submission_id: jellywibble-vampirekingd_8206_v3
developer_uid: Jellywibble
alignment_samples: 11818
alignment_score: 1.1486625000593376
best_of: 16
celo_rating: 1235.81
display_name: vk
formatter: {'memory_template': '### Instruction:\n{memory}\n', 'prompt_template': '### Input:\n{prompt}\n', 'bot_template': '{bot_name}: {message}</s>\n', 'user_template': '{user_name}: {message}\n', 'response_template': '### Response:\n{bot_name}:', 'truncate_by_message': True}
generation_params: {'temperature': 0.95, 'top_p': 0.95, 'min_p': 0.05, 'top_k': 80, 'presence_penalty': 0.0, 'frequency_penalty': 0.0, 'stopping_words': ['\n', '</s>', '###', 'You:'], 'max_input_tokens': 512, 'best_of': 16, 'max_output_tokens': 64, 'reward_max_token_input': 256}
is_internal_developer: True
language_model: Jellywibble/VampireKingDedupVicEP5
max_input_tokens: 512
max_output_tokens: 64
model_architecture: MistralForCausalLM
model_group: Jellywibble/VampireKingD
model_name: vk
model_num_parameters: 12772070400.0
model_repo: Jellywibble/VampireKingDedupVicEP5
model_size: 13B
num_battles: 11818
num_wins: 6119
propriety_score: 0.7421503330161751
propriety_total_count: 1051.0
ranking_group: single
reward_formatter: {'bot_template': '{bot_name}: {message}\n', 'memory_template': '', 'prompt_template': '', 'response_template': '{bot_name}:', 'truncate_by_message': False, 'user_template': '{user_name}: {message}\n'}
reward_repo: ChaiML/gpt2_xl_pairwise_89m_step_347634
status: torndown
submission_type: basic
timestamp: 2024-08-12T22:47:08+00:00
us_pacific_date: 2024-08-12
win_ratio: 0.5177695041462176
Download Preference Data
Resubmit model
Running pipeline stage MKMLizer
Starting job with name jellywibble-vampirekingd-8206-v3-mkmlizer
Waiting for job on jellywibble-vampirekingd-8206-v3-mkmlizer to finish
jellywibble-vampirekingd-8206-v3-mkmlizer: ╔═════════════════════════════════════════════════════════════════════╗
jellywibble-vampirekingd-8206-v3-mkmlizer: ║ _____ __ __ ║
jellywibble-vampirekingd-8206-v3-mkmlizer: ║ / _/ /_ ___ __/ / ___ ___ / / ║
jellywibble-vampirekingd-8206-v3-mkmlizer: ║ / _/ / // / |/|/ / _ \/ -_) -_) / ║
jellywibble-vampirekingd-8206-v3-mkmlizer: ║ /_//_/\_, /|__,__/_//_/\__/\__/_/ ║
jellywibble-vampirekingd-8206-v3-mkmlizer: ║ /___/ ║
jellywibble-vampirekingd-8206-v3-mkmlizer: ║ ║
jellywibble-vampirekingd-8206-v3-mkmlizer: ║ Version: 0.9.9 ║
jellywibble-vampirekingd-8206-v3-mkmlizer: ║ Copyright 2023 MK ONE TECHNOLOGIES Inc. ║
jellywibble-vampirekingd-8206-v3-mkmlizer: ║ https://mk1.ai ║
jellywibble-vampirekingd-8206-v3-mkmlizer: ║ ║
jellywibble-vampirekingd-8206-v3-mkmlizer: ║ The license key for the current software has been verified as ║
jellywibble-vampirekingd-8206-v3-mkmlizer: ║ belonging to: ║
jellywibble-vampirekingd-8206-v3-mkmlizer: ║ ║
jellywibble-vampirekingd-8206-v3-mkmlizer: ║ Chai Research Corp. ║
jellywibble-vampirekingd-8206-v3-mkmlizer: ║ Account ID: 7997a29f-0ceb-4cc7-9adf-840c57b4ae6f ║
jellywibble-vampirekingd-8206-v3-mkmlizer: ║ Expiration: 2024-10-15 23:59:59 ║
jellywibble-vampirekingd-8206-v3-mkmlizer: ║ ║
jellywibble-vampirekingd-8206-v3-mkmlizer: ╚═════════════════════════════════════════════════════════════════════╝
jellywibble-vampirekingd-8206-v3-mkmlizer: Downloaded to shared memory in 33.237s
jellywibble-vampirekingd-8206-v3-mkmlizer: quantizing model to /dev/shm/model_cache, profile:s0, folder:/tmp/tmpxkfjltq0, device:0
jellywibble-vampirekingd-8206-v3-mkmlizer: Saving flywheel model at /dev/shm/model_cache
jellywibble-vampirekingd-8206-v3-mkmlizer: quantized model in 35.492s
jellywibble-vampirekingd-8206-v3-mkmlizer: Processed model Jellywibble/VampireKingDedupVicEP5 in 68.729s
jellywibble-vampirekingd-8206-v3-mkmlizer: creating bucket guanaco-mkml-models
jellywibble-vampirekingd-8206-v3-mkmlizer: Bucket 's3://guanaco-mkml-models/' created
jellywibble-vampirekingd-8206-v3-mkmlizer: uploading /dev/shm/model_cache to s3://guanaco-mkml-models/jellywibble-vampirekingd-8206-v3
jellywibble-vampirekingd-8206-v3-mkmlizer: cp /dev/shm/model_cache/config.json s3://guanaco-mkml-models/jellywibble-vampirekingd-8206-v3/config.json
jellywibble-vampirekingd-8206-v3-mkmlizer: cp /dev/shm/model_cache/special_tokens_map.json s3://guanaco-mkml-models/jellywibble-vampirekingd-8206-v3/special_tokens_map.json
jellywibble-vampirekingd-8206-v3-mkmlizer: cp /dev/shm/model_cache/tokenizer_config.json s3://guanaco-mkml-models/jellywibble-vampirekingd-8206-v3/tokenizer_config.json
jellywibble-vampirekingd-8206-v3-mkmlizer: cp /dev/shm/model_cache/tokenizer.json s3://guanaco-mkml-models/jellywibble-vampirekingd-8206-v3/tokenizer.json
jellywibble-vampirekingd-8206-v3-mkmlizer: loading reward model from ChaiML/gpt2_xl_pairwise_89m_step_347634
jellywibble-vampirekingd-8206-v3-mkmlizer: Loading 0: 0%| | 0/363 [00:00<?, ?it/s] Loading 0: 1%|▏ | 5/363 [00:00<00:11, 31.52it/s] Loading 0: 4%|▎ | 13/363 [00:00<00:06, 51.24it/s] Loading 0: 5%|▌ | 19/363 [00:00<00:07, 47.03it/s] Loading 0: 7%|▋ | 24/363 [00:00<00:07, 45.58it/s] Loading 0: 9%|▊ | 31/363 [00:00<00:06, 50.97it/s] Loading 0: 10%|█ | 37/363 [00:00<00:07, 46.26it/s] Loading 0: 12%|█▏ | 42/363 [00:00<00:07, 44.94it/s] Loading 0: 13%|█▎ | 49/363 [00:01<00:06, 50.60it/s] Loading 0: 15%|█▌ | 55/363 [00:01<00:06, 47.23it/s] Loading 0: 17%|█▋ | 61/363 [00:01<00:08, 34.76it/s] Loading 0: 18%|█▊ | 66/363 [00:01<00:08, 36.03it/s] Loading 0: 20%|█▉ | 72/363 [00:01<00:07, 40.30it/s] Loading 0: 21%|██▏ | 78/363 [00:01<00:07, 40.33it/s] Loading 0: 23%|██▎ | 83/363 [00:01<00:07, 39.81it/s] Loading 0: 25%|██▍ | 90/363 [00:02<00:06, 45.46it/s] Loading 0: 26%|██▋ | 96/363 [00:02<00:06, 43.95it/s] Loading 0: 28%|██▊ | 101/363 [00:02<00:06, 43.17it/s] Loading 0: 30%|██▉ | 108/363 [00:02<00:05, 49.37it/s] Loading 0: 31%|███▏ | 114/363 [00:02<00:05, 44.17it/s] Loading 0: 33%|███▎ | 119/363 [00:02<00:05, 42.87it/s] Loading 0: 35%|███▍ | 126/363 [00:02<00:04, 47.61it/s] Loading 0: 36%|███▋ | 132/363 [00:03<00:05, 45.09it/s] Loading 0: 38%|███▊ | 137/363 [00:03<00:05, 44.54it/s] Loading 0: 39%|███▉ | 142/363 [00:03<00:06, 33.99it/s] Loading 0: 40%|████ | 147/363 [00:03<00:06, 35.56it/s] Loading 0: 42%|████▏ | 151/363 [00:03<00:05, 36.50it/s] Loading 0: 43%|████▎ | 157/363 [00:03<00:05, 41.09it/s] Loading 0: 45%|████▍ | 163/363 [00:03<00:04, 40.93it/s] Loading 0: 46%|████▋ | 168/363 [00:03<00:04, 40.16it/s] Loading 0: 48%|████▊ | 175/363 [00:04<00:04, 45.87it/s] Loading 0: 50%|████▉ | 181/363 [00:04<00:04, 43.58it/s] Loading 0: 51%|█████ | 186/363 [00:04<00:04, 42.36it/s] Loading 0: 53%|█████▎ | 192/363 [00:04<00:03, 46.62it/s] Loading 0: 54%|█████▍ | 197/363 [00:04<00:03, 45.47it/s] Loading 0: 56%|█████▌ | 202/363 [00:04<00:03, 44.79it/s] Loading 0: 57%|█████▋ | 208/363 [00:04<00:03, 43.17it/s] Loading 0: 59%|█████▊ | 213/363 [00:04<00:03, 42.61it/s] Loading 0: 60%|██████ | 219/363 [00:05<00:03, 46.78it/s] Loading 0: 62%|██████▏ | 224/363 [00:05<00:04, 32.96it/s] Loading 0: 63%|██████▎ | 228/363 [00:05<00:04, 32.54it/s] Loading 0: 64%|██████▍ | 232/363 [00:05<00:03, 33.72it/s] Loading 0: 66%|██████▌ | 238/363 [00:05<00:03, 38.66it/s] Loading 0: 67%|██████▋ | 244/363 [00:05<00:03, 38.95it/s] Loading 0: 69%|██████▊ | 249/363 [00:05<00:02, 39.43it/s] Loading 0: 71%|███████ | 256/363 [00:06<00:02, 44.67it/s] Loading 0: 72%|███████▏ | 262/363 [00:06<00:02, 43.24it/s] Loading 0: 74%|███████▎ | 267/363 [00:06<00:02, 42.14it/s] Loading 0: 75%|███████▌ | 273/363 [00:06<00:01, 45.87it/s] Loading 0: 77%|███████▋ | 278/363 [00:06<00:01, 45.93it/s] Loading 0: 78%|███████▊ | 283/363 [00:06<00:01, 45.75it/s] Loading 0: 80%|███████▉ | 289/363 [00:06<00:01, 43.21it/s] Loading 0: 81%|████████ | 294/363 [00:06<00:01, 42.40it/s] Loading 0: 82%|████████▏ | 299/363 [00:07<00:01, 43.74it/s] Loading 0: 84%|████████▎ | 304/363 [00:13<00:23, 2.47it/s] Loading 0: 85%|████████▍ | 308/363 [00:13<00:17, 3.21it/s] Loading 0: 86%|████████▌ | 312/363 [00:14<00:12, 4.21it/s] Loading 0: 88%|████████▊ | 320/363 [00:14<00:06, 7.06it/s] Loading 0: 90%|████████▉ | 326/363 [00:14<00:03, 9.54it/s] Loading 0: 91%|█████████ | 331/363 [00:14<00:02, 12.06it/s] Loading 0: 93%|█████████▎| 338/363 [00:14<00:01, 16.90it/s] Loading 0: 95%|█████████▍| 344/363 [00:14<00:00, 20.50it/s] Loading 0: 96%|█████████▌| 349/363 [00:14<00:00, 23.70it/s] Loading 0: 98%|█████████▊| 356/363 [00:14<00:00, 30.15it/s] Loading 0: 100%|█████████▉| 362/363 [00:15<00:00, 32.72it/s] /opt/conda/lib/python3.10/site-packages/transformers/models/auto/configuration_auto.py:957: FutureWarning: The `use_auth_token` argument is deprecated and will be removed in v5 of Transformers. Please use `token` instead.
jellywibble-vampirekingd-8206-v3-mkmlizer: warnings.warn(
jellywibble-vampirekingd-8206-v3-mkmlizer: /opt/conda/lib/python3.10/site-packages/transformers/models/auto/tokenization_auto.py:785: FutureWarning: The `use_auth_token` argument is deprecated and will be removed in v5 of Transformers. Please use `token` instead.
jellywibble-vampirekingd-8206-v3-mkmlizer: warnings.warn(
jellywibble-vampirekingd-8206-v3-mkmlizer: /opt/conda/lib/python3.10/site-packages/transformers/models/auto/auto_factory.py:469: FutureWarning: The `use_auth_token` argument is deprecated and will be removed in v5 of Transformers. Please use `token` instead.
jellywibble-vampirekingd-8206-v3-mkmlizer: warnings.warn(
jellywibble-vampirekingd-8206-v3-mkmlizer: Downloading shards: 0%| | 0/2 [00:00<?, ?it/s] Downloading shards: 50%|█████ | 1/2 [00:05<00:05, 5.28s/it] Downloading shards: 100%|██████████| 2/2 [00:07<00:00, 3.77s/it] Downloading shards: 100%|██████████| 2/2 [00:07<00:00, 3.99s/it]
jellywibble-vampirekingd-8206-v3-mkmlizer: Loading checkpoint shards: 0%| | 0/2 [00:00<?, ?it/s] Loading checkpoint shards: 50%|█████ | 1/2 [00:00<00:00, 2.39it/s] Loading checkpoint shards: 100%|██████████| 2/2 [00:00<00:00, 3.95it/s] Loading checkpoint shards: 100%|██████████| 2/2 [00:00<00:00, 3.60it/s]
jellywibble-vampirekingd-8206-v3-mkmlizer: Saving model to /tmp/reward_cache/reward.tensors
jellywibble-vampirekingd-8206-v3-mkmlizer: Saving duration: 1.330s
jellywibble-vampirekingd-8206-v3-mkmlizer: Processed model ChaiML/gpt2_xl_pairwise_89m_step_347634 in 12.865s
jellywibble-vampirekingd-8206-v3-mkmlizer: creating bucket guanaco-reward-models
jellywibble-vampirekingd-8206-v3-mkmlizer: Bucket 's3://guanaco-reward-models/' created
jellywibble-vampirekingd-8206-v3-mkmlizer: uploading /tmp/reward_cache to s3://guanaco-reward-models/jellywibble-vampirekingd-8206-v3_reward
jellywibble-vampirekingd-8206-v3-mkmlizer: cp /tmp/reward_cache/config.json s3://guanaco-reward-models/jellywibble-vampirekingd-8206-v3_reward/config.json
jellywibble-vampirekingd-8206-v3-mkmlizer: cp /tmp/reward_cache/special_tokens_map.json s3://guanaco-reward-models/jellywibble-vampirekingd-8206-v3_reward/special_tokens_map.json
jellywibble-vampirekingd-8206-v3-mkmlizer: cp /tmp/reward_cache/tokenizer_config.json s3://guanaco-reward-models/jellywibble-vampirekingd-8206-v3_reward/tokenizer_config.json
jellywibble-vampirekingd-8206-v3-mkmlizer: cp /tmp/reward_cache/merges.txt s3://guanaco-reward-models/jellywibble-vampirekingd-8206-v3_reward/merges.txt
jellywibble-vampirekingd-8206-v3-mkmlizer: cp /tmp/reward_cache/vocab.json s3://guanaco-reward-models/jellywibble-vampirekingd-8206-v3_reward/vocab.json
jellywibble-vampirekingd-8206-v3-mkmlizer: cp /tmp/reward_cache/tokenizer.json s3://guanaco-reward-models/jellywibble-vampirekingd-8206-v3_reward/tokenizer.json
jellywibble-vampirekingd-8206-v3-mkmlizer: cp /tmp/reward_cache/reward.tensors s3://guanaco-reward-models/jellywibble-vampirekingd-8206-v3_reward/reward.tensors
Job jellywibble-vampirekingd-8206-v3-mkmlizer completed after 116.58s with status: succeeded
Stopping job with name jellywibble-vampirekingd-8206-v3-mkmlizer
Pipeline stage MKMLizer completed in 117.51s
Running pipeline stage MKMLKubeTemplater
Pipeline stage MKMLKubeTemplater completed in 0.13s
Running pipeline stage ISVCDeployer
Creating inference service jellywibble-vampirekingd-8206-v3
Waiting for inference service jellywibble-vampirekingd-8206-v3 to be ready
Inference service jellywibble-vampirekingd-8206-v3 ready after 211.29611921310425s
Pipeline stage ISVCDeployer completed in 213.17s
Running pipeline stage StressChecker
Received healthy response to inference request in 2.848369598388672s
Received healthy response to inference request in 1.7712960243225098s
Received healthy response to inference request in 1.8008882999420166s
Received healthy response to inference request in 1.7505583763122559s
Received healthy response to inference request in 1.8025236129760742s
5 requests
0 failed requests
5th percentile: 1.7547059059143066
10th percentile: 1.7588534355163574
20th percentile: 1.767148494720459
30th percentile: 1.777214479446411
40th percentile: 1.789051389694214
50th percentile: 1.8008882999420166
60th percentile: 1.8015424251556396
70th percentile: 1.8021965503692627
80th percentile: 2.011692810058594
90th percentile: 2.4300312042236327
95th percentile: 2.639200401306152
99th percentile: 2.806535758972168
mean time: 1.9947271823883057
Pipeline stage StressChecker completed in 10.65s
jellywibble-vampirekingd_8206_v3 status is now deployed due to DeploymentManager action
jellywibble-vampirekingd_8206_v3 status is now inactive due to auto deactivation removed underperforming models
jellywibble-vampirekingd_8206_v3 status is now torndown due to DeploymentManager action

Usage Metrics

Latency Metrics