submission_id: mistralai-mixtral-8x7b_3473_v119
developer_uid: Jellywibble
alignment_samples: 0
best_of: 4
celo_rating: 1172.21
display_name: mistral-mem-in-prompt
formatter: {'memory_template': '', 'prompt_template': '<s>[INST] This is an entertaining conversation. You are {bot_name} who has the persona: {memory}.\nEngage in a chat with {user_name} while staying in character. Try to flirt with {user_name}. Engage in *roleplay* actions. Describe the scene dramatically. \n{prompt}\n', 'bot_template': '{bot_name}: {message}</s>', 'user_template': '[INST] {user_name}: {message} [/INST]', 'response_template': '{bot_name}:', 'truncate_by_message': False}
generation_params: {'temperature': 0.9, 'top_p': 1.0, 'min_p': 0.0, 'top_k': 50, 'presence_penalty': 0.5, 'frequency_penalty': 0.5, 'stopping_words': ['\n', '</s>', '<|user|>', '###'], 'max_input_tokens': 512, 'best_of': 4, 'max_output_tokens': 64, 'reward_max_token_input': 256}
is_internal_developer: True
language_model: mistralai/Mixtral-8x7B-Instruct-v0.1
max_input_tokens: 512
max_output_tokens: 64
model_architecture: MixtralForCausalLM
model_group: mistralai/Mixtral-8x7B-I
model_name: mistral-mem-in-prompt
model_num_parameters: 46702792704.0
model_repo: mistralai/Mixtral-8x7B-Instruct-v0.1
model_size: 47B
num_battles: 10204
num_wins: 4483
propriety_score: 0.7219003476245655
propriety_total_count: 863.0
ranking_group: single
reward_formatter: {'bot_template': 'Bot: {message}\n', 'memory_template': 'Memory: {memory}\n', 'prompt_template': '{prompt}\n', 'response_template': 'Bot:', 'truncate_by_message': False, 'user_template': 'User: {message}\n'}
reward_repo: ChaiML/gpt2_xl_pairwise_89m_step_347634
status: torndown
submission_type: basic
timestamp: 2024-08-05T22:59:04+00:00
us_pacific_date: 2024-08-05
win_ratio: 0.4393375147001176
Download Preference Data
Resubmit model
Running pipeline stage MKMLizer
Starting job with name mistralai-mixtral-8x7b-3473-v119-mkmlizer
Waiting for job on mistralai-mixtral-8x7b-3473-v119-mkmlizer to finish
Stopping job with name mistralai-mixtral-8x7b-3473-v119-mkmlizer
%s, retrying in %s seconds...
Starting job with name mistralai-mixtral-8x7b-3473-v119-mkmlizer
Waiting for job on mistralai-mixtral-8x7b-3473-v119-mkmlizer to finish
Stopping job with name mistralai-mixtral-8x7b-3473-v119-mkmlizer
%s, retrying in %s seconds...
Starting job with name mistralai-mixtral-8x7b-3473-v119-mkmlizer
Waiting for job on mistralai-mixtral-8x7b-3473-v119-mkmlizer to finish
mistralai-mixtral-8x7b-3473-v119-mkmlizer: ╔═════════════════════════════════════════════════════════════════════╗
mistralai-mixtral-8x7b-3473-v119-mkmlizer: ║ _____ __ __ ║
mistralai-mixtral-8x7b-3473-v119-mkmlizer: ║ / _/ /_ ___ __/ / ___ ___ / / ║
mistralai-mixtral-8x7b-3473-v119-mkmlizer: ║ / _/ / // / |/|/ / _ \/ -_) -_) / ║
mistralai-mixtral-8x7b-3473-v119-mkmlizer: ║ /_//_/\_, /|__,__/_//_/\__/\__/_/ ║
mistralai-mixtral-8x7b-3473-v119-mkmlizer: ║ /___/ ║
mistralai-mixtral-8x7b-3473-v119-mkmlizer: ║ ║
mistralai-mixtral-8x7b-3473-v119-mkmlizer: ║ Version: 0.9.9 ║
mistralai-mixtral-8x7b-3473-v119-mkmlizer: ║ Copyright 2023 MK ONE TECHNOLOGIES Inc. ║
mistralai-mixtral-8x7b-3473-v119-mkmlizer: ║ https://mk1.ai ║
mistralai-mixtral-8x7b-3473-v119-mkmlizer: ║ ║
mistralai-mixtral-8x7b-3473-v119-mkmlizer: ║ The license key for the current software has been verified as ║
mistralai-mixtral-8x7b-3473-v119-mkmlizer: ║ belonging to: ║
mistralai-mixtral-8x7b-3473-v119-mkmlizer: ║ ║
mistralai-mixtral-8x7b-3473-v119-mkmlizer: ║ Chai Research Corp. ║
mistralai-mixtral-8x7b-3473-v119-mkmlizer: ║ Account ID: 7997a29f-0ceb-4cc7-9adf-840c57b4ae6f ║
mistralai-mixtral-8x7b-3473-v119-mkmlizer: ║ Expiration: 2024-10-15 23:59:59 ║
mistralai-mixtral-8x7b-3473-v119-mkmlizer: ║ ║
mistralai-mixtral-8x7b-3473-v119-mkmlizer: ╚═════════════════════════════════════════════════════════════════════╝
mistralai-mixtral-8x7b-3473-v119-mkmlizer: Downloaded to shared memory in 194.687s
mistralai-mixtral-8x7b-3473-v119-mkmlizer: quantizing model to /dev/shm/model_cache, profile:s0, folder:/tmp/tmpc8duvjsg, device:0
mistralai-mixtral-8x7b-3473-v119-mkmlizer: Saving flywheel model at /dev/shm/model_cache
Failed to get response for submission chaiml-elo-alignment-run-3_v17: ('http://chaiml-elo-alignment-run-3-v17-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'read tcp 127.0.0.1:36132->127.0.0.1:8080: read: connection reset by peer\n')
mistralai-mixtral-8x7b-3473-v119-mkmlizer: quantized model in 94.100s
mistralai-mixtral-8x7b-3473-v119-mkmlizer: Processed model mistralai/Mixtral-8x7B-Instruct-v0.1 in 288.788s
mistralai-mixtral-8x7b-3473-v119-mkmlizer: creating bucket guanaco-mkml-models
mistralai-mixtral-8x7b-3473-v119-mkmlizer: Bucket 's3://guanaco-mkml-models/' created
mistralai-mixtral-8x7b-3473-v119-mkmlizer: uploading /dev/shm/model_cache to s3://guanaco-mkml-models/mistralai-mixtral-8x7b-3473-v119
mistralai-mixtral-8x7b-3473-v119-mkmlizer: cp /dev/shm/model_cache/config.json s3://guanaco-mkml-models/mistralai-mixtral-8x7b-3473-v119/config.json
mistralai-mixtral-8x7b-3473-v119-mkmlizer: cp /dev/shm/model_cache/tokenizer.json s3://guanaco-mkml-models/mistralai-mixtral-8x7b-3473-v119/tokenizer.json
mistralai-mixtral-8x7b-3473-v119-mkmlizer: cp /dev/shm/model_cache/flywheel_model.3.safetensors s3://guanaco-mkml-models/mistralai-mixtral-8x7b-3473-v119/flywheel_model.3.safetensors
mistralai-mixtral-8x7b-3473-v119-mkmlizer: cp /dev/shm/model_cache/flywheel_model.2.safetensors s3://guanaco-mkml-models/mistralai-mixtral-8x7b-3473-v119/flywheel_model.2.safetensors
mistralai-mixtral-8x7b-3473-v119-mkmlizer: loading reward model from ChaiML/gpt2_xl_pairwise_89m_step_347634
mistralai-mixtral-8x7b-3473-v119-mkmlizer: Loading 0: 0%| | 0/995 [00:00<?, ?it/s] Loading 0: 0%| | 4/995 [00:00<00:29, 33.84it/s] Loading 0: 1%| | 8/995 [00:00<00:31, 30.88it/s] Loading 0: 1%| | 12/995 [00:00<00:32, 29.91it/s] Loading 0: 2%|▏ | 16/995 [00:00<00:38, 25.43it/s] Loading 0: 2%|▏ | 19/995 [00:00<00:38, 25.44it/s] Loading 0: 2%|▏ | 22/995 [00:00<00:38, 25.19it/s] Loading 0: 3%|▎ | 25/995 [00:00<00:38, 25.40it/s] Loading 0: 3%|▎ | 34/995 [00:01<00:23, 41.47it/s] Loading 0: 4%|▍ | 39/995 [00:01<00:29, 32.13it/s] Loading 0: 4%|▍ | 43/995 [00:01<00:30, 31.06it/s] Loading 0: 5%|▍ | 47/995 [00:01<00:30, 31.47it/s] Loading 0: 5%|▌ | 52/995 [00:01<00:41, 22.67it/s] Loading 0: 6%|▌ | 55/995 [00:02<00:41, 22.76it/s] Loading 0: 6%|▌ | 58/995 [00:02<00:40, 22.97it/s] Loading 0: 6%|▌ | 61/995 [00:02<00:40, 23.35it/s] Loading 0: 7%|▋ | 66/995 [00:02<00:33, 27.74it/s] Loading 0: 7%|▋ | 69/995 [00:02<00:34, 26.75it/s] Loading 0: 7%|▋ | 72/995 [00:02<00:35, 25.82it/s] Loading 0: 8%|▊ | 75/995 [00:02<00:36, 25.43it/s] Loading 0: 8%|▊ | 78/995 [00:02<00:37, 24.57it/s] Loading 0: 8%|▊ | 81/995 [00:03<00:37, 24.34it/s] Loading 0: 8%|▊ | 84/995 [00:03<00:37, 24.33it/s] Loading 0: 9%|▊ | 87/995 [00:03<00:37, 24.37it/s] Loading 0: 10%|▉ | 95/995 [00:03<00:23, 37.88it/s] Loading 0: 10%|▉ | 99/995 [00:03<00:26, 33.39it/s] Loading 0: 10%|█ | 103/995 [00:03<00:26, 34.29it/s] Loading 0: 11%|█ | 107/995 [00:03<00:36, 24.60it/s] Loading 0: 11%|█ | 110/995 [00:04<00:36, 24.53it/s] Loading 0: 11%|█▏ | 113/995 [00:04<00:37, 23.51it/s] Loading 0: 12%|█▏ | 116/995 [00:04<00:37, 23.23it/s] Loading 0: 12%|█▏ | 119/995 [00:04<00:37, 23.33it/s] Loading 0: 12%|█▏ | 122/995 [00:04<00:37, 23.53it/s] Loading 0: 13%|█▎ | 126/995 [00:04<00:32, 27.00it/s] Loading 0: 13%|█▎ | 129/995 [00:04<00:33, 25.96it/s] Loading 0: 13%|█▎ | 132/995 [00:04<00:34, 25.14it/s] Loading 0: 14%|█▎ | 135/995 [00:05<00:35, 24.15it/s] Loading 0: 14%|█▍ | 138/995 [00:05<00:35, 24.01it/s] Loading 0: 14%|█▍ | 141/995 [00:05<00:35, 23.91it/s] Loading 0: 14%|█▍ | 144/995 [00:05<00:35, 23.99it/s] Loading 0: 15%|█▍ | 147/995 [00:05<00:35, 24.06it/s] Loading 0: 15%|█▌ | 150/995 [00:05<00:33, 25.17it/s] Loading 0: 16%|█▌ | 161/995 [00:05<00:20, 40.75it/s] Loading 0: 17%|█▋ | 165/995 [00:06<00:32, 25.87it/s] Loading 0: 17%|█▋ | 169/995 [00:06<00:31, 26.29it/s] Loading 0: 17%|█▋ | 172/995 [00:06<00:32, 25.68it/s] Loading 0: 18%|█▊ | 175/995 [00:06<00:33, 24.81it/s] Loading 0: 18%|█▊ | 178/995 [00:06<00:33, 24.54it/s] Loading 0: 18%|█▊ | 181/995 [00:06<00:33, 24.46it/s] Loading 0: 18%|█▊ | 184/995 [00:06<00:33, 24.22it/s] Loading 0: 19%|█▉ | 188/995 [00:07<00:29, 27.48it/s] Loading 0: 19%|█▉ | 191/995 [00:07<00:30, 26.43it/s] Loading 0: 19%|█▉ | 194/995 [00:07<00:31, 25.65it/s] Loading 0: 20%|█▉ | 197/995 [00:07<00:31, 25.09it/s] Loading 0: 20%|██ | 200/995 [00:07<00:31, 24.89it/s] Loading 0: 20%|██ | 203/995 [00:07<00:31, 24.86it/s] Loading 0: 21%|██ | 210/995 [00:08<00:35, 21.86it/s] Loading 0: 21%|██▏ | 213/995 [00:08<00:35, 22.33it/s] Loading 0: 22%|██▏ | 216/995 [00:08<00:34, 22.76it/s] Loading 0: 22%|██▏ | 221/995 [00:08<00:28, 27.07it/s] Loading 0: 23%|██▎ | 224/995 [00:08<00:29, 26.43it/s] Loading 0: 23%|██▎ | 227/995 [00:08<00:29, 25.97it/s] Loading 0: 23%|██▎ | 230/995 [00:08<00:30, 25.48it/s] Loading 0: 23%|██▎ | 233/995 [00:08<00:30, 25.07it/s] Loading 0: 24%|██▎ | 236/995 [00:09<00:30, 24.88it/s] Loading 0: 24%|██▍ | 239/995 [00:09<00:30, 24.91it/s] Loading 0: 24%|██▍ | 242/995 [00:09<00:30, 24.44it/s] Loading 0: 25%|██▌ | 251/995 [00:09<00:18, 39.65it/s] Loading 0: 26%|██▌ | 256/995 [00:09<00:22, 32.25it/s] Loading 0: 26%|██▌ | 260/995 [00:09<00:22, 32.32it/s] Loading 0: 27%|██▋ | 265/995 [00:10<00:28, 25.67it/s] Loading 0: 27%|██▋ | 268/995 [00:10<00:27, 26.18it/s] Loading 0: 27%|██▋ | 273/995 [00:10<00:23, 30.73it/s] Loading 0: 28%|██▊ | 277/995 [00:10<00:26, 26.75it/s] Loading 0: 28%|██▊ | 281/995 [00:10<00:26, 26.74it/s] Loading 0: 29%|██▊ | 284/995 [00:25<14:05, 1.19s/it] Loading 0: 29%|██▊ | 286/995 [00:25<11:42, 1.01it/s] Loading 0: 29%|██▉ | 291/995 [00:25<07:12, 1.63it/s] Loading 0: 30%|██▉ | 294/995 [00:25<05:28, 2.13it/s] Loading 0: 30%|██▉ | 297/995 [00:26<04:08, 2.81it/s] Loading 0: 30%|███ | 300/995 [00:26<03:07, 3.71it/s] Loading 0: 30%|███ | 303/995 [00:26<02:21, 4.88it/s] Loading 0: 31%|███ | 306/995 [00:26<01:48, 6.34it/s] Loading 0: 31%|███ | 309/995 [00:26<01:25, 8.07it/s] Loading 0: 31%|███▏ | 312/995 [00:26<01:08, 10.03it/s] Loading 0: 32%|███▏ | 320/995 [00:27<00:44, 15.05it/s] Loading 0: 32%|███▏ | 323/995 [00:27<00:40, 16.43it/s] Loading 0: 33%|███▎ | 326/995 [00:27<00:37, 17.92it/s] Loading 0: 33%|███▎ | 329/995 [00:27<00:34, 19.38it/s] Loading 0: 33%|███▎ | 332/995 [00:27<00:32, 20.54it/s] Loading 0: 34%|███▎ | 335/995 [00:27<00:30, 21.30it/s] Loading 0: 34%|███▍ | 338/995 [00:27<00:29, 22.32it/s] Loading 0: 34%|███▍ | 343/995 [00:27<00:23, 27.30it/s] Loading 0: 35%|███▍ | 346/995 [00:28<00:24, 26.20it/s] Loading 0: 35%|███▌ | 349/995 [00:28<00:25, 25.75it/s] Loading 0: 35%|███▌ | 352/995 [00:28<00:25, 25.46it/s] Loading 0: 36%|███▌ | 355/995 [00:28<00:25, 25.07it/s] Loading 0: 36%|███▌ | 358/995 [00:28<00:26, 24.47it/s] Loading 0: 36%|███▋ | 361/995 [00:28<00:25, 24.47it/s] Loading 0: 37%|███▋ | 368/995 [00:28<00:28, 21.90it/s] Loading 0: 37%|███▋ | 371/995 [00:29<00:27, 22.45it/s] Loading 0: 38%|███▊ | 376/995 [00:29<00:23, 26.67it/s] Loading 0: 38%|███▊ | 379/995 [00:29<00:23, 26.01it/s] Loading 0: 38%|███▊ | 382/995 [00:29<00:24, 24.92it/s] Loading 0: 39%|███▊ | 385/995 [00:29<00:24, 24.80it/s] Loading 0: 39%|███▉ | 388/995 [00:29<00:24, 24.61it/s] Loading 0: 39%|███▉ | 391/995 [00:29<00:24, 24.48it/s] Loading 0: 40%|███▉ | 394/995 [00:29<00:24, 24.31it/s] Loading 0: 40%|███▉ | 397/995 [00:30<00:24, 24.36it/s] Loading 0: 41%|████ | 405/995 [00:30<00:15, 37.74it/s] Loading 0: 41%|████ | 409/995 [00:30<00:16, 34.69it/s] Loading 0: 42%|████▏ | 413/995 [00:30<00:20, 28.47it/s] Loading 0: 42%|████▏ | 417/995 [00:30<00:20, 28.00it/s] Loading 0: 43%|████▎ | 423/995 [00:30<00:22, 25.48it/s] Loading 0: 43%|████▎ | 426/995 [00:31<00:22, 25.28it/s] Loading 0: 43%|████▎ | 429/995 [00:31<00:22, 25.10it/s] Loading 0: 43%|████▎ | 432/995 [00:31<00:22, 24.79it/s] Loading 0: 44%|████▍ | 436/995 [00:31<00:19, 28.13it/s] Loading 0: 44%|████▍ | 439/995 [00:31<00:20, 27.32it/s] Loading 0: 44%|████▍ | 442/995 [00:31<00:20, 26.47it/s] Loading 0: 45%|████▍ | 445/995 [00:31<00:21, 25.70it/s] Loading 0: 45%|████▌ | 448/995 [00:31<00:21, 24.90it/s] Loading 0: 45%|████▌ | 451/995 [00:32<00:21, 24.84it/s] Loading 0: 46%|████▌ | 454/995 [00:32<00:21, 24.81it/s] Loading 0: 46%|████▌ | 457/995 [00:32<00:21, 24.79it/s] Loading 0: 47%|████▋ | 463/995 [00:32<00:15, 33.65it/s] Loading 0: 47%|████▋ | 468/995 [00:32<00:14, 37.23it/s] Loading 0: 47%|████▋ | 472/995 [00:32<00:17, 29.84it/s] Loading 0: 48%|████▊ | 478/995 [00:33<00:20, 25.41it/s] Loading 0: 48%|████▊ | 481/995 [00:33<00:20, 25.14it/s] Loading 0: 49%|████▊ | 484/995 [00:33<00:20, 24.97it/s] Loading 0: 49%|████▉ | 487/995 [00:33<00:20, 24.98it/s] Loading 0: 49%|████▉ | 490/995 [00:33<00:20, 24.93it/s] Loading 0: 50%|████▉ | 493/995 [00:33<00:20, 24.67it/s] Loading 0: 50%|████▉ | 496/995 [00:33<00:19, 25.85it/s] Loading 0: 50%|█████ | 500/995 [00:33<00:17, 28.03it/s] Loading 0: 51%|█████ | 503/995 [00:33<00:18, 26.96it/s] Loading 0: 51%|█████ | 506/995 [00:34<00:18, 25.92it/s] Loading 0: 51%|█████ | 509/995 [00:34<00:19, 25.56it/s] Loading 0: 51%|█████▏ | 512/995 [00:34<00:19, 25.09it/s] Loading 0: 52%|█████▏ | 515/995 [00:34<00:19, 24.69it/s] Loading 0: 52%|█████▏ | 518/995 [00:34<00:19, 24.60it/s] Loading 0: 53%|█████▎ | 525/995 [00:34<00:13, 34.89it/s] Loading 0: 53%|█████▎ | 529/995 [00:34<00:18, 24.83it/s] Loading 0: 53%|█████▎ | 532/995 [00:35<00:18, 24.96it/s] Loading 0: 54%|█████▍ | 535/995 [00:35<00:18, 25.02it/s] Loading 0: 54%|█████▍ | 538/995 [00:35<00:18, 24.90it/s] Loading 0: 54%|█████▍ | 541/995 [00:35<00:18, 24.89it/s] Loading 0: 55%|█████▍ | 544/995 [00:35<00:18, 24.77it/s] Loading 0: 55%|█████▍ | 547/995 [00:35<00:18, 24.68it/s] Loading 0: 55%|█████▌ | 550/995 [00:35<00:18, 24.57it/s] Loading 0: 56%|█████▌ | 556/995 [00:35<00:13, 33.39it/s] Loading 0: 56%|█████▋ | 561/995 [00:36<00:11, 36.56it/s] Loading 0: 56%|█████▋ | 561/995 [00:46<00:11, 36.56it/s] Loading 0: 56%|█████▋ | 562/995 [00:51<10:04, 1.40s/it] Loading 0: 57%|█████▋ | 564/995 [00:51<08:03, 1.12s/it] Loading 0: 57%|█████▋ | 568/995 [00:51<05:12, 1.36it/s] Loading 0: 57%|█████▋ | 571/995 [00:51<03:47, 1.86it/s] Loading 0: 58%|█████▊ | 574/995 [00:51<02:45, 2.54it/s] Loading 0: 58%|█████▊ | 580/995 [00:51<01:32, 4.47it/s] Loading 0: 59%|█████▊ | 584/995 [00:52<01:14, 5.51it/s] Loading 0: 59%|█████▉ | 587/995 [00:52<01:00, 6.78it/s] Loading 0: 59%|█████▉ | 591/995 [00:52<00:44, 9.14it/s] Loading 0: 60%|█████▉ | 594/995 [00:52<00:37, 10.81it/s] Loading 0: 60%|██████ | 597/995 [00:52<00:31, 12.65it/s] Loading 0: 60%|██████ | 600/995 [00:52<00:27, 14.47it/s] Loading 0: 61%|██████ | 603/995 [00:52<00:24, 16.24it/s] Loading 0: 61%|██████ | 606/995 [00:53<00:22, 17.40it/s] Loading 0: 61%|██████ | 609/995 [00:53<00:20, 18.85it/s] Loading 0: 62%|██████▏ | 612/995 [00:53<00:18, 20.17it/s] Loading 0: 62%|██████▏ | 615/995 [00:53<00:17, 22.15it/s] Loading 0: 63%|██████▎ | 622/995 [00:53<00:11, 33.00it/s] Loading 0: 63%|██████▎ | 626/995 [00:53<00:11, 31.48it/s] Loading 0: 63%|██████▎ | 630/995 [00:53<00:13, 26.30it/s] Loading 0: 64%|██████▍ | 636/995 [00:54<00:14, 24.44it/s] Loading 0: 64%|██████▍ | 639/995 [00:54<00:14, 24.29it/s] Loading 0: 65%|██████▍ | 642/995 [00:54<00:14, 24.04it/s] Loading 0: 65%|██████▍ | 645/995 [00:54<00:14, 24.15it/s] Loading 0: 65%|██████▌ | 648/995 [00:54<00:14, 23.89it/s] Loading 0: 65%|██████▌ | 651/995 [00:54<00:13, 25.25it/s] Loading 0: 66%|██████▌ | 655/995 [00:54<00:12, 27.03it/s] Loading 0: 66%|██████▌ | 658/995 [00:55<00:12, 26.53it/s] Loading 0: 66%|██████▋ | 661/995 [00:55<00:12, 25.96it/s] Loading 0: 67%|██████▋ | 664/995 [00:55<00:13, 25.23it/s] Loading 0: 67%|██████▋ | 667/995 [00:55<00:13, 24.99it/s] Loading 0: 67%|██████▋ | 670/995 [00:55<00:13, 24.58it/s] Loading 0: 68%|██████▊ | 673/995 [00:55<00:13, 24.40it/s] Loading 0: 68%|██████▊ | 676/995 [00:55<00:13, 24.04it/s] Loading 0: 69%|██████▉ | 685/995 [00:55<00:07, 39.17it/s] Loading 0: 69%|██████▉ | 691/995 [00:56<00:11, 27.56it/s] Loading 0: 70%|██████▉ | 695/995 [00:56<00:10, 27.38it/s] Loading 0: 70%|███████ | 699/995 [00:56<00:10, 27.70it/s] Loading 0: 71%|███████ | 703/995 [00:56<00:11, 24.78it/s] Loading 0: 71%|███████ | 706/995 [00:56<00:11, 24.85it/s] Loading 0: 71%|███████▏ | 709/995 [00:57<00:11, 24.94it/s] Loading 0: 72%|███████▏ | 712/995 [00:57<00:11, 24.64it/s] Loading 0: 72%|███████▏ | 717/995 [00:57<00:09, 29.23it/s] Loading 0: 72%|███████▏ | 721/995 [00:57<00:09, 28.12it/s] Loading 0: 73%|███████▎ | 724/995 [00:57<00:09, 27.26it/s] Loading 0: 73%|███████▎ | 727/995 [00:57<00:10, 26.54it/s] Loading 0: 73%|███████▎ | 730/995 [00:57<00:10, 25.81it/s] Loading 0: 74%|███████▎ | 733/995 [00:57<00:10, 25.49it/s] Loading 0: 74%|███████▍ | 739/995 [00:58<00:10, 23.71it/s] Loading 0: 75%|███████▍ | 742/995 [00:58<00:10, 23.84it/s] Loading 0: 75%|███████▍ | 746/995 [00:58<00:09, 27.01it/s] Loading 0: 75%|███████▌ | 749/995 [00:58<00:09, 26.27it/s] Loading 0: 76%|███████▌ | 752/995 [00:58<00:09, 25.82it/s] Loading 0: 76%|███████▌ | 755/995 [00:58<00:09, 25.35it/s] Loading 0: 76%|███████▌ | 758/995 [00:58<00:09, 24.95it/s] Loading 0: 76%|███████▋ | 761/995 [00:59<00:09, 24.90it/s] Loading 0: 77%|███████▋ | 764/995 [00:59<00:09, 24.69it/s] Loading 0: 77%|███████▋ | 767/995 [00:59<00:09, 24.21it/s] Loading 0: 77%|███████▋ | 770/995 [00:59<00:09, 24.74it/s] Loading 0: 78%|███████▊ | 777/995 [00:59<00:06, 36.28it/s] Loading 0: 78%|███████▊ | 781/995 [00:59<00:06, 33.58it/s] Loading 0: 79%|███████▉ | 785/995 [00:59<00:07, 27.06it/s] Loading 0: 79%|███████▉ | 789/995 [00:59<00:07, 28.41it/s] Loading 0: 80%|███████▉ | 794/995 [01:00<00:08, 23.25it/s] Loading 0: 80%|████████ | 797/995 [01:00<00:08, 23.16it/s] Loading 0: 80%|████████ | 800/995 [01:00<00:08, 23.40it/s] Loading 0: 81%|████████ | 803/995 [01:00<00:08, 23.47it/s] Loading 0: 81%|████████ | 806/995 [01:00<00:07, 24.88it/s] Loading 0: 81%|████████▏ | 810/995 [01:00<00:06, 26.77it/s] Loading 0: 82%|████████▏ | 813/995 [01:00<00:07, 25.63it/s] Loading 0: 82%|████████▏ | 816/995 [01:01<00:07, 24.90it/s] Loading 0: 82%|████████▏ | 819/995 [01:01<00:07, 24.71it/s] Loading 0: 83%|████████▎ | 822/995 [01:01<00:07, 24.41it/s] Loading 0: 83%|████████▎ | 825/995 [01:01<00:07, 24.27it/s] Loading 0: 83%|████████▎ | 828/995 [01:01<00:06, 24.00it/s] Loading 0: 84%|████████▎ | 831/995 [01:01<00:06, 23.53it/s] Loading 0: 84%|████████▍ | 839/995 [01:01<00:04, 36.99it/s] Loading 0: 85%|████████▍ | 842/995 [01:16<00:04, 36.99it/s] Loading 0: 85%|████████▍ | 843/995 [01:16<02:40, 1.06s/it] Loading 0: 85%|████████▌ | 848/995 [01:17<01:45, 1.40it/s] Loading 0: 86%|████████▌ | 853/995 [01:17<01:13, 1.93it/s] Loading 0: 86%|████████▌ | 857/995 [01:17<00:53, 2.57it/s] Loading 0: 87%|████████▋ | 861/995 [01:17<00:39, 3.39it/s] Loading 0: 87%|████████▋ | 864/995 [01:18<00:31, 4.22it/s] Loading 0: 87%|████████▋ | 867/995 [01:18<00:24, 5.30it/s] Loading 0: 88%|████████▊ | 872/995 [01:18<00:15, 7.82it/s] Loading 0: 88%|████████▊ | 876/995 [01:18<00:12, 9.90it/s] Loading 0: 88%|████████▊ | 879/995 [01:18<00:10, 11.55it/s] Loading 0: 89%|████████▊ | 882/995 [01:18<00:08, 13.38it/s] Loading 0: 89%|████████▉ | 885/995 [01:18<00:07, 15.23it/s] Loading 0: 89%|████████▉ | 888/995 [01:18<00:06, 17.03it/s] Loading 0: 90%|████████▉ | 891/995 [01:19<00:05, 18.72it/s] Loading 0: 90%|█████████ | 897/995 [01:19<00:04, 20.10it/s] Loading 0: 91%|█████████ | 901/995 [01:19<00:04, 23.42it/s] Loading 0: 91%|█████████ | 904/995 [01:19<00:03, 23.25it/s] Loading 0: 91%|█████████ | 907/995 [01:19<00:03, 23.49it/s] Loading 0: 91%|█████████▏| 910/995 [01:19<00:03, 23.21it/s] Loading 0: 92%|█████████▏| 913/995 [01:19<00:03, 23.35it/s] Loading 0: 92%|█████████▏| 916/995 [01:20<00:03, 23.18it/s] Loading 0: 92%|█████████▏| 919/995 [01:20<00:03, 23.46it/s] Loading 0: 93%|█████████▎| 922/995 [01:20<00:03, 23.90it/s] Loading 0: 93%|█████████▎| 925/995 [01:20<00:02, 25.39it/s] Loading 0: 94%|█████████▍| 933/995 [01:20<00:01, 38.24it/s] Loading 0: 94%|█████████▍| 937/995 [01:20<00:01, 30.50it/s] Loading 0: 95%|█████████▍| 941/995 [01:20<00:01, 28.74it/s] Loading 0: 95%|█████████▍| 945/995 [01:21<00:01, 28.54it/s] Loading 0: 96%|█████████▌| 951/995 [01:21<00:01, 31.25it/s] Loading 0: 96%|█████████▌| 955/995 [01:22<00:05, 7.68it/s] Loading 0: 96%|█████████▋| 958/995 [01:22<00:04, 9.05it/s] Loading 0: 97%|█████████▋| 961/995 [01:23<00:03, 10.61it/s] Loading 0: 97%|█████████▋| 966/995 [01:23<00:02, 14.46it/s] Loading 0: 97%|█████████▋| 969/995 [01:23<00:01, 15.98it/s] Loading 0: 98%|█████████▊| 972/995 [01:23<00:01, 17.54it/s] Loading 0: 98%|█████████▊| 975/995 [01:23<00:01, 18.87it/s] Loading 0: 98%|█████████▊| 978/995 [01:23<00:00, 20.14it/s] Loading 0: 99%|█████████▊| 981/995 [01:23<00:00, 20.97it/s] Loading 0: 99%|█████████▉| 984/995 [01:23<00:00, 21.75it/s] Loading 0: 99%|█████████▉| 987/995 [01:24<00:00, 22.29it/s] /opt/conda/lib/python3.10/site-packages/transformers/models/auto/configuration_auto.py:957: FutureWarning: The `use_auth_token` argument is deprecated and will be removed in v5 of Transformers. Please use `token` instead.
mistralai-mixtral-8x7b-3473-v119-mkmlizer: warnings.warn(
mistralai-mixtral-8x7b-3473-v119-mkmlizer: /opt/conda/lib/python3.10/site-packages/transformers/models/auto/tokenization_auto.py:785: FutureWarning: The `use_auth_token` argument is deprecated and will be removed in v5 of Transformers. Please use `token` instead.
mistralai-mixtral-8x7b-3473-v119-mkmlizer: warnings.warn(
mistralai-mixtral-8x7b-3473-v119-mkmlizer: /opt/conda/lib/python3.10/site-packages/transformers/models/auto/auto_factory.py:469: FutureWarning: The `use_auth_token` argument is deprecated and will be removed in v5 of Transformers. Please use `token` instead.
mistralai-mixtral-8x7b-3473-v119-mkmlizer: warnings.warn(
mistralai-mixtral-8x7b-3473-v119-mkmlizer: Downloading shards: 0%| | 0/2 [00:00<?, ?it/s] Downloading shards: 50%|█████ | 1/2 [00:05<00:05, 5.34s/it] Downloading shards: 100%|██████████| 2/2 [00:09<00:00, 4.38s/it] Downloading shards: 100%|██████████| 2/2 [00:09<00:00, 4.52s/it]
mistralai-mixtral-8x7b-3473-v119-mkmlizer: Loading checkpoint shards: 0%| | 0/2 [00:00<?, ?it/s] Loading checkpoint shards: 50%|█████ | 1/2 [00:00<00:00, 2.43it/s] Loading checkpoint shards: 100%|██████████| 2/2 [00:00<00:00, 4.01it/s] Loading checkpoint shards: 100%|██████████| 2/2 [00:00<00:00, 3.65it/s]
mistralai-mixtral-8x7b-3473-v119-mkmlizer: Saving model to /tmp/reward_cache/reward.tensors
mistralai-mixtral-8x7b-3473-v119-mkmlizer: Saving duration: 1.408s
mistralai-mixtral-8x7b-3473-v119-mkmlizer: Processed model ChaiML/gpt2_xl_pairwise_89m_step_347634 in 15.276s
mistralai-mixtral-8x7b-3473-v119-mkmlizer: creating bucket guanaco-reward-models
mistralai-mixtral-8x7b-3473-v119-mkmlizer: cp /tmp/reward_cache/tokenizer_config.json s3://guanaco-reward-models/mistralai-mixtral-8x7b-3473-v119_reward/tokenizer_config.json
mistralai-mixtral-8x7b-3473-v119-mkmlizer: cp /tmp/reward_cache/special_tokens_map.json s3://guanaco-reward-models/mistralai-mixtral-8x7b-3473-v119_reward/special_tokens_map.json
mistralai-mixtral-8x7b-3473-v119-mkmlizer: cp /tmp/reward_cache/config.json s3://guanaco-reward-models/mistralai-mixtral-8x7b-3473-v119_reward/config.json
mistralai-mixtral-8x7b-3473-v119-mkmlizer: cp /tmp/reward_cache/merges.txt s3://guanaco-reward-models/mistralai-mixtral-8x7b-3473-v119_reward/merges.txt
mistralai-mixtral-8x7b-3473-v119-mkmlizer: cp /tmp/reward_cache/vocab.json s3://guanaco-reward-models/mistralai-mixtral-8x7b-3473-v119_reward/vocab.json
mistralai-mixtral-8x7b-3473-v119-mkmlizer: cp /tmp/reward_cache/tokenizer.json s3://guanaco-reward-models/mistralai-mixtral-8x7b-3473-v119_reward/tokenizer.json
mistralai-mixtral-8x7b-3473-v119-mkmlizer: cp /tmp/reward_cache/reward.tensors s3://guanaco-reward-models/mistralai-mixtral-8x7b-3473-v119_reward/reward.tensors
Job mistralai-mixtral-8x7b-3473-v119-mkmlizer completed after 353.08s with status: succeeded
Stopping job with name mistralai-mixtral-8x7b-3473-v119-mkmlizer
Pipeline stage MKMLizer completed in 366.24s
Running pipeline stage MKMLKubeTemplater
Pipeline stage MKMLKubeTemplater completed in 0.11s
Running pipeline stage ISVCDeployer
Creating inference service mistralai-mixtral-8x7b-3473-v119
Waiting for inference service mistralai-mixtral-8x7b-3473-v119 to be ready
Connection pool is full, discarding connection: %s. Connection pool size: %s
Inference service mistralai-mixtral-8x7b-3473-v119 ready after 171.05421352386475s
Pipeline stage ISVCDeployer completed in 173.04s
Running pipeline stage StressChecker
Received healthy response to inference request in 2.461535930633545s
Received healthy response to inference request in 1.5331847667694092s
Received healthy response to inference request in 1.4664897918701172s
Received healthy response to inference request in 1.0900461673736572s
Received healthy response to inference request in 1.318744421005249s
5 requests
0 failed requests
5th percentile: 1.1357858180999756
10th percentile: 1.181525468826294
20th percentile: 1.2730047702789307
30th percentile: 1.3482934951782226
40th percentile: 1.40739164352417
50th percentile: 1.4664897918701172
60th percentile: 1.493167781829834
70th percentile: 1.5198457717895508
80th percentile: 1.7188549995422364
90th percentile: 2.090195465087891
95th percentile: 2.2758656978607177
99th percentile: 2.4244018840789794
mean time: 1.5740002155303956
Pipeline stage StressChecker completed in 8.74s
mistralai-mixtral-8x7b_3473_v119 status is now deployed due to DeploymentManager action
mistralai-mixtral-8x7b_3473_v119 status is now inactive due to auto deactivation removed underperforming models
mistralai-mixtral-8x7b_3473_v119 status is now torndown due to DeploymentManager action

Usage Metrics

Latency Metrics