submission_id: mistralai-mixtral-8x7b_3473_v106
developer_uid: chai_backend_admin
alignment_samples: 719803
alignment_score: 0.5528901895678846
best_of: 4
celo_rating: 1170.77
display_name: mistralai-mixtral-8x7b_3473_v106
formatter: {'memory_template': '<s>[INST] This is an entertaining conversation. You are {bot_name} who has the persona: {memory}.\nEngage in a chat with {user_name} while staying in character. Try to flirt with {user_name}. Engage in *roleplay* actions. Describe the scene dramatically. \n', 'prompt_template': '{prompt}\n', 'bot_template': '{bot_name}: {message}</s>', 'user_template': '[INST] {user_name}: {message} [/INST]', 'response_template': '{bot_name}:', 'truncate_by_message': False}
generation_params: {'temperature': 0.9, 'top_p': 1.0, 'min_p': 0.0, 'top_k': 50, 'presence_penalty': 0.5, 'frequency_penalty': 0.5, 'stopping_words': ['\n', '</s>', '<|user|>', '###'], 'max_input_tokens': 512, 'best_of': 4, 'max_output_tokens': 64, 'reward_max_token_input': 256}
is_internal_developer: True
language_model: mistralai/Mixtral-8x7B-Instruct-v0.1
max_input_tokens: 512
max_output_tokens: 64
model_architecture: MixtralForCausalLM
model_group: mistralai/Mixtral-8x7B-I
model_name: mistralai-mixtral-8x7b_3473_v106
model_num_parameters: 46702792704.0
model_repo: mistralai/Mixtral-8x7B-Instruct-v0.1
model_size: 47B
num_battles: 758492
num_wins: 311780
propriety_score: 0.7342363381597384
propriety_total_count: 70653.0
ranking_group: single
reward_formatter: {'bot_template': 'Bot: {message}\n', 'memory_template': '', 'prompt_template': '', 'response_template': 'Bot:', 'truncate_by_message': False, 'user_template': 'User: {message}\n'}
reward_repo: ChaiML/gpt2_xl_pairwise_89m_step_347634
status: torndown
submission_type: basic
timestamp: 2024-08-03T21:30:08+00:00
us_pacific_date: 2024-08-03
win_ratio: 0.4110524567167485
Download Preference Data
Resubmit model
Running pipeline stage MKMLizer
Starting job with name mistralai-mixtral-8x7b-3473-v106-mkmlizer
Waiting for job on mistralai-mixtral-8x7b-3473-v106-mkmlizer to finish
mistralai-mixtral-8x7b-3473-v106-mkmlizer: ╔═════════════════════════════════════════════════════════════════════╗
mistralai-mixtral-8x7b-3473-v106-mkmlizer: ║ _____ __ __ ║
mistralai-mixtral-8x7b-3473-v106-mkmlizer: ║ / _/ /_ ___ __/ / ___ ___ / / ║
mistralai-mixtral-8x7b-3473-v106-mkmlizer: ║ / _/ / // / |/|/ / _ \/ -_) -_) / ║
mistralai-mixtral-8x7b-3473-v106-mkmlizer: ║ /_//_/\_, /|__,__/_//_/\__/\__/_/ ║
mistralai-mixtral-8x7b-3473-v106-mkmlizer: ║ /___/ ║
mistralai-mixtral-8x7b-3473-v106-mkmlizer: ║ ║
mistralai-mixtral-8x7b-3473-v106-mkmlizer: ║ Version: 0.9.9 ║
mistralai-mixtral-8x7b-3473-v106-mkmlizer: ║ Copyright 2023 MK ONE TECHNOLOGIES Inc. ║
mistralai-mixtral-8x7b-3473-v106-mkmlizer: ║ https://mk1.ai ║
mistralai-mixtral-8x7b-3473-v106-mkmlizer: ║ ║
mistralai-mixtral-8x7b-3473-v106-mkmlizer: ║ The license key for the current software has been verified as ║
mistralai-mixtral-8x7b-3473-v106-mkmlizer: ║ belonging to: ║
mistralai-mixtral-8x7b-3473-v106-mkmlizer: ║ ║
mistralai-mixtral-8x7b-3473-v106-mkmlizer: ║ Chai Research Corp. ║
mistralai-mixtral-8x7b-3473-v106-mkmlizer: ║ Account ID: 7997a29f-0ceb-4cc7-9adf-840c57b4ae6f ║
mistralai-mixtral-8x7b-3473-v106-mkmlizer: ║ Expiration: 2024-10-15 23:59:59 ║
mistralai-mixtral-8x7b-3473-v106-mkmlizer: ║ ║
mistralai-mixtral-8x7b-3473-v106-mkmlizer: ╚═════════════════════════════════════════════════════════════════════╝
mistralai-mixtral-8x7b-3473-v106-mkmlizer: Downloaded to shared memory in 197.086s
mistralai-mixtral-8x7b-3473-v106-mkmlizer: quantizing model to /dev/shm/model_cache, profile:s0, folder:/tmp/tmp3lbm6mlv, device:0
mistralai-mixtral-8x7b-3473-v106-mkmlizer: Saving flywheel model at /dev/shm/model_cache
mistralai-mixtral-8x7b-3473-v106-mkmlizer: creating bucket guanaco-mkml-models
mistralai-mixtral-8x7b-3473-v106-mkmlizer: Bucket 's3://guanaco-mkml-models/' created
mistralai-mixtral-8x7b-3473-v106-mkmlizer: uploading /dev/shm/model_cache to s3://guanaco-mkml-models/mistralai-mixtral-8x7b-3473-v106
mistralai-mixtral-8x7b-3473-v106-mkmlizer: cp /dev/shm/model_cache/config.json s3://guanaco-mkml-models/mistralai-mixtral-8x7b-3473-v106/config.json
mistralai-mixtral-8x7b-3473-v106-mkmlizer: cp /dev/shm/model_cache/special_tokens_map.json s3://guanaco-mkml-models/mistralai-mixtral-8x7b-3473-v106/special_tokens_map.json
mistralai-mixtral-8x7b-3473-v106-mkmlizer: cp /dev/shm/model_cache/tokenizer_config.json s3://guanaco-mkml-models/mistralai-mixtral-8x7b-3473-v106/tokenizer_config.json
mistralai-mixtral-8x7b-3473-v106-mkmlizer: cp /dev/shm/model_cache/tokenizer.model s3://guanaco-mkml-models/mistralai-mixtral-8x7b-3473-v106/tokenizer.model
mistralai-mixtral-8x7b-3473-v106-mkmlizer: cp /dev/shm/model_cache/tokenizer.json s3://guanaco-mkml-models/mistralai-mixtral-8x7b-3473-v106/tokenizer.json
mistralai-mixtral-8x7b-3473-v106-mkmlizer: cp /dev/shm/model_cache/flywheel_model.3.safetensors s3://guanaco-mkml-models/mistralai-mixtral-8x7b-3473-v106/flywheel_model.3.safetensors
mistralai-mixtral-8x7b-3473-v106-mkmlizer: cp /dev/shm/model_cache/flywheel_model.2.safetensors s3://guanaco-mkml-models/mistralai-mixtral-8x7b-3473-v106/flywheel_model.2.safetensors
mistralai-mixtral-8x7b-3473-v106-mkmlizer: cp /dev/shm/model_cache/flywheel_model.0.safetensors s3://guanaco-mkml-models/mistralai-mixtral-8x7b-3473-v106/flywheel_model.0.safetensors
mistralai-mixtral-8x7b-3473-v106-mkmlizer: cp /dev/shm/model_cache/flywheel_model.1.safetensors s3://guanaco-mkml-models/mistralai-mixtral-8x7b-3473-v106/flywheel_model.1.safetensors
mistralai-mixtral-8x7b-3473-v106-mkmlizer: loading reward model from ChaiML/gpt2_xl_pairwise_89m_step_347634
mistralai-mixtral-8x7b-3473-v106-mkmlizer: Loading 0: 0%| | 0/995 [00:00<?, ?it/s] Loading 0: 0%| | 4/995 [00:00<00:39, 24.91it/s] Loading 0: 1%| | 7/995 [00:00<00:42, 23.47it/s] Loading 0: 1%| | 10/995 [00:00<00:40, 24.55it/s] Loading 0: 1%|▏ | 13/995 [00:00<00:46, 20.91it/s] Loading 0: 2%|▏ | 16/995 [00:00<00:48, 20.06it/s] Loading 0: 2%|▏ | 19/995 [00:00<00:46, 21.11it/s] Loading 0: 2%|▏ | 22/995 [00:01<00:45, 21.45it/s] Loading 0: 3%|▎ | 25/995 [00:01<00:46, 21.08it/s] Loading 0: 3%|▎ | 32/995 [00:01<00:29, 32.64it/s] Loading 0: 4%|▎ | 36/995 [00:01<00:33, 28.85it/s] Loading 0: 4%|▍ | 40/995 [00:01<00:33, 28.14it/s] Loading 0: 4%|▍ | 44/995 [00:01<00:38, 24.46it/s] Loading 0: 5%|▌ | 51/995 [00:01<00:29, 32.14it/s] Loading 0: 6%|▌ | 55/995 [00:02<00:47, 19.64it/s] Loading 0: 6%|▌ | 58/995 [00:02<00:46, 20.18it/s] Loading 0: 6%|▌ | 61/995 [00:02<00:45, 20.63it/s] Loading 0: 7%|▋ | 66/995 [00:02<00:36, 25.25it/s] Loading 0: 7%|▋ | 70/995 [00:02<00:35, 26.03it/s] Loading 0: 7%|▋ | 73/995 [00:03<00:35, 25.79it/s] Loading 0: 8%|▊ | 76/995 [00:03<00:35, 25.95it/s] Loading 0: 8%|▊ | 79/995 [00:03<00:35, 26.01it/s] Loading 0: 8%|▊ | 82/995 [00:03<00:35, 25.71it/s] Loading 0: 9%|▊ | 85/995 [00:03<00:35, 25.74it/s] Loading 0: 9%|▉ | 91/995 [00:03<00:26, 34.25it/s] Loading 0: 10%|▉ | 95/995 [00:03<00:25, 35.22it/s] Loading 0: 10%|▉ | 99/995 [00:03<00:27, 32.85it/s] Loading 0: 10%|█ | 103/995 [00:03<00:26, 34.20it/s] Loading 0: 11%|█ | 107/995 [00:04<00:35, 25.08it/s] Loading 0: 11%|█ | 110/995 [00:04<00:35, 24.67it/s] Loading 0: 11%|█▏ | 113/995 [00:04<00:35, 24.58it/s] Loading 0: 12%|█▏ | 116/995 [00:04<00:36, 24.22it/s] Loading 0: 12%|█▏ | 119/995 [00:04<00:37, 23.13it/s] Loading 0: 12%|█▏ | 122/995 [00:04<00:37, 23.07it/s] Loading 0: 13%|█▎ | 126/995 [00:04<00:33, 26.05it/s] Loading 0: 13%|█▎ | 129/995 [00:05<00:33, 25.72it/s] Loading 0: 13%|█▎ | 132/995 [00:05<00:34, 24.67it/s] Loading 0: 14%|█▎ | 135/995 [00:05<00:35, 24.24it/s] Loading 0: 14%|█▍ | 138/995 [00:05<00:35, 24.31it/s] Loading 0: 14%|█▍ | 141/995 [00:05<00:36, 23.45it/s] Loading 0: 14%|█▍ | 144/995 [00:05<00:36, 23.33it/s] Loading 0: 15%|█▍ | 147/995 [00:05<00:43, 19.61it/s] Loading 0: 15%|█▌ | 150/995 [00:06<00:39, 21.47it/s] Loading 0: 16%|█▌ | 161/995 [00:06<00:22, 36.26it/s] Loading 0: 17%|█▋ | 165/995 [00:06<00:34, 23.80it/s] Loading 0: 17%|█▋ | 168/995 [00:06<00:34, 23.64it/s] Loading 0: 17%|█▋ | 171/995 [00:06<00:34, 23.61it/s] Loading 0: 17%|█▋ | 174/995 [00:06<00:35, 23.37it/s] Loading 0: 18%|█▊ | 177/995 [00:07<00:35, 23.32it/s] Loading 0: 18%|█▊ | 180/995 [00:07<00:35, 23.25it/s] Loading 0: 18%|█▊ | 183/995 [00:07<00:34, 23.21it/s] Loading 0: 19%|█▊ | 186/995 [00:07<00:32, 24.69it/s] Loading 0: 19%|█▉ | 190/995 [00:07<00:30, 26.53it/s] Loading 0: 19%|█▉ | 193/995 [00:07<00:30, 26.16it/s] Loading 0: 20%|█▉ | 196/995 [00:07<00:30, 25.95it/s] Loading 0: 20%|██ | 199/995 [00:07<00:30, 25.69it/s] Loading 0: 20%|██ | 202/995 [00:08<00:30, 25.64it/s] Loading 0: 21%|██ | 209/995 [00:08<00:21, 36.01it/s] Loading 0: 21%|██▏ | 213/995 [00:08<00:37, 21.04it/s] Loading 0: 22%|██▏ | 216/995 [00:08<00:36, 21.55it/s] Loading 0: 22%|██▏ | 221/995 [00:08<00:29, 25.94it/s] Loading 0: 23%|██▎ | 225/995 [00:08<00:28, 26.65it/s] Loading 0: 23%|██▎ | 229/995 [00:09<00:27, 27.46it/s] Loading 0: 23%|██▎ | 233/995 [00:09<00:30, 24.74it/s] Loading 0: 24%|██▎ | 236/995 [00:09<00:31, 24.29it/s] Loading 0: 24%|██▍ | 239/995 [00:09<00:32, 23.43it/s] Loading 0: 24%|██▍ | 242/995 [00:09<00:33, 22.24it/s] Loading 0: 25%|██▌ | 250/995 [00:09<00:21, 34.11it/s] Loading 0: 26%|██▌ | 254/995 [00:10<00:24, 30.04it/s] Loading 0: 26%|██▌ | 258/995 [00:10<00:28, 25.62it/s] Loading 0: 27%|██▋ | 264/995 [00:10<00:22, 32.17it/s] Loading 0: 27%|██▋ | 268/995 [00:10<00:31, 23.04it/s] Loading 0: 27%|██▋ | 272/995 [00:10<00:27, 25.86it/s] Loading 0: 28%|██▊ | 276/995 [00:10<00:28, 25.56it/s] Loading 0: 28%|██▊ | 280/995 [00:11<00:31, 23.00it/s] Loading 0: 28%|██▊ | 282/995 [00:26<00:30, 23.00it/s] Loading 0: 28%|██▊ | 283/995 [00:26<15:04, 1.27s/it] Loading 0: 29%|██▊ | 285/995 [00:27<12:25, 1.05s/it] Loading 0: 29%|██▉ | 288/995 [00:27<09:03, 1.30it/s] Loading 0: 29%|██▉ | 291/995 [00:27<06:37, 1.77it/s] Loading 0: 30%|██▉ | 294/995 [00:27<04:49, 2.42it/s] Loading 0: 30%|██▉ | 297/995 [00:27<03:33, 3.28it/s] Loading 0: 30%|███ | 300/995 [00:27<02:37, 4.40it/s] Loading 0: 30%|███ | 303/995 [00:27<01:58, 5.84it/s] Loading 0: 31%|███ | 306/995 [00:28<01:31, 7.55it/s] Loading 0: 31%|███ | 309/995 [00:28<01:11, 9.55it/s] Loading 0: 31%|███▏ | 312/995 [00:28<00:58, 11.68it/s] Loading 0: 32%|███▏ | 320/995 [00:28<00:39, 17.06it/s] Loading 0: 32%|███▏ | 323/995 [00:28<00:36, 18.40it/s] Loading 0: 33%|███▎ | 326/995 [00:28<00:34, 19.53it/s] Loading 0: 33%|███▎ | 329/995 [00:28<00:31, 20.82it/s] Loading 0: 33%|███▎ | 332/995 [00:29<00:30, 21.97it/s] Loading 0: 34%|███▎ | 335/995 [00:29<00:28, 22.83it/s] Loading 0: 34%|███▍ | 338/995 [00:29<00:28, 23.08it/s] Loading 0: 34%|███▍ | 341/995 [00:29<00:27, 23.59it/s] Loading 0: 35%|███▍ | 345/995 [00:29<00:25, 25.12it/s] Loading 0: 35%|███▍ | 348/995 [00:29<00:27, 23.69it/s] Loading 0: 35%|███▌ | 351/995 [00:29<00:27, 23.26it/s] Loading 0: 36%|███▌ | 354/995 [00:29<00:27, 23.48it/s] Loading 0: 36%|███▌ | 357/995 [00:30<00:27, 23.57it/s] Loading 0: 36%|███▌ | 360/995 [00:30<00:27, 23.43it/s] Loading 0: 37%|███▋ | 367/995 [00:30<00:19, 32.70it/s] Loading 0: 37%|███▋ | 371/995 [00:30<00:33, 18.48it/s] Loading 0: 38%|███▊ | 376/995 [00:30<00:27, 22.18it/s] Loading 0: 38%|███▊ | 379/995 [00:31<00:28, 21.90it/s] Loading 0: 38%|███▊ | 382/995 [00:31<00:28, 21.84it/s] Loading 0: 39%|███▊ | 385/995 [00:31<00:27, 22.17it/s] Loading 0: 39%|███▉ | 388/995 [00:31<00:27, 22.41it/s] Loading 0: 39%|███▉ | 391/995 [00:31<00:26, 22.41it/s] Loading 0: 40%|███▉ | 394/995 [00:31<00:26, 22.26it/s] Loading 0: 40%|███▉ | 397/995 [00:31<00:26, 22.21it/s] Loading 0: 41%|████ | 405/995 [00:31<00:17, 34.47it/s] Loading 0: 41%|████ | 409/995 [00:32<00:18, 31.88it/s] Loading 0: 42%|████▏ | 413/995 [00:32<00:22, 25.63it/s] Loading 0: 42%|████▏ | 416/995 [00:32<00:22, 25.22it/s] Loading 0: 42%|████▏ | 422/995 [00:32<00:17, 32.64it/s] Loading 0: 43%|████▎ | 426/995 [00:32<00:24, 23.08it/s] Loading 0: 43%|████▎ | 429/995 [00:33<00:24, 23.50it/s] Loading 0: 43%|████▎ | 432/995 [00:33<00:23, 23.79it/s] Loading 0: 44%|████▍ | 437/995 [00:33<00:19, 28.50it/s] Loading 0: 44%|████▍ | 441/995 [00:33<00:21, 25.73it/s] Loading 0: 45%|████▍ | 444/995 [00:33<00:21, 25.70it/s] Loading 0: 45%|████▍ | 447/995 [00:33<00:21, 25.88it/s] Loading 0: 45%|████▌ | 450/995 [00:33<00:21, 25.87it/s] Loading 0: 46%|████▌ | 453/995 [00:33<00:21, 25.70it/s] Loading 0: 46%|████▌ | 456/995 [00:34<00:21, 25.61it/s] Loading 0: 46%|████▌ | 459/995 [00:34<00:21, 24.86it/s] Loading 0: 47%|████▋ | 468/995 [00:34<00:13, 40.18it/s] Loading 0: 48%|████▊ | 473/995 [00:34<00:15, 33.62it/s] Loading 0: 48%|████▊ | 478/995 [00:34<00:19, 26.25it/s] Loading 0: 48%|████▊ | 482/995 [00:34<00:19, 25.69it/s] Loading 0: 49%|████▊ | 485/995 [00:35<00:20, 24.80it/s] Loading 0: 49%|████▉ | 488/995 [00:35<00:20, 24.37it/s] Loading 0: 49%|████▉ | 491/995 [00:35<00:21, 23.21it/s] Loading 0: 50%|████▉ | 494/995 [00:35<00:22, 22.51it/s] Loading 0: 50%|█████ | 498/995 [00:35<00:19, 25.33it/s] Loading 0: 50%|█████ | 501/995 [00:35<00:20, 23.74it/s] Loading 0: 51%|█████ | 504/995 [00:35<00:22, 22.24it/s] Loading 0: 51%|█████ | 507/995 [00:36<00:21, 22.48it/s] Loading 0: 51%|█████▏ | 510/995 [00:36<00:21, 22.56it/s] Loading 0: 52%|█████▏ | 513/995 [00:36<00:21, 22.08it/s] Loading 0: 52%|█████▏ | 516/995 [00:36<00:22, 21.70it/s] Loading 0: 52%|█████▏ | 519/995 [00:36<00:22, 21.27it/s] Loading 0: 53%|█████▎ | 525/995 [00:36<00:16, 28.57it/s] Loading 0: 53%|█████▎ | 528/995 [00:37<00:23, 19.60it/s] Loading 0: 53%|█████▎ | 531/995 [00:37<00:22, 20.41it/s] Loading 0: 54%|█████▎ | 534/995 [00:37<00:21, 20.98it/s] Loading 0: 54%|█████▍ | 537/995 [00:37<00:21, 21.12it/s] Loading 0: 54%|█████▍ | 540/995 [00:37<00:21, 20.80it/s] Loading 0: 55%|█████▍ | 543/995 [00:37<00:20, 21.53it/s] Loading 0: 55%|█████▍ | 546/995 [00:37<00:19, 22.55it/s] Loading 0: 55%|█████▌ | 549/995 [00:37<00:19, 23.33it/s] Loading 0: 55%|█████▌ | 552/995 [00:38<00:18, 23.53it/s] Loading 0: 56%|█████▋ | 561/995 [00:38<00:11, 38.83it/s] Loading 0: 56%|█████▋ | 561/995 [00:53<00:11, 38.83it/s] Loading 0: 56%|█████▋ | 562/995 [00:53<09:17, 1.29s/it] Loading 0: 57%|█████▋ | 564/995 [00:53<07:35, 1.06s/it] Loading 0: 57%|█████▋ | 568/995 [00:54<05:04, 1.40it/s] Loading 0: 57%|█████▋ | 571/995 [00:54<03:43, 1.89it/s] Loading 0: 58%|█████▊ | 574/995 [00:54<02:45, 2.55it/s] Loading 0: 58%|█████▊ | 580/995 [00:54<01:33, 4.42it/s] Loading 0: 59%|█████▊ | 584/995 [00:54<01:15, 5.47it/s] Loading 0: 59%|█████▉ | 587/995 [00:54<01:00, 6.74it/s] Loading 0: 59%|█████▉ | 591/995 [00:55<00:44, 9.13it/s] Loading 0: 60%|█████▉ | 595/995 [00:55<00:34, 11.61it/s] Loading 0: 60%|██████ | 599/995 [00:55<00:29, 13.25it/s] Loading 0: 61%|██████ | 602/995 [00:55<00:26, 14.95it/s] Loading 0: 61%|██████ | 605/995 [00:55<00:23, 16.61it/s] Loading 0: 61%|██████ | 608/995 [00:55<00:21, 18.36it/s] Loading 0: 61%|██████▏ | 611/995 [00:55<00:19, 19.89it/s] Loading 0: 62%|██████▏ | 614/995 [00:55<00:18, 20.57it/s] Loading 0: 63%|██████▎ | 623/995 [00:56<00:10, 34.59it/s] Loading 0: 63%|██████▎ | 628/995 [00:56<00:12, 29.74it/s] Loading 0: 64%|██████▎ | 632/995 [00:56<00:11, 31.73it/s] Loading 0: 64%|██████▍ | 636/995 [00:56<00:14, 24.59it/s] Loading 0: 64%|██████▍ | 640/995 [00:56<00:14, 24.85it/s] Loading 0: 65%|██████▍ | 643/995 [00:56<00:14, 24.88it/s] Loading 0: 65%|██████▍ | 646/995 [00:57<00:14, 24.92it/s] Loading 0: 65%|██████▌ | 649/995 [00:57<00:13, 25.02it/s] Loading 0: 66%|██████▌ | 653/995 [00:57<00:12, 28.18it/s] Loading 0: 66%|██████▌ | 656/995 [00:57<00:12, 27.18it/s] Loading 0: 66%|██████▌ | 659/995 [00:57<00:12, 26.32it/s] Loading 0: 67%|██████▋ | 662/995 [00:57<00:13, 24.91it/s] Loading 0: 67%|██████▋ | 665/995 [00:57<00:13, 24.84it/s] Loading 0: 67%|██████▋ | 668/995 [00:57<00:13, 25.00it/s] Loading 0: 67%|██████▋ | 671/995 [00:58<00:13, 24.87it/s] Loading 0: 68%|██████▊ | 674/995 [00:58<00:13, 24.56it/s] Loading 0: 68%|██████▊ | 680/995 [00:58<00:09, 33.42it/s] Loading 0: 69%|██████▉ | 685/995 [00:58<00:08, 37.24it/s] Loading 0: 69%|██████▉ | 691/995 [00:58<00:11, 26.96it/s] Loading 0: 70%|██████▉ | 695/995 [00:58<00:10, 27.33it/s] Loading 0: 70%|███████ | 699/995 [00:58<00:10, 27.88it/s] Loading 0: 71%|███████ | 703/995 [00:59<00:11, 25.24it/s] Loading 0: 71%|███████ | 706/995 [00:59<00:11, 24.94it/s] Loading 0: 71%|███████▏ | 709/995 [00:59<00:11, 24.52it/s] Loading 0: 72%|███████▏ | 712/995 [00:59<00:11, 24.36it/s] Loading 0: 72%|███████▏ | 717/995 [00:59<00:09, 29.10it/s] Loading 0: 72%|███████▏ | 721/995 [00:59<00:09, 28.94it/s] Loading 0: 73%|███████▎ | 724/995 [00:59<00:09, 27.74it/s] Loading 0: 73%|███████▎ | 727/995 [01:00<00:09, 26.91it/s] Loading 0: 73%|███████▎ | 730/995 [01:00<00:10, 26.46it/s] Loading 0: 74%|███████▎ | 733/995 [01:00<00:10, 25.58it/s] Loading 0: 74%|███████▍ | 739/995 [01:00<00:10, 24.29it/s] Loading 0: 75%|███████▍ | 742/995 [01:00<00:10, 24.28it/s] Loading 0: 75%|███████▍ | 746/995 [01:00<00:09, 27.41it/s] Loading 0: 75%|███████▌ | 749/995 [01:00<00:09, 26.60it/s] Loading 0: 76%|███████▌ | 752/995 [01:01<00:09, 26.37it/s] Loading 0: 76%|███████▌ | 755/995 [01:01<00:09, 26.20it/s] Loading 0: 76%|███████▌ | 758/995 [01:01<00:09, 26.00it/s] Loading 0: 76%|███████▋ | 761/995 [01:01<00:09, 25.08it/s] Loading 0: 77%|███████▋ | 764/995 [01:01<00:09, 25.27it/s] Loading 0: 77%|███████▋ | 767/995 [01:01<00:08, 25.35it/s] Loading 0: 78%|███████▊ | 773/995 [01:01<00:06, 34.17it/s] Loading 0: 78%|███████▊ | 778/995 [01:01<00:05, 37.26it/s] Loading 0: 79%|███████▊ | 782/995 [01:02<00:07, 29.87it/s] Loading 0: 79%|███████▉ | 786/995 [01:02<00:07, 28.66it/s] Loading 0: 79%|███████▉ | 791/995 [01:02<00:06, 33.05it/s] Loading 0: 80%|███████▉ | 795/995 [01:02<00:08, 24.33it/s] Loading 0: 80%|████████ | 798/995 [01:02<00:08, 24.16it/s] Loading 0: 81%|████████ | 801/995 [01:02<00:07, 24.42it/s] Loading 0: 81%|████████ | 804/995 [01:02<00:07, 24.61it/s] Loading 0: 81%|████████ | 808/995 [01:03<00:06, 27.53it/s] Loading 0: 82%|████████▏ | 811/995 [01:03<00:06, 26.64it/s] Loading 0: 82%|████████▏ | 814/995 [01:03<00:06, 26.05it/s] Loading 0: 82%|████████▏ | 817/995 [01:03<00:06, 25.67it/s] Loading 0: 82%|████████▏ | 820/995 [01:03<00:07, 24.33it/s] Loading 0: 83%|████████▎ | 823/995 [01:03<00:07, 24.29it/s] Loading 0: 83%|████████▎ | 826/995 [01:03<00:06, 24.23it/s] Loading 0: 83%|████████▎ | 829/995 [01:03<00:06, 24.20it/s] Loading 0: 84%|████████▎ | 832/995 [01:04<00:06, 25.49it/s] Loading 0: 84%|████████▍ | 839/995 [01:04<00:04, 37.26it/s] Loading 0: 85%|████████▍ | 842/995 [01:19<00:04, 37.26it/s] Loading 0: 85%|████████▍ | 843/995 [01:19<02:49, 1.12s/it] Loading 0: 85%|████████▌ | 848/995 [01:19<01:49, 1.34it/s] Loading 0: 86%|████████▌ | 853/995 [01:19<01:15, 1.88it/s] Loading 0: 86%|████████▌ | 857/995 [01:20<00:54, 2.51it/s] Loading 0: 86%|████████▋ | 860/995 [01:20<00:42, 3.15it/s] Loading 0: 87%|████████▋ | 863/995 [01:20<00:33, 3.99it/s] Loading 0: 87%|████████▋ | 866/995 [01:20<00:25, 5.09it/s] Loading 0: 87%|████████▋ | 870/995 [01:20<00:17, 7.14it/s] Loading 0: 88%|████████▊ | 873/995 [01:20<00:13, 8.77it/s] Loading 0: 88%|████████▊ | 876/995 [01:20<00:11, 10.66it/s] Loading 0: 88%|████████▊ | 879/995 [01:20<00:09, 12.64it/s] Loading 0: 89%|████████▊ | 882/995 [01:21<00:07, 14.68it/s] Loading 0: 89%|████████▉ | 885/995 [01:21<00:06, 16.70it/s] Loading 0: 89%|████████▉ | 888/995 [01:21<00:05, 18.26it/s] Loading 0: 90%|████████▉ | 891/995 [01:21<00:05, 19.43it/s] Loading 0: 90%|█████████ | 897/995 [01:21<00:04, 20.07it/s] Loading 0: 91%|█████████ | 901/995 [01:21<00:04, 23.43it/s] Loading 0: 91%|█████████ | 904/995 [01:21<00:03, 23.57it/s] Loading 0: 91%|█████████ | 907/995 [01:22<00:03, 23.93it/s] Loading 0: 91%|█████████▏| 910/995 [01:22<00:03, 24.15it/s] Loading 0: 92%|█████████▏| 913/995 [01:22<00:03, 24.11it/s] Loading 0: 92%|█████████▏| 916/995 [01:22<00:03, 24.43it/s] Loading 0: 92%|█████████▏| 919/995 [01:22<00:03, 24.78it/s] Loading 0: 93%|█████████▎| 922/995 [01:22<00:02, 24.95it/s] Loading 0: 93%|█████████▎| 928/995 [01:22<00:01, 33.55it/s] Loading 0: 94%|█████████▍| 933/995 [01:22<00:01, 36.27it/s] Loading 0: 94%|█████████▍| 937/995 [01:23<00:02, 28.78it/s] Loading 0: 95%|█████████▍| 941/995 [01:23<00:01, 28.12it/s] Loading 0: 95%|█████████▍| 945/995 [01:23<00:01, 27.95it/s] Loading 0: 96%|█████████▌| 951/995 [01:23<00:01, 31.03it/s] Loading 0: 96%|█████████▌| 955/995 [01:25<00:05, 7.51it/s] Loading 0: 96%|█████████▋| 958/995 [01:25<00:04, 8.89it/s] Loading 0: 97%|█████████▋| 961/995 [01:25<00:03, 10.55it/s] Loading 0: 97%|█████████▋| 966/995 [01:25<00:02, 14.45it/s] Loading 0: 97%|█████████▋| 969/995 [01:25<00:01, 16.00it/s] Loading 0: 98%|█████████▊| 972/995 [01:25<00:01, 17.51it/s] Loading 0: 98%|█████████▊| 975/995 [01:25<00:01, 19.11it/s] Loading 0: 98%|█████████▊| 978/995 [01:26<00:00, 20.37it/s] Loading 0: 99%|█████████▊| 981/995 [01:26<00:00, 21.38it/s] Loading 0: 99%|█████████▉| 984/995 [01:26<00:00, 22.07it/s] Loading 0: 99%|█████████▉| 987/995 [01:26<00:00, 22.90it/s] /opt/conda/lib/python3.10/site-packages/transformers/models/auto/configuration_auto.py:957: FutureWarning: The `use_auth_token` argument is deprecated and will be removed in v5 of Transformers. Please use `token` instead.
mistralai-mixtral-8x7b-3473-v106-mkmlizer: warnings.warn(
mistralai-mixtral-8x7b-3473-v106-mkmlizer: /opt/conda/lib/python3.10/site-packages/transformers/models/auto/tokenization_auto.py:785: FutureWarning: The `use_auth_token` argument is deprecated and will be removed in v5 of Transformers. Please use `token` instead.
mistralai-mixtral-8x7b-3473-v106-mkmlizer: warnings.warn(
mistralai-mixtral-8x7b-3473-v106-mkmlizer: /opt/conda/lib/python3.10/site-packages/transformers/models/auto/auto_factory.py:469: FutureWarning: The `use_auth_token` argument is deprecated and will be removed in v5 of Transformers. Please use `token` instead.
mistralai-mixtral-8x7b-3473-v106-mkmlizer: warnings.warn(
Retrying (%r) after connection broken by '%r': %s
Retrying (%r) after connection broken by '%r': %s
Retrying (%r) after connection broken by '%r': %s
mistralai-mixtral-8x7b-3473-v106-mkmlizer: Downloading shards: 0%| | 0/2 [00:00<?, ?it/s] Downloading shards: 50%|█████ | 1/2 [00:05<00:05, 5.42s/it] Downloading shards: 100%|██████████| 2/2 [00:08<00:00, 4.05s/it] Downloading shards: 100%|██████████| 2/2 [00:08<00:00, 4.26s/it]
mistralai-mixtral-8x7b-3473-v106-mkmlizer: Loading checkpoint shards: 0%| | 0/2 [00:00<?, ?it/s] Loading checkpoint shards: 50%|█████ | 1/2 [00:00<00:00, 2.28it/s] Loading checkpoint shards: 100%|██████████| 2/2 [00:00<00:00, 3.72it/s] Loading checkpoint shards: 100%|██████████| 2/2 [00:00<00:00, 3.40it/s]
mistralai-mixtral-8x7b-3473-v106-mkmlizer: Saving model to /tmp/reward_cache/reward.tensors
mistralai-mixtral-8x7b-3473-v106-mkmlizer: Saving duration: 1.380s
mistralai-mixtral-8x7b-3473-v106-mkmlizer: Processed model ChaiML/gpt2_xl_pairwise_89m_step_347634 in 13.840s
mistralai-mixtral-8x7b-3473-v106-mkmlizer: creating bucket guanaco-reward-models
mistralai-mixtral-8x7b-3473-v106-mkmlizer: Bucket 's3://guanaco-reward-models/' created
mistralai-mixtral-8x7b-3473-v106-mkmlizer: uploading /tmp/reward_cache to s3://guanaco-reward-models/mistralai-mixtral-8x7b-3473-v106_reward
mistralai-mixtral-8x7b-3473-v106-mkmlizer: cp /tmp/reward_cache/config.json s3://guanaco-reward-models/mistralai-mixtral-8x7b-3473-v106_reward/config.json
mistralai-mixtral-8x7b-3473-v106-mkmlizer: cp /tmp/reward_cache/special_tokens_map.json s3://guanaco-reward-models/mistralai-mixtral-8x7b-3473-v106_reward/special_tokens_map.json
mistralai-mixtral-8x7b-3473-v106-mkmlizer: cp /tmp/reward_cache/tokenizer_config.json s3://guanaco-reward-models/mistralai-mixtral-8x7b-3473-v106_reward/tokenizer_config.json
mistralai-mixtral-8x7b-3473-v106-mkmlizer: cp /tmp/reward_cache/merges.txt s3://guanaco-reward-models/mistralai-mixtral-8x7b-3473-v106_reward/merges.txt
mistralai-mixtral-8x7b-3473-v106-mkmlizer: cp /tmp/reward_cache/vocab.json s3://guanaco-reward-models/mistralai-mixtral-8x7b-3473-v106_reward/vocab.json
mistralai-mixtral-8x7b-3473-v106-mkmlizer: cp /tmp/reward_cache/tokenizer.json s3://guanaco-reward-models/mistralai-mixtral-8x7b-3473-v106_reward/tokenizer.json
mistralai-mixtral-8x7b-3473-v106-mkmlizer: cp /tmp/reward_cache/reward.tensors s3://guanaco-reward-models/mistralai-mixtral-8x7b-3473-v106_reward/reward.tensors
Job mistralai-mixtral-8x7b-3473-v106-mkmlizer completed after 561.1s with status: succeeded
Stopping job with name mistralai-mixtral-8x7b-3473-v106-mkmlizer
Pipeline stage MKMLizer completed in 563.03s
Running pipeline stage MKMLKubeTemplater
Pipeline stage MKMLKubeTemplater completed in 0.45s
Running pipeline stage ISVCDeployer
Creating inference service mistralai-mixtral-8x7b-3473-v106
Waiting for inference service mistralai-mixtral-8x7b-3473-v106 to be ready
Inference service mistralai-mixtral-8x7b-3473-v106 ready after 152.83423805236816s
Pipeline stage ISVCDeployer completed in 154.97s
Running pipeline stage StressChecker
HTTP Request: %s %s "%s %d %s"
Received healthy response to inference request in 3.39204478263855s
HTTP Request: %s %s "%s %d %s"
Received healthy response to inference request in 1.8455009460449219s
HTTP Request: %s %s "%s %d %s"
Received healthy response to inference request in 1.8285801410675049s
HTTP Request: %s %s "%s %d %s"
Received healthy response to inference request in 1.6383411884307861s
HTTP Request: %s %s "%s %d %s"
Received healthy response to inference request in 1.8404710292816162s
5 requests
0 failed requests
5th percentile: 1.6763889789581299
10th percentile: 1.7144367694854736
20th percentile: 1.7905323505401611
30th percentile: 1.830958318710327
40th percentile: 1.8357146739959718
50th percentile: 1.8404710292816162
60th percentile: 1.8424829959869384
70th percentile: 1.8444949626922607
80th percentile: 2.154809713363648
90th percentile: 2.7734272480010986
95th percentile: 3.0827360153198238
99th percentile: 3.3301830291748047
mean time: 2.1089876174926756
Pipeline stage StressChecker completed in 14.55s
mistralai-mixtral-8x7b_3473_v106 status is now deployed due to DeploymentManager action
mistralai-mixtral-8x7b_3473_v106 status is now inactive due to auto deactivation removed underperforming models
mistralai-mixtral-8x7b_3473_v106 status is now torndown due to DeploymentManager action

Usage Metrics

Latency Metrics