submission_id: mistralai-mixtral-8x7b_3473_v112
developer_uid: chai_backend_admin
alignment_samples: 0
best_of: 4
celo_rating: 1177.64
display_name: mistralai-mixtral-8x7b_3473_v112
formatter: {'memory_template': '<s>[INST] This is an entertaining conversation. You are {bot_name} who has the persona: {memory}.\nEngage in a chat with {user_name} while staying in character. Try to flirt with {user_name}. Engage in *roleplay* actions. Describe the scene dramatically. \n', 'prompt_template': '{prompt}\n', 'bot_template': '{bot_name}: {message}</s>', 'user_template': '[INST] {user_name}: {message} [/INST]', 'response_template': '{bot_name}:', 'truncate_by_message': False}
generation_params: {'temperature': 0.9, 'top_p': 1.0, 'min_p': 0.0, 'top_k': 50, 'presence_penalty': 0.5, 'frequency_penalty': 0.5, 'stopping_words': ['\n', '</s>', '<|user|>', '###'], 'max_input_tokens': 512, 'best_of': 4, 'max_output_tokens': 64, 'reward_max_token_input': 256}
is_internal_developer: True
language_model: mistralai/Mixtral-8x7B-Instruct-v0.1
max_input_tokens: 512
max_output_tokens: 64
model_architecture: MixtralForCausalLM
model_group: mistralai/Mixtral-8x7B-I
model_name: mistralai-mixtral-8x7b_3473_v112
model_num_parameters: 46702792704.0
model_repo: mistralai/Mixtral-8x7B-Instruct-v0.1
model_size: 47B
num_battles: 38861
num_wins: 20340
propriety_score: 0.7237107461826563
propriety_total_count: 3471.0
ranking_group: single
reward_formatter: {'bot_template': 'Bot: {message}\n', 'memory_template': '', 'prompt_template': '', 'response_template': 'Bot:', 'truncate_by_message': False, 'user_template': 'User: {message}\n'}
reward_repo: ChaiML/llama_8b_pairwise_64m_256_tokens_step_249984
status: torndown
submission_type: basic
timestamp: 2024-08-05T15:03:39+00:00
us_pacific_date: 2024-08-05
win_ratio: 0.5234039268160882
Download Preference Data
Resubmit model
Running pipeline stage MKMLizer
Starting job with name mistralai-mixtral-8x7b-3473-v112-mkmlizer
Waiting for job on mistralai-mixtral-8x7b-3473-v112-mkmlizer to finish
mistralai-mixtral-8x7b-3473-v112-mkmlizer: ╔═════════════════════════════════════════════════════════════════════╗
mistralai-mixtral-8x7b-3473-v112-mkmlizer: ║ _____ __ __ ║
mistralai-mixtral-8x7b-3473-v112-mkmlizer: ║ / _/ /_ ___ __/ / ___ ___ / / ║
mistralai-mixtral-8x7b-3473-v112-mkmlizer: ║ / _/ / // / |/|/ / _ \/ -_) -_) / ║
mistralai-mixtral-8x7b-3473-v112-mkmlizer: ║ /_//_/\_, /|__,__/_//_/\__/\__/_/ ║
mistralai-mixtral-8x7b-3473-v112-mkmlizer: ║ /___/ ║
mistralai-mixtral-8x7b-3473-v112-mkmlizer: ║ ║
mistralai-mixtral-8x7b-3473-v112-mkmlizer: ║ Version: 0.9.9 ║
mistralai-mixtral-8x7b-3473-v112-mkmlizer: ║ Copyright 2023 MK ONE TECHNOLOGIES Inc. ║
mistralai-mixtral-8x7b-3473-v112-mkmlizer: ║ https://mk1.ai ║
mistralai-mixtral-8x7b-3473-v112-mkmlizer: ║ ║
mistralai-mixtral-8x7b-3473-v112-mkmlizer: ║ The license key for the current software has been verified as ║
mistralai-mixtral-8x7b-3473-v112-mkmlizer: ║ belonging to: ║
mistralai-mixtral-8x7b-3473-v112-mkmlizer: ║ ║
mistralai-mixtral-8x7b-3473-v112-mkmlizer: ║ Chai Research Corp. ║
mistralai-mixtral-8x7b-3473-v112-mkmlizer: ║ Account ID: 7997a29f-0ceb-4cc7-9adf-840c57b4ae6f ║
mistralai-mixtral-8x7b-3473-v112-mkmlizer: ║ Expiration: 2024-10-15 23:59:59 ║
mistralai-mixtral-8x7b-3473-v112-mkmlizer: ║ ║
mistralai-mixtral-8x7b-3473-v112-mkmlizer: ╚═════════════════════════════════════════════════════════════════════╝
mistralai-mixtral-8x7b-3473-v112-mkmlizer: Downloaded to shared memory in 185.320s
mistralai-mixtral-8x7b-3473-v112-mkmlizer: quantizing model to /dev/shm/model_cache, profile:s0, folder:/tmp/tmp9dxqcwhv, device:0
mistralai-mixtral-8x7b-3473-v112-mkmlizer: Saving flywheel model at /dev/shm/model_cache
mistralai-mixtral-8x7b-3473-v112-mkmlizer: quantized model in 92.537s
mistralai-mixtral-8x7b-3473-v112-mkmlizer: Processed model mistralai/Mixtral-8x7B-Instruct-v0.1 in 277.857s
mistralai-mixtral-8x7b-3473-v112-mkmlizer: creating bucket guanaco-mkml-models
mistralai-mixtral-8x7b-3473-v112-mkmlizer: Bucket 's3://guanaco-mkml-models/' created
mistralai-mixtral-8x7b-3473-v112-mkmlizer: uploading /dev/shm/model_cache to s3://guanaco-mkml-models/mistralai-mixtral-8x7b-3473-v112
mistralai-mixtral-8x7b-3473-v112-mkmlizer: cp /dev/shm/model_cache/config.json s3://guanaco-mkml-models/mistralai-mixtral-8x7b-3473-v112/config.json
mistralai-mixtral-8x7b-3473-v112-mkmlizer: cp /dev/shm/model_cache/special_tokens_map.json s3://guanaco-mkml-models/mistralai-mixtral-8x7b-3473-v112/special_tokens_map.json
mistralai-mixtral-8x7b-3473-v112-mkmlizer: cp /dev/shm/model_cache/tokenizer_config.json s3://guanaco-mkml-models/mistralai-mixtral-8x7b-3473-v112/tokenizer_config.json
mistralai-mixtral-8x7b-3473-v112-mkmlizer: cp /dev/shm/model_cache/tokenizer.model s3://guanaco-mkml-models/mistralai-mixtral-8x7b-3473-v112/tokenizer.model
mistralai-mixtral-8x7b-3473-v112-mkmlizer: cp /dev/shm/model_cache/tokenizer.json s3://guanaco-mkml-models/mistralai-mixtral-8x7b-3473-v112/tokenizer.json
mistralai-mixtral-8x7b-3473-v112-mkmlizer: cp /dev/shm/model_cache/flywheel_model.3.safetensors s3://guanaco-mkml-models/mistralai-mixtral-8x7b-3473-v112/flywheel_model.3.safetensors
mistralai-mixtral-8x7b-3473-v112-mkmlizer: cp /dev/shm/model_cache/flywheel_model.0.safetensors s3://guanaco-mkml-models/mistralai-mixtral-8x7b-3473-v112/flywheel_model.0.safetensors
mistralai-mixtral-8x7b-3473-v112-mkmlizer: cp /dev/shm/model_cache/flywheel_model.1.safetensors s3://guanaco-mkml-models/mistralai-mixtral-8x7b-3473-v112/flywheel_model.1.safetensors
mistralai-mixtral-8x7b-3473-v112-mkmlizer: cp /dev/shm/model_cache/flywheel_model.2.safetensors s3://guanaco-mkml-models/mistralai-mixtral-8x7b-3473-v112/flywheel_model.2.safetensors
mistralai-mixtral-8x7b-3473-v112-mkmlizer: loading reward model from ChaiML/llama_8b_pairwise_64m_256_tokens_step_249984
mistralai-mixtral-8x7b-3473-v112-mkmlizer: Loading 0: 0%| | 0/995 [00:00<?, ?it/s] Loading 0: 0%| | 4/995 [00:00<00:28, 34.21it/s] Loading 0: 1%| | 8/995 [00:00<00:32, 30.11it/s] Loading 0: 1%| | 12/995 [00:00<00:33, 29.58it/s] Loading 0: 2%|▏ | 15/995 [00:00<00:35, 27.50it/s] Loading 0: 2%|▏ | 18/995 [00:00<00:36, 26.45it/s] Loading 0: 2%|▏ | 21/995 [00:00<00:37, 25.88it/s] Loading 0: 2%|▏ | 24/995 [00:00<00:38, 25.42it/s] Loading 0: 3%|▎ | 32/995 [00:01<00:27, 35.12it/s] Loading 0: 4%|▎ | 36/995 [00:01<00:28, 33.23it/s] Loading 0: 4%|▍ | 40/995 [00:01<00:29, 32.32it/s] Loading 0: 4%|▍ | 44/995 [00:01<00:34, 27.53it/s] Loading 0: 5%|▌ | 51/995 [00:01<00:26, 35.88it/s] Loading 0: 6%|▌ | 55/995 [00:02<00:42, 22.15it/s] Loading 0: 6%|▌ | 58/995 [00:02<00:40, 22.97it/s] Loading 0: 6%|▌ | 61/995 [00:02<00:39, 23.63it/s] Loading 0: 7%|▋ | 66/995 [00:02<00:32, 28.15it/s] Loading 0: 7%|▋ | 70/995 [00:02<00:32, 28.22it/s] Loading 0: 7%|▋ | 74/995 [00:02<00:31, 28.85it/s] Loading 0: 8%|▊ | 78/995 [00:02<00:35, 25.51it/s] Loading 0: 8%|▊ | 81/995 [00:02<00:36, 24.89it/s] Loading 0: 8%|▊ | 84/995 [00:03<00:36, 25.11it/s] Loading 0: 9%|▊ | 87/995 [00:03<00:36, 24.97it/s] Loading 0: 10%|▉ | 95/995 [00:03<00:23, 37.99it/s] Loading 0: 10%|█ | 100/995 [00:03<00:28, 31.67it/s] Loading 0: 11%|█ | 107/995 [00:03<00:31, 28.04it/s] Loading 0: 11%|█ | 111/995 [00:04<00:34, 25.63it/s] Loading 0: 11%|█▏ | 114/995 [00:04<00:34, 25.55it/s] Loading 0: 12%|█▏ | 117/995 [00:04<00:34, 25.64it/s] Loading 0: 12%|█▏ | 120/995 [00:04<00:34, 25.73it/s] Loading 0: 12%|█▏ | 123/995 [00:04<00:33, 25.74it/s] Loading 0: 13%|█▎ | 128/995 [00:04<00:28, 30.30it/s] Loading 0: 13%|█▎ | 132/995 [00:04<00:29, 29.49it/s] Loading 0: 14%|█▎ | 136/995 [00:04<00:28, 29.80it/s] Loading 0: 14%|█▍ | 140/995 [00:05<00:32, 26.06it/s] Loading 0: 14%|█▍ | 143/995 [00:05<00:32, 26.08it/s] Loading 0: 15%|█▍ | 146/995 [00:05<00:32, 25.83it/s] Loading 0: 15%|█▍ | 149/995 [00:05<00:32, 25.85it/s] Loading 0: 16%|█▌ | 161/995 [00:05<00:18, 44.14it/s] Loading 0: 17%|█▋ | 166/995 [00:05<00:28, 29.32it/s] Loading 0: 17%|█▋ | 170/995 [00:06<00:31, 26.44it/s] Loading 0: 17%|█▋ | 174/995 [00:06<00:30, 26.76it/s] Loading 0: 18%|█▊ | 177/995 [00:06<00:30, 26.67it/s] Loading 0: 18%|█▊ | 180/995 [00:06<00:31, 26.16it/s] Loading 0: 18%|█▊ | 183/995 [00:06<00:31, 25.83it/s] Loading 0: 19%|█▉ | 188/995 [00:06<00:26, 30.21it/s] Loading 0: 19%|█▉ | 192/995 [00:06<00:27, 29.58it/s] Loading 0: 20%|█▉ | 196/995 [00:07<00:30, 26.16it/s] Loading 0: 20%|██ | 199/995 [00:07<00:30, 26.09it/s] Loading 0: 20%|██ | 202/995 [00:07<00:30, 25.94it/s] Loading 0: 21%|██ | 209/995 [00:07<00:22, 35.39it/s] Loading 0: 21%|██▏ | 213/995 [00:07<00:36, 21.41it/s] Loading 0: 22%|██▏ | 216/995 [00:07<00:34, 22.33it/s] Loading 0: 22%|██▏ | 221/995 [00:08<00:28, 26.78it/s] Loading 0: 23%|██▎ | 225/995 [00:08<00:28, 27.16it/s] Loading 0: 23%|██▎ | 229/995 [00:08<00:27, 27.79it/s] Loading 0: 23%|██▎ | 233/995 [00:08<00:30, 25.09it/s] Loading 0: 24%|██▎ | 236/995 [00:08<00:30, 25.17it/s] Loading 0: 24%|██▍ | 239/995 [00:08<00:30, 24.96it/s] Loading 0: 24%|██▍ | 242/995 [00:08<00:29, 25.41it/s] Loading 0: 25%|██▌ | 251/995 [00:08<00:18, 39.93it/s] Loading 0: 26%|██▌ | 256/995 [00:09<00:23, 32.01it/s] Loading 0: 26%|██▌ | 260/995 [00:09<00:22, 32.62it/s] Loading 0: 27%|██▋ | 265/995 [00:09<00:27, 26.30it/s] Loading 0: 27%|██▋ | 269/995 [00:09<00:25, 28.82it/s] Loading 0: 28%|██▊ | 274/995 [00:09<00:24, 29.07it/s] Loading 0: 28%|██▊ | 278/995 [00:09<00:24, 28.69it/s] Loading 0: 28%|██▊ | 282/995 [00:10<00:24, 28.96it/s] Loading 0: 29%|██▊ | 286/995 [00:25<13:07, 1.11s/it] Loading 0: 29%|██▉ | 291/995 [00:25<08:44, 1.34it/s] Loading 0: 30%|██▉ | 295/995 [00:25<06:23, 1.83it/s] Loading 0: 30%|███ | 299/995 [00:25<04:39, 2.49it/s] Loading 0: 30%|███ | 303/995 [00:26<03:27, 3.33it/s] Loading 0: 31%|███ | 306/995 [00:26<02:44, 4.19it/s] Loading 0: 31%|███ | 309/995 [00:26<02:09, 5.31it/s] Loading 0: 31%|███▏ | 312/995 [00:26<01:41, 6.71it/s] Loading 0: 32%|███▏ | 320/995 [00:26<01:01, 10.96it/s] Loading 0: 32%|███▏ | 323/995 [00:26<00:53, 12.47it/s] Loading 0: 33%|███▎ | 326/995 [00:26<00:47, 14.14it/s] Loading 0: 33%|███▎ | 329/995 [00:27<00:41, 15.92it/s] Loading 0: 33%|███▎ | 332/995 [00:27<00:37, 17.62it/s] Loading 0: 34%|███▎ | 335/995 [00:27<00:34, 19.37it/s] Loading 0: 34%|███▍ | 338/995 [00:27<00:31, 20.81it/s] Loading 0: 34%|███▍ | 343/995 [00:27<00:24, 26.24it/s] Loading 0: 35%|███▍ | 347/995 [00:27<00:23, 27.42it/s] Loading 0: 35%|███▌ | 351/995 [00:27<00:25, 24.82it/s] Loading 0: 36%|███▌ | 354/995 [00:27<00:25, 25.02it/s] Loading 0: 36%|███▌ | 357/995 [00:28<00:25, 25.29it/s] Loading 0: 36%|███▌ | 360/995 [00:28<00:24, 25.56it/s] Loading 0: 37%|███▋ | 367/995 [00:28<00:17, 35.18it/s] Loading 0: 37%|███▋ | 371/995 [00:28<00:29, 21.39it/s] Loading 0: 38%|███▊ | 376/995 [00:28<00:24, 25.59it/s] Loading 0: 38%|███▊ | 380/995 [00:28<00:23, 26.44it/s] Loading 0: 39%|███▊ | 384/995 [00:29<00:22, 27.38it/s] Loading 0: 39%|███▉ | 388/995 [00:29<00:24, 24.80it/s] Loading 0: 39%|███▉ | 391/995 [00:29<00:24, 25.13it/s] Loading 0: 40%|███▉ | 394/995 [00:29<00:23, 25.11it/s] Loading 0: 40%|███▉ | 397/995 [00:29<00:23, 25.09it/s] Loading 0: 41%|████ | 406/995 [00:29<00:14, 39.87it/s] Loading 0: 41%|████▏ | 411/995 [00:29<00:17, 32.75it/s] Loading 0: 42%|████▏ | 415/995 [00:30<00:18, 31.57it/s] Loading 0: 42%|████▏ | 419/995 [00:30<00:17, 33.13it/s] Loading 0: 43%|████▎ | 423/995 [00:30<00:23, 24.59it/s] Loading 0: 43%|████▎ | 426/995 [00:30<00:23, 24.58it/s] Loading 0: 43%|████▎ | 429/995 [00:30<00:22, 24.96it/s] Loading 0: 43%|████▎ | 432/995 [00:30<00:22, 25.25it/s] Loading 0: 44%|████▍ | 436/995 [00:30<00:19, 28.54it/s] Loading 0: 44%|████▍ | 440/995 [00:31<00:19, 28.42it/s] Loading 0: 45%|████▍ | 443/995 [00:31<00:20, 26.81it/s] Loading 0: 45%|████▍ | 446/995 [00:31<00:20, 26.45it/s] Loading 0: 45%|████▌ | 449/995 [00:31<00:21, 25.86it/s] Loading 0: 45%|████▌ | 452/995 [00:31<00:21, 25.38it/s] Loading 0: 46%|████▌ | 455/995 [00:31<00:21, 25.37it/s] Loading 0: 46%|████▌ | 458/995 [00:31<00:21, 25.52it/s] Loading 0: 47%|████▋ | 466/995 [00:31<00:15, 34.62it/s] Loading 0: 47%|████▋ | 470/995 [00:32<00:16, 32.69it/s] Loading 0: 48%|████▊ | 474/995 [00:32<00:15, 34.22it/s] Loading 0: 48%|████▊ | 478/995 [00:32<00:20, 24.84it/s] Loading 0: 48%|████▊ | 481/995 [00:32<00:20, 24.84it/s] Loading 0: 49%|████▊ | 484/995 [00:32<00:20, 25.13it/s] Loading 0: 49%|████▉ | 487/995 [00:32<00:20, 24.98it/s] Loading 0: 49%|████▉ | 490/995 [00:32<00:20, 25.19it/s] Loading 0: 50%|████▉ | 493/995 [00:33<00:19, 25.42it/s] Loading 0: 50%|█████ | 498/995 [00:33<00:16, 30.49it/s] Loading 0: 50%|█████ | 502/995 [00:33<00:16, 29.56it/s] Loading 0: 51%|█████ | 506/995 [00:33<00:18, 26.11it/s] Loading 0: 51%|█████ | 509/995 [00:33<00:18, 26.02it/s] Loading 0: 51%|█████▏ | 512/995 [00:33<00:19, 24.97it/s] Loading 0: 52%|█████▏ | 515/995 [00:33<00:19, 25.18it/s] Loading 0: 52%|█████▏ | 518/995 [00:33<00:18, 25.44it/s] Loading 0: 53%|█████▎ | 525/995 [00:34<00:13, 35.42it/s] Loading 0: 53%|█████▎ | 529/995 [00:34<00:18, 24.70it/s] Loading 0: 53%|█████▎ | 532/995 [00:34<00:18, 25.14it/s] Loading 0: 54%|█████▍ | 535/995 [00:34<00:18, 25.13it/s] Loading 0: 54%|█████▍ | 538/995 [00:34<00:18, 25.15it/s] Loading 0: 54%|█████▍ | 541/995 [00:34<00:17, 25.33it/s] Loading 0: 55%|█████▍ | 544/995 [00:34<00:17, 25.25it/s] Loading 0: 55%|█████▍ | 547/995 [00:35<00:17, 25.43it/s] Loading 0: 55%|█████▌ | 550/995 [00:35<00:17, 25.56it/s] Loading 0: 56%|█████▌ | 557/995 [00:35<00:11, 36.84it/s] Loading 0: 56%|█████▋ | 561/995 [00:35<00:11, 37.31it/s] Loading 0: 56%|█████▋ | 561/995 [00:50<00:11, 37.31it/s] Loading 0: 56%|█████▋ | 562/995 [00:50<10:09, 1.41s/it] Loading 0: 57%|█████▋ | 564/995 [00:50<08:05, 1.13s/it] Loading 0: 57%|█████▋ | 568/995 [00:50<05:12, 1.37it/s] Loading 0: 57%|█████▋ | 571/995 [00:50<03:46, 1.87it/s] Loading 0: 58%|█████▊ | 574/995 [00:50<02:44, 2.56it/s] Loading 0: 58%|█████▊ | 580/995 [00:51<01:31, 4.53it/s] Loading 0: 59%|█████▊ | 584/995 [00:51<01:13, 5.62it/s] Loading 0: 59%|█████▉ | 587/995 [00:51<00:59, 6.90it/s] Loading 0: 59%|█████▉ | 591/995 [00:51<00:43, 9.35it/s] Loading 0: 60%|█████▉ | 595/995 [00:51<00:33, 11.95it/s] Loading 0: 60%|██████ | 599/995 [00:51<00:29, 13.63it/s] Loading 0: 61%|██████ | 602/995 [00:52<00:25, 15.17it/s] Loading 0: 61%|██████ | 605/995 [00:52<00:22, 17.04it/s] Loading 0: 61%|██████ | 608/995 [00:52<00:20, 18.76it/s] Loading 0: 61%|██████▏ | 611/995 [00:52<00:18, 20.22it/s] Loading 0: 62%|██████▏ | 614/995 [00:52<00:17, 21.54it/s] Loading 0: 63%|██████▎ | 623/995 [00:52<00:10, 36.10it/s] Loading 0: 63%|██████▎ | 628/995 [00:52<00:11, 30.69it/s] Loading 0: 64%|██████▎ | 633/995 [00:52<00:10, 34.32it/s] Loading 0: 64%|██████▍ | 637/995 [00:53<00:14, 25.10it/s] Loading 0: 64%|██████▍ | 641/995 [00:53<00:15, 23.10it/s] Loading 0: 65%|██████▍ | 644/995 [00:53<00:14, 23.44it/s] Loading 0: 65%|██████▌ | 647/995 [00:53<00:14, 23.37it/s] Loading 0: 65%|██████▌ | 650/995 [00:53<00:14, 23.74it/s] Loading 0: 66%|██████▌ | 655/995 [00:53<00:12, 28.31it/s] Loading 0: 66%|██████▌ | 659/995 [00:54<00:11, 28.26it/s] Loading 0: 67%|██████▋ | 662/995 [00:54<00:12, 27.49it/s] Loading 0: 67%|██████▋ | 665/995 [00:54<00:12, 26.78it/s] Loading 0: 67%|██████▋ | 668/995 [00:54<00:12, 26.41it/s] Loading 0: 67%|██████▋ | 671/995 [00:54<00:12, 25.68it/s] Loading 0: 68%|██████▊ | 674/995 [00:54<00:12, 24.80it/s] Loading 0: 68%|██████▊ | 680/995 [00:54<00:09, 33.55it/s] Loading 0: 69%|██████▉ | 685/995 [00:54<00:08, 36.97it/s] Loading 0: 69%|██████▉ | 691/995 [00:55<00:11, 25.56it/s] Loading 0: 70%|██████▉ | 695/995 [00:55<00:11, 26.27it/s] Loading 0: 70%|███████ | 699/995 [00:55<00:10, 27.04it/s] Loading 0: 71%|███████ | 703/995 [00:55<00:11, 25.08it/s] Loading 0: 71%|███████ | 706/995 [00:55<00:11, 25.32it/s] Loading 0: 71%|███████▏ | 709/995 [00:55<00:11, 25.41it/s] Loading 0: 72%|███████▏ | 712/995 [00:56<00:11, 24.93it/s] Loading 0: 72%|███████▏ | 717/995 [00:56<00:09, 29.72it/s] Loading 0: 72%|███████▏ | 721/995 [00:56<00:09, 28.99it/s] Loading 0: 73%|███████▎ | 724/995 [00:56<00:09, 27.90it/s] Loading 0: 73%|███████▎ | 727/995 [00:56<00:09, 27.48it/s] Loading 0: 73%|███████▎ | 730/995 [00:56<00:09, 27.08it/s] Loading 0: 74%|███████▎ | 733/995 [00:56<00:09, 26.54it/s] Loading 0: 74%|███████▍ | 739/995 [00:57<00:10, 24.12it/s] Loading 0: 75%|███████▍ | 742/995 [00:57<00:10, 24.33it/s] Loading 0: 75%|███████▍ | 746/995 [00:57<00:08, 27.74it/s] Loading 0: 75%|███████▌ | 749/995 [00:57<00:09, 27.02it/s] Loading 0: 76%|███████▌ | 752/995 [00:57<00:09, 26.23it/s] Loading 0: 76%|███████▌ | 755/995 [00:57<00:09, 26.01it/s] Loading 0: 76%|███████▌ | 758/995 [00:57<00:09, 25.87it/s] Loading 0: 76%|███████▋ | 761/995 [00:57<00:09, 25.46it/s] Loading 0: 77%|███████▋ | 764/995 [00:58<00:09, 24.95it/s] Loading 0: 77%|███████▋ | 767/995 [00:58<00:09, 25.12it/s] Loading 0: 78%|███████▊ | 773/995 [00:58<00:06, 34.15it/s] Loading 0: 78%|███████▊ | 778/995 [00:58<00:05, 37.22it/s] Loading 0: 79%|███████▊ | 782/995 [00:58<00:07, 30.13it/s] Loading 0: 79%|███████▉ | 786/995 [00:58<00:07, 29.44it/s] Loading 0: 79%|███████▉ | 790/995 [00:58<00:06, 31.65it/s] Loading 0: 80%|███████▉ | 794/995 [00:59<00:08, 23.39it/s] Loading 0: 80%|████████ | 797/995 [00:59<00:08, 23.77it/s] Loading 0: 80%|████████ | 800/995 [00:59<00:08, 23.86it/s] Loading 0: 81%|████████ | 803/995 [00:59<00:07, 24.25it/s] Loading 0: 81%|████████ | 808/995 [00:59<00:06, 28.88it/s] Loading 0: 82%|████████▏ | 812/995 [00:59<00:06, 28.72it/s] Loading 0: 82%|████████▏ | 815/995 [00:59<00:06, 27.16it/s] Loading 0: 82%|████████▏ | 818/995 [00:59<00:06, 26.31it/s] Loading 0: 83%|████████▎ | 821/995 [01:00<00:06, 26.08it/s] Loading 0: 83%|████████▎ | 824/995 [01:00<00:06, 25.58it/s] Loading 0: 83%|████████▎ | 827/995 [01:00<00:06, 25.37it/s] Loading 0: 83%|████████▎ | 830/995 [01:00<00:06, 25.45it/s] Loading 0: 84%|████████▍ | 838/995 [01:00<00:04, 34.34it/s] Loading 0: 85%|████████▍ | 842/995 [01:00<00:04, 32.26it/s] Loading 0: 85%|████████▌ | 846/995 [01:15<02:38, 1.06s/it] Loading 0: 85%|████████▌ | 848/995 [01:15<02:12, 1.11it/s] Loading 0: 86%|████████▌ | 851/995 [01:16<01:39, 1.44it/s] Loading 0: 86%|████████▌ | 854/995 [01:16<01:12, 1.94it/s] Loading 0: 86%|████████▌ | 857/995 [01:16<00:52, 2.61it/s] Loading 0: 86%|████████▋ | 860/995 [01:16<00:38, 3.52it/s] Loading 0: 87%|████████▋ | 863/995 [01:16<00:28, 4.69it/s] Loading 0: 87%|████████▋ | 866/995 [01:16<00:20, 6.17it/s] Loading 0: 88%|████████▊ | 871/995 [01:16<00:13, 9.53it/s] Loading 0: 88%|████████▊ | 874/995 [01:16<00:10, 11.39it/s] Loading 0: 88%|████████▊ | 877/995 [01:17<00:08, 13.34it/s] Loading 0: 88%|████████▊ | 880/995 [01:17<00:07, 15.40it/s] Loading 0: 89%|████████▊ | 883/995 [01:17<00:06, 17.38it/s] Loading 0: 89%|████████▉ | 886/995 [01:17<00:05, 19.24it/s] Loading 0: 89%|████████▉ | 889/995 [01:17<00:05, 20.76it/s] Loading 0: 90%|████████▉ | 893/995 [01:17<00:04, 25.01it/s] Loading 0: 90%|█████████ | 897/995 [01:17<00:04, 20.21it/s] Loading 0: 91%|█████████ | 901/995 [01:18<00:03, 23.96it/s] Loading 0: 91%|█████████ | 904/995 [01:18<00:03, 24.39it/s] Loading 0: 91%|█████████ | 907/995 [01:18<00:03, 24.80it/s] Loading 0: 91%|█████████▏| 910/995 [01:18<00:03, 25.30it/s] Loading 0: 92%|█████████▏| 913/995 [01:18<00:03, 25.15it/s] Loading 0: 92%|█████████▏| 916/995 [01:18<00:03, 24.74it/s] Loading 0: 92%|█████████▏| 919/995 [01:18<00:03, 24.95it/s] Loading 0: 93%|█████████▎| 922/995 [01:18<00:02, 25.14it/s] Loading 0: 93%|█████████▎| 928/995 [01:18<00:01, 34.22it/s] Loading 0: 94%|█████████▍| 933/995 [01:19<00:01, 37.34it/s] Loading 0: 94%|█████████▍| 937/995 [01:19<00:01, 30.33it/s] Loading 0: 95%|█████████▍| 941/995 [01:19<00:01, 30.17it/s] Loading 0: 95%|█████████▍| 945/995 [01:19<00:01, 30.26it/s] Loading 0: 96%|█████████▌| 951/995 [01:19<00:01, 33.28it/s] Loading 0: 96%|█████████▌| 955/995 [01:21<00:05, 7.59it/s] Loading 0: 96%|█████████▋| 958/995 [01:21<00:04, 9.01it/s] Loading 0: 97%|█████████▋| 961/995 [01:21<00:03, 10.70it/s] Loading 0: 97%|█████████▋| 966/995 [01:21<00:01, 14.78it/s] Loading 0: 97%|█████████▋| 970/995 [01:21<00:01, 17.19it/s] Loading 0: 98%|█████████▊| 973/995 [01:21<00:01, 18.56it/s] Loading 0: 98%|█████████▊| 976/995 [01:22<00:00, 20.05it/s] Loading 0: 98%|█████████▊| 979/995 [01:22<00:00, 21.38it/s] Loading 0: 99%|█████████▊| 982/995 [01:22<00:00, 22.45it/s] Loading 0: 99%|█████████▉| 985/995 [01:22<00:00, 23.13it/s] Loading 0: 100%|█████████▉| 992/995 [01:22<00:00, 33.98it/s] /opt/conda/lib/python3.10/site-packages/transformers/models/auto/configuration_auto.py:957: FutureWarning: The `use_auth_token` argument is deprecated and will be removed in v5 of Transformers. Please use `token` instead.
mistralai-mixtral-8x7b-3473-v112-mkmlizer: warnings.warn(
mistralai-mixtral-8x7b-3473-v112-mkmlizer: /opt/conda/lib/python3.10/site-packages/transformers/models/auto/tokenization_auto.py:785: FutureWarning: The `use_auth_token` argument is deprecated and will be removed in v5 of Transformers. Please use `token` instead.
mistralai-mixtral-8x7b-3473-v112-mkmlizer: warnings.warn(
mistralai-mixtral-8x7b-3473-v112-mkmlizer: Downloading shards: 0%| | 0/4 [00:00<?, ?it/s] Downloading shards: 25%|██▌ | 1/4 [00:05<00:16, 5.52s/it] Downloading shards: 50%|█████ | 2/4 [00:11<00:11, 5.59s/it] Downloading shards: 75%|███████▌ | 3/4 [00:16<00:05, 5.55s/it] Downloading shards: 100%|██████████| 4/4 [00:19<00:00, 4.39s/it] Downloading shards: 100%|██████████| 4/4 [00:19<00:00, 4.82s/it]
mistralai-mixtral-8x7b-3473-v112-mkmlizer: Loading checkpoint shards: 0%| | 0/4 [00:00<?, ?it/s] Loading checkpoint shards: 50%|█████ | 2/4 [00:00<00:00, 17.17it/s] Loading checkpoint shards: 100%|██████████| 4/4 [00:00<00:00, 18.93it/s]
mistralai-mixtral-8x7b-3473-v112-mkmlizer: Saving model to /tmp/reward_cache/reward.tensors
mistralai-mixtral-8x7b-3473-v112-mkmlizer: Saving duration: 5.980s
mistralai-mixtral-8x7b-3473-v112-mkmlizer: Processed model ChaiML/llama_8b_pairwise_64m_256_tokens_step_249984 in 28.353s
mistralai-mixtral-8x7b-3473-v112-mkmlizer: creating bucket guanaco-reward-models
mistralai-mixtral-8x7b-3473-v112-mkmlizer: Bucket 's3://guanaco-reward-models/' created
mistralai-mixtral-8x7b-3473-v112-mkmlizer: uploading /tmp/reward_cache to s3://guanaco-reward-models/mistralai-mixtral-8x7b-3473-v112_reward
mistralai-mixtral-8x7b-3473-v112-mkmlizer: cp /tmp/reward_cache/config.json s3://guanaco-reward-models/mistralai-mixtral-8x7b-3473-v112_reward/config.json
mistralai-mixtral-8x7b-3473-v112-mkmlizer: cp /tmp/reward_cache/special_tokens_map.json s3://guanaco-reward-models/mistralai-mixtral-8x7b-3473-v112_reward/special_tokens_map.json
mistralai-mixtral-8x7b-3473-v112-mkmlizer: cp /tmp/reward_cache/tokenizer_config.json s3://guanaco-reward-models/mistralai-mixtral-8x7b-3473-v112_reward/tokenizer_config.json
mistralai-mixtral-8x7b-3473-v112-mkmlizer: cp /tmp/reward_cache/tokenizer.json s3://guanaco-reward-models/mistralai-mixtral-8x7b-3473-v112_reward/tokenizer.json
mistralai-mixtral-8x7b-3473-v112-mkmlizer: cp /tmp/reward_cache/reward.tensors s3://guanaco-reward-models/mistralai-mixtral-8x7b-3473-v112_reward/reward.tensors
Job mistralai-mixtral-8x7b-3473-v112-mkmlizer completed after 367.81s with status: succeeded
Stopping job with name mistralai-mixtral-8x7b-3473-v112-mkmlizer
Pipeline stage MKMLizer completed in 370.11s
Running pipeline stage MKMLKubeTemplater
Pipeline stage MKMLKubeTemplater completed in 0.47s
Running pipeline stage ISVCDeployer
Creating inference service mistralai-mixtral-8x7b-3473-v112
Waiting for inference service mistralai-mixtral-8x7b-3473-v112 to be ready
Inference service mistralai-mixtral-8x7b-3473-v112 ready after 162.69465589523315s
Pipeline stage ISVCDeployer completed in 165.17s
Running pipeline stage StressChecker
HTTP Request: %s %s "%s %d %s"
Received healthy response to inference request in 2.6089107990264893s
HTTP Request: %s %s "%s %d %s"
Received healthy response to inference request in 2.239795207977295s
HTTP Request: %s %s "%s %d %s"
Received healthy response to inference request in 1.835770845413208s
HTTP Request: %s %s "%s %d %s"
Received healthy response to inference request in 2.2035958766937256s
HTTP Request: %s %s "%s %d %s"
Received healthy response to inference request in 2.400935173034668s
5 requests
0 failed requests
5th percentile: 1.9093358516693115
10th percentile: 1.982900857925415
20th percentile: 2.130030870437622
30th percentile: 2.2108357429504393
40th percentile: 2.225315475463867
50th percentile: 2.239795207977295
60th percentile: 2.304251194000244
70th percentile: 2.3687071800231934
80th percentile: 2.442530298233032
90th percentile: 2.525720548629761
95th percentile: 2.567315673828125
99th percentile: 2.6005917739868165
mean time: 2.257801580429077
Pipeline stage StressChecker completed in 15.40s
mistralai-mixtral-8x7b_3473_v112 status is now deployed due to DeploymentManager action
mistralai-mixtral-8x7b_3473_v112 status is now inactive due to auto deactivation removed underperforming models
mistralai-mixtral-8x7b_3473_v112 status is now torndown due to DeploymentManager action

Usage Metrics

Latency Metrics