submission_id: mistralai-mixtral-8x7b-_3473_v99
developer_uid: chai_backend_admin
alignment_samples: 0
best_of: 4
celo_rating: 1169.25
display_name: mistralai-mixtral-8x7b-_3473_v99
formatter: {'memory_template': '<s>[INST] This is an entertaining conversation. You are {bot_name} who has the persona: {memory}.\nEngage in a chat with {user_name} while staying in character. Try to flirt with {user_name}. Engage in *roleplay* actions. Describe the scene dramatically. \n', 'prompt_template': '{prompt}\n', 'bot_template': '{bot_name}: {message}</s>', 'user_template': '[INST] {user_name}: {message} [/INST]', 'response_template': '{bot_name}:', 'truncate_by_message': False}
generation_params: {'temperature': 0.9, 'top_p': 1.0, 'min_p': 0.0, 'top_k': 50, 'presence_penalty': 0.5, 'frequency_penalty': 0.5, 'stopping_words': ['\n', '</s>', '<|user|>', '###'], 'max_input_tokens': 512, 'best_of': 4, 'max_output_tokens': 64, 'reward_max_token_input': 256}
is_internal_developer: True
language_model: mistralai/Mixtral-8x7B-Instruct-v0.1
max_input_tokens: 512
max_output_tokens: 64
model_architecture: MixtralForCausalLM
model_group: mistralai/Mixtral-8x7B-I
model_name: mistralai-mixtral-8x7b-_3473_v99
model_num_parameters: 46702792704.0
model_repo: mistralai/Mixtral-8x7B-Instruct-v0.1
model_size: 47B
num_battles: 13806
num_wins: 6136
propriety_score: 0.7378246753246753
propriety_total_count: 1232.0
ranking_group: single
reward_formatter: {'bot_template': 'Bot: {message}\n', 'memory_template': 'Memory: {memory}\n', 'prompt_template': '{prompt}\n', 'response_template': 'Bot:', 'truncate_by_message': False, 'user_template': 'User: {message}\n'}
reward_repo: ChaiML/gpt2_xl_pairwise_89m_step_347634
status: torndown
submission_type: basic
timestamp: 2024-07-30T15:36:04+00:00
us_pacific_date: 2024-07-30
win_ratio: 0.4444444444444444
Download Preference Data
Resubmit model
Running pipeline stage MKMLizer
Starting job with name mistralai-mixtral-8x7b-3473-v99-mkmlizer
Waiting for job on mistralai-mixtral-8x7b-3473-v99-mkmlizer to finish
mistralai-mixtral-8x7b-3473-v99-mkmlizer: ╔═════════════════════════════════════════════════════════════════════╗
mistralai-mixtral-8x7b-3473-v99-mkmlizer: ║ _____ __ __ ║
mistralai-mixtral-8x7b-3473-v99-mkmlizer: ║ / _/ /_ ___ __/ / ___ ___ / / ║
mistralai-mixtral-8x7b-3473-v99-mkmlizer: ║ / _/ / // / |/|/ / _ \/ -_) -_) / ║
mistralai-mixtral-8x7b-3473-v99-mkmlizer: ║ /_//_/\_, /|__,__/_//_/\__/\__/_/ ║
mistralai-mixtral-8x7b-3473-v99-mkmlizer: ║ /___/ ║
mistralai-mixtral-8x7b-3473-v99-mkmlizer: ║ ║
mistralai-mixtral-8x7b-3473-v99-mkmlizer: ║ Version: 0.9.7 ║
mistralai-mixtral-8x7b-3473-v99-mkmlizer: ║ Copyright 2023 MK ONE TECHNOLOGIES Inc. ║
mistralai-mixtral-8x7b-3473-v99-mkmlizer: ║ https://mk1.ai ║
mistralai-mixtral-8x7b-3473-v99-mkmlizer: ║ ║
mistralai-mixtral-8x7b-3473-v99-mkmlizer: ║ The license key for the current software has been verified as ║
mistralai-mixtral-8x7b-3473-v99-mkmlizer: ║ belonging to: ║
mistralai-mixtral-8x7b-3473-v99-mkmlizer: ║ ║
mistralai-mixtral-8x7b-3473-v99-mkmlizer: ║ Chai Research Corp. ║
mistralai-mixtral-8x7b-3473-v99-mkmlizer: ║ Account ID: 7997a29f-0ceb-4cc7-9adf-840c57b4ae6f ║
mistralai-mixtral-8x7b-3473-v99-mkmlizer: ║ Expiration: 2024-10-15 23:59:59 ║
mistralai-mixtral-8x7b-3473-v99-mkmlizer: ║ ║
mistralai-mixtral-8x7b-3473-v99-mkmlizer: ╚═════════════════════════════════════════════════════════════════════╝
mistralai-mixtral-8x7b-3473-v99-mkmlizer: Downloaded to shared memory in 194.267s
mistralai-mixtral-8x7b-3473-v99-mkmlizer: quantizing model to /dev/shm/model_cache, profile:s0, folder:/tmp/tmpvb0wl14s, device:0
mistralai-mixtral-8x7b-3473-v99-mkmlizer: Saving flywheel model at /dev/shm/model_cache
mistralai-mixtral-8x7b-3473-v99-mkmlizer: quantized model in 93.341s
mistralai-mixtral-8x7b-3473-v99-mkmlizer: Processed model mistralai/Mixtral-8x7B-Instruct-v0.1 in 287.608s
mistralai-mixtral-8x7b-3473-v99-mkmlizer: creating bucket guanaco-mkml-models
mistralai-mixtral-8x7b-3473-v99-mkmlizer: Bucket 's3://guanaco-mkml-models/' created
mistralai-mixtral-8x7b-3473-v99-mkmlizer: uploading /dev/shm/model_cache to s3://guanaco-mkml-models/mistralai-mixtral-8x7b-3473-v99
mistralai-mixtral-8x7b-3473-v99-mkmlizer: cp /dev/shm/model_cache/config.json s3://guanaco-mkml-models/mistralai-mixtral-8x7b-3473-v99/config.json
mistralai-mixtral-8x7b-3473-v99-mkmlizer: cp /dev/shm/model_cache/special_tokens_map.json s3://guanaco-mkml-models/mistralai-mixtral-8x7b-3473-v99/special_tokens_map.json
mistralai-mixtral-8x7b-3473-v99-mkmlizer: cp /dev/shm/model_cache/tokenizer_config.json s3://guanaco-mkml-models/mistralai-mixtral-8x7b-3473-v99/tokenizer_config.json
mistralai-mixtral-8x7b-3473-v99-mkmlizer: cp /dev/shm/model_cache/tokenizer.model s3://guanaco-mkml-models/mistralai-mixtral-8x7b-3473-v99/tokenizer.model
mistralai-mixtral-8x7b-3473-v99-mkmlizer: cp /dev/shm/model_cache/tokenizer.json s3://guanaco-mkml-models/mistralai-mixtral-8x7b-3473-v99/tokenizer.json
mistralai-mixtral-8x7b-3473-v99-mkmlizer: cp /dev/shm/model_cache/flywheel_model.3.safetensors s3://guanaco-mkml-models/mistralai-mixtral-8x7b-3473-v99/flywheel_model.3.safetensors
mistralai-mixtral-8x7b-3473-v99-mkmlizer: cp /dev/shm/model_cache/flywheel_model.2.safetensors s3://guanaco-mkml-models/mistralai-mixtral-8x7b-3473-v99/flywheel_model.2.safetensors
mistralai-mixtral-8x7b-3473-v99-mkmlizer: cp /dev/shm/model_cache/flywheel_model.0.safetensors s3://guanaco-mkml-models/mistralai-mixtral-8x7b-3473-v99/flywheel_model.0.safetensors
mistralai-mixtral-8x7b-3473-v99-mkmlizer: cp /dev/shm/model_cache/flywheel_model.1.safetensors s3://guanaco-mkml-models/mistralai-mixtral-8x7b-3473-v99/flywheel_model.1.safetensors
mistralai-mixtral-8x7b-3473-v99-mkmlizer: loading reward model from ChaiML/gpt2_xl_pairwise_89m_step_347634
mistralai-mixtral-8x7b-3473-v99-mkmlizer: Loading 0: 0%| | 0/995 [00:00<?, ?it/s] Loading 0: 0%| | 4/995 [00:00<00:31, 31.22it/s] Loading 0: 1%| | 8/995 [00:00<00:34, 28.38it/s] Loading 0: 1%| | 11/995 [00:00<00:36, 26.98it/s] Loading 0: 1%|▏ | 14/995 [00:00<00:37, 26.07it/s] Loading 0: 2%|▏ | 17/995 [00:00<00:38, 25.60it/s] Loading 0: 2%|▏ | 20/995 [00:00<00:38, 25.42it/s] Loading 0: 2%|▏ | 23/995 [00:00<00:37, 25.61it/s] Loading 0: 3%|▎ | 26/995 [00:00<00:36, 26.66it/s] Loading 0: 3%|▎ | 33/995 [00:01<00:24, 38.82it/s] Loading 0: 4%|▍ | 38/995 [00:01<00:31, 30.56it/s] Loading 0: 4%|▍ | 42/995 [00:01<00:31, 30.06it/s] Loading 0: 5%|▍ | 46/995 [00:01<00:31, 29.97it/s] Loading 0: 5%|▌ | 52/995 [00:01<00:38, 24.38it/s] Loading 0: 6%|▌ | 55/995 [00:02<00:38, 24.42it/s] Loading 0: 6%|▌ | 58/995 [00:02<00:37, 24.72it/s] Loading 0: 6%|▌ | 61/995 [00:02<00:37, 24.64it/s] Loading 0: 7%|▋ | 66/995 [00:02<00:32, 28.65it/s] Loading 0: 7%|▋ | 69/995 [00:02<00:33, 27.54it/s] Loading 0: 7%|▋ | 72/995 [00:02<00:34, 27.04it/s] Loading 0: 8%|▊ | 75/995 [00:02<00:34, 26.64it/s] Loading 0: 8%|▊ | 78/995 [00:02<00:34, 26.41it/s] Loading 0: 8%|▊ | 81/995 [00:02<00:35, 25.95it/s] Loading 0: 8%|▊ | 84/995 [00:03<00:36, 25.10it/s] Loading 0: 9%|▊ | 87/995 [00:03<00:37, 24.18it/s] Loading 0: 10%|▉ | 95/995 [00:03<00:24, 37.29it/s] Loading 0: 10%|▉ | 99/995 [00:03<00:27, 32.38it/s] Loading 0: 10%|█ | 103/995 [00:03<00:27, 32.06it/s] Loading 0: 11%|█ | 107/995 [00:03<00:36, 24.15it/s] Loading 0: 11%|█ | 110/995 [00:04<00:36, 24.16it/s] Loading 0: 11%|█▏ | 113/995 [00:04<00:36, 24.35it/s] Loading 0: 12%|█▏ | 116/995 [00:04<00:35, 24.72it/s] Loading 0: 12%|█▏ | 119/995 [00:04<00:34, 25.04it/s] Loading 0: 12%|█▏ | 122/995 [00:04<00:34, 25.29it/s] Loading 0: 13%|█▎ | 126/995 [00:04<00:30, 28.75it/s] Loading 0: 13%|█▎ | 129/995 [00:04<00:31, 27.89it/s] Loading 0: 13%|█▎ | 132/995 [00:04<00:32, 26.96it/s] Loading 0: 14%|█▎ | 135/995 [00:04<00:33, 25.96it/s] Loading 0: 14%|█▍ | 138/995 [00:05<00:34, 25.16it/s] Loading 0: 14%|█▍ | 141/995 [00:05<00:34, 24.83it/s] Loading 0: 14%|█▍ | 144/995 [00:05<00:34, 24.52it/s] Loading 0: 15%|█▍ | 147/995 [00:05<00:34, 24.80it/s] Loading 0: 15%|█▌ | 153/995 [00:05<00:24, 33.87it/s] Loading 0: 16%|█▌ | 161/995 [00:05<00:19, 42.15it/s] Loading 0: 17%|█▋ | 166/995 [00:06<00:30, 26.93it/s] Loading 0: 17%|█▋ | 170/995 [00:06<00:33, 24.74it/s] Loading 0: 17%|█▋ | 173/995 [00:06<00:32, 25.04it/s] Loading 0: 18%|█▊ | 176/995 [00:06<00:33, 24.80it/s] Loading 0: 18%|█▊ | 179/995 [00:06<00:32, 25.06it/s] Loading 0: 18%|█▊ | 182/995 [00:06<00:32, 25.25it/s] Loading 0: 19%|█▊ | 185/995 [00:06<00:32, 25.31it/s] Loading 0: 19%|█▉ | 190/995 [00:06<00:27, 29.69it/s] Loading 0: 19%|█▉ | 194/995 [00:07<00:27, 29.11it/s] Loading 0: 20%|█▉ | 197/995 [00:07<00:28, 27.61it/s] Loading 0: 20%|██ | 200/995 [00:07<00:30, 26.20it/s] Loading 0: 20%|██ | 203/995 [00:07<00:30, 25.91it/s] Loading 0: 21%|██ | 210/995 [00:07<00:34, 22.96it/s] Loading 0: 21%|██▏ | 213/995 [00:07<00:33, 23.14it/s] Loading 0: 22%|██▏ | 216/995 [00:08<00:32, 23.79it/s] Loading 0: 22%|██▏ | 221/995 [00:08<00:27, 27.96it/s] Loading 0: 23%|██▎ | 224/995 [00:08<00:29, 26.49it/s] Loading 0: 23%|██▎ | 227/995 [00:08<00:30, 25.52it/s] Loading 0: 23%|██▎ | 230/995 [00:08<00:31, 24.51it/s] Loading 0: 23%|██▎ | 233/995 [00:08<00:30, 24.84it/s] Loading 0: 24%|██▎ | 236/995 [00:08<00:30, 25.03it/s] Loading 0: 24%|██▍ | 239/995 [00:08<00:29, 25.23it/s] Loading 0: 24%|██▍ | 242/995 [00:09<00:29, 25.32it/s] Loading 0: 25%|██▌ | 251/995 [00:09<00:18, 41.02it/s] Loading 0: 26%|██▌ | 256/995 [00:09<00:22, 32.82it/s] Loading 0: 26%|██▌ | 260/995 [00:09<00:22, 33.28it/s] Loading 0: 27%|██▋ | 265/995 [00:09<00:27, 26.17it/s] Loading 0: 27%|██▋ | 269/995 [00:09<00:25, 28.36it/s] Loading 0: 28%|██▊ | 274/995 [00:10<00:24, 29.29it/s] Loading 0: 28%|██▊ | 278/995 [00:10<00:25, 28.11it/s] Loading 0: 28%|██▊ | 282/995 [00:10<00:25, 28.15it/s] Loading 0: 29%|██▊ | 285/995 [00:25<13:50, 1.17s/it] Loading 0: 29%|██▊ | 286/995 [00:25<12:37, 1.07s/it] Loading 0: 29%|██▉ | 291/995 [00:25<07:31, 1.56it/s] Loading 0: 30%|██▉ | 294/995 [00:25<05:39, 2.06it/s] Loading 0: 30%|██▉ | 297/995 [00:25<04:14, 2.74it/s] Loading 0: 30%|███ | 300/995 [00:26<03:10, 3.65it/s] Loading 0: 30%|███ | 303/995 [00:26<02:23, 4.84it/s] Loading 0: 31%|███ | 306/995 [00:26<01:48, 6.33it/s] Loading 0: 31%|███ | 309/995 [00:26<01:24, 8.12it/s] Loading 0: 31%|███▏ | 312/995 [00:26<01:07, 10.13it/s] Loading 0: 32%|███▏ | 320/995 [00:26<00:43, 15.59it/s] Loading 0: 32%|███▏ | 323/995 [00:26<00:40, 16.49it/s] Loading 0: 33%|███▎ | 326/995 [00:27<00:37, 17.83it/s] Loading 0: 33%|███▎ | 329/995 [00:27<00:34, 19.06it/s] Loading 0: 33%|███▎ | 332/995 [00:27<00:32, 20.13it/s] Loading 0: 34%|███▎ | 335/995 [00:27<00:31, 21.12it/s] Loading 0: 34%|███▍ | 338/995 [00:27<00:30, 21.81it/s] Loading 0: 34%|███▍ | 341/995 [00:27<00:27, 23.45it/s] Loading 0: 35%|███▍ | 345/995 [00:27<00:24, 26.02it/s] Loading 0: 35%|███▍ | 348/995 [00:27<00:25, 25.46it/s] Loading 0: 35%|███▌ | 351/995 [00:27<00:25, 24.77it/s] Loading 0: 36%|███▌ | 354/995 [00:28<00:26, 24.36it/s] Loading 0: 36%|███▌ | 357/995 [00:28<00:26, 24.15it/s] Loading 0: 36%|███▌ | 360/995 [00:28<00:26, 24.31it/s] Loading 0: 37%|███▋ | 367/995 [00:28<00:18, 34.20it/s] Loading 0: 37%|███▋ | 371/995 [00:28<00:30, 20.37it/s] Loading 0: 38%|███▊ | 376/995 [00:29<00:25, 24.19it/s] Loading 0: 38%|███▊ | 380/995 [00:29<00:24, 24.78it/s] Loading 0: 38%|███▊ | 383/995 [00:29<00:25, 24.37it/s] Loading 0: 39%|███▉ | 386/995 [00:29<00:25, 23.42it/s] Loading 0: 39%|███▉ | 389/995 [00:29<00:25, 23.69it/s] Loading 0: 39%|███▉ | 392/995 [00:29<00:25, 24.08it/s] Loading 0: 40%|███▉ | 395/995 [00:29<00:25, 23.91it/s] Loading 0: 40%|████ | 398/995 [00:29<00:23, 25.16it/s] Loading 0: 41%|████ | 406/995 [00:30<00:15, 38.16it/s] Loading 0: 41%|████ | 410/995 [00:30<00:18, 30.87it/s] Loading 0: 42%|████▏ | 414/995 [00:30<00:19, 29.53it/s] Loading 0: 42%|████▏ | 418/995 [00:30<00:19, 30.19it/s] Loading 0: 43%|████▎ | 423/995 [00:30<00:22, 24.89it/s] Loading 0: 43%|████▎ | 426/995 [00:30<00:22, 24.79it/s] Loading 0: 43%|████▎ | 429/995 [00:31<00:22, 24.82it/s] Loading 0: 43%|████▎ | 432/995 [00:31<00:23, 24.42it/s] Loading 0: 44%|████▍ | 436/995 [00:31<00:20, 27.69it/s] Loading 0: 44%|████▍ | 439/995 [00:31<00:20, 26.79it/s] Loading 0: 44%|████▍ | 442/995 [00:31<00:21, 26.14it/s] Loading 0: 45%|████▍ | 445/995 [00:31<00:21, 25.34it/s] Loading 0: 45%|████▌ | 448/995 [00:31<00:22, 24.84it/s] Loading 0: 45%|████▌ | 451/995 [00:31<00:22, 24.14it/s] Loading 0: 46%|████▌ | 454/995 [00:32<00:23, 23.44it/s] Loading 0: 46%|████▌ | 457/995 [00:32<00:22, 24.18it/s] Loading 0: 47%|████▋ | 464/995 [00:32<00:15, 35.33it/s] Loading 0: 47%|████▋ | 469/995 [00:32<00:16, 32.86it/s] Loading 0: 48%|████▊ | 473/995 [00:32<00:16, 32.40it/s] Loading 0: 48%|████▊ | 478/995 [00:32<00:19, 26.10it/s] Loading 0: 48%|████▊ | 481/995 [00:32<00:20, 25.43it/s] Loading 0: 49%|████▊ | 484/995 [00:33<00:20, 25.21it/s] Loading 0: 49%|████▉ | 487/995 [00:33<00:20, 24.79it/s] Loading 0: 49%|████▉ | 490/995 [00:33<00:20, 24.40it/s] Loading 0: 50%|████▉ | 493/995 [00:33<00:20, 23.97it/s] Loading 0: 50%|████▉ | 496/995 [00:33<00:20, 24.36it/s] Loading 0: 50%|█████ | 500/995 [00:33<00:18, 26.61it/s] Loading 0: 51%|█████ | 503/995 [00:33<00:19, 25.88it/s] Loading 0: 51%|█████ | 506/995 [00:33<00:19, 25.38it/s] Loading 0: 51%|█████ | 509/995 [00:34<00:19, 25.21it/s] Loading 0: 51%|█████▏ | 512/995 [00:34<00:19, 25.21it/s] Loading 0: 52%|█████▏ | 515/995 [00:34<00:19, 24.42it/s] Loading 0: 52%|█████▏ | 518/995 [00:34<00:19, 24.37it/s] Loading 0: 53%|█████▎ | 525/995 [00:34<00:13, 33.91it/s] Loading 0: 53%|█████▎ | 529/995 [00:34<00:19, 23.68it/s] Loading 0: 53%|█████▎ | 532/995 [00:34<00:19, 23.68it/s] Loading 0: 54%|█████▍ | 535/995 [00:35<00:19, 23.67it/s] Loading 0: 54%|█████▍ | 538/995 [00:35<00:19, 23.51it/s] Loading 0: 54%|█████▍ | 541/995 [00:35<00:19, 23.61it/s] Loading 0: 55%|█████▍ | 544/995 [00:35<00:18, 23.77it/s] Loading 0: 55%|█████▍ | 547/995 [00:35<00:18, 23.59it/s] Loading 0: 55%|█████▌ | 550/995 [00:35<00:19, 23.24it/s] Loading 0: 56%|█████▌ | 553/995 [00:35<00:17, 24.80it/s] Loading 0: 56%|█████▋ | 561/995 [00:35<00:11, 37.44it/s] Loading 0: 56%|█████▋ | 561/995 [00:50<00:11, 37.44it/s] Loading 0: 56%|█████▋ | 562/995 [00:50<09:31, 1.32s/it] Loading 0: 57%|█████▋ | 564/995 [00:51<07:42, 1.07s/it] Loading 0: 57%|█████▋ | 568/995 [00:51<05:03, 1.41it/s] Loading 0: 57%|█████▋ | 571/995 [00:51<03:42, 1.91it/s] Loading 0: 58%|█████▊ | 574/995 [00:51<02:43, 2.58it/s] Loading 0: 58%|█████▊ | 581/995 [00:51<01:30, 4.58it/s] Loading 0: 59%|█████▊ | 584/995 [00:52<01:13, 5.61it/s] Loading 0: 59%|█████▉ | 587/995 [00:52<00:58, 6.93it/s] Loading 0: 59%|█████▉ | 591/995 [00:52<00:42, 9.41it/s] Loading 0: 60%|█████▉ | 594/995 [00:52<00:35, 11.24it/s] Loading 0: 60%|██████ | 597/995 [00:52<00:30, 13.18it/s] Loading 0: 60%|██████ | 600/995 [00:52<00:26, 14.96it/s] Loading 0: 61%|██████ | 603/995 [00:52<00:23, 17.00it/s] Loading 0: 61%|██████ | 606/995 [00:52<00:20, 18.68it/s] Loading 0: 61%|██████ | 609/995 [00:53<00:19, 20.25it/s] Loading 0: 62%|██████▏ | 612/995 [00:53<00:17, 21.82it/s] Loading 0: 62%|██████▏ | 615/995 [00:53<00:16, 23.43it/s] Loading 0: 63%|██████▎ | 623/995 [00:53<00:10, 36.41it/s] Loading 0: 63%|██████▎ | 628/995 [00:53<00:11, 31.00it/s] Loading 0: 64%|██████▎ | 633/995 [00:53<00:10, 35.04it/s] Loading 0: 64%|██████▍ | 637/995 [00:53<00:14, 25.40it/s] Loading 0: 64%|██████▍ | 641/995 [00:54<00:14, 23.76it/s] Loading 0: 65%|██████▍ | 644/995 [00:54<00:15, 23.38it/s] Loading 0: 65%|██████▌ | 647/995 [00:54<00:14, 23.88it/s] Loading 0: 65%|██████▌ | 650/995 [00:54<00:14, 24.31it/s] Loading 0: 66%|██████▌ | 655/995 [00:54<00:11, 29.21it/s] Loading 0: 66%|██████▌ | 659/995 [00:54<00:11, 28.87it/s] Loading 0: 67%|██████▋ | 663/995 [00:54<00:11, 28.47it/s] Loading 0: 67%|██████▋ | 666/995 [00:55<00:12, 27.10it/s] Loading 0: 67%|██████▋ | 669/995 [00:55<00:12, 25.96it/s] Loading 0: 68%|██████▊ | 672/995 [00:55<00:12, 25.97it/s] Loading 0: 68%|██████▊ | 675/995 [00:55<00:12, 25.58it/s] Loading 0: 69%|██████▊ | 683/995 [00:55<00:09, 33.95it/s] Loading 0: 69%|██████▉ | 690/995 [00:55<00:07, 39.77it/s] Loading 0: 70%|██████▉ | 694/995 [00:56<00:13, 22.30it/s] Loading 0: 70%|███████ | 698/995 [00:56<00:12, 23.54it/s] Loading 0: 70%|███████ | 701/995 [00:56<00:12, 24.04it/s] Loading 0: 71%|███████ | 704/995 [00:56<00:12, 24.23it/s] Loading 0: 71%|███████ | 707/995 [00:56<00:11, 24.43it/s] Loading 0: 71%|███████▏ | 710/995 [00:56<00:11, 24.75it/s] Loading 0: 72%|███████▏ | 715/995 [00:56<00:09, 29.56it/s] Loading 0: 72%|███████▏ | 719/995 [00:57<00:09, 28.44it/s] Loading 0: 73%|███████▎ | 722/995 [00:57<00:09, 27.87it/s] Loading 0: 73%|███████▎ | 725/995 [00:57<00:09, 27.14it/s] Loading 0: 73%|███████▎ | 728/995 [00:57<00:10, 26.48it/s] Loading 0: 73%|███████▎ | 731/995 [00:57<00:10, 25.47it/s] Loading 0: 74%|███████▍ | 734/995 [00:57<00:09, 26.29it/s] Loading 0: 74%|███████▍ | 739/995 [00:57<00:11, 22.62it/s] Loading 0: 75%|███████▍ | 742/995 [00:57<00:10, 23.10it/s] Loading 0: 75%|███████▍ | 746/995 [00:58<00:09, 26.78it/s] Loading 0: 75%|███████▌ | 749/995 [00:58<00:09, 26.05it/s] Loading 0: 76%|███████▌ | 752/995 [00:58<00:09, 24.77it/s] Loading 0: 76%|███████▌ | 755/995 [00:58<00:09, 24.76it/s] Loading 0: 76%|███████▌ | 758/995 [00:58<00:09, 24.52it/s] Loading 0: 76%|███████▋ | 761/995 [00:58<00:09, 23.94it/s] Loading 0: 77%|███████▋ | 764/995 [00:58<00:09, 23.69it/s] Loading 0: 77%|███████▋ | 767/995 [00:58<00:09, 23.44it/s] Loading 0: 78%|███████▊ | 773/995 [00:59<00:06, 32.18it/s] Loading 0: 78%|███████▊ | 778/995 [00:59<00:06, 36.16it/s] Loading 0: 79%|███████▊ | 782/995 [00:59<00:07, 29.18it/s] Loading 0: 79%|███████▉ | 786/995 [00:59<00:07, 28.44it/s] Loading 0: 79%|███████▉ | 791/995 [00:59<00:06, 32.92it/s] Loading 0: 80%|███████▉ | 795/995 [00:59<00:08, 24.64it/s] Loading 0: 80%|████████ | 798/995 [01:00<00:08, 24.47it/s] Loading 0: 81%|████████ | 801/995 [01:00<00:08, 23.93it/s] Loading 0: 81%|████████ | 804/995 [01:00<00:07, 24.11it/s] Loading 0: 81%|████████ | 808/995 [01:00<00:06, 27.27it/s] Loading 0: 82%|████████▏ | 811/995 [01:00<00:06, 26.87it/s] Loading 0: 82%|████████▏ | 814/995 [01:00<00:06, 26.06it/s] Loading 0: 82%|████████▏ | 817/995 [01:00<00:06, 26.22it/s] Loading 0: 82%|████████▏ | 820/995 [01:00<00:06, 25.68it/s] Loading 0: 83%|████████▎ | 823/995 [01:01<00:06, 25.87it/s] Loading 0: 83%|████████▎ | 826/995 [01:01<00:06, 25.26it/s] Loading 0: 83%|████████▎ | 829/995 [01:01<00:06, 24.32it/s] Loading 0: 84%|████████▍ | 835/995 [01:01<00:04, 33.16it/s] Loading 0: 84%|████████▍ | 840/995 [01:01<00:04, 37.33it/s] Loading 0: 85%|████████▍ | 844/995 [01:16<02:43, 1.09s/it] Loading 0: 85%|████████▌ | 848/995 [01:16<01:55, 1.27it/s] Loading 0: 86%|████████▌ | 852/995 [01:16<01:23, 1.71it/s] Loading 0: 86%|████████▌ | 855/995 [01:17<01:03, 2.19it/s] Loading 0: 86%|████████▌ | 858/995 [01:17<00:48, 2.85it/s] Loading 0: 87%|████████▋ | 861/995 [01:17<00:35, 3.74it/s] Loading 0: 87%|████████▋ | 864/995 [01:17<00:26, 4.89it/s] Loading 0: 87%|████████▋ | 867/995 [01:17<00:20, 6.35it/s] Loading 0: 88%|████████▊ | 872/995 [01:17<00:12, 9.66it/s] Loading 0: 88%|████████▊ | 876/995 [01:17<00:09, 12.28it/s] Loading 0: 88%|████████▊ | 880/995 [01:17<00:07, 15.04it/s] Loading 0: 89%|████████▉ | 884/995 [01:18<00:06, 16.60it/s] Loading 0: 89%|████████▉ | 887/995 [01:18<00:06, 17.95it/s] Loading 0: 89%|████████▉ | 890/995 [01:18<00:05, 19.24it/s] Loading 0: 90%|█████████ | 896/995 [01:18<00:03, 26.34it/s] Loading 0: 90%|█████████ | 900/995 [01:18<00:04, 20.75it/s] Loading 0: 91%|█████████ | 903/995 [01:18<00:04, 20.99it/s] Loading 0: 91%|█████████ | 906/995 [01:19<00:04, 22.01it/s] Loading 0: 91%|█████████▏| 909/995 [01:19<00:03, 22.80it/s] Loading 0: 92%|█████████▏| 912/995 [01:19<00:03, 23.66it/s] Loading 0: 92%|█████████▏| 915/995 [01:19<00:03, 24.28it/s] Loading 0: 92%|█████████▏| 918/995 [01:19<00:03, 24.92it/s] Loading 0: 93%|█████████▎| 921/995 [01:19<00:02, 25.10it/s] Loading 0: 93%|█████████▎| 924/995 [01:19<00:02, 24.90it/s] Loading 0: 94%|█████████▍| 933/995 [01:19<00:01, 40.21it/s] Loading 0: 94%|█████████▍| 938/995 [01:20<00:01, 31.64it/s] Loading 0: 95%|█████████▍| 942/995 [01:20<00:01, 31.19it/s] Loading 0: 95%|█████████▌| 946/995 [01:20<00:01, 26.81it/s] Loading 0: 96%|█████████▌| 952/995 [01:21<00:05, 8.51it/s] Loading 0: 96%|█████████▌| 955/995 [01:22<00:04, 9.84it/s] Loading 0: 96%|█████████▋| 958/995 [01:22<00:03, 11.29it/s] Loading 0: 97%|█████████▋| 961/995 [01:22<00:02, 12.87it/s] Loading 0: 97%|█████████▋| 966/995 [01:22<00:01, 17.33it/s] Loading 0: 97%|█████████▋| 969/995 [01:22<00:01, 18.84it/s] Loading 0: 98%|█████████▊| 972/995 [01:22<00:01, 20.25it/s] Loading 0: 98%|█████████▊| 975/995 [01:22<00:00, 21.59it/s] Loading 0: 98%|█████████▊| 978/995 [01:22<00:00, 22.68it/s] Loading 0: 99%|█████████▊| 981/995 [01:23<00:00, 23.44it/s] Loading 0: 99%|█████████▉| 984/995 [01:23<00:00, 23.72it/s] Loading 0: 99%|█████████▉| 987/995 [01:23<00:00, 24.60it/s] /opt/conda/lib/python3.10/site-packages/transformers/models/auto/configuration_auto.py:957: FutureWarning: The `use_auth_token` argument is deprecated and will be removed in v5 of Transformers. Please use `token` instead.
mistralai-mixtral-8x7b-3473-v99-mkmlizer: warnings.warn(
mistralai-mixtral-8x7b-3473-v99-mkmlizer: /opt/conda/lib/python3.10/site-packages/transformers/models/auto/tokenization_auto.py:785: FutureWarning: The `use_auth_token` argument is deprecated and will be removed in v5 of Transformers. Please use `token` instead.
mistralai-mixtral-8x7b-3473-v99-mkmlizer: warnings.warn(
mistralai-mixtral-8x7b-3473-v99-mkmlizer: Downloading shards: 0%| | 0/2 [00:00<?, ?it/s] Downloading shards: 50%|█████ | 1/2 [00:05<00:05, 5.43s/it] Downloading shards: 100%|██████████| 2/2 [00:08<00:00, 4.19s/it] Downloading shards: 100%|██████████| 2/2 [00:08<00:00, 4.37s/it]
mistralai-mixtral-8x7b-3473-v99-mkmlizer: Loading checkpoint shards: 0%| | 0/2 [00:00<?, ?it/s] Loading checkpoint shards: 50%|█████ | 1/2 [00:00<00:00, 2.34it/s] Loading checkpoint shards: 100%|██████████| 2/2 [00:00<00:00, 3.85it/s] Loading checkpoint shards: 100%|██████████| 2/2 [00:00<00:00, 3.51it/s]
mistralai-mixtral-8x7b-3473-v99-mkmlizer: Saving model to /tmp/reward_cache/reward.tensors
mistralai-mixtral-8x7b-3473-v99-mkmlizer: creating bucket guanaco-reward-models
mistralai-mixtral-8x7b-3473-v99-mkmlizer: Bucket 's3://guanaco-reward-models/' created
mistralai-mixtral-8x7b-3473-v99-mkmlizer: uploading /tmp/reward_cache to s3://guanaco-reward-models/mistralai-mixtral-8x7b-3473-v99_reward
mistralai-mixtral-8x7b-3473-v99-mkmlizer: cp /tmp/reward_cache/config.json s3://guanaco-reward-models/mistralai-mixtral-8x7b-3473-v99_reward/config.json
mistralai-mixtral-8x7b-3473-v99-mkmlizer: cp /tmp/reward_cache/special_tokens_map.json s3://guanaco-reward-models/mistralai-mixtral-8x7b-3473-v99_reward/special_tokens_map.json
mistralai-mixtral-8x7b-3473-v99-mkmlizer: cp /tmp/reward_cache/tokenizer_config.json s3://guanaco-reward-models/mistralai-mixtral-8x7b-3473-v99_reward/tokenizer_config.json
mistralai-mixtral-8x7b-3473-v99-mkmlizer: cp /tmp/reward_cache/merges.txt s3://guanaco-reward-models/mistralai-mixtral-8x7b-3473-v99_reward/merges.txt
mistralai-mixtral-8x7b-3473-v99-mkmlizer: cp /tmp/reward_cache/vocab.json s3://guanaco-reward-models/mistralai-mixtral-8x7b-3473-v99_reward/vocab.json
mistralai-mixtral-8x7b-3473-v99-mkmlizer: cp /tmp/reward_cache/tokenizer.json s3://guanaco-reward-models/mistralai-mixtral-8x7b-3473-v99_reward/tokenizer.json
mistralai-mixtral-8x7b-3473-v99-mkmlizer: cp /tmp/reward_cache/reward.tensors s3://guanaco-reward-models/mistralai-mixtral-8x7b-3473-v99_reward/reward.tensors
Job mistralai-mixtral-8x7b-3473-v99-mkmlizer completed after 359.51s with status: succeeded
Stopping job with name mistralai-mixtral-8x7b-3473-v99-mkmlizer
Pipeline stage MKMLizer completed in 361.30s
Running pipeline stage MKMLKubeTemplater
Pipeline stage MKMLKubeTemplater completed in 0.44s
Running pipeline stage ISVCDeployer
Creating inference service mistralai-mixtral-8x7b-3473-v99
Waiting for inference service mistralai-mixtral-8x7b-3473-v99 to be ready
Inference service mistralai-mixtral-8x7b-3473-v99 ready after 121.99111866950989s
Pipeline stage ISVCDeployer completed in 124.16s
Running pipeline stage StressChecker
HTTP Request: %s %s "%s %d %s"
Received healthy response to inference request in 3.580219030380249s
HTTP Request: %s %s "%s %d %s"
Received healthy response to inference request in 2.1819589138031006s
HTTP Request: %s %s "%s %d %s"
Received healthy response to inference request in 2.589265823364258s
HTTP Request: %s %s "%s %d %s"
Received healthy response to inference request in 1.8569509983062744s
HTTP Request: %s %s "%s %d %s"
Received healthy response to inference request in 1.6616909503936768s
5 requests
0 failed requests
5th percentile: 1.7007429599761963
10th percentile: 1.7397949695587158
20th percentile: 1.8178989887237549
30th percentile: 1.9219525814056397
40th percentile: 2.05195574760437
50th percentile: 2.1819589138031006
60th percentile: 2.3448816776275634
70th percentile: 2.507804441452026
80th percentile: 2.7874564647674562
90th percentile: 3.1838377475738526
95th percentile: 3.3820283889770506
99th percentile: 3.5405809020996095
mean time: 2.3740171432495116
Pipeline stage StressChecker completed in 16.03s
mistralai-mixtral-8x7b-_3473_v99 status is now deployed due to DeploymentManager action
mistralai-mixtral-8x7b-_3473_v99 status is now inactive due to auto deactivation removed underperforming models
admin requested tearing down of mistralai-mixtral-8x7b-_3473_v99
Running pipeline stage ISVCDeleter
Checking if service mistralai-mixtral-8x7b-3473-v99 is running
Tearing down inference service mistralai-mixtral-8x7b-3473-v99
Service mistralai-mixtral-8x7b-3473-v99 has been torndown
Pipeline stage ISVCDeleter completed in 4.70s
Running pipeline stage MKMLModelDeleter
Cleaning model data from S3
Cleaning model data from model cache
Deleting key mistralai-mixtral-8x7b-3473-v99/config.json from bucket guanaco-mkml-models
Deleting key mistralai-mixtral-8x7b-3473-v99/flywheel_model.0.safetensors from bucket guanaco-mkml-models
Deleting key mistralai-mixtral-8x7b-3473-v99/flywheel_model.1.safetensors from bucket guanaco-mkml-models
Deleting key mistralai-mixtral-8x7b-3473-v99/flywheel_model.2.safetensors from bucket guanaco-mkml-models
Deleting key mistralai-mixtral-8x7b-3473-v99/flywheel_model.3.safetensors from bucket guanaco-mkml-models
Deleting key mistralai-mixtral-8x7b-3473-v99/special_tokens_map.json from bucket guanaco-mkml-models
Deleting key mistralai-mixtral-8x7b-3473-v99/tokenizer.json from bucket guanaco-mkml-models
Deleting key mistralai-mixtral-8x7b-3473-v99/tokenizer.model from bucket guanaco-mkml-models
Deleting key mistralai-mixtral-8x7b-3473-v99/tokenizer_config.json from bucket guanaco-mkml-models
Cleaning model data from model cache
Deleting key mistralai-mixtral-8x7b-3473-v99_reward/config.json from bucket guanaco-reward-models
Deleting key mistralai-mixtral-8x7b-3473-v99_reward/merges.txt from bucket guanaco-reward-models
Deleting key mistralai-mixtral-8x7b-3473-v99_reward/reward.tensors from bucket guanaco-reward-models
Deleting key mistralai-mixtral-8x7b-3473-v99_reward/special_tokens_map.json from bucket guanaco-reward-models
Deleting key mistralai-mixtral-8x7b-3473-v99_reward/tokenizer.json from bucket guanaco-reward-models
Deleting key mistralai-mixtral-8x7b-3473-v99_reward/tokenizer_config.json from bucket guanaco-reward-models
Deleting key mistralai-mixtral-8x7b-3473-v99_reward/vocab.json from bucket guanaco-reward-models
Pipeline stage MKMLModelDeleter completed in 10.00s
mistralai-mixtral-8x7b-_3473_v99 status is now torndown due to DeploymentManager action

Usage Metrics

Latency Metrics