mistralai-mixtral-8x7b_3473

developer_uid: robert_irvine
submission_id: mistralai-mixtral-8x7b_3473_v115
model_name: mistralai-mixtral-8x7b_3473_v115
model_group: mistralai/Mixtral-8x7B-I
status: torndown
timestamp: 2024-08-05T19:19:07+00:00
num_battles: 35379
num_wins: 18084
celo_rating: 1158.62
family_friendly_score: 0.0
submission_type: basic
model_repo: mistralai/Mixtral-8x7B-Instruct-v0.1
model_architecture: MixtralForCausalLM
reward_repo: rirv938/reward_gpt2_preference_24m_e2
model_num_parameters: 46702792704.0
best_of: 4
max_input_tokens: 512
max_output_tokens: 64
display_name: mistralai-mixtral-8x7b_3473_v115
is_internal_developer: True
language_model: mistralai/Mixtral-8x7B-Instruct-v0.1
model_size: 47B
ranking_group: single
us_pacific_date: 2024-08-05
win_ratio: 0.511150682608327
generation_params: {'temperature': 0.9, 'top_p': 1.0, 'min_p': 0.0, 'top_k': 50, 'presence_penalty': 0.5, 'frequency_penalty': 0.5, 'stopping_words': ['\n', '</s>', '<|user|>', '###'], 'max_input_tokens': 512, 'best_of': 4, 'max_output_tokens': 64, 'reward_max_token_input': 256}
formatter: {'memory_template': '<s>[INST] This is an entertaining conversation. You are {bot_name} who has the persona: {memory}.\nEngage in a chat with {user_name} while staying in character. Try to flirt with {user_name}. Engage in *roleplay* actions. Describe the scene dramatically. \n', 'prompt_template': '{prompt}\n', 'bot_template': '{bot_name}: {message}</s>', 'user_template': '[INST] {user_name}: {message} [/INST]', 'response_template': '{bot_name}:', 'truncate_by_message': False}
reward_formatter: {'bot_template': 'Bot: {message}\n', 'memory_template': '', 'prompt_template': '', 'response_template': 'Bot:', 'truncate_by_message': False, 'user_template': 'User: {message}\n'}
Resubmit model
Running pipeline stage MKMLizer
Starting job with name mistralai-mixtral-8x7b-3473-v115-mkmlizer
Waiting for job on mistralai-mixtral-8x7b-3473-v115-mkmlizer to finish
Stopping job with name mistralai-mixtral-8x7b-3473-v115-mkmlizer
%s, retrying in %s seconds...
Starting job with name mistralai-mixtral-8x7b-3473-v115-mkmlizer
Waiting for job on mistralai-mixtral-8x7b-3473-v115-mkmlizer to finish
mistralai-mixtral-8x7b-3473-v115-mkmlizer: ╔═════════════════════════════════════════════════════════════════════╗
mistralai-mixtral-8x7b-3473-v115-mkmlizer: ║     _____            __           __                                ║
mistralai-mixtral-8x7b-3473-v115-mkmlizer: ║    / _/ /_ ___    __/ /  ___ ___ / /                                ║
mistralai-mixtral-8x7b-3473-v115-mkmlizer: ║   / _/ / // / |/|/ / _ \/ -_) -_) /                                 ║
mistralai-mixtral-8x7b-3473-v115-mkmlizer: ║  /_//_/\_, /|__,__/_//_/\__/\__/_/                                  ║
mistralai-mixtral-8x7b-3473-v115-mkmlizer: ║       /___/                                                         ║
mistralai-mixtral-8x7b-3473-v115-mkmlizer: ║                                                                     ║
mistralai-mixtral-8x7b-3473-v115-mkmlizer: ║  Version: 0.9.9                                                     ║
mistralai-mixtral-8x7b-3473-v115-mkmlizer: ║  Copyright 2023 MK ONE TECHNOLOGIES Inc.                            ║
mistralai-mixtral-8x7b-3473-v115-mkmlizer: ║  https://mk1.ai                                                     ║
mistralai-mixtral-8x7b-3473-v115-mkmlizer: ║                                                                     ║
mistralai-mixtral-8x7b-3473-v115-mkmlizer: ║  The license key for the current software has been verified as      ║
mistralai-mixtral-8x7b-3473-v115-mkmlizer: ║  belonging to:                                                      ║
mistralai-mixtral-8x7b-3473-v115-mkmlizer: ║                                                                     ║
mistralai-mixtral-8x7b-3473-v115-mkmlizer: ║  Chai Research Corp.                                                ║
mistralai-mixtral-8x7b-3473-v115-mkmlizer: ║  Account ID: 7997a29f-0ceb-4cc7-9adf-840c57b4ae6f                   ║
mistralai-mixtral-8x7b-3473-v115-mkmlizer: ║  Expiration: 2024-10-15 23:59:59                                    ║
mistralai-mixtral-8x7b-3473-v115-mkmlizer: ║                                                                     ║
mistralai-mixtral-8x7b-3473-v115-mkmlizer: ╚═════════════════════════════════════════════════════════════════════╝
mistralai-mixtral-8x7b-3473-v115-mkmlizer: Downloaded to shared memory in 184.362s
mistralai-mixtral-8x7b-3473-v115-mkmlizer: quantizing model to /dev/shm/model_cache, profile:s0, folder:/tmp/tmpmkg7i4jp, device:0
mistralai-mixtral-8x7b-3473-v115-mkmlizer: Saving flywheel model at /dev/shm/model_cache
mistralai-mixtral-8x7b-3473-v115-mkmlizer: quantized model in 93.772s
mistralai-mixtral-8x7b-3473-v115-mkmlizer: Processed model mistralai/Mixtral-8x7B-Instruct-v0.1 in 278.135s
mistralai-mixtral-8x7b-3473-v115-mkmlizer: creating bucket guanaco-mkml-models
mistralai-mixtral-8x7b-3473-v115-mkmlizer: Bucket 's3://guanaco-mkml-models/' created
mistralai-mixtral-8x7b-3473-v115-mkmlizer: uploading /dev/shm/model_cache to s3://guanaco-mkml-models/mistralai-mixtral-8x7b-3473-v115
mistralai-mixtral-8x7b-3473-v115-mkmlizer: cp /dev/shm/model_cache/special_tokens_map.json s3://guanaco-mkml-models/mistralai-mixtral-8x7b-3473-v115/special_tokens_map.json
mistralai-mixtral-8x7b-3473-v115-mkmlizer: cp /dev/shm/model_cache/tokenizer_config.json s3://guanaco-mkml-models/mistralai-mixtral-8x7b-3473-v115/tokenizer_config.json
mistralai-mixtral-8x7b-3473-v115-mkmlizer: cp /dev/shm/model_cache/tokenizer.json s3://guanaco-mkml-models/mistralai-mixtral-8x7b-3473-v115/tokenizer.json
mistralai-mixtral-8x7b-3473-v115-mkmlizer: cp /dev/shm/model_cache/tokenizer.model s3://guanaco-mkml-models/mistralai-mixtral-8x7b-3473-v115/tokenizer.model
mistralai-mixtral-8x7b-3473-v115-mkmlizer: cp /dev/shm/model_cache/flywheel_model.3.safetensors s3://guanaco-mkml-models/mistralai-mixtral-8x7b-3473-v115/flywheel_model.3.safetensors
mistralai-mixtral-8x7b-3473-v115-mkmlizer: cp /dev/shm/model_cache/flywheel_model.0.safetensors s3://guanaco-mkml-models/mistralai-mixtral-8x7b-3473-v115/flywheel_model.0.safetensors
mistralai-mixtral-8x7b-3473-v115-mkmlizer: cp /dev/shm/model_cache/flywheel_model.1.safetensors s3://guanaco-mkml-models/mistralai-mixtral-8x7b-3473-v115/flywheel_model.1.safetensors
mistralai-mixtral-8x7b-3473-v115-mkmlizer: cp /dev/shm/model_cache/flywheel_model.2.safetensors s3://guanaco-mkml-models/mistralai-mixtral-8x7b-3473-v115/flywheel_model.2.safetensors
mistralai-mixtral-8x7b-3473-v115-mkmlizer: loading reward model from rirv938/reward_gpt2_preference_24m_e2
mistralai-mixtral-8x7b-3473-v115-mkmlizer: 
Loading 0:   0%|          | 0/995 [00:00<?, ?it/s]
Loading 0:   0%|          | 4/995 [00:00<00:28, 34.37it/s]
Loading 0:   1%|          | 8/995 [00:00<00:32, 30.35it/s]
Loading 0:   1%|          | 12/995 [00:00<00:32, 30.36it/s]
Loading 0:   2%|▏         | 16/995 [00:00<00:38, 25.53it/s]
Loading 0:   2%|▏         | 19/995 [00:00<00:39, 24.97it/s]
Loading 0:   2%|▏         | 22/995 [00:00<00:39, 24.90it/s]
Loading 0:   3%|▎         | 25/995 [00:00<00:38, 24.99it/s]
Loading 0:   3%|▎         | 34/995 [00:01<00:23, 40.63it/s]
Loading 0:   4%|▍         | 39/995 [00:01<00:29, 32.50it/s]
Loading 0:   4%|▍         | 43/995 [00:01<00:29, 31.79it/s]
Loading 0:   5%|▍         | 47/995 [00:01<00:29, 32.13it/s]
Loading 0:   5%|▌         | 52/995 [00:01<00:39, 23.71it/s]
Loading 0:   6%|▌         | 55/995 [00:01<00:39, 23.83it/s]
Loading 0:   6%|▌         | 58/995 [00:02<00:38, 24.29it/s]
Loading 0:   6%|▌         | 61/995 [00:02<00:38, 24.46it/s]
Loading 0:   7%|▋         | 66/995 [00:02<00:33, 28.05it/s]
Loading 0:   7%|▋         | 69/995 [00:02<00:34, 27.11it/s]
Loading 0:   7%|▋         | 72/995 [00:02<00:34, 26.46it/s]
Loading 0:   8%|▊         | 75/995 [00:02<00:35, 25.94it/s]
Loading 0:   8%|▊         | 78/995 [00:02<00:36, 25.43it/s]
Loading 0:   8%|▊         | 81/995 [00:02<00:37, 24.64it/s]
Loading 0:   8%|▊         | 84/995 [00:03<00:36, 25.08it/s]
Loading 0:   9%|▊         | 87/995 [00:03<00:35, 25.41it/s]
Loading 0:  10%|▉         | 96/995 [00:03<00:22, 40.59it/s]
Loading 0:  10%|█         | 101/995 [00:03<00:26, 33.20it/s]
Loading 0:  11%|█         | 107/995 [00:03<00:30, 28.84it/s]
Loading 0:  11%|█         | 111/995 [00:03<00:33, 26.34it/s]
Loading 0:  11%|█▏        | 114/995 [00:04<00:33, 26.36it/s]
Loading 0:  12%|█▏        | 117/995 [00:04<00:33, 26.17it/s]
Loading 0:  12%|█▏        | 120/995 [00:04<00:34, 25.70it/s]
Loading 0:  12%|█▏        | 123/995 [00:04<00:34, 25.36it/s]
Loading 0:  13%|█▎        | 128/995 [00:04<00:29, 29.71it/s]
Loading 0:  13%|█▎        | 132/995 [00:04<00:29, 29.24it/s]
Loading 0:  14%|█▎        | 135/995 [00:04<00:30, 28.36it/s]
Loading 0:  14%|█▍        | 138/995 [00:04<00:30, 27.65it/s]
Loading 0:  14%|█▍        | 141/995 [00:05<00:31, 27.00it/s]
Loading 0:  14%|█▍        | 144/995 [00:05<00:32, 26.55it/s]
Loading 0:  15%|█▍        | 147/995 [00:05<00:32, 26.45it/s]
Loading 0:  15%|█▌        | 154/995 [00:05<00:22, 37.47it/s]
Loading 0:  16%|█▌        | 161/995 [00:05<00:20, 41.70it/s]
Loading 0:  17%|█▋        | 166/995 [00:05<00:30, 27.13it/s]
Loading 0:  17%|█▋        | 170/995 [00:06<00:33, 24.94it/s]
Loading 0:  17%|█▋        | 173/995 [00:06<00:32, 25.12it/s]
Loading 0:  18%|█▊        | 176/995 [00:06<00:32, 25.15it/s]
Loading 0:  18%|█▊        | 179/995 [00:06<00:32, 25.01it/s]
Loading 0:  18%|█▊        | 182/995 [00:06<00:32, 24.94it/s]
Loading 0:  19%|█▊        | 185/995 [00:06<00:32, 25.20it/s]
Loading 0:  19%|█▉        | 190/995 [00:06<00:27, 29.71it/s]
Loading 0:  19%|█▉        | 194/995 [00:06<00:27, 29.17it/s]
Loading 0:  20%|█▉        | 197/995 [00:07<00:28, 28.08it/s]
Loading 0:  20%|██        | 200/995 [00:07<00:29, 26.69it/s]
Loading 0:  20%|██        | 203/995 [00:07<00:30, 26.40it/s]
Loading 0:  21%|██        | 210/995 [00:07<00:33, 23.27it/s]
Loading 0:  21%|██▏       | 213/995 [00:07<00:33, 23.67it/s]
Loading 0:  22%|██▏       | 216/995 [00:07<00:32, 24.19it/s]
Loading 0:  22%|██▏       | 221/995 [00:08<00:26, 28.94it/s]
Loading 0:  23%|██▎       | 225/995 [00:08<00:26, 28.59it/s]
Loading 0:  23%|██▎       | 228/995 [00:08<00:27, 27.90it/s]
Loading 0:  23%|██▎       | 231/995 [00:08<00:27, 27.33it/s]
Loading 0:  24%|██▎       | 234/995 [00:08<00:28, 26.91it/s]
Loading 0:  24%|██▍       | 237/995 [00:08<00:29, 25.93it/s]
Loading 0:  24%|██▍       | 240/995 [00:08<00:29, 25.49it/s]
Loading 0:  24%|██▍       | 243/995 [00:08<00:29, 25.88it/s]
Loading 0:  25%|██▌       | 250/995 [00:08<00:20, 36.59it/s]
Loading 0:  26%|██▌       | 254/995 [00:09<00:21, 34.34it/s]
Loading 0:  26%|██▌       | 258/995 [00:09<00:25, 28.94it/s]
Loading 0:  27%|██▋       | 265/995 [00:09<00:27, 26.60it/s]
Loading 0:  27%|██▋       | 269/995 [00:09<00:25, 28.82it/s]
Loading 0:  28%|██▊       | 274/995 [00:09<00:25, 28.77it/s]
Loading 0:  28%|██▊       | 278/995 [00:10<00:25, 28.62it/s]
Loading 0:  28%|██▊       | 281/995 [00:10<00:25, 27.74it/s]
Loading 0:  28%|██▊       | 283/995 [00:25<00:25, 27.74it/s]
Loading 0:  29%|██▊       | 284/995 [00:25<14:52,  1.26s/it]
Loading 0:  29%|██▊       | 286/995 [00:25<12:17,  1.04s/it]
Loading 0:  29%|██▉       | 291/995 [00:25<07:28,  1.57it/s]
Loading 0:  30%|██▉       | 294/995 [00:25<05:39,  2.06it/s]
Loading 0:  30%|██▉       | 297/995 [00:26<04:15,  2.73it/s]
Loading 0:  30%|███       | 300/995 [00:26<03:11,  3.63it/s]
Loading 0:  30%|███       | 303/995 [00:26<02:24,  4.78it/s]
Loading 0:  31%|███       | 306/995 [00:26<01:50,  6.23it/s]
Loading 0:  31%|███       | 309/995 [00:26<01:25,  8.00it/s]
Loading 0:  31%|███▏      | 312/995 [00:26<01:07, 10.05it/s]
Loading 0:  32%|███▏      | 320/995 [00:26<00:43, 15.45it/s]
Loading 0:  32%|███▏      | 323/995 [00:27<00:40, 16.71it/s]
Loading 0:  33%|███▎      | 326/995 [00:27<00:36, 18.13it/s]
Loading 0:  33%|███▎      | 329/995 [00:27<00:34, 19.58it/s]
Loading 0:  33%|███▎      | 332/995 [00:27<00:31, 20.98it/s]
Loading 0:  34%|███▎      | 335/995 [00:27<00:29, 22.16it/s]
Loading 0:  34%|███▍      | 338/995 [00:27<00:28, 22.96it/s]
Loading 0:  34%|███▍      | 343/995 [00:27<00:23, 28.23it/s]
Loading 0:  35%|███▍      | 347/995 [00:27<00:22, 28.84it/s]
Loading 0:  35%|███▌      | 351/995 [00:28<00:25, 25.51it/s]
Loading 0:  36%|███▌      | 354/995 [00:28<00:25, 25.36it/s]
Loading 0:  36%|███▌      | 357/995 [00:28<00:25, 25.31it/s]
Loading 0:  36%|███▌      | 360/995 [00:28<00:25, 24.97it/s]
Loading 0:  37%|███▋      | 367/995 [00:28<00:18, 34.03it/s]
Loading 0:  37%|███▋      | 371/995 [00:28<00:29, 21.45it/s]
Loading 0:  38%|███▊      | 376/995 [00:29<00:24, 25.50it/s]
Loading 0:  38%|███▊      | 380/995 [00:29<00:23, 26.13it/s]
Loading 0:  39%|███▊      | 384/995 [00:29<00:23, 26.22it/s]
Loading 0:  39%|███▉      | 387/995 [00:29<00:23, 25.59it/s]
Loading 0:  39%|███▉      | 390/995 [00:29<00:23, 25.25it/s]
Loading 0:  39%|███▉      | 393/995 [00:29<00:23, 25.21it/s]
Loading 0:  40%|███▉      | 396/995 [00:29<00:23, 25.35it/s]
Loading 0:  41%|████      | 404/995 [00:30<00:17, 34.54it/s]
Loading 0:  41%|████      | 408/995 [00:30<00:18, 32.48it/s]
Loading 0:  41%|████▏     | 412/995 [00:30<00:18, 31.55it/s]
Loading 0:  42%|████▏     | 416/995 [00:30<00:21, 27.37it/s]
Loading 0:  43%|████▎     | 423/995 [00:30<00:21, 26.37it/s]
Loading 0:  43%|████▎     | 426/995 [00:30<00:21, 26.09it/s]
Loading 0:  43%|████▎     | 429/995 [00:31<00:21, 26.10it/s]
Loading 0:  43%|████▎     | 432/995 [00:31<00:21, 26.25it/s]
Loading 0:  44%|████▍     | 437/995 [00:31<00:18, 30.82it/s]
Loading 0:  44%|████▍     | 441/995 [00:31<00:20, 26.80it/s]
Loading 0:  45%|████▍     | 444/995 [00:31<00:20, 26.64it/s]
Loading 0:  45%|████▍     | 447/995 [00:31<00:20, 26.55it/s]
Loading 0:  45%|████▌     | 450/995 [00:31<00:20, 26.24it/s]
Loading 0:  46%|████▌     | 453/995 [00:31<00:20, 25.81it/s]
Loading 0:  46%|████▌     | 456/995 [00:32<00:20, 25.91it/s]
Loading 0:  46%|████▌     | 459/995 [00:32<00:20, 25.99it/s]
Loading 0:  47%|████▋     | 467/995 [00:32<00:13, 40.08it/s]
Loading 0:  47%|████▋     | 472/995 [00:32<00:16, 32.42it/s]
Loading 0:  48%|████▊     | 478/995 [00:32<00:18, 28.16it/s]
Loading 0:  48%|████▊     | 482/995 [00:32<00:18, 28.49it/s]
Loading 0:  49%|████▉     | 486/995 [00:33<00:20, 24.86it/s]
Loading 0:  49%|████▉     | 489/995 [00:33<00:20, 24.97it/s]
Loading 0:  49%|████▉     | 492/995 [00:33<00:19, 25.19it/s]
Loading 0:  50%|████▉     | 495/995 [00:33<00:19, 25.08it/s]
Loading 0:  50%|█████     | 500/995 [00:33<00:16, 29.31it/s]
Loading 0:  51%|█████     | 504/995 [00:33<00:16, 28.99it/s]
Loading 0:  51%|█████     | 507/995 [00:33<00:17, 27.70it/s]
Loading 0:  51%|█████▏    | 510/995 [00:33<00:18, 26.93it/s]
Loading 0:  52%|█████▏    | 513/995 [00:34<00:18, 26.18it/s]
Loading 0:  52%|█████▏    | 516/995 [00:34<00:18, 26.04it/s]
Loading 0:  52%|█████▏    | 519/995 [00:34<00:18, 25.89it/s]
Loading 0:  53%|█████▎    | 526/995 [00:34<00:20, 22.69it/s]
Loading 0:  53%|█████▎    | 531/995 [00:34<00:17, 26.83it/s]
Loading 0:  54%|█████▎    | 534/995 [00:34<00:17, 26.62it/s]
Loading 0:  54%|█████▍    | 537/995 [00:34<00:17, 26.18it/s]
Loading 0:  54%|█████▍    | 540/995 [00:35<00:17, 25.76it/s]
Loading 0:  55%|█████▍    | 543/995 [00:35<00:17, 25.71it/s]
Loading 0:  55%|█████▍    | 546/995 [00:35<00:17, 25.64it/s]
Loading 0:  55%|█████▌    | 549/995 [00:35<00:17, 25.72it/s]
Loading 0:  55%|█████▌    | 552/995 [00:35<00:17, 25.63it/s]
Loading 0:  56%|█████▋    | 560/995 [00:35<00:10, 39.73it/s]
Loading 0:  56%|█████▋    | 561/995 [00:50<00:10, 39.73it/s]
Loading 0:  56%|█████▋    | 562/995 [00:50<08:50,  1.23s/it]
Loading 0:  57%|█████▋    | 564/995 [00:51<07:16,  1.01s/it]
Loading 0:  57%|█████▋    | 568/995 [00:51<04:52,  1.46it/s]
Loading 0:  57%|█████▋    | 571/995 [00:51<03:37,  1.95it/s]
Loading 0:  58%|█████▊    | 574/995 [00:51<02:40,  2.62it/s]
Loading 0:  58%|█████▊    | 581/995 [00:51<01:29,  4.61it/s]
Loading 0:  59%|█████▊    | 584/995 [00:52<01:12,  5.66it/s]
Loading 0:  59%|█████▉    | 587/995 [00:52<00:58,  7.01it/s]
Loading 0:  59%|█████▉    | 591/995 [00:52<00:42,  9.47it/s]
Loading 0:  60%|█████▉    | 594/995 [00:52<00:35, 11.21it/s]
Loading 0:  60%|██████    | 597/995 [00:52<00:30, 13.20it/s]
Loading 0:  60%|██████    | 600/995 [00:52<00:25, 15.20it/s]
Loading 0:  61%|██████    | 603/995 [00:52<00:23, 16.58it/s]
Loading 0:  61%|██████    | 606/995 [00:52<00:21, 18.25it/s]
Loading 0:  61%|██████    | 609/995 [00:52<00:19, 19.95it/s]
Loading 0:  62%|██████▏   | 612/995 [00:53<00:17, 21.36it/s]
Loading 0:  62%|██████▏   | 619/995 [00:53<00:11, 32.08it/s]
Loading 0:  63%|██████▎   | 623/995 [00:53<00:11, 33.18it/s]
Loading 0:  63%|██████▎   | 627/995 [00:53<00:12, 28.32it/s]
Loading 0:  63%|██████▎   | 631/995 [00:53<00:12, 29.71it/s]
Loading 0:  64%|██████▍   | 636/995 [00:53<00:14, 25.08it/s]
Loading 0:  64%|██████▍   | 639/995 [00:53<00:14, 24.49it/s]
Loading 0:  65%|██████▍   | 642/995 [00:54<00:14, 24.34it/s]
Loading 0:  65%|██████▍   | 645/995 [00:54<00:14, 24.14it/s]
Loading 0:  65%|██████▌   | 648/995 [00:54<00:14, 24.42it/s]
Loading 0:  66%|██████▌   | 653/995 [00:54<00:11, 28.98it/s]
Loading 0:  66%|██████▌   | 656/995 [00:54<00:12, 28.17it/s]
Loading 0:  66%|██████▌   | 659/995 [00:54<00:12, 27.56it/s]
Loading 0:  67%|██████▋   | 662/995 [00:54<00:12, 26.98it/s]
Loading 0:  67%|██████▋   | 665/995 [00:54<00:12, 26.01it/s]
Loading 0:  67%|██████▋   | 668/995 [00:55<00:12, 25.20it/s]
Loading 0:  67%|██████▋   | 671/995 [00:55<00:12, 25.22it/s]
Loading 0:  68%|██████▊   | 674/995 [00:55<00:12, 25.11it/s]
Loading 0:  68%|██████▊   | 681/995 [00:55<00:08, 36.37it/s]
Loading 0:  69%|██████▉   | 688/995 [00:55<00:06, 44.98it/s]
Loading 0:  70%|██████▉   | 693/995 [00:55<00:11, 26.83it/s]
Loading 0:  70%|███████   | 697/995 [00:56<00:11, 25.14it/s]
Loading 0:  70%|███████   | 701/995 [00:56<00:11, 25.90it/s]
Loading 0:  71%|███████   | 705/995 [00:56<00:10, 27.03it/s]
Loading 0:  71%|███████▏  | 709/995 [00:56<00:11, 25.09it/s]
Loading 0:  72%|███████▏  | 712/995 [00:56<00:11, 25.17it/s]
Loading 0:  72%|███████▏  | 717/995 [00:56<00:09, 29.55it/s]
Loading 0:  72%|███████▏  | 721/995 [00:56<00:09, 28.19it/s]
Loading 0:  73%|███████▎  | 724/995 [00:57<00:09, 27.39it/s]
Loading 0:  73%|███████▎  | 727/995 [00:57<00:10, 26.40it/s]
Loading 0:  73%|███████▎  | 730/995 [00:57<00:10, 25.29it/s]
Loading 0:  74%|███████▎  | 733/995 [00:57<00:10, 25.24it/s]
Loading 0:  74%|███████▍  | 739/995 [00:57<00:11, 23.17it/s]
Loading 0:  75%|███████▍  | 742/995 [00:57<00:10, 23.58it/s]
Loading 0:  75%|███████▍  | 746/995 [00:57<00:09, 26.88it/s]
Loading 0:  75%|███████▌  | 749/995 [00:58<00:09, 26.58it/s]
Loading 0:  76%|███████▌  | 752/995 [00:58<00:09, 26.36it/s]
Loading 0:  76%|███████▌  | 755/995 [00:58<00:09, 26.05it/s]
Loading 0:  76%|███████▌  | 758/995 [00:58<00:09, 25.64it/s]
Loading 0:  76%|███████▋  | 761/995 [00:58<00:09, 25.47it/s]
Loading 0:  77%|███████▋  | 764/995 [00:58<00:09, 25.08it/s]
Loading 0:  77%|███████▋  | 767/995 [00:58<00:09, 24.82it/s]
Loading 0:  77%|███████▋  | 770/995 [00:58<00:08, 26.01it/s]
Loading 0:  78%|███████▊  | 778/995 [00:59<00:05, 38.86it/s]
Loading 0:  79%|███████▊  | 782/995 [00:59<00:06, 30.55it/s]
Loading 0:  79%|███████▉  | 786/995 [00:59<00:07, 29.48it/s]
Loading 0:  79%|███████▉  | 790/995 [00:59<00:06, 31.68it/s]
Loading 0:  80%|███████▉  | 794/995 [00:59<00:08, 23.58it/s]
Loading 0:  80%|████████  | 797/995 [00:59<00:08, 23.51it/s]
Loading 0:  80%|████████  | 800/995 [00:59<00:08, 24.11it/s]
Loading 0:  81%|████████  | 803/995 [01:00<00:07, 24.50it/s]
Loading 0:  81%|████████  | 808/995 [01:00<00:06, 29.01it/s]
Loading 0:  82%|████████▏ | 812/995 [01:00<00:06, 29.29it/s]
Loading 0:  82%|████████▏ | 816/995 [01:00<00:06, 25.70it/s]
Loading 0:  82%|████████▏ | 819/995 [01:00<00:06, 25.36it/s]
Loading 0:  83%|████████▎ | 822/995 [01:00<00:07, 24.32it/s]
Loading 0:  83%|████████▎ | 825/995 [01:00<00:07, 24.26it/s]
Loading 0:  83%|████████▎ | 828/995 [01:01<00:06, 23.98it/s]
Loading 0:  84%|████████▎ | 831/995 [01:01<00:06, 24.02it/s]
Loading 0:  84%|████████▍ | 839/995 [01:01<00:04, 37.76it/s]
Loading 0:  85%|████████▍ | 842/995 [01:16<00:04, 37.76it/s]
Loading 0:  85%|████████▍ | 843/995 [01:16<02:42,  1.07s/it]
Loading 0:  85%|████████▌ | 848/995 [01:16<01:46,  1.37it/s]
Loading 0:  86%|████████▌ | 853/995 [01:17<01:14,  1.90it/s]
Loading 0:  86%|████████▌ | 857/995 [01:17<00:54,  2.53it/s]
Loading 0:  87%|████████▋ | 861/995 [01:17<00:40,  3.33it/s]
Loading 0:  87%|████████▋ | 864/995 [01:17<00:31,  4.16it/s]
Loading 0:  87%|████████▋ | 867/995 [01:17<00:24,  5.22it/s]
Loading 0:  88%|████████▊ | 872/995 [01:18<00:15,  7.74it/s]
Loading 0:  88%|████████▊ | 876/995 [01:18<00:12,  9.86it/s]
Loading 0:  88%|████████▊ | 880/995 [01:18<00:09, 12.34it/s]
Loading 0:  89%|████████▉ | 884/995 [01:18<00:07, 14.10it/s]
Loading 0:  89%|████████▉ | 887/995 [01:18<00:06, 15.80it/s]
Loading 0:  89%|████████▉ | 890/995 [01:18<00:05, 17.54it/s]
Loading 0:  90%|█████████ | 896/995 [01:18<00:04, 23.81it/s]
Loading 0:  90%|█████████ | 900/995 [01:19<00:04, 20.51it/s]
Loading 0:  91%|█████████ | 903/995 [01:19<00:04, 21.61it/s]
Loading 0:  91%|█████████ | 906/995 [01:19<00:03, 22.54it/s]
Loading 0:  91%|█████████▏| 909/995 [01:19<00:03, 23.35it/s]
Loading 0:  92%|█████████▏| 912/995 [01:19<00:03, 23.94it/s]
Loading 0:  92%|█████████▏| 915/995 [01:19<00:03, 24.47it/s]
Loading 0:  92%|█████████▏| 918/995 [01:19<00:03, 24.48it/s]
Loading 0:  93%|█████████▎| 921/995 [01:19<00:03, 24.53it/s]
Loading 0:  93%|█████████▎| 924/995 [01:20<00:02, 24.02it/s]
Loading 0:  94%|█████████▍| 933/995 [01:20<00:01, 39.51it/s]
Loading 0:  94%|█████████▍| 938/995 [01:20<00:01, 32.36it/s]
Loading 0:  95%|█████████▍| 942/995 [01:20<00:01, 31.72it/s]
Loading 0:  95%|█████████▌| 946/995 [01:20<00:01, 27.97it/s]
Loading 0:  96%|█████████▌| 952/995 [01:22<00:05,  8.44it/s]
Loading 0:  96%|█████████▌| 955/995 [01:22<00:04,  9.84it/s]
Loading 0:  96%|█████████▋| 958/995 [01:22<00:03, 11.44it/s]
Loading 0:  97%|█████████▋| 961/995 [01:22<00:02, 13.21it/s]
Loading 0:  97%|█████████▋| 966/995 [01:22<00:01, 17.78it/s]
Loading 0:  97%|█████████▋| 970/995 [01:22<00:01, 20.15it/s]
Loading 0:  98%|█████████▊| 973/995 [01:23<00:01, 21.21it/s]
Loading 0:  98%|█████████▊| 976/995 [01:23<00:00, 22.11it/s]
Loading 0:  98%|█████████▊| 979/995 [01:23<00:00, 23.17it/s]
Loading 0:  99%|█████████▊| 982/995 [01:23<00:00, 24.04it/s]
Loading 0:  99%|█████████▉| 985/995 [01:23<00:00, 24.47it/s]
Loading 0: 100%|█████████▉| 991/995 [01:23<00:00, 32.92it/s]
                                                            
/opt/conda/lib/python3.10/site-packages/transformers/models/auto/configuration_auto.py:957: FutureWarning: The `use_auth_token` argument is deprecated and will be removed in v5 of Transformers. Please use `token` instead.
mistralai-mixtral-8x7b-3473-v115-mkmlizer:   warnings.warn(
mistralai-mixtral-8x7b-3473-v115-mkmlizer: /opt/conda/lib/python3.10/site-packages/transformers/models/auto/tokenization_auto.py:785: FutureWarning: The `use_auth_token` argument is deprecated and will be removed in v5 of Transformers. Please use `token` instead.
mistralai-mixtral-8x7b-3473-v115-mkmlizer:   warnings.warn(
mistralai-mixtral-8x7b-3473-v115-mkmlizer: /opt/conda/lib/python3.10/site-packages/transformers/models/auto/auto_factory.py:469: FutureWarning: The `use_auth_token` argument is deprecated and will be removed in v5 of Transformers. Please use `token` instead.
mistralai-mixtral-8x7b-3473-v115-mkmlizer:   warnings.warn(
mistralai-mixtral-8x7b-3473-v115-mkmlizer: Saving model to /tmp/reward_cache/reward.tensors
mistralai-mixtral-8x7b-3473-v115-mkmlizer: Saving duration: 0.120s
mistralai-mixtral-8x7b-3473-v115-mkmlizer: Processed model rirv938/reward_gpt2_preference_24m_e2 in 6.225s
mistralai-mixtral-8x7b-3473-v115-mkmlizer: creating bucket guanaco-reward-models
mistralai-mixtral-8x7b-3473-v115-mkmlizer: Bucket 's3://guanaco-reward-models/' created
mistralai-mixtral-8x7b-3473-v115-mkmlizer: uploading /tmp/reward_cache to s3://guanaco-reward-models/mistralai-mixtral-8x7b-3473-v115_reward
mistralai-mixtral-8x7b-3473-v115-mkmlizer: cp /tmp/reward_cache/config.json s3://guanaco-reward-models/mistralai-mixtral-8x7b-3473-v115_reward/config.json
mistralai-mixtral-8x7b-3473-v115-mkmlizer: cp /tmp/reward_cache/special_tokens_map.json s3://guanaco-reward-models/mistralai-mixtral-8x7b-3473-v115_reward/special_tokens_map.json
mistralai-mixtral-8x7b-3473-v115-mkmlizer: cp /tmp/reward_cache/tokenizer_config.json s3://guanaco-reward-models/mistralai-mixtral-8x7b-3473-v115_reward/tokenizer_config.json
mistralai-mixtral-8x7b-3473-v115-mkmlizer: cp /tmp/reward_cache/merges.txt s3://guanaco-reward-models/mistralai-mixtral-8x7b-3473-v115_reward/merges.txt
mistralai-mixtral-8x7b-3473-v115-mkmlizer: cp /tmp/reward_cache/vocab.json s3://guanaco-reward-models/mistralai-mixtral-8x7b-3473-v115_reward/vocab.json
mistralai-mixtral-8x7b-3473-v115-mkmlizer: cp /tmp/reward_cache/tokenizer.json s3://guanaco-reward-models/mistralai-mixtral-8x7b-3473-v115_reward/tokenizer.json
mistralai-mixtral-8x7b-3473-v115-mkmlizer: cp /tmp/reward_cache/reward.tensors s3://guanaco-reward-models/mistralai-mixtral-8x7b-3473-v115_reward/reward.tensors
Job mistralai-mixtral-8x7b-3473-v115-mkmlizer completed after 331.73s with status: succeeded
Stopping job with name mistralai-mixtral-8x7b-3473-v115-mkmlizer
Pipeline stage MKMLizer completed in 333.14s
Running pipeline stage MKMLKubeTemplater
Pipeline stage MKMLKubeTemplater completed in 0.10s
Running pipeline stage ISVCDeployer
Creating inference service mistralai-mixtral-8x7b-3473-v115
Waiting for inference service mistralai-mixtral-8x7b-3473-v115 to be ready
Inference service mistralai-mixtral-8x7b-3473-v115 ready after 171.081547498703s
Pipeline stage ISVCDeployer completed in 172.91s
Running pipeline stage StressChecker
Received healthy response to inference request in 3.0272819995880127s
Received healthy response to inference request in 1.0991971492767334s
Received healthy response to inference request in 1.481083869934082s
Received healthy response to inference request in 1.1820259094238281s
Received healthy response to inference request in 1.4648654460906982s
5 requests
0 failed requests
5th percentile: 1.1157629013061523
10th percentile: 1.1323286533355712
20th percentile: 1.1654601573944092
30th percentile: 1.2385938167572021
40th percentile: 1.3517296314239502
50th percentile: 1.4648654460906982
60th percentile: 1.4713528156280518
70th percentile: 1.4778401851654053
80th percentile: 1.7903234958648684
90th percentile: 2.4088027477264404
95th percentile: 2.718042373657226
99th percentile: 2.9654340744018555
mean time: 1.650890874862671
Pipeline stage StressChecker completed in 9.09s
mistralai-mixtral-8x7b_3473_v115 status is now deployed due to DeploymentManager action
mistralai-mixtral-8x7b_3473_v115 status is now inactive due to auto deactivation removed underperforming models
mistralai-mixtral-8x7b_3473_v115 status is now torndown due to DeploymentManager action