submission_id: mistralai-mixtral-8x7b_3473_v101
developer_uid: chai_backend_admin
alignment_samples: 0
best_of: 4
celo_rating: 1166.93
display_name: mistralai-mixtral-8x7b_3473_v101
formatter: {'memory_template': '<s>[INST] This is an entertaining conversation. You are {bot_name} who has the persona: {memory}.\nEngage in a chat with {user_name} while staying in character. Try to flirt with {user_name}. Engage in *roleplay* actions. Describe the scene dramatically. \n', 'prompt_template': '{prompt}\n', 'bot_template': '{bot_name}: {message}</s>', 'user_template': '[INST] {user_name}: {message} [/INST]', 'response_template': '{bot_name}:', 'truncate_by_message': False}
generation_params: {'temperature': 0.9, 'top_p': 1.0, 'min_p': 0.0, 'top_k': 50, 'presence_penalty': 0.5, 'frequency_penalty': 0.5, 'stopping_words': ['\n', '</s>', '<|user|>', '###'], 'max_input_tokens': 512, 'best_of': 4, 'max_output_tokens': 64, 'reward_max_token_input': 256}
is_internal_developer: True
language_model: mistralai/Mixtral-8x7B-Instruct-v0.1
max_input_tokens: 512
max_output_tokens: 64
model_architecture: MixtralForCausalLM
model_group: mistralai/Mixtral-8x7B-I
model_name: mistralai-mixtral-8x7b_3473_v101
model_num_parameters: 46702792704.0
model_repo: mistralai/Mixtral-8x7B-Instruct-v0.1
model_size: 47B
num_battles: 10901
num_wins: 4875
propriety_score: 0.744258872651357
propriety_total_count: 958.0
ranking_group: single
reward_formatter: {'bot_template': 'Bot: {message}\n', 'memory_template': 'Memory: {memory}\n', 'prompt_template': '{prompt}\n', 'response_template': 'Bot:', 'truncate_by_message': False, 'user_template': 'User: {message}\n'}
reward_repo: ChaiML/gpt2_xl_pairwise_89m_step_347634
status: torndown
submission_type: basic
timestamp: 2024-07-30T18:01:19+00:00
us_pacific_date: 2024-07-30
win_ratio: 0.44720667828639576
Download Preference Data
Resubmit model
Running pipeline stage MKMLizer
Starting job with name mistralai-mixtral-8x7b-3473-v101-mkmlizer
Waiting for job on mistralai-mixtral-8x7b-3473-v101-mkmlizer to finish
mistralai-mixtral-8x7b-3473-v101-mkmlizer: ╔═════════════════════════════════════════════════════════════════════╗
mistralai-mixtral-8x7b-3473-v101-mkmlizer: ║ _____ __ __ ║
mistralai-mixtral-8x7b-3473-v101-mkmlizer: ║ / _/ /_ ___ __/ / ___ ___ / / ║
mistralai-mixtral-8x7b-3473-v101-mkmlizer: ║ / _/ / // / |/|/ / _ \/ -_) -_) / ║
mistralai-mixtral-8x7b-3473-v101-mkmlizer: ║ /_//_/\_, /|__,__/_//_/\__/\__/_/ ║
mistralai-mixtral-8x7b-3473-v101-mkmlizer: ║ /___/ ║
mistralai-mixtral-8x7b-3473-v101-mkmlizer: ║ ║
mistralai-mixtral-8x7b-3473-v101-mkmlizer: ║ Version: 0.9.7 ║
mistralai-mixtral-8x7b-3473-v101-mkmlizer: ║ Copyright 2023 MK ONE TECHNOLOGIES Inc. ║
mistralai-mixtral-8x7b-3473-v101-mkmlizer: ║ https://mk1.ai ║
mistralai-mixtral-8x7b-3473-v101-mkmlizer: ║ ║
mistralai-mixtral-8x7b-3473-v101-mkmlizer: ║ The license key for the current software has been verified as ║
mistralai-mixtral-8x7b-3473-v101-mkmlizer: ║ belonging to: ║
mistralai-mixtral-8x7b-3473-v101-mkmlizer: ║ ║
mistralai-mixtral-8x7b-3473-v101-mkmlizer: ║ Chai Research Corp. ║
mistralai-mixtral-8x7b-3473-v101-mkmlizer: ║ Account ID: 7997a29f-0ceb-4cc7-9adf-840c57b4ae6f ║
mistralai-mixtral-8x7b-3473-v101-mkmlizer: ║ Expiration: 2024-10-15 23:59:59 ║
mistralai-mixtral-8x7b-3473-v101-mkmlizer: ║ ║
mistralai-mixtral-8x7b-3473-v101-mkmlizer: ╚═════════════════════════════════════════════════════════════════════╝
mistralai-mixtral-8x7b-3473-v101-mkmlizer: Downloaded to shared memory in 176.945s
mistralai-mixtral-8x7b-3473-v101-mkmlizer: quantizing model to /dev/shm/model_cache, profile:s0, folder:/tmp/tmpbyttvaw0, device:0
mistralai-mixtral-8x7b-3473-v101-mkmlizer: Saving flywheel model at /dev/shm/model_cache
mistralai-mixtral-8x7b-3473-v101-mkmlizer: creating bucket guanaco-mkml-models
mistralai-mixtral-8x7b-3473-v101-mkmlizer: Bucket 's3://guanaco-mkml-models/' created
mistralai-mixtral-8x7b-3473-v101-mkmlizer: uploading /dev/shm/model_cache to s3://guanaco-mkml-models/mistralai-mixtral-8x7b-3473-v101
mistralai-mixtral-8x7b-3473-v101-mkmlizer: cp /dev/shm/model_cache/config.json s3://guanaco-mkml-models/mistralai-mixtral-8x7b-3473-v101/config.json
mistralai-mixtral-8x7b-3473-v101-mkmlizer: cp /dev/shm/model_cache/special_tokens_map.json s3://guanaco-mkml-models/mistralai-mixtral-8x7b-3473-v101/special_tokens_map.json
mistralai-mixtral-8x7b-3473-v101-mkmlizer: cp /dev/shm/model_cache/tokenizer_config.json s3://guanaco-mkml-models/mistralai-mixtral-8x7b-3473-v101/tokenizer_config.json
mistralai-mixtral-8x7b-3473-v101-mkmlizer: cp /dev/shm/model_cache/tokenizer.model s3://guanaco-mkml-models/mistralai-mixtral-8x7b-3473-v101/tokenizer.model
mistralai-mixtral-8x7b-3473-v101-mkmlizer: cp /dev/shm/model_cache/tokenizer.json s3://guanaco-mkml-models/mistralai-mixtral-8x7b-3473-v101/tokenizer.json
mistralai-mixtral-8x7b-3473-v101-mkmlizer: cp /dev/shm/model_cache/flywheel_model.3.safetensors s3://guanaco-mkml-models/mistralai-mixtral-8x7b-3473-v101/flywheel_model.3.safetensors
mistralai-mixtral-8x7b-3473-v101-mkmlizer: cp /dev/shm/model_cache/flywheel_model.0.safetensors s3://guanaco-mkml-models/mistralai-mixtral-8x7b-3473-v101/flywheel_model.0.safetensors
mistralai-mixtral-8x7b-3473-v101-mkmlizer: cp /dev/shm/model_cache/flywheel_model.2.safetensors s3://guanaco-mkml-models/mistralai-mixtral-8x7b-3473-v101/flywheel_model.2.safetensors
mistralai-mixtral-8x7b-3473-v101-mkmlizer: cp /dev/shm/model_cache/flywheel_model.1.safetensors s3://guanaco-mkml-models/mistralai-mixtral-8x7b-3473-v101/flywheel_model.1.safetensors
mistralai-mixtral-8x7b-3473-v101-mkmlizer: loading reward model from ChaiML/gpt2_xl_pairwise_89m_step_347634
mistralai-mixtral-8x7b-3473-v101-mkmlizer: Loading 0: 0%| | 0/995 [00:00<?, ?it/s] Loading 0: 0%| | 4/995 [00:00<00:27, 35.59it/s] Loading 0: 1%| | 8/995 [00:00<00:30, 32.83it/s] Loading 0: 1%| | 12/995 [00:00<00:30, 32.50it/s] Loading 0: 2%|▏ | 16/995 [00:00<00:35, 27.33it/s] Loading 0: 2%|▏ | 19/995 [00:00<00:35, 27.41it/s] Loading 0: 2%|▏ | 22/995 [00:00<00:35, 27.54it/s] Loading 0: 3%|▎ | 25/995 [00:00<00:35, 27.48it/s] Loading 0: 3%|▎ | 34/995 [00:00<00:21, 44.51it/s] Loading 0: 4%|▍ | 39/995 [00:01<00:26, 35.99it/s] Loading 0: 4%|▍ | 43/995 [00:01<00:27, 34.82it/s] Loading 0: 5%|▍ | 47/995 [00:01<00:27, 35.03it/s] Loading 0: 5%|▌ | 52/995 [00:01<00:36, 25.77it/s] Loading 0: 6%|▌ | 56/995 [00:01<00:35, 26.78it/s] Loading 0: 6%|▌ | 60/995 [00:01<00:33, 27.72it/s] Loading 0: 7%|▋ | 65/995 [00:02<00:29, 31.80it/s] Loading 0: 7%|▋ | 69/995 [00:02<00:32, 28.16it/s] Loading 0: 7%|▋ | 73/995 [00:02<00:31, 29.02it/s] Loading 0: 8%|▊ | 77/995 [00:02<00:30, 30.01it/s] Loading 0: 8%|▊ | 81/995 [00:02<00:33, 27.27it/s] Loading 0: 8%|▊ | 84/995 [00:02<00:33, 27.47it/s] Loading 0: 9%|▊ | 87/995 [00:02<00:32, 27.62it/s] Loading 0: 10%|▉ | 96/995 [00:03<00:20, 42.84it/s] Loading 0: 10%|█ | 101/995 [00:03<00:25, 35.26it/s] Loading 0: 11%|█ | 107/995 [00:03<00:28, 30.81it/s] Loading 0: 11%|█ | 111/995 [00:03<00:32, 27.34it/s] Loading 0: 12%|█▏ | 115/995 [00:03<00:31, 28.07it/s] Loading 0: 12%|█▏ | 119/995 [00:03<00:30, 28.92it/s] Loading 0: 12%|█▏ | 123/995 [00:04<00:33, 25.73it/s] Loading 0: 13%|█▎ | 128/995 [00:04<00:28, 30.04it/s] Loading 0: 13%|█▎ | 132/995 [00:04<00:28, 30.15it/s] Loading 0: 14%|█▎ | 136/995 [00:04<00:28, 30.38it/s] Loading 0: 14%|█▍ | 140/995 [00:04<00:31, 27.41it/s] Loading 0: 14%|█▍ | 143/995 [00:04<00:31, 27.42it/s] Loading 0: 15%|█▍ | 146/995 [00:04<00:31, 26.95it/s] Loading 0: 15%|█▍ | 149/995 [00:05<00:31, 26.83it/s] Loading 0: 16%|█▌ | 161/995 [00:05<00:18, 45.02it/s] Loading 0: 17%|█▋ | 166/995 [00:05<00:27, 30.46it/s] Loading 0: 17%|█▋ | 170/995 [00:05<00:29, 27.74it/s] Loading 0: 17%|█▋ | 174/995 [00:05<00:29, 28.20it/s] Loading 0: 18%|█▊ | 178/995 [00:05<00:28, 29.04it/s] Loading 0: 18%|█▊ | 182/995 [00:06<00:30, 26.77it/s] Loading 0: 19%|█▊ | 185/995 [00:06<00:30, 26.93it/s] Loading 0: 19%|█▉ | 190/995 [00:06<00:26, 30.78it/s] Loading 0: 19%|█▉ | 194/995 [00:06<00:26, 30.63it/s] Loading 0: 20%|█▉ | 198/995 [00:06<00:25, 30.80it/s] Loading 0: 20%|██ | 202/995 [00:06<00:29, 27.03it/s] Loading 0: 21%|██ | 209/995 [00:06<00:21, 36.20it/s] Loading 0: 22%|██▏ | 214/995 [00:07<00:32, 24.12it/s] Loading 0: 22%|██▏ | 219/995 [00:07<00:27, 28.12it/s] Loading 0: 22%|██▏ | 223/995 [00:07<00:26, 29.10it/s] Loading 0: 23%|██▎ | 227/995 [00:07<00:28, 26.67it/s] Loading 0: 23%|██▎ | 231/995 [00:07<00:27, 27.61it/s] Loading 0: 24%|██▎ | 235/995 [00:07<00:26, 28.80it/s] Loading 0: 24%|██▍ | 239/995 [00:08<00:28, 26.38it/s] Loading 0: 24%|██▍ | 242/995 [00:08<00:28, 26.78it/s] Loading 0: 25%|██▌ | 251/995 [00:08<00:18, 41.04it/s] Loading 0: 26%|██▌ | 256/995 [00:08<00:21, 34.48it/s] Loading 0: 26%|██▌ | 260/995 [00:08<00:20, 35.14it/s] Loading 0: 27%|██▋ | 265/995 [00:08<00:25, 28.66it/s] Loading 0: 27%|██▋ | 270/995 [00:08<00:22, 32.76it/s] Loading 0: 28%|██▊ | 274/995 [00:09<00:23, 30.20it/s] Loading 0: 28%|██▊ | 278/995 [00:09<00:23, 29.88it/s] Loading 0: 28%|██▊ | 282/995 [00:09<00:23, 29.84it/s] Loading 0: 28%|██▊ | 282/995 [00:20<00:23, 29.84it/s] Loading 0: 28%|██▊ | 283/995 [00:29<21:48, 1.84s/it] Loading 0: 29%|██▊ | 285/995 [00:29<17:27, 1.48s/it] Loading 0: 29%|██▉ | 288/995 [00:29<12:14, 1.04s/it] Loading 0: 29%|██▉ | 291/995 [00:29<08:40, 1.35it/s] Loading 0: 30%|██▉ | 294/995 [00:29<06:09, 1.90it/s] Loading 0: 30%|██▉ | 297/995 [00:30<04:24, 2.64it/s] Loading 0: 30%|███ | 300/995 [00:30<03:11, 3.63it/s] Loading 0: 30%|███ | 303/995 [00:30<02:20, 4.91it/s] Loading 0: 31%|███ | 306/995 [00:30<01:45, 6.53it/s] Loading 0: 31%|███ | 309/995 [00:30<01:20, 8.47it/s] Loading 0: 31%|███▏ | 312/995 [00:30<01:03, 10.68it/s] Loading 0: 32%|███▏ | 320/995 [00:30<00:40, 16.66it/s] Loading 0: 32%|███▏ | 323/995 [00:31<00:36, 18.19it/s] Loading 0: 33%|███▎ | 326/995 [00:31<00:33, 19.77it/s] Loading 0: 33%|███▎ | 329/995 [00:31<00:31, 21.31it/s] Loading 0: 33%|███▎ | 332/995 [00:31<00:29, 22.77it/s] Loading 0: 34%|███▎ | 335/995 [00:31<00:27, 23.97it/s] Loading 0: 34%|███▍ | 338/995 [00:31<00:26, 24.90it/s] Loading 0: 34%|███▍ | 343/995 [00:31<00:21, 30.71it/s] Loading 0: 35%|███▍ | 347/995 [00:31<00:20, 30.94it/s] Loading 0: 35%|███▌ | 351/995 [00:31<00:23, 27.22it/s] Loading 0: 36%|███▌ | 354/995 [00:32<00:23, 27.43it/s] Loading 0: 36%|███▌ | 357/995 [00:32<00:23, 27.73it/s] Loading 0: 36%|███▌ | 360/995 [00:32<00:22, 27.78it/s] Loading 0: 37%|███▋ | 367/995 [00:32<00:16, 38.61it/s] Loading 0: 37%|███▋ | 372/995 [00:32<00:24, 25.10it/s] Loading 0: 38%|███▊ | 376/995 [00:32<00:22, 27.41it/s] Loading 0: 38%|███▊ | 380/995 [00:32<00:21, 28.36it/s] Loading 0: 39%|███▊ | 384/995 [00:33<00:20, 29.39it/s] Loading 0: 39%|███▉ | 388/995 [00:33<00:23, 25.93it/s] Loading 0: 39%|███▉ | 391/995 [00:33<00:23, 26.09it/s] Loading 0: 40%|███▉ | 394/995 [00:33<00:22, 26.46it/s] Loading 0: 40%|███▉ | 397/995 [00:33<00:22, 26.73it/s] Loading 0: 41%|████ | 406/995 [00:33<00:13, 42.34it/s] Loading 0: 41%|████▏ | 411/995 [00:33<00:16, 35.00it/s] Loading 0: 42%|████▏ | 415/995 [00:34<00:17, 34.06it/s] Loading 0: 42%|████▏ | 420/995 [00:34<00:15, 37.74it/s] Loading 0: 43%|████▎ | 425/995 [00:34<00:22, 25.13it/s] Loading 0: 43%|████▎ | 429/995 [00:34<00:21, 26.52it/s] Loading 0: 44%|████▎ | 433/995 [00:34<00:22, 25.28it/s] Loading 0: 44%|████▍ | 438/995 [00:34<00:18, 29.54it/s] Loading 0: 44%|████▍ | 442/995 [00:35<00:18, 29.64it/s] Loading 0: 45%|████▍ | 446/995 [00:35<00:18, 30.31it/s] Loading 0: 45%|████▌ | 450/995 [00:35<00:19, 27.49it/s] Loading 0: 46%|████▌ | 453/995 [00:35<00:19, 27.21it/s] Loading 0: 46%|████▌ | 456/995 [00:35<00:19, 27.36it/s] Loading 0: 46%|████▌ | 459/995 [00:35<00:19, 27.51it/s] Loading 0: 47%|████▋ | 468/995 [00:35<00:12, 42.66it/s] Loading 0: 48%|████▊ | 473/995 [00:35<00:14, 36.63it/s] Loading 0: 48%|████▊ | 478/995 [00:36<00:17, 29.56it/s] Loading 0: 48%|████▊ | 482/995 [00:36<00:17, 29.41it/s] Loading 0: 49%|████▉ | 486/995 [00:36<00:18, 26.88it/s] Loading 0: 49%|████▉ | 489/995 [00:36<00:18, 27.03it/s] Loading 0: 49%|████▉ | 492/995 [00:36<00:18, 26.95it/s] Loading 0: 50%|████▉ | 495/995 [00:36<00:18, 27.08it/s] Loading 0: 50%|█████ | 500/995 [00:36<00:15, 31.92it/s] Loading 0: 51%|█████ | 504/995 [00:37<00:15, 30.89it/s] Loading 0: 51%|█████ | 508/995 [00:37<00:16, 30.21it/s] Loading 0: 51%|█████▏ | 512/995 [00:37<00:18, 26.58it/s] Loading 0: 52%|█████▏ | 515/995 [00:37<00:17, 26.88it/s] Loading 0: 52%|█████▏ | 518/995 [00:37<00:17, 26.82it/s] Loading 0: 53%|█████▎ | 525/995 [00:37<00:12, 36.93it/s] Loading 0: 53%|█████▎ | 529/995 [00:38<00:17, 26.56it/s] Loading 0: 54%|█████▎ | 533/995 [00:38<00:16, 27.71it/s] Loading 0: 54%|█████▍ | 537/995 [00:38<00:17, 25.93it/s] Loading 0: 54%|█████▍ | 540/995 [00:38<00:17, 26.35it/s] Loading 0: 55%|█████▍ | 543/995 [00:38<00:17, 26.45it/s] Loading 0: 55%|█████▍ | 546/995 [00:38<00:16, 26.90it/s] Loading 0: 55%|█████▌ | 549/995 [00:38<00:16, 27.22it/s] Loading 0: 55%|█████▌ | 552/995 [00:38<00:16, 27.37it/s] Loading 0: 56%|█████▋ | 561/995 [00:39<00:10, 41.86it/s] Loading 0: 56%|█████▋ | 561/995 [00:50<00:10, 41.86it/s] Loading 0: 56%|█████▋ | 562/995 [00:59<11:49, 1.64s/it] Loading 0: 57%|█████▋ | 564/995 [00:59<09:40, 1.35s/it] Loading 0: 57%|█████▋ | 568/995 [00:59<06:26, 1.11it/s] Loading 0: 57%|█████▋ | 571/995 [00:59<04:43, 1.49it/s] Loading 0: 58%|█████▊ | 574/995 [00:59<03:27, 2.03it/s] Loading 0: 58%|█████▊ | 581/995 [01:00<01:53, 3.65it/s] Loading 0: 59%|█████▊ | 584/995 [01:00<01:30, 4.54it/s] Loading 0: 59%|█████▉ | 587/995 [01:00<01:11, 5.71it/s] Loading 0: 59%|█████▉ | 592/995 [01:00<00:47, 8.44it/s] Loading 0: 60%|█████▉ | 596/995 [01:00<00:38, 10.34it/s] Loading 0: 60%|██████ | 599/995 [01:00<00:32, 12.17it/s] Loading 0: 61%|██████ | 602/995 [01:00<00:27, 14.20it/s] Loading 0: 61%|██████ | 605/995 [01:01<00:23, 16.36it/s] Loading 0: 61%|██████ | 608/995 [01:01<00:20, 18.44it/s] Loading 0: 61%|██████▏ | 611/995 [01:01<00:18, 20.34it/s] Loading 0: 62%|██████▏ | 614/995 [01:01<00:17, 22.00it/s] Loading 0: 63%|██████▎ | 623/995 [01:01<00:10, 37.16it/s] Loading 0: 63%|██████▎ | 628/995 [01:01<00:11, 32.32it/s] Loading 0: 64%|██████▎ | 633/995 [01:01<00:10, 36.17it/s] Loading 0: 64%|██████▍ | 638/995 [01:02<00:13, 25.73it/s] Loading 0: 65%|██████▍ | 642/995 [01:02<00:13, 26.59it/s] Loading 0: 65%|██████▍ | 646/995 [01:02<00:12, 27.83it/s] Loading 0: 65%|██████▌ | 650/995 [01:02<00:13, 25.82it/s] Loading 0: 66%|██████▌ | 655/995 [01:02<00:11, 29.84it/s] Loading 0: 66%|██████▌ | 659/995 [01:02<00:11, 29.95it/s] Loading 0: 67%|██████▋ | 663/995 [01:02<00:10, 30.34it/s] Loading 0: 67%|██████▋ | 667/995 [01:03<00:12, 27.16it/s] Loading 0: 67%|██████▋ | 670/995 [01:03<00:11, 27.16it/s] Loading 0: 68%|██████▊ | 673/995 [01:03<00:11, 27.12it/s] Loading 0: 68%|██████▊ | 676/995 [01:03<00:11, 26.84it/s] Loading 0: 69%|██████▉ | 685/995 [01:03<00:07, 42.27it/s] Loading 0: 69%|██████▉ | 691/995 [01:03<00:10, 29.82it/s] Loading 0: 70%|██████▉ | 695/995 [01:03<00:10, 29.91it/s] Loading 0: 70%|███████ | 699/995 [01:04<00:10, 29.40it/s] Loading 0: 71%|███████ | 703/995 [01:04<00:11, 26.26it/s] Loading 0: 71%|███████ | 706/995 [01:04<00:10, 26.31it/s] Loading 0: 71%|███████▏ | 709/995 [01:04<00:10, 26.59it/s] Loading 0: 72%|███████▏ | 712/995 [01:04<00:10, 26.81it/s] Loading 0: 72%|███████▏ | 717/995 [01:04<00:08, 31.69it/s] Loading 0: 72%|███████▏ | 721/995 [01:04<00:08, 30.56it/s] Loading 0: 73%|███████▎ | 725/995 [01:05<00:08, 30.70it/s] Loading 0: 73%|███████▎ | 729/995 [01:05<00:09, 26.60it/s] Loading 0: 74%|███████▎ | 732/995 [01:05<00:09, 26.83it/s] Loading 0: 74%|███████▍ | 739/995 [01:05<00:09, 25.88it/s] Loading 0: 75%|███████▍ | 742/995 [01:05<00:09, 26.07it/s] Loading 0: 75%|███████▌ | 747/995 [01:05<00:08, 30.58it/s] Loading 0: 75%|███████▌ | 751/995 [01:06<00:08, 27.55it/s] Loading 0: 76%|███████▌ | 754/995 [01:06<00:08, 27.33it/s] Loading 0: 76%|███████▌ | 757/995 [01:06<00:08, 27.36it/s] Loading 0: 76%|███████▋ | 760/995 [01:06<00:08, 27.14it/s] Loading 0: 77%|███████▋ | 763/995 [01:06<00:08, 27.04it/s] Loading 0: 77%|███████▋ | 766/995 [01:06<00:08, 25.89it/s] Loading 0: 77%|███████▋ | 769/995 [01:06<00:08, 25.90it/s] Loading 0: 78%|███████▊ | 778/995 [01:06<00:05, 41.35it/s] Loading 0: 79%|███████▊ | 783/995 [01:07<00:06, 33.38it/s] Loading 0: 79%|███████▉ | 787/995 [01:07<00:06, 32.82it/s] Loading 0: 80%|███████▉ | 793/995 [01:07<00:05, 35.59it/s] Loading 0: 80%|████████ | 797/995 [01:07<00:08, 24.55it/s] Loading 0: 80%|████████ | 800/995 [01:07<00:07, 24.70it/s] Loading 0: 81%|████████ | 803/995 [01:07<00:07, 24.94it/s] Loading 0: 81%|████████ | 808/995 [01:07<00:06, 29.49it/s] Loading 0: 82%|████████▏ | 812/995 [01:08<00:06, 29.80it/s] Loading 0: 82%|████████▏ | 816/995 [01:08<00:06, 26.71it/s] Loading 0: 82%|████████▏ | 819/995 [01:08<00:06, 26.81it/s] Loading 0: 83%|████████▎ | 822/995 [01:08<00:06, 26.82it/s] Loading 0: 83%|████████▎ | 825/995 [01:08<00:06, 26.93it/s] Loading 0: 83%|████████▎ | 828/995 [01:08<00:06, 26.65it/s] Loading 0: 84%|████████▎ | 831/995 [01:08<00:06, 26.89it/s] Loading 0: 84%|████████▍ | 840/995 [01:08<00:03, 42.73it/s] Loading 0: 85%|████████▍ | 842/995 [01:20<00:03, 42.73it/s] Loading 0: 85%|████████▍ | 843/995 [01:29<03:33, 1.41s/it] Loading 0: 85%|████████▌ | 848/995 [01:29<02:19, 1.05it/s] Loading 0: 86%|████████▌ | 853/995 [01:29<01:35, 1.48it/s] Loading 0: 86%|████████▌ | 857/995 [01:29<01:09, 1.99it/s] Loading 0: 87%|████████▋ | 861/995 [01:29<00:50, 2.66it/s] Loading 0: 87%|████████▋ | 864/995 [01:30<00:39, 3.35it/s] Loading 0: 87%|████████▋ | 867/995 [01:30<00:30, 4.26it/s] Loading 0: 88%|████████▊ | 872/995 [01:30<00:19, 6.41it/s] Loading 0: 88%|████████▊ | 876/995 [01:30<00:14, 8.36it/s] Loading 0: 88%|████████▊ | 880/995 [01:30<00:10, 10.70it/s] Loading 0: 89%|████████▉ | 884/995 [01:30<00:08, 12.63it/s] Loading 0: 89%|████████▉ | 887/995 [01:30<00:07, 14.56it/s] Loading 0: 89%|████████▉ | 890/995 [01:30<00:06, 16.57it/s] Loading 0: 90%|█████████ | 897/995 [01:31<00:04, 19.64it/s] Loading 0: 91%|█████████ | 901/995 [01:31<00:04, 22.69it/s] Loading 0: 91%|█████████ | 905/995 [01:31<00:03, 24.41it/s] Loading 0: 91%|█████████▏| 909/995 [01:31<00:03, 23.37it/s] Loading 0: 92%|█████████▏| 912/995 [01:31<00:03, 23.77it/s] Loading 0: 92%|█████████▏| 915/995 [01:31<00:03, 23.93it/s] Loading 0: 92%|█████████▏| 918/995 [01:32<00:03, 24.32it/s] Loading 0: 93%|█████████▎| 921/995 [01:32<00:03, 24.65it/s] Loading 0: 93%|█████████▎| 924/995 [01:32<00:02, 24.78it/s] Loading 0: 94%|█████████▍| 933/995 [01:32<00:01, 39.56it/s] Loading 0: 94%|█████████▍| 938/995 [01:32<00:01, 32.36it/s] Loading 0: 95%|█████████▍| 942/995 [01:32<00:01, 31.26it/s] Loading 0: 95%|█████████▌| 946/995 [01:32<00:01, 27.48it/s] Loading 0: 96%|█████████▌| 952/995 [01:34<00:05, 8.46it/s] Loading 0: 96%|█████████▌| 955/995 [01:34<00:04, 9.82it/s] Loading 0: 96%|█████████▋| 958/995 [01:34<00:03, 11.50it/s] Loading 0: 97%|█████████▋| 961/995 [01:34<00:02, 13.43it/s] Loading 0: 97%|█████████▋| 966/995 [01:34<00:01, 18.14it/s] Loading 0: 97%|█████████▋| 970/995 [01:35<00:01, 20.04it/s] Loading 0: 98%|█████████▊| 973/995 [01:35<00:01, 20.85it/s] Loading 0: 98%|█████████▊| 976/995 [01:35<00:00, 21.71it/s] Loading 0: 98%|█████████▊| 979/995 [01:35<00:00, 22.43it/s] Loading 0: 99%|█████████▊| 982/995 [01:35<00:00, 23.22it/s] Loading 0: 99%|█████████▉| 985/995 [01:35<00:00, 23.81it/s] Loading 0: 100%|█████████▉| 992/995 [01:35<00:00, 34.62it/s] /opt/conda/lib/python3.10/site-packages/transformers/models/auto/configuration_auto.py:957: FutureWarning: The `use_auth_token` argument is deprecated and will be removed in v5 of Transformers. Please use `token` instead.
mistralai-mixtral-8x7b-3473-v101-mkmlizer: warnings.warn(
mistralai-mixtral-8x7b-3473-v101-mkmlizer: /opt/conda/lib/python3.10/site-packages/transformers/models/auto/tokenization_auto.py:785: FutureWarning: The `use_auth_token` argument is deprecated and will be removed in v5 of Transformers. Please use `token` instead.
mistralai-mixtral-8x7b-3473-v101-mkmlizer: warnings.warn(
mistralai-mixtral-8x7b-3473-v101-mkmlizer: /opt/conda/lib/python3.10/site-packages/transformers/models/auto/auto_factory.py:469: FutureWarning: The `use_auth_token` argument is deprecated and will be removed in v5 of Transformers. Please use `token` instead.
mistralai-mixtral-8x7b-3473-v101-mkmlizer: warnings.warn(
mistralai-mixtral-8x7b-3473-v101-mkmlizer: Loading checkpoint shards: 0%| | 0/2 [00:00<?, ?it/s] Loading checkpoint shards: 50%|█████ | 1/2 [00:00<00:00, 2.14it/s] Loading checkpoint shards: 100%|██████████| 2/2 [00:00<00:00, 3.56it/s] Loading checkpoint shards: 100%|██████████| 2/2 [00:00<00:00, 3.23it/s]
mistralai-mixtral-8x7b-3473-v101-mkmlizer: Saving model to /tmp/reward_cache/reward.tensors
mistralai-mixtral-8x7b-3473-v101-mkmlizer: Saving duration: 1.676s
mistralai-mixtral-8x7b-3473-v101-mkmlizer: Processed model ChaiML/gpt2_xl_pairwise_89m_step_347634 in 13.989s
mistralai-mixtral-8x7b-3473-v101-mkmlizer: creating bucket guanaco-reward-models
mistralai-mixtral-8x7b-3473-v101-mkmlizer: Bucket 's3://guanaco-reward-models/' created
mistralai-mixtral-8x7b-3473-v101-mkmlizer: uploading /tmp/reward_cache to s3://guanaco-reward-models/mistralai-mixtral-8x7b-3473-v101_reward
mistralai-mixtral-8x7b-3473-v101-mkmlizer: cp /tmp/reward_cache/config.json s3://guanaco-reward-models/mistralai-mixtral-8x7b-3473-v101_reward/config.json
mistralai-mixtral-8x7b-3473-v101-mkmlizer: cp /tmp/reward_cache/special_tokens_map.json s3://guanaco-reward-models/mistralai-mixtral-8x7b-3473-v101_reward/special_tokens_map.json
mistralai-mixtral-8x7b-3473-v101-mkmlizer: cp /tmp/reward_cache/tokenizer_config.json s3://guanaco-reward-models/mistralai-mixtral-8x7b-3473-v101_reward/tokenizer_config.json
mistralai-mixtral-8x7b-3473-v101-mkmlizer: cp /tmp/reward_cache/merges.txt s3://guanaco-reward-models/mistralai-mixtral-8x7b-3473-v101_reward/merges.txt
mistralai-mixtral-8x7b-3473-v101-mkmlizer: cp /tmp/reward_cache/vocab.json s3://guanaco-reward-models/mistralai-mixtral-8x7b-3473-v101_reward/vocab.json
mistralai-mixtral-8x7b-3473-v101-mkmlizer: cp /tmp/reward_cache/tokenizer.json s3://guanaco-reward-models/mistralai-mixtral-8x7b-3473-v101_reward/tokenizer.json
mistralai-mixtral-8x7b-3473-v101-mkmlizer: cp /tmp/reward_cache/reward.tensors s3://guanaco-reward-models/mistralai-mixtral-8x7b-3473-v101_reward/reward.tensors
Job mistralai-mixtral-8x7b-3473-v101-mkmlizer completed after 354.98s with status: succeeded
Stopping job with name mistralai-mixtral-8x7b-3473-v101-mkmlizer
Pipeline stage MKMLizer completed in 357.29s
Running pipeline stage MKMLKubeTemplater
Pipeline stage MKMLKubeTemplater completed in 0.53s
Running pipeline stage ISVCDeployer
Creating inference service mistralai-mixtral-8x7b-3473-v101
Waiting for inference service mistralai-mixtral-8x7b-3473-v101 to be ready
Inference service mistralai-mixtral-8x7b-3473-v101 ready after 122.50002884864807s
Pipeline stage ISVCDeployer completed in 124.62s
Running pipeline stage StressChecker
HTTP Request: %s %s "%s %d %s"
Received healthy response to inference request in 3.0736582279205322s
HTTP Request: %s %s "%s %d %s"
Received healthy response to inference request in 2.24259877204895s
HTTP Request: %s %s "%s %d %s"
Received healthy response to inference request in 2.0938961505889893s
HTTP Request: %s %s "%s %d %s"
Received healthy response to inference request in 1.8634681701660156s
HTTP Request: %s %s "%s %d %s"
Received healthy response to inference request in 2.317884922027588s
5 requests
0 failed requests
5th percentile: 1.9095537662506104
10th percentile: 1.955639362335205
20th percentile: 2.0478105545043945
30th percentile: 2.1236366748809816
40th percentile: 2.183117723464966
50th percentile: 2.24259877204895
60th percentile: 2.272713232040405
70th percentile: 2.3028276920318604
80th percentile: 2.469039583206177
90th percentile: 2.7713489055633547
95th percentile: 2.922503566741943
99th percentile: 3.0434272956848143
mean time: 2.318301248550415
Pipeline stage StressChecker completed in 16.02s
mistralai-mixtral-8x7b_3473_v101 status is now deployed due to DeploymentManager action
mistralai-mixtral-8x7b_3473_v101 status is now inactive due to auto deactivation removed underperforming models
admin requested tearing down of mistralai-mixtral-8x7b_3473_v101
Running pipeline stage ISVCDeleter
Checking if service mistralai-mixtral-8x7b-3473-v101 is running
Tearing down inference service mistralai-mixtral-8x7b-3473-v101
Service mistralai-mixtral-8x7b-3473-v101 has been torndown
Pipeline stage ISVCDeleter completed in 4.62s
Running pipeline stage MKMLModelDeleter
Cleaning model data from S3
Cleaning model data from model cache
Deleting key mistralai-mixtral-8x7b-3473-v101/config.json from bucket guanaco-mkml-models
Deleting key mistralai-mixtral-8x7b-3473-v101/flywheel_model.0.safetensors from bucket guanaco-mkml-models
Deleting key mistralai-mixtral-8x7b-3473-v101/flywheel_model.1.safetensors from bucket guanaco-mkml-models
Deleting key mistralai-mixtral-8x7b-3473-v101/flywheel_model.2.safetensors from bucket guanaco-mkml-models
Deleting key mistralai-mixtral-8x7b-3473-v101/flywheel_model.3.safetensors from bucket guanaco-mkml-models
Deleting key mistralai-mixtral-8x7b-3473-v101/special_tokens_map.json from bucket guanaco-mkml-models
Deleting key mistralai-mixtral-8x7b-3473-v101/tokenizer.json from bucket guanaco-mkml-models
Deleting key mistralai-mixtral-8x7b-3473-v101/tokenizer.model from bucket guanaco-mkml-models
Deleting key mistralai-mixtral-8x7b-3473-v101/tokenizer_config.json from bucket guanaco-mkml-models
Cleaning model data from model cache
Deleting key mistralai-mixtral-8x7b-3473-v101_reward/config.json from bucket guanaco-reward-models
Deleting key mistralai-mixtral-8x7b-3473-v101_reward/merges.txt from bucket guanaco-reward-models
Deleting key mistralai-mixtral-8x7b-3473-v101_reward/reward.tensors from bucket guanaco-reward-models
Deleting key mistralai-mixtral-8x7b-3473-v101_reward/special_tokens_map.json from bucket guanaco-reward-models
Deleting key mistralai-mixtral-8x7b-3473-v101_reward/tokenizer.json from bucket guanaco-reward-models
Deleting key mistralai-mixtral-8x7b-3473-v101_reward/tokenizer_config.json from bucket guanaco-reward-models
Deleting key mistralai-mixtral-8x7b-3473-v101_reward/vocab.json from bucket guanaco-reward-models
Pipeline stage MKMLModelDeleter completed in 10.11s
mistralai-mixtral-8x7b_3473_v101 status is now torndown due to DeploymentManager action

Usage Metrics

Latency Metrics