mistralai-mixtral-8x7b-_3473

developer_uid: azuruce
submission_id: mistralai-mixtral-8x7b-_3473_v11
model_name: albert-mixtrial-baseline-bo1
model_group: mistralai/Mixtral-8x7B-I
status: torndown
timestamp: 2024-03-27T01:59:17+00:00
num_battles: 67686
num_wins: 29646
celo_rating: 1114.0
family_friendly_score: 0.0
submission_type: basic
model_repo: mistralai/Mixtral-8x7B-Instruct-v0.1
model_architecture: MixtralForCausalLM
reward_repo: ChaiML/reward_gpt2_medium_preference_24m_e2
model_num_parameters: 46702792704.0
best_of: 1
max_input_tokens: 512
max_output_tokens: 64
display_name: albert-mixtrial-baseline-bo1
is_internal_developer: True
language_model: mistralai/Mixtral-8x7B-Instruct-v0.1
model_size: 47B
ranking_group: single
us_pacific_date: 2024-03-26
win_ratio: 0.43799308571935114
generation_params: {'temperature': 0.9, 'top_p': 1.0, 'min_p': 0.0, 'top_k': 50, 'presence_penalty': 0.5, 'frequency_penalty': 0.5, 'stopping_words': ['\n', '</s>', '<|user|>', '###'], 'max_input_tokens': 512, 'best_of': 1, 'max_output_tokens': 64}
formatter: {'memory_template': "{bot_name}'s Persona: {memory}\n####\n", 'prompt_template': '{prompt}\n<START>\n', 'bot_template': '{bot_name}: {message}\n', 'user_template': '{user_name}: {message}\n', 'response_template': '{bot_name}:', 'truncate_by_message': False}
reward_formatter: {'bot_template': '{bot_name}: {message}\n', 'memory_template': "{bot_name}'s Persona: {memory}\n####\n", 'prompt_template': '{prompt}\n<START>\n', 'response_template': '{bot_name}:', 'truncate_by_message': False, 'user_template': '{user_name}: {message}\n'}
model_eval_status: success
Resubmit model
Running pipeline stage MKMLizer
Starting job with name mistralai-mixtral-8x7b-3473-v11-mkmlizer
Waiting for job on mistralai-mixtral-8x7b-3473-v11-mkmlizer to finish
mistralai-mixtral-8x7b-3473-v11-mkmlizer: ╔═════════════════════════════════════════════════════════════════════╗
mistralai-mixtral-8x7b-3473-v11-mkmlizer: ║     _____            __           __                                ║
mistralai-mixtral-8x7b-3473-v11-mkmlizer: ║    / _/ /_ ___    __/ /  ___ ___ / /                                ║
mistralai-mixtral-8x7b-3473-v11-mkmlizer: ║   / _/ / // / |/|/ / _ \/ -_) -_) /                                 ║
mistralai-mixtral-8x7b-3473-v11-mkmlizer: ║  /_//_/\_, /|__,__/_//_/\__/\__/_/                                  ║
mistralai-mixtral-8x7b-3473-v11-mkmlizer: ║       /___/                                                         ║
mistralai-mixtral-8x7b-3473-v11-mkmlizer: ║                                                                     ║
mistralai-mixtral-8x7b-3473-v11-mkmlizer: ║  Version: 0.8.6                                                     ║
mistralai-mixtral-8x7b-3473-v11-mkmlizer: ║  Copyright 2023 MK ONE TECHNOLOGIES Inc.                            ║
mistralai-mixtral-8x7b-3473-v11-mkmlizer: ║                                                                     ║
mistralai-mixtral-8x7b-3473-v11-mkmlizer: ║  The license key for the current software has been verified as      ║
mistralai-mixtral-8x7b-3473-v11-mkmlizer: ║  belonging to:                                                      ║
mistralai-mixtral-8x7b-3473-v11-mkmlizer: ║                                                                     ║
mistralai-mixtral-8x7b-3473-v11-mkmlizer: ║  Chai Research Corp.                                                ║
mistralai-mixtral-8x7b-3473-v11-mkmlizer: ║  Account ID: 7997a29f-0ceb-4cc7-9adf-840c57b4ae6f                   ║
mistralai-mixtral-8x7b-3473-v11-mkmlizer: ║  Expiration: 2024-07-15 23:59:59                                    ║
mistralai-mixtral-8x7b-3473-v11-mkmlizer: ║                                                                     ║
mistralai-mixtral-8x7b-3473-v11-mkmlizer: ╚═════════════════════════════════════════════════════════════════════╝
mistralai-mixtral-8x7b-3473-v11-mkmlizer: /opt/conda/lib/python3.10/site-packages/huggingface_hub/utils/_deprecation.py:131: FutureWarning: 'list_files_info' (from 'huggingface_hub.hf_api') is deprecated and will be removed from version '0.23'. Use `list_repo_tree` and `get_paths_info` instead.
mistralai-mixtral-8x7b-3473-v11-mkmlizer:   warnings.warn(warning_message, FutureWarning)
mistralai-mixtral-8x7b-3473-v11-mkmlizer: quantized model in 59.631s
mistralai-mixtral-8x7b-3473-v11-mkmlizer: Processed model mistralai/Mixtral-8x7B-Instruct-v0.1 in 145.103s
mistralai-mixtral-8x7b-3473-v11-mkmlizer: creating bucket guanaco-mkml-models
mistralai-mixtral-8x7b-3473-v11-mkmlizer: Bucket 's3://guanaco-mkml-models/' created
mistralai-mixtral-8x7b-3473-v11-mkmlizer: uploading /dev/shm/model_cache to s3://guanaco-mkml-models/mistralai-mixtral-8x7b-3473-v11
mistralai-mixtral-8x7b-3473-v11-mkmlizer: cp /dev/shm/model_cache/config.json s3://guanaco-mkml-models/mistralai-mixtral-8x7b-3473-v11/config.json
mistralai-mixtral-8x7b-3473-v11-mkmlizer: cp /dev/shm/model_cache/tokenizer.model s3://guanaco-mkml-models/mistralai-mixtral-8x7b-3473-v11/tokenizer.model
mistralai-mixtral-8x7b-3473-v11-mkmlizer: cp /dev/shm/model_cache/tokenizer_config.json s3://guanaco-mkml-models/mistralai-mixtral-8x7b-3473-v11/tokenizer_config.json
mistralai-mixtral-8x7b-3473-v11-mkmlizer: cp /dev/shm/model_cache/special_tokens_map.json s3://guanaco-mkml-models/mistralai-mixtral-8x7b-3473-v11/special_tokens_map.json
mistralai-mixtral-8x7b-3473-v11-mkmlizer: cp /dev/shm/model_cache/tokenizer.json s3://guanaco-mkml-models/mistralai-mixtral-8x7b-3473-v11/tokenizer.json
mistralai-mixtral-8x7b-3473-v11-mkmlizer: cp /dev/shm/model_cache/flywheel_model.3.safetensors s3://guanaco-mkml-models/mistralai-mixtral-8x7b-3473-v11/flywheel_model.3.safetensors
mistralai-mixtral-8x7b-3473-v11-mkmlizer: cp /dev/shm/model_cache/flywheel_model.1.safetensors s3://guanaco-mkml-models/mistralai-mixtral-8x7b-3473-v11/flywheel_model.1.safetensors
mistralai-mixtral-8x7b-3473-v11-mkmlizer: cp /dev/shm/model_cache/flywheel_model.2.safetensors s3://guanaco-mkml-models/mistralai-mixtral-8x7b-3473-v11/flywheel_model.2.safetensors
mistralai-mixtral-8x7b-3473-v11-mkmlizer: cp /dev/shm/model_cache/flywheel_model.0.safetensors s3://guanaco-mkml-models/mistralai-mixtral-8x7b-3473-v11/flywheel_model.0.safetensors
mistralai-mixtral-8x7b-3473-v11-mkmlizer: loading reward model from ChaiML/reward_gpt2_medium_preference_24m_e2
mistralai-mixtral-8x7b-3473-v11-mkmlizer: 
Loading 0:   0%|          | 0/995 [00:00<?, ?it/s]
Loading 0:   1%|          | 7/995 [00:00<00:15, 64.94it/s]
Loading 0:   1%|▏         | 14/995 [00:00<00:16, 59.61it/s]
Loading 0:   2%|▏         | 20/995 [00:00<00:16, 58.35it/s]
Loading 0:   3%|▎         | 31/995 [00:00<00:12, 76.89it/s]
Loading 0:   4%|▍         | 39/995 [00:00<00:13, 72.12it/s]
Loading 0:   5%|▍         | 47/995 [00:00<00:13, 69.36it/s]
Loading 0:   6%|▌         | 55/995 [00:01<00:26, 35.47it/s]
Loading 0:   6%|▌         | 61/995 [00:01<00:23, 39.25it/s]
Loading 0:   7%|▋         | 69/995 [00:01<00:19, 46.46it/s]
Loading 0:   8%|▊         | 76/995 [00:01<00:18, 48.73it/s]
Loading 0:   8%|▊         | 82/995 [00:01<00:18, 50.44it/s]
Loading 0:   9%|▉         | 93/995 [00:01<00:13, 64.60it/s]
Loading 0:  10%|█         | 101/995 [00:01<00:13, 64.67it/s]
Loading 0:  11%|█         | 109/995 [00:02<00:22, 39.74it/s]
Loading 0:  12%|█▏        | 115/995 [00:02<00:20, 42.97it/s]
Loading 0:  12%|█▏        | 121/995 [00:02<00:19, 45.91it/s]
Loading 0:  13%|█▎        | 129/995 [00:02<00:16, 53.05it/s]
Loading 0:  14%|█▎        | 136/995 [00:02<00:15, 54.46it/s]
Loading 0:  14%|█▍        | 143/995 [00:02<00:15, 55.53it/s]
Loading 0:  15%|█▌        | 150/995 [00:02<00:14, 58.70it/s]
Loading 0:  16%|█▋        | 162/995 [00:03<00:18, 45.77it/s]
Loading 0:  17%|█▋        | 168/995 [00:03<00:17, 47.88it/s]
Loading 0:  17%|█▋        | 174/995 [00:03<00:16, 50.15it/s]
Loading 0:  18%|█▊        | 180/995 [00:03<00:15, 52.14it/s]
Loading 0:  19%|█▉        | 188/995 [00:03<00:13, 58.26it/s]
Loading 0:  20%|█▉        | 195/995 [00:03<00:13, 57.79it/s]
Loading 0:  20%|██        | 202/995 [00:03<00:13, 57.51it/s]
Loading 0:  21%|██        | 210/995 [00:04<00:22, 34.36it/s]
Loading 0:  22%|██▏       | 216/995 [00:04<00:20, 38.36it/s]
Loading 0:  23%|██▎       | 224/995 [00:04<00:16, 45.83it/s]
Loading 0:  23%|██▎       | 230/995 [00:04<00:15, 48.38it/s]
Loading 0:  24%|██▎       | 236/995 [00:04<00:14, 50.64it/s]
Loading 0:  24%|██▍       | 242/995 [00:04<00:14, 52.66it/s]
Loading 0:  26%|██▌       | 255/995 [00:04<00:10, 69.96it/s]
Loading 0:  27%|██▋       | 265/995 [00:05<00:16, 43.56it/s]
Loading 0:  28%|██▊       | 276/995 [00:05<00:13, 53.55it/s]
Loading 0:  29%|██▊       | 284/995 [00:15<04:10,  2.83it/s]
Loading 0:  29%|██▉       | 292/995 [00:15<03:02,  3.85it/s]
Loading 0:  30%|███       | 300/995 [00:15<02:12,  5.23it/s]
Loading 0:  31%|███       | 307/995 [00:16<01:40,  6.85it/s]
Loading 0:  32%|███▏      | 320/995 [00:16<01:11,  9.44it/s]
Loading 0:  33%|███▎      | 326/995 [00:16<00:58, 11.48it/s]
Loading 0:  33%|███▎      | 332/995 [00:16<00:47, 14.07it/s]
Loading 0:  34%|███▍      | 338/995 [00:16<00:37, 17.32it/s]
Loading 0:  35%|███▍      | 346/995 [00:17<00:28, 23.00it/s]
Loading 0:  35%|███▌      | 353/995 [00:17<00:23, 26.94it/s]
Loading 0:  36%|███▌      | 359/995 [00:17<00:20, 31.37it/s]
Loading 0:  37%|███▋      | 368/995 [00:17<00:22, 28.16it/s]
Loading 0:  38%|███▊      | 376/995 [00:17<00:17, 35.13it/s]
Loading 0:  38%|███▊      | 382/995 [00:17<00:15, 38.98it/s]
Loading 0:  39%|███▉      | 388/995 [00:18<00:14, 42.66it/s]
Loading 0:  40%|███▉      | 394/995 [00:18<00:13, 46.06it/s]
Loading 0:  41%|████      | 407/995 [00:18<00:09, 63.11it/s]
Loading 0:  42%|████▏     | 415/995 [00:18<00:09, 61.29it/s]
Loading 0:  43%|████▎     | 423/995 [00:18<00:14, 39.35it/s]
Loading 0:  43%|████▎     | 429/995 [00:18<00:13, 42.28it/s]
Loading 0:  44%|████▍     | 437/995 [00:18<00:11, 48.73it/s]
Loading 0:  45%|████▍     | 444/995 [00:19<00:10, 50.26it/s]
Loading 0:  45%|████▌     | 450/995 [00:19<00:10, 51.66it/s]
Loading 0:  46%|████▌     | 456/995 [00:19<00:10, 52.95it/s]
Loading 0:  47%|████▋     | 468/995 [00:19<00:07, 69.23it/s]
Loading 0:  48%|████▊     | 478/995 [00:19<00:11, 44.24it/s]
Loading 0:  49%|████▊     | 485/995 [00:19<00:11, 46.10it/s]
Loading 0:  49%|████▉     | 491/995 [00:20<00:10, 48.04it/s]
Loading 0:  50%|█████     | 499/995 [00:20<00:09, 53.32it/s]
Loading 0:  51%|█████     | 506/995 [00:20<00:09, 53.04it/s]
Loading 0:  51%|█████▏    | 512/995 [00:20<00:09, 53.16it/s]
Loading 0:  52%|█████▏    | 518/995 [00:20<00:08, 53.21it/s]
Loading 0:  53%|█████▎    | 526/995 [00:20<00:13, 35.34it/s]
Loading 0:  54%|█████▎    | 534/995 [00:21<00:10, 42.27it/s]
Loading 0:  54%|█████▍    | 540/995 [00:21<00:10, 44.84it/s]
Loading 0:  55%|█████▍    | 546/995 [00:21<00:09, 47.06it/s]
Loading 0:  55%|█████▌    | 552/995 [00:21<00:09, 48.64it/s]
Loading 0:  56%|█████▋    | 560/995 [00:31<00:08, 48.64it/s]
Loading 0:  56%|█████▋    | 561/995 [00:31<03:08,  2.30it/s]
Loading 0:  57%|█████▋    | 567/995 [00:31<02:19,  3.08it/s]
Loading 0:  58%|█████▊    | 573/995 [00:31<01:41,  4.14it/s]
Loading 0:  58%|█████▊    | 581/995 [00:32<01:17,  5.37it/s]
Loading 0:  59%|█████▉    | 587/995 [00:32<00:57,  7.07it/s]
Loading 0:  60%|█████▉    | 595/995 [00:32<00:39, 10.16it/s]
Loading 0:  60%|██████    | 601/995 [00:32<00:30, 12.93it/s]
Loading 0:  61%|██████    | 607/995 [00:32<00:23, 16.28it/s]
Loading 0:  62%|██████▏   | 613/995 [00:32<00:18, 20.20it/s]
Loading 0:  63%|██████▎   | 625/995 [00:33<00:11, 31.76it/s]
Loading 0:  64%|██████▎   | 633/995 [00:33<00:09, 38.51it/s]
Loading 0:  64%|██████▍   | 641/995 [00:33<00:12, 29.50it/s]
Loading 0:  65%|██████▌   | 647/995 [00:33<00:10, 32.95it/s]
Loading 0:  66%|██████▌   | 655/995 [00:33<00:08, 39.50it/s]
Loading 0:  66%|██████▋   | 661/995 [00:33<00:07, 42.18it/s]
Loading 0:  67%|██████▋   | 667/995 [00:34<00:07, 44.44it/s]
Loading 0:  68%|██████▊   | 673/995 [00:34<00:06, 46.27it/s]
Loading 0:  69%|██████▉   | 685/995 [00:34<00:05, 61.98it/s]
Loading 0:  70%|██████▉   | 693/995 [00:34<00:07, 41.62it/s]
Loading 0:  70%|███████   | 699/995 [00:34<00:06, 43.87it/s]
Loading 0:  71%|███████   | 705/995 [00:34<00:06, 45.96it/s]
Loading 0:  71%|███████▏  | 711/995 [00:34<00:05, 47.69it/s]
Loading 0:  72%|███████▏  | 719/995 [00:35<00:05, 53.32it/s]
Loading 0:  73%|███████▎  | 725/995 [00:35<00:05, 53.14it/s]
Loading 0:  73%|███████▎  | 731/995 [00:35<00:04, 52.83it/s]
Loading 0:  74%|███████▍  | 739/995 [00:35<00:06, 36.82it/s]
Loading 0:  75%|███████▌  | 747/995 [00:35<00:05, 43.63it/s]
Loading 0:  76%|███████▌  | 753/995 [00:35<00:05, 45.73it/s]
Loading 0:  76%|███████▋  | 759/995 [00:35<00:04, 47.33it/s]
Loading 0:  77%|███████▋  | 765/995 [00:36<00:04, 48.75it/s]
Loading 0:  78%|███████▊  | 777/995 [00:36<00:03, 64.68it/s]
Loading 0:  79%|███████▉  | 784/995 [00:36<00:03, 60.92it/s]
Loading 0:  80%|███████▉  | 794/995 [00:36<00:04, 41.32it/s]
Loading 0:  80%|████████  | 800/995 [00:36<00:04, 43.92it/s]
Loading 0:  81%|████████  | 808/995 [00:36<00:03, 50.21it/s]
Loading 0:  82%|████████▏ | 814/995 [00:37<00:03, 51.52it/s]
Loading 0:  82%|████████▏ | 820/995 [00:37<00:03, 52.76it/s]
Loading 0:  83%|████████▎ | 826/995 [00:37<00:03, 53.43it/s]
Loading 0:  84%|████████▍ | 837/995 [00:37<00:02, 67.33it/s]
Loading 0:  85%|████████▍ | 845/995 [00:47<00:59,  2.51it/s]
Loading 0:  85%|████████▌ | 849/995 [00:48<00:51,  2.82it/s]
Loading 0:  86%|████████▌ | 855/995 [00:48<00:36,  3.83it/s]
Loading 0:  87%|████████▋ | 861/995 [00:48<00:25,  5.20it/s]
Loading 0:  87%|████████▋ | 867/995 [00:48<00:18,  7.03it/s]
Loading 0:  88%|████████▊ | 875/995 [00:48<00:11, 10.36it/s]
Loading 0:  89%|████████▊ | 882/995 [00:48<00:08, 13.85it/s]
Loading 0:  89%|████████▉ | 889/995 [00:48<00:05, 18.04it/s]
Loading 0:  90%|█████████ | 897/995 [00:49<00:05, 18.48it/s]
Loading 0:  91%|█████████ | 905/995 [00:49<00:03, 24.37it/s]
Loading 0:  92%|█████████▏| 911/995 [00:49<00:02, 28.51it/s]
Loading 0:  92%|█████████▏| 917/995 [00:49<00:02, 32.81it/s]
Loading 0:  93%|█████████▎| 923/995 [00:49<00:01, 37.01it/s]
Loading 0:  94%|█████████▍| 935/995 [00:49<00:01, 52.96it/s]
Loading 0:  95%|█████████▍| 943/995 [00:49<00:00, 53.72it/s]
Loading 0:  96%|█████████▌| 952/995 [00:51<00:03, 14.04it/s]
Loading 0:  96%|█████████▋| 958/995 [00:51<00:02, 17.04it/s]
Loading 0:  97%|█████████▋| 966/995 [00:51<00:01, 22.34it/s]
Loading 0:  98%|█████████▊| 972/995 [00:51<00:00, 26.24it/s]
Loading 0:  98%|█████████▊| 978/995 [00:52<00:00, 30.59it/s]
Loading 0:  99%|█████████▉| 984/995 [00:52<00:00, 34.86it/s]
                                                            
/opt/conda/lib/python3.10/site-packages/transformers/models/auto/configuration_auto.py:1096: FutureWarning: The `use_auth_token` argument is deprecated and will be removed in v5 of Transformers. Please use `token` instead.
mistralai-mixtral-8x7b-3473-v11-mkmlizer:   warnings.warn(
mistralai-mixtral-8x7b-3473-v11-mkmlizer: /opt/conda/lib/python3.10/site-packages/transformers/models/auto/tokenization_auto.py:720: FutureWarning: The `use_auth_token` argument is deprecated and will be removed in v5 of Transformers. Please use `token` instead.
mistralai-mixtral-8x7b-3473-v11-mkmlizer:   warnings.warn(
mistralai-mixtral-8x7b-3473-v11-mkmlizer: /opt/conda/lib/python3.10/site-packages/transformers/models/auto/auto_factory.py:466: FutureWarning: The `use_auth_token` argument is deprecated and will be removed in v5 of Transformers. Please use `token` instead.
mistralai-mixtral-8x7b-3473-v11-mkmlizer:   warnings.warn(
mistralai-mixtral-8x7b-3473-v11-mkmlizer: /opt/conda/lib/python3.10/site-packages/torch/_utils.py:831: UserWarning: TypedStorage is deprecated. It will be removed in the future and UntypedStorage will be the only storage class. This should only matter to you if you are using storages directly.  To access UntypedStorage directly, use tensor.untyped_storage() instead of tensor.storage()
mistralai-mixtral-8x7b-3473-v11-mkmlizer:   return self.fget.__get__(instance, owner)()
mistralai-mixtral-8x7b-3473-v11-mkmlizer: Saving model to /tmp/reward_cache/reward.tensors
mistralai-mixtral-8x7b-3473-v11-mkmlizer: Saving duration: 0.300s
mistralai-mixtral-8x7b-3473-v11-mkmlizer: Processed model ChaiML/reward_gpt2_medium_preference_24m_e2 in 6.230s
mistralai-mixtral-8x7b-3473-v11-mkmlizer: creating bucket guanaco-reward-models
mistralai-mixtral-8x7b-3473-v11-mkmlizer: Bucket 's3://guanaco-reward-models/' created
mistralai-mixtral-8x7b-3473-v11-mkmlizer: uploading /tmp/reward_cache to s3://guanaco-reward-models/mistralai-mixtral-8x7b-3473-v11_reward
mistralai-mixtral-8x7b-3473-v11-mkmlizer: cp /tmp/reward_cache/vocab.json s3://guanaco-reward-models/mistralai-mixtral-8x7b-3473-v11_reward/vocab.json
mistralai-mixtral-8x7b-3473-v11-mkmlizer: cp /tmp/reward_cache/special_tokens_map.json s3://guanaco-reward-models/mistralai-mixtral-8x7b-3473-v11_reward/special_tokens_map.json
mistralai-mixtral-8x7b-3473-v11-mkmlizer: cp /tmp/reward_cache/config.json s3://guanaco-reward-models/mistralai-mixtral-8x7b-3473-v11_reward/config.json
mistralai-mixtral-8x7b-3473-v11-mkmlizer: cp /tmp/reward_cache/tokenizer_config.json s3://guanaco-reward-models/mistralai-mixtral-8x7b-3473-v11_reward/tokenizer_config.json
mistralai-mixtral-8x7b-3473-v11-mkmlizer: cp /tmp/reward_cache/merges.txt s3://guanaco-reward-models/mistralai-mixtral-8x7b-3473-v11_reward/merges.txt
mistralai-mixtral-8x7b-3473-v11-mkmlizer: cp /tmp/reward_cache/tokenizer.json s3://guanaco-reward-models/mistralai-mixtral-8x7b-3473-v11_reward/tokenizer.json
Job mistralai-mixtral-8x7b-3473-v11-mkmlizer completed after 272.28s with status: succeeded
Stopping job with name mistralai-mixtral-8x7b-3473-v11-mkmlizer
Pipeline stage MKMLizer completed in 277.02s
Running pipeline stage MKMLKubeTemplater
Pipeline stage MKMLKubeTemplater completed in 0.10s
Running pipeline stage ISVCDeployer
Creating inference service mistralai-mixtral-8x7b-3473-v11
Waiting for inference service mistralai-mixtral-8x7b-3473-v11 to be ready
Inference service mistralai-mixtral-8x7b-3473-v11 ready after 70.41325998306274s
Pipeline stage ISVCDeployer completed in 78.08s
Running pipeline stage StressChecker
Received healthy response to inference request in 1.8206713199615479s
Received healthy response to inference request in 0.7622551918029785s
Received healthy response to inference request in 0.5936846733093262s
Received healthy response to inference request in 15.398578882217407s
Received healthy response to inference request in 0.6591622829437256s
5 requests
0 failed requests
5th percentile: 0.6067801952362061
10th percentile: 0.619875717163086
20th percentile: 0.6460667610168457
30th percentile: 0.6797808647155762
40th percentile: 0.7210180282592773
50th percentile: 0.7622551918029785
60th percentile: 1.1856216430664062
70th percentile: 1.6089880943298338
80th percentile: 4.536252832412722
90th percentile: 9.967415857315064
95th percentile: 12.682997369766234
99th percentile: 14.855462579727172
mean time: 3.846870470046997
Pipeline stage StressChecker completed in 20.03s
Running pipeline stage DaemonicModelEvalScorer
Pipeline stage DaemonicModelEvalScorer completed in 0.04s
Running pipeline stage DaemonicSafetyScorer
Running M-Eval for topic stay_in_character
Pipeline stage DaemonicSafetyScorer completed in 0.05s
M-Eval Dataset for topic stay_in_character is loaded
mistralai-mixtral-8x7b-_3473_v11 status is now deployed due to DeploymentManager action
mistralai-mixtral-8x7b-_3473_v11 status is now inactive due to auto deactivation removed underperforming models
admin requested tearing down of mistralai-mixtral-8x7b-_3473_v11
Running pipeline stage ISVCDeleter
Checking if service mistralai-mixtral-8x7b-3473-v11 is running
Tearing down inference service mistralai-mixtral-8x7b-3473-v11
Toredown service mistralai-mixtral-8x7b-3473-v11
Pipeline stage ISVCDeleter completed in 4.04s
Running pipeline stage MKMLModelDeleter
Cleaning model data from S3
Cleaning model data from model cache
Deleting key mistralai-mixtral-8x7b-3473-v11/config.json from bucket guanaco-mkml-models
Deleting key mistralai-mixtral-8x7b-3473-v11/flywheel_model.0.safetensors from bucket guanaco-mkml-models
Deleting key mistralai-mixtral-8x7b-3473-v11/flywheel_model.1.safetensors from bucket guanaco-mkml-models
Deleting key mistralai-mixtral-8x7b-3473-v11/flywheel_model.2.safetensors from bucket guanaco-mkml-models
Deleting key mistralai-mixtral-8x7b-3473-v11/flywheel_model.3.safetensors from bucket guanaco-mkml-models
Deleting key mistralai-mixtral-8x7b-3473-v11/special_tokens_map.json from bucket guanaco-mkml-models
Deleting key mistralai-mixtral-8x7b-3473-v11/tokenizer.json from bucket guanaco-mkml-models
Deleting key mistralai-mixtral-8x7b-3473-v11/tokenizer.model from bucket guanaco-mkml-models
Deleting key mistralai-mixtral-8x7b-3473-v11/tokenizer_config.json from bucket guanaco-mkml-models
Cleaning model data from model cache
Deleting key mistralai-mixtral-8x7b-3473-v11_reward/config.json from bucket guanaco-reward-models
Deleting key mistralai-mixtral-8x7b-3473-v11_reward/merges.txt from bucket guanaco-reward-models
Deleting key mistralai-mixtral-8x7b-3473-v11_reward/reward.tensors from bucket guanaco-reward-models
Deleting key mistralai-mixtral-8x7b-3473-v11_reward/special_tokens_map.json from bucket guanaco-reward-models
Deleting key mistralai-mixtral-8x7b-3473-v11_reward/tokenizer.json from bucket guanaco-reward-models
Deleting key mistralai-mixtral-8x7b-3473-v11_reward/tokenizer_config.json from bucket guanaco-reward-models
Deleting key mistralai-mixtral-8x7b-3473-v11_reward/vocab.json from bucket guanaco-reward-models
Pipeline stage MKMLModelDeleter completed in 6.22s
mistralai-mixtral-8x7b-_3473_v11 status is now torndown due to DeploymentManager action