developer_uid: NischayDnk
submission_id: chaiml-mistral32-deepse_89261_v7
model_name: chaiml-mistral32-deepse_89261_v7
model_group: ChaiML/mistral32_deepsee
status: torndown
timestamp: 2025-07-25T00:03:28+00:00
num_battles: 6178
num_wins: 3370
celo_rating: 1312.68
family_friendly_score: 0.4972
family_friendly_standard_error: 0.0070709569366529165
submission_type: basic
model_repo: ChaiML/mistral32_deepseek_cpos3_cpos21322_sfts11306_seq96_exp6
model_architecture: MistralForCausalLM
model_num_parameters: 24096691200.0
best_of: 8
max_input_tokens: 1024
max_output_tokens: 64
reward_model: default
latencies: [{'batch_size': 1, 'throughput': 0.5517255709183, 'latency_mean': 1.8123756861686706, 'latency_p50': 1.814539909362793, 'latency_p90': 2.0003203630447386}, {'batch_size': 3, 'throughput': 1.0919669686542268, 'latency_mean': 2.7309290063381195, 'latency_p50': 2.720894694328308, 'latency_p90': 3.0345752000808717}, {'batch_size': 5, 'throughput': 1.3701543051064393, 'latency_mean': 3.622377506494522, 'latency_p50': 3.666927456855774, 'latency_p90': 3.99708034992218}, {'batch_size': 6, 'throughput': 1.4955711846392794, 'latency_mean': 3.9855480098724367, 'latency_p50': 3.9629576206207275, 'latency_p90': 4.4643680334091185}, {'batch_size': 8, 'throughput': 1.6180550685689674, 'latency_mean': 4.906414461135864, 'latency_p50': 4.880770921707153, 'latency_p90': 5.5510499477386475}, {'batch_size': 10, 'throughput': 1.6931560037729534, 'latency_mean': 5.854788485765457, 'latency_p50': 5.866108059883118, 'latency_p90': 6.552687072753907}]
gpu_counts: {'NVIDIA A100-SXM4-80GB': 1}
display_name: chaiml-mistral32-deepse_89261_v7
is_internal_developer: False
language_model: ChaiML/mistral32_deepseek_cpos3_cpos21322_sfts11306_seq96_exp6
model_size: 24B
ranking_group: single
throughput_3p7s: 1.41
us_pacific_date: 2025-07-24
win_ratio: 0.5454839753965685
generation_params: {'temperature': 0.7, 'top_p': 0.95, 'min_p': 0.025, 'top_k': 80, 'presence_penalty': 0.4, 'frequency_penalty': 0.4, 'stopping_words': ['</s>', '###', '<|im_start|>', '<|im_end|>', '\n'], 'max_input_tokens': 1024, 'best_of': 8, 'max_output_tokens': 64}
formatter: {'memory_template': '<|system|>Family Friendly{memory}\n', 'prompt_template': '', 'bot_template': '<|im_start|>assistant\n{message}<|im_end|>\n', 'user_template': '<|im_start|>user\nYou:{message}<|im_end|>\n', 'response_template': '<|im_start|>assistant\n', 'truncate_by_message': True}
Resubmit model
Shutdown handler not registered because Python interpreter is not running in the main thread
run pipeline %s
run pipeline stage %s
Running pipeline stage MKMLizer
Starting job with name chaiml-mistral32-deepse-89261-v7-mkmlizer
Waiting for job on chaiml-mistral32-deepse-89261-v7-mkmlizer to finish
chaiml-mistral32-deepse-89261-v7-mkmlizer: ╔═════════════════════════════════════════════════════════════════════╗
chaiml-mistral32-deepse-89261-v7-mkmlizer: ║                                                                     ║
chaiml-mistral32-deepse-89261-v7-mkmlizer: ║   ██████   ██████  █████   ████  ████                               ║
chaiml-mistral32-deepse-89261-v7-mkmlizer: ║  ░░██████ ██████  ░░███   ███░  ░░███                               ║
chaiml-mistral32-deepse-89261-v7-mkmlizer: ║   ░███░█████░███   ░███  ███     ░███                               ║
chaiml-mistral32-deepse-89261-v7-mkmlizer: ║   ░███░░███ ░███   ░███████      ░███                               ║
chaiml-mistral32-deepse-89261-v7-mkmlizer: ║   ░███ ░░░  ░███   ░███░░███     ░███                               ║
chaiml-mistral32-deepse-89261-v7-mkmlizer: ║   ░███      ░███   ░███ ░░███    ░███                               ║
chaiml-mistral32-deepse-89261-v7-mkmlizer: ║   █████     █████  █████ ░░████  █████                              ║
chaiml-mistral32-deepse-89261-v7-mkmlizer: ║  ░░░░░     ░░░░░  ░░░░░   ░░░░  ░░░░░                               ║
chaiml-mistral32-deepse-89261-v7-mkmlizer: ║                                                                     ║
chaiml-mistral32-deepse-89261-v7-mkmlizer: ║  Version: 0.29.15                                                   ║
chaiml-mistral32-deepse-89261-v7-mkmlizer: ║  Features: FLYWHEEL, CUDA                                           ║
chaiml-mistral32-deepse-89261-v7-mkmlizer: ║  Copyright 2023-2025 MK ONE TECHNOLOGIES Inc.                       ║
chaiml-mistral32-deepse-89261-v7-mkmlizer: ║  https://mk1.ai                                                     ║
chaiml-mistral32-deepse-89261-v7-mkmlizer: ║                                                                     ║
chaiml-mistral32-deepse-89261-v7-mkmlizer: ║  The license key for the current software has been verified as      ║
chaiml-mistral32-deepse-89261-v7-mkmlizer: ║  belonging to:                                                      ║
chaiml-mistral32-deepse-89261-v7-mkmlizer: ║                                                                     ║
chaiml-mistral32-deepse-89261-v7-mkmlizer: ║  Chai Research Corp.                                                ║
chaiml-mistral32-deepse-89261-v7-mkmlizer: ║  Account ID: 7997a29f-0ceb-4cc7-9adf-840c57b4ae6f                   ║
chaiml-mistral32-deepse-89261-v7-mkmlizer: ║  Expiration: 2028-03-31 23:59:59                                    ║
chaiml-mistral32-deepse-89261-v7-mkmlizer: ║                                                                     ║
chaiml-mistral32-deepse-89261-v7-mkmlizer: ╚═════════════════════════════════════════════════════════════════════╝
Failed to get response for submission chaiml-next-door-annoyi_15417_v1: ('http://chaiml-next-door-annoyi-15417-v1-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', '')
chaiml-mistral32-deepse-89261-v7-mkmlizer: Downloaded to shared memory in 57.436s
chaiml-mistral32-deepse-89261-v7-mkmlizer: Checking if ChaiML/mistral32_deepseek_cpos3_cpos21322_sfts11306_seq96_exp6 already exists in ChaiML
chaiml-mistral32-deepse-89261-v7-mkmlizer: quantizing model to /dev/shm/model_cache, profile:q4, folder:/tmp/tmpylk10gpo, device:0
chaiml-mistral32-deepse-89261-v7-mkmlizer: Saving flywheel model at /dev/shm/model_cache
Failed to get response for submission chaiml-next-door-annoyi_15417_v1: ('http://chaiml-next-door-annoyi-15417-v1-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', '')
Failed to get response for submission chaiml-next-door-annoyi_15417_v1: ('http://chaiml-next-door-annoyi-15417-v1-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', '')
chaiml-mistral32-deepse-89261-v7-mkmlizer: quantized model in 320.799s
chaiml-mistral32-deepse-89261-v7-mkmlizer: Processed model ChaiML/mistral32_deepseek_cpos3_cpos21322_sfts11306_seq96_exp6 in 378.236s
chaiml-mistral32-deepse-89261-v7-mkmlizer: creating bucket guanaco-mkml-models
chaiml-mistral32-deepse-89261-v7-mkmlizer: Bucket 's3://guanaco-mkml-models/' created
chaiml-mistral32-deepse-89261-v7-mkmlizer: uploading /dev/shm/model_cache to s3://guanaco-mkml-models/chaiml-mistral32-deepse-89261-v7/nvidia
chaiml-mistral32-deepse-89261-v7-mkmlizer: cp /dev/shm/model_cache/config.json s3://guanaco-mkml-models/chaiml-mistral32-deepse-89261-v7/nvidia/config.json
chaiml-mistral32-deepse-89261-v7-mkmlizer: cp /dev/shm/model_cache/special_tokens_map.json s3://guanaco-mkml-models/chaiml-mistral32-deepse-89261-v7/nvidia/special_tokens_map.json
chaiml-mistral32-deepse-89261-v7-mkmlizer: cp /dev/shm/model_cache/tokenizer_config.json s3://guanaco-mkml-models/chaiml-mistral32-deepse-89261-v7/nvidia/tokenizer_config.json
chaiml-mistral32-deepse-89261-v7-mkmlizer: cp /dev/shm/model_cache/tokenizer.json s3://guanaco-mkml-models/chaiml-mistral32-deepse-89261-v7/nvidia/tokenizer.json
chaiml-mistral32-deepse-89261-v7-mkmlizer: cp /dev/shm/model_cache/flywheel_model.1.safetensors s3://guanaco-mkml-models/chaiml-mistral32-deepse-89261-v7/nvidia/flywheel_model.1.safetensors
chaiml-mistral32-deepse-89261-v7-mkmlizer: cp /dev/shm/model_cache/flywheel_model.0.safetensors s3://guanaco-mkml-models/chaiml-mistral32-deepse-89261-v7/nvidia/flywheel_model.0.safetensors
chaiml-mistral32-deepse-89261-v7-mkmlizer: 
Loading 0:   0%|          | 0/363 [00:00<?, ?it/s]
Loading 0:   1%|          | 2/363 [00:00<00:35, 10.12it/s]
Loading 0:   1%|          | 4/363 [00:02<04:09,  1.44it/s]
Loading 0:   1%|▏         | 5/363 [00:04<06:48,  1.14s/it]
Loading 0:   2%|▏         | 6/363 [00:06<08:51,  1.49s/it]
Loading 0:   2%|▏         | 9/363 [00:07<04:15,  1.39it/s]
Loading 0:   3%|▎         | 10/363 [00:07<03:45,  1.57it/s]
Loading 0:   3%|▎         | 11/363 [00:07<03:04,  1.90it/s]
Loading 0:   4%|▎         | 13/363 [00:09<04:23,  1.33it/s]
Loading 0:   4%|▍         | 14/363 [00:12<06:14,  1.07s/it]
Loading 0:   4%|▍         | 15/363 [00:14<07:56,  1.37s/it]
Loading 0:   5%|▍         | 18/363 [00:14<04:13,  1.36it/s]
Loading 0:   5%|▌         | 19/363 [00:15<03:42,  1.54it/s]
Loading 0:   6%|▌         | 20/363 [00:15<03:04,  1.86it/s]
Loading 0:   6%|▌         | 22/363 [00:17<04:29,  1.27it/s]
Loading 0:   6%|▋         | 23/363 [00:19<06:11,  1.09s/it]
Loading 0:   7%|▋         | 24/363 [00:22<07:46,  1.38s/it]
Loading 0:   7%|▋         | 27/363 [00:22<04:09,  1.34it/s]
Loading 0:   8%|▊         | 28/363 [00:22<03:39,  1.53it/s]
Loading 0:   8%|▊         | 29/363 [00:22<03:02,  1.83it/s]
Loading 0:   9%|▊         | 31/363 [00:25<04:11,  1.32it/s]
Loading 0:   9%|▉         | 32/363 [00:27<05:54,  1.07s/it]
Loading 0:   9%|▉         | 33/363 [00:29<07:28,  1.36s/it]
Loading 0:  10%|▉         | 36/363 [00:29<04:00,  1.36it/s]
Loading 0:  10%|█         | 37/363 [00:30<03:31,  1.54it/s]
Loading 0:  10%|█         | 38/363 [00:30<02:55,  1.86it/s]
Loading 0:  11%|█         | 40/363 [00:32<04:03,  1.32it/s]
Loading 0:  11%|█▏        | 41/363 [00:34<05:43,  1.07s/it]
Loading 0:  12%|█▏        | 42/363 [00:37<07:14,  1.35s/it]
Loading 0:  12%|█▏        | 45/363 [00:37<03:53,  1.36it/s]
Loading 0:  13%|█▎        | 46/363 [00:37<03:24,  1.55it/s]
Loading 0:  13%|█▎        | 47/363 [00:37<02:50,  1.86it/s]
Loading 0:  13%|█▎        | 49/363 [00:40<03:56,  1.33it/s]
Loading 0:  14%|█▍        | 50/363 [00:42<05:33,  1.07s/it]
Loading 0:  14%|█▍        | 51/363 [00:44<07:03,  1.36s/it]
Loading 0:  15%|█▍        | 54/363 [00:44<03:47,  1.36it/s]
Loading 0:  15%|█▌        | 55/363 [00:45<03:19,  1.54it/s]
Loading 0:  15%|█▌        | 56/363 [00:45<02:45,  1.86it/s]
Loading 0:  16%|█▌        | 58/363 [00:47<03:51,  1.32it/s]
Loading 0:  16%|█▋        | 59/363 [00:50<05:37,  1.11s/it]
Loading 0:  17%|█▋        | 60/363 [00:52<07:00,  1.39s/it]
Loading 0:  17%|█▋        | 63/363 [00:52<03:45,  1.33it/s]
Loading 0:  18%|█▊        | 64/363 [00:52<03:17,  1.51it/s]
Loading 0:  18%|█▊        | 65/363 [00:53<02:43,  1.82it/s]
Loading 0:  18%|█▊        | 67/363 [00:55<03:46,  1.30it/s]
Loading 0:  19%|█▊        | 68/363 [00:57<05:17,  1.08s/it]
Loading 0:  19%|█▉        | 69/363 [00:59<06:41,  1.36s/it]
Loading 0:  20%|█▉        | 72/363 [01:00<03:35,  1.35it/s]
Loading 0:  20%|██        | 73/363 [01:00<03:08,  1.54it/s]
Loading 0:  20%|██        | 74/363 [01:00<02:36,  1.85it/s]
Loading 0:  21%|██        | 76/363 [01:02<03:37,  1.32it/s]
Loading 0:  21%|██        | 77/363 [01:05<05:05,  1.07s/it]
Loading 0:  21%|██▏       | 78/363 [01:07<06:27,  1.36s/it]
Loading 0:  22%|██▏       | 81/363 [01:07<03:27,  1.36it/s]
Loading 0:  23%|██▎       | 82/363 [01:08<03:02,  1.54it/s]
Loading 0:  23%|██▎       | 83/363 [01:08<02:31,  1.85it/s]
Loading 0:  23%|██▎       | 85/363 [01:10<03:30,  1.32it/s]
Loading 0:  24%|██▎       | 86/363 [01:12<04:55,  1.07s/it]
Loading 0:  24%|██▍       | 87/363 [01:14<06:14,  1.36s/it]
Loading 0:  25%|██▍       | 90/363 [01:15<03:20,  1.36it/s]
Loading 0:  25%|██▌       | 91/363 [01:15<02:56,  1.54it/s]
Loading 0:  25%|██▌       | 92/363 [01:15<02:26,  1.85it/s]
Loading 0:  26%|██▌       | 94/363 [01:17<03:23,  1.32it/s]
Loading 0:  26%|██▌       | 95/363 [01:20<04:46,  1.07s/it]
Loading 0:  26%|██▋       | 96/363 [01:22<06:12,  1.40s/it]
Loading 0:  27%|██▋       | 99/363 [01:22<03:19,  1.32it/s]
Loading 0:  28%|██▊       | 100/363 [01:23<02:54,  1.51it/s]
Loading 0:  28%|██▊       | 101/363 [01:23<02:24,  1.81it/s]
Loading 0:  28%|██▊       | 103/363 [01:25<03:19,  1.31it/s]
Loading 0:  29%|██▊       | 104/363 [01:27<04:39,  1.08s/it]
Loading 0:  29%|██▉       | 105/363 [01:30<05:52,  1.37s/it]
Loading 0:  30%|██▉       | 108/363 [01:30<03:08,  1.35it/s]
Loading 0:  30%|███       | 109/363 [01:30<02:45,  1.53it/s]
Loading 0:  30%|███       | 110/363 [01:30<02:16,  1.85it/s]
Loading 0:  31%|███       | 112/363 [01:33<03:10,  1.32it/s]
Loading 0:  31%|███       | 113/363 [01:35<04:27,  1.07s/it]
Loading 0:  31%|███▏      | 114/363 [01:37<05:39,  1.36s/it]
Loading 0:  32%|███▏      | 117/363 [01:37<03:01,  1.35it/s]
Loading 0:  33%|███▎      | 118/363 [01:38<02:39,  1.54it/s]
Loading 0:  33%|███▎      | 119/363 [01:38<02:12,  1.84it/s]
Loading 0:  33%|███▎      | 121/363 [01:40<03:02,  1.32it/s]
Loading 0:  34%|███▎      | 122/363 [01:42<04:17,  1.07s/it]
Loading 0:  34%|███▍      | 123/363 [01:45<05:25,  1.35s/it]
Loading 0:  35%|███▍      | 126/363 [01:45<02:53,  1.36it/s]
Loading 0:  35%|███▍      | 127/363 [01:45<02:32,  1.55it/s]
Loading 0:  35%|███▌      | 128/363 [01:45<02:06,  1.86it/s]
Loading 0:  36%|███▌      | 130/363 [01:48<02:56,  1.32it/s]
Loading 0:  36%|███▌      | 131/363 [01:50<04:08,  1.07s/it]
Loading 0:  36%|███▋      | 132/363 [01:52<05:13,  1.36s/it]
Loading 0:  37%|███▋      | 135/363 [01:52<02:47,  1.36it/s]
Loading 0:  37%|███▋      | 136/363 [01:53<02:27,  1.54it/s]
Loading 0:  38%|███▊      | 137/363 [01:53<02:01,  1.85it/s]
Loading 0:  38%|███▊      | 139/363 [01:55<02:56,  1.27it/s]
Loading 0:  39%|███▊      | 140/363 [01:57<04:03,  1.09s/it]
Loading 0:  39%|███▉      | 141/363 [02:00<05:05,  1.38s/it]
Loading 0:  40%|███▉      | 144/363 [02:00<02:42,  1.34it/s]
Loading 0:  40%|███▉      | 145/363 [02:00<02:22,  1.53it/s]
Loading 0:  40%|████      | 146/363 [02:01<01:58,  1.84it/s]
Loading 0:  41%|████      | 148/363 [02:03<02:42,  1.32it/s]
Loading 0:  41%|████      | 149/363 [02:05<03:48,  1.07s/it]
Loading 0:  41%|████▏     | 150/363 [02:07<04:49,  1.36s/it]
Loading 0:  42%|████▏     | 153/363 [02:08<02:34,  1.36it/s]
Loading 0:  42%|████▏     | 154/363 [02:08<02:15,  1.54it/s]
Loading 0:  43%|████▎     | 155/363 [02:08<01:52,  1.85it/s]
Loading 0:  43%|████▎     | 157/363 [02:10<02:35,  1.32it/s]
Loading 0:  44%|████▎     | 158/363 [02:12<03:38,  1.07s/it]
Loading 0:  44%|████▍     | 159/363 [02:15<04:35,  1.35s/it]
Loading 0:  45%|████▍     | 162/363 [02:15<02:27,  1.37it/s]
Loading 0:  45%|████▍     | 163/363 [02:15<02:09,  1.55it/s]
Loading 0:  45%|████▌     | 164/363 [02:16<01:46,  1.86it/s]
Loading 0:  46%|████▌     | 166/363 [02:18<02:28,  1.33it/s]
Loading 0:  46%|████▌     | 167/363 [02:20<03:28,  1.06s/it]
Loading 0:  46%|████▋     | 168/363 [02:22<04:22,  1.35s/it]
Loading 0:  47%|████▋     | 171/363 [02:23<02:20,  1.37it/s]
Loading 0:  47%|████▋     | 172/363 [02:23<02:03,  1.55it/s]
Loading 0:  48%|████▊     | 173/363 [02:23<01:41,  1.87it/s]
Loading 0:  48%|████▊     | 175/363 [02:25<02:21,  1.33it/s]
Loading 0:  48%|████▊     | 176/363 [02:28<03:28,  1.11s/it]
Loading 0:  49%|████▉     | 177/363 [02:30<04:19,  1.39s/it]
Loading 0:  50%|████▉     | 180/363 [02:30<02:17,  1.33it/s]
Loading 0:  50%|████▉     | 181/363 [02:31<02:00,  1.51it/s]
Loading 0:  50%|█████     | 182/363 [02:31<01:39,  1.81it/s]
Loading 0:  51%|█████     | 184/363 [02:33<02:15,  1.32it/s]
Loading 0:  51%|█████     | 185/363 [02:35<03:09,  1.06s/it]
Loading 0:  51%|█████     | 186/363 [02:37<03:57,  1.34s/it]
Loading 0:  52%|█████▏    | 189/363 [02:38<02:06,  1.37it/s]
Loading 0:  52%|█████▏    | 190/363 [02:38<01:50,  1.56it/s]
Loading 0:  53%|█████▎    | 191/363 [02:38<01:31,  1.88it/s]
Loading 0:  53%|█████▎    | 193/363 [02:40<02:06,  1.34it/s]
Loading 0:  53%|█████▎    | 194/363 [02:42<02:57,  1.05s/it]
Loading 0:  54%|█████▎    | 195/363 [02:45<03:44,  1.34s/it]
Loading 0:  55%|█████▍    | 198/363 [02:45<01:59,  1.38it/s]
Loading 0:  55%|█████▍    | 199/363 [02:45<01:44,  1.57it/s]
Loading 0:  55%|█████▌    | 200/363 [02:45<01:26,  1.88it/s]
Loading 0:  56%|█████▌    | 202/363 [02:48<02:00,  1.34it/s]
Loading 0:  56%|█████▌    | 203/363 [02:50<02:48,  1.05s/it]
Loading 0:  56%|█████▌    | 204/363 [02:52<03:32,  1.33s/it]
Loading 0:  57%|█████▋    | 207/363 [02:52<01:52,  1.38it/s]
Loading 0:  57%|█████▋    | 208/363 [02:53<01:38,  1.57it/s]
Loading 0:  58%|█████▊    | 209/363 [02:53<01:21,  1.89it/s]
Loading 0:  58%|█████▊    | 211/363 [02:55<01:52,  1.35it/s]
Loading 0:  58%|█████▊    | 212/363 [02:57<02:38,  1.05s/it]
Loading 0:  59%|█████▊    | 213/363 [03:00<03:27,  1.38s/it]
Loading 0:  60%|█████▉    | 216/363 [03:00<01:49,  1.34it/s]
Loading 0:  60%|█████▉    | 217/363 [03:00<01:35,  1.52it/s]
Loading 0:  60%|██████    | 218/363 [03:00<01:19,  1.83it/s]
Loading 0:  61%|██████    | 220/363 [03:03<01:47,  1.33it/s]
Loading 0:  61%|██████    | 221/363 [03:05<02:30,  1.06s/it]
Loading 0:  61%|██████    | 222/363 [03:07<03:08,  1.34s/it]
Loading 0:  62%|██████▏   | 225/363 [03:07<01:40,  1.38it/s]
Loading 0:  62%|██████▏   | 226/363 [03:08<01:27,  1.57it/s]
Loading 0:  63%|██████▎   | 227/363 [03:08<01:12,  1.89it/s]
Loading 0:  63%|██████▎   | 229/363 [03:10<01:39,  1.35it/s]
Loading 0:  63%|██████▎   | 230/363 [03:12<02:19,  1.05s/it]
Loading 0:  64%|██████▎   | 231/363 [03:29<10:28,  4.76s/it]
Loading 0:  64%|██████▍   | 234/363 [03:29<05:13,  2.43s/it]
Loading 0:  65%|██████▍   | 235/363 [03:30<04:17,  2.01s/it]
Loading 0:  65%|██████▌   | 236/363 [03:30<03:23,  1.60s/it]
Loading 0:  66%|██████▌   | 238/363 [03:32<02:52,  1.38s/it]
Loading 0:  66%|██████▌   | 239/363 [03:34<03:07,  1.52s/it]
Loading 0:  66%|██████▌   | 240/363 [03:36<03:27,  1.69s/it]
Loading 0:  67%|██████▋   | 243/363 [03:36<01:47,  1.12it/s]
Loading 0:  67%|██████▋   | 244/363 [03:37<01:31,  1.30it/s]
Loading 0:  67%|██████▋   | 245/363 [03:37<01:15,  1.57it/s]
Loading 0:  68%|██████▊   | 247/363 [03:39<01:33,  1.24it/s]
Loading 0:  68%|██████▊   | 248/363 [03:41<02:06,  1.10s/it]
Loading 0:  69%|██████▊   | 249/363 [03:43<02:35,  1.37s/it]
Loading 0:  69%|██████▉   | 252/363 [03:44<01:21,  1.36it/s]
Loading 0:  70%|██████▉   | 253/363 [03:44<01:10,  1.56it/s]
Loading 0:  70%|██████▉   | 254/363 [03:44<00:57,  1.88it/s]
Loading 0:  71%|███████   | 256/363 [03:46<01:23,  1.29it/s]
Loading 0:  71%|███████   | 257/363 [03:49<01:53,  1.07s/it]
Loading 0:  71%|███████   | 258/363 [03:51<02:22,  1.35s/it]
Loading 0:  72%|███████▏  | 261/363 [03:51<01:14,  1.38it/s]
Loading 0:  72%|███████▏  | 262/363 [03:51<01:03,  1.58it/s]
Loading 0:  72%|███████▏  | 263/363 [03:52<00:52,  1.90it/s]
Loading 0:  73%|███████▎  | 265/363 [03:54<01:09,  1.40it/s]
Loading 0:  73%|███████▎  | 266/363 [03:56<01:39,  1.03s/it]
Loading 0:  74%|███████▎  | 267/363 [03:58<02:06,  1.32s/it]
Loading 0:  74%|███████▍  | 270/363 [03:58<01:05,  1.41it/s]
Loading 0:  75%|███████▍  | 271/363 [03:59<00:57,  1.61it/s]
Loading 0:  75%|███████▍  | 272/363 [03:59<00:46,  1.94it/s]
Loading 0:  75%|███████▌  | 274/363 [04:01<01:05,  1.36it/s]
Loading 0:  76%|███████▌  | 275/363 [04:03<01:31,  1.04s/it]
Loading 0:  76%|███████▌  | 276/363 [04:05<01:55,  1.33s/it]
Loading 0:  77%|███████▋  | 279/363 [04:06<00:59,  1.40it/s]
Loading 0:  77%|███████▋  | 280/363 [04:06<00:51,  1.61it/s]
Loading 0:  77%|███████▋  | 281/363 [04:06<00:42,  1.93it/s]
Loading 0:  78%|███████▊  | 283/363 [04:08<00:58,  1.36it/s]
Loading 0:  78%|███████▊  | 284/363 [04:10<01:22,  1.04s/it]
Loading 0:  79%|███████▊  | 285/363 [04:13<01:43,  1.33s/it]
Loading 0:  79%|███████▉  | 288/363 [04:13<00:53,  1.40it/s]
Loading 0:  80%|███████▉  | 289/363 [04:13<00:46,  1.60it/s]
Loading 0:  80%|███████▉  | 290/363 [04:13<00:38,  1.92it/s]
Loading 0:  80%|████████  | 292/363 [04:15<00:52,  1.35it/s]
Loading 0:  81%|████████  | 293/363 [04:18<01:16,  1.09s/it]
Loading 0:  81%|████████  | 294/363 [04:20<01:33,  1.36s/it]
Loading 0:  82%|████████▏ | 297/363 [04:20<00:48,  1.37it/s]
Loading 0:  82%|████████▏ | 298/363 [04:21<00:41,  1.56it/s]
Loading 0:  82%|████████▏ | 299/363 [04:21<00:34,  1.87it/s]
Loading 0:  83%|████████▎ | 301/363 [04:23<00:46,  1.32it/s]
Loading 0:  83%|████████▎ | 302/363 [04:25<01:04,  1.06s/it]
Loading 0:  83%|████████▎ | 303/363 [04:28<01:20,  1.34s/it]
Loading 0:  84%|████████▍ | 306/363 [04:28<00:41,  1.38it/s]
Loading 0:  85%|████████▍ | 307/363 [04:28<00:35,  1.58it/s]
Loading 0:  85%|████████▍ | 308/363 [04:28<00:28,  1.90it/s]
Loading 0:  85%|████████▌ | 310/363 [04:30<00:39,  1.35it/s]
Loading 0:  86%|████████▌ | 311/363 [04:33<00:54,  1.05s/it]
Loading 0:  86%|████████▌ | 312/363 [04:35<01:07,  1.33s/it]
Loading 0:  87%|████████▋ | 315/363 [04:35<00:34,  1.39it/s]
Loading 0:  87%|████████▋ | 316/363 [04:35<00:29,  1.58it/s]
Loading 0:  87%|████████▋ | 317/363 [04:36<00:24,  1.90it/s]
Loading 0:  88%|████████▊ | 319/363 [04:38<00:32,  1.35it/s]
Loading 0:  88%|████████▊ | 320/363 [04:40<00:45,  1.05s/it]
Loading 0:  88%|████████▊ | 321/363 [04:42<00:55,  1.33s/it]
Loading 0:  89%|████████▉ | 324/363 [04:43<00:28,  1.39it/s]
Loading 0:  90%|████████▉ | 325/363 [04:43<00:24,  1.58it/s]
Loading 0:  90%|████████▉ | 326/363 [04:43<00:19,  1.89it/s]
Loading 0:  90%|█████████ | 328/363 [04:45<00:26,  1.34it/s]
Loading 0:  91%|█████████ | 329/363 [04:47<00:35,  1.06s/it]
Loading 0:  91%|█████████ | 330/363 [04:50<00:45,  1.38s/it]
Loading 0:  92%|█████████▏| 333/363 [04:50<00:22,  1.34it/s]
Loading 0:  92%|█████████▏| 334/363 [04:50<00:19,  1.52it/s]
Loading 0:  92%|█████████▏| 335/363 [04:51<00:15,  1.84it/s]
Loading 0:  93%|█████████▎| 337/363 [04:53<00:19,  1.32it/s]
Loading 0:  93%|█████████▎| 338/363 [04:55<00:26,  1.06s/it]
Loading 0:  93%|█████████▎| 339/363 [04:57<00:32,  1.34s/it]
Loading 0:  94%|█████████▍| 342/363 [04:58<00:15,  1.37it/s]
Loading 0:  94%|█████████▍| 343/363 [04:58<00:12,  1.56it/s]
Loading 0:  95%|█████████▍| 344/363 [04:58<00:10,  1.88it/s]
Loading 0:  95%|█████████▌| 346/363 [05:00<00:12,  1.34it/s]
Loading 0:  96%|█████████▌| 347/363 [05:02<00:16,  1.06s/it]
Loading 0:  96%|█████████▌| 348/363 [05:05<00:20,  1.34s/it]
Loading 0:  97%|█████████▋| 351/363 [05:05<00:08,  1.38it/s]
Loading 0:  97%|█████████▋| 352/363 [05:05<00:07,  1.56it/s]
Loading 0:  97%|█████████▋| 353/363 [05:05<00:05,  1.88it/s]
Loading 0:  98%|█████████▊| 355/363 [05:08<00:05,  1.34it/s]
Loading 0:  98%|█████████▊| 356/363 [05:10<00:07,  1.06s/it]
Loading 0:  98%|█████████▊| 357/363 [05:12<00:08,  1.34s/it]
Loading 0:  99%|█████████▉| 360/363 [05:12<00:02,  1.38it/s]
Loading 0:  99%|█████████▉| 361/363 [05:13<00:01,  1.56it/s]
Loading 0: 100%|█████████▉| 362/363 [05:13<00:00,  1.88it/s]
                                                            

Job chaiml-mistral32-deepse-89261-v7-mkmlizer completed after 407.78s with status: succeeded
Stopping job with name chaiml-mistral32-deepse-89261-v7-mkmlizer
Pipeline stage MKMLizer completed in 408.49s
run pipeline stage %s
Running pipeline stage MKMLTemplater
Pipeline stage MKMLTemplater completed in 0.21s
run pipeline stage %s
Running pipeline stage MKMLDeployer
Creating inference service chaiml-mistral32-deepse-89261-v7
Waiting for inference service chaiml-mistral32-deepse-89261-v7 to be ready
Inference service chaiml-mistral32-deepse-89261-v7 ready after 171.70196175575256s
Pipeline stage MKMLDeployer completed in 172.34s
run pipeline stage %s
Running pipeline stage StressChecker
HTTPConnectionPool(host='guanaco-submitter.guanaco-backend.k2.chaiverse.com', port=80): Read timed out. (read timeout=20)
Received unhealthy response to inference request!
Received healthy response to inference request in 2.636392116546631s
Received healthy response to inference request in 2.249394416809082s
Received healthy response to inference request in 2.282522678375244s
Received healthy response to inference request in 3.8764150142669678s
5 requests
1 failed requests
5th percentile: 2.2560200691223145
10th percentile: 2.262645721435547
20th percentile: 2.2758970260620117
30th percentile: 2.3532965660095213
40th percentile: 2.494844341278076
50th percentile: 2.636392116546631
60th percentile: 3.1324012756347654
70th percentile: 3.6284104347229
80th percentile: 7.157454109191898
90th percentile: 13.719532299041749
95th percentile: 17.000571393966673
99th percentile: 19.625402669906617
mean time: 6.265266942977905
%s, retrying in %s seconds...
Received healthy response to inference request in 3.085073471069336s
Received healthy response to inference request in 2.9380979537963867s
Received healthy response to inference request in 2.9495134353637695s
Received healthy response to inference request in 3.3049869537353516s
Received healthy response to inference request in 1.9340684413909912s
5 requests
0 failed requests
5th percentile: 2.1348743438720703
10th percentile: 2.3356802463531494
20th percentile: 2.7372920513153076
30th percentile: 2.9403810501098633
40th percentile: 2.9449472427368164
50th percentile: 2.9495134353637695
60th percentile: 3.003737449645996
70th percentile: 3.0579614639282227
80th percentile: 3.1290561676025392
90th percentile: 3.2170215606689454
95th percentile: 3.2610042572021483
99th percentile: 3.296190414428711
mean time: 2.842348051071167
Pipeline stage StressChecker completed in 49.28s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyTriggerPipeline
run_pipeline:run_in_cloud %s
starting trigger_guanaco_pipeline args=%s
triggered trigger_guanaco_pipeline args=%s
Pipeline stage OfflineFamilyFriendlyTriggerPipeline completed in 0.75s
run pipeline stage %s
Running pipeline stage TriggerMKMLProfilingPipeline
run_pipeline:run_in_cloud %s
starting trigger_guanaco_pipeline args=%s
triggered trigger_guanaco_pipeline args=%s
Pipeline stage TriggerMKMLProfilingPipeline completed in 0.77s
Shutdown handler de-registered
chaiml-mistral32-deepse_89261_v7 status is now deployed due to DeploymentManager action
Shutdown handler registered
run pipeline %s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyScorer
Evaluating %s Family Friendly Score with %s threads
Pipeline stage OfflineFamilyFriendlyScorer completed in 2946.13s
Shutdown handler de-registered
chaiml-mistral32-deepse_89261_v7 status is now inactive due to auto deactivation removed underperforming models
chaiml-mistral32-deepse_89261_v7 status is now torndown due to DeploymentManager action