developer_uid: NischayDnk
submission_id: mistralai-mistral-smal_78605_v28
model_name: mistralai-mistral-smal_78605_v28
model_group: mistralai/Mistral-Small-
status: torndown
timestamp: 2025-05-23T13:26:44+00:00
num_battles: 9205
num_wins: 4484
celo_rating: 1280.05
family_friendly_score: 0.6448
family_friendly_standard_error: 0.006768056737350834
submission_type: basic
model_repo: mistralai/Mistral-Small-24B-Instruct-2501
model_architecture: MistralForCausalLM
model_num_parameters: 24096691200.0
best_of: 8
max_input_tokens: 1024
max_output_tokens: 64
reward_model: default
display_name: mistralai-mistral-smal_78605_v28
is_internal_developer: False
language_model: mistralai/Mistral-Small-24B-Instruct-2501
model_size: 24B
ranking_group: single
us_pacific_date: 2025-05-23
win_ratio: 0.4871265616512765
generation_params: {'temperature': 0.45, 'top_p': 0.95, 'min_p': 0.025, 'top_k': 60, 'presence_penalty': 0.4, 'frequency_penalty': 0.4, 'stopping_words': ['<|im_end|>', '\n', '<|im_start|>'], 'max_input_tokens': 1024, 'best_of': 8, 'max_output_tokens': 64}
formatter: {'memory_template': '<|system|>Family Friendly{memory}\n', 'prompt_template': '', 'bot_template': '<|im_start|>assistant\n{message}<|im_end|>\n', 'user_template': '<|im_start|>user\nYou:{message}<|im_end|>\n', 'response_template': '<|im_start|>assistant\n', 'truncate_by_message': False}
Resubmit model
Shutdown handler not registered because Python interpreter is not running in the main thread
run pipeline %s
run pipeline stage %s
Running pipeline stage MKMLizer
Starting job with name mistralai-mistral-smal-78605-v28-mkmlizer
Waiting for job on mistralai-mistral-smal-78605-v28-mkmlizer to finish
mistralai-mistral-smal-78605-v28-mkmlizer: ╔═════════════════════════════════════════════════════════════════════╗
mistralai-mistral-smal-78605-v28-mkmlizer: ║     _____            __           __                                ║
mistralai-mistral-smal-78605-v28-mkmlizer: ║    / _/ /_ ___    __/ /  ___ ___ / /                                ║
mistralai-mistral-smal-78605-v28-mkmlizer: ║   / _/ / // / |/|/ / _ \/ -_) -_) /                                 ║
mistralai-mistral-smal-78605-v28-mkmlizer: ║  /_//_/\_, /|__,__/_//_/\__/\__/_/                                  ║
mistralai-mistral-smal-78605-v28-mkmlizer: ║       /___/                                                         ║
mistralai-mistral-smal-78605-v28-mkmlizer: ║                                                                     ║
mistralai-mistral-smal-78605-v28-mkmlizer: ║  Version: 0.12.8                                                    ║
mistralai-mistral-smal-78605-v28-mkmlizer: ║  Copyright 2023 MK ONE TECHNOLOGIES Inc.                            ║
mistralai-mistral-smal-78605-v28-mkmlizer: ║  https://mk1.ai                                                     ║
mistralai-mistral-smal-78605-v28-mkmlizer: ║                                                                     ║
mistralai-mistral-smal-78605-v28-mkmlizer: ║  The license key for the current software has been verified as      ║
mistralai-mistral-smal-78605-v28-mkmlizer: ║  belonging to:                                                      ║
mistralai-mistral-smal-78605-v28-mkmlizer: ║                                                                     ║
mistralai-mistral-smal-78605-v28-mkmlizer: ║  Chai Research Corp.                                                ║
mistralai-mistral-smal-78605-v28-mkmlizer: ║  Account ID: 7997a29f-0ceb-4cc7-9adf-840c57b4ae6f                   ║
mistralai-mistral-smal-78605-v28-mkmlizer: ║  Expiration: 2028-03-31 23:59:59                                    ║
mistralai-mistral-smal-78605-v28-mkmlizer: ║                                                                     ║
mistralai-mistral-smal-78605-v28-mkmlizer: ╚═════════════════════════════════════════════════════════════════════╝
mistralai-mistral-smal-78605-v28-mkmlizer: quantized model in 54.709s
mistralai-mistral-smal-78605-v28-mkmlizer: Processed model mistralai/Mistral-Small-24B-Instruct-2501 in 148.454s
mistralai-mistral-smal-78605-v28-mkmlizer: creating bucket guanaco-mkml-models
mistralai-mistral-smal-78605-v28-mkmlizer: Bucket 's3://guanaco-mkml-models/' created
mistralai-mistral-smal-78605-v28-mkmlizer: uploading /dev/shm/model_cache to s3://guanaco-mkml-models/mistralai-mistral-smal-78605-v28
mistralai-mistral-smal-78605-v28-mkmlizer: cp /dev/shm/model_cache/config.json s3://guanaco-mkml-models/mistralai-mistral-smal-78605-v28/config.json
mistralai-mistral-smal-78605-v28-mkmlizer: cp /dev/shm/model_cache/special_tokens_map.json s3://guanaco-mkml-models/mistralai-mistral-smal-78605-v28/special_tokens_map.json
mistralai-mistral-smal-78605-v28-mkmlizer: cp /dev/shm/model_cache/tokenizer_config.json s3://guanaco-mkml-models/mistralai-mistral-smal-78605-v28/tokenizer_config.json
mistralai-mistral-smal-78605-v28-mkmlizer: cp /dev/shm/model_cache/tokenizer.json s3://guanaco-mkml-models/mistralai-mistral-smal-78605-v28/tokenizer.json
mistralai-mistral-smal-78605-v28-mkmlizer: cp /dev/shm/model_cache/flywheel_model.1.safetensors s3://guanaco-mkml-models/mistralai-mistral-smal-78605-v28/flywheel_model.1.safetensors
mistralai-mistral-smal-78605-v28-mkmlizer: cp /dev/shm/model_cache/flywheel_model.0.safetensors s3://guanaco-mkml-models/mistralai-mistral-smal-78605-v28/flywheel_model.0.safetensors
mistralai-mistral-smal-78605-v28-mkmlizer: 
Loading 0:   0%|          | 0/363 [00:00<?, ?it/s]
Loading 0:   1%|▏         | 5/363 [00:00<00:17, 20.79it/s]
Loading 0:   3%|▎         | 12/363 [00:00<00:09, 35.59it/s]
Loading 0:   5%|▍         | 17/363 [00:00<00:10, 32.81it/s]
Loading 0:   6%|▌         | 21/363 [00:00<00:10, 33.21it/s]
Loading 0:   7%|▋         | 25/363 [00:00<00:11, 29.65it/s]
Loading 0:   9%|▉         | 32/363 [00:00<00:09, 34.98it/s]
Loading 0:  10%|▉         | 36/363 [00:01<00:15, 21.73it/s]
Loading 0:  11%|█         | 40/363 [00:01<00:13, 23.66it/s]
Loading 0:  12%|█▏        | 43/363 [00:01<00:13, 24.04it/s]
Loading 0:  13%|█▎        | 48/363 [00:01<00:11, 26.82it/s]
Loading 0:  14%|█▍        | 51/363 [00:01<00:12, 24.43it/s]
Loading 0:  16%|█▌        | 57/363 [00:02<00:10, 29.48it/s]
Loading 0:  17%|█▋        | 61/363 [00:02<00:10, 27.61it/s]
Loading 0:  18%|█▊        | 65/363 [00:02<00:10, 28.10it/s]
Loading 0:  19%|█▉        | 70/363 [00:02<00:11, 25.15it/s]
Loading 0:  20%|██        | 73/363 [00:02<00:13, 21.06it/s]
Loading 0:  22%|██▏       | 79/363 [00:02<00:10, 25.97it/s]
Loading 0:  23%|██▎       | 82/363 [00:03<00:11, 25.12it/s]
Loading 0:  24%|██▎       | 86/363 [00:03<00:10, 26.66it/s]
Loading 0:  25%|██▍       | 89/363 [00:03<00:10, 26.52it/s]
Loading 0:  25%|██▌       | 92/363 [00:03<00:12, 21.24it/s]
Loading 0:  27%|██▋       | 99/363 [00:03<00:09, 28.15it/s]
Loading 0:  28%|██▊       | 102/363 [00:03<00:10, 25.13it/s]
Loading 0:  29%|██▉       | 106/363 [00:03<00:09, 28.01it/s]
Loading 0:  30%|███       | 110/363 [00:04<00:10, 24.00it/s]
Loading 0:  31%|███       | 113/363 [00:04<00:12, 20.14it/s]
Loading 0:  33%|███▎      | 120/363 [00:04<00:08, 27.07it/s]
Loading 0:  34%|███▍      | 123/363 [00:04<00:09, 24.43it/s]
Loading 0:  36%|███▌      | 129/363 [00:04<00:08, 28.15it/s]
Loading 0:  36%|███▋      | 132/363 [00:05<00:09, 25.06it/s]
Loading 0:  38%|███▊      | 138/363 [00:05<00:07, 28.73it/s]
Loading 0:  39%|███▉      | 141/363 [00:05<00:08, 25.44it/s]
Loading 0:  40%|████      | 147/363 [00:05<00:06, 31.63it/s]
Loading 0:  42%|████▏     | 151/363 [00:05<00:09, 23.47it/s]
Loading 0:  42%|████▏     | 154/363 [00:05<00:09, 22.20it/s]
Loading 0:  43%|████▎     | 157/363 [00:06<00:08, 23.42it/s]
Loading 0:  44%|████▍     | 160/363 [00:06<00:08, 23.68it/s]
Loading 0:  45%|████▌     | 165/363 [00:06<00:07, 26.45it/s]
Loading 0:  46%|████▋     | 168/363 [00:06<00:08, 24.31it/s]
Loading 0:  48%|████▊     | 174/363 [00:06<00:06, 29.18it/s]
Loading 0:  49%|████▉     | 177/363 [00:06<00:07, 25.57it/s]
Loading 0:  50%|█████     | 182/363 [00:06<00:06, 27.88it/s]
Loading 0:  52%|█████▏    | 187/363 [00:07<00:06, 25.47it/s]
Loading 0:  52%|█████▏    | 190/363 [00:07<00:07, 23.31it/s]
Loading 0:  53%|█████▎    | 193/363 [00:07<00:07, 24.10it/s]
Loading 0:  54%|█████▍    | 196/363 [00:07<00:06, 24.43it/s]
Loading 0:  55%|█████▌    | 200/363 [00:22<00:06, 24.43it/s]
Loading 0:  55%|█████▌    | 201/363 [00:22<03:02,  1.13s/it]
Loading 0:  56%|█████▌    | 203/363 [00:22<02:31,  1.06it/s]
Loading 0:  57%|█████▋    | 208/363 [00:22<01:31,  1.70it/s]
Loading 0:  58%|█████▊    | 211/363 [00:22<01:10,  2.17it/s]
Loading 0:  59%|█████▉    | 214/363 [00:22<00:52,  2.83it/s]
Loading 0:  60%|██████    | 218/363 [00:23<00:36,  4.02it/s]
Loading 0:  61%|██████    | 221/363 [00:23<00:27,  5.13it/s]
Loading 0:  62%|██████▏   | 224/363 [00:23<00:22,  6.13it/s]
Loading 0:  63%|██████▎   | 228/363 [00:23<00:15,  8.61it/s]
Loading 0:  64%|██████▎   | 231/363 [00:23<00:13, 10.02it/s]
Loading 0:  65%|██████▌   | 237/363 [00:23<00:08, 14.75it/s]
Loading 0:  66%|██████▌   | 240/363 [00:23<00:07, 15.48it/s]
Loading 0:  68%|██████▊   | 246/363 [00:24<00:05, 20.53it/s]
Loading 0:  69%|██████▊   | 249/363 [00:24<00:05, 19.74it/s]
Loading 0:  70%|███████   | 255/363 [00:24<00:04, 24.47it/s]
Loading 0:  71%|███████   | 258/363 [00:24<00:04, 23.00it/s]
Loading 0:  73%|███████▎  | 264/363 [00:24<00:03, 29.77it/s]
Loading 0:  74%|███████▍  | 268/363 [00:24<00:04, 23.06it/s]
Loading 0:  75%|███████▍  | 271/363 [00:25<00:04, 21.91it/s]
Loading 0:  75%|███████▌  | 274/363 [00:25<00:03, 22.99it/s]
Loading 0:  76%|███████▋  | 277/363 [00:25<00:03, 23.39it/s]
Loading 0:  78%|███████▊  | 282/363 [00:25<00:03, 26.38it/s]
Loading 0:  79%|███████▊  | 285/363 [00:25<00:03, 23.55it/s]
Loading 0:  80%|████████  | 291/363 [00:25<00:02, 28.23it/s]
Loading 0:  81%|████████  | 294/363 [00:26<00:02, 24.63it/s]
Loading 0:  82%|████████▏ | 299/363 [00:26<00:02, 26.41it/s]
Loading 0:  84%|████████▎ | 304/363 [00:26<00:02, 23.65it/s]
Loading 0:  85%|████████▍ | 307/363 [00:26<00:02, 22.00it/s]
Loading 0:  85%|████████▌ | 310/363 [00:26<00:02, 22.76it/s]
Loading 0:  86%|████████▌ | 313/363 [00:26<00:02, 22.76it/s]
Loading 0:  88%|████████▊ | 318/363 [00:27<00:01, 25.81it/s]
Loading 0:  88%|████████▊ | 321/363 [00:27<00:01, 22.94it/s]
Loading 0:  90%|█████████ | 327/363 [00:27<00:01, 27.63it/s]
Loading 0:  91%|█████████ | 330/363 [00:27<00:01, 24.56it/s]
Loading 0:  92%|█████████▏| 335/363 [00:27<00:01, 26.71it/s]
Loading 0:  93%|█████████▎| 338/363 [00:27<00:00, 25.52it/s]
Loading 0:  94%|█████████▍| 341/363 [00:34<00:13,  1.68it/s]
Loading 0:  96%|█████████▌| 347/363 [00:34<00:05,  2.79it/s]
Loading 0:  96%|█████████▋| 350/363 [00:34<00:03,  3.51it/s]
Loading 0:  98%|█████████▊| 355/363 [00:35<00:01,  5.18it/s]
Loading 0:  99%|█████████▉| 359/363 [00:35<00:00,  6.70it/s]
                                                            

Job mistralai-mistral-smal-78605-v28-mkmlizer completed after 176.08s with status: succeeded
Stopping job with name mistralai-mistral-smal-78605-v28-mkmlizer
Pipeline stage MKMLizer completed in 176.63s
run pipeline stage %s
Running pipeline stage MKMLTemplater
Pipeline stage MKMLTemplater completed in 0.15s
run pipeline stage %s
Running pipeline stage MKMLDeployer
Creating inference service mistralai-mistral-smal-78605-v28
Waiting for inference service mistralai-mistral-smal-78605-v28 to be ready
Inference service mistralai-mistral-smal-78605-v28 ready after 160.69801664352417s
Pipeline stage MKMLDeployer completed in 161.28s
run pipeline stage %s
Running pipeline stage StressChecker
Received healthy response to inference request in 1.9692347049713135s
Received healthy response to inference request in 1.451563835144043s
Received healthy response to inference request in 2.031045913696289s
Received healthy response to inference request in 1.8941819667816162s
Received healthy response to inference request in 1.7741446495056152s
5 requests
0 failed requests
5th percentile: 1.5160799980163575
10th percentile: 1.5805961608886718
20th percentile: 1.7096284866333007
30th percentile: 1.7981521129608153
40th percentile: 1.8461670398712158
50th percentile: 1.8941819667816162
60th percentile: 1.924203062057495
70th percentile: 1.954224157333374
80th percentile: 1.9815969467163086
90th percentile: 2.006321430206299
95th percentile: 2.018683671951294
99th percentile: 2.02857346534729
mean time: 1.8240342140197754
Pipeline stage StressChecker completed in 10.48s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyTriggerPipeline
run_pipeline:run_in_cloud %s
starting trigger_guanaco_pipeline args=%s
triggered trigger_guanaco_pipeline args=%s
Pipeline stage OfflineFamilyFriendlyTriggerPipeline completed in 0.74s
run pipeline stage %s
Running pipeline stage TriggerMKMLProfilingPipeline
run_pipeline:run_in_cloud %s
starting trigger_guanaco_pipeline args=%s
triggered trigger_guanaco_pipeline args=%s
Pipeline stage TriggerMKMLProfilingPipeline completed in 0.71s
Shutdown handler de-registered
mistralai-mistral-smal_78605_v28 status is now deployed due to DeploymentManager action
Shutdown handler registered
run pipeline %s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyScorer
Evaluating %s Family Friendly Score with %s threads
Pipeline stage OfflineFamilyFriendlyScorer completed in 3066.31s
Shutdown handler de-registered
mistralai-mistral-smal_78605_v28 status is now inactive due to auto deactivation removed underperforming models
mistralai-mistral-smal_78605_v28 status is now torndown due to DeploymentManager action