chaiml-mistral31-trysim_80512

developer_uid: chai_backend_admin
submission_id: chaiml-mistral31-trysim_80512_v2
model_name: chaiml-mistral31-trysim_80512_v2
model_group: ChaiML/mistral31-trysimp
status: torndown
timestamp: 2025-04-21T19:17:59+00:00
num_battles: 9380
num_wins: 5292
celo_rating: 1333.93
family_friendly_score: 0.0
family_friendly_standard_error: 0.0
submission_type: basic
model_repo: ChaiML/mistral31-trysimponew-1350pref-v2
model_architecture: MistralForCausalLM
model_num_parameters: 24096691200.0
best_of: 8
max_input_tokens: 1024
max_output_tokens: 64
reward_model: default
display_name: chaiml-mistral31-trysim_80512_v2
is_internal_developer: True
language_model: ChaiML/mistral31-trysimponew-1350pref-v2
model_size: 24B
ranking_group: single
us_pacific_date: 2025-04-21
win_ratio: 0.564179104477612
generation_params: {'temperature': 0.5, 'top_p': 0.95, 'min_p': 0.025, 'top_k': 70, 'presence_penalty': 0.45, 'frequency_penalty': 0.45, 'stopping_words': ['<|im_start|>', '<|im_end|>', '\n'], 'max_input_tokens': 1024, 'best_of': 8, 'max_output_tokens': 64}
formatter: {'memory_template': '<|system|>Family Friendly{memory}\n', 'prompt_template': '', 'bot_template': '<|im_start|>assistant\n{message}<|im_end|>\n', 'user_template': '<|im_start|>user\nYou:{message}<|im_end|>\n', 'response_template': '<|im_start|>assistant\n', 'truncate_by_message': False}
Resubmit model
Shutdown handler not registered because Python interpreter is not running in the main thread
run pipeline %s
run pipeline stage %s
Running pipeline stage MKMLizer
Starting job with name chaiml-mistral31-trysim-80512-v2-mkmlizer
Waiting for job on chaiml-mistral31-trysim-80512-v2-mkmlizer to finish
chaiml-mistral31-trysim-80512-v2-mkmlizer: ╔═════════════════════════════════════════════════════════════════════╗
chaiml-mistral31-trysim-80512-v2-mkmlizer: ║     _____            __           __                                ║
chaiml-mistral31-trysim-80512-v2-mkmlizer: ║    / _/ /_ ___    __/ /  ___ ___ / /                                ║
chaiml-mistral31-trysim-80512-v2-mkmlizer: ║   / _/ / // / |/|/ / _ \/ -_) -_) /                                 ║
chaiml-mistral31-trysim-80512-v2-mkmlizer: ║  /_//_/\_, /|__,__/_//_/\__/\__/_/                                  ║
chaiml-mistral31-trysim-80512-v2-mkmlizer: ║       /___/                                                         ║
chaiml-mistral31-trysim-80512-v2-mkmlizer: ║                                                                     ║
chaiml-mistral31-trysim-80512-v2-mkmlizer: ║  Version: 0.12.8                                                    ║
chaiml-mistral31-trysim-80512-v2-mkmlizer: ║  Copyright 2023 MK ONE TECHNOLOGIES Inc.                            ║
chaiml-mistral31-trysim-80512-v2-mkmlizer: ║  https://mk1.ai                                                     ║
chaiml-mistral31-trysim-80512-v2-mkmlizer: ║                                                                     ║
chaiml-mistral31-trysim-80512-v2-mkmlizer: ║  The license key for the current software has been verified as      ║
chaiml-mistral31-trysim-80512-v2-mkmlizer: ║  belonging to:                                                      ║
chaiml-mistral31-trysim-80512-v2-mkmlizer: ║                                                                     ║
chaiml-mistral31-trysim-80512-v2-mkmlizer: ║  Chai Research Corp.                                                ║
chaiml-mistral31-trysim-80512-v2-mkmlizer: ║  Account ID: 7997a29f-0ceb-4cc7-9adf-840c57b4ae6f                   ║
chaiml-mistral31-trysim-80512-v2-mkmlizer: ║  Expiration: 2028-03-31 23:59:59                                    ║
chaiml-mistral31-trysim-80512-v2-mkmlizer: ║                                                                     ║
chaiml-mistral31-trysim-80512-v2-mkmlizer: ╚═════════════════════════════════════════════════════════════════════╝
chaiml-mistral31-trysim-80512-v2-mkmlizer: Downloaded to shared memory in 62.479s
chaiml-mistral31-trysim-80512-v2-mkmlizer: quantizing model to /dev/shm/model_cache, profile:s0, folder:/tmp/tmpoyj_ogy3, device:0
chaiml-mistral31-trysim-80512-v2-mkmlizer: Saving flywheel model at /dev/shm/model_cache
chaiml-mistral31-trysim-80512-v2-mkmlizer: quantized model in 52.748s
chaiml-mistral31-trysim-80512-v2-mkmlizer: Processed model ChaiML/mistral31-trysimponew-1350pref-v2 in 115.227s
chaiml-mistral31-trysim-80512-v2-mkmlizer: creating bucket guanaco-mkml-models
chaiml-mistral31-trysim-80512-v2-mkmlizer: Bucket 's3://guanaco-mkml-models/' created
chaiml-mistral31-trysim-80512-v2-mkmlizer: uploading /dev/shm/model_cache to s3://guanaco-mkml-models/chaiml-mistral31-trysim-80512-v2
chaiml-mistral31-trysim-80512-v2-mkmlizer: cp /dev/shm/model_cache/config.json s3://guanaco-mkml-models/chaiml-mistral31-trysim-80512-v2/config.json
chaiml-mistral31-trysim-80512-v2-mkmlizer: cp /dev/shm/model_cache/special_tokens_map.json s3://guanaco-mkml-models/chaiml-mistral31-trysim-80512-v2/special_tokens_map.json
chaiml-mistral31-trysim-80512-v2-mkmlizer: cp /dev/shm/model_cache/tokenizer_config.json s3://guanaco-mkml-models/chaiml-mistral31-trysim-80512-v2/tokenizer_config.json
chaiml-mistral31-trysim-80512-v2-mkmlizer: cp /dev/shm/model_cache/tokenizer.json s3://guanaco-mkml-models/chaiml-mistral31-trysim-80512-v2/tokenizer.json
chaiml-mistral31-trysim-80512-v2-mkmlizer: cp /dev/shm/model_cache/flywheel_model.1.safetensors s3://guanaco-mkml-models/chaiml-mistral31-trysim-80512-v2/flywheel_model.1.safetensors
chaiml-mistral31-trysim-80512-v2-mkmlizer: cp /dev/shm/model_cache/flywheel_model.0.safetensors s3://guanaco-mkml-models/chaiml-mistral31-trysim-80512-v2/flywheel_model.0.safetensors
chaiml-mistral31-trysim-80512-v2-mkmlizer: 
Loading 0:   0%|          | 0/363 [00:00<?, ?it/s]
Loading 0:   1%|▏         | 5/363 [00:00<00:15, 22.40it/s]
Loading 0:   3%|▎         | 12/363 [00:00<00:09, 38.68it/s]
Loading 0:   5%|▍         | 17/363 [00:00<00:10, 34.58it/s]
Loading 0:   6%|▌         | 21/363 [00:00<00:09, 35.53it/s]
Loading 0:   7%|▋         | 25/363 [00:00<00:10, 31.76it/s]
Loading 0:   9%|▉         | 32/363 [00:00<00:09, 36.60it/s]
Loading 0:  10%|▉         | 36/363 [00:01<00:14, 23.25it/s]
Loading 0:  11%|█         | 40/363 [00:01<00:12, 25.96it/s]
Loading 0:  12%|█▏        | 44/363 [00:01<00:11, 27.27it/s]
Loading 0:  13%|█▎        | 48/363 [00:01<00:10, 29.36it/s]
Loading 0:  14%|█▍        | 52/363 [00:01<00:11, 27.96it/s]
Loading 0:  16%|█▌        | 57/363 [00:01<00:09, 30.75it/s]
Loading 0:  17%|█▋        | 61/363 [00:02<00:10, 28.48it/s]
Loading 0:  18%|█▊        | 65/363 [00:02<00:10, 28.62it/s]
Loading 0:  19%|█▉        | 70/363 [00:02<00:11, 25.47it/s]
Loading 0:  20%|██        | 73/363 [00:02<00:13, 21.93it/s]
Loading 0:  22%|██▏       | 79/363 [00:02<00:10, 27.09it/s]
Loading 0:  23%|██▎       | 82/363 [00:02<00:10, 26.57it/s]
Loading 0:  24%|██▎       | 86/363 [00:03<00:09, 27.87it/s]
Loading 0:  25%|██▍       | 89/363 [00:03<00:09, 27.82it/s]
Loading 0:  25%|██▌       | 92/363 [00:03<00:12, 21.75it/s]
Loading 0:  27%|██▋       | 99/363 [00:03<00:09, 29.09it/s]
Loading 0:  28%|██▊       | 103/363 [00:03<00:09, 28.25it/s]
Loading 0:  29%|██▉       | 107/363 [00:03<00:10, 23.69it/s]
Loading 0:  31%|███       | 112/363 [00:04<00:09, 26.70it/s]
Loading 0:  32%|███▏      | 115/363 [00:04<00:09, 26.74it/s]
Loading 0:  33%|███▎      | 120/363 [00:04<00:08, 29.21it/s]
Loading 0:  34%|███▍      | 124/363 [00:04<00:08, 28.43it/s]
Loading 0:  36%|███▌      | 129/363 [00:04<00:07, 30.55it/s]
Loading 0:  37%|███▋      | 133/363 [00:04<00:08, 28.30it/s]
Loading 0:  38%|███▊      | 138/363 [00:04<00:07, 30.48it/s]
Loading 0:  39%|███▉      | 142/363 [00:05<00:07, 28.58it/s]
Loading 0:  40%|████      | 147/363 [00:05<00:06, 33.06it/s]
Loading 0:  42%|████▏     | 151/363 [00:05<00:08, 24.60it/s]
Loading 0:  42%|████▏     | 154/363 [00:05<00:09, 23.04it/s]
Loading 0:  43%|████▎     | 157/363 [00:05<00:08, 23.84it/s]
Loading 0:  44%|████▍     | 160/363 [00:05<00:08, 24.54it/s]
Loading 0:  45%|████▌     | 165/363 [00:05<00:07, 27.85it/s]
Loading 0:  46%|████▋     | 168/363 [00:06<00:07, 24.93it/s]
Loading 0:  48%|████▊     | 174/363 [00:06<00:06, 29.32it/s]
Loading 0:  49%|████▉     | 177/363 [00:06<00:06, 26.60it/s]
Loading 0:  50%|█████     | 182/363 [00:06<00:06, 29.23it/s]
Loading 0:  52%|█████▏    | 187/363 [00:06<00:06, 25.88it/s]
Loading 0:  52%|█████▏    | 190/363 [00:06<00:07, 22.94it/s]
Loading 0:  53%|█████▎    | 193/363 [00:07<00:07, 24.11it/s]
Loading 0:  54%|█████▍    | 196/363 [00:07<00:06, 24.46it/s]
Loading 0:  55%|█████▌    | 200/363 [00:21<00:06, 24.46it/s]
Loading 0:  55%|█████▌    | 201/363 [00:21<02:55,  1.09s/it]
Loading 0:  56%|█████▌    | 203/363 [00:21<02:25,  1.10it/s]
Loading 0:  57%|█████▋    | 208/363 [00:21<01:27,  1.78it/s]
Loading 0:  58%|█████▊    | 211/363 [00:21<01:07,  2.26it/s]
Loading 0:  59%|█████▉    | 214/363 [00:21<00:50,  2.95it/s]
Loading 0:  60%|██████    | 218/363 [00:21<00:34,  4.21it/s]
Loading 0:  61%|██████    | 221/363 [00:22<00:26,  5.36it/s]
Loading 0:  62%|██████▏   | 224/363 [00:22<00:21,  6.36it/s]
Loading 0:  63%|██████▎   | 229/363 [00:22<00:14,  9.40it/s]
Loading 0:  64%|██████▍   | 232/363 [00:22<00:11, 11.15it/s]
Loading 0:  65%|██████▌   | 237/363 [00:22<00:08, 14.89it/s]
Loading 0:  66%|██████▌   | 240/363 [00:22<00:07, 15.81it/s]
Loading 0:  68%|██████▊   | 246/363 [00:23<00:05, 21.04it/s]
Loading 0:  69%|██████▊   | 249/363 [00:23<00:05, 20.47it/s]
Loading 0:  70%|███████   | 255/363 [00:23<00:04, 25.54it/s]
Loading 0:  71%|███████▏  | 259/363 [00:23<00:04, 25.20it/s]
Loading 0:  73%|███████▎  | 264/363 [00:23<00:03, 29.88it/s]
Loading 0:  74%|███████▍  | 268/363 [00:23<00:04, 22.82it/s]
Loading 0:  75%|███████▍  | 271/363 [00:24<00:04, 21.89it/s]
Loading 0:  75%|███████▌  | 274/363 [00:24<00:03, 22.91it/s]
Loading 0:  76%|███████▋  | 277/363 [00:24<00:03, 23.64it/s]
Loading 0:  78%|███████▊  | 282/363 [00:24<00:03, 26.78it/s]
Loading 0:  79%|███████▊  | 285/363 [00:24<00:03, 24.35it/s]
Loading 0:  80%|████████  | 291/363 [00:24<00:02, 28.86it/s]
Loading 0:  81%|████████  | 294/363 [00:24<00:02, 25.62it/s]
Loading 0:  82%|████████▏ | 299/363 [00:25<00:02, 28.37it/s]
Loading 0:  84%|████████▎ | 304/363 [00:25<00:02, 25.24it/s]
Loading 0:  85%|████████▍ | 307/363 [00:25<00:02, 23.58it/s]
Loading 0:  85%|████████▌ | 310/363 [00:25<00:02, 24.64it/s]
Loading 0:  86%|████████▌ | 313/363 [00:25<00:02, 24.54it/s]
Loading 0:  88%|████████▊ | 318/363 [00:25<00:01, 27.18it/s]
Loading 0:  88%|████████▊ | 321/363 [00:26<00:01, 24.64it/s]
Loading 0:  90%|█████████ | 327/363 [00:26<00:01, 29.31it/s]
Loading 0:  91%|█████████ | 330/363 [00:26<00:01, 26.05it/s]
Loading 0:  92%|█████████▏| 335/363 [00:26<00:00, 28.06it/s]
Loading 0:  93%|█████████▎| 338/363 [00:26<00:00, 26.41it/s]
Loading 0:  94%|█████████▍| 341/363 [00:33<00:13,  1.69it/s]
Loading 0:  96%|█████████▌| 347/363 [00:33<00:05,  2.82it/s]
Loading 0:  96%|█████████▋| 350/363 [00:33<00:03,  3.54it/s]
Loading 0:  98%|█████████▊| 355/363 [00:33<00:01,  5.26it/s]
Loading 0:  99%|█████████▉| 359/363 [00:33<00:00,  6.84it/s]
                                                            
Job chaiml-mistral31-trysim-80512-v2-mkmlizer completed after 143.27s with status: succeeded
Stopping job with name chaiml-mistral31-trysim-80512-v2-mkmlizer
Pipeline stage MKMLizer completed in 144.82s
run pipeline stage %s
Running pipeline stage MKMLTemplater
Pipeline stage MKMLTemplater completed in 0.45s
run pipeline stage %s
Running pipeline stage MKMLDeployer
Creating inference service chaiml-mistral31-trysim-80512-v2
Waiting for inference service chaiml-mistral31-trysim-80512-v2 to be ready
Inference service chaiml-mistral31-trysim-80512-v2 ready after 131.79082131385803s
Pipeline stage MKMLDeployer completed in 133.61s
run pipeline stage %s
Running pipeline stage StressChecker
HTTP Request: %s %s "%s %d %s"
Received healthy response to inference request in 4.687318563461304s
HTTP Request: %s %s "%s %d %s"
Received healthy response to inference request in 2.670516014099121s
HTTP Request: %s %s "%s %d %s"
Received healthy response to inference request in 3.039656400680542s
HTTP Request: %s %s "%s %d %s"
Received healthy response to inference request in 2.6996099948883057s
HTTP Request: %s %s "%s %d %s"
Received healthy response to inference request in 2.8731777667999268s
5 requests
0 failed requests
5th percentile: 2.676334810256958
10th percentile: 2.682153606414795
20th percentile: 2.6937911987304686
30th percentile: 2.7343235492706297
40th percentile: 2.8037506580352782
50th percentile: 2.8731777667999268
60th percentile: 2.939769220352173
70th percentile: 3.006360673904419
80th percentile: 3.3691888332366946
90th percentile: 4.028253698348999
95th percentile: 4.357786130905151
99th percentile: 4.621412076950073
mean time: 3.19405574798584
Pipeline stage StressChecker completed in 19.68s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyTriggerPipeline
Pipeline stage OfflineFamilyFriendlyTriggerPipeline completed in 0.45s
run pipeline stage %s
Running pipeline stage TriggerMKMLProfilingPipeline
Pipeline stage TriggerMKMLProfilingPipeline completed in 0.46s
Shutdown handler de-registered
chaiml-mistral31-trysim_80512_v2 status is now deployed due to DeploymentManager action
chaiml-mistral31-trysim_80512_v2 status is now inactive due to auto deactivation removed underperforming models
chaiml-mistral31-trysim_80512_v2 status is now torndown due to DeploymentManager action