mistralai-mistral-nem_93303

developer_uid: NischayDnk
submission_id: mistralai-mistral-nem_93303_v482
model_name: mistralai-mistral-nem_93303_v482
model_group: mistralai/Mistral-Nemo-I
status: torndown
timestamp: 2025-06-13T23:54:36+00:00
num_battles: 9043
num_wins: 3679
celo_rating: 1193.95
family_friendly_score: 0.6678
family_friendly_standard_error: 0.00666097830652525
submission_type: basic
model_repo: mistralai/Mistral-Nemo-Instruct-2407
model_architecture: MistralForCausalLM
model_num_parameters: 12772070400.0
best_of: 1
max_input_tokens: 1024
max_output_tokens: 64
reward_model: default
display_name: mistralai-mistral-nem_93303_v482
is_internal_developer: False
language_model: mistralai/Mistral-Nemo-Instruct-2407
model_size: 13B
ranking_group: single
us_pacific_date: 2025-06-13
win_ratio: 0.4068340152604224
generation_params: {'temperature': 0.45, 'top_p': 0.95, 'min_p': 0.025, 'top_k': 60, 'presence_penalty': 0.4, 'frequency_penalty': 0.4, 'stopping_words': ['\n', '<|im_start|>', '<|im_end|>'], 'max_input_tokens': 1024, 'best_of': 1, 'max_output_tokens': 64}
formatter: {'memory_template': '<|im_start|>system\n{memory}<|im_end|>\n', 'prompt_template': '', 'bot_template': '<|im_start|>assistant\n{message}<|im_end|>\n', 'user_template': '<|im_start|>user\n{message}<|im_end|>\n', 'response_template': '<|im_start|>assistant\n', 'truncate_by_message': False}
Resubmit model
Shutdown handler not registered because Python interpreter is not running in the main thread
run pipeline %s
run pipeline stage %s
Running pipeline stage MKMLizer
Starting job with name mistralai-mistral-nem-93303-v482-mkmlizer
Waiting for job on mistralai-mistral-nem-93303-v482-mkmlizer to finish
mistralai-mistral-nem-93303-v482-mkmlizer: ╔═════════════════════════════════════════════════════════════════════╗
mistralai-mistral-nem-93303-v482-mkmlizer: ║                                                                     ║
mistralai-mistral-nem-93303-v482-mkmlizer: ║   ██████   ██████  █████   ████  ████                               ║
mistralai-mistral-nem-93303-v482-mkmlizer: ║  ░░██████ ██████  ░░███   ███░  ░░███                               ║
mistralai-mistral-nem-93303-v482-mkmlizer: ║   ░███░█████░███   ░███  ███     ░███                               ║
mistralai-mistral-nem-93303-v482-mkmlizer: ║   ░███░░███ ░███   ░███████      ░███                               ║
mistralai-mistral-nem-93303-v482-mkmlizer: ║   ░███ ░░░  ░███   ░███░░███     ░███                               ║
mistralai-mistral-nem-93303-v482-mkmlizer: ║   ░███      ░███   ░███ ░░███    ░███                               ║
mistralai-mistral-nem-93303-v482-mkmlizer: ║   █████     █████  █████ ░░████  █████                              ║
mistralai-mistral-nem-93303-v482-mkmlizer: ║  ░░░░░     ░░░░░  ░░░░░   ░░░░  ░░░░░                               ║
mistralai-mistral-nem-93303-v482-mkmlizer: ║                                                                     ║
mistralai-mistral-nem-93303-v482-mkmlizer: ║  Version: 0.27.1+vampire_v3                                         ║
mistralai-mistral-nem-93303-v482-mkmlizer: ║  Features: FLYWHEEL, CUDA                                           ║
mistralai-mistral-nem-93303-v482-mkmlizer: ║  Copyright 2023-2025 MK ONE TECHNOLOGIES Inc.                       ║
mistralai-mistral-nem-93303-v482-mkmlizer: ║  https://mk1.ai                                                     ║
mistralai-mistral-nem-93303-v482-mkmlizer: ║                                                                     ║
mistralai-mistral-nem-93303-v482-mkmlizer: ║  The license key for the current software has been verified as      ║
mistralai-mistral-nem-93303-v482-mkmlizer: ║  belonging to:                                                      ║
mistralai-mistral-nem-93303-v482-mkmlizer: ║                                                                     ║
mistralai-mistral-nem-93303-v482-mkmlizer: ║  Chai Research Corp.                                                ║
mistralai-mistral-nem-93303-v482-mkmlizer: ║  Account ID: 7997a29f-0ceb-4cc7-9adf-840c57b4ae6f                   ║
mistralai-mistral-nem-93303-v482-mkmlizer: ║  Expiration: 2028-03-31 23:59:59                                    ║
mistralai-mistral-nem-93303-v482-mkmlizer: ║                                                                     ║
mistralai-mistral-nem-93303-v482-mkmlizer: ╚═════════════════════════════════════════════════════════════════════╝
mistralai-mistral-nem-93303-v482-mkmlizer: Downloaded to shared memory in 50.773s
mistralai-mistral-nem-93303-v482-mkmlizer: quantizing model to /dev/shm/model_cache, profile:s0, folder:/tmp/tmprg2gkkrz, device:0
mistralai-mistral-nem-93303-v482-mkmlizer: Saving flywheel model at /dev/shm/model_cache
mistralai-mistral-nem-93303-v482-mkmlizer: quantized model in 30.441s
mistralai-mistral-nem-93303-v482-mkmlizer: Processed model mistralai/Mistral-Nemo-Instruct-2407 in 81.214s
mistralai-mistral-nem-93303-v482-mkmlizer: creating bucket guanaco-mkml-models
mistralai-mistral-nem-93303-v482-mkmlizer: Bucket 's3://guanaco-mkml-models/' created
mistralai-mistral-nem-93303-v482-mkmlizer: uploading /dev/shm/model_cache to s3://guanaco-mkml-models/mistralai-mistral-nem-93303-v482
mistralai-mistral-nem-93303-v482-mkmlizer: cp /dev/shm/model_cache/config.json s3://guanaco-mkml-models/mistralai-mistral-nem-93303-v482/config.json
mistralai-mistral-nem-93303-v482-mkmlizer: cp /dev/shm/model_cache/special_tokens_map.json s3://guanaco-mkml-models/mistralai-mistral-nem-93303-v482/special_tokens_map.json
mistralai-mistral-nem-93303-v482-mkmlizer: cp /dev/shm/model_cache/tokenizer_config.json s3://guanaco-mkml-models/mistralai-mistral-nem-93303-v482/tokenizer_config.json
mistralai-mistral-nem-93303-v482-mkmlizer: cp /dev/shm/model_cache/tokenizer.json s3://guanaco-mkml-models/mistralai-mistral-nem-93303-v482/tokenizer.json
mistralai-mistral-nem-93303-v482-mkmlizer: cp /dev/shm/model_cache/flywheel_model.0.safetensors s3://guanaco-mkml-models/mistralai-mistral-nem-93303-v482/flywheel_model.0.safetensors
mistralai-mistral-nem-93303-v482-mkmlizer: 
Loading 0:   0%|          | 0/363 [00:00<?, ?it/s]
Loading 0:   1%|▏         | 5/363 [00:00<00:11, 29.97it/s]
Loading 0:   4%|▎         | 13/363 [00:00<00:06, 50.05it/s]
Loading 0:   5%|▌         | 19/363 [00:00<00:07, 45.91it/s]
Loading 0:   7%|▋         | 24/363 [00:00<00:07, 44.84it/s]
Loading 0:   9%|▊         | 31/363 [00:00<00:06, 51.05it/s]
Loading 0:  10%|█         | 37/363 [00:00<00:07, 45.84it/s]
Loading 0:  12%|█▏        | 42/363 [00:00<00:07, 44.25it/s]
Loading 0:  13%|█▎        | 49/363 [00:01<00:06, 48.87it/s]
Loading 0:  15%|█▌        | 55/363 [00:01<00:06, 45.36it/s]
Loading 0:  17%|█▋        | 61/363 [00:01<00:08, 34.68it/s]
Loading 0:  18%|█▊        | 65/363 [00:01<00:08, 33.28it/s]
Loading 0:  20%|█▉        | 72/363 [00:01<00:07, 39.90it/s]
Loading 0:  21%|██▏       | 78/363 [00:01<00:07, 39.06it/s]
Loading 0:  23%|██▎       | 83/363 [00:02<00:07, 38.06it/s]
Loading 0:  25%|██▍       | 89/363 [00:02<00:06, 42.38it/s]
Loading 0:  26%|██▌       | 94/363 [00:02<00:06, 42.63it/s]
Loading 0:  27%|██▋       | 99/363 [00:02<00:06, 43.78it/s]
Loading 0:  29%|██▉       | 105/363 [00:02<00:06, 42.01it/s]
Loading 0:  31%|███       | 112/363 [00:02<00:05, 46.18it/s]
Loading 0:  32%|███▏      | 117/363 [00:02<00:05, 44.09it/s]
Loading 0:  34%|███▎      | 122/363 [00:02<00:05, 45.05it/s]
Loading 0:  35%|███▍      | 127/363 [00:03<00:06, 37.04it/s]
Loading 0:  37%|███▋      | 134/363 [00:03<00:05, 43.89it/s]
Loading 0:  38%|███▊      | 139/363 [00:03<00:05, 42.26it/s]
Loading 0:  40%|███▉      | 144/363 [00:03<00:07, 27.63it/s]
Loading 0:  41%|████      | 149/363 [00:03<00:07, 29.70it/s]
Loading 0:  43%|████▎     | 156/363 [00:03<00:05, 36.50it/s]
Loading 0:  44%|████▍     | 161/363 [00:03<00:05, 38.74it/s]
Loading 0:  46%|████▌     | 166/363 [00:04<00:04, 40.42it/s]
Loading 0:  47%|████▋     | 172/363 [00:04<00:04, 40.32it/s]
Loading 0:  49%|████▉     | 177/363 [00:04<00:04, 39.96it/s]
Loading 0:  51%|█████     | 184/363 [00:04<00:03, 45.01it/s]
Loading 0:  52%|█████▏    | 190/363 [00:04<00:04, 42.28it/s]
Loading 0:  54%|█████▎    | 195/363 [00:04<00:04, 40.38it/s]
Loading 0:  55%|█████▌    | 201/363 [00:04<00:03, 43.85it/s]
Loading 0:  57%|█████▋    | 206/363 [00:05<00:03, 42.97it/s]
Loading 0:  58%|█████▊    | 211/363 [00:05<00:03, 42.15it/s]
Loading 0:  60%|█████▉    | 216/363 [00:05<00:03, 43.96it/s]
Loading 0:  61%|██████    | 222/363 [00:05<00:03, 41.98it/s]
Loading 0:  63%|██████▎   | 227/363 [00:05<00:04, 30.28it/s]
Loading 0:  64%|██████▎   | 231/363 [00:05<00:04, 30.52it/s]
Loading 0:  65%|██████▌   | 237/363 [00:05<00:03, 34.93it/s]
Loading 0:  66%|██████▋   | 241/363 [00:06<00:03, 35.69it/s]
Loading 0:  68%|██████▊   | 247/363 [00:06<00:02, 39.98it/s]
Loading 0:  70%|██████▉   | 253/363 [00:06<00:02, 39.89it/s]
Loading 0:  71%|███████   | 258/363 [00:06<00:02, 38.27it/s]
Loading 0:  73%|███████▎  | 264/363 [00:06<00:02, 43.14it/s]
Loading 0:  74%|███████▍  | 269/363 [00:06<00:02, 42.20it/s]
Loading 0:  75%|███████▌  | 274/363 [00:06<00:02, 41.63it/s]
Loading 0:  77%|███████▋  | 279/363 [00:06<00:02, 41.78it/s]
Loading 0:  78%|███████▊  | 284/363 [00:07<00:02, 35.76it/s]
Loading 0:  80%|████████  | 291/363 [00:07<00:01, 42.64it/s]
Loading 0:  82%|████████▏ | 296/363 [00:07<00:01, 42.28it/s]
Loading 0:  83%|████████▎ | 302/363 [00:07<00:01, 46.04it/s]
Loading 0:  85%|████████▍ | 307/363 [00:08<00:02, 21.93it/s]
Loading 0:  86%|████████▌ | 312/363 [00:08<00:02, 24.07it/s]
Loading 0:  88%|████████▊ | 320/363 [00:08<00:01, 32.35it/s]
Loading 0:  90%|████████▉ | 326/363 [00:08<00:01, 34.62it/s]
Loading 0:  91%|█████████ | 331/363 [00:08<00:00, 35.77it/s]
Loading 0:  93%|█████████▎| 338/363 [00:08<00:00, 41.25it/s]
Loading 0:  94%|█████████▍| 343/363 [00:08<00:00, 42.69it/s]
Loading 0:  96%|█████████▌| 348/363 [00:08<00:00, 36.52it/s]
Loading 0:  98%|█████████▊| 356/363 [00:09<00:00, 44.35it/s]
Loading 0:  99%|█████████▉| 361/363 [00:09<00:00, 45.31it/s]
                                                            
Job mistralai-mistral-nem-93303-v482-mkmlizer completed after 104.45s with status: succeeded
Stopping job with name mistralai-mistral-nem-93303-v482-mkmlizer
Pipeline stage MKMLizer completed in 105.12s
run pipeline stage %s
Running pipeline stage MKMLTemplater
Pipeline stage MKMLTemplater completed in 0.16s
run pipeline stage %s
Running pipeline stage MKMLDeployer
Creating inference service mistralai-mistral-nem-93303-v482
Waiting for inference service mistralai-mistral-nem-93303-v482 to be ready
Inference service mistralai-mistral-nem-93303-v482 ready after 100.53534173965454s
Pipeline stage MKMLDeployer completed in 101.15s
run pipeline stage %s
Running pipeline stage StressChecker
Received healthy response to inference request in 1.2741568088531494s
Received healthy response to inference request in 1.3465712070465088s
Received healthy response to inference request in 0.8750717639923096s
Received healthy response to inference request in 0.7622463703155518s
Received healthy response to inference request in 0.578200101852417s
5 requests
0 failed requests
5th percentile: 0.615009355545044
10th percentile: 0.6518186092376709
20th percentile: 0.7254371166229248
30th percentile: 0.7848114490509033
40th percentile: 0.8299416065216064
50th percentile: 0.8750717639923096
60th percentile: 1.0347057819366454
70th percentile: 1.1943397998809813
80th percentile: 1.2886396884918212
90th percentile: 1.317605447769165
95th percentile: 1.332088327407837
99th percentile: 1.3436746311187744
mean time: 0.9672492504119873
Pipeline stage StressChecker completed in 6.17s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyTriggerPipeline
run_pipeline:run_in_cloud %s
starting trigger_guanaco_pipeline args=%s
triggered trigger_guanaco_pipeline args=%s
Pipeline stage OfflineFamilyFriendlyTriggerPipeline completed in 0.68s
run pipeline stage %s
Running pipeline stage TriggerMKMLProfilingPipeline
run_pipeline:run_in_cloud %s
starting trigger_guanaco_pipeline args=%s
triggered trigger_guanaco_pipeline args=%s
Pipeline stage TriggerMKMLProfilingPipeline completed in 0.69s
Shutdown handler de-registered
mistralai-mistral-nem_93303_v482 status is now deployed due to DeploymentManager action
Shutdown handler registered
run pipeline %s
run pipeline stage %s
Running pipeline stage MKMLProfilerDeleter
Skipping teardown as no inference service was successfully deployed
Pipeline stage MKMLProfilerDeleter completed in 0.11s
run pipeline stage %s
Running pipeline stage MKMLProfilerTemplater
Pipeline stage MKMLProfilerTemplater completed in 0.12s
run pipeline stage %s
Running pipeline stage MKMLProfilerDeployer
Creating inference service mistralai-mistral-nem-93303-v482-profiler
Waiting for inference service mistralai-mistral-nem-93303-v482-profiler to be ready
Shutdown handler registered
run pipeline %s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyScorer
Evaluating %s Family Friendly Score with %s threads
Pipeline stage OfflineFamilyFriendlyScorer completed in 1984.49s
Shutdown handler de-registered
mistralai-mistral-nem_93303_v482 status is now inactive due to auto deactivation removed underperforming models
mistralai-mistral-nem_93303_v482 status is now torndown due to DeploymentManager action