developer_uid: azuruce
submission_id: mistralai-mistral-nem_93303_v260
model_name: mistralai-mistral-nem_93303_v260
model_group: mistralai/Mistral-Nemo-I
status: torndown
timestamp: 2025-01-10T18:43:21+00:00
num_battles: 6943
num_wins: 3463
celo_rating: 1254.74
family_friendly_score: 0.595
family_friendly_standard_error: 0.00694226187924368
submission_type: basic
model_repo: mistralai/Mistral-Nemo-Instruct-2407
model_architecture: MistralForCausalLM
model_num_parameters: 12772070400.0
best_of: 8
max_input_tokens: 1024
max_output_tokens: 64
reward_model: default
latencies: [{'batch_size': 1, 'throughput': 0.6122947152102742, 'latency_mean': 1.6331088984012603, 'latency_p50': 1.641366720199585, 'latency_p90': 1.7914663076400756}, {'batch_size': 3, 'throughput': 1.1401774501574313, 'latency_mean': 2.6303188180923462, 'latency_p50': 2.6362725496292114, 'latency_p90': 2.9065468072891236}, {'batch_size': 5, 'throughput': 1.3508827901753202, 'latency_mean': 3.690356212854385, 'latency_p50': 3.7234331369400024, 'latency_p90': 4.1139010190963745}, {'batch_size': 6, 'throughput': 1.441642487708113, 'latency_mean': 4.139590975046158, 'latency_p50': 4.136390328407288, 'latency_p90': 4.681034731864929}, {'batch_size': 8, 'throughput': 1.4948818078187975, 'latency_mean': 5.310092251300812, 'latency_p50': 5.353640556335449, 'latency_p90': 5.985404753684997}, {'batch_size': 10, 'throughput': 1.5478434292416143, 'latency_mean': 6.4119965362548825, 'latency_p50': 6.400988340377808, 'latency_p90': 7.227708840370179}]
gpu_counts: {'NVIDIA RTX A5000': 1}
display_name: mistralai-mistral-nem_93303_v260
is_internal_developer: False
language_model: mistralai/Mistral-Nemo-Instruct-2407
model_size: 13B
ranking_group: single
throughput_3p7s: 1.36
us_pacific_date: 2025-01-10
win_ratio: 0.49877574535503383
generation_params: {'temperature': 1.0, 'top_p': 0.99, 'min_p': 0.0, 'top_k': 80, 'presence_penalty': 0.0, 'frequency_penalty': 0.0, 'stopping_words': ['<|eot_id|>,', 'Bot:,', '\n,', '</s>,', 'Me:', '####,', '<|im_end|>,', 'You:,', 'User:,'], 'max_input_tokens': 1024, 'best_of': 8, 'max_output_tokens': 64}
formatter: {'memory_template': '<|im_start|>system\n{memory}<|im_end|>\n', 'prompt_template': '<|im_start|>user\n{prompt}<|im_end|>\n', 'bot_template': '{bot_name}: {message}\n', 'user_template': '{user_name}: {message}\n', 'response_template': '{bot_name}:', 'truncate_by_message': False}
Resubmit model
Shutdown handler not registered because Python interpreter is not running in the main thread
run pipeline %s
run pipeline stage %s
Running pipeline stage MKMLizer
Starting job with name mistralai-mistral-nem-93303-v260-mkmlizer
Waiting for job on mistralai-mistral-nem-93303-v260-mkmlizer to finish
mistralai-mistral-nem-93303-v260-mkmlizer: ╔═════════════════════════════════════════════════════════════════════╗
mistralai-mistral-nem-93303-v260-mkmlizer: ║     _____            __           __                                ║
mistralai-mistral-nem-93303-v260-mkmlizer: ║    / _/ /_ ___    __/ /  ___ ___ / /                                ║
mistralai-mistral-nem-93303-v260-mkmlizer: ║   / _/ / // / |/|/ / _ \/ -_) -_) /                                 ║
mistralai-mistral-nem-93303-v260-mkmlizer: ║  /_//_/\_, /|__,__/_//_/\__/\__/_/                                  ║
mistralai-mistral-nem-93303-v260-mkmlizer: ║       /___/                                                         ║
mistralai-mistral-nem-93303-v260-mkmlizer: ║                                                                     ║
mistralai-mistral-nem-93303-v260-mkmlizer: ║  Version: 0.11.12                                                   ║
mistralai-mistral-nem-93303-v260-mkmlizer: ║  Copyright 2023 MK ONE TECHNOLOGIES Inc.                            ║
mistralai-mistral-nem-93303-v260-mkmlizer: ║  https://mk1.ai                                                     ║
mistralai-mistral-nem-93303-v260-mkmlizer: ║                                                                     ║
mistralai-mistral-nem-93303-v260-mkmlizer: ║  The license key for the current software has been verified as      ║
mistralai-mistral-nem-93303-v260-mkmlizer: ║  belonging to:                                                      ║
mistralai-mistral-nem-93303-v260-mkmlizer: ║                                                                     ║
mistralai-mistral-nem-93303-v260-mkmlizer: ║  Chai Research Corp.                                                ║
mistralai-mistral-nem-93303-v260-mkmlizer: ║  Account ID: 7997a29f-0ceb-4cc7-9adf-840c57b4ae6f                   ║
mistralai-mistral-nem-93303-v260-mkmlizer: ║  Expiration: 2025-04-15 23:59:59                                    ║
mistralai-mistral-nem-93303-v260-mkmlizer: ║                                                                     ║
mistralai-mistral-nem-93303-v260-mkmlizer: ╚═════════════════════════════════════════════════════════════════════╝
mistralai-mistral-nem-93303-v260-mkmlizer: Downloaded to shared memory in 59.081s
mistralai-mistral-nem-93303-v260-mkmlizer: quantizing model to /dev/shm/model_cache, profile:s0, folder:/tmp/tmp6tse2awk, device:0
mistralai-mistral-nem-93303-v260-mkmlizer: Saving flywheel model at /dev/shm/model_cache
mistralai-mistral-nem-93303-v260-mkmlizer: quantized model in 36.052s
mistralai-mistral-nem-93303-v260-mkmlizer: Processed model mistralai/Mistral-Nemo-Instruct-2407 in 95.133s
mistralai-mistral-nem-93303-v260-mkmlizer: creating bucket guanaco-mkml-models
mistralai-mistral-nem-93303-v260-mkmlizer: Bucket 's3://guanaco-mkml-models/' created
mistralai-mistral-nem-93303-v260-mkmlizer: uploading /dev/shm/model_cache to s3://guanaco-mkml-models/mistralai-mistral-nem-93303-v260
mistralai-mistral-nem-93303-v260-mkmlizer: cp /dev/shm/model_cache/special_tokens_map.json s3://guanaco-mkml-models/mistralai-mistral-nem-93303-v260/special_tokens_map.json
mistralai-mistral-nem-93303-v260-mkmlizer: cp /dev/shm/model_cache/config.json s3://guanaco-mkml-models/mistralai-mistral-nem-93303-v260/config.json
mistralai-mistral-nem-93303-v260-mkmlizer: cp /dev/shm/model_cache/tokenizer_config.json s3://guanaco-mkml-models/mistralai-mistral-nem-93303-v260/tokenizer_config.json
mistralai-mistral-nem-93303-v260-mkmlizer: cp /dev/shm/model_cache/tokenizer.json s3://guanaco-mkml-models/mistralai-mistral-nem-93303-v260/tokenizer.json
mistralai-mistral-nem-93303-v260-mkmlizer: cp /dev/shm/model_cache/flywheel_model.0.safetensors s3://guanaco-mkml-models/mistralai-mistral-nem-93303-v260/flywheel_model.0.safetensors
mistralai-mistral-nem-93303-v260-mkmlizer: 
Loading 0:   0%|          | 0/363 [00:00<?, ?it/s]
Loading 0:   1%|▏         | 5/363 [00:00<00:11, 31.05it/s]
Loading 0:   4%|▎         | 13/363 [00:00<00:06, 51.97it/s]
Loading 0:   5%|▌         | 19/363 [00:00<00:07, 47.94it/s]
Loading 0:   7%|▋         | 25/363 [00:00<00:06, 49.27it/s]
Loading 0:   9%|▊         | 31/363 [00:00<00:06, 51.60it/s]
Loading 0:  10%|█         | 37/363 [00:00<00:06, 47.12it/s]
Loading 0:  12%|█▏        | 42/363 [00:00<00:06, 46.15it/s]
Loading 0:  13%|█▎        | 49/363 [00:00<00:06, 52.21it/s]
Loading 0:  15%|█▌        | 55/363 [00:01<00:06, 49.08it/s]
Loading 0:  17%|█▋        | 61/363 [00:01<00:08, 37.55it/s]
Loading 0:  18%|█▊        | 66/363 [00:01<00:07, 37.60it/s]
Loading 0:  20%|█▉        | 72/363 [00:01<00:06, 41.82it/s]
Loading 0:  21%|██▏       | 78/363 [00:01<00:06, 40.94it/s]
Loading 0:  23%|██▎       | 83/363 [00:01<00:06, 40.73it/s]
Loading 0:  25%|██▍       | 90/363 [00:02<00:05, 46.11it/s]
Loading 0:  26%|██▋       | 96/363 [00:02<00:06, 44.20it/s]
Loading 0:  28%|██▊       | 101/363 [00:02<00:06, 41.67it/s]
Loading 0:  30%|███       | 109/363 [00:02<00:05, 50.19it/s]
Loading 0:  32%|███▏      | 115/363 [00:02<00:05, 46.38it/s]
Loading 0:  33%|███▎      | 120/363 [00:02<00:05, 43.81it/s]
Loading 0:  35%|███▍      | 126/363 [00:02<00:05, 46.12it/s]
Loading 0:  36%|███▌      | 131/363 [00:02<00:04, 46.75it/s]
Loading 0:  37%|███▋      | 136/363 [00:03<00:05, 39.65it/s]
Loading 0:  39%|███▉      | 142/363 [00:03<00:06, 33.26it/s]
Loading 0:  40%|████      | 146/363 [00:03<00:06, 34.30it/s]
Loading 0:  41%|████▏     | 150/363 [00:03<00:06, 33.69it/s]
Loading 0:  43%|████▎     | 156/363 [00:03<00:05, 39.01it/s]
Loading 0:  44%|████▍     | 161/363 [00:03<00:05, 40.29it/s]
Loading 0:  46%|████▌     | 166/363 [00:03<00:04, 42.23it/s]
Loading 0:  47%|████▋     | 172/363 [00:04<00:04, 42.58it/s]
Loading 0:  49%|████▉     | 177/363 [00:04<00:04, 42.15it/s]
Loading 0:  50%|█████     | 183/363 [00:04<00:03, 46.15it/s]
Loading 0:  52%|█████▏    | 188/363 [00:04<00:03, 46.77it/s]
Loading 0:  53%|█████▎    | 193/363 [00:04<00:03, 47.28it/s]
Loading 0:  55%|█████▍    | 199/363 [00:04<00:03, 43.12it/s]
Loading 0:  56%|█████▌    | 204/363 [00:04<00:03, 41.33it/s]
Loading 0:  58%|█████▊    | 211/363 [00:04<00:03, 46.67it/s]
Loading 0:  60%|█████▉    | 217/363 [00:05<00:03, 44.88it/s]
Loading 0:  61%|██████▏   | 223/363 [00:05<00:03, 36.01it/s]
Loading 0:  63%|██████▎   | 227/363 [00:05<00:03, 36.39it/s]
Loading 0:  64%|██████▎   | 231/363 [00:05<00:03, 35.32it/s]
Loading 0:  65%|██████▌   | 237/363 [00:05<00:03, 40.40it/s]
Loading 0:  67%|██████▋   | 242/363 [00:05<00:02, 41.30it/s]
Loading 0:  68%|██████▊   | 247/363 [00:05<00:02, 42.73it/s]
Loading 0:  70%|██████▉   | 253/363 [00:05<00:02, 42.47it/s]
Loading 0:  71%|███████   | 258/363 [00:06<00:02, 41.97it/s]
Loading 0:  73%|███████▎  | 264/363 [00:06<00:02, 46.21it/s]
Loading 0:  74%|███████▍  | 269/363 [00:06<00:02, 45.77it/s]
Loading 0:  75%|███████▌  | 274/363 [00:06<00:01, 46.06it/s]
Loading 0:  77%|███████▋  | 279/363 [00:06<00:01, 45.56it/s]
Loading 0:  78%|███████▊  | 284/363 [00:06<00:02, 37.13it/s]
Loading 0:  80%|████████  | 291/363 [00:06<00:01, 44.79it/s]
Loading 0:  82%|████████▏ | 296/363 [00:06<00:01, 45.16it/s]
Loading 0:  83%|████████▎ | 303/363 [00:07<00:01, 45.33it/s]
Loading 0:  85%|████████▍ | 308/363 [00:13<00:20,  2.63it/s]
Loading 0:  86%|████████▌ | 312/363 [00:14<00:15,  3.35it/s]
Loading 0:  88%|████████▊ | 320/363 [00:14<00:07,  5.43it/s]
Loading 0:  90%|████████▉ | 326/363 [00:14<00:05,  7.39it/s]
Loading 0:  91%|█████████ | 331/363 [00:14<00:03,  9.48it/s]
Loading 0:  93%|█████████▎| 338/363 [00:14<00:01, 13.47it/s]
Loading 0:  95%|█████████▍| 344/363 [00:14<00:01, 17.08it/s]
Loading 0:  96%|█████████▋| 350/363 [00:14<00:00, 21.43it/s]
Loading 0:  98%|█████████▊| 356/363 [00:14<00:00, 26.37it/s]
Loading 0: 100%|█████████▉| 362/363 [00:14<00:00, 30.30it/s]
                                                            

Job mistralai-mistral-nem-93303-v260-mkmlizer completed after 134.98s with status: succeeded
Stopping job with name mistralai-mistral-nem-93303-v260-mkmlizer
Pipeline stage MKMLizer completed in 135.52s
run pipeline stage %s
Running pipeline stage MKMLTemplater
Pipeline stage MKMLTemplater completed in 0.19s
run pipeline stage %s
Running pipeline stage MKMLDeployer
Creating inference service mistralai-mistral-nem-93303-v260
Waiting for inference service mistralai-mistral-nem-93303-v260 to be ready
Inference service mistralai-mistral-nem-93303-v260 ready after 371.7305705547333s
Pipeline stage MKMLDeployer completed in 372.92s
run pipeline stage %s
Running pipeline stage StressChecker
Received healthy response to inference request in 2.077793836593628s
Received healthy response to inference request in 2.310101270675659s
Received healthy response to inference request in 7.79150652885437s
Received healthy response to inference request in 1.9953598976135254s
Received healthy response to inference request in 7.128508567810059s
5 requests
0 failed requests
5th percentile: 2.011846685409546
10th percentile: 2.0283334732055662
20th percentile: 2.0613070487976075
30th percentile: 2.124255323410034
40th percentile: 2.217178297042847
50th percentile: 2.310101270675659
60th percentile: 4.237464189529419
70th percentile: 6.164827108383178
80th percentile: 7.261108160018921
90th percentile: 7.5263073444366455
95th percentile: 7.658906936645508
99th percentile: 7.764986610412597
mean time: 4.260654020309448
%s, retrying in %s seconds...
Received healthy response to inference request in 1.903688669204712s
Received healthy response to inference request in 2.193024158477783s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Received healthy response to inference request in 1.837599277496338s
Received healthy response to inference request in 1.658860445022583s
Received healthy response to inference request in 2.1634325981140137s
5 requests
0 failed requests
5th percentile: 1.694608211517334
10th percentile: 1.730355978012085
20th percentile: 1.801851511001587
30th percentile: 1.8508171558380127
40th percentile: 1.8772529125213624
50th percentile: 1.903688669204712
60th percentile: 2.0075862407684326
70th percentile: 2.1114838123321533
80th percentile: 2.1693509101867674
90th percentile: 2.1811875343322753
95th percentile: 2.1871058464050295
99th percentile: 2.1918404960632323
mean time: 1.9513210296630858
Pipeline stage StressChecker completed in 34.31s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyTriggerPipeline
run_pipeline:run_in_cloud %s
starting trigger_guanaco_pipeline args=%s
triggered trigger_guanaco_pipeline args=%s
Pipeline stage OfflineFamilyFriendlyTriggerPipeline completed in 0.75s
run pipeline stage %s
Running pipeline stage TriggerMKMLProfilingPipeline
run_pipeline:run_in_cloud %s
starting trigger_guanaco_pipeline args=%s
triggered trigger_guanaco_pipeline args=%s
Pipeline stage TriggerMKMLProfilingPipeline completed in 0.83s
Shutdown handler de-registered
mistralai-mistral-nem_93303_v260 status is now deployed due to DeploymentManager action
Shutdown handler registered
run pipeline %s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyScorer
Evaluating %s Family Friendly Score with %s threads
Pipeline stage OfflineFamilyFriendlyScorer completed in 2851.88s
Shutdown handler de-registered
mistralai-mistral-nem_93303_v260 status is now inactive due to auto deactivation removed underperforming models
mistralai-mistral-nem_93303_v260 status is now torndown due to DeploymentManager action