mistralai-mistral-nem_93303

developer_uid: zhenzhe

submission_id: mistralai-mistral-nem_93303_v449

model_name: bird_test

model_group: mistralai/Mistral-Nemo-I

status: torndown

timestamp: 2025-03-01T04:28:13+00:00

num_battles: 10478

num_wins: 4754

celo_rating: 1241.7

family_friendly_score: 0.5958

family_friendly_standard_error: 0.0069400628239231375

submission_type: basic

model_repo: mistralai/Mistral-Nemo-Instruct-2407

model_architecture: MistralForCausalLM

model_num_parameters: 12772070400.0

best_of: 8

max_input_tokens: 1024

max_output_tokens: 64

reward_model: default

latencies: [{'batch_size': 1, 'throughput': 0.5993420067544807, 'latency_mean': 1.6684273898601532, 'latency_p50': 1.660980224609375, 'latency_p90': 1.8483203887939452}, {'batch_size': 3, 'throughput': 1.0948755955414844, 'latency_mean': 2.735256197452545, 'latency_p50': 2.747364044189453, 'latency_p90': 3.0191949367523194}, {'batch_size': 5, 'throughput': 1.308515776550618, 'latency_mean': 3.807386953830719, 'latency_p50': 3.82863986492157, 'latency_p90': 4.26117594242096}, {'batch_size': 6, 'throughput': 1.3736042237790174, 'latency_mean': 4.346028317213058, 'latency_p50': 4.361011981964111, 'latency_p90': 4.849549126625061}, {'batch_size': 8, 'throughput': 1.428934600742799, 'latency_mean': 5.554532387256622, 'latency_p50': 5.609978199005127, 'latency_p90': 6.230417037010192}, {'batch_size': 10, 'throughput': 1.4688422061002735, 'latency_mean': 6.754300361871719, 'latency_p50': 6.681875228881836, 'latency_p90': 7.695715022087097}]

gpu_counts: {'NVIDIA RTX A5000': 1}

display_name: bird_test

is_internal_developer: False

language_model: mistralai/Mistral-Nemo-Instruct-2407

model_size: 13B

ranking_group: single

throughput_3p7s: 1.3

us_pacific_date: 2025-02-28

win_ratio: 0.45371254056117577

generation_params: {'temperature': 0.95, 'top_p': 0.95, 'min_p': 0.05, 'top_k': 80, 'presence_penalty': 0.0, 'frequency_penalty': 0.0, 'stopping_words': ['User:', 'Bot:', 'Me:', '</s>', '\n', '####', 'You:'], 'max_input_tokens': 1024, 'best_of': 8, 'max_output_tokens': 64}

formatter: {'memory_template': "{bot_name}'s Persona: {memory}\n####\n", 'prompt_template': '{prompt}\n<START>\n', 'bot_template': '{bot_name}: {message}\n', 'user_template': '{user_name}: {message}\n', 'response_template': '{bot_name}:', 'truncate_by_message': False}

Resubmit model

Shutdown handler not registered because Python interpreter is not running in the main thread
run pipeline %s
run pipeline stage %s
Running pipeline stage MKMLizer
Starting job with name mistralai-mistral-nem-93303-v449-mkmlizer
Waiting for job on mistralai-mistral-nem-93303-v449-mkmlizer to finish
mistralai-mistral-nem-93303-v449-mkmlizer: ╔═════════════════════════════════════════════════════════════════════╗
mistralai-mistral-nem-93303-v449-mkmlizer: ║     _____            __           __                                ║
mistralai-mistral-nem-93303-v449-mkmlizer: ║    / _/ /_ ___    __/ /  ___ ___ / /                                ║
mistralai-mistral-nem-93303-v449-mkmlizer: ║   / _/ / // / |/|/ / _ \/ -_) -_) /                                 ║
mistralai-mistral-nem-93303-v449-mkmlizer: ║  /_//_/\_, /|__,__/_//_/\__/\__/_/                                  ║
mistralai-mistral-nem-93303-v449-mkmlizer: ║       /___/                                                         ║
mistralai-mistral-nem-93303-v449-mkmlizer: ║                                                                     ║
mistralai-mistral-nem-93303-v449-mkmlizer: ║  Version: 0.12.8                                                    ║
mistralai-mistral-nem-93303-v449-mkmlizer: ║  Copyright 2023 MK ONE TECHNOLOGIES Inc.                            ║
mistralai-mistral-nem-93303-v449-mkmlizer: ║  https://mk1.ai                                                     ║
mistralai-mistral-nem-93303-v449-mkmlizer: ║                                                                     ║
mistralai-mistral-nem-93303-v449-mkmlizer: ║  The license key for the current software has been verified as      ║
mistralai-mistral-nem-93303-v449-mkmlizer: ║  belonging to:                                                      ║
mistralai-mistral-nem-93303-v449-mkmlizer: ║                                                                     ║
mistralai-mistral-nem-93303-v449-mkmlizer: ║  Chai Research Corp.                                                ║
mistralai-mistral-nem-93303-v449-mkmlizer: ║  Account ID: 7997a29f-0ceb-4cc7-9adf-840c57b4ae6f                   ║
mistralai-mistral-nem-93303-v449-mkmlizer: ║  Expiration: 2025-04-15 23:59:59                                    ║
mistralai-mistral-nem-93303-v449-mkmlizer: ║                                                                     ║
mistralai-mistral-nem-93303-v449-mkmlizer: ╚═════════════════════════════════════════════════════════════════════╝
mistralai-mistral-nem-93303-v449-mkmlizer: Downloaded to shared memory in 51.494s
mistralai-mistral-nem-93303-v449-mkmlizer: quantizing model to /dev/shm/model_cache, profile:s0, folder:/tmp/tmps4whq_lo, device:0
mistralai-mistral-nem-93303-v449-mkmlizer: Saving flywheel model at /dev/shm/model_cache
mistralai-mistral-nem-93303-v449-mkmlizer: quantized model in 35.205s
mistralai-mistral-nem-93303-v449-mkmlizer: Processed model mistralai/Mistral-Nemo-Instruct-2407 in 86.700s
mistralai-mistral-nem-93303-v449-mkmlizer: creating bucket guanaco-mkml-models
mistralai-mistral-nem-93303-v449-mkmlizer: Bucket 's3://guanaco-mkml-models/' created
mistralai-mistral-nem-93303-v449-mkmlizer: uploading /dev/shm/model_cache to s3://guanaco-mkml-models/mistralai-mistral-nem-93303-v449
mistralai-mistral-nem-93303-v449-mkmlizer: cp /dev/shm/model_cache/config.json s3://guanaco-mkml-models/mistralai-mistral-nem-93303-v449/config.json
mistralai-mistral-nem-93303-v449-mkmlizer: cp /dev/shm/model_cache/special_tokens_map.json s3://guanaco-mkml-models/mistralai-mistral-nem-93303-v449/special_tokens_map.json
mistralai-mistral-nem-93303-v449-mkmlizer: cp /dev/shm/model_cache/tokenizer_config.json s3://guanaco-mkml-models/mistralai-mistral-nem-93303-v449/tokenizer_config.json
mistralai-mistral-nem-93303-v449-mkmlizer: cp /dev/shm/model_cache/tokenizer.json s3://guanaco-mkml-models/mistralai-mistral-nem-93303-v449/tokenizer.json
mistralai-mistral-nem-93303-v449-mkmlizer: cp /dev/shm/model_cache/flywheel_model.0.safetensors s3://guanaco-mkml-models/mistralai-mistral-nem-93303-v449/flywheel_model.0.safetensors
mistralai-mistral-nem-93303-v449-mkmlizer: 
Loading 0:   0%|          | 0/363 [00:00<?, ?it/s]
Loading 0:   1%|▏         | 5/363 [00:00<00:11, 30.65it/s]
Loading 0:   4%|▎         | 13/363 [00:00<00:06, 51.98it/s]
Loading 0:   5%|▌         | 19/363 [00:00<00:06, 49.65it/s]
Loading 0:   7%|▋         | 25/363 [00:00<00:06, 51.12it/s]
Loading 0:   9%|▊         | 31/363 [00:00<00:06, 53.51it/s]
Loading 0:  10%|█         | 37/363 [00:00<00:06, 51.18it/s]
Loading 0:  12%|█▏        | 43/363 [00:00<00:06, 52.10it/s]
Loading 0:  13%|█▎        | 49/363 [00:00<00:05, 52.66it/s]
Loading 0:  15%|█▌        | 55/363 [00:01<00:06, 49.76it/s]
Loading 0:  17%|█▋        | 61/363 [00:01<00:08, 37.68it/s]
Loading 0:  18%|█▊        | 66/363 [00:01<00:07, 37.61it/s]
Loading 0:  20%|█▉        | 72/363 [00:01<00:06, 41.65it/s]
Loading 0:  21%|██▏       | 78/363 [00:01<00:06, 41.72it/s]
Loading 0:  23%|██▎       | 83/363 [00:01<00:06, 41.72it/s]
Loading 0:  25%|██▍       | 90/363 [00:01<00:05, 46.60it/s]
Loading 0:  26%|██▋       | 96/363 [00:02<00:05, 44.71it/s]
Loading 0:  28%|██▊       | 101/363 [00:02<00:05, 44.72it/s]
Loading 0:  30%|███       | 109/363 [00:02<00:04, 52.90it/s]
Loading 0:  32%|███▏      | 115/363 [00:02<00:05, 48.16it/s]
Loading 0:  33%|███▎      | 121/363 [00:02<00:05, 47.81it/s]
Loading 0:  35%|███▍      | 127/363 [00:02<00:05, 42.92it/s]
Loading 0:  37%|███▋      | 134/363 [00:02<00:05, 45.61it/s]
Loading 0:  38%|███▊      | 139/363 [00:03<00:04, 45.52it/s]
Loading 0:  40%|███▉      | 144/363 [00:03<00:07, 29.99it/s]
Loading 0:  41%|████      | 149/363 [00:03<00:06, 31.75it/s]
Loading 0:  43%|████▎     | 156/363 [00:03<00:05, 39.14it/s]
Loading 0:  44%|████▍     | 161/363 [00:03<00:04, 40.81it/s]
Loading 0:  46%|████▌     | 166/363 [00:03<00:04, 42.26it/s]
Loading 0:  47%|████▋     | 172/363 [00:03<00:04, 41.21it/s]
Loading 0:  49%|████▉     | 177/363 [00:04<00:04, 39.54it/s]
Loading 0:  50%|█████     | 183/363 [00:04<00:04, 43.37it/s]
Loading 0:  52%|█████▏    | 188/363 [00:04<00:04, 43.30it/s]
Loading 0:  53%|█████▎    | 193/363 [00:04<00:03, 43.31it/s]
Loading 0:  55%|█████▍    | 199/363 [00:04<00:03, 42.59it/s]
Loading 0:  56%|█████▌    | 204/363 [00:04<00:03, 41.73it/s]
Loading 0:  58%|█████▊    | 210/363 [00:04<00:03, 45.19it/s]
Loading 0:  59%|█████▉    | 215/363 [00:04<00:03, 44.34it/s]
Loading 0:  61%|██████    | 222/363 [00:05<00:03, 45.66it/s]
Loading 0:  63%|██████▎   | 227/363 [00:05<00:04, 32.39it/s]
Loading 0:  64%|██████▎   | 231/363 [00:05<00:04, 32.44it/s]
Loading 0:  65%|██████▌   | 237/363 [00:05<00:03, 37.41it/s]
Loading 0:  67%|██████▋   | 242/363 [00:05<00:03, 40.24it/s]
Loading 0:  68%|██████▊   | 247/363 [00:05<00:02, 41.84it/s]
Loading 0:  70%|██████▉   | 253/363 [00:05<00:02, 42.70it/s]
Loading 0:  71%|███████   | 258/363 [00:06<00:02, 43.19it/s]
Loading 0:  73%|███████▎  | 265/363 [00:06<00:02, 48.94it/s]
Loading 0:  75%|███████▍  | 271/363 [00:06<00:01, 47.27it/s]
Loading 0:  76%|███████▌  | 276/363 [00:06<00:01, 44.28it/s]
Loading 0:  78%|███████▊  | 283/363 [00:06<00:01, 49.49it/s]
Loading 0:  80%|███████▉  | 289/363 [00:06<00:01, 48.30it/s]
Loading 0:  81%|████████  | 294/363 [00:06<00:01, 46.86it/s]
Loading 0:  83%|████████▎ | 300/363 [00:06<00:01, 50.12it/s]
Loading 0:  84%|████████▍ | 306/363 [00:13<00:20,  2.80it/s]
Loading 0:  85%|████████▌ | 310/363 [00:13<00:14,  3.53it/s]
Loading 0:  87%|████████▋ | 315/363 [00:13<00:09,  4.82it/s]
Loading 0:  88%|████████▊ | 321/363 [00:13<00:06,  6.77it/s]
Loading 0:  91%|█████████ | 329/363 [00:14<00:03, 10.45it/s]
Loading 0:  92%|█████████▏| 335/363 [00:14<00:02, 13.50it/s]
Loading 0:  94%|█████████▎| 340/363 [00:14<00:01, 16.27it/s]
Loading 0:  96%|█████████▌| 347/363 [00:14<00:00, 21.62it/s]
Loading 0:  97%|█████████▋| 353/363 [00:14<00:00, 25.42it/s]
Loading 0:  99%|█████████▊| 358/363 [00:14<00:00, 28.57it/s]
                                                            
Job mistralai-mistral-nem-93303-v449-mkmlizer completed after 114.21s with status: succeeded
Stopping job with name mistralai-mistral-nem-93303-v449-mkmlizer
Pipeline stage MKMLizer completed in 114.70s
run pipeline stage %s
Running pipeline stage MKMLTemplater
Pipeline stage MKMLTemplater completed in 0.15s
run pipeline stage %s
Running pipeline stage MKMLDeployer
Creating inference service mistralai-mistral-nem-93303-v449
Waiting for inference service mistralai-mistral-nem-93303-v449 to be ready
Inference service mistralai-mistral-nem-93303-v449 ready after 140.4805941581726s
Pipeline stage MKMLDeployer completed in 141.00s
run pipeline stage %s
Running pipeline stage StressChecker
Received healthy response to inference request in 2.440378189086914s
Received healthy response to inference request in 1.4585785865783691s
Received healthy response to inference request in 1.9991202354431152s
Received healthy response to inference request in 1.3814945220947266s
Received healthy response to inference request in 1.6151280403137207s
5 requests
0 failed requests
5th percentile: 1.396911334991455
10th percentile: 1.4123281478881835
20th percentile: 1.4431617736816407
30th percentile: 1.4898884773254395
40th percentile: 1.55250825881958
50th percentile: 1.6151280403137207
60th percentile: 1.7687249183654785
70th percentile: 1.9223217964172363
80th percentile: 2.087371826171875
90th percentile: 2.2638750076293945
95th percentile: 2.3521265983581543
99th percentile: 2.422727870941162
mean time: 1.778939914703369
Pipeline stage StressChecker completed in 10.37s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyTriggerPipeline
run_pipeline:run_in_cloud %s
starting trigger_guanaco_pipeline args=%s
triggered trigger_guanaco_pipeline args=%s
Pipeline stage OfflineFamilyFriendlyTriggerPipeline completed in 0.69s
run pipeline stage %s
Running pipeline stage TriggerMKMLProfilingPipeline
run_pipeline:run_in_cloud %s
starting trigger_guanaco_pipeline args=%s
triggered trigger_guanaco_pipeline args=%s
Pipeline stage TriggerMKMLProfilingPipeline completed in 0.71s
Shutdown handler de-registered
mistralai-mistral-nem_93303_v449 status is now deployed due to DeploymentManager action
Shutdown handler registered
run pipeline %s
run pipeline stage %s
Running pipeline stage MKMLProfilerDeleter
Skipping teardown as no inference service was successfully deployed
Pipeline stage MKMLProfilerDeleter completed in 0.11s
run pipeline stage %s
Running pipeline stage MKMLProfilerTemplater
Pipeline stage MKMLProfilerTemplater completed in 0.10s
run pipeline stage %s
Running pipeline stage MKMLProfilerDeployer
Creating inference service mistralai-mistral-nem-93303-v449-profiler
Waiting for inference service mistralai-mistral-nem-93303-v449-profiler to be ready
Shutdown handler registered
run pipeline %s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyScorer
Evaluating %s Family Friendly Score with %s threads
%s, retrying in %s seconds...
Evaluating %s Family Friendly Score with %s threads
%s, retrying in %s seconds...
Evaluating %s Family Friendly Score with %s threads
Pipeline stage OfflineFamilyFriendlyScorer completed in 4230.08s
Shutdown handler de-registered
mistralai-mistral-nem_93303_v449 status is now inactive due to auto deactivation removed underperforming models
mistralai-mistral-nem_93303_v449 status is now torndown due to DeploymentManager action