developer_uid: NischayDnk
submission_id: mistralai-mistral-nem_93303_v249
model_name: hightemp-Mistral
model_group: mistralai/Mistral-Nemo-I
status: torndown
timestamp: 2025-01-04T04:18:49+00:00
num_battles: 16692
num_wins: 7782
celo_rating: 1236.24
family_friendly_score: 0.5984
family_friendly_standard_error: 0.006932783567947294
submission_type: basic
model_repo: mistralai/Mistral-Nemo-Instruct-2407
model_architecture: MistralForCausalLM
model_num_parameters: 12772070400.0
best_of: 8
max_input_tokens: 1024
max_output_tokens: 64
reward_model: default
latencies: [{'batch_size': 1, 'throughput': 0.6132864486382027, 'latency_mean': 1.6304677307605744, 'latency_p50': 1.6429846286773682, 'latency_p90': 1.7726505756378172}, {'batch_size': 3, 'throughput': 1.124390293859296, 'latency_mean': 2.666370356082916, 'latency_p50': 2.6575692892074585, 'latency_p90': 2.927035903930664}, {'batch_size': 5, 'throughput': 1.3527872182910308, 'latency_mean': 3.6791017496585847, 'latency_p50': 3.6517133712768555, 'latency_p90': 4.1447594165802}, {'batch_size': 6, 'throughput': 1.418921506778588, 'latency_mean': 4.203370831012726, 'latency_p50': 4.191584348678589, 'latency_p90': 4.707057547569275}, {'batch_size': 8, 'throughput': 1.4939382727897732, 'latency_mean': 5.323624559640884, 'latency_p50': 5.32179582118988, 'latency_p90': 5.973917531967163}, {'batch_size': 10, 'throughput': 1.5224303765989633, 'latency_mean': 6.5176846408844, 'latency_p50': 6.531891345977783, 'latency_p90': 7.386304926872254}]
gpu_counts: {'NVIDIA RTX A5000': 1}
display_name: hightemp-Mistral
is_internal_developer: False
language_model: mistralai/Mistral-Nemo-Instruct-2407
model_size: 13B
ranking_group: single
throughput_3p7s: 1.36
us_pacific_date: 2025-01-03
win_ratio: 0.46621135873472325
generation_params: {'temperature': 1.2, 'top_p': 1.0, 'min_p': 0.0, 'top_k': 40, 'presence_penalty': 0.0, 'frequency_penalty': 0.0, 'stopping_words': ['\n'], 'max_input_tokens': 1024, 'best_of': 8, 'max_output_tokens': 64}
formatter: {'memory_template': "{bot_name}'s Persona: {memory}\n####\n", 'prompt_template': '{prompt}\n<START>\n', 'bot_template': '{bot_name}: {message}\n', 'user_template': '{user_name}: {message}\n', 'response_template': '{bot_name}:', 'truncate_by_message': False}
Resubmit model
Shutdown handler not registered because Python interpreter is not running in the main thread
run pipeline %s
run pipeline stage %s
Running pipeline stage MKMLizer
Starting job with name mistralai-mistral-nem-93303-v249-mkmlizer
Waiting for job on mistralai-mistral-nem-93303-v249-mkmlizer to finish
mistralai-mistral-nem-93303-v249-mkmlizer: ╔═════════════════════════════════════════════════════════════════════╗
mistralai-mistral-nem-93303-v249-mkmlizer: ║     _____            __           __                                ║
mistralai-mistral-nem-93303-v249-mkmlizer: ║    / _/ /_ ___    __/ /  ___ ___ / /                                ║
mistralai-mistral-nem-93303-v249-mkmlizer: ║   / _/ / // / |/|/ / _ \/ -_) -_) /                                 ║
mistralai-mistral-nem-93303-v249-mkmlizer: ║  /_//_/\_, /|__,__/_//_/\__/\__/_/                                  ║
mistralai-mistral-nem-93303-v249-mkmlizer: ║       /___/                                                         ║
mistralai-mistral-nem-93303-v249-mkmlizer: ║                                                                     ║
mistralai-mistral-nem-93303-v249-mkmlizer: ║  Version: 0.11.12                                                   ║
mistralai-mistral-nem-93303-v249-mkmlizer: ║  Copyright 2023 MK ONE TECHNOLOGIES Inc.                            ║
mistralai-mistral-nem-93303-v249-mkmlizer: ║  https://mk1.ai                                                     ║
mistralai-mistral-nem-93303-v249-mkmlizer: ║                                                                     ║
mistralai-mistral-nem-93303-v249-mkmlizer: ║  The license key for the current software has been verified as      ║
mistralai-mistral-nem-93303-v249-mkmlizer: ║  belonging to:                                                      ║
mistralai-mistral-nem-93303-v249-mkmlizer: ║                                                                     ║
mistralai-mistral-nem-93303-v249-mkmlizer: ║  Chai Research Corp.                                                ║
mistralai-mistral-nem-93303-v249-mkmlizer: ║  Account ID: 7997a29f-0ceb-4cc7-9adf-840c57b4ae6f                   ║
mistralai-mistral-nem-93303-v249-mkmlizer: ║  Expiration: 2025-04-15 23:59:59                                    ║
mistralai-mistral-nem-93303-v249-mkmlizer: ║                                                                     ║
mistralai-mistral-nem-93303-v249-mkmlizer: ╚═════════════════════════════════════════════════════════════════════╝
Failed to get response for submission mistralai-mistral-nem_93303_v248: HTTPConnectionPool(host='mistralai-mistral-nem-93303-v248-predictor.tenant-chaiml-guanaco.k.chaiverse.com', port=80): Read timed out. (read timeout=12.0)
mistralai-mistral-nem-93303-v249-mkmlizer: Downloaded to shared memory in 54.457s
mistralai-mistral-nem-93303-v249-mkmlizer: quantizing model to /dev/shm/model_cache, profile:s0, folder:/tmp/tmpkwiye65o, device:0
mistralai-mistral-nem-93303-v249-mkmlizer: Saving flywheel model at /dev/shm/model_cache
mistralai-mistral-nem-93303-v249-mkmlizer: quantized model in 35.325s
mistralai-mistral-nem-93303-v249-mkmlizer: Processed model mistralai/Mistral-Nemo-Instruct-2407 in 89.783s
mistralai-mistral-nem-93303-v249-mkmlizer: creating bucket guanaco-mkml-models
mistralai-mistral-nem-93303-v249-mkmlizer: Bucket 's3://guanaco-mkml-models/' created
mistralai-mistral-nem-93303-v249-mkmlizer: uploading /dev/shm/model_cache to s3://guanaco-mkml-models/mistralai-mistral-nem-93303-v249
mistralai-mistral-nem-93303-v249-mkmlizer: cp /dev/shm/model_cache/config.json s3://guanaco-mkml-models/mistralai-mistral-nem-93303-v249/config.json
mistralai-mistral-nem-93303-v249-mkmlizer: cp /dev/shm/model_cache/special_tokens_map.json s3://guanaco-mkml-models/mistralai-mistral-nem-93303-v249/special_tokens_map.json
mistralai-mistral-nem-93303-v249-mkmlizer: cp /dev/shm/model_cache/tokenizer_config.json s3://guanaco-mkml-models/mistralai-mistral-nem-93303-v249/tokenizer_config.json
mistralai-mistral-nem-93303-v249-mkmlizer: cp /dev/shm/model_cache/tokenizer.json s3://guanaco-mkml-models/mistralai-mistral-nem-93303-v249/tokenizer.json
Failed to get response for submission mistralai-mistral-nem_93303_v248: HTTPConnectionPool(host='mistralai-mistral-nem-93303-v248-predictor.tenant-chaiml-guanaco.k.chaiverse.com', port=80): Read timed out. (read timeout=12.0)
mistralai-mistral-nem-93303-v249-mkmlizer: cp /dev/shm/model_cache/flywheel_model.0.safetensors s3://guanaco-mkml-models/mistralai-mistral-nem-93303-v249/flywheel_model.0.safetensors
mistralai-mistral-nem-93303-v249-mkmlizer: 
Loading 0:   0%|          | 0/363 [00:00<?, ?it/s]
Loading 0:   1%|▏         | 5/363 [00:00<00:11, 32.31it/s]
Loading 0:   4%|▎         | 13/363 [00:00<00:06, 54.97it/s]
Loading 0:   5%|▌         | 19/363 [00:00<00:06, 51.45it/s]
Loading 0:   7%|▋         | 25/363 [00:00<00:06, 52.85it/s]
Loading 0:   9%|▉         | 32/363 [00:00<00:06, 48.30it/s]
Loading 0:  11%|█         | 40/363 [00:00<00:05, 56.94it/s]
Loading 0:  13%|█▎        | 47/363 [00:00<00:05, 56.31it/s]
Loading 0:  15%|█▍        | 53/363 [00:00<00:05, 55.11it/s]
Loading 0:  17%|█▋        | 60/363 [00:01<00:05, 54.39it/s]
Loading 0:  18%|█▊        | 66/363 [00:01<00:07, 38.05it/s]
Loading 0:  20%|█▉        | 72/363 [00:01<00:06, 42.31it/s]
Loading 0:  21%|██▏       | 78/363 [00:01<00:06, 41.97it/s]
Loading 0:  23%|██▎       | 83/363 [00:01<00:06, 43.11it/s]
Loading 0:  25%|██▍       | 90/363 [00:01<00:05, 47.50it/s]
Loading 0:  26%|██▋       | 96/363 [00:02<00:05, 45.22it/s]
Loading 0:  28%|██▊       | 101/363 [00:02<00:05, 44.08it/s]
Loading 0:  30%|███       | 109/363 [00:02<00:04, 52.85it/s]
Loading 0:  32%|███▏      | 115/363 [00:02<00:05, 48.74it/s]
Loading 0:  33%|███▎      | 121/363 [00:02<00:05, 48.34it/s]
Loading 0:  35%|███▍      | 127/363 [00:02<00:05, 43.80it/s]
Loading 0:  37%|███▋      | 135/363 [00:02<00:04, 50.37it/s]
Loading 0:  39%|███▉      | 141/363 [00:02<00:04, 45.93it/s]
Loading 0:  40%|████      | 146/363 [00:03<00:06, 32.88it/s]
Loading 0:  41%|████▏     | 150/363 [00:03<00:06, 33.01it/s]
Loading 0:  43%|████▎     | 157/363 [00:03<00:05, 39.21it/s]
Loading 0:  45%|████▍     | 163/363 [00:03<00:04, 40.10it/s]
Loading 0:  46%|████▋     | 168/363 [00:03<00:04, 41.43it/s]
Loading 0:  48%|████▊     | 175/363 [00:03<00:03, 47.10it/s]
Loading 0:  50%|████▉     | 181/363 [00:04<00:04, 44.55it/s]
Loading 0:  51%|█████     | 186/363 [00:04<00:04, 42.74it/s]
Loading 0:  53%|█████▎    | 192/363 [00:04<00:03, 46.28it/s]
Loading 0:  54%|█████▍    | 197/363 [00:04<00:03, 46.10it/s]
Loading 0:  56%|█████▌    | 202/363 [00:04<00:03, 46.90it/s]
Loading 0:  57%|█████▋    | 208/363 [00:04<00:03, 45.56it/s]
Loading 0:  59%|█████▊    | 213/363 [00:04<00:03, 43.72it/s]
Loading 0:  60%|██████    | 219/363 [00:04<00:03, 47.72it/s]
Loading 0:  62%|██████▏   | 224/363 [00:05<00:03, 36.27it/s]
Loading 0:  63%|██████▎   | 229/363 [00:05<00:03, 36.71it/s]
Loading 0:  64%|██████▍   | 234/363 [00:05<00:03, 39.26it/s]
Loading 0:  66%|██████▌   | 239/363 [00:05<00:03, 33.11it/s]
Loading 0:  68%|██████▊   | 246/363 [00:05<00:02, 39.22it/s]
Loading 0:  69%|██████▉   | 251/363 [00:05<00:02, 39.09it/s]
Loading 0:  71%|███████   | 256/363 [00:05<00:02, 38.19it/s]
Loading 0:  72%|███████▏  | 260/363 [00:05<00:02, 38.20it/s]
Loading 0:  73%|███████▎  | 264/363 [00:06<00:02, 38.55it/s]
Loading 0:  74%|███████▍  | 269/363 [00:06<00:02, 40.86it/s]
Loading 0:  75%|███████▌  | 274/363 [00:06<00:02, 42.85it/s]
Loading 0:  77%|███████▋  | 280/363 [00:06<00:01, 42.34it/s]
Loading 0:  79%|███████▊  | 285/363 [00:06<00:01, 42.24it/s]
Loading 0:  80%|████████  | 292/363 [00:06<00:01, 47.46it/s]
Loading 0:  82%|████████▏ | 298/363 [00:06<00:01, 44.44it/s]
Loading 0:  84%|████████▎ | 304/363 [00:13<00:21,  2.77it/s]
Loading 0:  85%|████████▍ | 308/363 [00:13<00:15,  3.51it/s]
Loading 0:  86%|████████▌ | 312/363 [00:13<00:11,  4.51it/s]
Loading 0:  88%|████████▊ | 320/363 [00:13<00:05,  7.36it/s]
Loading 0:  90%|████████▉ | 326/363 [00:14<00:03,  9.91it/s]
Loading 0:  91%|█████████ | 331/363 [00:14<00:02, 12.51it/s]
Loading 0:  93%|█████████▎| 338/363 [00:14<00:01, 17.45it/s]
Loading 0:  95%|█████████▍| 344/363 [00:14<00:00, 21.53it/s]
Loading 0:  96%|█████████▋| 350/363 [00:14<00:00, 26.32it/s]
Loading 0:  98%|█████████▊| 356/363 [00:14<00:00, 31.59it/s]
Loading 0: 100%|█████████▉| 362/363 [00:14<00:00, 33.15it/s]
                                                            

Job mistralai-mistral-nem-93303-v249-mkmlizer completed after 114.56s with status: succeeded
Stopping job with name mistralai-mistral-nem-93303-v249-mkmlizer
Pipeline stage MKMLizer completed in 115.08s
run pipeline stage %s
Running pipeline stage MKMLTemplater
Failed to get response for submission mistralai-mistral-nem_93303_v248: HTTPConnectionPool(host='mistralai-mistral-nem-93303-v248-predictor.tenant-chaiml-guanaco.k.chaiverse.com', port=80): Read timed out. (read timeout=12.0)
Pipeline stage MKMLTemplater completed in 0.19s
run pipeline stage %s
Running pipeline stage MKMLDeployer
Creating inference service mistralai-mistral-nem-93303-v249
Waiting for inference service mistralai-mistral-nem-93303-v249 to be ready
Failed to get response for submission mistralai-mistral-nem_93303_v248: HTTPConnectionPool(host='mistralai-mistral-nem-93303-v248-predictor.tenant-chaiml-guanaco.k.chaiverse.com', port=80): Read timed out. (read timeout=12.0)
Failed to get response for submission mistralai-mistral-nem_93303_v248: HTTPConnectionPool(host='mistralai-mistral-nem-93303-v248-predictor.tenant-chaiml-guanaco.k.chaiverse.com', port=80): Read timed out. (read timeout=12.0)
Failed to get response for submission mistralai-mistral-nem_93303_v248: HTTPConnectionPool(host='mistralai-mistral-nem-93303-v248-predictor.tenant-chaiml-guanaco.k.chaiverse.com', port=80): Read timed out. (read timeout=12.0)
Failed to get response for submission mistralai-mistral-nem_93303_v248: HTTPConnectionPool(host='mistralai-mistral-nem-93303-v248-predictor.tenant-chaiml-guanaco.k.chaiverse.com', port=80): Read timed out. (read timeout=12.0)
Failed to get response for submission mistralai-mistral-nem_93303_v248: HTTPConnectionPool(host='mistralai-mistral-nem-93303-v248-predictor.tenant-chaiml-guanaco.k.chaiverse.com', port=80): Read timed out. (read timeout=12.0)
Failed to get response for submission mistralai-mistral-nem_93303_v248: HTTPConnectionPool(host='mistralai-mistral-nem-93303-v248-predictor.tenant-chaiml-guanaco.k.chaiverse.com', port=80): Read timed out. (read timeout=12.0)
Inference service mistralai-mistral-nem-93303-v249 ready after 331.44571113586426s
Pipeline stage MKMLDeployer completed in 332.00s
run pipeline stage %s
Running pipeline stage StressChecker
Received healthy response to inference request in 1.618299961090088s
Received healthy response to inference request in 2.062264919281006s
Received healthy response to inference request in 1.8222124576568604s
Received healthy response to inference request in 1.8009629249572754s
Received healthy response to inference request in 1.6614243984222412s
5 requests
0 failed requests
5th percentile: 1.6269248485565186
10th percentile: 1.6355497360229492
20th percentile: 1.6527995109558105
30th percentile: 1.6893321037292481
40th percentile: 1.7451475143432618
50th percentile: 1.8009629249572754
60th percentile: 1.8094627380371093
70th percentile: 1.8179625511169433
80th percentile: 1.8702229499816896
90th percentile: 1.9662439346313476
95th percentile: 2.014254426956177
99th percentile: 2.0526628208160402
mean time: 1.7930329322814942
Pipeline stage StressChecker completed in 10.33s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyTriggerPipeline
run_pipeline:run_in_cloud %s
starting trigger_guanaco_pipeline args=%s
triggered trigger_guanaco_pipeline args=%s
Pipeline stage OfflineFamilyFriendlyTriggerPipeline completed in 0.73s
run pipeline stage %s
Running pipeline stage TriggerMKMLProfilingPipeline
run_pipeline:run_in_cloud %s
starting trigger_guanaco_pipeline args=%s
triggered trigger_guanaco_pipeline args=%s
Pipeline stage TriggerMKMLProfilingPipeline completed in 0.70s
Shutdown handler de-registered
mistralai-mistral-nem_93303_v249 status is now deployed due to DeploymentManager action
Shutdown handler registered
run pipeline %s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyScorer
Evaluating %s Family Friendly Score with %s threads
Pipeline stage OfflineFamilyFriendlyScorer completed in 2869.35s
Shutdown handler de-registered
mistralai-mistral-nem_93303_v249 status is now inactive due to auto deactivation removed underperforming models
mistralai-mistral-nem_93303_v249 status is now torndown due to DeploymentManager action
mistralai-mistral-nem_93303_v249 status is now torndown due to DeploymentManager action
mistralai-mistral-nem_93303_v249 status is now torndown due to DeploymentManager action