mistralai-mistral-sma_53415

developer_uid: azuruce

submission_id: mistralai-mistral-sma_53415_v105

model_name: mistralai-mistral-sma_53415_v105

model_group: mistralai/Mistral-Small-

status: torndown

timestamp: 2025-01-10T19:02:55+00:00

num_battles: 8153

num_wins: 3935

celo_rating: 1241.27

family_friendly_score: 0.5988

family_friendly_standard_error: 0.006931645692041681

submission_type: basic

model_repo: mistralai/Mistral-Small-Instruct-2409

model_architecture: MistralForCausalLM

model_num_parameters: 22247282688.0

best_of: 8

max_input_tokens: 1024

max_output_tokens: 64

latencies: [{'batch_size': 1, 'throughput': 0.3792297555532471, 'latency_mean': 2.6368523454666137, 'latency_p50': 2.6461453437805176, 'latency_p90': 2.921352910995483}, {'batch_size': 2, 'throughput': 0.6060270349062757, 'latency_mean': 3.2981177985668184, 'latency_p50': 3.28867244720459, 'latency_p90': 3.6257739305496215}, {'batch_size': 3, 'throughput': 0.7635480648274316, 'latency_mean': 3.915440343618393, 'latency_p50': 3.9221326112747192, 'latency_p90': 4.335126781463623}, {'batch_size': 4, 'throughput': 0.8847449432507208, 'latency_mean': 4.5104432606697085, 'latency_p50': 4.500641465187073, 'latency_p90': 4.966550421714783}, {'batch_size': 5, 'throughput': 0.9727584252643632, 'latency_mean': 5.108116525411606, 'latency_p50': 5.082103729248047, 'latency_p90': 5.6923583984375}]

gpu_counts: {'NVIDIA RTX A6000': 1}

display_name: mistralai-mistral-sma_53415_v105

is_internal_developer: True

language_model: mistralai/Mistral-Small-Instruct-2409

model_size: 22B

ranking_group: single

throughput_3p7s: 0.72

us_pacific_date: 2025-01-10

win_ratio: 0.4826444253648964

generation_params: {'temperature': 1.0, 'top_p': 0.99, 'min_p': 0.0, 'top_k': 80, 'presence_penalty': 0.0, 'frequency_penalty': 0.0, 'stopping_words': ['\n,', '</s>,', '####,', 'Bot:,', 'User:,', 'You:,', '<|im_end|>,', '<|eot_id|>,', 'Me:'], 'max_input_tokens': 1024, 'best_of': 8, 'max_output_tokens': 64}

formatter: {'memory_template': '<|im_start|>system\n{memory}<|im_end|>\n', 'prompt_template': '<|im_start|>user\n{prompt}<|im_end|>\n', 'bot_template': '{bot_name}: {message}\n', 'user_template': '{user_name}: {message}\n', 'response_template': '{bot_name}:', 'truncate_by_message': False}

Resubmit model

Shutdown handler not registered because Python interpreter is not running in the main thread
run pipeline %s
run pipeline stage %s
Running pipeline stage MKMLizer
Starting job with name mistralai-mistral-sma-53415-v105-mkmlizer
Waiting for job on mistralai-mistral-sma-53415-v105-mkmlizer to finish
Failed to get response for submission mistralai-mistral-sma_53415_v101: HTTPConnectionPool(host='mistralai-mistral-sma-53415-v101-predictor.tenant-chaiml-guanaco.k.chaiverse.com', port=80): Read timed out. (read timeout=12.0)
mistralai-mistral-sma-53415-v105-mkmlizer: ╔═════════════════════════════════════════════════════════════════════╗
mistralai-mistral-sma-53415-v105-mkmlizer: ║     _____            __           __                                ║
mistralai-mistral-sma-53415-v105-mkmlizer: ║    / _/ /_ ___    __/ /  ___ ___ / /                                ║
mistralai-mistral-sma-53415-v105-mkmlizer: ║   / _/ / // / |/|/ / _ \/ -_) -_) /                                 ║
mistralai-mistral-sma-53415-v105-mkmlizer: ║  /_//_/\_, /|__,__/_//_/\__/\__/_/                                  ║
mistralai-mistral-sma-53415-v105-mkmlizer: ║       /___/                                                         ║
mistralai-mistral-sma-53415-v105-mkmlizer: ║                                                                     ║
mistralai-mistral-sma-53415-v105-mkmlizer: ║  Version: 0.11.12                                                   ║
mistralai-mistral-sma-53415-v105-mkmlizer: ║  Copyright 2023 MK ONE TECHNOLOGIES Inc.                            ║
mistralai-mistral-sma-53415-v105-mkmlizer: ║  https://mk1.ai                                                     ║
mistralai-mistral-sma-53415-v105-mkmlizer: ║                                                                     ║
mistralai-mistral-sma-53415-v105-mkmlizer: ║  The license key for the current software has been verified as      ║
mistralai-mistral-sma-53415-v105-mkmlizer: ║  belonging to:                                                      ║
mistralai-mistral-sma-53415-v105-mkmlizer: ║                                                                     ║
mistralai-mistral-sma-53415-v105-mkmlizer: ║  Chai Research Corp.                                                ║
mistralai-mistral-sma-53415-v105-mkmlizer: ║  Account ID: 7997a29f-0ceb-4cc7-9adf-840c57b4ae6f                   ║
mistralai-mistral-sma-53415-v105-mkmlizer: ║  Expiration: 2025-04-15 23:59:59                                    ║
mistralai-mistral-sma-53415-v105-mkmlizer: ║                                                                     ║
mistralai-mistral-sma-53415-v105-mkmlizer: ╚═════════════════════════════════════════════════════════════════════╝
mistralai-mistral-sma-53415-v105-mkmlizer: Downloaded to shared memory in 97.132s
mistralai-mistral-sma-53415-v105-mkmlizer: quantizing model to /dev/shm/model_cache, profile:s0, folder:/tmp/tmp158df_on, device:0
mistralai-mistral-sma-53415-v105-mkmlizer: Saving flywheel model at /dev/shm/model_cache
mistralai-mistral-sma-53415-v105-mkmlizer: quantized model in 46.454s
mistralai-mistral-sma-53415-v105-mkmlizer: Processed model mistralai/Mistral-Small-Instruct-2409 in 143.586s
mistralai-mistral-sma-53415-v105-mkmlizer: Bucket 's3://guanaco-mkml-models/' created
mistralai-mistral-sma-53415-v105-mkmlizer: uploading /dev/shm/model_cache to s3://guanaco-mkml-models/mistralai-mistral-sma-53415-v105
mistralai-mistral-sma-53415-v105-mkmlizer: cp /dev/shm/model_cache/config.json s3://guanaco-mkml-models/mistralai-mistral-sma-53415-v105/config.json
mistralai-mistral-sma-53415-v105-mkmlizer: cp /dev/shm/model_cache/special_tokens_map.json s3://guanaco-mkml-models/mistralai-mistral-sma-53415-v105/special_tokens_map.json
mistralai-mistral-sma-53415-v105-mkmlizer: cp /dev/shm/model_cache/tokenizer_config.json s3://guanaco-mkml-models/mistralai-mistral-sma-53415-v105/tokenizer_config.json
mistralai-mistral-sma-53415-v105-mkmlizer: cp /dev/shm/model_cache/tokenizer.model s3://guanaco-mkml-models/mistralai-mistral-sma-53415-v105/tokenizer.model
mistralai-mistral-sma-53415-v105-mkmlizer: cp /dev/shm/model_cache/tokenizer.json s3://guanaco-mkml-models/mistralai-mistral-sma-53415-v105/tokenizer.json
mistralai-mistral-sma-53415-v105-mkmlizer: cp /dev/shm/model_cache/flywheel_model.1.safetensors s3://guanaco-mkml-models/mistralai-mistral-sma-53415-v105/flywheel_model.1.safetensors
mistralai-mistral-sma-53415-v105-mkmlizer: cp /dev/shm/model_cache/flywheel_model.0.safetensors s3://guanaco-mkml-models/mistralai-mistral-sma-53415-v105/flywheel_model.0.safetensors
mistralai-mistral-sma-53415-v105-mkmlizer: 
Loading 0:   0%|          | 0/507 [00:00<?, ?it/s]
Loading 0:   1%|          | 5/507 [00:00<00:21, 23.18it/s]
Loading 0:   2%|▏         | 10/507 [00:00<00:15, 31.34it/s]
Loading 0:   3%|▎         | 14/507 [00:00<00:17, 27.76it/s]
Loading 0:   4%|▎         | 19/507 [00:00<00:14, 32.98it/s]
Loading 0:   5%|▍         | 23/507 [00:00<00:16, 29.12it/s]
Loading 0:   6%|▌         | 28/507 [00:00<00:14, 33.68it/s]
Loading 0:   6%|▋         | 32/507 [00:01<00:16, 28.76it/s]
Loading 0:   7%|▋         | 37/507 [00:01<00:14, 33.38it/s]
Loading 0:   8%|▊         | 41/507 [00:01<00:14, 31.19it/s]
Loading 0:   9%|▉         | 46/507 [00:01<00:13, 35.21it/s]
Loading 0:  10%|▉         | 50/507 [00:01<00:14, 31.35it/s]
Loading 0:  11%|█         | 54/507 [00:01<00:19, 23.04it/s]
Loading 0:  11%|█         | 57/507 [00:02<00:20, 22.39it/s]
Loading 0:  12%|█▏        | 61/507 [00:02<00:17, 25.78it/s]
Loading 0:  13%|█▎        | 65/507 [00:02<00:17, 25.17it/s]
Loading 0:  14%|█▍        | 71/507 [00:02<00:13, 32.21it/s]
Loading 0:  15%|█▍        | 75/507 [00:02<00:13, 31.72it/s]
Loading 0:  16%|█▌        | 80/507 [00:02<00:13, 31.40it/s]
Loading 0:  17%|█▋        | 85/507 [00:02<00:12, 35.05it/s]
Loading 0:  18%|█▊        | 89/507 [00:02<00:13, 30.41it/s]
Loading 0:  19%|█▊        | 94/507 [00:03<00:12, 33.93it/s]
Loading 0:  19%|█▉        | 98/507 [00:03<00:13, 31.39it/s]
Loading 0:  21%|██        | 105/507 [00:03<00:10, 36.69it/s]
Loading 0:  21%|██▏       | 109/507 [00:03<00:11, 33.21it/s]
Loading 0:  22%|██▏       | 113/507 [00:03<00:15, 25.19it/s]
Loading 0:  23%|██▎       | 116/507 [00:04<00:17, 22.30it/s]
Loading 0:  24%|██▍       | 122/507 [00:04<00:15, 24.66it/s]
Loading 0:  25%|██▌       | 127/507 [00:04<00:13, 29.05it/s]
Loading 0:  26%|██▌       | 131/507 [00:04<00:13, 28.24it/s]
Loading 0:  27%|██▋       | 136/507 [00:04<00:11, 32.46it/s]
Loading 0:  28%|██▊       | 140/507 [00:04<00:12, 29.99it/s]
Loading 0:  29%|██▉       | 147/507 [00:04<00:09, 36.76it/s]
Loading 0:  30%|██▉       | 151/507 [00:04<00:09, 36.20it/s]
Loading 0:  31%|███       | 156/507 [00:05<00:09, 38.46it/s]
Loading 0:  32%|███▏      | 160/507 [00:05<00:09, 36.45it/s]
Loading 0:  32%|███▏      | 164/507 [00:05<00:09, 35.31it/s]
Loading 0:  33%|███▎      | 169/507 [00:05<00:11, 30.08it/s]
Loading 0:  34%|███▍      | 173/507 [00:05<00:11, 29.25it/s]
Loading 0:  35%|███▍      | 177/507 [00:05<00:12, 26.42it/s]
Loading 0:  36%|███▌      | 181/507 [00:06<00:11, 27.56it/s]
Loading 0:  36%|███▋      | 185/507 [00:06<00:12, 26.17it/s]
Loading 0:  37%|███▋      | 190/507 [00:06<00:10, 31.21it/s]
Loading 0:  38%|███▊      | 194/507 [00:06<00:10, 28.75it/s]
Loading 0:  39%|███▉      | 199/507 [00:06<00:09, 32.93it/s]
Loading 0:  40%|████      | 203/507 [00:06<00:10, 29.69it/s]
Loading 0:  41%|████      | 208/507 [00:06<00:08, 33.48it/s]
Loading 0:  42%|████▏     | 212/507 [00:07<00:09, 29.59it/s]
Loading 0:  43%|████▎     | 217/507 [00:07<00:08, 33.52it/s]
Loading 0:  44%|████▎     | 221/507 [00:07<00:09, 30.86it/s]
Loading 0:  44%|████▍     | 225/507 [00:07<00:11, 23.50it/s]
Loading 0:  45%|████▌     | 230/507 [00:07<00:11, 24.98it/s]
Loading 0:  46%|████▋     | 235/507 [00:07<00:09, 29.56it/s]
Loading 0:  47%|████▋     | 239/507 [00:08<00:09, 27.84it/s]
Loading 0:  49%|████▊     | 246/507 [00:08<00:07, 34.36it/s]
Loading 0:  49%|████▉     | 250/507 [00:08<00:07, 32.64it/s]
Loading 0:  50%|█████     | 254/507 [00:08<00:07, 33.27it/s]
Loading 0:  51%|█████     | 258/507 [00:08<00:08, 29.21it/s]
Loading 0:  52%|█████▏    | 262/507 [00:08<00:08, 30.14it/s]
Loading 0:  52%|█████▏    | 266/507 [00:08<00:08, 28.00it/s]
Loading 0:  54%|█████▍    | 273/507 [00:09<00:06, 34.62it/s]
Loading 0:  55%|█████▍    | 277/507 [00:09<00:06, 33.47it/s]
Loading 0:  55%|█████▌    | 281/507 [00:09<00:06, 34.36it/s]
Loading 0:  56%|█████▌    | 285/507 [00:09<00:09, 24.41it/s]
Loading 0:  57%|█████▋    | 288/507 [00:09<00:09, 23.59it/s]
Loading 0:  58%|█████▊    | 293/507 [00:09<00:08, 24.14it/s]
Loading 0:  59%|█████▉    | 298/507 [00:10<00:07, 28.17it/s]
Loading 0:  59%|█████▉    | 299/507 [00:20<00:07, 28.17it/s]
Loading 0:  59%|█████▉    | 300/507 [00:24<04:08,  1.20s/it]
Loading 0:  60%|█████▉    | 302/507 [00:24<03:23,  1.01it/s]
Loading 0:  61%|██████    | 307/507 [00:24<02:01,  1.64it/s]
Loading 0:  61%|██████    | 310/507 [00:25<01:31,  2.14it/s]
Loading 0:  62%|██████▏   | 313/507 [00:25<01:08,  2.85it/s]
Loading 0:  63%|██████▎   | 318/507 [00:25<00:42,  4.44it/s]
Loading 0:  63%|██████▎   | 321/507 [00:25<00:33,  5.56it/s]
Loading 0:  64%|██████▍   | 325/507 [00:25<00:24,  7.58it/s]
Loading 0:  65%|██████▍   | 329/507 [00:25<00:18,  9.63it/s]
Loading 0:  66%|██████▌   | 334/507 [00:25<00:12, 13.52it/s]
Loading 0:  67%|██████▋   | 339/507 [00:25<00:09, 17.34it/s]
Loading 0:  68%|██████▊   | 343/507 [00:26<00:10, 16.02it/s]
Loading 0:  68%|██████▊   | 347/507 [00:26<00:09, 17.47it/s]
Loading 0:  69%|██████▉   | 352/507 [00:26<00:07, 22.02it/s]
Loading 0:  70%|███████   | 356/507 [00:26<00:06, 22.07it/s]
Loading 0:  71%|███████   | 361/507 [00:26<00:05, 26.62it/s]
Loading 0:  72%|███████▏  | 365/507 [00:27<00:05, 25.36it/s]
Loading 0:  73%|███████▎  | 372/507 [00:27<00:04, 32.07it/s]
Loading 0:  74%|███████▍  | 376/507 [00:27<00:04, 31.26it/s]
Loading 0:  75%|███████▌  | 381/507 [00:27<00:03, 32.71it/s]
Loading 0:  76%|███████▌  | 385/507 [00:27<00:03, 31.58it/s]
Loading 0:  77%|███████▋  | 389/507 [00:27<00:03, 31.53it/s]
Loading 0:  78%|███████▊  | 393/507 [00:27<00:03, 30.96it/s]
Loading 0:  78%|███████▊  | 397/507 [00:28<00:04, 23.82it/s]
Loading 0:  79%|███████▉  | 401/507 [00:28<00:04, 23.36it/s]
Loading 0:  80%|████████  | 406/507 [00:28<00:03, 28.34it/s]
Loading 0:  81%|████████  | 410/507 [00:28<00:03, 26.75it/s]
Loading 0:  82%|████████▏ | 415/507 [00:28<00:02, 31.07it/s]
Loading 0:  83%|████████▎ | 419/507 [00:28<00:03, 27.89it/s]
Loading 0:  84%|████████▎ | 424/507 [00:28<00:02, 32.23it/s]
Loading 0:  84%|████████▍ | 428/507 [00:29<00:02, 29.08it/s]
Loading 0:  85%|████████▌ | 433/507 [00:29<00:02, 33.63it/s]
Loading 0:  86%|████████▌ | 437/507 [00:29<00:02, 30.37it/s]
Loading 0:  87%|████████▋ | 442/507 [00:29<00:01, 34.82it/s]
Loading 0:  88%|████████▊ | 446/507 [00:29<00:01, 30.63it/s]
Loading 0:  89%|████████▉ | 452/507 [00:29<00:01, 36.93it/s]
Loading 0:  90%|█████████ | 457/507 [00:32<00:07,  6.33it/s]
Loading 0:  91%|█████████ | 460/507 [00:32<00:06,  7.43it/s]
Loading 0:  92%|█████████▏| 465/507 [00:32<00:04,  9.81it/s]
Loading 0:  93%|█████████▎| 472/507 [00:32<00:02, 14.52it/s]
Loading 0:  94%|█████████▍| 476/507 [00:32<00:01, 16.57it/s]
Loading 0:  95%|█████████▍| 481/507 [00:32<00:01, 20.12it/s]
Loading 0:  96%|█████████▌| 485/507 [00:32<00:01, 21.99it/s]
Loading 0:  97%|█████████▋| 490/507 [00:33<00:00, 26.08it/s]
Loading 0:  97%|█████████▋| 494/507 [00:33<00:00, 27.57it/s]
Loading 0:  98%|█████████▊| 499/507 [00:33<00:00, 31.16it/s]
Loading 0:  99%|█████████▉| 503/507 [00:33<00:00, 31.69it/s]
                                                            
Job mistralai-mistral-sma-53415-v105-mkmlizer completed after 176.13s with status: succeeded
Stopping job with name mistralai-mistral-sma-53415-v105-mkmlizer
Pipeline stage MKMLizer completed in 176.63s
run pipeline stage %s
Running pipeline stage MKMLTemplater
Pipeline stage MKMLTemplater completed in 0.16s
run pipeline stage %s
Running pipeline stage MKMLDeployer
Creating inference service mistralai-mistral-sma-53415-v105
Waiting for inference service mistralai-mistral-sma-53415-v105 to be ready
Inference service mistralai-mistral-sma-53415-v105 ready after 391.4121470451355s
Pipeline stage MKMLDeployer completed in 391.99s
run pipeline stage %s
Running pipeline stage StressChecker
Received healthy response to inference request in 2.99969220161438s
Received healthy response to inference request in 2.6187081336975098s
Received healthy response to inference request in 8.225366353988647s
Received healthy response to inference request in 3.116551637649536s
Received healthy response to inference request in 7.947529077529907s
5 requests
0 failed requests
5th percentile: 2.6949049472808837
10th percentile: 2.7711017608642576
20th percentile: 2.923495388031006
30th percentile: 3.0230640888214113
40th percentile: 3.0698078632354737
50th percentile: 3.116551637649536
60th percentile: 5.048942613601684
70th percentile: 6.981333589553832
80th percentile: 8.003096532821655
90th percentile: 8.114231443405151
95th percentile: 8.1697988986969
99th percentile: 8.214252862930298
mean time: 4.981569480895996
%s, retrying in %s seconds...
Received healthy response to inference request in 2.6282575130462646s
Received healthy response to inference request in 3.9943509101867676s
Received healthy response to inference request in 2.858546495437622s
Received healthy response to inference request in 7.945814371109009s
Received healthy response to inference request in 2.694396734237671s
5 requests
0 failed requests
5th percentile: 2.641485357284546
10th percentile: 2.654713201522827
20th percentile: 2.6811688899993897
30th percentile: 2.7272266864776613
40th percentile: 2.7928865909576417
50th percentile: 2.858546495437622
60th percentile: 3.31286826133728
70th percentile: 3.7671900272369383
80th percentile: 4.784643602371217
90th percentile: 6.365228986740113
95th percentile: 7.1555216789245595
99th percentile: 7.787755832672119
mean time: 4.024273204803467
%s, retrying in %s seconds...
Received healthy response to inference request in 2.746978998184204s
Received healthy response to inference request in 2.8656702041625977s
Received healthy response to inference request in 2.7049994468688965s
Received healthy response to inference request in 2.7291207313537598s
Received healthy response to inference request in 2.765864849090576s
5 requests
0 failed requests
5th percentile: 2.7098237037658692
10th percentile: 2.714647960662842
20th percentile: 2.724296474456787
30th percentile: 2.7326923847198485
40th percentile: 2.7398356914520265
50th percentile: 2.746978998184204
60th percentile: 2.7545333385467528
70th percentile: 2.762087678909302
80th percentile: 2.7858259201049806
90th percentile: 2.825748062133789
95th percentile: 2.845709133148193
99th percentile: 2.8616779899597167
mean time: 2.762526845932007
Pipeline stage StressChecker completed in 62.98s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyTriggerPipeline
run_pipeline:run_in_cloud %s
starting trigger_guanaco_pipeline args=%s
triggered trigger_guanaco_pipeline args=%s
Pipeline stage OfflineFamilyFriendlyTriggerPipeline completed in 0.68s
run pipeline stage %s
Running pipeline stage TriggerMKMLProfilingPipeline
run_pipeline:run_in_cloud %s
starting trigger_guanaco_pipeline args=%s
triggered trigger_guanaco_pipeline args=%s
Pipeline stage TriggerMKMLProfilingPipeline completed in 0.63s
Shutdown handler de-registered
mistralai-mistral-sma_53415_v105 status is now deployed due to DeploymentManager action
Shutdown handler registered
run pipeline %s
run pipeline stage %s
Running pipeline stage MKMLProfilerDeleter
Skipping teardown as no inference service was successfully deployed
Pipeline stage MKMLProfilerDeleter completed in 0.13s
run pipeline stage %s
Running pipeline stage MKMLProfilerTemplater
Pipeline stage MKMLProfilerTemplater completed in 0.11s
run pipeline stage %s
Running pipeline stage MKMLProfilerDeployer
Creating inference service mistralai-mistral-sma-53415-v105-profiler
Waiting for inference service mistralai-mistral-sma-53415-v105-profiler to be ready
Shutdown handler registered
run pipeline %s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyScorer
Evaluating %s Family Friendly Score with %s threads
Pipeline stage OfflineFamilyFriendlyScorer completed in 3434.92s
Shutdown handler de-registered
mistralai-mistral-sma_53415_v105 status is now inactive due to auto deactivation removed underperforming models
mistralai-mistral-sma_53415_v105 status is now torndown due to DeploymentManager action