mistralai-mistral-small_5341

developer_uid: azuruce
submission_id: mistralai-mistral-small_5341_v69
model_name: small-ba
model_group: mistralai/Mistral-Small-
status: torndown
timestamp: 2024-11-03T05:33:48+00:00
num_battles: 9099
num_wins: 4342
celo_rating: 1224.47
family_friendly_score: 0.5828
family_friendly_standard_error: 0.006973437602789603
submission_type: basic
model_repo: mistralai/Mistral-Small-Instruct-2409
model_architecture: MistralForCausalLM
model_num_parameters: 22247282688.0
best_of: 4
max_input_tokens: 1024
max_output_tokens: 64
display_name: small-ba
is_internal_developer: True
language_model: mistralai/Mistral-Small-Instruct-2409
model_size: 22B
ranking_group: single
us_pacific_date: 2024-11-02
win_ratio: 0.4771952961863941
generation_params: {'temperature': 1.0, 'top_p': 1.0, 'min_p': 0.0, 'top_k': 40, 'presence_penalty': 0.0, 'frequency_penalty': 0.0, 'stopping_words': ['\n', '<|eot_id|>', '<|end_of_text|>', 'You:'], 'max_input_tokens': 1024, 'best_of': 4, 'max_output_tokens': 64}
formatter: {'memory_template': '', 'prompt_template': '', 'bot_template': '{bot_name}: {message}\n', 'user_template': '{user_name}: {message}\n', 'response_template': '### Response:\n{bot_name}:', 'truncate_by_message': False}
Resubmit model
Shutdown handler not registered because Python interpreter is not running in the main thread
run pipeline %s
run pipeline stage %s
Running pipeline stage MKMLizer
Starting job with name mistralai-mistral-small-5341-v69-mkmlizer
Waiting for job on mistralai-mistral-small-5341-v69-mkmlizer to finish
mistralai-mistral-small-5341-v69-mkmlizer: ╔═════════════════════════════════════════════════════════════════════╗
mistralai-mistral-small-5341-v69-mkmlizer: ║     _____            __           __                                ║
mistralai-mistral-small-5341-v69-mkmlizer: ║    / _/ /_ ___    __/ /  ___ ___ / /                                ║
mistralai-mistral-small-5341-v69-mkmlizer: ║   / _/ / // / |/|/ / _ \/ -_) -_) /                                 ║
mistralai-mistral-small-5341-v69-mkmlizer: ║  /_//_/\_, /|__,__/_//_/\__/\__/_/                                  ║
mistralai-mistral-small-5341-v69-mkmlizer: ║       /___/                                                         ║
mistralai-mistral-small-5341-v69-mkmlizer: ║                                                                     ║
mistralai-mistral-small-5341-v69-mkmlizer: ║  Version: 0.11.12                                                   ║
mistralai-mistral-small-5341-v69-mkmlizer: ║  Copyright 2023 MK ONE TECHNOLOGIES Inc.                            ║
mistralai-mistral-small-5341-v69-mkmlizer: ║  https://mk1.ai                                                     ║
mistralai-mistral-small-5341-v69-mkmlizer: ║                                                                     ║
mistralai-mistral-small-5341-v69-mkmlizer: ║  The license key for the current software has been verified as      ║
mistralai-mistral-small-5341-v69-mkmlizer: ║  belonging to:                                                      ║
mistralai-mistral-small-5341-v69-mkmlizer: ║                                                                     ║
mistralai-mistral-small-5341-v69-mkmlizer: ║  Chai Research Corp.                                                ║
mistralai-mistral-small-5341-v69-mkmlizer: ║  Account ID: 7997a29f-0ceb-4cc7-9adf-840c57b4ae6f                   ║
mistralai-mistral-small-5341-v69-mkmlizer: ║  Expiration: 2025-01-15 23:59:59                                    ║
mistralai-mistral-small-5341-v69-mkmlizer: ║                                                                     ║
mistralai-mistral-small-5341-v69-mkmlizer: ╚═════════════════════════════════════════════════════════════════════╝
Failed to get response for submission chaiml-nemo-20241010-tie_5991_v2: ('http://chaiml-nemo-20241010-tie-5991-v2-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'read tcp 127.0.0.1:55110->127.0.0.1:8080: read: connection reset by peer\n')
Connection pool is full, discarding connection: %s. Connection pool size: %s
mistralai-mistral-small-5341-v69-mkmlizer: Downloaded to shared memory in 89.531s
mistralai-mistral-small-5341-v69-mkmlizer: quantizing model to /dev/shm/model_cache, profile:s0, folder:/tmp/tmpdf0_vocw, device:0
mistralai-mistral-small-5341-v69-mkmlizer: Saving flywheel model at /dev/shm/model_cache
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
mistralai-mistral-small-5341-v69-mkmlizer: quantized model in 44.639s
mistralai-mistral-small-5341-v69-mkmlizer: Processed model mistralai/Mistral-Small-Instruct-2409 in 134.170s
mistralai-mistral-small-5341-v69-mkmlizer: creating bucket guanaco-mkml-models
mistralai-mistral-small-5341-v69-mkmlizer: Bucket 's3://guanaco-mkml-models/' created
mistralai-mistral-small-5341-v69-mkmlizer: uploading /dev/shm/model_cache to s3://guanaco-mkml-models/mistralai-mistral-small-5341-v69
mistralai-mistral-small-5341-v69-mkmlizer: cp /dev/shm/model_cache/config.json s3://guanaco-mkml-models/mistralai-mistral-small-5341-v69/config.json
mistralai-mistral-small-5341-v69-mkmlizer: cp /dev/shm/model_cache/special_tokens_map.json s3://guanaco-mkml-models/mistralai-mistral-small-5341-v69/special_tokens_map.json
mistralai-mistral-small-5341-v69-mkmlizer: cp /dev/shm/model_cache/tokenizer_config.json s3://guanaco-mkml-models/mistralai-mistral-small-5341-v69/tokenizer_config.json
mistralai-mistral-small-5341-v69-mkmlizer: cp /dev/shm/model_cache/tokenizer.model s3://guanaco-mkml-models/mistralai-mistral-small-5341-v69/tokenizer.model
mistralai-mistral-small-5341-v69-mkmlizer: cp /dev/shm/model_cache/tokenizer.json s3://guanaco-mkml-models/mistralai-mistral-small-5341-v69/tokenizer.json
mistralai-mistral-small-5341-v69-mkmlizer: cp /dev/shm/model_cache/flywheel_model.1.safetensors s3://guanaco-mkml-models/mistralai-mistral-small-5341-v69/flywheel_model.1.safetensors
mistralai-mistral-small-5341-v69-mkmlizer: cp /dev/shm/model_cache/flywheel_model.0.safetensors s3://guanaco-mkml-models/mistralai-mistral-small-5341-v69/flywheel_model.0.safetensors
mistralai-mistral-small-5341-v69-mkmlizer: 
Loading 0:   0%|          | 0/507 [00:00<?, ?it/s]
Loading 0:   1%|          | 5/507 [00:00<00:19, 25.24it/s]
Loading 0:   2%|▏         | 10/507 [00:00<00:14, 34.73it/s]
Loading 0:   3%|▎         | 14/507 [00:00<00:16, 29.98it/s]
Loading 0:   4%|▍         | 21/507 [00:00<00:12, 38.43it/s]
Loading 0:   5%|▌         | 26/507 [00:00<00:13, 36.34it/s]
Loading 0:   6%|▌         | 30/507 [00:00<00:13, 35.71it/s]
Loading 0:   7%|▋         | 34/507 [00:01<00:14, 33.18it/s]
Loading 0:   8%|▊         | 39/507 [00:01<00:13, 35.23it/s]
Loading 0:   8%|▊         | 43/507 [00:01<00:13, 34.06it/s]
Loading 0:   9%|▉         | 47/507 [00:01<00:13, 33.75it/s]
Loading 0:  10%|█         | 51/507 [00:01<00:13, 34.11it/s]
Loading 0:  11%|█         | 55/507 [00:01<00:17, 25.24it/s]
Loading 0:  11%|█▏        | 58/507 [00:01<00:17, 26.12it/s]
Loading 0:  12%|█▏        | 63/507 [00:01<00:15, 29.57it/s]
Loading 0:  13%|█▎        | 67/507 [00:02<00:14, 30.62it/s]
Loading 0:  14%|█▍        | 73/507 [00:02<00:12, 33.90it/s]
Loading 0:  16%|█▌        | 80/507 [00:02<00:11, 36.06it/s]
Loading 0:  17%|█▋        | 87/507 [00:02<00:10, 41.62it/s]
Loading 0:  18%|█▊        | 92/507 [00:02<00:10, 40.34it/s]
Loading 0:  19%|█▉        | 97/507 [00:02<00:10, 39.74it/s]
Loading 0:  20%|██        | 102/507 [00:02<00:09, 40.72it/s]
Loading 0:  21%|██        | 107/507 [00:03<00:11, 34.54it/s]
Loading 0:  22%|██▏       | 113/507 [00:03<00:12, 31.39it/s]
Loading 0:  23%|██▎       | 117/507 [00:03<00:13, 29.02it/s]
Loading 0:  24%|██▍       | 122/507 [00:03<00:13, 29.52it/s]
Loading 0:  25%|██▌       | 129/507 [00:03<00:10, 35.28it/s]
Loading 0:  26%|██▌       | 133/507 [00:03<00:10, 34.93it/s]
Loading 0:  27%|██▋       | 138/507 [00:04<00:09, 37.11it/s]
Loading 0:  28%|██▊       | 142/507 [00:04<00:09, 36.60it/s]
Loading 0:  29%|██▉       | 147/507 [00:04<00:09, 38.58it/s]
Loading 0:  30%|██▉       | 151/507 [00:04<00:09, 36.42it/s]
Loading 0:  31%|███       | 156/507 [00:04<00:09, 38.04it/s]
Loading 0:  32%|███▏      | 160/507 [00:04<00:09, 36.57it/s]
Loading 0:  32%|███▏      | 164/507 [00:04<00:10, 34.24it/s]
Loading 0:  33%|███▎      | 168/507 [00:04<00:09, 35.47it/s]
Loading 0:  34%|███▍      | 172/507 [00:05<00:13, 23.95it/s]
Loading 0:  35%|███▍      | 176/507 [00:05<00:13, 23.65it/s]
Loading 0:  36%|███▌      | 183/507 [00:05<00:10, 30.80it/s]
Loading 0:  37%|███▋      | 187/507 [00:05<00:10, 30.37it/s]
Loading 0:  38%|███▊      | 192/507 [00:05<00:09, 32.39it/s]
Loading 0:  39%|███▊      | 196/507 [00:05<00:09, 31.48it/s]
Loading 0:  40%|███▉      | 201/507 [00:06<00:08, 34.52it/s]
Loading 0:  40%|████      | 205/507 [00:06<00:08, 34.29it/s]
Loading 0:  41%|████▏     | 210/507 [00:06<00:08, 36.09it/s]
Loading 0:  42%|████▏     | 214/507 [00:06<00:08, 35.08it/s]
Loading 0:  43%|████▎     | 218/507 [00:06<00:08, 34.51it/s]
Loading 0:  44%|████▍     | 222/507 [00:06<00:08, 34.00it/s]
Loading 0:  45%|████▍     | 226/507 [00:06<00:11, 25.16it/s]
Loading 0:  45%|████▌     | 230/507 [00:07<00:10, 25.39it/s]
Loading 0:  47%|████▋     | 237/507 [00:07<00:08, 32.70it/s]
Loading 0:  48%|████▊     | 241/507 [00:07<00:08, 32.97it/s]
Loading 0:  49%|████▊     | 246/507 [00:07<00:07, 34.39it/s]
Loading 0:  49%|████▉     | 250/507 [00:07<00:07, 33.29it/s]
Loading 0:  50%|█████     | 255/507 [00:07<00:06, 36.05it/s]
Loading 0:  51%|█████     | 259/507 [00:07<00:07, 35.32it/s]
Loading 0:  52%|█████▏    | 264/507 [00:07<00:06, 36.93it/s]
Loading 0:  53%|█████▎    | 268/507 [00:08<00:06, 36.19it/s]
Loading 0:  54%|█████▍    | 273/507 [00:08<00:06, 38.82it/s]
Loading 0:  55%|█████▍    | 277/507 [00:08<00:06, 37.33it/s]
Loading 0:  56%|█████▌    | 282/507 [00:08<00:05, 39.70it/s]
Loading 0:  57%|█████▋    | 287/507 [00:08<00:08, 25.73it/s]
Loading 0:  58%|█████▊    | 293/507 [00:08<00:07, 28.80it/s]
Loading 0:  59%|█████▉    | 299/507 [00:23<00:07, 28.80it/s]
Loading 0:  59%|█████▉    | 300/507 [00:23<02:41,  1.28it/s]
Loading 0:  60%|█████▉    | 302/507 [00:23<02:21,  1.45it/s]
Loading 0:  61%|██████    | 307/507 [00:23<01:36,  2.08it/s]
Loading 0:  61%|██████    | 310/507 [00:23<01:16,  2.58it/s]
Loading 0:  62%|██████▏   | 313/507 [00:23<00:59,  3.27it/s]
Loading 0:  63%|██████▎   | 318/507 [00:23<00:38,  4.86it/s]
Loading 0:  63%|██████▎   | 321/507 [00:24<00:31,  5.98it/s]
Loading 0:  64%|██████▍   | 325/507 [00:24<00:22,  8.08it/s]
Loading 0:  65%|██████▍   | 329/507 [00:24<00:17, 10.25it/s]
Loading 0:  66%|██████▌   | 335/507 [00:24<00:11, 14.87it/s]
Loading 0:  67%|██████▋   | 340/507 [00:24<00:10, 16.52it/s]
Loading 0:  68%|██████▊   | 344/507 [00:24<00:08, 19.01it/s]
Loading 0:  69%|██████▊   | 348/507 [00:24<00:07, 20.55it/s]
Loading 0:  70%|██████▉   | 354/507 [00:25<00:05, 25.74it/s]
Loading 0:  71%|███████   | 358/507 [00:25<00:05, 27.32it/s]
Loading 0:  72%|███████▏  | 363/507 [00:25<00:04, 30.85it/s]
Loading 0:  72%|███████▏  | 367/507 [00:25<00:04, 31.57it/s]
Loading 0:  73%|███████▎  | 372/507 [00:25<00:03, 34.36it/s]
Loading 0:  74%|███████▍  | 376/507 [00:25<00:04, 32.10it/s]
Loading 0:  75%|███████▌  | 381/507 [00:25<00:03, 33.69it/s]
Loading 0:  76%|███████▌  | 385/507 [00:25<00:03, 32.57it/s]
Loading 0:  77%|███████▋  | 389/507 [00:26<00:03, 33.02it/s]
Loading 0:  78%|███████▊  | 393/507 [00:26<00:03, 33.66it/s]
Loading 0:  78%|███████▊  | 397/507 [00:26<00:04, 25.95it/s]
Loading 0:  79%|███████▉  | 401/507 [00:26<00:04, 25.63it/s]
Loading 0:  80%|████████  | 408/507 [00:26<00:02, 33.47it/s]
Loading 0:  81%|████████▏ | 412/507 [00:26<00:02, 33.35it/s]
Loading 0:  82%|████████▏ | 417/507 [00:26<00:02, 35.11it/s]
Loading 0:  83%|████████▎ | 421/507 [00:27<00:02, 34.48it/s]
Loading 0:  84%|████████▍ | 426/507 [00:27<00:02, 37.36it/s]
Loading 0:  85%|████████▍ | 430/507 [00:27<00:02, 36.33it/s]
Loading 0:  86%|████████▌ | 435/507 [00:27<00:01, 37.59it/s]
Loading 0:  87%|████████▋ | 439/507 [00:27<00:01, 36.87it/s]
Loading 0:  88%|████████▊ | 444/507 [00:27<00:01, 39.52it/s]
Loading 0:  88%|████████▊ | 448/507 [00:27<00:01, 38.12it/s]
Loading 0:  89%|████████▉ | 453/507 [00:27<00:01, 41.29it/s]
Loading 0:  90%|█████████ | 458/507 [00:30<00:07,  6.36it/s]
Loading 0:  91%|█████████ | 461/507 [00:30<00:06,  7.60it/s]
Loading 0:  92%|█████████▏| 465/507 [00:30<00:04,  9.66it/s]
Loading 0:  93%|█████████▎| 472/507 [00:30<00:02, 14.91it/s]
Loading 0:  94%|█████████▍| 476/507 [00:30<00:01, 17.45it/s]
Loading 0:  95%|█████████▍| 481/507 [00:30<00:01, 21.20it/s]
Loading 0:  96%|█████████▌| 485/507 [00:30<00:00, 23.15it/s]
Loading 0:  97%|█████████▋| 490/507 [00:30<00:00, 27.52it/s]
Loading 0:  97%|█████████▋| 494/507 [00:31<00:00, 27.59it/s]
Loading 0:  98%|█████████▊| 499/507 [00:31<00:00, 30.66it/s]
Loading 0:  99%|█████████▉| 503/507 [00:31<00:00, 31.32it/s]
                                                            
Job mistralai-mistral-small-5341-v69-mkmlizer completed after 165.96s with status: succeeded
Stopping job with name mistralai-mistral-small-5341-v69-mkmlizer
Pipeline stage MKMLizer completed in 166.64s
run pipeline stage %s
Running pipeline stage MKMLTemplater
Pipeline stage MKMLTemplater completed in 0.21s
run pipeline stage %s
Running pipeline stage MKMLDeployer
Creating inference service mistralai-mistral-small-5341-v69
Waiting for inference service mistralai-mistral-small-5341-v69 to be ready
Inference service mistralai-mistral-small-5341-v69 ready after 150.6823444366455s
Pipeline stage MKMLDeployer completed in 151.31s
run pipeline stage %s
Running pipeline stage StressChecker
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
HTTPConnectionPool(host='guanaco-submitter.guanaco-backend.k2.chaiverse.com', port=80): Read timed out. (read timeout=20)
Received unhealthy response to inference request!
Received healthy response to inference request in 2.8026435375213623s
Received healthy response to inference request in 2.300854206085205s
Received healthy response to inference request in 2.049652099609375s
Received healthy response to inference request in 2.237321138381958s
5 requests
1 failed requests
5th percentile: 2.0871859073638914
10th percentile: 2.1247197151184083
20th percentile: 2.1997873306274416
30th percentile: 2.2500277519226075
40th percentile: 2.275440979003906
50th percentile: 2.300854206085205
60th percentile: 2.501569938659668
70th percentile: 2.7022856712341308
80th percentile: 6.329035043716434
90th percentile: 13.38181805610657
95th percentile: 16.908209562301632
99th percentile: 19.72932276725769
mean time: 5.965014410018921
%s, retrying in %s seconds...
Received healthy response to inference request in 1.6894762516021729s
Received healthy response to inference request in 2.37803053855896s
Received healthy response to inference request in 2.7168734073638916s
Received healthy response to inference request in 3.128556728363037s
Received healthy response to inference request in 2.337890625s
5 requests
0 failed requests
5th percentile: 1.8191591262817384
10th percentile: 1.9488420009613037
20th percentile: 2.2082077503204345
30th percentile: 2.345918607711792
40th percentile: 2.361974573135376
50th percentile: 2.37803053855896
60th percentile: 2.5135676860809326
70th percentile: 2.6491048336029053
80th percentile: 2.7992100715637207
90th percentile: 2.963883399963379
95th percentile: 3.046220064163208
99th percentile: 3.1120893955230713
mean time: 2.4501655101776123
Pipeline stage StressChecker completed in 45.28s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyTriggerPipeline
run_pipeline:run_in_cloud %s
starting trigger_guanaco_pipeline args=%s
triggered trigger_guanaco_pipeline args=%s
Pipeline stage OfflineFamilyFriendlyTriggerPipeline completed in 2.52s
Shutdown handler de-registered
mistralai-mistral-small_5341_v69 status is now deployed due to DeploymentManager action
Shutdown handler registered
run pipeline %s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyScorer
Evaluating %s Family Friendly Score with %s threads
Pipeline stage OfflineFamilyFriendlyScorer completed in 2984.17s
Shutdown handler de-registered
mistralai-mistral-small_5341_v69 status is now inactive due to auto deactivation removed underperforming models
mistralai-mistral-small_5341_v69 status is now torndown due to DeploymentManager action