mistralai-mistral-small_5341

developer_uid: azuruce
submission_id: mistralai-mistral-small_5341_v70
model_name: small-addict-bo8-param
model_group: mistralai/Mistral-Small-
status: torndown
timestamp: 2024-11-03T06:01:03+00:00
num_battles: 13709
num_wins: 6592
celo_rating: 1233.68
family_friendly_score: 0.5714
family_friendly_standard_error: 0.006998600431514861
submission_type: basic
model_repo: mistralai/Mistral-Small-Instruct-2409
model_architecture: MistralForCausalLM
model_num_parameters: 22247282688.0
best_of: 8
max_input_tokens: 1024
max_output_tokens: 64
display_name: small-addict-bo8-param
is_internal_developer: True
language_model: mistralai/Mistral-Small-Instruct-2409
model_size: 22B
ranking_group: single
us_pacific_date: 2024-11-02
win_ratio: 0.4808519950397549
generation_params: {'temperature': 0.9, 'top_p': 1.0, 'min_p': 0.05, 'top_k': 80, 'presence_penalty': 0.0, 'frequency_penalty': 0.0, 'stopping_words': ['\n', '</s>', '####', 'Bot:', 'User:', 'You:', '<|im_end|>', '<|eot_id|>'], 'max_input_tokens': 1024, 'best_of': 8, 'max_output_tokens': 64}
formatter: {'memory_template': '', 'prompt_template': '', 'bot_template': '{bot_name}: {message}\n', 'user_template': '{user_name}: {message}\n', 'response_template': '{bot_name}:', 'truncate_by_message': False}
Resubmit model
Shutdown handler not registered because Python interpreter is not running in the main thread
run pipeline %s
run pipeline stage %s
Running pipeline stage MKMLizer
Starting job with name mistralai-mistral-small-5341-v70-mkmlizer
Waiting for job on mistralai-mistral-small-5341-v70-mkmlizer to finish
mistralai-mistral-small-5341-v70-mkmlizer: ╔═════════════════════════════════════════════════════════════════════╗
mistralai-mistral-small-5341-v70-mkmlizer: ║     _____            __           __                                ║
mistralai-mistral-small-5341-v70-mkmlizer: ║    / _/ /_ ___    __/ /  ___ ___ / /                                ║
mistralai-mistral-small-5341-v70-mkmlizer: ║   / _/ / // / |/|/ / _ \/ -_) -_) /                                 ║
mistralai-mistral-small-5341-v70-mkmlizer: ║  /_//_/\_, /|__,__/_//_/\__/\__/_/                                  ║
mistralai-mistral-small-5341-v70-mkmlizer: ║       /___/                                                         ║
mistralai-mistral-small-5341-v70-mkmlizer: ║                                                                     ║
mistralai-mistral-small-5341-v70-mkmlizer: ║  Version: 0.11.12                                                   ║
mistralai-mistral-small-5341-v70-mkmlizer: ║  Copyright 2023 MK ONE TECHNOLOGIES Inc.                            ║
mistralai-mistral-small-5341-v70-mkmlizer: ║  https://mk1.ai                                                     ║
mistralai-mistral-small-5341-v70-mkmlizer: ║                                                                     ║
mistralai-mistral-small-5341-v70-mkmlizer: ║  The license key for the current software has been verified as      ║
mistralai-mistral-small-5341-v70-mkmlizer: ║  belonging to:                                                      ║
mistralai-mistral-small-5341-v70-mkmlizer: ║                                                                     ║
mistralai-mistral-small-5341-v70-mkmlizer: ║  Chai Research Corp.                                                ║
mistralai-mistral-small-5341-v70-mkmlizer: ║  Account ID: 7997a29f-0ceb-4cc7-9adf-840c57b4ae6f                   ║
mistralai-mistral-small-5341-v70-mkmlizer: ║  Expiration: 2025-01-15 23:59:59                                    ║
mistralai-mistral-small-5341-v70-mkmlizer: ║                                                                     ║
mistralai-mistral-small-5341-v70-mkmlizer: ╚═════════════════════════════════════════════════════════════════════╝
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
mistralai-mistral-small-5341-v70-mkmlizer: Downloaded to shared memory in 97.721s
mistralai-mistral-small-5341-v70-mkmlizer: quantizing model to /dev/shm/model_cache, profile:s0, folder:/tmp/tmpfa33eqi7, device:0
mistralai-mistral-small-5341-v70-mkmlizer: Saving flywheel model at /dev/shm/model_cache
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
mistralai-mistral-small-5341-v70-mkmlizer: quantized model in 46.407s
mistralai-mistral-small-5341-v70-mkmlizer: Processed model mistralai/Mistral-Small-Instruct-2409 in 144.127s
mistralai-mistral-small-5341-v70-mkmlizer: creating bucket guanaco-mkml-models
Connection pool is full, discarding connection: %s. Connection pool size: %s
mistralai-mistral-small-5341-v70-mkmlizer: Bucket 's3://guanaco-mkml-models/' created
mistralai-mistral-small-5341-v70-mkmlizer: uploading /dev/shm/model_cache to s3://guanaco-mkml-models/mistralai-mistral-small-5341-v70
mistralai-mistral-small-5341-v70-mkmlizer: cp /dev/shm/model_cache/special_tokens_map.json s3://guanaco-mkml-models/mistralai-mistral-small-5341-v70/special_tokens_map.json
mistralai-mistral-small-5341-v70-mkmlizer: cp /dev/shm/model_cache/config.json s3://guanaco-mkml-models/mistralai-mistral-small-5341-v70/config.json
mistralai-mistral-small-5341-v70-mkmlizer: cp /dev/shm/model_cache/tokenizer_config.json s3://guanaco-mkml-models/mistralai-mistral-small-5341-v70/tokenizer_config.json
mistralai-mistral-small-5341-v70-mkmlizer: cp /dev/shm/model_cache/tokenizer.model s3://guanaco-mkml-models/mistralai-mistral-small-5341-v70/tokenizer.model
mistralai-mistral-small-5341-v70-mkmlizer: cp /dev/shm/model_cache/tokenizer.json s3://guanaco-mkml-models/mistralai-mistral-small-5341-v70/tokenizer.json
mistralai-mistral-small-5341-v70-mkmlizer: cp /dev/shm/model_cache/flywheel_model.0.safetensors s3://guanaco-mkml-models/mistralai-mistral-small-5341-v70/flywheel_model.0.safetensors
mistralai-mistral-small-5341-v70-mkmlizer: 
Loading 0:   0%|          | 0/507 [00:00<?, ?it/s]
Loading 0:   1%|          | 5/507 [00:00<00:19, 25.20it/s]
Loading 0:   2%|▏         | 10/507 [00:00<00:14, 35.09it/s]
Loading 0:   3%|▎         | 14/507 [00:00<00:16, 29.49it/s]
Loading 0:   4%|▍         | 21/507 [00:00<00:12, 37.65it/s]
Loading 0:   5%|▍         | 25/507 [00:00<00:13, 34.62it/s]
Loading 0:   6%|▌         | 30/507 [00:00<00:13, 35.30it/s]
Loading 0:   7%|▋         | 34/507 [00:01<00:14, 33.16it/s]
Loading 0:   7%|▋         | 38/507 [00:01<00:13, 34.51it/s]
Loading 0:   8%|▊         | 42/507 [00:01<00:15, 29.80it/s]
Loading 0:   9%|▉         | 46/507 [00:01<00:14, 31.92it/s]
Loading 0:  10%|▉         | 50/507 [00:01<00:15, 29.62it/s]
Loading 0:  11%|█         | 54/507 [00:01<00:19, 23.05it/s]
Loading 0:  11%|█         | 57/507 [00:01<00:19, 22.78it/s]
Loading 0:  12%|█▏        | 61/507 [00:02<00:17, 26.18it/s]
Loading 0:  13%|█▎        | 65/507 [00:02<00:17, 25.27it/s]
Loading 0:  14%|█▍        | 70/507 [00:02<00:14, 30.04it/s]
Loading 0:  15%|█▍        | 75/507 [00:02<00:12, 33.58it/s]
Loading 0:  16%|█▌        | 80/507 [00:02<00:13, 32.67it/s]
Loading 0:  17%|█▋        | 85/507 [00:02<00:11, 36.47it/s]
Loading 0:  18%|█▊        | 89/507 [00:02<00:13, 31.88it/s]
Loading 0:  19%|█▊        | 94/507 [00:02<00:11, 36.05it/s]
Loading 0:  19%|█▉        | 98/507 [00:03<00:12, 31.89it/s]
Loading 0:  20%|██        | 103/507 [00:03<00:11, 35.08it/s]
Loading 0:  21%|██        | 107/507 [00:03<00:12, 31.14it/s]
Loading 0:  22%|██▏       | 112/507 [00:03<00:11, 34.19it/s]
Loading 0:  23%|██▎       | 116/507 [00:03<00:16, 23.36it/s]
Loading 0:  24%|██▍       | 122/507 [00:04<00:14, 25.74it/s]
Loading 0:  25%|██▌       | 127/507 [00:04<00:12, 29.85it/s]
Loading 0:  26%|██▌       | 131/507 [00:04<00:13, 27.97it/s]
Loading 0:  27%|██▋       | 136/507 [00:04<00:11, 31.99it/s]
Loading 0:  28%|██▊       | 140/507 [00:04<00:12, 29.46it/s]
Loading 0:  29%|██▊       | 145/507 [00:04<00:10, 33.78it/s]
Loading 0:  29%|██▉       | 149/507 [00:04<00:11, 30.64it/s]
Loading 0:  30%|███       | 154/507 [00:04<00:10, 34.50it/s]
Loading 0:  31%|███       | 158/507 [00:05<00:11, 30.50it/s]
Loading 0:  32%|███▏      | 163/507 [00:05<00:10, 34.39it/s]
Loading 0:  33%|███▎      | 168/507 [00:05<00:09, 35.09it/s]
Loading 0:  34%|███▍      | 172/507 [00:05<00:13, 24.37it/s]
Loading 0:  35%|███▍      | 176/507 [00:05<00:13, 23.87it/s]
Loading 0:  36%|███▌      | 181/507 [00:05<00:11, 28.09it/s]
Loading 0:  36%|███▋      | 185/507 [00:06<00:11, 27.36it/s]
Loading 0:  38%|███▊      | 192/507 [00:06<00:09, 33.95it/s]
Loading 0:  39%|███▊      | 196/507 [00:06<00:09, 33.28it/s]
Loading 0:  40%|███▉      | 201/507 [00:06<00:08, 34.92it/s]
Loading 0:  40%|████      | 205/507 [00:06<00:09, 33.31it/s]
Loading 0:  41%|████▏     | 210/507 [00:06<00:08, 33.96it/s]
Loading 0:  42%|████▏     | 214/507 [00:06<00:08, 32.90it/s]
Loading 0:  43%|████▎     | 218/507 [00:07<00:08, 32.39it/s]
Loading 0:  44%|████▍     | 222/507 [00:07<00:08, 32.50it/s]
Loading 0:  45%|████▍     | 226/507 [00:07<00:11, 25.16it/s]
Loading 0:  45%|████▌     | 230/507 [00:07<00:11, 24.92it/s]
Loading 0:  46%|████▋     | 235/507 [00:07<00:09, 29.96it/s]
Loading 0:  47%|████▋     | 239/507 [00:07<00:09, 27.77it/s]
Loading 0:  48%|████▊     | 245/507 [00:07<00:07, 34.80it/s]
Loading 0:  49%|████▉     | 249/507 [00:08<00:08, 31.17it/s]
Loading 0:  50%|████▉     | 253/507 [00:08<00:07, 32.90it/s]
Loading 0:  51%|█████     | 257/507 [00:08<00:09, 27.78it/s]
Loading 0:  52%|█████▏    | 262/507 [00:08<00:07, 31.94it/s]
Loading 0:  52%|█████▏    | 266/507 [00:08<00:08, 28.68it/s]
Loading 0:  53%|█████▎    | 271/507 [00:08<00:07, 33.06it/s]
Loading 0:  54%|█████▍    | 275/507 [00:09<00:07, 29.74it/s]
Loading 0:  55%|█████▌    | 281/507 [00:09<00:06, 36.07it/s]
Loading 0:  56%|█████▌    | 285/507 [00:09<00:08, 25.59it/s]
Loading 0:  57%|█████▋    | 289/507 [00:09<00:08, 25.68it/s]
Loading 0:  58%|█████▊    | 293/507 [00:09<00:08, 25.01it/s]
Loading 0:  59%|█████▉    | 298/507 [00:09<00:06, 29.92it/s]
Loading 0:  59%|█████▉    | 299/507 [00:20<00:06, 29.92it/s]
Loading 0:  59%|█████▉    | 300/507 [00:24<04:09,  1.21s/it]
Loading 0:  60%|█████▉    | 302/507 [00:24<03:24,  1.00it/s]
Loading 0:  61%|██████    | 307/507 [00:24<02:02,  1.63it/s]
Loading 0:  61%|██████    | 310/507 [00:25<01:32,  2.13it/s]
Loading 0:  62%|██████▏   | 313/507 [00:25<01:08,  2.81it/s]
Loading 0:  62%|██████▏   | 316/507 [00:25<00:51,  3.74it/s]
Loading 0:  63%|██████▎   | 320/507 [00:25<00:35,  5.24it/s]
Loading 0:  64%|██████▍   | 325/507 [00:25<00:23,  7.89it/s]
Loading 0:  65%|██████▍   | 329/507 [00:25<00:18,  9.84it/s]
Loading 0:  66%|██████▌   | 334/507 [00:25<00:12, 13.44it/s]
Loading 0:  67%|██████▋   | 339/507 [00:26<00:10, 16.65it/s]
Loading 0:  68%|██████▊   | 343/507 [00:26<00:10, 15.34it/s]
Loading 0:  68%|██████▊   | 347/507 [00:26<00:09, 16.97it/s]
Loading 0:  69%|██████▉   | 352/507 [00:26<00:07, 21.69it/s]
Loading 0:  70%|███████   | 356/507 [00:26<00:06, 22.34it/s]
Loading 0:  71%|███████   | 361/507 [00:26<00:05, 27.30it/s]
Loading 0:  72%|███████▏  | 365/507 [00:27<00:05, 26.52it/s]
Loading 0:  73%|███████▎  | 370/507 [00:27<00:04, 31.04it/s]
Loading 0:  74%|███████▍  | 374/507 [00:27<00:04, 28.70it/s]
Loading 0:  75%|███████▍  | 379/507 [00:27<00:04, 31.82it/s]
Loading 0:  76%|███████▌  | 383/507 [00:27<00:04, 28.86it/s]
Loading 0:  77%|███████▋  | 388/507 [00:27<00:03, 33.34it/s]
Loading 0:  77%|███████▋  | 392/507 [00:27<00:03, 31.64it/s]
Loading 0:  78%|███████▊  | 396/507 [00:28<00:04, 25.12it/s]
Loading 0:  79%|███████▉  | 401/507 [00:28<00:03, 26.64it/s]
Loading 0:  80%|████████  | 406/507 [00:28<00:03, 31.07it/s]
Loading 0:  81%|████████  | 410/507 [00:28<00:03, 28.91it/s]
Loading 0:  82%|████████▏ | 415/507 [00:28<00:02, 33.25it/s]
Loading 0:  83%|████████▎ | 419/507 [00:28<00:02, 30.35it/s]
Loading 0:  84%|████████▎ | 424/507 [00:28<00:02, 34.64it/s]
Loading 0:  84%|████████▍ | 428/507 [00:29<00:02, 31.01it/s]
Loading 0:  85%|████████▌ | 433/507 [00:29<00:02, 35.08it/s]
Loading 0:  86%|████████▌ | 437/507 [00:29<00:02, 31.41it/s]
Loading 0:  87%|████████▋ | 442/507 [00:29<00:01, 35.26it/s]
Loading 0:  88%|████████▊ | 446/507 [00:29<00:01, 30.93it/s]
Loading 0:  89%|████████▉ | 453/507 [00:29<00:01, 38.47it/s]
Loading 0:  90%|█████████ | 458/507 [00:32<00:07,  6.61it/s]
Loading 0:  91%|█████████ | 461/507 [00:32<00:05,  7.74it/s]
Loading 0:  92%|█████████▏| 465/507 [00:32<00:04,  9.53it/s]
Loading 0:  93%|█████████▎| 470/507 [00:32<00:02, 12.95it/s]
Loading 0:  93%|█████████▎| 474/507 [00:32<00:02, 14.92it/s]
Loading 0:  95%|█████████▍| 481/507 [00:32<00:01, 21.09it/s]
Loading 0:  96%|█████████▌| 485/507 [00:32<00:00, 22.87it/s]
Loading 0:  97%|█████████▋| 490/507 [00:32<00:00, 26.39it/s]
Loading 0:  97%|█████████▋| 494/507 [00:33<00:00, 27.81it/s]
Loading 0:  98%|█████████▊| 499/507 [00:33<00:00, 31.10it/s]
Loading 0:  99%|█████████▉| 503/507 [00:33<00:00, 30.78it/s]
                                                            
Job mistralai-mistral-small-5341-v70-mkmlizer completed after 175.75s with status: succeeded
Stopping job with name mistralai-mistral-small-5341-v70-mkmlizer
Pipeline stage MKMLizer completed in 176.34s
run pipeline stage %s
Running pipeline stage MKMLTemplater
Pipeline stage MKMLTemplater completed in 0.21s
run pipeline stage %s
Running pipeline stage MKMLDeployer
Creating inference service mistralai-mistral-small-5341-v70
Waiting for inference service mistralai-mistral-small-5341-v70 to be ready
Inference service mistralai-mistral-small-5341-v70 ready after 151.2010543346405s
Pipeline stage MKMLDeployer completed in 151.83s
run pipeline stage %s
Running pipeline stage StressChecker
Received healthy response to inference request in 2.3114326000213623s
Received healthy response to inference request in 2.0018107891082764s
Failed to get response for submission mistralai-mistral-nemo_9330_v186: ('http://mistralai-mistral-nemo-9330-v186-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'upstream connect error or disconnect/reset before headers. reset reason: connection timeout')
Received healthy response to inference request in 2.7565672397613525s
Received healthy response to inference request in 1.4743878841400146s
Received healthy response to inference request in 2.2911274433135986s
5 requests
0 failed requests
5th percentile: 1.579872465133667
10th percentile: 1.6853570461273193
20th percentile: 1.896326208114624
30th percentile: 2.059674119949341
40th percentile: 2.1754007816314695
50th percentile: 2.2911274433135986
60th percentile: 2.299249505996704
70th percentile: 2.3073715686798097
80th percentile: 2.4004595279693604
90th percentile: 2.5785133838653564
95th percentile: 2.6675403118133545
99th percentile: 2.738761854171753
mean time: 2.1670651912689207
Pipeline stage StressChecker completed in 12.36s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyTriggerPipeline
run_pipeline:run_in_cloud %s
starting trigger_guanaco_pipeline args=%s
triggered trigger_guanaco_pipeline args=%s
Pipeline stage OfflineFamilyFriendlyTriggerPipeline completed in 2.68s
Shutdown handler de-registered
mistralai-mistral-small_5341_v70 status is now deployed due to DeploymentManager action
Shutdown handler registered
run pipeline %s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyScorer
Evaluating %s Family Friendly Score with %s threads
Pipeline stage OfflineFamilyFriendlyScorer completed in 3230.02s
Shutdown handler de-registered
mistralai-mistral-small_5341_v70 status is now inactive due to auto deactivation removed underperforming models
mistralai-mistral-small_5341_v70 status is now torndown due to DeploymentManager action