nbeerbower-mistral-small_1103

developer_uid: zonemercy
submission_id: nbeerbower-mistral-small_1103_v1
model_name: nbeerbower-mistral-small_1103_v1
model_group: nbeerbower/Mistral-Small
status: torndown
timestamp: 2024-10-10T22:56:05+00:00
num_battles: 6277
num_wins: 2974
celo_rating: 1232.97
family_friendly_score: 0.5587890951664158
family_friendly_standard_error: 0.006387764842049729
submission_type: basic
model_repo: nbeerbower/Mistral-Small-Gutenberg-Doppel-22B
model_architecture: MistralForCausalLM
model_num_parameters: 22247282688.0
best_of: 8
max_input_tokens: 1024
max_output_tokens: 64
display_name: nbeerbower-mistral-small_1103_v1
is_internal_developer: True
language_model: nbeerbower/Mistral-Small-Gutenberg-Doppel-22B
model_size: 22B
ranking_group: single
us_pacific_date: 2024-10-10
win_ratio: 0.47379321331846425
generation_params: {'temperature': 1.0, 'top_p': 1.0, 'min_p': 0.0, 'top_k': 40, 'presence_penalty': 0.0, 'frequency_penalty': 0.0, 'stopping_words': ['\n', '</s>', '####', 'Bot:', 'User:', 'You:', '<|im_end|>', '<|eot_id|>'], 'max_input_tokens': 1024, 'best_of': 8, 'max_output_tokens': 64}
formatter: {'memory_template': '', 'prompt_template': '', 'bot_template': '{bot_name}: {message}\n', 'user_template': '{user_name}: {message}\n', 'response_template': '{bot_name}:', 'truncate_by_message': False}
Resubmit model
Shutdown handler not registered because Python interpreter is not running in the main thread
run pipeline %s
run pipeline stage %s
Running pipeline stage MKMLizer
Starting job with name nbeerbower-mistral-small-1103-v1-mkmlizer
Waiting for job on nbeerbower-mistral-small-1103-v1-mkmlizer to finish
nbeerbower-mistral-small-1103-v1-mkmlizer: ╔═════════════════════════════════════════════════════════════════════╗
nbeerbower-mistral-small-1103-v1-mkmlizer: ║     _____            __           __                                ║
nbeerbower-mistral-small-1103-v1-mkmlizer: ║    / _/ /_ ___    __/ /  ___ ___ / /                                ║
nbeerbower-mistral-small-1103-v1-mkmlizer: ║   / _/ / // / |/|/ / _ \/ -_) -_) /                                 ║
nbeerbower-mistral-small-1103-v1-mkmlizer: ║  /_//_/\_, /|__,__/_//_/\__/\__/_/                                  ║
nbeerbower-mistral-small-1103-v1-mkmlizer: ║       /___/                                                         ║
nbeerbower-mistral-small-1103-v1-mkmlizer: ║                                                                     ║
nbeerbower-mistral-small-1103-v1-mkmlizer: ║  Version: 0.11.12                                                   ║
nbeerbower-mistral-small-1103-v1-mkmlizer: ║  Copyright 2023 MK ONE TECHNOLOGIES Inc.                            ║
nbeerbower-mistral-small-1103-v1-mkmlizer: ║  https://mk1.ai                                                     ║
nbeerbower-mistral-small-1103-v1-mkmlizer: ║                                                                     ║
nbeerbower-mistral-small-1103-v1-mkmlizer: ║  The license key for the current software has been verified as      ║
nbeerbower-mistral-small-1103-v1-mkmlizer: ║  belonging to:                                                      ║
nbeerbower-mistral-small-1103-v1-mkmlizer: ║                                                                     ║
nbeerbower-mistral-small-1103-v1-mkmlizer: ║  Chai Research Corp.                                                ║
nbeerbower-mistral-small-1103-v1-mkmlizer: ║  Account ID: 7997a29f-0ceb-4cc7-9adf-840c57b4ae6f                   ║
nbeerbower-mistral-small-1103-v1-mkmlizer: ║  Expiration: 2024-10-15 23:59:59                                    ║
nbeerbower-mistral-small-1103-v1-mkmlizer: ║                                                                     ║
nbeerbower-mistral-small-1103-v1-mkmlizer: ╚═════════════════════════════════════════════════════════════════════╝
nbeerbower-mistral-small-1103-v1-mkmlizer: Downloaded to shared memory in 92.216s
nbeerbower-mistral-small-1103-v1-mkmlizer: quantizing model to /dev/shm/model_cache, profile:s0, folder:/tmp/tmpxb40h96z, device:0
nbeerbower-mistral-small-1103-v1-mkmlizer: Saving flywheel model at /dev/shm/model_cache
nbeerbower-mistral-small-1103-v1-mkmlizer: quantized model in 45.419s
nbeerbower-mistral-small-1103-v1-mkmlizer: Processed model nbeerbower/Mistral-Small-Gutenberg-Doppel-22B in 137.635s
nbeerbower-mistral-small-1103-v1-mkmlizer: creating bucket guanaco-mkml-models
nbeerbower-mistral-small-1103-v1-mkmlizer: Bucket 's3://guanaco-mkml-models/' created
nbeerbower-mistral-small-1103-v1-mkmlizer: uploading /dev/shm/model_cache to s3://guanaco-mkml-models/nbeerbower-mistral-small-1103-v1
nbeerbower-mistral-small-1103-v1-mkmlizer: cp /dev/shm/model_cache/config.json s3://guanaco-mkml-models/nbeerbower-mistral-small-1103-v1/config.json
nbeerbower-mistral-small-1103-v1-mkmlizer: cp /dev/shm/model_cache/special_tokens_map.json s3://guanaco-mkml-models/nbeerbower-mistral-small-1103-v1/special_tokens_map.json
nbeerbower-mistral-small-1103-v1-mkmlizer: cp /dev/shm/model_cache/tokenizer_config.json s3://guanaco-mkml-models/nbeerbower-mistral-small-1103-v1/tokenizer_config.json
nbeerbower-mistral-small-1103-v1-mkmlizer: cp /dev/shm/model_cache/tokenizer.json s3://guanaco-mkml-models/nbeerbower-mistral-small-1103-v1/tokenizer.json
nbeerbower-mistral-small-1103-v1-mkmlizer: cp /dev/shm/model_cache/flywheel_model.1.safetensors s3://guanaco-mkml-models/nbeerbower-mistral-small-1103-v1/flywheel_model.1.safetensors
nbeerbower-mistral-small-1103-v1-mkmlizer: cp /dev/shm/model_cache/flywheel_model.0.safetensors s3://guanaco-mkml-models/nbeerbower-mistral-small-1103-v1/flywheel_model.0.safetensors
nbeerbower-mistral-small-1103-v1-mkmlizer: 
Loading 0:   0%|          | 0/507 [00:00<?, ?it/s]
Loading 0:   1%|          | 5/507 [00:00<00:18, 27.12it/s]
Loading 0:   2%|▏         | 12/507 [00:00<00:11, 41.48it/s]
Loading 0:   3%|▎         | 17/507 [00:00<00:12, 39.37it/s]
Loading 0:   4%|▍         | 22/507 [00:00<00:12, 38.12it/s]
Loading 0:   5%|▌         | 26/507 [00:00<00:12, 37.83it/s]
Loading 0:   6%|▌         | 30/507 [00:00<00:12, 37.21it/s]
Loading 0:   7%|▋         | 34/507 [00:00<00:13, 35.09it/s]
Loading 0:   8%|▊         | 39/507 [00:01<00:13, 35.67it/s]
Loading 0:   8%|▊         | 43/507 [00:01<00:13, 33.81it/s]
Loading 0:   9%|▉         | 47/507 [00:01<00:13, 33.36it/s]
Loading 0:  10%|█         | 51/507 [00:01<00:13, 33.41it/s]
Loading 0:  11%|█         | 55/507 [00:01<00:20, 22.34it/s]
Loading 0:  12%|█▏        | 59/507 [00:01<00:17, 25.17it/s]
Loading 0:  12%|█▏        | 63/507 [00:01<00:16, 27.59it/s]
Loading 0:  13%|█▎        | 67/507 [00:02<00:15, 27.74it/s]
Loading 0:  14%|█▍        | 72/507 [00:02<00:13, 32.00it/s]
Loading 0:  15%|█▌        | 78/507 [00:02<00:11, 36.37it/s]
Loading 0:  16%|█▌        | 82/507 [00:02<00:12, 35.08it/s]
Loading 0:  17%|█▋        | 87/507 [00:02<00:11, 36.68it/s]
Loading 0:  18%|█▊        | 91/507 [00:02<00:11, 34.89it/s]
Loading 0:  19%|█▉        | 96/507 [00:02<00:11, 36.55it/s]
Loading 0:  20%|█▉        | 100/507 [00:02<00:11, 35.44it/s]
Loading 0:  21%|██        | 105/507 [00:03<00:10, 37.06it/s]
Loading 0:  21%|██▏       | 109/507 [00:03<00:11, 35.43it/s]
Loading 0:  22%|██▏       | 113/507 [00:03<00:15, 24.94it/s]
Loading 0:  23%|██▎       | 116/507 [00:03<00:16, 23.40it/s]
Loading 0:  24%|██▍       | 122/507 [00:03<00:14, 26.51it/s]
Loading 0:  25%|██▌       | 127/507 [00:03<00:12, 31.20it/s]
Loading 0:  26%|██▌       | 131/507 [00:04<00:13, 28.92it/s]
Loading 0:  27%|██▋       | 138/507 [00:04<00:10, 34.93it/s]
Loading 0:  28%|██▊       | 142/507 [00:04<00:10, 33.84it/s]
Loading 0:  29%|██▉       | 147/507 [00:04<00:10, 35.21it/s]
Loading 0:  30%|██▉       | 151/507 [00:04<00:11, 32.28it/s]
Loading 0:  31%|███       | 156/507 [00:04<00:10, 33.48it/s]
Loading 0:  32%|███▏      | 160/507 [00:04<00:10, 31.57it/s]
Loading 0:  32%|███▏      | 164/507 [00:05<00:11, 30.84it/s]
Loading 0:  33%|███▎      | 169/507 [00:05<00:14, 24.04it/s]
Loading 0:  34%|███▍      | 172/507 [00:05<00:13, 24.00it/s]
Loading 0:  35%|███▍      | 176/507 [00:05<00:13, 24.29it/s]
Loading 0:  36%|███▌      | 183/507 [00:05<00:10, 31.34it/s]
Loading 0:  37%|███▋      | 187/507 [00:05<00:10, 31.41it/s]
Loading 0:  38%|███▊      | 192/507 [00:06<00:09, 33.14it/s]
Loading 0:  39%|███▊      | 196/507 [00:06<00:09, 32.85it/s]
Loading 0:  40%|███▉      | 201/507 [00:06<00:08, 34.47it/s]
Loading 0:  40%|████      | 205/507 [00:06<00:09, 33.53it/s]
Loading 0:  41%|████▏     | 210/507 [00:06<00:08, 35.41it/s]
Loading 0:  42%|████▏     | 214/507 [00:06<00:08, 34.57it/s]
Loading 0:  43%|████▎     | 218/507 [00:06<00:08, 33.89it/s]
Loading 0:  44%|████▍     | 222/507 [00:06<00:08, 33.82it/s]
Loading 0:  45%|████▍     | 226/507 [00:07<00:11, 24.23it/s]
Loading 0:  45%|████▌     | 230/507 [00:07<00:11, 24.69it/s]
Loading 0:  47%|████▋     | 237/507 [00:07<00:08, 32.07it/s]
Loading 0:  48%|████▊     | 241/507 [00:07<00:08, 32.17it/s]
Loading 0:  49%|████▊     | 246/507 [00:07<00:07, 34.15it/s]
Loading 0:  49%|████▉     | 250/507 [00:07<00:07, 33.96it/s]
Loading 0:  50%|█████     | 255/507 [00:08<00:07, 35.83it/s]
Loading 0:  51%|█████     | 259/507 [00:08<00:07, 33.94it/s]
Loading 0:  52%|█████▏    | 264/507 [00:08<00:06, 34.81it/s]
Loading 0:  53%|█████▎    | 268/507 [00:08<00:07, 32.50it/s]
Loading 0:  54%|█████▍    | 273/507 [00:08<00:06, 34.47it/s]
Loading 0:  55%|█████▍    | 277/507 [00:08<00:06, 33.66it/s]
Loading 0:  56%|█████▌    | 282/507 [00:08<00:06, 36.99it/s]
Loading 0:  56%|█████▋    | 286/507 [00:09<00:08, 25.06it/s]
Loading 0:  57%|█████▋    | 290/507 [00:09<00:07, 27.44it/s]
Loading 0:  58%|█████▊    | 294/507 [00:09<00:07, 27.19it/s]
Loading 0:  59%|█████▉    | 299/507 [00:09<00:06, 32.12it/s]
Loading 0:  59%|█████▉    | 299/507 [00:20<00:06, 32.12it/s]
Loading 0:  59%|█████▉    | 300/507 [00:24<04:33,  1.32s/it]
Loading 0:  60%|█████▉    | 302/507 [00:24<03:39,  1.07s/it]
Loading 0:  61%|██████    | 307/507 [00:24<02:07,  1.56it/s]
Loading 0:  61%|██████    | 310/507 [00:24<01:35,  2.06it/s]
Loading 0:  62%|██████▏   | 313/507 [00:24<01:10,  2.76it/s]
Loading 0:  63%|██████▎   | 318/507 [00:24<00:43,  4.35it/s]
Loading 0:  63%|██████▎   | 321/507 [00:25<00:34,  5.46it/s]
Loading 0:  64%|██████▍   | 325/507 [00:25<00:24,  7.56it/s]
Loading 0:  65%|██████▍   | 329/507 [00:25<00:18,  9.68it/s]
Loading 0:  66%|██████▌   | 334/507 [00:25<00:12, 13.34it/s]
Loading 0:  67%|██████▋   | 339/507 [00:25<00:09, 16.98it/s]
Loading 0:  68%|██████▊   | 343/507 [00:25<00:10, 14.95it/s]
Loading 0:  68%|██████▊   | 347/507 [00:26<00:09, 16.68it/s]
Loading 0:  69%|██████▉   | 352/507 [00:26<00:07, 21.16it/s]
Loading 0:  70%|███████   | 356/507 [00:26<00:06, 21.74it/s]
Loading 0:  72%|███████▏  | 363/507 [00:26<00:05, 28.42it/s]
Loading 0:  72%|███████▏  | 367/507 [00:26<00:04, 28.70it/s]
Loading 0:  73%|███████▎  | 372/507 [00:26<00:04, 30.69it/s]
Loading 0:  74%|███████▍  | 376/507 [00:26<00:04, 30.39it/s]
Loading 0:  75%|███████▌  | 381/507 [00:27<00:03, 32.32it/s]
Loading 0:  76%|███████▌  | 385/507 [00:27<00:03, 31.45it/s]
Loading 0:  77%|███████▋  | 389/507 [00:27<00:03, 31.18it/s]
Loading 0:  78%|███████▊  | 393/507 [00:27<00:03, 30.93it/s]
Loading 0:  78%|███████▊  | 397/507 [00:27<00:04, 22.06it/s]
Loading 0:  79%|███████▉  | 401/507 [00:27<00:04, 22.47it/s]
Loading 0:  80%|████████  | 406/507 [00:28<00:03, 27.04it/s]
Loading 0:  81%|████████  | 410/507 [00:28<00:03, 26.07it/s]
Loading 0:  82%|████████▏ | 417/507 [00:28<00:02, 32.75it/s]
Loading 0:  83%|████████▎ | 421/507 [00:28<00:02, 32.77it/s]
Loading 0:  84%|████████▍ | 426/507 [00:28<00:02, 35.13it/s]
Loading 0:  85%|████████▍ | 430/507 [00:28<00:02, 33.83it/s]
Loading 0:  86%|████████▌ | 435/507 [00:28<00:02, 34.87it/s]
Loading 0:  87%|████████▋ | 439/507 [00:28<00:02, 33.90it/s]
Loading 0:  88%|████████▊ | 444/507 [00:29<00:01, 35.66it/s]
Loading 0:  88%|████████▊ | 448/507 [00:29<00:01, 34.47it/s]
Loading 0:  89%|████████▉ | 453/507 [00:29<00:01, 37.34it/s]
Loading 0:  90%|█████████ | 457/507 [00:31<00:08,  5.76it/s]
Loading 0:  91%|█████████ | 460/507 [00:31<00:06,  6.82it/s]
Loading 0:  92%|█████████▏| 465/507 [00:31<00:04,  9.23it/s]
Loading 0:  93%|█████████▎| 470/507 [00:32<00:02, 12.62it/s]
Loading 0:  93%|█████████▎| 474/507 [00:32<00:02, 14.46it/s]
Loading 0:  94%|█████████▍| 479/507 [00:32<00:01, 18.69it/s]
Loading 0:  95%|█████████▌| 483/507 [00:32<00:01, 19.82it/s]
Loading 0:  96%|█████████▋| 488/507 [00:32<00:00, 24.60it/s]
Loading 0:  97%|█████████▋| 492/507 [00:32<00:00, 24.51it/s]
Loading 0:  98%|█████████▊| 497/507 [00:32<00:00, 28.68it/s]
Loading 0:  99%|█████████▉| 501/507 [00:33<00:00, 27.42it/s]
Loading 0: 100%|█████████▉| 506/507 [00:33<00:00, 31.95it/s]
                                                            
Job nbeerbower-mistral-small-1103-v1-mkmlizer completed after 175.56s with status: succeeded
Stopping job with name nbeerbower-mistral-small-1103-v1-mkmlizer
Pipeline stage MKMLizer completed in 177.18s
run pipeline stage %s
Running pipeline stage MKMLTemplater
Pipeline stage MKMLTemplater completed in 0.18s
run pipeline stage %s
Running pipeline stage MKMLDeployer
Creating inference service nbeerbower-mistral-small-1103-v1
Waiting for inference service nbeerbower-mistral-small-1103-v1 to be ready
Inference service nbeerbower-mistral-small-1103-v1 ready after 140.579669713974s
Pipeline stage MKMLDeployer completed in 141.14s
run pipeline stage %s
Running pipeline stage StressChecker
Received healthy response to inference request in 2.1891074180603027s
Received healthy response to inference request in 2.584876537322998s
Received healthy response to inference request in 2.4106218814849854s
Received healthy response to inference request in 2.067958116531372s
Received healthy response to inference request in 2.3416588306427s
5 requests
0 failed requests
5th percentile: 2.0921879768371583
10th percentile: 2.1164178371429445
20th percentile: 2.1648775577545165
30th percentile: 2.2196177005767823
40th percentile: 2.280638265609741
50th percentile: 2.3416588306427
60th percentile: 2.3692440509796144
70th percentile: 2.3968292713165282
80th percentile: 2.445472812652588
90th percentile: 2.515174674987793
95th percentile: 2.5500256061553954
99th percentile: 2.5779063510894775
mean time: 2.3188445568084717
Pipeline stage StressChecker completed in 13.07s
Shutdown handler de-registered
nbeerbower-mistral-small_1103_v1 status is now deployed due to DeploymentManager action
nbeerbower-mistral-small_1103_v1 status is now inactive due to auto deactivation removed underperforming models
nbeerbower-mistral-small_1103_v1 status is now torndown due to DeploymentManager action