raifle-acolyte-22b

developer_uid: zonemercy
submission_id: raifle-acolyte-22b_v1
model_name: raifle-acolyte-22b_v1
model_group: rAIfle/Acolyte-22B
status: torndown
timestamp: 2024-10-10T22:57:58+00:00
num_battles: 6072
num_wins: 2938
celo_rating: 1238.78
family_friendly_score: 0.5474489210284665
family_friendly_standard_error: 0.006502955252127388
submission_type: basic
model_repo: rAIfle/Acolyte-22B
model_architecture: MistralForCausalLM
model_num_parameters: 22247282688.0
best_of: 8
max_input_tokens: 1024
max_output_tokens: 64
display_name: raifle-acolyte-22b_v1
is_internal_developer: True
language_model: rAIfle/Acolyte-22B
model_size: 22B
ranking_group: single
us_pacific_date: 2024-10-10
win_ratio: 0.48386034255599475
generation_params: {'temperature': 1.0, 'top_p': 1.0, 'min_p': 0.0, 'top_k': 40, 'presence_penalty': 0.0, 'frequency_penalty': 0.0, 'stopping_words': ['\n', '</s>', '####', 'Bot:', 'User:', 'You:', '<|im_end|>', '<|eot_id|>'], 'max_input_tokens': 1024, 'best_of': 8, 'max_output_tokens': 64}
formatter: {'memory_template': '', 'prompt_template': '', 'bot_template': '{bot_name}: {message}\n', 'user_template': '{user_name}: {message}\n', 'response_template': '{bot_name}:', 'truncate_by_message': False}
Resubmit model
Shutdown handler not registered because Python interpreter is not running in the main thread
run pipeline %s
run pipeline stage %s
Running pipeline stage MKMLizer
Starting job with name raifle-acolyte-22b-v1-mkmlizer
Waiting for job on raifle-acolyte-22b-v1-mkmlizer to finish
raifle-acolyte-22b-v1-mkmlizer: ╔═════════════════════════════════════════════════════════════════════╗
raifle-acolyte-22b-v1-mkmlizer: ║     _____            __           __                                ║
raifle-acolyte-22b-v1-mkmlizer: ║    / _/ /_ ___    __/ /  ___ ___ / /                                ║
raifle-acolyte-22b-v1-mkmlizer: ║   / _/ / // / |/|/ / _ \/ -_) -_) /                                 ║
raifle-acolyte-22b-v1-mkmlizer: ║  /_//_/\_, /|__,__/_//_/\__/\__/_/                                  ║
raifle-acolyte-22b-v1-mkmlizer: ║       /___/                                                         ║
raifle-acolyte-22b-v1-mkmlizer: ║                                                                     ║
raifle-acolyte-22b-v1-mkmlizer: ║  Version: 0.11.12                                                   ║
raifle-acolyte-22b-v1-mkmlizer: ║  Copyright 2023 MK ONE TECHNOLOGIES Inc.                            ║
raifle-acolyte-22b-v1-mkmlizer: ║  https://mk1.ai                                                     ║
raifle-acolyte-22b-v1-mkmlizer: ║                                                                     ║
raifle-acolyte-22b-v1-mkmlizer: ║  The license key for the current software has been verified as      ║
raifle-acolyte-22b-v1-mkmlizer: ║  belonging to:                                                      ║
raifle-acolyte-22b-v1-mkmlizer: ║                                                                     ║
raifle-acolyte-22b-v1-mkmlizer: ║  Chai Research Corp.                                                ║
raifle-acolyte-22b-v1-mkmlizer: ║  Account ID: 7997a29f-0ceb-4cc7-9adf-840c57b4ae6f                   ║
raifle-acolyte-22b-v1-mkmlizer: ║  Expiration: 2024-10-15 23:59:59                                    ║
raifle-acolyte-22b-v1-mkmlizer: ║                                                                     ║
raifle-acolyte-22b-v1-mkmlizer: ╚═════════════════════════════════════════════════════════════════════╝
Retrying (%r) after connection broken by '%r': %s
raifle-acolyte-22b-v1-mkmlizer: Downloaded to shared memory in 86.458s
raifle-acolyte-22b-v1-mkmlizer: quantizing model to /dev/shm/model_cache, profile:s0, folder:/tmp/tmpx5e2311z, device:0
raifle-acolyte-22b-v1-mkmlizer: Saving flywheel model at /dev/shm/model_cache
raifle-acolyte-22b-v1-mkmlizer: quantized model in 45.397s
raifle-acolyte-22b-v1-mkmlizer: Processed model rAIfle/Acolyte-22B in 131.855s
raifle-acolyte-22b-v1-mkmlizer: creating bucket guanaco-mkml-models
raifle-acolyte-22b-v1-mkmlizer: Bucket 's3://guanaco-mkml-models/' created
raifle-acolyte-22b-v1-mkmlizer: uploading /dev/shm/model_cache to s3://guanaco-mkml-models/raifle-acolyte-22b-v1
raifle-acolyte-22b-v1-mkmlizer: cp /dev/shm/model_cache/config.json s3://guanaco-mkml-models/raifle-acolyte-22b-v1/config.json
raifle-acolyte-22b-v1-mkmlizer: cp /dev/shm/model_cache/special_tokens_map.json s3://guanaco-mkml-models/raifle-acolyte-22b-v1/special_tokens_map.json
raifle-acolyte-22b-v1-mkmlizer: cp /dev/shm/model_cache/tokenizer_config.json s3://guanaco-mkml-models/raifle-acolyte-22b-v1/tokenizer_config.json
raifle-acolyte-22b-v1-mkmlizer: cp /dev/shm/model_cache/tokenizer.model s3://guanaco-mkml-models/raifle-acolyte-22b-v1/tokenizer.model
raifle-acolyte-22b-v1-mkmlizer: cp /dev/shm/model_cache/tokenizer.json s3://guanaco-mkml-models/raifle-acolyte-22b-v1/tokenizer.json
raifle-acolyte-22b-v1-mkmlizer: cp /dev/shm/model_cache/flywheel_model.1.safetensors s3://guanaco-mkml-models/raifle-acolyte-22b-v1/flywheel_model.1.safetensors
raifle-acolyte-22b-v1-mkmlizer: cp /dev/shm/model_cache/flywheel_model.0.safetensors s3://guanaco-mkml-models/raifle-acolyte-22b-v1/flywheel_model.0.safetensors
raifle-acolyte-22b-v1-mkmlizer: 
Loading 0:   0%|          | 0/507 [00:00<?, ?it/s]
Loading 0:   0%|          | 2/507 [00:01<07:46,  1.08it/s]
Loading 0:   1%|          | 6/507 [00:02<02:15,  3.69it/s]
Loading 0:   3%|▎         | 13/507 [00:02<00:52,  9.44it/s]
Loading 0:   3%|▎         | 17/507 [00:02<00:38, 12.59it/s]
Loading 0:   4%|▍         | 22/507 [00:02<00:28, 17.21it/s]
Loading 0:   5%|▌         | 26/507 [00:02<00:23, 20.09it/s]
Loading 0:   6%|▌         | 31/507 [00:02<00:19, 24.87it/s]
Loading 0:   7%|▋         | 35/507 [00:02<00:17, 26.60it/s]
Loading 0:   8%|▊         | 40/507 [00:02<00:15, 30.21it/s]
Loading 0:   9%|▊         | 44/507 [00:02<00:14, 31.33it/s]
Loading 0:  10%|▉         | 49/507 [00:03<00:13, 34.26it/s]
Loading 0:  10%|█         | 53/507 [00:03<00:19, 23.66it/s]
Loading 0:  11%|█▏        | 58/507 [00:03<00:16, 27.38it/s]
Loading 0:  12%|█▏        | 62/507 [00:03<00:15, 28.98it/s]
Loading 0:  13%|█▎        | 67/507 [00:03<00:13, 31.90it/s]
Loading 0:  14%|█▍        | 71/507 [00:03<00:13, 32.48it/s]
Loading 0:  15%|█▍        | 76/507 [00:03<00:12, 35.01it/s]
Loading 0:  16%|█▌        | 80/507 [00:04<00:12, 34.31it/s]
Loading 0:  17%|█▋        | 85/507 [00:04<00:11, 36.61it/s]
Loading 0:  18%|█▊        | 89/507 [00:04<00:11, 36.28it/s]
Loading 0:  19%|█▊        | 94/507 [00:04<00:10, 37.67it/s]
Loading 0:  19%|█▉        | 98/507 [00:04<00:11, 36.75it/s]
Loading 0:  20%|██        | 103/507 [00:04<00:10, 38.54it/s]
Loading 0:  21%|██        | 107/507 [00:05<00:15, 25.57it/s]
Loading 0:  22%|██▏       | 112/507 [00:05<00:13, 29.43it/s]
Loading 0:  23%|██▎       | 116/507 [00:05<00:12, 30.31it/s]
Loading 0:  24%|██▍       | 121/507 [00:05<00:11, 33.01it/s]
Loading 0:  25%|██▍       | 125/507 [00:05<00:11, 33.43it/s]
Loading 0:  26%|██▌       | 130/507 [00:05<00:10, 35.44it/s]
Loading 0:  26%|██▋       | 134/507 [00:05<00:10, 34.47it/s]
Loading 0:  27%|██▋       | 139/507 [00:05<00:10, 36.13it/s]
Loading 0:  28%|██▊       | 143/507 [00:05<00:10, 35.14it/s]
Loading 0:  29%|██▉       | 148/507 [00:06<00:09, 37.03it/s]
Loading 0:  30%|██▉       | 152/507 [00:06<00:09, 35.75it/s]
Loading 0:  31%|███       | 157/507 [00:06<00:09, 37.15it/s]
Loading 0:  32%|███▏      | 161/507 [00:06<00:09, 35.26it/s]
Loading 0:  33%|███▎      | 165/507 [00:06<00:13, 25.35it/s]
Loading 0:  33%|███▎      | 168/507 [00:06<00:14, 24.03it/s]
Loading 0:  35%|███▍      | 175/507 [00:07<00:10, 31.95it/s]
Loading 0:  35%|███▌      | 179/507 [00:07<00:10, 32.43it/s]
Loading 0:  36%|███▋      | 184/507 [00:07<00:09, 35.01it/s]
Loading 0:  37%|███▋      | 188/507 [00:07<00:09, 34.75it/s]
Loading 0:  38%|███▊      | 193/507 [00:07<00:08, 36.65it/s]
Loading 0:  39%|███▉      | 197/507 [00:07<00:08, 35.45it/s]
Loading 0:  40%|███▉      | 202/507 [00:07<00:08, 37.16it/s]
Loading 0:  41%|████      | 206/507 [00:07<00:08, 35.77it/s]
Loading 0:  42%|████▏     | 211/507 [00:07<00:07, 37.15it/s]
Loading 0:  42%|████▏     | 215/507 [00:08<00:08, 36.34it/s]
Loading 0:  43%|████▎     | 220/507 [00:08<00:07, 38.41it/s]
Loading 0:  44%|████▍     | 224/507 [00:08<00:11, 24.87it/s]
Loading 0:  45%|████▌     | 229/507 [00:08<00:09, 28.78it/s]
Loading 0:  46%|████▌     | 233/507 [00:08<00:09, 29.63it/s]
Loading 0:  47%|████▋     | 238/507 [00:08<00:08, 32.71it/s]
Loading 0:  48%|████▊     | 242/507 [00:09<00:07, 33.21it/s]
Loading 0:  49%|████▊     | 247/507 [00:09<00:07, 35.85it/s]
Loading 0:  50%|████▉     | 251/507 [00:09<00:07, 35.27it/s]
Loading 0:  50%|█████     | 256/507 [00:09<00:06, 37.11it/s]
Loading 0:  51%|█████▏    | 260/507 [00:09<00:06, 36.02it/s]
Loading 0:  52%|█████▏    | 265/507 [00:09<00:06, 37.91it/s]
Loading 0:  53%|█████▎    | 269/507 [00:09<00:06, 36.64it/s]
Loading 0:  54%|█████▍    | 274/507 [00:09<00:06, 38.40it/s]
Loading 0:  55%|█████▍    | 278/507 [00:10<00:09, 25.34it/s]
Loading 0:  56%|█████▌    | 283/507 [00:10<00:07, 28.84it/s]
Loading 0:  57%|█████▋    | 287/507 [00:10<00:07, 30.15it/s]
Loading 0:  58%|█████▊    | 292/507 [00:10<00:06, 33.44it/s]
Loading 0:  58%|█████▊    | 296/507 [00:25<03:45,  1.07s/it]
Loading 0:  59%|█████▉    | 299/507 [00:26<02:54,  1.19it/s]
Loading 0:  60%|█████▉    | 303/507 [00:26<02:03,  1.65it/s]
Loading 0:  61%|██████    | 310/507 [00:26<01:09,  2.83it/s]
Loading 0:  62%|██████▏   | 314/507 [00:26<00:52,  3.71it/s]
Loading 0:  63%|██████▎   | 319/507 [00:26<00:36,  5.21it/s]
Loading 0:  64%|██████▎   | 323/507 [00:26<00:27,  6.73it/s]
Loading 0:  65%|██████▍   | 328/507 [00:26<00:19,  9.24it/s]
Loading 0:  65%|██████▌   | 332/507 [00:26<00:15, 11.50it/s]
Loading 0:  66%|██████▋   | 336/507 [00:27<00:13, 12.46it/s]
Loading 0:  67%|██████▋   | 340/507 [00:27<00:11, 14.46it/s]
Loading 0:  68%|██████▊   | 344/507 [00:27<00:09, 17.40it/s]
Loading 0:  69%|██████▊   | 348/507 [00:27<00:08, 19.40it/s]
Loading 0:  70%|███████   | 355/507 [00:27<00:05, 26.69it/s]
Loading 0:  71%|███████   | 359/507 [00:27<00:05, 27.85it/s]
Loading 0:  72%|███████▏  | 364/507 [00:28<00:04, 30.99it/s]
Loading 0:  73%|███████▎  | 368/507 [00:28<00:04, 31.26it/s]
Loading 0:  74%|███████▎  | 373/507 [00:28<00:04, 33.47it/s]
Loading 0:  74%|███████▍  | 377/507 [00:28<00:03, 33.05it/s]
Loading 0:  75%|███████▌  | 382/507 [00:28<00:03, 34.98it/s]
Loading 0:  76%|███████▌  | 386/507 [00:28<00:03, 33.80it/s]
Loading 0:  77%|███████▋  | 391/507 [00:28<00:03, 35.04it/s]
Loading 0:  78%|███████▊  | 395/507 [00:29<00:04, 24.60it/s]
Loading 0:  79%|███████▉  | 400/507 [00:29<00:03, 28.34it/s]
Loading 0:  80%|███████▉  | 404/507 [00:29<00:03, 29.15it/s]
Loading 0:  81%|████████  | 409/507 [00:29<00:03, 32.38it/s]
Loading 0:  81%|████████▏ | 413/507 [00:29<00:02, 32.16it/s]
Loading 0:  82%|████████▏ | 418/507 [00:29<00:02, 34.38it/s]
Loading 0:  83%|████████▎ | 422/507 [00:29<00:02, 34.34it/s]
Loading 0:  84%|████████▍ | 427/507 [00:29<00:02, 36.41it/s]
Loading 0:  85%|████████▌ | 431/507 [00:30<00:02, 34.87it/s]
Loading 0:  86%|████████▌ | 436/507 [00:30<00:01, 37.03it/s]
Loading 0:  87%|████████▋ | 440/507 [00:30<00:01, 36.34it/s]
Loading 0:  88%|████████▊ | 445/507 [00:30<00:01, 38.18it/s]
Loading 0:  89%|████████▊ | 449/507 [00:30<00:02, 25.66it/s]
Loading 0:  90%|████████▉ | 454/507 [00:30<00:01, 29.41it/s]
Loading 0:  90%|█████████ | 458/507 [00:30<00:01, 30.31it/s]
Loading 0:  91%|█████████▏| 463/507 [00:31<00:01, 32.08it/s]
Loading 0:  92%|█████████▏| 467/507 [00:31<00:01, 32.57it/s]
Loading 0:  93%|█████████▎| 472/507 [00:31<00:00, 35.17it/s]
Loading 0:  94%|█████████▍| 476/507 [00:31<00:00, 34.75it/s]
Loading 0:  95%|█████████▍| 481/507 [00:31<00:00, 36.77it/s]
Loading 0:  96%|█████████▌| 485/507 [00:31<00:00, 35.27it/s]
Loading 0:  97%|█████████▋| 490/507 [00:31<00:00, 37.09it/s]
Loading 0:  97%|█████████▋| 494/507 [00:31<00:00, 35.69it/s]
Loading 0:  98%|█████████▊| 499/507 [00:32<00:00, 37.53it/s]
Loading 0:  99%|█████████▉| 503/507 [00:32<00:00, 36.79it/s]
Loading 0: 100%|██████████| 507/507 [00:32<00:00, 27.28it/s]
                                                            
Job raifle-acolyte-22b-v1-mkmlizer completed after 155.55s with status: succeeded
Stopping job with name raifle-acolyte-22b-v1-mkmlizer
Pipeline stage MKMLizer completed in 156.11s
run pipeline stage %s
Running pipeline stage MKMLTemplater
Pipeline stage MKMLTemplater completed in 0.20s
run pipeline stage %s
Running pipeline stage MKMLDeployer
Creating inference service raifle-acolyte-22b-v1
Waiting for inference service raifle-acolyte-22b-v1 to be ready
Inference service raifle-acolyte-22b-v1 ready after 150.51849007606506s
Pipeline stage MKMLDeployer completed in 151.27s
run pipeline stage %s
Running pipeline stage StressChecker
Received healthy response to inference request in 2.714338779449463s
Received healthy response to inference request in 2.2844595909118652s
Received healthy response to inference request in 2.4402568340301514s
Received healthy response to inference request in 2.4406228065490723s
Received healthy response to inference request in 2.1135642528533936s
5 requests
0 failed requests
5th percentile: 2.147743320465088
10th percentile: 2.1819223880767824
20th percentile: 2.250280523300171
30th percentile: 2.3156190395355223
40th percentile: 2.377937936782837
50th percentile: 2.4402568340301514
60th percentile: 2.4404032230377197
70th percentile: 2.440549612045288
80th percentile: 2.4953660011291503
90th percentile: 2.604852390289307
95th percentile: 2.659595584869385
99th percentile: 2.7033901405334473
mean time: 2.3986484527587892
Pipeline stage StressChecker completed in 13.89s
Shutdown handler de-registered
raifle-acolyte-22b_v1 status is now deployed due to DeploymentManager action
raifle-acolyte-22b_v1 status is now inactive due to auto deactivation removed underperforming models
raifle-acolyte-22b_v1 status is now torndown due to DeploymentManager action