developer_uid: azuruce
submission_id: chaiml-nemo-20241017-mod_6892_v4
model_name: chaiml-nemo-20241017-mod_6892_v4
model_group: ChaiML/nemo-20241017-mod
status: torndown
timestamp: 2024-10-18T16:05:17+00:00
num_battles: 7644
num_wins: 3741
celo_rating: 1246.29
family_friendly_score: 0.589530176922727
family_friendly_standard_error: 0.005721481177521448
submission_type: basic
model_repo: ChaiML/nemo-20241017-model_stock-remerge_v2_5merge-albert
model_architecture: MistralForCausalLM
model_num_parameters: 12772070400.0
best_of: 8
max_input_tokens: 1024
max_output_tokens: 64
display_name: chaiml-nemo-20241017-mod_6892_v4
is_internal_developer: True
language_model: ChaiML/nemo-20241017-model_stock-remerge_v2_5merge-albert
model_size: 13B
ranking_group: single
us_pacific_date: 2024-10-18
win_ratio: 0.489403453689168
generation_params: {'temperature': 1.0, 'top_p': 1.0, 'min_p': 0.0, 'top_k': 40, 'presence_penalty': 0.0, 'frequency_penalty': 0.0, 'stopping_words': ['\n', '</s>', '###', 'Bot:', 'User:', 'You:', '<|im_end|>'], 'max_input_tokens': 1024, 'best_of': 8, 'max_output_tokens': 64}
formatter: {'memory_template': '', 'prompt_template': '', 'bot_template': '{bot_name}: {message}\n', 'user_template': '{user_name}: {message}\n', 'response_template': '### Response:\n{bot_name}:', 'truncate_by_message': False}
Resubmit model
Shutdown handler not registered because Python interpreter is not running in the main thread
run pipeline %s
run pipeline stage %s
Running pipeline stage MKMLizer
Starting job with name chaiml-nemo-20241017-mod-6892-v4-mkmlizer
Waiting for job on chaiml-nemo-20241017-mod-6892-v4-mkmlizer to finish
chaiml-nemo-20241017-mod-6892-v4-mkmlizer: ╔═════════════════════════════════════════════════════════════════════╗
chaiml-nemo-20241017-mod-6892-v4-mkmlizer: ║     _____            __           __                                ║
chaiml-nemo-20241017-mod-6892-v4-mkmlizer: ║    / _/ /_ ___    __/ /  ___ ___ / /                                ║
chaiml-nemo-20241017-mod-6892-v4-mkmlizer: ║   / _/ / // / |/|/ / _ \/ -_) -_) /                                 ║
chaiml-nemo-20241017-mod-6892-v4-mkmlizer: ║  /_//_/\_, /|__,__/_//_/\__/\__/_/                                  ║
chaiml-nemo-20241017-mod-6892-v4-mkmlizer: ║       /___/                                                         ║
chaiml-nemo-20241017-mod-6892-v4-mkmlizer: ║                                                                     ║
chaiml-nemo-20241017-mod-6892-v4-mkmlizer: ║  Version: 0.11.12                                                   ║
chaiml-nemo-20241017-mod-6892-v4-mkmlizer: ║  Copyright 2023 MK ONE TECHNOLOGIES Inc.                            ║
chaiml-nemo-20241017-mod-6892-v4-mkmlizer: ║  https://mk1.ai                                                     ║
chaiml-nemo-20241017-mod-6892-v4-mkmlizer: ║                                                                     ║
chaiml-nemo-20241017-mod-6892-v4-mkmlizer: ║  The license key for the current software has been verified as      ║
chaiml-nemo-20241017-mod-6892-v4-mkmlizer: ║  belonging to:                                                      ║
chaiml-nemo-20241017-mod-6892-v4-mkmlizer: ║                                                                     ║
chaiml-nemo-20241017-mod-6892-v4-mkmlizer: ║  Chai Research Corp.                                                ║
chaiml-nemo-20241017-mod-6892-v4-mkmlizer: ║  Account ID: 7997a29f-0ceb-4cc7-9adf-840c57b4ae6f                   ║
chaiml-nemo-20241017-mod-6892-v4-mkmlizer: ║  Expiration: 2025-01-15 23:59:59                                    ║
chaiml-nemo-20241017-mod-6892-v4-mkmlizer: ║                                                                     ║
chaiml-nemo-20241017-mod-6892-v4-mkmlizer: ╚═════════════════════════════════════════════════════════════════════╝
chaiml-nemo-20241017-mod-6892-v4-mkmlizer: Downloaded to shared memory in 29.251s
chaiml-nemo-20241017-mod-6892-v4-mkmlizer: quantizing model to /dev/shm/model_cache, profile:s0, folder:/tmp/tmpcrpdo536, device:0
chaiml-nemo-20241017-mod-6892-v4-mkmlizer: Saving flywheel model at /dev/shm/model_cache
chaiml-nemo-20241017-mod-6892-v4-mkmlizer: quantized model in 38.846s
chaiml-nemo-20241017-mod-6892-v4-mkmlizer: Processed model ChaiML/nemo-20241017-model_stock-remerge_v2_5merge-albert in 68.097s
chaiml-nemo-20241017-mod-6892-v4-mkmlizer: creating bucket guanaco-mkml-models
chaiml-nemo-20241017-mod-6892-v4-mkmlizer: Bucket 's3://guanaco-mkml-models/' created
chaiml-nemo-20241017-mod-6892-v4-mkmlizer: uploading /dev/shm/model_cache to s3://guanaco-mkml-models/chaiml-nemo-20241017-mod-6892-v4
chaiml-nemo-20241017-mod-6892-v4-mkmlizer: cp /dev/shm/model_cache/config.json s3://guanaco-mkml-models/chaiml-nemo-20241017-mod-6892-v4/config.json
chaiml-nemo-20241017-mod-6892-v4-mkmlizer: cp /dev/shm/model_cache/special_tokens_map.json s3://guanaco-mkml-models/chaiml-nemo-20241017-mod-6892-v4/special_tokens_map.json
chaiml-nemo-20241017-mod-6892-v4-mkmlizer: cp /dev/shm/model_cache/tokenizer_config.json s3://guanaco-mkml-models/chaiml-nemo-20241017-mod-6892-v4/tokenizer_config.json
chaiml-nemo-20241017-mod-6892-v4-mkmlizer: cp /dev/shm/model_cache/tokenizer.json s3://guanaco-mkml-models/chaiml-nemo-20241017-mod-6892-v4/tokenizer.json
chaiml-nemo-20241017-mod-6892-v4-mkmlizer: cp /dev/shm/model_cache/flywheel_model.0.safetensors s3://guanaco-mkml-models/chaiml-nemo-20241017-mod-6892-v4/flywheel_model.0.safetensors
chaiml-nemo-20241017-mod-6892-v4-mkmlizer: 
Loading 0:   0%|          | 0/363 [00:00<?, ?it/s]
Loading 0:   1%|          | 2/363 [00:06<18:46,  3.12s/it]
Loading 0:   2%|▏         | 6/363 [00:06<05:00,  1.19it/s]
Loading 0:   3%|▎         | 11/363 [00:06<02:12,  2.66it/s]
Loading 0:   4%|▍         | 15/363 [00:06<01:24,  4.11it/s]
Loading 0:   6%|▌         | 20/363 [00:06<00:52,  6.50it/s]
Loading 0:   7%|▋         | 24/363 [00:06<00:39,  8.53it/s]
Loading 0:   9%|▊         | 31/363 [00:07<00:24, 13.77it/s]
Loading 0:  10%|▉         | 35/363 [00:07<00:20, 16.18it/s]
Loading 0:  11%|█         | 40/363 [00:07<00:20, 15.46it/s]
Loading 0:  12%|█▏        | 43/363 [00:07<00:19, 16.57it/s]
Loading 0:  13%|█▎        | 47/363 [00:07<00:15, 19.88it/s]
Loading 0:  14%|█▍        | 51/363 [00:07<00:14, 21.02it/s]
Loading 0:  15%|█▌        | 56/363 [00:08<00:12, 24.78it/s]
Loading 0:  17%|█▋        | 60/363 [00:08<00:12, 24.81it/s]
Loading 0:  18%|█▊        | 65/363 [00:08<00:10, 29.29it/s]
Loading 0:  19%|█▉        | 69/363 [00:08<00:10, 27.97it/s]
Loading 0:  20%|██        | 74/363 [00:08<00:08, 32.50it/s]
Loading 0:  21%|██▏       | 78/363 [00:08<00:09, 31.48it/s]
Loading 0:  23%|██▎       | 85/363 [00:08<00:06, 40.03it/s]
Loading 0:  25%|██▍       | 90/363 [00:09<00:06, 40.79it/s]
Loading 0:  26%|██▌       | 95/363 [00:09<00:06, 42.06it/s]
Loading 0:  28%|██▊       | 101/363 [00:09<00:06, 40.94it/s]
Loading 0:  29%|██▉       | 106/363 [00:09<00:06, 39.80it/s]
Loading 0:  31%|███       | 112/363 [00:09<00:05, 44.39it/s]
Loading 0:  32%|███▏      | 117/363 [00:09<00:05, 42.93it/s]
Loading 0:  34%|███▎      | 122/363 [00:09<00:08, 29.27it/s]
Loading 0:  35%|███▍      | 126/363 [00:10<00:07, 30.26it/s]
Loading 0:  36%|███▌      | 130/363 [00:10<00:07, 31.60it/s]
Loading 0:  37%|███▋      | 134/363 [00:10<00:07, 30.56it/s]
Loading 0:  38%|███▊      | 138/363 [00:10<00:07, 31.94it/s]
Loading 0:  39%|███▉      | 142/363 [00:10<00:07, 27.83it/s]
Loading 0:  40%|████      | 146/363 [00:10<00:07, 30.43it/s]
Loading 0:  41%|████▏     | 150/363 [00:10<00:07, 28.76it/s]
Loading 0:  43%|████▎     | 155/363 [00:10<00:06, 33.52it/s]
Loading 0:  44%|████▍     | 159/363 [00:11<00:06, 31.03it/s]
Loading 0:  45%|████▌     | 164/363 [00:11<00:05, 34.35it/s]
Loading 0:  46%|████▋     | 168/363 [00:11<00:06, 29.76it/s]
Loading 0:  48%|████▊     | 173/363 [00:11<00:05, 33.96it/s]
Loading 0:  49%|████▉     | 177/363 [00:11<00:06, 30.56it/s]
Loading 0:  50%|█████     | 182/363 [00:11<00:05, 33.81it/s]
Loading 0:  51%|█████     | 186/363 [00:12<00:05, 29.75it/s]
Loading 0:  53%|█████▎    | 191/363 [00:12<00:05, 33.09it/s]
Loading 0:  54%|█████▎    | 195/363 [00:12<00:05, 29.99it/s]
Loading 0:  55%|█████▌    | 200/363 [00:12<00:04, 32.60it/s]
Loading 0:  56%|█████▌    | 204/363 [00:12<00:07, 20.71it/s]
Loading 0:  58%|█████▊    | 209/363 [00:12<00:06, 25.08it/s]
Loading 0:  59%|█████▊    | 213/363 [00:13<00:06, 24.76it/s]
Loading 0:  60%|██████    | 218/363 [00:13<00:04, 29.16it/s]
Loading 0:  61%|██████    | 222/363 [00:13<00:05, 27.70it/s]
Loading 0:  63%|██████▎   | 227/363 [00:13<00:04, 31.53it/s]
Loading 0:  64%|██████▎   | 231/363 [00:13<00:04, 29.36it/s]
Loading 0:  65%|██████▌   | 236/363 [00:13<00:03, 33.90it/s]
Loading 0:  66%|██████▌   | 240/363 [00:13<00:03, 31.87it/s]
Loading 0:  68%|██████▊   | 247/363 [00:13<00:02, 38.83it/s]
Loading 0:  69%|██████▉   | 252/363 [00:14<00:02, 38.30it/s]
Loading 0:  71%|███████   | 257/363 [00:14<00:02, 38.37it/s]
Loading 0:  72%|███████▏  | 261/363 [00:14<00:02, 38.64it/s]
Loading 0:  73%|███████▎  | 265/363 [00:14<00:02, 37.53it/s]
Loading 0:  74%|███████▍  | 269/363 [00:14<00:02, 36.44it/s]
Loading 0:  76%|███████▌  | 275/363 [00:14<00:02, 40.49it/s]
Loading 0:  77%|███████▋  | 281/363 [00:14<00:02, 40.85it/s]
Loading 0:  79%|███████▉  | 286/363 [00:15<00:02, 27.17it/s]
Loading 0:  80%|████████  | 292/363 [00:15<00:02, 32.08it/s]
Loading 0:  82%|████████▏ | 296/363 [00:15<00:02, 33.05it/s]
Loading 0:  83%|████████▎ | 301/363 [00:15<00:01, 36.11it/s]
Loading 0:  84%|████████▍ | 306/363 [00:15<00:01, 36.64it/s]
Loading 0:  86%|████████▌ | 311/363 [00:15<00:01, 37.98it/s]
Loading 0:  87%|████████▋ | 316/363 [00:15<00:01, 40.35it/s]
Loading 0:  88%|████████▊ | 321/363 [00:16<00:01, 34.04it/s]
Loading 0:  90%|█████████ | 328/363 [00:16<00:00, 41.24it/s]
Loading 0:  92%|█████████▏| 333/363 [00:16<00:00, 40.16it/s]
Loading 0:  93%|█████████▎| 338/363 [00:16<00:00, 37.26it/s]
Loading 0:  94%|█████████▍| 342/363 [00:16<00:00, 36.33it/s]
Loading 0:  96%|█████████▌| 347/363 [00:16<00:00, 38.55it/s]
Loading 0:  97%|█████████▋| 351/363 [00:16<00:00, 38.85it/s]
Loading 0:  98%|█████████▊| 355/363 [00:16<00:00, 37.40it/s]
Loading 0:  99%|█████████▉| 359/363 [00:17<00:00, 34.73it/s]
                                                            

Job chaiml-nemo-20241017-mod-6892-v4-mkmlizer completed after 93.92s with status: succeeded
Stopping job with name chaiml-nemo-20241017-mod-6892-v4-mkmlizer
Pipeline stage MKMLizer completed in 94.39s
run pipeline stage %s
Running pipeline stage MKMLTemplater
Pipeline stage MKMLTemplater completed in 0.17s
run pipeline stage %s
Running pipeline stage MKMLDeployer
Creating inference service chaiml-nemo-20241017-mod-6892-v4
Waiting for inference service chaiml-nemo-20241017-mod-6892-v4 to be ready
Inference service chaiml-nemo-20241017-mod-6892-v4 ready after 260.98367738723755s
Pipeline stage MKMLDeployer completed in 261.56s
run pipeline stage %s
Running pipeline stage StressChecker
Received healthy response to inference request in 1.9215664863586426s
Received healthy response to inference request in 1.7542881965637207s
Received healthy response to inference request in 1.4615862369537354s
Received healthy response to inference request in 1.9576952457427979s
Received healthy response to inference request in 1.4871399402618408s
5 requests
0 failed requests
5th percentile: 1.4666969776153564
10th percentile: 1.4718077182769775
20th percentile: 1.4820291996002197
30th percentile: 1.5405695915222168
40th percentile: 1.6474288940429687
50th percentile: 1.7542881965637207
60th percentile: 1.8211995124816895
70th percentile: 1.8881108283996582
80th percentile: 1.9287922382354736
90th percentile: 1.9432437419891357
95th percentile: 1.9504694938659668
99th percentile: 1.9562500953674316
mean time: 1.7164552211761475
Pipeline stage StressChecker completed in 10.22s
Shutdown handler de-registered
chaiml-nemo-20241017-mod_6892_v4 status is now deployed due to DeploymentManager action
chaiml-nemo-20241017-mod_6892_v4 status is now inactive due to auto deactivation removed underperforming models
chaiml-nemo-20241017-mod_6892_v4 status is now torndown due to DeploymentManager action