developer_uid: azuruce
submission_id: chaiml-nemo-20241017-dar_6728_v2
model_name: chaiml-nemo-20241017-dar_6728_v2
model_group: ChaiML/nemo-20241017-dar
status: torndown
timestamp: 2024-10-18T16:05:51+00:00
num_battles: 7790
num_wins: 4068
celo_rating: 1266.53
family_friendly_score: 0.5906731113498032
family_friendly_standard_error: 0.005671513035796936
submission_type: basic
model_repo: ChaiML/nemo-20241017-dare_linear-remerge_v2_5merge-albert
model_architecture: MistralForCausalLM
model_num_parameters: 12772070400.0
best_of: 8
max_input_tokens: 1024
max_output_tokens: 64
display_name: chaiml-nemo-20241017-dar_6728_v2
is_internal_developer: True
language_model: ChaiML/nemo-20241017-dare_linear-remerge_v2_5merge-albert
model_size: 13B
ranking_group: single
us_pacific_date: 2024-10-18
win_ratio: 0.5222079589216945
generation_params: {'temperature': 1.0, 'top_p': 1.0, 'min_p': 0.0, 'top_k': 40, 'presence_penalty': 0.0, 'frequency_penalty': 0.0, 'stopping_words': ['\n', '</s>', '###', 'Bot:', 'User:', 'You:', '<|im_end|>'], 'max_input_tokens': 1024, 'best_of': 8, 'max_output_tokens': 64}
formatter: {'memory_template': '', 'prompt_template': '', 'bot_template': '{bot_name}: {message}\n', 'user_template': '{user_name}: {message}\n', 'response_template': '### Response:\n{bot_name}:', 'truncate_by_message': False}
Resubmit model
Shutdown handler not registered because Python interpreter is not running in the main thread
run pipeline %s
run pipeline stage %s
Running pipeline stage MKMLizer
Starting job with name chaiml-nemo-20241017-dar-6728-v2-mkmlizer
Waiting for job on chaiml-nemo-20241017-dar-6728-v2-mkmlizer to finish
chaiml-nemo-20241017-dar-6728-v2-mkmlizer: ╔═════════════════════════════════════════════════════════════════════╗
chaiml-nemo-20241017-dar-6728-v2-mkmlizer: ║     _____            __           __                                ║
chaiml-nemo-20241017-dar-6728-v2-mkmlizer: ║    / _/ /_ ___    __/ /  ___ ___ / /                                ║
chaiml-nemo-20241017-dar-6728-v2-mkmlizer: ║   / _/ / // / |/|/ / _ \/ -_) -_) /                                 ║
chaiml-nemo-20241017-dar-6728-v2-mkmlizer: ║  /_//_/\_, /|__,__/_//_/\__/\__/_/                                  ║
chaiml-nemo-20241017-dar-6728-v2-mkmlizer: ║       /___/                                                         ║
chaiml-nemo-20241017-dar-6728-v2-mkmlizer: ║                                                                     ║
chaiml-nemo-20241017-dar-6728-v2-mkmlizer: ║  Version: 0.11.12                                                   ║
chaiml-nemo-20241017-dar-6728-v2-mkmlizer: ║  Copyright 2023 MK ONE TECHNOLOGIES Inc.                            ║
chaiml-nemo-20241017-dar-6728-v2-mkmlizer: ║  https://mk1.ai                                                     ║
chaiml-nemo-20241017-dar-6728-v2-mkmlizer: ║                                                                     ║
chaiml-nemo-20241017-dar-6728-v2-mkmlizer: ║  The license key for the current software has been verified as      ║
chaiml-nemo-20241017-dar-6728-v2-mkmlizer: ║  belonging to:                                                      ║
chaiml-nemo-20241017-dar-6728-v2-mkmlizer: ║                                                                     ║
chaiml-nemo-20241017-dar-6728-v2-mkmlizer: ║  Chai Research Corp.                                                ║
chaiml-nemo-20241017-dar-6728-v2-mkmlizer: ║  Account ID: 7997a29f-0ceb-4cc7-9adf-840c57b4ae6f                   ║
chaiml-nemo-20241017-dar-6728-v2-mkmlizer: ║  Expiration: 2025-01-15 23:59:59                                    ║
chaiml-nemo-20241017-dar-6728-v2-mkmlizer: ║                                                                     ║
chaiml-nemo-20241017-dar-6728-v2-mkmlizer: ╚═════════════════════════════════════════════════════════════════════╝
chaiml-nemo-20241017-dar-6728-v2-mkmlizer: Downloaded to shared memory in 32.991s
chaiml-nemo-20241017-dar-6728-v2-mkmlizer: quantizing model to /dev/shm/model_cache, profile:s0, folder:/tmp/tmpmzf8owj7, device:0
chaiml-nemo-20241017-dar-6728-v2-mkmlizer: Saving flywheel model at /dev/shm/model_cache
chaiml-nemo-20241017-dar-6728-v2-mkmlizer: quantized model in 36.147s
chaiml-nemo-20241017-dar-6728-v2-mkmlizer: Processed model ChaiML/nemo-20241017-dare_linear-remerge_v2_5merge-albert in 69.138s
chaiml-nemo-20241017-dar-6728-v2-mkmlizer: creating bucket guanaco-mkml-models
chaiml-nemo-20241017-dar-6728-v2-mkmlizer: Bucket 's3://guanaco-mkml-models/' created
chaiml-nemo-20241017-dar-6728-v2-mkmlizer: uploading /dev/shm/model_cache to s3://guanaco-mkml-models/chaiml-nemo-20241017-dar-6728-v2
chaiml-nemo-20241017-dar-6728-v2-mkmlizer: cp /dev/shm/model_cache/config.json s3://guanaco-mkml-models/chaiml-nemo-20241017-dar-6728-v2/config.json
chaiml-nemo-20241017-dar-6728-v2-mkmlizer: cp /dev/shm/model_cache/special_tokens_map.json s3://guanaco-mkml-models/chaiml-nemo-20241017-dar-6728-v2/special_tokens_map.json
chaiml-nemo-20241017-dar-6728-v2-mkmlizer: cp /dev/shm/model_cache/tokenizer_config.json s3://guanaco-mkml-models/chaiml-nemo-20241017-dar-6728-v2/tokenizer_config.json
chaiml-nemo-20241017-dar-6728-v2-mkmlizer: cp /dev/shm/model_cache/tokenizer.json s3://guanaco-mkml-models/chaiml-nemo-20241017-dar-6728-v2/tokenizer.json
chaiml-nemo-20241017-dar-6728-v2-mkmlizer: cp /dev/shm/model_cache/flywheel_model.0.safetensors s3://guanaco-mkml-models/chaiml-nemo-20241017-dar-6728-v2/flywheel_model.0.safetensors
chaiml-nemo-20241017-dar-6728-v2-mkmlizer: 
Loading 0:   0%|          | 0/363 [00:00<?, ?it/s]
Loading 0:   1%|          | 2/363 [00:06<18:27,  3.07s/it]
Loading 0:   2%|▏         | 6/363 [00:06<04:55,  1.21it/s]
Loading 0:   4%|▎         | 13/363 [00:06<01:45,  3.33it/s]
Loading 0:   5%|▍         | 18/363 [00:06<01:05,  5.29it/s]
Loading 0:   6%|▋         | 23/363 [00:06<00:43,  7.81it/s]
Loading 0:   8%|▊         | 29/363 [00:06<00:29, 11.39it/s]
Loading 0:   9%|▉         | 34/363 [00:06<00:22, 14.43it/s]
Loading 0:  11%|█         | 40/363 [00:07<00:20, 15.62it/s]
Loading 0:  12%|█▏        | 44/363 [00:07<00:17, 18.21it/s]
Loading 0:  14%|█▍        | 50/363 [00:07<00:13, 23.82it/s]
Loading 0:  15%|█▌        | 56/363 [00:07<00:10, 27.99it/s]
Loading 0:  17%|█▋        | 61/363 [00:07<00:09, 30.86it/s]
Loading 0:  19%|█▊        | 68/363 [00:07<00:07, 37.34it/s]
Loading 0:  20%|██        | 74/363 [00:07<00:07, 38.58it/s]
Loading 0:  22%|██▏       | 79/363 [00:08<00:07, 39.89it/s]
Loading 0:  24%|██▎       | 86/363 [00:08<00:06, 46.01it/s]
Loading 0:  25%|██▌       | 92/363 [00:08<00:06, 42.98it/s]
Loading 0:  27%|██▋       | 97/363 [00:08<00:06, 42.00it/s]
Loading 0:  29%|██▊       | 104/363 [00:08<00:05, 47.07it/s]
Loading 0:  30%|███       | 110/363 [00:08<00:05, 46.55it/s]
Loading 0:  32%|███▏      | 115/363 [00:08<00:05, 45.39it/s]
Loading 0:  33%|███▎      | 121/363 [00:09<00:07, 34.23it/s]
Loading 0:  34%|███▍      | 125/363 [00:09<00:06, 34.17it/s]
Loading 0:  36%|███▌      | 130/363 [00:09<00:06, 37.48it/s]
Loading 0:  37%|███▋      | 135/363 [00:09<00:05, 40.20it/s]
Loading 0:  39%|███▊      | 140/363 [00:09<00:05, 42.15it/s]
Loading 0:  40%|████      | 146/363 [00:09<00:05, 42.09it/s]
Loading 0:  42%|████▏     | 151/363 [00:09<00:05, 40.83it/s]
Loading 0:  44%|████▎     | 158/363 [00:09<00:04, 45.57it/s]
Loading 0:  45%|████▍     | 163/363 [00:10<00:04, 46.67it/s]
Loading 0:  46%|████▋     | 168/363 [00:10<00:05, 35.98it/s]
Loading 0:  48%|████▊     | 175/363 [00:10<00:04, 41.97it/s]
Loading 0:  50%|████▉     | 180/363 [00:10<00:04, 41.48it/s]
Loading 0:  51%|█████     | 185/363 [00:10<00:04, 41.87it/s]
Loading 0:  53%|█████▎    | 191/363 [00:10<00:04, 41.36it/s]
Loading 0:  54%|█████▍    | 196/363 [00:10<00:04, 40.25it/s]
Loading 0:  56%|█████▌    | 202/363 [00:11<00:05, 30.50it/s]
Loading 0:  57%|█████▋    | 206/363 [00:11<00:05, 31.30it/s]
Loading 0:  58%|█████▊    | 212/363 [00:11<00:04, 35.82it/s]
Loading 0:  60%|█████▉    | 216/363 [00:11<00:04, 36.47it/s]
Loading 0:  61%|██████    | 220/363 [00:11<00:03, 36.98it/s]
Loading 0:  62%|██████▏   | 225/363 [00:11<00:03, 39.82it/s]
Loading 0:  63%|██████▎   | 230/363 [00:11<00:03, 42.32it/s]
Loading 0:  65%|██████▌   | 236/363 [00:11<00:02, 42.80it/s]
Loading 0:  66%|██████▋   | 241/363 [00:12<00:02, 42.49it/s]
Loading 0:  68%|██████▊   | 248/363 [00:12<00:02, 47.71it/s]
Loading 0:  70%|██████▉   | 253/363 [00:12<00:02, 48.13it/s]
Loading 0:  71%|███████   | 258/363 [00:12<00:02, 40.00it/s]
Loading 0:  73%|███████▎  | 266/363 [00:12<00:02, 47.52it/s]
Loading 0:  75%|███████▍  | 271/363 [00:12<00:01, 47.60it/s]
Loading 0:  76%|███████▌  | 276/363 [00:12<00:02, 40.69it/s]
Loading 0:  78%|███████▊  | 283/363 [00:13<00:02, 33.79it/s]
Loading 0:  79%|███████▉  | 287/363 [00:13<00:02, 34.15it/s]
Loading 0:  81%|████████  | 293/363 [00:13<00:01, 38.64it/s]
Loading 0:  82%|████████▏ | 299/363 [00:13<00:01, 40.40it/s]
Loading 0:  84%|████████▎ | 304/363 [00:13<00:01, 40.37it/s]
Loading 0:  86%|████████▌ | 311/363 [00:13<00:01, 45.57it/s]
Loading 0:  87%|████████▋ | 317/363 [00:13<00:01, 44.69it/s]
Loading 0:  89%|████████▊ | 322/363 [00:14<00:00, 42.77it/s]
Loading 0:  90%|█████████ | 328/363 [00:14<00:00, 46.45it/s]
Loading 0:  92%|█████████▏| 333/363 [00:14<00:00, 47.04it/s]
Loading 0:  93%|█████████▎| 339/363 [00:14<00:00, 42.98it/s]
Loading 0:  95%|█████████▌| 346/363 [00:14<00:00, 49.48it/s]
Loading 0:  97%|█████████▋| 352/363 [00:14<00:00, 50.96it/s]
Loading 0:  99%|█████████▊| 358/363 [00:14<00:00, 46.05it/s]
                                                            

Job chaiml-nemo-20241017-dar-6728-v2-mkmlizer completed after 93.48s with status: succeeded
Stopping job with name chaiml-nemo-20241017-dar-6728-v2-mkmlizer
Pipeline stage MKMLizer completed in 94.07s
run pipeline stage %s
Running pipeline stage MKMLTemplater
Pipeline stage MKMLTemplater completed in 0.18s
run pipeline stage %s
Running pipeline stage MKMLDeployer
Creating inference service chaiml-nemo-20241017-dar-6728-v2
Waiting for inference service chaiml-nemo-20241017-dar-6728-v2 to be ready
Inference service chaiml-nemo-20241017-dar-6728-v2 ready after 190.93827557563782s
Pipeline stage MKMLDeployer completed in 191.55s
run pipeline stage %s
Running pipeline stage StressChecker
Received healthy response to inference request in 2.03255558013916s
Received healthy response to inference request in 1.522388219833374s
Received healthy response to inference request in 1.456514835357666s
Received healthy response to inference request in 1.4722344875335693s
Received healthy response to inference request in 1.9293079376220703s
5 requests
0 failed requests
5th percentile: 1.4596587657928466
10th percentile: 1.4628026962280274
20th percentile: 1.4690905570983888
30th percentile: 1.4822652339935303
40th percentile: 1.5023267269134521
50th percentile: 1.522388219833374
60th percentile: 1.6851561069488525
70th percentile: 1.847923994064331
80th percentile: 1.9499574661254884
90th percentile: 1.9912565231323243
95th percentile: 2.011906051635742
99th percentile: 2.0284256744384765
mean time: 1.682600212097168
Pipeline stage StressChecker completed in 9.72s
Shutdown handler de-registered
chaiml-nemo-20241017-dar_6728_v2 status is now deployed due to DeploymentManager action
chaiml-nemo-20241017-dar_6728_v2 status is now inactive due to auto deactivation removed underperforming models
chaiml-nemo-20241017-dar_6728_v2 status is now torndown due to DeploymentManager action