developer_uid: azuruce
submission_id: chaiml-nemo-chai-5merge-ties_v5
model_name: chaiml-nemo-chai-5merge-ties_v3
model_group: ChaiML/nemo-chai-5merge-
status: torndown
timestamp: 2024-10-08T20:30:40+00:00
num_battles: 7555
num_wins: 3961
celo_rating: 1273.6
family_friendly_score: 0.5741203125722476
family_friendly_standard_error: 0.00579666208109646
submission_type: basic
model_repo: ChaiML/nemo-chai-5merge-ties
model_architecture: MistralForCausalLM
model_num_parameters: 12772070400.0
best_of: 8
max_input_tokens: 1024
max_output_tokens: 64
display_name: chaiml-nemo-chai-5merge-ties_v3
is_internal_developer: True
language_model: ChaiML/nemo-chai-5merge-ties
model_size: 13B
ranking_group: single
us_pacific_date: 2024-10-08
win_ratio: 0.5242885506287227
generation_params: {'temperature': 1.0, 'top_p': 1.0, 'min_p': 0.0, 'top_k': 40, 'presence_penalty': 0.0, 'frequency_penalty': 0.0, 'stopping_words': ['\n', '<|eot_id|>', '<|end_of_text|>', '\nYou:'], 'max_input_tokens': 1024, 'best_of': 8, 'max_output_tokens': 64}
formatter: {'memory_template': '', 'prompt_template': '', 'bot_template': '{bot_name}: {message}\n', 'user_template': '{user_name}: {message}\n', 'response_template': '### Response:\n{bot_name}:', 'truncate_by_message': False}
Resubmit model
Shutdown handler not registered because Python interpreter is not running in the main thread
run pipeline %s
run pipeline stage %s
Running pipeline stage MKMLizer
Starting job with name chaiml-nemo-chai-5merge-ties-v5-mkmlizer
Waiting for job on chaiml-nemo-chai-5merge-ties-v5-mkmlizer to finish
chaiml-nemo-chai-5merge-ties-v5-mkmlizer: ╔═════════════════════════════════════════════════════════════════════╗
chaiml-nemo-chai-5merge-ties-v5-mkmlizer: ║     _____            __           __                                ║
chaiml-nemo-chai-5merge-ties-v5-mkmlizer: ║    / _/ /_ ___    __/ /  ___ ___ / /                                ║
chaiml-nemo-chai-5merge-ties-v5-mkmlizer: ║   / _/ / // / |/|/ / _ \/ -_) -_) /                                 ║
chaiml-nemo-chai-5merge-ties-v5-mkmlizer: ║  /_//_/\_, /|__,__/_//_/\__/\__/_/                                  ║
chaiml-nemo-chai-5merge-ties-v5-mkmlizer: ║       /___/                                                         ║
chaiml-nemo-chai-5merge-ties-v5-mkmlizer: ║                                                                     ║
chaiml-nemo-chai-5merge-ties-v5-mkmlizer: ║  Version: 0.11.12                                                   ║
chaiml-nemo-chai-5merge-ties-v5-mkmlizer: ║  Copyright 2023 MK ONE TECHNOLOGIES Inc.                            ║
chaiml-nemo-chai-5merge-ties-v5-mkmlizer: ║  https://mk1.ai                                                     ║
chaiml-nemo-chai-5merge-ties-v5-mkmlizer: ║                                                                     ║
chaiml-nemo-chai-5merge-ties-v5-mkmlizer: ║  The license key for the current software has been verified as      ║
chaiml-nemo-chai-5merge-ties-v5-mkmlizer: ║  belonging to:                                                      ║
chaiml-nemo-chai-5merge-ties-v5-mkmlizer: ║                                                                     ║
chaiml-nemo-chai-5merge-ties-v5-mkmlizer: ║  Chai Research Corp.                                                ║
chaiml-nemo-chai-5merge-ties-v5-mkmlizer: ║  Account ID: 7997a29f-0ceb-4cc7-9adf-840c57b4ae6f                   ║
chaiml-nemo-chai-5merge-ties-v5-mkmlizer: ║  Expiration: 2024-10-15 23:59:59                                    ║
chaiml-nemo-chai-5merge-ties-v5-mkmlizer: ║                                                                     ║
chaiml-nemo-chai-5merge-ties-v5-mkmlizer: ╚═════════════════════════════════════════════════════════════════════╝
chaiml-nemo-chai-5merge-ties-v5-mkmlizer: Downloaded to shared memory in 45.532s
chaiml-nemo-chai-5merge-ties-v5-mkmlizer: quantizing model to /dev/shm/model_cache, profile:s0, folder:/tmp/tmp049azikv, device:0
chaiml-nemo-chai-5merge-ties-v5-mkmlizer: Saving flywheel model at /dev/shm/model_cache
chaiml-nemo-chai-5merge-ties-v5-mkmlizer: quantized model in 38.264s
chaiml-nemo-chai-5merge-ties-v5-mkmlizer: Processed model ChaiML/nemo-chai-5merge-ties in 83.796s
chaiml-nemo-chai-5merge-ties-v5-mkmlizer: creating bucket guanaco-mkml-models
chaiml-nemo-chai-5merge-ties-v5-mkmlizer: Bucket 's3://guanaco-mkml-models/' created
chaiml-nemo-chai-5merge-ties-v5-mkmlizer: uploading /dev/shm/model_cache to s3://guanaco-mkml-models/chaiml-nemo-chai-5merge-ties-v5
chaiml-nemo-chai-5merge-ties-v5-mkmlizer: cp /dev/shm/model_cache/config.json s3://guanaco-mkml-models/chaiml-nemo-chai-5merge-ties-v5/config.json
chaiml-nemo-chai-5merge-ties-v5-mkmlizer: cp /dev/shm/model_cache/special_tokens_map.json s3://guanaco-mkml-models/chaiml-nemo-chai-5merge-ties-v5/special_tokens_map.json
chaiml-nemo-chai-5merge-ties-v5-mkmlizer: cp /dev/shm/model_cache/tokenizer_config.json s3://guanaco-mkml-models/chaiml-nemo-chai-5merge-ties-v5/tokenizer_config.json
chaiml-nemo-chai-5merge-ties-v5-mkmlizer: cp /dev/shm/model_cache/tokenizer.json s3://guanaco-mkml-models/chaiml-nemo-chai-5merge-ties-v5/tokenizer.json
chaiml-nemo-chai-5merge-ties-v5-mkmlizer: cp /dev/shm/model_cache/flywheel_model.0.safetensors s3://guanaco-mkml-models/chaiml-nemo-chai-5merge-ties-v5/flywheel_model.0.safetensors
chaiml-nemo-chai-5merge-ties-v5-mkmlizer: 
Loading 0:   0%|          | 0/363 [00:00<?, ?it/s]
Loading 0:   1%|          | 2/363 [00:06<18:44,  3.12s/it]
Loading 0:   2%|▏         | 6/363 [00:06<05:00,  1.19it/s]
Loading 0:   3%|▎         | 11/363 [00:06<02:11,  2.67it/s]
Loading 0:   4%|▍         | 15/363 [00:06<01:23,  4.17it/s]
Loading 0:   6%|▌         | 22/363 [00:06<00:43,  7.79it/s]
Loading 0:   7%|▋         | 27/363 [00:06<00:31, 10.72it/s]
Loading 0:   9%|▉         | 32/363 [00:06<00:23, 14.23it/s]
Loading 0:  10%|█         | 37/363 [00:07<00:17, 18.29it/s]
Loading 0:  12%|█▏        | 42/363 [00:07<00:20, 15.62it/s]
Loading 0:  13%|█▎        | 49/363 [00:07<00:14, 22.22it/s]
Loading 0:  15%|█▍        | 54/363 [00:07<00:12, 25.46it/s]
Loading 0:  16%|█▋        | 59/363 [00:07<00:10, 28.42it/s]
Loading 0:  18%|█▊        | 64/363 [00:07<00:09, 31.87it/s]
Loading 0:  19%|█▉        | 69/363 [00:08<00:10, 28.72it/s]
Loading 0:  21%|██        | 76/363 [00:08<00:08, 35.76it/s]
Loading 0:  22%|██▏       | 81/363 [00:08<00:07, 36.22it/s]
Loading 0:  24%|██▎       | 86/363 [00:08<00:07, 36.39it/s]
Loading 0:  25%|██▌       | 91/363 [00:08<00:07, 38.04it/s]
Loading 0:  26%|██▋       | 96/363 [00:08<00:08, 31.22it/s]
Loading 0:  28%|██▊       | 103/363 [00:09<00:06, 38.21it/s]
Loading 0:  30%|██▉       | 108/363 [00:09<00:06, 37.80it/s]
Loading 0:  31%|███       | 113/363 [00:09<00:06, 37.72it/s]
Loading 0:  33%|███▎      | 118/363 [00:09<00:06, 39.12it/s]
Loading 0:  34%|███▍      | 123/363 [00:09<00:10, 22.38it/s]
Loading 0:  36%|███▌      | 130/363 [00:10<00:08, 28.62it/s]
Loading 0:  37%|███▋      | 134/363 [00:10<00:07, 29.34it/s]
Loading 0:  38%|███▊      | 139/363 [00:10<00:06, 33.05it/s]
Loading 0:  40%|███▉      | 144/363 [00:10<00:06, 34.24it/s]
Loading 0:  41%|████      | 149/363 [00:10<00:05, 36.02it/s]
Loading 0:  42%|████▏     | 153/363 [00:10<00:05, 36.54it/s]
Loading 0:  44%|████▎     | 158/363 [00:10<00:05, 38.00it/s]
Loading 0:  45%|████▍     | 163/363 [00:10<00:04, 40.14it/s]
Loading 0:  46%|████▋     | 168/363 [00:11<00:05, 33.44it/s]
Loading 0:  48%|████▊     | 175/363 [00:11<00:04, 40.03it/s]
Loading 0:  50%|████▉     | 180/363 [00:11<00:04, 39.25it/s]
Loading 0:  51%|█████     | 185/363 [00:11<00:04, 39.16it/s]
Loading 0:  52%|█████▏    | 190/363 [00:11<00:04, 39.90it/s]
Loading 0:  54%|█████▎    | 195/363 [00:11<00:04, 33.84it/s]
Loading 0:  56%|█████▌    | 202/363 [00:12<00:05, 29.08it/s]
Loading 0:  57%|█████▋    | 206/363 [00:12<00:05, 29.91it/s]
Loading 0:  58%|█████▊    | 212/363 [00:12<00:04, 35.05it/s]
Loading 0:  60%|██████    | 218/363 [00:12<00:03, 36.88it/s]
Loading 0:  61%|██████    | 222/363 [00:12<00:04, 33.44it/s]
Loading 0:  63%|██████▎   | 229/363 [00:12<00:03, 40.72it/s]
Loading 0:  64%|██████▍   | 234/363 [00:12<00:03, 42.08it/s]
Loading 0:  66%|██████▌   | 239/363 [00:12<00:02, 43.24it/s]
Loading 0:  67%|██████▋   | 244/363 [00:12<00:02, 44.57it/s]
Loading 0:  69%|██████▊   | 249/363 [00:13<00:03, 36.80it/s]
Loading 0:  71%|███████   | 256/363 [00:13<00:02, 44.42it/s]
Loading 0:  72%|███████▏  | 261/363 [00:13<00:02, 42.82it/s]
Loading 0:  73%|███████▎  | 266/363 [00:13<00:02, 41.77it/s]
Loading 0:  75%|███████▍  | 271/363 [00:13<00:02, 41.99it/s]
Loading 0:  76%|███████▌  | 276/363 [00:13<00:02, 35.48it/s]
Loading 0:  78%|███████▊  | 283/363 [00:14<00:02, 29.83it/s]
Loading 0:  79%|███████▉  | 287/363 [00:14<00:02, 30.64it/s]
Loading 0:  81%|████████  | 293/363 [00:14<00:01, 35.23it/s]
Loading 0:  82%|████████▏ | 298/363 [00:14<00:01, 38.33it/s]
Loading 0:  83%|████████▎ | 303/363 [00:14<00:01, 32.78it/s]
Loading 0:  85%|████████▌ | 310/363 [00:14<00:01, 39.55it/s]
Loading 0:  87%|████████▋ | 315/363 [00:14<00:01, 39.89it/s]
Loading 0:  88%|████████▊ | 320/363 [00:15<00:01, 41.39it/s]
Loading 0:  90%|████████▉ | 325/363 [00:15<00:00, 43.24it/s]
Loading 0:  91%|█████████ | 330/363 [00:15<00:00, 36.04it/s]
Loading 0:  93%|█████████▎| 338/363 [00:15<00:00, 43.04it/s]
Loading 0:  94%|█████████▍| 343/363 [00:15<00:00, 43.64it/s]
Loading 0:  96%|█████████▌| 348/363 [00:15<00:00, 34.20it/s]
Loading 0:  98%|█████████▊| 355/363 [00:15<00:00, 41.59it/s]
Loading 0:  99%|█████████▉| 360/363 [00:16<00:00, 41.00it/s]
                                                            

Job chaiml-nemo-chai-5merge-ties-v5-mkmlizer completed after 121.95s with status: succeeded
Stopping job with name chaiml-nemo-chai-5merge-ties-v5-mkmlizer
Pipeline stage MKMLizer completed in 122.50s
run pipeline stage %s
Running pipeline stage MKMLTemplater
Pipeline stage MKMLTemplater completed in 0.16s
run pipeline stage %s
Running pipeline stage MKMLDeployer
Creating inference service chaiml-nemo-chai-5merge-ties-v5
Waiting for inference service chaiml-nemo-chai-5merge-ties-v5 to be ready
Inference service chaiml-nemo-chai-5merge-ties-v5 ready after 180.81880974769592s
Pipeline stage MKMLDeployer completed in 181.35s
run pipeline stage %s
Running pipeline stage StressChecker
Received healthy response to inference request in 2.3061704635620117s
Received healthy response to inference request in 1.7466278076171875s
Received healthy response to inference request in 1.4576129913330078s
Received healthy response to inference request in 1.758101463317871s
Received healthy response to inference request in 1.4534857273101807s
5 requests
0 failed requests
5th percentile: 1.4543111801147461
10th percentile: 1.4551366329193116
20th percentile: 1.4567875385284423
30th percentile: 1.5154159545898438
40th percentile: 1.6310218811035155
50th percentile: 1.7466278076171875
60th percentile: 1.7512172698974608
70th percentile: 1.7558067321777344
80th percentile: 1.8677152633666994
90th percentile: 2.0869428634643556
95th percentile: 2.1965566635131837
99th percentile: 2.284247703552246
mean time: 1.7443996906280517
Pipeline stage StressChecker completed in 10.81s
Shutdown handler de-registered
chaiml-nemo-chai-5merge-ties_v5 status is now deployed due to DeploymentManager action
chaiml-nemo-chai-5merge-ties_v5 status is now inactive due to auto deactivation removed underperforming models
chaiml-nemo-chai-5merge-ties_v5 status is now torndown due to DeploymentManager action