chaiml-nemo-20241010-ti_5991

submission_id: chaiml-nemo-20241010-ti_5991_v59

developer_uid: chai_backend_admin

best_of: 4

celo_rating: 1260.76

display_name: chaiml-nemo-20241010-ti_5991_v59

family_friendly_score: 0.5748049403449844

family_friendly_standard_error: 0.00504115174743818

formatter: {'memory_template': '', 'prompt_template': '', 'bot_template': '{bot_name}: {message}\n', 'user_template': '{user_name}: {message}\n', 'response_template': '### Response:\n{bot_name}:', 'truncate_by_message': False}

generation_params: {'temperature': 1.0, 'top_p': 1.0, 'min_p': 0.0, 'top_k': 40, 'presence_penalty': 0.0, 'frequency_penalty': 0.0, 'stopping_words': ['\n', '<|eot_id|>', '<|end_of_text|>', 'You:'], 'max_input_tokens': 1024, 'best_of': 4, 'max_output_tokens': 64}

is_internal_developer: True

language_model: ChaiML/nemo-20241010_tier_merge_v4-albert

max_input_tokens: 1024

max_output_tokens: 64

model_architecture: MistralForCausalLM

model_group: ChaiML/nemo-20241010_tie

model_name: chaiml-nemo-20241010-ti_5991_v59

model_num_parameters: 12772070400.0

model_repo: ChaiML/nemo-20241010_tier_merge_v4-albert

model_size: 13B

num_battles: 10231

num_wins: 5243

ranking_group: single

status: torndown

submission_type: basic

timestamp: 2024-10-13T09:00:45+00:00

us_pacific_date: 2024-10-13

win_ratio: 0.5124621249144756

Resubmit model

Shutdown handler not registered because Python interpreter is not running in the main thread
run pipeline %s
run pipeline stage %s
Running pipeline stage MKMLizer
Starting job with name chaiml-nemo-20241010-ti-5991-v59-mkmlizer
Waiting for job on chaiml-nemo-20241010-ti-5991-v59-mkmlizer to finish
chaiml-nemo-20241010-ti-5991-v59-mkmlizer: ╔═════════════════════════════════════════════════════════════════════╗
chaiml-nemo-20241010-ti-5991-v59-mkmlizer: ║     _____            __           __                                ║
chaiml-nemo-20241010-ti-5991-v59-mkmlizer: ║    / _/ /_ ___    __/ /  ___ ___ / /                                ║
chaiml-nemo-20241010-ti-5991-v59-mkmlizer: ║   / _/ / // / |/|/ / _ \/ -_) -_) /                                 ║
chaiml-nemo-20241010-ti-5991-v59-mkmlizer: ║  /_//_/\_, /|__,__/_//_/\__/\__/_/                                  ║
chaiml-nemo-20241010-ti-5991-v59-mkmlizer: ║       /___/                                                         ║
chaiml-nemo-20241010-ti-5991-v59-mkmlizer: ║                                                                     ║
chaiml-nemo-20241010-ti-5991-v59-mkmlizer: ║  Version: 0.11.12                                                   ║
chaiml-nemo-20241010-ti-5991-v59-mkmlizer: ║  Copyright 2023 MK ONE TECHNOLOGIES Inc.                            ║
chaiml-nemo-20241010-ti-5991-v59-mkmlizer: ║  https://mk1.ai                                                     ║
chaiml-nemo-20241010-ti-5991-v59-mkmlizer: ║                                                                     ║
chaiml-nemo-20241010-ti-5991-v59-mkmlizer: ║  The license key for the current software has been verified as      ║
chaiml-nemo-20241010-ti-5991-v59-mkmlizer: ║  belonging to:                                                      ║
chaiml-nemo-20241010-ti-5991-v59-mkmlizer: ║                                                                     ║
chaiml-nemo-20241010-ti-5991-v59-mkmlizer: ║  Chai Research Corp.                                                ║
chaiml-nemo-20241010-ti-5991-v59-mkmlizer: ║  Account ID: 7997a29f-0ceb-4cc7-9adf-840c57b4ae6f                   ║
chaiml-nemo-20241010-ti-5991-v59-mkmlizer: ║  Expiration: 2024-10-15 23:59:59                                    ║
chaiml-nemo-20241010-ti-5991-v59-mkmlizer: ║                                                                     ║
chaiml-nemo-20241010-ti-5991-v59-mkmlizer: ╚═════════════════════════════════════════════════════════════════════╝
chaiml-nemo-20241010-ti-5991-v59-mkmlizer: Downloaded to shared memory in 28.400s
chaiml-nemo-20241010-ti-5991-v59-mkmlizer: quantizing model to /dev/shm/model_cache, profile:s0, folder:/tmp/tmpeb0k8ied, device:0
chaiml-nemo-20241010-ti-5991-v59-mkmlizer: Saving flywheel model at /dev/shm/model_cache
chaiml-nemo-20241010-ti-5991-v59-mkmlizer: quantized model in 35.917s
chaiml-nemo-20241010-ti-5991-v59-mkmlizer: Processed model ChaiML/nemo-20241010_tier_merge_v4-albert in 64.318s
chaiml-nemo-20241010-ti-5991-v59-mkmlizer: creating bucket guanaco-mkml-models
chaiml-nemo-20241010-ti-5991-v59-mkmlizer: Bucket 's3://guanaco-mkml-models/' created
chaiml-nemo-20241010-ti-5991-v59-mkmlizer: uploading /dev/shm/model_cache to s3://guanaco-mkml-models/chaiml-nemo-20241010-ti-5991-v59
chaiml-nemo-20241010-ti-5991-v59-mkmlizer: cp /dev/shm/model_cache/config.json s3://guanaco-mkml-models/chaiml-nemo-20241010-ti-5991-v59/config.json
chaiml-nemo-20241010-ti-5991-v59-mkmlizer: cp /dev/shm/model_cache/special_tokens_map.json s3://guanaco-mkml-models/chaiml-nemo-20241010-ti-5991-v59/special_tokens_map.json
chaiml-nemo-20241010-ti-5991-v59-mkmlizer: cp /dev/shm/model_cache/tokenizer_config.json s3://guanaco-mkml-models/chaiml-nemo-20241010-ti-5991-v59/tokenizer_config.json
chaiml-nemo-20241010-ti-5991-v59-mkmlizer: cp /dev/shm/model_cache/flywheel_model.0.safetensors s3://guanaco-mkml-models/chaiml-nemo-20241010-ti-5991-v59/flywheel_model.0.safetensors
chaiml-nemo-20241010-ti-5991-v59-mkmlizer: 
Loading 0:   0%|          | 0/363 [00:00<?, ?it/s]
Loading 0:   1%|          | 2/363 [00:06<18:06,  3.01s/it]
Loading 0:   2%|▏         | 6/363 [00:06<04:50,  1.23it/s]
Loading 0:   3%|▎         | 11/363 [00:06<02:07,  2.77it/s]
Loading 0:   4%|▍         | 15/363 [00:06<01:20,  4.32it/s]
Loading 0:   6%|▌         | 22/363 [00:06<00:42,  8.04it/s]
Loading 0:   7%|▋         | 27/363 [00:06<00:30, 11.04it/s]
Loading 0:   9%|▉         | 32/363 [00:06<00:22, 14.66it/s]
Loading 0:  10%|█         | 38/363 [00:06<00:17, 18.85it/s]
Loading 0:  12%|█▏        | 43/363 [00:07<00:18, 17.69it/s]
Loading 0:  14%|█▍        | 50/363 [00:07<00:13, 23.86it/s]
Loading 0:  15%|█▌        | 55/363 [00:07<00:11, 27.68it/s]
Loading 0:  17%|█▋        | 60/363 [00:07<00:11, 27.41it/s]
Loading 0:  18%|█▊        | 67/363 [00:07<00:08, 34.86it/s]
Loading 0:  20%|█▉        | 72/363 [00:07<00:07, 36.99it/s]
Loading 0:  21%|██        | 77/363 [00:07<00:07, 39.03it/s]
Loading 0:  23%|██▎       | 83/363 [00:08<00:07, 39.04it/s]
Loading 0:  24%|██▍       | 88/363 [00:08<00:06, 39.64it/s]
Loading 0:  26%|██▌       | 94/363 [00:08<00:06, 43.36it/s]
Loading 0:  27%|██▋       | 99/363 [00:08<00:06, 43.76it/s]
Loading 0:  29%|██▊       | 104/363 [00:08<00:05, 44.66it/s]
Loading 0:  30%|███       | 110/363 [00:08<00:05, 42.46it/s]
Loading 0:  32%|███▏      | 115/363 [00:08<00:05, 41.69it/s]
Loading 0:  33%|███▎      | 121/363 [00:09<00:07, 32.10it/s]
Loading 0:  34%|███▍      | 125/363 [00:09<00:07, 33.14it/s]
Loading 0:  36%|███▌      | 131/363 [00:09<00:06, 37.83it/s]
Loading 0:  38%|███▊      | 137/363 [00:09<00:05, 38.04it/s]
Loading 0:  39%|███▉      | 142/363 [00:09<00:05, 38.50it/s]
Loading 0:  41%|████      | 148/363 [00:09<00:04, 43.47it/s]
Loading 0:  42%|████▏     | 153/363 [00:09<00:04, 42.55it/s]
Loading 0:  44%|████▎     | 158/363 [00:09<00:04, 43.21it/s]
Loading 0:  45%|████▌     | 164/363 [00:10<00:04, 41.67it/s]
Loading 0:  47%|████▋     | 169/363 [00:10<00:04, 41.38it/s]
Loading 0:  48%|████▊     | 175/363 [00:10<00:04, 44.89it/s]
Loading 0:  50%|████▉     | 180/363 [00:10<00:04, 44.75it/s]
Loading 0:  51%|█████     | 185/363 [00:10<00:04, 43.52it/s]
Loading 0:  53%|█████▎    | 191/363 [00:10<00:04, 41.65it/s]
Loading 0:  54%|█████▍    | 196/363 [00:10<00:04, 41.71it/s]
Loading 0:  56%|█████▌    | 202/363 [00:11<00:04, 32.44it/s]
Loading 0:  57%|█████▋    | 206/363 [00:11<00:04, 33.57it/s]
Loading 0:  58%|█████▊    | 212/363 [00:11<00:03, 37.89it/s]
Loading 0:  60%|█████▉    | 217/363 [00:11<00:03, 39.87it/s]
Loading 0:  61%|██████    | 222/363 [00:11<00:04, 34.43it/s]
Loading 0:  63%|██████▎   | 230/363 [00:11<00:03, 42.65it/s]
Loading 0:  65%|██████▌   | 236/363 [00:11<00:03, 41.18it/s]
Loading 0:  66%|██████▋   | 241/363 [00:12<00:02, 40.82it/s]
Loading 0:  68%|██████▊   | 247/363 [00:12<00:02, 45.26it/s]
Loading 0:  69%|██████▉   | 252/363 [00:12<00:02, 45.02it/s]
Loading 0:  71%|███████   | 257/363 [00:12<00:02, 45.50it/s]
Loading 0:  72%|███████▏  | 263/363 [00:12<00:02, 43.08it/s]
Loading 0:  74%|███████▍  | 268/363 [00:12<00:02, 42.11it/s]
Loading 0:  75%|███████▌  | 274/363 [00:12<00:01, 46.51it/s]
Loading 0:  77%|███████▋  | 279/363 [00:12<00:01, 45.91it/s]
Loading 0:  78%|███████▊  | 284/363 [00:13<00:02, 31.54it/s]
Loading 0:  80%|███████▉  | 289/363 [00:13<00:02, 35.27it/s]
Loading 0:  81%|████████  | 294/363 [00:13<00:02, 32.17it/s]
Loading 0:  83%|████████▎ | 301/363 [00:13<00:01, 39.87it/s]
Loading 0:  84%|████████▍ | 306/363 [00:13<00:01, 41.07it/s]
Loading 0:  86%|████████▌ | 311/363 [00:13<00:01, 42.34it/s]
Loading 0:  87%|████████▋ | 317/363 [00:13<00:01, 40.90it/s]
Loading 0:  89%|████████▊ | 322/363 [00:14<00:00, 41.15it/s]
Loading 0:  90%|█████████ | 328/363 [00:14<00:00, 45.20it/s]
Loading 0:  92%|█████████▏| 333/363 [00:14<00:00, 44.93it/s]
Loading 0:  93%|█████████▎| 338/363 [00:14<00:00, 45.72it/s]
Loading 0:  95%|█████████▍| 344/363 [00:14<00:00, 43.44it/s]
Loading 0:  96%|█████████▌| 349/363 [00:14<00:00, 42.74it/s]
Loading 0:  98%|█████████▊| 356/363 [00:14<00:00, 47.57it/s]
Loading 0:  99%|█████████▉| 361/363 [00:14<00:00, 47.98it/s]
                                                            
Job chaiml-nemo-20241010-ti-5991-v59-mkmlizer completed after 83.75s with status: succeeded
Stopping job with name chaiml-nemo-20241010-ti-5991-v59-mkmlizer
Pipeline stage MKMLizer completed in 84.28s
run pipeline stage %s
Running pipeline stage MKMLTemplater
Pipeline stage MKMLTemplater completed in 0.16s
run pipeline stage %s
Running pipeline stage MKMLDeployer
Creating inference service chaiml-nemo-20241010-ti-5991-v59
Waiting for inference service chaiml-nemo-20241010-ti-5991-v59 to be ready
Inference service chaiml-nemo-20241010-ti-5991-v59 ready after 170.643887758255s
Pipeline stage MKMLDeployer completed in 171.18s
run pipeline stage %s
Running pipeline stage StressChecker
Received healthy response to inference request in 2.0016627311706543s
Received healthy response to inference request in 2.1295876502990723s
Received healthy response to inference request in 1.7452647686004639s
Received healthy response to inference request in 1.719316005706787s
Received healthy response to inference request in 1.532515287399292s
5 requests
0 failed requests
5th percentile: 1.569875431060791
10th percentile: 1.60723557472229
20th percentile: 1.6819558620452881
30th percentile: 1.7245057582855225
40th percentile: 1.7348852634429932
50th percentile: 1.7452647686004639
60th percentile: 1.8478239536285401
70th percentile: 1.9503831386566162
80th percentile: 2.0272477149963377
90th percentile: 2.078417682647705
95th percentile: 2.104002666473389
99th percentile: 2.1244706535339355
mean time: 1.825669288635254
Pipeline stage StressChecker completed in 10.42s
Shutdown handler de-registered
chaiml-nemo-20241010-ti_5991_v59 status is now deployed due to DeploymentManager action
chaiml-nemo-20241010-ti_5991_v59 status is now inactive due to auto deactivation removed underperforming models
chaiml-nemo-20241010-ti_5991_v59 status is now torndown due to DeploymentManager action