chaiml-nemo-20241010-t_5991

submission_id: chaiml-nemo-20241010-t_5991_v110

developer_uid: chai_backend_admin

best_of: 4

celo_rating: 1262.33

display_name: chaiml-nemo-20241010-t_5991_v110

family_friendly_score: 0.584083704303251

family_friendly_standard_error: 0.004927540163129929

formatter: {'memory_template': '', 'prompt_template': '', 'bot_template': '{bot_name}: {message}\n', 'user_template': '{user_name}: {message}\n', 'response_template': '### Response:\n{bot_name}:', 'truncate_by_message': False}

generation_params: {'temperature': 1.0, 'top_p': 1.0, 'min_p': 0.0, 'top_k': 40, 'presence_penalty': 0.0, 'frequency_penalty': 0.0, 'stopping_words': ['\n', '<|eot_id|>', '<|end_of_text|>', 'You:'], 'max_input_tokens': 1024, 'best_of': 4, 'max_output_tokens': 64}

is_internal_developer: True

language_model: ChaiML/nemo-20241010_tier_merge_v4-albert

max_input_tokens: 1024

max_output_tokens: 64

model_architecture: MistralForCausalLM

model_group: ChaiML/nemo-20241010_tie

model_name: chaiml-nemo-20241010-t_5991_v110

model_num_parameters: 12772070400.0

model_repo: ChaiML/nemo-20241010_tier_merge_v4-albert

model_size: 13B

num_battles: 10655

num_wins: 5472

ranking_group: single

status: torndown

submission_type: basic

timestamp: 2024-10-14T13:00:36+00:00

us_pacific_date: 2024-10-14

win_ratio: 0.5135617081182543

Resubmit model

Shutdown handler not registered because Python interpreter is not running in the main thread
run pipeline %s
run pipeline stage %s
Running pipeline stage MKMLizer
Starting job with name chaiml-nemo-20241010-t-5991-v110-mkmlizer
Waiting for job on chaiml-nemo-20241010-t-5991-v110-mkmlizer to finish
chaiml-nemo-20241010-t-5991-v110-mkmlizer: ╔═════════════════════════════════════════════════════════════════════╗
chaiml-nemo-20241010-t-5991-v110-mkmlizer: ║     _____            __           __                                ║
chaiml-nemo-20241010-t-5991-v110-mkmlizer: ║    / _/ /_ ___    __/ /  ___ ___ / /                                ║
chaiml-nemo-20241010-t-5991-v110-mkmlizer: ║   / _/ / // / |/|/ / _ \/ -_) -_) /                                 ║
chaiml-nemo-20241010-t-5991-v110-mkmlizer: ║  /_//_/\_, /|__,__/_//_/\__/\__/_/                                  ║
chaiml-nemo-20241010-t-5991-v110-mkmlizer: ║       /___/                                                         ║
chaiml-nemo-20241010-t-5991-v110-mkmlizer: ║                                                                     ║
chaiml-nemo-20241010-t-5991-v110-mkmlizer: ║  Version: 0.11.12                                                   ║
chaiml-nemo-20241010-t-5991-v110-mkmlizer: ║  Copyright 2023 MK ONE TECHNOLOGIES Inc.                            ║
chaiml-nemo-20241010-t-5991-v110-mkmlizer: ║  https://mk1.ai                                                     ║
chaiml-nemo-20241010-t-5991-v110-mkmlizer: ║                                                                     ║
chaiml-nemo-20241010-t-5991-v110-mkmlizer: ║  The license key for the current software has been verified as      ║
chaiml-nemo-20241010-t-5991-v110-mkmlizer: ║  belonging to:                                                      ║
chaiml-nemo-20241010-t-5991-v110-mkmlizer: ║                                                                     ║
chaiml-nemo-20241010-t-5991-v110-mkmlizer: ║  Chai Research Corp.                                                ║
chaiml-nemo-20241010-t-5991-v110-mkmlizer: ║  Account ID: 7997a29f-0ceb-4cc7-9adf-840c57b4ae6f                   ║
chaiml-nemo-20241010-t-5991-v110-mkmlizer: ║  Expiration: 2024-10-15 23:59:59                                    ║
chaiml-nemo-20241010-t-5991-v110-mkmlizer: ║                                                                     ║
chaiml-nemo-20241010-t-5991-v110-mkmlizer: ╚═════════════════════════════════════════════════════════════════════╝
chaiml-nemo-20241010-t-5991-v110-mkmlizer: Downloaded to shared memory in 31.496s
chaiml-nemo-20241010-t-5991-v110-mkmlizer: quantizing model to /dev/shm/model_cache, profile:s0, folder:/tmp/tmpv1mz6__5, device:0
chaiml-nemo-20241010-t-5991-v110-mkmlizer: Saving flywheel model at /dev/shm/model_cache
chaiml-nemo-20241010-t-5991-v110-mkmlizer: quantized model in 36.862s
chaiml-nemo-20241010-t-5991-v110-mkmlizer: Processed model ChaiML/nemo-20241010_tier_merge_v4-albert in 68.358s
chaiml-nemo-20241010-t-5991-v110-mkmlizer: creating bucket guanaco-mkml-models
chaiml-nemo-20241010-t-5991-v110-mkmlizer: Bucket 's3://guanaco-mkml-models/' created
chaiml-nemo-20241010-t-5991-v110-mkmlizer: uploading /dev/shm/model_cache to s3://guanaco-mkml-models/chaiml-nemo-20241010-t-5991-v110
chaiml-nemo-20241010-t-5991-v110-mkmlizer: cp /dev/shm/model_cache/special_tokens_map.json s3://guanaco-mkml-models/chaiml-nemo-20241010-t-5991-v110/special_tokens_map.json
chaiml-nemo-20241010-t-5991-v110-mkmlizer: cp /dev/shm/model_cache/config.json s3://guanaco-mkml-models/chaiml-nemo-20241010-t-5991-v110/config.json
chaiml-nemo-20241010-t-5991-v110-mkmlizer: cp /dev/shm/model_cache/tokenizer_config.json s3://guanaco-mkml-models/chaiml-nemo-20241010-t-5991-v110/tokenizer_config.json
chaiml-nemo-20241010-t-5991-v110-mkmlizer: cp /dev/shm/model_cache/tokenizer.json s3://guanaco-mkml-models/chaiml-nemo-20241010-t-5991-v110/tokenizer.json
chaiml-nemo-20241010-t-5991-v110-mkmlizer: cp /dev/shm/model_cache/flywheel_model.0.safetensors s3://guanaco-mkml-models/chaiml-nemo-20241010-t-5991-v110/flywheel_model.0.safetensors
chaiml-nemo-20241010-t-5991-v110-mkmlizer: 
Loading 0:   0%|          | 0/363 [00:00<?, ?it/s]
Loading 0:   1%|          | 2/363 [00:06<18:20,  3.05s/it]
Loading 0:   2%|▏         | 6/363 [00:06<04:54,  1.21it/s]
Loading 0:   4%|▎         | 13/363 [00:06<01:44,  3.34it/s]
Loading 0:   5%|▍         | 17/363 [00:06<01:10,  4.88it/s]
Loading 0:   6%|▋         | 23/363 [00:06<00:42,  7.97it/s]
Loading 0:   8%|▊         | 29/363 [00:06<00:29, 11.46it/s]
Loading 0:   9%|▉         | 34/363 [00:06<00:22, 14.76it/s]
Loading 0:  11%|█         | 40/363 [00:07<00:19, 16.39it/s]
Loading 0:  12%|█▏        | 44/363 [00:07<00:16, 18.79it/s]
Loading 0:  14%|█▍        | 50/363 [00:07<00:12, 24.24it/s]
Loading 0:  15%|█▌        | 56/363 [00:07<00:10, 28.00it/s]
Loading 0:  17%|█▋        | 61/363 [00:07<00:09, 30.74it/s]
Loading 0:  19%|█▊        | 68/363 [00:07<00:07, 37.15it/s]
Loading 0:  20%|██        | 74/363 [00:07<00:07, 38.74it/s]
Loading 0:  22%|██▏       | 79/363 [00:08<00:07, 38.01it/s]
Loading 0:  24%|██▎       | 86/363 [00:08<00:06, 43.33it/s]
Loading 0:  25%|██▌       | 92/363 [00:08<00:06, 42.75it/s]
Loading 0:  27%|██▋       | 97/363 [00:08<00:06, 41.46it/s]
Loading 0:  29%|██▊       | 104/363 [00:08<00:05, 46.41it/s]
Loading 0:  30%|███       | 110/363 [00:08<00:05, 44.36it/s]
Loading 0:  32%|███▏      | 115/363 [00:08<00:05, 41.56it/s]
Loading 0:  33%|███▎      | 121/363 [00:09<00:07, 31.42it/s]
Loading 0:  34%|███▍      | 125/363 [00:09<00:07, 31.77it/s]
Loading 0:  36%|███▌      | 130/363 [00:09<00:06, 35.29it/s]
Loading 0:  37%|███▋      | 134/363 [00:09<00:06, 33.51it/s]
Loading 0:  38%|███▊      | 139/363 [00:09<00:06, 36.31it/s]
Loading 0:  39%|███▉      | 143/363 [00:09<00:06, 35.63it/s]
Loading 0:  41%|████      | 149/363 [00:09<00:05, 39.60it/s]
Loading 0:  42%|████▏     | 154/363 [00:09<00:05, 41.38it/s]
Loading 0:  44%|████▍     | 159/363 [00:10<00:05, 35.33it/s]
Loading 0:  46%|████▌     | 166/363 [00:10<00:04, 43.11it/s]
Loading 0:  47%|████▋     | 171/363 [00:10<00:04, 40.83it/s]
Loading 0:  48%|████▊     | 176/363 [00:10<00:04, 40.77it/s]
Loading 0:  50%|████▉     | 181/363 [00:10<00:04, 41.49it/s]
Loading 0:  51%|█████     | 186/363 [00:10<00:05, 35.21it/s]
Loading 0:  53%|█████▎    | 194/363 [00:10<00:03, 43.93it/s]
Loading 0:  55%|█████▌    | 200/363 [00:11<00:03, 42.55it/s]
Loading 0:  56%|█████▋    | 205/363 [00:11<00:05, 27.91it/s]
Loading 0:  58%|█████▊    | 211/363 [00:11<00:04, 32.41it/s]
Loading 0:  60%|█████▉    | 216/363 [00:11<00:04, 33.75it/s]
Loading 0:  61%|██████    | 221/363 [00:11<00:03, 35.63it/s]
Loading 0:  62%|██████▏   | 226/363 [00:11<00:03, 38.05it/s]
Loading 0:  64%|██████▎   | 231/363 [00:12<00:03, 35.42it/s]
Loading 0:  66%|██████▌   | 239/363 [00:12<00:02, 43.75it/s]
Loading 0:  67%|██████▋   | 245/363 [00:12<00:02, 40.26it/s]
Loading 0:  69%|██████▉   | 250/363 [00:12<00:02, 40.13it/s]
Loading 0:  71%|███████   | 257/363 [00:12<00:02, 45.56it/s]
Loading 0:  72%|███████▏  | 263/363 [00:12<00:02, 45.03it/s]
Loading 0:  74%|███████▍  | 268/363 [00:12<00:02, 44.16it/s]
Loading 0:  75%|███████▌  | 274/363 [00:13<00:01, 48.07it/s]
Loading 0:  77%|███████▋  | 279/363 [00:13<00:01, 45.93it/s]
Loading 0:  78%|███████▊  | 284/363 [00:13<00:02, 31.63it/s]
Loading 0:  79%|███████▉  | 288/363 [00:13<00:02, 33.17it/s]
Loading 0:  81%|████████  | 293/363 [00:13<00:01, 36.16it/s]
Loading 0:  82%|████████▏ | 298/363 [00:13<00:01, 39.23it/s]
Loading 0:  83%|████████▎ | 303/363 [00:13<00:01, 33.94it/s]
Loading 0:  85%|████████▌ | 310/363 [00:14<00:01, 41.67it/s]
Loading 0:  87%|████████▋ | 315/363 [00:14<00:01, 42.39it/s]
Loading 0:  88%|████████▊ | 320/363 [00:14<00:00, 43.22it/s]
Loading 0:  90%|████████▉ | 326/363 [00:14<00:00, 41.91it/s]
Loading 0:  91%|█████████ | 331/363 [00:14<00:00, 40.23it/s]
Loading 0:  93%|█████████▎| 338/363 [00:14<00:00, 45.64it/s]
Loading 0:  94%|█████████▍| 343/363 [00:14<00:00, 45.86it/s]
Loading 0:  96%|█████████▌| 348/363 [00:14<00:00, 36.24it/s]
Loading 0:  98%|█████████▊| 355/363 [00:15<00:00, 42.31it/s]
Loading 0:  99%|█████████▉| 360/363 [00:15<00:00, 43.03it/s]
                                                            
Job chaiml-nemo-20241010-t-5991-v110-mkmlizer completed after 93.91s with status: succeeded
Stopping job with name chaiml-nemo-20241010-t-5991-v110-mkmlizer
Pipeline stage MKMLizer completed in 94.46s
run pipeline stage %s
Running pipeline stage MKMLTemplater
Pipeline stage MKMLTemplater completed in 0.17s
run pipeline stage %s
Running pipeline stage MKMLDeployer
Creating inference service chaiml-nemo-20241010-t-5991-v110
Waiting for inference service chaiml-nemo-20241010-t-5991-v110 to be ready
Inference service chaiml-nemo-20241010-t-5991-v110 ready after 181.01652765274048s
Pipeline stage MKMLDeployer completed in 181.91s
run pipeline stage %s
Running pipeline stage StressChecker
Received healthy response to inference request in 1.8962736129760742s
Received healthy response to inference request in 1.6722338199615479s
Received healthy response to inference request in 1.4895269870758057s
Received healthy response to inference request in 1.360792875289917s
Received healthy response to inference request in 1.3215217590332031s
5 requests
0 failed requests
5th percentile: 1.329375982284546
10th percentile: 1.3372302055358887
20th percentile: 1.3529386520385742
30th percentile: 1.3865396976470947
40th percentile: 1.4380333423614502
50th percentile: 1.4895269870758057
60th percentile: 1.5626097202301026
70th percentile: 1.6356924533843993
80th percentile: 1.7170417785644532
90th percentile: 1.8066576957702636
95th percentile: 1.851465654373169
99th percentile: 1.8873120212554932
mean time: 1.5480698108673097
Pipeline stage StressChecker completed in 9.26s
Shutdown handler de-registered
chaiml-nemo-20241010-t_5991_v110 status is now deployed due to DeploymentManager action
chaiml-nemo-20241010-t_5991_v110 status is now inactive due to auto deactivation removed underperforming models
chaiml-nemo-20241010-t_5991_v110 status is now torndown due to DeploymentManager action