chaiml-nemo-20241010-t_5991

submission_id: chaiml-nemo-20241010-t_5991_v135
developer_uid: chai_backend_admin
best_of: 4
celo_rating: 1262.04
display_name: chaiml-nemo-20241010-t_5991_v135
family_friendly_score: 0.5747234576348501
family_friendly_standard_error: 0.006436249080570099
formatter: {'memory_template': '', 'prompt_template': '', 'bot_template': '{bot_name}: {message}\n', 'user_template': '{user_name}: {message}\n', 'response_template': '### Response:\n{bot_name}:', 'truncate_by_message': False}
generation_params: {'temperature': 1.0, 'top_p': 1.0, 'min_p': 0.0, 'top_k': 40, 'presence_penalty': 0.0, 'frequency_penalty': 0.0, 'stopping_words': ['\n', '<|eot_id|>', '<|end_of_text|>', 'You:'], 'max_input_tokens': 1024, 'best_of': 4, 'max_output_tokens': 64}
is_internal_developer: True
language_model: ChaiML/nemo-20241010_tier_merge_v4-albert
max_input_tokens: 1024
max_output_tokens: 64
model_architecture: MistralForCausalLM
model_group: ChaiML/nemo-20241010_tie
model_name: chaiml-nemo-20241010-t_5991_v135
model_num_parameters: 12772070400.0
model_repo: ChaiML/nemo-20241010_tier_merge_v4-albert
model_size: 13B
num_battles: 6115
num_wins: 3363
ranking_group: single
status: torndown
submission_type: basic
timestamp: 2024-10-15T03:00:19+00:00
us_pacific_date: 2024-10-14
win_ratio: 0.5499591169255929
Resubmit model
Shutdown handler not registered because Python interpreter is not running in the main thread
run pipeline %s
run pipeline stage %s
Running pipeline stage MKMLizer
Starting job with name chaiml-nemo-20241010-t-5991-v135-mkmlizer
Waiting for job on chaiml-nemo-20241010-t-5991-v135-mkmlizer to finish
chaiml-nemo-20241010-t-5991-v135-mkmlizer: ╔═════════════════════════════════════════════════════════════════════╗
chaiml-nemo-20241010-t-5991-v135-mkmlizer: ║     _____            __           __                                ║
chaiml-nemo-20241010-t-5991-v135-mkmlizer: ║    / _/ /_ ___    __/ /  ___ ___ / /                                ║
chaiml-nemo-20241010-t-5991-v135-mkmlizer: ║   / _/ / // / |/|/ / _ \/ -_) -_) /                                 ║
chaiml-nemo-20241010-t-5991-v135-mkmlizer: ║  /_//_/\_, /|__,__/_//_/\__/\__/_/                                  ║
chaiml-nemo-20241010-t-5991-v135-mkmlizer: ║       /___/                                                         ║
chaiml-nemo-20241010-t-5991-v135-mkmlizer: ║                                                                     ║
chaiml-nemo-20241010-t-5991-v135-mkmlizer: ║  Version: 0.11.12                                                   ║
chaiml-nemo-20241010-t-5991-v135-mkmlizer: ║  Copyright 2023 MK ONE TECHNOLOGIES Inc.                            ║
chaiml-nemo-20241010-t-5991-v135-mkmlizer: ║  https://mk1.ai                                                     ║
chaiml-nemo-20241010-t-5991-v135-mkmlizer: ║                                                                     ║
chaiml-nemo-20241010-t-5991-v135-mkmlizer: ║  The license key for the current software has been verified as      ║
chaiml-nemo-20241010-t-5991-v135-mkmlizer: ║  belonging to:                                                      ║
chaiml-nemo-20241010-t-5991-v135-mkmlizer: ║                                                                     ║
chaiml-nemo-20241010-t-5991-v135-mkmlizer: ║  Chai Research Corp.                                                ║
chaiml-nemo-20241010-t-5991-v135-mkmlizer: ║  Account ID: 7997a29f-0ceb-4cc7-9adf-840c57b4ae6f                   ║
chaiml-nemo-20241010-t-5991-v135-mkmlizer: ║  Expiration: 2024-10-15 23:59:59                                    ║
chaiml-nemo-20241010-t-5991-v135-mkmlizer: ║                                                                     ║
chaiml-nemo-20241010-t-5991-v135-mkmlizer: ╚═════════════════════════════════════════════════════════════════════╝
chaiml-nemo-20241010-t-5991-v135-mkmlizer: Downloaded to shared memory in 30.568s
chaiml-nemo-20241010-t-5991-v135-mkmlizer: quantizing model to /dev/shm/model_cache, profile:s0, folder:/tmp/tmpf1aau4rd, device:0
chaiml-nemo-20241010-t-5991-v135-mkmlizer: Saving flywheel model at /dev/shm/model_cache
chaiml-nemo-20241010-t-5991-v135-mkmlizer: quantized model in 36.302s
chaiml-nemo-20241010-t-5991-v135-mkmlizer: Processed model ChaiML/nemo-20241010_tier_merge_v4-albert in 66.869s
chaiml-nemo-20241010-t-5991-v135-mkmlizer: creating bucket guanaco-mkml-models
chaiml-nemo-20241010-t-5991-v135-mkmlizer: Bucket 's3://guanaco-mkml-models/' created
chaiml-nemo-20241010-t-5991-v135-mkmlizer: uploading /dev/shm/model_cache to s3://guanaco-mkml-models/chaiml-nemo-20241010-t-5991-v135
chaiml-nemo-20241010-t-5991-v135-mkmlizer: cp /dev/shm/model_cache/config.json s3://guanaco-mkml-models/chaiml-nemo-20241010-t-5991-v135/config.json
chaiml-nemo-20241010-t-5991-v135-mkmlizer: cp /dev/shm/model_cache/special_tokens_map.json s3://guanaco-mkml-models/chaiml-nemo-20241010-t-5991-v135/special_tokens_map.json
chaiml-nemo-20241010-t-5991-v135-mkmlizer: cp /dev/shm/model_cache/tokenizer_config.json s3://guanaco-mkml-models/chaiml-nemo-20241010-t-5991-v135/tokenizer_config.json
chaiml-nemo-20241010-t-5991-v135-mkmlizer: cp /dev/shm/model_cache/tokenizer.json s3://guanaco-mkml-models/chaiml-nemo-20241010-t-5991-v135/tokenizer.json
chaiml-nemo-20241010-t-5991-v135-mkmlizer: cp /dev/shm/model_cache/flywheel_model.0.safetensors s3://guanaco-mkml-models/chaiml-nemo-20241010-t-5991-v135/flywheel_model.0.safetensors
chaiml-nemo-20241010-t-5991-v135-mkmlizer: 
Loading 0:   0%|          | 0/363 [00:00<?, ?it/s]
Loading 0:   1%|          | 2/363 [00:06<18:07,  3.01s/it]
Loading 0:   2%|▏         | 6/363 [00:06<04:50,  1.23it/s]
Loading 0:   3%|▎         | 11/363 [00:06<02:07,  2.76it/s]
Loading 0:   4%|▍         | 15/363 [00:06<01:20,  4.31it/s]
Loading 0:   6%|▌         | 22/363 [00:06<00:42,  8.07it/s]
Loading 0:   7%|▋         | 27/363 [00:06<00:30, 11.10it/s]
Loading 0:   9%|▉         | 32/363 [00:06<00:22, 14.71it/s]
Loading 0:  10%|█         | 38/363 [00:06<00:17, 18.91it/s]
Loading 0:  12%|█▏        | 43/363 [00:07<00:18, 17.38it/s]
Loading 0:  13%|█▎        | 49/363 [00:07<00:14, 22.40it/s]
Loading 0:  15%|█▍        | 53/363 [00:07<00:12, 24.72it/s]
Loading 0:  16%|█▌        | 58/363 [00:07<00:10, 29.17it/s]
Loading 0:  17%|█▋        | 63/363 [00:07<00:09, 32.32it/s]
Loading 0:  19%|█▊        | 68/363 [00:07<00:08, 35.20it/s]
Loading 0:  20%|██        | 74/363 [00:07<00:08, 36.00it/s]
Loading 0:  22%|██▏       | 79/363 [00:08<00:07, 35.99it/s]
Loading 0:  23%|██▎       | 85/363 [00:08<00:06, 40.50it/s]
Loading 0:  25%|██▍       | 90/363 [00:08<00:06, 40.88it/s]
Loading 0:  26%|██▌       | 95/363 [00:08<00:06, 41.93it/s]
Loading 0:  28%|██▊       | 101/363 [00:08<00:06, 41.00it/s]
Loading 0:  29%|██▉       | 106/363 [00:08<00:06, 40.38it/s]
Loading 0:  31%|███       | 112/363 [00:08<00:05, 43.84it/s]
Loading 0:  32%|███▏      | 117/363 [00:08<00:05, 43.04it/s]
Loading 0:  34%|███▎      | 122/363 [00:09<00:08, 29.81it/s]
Loading 0:  35%|███▍      | 127/363 [00:09<00:07, 33.61it/s]
Loading 0:  36%|███▌      | 131/363 [00:09<00:06, 34.95it/s]
Loading 0:  38%|███▊      | 137/363 [00:09<00:06, 35.72it/s]
Loading 0:  39%|███▉      | 141/363 [00:09<00:06, 34.19it/s]
Loading 0:  41%|████      | 148/363 [00:09<00:05, 41.98it/s]
Loading 0:  42%|████▏     | 153/363 [00:09<00:04, 42.05it/s]
Loading 0:  44%|████▎     | 158/363 [00:10<00:04, 42.02it/s]
Loading 0:  45%|████▍     | 163/363 [00:10<00:04, 42.22it/s]
Loading 0:  46%|████▋     | 168/363 [00:10<00:05, 35.47it/s]
Loading 0:  48%|████▊     | 175/363 [00:10<00:04, 42.65it/s]
Loading 0:  50%|████▉     | 180/363 [00:10<00:04, 42.56it/s]
Loading 0:  51%|█████     | 185/363 [00:10<00:04, 42.85it/s]
Loading 0:  52%|█████▏    | 190/363 [00:10<00:03, 44.42it/s]
Loading 0:  54%|█████▎    | 195/363 [00:11<00:04, 37.20it/s]
Loading 0:  56%|█████▌    | 202/363 [00:11<00:05, 30.18it/s]
Loading 0:  57%|█████▋    | 206/363 [00:11<00:05, 30.86it/s]
Loading 0:  58%|█████▊    | 211/363 [00:11<00:04, 34.57it/s]
Loading 0:  59%|█████▉    | 215/363 [00:11<00:04, 33.98it/s]
Loading 0:  61%|██████    | 221/363 [00:11<00:03, 38.34it/s]
Loading 0:  62%|██████▏   | 226/363 [00:11<00:03, 41.06it/s]
Loading 0:  64%|██████▎   | 231/363 [00:12<00:03, 35.28it/s]
Loading 0:  66%|██████▌   | 238/363 [00:12<00:02, 42.74it/s]
Loading 0:  67%|██████▋   | 243/363 [00:12<00:02, 43.42it/s]
Loading 0:  68%|██████▊   | 248/363 [00:12<00:02, 44.14it/s]
Loading 0:  70%|██████▉   | 254/363 [00:12<00:02, 42.34it/s]
Loading 0:  71%|███████▏  | 259/363 [00:12<00:02, 42.16it/s]
Loading 0:  73%|███████▎  | 265/363 [00:12<00:02, 46.36it/s]
Loading 0:  74%|███████▍  | 270/363 [00:12<00:02, 45.95it/s]
Loading 0:  76%|███████▌  | 275/363 [00:13<00:01, 46.03it/s]
Loading 0:  77%|███████▋  | 281/363 [00:13<00:01, 43.33it/s]
Loading 0:  79%|███████▉  | 286/363 [00:13<00:02, 29.33it/s]
Loading 0:  80%|████████  | 292/363 [00:13<00:02, 35.02it/s]
Loading 0:  82%|████████▏ | 297/363 [00:13<00:01, 37.37it/s]
Loading 0:  83%|████████▎ | 302/363 [00:13<00:01, 38.30it/s]
Loading 0:  85%|████████▍ | 307/363 [00:13<00:01, 40.56it/s]
Loading 0:  86%|████████▌ | 312/363 [00:14<00:01, 35.25it/s]
Loading 0:  88%|████████▊ | 320/363 [00:14<00:00, 43.62it/s]
Loading 0:  90%|████████▉ | 326/363 [00:14<00:00, 41.94it/s]
Loading 0:  91%|█████████ | 331/363 [00:14<00:00, 41.48it/s]
Loading 0:  93%|█████████▎| 337/363 [00:14<00:00, 45.75it/s]
Loading 0:  94%|█████████▍| 342/363 [00:14<00:00, 45.24it/s]
Loading 0:  96%|█████████▌| 347/363 [00:14<00:00, 45.23it/s]
Loading 0:  97%|█████████▋| 353/363 [00:15<00:00, 42.49it/s]
Loading 0:  99%|█████████▊| 358/363 [00:15<00:00, 41.90it/s]
                                                            
Job chaiml-nemo-20241010-t-5991-v135-mkmlizer completed after 94.18s with status: succeeded
Stopping job with name chaiml-nemo-20241010-t-5991-v135-mkmlizer
Pipeline stage MKMLizer completed in 94.72s
run pipeline stage %s
Running pipeline stage MKMLTemplater
Pipeline stage MKMLTemplater completed in 0.19s
run pipeline stage %s
Running pipeline stage MKMLDeployer
Creating inference service chaiml-nemo-20241010-t-5991-v135
Waiting for inference service chaiml-nemo-20241010-t-5991-v135 to be ready
Inference service chaiml-nemo-20241010-t-5991-v135 ready after 150.76031827926636s
Pipeline stage MKMLDeployer completed in 151.26s
run pipeline stage %s
Running pipeline stage StressChecker
HTTPConnectionPool(host='guanaco-submitter.guanaco-backend.k2.chaiverse.com', port=80): Read timed out. (read timeout=20)
Received unhealthy response to inference request!
Received healthy response to inference request in 1.8322505950927734s
Received healthy response to inference request in 1.6616151332855225s
Received healthy response to inference request in 1.2918624877929688s
Received healthy response to inference request in 1.5386242866516113s
5 requests
1 failed requests
5th percentile: 1.3412148475646972
10th percentile: 1.3905672073364257
20th percentile: 1.4892719268798829
30th percentile: 1.5632224559783936
40th percentile: 1.612418794631958
50th percentile: 1.6616151332855225
60th percentile: 1.7298693180084228
70th percentile: 1.7981235027313232
80th percentile: 5.500444936752323
90th percentile: 12.836833620071413
95th percentile: 16.505027961730953
99th percentile: 19.439583435058594
mean time: 5.299514961242676
%s, retrying in %s seconds...
Received healthy response to inference request in 1.3355073928833008s
Received healthy response to inference request in 1.358034610748291s
Received healthy response to inference request in 1.3974425792694092s
Received healthy response to inference request in 1.4417304992675781s
Received healthy response to inference request in 1.5425810813903809s
5 requests
0 failed requests
5th percentile: 1.3400128364562989
10th percentile: 1.344518280029297
20th percentile: 1.353529167175293
30th percentile: 1.3659162044525146
40th percentile: 1.381679391860962
50th percentile: 1.3974425792694092
60th percentile: 1.4151577472686767
70th percentile: 1.4328729152679442
80th percentile: 1.4619006156921386
90th percentile: 1.5022408485412597
95th percentile: 1.5224109649658204
99th percentile: 1.5385470581054688
mean time: 1.415059232711792
Pipeline stage StressChecker completed in 36.05s
Shutdown handler de-registered
chaiml-nemo-20241010-t_5991_v135 status is now deployed due to DeploymentManager action
chaiml-nemo-20241010-t_5991_v135 status is now inactive due to auto deactivation removed underperforming models
chaiml-nemo-20241010-t_5991_v135 status is now torndown due to DeploymentManager action