chaiml-nemo-20241010-ti_5991

developer_uid: chai_backend_admin

submission_id: chaiml-nemo-20241010-ti_5991_v52

model_name: chaiml-nemo-20241010-ti_5991_v52

model_group: ChaiML/nemo-20241010_tie

status: torndown

timestamp: 2024-10-13T05:00:27+00:00

num_battles: 6369

num_wins: 3262

celo_rating: 1262.43

family_friendly_score: 0.5773059057730591

family_friendly_standard_error: 0.0063318545708251055

submission_type: basic

model_repo: ChaiML/nemo-20241010_tier_merge_v4-albert

model_architecture: MistralForCausalLM

model_num_parameters: 12772070400.0

best_of: 4

max_input_tokens: 1024

max_output_tokens: 64

display_name: chaiml-nemo-20241010-ti_5991_v52

is_internal_developer: True

language_model: ChaiML/nemo-20241010_tier_merge_v4-albert

model_size: 13B

ranking_group: single

us_pacific_date: 2024-10-12

win_ratio: 0.5121683152771236

generation_params: {'temperature': 1.0, 'top_p': 1.0, 'min_p': 0.0, 'top_k': 40, 'presence_penalty': 0.0, 'frequency_penalty': 0.0, 'stopping_words': ['\n', '<|eot_id|>', '<|end_of_text|>', 'You:'], 'max_input_tokens': 1024, 'best_of': 4, 'max_output_tokens': 64}

formatter: {'memory_template': '', 'prompt_template': '', 'bot_template': '{bot_name}: {message}\n', 'user_template': '{user_name}: {message}\n', 'response_template': '### Response:\n{bot_name}:', 'truncate_by_message': False}

Resubmit model

Shutdown handler not registered because Python interpreter is not running in the main thread
run pipeline %s
run pipeline stage %s
Running pipeline stage MKMLizer
Starting job with name chaiml-nemo-20241010-ti-5991-v52-mkmlizer
Waiting for job on chaiml-nemo-20241010-ti-5991-v52-mkmlizer to finish
chaiml-nemo-20241010-ti-5991-v52-mkmlizer: ╔═════════════════════════════════════════════════════════════════════╗
chaiml-nemo-20241010-ti-5991-v52-mkmlizer: ║     _____            __           __                                ║
chaiml-nemo-20241010-ti-5991-v52-mkmlizer: ║    / _/ /_ ___    __/ /  ___ ___ / /                                ║
chaiml-nemo-20241010-ti-5991-v52-mkmlizer: ║   / _/ / // / |/|/ / _ \/ -_) -_) /                                 ║
chaiml-nemo-20241010-ti-5991-v52-mkmlizer: ║  /_//_/\_, /|__,__/_//_/\__/\__/_/                                  ║
chaiml-nemo-20241010-ti-5991-v52-mkmlizer: ║       /___/                                                         ║
chaiml-nemo-20241010-ti-5991-v52-mkmlizer: ║                                                                     ║
chaiml-nemo-20241010-ti-5991-v52-mkmlizer: ║  Version: 0.11.12                                                   ║
chaiml-nemo-20241010-ti-5991-v52-mkmlizer: ║  Copyright 2023 MK ONE TECHNOLOGIES Inc.                            ║
chaiml-nemo-20241010-ti-5991-v52-mkmlizer: ║  https://mk1.ai                                                     ║
chaiml-nemo-20241010-ti-5991-v52-mkmlizer: ║                                                                     ║
chaiml-nemo-20241010-ti-5991-v52-mkmlizer: ║  The license key for the current software has been verified as      ║
chaiml-nemo-20241010-ti-5991-v52-mkmlizer: ║  belonging to:                                                      ║
chaiml-nemo-20241010-ti-5991-v52-mkmlizer: ║                                                                     ║
chaiml-nemo-20241010-ti-5991-v52-mkmlizer: ║  Chai Research Corp.                                                ║
chaiml-nemo-20241010-ti-5991-v52-mkmlizer: ║  Account ID: 7997a29f-0ceb-4cc7-9adf-840c57b4ae6f                   ║
chaiml-nemo-20241010-ti-5991-v52-mkmlizer: ║  Expiration: 2024-10-15 23:59:59                                    ║
chaiml-nemo-20241010-ti-5991-v52-mkmlizer: ║                                                                     ║
chaiml-nemo-20241010-ti-5991-v52-mkmlizer: ╚═════════════════════════════════════════════════════════════════════╝
chaiml-nemo-20241010-ti-5991-v52-mkmlizer: Downloaded to shared memory in 35.013s
chaiml-nemo-20241010-ti-5991-v52-mkmlizer: quantizing model to /dev/shm/model_cache, profile:s0, folder:/tmp/tmpj6hi86f9, device:0
chaiml-nemo-20241010-ti-5991-v52-mkmlizer: Saving flywheel model at /dev/shm/model_cache
chaiml-nemo-20241010-ti-5991-v52-mkmlizer: quantized model in 36.353s
chaiml-nemo-20241010-ti-5991-v52-mkmlizer: Processed model ChaiML/nemo-20241010_tier_merge_v4-albert in 71.367s
chaiml-nemo-20241010-ti-5991-v52-mkmlizer: creating bucket guanaco-mkml-models
chaiml-nemo-20241010-ti-5991-v52-mkmlizer: Bucket 's3://guanaco-mkml-models/' created
chaiml-nemo-20241010-ti-5991-v52-mkmlizer: uploading /dev/shm/model_cache to s3://guanaco-mkml-models/chaiml-nemo-20241010-ti-5991-v52
chaiml-nemo-20241010-ti-5991-v52-mkmlizer: cp /dev/shm/model_cache/special_tokens_map.json s3://guanaco-mkml-models/chaiml-nemo-20241010-ti-5991-v52/special_tokens_map.json
chaiml-nemo-20241010-ti-5991-v52-mkmlizer: cp /dev/shm/model_cache/config.json s3://guanaco-mkml-models/chaiml-nemo-20241010-ti-5991-v52/config.json
chaiml-nemo-20241010-ti-5991-v52-mkmlizer: cp /dev/shm/model_cache/tokenizer_config.json s3://guanaco-mkml-models/chaiml-nemo-20241010-ti-5991-v52/tokenizer_config.json
chaiml-nemo-20241010-ti-5991-v52-mkmlizer: cp /dev/shm/model_cache/tokenizer.json s3://guanaco-mkml-models/chaiml-nemo-20241010-ti-5991-v52/tokenizer.json
chaiml-nemo-20241010-ti-5991-v52-mkmlizer: cp /dev/shm/model_cache/flywheel_model.0.safetensors s3://guanaco-mkml-models/chaiml-nemo-20241010-ti-5991-v52/flywheel_model.0.safetensors
chaiml-nemo-20241010-ti-5991-v52-mkmlizer: 
Loading 0:   0%|          | 0/363 [00:00<?, ?it/s]
Loading 0:   1%|          | 2/363 [00:06<18:20,  3.05s/it]
Loading 0:   2%|▏         | 6/363 [00:06<04:54,  1.21it/s]
Loading 0:   3%|▎         | 11/363 [00:06<02:08,  2.73it/s]
Loading 0:   4%|▍         | 15/363 [00:06<01:21,  4.28it/s]
Loading 0:   6%|▋         | 23/363 [00:06<00:40,  8.49it/s]
Loading 0:   8%|▊         | 29/363 [00:06<00:28, 11.92it/s]
Loading 0:   9%|▉         | 34/363 [00:06<00:21, 15.12it/s]
Loading 0:  11%|█         | 40/363 [00:07<00:19, 16.85it/s]
Loading 0:  12%|█▏        | 44/363 [00:07<00:16, 19.28it/s]
Loading 0:  14%|█▍        | 50/363 [00:07<00:12, 24.69it/s]
Loading 0:  15%|█▌        | 55/363 [00:07<00:10, 28.94it/s]
Loading 0:  17%|█▋        | 60/363 [00:07<00:10, 28.29it/s]
Loading 0:  19%|█▊        | 68/363 [00:07<00:08, 36.79it/s]
Loading 0:  20%|██        | 73/363 [00:07<00:07, 38.83it/s]
Loading 0:  21%|██▏       | 78/363 [00:08<00:08, 34.58it/s]
Loading 0:  23%|██▎       | 85/363 [00:08<00:06, 41.69it/s]
Loading 0:  25%|██▍       | 90/363 [00:08<00:06, 42.93it/s]
Loading 0:  26%|██▌       | 95/363 [00:08<00:06, 43.34it/s]
Loading 0:  28%|██▊       | 101/363 [00:08<00:06, 43.08it/s]
Loading 0:  29%|██▉       | 106/363 [00:08<00:05, 43.04it/s]
Loading 0:  31%|███       | 112/363 [00:08<00:05, 47.16it/s]
Loading 0:  32%|███▏      | 117/363 [00:08<00:05, 46.22it/s]
Loading 0:  34%|███▎      | 122/363 [00:09<00:07, 31.96it/s]
Loading 0:  35%|███▍      | 126/363 [00:09<00:07, 33.49it/s]
Loading 0:  36%|███▌      | 131/363 [00:09<00:06, 36.83it/s]
Loading 0:  37%|███▋      | 136/363 [00:09<00:05, 38.63it/s]
Loading 0:  39%|███▉      | 141/363 [00:09<00:06, 34.15it/s]
Loading 0:  41%|████      | 149/363 [00:09<00:05, 42.48it/s]
Loading 0:  43%|████▎     | 155/363 [00:09<00:04, 41.65it/s]
Loading 0:  44%|████▍     | 160/363 [00:10<00:05, 40.49it/s]
Loading 0:  46%|████▌     | 167/363 [00:10<00:04, 45.03it/s]
Loading 0:  47%|████▋     | 172/363 [00:10<00:04, 46.18it/s]
Loading 0:  49%|████▉     | 177/363 [00:10<00:04, 39.02it/s]
Loading 0:  51%|█████     | 185/363 [00:10<00:03, 46.28it/s]
Loading 0:  53%|█████▎    | 191/363 [00:10<00:03, 44.87it/s]
Loading 0:  54%|█████▍    | 196/363 [00:10<00:03, 44.62it/s]
Loading 0:  56%|█████▌    | 202/363 [00:11<00:04, 34.30it/s]
Loading 0:  57%|█████▋    | 206/363 [00:11<00:04, 34.41it/s]
Loading 0:  58%|█████▊    | 212/363 [00:11<00:03, 39.20it/s]
Loading 0:  60%|██████    | 218/363 [00:11<00:03, 39.50it/s]
Loading 0:  61%|██████▏   | 223/363 [00:11<00:03, 39.23it/s]
Loading 0:  63%|██████▎   | 230/363 [00:11<00:02, 45.06it/s]
Loading 0:  65%|██████▌   | 236/363 [00:11<00:02, 44.62it/s]
Loading 0:  66%|██████▋   | 241/363 [00:12<00:02, 43.56it/s]
Loading 0:  68%|██████▊   | 248/363 [00:12<00:02, 47.93it/s]
Loading 0:  70%|██████▉   | 254/363 [00:12<00:02, 45.97it/s]
Loading 0:  71%|███████▏  | 259/363 [00:12<00:02, 44.52it/s]
Loading 0:  73%|███████▎  | 266/363 [00:12<00:01, 48.82it/s]
Loading 0:  75%|███████▍  | 272/363 [00:12<00:01, 46.47it/s]
Loading 0:  76%|███████▋  | 277/363 [00:12<00:01, 45.04it/s]
Loading 0:  78%|███████▊  | 283/363 [00:13<00:02, 34.54it/s]
Loading 0:  79%|███████▉  | 287/363 [00:13<00:02, 34.63it/s]
Loading 0:  81%|████████  | 293/363 [00:13<00:01, 38.73it/s]
Loading 0:  82%|████████▏ | 299/363 [00:13<00:01, 38.26it/s]
Loading 0:  84%|████████▎ | 304/363 [00:13<00:01, 36.72it/s]
Loading 0:  85%|████████▌ | 310/363 [00:13<00:01, 40.61it/s]
Loading 0:  87%|████████▋ | 315/363 [00:13<00:01, 40.30it/s]
Loading 0:  88%|████████▊ | 320/363 [00:13<00:01, 41.76it/s]
Loading 0:  90%|████████▉ | 325/363 [00:14<00:00, 42.92it/s]
Loading 0:  91%|█████████ | 330/363 [00:14<00:00, 36.28it/s]
Loading 0:  93%|█████████▎| 337/363 [00:14<00:00, 43.98it/s]
Loading 0:  94%|█████████▍| 342/363 [00:14<00:00, 43.42it/s]
Loading 0:  96%|█████████▌| 347/363 [00:14<00:00, 42.34it/s]
Loading 0:  97%|█████████▋| 353/363 [00:14<00:00, 41.69it/s]
Loading 0:  99%|█████████▊| 358/363 [00:14<00:00, 41.90it/s]
                                                            
Job chaiml-nemo-20241010-ti-5991-v52-mkmlizer completed after 93.88s with status: succeeded
Stopping job with name chaiml-nemo-20241010-ti-5991-v52-mkmlizer
Pipeline stage MKMLizer completed in 94.45s
run pipeline stage %s
Running pipeline stage MKMLTemplater
Pipeline stage MKMLTemplater completed in 0.17s
run pipeline stage %s
Running pipeline stage MKMLDeployer
Creating inference service chaiml-nemo-20241010-ti-5991-v52
Waiting for inference service chaiml-nemo-20241010-ti-5991-v52 to be ready
Inference service chaiml-nemo-20241010-ti-5991-v52 ready after 170.6105179786682s
Pipeline stage MKMLDeployer completed in 171.22s
run pipeline stage %s
Running pipeline stage StressChecker
Received healthy response to inference request in 2.9272091388702393s
Received healthy response to inference request in 1.550969123840332s
Received healthy response to inference request in 1.8910293579101562s
Received healthy response to inference request in 1.4452316761016846s
Received healthy response to inference request in 2.124035120010376s
5 requests
0 failed requests
5th percentile: 1.466379165649414
10th percentile: 1.4875266551971436
20th percentile: 1.5298216342926025
30th percentile: 1.618981170654297
40th percentile: 1.7550052642822265
50th percentile: 1.8910293579101562
60th percentile: 1.984231662750244
70th percentile: 2.077433967590332
80th percentile: 2.284669923782349
90th percentile: 2.605939531326294
95th percentile: 2.7665743350982663
99th percentile: 2.8950821781158447
mean time: 1.9876948833465575
Pipeline stage StressChecker completed in 11.51s
Shutdown handler de-registered
chaiml-nemo-20241010-ti_5991_v52 status is now deployed due to DeploymentManager action
chaiml-nemo-20241010-ti_5991_v52 status is now inactive due to auto deactivation removed underperforming models
chaiml-nemo-20241010-ti_5991_v52 status is now torndown due to DeploymentManager action