bbchicago-nana-nemo-12b-v1-0

developer_uid: Bbbrun0

submission_id: bbchicago-nana-nemo-12b-v1-0_v8

model_name: bbchicago-nana-nemo-12b-v1-0_v1

model_group: BBChicago/Nana-nemo-12B_

status: torndown

timestamp: 2024-10-14T02:11:50+00:00

num_battles: 12246

num_wins: 7056

celo_rating: 1293.33

family_friendly_score: 0.5576935615321923

family_friendly_standard_error: 0.004872469366882793

submission_type: basic

model_repo: BBChicago/Nana-nemo-12B_v1.0

model_architecture: MistralForCausalLM

model_num_parameters: 12772070400.0

best_of: 8

max_input_tokens: 1024

max_output_tokens: 64

display_name: bbchicago-nana-nemo-12b-v1-0_v1

is_internal_developer: False

language_model: BBChicago/Nana-nemo-12B_v1.0

model_size: 13B

ranking_group: single

us_pacific_date: 2024-10-13

win_ratio: 0.576188143067124

generation_params: {'temperature': 0.95, 'top_p': 0.95, 'min_p': 0.05, 'top_k': 80, 'presence_penalty': 0.0, 'frequency_penalty': 0.0, 'stopping_words': ['\n', '</s>', 'Bot:', 'User:', 'You:', 'Me:', '####'], 'max_input_tokens': 1024, 'best_of': 8, 'max_output_tokens': 64}

formatter: {'memory_template': "{bot_name}'s Persona: {memory}\n####\n", 'prompt_template': '{prompt}\n<START>\n', 'bot_template': '{bot_name}: {message}\n', 'user_template': '{user_name}: {message}\n', 'response_template': '{bot_name}:', 'truncate_by_message': False}

Resubmit model

Shutdown handler not registered because Python interpreter is not running in the main thread
run pipeline %s
run pipeline stage %s
Running pipeline stage MKMLizer
Starting job with name bbchicago-nana-nemo-12b-v1-0-v8-mkmlizer
Waiting for job on bbchicago-nana-nemo-12b-v1-0-v8-mkmlizer to finish
bbchicago-nana-nemo-12b-v1-0-v8-mkmlizer: ╔═════════════════════════════════════════════════════════════════════╗
bbchicago-nana-nemo-12b-v1-0-v8-mkmlizer: ║     _____            __           __                                ║
bbchicago-nana-nemo-12b-v1-0-v8-mkmlizer: ║    / _/ /_ ___    __/ /  ___ ___ / /                                ║
bbchicago-nana-nemo-12b-v1-0-v8-mkmlizer: ║   / _/ / // / |/|/ / _ \/ -_) -_) /                                 ║
bbchicago-nana-nemo-12b-v1-0-v8-mkmlizer: ║  /_//_/\_, /|__,__/_//_/\__/\__/_/                                  ║
bbchicago-nana-nemo-12b-v1-0-v8-mkmlizer: ║       /___/                                                         ║
bbchicago-nana-nemo-12b-v1-0-v8-mkmlizer: ║                                                                     ║
bbchicago-nana-nemo-12b-v1-0-v8-mkmlizer: ║  Version: 0.11.12                                                   ║
bbchicago-nana-nemo-12b-v1-0-v8-mkmlizer: ║  Copyright 2023 MK ONE TECHNOLOGIES Inc.                            ║
bbchicago-nana-nemo-12b-v1-0-v8-mkmlizer: ║  https://mk1.ai                                                     ║
bbchicago-nana-nemo-12b-v1-0-v8-mkmlizer: ║                                                                     ║
bbchicago-nana-nemo-12b-v1-0-v8-mkmlizer: ║  The license key for the current software has been verified as      ║
bbchicago-nana-nemo-12b-v1-0-v8-mkmlizer: ║  belonging to:                                                      ║
bbchicago-nana-nemo-12b-v1-0-v8-mkmlizer: ║                                                                     ║
bbchicago-nana-nemo-12b-v1-0-v8-mkmlizer: ║  Chai Research Corp.                                                ║
bbchicago-nana-nemo-12b-v1-0-v8-mkmlizer: ║  Account ID: 7997a29f-0ceb-4cc7-9adf-840c57b4ae6f                   ║
bbchicago-nana-nemo-12b-v1-0-v8-mkmlizer: ║  Expiration: 2024-10-15 23:59:59                                    ║
bbchicago-nana-nemo-12b-v1-0-v8-mkmlizer: ║                                                                     ║
bbchicago-nana-nemo-12b-v1-0-v8-mkmlizer: ╚═════════════════════════════════════════════════════════════════════╝
bbchicago-nana-nemo-12b-v1-0-v8-mkmlizer: Downloaded to shared memory in 55.616s
bbchicago-nana-nemo-12b-v1-0-v8-mkmlizer: quantizing model to /dev/shm/model_cache, profile:s0, folder:/tmp/tmpaocswg9q, device:0
bbchicago-nana-nemo-12b-v1-0-v8-mkmlizer: Saving flywheel model at /dev/shm/model_cache
bbchicago-nana-nemo-12b-v1-0-v8-mkmlizer: creating bucket guanaco-mkml-models
bbchicago-nana-nemo-12b-v1-0-v8-mkmlizer: Bucket 's3://guanaco-mkml-models/' created
bbchicago-nana-nemo-12b-v1-0-v8-mkmlizer: uploading /dev/shm/model_cache to s3://guanaco-mkml-models/bbchicago-nana-nemo-12b-v1-0-v8
bbchicago-nana-nemo-12b-v1-0-v8-mkmlizer: cp /dev/shm/model_cache/config.json s3://guanaco-mkml-models/bbchicago-nana-nemo-12b-v1-0-v8/config.json
bbchicago-nana-nemo-12b-v1-0-v8-mkmlizer: cp /dev/shm/model_cache/special_tokens_map.json s3://guanaco-mkml-models/bbchicago-nana-nemo-12b-v1-0-v8/special_tokens_map.json
bbchicago-nana-nemo-12b-v1-0-v8-mkmlizer: cp /dev/shm/model_cache/tokenizer_config.json s3://guanaco-mkml-models/bbchicago-nana-nemo-12b-v1-0-v8/tokenizer_config.json
bbchicago-nana-nemo-12b-v1-0-v8-mkmlizer: cp /dev/shm/model_cache/tokenizer.json s3://guanaco-mkml-models/bbchicago-nana-nemo-12b-v1-0-v8/tokenizer.json
bbchicago-nana-nemo-12b-v1-0-v8-mkmlizer: cp /dev/shm/model_cache/flywheel_model.0.safetensors s3://guanaco-mkml-models/bbchicago-nana-nemo-12b-v1-0-v8/flywheel_model.0.safetensors
bbchicago-nana-nemo-12b-v1-0-v8-mkmlizer: 
Loading 0:   0%|          | 0/363 [00:00<?, ?it/s]
Loading 0:   1%|▏         | 5/363 [00:00<00:12, 27.94it/s]
Loading 0:   3%|▎         | 12/363 [00:00<00:07, 45.48it/s]
Loading 0:   5%|▍         | 18/363 [00:00<00:07, 44.95it/s]
Loading 0:   6%|▋         | 23/363 [00:00<00:09, 34.57it/s]
Loading 0:   9%|▊         | 31/363 [00:00<00:07, 44.56it/s]
Loading 0:  10%|█         | 37/363 [00:00<00:07, 43.47it/s]
Loading 0:  12%|█▏        | 42/363 [00:01<00:07, 41.84it/s]
Loading 0:  13%|█▎        | 49/363 [00:01<00:06, 47.07it/s]
Loading 0:  15%|█▍        | 54/363 [00:01<00:06, 46.00it/s]
Loading 0:  16%|█▋        | 59/363 [00:01<00:06, 44.11it/s]
Loading 0:  18%|█▊        | 64/363 [00:01<00:12, 24.26it/s]
Loading 0:  20%|█▉        | 72/363 [00:01<00:08, 32.54it/s]
Loading 0:  21%|██        | 77/363 [00:02<00:08, 35.60it/s]
Loading 0:  23%|██▎       | 82/363 [00:02<00:08, 32.82it/s]
Loading 0:  25%|██▍       | 89/363 [00:02<00:06, 40.30it/s]
Loading 0:  26%|██▌       | 94/363 [00:02<00:06, 40.19it/s]
Loading 0:  27%|██▋       | 99/363 [00:02<00:06, 40.80it/s]
Loading 0:  29%|██▊       | 104/363 [00:02<00:06, 40.84it/s]
Loading 0:  30%|███       | 109/363 [00:02<00:06, 40.77it/s]
Loading 0:  31%|███▏      | 114/363 [00:03<00:07, 34.12it/s]
Loading 0:  33%|███▎      | 118/363 [00:03<00:07, 32.16it/s]
Loading 0:  34%|███▍      | 125/363 [00:03<00:06, 39.05it/s]
Loading 0:  36%|███▌      | 130/363 [00:03<00:06, 37.57it/s]
Loading 0:  37%|███▋      | 134/363 [00:03<00:06, 37.02it/s]
Loading 0:  38%|███▊      | 138/363 [00:03<00:06, 34.00it/s]
Loading 0:  39%|███▉      | 142/363 [00:04<00:09, 22.90it/s]
Loading 0:  40%|███▉      | 145/363 [00:04<00:09, 23.22it/s]
Loading 0:  41%|████      | 149/363 [00:04<00:09, 23.69it/s]
Loading 0:  43%|████▎     | 156/363 [00:04<00:06, 31.48it/s]
Loading 0:  44%|████▍     | 160/363 [00:04<00:06, 31.62it/s]
Loading 0:  45%|████▌     | 165/363 [00:04<00:05, 34.50it/s]
Loading 0:  47%|████▋     | 169/363 [00:04<00:05, 33.48it/s]
Loading 0:  48%|████▊     | 174/363 [00:04<00:05, 36.09it/s]
Loading 0:  49%|████▉     | 178/363 [00:05<00:05, 34.61it/s]
Loading 0:  50%|█████     | 183/363 [00:05<00:04, 36.73it/s]
Loading 0:  52%|█████▏    | 187/363 [00:05<00:05, 34.58it/s]
Loading 0:  53%|█████▎    | 192/363 [00:05<00:04, 36.15it/s]
Loading 0:  54%|█████▍    | 196/363 [00:05<00:04, 34.55it/s]
Loading 0:  55%|█████▌    | 201/363 [00:05<00:04, 36.66it/s]
Loading 0:  56%|█████▋    | 205/363 [00:05<00:04, 34.56it/s]
Loading 0:  58%|█████▊    | 210/363 [00:05<00:04, 36.35it/s]
Loading 0:  59%|█████▉    | 214/363 [00:06<00:04, 35.00it/s]
Loading 0:  60%|██████    | 218/363 [00:06<00:04, 34.23it/s]
Loading 0:  61%|██████    | 222/363 [00:06<00:03, 35.42it/s]
Loading 0:  62%|██████▏   | 226/363 [00:06<00:06, 22.47it/s]
Loading 0:  63%|██████▎   | 230/363 [00:06<00:05, 22.98it/s]
Loading 0:  65%|██████▌   | 237/363 [00:06<00:04, 30.43it/s]
Loading 0:  66%|██████▋   | 241/363 [00:07<00:04, 29.81it/s]
Loading 0:  68%|██████▊   | 246/363 [00:07<00:03, 32.71it/s]
Loading 0:  69%|██████▉   | 250/363 [00:07<00:03, 32.34it/s]
Loading 0:  70%|███████   | 255/363 [00:07<00:03, 34.54it/s]
Loading 0:  71%|███████▏  | 259/363 [00:07<00:03, 33.34it/s]
Loading 0:  72%|███████▏  | 263/363 [00:07<00:02, 34.80it/s]
Loading 0:  74%|███████▎  | 267/363 [00:07<00:03, 31.65it/s]
Loading 0:  75%|███████▍  | 271/363 [00:07<00:02, 33.59it/s]
Loading 0:  76%|███████▌  | 275/363 [00:08<00:02, 31.26it/s]
Loading 0:  78%|███████▊  | 282/363 [00:08<00:02, 38.20it/s]
Loading 0:  79%|███████▉  | 286/363 [00:08<00:02, 35.90it/s]
Loading 0:  80%|████████  | 291/363 [00:08<00:01, 38.23it/s]
Loading 0:  81%|████████▏ | 295/363 [00:08<00:01, 36.75it/s]
Loading 0:  82%|████████▏ | 299/363 [00:08<00:01, 35.91it/s]
Loading 0:  84%|████████▎ | 304/363 [00:15<00:28,  2.10it/s]
Loading 0:  85%|████████▍ | 307/363 [00:15<00:21,  2.64it/s]
Loading 0:  86%|████████▌ | 312/363 [00:15<00:13,  3.88it/s]
Loading 0:  88%|████████▊ | 320/363 [00:15<00:06,  6.63it/s]
Loading 0:  90%|████████▉ | 326/363 [00:16<00:04,  9.06it/s]
Loading 0:  91%|█████████ | 331/363 [00:16<00:02, 11.64it/s]
Loading 0:  93%|█████████▎| 338/363 [00:16<00:01, 16.43it/s]
Loading 0:  95%|█████████▍| 344/363 [00:16<00:00, 20.17it/s]
Loading 0:  96%|█████████▌| 349/363 [00:16<00:00, 23.53it/s]
Loading 0:  98%|█████████▊| 356/363 [00:16<00:00, 30.34it/s]
Loading 0: 100%|█████████▉| 362/363 [00:16<00:00, 33.40it/s]
                                                            
Job bbchicago-nana-nemo-12b-v1-0-v8-mkmlizer completed after 124.63s with status: succeeded
Stopping job with name bbchicago-nana-nemo-12b-v1-0-v8-mkmlizer
Pipeline stage MKMLizer completed in 125.20s
run pipeline stage %s
Running pipeline stage MKMLTemplater
Pipeline stage MKMLTemplater completed in 0.18s
run pipeline stage %s
Running pipeline stage MKMLDeployer
Creating inference service bbchicago-nana-nemo-12b-v1-0-v8
Waiting for inference service bbchicago-nana-nemo-12b-v1-0-v8 to be ready
Inference service bbchicago-nana-nemo-12b-v1-0-v8 ready after 160.80286717414856s
Pipeline stage MKMLDeployer completed in 161.43s
run pipeline stage %s
Running pipeline stage StressChecker
Received healthy response to inference request in 2.0682833194732666s
Received healthy response to inference request in 2.1175270080566406s
Received healthy response to inference request in 1.8748669624328613s
Received healthy response to inference request in 1.6864426136016846s
Received healthy response to inference request in 1.8166940212249756s
5 requests
0 failed requests
5th percentile: 1.7124928951263427
10th percentile: 1.7385431766510009
20th percentile: 1.7906437397003174
30th percentile: 1.8283286094665527
40th percentile: 1.851597785949707
50th percentile: 1.8748669624328613
60th percentile: 1.9522335052490234
70th percentile: 2.0296000480651855
80th percentile: 2.0781320571899413
90th percentile: 2.097829532623291
95th percentile: 2.107678270339966
99th percentile: 2.1155572605133055
mean time: 1.9127627849578857
Pipeline stage StressChecker completed in 11.20s
Shutdown handler de-registered
bbchicago-nana-nemo-12b-v1-0_v8 status is now deployed due to DeploymentManager action
bbchicago-nana-nemo-12b-v1-0_v8 status is now inactive due to auto deactivation removed underperforming models
bbchicago-nana-nemo-12b-v1-0_v8 status is now torndown due to DeploymentManager action