bbchicago-nana-nemo-12b-_6496

submission_id: bbchicago-nana-nemo-12b-_6496_v1

developer_uid: Bbbrun0

best_of: 8

celo_rating: 827.6

display_name: bbchicago-nana-nemo-12b-v1-0_v2

family_friendly_score: 0.5763476620669116

family_friendly_standard_error: 0.00488743630066593

formatter: {'memory_template': "{bot_name}'s Persona: {memory}\n####\n", 'prompt_template': '{prompt}\n<START>\n', 'bot_template': '{bot_name}: {message}\n', 'user_template': '{user_name}: {message}\n', 'response_template': '{bot_name}:', 'truncate_by_message': False}

generation_params: {'temperature': 0.95, 'top_p': 0.95, 'min_p': 0.05, 'top_k': 80, 'presence_penalty': 0.0, 'frequency_penalty': 0.0, 'stopping_words': ['\n', '</s>', 'Bot:', 'User:', 'You:', 'Me:', '####'], 'max_input_tokens': 1024, 'best_of': 8, 'max_output_tokens': 64}

is_internal_developer: False

language_model: BBChicago/Nana-nemo-12B_v1.0-FP8-Dynamic

max_input_tokens: 1024

max_output_tokens: 64

model_architecture: MistralForCausalLM

model_group: BBChicago/Nana-nemo-12B_

model_name: bbchicago-nana-nemo-12b-v1-0_v2

model_num_parameters: 12772070400.0

model_repo: BBChicago/Nana-nemo-12B_v1.0-FP8-Dynamic

model_size: 13B

num_battles: 12315

num_wins: 1028

ranking_group: single

status: torndown

submission_type: basic

timestamp: 2024-10-21T02:26:56+00:00

us_pacific_date: 2024-10-20

win_ratio: 0.0834754364596021

Resubmit model

Shutdown handler not registered because Python interpreter is not running in the main thread
run pipeline %s
run pipeline stage %s
Running pipeline stage MKMLizer
Starting job with name bbchicago-nana-nemo-12b-6496-v1-mkmlizer
Waiting for job on bbchicago-nana-nemo-12b-6496-v1-mkmlizer to finish
bbchicago-nana-nemo-12b-6496-v1-mkmlizer: ╔═════════════════════════════════════════════════════════════════════╗
bbchicago-nana-nemo-12b-6496-v1-mkmlizer: ║     _____            __           __                                ║
bbchicago-nana-nemo-12b-6496-v1-mkmlizer: ║    / _/ /_ ___    __/ /  ___ ___ / /                                ║
bbchicago-nana-nemo-12b-6496-v1-mkmlizer: ║   / _/ / // / |/|/ / _ \/ -_) -_) /                                 ║
bbchicago-nana-nemo-12b-6496-v1-mkmlizer: ║  /_//_/\_, /|__,__/_//_/\__/\__/_/                                  ║
bbchicago-nana-nemo-12b-6496-v1-mkmlizer: ║       /___/                                                         ║
bbchicago-nana-nemo-12b-6496-v1-mkmlizer: ║                                                                     ║
bbchicago-nana-nemo-12b-6496-v1-mkmlizer: ║  Version: 0.11.12                                                   ║
bbchicago-nana-nemo-12b-6496-v1-mkmlizer: ║  Copyright 2023 MK ONE TECHNOLOGIES Inc.                            ║
bbchicago-nana-nemo-12b-6496-v1-mkmlizer: ║  https://mk1.ai                                                     ║
bbchicago-nana-nemo-12b-6496-v1-mkmlizer: ║                                                                     ║
bbchicago-nana-nemo-12b-6496-v1-mkmlizer: ║  The license key for the current software has been verified as      ║
bbchicago-nana-nemo-12b-6496-v1-mkmlizer: ║  belonging to:                                                      ║
bbchicago-nana-nemo-12b-6496-v1-mkmlizer: ║                                                                     ║
bbchicago-nana-nemo-12b-6496-v1-mkmlizer: ║  Chai Research Corp.                                                ║
bbchicago-nana-nemo-12b-6496-v1-mkmlizer: ║  Account ID: 7997a29f-0ceb-4cc7-9adf-840c57b4ae6f                   ║
bbchicago-nana-nemo-12b-6496-v1-mkmlizer: ║  Expiration: 2025-01-15 23:59:59                                    ║
bbchicago-nana-nemo-12b-6496-v1-mkmlizer: ║                                                                     ║
bbchicago-nana-nemo-12b-6496-v1-mkmlizer: ╚═════════════════════════════════════════════════════════════════════╝
bbchicago-nana-nemo-12b-6496-v1-mkmlizer: Downloaded to shared memory in 27.695s
bbchicago-nana-nemo-12b-6496-v1-mkmlizer: quantizing model to /dev/shm/model_cache, profile:s0, folder:/tmp/tmpx61xhf1b, device:0
bbchicago-nana-nemo-12b-6496-v1-mkmlizer: Saving flywheel model at /dev/shm/model_cache
bbchicago-nana-nemo-12b-6496-v1-mkmlizer: quantized model in 34.154s
bbchicago-nana-nemo-12b-6496-v1-mkmlizer: Processed model BBChicago/Nana-nemo-12B_v1.0-FP8-Dynamic in 61.850s
bbchicago-nana-nemo-12b-6496-v1-mkmlizer: creating bucket guanaco-mkml-models
bbchicago-nana-nemo-12b-6496-v1-mkmlizer: Bucket 's3://guanaco-mkml-models/' created
bbchicago-nana-nemo-12b-6496-v1-mkmlizer: uploading /dev/shm/model_cache to s3://guanaco-mkml-models/bbchicago-nana-nemo-12b-6496-v1
bbchicago-nana-nemo-12b-6496-v1-mkmlizer: cp /dev/shm/model_cache/special_tokens_map.json s3://guanaco-mkml-models/bbchicago-nana-nemo-12b-6496-v1/special_tokens_map.json
bbchicago-nana-nemo-12b-6496-v1-mkmlizer: cp /dev/shm/model_cache/config.json s3://guanaco-mkml-models/bbchicago-nana-nemo-12b-6496-v1/config.json
bbchicago-nana-nemo-12b-6496-v1-mkmlizer: cp /dev/shm/model_cache/tokenizer_config.json s3://guanaco-mkml-models/bbchicago-nana-nemo-12b-6496-v1/tokenizer_config.json
bbchicago-nana-nemo-12b-6496-v1-mkmlizer: cp /dev/shm/model_cache/tokenizer.json s3://guanaco-mkml-models/bbchicago-nana-nemo-12b-6496-v1/tokenizer.json
bbchicago-nana-nemo-12b-6496-v1-mkmlizer: cp /dev/shm/model_cache/flywheel_model.0.safetensors s3://guanaco-mkml-models/bbchicago-nana-nemo-12b-6496-v1/flywheel_model.0.safetensors
bbchicago-nana-nemo-12b-6496-v1-mkmlizer: 
Loading 0:   0%|          | 0/643 [00:00<?, ?it/s]
Loading 0:   1%|          | 7/643 [00:00<00:11, 56.28it/s]
Loading 0:   3%|▎         | 20/643 [00:00<00:06, 95.34it/s]
Loading 0:   5%|▍         | 32/643 [00:00<00:06, 89.80it/s]
Loading 0:   7%|▋         | 44/643 [00:00<00:06, 99.81it/s]
Loading 0:   9%|▊         | 55/643 [00:00<00:06, 86.17it/s]
Loading 0:  11%|█         | 71/643 [00:00<00:06, 89.91it/s]
Loading 0:  14%|█▎        | 88/643 [00:00<00:05, 101.42it/s]
Loading 0:  16%|█▌        | 100/643 [00:01<00:05, 105.46it/s]
Loading 0:  17%|█▋        | 111/643 [00:01<00:05, 93.84it/s] 
Loading 0:  20%|█▉        | 127/643 [00:01<00:05, 95.56it/s]
Loading 0:  22%|██▏       | 143/643 [00:01<00:05, 96.12it/s]
Loading 0:  25%|██▍       | 159/643 [00:01<00:04, 97.30it/s]
Loading 0:  27%|██▋       | 175/643 [00:01<00:04, 98.04it/s]
Loading 0:  30%|██▉       | 191/643 [00:01<00:04, 98.14it/s]
Loading 0:  32%|███▏      | 207/643 [00:02<00:04, 98.46it/s]
Loading 0:  34%|███▍      | 218/643 [00:02<00:05, 77.79it/s]
Loading 0:  35%|███▌      | 227/643 [00:02<00:05, 78.75it/s]
Loading 0:  37%|███▋      | 240/643 [00:02<00:04, 81.41it/s]
Loading 0:  39%|███▉      | 252/643 [00:02<00:04, 88.68it/s]
Loading 0:  41%|████      | 263/643 [00:02<00:04, 83.56it/s]
Loading 0:  43%|████▎     | 279/643 [00:03<00:04, 87.74it/s]
Loading 0:  46%|████▌     | 293/643 [00:03<00:03, 97.36it/s]
Loading 0:  47%|████▋     | 304/643 [00:03<00:03, 89.70it/s]
Loading 0:  49%|████▉     | 314/643 [00:03<00:03, 91.85it/s]
Loading 0:  51%|█████     | 327/643 [00:03<00:03, 88.50it/s]
Loading 0:  53%|█████▎    | 343/643 [00:03<00:03, 92.40it/s]
Loading 0:  56%|█████▌    | 359/643 [00:03<00:02, 95.36it/s]
Loading 0:  58%|█████▊    | 375/643 [00:04<00:02, 96.02it/s]
Loading 0:  61%|██████    | 391/643 [00:04<00:02, 96.76it/s]
Loading 0:  63%|██████▎   | 407/643 [00:04<00:02, 95.02it/s]
Loading 0:  65%|██████▌   | 421/643 [00:04<00:02, 103.41it/s]
Loading 0:  67%|██████▋   | 432/643 [00:04<00:02, 94.43it/s] 
Loading 0:  69%|██████▊   | 442/643 [00:04<00:02, 94.60it/s]
Loading 0:  70%|███████   | 453/643 [00:04<00:01, 97.22it/s]
Loading 0:  72%|███████▏  | 464/643 [00:05<00:01, 91.16it/s]
Loading 0:  74%|███████▍  | 476/643 [00:05<00:01, 97.55it/s]
Loading 0:  76%|███████▌  | 487/643 [00:05<00:01, 87.28it/s]
Loading 0:  79%|███████▊  | 506/643 [00:05<00:01, 99.01it/s]
Loading 0:  80%|████████  | 517/643 [00:12<00:20,  6.09it/s]
Loading 0:  82%|████████▏ | 529/643 [00:12<00:13,  8.26it/s]
Loading 0:  84%|████████▎ | 537/643 [00:12<00:10, 10.15it/s]
Loading 0:  86%|████████▌ | 552/643 [00:12<00:06, 14.93it/s]
Loading 0:  88%|████████▊ | 568/643 [00:12<00:03, 21.39it/s]
Loading 0:  91%|█████████ | 584/643 [00:12<00:02, 29.08it/s]
Loading 0:  93%|█████████▎| 598/643 [00:13<00:01, 37.95it/s]
Loading 0:  95%|█████████▍| 609/643 [00:13<00:00, 43.65it/s]
Loading 0:  96%|█████████▋| 619/643 [00:13<00:00, 50.63it/s]
Loading 0:  98%|█████████▊| 632/643 [00:13<00:00, 57.14it/s]
                                                            
Job bbchicago-nana-nemo-12b-6496-v1-mkmlizer completed after 84.15s with status: succeeded
Stopping job with name bbchicago-nana-nemo-12b-6496-v1-mkmlizer
Pipeline stage MKMLizer completed in 84.74s
run pipeline stage %s
Running pipeline stage MKMLTemplater
Pipeline stage MKMLTemplater completed in 0.20s
run pipeline stage %s
Running pipeline stage MKMLDeployer
Creating inference service bbchicago-nana-nemo-12b-6496-v1
Waiting for inference service bbchicago-nana-nemo-12b-6496-v1 to be ready
Inference service bbchicago-nana-nemo-12b-6496-v1 ready after 150.6494598388672s
Pipeline stage MKMLDeployer completed in 151.42s
run pipeline stage %s
Running pipeline stage StressChecker
Received healthy response to inference request in 2.1902880668640137s
Received healthy response to inference request in 1.6753582954406738s
Received healthy response to inference request in 1.6646268367767334s
Received healthy response to inference request in 1.77199125289917s
Received healthy response to inference request in 1.6992120742797852s
5 requests
0 failed requests
5th percentile: 1.6667731285095215
10th percentile: 1.6689194202423097
20th percentile: 1.6732120037078857
30th percentile: 1.680129051208496
40th percentile: 1.6896705627441406
50th percentile: 1.6992120742797852
60th percentile: 1.728323745727539
70th percentile: 1.757435417175293
80th percentile: 1.8556506156921388
90th percentile: 2.0229693412780763
95th percentile: 2.1066287040710447
99th percentile: 2.17355619430542
mean time: 1.8002953052520752
Pipeline stage StressChecker completed in 10.42s
Shutdown handler de-registered
bbchicago-nana-nemo-12b-_6496_v1 status is now deployed due to DeploymentManager action
bbchicago-nana-nemo-12b-_6496_v1 status is now inactive due to auto deactivation removed underperforming models
bbchicago-nana-nemo-12b-_6496_v1 status is now torndown due to DeploymentManager action