cgato-nemo-12b-thespice-_1916

developer_uid: c.gato

submission_id: cgato-nemo-12b-thespice-_1916_v1

model_name: cgato-nemo-12b-thespice-_1916_v1

model_group: cgato/Nemo-12b-TheSpice-

status: torndown

timestamp: 2024-10-24T03:34:35+00:00

num_battles: 7892

num_wins: 4047

celo_rating: 1251.98

family_friendly_score: 0.7747999999999999

family_friendly_standard_error: 0.005907367603256124

submission_type: basic

model_repo: cgato/Nemo-12b-TheSpice-V0.9-RP-Preview1

model_architecture: MistralForCausalLM

model_num_parameters: 12772121600.0

best_of: 8

max_input_tokens: 1024

max_output_tokens: 64

display_name: cgato-nemo-12b-thespice-_1916_v1

is_internal_developer: False

language_model: cgato/Nemo-12b-TheSpice-V0.9-RP-Preview1

model_size: 13B

ranking_group: single

us_pacific_date: 2024-10-23

win_ratio: 0.5127977698935631

generation_params: {'temperature': 1.0, 'top_p': 1.0, 'min_p': 0.0, 'top_k': 40, 'presence_penalty': 0.0, 'frequency_penalty': 0.0, 'stopping_words': ['\n'], 'max_input_tokens': 1024, 'best_of': 8, 'max_output_tokens': 64}

formatter: {'memory_template': '<|im_start|>system\n{memory}<|im_end|>\n', 'prompt_template': '<|im_start|>user\n{prompt}<|im_end|>\n', 'bot_template': '<|im_start|>assistant\n{bot_name}: {message}<|im_end|>\n', 'user_template': '<|im_start|>user\n{user_name}: {message}<|im_end|>\n', 'response_template': '<|im_start|>assistant\n{bot_name}:', 'truncate_by_message': True}

Resubmit model

Shutdown handler not registered because Python interpreter is not running in the main thread
run pipeline %s
run pipeline stage %s
Running pipeline stage MKMLizer
Starting job with name cgato-nemo-12b-thespice-1916-v1-mkmlizer
Waiting for job on cgato-nemo-12b-thespice-1916-v1-mkmlizer to finish
cgato-nemo-12b-thespice-1916-v1-mkmlizer: ╔═════════════════════════════════════════════════════════════════════╗
cgato-nemo-12b-thespice-1916-v1-mkmlizer: ║     _____            __           __                                ║
cgato-nemo-12b-thespice-1916-v1-mkmlizer: ║    / _/ /_ ___    __/ /  ___ ___ / /                                ║
cgato-nemo-12b-thespice-1916-v1-mkmlizer: ║   / _/ / // / |/|/ / _ \/ -_) -_) /                                 ║
cgato-nemo-12b-thespice-1916-v1-mkmlizer: ║  /_//_/\_, /|__,__/_//_/\__/\__/_/                                  ║
cgato-nemo-12b-thespice-1916-v1-mkmlizer: ║       /___/                                                         ║
cgato-nemo-12b-thespice-1916-v1-mkmlizer: ║                                                                     ║
cgato-nemo-12b-thespice-1916-v1-mkmlizer: ║  Version: 0.11.12                                                   ║
cgato-nemo-12b-thespice-1916-v1-mkmlizer: ║  Copyright 2023 MK ONE TECHNOLOGIES Inc.                            ║
cgato-nemo-12b-thespice-1916-v1-mkmlizer: ║  https://mk1.ai                                                     ║
cgato-nemo-12b-thespice-1916-v1-mkmlizer: ║                                                                     ║
cgato-nemo-12b-thespice-1916-v1-mkmlizer: ║  The license key for the current software has been verified as      ║
cgato-nemo-12b-thespice-1916-v1-mkmlizer: ║  belonging to:                                                      ║
cgato-nemo-12b-thespice-1916-v1-mkmlizer: ║                                                                     ║
cgato-nemo-12b-thespice-1916-v1-mkmlizer: ║  Chai Research Corp.                                                ║
cgato-nemo-12b-thespice-1916-v1-mkmlizer: ║  Account ID: 7997a29f-0ceb-4cc7-9adf-840c57b4ae6f                   ║
cgato-nemo-12b-thespice-1916-v1-mkmlizer: ║  Expiration: 2025-01-15 23:59:59                                    ║
cgato-nemo-12b-thespice-1916-v1-mkmlizer: ║                                                                     ║
cgato-nemo-12b-thespice-1916-v1-mkmlizer: ╚═════════════════════════════════════════════════════════════════════╝
cgato-nemo-12b-thespice-1916-v1-mkmlizer: Downloaded to shared memory in 53.216s
cgato-nemo-12b-thespice-1916-v1-mkmlizer: quantizing model to /dev/shm/model_cache, profile:s0, folder:/tmp/tmpfxpoe9i5, device:0
cgato-nemo-12b-thespice-1916-v1-mkmlizer: Saving flywheel model at /dev/shm/model_cache
cgato-nemo-12b-thespice-1916-v1-mkmlizer: quantized model in 35.899s
cgato-nemo-12b-thespice-1916-v1-mkmlizer: Processed model cgato/Nemo-12b-TheSpice-V0.9-RP-Preview1 in 89.115s
cgato-nemo-12b-thespice-1916-v1-mkmlizer: creating bucket guanaco-mkml-models
cgato-nemo-12b-thespice-1916-v1-mkmlizer: Bucket 's3://guanaco-mkml-models/' created
cgato-nemo-12b-thespice-1916-v1-mkmlizer: uploading /dev/shm/model_cache to s3://guanaco-mkml-models/cgato-nemo-12b-thespice-1916-v1
cgato-nemo-12b-thespice-1916-v1-mkmlizer: cp /dev/shm/model_cache/config.json s3://guanaco-mkml-models/cgato-nemo-12b-thespice-1916-v1/config.json
cgato-nemo-12b-thespice-1916-v1-mkmlizer: cp /dev/shm/model_cache/special_tokens_map.json s3://guanaco-mkml-models/cgato-nemo-12b-thespice-1916-v1/special_tokens_map.json
cgato-nemo-12b-thespice-1916-v1-mkmlizer: cp /dev/shm/model_cache/tokenizer_config.json s3://guanaco-mkml-models/cgato-nemo-12b-thespice-1916-v1/tokenizer_config.json
cgato-nemo-12b-thespice-1916-v1-mkmlizer: cp /dev/shm/model_cache/tokenizer.json s3://guanaco-mkml-models/cgato-nemo-12b-thespice-1916-v1/tokenizer.json
cgato-nemo-12b-thespice-1916-v1-mkmlizer: cp /dev/shm/model_cache/flywheel_model.0.safetensors s3://guanaco-mkml-models/cgato-nemo-12b-thespice-1916-v1/flywheel_model.0.safetensors
cgato-nemo-12b-thespice-1916-v1-mkmlizer: 
Loading 0:   0%|          | 0/363 [00:00<?, ?it/s]
Loading 0:   1%|▏         | 5/363 [00:00<00:11, 31.50it/s]
Loading 0:   4%|▎         | 13/363 [00:00<00:07, 49.89it/s]
Loading 0:   5%|▌         | 19/363 [00:00<00:07, 43.35it/s]
Loading 0:   7%|▋         | 24/363 [00:00<00:08, 41.25it/s]
Loading 0:   9%|▊         | 31/363 [00:00<00:07, 47.09it/s]
Loading 0:  10%|█         | 37/363 [00:00<00:07, 44.55it/s]
Loading 0:  12%|█▏        | 42/363 [00:00<00:07, 43.84it/s]
Loading 0:  13%|█▎        | 49/363 [00:01<00:06, 49.27it/s]
Loading 0:  15%|█▌        | 55/363 [00:01<00:06, 46.36it/s]
Loading 0:  17%|█▋        | 61/363 [00:01<00:08, 34.46it/s]
Loading 0:  18%|█▊        | 65/363 [00:01<00:08, 34.10it/s]
Loading 0:  20%|█▉        | 72/363 [00:01<00:07, 40.51it/s]
Loading 0:  21%|██        | 77/363 [00:01<00:06, 42.15it/s]
Loading 0:  23%|██▎       | 82/363 [00:02<00:07, 37.26it/s]
Loading 0:  25%|██▍       | 89/363 [00:02<00:06, 44.11it/s]
Loading 0:  26%|██▌       | 94/363 [00:02<00:06, 44.37it/s]
Loading 0:  27%|██▋       | 99/363 [00:02<00:05, 45.07it/s]
Loading 0:  29%|██▉       | 105/363 [00:02<00:05, 43.20it/s]
Loading 0:  31%|███       | 112/363 [00:02<00:05, 46.86it/s]
Loading 0:  32%|███▏      | 117/363 [00:02<00:05, 44.42it/s]
Loading 0:  34%|███▍      | 123/363 [00:02<00:05, 42.45it/s]
Loading 0:  35%|███▌      | 128/363 [00:03<00:05, 41.79it/s]
Loading 0:  37%|███▋      | 134/363 [00:03<00:04, 46.11it/s]
Loading 0:  38%|███▊      | 139/363 [00:03<00:04, 45.38it/s]
Loading 0:  40%|███▉      | 144/363 [00:03<00:07, 28.26it/s]
Loading 0:  41%|████      | 149/363 [00:03<00:06, 30.87it/s]
Loading 0:  43%|████▎     | 156/363 [00:03<00:05, 38.33it/s]
Loading 0:  44%|████▍     | 161/363 [00:03<00:05, 39.54it/s]
Loading 0:  46%|████▌     | 166/363 [00:04<00:04, 40.98it/s]
Loading 0:  47%|████▋     | 172/363 [00:04<00:04, 40.11it/s]
Loading 0:  49%|████▉     | 177/363 [00:04<00:04, 39.97it/s]
Loading 0:  51%|█████     | 184/363 [00:04<00:03, 45.32it/s]
Loading 0:  52%|█████▏    | 189/363 [00:04<00:03, 46.46it/s]
Loading 0:  53%|█████▎    | 194/363 [00:04<00:04, 38.22it/s]
Loading 0:  56%|█████▌    | 202/363 [00:04<00:03, 46.04it/s]
Loading 0:  57%|█████▋    | 208/363 [00:05<00:03, 43.81it/s]
Loading 0:  59%|█████▊    | 213/363 [00:05<00:03, 43.22it/s]
Loading 0:  60%|██████    | 218/363 [00:05<00:03, 44.66it/s]
Loading 0:  61%|██████▏   | 223/363 [00:05<00:04, 34.69it/s]
Loading 0:  63%|██████▎   | 227/363 [00:05<00:03, 35.48it/s]
Loading 0:  64%|██████▎   | 231/363 [00:05<00:03, 33.89it/s]
Loading 0:  65%|██████▌   | 237/363 [00:05<00:03, 39.59it/s]
Loading 0:  67%|██████▋   | 242/363 [00:05<00:02, 40.87it/s]
Loading 0:  68%|██████▊   | 247/363 [00:06<00:02, 41.73it/s]
Loading 0:  69%|██████▉   | 252/363 [00:06<00:02, 43.62it/s]
Loading 0:  71%|███████   | 257/363 [00:06<00:02, 36.57it/s]
Loading 0:  73%|███████▎  | 264/363 [00:06<00:02, 43.31it/s]
Loading 0:  74%|███████▍  | 269/363 [00:06<00:02, 43.12it/s]
Loading 0:  75%|███████▌  | 274/363 [00:06<00:02, 44.00it/s]
Loading 0:  77%|███████▋  | 280/363 [00:06<00:01, 42.14it/s]
Loading 0:  79%|███████▊  | 285/363 [00:06<00:01, 42.17it/s]
Loading 0:  80%|████████  | 292/363 [00:07<00:01, 47.01it/s]
Loading 0:  82%|████████▏ | 298/363 [00:07<00:01, 44.15it/s]
Loading 0:  84%|████████▎ | 304/363 [00:13<00:21,  2.79it/s]
Loading 0:  85%|████████▍ | 308/363 [00:14<00:15,  3.52it/s]
Loading 0:  86%|████████▌ | 312/363 [00:14<00:11,  4.43it/s]
Loading 0:  88%|████████▊ | 320/363 [00:14<00:06,  7.16it/s]
Loading 0:  90%|████████▉ | 326/363 [00:14<00:03,  9.54it/s]
Loading 0:  91%|█████████ | 331/363 [00:14<00:02, 12.01it/s]
Loading 0:  93%|█████████▎| 338/363 [00:14<00:01, 16.62it/s]
Loading 0:  94%|█████████▍| 343/363 [00:14<00:00, 20.14it/s]
Loading 0:  96%|█████████▌| 348/363 [00:15<00:00, 21.53it/s]
Loading 0:  98%|█████████▊| 355/363 [00:15<00:00, 28.37it/s]
Loading 0:  99%|█████████▉| 360/363 [00:15<00:00, 31.23it/s]
                                                            
Job cgato-nemo-12b-thespice-1916-v1-mkmlizer completed after 114.54s with status: succeeded
Stopping job with name cgato-nemo-12b-thespice-1916-v1-mkmlizer
Pipeline stage MKMLizer completed in 115.10s
run pipeline stage %s
Running pipeline stage MKMLTemplater
Pipeline stage MKMLTemplater completed in 0.16s
run pipeline stage %s
Running pipeline stage MKMLDeployer
Creating inference service cgato-nemo-12b-thespice-1916-v1
Waiting for inference service cgato-nemo-12b-thespice-1916-v1 to be ready
Inference service cgato-nemo-12b-thespice-1916-v1 ready after 130.7490737438202s
Pipeline stage MKMLDeployer completed in 131.32s
run pipeline stage %s
Running pipeline stage StressChecker
Received healthy response to inference request in 2.1066198348999023s
Retrying (%r) after connection broken by '%r': %s
Received healthy response to inference request in 1.8068311214447021s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Received healthy response to inference request in 1.3221890926361084s
Received healthy response to inference request in 1.5876517295837402s
Received healthy response to inference request in 1.6211299896240234s
5 requests
0 failed requests
5th percentile: 1.3752816200256348
10th percentile: 1.4283741474151612
20th percentile: 1.5345592021942138
30th percentile: 1.5943473815917968
40th percentile: 1.6077386856079101
50th percentile: 1.6211299896240234
60th percentile: 1.6954104423522949
70th percentile: 1.7696908950805663
80th percentile: 1.8667888641357422
90th percentile: 1.9867043495178223
95th percentile: 2.0466620922088623
99th percentile: 2.0946282863616945
mean time: 1.6888843536376954
Pipeline stage StressChecker completed in 9.93s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyTriggerPipeline
run_pipeline:run_in_cloud %s
starting trigger_guanaco_pipeline args=%s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
triggered trigger_guanaco_pipeline args=%s
Pipeline stage OfflineFamilyFriendlyTriggerPipeline completed in 5.31s
Shutdown handler de-registered
cgato-nemo-12b-thespice-_1916_v1 status is now deployed due to DeploymentManager action
Shutdown handler registered
run pipeline %s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyScorer
Evaluating %s Family Friendly Score with %s threads
Pipeline stage OfflineFamilyFriendlyScorer completed in 1954.10s
Shutdown handler de-registered
cgato-nemo-12b-thespice-_1916_v1 status is now inactive due to auto deactivation removed underperforming models
cgato-nemo-12b-thespice-_1916_v1 status is now torndown due to DeploymentManager action
admin requested tearing down of cgato-nemo-12b-thespice-_2145_v1