cgato-nemo-12b-thespice-_2145

developer_uid: c.gato

submission_id: cgato-nemo-12b-thespice-_2145_v2

model_name: cgato-nemo-12b-thespice-_2145_v2

model_group: cgato/Nemo-12b-TheSpice-

status: torndown

timestamp: 2024-10-30T16:37:37+00:00

num_battles: 10282

num_wins: 5926

celo_rating: 1279.19

family_friendly_score: 0.5556

family_friendly_standard_error: 0.007027213387965389

submission_type: basic

model_repo: cgato/Nemo-12b-TheSpice-V0.9-All-v2-KTO-v0.1-E1-1657

model_architecture: MistralForCausalLM

model_num_parameters: 12772121600.0

best_of: 8

max_input_tokens: 1024

max_output_tokens: 64

display_name: cgato-nemo-12b-thespice-_2145_v2

is_internal_developer: False

language_model: cgato/Nemo-12b-TheSpice-V0.9-All-v2-KTO-v0.1-E1-1657

model_size: 13B

ranking_group: single

us_pacific_date: 2024-10-30

win_ratio: 0.576347014199572

generation_params: {'temperature': 1.0, 'top_p': 1.0, 'min_p': 0.0, 'top_k': 40, 'presence_penalty': 0.0, 'frequency_penalty': 0.0, 'stopping_words': ['\n'], 'max_input_tokens': 1024, 'best_of': 8, 'max_output_tokens': 64}

formatter: {'memory_template': '<|im_start|>system\n{memory}<|im_end|>\n', 'prompt_template': '<|im_start|>user\n{prompt}<|im_end|>\n', 'bot_template': '<|im_start|>assistant\n{bot_name}: {message}<|im_end|>\n', 'user_template': '<|im_start|>user\n{user_name}: {message}<|im_end|>\n', 'response_template': '<|im_start|>assistant\n{bot_name}:', 'truncate_by_message': True}

Resubmit model

Shutdown handler not registered because Python interpreter is not running in the main thread
run pipeline %s
run pipeline stage %s
Running pipeline stage MKMLizer
Starting job with name cgato-nemo-12b-thespice-2145-v2-mkmlizer
Waiting for job on cgato-nemo-12b-thespice-2145-v2-mkmlizer to finish
cgato-nemo-12b-thespice-2145-v2-mkmlizer: ╔═════════════════════════════════════════════════════════════════════╗
cgato-nemo-12b-thespice-2145-v2-mkmlizer: ║     _____            __           __                                ║
cgato-nemo-12b-thespice-2145-v2-mkmlizer: ║    / _/ /_ ___    __/ /  ___ ___ / /                                ║
cgato-nemo-12b-thespice-2145-v2-mkmlizer: ║   / _/ / // / |/|/ / _ \/ -_) -_) /                                 ║
cgato-nemo-12b-thespice-2145-v2-mkmlizer: ║  /_//_/\_, /|__,__/_//_/\__/\__/_/                                  ║
cgato-nemo-12b-thespice-2145-v2-mkmlizer: ║       /___/                                                         ║
cgato-nemo-12b-thespice-2145-v2-mkmlizer: ║                                                                     ║
cgato-nemo-12b-thespice-2145-v2-mkmlizer: ║  Version: 0.11.12                                                   ║
cgato-nemo-12b-thespice-2145-v2-mkmlizer: ║  Copyright 2023 MK ONE TECHNOLOGIES Inc.                            ║
cgato-nemo-12b-thespice-2145-v2-mkmlizer: ║  https://mk1.ai                                                     ║
cgato-nemo-12b-thespice-2145-v2-mkmlizer: ║                                                                     ║
cgato-nemo-12b-thespice-2145-v2-mkmlizer: ║  The license key for the current software has been verified as      ║
cgato-nemo-12b-thespice-2145-v2-mkmlizer: ║  belonging to:                                                      ║
cgato-nemo-12b-thespice-2145-v2-mkmlizer: ║                                                                     ║
cgato-nemo-12b-thespice-2145-v2-mkmlizer: ║  Chai Research Corp.                                                ║
cgato-nemo-12b-thespice-2145-v2-mkmlizer: ║  Account ID: 7997a29f-0ceb-4cc7-9adf-840c57b4ae6f                   ║
cgato-nemo-12b-thespice-2145-v2-mkmlizer: ║  Expiration: 2025-01-15 23:59:59                                    ║
cgato-nemo-12b-thespice-2145-v2-mkmlizer: ║                                                                     ║
cgato-nemo-12b-thespice-2145-v2-mkmlizer: ╚═════════════════════════════════════════════════════════════════════╝
cgato-nemo-12b-thespice-2145-v2-mkmlizer: Downloaded to shared memory in 50.506s
cgato-nemo-12b-thespice-2145-v2-mkmlizer: quantizing model to /dev/shm/model_cache, profile:s0, folder:/tmp/tmp1yd0i4ok, device:0
cgato-nemo-12b-thespice-2145-v2-mkmlizer: Saving flywheel model at /dev/shm/model_cache
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
cgato-nemo-12b-thespice-2145-v2-mkmlizer: quantized model in 36.298s
cgato-nemo-12b-thespice-2145-v2-mkmlizer: Processed model cgato/Nemo-12b-TheSpice-V0.9-All-v2-KTO-v0.1-E1-1657 in 86.804s
cgato-nemo-12b-thespice-2145-v2-mkmlizer: creating bucket guanaco-mkml-models
cgato-nemo-12b-thespice-2145-v2-mkmlizer: Bucket 's3://guanaco-mkml-models/' created
cgato-nemo-12b-thespice-2145-v2-mkmlizer: uploading /dev/shm/model_cache to s3://guanaco-mkml-models/cgato-nemo-12b-thespice-2145-v2
cgato-nemo-12b-thespice-2145-v2-mkmlizer: cp /dev/shm/model_cache/special_tokens_map.json s3://guanaco-mkml-models/cgato-nemo-12b-thespice-2145-v2/special_tokens_map.json
cgato-nemo-12b-thespice-2145-v2-mkmlizer: cp /dev/shm/model_cache/config.json s3://guanaco-mkml-models/cgato-nemo-12b-thespice-2145-v2/config.json
cgato-nemo-12b-thespice-2145-v2-mkmlizer: cp /dev/shm/model_cache/tokenizer_config.json s3://guanaco-mkml-models/cgato-nemo-12b-thespice-2145-v2/tokenizer_config.json
cgato-nemo-12b-thespice-2145-v2-mkmlizer: cp /dev/shm/model_cache/tokenizer.json s3://guanaco-mkml-models/cgato-nemo-12b-thespice-2145-v2/tokenizer.json
cgato-nemo-12b-thespice-2145-v2-mkmlizer: 
Loading 0:   0%|          | 0/363 [00:00<?, ?it/s]
Loading 0:   1%|▏         | 5/363 [00:00<00:11, 30.13it/s]
Loading 0:   4%|▎         | 13/363 [00:00<00:06, 50.83it/s]
Loading 0:   5%|▌         | 19/363 [00:00<00:07, 48.14it/s]
Loading 0:   7%|▋         | 25/363 [00:00<00:07, 48.24it/s]
Loading 0:   9%|▊         | 31/363 [00:00<00:06, 50.35it/s]
Loading 0:  10%|█         | 37/363 [00:00<00:06, 47.74it/s]
Loading 0:  12%|█▏        | 42/363 [00:00<00:07, 45.81it/s]
Loading 0:  13%|█▎        | 49/363 [00:01<00:06, 50.85it/s]
Loading 0:  15%|█▌        | 55/363 [00:01<00:06, 48.39it/s]
Loading 0:  17%|█▋        | 61/363 [00:01<00:09, 32.88it/s]
Loading 0:  18%|█▊        | 65/363 [00:01<00:09, 32.32it/s]
Loading 0:  20%|█▉        | 72/363 [00:01<00:07, 39.06it/s]
Loading 0:  21%|██        | 77/363 [00:01<00:06, 41.34it/s]
Loading 0:  23%|██▎       | 82/363 [00:01<00:07, 37.23it/s]
Loading 0:  25%|██▍       | 90/363 [00:02<00:06, 44.79it/s]
Loading 0:  26%|██▌       | 95/363 [00:02<00:05, 45.98it/s]
Loading 0:  28%|██▊       | 100/363 [00:02<00:06, 37.90it/s]
Loading 0:  29%|██▉       | 106/363 [00:02<00:06, 42.21it/s]
Loading 0:  31%|███       | 112/363 [00:02<00:05, 46.31it/s]
Loading 0:  33%|███▎      | 118/363 [00:02<00:06, 36.96it/s]
Loading 0:  34%|███▍      | 125/363 [00:02<00:05, 42.08it/s]
Loading 0:  36%|███▌      | 130/363 [00:03<00:05, 40.55it/s]
Loading 0:  37%|███▋      | 135/363 [00:03<00:05, 39.69it/s]
Loading 0:  39%|███▊      | 140/363 [00:03<00:05, 40.77it/s]
Loading 0:  40%|███▉      | 145/363 [00:03<00:08, 26.49it/s]
Loading 0:  41%|████      | 149/363 [00:03<00:07, 27.58it/s]
Loading 0:  43%|████▎     | 156/363 [00:03<00:05, 35.60it/s]
Loading 0:  44%|████▍     | 161/363 [00:04<00:05, 38.01it/s]
Loading 0:  46%|████▌     | 166/363 [00:04<00:04, 40.47it/s]
Loading 0:  47%|████▋     | 172/363 [00:04<00:04, 41.33it/s]
Loading 0:  49%|████▉     | 177/363 [00:04<00:04, 41.20it/s]
Loading 0:  51%|█████     | 184/363 [00:04<00:03, 47.13it/s]
Loading 0:  52%|█████▏    | 190/363 [00:04<00:03, 45.56it/s]
Loading 0:  54%|█████▎    | 195/363 [00:04<00:03, 45.21it/s]
Loading 0:  56%|█████▌    | 202/363 [00:04<00:03, 50.45it/s]
Loading 0:  57%|█████▋    | 208/363 [00:05<00:03, 48.77it/s]
Loading 0:  59%|█████▊    | 213/363 [00:05<00:03, 44.45it/s]
Loading 0:  60%|██████    | 218/363 [00:05<00:03, 45.39it/s]
Loading 0:  61%|██████▏   | 223/363 [00:05<00:04, 33.18it/s]
Loading 0:  63%|██████▎   | 227/363 [00:05<00:03, 34.35it/s]
Loading 0:  64%|██████▎   | 231/363 [00:05<00:03, 34.21it/s]
Loading 0:  66%|██████▌   | 238/363 [00:05<00:03, 40.96it/s]
Loading 0:  67%|██████▋   | 244/363 [00:05<00:02, 41.48it/s]
Loading 0:  69%|██████▊   | 249/363 [00:06<00:02, 41.93it/s]
Loading 0:  71%|███████   | 256/363 [00:06<00:02, 47.35it/s]
Loading 0:  72%|███████▏  | 262/363 [00:06<00:02, 44.94it/s]
Loading 0:  74%|███████▎  | 267/363 [00:06<00:02, 44.41it/s]
Loading 0:  75%|███████▌  | 274/363 [00:06<00:01, 49.61it/s]
Loading 0:  77%|███████▋  | 280/363 [00:06<00:01, 47.34it/s]
Loading 0:  79%|███████▊  | 285/363 [00:06<00:01, 44.55it/s]
Loading 0:  80%|████████  | 292/363 [00:06<00:01, 48.62it/s]
Loading 0:  82%|████████▏ | 298/363 [00:07<00:01, 46.08it/s]
Loading 0:  84%|████████▎ | 304/363 [00:14<00:20,  2.81it/s]
Loading 0:  85%|████████▍ | 308/363 [00:14<00:15,  3.51it/s]
Loading 0:  86%|████████▌ | 312/363 [00:14<00:11,  4.41it/s]
Loading 0:  88%|████████▊ | 320/363 [00:14<00:06,  7.14it/s]
Loading 0:  90%|████████▉ | 326/363 [00:14<00:03,  9.56it/s]
Loading 0:  91%|█████████ | 331/363 [00:14<00:02, 11.95it/s]
Loading 0:  93%|█████████▎| 338/363 [00:14<00:01, 16.52it/s]
Loading 0:  95%|█████████▍| 344/363 [00:14<00:00, 20.00it/s]
Loading 0:  96%|█████████▌| 349/363 [00:15<00:00, 22.92it/s]
Loading 0:  98%|█████████▊| 356/363 [00:15<00:00, 29.38it/s]
Loading 0: 100%|█████████▉| 362/363 [00:15<00:00, 31.77it/s]
                                                            
Job cgato-nemo-12b-thespice-2145-v2-mkmlizer completed after 115.26s with status: succeeded
Stopping job with name cgato-nemo-12b-thespice-2145-v2-mkmlizer
Pipeline stage MKMLizer completed in 115.90s
run pipeline stage %s
Running pipeline stage MKMLTemplater
Pipeline stage MKMLTemplater completed in 0.23s
run pipeline stage %s
Running pipeline stage MKMLDeployer
Creating inference service cgato-nemo-12b-thespice-2145-v2
Waiting for inference service cgato-nemo-12b-thespice-2145-v2 to be ready
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Inference service cgato-nemo-12b-thespice-2145-v2 ready after 131.0758011341095s
Pipeline stage MKMLDeployer completed in 131.76s
run pipeline stage %s
Running pipeline stage StressChecker
Received healthy response to inference request in 2.334930896759033s
Received healthy response to inference request in 2.2318849563598633s
Received healthy response to inference request in 2.1455249786376953s
Received healthy response to inference request in 1.7648305892944336s
Received healthy response to inference request in 1.8048043251037598s
5 requests
0 failed requests
5th percentile: 1.772825336456299
10th percentile: 1.780820083618164
20th percentile: 1.7968095779418944
30th percentile: 1.872948455810547
40th percentile: 2.009236717224121
50th percentile: 2.1455249786376953
60th percentile: 2.1800689697265625
70th percentile: 2.2146129608154297
80th percentile: 2.252494144439697
90th percentile: 2.293712520599365
95th percentile: 2.3143217086791994
99th percentile: 2.3308090591430664
mean time: 2.056395149230957
Pipeline stage StressChecker completed in 12.28s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyTriggerPipeline
run_pipeline:run_in_cloud %s
starting trigger_guanaco_pipeline args=%s
triggered trigger_guanaco_pipeline args=%s
Pipeline stage OfflineFamilyFriendlyTriggerPipeline completed in 5.93s
Shutdown handler de-registered
cgato-nemo-12b-thespice-_2145_v2 status is now deployed due to DeploymentManager action
Shutdown handler registered
run pipeline %s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyScorer
Evaluating %s Family Friendly Score with %s threads
Pipeline stage OfflineFamilyFriendlyScorer completed in 2570.93s
Shutdown handler de-registered
cgato-nemo-12b-thespice-_2145_v2 status is now inactive due to auto deactivation removed underperforming models
cgato-nemo-12b-thespice-_2145_v2 status is now torndown due to DeploymentManager action
cgato-nemo-12b-thespice-_2145_v2 status is now torndown due to DeploymentManager action