submission_id: cgato-nemo-12b-thespice-_2145_v2
developer_uid: c.gato
best_of: 8
celo_rating: 1279.19
display_name: cgato-nemo-12b-thespice-_2145_v2
family_friendly_score: 0.5556
family_friendly_standard_error: 0.007027213387965389
formatter: {'memory_template': '<|im_start|>system\n{memory}<|im_end|>\n', 'prompt_template': '<|im_start|>user\n{prompt}<|im_end|>\n', 'bot_template': '<|im_start|>assistant\n{bot_name}: {message}<|im_end|>\n', 'user_template': '<|im_start|>user\n{user_name}: {message}<|im_end|>\n', 'response_template': '<|im_start|>assistant\n{bot_name}:', 'truncate_by_message': True}
generation_params: {'temperature': 1.0, 'top_p': 1.0, 'min_p': 0.0, 'top_k': 40, 'presence_penalty': 0.0, 'frequency_penalty': 0.0, 'stopping_words': ['\n'], 'max_input_tokens': 1024, 'best_of': 8, 'max_output_tokens': 64}
is_internal_developer: False
language_model: cgato/Nemo-12b-TheSpice-V0.9-All-v2-KTO-v0.1-E1-1657
max_input_tokens: 1024
max_output_tokens: 64
model_architecture: MistralForCausalLM
model_group: cgato/Nemo-12b-TheSpice-
model_name: cgato-nemo-12b-thespice-_2145_v2
model_num_parameters: 12772121600.0
model_repo: cgato/Nemo-12b-TheSpice-V0.9-All-v2-KTO-v0.1-E1-1657
model_size: 13B
num_battles: 10282
num_wins: 5926
ranking_group: single
status: inactive
submission_type: basic
timestamp: 2024-10-30T16:37:37+00:00
us_pacific_date: 2024-10-30
win_ratio: 0.576347014199572
Resubmit model
Shutdown handler not registered because Python interpreter is not running in the main thread
run pipeline %s
run pipeline stage %s
Running pipeline stage MKMLizer
Starting job with name cgato-nemo-12b-thespice-2145-v2-mkmlizer
Waiting for job on cgato-nemo-12b-thespice-2145-v2-mkmlizer to finish
cgato-nemo-12b-thespice-2145-v2-mkmlizer: ╔═════════════════════════════════════════════════════════════════════╗
cgato-nemo-12b-thespice-2145-v2-mkmlizer: ║ _____ __ __ ║
cgato-nemo-12b-thespice-2145-v2-mkmlizer: ║ / _/ /_ ___ __/ / ___ ___ / / ║
cgato-nemo-12b-thespice-2145-v2-mkmlizer: ║ / _/ / // / |/|/ / _ \/ -_) -_) / ║
cgato-nemo-12b-thespice-2145-v2-mkmlizer: ║ /_//_/\_, /|__,__/_//_/\__/\__/_/ ║
cgato-nemo-12b-thespice-2145-v2-mkmlizer: ║ /___/ ║
cgato-nemo-12b-thespice-2145-v2-mkmlizer: ║ ║
cgato-nemo-12b-thespice-2145-v2-mkmlizer: ║ Version: 0.11.12 ║
cgato-nemo-12b-thespice-2145-v2-mkmlizer: ║ Copyright 2023 MK ONE TECHNOLOGIES Inc. ║
cgato-nemo-12b-thespice-2145-v2-mkmlizer: ║ https://mk1.ai ║
cgato-nemo-12b-thespice-2145-v2-mkmlizer: ║ ║
cgato-nemo-12b-thespice-2145-v2-mkmlizer: ║ The license key for the current software has been verified as ║
cgato-nemo-12b-thespice-2145-v2-mkmlizer: ║ belonging to: ║
cgato-nemo-12b-thespice-2145-v2-mkmlizer: ║ ║
cgato-nemo-12b-thespice-2145-v2-mkmlizer: ║ Chai Research Corp. ║
cgato-nemo-12b-thespice-2145-v2-mkmlizer: ║ Account ID: 7997a29f-0ceb-4cc7-9adf-840c57b4ae6f ║
cgato-nemo-12b-thespice-2145-v2-mkmlizer: ║ Expiration: 2025-01-15 23:59:59 ║
cgato-nemo-12b-thespice-2145-v2-mkmlizer: ║ ║
cgato-nemo-12b-thespice-2145-v2-mkmlizer: ╚═════════════════════════════════════════════════════════════════════╝
cgato-nemo-12b-thespice-2145-v2-mkmlizer: Downloaded to shared memory in 50.506s
cgato-nemo-12b-thespice-2145-v2-mkmlizer: quantizing model to /dev/shm/model_cache, profile:s0, folder:/tmp/tmp1yd0i4ok, device:0
cgato-nemo-12b-thespice-2145-v2-mkmlizer: Saving flywheel model at /dev/shm/model_cache
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
cgato-nemo-12b-thespice-2145-v2-mkmlizer: quantized model in 36.298s
cgato-nemo-12b-thespice-2145-v2-mkmlizer: Processed model cgato/Nemo-12b-TheSpice-V0.9-All-v2-KTO-v0.1-E1-1657 in 86.804s
cgato-nemo-12b-thespice-2145-v2-mkmlizer: creating bucket guanaco-mkml-models
cgato-nemo-12b-thespice-2145-v2-mkmlizer: Bucket 's3://guanaco-mkml-models/' created
cgato-nemo-12b-thespice-2145-v2-mkmlizer: uploading /dev/shm/model_cache to s3://guanaco-mkml-models/cgato-nemo-12b-thespice-2145-v2
cgato-nemo-12b-thespice-2145-v2-mkmlizer: cp /dev/shm/model_cache/special_tokens_map.json s3://guanaco-mkml-models/cgato-nemo-12b-thespice-2145-v2/special_tokens_map.json
cgato-nemo-12b-thespice-2145-v2-mkmlizer: cp /dev/shm/model_cache/config.json s3://guanaco-mkml-models/cgato-nemo-12b-thespice-2145-v2/config.json
cgato-nemo-12b-thespice-2145-v2-mkmlizer: cp /dev/shm/model_cache/tokenizer_config.json s3://guanaco-mkml-models/cgato-nemo-12b-thespice-2145-v2/tokenizer_config.json
cgato-nemo-12b-thespice-2145-v2-mkmlizer: cp /dev/shm/model_cache/tokenizer.json s3://guanaco-mkml-models/cgato-nemo-12b-thespice-2145-v2/tokenizer.json
cgato-nemo-12b-thespice-2145-v2-mkmlizer: Loading 0: 0%| | 0/363 [00:00<?, ?it/s] Loading 0: 1%|▏ | 5/363 [00:00<00:11, 30.13it/s] Loading 0: 4%|▎ | 13/363 [00:00<00:06, 50.83it/s] Loading 0: 5%|▌ | 19/363 [00:00<00:07, 48.14it/s] Loading 0: 7%|▋ | 25/363 [00:00<00:07, 48.24it/s] Loading 0: 9%|▊ | 31/363 [00:00<00:06, 50.35it/s] Loading 0: 10%|█ | 37/363 [00:00<00:06, 47.74it/s] Loading 0: 12%|█▏ | 42/363 [00:00<00:07, 45.81it/s] Loading 0: 13%|█▎ | 49/363 [00:01<00:06, 50.85it/s] Loading 0: 15%|█▌ | 55/363 [00:01<00:06, 48.39it/s] Loading 0: 17%|█▋ | 61/363 [00:01<00:09, 32.88it/s] Loading 0: 18%|█▊ | 65/363 [00:01<00:09, 32.32it/s] Loading 0: 20%|█▉ | 72/363 [00:01<00:07, 39.06it/s] Loading 0: 21%|██ | 77/363 [00:01<00:06, 41.34it/s] Loading 0: 23%|██▎ | 82/363 [00:01<00:07, 37.23it/s] Loading 0: 25%|██▍ | 90/363 [00:02<00:06, 44.79it/s] Loading 0: 26%|██▌ | 95/363 [00:02<00:05, 45.98it/s] Loading 0: 28%|██▊ | 100/363 [00:02<00:06, 37.90it/s] Loading 0: 29%|██▉ | 106/363 [00:02<00:06, 42.21it/s] Loading 0: 31%|███ | 112/363 [00:02<00:05, 46.31it/s] Loading 0: 33%|███▎ | 118/363 [00:02<00:06, 36.96it/s] Loading 0: 34%|███▍ | 125/363 [00:02<00:05, 42.08it/s] Loading 0: 36%|███▌ | 130/363 [00:03<00:05, 40.55it/s] Loading 0: 37%|███▋ | 135/363 [00:03<00:05, 39.69it/s] Loading 0: 39%|███▊ | 140/363 [00:03<00:05, 40.77it/s] Loading 0: 40%|███▉ | 145/363 [00:03<00:08, 26.49it/s] Loading 0: 41%|████ | 149/363 [00:03<00:07, 27.58it/s] Loading 0: 43%|████▎ | 156/363 [00:03<00:05, 35.60it/s] Loading 0: 44%|████▍ | 161/363 [00:04<00:05, 38.01it/s] Loading 0: 46%|████▌ | 166/363 [00:04<00:04, 40.47it/s] Loading 0: 47%|████▋ | 172/363 [00:04<00:04, 41.33it/s] Loading 0: 49%|████▉ | 177/363 [00:04<00:04, 41.20it/s] Loading 0: 51%|█████ | 184/363 [00:04<00:03, 47.13it/s] Loading 0: 52%|█████▏ | 190/363 [00:04<00:03, 45.56it/s] Loading 0: 54%|█████▎ | 195/363 [00:04<00:03, 45.21it/s] Loading 0: 56%|█████▌ | 202/363 [00:04<00:03, 50.45it/s] Loading 0: 57%|█████▋ | 208/363 [00:05<00:03, 48.77it/s] Loading 0: 59%|█████▊ | 213/363 [00:05<00:03, 44.45it/s] Loading 0: 60%|██████ | 218/363 [00:05<00:03, 45.39it/s] Loading 0: 61%|██████▏ | 223/363 [00:05<00:04, 33.18it/s] Loading 0: 63%|██████▎ | 227/363 [00:05<00:03, 34.35it/s] Loading 0: 64%|██████▎ | 231/363 [00:05<00:03, 34.21it/s] Loading 0: 66%|██████▌ | 238/363 [00:05<00:03, 40.96it/s] Loading 0: 67%|██████▋ | 244/363 [00:05<00:02, 41.48it/s] Loading 0: 69%|██████▊ | 249/363 [00:06<00:02, 41.93it/s] Loading 0: 71%|███████ | 256/363 [00:06<00:02, 47.35it/s] Loading 0: 72%|███████▏ | 262/363 [00:06<00:02, 44.94it/s] Loading 0: 74%|███████▎ | 267/363 [00:06<00:02, 44.41it/s] Loading 0: 75%|███████▌ | 274/363 [00:06<00:01, 49.61it/s] Loading 0: 77%|███████▋ | 280/363 [00:06<00:01, 47.34it/s] Loading 0: 79%|███████▊ | 285/363 [00:06<00:01, 44.55it/s] Loading 0: 80%|████████ | 292/363 [00:06<00:01, 48.62it/s] Loading 0: 82%|████████▏ | 298/363 [00:07<00:01, 46.08it/s] Loading 0: 84%|████████▎ | 304/363 [00:14<00:20, 2.81it/s] Loading 0: 85%|████████▍ | 308/363 [00:14<00:15, 3.51it/s] Loading 0: 86%|████████▌ | 312/363 [00:14<00:11, 4.41it/s] Loading 0: 88%|████████▊ | 320/363 [00:14<00:06, 7.14it/s] Loading 0: 90%|████████▉ | 326/363 [00:14<00:03, 9.56it/s] Loading 0: 91%|█████████ | 331/363 [00:14<00:02, 11.95it/s] Loading 0: 93%|█████████▎| 338/363 [00:14<00:01, 16.52it/s] Loading 0: 95%|█████████▍| 344/363 [00:14<00:00, 20.00it/s] Loading 0: 96%|█████████▌| 349/363 [00:15<00:00, 22.92it/s] Loading 0: 98%|█████████▊| 356/363 [00:15<00:00, 29.38it/s] Loading 0: 100%|█████████▉| 362/363 [00:15<00:00, 31.77it/s]
Job cgato-nemo-12b-thespice-2145-v2-mkmlizer completed after 115.26s with status: succeeded
Stopping job with name cgato-nemo-12b-thespice-2145-v2-mkmlizer
Pipeline stage MKMLizer completed in 115.90s
run pipeline stage %s
Running pipeline stage MKMLTemplater
Pipeline stage MKMLTemplater completed in 0.23s
run pipeline stage %s
Running pipeline stage MKMLDeployer
Creating inference service cgato-nemo-12b-thespice-2145-v2
Waiting for inference service cgato-nemo-12b-thespice-2145-v2 to be ready
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Inference service cgato-nemo-12b-thespice-2145-v2 ready after 131.0758011341095s
Pipeline stage MKMLDeployer completed in 131.76s
run pipeline stage %s
Running pipeline stage StressChecker
Received healthy response to inference request in 2.334930896759033s
Received healthy response to inference request in 2.2318849563598633s
Received healthy response to inference request in 2.1455249786376953s
Received healthy response to inference request in 1.7648305892944336s
Received healthy response to inference request in 1.8048043251037598s
5 requests
0 failed requests
5th percentile: 1.772825336456299
10th percentile: 1.780820083618164
20th percentile: 1.7968095779418944
30th percentile: 1.872948455810547
40th percentile: 2.009236717224121
50th percentile: 2.1455249786376953
60th percentile: 2.1800689697265625
70th percentile: 2.2146129608154297
80th percentile: 2.252494144439697
90th percentile: 2.293712520599365
95th percentile: 2.3143217086791994
99th percentile: 2.3308090591430664
mean time: 2.056395149230957
Pipeline stage StressChecker completed in 12.28s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyTriggerPipeline
run_pipeline:run_in_cloud %s
starting trigger_guanaco_pipeline args=%s
triggered trigger_guanaco_pipeline args=%s
Pipeline stage OfflineFamilyFriendlyTriggerPipeline completed in 5.93s
Shutdown handler de-registered
cgato-nemo-12b-thespice-_2145_v2 status is now deployed due to DeploymentManager action
Shutdown handler registered
run pipeline %s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyScorer
Evaluating %s Family Friendly Score with %s threads
Pipeline stage OfflineFamilyFriendlyScorer completed in 2570.93s
Shutdown handler de-registered
cgato-nemo-12b-thespice-_2145_v2 status is now inactive due to auto deactivation removed underperforming models