cgato-l3-1-8b-thespice-v_8134

submission_id: cgato-l3-1-8b-thespice-v_8134_v1

developer_uid: c.gato

best_of: 8

celo_rating: 1229.69

display_name: cgato-l3-1-8b-thespice-v_8134_v1

family_friendly_score: 0.0

formatter: {'memory_template': '<|im_start|>system\n{memory}<|im_end|>\n', 'prompt_template': '<|im_start|>user\n{prompt}<|im_end|>\n', 'bot_template': '<|im_start|>assistant\n{bot_name}: {message}<|im_end|>\n', 'user_template': '<|im_start|>user\n{user_name}: {message}<|im_end|>\n', 'response_template': '<|im_start|>assistant\n{bot_name}:', 'truncate_by_message': True}

generation_params: {'temperature': 1.0, 'top_p': 1.0, 'min_p': 0.0, 'top_k': 40, 'presence_penalty': 0.0, 'frequency_penalty': 0.0, 'stopping_words': ['\n'], 'max_input_tokens': 1024, 'best_of': 8, 'max_output_tokens': 64}

is_internal_developer: False

language_model: cgato/L3.1-8b-TheSpice-V0.9-RP-Preview1

max_input_tokens: 1024

max_output_tokens: 64

model_architecture: LlamaForCausalLM

model_group: cgato/L3.1-8b-TheSpice-V

model_name: cgato-l3-1-8b-thespice-v_8134_v1

model_num_parameters: 8030294016.0

model_repo: cgato/L3.1-8b-TheSpice-V0.9-RP-Preview1

model_size: 8B

num_battles: 54136

num_wins: 25326

ranking_group: single

status: torndown

submission_type: basic

timestamp: 2024-10-22T20:37:11+00:00

us_pacific_date: 2024-10-22

win_ratio: 0.46782178217821785

Resubmit model

Shutdown handler not registered because Python interpreter is not running in the main thread
run pipeline %s
run pipeline stage %s
Running pipeline stage MKMLizer
Starting job with name cgato-l3-1-8b-thespice-v-8134-v1-mkmlizer
Waiting for job on cgato-l3-1-8b-thespice-v-8134-v1-mkmlizer to finish
cgato-l3-1-8b-thespice-v-8134-v1-mkmlizer: ╔═════════════════════════════════════════════════════════════════════╗
cgato-l3-1-8b-thespice-v-8134-v1-mkmlizer: ║     _____            __           __                                ║
cgato-l3-1-8b-thespice-v-8134-v1-mkmlizer: ║    / _/ /_ ___    __/ /  ___ ___ / /                                ║
cgato-l3-1-8b-thespice-v-8134-v1-mkmlizer: ║   / _/ / // / |/|/ / _ \/ -_) -_) /                                 ║
cgato-l3-1-8b-thespice-v-8134-v1-mkmlizer: ║  /_//_/\_, /|__,__/_//_/\__/\__/_/                                  ║
cgato-l3-1-8b-thespice-v-8134-v1-mkmlizer: ║       /___/                                                         ║
cgato-l3-1-8b-thespice-v-8134-v1-mkmlizer: ║                                                                     ║
cgato-l3-1-8b-thespice-v-8134-v1-mkmlizer: ║  Version: 0.11.12                                                   ║
cgato-l3-1-8b-thespice-v-8134-v1-mkmlizer: ║  Copyright 2023 MK ONE TECHNOLOGIES Inc.                            ║
cgato-l3-1-8b-thespice-v-8134-v1-mkmlizer: ║  https://mk1.ai                                                     ║
cgato-l3-1-8b-thespice-v-8134-v1-mkmlizer: ║                                                                     ║
cgato-l3-1-8b-thespice-v-8134-v1-mkmlizer: ║  The license key for the current software has been verified as      ║
cgato-l3-1-8b-thespice-v-8134-v1-mkmlizer: ║  belonging to:                                                      ║
cgato-l3-1-8b-thespice-v-8134-v1-mkmlizer: ║                                                                     ║
cgato-l3-1-8b-thespice-v-8134-v1-mkmlizer: ║  Chai Research Corp.                                                ║
cgato-l3-1-8b-thespice-v-8134-v1-mkmlizer: ║  Account ID: 7997a29f-0ceb-4cc7-9adf-840c57b4ae6f                   ║
cgato-l3-1-8b-thespice-v-8134-v1-mkmlizer: ║  Expiration: 2025-01-15 23:59:59                                    ║
cgato-l3-1-8b-thespice-v-8134-v1-mkmlizer: ║                                                                     ║
cgato-l3-1-8b-thespice-v-8134-v1-mkmlizer: ╚═════════════════════════════════════════════════════════════════════╝
cgato-l3-1-8b-thespice-v-8134-v1-mkmlizer: Downloaded to shared memory in 37.141s
cgato-l3-1-8b-thespice-v-8134-v1-mkmlizer: quantizing model to /dev/shm/model_cache, profile:s0, folder:/tmp/tmpwblru8wq, device:0
cgato-l3-1-8b-thespice-v-8134-v1-mkmlizer: Saving flywheel model at /dev/shm/model_cache
cgato-l3-1-8b-thespice-v-8134-v1-mkmlizer: quantized model in 26.168s
cgato-l3-1-8b-thespice-v-8134-v1-mkmlizer: Processed model cgato/L3.1-8b-TheSpice-V0.9-RP-Preview1 in 63.309s
cgato-l3-1-8b-thespice-v-8134-v1-mkmlizer: creating bucket guanaco-mkml-models
cgato-l3-1-8b-thespice-v-8134-v1-mkmlizer: Bucket 's3://guanaco-mkml-models/' created
cgato-l3-1-8b-thespice-v-8134-v1-mkmlizer: uploading /dev/shm/model_cache to s3://guanaco-mkml-models/cgato-l3-1-8b-thespice-v-8134-v1
cgato-l3-1-8b-thespice-v-8134-v1-mkmlizer: cp /dev/shm/model_cache/config.json s3://guanaco-mkml-models/cgato-l3-1-8b-thespice-v-8134-v1/config.json
cgato-l3-1-8b-thespice-v-8134-v1-mkmlizer: cp /dev/shm/model_cache/special_tokens_map.json s3://guanaco-mkml-models/cgato-l3-1-8b-thespice-v-8134-v1/special_tokens_map.json
cgato-l3-1-8b-thespice-v-8134-v1-mkmlizer: cp /dev/shm/model_cache/tokenizer_config.json s3://guanaco-mkml-models/cgato-l3-1-8b-thespice-v-8134-v1/tokenizer_config.json
cgato-l3-1-8b-thespice-v-8134-v1-mkmlizer: cp /dev/shm/model_cache/tokenizer.json s3://guanaco-mkml-models/cgato-l3-1-8b-thespice-v-8134-v1/tokenizer.json
cgato-l3-1-8b-thespice-v-8134-v1-mkmlizer: cp /dev/shm/model_cache/flywheel_model.0.safetensors s3://guanaco-mkml-models/cgato-l3-1-8b-thespice-v-8134-v1/flywheel_model.0.safetensors
cgato-l3-1-8b-thespice-v-8134-v1-mkmlizer: 
Loading 0:   0%|          | 0/291 [00:00<?, ?it/s]
Loading 0:   2%|▏         | 5/291 [00:00<00:08, 32.43it/s]
Loading 0:   4%|▍         | 13/291 [00:00<00:05, 55.17it/s]
Loading 0:   7%|▋         | 20/291 [00:00<00:04, 54.65it/s]
Loading 0:   9%|▉         | 26/291 [00:00<00:04, 54.53it/s]
Loading 0:  11%|█         | 32/291 [00:00<00:05, 47.66it/s]
Loading 0:  14%|█▍        | 41/291 [00:00<00:04, 50.54it/s]
Loading 0:  17%|█▋        | 50/291 [00:00<00:04, 52.55it/s]
Loading 0:  20%|██        | 59/291 [00:01<00:04, 53.75it/s]
Loading 0:  23%|██▎       | 67/291 [00:01<00:03, 59.67it/s]
Loading 0:  25%|██▌       | 74/291 [00:01<00:03, 57.66it/s]
Loading 0:  27%|██▋       | 80/291 [00:01<00:03, 55.97it/s]
Loading 0:  30%|██▉       | 86/291 [00:01<00:06, 33.43it/s]
Loading 0:  32%|███▏      | 94/291 [00:01<00:04, 41.26it/s]
Loading 0:  34%|███▍      | 100/291 [00:02<00:04, 41.30it/s]
Loading 0:  36%|███▋      | 106/291 [00:02<00:04, 41.94it/s]
Loading 0:  38%|███▊      | 111/291 [00:02<00:04, 42.63it/s]
Loading 0:  40%|███▉      | 116/291 [00:02<00:03, 44.08it/s]
Loading 0:  42%|████▏     | 122/291 [00:02<00:03, 42.38it/s]
Loading 0:  45%|████▍     | 130/291 [00:02<00:03, 49.88it/s]
Loading 0:  47%|████▋     | 136/291 [00:02<00:03, 44.60it/s]
Loading 0:  48%|████▊     | 141/291 [00:03<00:03, 43.82it/s]
Loading 0:  51%|█████     | 148/291 [00:03<00:02, 49.61it/s]
Loading 0:  53%|█████▎    | 154/291 [00:03<00:02, 46.90it/s]
Loading 0:  55%|█████▍    | 159/291 [00:03<00:02, 45.55it/s]
Loading 0:  57%|█████▋    | 165/291 [00:03<00:02, 49.08it/s]
Loading 0:  59%|█████▉    | 171/291 [00:03<00:02, 50.05it/s]
Loading 0:  61%|██████    | 177/291 [00:03<00:02, 46.79it/s]
Loading 0:  63%|██████▎   | 182/291 [00:03<00:02, 44.56it/s]
Loading 0:  64%|██████▍   | 187/291 [00:04<00:03, 34.59it/s]
Loading 0:  66%|██████▌   | 191/291 [00:04<00:02, 35.19it/s]
Loading 0:  67%|██████▋   | 195/291 [00:04<00:02, 32.93it/s]
Loading 0:  68%|██████▊   | 199/291 [00:04<00:02, 34.35it/s]
Loading 0:  70%|██████▉   | 203/291 [00:04<00:02, 34.18it/s]
Loading 0:  73%|███████▎  | 211/291 [00:04<00:01, 45.34it/s]
Loading 0:  75%|███████▍  | 217/291 [00:04<00:01, 45.12it/s]
Loading 0:  77%|███████▋  | 223/291 [00:04<00:01, 48.47it/s]
Loading 0:  79%|███████▉  | 230/291 [00:05<00:01, 45.03it/s]
Loading 0:  82%|████████▏ | 238/291 [00:05<00:01, 52.24it/s]
Loading 0:  84%|████████▍ | 244/291 [00:05<00:00, 47.61it/s]
Loading 0:  86%|████████▌ | 250/291 [00:05<00:00, 48.30it/s]
Loading 0:  88%|████████▊ | 256/291 [00:05<00:00, 51.07it/s]
Loading 0:  90%|█████████ | 262/291 [00:05<00:00, 48.81it/s]
Loading 0:  92%|█████████▏| 268/291 [00:05<00:00, 49.07it/s]
Loading 0:  94%|█████████▍| 274/291 [00:05<00:00, 51.50it/s]
Loading 0:  96%|█████████▌| 280/291 [00:06<00:00, 49.92it/s]
Loading 0:  98%|█████████▊| 286/291 [00:06<00:00, 45.97it/s]
Loading 0: 100%|██████████| 291/291 [00:11<00:00,  3.39it/s]
                                                            
Job cgato-l3-1-8b-thespice-v-8134-v1-mkmlizer completed after 84.18s with status: succeeded
Stopping job with name cgato-l3-1-8b-thespice-v-8134-v1-mkmlizer
Pipeline stage MKMLizer completed in 84.70s
run pipeline stage %s
Running pipeline stage MKMLTemplater
Pipeline stage MKMLTemplater completed in 0.15s
run pipeline stage %s
Running pipeline stage MKMLDeployer
Creating inference service cgato-l3-1-8b-thespice-v-8134-v1
Waiting for inference service cgato-l3-1-8b-thespice-v-8134-v1 to be ready
Inference service cgato-l3-1-8b-thespice-v-8134-v1 ready after 150.73736381530762s
Pipeline stage MKMLDeployer completed in 151.27s
run pipeline stage %s
Running pipeline stage StressChecker
Received healthy response to inference request in 1.6658201217651367s
Received healthy response to inference request in 1.2165701389312744s
Received healthy response to inference request in 1.4611432552337646s
Received healthy response to inference request in 1.279024362564087s
Received healthy response to inference request in 1.601529598236084s
5 requests
0 failed requests
5th percentile: 1.229060983657837
10th percentile: 1.2415518283843994
20th percentile: 1.2665335178375243
30th percentile: 1.3154481410980225
40th percentile: 1.3882956981658936
50th percentile: 1.4611432552337646
60th percentile: 1.5172977924346924
70th percentile: 1.57345232963562
80th percentile: 1.6143877029418945
90th percentile: 1.6401039123535157
95th percentile: 1.6529620170593262
99th percentile: 1.6632485008239746
mean time: 1.4448174953460693
Pipeline stage StressChecker completed in 8.59s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyTriggerPipeline
run_pipeline:run_in_cloud %s
starting trigger_guanaco_pipeline args=%s
Pipeline stage OfflineFamilyFriendlyTriggerPipeline completed in 1.00s
Shutdown handler de-registered
cgato-l3-1-8b-thespice-v_8134_v1 status is now deployed due to DeploymentManager action
cgato-l3-1-8b-thespice-v_8134_v1 status is now inactive due to auto deactivation removed underperforming models
cgato-l3-1-8b-thespice-v_8134_v1 status is now torndown due to DeploymentManager action