submission_id: turboderp-llama3-turbca_4336_v22
developer_uid: rey-yan
alignment_samples: 12799
alignment_score: -1.4648009009547658
best_of: 16
celo_rating: 1226.44
display_name: turboderp-llama3-turbcat
formatter: {'memory_template': '<|begin_of_text|><|start_header_id|>system<|end_header_id|>\n\n{memory}<|eot_id|>', 'prompt_template': '<|start_header_id|>system<|end_header_id|>\n\nThe following message provides the necessary information about the below conversation and the characters in the conversation.\n{prompt}\nThe conversation below will be carried out according to information in the above text.<|eot_id|>', 'bot_template': '<|start_header_id|>assistant<|end_header_id|>\n\n{bot_name}: {message}<|eot_id|>', 'user_template': '<|start_header_id|>user<|end_header_id|>\n\n{user_name}: {message}<|eot_id|>', 'response_template': '<|start_header_id|>assistant<|end_header_id|>\n\n{bot_name}:', 'truncate_by_message': False}
generation_params: {'temperature': 0.9, 'top_p': 0.8, 'min_p': 0.0, 'top_k': 40, 'presence_penalty': 0.0, 'frequency_penalty': 0.0, 'stopping_words': ['\n'], 'max_input_tokens': 512, 'best_of': 16, 'max_output_tokens': 64}
is_internal_developer: False
language_model: turboderp/llama3-turbcat-instruct-8b
max_input_tokens: 512
max_output_tokens: 64
model_architecture: LlamaForCausalLM
model_group: turboderp/llama3-turbcat
model_name: turboderp-llama3-turbcat
model_num_parameters: 8030261248.0
model_repo: turboderp/llama3-turbcat-instruct-8b
model_size: 8B
num_battles: 12799
num_wins: 6358
propriety_score: 0.7209737827715356
propriety_total_count: 1068.0
ranking_group: single
status: torndown
submission_type: basic
timestamp: 2024-08-16T19:44:29+00:00
us_pacific_date: 2024-08-16
win_ratio: 0.49675755918431125
Download Preferencedata
Resubmit model
Running pipeline stage MKMLizer
Starting job with name turboderp-llama3-turbca-4336-v22-mkmlizer
Waiting for job on turboderp-llama3-turbca-4336-v22-mkmlizer to finish
turboderp-llama3-turbca-4336-v22-mkmlizer: ╔═════════════════════════════════════════════════════════════════════╗
turboderp-llama3-turbca-4336-v22-mkmlizer: ║ _____ __ __ ║
turboderp-llama3-turbca-4336-v22-mkmlizer: ║ / _/ /_ ___ __/ / ___ ___ / / ║
turboderp-llama3-turbca-4336-v22-mkmlizer: ║ / _/ / // / |/|/ / _ \/ -_) -_) / ║
turboderp-llama3-turbca-4336-v22-mkmlizer: ║ /_//_/\_, /|__,__/_//_/\__/\__/_/ ║
turboderp-llama3-turbca-4336-v22-mkmlizer: ║ /___/ ║
turboderp-llama3-turbca-4336-v22-mkmlizer: ║ ║
turboderp-llama3-turbca-4336-v22-mkmlizer: ║ Version: 0.9.11 ║
turboderp-llama3-turbca-4336-v22-mkmlizer: ║ Copyright 2023 MK ONE TECHNOLOGIES Inc. ║
turboderp-llama3-turbca-4336-v22-mkmlizer: ║ https://mk1.ai ║
turboderp-llama3-turbca-4336-v22-mkmlizer: ║ ║
turboderp-llama3-turbca-4336-v22-mkmlizer: ║ The license key for the current software has been verified as ║
turboderp-llama3-turbca-4336-v22-mkmlizer: ║ belonging to: ║
turboderp-llama3-turbca-4336-v22-mkmlizer: ║ ║
turboderp-llama3-turbca-4336-v22-mkmlizer: ║ Chai Research Corp. ║
turboderp-llama3-turbca-4336-v22-mkmlizer: ║ Account ID: 7997a29f-0ceb-4cc7-9adf-840c57b4ae6f ║
turboderp-llama3-turbca-4336-v22-mkmlizer: ║ Expiration: 2024-10-15 23:59:59 ║
turboderp-llama3-turbca-4336-v22-mkmlizer: ║ ║
turboderp-llama3-turbca-4336-v22-mkmlizer: ╚═════════════════════════════════════════════════════════════════════╝
turboderp-llama3-turbca-4336-v22-mkmlizer: Downloaded to shared memory in 26.242s
turboderp-llama3-turbca-4336-v22-mkmlizer: quantizing model to /dev/shm/model_cache, profile:s0, folder:/tmp/tmp84g57ofs, device:0
turboderp-llama3-turbca-4336-v22-mkmlizer: Saving flywheel model at /dev/shm/model_cache
turboderp-llama3-turbca-4336-v22-mkmlizer: quantized model in 25.906s
turboderp-llama3-turbca-4336-v22-mkmlizer: Processed model turboderp/llama3-turbcat-instruct-8b in 52.148s
turboderp-llama3-turbca-4336-v22-mkmlizer: creating bucket guanaco-mkml-models
turboderp-llama3-turbca-4336-v22-mkmlizer: Bucket 's3://guanaco-mkml-models/' created
turboderp-llama3-turbca-4336-v22-mkmlizer: uploading /dev/shm/model_cache to s3://guanaco-mkml-models/turboderp-llama3-turbca-4336-v22
turboderp-llama3-turbca-4336-v22-mkmlizer: cp /dev/shm/model_cache/config.json s3://guanaco-mkml-models/turboderp-llama3-turbca-4336-v22/config.json
turboderp-llama3-turbca-4336-v22-mkmlizer: cp /dev/shm/model_cache/special_tokens_map.json s3://guanaco-mkml-models/turboderp-llama3-turbca-4336-v22/special_tokens_map.json
turboderp-llama3-turbca-4336-v22-mkmlizer: cp /dev/shm/model_cache/tokenizer_config.json s3://guanaco-mkml-models/turboderp-llama3-turbca-4336-v22/tokenizer_config.json
turboderp-llama3-turbca-4336-v22-mkmlizer: cp /dev/shm/model_cache/tokenizer.json s3://guanaco-mkml-models/turboderp-llama3-turbca-4336-v22/tokenizer.json
turboderp-llama3-turbca-4336-v22-mkmlizer: cp /dev/shm/model_cache/flywheel_model.0.safetensors s3://guanaco-mkml-models/turboderp-llama3-turbca-4336-v22/flywheel_model.0.safetensors
turboderp-llama3-turbca-4336-v22-mkmlizer: Loading 0: 0%| | 0/291 [00:00<?, ?it/s] Loading 0: 2%|▏ | 5/291 [00:00<00:08, 34.22it/s] Loading 0: 4%|▍ | 13/291 [00:00<00:04, 55.70it/s] Loading 0: 7%|▋ | 19/291 [00:00<00:05, 49.64it/s] Loading 0: 9%|▊ | 25/291 [00:00<00:05, 51.95it/s] Loading 0: 11%|█ | 32/291 [00:00<00:05, 47.77it/s] Loading 0: 14%|█▍ | 41/291 [00:00<00:04, 50.34it/s] Loading 0: 17%|█▋ | 50/291 [00:00<00:04, 51.69it/s] Loading 0: 20%|██ | 59/291 [00:01<00:04, 52.54it/s] Loading 0: 23%|██▎ | 68/291 [00:01<00:04, 52.93it/s] Loading 0: 26%|██▋ | 77/291 [00:01<00:03, 53.52it/s] Loading 0: 29%|██▊ | 83/291 [00:01<00:04, 41.98it/s] Loading 0: 30%|███ | 88/291 [00:01<00:04, 42.55it/s] Loading 0: 32%|███▏ | 94/291 [00:01<00:04, 46.12it/s] Loading 0: 34%|███▍ | 100/291 [00:02<00:04, 45.53it/s] Loading 0: 36%|███▌ | 105/291 [00:02<00:04, 44.92it/s] Loading 0: 38%|███▊ | 112/291 [00:02<00:03, 50.32it/s] Loading 0: 41%|████ | 118/291 [00:02<00:03, 48.01it/s] Loading 0: 42%|████▏ | 123/291 [00:02<00:03, 47.87it/s] Loading 0: 45%|████▍ | 130/291 [00:02<00:03, 52.92it/s] Loading 0: 47%|████▋ | 136/291 [00:02<00:03, 49.05it/s] Loading 0: 49%|████▉ | 142/291 [00:02<00:02, 50.63it/s] Loading 0: 51%|█████ | 148/291 [00:03<00:02, 52.71it/s] Loading 0: 53%|█████▎ | 154/291 [00:03<00:02, 49.90it/s] Loading 0: 55%|█████▍ | 160/291 [00:03<00:02, 51.26it/s] Loading 0: 57%|█████▋ | 166/291 [00:03<00:02, 53.04it/s] Loading 0: 59%|█████▉ | 172/291 [00:03<00:02, 50.15it/s] Loading 0: 62%|██████▏ | 179/291 [00:03<00:02, 54.71it/s] Loading 0: 64%|██████▎ | 185/291 [00:03<00:01, 55.04it/s] Loading 0: 66%|██████▌ | 191/291 [00:04<00:02, 35.60it/s] Loading 0: 67%|██████▋ | 196/291 [00:04<00:02, 37.25it/s] Loading 0: 69%|██████▉ | 202/291 [00:04<00:02, 41.35it/s] Loading 0: 71%|███████▏ | 208/291 [00:04<00:01, 41.84it/s] Loading 0: 73%|███████▎ | 213/291 [00:04<00:01, 42.88it/s] Loading 0: 76%|███████▌ | 221/291 [00:04<00:01, 44.48it/s] Loading 0: 79%|███████▊ | 229/291 [00:04<00:01, 51.90it/s] Loading 0: 81%|████████ | 235/291 [00:04<00:01, 49.34it/s] Loading 0: 83%|████████▎ | 241/291 [00:05<00:00, 51.20it/s] Loading 0: 85%|████████▍ | 247/291 [00:05<00:00, 52.36it/s] Loading 0: 87%|████████▋ | 253/291 [00:05<00:00, 49.25it/s] Loading 0: 89%|████████▉ | 259/291 [00:05<00:00, 51.09it/s] Loading 0: 91%|█████████▏| 266/291 [00:05<00:00, 47.54it/s] Loading 0: 94%|█████████▍| 274/291 [00:05<00:00, 54.76it/s] Loading 0: 96%|█████████▌| 280/291 [00:05<00:00, 51.16it/s] Loading 0: 98%|█████████▊| 286/291 [00:05<00:00, 46.49it/s] Loading 0: 100%|██████████| 291/291 [00:11<00:00, 3.52it/s]
Job turboderp-llama3-turbca-4336-v22-mkmlizer completed after 73.83s with status: succeeded
Stopping job with name turboderp-llama3-turbca-4336-v22-mkmlizer
Pipeline stage MKMLizer completed in 74.87s
Running pipeline stage MKMLKubeTemplater
Pipeline stage MKMLKubeTemplater completed in 0.10s
Running pipeline stage ISVCDeployer
Creating inference service turboderp-llama3-turbca-4336-v22
Waiting for inference service turboderp-llama3-turbca-4336-v22 to be ready
Failed to get response for submission blend_filor_2024-08-16: ('http://neversleep-noromaid-v0-8068-v149-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'read tcp 127.0.0.1:33266->127.0.0.1:8080: read: connection reset by peer\n')
Inference service turboderp-llama3-turbca-4336-v22 ready after 241.47041296958923s
Pipeline stage ISVCDeployer completed in 243.06s
Running pipeline stage StressChecker
Received healthy response to inference request in 2.0801312923431396s
Received healthy response to inference request in 1.7143185138702393s
Received healthy response to inference request in 1.561471939086914s
Received healthy response to inference request in 1.3700635433197021s
Received healthy response to inference request in 1.5206191539764404s
5 requests
0 failed requests
5th percentile: 1.4001746654510498
10th percentile: 1.4302857875823975
20th percentile: 1.4905080318450927
30th percentile: 1.5287897109985351
40th percentile: 1.5451308250427247
50th percentile: 1.561471939086914
60th percentile: 1.6226105690002441
70th percentile: 1.6837491989135742
80th percentile: 1.7874810695648193
90th percentile: 1.9338061809539795
95th percentile: 2.0069687366485596
99th percentile: 2.065498781204224
mean time: 1.6493208885192872
Pipeline stage StressChecker completed in 8.96s
turboderp-llama3-turbca_4336_v22 status is now deployed due to DeploymentManager action
turboderp-llama3-turbca_4336_v22 status is now inactive due to auto deactivation removed underperforming models
turboderp-llama3-turbca_4336_v22 status is now torndown due to DeploymentManager action

Usage Metrics

Latency Metrics