submission_id: turboderp-llama3-turbca_4336_v20
developer_uid: rey-yan
alignment_samples: 10928
alignment_score: -1.292857534700583
best_of: 16
celo_rating: 1223.84
display_name: turboderp-llama3-turbcat
formatter: {'memory_template': '<|begin_of_text|><|start_header_id|>system<|end_header_id|>\n\n{memory}<|eot_id|>', 'prompt_template': '<|start_header_id|>system<|end_header_id|>\n\nThe following message provides the necessary information about the below conversation and the characters in the conversation.\n{prompt}\nThe conversation below will be carried out according to information in the above text.<|eot_id|>', 'bot_template': '<|start_header_id|>assistant<|end_header_id|>\n\n{bot_name}: {message}<|eot_id|>', 'user_template': '<|start_header_id|>user<|end_header_id|>\n\n{user_name}: {message}<|eot_id|>', 'response_template': '<|start_header_id|>assistant<|end_header_id|>\n\n{bot_name}:', 'truncate_by_message': False}
generation_params: {'temperature': 0.9, 'top_p': 0.8, 'min_p': 0.0, 'top_k': 40, 'presence_penalty': 0.0, 'frequency_penalty': 0.0, 'stopping_words': ['\n'], 'max_input_tokens': 512, 'best_of': 16, 'max_output_tokens': 64}
is_internal_developer: False
language_model: turboderp/llama3-turbcat-instruct-8b
max_input_tokens: 512
max_output_tokens: 64
model_architecture: LlamaForCausalLM
model_group: turboderp/llama3-turbcat
model_name: turboderp-llama3-turbcat
model_num_parameters: 8030261248.0
model_repo: turboderp/llama3-turbcat-instruct-8b
model_size: 8B
num_battles: 10928
num_wins: 5384
propriety_score: 0.7413249211356467
propriety_total_count: 951.0
ranking_group: single
status: torndown
submission_type: basic
timestamp: 2024-08-16T19:43:14+00:00
us_pacific_date: 2024-08-16
win_ratio: 0.49267935578330896
Download Preferencedata
Resubmit model
Running pipeline stage MKMLizer
Starting job with name turboderp-llama3-turbca-4336-v20-mkmlizer
Waiting for job on turboderp-llama3-turbca-4336-v20-mkmlizer to finish
turboderp-llama3-turbca-4336-v20-mkmlizer: ╔═════════════════════════════════════════════════════════════════════╗
turboderp-llama3-turbca-4336-v20-mkmlizer: ║ _____ __ __ ║
turboderp-llama3-turbca-4336-v20-mkmlizer: ║ / _/ /_ ___ __/ / ___ ___ / / ║
turboderp-llama3-turbca-4336-v20-mkmlizer: ║ / _/ / // / |/|/ / _ \/ -_) -_) / ║
turboderp-llama3-turbca-4336-v20-mkmlizer: ║ /_//_/\_, /|__,__/_//_/\__/\__/_/ ║
turboderp-llama3-turbca-4336-v20-mkmlizer: ║ /___/ ║
turboderp-llama3-turbca-4336-v20-mkmlizer: ║ ║
turboderp-llama3-turbca-4336-v20-mkmlizer: ║ Version: 0.9.11 ║
turboderp-llama3-turbca-4336-v20-mkmlizer: ║ Copyright 2023 MK ONE TECHNOLOGIES Inc. ║
turboderp-llama3-turbca-4336-v20-mkmlizer: ║ https://mk1.ai ║
turboderp-llama3-turbca-4336-v20-mkmlizer: ║ ║
turboderp-llama3-turbca-4336-v20-mkmlizer: ║ The license key for the current software has been verified as ║
turboderp-llama3-turbca-4336-v20-mkmlizer: ║ belonging to: ║
turboderp-llama3-turbca-4336-v20-mkmlizer: ║ ║
turboderp-llama3-turbca-4336-v20-mkmlizer: ║ Chai Research Corp. ║
turboderp-llama3-turbca-4336-v20-mkmlizer: ║ Account ID: 7997a29f-0ceb-4cc7-9adf-840c57b4ae6f ║
turboderp-llama3-turbca-4336-v20-mkmlizer: ║ Expiration: 2024-10-15 23:59:59 ║
turboderp-llama3-turbca-4336-v20-mkmlizer: ║ ║
turboderp-llama3-turbca-4336-v20-mkmlizer: ╚═════════════════════════════════════════════════════════════════════╝
turboderp-llama3-turbca-4336-v20-mkmlizer: Downloaded to shared memory in 22.116s
turboderp-llama3-turbca-4336-v20-mkmlizer: quantizing model to /dev/shm/model_cache, profile:s0, folder:/tmp/tmp8ix8nv5i, device:0
turboderp-llama3-turbca-4336-v20-mkmlizer: Saving flywheel model at /dev/shm/model_cache
turboderp-llama3-turbca-4336-v20-mkmlizer: quantized model in 25.853s
turboderp-llama3-turbca-4336-v20-mkmlizer: Processed model turboderp/llama3-turbcat-instruct-8b in 47.969s
turboderp-llama3-turbca-4336-v20-mkmlizer: creating bucket guanaco-mkml-models
turboderp-llama3-turbca-4336-v20-mkmlizer: Bucket 's3://guanaco-mkml-models/' created
turboderp-llama3-turbca-4336-v20-mkmlizer: uploading /dev/shm/model_cache to s3://guanaco-mkml-models/turboderp-llama3-turbca-4336-v20
turboderp-llama3-turbca-4336-v20-mkmlizer: cp /dev/shm/model_cache/config.json s3://guanaco-mkml-models/turboderp-llama3-turbca-4336-v20/config.json
turboderp-llama3-turbca-4336-v20-mkmlizer: cp /dev/shm/model_cache/special_tokens_map.json s3://guanaco-mkml-models/turboderp-llama3-turbca-4336-v20/special_tokens_map.json
turboderp-llama3-turbca-4336-v20-mkmlizer: cp /dev/shm/model_cache/tokenizer_config.json s3://guanaco-mkml-models/turboderp-llama3-turbca-4336-v20/tokenizer_config.json
turboderp-llama3-turbca-4336-v20-mkmlizer: cp /dev/shm/model_cache/tokenizer.json s3://guanaco-mkml-models/turboderp-llama3-turbca-4336-v20/tokenizer.json
turboderp-llama3-turbca-4336-v20-mkmlizer: cp /dev/shm/model_cache/flywheel_model.0.safetensors s3://guanaco-mkml-models/turboderp-llama3-turbca-4336-v20/flywheel_model.0.safetensors
turboderp-llama3-turbca-4336-v20-mkmlizer: Loading 0: 0%| | 0/291 [00:00<?, ?it/s] Loading 0: 2%|▏ | 5/291 [00:00<00:08, 33.58it/s] Loading 0: 4%|▍ | 13/291 [00:00<00:04, 56.11it/s] Loading 0: 7%|▋ | 20/291 [00:00<00:05, 52.54it/s] Loading 0: 9%|▉ | 26/291 [00:00<00:05, 51.65it/s] Loading 0: 11%|█ | 32/291 [00:00<00:05, 44.12it/s] Loading 0: 14%|█▎ | 40/291 [00:00<00:04, 53.16it/s] Loading 0: 16%|█▌ | 46/291 [00:00<00:04, 49.37it/s] Loading 0: 18%|█▊ | 52/291 [00:01<00:04, 51.36it/s] Loading 0: 20%|██ | 59/291 [00:01<00:04, 47.55it/s] Loading 0: 23%|██▎ | 68/291 [00:01<00:04, 48.99it/s] Loading 0: 26%|██▌ | 76/291 [00:01<00:03, 55.70it/s] Loading 0: 28%|██▊ | 82/291 [00:01<00:04, 51.66it/s] Loading 0: 30%|███ | 88/291 [00:01<00:05, 36.16it/s] Loading 0: 32%|███▏ | 93/291 [00:02<00:05, 38.65it/s] Loading 0: 34%|███▎ | 98/291 [00:02<00:04, 40.30it/s] Loading 0: 35%|███▌ | 103/291 [00:02<00:04, 41.55it/s] Loading 0: 37%|███▋ | 108/291 [00:02<00:04, 43.59it/s] Loading 0: 39%|███▉ | 113/291 [00:02<00:04, 37.70it/s] Loading 0: 42%|████▏ | 121/291 [00:02<00:03, 46.02it/s] Loading 0: 44%|████▎ | 127/291 [00:02<00:03, 44.72it/s] Loading 0: 45%|████▌ | 132/291 [00:02<00:03, 44.80it/s] Loading 0: 48%|████▊ | 139/291 [00:03<00:03, 49.95it/s] Loading 0: 50%|████▉ | 145/291 [00:03<00:03, 47.45it/s] Loading 0: 52%|█████▏ | 150/291 [00:03<00:03, 46.59it/s] Loading 0: 54%|█████▍ | 157/291 [00:03<00:02, 51.77it/s] Loading 0: 56%|█████▌ | 163/291 [00:03<00:02, 45.21it/s] Loading 0: 58%|█████▊ | 168/291 [00:03<00:02, 44.67it/s] Loading 0: 60%|██████ | 176/291 [00:03<00:02, 53.06it/s] Loading 0: 63%|██████▎ | 182/291 [00:03<00:02, 44.89it/s] Loading 0: 64%|██████▍ | 187/291 [00:04<00:03, 34.18it/s] Loading 0: 66%|██████▌ | 192/291 [00:04<00:02, 36.34it/s] Loading 0: 68%|██████▊ | 197/291 [00:04<00:02, 38.63it/s] Loading 0: 70%|██████▉ | 203/291 [00:04<00:02, 37.28it/s] Loading 0: 73%|███████▎ | 211/291 [00:04<00:01, 45.50it/s] Loading 0: 75%|███████▍ | 217/291 [00:04<00:01, 43.51it/s] Loading 0: 76%|███████▋ | 222/291 [00:04<00:01, 42.90it/s] Loading 0: 79%|███████▊ | 229/291 [00:05<00:01, 48.25it/s] Loading 0: 81%|████████ | 235/291 [00:05<00:01, 46.25it/s] Loading 0: 82%|████████▏ | 240/291 [00:05<00:01, 46.54it/s] Loading 0: 85%|████████▍ | 247/291 [00:05<00:00, 52.22it/s] Loading 0: 87%|████████▋ | 253/291 [00:05<00:00, 48.81it/s] Loading 0: 89%|████████▉ | 259/291 [00:05<00:00, 50.20it/s] Loading 0: 91%|█████████ | 265/291 [00:05<00:00, 51.82it/s] Loading 0: 93%|█████████▎| 271/291 [00:05<00:00, 48.69it/s] Loading 0: 95%|█████████▍| 276/291 [00:06<00:00, 47.13it/s] Loading 0: 97%|█████████▋| 282/291 [00:06<00:00, 42.75it/s] Loading 0: 99%|█████████▊| 287/291 [00:11<00:01, 3.29it/s]
Job turboderp-llama3-turbca-4336-v20-mkmlizer completed after 73.47s with status: succeeded
Stopping job with name turboderp-llama3-turbca-4336-v20-mkmlizer
Pipeline stage MKMLizer completed in 74.29s
Running pipeline stage MKMLKubeTemplater
Pipeline stage MKMLKubeTemplater completed in 0.38s
Running pipeline stage ISVCDeployer
Creating inference service turboderp-llama3-turbca-4336-v20
Waiting for inference service turboderp-llama3-turbca-4336-v20 to be ready
Failed to get response for submission blend_nibok_2024-08-16: ('http://chaiml-elo-alignment-run-3-v34-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', '{"error":"ValueError : [TypeError(\\"\'numpy.int64\' object is not iterable\\"), TypeError(\'vars() argument must have __dict__ attribute\')]"}')
Inference service turboderp-llama3-turbca-4336-v20 ready after 231.56995964050293s
Pipeline stage ISVCDeployer completed in 233.11s
Running pipeline stage StressChecker
Received healthy response to inference request in 1.9641518592834473s
Received healthy response to inference request in 1.5205647945404053s
Received healthy response to inference request in 1.4246673583984375s
Received healthy response to inference request in 1.3460512161254883s
Received healthy response to inference request in 2.019223928451538s
5 requests
0 failed requests
5th percentile: 1.3617744445800781
10th percentile: 1.377497673034668
20th percentile: 1.4089441299438477
30th percentile: 1.443846845626831
40th percentile: 1.4822058200836181
50th percentile: 1.5205647945404053
60th percentile: 1.697999620437622
70th percentile: 1.8754344463348387
80th percentile: 1.9751662731170654
90th percentile: 1.9971951007843018
95th percentile: 2.0082095146179197
99th percentile: 2.0170210456848143
mean time: 1.6549318313598633
Pipeline stage StressChecker completed in 8.96s
turboderp-llama3-turbca_4336_v20 status is now deployed due to DeploymentManager action
turboderp-llama3-turbca_4336_v20 status is now inactive due to auto deactivation removed underperforming models
turboderp-llama3-turbca_4336_v20 status is now torndown due to DeploymentManager action

Usage Metrics

Latency Metrics