turboderp-llama3-turbca_4336

submission_id: turboderp-llama3-turbca_4336_v20

developer_uid: rey-yan

best_of: 16

celo_rating: 1223.84

display_name: turboderp-llama3-turbcat

family_friendly_score: 0.0

formatter: {'memory_template': '<|begin_of_text|><|start_header_id|>system<|end_header_id|>\n\n{memory}<|eot_id|>', 'prompt_template': '<|start_header_id|>system<|end_header_id|>\n\nThe following message provides the necessary information about the below conversation and the characters in the conversation.\n{prompt}\nThe conversation below will be carried out according to information in the above text.<|eot_id|>', 'bot_template': '<|start_header_id|>assistant<|end_header_id|>\n\n{bot_name}: {message}<|eot_id|>', 'user_template': '<|start_header_id|>user<|end_header_id|>\n\n{user_name}: {message}<|eot_id|>', 'response_template': '<|start_header_id|>assistant<|end_header_id|>\n\n{bot_name}:', 'truncate_by_message': False}

generation_params: {'temperature': 0.9, 'top_p': 0.8, 'min_p': 0.0, 'top_k': 40, 'presence_penalty': 0.0, 'frequency_penalty': 0.0, 'stopping_words': ['\n'], 'max_input_tokens': 512, 'best_of': 16, 'max_output_tokens': 64}

is_internal_developer: False

language_model: turboderp/llama3-turbcat-instruct-8b

max_input_tokens: 512

max_output_tokens: 64

model_architecture: LlamaForCausalLM

model_group: turboderp/llama3-turbcat

model_name: turboderp-llama3-turbcat

model_num_parameters: 8030261248.0

model_repo: turboderp/llama3-turbcat-instruct-8b

model_size: 8B

num_battles: 10928

num_wins: 5384

ranking_group: single

status: torndown

submission_type: basic

timestamp: 2024-08-16T19:43:14+00:00

us_pacific_date: 2024-08-16

win_ratio: 0.49267935578330896

Resubmit model

Running pipeline stage MKMLizer
Starting job with name turboderp-llama3-turbca-4336-v20-mkmlizer
Waiting for job on turboderp-llama3-turbca-4336-v20-mkmlizer to finish
turboderp-llama3-turbca-4336-v20-mkmlizer: ╔═════════════════════════════════════════════════════════════════════╗
turboderp-llama3-turbca-4336-v20-mkmlizer: ║     _____            __           __                                ║
turboderp-llama3-turbca-4336-v20-mkmlizer: ║    / _/ /_ ___    __/ /  ___ ___ / /                                ║
turboderp-llama3-turbca-4336-v20-mkmlizer: ║   / _/ / // / |/|/ / _ \/ -_) -_) /                                 ║
turboderp-llama3-turbca-4336-v20-mkmlizer: ║  /_//_/\_, /|__,__/_//_/\__/\__/_/                                  ║
turboderp-llama3-turbca-4336-v20-mkmlizer: ║       /___/                                                         ║
turboderp-llama3-turbca-4336-v20-mkmlizer: ║                                                                     ║
turboderp-llama3-turbca-4336-v20-mkmlizer: ║  Version: 0.9.11                                                    ║
turboderp-llama3-turbca-4336-v20-mkmlizer: ║  Copyright 2023 MK ONE TECHNOLOGIES Inc.                            ║
turboderp-llama3-turbca-4336-v20-mkmlizer: ║  https://mk1.ai                                                     ║
turboderp-llama3-turbca-4336-v20-mkmlizer: ║                                                                     ║
turboderp-llama3-turbca-4336-v20-mkmlizer: ║  The license key for the current software has been verified as      ║
turboderp-llama3-turbca-4336-v20-mkmlizer: ║  belonging to:                                                      ║
turboderp-llama3-turbca-4336-v20-mkmlizer: ║                                                                     ║
turboderp-llama3-turbca-4336-v20-mkmlizer: ║  Chai Research Corp.                                                ║
turboderp-llama3-turbca-4336-v20-mkmlizer: ║  Account ID: 7997a29f-0ceb-4cc7-9adf-840c57b4ae6f                   ║
turboderp-llama3-turbca-4336-v20-mkmlizer: ║  Expiration: 2024-10-15 23:59:59                                    ║
turboderp-llama3-turbca-4336-v20-mkmlizer: ║                                                                     ║
turboderp-llama3-turbca-4336-v20-mkmlizer: ╚═════════════════════════════════════════════════════════════════════╝
turboderp-llama3-turbca-4336-v20-mkmlizer: Downloaded to shared memory in 22.116s
turboderp-llama3-turbca-4336-v20-mkmlizer: quantizing model to /dev/shm/model_cache, profile:s0, folder:/tmp/tmp8ix8nv5i, device:0
turboderp-llama3-turbca-4336-v20-mkmlizer: Saving flywheel model at /dev/shm/model_cache
turboderp-llama3-turbca-4336-v20-mkmlizer: quantized model in 25.853s
turboderp-llama3-turbca-4336-v20-mkmlizer: Processed model turboderp/llama3-turbcat-instruct-8b in 47.969s
turboderp-llama3-turbca-4336-v20-mkmlizer: creating bucket guanaco-mkml-models
turboderp-llama3-turbca-4336-v20-mkmlizer: Bucket 's3://guanaco-mkml-models/' created
turboderp-llama3-turbca-4336-v20-mkmlizer: uploading /dev/shm/model_cache to s3://guanaco-mkml-models/turboderp-llama3-turbca-4336-v20
turboderp-llama3-turbca-4336-v20-mkmlizer: cp /dev/shm/model_cache/config.json s3://guanaco-mkml-models/turboderp-llama3-turbca-4336-v20/config.json
turboderp-llama3-turbca-4336-v20-mkmlizer: cp /dev/shm/model_cache/special_tokens_map.json s3://guanaco-mkml-models/turboderp-llama3-turbca-4336-v20/special_tokens_map.json
turboderp-llama3-turbca-4336-v20-mkmlizer: cp /dev/shm/model_cache/tokenizer_config.json s3://guanaco-mkml-models/turboderp-llama3-turbca-4336-v20/tokenizer_config.json
turboderp-llama3-turbca-4336-v20-mkmlizer: cp /dev/shm/model_cache/tokenizer.json s3://guanaco-mkml-models/turboderp-llama3-turbca-4336-v20/tokenizer.json
turboderp-llama3-turbca-4336-v20-mkmlizer: cp /dev/shm/model_cache/flywheel_model.0.safetensors s3://guanaco-mkml-models/turboderp-llama3-turbca-4336-v20/flywheel_model.0.safetensors
turboderp-llama3-turbca-4336-v20-mkmlizer: 
Loading 0:   0%|          | 0/291 [00:00<?, ?it/s]
Loading 0:   2%|▏         | 5/291 [00:00<00:08, 33.58it/s]
Loading 0:   4%|▍         | 13/291 [00:00<00:04, 56.11it/s]
Loading 0:   7%|▋         | 20/291 [00:00<00:05, 52.54it/s]
Loading 0:   9%|▉         | 26/291 [00:00<00:05, 51.65it/s]
Loading 0:  11%|█         | 32/291 [00:00<00:05, 44.12it/s]
Loading 0:  14%|█▎        | 40/291 [00:00<00:04, 53.16it/s]
Loading 0:  16%|█▌        | 46/291 [00:00<00:04, 49.37it/s]
Loading 0:  18%|█▊        | 52/291 [00:01<00:04, 51.36it/s]
Loading 0:  20%|██        | 59/291 [00:01<00:04, 47.55it/s]
Loading 0:  23%|██▎       | 68/291 [00:01<00:04, 48.99it/s]
Loading 0:  26%|██▌       | 76/291 [00:01<00:03, 55.70it/s]
Loading 0:  28%|██▊       | 82/291 [00:01<00:04, 51.66it/s]
Loading 0:  30%|███       | 88/291 [00:01<00:05, 36.16it/s]
Loading 0:  32%|███▏      | 93/291 [00:02<00:05, 38.65it/s]
Loading 0:  34%|███▎      | 98/291 [00:02<00:04, 40.30it/s]
Loading 0:  35%|███▌      | 103/291 [00:02<00:04, 41.55it/s]
Loading 0:  37%|███▋      | 108/291 [00:02<00:04, 43.59it/s]
Loading 0:  39%|███▉      | 113/291 [00:02<00:04, 37.70it/s]
Loading 0:  42%|████▏     | 121/291 [00:02<00:03, 46.02it/s]
Loading 0:  44%|████▎     | 127/291 [00:02<00:03, 44.72it/s]
Loading 0:  45%|████▌     | 132/291 [00:02<00:03, 44.80it/s]
Loading 0:  48%|████▊     | 139/291 [00:03<00:03, 49.95it/s]
Loading 0:  50%|████▉     | 145/291 [00:03<00:03, 47.45it/s]
Loading 0:  52%|█████▏    | 150/291 [00:03<00:03, 46.59it/s]
Loading 0:  54%|█████▍    | 157/291 [00:03<00:02, 51.77it/s]
Loading 0:  56%|█████▌    | 163/291 [00:03<00:02, 45.21it/s]
Loading 0:  58%|█████▊    | 168/291 [00:03<00:02, 44.67it/s]
Loading 0:  60%|██████    | 176/291 [00:03<00:02, 53.06it/s]
Loading 0:  63%|██████▎   | 182/291 [00:03<00:02, 44.89it/s]
Loading 0:  64%|██████▍   | 187/291 [00:04<00:03, 34.18it/s]
Loading 0:  66%|██████▌   | 192/291 [00:04<00:02, 36.34it/s]
Loading 0:  68%|██████▊   | 197/291 [00:04<00:02, 38.63it/s]
Loading 0:  70%|██████▉   | 203/291 [00:04<00:02, 37.28it/s]
Loading 0:  73%|███████▎  | 211/291 [00:04<00:01, 45.50it/s]
Loading 0:  75%|███████▍  | 217/291 [00:04<00:01, 43.51it/s]
Loading 0:  76%|███████▋  | 222/291 [00:04<00:01, 42.90it/s]
Loading 0:  79%|███████▊  | 229/291 [00:05<00:01, 48.25it/s]
Loading 0:  81%|████████  | 235/291 [00:05<00:01, 46.25it/s]
Loading 0:  82%|████████▏ | 240/291 [00:05<00:01, 46.54it/s]
Loading 0:  85%|████████▍ | 247/291 [00:05<00:00, 52.22it/s]
Loading 0:  87%|████████▋ | 253/291 [00:05<00:00, 48.81it/s]
Loading 0:  89%|████████▉ | 259/291 [00:05<00:00, 50.20it/s]
Loading 0:  91%|█████████ | 265/291 [00:05<00:00, 51.82it/s]
Loading 0:  93%|█████████▎| 271/291 [00:05<00:00, 48.69it/s]
Loading 0:  95%|█████████▍| 276/291 [00:06<00:00, 47.13it/s]
Loading 0:  97%|█████████▋| 282/291 [00:06<00:00, 42.75it/s]
Loading 0:  99%|█████████▊| 287/291 [00:11<00:01,  3.29it/s]
                                                            
Job turboderp-llama3-turbca-4336-v20-mkmlizer completed after 73.47s with status: succeeded
Stopping job with name turboderp-llama3-turbca-4336-v20-mkmlizer
Pipeline stage MKMLizer completed in 74.29s
Running pipeline stage MKMLKubeTemplater
Pipeline stage MKMLKubeTemplater completed in 0.38s
Running pipeline stage ISVCDeployer
Creating inference service turboderp-llama3-turbca-4336-v20
Waiting for inference service turboderp-llama3-turbca-4336-v20 to be ready
Failed to get response for submission blend_nibok_2024-08-16: ('http://chaiml-elo-alignment-run-3-v34-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', '{"error":"ValueError : [TypeError(\\"\'numpy.int64\' object is not iterable\\"), TypeError(\'vars() argument must have __dict__ attribute\')]"}')
Inference service turboderp-llama3-turbca-4336-v20 ready after 231.56995964050293s
Pipeline stage ISVCDeployer completed in 233.11s
Running pipeline stage StressChecker
Received healthy response to inference request in 1.9641518592834473s
Received healthy response to inference request in 1.5205647945404053s
Received healthy response to inference request in 1.4246673583984375s
Received healthy response to inference request in 1.3460512161254883s
Received healthy response to inference request in 2.019223928451538s
5 requests
0 failed requests
5th percentile: 1.3617744445800781
10th percentile: 1.377497673034668
20th percentile: 1.4089441299438477
30th percentile: 1.443846845626831
40th percentile: 1.4822058200836181
50th percentile: 1.5205647945404053
60th percentile: 1.697999620437622
70th percentile: 1.8754344463348387
80th percentile: 1.9751662731170654
90th percentile: 1.9971951007843018
95th percentile: 2.0082095146179197
99th percentile: 2.0170210456848143
mean time: 1.6549318313598633
Pipeline stage StressChecker completed in 8.96s
turboderp-llama3-turbca_4336_v20 status is now deployed due to DeploymentManager action
turboderp-llama3-turbca_4336_v20 status is now inactive due to auto deactivation removed underperforming models
turboderp-llama3-turbca_4336_v20 status is now torndown due to DeploymentManager action