turboderp-llama3-turbca_4336

submission_id: turboderp-llama3-turbca_4336_v18

developer_uid: rey-yan

best_of: 16

celo_rating: 1228.31

display_name: turboderp-llama3-turbcat

family_friendly_score: 0.0

formatter: {'memory_template': '<|begin_of_text|><|start_header_id|>system<|end_header_id|>\n\n{memory}<|eot_id|>', 'prompt_template': '<|start_header_id|>system<|end_header_id|>\n\nThe following message provides the necessary information about the below conversation and the characters in the conversation.\n{prompt}\nThe conversation below will be carried out according to information in the above text.<|eot_id|>', 'bot_template': '<|start_header_id|>assistant<|end_header_id|>\n\n{bot_name}: {message}<|eot_id|>', 'user_template': '<|start_header_id|>user<|end_header_id|>\n\n{user_name}: {message}<|eot_id|>', 'response_template': '<|start_header_id|>assistant<|end_header_id|>\n\n{bot_name}:', 'truncate_by_message': False}

generation_params: {'temperature': 0.9, 'top_p': 0.8, 'min_p': 0.0, 'top_k': 40, 'presence_penalty': 0.0, 'frequency_penalty': 0.0, 'stopping_words': ['\n'], 'max_input_tokens': 512, 'best_of': 16, 'max_output_tokens': 64}

is_internal_developer: False

language_model: turboderp/llama3-turbcat-instruct-8b

max_input_tokens: 512

max_output_tokens: 64

model_architecture: LlamaForCausalLM

model_group: turboderp/llama3-turbcat

model_name: turboderp-llama3-turbcat

model_num_parameters: 8030261248.0

model_repo: turboderp/llama3-turbcat-instruct-8b

model_size: 8B

num_battles: 10654

num_wins: 5321

ranking_group: single

status: torndown

submission_type: basic

timestamp: 2024-08-16T19:42:43+00:00

us_pacific_date: 2024-08-16

win_ratio: 0.49943683123709404

Resubmit model

Running pipeline stage MKMLizer
Starting job with name turboderp-llama3-turbca-4336-v18-mkmlizer
Waiting for job on turboderp-llama3-turbca-4336-v18-mkmlizer to finish
Stopping job with name turboderp-llama3-turbca-4336-v18-mkmlizer
%s, retrying in %s seconds...
Starting job with name turboderp-llama3-turbca-4336-v18-mkmlizer
Waiting for job on turboderp-llama3-turbca-4336-v18-mkmlizer to finish
turboderp-llama3-turbca-4336-v18-mkmlizer: ╔═════════════════════════════════════════════════════════════════════╗
turboderp-llama3-turbca-4336-v18-mkmlizer: ║     _____            __           __                                ║
turboderp-llama3-turbca-4336-v18-mkmlizer: ║    / _/ /_ ___    __/ /  ___ ___ / /                                ║
turboderp-llama3-turbca-4336-v18-mkmlizer: ║   / _/ / // / |/|/ / _ \/ -_) -_) /                                 ║
turboderp-llama3-turbca-4336-v18-mkmlizer: ║  /_//_/\_, /|__,__/_//_/\__/\__/_/                                  ║
turboderp-llama3-turbca-4336-v18-mkmlizer: ║       /___/                                                         ║
turboderp-llama3-turbca-4336-v18-mkmlizer: ║                                                                     ║
turboderp-llama3-turbca-4336-v18-mkmlizer: ║  Version: 0.9.11                                                    ║
turboderp-llama3-turbca-4336-v18-mkmlizer: ║  Copyright 2023 MK ONE TECHNOLOGIES Inc.                            ║
turboderp-llama3-turbca-4336-v18-mkmlizer: ║  https://mk1.ai                                                     ║
turboderp-llama3-turbca-4336-v18-mkmlizer: ║                                                                     ║
turboderp-llama3-turbca-4336-v18-mkmlizer: ║  The license key for the current software has been verified as      ║
turboderp-llama3-turbca-4336-v18-mkmlizer: ║  belonging to:                                                      ║
turboderp-llama3-turbca-4336-v18-mkmlizer: ║                                                                     ║
turboderp-llama3-turbca-4336-v18-mkmlizer: ║  Chai Research Corp.                                                ║
turboderp-llama3-turbca-4336-v18-mkmlizer: ║  Account ID: 7997a29f-0ceb-4cc7-9adf-840c57b4ae6f                   ║
turboderp-llama3-turbca-4336-v18-mkmlizer: ║  Expiration: 2024-10-15 23:59:59                                    ║
turboderp-llama3-turbca-4336-v18-mkmlizer: ║                                                                     ║
turboderp-llama3-turbca-4336-v18-mkmlizer: ╚═════════════════════════════════════════════════════════════════════╝
turboderp-llama3-turbca-4336-v18-mkmlizer: Downloaded to shared memory in 41.880s
turboderp-llama3-turbca-4336-v18-mkmlizer: quantizing model to /dev/shm/model_cache, profile:s0, folder:/tmp/tmppjzd360a, device:0
turboderp-llama3-turbca-4336-v18-mkmlizer: Saving flywheel model at /dev/shm/model_cache
turboderp-llama3-turbca-4336-v18-mkmlizer: quantized model in 26.792s
turboderp-llama3-turbca-4336-v18-mkmlizer: Processed model turboderp/llama3-turbcat-instruct-8b in 68.672s
turboderp-llama3-turbca-4336-v18-mkmlizer: creating bucket guanaco-mkml-models
turboderp-llama3-turbca-4336-v18-mkmlizer: Bucket 's3://guanaco-mkml-models/' created
turboderp-llama3-turbca-4336-v18-mkmlizer: uploading /dev/shm/model_cache to s3://guanaco-mkml-models/turboderp-llama3-turbca-4336-v18
turboderp-llama3-turbca-4336-v18-mkmlizer: cp /dev/shm/model_cache/config.json s3://guanaco-mkml-models/turboderp-llama3-turbca-4336-v18/config.json
turboderp-llama3-turbca-4336-v18-mkmlizer: cp /dev/shm/model_cache/special_tokens_map.json s3://guanaco-mkml-models/turboderp-llama3-turbca-4336-v18/special_tokens_map.json
turboderp-llama3-turbca-4336-v18-mkmlizer: cp /dev/shm/model_cache/tokenizer_config.json s3://guanaco-mkml-models/turboderp-llama3-turbca-4336-v18/tokenizer_config.json
turboderp-llama3-turbca-4336-v18-mkmlizer: cp /dev/shm/model_cache/tokenizer.json s3://guanaco-mkml-models/turboderp-llama3-turbca-4336-v18/tokenizer.json
turboderp-llama3-turbca-4336-v18-mkmlizer: cp /dev/shm/model_cache/flywheel_model.0.safetensors s3://guanaco-mkml-models/turboderp-llama3-turbca-4336-v18/flywheel_model.0.safetensors
turboderp-llama3-turbca-4336-v18-mkmlizer: 
Loading 0:   0%|          | 0/291 [00:00<?, ?it/s]
Loading 0:   2%|▏         | 5/291 [00:00<00:08, 31.80it/s]
Loading 0:   4%|▍         | 13/291 [00:00<00:05, 53.13it/s]
Loading 0:   7%|▋         | 19/291 [00:00<00:05, 46.14it/s]
Loading 0:   8%|▊         | 24/291 [00:00<00:05, 45.01it/s]
Loading 0:  11%|█         | 31/291 [00:00<00:05, 50.46it/s]
Loading 0:  13%|█▎        | 37/291 [00:00<00:05, 45.68it/s]
Loading 0:  14%|█▍        | 42/291 [00:00<00:05, 45.17it/s]
Loading 0:  17%|█▋        | 49/291 [00:01<00:04, 50.47it/s]
Loading 0:  19%|█▉        | 55/291 [00:01<00:05, 46.46it/s]
Loading 0:  21%|██        | 60/291 [00:01<00:05, 45.99it/s]
Loading 0:  23%|██▎       | 67/291 [00:01<00:04, 51.52it/s]
Loading 0:  25%|██▌       | 73/291 [00:01<00:04, 46.65it/s]
Loading 0:  27%|██▋       | 78/291 [00:01<00:04, 45.54it/s]
Loading 0:  29%|██▊       | 83/291 [00:01<00:06, 32.18it/s]
Loading 0:  30%|██▉       | 87/291 [00:02<00:06, 32.17it/s]
Loading 0:  32%|███▏      | 93/291 [00:02<00:05, 37.60it/s]
Loading 0:  34%|███▎      | 98/291 [00:02<00:04, 39.61it/s]
Loading 0:  35%|███▌      | 103/291 [00:02<00:04, 41.46it/s]
Loading 0:  37%|███▋      | 108/291 [00:02<00:04, 43.27it/s]
Loading 0:  39%|███▉      | 113/291 [00:02<00:04, 37.27it/s]
Loading 0:  42%|████▏     | 121/291 [00:02<00:03, 45.80it/s]
Loading 0:  44%|████▎     | 127/291 [00:02<00:03, 43.19it/s]
Loading 0:  45%|████▌     | 132/291 [00:03<00:03, 42.71it/s]
Loading 0:  48%|████▊     | 139/291 [00:03<00:03, 47.66it/s]
Loading 0:  50%|████▉     | 145/291 [00:03<00:03, 44.36it/s]
Loading 0:  52%|█████▏    | 150/291 [00:03<00:03, 43.70it/s]
Loading 0:  54%|█████▍    | 157/291 [00:03<00:02, 48.83it/s]
Loading 0:  56%|█████▌    | 163/291 [00:03<00:02, 45.26it/s]
Loading 0:  58%|█████▊    | 168/291 [00:03<00:02, 44.33it/s]
Loading 0:  59%|█████▉    | 173/291 [00:03<00:02, 45.40it/s]
Loading 0:  62%|██████▏   | 180/291 [00:04<00:02, 50.03it/s]
Loading 0:  64%|██████▍   | 186/291 [00:04<00:02, 45.91it/s]
Loading 0:  66%|██████▌   | 191/291 [00:04<00:03, 31.78it/s]
Loading 0:  67%|██████▋   | 195/291 [00:04<00:02, 32.15it/s]
Loading 0:  69%|██████▉   | 202/291 [00:04<00:02, 38.84it/s]
Loading 0:  71%|███████▏  | 208/291 [00:04<00:02, 38.08it/s]
Loading 0:  73%|███████▎  | 213/291 [00:05<00:02, 38.23it/s]
Loading 0:  75%|███████▌  | 219/291 [00:05<00:01, 42.56it/s]
Loading 0:  77%|███████▋  | 224/291 [00:05<00:01, 43.50it/s]
Loading 0:  79%|███████▊  | 229/291 [00:05<00:01, 44.07it/s]
Loading 0:  81%|████████  | 235/291 [00:05<00:01, 41.96it/s]
Loading 0:  82%|████████▏ | 240/291 [00:05<00:01, 42.30it/s]
Loading 0:  85%|████████▍ | 246/291 [00:05<00:00, 46.53it/s]
Loading 0:  86%|████████▋ | 251/291 [00:05<00:00, 46.33it/s]
Loading 0:  88%|████████▊ | 256/291 [00:05<00:00, 45.82it/s]
Loading 0:  90%|█████████ | 262/291 [00:06<00:00, 43.48it/s]
Loading 0:  92%|█████████▏| 267/291 [00:06<00:00, 42.91it/s]
Loading 0:  94%|█████████▍| 274/291 [00:06<00:00, 48.25it/s]
Loading 0:  96%|█████████▌| 280/291 [00:06<00:00, 44.84it/s]
Loading 0:  98%|█████████▊| 285/291 [00:06<00:00, 44.25it/s]
Loading 0: 100%|█████████▉| 290/291 [00:12<00:00,  3.16it/s]
                                                            
Job turboderp-llama3-turbca-4336-v18-mkmlizer completed after 94.48s with status: succeeded
Stopping job with name turboderp-llama3-turbca-4336-v18-mkmlizer
Pipeline stage MKMLizer completed in 96.20s
Running pipeline stage MKMLKubeTemplater
Pipeline stage MKMLKubeTemplater completed in 0.10s
Running pipeline stage ISVCDeployer
Creating inference service turboderp-llama3-turbca-4336-v18
Waiting for inference service turboderp-llama3-turbca-4336-v18 to be ready
Inference service turboderp-llama3-turbca-4336-v18 ready after 241.91295671463013s
Pipeline stage ISVCDeployer completed in 244.11s
Running pipeline stage StressChecker
Received healthy response to inference request in 2.271116256713867s
Received healthy response to inference request in 1.5925796031951904s
Received healthy response to inference request in 1.5332200527191162s
Received healthy response to inference request in 1.2568342685699463s
Received healthy response to inference request in 1.520796298980713s
5 requests
0 failed requests
5th percentile: 1.3096266746520997
10th percentile: 1.3624190807342529
20th percentile: 1.4680038928985595
30th percentile: 1.5232810497283935
40th percentile: 1.528250551223755
50th percentile: 1.5332200527191162
60th percentile: 1.5569638729095459
70th percentile: 1.5807076930999755
80th percentile: 1.728286933898926
90th percentile: 1.9997015953063966
95th percentile: 2.1354089260101317
99th percentile: 2.24397479057312
mean time: 1.6349092960357665
Pipeline stage StressChecker completed in 9.02s
turboderp-llama3-turbca_4336_v18 status is now deployed due to DeploymentManager action
turboderp-llama3-turbca_4336_v18 status is now inactive due to auto deactivation removed underperforming models
turboderp-llama3-turbca_4336_v18 status is now torndown due to DeploymentManager action