submission_id: undi95-meta-llama-3-1-8b_1733_v2
developer_uid: Meliodia
alignment_samples: 0
best_of: 16
celo_rating: 1224.36
display_name: zonemercy-burly-blue-cp2500_v5
formatter: {'memory_template': "{bot_name}'s Persona: {memory}\n####\n", 'prompt_template': '{prompt}\n<START>\n', 'bot_template': '{bot_name}: {message}\n', 'user_template': '{user_name}: {message}\n', 'response_template': '{bot_name}:', 'truncate_by_message': False}
generation_params: {'temperature': 0.95, 'top_p': 0.95, 'min_p': 0.05, 'top_k': 80, 'presence_penalty': 0.0, 'frequency_penalty': 0.0, 'stopping_words': ['\n', '<|eot_id|>'], 'max_input_tokens': 512, 'best_of': 16, 'max_output_tokens': 64, 'reward_max_token_input': 256}
is_internal_developer: True
language_model: Undi95/Meta-Llama-3.1-8B-Claude
max_input_tokens: 512
max_output_tokens: 64
model_architecture: LlamaForCausalLM
model_group: Undi95/Meta-Llama-3.1-8B
model_name: zonemercy-burly-blue-cp2500_v5
model_num_parameters: 8030261248.0
model_repo: Undi95/Meta-Llama-3.1-8B-Claude
model_size: 8B
num_battles: 14009
num_wins: 7444
propriety_score: 0.7256916996047431
propriety_total_count: 1265.0
ranking_group: single
reward_formatter: {'bot_template': '{bot_name}: {message}\n', 'memory_template': "''", 'prompt_template': "''", 'response_template': '{bot_name}:', 'truncate_by_message': False, 'user_template': '{user_name}: {message}\n'}
reward_repo: ChaiML/gpt2_xl_pairwise_89m_step_347634
status: torndown
submission_type: basic
timestamp: 2024-07-26T20:07:53+00:00
us_pacific_date: 2024-07-26
win_ratio: 0.531372688985652
Download Preference Data
Resubmit model
Running pipeline stage MKMLizer
Starting job with name undi95-meta-llama-3-1-8b-1733-v2-mkmlizer
Waiting for job on undi95-meta-llama-3-1-8b-1733-v2-mkmlizer to finish
undi95-meta-llama-3-1-8b-1733-v2-mkmlizer: ╔═════════════════════════════════════════════════════════════════════╗
undi95-meta-llama-3-1-8b-1733-v2-mkmlizer: ║ _____ __ __ ║
undi95-meta-llama-3-1-8b-1733-v2-mkmlizer: ║ / _/ /_ ___ __/ / ___ ___ / / ║
undi95-meta-llama-3-1-8b-1733-v2-mkmlizer: ║ / _/ / // / |/|/ / _ \/ -_) -_) / ║
undi95-meta-llama-3-1-8b-1733-v2-mkmlizer: ║ /_//_/\_, /|__,__/_//_/\__/\__/_/ ║
undi95-meta-llama-3-1-8b-1733-v2-mkmlizer: ║ /___/ ║
undi95-meta-llama-3-1-8b-1733-v2-mkmlizer: ║ ║
undi95-meta-llama-3-1-8b-1733-v2-mkmlizer: ║ Version: 0.9.7 ║
undi95-meta-llama-3-1-8b-1733-v2-mkmlizer: ║ Copyright 2023 MK ONE TECHNOLOGIES Inc. ║
undi95-meta-llama-3-1-8b-1733-v2-mkmlizer: ║ https://mk1.ai ║
undi95-meta-llama-3-1-8b-1733-v2-mkmlizer: ║ ║
undi95-meta-llama-3-1-8b-1733-v2-mkmlizer: ║ The license key for the current software has been verified as ║
undi95-meta-llama-3-1-8b-1733-v2-mkmlizer: ║ belonging to: ║
undi95-meta-llama-3-1-8b-1733-v2-mkmlizer: ║ ║
undi95-meta-llama-3-1-8b-1733-v2-mkmlizer: ║ Chai Research Corp. ║
undi95-meta-llama-3-1-8b-1733-v2-mkmlizer: ║ Account ID: 7997a29f-0ceb-4cc7-9adf-840c57b4ae6f ║
undi95-meta-llama-3-1-8b-1733-v2-mkmlizer: ║ Expiration: 2024-10-15 23:59:59 ║
undi95-meta-llama-3-1-8b-1733-v2-mkmlizer: ║ ║
undi95-meta-llama-3-1-8b-1733-v2-mkmlizer: ╚═════════════════════════════════════════════════════════════════════╝
undi95-meta-llama-3-1-8b-1733-v2-mkmlizer: quantized model in 28.668s
undi95-meta-llama-3-1-8b-1733-v2-mkmlizer: Processed model Undi95/Meta-Llama-3.1-8B-Claude in 75.210s
undi95-meta-llama-3-1-8b-1733-v2-mkmlizer: creating bucket guanaco-mkml-models
undi95-meta-llama-3-1-8b-1733-v2-mkmlizer: Bucket 's3://guanaco-mkml-models/' created
undi95-meta-llama-3-1-8b-1733-v2-mkmlizer: uploading /dev/shm/model_cache to s3://guanaco-mkml-models/undi95-meta-llama-3-1-8b-1733-v2
undi95-meta-llama-3-1-8b-1733-v2-mkmlizer: cp /dev/shm/model_cache/config.json s3://guanaco-mkml-models/undi95-meta-llama-3-1-8b-1733-v2/config.json
undi95-meta-llama-3-1-8b-1733-v2-mkmlizer: cp /dev/shm/model_cache/special_tokens_map.json s3://guanaco-mkml-models/undi95-meta-llama-3-1-8b-1733-v2/special_tokens_map.json
undi95-meta-llama-3-1-8b-1733-v2-mkmlizer: cp /dev/shm/model_cache/tokenizer_config.json s3://guanaco-mkml-models/undi95-meta-llama-3-1-8b-1733-v2/tokenizer_config.json
undi95-meta-llama-3-1-8b-1733-v2-mkmlizer: cp /dev/shm/model_cache/tokenizer.json s3://guanaco-mkml-models/undi95-meta-llama-3-1-8b-1733-v2/tokenizer.json
undi95-meta-llama-3-1-8b-1733-v2-mkmlizer: cp /dev/shm/model_cache/flywheel_model.0.safetensors s3://guanaco-mkml-models/undi95-meta-llama-3-1-8b-1733-v2/flywheel_model.0.safetensors
undi95-meta-llama-3-1-8b-1733-v2-mkmlizer: loading reward model from ChaiML/gpt2_xl_pairwise_89m_step_347634
undi95-meta-llama-3-1-8b-1733-v2-mkmlizer: Loading 0: 0%| | 0/291 [00:00<?, ?it/s] Loading 0: 2%|▏ | 5/291 [00:00<00:10, 26.32it/s] Loading 0: 4%|▍ | 12/291 [00:00<00:07, 35.73it/s] Loading 0: 5%|▌ | 16/291 [00:00<00:08, 33.26it/s] Loading 0: 7%|▋ | 21/291 [00:00<00:07, 36.27it/s] Loading 0: 9%|▊ | 25/291 [00:00<00:07, 34.10it/s] Loading 0: 10%|█ | 30/291 [00:00<00:06, 37.90it/s] Loading 0: 12%|█▏ | 34/291 [00:01<00:10, 24.79it/s] Loading 0: 13%|█▎ | 38/291 [00:01<00:09, 26.22it/s] Loading 0: 14%|█▍ | 42/291 [00:01<00:09, 25.43it/s] Loading 0: 16%|█▋ | 48/291 [00:01<00:07, 30.85it/s] Loading 0: 18%|█▊ | 52/291 [00:01<00:07, 30.67it/s] Loading 0: 20%|█▉ | 57/291 [00:01<00:06, 33.73it/s] Loading 0: 21%|██ | 61/291 [00:01<00:07, 32.52it/s] Loading 0: 23%|██▎ | 66/291 [00:02<00:06, 35.35it/s] Loading 0: 24%|██▍ | 70/291 [00:02<00:06, 31.86it/s] Loading 0: 25%|██▌ | 74/291 [00:02<00:06, 32.26it/s] Loading 0: 27%|██▋ | 78/291 [00:02<00:06, 32.47it/s] Loading 0: 28%|██▊ | 82/291 [00:02<00:09, 22.80it/s] Loading 0: 29%|██▉ | 85/291 [00:02<00:08, 23.93it/s] Loading 0: 31%|███ | 90/291 [00:03<00:07, 27.93it/s] Loading 0: 32%|███▏ | 94/291 [00:03<00:06, 28.68it/s] Loading 0: 34%|███▍ | 99/291 [00:03<00:05, 32.40it/s] Loading 0: 35%|███▌ | 103/291 [00:03<00:05, 31.95it/s] Loading 0: 37%|███▋ | 108/291 [00:03<00:05, 34.28it/s] Loading 0: 38%|███▊ | 112/291 [00:03<00:05, 32.91it/s] Loading 0: 40%|███▉ | 116/291 [00:03<00:05, 33.05it/s] Loading 0: 41%|████ | 120/291 [00:03<00:04, 34.27it/s] Loading 0: 43%|████▎ | 126/291 [00:03<00:04, 39.90it/s] Loading 0: 45%|████▌ | 131/291 [00:04<00:03, 40.86it/s] Loading 0: 47%|████▋ | 136/291 [00:04<00:06, 25.49it/s] Loading 0: 48%|████▊ | 140/291 [00:04<00:05, 25.35it/s] Loading 0: 51%|█████ | 147/291 [00:04<00:04, 32.32it/s] Loading 0: 52%|█████▏ | 151/291 [00:04<00:04, 30.99it/s] Loading 0: 54%|█████▎ | 156/291 [00:05<00:04, 33.62it/s] Loading 0: 55%|█████▍ | 160/291 [00:05<00:03, 32.97it/s] Loading 0: 57%|█████▋ | 165/291 [00:05<00:03, 35.14it/s] Loading 0: 58%|█████▊ | 169/291 [00:05<00:03, 33.84it/s] Loading 0: 60%|█████▉ | 174/291 [00:05<00:03, 36.08it/s] Loading 0: 61%|██████ | 178/291 [00:05<00:03, 34.55it/s] Loading 0: 63%|██████▎ | 184/291 [00:05<00:02, 39.97it/s] Loading 0: 65%|██████▍ | 189/291 [00:06<00:04, 23.96it/s] Loading 0: 67%|██████▋ | 194/291 [00:06<00:03, 25.62it/s] Loading 0: 69%|██████▉ | 201/291 [00:06<00:02, 32.19it/s] Loading 0: 70%|███████ | 205/291 [00:06<00:02, 31.92it/s] Loading 0: 72%|███████▏ | 210/291 [00:06<00:02, 34.54it/s] Loading 0: 74%|███████▎ | 214/291 [00:06<00:02, 33.11it/s] Loading 0: 75%|███████▌ | 219/291 [00:06<00:02, 35.77it/s] Loading 0: 77%|███████▋ | 223/291 [00:07<00:01, 34.33it/s] Loading 0: 78%|███████▊ | 227/291 [00:07<00:01, 34.28it/s] Loading 0: 79%|███████▉ | 231/291 [00:07<00:01, 34.41it/s] Loading 0: 81%|████████ | 235/291 [00:07<00:02, 25.10it/s] Loading 0: 82%|████████▏ | 239/291 [00:07<00:02, 25.34it/s] Loading 0: 85%|████████▍ | 246/291 [00:07<00:01, 33.30it/s] Loading 0: 86%|████████▌ | 250/291 [00:07<00:01, 32.56it/s] Loading 0: 88%|████████▊ | 255/291 [00:08<00:01, 35.50it/s] Loading 0: 89%|████████▉ | 259/291 [00:08<00:00, 34.18it/s] Loading 0: 91%|█████████ | 264/291 [00:08<00:00, 37.02it/s] Loading 0: 92%|█████████▏| 268/291 [00:08<00:00, 35.47it/s] Loading 0: 94%|█████████▍| 273/291 [00:08<00:00, 37.62it/s] Loading 0: 95%|█████████▌| 277/291 [00:08<00:00, 35.85it/s] Loading 0: 97%|█████████▋| 281/291 [00:08<00:00, 35.57it/s] Loading 0: 98%|█████████▊| 286/291 [00:14<00:01, 2.66it/s] Loading 0: 99%|█████████▉| 289/291 [00:14<00:00, 3.30it/s] /opt/conda/lib/python3.10/site-packages/transformers/models/auto/configuration_auto.py:957: FutureWarning: The `use_auth_token` argument is deprecated and will be removed in v5 of Transformers. Please use `token` instead.
undi95-meta-llama-3-1-8b-1733-v2-mkmlizer: warnings.warn(
undi95-meta-llama-3-1-8b-1733-v2-mkmlizer: /opt/conda/lib/python3.10/site-packages/transformers/models/auto/tokenization_auto.py:785: FutureWarning: The `use_auth_token` argument is deprecated and will be removed in v5 of Transformers. Please use `token` instead.
undi95-meta-llama-3-1-8b-1733-v2-mkmlizer: warnings.warn(
undi95-meta-llama-3-1-8b-1733-v2-mkmlizer: /opt/conda/lib/python3.10/site-packages/transformers/models/auto/auto_factory.py:469: FutureWarning: The `use_auth_token` argument is deprecated and will be removed in v5 of Transformers. Please use `token` instead.
undi95-meta-llama-3-1-8b-1733-v2-mkmlizer: warnings.warn(
undi95-meta-llama-3-1-8b-1733-v2-mkmlizer: Downloading shards: 0%| | 0/2 [00:00<?, ?it/s] Downloading shards: 50%|█████ | 1/2 [00:05<00:05, 5.53s/it] Downloading shards: 100%|██████████| 2/2 [00:08<00:00, 3.95s/it] Downloading shards: 100%|██████████| 2/2 [00:08<00:00, 4.18s/it]
undi95-meta-llama-3-1-8b-1733-v2-mkmlizer: Loading checkpoint shards: 0%| | 0/2 [00:00<?, ?it/s] Loading checkpoint shards: 50%|█████ | 1/2 [00:00<00:00, 2.36it/s] Loading checkpoint shards: 100%|██████████| 2/2 [00:00<00:00, 3.96it/s] Loading checkpoint shards: 100%|██████████| 2/2 [00:00<00:00, 3.59it/s]
undi95-meta-llama-3-1-8b-1733-v2-mkmlizer: Saving model to /tmp/reward_cache/reward.tensors
undi95-meta-llama-3-1-8b-1733-v2-mkmlizer: Saving duration: 1.325s
undi95-meta-llama-3-1-8b-1733-v2-mkmlizer: Processed model ChaiML/gpt2_xl_pairwise_89m_step_347634 in 13.148s
undi95-meta-llama-3-1-8b-1733-v2-mkmlizer: Bucket 's3://guanaco-reward-models/' created
undi95-meta-llama-3-1-8b-1733-v2-mkmlizer: uploading /tmp/reward_cache to s3://guanaco-reward-models/undi95-meta-llama-3-1-8b-1733-v2_reward
undi95-meta-llama-3-1-8b-1733-v2-mkmlizer: cp /tmp/reward_cache/config.json s3://guanaco-reward-models/undi95-meta-llama-3-1-8b-1733-v2_reward/config.json
undi95-meta-llama-3-1-8b-1733-v2-mkmlizer: cp /tmp/reward_cache/tokenizer_config.json s3://guanaco-reward-models/undi95-meta-llama-3-1-8b-1733-v2_reward/tokenizer_config.json
undi95-meta-llama-3-1-8b-1733-v2-mkmlizer: cp /tmp/reward_cache/special_tokens_map.json s3://guanaco-reward-models/undi95-meta-llama-3-1-8b-1733-v2_reward/special_tokens_map.json
undi95-meta-llama-3-1-8b-1733-v2-mkmlizer: cp /tmp/reward_cache/merges.txt s3://guanaco-reward-models/undi95-meta-llama-3-1-8b-1733-v2_reward/merges.txt
undi95-meta-llama-3-1-8b-1733-v2-mkmlizer: cp /tmp/reward_cache/vocab.json s3://guanaco-reward-models/undi95-meta-llama-3-1-8b-1733-v2_reward/vocab.json
undi95-meta-llama-3-1-8b-1733-v2-mkmlizer: cp /tmp/reward_cache/tokenizer.json s3://guanaco-reward-models/undi95-meta-llama-3-1-8b-1733-v2_reward/tokenizer.json
undi95-meta-llama-3-1-8b-1733-v2-mkmlizer: cp /tmp/reward_cache/reward.tensors s3://guanaco-reward-models/undi95-meta-llama-3-1-8b-1733-v2_reward/reward.tensors
Job undi95-meta-llama-3-1-8b-1733-v2-mkmlizer completed after 115.82s with status: succeeded
Stopping job with name undi95-meta-llama-3-1-8b-1733-v2-mkmlizer
Pipeline stage MKMLizer completed in 117.06s
Running pipeline stage MKMLKubeTemplater
Pipeline stage MKMLKubeTemplater completed in 0.10s
Running pipeline stage ISVCDeployer
Creating inference service undi95-meta-llama-3-1-8b-1733-v2
Waiting for inference service undi95-meta-llama-3-1-8b-1733-v2 to be ready
Inference service undi95-meta-llama-3-1-8b-1733-v2 ready after 90.57764291763306s
Pipeline stage ISVCDeployer completed in 92.49s
Running pipeline stage StressChecker
Received healthy response to inference request in 2.089421033859253s
Received healthy response to inference request in 1.2639129161834717s
Received healthy response to inference request in 1.24735689163208s
Received healthy response to inference request in 1.214721441268921s
Received healthy response to inference request in 1.2356531620025635s
5 requests
0 failed requests
5th percentile: 1.2189077854156494
10th percentile: 1.2230941295623778
20th percentile: 1.231466817855835
30th percentile: 1.2379939079284668
40th percentile: 1.2426753997802735
50th percentile: 1.24735689163208
60th percentile: 1.2539793014526368
70th percentile: 1.2606017112731933
80th percentile: 1.429014539718628
90th percentile: 1.7592177867889405
95th percentile: 1.9243194103240966
99th percentile: 2.056400709152222
mean time: 1.410213088989258
Pipeline stage StressChecker completed in 7.72s
undi95-meta-llama-3-1-8b_1733_v2 status is now deployed due to DeploymentManager action
undi95-meta-llama-3-1-8b_1733_v2 status is now inactive due to auto deactivation removed underperforming models
admin requested tearing down of undi95-meta-llama-3-1-8b_1733_v2
Running pipeline stage ISVCDeleter
Checking if service undi95-meta-llama-3-1-8b-1733-v2 is running
Tearing down inference service undi95-meta-llama-3-1-8b-1733-v2
Service undi95-meta-llama-3-1-8b-1733-v2 has been torndown
Pipeline stage ISVCDeleter completed in 4.02s
Running pipeline stage MKMLModelDeleter
Cleaning model data from S3
Cleaning model data from model cache
Deleting key undi95-meta-llama-3-1-8b-1733-v2/config.json from bucket guanaco-mkml-models
Deleting key undi95-meta-llama-3-1-8b-1733-v2/flywheel_model.0.safetensors from bucket guanaco-mkml-models
Deleting key undi95-meta-llama-3-1-8b-1733-v2/special_tokens_map.json from bucket guanaco-mkml-models
Deleting key undi95-meta-llama-3-1-8b-1733-v2/tokenizer.json from bucket guanaco-mkml-models
Deleting key undi95-meta-llama-3-1-8b-1733-v2/tokenizer_config.json from bucket guanaco-mkml-models
Cleaning model data from model cache
Deleting key undi95-meta-llama-3-1-8b-1733-v2_reward/config.json from bucket guanaco-reward-models
Deleting key undi95-meta-llama-3-1-8b-1733-v2_reward/merges.txt from bucket guanaco-reward-models
Deleting key undi95-meta-llama-3-1-8b-1733-v2_reward/reward.tensors from bucket guanaco-reward-models
Deleting key undi95-meta-llama-3-1-8b-1733-v2_reward/special_tokens_map.json from bucket guanaco-reward-models
Deleting key undi95-meta-llama-3-1-8b-1733-v2_reward/tokenizer.json from bucket guanaco-reward-models
Deleting key undi95-meta-llama-3-1-8b-1733-v2_reward/tokenizer_config.json from bucket guanaco-reward-models
Deleting key undi95-meta-llama-3-1-8b-1733-v2_reward/vocab.json from bucket guanaco-reward-models
Pipeline stage MKMLModelDeleter completed in 5.80s
undi95-meta-llama-3-1-8b_1733_v2 status is now torndown due to DeploymentManager action

Usage Metrics

Latency Metrics