submission_id: nousresearch-meta-llama_4941_v97
developer_uid: chai_backend_admin
alignment_samples: 0
best_of: 16
celo_rating: 1188.55
display_name: nousresearch-meta-llama_4941_v97
formatter: {'memory_template': "{bot_name}'s Persona: {memory}\n####\n", 'prompt_template': '{prompt}\n<START>\n', 'bot_template': '{bot_name}: {message}\n', 'user_template': '{user_name}: {message}\n', 'response_template': '{bot_name}:', 'truncate_by_message': False}
generation_params: {'temperature': 1.0, 'top_p': 1.0, 'min_p': 0.0, 'top_k': 40, 'presence_penalty': 0.0, 'frequency_penalty': 0.0, 'stopping_words': ['\n'], 'max_input_tokens': 512, 'best_of': 16, 'max_output_tokens': 64, 'reward_max_token_input': 256}
is_internal_developer: True
language_model: NousResearch/Meta-Llama-3-8B-Instruct
max_input_tokens: 512
max_output_tokens: 64
model_architecture: LlamaForCausalLM
model_group: NousResearch/Meta-Llama-
model_name: nousresearch-meta-llama_4941_v97
model_num_parameters: 8030261248.0
model_repo: NousResearch/Meta-Llama-3-8B-Instruct
model_size: 8B
num_battles: 10487
num_wins: 5078
propriety_score: 0.6967471143756558
propriety_total_count: 953.0
ranking_group: single
reward_formatter: {'bot_template': '{bot_name}: {message}\n', 'memory_template': "{bot_name}'s Persona: {memory}\n####\n", 'prompt_template': '{prompt}\n<START>\n', 'response_template': '{bot_name}:', 'truncate_by_message': False, 'user_template': '{user_name}: {message}\n'}
reward_repo: ChaiML/gpt2_xl_pairwise_89m_step_347634
status: torndown
submission_type: basic
timestamp: 2024-08-02T04:07:22+00:00
us_pacific_date: 2024-08-01
win_ratio: 0.4842185563078097
Download Preference Data
Resubmit model
Running pipeline stage MKMLizer
Starting job with name nousresearch-meta-llama-4941-v97-mkmlizer
Waiting for job on nousresearch-meta-llama-4941-v97-mkmlizer to finish
nousresearch-meta-llama-4941-v97-mkmlizer: ╔═════════════════════════════════════════════════════════════════════╗
nousresearch-meta-llama-4941-v97-mkmlizer: ║ _____ __ __ ║
nousresearch-meta-llama-4941-v97-mkmlizer: ║ / _/ /_ ___ __/ / ___ ___ / / ║
nousresearch-meta-llama-4941-v97-mkmlizer: ║ / _/ / // / |/|/ / _ \/ -_) -_) / ║
nousresearch-meta-llama-4941-v97-mkmlizer: ║ /_//_/\_, /|__,__/_//_/\__/\__/_/ ║
nousresearch-meta-llama-4941-v97-mkmlizer: ║ /___/ ║
nousresearch-meta-llama-4941-v97-mkmlizer: ║ ║
nousresearch-meta-llama-4941-v97-mkmlizer: ║ Version: 0.9.9 ║
nousresearch-meta-llama-4941-v97-mkmlizer: ║ Copyright 2023 MK ONE TECHNOLOGIES Inc. ║
nousresearch-meta-llama-4941-v97-mkmlizer: ║ https://mk1.ai ║
nousresearch-meta-llama-4941-v97-mkmlizer: ║ ║
nousresearch-meta-llama-4941-v97-mkmlizer: ║ The license key for the current software has been verified as ║
nousresearch-meta-llama-4941-v97-mkmlizer: ║ belonging to: ║
nousresearch-meta-llama-4941-v97-mkmlizer: ║ ║
nousresearch-meta-llama-4941-v97-mkmlizer: ║ Chai Research Corp. ║
nousresearch-meta-llama-4941-v97-mkmlizer: ║ Account ID: 7997a29f-0ceb-4cc7-9adf-840c57b4ae6f ║
nousresearch-meta-llama-4941-v97-mkmlizer: ║ Expiration: 2024-10-15 23:59:59 ║
nousresearch-meta-llama-4941-v97-mkmlizer: ║ ║
nousresearch-meta-llama-4941-v97-mkmlizer: ╚═════════════════════════════════════════════════════════════════════╝
nousresearch-meta-llama-4941-v97-mkmlizer: Downloaded to shared memory in 21.419s
nousresearch-meta-llama-4941-v97-mkmlizer: quantizing model to /dev/shm/model_cache, profile:s0, folder:/tmp/tmp7z96uzo6, device:0
nousresearch-meta-llama-4941-v97-mkmlizer: Saving flywheel model at /dev/shm/model_cache
nousresearch-meta-llama-4941-v97-mkmlizer: quantized model in 26.021s
nousresearch-meta-llama-4941-v97-mkmlizer: Processed model NousResearch/Meta-Llama-3-8B-Instruct in 47.440s
nousresearch-meta-llama-4941-v97-mkmlizer: creating bucket guanaco-mkml-models
nousresearch-meta-llama-4941-v97-mkmlizer: Bucket 's3://guanaco-mkml-models/' created
nousresearch-meta-llama-4941-v97-mkmlizer: uploading /dev/shm/model_cache to s3://guanaco-mkml-models/nousresearch-meta-llama-4941-v97
nousresearch-meta-llama-4941-v97-mkmlizer: cp /dev/shm/model_cache/special_tokens_map.json s3://guanaco-mkml-models/nousresearch-meta-llama-4941-v97/special_tokens_map.json
nousresearch-meta-llama-4941-v97-mkmlizer: cp /dev/shm/model_cache/config.json s3://guanaco-mkml-models/nousresearch-meta-llama-4941-v97/config.json
nousresearch-meta-llama-4941-v97-mkmlizer: cp /dev/shm/model_cache/tokenizer_config.json s3://guanaco-mkml-models/nousresearch-meta-llama-4941-v97/tokenizer_config.json
nousresearch-meta-llama-4941-v97-mkmlizer: cp /dev/shm/model_cache/tokenizer.json s3://guanaco-mkml-models/nousresearch-meta-llama-4941-v97/tokenizer.json
nousresearch-meta-llama-4941-v97-mkmlizer: cp /dev/shm/model_cache/flywheel_model.0.safetensors s3://guanaco-mkml-models/nousresearch-meta-llama-4941-v97/flywheel_model.0.safetensors
nousresearch-meta-llama-4941-v97-mkmlizer: loading reward model from ChaiML/gpt2_xl_pairwise_89m_step_347634
nousresearch-meta-llama-4941-v97-mkmlizer: Loading 0: 0%| | 0/291 [00:00<?, ?it/s] Loading 0: 2%|▏ | 5/291 [00:00<00:08, 34.92it/s] Loading 0: 4%|▍ | 13/291 [00:00<00:04, 56.17it/s] Loading 0: 7%|▋ | 19/291 [00:00<00:05, 51.08it/s] Loading 0: 9%|▊ | 25/291 [00:00<00:04, 53.67it/s] Loading 0: 11%|█ | 32/291 [00:00<00:05, 49.28it/s] Loading 0: 14%|█▍ | 41/291 [00:00<00:04, 51.73it/s] Loading 0: 17%|█▋ | 50/291 [00:00<00:04, 52.46it/s] Loading 0: 20%|█▉ | 58/291 [00:01<00:03, 58.92it/s] Loading 0: 22%|██▏ | 65/291 [00:01<00:03, 56.96it/s] Loading 0: 24%|██▍ | 71/291 [00:01<00:03, 55.50it/s] Loading 0: 26%|██▋ | 77/291 [00:01<00:04, 48.52it/s] Loading 0: 29%|██▊ | 83/291 [00:01<00:05, 37.73it/s] Loading 0: 30%|███ | 88/291 [00:01<00:05, 38.33it/s] Loading 0: 32%|███▏ | 94/291 [00:01<00:04, 42.57it/s] Loading 0: 34%|███▍ | 100/291 [00:02<00:04, 42.02it/s] Loading 0: 36%|███▌ | 105/291 [00:02<00:04, 43.35it/s] Loading 0: 38%|███▊ | 112/291 [00:02<00:03, 49.31it/s] Loading 0: 41%|████ | 118/291 [00:02<00:03, 47.94it/s] Loading 0: 42%|████▏ | 123/291 [00:02<00:03, 47.52it/s] Loading 0: 45%|████▍ | 130/291 [00:02<00:03, 51.80it/s] Loading 0: 47%|████▋ | 136/291 [00:02<00:03, 48.95it/s] Loading 0: 49%|████▉ | 142/291 [00:02<00:03, 49.01it/s] Loading 0: 51%|█████ | 148/291 [00:03<00:02, 51.80it/s] Loading 0: 53%|█████▎ | 154/291 [00:03<00:02, 48.94it/s] Loading 0: 55%|█████▍ | 159/291 [00:03<00:02, 48.44it/s] Loading 0: 57%|█████▋ | 166/291 [00:03<00:02, 52.99it/s] Loading 0: 59%|█████▉ | 172/291 [00:03<00:02, 49.28it/s] Loading 0: 62%|██████▏ | 179/291 [00:03<00:02, 52.48it/s] Loading 0: 64%|██████▎ | 185/291 [00:03<00:01, 54.05it/s] Loading 0: 66%|██████▌ | 191/291 [00:04<00:02, 34.15it/s] Loading 0: 67%|██████▋ | 196/291 [00:04<00:02, 36.34it/s] Loading 0: 70%|██████▉ | 203/291 [00:04<00:02, 37.89it/s] Loading 0: 73%|███████▎ | 211/291 [00:04<00:01, 45.69it/s] Loading 0: 75%|███████▍ | 217/291 [00:04<00:01, 45.12it/s] Loading 0: 76%|███████▋ | 222/291 [00:04<00:01, 45.41it/s] Loading 0: 79%|███████▊ | 229/291 [00:04<00:01, 50.26it/s] Loading 0: 81%|████████ | 235/291 [00:04<00:01, 47.60it/s] Loading 0: 82%|████████▏ | 240/291 [00:05<00:01, 47.02it/s] Loading 0: 85%|████████▍ | 246/291 [00:05<00:00, 49.80it/s] Loading 0: 87%|████████▋ | 252/291 [00:05<00:00, 51.18it/s] Loading 0: 89%|████████▊ | 258/291 [00:05<00:00, 44.87it/s] Loading 0: 91%|█████████ | 265/291 [00:05<00:00, 50.29it/s] Loading 0: 93%|█████████▎| 271/291 [00:05<00:00, 47.43it/s] Loading 0: 95%|█████████▍| 276/291 [00:05<00:00, 46.76it/s] Loading 0: 97%|█████████▋| 282/291 [00:05<00:00, 43.70it/s] Loading 0: 99%|█████████▊| 287/291 [00:11<00:01, 3.28it/s] /opt/conda/lib/python3.10/site-packages/transformers/models/auto/configuration_auto.py:957: FutureWarning: The `use_auth_token` argument is deprecated and will be removed in v5 of Transformers. Please use `token` instead.
nousresearch-meta-llama-4941-v97-mkmlizer: warnings.warn(
nousresearch-meta-llama-4941-v97-mkmlizer: Downloading shards: 0%| | 0/2 [00:00<?, ?it/s] Downloading shards: 50%|█████ | 1/2 [00:05<00:05, 5.22s/it] Downloading shards: 100%|██████████| 2/2 [00:08<00:00, 3.86s/it] Downloading shards: 100%|██████████| 2/2 [00:08<00:00, 4.06s/it]
nousresearch-meta-llama-4941-v97-mkmlizer: Loading checkpoint shards: 0%| | 0/2 [00:00<?, ?it/s] Loading checkpoint shards: 50%|█████ | 1/2 [00:00<00:00, 2.37it/s] Loading checkpoint shards: 100%|██████████| 2/2 [00:00<00:00, 3.92it/s] Loading checkpoint shards: 100%|██████████| 2/2 [00:00<00:00, 3.57it/s]
nousresearch-meta-llama-4941-v97-mkmlizer: Saving model to /tmp/reward_cache/reward.tensors
nousresearch-meta-llama-4941-v97-mkmlizer: Bucket 's3://guanaco-reward-models/' created
nousresearch-meta-llama-4941-v97-mkmlizer: uploading /tmp/reward_cache to s3://guanaco-reward-models/nousresearch-meta-llama-4941-v97_reward
nousresearch-meta-llama-4941-v97-mkmlizer: cp /tmp/reward_cache/config.json s3://guanaco-reward-models/nousresearch-meta-llama-4941-v97_reward/config.json
nousresearch-meta-llama-4941-v97-mkmlizer: cp /tmp/reward_cache/special_tokens_map.json s3://guanaco-reward-models/nousresearch-meta-llama-4941-v97_reward/special_tokens_map.json
nousresearch-meta-llama-4941-v97-mkmlizer: cp /tmp/reward_cache/tokenizer_config.json s3://guanaco-reward-models/nousresearch-meta-llama-4941-v97_reward/tokenizer_config.json
nousresearch-meta-llama-4941-v97-mkmlizer: cp /tmp/reward_cache/merges.txt s3://guanaco-reward-models/nousresearch-meta-llama-4941-v97_reward/merges.txt
nousresearch-meta-llama-4941-v97-mkmlizer: cp /tmp/reward_cache/vocab.json s3://guanaco-reward-models/nousresearch-meta-llama-4941-v97_reward/vocab.json
nousresearch-meta-llama-4941-v97-mkmlizer: cp /tmp/reward_cache/tokenizer.json s3://guanaco-reward-models/nousresearch-meta-llama-4941-v97_reward/tokenizer.json
nousresearch-meta-llama-4941-v97-mkmlizer: cp /tmp/reward_cache/reward.tensors s3://guanaco-reward-models/nousresearch-meta-llama-4941-v97_reward/reward.tensors
Job nousresearch-meta-llama-4941-v97-mkmlizer completed after 92.41s with status: succeeded
Stopping job with name nousresearch-meta-llama-4941-v97-mkmlizer
Pipeline stage MKMLizer completed in 94.33s
Running pipeline stage MKMLKubeTemplater
Pipeline stage MKMLKubeTemplater completed in 0.43s
Running pipeline stage ISVCDeployer
Creating inference service nousresearch-meta-llama-4941-v97
Waiting for inference service nousresearch-meta-llama-4941-v97 to be ready
Inference service nousresearch-meta-llama-4941-v97 ready after 142.62434196472168s
Pipeline stage ISVCDeployer completed in 144.29s
Running pipeline stage StressChecker
HTTP Request: %s %s "%s %d %s"
Received healthy response to inference request in 9.589207649230957s
HTTP Request: %s %s "%s %d %s"
Received healthy response to inference request in 2.6322548389434814s
HTTP Request: %s %s "%s %d %s"
Received healthy response to inference request in 2.528519868850708s
HTTP Request: %s %s "%s %d %s"
Received healthy response to inference request in 2.2350690364837646s
HTTP Request: %s %s "%s %d %s"
Received healthy response to inference request in 2.435889720916748s
5 requests
0 failed requests
5th percentile: 2.2752331733703612
10th percentile: 2.315397310256958
20th percentile: 2.3957255840301515
30th percentile: 2.45441575050354
40th percentile: 2.491467809677124
50th percentile: 2.528519868850708
60th percentile: 2.5700138568878175
70th percentile: 2.611507844924927
80th percentile: 4.023645401000978
90th percentile: 6.806426525115967
95th percentile: 8.19781708717346
99th percentile: 9.310929536819458
mean time: 3.8841882228851317
Pipeline stage StressChecker completed in 23.18s
nousresearch-meta-llama_4941_v97 status is now deployed due to DeploymentManager action
nousresearch-meta-llama_4941_v97 status is now inactive due to auto deactivation removed underperforming models
admin requested tearing down of nousresearch-meta-llama_4941_v97
Running pipeline stage ISVCDeleter
Checking if service nousresearch-meta-llama-4941-v97 is running
Tearing down inference service nousresearch-meta-llama-4941-v97
Service nousresearch-meta-llama-4941-v97 has been torndown
Pipeline stage ISVCDeleter completed in 4.61s
Running pipeline stage MKMLModelDeleter
Cleaning model data from S3
Cleaning model data from model cache
Deleting key nousresearch-meta-llama-4941-v97/config.json from bucket guanaco-mkml-models
Deleting key nousresearch-meta-llama-4941-v97/flywheel_model.0.safetensors from bucket guanaco-mkml-models
Deleting key nousresearch-meta-llama-4941-v97/special_tokens_map.json from bucket guanaco-mkml-models
Deleting key nousresearch-meta-llama-4941-v97/tokenizer.json from bucket guanaco-mkml-models
Deleting key nousresearch-meta-llama-4941-v97/tokenizer_config.json from bucket guanaco-mkml-models
Cleaning model data from model cache
Deleting key nousresearch-meta-llama-4941-v97_reward/config.json from bucket guanaco-reward-models
Deleting key nousresearch-meta-llama-4941-v97_reward/merges.txt from bucket guanaco-reward-models
Deleting key nousresearch-meta-llama-4941-v97_reward/reward.tensors from bucket guanaco-reward-models
Deleting key nousresearch-meta-llama-4941-v97_reward/special_tokens_map.json from bucket guanaco-reward-models
Deleting key nousresearch-meta-llama-4941-v97_reward/tokenizer.json from bucket guanaco-reward-models
Deleting key nousresearch-meta-llama-4941-v97_reward/tokenizer_config.json from bucket guanaco-reward-models
Deleting key nousresearch-meta-llama-4941-v97_reward/vocab.json from bucket guanaco-reward-models
Pipeline stage MKMLModelDeleter completed in 5.78s
nousresearch-meta-llama_4941_v97 status is now torndown due to DeploymentManager action

Usage Metrics

Latency Metrics