submission_id: nousresearch-meta-llama_4939_v11
developer_uid: end_to_end_test
best_of: 4
display_name: nousresearch-meta-llama_4939_v11
family_friendly_score: 0.0
formatter: {'memory_template': "{bot_name}'s Persona: {memory}\n####\n", 'prompt_template': '{prompt}\n<START>\n', 'bot_template': '{bot_name}: {message}\n', 'user_template': '{user_name}: {message}\n', 'response_template': '{bot_name}:', 'truncate_by_message': False}
generation_params: {'temperature': 1.0, 'top_p': 0.99, 'min_p': 0.1, 'top_k': 40, 'presence_penalty': 0.0, 'frequency_penalty': 0.0, 'stopping_words': ['\n'], 'max_input_tokens': 512, 'best_of': 4, 'max_output_tokens': 64}
ineligible_reason: model is only for e2e test
is_internal_developer: True
language_model: NousResearch/Meta-Llama-3.1-8B-Instruct
max_input_tokens: 512
max_output_tokens: 64
model_architecture: LlamaForCausalLM
model_group: NousResearch/Meta-Llama-
model_name: nousresearch-meta-llama_4939_v11
model_num_parameters: 8030261248.0
model_repo: NousResearch/Meta-Llama-3.1-8B-Instruct
model_size: 8B
num_battles: 317
num_wins: 137
ranking_group: single
status: torndown
submission_type: basic
timestamp: 2024-08-29T05:07:28+00:00
us_pacific_date: 2024-08-28
win_ratio: 0.43217665615141954
Download Preference Data
Resubmit model
Running pipeline stage MKMLizer
Deleting key nousresearch-meta-llama-4939-v10/flywheel_model.0.safetensors from bucket guanaco-mkml-models
Starting job with name nousresearch-meta-llama-4939-v11-mkmlizer
Waiting for job on nousresearch-meta-llama-4939-v11-mkmlizer to finish
Deleting key nousresearch-meta-llama-4939-v10/special_tokens_map.json from bucket guanaco-mkml-models
Deleting key nousresearch-meta-llama-4939-v10/tokenizer.json from bucket guanaco-mkml-models
Deleting key nousresearch-meta-llama-4939-v10/tokenizer_config.json from bucket guanaco-mkml-models
Pipeline stage MKMLModelDeleter completed in 15.72s
nousresearch-meta-llama_4939_v10 status is now torndown due to DeploymentManager action
nousresearch-meta-llama-4939-v11-mkmlizer: ╔═════════════════════════════════════════════════════════════════════╗
nousresearch-meta-llama-4939-v11-mkmlizer: ║     _____            __           __                                ║
nousresearch-meta-llama-4939-v11-mkmlizer: ║    / _/ /_ ___    __/ /  ___ ___ / /                                ║
nousresearch-meta-llama-4939-v11-mkmlizer: ║   / _/ / // / |/|/ / _ \/ -_) -_) /                                 ║
nousresearch-meta-llama-4939-v11-mkmlizer: ║  /_//_/\_, /|__,__/_//_/\__/\__/_/                                  ║
nousresearch-meta-llama-4939-v11-mkmlizer: ║       /___/                                                         ║
nousresearch-meta-llama-4939-v11-mkmlizer: ║                                                                     ║
nousresearch-meta-llama-4939-v11-mkmlizer: ║  Version: 0.10.1                                                    ║
nousresearch-meta-llama-4939-v11-mkmlizer: ║  Copyright 2023 MK ONE TECHNOLOGIES Inc.                            ║
nousresearch-meta-llama-4939-v11-mkmlizer: ║  https://mk1.ai                                                     ║
nousresearch-meta-llama-4939-v11-mkmlizer: ║                                                                     ║
nousresearch-meta-llama-4939-v11-mkmlizer: ║  The license key for the current software has been verified as      ║
nousresearch-meta-llama-4939-v11-mkmlizer: ║  belonging to:                                                      ║
nousresearch-meta-llama-4939-v11-mkmlizer: ║                                                                     ║
nousresearch-meta-llama-4939-v11-mkmlizer: ║  Chai Research Corp.                                                ║
nousresearch-meta-llama-4939-v11-mkmlizer: ║  Account ID: 7997a29f-0ceb-4cc7-9adf-840c57b4ae6f                   ║
nousresearch-meta-llama-4939-v11-mkmlizer: ║  Expiration: 2024-10-15 23:59:59                                    ║
nousresearch-meta-llama-4939-v11-mkmlizer: ║                                                                     ║
nousresearch-meta-llama-4939-v11-mkmlizer: ╚═════════════════════════════════════════════════════════════════════╝
nousresearch-meta-llama-4939-v11-mkmlizer: quantized model in 26.237s
nousresearch-meta-llama-4939-v11-mkmlizer: Processed model NousResearch/Meta-Llama-3.1-8B-Instruct in 75.169s
nousresearch-meta-llama-4939-v11-mkmlizer: creating bucket guanaco-mkml-models
nousresearch-meta-llama-4939-v11-mkmlizer: Bucket 's3://guanaco-mkml-models/' created
nousresearch-meta-llama-4939-v11-mkmlizer: uploading /dev/shm/model_cache to s3://guanaco-mkml-models/nousresearch-meta-llama-4939-v11
nousresearch-meta-llama-4939-v11-mkmlizer: cp /dev/shm/model_cache/config.json s3://guanaco-mkml-models/nousresearch-meta-llama-4939-v11/config.json
nousresearch-meta-llama-4939-v11-mkmlizer: cp /dev/shm/model_cache/special_tokens_map.json s3://guanaco-mkml-models/nousresearch-meta-llama-4939-v11/special_tokens_map.json
nousresearch-meta-llama-4939-v11-mkmlizer: cp /dev/shm/model_cache/tokenizer_config.json s3://guanaco-mkml-models/nousresearch-meta-llama-4939-v11/tokenizer_config.json
nousresearch-meta-llama-4939-v11-mkmlizer: cp /dev/shm/model_cache/tokenizer.json s3://guanaco-mkml-models/nousresearch-meta-llama-4939-v11/tokenizer.json
nousresearch-meta-llama-4939-v11-mkmlizer: cp /dev/shm/model_cache/flywheel_model.0.safetensors s3://guanaco-mkml-models/nousresearch-meta-llama-4939-v11/flywheel_model.0.safetensors
nousresearch-meta-llama-4939-v11-mkmlizer: 
Loading 0:   0%|          | 0/291 [00:00<?, ?it/s]
Loading 0:   2%|▏         | 5/291 [00:00<00:07, 36.46it/s]
Loading 0:   5%|▍         | 14/291 [00:00<00:06, 44.84it/s]
Loading 0:   8%|▊         | 22/291 [00:00<00:04, 54.75it/s]
Loading 0:  10%|▉         | 28/291 [00:00<00:05, 51.01it/s]
Loading 0:  12%|█▏        | 34/291 [00:00<00:05, 51.19it/s]
Loading 0:  14%|█▎        | 40/291 [00:00<00:04, 52.55it/s]
Loading 0:  16%|█▌        | 46/291 [00:00<00:04, 50.34it/s]
Loading 0:  18%|█▊        | 53/291 [00:01<00:04, 55.21it/s]
Loading 0:  20%|██        | 59/291 [00:01<00:04, 46.41it/s]
Loading 0:  23%|██▎       | 67/291 [00:01<00:04, 54.47it/s]
Loading 0:  25%|██▌       | 73/291 [00:01<00:04, 51.05it/s]
Loading 0:  27%|██▋       | 79/291 [00:01<00:04, 50.01it/s]
Loading 0:  29%|██▉       | 85/291 [00:01<00:05, 38.31it/s]
Loading 0:  31%|███▏      | 91/291 [00:01<00:05, 39.51it/s]
Loading 0:  33%|███▎      | 96/291 [00:02<00:04, 39.43it/s]
Loading 0:  35%|███▌      | 103/291 [00:02<00:04, 45.47it/s]
Loading 0:  37%|███▋      | 109/291 [00:02<00:03, 45.87it/s]
Loading 0:  39%|███▉      | 114/291 [00:02<00:03, 45.38it/s]
Loading 0:  41%|████      | 120/291 [00:02<00:03, 47.64it/s]
Loading 0:  43%|████▎     | 125/291 [00:02<00:03, 46.48it/s]
Loading 0:  45%|████▌     | 131/291 [00:02<00:03, 41.04it/s]
Loading 0:  47%|████▋     | 138/291 [00:02<00:03, 45.82it/s]
Loading 0:  49%|████▉     | 143/291 [00:03<00:03, 46.46it/s]
Loading 0:  51%|█████     | 149/291 [00:03<00:03, 42.53it/s]
Loading 0:  54%|█████▍    | 157/291 [00:03<00:02, 49.84it/s]
Loading 0:  56%|█████▌    | 163/291 [00:03<00:02, 46.91it/s]
Loading 0:  58%|█████▊    | 168/291 [00:03<00:02, 45.16it/s]
Loading 0:  59%|█████▉    | 173/291 [00:03<00:02, 44.83it/s]
Loading 0:  62%|██████▏   | 179/291 [00:03<00:02, 48.61it/s]
Loading 0:  63%|██████▎   | 184/291 [00:03<00:02, 48.22it/s]
Loading 0:  65%|██████▍   | 189/291 [00:04<00:03, 30.92it/s]
Loading 0:  67%|██████▋   | 194/291 [00:04<00:02, 32.80it/s]
Loading 0:  69%|██████▉   | 202/291 [00:04<00:02, 41.27it/s]
Loading 0:  71%|███████▏  | 208/291 [00:04<00:02, 40.81it/s]
Loading 0:  73%|███████▎  | 213/291 [00:04<00:01, 40.55it/s]
Loading 0:  75%|███████▌  | 219/291 [00:04<00:01, 44.36it/s]
Loading 0:  77%|███████▋  | 225/291 [00:04<00:01, 47.70it/s]
Loading 0:  79%|███████▉  | 231/291 [00:05<00:01, 42.42it/s]
Loading 0:  81%|████████▏ | 237/291 [00:05<00:01, 45.24it/s]
Loading 0:  83%|████████▎ | 242/291 [00:05<00:01, 46.07it/s]
Loading 0:  85%|████████▌ | 248/291 [00:05<00:01, 41.55it/s]
Loading 0:  88%|████████▊ | 256/291 [00:05<00:00, 49.44it/s]
Loading 0:  90%|█████████ | 262/291 [00:05<00:00, 47.02it/s]
Loading 0:  92%|█████████▏| 267/291 [00:05<00:00, 45.02it/s]
Loading 0:  94%|█████████▍| 273/291 [00:06<00:00, 48.07it/s]
Loading 0:  96%|█████████▌| 278/291 [00:06<00:00, 48.25it/s]
Loading 0:  97%|█████████▋| 283/291 [00:06<00:00, 42.85it/s]
Loading 0:  99%|█████████▉| 288/291 [00:11<00:00,  3.10it/s]
                                                            

Job nousresearch-meta-llama-4939-v11-mkmlizer completed after 98.55s with status: succeeded
Stopping job with name nousresearch-meta-llama-4939-v11-mkmlizer
Pipeline stage MKMLizer completed in 99.90s
Running pipeline stage MKMLKubeTemplater
Pipeline stage MKMLKubeTemplater completed in 0.39s
Running pipeline stage ISVCDeployer
Creating inference service nousresearch-meta-llama-4939-v11
Waiting for inference service nousresearch-meta-llama-4939-v11 to be ready
Inference service nousresearch-meta-llama-4939-v11 ready after 182.3978831768036s
Pipeline stage ISVCDeployer completed in 183.47s
Running pipeline stage StressChecker
Received healthy response to inference request in 2.5769948959350586s
Received healthy response to inference request in 2.0865530967712402s
Received healthy response to inference request in 1.9002721309661865s
Received healthy response to inference request in 1.848606824874878s
Received healthy response to inference request in 1.4374849796295166s
5 requests
0 failed requests
5th percentile: 1.5197093486785889
10th percentile: 1.6019337177276611
20th percentile: 1.7663824558258057
30th percentile: 1.8589398860931396
40th percentile: 1.879606008529663
50th percentile: 1.9002721309661865
60th percentile: 1.974784517288208
70th percentile: 2.0492969036102293
80th percentile: 2.184641456604004
90th percentile: 2.380818176269531
95th percentile: 2.478906536102295
99th percentile: 2.5573772239685058
mean time: 1.969982385635376
Pipeline stage StressChecker completed in 11.99s
nousresearch-meta-llama_4939_v11 status is now deployed due to DeploymentManager action
nousresearch-meta-llama_4939_v11 status is now inactive due to admin request
admin requested tearing down of nousresearch-meta-llama_4939_v11
pipeline stage ISVCDeleter: starting
pipeline stage ISVCDeleter: trying
Checking if service nousresearch-meta-llama-4939-v11 is running
Tearing down inference service nousresearch-meta-llama-4939-v11
Service nousresearch-meta-llama-4939-v11 has been torndown
pipeline stage ISVCDeleter: completed in 9.36s
pipeline stage MKMLModelDeleter: starting
pipeline stage MKMLModelDeleter: trying
Cleaning model data from S3
Cleaning model data from model cache
Deleting key nousresearch-meta-llama-4939-v11/config.json from bucket guanaco-mkml-models
Deleting key nousresearch-meta-llama-4939-v11/flywheel_model.0.safetensors from bucket guanaco-mkml-models
pipeline stage MKMLizer: starting
pipeline stage MKMLizer: trying
Starting job with name nousresearch-meta-llama-4939-v12-mkmlizer
Deleting key nousresearch-meta-llama-4939-v11/special_tokens_map.json from bucket guanaco-mkml-models
Deleting key nousresearch-meta-llama-4939-v11/tokenizer.json from bucket guanaco-mkml-models
Waiting for job on nousresearch-meta-llama-4939-v12-mkmlizer to finish
Deleting key nousresearch-meta-llama-4939-v11/tokenizer_config.json from bucket guanaco-mkml-models
pipeline stage MKMLModelDeleter: completed in 7.84s
nousresearch-meta-llama_4939_v11 status is now torndown due to DeploymentManager action