submission_id: nousresearch-meta-llama_4939_v15
developer_uid: end_to_end_test
best_of: 4
celo_rating: 1185.74
display_name: nousresearch-meta-llama_4939_v15
family_friendly_score: 0.0
formatter: {'memory_template': "{bot_name}'s Persona: {memory}\n####\n", 'prompt_template': '{prompt}\n<START>\n', 'bot_template': '{bot_name}: {message}\n', 'user_template': '{user_name}: {message}\n', 'response_template': '{bot_name}:', 'truncate_by_message': False}
generation_params: {'temperature': 1.0, 'top_p': 0.99, 'min_p': 0.1, 'top_k': 40, 'presence_penalty': 0.0, 'frequency_penalty': 0.0, 'stopping_words': ['\n'], 'max_input_tokens': 512, 'best_of': 4, 'max_output_tokens': 64}
ineligible_reason: model is only for e2e test
is_internal_developer: True
language_model: NousResearch/Meta-Llama-3.1-8B-Instruct
max_input_tokens: 512
max_output_tokens: 64
model_architecture: LlamaForCausalLM
model_group: NousResearch/Meta-Llama-
model_name: nousresearch-meta-llama_4939_v15
model_num_parameters: 8030261248.0
model_repo: NousResearch/Meta-Llama-3.1-8B-Instruct
model_size: 8B
num_battles: 1005
num_wins: 430
ranking_group: single
status: torndown
submission_type: basic
timestamp: 2024-08-29T05:41:48+00:00
us_pacific_date: 2024-08-28
win_ratio: 0.42786069651741293
Download Preference Data
Resubmit model
pipeline stage MKMLizer: starting
pipeline stage MKMLizer:  - attemping
Starting job with name nousresearch-meta-llama-4939-v15-mkmlizer
Waiting for job on nousresearch-meta-llama-4939-v15-mkmlizer to finish
nousresearch-meta-llama-4939-v15-mkmlizer: ╔═════════════════════════════════════════════════════════════════════╗
nousresearch-meta-llama-4939-v15-mkmlizer: ║     _____            __           __                                ║
nousresearch-meta-llama-4939-v15-mkmlizer: ║    / _/ /_ ___    __/ /  ___ ___ / /                                ║
nousresearch-meta-llama-4939-v15-mkmlizer: ║   / _/ / // / |/|/ / _ \/ -_) -_) /                                 ║
nousresearch-meta-llama-4939-v15-mkmlizer: ║  /_//_/\_, /|__,__/_//_/\__/\__/_/                                  ║
nousresearch-meta-llama-4939-v15-mkmlizer: ║       /___/                                                         ║
nousresearch-meta-llama-4939-v15-mkmlizer: ║                                                                     ║
nousresearch-meta-llama-4939-v15-mkmlizer: ║  Version: 0.10.1                                                    ║
nousresearch-meta-llama-4939-v15-mkmlizer: ║  Copyright 2023 MK ONE TECHNOLOGIES Inc.                            ║
nousresearch-meta-llama-4939-v15-mkmlizer: ║  https://mk1.ai                                                     ║
nousresearch-meta-llama-4939-v15-mkmlizer: ║                                                                     ║
nousresearch-meta-llama-4939-v15-mkmlizer: ║  The license key for the current software has been verified as      ║
nousresearch-meta-llama-4939-v15-mkmlizer: ║  belonging to:                                                      ║
nousresearch-meta-llama-4939-v15-mkmlizer: ║                                                                     ║
nousresearch-meta-llama-4939-v15-mkmlizer: ║  Chai Research Corp.                                                ║
nousresearch-meta-llama-4939-v15-mkmlizer: ║  Account ID: 7997a29f-0ceb-4cc7-9adf-840c57b4ae6f                   ║
nousresearch-meta-llama-4939-v15-mkmlizer: ║  Expiration: 2024-10-15 23:59:59                                    ║
nousresearch-meta-llama-4939-v15-mkmlizer: ║                                                                     ║
nousresearch-meta-llama-4939-v15-mkmlizer: ╚═════════════════════════════════════════════════════════════════════╝
nousresearch-meta-llama-4939-v15-mkmlizer: Downloaded to shared memory in 40.354s
nousresearch-meta-llama-4939-v15-mkmlizer: quantizing model to /dev/shm/model_cache, profile:s0, folder:/tmp/tmpqdzhcd8l, device:0
nousresearch-meta-llama-4939-v15-mkmlizer: Saving flywheel model at /dev/shm/model_cache
nousresearch-meta-llama-4939-v15-mkmlizer: quantized model in 26.360s
nousresearch-meta-llama-4939-v15-mkmlizer: Processed model NousResearch/Meta-Llama-3.1-8B-Instruct in 66.715s
nousresearch-meta-llama-4939-v15-mkmlizer: creating bucket guanaco-mkml-models
nousresearch-meta-llama-4939-v15-mkmlizer: Bucket 's3://guanaco-mkml-models/' created
nousresearch-meta-llama-4939-v15-mkmlizer: uploading /dev/shm/model_cache to s3://guanaco-mkml-models/nousresearch-meta-llama-4939-v15
nousresearch-meta-llama-4939-v15-mkmlizer: cp /dev/shm/model_cache/config.json s3://guanaco-mkml-models/nousresearch-meta-llama-4939-v15/config.json
nousresearch-meta-llama-4939-v15-mkmlizer: cp /dev/shm/model_cache/special_tokens_map.json s3://guanaco-mkml-models/nousresearch-meta-llama-4939-v15/special_tokens_map.json
nousresearch-meta-llama-4939-v15-mkmlizer: cp /dev/shm/model_cache/tokenizer_config.json s3://guanaco-mkml-models/nousresearch-meta-llama-4939-v15/tokenizer_config.json
nousresearch-meta-llama-4939-v15-mkmlizer: cp /dev/shm/model_cache/tokenizer.json s3://guanaco-mkml-models/nousresearch-meta-llama-4939-v15/tokenizer.json
nousresearch-meta-llama-4939-v15-mkmlizer: cp /dev/shm/model_cache/flywheel_model.0.safetensors s3://guanaco-mkml-models/nousresearch-meta-llama-4939-v15/flywheel_model.0.safetensors
nousresearch-meta-llama-4939-v15-mkmlizer: 
Loading 0:   0%|          | 0/291 [00:00<?, ?it/s]
Loading 0:   2%|▏         | 5/291 [00:00<00:08, 31.79it/s]
Loading 0:   4%|▍         | 13/291 [00:00<00:05, 51.49it/s]
Loading 0:   7%|▋         | 19/291 [00:00<00:05, 46.82it/s]
Loading 0:   8%|▊         | 24/291 [00:00<00:05, 45.93it/s]
Loading 0:  11%|█         | 32/291 [00:00<00:05, 45.26it/s]
Loading 0:  14%|█▎        | 40/291 [00:00<00:04, 52.59it/s]
Loading 0:  16%|█▌        | 46/291 [00:00<00:05, 45.60it/s]
Loading 0:  18%|█▊        | 51/291 [00:01<00:05, 44.17it/s]
Loading 0:  20%|█▉        | 58/291 [00:01<00:04, 48.71it/s]
Loading 0:  22%|██▏       | 64/291 [00:01<00:04, 45.55it/s]
Loading 0:  24%|██▎       | 69/291 [00:01<00:04, 44.93it/s]
Loading 0:  26%|██▌       | 76/291 [00:01<00:04, 49.28it/s]
Loading 0:  28%|██▊       | 82/291 [00:01<00:04, 45.63it/s]
Loading 0:  30%|██▉       | 87/291 [00:02<00:06, 31.30it/s]
Loading 0:  32%|███▏      | 93/291 [00:02<00:05, 36.45it/s]
Loading 0:  34%|███▎      | 98/291 [00:02<00:04, 38.68it/s]
Loading 0:  36%|███▌      | 104/291 [00:02<00:05, 36.51it/s]
Loading 0:  38%|███▊      | 111/291 [00:02<00:04, 43.08it/s]
Loading 0:  40%|███▉      | 116/291 [00:02<00:04, 43.70it/s]
Loading 0:  42%|████▏     | 121/291 [00:02<00:03, 44.87it/s]
Loading 0:  44%|████▎     | 127/291 [00:02<00:03, 44.27it/s]
Loading 0:  45%|████▌     | 132/291 [00:03<00:03, 44.36it/s]
Loading 0:  48%|████▊     | 139/291 [00:03<00:02, 50.74it/s]
Loading 0:  50%|████▉     | 145/291 [00:03<00:03, 45.28it/s]
Loading 0:  52%|█████▏    | 150/291 [00:03<00:03, 44.83it/s]
Loading 0:  54%|█████▍    | 157/291 [00:03<00:02, 50.43it/s]
Loading 0:  56%|█████▌    | 163/291 [00:03<00:02, 45.08it/s]
Loading 0:  58%|█████▊    | 168/291 [00:03<00:02, 44.87it/s]
Loading 0:  60%|██████    | 175/291 [00:03<00:02, 50.60it/s]
Loading 0:  62%|██████▏   | 181/291 [00:04<00:02, 42.90it/s]
Loading 0:  64%|██████▍   | 187/291 [00:04<00:02, 36.45it/s]
Loading 0:  66%|██████▌   | 192/291 [00:04<00:02, 37.28it/s]
Loading 0:  68%|██████▊   | 197/291 [00:04<00:02, 39.77it/s]
Loading 0:  70%|██████▉   | 203/291 [00:04<00:02, 36.95it/s]
Loading 0:  72%|███████▏  | 210/291 [00:04<00:01, 43.81it/s]
Loading 0:  74%|███████▍  | 215/291 [00:04<00:01, 44.94it/s]
Loading 0:  76%|███████▌  | 221/291 [00:05<00:01, 42.83it/s]
Loading 0:  79%|███████▊  | 229/291 [00:05<00:01, 50.63it/s]
Loading 0:  81%|████████  | 235/291 [00:05<00:01, 49.14it/s]
Loading 0:  83%|████████▎ | 241/291 [00:05<00:01, 48.68it/s]
Loading 0:  85%|████████▍ | 247/291 [00:05<00:00, 51.43it/s]
Loading 0:  87%|████████▋ | 253/291 [00:05<00:00, 46.27it/s]
Loading 0:  89%|████████▊ | 258/291 [00:05<00:00, 45.53it/s]
Loading 0:  91%|█████████ | 265/291 [00:05<00:00, 50.54it/s]
Loading 0:  93%|█████████▎| 271/291 [00:06<00:00, 46.92it/s]
Loading 0:  95%|█████████▍| 276/291 [00:06<00:00, 44.88it/s]
Loading 0:  97%|█████████▋| 282/291 [00:06<00:00, 43.05it/s]
Loading 0:  99%|█████████▊| 287/291 [00:11<00:01,  3.25it/s]
                                                            

Job nousresearch-meta-llama-4939-v15-mkmlizer completed after 87.48s with status: succeeded
Stopping job with name nousresearch-meta-llama-4939-v15-mkmlizer
pipeline stage MKMLizer: completed in 88.75s
pipeline stage MKMLKubeTemplater: starting
pipeline stage MKMLKubeTemplater:  - attemping
pipeline stage MKMLKubeTemplater: completed in 0.40s
pipeline stage ISVCDeployer: starting
pipeline stage ISVCDeployer:  - attemping
Creating inference service nousresearch-meta-llama-4939-v15
Waiting for inference service nousresearch-meta-llama-4939-v15 to be ready
Inference service nousresearch-meta-llama-4939-v15 ready after 182.7077190876007s
pipeline stage ISVCDeployer: completed in 183.88s
pipeline stage StressChecker: starting
pipeline stage StressChecker:  - attemping
Received healthy response to inference request in 2.2089500427246094s
Received healthy response to inference request in 1.6442949771881104s
Received healthy response to inference request in 1.2211380004882812s
Received healthy response to inference request in 1.6727261543273926s
Received healthy response to inference request in 1.6641321182250977s
5 requests
0 failed requests
5th percentile: 1.305769395828247
10th percentile: 1.3904007911682128
20th percentile: 1.5596635818481446
30th percentile: 1.6482624053955077
40th percentile: 1.6561972618103027
50th percentile: 1.6641321182250977
60th percentile: 1.6675697326660157
70th percentile: 1.6710073471069335
80th percentile: 1.779970932006836
90th percentile: 1.9944604873657228
95th percentile: 2.1017052650451657
99th percentile: 2.187501087188721
mean time: 1.6822482585906982
pipeline stage StressChecker: completed in 10.79s
nousresearch-meta-llama_4939_v15 status is now deployed due to DeploymentManager action
nousresearch-meta-llama_4939_v15 status is now inactive due to admin request
admin requested tearing down of nousresearch-meta-llama_4939_v15
nousresearch-meta-llama_4939_v15 status is now torndown due to DeploymentManager action
nousresearch-meta-llama_4939_v15 status is now torndown due to DeploymentManager action