nousresearch-meta-llama_4939

submission_id: nousresearch-meta-llama_4939_v59
developer_uid: end_to_end_test
best_of: 4
celo_rating: 1187.19
display_name: nousresearch-meta-llama_4939_v59
family_friendly_score: 0.0
formatter: {'memory_template': "{bot_name}'s Persona: {memory}\n####\n", 'prompt_template': '{prompt}\n<START>\n', 'bot_template': '{bot_name}: {message}\n', 'user_template': '{user_name}: {message}\n', 'response_template': '{bot_name}:', 'truncate_by_message': False}
generation_params: {'temperature': 1.0, 'top_p': 0.99, 'min_p': 0.1, 'top_k': 40, 'presence_penalty': 0.0, 'frequency_penalty': 0.0, 'stopping_words': ['\n'], 'max_input_tokens': 512, 'best_of': 4, 'max_output_tokens': 64}
ineligible_reason: model is only for e2e test
is_internal_developer: True
language_model: NousResearch/Meta-Llama-3.1-8B-Instruct
max_input_tokens: 512
max_output_tokens: 64
model_architecture: LlamaForCausalLM
model_group: NousResearch/Meta-Llama-
model_name: nousresearch-meta-llama_4939_v59
model_num_parameters: 8030261248.0
model_repo: NousResearch/Meta-Llama-3.1-8B-Instruct
model_size: 8B
num_battles: 15669
num_wins: 6728
ranking_group: single
status: torndown
submission_type: basic
timestamp: 2024-09-01T05:32:48+00:00
us_pacific_date: 2024-08-31
win_ratio: 0.4293828578722318
Download Preference Data
Resubmit model
run pipeline %s
Pipeline stage MKMLModelDeleter completed in 14.76s
nousresearch-meta-llama_4939_v58 status is now torndown due to DeploymentManager action
Running pipeline stage MKMLizer
Starting job with name nousresearch-meta-llama-4939-v59-mkmlizer
Waiting for job on nousresearch-meta-llama-4939-v59-mkmlizer to finish
nousresearch-meta-llama-4939-v59-mkmlizer: ╔═════════════════════════════════════════════════════════════════════╗
nousresearch-meta-llama-4939-v59-mkmlizer: ║     _____            __           __                                ║
nousresearch-meta-llama-4939-v59-mkmlizer: ║    / _/ /_ ___    __/ /  ___ ___ / /                                ║
nousresearch-meta-llama-4939-v59-mkmlizer: ║   / _/ / // / |/|/ / _ \/ -_) -_) /                                 ║
nousresearch-meta-llama-4939-v59-mkmlizer: ║  /_//_/\_, /|__,__/_//_/\__/\__/_/                                  ║
nousresearch-meta-llama-4939-v59-mkmlizer: ║       /___/                                                         ║
nousresearch-meta-llama-4939-v59-mkmlizer: ║                                                                     ║
nousresearch-meta-llama-4939-v59-mkmlizer: ║  Version: 0.10.1                                                    ║
nousresearch-meta-llama-4939-v59-mkmlizer: ║  Copyright 2023 MK ONE TECHNOLOGIES Inc.                            ║
nousresearch-meta-llama-4939-v59-mkmlizer: ║  https://mk1.ai                                                     ║
nousresearch-meta-llama-4939-v59-mkmlizer: ║                                                                     ║
nousresearch-meta-llama-4939-v59-mkmlizer: ║  The license key for the current software has been verified as      ║
nousresearch-meta-llama-4939-v59-mkmlizer: ║  belonging to:                                                      ║
nousresearch-meta-llama-4939-v59-mkmlizer: ║                                                                     ║
nousresearch-meta-llama-4939-v59-mkmlizer: ║  Chai Research Corp.                                                ║
nousresearch-meta-llama-4939-v59-mkmlizer: ║  Account ID: 7997a29f-0ceb-4cc7-9adf-840c57b4ae6f                   ║
nousresearch-meta-llama-4939-v59-mkmlizer: ║  Expiration: 2024-10-15 23:59:59                                    ║
nousresearch-meta-llama-4939-v59-mkmlizer: ║                                                                     ║
nousresearch-meta-llama-4939-v59-mkmlizer: ╚═════════════════════════════════════════════════════════════════════╝
nousresearch-meta-llama-4939-v59-mkmlizer: Downloaded to shared memory in 34.143s
nousresearch-meta-llama-4939-v59-mkmlizer: quantizing model to /dev/shm/model_cache, profile:s0, folder:/tmp/tmpbiurwo15, device:0
nousresearch-meta-llama-4939-v59-mkmlizer: Saving flywheel model at /dev/shm/model_cache
nousresearch-meta-llama-4939-v59-mkmlizer: quantized model in 26.320s
nousresearch-meta-llama-4939-v59-mkmlizer: Processed model NousResearch/Meta-Llama-3.1-8B-Instruct in 60.464s
nousresearch-meta-llama-4939-v59-mkmlizer: creating bucket guanaco-mkml-models
nousresearch-meta-llama-4939-v59-mkmlizer: Bucket 's3://guanaco-mkml-models/' created
nousresearch-meta-llama-4939-v59-mkmlizer: uploading /dev/shm/model_cache to s3://guanaco-mkml-models/nousresearch-meta-llama-4939-v59
nousresearch-meta-llama-4939-v59-mkmlizer: cp /dev/shm/model_cache/config.json s3://guanaco-mkml-models/nousresearch-meta-llama-4939-v59/config.json
nousresearch-meta-llama-4939-v59-mkmlizer: cp /dev/shm/model_cache/special_tokens_map.json s3://guanaco-mkml-models/nousresearch-meta-llama-4939-v59/special_tokens_map.json
nousresearch-meta-llama-4939-v59-mkmlizer: cp /dev/shm/model_cache/tokenizer_config.json s3://guanaco-mkml-models/nousresearch-meta-llama-4939-v59/tokenizer_config.json
nousresearch-meta-llama-4939-v59-mkmlizer: cp /dev/shm/model_cache/tokenizer.json s3://guanaco-mkml-models/nousresearch-meta-llama-4939-v59/tokenizer.json
nousresearch-meta-llama-4939-v59-mkmlizer: cp /dev/shm/model_cache/flywheel_model.0.safetensors s3://guanaco-mkml-models/nousresearch-meta-llama-4939-v59/flywheel_model.0.safetensors
nousresearch-meta-llama-4939-v59-mkmlizer: 
Loading 0:   0%|          | 0/291 [00:00<?, ?it/s]
Loading 0:   2%|▏         | 5/291 [00:00<00:07, 36.71it/s]
Loading 0:   4%|▍         | 13/291 [00:00<00:04, 58.38it/s]
Loading 0:   7%|▋         | 20/291 [00:00<00:05, 51.88it/s]
Loading 0:   9%|▉         | 26/291 [00:00<00:05, 51.99it/s]
Loading 0:  11%|█         | 32/291 [00:00<00:05, 44.29it/s]
Loading 0:  14%|█▎        | 40/291 [00:00<00:04, 52.70it/s]
Loading 0:  16%|█▌        | 46/291 [00:00<00:05, 48.56it/s]
Loading 0:  18%|█▊        | 52/291 [00:01<00:04, 49.58it/s]
Loading 0:  20%|█▉        | 58/291 [00:01<00:04, 51.57it/s]
Loading 0:  22%|██▏       | 64/291 [00:01<00:04, 47.64it/s]
Loading 0:  24%|██▎       | 69/291 [00:01<00:04, 46.20it/s]
Loading 0:  26%|██▌       | 75/291 [00:01<00:04, 49.66it/s]
Loading 0:  28%|██▊       | 81/291 [00:01<00:04, 51.07it/s]
Loading 0:  30%|██▉       | 87/291 [00:01<00:06, 31.31it/s]
Loading 0:  32%|███▏      | 94/291 [00:02<00:05, 37.49it/s]
Loading 0:  34%|███▍      | 100/291 [00:02<00:05, 37.32it/s]
Loading 0:  36%|███▌      | 105/291 [00:02<00:04, 38.59it/s]
Loading 0:  38%|███▊      | 111/291 [00:02<00:04, 42.79it/s]
Loading 0:  40%|███▉      | 116/291 [00:02<00:04, 43.73it/s]
Loading 0:  42%|████▏     | 121/291 [00:02<00:03, 44.88it/s]
Loading 0:  44%|████▎     | 127/291 [00:02<00:03, 42.37it/s]
Loading 0:  45%|████▌     | 132/291 [00:02<00:03, 42.29it/s]
Loading 0:  47%|████▋     | 138/291 [00:03<00:03, 46.26it/s]
Loading 0:  49%|████▉     | 143/291 [00:03<00:03, 46.27it/s]
Loading 0:  51%|█████     | 148/291 [00:03<00:03, 46.25it/s]
Loading 0:  53%|█████▎    | 154/291 [00:03<00:03, 42.23it/s]
Loading 0:  55%|█████▍    | 159/291 [00:03<00:03, 42.53it/s]
Loading 0:  57%|█████▋    | 165/291 [00:03<00:02, 46.00it/s]
Loading 0:  58%|█████▊    | 170/291 [00:03<00:02, 45.20it/s]
Loading 0:  60%|██████    | 176/291 [00:03<00:02, 48.49it/s]
Loading 0:  62%|██████▏   | 181/291 [00:04<00:02, 39.89it/s]
Loading 0:  64%|██████▍   | 187/291 [00:04<00:03, 32.75it/s]
Loading 0:  66%|██████▌   | 192/291 [00:04<00:02, 34.98it/s]
Loading 0:  67%|██████▋   | 196/291 [00:04<00:02, 35.68it/s]
Loading 0:  69%|██████▉   | 202/291 [00:04<00:02, 40.25it/s]
Loading 0:  71%|███████▏  | 208/291 [00:04<00:02, 39.23it/s]
Loading 0:  73%|███████▎  | 213/291 [00:04<00:01, 39.43it/s]
Loading 0:  75%|███████▌  | 219/291 [00:05<00:01, 44.33it/s]
Loading 0:  77%|███████▋  | 224/291 [00:05<00:01, 44.98it/s]
Loading 0:  79%|███████▊  | 229/291 [00:05<00:01, 45.69it/s]
Loading 0:  81%|████████  | 235/291 [00:05<00:01, 42.85it/s]
Loading 0:  82%|████████▏ | 240/291 [00:05<00:01, 41.58it/s]
Loading 0:  85%|████████▍ | 246/291 [00:05<00:00, 46.10it/s]
Loading 0:  86%|████████▋ | 251/291 [00:05<00:00, 46.07it/s]
Loading 0:  88%|████████▊ | 256/291 [00:05<00:00, 45.59it/s]
Loading 0:  90%|█████████ | 262/291 [00:06<00:00, 43.16it/s]
Loading 0:  92%|█████████▏| 267/291 [00:06<00:00, 42.56it/s]
Loading 0:  94%|█████████▍| 274/291 [00:06<00:00, 47.40it/s]
Loading 0:  96%|█████████▌| 280/291 [00:06<00:00, 44.08it/s]
Loading 0:  98%|█████████▊| 285/291 [00:06<00:00, 44.23it/s]
Loading 0: 100%|█████████▉| 290/291 [00:11<00:00,  3.20it/s]
                                                            
Job nousresearch-meta-llama-4939-v59-mkmlizer completed after 87.93s with status: succeeded
Stopping job with name nousresearch-meta-llama-4939-v59-mkmlizer
Pipeline stage MKMLizer completed in 88.89s
run pipeline stage %s
Running pipeline stage MKMLTemplater
Pipeline stage MKMLTemplater completed in 0.38s
run pipeline stage %s
Running pipeline stage MKMLDeployer
Creating inference service nousresearch-meta-llama-4939-v59
Waiting for inference service nousresearch-meta-llama-4939-v59 to be ready
Inference service nousresearch-meta-llama-4939-v59 ready after 192.7329227924347s
Pipeline stage MKMLDeployer completed in 193.65s
run pipeline stage %s
Running pipeline stage StressChecker
Received healthy response to inference request in 9.089202165603638s
Received healthy response to inference request in 1.693817138671875s
Received healthy response to inference request in 1.8318839073181152s
Received healthy response to inference request in 1.2293369770050049s
Received healthy response to inference request in 1.237847089767456s
5 requests
0 failed requests
5th percentile: 1.231038999557495
10th percentile: 1.2327410221099853
20th percentile: 1.2361450672149659
30th percentile: 1.32904109954834
40th percentile: 1.5114291191101075
50th percentile: 1.693817138671875
60th percentile: 1.749043846130371
70th percentile: 1.8042705535888672
80th percentile: 3.283347558975221
90th percentile: 6.186274862289429
95th percentile: 7.637738513946532
99th percentile: 8.798909435272217
mean time: 3.016417455673218
Pipeline stage StressChecker completed in 17.37s
run pipeline stage %s
Running pipeline stage TriggerMKMLProfilingPipeline
run_pipeline:run_in_cloud %s
starting trigger_guanaco_pipeline args=%s
Pipeline stage TriggerMKMLProfilingPipeline completed in 2.62s
nousresearch-meta-llama_4939_v59 status is now deployed due to DeploymentManager action
run pipeline %s
run pipeline stage %s
Running pipeline stage MKMLProfilerTemplater
Pipeline stage MKMLProfilerTemplater completed in 0.14s
run pipeline stage %s
Running pipeline stage MKMLProfilerDeployer
Creating inference service nousresearch-meta-llama-4939-v59-profiler
Waiting for inference service nousresearch-meta-llama-4939-v59-profiler to be ready
Inference service nousresearch-meta-llama-4939-v59-profiler ready after 220.47178673744202s
Pipeline stage MKMLProfilerDeployer completed in 220.86s
run pipeline stage %s
Running pipeline stage MKMLProfilerRunner
script pods %s
Pipeline stage MKMLProfilerRunner completed in 0.36s
run pipeline stage %s
Running pipeline stage MKMLProfilerDeleter
Checking if service nousresearch-meta-llama-4939-v59-profiler is running
Tearing down inference service nousresearch-meta-llama-4939-v59-profiler
Service nousresearch-meta-llama-4939-v59-profiler has been torndown
Pipeline stage MKMLProfilerDeleter completed in 1.99s
run pipeline %s
run pipeline stage %s
Running pipeline stage MKMLProfilerTemplater
Pipeline stage %s skipped, reason=%s
Pipeline stage MKMLProfilerTemplater completed in 0.30s
run pipeline stage %s
Running pipeline stage MKMLProfilerDeployer
Creating inference service nousresearch-meta-llama-4939-v59-profiler
Waiting for inference service nousresearch-meta-llama-4939-v59-profiler to be ready
Inference service nousresearch-meta-llama-4939-v59-profiler ready after 101.15375590324402s
Pipeline stage MKMLProfilerDeployer completed in 102.24s
run pipeline stage %s
Running pipeline stage MKMLProfilerRunner
kubectl cp /code/guanaco/guanaco_inference_services/scripts tenant-chaiml-guanaco/nousresearch-meta-lle265c31c313791f1dd45ca2f38171ef3-deploxlmfk:/code/chaiverse_profiler_1725169470
kubectl exec -it nousresearch-meta-lle265c31c313791f1dd45ca2f38171ef3-deploxlmfk -- sh -c 'cd /code/chaiverse_profiler_1725169470 && chmod +x profiles.py && python profiles.py profile --best_of_n 4 --batches 1,5,10,15,20,25,30,35,40,45,50,55,60,65,70,75,80,85,90,95,100,105,110,115,120,125,130,135,140,145,150,155,160,165,170,175,180,185,190,195 --samples 200 --input_tokens 512 --output_tokens 64 --summary /code/chaiverse_profiler_1725169470/summary.json'
%s, retrying in %s seconds...
kubectl cp /code/guanaco/guanaco_inference_services/scripts tenant-chaiml-guanaco/nousresearch-meta-lle265c31c313791f1dd45ca2f38171ef3-deploxlmfk:/code/chaiverse_profiler_1725169914
kubectl exec -it nousresearch-meta-lle265c31c313791f1dd45ca2f38171ef3-deploxlmfk -- sh -c 'cd /code/chaiverse_profiler_1725169914 && chmod +x profiles.py && python profiles.py profile --best_of_n 4 --batches 1,5,10,15,20,25,30,35,40,45,50,55,60,65,70,75,80,85,90,95,100,105,110,115,120,125,130,135,140,145,150,155,160,165,170,175,180,185,190,195 --samples 200 --input_tokens 512 --output_tokens 64 --summary /code/chaiverse_profiler_1725169914/summary.json'
nousresearch-meta-llama_4939_v59 status is now inactive due to auto deactivation removed underperforming models
run pipeline %s
run pipeline stage %s
Running pipeline stage MKMLProfilerTemplater
Pipeline stage %s skipped, reason=%s
Pipeline stage MKMLProfilerTemplater completed in 0.07s
run pipeline stage %s
Running pipeline stage MKMLProfilerDeployer
Creating inference service nousresearch-meta-llama-4939-v59-profiler
Waiting for inference service nousresearch-meta-llama-4939-v59-profiler to be ready
run pipeline %s
run pipeline stage %s
Running pipeline stage MKMLProfilerTemplater
Pipeline stage %s skipped, reason=%s
Pipeline stage MKMLProfilerTemplater completed in 0.08s
run pipeline stage %s
Running pipeline stage MKMLProfilerDeployer
Creating inference service nousresearch-meta-llama-4939-v59-profiler
Waiting for inference service nousresearch-meta-llama-4939-v59-profiler to be ready
nousresearch-meta-llama_4939_v59 status is now torndown due to DeploymentManager action