submission_id: nousresearch-meta-llama_4939_v39
developer_uid: end_to_end_test
best_of: 4
celo_rating: 1185.64
display_name: nousresearch-meta-llama_4939_v39
family_friendly_score: 0.0
formatter: {'memory_template': "{bot_name}'s Persona: {memory}\n####\n", 'prompt_template': '{prompt}\n<START>\n', 'bot_template': '{bot_name}: {message}\n', 'user_template': '{user_name}: {message}\n', 'response_template': '{bot_name}:', 'truncate_by_message': False}
generation_params: {'temperature': 1.0, 'top_p': 0.99, 'min_p': 0.1, 'top_k': 40, 'presence_penalty': 0.0, 'frequency_penalty': 0.0, 'stopping_words': ['\n'], 'max_input_tokens': 512, 'best_of': 4, 'max_output_tokens': 64}
ineligible_reason: model is only for e2e test
is_internal_developer: True
language_model: NousResearch/Meta-Llama-3.1-8B-Instruct
max_input_tokens: 512
max_output_tokens: 64
model_architecture: LlamaForCausalLM
model_group: NousResearch/Meta-Llama-
model_name: nousresearch-meta-llama_4939_v39
model_num_parameters: 8030261248.0
model_repo: NousResearch/Meta-Llama-3.1-8B-Instruct
model_size: 8B
num_battles: 8687
num_wins: 3716
ranking_group: single
status: torndown
submission_type: basic
timestamp: 2024-08-30T19:22:28+00:00
us_pacific_date: 2024-08-30
win_ratio: 0.42776562679866464
Download Preference Data
Resubmit model
Running pipeline stage MKMLizer
Deleting key nousresearch-meta-llama-4939-v38/tokenizer.json from bucket guanaco-mkml-models
Starting job with name nousresearch-meta-llama-4939-v39-mkmlizer
Deleting key nousresearch-meta-llama-4939-v38/tokenizer_config.json from bucket guanaco-mkml-models
Waiting for job on nousresearch-meta-llama-4939-v39-mkmlizer to finish
Pipeline stage MKMLModelDeleter completed in 15.42s
nousresearch-meta-llama_4939_v38 status is now torndown due to DeploymentManager action
nousresearch-meta-llama-4939-v39-mkmlizer: ╔═════════════════════════════════════════════════════════════════════╗
nousresearch-meta-llama-4939-v39-mkmlizer: ║     _____            __           __                                ║
nousresearch-meta-llama-4939-v39-mkmlizer: ║    / _/ /_ ___    __/ /  ___ ___ / /                                ║
nousresearch-meta-llama-4939-v39-mkmlizer: ║   / _/ / // / |/|/ / _ \/ -_) -_) /                                 ║
nousresearch-meta-llama-4939-v39-mkmlizer: ║  /_//_/\_, /|__,__/_//_/\__/\__/_/                                  ║
nousresearch-meta-llama-4939-v39-mkmlizer: ║       /___/                                                         ║
nousresearch-meta-llama-4939-v39-mkmlizer: ║                                                                     ║
nousresearch-meta-llama-4939-v39-mkmlizer: ║  Version: 0.10.1                                                    ║
nousresearch-meta-llama-4939-v39-mkmlizer: ║  Copyright 2023 MK ONE TECHNOLOGIES Inc.                            ║
nousresearch-meta-llama-4939-v39-mkmlizer: ║  https://mk1.ai                                                     ║
nousresearch-meta-llama-4939-v39-mkmlizer: ║                                                                     ║
nousresearch-meta-llama-4939-v39-mkmlizer: ║  The license key for the current software has been verified as      ║
nousresearch-meta-llama-4939-v39-mkmlizer: ║  belonging to:                                                      ║
nousresearch-meta-llama-4939-v39-mkmlizer: ║                                                                     ║
nousresearch-meta-llama-4939-v39-mkmlizer: ║  Chai Research Corp.                                                ║
nousresearch-meta-llama-4939-v39-mkmlizer: ║  Account ID: 7997a29f-0ceb-4cc7-9adf-840c57b4ae6f                   ║
nousresearch-meta-llama-4939-v39-mkmlizer: ║  Expiration: 2024-10-15 23:59:59                                    ║
nousresearch-meta-llama-4939-v39-mkmlizer: ║                                                                     ║
nousresearch-meta-llama-4939-v39-mkmlizer: ╚═════════════════════════════════════════════════════════════════════╝
nousresearch-meta-llama-4939-v39-mkmlizer: Downloaded to shared memory in 56.119s
nousresearch-meta-llama-4939-v39-mkmlizer: quantizing model to /dev/shm/model_cache, profile:s0, folder:/tmp/tmp8xm35jeb, device:0
nousresearch-meta-llama-4939-v39-mkmlizer: Saving flywheel model at /dev/shm/model_cache
nousresearch-meta-llama-4939-v39-mkmlizer: quantized model in 25.977s
nousresearch-meta-llama-4939-v39-mkmlizer: Processed model NousResearch/Meta-Llama-3.1-8B-Instruct in 82.096s
nousresearch-meta-llama-4939-v39-mkmlizer: creating bucket guanaco-mkml-models
nousresearch-meta-llama-4939-v39-mkmlizer: Bucket 's3://guanaco-mkml-models/' created
nousresearch-meta-llama-4939-v39-mkmlizer: uploading /dev/shm/model_cache to s3://guanaco-mkml-models/nousresearch-meta-llama-4939-v39
nousresearch-meta-llama-4939-v39-mkmlizer: cp /dev/shm/model_cache/config.json s3://guanaco-mkml-models/nousresearch-meta-llama-4939-v39/config.json
nousresearch-meta-llama-4939-v39-mkmlizer: cp /dev/shm/model_cache/special_tokens_map.json s3://guanaco-mkml-models/nousresearch-meta-llama-4939-v39/special_tokens_map.json
nousresearch-meta-llama-4939-v39-mkmlizer: cp /dev/shm/model_cache/tokenizer_config.json s3://guanaco-mkml-models/nousresearch-meta-llama-4939-v39/tokenizer_config.json
nousresearch-meta-llama-4939-v39-mkmlizer: cp /dev/shm/model_cache/tokenizer.json s3://guanaco-mkml-models/nousresearch-meta-llama-4939-v39/tokenizer.json
nousresearch-meta-llama-4939-v39-mkmlizer: cp /dev/shm/model_cache/flywheel_model.0.safetensors s3://guanaco-mkml-models/nousresearch-meta-llama-4939-v39/flywheel_model.0.safetensors
nousresearch-meta-llama-4939-v39-mkmlizer: 
Loading 0:   0%|          | 0/291 [00:00<?, ?it/s]
Loading 0:   2%|▏         | 5/291 [00:00<00:08, 34.94it/s]
Loading 0:   4%|▍         | 13/291 [00:00<00:04, 55.94it/s]
Loading 0:   7%|▋         | 19/291 [00:00<00:05, 50.45it/s]
Loading 0:   9%|▊         | 25/291 [00:00<00:05, 51.61it/s]
Loading 0:  11%|█         | 32/291 [00:00<00:05, 47.06it/s]
Loading 0:  14%|█▍        | 41/291 [00:00<00:05, 49.53it/s]
Loading 0:  17%|█▋        | 49/291 [00:00<00:04, 55.85it/s]
Loading 0:  19%|█▉        | 55/291 [00:01<00:04, 52.24it/s]
Loading 0:  21%|██        | 61/291 [00:01<00:04, 51.33it/s]
Loading 0:  23%|██▎       | 68/291 [00:01<00:04, 47.01it/s]
Loading 0:  26%|██▌       | 75/291 [00:01<00:04, 51.91it/s]
Loading 0:  28%|██▊       | 81/291 [00:01<00:03, 53.33it/s]
Loading 0:  30%|██▉       | 87/291 [00:01<00:05, 34.41it/s]
Loading 0:  32%|███▏      | 94/291 [00:02<00:04, 40.50it/s]
Loading 0:  34%|███▍      | 100/291 [00:02<00:04, 40.30it/s]
Loading 0:  36%|███▌      | 105/291 [00:02<00:04, 40.92it/s]
Loading 0:  38%|███▊      | 112/291 [00:02<00:03, 46.77it/s]
Loading 0:  41%|████      | 118/291 [00:02<00:03, 44.59it/s]
Loading 0:  42%|████▏     | 123/291 [00:02<00:03, 44.95it/s]
Loading 0:  45%|████▍     | 130/291 [00:02<00:03, 50.45it/s]
Loading 0:  47%|████▋     | 136/291 [00:02<00:03, 45.78it/s]
Loading 0:  48%|████▊     | 141/291 [00:03<00:03, 43.65it/s]
Loading 0:  51%|█████     | 148/291 [00:03<00:02, 48.65it/s]
Loading 0:  53%|█████▎    | 154/291 [00:03<00:03, 45.03it/s]
Loading 0:  55%|█████▍    | 159/291 [00:03<00:02, 44.04it/s]
Loading 0:  57%|█████▋    | 165/291 [00:03<00:02, 47.85it/s]
Loading 0:  58%|█████▊    | 170/291 [00:03<00:02, 46.88it/s]
Loading 0:  61%|██████    | 177/291 [00:03<00:02, 47.50it/s]
Loading 0:  63%|██████▎   | 182/291 [00:03<00:02, 46.69it/s]
Loading 0:  64%|██████▍   | 187/291 [00:04<00:02, 36.27it/s]
Loading 0:  66%|██████▌   | 192/291 [00:04<00:02, 37.97it/s]
Loading 0:  68%|██████▊   | 197/291 [00:04<00:02, 40.56it/s]
Loading 0:  69%|██████▉   | 202/291 [00:04<00:02, 42.56it/s]
Loading 0:  71%|███████▏  | 208/291 [00:04<00:02, 41.01it/s]
Loading 0:  73%|███████▎  | 213/291 [00:04<00:01, 41.94it/s]
Loading 0:  76%|███████▌  | 220/291 [00:04<00:01, 48.41it/s]
Loading 0:  78%|███████▊  | 226/291 [00:04<00:01, 47.04it/s]
Loading 0:  79%|███████▉  | 231/291 [00:05<00:01, 46.77it/s]
Loading 0:  82%|████████▏ | 238/291 [00:05<00:01, 51.10it/s]
Loading 0:  84%|████████▍ | 244/291 [00:05<00:01, 46.80it/s]
Loading 0:  86%|████████▌ | 249/291 [00:05<00:00, 46.40it/s]
Loading 0:  88%|████████▊ | 256/291 [00:05<00:00, 50.84it/s]
Loading 0:  90%|█████████ | 262/291 [00:05<00:00, 46.64it/s]
Loading 0:  92%|█████████▏| 267/291 [00:05<00:00, 45.71it/s]
Loading 0:  94%|█████████▍| 273/291 [00:05<00:00, 48.82it/s]
Loading 0:  96%|█████████▌| 278/291 [00:06<00:00, 48.86it/s]
Loading 0:  97%|█████████▋| 283/291 [00:06<00:00, 42.77it/s]
Loading 0:  99%|█████████▉| 288/291 [00:11<00:00,  3.09it/s]
                                                            

Job nousresearch-meta-llama-4939-v39-mkmlizer completed after 108.07s with status: succeeded
Stopping job with name nousresearch-meta-llama-4939-v39-mkmlizer
Pipeline stage MKMLizer completed in 109.42s
Running pipeline stage MKMLTemplater
Pipeline stage MKMLTemplater completed in 0.36s
Running pipeline stage MKMLDeployer
Creating inference service nousresearch-meta-llama-4939-v39
Waiting for inference service nousresearch-meta-llama-4939-v39 to be ready
Inference service nousresearch-meta-llama-4939-v39 ready after 191.65926718711853s
Pipeline stage MKMLDeployer completed in 192.59s
Running pipeline stage StressChecker
Received healthy response to inference request in 1.7822000980377197s
Received healthy response to inference request in 1.2912828922271729s
Received healthy response to inference request in 2.868964195251465s
Received healthy response to inference request in 1.4418785572052002s
Received healthy response to inference request in 1.0637328624725342s
5 requests
0 failed requests
5th percentile: 1.1092428684234619
10th percentile: 1.1547528743743896
20th percentile: 1.2457728862762452
30th percentile: 1.3214020252227783
40th percentile: 1.3816402912139893
50th percentile: 1.4418785572052002
60th percentile: 1.578007173538208
70th percentile: 1.7141357898712157
80th percentile: 1.999552917480469
90th percentile: 2.434258556365967
95th percentile: 2.6516113758087156
99th percentile: 2.825493631362915
mean time: 1.6896117210388184
Pipeline stage StressChecker completed in 10.57s
Running pipeline stage TriggerMKMLProfilingPipeline
starting trigger_guanaco_pipeline %s
triggered trigger_guanaco_pipeline %s
Pipeline stage TriggerMKMLProfilingPipeline completed in 2.46s
nousresearch-meta-llama_4939_v39 status is now deployed due to DeploymentManager action
nousresearch-meta-llama_4939_v39 status is now inactive due to admin request
Starting job with name nousresearch-meta-llama-4939-v40-mkmlizer
nousresearch-meta-llama_4939_v39 status is now torndown due to DeploymentManager action