nousresearch-meta-llama_4939

submission_id: nousresearch-meta-llama_4939_v46

developer_uid: end_to_end_test

best_of: 4

celo_rating: 1181.54

display_name: nousresearch-meta-llama_4939_v46

family_friendly_score: 0.0

formatter: {'memory_template': "{bot_name}'s Persona: {memory}\n####\n", 'prompt_template': '{prompt}\n<START>\n', 'bot_template': '{bot_name}: {message}\n', 'user_template': '{user_name}: {message}\n', 'response_template': '{bot_name}:', 'truncate_by_message': False}

generation_params: {'temperature': 1.0, 'top_p': 0.99, 'min_p': 0.1, 'top_k': 40, 'presence_penalty': 0.0, 'frequency_penalty': 0.0, 'stopping_words': ['\n'], 'max_input_tokens': 512, 'best_of': 4, 'max_output_tokens': 64}

ineligible_reason: model is only for e2e test

is_internal_developer: True

language_model: NousResearch/Meta-Llama-3.1-8B-Instruct

max_input_tokens: 512

max_output_tokens: 64

model_architecture: LlamaForCausalLM

model_group: NousResearch/Meta-Llama-

model_name: nousresearch-meta-llama_4939_v46

model_num_parameters: 8030261248.0

model_repo: NousResearch/Meta-Llama-3.1-8B-Instruct

model_size: 8B

num_battles: 2775

num_wins: 1170

ranking_group: single

status: torndown

submission_type: basic

timestamp: 2024-08-30T22:34:30+00:00

us_pacific_date: 2024-08-30

win_ratio: 0.42162162162162165

Download Preference Data

Resubmit model

Running pipeline stage MKMLizer
Deleting key nousresearch-meta-llama-4939-v45/tokenizer.json from bucket guanaco-mkml-models
Starting job with name nousresearch-meta-llama-4939-v46-mkmlizer
Deleting key nousresearch-meta-llama-4939-v45/tokenizer_config.json from bucket guanaco-mkml-models
Waiting for job on nousresearch-meta-llama-4939-v46-mkmlizer to finish
Pipeline stage MKMLModelDeleter completed in 8.24s
nousresearch-meta-llama_4939_v45 status is now torndown due to DeploymentManager action
nousresearch-meta-llama-4939-v46-mkmlizer: ╔═════════════════════════════════════════════════════════════════════╗
nousresearch-meta-llama-4939-v46-mkmlizer: ║     _____            __           __                                ║
nousresearch-meta-llama-4939-v46-mkmlizer: ║    / _/ /_ ___    __/ /  ___ ___ / /                                ║
nousresearch-meta-llama-4939-v46-mkmlizer: ║   / _/ / // / |/|/ / _ \/ -_) -_) /                                 ║
nousresearch-meta-llama-4939-v46-mkmlizer: ║  /_//_/\_, /|__,__/_//_/\__/\__/_/                                  ║
nousresearch-meta-llama-4939-v46-mkmlizer: ║       /___/                                                         ║
nousresearch-meta-llama-4939-v46-mkmlizer: ║                                                                     ║
nousresearch-meta-llama-4939-v46-mkmlizer: ║  Version: 0.10.1                                                    ║
nousresearch-meta-llama-4939-v46-mkmlizer: ║  Copyright 2023 MK ONE TECHNOLOGIES Inc.                            ║
nousresearch-meta-llama-4939-v46-mkmlizer: ║  https://mk1.ai                                                     ║
nousresearch-meta-llama-4939-v46-mkmlizer: ║                                                                     ║
nousresearch-meta-llama-4939-v46-mkmlizer: ║  The license key for the current software has been verified as      ║
nousresearch-meta-llama-4939-v46-mkmlizer: ║  belonging to:                                                      ║
nousresearch-meta-llama-4939-v46-mkmlizer: ║                                                                     ║
nousresearch-meta-llama-4939-v46-mkmlizer: ║  Chai Research Corp.                                                ║
nousresearch-meta-llama-4939-v46-mkmlizer: ║  Account ID: 7997a29f-0ceb-4cc7-9adf-840c57b4ae6f                   ║
nousresearch-meta-llama-4939-v46-mkmlizer: ║  Expiration: 2024-10-15 23:59:59                                    ║
nousresearch-meta-llama-4939-v46-mkmlizer: ║                                                                     ║
nousresearch-meta-llama-4939-v46-mkmlizer: ╚═════════════════════════════════════════════════════════════════════╝
nousresearch-meta-llama-4939-v46-mkmlizer: Downloaded to shared memory in 51.397s
nousresearch-meta-llama-4939-v46-mkmlizer: quantizing model to /dev/shm/model_cache, profile:s0, folder:/tmp/tmpovn3eu8m, device:0
nousresearch-meta-llama-4939-v46-mkmlizer: Saving flywheel model at /dev/shm/model_cache
nousresearch-meta-llama-4939-v46-mkmlizer: quantized model in 25.727s
nousresearch-meta-llama-4939-v46-mkmlizer: Processed model NousResearch/Meta-Llama-3.1-8B-Instruct in 77.125s
nousresearch-meta-llama-4939-v46-mkmlizer: creating bucket guanaco-mkml-models
nousresearch-meta-llama-4939-v46-mkmlizer: Bucket 's3://guanaco-mkml-models/' created
nousresearch-meta-llama-4939-v46-mkmlizer: uploading /dev/shm/model_cache to s3://guanaco-mkml-models/nousresearch-meta-llama-4939-v46
nousresearch-meta-llama-4939-v46-mkmlizer: cp /dev/shm/model_cache/config.json s3://guanaco-mkml-models/nousresearch-meta-llama-4939-v46/config.json
nousresearch-meta-llama-4939-v46-mkmlizer: cp /dev/shm/model_cache/special_tokens_map.json s3://guanaco-mkml-models/nousresearch-meta-llama-4939-v46/special_tokens_map.json
nousresearch-meta-llama-4939-v46-mkmlizer: cp /dev/shm/model_cache/tokenizer_config.json s3://guanaco-mkml-models/nousresearch-meta-llama-4939-v46/tokenizer_config.json
nousresearch-meta-llama-4939-v46-mkmlizer: cp /dev/shm/model_cache/tokenizer.json s3://guanaco-mkml-models/nousresearch-meta-llama-4939-v46/tokenizer.json
nousresearch-meta-llama-4939-v46-mkmlizer: cp /dev/shm/model_cache/flywheel_model.0.safetensors s3://guanaco-mkml-models/nousresearch-meta-llama-4939-v46/flywheel_model.0.safetensors
nousresearch-meta-llama-4939-v46-mkmlizer: 
Loading 0:   0%|          | 0/291 [00:00<?, ?it/s]
Loading 0:   2%|▏         | 5/291 [00:00<00:07, 38.20it/s]
Loading 0:   5%|▍         | 14/291 [00:00<00:05, 47.52it/s]
Loading 0:   8%|▊         | 23/291 [00:00<00:05, 51.39it/s]
Loading 0:  11%|█         | 32/291 [00:00<00:04, 52.33it/s]
Loading 0:  14%|█▍        | 41/291 [00:00<00:04, 52.35it/s]
Loading 0:  17%|█▋        | 50/291 [00:00<00:04, 53.32it/s]
Loading 0:  20%|██        | 59/291 [00:01<00:04, 53.21it/s]
Loading 0:  23%|██▎       | 67/291 [00:01<00:03, 58.89it/s]
Loading 0:  25%|██▌       | 74/291 [00:01<00:03, 56.59it/s]
Loading 0:  27%|██▋       | 80/291 [00:01<00:03, 54.45it/s]
Loading 0:  30%|██▉       | 86/291 [00:01<00:05, 36.77it/s]
Loading 0:  32%|███▏      | 94/291 [00:01<00:04, 44.34it/s]
Loading 0:  34%|███▍      | 100/291 [00:02<00:04, 43.99it/s]
Loading 0:  36%|███▋      | 106/291 [00:02<00:04, 45.68it/s]
Loading 0:  38%|███▊      | 112/291 [00:02<00:03, 48.84it/s]
Loading 0:  41%|████      | 118/291 [00:02<00:03, 47.07it/s]
Loading 0:  43%|████▎     | 124/291 [00:02<00:03, 47.12it/s]
Loading 0:  45%|████▍     | 130/291 [00:02<00:03, 50.13it/s]
Loading 0:  47%|████▋     | 136/291 [00:02<00:03, 47.87it/s]
Loading 0:  48%|████▊     | 141/291 [00:02<00:03, 47.49it/s]
Loading 0:  51%|█████     | 148/291 [00:02<00:02, 52.34it/s]
Loading 0:  53%|█████▎    | 154/291 [00:03<00:02, 48.62it/s]
Loading 0:  55%|█████▍    | 159/291 [00:03<00:02, 47.75it/s]
Loading 0:  57%|█████▋    | 166/291 [00:03<00:02, 51.83it/s]
Loading 0:  59%|█████▉    | 172/291 [00:03<00:02, 48.29it/s]
Loading 0:  62%|██████▏   | 179/291 [00:03<00:02, 52.18it/s]
Loading 0:  64%|██████▎   | 185/291 [00:03<00:01, 53.70it/s]
Loading 0:  66%|██████▌   | 191/291 [00:04<00:02, 36.30it/s]
Loading 0:  67%|██████▋   | 196/291 [00:04<00:02, 38.23it/s]
Loading 0:  69%|██████▉   | 201/291 [00:04<00:02, 40.73it/s]
Loading 0:  71%|███████   | 206/291 [00:04<00:01, 42.68it/s]
Loading 0:  73%|███████▎  | 211/291 [00:04<00:01, 43.64it/s]
Loading 0:  75%|███████▍  | 217/291 [00:04<00:01, 42.57it/s]
Loading 0:  76%|███████▋  | 222/291 [00:04<00:01, 43.66it/s]
Loading 0:  79%|███████▊  | 229/291 [00:04<00:01, 49.36it/s]
Loading 0:  81%|████████  | 235/291 [00:04<00:01, 47.42it/s]
Loading 0:  82%|████████▏ | 240/291 [00:05<00:01, 47.33it/s]
Loading 0:  85%|████████▍ | 247/291 [00:05<00:00, 51.55it/s]
Loading 0:  87%|████████▋ | 253/291 [00:05<00:00, 48.13it/s]
Loading 0:  89%|████████▊ | 258/291 [00:05<00:00, 47.54it/s]
Loading 0:  91%|█████████ | 265/291 [00:05<00:00, 52.08it/s]
Loading 0:  93%|█████████▎| 271/291 [00:05<00:00, 48.17it/s]
Loading 0:  95%|█████████▍| 276/291 [00:05<00:00, 47.65it/s]
Loading 0:  97%|█████████▋| 281/291 [00:05<00:00, 47.30it/s]
Loading 0:  98%|█████████▊| 286/291 [00:06<00:00, 42.08it/s]
Loading 0: 100%|██████████| 291/291 [00:11<00:00,  3.12it/s]
                                                            
Job nousresearch-meta-llama-4939-v46-mkmlizer completed after 98.5s with status: succeeded
Stopping job with name nousresearch-meta-llama-4939-v46-mkmlizer
Pipeline stage MKMLizer completed in 99.78s
Running pipeline stage MKMLTemplater
Pipeline stage MKMLTemplater completed in 0.28s
Running pipeline stage MKMLDeployer
Creating inference service nousresearch-meta-llama-4939-v46
Waiting for inference service nousresearch-meta-llama-4939-v46 to be ready
Inference service nousresearch-meta-llama-4939-v46 ready after 322.7609157562256s
Pipeline stage MKMLDeployer completed in 323.62s
Running pipeline stage StressChecker
Received healthy response to inference request in 2.2759909629821777s
Received healthy response to inference request in 1.4656038284301758s
Received healthy response to inference request in 1.6044979095458984s
Received healthy response to inference request in 1.7303009033203125s
Received healthy response to inference request in 1.2719299793243408s
5 requests
0 failed requests
5th percentile: 1.310664749145508
10th percentile: 1.3493995189666748
20th percentile: 1.4268690586090087
30th percentile: 1.4933826446533203
40th percentile: 1.5489402770996095
50th percentile: 1.6044979095458984
60th percentile: 1.6548191070556642
70th percentile: 1.7051403045654296
80th percentile: 1.8394389152526855
90th percentile: 2.0577149391174316
95th percentile: 2.1668529510498047
99th percentile: 2.254163360595703
mean time: 1.6696647167205811
Pipeline stage StressChecker completed in 10.82s
Running pipeline stage TriggerMKMLProfilingPipeline
starting trigger_guanaco_pipeline %s
Pipeline stage TriggerMKMLProfilingPipeline completed in 5.97s
nousresearch-meta-llama_4939_v46 status is now deployed due to DeploymentManager action
Running pipeline stage MKMLProfilerTemplater
Pipeline stage MKMLProfilerTemplater completed in 0.17s
Running pipeline stage MKMLProfilerDeployer
Creating inference service nousresearch-meta-llama-4939-v46-profiler
Waiting for inference service nousresearch-meta-llama-4939-v46-profiler to be ready
Inference service nousresearch-meta-llama-4939-v46-profiler ready after 190.4205777645111s
Pipeline stage MKMLProfilerDeployer completed in 190.79s
Running pipeline stage MKMLProfilerRunner
script pods %s
Pipeline stage MKMLProfilerRunner completed in 0.38s
Running pipeline stage MKMLProfilerDeleter
Checking if service nousresearch-meta-llama-4939-v46-profiler is running
Tearing down inference service nousresearch-meta-llama-4939-v46-profiler
Service nousresearch-meta-llama-4939-v46-profiler has been torndown
Pipeline stage MKMLProfilerDeleter completed in 1.97s
nousresearch-meta-llama_4939_v46 status is now inactive due to admin request
Running pipeline stage MKMLizer
nousresearch-meta-llama_4939_v46 status is now torndown due to DeploymentManager action