submission_id: nousresearch-meta-llama_4939_v26
developer_uid: end_to_end_test
best_of: 4
celo_rating: 1189.32
display_name: nousresearch-meta-llama_4939_v26
family_friendly_score: 0.0
formatter: {'memory_template': "{bot_name}'s Persona: {memory}\n####\n", 'prompt_template': '{prompt}\n<START>\n', 'bot_template': '{bot_name}: {message}\n', 'user_template': '{user_name}: {message}\n', 'response_template': '{bot_name}:', 'truncate_by_message': False}
generation_params: {'temperature': 1.0, 'top_p': 0.99, 'min_p': 0.1, 'top_k': 40, 'presence_penalty': 0.0, 'frequency_penalty': 0.0, 'stopping_words': ['\n'], 'max_input_tokens': 512, 'best_of': 4, 'max_output_tokens': 64}
ineligible_reason: model is only for e2e test
is_internal_developer: True
language_model: NousResearch/Meta-Llama-3.1-8B-Instruct
max_input_tokens: 512
max_output_tokens: 64
model_architecture: LlamaForCausalLM
model_group: NousResearch/Meta-Llama-
model_name: nousresearch-meta-llama_4939_v26
model_num_parameters: 8030261248.0
model_repo: NousResearch/Meta-Llama-3.1-8B-Instruct
model_size: 8B
num_battles: 13767
num_wins: 5953
ranking_group: single
status: torndown
submission_type: basic
timestamp: 2024-08-29T18:25:06+00:00
us_pacific_date: 2024-08-29
win_ratio: 0.43241083750998766
Download Preference Data
Resubmit model
Running pipeline stage MKMLizer
Starting job with name nousresearch-meta-llama-4939-v26-mkmlizer
Waiting for job on nousresearch-meta-llama-4939-v26-mkmlizer to finish
nousresearch-meta-llama-4939-v26-mkmlizer: ╔═════════════════════════════════════════════════════════════════════╗
nousresearch-meta-llama-4939-v26-mkmlizer: ║     _____            __           __                                ║
nousresearch-meta-llama-4939-v26-mkmlizer: ║    / _/ /_ ___    __/ /  ___ ___ / /                                ║
nousresearch-meta-llama-4939-v26-mkmlizer: ║   / _/ / // / |/|/ / _ \/ -_) -_) /                                 ║
nousresearch-meta-llama-4939-v26-mkmlizer: ║  /_//_/\_, /|__,__/_//_/\__/\__/_/                                  ║
nousresearch-meta-llama-4939-v26-mkmlizer: ║       /___/                                                         ║
nousresearch-meta-llama-4939-v26-mkmlizer: ║                                                                     ║
nousresearch-meta-llama-4939-v26-mkmlizer: ║  Version: 0.10.1                                                    ║
nousresearch-meta-llama-4939-v26-mkmlizer: ║  Copyright 2023 MK ONE TECHNOLOGIES Inc.                            ║
nousresearch-meta-llama-4939-v26-mkmlizer: ║  https://mk1.ai                                                     ║
nousresearch-meta-llama-4939-v26-mkmlizer: ║                                                                     ║
nousresearch-meta-llama-4939-v26-mkmlizer: ║  The license key for the current software has been verified as      ║
nousresearch-meta-llama-4939-v26-mkmlizer: ║  belonging to:                                                      ║
nousresearch-meta-llama-4939-v26-mkmlizer: ║                                                                     ║
nousresearch-meta-llama-4939-v26-mkmlizer: ║  Chai Research Corp.                                                ║
nousresearch-meta-llama-4939-v26-mkmlizer: ║  Account ID: 7997a29f-0ceb-4cc7-9adf-840c57b4ae6f                   ║
nousresearch-meta-llama-4939-v26-mkmlizer: ║  Expiration: 2024-10-15 23:59:59                                    ║
nousresearch-meta-llama-4939-v26-mkmlizer: ║                                                                     ║
nousresearch-meta-llama-4939-v26-mkmlizer: ╚═════════════════════════════════════════════════════════════════════╝
nousresearch-meta-llama-4939-v26-mkmlizer: Downloaded to shared memory in 113.080s
nousresearch-meta-llama-4939-v26-mkmlizer: quantizing model to /dev/shm/model_cache, profile:s0, folder:/tmp/tmpqli1_x_5, device:0
nousresearch-meta-llama-4939-v26-mkmlizer: Saving flywheel model at /dev/shm/model_cache
nousresearch-meta-llama-4939-v26-mkmlizer: quantized model in 26.260s
nousresearch-meta-llama-4939-v26-mkmlizer: Processed model NousResearch/Meta-Llama-3.1-8B-Instruct in 139.340s
nousresearch-meta-llama-4939-v26-mkmlizer: creating bucket guanaco-mkml-models
nousresearch-meta-llama-4939-v26-mkmlizer: Bucket 's3://guanaco-mkml-models/' created
nousresearch-meta-llama-4939-v26-mkmlizer: uploading /dev/shm/model_cache to s3://guanaco-mkml-models/nousresearch-meta-llama-4939-v26
nousresearch-meta-llama-4939-v26-mkmlizer: cp /dev/shm/model_cache/config.json s3://guanaco-mkml-models/nousresearch-meta-llama-4939-v26/config.json
nousresearch-meta-llama-4939-v26-mkmlizer: cp /dev/shm/model_cache/special_tokens_map.json s3://guanaco-mkml-models/nousresearch-meta-llama-4939-v26/special_tokens_map.json
nousresearch-meta-llama-4939-v26-mkmlizer: cp /dev/shm/model_cache/tokenizer_config.json s3://guanaco-mkml-models/nousresearch-meta-llama-4939-v26/tokenizer_config.json
nousresearch-meta-llama-4939-v26-mkmlizer: cp /dev/shm/model_cache/tokenizer.json s3://guanaco-mkml-models/nousresearch-meta-llama-4939-v26/tokenizer.json
nousresearch-meta-llama-4939-v26-mkmlizer: cp /dev/shm/model_cache/flywheel_model.0.safetensors s3://guanaco-mkml-models/nousresearch-meta-llama-4939-v26/flywheel_model.0.safetensors
nousresearch-meta-llama-4939-v26-mkmlizer: 
Loading 0:   0%|          | 0/291 [00:00<?, ?it/s]
Loading 0:   2%|▏         | 5/291 [00:00<00:08, 35.09it/s]
Loading 0:   5%|▍         | 14/291 [00:00<00:05, 47.88it/s]
Loading 0:   8%|▊         | 22/291 [00:00<00:04, 57.13it/s]
Loading 0:  10%|▉         | 28/291 [00:00<00:04, 53.14it/s]
Loading 0:  12%|█▏        | 35/291 [00:00<00:04, 58.13it/s]
Loading 0:  14%|█▍        | 41/291 [00:00<00:05, 49.56it/s]
Loading 0:  17%|█▋        | 49/291 [00:00<00:04, 55.45it/s]
Loading 0:  19%|█▉        | 55/291 [00:01<00:04, 51.96it/s]
Loading 0:  21%|██        | 61/291 [00:01<00:04, 50.46it/s]
Loading 0:  23%|██▎       | 67/291 [00:01<00:04, 52.46it/s]
Loading 0:  25%|██▌       | 73/291 [00:01<00:04, 49.04it/s]
Loading 0:  27%|██▋       | 79/291 [00:01<00:04, 49.32it/s]
Loading 0:  29%|██▉       | 85/291 [00:01<00:05, 35.93it/s]
Loading 0:  31%|███▏      | 91/291 [00:01<00:05, 37.46it/s]
Loading 0:  33%|███▎      | 96/291 [00:02<00:04, 39.29it/s]
Loading 0:  35%|███▌      | 103/291 [00:02<00:04, 46.22it/s]
Loading 0:  37%|███▋      | 109/291 [00:02<00:04, 45.17it/s]
Loading 0:  39%|███▉      | 114/291 [00:02<00:04, 43.18it/s]
Loading 0:  41%|████      | 120/291 [00:02<00:03, 46.91it/s]
Loading 0:  43%|████▎     | 125/291 [00:02<00:03, 45.38it/s]
Loading 0:  45%|████▍     | 130/291 [00:02<00:03, 44.11it/s]
Loading 0:  47%|████▋     | 136/291 [00:02<00:03, 41.57it/s]
Loading 0:  48%|████▊     | 141/291 [00:03<00:03, 42.18it/s]
Loading 0:  51%|█████     | 147/291 [00:03<00:03, 46.15it/s]
Loading 0:  52%|█████▏    | 152/291 [00:03<00:03, 46.32it/s]
Loading 0:  54%|█████▍    | 158/291 [00:03<00:03, 41.06it/s]
Loading 0:  57%|█████▋    | 165/291 [00:03<00:02, 47.49it/s]
Loading 0:  59%|█████▉    | 171/291 [00:03<00:02, 49.33it/s]
Loading 0:  61%|██████    | 177/291 [00:03<00:02, 45.48it/s]
Loading 0:  63%|██████▎   | 182/291 [00:03<00:02, 43.87it/s]
Loading 0:  64%|██████▍   | 187/291 [00:04<00:03, 33.68it/s]
Loading 0:  66%|██████▌   | 192/291 [00:04<00:02, 35.48it/s]
Loading 0:  67%|██████▋   | 196/291 [00:04<00:02, 36.22it/s]
Loading 0:  69%|██████▉   | 201/291 [00:04<00:02, 39.51it/s]
Loading 0:  71%|███████   | 206/291 [00:04<00:02, 40.32it/s]
Loading 0:  73%|███████▎  | 211/291 [00:04<00:01, 42.40it/s]
Loading 0:  75%|███████▍  | 217/291 [00:04<00:01, 41.91it/s]
Loading 0:  76%|███████▋  | 222/291 [00:05<00:01, 42.25it/s]
Loading 0:  79%|███████▊  | 229/291 [00:05<00:01, 47.67it/s]
Loading 0:  80%|████████  | 234/291 [00:05<00:01, 47.86it/s]
Loading 0:  82%|████████▏ | 239/291 [00:05<00:01, 39.80it/s]
Loading 0:  85%|████████▍ | 247/291 [00:05<00:00, 48.57it/s]
Loading 0:  87%|████████▋ | 253/291 [00:05<00:00, 44.68it/s]
Loading 0:  89%|████████▊ | 258/291 [00:05<00:00, 45.44it/s]
Loading 0:  91%|█████████ | 264/291 [00:05<00:00, 48.57it/s]
Loading 0:  93%|█████████▎| 270/291 [00:05<00:00, 50.07it/s]
Loading 0:  95%|█████████▍| 276/291 [00:06<00:00, 46.23it/s]
Loading 0:  97%|█████████▋| 282/291 [00:06<00:00, 43.65it/s]
Loading 0:  99%|█████████▊| 287/291 [00:11<00:01,  3.31it/s]
                                                            

Job nousresearch-meta-llama-4939-v26-mkmlizer completed after 160.9s with status: succeeded
Stopping job with name nousresearch-meta-llama-4939-v26-mkmlizer
Pipeline stage MKMLizer completed in 161.97s
Running pipeline stage MKMLKubeTemplater
Pipeline stage MKMLKubeTemplater completed in 0.37s
Running pipeline stage ISVCDeployer
Creating inference service nousresearch-meta-llama-4939-v26
Waiting for inference service nousresearch-meta-llama-4939-v26 to be ready
Inference service nousresearch-meta-llama-4939-v26 ready after 181.7695119380951s
Pipeline stage ISVCDeployer completed in 182.71s
Running pipeline stage StressChecker
Received healthy response to inference request in 2.555398941040039s
Received healthy response to inference request in 1.3505868911743164s
Received healthy response to inference request in 1.2199909687042236s
Received healthy response to inference request in 1.406332015991211s
Received healthy response to inference request in 1.5321331024169922s
5 requests
0 failed requests
5th percentile: 1.2461101531982421
10th percentile: 1.2722293376922607
20th percentile: 1.324467706680298
30th percentile: 1.3617359161376954
40th percentile: 1.3840339660644532
50th percentile: 1.406332015991211
60th percentile: 1.4566524505615235
70th percentile: 1.506972885131836
80th percentile: 1.7367862701416017
90th percentile: 2.1460926055908205
95th percentile: 2.3507457733154293
99th percentile: 2.5144683074951173
mean time: 1.6128883838653565
Pipeline stage StressChecker completed in 10.05s
nousresearch-meta-llama_4939_v26 status is now deployed due to DeploymentManager action
nousresearch-meta-llama_4939_v26 status is now inactive due to auto deactivation removed underperforming models
Running pipeline stage MKMLDeleter
Checking if service nousresearch-meta-llama-4939-v26 is running
Tearing down inference service nousresearch-meta-llama-4939-v26
Service nousresearch-meta-llama-4939-v26 has been torndown
Pipeline stage MKMLDeleter completed in 3.17s
Running pipeline stage MKMLModelDeleter
Cleaning model data from S3
Cleaning model data from model cache
Deleting key nousresearch-meta-llama-4939-v26/config.json from bucket guanaco-mkml-models
Deleting key nousresearch-meta-llama-4939-v26/flywheel_model.0.safetensors from bucket guanaco-mkml-models
Deleting key nousresearch-meta-llama-4939-v26/special_tokens_map.json from bucket guanaco-mkml-models
Deleting key nousresearch-meta-llama-4939-v26/tokenizer.json from bucket guanaco-mkml-models
Deleting key nousresearch-meta-llama-4939-v26/tokenizer_config.json from bucket guanaco-mkml-models
Pipeline stage MKMLModelDeleter completed in 2.42s
Running pipeline stage MKMLDeleter
Pipeline stage %s skipped, reason=%s
Pipeline stage MKMLDeleter completed in 0.29s
Running pipeline stage MKMLModelDeleter
Pipeline stage %s skipped, reason=%s
Pipeline stage MKMLModelDeleter completed in 0.18s
Running pipeline stage MKMLDeleter
Pipeline stage %s skipped, reason=%s
Pipeline stage MKMLDeleter completed in 0.08s
Running pipeline stage MKMLModelDeleter
Pipeline stage %s skipped, reason=%s
Pipeline stage MKMLModelDeleter completed in 0.07s
nousresearch-meta-llama_4939_v26 status is now torndown due to DeploymentManager action