submission_id: nousresearch-meta-llama_4939_v45
developer_uid: end_to_end_test
best_of: 4
celo_rating: 1179.16
display_name: nousresearch-meta-llama_4939_v45
family_friendly_score: 0.0
formatter: {'memory_template': "{bot_name}'s Persona: {memory}\n####\n", 'prompt_template': '{prompt}\n<START>\n', 'bot_template': '{bot_name}: {message}\n', 'user_template': '{user_name}: {message}\n', 'response_template': '{bot_name}:', 'truncate_by_message': False}
generation_params: {'temperature': 1.0, 'top_p': 0.99, 'min_p': 0.1, 'top_k': 40, 'presence_penalty': 0.0, 'frequency_penalty': 0.0, 'stopping_words': ['\n'], 'max_input_tokens': 512, 'best_of': 4, 'max_output_tokens': 64}
ineligible_reason: model is only for e2e test
is_internal_developer: True
language_model: NousResearch/Meta-Llama-3.1-8B-Instruct
max_input_tokens: 512
max_output_tokens: 64
model_architecture: LlamaForCausalLM
model_group: NousResearch/Meta-Llama-
model_name: nousresearch-meta-llama_4939_v45
model_num_parameters: 8030261248.0
model_repo: NousResearch/Meta-Llama-3.1-8B-Instruct
model_size: 8B
num_battles: 2406
num_wins: 1005
ranking_group: single
status: torndown
submission_type: basic
timestamp: 2024-08-30T22:06:45+00:00
us_pacific_date: 2024-08-30
win_ratio: 0.4177057356608479
Download Preference Data
Resubmit model
Running pipeline stage MKMLizer
Starting job with name nousresearch-meta-llama-4939-v45-mkmlizer
Waiting for job on nousresearch-meta-llama-4939-v45-mkmlizer to finish
nousresearch-meta-llama-4939-v45-mkmlizer: ╔═════════════════════════════════════════════════════════════════════╗
nousresearch-meta-llama-4939-v45-mkmlizer: ║     _____            __           __                                ║
nousresearch-meta-llama-4939-v45-mkmlizer: ║    / _/ /_ ___    __/ /  ___ ___ / /                                ║
nousresearch-meta-llama-4939-v45-mkmlizer: ║   / _/ / // / |/|/ / _ \/ -_) -_) /                                 ║
nousresearch-meta-llama-4939-v45-mkmlizer: ║  /_//_/\_, /|__,__/_//_/\__/\__/_/                                  ║
nousresearch-meta-llama-4939-v45-mkmlizer: ║       /___/                                                         ║
nousresearch-meta-llama-4939-v45-mkmlizer: ║                                                                     ║
nousresearch-meta-llama-4939-v45-mkmlizer: ║  Version: 0.10.1                                                    ║
nousresearch-meta-llama-4939-v45-mkmlizer: ║  Copyright 2023 MK ONE TECHNOLOGIES Inc.                            ║
nousresearch-meta-llama-4939-v45-mkmlizer: ║  https://mk1.ai                                                     ║
nousresearch-meta-llama-4939-v45-mkmlizer: ║                                                                     ║
nousresearch-meta-llama-4939-v45-mkmlizer: ║  The license key for the current software has been verified as      ║
nousresearch-meta-llama-4939-v45-mkmlizer: ║  belonging to:                                                      ║
nousresearch-meta-llama-4939-v45-mkmlizer: ║                                                                     ║
nousresearch-meta-llama-4939-v45-mkmlizer: ║  Chai Research Corp.                                                ║
nousresearch-meta-llama-4939-v45-mkmlizer: ║  Account ID: 7997a29f-0ceb-4cc7-9adf-840c57b4ae6f                   ║
nousresearch-meta-llama-4939-v45-mkmlizer: ║  Expiration: 2024-10-15 23:59:59                                    ║
nousresearch-meta-llama-4939-v45-mkmlizer: ║                                                                     ║
nousresearch-meta-llama-4939-v45-mkmlizer: ╚═════════════════════════════════════════════════════════════════════╝
nousresearch-meta-llama-4939-v45-mkmlizer: Downloaded to shared memory in 36.434s
nousresearch-meta-llama-4939-v45-mkmlizer: quantizing model to /dev/shm/model_cache, profile:s0, folder:/tmp/tmpy6tdl9k3, device:0
nousresearch-meta-llama-4939-v45-mkmlizer: Saving flywheel model at /dev/shm/model_cache
nousresearch-meta-llama-4939-v45-mkmlizer: quantized model in 26.198s
nousresearch-meta-llama-4939-v45-mkmlizer: Processed model NousResearch/Meta-Llama-3.1-8B-Instruct in 62.633s
nousresearch-meta-llama-4939-v45-mkmlizer: creating bucket guanaco-mkml-models
nousresearch-meta-llama-4939-v45-mkmlizer: Bucket 's3://guanaco-mkml-models/' created
nousresearch-meta-llama-4939-v45-mkmlizer: uploading /dev/shm/model_cache to s3://guanaco-mkml-models/nousresearch-meta-llama-4939-v45
nousresearch-meta-llama-4939-v45-mkmlizer: cp /dev/shm/model_cache/config.json s3://guanaco-mkml-models/nousresearch-meta-llama-4939-v45/config.json
nousresearch-meta-llama-4939-v45-mkmlizer: cp /dev/shm/model_cache/special_tokens_map.json s3://guanaco-mkml-models/nousresearch-meta-llama-4939-v45/special_tokens_map.json
nousresearch-meta-llama-4939-v45-mkmlizer: cp /dev/shm/model_cache/tokenizer_config.json s3://guanaco-mkml-models/nousresearch-meta-llama-4939-v45/tokenizer_config.json
nousresearch-meta-llama-4939-v45-mkmlizer: cp /dev/shm/model_cache/tokenizer.json s3://guanaco-mkml-models/nousresearch-meta-llama-4939-v45/tokenizer.json
Job nousresearch-meta-llama-4939-v45-mkmlizer completed after 87.24s with status: succeeded
Stopping job with name nousresearch-meta-llama-4939-v45-mkmlizer
Pipeline stage MKMLizer completed in 88.41s
Running pipeline stage MKMLTemplater
Pipeline stage MKMLTemplater completed in 0.28s
Running pipeline stage MKMLDeployer
Creating inference service nousresearch-meta-llama-4939-v45
Waiting for inference service nousresearch-meta-llama-4939-v45 to be ready
Inference service nousresearch-meta-llama-4939-v45 ready after 171.65142917633057s
Pipeline stage MKMLDeployer completed in 172.63s
Running pipeline stage StressChecker
Received healthy response to inference request in 1.7571702003479004s
Received healthy response to inference request in 2.2805678844451904s
Received healthy response to inference request in 1.5669670104980469s
Received healthy response to inference request in 1.2895917892456055s
Received healthy response to inference request in 2.013291120529175s
5 requests
0 failed requests
5th percentile: 1.3450668334960938
10th percentile: 1.4005418777465821
20th percentile: 1.5114919662475585
30th percentile: 1.6050076484680176
40th percentile: 1.681088924407959
50th percentile: 1.7571702003479004
60th percentile: 1.85961856842041
70th percentile: 1.96206693649292
80th percentile: 2.066746473312378
90th percentile: 2.173657178878784
95th percentile: 2.2271125316619873
99th percentile: 2.26987681388855
mean time: 1.7815176010131837
Pipeline stage StressChecker completed in 11.12s
Running pipeline stage TriggerMKMLProfilingPipeline
starting trigger_guanaco_pipeline %s
Pipeline stage TriggerMKMLProfilingPipeline completed in 2.20s
nousresearch-meta-llama_4939_v45 status is now deployed due to DeploymentManager action
Running pipeline stage MKMLProfilerTemplater
Pipeline stage MKMLProfilerTemplater completed in 0.13s
Running pipeline stage MKMLProfilerDeployer
Creating inference service nousresearch-meta-llama-4939-v45-profiler
Waiting for inference service nousresearch-meta-llama-4939-v45-profiler to be ready
Inference service nousresearch-meta-llama-4939-v45-profiler ready after 190.7116289138794s
Pipeline stage MKMLProfilerDeployer completed in 191.11s
Running pipeline stage MKMLProfilerRunner
script pods %s
Pipeline stage MKMLProfilerRunner completed in 0.40s
Running pipeline stage MKMLProfilerDeleter
Checking if service nousresearch-meta-llama-4939-v45-profiler is running
Tearing down inference service nousresearch-meta-llama-4939-v45-profiler
Service nousresearch-meta-llama-4939-v45-profiler has been torndown
Pipeline stage MKMLProfilerDeleter completed in 1.88s
nousresearch-meta-llama_4939_v45 status is now inactive due to admin request
nousresearch-meta-llama_4939_v45 status is now torndown due to DeploymentManager action