submission_id: chaiml-llama-8b-pairwis_8189_v34
developer_uid: chai_backend_admin
best_of: 1
celo_rating: 1238.42
display_name: chaiml-llama-8b-pairwis_8189_v34
family_friendly_score: 0.5660712948401108
family_friendly_standard_error: 0.005220918840525811
formatter: {'memory_template': '', 'prompt_template': '', 'bot_template': '{bot_name}: {message}\n', 'user_template': '{user_name}: {message}\n', 'response_template': '', 'truncate_by_message': True}
generation_params: {'temperature': 1.0, 'top_p': 1.0, 'min_p': 0.0, 'top_k': 40, 'presence_penalty': 0.0, 'frequency_penalty': 0.0, 'stopping_words': ['\n'], 'max_input_tokens': 128, 'best_of': 1, 'max_output_tokens': 1}
ineligible_reason: max_output_tokens!=64
is_internal_developer: True
language_model: ChaiML/llama_8b_pairwise_64m_256_tokens_step_249984
max_input_tokens: 128
max_output_tokens: 1
model_architecture: LlamaForSequenceClassification
model_group: ChaiML/llama_8b_pairwise
model_name: chaiml-llama-8b-pairwis_8189_v34
model_num_parameters: 8030261248.0
model_repo: ChaiML/llama_8b_pairwise_64m_256_tokens_step_249984
model_size: 8B
num_battles: 9712
num_wins: 4689
ranking_group: single
status: torndown
submission_type: basic
timestamp: 2024-10-13T23:28:45+00:00
us_pacific_date: 2024-10-13
win_ratio: 0.4828047775947282
Download Preference Data
Resubmit model
Shutdown handler not registered because Python interpreter is not running in the main thread
run pipeline %s
run pipeline stage %s
Running pipeline stage MKMLizer
Starting job with name chaiml-llama-8b-pairwis-8189-v34-mkmlizer
Waiting for job on chaiml-llama-8b-pairwis-8189-v34-mkmlizer to finish
chaiml-llama-8b-pairwis-8189-v34-mkmlizer: ╔═════════════════════════════════════════════════════════════════════╗
chaiml-llama-8b-pairwis-8189-v34-mkmlizer: ║     _____            __           __                                ║
chaiml-llama-8b-pairwis-8189-v34-mkmlizer: ║    / _/ /_ ___    __/ /  ___ ___ / /                                ║
chaiml-llama-8b-pairwis-8189-v34-mkmlizer: ║   / _/ / // / |/|/ / _ \/ -_) -_) /                                 ║
chaiml-llama-8b-pairwis-8189-v34-mkmlizer: ║  /_//_/\_, /|__,__/_//_/\__/\__/_/                                  ║
chaiml-llama-8b-pairwis-8189-v34-mkmlizer: ║       /___/                                                         ║
chaiml-llama-8b-pairwis-8189-v34-mkmlizer: ║                                                                     ║
chaiml-llama-8b-pairwis-8189-v34-mkmlizer: ║  Version: 0.11.12                                                   ║
chaiml-llama-8b-pairwis-8189-v34-mkmlizer: ║  Copyright 2023 MK ONE TECHNOLOGIES Inc.                            ║
chaiml-llama-8b-pairwis-8189-v34-mkmlizer: ║  https://mk1.ai                                                     ║
chaiml-llama-8b-pairwis-8189-v34-mkmlizer: ║                                                                     ║
chaiml-llama-8b-pairwis-8189-v34-mkmlizer: ║  The license key for the current software has been verified as      ║
chaiml-llama-8b-pairwis-8189-v34-mkmlizer: ║  belonging to:                                                      ║
chaiml-llama-8b-pairwis-8189-v34-mkmlizer: ║                                                                     ║
chaiml-llama-8b-pairwis-8189-v34-mkmlizer: ║  Chai Research Corp.                                                ║
chaiml-llama-8b-pairwis-8189-v34-mkmlizer: ║  Account ID: 7997a29f-0ceb-4cc7-9adf-840c57b4ae6f                   ║
chaiml-llama-8b-pairwis-8189-v34-mkmlizer: ║  Expiration: 2024-10-15 23:59:59                                    ║
chaiml-llama-8b-pairwis-8189-v34-mkmlizer: ║                                                                     ║
chaiml-llama-8b-pairwis-8189-v34-mkmlizer: ╚═════════════════════════════════════════════════════════════════════╝
chaiml-llama-8b-pairwis-8189-v34-mkmlizer: Downloaded to shared memory in 21.896s
chaiml-llama-8b-pairwis-8189-v34-mkmlizer: quantizing model to /dev/shm/model_cache, profile:t0, folder:/tmp/tmplsfos9rt, device:0
chaiml-llama-8b-pairwis-8189-v34-mkmlizer: Saving flywheel model at /dev/shm/model_cache
chaiml-llama-8b-pairwis-8189-v34-mkmlizer: quantized model in 83.604s
chaiml-llama-8b-pairwis-8189-v34-mkmlizer: Processed model ChaiML/llama_8b_pairwise_64m_256_tokens_step_249984 in 105.500s
chaiml-llama-8b-pairwis-8189-v34-mkmlizer: creating bucket guanaco-mkml-models
chaiml-llama-8b-pairwis-8189-v34-mkmlizer: Bucket 's3://guanaco-mkml-models/' created
chaiml-llama-8b-pairwis-8189-v34-mkmlizer: uploading /dev/shm/model_cache to s3://guanaco-mkml-models/chaiml-llama-8b-pairwis-8189-v34
chaiml-llama-8b-pairwis-8189-v34-mkmlizer: cp /dev/shm/model_cache/config.json s3://guanaco-mkml-models/chaiml-llama-8b-pairwis-8189-v34/config.json
chaiml-llama-8b-pairwis-8189-v34-mkmlizer: cp /dev/shm/model_cache/special_tokens_map.json s3://guanaco-mkml-models/chaiml-llama-8b-pairwis-8189-v34/special_tokens_map.json
chaiml-llama-8b-pairwis-8189-v34-mkmlizer: cp /dev/shm/model_cache/tokenizer_config.json s3://guanaco-mkml-models/chaiml-llama-8b-pairwis-8189-v34/tokenizer_config.json
chaiml-llama-8b-pairwis-8189-v34-mkmlizer: cp /dev/shm/model_cache/tokenizer.json s3://guanaco-mkml-models/chaiml-llama-8b-pairwis-8189-v34/tokenizer.json
Job chaiml-llama-8b-pairwis-8189-v34-mkmlizer completed after 132.6s with status: succeeded
Stopping job with name chaiml-llama-8b-pairwis-8189-v34-mkmlizer
Pipeline stage MKMLizer completed in 134.01s
run pipeline stage %s
Running pipeline stage MKMLTemplater
Pipeline stage MKMLTemplater completed in 0.43s
run pipeline stage %s
Running pipeline stage MKMLDeployer
Creating inference service chaiml-llama-8b-pairwis-8189-v34
Waiting for inference service chaiml-llama-8b-pairwis-8189-v34 to be ready
Inference service chaiml-llama-8b-pairwis-8189-v34 ready after 172.51803588867188s
Pipeline stage MKMLDeployer completed in 174.27s
run pipeline stage %s
Running pipeline stage StressChecker
HTTP Request: %s %s "%s %d %s"
Received healthy response to inference request in 6.7106688022613525s
HTTP Request: %s %s "%s %d %s"
Received healthy response to inference request in 2.790184259414673s
HTTP Request: %s %s "%s %d %s"
Received healthy response to inference request in 2.1871402263641357s
HTTP Request: %s %s "%s %d %s"
Received healthy response to inference request in 3.253571033477783s
HTTP Request: %s %s "%s %d %s"
Received healthy response to inference request in 3.6233129501342773s
5 requests
0 failed requests
5th percentile: 2.307749032974243
10th percentile: 2.4283578395843506
20th percentile: 2.6695754528045654
30th percentile: 2.882861614227295
40th percentile: 3.068216323852539
50th percentile: 3.253571033477783
60th percentile: 3.401467800140381
70th percentile: 3.5493645668029785
80th percentile: 4.240784120559693
90th percentile: 5.475726461410523
95th percentile: 6.093197631835937
99th percentile: 6.58717456817627
mean time: 3.7129754543304445
Pipeline stage StressChecker completed in 22.17s
Shutdown handler de-registered
chaiml-llama-8b-pairwis_8189_v34 status is now deployed due to DeploymentManager action
chaiml-llama-8b-pairwis_8189_v34 status is now inactive due to auto deactivation removed underperforming models
chaiml-llama-8b-pairwis_8189_v34 status is now torndown due to DeploymentManager action