chaiml-llama-8b-pairwis_8189

submission_id: chaiml-llama-8b-pairwis_8189_v14

developer_uid: chai_backend_admin

best_of: 1

celo_rating: 1224.55

display_name: chaiml-llama-8b-pairwis_8189_v14

family_friendly_score: 0.0

formatter: {'memory_template': '', 'prompt_template': '', 'bot_template': '{bot_name}: {message}\n', 'user_template': '{user_name}: {message}\n', 'response_template': '', 'truncate_by_message': True}

generation_params: {'temperature': 1.0, 'top_p': 1.0, 'min_p': 0.0, 'top_k': 40, 'presence_penalty': 0.0, 'frequency_penalty': 0.0, 'stopping_words': ['\n'], 'max_input_tokens': 256, 'best_of': 1, 'max_output_tokens': 1}

ineligible_reason: max_output_tokens!=64

is_internal_developer: True

language_model: ChaiML/llama_8b_pairwise_64m_256_tokens_step_249984

max_input_tokens: 256

max_output_tokens: 1

model_architecture: LlamaForSequenceClassification

model_group: ChaiML/llama_8b_pairwise

model_name: chaiml-llama-8b-pairwis_8189_v14

model_num_parameters: 8030261248.0

model_repo: ChaiML/llama_8b_pairwise_64m_256_tokens_step_249984

model_size: 8B

num_battles: 9952

num_wins: 4825

ranking_group: single

status: torndown

submission_type: basic

timestamp: 2024-08-17T05:08:37+00:00

us_pacific_date: 2024-08-16

win_ratio: 0.48482717041800644

Download Preference Data

Resubmit model

Running pipeline stage MKMLizer
Starting job with name chaiml-llama-8b-pairwis-8189-v14-mkmlizer
Waiting for job on chaiml-llama-8b-pairwis-8189-v14-mkmlizer to finish
chaiml-llama-8b-pairwis-8189-v14-mkmlizer: ╔═════════════════════════════════════════════════════════════════════╗
chaiml-llama-8b-pairwis-8189-v14-mkmlizer: ║     _____            __           __                                ║
chaiml-llama-8b-pairwis-8189-v14-mkmlizer: ║    / _/ /_ ___    __/ /  ___ ___ / /                                ║
chaiml-llama-8b-pairwis-8189-v14-mkmlizer: ║   / _/ / // / |/|/ / _ \/ -_) -_) /                                 ║
chaiml-llama-8b-pairwis-8189-v14-mkmlizer: ║  /_//_/\_, /|__,__/_//_/\__/\__/_/                                  ║
chaiml-llama-8b-pairwis-8189-v14-mkmlizer: ║       /___/                                                         ║
chaiml-llama-8b-pairwis-8189-v14-mkmlizer: ║                                                                     ║
chaiml-llama-8b-pairwis-8189-v14-mkmlizer: ║  Version: 0.9.11                                                    ║
chaiml-llama-8b-pairwis-8189-v14-mkmlizer: ║  Copyright 2023 MK ONE TECHNOLOGIES Inc.                            ║
chaiml-llama-8b-pairwis-8189-v14-mkmlizer: ║  https://mk1.ai                                                     ║
chaiml-llama-8b-pairwis-8189-v14-mkmlizer: ║                                                                     ║
chaiml-llama-8b-pairwis-8189-v14-mkmlizer: ║  The license key for the current software has been verified as      ║
chaiml-llama-8b-pairwis-8189-v14-mkmlizer: ║  belonging to:                                                      ║
chaiml-llama-8b-pairwis-8189-v14-mkmlizer: ║                                                                     ║
chaiml-llama-8b-pairwis-8189-v14-mkmlizer: ║  Chai Research Corp.                                                ║
chaiml-llama-8b-pairwis-8189-v14-mkmlizer: ║  Account ID: 7997a29f-0ceb-4cc7-9adf-840c57b4ae6f                   ║
chaiml-llama-8b-pairwis-8189-v14-mkmlizer: ║  Expiration: 2024-10-15 23:59:59                                    ║
chaiml-llama-8b-pairwis-8189-v14-mkmlizer: ║                                                                     ║
chaiml-llama-8b-pairwis-8189-v14-mkmlizer: ╚═════════════════════════════════════════════════════════════════════╝
chaiml-llama-8b-pairwis-8189-v14-mkmlizer: Downloaded to shared memory in 22.871s
chaiml-llama-8b-pairwis-8189-v14-mkmlizer: quantizing model to /dev/shm/model_cache, profile:t0, folder:/tmp/tmp9uenry68, device:0
chaiml-llama-8b-pairwis-8189-v14-mkmlizer: Saving flywheel model at /dev/shm/model_cache
chaiml-llama-8b-pairwis-8189-v14-mkmlizer: quantized model in 84.004s
chaiml-llama-8b-pairwis-8189-v14-mkmlizer: Processed model ChaiML/llama_8b_pairwise_64m_256_tokens_step_249984 in 106.876s
chaiml-llama-8b-pairwis-8189-v14-mkmlizer: creating bucket guanaco-mkml-models
chaiml-llama-8b-pairwis-8189-v14-mkmlizer: Bucket 's3://guanaco-mkml-models/' created
chaiml-llama-8b-pairwis-8189-v14-mkmlizer: uploading /dev/shm/model_cache to s3://guanaco-mkml-models/chaiml-llama-8b-pairwis-8189-v14
chaiml-llama-8b-pairwis-8189-v14-mkmlizer: cp /dev/shm/model_cache/config.json s3://guanaco-mkml-models/chaiml-llama-8b-pairwis-8189-v14/config.json
chaiml-llama-8b-pairwis-8189-v14-mkmlizer: cp /dev/shm/model_cache/special_tokens_map.json s3://guanaco-mkml-models/chaiml-llama-8b-pairwis-8189-v14/special_tokens_map.json
chaiml-llama-8b-pairwis-8189-v14-mkmlizer: cp /dev/shm/model_cache/tokenizer_config.json s3://guanaco-mkml-models/chaiml-llama-8b-pairwis-8189-v14/tokenizer_config.json
chaiml-llama-8b-pairwis-8189-v14-mkmlizer: cp /dev/shm/model_cache/tokenizer.json s3://guanaco-mkml-models/chaiml-llama-8b-pairwis-8189-v14/tokenizer.json
chaiml-llama-8b-pairwis-8189-v14-mkmlizer: cp /dev/shm/model_cache/flywheel_model.0.safetensors s3://guanaco-mkml-models/chaiml-llama-8b-pairwis-8189-v14/flywheel_model.0.safetensors
Job chaiml-llama-8b-pairwis-8189-v14-mkmlizer completed after 133.08s with status: succeeded
Stopping job with name chaiml-llama-8b-pairwis-8189-v14-mkmlizer
Pipeline stage MKMLizer completed in 135.01s
Running pipeline stage MKMLKubeTemplater
Pipeline stage MKMLKubeTemplater completed in 0.43s
Running pipeline stage ISVCDeployer
Creating inference service chaiml-llama-8b-pairwis-8189-v14
Waiting for inference service chaiml-llama-8b-pairwis-8189-v14 to be ready
Inference service chaiml-llama-8b-pairwis-8189-v14 ready after 233.79407691955566s
Pipeline stage ISVCDeployer completed in 235.44s
Running pipeline stage StressChecker
HTTP Request: %s %s "%s %d %s"
Received healthy response to inference request in 8.283552169799805s
HTTP Request: %s %s "%s %d %s"
Received healthy response to inference request in 5.238374710083008s
HTTP Request: %s %s "%s %d %s"
Received healthy response to inference request in 4.204387187957764s
HTTP Request: %s %s "%s %d %s"
Received healthy response to inference request in 3.9359030723571777s
HTTP Request: %s %s "%s %d %s"
Received healthy response to inference request in 5.234715223312378s
5 requests
0 failed requests
5th percentile: 3.989599895477295
10th percentile: 4.0432967185974125
20th percentile: 4.150690364837646
30th percentile: 4.410452795028687
40th percentile: 4.822584009170532
50th percentile: 5.234715223312378
60th percentile: 5.23617901802063
70th percentile: 5.237642812728882
80th percentile: 5.847410202026368
90th percentile: 7.065481185913086
95th percentile: 7.674516677856444
99th percentile: 8.161745071411133
mean time: 5.379386472702026
Pipeline stage StressChecker completed in 30.20s
chaiml-llama-8b-pairwis_8189_v14 status is now deployed due to DeploymentManager action
chaiml-llama-8b-pairwis_8189_v14 status is now inactive due to auto deactivation removed underperforming models
chaiml-llama-8b-pairwis_8189_v14 status is now torndown due to DeploymentManager action