submission_id: rica40325-yui-new-chai_v9
developer_uid: rica40325
alignment_samples: 10412
alignment_score: 1.6667864057285793
best_of: 1
celo_rating: 1131.43
display_name: rica40325-yui-new-chai_v9
formatter: {'memory_template': "{bot_name}'s Persona: {memory}\n####\n", 'prompt_template': '{prompt}\n<START>\n', 'bot_template': '{bot_name}: {message}\n', 'user_template': '{user_name}: {message}\n', 'response_template': '{bot_name}:', 'truncate_by_message': False}
generation_params: {'temperature': 1.0, 'top_p': 1.0, 'min_p': 0.0, 'top_k': 40, 'presence_penalty': 0.0, 'frequency_penalty': 0.0, 'stopping_words': ['\n'], 'max_input_tokens': 512, 'best_of': 1, 'max_output_tokens': 64}
is_internal_developer: False
language_model: rica40325/yui-new-chai
max_input_tokens: 512
max_output_tokens: 64
model_architecture: LlamaForCausalLM
model_group: rica40325/yui-new-chai
model_name: rica40325-yui-new-chai_v9
model_num_parameters: 8030261248.0
model_repo: rica40325/yui-new-chai
model_size: 8B
num_battles: 10412
num_wins: 3676
propriety_score: 0.7315842583249244
propriety_total_count: 991.0
ranking_group: single
status: inactive
submission_type: basic
timestamp: 2024-08-24T16:47:55+00:00
us_pacific_date: 2024-08-24
win_ratio: 0.3530541682673838
Download Preference Data
Resubmit model
Running pipeline stage MKMLizer
Starting job with name rica40325-yui-new-chai-v9-mkmlizer
Waiting for job on rica40325-yui-new-chai-v9-mkmlizer to finish
rica40325-yui-new-chai-v9-mkmlizer: ╔═════════════════════════════════════════════════════════════════════╗
rica40325-yui-new-chai-v9-mkmlizer: ║ _____ __ __ ║
rica40325-yui-new-chai-v9-mkmlizer: ║ / _/ /_ ___ __/ / ___ ___ / / ║
rica40325-yui-new-chai-v9-mkmlizer: ║ / _/ / // / |/|/ / _ \/ -_) -_) / ║
rica40325-yui-new-chai-v9-mkmlizer: ║ /_//_/\_, /|__,__/_//_/\__/\__/_/ ║
rica40325-yui-new-chai-v9-mkmlizer: ║ /___/ ║
rica40325-yui-new-chai-v9-mkmlizer: ║ ║
rica40325-yui-new-chai-v9-mkmlizer: ║ Version: 0.10.1 ║
rica40325-yui-new-chai-v9-mkmlizer: ║ Copyright 2023 MK ONE TECHNOLOGIES Inc. ║
rica40325-yui-new-chai-v9-mkmlizer: ║ https://mk1.ai ║
rica40325-yui-new-chai-v9-mkmlizer: ║ ║
rica40325-yui-new-chai-v9-mkmlizer: ║ The license key for the current software has been verified as ║
rica40325-yui-new-chai-v9-mkmlizer: ║ belonging to: ║
rica40325-yui-new-chai-v9-mkmlizer: ║ ║
rica40325-yui-new-chai-v9-mkmlizer: ║ Chai Research Corp. ║
rica40325-yui-new-chai-v9-mkmlizer: ║ Account ID: 7997a29f-0ceb-4cc7-9adf-840c57b4ae6f ║
rica40325-yui-new-chai-v9-mkmlizer: ║ Expiration: 2024-10-15 23:59:59 ║
rica40325-yui-new-chai-v9-mkmlizer: ║ ║
rica40325-yui-new-chai-v9-mkmlizer: ╚═════════════════════════════════════════════════════════════════════╝
rica40325-yui-new-chai-v9-mkmlizer: Downloaded to shared memory in 20.514s
rica40325-yui-new-chai-v9-mkmlizer: quantizing model to /dev/shm/model_cache, profile:s0, folder:/tmp/tmpkq_drjpl, device:0
rica40325-yui-new-chai-v9-mkmlizer: Saving flywheel model at /dev/shm/model_cache
rica40325-yui-new-chai-v9-mkmlizer: quantized model in 25.740s
rica40325-yui-new-chai-v9-mkmlizer: Processed model rica40325/yui-new-chai in 46.254s
rica40325-yui-new-chai-v9-mkmlizer: creating bucket guanaco-mkml-models
rica40325-yui-new-chai-v9-mkmlizer: Bucket 's3://guanaco-mkml-models/' created
rica40325-yui-new-chai-v9-mkmlizer: uploading /dev/shm/model_cache to s3://guanaco-mkml-models/rica40325-yui-new-chai-v9
rica40325-yui-new-chai-v9-mkmlizer: cp /dev/shm/model_cache/special_tokens_map.json s3://guanaco-mkml-models/rica40325-yui-new-chai-v9/special_tokens_map.json
rica40325-yui-new-chai-v9-mkmlizer: cp /dev/shm/model_cache/config.json s3://guanaco-mkml-models/rica40325-yui-new-chai-v9/config.json
rica40325-yui-new-chai-v9-mkmlizer: cp /dev/shm/model_cache/tokenizer_config.json s3://guanaco-mkml-models/rica40325-yui-new-chai-v9/tokenizer_config.json
rica40325-yui-new-chai-v9-mkmlizer: cp /dev/shm/model_cache/tokenizer.json s3://guanaco-mkml-models/rica40325-yui-new-chai-v9/tokenizer.json
rica40325-yui-new-chai-v9-mkmlizer: cp /dev/shm/model_cache/flywheel_model.0.safetensors s3://guanaco-mkml-models/rica40325-yui-new-chai-v9/flywheel_model.0.safetensors
Job rica40325-yui-new-chai-v9-mkmlizer completed after 74.41s with status: succeeded
Stopping job with name rica40325-yui-new-chai-v9-mkmlizer
Pipeline stage MKMLizer completed in 75.30s
Running pipeline stage MKMLKubeTemplater
Pipeline stage MKMLKubeTemplater completed in 0.10s
Running pipeline stage ISVCDeployer
Creating inference service rica40325-yui-new-chai-v9
Waiting for inference service rica40325-yui-new-chai-v9 to be ready
Failed to get response for submission blend_remul_2024-08-22: ('http://zonemercy-lexical-nemo-1518-v18-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'request timeout')
Failed to get response for submission blend_susol_2024-08-22: ('http://zonemercy-lexical-nemo-1518-v18-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'request timeout')
Failed to get response for submission blend_dedat_2024-08-16: ('http://zonemercy-lexical-nemo-1518-v18-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'request timeout')
Failed to get response for submission blend_remul_2024-08-22: ('http://zonemercy-lexical-nemo-1518-v18-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'request timeout')
Failed to get response for submission blend_lobuf_2024-08-22: ('http://zonemercy-lexical-nemo-1518-v18-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'request timeout')
Failed to get response for submission blend_fedek_2024-08-24: ('http://zonemercy-lexical-nemo-1518-v18-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'request timeout')
Inference service rica40325-yui-new-chai-v9 ready after 211.57794952392578s
Pipeline stage ISVCDeployer completed in 212.40s
Running pipeline stage StressChecker
Received healthy response to inference request in 1.786010503768921s
Received healthy response to inference request in 1.089932918548584s
Received healthy response to inference request in 3.1043028831481934s
Received healthy response to inference request in 1.7606492042541504s
Received healthy response to inference request in 0.8891234397888184s
5 requests
0 failed requests
5th percentile: 0.9292853355407715
10th percentile: 0.9694472312927246
20th percentile: 1.0497710227966308
30th percentile: 1.2240761756896972
40th percentile: 1.492362689971924
50th percentile: 1.7606492042541504
60th percentile: 1.7707937240600586
70th percentile: 1.7809382438659669
80th percentile: 2.0496689796447756
90th percentile: 2.5769859313964845
95th percentile: 2.8406444072723387
99th percentile: 3.0515711879730225
mean time: 1.7260037899017333
Pipeline stage StressChecker completed in 9.42s
rica40325-yui-new-chai_v9 status is now deployed due to DeploymentManager action
rica40325-yui-new-chai_v9 status is now inactive due to auto deactivation removed underperforming models

Usage Metrics

Latency Metrics