developer_uid: junhua024
submission_id: junhua024-chai-16-full-_69709_v2
model_name: junhua024-chai-16-full-_69709_v2
model_group: junhua024/chai_16_full_1
status: torndown
timestamp: 2025-07-19T16:42:03+00:00
num_battles: 8523
num_wins: 4487
celo_rating: 1274.4
family_friendly_score: 0.5542
family_friendly_standard_error: 0.007029400543431851
submission_type: basic
model_repo: junhua024/chai_16_full_104_o_ffn_1925
model_architecture: MistralForCausalLM
model_num_parameters: 12772070400.0
best_of: 8
max_input_tokens: 1024
max_output_tokens: 64
reward_model: default
latencies: [{'batch_size': 1, 'throughput': 0.5998966147258309, 'latency_mean': 1.6668483877182008, 'latency_p50': 1.6536779403686523, 'latency_p90': 1.841061520576477}, {'batch_size': 3, 'throughput': 1.0803409433560678, 'latency_mean': 2.769555119276047, 'latency_p50': 2.755764126777649, 'latency_p90': 3.1413163423538206}, {'batch_size': 5, 'throughput': 1.2666298881401659, 'latency_mean': 3.9260830771923065, 'latency_p50': 3.9172223806381226, 'latency_p90': 4.451226544380188}, {'batch_size': 6, 'throughput': 1.3390328388256223, 'latency_mean': 4.460163662433624, 'latency_p50': 4.457961797714233, 'latency_p90': 5.01838321685791}, {'batch_size': 8, 'throughput': 1.4070547085449288, 'latency_mean': 5.658371437788009, 'latency_p50': 5.689339280128479, 'latency_p90': 6.3265187501907345}, {'batch_size': 10, 'throughput': 1.438273962336588, 'latency_mean': 6.892365409135818, 'latency_p50': 6.903557181358337, 'latency_p90': 7.867212343215942}]
gpu_counts: {'NVIDIA RTX A5000': 1}
display_name: junhua024-chai-16-full-_69709_v2
is_internal_developer: False
language_model: junhua024/chai_16_full_104_o_ffn_1925
model_size: 13B
ranking_group: single
throughput_3p7s: 1.25
us_pacific_date: 2025-07-19
win_ratio: 0.5264578200164262
generation_params: {'temperature': 1.0, 'top_p': 0.88, 'min_p': 0.0, 'top_k': 10, 'presence_penalty': 0.0, 'frequency_penalty': 0.0, 'stopping_words': ['\n'], 'max_input_tokens': 1024, 'best_of': 8, 'max_output_tokens': 64}
formatter: {'memory_template': "{bot_name}'s Persona: {memory}\n####\n", 'prompt_template': '{prompt}\n<START>\n', 'bot_template': '{bot_name}: {message}\n', 'user_template': '{user_name}: {message}\n', 'response_template': '{bot_name}:', 'truncate_by_message': False}
Resubmit model
Shutdown handler not registered because Python interpreter is not running in the main thread
run pipeline %s
run pipeline stage %s
Running pipeline stage MKMLizer
Starting job with name junhua024-chai-16-full-69709-v2-mkmlizer
Waiting for job on junhua024-chai-16-full-69709-v2-mkmlizer to finish
junhua024-chai-16-full-69709-v2-mkmlizer: ╔═════════════════════════════════════════════════════════════════════╗
junhua024-chai-16-full-69709-v2-mkmlizer: ║                                                                     ║
junhua024-chai-16-full-69709-v2-mkmlizer: ║   ██████   ██████  █████   ████  ████                               ║
junhua024-chai-16-full-69709-v2-mkmlizer: ║  ░░██████ ██████  ░░███   ███░  ░░███                               ║
junhua024-chai-16-full-69709-v2-mkmlizer: ║   ░███░█████░███   ░███  ███     ░███                               ║
junhua024-chai-16-full-69709-v2-mkmlizer: ║   ░███░░███ ░███   ░███████      ░███                               ║
junhua024-chai-16-full-69709-v2-mkmlizer: ║   ░███ ░░░  ░███   ░███░░███     ░███                               ║
junhua024-chai-16-full-69709-v2-mkmlizer: ║   ░███      ░███   ░███ ░░███    ░███                               ║
junhua024-chai-16-full-69709-v2-mkmlizer: ║   █████     █████  █████ ░░████  █████                              ║
junhua024-chai-16-full-69709-v2-mkmlizer: ║  ░░░░░     ░░░░░  ░░░░░   ░░░░  ░░░░░                               ║
junhua024-chai-16-full-69709-v2-mkmlizer: ║                                                                     ║
junhua024-chai-16-full-69709-v2-mkmlizer: ║  Version: 0.29.15                                                   ║
junhua024-chai-16-full-69709-v2-mkmlizer: ║  Features: FLYWHEEL, CUDA                                           ║
junhua024-chai-16-full-69709-v2-mkmlizer: ║  Copyright 2023-2025 MK ONE TECHNOLOGIES Inc.                       ║
junhua024-chai-16-full-69709-v2-mkmlizer: ║  https://mk1.ai                                                     ║
junhua024-chai-16-full-69709-v2-mkmlizer: ║                                                                     ║
junhua024-chai-16-full-69709-v2-mkmlizer: ║  The license key for the current software has been verified as      ║
junhua024-chai-16-full-69709-v2-mkmlizer: ║  belonging to:                                                      ║
junhua024-chai-16-full-69709-v2-mkmlizer: ║                                                                     ║
junhua024-chai-16-full-69709-v2-mkmlizer: ║  Chai Research Corp.                                                ║
junhua024-chai-16-full-69709-v2-mkmlizer: ║  Account ID: 7997a29f-0ceb-4cc7-9adf-840c57b4ae6f                   ║
junhua024-chai-16-full-69709-v2-mkmlizer: ║  Expiration: 2028-03-31 23:59:59                                    ║
junhua024-chai-16-full-69709-v2-mkmlizer: ║                                                                     ║
junhua024-chai-16-full-69709-v2-mkmlizer: ╚═════════════════════════════════════════════════════════════════════╝
junhua024-chai-16-full-69709-v2-mkmlizer: Xet Storage is enabled for this repo, but the 'hf_xet' package is not installed. Falling back to regular HTTP download. For better performance, install the package with: `pip install huggingface_hub[hf_xet]` or `pip install hf_xet`
junhua024-chai-16-full-69709-v2-mkmlizer: Xet Storage is enabled for this repo, but the 'hf_xet' package is not installed. Falling back to regular HTTP download. For better performance, install the package with: `pip install huggingface_hub[hf_xet]` or `pip install hf_xet`
Failed to get response for submission junhua024-chai-16-full-_74386_v3: HTTPConnectionPool(host='junhua024-chai-16-full-74386-v3-predictor.tenant-chaiml-guanaco.k.chaiverse.com', port=80): Read timed out. (read timeout=12.0)
junhua024-chai-16-full-69709-v2-mkmlizer: Xet Storage is enabled for this repo, but the 'hf_xet' package is not installed. Falling back to regular HTTP download. For better performance, install the package with: `pip install huggingface_hub[hf_xet]` or `pip install hf_xet`
junhua024-chai-16-full-69709-v2-mkmlizer: Xet Storage is enabled for this repo, but the 'hf_xet' package is not installed. Falling back to regular HTTP download. For better performance, install the package with: `pip install huggingface_hub[hf_xet]` or `pip install hf_xet`
junhua024-chai-16-full-69709-v2-mkmlizer: Xet Storage is enabled for this repo, but the 'hf_xet' package is not installed. Falling back to regular HTTP download. For better performance, install the package with: `pip install huggingface_hub[hf_xet]` or `pip install hf_xet`
junhua024-chai-16-full-69709-v2-mkmlizer: Xet Storage is enabled for this repo, but the 'hf_xet' package is not installed. Falling back to regular HTTP download. For better performance, install the package with: `pip install huggingface_hub[hf_xet]` or `pip install hf_xet`
junhua024-chai-16-full-69709-v2-mkmlizer: Xet Storage is enabled for this repo, but the 'hf_xet' package is not installed. Falling back to regular HTTP download. For better performance, install the package with: `pip install huggingface_hub[hf_xet]` or `pip install hf_xet`
junhua024-chai-16-full-69709-v2-mkmlizer: Xet Storage is enabled for this repo, but the 'hf_xet' package is not installed. Falling back to regular HTTP download. For better performance, install the package with: `pip install huggingface_hub[hf_xet]` or `pip install hf_xet`
junhua024-chai-16-full-69709-v2-mkmlizer: Xet Storage is enabled for this repo, but the 'hf_xet' package is not installed. Falling back to regular HTTP download. For better performance, install the package with: `pip install huggingface_hub[hf_xet]` or `pip install hf_xet`
junhua024-chai-16-full-69709-v2-mkmlizer: Xet Storage is enabled for this repo, but the 'hf_xet' package is not installed. Falling back to regular HTTP download. For better performance, install the package with: `pip install huggingface_hub[hf_xet]` or `pip install hf_xet`
junhua024-chai-16-full-69709-v2-mkmlizer: Xet Storage is enabled for this repo, but the 'hf_xet' package is not installed. Falling back to regular HTTP download. For better performance, install the package with: `pip install huggingface_hub[hf_xet]` or `pip install hf_xet`
junhua024-chai-16-full-69709-v2-mkmlizer: Xet Storage is enabled for this repo, but the 'hf_xet' package is not installed. Falling back to regular HTTP download. For better performance, install the package with: `pip install huggingface_hub[hf_xet]` or `pip install hf_xet`
junhua024-chai-16-full-69709-v2-mkmlizer: Xet Storage is enabled for this repo, but the 'hf_xet' package is not installed. Falling back to regular HTTP download. For better performance, install the package with: `pip install huggingface_hub[hf_xet]` or `pip install hf_xet`
junhua024-chai-16-full-69709-v2-mkmlizer: Xet Storage is enabled for this repo, but the 'hf_xet' package is not installed. Falling back to regular HTTP download. For better performance, install the package with: `pip install huggingface_hub[hf_xet]` or `pip install hf_xet`
junhua024-chai-16-full-69709-v2-mkmlizer: Xet Storage is enabled for this repo, but the 'hf_xet' package is not installed. Falling back to regular HTTP download. For better performance, install the package with: `pip install huggingface_hub[hf_xet]` or `pip install hf_xet`
junhua024-chai-16-full-69709-v2-mkmlizer: Xet Storage is enabled for this repo, but the 'hf_xet' package is not installed. Falling back to regular HTTP download. For better performance, install the package with: `pip install huggingface_hub[hf_xet]` or `pip install hf_xet`
Failed to get response for submission junhua024-chai-16-full-_74386_v3: HTTPConnectionPool(host='junhua024-chai-16-full-74386-v3-predictor.tenant-chaiml-guanaco.k.chaiverse.com', port=80): Read timed out. (read timeout=12.0)
junhua024-chai-16-full-69709-v2-mkmlizer: Xet Storage is enabled for this repo, but the 'hf_xet' package is not installed. Falling back to regular HTTP download. For better performance, install the package with: `pip install huggingface_hub[hf_xet]` or `pip install hf_xet`
junhua024-chai-16-full-69709-v2-mkmlizer: Xet Storage is enabled for this repo, but the 'hf_xet' package is not installed. Falling back to regular HTTP download. For better performance, install the package with: `pip install huggingface_hub[hf_xet]` or `pip install hf_xet`
junhua024-chai-16-full-69709-v2-mkmlizer: Xet Storage is enabled for this repo, but the 'hf_xet' package is not installed. Falling back to regular HTTP download. For better performance, install the package with: `pip install huggingface_hub[hf_xet]` or `pip install hf_xet`
junhua024-chai-16-full-69709-v2-mkmlizer: Xet Storage is enabled for this repo, but the 'hf_xet' package is not installed. Falling back to regular HTTP download. For better performance, install the package with: `pip install huggingface_hub[hf_xet]` or `pip install hf_xet`
Failed to get response for submission junhua024-chai-16-full-_74386_v4: HTTPConnectionPool(host='junhua024-chai-16-full-74386-v4-predictor.tenant-chaiml-guanaco.k.chaiverse.com', port=80): Read timed out. (read timeout=12.0)
Failed to get response for submission junhua024-chai-16-full-_74386_v3: HTTPConnectionPool(host='junhua024-chai-16-full-74386-v3-predictor.tenant-chaiml-guanaco.k.chaiverse.com', port=80): Read timed out. (read timeout=12.0)
junhua024-chai-16-full-69709-v2-mkmlizer: Xet Storage is enabled for this repo, but the 'hf_xet' package is not installed. Falling back to regular HTTP download. For better performance, install the package with: `pip install huggingface_hub[hf_xet]` or `pip install hf_xet`
junhua024-chai-16-full-69709-v2-mkmlizer: Xet Storage is enabled for this repo, but the 'hf_xet' package is not installed. Falling back to regular HTTP download. For better performance, install the package with: `pip install huggingface_hub[hf_xet]` or `pip install hf_xet`
junhua024-chai-16-full-69709-v2-mkmlizer: Xet Storage is enabled for this repo, but the 'hf_xet' package is not installed. Falling back to regular HTTP download. For better performance, install the package with: `pip install huggingface_hub[hf_xet]` or `pip install hf_xet`
junhua024-chai-16-full-69709-v2-mkmlizer: Xet Storage is enabled for this repo, but the 'hf_xet' package is not installed. Falling back to regular HTTP download. For better performance, install the package with: `pip install huggingface_hub[hf_xet]` or `pip install hf_xet`
junhua024-chai-16-full-69709-v2-mkmlizer: Xet Storage is enabled for this repo, but the 'hf_xet' package is not installed. Falling back to regular HTTP download. For better performance, install the package with: `pip install huggingface_hub[hf_xet]` or `pip install hf_xet`
junhua024-chai-16-full-69709-v2-mkmlizer: Downloaded to shared memory in 130.248s
junhua024-chai-16-full-69709-v2-mkmlizer: Checking if junhua024/chai_16_full_104_o_ffn_1925 already exists in ChaiML
junhua024-chai-16-full-69709-v2-mkmlizer: quantizing model to /dev/shm/model_cache, profile:s0, folder:/tmp/tmp39mkiwa0, device:0
junhua024-chai-16-full-69709-v2-mkmlizer: Saving flywheel model at /dev/shm/model_cache
Failed to get response for submission junhua024-chai-16-full-_74386_v1: HTTPConnectionPool(host='junhua024-chai-16-full-74386-v1-predictor.tenant-chaiml-guanaco.k.chaiverse.com', port=80): Read timed out. (read timeout=12.0)
junhua024-chai-16-full-69709-v2-mkmlizer: quantized model in 32.173s
junhua024-chai-16-full-69709-v2-mkmlizer: Processed model junhua024/chai_16_full_104_o_ffn_1925 in 162.501s
junhua024-chai-16-full-69709-v2-mkmlizer: creating bucket guanaco-mkml-models
junhua024-chai-16-full-69709-v2-mkmlizer: Bucket 's3://guanaco-mkml-models/' created
junhua024-chai-16-full-69709-v2-mkmlizer: uploading /dev/shm/model_cache to s3://guanaco-mkml-models/junhua024-chai-16-full-69709-v2/nvidia
junhua024-chai-16-full-69709-v2-mkmlizer: cp /dev/shm/model_cache/config.json s3://guanaco-mkml-models/junhua024-chai-16-full-69709-v2/nvidia/config.json
junhua024-chai-16-full-69709-v2-mkmlizer: cp /dev/shm/model_cache/special_tokens_map.json s3://guanaco-mkml-models/junhua024-chai-16-full-69709-v2/nvidia/special_tokens_map.json
junhua024-chai-16-full-69709-v2-mkmlizer: cp /dev/shm/model_cache/tokenizer_config.json s3://guanaco-mkml-models/junhua024-chai-16-full-69709-v2/nvidia/tokenizer_config.json
junhua024-chai-16-full-69709-v2-mkmlizer: cp /dev/shm/model_cache/tokenizer.json s3://guanaco-mkml-models/junhua024-chai-16-full-69709-v2/nvidia/tokenizer.json
Job junhua024-chai-16-full-69709-v2-mkmlizer completed after 193.09s with status: succeeded
Stopping job with name junhua024-chai-16-full-69709-v2-mkmlizer
Pipeline stage MKMLizer completed in 193.63s
run pipeline stage %s
Running pipeline stage MKMLTemplater
Pipeline stage MKMLTemplater completed in 0.16s
run pipeline stage %s
Running pipeline stage MKMLDeployer
Creating inference service junhua024-chai-16-full-69709-v2
Waiting for inference service junhua024-chai-16-full-69709-v2 to be ready
Failed to get response for submission chaiml-nis-qwen32b-sim_98336_v34: HTTPConnectionPool(host='chaiml-nis-qwen32b-sim-98336-v34-predictor.tenant-chaiml-guanaco.k.chaiverse.com', port=80): Read timed out. (read timeout=12.0)
Failed to get response for submission junhua024-chai-16-full-_74386_v4: HTTPConnectionPool(host='junhua024-chai-16-full-74386-v4-predictor.tenant-chaiml-guanaco.k.chaiverse.com', port=80): Read timed out. (read timeout=12.0)
Failed to get response for submission junhua024-chai-16-full-_74386_v2: HTTPConnectionPool(host='junhua024-chai-16-full-74386-v2-predictor.tenant-chaiml-guanaco.k.chaiverse.com', port=80): Read timed out. (read timeout=12.0)
Failed to get response for submission junhua024-chai-16-full-_74386_v2: HTTPConnectionPool(host='junhua024-chai-16-full-74386-v2-predictor.tenant-chaiml-guanaco.k.chaiverse.com', port=80): Read timed out. (read timeout=12.0)
Failed to get response for submission junhua024-chai-16-full-_74386_v4: HTTPConnectionPool(host='junhua024-chai-16-full-74386-v4-predictor.tenant-chaiml-guanaco.k.chaiverse.com', port=80): Read timed out. (read timeout=12.0)
Failed to get response for submission junhua024-chai-16-full-_74386_v1: HTTPConnectionPool(host='junhua024-chai-16-full-74386-v1-predictor.tenant-chaiml-guanaco.k.chaiverse.com', port=80): Read timed out. (read timeout=12.0)
Inference service junhua024-chai-16-full-69709-v2 ready after 332.3664755821228s
Pipeline stage MKMLDeployer completed in 332.95s
run pipeline stage %s
Running pipeline stage StressChecker
Received healthy response to inference request in 2.4765331745147705s
Received healthy response to inference request in 1.7786586284637451s
Failed to get response for submission junhua024-chai-16-full-_74386_v4: HTTPConnectionPool(host='junhua024-chai-16-full-74386-v4-predictor.tenant-chaiml-guanaco.k.chaiverse.com', port=80): Read timed out. (read timeout=12.0)
Received healthy response to inference request in 1.5080583095550537s
Received healthy response to inference request in 1.5903255939483643s
Received healthy response to inference request in 1.7500412464141846s
5 requests
0 failed requests
5th percentile: 1.5245117664337158
10th percentile: 1.5409652233123778
20th percentile: 1.5738721370697022
30th percentile: 1.6222687244415284
40th percentile: 1.6861549854278564
50th percentile: 1.7500412464141846
60th percentile: 1.7614881992340088
70th percentile: 1.772935152053833
80th percentile: 1.9182335376739503
90th percentile: 2.1973833560943605
95th percentile: 2.336958265304565
99th percentile: 2.4486181926727295
mean time: 1.8207233905792237
Pipeline stage StressChecker completed in 10.69s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyTriggerPipeline
run_pipeline:run_in_cloud %s
starting trigger_guanaco_pipeline args=%s
triggered trigger_guanaco_pipeline args=%s
Pipeline stage OfflineFamilyFriendlyTriggerPipeline completed in 0.70s
run pipeline stage %s
Running pipeline stage TriggerMKMLProfilingPipeline
run_pipeline:run_in_cloud %s
starting trigger_guanaco_pipeline args=%s
triggered trigger_guanaco_pipeline args=%s
Pipeline stage TriggerMKMLProfilingPipeline completed in 0.68s
Shutdown handler de-registered
junhua024-chai-16-full-_69709_v2 status is now deployed due to DeploymentManager action
Shutdown handler registered
run pipeline %s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyScorer
Evaluating %s Family Friendly Score with %s threads
%s, retrying in %s seconds...
Evaluating %s Family Friendly Score with %s threads
%s, retrying in %s seconds...
Evaluating %s Family Friendly Score with %s threads
clean up pipeline due to error=DeploymentChecksError('None: None')
Shutdown handler de-registered
Shutdown handler registered
run pipeline %s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyScorer
Evaluating %s Family Friendly Score with %s threads
%s, retrying in %s seconds...
Evaluating %s Family Friendly Score with %s threads
%s, retrying in %s seconds...
Evaluating %s Family Friendly Score with %s threads
Pipeline stage OfflineFamilyFriendlyScorer completed in 4659.14s
Shutdown handler de-registered
junhua024-chai-16-full-_69709_v2 status is now inactive due to auto deactivation removed underperforming models
junhua024-chai-16-full-_69709_v2 status is now torndown due to DeploymentManager action