chaiml-q80b-a3b-chai-11_41637

developer_uid: chai_backend_admin

submission_id: chaiml-q80b-a3b-chai-11_41637_v4

model_name: chaiml-q80b-a3b-chai-11_41637_v4

model_group: ChaiML/Q80B-A3B-CHAI-11K

status: protected

timestamp: 2025-11-25T00:21:45+00:00

num_battles: 5639

num_wins: 2909

celo_rating: 1318.82

family_friendly_score: 0.0

family_friendly_standard_error: 0.0

submission_type: basic

model_repo: ChaiML/Q80B-A3B-CHAI-11K-Filtered-Payload-EP2

model_architecture: Qwen3NextForCausalLM

model_num_parameters: 2585462784.0

best_of: 8

max_input_tokens: 2048

max_output_tokens: 70

reward_model: default

display_name: chaiml-q80b-a3b-chai-11_41637_v4

ineligible_reason: max_output_tokens!=64

is_internal_developer: True

language_model: ChaiML/Q80B-A3B-CHAI-11K-Filtered-Payload-EP2

model_size: 3B

ranking_group: single

us_pacific_date: 2025-11-24

win_ratio: 0.5158716084412129

generation_params: {'temperature': 1.0, 'top_p': 1.0, 'min_p': 0.0, 'top_k': 40, 'presence_penalty': 0.0, 'frequency_penalty': 0.0, 'stopping_words': ['<|im_end|>', '####'], 'max_input_tokens': 2048, 'best_of': 8, 'max_output_tokens': 70}

formatter: {'memory_template': "<|im_start|>system\n{bot_name}'s persona: {memory}<|im_end|>\n", 'prompt_template': '', 'bot_template': '<|im_start|>assistant\n{message}<|im_end|>\n', 'user_template': '<|im_start|>user\n{message}<|im_end|>\n', 'response_template': '<|im_start|>assistant\n', 'truncate_by_message': False}

Resubmit model

Shutdown handler not registered because Python interpreter is not running in the main thread
run pipeline %s
run pipeline stage %s
Running pipeline stage VLLMTemplater
Pipeline stage VLLMTemplater completed in 0.15s
run pipeline stage %s
Running pipeline stage VLLMDeployer
Creating inference service chaiml-q80b-a3b-chai-11-41637-v4
Waiting for inference service chaiml-q80b-a3b-chai-11-41637-v4 to be ready
Inference service chaiml-q80b-a3b-chai-11-41637-v4 ready after 481.188125371933s
Pipeline stage VLLMDeployer completed in 481.59s
run pipeline stage %s
Running pipeline stage StressChecker
{"detail":"HTTPConnectionPool(host='chaiml-q80b-a3b-chai-11-41637-v4-predictor.tenant-chaiml-guanaco.k2.chaiverse.com', port=80): Read timed out. (read timeout=12.0)"}
Received unhealthy response to inference request!
{"detail":"HTTPConnectionPool(host='chaiml-q80b-a3b-chai-11-41637-v4-predictor.tenant-chaiml-guanaco.k2.chaiverse.com', port=80): Read timed out. (read timeout=12.0)"}
Received unhealthy response to inference request!
{"detail":"HTTPConnectionPool(host='chaiml-q80b-a3b-chai-11-41637-v4-predictor.tenant-chaiml-guanaco.k2.chaiverse.com', port=80): Read timed out. (read timeout=12.0)"}
Received unhealthy response to inference request!
{"detail":"HTTPConnectionPool(host='chaiml-q80b-a3b-chai-11-41637-v4-predictor.tenant-chaiml-guanaco.k2.chaiverse.com', port=80): Read timed out. (read timeout=12.0)"}
Received unhealthy response to inference request!
{"detail":"HTTPConnectionPool(host='chaiml-q80b-a3b-chai-11-41637-v4-predictor.tenant-chaiml-guanaco.k2.chaiverse.com', port=80): Read timed out. (read timeout=12.0)"}
Received unhealthy response to inference request!
{"detail":"HTTPConnectionPool(host='chaiml-q80b-a3b-chai-11-41637-v4-predictor.tenant-chaiml-guanaco.k2.chaiverse.com', port=80): Read timed out. (read timeout=12.0)"}
Received unhealthy response to inference request!
{"detail":"HTTPConnectionPool(host='chaiml-q80b-a3b-chai-11-41637-v4-predictor.tenant-chaiml-guanaco.k2.chaiverse.com', port=80): Read timed out. (read timeout=12.0)"}
Received unhealthy response to inference request!
{"detail":"HTTPConnectionPool(host='chaiml-q80b-a3b-chai-11-41637-v4-predictor.tenant-chaiml-guanaco.k2.chaiverse.com', port=80): Read timed out. (read timeout=12.0)"}
Received unhealthy response to inference request!
Received healthy response to inference request in 2.1814122200012207s
Failed to get response for submission chaiml-2fe5-v1-dpo-lr15-b025_v2: HTTPConnectionPool(host='chaiml-2fe5-v1-dpo-lr15-b025-v2-predictor.tenant-chaiml-guanaco.kchai-coreweave-us-east-04a.chaiverse.com', port=80): Read timed out. (read timeout=12.0)
{"detail":"HTTPConnectionPool(host='chaiml-q80b-a3b-chai-11-41637-v4-predictor.tenant-chaiml-guanaco.k2.chaiverse.com', port=80): Read timed out. (read timeout=12.0)"}
Received unhealthy response to inference request!
Received healthy response to inference request in 2.3308005332946777s
Received healthy response to inference request in 10.284980773925781s
Received healthy response to inference request in 1.1638131141662598s
{"detail":"HTTPConnectionPool(host='chaiml-q80b-a3b-chai-11-41637-v4-predictor.tenant-chaiml-guanaco.k2.chaiverse.com', port=80): Read timed out. (read timeout=12.0)"}
Received unhealthy response to inference request!
Received healthy response to inference request in 1.848022699356079s
Received healthy response to inference request in 1.41282320022583s
Received healthy response to inference request in 1.9655303955078125s
Received healthy response to inference request in 1.466278076171875s
Received healthy response to inference request in 1.3331177234649658s
Received healthy response to inference request in 2.0441055297851562s
Received healthy response to inference request in 1.9758062362670898s
Received healthy response to inference request in 2.1701014041900635s
Received healthy response to inference request in 1.3516855239868164s
Received healthy response to inference request in 1.7328178882598877s
Received healthy response to inference request in 1.0865380764007568s
Received healthy response to inference request in 1.7277278900146484s
{"detail":"HTTPConnectionPool(host='chaiml-q80b-a3b-chai-11-41637-v4-predictor.tenant-chaiml-guanaco.k2.chaiverse.com', port=80): Read timed out. (read timeout=12.0)"}
Received unhealthy response to inference request!
Received healthy response to inference request in 1.130840539932251s
Received healthy response to inference request in 1.1246280670166016s
{"detail":"HTTPConnectionPool(host='chaiml-q80b-a3b-chai-11-41637-v4-predictor.tenant-chaiml-guanaco.k2.chaiverse.com', port=80): Read timed out. (read timeout=12.0)"}
Received unhealthy response to inference request!
30 requests
12 failed requests
5th percentile: 1.1274236798286439
10th percentile: 1.1605158567428588
20th percentile: 1.4005956649780273
30th percentile: 1.7312908887863159
40th percentile: 1.971695899963379
50th percentile: 2.175756812095642
60th percentile: 11.032228565216062
70th percentile: 12.164244747161865
80th percentile: 12.185458040237426
90th percentile: 12.260016942024231
95th percentile: 12.366966700553894
99th percentile: 12.405669963359832
mean time: 6.170587762196859
%s, retrying in %s seconds...
Received healthy response to inference request in 1.1687779426574707s
Received healthy response to inference request in 1.2242951393127441s
Received healthy response to inference request in 1.243717908859253s
Received healthy response to inference request in 10.86894702911377s
Received healthy response to inference request in 2.3587329387664795s
Received healthy response to inference request in 1.3535876274108887s
Received healthy response to inference request in 1.3692958354949951s
Received healthy response to inference request in 1.1726961135864258s
Received healthy response to inference request in 1.416722059249878s
Received healthy response to inference request in 1.4039950370788574s
Received healthy response to inference request in 1.766036033630371s
Received healthy response to inference request in 1.6583263874053955s
Received healthy response to inference request in 1.276099681854248s
Received healthy response to inference request in 1.413454294204712s
Received healthy response to inference request in 1.886552333831787s
Received healthy response to inference request in 1.8323924541473389s
Received healthy response to inference request in 1.4025957584381104s
Received healthy response to inference request in 1.2967565059661865s
Received healthy response to inference request in 1.377347707748413s
Received healthy response to inference request in 1.8858997821807861s
Received healthy response to inference request in 1.1023776531219482s
Received healthy response to inference request in 1.351776123046875s
Received healthy response to inference request in 1.152395248413086s
Received healthy response to inference request in 1.437596082687378s
Received healthy response to inference request in 1.7238340377807617s
Received healthy response to inference request in 1.1106388568878174s
Received healthy response to inference request in 1.1656348705291748s
Received healthy response to inference request in 1.9337646961212158s
Received healthy response to inference request in 1.7264552116394043s
Received healthy response to inference request in 1.442744493484497s
30 requests
0 failed requests
5th percentile: 1.1294292330741882
10th percentile: 1.164310908317566
20th percentile: 1.2139753341674806
30th percentile: 1.290559458732605
40th percentile: 1.3630125522613525
50th percentile: 1.4032953977584839
60th percentile: 1.4250716686248779
70th percentile: 1.6779786825180052
80th percentile: 1.7793073177337648
90th percentile: 1.8912735700607302
95th percentile: 2.1674972295761097
99th percentile: 8.400984942913063
mean time: 1.7841148614883422
Pipeline stage StressChecker completed in 242.94s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyTriggerPipeline
run_pipeline:run_in_cloud %s
starting trigger_guanaco_pipeline args=%s
triggered trigger_guanaco_pipeline args=%s
Pipeline stage OfflineFamilyFriendlyTriggerPipeline completed in 0.64s
Shutdown handler de-registered
chaiml-q80b-a3b-chai-11_41637_v4 status is now deployed due to DeploymentManager action
Shutdown handler registered
run pipeline %s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyScorer
Evaluating %s Family Friendly Score with %s threads
%s, retrying in %s seconds...
Evaluating %s Family Friendly Score with %s threads
%s, retrying in %s seconds...
Evaluating %s Family Friendly Score with %s threads
clean up pipeline due to error=DeploymentChecksError('None: None')
Shutdown handler de-registered
Shutdown handler registered
run pipeline %s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyScorer
Evaluating %s Family Friendly Score with %s threads
%s, retrying in %s seconds...
Evaluating %s Family Friendly Score with %s threads
%s, retrying in %s seconds...
Evaluating %s Family Friendly Score with %s threads
Pipeline stage OfflineFamilyFriendlyScorer completed in 5149.63s
Shutdown handler de-registered
chaiml-q80b-a3b-chai-11_41637_v4 status is now protected due to ABTestQueueItem
chaiml-q80b-a3b-chai-11_41637_v4 status is now protected due to ABTestQueueItem
chaiml-q80b-a3b-chai-11_41637_v4 status is now protected due to ABTestQueueItem