chaiml-mega-v1-sonnetwi_11582

developer_uid: zonemercy
submission_id: chaiml-mega-v1-sonnetwi_11582_v3
model_name: chaiml-mega-v1-sonnetwi_11582_v3
model_group: ChaiML/mega-v1-sonnetwin
status: torndown
timestamp: 2026-03-31T15:56:09+00:00
num_battles: 8018
num_wins: 4122
celo_rating: 1304.69
family_friendly_score: 0.0
family_friendly_standard_error: 0.0
submission_type: basic
model_repo: ChaiML/mega-v1-sonnetwintop2-q27b-lr5e6ep2g8
model_architecture: Qwen3_5ForConditionalGeneration
model_num_parameters: 23564784640.0
best_of: 8
max_input_tokens: 2048
max_output_tokens: 80
reward_model: default
display_name: chaiml-mega-v1-sonnetwi_11582_v3
ineligible_reason: max_output_tokens!=64
is_internal_developer: True
language_model: ChaiML/mega-v1-sonnetwintop2-q27b-lr5e6ep2g8
model_size: 24B
ranking_group: single
us_pacific_date: 2026-03-28
win_ratio: 0.5140932900972811
generation_params: {'temperature': 1.0, 'top_p': 1.0, 'min_p': 0.0, 'top_k': 40, 'presence_penalty': 0.0, 'frequency_penalty': 0.0, 'stopping_words': ['<|im_end|>', '<|assistant|>', '<|user|>', '</s>', '####'], 'max_input_tokens': 2048, 'best_of': 8, 'max_output_tokens': 80}
formatter: {'memory_template': "<|im_start|>system\n{bot_name}'s persona: {memory}<|im_end|>\n", 'prompt_template': '', 'bot_template': '<|im_start|>assistant\n{bot_name}: {message}<|im_end|>\n', 'user_template': '<|im_start|>user\n{message}<|im_end|>\n', 'response_template': '<|im_start|>assistant\n{bot_name}:', 'truncate_by_message': True}
Resubmit model
Shutdown handler not registered because Python interpreter is not running in the main thread
run pipeline %s
run pipeline stage %s
Running pipeline stage VLLMUploader
Starting job with name chaiml-mega-v1-sonnetwi-11582-v3-uploader
Waiting for job on chaiml-mega-v1-sonnetwi-11582-v3-uploader to finish
chaiml-mega-v1-sonnetwi-11582-v3-uploader: Using quantization_mode: fp8
chaiml-mega-v1-sonnetwi-11582-v3-uploader: Checking if ChaiML/mega-v1-sonnetwintop2-q27b-lr5e6ep2g8-FP8 already exists in ChaiML
chaiml-mega-v1-sonnetwi-11582-v3-uploader: Model already exists. Downloading to /dev/shm/model_output...
chaiml-mega-v1-sonnetwi-11582-v3-uploader: Downloading snapshot of ChaiML/mega-v1-sonnetwintop2-q27b-lr5e6ep2g8-FP8...
2026-03-28T13:07:43.899189+00:00 monitor updated for chaiml-mega-v1-sonnetwi_11582_v3
chaiml-mega-v1-sonnetwi-11582-v3-uploader: Downloaded in 36.812s
chaiml-mega-v1-sonnetwi-11582-v3-uploader: Processed model ChaiML/mega-v1-sonnetwintop2-q27b-lr5e6ep2g8 in 39.309s
chaiml-mega-v1-sonnetwi-11582-v3-uploader: creating bucket guanaco-vllm-models
chaiml-mega-v1-sonnetwi-11582-v3-uploader: /usr/lib/python3/dist-packages/S3/BaseUtils.py:56: SyntaxWarning: invalid escape sequence '\.'
chaiml-mega-v1-sonnetwi-11582-v3-uploader:   RE_S3_DATESTRING = re.compile('\.[0-9]*(?:[Z\\-\\+]*?)')
chaiml-mega-v1-sonnetwi-11582-v3-uploader: /usr/lib/python3/dist-packages/S3/BaseUtils.py:57: SyntaxWarning: invalid escape sequence '\s'
chaiml-mega-v1-sonnetwi-11582-v3-uploader:   RE_XML_NAMESPACE = re.compile(b'^(<?[^>]+?>\s*|\s*)(<\w+) xmlns=[\'"](https?://[^\'"]+)[\'"]', re.MULTILINE)
chaiml-mega-v1-sonnetwi-11582-v3-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:240: SyntaxWarning: invalid escape sequence '\.'
chaiml-mega-v1-sonnetwi-11582-v3-uploader:   invalid = re.search("([^a-z0-9\.-])", bucket, re.UNICODE)
chaiml-mega-v1-sonnetwi-11582-v3-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:244: SyntaxWarning: invalid escape sequence '\.'
chaiml-mega-v1-sonnetwi-11582-v3-uploader:   invalid = re.search("([^A-Za-z0-9\._-])", bucket, re.UNICODE)
chaiml-mega-v1-sonnetwi-11582-v3-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:255: SyntaxWarning: invalid escape sequence '\.'
chaiml-mega-v1-sonnetwi-11582-v3-uploader:   if re.search("-\.", bucket, re.UNICODE):
chaiml-mega-v1-sonnetwi-11582-v3-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:257: SyntaxWarning: invalid escape sequence '\.'
chaiml-mega-v1-sonnetwi-11582-v3-uploader:   if re.search("\.\.", bucket, re.UNICODE):
chaiml-mega-v1-sonnetwi-11582-v3-uploader: /usr/lib/python3/dist-packages/S3/S3Uri.py:155: SyntaxWarning: invalid escape sequence '\w'
chaiml-mega-v1-sonnetwi-11582-v3-uploader:   _re = re.compile("^(\w+://)?(.*)", re.UNICODE)
chaiml-mega-v1-sonnetwi-11582-v3-uploader: /usr/lib/python3/dist-packages/S3/FileLists.py:480: SyntaxWarning: invalid escape sequence '\*'
chaiml-mega-v1-sonnetwi-11582-v3-uploader:   wildcard_split_result = re.split("\*|\?", uri_str, maxsplit=1)
chaiml-mega-v1-sonnetwi-11582-v3-uploader: Bucket 's3://guanaco-vllm-models/' created
chaiml-mega-v1-sonnetwi-11582-v3-uploader: uploading /dev/shm/model_output to s3://guanaco-vllm-models/chaiml-mega-v1-sonnetwi-11582-v3/default
chaiml-mega-v1-sonnetwi-11582-v3-uploader: cp /dev/shm/model_output/.gitattributes s3://guanaco-vllm-models/chaiml-mega-v1-sonnetwi-11582-v3/default/.gitattributes
chaiml-mega-v1-sonnetwi-11582-v3-uploader: cp /dev/shm/model_output/tokenizer_config.json s3://guanaco-vllm-models/chaiml-mega-v1-sonnetwi-11582-v3/default/tokenizer_config.json
chaiml-mega-v1-sonnetwi-11582-v3-uploader: cp /dev/shm/model_output/chat_template.jinja s3://guanaco-vllm-models/chaiml-mega-v1-sonnetwi-11582-v3/default/chat_template.jinja
chaiml-mega-v1-sonnetwi-11582-v3-uploader: cp /dev/shm/model_output/recipe.yaml s3://guanaco-vllm-models/chaiml-mega-v1-sonnetwi-11582-v3/default/recipe.yaml
chaiml-mega-v1-sonnetwi-11582-v3-uploader: cp /dev/shm/model_output/generation_config.json s3://guanaco-vllm-models/chaiml-mega-v1-sonnetwi-11582-v3/default/generation_config.json
chaiml-mega-v1-sonnetwi-11582-v3-uploader: cp /dev/shm/model_output/config.json s3://guanaco-vllm-models/chaiml-mega-v1-sonnetwi-11582-v3/default/config.json
chaiml-mega-v1-sonnetwi-11582-v3-uploader: cp /dev/shm/model_output/tokenizer.json s3://guanaco-vllm-models/chaiml-mega-v1-sonnetwi-11582-v3/default/tokenizer.json
2026-03-28T13:08:44.142072+00:00 monitor updated for chaiml-mega-v1-sonnetwi_11582_v3
chaiml-mega-v1-sonnetwi-11582-v3-uploader: cp /dev/shm/model_output/model.safetensors s3://guanaco-vllm-models/chaiml-mega-v1-sonnetwi-11582-v3/default/model.safetensors
Job chaiml-mega-v1-sonnetwi-11582-v3-uploader completed after 133.84s with status: succeeded
Stopping job with name chaiml-mega-v1-sonnetwi-11582-v3-uploader
Pipeline stage VLLMUploader completed in 134.47s
run pipeline stage %s
Running pipeline stage VLLMUploaderAMD
Pipeline stage vllm_upload_amd skipped, reason=not amd cluster
Pipeline stage VLLMUploaderAMD completed in 0.10s
run pipeline stage %s
Running pipeline stage VLLMTemplater
Pipeline stage VLLMTemplater completed in 1.81s
run pipeline stage %s
Running pipeline stage VLLMDeployer
Creating inference service chaiml-mega-v1-sonnetwi-11582-v3
Waiting for inference service chaiml-mega-v1-sonnetwi-11582-v3 to be ready
2026-03-28T13:09:44.230310+00:00 monitor updated for chaiml-mega-v1-sonnetwi_11582_v3
Failed to get response for submission chaiml-gspo-glm47-combi_10268_v1: ('http://chaiml-gspo-glm47-combi-10268-v1-predictor.tenant-chaiml-guanaco.k2.chaiverse.com/v1/completions', 'activator request timeout')
2026-03-28T13:10:44.328928+00:00 monitor updated for chaiml-mega-v1-sonnetwi_11582_v3
Failed to get response for submission chaiml-gspo-glm47-cas72_44260_v1: ('http://chaiml-gspo-glm47-cas72-44260-v1-predictor.tenant-chaiml-guanaco.k2.chaiverse.com/v1/completions', 'activator request timeout')
2026-03-28T13:11:44.429216+00:00 monitor updated for chaiml-mega-v1-sonnetwi_11582_v3
Inference service chaiml-mega-v1-sonnetwi-11582-v3 ready after 180.21354603767395s
Pipeline stage VLLMDeployer completed in 180.68s
run pipeline stage %s
Running pipeline stage StressChecker
HTTPConnectionPool(host='guanaco-submitter-v2.guanaco-backend.kchai-google-us-east4.chaiverse.com', port=80): Read timed out. (read timeout=20)
Received unhealthy response to inference request!
HTTPConnectionPool(host='guanaco-submitter-v2.guanaco-backend.kchai-google-us-east4.chaiverse.com', port=80): Read timed out. (read timeout=20)
Received unhealthy response to inference request!
2026-03-28T13:12:44.568616+00:00 monitor updated for chaiml-mega-v1-sonnetwi_11582_v3
HTTPConnectionPool(host='guanaco-submitter-v2.guanaco-backend.kchai-google-us-east4.chaiverse.com', port=80): Read timed out. (read timeout=20)
Received unhealthy response to inference request!
HTTPConnectionPool(host='guanaco-submitter-v2.guanaco-backend.kchai-google-us-east4.chaiverse.com', port=80): Read timed out. (read timeout=20)
Received unhealthy response to inference request!
Received healthy response to inference request in 2.5172557830810547s
Received healthy response to inference request in 9.23505187034607s
2026-03-28T13:13:44.666407+00:00 monitor updated for chaiml-mega-v1-sonnetwi_11582_v3
HTTPConnectionPool(host='guanaco-submitter-v2.guanaco-backend.kchai-google-us-east4.chaiverse.com', port=80): Read timed out. (read timeout=20)
Received unhealthy response to inference request!
HTTPConnectionPool(host='guanaco-submitter-v2.guanaco-backend.kchai-google-us-east4.chaiverse.com', port=80): Read timed out. (read timeout=20)
Received unhealthy response to inference request!
HTTPConnectionPool(host='guanaco-submitter-v2.guanaco-backend.kchai-google-us-east4.chaiverse.com', port=80): Read timed out. (read timeout=20)
Received unhealthy response to inference request!
{"detail":"('http://chaiml-mega-v1-sonnetwi-11582-v3-predictor.tenant-chaiml-guanaco.k2.chaiverse.com/v1/completions', 'upstream connect error or disconnect/reset before headers. reset reason: connection termination')"}
Received unhealthy response to inference request!
2026-03-28T13:14:44.791826+00:00 monitor updated for chaiml-mega-v1-sonnetwi_11582_v3
Received healthy response to inference request in 4.443531513214111s
HTTPConnectionPool(host='guanaco-submitter-v2.guanaco-backend.kchai-google-us-east4.chaiverse.com', port=80): Read timed out. (read timeout=20)
Received unhealthy response to inference request!
Received healthy response to inference request in 2.131904125213623s
Received healthy response to inference request in 1.8496696949005127s
Received healthy response to inference request in 2.143590211868286s
Received healthy response to inference request in 14.103766441345215s
Received healthy response to inference request in 1.9685630798339844s
Received healthy response to inference request in 2.100817918777466s
Received healthy response to inference request in 1.9155569076538086s
Received healthy response to inference request in 1.88680100440979s
Received healthy response to inference request in 1.973780870437622s
Received healthy response to inference request in 1.8336670398712158s
Received healthy response to inference request in 2.0990357398986816s
2026-03-28T13:15:44.892974+00:00 monitor updated for chaiml-mega-v1-sonnetwi_11582_v3
Received healthy response to inference request in 1.943695306777954s
Received healthy response to inference request in 1.9913761615753174s
Received healthy response to inference request in 1.913539171218872s
Received healthy response to inference request in 2.005061149597168s
Received healthy response to inference request in 2.0117695331573486s
Received healthy response to inference request in 1.9579529762268066s
Received healthy response to inference request in 2.077498435974121s
30 requests
9 failed requests
5th percentile: 1.8663787841796875
10th percentile: 1.910865354537964
20th percentile: 1.955101442337036
30th percentile: 1.9860975742340088
40th percentile: 2.0512068748474124
50th percentile: 2.1163610219955444
60th percentile: 3.2877660751342748
70th percentile: 11.042909336090075
80th percentile: 20.131244802474974
90th percentile: 20.161127161979675
95th percentile: 20.172721683979034
99th percentile: 20.40861267566681
mean time: 7.845960179964702
%s, retrying in %s seconds...
Received healthy response to inference request in 1.59611177444458s
Received healthy response to inference request in 1.7523980140686035s
Received healthy response to inference request in 1.8625690937042236s
Received healthy response to inference request in 2.0186679363250732s
Received healthy response to inference request in 1.7880041599273682s
Received healthy response to inference request in 2.0521397590637207s
Received healthy response to inference request in 1.8008396625518799s
Received healthy response to inference request in 2.136003017425537s
Received healthy response to inference request in 1.9024736881256104s
Received healthy response to inference request in 2.063307046890259s
Received healthy response to inference request in 1.983058214187622s
Received healthy response to inference request in 2.016303062438965s
Received healthy response to inference request in 2.056511163711548s
Received healthy response to inference request in 1.7393412590026855s
Received healthy response to inference request in 1.9153001308441162s
Received healthy response to inference request in 2.4910478591918945s
Received healthy response to inference request in 1.9735138416290283s
Received healthy response to inference request in 1.8750801086425781s
Received healthy response to inference request in 2.335402488708496s
Received healthy response to inference request in 2.439774990081787s
Received healthy response to inference request in 1.8632254600524902s
Received healthy response to inference request in 2.426558256149292s
2026-03-28T13:16:45.001684+00:00 monitor updated for chaiml-mega-v1-sonnetwi_11582_v3
Received healthy response to inference request in 2.059913396835327s
Received healthy response to inference request in 1.8356430530548096s
Received healthy response to inference request in 2.7316014766693115s
Received healthy response to inference request in 1.9760961532592773s
Received healthy response to inference request in 1.9626827239990234s
Received healthy response to inference request in 2.0964229106903076s
Received healthy response to inference request in 2.5972728729248047s
Received healthy response to inference request in 2.5907604694366455s
30 requests
0 failed requests
5th percentile: 1.7452167987823486
10th percentile: 1.7844435453414917
20th percentile: 1.8571838855743408
30th percentile: 1.8942556142807008
40th percentile: 1.9691813945770265
50th percentile: 1.9996806383132935
60th percentile: 2.0538883209228516
70th percentile: 2.0732418060302735
80th percentile: 2.3536336421966557
90th percentile: 2.5010191202163696
95th percentile: 2.594342291355133
99th percentile: 2.6926461815834046
mean time: 2.0646008014678956
Pipeline stage StressChecker completed in 302.99s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyTriggerPipeline
run_pipeline:run_in_cloud %s
starting trigger_guanaco_pipeline args=%s
triggered trigger_guanaco_pipeline args=%s
Pipeline stage OfflineFamilyFriendlyTriggerPipeline completed in 0.81s
Shutdown handler de-registered
chaiml-mega-v1-sonnetwi_11582_v3 status is now deployed due to DeploymentManager action
chaiml-mega-v1-sonnetwi_11582_v3 status is now inactive due to admin request
chaiml-mega-v1-sonnetwi_11582_v3 status is now torndown due to DeploymentManager action