developer_uid: zonemercy
submission_id: chaiml-pony-d3a-mv1-son_75599_v6
model_name: chaiml-pony-d3a-mv1-son_75599_v6
model_group: ChaiML/pony-d3a-mv1-sonn
status: deployed
timestamp: 2026-03-28T17:24:46+00:00
num_battles: 5841
num_wins: 3046
celo_rating: 1309.78
family_friendly_score: 0.0
family_friendly_standard_error: 0.0
submission_type: basic
model_repo: ChaiML/pony-d3a-mv1-sonnetwintop2-q35b-lr5e6ep2g8
model_architecture: Qwen3_5MoeForConditionalGeneration
model_num_parameters: 33753909248.0
best_of: 16
max_input_tokens: 2048
max_output_tokens: 80
reward_model: default
display_name: chaiml-pony-d3a-mv1-son_75599_v6
ineligible_reason: max_output_tokens!=64
is_internal_developer: True
language_model: ChaiML/pony-d3a-mv1-sonnetwintop2-q35b-lr5e6ep2g8
model_size: 34B
ranking_group: single
us_pacific_date: 2026-03-28
win_ratio: 0.5214860469097757
generation_params: {'temperature': 1.0, 'top_p': 1.0, 'min_p': 0.0, 'top_k': 40, 'presence_penalty': 0.8, 'frequency_penalty': 0.0, 'stopping_words': ['<|assistant|>', '<|im_end|>', '</s>', '<|user|>', '####'], 'max_input_tokens': 2048, 'best_of': 16, 'max_output_tokens': 80}
formatter: {'memory_template': "<|im_start|>system\n{bot_name}'s persona: {memory}<|im_end|>\n", 'prompt_template': '', 'bot_template': '<|im_start|>assistant\n{bot_name}: {message}<|im_end|>\n', 'user_template': '<|im_start|>user\n{message}<|im_end|>\n', 'response_template': '<|im_start|>assistant\n{bot_name}:', 'truncate_by_message': True}
Resubmit model
Shutdown handler not registered because Python interpreter is not running in the main thread
run pipeline %s
run pipeline stage %s
Running pipeline stage VLLMUploader
Starting job with name chaiml-pony-d3a-mv1-son-75599-v6-uploader
Waiting for job on chaiml-pony-d3a-mv1-son-75599-v6-uploader to finish
chaiml-pony-d3a-mv1-son-75599-v6-uploader: Using quantization_mode: fp8
chaiml-pony-d3a-mv1-son-75599-v6-uploader: Model already exists. Downloading to /dev/shm/model_output...
chaiml-pony-d3a-mv1-son-75599-v6-uploader: Downloading snapshot of ChaiML/pony-d3a-mv1-sonnetwintop2-q35b-lr5e6ep2g8-FP8...
2026-03-28T17:14:54.525848+00:00 monitor updated for chaiml-pony-d3a-mv1-son_75599_v6
chaiml-pony-d3a-mv1-son-75599-v6-uploader: Downloaded in 36.577s
chaiml-pony-d3a-mv1-son-75599-v6-uploader: Processed model ChaiML/pony-d3a-mv1-sonnetwintop2-q35b-lr5e6ep2g8 in 39.733s
chaiml-pony-d3a-mv1-son-75599-v6-uploader: creating bucket guanaco-vllm-models
chaiml-pony-d3a-mv1-son-75599-v6-uploader: /usr/lib/python3/dist-packages/S3/BaseUtils.py:56: SyntaxWarning: invalid escape sequence '\.'
chaiml-pony-d3a-mv1-son-75599-v6-uploader:   RE_S3_DATESTRING = re.compile('\.[0-9]*(?:[Z\\-\\+]*?)')
chaiml-pony-d3a-mv1-son-75599-v6-uploader: /usr/lib/python3/dist-packages/S3/BaseUtils.py:57: SyntaxWarning: invalid escape sequence '\s'
chaiml-pony-d3a-mv1-son-75599-v6-uploader:   RE_XML_NAMESPACE = re.compile(b'^(<?[^>]+?>\s*|\s*)(<\w+) xmlns=[\'"](https?://[^\'"]+)[\'"]', re.MULTILINE)
chaiml-pony-d3a-mv1-son-75599-v6-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:240: SyntaxWarning: invalid escape sequence '\.'
chaiml-pony-d3a-mv1-son-75599-v6-uploader:   invalid = re.search("([^a-z0-9\.-])", bucket, re.UNICODE)
chaiml-pony-d3a-mv1-son-75599-v6-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:244: SyntaxWarning: invalid escape sequence '\.'
chaiml-pony-d3a-mv1-son-75599-v6-uploader:   invalid = re.search("([^A-Za-z0-9\._-])", bucket, re.UNICODE)
chaiml-pony-d3a-mv1-son-75599-v6-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:255: SyntaxWarning: invalid escape sequence '\.'
chaiml-pony-d3a-mv1-son-75599-v6-uploader:   if re.search("-\.", bucket, re.UNICODE):
chaiml-pony-d3a-mv1-son-75599-v6-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:257: SyntaxWarning: invalid escape sequence '\.'
chaiml-pony-d3a-mv1-son-75599-v6-uploader:   if re.search("\.\.", bucket, re.UNICODE):
chaiml-pony-d3a-mv1-son-75599-v6-uploader: /usr/lib/python3/dist-packages/S3/S3Uri.py:155: SyntaxWarning: invalid escape sequence '\w'
chaiml-pony-d3a-mv1-son-75599-v6-uploader:   _re = re.compile("^(\w+://)?(.*)", re.UNICODE)
chaiml-pony-d3a-mv1-son-75599-v6-uploader: /usr/lib/python3/dist-packages/S3/FileLists.py:480: SyntaxWarning: invalid escape sequence '\*'
chaiml-pony-d3a-mv1-son-75599-v6-uploader:   wildcard_split_result = re.split("\*|\?", uri_str, maxsplit=1)
chaiml-pony-d3a-mv1-son-75599-v6-uploader: Bucket 's3://guanaco-vllm-models/' created
chaiml-pony-d3a-mv1-son-75599-v6-uploader: uploading /dev/shm/model_output to s3://guanaco-vllm-models/chaiml-pony-d3a-mv1-son-75599-v6/default
chaiml-pony-d3a-mv1-son-75599-v6-uploader: cp /dev/shm/model_output/.gitattributes s3://guanaco-vllm-models/chaiml-pony-d3a-mv1-son-75599-v6/default/.gitattributes
chaiml-pony-d3a-mv1-son-75599-v6-uploader: cp /dev/shm/model_output/config.json s3://guanaco-vllm-models/chaiml-pony-d3a-mv1-son-75599-v6/default/config.json
chaiml-pony-d3a-mv1-son-75599-v6-uploader: cp /dev/shm/model_output/recipe.yaml s3://guanaco-vllm-models/chaiml-pony-d3a-mv1-son-75599-v6/default/recipe.yaml
chaiml-pony-d3a-mv1-son-75599-v6-uploader: cp /dev/shm/model_output/tokenizer_config.json s3://guanaco-vllm-models/chaiml-pony-d3a-mv1-son-75599-v6/default/tokenizer_config.json
chaiml-pony-d3a-mv1-son-75599-v6-uploader: cp /dev/shm/model_output/generation_config.json s3://guanaco-vllm-models/chaiml-pony-d3a-mv1-son-75599-v6/default/generation_config.json
chaiml-pony-d3a-mv1-son-75599-v6-uploader: cp /dev/shm/model_output/chat_template.jinja s3://guanaco-vllm-models/chaiml-pony-d3a-mv1-son-75599-v6/default/chat_template.jinja
chaiml-pony-d3a-mv1-son-75599-v6-uploader: cp /dev/shm/model_output/tokenizer.json s3://guanaco-vllm-models/chaiml-pony-d3a-mv1-son-75599-v6/default/tokenizer.json
2026-03-28T17:15:54.625201+00:00 monitor updated for chaiml-pony-d3a-mv1-son_75599_v6
chaiml-pony-d3a-mv1-son-75599-v6-uploader: cp /dev/shm/model_output/model.safetensors s3://guanaco-vllm-models/chaiml-pony-d3a-mv1-son-75599-v6/default/model.safetensors
Job chaiml-pony-d3a-mv1-son-75599-v6-uploader completed after 153.24s with status: succeeded
Stopping job with name chaiml-pony-d3a-mv1-son-75599-v6-uploader
Pipeline stage VLLMUploader completed in 154.02s
run pipeline stage %s
Running pipeline stage VLLMUploaderAMD
Pipeline stage vllm_upload_amd skipped, reason=not amd cluster
Pipeline stage VLLMUploaderAMD completed in 0.09s
run pipeline stage %s
Running pipeline stage VLLMTemplater
Pipeline stage VLLMTemplater completed in 3.00s
run pipeline stage %s
Running pipeline stage VLLMDeployer
Creating inference service chaiml-pony-d3a-mv1-son-75599-v6
Waiting for inference service chaiml-pony-d3a-mv1-son-75599-v6 to be ready
2026-03-28T17:16:54.711658+00:00 monitor updated for chaiml-pony-d3a-mv1-son_75599_v6
2026-03-28T17:17:54.809809+00:00 monitor updated for chaiml-pony-d3a-mv1-son_75599_v6
Unable to record family friendly update due to error: Invalid JSON input: JSON must contain 'User Safety' and 'Response Safety' fields
2026-03-28T17:18:54.904412+00:00 monitor updated for chaiml-pony-d3a-mv1-son_75599_v6
2026-03-28T17:19:55.012229+00:00 monitor updated for chaiml-pony-d3a-mv1-son_75599_v6
Inference service chaiml-pony-d3a-mv1-son-75599-v6 ready after 241.3057301044464s
Pipeline stage VLLMDeployer completed in 241.98s
run pipeline stage %s
Running pipeline stage StressChecker
HTTPConnectionPool(host='guanaco-submitter-v2.guanaco-backend.kchai-google-us-east4.chaiverse.com', port=80): Read timed out. (read timeout=20)
Received unhealthy response to inference request!
2026-03-28T17:20:55.111864+00:00 monitor updated for chaiml-pony-d3a-mv1-son_75599_v6
HTTPConnectionPool(host='guanaco-submitter-v2.guanaco-backend.kchai-google-us-east4.chaiverse.com', port=80): Read timed out. (read timeout=20)
Received unhealthy response to inference request!
HTTPConnectionPool(host='guanaco-submitter-v2.guanaco-backend.kchai-google-us-east4.chaiverse.com', port=80): Read timed out. (read timeout=20)
Received unhealthy response to inference request!
HTTPConnectionPool(host='guanaco-submitter-v2.guanaco-backend.kchai-google-us-east4.chaiverse.com', port=80): Read timed out. (read timeout=20)
Received unhealthy response to inference request!
2026-03-28T17:21:55.209830+00:00 monitor updated for chaiml-pony-d3a-mv1-son_75599_v6
HTTPConnectionPool(host='guanaco-submitter-v2.guanaco-backend.kchai-google-us-east4.chaiverse.com', port=80): Read timed out. (read timeout=20)
Received unhealthy response to inference request!
Received healthy response to inference request in 5.7422003746032715s
Received healthy response to inference request in 2.826416254043579s
Received healthy response to inference request in 5.900695085525513s
Received healthy response to inference request in 5.812366485595703s
Received healthy response to inference request in 12.966366291046143s
Received healthy response to inference request in 2.0735671520233154s
Received healthy response to inference request in 2.369680166244507s
Received healthy response to inference request in 2.105886936187744s
2026-03-28T17:22:55.309926+00:00 monitor updated for chaiml-pony-d3a-mv1-son_75599_v6
Received healthy response to inference request in 2.1633524894714355s
Received healthy response to inference request in 2.108327865600586s
Received healthy response to inference request in 2.2699854373931885s
Received healthy response to inference request in 2.6310229301452637s
Received healthy response to inference request in 2.317448616027832s
Received healthy response to inference request in 2.1344659328460693s
Received healthy response to inference request in 2.107024669647217s
Received healthy response to inference request in 2.2007009983062744s
Received healthy response to inference request in 2.1653335094451904s
Received healthy response to inference request in 2.106013774871826s
Received healthy response to inference request in 2.34771990776062s
Received healthy response to inference request in 2.166152000427246s
Received healthy response to inference request in 2.1302425861358643s
Received healthy response to inference request in 2.591196298599243s
Received healthy response to inference request in 2.842437744140625s
Received healthy response to inference request in 2.246098518371582s
Received healthy response to inference request in 2.1069436073303223s
30 requests
5 failed requests
5th percentile: 2.105944013595581
10th percentile: 2.1068506240844727
20th percentile: 2.1258596420288085
30th percentile: 2.164739203453064
40th percentile: 2.227939510345459
50th percentile: 2.332584261894226
60th percentile: 2.6071269512176514
70th percentile: 3.7123665332794107
80th percentile: 7.313829326629659
90th percentile: 20.13575212955475
95th percentile: 20.13882977962494
99th percentile: 20.154736580848695
mean time: 5.971169312795003
%s, retrying in %s seconds...
Received healthy response to inference request in 2.156869411468506s
Received healthy response to inference request in 2.11749005317688s
Received healthy response to inference request in 2.0718116760253906s
Received healthy response to inference request in 2.0345818996429443s
Received healthy response to inference request in 2.206124782562256s
Received healthy response to inference request in 2.1781535148620605s
Received healthy response to inference request in 2.2026638984680176s
Received healthy response to inference request in 2.3673040866851807s
2026-03-28T17:23:55.499517+00:00 monitor updated for chaiml-pony-d3a-mv1-son_75599_v6
Received healthy response to inference request in 2.146496057510376s
Received healthy response to inference request in 2.0793120861053467s
Received healthy response to inference request in 2.0698797702789307s
Received healthy response to inference request in 2.177983522415161s
Received healthy response to inference request in 2.117332696914673s
Received healthy response to inference request in 2.0692591667175293s
Received healthy response to inference request in 2.064624786376953s
Received healthy response to inference request in 2.1406373977661133s
Received healthy response to inference request in 2.285778760910034s
Received healthy response to inference request in 2.1552655696868896s
Received healthy response to inference request in 2.234119415283203s
Received healthy response to inference request in 2.1702287197113037s
Received healthy response to inference request in 2.124424457550049s
Received healthy response to inference request in 2.1949310302734375s
Received healthy response to inference request in 2.1868081092834473s
Received healthy response to inference request in 2.151888847351074s
Received healthy response to inference request in 2.229933261871338s
Received healthy response to inference request in 2.2018353939056396s
Received healthy response to inference request in 2.166472911834717s
Received healthy response to inference request in 2.4603748321533203s
Received healthy response to inference request in 2.141930341720581s
Received healthy response to inference request in 2.0901620388031006s
30 requests
0 failed requests
5th percentile: 2.0667102575302123
10th percentile: 2.0698177099227903
20th percentile: 2.08799204826355
30th percentile: 2.122344136238098
40th percentile: 2.144669771194458
50th percentile: 2.1560674905776978
60th percentile: 2.1733306407928468
70th percentile: 2.1892449855804443
80th percentile: 2.203356075286865
90th percentile: 2.2392853498458862
95th percentile: 2.3306176900863647
99th percentile: 2.43338431596756
mean time: 2.166489283243815
Pipeline stage StressChecker completed in 250.61s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyTriggerPipeline
run_pipeline:run_in_cloud %s
starting trigger_guanaco_pipeline args=%s
triggered trigger_guanaco_pipeline args=%s
Pipeline stage OfflineFamilyFriendlyTriggerPipeline completed in 1.12s
Shutdown handler de-registered
chaiml-pony-d3a-mv1-son_75599_v6 status is now deployed due to DeploymentManager action