developer_uid: zonemercy
submission_id: chaiml-pony-d3a-mv1-son_96936_v2
model_name: chaiml-pony-d3a-mv1-son_96936_v2
model_group: ChaiML/pony-d3a-mv1-sonn
status: torndown
timestamp: 2026-03-28T17:01:00+00:00
num_battles: 10929
num_wins: 5709
celo_rating: 8468.98
family_friendly_score: 0.0
family_friendly_standard_error: 0.0
submission_type: basic
model_repo: ChaiML/pony-d3a-mv1-sonnetwintop2-q35b-lr5e6ep1g8
model_architecture: Qwen3_5MoeForConditionalGeneration
model_num_parameters: 33753909248.0
best_of: 16
max_input_tokens: 2048
max_output_tokens: 80
reward_model: default
display_name: chaiml-pony-d3a-mv1-son_96936_v2
ineligible_reason: max_output_tokens!=64
is_internal_developer: True
language_model: ChaiML/pony-d3a-mv1-sonnetwintop2-q35b-lr5e6ep1g8
model_size: 34B
ranking_group: single
us_pacific_date: 2026-03-25
win_ratio: 0.522371671699149
generation_params: {'temperature': 1.0, 'top_p': 1.0, 'min_p': 0.0, 'top_k': 40, 'presence_penalty': 0.8, 'frequency_penalty': 0.0, 'stopping_words': ['<|user|>', '<|im_end|>', '<|assistant|>', '</s>', '####'], 'max_input_tokens': 2048, 'best_of': 16, 'max_output_tokens': 80}
formatter: {'memory_template': "<|im_start|>system\n{bot_name}'s persona: {memory}<|im_end|>\n", 'prompt_template': '', 'bot_template': '<|im_start|>assistant\n{bot_name}: {message}<|im_end|>\n', 'user_template': '<|im_start|>user\n{message}<|im_end|>\n', 'response_template': '<|im_start|>assistant\n{bot_name}:', 'truncate_by_message': True}
Resubmit model
Shutdown handler not registered because Python interpreter is not running in the main thread
run pipeline %s
run pipeline stage %s
Running pipeline stage VLLMUploader
Starting job with name chaiml-pony-d3a-mv1-son-96936-v2-uploader
Waiting for job on chaiml-pony-d3a-mv1-son-96936-v2-uploader to finish
chaiml-pony-d3a-mv1-son-96936-v2-uploader: Using quantization_mode: none
chaiml-pony-d3a-mv1-son-96936-v2-uploader: Downloading snapshot of ChaiML/pony-d3a-mv1-sonnetwintop2-q35b-lr5e6ep1g8...
chaiml-pony-d3a-mv1-son-96936-v2-uploader: Downloaded in 24.572s
2026-03-25T14:42:21.853541+00:00 monitor updated for chaiml-pony-d3a-mv1-son_96936_v2
chaiml-pony-d3a-mv1-son-96936-v2-uploader: Processed model ChaiML/pony-d3a-mv1-sonnetwintop2-q35b-lr5e6ep1g8 in 51.100s
chaiml-pony-d3a-mv1-son-96936-v2-uploader: creating bucket guanaco-vllm-models
chaiml-pony-d3a-mv1-son-96936-v2-uploader: /usr/lib/python3/dist-packages/S3/BaseUtils.py:56: SyntaxWarning: invalid escape sequence '\.'
chaiml-pony-d3a-mv1-son-96936-v2-uploader:   RE_S3_DATESTRING = re.compile('\.[0-9]*(?:[Z\\-\\+]*?)')
chaiml-pony-d3a-mv1-son-96936-v2-uploader: /usr/lib/python3/dist-packages/S3/BaseUtils.py:57: SyntaxWarning: invalid escape sequence '\s'
chaiml-pony-d3a-mv1-son-96936-v2-uploader:   RE_XML_NAMESPACE = re.compile(b'^(<?[^>]+?>\s*|\s*)(<\w+) xmlns=[\'"](https?://[^\'"]+)[\'"]', re.MULTILINE)
chaiml-pony-d3a-mv1-son-96936-v2-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:240: SyntaxWarning: invalid escape sequence '\.'
chaiml-pony-d3a-mv1-son-96936-v2-uploader:   invalid = re.search("([^a-z0-9\.-])", bucket, re.UNICODE)
chaiml-pony-d3a-mv1-son-96936-v2-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:244: SyntaxWarning: invalid escape sequence '\.'
chaiml-pony-d3a-mv1-son-96936-v2-uploader:   invalid = re.search("([^A-Za-z0-9\._-])", bucket, re.UNICODE)
chaiml-pony-d3a-mv1-son-96936-v2-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:255: SyntaxWarning: invalid escape sequence '\.'
chaiml-pony-d3a-mv1-son-96936-v2-uploader:   if re.search("-\.", bucket, re.UNICODE):
chaiml-pony-d3a-mv1-son-96936-v2-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:257: SyntaxWarning: invalid escape sequence '\.'
chaiml-pony-d3a-mv1-son-96936-v2-uploader:   if re.search("\.\.", bucket, re.UNICODE):
chaiml-pony-d3a-mv1-son-96936-v2-uploader: /usr/lib/python3/dist-packages/S3/S3Uri.py:155: SyntaxWarning: invalid escape sequence '\w'
chaiml-pony-d3a-mv1-son-96936-v2-uploader:   _re = re.compile("^(\w+://)?(.*)", re.UNICODE)
chaiml-pony-d3a-mv1-son-96936-v2-uploader: /usr/lib/python3/dist-packages/S3/FileLists.py:480: SyntaxWarning: invalid escape sequence '\*'
chaiml-pony-d3a-mv1-son-96936-v2-uploader:   wildcard_split_result = re.split("\*|\?", uri_str, maxsplit=1)
chaiml-pony-d3a-mv1-son-96936-v2-uploader: Bucket 's3://guanaco-vllm-models/' created
chaiml-pony-d3a-mv1-son-96936-v2-uploader: uploading /dev/shm/model_output to s3://guanaco-vllm-models/chaiml-pony-d3a-mv1-son-96936-v2/default
chaiml-pony-d3a-mv1-son-96936-v2-uploader: cp /dev/shm/model_output/.gitattributes s3://guanaco-vllm-models/chaiml-pony-d3a-mv1-son-96936-v2/default/.gitattributes
chaiml-pony-d3a-mv1-son-96936-v2-uploader: cp /dev/shm/model_output/tokenizer_config.json s3://guanaco-vllm-models/chaiml-pony-d3a-mv1-son-96936-v2/default/tokenizer_config.json
chaiml-pony-d3a-mv1-son-96936-v2-uploader: cp /dev/shm/model_output/generation_config.json s3://guanaco-vllm-models/chaiml-pony-d3a-mv1-son-96936-v2/default/generation_config.json
chaiml-pony-d3a-mv1-son-96936-v2-uploader: cp /dev/shm/model_output/preprocessor_config.json s3://guanaco-vllm-models/chaiml-pony-d3a-mv1-son-96936-v2/default/preprocessor_config.json
chaiml-pony-d3a-mv1-son-96936-v2-uploader: cp /dev/shm/model_output/README.md s3://guanaco-vllm-models/chaiml-pony-d3a-mv1-son-96936-v2/default/README.md
chaiml-pony-d3a-mv1-son-96936-v2-uploader: cp /dev/shm/model_output/processor_config.json s3://guanaco-vllm-models/chaiml-pony-d3a-mv1-son-96936-v2/default/processor_config.json
chaiml-pony-d3a-mv1-son-96936-v2-uploader: cp /dev/shm/model_output/config.json s3://guanaco-vllm-models/chaiml-pony-d3a-mv1-son-96936-v2/default/config.json
chaiml-pony-d3a-mv1-son-96936-v2-uploader: cp /dev/shm/model_output/added_tokens.json s3://guanaco-vllm-models/chaiml-pony-d3a-mv1-son-96936-v2/default/added_tokens.json
chaiml-pony-d3a-mv1-son-96936-v2-uploader: cp /dev/shm/model_output/chat_template.jinja s3://guanaco-vllm-models/chaiml-pony-d3a-mv1-son-96936-v2/default/chat_template.jinja
chaiml-pony-d3a-mv1-son-96936-v2-uploader: cp /dev/shm/model_output/args.json s3://guanaco-vllm-models/chaiml-pony-d3a-mv1-son-96936-v2/default/args.json
chaiml-pony-d3a-mv1-son-96936-v2-uploader: cp /dev/shm/model_output/special_tokens_map.json s3://guanaco-vllm-models/chaiml-pony-d3a-mv1-son-96936-v2/default/special_tokens_map.json
chaiml-pony-d3a-mv1-son-96936-v2-uploader: cp /dev/shm/model_output/model.safetensors.index.json s3://guanaco-vllm-models/chaiml-pony-d3a-mv1-son-96936-v2/default/model.safetensors.index.json
chaiml-pony-d3a-mv1-son-96936-v2-uploader: cp /dev/shm/model_output/merges.txt s3://guanaco-vllm-models/chaiml-pony-d3a-mv1-son-96936-v2/default/merges.txt
chaiml-pony-d3a-mv1-son-96936-v2-uploader: cp /dev/shm/model_output/vocab.json s3://guanaco-vllm-models/chaiml-pony-d3a-mv1-son-96936-v2/default/vocab.json
chaiml-pony-d3a-mv1-son-96936-v2-uploader: cp /dev/shm/model_output/tokenizer.json s3://guanaco-vllm-models/chaiml-pony-d3a-mv1-son-96936-v2/default/tokenizer.json
chaiml-pony-d3a-mv1-son-96936-v2-uploader: cp /dev/shm/model_output/model-00016-of-00016.safetensors s3://guanaco-vllm-models/chaiml-pony-d3a-mv1-son-96936-v2/default/model-00016-of-00016.safetensors
chaiml-pony-d3a-mv1-son-96936-v2-uploader: cp /dev/shm/model_output/model-00010-of-00016.safetensors s3://guanaco-vllm-models/chaiml-pony-d3a-mv1-son-96936-v2/default/model-00010-of-00016.safetensors
chaiml-pony-d3a-mv1-son-96936-v2-uploader: cp /dev/shm/model_output/model-00007-of-00016.safetensors s3://guanaco-vllm-models/chaiml-pony-d3a-mv1-son-96936-v2/default/model-00007-of-00016.safetensors
chaiml-pony-d3a-mv1-son-96936-v2-uploader: cp /dev/shm/model_output/model-00013-of-00016.safetensors s3://guanaco-vllm-models/chaiml-pony-d3a-mv1-son-96936-v2/default/model-00013-of-00016.safetensors
chaiml-pony-d3a-mv1-son-96936-v2-uploader: cp /dev/shm/model_output/model-00004-of-00016.safetensors s3://guanaco-vllm-models/chaiml-pony-d3a-mv1-son-96936-v2/default/model-00004-of-00016.safetensors
chaiml-pony-d3a-mv1-son-96936-v2-uploader: cp /dev/shm/model_output/model-00001-of-00016.safetensors s3://guanaco-vllm-models/chaiml-pony-d3a-mv1-son-96936-v2/default/model-00001-of-00016.safetensors
chaiml-pony-d3a-mv1-son-96936-v2-uploader: cp /dev/shm/model_output/model-00011-of-00016.safetensors s3://guanaco-vllm-models/chaiml-pony-d3a-mv1-son-96936-v2/default/model-00011-of-00016.safetensors
chaiml-pony-d3a-mv1-son-96936-v2-uploader: cp /dev/shm/model_output/model-00005-of-00016.safetensors s3://guanaco-vllm-models/chaiml-pony-d3a-mv1-son-96936-v2/default/model-00005-of-00016.safetensors
chaiml-pony-d3a-mv1-son-96936-v2-uploader: cp /dev/shm/model_output/model-00009-of-00016.safetensors s3://guanaco-vllm-models/chaiml-pony-d3a-mv1-son-96936-v2/default/model-00009-of-00016.safetensors
chaiml-pony-d3a-mv1-son-96936-v2-uploader: cp /dev/shm/model_output/model-00006-of-00016.safetensors s3://guanaco-vllm-models/chaiml-pony-d3a-mv1-son-96936-v2/default/model-00006-of-00016.safetensors
chaiml-pony-d3a-mv1-son-96936-v2-uploader: cp /dev/shm/model_output/model-00012-of-00016.safetensors s3://guanaco-vllm-models/chaiml-pony-d3a-mv1-son-96936-v2/default/model-00012-of-00016.safetensors
chaiml-pony-d3a-mv1-son-96936-v2-uploader: cp /dev/shm/model_output/model-00003-of-00016.safetensors s3://guanaco-vllm-models/chaiml-pony-d3a-mv1-son-96936-v2/default/model-00003-of-00016.safetensors
chaiml-pony-d3a-mv1-son-96936-v2-uploader: cp /dev/shm/model_output/model-00015-of-00016.safetensors s3://guanaco-vllm-models/chaiml-pony-d3a-mv1-son-96936-v2/default/model-00015-of-00016.safetensors
Job chaiml-pony-d3a-mv1-son-96936-v2-uploader completed after 83.47s with status: succeeded
Stopping job with name chaiml-pony-d3a-mv1-son-96936-v2-uploader
Pipeline stage VLLMUploader completed in 84.18s
run pipeline stage %s
Running pipeline stage VLLMTemplater
Pipeline stage VLLMTemplater completed in 2.91s
run pipeline stage %s
Running pipeline stage VLLMDeployer
Creating inference service chaiml-pony-d3a-mv1-son-96936-v2
Waiting for inference service chaiml-pony-d3a-mv1-son-96936-v2 to be ready
2026-03-25T14:43:21.935634+00:00 monitor updated for chaiml-pony-d3a-mv1-son_96936_v2
2026-03-25T14:44:22.025349+00:00 monitor updated for chaiml-pony-d3a-mv1-son_96936_v2
2026-03-25T14:45:22.123458+00:00 monitor updated for chaiml-pony-d3a-mv1-son_96936_v2
Inference service chaiml-pony-d3a-mv1-son-96936-v2 ready after 191.25865244865417s
Pipeline stage VLLMDeployer completed in 192.39s
run pipeline stage %s
Running pipeline stage StressChecker
HTTPConnectionPool(host='guanaco-submitter-v2.guanaco-backend.kchai-google-us-east4.chaiverse.com', port=80): Read timed out. (read timeout=20)
Received unhealthy response to inference request!
2026-03-25T14:46:22.238829+00:00 monitor updated for chaiml-pony-d3a-mv1-son_96936_v2
HTTPConnectionPool(host='guanaco-submitter-v2.guanaco-backend.kchai-google-us-east4.chaiverse.com', port=80): Read timed out. (read timeout=20)
Received unhealthy response to inference request!
HTTPConnectionPool(host='guanaco-submitter-v2.guanaco-backend.kchai-google-us-east4.chaiverse.com', port=80): Read timed out. (read timeout=20)
Received unhealthy response to inference request!
2026-03-25T14:47:22.407150+00:00 monitor updated for chaiml-pony-d3a-mv1-son_96936_v2
HTTPConnectionPool(host='guanaco-submitter-v2.guanaco-backend.kchai-google-us-east4.chaiverse.com', port=80): Read timed out. (read timeout=20)
Received unhealthy response to inference request!
HTTPConnectionPool(host='guanaco-submitter-v2.guanaco-backend.kchai-google-us-east4.chaiverse.com', port=80): Read timed out. (read timeout=20)
Received unhealthy response to inference request!
Received healthy response to inference request in 14.28112006187439s
Received healthy response to inference request in 6.407735824584961s
Received healthy response to inference request in 7.388978481292725s
Received healthy response to inference request in 2.0119850635528564s
Received healthy response to inference request in 1.4359242916107178s
Received healthy response to inference request in 1.627561092376709s
2026-03-25T14:48:22.501525+00:00 monitor updated for chaiml-pony-d3a-mv1-son_96936_v2
Received healthy response to inference request in 7.04900598526001s
Received healthy response to inference request in 2.0169193744659424s
Received healthy response to inference request in 2.857800245285034s
Received healthy response to inference request in 1.8436956405639648s
Received healthy response to inference request in 1.4391698837280273s
Received healthy response to inference request in 1.498490571975708s
Received healthy response to inference request in 1.4795987606048584s
Received healthy response to inference request in 1.449857234954834s
Received healthy response to inference request in 2.0946667194366455s
Received healthy response to inference request in 2.161119222640991s
Received healthy response to inference request in 1.7317776679992676s
Received healthy response to inference request in 1.7154996395111084s
Received healthy response to inference request in 2.458986520767212s
Received healthy response to inference request in 1.6584131717681885s
Received healthy response to inference request in 1.4930219650268555s
Received healthy response to inference request in 1.5505890846252441s
Received healthy response to inference request in 2.2297515869140625s
Received healthy response to inference request in 1.502413272857666s
Received healthy response to inference request in 1.4333457946777344s
30 requests
5 failed requests
5th percentile: 1.437384808063507
10th percentile: 1.4487884998321534
20th percentile: 1.4973968505859374
30th percentile: 1.6044694900512695
40th percentile: 1.725266456604004
50th percentile: 2.0144522190093994
60th percentile: 2.1885721683502197
70th percentile: 3.9227809190750023
80th percentile: 8.767406797409077
90th percentile: 20.11837296485901
95th percentile: 20.1275661110878
99th percentile: 20.249725887775423
mean time: 5.786335897445679
%s, retrying in %s seconds...
Received healthy response to inference request in 1.368778944015503s
Received healthy response to inference request in 1.3117680549621582s
Received healthy response to inference request in 2.0351130962371826s
Received healthy response to inference request in 1.4925479888916016s
Received healthy response to inference request in 1.335150957107544s
Received healthy response to inference request in 1.8455092906951904s
Received healthy response to inference request in 1.7321436405181885s
Received healthy response to inference request in 1.8882989883422852s
Received healthy response to inference request in 1.510739803314209s
Received healthy response to inference request in 2.159548044204712s
Received healthy response to inference request in 1.522322654724121s
Received healthy response to inference request in 1.500312089920044s
Received healthy response to inference request in 1.6069271564483643s
Received healthy response to inference request in 1.622978687286377s
2026-03-25T14:49:22.592878+00:00 monitor updated for chaiml-pony-d3a-mv1-son_96936_v2
Received healthy response to inference request in 1.6619784832000732s
Received healthy response to inference request in 1.3916773796081543s
Received healthy response to inference request in 1.4486727714538574s
Received healthy response to inference request in 1.7303729057312012s
Received healthy response to inference request in 1.4219472408294678s
Received healthy response to inference request in 1.4858546257019043s
Received healthy response to inference request in 1.456843614578247s
Received healthy response to inference request in 1.4300026893615723s
Received healthy response to inference request in 1.9493210315704346s
Received healthy response to inference request in 1.4468908309936523s
Received healthy response to inference request in 1.4178738594055176s
Received healthy response to inference request in 1.529299020767212s
Received healthy response to inference request in 1.501824140548706s
Received healthy response to inference request in 1.7544002532958984s
Received healthy response to inference request in 1.4446651935577393s
Received healthy response to inference request in 1.6917362213134766s
30 requests
0 failed requests
5th percentile: 1.3502835512161255
10th percentile: 1.3893875360488892
20th percentile: 1.4283915996551513
30th percentile: 1.4481381893157959
40th percentile: 1.4898706436157227
50th percentile: 1.5062819719314575
60th percentile: 1.5603502750396727
70th percentile: 1.670905804634094
80th percentile: 1.7365949630737305
90th percentile: 1.8944011926651
95th percentile: 1.9965066671371456
99th percentile: 2.1234619092941287
mean time: 1.5898499886194866
Pipeline stage StressChecker completed in 226.72s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyTriggerPipeline
run_pipeline:run_in_cloud %s
starting trigger_guanaco_pipeline args=%s
triggered trigger_guanaco_pipeline args=%s
Pipeline stage OfflineFamilyFriendlyTriggerPipeline completed in 2.60s
Shutdown handler de-registered
chaiml-pony-d3a-mv1-son_96936_v2 status is now deployed due to DeploymentManager action
chaiml-pony-d3a-mv1-son_96936_v2 status is now inactive due to auto deactivation removed underperforming models
chaiml-pony-d3a-mv1-son_96936_v2 status is now torndown due to DeploymentManager action