developer_uid: zonemercy
submission_id: chaiml-pony-d1-q235b-pv_97292_v4
model_name: chaiml-pony-d1-q235b-pv_97292_v4
model_group: ChaiML/pony-d1-q235b-pv1
status: inactive
timestamp: 2026-03-04T18:17:20+00:00
num_battles: 12556
num_wins: 7069
celo_rating: 1343.38
family_friendly_score: 0.0
family_friendly_standard_error: 0.0
submission_type: basic
model_repo: ChaiML/pony-d1-q235b-pv1-lr5e6ep2r64g4
model_architecture: Qwen3MoeForCausalLM
model_num_parameters: 18790207488.0
best_of: 8
max_input_tokens: 2048
max_output_tokens: 80
reward_model: default
display_name: chaiml-pony-d1-q235b-pv_97292_v4
ineligible_reason: max_output_tokens!=64
is_internal_developer: True
language_model: ChaiML/pony-d1-q235b-pv1-lr5e6ep2r64g4
model_size: 19B
ranking_group: single
us_pacific_date: 2026-03-04
win_ratio: 0.5629977699904428
generation_params: {'temperature': 1.0, 'top_p': 1.0, 'min_p': 0.0, 'top_k': 40, 'presence_penalty': 0.0, 'frequency_penalty': 0.0, 'stopping_words': ['<|assistant|>', '</s>', '<|user|>', '####', '<|im_end|>'], 'max_input_tokens': 2048, 'best_of': 8, 'max_output_tokens': 80}
formatter: {'memory_template': "<|im_start|>system\n{bot_name}'s persona: {memory}<|im_end|>\n", 'prompt_template': '', 'bot_template': '<|im_start|>assistant\n{bot_name}: {message}<|im_end|>\n', 'user_template': '<|im_start|>user\n{message}<|im_end|>\n', 'response_template': '<|im_start|>assistant\n{bot_name}:', 'truncate_by_message': True}
Resubmit model
Shutdown handler not registered because Python interpreter is not running in the main thread
run pipeline %s
run pipeline stage %s
Running pipeline stage VLLMUploader
Starting job with name chaiml-pony-d1-q235b-pv-97292-v4-uploader
Waiting for job on chaiml-pony-d1-q235b-pv-97292-v4-uploader to finish
chaiml-pony-d1-q235b-pv-97292-v4-uploader: Using quantization_mode: w4a16
chaiml-pony-d1-q235b-pv-97292-v4-uploader: Checking if ChaiML/pony-d1-q235b-pv1-lr5e6ep2r64g4-W4A16 already exists in ChaiML
chaiml-pony-d1-q235b-pv-97292-v4-uploader: Model already exists. Downloading to /dev/shm/model_output...
chaiml-pony-d1-q235b-pv-97292-v4-uploader: Downloading snapshot of ChaiML/pony-d1-q235b-pv1-lr5e6ep2r64g4-W4A16...
chaiml-pony-d1-q235b-pv-97292-v4-uploader: Downloaded in 59.456s
chaiml-pony-d1-q235b-pv-97292-v4-uploader: Processed model ChaiML/pony-d1-q235b-pv1-lr5e6ep2r64g4 in 60.033s
chaiml-pony-d1-q235b-pv-97292-v4-uploader: creating bucket guanaco-vllm-models
chaiml-pony-d1-q235b-pv-97292-v4-uploader: /usr/lib/python3/dist-packages/S3/BaseUtils.py:56: SyntaxWarning: invalid escape sequence '\.'
chaiml-pony-d1-q235b-pv-97292-v4-uploader:   RE_S3_DATESTRING = re.compile('\.[0-9]*(?:[Z\\-\\+]*?)')
chaiml-pony-d1-q235b-pv-97292-v4-uploader: /usr/lib/python3/dist-packages/S3/BaseUtils.py:57: SyntaxWarning: invalid escape sequence '\s'
chaiml-pony-d1-q235b-pv-97292-v4-uploader:   RE_XML_NAMESPACE = re.compile(b'^(<?[^>]+?>\s*|\s*)(<\w+) xmlns=[\'"](https?://[^\'"]+)[\'"]', re.MULTILINE)
chaiml-pony-d1-q235b-pv-97292-v4-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:240: SyntaxWarning: invalid escape sequence '\.'
chaiml-pony-d1-q235b-pv-97292-v4-uploader:   invalid = re.search("([^a-z0-9\.-])", bucket, re.UNICODE)
chaiml-pony-d1-q235b-pv-97292-v4-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:244: SyntaxWarning: invalid escape sequence '\.'
chaiml-pony-d1-q235b-pv-97292-v4-uploader:   invalid = re.search("([^A-Za-z0-9\._-])", bucket, re.UNICODE)
chaiml-pony-d1-q235b-pv-97292-v4-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:255: SyntaxWarning: invalid escape sequence '\.'
chaiml-pony-d1-q235b-pv-97292-v4-uploader:   if re.search("-\.", bucket, re.UNICODE):
chaiml-pony-d1-q235b-pv-97292-v4-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:257: SyntaxWarning: invalid escape sequence '\.'
chaiml-pony-d1-q235b-pv-97292-v4-uploader:   if re.search("\.\.", bucket, re.UNICODE):
chaiml-pony-d1-q235b-pv-97292-v4-uploader: /usr/lib/python3/dist-packages/S3/S3Uri.py:155: SyntaxWarning: invalid escape sequence '\w'
chaiml-pony-d1-q235b-pv-97292-v4-uploader:   _re = re.compile("^(\w+://)?(.*)", re.UNICODE)
chaiml-pony-d1-q235b-pv-97292-v4-uploader: /usr/lib/python3/dist-packages/S3/FileLists.py:480: SyntaxWarning: invalid escape sequence '\*'
chaiml-pony-d1-q235b-pv-97292-v4-uploader:   wildcard_split_result = re.split("\*|\?", uri_str, maxsplit=1)
chaiml-pony-d1-q235b-pv-97292-v4-uploader: Bucket 's3://guanaco-vllm-models/' created
chaiml-pony-d1-q235b-pv-97292-v4-uploader: uploading /dev/shm/model_output to s3://guanaco-vllm-models/chaiml-pony-d1-q235b-pv-97292-v4/default
chaiml-pony-d1-q235b-pv-97292-v4-uploader: cp /dev/shm/model_output/chat_template.jinja s3://guanaco-vllm-models/chaiml-pony-d1-q235b-pv-97292-v4/default/chat_template.jinja
chaiml-pony-d1-q235b-pv-97292-v4-uploader: cp /dev/shm/model_output/quantization_config.json s3://guanaco-vllm-models/chaiml-pony-d1-q235b-pv-97292-v4/default/quantization_config.json
chaiml-pony-d1-q235b-pv-97292-v4-uploader: cp /dev/shm/model_output/.gitattributes s3://guanaco-vllm-models/chaiml-pony-d1-q235b-pv-97292-v4/default/.gitattributes
chaiml-pony-d1-q235b-pv-97292-v4-uploader: cp /dev/shm/model_output/added_tokens.json s3://guanaco-vllm-models/chaiml-pony-d1-q235b-pv-97292-v4/default/added_tokens.json
chaiml-pony-d1-q235b-pv-97292-v4-uploader: cp /dev/shm/model_output/config.json s3://guanaco-vllm-models/chaiml-pony-d1-q235b-pv-97292-v4/default/config.json
chaiml-pony-d1-q235b-pv-97292-v4-uploader: cp /dev/shm/model_output/generation_config.json s3://guanaco-vllm-models/chaiml-pony-d1-q235b-pv-97292-v4/default/generation_config.json
chaiml-pony-d1-q235b-pv-97292-v4-uploader: cp /dev/shm/model_output/tokenizer_config.json s3://guanaco-vllm-models/chaiml-pony-d1-q235b-pv-97292-v4/default/tokenizer_config.json
chaiml-pony-d1-q235b-pv-97292-v4-uploader: cp /dev/shm/model_output/special_tokens_map.json s3://guanaco-vllm-models/chaiml-pony-d1-q235b-pv-97292-v4/default/special_tokens_map.json
chaiml-pony-d1-q235b-pv-97292-v4-uploader: cp /dev/shm/model_output/vocab.json s3://guanaco-vllm-models/chaiml-pony-d1-q235b-pv-97292-v4/default/vocab.json
chaiml-pony-d1-q235b-pv-97292-v4-uploader: cp /dev/shm/model_output/tokenizer.json s3://guanaco-vllm-models/chaiml-pony-d1-q235b-pv-97292-v4/default/tokenizer.json
chaiml-pony-d1-q235b-pv-97292-v4-uploader: cp /dev/shm/model_output/model.safetensors.index.json s3://guanaco-vllm-models/chaiml-pony-d1-q235b-pv-97292-v4/default/model.safetensors.index.json
chaiml-pony-d1-q235b-pv-97292-v4-uploader: cp /dev/shm/model_output/model-00027-of-00027.safetensors s3://guanaco-vllm-models/chaiml-pony-d1-q235b-pv-97292-v4/default/model-00027-of-00027.safetensors
chaiml-pony-d1-q235b-pv-97292-v4-uploader: cp /dev/shm/model_output/model-00023-of-00027.safetensors s3://guanaco-vllm-models/chaiml-pony-d1-q235b-pv-97292-v4/default/model-00023-of-00027.safetensors
chaiml-pony-d1-q235b-pv-97292-v4-uploader: cp /dev/shm/model_output/model-00014-of-00027.safetensors s3://guanaco-vllm-models/chaiml-pony-d1-q235b-pv-97292-v4/default/model-00014-of-00027.safetensors
chaiml-pony-d1-q235b-pv-97292-v4-uploader: cp /dev/shm/model_output/model-00021-of-00027.safetensors s3://guanaco-vllm-models/chaiml-pony-d1-q235b-pv-97292-v4/default/model-00021-of-00027.safetensors
chaiml-pony-d1-q235b-pv-97292-v4-uploader: cp /dev/shm/model_output/model-00007-of-00027.safetensors s3://guanaco-vllm-models/chaiml-pony-d1-q235b-pv-97292-v4/default/model-00007-of-00027.safetensors
chaiml-pony-d1-q235b-pv-97292-v4-uploader: cp /dev/shm/model_output/model-00003-of-00027.safetensors s3://guanaco-vllm-models/chaiml-pony-d1-q235b-pv-97292-v4/default/model-00003-of-00027.safetensors
chaiml-pony-d1-q235b-pv-97292-v4-uploader: cp /dev/shm/model_output/model-00012-of-00027.safetensors s3://guanaco-vllm-models/chaiml-pony-d1-q235b-pv-97292-v4/default/model-00012-of-00027.safetensors
chaiml-pony-d1-q235b-pv-97292-v4-uploader: cp /dev/shm/model_output/model-00002-of-00027.safetensors s3://guanaco-vllm-models/chaiml-pony-d1-q235b-pv-97292-v4/default/model-00002-of-00027.safetensors
chaiml-pony-d1-q235b-pv-97292-v4-uploader: cp /dev/shm/model_output/model-00018-of-00027.safetensors s3://guanaco-vllm-models/chaiml-pony-d1-q235b-pv-97292-v4/default/model-00018-of-00027.safetensors
chaiml-pony-d1-q235b-pv-97292-v4-uploader: cp /dev/shm/model_output/model-00020-of-00027.safetensors s3://guanaco-vllm-models/chaiml-pony-d1-q235b-pv-97292-v4/default/model-00020-of-00027.safetensors
chaiml-pony-d1-q235b-pv-97292-v4-uploader: cp /dev/shm/model_output/model-00022-of-00027.safetensors s3://guanaco-vllm-models/chaiml-pony-d1-q235b-pv-97292-v4/default/model-00022-of-00027.safetensors
chaiml-pony-d1-q235b-pv-97292-v4-uploader: cp /dev/shm/model_output/model-00011-of-00027.safetensors s3://guanaco-vllm-models/chaiml-pony-d1-q235b-pv-97292-v4/default/model-00011-of-00027.safetensors
chaiml-pony-d1-q235b-pv-97292-v4-uploader: cp /dev/shm/model_output/model-00016-of-00027.safetensors s3://guanaco-vllm-models/chaiml-pony-d1-q235b-pv-97292-v4/default/model-00016-of-00027.safetensors
chaiml-pony-d1-q235b-pv-97292-v4-uploader: cp /dev/shm/model_output/model-00008-of-00027.safetensors s3://guanaco-vllm-models/chaiml-pony-d1-q235b-pv-97292-v4/default/model-00008-of-00027.safetensors
chaiml-pony-d1-q235b-pv-97292-v4-uploader: cp /dev/shm/model_output/model-00017-of-00027.safetensors s3://guanaco-vllm-models/chaiml-pony-d1-q235b-pv-97292-v4/default/model-00017-of-00027.safetensors
chaiml-pony-d1-q235b-pv-97292-v4-uploader: cp /dev/shm/model_output/model-00024-of-00027.safetensors s3://guanaco-vllm-models/chaiml-pony-d1-q235b-pv-97292-v4/default/model-00024-of-00027.safetensors
chaiml-pony-d1-q235b-pv-97292-v4-uploader: cp /dev/shm/model_output/model-00001-of-00027.safetensors s3://guanaco-vllm-models/chaiml-pony-d1-q235b-pv-97292-v4/default/model-00001-of-00027.safetensors
chaiml-pony-d1-q235b-pv-97292-v4-uploader: cp /dev/shm/model_output/model-00009-of-00027.safetensors s3://guanaco-vllm-models/chaiml-pony-d1-q235b-pv-97292-v4/default/model-00009-of-00027.safetensors
chaiml-pony-d1-q235b-pv-97292-v4-uploader: cp /dev/shm/model_output/model-00004-of-00027.safetensors s3://guanaco-vllm-models/chaiml-pony-d1-q235b-pv-97292-v4/default/model-00004-of-00027.safetensors
chaiml-pony-d1-q235b-pv-97292-v4-uploader: cp /dev/shm/model_output/model-00025-of-00027.safetensors s3://guanaco-vllm-models/chaiml-pony-d1-q235b-pv-97292-v4/default/model-00025-of-00027.safetensors
chaiml-pony-d1-q235b-pv-97292-v4-uploader: cp /dev/shm/model_output/model-00006-of-00027.safetensors s3://guanaco-vllm-models/chaiml-pony-d1-q235b-pv-97292-v4/default/model-00006-of-00027.safetensors
chaiml-pony-d1-q235b-pv-97292-v4-uploader: cp /dev/shm/model_output/model-00013-of-00027.safetensors s3://guanaco-vllm-models/chaiml-pony-d1-q235b-pv-97292-v4/default/model-00013-of-00027.safetensors
chaiml-pony-d1-q235b-pv-97292-v4-uploader: cp /dev/shm/model_output/model-00005-of-00027.safetensors s3://guanaco-vllm-models/chaiml-pony-d1-q235b-pv-97292-v4/default/model-00005-of-00027.safetensors
chaiml-pony-d1-q235b-pv-97292-v4-uploader: cp /dev/shm/model_output/model-00019-of-00027.safetensors s3://guanaco-vllm-models/chaiml-pony-d1-q235b-pv-97292-v4/default/model-00019-of-00027.safetensors
chaiml-pony-d1-q235b-pv-97292-v4-uploader: cp /dev/shm/model_output/model-00010-of-00027.safetensors s3://guanaco-vllm-models/chaiml-pony-d1-q235b-pv-97292-v4/default/model-00010-of-00027.safetensors
chaiml-pony-d1-q235b-pv-97292-v4-uploader: cp /dev/shm/model_output/model-00026-of-00027.safetensors s3://guanaco-vllm-models/chaiml-pony-d1-q235b-pv-97292-v4/default/model-00026-of-00027.safetensors
chaiml-pony-d1-q235b-pv-97292-v4-uploader: cp /dev/shm/model_output/model-00015-of-00027.safetensors s3://guanaco-vllm-models/chaiml-pony-d1-q235b-pv-97292-v4/default/model-00015-of-00027.safetensors
Job chaiml-pony-d1-q235b-pv-97292-v4-uploader completed after 159.21s with status: succeeded
Stopping job with name chaiml-pony-d1-q235b-pv-97292-v4-uploader
Pipeline stage VLLMUploader completed in 160.17s
run pipeline stage %s
Running pipeline stage VLLMTemplater
Pipeline stage VLLMTemplater completed in 0.24s
run pipeline stage %s
Running pipeline stage VLLMDeployer
Creating inference service chaiml-pony-d1-q235b-pv-97292-v4
Waiting for inference service chaiml-pony-d1-q235b-pv-97292-v4 to be ready
Inference service chaiml-pony-d1-q235b-pv-97292-v4 ready after 410.4109046459198s
Pipeline stage VLLMDeployer completed in 411.07s
run pipeline stage %s
Running pipeline stage StressChecker
HTTPConnectionPool(host='guanaco-submitter.guanaco-backend.k2.chaiverse.com', port=80): Read timed out. (read timeout=20)
Received unhealthy response to inference request!
Received healthy response to inference request in 1.9463603496551514s
Received healthy response to inference request in 2.191460609436035s
Received healthy response to inference request in 2.1644480228424072s
Received healthy response to inference request in 1.978987455368042s
Received healthy response to inference request in 1.9972047805786133s
Received healthy response to inference request in 1.8927850723266602s
Received healthy response to inference request in 1.91475248336792s
Received healthy response to inference request in 2.1360859870910645s
Received healthy response to inference request in 1.8810713291168213s
Received healthy response to inference request in 2.098517894744873s
Received healthy response to inference request in 1.9155333042144775s
Received healthy response to inference request in 1.8628149032592773s
Received healthy response to inference request in 2.1819937229156494s
Received healthy response to inference request in 2.0943193435668945s
Received healthy response to inference request in 2.187410831451416s
Received healthy response to inference request in 2.7060019969940186s
Received healthy response to inference request in 1.9824390411376953s
Received healthy response to inference request in 1.9899568557739258s
Received healthy response to inference request in 2.1084048748016357s
Received healthy response to inference request in 2.2210755348205566s
Received healthy response to inference request in 1.9953176975250244s
Received healthy response to inference request in 1.886974573135376s
Received healthy response to inference request in 2.007500648498535s
Received healthy response to inference request in 1.9928290843963623s
Received healthy response to inference request in 1.8799810409545898s
Received healthy response to inference request in 2.6469874382019043s
Received healthy response to inference request in 1.991896152496338s
Received healthy response to inference request in 1.9427695274353027s
Received healthy response to inference request in 2.0893139839172363s
30 requests
1 failed requests
5th percentile: 1.880471670627594
10th percentile: 1.8863842487335205
20th percentile: 1.915377140045166
30th percentile: 1.9691993236541747
40th percentile: 1.9911204338073731
50th percentile: 1.9962612390518188
60th percentile: 2.0913161277770995
70th percentile: 2.1167092084884644
80th percentile: 2.1830771446228026
90th percentile: 2.263666725158692
95th percentile: 2.679445445537567
99th percentile: 15.177757880687729
mean time: 2.6719016631444297
%s, retrying in %s seconds...
Received healthy response to inference request in 2.2931203842163086s
Received healthy response to inference request in 2.2756705284118652s
Received healthy response to inference request in 1.9953088760375977s
Received healthy response to inference request in 1.9238824844360352s
Received healthy response to inference request in 2.0536224842071533s
Received healthy response to inference request in 1.9850006103515625s
Received healthy response to inference request in 2.1170833110809326s
Received healthy response to inference request in 1.9572629928588867s
Received healthy response to inference request in 2.1836025714874268s
Received healthy response to inference request in 2.102477788925171s
Received healthy response to inference request in 1.8629257678985596s
Received healthy response to inference request in 1.9547691345214844s
Received healthy response to inference request in 1.905310869216919s
Received healthy response to inference request in 1.9230239391326904s
Received healthy response to inference request in 1.83935546875s
Received healthy response to inference request in 2.0169661045074463s
Received healthy response to inference request in 2.404712677001953s
Received healthy response to inference request in 1.9184603691101074s
Received healthy response to inference request in 2.010448455810547s
Received healthy response to inference request in 2.4254698753356934s
Received healthy response to inference request in 2.2294859886169434s
Received healthy response to inference request in 2.232821226119995s
Received healthy response to inference request in 1.904036521911621s
Received healthy response to inference request in 1.9145042896270752s
Received healthy response to inference request in 1.9115586280822754s
Received healthy response to inference request in 1.9047577381134033s
Received healthy response to inference request in 1.871291160583496s
Received healthy response to inference request in 1.9122490882873535s
Received healthy response to inference request in 2.0655312538146973s
Received healthy response to inference request in 1.889247179031372s
30 requests
0 failed requests
5th percentile: 1.866690194606781
10th percentile: 1.8874515771865845
20th percentile: 1.9052002429962158
30th percentile: 1.9138277292251586
40th percentile: 1.9235390663146972
50th percentile: 1.9711318016052246
60th percentile: 2.013055515289307
70th percentile: 2.0766152143478394
80th percentile: 2.1927792549133303
90th percentile: 2.2774155139923096
95th percentile: 2.3544961452484126
99th percentile: 2.4194502878189086
mean time: 2.0327985922495526
Pipeline stage StressChecker completed in 157.88s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyTriggerPipeline
run_pipeline:run_in_cloud %s
starting trigger_guanaco_pipeline args=%s
triggered trigger_guanaco_pipeline args=%s
Pipeline stage OfflineFamilyFriendlyTriggerPipeline completed in 2.11s
Shutdown handler de-registered
chaiml-pony-d1-q235b-pv_97292_v4 status is now deployed due to DeploymentManager action
chaiml-pony-d1-q235b-pv_97292_v4 status is now inactive due to auto deactivation removed underperforming models