chaiml-pony-v2-q235b-lr_18913

developer_uid: zonemercy
submission_id: chaiml-pony-v2-q235b-lr_18913_v3
model_name: chaiml-pony-v2-q235b-lr_18913_v3
model_group: ChaiML/pony-v2-q235b-lr1
status: torndown
timestamp: 2026-03-02T16:21:11+00:00
num_battles: 11532
num_wins: 6276
celo_rating: 1321.34
family_friendly_score: 0.0
family_friendly_standard_error: 0.0
submission_type: basic
model_repo: ChaiML/pony-v2-q235b-lr1e4ep1r64g4
model_architecture: Qwen3MoeForCausalLM
model_num_parameters: 18790207488.0
best_of: 8
max_input_tokens: 2048
max_output_tokens: 80
reward_model: default
display_name: chaiml-pony-v2-q235b-lr_18913_v3
ineligible_reason: max_output_tokens!=64
is_internal_developer: True
language_model: ChaiML/pony-v2-q235b-lr1e4ep1r64g4
model_size: 19B
ranking_group: single
us_pacific_date: 2026-02-27
win_ratio: 0.5442247658688866
generation_params: {'temperature': 1.0, 'top_p': 1.0, 'min_p': 0.0, 'top_k': 40, 'presence_penalty': 0.0, 'frequency_penalty': 0.0, 'stopping_words': ['<|assistant|>', '<|user|>', '<|im_end|>', '</s>', '####', '</think>'], 'max_input_tokens': 2048, 'best_of': 8, 'max_output_tokens': 80}
formatter: {'memory_template': "<|im_start|>system\n{bot_name}'s persona: {memory}<|im_end|>\n", 'prompt_template': '', 'bot_template': '<|im_start|>assistant\n{bot_name}: {message}<|im_end|>\n', 'user_template': '<|im_start|>user\n{message}<|im_end|>\n', 'response_template': '<|im_start|>assistant\n{bot_name}:', 'truncate_by_message': True}
Resubmit model
Shutdown handler not registered because Python interpreter is not running in the main thread
run pipeline %s
run pipeline stage %s
Running pipeline stage VLLMUploader
Starting job with name chaiml-pony-v2-q235b-lr-18913-v3-uploader
Waiting for job on chaiml-pony-v2-q235b-lr-18913-v3-uploader to finish
chaiml-pony-v2-q235b-lr-18913-v3-uploader: Using quantization_mode: w4a16
chaiml-pony-v2-q235b-lr-18913-v3-uploader: Checking if ChaiML/pony-v2-q235b-lr1e4ep1r64g4-W4A16 already exists in ChaiML
chaiml-pony-v2-q235b-lr-18913-v3-uploader: Model already exists. Downloading to /dev/shm/model_output...
chaiml-pony-v2-q235b-lr-18913-v3-uploader: Downloading snapshot of ChaiML/pony-v2-q235b-lr1e4ep1r64g4-W4A16...
chaiml-pony-v2-q235b-lr-18913-v3-uploader: Downloaded in 54.129s
chaiml-pony-v2-q235b-lr-18913-v3-uploader: Processed model ChaiML/pony-v2-q235b-lr1e4ep1r64g4 in 54.759s
chaiml-pony-v2-q235b-lr-18913-v3-uploader: creating bucket guanaco-vllm-models
chaiml-pony-v2-q235b-lr-18913-v3-uploader: /usr/lib/python3/dist-packages/S3/BaseUtils.py:56: SyntaxWarning: invalid escape sequence '\.'
chaiml-pony-v2-q235b-lr-18913-v3-uploader:   RE_S3_DATESTRING = re.compile('\.[0-9]*(?:[Z\\-\\+]*?)')
chaiml-pony-v2-q235b-lr-18913-v3-uploader: /usr/lib/python3/dist-packages/S3/BaseUtils.py:57: SyntaxWarning: invalid escape sequence '\s'
chaiml-pony-v2-q235b-lr-18913-v3-uploader:   RE_XML_NAMESPACE = re.compile(b'^(<?[^>]+?>\s*|\s*)(<\w+) xmlns=[\'"](https?://[^\'"]+)[\'"]', re.MULTILINE)
chaiml-pony-v2-q235b-lr-18913-v3-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:240: SyntaxWarning: invalid escape sequence '\.'
chaiml-pony-v2-q235b-lr-18913-v3-uploader:   invalid = re.search("([^a-z0-9\.-])", bucket, re.UNICODE)
chaiml-pony-v2-q235b-lr-18913-v3-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:244: SyntaxWarning: invalid escape sequence '\.'
chaiml-pony-v2-q235b-lr-18913-v3-uploader:   invalid = re.search("([^A-Za-z0-9\._-])", bucket, re.UNICODE)
chaiml-pony-v2-q235b-lr-18913-v3-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:255: SyntaxWarning: invalid escape sequence '\.'
chaiml-pony-v2-q235b-lr-18913-v3-uploader:   if re.search("-\.", bucket, re.UNICODE):
chaiml-pony-v2-q235b-lr-18913-v3-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:257: SyntaxWarning: invalid escape sequence '\.'
chaiml-pony-v2-q235b-lr-18913-v3-uploader:   if re.search("\.\.", bucket, re.UNICODE):
chaiml-pony-v2-q235b-lr-18913-v3-uploader: /usr/lib/python3/dist-packages/S3/S3Uri.py:155: SyntaxWarning: invalid escape sequence '\w'
chaiml-pony-v2-q235b-lr-18913-v3-uploader:   _re = re.compile("^(\w+://)?(.*)", re.UNICODE)
chaiml-pony-v2-q235b-lr-18913-v3-uploader: /usr/lib/python3/dist-packages/S3/FileLists.py:480: SyntaxWarning: invalid escape sequence '\*'
chaiml-pony-v2-q235b-lr-18913-v3-uploader:   wildcard_split_result = re.split("\*|\?", uri_str, maxsplit=1)
chaiml-pony-v2-q235b-lr-18913-v3-uploader: Bucket 's3://guanaco-vllm-models/' created
chaiml-pony-v2-q235b-lr-18913-v3-uploader: uploading /dev/shm/model_output to s3://guanaco-vllm-models/chaiml-pony-v2-q235b-lr-18913-v3/default
chaiml-pony-v2-q235b-lr-18913-v3-uploader: cp /dev/shm/model_output/added_tokens.json s3://guanaco-vllm-models/chaiml-pony-v2-q235b-lr-18913-v3/default/added_tokens.json
chaiml-pony-v2-q235b-lr-18913-v3-uploader: cp /dev/shm/model_output/chat_template.jinja s3://guanaco-vllm-models/chaiml-pony-v2-q235b-lr-18913-v3/default/chat_template.jinja
chaiml-pony-v2-q235b-lr-18913-v3-uploader: cp /dev/shm/model_output/tokenizer_config.json s3://guanaco-vllm-models/chaiml-pony-v2-q235b-lr-18913-v3/default/tokenizer_config.json
chaiml-pony-v2-q235b-lr-18913-v3-uploader: cp /dev/shm/model_output/generation_config.json s3://guanaco-vllm-models/chaiml-pony-v2-q235b-lr-18913-v3/default/generation_config.json
chaiml-pony-v2-q235b-lr-18913-v3-uploader: cp /dev/shm/model_output/tokenizer.json s3://guanaco-vllm-models/chaiml-pony-v2-q235b-lr-18913-v3/default/tokenizer.json
chaiml-pony-v2-q235b-lr-18913-v3-uploader: cp /dev/shm/model_output/model.safetensors.index.json s3://guanaco-vllm-models/chaiml-pony-v2-q235b-lr-18913-v3/default/model.safetensors.index.json
chaiml-pony-v2-q235b-lr-18913-v3-uploader: cp /dev/shm/model_output/merges.txt s3://guanaco-vllm-models/chaiml-pony-v2-q235b-lr-18913-v3/default/merges.txt
chaiml-pony-v2-q235b-lr-18913-v3-uploader: cp /dev/shm/model_output/quantization_config.json s3://guanaco-vllm-models/chaiml-pony-v2-q235b-lr-18913-v3/default/quantization_config.json
chaiml-pony-v2-q235b-lr-18913-v3-uploader: cp /dev/shm/model_output/vocab.json s3://guanaco-vllm-models/chaiml-pony-v2-q235b-lr-18913-v3/default/vocab.json
chaiml-pony-v2-q235b-lr-18913-v3-uploader: cp /dev/shm/model_output/.gitattributes s3://guanaco-vllm-models/chaiml-pony-v2-q235b-lr-18913-v3/default/.gitattributes
chaiml-pony-v2-q235b-lr-18913-v3-uploader: cp /dev/shm/model_output/config.json s3://guanaco-vllm-models/chaiml-pony-v2-q235b-lr-18913-v3/default/config.json
chaiml-pony-v2-q235b-lr-18913-v3-uploader: cp /dev/shm/model_output/special_tokens_map.json s3://guanaco-vllm-models/chaiml-pony-v2-q235b-lr-18913-v3/default/special_tokens_map.json
chaiml-pony-v2-q235b-lr-18913-v3-uploader: cp /dev/shm/model_output/model-00027-of-00027.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-q235b-lr-18913-v3/default/model-00027-of-00027.safetensors
chaiml-pony-v2-q235b-lr-18913-v3-uploader: cp /dev/shm/model_output/model-00013-of-00027.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-q235b-lr-18913-v3/default/model-00013-of-00027.safetensors
chaiml-pony-v2-q235b-lr-18913-v3-uploader: cp /dev/shm/model_output/model-00015-of-00027.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-q235b-lr-18913-v3/default/model-00015-of-00027.safetensors
chaiml-pony-v2-q235b-lr-18913-v3-uploader: cp /dev/shm/model_output/model-00019-of-00027.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-q235b-lr-18913-v3/default/model-00019-of-00027.safetensors
chaiml-pony-v2-q235b-lr-18913-v3-uploader: cp /dev/shm/model_output/model-00002-of-00027.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-q235b-lr-18913-v3/default/model-00002-of-00027.safetensors
chaiml-pony-v2-q235b-lr-18913-v3-uploader: cp /dev/shm/model_output/model-00014-of-00027.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-q235b-lr-18913-v3/default/model-00014-of-00027.safetensors
chaiml-pony-v2-q235b-lr-18913-v3-uploader: cp /dev/shm/model_output/model-00012-of-00027.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-q235b-lr-18913-v3/default/model-00012-of-00027.safetensors
chaiml-pony-v2-q235b-lr-18913-v3-uploader: cp /dev/shm/model_output/model-00008-of-00027.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-q235b-lr-18913-v3/default/model-00008-of-00027.safetensors
chaiml-pony-v2-q235b-lr-18913-v3-uploader: cp /dev/shm/model_output/model-00009-of-00027.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-q235b-lr-18913-v3/default/model-00009-of-00027.safetensors
chaiml-pony-v2-q235b-lr-18913-v3-uploader: cp /dev/shm/model_output/model-00021-of-00027.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-q235b-lr-18913-v3/default/model-00021-of-00027.safetensors
chaiml-pony-v2-q235b-lr-18913-v3-uploader: cp /dev/shm/model_output/model-00023-of-00027.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-q235b-lr-18913-v3/default/model-00023-of-00027.safetensors
chaiml-pony-v2-q235b-lr-18913-v3-uploader: cp /dev/shm/model_output/model-00016-of-00027.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-q235b-lr-18913-v3/default/model-00016-of-00027.safetensors
chaiml-pony-v2-q235b-lr-18913-v3-uploader: cp /dev/shm/model_output/model-00026-of-00027.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-q235b-lr-18913-v3/default/model-00026-of-00027.safetensors
chaiml-pony-v2-q235b-lr-18913-v3-uploader: cp /dev/shm/model_output/model-00024-of-00027.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-q235b-lr-18913-v3/default/model-00024-of-00027.safetensors
chaiml-pony-v2-q235b-lr-18913-v3-uploader: cp /dev/shm/model_output/model-00004-of-00027.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-q235b-lr-18913-v3/default/model-00004-of-00027.safetensors
chaiml-pony-v2-q235b-lr-18913-v3-uploader: cp /dev/shm/model_output/model-00007-of-00027.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-q235b-lr-18913-v3/default/model-00007-of-00027.safetensors
chaiml-pony-v2-q235b-lr-18913-v3-uploader: cp /dev/shm/model_output/model-00025-of-00027.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-q235b-lr-18913-v3/default/model-00025-of-00027.safetensors
chaiml-pony-v2-q235b-lr-18913-v3-uploader: cp /dev/shm/model_output/model-00001-of-00027.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-q235b-lr-18913-v3/default/model-00001-of-00027.safetensors
chaiml-pony-v2-q235b-lr-18913-v3-uploader: cp /dev/shm/model_output/model-00017-of-00027.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-q235b-lr-18913-v3/default/model-00017-of-00027.safetensors
chaiml-pony-v2-q235b-lr-18913-v3-uploader: cp /dev/shm/model_output/model-00022-of-00027.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-q235b-lr-18913-v3/default/model-00022-of-00027.safetensors
chaiml-pony-v2-q235b-lr-18913-v3-uploader: cp /dev/shm/model_output/model-00011-of-00027.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-q235b-lr-18913-v3/default/model-00011-of-00027.safetensors
chaiml-pony-v2-q235b-lr-18913-v3-uploader: cp /dev/shm/model_output/model-00020-of-00027.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-q235b-lr-18913-v3/default/model-00020-of-00027.safetensors
chaiml-pony-v2-q235b-lr-18913-v3-uploader: cp /dev/shm/model_output/model-00006-of-00027.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-q235b-lr-18913-v3/default/model-00006-of-00027.safetensors
chaiml-pony-v2-q235b-lr-18913-v3-uploader: cp /dev/shm/model_output/model-00018-of-00027.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-q235b-lr-18913-v3/default/model-00018-of-00027.safetensors
chaiml-pony-v2-q235b-lr-18913-v3-uploader: cp /dev/shm/model_output/model-00010-of-00027.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-q235b-lr-18913-v3/default/model-00010-of-00027.safetensors
chaiml-pony-v2-q235b-lr-18913-v3-uploader: cp /dev/shm/model_output/model-00003-of-00027.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-q235b-lr-18913-v3/default/model-00003-of-00027.safetensors
chaiml-pony-v2-q235b-lr-18913-v3-uploader: cp /dev/shm/model_output/model-00005-of-00027.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-q235b-lr-18913-v3/default/model-00005-of-00027.safetensors
Job chaiml-pony-v2-q235b-lr-18913-v3-uploader completed after 159.45s with status: succeeded
Stopping job with name chaiml-pony-v2-q235b-lr-18913-v3-uploader
Pipeline stage VLLMUploader completed in 159.87s
run pipeline stage %s
Running pipeline stage VLLMTemplater
Pipeline stage VLLMTemplater completed in 0.30s
run pipeline stage %s
Running pipeline stage VLLMDeployer
Creating inference service chaiml-pony-v2-q235b-lr-18913-v3
Waiting for inference service chaiml-pony-v2-q235b-lr-18913-v3 to be ready
Failed to get response for submission chaiml-98p-2ff-chaiml-m_32069_v1: ('http://chaiml-98p-2ff-chaiml-m-32069-v1-predictor.tenant-chaiml-guanaco.kchai-coreweave-us-east-04a.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', '')
Inference service chaiml-pony-v2-q235b-lr-18913-v3 ready after 491.55557465553284s
Pipeline stage VLLMDeployer completed in 492.02s
run pipeline stage %s
Running pipeline stage StressChecker
Received healthy response to inference request in 2.012413263320923s
Received healthy response to inference request in 2.052706241607666s
Received healthy response to inference request in 2.276116371154785s
Received healthy response to inference request in 2.3886096477508545s
Received healthy response to inference request in 2.182119131088257s
Received healthy response to inference request in 1.9724838733673096s
Received healthy response to inference request in 2.0915310382843018s
Received healthy response to inference request in 1.9364190101623535s
Received healthy response to inference request in 2.048860549926758s
Received healthy response to inference request in 2.019212007522583s
Received healthy response to inference request in 1.9996130466461182s
Received healthy response to inference request in 2.3632407188415527s
Received healthy response to inference request in 2.2645626068115234s
Received healthy response to inference request in 2.268221378326416s
Received healthy response to inference request in 1.9922099113464355s
Received healthy response to inference request in 1.9203126430511475s
Received healthy response to inference request in 1.8812243938446045s
Received healthy response to inference request in 1.9167611598968506s
Received healthy response to inference request in 1.9927818775177002s
Received healthy response to inference request in 2.0916361808776855s
Received healthy response to inference request in 1.9138550758361816s
Received healthy response to inference request in 1.9021897315979004s
Received healthy response to inference request in 1.8894805908203125s
Received healthy response to inference request in 1.8822429180145264s
Received healthy response to inference request in 1.8765857219696045s
Received healthy response to inference request in 2.003929853439331s
Received healthy response to inference request in 1.9202706813812256s
Received healthy response to inference request in 1.9507989883422852s
Received healthy response to inference request in 2.0905425548553467s
Received healthy response to inference request in 2.265597343444824s
30 requests
0 failed requests
5th percentile: 1.8816827297210694
10th percentile: 1.8887568235397338
20th percentile: 1.9161799430847168
30th percentile: 1.9315871000289917
40th percentile: 1.9843194961547852
50th percentile: 2.0017714500427246
60th percentile: 2.031071424484253
70th percentile: 2.0908390998840334
80th percentile: 2.1986078262329105
90th percentile: 2.269010877609253
95th percentile: 2.324034762382507
99th percentile: 2.381252658367157
mean time: 2.0455509503682454
Pipeline stage StressChecker completed in 64.78s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyTriggerPipeline
run_pipeline:run_in_cloud %s
starting trigger_guanaco_pipeline args=%s
triggered trigger_guanaco_pipeline args=%s
Pipeline stage OfflineFamilyFriendlyTriggerPipeline completed in 0.77s
Shutdown handler de-registered
chaiml-pony-v2-q235b-lr_18913_v3 status is now deployed due to DeploymentManager action
chaiml-pony-v2-q235b-lr_18913_v3 status is now inactive due to auto deactivation removed underperforming models
chaiml-pony-v2-q235b-lr_18913_v3 status is now torndown due to DeploymentManager action