developer_uid: richhx
submission_id: chaiml-pony-d3-g46-pv2-l_7830_v2
model_name: chaiml-pony-d3-g46-pv2-l_7830_v2
model_group: ChaiML/pony-d3-g46-pv2-l
status: deployed
timestamp: 2026-03-27T18:05:55+00:00
num_battles: 9166
num_wins: 5277
celo_rating: 1346.86
family_friendly_score: 0.0
family_friendly_standard_error: 0.0
submission_type: basic
model_repo: ChaiML/pony-d3-g46-pv2-lr5e6ep1r64b8-FP8
model_architecture: Glm4MoeForCausalLM
model_num_parameters: 24110003200.0
best_of: 8
max_input_tokens: 1500
max_output_tokens: 80
reward_model: default
display_name: chaiml-pony-d3-g46-pv2-l_7830_v2
ineligible_reason: max_output_tokens!=64
is_internal_developer: True
language_model: ChaiML/pony-d3-g46-pv2-lr5e6ep1r64b8-FP8
model_size: 24B
ranking_group: single
us_pacific_date: 2026-03-27
win_ratio: 0.5757145974252673
generation_params: {'temperature': 1.0, 'top_p': 0.95, 'min_p': 0.0, 'top_k': 60, 'presence_penalty': 0.0, 'frequency_penalty': 0.0, 'stopping_words': ['<|im_end|>', '</s>', '<|user|>', '####', '<|assistant|>'], 'max_input_tokens': 1500, 'best_of': 8, 'max_output_tokens': 80}
formatter: {'memory_template': "[gMASK]<sop><|system|>\n{bot_name}'s persona: {memory}", 'prompt_template': '', 'bot_template': '<|assistant|>\n{bot_name}: {message}', 'user_template': '<|user|>\n{message}', 'response_template': '<|assistant|>\n<think></think>\n{bot_name}:', 'truncate_by_message': True}
Resubmit model
Shutdown handler not registered because Python interpreter is not running in the main thread
run pipeline %s
run pipeline stage %s
Running pipeline stage VLLMUploader
Starting job with name chaiml-pony-d3-g46-pv2-l-7830-v2-uploader
Waiting for job on chaiml-pony-d3-g46-pv2-l-7830-v2-uploader to finish
chaiml-pony-d3-g46-pv2-l-7830-v2-uploader: {repo_id} is already quantized
chaiml-pony-d3-g46-pv2-l-7830-v2-uploader: Using quantization_mode: none
chaiml-pony-d3-g46-pv2-l-7830-v2-uploader: Downloading snapshot of ChaiML/pony-d3-g46-pv2-lr5e6ep1r64b8-FP8...
2026-03-27T17:52:36.489893+00:00 monitor updated for chaiml-pony-d3-g46-pv2-_13283_v2
2026-03-27T17:52:38.311253+00:00 monitor updated for chaiml-pony-d3-g46-pv2-l_7830_v2
2026-03-27T17:53:36.793451+00:00 monitor updated for chaiml-pony-d3-g46-pv2-_13283_v2
2026-03-27T17:53:38.623510+00:00 monitor updated for chaiml-pony-d3-g46-pv2-l_7830_v2
chaiml-pony-d3-g46-pv2-13283-v2-uploader: Downloaded in 80.608s
2026-03-27T17:54:37.099810+00:00 monitor updated for chaiml-pony-d3-g46-pv2-_13283_v2
chaiml-pony-d3-g46-pv2-l-7830-v2-uploader: Downloaded in 135.875s
2026-03-27T17:54:38.919904+00:00 monitor updated for chaiml-pony-d3-g46-pv2-l_7830_v2
chaiml-pony-d3-g46-pv2-13283-v2-uploader: Processed model ChaiML/pony-d3-g46-pv2-lr5e6ep1r64b8-W4A16 in 149.466s
chaiml-pony-d3-g46-pv2-13283-v2-uploader: creating bucket guanaco-vllm-models
chaiml-pony-d3-g46-pv2-13283-v2-uploader: /usr/lib/python3/dist-packages/S3/BaseUtils.py:56: SyntaxWarning: invalid escape sequence '\.'
chaiml-pony-d3-g46-pv2-13283-v2-uploader:   RE_S3_DATESTRING = re.compile('\.[0-9]*(?:[Z\\-\\+]*?)')
chaiml-pony-d3-g46-pv2-13283-v2-uploader: /usr/lib/python3/dist-packages/S3/BaseUtils.py:57: SyntaxWarning: invalid escape sequence '\s'
chaiml-pony-d3-g46-pv2-13283-v2-uploader:   RE_XML_NAMESPACE = re.compile(b'^(<?[^>]+?>\s*|\s*)(<\w+) xmlns=[\'"](https?://[^\'"]+)[\'"]', re.MULTILINE)
chaiml-pony-d3-g46-pv2-13283-v2-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:240: SyntaxWarning: invalid escape sequence '\.'
chaiml-pony-d3-g46-pv2-13283-v2-uploader:   invalid = re.search("([^a-z0-9\.-])", bucket, re.UNICODE)
chaiml-pony-d3-g46-pv2-13283-v2-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:244: SyntaxWarning: invalid escape sequence '\.'
chaiml-pony-d3-g46-pv2-13283-v2-uploader:   invalid = re.search("([^A-Za-z0-9\._-])", bucket, re.UNICODE)
chaiml-pony-d3-g46-pv2-13283-v2-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:255: SyntaxWarning: invalid escape sequence '\.'
chaiml-pony-d3-g46-pv2-13283-v2-uploader:   if re.search("-\.", bucket, re.UNICODE):
chaiml-pony-d3-g46-pv2-13283-v2-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:257: SyntaxWarning: invalid escape sequence '\.'
chaiml-pony-d3-g46-pv2-13283-v2-uploader:   if re.search("\.\.", bucket, re.UNICODE):
chaiml-pony-d3-g46-pv2-13283-v2-uploader: /usr/lib/python3/dist-packages/S3/S3Uri.py:155: SyntaxWarning: invalid escape sequence '\w'
chaiml-pony-d3-g46-pv2-13283-v2-uploader:   _re = re.compile("^(\w+://)?(.*)", re.UNICODE)
chaiml-pony-d3-g46-pv2-13283-v2-uploader: /usr/lib/python3/dist-packages/S3/FileLists.py:480: SyntaxWarning: invalid escape sequence '\*'
chaiml-pony-d3-g46-pv2-13283-v2-uploader:   wildcard_split_result = re.split("\*|\?", uri_str, maxsplit=1)
chaiml-pony-d3-g46-pv2-13283-v2-uploader: Bucket 's3://guanaco-vllm-models/' created
chaiml-pony-d3-g46-pv2-13283-v2-uploader: uploading /dev/shm/model_output to s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-13283-v2/default
chaiml-pony-d3-g46-pv2-13283-v2-uploader: cp /dev/shm/model_output/generation_config.json s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-13283-v2/default/generation_config.json
chaiml-pony-d3-g46-pv2-13283-v2-uploader: cp /dev/shm/model_output/chat_template.jinja s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-13283-v2/default/chat_template.jinja
chaiml-pony-d3-g46-pv2-13283-v2-uploader: cp /dev/shm/model_output/config.json s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-13283-v2/default/config.json
chaiml-pony-d3-g46-pv2-13283-v2-uploader: cp /dev/shm/model_output/.gitattributes s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-13283-v2/default/.gitattributes
chaiml-pony-d3-g46-pv2-13283-v2-uploader: cp /dev/shm/model_output/quantization_config.json s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-13283-v2/default/quantization_config.json
chaiml-pony-d3-g46-pv2-13283-v2-uploader: cp /dev/shm/model_output/tokenizer_config.json s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-13283-v2/default/tokenizer_config.json
chaiml-pony-d3-g46-pv2-13283-v2-uploader: cp /dev/shm/model_output/tokenizer.json s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-13283-v2/default/tokenizer.json
chaiml-pony-d3-g46-pv2-13283-v2-uploader: cp /dev/shm/model_output/model.safetensors.index.json s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-13283-v2/default/model.safetensors.index.json
chaiml-pony-d3-g46-pv2-13283-v2-uploader: cp /dev/shm/model_output/model-00038-of-00038.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-13283-v2/default/model-00038-of-00038.safetensors
chaiml-pony-d3-g46-pv2-13283-v2-uploader: cp /dev/shm/model_output/model-00037-of-00038.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-13283-v2/default/model-00037-of-00038.safetensors
chaiml-pony-d3-g46-pv2-13283-v2-uploader: cp /dev/shm/model_output/model-00036-of-00038.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-13283-v2/default/model-00036-of-00038.safetensors
chaiml-pony-d3-g46-pv2-13283-v2-uploader: cp /dev/shm/model_output/model-00007-of-00038.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-13283-v2/default/model-00007-of-00038.safetensors
2026-03-27T17:55:37.404140+00:00 monitor updated for chaiml-pony-d3-g46-pv2-_13283_v2
2026-03-27T17:55:39.195023+00:00 monitor updated for chaiml-pony-d3-g46-pv2-l_7830_v2
chaiml-pony-d3-g46-pv2-13283-v2-uploader: cp /dev/shm/model_output/model-00035-of-00038.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-13283-v2/default/model-00035-of-00038.safetensors
chaiml-pony-d3-g46-pv2-13283-v2-uploader: cp /dev/shm/model_output/model-00011-of-00038.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-13283-v2/default/model-00011-of-00038.safetensors
chaiml-pony-d3-g46-pv2-13283-v2-uploader: cp /dev/shm/model_output/model-00016-of-00038.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-13283-v2/default/model-00016-of-00038.safetensors
chaiml-pony-d3-g46-pv2-13283-v2-uploader: cp /dev/shm/model_output/model-00014-of-00038.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-13283-v2/default/model-00014-of-00038.safetensors
chaiml-pony-d3-g46-pv2-13283-v2-uploader: cp /dev/shm/model_output/model-00027-of-00038.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-13283-v2/default/model-00027-of-00038.safetensors
chaiml-pony-d3-g46-pv2-13283-v2-uploader: cp /dev/shm/model_output/model-00001-of-00038.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-13283-v2/default/model-00001-of-00038.safetensors
chaiml-pony-d3-g46-pv2-13283-v2-uploader: cp /dev/shm/model_output/model-00006-of-00038.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-13283-v2/default/model-00006-of-00038.safetensors
chaiml-pony-d3-g46-pv2-13283-v2-uploader: cp /dev/shm/model_output/model-00029-of-00038.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-13283-v2/default/model-00029-of-00038.safetensors
chaiml-pony-d3-g46-pv2-13283-v2-uploader: cp /dev/shm/model_output/model-00017-of-00038.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-13283-v2/default/model-00017-of-00038.safetensors
chaiml-pony-d3-g46-pv2-13283-v2-uploader: cp /dev/shm/model_output/model-00018-of-00038.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-13283-v2/default/model-00018-of-00038.safetensors
chaiml-pony-d3-g46-pv2-13283-v2-uploader: cp /dev/shm/model_output/model-00021-of-00038.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-13283-v2/default/model-00021-of-00038.safetensors
chaiml-pony-d3-g46-pv2-13283-v2-uploader: cp /dev/shm/model_output/model-00002-of-00038.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-13283-v2/default/model-00002-of-00038.safetensors
chaiml-pony-d3-g46-pv2-13283-v2-uploader: cp /dev/shm/model_output/model-00005-of-00038.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-13283-v2/default/model-00005-of-00038.safetensors
chaiml-pony-d3-g46-pv2-13283-v2-uploader: cp /dev/shm/model_output/model-00004-of-00038.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-13283-v2/default/model-00004-of-00038.safetensors
chaiml-pony-d3-g46-pv2-13283-v2-uploader: cp /dev/shm/model_output/model-00030-of-00038.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-13283-v2/default/model-00030-of-00038.safetensors
chaiml-pony-d3-g46-pv2-13283-v2-uploader: cp /dev/shm/model_output/model-00024-of-00038.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-13283-v2/default/model-00024-of-00038.safetensors
chaiml-pony-d3-g46-pv2-13283-v2-uploader: cp /dev/shm/model_output/model-00009-of-00038.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-13283-v2/default/model-00009-of-00038.safetensors
chaiml-pony-d3-g46-pv2-13283-v2-uploader: cp /dev/shm/model_output/model-00033-of-00038.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-13283-v2/default/model-00033-of-00038.safetensors
chaiml-pony-d3-g46-pv2-13283-v2-uploader: cp /dev/shm/model_output/model-00025-of-00038.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-13283-v2/default/model-00025-of-00038.safetensors
chaiml-pony-d3-g46-pv2-13283-v2-uploader: cp /dev/shm/model_output/model-00019-of-00038.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-13283-v2/default/model-00019-of-00038.safetensors
chaiml-pony-d3-g46-pv2-13283-v2-uploader: cp /dev/shm/model_output/model-00012-of-00038.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-13283-v2/default/model-00012-of-00038.safetensors
chaiml-pony-d3-g46-pv2-13283-v2-uploader: cp /dev/shm/model_output/model-00034-of-00038.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-13283-v2/default/model-00034-of-00038.safetensors
chaiml-pony-d3-g46-pv2-13283-v2-uploader: cp /dev/shm/model_output/model-00015-of-00038.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-13283-v2/default/model-00015-of-00038.safetensors
chaiml-pony-d3-g46-pv2-13283-v2-uploader: cp /dev/shm/model_output/model-00013-of-00038.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-13283-v2/default/model-00013-of-00038.safetensors
chaiml-pony-d3-g46-pv2-13283-v2-uploader: cp /dev/shm/model_output/model-00022-of-00038.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-13283-v2/default/model-00022-of-00038.safetensors
chaiml-pony-d3-g46-pv2-13283-v2-uploader: cp /dev/shm/model_output/model-00008-of-00038.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-13283-v2/default/model-00008-of-00038.safetensors
chaiml-pony-d3-g46-pv2-13283-v2-uploader: cp /dev/shm/model_output/model-00023-of-00038.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-13283-v2/default/model-00023-of-00038.safetensors
chaiml-pony-d3-g46-pv2-13283-v2-uploader: cp /dev/shm/model_output/model-00026-of-00038.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-13283-v2/default/model-00026-of-00038.safetensors
chaiml-pony-d3-g46-pv2-13283-v2-uploader: cp /dev/shm/model_output/model-00020-of-00038.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-13283-v2/default/model-00020-of-00038.safetensors
chaiml-pony-d3-g46-pv2-13283-v2-uploader: cp /dev/shm/model_output/model-00010-of-00038.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-13283-v2/default/model-00010-of-00038.safetensors
chaiml-pony-d3-g46-pv2-13283-v2-uploader: cp /dev/shm/model_output/model-00028-of-00038.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-13283-v2/default/model-00028-of-00038.safetensors
chaiml-pony-d3-g46-pv2-13283-v2-uploader: cp /dev/shm/model_output/model-00003-of-00038.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-13283-v2/default/model-00003-of-00038.safetensors
chaiml-pony-d3-g46-pv2-13283-v2-uploader: cp /dev/shm/model_output/model-00031-of-00038.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-13283-v2/default/model-00031-of-00038.safetensors
chaiml-pony-d3-g46-pv2-13283-v2-uploader: cp /dev/shm/model_output/model-00032-of-00038.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-13283-v2/default/model-00032-of-00038.safetensors
Job chaiml-pony-d3-g46-pv2-13283-v2-uploader completed after 266.65s with status: succeeded
Stopping job with name chaiml-pony-d3-g46-pv2-13283-v2-uploader
Pipeline stage VLLMUploader completed in 268.27s
run pipeline stage %s
Running pipeline stage VLLMUploaderAMD
Pipeline stage vllm_upload_amd skipped, reason=not amd cluster
Pipeline stage VLLMUploaderAMD completed in 0.43s
run pipeline stage %s
Running pipeline stage VLLMTemplater
Pipeline stage VLLMTemplater completed in 1.38s
run pipeline stage %s
Running pipeline stage VLLMDeployer
Creating inference service chaiml-pony-d3-g46-pv2-13283-v2
Waiting for inference service chaiml-pony-d3-g46-pv2-13283-v2 to be ready
chaiml-pony-d3-g46-pv2-l-7830-v2-uploader: Processed model ChaiML/pony-d3-g46-pv2-lr5e6ep1r64b8-FP8 in 262.329s
chaiml-pony-d3-g46-pv2-l-7830-v2-uploader: creating bucket guanaco-vllm-models
chaiml-pony-d3-g46-pv2-l-7830-v2-uploader: /usr/lib/python3/dist-packages/S3/BaseUtils.py:56: SyntaxWarning: invalid escape sequence '\.'
chaiml-pony-d3-g46-pv2-l-7830-v2-uploader:   RE_S3_DATESTRING = re.compile('\.[0-9]*(?:[Z\\-\\+]*?)')
2026-03-27T17:56:44.236689+00:00 monitor updated for chaiml-pony-d3-g46-pv2-_13283_v2
2026-03-27T17:56:44.325319+00:00 monitor updated for chaiml-pony-d3-g46-pv2-l_7830_v2
chaiml-pony-d3-g46-pv2-l-7830-v2-uploader: /usr/lib/python3/dist-packages/S3/BaseUtils.py:57: SyntaxWarning: invalid escape sequence '\s'
chaiml-pony-d3-g46-pv2-l-7830-v2-uploader:   RE_XML_NAMESPACE = re.compile(b'^(<?[^>]+?>\s*|\s*)(<\w+) xmlns=[\'"](https?://[^\'"]+)[\'"]', re.MULTILINE)
chaiml-pony-d3-g46-pv2-l-7830-v2-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:240: SyntaxWarning: invalid escape sequence '\.'
chaiml-pony-d3-g46-pv2-l-7830-v2-uploader:   invalid = re.search("([^a-z0-9\.-])", bucket, re.UNICODE)
chaiml-pony-d3-g46-pv2-l-7830-v2-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:244: SyntaxWarning: invalid escape sequence '\.'
chaiml-pony-d3-g46-pv2-l-7830-v2-uploader:   invalid = re.search("([^A-Za-z0-9\._-])", bucket, re.UNICODE)
chaiml-pony-d3-g46-pv2-l-7830-v2-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:255: SyntaxWarning: invalid escape sequence '\.'
chaiml-pony-d3-g46-pv2-l-7830-v2-uploader:   if re.search("-\.", bucket, re.UNICODE):
chaiml-pony-d3-g46-pv2-l-7830-v2-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:257: SyntaxWarning: invalid escape sequence '\.'
chaiml-pony-d3-g46-pv2-l-7830-v2-uploader:   if re.search("\.\.", bucket, re.UNICODE):
chaiml-pony-d3-g46-pv2-l-7830-v2-uploader: /usr/lib/python3/dist-packages/S3/S3Uri.py:155: SyntaxWarning: invalid escape sequence '\w'
chaiml-pony-d3-g46-pv2-l-7830-v2-uploader:   _re = re.compile("^(\w+://)?(.*)", re.UNICODE)
chaiml-pony-d3-g46-pv2-l-7830-v2-uploader: /usr/lib/python3/dist-packages/S3/FileLists.py:480: SyntaxWarning: invalid escape sequence '\*'
chaiml-pony-d3-g46-pv2-l-7830-v2-uploader:   wildcard_split_result = re.split("\*|\?", uri_str, maxsplit=1)
chaiml-pony-d3-g46-pv2-l-7830-v2-uploader: Bucket 's3://guanaco-vllm-models/' created
chaiml-pony-d3-g46-pv2-l-7830-v2-uploader: uploading /dev/shm/model_output to s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-l-7830-v2/default
chaiml-pony-d3-g46-pv2-l-7830-v2-uploader: cp /dev/shm/model_output/.gitattributes s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-l-7830-v2/default/.gitattributes
chaiml-pony-d3-g46-pv2-l-7830-v2-uploader: cp /dev/shm/model_output/chat_template.jinja s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-l-7830-v2/default/chat_template.jinja
chaiml-pony-d3-g46-pv2-l-7830-v2-uploader: cp /dev/shm/model_output/config.json s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-l-7830-v2/default/config.json
chaiml-pony-d3-g46-pv2-l-7830-v2-uploader: cp /dev/shm/model_output/generation_config.json s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-l-7830-v2/default/generation_config.json
chaiml-pony-d3-g46-pv2-l-7830-v2-uploader: cp /dev/shm/model_output/model-00072-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-l-7830-v2/default/model-00072-of-00072.safetensors
chaiml-pony-d3-g46-pv2-l-7830-v2-uploader: cp /dev/shm/model_output/model-00071-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-l-7830-v2/default/model-00071-of-00072.safetensors
2026-03-27T17:57:44.699417+00:00 monitor updated for chaiml-pony-d3-g46-pv2-_13283_v2
2026-03-27T17:57:44.811664+00:00 monitor updated for chaiml-pony-d3-g46-pv2-l_7830_v2
chaiml-pony-d3-g46-pv2-l-7830-v2-uploader: cp /dev/shm/model_output/model-00003-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-l-7830-v2/default/model-00003-of-00072.safetensors
chaiml-pony-d3-g46-pv2-l-7830-v2-uploader: cp /dev/shm/model_output/model-00027-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-l-7830-v2/default/model-00027-of-00072.safetensors
chaiml-pony-d3-g46-pv2-l-7830-v2-uploader: cp /dev/shm/model_output/model-00030-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-l-7830-v2/default/model-00030-of-00072.safetensors
chaiml-pony-d3-g46-pv2-l-7830-v2-uploader: cp /dev/shm/model_output/model-00044-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-l-7830-v2/default/model-00044-of-00072.safetensors
chaiml-pony-d3-g46-pv2-l-7830-v2-uploader: cp /dev/shm/model_output/model-00047-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-l-7830-v2/default/model-00047-of-00072.safetensors
chaiml-pony-d3-g46-pv2-l-7830-v2-uploader: cp /dev/shm/model_output/model-00037-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-l-7830-v2/default/model-00037-of-00072.safetensors
chaiml-pony-d3-g46-pv2-l-7830-v2-uploader: cp /dev/shm/model_output/model-00045-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-l-7830-v2/default/model-00045-of-00072.safetensors
chaiml-pony-d3-g46-pv2-l-7830-v2-uploader: cp /dev/shm/model_output/model-00035-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-l-7830-v2/default/model-00035-of-00072.safetensors
chaiml-pony-d3-g46-pv2-l-7830-v2-uploader: cp /dev/shm/model_output/model-00031-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-l-7830-v2/default/model-00031-of-00072.safetensors
chaiml-pony-d3-g46-pv2-l-7830-v2-uploader: cp /dev/shm/model_output/model-00010-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-l-7830-v2/default/model-00010-of-00072.safetensors
chaiml-pony-d3-g46-pv2-l-7830-v2-uploader: cp /dev/shm/model_output/model-00023-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-l-7830-v2/default/model-00023-of-00072.safetensors
chaiml-pony-d3-g46-pv2-l-7830-v2-uploader: cp /dev/shm/model_output/model-00004-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-l-7830-v2/default/model-00004-of-00072.safetensors
chaiml-pony-d3-g46-pv2-l-7830-v2-uploader: cp /dev/shm/model_output/model-00039-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-l-7830-v2/default/model-00039-of-00072.safetensors
chaiml-pony-d3-g46-pv2-l-7830-v2-uploader: cp /dev/shm/model_output/model-00049-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-l-7830-v2/default/model-00049-of-00072.safetensors
chaiml-pony-d3-g46-pv2-l-7830-v2-uploader: cp /dev/shm/model_output/model-00043-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-l-7830-v2/default/model-00043-of-00072.safetensors
chaiml-pony-d3-g46-pv2-l-7830-v2-uploader: cp /dev/shm/model_output/model-00024-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-l-7830-v2/default/model-00024-of-00072.safetensors
chaiml-pony-d3-g46-pv2-l-7830-v2-uploader: cp /dev/shm/model_output/model-00021-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-l-7830-v2/default/model-00021-of-00072.safetensors
Job chaiml-pony-d3-g46-pv2-l-7830-v2-uploader completed after 396.82s with status: succeeded
Stopping job with name chaiml-pony-d3-g46-pv2-l-7830-v2-uploader
Pipeline stage VLLMUploader completed in 398.77s
run pipeline stage %s
Running pipeline stage VLLMUploaderAMD
Pipeline stage vllm_upload_amd skipped, reason=not amd cluster
Pipeline stage VLLMUploaderAMD completed in 0.46s
run pipeline stage %s
Running pipeline stage VLLMTemplater
Pipeline stage VLLMTemplater completed in 1.36s
run pipeline stage %s
Running pipeline stage VLLMDeployer
Creating inference service chaiml-pony-d3-g46-pv2-l-7830-v2
Waiting for inference service chaiml-pony-d3-g46-pv2-l-7830-v2 to be ready
2026-03-27T17:58:45.109808+00:00 monitor updated for chaiml-pony-d3-g46-pv2-_13283_v2
2026-03-27T17:58:45.228509+00:00 monitor updated for chaiml-pony-d3-g46-pv2-l_7830_v2
2026-03-27T17:59:45.524795+00:00 monitor updated for chaiml-pony-d3-g46-pv2-_13283_v2
2026-03-27T17:59:45.666743+00:00 monitor updated for chaiml-pony-d3-g46-pv2-l_7830_v2
Inference service chaiml-pony-d3-g46-pv2-13283-v2 ready after 264.34930086135864s
Pipeline stage VLLMDeployer completed in 266.21s
run pipeline stage %s
Running pipeline stage StressChecker
Received healthy response to inference request in 8.776982307434082s
2026-03-27T18:00:46.043552+00:00 monitor updated for chaiml-pony-d3-g46-pv2-l_7830_v2
2026-03-27T18:00:45.916113+00:00 monitor updated for chaiml-pony-d3-g46-pv2-_13283_v2
Received healthy response to inference request in 8.924762964248657s
Received healthy response to inference request in 8.77453351020813s
Received healthy response to inference request in 2.0980658531188965s
Received healthy response to inference request in 8.901828289031982s
Received healthy response to inference request in 2.121722459793091s
Received healthy response to inference request in 2.10976243019104s
Received healthy response to inference request in 9.045119524002075s
Received healthy response to inference request in 2.3254706859588623s
Received healthy response to inference request in 2.185009479522705s
Received healthy response to inference request in 2.1378841400146484s
Received healthy response to inference request in 2.1490557193756104s
Received healthy response to inference request in 2.1174306869506836s
Received healthy response to inference request in 2.0651040077209473s
2026-03-27T18:01:48.873548+00:00 monitor updated for chaiml-pony-d3-g46-pv2-l_7830_v2
2026-03-27T18:01:49.031063+00:00 monitor updated for chaiml-pony-d3-g46-pv2-_13283_v2
Received healthy response to inference request in 2.0809617042541504s
Received healthy response to inference request in 2.1797070503234863s
Received healthy response to inference request in 2.309887647628784s
Received healthy response to inference request in 2.0939862728118896s
Received healthy response to inference request in 2.0891551971435547s
Received healthy response to inference request in 2.1326324939727783s
Received healthy response to inference request in 2.10388445854187s
Received healthy response to inference request in 2.110459089279175s
Received healthy response to inference request in 2.101132392883301s
Received healthy response to inference request in 2.258213996887207s
Received healthy response to inference request in 2.270087242126465s
Received healthy response to inference request in 2.6971116065979004s
Received healthy response to inference request in 2.4207780361175537s
Received healthy response to inference request in 2.113703727722168s
Received healthy response to inference request in 4.383466482162476s
Received healthy response to inference request in 2.212412118911743s
30 requests
0 failed requests
5th percentile: 2.084648776054382
10th percentile: 2.093503165245056
20th percentile: 2.1033340454101563
30th percentile: 2.11273033618927
40th percentile: 2.1282684803009033
50th percentile: 2.1643813848495483
60th percentile: 2.2307328701019284
70th percentile: 2.3145625591278076
80th percentile: 3.0343825817108203
90th percentile: 8.789466905593873
95th percentile: 8.914442360401154
99th percentile: 9.010216121673585
mean time: 3.3763437191645305
Pipeline stage StressChecker completed in 118.35s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyTriggerPipeline
run_pipeline:run_in_cloud %s
starting trigger_guanaco_pipeline args=%s
triggered trigger_guanaco_pipeline args=%s
Pipeline stage OfflineFamilyFriendlyTriggerPipeline completed in 1.91s
Shutdown handler de-registered
chaiml-pony-d3-g46-pv2-_13283_v2 status is now deployed due to DeploymentManager action
2026-03-27T18:02:49.221757+00:00 monitor updated for chaiml-pony-d3-g46-pv2-l_7830_v2
2026-03-27T18:03:49.640630+00:00 monitor updated for chaiml-pony-d3-g46-pv2-l_7830_v2
Inference service chaiml-pony-d3-g46-pv2-l-7830-v2 ready after 344.5378098487854s
Pipeline stage VLLMDeployer completed in 346.21s
run pipeline stage %s
Running pipeline stage StressChecker
Received healthy response to inference request in 9.357629776000977s
Received healthy response to inference request in 2.357419013977051s
Received healthy response to inference request in 9.264024257659912s
Received healthy response to inference request in 9.215418577194214s
Received healthy response to inference request in 2.0730526447296143s
Received healthy response to inference request in 2.781193733215332s
2026-03-27T18:04:49.881796+00:00 monitor updated for chaiml-pony-d3-g46-pv2-l_7830_v2
Received healthy response to inference request in 9.690949201583862s
Received healthy response to inference request in 2.2391862869262695s
Received healthy response to inference request in 2.2755022048950195s
Received healthy response to inference request in 2.074350357055664s
Received healthy response to inference request in 2.054016351699829s
Received healthy response to inference request in 3.738399028778076s
Received healthy response to inference request in 2.0848371982574463s
Received healthy response to inference request in 2.231809616088867s
Received healthy response to inference request in 2.761073589324951s
Received healthy response to inference request in 2.278040885925293s
Received healthy response to inference request in 2.0754988193511963s
Received healthy response to inference request in 2.169954299926758s
Received healthy response to inference request in 2.529632091522217s
Received healthy response to inference request in 2.097946882247925s
Received healthy response to inference request in 2.1783249378204346s
Received healthy response to inference request in 2.210463523864746s
Received healthy response to inference request in 2.2445788383483887s
Received healthy response to inference request in 2.225224494934082s
Received healthy response to inference request in 2.1280908584594727s
Received healthy response to inference request in 2.6315762996673584s
Received healthy response to inference request in 2.0758676528930664s
Received healthy response to inference request in 2.273595094680786s
Received healthy response to inference request in 2.1003034114837646s
2026-03-27T18:05:50.103947+00:00 monitor updated for chaiml-pony-d3-g46-pv2-l_7830_v2
Received healthy response to inference request in 2.079530715942383s
30 requests
0 failed requests
5th percentile: 2.0736366152763366
10th percentile: 2.0753839731216432
20th percentile: 2.0837759017944335
30th percentile: 2.1197546243667604
40th percentile: 2.1976080894470216
50th percentile: 2.2354979515075684
60th percentile: 2.2743579387664794
70th percentile: 2.4090829372406
80th percentile: 2.7650976181030273
90th percentile: 9.220279145240784
95th percentile: 9.315507292747498
99th percentile: 9.594286568164826
mean time: 3.249916354815165
Pipeline stage StressChecker completed in 105.53s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyTriggerPipeline
run_pipeline:run_in_cloud %s
starting trigger_guanaco_pipeline args=%s
triggered trigger_guanaco_pipeline args=%s
Pipeline stage OfflineFamilyFriendlyTriggerPipeline completed in 1.29s
Shutdown handler de-registered
chaiml-pony-d3-g46-pv2-l_7830_v2 status is now deployed due to DeploymentManager action