developer_uid: zonemercy
submission_id: chaiml-pony-d3-g46-pv2-_64172_v9
model_name: chaiml-pony-d3-g46-pv2-_64172_v9
model_group: ChaiML/pony-d3-g46-pv2-l
status: torndown
timestamp: 2026-03-12T16:02:00+00:00
num_battles: 12204
num_wins: 6738
celo_rating: 1336.49
family_friendly_score: 0.0
family_friendly_standard_error: 0.0
submission_type: basic
model_repo: ChaiML/pony-d3-g46-pv2-lr5e6ep2r64b8
model_architecture: Glm4MoeForCausalLM
model_num_parameters: 24110003200.0
best_of: 8
max_input_tokens: 2048
max_output_tokens: 72
reward_model: default
display_name: chaiml-pony-d3-g46-pv2-_64172_v9
ineligible_reason: max_output_tokens!=64
is_internal_developer: True
language_model: ChaiML/pony-d3-g46-pv2-lr5e6ep2r64b8
model_size: 24B
ranking_group: single
us_pacific_date: 2026-03-09
win_ratio: 0.5521140609636185
generation_params: {'temperature': 1.0, 'top_p': 0.95, 'min_p': 0.0, 'top_k': 60, 'presence_penalty': 0.0, 'frequency_penalty': 0.0, 'stopping_words': ['<|assistant|>', '<|user|>', '<|im_end|>', '</s>', '####'], 'max_input_tokens': 2048, 'best_of': 8, 'max_output_tokens': 72}
formatter: {'memory_template': "[gMASK]<sop><|system|>\n{bot_name}'s persona: {memory}", 'prompt_template': '', 'bot_template': '<|assistant|>\n{message}', 'user_template': '<|user|>\n{message}', 'response_template': '<|assistant|>\n<think></think>\n{bot_name}:', 'truncate_by_message': True}
Resubmit model
Shutdown handler not registered because Python interpreter is not running in the main thread
run pipeline %s
run pipeline stage %s
Running pipeline stage VLLMUploader
Starting job with name chaiml-pony-d3-g46-pv2-64172-v9-uploader
Waiting for job on chaiml-pony-d3-g46-pv2-64172-v9-uploader to finish
chaiml-pony-d3-g46-pv2-64172-v9-uploader: Using quantization_mode: fp8
chaiml-pony-d3-g46-pv2-64172-v9-uploader: Checking if ChaiML/pony-d3-g46-pv2-lr5e6ep2r64b8-FP8 already exists in ChaiML
chaiml-pony-d3-g46-pv2-64172-v9-uploader: Model already exists. Downloading to /dev/shm/model_output...
chaiml-pony-d3-g46-pv2-64172-v9-uploader: Downloading snapshot of ChaiML/pony-d3-g46-pv2-lr5e6ep2r64b8-FP8...
chaiml-pony-d3-g46-pv2-64172-v9-uploader: Downloaded in 114.859s
chaiml-pony-d3-g46-pv2-64172-v9-uploader: Processed model ChaiML/pony-d3-g46-pv2-lr5e6ep2r64b8 in 118.413s
chaiml-pony-d3-g46-pv2-64172-v9-uploader: creating bucket guanaco-vllm-models
chaiml-pony-d3-g46-pv2-64172-v9-uploader: /usr/lib/python3/dist-packages/S3/BaseUtils.py:56: SyntaxWarning: invalid escape sequence '\.'
chaiml-pony-d3-g46-pv2-64172-v9-uploader:   RE_S3_DATESTRING = re.compile('\.[0-9]*(?:[Z\\-\\+]*?)')
chaiml-pony-d3-g46-pv2-64172-v9-uploader: /usr/lib/python3/dist-packages/S3/BaseUtils.py:57: SyntaxWarning: invalid escape sequence '\s'
chaiml-pony-d3-g46-pv2-64172-v9-uploader:   RE_XML_NAMESPACE = re.compile(b'^(<?[^>]+?>\s*|\s*)(<\w+) xmlns=[\'"](https?://[^\'"]+)[\'"]', re.MULTILINE)
chaiml-pony-d3-g46-pv2-64172-v9-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:240: SyntaxWarning: invalid escape sequence '\.'
chaiml-pony-d3-g46-pv2-64172-v9-uploader:   invalid = re.search("([^a-z0-9\.-])", bucket, re.UNICODE)
chaiml-pony-d3-g46-pv2-64172-v9-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:244: SyntaxWarning: invalid escape sequence '\.'
chaiml-pony-d3-g46-pv2-64172-v9-uploader:   invalid = re.search("([^A-Za-z0-9\._-])", bucket, re.UNICODE)
chaiml-pony-d3-g46-pv2-64172-v9-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:255: SyntaxWarning: invalid escape sequence '\.'
chaiml-pony-d3-g46-pv2-64172-v9-uploader:   if re.search("-\.", bucket, re.UNICODE):
chaiml-pony-d3-g46-pv2-64172-v9-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:257: SyntaxWarning: invalid escape sequence '\.'
chaiml-pony-d3-g46-pv2-64172-v9-uploader:   if re.search("\.\.", bucket, re.UNICODE):
chaiml-pony-d3-g46-pv2-64172-v9-uploader: /usr/lib/python3/dist-packages/S3/S3Uri.py:155: SyntaxWarning: invalid escape sequence '\w'
chaiml-pony-d3-g46-pv2-64172-v9-uploader:   _re = re.compile("^(\w+://)?(.*)", re.UNICODE)
chaiml-pony-d3-g46-pv2-64172-v9-uploader: /usr/lib/python3/dist-packages/S3/FileLists.py:480: SyntaxWarning: invalid escape sequence '\*'
chaiml-pony-d3-g46-pv2-64172-v9-uploader:   wildcard_split_result = re.split("\*|\?", uri_str, maxsplit=1)
chaiml-pony-d3-g46-pv2-64172-v9-uploader: Bucket 's3://guanaco-vllm-models/' created
chaiml-pony-d3-g46-pv2-64172-v9-uploader: uploading /dev/shm/model_output to s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-64172-v9/default
chaiml-pony-d3-g46-pv2-64172-v9-uploader: cp /dev/shm/model_output/.gitattributes s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-64172-v9/default/.gitattributes
chaiml-pony-d3-g46-pv2-64172-v9-uploader: cp /dev/shm/model_output/config.json s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-64172-v9/default/config.json
chaiml-pony-d3-g46-pv2-64172-v9-uploader: cp /dev/shm/model_output/generation_config.json s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-64172-v9/default/generation_config.json
chaiml-pony-d3-g46-pv2-64172-v9-uploader: cp /dev/shm/model_output/chat_template.jinja s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-64172-v9/default/chat_template.jinja
chaiml-pony-d3-g46-pv2-64172-v9-uploader: cp /dev/shm/model_output/special_tokens_map.json s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-64172-v9/default/special_tokens_map.json
chaiml-pony-d3-g46-pv2-64172-v9-uploader: cp /dev/shm/model_output/tokenizer_config.json s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-64172-v9/default/tokenizer_config.json
chaiml-pony-d3-g46-pv2-64172-v9-uploader: cp /dev/shm/model_output/recipe.yaml s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-64172-v9/default/recipe.yaml
chaiml-pony-d3-g46-pv2-64172-v9-uploader: cp /dev/shm/model_output/model.safetensors.index.json s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-64172-v9/default/model.safetensors.index.json
chaiml-pony-d3-g46-pv2-64172-v9-uploader: cp /dev/shm/model_output/tokenizer.json s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-64172-v9/default/tokenizer.json
chaiml-pony-d3-g46-pv2-64172-v9-uploader: cp /dev/shm/model_output/model-00072-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-64172-v9/default/model-00072-of-00072.safetensors
chaiml-pony-d3-g46-pv2-64172-v9-uploader: cp /dev/shm/model_output/model-00071-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-64172-v9/default/model-00071-of-00072.safetensors
chaiml-pony-d3-g46-pv2-64172-v9-uploader: cp /dev/shm/model_output/model-00038-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-64172-v9/default/model-00038-of-00072.safetensors
chaiml-pony-d3-g46-pv2-64172-v9-uploader: cp /dev/shm/model_output/model-00020-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-64172-v9/default/model-00020-of-00072.safetensors
chaiml-pony-d3-g46-pv2-64172-v9-uploader: cp /dev/shm/model_output/model-00050-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-64172-v9/default/model-00050-of-00072.safetensors
chaiml-pony-d3-g46-pv2-64172-v9-uploader: cp /dev/shm/model_output/model-00012-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-64172-v9/default/model-00012-of-00072.safetensors
chaiml-pony-d3-g46-pv2-64172-v9-uploader: cp /dev/shm/model_output/model-00054-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-64172-v9/default/model-00054-of-00072.safetensors
chaiml-pony-d3-g46-pv2-64172-v9-uploader: cp /dev/shm/model_output/model-00021-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-64172-v9/default/model-00021-of-00072.safetensors
chaiml-pony-d3-g46-pv2-64172-v9-uploader: cp /dev/shm/model_output/model-00042-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-64172-v9/default/model-00042-of-00072.safetensors
chaiml-pony-d3-g46-pv2-64172-v9-uploader: cp /dev/shm/model_output/model-00034-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-64172-v9/default/model-00034-of-00072.safetensors
chaiml-pony-d3-g46-pv2-64172-v9-uploader: cp /dev/shm/model_output/model-00041-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-64172-v9/default/model-00041-of-00072.safetensors
chaiml-pony-d3-g46-pv2-64172-v9-uploader: cp /dev/shm/model_output/model-00058-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-64172-v9/default/model-00058-of-00072.safetensors
chaiml-pony-d3-g46-pv2-64172-v9-uploader: cp /dev/shm/model_output/model-00057-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-64172-v9/default/model-00057-of-00072.safetensors
chaiml-pony-d3-g46-pv2-64172-v9-uploader: cp /dev/shm/model_output/model-00004-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-64172-v9/default/model-00004-of-00072.safetensors
chaiml-pony-d3-g46-pv2-64172-v9-uploader: cp /dev/shm/model_output/model-00026-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-64172-v9/default/model-00026-of-00072.safetensors
chaiml-pony-d3-g46-pv2-64172-v9-uploader: cp /dev/shm/model_output/model-00065-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-64172-v9/default/model-00065-of-00072.safetensors
chaiml-pony-d3-g46-pv2-64172-v9-uploader: cp /dev/shm/model_output/model-00056-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-64172-v9/default/model-00056-of-00072.safetensors
chaiml-pony-d3-g46-pv2-64172-v9-uploader: cp /dev/shm/model_output/model-00003-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-64172-v9/default/model-00003-of-00072.safetensors
chaiml-pony-d3-g46-pv2-64172-v9-uploader: cp /dev/shm/model_output/model-00064-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-64172-v9/default/model-00064-of-00072.safetensors
chaiml-pony-d3-g46-pv2-64172-v9-uploader: cp /dev/shm/model_output/model-00068-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-64172-v9/default/model-00068-of-00072.safetensors
chaiml-pony-d3-g46-pv2-64172-v9-uploader: cp /dev/shm/model_output/model-00014-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-64172-v9/default/model-00014-of-00072.safetensors
chaiml-pony-d3-g46-pv2-64172-v9-uploader: cp /dev/shm/model_output/model-00025-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-64172-v9/default/model-00025-of-00072.safetensors
chaiml-pony-d3-g46-pv2-64172-v9-uploader: cp /dev/shm/model_output/model-00052-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-64172-v9/default/model-00052-of-00072.safetensors
chaiml-pony-d3-g46-pv2-64172-v9-uploader: cp /dev/shm/model_output/model-00016-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-64172-v9/default/model-00016-of-00072.safetensors
chaiml-pony-d3-g46-pv2-64172-v9-uploader: cp /dev/shm/model_output/model-00047-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-64172-v9/default/model-00047-of-00072.safetensors
chaiml-pony-d3-g46-pv2-64172-v9-uploader: cp /dev/shm/model_output/model-00009-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-64172-v9/default/model-00009-of-00072.safetensors
chaiml-pony-d3-g46-pv2-64172-v9-uploader: cp /dev/shm/model_output/model-00059-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-64172-v9/default/model-00059-of-00072.safetensors
chaiml-pony-d3-g46-pv2-64172-v9-uploader: cp /dev/shm/model_output/model-00007-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-64172-v9/default/model-00007-of-00072.safetensors
chaiml-pony-d3-g46-pv2-64172-v9-uploader: cp /dev/shm/model_output/model-00051-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-64172-v9/default/model-00051-of-00072.safetensors
chaiml-pony-d3-g46-pv2-64172-v9-uploader: cp /dev/shm/model_output/model-00063-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-64172-v9/default/model-00063-of-00072.safetensors
chaiml-pony-d3-g46-pv2-64172-v9-uploader: cp /dev/shm/model_output/model-00033-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-64172-v9/default/model-00033-of-00072.safetensors
chaiml-pony-d3-g46-pv2-64172-v9-uploader: cp /dev/shm/model_output/model-00046-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-64172-v9/default/model-00046-of-00072.safetensors
chaiml-pony-d3-g46-pv2-64172-v9-uploader: cp /dev/shm/model_output/model-00010-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-64172-v9/default/model-00010-of-00072.safetensors
chaiml-pony-d3-g46-pv2-64172-v9-uploader: cp /dev/shm/model_output/model-00055-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-64172-v9/default/model-00055-of-00072.safetensors
chaiml-pony-d3-g46-pv2-64172-v9-uploader: cp /dev/shm/model_output/model-00029-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-64172-v9/default/model-00029-of-00072.safetensors
chaiml-pony-d3-g46-pv2-64172-v9-uploader: cp /dev/shm/model_output/model-00044-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-64172-v9/default/model-00044-of-00072.safetensors
chaiml-pony-d3-g46-pv2-64172-v9-uploader: cp /dev/shm/model_output/model-00032-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-64172-v9/default/model-00032-of-00072.safetensors
chaiml-pony-d3-g46-pv2-64172-v9-uploader: cp /dev/shm/model_output/model-00022-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-64172-v9/default/model-00022-of-00072.safetensors
chaiml-pony-d3-g46-pv2-64172-v9-uploader: cp /dev/shm/model_output/model-00019-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-64172-v9/default/model-00019-of-00072.safetensors
chaiml-pony-d3-g46-pv2-64172-v9-uploader: cp /dev/shm/model_output/model-00001-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-64172-v9/default/model-00001-of-00072.safetensors
chaiml-pony-d3-g46-pv2-64172-v9-uploader: cp /dev/shm/model_output/model-00015-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-64172-v9/default/model-00015-of-00072.safetensors
chaiml-pony-d3-g46-pv2-64172-v9-uploader: cp /dev/shm/model_output/model-00070-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-64172-v9/default/model-00070-of-00072.safetensors
chaiml-pony-d3-g46-pv2-64172-v9-uploader: cp /dev/shm/model_output/model-00067-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-64172-v9/default/model-00067-of-00072.safetensors
chaiml-pony-d3-g46-pv2-64172-v9-uploader: cp /dev/shm/model_output/model-00036-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-64172-v9/default/model-00036-of-00072.safetensors
chaiml-pony-d3-g46-pv2-64172-v9-uploader: cp /dev/shm/model_output/model-00060-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-64172-v9/default/model-00060-of-00072.safetensors
chaiml-pony-d3-g46-pv2-64172-v9-uploader: cp /dev/shm/model_output/model-00045-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-64172-v9/default/model-00045-of-00072.safetensors
chaiml-pony-d3-g46-pv2-64172-v9-uploader: cp /dev/shm/model_output/model-00017-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-64172-v9/default/model-00017-of-00072.safetensors
chaiml-pony-d3-g46-pv2-64172-v9-uploader: cp /dev/shm/model_output/model-00048-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-64172-v9/default/model-00048-of-00072.safetensors
chaiml-pony-d3-g46-pv2-64172-v9-uploader: cp /dev/shm/model_output/model-00023-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-64172-v9/default/model-00023-of-00072.safetensors
chaiml-pony-d3-g46-pv2-64172-v9-uploader: cp /dev/shm/model_output/model-00031-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-64172-v9/default/model-00031-of-00072.safetensors
chaiml-pony-d3-g46-pv2-64172-v9-uploader: cp /dev/shm/model_output/model-00018-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-64172-v9/default/model-00018-of-00072.safetensors
chaiml-pony-d3-g46-pv2-64172-v9-uploader: cp /dev/shm/model_output/model-00043-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-64172-v9/default/model-00043-of-00072.safetensors
chaiml-pony-d3-g46-pv2-64172-v9-uploader: cp /dev/shm/model_output/model-00061-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-64172-v9/default/model-00061-of-00072.safetensors
chaiml-pony-d3-g46-pv2-64172-v9-uploader: cp /dev/shm/model_output/model-00062-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-64172-v9/default/model-00062-of-00072.safetensors
chaiml-pony-d3-g46-pv2-64172-v9-uploader: cp /dev/shm/model_output/model-00006-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-64172-v9/default/model-00006-of-00072.safetensors
chaiml-pony-d3-g46-pv2-64172-v9-uploader: cp /dev/shm/model_output/model-00002-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-64172-v9/default/model-00002-of-00072.safetensors
chaiml-pony-d3-g46-pv2-64172-v9-uploader: cp /dev/shm/model_output/model-00005-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-64172-v9/default/model-00005-of-00072.safetensors
chaiml-pony-d3-g46-pv2-64172-v9-uploader: cp /dev/shm/model_output/model-00024-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-64172-v9/default/model-00024-of-00072.safetensors
chaiml-pony-d3-g46-pv2-64172-v9-uploader: cp /dev/shm/model_output/model-00053-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-64172-v9/default/model-00053-of-00072.safetensors
chaiml-pony-d3-g46-pv2-64172-v9-uploader: cp /dev/shm/model_output/model-00040-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-64172-v9/default/model-00040-of-00072.safetensors
chaiml-pony-d3-g46-pv2-64172-v9-uploader: cp /dev/shm/model_output/model-00069-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-64172-v9/default/model-00069-of-00072.safetensors
chaiml-pony-d3-g46-pv2-64172-v9-uploader: cp /dev/shm/model_output/model-00037-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-64172-v9/default/model-00037-of-00072.safetensors
chaiml-pony-d3-g46-pv2-64172-v9-uploader: cp /dev/shm/model_output/model-00028-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-64172-v9/default/model-00028-of-00072.safetensors
chaiml-pony-d3-g46-pv2-64172-v9-uploader: cp /dev/shm/model_output/model-00035-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-64172-v9/default/model-00035-of-00072.safetensors
chaiml-pony-d3-g46-pv2-64172-v9-uploader: cp /dev/shm/model_output/model-00039-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-64172-v9/default/model-00039-of-00072.safetensors
chaiml-pony-d3-g46-pv2-64172-v9-uploader: cp /dev/shm/model_output/model-00008-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-64172-v9/default/model-00008-of-00072.safetensors
chaiml-pony-d3-g46-pv2-64172-v9-uploader: cp /dev/shm/model_output/model-00011-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-64172-v9/default/model-00011-of-00072.safetensors
chaiml-pony-d3-g46-pv2-64172-v9-uploader: cp /dev/shm/model_output/model-00066-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-64172-v9/default/model-00066-of-00072.safetensors
chaiml-pony-d3-g46-pv2-64172-v9-uploader: cp /dev/shm/model_output/model-00030-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-64172-v9/default/model-00030-of-00072.safetensors
chaiml-pony-d3-g46-pv2-64172-v9-uploader: cp /dev/shm/model_output/model-00013-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-64172-v9/default/model-00013-of-00072.safetensors
chaiml-pony-d3-g46-pv2-64172-v9-uploader: cp /dev/shm/model_output/model-00049-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-64172-v9/default/model-00049-of-00072.safetensors
chaiml-pony-d3-g46-pv2-64172-v9-uploader: cp /dev/shm/model_output/model-00027-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-64172-v9/default/model-00027-of-00072.safetensors
Job chaiml-pony-d3-g46-pv2-64172-v9-uploader completed after 261.36s with status: succeeded
Stopping job with name chaiml-pony-d3-g46-pv2-64172-v9-uploader
Pipeline stage VLLMUploader completed in 261.89s
run pipeline stage %s
Running pipeline stage VLLMTemplater
Pipeline stage VLLMTemplater completed in 1.38s
run pipeline stage %s
Running pipeline stage VLLMDeployer
Creating inference service chaiml-pony-d3-g46-pv2-64172-v9
Waiting for inference service chaiml-pony-d3-g46-pv2-64172-v9 to be ready
Inference service chaiml-pony-d3-g46-pv2-64172-v9 ready after 561.2131116390228s
Pipeline stage VLLMDeployer completed in 562.28s
run pipeline stage %s
Running pipeline stage StressChecker
Received healthy response to inference request in 1.967954158782959s
Received healthy response to inference request in 1.9318437576293945s
Received healthy response to inference request in 1.9220592975616455s
Received healthy response to inference request in 1.878354549407959s
Received healthy response to inference request in 1.888808012008667s
Received healthy response to inference request in 1.9465692043304443s
Received healthy response to inference request in 1.8773152828216553s
Received healthy response to inference request in 1.8645670413970947s
Received healthy response to inference request in 2.5996742248535156s
Received healthy response to inference request in 1.9356074333190918s
Received healthy response to inference request in 1.8802382946014404s
Received healthy response to inference request in 1.8476309776306152s
Received healthy response to inference request in 1.9370193481445312s
Received healthy response to inference request in 1.8682889938354492s
Received healthy response to inference request in 1.9130189418792725s
Received healthy response to inference request in 1.872483253479004s
Received healthy response to inference request in 2.0588154792785645s
Received healthy response to inference request in 1.9239962100982666s
Received healthy response to inference request in 1.9310665130615234s
Received healthy response to inference request in 1.868997573852539s
Received healthy response to inference request in 1.9425427913665771s
Received healthy response to inference request in 1.9359321594238281s
Received healthy response to inference request in 1.9606010913848877s
Received healthy response to inference request in 1.90079927444458s
Received healthy response to inference request in 1.866685152053833s
Received healthy response to inference request in 1.8762083053588867s
Received healthy response to inference request in 1.8789701461791992s
Received healthy response to inference request in 1.8713228702545166s
Received healthy response to inference request in 2.0057880878448486s
Received healthy response to inference request in 1.8929498195648193s
30 requests
0 failed requests
5th percentile: 1.865520191192627
10th percentile: 1.8681286096572876
20th percentile: 1.8722511768341064
30th percentile: 1.8780427694320678
40th percentile: 1.8853801250457765
50th percentile: 1.9069091081619263
60th percentile: 1.9268243312835693
70th percentile: 1.9357048511505126
80th percentile: 1.9433480739593505
90th percentile: 1.971737551689148
95th percentile: 2.034953153133392
99th percentile: 2.4428251886367804
mean time: 1.9348702748616537
Pipeline stage StressChecker completed in 65.06s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyTriggerPipeline
run_pipeline:run_in_cloud %s
starting trigger_guanaco_pipeline args=%s
triggered trigger_guanaco_pipeline args=%s
Pipeline stage OfflineFamilyFriendlyTriggerPipeline completed in 1.09s
Shutdown handler de-registered
chaiml-pony-d3-g46-pv2-_64172_v9 status is now deployed due to DeploymentManager action
chaiml-pony-d3-g46-pv2-_64172_v9 status is now inactive due to auto deactivation removed underperforming models
chaiml-pony-d3-g46-pv2-_64172_v9 status is now torndown due to DeploymentManager action