developer_uid: zonemercy
submission_id: chaiml-pony-d3-g46-pv2-_41910_v4
model_name: chaiml-pony-d3-g46-pv2-_41910_v4
model_group: ChaiML/pony-d3-g46-pv2-l
status: torndown
timestamp: 2026-03-09T16:31:53+00:00
num_battles: 10960
num_wins: 6300
celo_rating: 1351.64
family_friendly_score: 0.0
family_friendly_standard_error: 0.0
submission_type: basic
model_repo: ChaiML/pony-d3-g46-pv2-lr5e6ep1r64b8
model_architecture: Glm4MoeForCausalLM
model_num_parameters: 24110003200.0
best_of: 8
max_input_tokens: 2048
max_output_tokens: 80
reward_model: default
display_name: chaiml-pony-d3-g46-pv2-_41910_v4
ineligible_reason: max_output_tokens!=64
is_internal_developer: True
language_model: ChaiML/pony-d3-g46-pv2-lr5e6ep1r64b8
model_size: 24B
ranking_group: single
us_pacific_date: 2026-03-06
win_ratio: 0.5748175182481752
generation_params: {'temperature': 1.0, 'top_p': 0.95, 'min_p': 0.0, 'top_k': 60, 'presence_penalty': 0.0, 'frequency_penalty': 0.0, 'stopping_words': ['<|assistant|>', '<|user|>', '<|im_end|>', '</s>', '####'], 'max_input_tokens': 2048, 'best_of': 8, 'max_output_tokens': 80}
formatter: {'memory_template': "[gMASK]<sop><|system|>\n{bot_name}'s persona: {memory}", 'prompt_template': '', 'bot_template': '<|assistant|>\n{message}', 'user_template': '<|user|>\n{message}', 'response_template': '<|assistant|>\n<think></think>\n{bot_name}:', 'truncate_by_message': True}
Resubmit model
Shutdown handler not registered because Python interpreter is not running in the main thread
run pipeline %s
run pipeline stage %s
Running pipeline stage VLLMUploader
Starting job with name chaiml-pony-d3-g46-pv2-41910-v4-uploader
Waiting for job on chaiml-pony-d3-g46-pv2-41910-v4-uploader to finish
chaiml-pony-d3-g46-pv2-41910-v4-uploader: Using quantization_mode: fp8
chaiml-pony-d3-g46-pv2-41910-v4-uploader: Checking if ChaiML/pony-d3-g46-pv2-lr5e6ep1r64b8-FP8 already exists in ChaiML
chaiml-pony-d3-g46-pv2-41910-v4-uploader: Model already exists. Downloading to /dev/shm/model_output...
chaiml-pony-d3-g46-pv2-41910-v4-uploader: Downloading snapshot of ChaiML/pony-d3-g46-pv2-lr5e6ep1r64b8-FP8...
chaiml-pony-d3-g46-pv2-41910-v4-uploader: Downloaded in 120.083s
chaiml-pony-d3-g46-pv2-41910-v4-uploader: Processed model ChaiML/pony-d3-g46-pv2-lr5e6ep1r64b8 in 123.664s
chaiml-pony-d3-g46-pv2-41910-v4-uploader: creating bucket guanaco-vllm-models
chaiml-pony-d3-g46-pv2-41910-v4-uploader: /usr/lib/python3/dist-packages/S3/BaseUtils.py:56: SyntaxWarning: invalid escape sequence '\.'
chaiml-pony-d3-g46-pv2-41910-v4-uploader:   RE_S3_DATESTRING = re.compile('\.[0-9]*(?:[Z\\-\\+]*?)')
chaiml-pony-d3-g46-pv2-41910-v4-uploader: /usr/lib/python3/dist-packages/S3/BaseUtils.py:57: SyntaxWarning: invalid escape sequence '\s'
chaiml-pony-d3-g46-pv2-41910-v4-uploader:   RE_XML_NAMESPACE = re.compile(b'^(<?[^>]+?>\s*|\s*)(<\w+) xmlns=[\'"](https?://[^\'"]+)[\'"]', re.MULTILINE)
chaiml-pony-d3-g46-pv2-41910-v4-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:240: SyntaxWarning: invalid escape sequence '\.'
chaiml-pony-d3-g46-pv2-41910-v4-uploader:   invalid = re.search("([^a-z0-9\.-])", bucket, re.UNICODE)
chaiml-pony-d3-g46-pv2-41910-v4-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:244: SyntaxWarning: invalid escape sequence '\.'
chaiml-pony-d3-g46-pv2-41910-v4-uploader:   invalid = re.search("([^A-Za-z0-9\._-])", bucket, re.UNICODE)
chaiml-pony-d3-g46-pv2-41910-v4-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:255: SyntaxWarning: invalid escape sequence '\.'
chaiml-pony-d3-g46-pv2-41910-v4-uploader:   if re.search("-\.", bucket, re.UNICODE):
chaiml-pony-d3-g46-pv2-41910-v4-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:257: SyntaxWarning: invalid escape sequence '\.'
chaiml-pony-d3-g46-pv2-41910-v4-uploader:   if re.search("\.\.", bucket, re.UNICODE):
chaiml-pony-d3-g46-pv2-41910-v4-uploader: /usr/lib/python3/dist-packages/S3/S3Uri.py:155: SyntaxWarning: invalid escape sequence '\w'
chaiml-pony-d3-g46-pv2-41910-v4-uploader:   _re = re.compile("^(\w+://)?(.*)", re.UNICODE)
chaiml-pony-d3-g46-pv2-41910-v4-uploader: /usr/lib/python3/dist-packages/S3/FileLists.py:480: SyntaxWarning: invalid escape sequence '\*'
chaiml-pony-d3-g46-pv2-41910-v4-uploader:   wildcard_split_result = re.split("\*|\?", uri_str, maxsplit=1)
chaiml-pony-d3-g46-pv2-41910-v4-uploader: Bucket 's3://guanaco-vllm-models/' created
chaiml-pony-d3-g46-pv2-41910-v4-uploader: uploading /dev/shm/model_output to s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-41910-v4/default
chaiml-pony-d3-g46-pv2-41910-v4-uploader: cp /dev/shm/model_output/config.json s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-41910-v4/default/config.json
chaiml-pony-d3-g46-pv2-41910-v4-uploader: cp /dev/shm/model_output/generation_config.json s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-41910-v4/default/generation_config.json
chaiml-pony-d3-g46-pv2-41910-v4-uploader: cp /dev/shm/model_output/chat_template.jinja s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-41910-v4/default/chat_template.jinja
chaiml-pony-d3-g46-pv2-41910-v4-uploader: cp /dev/shm/model_output/.gitattributes s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-41910-v4/default/.gitattributes
chaiml-pony-d3-g46-pv2-41910-v4-uploader: cp /dev/shm/model_output/recipe.yaml s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-41910-v4/default/recipe.yaml
chaiml-pony-d3-g46-pv2-41910-v4-uploader: cp /dev/shm/model_output/special_tokens_map.json s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-41910-v4/default/special_tokens_map.json
chaiml-pony-d3-g46-pv2-41910-v4-uploader: cp /dev/shm/model_output/tokenizer_config.json s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-41910-v4/default/tokenizer_config.json
chaiml-pony-d3-g46-pv2-41910-v4-uploader: cp /dev/shm/model_output/model.safetensors.index.json s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-41910-v4/default/model.safetensors.index.json
chaiml-pony-d3-g46-pv2-41910-v4-uploader: cp /dev/shm/model_output/tokenizer.json s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-41910-v4/default/tokenizer.json
chaiml-pony-d3-g46-pv2-41910-v4-uploader: cp /dev/shm/model_output/model-00072-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-41910-v4/default/model-00072-of-00072.safetensors
chaiml-pony-d3-g46-pv2-41910-v4-uploader: cp /dev/shm/model_output/model-00071-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-41910-v4/default/model-00071-of-00072.safetensors
chaiml-pony-d3-g46-pv2-41910-v4-uploader: cp /dev/shm/model_output/model-00057-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-41910-v4/default/model-00057-of-00072.safetensors
chaiml-pony-d3-g46-pv2-41910-v4-uploader: cp /dev/shm/model_output/model-00009-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-41910-v4/default/model-00009-of-00072.safetensors
chaiml-pony-d3-g46-pv2-41910-v4-uploader: cp /dev/shm/model_output/model-00030-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-41910-v4/default/model-00030-of-00072.safetensors
chaiml-pony-d3-g46-pv2-41910-v4-uploader: cp /dev/shm/model_output/model-00035-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-41910-v4/default/model-00035-of-00072.safetensors
chaiml-pony-d3-g46-pv2-41910-v4-uploader: cp /dev/shm/model_output/model-00036-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-41910-v4/default/model-00036-of-00072.safetensors
chaiml-pony-d3-g46-pv2-41910-v4-uploader: cp /dev/shm/model_output/model-00053-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-41910-v4/default/model-00053-of-00072.safetensors
chaiml-pony-d3-g46-pv2-41910-v4-uploader: cp /dev/shm/model_output/model-00032-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-41910-v4/default/model-00032-of-00072.safetensors
chaiml-pony-d3-g46-pv2-41910-v4-uploader: cp /dev/shm/model_output/model-00062-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-41910-v4/default/model-00062-of-00072.safetensors
chaiml-pony-d3-g46-pv2-41910-v4-uploader: cp /dev/shm/model_output/model-00015-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-41910-v4/default/model-00015-of-00072.safetensors
chaiml-pony-d3-g46-pv2-41910-v4-uploader: cp /dev/shm/model_output/model-00028-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-41910-v4/default/model-00028-of-00072.safetensors
chaiml-pony-d3-g46-pv2-41910-v4-uploader: cp /dev/shm/model_output/model-00056-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-41910-v4/default/model-00056-of-00072.safetensors
chaiml-pony-d3-g46-pv2-41910-v4-uploader: cp /dev/shm/model_output/model-00070-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-41910-v4/default/model-00070-of-00072.safetensors
chaiml-pony-d3-g46-pv2-41910-v4-uploader: cp /dev/shm/model_output/model-00039-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-41910-v4/default/model-00039-of-00072.safetensors
chaiml-pony-d3-g46-pv2-41910-v4-uploader: cp /dev/shm/model_output/model-00050-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-41910-v4/default/model-00050-of-00072.safetensors
chaiml-pony-d3-g46-pv2-41910-v4-uploader: cp /dev/shm/model_output/model-00059-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-41910-v4/default/model-00059-of-00072.safetensors
chaiml-pony-d3-g46-pv2-41910-v4-uploader: cp /dev/shm/model_output/model-00047-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-41910-v4/default/model-00047-of-00072.safetensors
chaiml-pony-d3-g46-pv2-41910-v4-uploader: cp /dev/shm/model_output/model-00069-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-41910-v4/default/model-00069-of-00072.safetensors
chaiml-pony-d3-g46-pv2-41910-v4-uploader: cp /dev/shm/model_output/model-00054-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-41910-v4/default/model-00054-of-00072.safetensors
chaiml-pony-d3-g46-pv2-41910-v4-uploader: cp /dev/shm/model_output/model-00025-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-41910-v4/default/model-00025-of-00072.safetensors
chaiml-pony-d3-g46-pv2-41910-v4-uploader: cp /dev/shm/model_output/model-00017-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-41910-v4/default/model-00017-of-00072.safetensors
chaiml-pony-d3-g46-pv2-41910-v4-uploader: cp /dev/shm/model_output/model-00012-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-41910-v4/default/model-00012-of-00072.safetensors
chaiml-pony-d3-g46-pv2-41910-v4-uploader: cp /dev/shm/model_output/model-00023-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-41910-v4/default/model-00023-of-00072.safetensors
chaiml-pony-d3-g46-pv2-41910-v4-uploader: cp /dev/shm/model_output/model-00014-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-41910-v4/default/model-00014-of-00072.safetensors
chaiml-pony-d3-g46-pv2-41910-v4-uploader: cp /dev/shm/model_output/model-00034-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-41910-v4/default/model-00034-of-00072.safetensors
chaiml-pony-d3-g46-pv2-41910-v4-uploader: cp /dev/shm/model_output/model-00021-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-41910-v4/default/model-00021-of-00072.safetensors
chaiml-pony-d3-g46-pv2-41910-v4-uploader: cp /dev/shm/model_output/model-00067-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-41910-v4/default/model-00067-of-00072.safetensors
chaiml-pony-d3-g46-pv2-41910-v4-uploader: cp /dev/shm/model_output/model-00068-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-41910-v4/default/model-00068-of-00072.safetensors
chaiml-pony-d3-g46-pv2-41910-v4-uploader: cp /dev/shm/model_output/model-00013-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-41910-v4/default/model-00013-of-00072.safetensors
chaiml-pony-d3-g46-pv2-41910-v4-uploader: cp /dev/shm/model_output/model-00024-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-41910-v4/default/model-00024-of-00072.safetensors
chaiml-pony-d3-g46-pv2-41910-v4-uploader: cp /dev/shm/model_output/model-00016-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-41910-v4/default/model-00016-of-00072.safetensors
chaiml-pony-d3-g46-pv2-41910-v4-uploader: cp /dev/shm/model_output/model-00022-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-41910-v4/default/model-00022-of-00072.safetensors
chaiml-pony-d3-g46-pv2-41910-v4-uploader: cp /dev/shm/model_output/model-00066-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-41910-v4/default/model-00066-of-00072.safetensors
chaiml-pony-d3-g46-pv2-41910-v4-uploader: cp /dev/shm/model_output/model-00058-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-41910-v4/default/model-00058-of-00072.safetensors
chaiml-pony-d3-g46-pv2-41910-v4-uploader: cp /dev/shm/model_output/model-00065-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-41910-v4/default/model-00065-of-00072.safetensors
chaiml-pony-d3-g46-pv2-41910-v4-uploader: cp /dev/shm/model_output/model-00010-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-41910-v4/default/model-00010-of-00072.safetensors
chaiml-pony-d3-g46-pv2-41910-v4-uploader: cp /dev/shm/model_output/model-00004-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-41910-v4/default/model-00004-of-00072.safetensors
chaiml-pony-d3-g46-pv2-41910-v4-uploader: cp /dev/shm/model_output/model-00018-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-41910-v4/default/model-00018-of-00072.safetensors
chaiml-pony-d3-g46-pv2-41910-v4-uploader: cp /dev/shm/model_output/model-00048-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-41910-v4/default/model-00048-of-00072.safetensors
chaiml-pony-d3-g46-pv2-41910-v4-uploader: cp /dev/shm/model_output/model-00007-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-41910-v4/default/model-00007-of-00072.safetensors
chaiml-pony-d3-g46-pv2-41910-v4-uploader: cp /dev/shm/model_output/model-00063-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-41910-v4/default/model-00063-of-00072.safetensors
chaiml-pony-d3-g46-pv2-41910-v4-uploader: cp /dev/shm/model_output/model-00011-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-41910-v4/default/model-00011-of-00072.safetensors
chaiml-pony-d3-g46-pv2-41910-v4-uploader: cp /dev/shm/model_output/model-00038-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-41910-v4/default/model-00038-of-00072.safetensors
chaiml-pony-d3-g46-pv2-41910-v4-uploader: cp /dev/shm/model_output/model-00003-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-41910-v4/default/model-00003-of-00072.safetensors
chaiml-pony-d3-g46-pv2-41910-v4-uploader: cp /dev/shm/model_output/model-00064-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-41910-v4/default/model-00064-of-00072.safetensors
chaiml-pony-d3-g46-pv2-41910-v4-uploader: cp /dev/shm/model_output/model-00060-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-41910-v4/default/model-00060-of-00072.safetensors
chaiml-pony-d3-g46-pv2-41910-v4-uploader: cp /dev/shm/model_output/model-00055-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-41910-v4/default/model-00055-of-00072.safetensors
chaiml-pony-d3-g46-pv2-41910-v4-uploader: cp /dev/shm/model_output/model-00001-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-41910-v4/default/model-00001-of-00072.safetensors
chaiml-pony-d3-g46-pv2-41910-v4-uploader: cp /dev/shm/model_output/model-00037-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-41910-v4/default/model-00037-of-00072.safetensors
chaiml-pony-d3-g46-pv2-41910-v4-uploader: cp /dev/shm/model_output/model-00031-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-41910-v4/default/model-00031-of-00072.safetensors
chaiml-pony-d3-g46-pv2-41910-v4-uploader: cp /dev/shm/model_output/model-00041-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-41910-v4/default/model-00041-of-00072.safetensors
chaiml-pony-d3-g46-pv2-41910-v4-uploader: cp /dev/shm/model_output/model-00045-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-41910-v4/default/model-00045-of-00072.safetensors
chaiml-pony-d3-g46-pv2-41910-v4-uploader: cp /dev/shm/model_output/model-00046-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-41910-v4/default/model-00046-of-00072.safetensors
chaiml-pony-d3-g46-pv2-41910-v4-uploader: cp /dev/shm/model_output/model-00040-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-41910-v4/default/model-00040-of-00072.safetensors
chaiml-pony-d3-g46-pv2-41910-v4-uploader: cp /dev/shm/model_output/model-00033-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-41910-v4/default/model-00033-of-00072.safetensors
chaiml-pony-d3-g46-pv2-41910-v4-uploader: cp /dev/shm/model_output/model-00006-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-41910-v4/default/model-00006-of-00072.safetensors
chaiml-pony-d3-g46-pv2-41910-v4-uploader: cp /dev/shm/model_output/model-00061-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-41910-v4/default/model-00061-of-00072.safetensors
chaiml-pony-d3-g46-pv2-41910-v4-uploader: cp /dev/shm/model_output/model-00026-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-41910-v4/default/model-00026-of-00072.safetensors
chaiml-pony-d3-g46-pv2-41910-v4-uploader: cp /dev/shm/model_output/model-00051-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-41910-v4/default/model-00051-of-00072.safetensors
chaiml-pony-d3-g46-pv2-41910-v4-uploader: cp /dev/shm/model_output/model-00002-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-41910-v4/default/model-00002-of-00072.safetensors
chaiml-pony-d3-g46-pv2-41910-v4-uploader: cp /dev/shm/model_output/model-00008-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-41910-v4/default/model-00008-of-00072.safetensors
chaiml-pony-d3-g46-pv2-41910-v4-uploader: cp /dev/shm/model_output/model-00005-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-41910-v4/default/model-00005-of-00072.safetensors
chaiml-pony-d3-g46-pv2-41910-v4-uploader: cp /dev/shm/model_output/model-00027-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-41910-v4/default/model-00027-of-00072.safetensors
chaiml-pony-d3-g46-pv2-41910-v4-uploader: cp /dev/shm/model_output/model-00029-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-41910-v4/default/model-00029-of-00072.safetensors
chaiml-pony-d3-g46-pv2-41910-v4-uploader: cp /dev/shm/model_output/model-00052-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-41910-v4/default/model-00052-of-00072.safetensors
chaiml-pony-d3-g46-pv2-41910-v4-uploader: cp /dev/shm/model_output/model-00044-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-41910-v4/default/model-00044-of-00072.safetensors
chaiml-pony-d3-g46-pv2-41910-v4-uploader: cp /dev/shm/model_output/model-00042-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-41910-v4/default/model-00042-of-00072.safetensors
chaiml-pony-d3-g46-pv2-41910-v4-uploader: cp /dev/shm/model_output/model-00043-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-41910-v4/default/model-00043-of-00072.safetensors
chaiml-pony-d3-g46-pv2-41910-v4-uploader: cp /dev/shm/model_output/model-00020-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-41910-v4/default/model-00020-of-00072.safetensors
Job chaiml-pony-d3-g46-pv2-41910-v4-uploader completed after 269.62s with status: succeeded
Stopping job with name chaiml-pony-d3-g46-pv2-41910-v4-uploader
Pipeline stage VLLMUploader completed in 270.79s
run pipeline stage %s
Running pipeline stage VLLMTemplater
Pipeline stage VLLMTemplater completed in 1.28s
run pipeline stage %s
Running pipeline stage VLLMDeployer
Creating inference service chaiml-pony-d3-g46-pv2-41910-v4
Waiting for inference service chaiml-pony-d3-g46-pv2-41910-v4 to be ready
Inference service chaiml-pony-d3-g46-pv2-41910-v4 ready after 592.4148442745209s
Pipeline stage VLLMDeployer completed in 592.84s
run pipeline stage %s
Running pipeline stage StressChecker
Received healthy response to inference request in 2.2007906436920166s
Received healthy response to inference request in 2.1306824684143066s
Received healthy response to inference request in 2.1591665744781494s
Received healthy response to inference request in 2.141432046890259s
Received healthy response to inference request in 2.137155294418335s
Received healthy response to inference request in 2.5903429985046387s
Received healthy response to inference request in 2.0756995677948s
Received healthy response to inference request in 2.058302402496338s
Received healthy response to inference request in 2.1087515354156494s
Received healthy response to inference request in 2.1173415184020996s
Received healthy response to inference request in 2.069817304611206s
Received healthy response to inference request in 2.152473211288452s
Received healthy response to inference request in 2.3964710235595703s
Received healthy response to inference request in 2.5125768184661865s
Received healthy response to inference request in 2.30138897895813s
Received healthy response to inference request in 2.3965957164764404s
Received healthy response to inference request in 2.150205373764038s
Received healthy response to inference request in 2.147954225540161s
Received healthy response to inference request in 2.1025595664978027s
Received healthy response to inference request in 2.0662851333618164s
Received healthy response to inference request in 2.373999834060669s
Received healthy response to inference request in 2.3860249519348145s
Received healthy response to inference request in 2.1348695755004883s
Received healthy response to inference request in 2.125120162963867s
Received healthy response to inference request in 2.0701005458831787s
Received healthy response to inference request in 2.1182992458343506s
Received healthy response to inference request in 2.655325174331665s
Received healthy response to inference request in 2.173830032348633s
Received healthy response to inference request in 2.164134979248047s
Received healthy response to inference request in 2.104703664779663s
30 requests
0 failed requests
5th percentile: 2.0678746104240417
10th percentile: 2.0700722217559813
20th percentile: 2.104274845123291
30th percentile: 2.1180119276046754
40th percentile: 2.1331947326660154
50th percentile: 2.14469313621521
60th percentile: 2.155150556564331
70th percentile: 2.1819182157516477
80th percentile: 2.376404857635498
90th percentile: 2.408193826675415
95th percentile: 2.555348217487335
99th percentile: 2.6364803433418276
mean time: 2.210746685663859
Pipeline stage StressChecker completed in 70.84s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyTriggerPipeline
run_pipeline:run_in_cloud %s
starting trigger_guanaco_pipeline args=%s
triggered trigger_guanaco_pipeline args=%s
Pipeline stage OfflineFamilyFriendlyTriggerPipeline completed in 0.90s
Shutdown handler de-registered
chaiml-pony-d3-g46-pv2-_41910_v4 status is now deployed due to DeploymentManager action
chaiml-pony-d3-g46-pv2-_41910_v4 status is now inactive due to auto deactivation removed underperforming models
chaiml-pony-d3-g46-pv2-_41910_v4 status is now torndown due to DeploymentManager action