developer_uid: zonemercy
submission_id: chaiml-pony-d3-g46-pv2-_41910_v5
model_name: chaiml-pony-d3-g46-pv2-_41910_v5
model_group: ChaiML/pony-d3-g46-pv2-l
status: torndown
timestamp: 2026-03-09T16:30:07+00:00
num_battles: 10903
num_wins: 6240
celo_rating: 1350.31
family_friendly_score: 0.0
family_friendly_standard_error: 0.0
submission_type: basic
model_repo: ChaiML/pony-d3-g46-pv2-lr5e6ep1r64b8
model_architecture: Glm4MoeForCausalLM
model_num_parameters: 24110003200.0
best_of: 8
max_input_tokens: 2048
max_output_tokens: 80
reward_model: default
display_name: chaiml-pony-d3-g46-pv2-_41910_v5
ineligible_reason: max_output_tokens!=64
is_internal_developer: True
language_model: ChaiML/pony-d3-g46-pv2-lr5e6ep1r64b8
model_size: 24B
ranking_group: single
us_pacific_date: 2026-03-06
win_ratio: 0.5723195450793359
generation_params: {'temperature': 1.0, 'top_p': 0.95, 'min_p': 0.0, 'top_k': 60, 'presence_penalty': 0.0, 'frequency_penalty': 0.0, 'stopping_words': ['<|assistant|>', '<|user|>', '<|im_end|>', '</s>', '####'], 'max_input_tokens': 2048, 'best_of': 8, 'max_output_tokens': 80}
formatter: {'memory_template': "[gMASK]<sop><|system|>\n{bot_name}'s persona: {memory}.<|user|>\n", 'prompt_template': '', 'bot_template': '{bot_name}: {message}\n', 'user_template': '{user_name}: {message}\n', 'response_template': '<|assistant|>\n<think></think>\n{bot_name}:', 'truncate_by_message': True}
Resubmit model
Shutdown handler not registered because Python interpreter is not running in the main thread
run pipeline %s
run pipeline stage %s
Running pipeline stage VLLMUploader
Starting job with name chaiml-pony-d3-g46-pv2-41910-v5-uploader
Waiting for job on chaiml-pony-d3-g46-pv2-41910-v5-uploader to finish
chaiml-pony-d3-g46-pv2-41910-v5-uploader: Using quantization_mode: fp8
chaiml-pony-d3-g46-pv2-41910-v5-uploader: Checking if ChaiML/pony-d3-g46-pv2-lr5e6ep1r64b8-FP8 already exists in ChaiML
chaiml-pony-d3-g46-pv2-41910-v5-uploader: Model already exists. Downloading to /dev/shm/model_output...
chaiml-pony-d3-g46-pv2-41910-v5-uploader: Downloading snapshot of ChaiML/pony-d3-g46-pv2-lr5e6ep1r64b8-FP8...
chaiml-pony-d3-g46-pv2-41910-v5-uploader: Downloaded in 108.407s
chaiml-pony-d3-g46-pv2-41910-v5-uploader: Processed model ChaiML/pony-d3-g46-pv2-lr5e6ep1r64b8 in 111.920s
chaiml-pony-d3-g46-pv2-41910-v5-uploader: creating bucket guanaco-vllm-models
chaiml-pony-d3-g46-pv2-41910-v5-uploader: /usr/lib/python3/dist-packages/S3/BaseUtils.py:56: SyntaxWarning: invalid escape sequence '\.'
chaiml-pony-d3-g46-pv2-41910-v5-uploader:   RE_S3_DATESTRING = re.compile('\.[0-9]*(?:[Z\\-\\+]*?)')
chaiml-pony-d3-g46-pv2-41910-v5-uploader: /usr/lib/python3/dist-packages/S3/BaseUtils.py:57: SyntaxWarning: invalid escape sequence '\s'
chaiml-pony-d3-g46-pv2-41910-v5-uploader:   RE_XML_NAMESPACE = re.compile(b'^(<?[^>]+?>\s*|\s*)(<\w+) xmlns=[\'"](https?://[^\'"]+)[\'"]', re.MULTILINE)
chaiml-pony-d3-g46-pv2-41910-v5-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:240: SyntaxWarning: invalid escape sequence '\.'
chaiml-pony-d3-g46-pv2-41910-v5-uploader:   invalid = re.search("([^a-z0-9\.-])", bucket, re.UNICODE)
chaiml-pony-d3-g46-pv2-41910-v5-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:244: SyntaxWarning: invalid escape sequence '\.'
chaiml-pony-d3-g46-pv2-41910-v5-uploader:   invalid = re.search("([^A-Za-z0-9\._-])", bucket, re.UNICODE)
chaiml-pony-d3-g46-pv2-41910-v5-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:255: SyntaxWarning: invalid escape sequence '\.'
chaiml-pony-d3-g46-pv2-41910-v5-uploader:   if re.search("-\.", bucket, re.UNICODE):
chaiml-pony-d3-g46-pv2-41910-v5-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:257: SyntaxWarning: invalid escape sequence '\.'
chaiml-pony-d3-g46-pv2-41910-v5-uploader:   if re.search("\.\.", bucket, re.UNICODE):
chaiml-pony-d3-g46-pv2-41910-v5-uploader: /usr/lib/python3/dist-packages/S3/S3Uri.py:155: SyntaxWarning: invalid escape sequence '\w'
chaiml-pony-d3-g46-pv2-41910-v5-uploader:   _re = re.compile("^(\w+://)?(.*)", re.UNICODE)
chaiml-pony-d3-g46-pv2-41910-v5-uploader: /usr/lib/python3/dist-packages/S3/FileLists.py:480: SyntaxWarning: invalid escape sequence '\*'
chaiml-pony-d3-g46-pv2-41910-v5-uploader:   wildcard_split_result = re.split("\*|\?", uri_str, maxsplit=1)
chaiml-pony-d3-g46-pv2-41910-v5-uploader: Bucket 's3://guanaco-vllm-models/' created
chaiml-pony-d3-g46-pv2-41910-v5-uploader: uploading /dev/shm/model_output to s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-41910-v5/default
chaiml-pony-d3-g46-pv2-41910-v5-uploader: cp /dev/shm/model_output/chat_template.jinja s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-41910-v5/default/chat_template.jinja
chaiml-pony-d3-g46-pv2-41910-v5-uploader: cp /dev/shm/model_output/generation_config.json s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-41910-v5/default/generation_config.json
chaiml-pony-d3-g46-pv2-41910-v5-uploader: cp /dev/shm/model_output/special_tokens_map.json s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-41910-v5/default/special_tokens_map.json
chaiml-pony-d3-g46-pv2-41910-v5-uploader: cp /dev/shm/model_output/recipe.yaml s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-41910-v5/default/recipe.yaml
chaiml-pony-d3-g46-pv2-41910-v5-uploader: cp /dev/shm/model_output/config.json s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-41910-v5/default/config.json
chaiml-pony-d3-g46-pv2-41910-v5-uploader: cp /dev/shm/model_output/.gitattributes s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-41910-v5/default/.gitattributes
chaiml-pony-d3-g46-pv2-41910-v5-uploader: cp /dev/shm/model_output/model.safetensors.index.json s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-41910-v5/default/model.safetensors.index.json
chaiml-pony-d3-g46-pv2-41910-v5-uploader: cp /dev/shm/model_output/tokenizer_config.json s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-41910-v5/default/tokenizer_config.json
chaiml-pony-d3-g46-pv2-41910-v5-uploader: cp /dev/shm/model_output/tokenizer.json s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-41910-v5/default/tokenizer.json
chaiml-pony-d3-g46-pv2-41910-v5-uploader: cp /dev/shm/model_output/model-00072-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-41910-v5/default/model-00072-of-00072.safetensors
chaiml-pony-d3-g46-pv2-41910-v5-uploader: cp /dev/shm/model_output/model-00071-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-41910-v5/default/model-00071-of-00072.safetensors
chaiml-pony-d3-g46-pv2-41910-v5-uploader: cp /dev/shm/model_output/model-00046-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-41910-v5/default/model-00046-of-00072.safetensors
chaiml-pony-d3-g46-pv2-41910-v5-uploader: cp /dev/shm/model_output/model-00007-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-41910-v5/default/model-00007-of-00072.safetensors
chaiml-pony-d3-g46-pv2-41910-v5-uploader: cp /dev/shm/model_output/model-00060-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-41910-v5/default/model-00060-of-00072.safetensors
chaiml-pony-d3-g46-pv2-41910-v5-uploader: cp /dev/shm/model_output/model-00070-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-41910-v5/default/model-00070-of-00072.safetensors
chaiml-pony-d3-g46-pv2-41910-v5-uploader: cp /dev/shm/model_output/model-00068-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-41910-v5/default/model-00068-of-00072.safetensors
chaiml-pony-d3-g46-pv2-41910-v5-uploader: cp /dev/shm/model_output/model-00062-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-41910-v5/default/model-00062-of-00072.safetensors
chaiml-pony-d3-g46-pv2-41910-v5-uploader: cp /dev/shm/model_output/model-00018-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-41910-v5/default/model-00018-of-00072.safetensors
chaiml-pony-d3-g46-pv2-41910-v5-uploader: cp /dev/shm/model_output/model-00054-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-41910-v5/default/model-00054-of-00072.safetensors
chaiml-pony-d3-g46-pv2-41910-v5-uploader: cp /dev/shm/model_output/model-00039-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-41910-v5/default/model-00039-of-00072.safetensors
chaiml-pony-d3-g46-pv2-41910-v5-uploader: cp /dev/shm/model_output/model-00053-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-41910-v5/default/model-00053-of-00072.safetensors
chaiml-pony-d3-g46-pv2-41910-v5-uploader: cp /dev/shm/model_output/model-00014-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-41910-v5/default/model-00014-of-00072.safetensors
chaiml-pony-d3-g46-pv2-41910-v5-uploader: cp /dev/shm/model_output/model-00033-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-41910-v5/default/model-00033-of-00072.safetensors
chaiml-pony-d3-g46-pv2-41910-v5-uploader: cp /dev/shm/model_output/model-00069-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-41910-v5/default/model-00069-of-00072.safetensors
chaiml-pony-d3-g46-pv2-41910-v5-uploader: cp /dev/shm/model_output/model-00013-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-41910-v5/default/model-00013-of-00072.safetensors
chaiml-pony-d3-g46-pv2-41910-v5-uploader: cp /dev/shm/model_output/model-00051-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-41910-v5/default/model-00051-of-00072.safetensors
chaiml-pony-d3-g46-pv2-41910-v5-uploader: cp /dev/shm/model_output/model-00064-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-41910-v5/default/model-00064-of-00072.safetensors
chaiml-pony-d3-g46-pv2-41910-v5-uploader: cp /dev/shm/model_output/model-00024-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-41910-v5/default/model-00024-of-00072.safetensors
chaiml-pony-d3-g46-pv2-41910-v5-uploader: cp /dev/shm/model_output/model-00037-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-41910-v5/default/model-00037-of-00072.safetensors
chaiml-pony-d3-g46-pv2-41910-v5-uploader: cp /dev/shm/model_output/model-00017-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-41910-v5/default/model-00017-of-00072.safetensors
chaiml-pony-d3-g46-pv2-41910-v5-uploader: cp /dev/shm/model_output/model-00011-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-41910-v5/default/model-00011-of-00072.safetensors
chaiml-pony-d3-g46-pv2-41910-v5-uploader: cp /dev/shm/model_output/model-00003-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-41910-v5/default/model-00003-of-00072.safetensors
chaiml-pony-d3-g46-pv2-41910-v5-uploader: cp /dev/shm/model_output/model-00050-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-41910-v5/default/model-00050-of-00072.safetensors
chaiml-pony-d3-g46-pv2-41910-v5-uploader: cp /dev/shm/model_output/model-00031-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-41910-v5/default/model-00031-of-00072.safetensors
chaiml-pony-d3-g46-pv2-41910-v5-uploader: cp /dev/shm/model_output/model-00025-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-41910-v5/default/model-00025-of-00072.safetensors
chaiml-pony-d3-g46-pv2-41910-v5-uploader: cp /dev/shm/model_output/model-00004-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-41910-v5/default/model-00004-of-00072.safetensors
chaiml-pony-d3-g46-pv2-41910-v5-uploader: cp /dev/shm/model_output/model-00038-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-41910-v5/default/model-00038-of-00072.safetensors
chaiml-pony-d3-g46-pv2-41910-v5-uploader: cp /dev/shm/model_output/model-00041-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-41910-v5/default/model-00041-of-00072.safetensors
chaiml-pony-d3-g46-pv2-41910-v5-uploader: cp /dev/shm/model_output/model-00005-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-41910-v5/default/model-00005-of-00072.safetensors
chaiml-pony-d3-g46-pv2-41910-v5-uploader: cp /dev/shm/model_output/model-00009-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-41910-v5/default/model-00009-of-00072.safetensors
chaiml-pony-d3-g46-pv2-41910-v5-uploader: cp /dev/shm/model_output/model-00010-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-41910-v5/default/model-00010-of-00072.safetensors
chaiml-pony-d3-g46-pv2-41910-v5-uploader: cp /dev/shm/model_output/model-00002-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-41910-v5/default/model-00002-of-00072.safetensors
chaiml-pony-d3-g46-pv2-41910-v5-uploader: cp /dev/shm/model_output/model-00023-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-41910-v5/default/model-00023-of-00072.safetensors
chaiml-pony-d3-g46-pv2-41910-v5-uploader: cp /dev/shm/model_output/model-00042-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-41910-v5/default/model-00042-of-00072.safetensors
chaiml-pony-d3-g46-pv2-41910-v5-uploader: cp /dev/shm/model_output/model-00065-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-41910-v5/default/model-00065-of-00072.safetensors
chaiml-pony-d3-g46-pv2-41910-v5-uploader: cp /dev/shm/model_output/model-00045-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-41910-v5/default/model-00045-of-00072.safetensors
chaiml-pony-d3-g46-pv2-41910-v5-uploader: cp /dev/shm/model_output/model-00067-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-41910-v5/default/model-00067-of-00072.safetensors
chaiml-pony-d3-g46-pv2-41910-v5-uploader: cp /dev/shm/model_output/model-00035-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-41910-v5/default/model-00035-of-00072.safetensors
chaiml-pony-d3-g46-pv2-41910-v5-uploader: cp /dev/shm/model_output/model-00001-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-41910-v5/default/model-00001-of-00072.safetensors
chaiml-pony-d3-g46-pv2-41910-v5-uploader: cp /dev/shm/model_output/model-00040-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-41910-v5/default/model-00040-of-00072.safetensors
chaiml-pony-d3-g46-pv2-41910-v5-uploader: cp /dev/shm/model_output/model-00015-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-41910-v5/default/model-00015-of-00072.safetensors
chaiml-pony-d3-g46-pv2-41910-v5-uploader: cp /dev/shm/model_output/model-00066-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-41910-v5/default/model-00066-of-00072.safetensors
chaiml-pony-d3-g46-pv2-41910-v5-uploader: cp /dev/shm/model_output/model-00036-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-41910-v5/default/model-00036-of-00072.safetensors
chaiml-pony-d3-g46-pv2-41910-v5-uploader: cp /dev/shm/model_output/model-00012-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-41910-v5/default/model-00012-of-00072.safetensors
chaiml-pony-d3-g46-pv2-41910-v5-uploader: cp /dev/shm/model_output/model-00063-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-41910-v5/default/model-00063-of-00072.safetensors
chaiml-pony-d3-g46-pv2-41910-v5-uploader: cp /dev/shm/model_output/model-00029-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-41910-v5/default/model-00029-of-00072.safetensors
chaiml-pony-d3-g46-pv2-41910-v5-uploader: cp /dev/shm/model_output/model-00022-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-41910-v5/default/model-00022-of-00072.safetensors
chaiml-pony-d3-g46-pv2-41910-v5-uploader: cp /dev/shm/model_output/model-00055-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-41910-v5/default/model-00055-of-00072.safetensors
chaiml-pony-d3-g46-pv2-41910-v5-uploader: cp /dev/shm/model_output/model-00049-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-41910-v5/default/model-00049-of-00072.safetensors
chaiml-pony-d3-g46-pv2-41910-v5-uploader: cp /dev/shm/model_output/model-00019-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-41910-v5/default/model-00019-of-00072.safetensors
chaiml-pony-d3-g46-pv2-41910-v5-uploader: cp /dev/shm/model_output/model-00028-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-41910-v5/default/model-00028-of-00072.safetensors
chaiml-pony-d3-g46-pv2-41910-v5-uploader: cp /dev/shm/model_output/model-00061-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-41910-v5/default/model-00061-of-00072.safetensors
chaiml-pony-d3-g46-pv2-41910-v5-uploader: cp /dev/shm/model_output/model-00048-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-41910-v5/default/model-00048-of-00072.safetensors
chaiml-pony-d3-g46-pv2-41910-v5-uploader: cp /dev/shm/model_output/model-00047-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-41910-v5/default/model-00047-of-00072.safetensors
chaiml-pony-d3-g46-pv2-41910-v5-uploader: cp /dev/shm/model_output/model-00044-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-41910-v5/default/model-00044-of-00072.safetensors
chaiml-pony-d3-g46-pv2-41910-v5-uploader: cp /dev/shm/model_output/model-00032-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-41910-v5/default/model-00032-of-00072.safetensors
chaiml-pony-d3-g46-pv2-41910-v5-uploader: cp /dev/shm/model_output/model-00016-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-41910-v5/default/model-00016-of-00072.safetensors
chaiml-pony-d3-g46-pv2-41910-v5-uploader: cp /dev/shm/model_output/model-00006-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-41910-v5/default/model-00006-of-00072.safetensors
chaiml-pony-d3-g46-pv2-41910-v5-uploader: cp /dev/shm/model_output/model-00057-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-41910-v5/default/model-00057-of-00072.safetensors
chaiml-pony-d3-g46-pv2-41910-v5-uploader: cp /dev/shm/model_output/model-00043-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-41910-v5/default/model-00043-of-00072.safetensors
chaiml-pony-d3-g46-pv2-41910-v5-uploader: cp /dev/shm/model_output/model-00027-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-41910-v5/default/model-00027-of-00072.safetensors
chaiml-pony-d3-g46-pv2-41910-v5-uploader: cp /dev/shm/model_output/model-00052-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-41910-v5/default/model-00052-of-00072.safetensors
chaiml-pony-d3-g46-pv2-41910-v5-uploader: cp /dev/shm/model_output/model-00056-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-41910-v5/default/model-00056-of-00072.safetensors
Job chaiml-pony-d3-g46-pv2-41910-v5-uploader completed after 258.37s with status: succeeded
Stopping job with name chaiml-pony-d3-g46-pv2-41910-v5-uploader
Pipeline stage VLLMUploader completed in 258.91s
run pipeline stage %s
Running pipeline stage VLLMTemplater
Pipeline stage VLLMTemplater completed in 0.91s
run pipeline stage %s
Running pipeline stage VLLMDeployer
Creating inference service chaiml-pony-d3-g46-pv2-41910-v5
Waiting for inference service chaiml-pony-d3-g46-pv2-41910-v5 to be ready
Inference service chaiml-pony-d3-g46-pv2-41910-v5 ready after 621.6095933914185s
Pipeline stage VLLMDeployer completed in 622.08s
run pipeline stage %s
Running pipeline stage StressChecker
Received healthy response to inference request in 2.112271785736084s
Received healthy response to inference request in 2.175938844680786s
Received healthy response to inference request in 2.2179110050201416s
Received healthy response to inference request in 2.236872911453247s
Received healthy response to inference request in 2.219512701034546s
Received healthy response to inference request in 2.1085619926452637s
Received healthy response to inference request in 2.3362224102020264s
Received healthy response to inference request in 2.230780601501465s
Received healthy response to inference request in 2.1926534175872803s
Received healthy response to inference request in 2.59997820854187s
Received healthy response to inference request in 2.264695882797241s
Received healthy response to inference request in 2.506589412689209s
Received healthy response to inference request in 2.119173049926758s
Received healthy response to inference request in 2.082087278366089s
Received healthy response to inference request in 2.119929075241089s
Received healthy response to inference request in 2.13118577003479s
Received healthy response to inference request in 2.055910587310791s
Received healthy response to inference request in 2.5284423828125s
Received healthy response to inference request in 2.1772706508636475s
Received healthy response to inference request in 2.1802849769592285s
Received healthy response to inference request in 2.2126026153564453s
Received healthy response to inference request in 2.160534381866455s
Received healthy response to inference request in 2.0945894718170166s
Received healthy response to inference request in 2.248591899871826s
Received healthy response to inference request in 2.334373712539673s
Received healthy response to inference request in 2.0877187252044678s
Received healthy response to inference request in 2.1093077659606934s
Received healthy response to inference request in 2.2020103931427s
Received healthy response to inference request in 2.1818623542785645s
Received healthy response to inference request in 2.1185338497161865s
30 requests
0 failed requests
5th percentile: 2.0846214294433594
10th percentile: 2.0939023971557615
20th percentile: 2.1116789817810058
30th percentile: 2.1197022676467894
40th percentile: 2.1697770595550536
50th percentile: 2.1810736656188965
60th percentile: 2.2062472820281984
70th percentile: 2.2228930711746218
80th percentile: 2.251812696456909
90th percentile: 2.353259110450745
95th percentile: 2.518608546257019
99th percentile: 2.579232819080353
mean time: 2.211546603838603
Pipeline stage StressChecker completed in 74.39s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyTriggerPipeline
run_pipeline:run_in_cloud %s
starting trigger_guanaco_pipeline args=%s
triggered trigger_guanaco_pipeline args=%s
Pipeline stage OfflineFamilyFriendlyTriggerPipeline completed in 0.80s
Shutdown handler de-registered
chaiml-pony-d3-g46-pv2-_41910_v5 status is now deployed due to DeploymentManager action
chaiml-pony-d3-g46-pv2-_41910_v5 status is now inactive due to auto deactivation removed underperforming models
chaiml-pony-d3-g46-pv2-_41910_v5 status is now torndown due to DeploymentManager action