developer_uid: chai_backend_admin
submission_id: chaiml-pony-v2-g46-lr1_80834_v20
model_name: chaiml-pony-v2-g46-lr1_80834_v20
model_group: ChaiML/pony-v2-g46-lr1e4
status: inactive
timestamp: 2026-03-04T21:17:26+00:00
num_battles: 10890
num_wins: 6109
celo_rating: 1334.4
family_friendly_score: 0.0
family_friendly_standard_error: 0.0
submission_type: basic
model_repo: ChaiML/pony-v2-g46-lr1e4ep1r64b16
model_architecture: Glm4MoeForCausalLM
model_num_parameters: 24110003200.0
best_of: 8
max_input_tokens: 2048
max_output_tokens: 80
reward_model: default
display_name: chaiml-pony-v2-g46-lr1_80834_v20
ineligible_reason: max_output_tokens!=64
is_internal_developer: True
language_model: ChaiML/pony-v2-g46-lr1e4ep1r64b16
model_size: 24B
ranking_group: single
us_pacific_date: 2026-03-04
win_ratio: 0.5609733700642792
generation_params: {'temperature': 1.0, 'top_p': 1.0, 'min_p': 0.0, 'top_k': 40, 'presence_penalty': 0.0, 'frequency_penalty': 0.0, 'stopping_words': ['<|im_end|>', '</s>', '####', '<|user|>', '<|assistant|>'], 'max_input_tokens': 2048, 'best_of': 8, 'max_output_tokens': 80}
formatter: {'memory_template': "[gMASK]<sop><|system|>\n{bot_name}'s persona: {memory}.<|user|>\n", 'prompt_template': '', 'bot_template': '{bot_name}: {message}\n', 'user_template': '{user_name}: {message}\n', 'response_template': '<|assistant|>\n<think></think>\n{bot_name}:', 'truncate_by_message': True}
Resubmit model
Shutdown handler not registered because Python interpreter is not running in the main thread
run pipeline %s
run pipeline stage %s
Running pipeline stage VLLMUploader
Starting job with name chaiml-pony-v2-g46-lr1-80834-v20-uploader
Waiting for job on chaiml-pony-v2-g46-lr1-80834-v20-uploader to finish
chaiml-pony-v2-g46-lr1-80834-v20-uploader: Using quantization_mode: fp8
chaiml-pony-v2-g46-lr1-80834-v20-uploader: Checking if ChaiML/pony-v2-g46-lr1e4ep1r64b16-FP8 already exists in ChaiML
chaiml-pony-v2-g46-lr1-80834-v20-uploader: Model already exists. Downloading to /dev/shm/model_output...
chaiml-pony-v2-g46-lr1-80834-v20-uploader: Downloading snapshot of ChaiML/pony-v2-g46-lr1e4ep1r64b16-FP8...
chaiml-pony-v2-g46-lr1-80834-v20-uploader: Downloaded in 168.252s
chaiml-pony-v2-g46-lr1-80834-v20-uploader: Processed model ChaiML/pony-v2-g46-lr1e4ep1r64b16 in 171.754s
chaiml-pony-v2-g46-lr1-80834-v20-uploader: creating bucket guanaco-vllm-models
chaiml-pony-v2-g46-lr1-80834-v20-uploader: /usr/lib/python3/dist-packages/S3/BaseUtils.py:56: SyntaxWarning: invalid escape sequence '\.'
chaiml-pony-v2-g46-lr1-80834-v20-uploader:   RE_S3_DATESTRING = re.compile('\.[0-9]*(?:[Z\\-\\+]*?)')
chaiml-pony-v2-g46-lr1-80834-v20-uploader: /usr/lib/python3/dist-packages/S3/BaseUtils.py:57: SyntaxWarning: invalid escape sequence '\s'
chaiml-pony-v2-g46-lr1-80834-v20-uploader:   RE_XML_NAMESPACE = re.compile(b'^(<?[^>]+?>\s*|\s*)(<\w+) xmlns=[\'"](https?://[^\'"]+)[\'"]', re.MULTILINE)
chaiml-pony-v2-g46-lr1-80834-v20-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:240: SyntaxWarning: invalid escape sequence '\.'
chaiml-pony-v2-g46-lr1-80834-v20-uploader:   invalid = re.search("([^a-z0-9\.-])", bucket, re.UNICODE)
chaiml-pony-v2-g46-lr1-80834-v20-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:244: SyntaxWarning: invalid escape sequence '\.'
chaiml-pony-v2-g46-lr1-80834-v20-uploader:   invalid = re.search("([^A-Za-z0-9\._-])", bucket, re.UNICODE)
chaiml-pony-v2-g46-lr1-80834-v20-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:255: SyntaxWarning: invalid escape sequence '\.'
chaiml-pony-v2-g46-lr1-80834-v20-uploader:   if re.search("-\.", bucket, re.UNICODE):
chaiml-pony-v2-g46-lr1-80834-v20-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:257: SyntaxWarning: invalid escape sequence '\.'
chaiml-pony-v2-g46-lr1-80834-v20-uploader:   if re.search("\.\.", bucket, re.UNICODE):
chaiml-pony-v2-g46-lr1-80834-v20-uploader: /usr/lib/python3/dist-packages/S3/S3Uri.py:155: SyntaxWarning: invalid escape sequence '\w'
chaiml-pony-v2-g46-lr1-80834-v20-uploader:   _re = re.compile("^(\w+://)?(.*)", re.UNICODE)
chaiml-pony-v2-g46-lr1-80834-v20-uploader: /usr/lib/python3/dist-packages/S3/FileLists.py:480: SyntaxWarning: invalid escape sequence '\*'
chaiml-pony-v2-g46-lr1-80834-v20-uploader:   wildcard_split_result = re.split("\*|\?", uri_str, maxsplit=1)
chaiml-pony-v2-g46-lr1-80834-v20-uploader: Bucket 's3://guanaco-vllm-models/' created
chaiml-pony-v2-g46-lr1-80834-v20-uploader: uploading /dev/shm/model_output to s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v20/default
chaiml-pony-v2-g46-lr1-80834-v20-uploader: cp /dev/shm/model_output/chat_template.jinja s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v20/default/chat_template.jinja
chaiml-pony-v2-g46-lr1-80834-v20-uploader: cp /dev/shm/model_output/config.json s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v20/default/config.json
chaiml-pony-v2-g46-lr1-80834-v20-uploader: cp /dev/shm/model_output/.gitattributes s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v20/default/.gitattributes
chaiml-pony-v2-g46-lr1-80834-v20-uploader: cp /dev/shm/model_output/model.safetensors.index.json s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v20/default/model.safetensors.index.json
chaiml-pony-v2-g46-lr1-80834-v20-uploader: cp /dev/shm/model_output/tokenizer.json s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v20/default/tokenizer.json
chaiml-pony-v2-g46-lr1-80834-v20-uploader: cp /dev/shm/model_output/model-00072-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v20/default/model-00072-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v20-uploader: cp /dev/shm/model_output/model-00071-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v20/default/model-00071-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v20-uploader: cp /dev/shm/model_output/model-00032-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v20/default/model-00032-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v20-uploader: cp /dev/shm/model_output/model-00037-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v20/default/model-00037-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v20-uploader: cp /dev/shm/model_output/model-00045-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v20/default/model-00045-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v20-uploader: cp /dev/shm/model_output/model-00041-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v20/default/model-00041-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v20-uploader: cp /dev/shm/model_output/model-00029-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v20/default/model-00029-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v20-uploader: cp /dev/shm/model_output/model-00028-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v20/default/model-00028-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v20-uploader: cp /dev/shm/model_output/model-00024-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v20/default/model-00024-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v20-uploader: cp /dev/shm/model_output/model-00063-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v20/default/model-00063-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v20-uploader: cp /dev/shm/model_output/model-00008-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v20/default/model-00008-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v20-uploader: cp /dev/shm/model_output/model-00056-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v20/default/model-00056-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v20-uploader: cp /dev/shm/model_output/model-00016-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v20/default/model-00016-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v20-uploader: cp /dev/shm/model_output/model-00057-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v20/default/model-00057-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v20-uploader: cp /dev/shm/model_output/model-00049-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v20/default/model-00049-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v20-uploader: cp /dev/shm/model_output/model-00066-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v20/default/model-00066-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v20-uploader: cp /dev/shm/model_output/model-00011-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v20/default/model-00011-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v20-uploader: cp /dev/shm/model_output/model-00009-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v20/default/model-00009-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v20-uploader: cp /dev/shm/model_output/model-00040-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v20/default/model-00040-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v20-uploader: cp /dev/shm/model_output/model-00015-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v20/default/model-00015-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v20-uploader: cp /dev/shm/model_output/model-00034-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v20/default/model-00034-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v20-uploader: cp /dev/shm/model_output/model-00035-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v20/default/model-00035-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v20-uploader: cp /dev/shm/model_output/model-00043-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v20/default/model-00043-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v20-uploader: cp /dev/shm/model_output/model-00036-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v20/default/model-00036-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v20-uploader: cp /dev/shm/model_output/model-00044-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v20/default/model-00044-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v20-uploader: cp /dev/shm/model_output/model-00023-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v20/default/model-00023-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v20-uploader: cp /dev/shm/model_output/model-00042-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v20/default/model-00042-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v20-uploader: cp /dev/shm/model_output/model-00013-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v20/default/model-00013-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v20-uploader: cp /dev/shm/model_output/model-00012-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v20/default/model-00012-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v20-uploader: cp /dev/shm/model_output/model-00059-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v20/default/model-00059-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v20-uploader: cp /dev/shm/model_output/model-00026-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v20/default/model-00026-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v20-uploader: cp /dev/shm/model_output/model-00052-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v20/default/model-00052-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v20-uploader: cp /dev/shm/model_output/model-00007-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v20/default/model-00007-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v20-uploader: cp /dev/shm/model_output/model-00018-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v20/default/model-00018-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v20-uploader: cp /dev/shm/model_output/model-00061-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v20/default/model-00061-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v20-uploader: cp /dev/shm/model_output/model-00003-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v20/default/model-00003-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v20-uploader: cp /dev/shm/model_output/model-00033-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v20/default/model-00033-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v20-uploader: cp /dev/shm/model_output/model-00001-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v20/default/model-00001-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v20-uploader: cp /dev/shm/model_output/model-00010-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v20/default/model-00010-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v20-uploader: cp /dev/shm/model_output/model-00070-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v20/default/model-00070-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v20-uploader: cp /dev/shm/model_output/model-00050-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v20/default/model-00050-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v20-uploader: cp /dev/shm/model_output/model-00004-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v20/default/model-00004-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v20-uploader: cp /dev/shm/model_output/model-00060-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v20/default/model-00060-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v20-uploader: cp /dev/shm/model_output/model-00031-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v20/default/model-00031-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v20-uploader: cp /dev/shm/model_output/model-00055-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v20/default/model-00055-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v20-uploader: cp /dev/shm/model_output/model-00002-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v20/default/model-00002-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v20-uploader: cp /dev/shm/model_output/model-00020-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v20/default/model-00020-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v20-uploader: cp /dev/shm/model_output/model-00019-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v20/default/model-00019-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v20-uploader: cp /dev/shm/model_output/model-00058-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v20/default/model-00058-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v20-uploader: cp /dev/shm/model_output/model-00053-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v20/default/model-00053-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v20-uploader: cp /dev/shm/model_output/model-00014-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v20/default/model-00014-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v20-uploader: cp /dev/shm/model_output/model-00054-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v20/default/model-00054-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v20-uploader: cp /dev/shm/model_output/model-00027-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v20/default/model-00027-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v20-uploader: cp /dev/shm/model_output/model-00039-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v20/default/model-00039-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v20-uploader: cp /dev/shm/model_output/model-00067-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v20/default/model-00067-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v20-uploader: cp /dev/shm/model_output/model-00038-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v20/default/model-00038-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v20-uploader: cp /dev/shm/model_output/model-00006-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v20/default/model-00006-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v20-uploader: cp /dev/shm/model_output/model-00064-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v20/default/model-00064-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v20-uploader: cp /dev/shm/model_output/model-00068-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v20/default/model-00068-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v20-uploader: cp /dev/shm/model_output/model-00030-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v20/default/model-00030-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v20-uploader: cp /dev/shm/model_output/model-00021-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v20/default/model-00021-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v20-uploader: cp /dev/shm/model_output/model-00005-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v20/default/model-00005-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v20-uploader: cp /dev/shm/model_output/model-00017-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v20/default/model-00017-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v20-uploader: cp /dev/shm/model_output/model-00048-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v20/default/model-00048-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v20-uploader: cp /dev/shm/model_output/model-00022-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v20/default/model-00022-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v20-uploader: cp /dev/shm/model_output/model-00047-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v20/default/model-00047-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v20-uploader: cp /dev/shm/model_output/model-00046-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v20/default/model-00046-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v20-uploader: cp /dev/shm/model_output/model-00069-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v20/default/model-00069-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v20-uploader: cp /dev/shm/model_output/model-00025-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v20/default/model-00025-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v20-uploader: cp /dev/shm/model_output/model-00065-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v20/default/model-00065-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v20-uploader: cp /dev/shm/model_output/model-00062-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v20/default/model-00062-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v20-uploader: cp /dev/shm/model_output/model-00051-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v20/default/model-00051-of-00072.safetensors
Job chaiml-pony-v2-g46-lr1-80834-v20-uploader completed after 308.88s with status: succeeded
Stopping job with name chaiml-pony-v2-g46-lr1-80834-v20-uploader
Pipeline stage VLLMUploader completed in 309.43s
run pipeline stage %s
Running pipeline stage VLLMTemplater
Pipeline stage VLLMTemplater completed in 1.96s
run pipeline stage %s
Running pipeline stage VLLMDeployer
Creating inference service chaiml-pony-v2-g46-lr1-80834-v20
Waiting for inference service chaiml-pony-v2-g46-lr1-80834-v20 to be ready
Failed to get response for submission chaiml-llama31-mer-v2-_44570_v66: 'scores'
Inference service chaiml-pony-v2-g46-lr1-80834-v20 ready after 511.7796256542206s
Pipeline stage VLLMDeployer completed in 512.30s
run pipeline stage %s
Running pipeline stage StressChecker
Received healthy response to inference request in 2.1946747303009033s
Received healthy response to inference request in 2.275068998336792s
Received healthy response to inference request in 2.052184820175171s
Received healthy response to inference request in 2.006284475326538s
Received healthy response to inference request in 2.1342461109161377s
Received healthy response to inference request in 2.0663933753967285s
Received healthy response to inference request in 2.206462860107422s
Received healthy response to inference request in 2.1894893646240234s
Received healthy response to inference request in 2.395625591278076s
Received healthy response to inference request in 2.276726722717285s
Received healthy response to inference request in 1.9816648960113525s
Received healthy response to inference request in 2.2457761764526367s
Received healthy response to inference request in 2.599410057067871s
Received healthy response to inference request in 2.0522329807281494s
Received healthy response to inference request in 2.0887258052825928s
Received healthy response to inference request in 2.1110222339630127s
Received healthy response to inference request in 2.195038318634033s
Received healthy response to inference request in 2.129333734512329s
Received healthy response to inference request in 2.18121075630188s
Received healthy response to inference request in 2.4128053188323975s
Received healthy response to inference request in 2.1173911094665527s
Received healthy response to inference request in 2.0273826122283936s
Received healthy response to inference request in 2.1705968379974365s
Received healthy response to inference request in 2.4222729206085205s
Received healthy response to inference request in 2.008211851119995s
Received healthy response to inference request in 2.3422794342041016s
Received healthy response to inference request in 2.531935930252075s
Received healthy response to inference request in 2.279266595840454s
Received healthy response to inference request in 2.183959484100342s
Received healthy response to inference request in 2.3659090995788574s
30 requests
0 failed requests
5th percentile: 2.007151794433594
10th percentile: 2.0254655361175535
20th percentile: 2.063561296463013
30th percentile: 2.1154804468154906
40th percentile: 2.156056547164917
50th percentile: 2.1867244243621826
60th percentile: 2.1996081352233885
70th percentile: 2.27556631565094
80th percentile: 2.3470053672790527
90th percentile: 2.4137520790100098
95th percentile: 2.482587575912475
99th percentile: 2.5798425602912904
mean time: 2.208119440078735
Pipeline stage StressChecker completed in 70.57s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyTriggerPipeline
run_pipeline:run_in_cloud %s
starting trigger_guanaco_pipeline args=%s
triggered trigger_guanaco_pipeline args=%s
Pipeline stage OfflineFamilyFriendlyTriggerPipeline completed in 0.81s
Shutdown handler de-registered
chaiml-pony-v2-g46-lr1_80834_v20 status is now deployed due to DeploymentManager action
chaiml-pony-v2-g46-lr1_80834_v20 status is now inactive due to auto deactivation removed underperforming models