developer_uid: chai_backend_admin
submission_id: chaiml-pony-v2-g46-lr1_80834_v19
model_name: chaiml-pony-v2-g46-lr1_80834_v19
model_group: ChaiML/pony-v2-g46-lr1e4
status: inactive
timestamp: 2026-03-04T20:57:17+00:00
num_battles: 10386
num_wins: 5732
celo_rating: 1331.8
family_friendly_score: 0.0
family_friendly_standard_error: 0.0
submission_type: basic
model_repo: ChaiML/pony-v2-g46-lr1e4ep1r64b16
model_architecture: Glm4MoeForCausalLM
model_num_parameters: 24110003200.0
best_of: 8
max_input_tokens: 2048
max_output_tokens: 80
reward_model: default
display_name: chaiml-pony-v2-g46-lr1_80834_v19
ineligible_reason: max_output_tokens!=64
is_internal_developer: True
language_model: ChaiML/pony-v2-g46-lr1e4ep1r64b16
model_size: 24B
ranking_group: single
us_pacific_date: 2026-03-04
win_ratio: 0.551896784132486
generation_params: {'temperature': 1.0, 'top_p': 1.0, 'min_p': 0.0, 'top_k': 40, 'presence_penalty': 0.0, 'frequency_penalty': 0.0, 'stopping_words': ['<|user|>', '####', '<|im_end|>', '</s>', '<|assistant|>'], 'max_input_tokens': 2048, 'best_of': 8, 'max_output_tokens': 80}
formatter: {'memory_template': "[gMASK]<sop><|system|>\n{bot_name}'s persona: {memory}", 'prompt_template': '', 'bot_template': '<|assistant|>\n<think></think>\n{message}', 'user_template': '<|user|>\n{message}', 'response_template': '<|assistant|>\n<think></think>\n{bot_name}:', 'truncate_by_message': True}
Resubmit model
Shutdown handler not registered because Python interpreter is not running in the main thread
run pipeline %s
run pipeline stage %s
Running pipeline stage VLLMUploader
Starting job with name chaiml-pony-v2-g46-lr1-80834-v19-uploader
Waiting for job on chaiml-pony-v2-g46-lr1-80834-v19-uploader to finish
chaiml-pony-v2-g46-lr1-80834-v19-uploader: Using quantization_mode: fp8
chaiml-pony-v2-g46-lr1-80834-v19-uploader: Checking if ChaiML/pony-v2-g46-lr1e4ep1r64b16-FP8 already exists in ChaiML
chaiml-pony-v2-g46-lr1-80834-v19-uploader: Model already exists. Downloading to /dev/shm/model_output...
chaiml-pony-v2-g46-lr1-80834-v19-uploader: Downloading snapshot of ChaiML/pony-v2-g46-lr1e4ep1r64b16-FP8...
chaiml-pony-v2-g46-lr1-80834-v19-uploader: Downloaded in 150.881s
chaiml-pony-v2-g46-lr1-80834-v19-uploader: Processed model ChaiML/pony-v2-g46-lr1e4ep1r64b16 in 154.435s
chaiml-pony-v2-g46-lr1-80834-v19-uploader: creating bucket guanaco-vllm-models
chaiml-pony-v2-g46-lr1-80834-v19-uploader: /usr/lib/python3/dist-packages/S3/BaseUtils.py:56: SyntaxWarning: invalid escape sequence '\.'
chaiml-pony-v2-g46-lr1-80834-v19-uploader:   RE_S3_DATESTRING = re.compile('\.[0-9]*(?:[Z\\-\\+]*?)')
chaiml-pony-v2-g46-lr1-80834-v19-uploader: /usr/lib/python3/dist-packages/S3/BaseUtils.py:57: SyntaxWarning: invalid escape sequence '\s'
chaiml-pony-v2-g46-lr1-80834-v19-uploader:   RE_XML_NAMESPACE = re.compile(b'^(<?[^>]+?>\s*|\s*)(<\w+) xmlns=[\'"](https?://[^\'"]+)[\'"]', re.MULTILINE)
chaiml-pony-v2-g46-lr1-80834-v19-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:240: SyntaxWarning: invalid escape sequence '\.'
chaiml-pony-v2-g46-lr1-80834-v19-uploader:   invalid = re.search("([^a-z0-9\.-])", bucket, re.UNICODE)
chaiml-pony-v2-g46-lr1-80834-v19-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:244: SyntaxWarning: invalid escape sequence '\.'
chaiml-pony-v2-g46-lr1-80834-v19-uploader:   invalid = re.search("([^A-Za-z0-9\._-])", bucket, re.UNICODE)
chaiml-pony-v2-g46-lr1-80834-v19-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:255: SyntaxWarning: invalid escape sequence '\.'
chaiml-pony-v2-g46-lr1-80834-v19-uploader:   if re.search("-\.", bucket, re.UNICODE):
chaiml-pony-v2-g46-lr1-80834-v19-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:257: SyntaxWarning: invalid escape sequence '\.'
chaiml-pony-v2-g46-lr1-80834-v19-uploader:   if re.search("\.\.", bucket, re.UNICODE):
chaiml-pony-v2-g46-lr1-80834-v19-uploader: /usr/lib/python3/dist-packages/S3/S3Uri.py:155: SyntaxWarning: invalid escape sequence '\w'
chaiml-pony-v2-g46-lr1-80834-v19-uploader:   _re = re.compile("^(\w+://)?(.*)", re.UNICODE)
chaiml-pony-v2-g46-lr1-80834-v19-uploader: /usr/lib/python3/dist-packages/S3/FileLists.py:480: SyntaxWarning: invalid escape sequence '\*'
chaiml-pony-v2-g46-lr1-80834-v19-uploader:   wildcard_split_result = re.split("\*|\?", uri_str, maxsplit=1)
chaiml-pony-v2-g46-lr1-80834-v19-uploader: Bucket 's3://guanaco-vllm-models/' created
chaiml-pony-v2-g46-lr1-80834-v19-uploader: uploading /dev/shm/model_output to s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v19/default
chaiml-pony-v2-g46-lr1-80834-v19-uploader: cp /dev/shm/model_output/chat_template.jinja s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v19/default/chat_template.jinja
chaiml-pony-v2-g46-lr1-80834-v19-uploader: cp /dev/shm/model_output/generation_config.json s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v19/default/generation_config.json
chaiml-pony-v2-g46-lr1-80834-v19-uploader: cp /dev/shm/model_output/config.json s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v19/default/config.json
chaiml-pony-v2-g46-lr1-80834-v19-uploader: cp /dev/shm/model_output/tokenizer_config.json s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v19/default/tokenizer_config.json
chaiml-pony-v2-g46-lr1-80834-v19-uploader: cp /dev/shm/model_output/recipe.yaml s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v19/default/recipe.yaml
chaiml-pony-v2-g46-lr1-80834-v19-uploader: cp /dev/shm/model_output/model.safetensors.index.json s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v19/default/model.safetensors.index.json
chaiml-pony-v2-g46-lr1-80834-v19-uploader: cp /dev/shm/model_output/special_tokens_map.json s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v19/default/special_tokens_map.json
chaiml-pony-v2-g46-lr1-80834-v19-uploader: cp /dev/shm/model_output/.gitattributes s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v19/default/.gitattributes
chaiml-pony-v2-g46-lr1-80834-v19-uploader: cp /dev/shm/model_output/tokenizer.json s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v19/default/tokenizer.json
chaiml-pony-v2-g46-lr1-80834-v19-uploader: cp /dev/shm/model_output/model-00072-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v19/default/model-00072-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v19-uploader: cp /dev/shm/model_output/model-00071-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v19/default/model-00071-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v19-uploader: cp /dev/shm/model_output/model-00018-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v19/default/model-00018-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v19-uploader: cp /dev/shm/model_output/model-00005-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v19/default/model-00005-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v19-uploader: cp /dev/shm/model_output/model-00017-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v19/default/model-00017-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v19-uploader: cp /dev/shm/model_output/model-00001-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v19/default/model-00001-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v19-uploader: cp /dev/shm/model_output/model-00050-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v19/default/model-00050-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v19-uploader: cp /dev/shm/model_output/model-00016-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v19/default/model-00016-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v19-uploader: cp /dev/shm/model_output/model-00031-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v19/default/model-00031-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v19-uploader: cp /dev/shm/model_output/model-00011-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v19/default/model-00011-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v19-uploader: cp /dev/shm/model_output/model-00049-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v19/default/model-00049-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v19-uploader: cp /dev/shm/model_output/model-00015-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v19/default/model-00015-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v19-uploader: cp /dev/shm/model_output/model-00065-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v19/default/model-00065-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v19-uploader: cp /dev/shm/model_output/model-00062-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v19/default/model-00062-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v19-uploader: cp /dev/shm/model_output/model-00056-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v19/default/model-00056-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v19-uploader: cp /dev/shm/model_output/model-00032-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v19/default/model-00032-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v19-uploader: cp /dev/shm/model_output/model-00045-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v19/default/model-00045-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v19-uploader: cp /dev/shm/model_output/model-00066-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v19/default/model-00066-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v19-uploader: cp /dev/shm/model_output/model-00023-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v19/default/model-00023-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v19-uploader: cp /dev/shm/model_output/model-00041-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v19/default/model-00041-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v19-uploader: cp /dev/shm/model_output/model-00043-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v19/default/model-00043-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v19-uploader: cp /dev/shm/model_output/model-00020-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v19/default/model-00020-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v19-uploader: cp /dev/shm/model_output/model-00067-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v19/default/model-00067-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v19-uploader: cp /dev/shm/model_output/model-00028-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v19/default/model-00028-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v19-uploader: cp /dev/shm/model_output/model-00064-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v19/default/model-00064-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v19-uploader: cp /dev/shm/model_output/model-00055-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v19/default/model-00055-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v19-uploader: cp /dev/shm/model_output/model-00058-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v19/default/model-00058-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v19-uploader: cp /dev/shm/model_output/model-00035-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v19/default/model-00035-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v19-uploader: cp /dev/shm/model_output/model-00024-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v19/default/model-00024-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v19-uploader: cp /dev/shm/model_output/model-00069-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v19/default/model-00069-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v19-uploader: cp /dev/shm/model_output/model-00033-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v19/default/model-00033-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v19-uploader: cp /dev/shm/model_output/model-00063-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v19/default/model-00063-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v19-uploader: cp /dev/shm/model_output/model-00052-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v19/default/model-00052-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v19-uploader: cp /dev/shm/model_output/model-00037-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v19/default/model-00037-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v19-uploader: cp /dev/shm/model_output/model-00044-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v19/default/model-00044-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v19-uploader: cp /dev/shm/model_output/model-00042-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v19/default/model-00042-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v19-uploader: cp /dev/shm/model_output/model-00040-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v19/default/model-00040-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v19-uploader: cp /dev/shm/model_output/model-00006-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v19/default/model-00006-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v19-uploader: cp /dev/shm/model_output/model-00046-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v19/default/model-00046-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v19-uploader: cp /dev/shm/model_output/model-00009-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v19/default/model-00009-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v19-uploader: cp /dev/shm/model_output/model-00051-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v19/default/model-00051-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v19-uploader: cp /dev/shm/model_output/model-00022-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v19/default/model-00022-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v19-uploader: cp /dev/shm/model_output/model-00013-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v19/default/model-00013-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v19-uploader: cp /dev/shm/model_output/model-00054-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v19/default/model-00054-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v19-uploader: cp /dev/shm/model_output/model-00060-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v19/default/model-00060-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v19-uploader: cp /dev/shm/model_output/model-00068-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v19/default/model-00068-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v19-uploader: cp /dev/shm/model_output/model-00026-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v19/default/model-00026-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v19-uploader: cp /dev/shm/model_output/model-00007-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v19/default/model-00007-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v19-uploader: cp /dev/shm/model_output/model-00003-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v19/default/model-00003-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v19-uploader: cp /dev/shm/model_output/model-00059-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v19/default/model-00059-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v19-uploader: cp /dev/shm/model_output/model-00012-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v19/default/model-00012-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v19-uploader: cp /dev/shm/model_output/model-00030-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v19/default/model-00030-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v19-uploader: cp /dev/shm/model_output/model-00053-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v19/default/model-00053-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v19-uploader: cp /dev/shm/model_output/model-00014-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v19/default/model-00014-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v19-uploader: cp /dev/shm/model_output/model-00039-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v19/default/model-00039-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v19-uploader: cp /dev/shm/model_output/model-00057-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v19/default/model-00057-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v19-uploader: cp /dev/shm/model_output/model-00021-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v19/default/model-00021-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v19-uploader: cp /dev/shm/model_output/model-00025-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v19/default/model-00025-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v19-uploader: cp /dev/shm/model_output/model-00004-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v19/default/model-00004-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v19-uploader: cp /dev/shm/model_output/model-00029-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v19/default/model-00029-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v19-uploader: cp /dev/shm/model_output/model-00061-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v19/default/model-00061-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v19-uploader: cp /dev/shm/model_output/model-00002-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v19/default/model-00002-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v19-uploader: cp /dev/shm/model_output/model-00047-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v19/default/model-00047-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v19-uploader: cp /dev/shm/model_output/model-00008-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v19/default/model-00008-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v19-uploader: cp /dev/shm/model_output/model-00036-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v19/default/model-00036-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v19-uploader: cp /dev/shm/model_output/model-00038-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v19/default/model-00038-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v19-uploader: cp /dev/shm/model_output/model-00027-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v19/default/model-00027-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v19-uploader: cp /dev/shm/model_output/model-00010-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v19/default/model-00010-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v19-uploader: cp /dev/shm/model_output/model-00034-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v19/default/model-00034-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v19-uploader: cp /dev/shm/model_output/model-00019-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v19/default/model-00019-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v19-uploader: cp /dev/shm/model_output/model-00048-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v19/default/model-00048-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v19-uploader: cp /dev/shm/model_output/model-00070-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v19/default/model-00070-of-00072.safetensors
Job chaiml-pony-v2-g46-lr1-80834-v19-uploader completed after 269.03s with status: succeeded
Stopping job with name chaiml-pony-v2-g46-lr1-80834-v19-uploader
Pipeline stage VLLMUploader completed in 269.53s
run pipeline stage %s
Running pipeline stage VLLMTemplater
Pipeline stage VLLMTemplater completed in 1.15s
run pipeline stage %s
Running pipeline stage VLLMDeployer
Creating inference service chaiml-pony-v2-g46-lr1-80834-v19
Waiting for inference service chaiml-pony-v2-g46-lr1-80834-v19 to be ready
Inference service chaiml-pony-v2-g46-lr1-80834-v19 ready after 561.5167708396912s
Pipeline stage VLLMDeployer completed in 561.97s
run pipeline stage %s
Running pipeline stage StressChecker
Received healthy response to inference request in 2.3168578147888184s
Received healthy response to inference request in 2.258221387863159s
Received healthy response to inference request in 2.421053886413574s
Received healthy response to inference request in 2.4911117553710938s
Received healthy response to inference request in 2.3507940769195557s
Received healthy response to inference request in 2.2686214447021484s
Received healthy response to inference request in 2.1631975173950195s
Received healthy response to inference request in 2.283052444458008s
Received healthy response to inference request in 1.9963443279266357s
Received healthy response to inference request in 2.2216010093688965s
Received healthy response to inference request in 2.2819738388061523s
Received healthy response to inference request in 2.0555098056793213s
Received healthy response to inference request in 2.047070264816284s
Received healthy response to inference request in 2.1158790588378906s
Received healthy response to inference request in 1.984529972076416s
Received healthy response to inference request in 2.185537099838257s
Received healthy response to inference request in 2.0253708362579346s
Received healthy response to inference request in 2.055596351623535s
Received healthy response to inference request in 2.039478063583374s
Received healthy response to inference request in 2.01501727104187s
Received healthy response to inference request in 2.038170337677002s
Received healthy response to inference request in 2.5276248455047607s
Received healthy response to inference request in 2.1132967472076416s
Received healthy response to inference request in 2.0260825157165527s
Received healthy response to inference request in 2.072899341583252s
Received healthy response to inference request in 2.1055471897125244s
Received healthy response to inference request in 2.050386428833008s
Received healthy response to inference request in 2.165863513946533s
Received healthy response to inference request in 2.0889732837677s
Received healthy response to inference request in 2.016101837158203s
30 requests
0 failed requests
5th percentile: 2.004747152328491
10th percentile: 2.0159933805465697
20th percentile: 2.035752773284912
30th percentile: 2.049391579627991
40th percentile: 2.0659781455993653
50th percentile: 2.109421968460083
60th percentile: 2.164263916015625
70th percentile: 2.232587122917175
80th percentile: 2.2821895599365236
90th percentile: 2.3578200578689574
95th percentile: 2.45958571434021
99th percentile: 2.5170360493659976
mean time: 2.1593921422958373
Pipeline stage StressChecker completed in 67.84s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyTriggerPipeline
run_pipeline:run_in_cloud %s
starting trigger_guanaco_pipeline args=%s
triggered trigger_guanaco_pipeline args=%s
Pipeline stage OfflineFamilyFriendlyTriggerPipeline completed in 0.72s
Shutdown handler de-registered
chaiml-pony-v2-g46-lr1_80834_v19 status is now deployed due to DeploymentManager action
chaiml-pony-v2-g46-lr1_80834_v19 status is now inactive due to auto deactivation removed underperforming models