developer_uid: zonemercy
submission_id: chaiml-pony-v2-g46-lr1_80834_v26
model_name: chaiml-pony-v2-g46-lr1_80834_v26
model_group: ChaiML/pony-v2-g46-lr1e4
status: torndown
timestamp: 2026-03-08T14:32:23+00:00
num_battles: 10603
num_wins: 5948
celo_rating: 1359.06
family_friendly_score: 0.0
family_friendly_standard_error: 0.0
submission_type: basic
model_repo: ChaiML/pony-v2-g46-lr1e4ep1r64b16
model_architecture: Glm4MoeForCausalLM
model_num_parameters: 24110003200.0
best_of: 8
max_input_tokens: 2048
max_output_tokens: 80
reward_model: default
display_name: chaiml-pony-v2-g46-lr1_80834_v26
ineligible_reason: max_output_tokens!=64
is_internal_developer: True
language_model: ChaiML/pony-v2-g46-lr1e4ep1r64b16
model_size: 24B
ranking_group: single
us_pacific_date: 2026-03-05
win_ratio: 0.5609733094407243
generation_params: {'temperature': 1.0, 'top_p': 1.0, 'min_p': 0.0, 'top_k': 40, 'presence_penalty': 0.0, 'frequency_penalty': 0.0, 'stopping_words': ['<|assistant|>', '<|user|>', '<|im_end|>', '</s>', '####'], 'max_input_tokens': 2048, 'best_of': 8, 'max_output_tokens': 80}
formatter: {'memory_template': "[gMASK]<sop><|system|>\n{bot_name}'s persona: {memory}", 'prompt_template': '', 'bot_template': '<|assistant|>\n<think></think>\n{message}', 'user_template': '<|user|>\n{message}', 'response_template': '<|assistant|>\n<think></think>\n{bot_name}:', 'truncate_by_message': True}
Resubmit model
Shutdown handler not registered because Python interpreter is not running in the main thread
run pipeline %s
run pipeline stage %s
Running pipeline stage VLLMUploader
Starting job with name chaiml-pony-v2-g46-lr1-80834-v26-uploader
Waiting for job on chaiml-pony-v2-g46-lr1-80834-v26-uploader to finish
chaiml-pony-v2-g46-lr1-80834-v26-uploader: Using quantization_mode: fp8
chaiml-pony-v2-g46-lr1-80834-v26-uploader: Checking if ChaiML/pony-v2-g46-lr1e4ep1r64b16-FP8 already exists in ChaiML
chaiml-pony-v2-g46-lr1-80834-v26-uploader: Model already exists. Downloading to /dev/shm/model_output...
chaiml-pony-v2-g46-lr1-80834-v26-uploader: Downloading snapshot of ChaiML/pony-v2-g46-lr1e4ep1r64b16-FP8...
chaiml-pony-v2-g46-lr1-80834-v26-uploader: Downloaded in 396.957s
chaiml-pony-v2-g46-lr1-80834-v26-uploader: Processed model ChaiML/pony-v2-g46-lr1e4ep1r64b16 in 400.557s
chaiml-pony-v2-g46-lr1-80834-v26-uploader: creating bucket guanaco-vllm-models
chaiml-pony-v2-g46-lr1-80834-v26-uploader: /usr/lib/python3/dist-packages/S3/BaseUtils.py:56: SyntaxWarning: invalid escape sequence '\.'
chaiml-pony-v2-g46-lr1-80834-v26-uploader:   RE_S3_DATESTRING = re.compile('\.[0-9]*(?:[Z\\-\\+]*?)')
chaiml-pony-v2-g46-lr1-80834-v26-uploader: /usr/lib/python3/dist-packages/S3/BaseUtils.py:57: SyntaxWarning: invalid escape sequence '\s'
chaiml-pony-v2-g46-lr1-80834-v26-uploader:   RE_XML_NAMESPACE = re.compile(b'^(<?[^>]+?>\s*|\s*)(<\w+) xmlns=[\'"](https?://[^\'"]+)[\'"]', re.MULTILINE)
chaiml-pony-v2-g46-lr1-80834-v26-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:240: SyntaxWarning: invalid escape sequence '\.'
chaiml-pony-v2-g46-lr1-80834-v26-uploader:   invalid = re.search("([^a-z0-9\.-])", bucket, re.UNICODE)
chaiml-pony-v2-g46-lr1-80834-v26-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:244: SyntaxWarning: invalid escape sequence '\.'
chaiml-pony-v2-g46-lr1-80834-v26-uploader:   invalid = re.search("([^A-Za-z0-9\._-])", bucket, re.UNICODE)
chaiml-pony-v2-g46-lr1-80834-v26-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:255: SyntaxWarning: invalid escape sequence '\.'
chaiml-pony-v2-g46-lr1-80834-v26-uploader:   if re.search("-\.", bucket, re.UNICODE):
chaiml-pony-v2-g46-lr1-80834-v26-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:257: SyntaxWarning: invalid escape sequence '\.'
chaiml-pony-v2-g46-lr1-80834-v26-uploader:   if re.search("\.\.", bucket, re.UNICODE):
chaiml-pony-v2-g46-lr1-80834-v26-uploader: /usr/lib/python3/dist-packages/S3/S3Uri.py:155: SyntaxWarning: invalid escape sequence '\w'
chaiml-pony-v2-g46-lr1-80834-v26-uploader:   _re = re.compile("^(\w+://)?(.*)", re.UNICODE)
chaiml-pony-v2-g46-lr1-80834-v26-uploader: /usr/lib/python3/dist-packages/S3/FileLists.py:480: SyntaxWarning: invalid escape sequence '\*'
chaiml-pony-v2-g46-lr1-80834-v26-uploader:   wildcard_split_result = re.split("\*|\?", uri_str, maxsplit=1)
chaiml-pony-v2-g46-lr1-80834-v26-uploader: Bucket 's3://guanaco-vllm-models/' created
chaiml-pony-v2-g46-lr1-80834-v26-uploader: uploading /dev/shm/model_output to s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v26/default
chaiml-pony-v2-g46-lr1-80834-v26-uploader: cp /dev/shm/model_output/config.json s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v26/default/config.json
chaiml-pony-v2-g46-lr1-80834-v26-uploader: cp /dev/shm/model_output/.gitattributes s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v26/default/.gitattributes
chaiml-pony-v2-g46-lr1-80834-v26-uploader: cp /dev/shm/model_output/generation_config.json s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v26/default/generation_config.json
chaiml-pony-v2-g46-lr1-80834-v26-uploader: cp /dev/shm/model_output/chat_template.jinja s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v26/default/chat_template.jinja
chaiml-pony-v2-g46-lr1-80834-v26-uploader: cp /dev/shm/model_output/recipe.yaml s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v26/default/recipe.yaml
chaiml-pony-v2-g46-lr1-80834-v26-uploader: cp /dev/shm/model_output/tokenizer_config.json s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v26/default/tokenizer_config.json
chaiml-pony-v2-g46-lr1-80834-v26-uploader: cp /dev/shm/model_output/special_tokens_map.json s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v26/default/special_tokens_map.json
chaiml-pony-v2-g46-lr1-80834-v26-uploader: cp /dev/shm/model_output/tokenizer.json s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v26/default/tokenizer.json
chaiml-pony-v2-g46-lr1-80834-v26-uploader: cp /dev/shm/model_output/model.safetensors.index.json s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v26/default/model.safetensors.index.json
chaiml-pony-v2-g46-lr1-80834-v26-uploader: cp /dev/shm/model_output/model-00072-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v26/default/model-00072-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v26-uploader: cp /dev/shm/model_output/model-00071-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v26/default/model-00071-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v26-uploader: cp /dev/shm/model_output/model-00029-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v26/default/model-00029-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v26-uploader: cp /dev/shm/model_output/model-00060-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v26/default/model-00060-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v26-uploader: cp /dev/shm/model_output/model-00042-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v26/default/model-00042-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v26-uploader: cp /dev/shm/model_output/model-00017-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v26/default/model-00017-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v26-uploader: cp /dev/shm/model_output/model-00021-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v26/default/model-00021-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v26-uploader: cp /dev/shm/model_output/model-00035-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v26/default/model-00035-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v26-uploader: cp /dev/shm/model_output/model-00037-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v26/default/model-00037-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v26-uploader: cp /dev/shm/model_output/model-00018-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v26/default/model-00018-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v26-uploader: cp /dev/shm/model_output/model-00036-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v26/default/model-00036-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v26-uploader: cp /dev/shm/model_output/model-00024-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v26/default/model-00024-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v26-uploader: cp /dev/shm/model_output/model-00049-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v26/default/model-00049-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v26-uploader: cp /dev/shm/model_output/model-00064-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v26/default/model-00064-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v26-uploader: cp /dev/shm/model_output/model-00012-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v26/default/model-00012-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v26-uploader: cp /dev/shm/model_output/model-00011-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v26/default/model-00011-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v26-uploader: cp /dev/shm/model_output/model-00066-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v26/default/model-00066-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v26-uploader: cp /dev/shm/model_output/model-00070-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v26/default/model-00070-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v26-uploader: cp /dev/shm/model_output/model-00009-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v26/default/model-00009-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v26-uploader: cp /dev/shm/model_output/model-00019-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v26/default/model-00019-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v26-uploader: cp /dev/shm/model_output/model-00050-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v26/default/model-00050-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v26-uploader: cp /dev/shm/model_output/model-00015-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v26/default/model-00015-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v26-uploader: cp /dev/shm/model_output/model-00010-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v26/default/model-00010-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v26-uploader: cp /dev/shm/model_output/model-00020-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v26/default/model-00020-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v26-uploader: cp /dev/shm/model_output/model-00030-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v26/default/model-00030-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v26-uploader: cp /dev/shm/model_output/model-00048-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v26/default/model-00048-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v26-uploader: cp /dev/shm/model_output/model-00038-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v26/default/model-00038-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v26-uploader: cp /dev/shm/model_output/model-00068-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v26/default/model-00068-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v26-uploader: cp /dev/shm/model_output/model-00043-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v26/default/model-00043-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v26-uploader: cp /dev/shm/model_output/model-00034-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v26/default/model-00034-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v26-uploader: cp /dev/shm/model_output/model-00067-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v26/default/model-00067-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v26-uploader: cp /dev/shm/model_output/model-00023-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v26/default/model-00023-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v26-uploader: cp /dev/shm/model_output/model-00002-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v26/default/model-00002-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v26-uploader: cp /dev/shm/model_output/model-00032-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v26/default/model-00032-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v26-uploader: cp /dev/shm/model_output/model-00007-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v26/default/model-00007-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v26-uploader: cp /dev/shm/model_output/model-00061-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v26/default/model-00061-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v26-uploader: cp /dev/shm/model_output/model-00040-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v26/default/model-00040-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v26-uploader: cp /dev/shm/model_output/model-00063-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v26/default/model-00063-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v26-uploader: cp /dev/shm/model_output/model-00055-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v26/default/model-00055-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v26-uploader: cp /dev/shm/model_output/model-00056-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v26/default/model-00056-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v26-uploader: cp /dev/shm/model_output/model-00008-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v26/default/model-00008-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v26-uploader: cp /dev/shm/model_output/model-00027-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v26/default/model-00027-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v26-uploader: cp /dev/shm/model_output/model-00046-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v26/default/model-00046-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v26-uploader: cp /dev/shm/model_output/model-00001-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v26/default/model-00001-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v26-uploader: cp /dev/shm/model_output/model-00028-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v26/default/model-00028-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v26-uploader: cp /dev/shm/model_output/model-00051-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v26/default/model-00051-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v26-uploader: cp /dev/shm/model_output/model-00013-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v26/default/model-00013-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v26-uploader: cp /dev/shm/model_output/model-00006-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v26/default/model-00006-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v26-uploader: cp /dev/shm/model_output/model-00058-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v26/default/model-00058-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v26-uploader: cp /dev/shm/model_output/model-00026-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v26/default/model-00026-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v26-uploader: cp /dev/shm/model_output/model-00054-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v26/default/model-00054-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v26-uploader: cp /dev/shm/model_output/model-00052-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v26/default/model-00052-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v26-uploader: cp /dev/shm/model_output/model-00062-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v26/default/model-00062-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v26-uploader: cp /dev/shm/model_output/model-00053-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v26/default/model-00053-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v26-uploader: cp /dev/shm/model_output/model-00031-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v26/default/model-00031-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v26-uploader: cp /dev/shm/model_output/model-00039-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v26/default/model-00039-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v26-uploader: cp /dev/shm/model_output/model-00003-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v26/default/model-00003-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v26-uploader: cp /dev/shm/model_output/model-00014-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v26/default/model-00014-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v26-uploader: cp /dev/shm/model_output/model-00069-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v26/default/model-00069-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v26-uploader: cp /dev/shm/model_output/model-00059-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v26/default/model-00059-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v26-uploader: cp /dev/shm/model_output/model-00033-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v26/default/model-00033-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v26-uploader: cp /dev/shm/model_output/model-00041-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v26/default/model-00041-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v26-uploader: cp /dev/shm/model_output/model-00044-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v26/default/model-00044-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v26-uploader: cp /dev/shm/model_output/model-00057-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v26/default/model-00057-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v26-uploader: cp /dev/shm/model_output/model-00065-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v26/default/model-00065-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v26-uploader: cp /dev/shm/model_output/model-00004-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v26/default/model-00004-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v26-uploader: cp /dev/shm/model_output/model-00045-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v26/default/model-00045-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v26-uploader: cp /dev/shm/model_output/model-00022-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v26/default/model-00022-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v26-uploader: cp /dev/shm/model_output/model-00005-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v26/default/model-00005-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v26-uploader: cp /dev/shm/model_output/model-00016-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v26/default/model-00016-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v26-uploader: cp /dev/shm/model_output/model-00047-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v26/default/model-00047-of-00072.safetensors
Job chaiml-pony-v2-g46-lr1-80834-v26-uploader completed after 534.66s with status: succeeded
Stopping job with name chaiml-pony-v2-g46-lr1-80834-v26-uploader
Pipeline stage VLLMUploader completed in 535.23s
run pipeline stage %s
Running pipeline stage VLLMTemplater
Pipeline stage VLLMTemplater completed in 2.89s
run pipeline stage %s
Running pipeline stage VLLMDeployer
Creating inference service chaiml-pony-v2-g46-lr1-80834-v26
Waiting for inference service chaiml-pony-v2-g46-lr1-80834-v26 to be ready
Inference service chaiml-pony-v2-g46-lr1-80834-v26 ready after 552.0763885974884s
Pipeline stage VLLMDeployer completed in 552.80s
run pipeline stage %s
Running pipeline stage StressChecker
Received healthy response to inference request in 2.174210786819458s
Received healthy response to inference request in 2.2214362621307373s
Received healthy response to inference request in 2.382408857345581s
Received healthy response to inference request in 2.142321825027466s
Received healthy response to inference request in 2.133882761001587s
Received healthy response to inference request in 2.357252836227417s
Received healthy response to inference request in 2.2371764183044434s
Received healthy response to inference request in 2.228804349899292s
Received healthy response to inference request in 1.9983384609222412s
Received healthy response to inference request in 2.178926467895508s
Received healthy response to inference request in 2.5416316986083984s
Received healthy response to inference request in 2.1461143493652344s
Received healthy response to inference request in 2.1058349609375s
Received healthy response to inference request in 2.263383388519287s
Received healthy response to inference request in 2.5068697929382324s
Received healthy response to inference request in 2.480012893676758s
Received healthy response to inference request in 2.2005064487457275s
Received healthy response to inference request in 2.017792224884033s
Received healthy response to inference request in 2.0409021377563477s
Received healthy response to inference request in 2.001375913619995s
Received healthy response to inference request in 2.224670648574829s
Received healthy response to inference request in 2.8232619762420654s
Received healthy response to inference request in 2.0430519580841064s
Received healthy response to inference request in 2.308751344680786s
Received healthy response to inference request in 2.292548656463623s
Received healthy response to inference request in 2.6882498264312744s
Received healthy response to inference request in 2.0642871856689453s
Received healthy response to inference request in 2.961155414581299s
Received healthy response to inference request in 2.2173261642456055s
Received healthy response to inference request in 2.582897663116455s
30 requests
0 failed requests
5th percentile: 2.0087632536888123
10th percentile: 2.038591146469116
20th percentile: 2.097525405883789
30th percentile: 2.144976592063904
40th percentile: 2.19187445640564
50th percentile: 2.223053455352783
60th percentile: 2.247659206390381
70th percentile: 2.3233017921447754
80th percentile: 2.485384273529053
90th percentile: 2.593432879447937
95th percentile: 2.762506508827209
99th percentile: 2.921166317462921
mean time: 2.2855127890904745
Pipeline stage StressChecker completed in 72.49s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyTriggerPipeline
run_pipeline:run_in_cloud %s
starting trigger_guanaco_pipeline args=%s
triggered trigger_guanaco_pipeline args=%s
Pipeline stage OfflineFamilyFriendlyTriggerPipeline completed in 0.72s
Shutdown handler de-registered
chaiml-pony-v2-g46-lr1_80834_v26 status is now deployed due to DeploymentManager action
chaiml-pony-v2-g46-lr1_80834_v26 status is now inactive due to auto deactivation removed underperforming models
chaiml-pony-v2-g46-lr1_80834_v26 status is now torndown due to DeploymentManager action