developer_uid: zonemercy
submission_id: chaiml-pony-v2-g46-lr1_80834_v21
model_name: chaiml-pony-v2-g46-lr1_80834_v21
model_group: ChaiML/pony-v2-g46-lr1e4
status: torndown
timestamp: 2026-03-08T03:43:18+00:00
num_battles: 10670
num_wins: 5924
celo_rating: 1333.28
family_friendly_score: 0.0
family_friendly_standard_error: 0.0
submission_type: basic
model_repo: ChaiML/pony-v2-g46-lr1e4ep1r64b16
model_architecture: Glm4MoeForCausalLM
model_num_parameters: 24110003200.0
best_of: 8
max_input_tokens: 2048
max_output_tokens: 80
reward_model: default
display_name: chaiml-pony-v2-g46-lr1_80834_v21
ineligible_reason: max_output_tokens!=64
is_internal_developer: True
language_model: ChaiML/pony-v2-g46-lr1e4ep1r64b16
model_size: 24B
ranking_group: single
us_pacific_date: 2026-03-04
win_ratio: 0.5552014995313964
generation_params: {'temperature': 1.0, 'top_p': 1.0, 'min_p': 0.0, 'top_k': 40, 'presence_penalty': 0.0, 'frequency_penalty': 0.0, 'stopping_words': ['<|assistant|>', '<|user|>', '<|im_end|>', '</s>', '####'], 'max_input_tokens': 2048, 'best_of': 8, 'max_output_tokens': 80}
formatter: {'memory_template': "[gMASK]<sop><|system|>\n{bot_name}'s persona: {memory}", 'prompt_template': '', 'bot_template': '<|assistant|>\n<think></think>\n{message}', 'user_template': '<|user|>\n{message}', 'response_template': '<|assistant|>\n<think></think>\n{bot_name}:', 'truncate_by_message': True}
Resubmit model
Shutdown handler not registered because Python interpreter is not running in the main thread
run pipeline %s
run pipeline stage %s
Running pipeline stage VLLMUploader
Starting job with name chaiml-pony-v2-g46-lr1-80834-v21-uploader
Waiting for job on chaiml-pony-v2-g46-lr1-80834-v21-uploader to finish
chaiml-pony-v2-g46-lr1-80834-v21-uploader: Using quantization_mode: fp8
chaiml-pony-v2-g46-lr1-80834-v21-uploader: Checking if ChaiML/pony-v2-g46-lr1e4ep1r64b16-FP8 already exists in ChaiML
chaiml-pony-v2-g46-lr1-80834-v21-uploader: Model already exists. Downloading to /dev/shm/model_output...
chaiml-pony-v2-g46-lr1-80834-v21-uploader: Downloading snapshot of ChaiML/pony-v2-g46-lr1e4ep1r64b16-FP8...
chaiml-pony-v2-g46-lr1-80834-v21-uploader: Downloaded in 153.677s
chaiml-pony-v2-g46-lr1-80834-v21-uploader: Processed model ChaiML/pony-v2-g46-lr1e4ep1r64b16 in 157.147s
chaiml-pony-v2-g46-lr1-80834-v21-uploader: creating bucket guanaco-vllm-models
chaiml-pony-v2-g46-lr1-80834-v21-uploader: /usr/lib/python3/dist-packages/S3/BaseUtils.py:56: SyntaxWarning: invalid escape sequence '\.'
chaiml-pony-v2-g46-lr1-80834-v21-uploader:   RE_S3_DATESTRING = re.compile('\.[0-9]*(?:[Z\\-\\+]*?)')
chaiml-pony-v2-g46-lr1-80834-v21-uploader: /usr/lib/python3/dist-packages/S3/BaseUtils.py:57: SyntaxWarning: invalid escape sequence '\s'
chaiml-pony-v2-g46-lr1-80834-v21-uploader:   RE_XML_NAMESPACE = re.compile(b'^(<?[^>]+?>\s*|\s*)(<\w+) xmlns=[\'"](https?://[^\'"]+)[\'"]', re.MULTILINE)
chaiml-pony-v2-g46-lr1-80834-v21-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:240: SyntaxWarning: invalid escape sequence '\.'
chaiml-pony-v2-g46-lr1-80834-v21-uploader:   invalid = re.search("([^a-z0-9\.-])", bucket, re.UNICODE)
chaiml-pony-v2-g46-lr1-80834-v21-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:244: SyntaxWarning: invalid escape sequence '\.'
chaiml-pony-v2-g46-lr1-80834-v21-uploader:   invalid = re.search("([^A-Za-z0-9\._-])", bucket, re.UNICODE)
chaiml-pony-v2-g46-lr1-80834-v21-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:255: SyntaxWarning: invalid escape sequence '\.'
chaiml-pony-v2-g46-lr1-80834-v21-uploader:   if re.search("-\.", bucket, re.UNICODE):
chaiml-pony-v2-g46-lr1-80834-v21-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:257: SyntaxWarning: invalid escape sequence '\.'
chaiml-pony-v2-g46-lr1-80834-v21-uploader:   if re.search("\.\.", bucket, re.UNICODE):
chaiml-pony-v2-g46-lr1-80834-v21-uploader: /usr/lib/python3/dist-packages/S3/S3Uri.py:155: SyntaxWarning: invalid escape sequence '\w'
chaiml-pony-v2-g46-lr1-80834-v21-uploader:   _re = re.compile("^(\w+://)?(.*)", re.UNICODE)
chaiml-pony-v2-g46-lr1-80834-v21-uploader: /usr/lib/python3/dist-packages/S3/FileLists.py:480: SyntaxWarning: invalid escape sequence '\*'
chaiml-pony-v2-g46-lr1-80834-v21-uploader:   wildcard_split_result = re.split("\*|\?", uri_str, maxsplit=1)
chaiml-pony-v2-g46-lr1-80834-v21-uploader: Bucket 's3://guanaco-vllm-models/' created
chaiml-pony-v2-g46-lr1-80834-v21-uploader: uploading /dev/shm/model_output to s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v21/default
chaiml-pony-v2-g46-lr1-80834-v21-uploader: cp /dev/shm/model_output/config.json s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v21/default/config.json
chaiml-pony-v2-g46-lr1-80834-v21-uploader: cp /dev/shm/model_output/.gitattributes s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v21/default/.gitattributes
chaiml-pony-v2-g46-lr1-80834-v21-uploader: cp /dev/shm/model_output/generation_config.json s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v21/default/generation_config.json
chaiml-pony-v2-g46-lr1-80834-v21-uploader: cp /dev/shm/model_output/chat_template.jinja s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v21/default/chat_template.jinja
chaiml-pony-v2-g46-lr1-80834-v21-uploader: cp /dev/shm/model_output/tokenizer_config.json s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v21/default/tokenizer_config.json
chaiml-pony-v2-g46-lr1-80834-v21-uploader: cp /dev/shm/model_output/recipe.yaml s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v21/default/recipe.yaml
chaiml-pony-v2-g46-lr1-80834-v21-uploader: cp /dev/shm/model_output/special_tokens_map.json s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v21/default/special_tokens_map.json
chaiml-pony-v2-g46-lr1-80834-v21-uploader: cp /dev/shm/model_output/model.safetensors.index.json s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v21/default/model.safetensors.index.json
chaiml-pony-v2-g46-lr1-80834-v21-uploader: cp /dev/shm/model_output/tokenizer.json s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v21/default/tokenizer.json
chaiml-pony-v2-g46-lr1-80834-v21-uploader: cp /dev/shm/model_output/model-00072-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v21/default/model-00072-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v21-uploader: cp /dev/shm/model_output/model-00071-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v21/default/model-00071-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v21-uploader: cp /dev/shm/model_output/model-00025-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v21/default/model-00025-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v21-uploader: cp /dev/shm/model_output/model-00024-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v21/default/model-00024-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v21-uploader: cp /dev/shm/model_output/model-00042-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v21/default/model-00042-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v21-uploader: cp /dev/shm/model_output/model-00032-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v21/default/model-00032-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v21-uploader: cp /dev/shm/model_output/model-00069-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v21/default/model-00069-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v21-uploader: cp /dev/shm/model_output/model-00002-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v21/default/model-00002-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v21-uploader: cp /dev/shm/model_output/model-00068-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v21/default/model-00068-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v21-uploader: cp /dev/shm/model_output/model-00028-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v21/default/model-00028-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v21-uploader: cp /dev/shm/model_output/model-00003-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v21/default/model-00003-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v21-uploader: cp /dev/shm/model_output/model-00061-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v21/default/model-00061-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v21-uploader: cp /dev/shm/model_output/model-00043-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v21/default/model-00043-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v21-uploader: cp /dev/shm/model_output/model-00005-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v21/default/model-00005-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v21-uploader: cp /dev/shm/model_output/model-00063-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v21/default/model-00063-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v21-uploader: cp /dev/shm/model_output/model-00015-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v21/default/model-00015-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v21-uploader: cp /dev/shm/model_output/model-00020-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v21/default/model-00020-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v21-uploader: cp /dev/shm/model_output/model-00013-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v21/default/model-00013-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v21-uploader: cp /dev/shm/model_output/model-00059-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v21/default/model-00059-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v21-uploader: cp /dev/shm/model_output/model-00027-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v21/default/model-00027-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v21-uploader: cp /dev/shm/model_output/model-00016-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v21/default/model-00016-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v21-uploader: cp /dev/shm/model_output/model-00045-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v21/default/model-00045-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v21-uploader: cp /dev/shm/model_output/model-00039-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v21/default/model-00039-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v21-uploader: cp /dev/shm/model_output/model-00026-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v21/default/model-00026-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v21-uploader: cp /dev/shm/model_output/model-00033-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v21/default/model-00033-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v21-uploader: cp /dev/shm/model_output/model-00008-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v21/default/model-00008-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v21-uploader: cp /dev/shm/model_output/model-00006-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v21/default/model-00006-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v21-uploader: cp /dev/shm/model_output/model-00017-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v21/default/model-00017-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v21-uploader: cp /dev/shm/model_output/model-00035-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v21/default/model-00035-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v21-uploader: cp /dev/shm/model_output/model-00044-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v21/default/model-00044-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v21-uploader: cp /dev/shm/model_output/model-00047-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v21/default/model-00047-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v21-uploader: cp /dev/shm/model_output/model-00030-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v21/default/model-00030-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v21-uploader: cp /dev/shm/model_output/model-00055-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v21/default/model-00055-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v21-uploader: cp /dev/shm/model_output/model-00067-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v21/default/model-00067-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v21-uploader: cp /dev/shm/model_output/model-00034-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v21/default/model-00034-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v21-uploader: cp /dev/shm/model_output/model-00019-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v21/default/model-00019-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v21-uploader: cp /dev/shm/model_output/model-00037-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v21/default/model-00037-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v21-uploader: cp /dev/shm/model_output/model-00029-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v21/default/model-00029-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v21-uploader: cp /dev/shm/model_output/model-00058-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v21/default/model-00058-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v21-uploader: cp /dev/shm/model_output/model-00040-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v21/default/model-00040-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v21-uploader: cp /dev/shm/model_output/model-00066-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v21/default/model-00066-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v21-uploader: cp /dev/shm/model_output/model-00036-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v21/default/model-00036-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v21-uploader: cp /dev/shm/model_output/model-00046-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v21/default/model-00046-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v21-uploader: cp /dev/shm/model_output/model-00031-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v21/default/model-00031-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v21-uploader: cp /dev/shm/model_output/model-00021-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v21/default/model-00021-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v21-uploader: cp /dev/shm/model_output/model-00056-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v21/default/model-00056-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v21-uploader: cp /dev/shm/model_output/model-00022-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v21/default/model-00022-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v21-uploader: cp /dev/shm/model_output/model-00023-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v21/default/model-00023-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v21-uploader: cp /dev/shm/model_output/model-00070-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v21/default/model-00070-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v21-uploader: cp /dev/shm/model_output/model-00007-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v21/default/model-00007-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v21-uploader: cp /dev/shm/model_output/model-00051-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v21/default/model-00051-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v21-uploader: cp /dev/shm/model_output/model-00018-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v21/default/model-00018-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v21-uploader: cp /dev/shm/model_output/model-00001-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v21/default/model-00001-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v21-uploader: cp /dev/shm/model_output/model-00041-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v21/default/model-00041-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v21-uploader: cp /dev/shm/model_output/model-00010-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v21/default/model-00010-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v21-uploader: cp /dev/shm/model_output/model-00065-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v21/default/model-00065-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v21-uploader: cp /dev/shm/model_output/model-00038-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v21/default/model-00038-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v21-uploader: cp /dev/shm/model_output/model-00052-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v21/default/model-00052-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v21-uploader: cp /dev/shm/model_output/model-00057-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v21/default/model-00057-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v21-uploader: cp /dev/shm/model_output/model-00054-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v21/default/model-00054-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v21-uploader: cp /dev/shm/model_output/model-00011-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v21/default/model-00011-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v21-uploader: cp /dev/shm/model_output/model-00064-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v21/default/model-00064-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v21-uploader: cp /dev/shm/model_output/model-00009-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v21/default/model-00009-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v21-uploader: cp /dev/shm/model_output/model-00014-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v21/default/model-00014-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v21-uploader: cp /dev/shm/model_output/model-00062-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v21/default/model-00062-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v21-uploader: cp /dev/shm/model_output/model-00049-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v21/default/model-00049-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v21-uploader: cp /dev/shm/model_output/model-00048-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v21/default/model-00048-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v21-uploader: cp /dev/shm/model_output/model-00050-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v21/default/model-00050-of-00072.safetensors
Job chaiml-pony-v2-g46-lr1-80834-v21-uploader completed after 310.09s with status: succeeded
Stopping job with name chaiml-pony-v2-g46-lr1-80834-v21-uploader
Pipeline stage VLLMUploader completed in 310.60s
run pipeline stage %s
Running pipeline stage VLLMTemplater
Pipeline stage VLLMTemplater completed in 1.28s
run pipeline stage %s
Running pipeline stage VLLMDeployer
Creating inference service chaiml-pony-v2-g46-lr1-80834-v21
Waiting for inference service chaiml-pony-v2-g46-lr1-80834-v21 to be ready
Inference service chaiml-pony-v2-g46-lr1-80834-v21 ready after 651.7221994400024s
Pipeline stage VLLMDeployer completed in 652.56s
run pipeline stage %s
Running pipeline stage StressChecker
Received healthy response to inference request in 2.31506085395813s
Received healthy response to inference request in 2.4624550342559814s
Received healthy response to inference request in 4.095912933349609s
Received healthy response to inference request in 2.038285493850708s
Received healthy response to inference request in 2.384312391281128s
Received healthy response to inference request in 2.2824742794036865s
Received healthy response to inference request in 2.100215435028076s
Received healthy response to inference request in 2.0578672885894775s
Received healthy response to inference request in 2.0720598697662354s
Received healthy response to inference request in 2.068073034286499s
Received healthy response to inference request in 2.3052544593811035s
Received healthy response to inference request in 2.5099642276763916s
Received healthy response to inference request in 2.047544479370117s
Received healthy response to inference request in 2.0304982662200928s
Received healthy response to inference request in 2.0066769123077393s
Received healthy response to inference request in 1.977001428604126s
Received healthy response to inference request in 2.3348686695098877s
Received healthy response to inference request in 1.8961586952209473s
Received healthy response to inference request in 2.0745468139648438s
Received healthy response to inference request in 2.0080509185791016s
Received healthy response to inference request in 2.340590476989746s
Received healthy response to inference request in 2.1795904636383057s
Received healthy response to inference request in 2.016162395477295s
Received healthy response to inference request in 2.092625379562378s
Received healthy response to inference request in 2.046988010406494s
Received healthy response to inference request in 2.2679998874664307s
Received healthy response to inference request in 1.987682580947876s
Received healthy response to inference request in 2.0127878189086914s
Received healthy response to inference request in 2.0288567543029785s
Received healthy response to inference request in 2.294664144515991s
30 requests
0 failed requests
5th percentile: 1.9818079471588135
10th percentile: 2.0047774791717528
20th percentile: 2.015487480163574
30th percentile: 2.0359493255615235
40th percentile: 2.053738164901733
50th percentile: 2.0733033418655396
60th percentile: 2.131965446472168
70th percentile: 2.286131238937378
80th percentile: 2.3190224170684814
90th percentile: 2.3921266555786134
95th percentile: 2.4885850906372067
99th percentile: 3.6359878087043778
mean time: 2.211174313227336
Pipeline stage StressChecker completed in 70.35s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyTriggerPipeline
run_pipeline:run_in_cloud %s
starting trigger_guanaco_pipeline args=%s
triggered trigger_guanaco_pipeline args=%s
Pipeline stage OfflineFamilyFriendlyTriggerPipeline completed in 0.79s
Shutdown handler de-registered
chaiml-pony-v2-g46-lr1_80834_v21 status is now deployed due to DeploymentManager action
chaiml-pony-v2-g46-lr1_80834_v21 status is now inactive due to auto deactivation removed underperforming models
chaiml-pony-v2-g46-lr1_80834_v21 status is now torndown due to DeploymentManager action