chaiml-pony-d2-g46-o4-l_42687

developer_uid: zonemercy
submission_id: chaiml-pony-d2-g46-o4-l_42687_v1
model_name: chaiml-pony-d2-g46-o4-l_42687_v1
model_group: ChaiML/pony-d2-g46-o4-lr
status: torndown
timestamp: 2026-03-08T07:53:54+00:00
num_battles: 10873
num_wins: 6248
celo_rating: 1350.11
family_friendly_score: 0.0
family_friendly_standard_error: 0.0
submission_type: basic
model_repo: ChaiML/pony-d2-g46-o4-lr5e6ep2r64b8
model_architecture: Glm4MoeForCausalLM
model_num_parameters: 24110003200.0
best_of: 8
max_input_tokens: 2048
max_output_tokens: 80
reward_model: default
display_name: chaiml-pony-d2-g46-o4-l_42687_v1
ineligible_reason: max_output_tokens!=64
is_internal_developer: True
language_model: ChaiML/pony-d2-g46-o4-lr5e6ep2r64b8
model_size: 24B
ranking_group: single
us_pacific_date: 2026-03-04
win_ratio: 0.5746344155246942
generation_params: {'temperature': 1.0, 'top_p': 1.0, 'min_p': 0.0, 'top_k': 40, 'presence_penalty': 0.0, 'frequency_penalty': 0.0, 'stopping_words': ['<|assistant|>', '<|user|>', '<|im_end|>', '</s>', '####'], 'max_input_tokens': 2048, 'best_of': 8, 'max_output_tokens': 80}
formatter: {'memory_template': "[gMASK]<sop><|system|>\n{bot_name}'s persona: {memory}", 'prompt_template': '', 'bot_template': '<|assistant|>\n<think></think>\n{message}', 'user_template': '<|user|>\n{message}', 'response_template': '<|assistant|>\n<think></think>\n{bot_name}:', 'truncate_by_message': True}
Resubmit model
Shutdown handler not registered because Python interpreter is not running in the main thread
run pipeline %s
run pipeline stage %s
Running pipeline stage VLLMUploader
Starting job with name chaiml-pony-d2-g46-o4-l-42687-v1-uploader
Waiting for job on chaiml-pony-d2-g46-o4-l-42687-v1-uploader to finish
chaiml-pony-d2-g46-o4-l-42687-v1-uploader: Using quantization_mode: fp8
chaiml-pony-d2-g46-o4-l-42687-v1-uploader: Checking if ChaiML/pony-d2-g46-o4-lr5e6ep2r64b8-FP8 already exists in ChaiML
chaiml-pony-d2-g46-o4-l-42687-v1-uploader: Model already exists. Downloading to /dev/shm/model_output...
chaiml-pony-d2-g46-o4-l-42687-v1-uploader: Downloading snapshot of ChaiML/pony-d2-g46-o4-lr5e6ep2r64b8-FP8...
chaiml-pony-d2-g46-o4-l-42687-v1-uploader: Downloaded in 138.608s
chaiml-pony-d2-g46-o4-l-42687-v1-uploader: Processed model ChaiML/pony-d2-g46-o4-lr5e6ep2r64b8 in 142.162s
chaiml-pony-d2-g46-o4-l-42687-v1-uploader: creating bucket guanaco-vllm-models
chaiml-pony-d2-g46-o4-l-42687-v1-uploader: /usr/lib/python3/dist-packages/S3/BaseUtils.py:56: SyntaxWarning: invalid escape sequence '\.'
chaiml-pony-d2-g46-o4-l-42687-v1-uploader:   RE_S3_DATESTRING = re.compile('\.[0-9]*(?:[Z\\-\\+]*?)')
chaiml-pony-d2-g46-o4-l-42687-v1-uploader: /usr/lib/python3/dist-packages/S3/BaseUtils.py:57: SyntaxWarning: invalid escape sequence '\s'
chaiml-pony-d2-g46-o4-l-42687-v1-uploader:   RE_XML_NAMESPACE = re.compile(b'^(<?[^>]+?>\s*|\s*)(<\w+) xmlns=[\'"](https?://[^\'"]+)[\'"]', re.MULTILINE)
chaiml-pony-d2-g46-o4-l-42687-v1-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:240: SyntaxWarning: invalid escape sequence '\.'
chaiml-pony-d2-g46-o4-l-42687-v1-uploader:   invalid = re.search("([^a-z0-9\.-])", bucket, re.UNICODE)
chaiml-pony-d2-g46-o4-l-42687-v1-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:244: SyntaxWarning: invalid escape sequence '\.'
chaiml-pony-d2-g46-o4-l-42687-v1-uploader:   invalid = re.search("([^A-Za-z0-9\._-])", bucket, re.UNICODE)
chaiml-pony-d2-g46-o4-l-42687-v1-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:255: SyntaxWarning: invalid escape sequence '\.'
chaiml-pony-d2-g46-o4-l-42687-v1-uploader:   if re.search("-\.", bucket, re.UNICODE):
chaiml-pony-d2-g46-o4-l-42687-v1-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:257: SyntaxWarning: invalid escape sequence '\.'
chaiml-pony-d2-g46-o4-l-42687-v1-uploader:   if re.search("\.\.", bucket, re.UNICODE):
chaiml-pony-d2-g46-o4-l-42687-v1-uploader: /usr/lib/python3/dist-packages/S3/S3Uri.py:155: SyntaxWarning: invalid escape sequence '\w'
chaiml-pony-d2-g46-o4-l-42687-v1-uploader:   _re = re.compile("^(\w+://)?(.*)", re.UNICODE)
chaiml-pony-d2-g46-o4-l-42687-v1-uploader: /usr/lib/python3/dist-packages/S3/FileLists.py:480: SyntaxWarning: invalid escape sequence '\*'
chaiml-pony-d2-g46-o4-l-42687-v1-uploader:   wildcard_split_result = re.split("\*|\?", uri_str, maxsplit=1)
chaiml-pony-d2-g46-o4-l-42687-v1-uploader: Bucket 's3://guanaco-vllm-models/' created
chaiml-pony-d2-g46-o4-l-42687-v1-uploader: uploading /dev/shm/model_output to s3://guanaco-vllm-models/chaiml-pony-d2-g46-o4-l-42687-v1/default
chaiml-pony-d2-g46-o4-l-42687-v1-uploader: cp /dev/shm/model_output/generation_config.json s3://guanaco-vllm-models/chaiml-pony-d2-g46-o4-l-42687-v1/default/generation_config.json
chaiml-pony-d2-g46-o4-l-42687-v1-uploader: cp /dev/shm/model_output/chat_template.jinja s3://guanaco-vllm-models/chaiml-pony-d2-g46-o4-l-42687-v1/default/chat_template.jinja
chaiml-pony-d2-g46-o4-l-42687-v1-uploader: cp /dev/shm/model_output/.gitattributes s3://guanaco-vllm-models/chaiml-pony-d2-g46-o4-l-42687-v1/default/.gitattributes
chaiml-pony-d2-g46-o4-l-42687-v1-uploader: cp /dev/shm/model_output/config.json s3://guanaco-vllm-models/chaiml-pony-d2-g46-o4-l-42687-v1/default/config.json
chaiml-pony-d2-g46-o4-l-42687-v1-uploader: cp /dev/shm/model_output/recipe.yaml s3://guanaco-vllm-models/chaiml-pony-d2-g46-o4-l-42687-v1/default/recipe.yaml
chaiml-pony-d2-g46-o4-l-42687-v1-uploader: cp /dev/shm/model_output/special_tokens_map.json s3://guanaco-vllm-models/chaiml-pony-d2-g46-o4-l-42687-v1/default/special_tokens_map.json
chaiml-pony-d2-g46-o4-l-42687-v1-uploader: cp /dev/shm/model_output/tokenizer_config.json s3://guanaco-vllm-models/chaiml-pony-d2-g46-o4-l-42687-v1/default/tokenizer_config.json
chaiml-pony-d2-g46-o4-l-42687-v1-uploader: cp /dev/shm/model_output/model.safetensors.index.json s3://guanaco-vllm-models/chaiml-pony-d2-g46-o4-l-42687-v1/default/model.safetensors.index.json
chaiml-pony-d2-g46-o4-l-42687-v1-uploader: cp /dev/shm/model_output/tokenizer.json s3://guanaco-vllm-models/chaiml-pony-d2-g46-o4-l-42687-v1/default/tokenizer.json
chaiml-pony-d2-g46-o4-l-42687-v1-uploader: cp /dev/shm/model_output/model-00072-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d2-g46-o4-l-42687-v1/default/model-00072-of-00072.safetensors
chaiml-pony-d2-g46-o4-l-42687-v1-uploader: cp /dev/shm/model_output/model-00071-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d2-g46-o4-l-42687-v1/default/model-00071-of-00072.safetensors
chaiml-pony-d2-g46-o4-l-42687-v1-uploader: cp /dev/shm/model_output/model-00028-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d2-g46-o4-l-42687-v1/default/model-00028-of-00072.safetensors
chaiml-pony-d2-g46-o4-l-42687-v1-uploader: cp /dev/shm/model_output/model-00003-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d2-g46-o4-l-42687-v1/default/model-00003-of-00072.safetensors
chaiml-pony-d2-g46-o4-l-42687-v1-uploader: cp /dev/shm/model_output/model-00039-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d2-g46-o4-l-42687-v1/default/model-00039-of-00072.safetensors
chaiml-pony-d2-g46-o4-l-42687-v1-uploader: cp /dev/shm/model_output/model-00058-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d2-g46-o4-l-42687-v1/default/model-00058-of-00072.safetensors
chaiml-pony-d2-g46-o4-l-42687-v1-uploader: cp /dev/shm/model_output/model-00020-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d2-g46-o4-l-42687-v1/default/model-00020-of-00072.safetensors
chaiml-pony-d2-g46-o4-l-42687-v1-uploader: cp /dev/shm/model_output/model-00050-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d2-g46-o4-l-42687-v1/default/model-00050-of-00072.safetensors
chaiml-pony-d2-g46-o4-l-42687-v1-uploader: cp /dev/shm/model_output/model-00034-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d2-g46-o4-l-42687-v1/default/model-00034-of-00072.safetensors
chaiml-pony-d2-g46-o4-l-42687-v1-uploader: cp /dev/shm/model_output/model-00036-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d2-g46-o4-l-42687-v1/default/model-00036-of-00072.safetensors
chaiml-pony-d2-g46-o4-l-42687-v1-uploader: cp /dev/shm/model_output/model-00026-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d2-g46-o4-l-42687-v1/default/model-00026-of-00072.safetensors
chaiml-pony-d2-g46-o4-l-42687-v1-uploader: cp /dev/shm/model_output/model-00018-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d2-g46-o4-l-42687-v1/default/model-00018-of-00072.safetensors
chaiml-pony-d2-g46-o4-l-42687-v1-uploader: cp /dev/shm/model_output/model-00021-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d2-g46-o4-l-42687-v1/default/model-00021-of-00072.safetensors
chaiml-pony-d2-g46-o4-l-42687-v1-uploader: cp /dev/shm/model_output/model-00002-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d2-g46-o4-l-42687-v1/default/model-00002-of-00072.safetensors
chaiml-pony-d2-g46-o4-l-42687-v1-uploader: cp /dev/shm/model_output/model-00007-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d2-g46-o4-l-42687-v1/default/model-00007-of-00072.safetensors
chaiml-pony-d2-g46-o4-l-42687-v1-uploader: cp /dev/shm/model_output/model-00024-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d2-g46-o4-l-42687-v1/default/model-00024-of-00072.safetensors
chaiml-pony-d2-g46-o4-l-42687-v1-uploader: cp /dev/shm/model_output/model-00045-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d2-g46-o4-l-42687-v1/default/model-00045-of-00072.safetensors
chaiml-pony-d2-g46-o4-l-42687-v1-uploader: cp /dev/shm/model_output/model-00041-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d2-g46-o4-l-42687-v1/default/model-00041-of-00072.safetensors
chaiml-pony-d2-g46-o4-l-42687-v1-uploader: cp /dev/shm/model_output/model-00009-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d2-g46-o4-l-42687-v1/default/model-00009-of-00072.safetensors
chaiml-pony-d2-g46-o4-l-42687-v1-uploader: cp /dev/shm/model_output/model-00029-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d2-g46-o4-l-42687-v1/default/model-00029-of-00072.safetensors
chaiml-pony-d2-g46-o4-l-42687-v1-uploader: cp /dev/shm/model_output/model-00040-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d2-g46-o4-l-42687-v1/default/model-00040-of-00072.safetensors
chaiml-pony-d2-g46-o4-l-42687-v1-uploader: cp /dev/shm/model_output/model-00044-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d2-g46-o4-l-42687-v1/default/model-00044-of-00072.safetensors
chaiml-pony-d2-g46-o4-l-42687-v1-uploader: cp /dev/shm/model_output/model-00069-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d2-g46-o4-l-42687-v1/default/model-00069-of-00072.safetensors
chaiml-pony-d2-g46-o4-l-42687-v1-uploader: cp /dev/shm/model_output/model-00037-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d2-g46-o4-l-42687-v1/default/model-00037-of-00072.safetensors
chaiml-pony-d2-g46-o4-l-42687-v1-uploader: cp /dev/shm/model_output/model-00048-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d2-g46-o4-l-42687-v1/default/model-00048-of-00072.safetensors
chaiml-pony-d2-g46-o4-l-42687-v1-uploader: cp /dev/shm/model_output/model-00049-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d2-g46-o4-l-42687-v1/default/model-00049-of-00072.safetensors
chaiml-pony-d2-g46-o4-l-42687-v1-uploader: cp /dev/shm/model_output/model-00032-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d2-g46-o4-l-42687-v1/default/model-00032-of-00072.safetensors
chaiml-pony-d2-g46-o4-l-42687-v1-uploader: cp /dev/shm/model_output/model-00023-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d2-g46-o4-l-42687-v1/default/model-00023-of-00072.safetensors
chaiml-pony-d2-g46-o4-l-42687-v1-uploader: cp /dev/shm/model_output/model-00064-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d2-g46-o4-l-42687-v1/default/model-00064-of-00072.safetensors
chaiml-pony-d2-g46-o4-l-42687-v1-uploader: cp /dev/shm/model_output/model-00015-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d2-g46-o4-l-42687-v1/default/model-00015-of-00072.safetensors
chaiml-pony-d2-g46-o4-l-42687-v1-uploader: cp /dev/shm/model_output/model-00025-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d2-g46-o4-l-42687-v1/default/model-00025-of-00072.safetensors
chaiml-pony-d2-g46-o4-l-42687-v1-uploader: cp /dev/shm/model_output/model-00019-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d2-g46-o4-l-42687-v1/default/model-00019-of-00072.safetensors
chaiml-pony-d2-g46-o4-l-42687-v1-uploader: cp /dev/shm/model_output/model-00012-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d2-g46-o4-l-42687-v1/default/model-00012-of-00072.safetensors
chaiml-pony-d2-g46-o4-l-42687-v1-uploader: cp /dev/shm/model_output/model-00066-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d2-g46-o4-l-42687-v1/default/model-00066-of-00072.safetensors
chaiml-pony-d2-g46-o4-l-42687-v1-uploader: cp /dev/shm/model_output/model-00030-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d2-g46-o4-l-42687-v1/default/model-00030-of-00072.safetensors
chaiml-pony-d2-g46-o4-l-42687-v1-uploader: cp /dev/shm/model_output/model-00008-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d2-g46-o4-l-42687-v1/default/model-00008-of-00072.safetensors
chaiml-pony-d2-g46-o4-l-42687-v1-uploader: cp /dev/shm/model_output/model-00033-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d2-g46-o4-l-42687-v1/default/model-00033-of-00072.safetensors
chaiml-pony-d2-g46-o4-l-42687-v1-uploader: cp /dev/shm/model_output/model-00004-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d2-g46-o4-l-42687-v1/default/model-00004-of-00072.safetensors
chaiml-pony-d2-g46-o4-l-42687-v1-uploader: cp /dev/shm/model_output/model-00051-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d2-g46-o4-l-42687-v1/default/model-00051-of-00072.safetensors
chaiml-pony-d2-g46-o4-l-42687-v1-uploader: cp /dev/shm/model_output/model-00046-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d2-g46-o4-l-42687-v1/default/model-00046-of-00072.safetensors
chaiml-pony-d2-g46-o4-l-42687-v1-uploader: cp /dev/shm/model_output/model-00031-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d2-g46-o4-l-42687-v1/default/model-00031-of-00072.safetensors
chaiml-pony-d2-g46-o4-l-42687-v1-uploader: cp /dev/shm/model_output/model-00035-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d2-g46-o4-l-42687-v1/default/model-00035-of-00072.safetensors
chaiml-pony-d2-g46-o4-l-42687-v1-uploader: cp /dev/shm/model_output/model-00063-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d2-g46-o4-l-42687-v1/default/model-00063-of-00072.safetensors
chaiml-pony-d2-g46-o4-l-42687-v1-uploader: cp /dev/shm/model_output/model-00017-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d2-g46-o4-l-42687-v1/default/model-00017-of-00072.safetensors
chaiml-pony-d2-g46-o4-l-42687-v1-uploader: cp /dev/shm/model_output/model-00011-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d2-g46-o4-l-42687-v1/default/model-00011-of-00072.safetensors
chaiml-pony-d2-g46-o4-l-42687-v1-uploader: cp /dev/shm/model_output/model-00005-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d2-g46-o4-l-42687-v1/default/model-00005-of-00072.safetensors
chaiml-pony-d2-g46-o4-l-42687-v1-uploader: cp /dev/shm/model_output/model-00056-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d2-g46-o4-l-42687-v1/default/model-00056-of-00072.safetensors
chaiml-pony-d2-g46-o4-l-42687-v1-uploader: cp /dev/shm/model_output/model-00070-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d2-g46-o4-l-42687-v1/default/model-00070-of-00072.safetensors
chaiml-pony-d2-g46-o4-l-42687-v1-uploader: cp /dev/shm/model_output/model-00060-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d2-g46-o4-l-42687-v1/default/model-00060-of-00072.safetensors
chaiml-pony-d2-g46-o4-l-42687-v1-uploader: cp /dev/shm/model_output/model-00027-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d2-g46-o4-l-42687-v1/default/model-00027-of-00072.safetensors
chaiml-pony-d2-g46-o4-l-42687-v1-uploader: cp /dev/shm/model_output/model-00006-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d2-g46-o4-l-42687-v1/default/model-00006-of-00072.safetensors
chaiml-pony-d2-g46-o4-l-42687-v1-uploader: cp /dev/shm/model_output/model-00054-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d2-g46-o4-l-42687-v1/default/model-00054-of-00072.safetensors
chaiml-pony-d2-g46-o4-l-42687-v1-uploader: cp /dev/shm/model_output/model-00057-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d2-g46-o4-l-42687-v1/default/model-00057-of-00072.safetensors
chaiml-pony-d2-g46-o4-l-42687-v1-uploader: cp /dev/shm/model_output/model-00016-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d2-g46-o4-l-42687-v1/default/model-00016-of-00072.safetensors
chaiml-pony-d2-g46-o4-l-42687-v1-uploader: cp /dev/shm/model_output/model-00001-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d2-g46-o4-l-42687-v1/default/model-00001-of-00072.safetensors
chaiml-pony-d2-g46-o4-l-42687-v1-uploader: cp /dev/shm/model_output/model-00042-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d2-g46-o4-l-42687-v1/default/model-00042-of-00072.safetensors
chaiml-pony-d2-g46-o4-l-42687-v1-uploader: cp /dev/shm/model_output/model-00014-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d2-g46-o4-l-42687-v1/default/model-00014-of-00072.safetensors
chaiml-pony-d2-g46-o4-l-42687-v1-uploader: cp /dev/shm/model_output/model-00068-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d2-g46-o4-l-42687-v1/default/model-00068-of-00072.safetensors
chaiml-pony-d2-g46-o4-l-42687-v1-uploader: cp /dev/shm/model_output/model-00061-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d2-g46-o4-l-42687-v1/default/model-00061-of-00072.safetensors
chaiml-pony-d2-g46-o4-l-42687-v1-uploader: cp /dev/shm/model_output/model-00059-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d2-g46-o4-l-42687-v1/default/model-00059-of-00072.safetensors
chaiml-pony-d2-g46-o4-l-42687-v1-uploader: cp /dev/shm/model_output/model-00052-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d2-g46-o4-l-42687-v1/default/model-00052-of-00072.safetensors
chaiml-pony-d2-g46-o4-l-42687-v1-uploader: cp /dev/shm/model_output/model-00062-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d2-g46-o4-l-42687-v1/default/model-00062-of-00072.safetensors
Job chaiml-pony-d2-g46-o4-l-42687-v1-uploader completed after 279.15s with status: succeeded
Stopping job with name chaiml-pony-d2-g46-o4-l-42687-v1-uploader
Pipeline stage VLLMUploader completed in 279.73s
run pipeline stage %s
Running pipeline stage VLLMTemplater
Pipeline stage VLLMTemplater completed in 0.77s
run pipeline stage %s
Running pipeline stage VLLMDeployer
Creating inference service chaiml-pony-d2-g46-o4-l-42687-v1
Waiting for inference service chaiml-pony-d2-g46-o4-l-42687-v1 to be ready
Inference service chaiml-pony-d2-g46-o4-l-42687-v1 ready after 631.1298522949219s
Pipeline stage VLLMDeployer completed in 636.64s
run pipeline stage %s
Running pipeline stage StressChecker
Received healthy response to inference request in 2.212301015853882s
Received healthy response to inference request in 3.102065324783325s
Received healthy response to inference request in 2.0748465061187744s
Received healthy response to inference request in 2.3234729766845703s
Received healthy response to inference request in 1.9960625171661377s
Received healthy response to inference request in 2.323888063430786s
Received healthy response to inference request in 2.0013108253479004s
Received healthy response to inference request in 2.0046095848083496s
Received healthy response to inference request in 2.052694320678711s
Received healthy response to inference request in 2.23391056060791s
Received healthy response to inference request in 1.9518859386444092s
Received healthy response to inference request in 1.9737675189971924s
Received healthy response to inference request in 2.2451860904693604s
Received healthy response to inference request in 2.18019962310791s
Received healthy response to inference request in 2.1061058044433594s
Received healthy response to inference request in 1.9924368858337402s
Received healthy response to inference request in 2.0625357627868652s
Received healthy response to inference request in 2.444840908050537s
Received healthy response to inference request in 1.9778146743774414s
Received healthy response to inference request in 2.0399529933929443s
Received healthy response to inference request in 2.6562023162841797s
Received healthy response to inference request in 2.43009877204895s
Received healthy response to inference request in 2.518828868865967s
Received healthy response to inference request in 2.1447184085845947s
Received healthy response to inference request in 2.3361356258392334s
Received healthy response to inference request in 2.377065658569336s
Received healthy response to inference request in 2.9096872806549072s
Received healthy response to inference request in 2.107229709625244s
Received healthy response to inference request in 2.247556686401367s
Received healthy response to inference request in 2.028275489807129s
30 requests
0 failed requests
5th percentile: 1.9755887389183044
10th percentile: 1.9909746646881104
20th percentile: 2.00394983291626
30th percentile: 2.048871922492981
40th percentile: 2.0936020851135253
50th percentile: 2.1624590158462524
60th percentile: 2.23842077255249
70th percentile: 2.3235975027084352
80th percentile: 2.387672281265259
90th percentile: 2.5325662136077884
95th percentile: 2.795619046688079
99th percentile: 3.046275691986084
mean time: 2.2351895570755005
Pipeline stage StressChecker completed in 73.53s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyTriggerPipeline
run_pipeline:run_in_cloud %s
starting trigger_guanaco_pipeline args=%s
triggered trigger_guanaco_pipeline args=%s
Pipeline stage OfflineFamilyFriendlyTriggerPipeline completed in 0.72s
Shutdown handler de-registered
chaiml-pony-d2-g46-o4-l_42687_v1 status is now deployed due to DeploymentManager action
chaiml-pony-d2-g46-o4-l_42687_v1 status is now inactive due to auto deactivation removed underperforming models
chaiml-pony-d2-g46-o4-l_42687_v1 status is now torndown due to DeploymentManager action