developer_uid: zonemercy
submission_id: chaiml-pony-d3-g46-pv2-_41910_v3
model_name: chaiml-pony-d3-g46-pv2-_41910_v3
model_group: ChaiML/pony-d3-g46-pv2-l
status: torndown
timestamp: 2026-03-09T16:32:19+00:00
num_battles: 10998
num_wins: 6264
celo_rating: 1345.15
family_friendly_score: 0.0
family_friendly_standard_error: 0.0
submission_type: basic
model_repo: ChaiML/pony-d3-g46-pv2-lr5e6ep1r64b8
model_architecture: Glm4MoeForCausalLM
model_num_parameters: 24110003200.0
best_of: 8
max_input_tokens: 2048
max_output_tokens: 80
reward_model: default
display_name: chaiml-pony-d3-g46-pv2-_41910_v3
ineligible_reason: max_output_tokens!=64
is_internal_developer: True
language_model: ChaiML/pony-d3-g46-pv2-lr5e6ep1r64b8
model_size: 24B
ranking_group: single
us_pacific_date: 2026-03-06
win_ratio: 0.5695581014729951
generation_params: {'temperature': 1.0, 'top_p': 0.95, 'min_p': 0.0, 'top_k': 60, 'presence_penalty': 0.0, 'frequency_penalty': 0.0, 'stopping_words': ['<|assistant|>', '<|user|>', '<|im_end|>', '</s>', '####'], 'max_input_tokens': 2048, 'best_of': 8, 'max_output_tokens': 80}
formatter: {'memory_template': "[gMASK]<sop><|system|>\n{bot_name}'s persona: {memory}", 'prompt_template': '', 'bot_template': '<|assistant|>\n<think></think>\n{message}', 'user_template': '<|user|>\n{message}', 'response_template': '<|assistant|>\n<think></think>\n{bot_name}:', 'truncate_by_message': True}
Resubmit model
Shutdown handler not registered because Python interpreter is not running in the main thread
run pipeline %s
run pipeline stage %s
Running pipeline stage VLLMUploader
Starting job with name chaiml-pony-d3-g46-pv2-41910-v3-uploader
Waiting for job on chaiml-pony-d3-g46-pv2-41910-v3-uploader to finish
chaiml-pony-d3-g46-pv2-41910-v3-uploader: Using quantization_mode: fp8
chaiml-pony-d3-g46-pv2-41910-v3-uploader: Checking if ChaiML/pony-d3-g46-pv2-lr5e6ep1r64b8-FP8 already exists in ChaiML
chaiml-pony-d3-g46-pv2-41910-v3-uploader: Model already exists. Downloading to /dev/shm/model_output...
chaiml-pony-d3-g46-pv2-41910-v3-uploader: Downloading snapshot of ChaiML/pony-d3-g46-pv2-lr5e6ep1r64b8-FP8...
chaiml-pony-d3-g46-pv2-41910-v3-uploader: creating bucket guanaco-vllm-models
chaiml-pony-d3-g46-pv2-41910-v3-uploader: /usr/lib/python3/dist-packages/S3/BaseUtils.py:56: SyntaxWarning: invalid escape sequence '\.'
chaiml-pony-d3-g46-pv2-41910-v3-uploader:   RE_S3_DATESTRING = re.compile('\.[0-9]*(?:[Z\\-\\+]*?)')
chaiml-pony-d3-g46-pv2-41910-v3-uploader: /usr/lib/python3/dist-packages/S3/BaseUtils.py:57: SyntaxWarning: invalid escape sequence '\s'
chaiml-pony-d3-g46-pv2-41910-v3-uploader:   RE_XML_NAMESPACE = re.compile(b'^(<?[^>]+?>\s*|\s*)(<\w+) xmlns=[\'"](https?://[^\'"]+)[\'"]', re.MULTILINE)
chaiml-pony-d3-g46-pv2-41910-v3-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:240: SyntaxWarning: invalid escape sequence '\.'
chaiml-pony-d3-g46-pv2-41910-v3-uploader:   invalid = re.search("([^a-z0-9\.-])", bucket, re.UNICODE)
chaiml-pony-d3-g46-pv2-41910-v3-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:244: SyntaxWarning: invalid escape sequence '\.'
chaiml-pony-d3-g46-pv2-41910-v3-uploader:   invalid = re.search("([^A-Za-z0-9\._-])", bucket, re.UNICODE)
chaiml-pony-d3-g46-pv2-41910-v3-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:255: SyntaxWarning: invalid escape sequence '\.'
chaiml-pony-d3-g46-pv2-41910-v3-uploader:   if re.search("-\.", bucket, re.UNICODE):
chaiml-pony-d3-g46-pv2-41910-v3-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:257: SyntaxWarning: invalid escape sequence '\.'
chaiml-pony-d3-g46-pv2-41910-v3-uploader:   if re.search("\.\.", bucket, re.UNICODE):
chaiml-pony-d3-g46-pv2-41910-v3-uploader: /usr/lib/python3/dist-packages/S3/S3Uri.py:155: SyntaxWarning: invalid escape sequence '\w'
chaiml-pony-d3-g46-pv2-41910-v3-uploader:   _re = re.compile("^(\w+://)?(.*)", re.UNICODE)
chaiml-pony-d3-g46-pv2-41910-v3-uploader: /usr/lib/python3/dist-packages/S3/FileLists.py:480: SyntaxWarning: invalid escape sequence '\*'
chaiml-pony-d3-g46-pv2-41910-v3-uploader:   wildcard_split_result = re.split("\*|\?", uri_str, maxsplit=1)
chaiml-pony-d3-g46-pv2-41910-v3-uploader: Bucket 's3://guanaco-vllm-models/' created
chaiml-pony-d3-g46-pv2-41910-v3-uploader: uploading /dev/shm/model_output to s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-41910-v3/default
chaiml-pony-d3-g46-pv2-41910-v3-uploader: cp /dev/shm/model_output/generation_config.json s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-41910-v3/default/generation_config.json
chaiml-pony-d3-g46-pv2-41910-v3-uploader: cp /dev/shm/model_output/chat_template.jinja s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-41910-v3/default/chat_template.jinja
chaiml-pony-d3-g46-pv2-41910-v3-uploader: cp /dev/shm/model_output/.gitattributes s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-41910-v3/default/.gitattributes
chaiml-pony-d3-g46-pv2-41910-v3-uploader: cp /dev/shm/model_output/config.json s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-41910-v3/default/config.json
chaiml-pony-d3-g46-pv2-41910-v3-uploader: cp /dev/shm/model_output/tokenizer_config.json s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-41910-v3/default/tokenizer_config.json
chaiml-pony-d3-g46-pv2-41910-v3-uploader: cp /dev/shm/model_output/special_tokens_map.json s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-41910-v3/default/special_tokens_map.json
chaiml-pony-d3-g46-pv2-41910-v3-uploader: cp /dev/shm/model_output/recipe.yaml s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-41910-v3/default/recipe.yaml
chaiml-pony-d3-g46-pv2-41910-v3-uploader: cp /dev/shm/model_output/model.safetensors.index.json s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-41910-v3/default/model.safetensors.index.json
chaiml-pony-d3-g46-pv2-41910-v3-uploader: cp /dev/shm/model_output/tokenizer.json s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-41910-v3/default/tokenizer.json
chaiml-pony-d3-g46-pv2-41910-v3-uploader: cp /dev/shm/model_output/model-00072-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-41910-v3/default/model-00072-of-00072.safetensors
chaiml-pony-d3-g46-pv2-41910-v3-uploader: cp /dev/shm/model_output/model-00071-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-41910-v3/default/model-00071-of-00072.safetensors
chaiml-pony-d3-g46-pv2-41910-v3-uploader: cp /dev/shm/model_output/model-00022-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-41910-v3/default/model-00022-of-00072.safetensors
chaiml-pony-d3-g46-pv2-41910-v3-uploader: cp /dev/shm/model_output/model-00010-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-41910-v3/default/model-00010-of-00072.safetensors
chaiml-pony-d3-g46-pv2-41910-v3-uploader: cp /dev/shm/model_output/model-00003-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-41910-v3/default/model-00003-of-00072.safetensors
chaiml-pony-d3-g46-pv2-41910-v3-uploader: cp /dev/shm/model_output/model-00012-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-41910-v3/default/model-00012-of-00072.safetensors
chaiml-pony-d3-g46-pv2-41910-v3-uploader: cp /dev/shm/model_output/model-00039-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-41910-v3/default/model-00039-of-00072.safetensors
chaiml-pony-d3-g46-pv2-41910-v3-uploader: cp /dev/shm/model_output/model-00053-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-41910-v3/default/model-00053-of-00072.safetensors
chaiml-pony-d3-g46-pv2-41910-v3-uploader: cp /dev/shm/model_output/model-00057-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-41910-v3/default/model-00057-of-00072.safetensors
chaiml-pony-d3-g46-pv2-41910-v3-uploader: cp /dev/shm/model_output/model-00063-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-41910-v3/default/model-00063-of-00072.safetensors
chaiml-pony-d3-g46-pv2-41910-v3-uploader: cp /dev/shm/model_output/model-00019-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-41910-v3/default/model-00019-of-00072.safetensors
chaiml-pony-d3-g46-pv2-41910-v3-uploader: cp /dev/shm/model_output/model-00046-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-41910-v3/default/model-00046-of-00072.safetensors
chaiml-pony-d3-g46-pv2-41910-v3-uploader: cp /dev/shm/model_output/model-00020-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-41910-v3/default/model-00020-of-00072.safetensors
chaiml-pony-d3-g46-pv2-41910-v3-uploader: cp /dev/shm/model_output/model-00052-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-41910-v3/default/model-00052-of-00072.safetensors
chaiml-pony-d3-g46-pv2-41910-v3-uploader: cp /dev/shm/model_output/model-00016-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-41910-v3/default/model-00016-of-00072.safetensors
chaiml-pony-d3-g46-pv2-41910-v3-uploader: cp /dev/shm/model_output/model-00056-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-41910-v3/default/model-00056-of-00072.safetensors
chaiml-pony-d3-g46-pv2-41910-v3-uploader: cp /dev/shm/model_output/model-00025-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-41910-v3/default/model-00025-of-00072.safetensors
chaiml-pony-d3-g46-pv2-41910-v3-uploader: cp /dev/shm/model_output/model-00036-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-41910-v3/default/model-00036-of-00072.safetensors
chaiml-pony-d3-g46-pv2-41910-v3-uploader: cp /dev/shm/model_output/model-00059-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-41910-v3/default/model-00059-of-00072.safetensors
chaiml-pony-d3-g46-pv2-41910-v3-uploader: cp /dev/shm/model_output/model-00002-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-41910-v3/default/model-00002-of-00072.safetensors
chaiml-pony-d3-g46-pv2-41910-v3-uploader: cp /dev/shm/model_output/model-00055-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-41910-v3/default/model-00055-of-00072.safetensors
chaiml-pony-d3-g46-pv2-41910-v3-uploader: cp /dev/shm/model_output/model-00061-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-41910-v3/default/model-00061-of-00072.safetensors
chaiml-pony-d3-g46-pv2-41910-v3-uploader: cp /dev/shm/model_output/model-00042-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-41910-v3/default/model-00042-of-00072.safetensors
chaiml-pony-d3-g46-pv2-41910-v3-uploader: cp /dev/shm/model_output/model-00065-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-41910-v3/default/model-00065-of-00072.safetensors
chaiml-pony-d3-g46-pv2-41910-v3-uploader: cp /dev/shm/model_output/model-00037-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-41910-v3/default/model-00037-of-00072.safetensors
chaiml-pony-d3-g46-pv2-41910-v3-uploader: cp /dev/shm/model_output/model-00005-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-41910-v3/default/model-00005-of-00072.safetensors
chaiml-pony-d3-g46-pv2-41910-v3-uploader: cp /dev/shm/model_output/model-00044-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-41910-v3/default/model-00044-of-00072.safetensors
chaiml-pony-d3-g46-pv2-41910-v3-uploader: cp /dev/shm/model_output/model-00041-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-41910-v3/default/model-00041-of-00072.safetensors
chaiml-pony-d3-g46-pv2-41910-v3-uploader: cp /dev/shm/model_output/model-00043-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-41910-v3/default/model-00043-of-00072.safetensors
chaiml-pony-d3-g46-pv2-41910-v3-uploader: cp /dev/shm/model_output/model-00007-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-41910-v3/default/model-00007-of-00072.safetensors
chaiml-pony-d3-g46-pv2-41910-v3-uploader: cp /dev/shm/model_output/model-00047-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-41910-v3/default/model-00047-of-00072.safetensors
chaiml-pony-d3-g46-pv2-41910-v3-uploader: cp /dev/shm/model_output/model-00028-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-41910-v3/default/model-00028-of-00072.safetensors
chaiml-pony-d3-g46-pv2-41910-v3-uploader: cp /dev/shm/model_output/model-00049-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-41910-v3/default/model-00049-of-00072.safetensors
chaiml-pony-d3-g46-pv2-41910-v3-uploader: cp /dev/shm/model_output/model-00066-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-41910-v3/default/model-00066-of-00072.safetensors
chaiml-pony-d3-g46-pv2-41910-v3-uploader: cp /dev/shm/model_output/model-00033-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-41910-v3/default/model-00033-of-00072.safetensors
chaiml-pony-d3-g46-pv2-41910-v3-uploader: cp /dev/shm/model_output/model-00004-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-41910-v3/default/model-00004-of-00072.safetensors
chaiml-pony-d3-g46-pv2-41910-v3-uploader: cp /dev/shm/model_output/model-00034-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-41910-v3/default/model-00034-of-00072.safetensors
chaiml-pony-d3-g46-pv2-41910-v3-uploader: cp /dev/shm/model_output/model-00014-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-41910-v3/default/model-00014-of-00072.safetensors
chaiml-pony-d3-g46-pv2-41910-v3-uploader: cp /dev/shm/model_output/model-00040-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-41910-v3/default/model-00040-of-00072.safetensors
chaiml-pony-d3-g46-pv2-41910-v3-uploader: cp /dev/shm/model_output/model-00035-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-41910-v3/default/model-00035-of-00072.safetensors
chaiml-pony-d3-g46-pv2-41910-v3-uploader: cp /dev/shm/model_output/model-00006-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-41910-v3/default/model-00006-of-00072.safetensors
chaiml-pony-d3-g46-pv2-41910-v3-uploader: cp /dev/shm/model_output/model-00032-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-41910-v3/default/model-00032-of-00072.safetensors
chaiml-pony-d3-g46-pv2-41910-v3-uploader: cp /dev/shm/model_output/model-00030-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-41910-v3/default/model-00030-of-00072.safetensors
chaiml-pony-d3-g46-pv2-41910-v3-uploader: cp /dev/shm/model_output/model-00015-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-41910-v3/default/model-00015-of-00072.safetensors
chaiml-pony-d3-g46-pv2-41910-v3-uploader: cp /dev/shm/model_output/model-00048-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-41910-v3/default/model-00048-of-00072.safetensors
chaiml-pony-d3-g46-pv2-41910-v3-uploader: cp /dev/shm/model_output/model-00068-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-41910-v3/default/model-00068-of-00072.safetensors
chaiml-pony-d3-g46-pv2-41910-v3-uploader: cp /dev/shm/model_output/model-00008-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-41910-v3/default/model-00008-of-00072.safetensors
chaiml-pony-d3-g46-pv2-41910-v3-uploader: cp /dev/shm/model_output/model-00062-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-41910-v3/default/model-00062-of-00072.safetensors
chaiml-pony-d3-g46-pv2-41910-v3-uploader: cp /dev/shm/model_output/model-00069-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-41910-v3/default/model-00069-of-00072.safetensors
chaiml-pony-d3-g46-pv2-41910-v3-uploader: cp /dev/shm/model_output/model-00058-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-41910-v3/default/model-00058-of-00072.safetensors
chaiml-pony-d3-g46-pv2-41910-v3-uploader: cp /dev/shm/model_output/model-00024-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-41910-v3/default/model-00024-of-00072.safetensors
chaiml-pony-d3-g46-pv2-41910-v3-uploader: cp /dev/shm/model_output/model-00018-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-41910-v3/default/model-00018-of-00072.safetensors
chaiml-pony-d3-g46-pv2-41910-v3-uploader: cp /dev/shm/model_output/model-00067-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-41910-v3/default/model-00067-of-00072.safetensors
chaiml-pony-d3-g46-pv2-41910-v3-uploader: cp /dev/shm/model_output/model-00064-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-41910-v3/default/model-00064-of-00072.safetensors
chaiml-pony-d3-g46-pv2-41910-v3-uploader: cp /dev/shm/model_output/model-00070-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-41910-v3/default/model-00070-of-00072.safetensors
chaiml-pony-d3-g46-pv2-41910-v3-uploader: cp /dev/shm/model_output/model-00013-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-41910-v3/default/model-00013-of-00072.safetensors
chaiml-pony-d3-g46-pv2-41910-v3-uploader: cp /dev/shm/model_output/model-00017-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-41910-v3/default/model-00017-of-00072.safetensors
chaiml-pony-d3-g46-pv2-41910-v3-uploader: cp /dev/shm/model_output/model-00045-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-41910-v3/default/model-00045-of-00072.safetensors
chaiml-pony-d3-g46-pv2-41910-v3-uploader: cp /dev/shm/model_output/model-00051-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-41910-v3/default/model-00051-of-00072.safetensors
chaiml-pony-d3-g46-pv2-41910-v3-uploader: cp /dev/shm/model_output/model-00050-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-pv2-41910-v3/default/model-00050-of-00072.safetensors
Job chaiml-pony-d3-g46-pv2-41910-v3-uploader completed after 279.66s with status: succeeded
Stopping job with name chaiml-pony-d3-g46-pv2-41910-v3-uploader
Pipeline stage VLLMUploader completed in 280.35s
run pipeline stage %s
Running pipeline stage VLLMTemplater
Pipeline stage VLLMTemplater completed in 1.12s
run pipeline stage %s
Running pipeline stage VLLMDeployer
Creating inference service chaiml-pony-d3-g46-pv2-41910-v3
Waiting for inference service chaiml-pony-d3-g46-pv2-41910-v3 to be ready
Inference service chaiml-pony-d3-g46-pv2-41910-v3 ready after 612.7192516326904s
Pipeline stage VLLMDeployer completed in 613.47s
run pipeline stage %s
Running pipeline stage StressChecker
Received healthy response to inference request in 2.2439448833465576s
Received healthy response to inference request in 2.1608033180236816s
Received healthy response to inference request in 2.0342471599578857s
Received healthy response to inference request in 2.143216848373413s
Received healthy response to inference request in 2.138030767440796s
Received healthy response to inference request in 2.1647958755493164s
Received healthy response to inference request in 2.0314784049987793s
Received healthy response to inference request in 2.1656322479248047s
Received healthy response to inference request in 2.1178970336914062s
Received healthy response to inference request in 2.2312681674957275s
Received healthy response to inference request in 2.242431879043579s
Received healthy response to inference request in 2.326158285140991s
Received healthy response to inference request in 2.1311521530151367s
Received healthy response to inference request in 2.256105661392212s
Received healthy response to inference request in 2.6219913959503174s
Received healthy response to inference request in 2.121034622192383s
Received healthy response to inference request in 2.1429405212402344s
Received healthy response to inference request in 2.1467878818511963s
Received healthy response to inference request in 2.4996554851531982s
Received healthy response to inference request in 2.056234121322632s
Received healthy response to inference request in 2.3360438346862793s
Received healthy response to inference request in 2.326016902923584s
Received healthy response to inference request in 2.04233717918396s
Received healthy response to inference request in 2.09213924407959s
Received healthy response to inference request in 2.157503604888916s
Received healthy response to inference request in 2.7640585899353027s
Received healthy response to inference request in 2.2562150955200195s
Received healthy response to inference request in 2.0935537815093994s
Received healthy response to inference request in 2.2266182899475098s
Received healthy response to inference request in 2.265958547592163s
30 requests
0 failed requests
5th percentile: 2.0378876686096192
10th percentile: 2.0548444271087645
20th percentile: 2.113028383255005
30th percentile: 2.135967183113098
40th percentile: 2.145359468460083
50th percentile: 2.162799596786499
60th percentile: 2.2284782409667967
70th percentile: 2.247593116760254
80th percentile: 2.2779702186584476
90th percentile: 2.3524049997329715
95th percentile: 2.5669402360916136
99th percentile: 2.722859103679657
mean time: 2.217875059445699
Pipeline stage StressChecker completed in 72.06s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyTriggerPipeline
run_pipeline:run_in_cloud %s
starting trigger_guanaco_pipeline args=%s
triggered trigger_guanaco_pipeline args=%s
Pipeline stage OfflineFamilyFriendlyTriggerPipeline completed in 1.41s
Shutdown handler de-registered
chaiml-pony-d3-g46-pv2-_41910_v3 status is now deployed due to DeploymentManager action
chaiml-pony-d3-g46-pv2-_41910_v3 status is now inactive due to auto deactivation removed underperforming models
Deleting key chaiml-pony-d3-g46-pv2-64172-v2/default/tokenizer.json from bucket guanaco-vllm-models
chaiml-pony-d3-g46-pv2-_41910_v3 status is now torndown due to DeploymentManager action