developer_uid: zonemercy
submission_id: chaiml-pony-v2-g46-lr1_80834_v23
model_name: chaiml-pony-v2-g46-lr1_80834_v23
model_group: ChaiML/pony-v2-g46-lr1e4
status: torndown
timestamp: 2026-03-08T03:42:39+00:00
num_battles: 10519
num_wins: 5993
celo_rating: 1345.13
family_friendly_score: 0.0
family_friendly_standard_error: 0.0
submission_type: basic
model_repo: ChaiML/pony-v2-g46-lr1e4ep1r64b16
model_architecture: Glm4MoeForCausalLM
model_num_parameters: 24110003200.0
best_of: 8
max_input_tokens: 2048
max_output_tokens: 80
reward_model: default
display_name: chaiml-pony-v2-g46-lr1_80834_v23
ineligible_reason: max_output_tokens!=64
is_internal_developer: True
language_model: ChaiML/pony-v2-g46-lr1e4ep1r64b16
model_size: 24B
ranking_group: single
us_pacific_date: 2026-03-04
win_ratio: 0.5697309630192984
generation_params: {'temperature': 1.0, 'top_p': 1.0, 'min_p': 0.0, 'top_k': 40, 'presence_penalty': 0.0, 'frequency_penalty': 0.0, 'stopping_words': ['<|assistant|>', '<|user|>', '<|im_end|>', '</s>', '####'], 'max_input_tokens': 2048, 'best_of': 8, 'max_output_tokens': 80}
formatter: {'memory_template': "[gMASK]<sop><|system|>\n{bot_name}'s persona: {memory}.<|user|>\n", 'prompt_template': '', 'bot_template': '{bot_name}: {message}\n', 'user_template': '{user_name}: {message}\n', 'response_template': '<|assistant|>\n<think></think>\n{bot_name}:', 'truncate_by_message': True}
Resubmit model
Shutdown handler not registered because Python interpreter is not running in the main thread
run pipeline %s
run pipeline stage %s
Running pipeline stage VLLMUploader
Starting job with name chaiml-pony-v2-g46-lr1-80834-v23-uploader
Waiting for job on chaiml-pony-v2-g46-lr1-80834-v23-uploader to finish
chaiml-pony-v2-g46-lr1-80834-v23-uploader: Using quantization_mode: fp8
chaiml-pony-v2-g46-lr1-80834-v23-uploader: Checking if ChaiML/pony-v2-g46-lr1e4ep1r64b16-FP8 already exists in ChaiML
chaiml-pony-v2-g46-lr1-80834-v23-uploader: Model already exists. Downloading to /dev/shm/model_output...
chaiml-pony-v2-g46-lr1-80834-v23-uploader: Downloading snapshot of ChaiML/pony-v2-g46-lr1e4ep1r64b16-FP8...
chaiml-pony-v2-g46-lr1-80834-v23-uploader: Downloaded in 124.824s
chaiml-pony-v2-g46-lr1-80834-v23-uploader: Processed model ChaiML/pony-v2-g46-lr1e4ep1r64b16 in 128.361s
chaiml-pony-v2-g46-lr1-80834-v23-uploader: creating bucket guanaco-vllm-models
chaiml-pony-v2-g46-lr1-80834-v23-uploader: /usr/lib/python3/dist-packages/S3/BaseUtils.py:56: SyntaxWarning: invalid escape sequence '\.'
chaiml-pony-v2-g46-lr1-80834-v23-uploader:   RE_S3_DATESTRING = re.compile('\.[0-9]*(?:[Z\\-\\+]*?)')
chaiml-pony-v2-g46-lr1-80834-v23-uploader: /usr/lib/python3/dist-packages/S3/BaseUtils.py:57: SyntaxWarning: invalid escape sequence '\s'
chaiml-pony-v2-g46-lr1-80834-v23-uploader:   RE_XML_NAMESPACE = re.compile(b'^(<?[^>]+?>\s*|\s*)(<\w+) xmlns=[\'"](https?://[^\'"]+)[\'"]', re.MULTILINE)
chaiml-pony-v2-g46-lr1-80834-v23-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:240: SyntaxWarning: invalid escape sequence '\.'
chaiml-pony-v2-g46-lr1-80834-v23-uploader:   invalid = re.search("([^a-z0-9\.-])", bucket, re.UNICODE)
chaiml-pony-v2-g46-lr1-80834-v23-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:244: SyntaxWarning: invalid escape sequence '\.'
chaiml-pony-v2-g46-lr1-80834-v23-uploader:   invalid = re.search("([^A-Za-z0-9\._-])", bucket, re.UNICODE)
chaiml-pony-v2-g46-lr1-80834-v23-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:255: SyntaxWarning: invalid escape sequence '\.'
chaiml-pony-v2-g46-lr1-80834-v23-uploader:   if re.search("-\.", bucket, re.UNICODE):
chaiml-pony-v2-g46-lr1-80834-v23-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:257: SyntaxWarning: invalid escape sequence '\.'
chaiml-pony-v2-g46-lr1-80834-v23-uploader:   if re.search("\.\.", bucket, re.UNICODE):
chaiml-pony-v2-g46-lr1-80834-v23-uploader: /usr/lib/python3/dist-packages/S3/S3Uri.py:155: SyntaxWarning: invalid escape sequence '\w'
chaiml-pony-v2-g46-lr1-80834-v23-uploader:   _re = re.compile("^(\w+://)?(.*)", re.UNICODE)
chaiml-pony-v2-g46-lr1-80834-v23-uploader: /usr/lib/python3/dist-packages/S3/FileLists.py:480: SyntaxWarning: invalid escape sequence '\*'
chaiml-pony-v2-g46-lr1-80834-v23-uploader:   wildcard_split_result = re.split("\*|\?", uri_str, maxsplit=1)
chaiml-pony-v2-g46-lr1-80834-v23-uploader: Bucket 's3://guanaco-vllm-models/' created
chaiml-pony-v2-g46-lr1-80834-v23-uploader: uploading /dev/shm/model_output to s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v23/default
chaiml-pony-v2-g46-lr1-80834-v23-uploader: cp /dev/shm/model_output/generation_config.json s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v23/default/generation_config.json
chaiml-pony-v2-g46-lr1-80834-v23-uploader: cp /dev/shm/model_output/chat_template.jinja s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v23/default/chat_template.jinja
chaiml-pony-v2-g46-lr1-80834-v23-uploader: cp /dev/shm/model_output/recipe.yaml s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v23/default/recipe.yaml
chaiml-pony-v2-g46-lr1-80834-v23-uploader: cp /dev/shm/model_output/model.safetensors.index.json s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v23/default/model.safetensors.index.json
chaiml-pony-v2-g46-lr1-80834-v23-uploader: cp /dev/shm/model_output/config.json s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v23/default/config.json
chaiml-pony-v2-g46-lr1-80834-v23-uploader: cp /dev/shm/model_output/tokenizer.json s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v23/default/tokenizer.json
chaiml-pony-v2-g46-lr1-80834-v23-uploader: cp /dev/shm/model_output/.gitattributes s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v23/default/.gitattributes
chaiml-pony-v2-g46-lr1-80834-v23-uploader: cp /dev/shm/model_output/tokenizer_config.json s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v23/default/tokenizer_config.json
chaiml-pony-v2-g46-lr1-80834-v23-uploader: cp /dev/shm/model_output/special_tokens_map.json s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v23/default/special_tokens_map.json
chaiml-pony-v2-g46-lr1-80834-v23-uploader: cp /dev/shm/model_output/model-00072-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v23/default/model-00072-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v23-uploader: cp /dev/shm/model_output/model-00071-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v23/default/model-00071-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v23-uploader: cp /dev/shm/model_output/model-00018-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v23/default/model-00018-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v23-uploader: cp /dev/shm/model_output/model-00013-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v23/default/model-00013-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v23-uploader: cp /dev/shm/model_output/model-00034-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v23/default/model-00034-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v23-uploader: cp /dev/shm/model_output/model-00048-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v23/default/model-00048-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v23-uploader: cp /dev/shm/model_output/model-00028-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v23/default/model-00028-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v23-uploader: cp /dev/shm/model_output/model-00069-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v23/default/model-00069-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v23-uploader: cp /dev/shm/model_output/model-00007-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v23/default/model-00007-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v23-uploader: cp /dev/shm/model_output/model-00052-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v23/default/model-00052-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v23-uploader: cp /dev/shm/model_output/model-00037-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v23/default/model-00037-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v23-uploader: cp /dev/shm/model_output/model-00044-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v23/default/model-00044-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v23-uploader: cp /dev/shm/model_output/model-00035-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v23/default/model-00035-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v23-uploader: cp /dev/shm/model_output/model-00070-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v23/default/model-00070-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v23-uploader: cp /dev/shm/model_output/model-00012-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v23/default/model-00012-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v23-uploader: cp /dev/shm/model_output/model-00032-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v23/default/model-00032-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v23-uploader: cp /dev/shm/model_output/model-00015-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v23/default/model-00015-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v23-uploader: cp /dev/shm/model_output/model-00043-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v23/default/model-00043-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v23-uploader: cp /dev/shm/model_output/model-00054-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v23/default/model-00054-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v23-uploader: cp /dev/shm/model_output/model-00011-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v23/default/model-00011-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v23-uploader: cp /dev/shm/model_output/model-00019-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v23/default/model-00019-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v23-uploader: cp /dev/shm/model_output/model-00010-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v23/default/model-00010-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v23-uploader: cp /dev/shm/model_output/model-00014-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v23/default/model-00014-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v23-uploader: cp /dev/shm/model_output/model-00029-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v23/default/model-00029-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v23-uploader: cp /dev/shm/model_output/model-00003-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v23/default/model-00003-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v23-uploader: cp /dev/shm/model_output/model-00031-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v23/default/model-00031-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v23-uploader: cp /dev/shm/model_output/model-00046-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v23/default/model-00046-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v23-uploader: cp /dev/shm/model_output/model-00059-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v23/default/model-00059-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v23-uploader: cp /dev/shm/model_output/model-00038-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v23/default/model-00038-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v23-uploader: cp /dev/shm/model_output/model-00008-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v23/default/model-00008-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v23-uploader: cp /dev/shm/model_output/model-00039-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v23/default/model-00039-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v23-uploader: cp /dev/shm/model_output/model-00045-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v23/default/model-00045-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v23-uploader: cp /dev/shm/model_output/model-00062-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v23/default/model-00062-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v23-uploader: cp /dev/shm/model_output/model-00009-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v23/default/model-00009-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v23-uploader: cp /dev/shm/model_output/model-00063-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v23/default/model-00063-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v23-uploader: cp /dev/shm/model_output/model-00058-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v23/default/model-00058-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v23-uploader: cp /dev/shm/model_output/model-00051-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v23/default/model-00051-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v23-uploader: cp /dev/shm/model_output/model-00005-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v23/default/model-00005-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v23-uploader: cp /dev/shm/model_output/model-00023-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v23/default/model-00023-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v23-uploader: cp /dev/shm/model_output/model-00021-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v23/default/model-00021-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v23-uploader: cp /dev/shm/model_output/model-00006-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v23/default/model-00006-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v23-uploader: cp /dev/shm/model_output/model-00068-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v23/default/model-00068-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v23-uploader: cp /dev/shm/model_output/model-00064-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v23/default/model-00064-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v23-uploader: cp /dev/shm/model_output/model-00041-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v23/default/model-00041-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v23-uploader: cp /dev/shm/model_output/model-00002-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v23/default/model-00002-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v23-uploader: cp /dev/shm/model_output/model-00016-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v23/default/model-00016-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v23-uploader: cp /dev/shm/model_output/model-00057-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v23/default/model-00057-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v23-uploader: cp /dev/shm/model_output/model-00042-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v23/default/model-00042-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v23-uploader: cp /dev/shm/model_output/model-00066-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v23/default/model-00066-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v23-uploader: cp /dev/shm/model_output/model-00030-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v23/default/model-00030-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v23-uploader: cp /dev/shm/model_output/model-00027-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v23/default/model-00027-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v23-uploader: cp /dev/shm/model_output/model-00004-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v23/default/model-00004-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v23-uploader: cp /dev/shm/model_output/model-00065-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v23/default/model-00065-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v23-uploader: cp /dev/shm/model_output/model-00022-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v23/default/model-00022-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v23-uploader: cp /dev/shm/model_output/model-00040-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v23/default/model-00040-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v23-uploader: cp /dev/shm/model_output/model-00053-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v23/default/model-00053-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v23-uploader: cp /dev/shm/model_output/model-00033-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v23/default/model-00033-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v23-uploader: cp /dev/shm/model_output/model-00047-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v23/default/model-00047-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v23-uploader: cp /dev/shm/model_output/model-00061-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v23/default/model-00061-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v23-uploader: cp /dev/shm/model_output/model-00060-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v23/default/model-00060-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v23-uploader: cp /dev/shm/model_output/model-00036-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v23/default/model-00036-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v23-uploader: cp /dev/shm/model_output/model-00020-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v23/default/model-00020-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v23-uploader: cp /dev/shm/model_output/model-00017-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v23/default/model-00017-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v23-uploader: cp /dev/shm/model_output/model-00067-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v23/default/model-00067-of-00072.safetensors
chaiml-pony-v2-g46-lr1-80834-v23-uploader: cp /dev/shm/model_output/model-00001-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-v2-g46-lr1-80834-v23/default/model-00001-of-00072.safetensors
Job chaiml-pony-v2-g46-lr1-80834-v23-uploader completed after 288.14s with status: succeeded
Stopping job with name chaiml-pony-v2-g46-lr1-80834-v23-uploader
Pipeline stage VLLMUploader completed in 288.64s
run pipeline stage %s
Running pipeline stage VLLMTemplater
Pipeline stage VLLMTemplater completed in 0.98s
run pipeline stage %s
Running pipeline stage VLLMDeployer
Creating inference service chaiml-pony-v2-g46-lr1-80834-v23
Waiting for inference service chaiml-pony-v2-g46-lr1-80834-v23 to be ready
Inference service chaiml-pony-v2-g46-lr1-80834-v23 ready after 613.2675974369049s
Pipeline stage VLLMDeployer completed in 613.82s
run pipeline stage %s
Running pipeline stage StressChecker
Received healthy response to inference request in 2.493009328842163s
Received healthy response to inference request in 2.1893773078918457s
Received healthy response to inference request in 2.1352357864379883s
Received healthy response to inference request in 2.473761796951294s
Received healthy response to inference request in 2.1349689960479736s
Received healthy response to inference request in 2.123746156692505s
Received healthy response to inference request in 2.012669324874878s
Received healthy response to inference request in 2.060328722000122s
Received healthy response to inference request in 2.0467071533203125s
Received healthy response to inference request in 2.622436285018921s
Received healthy response to inference request in 1.992093563079834s
Received healthy response to inference request in 2.2920453548431396s
Received healthy response to inference request in 2.127974271774292s
Received healthy response to inference request in 2.0820200443267822s
Received healthy response to inference request in 2.0567874908447266s
Received healthy response to inference request in 2.004913330078125s
Received healthy response to inference request in 2.024637460708618s
Received healthy response to inference request in 2.3261396884918213s
Received healthy response to inference request in 2.0738203525543213s
Received healthy response to inference request in 2.25610089302063s
Received healthy response to inference request in 2.365957260131836s
Received healthy response to inference request in 2.0841352939605713s
Received healthy response to inference request in 2.1043217182159424s
Received healthy response to inference request in 2.4570395946502686s
Received healthy response to inference request in 2.4648094177246094s
Received healthy response to inference request in 2.390267848968506s
Received healthy response to inference request in 2.04996395111084s
Received healthy response to inference request in 2.010395050048828s
Received healthy response to inference request in 2.093000650405884s
Received healthy response to inference request in 2.122544288635254s
30 requests
0 failed requests
5th percentile: 2.0073801040649415
10th percentile: 2.012441897392273
20th percentile: 2.0493125915527344
30th percentile: 2.0697728633880614
40th percentile: 2.089454507827759
50th percentile: 2.1231452226638794
60th percentile: 2.1350757122039794
70th percentile: 2.2668842315673827
80th percentile: 2.37081937789917
90th percentile: 2.465704655647278
95th percentile: 2.484347939491272
99th percentile: 2.584902467727661
mean time: 2.189040279388428
Pipeline stage StressChecker completed in 69.68s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyTriggerPipeline
run_pipeline:run_in_cloud %s
starting trigger_guanaco_pipeline args=%s
triggered trigger_guanaco_pipeline args=%s
Pipeline stage OfflineFamilyFriendlyTriggerPipeline completed in 0.80s
Shutdown handler de-registered
chaiml-pony-v2-g46-lr1_80834_v23 status is now deployed due to DeploymentManager action
chaiml-pony-v2-g46-lr1_80834_v23 status is now inactive due to auto deactivation removed underperforming models
chaiml-pony-v2-g46-lr1_80834_v23 status is now torndown due to DeploymentManager action