developer_uid: zonemercy
submission_id: chaiml-pony-d3-g46-o4-l_49714_v1
model_name: chaiml-pony-d3-g46-o4-l_49714_v1
model_group: ChaiML/pony-d3-g46-o4-lr
status: torndown
timestamp: 2026-03-14T07:02:03+00:00
num_battles: 10773
num_wins: 6022
celo_rating: 1328.45
family_friendly_score: 0.0
family_friendly_standard_error: 0.0
submission_type: basic
model_repo: ChaiML/pony-d3-g46-o4-lr5e6ep1r64b8
model_architecture: Glm4MoeForCausalLM
model_num_parameters: 24110003200.0
best_of: 8
max_input_tokens: 2048
max_output_tokens: 80
reward_model: default
display_name: chaiml-pony-d3-g46-o4-l_49714_v1
ineligible_reason: max_output_tokens!=64
is_internal_developer: True
language_model: ChaiML/pony-d3-g46-o4-lr5e6ep1r64b8
model_size: 24B
ranking_group: single
us_pacific_date: 2026-03-10
win_ratio: 0.5589900677619976
generation_params: {'temperature': 1.0, 'top_p': 0.95, 'min_p': 0.0, 'top_k': 60, 'presence_penalty': 0.0, 'frequency_penalty': 0.0, 'stopping_words': ['<|assistant|>', '<|user|>', '<|im_end|>', '</s>', '####'], 'max_input_tokens': 2048, 'best_of': 8, 'max_output_tokens': 80}
formatter: {'memory_template': "[gMASK]<sop><|system|>\n{bot_name}'s persona: {memory}", 'prompt_template': '', 'bot_template': '<|assistant|>\n{message}', 'user_template': '<|user|>\n{message}', 'response_template': '<|assistant|>\n<think></think>\n{bot_name}:', 'truncate_by_message': True}
Resubmit model
Shutdown handler not registered because Python interpreter is not running in the main thread
run pipeline %s
run pipeline stage %s
Running pipeline stage VLLMUploader
Starting job with name chaiml-pony-d3-g46-o4-l-49714-v1-uploader
Waiting for job on chaiml-pony-d3-g46-o4-l-49714-v1-uploader to finish
chaiml-pony-d3-g46-o4-l-49714-v1-uploader: Using quantization_mode: fp8
chaiml-pony-d3-g46-o4-l-49714-v1-uploader: Checking if ChaiML/pony-d3-g46-o4-lr5e6ep1r64b8-FP8 already exists in ChaiML
chaiml-pony-d3-g46-o4-l-49714-v1-uploader: Downloading snapshot of ChaiML/pony-d3-g46-o4-lr5e6ep1r64b8...
2026-03-11T04:30:15.757002+00:00 monitor updated for chaiml-pony-d3-g46-o4-l_49714_v1
2026-03-11T04:31:15.843029+00:00 monitor updated for chaiml-pony-d3-g46-o4-l_49714_v1
2026-03-11T04:32:15.933704+00:00 monitor updated for chaiml-pony-d3-g46-o4-l_49714_v1
2026-03-11T04:33:16.020904+00:00 monitor updated for chaiml-pony-d3-g46-o4-l_49714_v1
chaiml-pony-d3-g46-o4-l-49714-v1-uploader: Downloaded in 243.351s
chaiml-pony-d3-g46-o4-l-49714-v1-uploader: Loading /tmp/model_input...
chaiml-pony-d3-g46-o4-l-49714-v1-uploader: The tokenizer you are loading from '/tmp/model_input' with an incorrect regex pattern: https://huggingface.co/mistralai/Mistral-Small-3.1-24B-Instruct-2503/discussions/84#69121093e8b480e709447d5e. This will lead to incorrect tokenization. You should set the `fix_mistral_regex=True` flag when loading this tokenizer to fix this issue.
chaiml-pony-d3-g46-o4-l-49714-v1-uploader: `torch_dtype` is deprecated! Use `dtype` instead!
2026-03-11T04:34:16.107786+00:00 monitor updated for chaiml-pony-d3-g46-o4-l_49714_v1
chaiml-pony-d3-g46-o4-l-49714-v1-uploader: Some parameters are on the meta device because they were offloaded to the cpu.
chaiml-pony-d3-g46-o4-l-49714-v1-uploader: Applying quantization...
chaiml-pony-d3-g46-o4-l-49714-v1-uploader: The tokenizer you are loading from '/tmp/model_input' with an incorrect regex pattern: https://huggingface.co/mistralai/Mistral-Small-3.1-24B-Instruct-2503/discussions/84#69121093e8b480e709447d5e. This will lead to incorrect tokenization. You should set the `fix_mistral_regex=True` flag when loading this tokenizer to fix this issue.
chaiml-pony-d3-g46-o4-l-49714-v1-uploader: 2026-03-10T21:34:42.860614-0700 | reset | INFO - Compression lifecycle reset
chaiml-pony-d3-g46-o4-l-49714-v1-uploader: 2026-03-10T21:34:42.934169-0700 | from_modifiers | INFO - Creating recipe from modifiers
2026-03-11T04:35:16.195043+00:00 monitor updated for chaiml-pony-d3-g46-o4-l_49714_v1
chaiml-pony-d3-g46-o4-l-49714-v1-uploader: Some parameters are on the meta device because they were offloaded to the cpu.
2026-03-11T04:36:29.149283+00:00 monitor updated for chaiml-pony-d3-g46-o4-l_49714_v1
2026-03-11T04:37:29.233675+00:00 monitor updated for chaiml-pony-d3-g46-o4-l_49714_v1
2026-03-11T04:38:29.328956+00:00 monitor updated for chaiml-pony-d3-g46-o4-l_49714_v1
2026-03-11T04:39:29.530258+00:00 monitor updated for chaiml-pony-d3-g46-o4-l_49714_v1
2026-03-11T04:40:29.624552+00:00 monitor updated for chaiml-pony-d3-g46-o4-l_49714_v1
2026-03-11T04:41:29.726407+00:00 monitor updated for chaiml-pony-d3-g46-o4-l_49714_v1
chaiml-pony-d3-g46-o4-l-49714-v1-uploader: 2026-03-10T21:41:53.360847-0700 | finalize | INFO - Compression lifecycle finalized for 1 modifiers
chaiml-pony-d3-g46-o4-l-49714-v1-uploader: 2026-03-10T21:42:00.508655-0700 | post_process | WARNING - Optimized model is not saved. To save, please provide`output_dir` as input arg.Ex. `oneshot(..., output_dir=...)`
chaiml-pony-d3-g46-o4-l-49714-v1-uploader: Saving to /dev/shm/model_output...
chaiml-pony-d3-g46-o4-l-49714-v1-uploader: 2026-03-10T21:42:00.544727-0700 | get_model_compressor | INFO - skip_sparsity_compression_stats set to True. Skipping sparsity compression statistic calculations. No sparsity compressor will be applied.
2026-03-11T04:42:29.817337+00:00 monitor updated for chaiml-pony-d3-g46-o4-l_49714_v1
2026-03-11T04:43:29.910158+00:00 monitor updated for chaiml-pony-d3-g46-o4-l_49714_v1
2026-03-11T04:44:30.003051+00:00 monitor updated for chaiml-pony-d3-g46-o4-l_49714_v1
2026-03-11T04:45:30.096273+00:00 monitor updated for chaiml-pony-d3-g46-o4-l_49714_v1
2026-03-11T04:46:30.192742+00:00 monitor updated for chaiml-pony-d3-g46-o4-l_49714_v1
2026-03-11T04:47:30.304255+00:00 monitor updated for chaiml-pony-d3-g46-o4-l_49714_v1
2026-03-11T04:48:30.396434+00:00 monitor updated for chaiml-pony-d3-g46-o4-l_49714_v1
2026-03-11T04:49:30.495928+00:00 monitor updated for chaiml-pony-d3-g46-o4-l_49714_v1
2026-03-11T04:50:30.593365+00:00 monitor updated for chaiml-pony-d3-g46-o4-l_49714_v1
2026-03-11T04:51:30.693507+00:00 monitor updated for chaiml-pony-d3-g46-o4-l_49714_v1
2026-03-11T04:52:30.785766+00:00 monitor updated for chaiml-pony-d3-g46-o4-l_49714_v1
2026-03-11T04:53:30.875625+00:00 monitor updated for chaiml-pony-d3-g46-o4-l_49714_v1
2026-03-11T04:54:30.968956+00:00 monitor updated for chaiml-pony-d3-g46-o4-l_49714_v1
2026-03-11T04:55:31.059268+00:00 monitor updated for chaiml-pony-d3-g46-o4-l_49714_v1
2026-03-11T04:56:31.156933+00:00 monitor updated for chaiml-pony-d3-g46-o4-l_49714_v1
2026-03-11T04:57:31.247141+00:00 monitor updated for chaiml-pony-d3-g46-o4-l_49714_v1
2026-03-11T04:58:31.337510+00:00 monitor updated for chaiml-pony-d3-g46-o4-l_49714_v1
2026-03-11T04:59:31.430549+00:00 monitor updated for chaiml-pony-d3-g46-o4-l_49714_v1
2026-03-11T05:00:31.529241+00:00 monitor updated for chaiml-pony-d3-g46-o4-l_49714_v1
2026-03-11T05:01:31.620178+00:00 monitor updated for chaiml-pony-d3-g46-o4-l_49714_v1
2026-03-11T05:02:31.710587+00:00 monitor updated for chaiml-pony-d3-g46-o4-l_49714_v1
2026-03-11T05:03:31.803682+00:00 monitor updated for chaiml-pony-d3-g46-o4-l_49714_v1
2026-03-11T05:04:31.906291+00:00 monitor updated for chaiml-pony-d3-g46-o4-l_49714_v1
2026-03-11T05:05:32.007429+00:00 monitor updated for chaiml-pony-d3-g46-o4-l_49714_v1
2026-03-11T05:06:32.242605+00:00 monitor updated for chaiml-pony-d3-g46-o4-l_49714_v1
2026-03-11T05:07:32.333555+00:00 monitor updated for chaiml-pony-d3-g46-o4-l_49714_v1
2026-03-11T05:08:32.424346+00:00 monitor updated for chaiml-pony-d3-g46-o4-l_49714_v1
2026-03-11T05:09:32.517062+00:00 monitor updated for chaiml-pony-d3-g46-o4-l_49714_v1
2026-03-11T05:10:32.608330+00:00 monitor updated for chaiml-pony-d3-g46-o4-l_49714_v1
2026-03-11T05:11:32.707742+00:00 monitor updated for chaiml-pony-d3-g46-o4-l_49714_v1
2026-03-11T05:12:32.805014+00:00 monitor updated for chaiml-pony-d3-g46-o4-l_49714_v1
2026-03-11T05:13:32.902546+00:00 monitor updated for chaiml-pony-d3-g46-o4-l_49714_v1
2026-03-11T05:14:32.997520+00:00 monitor updated for chaiml-pony-d3-g46-o4-l_49714_v1
2026-03-11T05:15:33.090042+00:00 monitor updated for chaiml-pony-d3-g46-o4-l_49714_v1
2026-03-11T05:16:33.200353+00:00 monitor updated for chaiml-pony-d3-g46-o4-l_49714_v1
2026-03-11T05:17:33.298402+00:00 monitor updated for chaiml-pony-d3-g46-o4-l_49714_v1
2026-03-11T05:18:33.574305+00:00 monitor updated for chaiml-pony-d3-g46-o4-l_49714_v1
2026-03-11T05:19:33.664190+00:00 monitor updated for chaiml-pony-d3-g46-o4-l_49714_v1
2026-03-11T05:20:33.758585+00:00 monitor updated for chaiml-pony-d3-g46-o4-l_49714_v1
2026-03-11T05:21:33.852187+00:00 monitor updated for chaiml-pony-d3-g46-o4-l_49714_v1
2026-03-11T05:22:33.942367+00:00 monitor updated for chaiml-pony-d3-g46-o4-l_49714_v1
2026-03-11T05:23:34.038289+00:00 monitor updated for chaiml-pony-d3-g46-o4-l_49714_v1
chaiml-pony-d3-g46-o4-l-49714-v1-uploader: Cleaning quantization config in /dev/shm/model_output
chaiml-pony-d3-g46-o4-l-49714-v1-uploader: Pushing to ChaiML/pony-d3-g46-o4-lr5e6ep1r64b8-FP8
chaiml-pony-d3-g46-o4-l-49714-v1-uploader: Checking if ChaiML/pony-d3-g46-o4-lr5e6ep1r64b8-FP8 already exists in ChaiML
chaiml-pony-d3-g46-o4-l-49714-v1-uploader: ---------- 2026-03-10 22:24:04 (0:00:00) ----------
chaiml-pony-d3-g46-o4-l-49714-v1-uploader: Files:   hashed 8/80 (28.2M/354.7G) | pre-uploaded: 0/1 (0.0/354.7G) (+72 unsure) | committed: 0/80 (0.0/354.7G) | ignored: 0
chaiml-pony-d3-g46-o4-l-49714-v1-uploader: Workers: hashing: 72 | get upload mode: 0 | pre-uploading: 1 | committing: 0 | waiting: 53
chaiml-pony-d3-g46-o4-l-49714-v1-uploader: ---------------------------------------------------
2026-03-11T05:24:34.137387+00:00 monitor updated for chaiml-pony-d3-g46-o4-l_49714_v1
chaiml-pony-d3-g46-o4-l-49714-v1-uploader: 







chaiml-pony-d3-g46-o4-l-49714-v1-uploader: ---------- 2026-03-10 22:25:05 (0:01:00) ----------
chaiml-pony-d3-g46-o4-l-49714-v1-uploader: Files:   hashed 80/80 (354.7G/354.7G) | pre-uploaded: 3/73 (5.1G/354.7G) | committed: 0/80 (0.0/354.7G) | ignored: 0
chaiml-pony-d3-g46-o4-l-49714-v1-uploader: Workers: hashing: 0 | get upload mode: 0 | pre-uploading: 70 | committing: 0 | waiting: 56
chaiml-pony-d3-g46-o4-l-49714-v1-uploader: ---------------------------------------------------
2026-03-11T05:25:34.231248+00:00 monitor updated for chaiml-pony-d3-g46-o4-l_49714_v1
chaiml-pony-d3-g46-o4-l-49714-v1-uploader:                              







chaiml-pony-d3-g46-o4-l-49714-v1-uploader: ---------- 2026-03-10 22:26:05 (0:02:00) ----------
chaiml-pony-d3-g46-o4-l-49714-v1-uploader: Files:   hashed 80/80 (354.7G/354.7G) | pre-uploaded: 11/73 (45.0G/354.7G) | committed: 0/80 (0.0/354.7G) | ignored: 0
chaiml-pony-d3-g46-o4-l-49714-v1-uploader: Workers: hashing: 0 | get upload mode: 0 | pre-uploading: 62 | committing: 0 | waiting: 64
chaiml-pony-d3-g46-o4-l-49714-v1-uploader: ---------------------------------------------------
2026-03-11T05:26:34.402213+00:00 monitor updated for chaiml-pony-d3-g46-o4-l_49714_v1
chaiml-pony-d3-g46-o4-l-49714-v1-uploader:                              







chaiml-pony-d3-g46-o4-l-49714-v1-uploader: ---------- 2026-03-10 22:27:05 (0:03:00) ----------
chaiml-pony-d3-g46-o4-l-49714-v1-uploader: Files:   hashed 80/80 (354.7G/354.7G) | pre-uploaded: 18/73 (80.0G/354.7G) | committed: 0/80 (0.0/354.7G) | ignored: 0
chaiml-pony-d3-g46-o4-l-49714-v1-uploader: Workers: hashing: 0 | get upload mode: 0 | pre-uploading: 55 | committing: 0 | waiting: 71
chaiml-pony-d3-g46-o4-l-49714-v1-uploader: ---------------------------------------------------
2026-03-11T05:27:34.601724+00:00 monitor updated for chaiml-pony-d3-g46-o4-l_49714_v1
chaiml-pony-d3-g46-o4-l-49714-v1-uploader:                              







chaiml-pony-d3-g46-o4-l-49714-v1-uploader: ---------- 2026-03-10 22:28:05 (0:04:00) ----------
chaiml-pony-d3-g46-o4-l-49714-v1-uploader: Files:   hashed 80/80 (354.7G/354.7G) | pre-uploaded: 20/73 (90.0G/354.7G) | committed: 0/80 (0.0/354.7G) | ignored: 0
chaiml-pony-d3-g46-o4-l-49714-v1-uploader: Workers: hashing: 0 | get upload mode: 0 | pre-uploading: 53 | committing: 0 | waiting: 73
chaiml-pony-d3-g46-o4-l-49714-v1-uploader: ---------------------------------------------------
2026-03-11T05:28:34.694183+00:00 monitor updated for chaiml-pony-d3-g46-o4-l_49714_v1
chaiml-pony-d3-g46-o4-l-49714-v1-uploader:                              







chaiml-pony-d3-g46-o4-l-49714-v1-uploader: ---------- 2026-03-10 22:29:05 (0:05:00) ----------
chaiml-pony-d3-g46-o4-l-49714-v1-uploader: Files:   hashed 80/80 (354.7G/354.7G) | pre-uploaded: 27/73 (124.9G/354.7G) | committed: 0/80 (0.0/354.7G) | ignored: 0
chaiml-pony-d3-g46-o4-l-49714-v1-uploader: Workers: hashing: 0 | get upload mode: 0 | pre-uploading: 46 | committing: 0 | waiting: 80
chaiml-pony-d3-g46-o4-l-49714-v1-uploader: ---------------------------------------------------
2026-03-11T05:29:34.784560+00:00 monitor updated for chaiml-pony-d3-g46-o4-l_49714_v1
chaiml-pony-d3-g46-o4-l-49714-v1-uploader:                              







chaiml-pony-d3-g46-o4-l-49714-v1-uploader: ---------- 2026-03-10 22:30:05 (0:06:00) ----------
chaiml-pony-d3-g46-o4-l-49714-v1-uploader: Files:   hashed 80/80 (354.7G/354.7G) | pre-uploaded: 35/73 (164.9G/354.7G) | committed: 0/80 (0.0/354.7G) | ignored: 0
chaiml-pony-d3-g46-o4-l-49714-v1-uploader: Workers: hashing: 0 | get upload mode: 0 | pre-uploading: 38 | committing: 0 | waiting: 88
chaiml-pony-d3-g46-o4-l-49714-v1-uploader: ---------------------------------------------------
2026-03-11T05:30:34.880831+00:00 monitor updated for chaiml-pony-d3-g46-o4-l_49714_v1
chaiml-pony-d3-g46-o4-l-49714-v1-uploader:                              







chaiml-pony-d3-g46-o4-l-49714-v1-uploader: ---------- 2026-03-10 22:31:05 (0:07:00) ----------
chaiml-pony-d3-g46-o4-l-49714-v1-uploader: Files:   hashed 80/80 (354.7G/354.7G) | pre-uploaded: 43/73 (204.8G/354.7G) | committed: 0/80 (0.0/354.7G) | ignored: 0
chaiml-pony-d3-g46-o4-l-49714-v1-uploader: Workers: hashing: 0 | get upload mode: 0 | pre-uploading: 30 | committing: 0 | waiting: 96
chaiml-pony-d3-g46-o4-l-49714-v1-uploader: ---------------------------------------------------
2026-03-11T05:31:34.996679+00:00 monitor updated for chaiml-pony-d3-g46-o4-l_49714_v1
chaiml-pony-d3-g46-o4-l-49714-v1-uploader:                              







chaiml-pony-d3-g46-o4-l-49714-v1-uploader: ---------- 2026-03-10 22:32:05 (0:08:00) ----------
chaiml-pony-d3-g46-o4-l-49714-v1-uploader: Files:   hashed 80/80 (354.7G/354.7G) | pre-uploaded: 50/73 (239.8G/354.7G) | committed: 0/80 (0.0/354.7G) | ignored: 0
chaiml-pony-d3-g46-o4-l-49714-v1-uploader: Workers: hashing: 0 | get upload mode: 0 | pre-uploading: 23 | committing: 0 | waiting: 103
chaiml-pony-d3-g46-o4-l-49714-v1-uploader: ---------------------------------------------------
2026-03-11T05:32:35.091290+00:00 monitor updated for chaiml-pony-d3-g46-o4-l_49714_v1
chaiml-pony-d3-g46-o4-l-49714-v1-uploader:                              







chaiml-pony-d3-g46-o4-l-49714-v1-uploader: ---------- 2026-03-10 22:33:05 (0:09:00) ----------
chaiml-pony-d3-g46-o4-l-49714-v1-uploader: Files:   hashed 80/80 (354.7G/354.7G) | pre-uploaded: 52/73 (249.8G/354.7G) | committed: 0/80 (0.0/354.7G) | ignored: 0
chaiml-pony-d3-g46-o4-l-49714-v1-uploader: Workers: hashing: 0 | get upload mode: 0 | pre-uploading: 21 | committing: 0 | waiting: 105
chaiml-pony-d3-g46-o4-l-49714-v1-uploader: ---------------------------------------------------
2026-03-11T05:33:35.193689+00:00 monitor updated for chaiml-pony-d3-g46-o4-l_49714_v1
chaiml-pony-d3-g46-o4-l-49714-v1-uploader:                              







chaiml-pony-d3-g46-o4-l-49714-v1-uploader: ---------- 2026-03-10 22:34:05 (0:10:00) ----------
chaiml-pony-d3-g46-o4-l-49714-v1-uploader: Files:   hashed 80/80 (354.7G/354.7G) | pre-uploaded: 59/73 (284.8G/354.7G) | committed: 0/80 (0.0/354.7G) | ignored: 0
chaiml-pony-d3-g46-o4-l-49714-v1-uploader: Workers: hashing: 0 | get upload mode: 0 | pre-uploading: 14 | committing: 0 | waiting: 112
chaiml-pony-d3-g46-o4-l-49714-v1-uploader: ---------------------------------------------------
2026-03-11T05:34:35.295377+00:00 monitor updated for chaiml-pony-d3-g46-o4-l_49714_v1
chaiml-pony-d3-g46-o4-l-49714-v1-uploader:                              







chaiml-pony-d3-g46-o4-l-49714-v1-uploader: ---------- 2026-03-10 22:35:05 (0:11:00) ----------
chaiml-pony-d3-g46-o4-l-49714-v1-uploader: Files:   hashed 80/80 (354.7G/354.7G) | pre-uploaded: 67/73 (324.7G/354.7G) | committed: 0/80 (0.0/354.7G) | ignored: 0
chaiml-pony-d3-g46-o4-l-49714-v1-uploader: Workers: hashing: 0 | get upload mode: 0 | pre-uploading: 6 | committing: 0 | waiting: 120
chaiml-pony-d3-g46-o4-l-49714-v1-uploader: ---------------------------------------------------
2026-03-11T05:35:35.389784+00:00 monitor updated for chaiml-pony-d3-g46-o4-l_49714_v1
chaiml-pony-d3-g46-o4-l-49714-v1-uploader:                              







chaiml-pony-d3-g46-o4-l-49714-v1-uploader: ---------- 2026-03-10 22:36:05 (0:12:00) ----------
chaiml-pony-d3-g46-o4-l-49714-v1-uploader: Files:   hashed 80/80 (354.7G/354.7G) | pre-uploaded: 73/73 (354.7G/354.7G) | committed: 0/80 (0.0/354.7G) | ignored: 0
chaiml-pony-d3-g46-o4-l-49714-v1-uploader: Workers: hashing: 0 | get upload mode: 0 | pre-uploading: 0 | committing: 1 | waiting: 125
chaiml-pony-d3-g46-o4-l-49714-v1-uploader: ---------------------------------------------------
2026-03-11T05:36:35.494127+00:00 monitor updated for chaiml-pony-d3-g46-o4-l_49714_v1
chaiml-pony-d3-g46-o4-l-49714-v1-uploader:                              







chaiml-pony-d3-g46-o4-l-49714-v1-uploader: ---------- 2026-03-10 22:37:05 (0:13:00) ----------
chaiml-pony-d3-g46-o4-l-49714-v1-uploader: Files:   hashed 80/80 (354.7G/354.7G) | pre-uploaded: 73/73 (354.7G/354.7G) | committed: 50/80 (204.9G/354.7G) | ignored: 0
chaiml-pony-d3-g46-o4-l-49714-v1-uploader: Workers: hashing: 0 | get upload mode: 0 | pre-uploading: 0 | committing: 1 | waiting: 125
chaiml-pony-d3-g46-o4-l-49714-v1-uploader: ---------------------------------------------------
2026-03-11T05:37:35.612442+00:00 monitor updated for chaiml-pony-d3-g46-o4-l_49714_v1
chaiml-pony-d3-g46-o4-l-49714-v1-uploader:                              Processed model ChaiML/pony-d3-g46-o4-lr5e6ep1r64b8 in 4058.516s
chaiml-pony-d3-g46-o4-l-49714-v1-uploader: creating bucket guanaco-vllm-models
chaiml-pony-d3-g46-o4-l-49714-v1-uploader: /usr/lib/python3/dist-packages/S3/BaseUtils.py:56: SyntaxWarning: invalid escape sequence '\.'
chaiml-pony-d3-g46-o4-l-49714-v1-uploader:   RE_S3_DATESTRING = re.compile('\.[0-9]*(?:[Z\\-\\+]*?)')
chaiml-pony-d3-g46-o4-l-49714-v1-uploader: /usr/lib/python3/dist-packages/S3/BaseUtils.py:57: SyntaxWarning: invalid escape sequence '\s'
chaiml-pony-d3-g46-o4-l-49714-v1-uploader:   RE_XML_NAMESPACE = re.compile(b'^(<?[^>]+?>\s*|\s*)(<\w+) xmlns=[\'"](https?://[^\'"]+)[\'"]', re.MULTILINE)
chaiml-pony-d3-g46-o4-l-49714-v1-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:240: SyntaxWarning: invalid escape sequence '\.'
chaiml-pony-d3-g46-o4-l-49714-v1-uploader:   invalid = re.search("([^a-z0-9\.-])", bucket, re.UNICODE)
chaiml-pony-d3-g46-o4-l-49714-v1-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:244: SyntaxWarning: invalid escape sequence '\.'
chaiml-pony-d3-g46-o4-l-49714-v1-uploader:   invalid = re.search("([^A-Za-z0-9\._-])", bucket, re.UNICODE)
chaiml-pony-d3-g46-o4-l-49714-v1-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:255: SyntaxWarning: invalid escape sequence '\.'
chaiml-pony-d3-g46-o4-l-49714-v1-uploader:   if re.search("-\.", bucket, re.UNICODE):
chaiml-pony-d3-g46-o4-l-49714-v1-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:257: SyntaxWarning: invalid escape sequence '\.'
chaiml-pony-d3-g46-o4-l-49714-v1-uploader:   if re.search("\.\.", bucket, re.UNICODE):
chaiml-pony-d3-g46-o4-l-49714-v1-uploader: /usr/lib/python3/dist-packages/S3/S3Uri.py:155: SyntaxWarning: invalid escape sequence '\w'
chaiml-pony-d3-g46-o4-l-49714-v1-uploader:   _re = re.compile("^(\w+://)?(.*)", re.UNICODE)
chaiml-pony-d3-g46-o4-l-49714-v1-uploader: /usr/lib/python3/dist-packages/S3/FileLists.py:480: SyntaxWarning: invalid escape sequence '\*'
chaiml-pony-d3-g46-o4-l-49714-v1-uploader:   wildcard_split_result = re.split("\*|\?", uri_str, maxsplit=1)
chaiml-pony-d3-g46-o4-l-49714-v1-uploader: Bucket 's3://guanaco-vllm-models/' created
chaiml-pony-d3-g46-o4-l-49714-v1-uploader: uploading /dev/shm/model_output to s3://guanaco-vllm-models/chaiml-pony-d3-g46-o4-l-49714-v1/default
chaiml-pony-d3-g46-o4-l-49714-v1-uploader: cp /dev/shm/model_output/config.json s3://guanaco-vllm-models/chaiml-pony-d3-g46-o4-l-49714-v1/default/config.json
chaiml-pony-d3-g46-o4-l-49714-v1-uploader: cp /dev/shm/model_output/chat_template.jinja s3://guanaco-vllm-models/chaiml-pony-d3-g46-o4-l-49714-v1/default/chat_template.jinja
chaiml-pony-d3-g46-o4-l-49714-v1-uploader: cp /dev/shm/model_output/generation_config.json s3://guanaco-vllm-models/chaiml-pony-d3-g46-o4-l-49714-v1/default/generation_config.json
chaiml-pony-d3-g46-o4-l-49714-v1-uploader: cp /dev/shm/model_output/recipe.yaml s3://guanaco-vllm-models/chaiml-pony-d3-g46-o4-l-49714-v1/default/recipe.yaml
chaiml-pony-d3-g46-o4-l-49714-v1-uploader: cp /dev/shm/model_output/special_tokens_map.json s3://guanaco-vllm-models/chaiml-pony-d3-g46-o4-l-49714-v1/default/special_tokens_map.json
chaiml-pony-d3-g46-o4-l-49714-v1-uploader: cp /dev/shm/model_output/tokenizer_config.json s3://guanaco-vllm-models/chaiml-pony-d3-g46-o4-l-49714-v1/default/tokenizer_config.json
chaiml-pony-d3-g46-o4-l-49714-v1-uploader: cp /dev/shm/model_output/model.safetensors.index.json s3://guanaco-vllm-models/chaiml-pony-d3-g46-o4-l-49714-v1/default/model.safetensors.index.json
chaiml-pony-d3-g46-o4-l-49714-v1-uploader: cp /dev/shm/model_output/tokenizer.json s3://guanaco-vllm-models/chaiml-pony-d3-g46-o4-l-49714-v1/default/tokenizer.json
chaiml-pony-d3-g46-o4-l-49714-v1-uploader: cp /dev/shm/model_output/model-00072-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-o4-l-49714-v1/default/model-00072-of-00072.safetensors
2026-03-11T05:38:35.708509+00:00 monitor updated for chaiml-pony-d3-g46-o4-l_49714_v1
chaiml-pony-d3-g46-o4-l-49714-v1-uploader: cp /dev/shm/model_output/model-00071-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-o4-l-49714-v1/default/model-00071-of-00072.safetensors
chaiml-pony-d3-g46-o4-l-49714-v1-uploader: cp /dev/shm/model_output/model-00006-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-o4-l-49714-v1/default/model-00006-of-00072.safetensors
chaiml-pony-d3-g46-o4-l-49714-v1-uploader: cp /dev/shm/model_output/model-00027-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-o4-l-49714-v1/default/model-00027-of-00072.safetensors
chaiml-pony-d3-g46-o4-l-49714-v1-uploader: cp /dev/shm/model_output/model-00002-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-o4-l-49714-v1/default/model-00002-of-00072.safetensors
chaiml-pony-d3-g46-o4-l-49714-v1-uploader: cp /dev/shm/model_output/model-00024-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-o4-l-49714-v1/default/model-00024-of-00072.safetensors
chaiml-pony-d3-g46-o4-l-49714-v1-uploader: cp /dev/shm/model_output/model-00004-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-o4-l-49714-v1/default/model-00004-of-00072.safetensors
chaiml-pony-d3-g46-o4-l-49714-v1-uploader: cp /dev/shm/model_output/model-00062-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-o4-l-49714-v1/default/model-00062-of-00072.safetensors
chaiml-pony-d3-g46-o4-l-49714-v1-uploader: cp /dev/shm/model_output/model-00069-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-o4-l-49714-v1/default/model-00069-of-00072.safetensors
chaiml-pony-d3-g46-o4-l-49714-v1-uploader: cp /dev/shm/model_output/model-00056-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-o4-l-49714-v1/default/model-00056-of-00072.safetensors
chaiml-pony-d3-g46-o4-l-49714-v1-uploader: cp /dev/shm/model_output/model-00042-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-o4-l-49714-v1/default/model-00042-of-00072.safetensors
chaiml-pony-d3-g46-o4-l-49714-v1-uploader: cp /dev/shm/model_output/model-00035-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-o4-l-49714-v1/default/model-00035-of-00072.safetensors
chaiml-pony-d3-g46-o4-l-49714-v1-uploader: cp /dev/shm/model_output/model-00032-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-o4-l-49714-v1/default/model-00032-of-00072.safetensors
chaiml-pony-d3-g46-o4-l-49714-v1-uploader: cp /dev/shm/model_output/model-00054-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-o4-l-49714-v1/default/model-00054-of-00072.safetensors
chaiml-pony-d3-g46-o4-l-49714-v1-uploader: cp /dev/shm/model_output/model-00041-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-o4-l-49714-v1/default/model-00041-of-00072.safetensors
chaiml-pony-d3-g46-o4-l-49714-v1-uploader: cp /dev/shm/model_output/model-00055-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-o4-l-49714-v1/default/model-00055-of-00072.safetensors
chaiml-pony-d3-g46-o4-l-49714-v1-uploader: cp /dev/shm/model_output/model-00019-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-o4-l-49714-v1/default/model-00019-of-00072.safetensors
chaiml-pony-d3-g46-o4-l-49714-v1-uploader: cp /dev/shm/model_output/model-00046-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-o4-l-49714-v1/default/model-00046-of-00072.safetensors
chaiml-pony-d3-g46-o4-l-49714-v1-uploader: cp /dev/shm/model_output/model-00022-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-o4-l-49714-v1/default/model-00022-of-00072.safetensors
chaiml-pony-d3-g46-o4-l-49714-v1-uploader: cp /dev/shm/model_output/model-00021-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-o4-l-49714-v1/default/model-00021-of-00072.safetensors
chaiml-pony-d3-g46-o4-l-49714-v1-uploader: cp /dev/shm/model_output/model-00031-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-o4-l-49714-v1/default/model-00031-of-00072.safetensors
chaiml-pony-d3-g46-o4-l-49714-v1-uploader: cp /dev/shm/model_output/model-00007-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-o4-l-49714-v1/default/model-00007-of-00072.safetensors
chaiml-pony-d3-g46-o4-l-49714-v1-uploader: cp /dev/shm/model_output/model-00067-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-o4-l-49714-v1/default/model-00067-of-00072.safetensors
chaiml-pony-d3-g46-o4-l-49714-v1-uploader: cp /dev/shm/model_output/model-00048-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-o4-l-49714-v1/default/model-00048-of-00072.safetensors
chaiml-pony-d3-g46-o4-l-49714-v1-uploader: cp /dev/shm/model_output/model-00009-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-o4-l-49714-v1/default/model-00009-of-00072.safetensors
chaiml-pony-d3-g46-o4-l-49714-v1-uploader: cp /dev/shm/model_output/model-00057-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-o4-l-49714-v1/default/model-00057-of-00072.safetensors
chaiml-pony-d3-g46-o4-l-49714-v1-uploader: cp /dev/shm/model_output/model-00003-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-o4-l-49714-v1/default/model-00003-of-00072.safetensors
chaiml-pony-d3-g46-o4-l-49714-v1-uploader: cp /dev/shm/model_output/model-00043-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-o4-l-49714-v1/default/model-00043-of-00072.safetensors
chaiml-pony-d3-g46-o4-l-49714-v1-uploader: cp /dev/shm/model_output/model-00039-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-o4-l-49714-v1/default/model-00039-of-00072.safetensors
chaiml-pony-d3-g46-o4-l-49714-v1-uploader: cp /dev/shm/model_output/model-00011-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-o4-l-49714-v1/default/model-00011-of-00072.safetensors
chaiml-pony-d3-g46-o4-l-49714-v1-uploader: cp /dev/shm/model_output/model-00028-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-o4-l-49714-v1/default/model-00028-of-00072.safetensors
chaiml-pony-d3-g46-o4-l-49714-v1-uploader: cp /dev/shm/model_output/model-00020-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-o4-l-49714-v1/default/model-00020-of-00072.safetensors
chaiml-pony-d3-g46-o4-l-49714-v1-uploader: cp /dev/shm/model_output/model-00051-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-o4-l-49714-v1/default/model-00051-of-00072.safetensors
chaiml-pony-d3-g46-o4-l-49714-v1-uploader: cp /dev/shm/model_output/model-00036-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-o4-l-49714-v1/default/model-00036-of-00072.safetensors
chaiml-pony-d3-g46-o4-l-49714-v1-uploader: cp /dev/shm/model_output/model-00005-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-o4-l-49714-v1/default/model-00005-of-00072.safetensors
chaiml-pony-d3-g46-o4-l-49714-v1-uploader: cp /dev/shm/model_output/model-00025-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-o4-l-49714-v1/default/model-00025-of-00072.safetensors
chaiml-pony-d3-g46-o4-l-49714-v1-uploader: cp /dev/shm/model_output/model-00014-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-o4-l-49714-v1/default/model-00014-of-00072.safetensors
chaiml-pony-d3-g46-o4-l-49714-v1-uploader: cp /dev/shm/model_output/model-00059-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-o4-l-49714-v1/default/model-00059-of-00072.safetensors
chaiml-pony-d3-g46-o4-l-49714-v1-uploader: cp /dev/shm/model_output/model-00030-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-o4-l-49714-v1/default/model-00030-of-00072.safetensors
chaiml-pony-d3-g46-o4-l-49714-v1-uploader: cp /dev/shm/model_output/model-00061-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-o4-l-49714-v1/default/model-00061-of-00072.safetensors
chaiml-pony-d3-g46-o4-l-49714-v1-uploader: cp /dev/shm/model_output/model-00040-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-o4-l-49714-v1/default/model-00040-of-00072.safetensors
chaiml-pony-d3-g46-o4-l-49714-v1-uploader: cp /dev/shm/model_output/model-00063-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-o4-l-49714-v1/default/model-00063-of-00072.safetensors
chaiml-pony-d3-g46-o4-l-49714-v1-uploader: cp /dev/shm/model_output/model-00065-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-o4-l-49714-v1/default/model-00065-of-00072.safetensors
chaiml-pony-d3-g46-o4-l-49714-v1-uploader: cp /dev/shm/model_output/model-00001-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-o4-l-49714-v1/default/model-00001-of-00072.safetensors
chaiml-pony-d3-g46-o4-l-49714-v1-uploader: cp /dev/shm/model_output/model-00026-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-o4-l-49714-v1/default/model-00026-of-00072.safetensors
chaiml-pony-d3-g46-o4-l-49714-v1-uploader: cp /dev/shm/model_output/model-00058-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-o4-l-49714-v1/default/model-00058-of-00072.safetensors
chaiml-pony-d3-g46-o4-l-49714-v1-uploader: cp /dev/shm/model_output/model-00068-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-o4-l-49714-v1/default/model-00068-of-00072.safetensors
chaiml-pony-d3-g46-o4-l-49714-v1-uploader: cp /dev/shm/model_output/model-00008-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-o4-l-49714-v1/default/model-00008-of-00072.safetensors
chaiml-pony-d3-g46-o4-l-49714-v1-uploader: cp /dev/shm/model_output/model-00018-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-o4-l-49714-v1/default/model-00018-of-00072.safetensors
chaiml-pony-d3-g46-o4-l-49714-v1-uploader: cp /dev/shm/model_output/model-00060-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-o4-l-49714-v1/default/model-00060-of-00072.safetensors
chaiml-pony-d3-g46-o4-l-49714-v1-uploader: cp /dev/shm/model_output/model-00045-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-o4-l-49714-v1/default/model-00045-of-00072.safetensors
chaiml-pony-d3-g46-o4-l-49714-v1-uploader: cp /dev/shm/model_output/model-00029-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-o4-l-49714-v1/default/model-00029-of-00072.safetensors
chaiml-pony-d3-g46-o4-l-49714-v1-uploader: cp /dev/shm/model_output/model-00034-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-o4-l-49714-v1/default/model-00034-of-00072.safetensors
chaiml-pony-d3-g46-o4-l-49714-v1-uploader: cp /dev/shm/model_output/model-00064-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-o4-l-49714-v1/default/model-00064-of-00072.safetensors
chaiml-pony-d3-g46-o4-l-49714-v1-uploader: cp /dev/shm/model_output/model-00044-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-o4-l-49714-v1/default/model-00044-of-00072.safetensors
chaiml-pony-d3-g46-o4-l-49714-v1-uploader: cp /dev/shm/model_output/model-00015-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-o4-l-49714-v1/default/model-00015-of-00072.safetensors
chaiml-pony-d3-g46-o4-l-49714-v1-uploader: cp /dev/shm/model_output/model-00016-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-o4-l-49714-v1/default/model-00016-of-00072.safetensors
chaiml-pony-d3-g46-o4-l-49714-v1-uploader: cp /dev/shm/model_output/model-00053-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-o4-l-49714-v1/default/model-00053-of-00072.safetensors
chaiml-pony-d3-g46-o4-l-49714-v1-uploader: cp /dev/shm/model_output/model-00047-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-o4-l-49714-v1/default/model-00047-of-00072.safetensors
chaiml-pony-d3-g46-o4-l-49714-v1-uploader: cp /dev/shm/model_output/model-00038-of-00072.safetensors s3://guanaco-vllm-models/chaiml-pony-d3-g46-o4-l-49714-v1/default/model-00038-of-00072.safetensors
Job chaiml-pony-d3-g46-o4-l-49714-v1-uploader completed after 4215.05s with status: succeeded
Stopping job with name chaiml-pony-d3-g46-o4-l-49714-v1-uploader
Pipeline stage VLLMUploader completed in 4215.58s
run pipeline stage %s
Running pipeline stage VLLMTemplater
Pipeline stage VLLMTemplater completed in 1.87s
run pipeline stage %s
Running pipeline stage VLLMDeployer
Creating inference service chaiml-pony-d3-g46-o4-l-49714-v1
Waiting for inference service chaiml-pony-d3-g46-o4-l-49714-v1 to be ready
2026-03-11T05:39:35.811135+00:00 monitor updated for chaiml-pony-d3-g46-o4-l_49714_v1
2026-03-11T05:40:35.920829+00:00 monitor updated for chaiml-pony-d3-g46-o4-l_49714_v1
2026-03-11T05:41:36.039099+00:00 monitor updated for chaiml-pony-d3-g46-o4-l_49714_v1
2026-03-11T05:42:36.145691+00:00 monitor updated for chaiml-pony-d3-g46-o4-l_49714_v1
2026-03-11T05:43:36.254061+00:00 monitor updated for chaiml-pony-d3-g46-o4-l_49714_v1
2026-03-11T05:44:36.362582+00:00 monitor updated for chaiml-pony-d3-g46-o4-l_49714_v1
2026-03-11T05:45:36.491314+00:00 monitor updated for chaiml-pony-d3-g46-o4-l_49714_v1
2026-03-11T05:46:36.605101+00:00 monitor updated for chaiml-pony-d3-g46-o4-l_49714_v1
Inference service chaiml-pony-d3-g46-o4-l-49714-v1 ready after 440.77838587760925s
Pipeline stage VLLMDeployer completed in 441.39s
run pipeline stage %s
Running pipeline stage StressChecker
Received healthy response to inference request in 9.40736198425293s
Received healthy response to inference request in 1.9645144939422607s
Received healthy response to inference request in 8.897323608398438s
Received healthy response to inference request in 8.918163061141968s
Received healthy response to inference request in 1.9935681819915771s
Received healthy response to inference request in 1.9872074127197266s
Received healthy response to inference request in 1.9772400856018066s
Received healthy response to inference request in 1.9620943069458008s
Received healthy response to inference request in 1.998673439025879s
2026-03-11T05:47:36.708136+00:00 monitor updated for chaiml-pony-d3-g46-o4-l_49714_v1
Received healthy response to inference request in 2.070978879928589s
Received healthy response to inference request in 1.9617435932159424s
Received healthy response to inference request in 1.9564943313598633s
Received healthy response to inference request in 1.9552297592163086s
Received healthy response to inference request in 1.9659461975097656s
Received healthy response to inference request in 1.958036184310913s
Received healthy response to inference request in 1.9174208641052246s
Received healthy response to inference request in 1.924708366394043s
Received healthy response to inference request in 9.05946683883667s
Received healthy response to inference request in 1.899214744567871s
Received healthy response to inference request in 1.9691340923309326s
Received healthy response to inference request in 1.964512586593628s
Received healthy response to inference request in 2.0549449920654297s
Received healthy response to inference request in 2.0277531147003174s
Received healthy response to inference request in 1.9693329334259033s
Received healthy response to inference request in 2.063753604888916s
Received healthy response to inference request in 1.9338583946228027s
Received healthy response to inference request in 1.9394805431365967s
Received healthy response to inference request in 1.9630255699157715s
Received healthy response to inference request in 1.9866149425506592s
Received healthy response to inference request in 1.9512696266174316s
30 requests
0 failed requests
5th percentile: 1.920700240135193
10th percentile: 1.9329433917999268
20th percentile: 1.9544377326965332
30th percentile: 1.9606313705444336
40th percentile: 1.9639177799224854
50th percentile: 1.9675401449203491
60th percentile: 1.9809900283813477
70th percentile: 1.9950997591018678
80th percentile: 2.0567067146301268
90th percentile: 8.899407553672791
95th percentile: 8.995880138874053
99th percentile: 9.306472392082215
mean time: 2.9199688911437987
Pipeline stage StressChecker completed in 91.50s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyTriggerPipeline
run_pipeline:run_in_cloud %s
starting trigger_guanaco_pipeline args=%s
triggered trigger_guanaco_pipeline args=%s
Pipeline stage OfflineFamilyFriendlyTriggerPipeline completed in 1.37s
Shutdown handler de-registered
chaiml-pony-d3-g46-o4-l_49714_v1 status is now deployed due to DeploymentManager action
chaiml-pony-d3-g46-o4-l_49714_v1 status is now inactive due to auto deactivation removed underperforming models
chaiml-pony-d3-g46-o4-l_49714_v1 status is now torndown due to DeploymentManager action