Shutdown handler not registered because Python interpreter is not running in the main thread
Connection pool is full, discarding connection: %s. Connection pool size: %s
run pipeline %s
run pipeline stage %s
Running pipeline stage VLLMUploader
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Starting job with name chaiml-shina-bondage-ant-8379-v1-uploader
Waiting for job on chaiml-shina-bondage-ant-8379-v1-uploader to finish
Connection pool is full, discarding connection: %s. Connection pool size: %s
Retrying (%r) after connection broken by '%r': %s
chaiml-shina-bondage-ant-8379-v1-uploader: Using quantization_mode: fp8
chaiml-shina-bondage-ant-8379-v1-uploader: Checking if ChaiML/Shina-bondage-anthro-furry-fluffy-drag260225204622_sft-FP8 already exists in ChaiML
chaiml-shina-bondage-ant-8379-v1-uploader: Downloading snapshot of ChaiML/Shina-bondage-anthro-furry-fluffy-drag260225204622_sft...
chaiml-shina-bondage-ant-8379-v1-uploader: Downloaded in 145.503s
chaiml-shina-bondage-ant-8379-v1-uploader: Loading /tmp/model_input...
chaiml-shina-bondage-ant-8379-v1-uploader: The tokenizer you are loading from '/tmp/model_input' with an incorrect regex pattern: https://huggingface.co/mistralai/Mistral-Small-3.1-24B-Instruct-2503/discussions/84#69121093e8b480e709447d5e. This will lead to incorrect tokenization. You should set the `fix_mistral_regex=True` flag when loading this tokenizer to fix this issue.
chaiml-shina-bondage-ant-8379-v1-uploader: `torch_dtype` is deprecated! Use `dtype` instead!
chaiml-shina-bondage-ant-8379-v1-uploader: Some parameters are on the meta device because they were offloaded to the cpu.
chaiml-shina-bondage-ant-8379-v1-uploader: Applying quantization...
chaiml-shina-bondage-ant-8379-v1-uploader: The tokenizer you are loading from '/tmp/model_input' with an incorrect regex pattern: https://huggingface.co/mistralai/Mistral-Small-3.1-24B-Instruct-2503/discussions/84#69121093e8b480e709447d5e. This will lead to incorrect tokenization. You should set the `fix_mistral_regex=True` flag when loading this tokenizer to fix this issue.
chaiml-shina-bondage-ant-8379-v1-uploader: 2026-02-25T12:59:19.105127-0800 | reset | INFO - Compression lifecycle reset
chaiml-shina-bondage-ant-8379-v1-uploader: 2026-02-25T12:59:19.106036-0800 | from_modifiers | INFO - Creating recipe from modifiers
chaiml-shina-bondage-ant-8379-v1-uploader: 2026-02-25T12:59:19.181197-0800 | initialize | INFO - Compression lifecycle initialized for 1 modifiers
chaiml-shina-bondage-ant-8379-v1-uploader: 2026-02-25T12:59:19.181463-0800 | IndependentPipeline | INFO - Inferred `DataFreePipeline` for `QuantizationModifier`
chaiml-shina-bondage-ant-8379-v1-uploader: Some parameters are on the meta device because they were offloaded to the cpu.
chaiml-shina-bondage-ant-8379-v1-uploader: 2026-02-25T12:59:48.734612-0800 | finalize | INFO - Compression lifecycle finalized for 1 modifiers
chaiml-shina-bondage-ant-8379-v1-uploader: 2026-02-25T12:59:50.839918-0800 | post_process | WARNING - Optimized model is not saved. To save, please provide`output_dir` as input arg.Ex. `oneshot(..., output_dir=...)`
chaiml-shina-bondage-ant-8379-v1-uploader: Saving to /dev/shm/model_output...
chaiml-shina-bondage-ant-8379-v1-uploader: 2026-02-25T12:59:50.866854-0800 | get_model_compressor | INFO - skip_sparsity_compression_stats set to True. Skipping sparsity compression statistic calculations. No sparsity compressor will be applied.
chaiml-shina-bondage-ant-8379-v1-uploader: Cleaning quantization config in /dev/shm/model_output
chaiml-shina-bondage-ant-8379-v1-uploader: Pushing to ChaiML/Shina-bondage-anthro-furry-fluffy-drag260225204622_sft-FP8
chaiml-shina-bondage-ant-8379-v1-uploader: Checking if ChaiML/Shina-bondage-anthro-furry-fluffy-drag260225204622_sft-FP8 already exists in ChaiML
chaiml-shina-bondage-ant-8379-v1-uploader: Creating repo ChaiML/Shina-bondage-anthro-furry-fluffy-drag260225204622_sft-FP8 and uploading /dev/shm/model_output to it
chaiml-shina-bondage-ant-8379-v1-uploader: ---------- 2026-02-25 13:00:37 (0:00:00) ----------
chaiml-shina-bondage-ant-8379-v1-uploader: Files:   hashed 3/13 (219.8K/24.9G) | pre-uploaded: 0/0 (0.0/24.9G) (+13 unsure) | committed: 0/13 (0.0/24.9G) | ignored: 0
chaiml-shina-bondage-ant-8379-v1-uploader: Workers: hashing: 12 | get upload mode: 1 | pre-uploading: 0 | committing: 0 | waiting: 113
chaiml-shina-bondage-ant-8379-v1-uploader: ---------------------------------------------------
chaiml-shina-bondage-ant-8379-v1-uploader: 







chaiml-shina-bondage-ant-8379-v1-uploader: ---------- 2026-02-25 13:01:37 (0:01:00) ----------
chaiml-shina-bondage-ant-8379-v1-uploader: Files:   hashed 13/13 (24.9G/24.9G) | pre-uploaded: 7/7 (24.9G/24.9G) | committed: 0/13 (0.0/24.9G) | ignored: 0
chaiml-shina-bondage-ant-8379-v1-uploader: Workers: hashing: 0 | get upload mode: 0 | pre-uploading: 0 | committing: 1 | waiting: 125
chaiml-shina-bondage-ant-8379-v1-uploader: ---------------------------------------------------
chaiml-shina-bondage-ant-8379-v1-uploader: creating bucket guanaco-vllm-models
chaiml-shina-bondage-ant-8379-v1-uploader: /usr/lib/python3/dist-packages/S3/BaseUtils.py:56: SyntaxWarning: invalid escape sequence '\.'
chaiml-shina-bondage-ant-8379-v1-uploader:   RE_S3_DATESTRING = re.compile('\.[0-9]*(?:[Z\\-\\+]*?)')
chaiml-shina-bondage-ant-8379-v1-uploader: /usr/lib/python3/dist-packages/S3/BaseUtils.py:57: SyntaxWarning: invalid escape sequence '\s'
chaiml-shina-bondage-ant-8379-v1-uploader:   RE_XML_NAMESPACE = re.compile(b'^(<?[^>]+?>\s*|\s*)(<\w+) xmlns=[\'"](https?://[^\'"]+)[\'"]', re.MULTILINE)
chaiml-shina-bondage-ant-8379-v1-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:240: SyntaxWarning: invalid escape sequence '\.'
chaiml-shina-bondage-ant-8379-v1-uploader:   invalid = re.search("([^a-z0-9\.-])", bucket, re.UNICODE)
chaiml-shina-bondage-ant-8379-v1-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:244: SyntaxWarning: invalid escape sequence '\.'
chaiml-shina-bondage-ant-8379-v1-uploader:   invalid = re.search("([^A-Za-z0-9\._-])", bucket, re.UNICODE)
chaiml-shina-bondage-ant-8379-v1-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:255: SyntaxWarning: invalid escape sequence '\.'
chaiml-shina-bondage-ant-8379-v1-uploader:   if re.search("-\.", bucket, re.UNICODE):
chaiml-shina-bondage-ant-8379-v1-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:257: SyntaxWarning: invalid escape sequence '\.'
chaiml-shina-bondage-ant-8379-v1-uploader:   if re.search("\.\.", bucket, re.UNICODE):
chaiml-shina-bondage-ant-8379-v1-uploader: /usr/lib/python3/dist-packages/S3/S3Uri.py:155: SyntaxWarning: invalid escape sequence '\w'
chaiml-shina-bondage-ant-8379-v1-uploader:   _re = re.compile("^(\w+://)?(.*)", re.UNICODE)
chaiml-shina-bondage-ant-8379-v1-uploader: /usr/lib/python3/dist-packages/S3/FileLists.py:480: SyntaxWarning: invalid escape sequence '\*'
chaiml-shina-bondage-ant-8379-v1-uploader:   wildcard_split_result = re.split("\*|\?", uri_str, maxsplit=1)
chaiml-shina-bondage-ant-8379-v1-uploader: Bucket 's3://guanaco-vllm-models/' created
chaiml-shina-bondage-ant-8379-v1-uploader: uploading /dev/shm/model_output to s3://guanaco-vllm-models/chaiml-shina-bondage-ant-8379-v1/default
chaiml-shina-bondage-ant-8379-v1-uploader: cp /dev/shm/model_output/config.json s3://guanaco-vllm-models/chaiml-shina-bondage-ant-8379-v1/default/config.json
chaiml-shina-bondage-ant-8379-v1-uploader: cp /dev/shm/model_output/generation_config.json s3://guanaco-vllm-models/chaiml-shina-bondage-ant-8379-v1/default/generation_config.json
chaiml-shina-bondage-ant-8379-v1-uploader: cp /dev/shm/model_output/model.safetensors.index.json s3://guanaco-vllm-models/chaiml-shina-bondage-ant-8379-v1/default/model.safetensors.index.json
chaiml-shina-bondage-ant-8379-v1-uploader: cp /dev/shm/model_output/recipe.yaml s3://guanaco-vllm-models/chaiml-shina-bondage-ant-8379-v1/default/recipe.yaml
chaiml-shina-bondage-ant-8379-v1-uploader: cp /dev/shm/model_output/tokenizer_config.json s3://guanaco-vllm-models/chaiml-shina-bondage-ant-8379-v1/default/tokenizer_config.json
chaiml-shina-bondage-ant-8379-v1-uploader: cp /dev/shm/model_output/special_tokens_map.json s3://guanaco-vllm-models/chaiml-shina-bondage-ant-8379-v1/default/special_tokens_map.json
chaiml-shina-bondage-ant-8379-v1-uploader: cp /dev/shm/model_output/tokenizer.json s3://guanaco-vllm-models/chaiml-shina-bondage-ant-8379-v1/default/tokenizer.json
chaiml-shina-bondage-ant-8379-v1-uploader: cp /dev/shm/model_output/model-00006-of-00006.safetensors s3://guanaco-vllm-models/chaiml-shina-bondage-ant-8379-v1/default/model-00006-of-00006.safetensors
chaiml-shina-bondage-ant-8379-v1-uploader: cp /dev/shm/model_output/model-00005-of-00006.safetensors s3://guanaco-vllm-models/chaiml-shina-bondage-ant-8379-v1/default/model-00005-of-00006.safetensors
chaiml-shina-bondage-ant-8379-v1-uploader: cp /dev/shm/model_output/model-00002-of-00006.safetensors s3://guanaco-vllm-models/chaiml-shina-bondage-ant-8379-v1/default/model-00002-of-00006.safetensors
chaiml-shina-bondage-ant-8379-v1-uploader: cp /dev/shm/model_output/model-00001-of-00006.safetensors s3://guanaco-vllm-models/chaiml-shina-bondage-ant-8379-v1/default/model-00001-of-00006.safetensors
chaiml-shina-bondage-ant-8379-v1-uploader: cp /dev/shm/model_output/model-00003-of-00006.safetensors s3://guanaco-vllm-models/chaiml-shina-bondage-ant-8379-v1/default/model-00003-of-00006.safetensors
chaiml-shina-bondage-ant-8379-v1-uploader: cp /dev/shm/model_output/model-00004-of-00006.safetensors s3://guanaco-vllm-models/chaiml-shina-bondage-ant-8379-v1/default/model-00004-of-00006.safetensors
Job chaiml-shina-bondage-ant-8379-v1-uploader completed after 365.79s with status: succeeded
Stopping job with name chaiml-shina-bondage-ant-8379-v1-uploader
Pipeline stage VLLMUploader completed in 368.40s
run pipeline stage %s
Running pipeline stage VLLMTemplater
Pipeline stage VLLMTemplater completed in 0.32s
run pipeline stage %s
Running pipeline stage VLLMDeployer
Creating inference service chaiml-shina-bondage-ant-8379-v1
Waiting for inference service chaiml-shina-bondage-ant-8379-v1 to be ready
Inference service chaiml-shina-bondage-ant-8379-v1 ready after 160.81255173683167s
Pipeline stage VLLMDeployer completed in 161.56s
run pipeline stage %s
Running pipeline stage StressChecker
Received healthy response to inference request in 2.803104877471924s
Received healthy response to inference request in 2.7437338829040527s
Received healthy response to inference request in 2.9412593841552734s
Received healthy response to inference request in 2.8190865516662598s
Received healthy response to inference request in 2.7608704566955566s
Received healthy response to inference request in 3.038794994354248s
Received healthy response to inference request in 2.667863368988037s
Received healthy response to inference request in 2.7369818687438965s
Received healthy response to inference request in 2.861752510070801s
Received healthy response to inference request in 2.6682729721069336s
Received healthy response to inference request in 3.3733954429626465s
Received healthy response to inference request in 2.8321616649627686s
Received healthy response to inference request in 2.950002670288086s
Received healthy response to inference request in 2.835901975631714s
Received healthy response to inference request in 2.9736621379852295s
Received healthy response to inference request in 2.6745851039886475s
Received healthy response to inference request in 2.719602346420288s
Received healthy response to inference request in 2.6679041385650635s
Received healthy response to inference request in 2.838935136795044s
Received healthy response to inference request in 2.7395238876342773s
Received healthy response to inference request in 2.9812240600585938s
Received healthy response to inference request in 2.6930694580078125s
Received healthy response to inference request in 2.8934693336486816s
Received healthy response to inference request in 3.61285400390625s
Received healthy response to inference request in 3.243400812149048s
Received healthy response to inference request in 2.7455830574035645s
Received healthy response to inference request in 2.9892566204071045s
Received healthy response to inference request in 2.712451696395874s
Received healthy response to inference request in 2.750088691711426s
Received healthy response to inference request in 2.7836976051330566s
30 requests
0 failed requests
5th percentile: 2.668070113658905
10th percentile: 2.673953890800476
20th percentile: 2.718172216415405
30th percentile: 2.74247088432312
40th percentile: 2.756557750701904
50th percentile: 2.811095714569092
60th percentile: 2.837115240097046
70th percentile: 2.907806348800659
80th percentile: 2.9751745223999024
90th percentile: 3.059255576133728
95th percentile: 3.3148978590965266
99th percentile: 3.5434110212326053
mean time: 2.868416357040405
Pipeline stage StressChecker completed in 89.03s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyTriggerPipeline
run_pipeline:run_in_cloud %s
starting trigger_guanaco_pipeline args=%s
triggered trigger_guanaco_pipeline args=%s
Pipeline stage OfflineFamilyFriendlyTriggerPipeline completed in 1.56s
Shutdown handler de-registered
chaiml-shina-bondage-ant_8379_v1 status is now deployed due to DeploymentManager action
chaiml-shina-bondage-ant_8379_v1 status is now inactive due to auto deactivation removed underperforming models