Shutdown handler not registered because Python interpreter is not running in the main thread
run pipeline %s
run pipeline stage %s
Running pipeline stage VLLMUploader
Starting job with name chaiml-4d70-fd43-linear-w01-v30-uploader
Waiting for job on chaiml-4d70-fd43-linear-w01-v30-uploader to finish
chaiml-4d70-fd43-linear-w01-v30-uploader: Using quantization_mode: none
chaiml-4d70-fd43-linear-w01-v30-uploader: Downloading snapshot of ChaiML/4d70-fd43-linear-w01...
chaiml-4d70-fd43-linear-w01-v30-uploader:
Fetching 14 files: 0%| | 0/14 [00:00<?, ?it/s]
Fetching 14 files: 7%|▋ | 1/14 [00:00<00:03, 3.48it/s]
Fetching 14 files: 43%|████▎ | 6/14 [00:12<00:17, 2.13s/it]
Fetching 14 files: 100%|██████████| 14/14 [00:12<00:00, 1.15it/s]
chaiml-4d70-fd43-linear-w01-v30-uploader: Downloaded in 12.310s
chaiml-4d70-fd43-linear-w01-v30-uploader: Processed model ChaiML/4d70-fd43-linear-w01 in 21.858s
chaiml-4d70-fd43-linear-w01-v30-uploader: creating bucket guanaco-vllm-models
chaiml-4d70-fd43-linear-w01-v30-uploader: /usr/lib/python3/dist-packages/S3/BaseUtils.py:56: SyntaxWarning: invalid escape sequence '\.'
chaiml-4d70-fd43-linear-w01-v30-uploader: RE_S3_DATESTRING = re.compile('\.[0-9]*(?:[Z\\-\\+]*?)')
chaiml-4d70-fd43-linear-w01-v30-uploader: /usr/lib/python3/dist-packages/S3/BaseUtils.py:57: SyntaxWarning: invalid escape sequence '\s'
chaiml-4d70-fd43-linear-w01-v30-uploader: RE_XML_NAMESPACE = re.compile(b'^(<?[^>]+?>\s*|\s*)(<\w+) xmlns=[\'"](https?://[^\'"]+)[\'"]', re.MULTILINE)
chaiml-4d70-fd43-linear-w01-v30-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:240: SyntaxWarning: invalid escape sequence '\.'
chaiml-4d70-fd43-linear-w01-v30-uploader: invalid = re.search("([^a-z0-9\.-])", bucket, re.UNICODE)
chaiml-4d70-fd43-linear-w01-v30-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:244: SyntaxWarning: invalid escape sequence '\.'
chaiml-4d70-fd43-linear-w01-v30-uploader: invalid = re.search("([^A-Za-z0-9\._-])", bucket, re.UNICODE)
chaiml-4d70-fd43-linear-w01-v30-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:255: SyntaxWarning: invalid escape sequence '\.'
chaiml-4d70-fd43-linear-w01-v30-uploader: if re.search("-\.", bucket, re.UNICODE):
chaiml-4d70-fd43-linear-w01-v30-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:257: SyntaxWarning: invalid escape sequence '\.'
chaiml-4d70-fd43-linear-w01-v30-uploader: if re.search("\.\.", bucket, re.UNICODE):
chaiml-4d70-fd43-linear-w01-v30-uploader: /usr/lib/python3/dist-packages/S3/S3Uri.py:155: SyntaxWarning: invalid escape sequence '\w'
chaiml-4d70-fd43-linear-w01-v30-uploader: _re = re.compile("^(\w+://)?(.*)", re.UNICODE)
chaiml-4d70-fd43-linear-w01-v30-uploader: /usr/lib/python3/dist-packages/S3/FileLists.py:480: SyntaxWarning: invalid escape sequence '\*'
chaiml-4d70-fd43-linear-w01-v30-uploader: wildcard_split_result = re.split("\*|\?", uri_str, maxsplit=1)
chaiml-4d70-fd43-linear-w01-v30-uploader: Bucket 's3://guanaco-vllm-models/' created
chaiml-4d70-fd43-linear-w01-v30-uploader: uploading /dev/shm/model_output to s3://guanaco-vllm-models/chaiml-4d70-fd43-linear-w01-v30
chaiml-4d70-fd43-linear-w01-v30-uploader: cp /dev/shm/model_output/.gitattributes s3://guanaco-vllm-models/chaiml-4d70-fd43-linear-w01-v30/.gitattributes
chaiml-4d70-fd43-linear-w01-v30-uploader: cp /dev/shm/model_output/mergekit_config.yaml s3://guanaco-vllm-models/chaiml-4d70-fd43-linear-w01-v30/mergekit_config.yaml
chaiml-4d70-fd43-linear-w01-v30-uploader: cp /dev/shm/model_output/special_tokens_map.json s3://guanaco-vllm-models/chaiml-4d70-fd43-linear-w01-v30/special_tokens_map.json
chaiml-4d70-fd43-linear-w01-v30-uploader: cp /dev/shm/model_output/config.json s3://guanaco-vllm-models/chaiml-4d70-fd43-linear-w01-v30/config.json
chaiml-4d70-fd43-linear-w01-v30-uploader: cp /dev/shm/model_output/mergekit_config.yml s3://guanaco-vllm-models/chaiml-4d70-fd43-linear-w01-v30/mergekit_config.yml
chaiml-4d70-fd43-linear-w01-v30-uploader: cp /dev/shm/model_output/tokenizer_config.json s3://guanaco-vllm-models/chaiml-4d70-fd43-linear-w01-v30/tokenizer_config.json
chaiml-4d70-fd43-linear-w01-v30-uploader: cp /dev/shm/model_output/README.md s3://guanaco-vllm-models/chaiml-4d70-fd43-linear-w01-v30/README.md
HTTP Request: %s %s "%s %d %s"
chaiml-4d70-fd43-linear-w01-v30-uploader: cp /dev/shm/model_output/model-00003-of-00005.safetensors s3://guanaco-vllm-models/chaiml-4d70-fd43-linear-w01-v30/model-00003-of-00005.safetensors
chaiml-4d70-fd43-linear-w01-v30-uploader: cp /dev/shm/model_output/model-00004-of-00005.safetensors s3://guanaco-vllm-models/chaiml-4d70-fd43-linear-w01-v30/model-00004-of-00005.safetensors
chaiml-4d70-fd43-linear-w01-v30-uploader: cp /dev/shm/model_output/model-00002-of-00005.safetensors s3://guanaco-vllm-models/chaiml-4d70-fd43-linear-w01-v30/model-00002-of-00005.safetensors
chaiml-4d70-fd43-linear-w01-v30-uploader: cp /dev/shm/model_output/model-00001-of-00005.safetensors s3://guanaco-vllm-models/chaiml-4d70-fd43-linear-w01-v30/model-00001-of-00005.safetensors
chaiml-4d70-fd43-linear-w01-v30-uploader: cp /dev/shm/model_output/model-00005-of-00005.safetensors s3://guanaco-vllm-models/chaiml-4d70-fd43-linear-w01-v30/model-00005-of-00005.safetensors
Job chaiml-4d70-fd43-linear-w01-v30-uploader completed after 176.12s with status: succeeded
Stopping job with name chaiml-4d70-fd43-linear-w01-v30-uploader
Pipeline stage VLLMUploader completed in 177.26s
run pipeline stage %s
Running pipeline stage VLLMTemplater
Pipeline stage VLLMTemplater completed in 0.15s
run pipeline stage %s
Running pipeline stage VLLMDeployer
Creating inference service chaiml-4d70-fd43-linear-w01-v30
Waiting for inference service chaiml-4d70-fd43-linear-w01-v30 to be ready
Inference service chaiml-4d70-fd43-linear-w01-v30 ready after 362.8756456375122s
Pipeline stage VLLMDeployer completed in 363.53s
run pipeline stage %s
Running pipeline stage StressChecker
Received healthy response to inference request in 1.581648349761963s
Received healthy response to inference request in 1.6627576351165771s
Received healthy response to inference request in 1.4576163291931152s
Received healthy response to inference request in 1.6008458137512207s
Received healthy response to inference request in 2.136488199234009s
Received healthy response to inference request in 1.4455628395080566s
Received healthy response to inference request in 1.9477312564849854s
Received healthy response to inference request in 2.122260332107544s
Received healthy response to inference request in 1.5939972400665283s
Received healthy response to inference request in 1.8323493003845215s
Received healthy response to inference request in 1.7048962116241455s
Received healthy response to inference request in 1.5027365684509277s
Received healthy response to inference request in 2.03647518157959s
Received healthy response to inference request in 1.5320119857788086s
Received healthy response to inference request in 1.5839343070983887s
Received healthy response to inference request in 2.665250062942505s
Received healthy response to inference request in 1.525693416595459s
Received healthy response to inference request in 1.4748082160949707s
Received healthy response to inference request in 1.6811323165893555s
Received healthy response to inference request in 1.6349456310272217s
Received healthy response to inference request in 1.555877447128296s
Received healthy response to inference request in 1.7839877605438232s
Received healthy response to inference request in 1.642773151397705s
Received healthy response to inference request in 1.788313627243042s
Received healthy response to inference request in 1.565345048904419s
Received healthy response to inference request in 1.5726940631866455s
Received healthy response to inference request in 1.8113396167755127s
Received healthy response to inference request in 1.5604333877563477s
Received healthy response to inference request in 1.731081485748291s
Received healthy response to inference request in 1.5453007221221924s
30 requests
0 failed requests
5th percentile: 1.46535267829895
10th percentile: 1.499943733215332
20th percentile: 1.5426429748535155
30th percentile: 1.5638715505599976
40th percentile: 1.5830199241638183
50th percentile: 1.6178957223892212
60th percentile: 1.6701075077056884
70th percentile: 1.7469533681869505
80th percentile: 1.8155415534973145
90th percentile: 2.0450536966323853
95th percentile: 2.1300856590271
99th percentile: 2.5119091224670416
mean time: 1.7093429168065388
Pipeline stage StressChecker completed in 53.92s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyTriggerPipeline
run_pipeline:run_in_cloud %s
starting trigger_guanaco_pipeline args=%s
triggered trigger_guanaco_pipeline args=%s
Pipeline stage OfflineFamilyFriendlyTriggerPipeline completed in 1.11s
Shutdown handler de-registered
chaiml-4d70-fd43-linear-w01_v30 status is now deployed due to DeploymentManager action