Shutdown handler not registered because Python interpreter is not running in the main thread
run pipeline %s
run pipeline stage %s
Running pipeline stage VLLMUploader
Starting job with name chaiml-4d70-fd43-linear-w01-v33-uploader
Waiting for job on chaiml-4d70-fd43-linear-w01-v33-uploader to finish
chaiml-4d70-fd43-linear-w01-v33-uploader: Using quantization_mode: none
chaiml-4d70-fd43-linear-w01-v33-uploader: Downloading snapshot of ChaiML/4d70-fd43-linear-w01...
chaiml-4d70-fd43-linear-w01-v33-uploader:
Fetching 14 files: 0%| | 0/14 [00:00<?, ?it/s]
Fetching 14 files: 7%|▋ | 1/14 [00:00<00:03, 3.52it/s]
Fetching 14 files: 43%|████▎ | 6/14 [00:11<00:16, 2.05s/it]
Fetching 14 files: 50%|█████ | 7/14 [00:12<00:11, 1.69s/it]
Fetching 14 files: 57%|█████▋ | 8/14 [00:12<00:08, 1.34s/it]
Fetching 14 files: 71%|███████▏ | 10/14 [00:12<00:03, 1.11it/s]
Fetching 14 files: 100%|██████████| 14/14 [00:12<00:00, 1.12it/s]
chaiml-4d70-fd43-linear-w01-v33-uploader: Downloaded in 12.649s
chaiml-4d70-fd43-linear-w01-v33-uploader: Processed model ChaiML/4d70-fd43-linear-w01 in 21.785s
chaiml-4d70-fd43-linear-w01-v33-uploader: creating bucket guanaco-vllm-models
chaiml-4d70-fd43-linear-w01-v33-uploader: /usr/lib/python3/dist-packages/S3/BaseUtils.py:56: SyntaxWarning: invalid escape sequence '\.'
chaiml-4d70-fd43-linear-w01-v33-uploader: RE_S3_DATESTRING = re.compile('\.[0-9]*(?:[Z\\-\\+]*?)')
chaiml-4d70-fd43-linear-w01-v33-uploader: /usr/lib/python3/dist-packages/S3/BaseUtils.py:57: SyntaxWarning: invalid escape sequence '\s'
chaiml-4d70-fd43-linear-w01-v33-uploader: RE_XML_NAMESPACE = re.compile(b'^(<?[^>]+?>\s*|\s*)(<\w+) xmlns=[\'"](https?://[^\'"]+)[\'"]', re.MULTILINE)
chaiml-4d70-fd43-linear-w01-v33-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:240: SyntaxWarning: invalid escape sequence '\.'
chaiml-4d70-fd43-linear-w01-v33-uploader: invalid = re.search("([^a-z0-9\.-])", bucket, re.UNICODE)
chaiml-4d70-fd43-linear-w01-v33-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:244: SyntaxWarning: invalid escape sequence '\.'
chaiml-4d70-fd43-linear-w01-v33-uploader: invalid = re.search("([^A-Za-z0-9\._-])", bucket, re.UNICODE)
chaiml-4d70-fd43-linear-w01-v33-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:255: SyntaxWarning: invalid escape sequence '\.'
chaiml-4d70-fd43-linear-w01-v33-uploader: if re.search("-\.", bucket, re.UNICODE):
chaiml-4d70-fd43-linear-w01-v33-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:257: SyntaxWarning: invalid escape sequence '\.'
chaiml-4d70-fd43-linear-w01-v33-uploader: if re.search("\.\.", bucket, re.UNICODE):
chaiml-4d70-fd43-linear-w01-v33-uploader: /usr/lib/python3/dist-packages/S3/S3Uri.py:155: SyntaxWarning: invalid escape sequence '\w'
chaiml-4d70-fd43-linear-w01-v33-uploader: _re = re.compile("^(\w+://)?(.*)", re.UNICODE)
chaiml-4d70-fd43-linear-w01-v33-uploader: /usr/lib/python3/dist-packages/S3/FileLists.py:480: SyntaxWarning: invalid escape sequence '\*'
chaiml-4d70-fd43-linear-w01-v33-uploader: wildcard_split_result = re.split("\*|\?", uri_str, maxsplit=1)
chaiml-4d70-fd43-linear-w01-v33-uploader: Bucket 's3://guanaco-vllm-models/' created
chaiml-4d70-fd43-linear-w01-v33-uploader: uploading /dev/shm/model_output to s3://guanaco-vllm-models/chaiml-4d70-fd43-linear-w01-v33
chaiml-4d70-fd43-linear-w01-v33-uploader: cp /dev/shm/model_output/special_tokens_map.json s3://guanaco-vllm-models/chaiml-4d70-fd43-linear-w01-v33/special_tokens_map.json
chaiml-4d70-fd43-linear-w01-v33-uploader: cp /dev/shm/model_output/mergekit_config.yaml s3://guanaco-vllm-models/chaiml-4d70-fd43-linear-w01-v33/mergekit_config.yaml
chaiml-4d70-fd43-linear-w01-v33-uploader: cp /dev/shm/model_output/config.json s3://guanaco-vllm-models/chaiml-4d70-fd43-linear-w01-v33/config.json
chaiml-4d70-fd43-linear-w01-v33-uploader: cp /dev/shm/model_output/README.md s3://guanaco-vllm-models/chaiml-4d70-fd43-linear-w01-v33/README.md
chaiml-4d70-fd43-linear-w01-v33-uploader: cp /dev/shm/model_output/mergekit_config.yml s3://guanaco-vllm-models/chaiml-4d70-fd43-linear-w01-v33/mergekit_config.yml
chaiml-4d70-fd43-linear-w01-v33-uploader: cp /dev/shm/model_output/model.safetensors.index.json s3://guanaco-vllm-models/chaiml-4d70-fd43-linear-w01-v33/model.safetensors.index.json
chaiml-4d70-fd43-linear-w01-v33-uploader: cp /dev/shm/model_output/.gitattributes s3://guanaco-vllm-models/chaiml-4d70-fd43-linear-w01-v33/.gitattributes
chaiml-4d70-fd43-linear-w01-v33-uploader: cp /dev/shm/model_output/tokenizer_config.json s3://guanaco-vllm-models/chaiml-4d70-fd43-linear-w01-v33/tokenizer_config.json
HTTP Request: %s %s "%s %d %s"
Unable to record family friendly update due to error: Invalid JSON input: Expecting value: line 1 column 1 (char 0)
chaiml-4d70-fd43-linear-w01-v33-uploader: cp /dev/shm/model_output/model-00004-of-00005.safetensors s3://guanaco-vllm-models/chaiml-4d70-fd43-linear-w01-v33/model-00004-of-00005.safetensors
chaiml-4d70-fd43-linear-w01-v33-uploader: cp /dev/shm/model_output/model-00005-of-00005.safetensors s3://guanaco-vllm-models/chaiml-4d70-fd43-linear-w01-v33/model-00005-of-00005.safetensors
chaiml-4d70-fd43-linear-w01-v33-uploader: cp /dev/shm/model_output/model-00003-of-00005.safetensors s3://guanaco-vllm-models/chaiml-4d70-fd43-linear-w01-v33/model-00003-of-00005.safetensors
chaiml-4d70-fd43-linear-w01-v33-uploader: cp /dev/shm/model_output/model-00002-of-00005.safetensors s3://guanaco-vllm-models/chaiml-4d70-fd43-linear-w01-v33/model-00002-of-00005.safetensors
chaiml-4d70-fd43-linear-w01-v33-uploader: cp /dev/shm/model_output/model-00001-of-00005.safetensors s3://guanaco-vllm-models/chaiml-4d70-fd43-linear-w01-v33/model-00001-of-00005.safetensors
Job chaiml-4d70-fd43-linear-w01-v33-uploader completed after 174.82s with status: succeeded
Stopping job with name chaiml-4d70-fd43-linear-w01-v33-uploader
Pipeline stage VLLMUploader completed in 175.35s
run pipeline stage %s
Running pipeline stage VLLMTemplater
Pipeline stage VLLMTemplater completed in 0.27s
run pipeline stage %s
Running pipeline stage VLLMDeployer
Creating inference service chaiml-4d70-fd43-linear-w01-v33
Waiting for inference service chaiml-4d70-fd43-linear-w01-v33 to be ready
HTTP Request: %s %s "%s %d %s"
Inference service chaiml-4d70-fd43-linear-w01-v33 ready after 261.83242177963257s
Pipeline stage VLLMDeployer completed in 262.58s
run pipeline stage %s
Running pipeline stage StressChecker
Received healthy response to inference request in 3.0844192504882812s
Received healthy response to inference request in 2.9333815574645996s
Received healthy response to inference request in 2.8076562881469727s
Received healthy response to inference request in 2.7244644165039062s
Received healthy response to inference request in 2.7713873386383057s
Received healthy response to inference request in 2.6937785148620605s
Received healthy response to inference request in 3.036090612411499s
Received healthy response to inference request in 2.9866247177124023s
Received healthy response to inference request in 3.199129104614258s
Received healthy response to inference request in 2.909358024597168s
Received healthy response to inference request in 2.7616021633148193s
Received healthy response to inference request in 2.768913984298706s
Received healthy response to inference request in 2.8544793128967285s
Received healthy response to inference request in 2.728353977203369s
Received healthy response to inference request in 3.0906786918640137s
Received healthy response to inference request in 2.7258543968200684s
Received healthy response to inference request in 3.019864320755005s
Received healthy response to inference request in 2.94773530960083s
Received healthy response to inference request in 2.841742515563965s
Received healthy response to inference request in 3.062133550643921s
Received healthy response to inference request in 3.0154337882995605s
Received healthy response to inference request in 3.0308728218078613s
Received healthy response to inference request in 2.7280566692352295s
Received healthy response to inference request in 2.8182854652404785s
Received healthy response to inference request in 3.340975046157837s
Received healthy response to inference request in 2.9421186447143555s
Received healthy response to inference request in 2.827510118484497s
Received healthy response to inference request in 2.9485650062561035s
Received healthy response to inference request in 3.0128884315490723s
Received healthy response to inference request in 3.0696802139282227s
30 requests
0 failed requests
5th percentile: 2.725089907646179
10th percentile: 2.7278364419937136
20th percentile: 2.767451620101929
30th percentile: 2.815096712112427
40th percentile: 2.849384593963623
50th percentile: 2.9377501010894775
60th percentile: 2.963788890838623
70th percentile: 3.016762948036194
80th percentile: 3.0412992000579835
90th percentile: 3.0850451946258546
95th percentile: 3.1503264188766478
99th percentile: 3.2998397231101992
mean time: 2.9227344751358033
Pipeline stage StressChecker completed in 90.15s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyTriggerPipeline
run_pipeline:run_in_cloud %s
starting trigger_guanaco_pipeline args=%s
triggered trigger_guanaco_pipeline args=%s
Pipeline stage OfflineFamilyFriendlyTriggerPipeline completed in 0.66s
Shutdown handler de-registered
chaiml-4d70-fd43-linear-w01_v33 status is now deployed due to DeploymentManager action
chaiml-4d70-fd43-linear-w01_v33 status is now inactive due to auto deactivation removed underperforming models
chaiml-4d70-fd43-linear-w01_v33 status is now torndown due to DeploymentManager action