Shutdown handler not registered because Python interpreter is not running in the main thread
run pipeline %s
run pipeline stage %s
Running pipeline stage VLLMUploader
Starting job with name chaiml-bol-v6-opusdv1b-lr-759-v4-uploader
Waiting for job on chaiml-bol-v6-opusdv1b-lr-759-v4-uploader to finish
chaiml-bol-v6-opusdv1b-lr-759-v4-uploader: Checking if ChaiML/bol-v6-opusdv1b-lr1e5ep2r64g4b01-W4A16 already exists in ChaiML
chaiml-bol-v6-opusdv1b-lr-759-v4-uploader: Model already exists. Downloading to /dev/shm/model_output...
chaiml-bol-v6-opusdv1b-lr-759-v4-uploader: Downloading snapshot of ChaiML/bol-v6-opusdv1b-lr1e5ep2r64g4b01-W4A16...
chaiml-bol-v6-opusdv1b-lr-759-v4-uploader:
Fetching 39 files: 0%| | 0/39 [00:00<?, ?it/s]
Fetching 39 files: 3%|▎ | 1/39 [00:00<00:12, 3.10it/s]
Fetching 39 files: 18%|█▊ | 7/39 [00:12<00:57, 1.79s/it]
Fetching 39 files: 21%|██ | 8/39 [00:12<00:48, 1.56s/it]
Fetching 39 files: 23%|██▎ | 9/39 [00:14<00:46, 1.54s/it]
Fetching 39 files: 33%|███▎ | 13/39 [00:14<00:20, 1.28it/s]
Fetching 39 files: 36%|███▌ | 14/39 [00:14<00:17, 1.45it/s]
Fetching 39 files: 38%|███▊ | 15/39 [00:23<00:52, 2.17s/it]
Fetching 39 files: 41%|████ | 16/39 [00:24<00:46, 2.00s/it]
Fetching 39 files: 44%|████▎ | 17/39 [00:25<00:36, 1.64s/it]
Fetching 39 files: 49%|████▊ | 19/39 [00:27<00:27, 1.36s/it]
Fetching 39 files: 51%|█████▏ | 20/39 [00:27<00:21, 1.15s/it]
Fetching 39 files: 54%|█████▍ | 21/39 [00:27<00:17, 1.03it/s]
Fetching 39 files: 59%|█████▉ | 23/39 [00:35<00:32, 2.05s/it]
Fetching 39 files: 62%|██████▏ | 24/39 [00:37<00:32, 2.15s/it]
Fetching 39 files: 64%|██████▍ | 25/39 [00:38<00:25, 1.84s/it]
Fetching 39 files: 67%|██████▋ | 26/39 [00:38<00:18, 1.44s/it]
Fetching 39 files: 69%|██████▉ | 27/39 [00:40<00:17, 1.45s/it]
Fetching 39 files: 72%|███████▏ | 28/39 [00:40<00:12, 1.12s/it]
Fetching 39 files: 74%|███████▍ | 29/39 [00:41<00:10, 1.00s/it]
Fetching 39 files: 79%|███████▉ | 31/39 [00:43<00:08, 1.02s/it]
Fetching 39 files: 82%|████████▏ | 32/39 [00:43<00:06, 1.10it/s]
Fetching 39 files: 100%|██████████| 39/39 [00:43<00:00, 1.12s/it]
chaiml-bol-v6-opusdv1b-lr-759-v4-uploader: Downloaded in 43.993s
chaiml-bol-v6-opusdv1b-lr-759-v4-uploader: Processed model ChaiML/bol-v6-opusdv1b-lr1e5ep2r64g4b01 in 44.526s
chaiml-bol-v6-opusdv1b-lr-759-v4-uploader: creating bucket guanaco-vllm-models
chaiml-bol-v6-opusdv1b-lr-759-v4-uploader: /usr/lib/python3/dist-packages/S3/BaseUtils.py:56: SyntaxWarning: invalid escape sequence '\.'
chaiml-bol-v6-opusdv1b-lr-759-v4-uploader: RE_S3_DATESTRING = re.compile('\.[0-9]*(?:[Z\\-\\+]*?)')
chaiml-bol-v6-opusdv1b-lr-759-v4-uploader: /usr/lib/python3/dist-packages/S3/BaseUtils.py:57: SyntaxWarning: invalid escape sequence '\s'
chaiml-bol-v6-opusdv1b-lr-759-v4-uploader: RE_XML_NAMESPACE = re.compile(b'^(<?[^>]+?>\s*|\s*)(<\w+) xmlns=[\'"](https?://[^\'"]+)[\'"]', re.MULTILINE)
chaiml-bol-v6-opusdv1b-lr-759-v4-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:240: SyntaxWarning: invalid escape sequence '\.'
chaiml-bol-v6-opusdv1b-lr-759-v4-uploader: invalid = re.search("([^a-z0-9\.-])", bucket, re.UNICODE)
chaiml-bol-v6-opusdv1b-lr-759-v4-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:244: SyntaxWarning: invalid escape sequence '\.'
chaiml-bol-v6-opusdv1b-lr-759-v4-uploader: invalid = re.search("([^A-Za-z0-9\._-])", bucket, re.UNICODE)
chaiml-bol-v6-opusdv1b-lr-759-v4-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:255: SyntaxWarning: invalid escape sequence '\.'
chaiml-bol-v6-opusdv1b-lr-759-v4-uploader: if re.search("-\.", bucket, re.UNICODE):
chaiml-bol-v6-opusdv1b-lr-759-v4-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:257: SyntaxWarning: invalid escape sequence '\.'
chaiml-bol-v6-opusdv1b-lr-759-v4-uploader: if re.search("\.\.", bucket, re.UNICODE):
chaiml-bol-v6-opusdv1b-lr-759-v4-uploader: /usr/lib/python3/dist-packages/S3/S3Uri.py:155: SyntaxWarning: invalid escape sequence '\w'
chaiml-bol-v6-opusdv1b-lr-759-v4-uploader: _re = re.compile("^(\w+://)?(.*)", re.UNICODE)
chaiml-bol-v6-opusdv1b-lr-759-v4-uploader: /usr/lib/python3/dist-packages/S3/FileLists.py:480: SyntaxWarning: invalid escape sequence '\*'
chaiml-bol-v6-opusdv1b-lr-759-v4-uploader: wildcard_split_result = re.split("\*|\?", uri_str, maxsplit=1)
chaiml-bol-v6-opusdv1b-lr-759-v4-uploader: Bucket 's3://guanaco-vllm-models/' created
chaiml-bol-v6-opusdv1b-lr-759-v4-uploader: uploading /dev/shm/model_output to s3://guanaco-vllm-models/chaiml-bol-v6-opusdv1b-lr-759-v4
chaiml-bol-v6-opusdv1b-lr-759-v4-uploader: cp /dev/shm/model_output/added_tokens.json s3://guanaco-vllm-models/chaiml-bol-v6-opusdv1b-lr-759-v4/added_tokens.json
chaiml-bol-v6-opusdv1b-lr-759-v4-uploader: cp /dev/shm/model_output/tokenizer_config.json s3://guanaco-vllm-models/chaiml-bol-v6-opusdv1b-lr-759-v4/tokenizer_config.json
chaiml-bol-v6-opusdv1b-lr-759-v4-uploader: cp /dev/shm/model_output/.gitattributes s3://guanaco-vllm-models/chaiml-bol-v6-opusdv1b-lr-759-v4/.gitattributes
chaiml-bol-v6-opusdv1b-lr-759-v4-uploader: cp /dev/shm/model_output/generation_config.json s3://guanaco-vllm-models/chaiml-bol-v6-opusdv1b-lr-759-v4/generation_config.json
chaiml-bol-v6-opusdv1b-lr-759-v4-uploader: cp /dev/shm/model_output/special_tokens_map.json s3://guanaco-vllm-models/chaiml-bol-v6-opusdv1b-lr-759-v4/special_tokens_map.json
chaiml-bol-v6-opusdv1b-lr-759-v4-uploader: cp /dev/shm/model_output/quantization_config.json s3://guanaco-vllm-models/chaiml-bol-v6-opusdv1b-lr-759-v4/quantization_config.json
chaiml-bol-v6-opusdv1b-lr-759-v4-uploader: cp /dev/shm/model_output/chat_template.jinja s3://guanaco-vllm-models/chaiml-bol-v6-opusdv1b-lr-759-v4/chat_template.jinja
chaiml-bol-v6-opusdv1b-lr-759-v4-uploader: cp /dev/shm/model_output/config.json s3://guanaco-vllm-models/chaiml-bol-v6-opusdv1b-lr-759-v4/config.json
chaiml-bol-v6-opusdv1b-lr-759-v4-uploader: cp /dev/shm/model_output/merges.txt s3://guanaco-vllm-models/chaiml-bol-v6-opusdv1b-lr-759-v4/merges.txt
chaiml-bol-v6-opusdv1b-lr-759-v4-uploader: cp /dev/shm/model_output/model.safetensors.index.json s3://guanaco-vllm-models/chaiml-bol-v6-opusdv1b-lr-759-v4/model.safetensors.index.json
chaiml-bol-v6-opusdv1b-lr-759-v4-uploader: cp /dev/shm/model_output/vocab.json s3://guanaco-vllm-models/chaiml-bol-v6-opusdv1b-lr-759-v4/vocab.json
chaiml-bol-v6-opusdv1b-lr-759-v4-uploader: cp /dev/shm/model_output/model-00027-of-00027.safetensors s3://guanaco-vllm-models/chaiml-bol-v6-opusdv1b-lr-759-v4/model-00027-of-00027.safetensors
HTTP Request: %s %s "%s %d %s"
HTTP Request: %s %s "%s %d %s"
chaiml-bol-v6-opusdv1b-lr-759-v4-uploader: cp /dev/shm/model_output/model-00011-of-00027.safetensors s3://guanaco-vllm-models/chaiml-bol-v6-opusdv1b-lr-759-v4/model-00011-of-00027.safetensors
chaiml-bol-v6-opusdv1b-lr-759-v4-uploader: cp /dev/shm/model_output/model-00018-of-00027.safetensors s3://guanaco-vllm-models/chaiml-bol-v6-opusdv1b-lr-759-v4/model-00018-of-00027.safetensors
chaiml-bol-v6-opusdv1b-lr-759-v4-uploader: cp /dev/shm/model_output/model-00002-of-00027.safetensors s3://guanaco-vllm-models/chaiml-bol-v6-opusdv1b-lr-759-v4/model-00002-of-00027.safetensors
chaiml-bol-v6-opusdv1b-lr-759-v4-uploader: cp /dev/shm/model_output/model-00014-of-00027.safetensors s3://guanaco-vllm-models/chaiml-bol-v6-opusdv1b-lr-759-v4/model-00014-of-00027.safetensors
chaiml-bol-v6-opusdv1b-lr-759-v4-uploader: cp /dev/shm/model_output/model-00020-of-00027.safetensors s3://guanaco-vllm-models/chaiml-bol-v6-opusdv1b-lr-759-v4/model-00020-of-00027.safetensors
chaiml-bol-v6-opusdv1b-lr-759-v4-uploader: cp /dev/shm/model_output/model-00008-of-00027.safetensors s3://guanaco-vllm-models/chaiml-bol-v6-opusdv1b-lr-759-v4/model-00008-of-00027.safetensors
chaiml-bol-v6-opusdv1b-lr-759-v4-uploader: cp /dev/shm/model_output/model-00026-of-00027.safetensors s3://guanaco-vllm-models/chaiml-bol-v6-opusdv1b-lr-759-v4/model-00026-of-00027.safetensors
chaiml-bol-v6-opusdv1b-lr-759-v4-uploader: cp /dev/shm/model_output/model-00019-of-00027.safetensors s3://guanaco-vllm-models/chaiml-bol-v6-opusdv1b-lr-759-v4/model-00019-of-00027.safetensors
chaiml-bol-v6-opusdv1b-lr-759-v4-uploader: cp /dev/shm/model_output/model-00004-of-00027.safetensors s3://guanaco-vllm-models/chaiml-bol-v6-opusdv1b-lr-759-v4/model-00004-of-00027.safetensors
chaiml-bol-v6-opusdv1b-lr-759-v4-uploader: cp /dev/shm/model_output/model-00024-of-00027.safetensors s3://guanaco-vllm-models/chaiml-bol-v6-opusdv1b-lr-759-v4/model-00024-of-00027.safetensors
chaiml-bol-v6-opusdv1b-lr-759-v4-uploader: cp /dev/shm/model_output/model-00016-of-00027.safetensors s3://guanaco-vllm-models/chaiml-bol-v6-opusdv1b-lr-759-v4/model-00016-of-00027.safetensors
chaiml-bol-v6-opusdv1b-lr-759-v4-uploader: cp /dev/shm/model_output/model-00009-of-00027.safetensors s3://guanaco-vllm-models/chaiml-bol-v6-opusdv1b-lr-759-v4/model-00009-of-00027.safetensors
chaiml-bol-v6-opusdv1b-lr-759-v4-uploader: cp /dev/shm/model_output/model-00001-of-00027.safetensors s3://guanaco-vllm-models/chaiml-bol-v6-opusdv1b-lr-759-v4/model-00001-of-00027.safetensors
chaiml-bol-v6-opusdv1b-lr-759-v4-uploader: cp /dev/shm/model_output/model-00023-of-00027.safetensors s3://guanaco-vllm-models/chaiml-bol-v6-opusdv1b-lr-759-v4/model-00023-of-00027.safetensors
chaiml-bol-v6-opusdv1b-lr-759-v4-uploader: cp /dev/shm/model_output/model-00003-of-00027.safetensors s3://guanaco-vllm-models/chaiml-bol-v6-opusdv1b-lr-759-v4/model-00003-of-00027.safetensors
chaiml-bol-v6-opusdv1b-lr-759-v4-uploader: cp /dev/shm/model_output/model-00006-of-00027.safetensors s3://guanaco-vllm-models/chaiml-bol-v6-opusdv1b-lr-759-v4/model-00006-of-00027.safetensors
chaiml-bol-v6-opusdv1b-lr-759-v4-uploader: cp /dev/shm/model_output/model-00013-of-00027.safetensors s3://guanaco-vllm-models/chaiml-bol-v6-opusdv1b-lr-759-v4/model-00013-of-00027.safetensors
chaiml-bol-v6-opusdv1b-lr-759-v4-uploader: cp /dev/shm/model_output/model-00021-of-00027.safetensors s3://guanaco-vllm-models/chaiml-bol-v6-opusdv1b-lr-759-v4/model-00021-of-00027.safetensors
chaiml-bol-v6-opusdv1b-lr-759-v4-uploader: cp /dev/shm/model_output/model-00022-of-00027.safetensors s3://guanaco-vllm-models/chaiml-bol-v6-opusdv1b-lr-759-v4/model-00022-of-00027.safetensors
chaiml-bol-v6-opusdv1b-lr-759-v4-uploader: cp /dev/shm/model_output/model-00025-of-00027.safetensors s3://guanaco-vllm-models/chaiml-bol-v6-opusdv1b-lr-759-v4/model-00025-of-00027.safetensors
chaiml-bol-v6-opusdv1b-lr-759-v4-uploader: cp /dev/shm/model_output/model-00012-of-00027.safetensors s3://guanaco-vllm-models/chaiml-bol-v6-opusdv1b-lr-759-v4/model-00012-of-00027.safetensors
chaiml-bol-v6-opusdv1b-lr-759-v4-uploader: cp /dev/shm/model_output/model-00010-of-00027.safetensors s3://guanaco-vllm-models/chaiml-bol-v6-opusdv1b-lr-759-v4/model-00010-of-00027.safetensors
chaiml-bol-v6-opusdv1b-lr-759-v4-uploader: cp /dev/shm/model_output/model-00017-of-00027.safetensors s3://guanaco-vllm-models/chaiml-bol-v6-opusdv1b-lr-759-v4/model-00017-of-00027.safetensors
chaiml-bol-v6-opusdv1b-lr-759-v4-uploader: cp /dev/shm/model_output/model-00015-of-00027.safetensors s3://guanaco-vllm-models/chaiml-bol-v6-opusdv1b-lr-759-v4/model-00015-of-00027.safetensors
chaiml-bol-v6-opusdv1b-lr-759-v4-uploader: cp /dev/shm/model_output/model-00007-of-00027.safetensors s3://guanaco-vllm-models/chaiml-bol-v6-opusdv1b-lr-759-v4/model-00007-of-00027.safetensors
chaiml-bol-v6-opusdv1b-lr-759-v4-uploader: cp /dev/shm/model_output/model-00005-of-00027.safetensors s3://guanaco-vllm-models/chaiml-bol-v6-opusdv1b-lr-759-v4/model-00005-of-00027.safetensors
Job chaiml-bol-v6-opusdv1b-lr-759-v4-uploader completed after 951.58s with status: succeeded
Stopping job with name chaiml-bol-v6-opusdv1b-lr-759-v4-uploader
Pipeline stage VLLMUploader completed in 951.93s
run pipeline stage %s
Running pipeline stage VLLMTemplater
Pipeline stage VLLMTemplater completed in 0.16s
run pipeline stage %s
Running pipeline stage VLLMDeployer
Creating inference service chaiml-bol-v6-opusdv1b-lr-759-v4
Waiting for inference service chaiml-bol-v6-opusdv1b-lr-759-v4 to be ready
HTTP Request: %s %s "%s %d %s"
HTTP Request: %s %s "%s %d %s"
Inference service chaiml-bol-v6-opusdv1b-lr-759-v4 ready after 1030.8029699325562s
Pipeline stage VLLMDeployer completed in 1031.64s
run pipeline stage %s
Running pipeline stage StressChecker
Received healthy response to inference request in 2.4397103786468506s
Received healthy response to inference request in 2.208319664001465s
Received healthy response to inference request in 1.7046668529510498s
Received healthy response to inference request in 1.7181527614593506s
Received healthy response to inference request in 1.9282853603363037s
Received healthy response to inference request in 1.6729674339294434s
Received healthy response to inference request in 2.239956855773926s
Received healthy response to inference request in 2.001289129257202s
Received healthy response to inference request in 2.658841848373413s
Received healthy response to inference request in 2.1895318031311035s
Received healthy response to inference request in 1.7978243827819824s
Received healthy response to inference request in 2.064875364303589s
Received healthy response to inference request in 2.0757107734680176s
Received healthy response to inference request in 1.8809618949890137s
Received healthy response to inference request in 1.8708393573760986s
Received healthy response to inference request in 2.061613082885742s
Received healthy response to inference request in 1.8259913921356201s
Received healthy response to inference request in 1.8521654605865479s
Received healthy response to inference request in 1.7605078220367432s
Received healthy response to inference request in 1.9748775959014893s
Received healthy response to inference request in 1.8544285297393799s
Received healthy response to inference request in 2.022759437561035s
Received healthy response to inference request in 1.692551612854004s
Received healthy response to inference request in 1.8265600204467773s
HTTP Request: %s %s "%s %d %s"
Received healthy response to inference request in 1.6566061973571777s
Received healthy response to inference request in 2.0120790004730225s
Received healthy response to inference request in 1.7285301685333252s
Received healthy response to inference request in 1.992093563079834s
Received healthy response to inference request in 1.8457083702087402s
Received healthy response to inference request in 1.9269561767578125s
30 requests
0 failed requests
5th percentile: 1.6817803144454957
10th percentile: 1.7034553289413452
20th percentile: 1.7541122913360596
30th percentile: 1.8263894319534302
40th percentile: 1.8535233020782471
50th percentile: 1.903959035873413
60th percentile: 1.981763982772827
70th percentile: 2.015283131599426
80th percentile: 2.0670424461364747
90th percentile: 2.211483383178711
95th percentile: 2.349821293354034
99th percentile: 2.59529372215271
mean time: 1.9495120763778686
Pipeline stage StressChecker completed in 63.14s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyTriggerPipeline
run_pipeline:run_in_cloud %s
starting trigger_guanaco_pipeline args=%s
triggered trigger_guanaco_pipeline args=%s
Pipeline stage OfflineFamilyFriendlyTriggerPipeline completed in 0.68s
Shutdown handler de-registered
chaiml-bol-v6-opusdv1b-lr_759_v4 status is now deployed due to DeploymentManager action
chaiml-bol-v6-opusdv1b-lr_759_v4 status is now inactive due to auto deactivation removed underperforming models