Shutdown handler not registered because Python interpreter is not running in the main thread
run pipeline %s
run pipeline stage %s
Running pipeline stage VLLMUploader
Starting job with name chaiml-kimid-v11-noname-97927-v5-uploader
Waiting for job on chaiml-kimid-v11-noname-97927-v5-uploader to finish
chaiml-kimid-v11-noname-97927-v5-uploader: Using quantization_mode: w4a16
chaiml-kimid-v11-noname-97927-v5-uploader: Checking if ChaiML/kimid-v11-noname-kimidv4ann-lr1e5ep2r64g8b01-W4A16 already exists in ChaiML
chaiml-kimid-v11-noname-97927-v5-uploader: Model already exists. Downloading to /dev/shm/model_output...
chaiml-kimid-v11-noname-97927-v5-uploader: Downloading snapshot of ChaiML/kimid-v11-noname-kimidv4ann-lr1e5ep2r64g8b01-W4A16...
chaiml-kimid-v11-noname-97927-v5-uploader:
Fetching 39 files: 0%| | 0/39 [00:00<?, ?it/s]
Fetching 39 files: 3%|▎ | 1/39 [00:00<00:13, 2.76it/s]
Fetching 39 files: 18%|█▊ | 7/39 [00:13<01:05, 2.06s/it]
Fetching 39 files: 21%|██ | 8/39 [00:15<00:59, 1.92s/it]
Fetching 39 files: 38%|███▊ | 15/39 [00:24<00:37, 1.56s/it]
Fetching 39 files: 41%|████ | 16/39 [00:26<00:36, 1.60s/it]
Fetching 39 files: 44%|████▎ | 17/39 [00:28<00:35, 1.63s/it]
Fetching 39 files: 49%|████▊ | 19/39 [00:30<00:28, 1.40s/it]
Fetching 39 files: 51%|█████▏ | 20/39 [00:30<00:23, 1.24s/it]
Fetching 39 files: 59%|█████▉ | 23/39 [00:38<00:28, 1.80s/it]
Fetching 39 files: 62%|██████▏ | 24/39 [00:40<00:27, 1.85s/it]
Fetching 39 files: 64%|██████▍ | 25/39 [00:40<00:21, 1.52s/it]
Fetching 39 files: 67%|██████▋ | 26/39 [00:41<00:17, 1.34s/it]
Fetching 39 files: 69%|██████▉ | 27/39 [00:42<00:15, 1.28s/it]
Fetching 39 files: 72%|███████▏ | 28/39 [00:43<00:13, 1.25s/it]
Fetching 39 files: 77%|███████▋ | 30/39 [00:44<00:09, 1.05s/it]
Fetching 39 files: 79%|███████▉ | 31/39 [00:46<00:10, 1.27s/it]
Fetching 39 files: 100%|██████████| 39/39 [00:46<00:00, 1.20s/it]
chaiml-kimid-v11-noname-97927-v5-uploader: Downloaded in 46.972s
chaiml-kimid-v11-noname-97927-v5-uploader: Processed model ChaiML/kimid-v11-noname-kimidv4ann-lr1e5ep2r64g8b01 in 47.723s
chaiml-kimid-v11-noname-97927-v5-uploader: creating bucket guanaco-vllm-models
chaiml-kimid-v11-noname-97927-v5-uploader: /usr/lib/python3/dist-packages/S3/BaseUtils.py:56: SyntaxWarning: invalid escape sequence '\.'
chaiml-kimid-v11-noname-97927-v5-uploader: RE_S3_DATESTRING = re.compile('\.[0-9]*(?:[Z\\-\\+]*?)')
chaiml-kimid-v11-noname-97927-v5-uploader: /usr/lib/python3/dist-packages/S3/BaseUtils.py:57: SyntaxWarning: invalid escape sequence '\s'
chaiml-kimid-v11-noname-97927-v5-uploader: RE_XML_NAMESPACE = re.compile(b'^(<?[^>]+?>\s*|\s*)(<\w+) xmlns=[\'"](https?://[^\'"]+)[\'"]', re.MULTILINE)
chaiml-kimid-v11-noname-97927-v5-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:240: SyntaxWarning: invalid escape sequence '\.'
chaiml-kimid-v11-noname-97927-v5-uploader: invalid = re.search("([^a-z0-9\.-])", bucket, re.UNICODE)
chaiml-kimid-v11-noname-97927-v5-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:244: SyntaxWarning: invalid escape sequence '\.'
chaiml-kimid-v11-noname-97927-v5-uploader: invalid = re.search("([^A-Za-z0-9\._-])", bucket, re.UNICODE)
chaiml-kimid-v11-noname-97927-v5-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:255: SyntaxWarning: invalid escape sequence '\.'
chaiml-kimid-v11-noname-97927-v5-uploader: if re.search("-\.", bucket, re.UNICODE):
chaiml-kimid-v11-noname-97927-v5-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:257: SyntaxWarning: invalid escape sequence '\.'
chaiml-kimid-v11-noname-97927-v5-uploader: if re.search("\.\.", bucket, re.UNICODE):
chaiml-kimid-v11-noname-97927-v5-uploader: /usr/lib/python3/dist-packages/S3/S3Uri.py:155: SyntaxWarning: invalid escape sequence '\w'
chaiml-kimid-v11-noname-97927-v5-uploader: _re = re.compile("^(\w+://)?(.*)", re.UNICODE)
chaiml-kimid-v11-noname-97927-v5-uploader: /usr/lib/python3/dist-packages/S3/FileLists.py:480: SyntaxWarning: invalid escape sequence '\*'
chaiml-kimid-v11-noname-97927-v5-uploader: wildcard_split_result = re.split("\*|\?", uri_str, maxsplit=1)
chaiml-kimid-v11-noname-97927-v5-uploader: Bucket 's3://guanaco-vllm-models/' created
chaiml-kimid-v11-noname-97927-v5-uploader: uploading /dev/shm/model_output to s3://guanaco-vllm-models/chaiml-kimid-v11-noname-97927-v5
chaiml-kimid-v11-noname-97927-v5-uploader: cp /dev/shm/model_output/chat_template.jinja s3://guanaco-vllm-models/chaiml-kimid-v11-noname-97927-v5/chat_template.jinja
chaiml-kimid-v11-noname-97927-v5-uploader: cp /dev/shm/model_output/added_tokens.json s3://guanaco-vllm-models/chaiml-kimid-v11-noname-97927-v5/added_tokens.json
chaiml-kimid-v11-noname-97927-v5-uploader: cp /dev/shm/model_output/generation_config.json s3://guanaco-vllm-models/chaiml-kimid-v11-noname-97927-v5/generation_config.json
chaiml-kimid-v11-noname-97927-v5-uploader: cp /dev/shm/model_output/.gitattributes s3://guanaco-vllm-models/chaiml-kimid-v11-noname-97927-v5/.gitattributes
chaiml-kimid-v11-noname-97927-v5-uploader: cp /dev/shm/model_output/tokenizer_config.json s3://guanaco-vllm-models/chaiml-kimid-v11-noname-97927-v5/tokenizer_config.json
chaiml-kimid-v11-noname-97927-v5-uploader: cp /dev/shm/model_output/quantization_config.json s3://guanaco-vllm-models/chaiml-kimid-v11-noname-97927-v5/quantization_config.json
chaiml-kimid-v11-noname-97927-v5-uploader: cp /dev/shm/model_output/special_tokens_map.json s3://guanaco-vllm-models/chaiml-kimid-v11-noname-97927-v5/special_tokens_map.json
chaiml-kimid-v11-noname-97927-v5-uploader: cp /dev/shm/model_output/config.json s3://guanaco-vllm-models/chaiml-kimid-v11-noname-97927-v5/config.json
chaiml-kimid-v11-noname-97927-v5-uploader: cp /dev/shm/model_output/merges.txt s3://guanaco-vllm-models/chaiml-kimid-v11-noname-97927-v5/merges.txt
chaiml-kimid-v11-noname-97927-v5-uploader: cp /dev/shm/model_output/vocab.json s3://guanaco-vllm-models/chaiml-kimid-v11-noname-97927-v5/vocab.json
chaiml-kimid-v11-noname-97927-v5-uploader: cp /dev/shm/model_output/model.safetensors.index.json s3://guanaco-vllm-models/chaiml-kimid-v11-noname-97927-v5/model.safetensors.index.json
chaiml-kimid-v11-noname-97927-v5-uploader: cp /dev/shm/model_output/tokenizer.json s3://guanaco-vllm-models/chaiml-kimid-v11-noname-97927-v5/tokenizer.json
chaiml-kimid-v11-noname-97927-v5-uploader: cp /dev/shm/model_output/model-00027-of-00027.safetensors s3://guanaco-vllm-models/chaiml-kimid-v11-noname-97927-v5/model-00027-of-00027.safetensors
chaiml-kimid-v11-noname-97927-v5-uploader: cp /dev/shm/model_output/model-00015-of-00027.safetensors s3://guanaco-vllm-models/chaiml-kimid-v11-noname-97927-v5/model-00015-of-00027.safetensors
chaiml-kimid-v11-noname-97927-v5-uploader: cp /dev/shm/model_output/model-00019-of-00027.safetensors s3://guanaco-vllm-models/chaiml-kimid-v11-noname-97927-v5/model-00019-of-00027.safetensors
chaiml-kimid-v11-noname-97927-v5-uploader: cp /dev/shm/model_output/model-00012-of-00027.safetensors s3://guanaco-vllm-models/chaiml-kimid-v11-noname-97927-v5/model-00012-of-00027.safetensors
chaiml-kimid-v11-noname-97927-v5-uploader: cp /dev/shm/model_output/model-00013-of-00027.safetensors s3://guanaco-vllm-models/chaiml-kimid-v11-noname-97927-v5/model-00013-of-00027.safetensors
chaiml-kimid-v11-noname-97927-v5-uploader: cp /dev/shm/model_output/model-00017-of-00027.safetensors s3://guanaco-vllm-models/chaiml-kimid-v11-noname-97927-v5/model-00017-of-00027.safetensors
chaiml-kimid-v11-noname-97927-v5-uploader: cp /dev/shm/model_output/model-00025-of-00027.safetensors s3://guanaco-vllm-models/chaiml-kimid-v11-noname-97927-v5/model-00025-of-00027.safetensors
chaiml-kimid-v11-noname-97927-v5-uploader: cp /dev/shm/model_output/model-00005-of-00027.safetensors s3://guanaco-vllm-models/chaiml-kimid-v11-noname-97927-v5/model-00005-of-00027.safetensors
chaiml-kimid-v11-noname-97927-v5-uploader: cp /dev/shm/model_output/model-00021-of-00027.safetensors s3://guanaco-vllm-models/chaiml-kimid-v11-noname-97927-v5/model-00021-of-00027.safetensors
chaiml-kimid-v11-noname-97927-v5-uploader: cp /dev/shm/model_output/model-00022-of-00027.safetensors s3://guanaco-vllm-models/chaiml-kimid-v11-noname-97927-v5/model-00022-of-00027.safetensors
chaiml-kimid-v11-noname-97927-v5-uploader: cp /dev/shm/model_output/model-00011-of-00027.safetensors s3://guanaco-vllm-models/chaiml-kimid-v11-noname-97927-v5/model-00011-of-00027.safetensors
chaiml-kimid-v11-noname-97927-v5-uploader: cp /dev/shm/model_output/model-00002-of-00027.safetensors s3://guanaco-vllm-models/chaiml-kimid-v11-noname-97927-v5/model-00002-of-00027.safetensors
HTTP Request: %s %s "%s %d %s"
chaiml-kimid-v11-noname-97927-v5-uploader: cp /dev/shm/model_output/model-00001-of-00027.safetensors s3://guanaco-vllm-models/chaiml-kimid-v11-noname-97927-v5/model-00001-of-00027.safetensors
chaiml-kimid-v11-noname-97927-v5-uploader: cp /dev/shm/model_output/model-00018-of-00027.safetensors s3://guanaco-vllm-models/chaiml-kimid-v11-noname-97927-v5/model-00018-of-00027.safetensors
chaiml-kimid-v11-noname-97927-v5-uploader: cp /dev/shm/model_output/model-00006-of-00027.safetensors s3://guanaco-vllm-models/chaiml-kimid-v11-noname-97927-v5/model-00006-of-00027.safetensors
chaiml-kimid-v11-noname-97927-v5-uploader: cp /dev/shm/model_output/model-00014-of-00027.safetensors s3://guanaco-vllm-models/chaiml-kimid-v11-noname-97927-v5/model-00014-of-00027.safetensors
chaiml-kimid-v11-noname-97927-v5-uploader: cp /dev/shm/model_output/model-00010-of-00027.safetensors s3://guanaco-vllm-models/chaiml-kimid-v11-noname-97927-v5/model-00010-of-00027.safetensors
chaiml-kimid-v11-noname-97927-v5-uploader: cp /dev/shm/model_output/model-00020-of-00027.safetensors s3://guanaco-vllm-models/chaiml-kimid-v11-noname-97927-v5/model-00020-of-00027.safetensors
chaiml-kimid-v11-noname-97927-v5-uploader: cp /dev/shm/model_output/model-00007-of-00027.safetensors s3://guanaco-vllm-models/chaiml-kimid-v11-noname-97927-v5/model-00007-of-00027.safetensors
chaiml-kimid-v11-noname-97927-v5-uploader: cp /dev/shm/model_output/model-00023-of-00027.safetensors s3://guanaco-vllm-models/chaiml-kimid-v11-noname-97927-v5/model-00023-of-00027.safetensors
chaiml-kimid-v11-noname-97927-v5-uploader: cp /dev/shm/model_output/model-00016-of-00027.safetensors s3://guanaco-vllm-models/chaiml-kimid-v11-noname-97927-v5/model-00016-of-00027.safetensors
chaiml-kimid-v11-noname-97927-v5-uploader: cp /dev/shm/model_output/model-00008-of-00027.safetensors s3://guanaco-vllm-models/chaiml-kimid-v11-noname-97927-v5/model-00008-of-00027.safetensors
chaiml-kimid-v11-noname-97927-v5-uploader: cp /dev/shm/model_output/model-00009-of-00027.safetensors s3://guanaco-vllm-models/chaiml-kimid-v11-noname-97927-v5/model-00009-of-00027.safetensors
chaiml-kimid-v11-noname-97927-v5-uploader: cp /dev/shm/model_output/model-00024-of-00027.safetensors s3://guanaco-vllm-models/chaiml-kimid-v11-noname-97927-v5/model-00024-of-00027.safetensors
HTTP Request: %s %s "%s %d %s"
chaiml-kimid-v11-noname-97927-v5-uploader: cp /dev/shm/model_output/model-00003-of-00027.safetensors s3://guanaco-vllm-models/chaiml-kimid-v11-noname-97927-v5/model-00003-of-00027.safetensors
chaiml-kimid-v11-noname-97927-v5-uploader: cp /dev/shm/model_output/model-00004-of-00027.safetensors s3://guanaco-vllm-models/chaiml-kimid-v11-noname-97927-v5/model-00004-of-00027.safetensors
Job chaiml-kimid-v11-noname-97927-v5-uploader completed after 942.19s with status: succeeded
Stopping job with name chaiml-kimid-v11-noname-97927-v5-uploader
Pipeline stage VLLMUploader completed in 942.54s
run pipeline stage %s
Running pipeline stage VLLMTemplater
Pipeline stage VLLMTemplater completed in 0.15s
run pipeline stage %s
Running pipeline stage VLLMDeployer
Creating inference service chaiml-kimid-v11-noname-97927-v5
Waiting for inference service chaiml-kimid-v11-noname-97927-v5 to be ready
HTTP Request: %s %s "%s %d %s"
HTTP Request: %s %s "%s %d %s"
Unable to record family friendly update due to error: Invalid JSON input: Expecting value: line 1 column 1 (char 0)
HTTP Request: %s %s "%s %d %s"
HTTP Request: %s %s "%s %d %s"
Inference service chaiml-kimid-v11-noname-97927-v5 ready after 990.6010580062866s
Pipeline stage VLLMDeployer completed in 991.08s
run pipeline stage %s
Running pipeline stage StressChecker
Received healthy response to inference request in 1.9145662784576416s
Received healthy response to inference request in 1.8128082752227783s
Received healthy response to inference request in 1.8658647537231445s
Received healthy response to inference request in 1.9287066459655762s
Received healthy response to inference request in 2.4665298461914062s
Received healthy response to inference request in 1.7383449077606201s
Received healthy response to inference request in 1.8674442768096924s
Received healthy response to inference request in 1.764197587966919s
Received healthy response to inference request in 1.9615063667297363s
Received healthy response to inference request in 1.9808878898620605s
Received healthy response to inference request in 1.911344289779663s
Received healthy response to inference request in 2.0257644653320312s
Received healthy response to inference request in 1.9787049293518066s
Received healthy response to inference request in 1.6722304821014404s
Received healthy response to inference request in 1.8573803901672363s
Received healthy response to inference request in 2.065765619277954s
Received healthy response to inference request in 1.7800097465515137s
Received healthy response to inference request in 2.0575435161590576s
Received healthy response to inference request in 1.9576044082641602s
Received healthy response to inference request in 2.2527554035186768s
Received healthy response to inference request in 1.9555368423461914s
Received healthy response to inference request in 1.7803723812103271s
Received healthy response to inference request in 2.1862807273864746s
Received healthy response to inference request in 1.8954191207885742s
Received healthy response to inference request in 1.8771142959594727s
Received healthy response to inference request in 2.0323259830474854s
Received healthy response to inference request in 1.8593721389770508s
Received healthy response to inference request in 2.536773681640625s
Received healthy response to inference request in 1.863361120223999s
Received healthy response to inference request in 1.9430134296417236s
30 requests
0 failed requests
5th percentile: 1.7499786138534545
10th percentile: 1.7784285306930543
20th percentile: 1.8484659671783448
30th percentile: 1.865113663673401
40th percentile: 1.8880971908569337
50th percentile: 1.9216364622116089
60th percentile: 1.9563638687133789
70th percentile: 1.9793598175048828
80th percentile: 2.0373694896698
90th percentile: 2.192928194999695
95th percentile: 2.3703313469886775
99th percentile: 2.516402969360352
mean time: 1.959650993347168
Pipeline stage StressChecker completed in 61.93s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyTriggerPipeline
run_pipeline:run_in_cloud %s
starting trigger_guanaco_pipeline args=%s
triggered trigger_guanaco_pipeline args=%s
Pipeline stage OfflineFamilyFriendlyTriggerPipeline completed in 0.66s
Shutdown handler de-registered
chaiml-kimid-v11-noname_97927_v5 status is now deployed due to DeploymentManager action
chaiml-kimid-v11-noname_97927_v5 status is now inactive due to auto deactivation removed underperforming models
chaiml-kimid-v11-noname_97927_v5 status is now torndown due to DeploymentManager action