Shutdown handler not registered because Python interpreter is not running in the main thread
run pipeline %s
run pipeline stage %s
Running pipeline stage VLLMUploader
Starting job with name chaiml-glm-air-4-5-sft-93550-v11-uploader
Waiting for job on chaiml-glm-air-4-5-sft-93550-v11-uploader to finish
Shutdown handler not registered because Python interpreter is not running in the main thread
run pipeline %s
run pipeline stage %s
Running pipeline stage VLLMUploader
Starting job with name chaiml-glm-air-4-5-sft-2-6195-v6-uploader
Waiting for job on chaiml-glm-air-4-5-sft-2-6195-v6-uploader to finish
chaiml-glm-air-4-5-sft-93550-v11-uploader: Using quantization_mode: none
chaiml-glm-air-4-5-sft-93550-v11-uploader: Downloading snapshot of ChaiML/glm_air_4_5_sft_20260227_q235_e2...
chaiml-glm-air-4-5-sft-2-6195-v6-uploader: Using quantization_mode: none
chaiml-glm-air-4-5-sft-2-6195-v6-uploader: Downloading snapshot of ChaiML/glm_air_4_5_sft_20260227_q235_e1...
chaiml-glm-air-4-5-sft-93550-v11-uploader: Downloaded in 73.273s
chaiml-glm-air-4-5-sft-2-6195-v6-uploader: Downloaded in 75.289s
chaiml-glm-air-4-5-sft-93550-v11-uploader: Processed model ChaiML/glm_air_4_5_sft_20260227_q235_e2 in 149.455s
chaiml-glm-air-4-5-sft-93550-v11-uploader: creating bucket guanaco-vllm-models
chaiml-glm-air-4-5-sft-93550-v11-uploader: /usr/lib/python3/dist-packages/S3/BaseUtils.py:56: SyntaxWarning: invalid escape sequence '\.'
chaiml-glm-air-4-5-sft-93550-v11-uploader: RE_S3_DATESTRING = re.compile('\.[0-9]*(?:[Z\\-\\+]*?)')
chaiml-glm-air-4-5-sft-93550-v11-uploader: /usr/lib/python3/dist-packages/S3/BaseUtils.py:57: SyntaxWarning: invalid escape sequence '\s'
chaiml-glm-air-4-5-sft-93550-v11-uploader: RE_XML_NAMESPACE = re.compile(b'^(<?[^>]+?>\s*|\s*)(<\w+) xmlns=[\'"](https?://[^\'"]+)[\'"]', re.MULTILINE)
chaiml-glm-air-4-5-sft-93550-v11-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:240: SyntaxWarning: invalid escape sequence '\.'
chaiml-glm-air-4-5-sft-93550-v11-uploader: invalid = re.search("([^a-z0-9\.-])", bucket, re.UNICODE)
chaiml-glm-air-4-5-sft-93550-v11-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:244: SyntaxWarning: invalid escape sequence '\.'
chaiml-glm-air-4-5-sft-93550-v11-uploader: invalid = re.search("([^A-Za-z0-9\._-])", bucket, re.UNICODE)
chaiml-glm-air-4-5-sft-93550-v11-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:255: SyntaxWarning: invalid escape sequence '\.'
chaiml-glm-air-4-5-sft-93550-v11-uploader: if re.search("-\.", bucket, re.UNICODE):
chaiml-glm-air-4-5-sft-93550-v11-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:257: SyntaxWarning: invalid escape sequence '\.'
chaiml-glm-air-4-5-sft-93550-v11-uploader: if re.search("\.\.", bucket, re.UNICODE):
chaiml-glm-air-4-5-sft-93550-v11-uploader: /usr/lib/python3/dist-packages/S3/S3Uri.py:155: SyntaxWarning: invalid escape sequence '\w'
chaiml-glm-air-4-5-sft-93550-v11-uploader: _re = re.compile("^(\w+://)?(.*)", re.UNICODE)
chaiml-glm-air-4-5-sft-93550-v11-uploader: /usr/lib/python3/dist-packages/S3/FileLists.py:480: SyntaxWarning: invalid escape sequence '\*'
chaiml-glm-air-4-5-sft-93550-v11-uploader: wildcard_split_result = re.split("\*|\?", uri_str, maxsplit=1)
chaiml-glm-air-4-5-sft-93550-v11-uploader: Bucket 's3://guanaco-vllm-models/' created
chaiml-glm-air-4-5-sft-93550-v11-uploader: uploading /dev/shm/model_output to s3://guanaco-vllm-models/chaiml-glm-air-4-5-sft-93550-v11/default
chaiml-glm-air-4-5-sft-93550-v11-uploader: cp /dev/shm/model_output/.gitattributes s3://guanaco-vllm-models/chaiml-glm-air-4-5-sft-93550-v11/default/.gitattributes
chaiml-glm-air-4-5-sft-93550-v11-uploader: cp /dev/shm/model_output/README.md s3://guanaco-vllm-models/chaiml-glm-air-4-5-sft-93550-v11/default/README.md
chaiml-glm-air-4-5-sft-93550-v11-uploader: cp /dev/shm/model_output/chat_template.jinja s3://guanaco-vllm-models/chaiml-glm-air-4-5-sft-93550-v11/default/chat_template.jinja
chaiml-glm-air-4-5-sft-93550-v11-uploader: cp /dev/shm/model_output/config.json s3://guanaco-vllm-models/chaiml-glm-air-4-5-sft-93550-v11/default/config.json
chaiml-glm-air-4-5-sft-93550-v11-uploader: cp /dev/shm/model_output/special_tokens_map.json s3://guanaco-vllm-models/chaiml-glm-air-4-5-sft-93550-v11/default/special_tokens_map.json
chaiml-glm-air-4-5-sft-93550-v11-uploader: cp /dev/shm/model_output/tokenizer_config.json s3://guanaco-vllm-models/chaiml-glm-air-4-5-sft-93550-v11/default/tokenizer_config.json
chaiml-glm-air-4-5-sft-93550-v11-uploader: cp /dev/shm/model_output/model.safetensors.index.json s3://guanaco-vllm-models/chaiml-glm-air-4-5-sft-93550-v11/default/model.safetensors.index.json
chaiml-glm-air-4-5-sft-93550-v11-uploader: cp /dev/shm/model_output/args.json s3://guanaco-vllm-models/chaiml-glm-air-4-5-sft-93550-v11/default/args.json
chaiml-glm-air-4-5-sft-2-6195-v6-uploader: Processed model ChaiML/glm_air_4_5_sft_20260227_q235_e1 in 153.041s
chaiml-glm-air-4-5-sft-2-6195-v6-uploader: creating bucket guanaco-vllm-models
chaiml-glm-air-4-5-sft-2-6195-v6-uploader: /usr/lib/python3/dist-packages/S3/BaseUtils.py:56: SyntaxWarning: invalid escape sequence '\.'
chaiml-glm-air-4-5-sft-2-6195-v6-uploader: RE_S3_DATESTRING = re.compile('\.[0-9]*(?:[Z\\-\\+]*?)')
chaiml-glm-air-4-5-sft-2-6195-v6-uploader: /usr/lib/python3/dist-packages/S3/BaseUtils.py:57: SyntaxWarning: invalid escape sequence '\s'
chaiml-glm-air-4-5-sft-2-6195-v6-uploader: RE_XML_NAMESPACE = re.compile(b'^(<?[^>]+?>\s*|\s*)(<\w+) xmlns=[\'"](https?://[^\'"]+)[\'"]', re.MULTILINE)
chaiml-glm-air-4-5-sft-2-6195-v6-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:240: SyntaxWarning: invalid escape sequence '\.'
chaiml-glm-air-4-5-sft-2-6195-v6-uploader: invalid = re.search("([^a-z0-9\.-])", bucket, re.UNICODE)
chaiml-glm-air-4-5-sft-2-6195-v6-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:244: SyntaxWarning: invalid escape sequence '\.'
chaiml-glm-air-4-5-sft-2-6195-v6-uploader: invalid = re.search("([^A-Za-z0-9\._-])", bucket, re.UNICODE)
chaiml-glm-air-4-5-sft-2-6195-v6-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:255: SyntaxWarning: invalid escape sequence '\.'
chaiml-glm-air-4-5-sft-2-6195-v6-uploader: if re.search("-\.", bucket, re.UNICODE):
chaiml-glm-air-4-5-sft-2-6195-v6-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:257: SyntaxWarning: invalid escape sequence '\.'
chaiml-glm-air-4-5-sft-2-6195-v6-uploader: if re.search("\.\.", bucket, re.UNICODE):
chaiml-glm-air-4-5-sft-2-6195-v6-uploader: /usr/lib/python3/dist-packages/S3/S3Uri.py:155: SyntaxWarning: invalid escape sequence '\w'
chaiml-glm-air-4-5-sft-2-6195-v6-uploader: _re = re.compile("^(\w+://)?(.*)", re.UNICODE)
chaiml-glm-air-4-5-sft-2-6195-v6-uploader: /usr/lib/python3/dist-packages/S3/FileLists.py:480: SyntaxWarning: invalid escape sequence '\*'
chaiml-glm-air-4-5-sft-2-6195-v6-uploader: wildcard_split_result = re.split("\*|\?", uri_str, maxsplit=1)
chaiml-glm-air-4-5-sft-2-6195-v6-uploader: Bucket 's3://guanaco-vllm-models/' created
chaiml-glm-air-4-5-sft-2-6195-v6-uploader: uploading /dev/shm/model_output to s3://guanaco-vllm-models/chaiml-glm-air-4-5-sft-2-6195-v6/default
chaiml-glm-air-4-5-sft-2-6195-v6-uploader: cp /dev/shm/model_output/.gitattributes s3://guanaco-vllm-models/chaiml-glm-air-4-5-sft-2-6195-v6/default/.gitattributes
chaiml-glm-air-4-5-sft-2-6195-v6-uploader: cp /dev/shm/model_output/chat_template.jinja s3://guanaco-vllm-models/chaiml-glm-air-4-5-sft-2-6195-v6/default/chat_template.jinja
chaiml-glm-air-4-5-sft-2-6195-v6-uploader: cp /dev/shm/model_output/config.json s3://guanaco-vllm-models/chaiml-glm-air-4-5-sft-2-6195-v6/default/config.json
chaiml-glm-air-4-5-sft-2-6195-v6-uploader: cp /dev/shm/model_output/args.json s3://guanaco-vllm-models/chaiml-glm-air-4-5-sft-2-6195-v6/default/args.json
chaiml-glm-air-4-5-sft-2-6195-v6-uploader: cp /dev/shm/model_output/README.md s3://guanaco-vllm-models/chaiml-glm-air-4-5-sft-2-6195-v6/default/README.md
chaiml-glm-air-4-5-sft-2-6195-v6-uploader: cp /dev/shm/model_output/special_tokens_map.json s3://guanaco-vllm-models/chaiml-glm-air-4-5-sft-2-6195-v6/default/special_tokens_map.json
chaiml-glm-air-4-5-sft-2-6195-v6-uploader: cp /dev/shm/model_output/tokenizer_config.json s3://guanaco-vllm-models/chaiml-glm-air-4-5-sft-2-6195-v6/default/tokenizer_config.json
chaiml-glm-air-4-5-sft-2-6195-v6-uploader: cp /dev/shm/model_output/model.safetensors.index.json s3://guanaco-vllm-models/chaiml-glm-air-4-5-sft-2-6195-v6/default/model.safetensors.index.json
chaiml-glm-air-4-5-sft-2-6195-v6-uploader: cp /dev/shm/model_output/tokenizer.json s3://guanaco-vllm-models/chaiml-glm-air-4-5-sft-2-6195-v6/default/tokenizer.json
chaiml-glm-air-4-5-sft-93550-v11-uploader: cp /dev/shm/model_output/model-00043-of-00043.safetensors s3://guanaco-vllm-models/chaiml-glm-air-4-5-sft-93550-v11/default/model-00043-of-00043.safetensors
chaiml-glm-air-4-5-sft-2-6195-v6-uploader: cp /dev/shm/model_output/model-00043-of-00043.safetensors s3://guanaco-vllm-models/chaiml-glm-air-4-5-sft-2-6195-v6/default/model-00043-of-00043.safetensors
chaiml-glm-air-4-5-sft-93550-v11-uploader: cp /dev/shm/model_output/model-00010-of-00043.safetensors s3://guanaco-vllm-models/chaiml-glm-air-4-5-sft-93550-v11/default/model-00010-of-00043.safetensors
chaiml-glm-air-4-5-sft-93550-v11-uploader: cp /dev/shm/model_output/model-00007-of-00043.safetensors s3://guanaco-vllm-models/chaiml-glm-air-4-5-sft-93550-v11/default/model-00007-of-00043.safetensors
chaiml-glm-air-4-5-sft-93550-v11-uploader: cp /dev/shm/model_output/model-00036-of-00043.safetensors s3://guanaco-vllm-models/chaiml-glm-air-4-5-sft-93550-v11/default/model-00036-of-00043.safetensors
chaiml-glm-air-4-5-sft-93550-v11-uploader: cp /dev/shm/model_output/model-00018-of-00043.safetensors s3://guanaco-vllm-models/chaiml-glm-air-4-5-sft-93550-v11/default/model-00018-of-00043.safetensors
chaiml-glm-air-4-5-sft-93550-v11-uploader: cp /dev/shm/model_output/model-00028-of-00043.safetensors s3://guanaco-vllm-models/chaiml-glm-air-4-5-sft-93550-v11/default/model-00028-of-00043.safetensors
chaiml-glm-air-4-5-sft-93550-v11-uploader: cp /dev/shm/model_output/model-00031-of-00043.safetensors s3://guanaco-vllm-models/chaiml-glm-air-4-5-sft-93550-v11/default/model-00031-of-00043.safetensors
chaiml-glm-air-4-5-sft-93550-v11-uploader: cp /dev/shm/model_output/model-00009-of-00043.safetensors s3://guanaco-vllm-models/chaiml-glm-air-4-5-sft-93550-v11/default/model-00009-of-00043.safetensors
chaiml-glm-air-4-5-sft-93550-v11-uploader: cp /dev/shm/model_output/model-00015-of-00043.safetensors s3://guanaco-vllm-models/chaiml-glm-air-4-5-sft-93550-v11/default/model-00015-of-00043.safetensors
chaiml-glm-air-4-5-sft-93550-v11-uploader: cp /dev/shm/model_output/model-00005-of-00043.safetensors s3://guanaco-vllm-models/chaiml-glm-air-4-5-sft-93550-v11/default/model-00005-of-00043.safetensors
chaiml-glm-air-4-5-sft-93550-v11-uploader: cp /dev/shm/model_output/model-00030-of-00043.safetensors s3://guanaco-vllm-models/chaiml-glm-air-4-5-sft-93550-v11/default/model-00030-of-00043.safetensors
chaiml-glm-air-4-5-sft-93550-v11-uploader: cp /dev/shm/model_output/model-00006-of-00043.safetensors s3://guanaco-vllm-models/chaiml-glm-air-4-5-sft-93550-v11/default/model-00006-of-00043.safetensors
chaiml-glm-air-4-5-sft-93550-v11-uploader: cp /dev/shm/model_output/model-00004-of-00043.safetensors s3://guanaco-vllm-models/chaiml-glm-air-4-5-sft-93550-v11/default/model-00004-of-00043.safetensors
chaiml-glm-air-4-5-sft-93550-v11-uploader: cp /dev/shm/model_output/model-00033-of-00043.safetensors s3://guanaco-vllm-models/chaiml-glm-air-4-5-sft-93550-v11/default/model-00033-of-00043.safetensors
chaiml-glm-air-4-5-sft-93550-v11-uploader: cp /dev/shm/model_output/model-00032-of-00043.safetensors s3://guanaco-vllm-models/chaiml-glm-air-4-5-sft-93550-v11/default/model-00032-of-00043.safetensors
chaiml-glm-air-4-5-sft-93550-v11-uploader: cp /dev/shm/model_output/model-00012-of-00043.safetensors s3://guanaco-vllm-models/chaiml-glm-air-4-5-sft-93550-v11/default/model-00012-of-00043.safetensors
chaiml-glm-air-4-5-sft-93550-v11-uploader: cp /dev/shm/model_output/model-00040-of-00043.safetensors s3://guanaco-vllm-models/chaiml-glm-air-4-5-sft-93550-v11/default/model-00040-of-00043.safetensors
chaiml-glm-air-4-5-sft-93550-v11-uploader: cp /dev/shm/model_output/model-00016-of-00043.safetensors s3://guanaco-vllm-models/chaiml-glm-air-4-5-sft-93550-v11/default/model-00016-of-00043.safetensors
chaiml-glm-air-4-5-sft-93550-v11-uploader: cp /dev/shm/model_output/model-00023-of-00043.safetensors s3://guanaco-vllm-models/chaiml-glm-air-4-5-sft-93550-v11/default/model-00023-of-00043.safetensors
chaiml-glm-air-4-5-sft-93550-v11-uploader: cp /dev/shm/model_output/model-00041-of-00043.safetensors s3://guanaco-vllm-models/chaiml-glm-air-4-5-sft-93550-v11/default/model-00041-of-00043.safetensors
chaiml-glm-air-4-5-sft-93550-v11-uploader: cp /dev/shm/model_output/model-00008-of-00043.safetensors s3://guanaco-vllm-models/chaiml-glm-air-4-5-sft-93550-v11/default/model-00008-of-00043.safetensors
chaiml-glm-air-4-5-sft-93550-v11-uploader: cp /dev/shm/model_output/model-00020-of-00043.safetensors s3://guanaco-vllm-models/chaiml-glm-air-4-5-sft-93550-v11/default/model-00020-of-00043.safetensors
chaiml-glm-air-4-5-sft-93550-v11-uploader: cp /dev/shm/model_output/model-00042-of-00043.safetensors s3://guanaco-vllm-models/chaiml-glm-air-4-5-sft-93550-v11/default/model-00042-of-00043.safetensors
chaiml-glm-air-4-5-sft-93550-v11-uploader: cp /dev/shm/model_output/model-00021-of-00043.safetensors s3://guanaco-vllm-models/chaiml-glm-air-4-5-sft-93550-v11/default/model-00021-of-00043.safetensors
chaiml-glm-air-4-5-sft-93550-v11-uploader: cp /dev/shm/model_output/model-00001-of-00043.safetensors s3://guanaco-vllm-models/chaiml-glm-air-4-5-sft-93550-v11/default/model-00001-of-00043.safetensors
chaiml-glm-air-4-5-sft-93550-v11-uploader: cp /dev/shm/model_output/model-00017-of-00043.safetensors s3://guanaco-vllm-models/chaiml-glm-air-4-5-sft-93550-v11/default/model-00017-of-00043.safetensors
chaiml-glm-air-4-5-sft-93550-v11-uploader: cp /dev/shm/model_output/model-00022-of-00043.safetensors s3://guanaco-vllm-models/chaiml-glm-air-4-5-sft-93550-v11/default/model-00022-of-00043.safetensors
chaiml-glm-air-4-5-sft-93550-v11-uploader: cp /dev/shm/model_output/model-00037-of-00043.safetensors s3://guanaco-vllm-models/chaiml-glm-air-4-5-sft-93550-v11/default/model-00037-of-00043.safetensors
chaiml-glm-air-4-5-sft-93550-v11-uploader: cp /dev/shm/model_output/model-00034-of-00043.safetensors s3://guanaco-vllm-models/chaiml-glm-air-4-5-sft-93550-v11/default/model-00034-of-00043.safetensors
chaiml-glm-air-4-5-sft-2-6195-v6-uploader: cp /dev/shm/model_output/model-00014-of-00043.safetensors s3://guanaco-vllm-models/chaiml-glm-air-4-5-sft-2-6195-v6/default/model-00014-of-00043.safetensors
chaiml-glm-air-4-5-sft-93550-v11-uploader: cp /dev/shm/model_output/model-00026-of-00043.safetensors s3://guanaco-vllm-models/chaiml-glm-air-4-5-sft-93550-v11/default/model-00026-of-00043.safetensors
chaiml-glm-air-4-5-sft-2-6195-v6-uploader: cp /dev/shm/model_output/model-00036-of-00043.safetensors s3://guanaco-vllm-models/chaiml-glm-air-4-5-sft-2-6195-v6/default/model-00036-of-00043.safetensors
chaiml-glm-air-4-5-sft-93550-v11-uploader: cp /dev/shm/model_output/model-00039-of-00043.safetensors s3://guanaco-vllm-models/chaiml-glm-air-4-5-sft-93550-v11/default/model-00039-of-00043.safetensors
chaiml-glm-air-4-5-sft-2-6195-v6-uploader: cp /dev/shm/model_output/model-00033-of-00043.safetensors s3://guanaco-vllm-models/chaiml-glm-air-4-5-sft-2-6195-v6/default/model-00033-of-00043.safetensors
chaiml-glm-air-4-5-sft-93550-v11-uploader: cp /dev/shm/model_output/model-00038-of-00043.safetensors s3://guanaco-vllm-models/chaiml-glm-air-4-5-sft-93550-v11/default/model-00038-of-00043.safetensors
chaiml-glm-air-4-5-sft-2-6195-v6-uploader: cp /dev/shm/model_output/model-00008-of-00043.safetensors s3://guanaco-vllm-models/chaiml-glm-air-4-5-sft-2-6195-v6/default/model-00008-of-00043.safetensors
chaiml-glm-air-4-5-sft-93550-v11-uploader: cp /dev/shm/model_output/model-00011-of-00043.safetensors s3://guanaco-vllm-models/chaiml-glm-air-4-5-sft-93550-v11/default/model-00011-of-00043.safetensors
chaiml-glm-air-4-5-sft-2-6195-v6-uploader: cp /dev/shm/model_output/model-00039-of-00043.safetensors s3://guanaco-vllm-models/chaiml-glm-air-4-5-sft-2-6195-v6/default/model-00039-of-00043.safetensors
chaiml-glm-air-4-5-sft-93550-v11-uploader: cp /dev/shm/model_output/model-00002-of-00043.safetensors s3://guanaco-vllm-models/chaiml-glm-air-4-5-sft-93550-v11/default/model-00002-of-00043.safetensors
chaiml-glm-air-4-5-sft-2-6195-v6-uploader: cp /dev/shm/model_output/model-00005-of-00043.safetensors s3://guanaco-vllm-models/chaiml-glm-air-4-5-sft-2-6195-v6/default/model-00005-of-00043.safetensors
chaiml-glm-air-4-5-sft-93550-v11-uploader: cp /dev/shm/model_output/model-00019-of-00043.safetensors s3://guanaco-vllm-models/chaiml-glm-air-4-5-sft-93550-v11/default/model-00019-of-00043.safetensors
chaiml-glm-air-4-5-sft-2-6195-v6-uploader: cp /dev/shm/model_output/model-00003-of-00043.safetensors s3://guanaco-vllm-models/chaiml-glm-air-4-5-sft-2-6195-v6/default/model-00003-of-00043.safetensors
chaiml-glm-air-4-5-sft-93550-v11-uploader: cp /dev/shm/model_output/model-00003-of-00043.safetensors s3://guanaco-vllm-models/chaiml-glm-air-4-5-sft-93550-v11/default/model-00003-of-00043.safetensors
chaiml-glm-air-4-5-sft-2-6195-v6-uploader: cp /dev/shm/model_output/model-00010-of-00043.safetensors s3://guanaco-vllm-models/chaiml-glm-air-4-5-sft-2-6195-v6/default/model-00010-of-00043.safetensors
chaiml-glm-air-4-5-sft-93550-v11-uploader: cp /dev/shm/model_output/model-00024-of-00043.safetensors s3://guanaco-vllm-models/chaiml-glm-air-4-5-sft-93550-v11/default/model-00024-of-00043.safetensors
chaiml-glm-air-4-5-sft-2-6195-v6-uploader: cp /dev/shm/model_output/model-00020-of-00043.safetensors s3://guanaco-vllm-models/chaiml-glm-air-4-5-sft-2-6195-v6/default/model-00020-of-00043.safetensors
chaiml-glm-air-4-5-sft-93550-v11-uploader: cp /dev/shm/model_output/model-00027-of-00043.safetensors s3://guanaco-vllm-models/chaiml-glm-air-4-5-sft-93550-v11/default/model-00027-of-00043.safetensors
chaiml-glm-air-4-5-sft-2-6195-v6-uploader: cp /dev/shm/model_output/model-00007-of-00043.safetensors s3://guanaco-vllm-models/chaiml-glm-air-4-5-sft-2-6195-v6/default/model-00007-of-00043.safetensors
chaiml-glm-air-4-5-sft-93550-v11-uploader: cp /dev/shm/model_output/model-00029-of-00043.safetensors s3://guanaco-vllm-models/chaiml-glm-air-4-5-sft-93550-v11/default/model-00029-of-00043.safetensors
chaiml-glm-air-4-5-sft-2-6195-v6-uploader: cp /dev/shm/model_output/model-00040-of-00043.safetensors s3://guanaco-vllm-models/chaiml-glm-air-4-5-sft-2-6195-v6/default/model-00040-of-00043.safetensors
chaiml-glm-air-4-5-sft-93550-v11-uploader: cp /dev/shm/model_output/model-00013-of-00043.safetensors s3://guanaco-vllm-models/chaiml-glm-air-4-5-sft-93550-v11/default/model-00013-of-00043.safetensors
chaiml-glm-air-4-5-sft-2-6195-v6-uploader: cp /dev/shm/model_output/model-00025-of-00043.safetensors s3://guanaco-vllm-models/chaiml-glm-air-4-5-sft-2-6195-v6/default/model-00025-of-00043.safetensors
chaiml-glm-air-4-5-sft-93550-v11-uploader: cp /dev/shm/model_output/model-00014-of-00043.safetensors s3://guanaco-vllm-models/chaiml-glm-air-4-5-sft-93550-v11/default/model-00014-of-00043.safetensors
chaiml-glm-air-4-5-sft-2-6195-v6-uploader: cp /dev/shm/model_output/model-00015-of-00043.safetensors s3://guanaco-vllm-models/chaiml-glm-air-4-5-sft-2-6195-v6/default/model-00015-of-00043.safetensors
chaiml-glm-air-4-5-sft-93550-v11-uploader: cp /dev/shm/model_output/model-00025-of-00043.safetensors s3://guanaco-vllm-models/chaiml-glm-air-4-5-sft-93550-v11/default/model-00025-of-00043.safetensors
chaiml-glm-air-4-5-sft-2-6195-v6-uploader: cp /dev/shm/model_output/model-00012-of-00043.safetensors s3://guanaco-vllm-models/chaiml-glm-air-4-5-sft-2-6195-v6/default/model-00012-of-00043.safetensors
chaiml-glm-air-4-5-sft-93550-v11-uploader: cp /dev/shm/model_output/model-00035-of-00043.safetensors s3://guanaco-vllm-models/chaiml-glm-air-4-5-sft-93550-v11/default/model-00035-of-00043.safetensors
chaiml-glm-air-4-5-sft-2-6195-v6-uploader: cp /dev/shm/model_output/model-00019-of-00043.safetensors s3://guanaco-vllm-models/chaiml-glm-air-4-5-sft-2-6195-v6/default/model-00019-of-00043.safetensors
chaiml-glm-air-4-5-sft-2-6195-v6-uploader: cp /dev/shm/model_output/model-00031-of-00043.safetensors s3://guanaco-vllm-models/chaiml-glm-air-4-5-sft-2-6195-v6/default/model-00031-of-00043.safetensors
chaiml-glm-air-4-5-sft-2-6195-v6-uploader: cp /dev/shm/model_output/model-00006-of-00043.safetensors s3://guanaco-vllm-models/chaiml-glm-air-4-5-sft-2-6195-v6/default/model-00006-of-00043.safetensors
chaiml-glm-air-4-5-sft-2-6195-v6-uploader: cp /dev/shm/model_output/model-00009-of-00043.safetensors s3://guanaco-vllm-models/chaiml-glm-air-4-5-sft-2-6195-v6/default/model-00009-of-00043.safetensors
chaiml-glm-air-4-5-sft-2-6195-v6-uploader: cp /dev/shm/model_output/model-00038-of-00043.safetensors s3://guanaco-vllm-models/chaiml-glm-air-4-5-sft-2-6195-v6/default/model-00038-of-00043.safetensors
chaiml-glm-air-4-5-sft-2-6195-v6-uploader: cp /dev/shm/model_output/model-00032-of-00043.safetensors s3://guanaco-vllm-models/chaiml-glm-air-4-5-sft-2-6195-v6/default/model-00032-of-00043.safetensors
chaiml-glm-air-4-5-sft-2-6195-v6-uploader: cp /dev/shm/model_output/model-00037-of-00043.safetensors s3://guanaco-vllm-models/chaiml-glm-air-4-5-sft-2-6195-v6/default/model-00037-of-00043.safetensors
chaiml-glm-air-4-5-sft-2-6195-v6-uploader: cp /dev/shm/model_output/model-00030-of-00043.safetensors s3://guanaco-vllm-models/chaiml-glm-air-4-5-sft-2-6195-v6/default/model-00030-of-00043.safetensors
chaiml-glm-air-4-5-sft-2-6195-v6-uploader: cp /dev/shm/model_output/model-00024-of-00043.safetensors s3://guanaco-vllm-models/chaiml-glm-air-4-5-sft-2-6195-v6/default/model-00024-of-00043.safetensors
chaiml-glm-air-4-5-sft-2-6195-v6-uploader: cp /dev/shm/model_output/model-00035-of-00043.safetensors s3://guanaco-vllm-models/chaiml-glm-air-4-5-sft-2-6195-v6/default/model-00035-of-00043.safetensors
chaiml-glm-air-4-5-sft-2-6195-v6-uploader: cp /dev/shm/model_output/model-00023-of-00043.safetensors s3://guanaco-vllm-models/chaiml-glm-air-4-5-sft-2-6195-v6/default/model-00023-of-00043.safetensors
chaiml-glm-air-4-5-sft-2-6195-v6-uploader: cp /dev/shm/model_output/model-00029-of-00043.safetensors s3://guanaco-vllm-models/chaiml-glm-air-4-5-sft-2-6195-v6/default/model-00029-of-00043.safetensors
chaiml-glm-air-4-5-sft-2-6195-v6-uploader: cp /dev/shm/model_output/model-00042-of-00043.safetensors s3://guanaco-vllm-models/chaiml-glm-air-4-5-sft-2-6195-v6/default/model-00042-of-00043.safetensors
chaiml-glm-air-4-5-sft-2-6195-v6-uploader: cp /dev/shm/model_output/model-00026-of-00043.safetensors s3://guanaco-vllm-models/chaiml-glm-air-4-5-sft-2-6195-v6/default/model-00026-of-00043.safetensors
chaiml-glm-air-4-5-sft-2-6195-v6-uploader: cp /dev/shm/model_output/model-00041-of-00043.safetensors s3://guanaco-vllm-models/chaiml-glm-air-4-5-sft-2-6195-v6/default/model-00041-of-00043.safetensors
chaiml-glm-air-4-5-sft-2-6195-v6-uploader: cp /dev/shm/model_output/model-00034-of-00043.safetensors s3://guanaco-vllm-models/chaiml-glm-air-4-5-sft-2-6195-v6/default/model-00034-of-00043.safetensors
chaiml-glm-air-4-5-sft-2-6195-v6-uploader: cp /dev/shm/model_output/model-00004-of-00043.safetensors s3://guanaco-vllm-models/chaiml-glm-air-4-5-sft-2-6195-v6/default/model-00004-of-00043.safetensors
chaiml-glm-air-4-5-sft-2-6195-v6-uploader: cp /dev/shm/model_output/model-00002-of-00043.safetensors s3://guanaco-vllm-models/chaiml-glm-air-4-5-sft-2-6195-v6/default/model-00002-of-00043.safetensors
chaiml-glm-air-4-5-sft-2-6195-v6-uploader: cp /dev/shm/model_output/model-00013-of-00043.safetensors s3://guanaco-vllm-models/chaiml-glm-air-4-5-sft-2-6195-v6/default/model-00013-of-00043.safetensors
chaiml-glm-air-4-5-sft-2-6195-v6-uploader: cp /dev/shm/model_output/model-00011-of-00043.safetensors s3://guanaco-vllm-models/chaiml-glm-air-4-5-sft-2-6195-v6/default/model-00011-of-00043.safetensors
chaiml-glm-air-4-5-sft-2-6195-v6-uploader: cp /dev/shm/model_output/model-00018-of-00043.safetensors s3://guanaco-vllm-models/chaiml-glm-air-4-5-sft-2-6195-v6/default/model-00018-of-00043.safetensors
chaiml-glm-air-4-5-sft-2-6195-v6-uploader: cp /dev/shm/model_output/model-00017-of-00043.safetensors s3://guanaco-vllm-models/chaiml-glm-air-4-5-sft-2-6195-v6/default/model-00017-of-00043.safetensors
chaiml-glm-air-4-5-sft-2-6195-v6-uploader: cp /dev/shm/model_output/model-00027-of-00043.safetensors s3://guanaco-vllm-models/chaiml-glm-air-4-5-sft-2-6195-v6/default/model-00027-of-00043.safetensors
chaiml-glm-air-4-5-sft-2-6195-v6-uploader: cp /dev/shm/model_output/model-00021-of-00043.safetensors s3://guanaco-vllm-models/chaiml-glm-air-4-5-sft-2-6195-v6/default/model-00021-of-00043.safetensors
chaiml-glm-air-4-5-sft-2-6195-v6-uploader: cp /dev/shm/model_output/model-00001-of-00043.safetensors s3://guanaco-vllm-models/chaiml-glm-air-4-5-sft-2-6195-v6/default/model-00001-of-00043.safetensors
chaiml-glm-air-4-5-sft-2-6195-v6-uploader: cp /dev/shm/model_output/model-00028-of-00043.safetensors s3://guanaco-vllm-models/chaiml-glm-air-4-5-sft-2-6195-v6/default/model-00028-of-00043.safetensors
chaiml-glm-air-4-5-sft-2-6195-v6-uploader: cp /dev/shm/model_output/model-00022-of-00043.safetensors s3://guanaco-vllm-models/chaiml-glm-air-4-5-sft-2-6195-v6/default/model-00022-of-00043.safetensors
chaiml-glm-air-4-5-sft-2-6195-v6-uploader: cp /dev/shm/model_output/model-00016-of-00043.safetensors s3://guanaco-vllm-models/chaiml-glm-air-4-5-sft-2-6195-v6/default/model-00016-of-00043.safetensors
Job chaiml-glm-air-4-5-sft-93550-v11-uploader completed after 228.69s with status: succeeded
Stopping job with name chaiml-glm-air-4-5-sft-93550-v11-uploader
Pipeline stage VLLMUploader completed in 230.26s
run pipeline stage %s
Running pipeline stage VLLMTemplater
Pipeline stage VLLMTemplater completed in 1.41s
run pipeline stage %s
Running pipeline stage VLLMDeployer
Creating inference service chaiml-glm-air-4-5-sft-93550-v11
Waiting for inference service chaiml-glm-air-4-5-sft-93550-v11 to be ready
Job chaiml-glm-air-4-5-sft-2-6195-v6-uploader completed after 231.22s with status: succeeded
Stopping job with name chaiml-glm-air-4-5-sft-2-6195-v6-uploader
Pipeline stage VLLMUploader completed in 233.04s
run pipeline stage %s
Running pipeline stage VLLMTemplater
Pipeline stage VLLMTemplater completed in 1.10s
run pipeline stage %s
Running pipeline stage VLLMDeployer
Creating inference service chaiml-glm-air-4-5-sft-2-6195-v6
Waiting for inference service chaiml-glm-air-4-5-sft-2-6195-v6 to be ready
Inference service chaiml-glm-air-4-5-sft-93550-v11 ready after 547.8920886516571s
Pipeline stage VLLMDeployer completed in 550.04s
run pipeline stage %s
Running pipeline stage StressChecker
Received healthy response to inference request in 3.0628647804260254s
Received healthy response to inference request in 3.3031792640686035s
Received healthy response to inference request in 2.368764638900757s
Received healthy response to inference request in 2.449923515319824s
Received healthy response to inference request in 2.4094319343566895s
Received healthy response to inference request in 2.22678804397583s
Received healthy response to inference request in 2.3433120250701904s
Received healthy response to inference request in 2.1084485054016113s
Received healthy response to inference request in 2.825714111328125s
Received healthy response to inference request in 2.4788005352020264s
Received healthy response to inference request in 2.4199934005737305s
Received healthy response to inference request in 2.399831533432007s
Received healthy response to inference request in 2.7055625915527344s
Received healthy response to inference request in 2.405071258544922s
Received healthy response to inference request in 2.1385738849639893s
Received healthy response to inference request in 2.2945711612701416s
Received healthy response to inference request in 2.738689661026001s
Received healthy response to inference request in 2.1535146236419678s
Received healthy response to inference request in 2.438866138458252s
Received healthy response to inference request in 2.1382088661193848s
Inference service chaiml-glm-air-4-5-sft-2-6195-v6 ready after 600.5047562122345s
Pipeline stage VLLMDeployer completed in 603.50s
run pipeline stage %s
Running pipeline stage StressChecker
Received healthy response to inference request in 2.19260573387146s
Received healthy response to inference request in 2.840985059738159s
Received healthy response to inference request in 2.354938268661499s
Received healthy response to inference request in 2.1615188121795654s
Received healthy response to inference request in 2.2443981170654297s
Received healthy response to inference request in 2.1672744750976562s
Received healthy response to inference request in 2.3331761360168457s
Received healthy response to inference request in 2.2421960830688477s
Received healthy response to inference request in 2.1936254501342773s
Received healthy response to inference request in 2.711482048034668s
Received healthy response to inference request in 2.0743837356567383s
Received healthy response to inference request in 2.0526204109191895s
Received healthy response to inference request in 2.365373373031616s
Received healthy response to inference request in 2.0624561309814453s
Received healthy response to inference request in 2.368744373321533s
Received healthy response to inference request in 2.7929606437683105s
Received healthy response to inference request in 3.0022358894348145s
Received healthy response to inference request in 2.1092302799224854s
30 requests
0 failed requests
Received healthy response to inference request in 2.6998038291931152s
5th percentile: 2.089712882041931
10th percentile: 2.109152102470398
20th percentile: 2.1505264759063722
30th percentile: 2.2168392658233644
40th percentile: 2.317734146118164
50th percentile: 2.361841320991516
60th percentile: 2.401927423477173
Received healthy response to inference request in 2.2673637866973877s
70th percentile: 2.425655221939087
80th percentile: 2.5241529464721686
90th percentile: 2.796235990524292
95th percentile: 2.9561469793319697
99th percentile: 3.2334880638122563
mean time: 2.4042264540990193
Received healthy response to inference request in 2.2224605083465576s
Pipeline stage StressChecker completed in 108.80s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyTriggerPipeline
run_pipeline:run_in_cloud %s
starting trigger_guanaco_pipeline args=%s
Received healthy response to inference request in 2.134348154067993s
triggered trigger_guanaco_pipeline args=%s
Pipeline stage OfflineFamilyFriendlyTriggerPipeline completed in 2.20s
Shutdown handler de-registered
chaiml-glm-air-4-5-sft_93550_v11 status is now deployed due to DeploymentManager action
chaiml-glm-air-4-5-sft_93550_v11 status is now inactive due to auto deactivation removed underperforming models