Shutdown handler not registered because Python interpreter is not running in the main thread
run pipeline %s
run pipeline stage %s
Running pipeline stage VLLMUploader
Starting job with name chaiml-grpo-glm50-combi-44700-v1-uploader
Waiting for job on chaiml-grpo-glm50-combi-44700-v1-uploader to finish
chaiml-grpo-glm50-combi-44700-v1-uploader: {repo_id} is already quantized
chaiml-grpo-glm50-combi-44700-v1-uploader: Using quantization_mode: none
chaiml-grpo-glm50-combi-44700-v1-uploader: Downloading snapshot of ChaiML/grpo-glm50-combine-rm3331-mega-kimi-mix8020-step300-int4...
2026-04-03T21:01:17.271525+00:00 monitor updated for chaiml-grpo-glm50-combi_44700_v1
2026-04-03T21:02:17.357487+00:00 monitor updated for chaiml-grpo-glm50-combi_44700_v1
2026-04-03T21:03:17.441341+00:00 monitor updated for chaiml-grpo-glm50-combi_44700_v1
chaiml-grpo-glm50-combi-44700-v1-uploader: Downloaded in 160.366s
2026-04-03T21:04:17.529988+00:00 monitor updated for chaiml-grpo-glm50-combi_44700_v1
2026-04-03T21:05:17.616406+00:00 monitor updated for chaiml-grpo-glm50-combi_44700_v1
chaiml-grpo-glm50-combi-44700-v1-uploader: Processed model ChaiML/grpo-glm50-combine-rm3331-mega-kimi-mix8020-step300-int4 in 297.573s
chaiml-grpo-glm50-combi-44700-v1-uploader: creating bucket guanaco-vllm-models
chaiml-grpo-glm50-combi-44700-v1-uploader: /usr/lib/python3/dist-packages/S3/BaseUtils.py:56: SyntaxWarning: invalid escape sequence '\.'
chaiml-grpo-glm50-combi-44700-v1-uploader: RE_S3_DATESTRING = re.compile('\.[0-9]*(?:[Z\\-\\+]*?)')
chaiml-grpo-glm50-combi-44700-v1-uploader: /usr/lib/python3/dist-packages/S3/BaseUtils.py:57: SyntaxWarning: invalid escape sequence '\s'
chaiml-grpo-glm50-combi-44700-v1-uploader: RE_XML_NAMESPACE = re.compile(b'^(<?[^>]+?>\s*|\s*)(<\w+) xmlns=[\'"](https?://[^\'"]+)[\'"]', re.MULTILINE)
chaiml-grpo-glm50-combi-44700-v1-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:240: SyntaxWarning: invalid escape sequence '\.'
chaiml-grpo-glm50-combi-44700-v1-uploader: invalid = re.search("([^a-z0-9\.-])", bucket, re.UNICODE)
chaiml-grpo-glm50-combi-44700-v1-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:244: SyntaxWarning: invalid escape sequence '\.'
chaiml-grpo-glm50-combi-44700-v1-uploader: invalid = re.search("([^A-Za-z0-9\._-])", bucket, re.UNICODE)
chaiml-grpo-glm50-combi-44700-v1-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:255: SyntaxWarning: invalid escape sequence '\.'
chaiml-grpo-glm50-combi-44700-v1-uploader: if re.search("-\.", bucket, re.UNICODE):
chaiml-grpo-glm50-combi-44700-v1-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:257: SyntaxWarning: invalid escape sequence '\.'
chaiml-grpo-glm50-combi-44700-v1-uploader: if re.search("\.\.", bucket, re.UNICODE):
chaiml-grpo-glm50-combi-44700-v1-uploader: /usr/lib/python3/dist-packages/S3/S3Uri.py:155: SyntaxWarning: invalid escape sequence '\w'
chaiml-grpo-glm50-combi-44700-v1-uploader: _re = re.compile("^(\w+://)?(.*)", re.UNICODE)
chaiml-grpo-glm50-combi-44700-v1-uploader: /usr/lib/python3/dist-packages/S3/FileLists.py:480: SyntaxWarning: invalid escape sequence '\*'
chaiml-grpo-glm50-combi-44700-v1-uploader: wildcard_split_result = re.split("\*|\?", uri_str, maxsplit=1)
chaiml-grpo-glm50-combi-44700-v1-uploader: Bucket 's3://guanaco-vllm-models/' created
chaiml-grpo-glm50-combi-44700-v1-uploader: uploading /dev/shm/model_output to s3://guanaco-vllm-models/chaiml-grpo-glm50-combi-44700-v1/default
chaiml-grpo-glm50-combi-44700-v1-uploader: cp /dev/shm/model_output/generation_config.json s3://guanaco-vllm-models/chaiml-grpo-glm50-combi-44700-v1/default/generation_config.json
chaiml-grpo-glm50-combi-44700-v1-uploader: cp /dev/shm/model_output/config.json s3://guanaco-vllm-models/chaiml-grpo-glm50-combi-44700-v1/default/config.json
chaiml-grpo-glm50-combi-44700-v1-uploader: cp /dev/shm/model_output/chat_template.jinja s3://guanaco-vllm-models/chaiml-grpo-glm50-combi-44700-v1/default/chat_template.jinja
chaiml-grpo-glm50-combi-44700-v1-uploader: cp /dev/shm/model_output/.gitattributes s3://guanaco-vllm-models/chaiml-grpo-glm50-combi-44700-v1/default/.gitattributes
chaiml-grpo-glm50-combi-44700-v1-uploader: cp /dev/shm/model_output/quantization_config.json s3://guanaco-vllm-models/chaiml-grpo-glm50-combi-44700-v1/default/quantization_config.json
chaiml-grpo-glm50-combi-44700-v1-uploader: cp /dev/shm/model_output/tokenizer_config.json s3://guanaco-vllm-models/chaiml-grpo-glm50-combi-44700-v1/default/tokenizer_config.json
chaiml-grpo-glm50-combi-44700-v1-uploader: cp /dev/shm/model_output/tokenizer.json s3://guanaco-vllm-models/chaiml-grpo-glm50-combi-44700-v1/default/tokenizer.json
chaiml-grpo-glm50-combi-44700-v1-uploader: cp /dev/shm/model_output/model.safetensors.index.json s3://guanaco-vllm-models/chaiml-grpo-glm50-combi-44700-v1/default/model.safetensors.index.json
2026-04-03T21:06:17.706662+00:00 monitor updated for chaiml-grpo-glm50-combi_44700_v1
chaiml-grpo-glm50-combi-44700-v1-uploader: cp /dev/shm/model_output/model-00075-of-00075.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm50-combi-44700-v1/default/model-00075-of-00075.safetensors
chaiml-grpo-glm50-combi-44700-v1-uploader: cp /dev/shm/model_output/model-00025-of-00075.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm50-combi-44700-v1/default/model-00025-of-00075.safetensors
chaiml-grpo-glm50-combi-44700-v1-uploader: cp /dev/shm/model_output/model-00012-of-00075.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm50-combi-44700-v1/default/model-00012-of-00075.safetensors
chaiml-grpo-glm50-combi-44700-v1-uploader: cp /dev/shm/model_output/model-00046-of-00075.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm50-combi-44700-v1/default/model-00046-of-00075.safetensors
chaiml-grpo-glm50-combi-44700-v1-uploader: cp /dev/shm/model_output/model-00043-of-00075.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm50-combi-44700-v1/default/model-00043-of-00075.safetensors
chaiml-grpo-glm50-combi-44700-v1-uploader: cp /dev/shm/model_output/model-00027-of-00075.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm50-combi-44700-v1/default/model-00027-of-00075.safetensors
chaiml-grpo-glm50-combi-44700-v1-uploader: cp /dev/shm/model_output/model-00004-of-00075.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm50-combi-44700-v1/default/model-00004-of-00075.safetensors
chaiml-grpo-glm50-combi-44700-v1-uploader: cp /dev/shm/model_output/model-00047-of-00075.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm50-combi-44700-v1/default/model-00047-of-00075.safetensors
chaiml-grpo-glm50-combi-44700-v1-uploader: cp /dev/shm/model_output/model-00058-of-00075.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm50-combi-44700-v1/default/model-00058-of-00075.safetensors
chaiml-grpo-glm50-combi-44700-v1-uploader: cp /dev/shm/model_output/model-00023-of-00075.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm50-combi-44700-v1/default/model-00023-of-00075.safetensors
chaiml-grpo-glm50-combi-44700-v1-uploader: cp /dev/shm/model_output/model-00018-of-00075.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm50-combi-44700-v1/default/model-00018-of-00075.safetensors
chaiml-grpo-glm50-combi-44700-v1-uploader: cp /dev/shm/model_output/model-00016-of-00075.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm50-combi-44700-v1/default/model-00016-of-00075.safetensors
chaiml-grpo-glm50-combi-44700-v1-uploader: cp /dev/shm/model_output/model-00038-of-00075.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm50-combi-44700-v1/default/model-00038-of-00075.safetensors
chaiml-grpo-glm50-combi-44700-v1-uploader: cp /dev/shm/model_output/model-00053-of-00075.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm50-combi-44700-v1/default/model-00053-of-00075.safetensors
chaiml-grpo-glm50-combi-44700-v1-uploader: cp /dev/shm/model_output/model-00002-of-00075.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm50-combi-44700-v1/default/model-00002-of-00075.safetensors
chaiml-grpo-glm50-combi-44700-v1-uploader: cp /dev/shm/model_output/model-00062-of-00075.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm50-combi-44700-v1/default/model-00062-of-00075.safetensors
chaiml-grpo-glm50-combi-44700-v1-uploader: cp /dev/shm/model_output/model-00029-of-00075.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm50-combi-44700-v1/default/model-00029-of-00075.safetensors
chaiml-grpo-glm50-combi-44700-v1-uploader: cp /dev/shm/model_output/model-00040-of-00075.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm50-combi-44700-v1/default/model-00040-of-00075.safetensors
chaiml-grpo-glm50-combi-44700-v1-uploader: cp /dev/shm/model_output/model-00009-of-00075.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm50-combi-44700-v1/default/model-00009-of-00075.safetensors
chaiml-grpo-glm50-combi-44700-v1-uploader: cp /dev/shm/model_output/model-00051-of-00075.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm50-combi-44700-v1/default/model-00051-of-00075.safetensors
chaiml-grpo-glm50-combi-44700-v1-uploader: cp /dev/shm/model_output/model-00020-of-00075.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm50-combi-44700-v1/default/model-00020-of-00075.safetensors
chaiml-grpo-glm50-combi-44700-v1-uploader: cp /dev/shm/model_output/model-00064-of-00075.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm50-combi-44700-v1/default/model-00064-of-00075.safetensors
chaiml-grpo-glm50-combi-44700-v1-uploader: cp /dev/shm/model_output/model-00054-of-00075.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm50-combi-44700-v1/default/model-00054-of-00075.safetensors
chaiml-grpo-glm50-combi-44700-v1-uploader: cp /dev/shm/model_output/model-00013-of-00075.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm50-combi-44700-v1/default/model-00013-of-00075.safetensors
chaiml-grpo-glm50-combi-44700-v1-uploader: cp /dev/shm/model_output/model-00011-of-00075.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm50-combi-44700-v1/default/model-00011-of-00075.safetensors
chaiml-grpo-glm50-combi-44700-v1-uploader: cp /dev/shm/model_output/model-00030-of-00075.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm50-combi-44700-v1/default/model-00030-of-00075.safetensors
chaiml-grpo-glm50-combi-44700-v1-uploader: cp /dev/shm/model_output/model-00050-of-00075.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm50-combi-44700-v1/default/model-00050-of-00075.safetensors
chaiml-grpo-glm50-combi-44700-v1-uploader: cp /dev/shm/model_output/model-00034-of-00075.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm50-combi-44700-v1/default/model-00034-of-00075.safetensors
chaiml-grpo-glm50-combi-44700-v1-uploader: cp /dev/shm/model_output/model-00069-of-00075.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm50-combi-44700-v1/default/model-00069-of-00075.safetensors
chaiml-grpo-glm50-combi-44700-v1-uploader: cp /dev/shm/model_output/model-00061-of-00075.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm50-combi-44700-v1/default/model-00061-of-00075.safetensors
chaiml-grpo-glm50-combi-44700-v1-uploader: cp /dev/shm/model_output/model-00055-of-00075.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm50-combi-44700-v1/default/model-00055-of-00075.safetensors
2026-04-03T21:07:17.797608+00:00 monitor updated for chaiml-grpo-glm50-combi_44700_v1
chaiml-grpo-glm50-combi-44700-v1-uploader: cp /dev/shm/model_output/model-00017-of-00075.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm50-combi-44700-v1/default/model-00017-of-00075.safetensors
chaiml-grpo-glm50-combi-44700-v1-uploader: cp /dev/shm/model_output/model-00045-of-00075.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm50-combi-44700-v1/default/model-00045-of-00075.safetensors
chaiml-grpo-glm50-combi-44700-v1-uploader: cp /dev/shm/model_output/model-00042-of-00075.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm50-combi-44700-v1/default/model-00042-of-00075.safetensors
chaiml-grpo-glm50-combi-44700-v1-uploader: cp /dev/shm/model_output/model-00035-of-00075.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm50-combi-44700-v1/default/model-00035-of-00075.safetensors
chaiml-grpo-glm50-combi-44700-v1-uploader: cp /dev/shm/model_output/model-00028-of-00075.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm50-combi-44700-v1/default/model-00028-of-00075.safetensors
chaiml-grpo-glm50-combi-44700-v1-uploader: cp /dev/shm/model_output/model-00024-of-00075.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm50-combi-44700-v1/default/model-00024-of-00075.safetensors
chaiml-grpo-glm50-combi-44700-v1-uploader: cp /dev/shm/model_output/model-00049-of-00075.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm50-combi-44700-v1/default/model-00049-of-00075.safetensors
chaiml-grpo-glm50-combi-44700-v1-uploader: cp /dev/shm/model_output/model-00041-of-00075.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm50-combi-44700-v1/default/model-00041-of-00075.safetensors
chaiml-grpo-glm50-combi-44700-v1-uploader: cp /dev/shm/model_output/model-00065-of-00075.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm50-combi-44700-v1/default/model-00065-of-00075.safetensors
chaiml-grpo-glm50-combi-44700-v1-uploader: cp /dev/shm/model_output/model-00006-of-00075.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm50-combi-44700-v1/default/model-00006-of-00075.safetensors
chaiml-grpo-glm50-combi-44700-v1-uploader: cp /dev/shm/model_output/model-00019-of-00075.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm50-combi-44700-v1/default/model-00019-of-00075.safetensors
chaiml-grpo-glm50-combi-44700-v1-uploader: cp /dev/shm/model_output/model-00056-of-00075.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm50-combi-44700-v1/default/model-00056-of-00075.safetensors
chaiml-grpo-glm50-combi-44700-v1-uploader: cp /dev/shm/model_output/model-00052-of-00075.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm50-combi-44700-v1/default/model-00052-of-00075.safetensors
chaiml-grpo-glm50-combi-44700-v1-uploader: cp /dev/shm/model_output/model-00071-of-00075.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm50-combi-44700-v1/default/model-00071-of-00075.safetensors
chaiml-grpo-glm50-combi-44700-v1-uploader: cp /dev/shm/model_output/model-00022-of-00075.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm50-combi-44700-v1/default/model-00022-of-00075.safetensors
chaiml-grpo-glm50-combi-44700-v1-uploader: cp /dev/shm/model_output/model-00031-of-00075.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm50-combi-44700-v1/default/model-00031-of-00075.safetensors
chaiml-grpo-glm50-combi-44700-v1-uploader: cp /dev/shm/model_output/model-00068-of-00075.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm50-combi-44700-v1/default/model-00068-of-00075.safetensors
chaiml-grpo-glm50-combi-44700-v1-uploader: cp /dev/shm/model_output/model-00007-of-00075.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm50-combi-44700-v1/default/model-00007-of-00075.safetensors
chaiml-grpo-glm50-combi-44700-v1-uploader: cp /dev/shm/model_output/model-00066-of-00075.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm50-combi-44700-v1/default/model-00066-of-00075.safetensors
chaiml-grpo-glm50-combi-44700-v1-uploader: cp /dev/shm/model_output/model-00073-of-00075.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm50-combi-44700-v1/default/model-00073-of-00075.safetensors
chaiml-grpo-glm50-combi-44700-v1-uploader: cp /dev/shm/model_output/model-00021-of-00075.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm50-combi-44700-v1/default/model-00021-of-00075.safetensors
chaiml-grpo-glm50-combi-44700-v1-uploader: cp /dev/shm/model_output/model-00072-of-00075.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm50-combi-44700-v1/default/model-00072-of-00075.safetensors
chaiml-grpo-glm50-combi-44700-v1-uploader: cp /dev/shm/model_output/model-00057-of-00075.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm50-combi-44700-v1/default/model-00057-of-00075.safetensors
chaiml-grpo-glm50-combi-44700-v1-uploader: cp /dev/shm/model_output/model-00014-of-00075.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm50-combi-44700-v1/default/model-00014-of-00075.safetensors
chaiml-grpo-glm50-combi-44700-v1-uploader: cp /dev/shm/model_output/model-00005-of-00075.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm50-combi-44700-v1/default/model-00005-of-00075.safetensors
chaiml-grpo-glm50-combi-44700-v1-uploader: cp /dev/shm/model_output/model-00032-of-00075.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm50-combi-44700-v1/default/model-00032-of-00075.safetensors
chaiml-grpo-glm50-combi-44700-v1-uploader: cp /dev/shm/model_output/model-00074-of-00075.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm50-combi-44700-v1/default/model-00074-of-00075.safetensors
chaiml-grpo-glm50-combi-44700-v1-uploader: cp /dev/shm/model_output/model-00070-of-00075.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm50-combi-44700-v1/default/model-00070-of-00075.safetensors
chaiml-grpo-glm50-combi-44700-v1-uploader: cp /dev/shm/model_output/model-00059-of-00075.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm50-combi-44700-v1/default/model-00059-of-00075.safetensors
chaiml-grpo-glm50-combi-44700-v1-uploader: cp /dev/shm/model_output/model-00044-of-00075.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm50-combi-44700-v1/default/model-00044-of-00075.safetensors
chaiml-grpo-glm50-combi-44700-v1-uploader: cp /dev/shm/model_output/model-00037-of-00075.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm50-combi-44700-v1/default/model-00037-of-00075.safetensors
chaiml-grpo-glm50-combi-44700-v1-uploader: cp /dev/shm/model_output/model-00033-of-00075.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm50-combi-44700-v1/default/model-00033-of-00075.safetensors
chaiml-grpo-glm50-combi-44700-v1-uploader: cp /dev/shm/model_output/model-00036-of-00075.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm50-combi-44700-v1/default/model-00036-of-00075.safetensors
chaiml-grpo-glm50-combi-44700-v1-uploader: cp /dev/shm/model_output/model-00039-of-00075.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm50-combi-44700-v1/default/model-00039-of-00075.safetensors
chaiml-grpo-glm50-combi-44700-v1-uploader: cp /dev/shm/model_output/model-00026-of-00075.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm50-combi-44700-v1/default/model-00026-of-00075.safetensors
chaiml-grpo-glm50-combi-44700-v1-uploader: cp /dev/shm/model_output/model-00063-of-00075.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm50-combi-44700-v1/default/model-00063-of-00075.safetensors
Job chaiml-grpo-glm50-combi-44700-v1-uploader completed after 453.15s with status: succeeded
Stopping job with name chaiml-grpo-glm50-combi-44700-v1-uploader
Pipeline stage VLLMUploader completed in 453.70s
run pipeline stage %s
Running pipeline stage VLLMUploaderAMD
Pipeline stage vllm_upload_amd skipped, reason=not amd cluster
Pipeline stage VLLMUploaderAMD completed in 0.13s
run pipeline stage %s
Running pipeline stage VLLMTemplater
Pipeline stage VLLMTemplater completed in 2.18s
run pipeline stage %s
Running pipeline stage VLLMDeployer
Creating inference service chaiml-grpo-glm50-combi-44700-v1
Waiting for inference service chaiml-grpo-glm50-combi-44700-v1 to be ready
2026-04-03T21:08:17.893860+00:00 monitor updated for chaiml-grpo-glm50-combi_44700_v1
2026-04-03T21:09:18.037195+00:00 monitor updated for chaiml-grpo-glm50-combi_44700_v1
2026-04-03T21:10:18.145748+00:00 monitor updated for chaiml-grpo-glm50-combi_44700_v1
2026-04-03T21:11:18.252884+00:00 monitor updated for chaiml-grpo-glm50-combi_44700_v1
2026-04-03T21:12:18.358203+00:00 monitor updated for chaiml-grpo-glm50-combi_44700_v1
Failed to get request counts for guanaco-submitter. Falling back to default
2026-04-03T21:13:18.461440+00:00 monitor updated for chaiml-grpo-glm50-combi_44700_v1
Inference service chaiml-grpo-glm50-combi-44700-v1 ready after 383.4645781517029s
Pipeline stage VLLMDeployer completed in 383.98s
run pipeline stage %s
Running pipeline stage StressChecker
2026-04-03T21:14:18.564174+00:00 monitor updated for chaiml-grpo-glm50-combi_44700_v1
Received healthy response to inference request in 11.278200626373291s
Received healthy response to inference request in 2.6550943851470947s
Received healthy response to inference request in 2.6962151527404785s
Received healthy response to inference request in 10.867646932601929s
Received healthy response to inference request in 2.746352195739746s
Received healthy response to inference request in 11.130393266677856s
Received healthy response to inference request in 2.658332586288452s
Received healthy response to inference request in 2.641101121902466s
Received healthy response to inference request in 3.043327808380127s
Received healthy response to inference request in 2.6729602813720703s
Received healthy response to inference request in 2.6348609924316406s
Received healthy response to inference request in 2.826648712158203s
2026-04-03T21:15:18.668289+00:00 monitor updated for chaiml-grpo-glm50-combi_44700_v1
Received healthy response to inference request in 2.984304428100586s
Received healthy response to inference request in 2.7463908195495605s
Received healthy response to inference request in 2.675795078277588s
Received healthy response to inference request in 2.8292319774627686s
Received healthy response to inference request in 2.952016830444336s
Received healthy response to inference request in 2.764601230621338s
Received healthy response to inference request in 2.589308023452759s
Received healthy response to inference request in 2.897188425064087s
Received healthy response to inference request in 2.6783175468444824s
Received healthy response to inference request in 2.841158151626587s
Received healthy response to inference request in 3.0376052856445312s
Received healthy response to inference request in 2.9090771675109863s
Received healthy response to inference request in 3.375385284423828s
Received healthy response to inference request in 2.7890841960906982s
Received healthy response to inference request in 2.6719369888305664s
Received healthy response to inference request in 2.915074110031128s
Received healthy response to inference request in 2.9050445556640625s
Received healthy response to inference request in 2.8658199310302734s
30 requests
0 failed requests
5th percentile: 2.637669050693512
10th percentile: 2.653695058822632
20th percentile: 2.6727556228637694
30th percentile: 2.6908458709716796
40th percentile: 2.757317066192627
50th percentile: 2.827940344810486
60th percentile: 2.8783673286437987
70th percentile: 2.910876250267029
80th percentile: 2.994964599609375
90th percentile: 4.124611449241649
95th percentile: 11.012157416343689
99th percentile: 11.235336492061615
mean time: 3.642615803082784
Pipeline stage StressChecker completed in 112.37s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyTriggerPipeline
run_pipeline:run_in_cloud %s
starting trigger_guanaco_pipeline args=%s
triggered trigger_guanaco_pipeline args=%s
Pipeline stage OfflineFamilyFriendlyTriggerPipeline completed in 0.76s
Shutdown handler de-registered
chaiml-grpo-glm50-combi_44700_v1 status is now deployed due to DeploymentManager action