Shutdown handler not registered because Python interpreter is not running in the main thread
run pipeline %s
run pipeline stage %s
Running pipeline stage VLLMUploader
Pipeline stage vllm_upload skipped, reason=amd cluster
Pipeline stage VLLMUploader completed in 0.09s
run pipeline stage %s
Running pipeline stage VLLMUploaderAMD
Starting job with name chaiml-kimid-v8b-kimidv-81429-v9-uploader-amd
Waiting for job on chaiml-kimid-v8b-kimidv-81429-v9-uploader-amd to finish
chaiml-kimid-v8b-kimidv-81429-v9-uploader-amd: {repo_id} is already quantized
chaiml-kimid-v8b-kimidv-81429-v9-uploader-amd: Using quantization_mode: none
chaiml-kimid-v8b-kimidv-81429-v9-uploader-amd: Downloading snapshot of ChaiML/kimid-v8b-kimidv5a-lr5e6ep2r64g4b01-FP8...
2026-04-02T17:46:07.273799+00:00 monitor updated for chaiml-kimid-v8b-kimidv_81429_v9
2026-04-02T17:47:07.367445+00:00 monitor updated for chaiml-kimid-v8b-kimidv_81429_v9
chaiml-kimid-v8b-kimidv-81429-v9-uploader-amd: Downloaded in 112.823s
2026-04-02T17:48:07.459505+00:00 monitor updated for chaiml-kimid-v8b-kimidv_81429_v9
2026-04-02T17:49:07.550590+00:00 monitor updated for chaiml-kimid-v8b-kimidv_81429_v9
chaiml-kimid-v8b-kimidv-81429-v9-uploader-amd: Processed model ChaiML/kimid-v8b-kimidv5a-lr5e6ep2r64g4b01-FP8 in 200.358s
chaiml-kimid-v8b-kimidv-81429-v9-uploader-amd: creating bucket guanaco-vllm-models
chaiml-kimid-v8b-kimidv-81429-v9-uploader-amd: /usr/lib/python3/dist-packages/S3/BaseUtils.py:56: SyntaxWarning: invalid escape sequence '\.'
chaiml-kimid-v8b-kimidv-81429-v9-uploader-amd: RE_S3_DATESTRING = re.compile('\.[0-9]*(?:[Z\\-\\+]*?)')
chaiml-kimid-v8b-kimidv-81429-v9-uploader-amd: /usr/lib/python3/dist-packages/S3/BaseUtils.py:57: SyntaxWarning: invalid escape sequence '\s'
chaiml-kimid-v8b-kimidv-81429-v9-uploader-amd: RE_XML_NAMESPACE = re.compile(b'^(<?[^>]+?>\s*|\s*)(<\w+) xmlns=[\'"](https?://[^\'"]+)[\'"]', re.MULTILINE)
chaiml-kimid-v8b-kimidv-81429-v9-uploader-amd: /usr/lib/python3/dist-packages/S3/Utils.py:240: SyntaxWarning: invalid escape sequence '\.'
chaiml-kimid-v8b-kimidv-81429-v9-uploader-amd: invalid = re.search("([^a-z0-9\.-])", bucket, re.UNICODE)
chaiml-kimid-v8b-kimidv-81429-v9-uploader-amd: /usr/lib/python3/dist-packages/S3/Utils.py:244: SyntaxWarning: invalid escape sequence '\.'
chaiml-kimid-v8b-kimidv-81429-v9-uploader-amd: invalid = re.search("([^A-Za-z0-9\._-])", bucket, re.UNICODE)
chaiml-kimid-v8b-kimidv-81429-v9-uploader-amd: /usr/lib/python3/dist-packages/S3/Utils.py:255: SyntaxWarning: invalid escape sequence '\.'
chaiml-kimid-v8b-kimidv-81429-v9-uploader-amd: if re.search("-\.", bucket, re.UNICODE):
chaiml-kimid-v8b-kimidv-81429-v9-uploader-amd: /usr/lib/python3/dist-packages/S3/Utils.py:257: SyntaxWarning: invalid escape sequence '\.'
chaiml-kimid-v8b-kimidv-81429-v9-uploader-amd: if re.search("\.\.", bucket, re.UNICODE):
chaiml-kimid-v8b-kimidv-81429-v9-uploader-amd: /usr/lib/python3/dist-packages/S3/S3Uri.py:155: SyntaxWarning: invalid escape sequence '\w'
chaiml-kimid-v8b-kimidv-81429-v9-uploader-amd: _re = re.compile("^(\w+://)?(.*)", re.UNICODE)
chaiml-kimid-v8b-kimidv-81429-v9-uploader-amd: /usr/lib/python3/dist-packages/S3/FileLists.py:480: SyntaxWarning: invalid escape sequence '\*'
chaiml-kimid-v8b-kimidv-81429-v9-uploader-amd: wildcard_split_result = re.split("\*|\?", uri_str, maxsplit=1)
chaiml-kimid-v8b-kimidv-81429-v9-uploader-amd: Bucket 's3://guanaco-vllm-models/' created
chaiml-kimid-v8b-kimidv-81429-v9-uploader-amd: uploading /dev/shm/model_output to s3://guanaco-vllm-models/chaiml-kimid-v8b-kimidv-81429-v9/amd
chaiml-kimid-v8b-kimidv-81429-v9-uploader-amd: cp /dev/shm/model_output/.gitattributes s3://guanaco-vllm-models/chaiml-kimid-v8b-kimidv-81429-v9/amd/.gitattributes
chaiml-kimid-v8b-kimidv-81429-v9-uploader-amd: cp /dev/shm/model_output/added_tokens.json s3://guanaco-vllm-models/chaiml-kimid-v8b-kimidv-81429-v9/amd/added_tokens.json
chaiml-kimid-v8b-kimidv-81429-v9-uploader-amd: cp /dev/shm/model_output/chat_template.jinja s3://guanaco-vllm-models/chaiml-kimid-v8b-kimidv-81429-v9/amd/chat_template.jinja
chaiml-kimid-v8b-kimidv-81429-v9-uploader-amd: cp /dev/shm/model_output/generation_config.json s3://guanaco-vllm-models/chaiml-kimid-v8b-kimidv-81429-v9/amd/generation_config.json
chaiml-kimid-v8b-kimidv-81429-v9-uploader-amd: cp /dev/shm/model_output/special_tokens_map.json s3://guanaco-vllm-models/chaiml-kimid-v8b-kimidv-81429-v9/amd/special_tokens_map.json
chaiml-kimid-v8b-kimidv-81429-v9-uploader-amd: cp /dev/shm/model_output/recipe.yaml s3://guanaco-vllm-models/chaiml-kimid-v8b-kimidv-81429-v9/amd/recipe.yaml
chaiml-kimid-v8b-kimidv-81429-v9-uploader-amd: cp /dev/shm/model_output/tokenizer_config.json s3://guanaco-vllm-models/chaiml-kimid-v8b-kimidv-81429-v9/amd/tokenizer_config.json
chaiml-kimid-v8b-kimidv-81429-v9-uploader-amd: cp /dev/shm/model_output/config.json s3://guanaco-vllm-models/chaiml-kimid-v8b-kimidv-81429-v9/amd/config.json
chaiml-kimid-v8b-kimidv-81429-v9-uploader-amd: cp /dev/shm/model_output/merges.txt s3://guanaco-vllm-models/chaiml-kimid-v8b-kimidv-81429-v9/amd/merges.txt
chaiml-kimid-v8b-kimidv-81429-v9-uploader-amd: cp /dev/shm/model_output/vocab.json s3://guanaco-vllm-models/chaiml-kimid-v8b-kimidv-81429-v9/amd/vocab.json
chaiml-kimid-v8b-kimidv-81429-v9-uploader-amd: cp /dev/shm/model_output/model.safetensors.index.json s3://guanaco-vllm-models/chaiml-kimid-v8b-kimidv-81429-v9/amd/model.safetensors.index.json
chaiml-kimid-v8b-kimidv-81429-v9-uploader-amd: cp /dev/shm/model_output/tokenizer.json s3://guanaco-vllm-models/chaiml-kimid-v8b-kimidv-81429-v9/amd/tokenizer.json
chaiml-kimid-v8b-kimidv-81429-v9-uploader-amd: cp /dev/shm/model_output/model-00048-of-00048.safetensors s3://guanaco-vllm-models/chaiml-kimid-v8b-kimidv-81429-v9/amd/model-00048-of-00048.safetensors
chaiml-kimid-v8b-kimidv-81429-v9-uploader-amd: cp /dev/shm/model_output/model-00044-of-00048.safetensors s3://guanaco-vllm-models/chaiml-kimid-v8b-kimidv-81429-v9/amd/model-00044-of-00048.safetensors
chaiml-kimid-v8b-kimidv-81429-v9-uploader-amd: cp /dev/shm/model_output/model-00012-of-00048.safetensors s3://guanaco-vllm-models/chaiml-kimid-v8b-kimidv-81429-v9/amd/model-00012-of-00048.safetensors
chaiml-kimid-v8b-kimidv-81429-v9-uploader-amd: cp /dev/shm/model_output/model-00013-of-00048.safetensors s3://guanaco-vllm-models/chaiml-kimid-v8b-kimidv-81429-v9/amd/model-00013-of-00048.safetensors
chaiml-kimid-v8b-kimidv-81429-v9-uploader-amd: cp /dev/shm/model_output/model-00005-of-00048.safetensors s3://guanaco-vllm-models/chaiml-kimid-v8b-kimidv-81429-v9/amd/model-00005-of-00048.safetensors
chaiml-kimid-v8b-kimidv-81429-v9-uploader-amd: cp /dev/shm/model_output/model-00024-of-00048.safetensors s3://guanaco-vllm-models/chaiml-kimid-v8b-kimidv-81429-v9/amd/model-00024-of-00048.safetensors
chaiml-kimid-v8b-kimidv-81429-v9-uploader-amd: cp /dev/shm/model_output/model-00030-of-00048.safetensors s3://guanaco-vllm-models/chaiml-kimid-v8b-kimidv-81429-v9/amd/model-00030-of-00048.safetensors
chaiml-kimid-v8b-kimidv-81429-v9-uploader-amd: cp /dev/shm/model_output/model-00004-of-00048.safetensors s3://guanaco-vllm-models/chaiml-kimid-v8b-kimidv-81429-v9/amd/model-00004-of-00048.safetensors
chaiml-kimid-v8b-kimidv-81429-v9-uploader-amd: cp /dev/shm/model_output/model-00033-of-00048.safetensors s3://guanaco-vllm-models/chaiml-kimid-v8b-kimidv-81429-v9/amd/model-00033-of-00048.safetensors
chaiml-kimid-v8b-kimidv-81429-v9-uploader-amd: cp /dev/shm/model_output/model-00011-of-00048.safetensors s3://guanaco-vllm-models/chaiml-kimid-v8b-kimidv-81429-v9/amd/model-00011-of-00048.safetensors
chaiml-kimid-v8b-kimidv-81429-v9-uploader-amd: cp /dev/shm/model_output/model-00023-of-00048.safetensors s3://guanaco-vllm-models/chaiml-kimid-v8b-kimidv-81429-v9/amd/model-00023-of-00048.safetensors
chaiml-kimid-v8b-kimidv-81429-v9-uploader-amd: cp /dev/shm/model_output/model-00022-of-00048.safetensors s3://guanaco-vllm-models/chaiml-kimid-v8b-kimidv-81429-v9/amd/model-00022-of-00048.safetensors
chaiml-kimid-v8b-kimidv-81429-v9-uploader-amd: cp /dev/shm/model_output/model-00016-of-00048.safetensors s3://guanaco-vllm-models/chaiml-kimid-v8b-kimidv-81429-v9/amd/model-00016-of-00048.safetensors
chaiml-kimid-v8b-kimidv-81429-v9-uploader-amd: cp /dev/shm/model_output/model-00010-of-00048.safetensors s3://guanaco-vllm-models/chaiml-kimid-v8b-kimidv-81429-v9/amd/model-00010-of-00048.safetensors
chaiml-kimid-v8b-kimidv-81429-v9-uploader-amd: cp /dev/shm/model_output/model-00029-of-00048.safetensors s3://guanaco-vllm-models/chaiml-kimid-v8b-kimidv-81429-v9/amd/model-00029-of-00048.safetensors
chaiml-kimid-v8b-kimidv-81429-v9-uploader-amd: cp /dev/shm/model_output/model-00025-of-00048.safetensors s3://guanaco-vllm-models/chaiml-kimid-v8b-kimidv-81429-v9/amd/model-00025-of-00048.safetensors
chaiml-kimid-v8b-kimidv-81429-v9-uploader-amd: cp /dev/shm/model_output/model-00018-of-00048.safetensors s3://guanaco-vllm-models/chaiml-kimid-v8b-kimidv-81429-v9/amd/model-00018-of-00048.safetensors
chaiml-kimid-v8b-kimidv-81429-v9-uploader-amd: cp /dev/shm/model_output/model-00020-of-00048.safetensors s3://guanaco-vllm-models/chaiml-kimid-v8b-kimidv-81429-v9/amd/model-00020-of-00048.safetensors
chaiml-kimid-v8b-kimidv-81429-v9-uploader-amd: cp /dev/shm/model_output/model-00031-of-00048.safetensors s3://guanaco-vllm-models/chaiml-kimid-v8b-kimidv-81429-v9/amd/model-00031-of-00048.safetensors
chaiml-kimid-v8b-kimidv-81429-v9-uploader-amd: cp /dev/shm/model_output/model-00008-of-00048.safetensors s3://guanaco-vllm-models/chaiml-kimid-v8b-kimidv-81429-v9/amd/model-00008-of-00048.safetensors
chaiml-kimid-v8b-kimidv-81429-v9-uploader-amd: cp /dev/shm/model_output/model-00039-of-00048.safetensors s3://guanaco-vllm-models/chaiml-kimid-v8b-kimidv-81429-v9/amd/model-00039-of-00048.safetensors
chaiml-kimid-v8b-kimidv-81429-v9-uploader-amd: cp /dev/shm/model_output/model-00009-of-00048.safetensors s3://guanaco-vllm-models/chaiml-kimid-v8b-kimidv-81429-v9/amd/model-00009-of-00048.safetensors
chaiml-kimid-v8b-kimidv-81429-v9-uploader-amd: cp /dev/shm/model_output/model-00034-of-00048.safetensors s3://guanaco-vllm-models/chaiml-kimid-v8b-kimidv-81429-v9/amd/model-00034-of-00048.safetensors
chaiml-kimid-v8b-kimidv-81429-v9-uploader-amd: cp /dev/shm/model_output/model-00014-of-00048.safetensors s3://guanaco-vllm-models/chaiml-kimid-v8b-kimidv-81429-v9/amd/model-00014-of-00048.safetensors
chaiml-kimid-v8b-kimidv-81429-v9-uploader-amd: cp /dev/shm/model_output/model-00021-of-00048.safetensors s3://guanaco-vllm-models/chaiml-kimid-v8b-kimidv-81429-v9/amd/model-00021-of-00048.safetensors
chaiml-kimid-v8b-kimidv-81429-v9-uploader-amd: cp /dev/shm/model_output/model-00001-of-00048.safetensors s3://guanaco-vllm-models/chaiml-kimid-v8b-kimidv-81429-v9/amd/model-00001-of-00048.safetensors
chaiml-kimid-v8b-kimidv-81429-v9-uploader-amd: cp /dev/shm/model_output/model-00027-of-00048.safetensors s3://guanaco-vllm-models/chaiml-kimid-v8b-kimidv-81429-v9/amd/model-00027-of-00048.safetensors
chaiml-kimid-v8b-kimidv-81429-v9-uploader-amd: cp /dev/shm/model_output/model-00037-of-00048.safetensors s3://guanaco-vllm-models/chaiml-kimid-v8b-kimidv-81429-v9/amd/model-00037-of-00048.safetensors
chaiml-kimid-v8b-kimidv-81429-v9-uploader-amd: cp /dev/shm/model_output/model-00006-of-00048.safetensors s3://guanaco-vllm-models/chaiml-kimid-v8b-kimidv-81429-v9/amd/model-00006-of-00048.safetensors
chaiml-kimid-v8b-kimidv-81429-v9-uploader-amd: cp /dev/shm/model_output/model-00028-of-00048.safetensors s3://guanaco-vllm-models/chaiml-kimid-v8b-kimidv-81429-v9/amd/model-00028-of-00048.safetensors
chaiml-kimid-v8b-kimidv-81429-v9-uploader-amd: cp /dev/shm/model_output/model-00038-of-00048.safetensors s3://guanaco-vllm-models/chaiml-kimid-v8b-kimidv-81429-v9/amd/model-00038-of-00048.safetensors
chaiml-kimid-v8b-kimidv-81429-v9-uploader-amd: cp /dev/shm/model_output/model-00017-of-00048.safetensors s3://guanaco-vllm-models/chaiml-kimid-v8b-kimidv-81429-v9/amd/model-00017-of-00048.safetensors
chaiml-kimid-v8b-kimidv-81429-v9-uploader-amd: cp /dev/shm/model_output/model-00047-of-00048.safetensors s3://guanaco-vllm-models/chaiml-kimid-v8b-kimidv-81429-v9/amd/model-00047-of-00048.safetensors
chaiml-kimid-v8b-kimidv-81429-v9-uploader-amd: cp /dev/shm/model_output/model-00015-of-00048.safetensors s3://guanaco-vllm-models/chaiml-kimid-v8b-kimidv-81429-v9/amd/model-00015-of-00048.safetensors
chaiml-kimid-v8b-kimidv-81429-v9-uploader-amd: cp /dev/shm/model_output/model-00040-of-00048.safetensors s3://guanaco-vllm-models/chaiml-kimid-v8b-kimidv-81429-v9/amd/model-00040-of-00048.safetensors
chaiml-kimid-v8b-kimidv-81429-v9-uploader-amd: cp /dev/shm/model_output/model-00043-of-00048.safetensors s3://guanaco-vllm-models/chaiml-kimid-v8b-kimidv-81429-v9/amd/model-00043-of-00048.safetensors
chaiml-kimid-v8b-kimidv-81429-v9-uploader-amd: cp /dev/shm/model_output/model-00002-of-00048.safetensors s3://guanaco-vllm-models/chaiml-kimid-v8b-kimidv-81429-v9/amd/model-00002-of-00048.safetensors
chaiml-kimid-v8b-kimidv-81429-v9-uploader-amd: cp /dev/shm/model_output/model-00035-of-00048.safetensors s3://guanaco-vllm-models/chaiml-kimid-v8b-kimidv-81429-v9/amd/model-00035-of-00048.safetensors
chaiml-kimid-v8b-kimidv-81429-v9-uploader-amd: cp /dev/shm/model_output/model-00046-of-00048.safetensors s3://guanaco-vllm-models/chaiml-kimid-v8b-kimidv-81429-v9/amd/model-00046-of-00048.safetensors
chaiml-kimid-v8b-kimidv-81429-v9-uploader-amd: cp /dev/shm/model_output/model-00019-of-00048.safetensors s3://guanaco-vllm-models/chaiml-kimid-v8b-kimidv-81429-v9/amd/model-00019-of-00048.safetensors
chaiml-kimid-v8b-kimidv-81429-v9-uploader-amd: cp /dev/shm/model_output/model-00042-of-00048.safetensors s3://guanaco-vllm-models/chaiml-kimid-v8b-kimidv-81429-v9/amd/model-00042-of-00048.safetensors
chaiml-kimid-v8b-kimidv-81429-v9-uploader-amd: cp /dev/shm/model_output/model-00045-of-00048.safetensors s3://guanaco-vllm-models/chaiml-kimid-v8b-kimidv-81429-v9/amd/model-00045-of-00048.safetensors
chaiml-kimid-v8b-kimidv-81429-v9-uploader-amd: cp /dev/shm/model_output/model-00007-of-00048.safetensors s3://guanaco-vllm-models/chaiml-kimid-v8b-kimidv-81429-v9/amd/model-00007-of-00048.safetensors
chaiml-kimid-v8b-kimidv-81429-v9-uploader-amd: cp /dev/shm/model_output/model-00041-of-00048.safetensors s3://guanaco-vllm-models/chaiml-kimid-v8b-kimidv-81429-v9/amd/model-00041-of-00048.safetensors
chaiml-kimid-v8b-kimidv-81429-v9-uploader-amd: cp /dev/shm/model_output/model-00036-of-00048.safetensors s3://guanaco-vllm-models/chaiml-kimid-v8b-kimidv-81429-v9/amd/model-00036-of-00048.safetensors
chaiml-kimid-v8b-kimidv-81429-v9-uploader-amd: cp /dev/shm/model_output/model-00032-of-00048.safetensors s3://guanaco-vllm-models/chaiml-kimid-v8b-kimidv-81429-v9/amd/model-00032-of-00048.safetensors
chaiml-kimid-v8b-kimidv-81429-v9-uploader-amd: cp /dev/shm/model_output/model-00003-of-00048.safetensors s3://guanaco-vllm-models/chaiml-kimid-v8b-kimidv-81429-v9/amd/model-00003-of-00048.safetensors
chaiml-kimid-v8b-kimidv-81429-v9-uploader-amd: cp /dev/shm/model_output/model-00026-of-00048.safetensors s3://guanaco-vllm-models/chaiml-kimid-v8b-kimidv-81429-v9/amd/model-00026-of-00048.safetensors
2026-04-02T17:50:07.641046+00:00 monitor updated for chaiml-kimid-v8b-kimidv_81429_v9
Job chaiml-kimid-v8b-kimidv-81429-v9-uploader-amd completed after 317.73s with status: succeeded
Stopping job with name chaiml-kimid-v8b-kimidv-81429-v9-uploader-amd
Pipeline stage VLLMUploaderAMD completed in 318.95s
run pipeline stage %s
Running pipeline stage VLLMTemplater
Pipeline stage VLLMTemplater completed in 2.56s
run pipeline stage %s
Running pipeline stage VLLMDeployer
Creating inference service chaiml-kimid-v8b-kimidv-81429-v9
Waiting for inference service chaiml-kimid-v8b-kimidv-81429-v9 to be ready
2026-04-02T17:51:07.738040+00:00 monitor updated for chaiml-kimid-v8b-kimidv_81429_v9
2026-04-02T17:52:07.884236+00:00 monitor updated for chaiml-kimid-v8b-kimidv_81429_v9
2026-04-02T17:53:07.978333+00:00 monitor updated for chaiml-kimid-v8b-kimidv_81429_v9
2026-04-02T17:54:08.077889+00:00 monitor updated for chaiml-kimid-v8b-kimidv_81429_v9
2026-04-02T17:55:08.178136+00:00 monitor updated for chaiml-kimid-v8b-kimidv_81429_v9
Inference service chaiml-kimid-v8b-kimidv-81429-v9 ready after 282.3095338344574s
Pipeline stage VLLMDeployer completed in 283.28s
run pipeline stage %s
Running pipeline stage StressChecker
HTTPConnectionPool(host='guanaco-submitter-v2.guanaco-backend.kchai-google-us-east4.chaiverse.com', port=80): Read timed out. (read timeout=20)
Received unhealthy response to inference request!
Received healthy response to inference request in 5.0772130489349365s
Received healthy response to inference request in 2.464578151702881s
Received healthy response to inference request in 3.094125270843506s
Received healthy response to inference request in 2.345810890197754s
Received healthy response to inference request in 2.420874834060669s
Received healthy response to inference request in 2.7289702892303467s
Received healthy response to inference request in 2.753814935684204s
Received healthy response to inference request in 2.583064317703247s
Received healthy response to inference request in 2.417417526245117s
Received healthy response to inference request in 2.7595560550689697s
Received healthy response to inference request in 2.4086246490478516s
Received healthy response to inference request in 2.365963935852051s
2026-04-02T17:56:08.273641+00:00 monitor updated for chaiml-kimid-v8b-kimidv_81429_v9
Received healthy response to inference request in 2.673427104949951s
Received healthy response to inference request in 2.6052141189575195s
Received healthy response to inference request in 2.3903915882110596s
Received healthy response to inference request in 2.402050018310547s
Received healthy response to inference request in 2.3770110607147217s
Received healthy response to inference request in 3.042306423187256s
Received healthy response to inference request in 2.505725622177124s
Received healthy response to inference request in 2.3935515880584717s
Received healthy response to inference request in 2.9633688926696777s
Received healthy response to inference request in 2.600370407104492s
Received healthy response to inference request in 2.4104292392730713s
Received healthy response to inference request in 2.4222917556762695s
Received healthy response to inference request in 2.501960515975952s
Received healthy response to inference request in 2.399334669113159s
Received healthy response to inference request in 2.385100841522217s
Received healthy response to inference request in 2.555868148803711s
Received healthy response to inference request in 2.541872501373291s
30 requests
1 failed requests
5th percentile: 2.3709351420402527
10th percentile: 2.3842918634414674
20th percentile: 2.398178052902222
30th percentile: 2.4098878622055055
40th percentile: 2.4217249870300295
50th percentile: 2.503843069076538
60th percentile: 2.5667466163635253
70th percentile: 2.6256780147552488
80th percentile: 2.7549631595611572
90th percentile: 3.047488307952881
95th percentile: 4.184823548793787
99th percentile: 15.786074452400221
mean time: 3.225013494491577
%s, retrying in %s seconds...
Received healthy response to inference request in 2.338043212890625s
Received healthy response to inference request in 2.823977470397949s
Received healthy response to inference request in 3.288203239440918s
Received healthy response to inference request in 2.274500608444214s
Received healthy response to inference request in 2.36212158203125s
2026-04-02T17:57:08.375623+00:00 monitor updated for chaiml-kimid-v8b-kimidv_81429_v9
Received healthy response to inference request in 2.3909618854522705s
Received healthy response to inference request in 2.2902257442474365s
Received healthy response to inference request in 2.3423616886138916s
Received healthy response to inference request in 2.3740875720977783s
Received healthy response to inference request in 2.3453667163848877s
Received healthy response to inference request in 2.7461893558502197s
Received healthy response to inference request in 2.294750690460205s
Received healthy response to inference request in 2.5343704223632812s
Received healthy response to inference request in 2.3908023834228516s
Received healthy response to inference request in 2.416804790496826s
Received healthy response to inference request in 2.6863629817962646s
Received healthy response to inference request in 2.4244399070739746s
Received healthy response to inference request in 2.4024970531463623s
Received healthy response to inference request in 2.410331964492798s
Received healthy response to inference request in 2.507631540298462s
Received healthy response to inference request in 2.4246673583984375s
Received healthy response to inference request in 2.4434916973114014s
Received healthy response to inference request in 2.702289342880249s
Received healthy response to inference request in 2.3377420902252197s
Received healthy response to inference request in 2.998591661453247s
Received healthy response to inference request in 3.0538389682769775s
Received healthy response to inference request in 2.303972005844116s
Received healthy response to inference request in 2.7849924564361572s
Received healthy response to inference request in 2.4237771034240723s
2026-04-02T17:58:08.474452+00:00 monitor updated for chaiml-kimid-v8b-kimidv_81429_v9
Received healthy response to inference request in 2.5405972003936768s
30 requests
0 failed requests
5th percentile: 2.2922619700431826
10th percentile: 2.3030498743057253
20th percentile: 2.3414979934692384
30th percentile: 2.37049777507782
40th percentile: 2.3978829860687254
50th percentile: 2.420290946960449
60th percentile: 2.432197093963623
70th percentile: 2.5362384557724
80th percentile: 2.711069345474243
90th percentile: 2.8414388895034794
95th percentile: 3.0289776802062987
99th percentile: 3.2202376008033755
mean time: 2.5219330231348676
Pipeline stage StressChecker completed in 178.98s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyTriggerPipeline
run_pipeline:run_in_cloud %s
starting trigger_guanaco_pipeline args=%s
triggered trigger_guanaco_pipeline args=%s
Pipeline stage OfflineFamilyFriendlyTriggerPipeline completed in 0.75s
Shutdown handler de-registered
chaiml-kimid-v8b-kimidv_81429_v9 status is now deployed due to DeploymentManager action
chaiml-kimid-v8b-kimidv_81429_v9 status is now inactive due to auto deactivation removed underperforming models
chaiml-kimid-v8b-kimidv_81429_v9 status is now inactive due to Froze recruitment for AB test 0403_baltimore