developer_uid: chai_backend_admin
submission_id: chaiml-gspo-glm46-combin_2308_v2
model_name: chaiml-gspo-glm46-combin_2308_v2
model_group: ChaiML/gspo-glm46-combin
status: inactive
timestamp: 2026-03-12T19:57:54+00:00
num_battles: 11020
num_wins: 5778
celo_rating: 1311.5
family_friendly_score: 0.0
family_friendly_standard_error: 0.0
submission_type: basic
model_repo: ChaiML/gspo-glm46-combine-rm55-step100
model_architecture: Glm4MoeForCausalLM
model_num_parameters: 24110003200.0
best_of: 8
max_input_tokens: 2048
max_output_tokens: 80
reward_model: default
display_name: chaiml-gspo-glm46-combin_2308_v2
ineligible_reason: max_output_tokens!=64
is_internal_developer: True
language_model: ChaiML/gspo-glm46-combine-rm55-step100
model_size: 24B
ranking_group: single
us_pacific_date: 2026-03-12
win_ratio: 0.5243194192377495
generation_params: {'temperature': 1.0, 'top_p': 0.95, 'min_p': 0.0, 'top_k': 60, 'presence_penalty': 0.0, 'frequency_penalty': 0.0, 'stopping_words': ['<|user|>', '<|im_end|>', '</s>', '<|assistant|>', '####'], 'max_input_tokens': 2048, 'best_of': 8, 'max_output_tokens': 80}
formatter: {'memory_template': "[gMASK]<sop><|system|>\n{bot_name}'s persona: {memory}.<|user|>\n", 'prompt_template': '', 'bot_template': '{bot_name}: {message}\n', 'user_template': '{user_name}: {message}\n', 'response_template': '<|assistant|>\n<think></think>\n{bot_name}:', 'truncate_by_message': True}
Resubmit model
Shutdown handler not registered because Python interpreter is not running in the main thread
run pipeline %s
run pipeline stage %s
Running pipeline stage VLLMUploader
Starting job with name chaiml-gspo-glm46-combin-2308-v2-uploader
Waiting for job on chaiml-gspo-glm46-combin-2308-v2-uploader to finish
2026-03-12T16:22:02.447956+00:00 monitor updated for chaiml-grpo-glm46-combi_19477_v2
chaiml-gspo-glm46-combin-2308-v2-uploader: Checking if ChaiML/gspo-glm46-combine-rm55-step100-FP8 already exists in ChaiML
chaiml-gspo-glm46-combin-2308-v2-uploader: Model already exists. Downloading to /dev/shm/model_output...
chaiml-gspo-glm46-combin-2308-v2-uploader: Downloading snapshot of ChaiML/gspo-glm46-combine-rm55-step100-FP8...
2026-03-12T16:22:37.647262+00:00 monitor updated for chaiml-gspo-glm46-combin_2308_v2
2026-03-12T16:23:02.589516+00:00 monitor updated for chaiml-grpo-glm46-combi_19477_v2
chaiml-grpo-glm46-combi-19477-v2-uploader: Downloaded in 160.474s
chaiml-grpo-glm46-combi-19477-v2-uploader: Processed model ChaiML/grpo-glm46-combine-rm82-lr5e6-step500 in 163.958s
chaiml-grpo-glm46-combi-19477-v2-uploader: creating bucket guanaco-vllm-models
chaiml-grpo-glm46-combi-19477-v2-uploader: /usr/lib/python3/dist-packages/S3/BaseUtils.py:56: SyntaxWarning: invalid escape sequence '\.'
chaiml-grpo-glm46-combi-19477-v2-uploader:   RE_S3_DATESTRING = re.compile('\.[0-9]*(?:[Z\\-\\+]*?)')
chaiml-grpo-glm46-combi-19477-v2-uploader: /usr/lib/python3/dist-packages/S3/BaseUtils.py:57: SyntaxWarning: invalid escape sequence '\s'
chaiml-grpo-glm46-combi-19477-v2-uploader:   RE_XML_NAMESPACE = re.compile(b'^(<?[^>]+?>\s*|\s*)(<\w+) xmlns=[\'"](https?://[^\'"]+)[\'"]', re.MULTILINE)
chaiml-grpo-glm46-combi-19477-v2-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:240: SyntaxWarning: invalid escape sequence '\.'
chaiml-grpo-glm46-combi-19477-v2-uploader:   invalid = re.search("([^a-z0-9\.-])", bucket, re.UNICODE)
chaiml-grpo-glm46-combi-19477-v2-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:244: SyntaxWarning: invalid escape sequence '\.'
chaiml-grpo-glm46-combi-19477-v2-uploader:   invalid = re.search("([^A-Za-z0-9\._-])", bucket, re.UNICODE)
chaiml-grpo-glm46-combi-19477-v2-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:255: SyntaxWarning: invalid escape sequence '\.'
2026-03-12T16:23:37.788745+00:00 monitor updated for chaiml-gspo-glm46-combin_2308_v2
chaiml-grpo-glm46-combi-19477-v2-uploader:   if re.search("-\.", bucket, re.UNICODE):
chaiml-grpo-glm46-combi-19477-v2-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:257: SyntaxWarning: invalid escape sequence '\.'
chaiml-grpo-glm46-combi-19477-v2-uploader:   if re.search("\.\.", bucket, re.UNICODE):
chaiml-grpo-glm46-combi-19477-v2-uploader: /usr/lib/python3/dist-packages/S3/S3Uri.py:155: SyntaxWarning: invalid escape sequence '\w'
chaiml-grpo-glm46-combi-19477-v2-uploader:   _re = re.compile("^(\w+://)?(.*)", re.UNICODE)
chaiml-grpo-glm46-combi-19477-v2-uploader: /usr/lib/python3/dist-packages/S3/FileLists.py:480: SyntaxWarning: invalid escape sequence '\*'
chaiml-grpo-glm46-combi-19477-v2-uploader:   wildcard_split_result = re.split("\*|\?", uri_str, maxsplit=1)
chaiml-grpo-glm46-combi-19477-v2-uploader: Bucket 's3://guanaco-vllm-models/' created
chaiml-grpo-glm46-combi-19477-v2-uploader: uploading /dev/shm/model_output to s3://guanaco-vllm-models/chaiml-grpo-glm46-combi-19477-v2/default
chaiml-grpo-glm46-combi-19477-v2-uploader: cp /dev/shm/model_output/.gitattributes s3://guanaco-vllm-models/chaiml-grpo-glm46-combi-19477-v2/default/.gitattributes
chaiml-grpo-glm46-combi-19477-v2-uploader: cp /dev/shm/model_output/generation_config.json s3://guanaco-vllm-models/chaiml-grpo-glm46-combi-19477-v2/default/generation_config.json
chaiml-grpo-glm46-combi-19477-v2-uploader: cp /dev/shm/model_output/chat_template.jinja s3://guanaco-vllm-models/chaiml-grpo-glm46-combi-19477-v2/default/chat_template.jinja
chaiml-grpo-glm46-combi-19477-v2-uploader: cp /dev/shm/model_output/config.json s3://guanaco-vllm-models/chaiml-grpo-glm46-combi-19477-v2/default/config.json
chaiml-grpo-glm46-combi-19477-v2-uploader: cp /dev/shm/model_output/special_tokens_map.json s3://guanaco-vllm-models/chaiml-grpo-glm46-combi-19477-v2/default/special_tokens_map.json
chaiml-grpo-glm46-combi-19477-v2-uploader: cp /dev/shm/model_output/recipe.yaml s3://guanaco-vllm-models/chaiml-grpo-glm46-combi-19477-v2/default/recipe.yaml
chaiml-grpo-glm46-combi-19477-v2-uploader: cp /dev/shm/model_output/tokenizer_config.json s3://guanaco-vllm-models/chaiml-grpo-glm46-combi-19477-v2/default/tokenizer_config.json
chaiml-grpo-glm46-combi-19477-v2-uploader: cp /dev/shm/model_output/model.safetensors.index.json s3://guanaco-vllm-models/chaiml-grpo-glm46-combi-19477-v2/default/model.safetensors.index.json
chaiml-grpo-glm46-combi-19477-v2-uploader: cp /dev/shm/model_output/tokenizer.json s3://guanaco-vllm-models/chaiml-grpo-glm46-combi-19477-v2/default/tokenizer.json
chaiml-grpo-glm46-combi-19477-v2-uploader: cp /dev/shm/model_output/model-00072-of-00072.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm46-combi-19477-v2/default/model-00072-of-00072.safetensors
2026-03-12T16:24:02.739295+00:00 monitor updated for chaiml-grpo-glm46-combi_19477_v2
chaiml-grpo-glm46-combi-19477-v2-uploader: cp /dev/shm/model_output/model-00071-of-00072.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm46-combi-19477-v2/default/model-00071-of-00072.safetensors
2026-03-12T16:24:38.012464+00:00 monitor updated for chaiml-gspo-glm46-combin_2308_v2
chaiml-grpo-glm46-combi-19477-v2-uploader: cp /dev/shm/model_output/model-00033-of-00072.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm46-combi-19477-v2/default/model-00033-of-00072.safetensors
chaiml-grpo-glm46-combi-19477-v2-uploader: cp /dev/shm/model_output/model-00027-of-00072.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm46-combi-19477-v2/default/model-00027-of-00072.safetensors
chaiml-grpo-glm46-combi-19477-v2-uploader: cp /dev/shm/model_output/model-00010-of-00072.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm46-combi-19477-v2/default/model-00010-of-00072.safetensors
chaiml-grpo-glm46-combi-19477-v2-uploader: cp /dev/shm/model_output/model-00048-of-00072.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm46-combi-19477-v2/default/model-00048-of-00072.safetensors
chaiml-grpo-glm46-combi-19477-v2-uploader: cp /dev/shm/model_output/model-00026-of-00072.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm46-combi-19477-v2/default/model-00026-of-00072.safetensors
chaiml-grpo-glm46-combi-19477-v2-uploader: cp /dev/shm/model_output/model-00020-of-00072.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm46-combi-19477-v2/default/model-00020-of-00072.safetensors
chaiml-grpo-glm46-combi-19477-v2-uploader: cp /dev/shm/model_output/model-00068-of-00072.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm46-combi-19477-v2/default/model-00068-of-00072.safetensors
chaiml-grpo-glm46-combi-19477-v2-uploader: cp /dev/shm/model_output/model-00046-of-00072.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm46-combi-19477-v2/default/model-00046-of-00072.safetensors
chaiml-grpo-glm46-combi-19477-v2-uploader: cp /dev/shm/model_output/model-00028-of-00072.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm46-combi-19477-v2/default/model-00028-of-00072.safetensors
chaiml-grpo-glm46-combi-19477-v2-uploader: cp /dev/shm/model_output/model-00008-of-00072.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm46-combi-19477-v2/default/model-00008-of-00072.safetensors
chaiml-grpo-glm46-combi-19477-v2-uploader: cp /dev/shm/model_output/model-00007-of-00072.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm46-combi-19477-v2/default/model-00007-of-00072.safetensors
chaiml-grpo-glm46-combi-19477-v2-uploader: cp /dev/shm/model_output/model-00034-of-00072.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm46-combi-19477-v2/default/model-00034-of-00072.safetensors
chaiml-grpo-glm46-combi-19477-v2-uploader: cp /dev/shm/model_output/model-00024-of-00072.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm46-combi-19477-v2/default/model-00024-of-00072.safetensors
chaiml-grpo-glm46-combi-19477-v2-uploader: cp /dev/shm/model_output/model-00029-of-00072.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm46-combi-19477-v2/default/model-00029-of-00072.safetensors
chaiml-grpo-glm46-combi-19477-v2-uploader: cp /dev/shm/model_output/model-00013-of-00072.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm46-combi-19477-v2/default/model-00013-of-00072.safetensors
chaiml-grpo-glm46-combi-19477-v2-uploader: cp /dev/shm/model_output/model-00011-of-00072.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm46-combi-19477-v2/default/model-00011-of-00072.safetensors
chaiml-grpo-glm46-combi-19477-v2-uploader: cp /dev/shm/model_output/model-00037-of-00072.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm46-combi-19477-v2/default/model-00037-of-00072.safetensors
chaiml-grpo-glm46-combi-19477-v2-uploader: cp /dev/shm/model_output/model-00031-of-00072.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm46-combi-19477-v2/default/model-00031-of-00072.safetensors
chaiml-grpo-glm46-combi-19477-v2-uploader: cp /dev/shm/model_output/model-00012-of-00072.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm46-combi-19477-v2/default/model-00012-of-00072.safetensors
chaiml-grpo-glm46-combi-19477-v2-uploader: cp /dev/shm/model_output/model-00058-of-00072.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm46-combi-19477-v2/default/model-00058-of-00072.safetensors
chaiml-grpo-glm46-combi-19477-v2-uploader: cp /dev/shm/model_output/model-00002-of-00072.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm46-combi-19477-v2/default/model-00002-of-00072.safetensors
chaiml-grpo-glm46-combi-19477-v2-uploader: cp /dev/shm/model_output/model-00045-of-00072.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm46-combi-19477-v2/default/model-00045-of-00072.safetensors
chaiml-grpo-glm46-combi-19477-v2-uploader: cp /dev/shm/model_output/model-00065-of-00072.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm46-combi-19477-v2/default/model-00065-of-00072.safetensors
chaiml-grpo-glm46-combi-19477-v2-uploader: cp /dev/shm/model_output/model-00044-of-00072.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm46-combi-19477-v2/default/model-00044-of-00072.safetensors
chaiml-grpo-glm46-combi-19477-v2-uploader: cp /dev/shm/model_output/model-00001-of-00072.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm46-combi-19477-v2/default/model-00001-of-00072.safetensors
chaiml-grpo-glm46-combi-19477-v2-uploader: cp /dev/shm/model_output/model-00035-of-00072.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm46-combi-19477-v2/default/model-00035-of-00072.safetensors
chaiml-grpo-glm46-combi-19477-v2-uploader: cp /dev/shm/model_output/model-00043-of-00072.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm46-combi-19477-v2/default/model-00043-of-00072.safetensors
chaiml-grpo-glm46-combi-19477-v2-uploader: cp /dev/shm/model_output/model-00069-of-00072.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm46-combi-19477-v2/default/model-00069-of-00072.safetensors
chaiml-grpo-glm46-combi-19477-v2-uploader: cp /dev/shm/model_output/model-00032-of-00072.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm46-combi-19477-v2/default/model-00032-of-00072.safetensors
chaiml-grpo-glm46-combi-19477-v2-uploader: cp /dev/shm/model_output/model-00055-of-00072.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm46-combi-19477-v2/default/model-00055-of-00072.safetensors
chaiml-grpo-glm46-combi-19477-v2-uploader: cp /dev/shm/model_output/model-00005-of-00072.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm46-combi-19477-v2/default/model-00005-of-00072.safetensors
chaiml-grpo-glm46-combi-19477-v2-uploader: cp /dev/shm/model_output/model-00030-of-00072.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm46-combi-19477-v2/default/model-00030-of-00072.safetensors
chaiml-grpo-glm46-combi-19477-v2-uploader: cp /dev/shm/model_output/model-00059-of-00072.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm46-combi-19477-v2/default/model-00059-of-00072.safetensors
chaiml-grpo-glm46-combi-19477-v2-uploader: cp /dev/shm/model_output/model-00019-of-00072.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm46-combi-19477-v2/default/model-00019-of-00072.safetensors
chaiml-grpo-glm46-combi-19477-v2-uploader: cp /dev/shm/model_output/model-00023-of-00072.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm46-combi-19477-v2/default/model-00023-of-00072.safetensors
chaiml-grpo-glm46-combi-19477-v2-uploader: cp /dev/shm/model_output/model-00004-of-00072.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm46-combi-19477-v2/default/model-00004-of-00072.safetensors
chaiml-grpo-glm46-combi-19477-v2-uploader: cp /dev/shm/model_output/model-00040-of-00072.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm46-combi-19477-v2/default/model-00040-of-00072.safetensors
chaiml-grpo-glm46-combi-19477-v2-uploader: cp /dev/shm/model_output/model-00063-of-00072.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm46-combi-19477-v2/default/model-00063-of-00072.safetensors
chaiml-grpo-glm46-combi-19477-v2-uploader: cp /dev/shm/model_output/model-00016-of-00072.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm46-combi-19477-v2/default/model-00016-of-00072.safetensors
chaiml-grpo-glm46-combi-19477-v2-uploader: cp /dev/shm/model_output/model-00053-of-00072.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm46-combi-19477-v2/default/model-00053-of-00072.safetensors
chaiml-grpo-glm46-combi-19477-v2-uploader: cp /dev/shm/model_output/model-00025-of-00072.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm46-combi-19477-v2/default/model-00025-of-00072.safetensors
chaiml-grpo-glm46-combi-19477-v2-uploader: cp /dev/shm/model_output/model-00049-of-00072.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm46-combi-19477-v2/default/model-00049-of-00072.safetensors
chaiml-grpo-glm46-combi-19477-v2-uploader: cp /dev/shm/model_output/model-00006-of-00072.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm46-combi-19477-v2/default/model-00006-of-00072.safetensors
chaiml-grpo-glm46-combi-19477-v2-uploader: cp /dev/shm/model_output/model-00054-of-00072.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm46-combi-19477-v2/default/model-00054-of-00072.safetensors
chaiml-grpo-glm46-combi-19477-v2-uploader: cp /dev/shm/model_output/model-00038-of-00072.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm46-combi-19477-v2/default/model-00038-of-00072.safetensors
chaiml-grpo-glm46-combi-19477-v2-uploader: cp /dev/shm/model_output/model-00060-of-00072.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm46-combi-19477-v2/default/model-00060-of-00072.safetensors
chaiml-grpo-glm46-combi-19477-v2-uploader: cp /dev/shm/model_output/model-00018-of-00072.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm46-combi-19477-v2/default/model-00018-of-00072.safetensors
chaiml-grpo-glm46-combi-19477-v2-uploader: cp /dev/shm/model_output/model-00064-of-00072.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm46-combi-19477-v2/default/model-00064-of-00072.safetensors
chaiml-grpo-glm46-combi-19477-v2-uploader: cp /dev/shm/model_output/model-00047-of-00072.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm46-combi-19477-v2/default/model-00047-of-00072.safetensors
chaiml-grpo-glm46-combi-19477-v2-uploader: cp /dev/shm/model_output/model-00051-of-00072.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm46-combi-19477-v2/default/model-00051-of-00072.safetensors
chaiml-grpo-glm46-combi-19477-v2-uploader: cp /dev/shm/model_output/model-00057-of-00072.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm46-combi-19477-v2/default/model-00057-of-00072.safetensors
chaiml-grpo-glm46-combi-19477-v2-uploader: cp /dev/shm/model_output/model-00021-of-00072.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm46-combi-19477-v2/default/model-00021-of-00072.safetensors
chaiml-grpo-glm46-combi-19477-v2-uploader: cp /dev/shm/model_output/model-00003-of-00072.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm46-combi-19477-v2/default/model-00003-of-00072.safetensors
chaiml-grpo-glm46-combi-19477-v2-uploader: cp /dev/shm/model_output/model-00009-of-00072.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm46-combi-19477-v2/default/model-00009-of-00072.safetensors
chaiml-grpo-glm46-combi-19477-v2-uploader: cp /dev/shm/model_output/model-00070-of-00072.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm46-combi-19477-v2/default/model-00070-of-00072.safetensors
chaiml-grpo-glm46-combi-19477-v2-uploader: cp /dev/shm/model_output/model-00067-of-00072.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm46-combi-19477-v2/default/model-00067-of-00072.safetensors
chaiml-grpo-glm46-combi-19477-v2-uploader: cp /dev/shm/model_output/model-00061-of-00072.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm46-combi-19477-v2/default/model-00061-of-00072.safetensors
chaiml-grpo-glm46-combi-19477-v2-uploader: cp /dev/shm/model_output/model-00015-of-00072.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm46-combi-19477-v2/default/model-00015-of-00072.safetensors
chaiml-grpo-glm46-combi-19477-v2-uploader: cp /dev/shm/model_output/model-00041-of-00072.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm46-combi-19477-v2/default/model-00041-of-00072.safetensors
chaiml-grpo-glm46-combi-19477-v2-uploader: cp /dev/shm/model_output/model-00036-of-00072.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm46-combi-19477-v2/default/model-00036-of-00072.safetensors
chaiml-grpo-glm46-combi-19477-v2-uploader: cp /dev/shm/model_output/model-00022-of-00072.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm46-combi-19477-v2/default/model-00022-of-00072.safetensors
chaiml-grpo-glm46-combi-19477-v2-uploader: cp /dev/shm/model_output/model-00066-of-00072.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm46-combi-19477-v2/default/model-00066-of-00072.safetensors
chaiml-grpo-glm46-combi-19477-v2-uploader: cp /dev/shm/model_output/model-00050-of-00072.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm46-combi-19477-v2/default/model-00050-of-00072.safetensors
chaiml-grpo-glm46-combi-19477-v2-uploader: cp /dev/shm/model_output/model-00039-of-00072.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm46-combi-19477-v2/default/model-00039-of-00072.safetensors
chaiml-grpo-glm46-combi-19477-v2-uploader: cp /dev/shm/model_output/model-00042-of-00072.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm46-combi-19477-v2/default/model-00042-of-00072.safetensors
chaiml-grpo-glm46-combi-19477-v2-uploader: cp /dev/shm/model_output/model-00056-of-00072.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm46-combi-19477-v2/default/model-00056-of-00072.safetensors
chaiml-grpo-glm46-combi-19477-v2-uploader: cp /dev/shm/model_output/model-00017-of-00072.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm46-combi-19477-v2/default/model-00017-of-00072.safetensors
chaiml-grpo-glm46-combi-19477-v2-uploader: cp /dev/shm/model_output/model-00052-of-00072.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm46-combi-19477-v2/default/model-00052-of-00072.safetensors
chaiml-grpo-glm46-combi-19477-v2-uploader: cp /dev/shm/model_output/model-00014-of-00072.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm46-combi-19477-v2/default/model-00014-of-00072.safetensors
chaiml-grpo-glm46-combi-19477-v2-uploader: cp /dev/shm/model_output/model-00062-of-00072.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm46-combi-19477-v2/default/model-00062-of-00072.safetensors
chaiml-gspo-glm46-combin-2308-v2-uploader: Downloaded in 158.020s
chaiml-gspo-glm46-combin-2308-v2-uploader: Processed model ChaiML/gspo-glm46-combine-rm55-step100 in 161.607s
2026-03-12T16:25:02.991926+00:00 monitor updated for chaiml-grpo-glm46-combi_19477_v2
Job chaiml-grpo-glm46-combi-19477-v2-uploader completed after 304.77s with status: succeeded
Stopping job with name chaiml-grpo-glm46-combi-19477-v2-uploader
Pipeline stage VLLMUploader completed in 306.07s
run pipeline stage %s
Running pipeline stage VLLMTemplater
Pipeline stage VLLMTemplater completed in 1.18s
run pipeline stage %s
Running pipeline stage VLLMDeployer
Creating inference service chaiml-grpo-glm46-combi-19477-v2
chaiml-gspo-glm46-combin-2308-v2-uploader: creating bucket guanaco-vllm-models
Waiting for inference service chaiml-grpo-glm46-combi-19477-v2 to be ready
chaiml-gspo-glm46-combin-2308-v2-uploader: /usr/lib/python3/dist-packages/S3/BaseUtils.py:56: SyntaxWarning: invalid escape sequence '\.'
chaiml-gspo-glm46-combin-2308-v2-uploader:   RE_S3_DATESTRING = re.compile('\.[0-9]*(?:[Z\\-\\+]*?)')
chaiml-gspo-glm46-combin-2308-v2-uploader: /usr/lib/python3/dist-packages/S3/BaseUtils.py:57: SyntaxWarning: invalid escape sequence '\s'
chaiml-gspo-glm46-combin-2308-v2-uploader:   RE_XML_NAMESPACE = re.compile(b'^(<?[^>]+?>\s*|\s*)(<\w+) xmlns=[\'"](https?://[^\'"]+)[\'"]', re.MULTILINE)
chaiml-gspo-glm46-combin-2308-v2-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:240: SyntaxWarning: invalid escape sequence '\.'
chaiml-gspo-glm46-combin-2308-v2-uploader:   invalid = re.search("([^a-z0-9\.-])", bucket, re.UNICODE)
chaiml-gspo-glm46-combin-2308-v2-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:244: SyntaxWarning: invalid escape sequence '\.'
chaiml-gspo-glm46-combin-2308-v2-uploader:   invalid = re.search("([^A-Za-z0-9\._-])", bucket, re.UNICODE)
chaiml-gspo-glm46-combin-2308-v2-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:255: SyntaxWarning: invalid escape sequence '\.'
chaiml-gspo-glm46-combin-2308-v2-uploader:   if re.search("-\.", bucket, re.UNICODE):
chaiml-gspo-glm46-combin-2308-v2-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:257: SyntaxWarning: invalid escape sequence '\.'
chaiml-gspo-glm46-combin-2308-v2-uploader:   if re.search("\.\.", bucket, re.UNICODE):
chaiml-gspo-glm46-combin-2308-v2-uploader: /usr/lib/python3/dist-packages/S3/S3Uri.py:155: SyntaxWarning: invalid escape sequence '\w'
chaiml-gspo-glm46-combin-2308-v2-uploader:   _re = re.compile("^(\w+://)?(.*)", re.UNICODE)
chaiml-gspo-glm46-combin-2308-v2-uploader: /usr/lib/python3/dist-packages/S3/FileLists.py:480: SyntaxWarning: invalid escape sequence '\*'
chaiml-gspo-glm46-combin-2308-v2-uploader:   wildcard_split_result = re.split("\*|\?", uri_str, maxsplit=1)
chaiml-gspo-glm46-combin-2308-v2-uploader: Bucket 's3://guanaco-vllm-models/' created
chaiml-gspo-glm46-combin-2308-v2-uploader: uploading /dev/shm/model_output to s3://guanaco-vllm-models/chaiml-gspo-glm46-combin-2308-v2/default
chaiml-gspo-glm46-combin-2308-v2-uploader: cp /dev/shm/model_output/.gitattributes s3://guanaco-vllm-models/chaiml-gspo-glm46-combin-2308-v2/default/.gitattributes
chaiml-gspo-glm46-combin-2308-v2-uploader: cp /dev/shm/model_output/generation_config.json s3://guanaco-vllm-models/chaiml-gspo-glm46-combin-2308-v2/default/generation_config.json
chaiml-gspo-glm46-combin-2308-v2-uploader: cp /dev/shm/model_output/tokenizer_config.json s3://guanaco-vllm-models/chaiml-gspo-glm46-combin-2308-v2/default/tokenizer_config.json
chaiml-gspo-glm46-combin-2308-v2-uploader: cp /dev/shm/model_output/model.safetensors.index.json s3://guanaco-vllm-models/chaiml-gspo-glm46-combin-2308-v2/default/model.safetensors.index.json
chaiml-gspo-glm46-combin-2308-v2-uploader: cp /dev/shm/model_output/chat_template.jinja s3://guanaco-vllm-models/chaiml-gspo-glm46-combin-2308-v2/default/chat_template.jinja
chaiml-gspo-glm46-combin-2308-v2-uploader: cp /dev/shm/model_output/config.json s3://guanaco-vllm-models/chaiml-gspo-glm46-combin-2308-v2/default/config.json
chaiml-gspo-glm46-combin-2308-v2-uploader: cp /dev/shm/model_output/tokenizer.json s3://guanaco-vllm-models/chaiml-gspo-glm46-combin-2308-v2/default/tokenizer.json
chaiml-gspo-glm46-combin-2308-v2-uploader: cp /dev/shm/model_output/recipe.yaml s3://guanaco-vllm-models/chaiml-gspo-glm46-combin-2308-v2/default/recipe.yaml
chaiml-gspo-glm46-combin-2308-v2-uploader: cp /dev/shm/model_output/special_tokens_map.json s3://guanaco-vllm-models/chaiml-gspo-glm46-combin-2308-v2/default/special_tokens_map.json
chaiml-gspo-glm46-combin-2308-v2-uploader: cp /dev/shm/model_output/model-00072-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm46-combin-2308-v2/default/model-00072-of-00072.safetensors
2026-03-12T16:25:38.504782+00:00 monitor updated for chaiml-gspo-glm46-combin_2308_v2
chaiml-gspo-glm46-combin-2308-v2-uploader: cp /dev/shm/model_output/model-00071-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm46-combin-2308-v2/default/model-00071-of-00072.safetensors
2026-03-12T16:26:03.466708+00:00 monitor updated for chaiml-grpo-glm46-combi_19477_v2
chaiml-gspo-glm46-combin-2308-v2-uploader: cp /dev/shm/model_output/model-00002-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm46-combin-2308-v2/default/model-00002-of-00072.safetensors
chaiml-gspo-glm46-combin-2308-v2-uploader: cp /dev/shm/model_output/model-00070-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm46-combin-2308-v2/default/model-00070-of-00072.safetensors
chaiml-gspo-glm46-combin-2308-v2-uploader: cp /dev/shm/model_output/model-00056-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm46-combin-2308-v2/default/model-00056-of-00072.safetensors
chaiml-gspo-glm46-combin-2308-v2-uploader: cp /dev/shm/model_output/model-00058-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm46-combin-2308-v2/default/model-00058-of-00072.safetensors
chaiml-gspo-glm46-combin-2308-v2-uploader: cp /dev/shm/model_output/model-00005-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm46-combin-2308-v2/default/model-00005-of-00072.safetensors
chaiml-gspo-glm46-combin-2308-v2-uploader: cp /dev/shm/model_output/model-00064-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm46-combin-2308-v2/default/model-00064-of-00072.safetensors
chaiml-gspo-glm46-combin-2308-v2-uploader: cp /dev/shm/model_output/model-00047-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm46-combin-2308-v2/default/model-00047-of-00072.safetensors
chaiml-gspo-glm46-combin-2308-v2-uploader: cp /dev/shm/model_output/model-00020-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm46-combin-2308-v2/default/model-00020-of-00072.safetensors
chaiml-gspo-glm46-combin-2308-v2-uploader: cp /dev/shm/model_output/model-00029-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm46-combin-2308-v2/default/model-00029-of-00072.safetensors
chaiml-gspo-glm46-combin-2308-v2-uploader: cp /dev/shm/model_output/model-00026-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm46-combin-2308-v2/default/model-00026-of-00072.safetensors
chaiml-gspo-glm46-combin-2308-v2-uploader: cp /dev/shm/model_output/model-00052-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm46-combin-2308-v2/default/model-00052-of-00072.safetensors
chaiml-gspo-glm46-combin-2308-v2-uploader: cp /dev/shm/model_output/model-00001-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm46-combin-2308-v2/default/model-00001-of-00072.safetensors
chaiml-gspo-glm46-combin-2308-v2-uploader: cp /dev/shm/model_output/model-00006-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm46-combin-2308-v2/default/model-00006-of-00072.safetensors
chaiml-gspo-glm46-combin-2308-v2-uploader: cp /dev/shm/model_output/model-00025-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm46-combin-2308-v2/default/model-00025-of-00072.safetensors
chaiml-gspo-glm46-combin-2308-v2-uploader: cp /dev/shm/model_output/model-00031-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm46-combin-2308-v2/default/model-00031-of-00072.safetensors
chaiml-gspo-glm46-combin-2308-v2-uploader: cp /dev/shm/model_output/model-00028-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm46-combin-2308-v2/default/model-00028-of-00072.safetensors
chaiml-gspo-glm46-combin-2308-v2-uploader: cp /dev/shm/model_output/model-00019-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm46-combin-2308-v2/default/model-00019-of-00072.safetensors
chaiml-gspo-glm46-combin-2308-v2-uploader: cp /dev/shm/model_output/model-00055-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm46-combin-2308-v2/default/model-00055-of-00072.safetensors
chaiml-gspo-glm46-combin-2308-v2-uploader: cp /dev/shm/model_output/model-00016-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm46-combin-2308-v2/default/model-00016-of-00072.safetensors
chaiml-gspo-glm46-combin-2308-v2-uploader: cp /dev/shm/model_output/model-00063-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm46-combin-2308-v2/default/model-00063-of-00072.safetensors
chaiml-gspo-glm46-combin-2308-v2-uploader: cp /dev/shm/model_output/model-00037-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm46-combin-2308-v2/default/model-00037-of-00072.safetensors
chaiml-gspo-glm46-combin-2308-v2-uploader: cp /dev/shm/model_output/model-00046-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm46-combin-2308-v2/default/model-00046-of-00072.safetensors
chaiml-gspo-glm46-combin-2308-v2-uploader: cp /dev/shm/model_output/model-00039-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm46-combin-2308-v2/default/model-00039-of-00072.safetensors
chaiml-gspo-glm46-combin-2308-v2-uploader: cp /dev/shm/model_output/model-00012-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm46-combin-2308-v2/default/model-00012-of-00072.safetensors
chaiml-gspo-glm46-combin-2308-v2-uploader: cp /dev/shm/model_output/model-00034-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm46-combin-2308-v2/default/model-00034-of-00072.safetensors
chaiml-gspo-glm46-combin-2308-v2-uploader: cp /dev/shm/model_output/model-00060-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm46-combin-2308-v2/default/model-00060-of-00072.safetensors
chaiml-gspo-glm46-combin-2308-v2-uploader: cp /dev/shm/model_output/model-00035-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm46-combin-2308-v2/default/model-00035-of-00072.safetensors
chaiml-gspo-glm46-combin-2308-v2-uploader: cp /dev/shm/model_output/model-00040-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm46-combin-2308-v2/default/model-00040-of-00072.safetensors
chaiml-gspo-glm46-combin-2308-v2-uploader: cp /dev/shm/model_output/model-00041-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm46-combin-2308-v2/default/model-00041-of-00072.safetensors
chaiml-gspo-glm46-combin-2308-v2-uploader: cp /dev/shm/model_output/model-00048-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm46-combin-2308-v2/default/model-00048-of-00072.safetensors
chaiml-gspo-glm46-combin-2308-v2-uploader: cp /dev/shm/model_output/model-00066-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm46-combin-2308-v2/default/model-00066-of-00072.safetensors
chaiml-gspo-glm46-combin-2308-v2-uploader: cp /dev/shm/model_output/model-00067-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm46-combin-2308-v2/default/model-00067-of-00072.safetensors
chaiml-gspo-glm46-combin-2308-v2-uploader: cp /dev/shm/model_output/model-00011-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm46-combin-2308-v2/default/model-00011-of-00072.safetensors
chaiml-gspo-glm46-combin-2308-v2-uploader: cp /dev/shm/model_output/model-00013-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm46-combin-2308-v2/default/model-00013-of-00072.safetensors
chaiml-gspo-glm46-combin-2308-v2-uploader: cp /dev/shm/model_output/model-00008-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm46-combin-2308-v2/default/model-00008-of-00072.safetensors
chaiml-gspo-glm46-combin-2308-v2-uploader: cp /dev/shm/model_output/model-00018-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm46-combin-2308-v2/default/model-00018-of-00072.safetensors
chaiml-gspo-glm46-combin-2308-v2-uploader: cp /dev/shm/model_output/model-00003-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm46-combin-2308-v2/default/model-00003-of-00072.safetensors
Job chaiml-gspo-glm46-combin-2308-v2-uploader completed after 298.13s with status: succeeded
Stopping job with name chaiml-gspo-glm46-combin-2308-v2-uploader
Pipeline stage VLLMUploader completed in 298.93s
run pipeline stage %s
Running pipeline stage VLLMTemplater
Pipeline stage VLLMTemplater completed in 1.09s
run pipeline stage %s
Running pipeline stage VLLMDeployer
2026-03-12T16:26:38.666250+00:00 monitor updated for chaiml-gspo-glm46-combin_2308_v2
Creating inference service chaiml-gspo-glm46-combin-2308-v2
Waiting for inference service chaiml-gspo-glm46-combin-2308-v2 to be ready
2026-03-12T16:27:03.628702+00:00 monitor updated for chaiml-grpo-glm46-combi_19477_v2
2026-03-12T16:27:38.855540+00:00 monitor updated for chaiml-gspo-glm46-combin_2308_v2
2026-03-12T16:28:03.819418+00:00 monitor updated for chaiml-grpo-glm46-combi_19477_v2
2026-03-12T16:28:39.027165+00:00 monitor updated for chaiml-gspo-glm46-combin_2308_v2
2026-03-12T16:29:03.988096+00:00 monitor updated for chaiml-grpo-glm46-combi_19477_v2
2026-03-12T16:29:39.236231+00:00 monitor updated for chaiml-gspo-glm46-combin_2308_v2
2026-03-12T16:30:04.180894+00:00 monitor updated for chaiml-grpo-glm46-combi_19477_v2
2026-03-12T16:30:39.438864+00:00 monitor updated for chaiml-gspo-glm46-combin_2308_v2
Inference service chaiml-grpo-glm46-combi-19477-v2 ready after 351.84844040870667s
Pipeline stage VLLMDeployer completed in 352.84s
run pipeline stage %s
Running pipeline stage StressChecker
2026-03-12T16:31:04.371652+00:00 monitor updated for chaiml-grpo-glm46-combi_19477_v2
Received healthy response to inference request in 9.258665084838867s
Received healthy response to inference request in 9.332569599151611s
Received healthy response to inference request in 1.9796473979949951s
Received healthy response to inference request in 8.913277626037598s
Received healthy response to inference request in 2.1534109115600586s
Received healthy response to inference request in 1.9950776100158691s
2026-03-12T16:31:39.607609+00:00 monitor updated for chaiml-gspo-glm46-combin_2308_v2
Received healthy response to inference request in 2.1589479446411133s
Received healthy response to inference request in 8.949825286865234s
Received healthy response to inference request in 9.175184488296509s
Received healthy response to inference request in 1.9742887020111084s
2026-03-12T16:32:04.554184+00:00 monitor updated for chaiml-grpo-glm46-combi_19477_v2
Received healthy response to inference request in 2.127040147781372s
Received healthy response to inference request in 2.013502836227417s
Received healthy response to inference request in 2.090557813644409s
Received healthy response to inference request in 2.1972997188568115s
Received healthy response to inference request in 2.190857172012329s
Received healthy response to inference request in 2.0132908821105957s
Received healthy response to inference request in 2.1784110069274902s
Received healthy response to inference request in 2.246751308441162s
Received healthy response to inference request in 2.1196861267089844s
Received healthy response to inference request in 2.004427909851074s
Received healthy response to inference request in 2.028691530227661s
Received healthy response to inference request in 2.1131627559661865s
Received healthy response to inference request in 2.029627799987793s
Received healthy response to inference request in 2.282078266143799s
2026-03-12T16:32:39.919376+00:00 monitor updated for chaiml-gspo-glm46-combin_2308_v2
Inference service chaiml-gspo-glm46-combin-2308-v2 ready after 361.4945635795593s
Pipeline stage VLLMDeployer completed in 362.42s
Received healthy response to inference request in 2.2171075344085693s
run pipeline stage %s
Running pipeline stage StressChecker
Received healthy response to inference request in 2.3072662353515625s
Received healthy response to inference request in 2.0240046977996826s
Received healthy response to inference request in 2.0804340839385986s
Received healthy response to inference request in 2.0486464500427246s
Received healthy response to inference request in 9.054048299789429s
Received healthy response to inference request in 2.2596614360809326s
30 requests
0 failed requests
5th percentile: 1.9865909934043884
10th percentile: 2.0034928798675535
20th percentile: 2.0219043254852296
30th percentile: 2.042940855026245
40th percentile: 2.1041207790374754
50th percentile: 2.1402255296707153
60th percentile: 2.1833894729614256
70th percentile: 2.226000666618347
80th percentile: 2.287115859985352
90th percentile: 8.972361207008362
95th percentile: 9.221098816394806
99th percentile: 9.311137290000916
mean time: 3.2821133454640705
Pipeline stage StressChecker completed in 113.05s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyTriggerPipeline
run_pipeline:run_in_cloud %s
starting trigger_guanaco_pipeline args=%s
triggered trigger_guanaco_pipeline args=%s
Pipeline stage OfflineFamilyFriendlyTriggerPipeline completed in 2.10s
Shutdown handler de-registered
chaiml-grpo-glm46-combi_19477_v2 status is now deployed due to DeploymentManager action
Received healthy response to inference request in 8.96865439414978s
Received healthy response to inference request in 2.2395331859588623s
Received healthy response to inference request in 9.289259672164917s
Received healthy response to inference request in 2.0295939445495605s
Received healthy response to inference request in 2.0384514331817627s
Received healthy response to inference request in 2.1354689598083496s
Received healthy response to inference request in 2.263150453567505s
Received healthy response to inference request in 1.997016191482544s
Received healthy response to inference request in 2.0096237659454346s
Received healthy response to inference request in 2.080315113067627s
Received healthy response to inference request in 2.1210572719573975s
Received healthy response to inference request in 2.144566059112549s
Received healthy response to inference request in 2.154137372970581s
Received healthy response to inference request in 2.2409744262695312s
Received healthy response to inference request in 2.378169536590576s
2026-03-12T16:33:40.136650+00:00 monitor updated for chaiml-gspo-glm46-combin_2308_v2
Received healthy response to inference request in 2.14094877243042s
Received healthy response to inference request in 2.1107256412506104s
Received healthy response to inference request in 2.042501449584961s
Received healthy response to inference request in 2.1940927505493164s
Received healthy response to inference request in 2.163144588470459s
Received healthy response to inference request in 2.357555627822876s
Received healthy response to inference request in 2.1209702491760254s
Received healthy response to inference request in 2.063310146331787s
Received healthy response to inference request in 2.2962658405303955s
Received healthy response to inference request in 2.1298201084136963s
Received healthy response to inference request in 2.15364933013916s
Received healthy response to inference request in 2.161022663116455s
Received healthy response to inference request in 2.232785224914551s
Received healthy response to inference request in 2.1216628551483154s
30 requests
0 failed requests
5th percentile: 2.018610346317291
10th percentile: 2.0375656843185426
20th percentile: 2.076914119720459
30th percentile: 2.121031165122986
40th percentile: 2.1332094192504885
50th percentile: 2.1491076946258545
60th percentile: 2.1618714332580566
70th percentile: 2.2348096132278443
80th percentile: 2.2697735309600833
90th percentile: 3.037218022346506
95th percentile: 9.015621042251587
99th percentile: 9.221048374176025
mean time: 2.847749177614848
Pipeline stage StressChecker completed in 96.78s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyTriggerPipeline
run_pipeline:run_in_cloud %s
starting trigger_guanaco_pipeline args=%s
triggered trigger_guanaco_pipeline args=%s
Pipeline stage OfflineFamilyFriendlyTriggerPipeline completed in 1.05s
Shutdown handler de-registered
chaiml-gspo-glm46-combin_2308_v2 status is now deployed due to DeploymentManager action
chaiml-gspo-glm46-combin_2308_v2 status is now inactive due to auto deactivation removed underperforming models