developer_uid: richhx
submission_id: chaiml-grpo-glm46-combi_79432_v6
model_name: chaiml-grpo-glm46-combi_79432_v6
model_group: ChaiML/grpo-glm46-combin
status: inactive
timestamp: 2026-03-19T16:36:47+00:00
num_battles: 12246
num_wins: 6745
celo_rating: 1331.86
family_friendly_score: 0.0
family_friendly_standard_error: 0.0
submission_type: basic
model_repo: ChaiML/grpo-glm46-combine-rm82-lr5e6-step150
model_architecture: Glm4MoeForCausalLM
model_num_parameters: 24110003200.0
best_of: 8
max_input_tokens: 1500
max_output_tokens: 80
reward_model: default
display_name: chaiml-grpo-glm46-combi_79432_v6
ineligible_reason: max_output_tokens!=64
is_internal_developer: True
language_model: ChaiML/grpo-glm46-combine-rm82-lr5e6-step150
model_size: 24B
ranking_group: single
us_pacific_date: 2026-03-19
win_ratio: 0.5507920953780826
generation_params: {'temperature': 1.0, 'top_p': 0.95, 'min_p': 0.0, 'top_k': 60, 'presence_penalty': 0.0, 'frequency_penalty': 0.0, 'stopping_words': ['####', '<|user|>', '<|assistant|>', '</s>', '<|im_end|>'], 'max_input_tokens': 1500, 'best_of': 8, 'max_output_tokens': 80}
formatter: {'memory_template': "[gMASK]<sop><|system|>\n{bot_name}'s persona: {memory}", 'prompt_template': '', 'bot_template': '<|assistant|>\n{bot_name}: {message}', 'user_template': '<|user|>\n{message}', 'response_template': '<|assistant|>\n<think></think>\n{bot_name}:', 'truncate_by_message': True}
Resubmit model
Shutdown handler not registered because Python interpreter is not running in the main thread
run pipeline %s
run pipeline stage %s
Running pipeline stage VLLMUploader
Starting job with name chaiml-grpo-glm46-combi-79432-v6-uploader
Waiting for job on chaiml-grpo-glm46-combi-79432-v6-uploader to finish
chaiml-grpo-glm46-combi-79432-v6-uploader: Using quantization_mode: fp8
chaiml-grpo-glm46-combi-79432-v6-uploader: Checking if ChaiML/grpo-glm46-combine-rm82-lr5e6-step150-FP8 already exists in ChaiML
chaiml-grpo-glm46-combi-79432-v6-uploader: Model already exists. Downloading to /dev/shm/model_output...
chaiml-grpo-glm46-combi-79432-v6-uploader: Downloading snapshot of ChaiML/grpo-glm46-combine-rm82-lr5e6-step150-FP8...
2026-03-19T15:10:48.757813+00:00 monitor updated for chaiml-grpo-glm46-combi_79432_v6
2026-03-19T15:11:48.851708+00:00 monitor updated for chaiml-grpo-glm46-combi_79432_v6
2026-03-19T15:12:48.945772+00:00 monitor updated for chaiml-grpo-glm46-combi_79432_v6
chaiml-grpo-glm46-combi-79432-v6-uploader: Downloaded in 162.392s
chaiml-grpo-glm46-combi-79432-v6-uploader: Processed model ChaiML/grpo-glm46-combine-rm82-lr5e6-step150 in 166.102s
chaiml-grpo-glm46-combi-79432-v6-uploader: creating bucket guanaco-vllm-models
chaiml-grpo-glm46-combi-79432-v6-uploader: /usr/lib/python3/dist-packages/S3/BaseUtils.py:56: SyntaxWarning: invalid escape sequence '\.'
chaiml-grpo-glm46-combi-79432-v6-uploader:   RE_S3_DATESTRING = re.compile('\.[0-9]*(?:[Z\\-\\+]*?)')
chaiml-grpo-glm46-combi-79432-v6-uploader: /usr/lib/python3/dist-packages/S3/BaseUtils.py:57: SyntaxWarning: invalid escape sequence '\s'
chaiml-grpo-glm46-combi-79432-v6-uploader:   RE_XML_NAMESPACE = re.compile(b'^(<?[^>]+?>\s*|\s*)(<\w+) xmlns=[\'"](https?://[^\'"]+)[\'"]', re.MULTILINE)
chaiml-grpo-glm46-combi-79432-v6-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:240: SyntaxWarning: invalid escape sequence '\.'
chaiml-grpo-glm46-combi-79432-v6-uploader:   invalid = re.search("([^a-z0-9\.-])", bucket, re.UNICODE)
chaiml-grpo-glm46-combi-79432-v6-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:244: SyntaxWarning: invalid escape sequence '\.'
chaiml-grpo-glm46-combi-79432-v6-uploader:   invalid = re.search("([^A-Za-z0-9\._-])", bucket, re.UNICODE)
chaiml-grpo-glm46-combi-79432-v6-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:255: SyntaxWarning: invalid escape sequence '\.'
chaiml-grpo-glm46-combi-79432-v6-uploader:   if re.search("-\.", bucket, re.UNICODE):
chaiml-grpo-glm46-combi-79432-v6-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:257: SyntaxWarning: invalid escape sequence '\.'
chaiml-grpo-glm46-combi-79432-v6-uploader:   if re.search("\.\.", bucket, re.UNICODE):
chaiml-grpo-glm46-combi-79432-v6-uploader: /usr/lib/python3/dist-packages/S3/S3Uri.py:155: SyntaxWarning: invalid escape sequence '\w'
chaiml-grpo-glm46-combi-79432-v6-uploader:   _re = re.compile("^(\w+://)?(.*)", re.UNICODE)
chaiml-grpo-glm46-combi-79432-v6-uploader: /usr/lib/python3/dist-packages/S3/FileLists.py:480: SyntaxWarning: invalid escape sequence '\*'
chaiml-grpo-glm46-combi-79432-v6-uploader:   wildcard_split_result = re.split("\*|\?", uri_str, maxsplit=1)
chaiml-grpo-glm46-combi-79432-v6-uploader: Bucket 's3://guanaco-vllm-models/' created
chaiml-grpo-glm46-combi-79432-v6-uploader: uploading /dev/shm/model_output to s3://guanaco-vllm-models/chaiml-grpo-glm46-combi-79432-v6/default
chaiml-grpo-glm46-combi-79432-v6-uploader: cp /dev/shm/model_output/chat_template.jinja s3://guanaco-vllm-models/chaiml-grpo-glm46-combi-79432-v6/default/chat_template.jinja
chaiml-grpo-glm46-combi-79432-v6-uploader: cp /dev/shm/model_output/.gitattributes s3://guanaco-vllm-models/chaiml-grpo-glm46-combi-79432-v6/default/.gitattributes
chaiml-grpo-glm46-combi-79432-v6-uploader: cp /dev/shm/model_output/config.json s3://guanaco-vllm-models/chaiml-grpo-glm46-combi-79432-v6/default/config.json
chaiml-grpo-glm46-combi-79432-v6-uploader: cp /dev/shm/model_output/generation_config.json s3://guanaco-vllm-models/chaiml-grpo-glm46-combi-79432-v6/default/generation_config.json
chaiml-grpo-glm46-combi-79432-v6-uploader: cp /dev/shm/model_output/tokenizer_config.json s3://guanaco-vllm-models/chaiml-grpo-glm46-combi-79432-v6/default/tokenizer_config.json
chaiml-grpo-glm46-combi-79432-v6-uploader: cp /dev/shm/model_output/special_tokens_map.json s3://guanaco-vllm-models/chaiml-grpo-glm46-combi-79432-v6/default/special_tokens_map.json
chaiml-grpo-glm46-combi-79432-v6-uploader: cp /dev/shm/model_output/recipe.yaml s3://guanaco-vllm-models/chaiml-grpo-glm46-combi-79432-v6/default/recipe.yaml
chaiml-grpo-glm46-combi-79432-v6-uploader: cp /dev/shm/model_output/model.safetensors.index.json s3://guanaco-vllm-models/chaiml-grpo-glm46-combi-79432-v6/default/model.safetensors.index.json
chaiml-grpo-glm46-combi-79432-v6-uploader: cp /dev/shm/model_output/tokenizer.json s3://guanaco-vllm-models/chaiml-grpo-glm46-combi-79432-v6/default/tokenizer.json
chaiml-grpo-glm46-combi-79432-v6-uploader: cp /dev/shm/model_output/model-00072-of-00072.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm46-combi-79432-v6/default/model-00072-of-00072.safetensors
2026-03-19T15:13:49.039011+00:00 monitor updated for chaiml-grpo-glm46-combi_79432_v6
chaiml-grpo-glm46-combi-79432-v6-uploader: cp /dev/shm/model_output/model-00071-of-00072.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm46-combi-79432-v6/default/model-00071-of-00072.safetensors
chaiml-grpo-glm46-combi-79432-v6-uploader: cp /dev/shm/model_output/model-00055-of-00072.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm46-combi-79432-v6/default/model-00055-of-00072.safetensors
chaiml-grpo-glm46-combi-79432-v6-uploader: cp /dev/shm/model_output/model-00042-of-00072.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm46-combi-79432-v6/default/model-00042-of-00072.safetensors
chaiml-grpo-glm46-combi-79432-v6-uploader: cp /dev/shm/model_output/model-00013-of-00072.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm46-combi-79432-v6/default/model-00013-of-00072.safetensors
chaiml-grpo-glm46-combi-79432-v6-uploader: cp /dev/shm/model_output/model-00046-of-00072.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm46-combi-79432-v6/default/model-00046-of-00072.safetensors
chaiml-grpo-glm46-combi-79432-v6-uploader: cp /dev/shm/model_output/model-00001-of-00072.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm46-combi-79432-v6/default/model-00001-of-00072.safetensors
chaiml-grpo-glm46-combi-79432-v6-uploader: cp /dev/shm/model_output/model-00015-of-00072.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm46-combi-79432-v6/default/model-00015-of-00072.safetensors
chaiml-grpo-glm46-combi-79432-v6-uploader: cp /dev/shm/model_output/model-00020-of-00072.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm46-combi-79432-v6/default/model-00020-of-00072.safetensors
chaiml-grpo-glm46-combi-79432-v6-uploader: cp /dev/shm/model_output/model-00012-of-00072.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm46-combi-79432-v6/default/model-00012-of-00072.safetensors
chaiml-grpo-glm46-combi-79432-v6-uploader: cp /dev/shm/model_output/model-00002-of-00072.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm46-combi-79432-v6/default/model-00002-of-00072.safetensors
chaiml-grpo-glm46-combi-79432-v6-uploader: cp /dev/shm/model_output/model-00006-of-00072.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm46-combi-79432-v6/default/model-00006-of-00072.safetensors
chaiml-grpo-glm46-combi-79432-v6-uploader: cp /dev/shm/model_output/model-00043-of-00072.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm46-combi-79432-v6/default/model-00043-of-00072.safetensors
chaiml-grpo-glm46-combi-79432-v6-uploader: cp /dev/shm/model_output/model-00066-of-00072.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm46-combi-79432-v6/default/model-00066-of-00072.safetensors
chaiml-grpo-glm46-combi-79432-v6-uploader: cp /dev/shm/model_output/model-00068-of-00072.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm46-combi-79432-v6/default/model-00068-of-00072.safetensors
chaiml-grpo-glm46-combi-79432-v6-uploader: cp /dev/shm/model_output/model-00049-of-00072.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm46-combi-79432-v6/default/model-00049-of-00072.safetensors
chaiml-grpo-glm46-combi-79432-v6-uploader: cp /dev/shm/model_output/model-00064-of-00072.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm46-combi-79432-v6/default/model-00064-of-00072.safetensors
chaiml-grpo-glm46-combi-79432-v6-uploader: cp /dev/shm/model_output/model-00044-of-00072.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm46-combi-79432-v6/default/model-00044-of-00072.safetensors
chaiml-grpo-glm46-combi-79432-v6-uploader: cp /dev/shm/model_output/model-00032-of-00072.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm46-combi-79432-v6/default/model-00032-of-00072.safetensors
chaiml-grpo-glm46-combi-79432-v6-uploader: cp /dev/shm/model_output/model-00031-of-00072.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm46-combi-79432-v6/default/model-00031-of-00072.safetensors
chaiml-grpo-glm46-combi-79432-v6-uploader: cp /dev/shm/model_output/model-00009-of-00072.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm46-combi-79432-v6/default/model-00009-of-00072.safetensors
chaiml-grpo-glm46-combi-79432-v6-uploader: cp /dev/shm/model_output/model-00061-of-00072.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm46-combi-79432-v6/default/model-00061-of-00072.safetensors
chaiml-grpo-glm46-combi-79432-v6-uploader: cp /dev/shm/model_output/model-00029-of-00072.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm46-combi-79432-v6/default/model-00029-of-00072.safetensors
chaiml-grpo-glm46-combi-79432-v6-uploader: cp /dev/shm/model_output/model-00050-of-00072.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm46-combi-79432-v6/default/model-00050-of-00072.safetensors
chaiml-grpo-glm46-combi-79432-v6-uploader: cp /dev/shm/model_output/model-00011-of-00072.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm46-combi-79432-v6/default/model-00011-of-00072.safetensors
chaiml-grpo-glm46-combi-79432-v6-uploader: cp /dev/shm/model_output/model-00019-of-00072.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm46-combi-79432-v6/default/model-00019-of-00072.safetensors
chaiml-grpo-glm46-combi-79432-v6-uploader: cp /dev/shm/model_output/model-00048-of-00072.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm46-combi-79432-v6/default/model-00048-of-00072.safetensors
chaiml-grpo-glm46-combi-79432-v6-uploader: cp /dev/shm/model_output/model-00067-of-00072.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm46-combi-79432-v6/default/model-00067-of-00072.safetensors
chaiml-grpo-glm46-combi-79432-v6-uploader: cp /dev/shm/model_output/model-00065-of-00072.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm46-combi-79432-v6/default/model-00065-of-00072.safetensors
chaiml-grpo-glm46-combi-79432-v6-uploader: cp /dev/shm/model_output/model-00060-of-00072.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm46-combi-79432-v6/default/model-00060-of-00072.safetensors
chaiml-grpo-glm46-combi-79432-v6-uploader: cp /dev/shm/model_output/model-00022-of-00072.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm46-combi-79432-v6/default/model-00022-of-00072.safetensors
chaiml-grpo-glm46-combi-79432-v6-uploader: cp /dev/shm/model_output/model-00041-of-00072.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm46-combi-79432-v6/default/model-00041-of-00072.safetensors
chaiml-grpo-glm46-combi-79432-v6-uploader: cp /dev/shm/model_output/model-00028-of-00072.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm46-combi-79432-v6/default/model-00028-of-00072.safetensors
chaiml-grpo-glm46-combi-79432-v6-uploader: cp /dev/shm/model_output/model-00007-of-00072.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm46-combi-79432-v6/default/model-00007-of-00072.safetensors
chaiml-grpo-glm46-combi-79432-v6-uploader: cp /dev/shm/model_output/model-00014-of-00072.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm46-combi-79432-v6/default/model-00014-of-00072.safetensors
chaiml-grpo-glm46-combi-79432-v6-uploader: cp /dev/shm/model_output/model-00021-of-00072.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm46-combi-79432-v6/default/model-00021-of-00072.safetensors
chaiml-grpo-glm46-combi-79432-v6-uploader: cp /dev/shm/model_output/model-00039-of-00072.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm46-combi-79432-v6/default/model-00039-of-00072.safetensors
chaiml-grpo-glm46-combi-79432-v6-uploader: cp /dev/shm/model_output/model-00033-of-00072.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm46-combi-79432-v6/default/model-00033-of-00072.safetensors
chaiml-grpo-glm46-combi-79432-v6-uploader: cp /dev/shm/model_output/model-00004-of-00072.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm46-combi-79432-v6/default/model-00004-of-00072.safetensors
chaiml-grpo-glm46-combi-79432-v6-uploader: cp /dev/shm/model_output/model-00027-of-00072.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm46-combi-79432-v6/default/model-00027-of-00072.safetensors
chaiml-grpo-glm46-combi-79432-v6-uploader: cp /dev/shm/model_output/model-00016-of-00072.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm46-combi-79432-v6/default/model-00016-of-00072.safetensors
chaiml-grpo-glm46-combi-79432-v6-uploader: cp /dev/shm/model_output/model-00063-of-00072.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm46-combi-79432-v6/default/model-00063-of-00072.safetensors
chaiml-grpo-glm46-combi-79432-v6-uploader: cp /dev/shm/model_output/model-00030-of-00072.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm46-combi-79432-v6/default/model-00030-of-00072.safetensors
chaiml-grpo-glm46-combi-79432-v6-uploader: cp /dev/shm/model_output/model-00017-of-00072.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm46-combi-79432-v6/default/model-00017-of-00072.safetensors
chaiml-grpo-glm46-combi-79432-v6-uploader: cp /dev/shm/model_output/model-00036-of-00072.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm46-combi-79432-v6/default/model-00036-of-00072.safetensors
chaiml-grpo-glm46-combi-79432-v6-uploader: cp /dev/shm/model_output/model-00058-of-00072.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm46-combi-79432-v6/default/model-00058-of-00072.safetensors
chaiml-grpo-glm46-combi-79432-v6-uploader: cp /dev/shm/model_output/model-00045-of-00072.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm46-combi-79432-v6/default/model-00045-of-00072.safetensors
chaiml-grpo-glm46-combi-79432-v6-uploader: cp /dev/shm/model_output/model-00034-of-00072.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm46-combi-79432-v6/default/model-00034-of-00072.safetensors
chaiml-grpo-glm46-combi-79432-v6-uploader: cp /dev/shm/model_output/model-00053-of-00072.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm46-combi-79432-v6/default/model-00053-of-00072.safetensors
chaiml-grpo-glm46-combi-79432-v6-uploader: cp /dev/shm/model_output/model-00040-of-00072.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm46-combi-79432-v6/default/model-00040-of-00072.safetensors
chaiml-grpo-glm46-combi-79432-v6-uploader: cp /dev/shm/model_output/model-00038-of-00072.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm46-combi-79432-v6/default/model-00038-of-00072.safetensors
chaiml-grpo-glm46-combi-79432-v6-uploader: cp /dev/shm/model_output/model-00010-of-00072.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm46-combi-79432-v6/default/model-00010-of-00072.safetensors
chaiml-grpo-glm46-combi-79432-v6-uploader: cp /dev/shm/model_output/model-00062-of-00072.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm46-combi-79432-v6/default/model-00062-of-00072.safetensors
chaiml-grpo-glm46-combi-79432-v6-uploader: cp /dev/shm/model_output/model-00008-of-00072.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm46-combi-79432-v6/default/model-00008-of-00072.safetensors
chaiml-grpo-glm46-combi-79432-v6-uploader: cp /dev/shm/model_output/model-00018-of-00072.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm46-combi-79432-v6/default/model-00018-of-00072.safetensors
chaiml-grpo-glm46-combi-79432-v6-uploader: cp /dev/shm/model_output/model-00052-of-00072.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm46-combi-79432-v6/default/model-00052-of-00072.safetensors
chaiml-grpo-glm46-combi-79432-v6-uploader: cp /dev/shm/model_output/model-00025-of-00072.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm46-combi-79432-v6/default/model-00025-of-00072.safetensors
chaiml-grpo-glm46-combi-79432-v6-uploader: cp /dev/shm/model_output/model-00003-of-00072.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm46-combi-79432-v6/default/model-00003-of-00072.safetensors
chaiml-grpo-glm46-combi-79432-v6-uploader: cp /dev/shm/model_output/model-00056-of-00072.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm46-combi-79432-v6/default/model-00056-of-00072.safetensors
chaiml-grpo-glm46-combi-79432-v6-uploader: cp /dev/shm/model_output/model-00070-of-00072.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm46-combi-79432-v6/default/model-00070-of-00072.safetensors
chaiml-grpo-glm46-combi-79432-v6-uploader: cp /dev/shm/model_output/model-00051-of-00072.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm46-combi-79432-v6/default/model-00051-of-00072.safetensors
chaiml-grpo-glm46-combi-79432-v6-uploader: cp /dev/shm/model_output/model-00005-of-00072.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm46-combi-79432-v6/default/model-00005-of-00072.safetensors
chaiml-grpo-glm46-combi-79432-v6-uploader: cp /dev/shm/model_output/model-00026-of-00072.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm46-combi-79432-v6/default/model-00026-of-00072.safetensors
chaiml-grpo-glm46-combi-79432-v6-uploader: cp /dev/shm/model_output/model-00057-of-00072.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm46-combi-79432-v6/default/model-00057-of-00072.safetensors
chaiml-grpo-glm46-combi-79432-v6-uploader: cp /dev/shm/model_output/model-00054-of-00072.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm46-combi-79432-v6/default/model-00054-of-00072.safetensors
chaiml-grpo-glm46-combi-79432-v6-uploader: cp /dev/shm/model_output/model-00069-of-00072.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm46-combi-79432-v6/default/model-00069-of-00072.safetensors
chaiml-grpo-glm46-combi-79432-v6-uploader: cp /dev/shm/model_output/model-00047-of-00072.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm46-combi-79432-v6/default/model-00047-of-00072.safetensors
chaiml-grpo-glm46-combi-79432-v6-uploader: cp /dev/shm/model_output/model-00035-of-00072.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm46-combi-79432-v6/default/model-00035-of-00072.safetensors
chaiml-grpo-glm46-combi-79432-v6-uploader: cp /dev/shm/model_output/model-00059-of-00072.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm46-combi-79432-v6/default/model-00059-of-00072.safetensors
chaiml-grpo-glm46-combi-79432-v6-uploader: cp /dev/shm/model_output/model-00024-of-00072.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm46-combi-79432-v6/default/model-00024-of-00072.safetensors
chaiml-grpo-glm46-combi-79432-v6-uploader: cp /dev/shm/model_output/model-00037-of-00072.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm46-combi-79432-v6/default/model-00037-of-00072.safetensors
chaiml-grpo-glm46-combi-79432-v6-uploader: cp /dev/shm/model_output/model-00023-of-00072.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm46-combi-79432-v6/default/model-00023-of-00072.safetensors
2026-03-19T15:14:49.151954+00:00 monitor updated for chaiml-grpo-glm46-combi_79432_v6
Job chaiml-grpo-glm46-combi-79432-v6-uploader completed after 309.14s with status: succeeded
Stopping job with name chaiml-grpo-glm46-combi-79432-v6-uploader
Pipeline stage VLLMUploader completed in 309.66s
run pipeline stage %s
Running pipeline stage VLLMTemplater
Pipeline stage VLLMTemplater completed in 2.34s
run pipeline stage %s
Running pipeline stage VLLMDeployer
Creating inference service chaiml-grpo-glm46-combi-79432-v6
Waiting for inference service chaiml-grpo-glm46-combi-79432-v6 to be ready
2026-03-19T15:15:49.260497+00:00 monitor updated for chaiml-grpo-glm46-combi_79432_v6
2026-03-19T15:16:49.368327+00:00 monitor updated for chaiml-grpo-glm46-combi_79432_v6
2026-03-19T15:17:49.487696+00:00 monitor updated for chaiml-grpo-glm46-combi_79432_v6
2026-03-19T15:18:49.593358+00:00 monitor updated for chaiml-grpo-glm46-combi_79432_v6
2026-03-19T15:19:49.699524+00:00 monitor updated for chaiml-grpo-glm46-combi_79432_v6
2026-03-19T15:20:49.806982+00:00 monitor updated for chaiml-grpo-glm46-combi_79432_v6
2026-03-19T15:21:49.910170+00:00 monitor updated for chaiml-grpo-glm46-combi_79432_v6
2026-03-19T15:22:50.010820+00:00 monitor updated for chaiml-grpo-glm46-combi_79432_v6
2026-03-19T15:23:50.109663+00:00 monitor updated for chaiml-grpo-glm46-combi_79432_v6
2026-03-19T15:24:50.212389+00:00 monitor updated for chaiml-grpo-glm46-combi_79432_v6
2026-03-19T15:25:50.330834+00:00 monitor updated for chaiml-grpo-glm46-combi_79432_v6
2026-03-19T15:26:50.441869+00:00 monitor updated for chaiml-grpo-glm46-combi_79432_v6
2026-03-19T15:27:50.552799+00:00 monitor updated for chaiml-grpo-glm46-combi_79432_v6
2026-03-19T15:28:50.661829+00:00 monitor updated for chaiml-grpo-glm46-combi_79432_v6
2026-03-19T15:29:50.780559+00:00 monitor updated for chaiml-grpo-glm46-combi_79432_v6
2026-03-19T15:30:50.888248+00:00 monitor updated for chaiml-grpo-glm46-combi_79432_v6
2026-03-19T15:31:51.004074+00:00 monitor updated for chaiml-grpo-glm46-combi_79432_v6
2026-03-19T15:32:51.115919+00:00 monitor updated for chaiml-grpo-glm46-combi_79432_v6
2026-03-19T15:33:51.223045+00:00 monitor updated for chaiml-grpo-glm46-combi_79432_v6
2026-03-19T15:34:51.340309+00:00 monitor updated for chaiml-grpo-glm46-combi_79432_v6
2026-03-19T15:35:51.469267+00:00 monitor updated for chaiml-grpo-glm46-combi_79432_v6
2026-03-19T15:36:51.576555+00:00 monitor updated for chaiml-grpo-glm46-combi_79432_v6
Inference service chaiml-grpo-glm46-combi-79432-v6 ready after 1316.9271078109741s
Pipeline stage VLLMDeployer completed in 1318.21s
run pipeline stage %s
Running pipeline stage StressChecker
HTTPConnectionPool(host='guanaco-submitter.guanaco-backend.kchai-google-us-east4.chaiverse.com', port=80): Read timed out. (read timeout=20)
Received unhealthy response to inference request!
HTTPConnectionPool(host='guanaco-submitter.guanaco-backend.kchai-google-us-east4.chaiverse.com', port=80): Read timed out. (read timeout=20)
Received unhealthy response to inference request!
2026-03-19T15:37:51.690119+00:00 monitor updated for chaiml-grpo-glm46-combi_79432_v6
HTTPConnectionPool(host='guanaco-submitter.guanaco-backend.kchai-google-us-east4.chaiverse.com', port=80): Read timed out. (read timeout=20)
Received unhealthy response to inference request!
Received healthy response to inference request in 19.678547382354736s
Received healthy response to inference request in 2.2948009967803955s
Received healthy response to inference request in 1.9265682697296143s
Received healthy response to inference request in 1.9510881900787354s
Received healthy response to inference request in 1.933312177658081s
Received healthy response to inference request in 1.9630258083343506s
Received healthy response to inference request in 2.053088903427124s
Received healthy response to inference request in 1.9345264434814453s
Received healthy response to inference request in 2.0030736923217773s
Received healthy response to inference request in 2.030102014541626s
Received healthy response to inference request in 1.9509263038635254s
Received healthy response to inference request in 1.8682212829589844s
Received healthy response to inference request in 2.097377300262451s
Received healthy response to inference request in 1.8915033340454102s
Received healthy response to inference request in 2.012512445449829s
Received healthy response to inference request in 1.964120626449585s
2026-03-19T15:38:51.802779+00:00 monitor updated for chaiml-grpo-glm46-combi_79432_v6
Received healthy response to inference request in 2.0383801460266113s
Received healthy response to inference request in 1.9329547882080078s
Received healthy response to inference request in 2.059682846069336s
Received healthy response to inference request in 1.939790964126587s
Received healthy response to inference request in 1.9879038333892822s
Received healthy response to inference request in 2.023116111755371s
Received healthy response to inference request in 1.9332945346832275s
Received healthy response to inference request in 1.950989007949829s
Received healthy response to inference request in 1.9350645542144775s
Received healthy response to inference request in 2.020193099975586s
Received healthy response to inference request in 2.1975343227386475s
30 requests
3 failed requests
5th percentile: 1.907282555103302
10th percentile: 1.9323161363601684
20th percentile: 1.9342835903167725
30th percentile: 1.9475857019424438
40th percentile: 1.9582507610321045
50th percentile: 1.9954887628555298
60th percentile: 2.0213623046875
70th percentile: 2.0427927732467652
80th percentile: 2.1174087047576906
90th percentile: 19.726142024993898
95th percentile: 20.15584969520569
99th percentile: 20.178758404254914
mean time: 4.402360486984253
%s, retrying in %s seconds...
Received healthy response to inference request in 1.926377773284912s
Received healthy response to inference request in 1.9269206523895264s
Received healthy response to inference request in 1.954728364944458s
Received healthy response to inference request in 2.007197141647339s
Received healthy response to inference request in 2.1104655265808105s
Received healthy response to inference request in 2.0245065689086914s
Received healthy response to inference request in 1.9629979133605957s
Received healthy response to inference request in 1.934709072113037s
Received healthy response to inference request in 1.940089464187622s
Received healthy response to inference request in 2.0466413497924805s
Received healthy response to inference request in 1.9996750354766846s
Received healthy response to inference request in 2.047024965286255s
Received healthy response to inference request in 2.0777626037597656s
Received healthy response to inference request in 2.0052709579467773s
Received healthy response to inference request in 2.168672561645508s
Received healthy response to inference request in 1.9262993335723877s
Received healthy response to inference request in 1.9052577018737793s
Received healthy response to inference request in 1.971825122833252s
2026-03-19T15:39:51.921555+00:00 monitor updated for chaiml-grpo-glm46-combi_79432_v6
Received healthy response to inference request in 2.1552700996398926s
Received healthy response to inference request in 2.128282070159912s
Received healthy response to inference request in 2.013699531555176s
Received healthy response to inference request in 2.055668592453003s
Received healthy response to inference request in 1.8913488388061523s
Received healthy response to inference request in 1.9195940494537354s
Received healthy response to inference request in 2.1771275997161865s
Received healthy response to inference request in 1.9293501377105713s
Received healthy response to inference request in 2.1643269062042236s
Received healthy response to inference request in 2.161094903945923s
Received healthy response to inference request in 2.0667366981506348s
Received healthy response to inference request in 2.1208906173706055s
30 requests
0 failed requests
5th percentile: 1.9117090582847596
10th percentile: 1.9256288051605224
20th percentile: 1.9288642406463623
30th percentile: 1.9503366947174072
40th percentile: 1.9885350704193117
50th percentile: 2.0104483366012573
60th percentile: 2.04679479598999
70th percentile: 2.070044469833374
80th percentile: 2.122368907928467
90th percentile: 2.161418104171753
95th percentile: 2.16671701669693
99th percentile: 2.17467563867569
mean time: 2.02399373849233
Pipeline stage StressChecker completed in 199.35s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyTriggerPipeline
run_pipeline:run_in_cloud %s
starting trigger_guanaco_pipeline args=%s
triggered trigger_guanaco_pipeline args=%s
Pipeline stage OfflineFamilyFriendlyTriggerPipeline completed in 0.88s
Shutdown handler de-registered
chaiml-grpo-glm46-combi_79432_v6 status is now deployed due to DeploymentManager action
chaiml-grpo-glm46-combi_79432_v6 status is now inactive due to auto deactivation removed underperforming models