chaiml-grpo-glm50-combi_43081

developer_uid: chai_backend_admin
submission_id: chaiml-grpo-glm50-combi_43081_v1
model_name: chaiml-grpo-glm50-combi_43081_v1
model_group: ChaiML/grpo-glm50-combin
status: inactive
timestamp: 2026-03-29T08:38:34+00:00
num_battles: 10914
num_wins: 4921
celo_rating: 1260.01
family_friendly_score: 0.0
family_friendly_standard_error: 0.0
submission_type: basic
model_repo: ChaiML/grpo-glm50-combine-rm721-mega-kimi-mix8020-step400-int4-fast
model_architecture: GlmMoeDsaForCausalLM
model_num_parameters: 31348144128.0
best_of: 8
max_input_tokens: 1500
max_output_tokens: 80
reward_model: default
display_name: chaiml-grpo-glm50-combi_43081_v1
ineligible_reason: max_output_tokens!=64
is_internal_developer: True
language_model: ChaiML/grpo-glm50-combine-rm721-mega-kimi-mix8020-step400-int4-fast
model_size: 31B
ranking_group: single
us_pacific_date: 2026-03-29
win_ratio: 0.4508887667216419
generation_params: {'temperature': 1.0, 'top_p': 0.95, 'min_p': 0.0, 'top_k': 60, 'presence_penalty': 0.0, 'frequency_penalty': 0.0, 'stopping_words': ['</s>', '####', '<|im_end|>', '<|assistant|>', '<|user|>'], 'max_input_tokens': 1500, 'best_of': 8, 'max_output_tokens': 80}
formatter: {'memory_template': "[gMASK]<sop><|system|>{bot_name}'s persona: {memory}", 'prompt_template': '', 'bot_template': '<|assistant|>{message}', 'user_template': '<|user|>{message}', 'response_template': '<|assistant|></think>', 'truncate_by_message': True}
Resubmit model
Shutdown handler not registered because Python interpreter is not running in the main thread
run pipeline %s
run pipeline stage %s
Running pipeline stage VLLMUploader
Starting job with name chaiml-grpo-glm50-combi-43081-v1-uploader
Waiting for job on chaiml-grpo-glm50-combi-43081-v1-uploader to finish
chaiml-grpo-glm50-combi-43081-v1-uploader: {repo_id} is already quantized
chaiml-grpo-glm50-combi-43081-v1-uploader: Using quantization_mode: none
chaiml-grpo-glm50-combi-43081-v1-uploader: Downloading snapshot of ChaiML/grpo-glm50-combine-rm721-mega-kimi-mix8020-step400-int4-fast...
2026-03-29T06:47:29.933913+00:00 monitor updated for chaiml-grpo-glm50-combi_43081_v1
2026-03-29T06:48:30.021486+00:00 monitor updated for chaiml-grpo-glm50-combi_43081_v1
chaiml-grpo-glm50-combi-43081-v1-uploader: Downloaded in 121.324s
2026-03-29T06:49:30.118833+00:00 monitor updated for chaiml-grpo-glm50-combi_43081_v1
2026-03-29T06:50:30.225371+00:00 monitor updated for chaiml-grpo-glm50-combi_43081_v1
2026-03-29T06:51:30.358397+00:00 monitor updated for chaiml-grpo-glm50-combi_43081_v1
chaiml-grpo-glm50-combi-43081-v1-uploader: Processed model ChaiML/grpo-glm50-combine-rm721-mega-kimi-mix8020-step400-int4-fast in 313.265s
chaiml-grpo-glm50-combi-43081-v1-uploader: creating bucket guanaco-vllm-models
chaiml-grpo-glm50-combi-43081-v1-uploader: /usr/lib/python3/dist-packages/S3/BaseUtils.py:56: SyntaxWarning: invalid escape sequence '\.'
chaiml-grpo-glm50-combi-43081-v1-uploader:   RE_S3_DATESTRING = re.compile('\.[0-9]*(?:[Z\\-\\+]*?)')
chaiml-grpo-glm50-combi-43081-v1-uploader: /usr/lib/python3/dist-packages/S3/BaseUtils.py:57: SyntaxWarning: invalid escape sequence '\s'
chaiml-grpo-glm50-combi-43081-v1-uploader:   RE_XML_NAMESPACE = re.compile(b'^(<?[^>]+?>\s*|\s*)(<\w+) xmlns=[\'"](https?://[^\'"]+)[\'"]', re.MULTILINE)
chaiml-grpo-glm50-combi-43081-v1-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:240: SyntaxWarning: invalid escape sequence '\.'
chaiml-grpo-glm50-combi-43081-v1-uploader:   invalid = re.search("([^a-z0-9\.-])", bucket, re.UNICODE)
chaiml-grpo-glm50-combi-43081-v1-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:244: SyntaxWarning: invalid escape sequence '\.'
chaiml-grpo-glm50-combi-43081-v1-uploader:   invalid = re.search("([^A-Za-z0-9\._-])", bucket, re.UNICODE)
chaiml-grpo-glm50-combi-43081-v1-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:255: SyntaxWarning: invalid escape sequence '\.'
chaiml-grpo-glm50-combi-43081-v1-uploader:   if re.search("-\.", bucket, re.UNICODE):
chaiml-grpo-glm50-combi-43081-v1-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:257: SyntaxWarning: invalid escape sequence '\.'
chaiml-grpo-glm50-combi-43081-v1-uploader:   if re.search("\.\.", bucket, re.UNICODE):
chaiml-grpo-glm50-combi-43081-v1-uploader: /usr/lib/python3/dist-packages/S3/S3Uri.py:155: SyntaxWarning: invalid escape sequence '\w'
chaiml-grpo-glm50-combi-43081-v1-uploader:   _re = re.compile("^(\w+://)?(.*)", re.UNICODE)
chaiml-grpo-glm50-combi-43081-v1-uploader: /usr/lib/python3/dist-packages/S3/FileLists.py:480: SyntaxWarning: invalid escape sequence '\*'
chaiml-grpo-glm50-combi-43081-v1-uploader:   wildcard_split_result = re.split("\*|\?", uri_str, maxsplit=1)
chaiml-grpo-glm50-combi-43081-v1-uploader: Bucket 's3://guanaco-vllm-models/' created
chaiml-grpo-glm50-combi-43081-v1-uploader: uploading /dev/shm/model_output to s3://guanaco-vllm-models/chaiml-grpo-glm50-combi-43081-v1/default
chaiml-grpo-glm50-combi-43081-v1-uploader: cp /dev/shm/model_output/generation_config.json s3://guanaco-vllm-models/chaiml-grpo-glm50-combi-43081-v1/default/generation_config.json
chaiml-grpo-glm50-combi-43081-v1-uploader: cp /dev/shm/model_output/.gitattributes s3://guanaco-vllm-models/chaiml-grpo-glm50-combi-43081-v1/default/.gitattributes
chaiml-grpo-glm50-combi-43081-v1-uploader: cp /dev/shm/model_output/chat_template.jinja s3://guanaco-vllm-models/chaiml-grpo-glm50-combi-43081-v1/default/chat_template.jinja
chaiml-grpo-glm50-combi-43081-v1-uploader: cp /dev/shm/model_output/config.json s3://guanaco-vllm-models/chaiml-grpo-glm50-combi-43081-v1/default/config.json
chaiml-grpo-glm50-combi-43081-v1-uploader: cp /dev/shm/model_output/model.safetensors.index.json s3://guanaco-vllm-models/chaiml-grpo-glm50-combi-43081-v1/default/model.safetensors.index.json
chaiml-grpo-glm50-combi-43081-v1-uploader: cp /dev/shm/model_output/tokenizer.json s3://guanaco-vllm-models/chaiml-grpo-glm50-combi-43081-v1/default/tokenizer.json
2026-03-29T06:52:30.458424+00:00 monitor updated for chaiml-grpo-glm50-combi_43081_v1
chaiml-grpo-glm50-combi-43081-v1-uploader: cp /dev/shm/model_output/model-00077-of-00078.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm50-combi-43081-v1/default/model-00077-of-00078.safetensors
chaiml-grpo-glm50-combi-43081-v1-uploader: cp /dev/shm/model_output/model-00078-of-00078.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm50-combi-43081-v1/default/model-00078-of-00078.safetensors
2026-03-29T06:53:30.548976+00:00 monitor updated for chaiml-grpo-glm50-combi_43081_v1
chaiml-grpo-glm50-combi-43081-v1-uploader: cp /dev/shm/model_output/model-00036-of-00078.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm50-combi-43081-v1/default/model-00036-of-00078.safetensors
chaiml-grpo-glm50-combi-43081-v1-uploader: cp /dev/shm/model_output/model-00010-of-00078.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm50-combi-43081-v1/default/model-00010-of-00078.safetensors
chaiml-grpo-glm50-combi-43081-v1-uploader: cp /dev/shm/model_output/model-00024-of-00078.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm50-combi-43081-v1/default/model-00024-of-00078.safetensors
chaiml-grpo-glm50-combi-43081-v1-uploader: cp /dev/shm/model_output/model-00052-of-00078.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm50-combi-43081-v1/default/model-00052-of-00078.safetensors
chaiml-grpo-glm50-combi-43081-v1-uploader: cp /dev/shm/model_output/model-00032-of-00078.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm50-combi-43081-v1/default/model-00032-of-00078.safetensors
chaiml-grpo-glm50-combi-43081-v1-uploader: cp /dev/shm/model_output/model-00070-of-00078.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm50-combi-43081-v1/default/model-00070-of-00078.safetensors
chaiml-grpo-glm50-combi-43081-v1-uploader: cp /dev/shm/model_output/model-00007-of-00078.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm50-combi-43081-v1/default/model-00007-of-00078.safetensors
chaiml-grpo-glm50-combi-43081-v1-uploader: cp /dev/shm/model_output/model-00018-of-00078.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm50-combi-43081-v1/default/model-00018-of-00078.safetensors
chaiml-grpo-glm50-combi-43081-v1-uploader: cp /dev/shm/model_output/model-00011-of-00078.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm50-combi-43081-v1/default/model-00011-of-00078.safetensors
chaiml-grpo-glm50-combi-43081-v1-uploader: cp /dev/shm/model_output/model-00034-of-00078.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm50-combi-43081-v1/default/model-00034-of-00078.safetensors
chaiml-grpo-glm50-combi-43081-v1-uploader: cp /dev/shm/model_output/model-00027-of-00078.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm50-combi-43081-v1/default/model-00027-of-00078.safetensors
chaiml-grpo-glm50-combi-43081-v1-uploader: cp /dev/shm/model_output/model-00028-of-00078.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm50-combi-43081-v1/default/model-00028-of-00078.safetensors
chaiml-grpo-glm50-combi-43081-v1-uploader: cp /dev/shm/model_output/model-00030-of-00078.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm50-combi-43081-v1/default/model-00030-of-00078.safetensors
chaiml-grpo-glm50-combi-43081-v1-uploader: cp /dev/shm/model_output/model-00025-of-00078.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm50-combi-43081-v1/default/model-00025-of-00078.safetensors
chaiml-grpo-glm50-combi-43081-v1-uploader: cp /dev/shm/model_output/model-00040-of-00078.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm50-combi-43081-v1/default/model-00040-of-00078.safetensors
chaiml-grpo-glm50-combi-43081-v1-uploader: cp /dev/shm/model_output/model-00044-of-00078.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm50-combi-43081-v1/default/model-00044-of-00078.safetensors
chaiml-grpo-glm50-combi-43081-v1-uploader: cp /dev/shm/model_output/model-00021-of-00078.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm50-combi-43081-v1/default/model-00021-of-00078.safetensors
chaiml-grpo-glm50-combi-43081-v1-uploader: cp /dev/shm/model_output/model-00008-of-00078.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm50-combi-43081-v1/default/model-00008-of-00078.safetensors
chaiml-grpo-glm50-combi-43081-v1-uploader: cp /dev/shm/model_output/model-00015-of-00078.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm50-combi-43081-v1/default/model-00015-of-00078.safetensors
chaiml-grpo-glm50-combi-43081-v1-uploader: cp /dev/shm/model_output/model-00023-of-00078.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm50-combi-43081-v1/default/model-00023-of-00078.safetensors
chaiml-grpo-glm50-combi-43081-v1-uploader: cp /dev/shm/model_output/model-00037-of-00078.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm50-combi-43081-v1/default/model-00037-of-00078.safetensors
chaiml-grpo-glm50-combi-43081-v1-uploader: cp /dev/shm/model_output/model-00031-of-00078.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm50-combi-43081-v1/default/model-00031-of-00078.safetensors
chaiml-grpo-glm50-combi-43081-v1-uploader: cp /dev/shm/model_output/model-00061-of-00078.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm50-combi-43081-v1/default/model-00061-of-00078.safetensors
chaiml-grpo-glm50-combi-43081-v1-uploader: cp /dev/shm/model_output/model-00053-of-00078.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm50-combi-43081-v1/default/model-00053-of-00078.safetensors
chaiml-grpo-glm50-combi-43081-v1-uploader: cp /dev/shm/model_output/model-00026-of-00078.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm50-combi-43081-v1/default/model-00026-of-00078.safetensors
chaiml-grpo-glm50-combi-43081-v1-uploader: cp /dev/shm/model_output/model-00012-of-00078.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm50-combi-43081-v1/default/model-00012-of-00078.safetensors
chaiml-grpo-glm50-combi-43081-v1-uploader: cp /dev/shm/model_output/model-00041-of-00078.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm50-combi-43081-v1/default/model-00041-of-00078.safetensors
chaiml-grpo-glm50-combi-43081-v1-uploader: cp /dev/shm/model_output/model-00045-of-00078.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm50-combi-43081-v1/default/model-00045-of-00078.safetensors
chaiml-grpo-glm50-combi-43081-v1-uploader: cp /dev/shm/model_output/model-00009-of-00078.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm50-combi-43081-v1/default/model-00009-of-00078.safetensors
chaiml-grpo-glm50-combi-43081-v1-uploader: cp /dev/shm/model_output/model-00001-of-00078.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm50-combi-43081-v1/default/model-00001-of-00078.safetensors
chaiml-grpo-glm50-combi-43081-v1-uploader: cp /dev/shm/model_output/model-00006-of-00078.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm50-combi-43081-v1/default/model-00006-of-00078.safetensors
chaiml-grpo-glm50-combi-43081-v1-uploader: cp /dev/shm/model_output/model-00005-of-00078.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm50-combi-43081-v1/default/model-00005-of-00078.safetensors
chaiml-grpo-glm50-combi-43081-v1-uploader: cp /dev/shm/model_output/model-00046-of-00078.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm50-combi-43081-v1/default/model-00046-of-00078.safetensors
chaiml-grpo-glm50-combi-43081-v1-uploader: cp /dev/shm/model_output/model-00063-of-00078.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm50-combi-43081-v1/default/model-00063-of-00078.safetensors
chaiml-grpo-glm50-combi-43081-v1-uploader: cp /dev/shm/model_output/model-00035-of-00078.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm50-combi-43081-v1/default/model-00035-of-00078.safetensors
chaiml-grpo-glm50-combi-43081-v1-uploader: cp /dev/shm/model_output/model-00019-of-00078.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm50-combi-43081-v1/default/model-00019-of-00078.safetensors
chaiml-grpo-glm50-combi-43081-v1-uploader: cp /dev/shm/model_output/model-00048-of-00078.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm50-combi-43081-v1/default/model-00048-of-00078.safetensors
chaiml-grpo-glm50-combi-43081-v1-uploader: cp /dev/shm/model_output/model-00020-of-00078.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm50-combi-43081-v1/default/model-00020-of-00078.safetensors
chaiml-grpo-glm50-combi-43081-v1-uploader: cp /dev/shm/model_output/model-00071-of-00078.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm50-combi-43081-v1/default/model-00071-of-00078.safetensors
chaiml-grpo-glm50-combi-43081-v1-uploader: cp /dev/shm/model_output/model-00054-of-00078.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm50-combi-43081-v1/default/model-00054-of-00078.safetensors
chaiml-grpo-glm50-combi-43081-v1-uploader: cp /dev/shm/model_output/model-00029-of-00078.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm50-combi-43081-v1/default/model-00029-of-00078.safetensors
chaiml-grpo-glm50-combi-43081-v1-uploader: cp /dev/shm/model_output/model-00074-of-00078.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm50-combi-43081-v1/default/model-00074-of-00078.safetensors
chaiml-grpo-glm50-combi-43081-v1-uploader: cp /dev/shm/model_output/model-00055-of-00078.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm50-combi-43081-v1/default/model-00055-of-00078.safetensors
chaiml-grpo-glm50-combi-43081-v1-uploader: cp /dev/shm/model_output/model-00016-of-00078.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm50-combi-43081-v1/default/model-00016-of-00078.safetensors
chaiml-grpo-glm50-combi-43081-v1-uploader: cp /dev/shm/model_output/model-00059-of-00078.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm50-combi-43081-v1/default/model-00059-of-00078.safetensors
chaiml-grpo-glm50-combi-43081-v1-uploader: cp /dev/shm/model_output/model-00065-of-00078.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm50-combi-43081-v1/default/model-00065-of-00078.safetensors
chaiml-grpo-glm50-combi-43081-v1-uploader: cp /dev/shm/model_output/model-00069-of-00078.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm50-combi-43081-v1/default/model-00069-of-00078.safetensors
chaiml-grpo-glm50-combi-43081-v1-uploader: cp /dev/shm/model_output/model-00002-of-00078.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm50-combi-43081-v1/default/model-00002-of-00078.safetensors
chaiml-grpo-glm50-combi-43081-v1-uploader: cp /dev/shm/model_output/model-00067-of-00078.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm50-combi-43081-v1/default/model-00067-of-00078.safetensors
chaiml-grpo-glm50-combi-43081-v1-uploader: cp /dev/shm/model_output/model-00073-of-00078.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm50-combi-43081-v1/default/model-00073-of-00078.safetensors
chaiml-grpo-glm50-combi-43081-v1-uploader: cp /dev/shm/model_output/model-00066-of-00078.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm50-combi-43081-v1/default/model-00066-of-00078.safetensors
chaiml-grpo-glm50-combi-43081-v1-uploader: cp /dev/shm/model_output/model-00049-of-00078.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm50-combi-43081-v1/default/model-00049-of-00078.safetensors
chaiml-grpo-glm50-combi-43081-v1-uploader: cp /dev/shm/model_output/model-00062-of-00078.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm50-combi-43081-v1/default/model-00062-of-00078.safetensors
chaiml-grpo-glm50-combi-43081-v1-uploader: cp /dev/shm/model_output/model-00060-of-00078.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm50-combi-43081-v1/default/model-00060-of-00078.safetensors
chaiml-grpo-glm50-combi-43081-v1-uploader: cp /dev/shm/model_output/model-00039-of-00078.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm50-combi-43081-v1/default/model-00039-of-00078.safetensors
chaiml-grpo-glm50-combi-43081-v1-uploader: cp /dev/shm/model_output/model-00057-of-00078.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm50-combi-43081-v1/default/model-00057-of-00078.safetensors
chaiml-grpo-glm50-combi-43081-v1-uploader: cp /dev/shm/model_output/model-00014-of-00078.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm50-combi-43081-v1/default/model-00014-of-00078.safetensors
chaiml-grpo-glm50-combi-43081-v1-uploader: cp /dev/shm/model_output/model-00047-of-00078.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm50-combi-43081-v1/default/model-00047-of-00078.safetensors
chaiml-grpo-glm50-combi-43081-v1-uploader: cp /dev/shm/model_output/model-00072-of-00078.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm50-combi-43081-v1/default/model-00072-of-00078.safetensors
chaiml-grpo-glm50-combi-43081-v1-uploader: cp /dev/shm/model_output/model-00017-of-00078.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm50-combi-43081-v1/default/model-00017-of-00078.safetensors
chaiml-grpo-glm50-combi-43081-v1-uploader: cp /dev/shm/model_output/model-00051-of-00078.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm50-combi-43081-v1/default/model-00051-of-00078.safetensors
chaiml-grpo-glm50-combi-43081-v1-uploader: cp /dev/shm/model_output/model-00004-of-00078.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm50-combi-43081-v1/default/model-00004-of-00078.safetensors
chaiml-grpo-glm50-combi-43081-v1-uploader: cp /dev/shm/model_output/model-00043-of-00078.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm50-combi-43081-v1/default/model-00043-of-00078.safetensors
chaiml-grpo-glm50-combi-43081-v1-uploader: cp /dev/shm/model_output/model-00068-of-00078.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm50-combi-43081-v1/default/model-00068-of-00078.safetensors
chaiml-grpo-glm50-combi-43081-v1-uploader: cp /dev/shm/model_output/model-00076-of-00078.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm50-combi-43081-v1/default/model-00076-of-00078.safetensors
chaiml-grpo-glm50-combi-43081-v1-uploader: cp /dev/shm/model_output/model-00042-of-00078.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm50-combi-43081-v1/default/model-00042-of-00078.safetensors
chaiml-grpo-glm50-combi-43081-v1-uploader: cp /dev/shm/model_output/model-00013-of-00078.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm50-combi-43081-v1/default/model-00013-of-00078.safetensors
chaiml-grpo-glm50-combi-43081-v1-uploader: cp /dev/shm/model_output/model-00022-of-00078.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm50-combi-43081-v1/default/model-00022-of-00078.safetensors
chaiml-grpo-glm50-combi-43081-v1-uploader: cp /dev/shm/model_output/model-00075-of-00078.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm50-combi-43081-v1/default/model-00075-of-00078.safetensors
chaiml-grpo-glm50-combi-43081-v1-uploader: cp /dev/shm/model_output/model-00003-of-00078.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm50-combi-43081-v1/default/model-00003-of-00078.safetensors
chaiml-grpo-glm50-combi-43081-v1-uploader: cp /dev/shm/model_output/model-00050-of-00078.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm50-combi-43081-v1/default/model-00050-of-00078.safetensors
chaiml-grpo-glm50-combi-43081-v1-uploader: cp /dev/shm/model_output/model-00056-of-00078.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm50-combi-43081-v1/default/model-00056-of-00078.safetensors
chaiml-grpo-glm50-combi-43081-v1-uploader: cp /dev/shm/model_output/model-00038-of-00078.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm50-combi-43081-v1/default/model-00038-of-00078.safetensors
chaiml-grpo-glm50-combi-43081-v1-uploader: cp /dev/shm/model_output/model-00058-of-00078.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm50-combi-43081-v1/default/model-00058-of-00078.safetensors
chaiml-grpo-glm50-combi-43081-v1-uploader: cp /dev/shm/model_output/model-00064-of-00078.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm50-combi-43081-v1/default/model-00064-of-00078.safetensors
chaiml-grpo-glm50-combi-43081-v1-uploader: cp /dev/shm/model_output/model-00033-of-00078.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm50-combi-43081-v1/default/model-00033-of-00078.safetensors
Job chaiml-grpo-glm50-combi-43081-v1-uploader completed after 463.19s with status: succeeded
Stopping job with name chaiml-grpo-glm50-combi-43081-v1-uploader
Pipeline stage VLLMUploader completed in 464.00s
run pipeline stage %s
Running pipeline stage VLLMUploaderAMD
Pipeline stage vllm_upload_amd skipped, reason=not amd cluster
Pipeline stage VLLMUploaderAMD completed in 0.14s
run pipeline stage %s
Running pipeline stage VLLMTemplater
Pipeline stage VLLMTemplater completed in 2.29s
run pipeline stage %s
Running pipeline stage VLLMDeployer
Creating inference service chaiml-grpo-glm50-combi-43081-v1
Waiting for inference service chaiml-grpo-glm50-combi-43081-v1 to be ready
2026-03-29T06:54:30.633049+00:00 monitor updated for chaiml-grpo-glm50-combi_43081_v1
2026-03-29T06:55:30.752169+00:00 monitor updated for chaiml-grpo-glm50-combi_43081_v1
2026-03-29T06:56:32.188175+00:00 monitor updated for chaiml-grpo-glm50-combi_43081_v1
2026-03-29T06:57:32.583534+00:00 monitor updated for chaiml-grpo-glm50-combi_43081_v1
2026-03-29T06:58:32.782085+00:00 monitor updated for chaiml-grpo-glm50-combi_43081_v1
Connection pool is full, discarding connection: %s. Connection pool size: %s
2026-03-29T06:59:33.077339+00:00 monitor updated for chaiml-grpo-glm50-combi_43081_v1
2026-03-29T07:00:33.301600+00:00 monitor updated for chaiml-grpo-glm50-combi_43081_v1
2026-03-29T07:01:33.452835+00:00 monitor updated for chaiml-grpo-glm50-combi_43081_v1
2026-03-29T07:02:33.552912+00:00 monitor updated for chaiml-grpo-glm50-combi_43081_v1
Inference service chaiml-grpo-glm50-combi-43081-v1 ready after 552.6963856220245s
Pipeline stage VLLMDeployer completed in 553.18s
run pipeline stage %s
Running pipeline stage StressChecker
2026-03-29T07:03:33.695190+00:00 monitor updated for chaiml-grpo-glm50-combi_43081_v1
Received healthy response to inference request in 10.705331802368164s
Received healthy response to inference request in 2.469191074371338s
Received healthy response to inference request in 10.98272442817688s
Received healthy response to inference request in 2.498880386352539s
Received healthy response to inference request in 2.617072105407715s
Received healthy response to inference request in 2.5658762454986572s
Received healthy response to inference request in 3.4024951457977295s
Received healthy response to inference request in 2.6375372409820557s
Received healthy response to inference request in 2.539755344390869s
Received healthy response to inference request in 2.5488696098327637s
Received healthy response to inference request in 2.436222791671753s
Received healthy response to inference request in 2.609368324279785s
Received healthy response to inference request in 2.581397533416748s
Received healthy response to inference request in 2.60237979888916s
Received healthy response to inference request in 2.5674266815185547s
Received healthy response to inference request in 2.605215549468994s
Received healthy response to inference request in 2.531141757965088s
2026-03-29T07:04:33.799899+00:00 monitor updated for chaiml-grpo-glm50-combi_43081_v1
Received healthy response to inference request in 2.526414632797241s
Received healthy response to inference request in 2.4014463424682617s
Received healthy response to inference request in 2.717108964920044s
Received healthy response to inference request in 2.7273519039154053s
Received healthy response to inference request in 2.359832286834717s
Received healthy response to inference request in 2.558429479598999s
Received healthy response to inference request in 2.6598901748657227s
Received healthy response to inference request in 2.457552433013916s
Received healthy response to inference request in 2.7161500453948975s
Received healthy response to inference request in 2.637009859085083s
Received healthy response to inference request in 2.6227807998657227s
Received healthy response to inference request in 2.865213394165039s
Received healthy response to inference request in 2.6232991218566895s
30 requests
0 failed requests
5th percentile: 2.4170957446098327
10th percentile: 2.4554194688796995
20th percentile: 2.5209077835083007
30th percentile: 2.5461353302001952
40th percentile: 2.566806507110596
50th percentile: 2.603797674179077
60th percentile: 2.619355583190918
70th percentile: 2.6371680736541747
80th percentile: 2.716341829299927
90th percentile: 2.9189415693283087
95th percentile: 7.4190553069114475
99th percentile: 10.902280566692353
mean time: 3.1591121753056846
Pipeline stage StressChecker completed in 98.93s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyTriggerPipeline
run_pipeline:run_in_cloud %s
starting trigger_guanaco_pipeline args=%s
triggered trigger_guanaco_pipeline args=%s
Pipeline stage OfflineFamilyFriendlyTriggerPipeline completed in 0.85s
Shutdown handler de-registered
chaiml-grpo-glm50-combi_43081_v1 status is now deployed due to DeploymentManager action
chaiml-grpo-glm50-combi_43081_v1 status is now inactive due to auto deactivation removed underperforming models