chaiml-grpo-glm50-combi_14294

developer_uid: chai_backend_admin
submission_id: chaiml-grpo-glm50-combi_14294_v1
model_name: chaiml-grpo-glm50-combi_14294_v1
model_group: ChaiML/grpo-glm50-combin
status: inactive
timestamp: 2026-04-02T20:17:33+00:00
num_battles: 11105
num_wins: 6456
celo_rating: 8388.89
family_friendly_score: 0.0
family_friendly_standard_error: 0.0
submission_type: basic
model_repo: ChaiML/grpo-glm50-combine-rm3331-mega-kimi-mix8020-step100-int4
model_architecture: GlmMoeDsaForCausalLM
model_num_parameters: 31348144128.0
best_of: 8
max_input_tokens: 1500
max_output_tokens: 80
reward_model: default
display_name: chaiml-grpo-glm50-combi_14294_v1
ineligible_reason: max_output_tokens!=64
is_internal_developer: True
language_model: ChaiML/grpo-glm50-combine-rm3331-mega-kimi-mix8020-step100-int4
model_size: 31B
ranking_group: single
us_pacific_date: 2026-04-02
win_ratio: 0.5813597478613237
generation_params: {'temperature': 1.0, 'top_p': 0.95, 'min_p': 0.0, 'top_k': 60, 'presence_penalty': 0.0, 'frequency_penalty': 0.0, 'stopping_words': ['<|user|>', '<|assistant|>', '</s>', '<|im_end|>', '####'], 'max_input_tokens': 1500, 'best_of': 8, 'max_output_tokens': 80}
formatter: {'memory_template': "[gMASK]<sop><|system|>{bot_name}'s persona: {memory}", 'prompt_template': '', 'bot_template': '<|assistant|>{message}', 'user_template': '<|user|>{message}', 'response_template': '<|assistant|></think>', 'truncate_by_message': True}
Resubmit model
Shutdown handler not registered because Python interpreter is not running in the main thread
run pipeline %s
run pipeline stage %s
Running pipeline stage VLLMUploader
Starting job with name chaiml-grpo-glm50-combi-14294-v1-uploader
Waiting for job on chaiml-grpo-glm50-combi-14294-v1-uploader to finish
chaiml-grpo-glm50-combi-14294-v1-uploader: {repo_id} is already quantized
chaiml-grpo-glm50-combi-14294-v1-uploader: Using quantization_mode: none
chaiml-grpo-glm50-combi-14294-v1-uploader: Downloading snapshot of ChaiML/grpo-glm50-combine-rm3331-mega-kimi-mix8020-step100-int4...
2026-04-02T17:01:36.318666+00:00 monitor updated for chaiml-grpo-glm50-combi_14294_v1
2026-04-02T17:02:36.405640+00:00 monitor updated for chaiml-grpo-glm50-combi_14294_v1
2026-04-02T17:03:36.515019+00:00 monitor updated for chaiml-grpo-glm50-combi_14294_v1
chaiml-grpo-glm50-combi-14294-v1-uploader: Downloaded in 194.137s
2026-04-02T17:04:36.603183+00:00 monitor updated for chaiml-grpo-glm50-combi_14294_v1
2026-04-02T17:05:36.691126+00:00 monitor updated for chaiml-grpo-glm50-combi_14294_v1
Retrying (%r) after connection broken by '%r': %s
2026-04-02T17:06:36.782263+00:00 monitor updated for chaiml-grpo-glm50-combi_14294_v1
chaiml-grpo-glm50-combi-14294-v1-uploader: Processed model ChaiML/grpo-glm50-combine-rm3331-mega-kimi-mix8020-step100-int4 in 333.236s
chaiml-grpo-glm50-combi-14294-v1-uploader: creating bucket guanaco-vllm-models
chaiml-grpo-glm50-combi-14294-v1-uploader: /usr/lib/python3/dist-packages/S3/BaseUtils.py:56: SyntaxWarning: invalid escape sequence '\.'
chaiml-grpo-glm50-combi-14294-v1-uploader:   RE_S3_DATESTRING = re.compile('\.[0-9]*(?:[Z\\-\\+]*?)')
chaiml-grpo-glm50-combi-14294-v1-uploader: /usr/lib/python3/dist-packages/S3/BaseUtils.py:57: SyntaxWarning: invalid escape sequence '\s'
chaiml-grpo-glm50-combi-14294-v1-uploader:   RE_XML_NAMESPACE = re.compile(b'^(<?[^>]+?>\s*|\s*)(<\w+) xmlns=[\'"](https?://[^\'"]+)[\'"]', re.MULTILINE)
chaiml-grpo-glm50-combi-14294-v1-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:240: SyntaxWarning: invalid escape sequence '\.'
chaiml-grpo-glm50-combi-14294-v1-uploader:   invalid = re.search("([^a-z0-9\.-])", bucket, re.UNICODE)
chaiml-grpo-glm50-combi-14294-v1-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:244: SyntaxWarning: invalid escape sequence '\.'
chaiml-grpo-glm50-combi-14294-v1-uploader:   invalid = re.search("([^A-Za-z0-9\._-])", bucket, re.UNICODE)
chaiml-grpo-glm50-combi-14294-v1-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:255: SyntaxWarning: invalid escape sequence '\.'
chaiml-grpo-glm50-combi-14294-v1-uploader:   if re.search("-\.", bucket, re.UNICODE):
chaiml-grpo-glm50-combi-14294-v1-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:257: SyntaxWarning: invalid escape sequence '\.'
chaiml-grpo-glm50-combi-14294-v1-uploader:   if re.search("\.\.", bucket, re.UNICODE):
chaiml-grpo-glm50-combi-14294-v1-uploader: /usr/lib/python3/dist-packages/S3/S3Uri.py:155: SyntaxWarning: invalid escape sequence '\w'
chaiml-grpo-glm50-combi-14294-v1-uploader:   _re = re.compile("^(\w+://)?(.*)", re.UNICODE)
chaiml-grpo-glm50-combi-14294-v1-uploader: /usr/lib/python3/dist-packages/S3/FileLists.py:480: SyntaxWarning: invalid escape sequence '\*'
chaiml-grpo-glm50-combi-14294-v1-uploader:   wildcard_split_result = re.split("\*|\?", uri_str, maxsplit=1)
chaiml-grpo-glm50-combi-14294-v1-uploader: Bucket 's3://guanaco-vllm-models/' created
chaiml-grpo-glm50-combi-14294-v1-uploader: uploading /dev/shm/model_output to s3://guanaco-vllm-models/chaiml-grpo-glm50-combi-14294-v1/default
chaiml-grpo-glm50-combi-14294-v1-uploader: cp /dev/shm/model_output/.gitattributes s3://guanaco-vllm-models/chaiml-grpo-glm50-combi-14294-v1/default/.gitattributes
chaiml-grpo-glm50-combi-14294-v1-uploader: cp /dev/shm/model_output/config.json s3://guanaco-vllm-models/chaiml-grpo-glm50-combi-14294-v1/default/config.json
chaiml-grpo-glm50-combi-14294-v1-uploader: cp /dev/shm/model_output/chat_template.jinja s3://guanaco-vllm-models/chaiml-grpo-glm50-combi-14294-v1/default/chat_template.jinja
chaiml-grpo-glm50-combi-14294-v1-uploader: cp /dev/shm/model_output/generation_config.json s3://guanaco-vllm-models/chaiml-grpo-glm50-combi-14294-v1/default/generation_config.json
chaiml-grpo-glm50-combi-14294-v1-uploader: cp /dev/shm/model_output/tokenizer_config.json s3://guanaco-vllm-models/chaiml-grpo-glm50-combi-14294-v1/default/tokenizer_config.json
chaiml-grpo-glm50-combi-14294-v1-uploader: cp /dev/shm/model_output/quantization_config.json s3://guanaco-vllm-models/chaiml-grpo-glm50-combi-14294-v1/default/quantization_config.json
chaiml-grpo-glm50-combi-14294-v1-uploader: cp /dev/shm/model_output/model.safetensors.index.json s3://guanaco-vllm-models/chaiml-grpo-glm50-combi-14294-v1/default/model.safetensors.index.json
chaiml-grpo-glm50-combi-14294-v1-uploader: cp /dev/shm/model_output/tokenizer.json s3://guanaco-vllm-models/chaiml-grpo-glm50-combi-14294-v1/default/tokenizer.json
chaiml-grpo-glm50-combi-14294-v1-uploader: cp /dev/shm/model_output/model-00075-of-00075.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm50-combi-14294-v1/default/model-00075-of-00075.safetensors
2026-04-02T17:07:36.891709+00:00 monitor updated for chaiml-grpo-glm50-combi_14294_v1
chaiml-grpo-glm50-combi-14294-v1-uploader: cp /dev/shm/model_output/model-00001-of-00075.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm50-combi-14294-v1/default/model-00001-of-00075.safetensors
chaiml-grpo-glm50-combi-14294-v1-uploader: cp /dev/shm/model_output/model-00030-of-00075.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm50-combi-14294-v1/default/model-00030-of-00075.safetensors
chaiml-grpo-glm50-combi-14294-v1-uploader: cp /dev/shm/model_output/model-00016-of-00075.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm50-combi-14294-v1/default/model-00016-of-00075.safetensors
chaiml-grpo-glm50-combi-14294-v1-uploader: cp /dev/shm/model_output/model-00032-of-00075.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm50-combi-14294-v1/default/model-00032-of-00075.safetensors
chaiml-grpo-glm50-combi-14294-v1-uploader: cp /dev/shm/model_output/model-00008-of-00075.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm50-combi-14294-v1/default/model-00008-of-00075.safetensors
chaiml-grpo-glm50-combi-14294-v1-uploader: cp /dev/shm/model_output/model-00006-of-00075.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm50-combi-14294-v1/default/model-00006-of-00075.safetensors
chaiml-grpo-glm50-combi-14294-v1-uploader: cp /dev/shm/model_output/model-00004-of-00075.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm50-combi-14294-v1/default/model-00004-of-00075.safetensors
chaiml-grpo-glm50-combi-14294-v1-uploader: cp /dev/shm/model_output/model-00054-of-00075.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm50-combi-14294-v1/default/model-00054-of-00075.safetensors
chaiml-grpo-glm50-combi-14294-v1-uploader: cp /dev/shm/model_output/model-00003-of-00075.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm50-combi-14294-v1/default/model-00003-of-00075.safetensors
chaiml-grpo-glm50-combi-14294-v1-uploader: cp /dev/shm/model_output/model-00047-of-00075.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm50-combi-14294-v1/default/model-00047-of-00075.safetensors
chaiml-grpo-glm50-combi-14294-v1-uploader: cp /dev/shm/model_output/model-00036-of-00075.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm50-combi-14294-v1/default/model-00036-of-00075.safetensors
chaiml-grpo-glm50-combi-14294-v1-uploader: cp /dev/shm/model_output/model-00044-of-00075.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm50-combi-14294-v1/default/model-00044-of-00075.safetensors
chaiml-grpo-glm50-combi-14294-v1-uploader: cp /dev/shm/model_output/model-00035-of-00075.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm50-combi-14294-v1/default/model-00035-of-00075.safetensors
chaiml-grpo-glm50-combi-14294-v1-uploader: cp /dev/shm/model_output/model-00034-of-00075.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm50-combi-14294-v1/default/model-00034-of-00075.safetensors
chaiml-grpo-glm50-combi-14294-v1-uploader: cp /dev/shm/model_output/model-00074-of-00075.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm50-combi-14294-v1/default/model-00074-of-00075.safetensors
chaiml-grpo-glm50-combi-14294-v1-uploader: cp /dev/shm/model_output/model-00024-of-00075.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm50-combi-14294-v1/default/model-00024-of-00075.safetensors
chaiml-grpo-glm50-combi-14294-v1-uploader: cp /dev/shm/model_output/model-00026-of-00075.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm50-combi-14294-v1/default/model-00026-of-00075.safetensors
chaiml-grpo-glm50-combi-14294-v1-uploader: cp /dev/shm/model_output/model-00017-of-00075.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm50-combi-14294-v1/default/model-00017-of-00075.safetensors
chaiml-grpo-glm50-combi-14294-v1-uploader: cp /dev/shm/model_output/model-00045-of-00075.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm50-combi-14294-v1/default/model-00045-of-00075.safetensors
chaiml-grpo-glm50-combi-14294-v1-uploader: cp /dev/shm/model_output/model-00011-of-00075.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm50-combi-14294-v1/default/model-00011-of-00075.safetensors
chaiml-grpo-glm50-combi-14294-v1-uploader: cp /dev/shm/model_output/model-00046-of-00075.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm50-combi-14294-v1/default/model-00046-of-00075.safetensors
chaiml-grpo-glm50-combi-14294-v1-uploader: cp /dev/shm/model_output/model-00010-of-00075.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm50-combi-14294-v1/default/model-00010-of-00075.safetensors
chaiml-grpo-glm50-combi-14294-v1-uploader: cp /dev/shm/model_output/model-00038-of-00075.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm50-combi-14294-v1/default/model-00038-of-00075.safetensors
chaiml-grpo-glm50-combi-14294-v1-uploader: cp /dev/shm/model_output/model-00043-of-00075.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm50-combi-14294-v1/default/model-00043-of-00075.safetensors
chaiml-grpo-glm50-combi-14294-v1-uploader: cp /dev/shm/model_output/model-00039-of-00075.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm50-combi-14294-v1/default/model-00039-of-00075.safetensors
chaiml-grpo-glm50-combi-14294-v1-uploader: cp /dev/shm/model_output/model-00028-of-00075.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm50-combi-14294-v1/default/model-00028-of-00075.safetensors
chaiml-grpo-glm50-combi-14294-v1-uploader: cp /dev/shm/model_output/model-00064-of-00075.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm50-combi-14294-v1/default/model-00064-of-00075.safetensors
chaiml-grpo-glm50-combi-14294-v1-uploader: cp /dev/shm/model_output/model-00009-of-00075.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm50-combi-14294-v1/default/model-00009-of-00075.safetensors
chaiml-grpo-glm50-combi-14294-v1-uploader: cp /dev/shm/model_output/model-00048-of-00075.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm50-combi-14294-v1/default/model-00048-of-00075.safetensors
chaiml-grpo-glm50-combi-14294-v1-uploader: cp /dev/shm/model_output/model-00020-of-00075.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm50-combi-14294-v1/default/model-00020-of-00075.safetensors
chaiml-grpo-glm50-combi-14294-v1-uploader: cp /dev/shm/model_output/model-00069-of-00075.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm50-combi-14294-v1/default/model-00069-of-00075.safetensors
chaiml-grpo-glm50-combi-14294-v1-uploader: cp /dev/shm/model_output/model-00055-of-00075.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm50-combi-14294-v1/default/model-00055-of-00075.safetensors
chaiml-grpo-glm50-combi-14294-v1-uploader: cp /dev/shm/model_output/model-00005-of-00075.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm50-combi-14294-v1/default/model-00005-of-00075.safetensors
chaiml-grpo-glm50-combi-14294-v1-uploader: cp /dev/shm/model_output/model-00018-of-00075.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm50-combi-14294-v1/default/model-00018-of-00075.safetensors
chaiml-grpo-glm50-combi-14294-v1-uploader: cp /dev/shm/model_output/model-00013-of-00075.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm50-combi-14294-v1/default/model-00013-of-00075.safetensors
chaiml-grpo-glm50-combi-14294-v1-uploader: cp /dev/shm/model_output/model-00068-of-00075.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm50-combi-14294-v1/default/model-00068-of-00075.safetensors
chaiml-grpo-glm50-combi-14294-v1-uploader: cp /dev/shm/model_output/model-00041-of-00075.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm50-combi-14294-v1/default/model-00041-of-00075.safetensors
chaiml-grpo-glm50-combi-14294-v1-uploader: cp /dev/shm/model_output/model-00056-of-00075.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm50-combi-14294-v1/default/model-00056-of-00075.safetensors
chaiml-grpo-glm50-combi-14294-v1-uploader: cp /dev/shm/model_output/model-00065-of-00075.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm50-combi-14294-v1/default/model-00065-of-00075.safetensors
chaiml-grpo-glm50-combi-14294-v1-uploader: cp /dev/shm/model_output/model-00052-of-00075.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm50-combi-14294-v1/default/model-00052-of-00075.safetensors
chaiml-grpo-glm50-combi-14294-v1-uploader: cp /dev/shm/model_output/model-00066-of-00075.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm50-combi-14294-v1/default/model-00066-of-00075.safetensors
chaiml-grpo-glm50-combi-14294-v1-uploader: cp /dev/shm/model_output/model-00050-of-00075.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm50-combi-14294-v1/default/model-00050-of-00075.safetensors
chaiml-grpo-glm50-combi-14294-v1-uploader: cp /dev/shm/model_output/model-00031-of-00075.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm50-combi-14294-v1/default/model-00031-of-00075.safetensors
chaiml-grpo-glm50-combi-14294-v1-uploader: cp /dev/shm/model_output/model-00042-of-00075.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm50-combi-14294-v1/default/model-00042-of-00075.safetensors
chaiml-grpo-glm50-combi-14294-v1-uploader: cp /dev/shm/model_output/model-00072-of-00075.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm50-combi-14294-v1/default/model-00072-of-00075.safetensors
chaiml-grpo-glm50-combi-14294-v1-uploader: cp /dev/shm/model_output/model-00051-of-00075.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm50-combi-14294-v1/default/model-00051-of-00075.safetensors
chaiml-grpo-glm50-combi-14294-v1-uploader: cp /dev/shm/model_output/model-00063-of-00075.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm50-combi-14294-v1/default/model-00063-of-00075.safetensors
chaiml-grpo-glm50-combi-14294-v1-uploader: cp /dev/shm/model_output/model-00029-of-00075.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm50-combi-14294-v1/default/model-00029-of-00075.safetensors
chaiml-grpo-glm50-combi-14294-v1-uploader: cp /dev/shm/model_output/model-00027-of-00075.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm50-combi-14294-v1/default/model-00027-of-00075.safetensors
chaiml-grpo-glm50-combi-14294-v1-uploader: cp /dev/shm/model_output/model-00023-of-00075.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm50-combi-14294-v1/default/model-00023-of-00075.safetensors
chaiml-grpo-glm50-combi-14294-v1-uploader: cp /dev/shm/model_output/model-00049-of-00075.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm50-combi-14294-v1/default/model-00049-of-00075.safetensors
chaiml-grpo-glm50-combi-14294-v1-uploader: cp /dev/shm/model_output/model-00021-of-00075.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm50-combi-14294-v1/default/model-00021-of-00075.safetensors
chaiml-grpo-glm50-combi-14294-v1-uploader: cp /dev/shm/model_output/model-00057-of-00075.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm50-combi-14294-v1/default/model-00057-of-00075.safetensors
chaiml-grpo-glm50-combi-14294-v1-uploader: cp /dev/shm/model_output/model-00037-of-00075.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm50-combi-14294-v1/default/model-00037-of-00075.safetensors
chaiml-grpo-glm50-combi-14294-v1-uploader: cp /dev/shm/model_output/model-00019-of-00075.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm50-combi-14294-v1/default/model-00019-of-00075.safetensors
chaiml-grpo-glm50-combi-14294-v1-uploader: cp /dev/shm/model_output/model-00025-of-00075.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm50-combi-14294-v1/default/model-00025-of-00075.safetensors
chaiml-grpo-glm50-combi-14294-v1-uploader: cp /dev/shm/model_output/model-00040-of-00075.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm50-combi-14294-v1/default/model-00040-of-00075.safetensors
chaiml-grpo-glm50-combi-14294-v1-uploader: cp /dev/shm/model_output/model-00022-of-00075.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm50-combi-14294-v1/default/model-00022-of-00075.safetensors
chaiml-grpo-glm50-combi-14294-v1-uploader: cp /dev/shm/model_output/model-00071-of-00075.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm50-combi-14294-v1/default/model-00071-of-00075.safetensors
chaiml-grpo-glm50-combi-14294-v1-uploader: cp /dev/shm/model_output/model-00059-of-00075.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm50-combi-14294-v1/default/model-00059-of-00075.safetensors
chaiml-grpo-glm50-combi-14294-v1-uploader: cp /dev/shm/model_output/model-00058-of-00075.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm50-combi-14294-v1/default/model-00058-of-00075.safetensors
chaiml-grpo-glm50-combi-14294-v1-uploader: cp /dev/shm/model_output/model-00061-of-00075.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm50-combi-14294-v1/default/model-00061-of-00075.safetensors
chaiml-grpo-glm50-combi-14294-v1-uploader: cp /dev/shm/model_output/model-00002-of-00075.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm50-combi-14294-v1/default/model-00002-of-00075.safetensors
chaiml-grpo-glm50-combi-14294-v1-uploader: cp /dev/shm/model_output/model-00067-of-00075.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm50-combi-14294-v1/default/model-00067-of-00075.safetensors
chaiml-grpo-glm50-combi-14294-v1-uploader: cp /dev/shm/model_output/model-00033-of-00075.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm50-combi-14294-v1/default/model-00033-of-00075.safetensors
chaiml-grpo-glm50-combi-14294-v1-uploader: cp /dev/shm/model_output/model-00014-of-00075.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm50-combi-14294-v1/default/model-00014-of-00075.safetensors
chaiml-grpo-glm50-combi-14294-v1-uploader: cp /dev/shm/model_output/model-00007-of-00075.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm50-combi-14294-v1/default/model-00007-of-00075.safetensors
chaiml-grpo-glm50-combi-14294-v1-uploader: cp /dev/shm/model_output/model-00015-of-00075.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm50-combi-14294-v1/default/model-00015-of-00075.safetensors
chaiml-grpo-glm50-combi-14294-v1-uploader: cp /dev/shm/model_output/model-00012-of-00075.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm50-combi-14294-v1/default/model-00012-of-00075.safetensors
chaiml-grpo-glm50-combi-14294-v1-uploader: cp /dev/shm/model_output/model-00062-of-00075.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm50-combi-14294-v1/default/model-00062-of-00075.safetensors
chaiml-grpo-glm50-combi-14294-v1-uploader: cp /dev/shm/model_output/model-00060-of-00075.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm50-combi-14294-v1/default/model-00060-of-00075.safetensors
chaiml-grpo-glm50-combi-14294-v1-uploader: cp /dev/shm/model_output/model-00053-of-00075.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm50-combi-14294-v1/default/model-00053-of-00075.safetensors
chaiml-grpo-glm50-combi-14294-v1-uploader: cp /dev/shm/model_output/model-00070-of-00075.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm50-combi-14294-v1/default/model-00070-of-00075.safetensors
chaiml-grpo-glm50-combi-14294-v1-uploader: cp /dev/shm/model_output/model-00073-of-00075.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm50-combi-14294-v1/default/model-00073-of-00075.safetensors
Job chaiml-grpo-glm50-combi-14294-v1-uploader completed after 476.23s with status: succeeded
Stopping job with name chaiml-grpo-glm50-combi-14294-v1-uploader
Pipeline stage VLLMUploader completed in 476.74s
run pipeline stage %s
Running pipeline stage VLLMUploaderAMD
Pipeline stage vllm_upload_amd skipped, reason=not amd cluster
Pipeline stage VLLMUploaderAMD completed in 0.13s
run pipeline stage %s
Running pipeline stage VLLMTemplater
2026-04-02T17:08:37.008559+00:00 monitor updated for chaiml-grpo-glm50-combi_14294_v1
Pipeline stage VLLMTemplater completed in 3.58s
run pipeline stage %s
Running pipeline stage VLLMDeployer
Creating inference service chaiml-grpo-glm50-combi-14294-v1
Waiting for inference service chaiml-grpo-glm50-combi-14294-v1 to be ready
2026-04-02T17:09:37.118523+00:00 monitor updated for chaiml-grpo-glm50-combi_14294_v1
2026-04-02T17:10:37.308207+00:00 monitor updated for chaiml-grpo-glm50-combi_14294_v1
2026-04-02T17:11:37.837480+00:00 monitor updated for chaiml-grpo-glm50-combi_14294_v1
2026-04-02T17:12:38.007609+00:00 monitor updated for chaiml-grpo-glm50-combi_14294_v1
2026-04-02T17:13:38.273184+00:00 monitor updated for chaiml-grpo-glm50-combi_14294_v1
2026-04-02T17:14:38.431038+00:00 monitor updated for chaiml-grpo-glm50-combi_14294_v1
2026-04-02T17:15:38.558152+00:00 monitor updated for chaiml-grpo-glm50-combi_14294_v1
2026-04-02T17:16:38.668668+00:00 monitor updated for chaiml-grpo-glm50-combi_14294_v1
2026-04-02T17:17:38.822310+00:00 monitor updated for chaiml-grpo-glm50-combi_14294_v1
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
2026-04-02T17:18:38.940004+00:00 monitor updated for chaiml-grpo-glm50-combi_14294_v1
2026-04-02T17:19:39.040100+00:00 monitor updated for chaiml-grpo-glm50-combi_14294_v1
Inference service chaiml-grpo-glm50-combi-14294-v1 ready after 681.8954572677612s
Pipeline stage VLLMDeployer completed in 682.40s
run pipeline stage %s
Running pipeline stage StressChecker
Received healthy response to inference request in 11.535403728485107s
Received healthy response to inference request in 2.6990854740142822s
Received healthy response to inference request in 3.2585768699645996s
Received healthy response to inference request in 3.0336644649505615s
Received healthy response to inference request in 2.8966634273529053s
Received healthy response to inference request in 2.7479543685913086s
Received healthy response to inference request in 2.7412707805633545s
2026-04-02T17:20:39.143301+00:00 monitor updated for chaiml-grpo-glm50-combi_14294_v1
Received healthy response to inference request in 2.855651378631592s
Received healthy response to inference request in 2.592733383178711s
Received healthy response to inference request in 3.038543224334717s
Received healthy response to inference request in 2.7072179317474365s
Received healthy response to inference request in 2.859074115753174s
Received healthy response to inference request in 3.321849822998047s
Received healthy response to inference request in 3.040285110473633s
Received healthy response to inference request in 2.9277915954589844s
Received healthy response to inference request in 2.8244681358337402s
Received healthy response to inference request in 3.09759783744812s
Received healthy response to inference request in 2.7578465938568115s
Received healthy response to inference request in 2.685659170150757s
Received healthy response to inference request in 3.3402645587921143s
Received healthy response to inference request in 2.7516636848449707s
Received healthy response to inference request in 3.019430160522461s
Received healthy response to inference request in 3.0243585109710693s
Received healthy response to inference request in 3.008145570755005s
Received healthy response to inference request in 2.7928690910339355s
Received healthy response to inference request in 2.807771921157837s
Received healthy response to inference request in 2.7689621448516846s
2026-04-02T17:21:39.260033+00:00 monitor updated for chaiml-grpo-glm50-combi_14294_v1
Received healthy response to inference request in 3.2113640308380127s
Received healthy response to inference request in 2.8427484035491943s
Received healthy response to inference request in 2.884941816329956s
30 requests
0 failed requests
5th percentile: 2.6917010068893434
10th percentile: 2.706404685974121
20th percentile: 2.7509218215942384
30th percentile: 2.78569700717926
40th percentile: 2.835436296463013
50th percentile: 2.872007966041565
60th percentile: 2.9599331855773925
70th percentile: 3.027150297164917
80th percentile: 3.0517476558685304
90th percentile: 3.2649041652679442
95th percentile: 3.331977927684784
99th percentile: 9.158813369274146
mean time: 3.202461910247803
Pipeline stage StressChecker completed in 108.72s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyTriggerPipeline
run_pipeline:run_in_cloud %s
starting trigger_guanaco_pipeline args=%s
triggered trigger_guanaco_pipeline args=%s
Pipeline stage OfflineFamilyFriendlyTriggerPipeline completed in 1.78s
Shutdown handler de-registered
chaiml-grpo-glm50-combi_14294_v1 status is now deployed due to DeploymentManager action
chaiml-grpo-glm50-combi_14294_v1 status is now inactive due to auto deactivation removed underperforming models