chaiml-gspo-glm47-combi_60653

developer_uid: richhx
submission_id: chaiml-gspo-glm47-combi_60653_v1
model_name: chaiml-gspo-glm47-combi_60653_v1
model_group: ChaiML/gspo-glm47-combin
status: deployed
timestamp: 2026-03-27T19:02:42+00:00
num_battles: 4045
num_wins: 2361
celo_rating: 1356.3
family_friendly_score: 0.0
family_friendly_standard_error: 0.0
submission_type: basic
model_repo: ChaiML/gspo-glm47-combine-rm82-kimi-data-step500-W4A16
model_architecture: Glm4MoeForCausalLM
model_num_parameters: 24110003200.0
best_of: 8
max_input_tokens: 1500
max_output_tokens: 80
reward_model: default
display_name: chaiml-gspo-glm47-combi_60653_v1
ineligible_reason: max_output_tokens!=64
is_internal_developer: True
language_model: ChaiML/gspo-glm47-combine-rm82-kimi-data-step500-W4A16
model_size: 24B
ranking_group: single
us_pacific_date: 2026-03-27
win_ratio: 0.5836835599505562
generation_params: {'temperature': 1.0, 'top_p': 0.95, 'min_p': 0.0, 'top_k': 60, 'presence_penalty': 0.0, 'frequency_penalty': 0.0, 'stopping_words': ['</s>', '<|user|>', '<|im_end|>', '<|assistant|>', '####'], 'max_input_tokens': 1500, 'best_of': 8, 'max_output_tokens': 80}
formatter: {'memory_template': "[gMASK]<sop><|system|>\n{bot_name}'s persona: {memory}", 'prompt_template': '', 'bot_template': '<|assistant|>\n{bot_name}: {message}', 'user_template': '<|user|>\n{message}', 'response_template': '<|assistant|>\n<think></think>\n{bot_name}:', 'truncate_by_message': True}
Resubmit model
Shutdown handler not registered because Python interpreter is not running in the main thread
run pipeline %s
run pipeline stage %s
Running pipeline stage VLLMUploader
Starting job with name chaiml-gspo-glm47-combi-60653-v1-uploader
Waiting for job on chaiml-gspo-glm47-combi-60653-v1-uploader to finish
Shutdown handler not registered because Python interpreter is not running in the main thread
run pipeline %s
run pipeline stage %s
Running pipeline stage VLLMUploader
Starting job with name chaiml-gspo-glm47-combi-10268-v1-uploader
Waiting for job on chaiml-gspo-glm47-combi-10268-v1-uploader to finish
chaiml-gspo-glm47-combi-60653-v1-uploader: {repo_id} is already quantized
chaiml-gspo-glm47-combi-60653-v1-uploader: Using quantization_mode: none
chaiml-gspo-glm47-combi-60653-v1-uploader: Downloading snapshot of ChaiML/gspo-glm47-combine-rm82-kimi-data-step500-W4A16...
chaiml-gspo-glm47-combi-10268-v1-uploader: {repo_id} is already quantized
chaiml-gspo-glm47-combi-10268-v1-uploader: Using quantization_mode: none
chaiml-gspo-glm47-combi-10268-v1-uploader: Downloading snapshot of ChaiML/gspo-glm47-combine-rm82-kimi-data-step500-FP8...
2026-03-27T18:52:44.511604+00:00 monitor updated for chaiml-gspo-glm47-combi_60653_v1
2026-03-27T18:52:46.374144+00:00 monitor updated for chaiml-gspo-glm47-combi_10268_v1
2026-03-27T18:53:44.805888+00:00 monitor updated for chaiml-gspo-glm47-combi_60653_v1
2026-03-27T18:53:46.687859+00:00 monitor updated for chaiml-gspo-glm47-combi_10268_v1
chaiml-gspo-glm47-combi-60653-v1-uploader: Downloaded in 82.318s
2026-03-27T18:54:45.174748+00:00 monitor updated for chaiml-gspo-glm47-combi_60653_v1
2026-03-27T18:54:47.015358+00:00 monitor updated for chaiml-gspo-glm47-combi_10268_v1
chaiml-gspo-glm47-combi-60653-v1-uploader: Processed model ChaiML/gspo-glm47-combine-rm82-kimi-data-step500-W4A16 in 153.856s
chaiml-gspo-glm47-combi-60653-v1-uploader: creating bucket guanaco-vllm-models
chaiml-gspo-glm47-combi-60653-v1-uploader: /usr/lib/python3/dist-packages/S3/BaseUtils.py:56: SyntaxWarning: invalid escape sequence '\.'
chaiml-gspo-glm47-combi-60653-v1-uploader:   RE_S3_DATESTRING = re.compile('\.[0-9]*(?:[Z\\-\\+]*?)')
chaiml-gspo-glm47-combi-60653-v1-uploader: /usr/lib/python3/dist-packages/S3/BaseUtils.py:57: SyntaxWarning: invalid escape sequence '\s'
chaiml-gspo-glm47-combi-60653-v1-uploader:   RE_XML_NAMESPACE = re.compile(b'^(<?[^>]+?>\s*|\s*)(<\w+) xmlns=[\'"](https?://[^\'"]+)[\'"]', re.MULTILINE)
chaiml-gspo-glm47-combi-60653-v1-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:240: SyntaxWarning: invalid escape sequence '\.'
chaiml-gspo-glm47-combi-60653-v1-uploader:   invalid = re.search("([^a-z0-9\.-])", bucket, re.UNICODE)
chaiml-gspo-glm47-combi-60653-v1-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:244: SyntaxWarning: invalid escape sequence '\.'
chaiml-gspo-glm47-combi-60653-v1-uploader:   invalid = re.search("([^A-Za-z0-9\._-])", bucket, re.UNICODE)
chaiml-gspo-glm47-combi-60653-v1-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:255: SyntaxWarning: invalid escape sequence '\.'
chaiml-gspo-glm47-combi-60653-v1-uploader:   if re.search("-\.", bucket, re.UNICODE):
chaiml-gspo-glm47-combi-60653-v1-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:257: SyntaxWarning: invalid escape sequence '\.'
chaiml-gspo-glm47-combi-60653-v1-uploader:   if re.search("\.\.", bucket, re.UNICODE):
chaiml-gspo-glm47-combi-10268-v1-uploader: Downloaded in 155.725s
chaiml-gspo-glm47-combi-60653-v1-uploader: /usr/lib/python3/dist-packages/S3/S3Uri.py:155: SyntaxWarning: invalid escape sequence '\w'
chaiml-gspo-glm47-combi-60653-v1-uploader:   _re = re.compile("^(\w+://)?(.*)", re.UNICODE)
chaiml-gspo-glm47-combi-60653-v1-uploader: /usr/lib/python3/dist-packages/S3/FileLists.py:480: SyntaxWarning: invalid escape sequence '\*'
chaiml-gspo-glm47-combi-60653-v1-uploader:   wildcard_split_result = re.split("\*|\?", uri_str, maxsplit=1)
chaiml-gspo-glm47-combi-60653-v1-uploader: Bucket 's3://guanaco-vllm-models/' created
chaiml-gspo-glm47-combi-60653-v1-uploader: uploading /dev/shm/model_output to s3://guanaco-vllm-models/chaiml-gspo-glm47-combi-60653-v1/default
chaiml-gspo-glm47-combi-60653-v1-uploader: cp /dev/shm/model_output/chat_template.jinja s3://guanaco-vllm-models/chaiml-gspo-glm47-combi-60653-v1/default/chat_template.jinja
chaiml-gspo-glm47-combi-60653-v1-uploader: cp /dev/shm/model_output/.gitattributes s3://guanaco-vllm-models/chaiml-gspo-glm47-combi-60653-v1/default/.gitattributes
chaiml-gspo-glm47-combi-60653-v1-uploader: cp /dev/shm/model_output/generation_config.json s3://guanaco-vllm-models/chaiml-gspo-glm47-combi-60653-v1/default/generation_config.json
chaiml-gspo-glm47-combi-60653-v1-uploader: cp /dev/shm/model_output/config.json s3://guanaco-vllm-models/chaiml-gspo-glm47-combi-60653-v1/default/config.json
chaiml-gspo-glm47-combi-60653-v1-uploader: cp /dev/shm/model_output/model.safetensors.index.json s3://guanaco-vllm-models/chaiml-gspo-glm47-combi-60653-v1/default/model.safetensors.index.json
chaiml-gspo-glm47-combi-60653-v1-uploader: cp /dev/shm/model_output/tokenizer_config.json s3://guanaco-vllm-models/chaiml-gspo-glm47-combi-60653-v1/default/tokenizer_config.json
chaiml-gspo-glm47-combi-60653-v1-uploader: cp /dev/shm/model_output/quantization_config.json s3://guanaco-vllm-models/chaiml-gspo-glm47-combi-60653-v1/default/quantization_config.json
chaiml-gspo-glm47-combi-60653-v1-uploader: cp /dev/shm/model_output/tokenizer.json s3://guanaco-vllm-models/chaiml-gspo-glm47-combi-60653-v1/default/tokenizer.json
chaiml-gspo-glm47-combi-60653-v1-uploader: cp /dev/shm/model_output/model-00038-of-00038.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm47-combi-60653-v1/default/model-00038-of-00038.safetensors
chaiml-gspo-glm47-combi-60653-v1-uploader: cp /dev/shm/model_output/model-00037-of-00038.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm47-combi-60653-v1/default/model-00037-of-00038.safetensors
chaiml-gspo-glm47-combi-60653-v1-uploader: cp /dev/shm/model_output/model-00036-of-00038.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm47-combi-60653-v1/default/model-00036-of-00038.safetensors
chaiml-gspo-glm47-combi-60653-v1-uploader: cp /dev/shm/model_output/model-00004-of-00038.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm47-combi-60653-v1/default/model-00004-of-00038.safetensors
chaiml-gspo-glm47-combi-60653-v1-uploader: cp /dev/shm/model_output/model-00001-of-00038.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm47-combi-60653-v1/default/model-00001-of-00038.safetensors
chaiml-gspo-glm47-combi-60653-v1-uploader: cp /dev/shm/model_output/model-00012-of-00038.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm47-combi-60653-v1/default/model-00012-of-00038.safetensors
chaiml-gspo-glm47-combi-60653-v1-uploader: cp /dev/shm/model_output/model-00020-of-00038.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm47-combi-60653-v1/default/model-00020-of-00038.safetensors
chaiml-gspo-glm47-combi-60653-v1-uploader: cp /dev/shm/model_output/model-00008-of-00038.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm47-combi-60653-v1/default/model-00008-of-00038.safetensors
chaiml-gspo-glm47-combi-60653-v1-uploader: cp /dev/shm/model_output/model-00023-of-00038.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm47-combi-60653-v1/default/model-00023-of-00038.safetensors
chaiml-gspo-glm47-combi-60653-v1-uploader: cp /dev/shm/model_output/model-00015-of-00038.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm47-combi-60653-v1/default/model-00015-of-00038.safetensors
chaiml-gspo-glm47-combi-60653-v1-uploader: cp /dev/shm/model_output/model-00027-of-00038.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm47-combi-60653-v1/default/model-00027-of-00038.safetensors
chaiml-gspo-glm47-combi-60653-v1-uploader: cp /dev/shm/model_output/model-00002-of-00038.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm47-combi-60653-v1/default/model-00002-of-00038.safetensors
chaiml-gspo-glm47-combi-60653-v1-uploader: cp /dev/shm/model_output/model-00028-of-00038.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm47-combi-60653-v1/default/model-00028-of-00038.safetensors
chaiml-gspo-glm47-combi-60653-v1-uploader: cp /dev/shm/model_output/model-00024-of-00038.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm47-combi-60653-v1/default/model-00024-of-00038.safetensors
chaiml-gspo-glm47-combi-60653-v1-uploader: cp /dev/shm/model_output/model-00026-of-00038.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm47-combi-60653-v1/default/model-00026-of-00038.safetensors
chaiml-gspo-glm47-combi-60653-v1-uploader: cp /dev/shm/model_output/model-00019-of-00038.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm47-combi-60653-v1/default/model-00019-of-00038.safetensors
chaiml-gspo-glm47-combi-60653-v1-uploader: cp /dev/shm/model_output/model-00014-of-00038.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm47-combi-60653-v1/default/model-00014-of-00038.safetensors
chaiml-gspo-glm47-combi-60653-v1-uploader: cp /dev/shm/model_output/model-00032-of-00038.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm47-combi-60653-v1/default/model-00032-of-00038.safetensors
chaiml-gspo-glm47-combi-60653-v1-uploader: cp /dev/shm/model_output/model-00021-of-00038.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm47-combi-60653-v1/default/model-00021-of-00038.safetensors
chaiml-gspo-glm47-combi-60653-v1-uploader: cp /dev/shm/model_output/model-00035-of-00038.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm47-combi-60653-v1/default/model-00035-of-00038.safetensors
chaiml-gspo-glm47-combi-60653-v1-uploader: cp /dev/shm/model_output/model-00018-of-00038.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm47-combi-60653-v1/default/model-00018-of-00038.safetensors
chaiml-gspo-glm47-combi-60653-v1-uploader: cp /dev/shm/model_output/model-00031-of-00038.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm47-combi-60653-v1/default/model-00031-of-00038.safetensors
chaiml-gspo-glm47-combi-60653-v1-uploader: cp /dev/shm/model_output/model-00022-of-00038.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm47-combi-60653-v1/default/model-00022-of-00038.safetensors
chaiml-gspo-glm47-combi-60653-v1-uploader: cp /dev/shm/model_output/model-00003-of-00038.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm47-combi-60653-v1/default/model-00003-of-00038.safetensors
chaiml-gspo-glm47-combi-60653-v1-uploader: cp /dev/shm/model_output/model-00005-of-00038.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm47-combi-60653-v1/default/model-00005-of-00038.safetensors
chaiml-gspo-glm47-combi-60653-v1-uploader: cp /dev/shm/model_output/model-00016-of-00038.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm47-combi-60653-v1/default/model-00016-of-00038.safetensors
chaiml-gspo-glm47-combi-60653-v1-uploader: cp /dev/shm/model_output/model-00030-of-00038.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm47-combi-60653-v1/default/model-00030-of-00038.safetensors
2026-03-27T18:55:45.489755+00:00 monitor updated for chaiml-gspo-glm47-combi_60653_v1
chaiml-gspo-glm47-combi-60653-v1-uploader: cp /dev/shm/model_output/model-00013-of-00038.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm47-combi-60653-v1/default/model-00013-of-00038.safetensors
chaiml-gspo-glm47-combi-60653-v1-uploader: cp /dev/shm/model_output/model-00025-of-00038.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm47-combi-60653-v1/default/model-00025-of-00038.safetensors
chaiml-gspo-glm47-combi-60653-v1-uploader: cp /dev/shm/model_output/model-00009-of-00038.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm47-combi-60653-v1/default/model-00009-of-00038.safetensors
chaiml-gspo-glm47-combi-60653-v1-uploader: cp /dev/shm/model_output/model-00006-of-00038.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm47-combi-60653-v1/default/model-00006-of-00038.safetensors
chaiml-gspo-glm47-combi-60653-v1-uploader: cp /dev/shm/model_output/model-00029-of-00038.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm47-combi-60653-v1/default/model-00029-of-00038.safetensors
chaiml-gspo-glm47-combi-60653-v1-uploader: cp /dev/shm/model_output/model-00011-of-00038.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm47-combi-60653-v1/default/model-00011-of-00038.safetensors
chaiml-gspo-glm47-combi-60653-v1-uploader: cp /dev/shm/model_output/model-00007-of-00038.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm47-combi-60653-v1/default/model-00007-of-00038.safetensors
2026-03-27T18:55:47.346936+00:00 monitor updated for chaiml-gspo-glm47-combi_10268_v1
chaiml-gspo-glm47-combi-60653-v1-uploader: cp /dev/shm/model_output/model-00010-of-00038.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm47-combi-60653-v1/default/model-00010-of-00038.safetensors
chaiml-gspo-glm47-combi-60653-v1-uploader: cp /dev/shm/model_output/model-00017-of-00038.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm47-combi-60653-v1/default/model-00017-of-00038.safetensors
chaiml-gspo-glm47-combi-60653-v1-uploader: cp /dev/shm/model_output/model-00034-of-00038.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm47-combi-60653-v1/default/model-00034-of-00038.safetensors
chaiml-gspo-glm47-combi-60653-v1-uploader: cp /dev/shm/model_output/model-00033-of-00038.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm47-combi-60653-v1/default/model-00033-of-00038.safetensors
Job chaiml-gspo-glm47-combi-60653-v1-uploader completed after 253.63s with status: succeeded
Stopping job with name chaiml-gspo-glm47-combi-60653-v1-uploader
Pipeline stage VLLMUploader completed in 255.02s
run pipeline stage %s
Running pipeline stage VLLMUploaderAMD
Pipeline stage vllm_upload_amd skipped, reason=not amd cluster
Pipeline stage VLLMUploaderAMD completed in 0.46s
run pipeline stage %s
Running pipeline stage VLLMTemplater
Pipeline stage VLLMTemplater completed in 1.47s
run pipeline stage %s
Running pipeline stage VLLMDeployer
Creating inference service chaiml-gspo-glm47-combi-60653-v1
Waiting for inference service chaiml-gspo-glm47-combi-60653-v1 to be ready
2026-03-27T18:56:45.974626+00:00 monitor updated for chaiml-gspo-glm47-combi_60653_v1
2026-03-27T18:56:47.838202+00:00 monitor updated for chaiml-gspo-glm47-combi_10268_v1
2026-03-27T18:57:46.337963+00:00 monitor updated for chaiml-gspo-glm47-combi_60653_v1
2026-03-27T18:57:48.928509+00:00 monitor updated for chaiml-gspo-glm47-combi_10268_v1
2026-03-27T18:58:46.690159+00:00 monitor updated for chaiml-gspo-glm47-combi_60653_v1
2026-03-27T18:58:49.268844+00:00 monitor updated for chaiml-gspo-glm47-combi_10268_v1
chaiml-gspo-glm47-combi-10268-v1-uploader: Processed model ChaiML/gspo-glm47-combine-rm82-kimi-data-step500-FP8 in 406.940s
chaiml-gspo-glm47-combi-10268-v1-uploader: /usr/lib/python3/dist-packages/S3/S3Uri.py:155: SyntaxWarning: invalid escape sequence '\w'
chaiml-gspo-glm47-combi-10268-v1-uploader:   _re = re.compile("^(\w+://)?(.*)", re.UNICODE)
chaiml-gspo-glm47-combi-10268-v1-uploader: /usr/lib/python3/dist-packages/S3/FileLists.py:480: SyntaxWarning: invalid escape sequence '\*'
chaiml-gspo-glm47-combi-10268-v1-uploader:   wildcard_split_result = re.split("\*|\?", uri_str, maxsplit=1)
chaiml-gspo-glm47-combi-10268-v1-uploader: Bucket 's3://guanaco-vllm-models/' created
chaiml-gspo-glm47-combi-10268-v1-uploader: uploading /dev/shm/model_output to s3://guanaco-vllm-models/chaiml-gspo-glm47-combi-10268-v1/default
chaiml-gspo-glm47-combi-10268-v1-uploader: cp /dev/shm/model_output/chat_template.jinja s3://guanaco-vllm-models/chaiml-gspo-glm47-combi-10268-v1/default/chat_template.jinja
chaiml-gspo-glm47-combi-10268-v1-uploader: cp /dev/shm/model_output/config.json s3://guanaco-vllm-models/chaiml-gspo-glm47-combi-10268-v1/default/config.json
chaiml-gspo-glm47-combi-10268-v1-uploader: cp /dev/shm/model_output/.gitattributes s3://guanaco-vllm-models/chaiml-gspo-glm47-combi-10268-v1/default/.gitattributes
chaiml-gspo-glm47-combi-10268-v1-uploader: cp /dev/shm/model_output/generation_config.json s3://guanaco-vllm-models/chaiml-gspo-glm47-combi-10268-v1/default/generation_config.json
chaiml-gspo-glm47-combi-10268-v1-uploader: cp /dev/shm/model_output/recipe.yaml s3://guanaco-vllm-models/chaiml-gspo-glm47-combi-10268-v1/default/recipe.yaml
chaiml-gspo-glm47-combi-10268-v1-uploader: cp /dev/shm/model_output/tokenizer_config.json s3://guanaco-vllm-models/chaiml-gspo-glm47-combi-10268-v1/default/tokenizer_config.json
chaiml-gspo-glm47-combi-10268-v1-uploader: cp /dev/shm/model_output/tokenizer.json s3://guanaco-vllm-models/chaiml-gspo-glm47-combi-10268-v1/default/tokenizer.json
chaiml-gspo-glm47-combi-10268-v1-uploader: cp /dev/shm/model_output/special_tokens_map.json s3://guanaco-vllm-models/chaiml-gspo-glm47-combi-10268-v1/default/special_tokens_map.json
chaiml-gspo-glm47-combi-10268-v1-uploader: cp /dev/shm/model_output/model.safetensors.index.json s3://guanaco-vllm-models/chaiml-gspo-glm47-combi-10268-v1/default/model.safetensors.index.json
2026-03-27T18:59:47.053309+00:00 monitor updated for chaiml-gspo-glm47-combi_60653_v1
2026-03-27T18:59:49.673694+00:00 monitor updated for chaiml-gspo-glm47-combi_10268_v1
chaiml-gspo-glm47-combi-10268-v1-uploader: cp /dev/shm/model_output/model-00072-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm47-combi-10268-v1/default/model-00072-of-00072.safetensors
chaiml-gspo-glm47-combi-10268-v1-uploader: cp /dev/shm/model_output/model-00071-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm47-combi-10268-v1/default/model-00071-of-00072.safetensors
chaiml-gspo-glm47-combi-10268-v1-uploader: cp /dev/shm/model_output/model-00006-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm47-combi-10268-v1/default/model-00006-of-00072.safetensors
chaiml-gspo-glm47-combi-10268-v1-uploader: cp /dev/shm/model_output/model-00021-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm47-combi-10268-v1/default/model-00021-of-00072.safetensors
chaiml-gspo-glm47-combi-10268-v1-uploader: cp /dev/shm/model_output/model-00030-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm47-combi-10268-v1/default/model-00030-of-00072.safetensors
chaiml-gspo-glm47-combi-10268-v1-uploader: cp /dev/shm/model_output/model-00014-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm47-combi-10268-v1/default/model-00014-of-00072.safetensors
chaiml-gspo-glm47-combi-10268-v1-uploader: cp /dev/shm/model_output/model-00002-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm47-combi-10268-v1/default/model-00002-of-00072.safetensors
chaiml-gspo-glm47-combi-10268-v1-uploader: cp /dev/shm/model_output/model-00069-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm47-combi-10268-v1/default/model-00069-of-00072.safetensors
Inference service chaiml-gspo-glm47-combi-60653-v1 ready after 273.44992876052856s
chaiml-gspo-glm47-combi-10268-v1-uploader: cp /dev/shm/model_output/model-00001-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm47-combi-10268-v1/default/model-00001-of-00072.safetensors
Pipeline stage VLLMDeployer completed in 275.71s
chaiml-gspo-glm47-combi-10268-v1-uploader: cp /dev/shm/model_output/model-00010-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm47-combi-10268-v1/default/model-00010-of-00072.safetensors
run pipeline stage %s
chaiml-gspo-glm47-combi-10268-v1-uploader: cp /dev/shm/model_output/model-00036-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm47-combi-10268-v1/default/model-00036-of-00072.safetensors
Running pipeline stage StressChecker
chaiml-gspo-glm47-combi-10268-v1-uploader: cp /dev/shm/model_output/model-00023-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm47-combi-10268-v1/default/model-00023-of-00072.safetensors
chaiml-gspo-glm47-combi-10268-v1-uploader: cp /dev/shm/model_output/model-00039-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm47-combi-10268-v1/default/model-00039-of-00072.safetensors
chaiml-gspo-glm47-combi-10268-v1-uploader: cp /dev/shm/model_output/model-00022-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm47-combi-10268-v1/default/model-00022-of-00072.safetensors
chaiml-gspo-glm47-combi-10268-v1-uploader: cp /dev/shm/model_output/model-00027-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm47-combi-10268-v1/default/model-00027-of-00072.safetensors
chaiml-gspo-glm47-combi-10268-v1-uploader: cp /dev/shm/model_output/model-00017-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm47-combi-10268-v1/default/model-00017-of-00072.safetensors
chaiml-gspo-glm47-combi-10268-v1-uploader: cp /dev/shm/model_output/model-00004-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm47-combi-10268-v1/default/model-00004-of-00072.safetensors
chaiml-gspo-glm47-combi-10268-v1-uploader: cp /dev/shm/model_output/model-00009-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm47-combi-10268-v1/default/model-00009-of-00072.safetensors
chaiml-gspo-glm47-combi-10268-v1-uploader: cp /dev/shm/model_output/model-00054-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm47-combi-10268-v1/default/model-00054-of-00072.safetensors
chaiml-gspo-glm47-combi-10268-v1-uploader: cp /dev/shm/model_output/model-00005-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm47-combi-10268-v1/default/model-00005-of-00072.safetensors
chaiml-gspo-glm47-combi-10268-v1-uploader: cp /dev/shm/model_output/model-00062-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm47-combi-10268-v1/default/model-00062-of-00072.safetensors
chaiml-gspo-glm47-combi-10268-v1-uploader: cp /dev/shm/model_output/model-00019-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm47-combi-10268-v1/default/model-00019-of-00072.safetensors
chaiml-gspo-glm47-combi-10268-v1-uploader: cp /dev/shm/model_output/model-00025-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm47-combi-10268-v1/default/model-00025-of-00072.safetensors
chaiml-gspo-glm47-combi-10268-v1-uploader: cp /dev/shm/model_output/model-00018-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm47-combi-10268-v1/default/model-00018-of-00072.safetensors
chaiml-gspo-glm47-combi-10268-v1-uploader: cp /dev/shm/model_output/model-00020-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm47-combi-10268-v1/default/model-00020-of-00072.safetensors
chaiml-gspo-glm47-combi-10268-v1-uploader: cp /dev/shm/model_output/model-00038-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm47-combi-10268-v1/default/model-00038-of-00072.safetensors
chaiml-gspo-glm47-combi-10268-v1-uploader: cp /dev/shm/model_output/model-00013-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm47-combi-10268-v1/default/model-00013-of-00072.safetensors
Received healthy response to inference request in 8.996192693710327s
2026-03-27T19:00:47.446943+00:00 monitor updated for chaiml-gspo-glm47-combi_60653_v1
chaiml-gspo-glm47-combi-10268-v1-uploader: cp /dev/shm/model_output/model-00026-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm47-combi-10268-v1/default/model-00026-of-00072.safetensors
2026-03-27T19:00:50.044533+00:00 monitor updated for chaiml-gspo-glm47-combi_10268_v1
chaiml-gspo-glm47-combi-10268-v1-uploader: cp /dev/shm/model_output/model-00034-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm47-combi-10268-v1/default/model-00034-of-00072.safetensors
chaiml-gspo-glm47-combi-10268-v1-uploader: cp /dev/shm/model_output/model-00064-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm47-combi-10268-v1/default/model-00064-of-00072.safetensors
chaiml-gspo-glm47-combi-10268-v1-uploader: cp /dev/shm/model_output/model-00042-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm47-combi-10268-v1/default/model-00042-of-00072.safetensors
chaiml-gspo-glm47-combi-10268-v1-uploader: cp /dev/shm/model_output/model-00011-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm47-combi-10268-v1/default/model-00011-of-00072.safetensors
chaiml-gspo-glm47-combi-10268-v1-uploader: cp /dev/shm/model_output/model-00066-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm47-combi-10268-v1/default/model-00066-of-00072.safetensors
chaiml-gspo-glm47-combi-10268-v1-uploader: cp /dev/shm/model_output/model-00028-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm47-combi-10268-v1/default/model-00028-of-00072.safetensors
chaiml-gspo-glm47-combi-10268-v1-uploader: cp /dev/shm/model_output/model-00037-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm47-combi-10268-v1/default/model-00037-of-00072.safetensors
chaiml-gspo-glm47-combi-10268-v1-uploader: cp /dev/shm/model_output/model-00057-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm47-combi-10268-v1/default/model-00057-of-00072.safetensors
chaiml-gspo-glm47-combi-10268-v1-uploader: cp /dev/shm/model_output/model-00033-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm47-combi-10268-v1/default/model-00033-of-00072.safetensors
chaiml-gspo-glm47-combi-10268-v1-uploader: cp /dev/shm/model_output/model-00012-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm47-combi-10268-v1/default/model-00012-of-00072.safetensors
chaiml-gspo-glm47-combi-10268-v1-uploader: cp /dev/shm/model_output/model-00046-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm47-combi-10268-v1/default/model-00046-of-00072.safetensors
chaiml-gspo-glm47-combi-10268-v1-uploader: cp /dev/shm/model_output/model-00051-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm47-combi-10268-v1/default/model-00051-of-00072.safetensors
chaiml-gspo-glm47-combi-10268-v1-uploader: cp /dev/shm/model_output/model-00008-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm47-combi-10268-v1/default/model-00008-of-00072.safetensors
chaiml-gspo-glm47-combi-10268-v1-uploader: cp /dev/shm/model_output/model-00061-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm47-combi-10268-v1/default/model-00061-of-00072.safetensors
chaiml-gspo-glm47-combi-10268-v1-uploader: cp /dev/shm/model_output/model-00044-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm47-combi-10268-v1/default/model-00044-of-00072.safetensors
chaiml-gspo-glm47-combi-10268-v1-uploader: cp /dev/shm/model_output/model-00043-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm47-combi-10268-v1/default/model-00043-of-00072.safetensors
chaiml-gspo-glm47-combi-10268-v1-uploader: cp /dev/shm/model_output/model-00031-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm47-combi-10268-v1/default/model-00031-of-00072.safetensors
chaiml-gspo-glm47-combi-10268-v1-uploader: cp /dev/shm/model_output/model-00053-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm47-combi-10268-v1/default/model-00053-of-00072.safetensors
chaiml-gspo-glm47-combi-10268-v1-uploader: cp /dev/shm/model_output/model-00041-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm47-combi-10268-v1/default/model-00041-of-00072.safetensors
chaiml-gspo-glm47-combi-10268-v1-uploader: cp /dev/shm/model_output/model-00065-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm47-combi-10268-v1/default/model-00065-of-00072.safetensors
chaiml-gspo-glm47-combi-10268-v1-uploader: cp /dev/shm/model_output/model-00040-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm47-combi-10268-v1/default/model-00040-of-00072.safetensors
chaiml-gspo-glm47-combi-10268-v1-uploader: cp /dev/shm/model_output/model-00052-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm47-combi-10268-v1/default/model-00052-of-00072.safetensors
chaiml-gspo-glm47-combi-10268-v1-uploader: cp /dev/shm/model_output/model-00047-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm47-combi-10268-v1/default/model-00047-of-00072.safetensors
chaiml-gspo-glm47-combi-10268-v1-uploader: cp /dev/shm/model_output/model-00060-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm47-combi-10268-v1/default/model-00060-of-00072.safetensors
chaiml-gspo-glm47-combi-10268-v1-uploader: cp /dev/shm/model_output/model-00048-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm47-combi-10268-v1/default/model-00048-of-00072.safetensors
Received healthy response to inference request in 8.633550643920898s
chaiml-gspo-glm47-combi-10268-v1-uploader: cp /dev/shm/model_output/model-00068-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm47-combi-10268-v1/default/model-00068-of-00072.safetensors
chaiml-gspo-glm47-combi-10268-v1-uploader: cp /dev/shm/model_output/model-00055-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm47-combi-10268-v1/default/model-00055-of-00072.safetensors
chaiml-gspo-glm47-combi-10268-v1-uploader: cp /dev/shm/model_output/model-00015-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm47-combi-10268-v1/default/model-00015-of-00072.safetensors
chaiml-gspo-glm47-combi-10268-v1-uploader: cp /dev/shm/model_output/model-00032-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm47-combi-10268-v1/default/model-00032-of-00072.safetensors
chaiml-gspo-glm47-combi-10268-v1-uploader: cp /dev/shm/model_output/model-00058-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm47-combi-10268-v1/default/model-00058-of-00072.safetensors
chaiml-gspo-glm47-combi-10268-v1-uploader: cp /dev/shm/model_output/model-00045-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm47-combi-10268-v1/default/model-00045-of-00072.safetensors
Received healthy response to inference request in 2.3250200748443604s
chaiml-gspo-glm47-combi-10268-v1-uploader: cp /dev/shm/model_output/model-00050-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm47-combi-10268-v1/default/model-00050-of-00072.safetensors
chaiml-gspo-glm47-combi-10268-v1-uploader: cp /dev/shm/model_output/model-00016-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm47-combi-10268-v1/default/model-00016-of-00072.safetensors
chaiml-gspo-glm47-combi-10268-v1-uploader: cp /dev/shm/model_output/model-00056-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm47-combi-10268-v1/default/model-00056-of-00072.safetensors
chaiml-gspo-glm47-combi-10268-v1-uploader: cp /dev/shm/model_output/model-00024-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm47-combi-10268-v1/default/model-00024-of-00072.safetensors
chaiml-gspo-glm47-combi-10268-v1-uploader: cp /dev/shm/model_output/model-00067-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm47-combi-10268-v1/default/model-00067-of-00072.safetensors
chaiml-gspo-glm47-combi-10268-v1-uploader: cp /dev/shm/model_output/model-00049-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm47-combi-10268-v1/default/model-00049-of-00072.safetensors
chaiml-gspo-glm47-combi-10268-v1-uploader: cp /dev/shm/model_output/model-00059-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm47-combi-10268-v1/default/model-00059-of-00072.safetensors
Received healthy response to inference request in 2.091007947921753s
chaiml-gspo-glm47-combi-10268-v1-uploader: cp /dev/shm/model_output/model-00063-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm47-combi-10268-v1/default/model-00063-of-00072.safetensors
chaiml-gspo-glm47-combi-10268-v1-uploader: cp /dev/shm/model_output/model-00070-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm47-combi-10268-v1/default/model-00070-of-00072.safetensors
Received healthy response to inference request in 2.2739686965942383s
Received healthy response to inference request in 2.1339540481567383s
Job chaiml-gspo-glm47-combi-10268-v1-uploader completed after 574.74s with status: succeeded
Stopping job with name chaiml-gspo-glm47-combi-10268-v1-uploader
Pipeline stage VLLMUploader completed in 576.47s
run pipeline stage %s
Running pipeline stage VLLMUploaderAMD
Pipeline stage vllm_upload_amd skipped, reason=not amd cluster
Pipeline stage VLLMUploaderAMD completed in 1.06s
run pipeline stage %s
Received healthy response to inference request in 8.950587511062622s
Running pipeline stage VLLMTemplater
Pipeline stage VLLMTemplater completed in 1.67s
run pipeline stage %s
Running pipeline stage VLLMDeployer
Received healthy response to inference request in 2.0924293994903564s
Creating inference service chaiml-gspo-glm47-combi-10268-v1
Waiting for inference service chaiml-gspo-glm47-combi-10268-v1 to be ready
Received healthy response to inference request in 8.984429597854614s
Received healthy response to inference request in 2.1899499893188477s
Received healthy response to inference request in 4.08288836479187s
Received healthy response to inference request in 2.1080148220062256s
Received healthy response to inference request in 2.1797285079956055s
Received healthy response to inference request in 2.1722702980041504s
Received healthy response to inference request in 2.0753087997436523s
Received healthy response to inference request in 2.1716723442077637s
2026-03-27T19:01:57.291191+00:00 monitor updated for chaiml-gspo-glm47-combi_60653_v1
2026-03-27T19:01:57.594498+00:00 monitor updated for chaiml-gspo-glm47-combi_10268_v1
Received healthy response to inference request in 2.163367986679077s
Received healthy response to inference request in 2.2109899520874023s
Received healthy response to inference request in 2.0892393589019775s
Received healthy response to inference request in 2.1947712898254395s
Received healthy response to inference request in 2.165525436401367s
Received healthy response to inference request in 2.1864638328552246s
Received healthy response to inference request in 2.1708076000213623s
Received healthy response to inference request in 2.4434499740600586s
Received healthy response to inference request in 2.2348949909210205s
Received healthy response to inference request in 2.4976837635040283s
Received healthy response to inference request in 2.0942625999450684s
Received healthy response to inference request in 2.1064441204071045s
Received healthy response to inference request in 2.2809486389160156s
Received healthy response to inference request in 2.112363576889038s
30 requests
0 failed requests
5th percentile: 2.0900352239608764
10th percentile: 2.0922872543334963
20th percentile: 2.1077006816864015
30th percentile: 2.1545438051223753
40th percentile: 2.171326446533203
50th percentile: 2.183096170425415
60th percentile: 2.2012587547302247
70th percentile: 2.2760626792907717
80th percentile: 2.454296731948853
90th percentile: 8.66525433063507
95th percentile: 8.969200658798218
99th percentile: 8.99278139591217
mean time: 3.1470728953679403
Pipeline stage StressChecker completed in 118.82s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyTriggerPipeline
run_pipeline:run_in_cloud %s
starting trigger_guanaco_pipeline args=%s
triggered trigger_guanaco_pipeline args=%s
Pipeline stage OfflineFamilyFriendlyTriggerPipeline completed in 2.55s
Shutdown handler de-registered
chaiml-gspo-glm47-combi_60653_v1 status is now deployed due to DeploymentManager action