chaiml-grpo-glm46-chai-_81744

developer_uid: richhx
submission_id: chaiml-grpo-glm46-chai-_81744_v7
model_name: chaiml-grpo-glm46-chai-_81744_v7
model_group: ChaiML/grpo-glm46-chai-r
status: torndown
timestamp: 2026-03-21T15:36:02+00:00
num_battles: 46
num_wins: 25
family_friendly_score: 0.0
family_friendly_standard_error: 0.0
submission_type: basic
model_repo: ChaiML/grpo-glm46-chai-rm-lr5e6-step150
model_architecture: Glm4MoeForCausalLM
model_num_parameters: 24110003200.0
best_of: 8
max_input_tokens: 1500
max_output_tokens: 80
reward_model: default
display_name: chaiml-grpo-glm46-chai-_81744_v7
ineligible_reason: max_output_tokens!=64
is_internal_developer: True
language_model: ChaiML/grpo-glm46-chai-rm-lr5e6-step150
model_size: 24B
ranking_group: single
us_pacific_date: 2026-03-21
win_ratio: 0.5434782608695652
generation_params: {'temperature': 1.0, 'top_p': 0.95, 'min_p': 0.0, 'top_k': 60, 'presence_penalty': 0.0, 'frequency_penalty': 0.0, 'stopping_words': ['<|assistant|>', '<|im_end|>', '<|user|>', '####', '</s>'], 'max_input_tokens': 1500, 'best_of': 8, 'max_output_tokens': 80}
formatter: {'memory_template': "[gMASK]<sop><|system|>\n{bot_name}'s persona: {memory}", 'prompt_template': '', 'bot_template': '<|assistant|>\n{bot_name}: {message}', 'user_template': '<|user|>\n{message}', 'response_template': '<|assistant|>\n<think></think>\n{bot_name}:', 'truncate_by_message': True}
ad_prompt_fallback_message: You know, with [Chai Ultra](chai://chai.ml/subscription/ultra), you'll have even more opportunities to express yourself, without any guardrails or distracting & annoying ads. We give our Ultra memebers 2x more memory, so your AI won't forget a thing. AND - You will get access to our premium, most advanced AI, it is more intelligent and how do I put it ... it will rock ur world more 😍🫶💞
Resubmit model
Shutdown handler not registered because Python interpreter is not running in the main thread
run pipeline %s
run pipeline stage %s
Running pipeline stage VLLMUploader
Starting job with name chaiml-grpo-glm46-chai-81744-v7-uploader
Waiting for job on chaiml-grpo-glm46-chai-81744-v7-uploader to finish
chaiml-grpo-glm46-combi-79432-v7-uploader: cp /dev/shm/model_output/model-00071-of-00072.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm46-combi-79432-v7/default/model-00071-of-00072.safetensors
chaiml-grpo-glm46-combi-79432-v7-uploader: cp /dev/shm/model_output/model-00011-of-00072.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm46-combi-79432-v7/default/model-00011-of-00072.safetensors
chaiml-grpo-glm46-combi-79432-v7-uploader: cp /dev/shm/model_output/model-00019-of-00072.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm46-combi-79432-v7/default/model-00019-of-00072.safetensors
chaiml-grpo-glm46-combi-79432-v7-uploader: cp /dev/shm/model_output/model-00038-of-00072.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm46-combi-79432-v7/default/model-00038-of-00072.safetensors
chaiml-grpo-glm46-combi-79432-v7-uploader: cp /dev/shm/model_output/model-00030-of-00072.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm46-combi-79432-v7/default/model-00030-of-00072.safetensors
chaiml-grpo-glm46-combi-79432-v7-uploader: cp /dev/shm/model_output/model-00031-of-00072.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm46-combi-79432-v7/default/model-00031-of-00072.safetensors
chaiml-grpo-glm46-combi-79432-v7-uploader: cp /dev/shm/model_output/model-00025-of-00072.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm46-combi-79432-v7/default/model-00025-of-00072.safetensors
chaiml-grpo-glm46-combi-79432-v7-uploader: cp /dev/shm/model_output/model-00015-of-00072.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm46-combi-79432-v7/default/model-00015-of-00072.safetensors
chaiml-grpo-glm46-combi-79432-v7-uploader: cp /dev/shm/model_output/model-00053-of-00072.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm46-combi-79432-v7/default/model-00053-of-00072.safetensors
chaiml-grpo-glm46-combi-79432-v7-uploader: cp /dev/shm/model_output/model-00032-of-00072.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm46-combi-79432-v7/default/model-00032-of-00072.safetensors
chaiml-grpo-glm46-combi-79432-v7-uploader: cp /dev/shm/model_output/model-00061-of-00072.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm46-combi-79432-v7/default/model-00061-of-00072.safetensors
chaiml-grpo-glm46-combi-79432-v7-uploader: cp /dev/shm/model_output/model-00041-of-00072.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm46-combi-79432-v7/default/model-00041-of-00072.safetensors
chaiml-grpo-glm46-combi-79432-v7-uploader: cp /dev/shm/model_output/model-00020-of-00072.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm46-combi-79432-v7/default/model-00020-of-00072.safetensors
chaiml-grpo-glm46-combi-79432-v7-uploader: cp /dev/shm/model_output/model-00006-of-00072.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm46-combi-79432-v7/default/model-00006-of-00072.safetensors
chaiml-grpo-glm46-combi-79432-v7-uploader: cp /dev/shm/model_output/model-00040-of-00072.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm46-combi-79432-v7/default/model-00040-of-00072.safetensors
chaiml-grpo-glm46-combi-79432-v7-uploader: cp /dev/shm/model_output/model-00028-of-00072.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm46-combi-79432-v7/default/model-00028-of-00072.safetensors
chaiml-grpo-glm46-combi-79432-v7-uploader: cp /dev/shm/model_output/model-00033-of-00072.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm46-combi-79432-v7/default/model-00033-of-00072.safetensors
chaiml-grpo-glm46-combi-79432-v7-uploader: cp /dev/shm/model_output/model-00070-of-00072.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm46-combi-79432-v7/default/model-00070-of-00072.safetensors
chaiml-grpo-glm46-combi-79432-v7-uploader: cp /dev/shm/model_output/model-00044-of-00072.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm46-combi-79432-v7/default/model-00044-of-00072.safetensors
chaiml-grpo-glm46-combi-79432-v7-uploader: cp /dev/shm/model_output/model-00014-of-00072.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm46-combi-79432-v7/default/model-00014-of-00072.safetensors
chaiml-grpo-glm46-combi-79432-v7-uploader: cp /dev/shm/model_output/model-00003-of-00072.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm46-combi-79432-v7/default/model-00003-of-00072.safetensors
chaiml-grpo-glm46-combi-79432-v7-uploader: cp /dev/shm/model_output/model-00035-of-00072.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm46-combi-79432-v7/default/model-00035-of-00072.safetensors
chaiml-grpo-glm46-combi-79432-v7-uploader: cp /dev/shm/model_output/model-00012-of-00072.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm46-combi-79432-v7/default/model-00012-of-00072.safetensors
chaiml-grpo-glm46-combi-79432-v7-uploader: cp /dev/shm/model_output/model-00042-of-00072.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm46-combi-79432-v7/default/model-00042-of-00072.safetensors
chaiml-grpo-glm46-combi-79432-v7-uploader: cp /dev/shm/model_output/model-00036-of-00072.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm46-combi-79432-v7/default/model-00036-of-00072.safetensors
chaiml-grpo-glm46-combi-79432-v7-uploader: cp /dev/shm/model_output/model-00046-of-00072.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm46-combi-79432-v7/default/model-00046-of-00072.safetensors
chaiml-grpo-glm46-combi-79432-v7-uploader: cp /dev/shm/model_output/model-00067-of-00072.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm46-combi-79432-v7/default/model-00067-of-00072.safetensors
chaiml-grpo-glm46-combi-79432-v7-uploader: cp /dev/shm/model_output/model-00010-of-00072.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm46-combi-79432-v7/default/model-00010-of-00072.safetensors
chaiml-grpo-glm46-combi-79432-v7-uploader: cp /dev/shm/model_output/model-00034-of-00072.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm46-combi-79432-v7/default/model-00034-of-00072.safetensors
chaiml-grpo-glm46-combi-79432-v7-uploader: cp /dev/shm/model_output/model-00043-of-00072.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm46-combi-79432-v7/default/model-00043-of-00072.safetensors
chaiml-grpo-glm46-combi-79432-v7-uploader: cp /dev/shm/model_output/model-00050-of-00072.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm46-combi-79432-v7/default/model-00050-of-00072.safetensors
chaiml-grpo-glm46-combi-79432-v7-uploader: cp /dev/shm/model_output/model-00054-of-00072.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm46-combi-79432-v7/default/model-00054-of-00072.safetensors
chaiml-grpo-glm46-combi-79432-v7-uploader: cp /dev/shm/model_output/model-00063-of-00072.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm46-combi-79432-v7/default/model-00063-of-00072.safetensors
chaiml-grpo-glm46-combi-79432-v7-uploader: cp /dev/shm/model_output/model-00062-of-00072.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm46-combi-79432-v7/default/model-00062-of-00072.safetensors
chaiml-grpo-glm46-combi-79432-v7-uploader: cp /dev/shm/model_output/model-00060-of-00072.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm46-combi-79432-v7/default/model-00060-of-00072.safetensors
chaiml-grpo-glm46-combi-79432-v7-uploader: cp /dev/shm/model_output/model-00016-of-00072.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm46-combi-79432-v7/default/model-00016-of-00072.safetensors
chaiml-grpo-glm46-combi-79432-v7-uploader: cp /dev/shm/model_output/model-00049-of-00072.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm46-combi-79432-v7/default/model-00049-of-00072.safetensors
chaiml-grpo-glm46-combi-79432-v7-uploader: cp /dev/shm/model_output/model-00021-of-00072.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm46-combi-79432-v7/default/model-00021-of-00072.safetensors
chaiml-grpo-glm46-combi-79432-v7-uploader: cp /dev/shm/model_output/model-00069-of-00072.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm46-combi-79432-v7/default/model-00069-of-00072.safetensors
chaiml-grpo-glm46-combi-79432-v7-uploader: cp /dev/shm/model_output/model-00004-of-00072.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm46-combi-79432-v7/default/model-00004-of-00072.safetensors
chaiml-grpo-glm46-combi-79432-v7-uploader: cp /dev/shm/model_output/model-00027-of-00072.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm46-combi-79432-v7/default/model-00027-of-00072.safetensors
chaiml-grpo-glm46-combi-79432-v7-uploader: cp /dev/shm/model_output/model-00002-of-00072.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm46-combi-79432-v7/default/model-00002-of-00072.safetensors
chaiml-grpo-glm46-combi-79432-v7-uploader: cp /dev/shm/model_output/model-00065-of-00072.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm46-combi-79432-v7/default/model-00065-of-00072.safetensors
chaiml-grpo-glm46-combi-79432-v7-uploader: cp /dev/shm/model_output/model-00009-of-00072.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm46-combi-79432-v7/default/model-00009-of-00072.safetensors
chaiml-grpo-glm46-combi-79432-v7-uploader: cp /dev/shm/model_output/model-00045-of-00072.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm46-combi-79432-v7/default/model-00045-of-00072.safetensors
chaiml-grpo-glm46-combi-79432-v7-uploader: cp /dev/shm/model_output/model-00052-of-00072.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm46-combi-79432-v7/default/model-00052-of-00072.safetensors
chaiml-grpo-glm46-combi-79432-v7-uploader: cp /dev/shm/model_output/model-00023-of-00072.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm46-combi-79432-v7/default/model-00023-of-00072.safetensors
chaiml-grpo-glm46-combi-79432-v7-uploader: cp /dev/shm/model_output/model-00029-of-00072.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm46-combi-79432-v7/default/model-00029-of-00072.safetensors
chaiml-grpo-glm46-combi-79432-v7-uploader: cp /dev/shm/model_output/model-00017-of-00072.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm46-combi-79432-v7/default/model-00017-of-00072.safetensors
chaiml-grpo-glm46-combi-79432-v7-uploader: cp /dev/shm/model_output/model-00026-of-00072.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm46-combi-79432-v7/default/model-00026-of-00072.safetensors
chaiml-grpo-glm46-combi-79432-v7-uploader: cp /dev/shm/model_output/model-00047-of-00072.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm46-combi-79432-v7/default/model-00047-of-00072.safetensors
chaiml-grpo-glm46-combi-79432-v7-uploader: cp /dev/shm/model_output/model-00057-of-00072.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm46-combi-79432-v7/default/model-00057-of-00072.safetensors
chaiml-grpo-glm46-combi-79432-v7-uploader: cp /dev/shm/model_output/model-00051-of-00072.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm46-combi-79432-v7/default/model-00051-of-00072.safetensors
chaiml-grpo-glm46-combi-79432-v7-uploader: cp /dev/shm/model_output/model-00005-of-00072.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm46-combi-79432-v7/default/model-00005-of-00072.safetensors
chaiml-grpo-glm46-combi-79432-v7-uploader: cp /dev/shm/model_output/model-00056-of-00072.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm46-combi-79432-v7/default/model-00056-of-00072.safetensors
chaiml-grpo-glm46-combi-79432-v7-uploader: cp /dev/shm/model_output/model-00008-of-00072.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm46-combi-79432-v7/default/model-00008-of-00072.safetensors
chaiml-grpo-glm46-combi-79432-v7-uploader: cp /dev/shm/model_output/model-00024-of-00072.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm46-combi-79432-v7/default/model-00024-of-00072.safetensors
chaiml-grpo-glm46-combi-79432-v7-uploader: cp /dev/shm/model_output/model-00013-of-00072.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm46-combi-79432-v7/default/model-00013-of-00072.safetensors
chaiml-grpo-glm46-combi-79432-v7-uploader: cp /dev/shm/model_output/model-00037-of-00072.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm46-combi-79432-v7/default/model-00037-of-00072.safetensors
chaiml-grpo-glm46-combi-79432-v7-uploader: cp /dev/shm/model_output/model-00048-of-00072.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm46-combi-79432-v7/default/model-00048-of-00072.safetensors
chaiml-grpo-glm46-combi-79432-v7-uploader: cp /dev/shm/model_output/model-00055-of-00072.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm46-combi-79432-v7/default/model-00055-of-00072.safetensors
Job chaiml-grpo-glm46-combi-79432-v7-uploader completed after 284.16s with status: succeeded
Stopping job with name chaiml-grpo-glm46-combi-79432-v7-uploader
Pipeline stage VLLMUploader completed in 284.83s
run pipeline stage %s
Running pipeline stage VLLMTemplater
Pipeline stage VLLMTemplater completed in 0.68s
run pipeline stage %s
Running pipeline stage VLLMDeployer
Creating inference service chaiml-grpo-glm46-combi-79432-v7
Waiting for inference service chaiml-grpo-glm46-combi-79432-v7 to be ready
chaiml-grpo-glm46-chai-81744-v7-uploader: Using quantization_mode: fp8
chaiml-grpo-glm46-chai-81744-v7-uploader: Checking if ChaiML/grpo-glm46-chai-rm-lr5e6-step150-FP8 already exists in ChaiML
chaiml-grpo-glm46-chai-81744-v7-uploader: Model already exists. Downloading to /dev/shm/model_output...
chaiml-grpo-glm46-chai-81744-v7-uploader: Downloading snapshot of ChaiML/grpo-glm46-chai-rm-lr5e6-step150-FP8...
2026-03-21T15:08:56.422274+00:00 monitor updated for chaiml-grpo-glm46-combi_79432_v7
2026-03-21T15:08:57.866386+00:00 monitor updated for chaiml-gspo-glm46-combi_11090_v7
2026-03-21T15:08:58.997880+00:00 monitor updated for chaiml-grpo-glm46-chai-_81744_v7
chaiml-gspo-glm46-combi-11090-v7-uploader: Downloaded in 136.696s
chaiml-gspo-glm46-combi-11090-v7-uploader: Processed model ChaiML/gspo-glm46-combine-rm55-step200 in 140.120s
chaiml-gspo-glm46-combi-11090-v7-uploader: creating bucket guanaco-vllm-models
chaiml-gspo-glm46-combi-11090-v7-uploader: /usr/lib/python3/dist-packages/S3/BaseUtils.py:56: SyntaxWarning: invalid escape sequence '\.'
chaiml-gspo-glm46-combi-11090-v7-uploader:   RE_S3_DATESTRING = re.compile('\.[0-9]*(?:[Z\\-\\+]*?)')
chaiml-gspo-glm46-combi-11090-v7-uploader: /usr/lib/python3/dist-packages/S3/BaseUtils.py:57: SyntaxWarning: invalid escape sequence '\s'
chaiml-gspo-glm46-combi-11090-v7-uploader:   RE_XML_NAMESPACE = re.compile(b'^(<?[^>]+?>\s*|\s*)(<\w+) xmlns=[\'"](https?://[^\'"]+)[\'"]', re.MULTILINE)
chaiml-gspo-glm46-combi-11090-v7-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:240: SyntaxWarning: invalid escape sequence '\.'
chaiml-gspo-glm46-combi-11090-v7-uploader:   invalid = re.search("([^a-z0-9\.-])", bucket, re.UNICODE)
chaiml-gspo-glm46-combi-11090-v7-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:244: SyntaxWarning: invalid escape sequence '\.'
chaiml-gspo-glm46-combi-11090-v7-uploader:   invalid = re.search("([^A-Za-z0-9\._-])", bucket, re.UNICODE)
chaiml-gspo-glm46-combi-11090-v7-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:255: SyntaxWarning: invalid escape sequence '\.'
chaiml-gspo-glm46-combi-11090-v7-uploader:   if re.search("-\.", bucket, re.UNICODE):
chaiml-gspo-glm46-combi-11090-v7-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:257: SyntaxWarning: invalid escape sequence '\.'
chaiml-gspo-glm46-combi-11090-v7-uploader:   if re.search("\.\.", bucket, re.UNICODE):
chaiml-gspo-glm46-combi-11090-v7-uploader: /usr/lib/python3/dist-packages/S3/S3Uri.py:155: SyntaxWarning: invalid escape sequence '\w'
chaiml-gspo-glm46-combi-11090-v7-uploader:   _re = re.compile("^(\w+://)?(.*)", re.UNICODE)
chaiml-gspo-glm46-combi-11090-v7-uploader: /usr/lib/python3/dist-packages/S3/FileLists.py:480: SyntaxWarning: invalid escape sequence '\*'
chaiml-gspo-glm46-combi-11090-v7-uploader:   wildcard_split_result = re.split("\*|\?", uri_str, maxsplit=1)
chaiml-gspo-glm46-combi-11090-v7-uploader: Bucket 's3://guanaco-vllm-models/' created
chaiml-gspo-glm46-combi-11090-v7-uploader: uploading /dev/shm/model_output to s3://guanaco-vllm-models/chaiml-gspo-glm46-combi-11090-v7/default
chaiml-gspo-glm46-combi-11090-v7-uploader: cp /dev/shm/model_output/chat_template.jinja s3://guanaco-vllm-models/chaiml-gspo-glm46-combi-11090-v7/default/chat_template.jinja
chaiml-gspo-glm46-combi-11090-v7-uploader: cp /dev/shm/model_output/.gitattributes s3://guanaco-vllm-models/chaiml-gspo-glm46-combi-11090-v7/default/.gitattributes
chaiml-gspo-glm46-combi-11090-v7-uploader: cp /dev/shm/model_output/generation_config.json s3://guanaco-vllm-models/chaiml-gspo-glm46-combi-11090-v7/default/generation_config.json
chaiml-gspo-glm46-combi-11090-v7-uploader: cp /dev/shm/model_output/config.json s3://guanaco-vllm-models/chaiml-gspo-glm46-combi-11090-v7/default/config.json
chaiml-gspo-glm46-combi-11090-v7-uploader: cp /dev/shm/model_output/tokenizer_config.json s3://guanaco-vllm-models/chaiml-gspo-glm46-combi-11090-v7/default/tokenizer_config.json
chaiml-gspo-glm46-combi-11090-v7-uploader: cp /dev/shm/model_output/recipe.yaml s3://guanaco-vllm-models/chaiml-gspo-glm46-combi-11090-v7/default/recipe.yaml
chaiml-gspo-glm46-combi-11090-v7-uploader: cp /dev/shm/model_output/special_tokens_map.json s3://guanaco-vllm-models/chaiml-gspo-glm46-combi-11090-v7/default/special_tokens_map.json
chaiml-gspo-glm46-combi-11090-v7-uploader: cp /dev/shm/model_output/model.safetensors.index.json s3://guanaco-vllm-models/chaiml-gspo-glm46-combi-11090-v7/default/model.safetensors.index.json
chaiml-gspo-glm46-combi-11090-v7-uploader: cp /dev/shm/model_output/tokenizer.json s3://guanaco-vllm-models/chaiml-gspo-glm46-combi-11090-v7/default/tokenizer.json
chaiml-gspo-glm46-combi-11090-v7-uploader: cp /dev/shm/model_output/model-00072-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm46-combi-11090-v7/default/model-00072-of-00072.safetensors
chaiml-gspo-glm46-combi-11090-v7-uploader: cp /dev/shm/model_output/model-00071-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm46-combi-11090-v7/default/model-00071-of-00072.safetensors
2026-03-21T15:09:56.619380+00:00 monitor updated for chaiml-grpo-glm46-combi_79432_v7
2026-03-21T15:09:58.067769+00:00 monitor updated for chaiml-gspo-glm46-combi_11090_v7
2026-03-21T15:09:59.205867+00:00 monitor updated for chaiml-grpo-glm46-chai-_81744_v7
chaiml-gspo-glm46-combi-11090-v7-uploader: cp /dev/shm/model_output/model-00010-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm46-combi-11090-v7/default/model-00010-of-00072.safetensors
chaiml-gspo-glm46-combi-11090-v7-uploader: cp /dev/shm/model_output/model-00008-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm46-combi-11090-v7/default/model-00008-of-00072.safetensors
chaiml-gspo-glm46-combi-11090-v7-uploader: cp /dev/shm/model_output/model-00035-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm46-combi-11090-v7/default/model-00035-of-00072.safetensors
chaiml-gspo-glm46-combi-11090-v7-uploader: cp /dev/shm/model_output/model-00037-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm46-combi-11090-v7/default/model-00037-of-00072.safetensors
chaiml-gspo-glm46-combi-11090-v7-uploader: cp /dev/shm/model_output/model-00013-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm46-combi-11090-v7/default/model-00013-of-00072.safetensors
chaiml-gspo-glm46-combi-11090-v7-uploader: cp /dev/shm/model_output/model-00031-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm46-combi-11090-v7/default/model-00031-of-00072.safetensors
chaiml-gspo-glm46-combi-11090-v7-uploader: cp /dev/shm/model_output/model-00042-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm46-combi-11090-v7/default/model-00042-of-00072.safetensors
chaiml-gspo-glm46-combi-11090-v7-uploader: cp /dev/shm/model_output/model-00041-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm46-combi-11090-v7/default/model-00041-of-00072.safetensors
chaiml-gspo-glm46-combi-11090-v7-uploader: cp /dev/shm/model_output/model-00009-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm46-combi-11090-v7/default/model-00009-of-00072.safetensors
chaiml-gspo-glm46-combi-11090-v7-uploader: cp /dev/shm/model_output/model-00057-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm46-combi-11090-v7/default/model-00057-of-00072.safetensors
chaiml-gspo-glm46-combi-11090-v7-uploader: cp /dev/shm/model_output/model-00004-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm46-combi-11090-v7/default/model-00004-of-00072.safetensors
chaiml-gspo-glm46-combi-11090-v7-uploader: cp /dev/shm/model_output/model-00058-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm46-combi-11090-v7/default/model-00058-of-00072.safetensors
chaiml-gspo-glm46-combi-11090-v7-uploader: cp /dev/shm/model_output/model-00063-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm46-combi-11090-v7/default/model-00063-of-00072.safetensors
chaiml-gspo-glm46-combi-11090-v7-uploader: cp /dev/shm/model_output/model-00034-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm46-combi-11090-v7/default/model-00034-of-00072.safetensors
chaiml-gspo-glm46-combi-11090-v7-uploader: cp /dev/shm/model_output/model-00017-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm46-combi-11090-v7/default/model-00017-of-00072.safetensors
chaiml-gspo-glm46-combi-11090-v7-uploader: cp /dev/shm/model_output/model-00039-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm46-combi-11090-v7/default/model-00039-of-00072.safetensors
chaiml-gspo-glm46-combi-11090-v7-uploader: cp /dev/shm/model_output/model-00054-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm46-combi-11090-v7/default/model-00054-of-00072.safetensors
chaiml-gspo-glm46-combi-11090-v7-uploader: cp /dev/shm/model_output/model-00036-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm46-combi-11090-v7/default/model-00036-of-00072.safetensors
chaiml-gspo-glm46-combi-11090-v7-uploader: cp /dev/shm/model_output/model-00011-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm46-combi-11090-v7/default/model-00011-of-00072.safetensors
chaiml-gspo-glm46-combi-11090-v7-uploader: cp /dev/shm/model_output/model-00062-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm46-combi-11090-v7/default/model-00062-of-00072.safetensors
chaiml-gspo-glm46-combi-11090-v7-uploader: cp /dev/shm/model_output/model-00045-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm46-combi-11090-v7/default/model-00045-of-00072.safetensors
chaiml-gspo-glm46-combi-11090-v7-uploader: cp /dev/shm/model_output/model-00061-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm46-combi-11090-v7/default/model-00061-of-00072.safetensors
chaiml-gspo-glm46-combi-11090-v7-uploader: cp /dev/shm/model_output/model-00070-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm46-combi-11090-v7/default/model-00070-of-00072.safetensors
chaiml-gspo-glm46-combi-11090-v7-uploader: cp /dev/shm/model_output/model-00032-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm46-combi-11090-v7/default/model-00032-of-00072.safetensors
chaiml-gspo-glm46-combi-11090-v7-uploader: cp /dev/shm/model_output/model-00049-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm46-combi-11090-v7/default/model-00049-of-00072.safetensors
chaiml-grpo-glm46-chai-81744-v7-uploader: Downloaded in 98.300s
chaiml-gspo-glm46-combi-11090-v7-uploader: cp /dev/shm/model_output/model-00043-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm46-combi-11090-v7/default/model-00043-of-00072.safetensors
chaiml-grpo-glm46-chai-81744-v7-uploader: Processed model ChaiML/grpo-glm46-chai-rm-lr5e6-step150 in 101.878s
chaiml-gspo-glm46-combi-11090-v7-uploader: cp /dev/shm/model_output/model-00065-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm46-combi-11090-v7/default/model-00065-of-00072.safetensors
chaiml-grpo-glm46-chai-81744-v7-uploader: creating bucket guanaco-vllm-models
chaiml-gspo-glm46-combi-11090-v7-uploader: cp /dev/shm/model_output/model-00007-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm46-combi-11090-v7/default/model-00007-of-00072.safetensors
chaiml-grpo-glm46-chai-81744-v7-uploader: /usr/lib/python3/dist-packages/S3/BaseUtils.py:56: SyntaxWarning: invalid escape sequence '\.'
chaiml-gspo-glm46-combi-11090-v7-uploader: cp /dev/shm/model_output/model-00052-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm46-combi-11090-v7/default/model-00052-of-00072.safetensors
chaiml-grpo-glm46-chai-81744-v7-uploader:   RE_S3_DATESTRING = re.compile('\.[0-9]*(?:[Z\\-\\+]*?)')
chaiml-gspo-glm46-combi-11090-v7-uploader: cp /dev/shm/model_output/model-00014-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm46-combi-11090-v7/default/model-00014-of-00072.safetensors
chaiml-grpo-glm46-chai-81744-v7-uploader: /usr/lib/python3/dist-packages/S3/BaseUtils.py:57: SyntaxWarning: invalid escape sequence '\s'
chaiml-gspo-glm46-combi-11090-v7-uploader: cp /dev/shm/model_output/model-00053-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm46-combi-11090-v7/default/model-00053-of-00072.safetensors
chaiml-grpo-glm46-chai-81744-v7-uploader:   RE_XML_NAMESPACE = re.compile(b'^(<?[^>]+?>\s*|\s*)(<\w+) xmlns=[\'"](https?://[^\'"]+)[\'"]', re.MULTILINE)
chaiml-gspo-glm46-combi-11090-v7-uploader: cp /dev/shm/model_output/model-00005-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm46-combi-11090-v7/default/model-00005-of-00072.safetensors
chaiml-grpo-glm46-chai-81744-v7-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:240: SyntaxWarning: invalid escape sequence '\.'
chaiml-gspo-glm46-combi-11090-v7-uploader: cp /dev/shm/model_output/model-00001-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm46-combi-11090-v7/default/model-00001-of-00072.safetensors
chaiml-grpo-glm46-chai-81744-v7-uploader:   invalid = re.search("([^a-z0-9\.-])", bucket, re.UNICODE)
chaiml-gspo-glm46-combi-11090-v7-uploader: cp /dev/shm/model_output/model-00012-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm46-combi-11090-v7/default/model-00012-of-00072.safetensors
chaiml-grpo-glm46-chai-81744-v7-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:244: SyntaxWarning: invalid escape sequence '\.'
chaiml-gspo-glm46-combi-11090-v7-uploader: cp /dev/shm/model_output/model-00016-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm46-combi-11090-v7/default/model-00016-of-00072.safetensors
chaiml-grpo-glm46-chai-81744-v7-uploader:   invalid = re.search("([^A-Za-z0-9\._-])", bucket, re.UNICODE)
chaiml-gspo-glm46-combi-11090-v7-uploader: cp /dev/shm/model_output/model-00064-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm46-combi-11090-v7/default/model-00064-of-00072.safetensors
chaiml-grpo-glm46-chai-81744-v7-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:255: SyntaxWarning: invalid escape sequence '\.'
chaiml-gspo-glm46-combi-11090-v7-uploader: cp /dev/shm/model_output/model-00056-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm46-combi-11090-v7/default/model-00056-of-00072.safetensors
chaiml-grpo-glm46-chai-81744-v7-uploader:   if re.search("-\.", bucket, re.UNICODE):
chaiml-gspo-glm46-combi-11090-v7-uploader: cp /dev/shm/model_output/model-00028-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm46-combi-11090-v7/default/model-00028-of-00072.safetensors
chaiml-grpo-glm46-chai-81744-v7-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:257: SyntaxWarning: invalid escape sequence '\.'
chaiml-gspo-glm46-combi-11090-v7-uploader: cp /dev/shm/model_output/model-00044-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm46-combi-11090-v7/default/model-00044-of-00072.safetensors
chaiml-grpo-glm46-chai-81744-v7-uploader:   if re.search("\.\.", bucket, re.UNICODE):
chaiml-gspo-glm46-combi-11090-v7-uploader: cp /dev/shm/model_output/model-00020-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm46-combi-11090-v7/default/model-00020-of-00072.safetensors
chaiml-grpo-glm46-chai-81744-v7-uploader: /usr/lib/python3/dist-packages/S3/S3Uri.py:155: SyntaxWarning: invalid escape sequence '\w'
chaiml-gspo-glm46-combi-11090-v7-uploader: cp /dev/shm/model_output/model-00019-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm46-combi-11090-v7/default/model-00019-of-00072.safetensors
chaiml-grpo-glm46-chai-81744-v7-uploader:   _re = re.compile("^(\w+://)?(.*)", re.UNICODE)
chaiml-gspo-glm46-combi-11090-v7-uploader: cp /dev/shm/model_output/model-00068-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm46-combi-11090-v7/default/model-00068-of-00072.safetensors
chaiml-grpo-glm46-chai-81744-v7-uploader: /usr/lib/python3/dist-packages/S3/FileLists.py:480: SyntaxWarning: invalid escape sequence '\*'
chaiml-gspo-glm46-combi-11090-v7-uploader: cp /dev/shm/model_output/model-00006-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm46-combi-11090-v7/default/model-00006-of-00072.safetensors
chaiml-grpo-glm46-chai-81744-v7-uploader:   wildcard_split_result = re.split("\*|\?", uri_str, maxsplit=1)
chaiml-gspo-glm46-combi-11090-v7-uploader: cp /dev/shm/model_output/model-00060-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm46-combi-11090-v7/default/model-00060-of-00072.safetensors
chaiml-grpo-glm46-chai-81744-v7-uploader: Bucket 's3://guanaco-vllm-models/' created
chaiml-gspo-glm46-combi-11090-v7-uploader: cp /dev/shm/model_output/model-00022-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm46-combi-11090-v7/default/model-00022-of-00072.safetensors
chaiml-grpo-glm46-chai-81744-v7-uploader: uploading /dev/shm/model_output to s3://guanaco-vllm-models/chaiml-grpo-glm46-chai-81744-v7/default
chaiml-gspo-glm46-combi-11090-v7-uploader: cp /dev/shm/model_output/model-00024-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm46-combi-11090-v7/default/model-00024-of-00072.safetensors
chaiml-grpo-glm46-chai-81744-v7-uploader: cp /dev/shm/model_output/.gitattributes s3://guanaco-vllm-models/chaiml-grpo-glm46-chai-81744-v7/default/.gitattributes
chaiml-gspo-glm46-combi-11090-v7-uploader: cp /dev/shm/model_output/model-00018-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm46-combi-11090-v7/default/model-00018-of-00072.safetensors
chaiml-grpo-glm46-chai-81744-v7-uploader: cp /dev/shm/model_output/config.json s3://guanaco-vllm-models/chaiml-grpo-glm46-chai-81744-v7/default/config.json
chaiml-gspo-glm46-combi-11090-v7-uploader: cp /dev/shm/model_output/model-00040-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm46-combi-11090-v7/default/model-00040-of-00072.safetensors
chaiml-grpo-glm46-chai-81744-v7-uploader: cp /dev/shm/model_output/recipe.yaml s3://guanaco-vllm-models/chaiml-grpo-glm46-chai-81744-v7/default/recipe.yaml
chaiml-gspo-glm46-combi-11090-v7-uploader: cp /dev/shm/model_output/model-00015-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm46-combi-11090-v7/default/model-00015-of-00072.safetensors
chaiml-grpo-glm46-chai-81744-v7-uploader: cp /dev/shm/model_output/special_tokens_map.json s3://guanaco-vllm-models/chaiml-grpo-glm46-chai-81744-v7/default/special_tokens_map.json
chaiml-gspo-glm46-combi-11090-v7-uploader: cp /dev/shm/model_output/model-00038-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm46-combi-11090-v7/default/model-00038-of-00072.safetensors
chaiml-grpo-glm46-chai-81744-v7-uploader: cp /dev/shm/model_output/generation_config.json s3://guanaco-vllm-models/chaiml-grpo-glm46-chai-81744-v7/default/generation_config.json
chaiml-gspo-glm46-combi-11090-v7-uploader: cp /dev/shm/model_output/model-00051-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm46-combi-11090-v7/default/model-00051-of-00072.safetensors
chaiml-grpo-glm46-chai-81744-v7-uploader: cp /dev/shm/model_output/chat_template.jinja s3://guanaco-vllm-models/chaiml-grpo-glm46-chai-81744-v7/default/chat_template.jinja
chaiml-gspo-glm46-combi-11090-v7-uploader: cp /dev/shm/model_output/model-00055-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm46-combi-11090-v7/default/model-00055-of-00072.safetensors
chaiml-grpo-glm46-chai-81744-v7-uploader: cp /dev/shm/model_output/tokenizer_config.json s3://guanaco-vllm-models/chaiml-grpo-glm46-chai-81744-v7/default/tokenizer_config.json
chaiml-gspo-glm46-combi-11090-v7-uploader: cp /dev/shm/model_output/model-00050-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm46-combi-11090-v7/default/model-00050-of-00072.safetensors
chaiml-grpo-glm46-chai-81744-v7-uploader: cp /dev/shm/model_output/tokenizer.json s3://guanaco-vllm-models/chaiml-grpo-glm46-chai-81744-v7/default/tokenizer.json
chaiml-gspo-glm46-combi-11090-v7-uploader: cp /dev/shm/model_output/model-00046-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm46-combi-11090-v7/default/model-00046-of-00072.safetensors
chaiml-grpo-glm46-chai-81744-v7-uploader: cp /dev/shm/model_output/model.safetensors.index.json s3://guanaco-vllm-models/chaiml-grpo-glm46-chai-81744-v7/default/model.safetensors.index.json
chaiml-gspo-glm46-combi-11090-v7-uploader: cp /dev/shm/model_output/model-00021-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm46-combi-11090-v7/default/model-00021-of-00072.safetensors
chaiml-gspo-glm46-combi-11090-v7-uploader: cp /dev/shm/model_output/model-00030-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm46-combi-11090-v7/default/model-00030-of-00072.safetensors
chaiml-gspo-glm46-combi-11090-v7-uploader: cp /dev/shm/model_output/model-00059-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm46-combi-11090-v7/default/model-00059-of-00072.safetensors
chaiml-gspo-glm46-combi-11090-v7-uploader: cp /dev/shm/model_output/model-00002-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm46-combi-11090-v7/default/model-00002-of-00072.safetensors
chaiml-gspo-glm46-combi-11090-v7-uploader: cp /dev/shm/model_output/model-00003-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm46-combi-11090-v7/default/model-00003-of-00072.safetensors
chaiml-gspo-glm46-combi-11090-v7-uploader: cp /dev/shm/model_output/model-00025-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm46-combi-11090-v7/default/model-00025-of-00072.safetensors
chaiml-gspo-glm46-combi-11090-v7-uploader: cp /dev/shm/model_output/model-00066-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm46-combi-11090-v7/default/model-00066-of-00072.safetensors
chaiml-gspo-glm46-combi-11090-v7-uploader: cp /dev/shm/model_output/model-00069-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm46-combi-11090-v7/default/model-00069-of-00072.safetensors
chaiml-gspo-glm46-combi-11090-v7-uploader: cp /dev/shm/model_output/model-00067-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm46-combi-11090-v7/default/model-00067-of-00072.safetensors
chaiml-gspo-glm46-combi-11090-v7-uploader: cp /dev/shm/model_output/model-00023-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm46-combi-11090-v7/default/model-00023-of-00072.safetensors
chaiml-gspo-glm46-combi-11090-v7-uploader: cp /dev/shm/model_output/model-00033-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm46-combi-11090-v7/default/model-00033-of-00072.safetensors
chaiml-gspo-glm46-combi-11090-v7-uploader: cp /dev/shm/model_output/model-00048-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm46-combi-11090-v7/default/model-00048-of-00072.safetensors
chaiml-gspo-glm46-combi-11090-v7-uploader: cp /dev/shm/model_output/model-00027-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm46-combi-11090-v7/default/model-00027-of-00072.safetensors
chaiml-gspo-glm46-combi-11090-v7-uploader: cp /dev/shm/model_output/model-00047-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm46-combi-11090-v7/default/model-00047-of-00072.safetensors
chaiml-gspo-glm46-combi-11090-v7-uploader: cp /dev/shm/model_output/model-00029-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm46-combi-11090-v7/default/model-00029-of-00072.safetensors
chaiml-grpo-glm46-chai-_81744_v7 status is now pending due to richhx requesting redeployment of chaiml-grpo-glm46-chai-_81744_v7
Shutdown handler not registered because Python interpreter is not running in the main thread
run pipeline %s
run pipeline stage %s
Running pipeline stage VLLMUploader
Starting job with name chaiml-grpo-glm46-chai-81744-v7-uploader
Job failed chaiml-grpo-glm46-chai-81744-v7-uploader: (409)
Reason: Conflict
HTTP response headers: HTTPHeaderDict({'Date': 'Sat, 21 Mar 2026 15:17:32 GMT', 'Content-Type': 'application/json', 'Content-Length': '280', 'Connection': 'keep-alive', 'audit-id': 'd71eec38-af6c-4f44-a4bc-6c168ea22d06', 'Cache-Control': 'no-cache, private', 'x-kubernetes-pf-flowschema-uid': '757159af-7462-4610-a5a7-751d984daa90', 'x-kubernetes-pf-prioritylevel-uid': '5f6bf145-f133-455b-a330-c9bf3e5ea771', 'cf-cache-status': 'DYNAMIC', 'set-cookie': '__cf_bm=-1774106252.3478396-1.0.1.1-w9L46xCBsMinSjxpRaVMgL4CrPBBV.Th2QV5IYqtHM9q.kVZBnTvZ4mg8UK9EINam7GXrNk1aysZDyvmF_uqPkYhDlR6nM4Jh10F1RtJtCaGYEiRONWAXVXvE_XhJ3nB; HttpOnly; Secure; Path=/; Domain=coreweave.com; Expires=Sat, 21 Mar 2026 15:47:32 GMT', 'Server': 'cloudflare', 'CF-RAY': '9dfdec8d2b91e5ba-IAD'})
HTTP response body: {"kind":"Status","apiVersion":"v1","metadata":{},"status":"Failure","message":"jobs.batch \"chaiml-grpo-glm46-chai-81744-v7-uploader\" already exists","reason":"AlreadyExists","details":{"name":"chaiml-grpo-glm46-chai-81744-v7-uploader","group":"batch","kind":"jobs"},"code":409}

Stopping job with name chaiml-grpo-glm46-chai-81744-v7-uploader
%s, retrying in %s seconds...
Starting job with name chaiml-grpo-glm46-chai-81744-v7-uploader
Waiting for job on chaiml-grpo-glm46-chai-81744-v7-uploader to finish
chaiml-gspo-glm46-combi-11090-v7-uploader: Using quantization_mode: fp8
2026-03-21T15:18:20.433194+00:00 monitor updated for chaiml-gspo-glm46-combi_11090_v7
chaiml-gspo-glm46-combi-11090-v7-uploader: Checking if ChaiML/gspo-glm46-combine-rm55-step200-FP8 already exists in ChaiML
chaiml-gspo-glm46-combi-11090-v7-uploader: Model already exists. Downloading to /dev/shm/model_output...
chaiml-gspo-glm46-combi-11090-v7-uploader: Downloading snapshot of ChaiML/gspo-glm46-combine-rm55-step200-FP8...
chaiml-grpo-glm46-chai-81744-v7-uploader: Using quantization_mode: fp8
chaiml-grpo-glm46-chai-81744-v7-uploader: Checking if ChaiML/grpo-glm46-chai-rm-lr5e6-step150-FP8 already exists in ChaiML
chaiml-grpo-glm46-chai-81744-v7-uploader: Model already exists. Downloading to /dev/shm/model_output...
chaiml-grpo-glm46-chai-81744-v7-uploader: Downloading snapshot of ChaiML/grpo-glm46-chai-rm-lr5e6-step150-FP8...
2026-03-21T15:18:31.510849+00:00 monitor updated for chaiml-grpo-glm46-chai-_81744_v7
2026-03-21T15:19:20.638320+00:00 monitor updated for chaiml-gspo-glm46-combi_11090_v7
2026-03-21T15:19:31.722732+00:00 monitor updated for chaiml-grpo-glm46-chai-_81744_v7
2026-03-21T15:20:20.991922+00:00 monitor updated for chaiml-gspo-glm46-combi_11090_v7
2026-03-21T15:20:31.971612+00:00 monitor updated for chaiml-grpo-glm46-chai-_81744_v7
chaiml-gspo-glm46-combi-11090-v7-uploader: Downloaded in 132.251s
chaiml-gspo-glm46-combi-11090-v7-uploader: Processed model ChaiML/gspo-glm46-combine-rm55-step200 in 135.874s
chaiml-gspo-glm46-combi-11090-v7-uploader: creating bucket guanaco-vllm-models
chaiml-gspo-glm46-combi-11090-v7-uploader: /usr/lib/python3/dist-packages/S3/BaseUtils.py:56: SyntaxWarning: invalid escape sequence '\.'
chaiml-gspo-glm46-combi-11090-v7-uploader:   RE_S3_DATESTRING = re.compile('\.[0-9]*(?:[Z\\-\\+]*?)')
chaiml-gspo-glm46-combi-11090-v7-uploader: /usr/lib/python3/dist-packages/S3/BaseUtils.py:57: SyntaxWarning: invalid escape sequence '\s'
chaiml-gspo-glm46-combi-11090-v7-uploader:   RE_XML_NAMESPACE = re.compile(b'^(<?[^>]+?>\s*|\s*)(<\w+) xmlns=[\'"](https?://[^\'"]+)[\'"]', re.MULTILINE)
chaiml-gspo-glm46-combi-11090-v7-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:240: SyntaxWarning: invalid escape sequence '\.'
chaiml-gspo-glm46-combi-11090-v7-uploader:   invalid = re.search("([^a-z0-9\.-])", bucket, re.UNICODE)
chaiml-gspo-glm46-combi-11090-v7-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:244: SyntaxWarning: invalid escape sequence '\.'
chaiml-grpo-glm46-chai-81744-v7-uploader: Downloaded in 135.607s
chaiml-gspo-glm46-combi-11090-v7-uploader:   invalid = re.search("([^A-Za-z0-9\._-])", bucket, re.UNICODE)
chaiml-grpo-glm46-chai-81744-v7-uploader: Processed model ChaiML/grpo-glm46-chai-rm-lr5e6-step150 in 139.060s
chaiml-gspo-glm46-combi-11090-v7-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:255: SyntaxWarning: invalid escape sequence '\.'
chaiml-gspo-glm46-combi-11090-v7-uploader:   if re.search("-\.", bucket, re.UNICODE):
chaiml-gspo-glm46-combi-11090-v7-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:257: SyntaxWarning: invalid escape sequence '\.'
chaiml-gspo-glm46-combi-11090-v7-uploader:   if re.search("\.\.", bucket, re.UNICODE):
chaiml-gspo-glm46-combi-11090-v7-uploader: /usr/lib/python3/dist-packages/S3/S3Uri.py:155: SyntaxWarning: invalid escape sequence '\w'
chaiml-gspo-glm46-combi-11090-v7-uploader:   _re = re.compile("^(\w+://)?(.*)", re.UNICODE)
chaiml-gspo-glm46-combi-11090-v7-uploader: /usr/lib/python3/dist-packages/S3/FileLists.py:480: SyntaxWarning: invalid escape sequence '\*'
chaiml-gspo-glm46-combi-11090-v7-uploader:   wildcard_split_result = re.split("\*|\?", uri_str, maxsplit=1)
chaiml-gspo-glm46-combi-11090-v7-uploader: Bucket 's3://guanaco-vllm-models/' created
chaiml-gspo-glm46-combi-11090-v7-uploader: uploading /dev/shm/model_output to s3://guanaco-vllm-models/chaiml-gspo-glm46-combi-11090-v7/default
chaiml-gspo-glm46-combi-11090-v7-uploader: DEBUG "sync /dev/shm/model_output/.gitattributes s3://guanaco-vllm-models/chaiml-gspo-glm46-combi-11090-v7/default/.gitattributes": object size matches
chaiml-gspo-glm46-combi-11090-v7-uploader: DEBUG "sync /dev/shm/model_output/chat_template.jinja s3://guanaco-vllm-models/chaiml-gspo-glm46-combi-11090-v7/default/chat_template.jinja": object size matches
chaiml-gspo-glm46-combi-11090-v7-uploader: DEBUG "sync /dev/shm/model_output/config.json s3://guanaco-vllm-models/chaiml-gspo-glm46-combi-11090-v7/default/config.json": object size matches
chaiml-gspo-glm46-combi-11090-v7-uploader: DEBUG "sync /dev/shm/model_output/generation_config.json s3://guanaco-vllm-models/chaiml-gspo-glm46-combi-11090-v7/default/generation_config.json": object size matches
chaiml-gspo-glm46-combi-11090-v7-uploader: DEBUG "sync /dev/shm/model_output/model-00001-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm46-combi-11090-v7/default/model-00001-of-00072.safetensors": object size matches
chaiml-gspo-glm46-combi-11090-v7-uploader: DEBUG "sync /dev/shm/model_output/model-00002-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm46-combi-11090-v7/default/model-00002-of-00072.safetensors": object size matches
chaiml-gspo-glm46-combi-11090-v7-uploader: DEBUG "sync /dev/shm/model_output/model-00003-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm46-combi-11090-v7/default/model-00003-of-00072.safetensors": object size matches
chaiml-gspo-glm46-combi-11090-v7-uploader: DEBUG "sync /dev/shm/model_output/model-00004-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm46-combi-11090-v7/default/model-00004-of-00072.safetensors": object size matches
chaiml-gspo-glm46-combi-11090-v7-uploader: DEBUG "sync /dev/shm/model_output/model-00005-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm46-combi-11090-v7/default/model-00005-of-00072.safetensors": object size matches
chaiml-gspo-glm46-combi-11090-v7-uploader: DEBUG "sync /dev/shm/model_output/model-00006-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm46-combi-11090-v7/default/model-00006-of-00072.safetensors": object size matches
chaiml-gspo-glm46-combi-11090-v7-uploader: DEBUG "sync /dev/shm/model_output/model-00007-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm46-combi-11090-v7/default/model-00007-of-00072.safetensors": object size matches
chaiml-gspo-glm46-combi-11090-v7-uploader: DEBUG "sync /dev/shm/model_output/model-00008-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm46-combi-11090-v7/default/model-00008-of-00072.safetensors": object size matches
chaiml-gspo-glm46-combi-11090-v7-uploader: DEBUG "sync /dev/shm/model_output/model-00009-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm46-combi-11090-v7/default/model-00009-of-00072.safetensors": object size matches
chaiml-gspo-glm46-combi-11090-v7-uploader: DEBUG "sync /dev/shm/model_output/model-00010-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm46-combi-11090-v7/default/model-00010-of-00072.safetensors": object size matches
chaiml-gspo-glm46-combi-11090-v7-uploader: DEBUG "sync /dev/shm/model_output/model-00011-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm46-combi-11090-v7/default/model-00011-of-00072.safetensors": object size matches
chaiml-gspo-glm46-combi-11090-v7-uploader: DEBUG "sync /dev/shm/model_output/model-00012-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm46-combi-11090-v7/default/model-00012-of-00072.safetensors": object size matches
chaiml-gspo-glm46-combi-11090-v7-uploader: DEBUG "sync /dev/shm/model_output/model-00013-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm46-combi-11090-v7/default/model-00013-of-00072.safetensors": object size matches
chaiml-gspo-glm46-combi-11090-v7-uploader: DEBUG "sync /dev/shm/model_output/model-00014-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm46-combi-11090-v7/default/model-00014-of-00072.safetensors": object size matches
chaiml-gspo-glm46-combi-11090-v7-uploader: DEBUG "sync /dev/shm/model_output/model-00015-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm46-combi-11090-v7/default/model-00015-of-00072.safetensors": object size matches
chaiml-gspo-glm46-combi-11090-v7-uploader: DEBUG "sync /dev/shm/model_output/model-00016-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm46-combi-11090-v7/default/model-00016-of-00072.safetensors": object size matches
chaiml-gspo-glm46-combi-11090-v7-uploader: DEBUG "sync /dev/shm/model_output/model-00017-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm46-combi-11090-v7/default/model-00017-of-00072.safetensors": object size matches
chaiml-gspo-glm46-combi-11090-v7-uploader: DEBUG "sync /dev/shm/model_output/model-00018-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm46-combi-11090-v7/default/model-00018-of-00072.safetensors": object size matches
chaiml-gspo-glm46-combi-11090-v7-uploader: DEBUG "sync /dev/shm/model_output/model-00019-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm46-combi-11090-v7/default/model-00019-of-00072.safetensors": object size matches
chaiml-gspo-glm46-combi-11090-v7-uploader: DEBUG "sync /dev/shm/model_output/model-00020-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm46-combi-11090-v7/default/model-00020-of-00072.safetensors": object size matches
chaiml-gspo-glm46-combi-11090-v7-uploader: DEBUG "sync /dev/shm/model_output/model-00021-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm46-combi-11090-v7/default/model-00021-of-00072.safetensors": object size matches
chaiml-gspo-glm46-combi-11090-v7-uploader: DEBUG "sync /dev/shm/model_output/model-00022-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm46-combi-11090-v7/default/model-00022-of-00072.safetensors": object size matches
chaiml-gspo-glm46-combi-11090-v7-uploader: DEBUG "sync /dev/shm/model_output/model-00023-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm46-combi-11090-v7/default/model-00023-of-00072.safetensors": object size matches
chaiml-gspo-glm46-combi-11090-v7-uploader: DEBUG "sync /dev/shm/model_output/model-00024-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm46-combi-11090-v7/default/model-00024-of-00072.safetensors": object size matches
chaiml-gspo-glm46-combi-11090-v7-uploader: DEBUG "sync /dev/shm/model_output/model-00025-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm46-combi-11090-v7/default/model-00025-of-00072.safetensors": object size matches
chaiml-gspo-glm46-combi-11090-v7-uploader: DEBUG "sync /dev/shm/model_output/model-00026-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm46-combi-11090-v7/default/model-00026-of-00072.safetensors": object size matches
chaiml-gspo-glm46-combi-11090-v7-uploader: DEBUG "sync /dev/shm/model_output/model-00027-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm46-combi-11090-v7/default/model-00027-of-00072.safetensors": object size matches
chaiml-gspo-glm46-combi-11090-v7-uploader: DEBUG "sync /dev/shm/model_output/model-00028-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm46-combi-11090-v7/default/model-00028-of-00072.safetensors": object size matches
chaiml-gspo-glm46-combi-11090-v7-uploader: DEBUG "sync /dev/shm/model_output/model-00029-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm46-combi-11090-v7/default/model-00029-of-00072.safetensors": object size matches
chaiml-gspo-glm46-combi-11090-v7-uploader: DEBUG "sync /dev/shm/model_output/model-00030-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm46-combi-11090-v7/default/model-00030-of-00072.safetensors": object size matches
chaiml-gspo-glm46-combi-11090-v7-uploader: DEBUG "sync /dev/shm/model_output/model-00031-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm46-combi-11090-v7/default/model-00031-of-00072.safetensors": object size matches
chaiml-gspo-glm46-combi-11090-v7-uploader: DEBUG "sync /dev/shm/model_output/model-00032-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm46-combi-11090-v7/default/model-00032-of-00072.safetensors": object size matches
chaiml-gspo-glm46-combi-11090-v7-uploader: DEBUG "sync /dev/shm/model_output/model-00033-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm46-combi-11090-v7/default/model-00033-of-00072.safetensors": object size matches
chaiml-gspo-glm46-combi-11090-v7-uploader: DEBUG "sync /dev/shm/model_output/model-00034-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm46-combi-11090-v7/default/model-00034-of-00072.safetensors": object size matches
chaiml-gspo-glm46-combi-11090-v7-uploader: DEBUG "sync /dev/shm/model_output/model-00035-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm46-combi-11090-v7/default/model-00035-of-00072.safetensors": object size matches
chaiml-gspo-glm46-combi-11090-v7-uploader: DEBUG "sync /dev/shm/model_output/model-00036-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm46-combi-11090-v7/default/model-00036-of-00072.safetensors": object size matches
chaiml-gspo-glm46-combi-11090-v7-uploader: DEBUG "sync /dev/shm/model_output/model-00037-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm46-combi-11090-v7/default/model-00037-of-00072.safetensors": object size matches
chaiml-gspo-glm46-combi-11090-v7-uploader: DEBUG "sync /dev/shm/model_output/model-00038-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm46-combi-11090-v7/default/model-00038-of-00072.safetensors": object size matches
chaiml-gspo-glm46-combi-11090-v7-uploader: DEBUG "sync /dev/shm/model_output/model-00039-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm46-combi-11090-v7/default/model-00039-of-00072.safetensors": object size matches
chaiml-gspo-glm46-combi-11090-v7-uploader: DEBUG "sync /dev/shm/model_output/model-00040-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm46-combi-11090-v7/default/model-00040-of-00072.safetensors": object size matches
chaiml-gspo-glm46-combi-11090-v7-uploader: DEBUG "sync /dev/shm/model_output/model-00041-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm46-combi-11090-v7/default/model-00041-of-00072.safetensors": object size matches
chaiml-gspo-glm46-combi-11090-v7-uploader: DEBUG "sync /dev/shm/model_output/model-00042-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm46-combi-11090-v7/default/model-00042-of-00072.safetensors": object size matches
chaiml-gspo-glm46-combi-11090-v7-uploader: DEBUG "sync /dev/shm/model_output/model-00043-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm46-combi-11090-v7/default/model-00043-of-00072.safetensors": object size matches
chaiml-gspo-glm46-combi-11090-v7-uploader: DEBUG "sync /dev/shm/model_output/model-00044-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm46-combi-11090-v7/default/model-00044-of-00072.safetensors": object size matches
chaiml-gspo-glm46-combi-11090-v7-uploader: DEBUG "sync /dev/shm/model_output/model-00045-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm46-combi-11090-v7/default/model-00045-of-00072.safetensors": object size matches
chaiml-grpo-glm46-chai-81744-v7-uploader: creating bucket guanaco-vllm-models
chaiml-gspo-glm46-combi-11090-v7-uploader: DEBUG "sync /dev/shm/model_output/model-00046-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm46-combi-11090-v7/default/model-00046-of-00072.safetensors": object size matches
chaiml-grpo-glm46-chai-81744-v7-uploader: /usr/lib/python3/dist-packages/S3/BaseUtils.py:56: SyntaxWarning: invalid escape sequence '\.'
chaiml-gspo-glm46-combi-11090-v7-uploader: DEBUG "sync /dev/shm/model_output/model-00047-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm46-combi-11090-v7/default/model-00047-of-00072.safetensors": object size matches
chaiml-grpo-glm46-chai-81744-v7-uploader:   RE_S3_DATESTRING = re.compile('\.[0-9]*(?:[Z\\-\\+]*?)')
chaiml-gspo-glm46-combi-11090-v7-uploader: DEBUG "sync /dev/shm/model_output/model-00048-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm46-combi-11090-v7/default/model-00048-of-00072.safetensors": object size matches
chaiml-grpo-glm46-chai-81744-v7-uploader: /usr/lib/python3/dist-packages/S3/BaseUtils.py:57: SyntaxWarning: invalid escape sequence '\s'
chaiml-gspo-glm46-combi-11090-v7-uploader: DEBUG "sync /dev/shm/model_output/model-00049-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm46-combi-11090-v7/default/model-00049-of-00072.safetensors": object size matches
chaiml-grpo-glm46-chai-81744-v7-uploader:   RE_XML_NAMESPACE = re.compile(b'^(<?[^>]+?>\s*|\s*)(<\w+) xmlns=[\'"](https?://[^\'"]+)[\'"]', re.MULTILINE)
chaiml-gspo-glm46-combi-11090-v7-uploader: DEBUG "sync /dev/shm/model_output/model-00050-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm46-combi-11090-v7/default/model-00050-of-00072.safetensors": object size matches
chaiml-grpo-glm46-chai-81744-v7-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:240: SyntaxWarning: invalid escape sequence '\.'
chaiml-gspo-glm46-combi-11090-v7-uploader: DEBUG "sync /dev/shm/model_output/model-00051-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm46-combi-11090-v7/default/model-00051-of-00072.safetensors": object size matches
chaiml-grpo-glm46-chai-81744-v7-uploader:   invalid = re.search("([^a-z0-9\.-])", bucket, re.UNICODE)
chaiml-gspo-glm46-combi-11090-v7-uploader: DEBUG "sync /dev/shm/model_output/model-00052-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm46-combi-11090-v7/default/model-00052-of-00072.safetensors": object size matches
chaiml-grpo-glm46-chai-81744-v7-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:244: SyntaxWarning: invalid escape sequence '\.'
chaiml-gspo-glm46-combi-11090-v7-uploader: DEBUG "sync /dev/shm/model_output/model-00053-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm46-combi-11090-v7/default/model-00053-of-00072.safetensors": object size matches
chaiml-grpo-glm46-chai-81744-v7-uploader:   invalid = re.search("([^A-Za-z0-9\._-])", bucket, re.UNICODE)
chaiml-gspo-glm46-combi-11090-v7-uploader: DEBUG "sync /dev/shm/model_output/model-00054-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm46-combi-11090-v7/default/model-00054-of-00072.safetensors": object size matches
chaiml-grpo-glm46-chai-81744-v7-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:255: SyntaxWarning: invalid escape sequence '\.'
chaiml-gspo-glm46-combi-11090-v7-uploader: DEBUG "sync /dev/shm/model_output/model-00055-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm46-combi-11090-v7/default/model-00055-of-00072.safetensors": object size matches
chaiml-grpo-glm46-chai-81744-v7-uploader:   if re.search("-\.", bucket, re.UNICODE):
chaiml-gspo-glm46-combi-11090-v7-uploader: DEBUG "sync /dev/shm/model_output/model-00056-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm46-combi-11090-v7/default/model-00056-of-00072.safetensors": object size matches
chaiml-grpo-glm46-chai-81744-v7-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:257: SyntaxWarning: invalid escape sequence '\.'
chaiml-gspo-glm46-combi-11090-v7-uploader: DEBUG "sync /dev/shm/model_output/model-00057-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm46-combi-11090-v7/default/model-00057-of-00072.safetensors": object size matches
chaiml-grpo-glm46-chai-81744-v7-uploader:   if re.search("\.\.", bucket, re.UNICODE):
chaiml-gspo-glm46-combi-11090-v7-uploader: DEBUG "sync /dev/shm/model_output/model-00058-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm46-combi-11090-v7/default/model-00058-of-00072.safetensors": object size matches
chaiml-grpo-glm46-chai-81744-v7-uploader: /usr/lib/python3/dist-packages/S3/S3Uri.py:155: SyntaxWarning: invalid escape sequence '\w'
chaiml-gspo-glm46-combi-11090-v7-uploader: DEBUG "sync /dev/shm/model_output/model-00059-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm46-combi-11090-v7/default/model-00059-of-00072.safetensors": object size matches
chaiml-grpo-glm46-chai-81744-v7-uploader:   _re = re.compile("^(\w+://)?(.*)", re.UNICODE)
chaiml-gspo-glm46-combi-11090-v7-uploader: DEBUG "sync /dev/shm/model_output/model-00060-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm46-combi-11090-v7/default/model-00060-of-00072.safetensors": object size matches
chaiml-grpo-glm46-chai-81744-v7-uploader: /usr/lib/python3/dist-packages/S3/FileLists.py:480: SyntaxWarning: invalid escape sequence '\*'
chaiml-gspo-glm46-combi-11090-v7-uploader: DEBUG "sync /dev/shm/model_output/model-00061-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm46-combi-11090-v7/default/model-00061-of-00072.safetensors": object size matches
chaiml-grpo-glm46-chai-81744-v7-uploader:   wildcard_split_result = re.split("\*|\?", uri_str, maxsplit=1)
chaiml-gspo-glm46-combi-11090-v7-uploader: DEBUG "sync /dev/shm/model_output/model-00062-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm46-combi-11090-v7/default/model-00062-of-00072.safetensors": object size matches
chaiml-grpo-glm46-chai-81744-v7-uploader: Bucket 's3://guanaco-vllm-models/' created
chaiml-gspo-glm46-combi-11090-v7-uploader: DEBUG "sync /dev/shm/model_output/model-00063-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm46-combi-11090-v7/default/model-00063-of-00072.safetensors": object size matches
chaiml-grpo-glm46-chai-81744-v7-uploader: uploading /dev/shm/model_output to s3://guanaco-vllm-models/chaiml-grpo-glm46-chai-81744-v7/default
chaiml-gspo-glm46-combi-11090-v7-uploader: DEBUG "sync /dev/shm/model_output/model-00064-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm46-combi-11090-v7/default/model-00064-of-00072.safetensors": object size matches
chaiml-grpo-glm46-chai-81744-v7-uploader: DEBUG "sync /dev/shm/model_output/.gitattributes s3://guanaco-vllm-models/chaiml-grpo-glm46-chai-81744-v7/default/.gitattributes": object size matches
chaiml-gspo-glm46-combi-11090-v7-uploader: DEBUG "sync /dev/shm/model_output/model-00065-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm46-combi-11090-v7/default/model-00065-of-00072.safetensors": object size matches
chaiml-grpo-glm46-chai-81744-v7-uploader: DEBUG "sync /dev/shm/model_output/chat_template.jinja s3://guanaco-vllm-models/chaiml-grpo-glm46-chai-81744-v7/default/chat_template.jinja": object size matches
chaiml-gspo-glm46-combi-11090-v7-uploader: DEBUG "sync /dev/shm/model_output/model-00066-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm46-combi-11090-v7/default/model-00066-of-00072.safetensors": object size matches
chaiml-grpo-glm46-chai-81744-v7-uploader: DEBUG "sync /dev/shm/model_output/config.json s3://guanaco-vllm-models/chaiml-grpo-glm46-chai-81744-v7/default/config.json": object size matches
chaiml-gspo-glm46-combi-11090-v7-uploader: DEBUG "sync /dev/shm/model_output/model-00067-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm46-combi-11090-v7/default/model-00067-of-00072.safetensors": object size matches
chaiml-grpo-glm46-chai-81744-v7-uploader: DEBUG "sync /dev/shm/model_output/generation_config.json s3://guanaco-vllm-models/chaiml-grpo-glm46-chai-81744-v7/default/generation_config.json": object size matches
chaiml-gspo-glm46-combi-11090-v7-uploader: DEBUG "sync /dev/shm/model_output/model-00068-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm46-combi-11090-v7/default/model-00068-of-00072.safetensors": object size matches
chaiml-grpo-glm46-chai-81744-v7-uploader: DEBUG "sync /dev/shm/model_output/model-00001-of-00072.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm46-chai-81744-v7/default/model-00001-of-00072.safetensors": object size matches
chaiml-gspo-glm46-combi-11090-v7-uploader: DEBUG "sync /dev/shm/model_output/model-00069-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm46-combi-11090-v7/default/model-00069-of-00072.safetensors": object size matches
chaiml-grpo-glm46-chai-81744-v7-uploader: DEBUG "sync /dev/shm/model_output/model-00002-of-00072.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm46-chai-81744-v7/default/model-00002-of-00072.safetensors": object size matches
chaiml-gspo-glm46-combi-11090-v7-uploader: DEBUG "sync /dev/shm/model_output/model-00070-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm46-combi-11090-v7/default/model-00070-of-00072.safetensors": object size matches
chaiml-grpo-glm46-chai-81744-v7-uploader: DEBUG "sync /dev/shm/model_output/model-00003-of-00072.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm46-chai-81744-v7/default/model-00003-of-00072.safetensors": object size matches
chaiml-gspo-glm46-combi-11090-v7-uploader: DEBUG "sync /dev/shm/model_output/model-00071-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm46-combi-11090-v7/default/model-00071-of-00072.safetensors": object size matches
chaiml-grpo-glm46-chai-81744-v7-uploader: DEBUG "sync /dev/shm/model_output/model-00004-of-00072.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm46-chai-81744-v7/default/model-00004-of-00072.safetensors": object size matches
chaiml-gspo-glm46-combi-11090-v7-uploader: DEBUG "sync /dev/shm/model_output/model-00072-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm46-combi-11090-v7/default/model-00072-of-00072.safetensors": object size matches
chaiml-grpo-glm46-chai-81744-v7-uploader: DEBUG "sync /dev/shm/model_output/model-00005-of-00072.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm46-chai-81744-v7/default/model-00005-of-00072.safetensors": object size matches
chaiml-gspo-glm46-combi-11090-v7-uploader: DEBUG "sync /dev/shm/model_output/model.safetensors.index.json s3://guanaco-vllm-models/chaiml-gspo-glm46-combi-11090-v7/default/model.safetensors.index.json": object size matches
chaiml-grpo-glm46-chai-81744-v7-uploader: DEBUG "sync /dev/shm/model_output/model-00006-of-00072.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm46-chai-81744-v7/default/model-00006-of-00072.safetensors": object size matches
chaiml-gspo-glm46-combi-11090-v7-uploader: DEBUG "sync /dev/shm/model_output/recipe.yaml s3://guanaco-vllm-models/chaiml-gspo-glm46-combi-11090-v7/default/recipe.yaml": object size matches
chaiml-grpo-glm46-chai-81744-v7-uploader: DEBUG "sync /dev/shm/model_output/model-00007-of-00072.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm46-chai-81744-v7/default/model-00007-of-00072.safetensors": object size matches
chaiml-gspo-glm46-combi-11090-v7-uploader: DEBUG "sync /dev/shm/model_output/special_tokens_map.json s3://guanaco-vllm-models/chaiml-gspo-glm46-combi-11090-v7/default/special_tokens_map.json": object size matches
chaiml-grpo-glm46-chai-81744-v7-uploader: DEBUG "sync /dev/shm/model_output/model-00008-of-00072.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm46-chai-81744-v7/default/model-00008-of-00072.safetensors": object size matches
chaiml-gspo-glm46-combi-11090-v7-uploader: DEBUG "sync /dev/shm/model_output/tokenizer.json s3://guanaco-vllm-models/chaiml-gspo-glm46-combi-11090-v7/default/tokenizer.json": object size matches
chaiml-grpo-glm46-chai-81744-v7-uploader: DEBUG "sync /dev/shm/model_output/model-00009-of-00072.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm46-chai-81744-v7/default/model-00009-of-00072.safetensors": object size matches
chaiml-gspo-glm46-combi-11090-v7-uploader: DEBUG "sync /dev/shm/model_output/tokenizer_config.json s3://guanaco-vllm-models/chaiml-gspo-glm46-combi-11090-v7/default/tokenizer_config.json": object size matches
chaiml-grpo-glm46-chai-81744-v7-uploader: DEBUG "sync /dev/shm/model_output/model-00010-of-00072.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm46-chai-81744-v7/default/model-00010-of-00072.safetensors": object size matches
chaiml-grpo-glm46-chai-81744-v7-uploader: DEBUG "sync /dev/shm/model_output/model-00011-of-00072.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm46-chai-81744-v7/default/model-00011-of-00072.safetensors": object size matches
chaiml-grpo-glm46-chai-81744-v7-uploader: DEBUG "sync /dev/shm/model_output/model-00012-of-00072.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm46-chai-81744-v7/default/model-00012-of-00072.safetensors": object size matches
chaiml-grpo-glm46-chai-81744-v7-uploader: DEBUG "sync /dev/shm/model_output/model-00013-of-00072.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm46-chai-81744-v7/default/model-00013-of-00072.safetensors": object size matches
chaiml-grpo-glm46-chai-81744-v7-uploader: DEBUG "sync /dev/shm/model_output/model-00014-of-00072.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm46-chai-81744-v7/default/model-00014-of-00072.safetensors": object size matches
chaiml-grpo-glm46-chai-81744-v7-uploader: DEBUG "sync /dev/shm/model_output/model-00015-of-00072.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm46-chai-81744-v7/default/model-00015-of-00072.safetensors": object size matches
chaiml-grpo-glm46-chai-81744-v7-uploader: DEBUG "sync /dev/shm/model_output/model-00016-of-00072.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm46-chai-81744-v7/default/model-00016-of-00072.safetensors": object size matches
chaiml-grpo-glm46-chai-81744-v7-uploader: DEBUG "sync /dev/shm/model_output/model-00017-of-00072.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm46-chai-81744-v7/default/model-00017-of-00072.safetensors": object size matches
chaiml-grpo-glm46-chai-81744-v7-uploader: DEBUG "sync /dev/shm/model_output/model-00018-of-00072.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm46-chai-81744-v7/default/model-00018-of-00072.safetensors": object size matches
chaiml-grpo-glm46-chai-81744-v7-uploader: DEBUG "sync /dev/shm/model_output/model-00019-of-00072.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm46-chai-81744-v7/default/model-00019-of-00072.safetensors": object size matches
chaiml-grpo-glm46-chai-81744-v7-uploader: DEBUG "sync /dev/shm/model_output/model-00020-of-00072.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm46-chai-81744-v7/default/model-00020-of-00072.safetensors": object size matches
chaiml-grpo-glm46-chai-81744-v7-uploader: DEBUG "sync /dev/shm/model_output/model-00021-of-00072.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm46-chai-81744-v7/default/model-00021-of-00072.safetensors": object size matches
chaiml-grpo-glm46-chai-81744-v7-uploader: DEBUG "sync /dev/shm/model_output/model-00022-of-00072.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm46-chai-81744-v7/default/model-00022-of-00072.safetensors": object size matches
chaiml-grpo-glm46-chai-81744-v7-uploader: DEBUG "sync /dev/shm/model_output/model-00023-of-00072.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm46-chai-81744-v7/default/model-00023-of-00072.safetensors": object size matches
chaiml-grpo-glm46-chai-81744-v7-uploader: DEBUG "sync /dev/shm/model_output/model-00024-of-00072.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm46-chai-81744-v7/default/model-00024-of-00072.safetensors": object size matches
chaiml-grpo-glm46-chai-81744-v7-uploader: DEBUG "sync /dev/shm/model_output/model-00025-of-00072.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm46-chai-81744-v7/default/model-00025-of-00072.safetensors": object size matches
chaiml-grpo-glm46-chai-81744-v7-uploader: DEBUG "sync /dev/shm/model_output/model-00026-of-00072.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm46-chai-81744-v7/default/model-00026-of-00072.safetensors": object size matches
chaiml-grpo-glm46-chai-81744-v7-uploader: DEBUG "sync /dev/shm/model_output/model-00027-of-00072.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm46-chai-81744-v7/default/model-00027-of-00072.safetensors": object size matches
chaiml-grpo-glm46-chai-81744-v7-uploader: DEBUG "sync /dev/shm/model_output/model-00028-of-00072.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm46-chai-81744-v7/default/model-00028-of-00072.safetensors": object size matches
chaiml-grpo-glm46-chai-81744-v7-uploader: DEBUG "sync /dev/shm/model_output/model-00029-of-00072.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm46-chai-81744-v7/default/model-00029-of-00072.safetensors": object size matches
chaiml-grpo-glm46-chai-81744-v7-uploader: DEBUG "sync /dev/shm/model_output/model-00030-of-00072.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm46-chai-81744-v7/default/model-00030-of-00072.safetensors": object size matches
chaiml-grpo-glm46-chai-81744-v7-uploader: DEBUG "sync /dev/shm/model_output/model-00031-of-00072.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm46-chai-81744-v7/default/model-00031-of-00072.safetensors": object size matches
chaiml-grpo-glm46-chai-81744-v7-uploader: DEBUG "sync /dev/shm/model_output/model-00032-of-00072.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm46-chai-81744-v7/default/model-00032-of-00072.safetensors": object size matches
chaiml-grpo-glm46-chai-81744-v7-uploader: DEBUG "sync /dev/shm/model_output/model-00033-of-00072.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm46-chai-81744-v7/default/model-00033-of-00072.safetensors": object size matches
chaiml-grpo-glm46-chai-81744-v7-uploader: DEBUG "sync /dev/shm/model_output/model-00034-of-00072.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm46-chai-81744-v7/default/model-00034-of-00072.safetensors": object size matches
chaiml-grpo-glm46-chai-81744-v7-uploader: DEBUG "sync /dev/shm/model_output/model-00035-of-00072.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm46-chai-81744-v7/default/model-00035-of-00072.safetensors": object size matches
chaiml-grpo-glm46-chai-81744-v7-uploader: DEBUG "sync /dev/shm/model_output/model-00036-of-00072.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm46-chai-81744-v7/default/model-00036-of-00072.safetensors": object size matches
chaiml-grpo-glm46-chai-81744-v7-uploader: DEBUG "sync /dev/shm/model_output/model-00037-of-00072.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm46-chai-81744-v7/default/model-00037-of-00072.safetensors": object size matches
chaiml-grpo-glm46-chai-81744-v7-uploader: DEBUG "sync /dev/shm/model_output/model-00038-of-00072.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm46-chai-81744-v7/default/model-00038-of-00072.safetensors": object size matches
chaiml-grpo-glm46-chai-81744-v7-uploader: DEBUG "sync /dev/shm/model_output/model-00039-of-00072.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm46-chai-81744-v7/default/model-00039-of-00072.safetensors": object size matches
chaiml-grpo-glm46-chai-81744-v7-uploader: DEBUG "sync /dev/shm/model_output/model-00040-of-00072.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm46-chai-81744-v7/default/model-00040-of-00072.safetensors": object size matches
chaiml-grpo-glm46-chai-81744-v7-uploader: DEBUG "sync /dev/shm/model_output/model-00041-of-00072.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm46-chai-81744-v7/default/model-00041-of-00072.safetensors": object size matches
chaiml-grpo-glm46-chai-81744-v7-uploader: DEBUG "sync /dev/shm/model_output/model-00042-of-00072.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm46-chai-81744-v7/default/model-00042-of-00072.safetensors": object size matches
chaiml-grpo-glm46-chai-81744-v7-uploader: DEBUG "sync /dev/shm/model_output/model-00043-of-00072.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm46-chai-81744-v7/default/model-00043-of-00072.safetensors": object size matches
chaiml-grpo-glm46-chai-81744-v7-uploader: DEBUG "sync /dev/shm/model_output/model-00044-of-00072.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm46-chai-81744-v7/default/model-00044-of-00072.safetensors": object size matches
chaiml-grpo-glm46-chai-81744-v7-uploader: DEBUG "sync /dev/shm/model_output/model-00045-of-00072.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm46-chai-81744-v7/default/model-00045-of-00072.safetensors": object size matches
chaiml-grpo-glm46-chai-81744-v7-uploader: DEBUG "sync /dev/shm/model_output/model-00046-of-00072.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm46-chai-81744-v7/default/model-00046-of-00072.safetensors": object size matches
chaiml-grpo-glm46-chai-81744-v7-uploader: DEBUG "sync /dev/shm/model_output/model-00047-of-00072.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm46-chai-81744-v7/default/model-00047-of-00072.safetensors": object size matches
chaiml-grpo-glm46-chai-81744-v7-uploader: DEBUG "sync /dev/shm/model_output/model-00048-of-00072.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm46-chai-81744-v7/default/model-00048-of-00072.safetensors": object size matches
chaiml-grpo-glm46-chai-81744-v7-uploader: DEBUG "sync /dev/shm/model_output/model-00049-of-00072.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm46-chai-81744-v7/default/model-00049-of-00072.safetensors": object size matches
chaiml-grpo-glm46-chai-81744-v7-uploader: DEBUG "sync /dev/shm/model_output/model-00050-of-00072.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm46-chai-81744-v7/default/model-00050-of-00072.safetensors": object size matches
chaiml-grpo-glm46-chai-81744-v7-uploader: DEBUG "sync /dev/shm/model_output/model-00051-of-00072.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm46-chai-81744-v7/default/model-00051-of-00072.safetensors": object size matches
chaiml-grpo-glm46-chai-81744-v7-uploader: DEBUG "sync /dev/shm/model_output/model-00052-of-00072.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm46-chai-81744-v7/default/model-00052-of-00072.safetensors": object size matches
Job chaiml-gspo-glm46-combi-11090-v7-uploader completed after 223.9s with status: succeeded
chaiml-grpo-glm46-chai-81744-v7-uploader: DEBUG "sync /dev/shm/model_output/model-00053-of-00072.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm46-chai-81744-v7/default/model-00053-of-00072.safetensors": object size matches
Stopping job with name chaiml-gspo-glm46-combi-11090-v7-uploader
chaiml-grpo-glm46-chai-81744-v7-uploader: DEBUG "sync /dev/shm/model_output/model-00054-of-00072.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm46-chai-81744-v7/default/model-00054-of-00072.safetensors": object size matches
Pipeline stage VLLMUploader completed in 225.49s
chaiml-grpo-glm46-chai-81744-v7-uploader: DEBUG "sync /dev/shm/model_output/model-00055-of-00072.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm46-chai-81744-v7/default/model-00055-of-00072.safetensors": object size matches
run pipeline stage %s
chaiml-grpo-glm46-chai-81744-v7-uploader: DEBUG "sync /dev/shm/model_output/model-00056-of-00072.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm46-chai-81744-v7/default/model-00056-of-00072.safetensors": object size matches
Running pipeline stage VLLMTemplater
chaiml-grpo-glm46-chai-81744-v7-uploader: DEBUG "sync /dev/shm/model_output/model-00057-of-00072.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm46-chai-81744-v7/default/model-00057-of-00072.safetensors": object size matches
chaiml-grpo-glm46-chai-81744-v7-uploader: DEBUG "sync /dev/shm/model_output/model-00058-of-00072.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm46-chai-81744-v7/default/model-00058-of-00072.safetensors": object size matches
chaiml-grpo-glm46-chai-81744-v7-uploader: DEBUG "sync /dev/shm/model_output/model-00059-of-00072.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm46-chai-81744-v7/default/model-00059-of-00072.safetensors": object size matches
chaiml-grpo-glm46-chai-81744-v7-uploader: DEBUG "sync /dev/shm/model_output/model-00060-of-00072.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm46-chai-81744-v7/default/model-00060-of-00072.safetensors": object size matches
Pipeline stage VLLMTemplater completed in 0.87s
chaiml-grpo-glm46-chai-81744-v7-uploader: DEBUG "sync /dev/shm/model_output/model-00061-of-00072.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm46-chai-81744-v7/default/model-00061-of-00072.safetensors": object size matches
run pipeline stage %s
chaiml-grpo-glm46-chai-81744-v7-uploader: DEBUG "sync /dev/shm/model_output/model-00062-of-00072.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm46-chai-81744-v7/default/model-00062-of-00072.safetensors": object size matches
Running pipeline stage VLLMDeployer
chaiml-grpo-glm46-chai-81744-v7-uploader: DEBUG "sync /dev/shm/model_output/model-00063-of-00072.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm46-chai-81744-v7/default/model-00063-of-00072.safetensors": object size matches
Creating inference service chaiml-gspo-glm46-combi-11090-v7
chaiml-grpo-glm46-chai-81744-v7-uploader: DEBUG "sync /dev/shm/model_output/model-00064-of-00072.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm46-chai-81744-v7/default/model-00064-of-00072.safetensors": object size matches
Waiting for inference service chaiml-gspo-glm46-combi-11090-v7 to be ready
chaiml-grpo-glm46-chai-81744-v7-uploader: DEBUG "sync /dev/shm/model_output/model-00065-of-00072.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm46-chai-81744-v7/default/model-00065-of-00072.safetensors": object size matches
chaiml-grpo-glm46-chai-81744-v7-uploader: DEBUG "sync /dev/shm/model_output/model-00066-of-00072.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm46-chai-81744-v7/default/model-00066-of-00072.safetensors": object size matches
chaiml-grpo-glm46-chai-81744-v7-uploader: DEBUG "sync /dev/shm/model_output/model-00067-of-00072.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm46-chai-81744-v7/default/model-00067-of-00072.safetensors": object size matches
chaiml-grpo-glm46-chai-81744-v7-uploader: DEBUG "sync /dev/shm/model_output/model-00068-of-00072.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm46-chai-81744-v7/default/model-00068-of-00072.safetensors": object size matches
chaiml-grpo-glm46-chai-81744-v7-uploader: DEBUG "sync /dev/shm/model_output/model-00069-of-00072.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm46-chai-81744-v7/default/model-00069-of-00072.safetensors": object size matches
chaiml-grpo-glm46-chai-81744-v7-uploader: DEBUG "sync /dev/shm/model_output/model-00070-of-00072.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm46-chai-81744-v7/default/model-00070-of-00072.safetensors": object size matches
chaiml-grpo-glm46-chai-81744-v7-uploader: DEBUG "sync /dev/shm/model_output/model-00071-of-00072.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm46-chai-81744-v7/default/model-00071-of-00072.safetensors": object size matches
chaiml-grpo-glm46-chai-81744-v7-uploader: DEBUG "sync /dev/shm/model_output/model-00072-of-00072.safetensors s3://guanaco-vllm-models/chaiml-grpo-glm46-chai-81744-v7/default/model-00072-of-00072.safetensors": object size matches
chaiml-grpo-glm46-chai-81744-v7-uploader: DEBUG "sync /dev/shm/model_output/model.safetensors.index.json s3://guanaco-vllm-models/chaiml-grpo-glm46-chai-81744-v7/default/model.safetensors.index.json": object size matches
chaiml-grpo-glm46-chai-81744-v7-uploader: DEBUG "sync /dev/shm/model_output/recipe.yaml s3://guanaco-vllm-models/chaiml-grpo-glm46-chai-81744-v7/default/recipe.yaml": object size matches
chaiml-grpo-glm46-chai-81744-v7-uploader: DEBUG "sync /dev/shm/model_output/special_tokens_map.json s3://guanaco-vllm-models/chaiml-grpo-glm46-chai-81744-v7/default/special_tokens_map.json": object size matches
chaiml-grpo-glm46-chai-81744-v7-uploader: DEBUG "sync /dev/shm/model_output/tokenizer.json s3://guanaco-vllm-models/chaiml-grpo-glm46-chai-81744-v7/default/tokenizer.json": object size matches
chaiml-grpo-glm46-chai-81744-v7-uploader: DEBUG "sync /dev/shm/model_output/tokenizer_config.json s3://guanaco-vllm-models/chaiml-grpo-glm46-chai-81744-v7/default/tokenizer_config.json": object size matches
2026-03-21T15:21:21.273318+00:00 monitor updated for chaiml-gspo-glm46-combi_11090_v7
Job chaiml-grpo-glm46-chai-81744-v7-uploader completed after 229.24s with status: succeeded
Stopping job with name chaiml-grpo-glm46-chai-81744-v7-uploader
Pipeline stage VLLMUploader completed in 231.00s
run pipeline stage %s
Running pipeline stage VLLMTemplater
Pipeline stage VLLMTemplater completed in 0.49s
run pipeline stage %s
Running pipeline stage VLLMDeployer
Creating inference service chaiml-grpo-glm46-chai-81744-v7
Waiting for inference service chaiml-grpo-glm46-chai-81744-v7 to be ready
2026-03-21T15:21:32.278675+00:00 monitor updated for chaiml-grpo-glm46-chai-_81744_v7
2026-03-21T15:22:21.548852+00:00 monitor updated for chaiml-gspo-glm46-combi_11090_v7
2026-03-21T15:22:32.595530+00:00 monitor updated for chaiml-grpo-glm46-chai-_81744_v7
2026-03-21T15:23:21.822177+00:00 monitor updated for chaiml-gspo-glm46-combi_11090_v7
2026-03-21T15:23:32.877964+00:00 monitor updated for chaiml-grpo-glm46-chai-_81744_v7
2026-03-21T15:24:22.138910+00:00 monitor updated for chaiml-gspo-glm46-combi_11090_v7
2026-03-21T15:24:33.125702+00:00 monitor updated for chaiml-grpo-glm46-chai-_81744_v7
2026-03-21T15:25:22.393474+00:00 monitor updated for chaiml-gspo-glm46-combi_11090_v7
2026-03-21T15:25:33.380871+00:00 monitor updated for chaiml-grpo-glm46-chai-_81744_v7
Shutdown handler not registered because Python interpreter is not running in the main thread
chaiml-grpo-glm46-chai-_81744_v7 status is now deployed due to DeploymentManager action
chaiml-grpo-glm46-chai-_81744_v7 status is now torndown due to DeploymentManager action