developer_uid: chai_backend_admin
submission_id: chaiml-gspo-glm47-combin_7698_v1
model_name: chaiml-gspo-glm47-combin_7698_v1
model_group: ChaiML/gspo-glm47-combin
status: torndown
timestamp: 2026-03-23T05:31:55+00:00
num_battles: 10284
num_wins: 5634
celo_rating: 1324.83
family_friendly_score: 0.0
family_friendly_standard_error: 0.0
submission_type: basic
model_repo: ChaiML/gspo-glm47-combine-rm82-zy-data-step100
model_architecture: Glm4MoeForCausalLM
model_num_parameters: 24110003200.0
best_of: 8
max_input_tokens: 1500
max_output_tokens: 80
reward_model: default
display_name: chaiml-gspo-glm47-combin_7698_v1
ineligible_reason: max_output_tokens!=64
is_internal_developer: True
language_model: ChaiML/gspo-glm47-combine-rm82-zy-data-step100
model_size: 24B
ranking_group: single
us_pacific_date: 2026-03-19
win_ratio: 0.5478413068844807
generation_params: {'temperature': 1.0, 'top_p': 0.95, 'min_p': 0.0, 'top_k': 60, 'presence_penalty': 0.0, 'frequency_penalty': 0.0, 'stopping_words': ['<|assistant|>', '<|im_end|>', '<|user|>', '</s>', '####'], 'max_input_tokens': 1500, 'best_of': 8, 'max_output_tokens': 80}
formatter: {'memory_template': "[gMASK]<sop><|system|>\n{bot_name}'s persona: {memory}", 'prompt_template': '', 'bot_template': '<|assistant|>\n{bot_name}: {message}', 'user_template': '<|user|>\n{message}', 'response_template': '<|assistant|>\n<think></think>\n{bot_name}:', 'truncate_by_message': True}
Resubmit model
Shutdown handler not registered because Python interpreter is not running in the main thread
run pipeline %s
run pipeline stage %s
Running pipeline stage VLLMUploader
Starting job with name chaiml-gspo-glm47-combin-7698-v1-uploader
Waiting for job on chaiml-gspo-glm47-combin-7698-v1-uploader to finish
Shutdown handler not registered because Python interpreter is not running in the main thread
run pipeline %s
run pipeline stage %s
Running pipeline stage VLLMUploader
Starting job with name chaiml-gspo-glm47-chai-41286-v1-uploader
Waiting for job on chaiml-gspo-glm47-chai-41286-v1-uploader to finish
chaiml-gspo-glm47-combin-7698-v1-uploader: Using quantization_mode: fp8
chaiml-gspo-glm47-combin-7698-v1-uploader: Checking if ChaiML/gspo-glm47-combine-rm82-zy-data-step100-FP8 already exists in ChaiML
chaiml-gspo-glm47-combin-7698-v1-uploader: Downloading snapshot of ChaiML/gspo-glm47-combine-rm82-zy-data-step100...
2026-03-19T17:44:48.120510+00:00 monitor updated for chaiml-gspo-glm47-combin_7698_v1
chaiml-gspo-glm47-chai-41286-v1-uploader: Using quantization_mode: fp8
chaiml-gspo-glm47-chai-41286-v1-uploader: Checking if ChaiML/gspo-glm47-chai-rm-zy-data-step100-FP8 already exists in ChaiML
chaiml-gspo-glm47-chai-41286-v1-uploader: Downloading snapshot of ChaiML/gspo-glm47-chai-rm-zy-data-step100...
2026-03-19T17:45:10.520842+00:00 monitor updated for chaiml-gspo-glm47-chai-_41286_v1
2026-03-19T17:45:48.255363+00:00 monitor updated for chaiml-gspo-glm47-combin_7698_v1
2026-03-19T17:46:10.665281+00:00 monitor updated for chaiml-gspo-glm47-chai-_41286_v1
2026-03-19T17:46:48.399894+00:00 monitor updated for chaiml-gspo-glm47-combin_7698_v1
2026-03-19T17:47:10.814260+00:00 monitor updated for chaiml-gspo-glm47-chai-_41286_v1
2026-03-19T17:47:48.547022+00:00 monitor updated for chaiml-gspo-glm47-combin_7698_v1
2026-03-19T17:48:10.955423+00:00 monitor updated for chaiml-gspo-glm47-chai-_41286_v1
2026-03-19T17:48:48.704322+00:00 monitor updated for chaiml-gspo-glm47-combin_7698_v1
chaiml-gspo-glm47-combin-7698-v1-uploader: Downloaded in 251.557s
chaiml-gspo-glm47-combin-7698-v1-uploader: Loading /tmp/model_input...
chaiml-gspo-glm47-combin-7698-v1-uploader: The tokenizer you are loading from '/tmp/model_input' with an incorrect regex pattern: https://huggingface.co/mistralai/Mistral-Small-3.1-24B-Instruct-2503/discussions/84#69121093e8b480e709447d5e. This will lead to incorrect tokenization. You should set the `fix_mistral_regex=True` flag when loading this tokenizer to fix this issue.
chaiml-gspo-glm47-combin-7698-v1-uploader: `torch_dtype` is deprecated! Use `dtype` instead!
chaiml-gspo-glm47-chai-41286-v1-uploader: Downloaded in 249.397s
chaiml-gspo-glm47-chai-41286-v1-uploader: Loading /tmp/model_input...
chaiml-gspo-glm47-chai-41286-v1-uploader: The tokenizer you are loading from '/tmp/model_input' with an incorrect regex pattern: https://huggingface.co/mistralai/Mistral-Small-3.1-24B-Instruct-2503/discussions/84#69121093e8b480e709447d5e. This will lead to incorrect tokenization. You should set the `fix_mistral_regex=True` flag when loading this tokenizer to fix this issue.
chaiml-gspo-glm47-chai-41286-v1-uploader: `torch_dtype` is deprecated! Use `dtype` instead!
2026-03-19T17:49:11.121276+00:00 monitor updated for chaiml-gspo-glm47-chai-_41286_v1
chaiml-gspo-glm47-combin-7698-v1-uploader: Some parameters are on the meta device because they were offloaded to the cpu.
chaiml-gspo-glm47-combin-7698-v1-uploader: Applying quantization...
chaiml-gspo-glm47-combin-7698-v1-uploader: The tokenizer you are loading from '/tmp/model_input' with an incorrect regex pattern: https://huggingface.co/mistralai/Mistral-Small-3.1-24B-Instruct-2503/discussions/84#69121093e8b480e709447d5e. This will lead to incorrect tokenization. You should set the `fix_mistral_regex=True` flag when loading this tokenizer to fix this issue.
chaiml-gspo-glm47-combin-7698-v1-uploader: 2026-03-19T10:49:41.644799-0700 | reset | INFO - Compression lifecycle reset
chaiml-gspo-glm47-combin-7698-v1-uploader: 2026-03-19T10:49:41.712298-0700 | from_modifiers | INFO - Creating recipe from modifiers
2026-03-19T17:49:48.846321+00:00 monitor updated for chaiml-gspo-glm47-combin_7698_v1
chaiml-gspo-glm47-chai-41286-v1-uploader: Some parameters are on the meta device because they were offloaded to the cpu.
chaiml-gspo-glm47-chai-41286-v1-uploader: Applying quantization...
chaiml-gspo-glm47-chai-41286-v1-uploader: The tokenizer you are loading from '/tmp/model_input' with an incorrect regex pattern: https://huggingface.co/mistralai/Mistral-Small-3.1-24B-Instruct-2503/discussions/84#69121093e8b480e709447d5e. This will lead to incorrect tokenization. You should set the `fix_mistral_regex=True` flag when loading this tokenizer to fix this issue.
chaiml-gspo-glm47-chai-41286-v1-uploader: 2026-03-19T10:49:52.017641-0700 | reset | INFO - Compression lifecycle reset
chaiml-gspo-glm47-chai-41286-v1-uploader: 2026-03-19T10:49:52.087884-0700 | from_modifiers | INFO - Creating recipe from modifiers
2026-03-19T17:50:11.261719+00:00 monitor updated for chaiml-gspo-glm47-chai-_41286_v1
chaiml-gspo-glm47-combin-7698-v1-uploader: 2026-03-19T10:50:41.551148-0700 | initialize | INFO - Compression lifecycle initialized for 1 modifiers
chaiml-gspo-glm47-combin-7698-v1-uploader: 2026-03-19T10:50:41.551636-0700 | IndependentPipeline | INFO - Inferred `DataFreePipeline` for `QuantizationModifier`
2026-03-19T17:50:48.997691+00:00 monitor updated for chaiml-gspo-glm47-combin_7698_v1
chaiml-gspo-glm47-chai-41286-v1-uploader: 2026-03-19T10:50:49.025729-0700 | initialize | INFO - Compression lifecycle initialized for 1 modifiers
chaiml-gspo-glm47-chai-41286-v1-uploader: 2026-03-19T10:50:49.026196-0700 | IndependentPipeline | INFO - Inferred `DataFreePipeline` for `QuantizationModifier`
2026-03-19T17:51:11.396357+00:00 monitor updated for chaiml-gspo-glm47-chai-_41286_v1
chaiml-gspo-glm47-combin-7698-v1-uploader: Some parameters are on the meta device because they were offloaded to the cpu.
chaiml-gspo-glm47-chai-41286-v1-uploader: Some parameters are on the meta device because they were offloaded to the cpu.
2026-03-19T17:51:49.143179+00:00 monitor updated for chaiml-gspo-glm47-combin_7698_v1
2026-03-19T17:52:11.539646+00:00 monitor updated for chaiml-gspo-glm47-chai-_41286_v1
2026-03-19T17:52:49.287623+00:00 monitor updated for chaiml-gspo-glm47-combin_7698_v1
2026-03-19T17:53:11.679999+00:00 monitor updated for chaiml-gspo-glm47-chai-_41286_v1
2026-03-19T17:53:49.432609+00:00 monitor updated for chaiml-gspo-glm47-combin_7698_v1
2026-03-19T17:54:11.817787+00:00 monitor updated for chaiml-gspo-glm47-chai-_41286_v1
2026-03-19T17:54:49.569465+00:00 monitor updated for chaiml-gspo-glm47-combin_7698_v1
2026-03-19T17:55:11.978170+00:00 monitor updated for chaiml-gspo-glm47-chai-_41286_v1
2026-03-19T17:55:49.718887+00:00 monitor updated for chaiml-gspo-glm47-combin_7698_v1
2026-03-19T17:56:12.299463+00:00 monitor updated for chaiml-gspo-glm47-chai-_41286_v1
2026-03-19T17:56:49.865664+00:00 monitor updated for chaiml-gspo-glm47-combin_7698_v1
2026-03-19T17:57:12.445716+00:00 monitor updated for chaiml-gspo-glm47-chai-_41286_v1
2026-03-19T17:57:50.006803+00:00 monitor updated for chaiml-gspo-glm47-combin_7698_v1
2026-03-19T17:58:12.596029+00:00 monitor updated for chaiml-gspo-glm47-chai-_41286_v1
2026-03-19T17:58:50.218986+00:00 monitor updated for chaiml-gspo-glm47-combin_7698_v1
2026-03-19T17:59:12.829460+00:00 monitor updated for chaiml-gspo-glm47-chai-_41286_v1
2026-03-19T17:59:50.355788+00:00 monitor updated for chaiml-gspo-glm47-combin_7698_v1
2026-03-19T18:00:12.978093+00:00 monitor updated for chaiml-gspo-glm47-chai-_41286_v1
2026-03-19T18:00:50.510071+00:00 monitor updated for chaiml-gspo-glm47-combin_7698_v1
2026-03-19T18:01:13.196269+00:00 monitor updated for chaiml-gspo-glm47-chai-_41286_v1
2026-03-19T18:01:55.881300+00:00 monitor updated for chaiml-gspo-glm47-combin_7698_v1
chaiml-gspo-glm47-chai-41286-v1-uploader: 2026-03-19T11:02:03.710472-0700 | finalize | INFO - Compression lifecycle finalized for 1 modifiers
2026-03-19T18:02:13.355615+00:00 monitor updated for chaiml-gspo-glm47-chai-_41286_v1
chaiml-gspo-glm47-chai-41286-v1-uploader: 2026-03-19T11:02:10.337718-0700 | post_process | WARNING - Optimized model is not saved. To save, please provide`output_dir` as input arg.Ex. `oneshot(..., output_dir=...)`
chaiml-gspo-glm47-chai-41286-v1-uploader: Saving to /dev/shm/model_output...
chaiml-gspo-glm47-chai-41286-v1-uploader: 2026-03-19T11:02:10.364376-0700 | get_model_compressor | INFO - skip_sparsity_compression_stats set to True. Skipping sparsity compression statistic calculations. No sparsity compressor will be applied.
2026-03-19T18:02:56.033577+00:00 monitor updated for chaiml-gspo-glm47-combin_7698_v1
2026-03-19T18:03:13.503259+00:00 monitor updated for chaiml-gspo-glm47-chai-_41286_v1
2026-03-19T18:03:56.191279+00:00 monitor updated for chaiml-gspo-glm47-combin_7698_v1
2026-03-19T18:04:13.662262+00:00 monitor updated for chaiml-gspo-glm47-chai-_41286_v1
2026-03-19T18:04:56.348216+00:00 monitor updated for chaiml-gspo-glm47-combin_7698_v1
2026-03-19T18:05:13.814516+00:00 monitor updated for chaiml-gspo-glm47-chai-_41286_v1
2026-03-19T18:05:56.504268+00:00 monitor updated for chaiml-gspo-glm47-combin_7698_v1
2026-03-19T18:06:13.969895+00:00 monitor updated for chaiml-gspo-glm47-chai-_41286_v1
2026-03-19T18:06:56.655937+00:00 monitor updated for chaiml-gspo-glm47-combin_7698_v1
2026-03-19T18:07:14.122832+00:00 monitor updated for chaiml-gspo-glm47-chai-_41286_v1
2026-03-19T18:07:56.799026+00:00 monitor updated for chaiml-gspo-glm47-combin_7698_v1
2026-03-19T18:08:14.268278+00:00 monitor updated for chaiml-gspo-glm47-chai-_41286_v1
2026-03-19T18:08:56.948653+00:00 monitor updated for chaiml-gspo-glm47-combin_7698_v1
2026-03-19T18:09:14.415736+00:00 monitor updated for chaiml-gspo-glm47-chai-_41286_v1
2026-03-19T18:09:57.102869+00:00 monitor updated for chaiml-gspo-glm47-combin_7698_v1
2026-03-19T18:10:14.561710+00:00 monitor updated for chaiml-gspo-glm47-chai-_41286_v1
2026-03-19T18:10:57.253165+00:00 monitor updated for chaiml-gspo-glm47-combin_7698_v1
2026-03-19T18:11:14.713444+00:00 monitor updated for chaiml-gspo-glm47-chai-_41286_v1
chaiml-gspo-glm47-combin-7698-v1-uploader: 2026-03-19T11:11:25.772366-0700 | finalize | INFO - Compression lifecycle finalized for 1 modifiers
chaiml-gspo-glm47-combin-7698-v1-uploader: 2026-03-19T11:11:31.932847-0700 | post_process | WARNING - Optimized model is not saved. To save, please provide`output_dir` as input arg.Ex. `oneshot(..., output_dir=...)`
chaiml-gspo-glm47-combin-7698-v1-uploader: Saving to /dev/shm/model_output...
chaiml-gspo-glm47-combin-7698-v1-uploader: 2026-03-19T11:11:31.960023-0700 | get_model_compressor | INFO - skip_sparsity_compression_stats set to True. Skipping sparsity compression statistic calculations. No sparsity compressor will be applied.
2026-03-19T18:11:57.408986+00:00 monitor updated for chaiml-gspo-glm47-combin_7698_v1
2026-03-19T18:12:14.861147+00:00 monitor updated for chaiml-gspo-glm47-chai-_41286_v1
2026-03-19T18:12:57.559739+00:00 monitor updated for chaiml-gspo-glm47-combin_7698_v1
2026-03-19T18:13:15.015421+00:00 monitor updated for chaiml-gspo-glm47-chai-_41286_v1
2026-03-19T18:13:57.712062+00:00 monitor updated for chaiml-gspo-glm47-combin_7698_v1
2026-03-19T18:14:15.230768+00:00 monitor updated for chaiml-gspo-glm47-chai-_41286_v1
2026-03-19T18:14:57.934084+00:00 monitor updated for chaiml-gspo-glm47-combin_7698_v1
2026-03-19T18:15:15.377968+00:00 monitor updated for chaiml-gspo-glm47-chai-_41286_v1
2026-03-19T18:15:58.107398+00:00 monitor updated for chaiml-gspo-glm47-combin_7698_v1
2026-03-19T18:16:15.529906+00:00 monitor updated for chaiml-gspo-glm47-chai-_41286_v1
2026-03-19T18:16:58.267364+00:00 monitor updated for chaiml-gspo-glm47-combin_7698_v1
2026-03-19T18:17:15.684050+00:00 monitor updated for chaiml-gspo-glm47-chai-_41286_v1
2026-03-19T18:17:58.419521+00:00 monitor updated for chaiml-gspo-glm47-combin_7698_v1
2026-03-19T18:18:15.831556+00:00 monitor updated for chaiml-gspo-glm47-chai-_41286_v1
2026-03-19T18:18:58.578967+00:00 monitor updated for chaiml-gspo-glm47-combin_7698_v1
2026-03-19T18:19:15.988086+00:00 monitor updated for chaiml-gspo-glm47-chai-_41286_v1
2026-03-19T18:19:58.734604+00:00 monitor updated for chaiml-gspo-glm47-combin_7698_v1
2026-03-19T18:20:16.145064+00:00 monitor updated for chaiml-gspo-glm47-chai-_41286_v1
2026-03-19T18:20:58.887310+00:00 monitor updated for chaiml-gspo-glm47-combin_7698_v1
2026-03-19T18:21:16.315893+00:00 monitor updated for chaiml-gspo-glm47-chai-_41286_v1
2026-03-19T18:21:59.052299+00:00 monitor updated for chaiml-gspo-glm47-combin_7698_v1
2026-03-19T18:22:16.470262+00:00 monitor updated for chaiml-gspo-glm47-chai-_41286_v1
2026-03-19T18:22:59.286623+00:00 monitor updated for chaiml-gspo-glm47-combin_7698_v1
2026-03-19T18:23:16.632111+00:00 monitor updated for chaiml-gspo-glm47-chai-_41286_v1
2026-03-19T18:23:59.446934+00:00 monitor updated for chaiml-gspo-glm47-combin_7698_v1
2026-03-19T18:24:16.796019+00:00 monitor updated for chaiml-gspo-glm47-chai-_41286_v1
2026-03-19T18:24:59.606589+00:00 monitor updated for chaiml-gspo-glm47-combin_7698_v1
2026-03-19T18:25:16.947172+00:00 monitor updated for chaiml-gspo-glm47-chai-_41286_v1
2026-03-19T18:25:59.867098+00:00 monitor updated for chaiml-gspo-glm47-combin_7698_v1
2026-03-19T18:26:17.111816+00:00 monitor updated for chaiml-gspo-glm47-chai-_41286_v1
2026-03-19T18:27:00.024952+00:00 monitor updated for chaiml-gspo-glm47-combin_7698_v1
2026-03-19T18:27:17.273035+00:00 monitor updated for chaiml-gspo-glm47-chai-_41286_v1
Retrying (%r) after connection broken by '%r': %s
2026-03-19T18:28:00.177509+00:00 monitor updated for chaiml-gspo-glm47-combin_7698_v1
2026-03-19T18:28:17.441803+00:00 monitor updated for chaiml-gspo-glm47-chai-_41286_v1
2026-03-19T18:29:00.331567+00:00 monitor updated for chaiml-gspo-glm47-combin_7698_v1
2026-03-19T18:29:17.592526+00:00 monitor updated for chaiml-gspo-glm47-chai-_41286_v1
2026-03-19T18:30:00.508403+00:00 monitor updated for chaiml-gspo-glm47-combin_7698_v1
2026-03-19T18:30:17.815074+00:00 monitor updated for chaiml-gspo-glm47-chai-_41286_v1
2026-03-19T18:31:00.669941+00:00 monitor updated for chaiml-gspo-glm47-combin_7698_v1
2026-03-19T18:31:17.974600+00:00 monitor updated for chaiml-gspo-glm47-chai-_41286_v1
2026-03-19T18:32:00.837448+00:00 monitor updated for chaiml-gspo-glm47-combin_7698_v1
2026-03-19T18:32:18.131566+00:00 monitor updated for chaiml-gspo-glm47-chai-_41286_v1
2026-03-19T18:33:00.989526+00:00 monitor updated for chaiml-gspo-glm47-combin_7698_v1
2026-03-19T18:33:18.294626+00:00 monitor updated for chaiml-gspo-glm47-chai-_41286_v1
2026-03-19T18:34:01.143041+00:00 monitor updated for chaiml-gspo-glm47-combin_7698_v1
2026-03-19T18:34:18.517707+00:00 monitor updated for chaiml-gspo-glm47-chai-_41286_v1
2026-03-19T18:35:01.406113+00:00 monitor updated for chaiml-gspo-glm47-combin_7698_v1
2026-03-19T18:35:18.732638+00:00 monitor updated for chaiml-gspo-glm47-chai-_41286_v1
2026-03-19T18:36:01.568779+00:00 monitor updated for chaiml-gspo-glm47-combin_7698_v1
2026-03-19T18:36:18.893772+00:00 monitor updated for chaiml-gspo-glm47-chai-_41286_v1
2026-03-19T18:37:01.752966+00:00 monitor updated for chaiml-gspo-glm47-combin_7698_v1
2026-03-19T18:37:19.110277+00:00 monitor updated for chaiml-gspo-glm47-chai-_41286_v1
2026-03-19T18:38:01.918588+00:00 monitor updated for chaiml-gspo-glm47-combin_7698_v1
2026-03-19T18:38:19.287247+00:00 monitor updated for chaiml-gspo-glm47-chai-_41286_v1
2026-03-19T18:39:02.088284+00:00 monitor updated for chaiml-gspo-glm47-combin_7698_v1
2026-03-19T18:39:19.458668+00:00 monitor updated for chaiml-gspo-glm47-chai-_41286_v1
2026-03-19T18:40:02.265473+00:00 monitor updated for chaiml-gspo-glm47-combin_7698_v1
2026-03-19T18:40:19.623259+00:00 monitor updated for chaiml-gspo-glm47-chai-_41286_v1
2026-03-19T18:41:02.543723+00:00 monitor updated for chaiml-gspo-glm47-combin_7698_v1
2026-03-19T18:41:19.818370+00:00 monitor updated for chaiml-gspo-glm47-chai-_41286_v1
2026-03-19T18:42:02.711436+00:00 monitor updated for chaiml-gspo-glm47-combin_7698_v1
2026-03-19T18:42:20.010303+00:00 monitor updated for chaiml-gspo-glm47-chai-_41286_v1
2026-03-19T18:43:02.883951+00:00 monitor updated for chaiml-gspo-glm47-combin_7698_v1
2026-03-19T18:43:20.219627+00:00 monitor updated for chaiml-gspo-glm47-chai-_41286_v1
chaiml-gspo-glm47-chai-41286-v1-uploader: Cleaning quantization config in /dev/shm/model_output
chaiml-gspo-glm47-chai-41286-v1-uploader: Pushing to ChaiML/gspo-glm47-chai-rm-zy-data-step100-FP8
chaiml-gspo-glm47-chai-41286-v1-uploader: Checking if ChaiML/gspo-glm47-chai-rm-zy-data-step100-FP8 already exists in ChaiML
chaiml-gspo-glm47-chai-41286-v1-uploader: Creating repo ChaiML/gspo-glm47-chai-rm-zy-data-step100-FP8 and uploading /dev/shm/model_output to it
chaiml-gspo-glm47-chai-41286-v1-uploader: ---------- 2026-03-19 11:43:45 (0:00:00) ----------
chaiml-gspo-glm47-chai-41286-v1-uploader: Files:   hashed 8/80 (28.2M/354.7G) | pre-uploaded: 0/1 (0.0/354.7G) (+72 unsure) | committed: 0/80 (0.0/354.7G) | ignored: 0
chaiml-gspo-glm47-chai-41286-v1-uploader: Workers: hashing: 72 | get upload mode: 0 | pre-uploading: 1 | committing: 0 | waiting: 53
chaiml-gspo-glm47-chai-41286-v1-uploader: ---------------------------------------------------
2026-03-19T18:44:03.050661+00:00 monitor updated for chaiml-gspo-glm47-combin_7698_v1
2026-03-19T18:44:20.513618+00:00 monitor updated for chaiml-gspo-glm47-chai-_41286_v1
chaiml-gspo-glm47-chai-41286-v1-uploader: 







chaiml-gspo-glm47-chai-41286-v1-uploader: ---------- 2026-03-19 11:44:45 (0:01:00) ----------
chaiml-gspo-glm47-chai-41286-v1-uploader: Files:   hashed 80/80 (354.7G/354.7G) | pre-uploaded: 3/73 (5.1G/354.7G) | committed: 0/80 (0.0/354.7G) | ignored: 0
chaiml-gspo-glm47-chai-41286-v1-uploader: Workers: hashing: 0 | get upload mode: 0 | pre-uploading: 70 | committing: 0 | waiting: 56
chaiml-gspo-glm47-chai-41286-v1-uploader: ---------------------------------------------------
2026-03-19T18:45:03.246588+00:00 monitor updated for chaiml-gspo-glm47-combin_7698_v1
2026-03-19T18:45:20.714248+00:00 monitor updated for chaiml-gspo-glm47-chai-_41286_v1
chaiml-gspo-glm47-chai-41286-v1-uploader:                              







chaiml-gspo-glm47-chai-41286-v1-uploader: ---------- 2026-03-19 11:45:45 (0:02:00) ----------
chaiml-gspo-glm47-chai-41286-v1-uploader: Files:   hashed 80/80 (354.7G/354.7G) | pre-uploaded: 3/73 (5.1G/354.7G) | committed: 0/80 (0.0/354.7G) | ignored: 0
chaiml-gspo-glm47-chai-41286-v1-uploader: Workers: hashing: 0 | get upload mode: 0 | pre-uploading: 70 | committing: 0 | waiting: 56
chaiml-gspo-glm47-chai-41286-v1-uploader: ---------------------------------------------------
2026-03-19T18:46:03.437692+00:00 monitor updated for chaiml-gspo-glm47-combin_7698_v1
2026-03-19T18:46:20.908461+00:00 monitor updated for chaiml-gspo-glm47-chai-_41286_v1
chaiml-gspo-glm47-chai-41286-v1-uploader:                              







chaiml-gspo-glm47-chai-41286-v1-uploader: ---------- 2026-03-19 11:46:45 (0:03:00) ----------
chaiml-gspo-glm47-chai-41286-v1-uploader: Files:   hashed 80/80 (354.7G/354.7G) | pre-uploaded: 11/73 (45.0G/354.7G) | committed: 0/80 (0.0/354.7G) | ignored: 0
chaiml-gspo-glm47-chai-41286-v1-uploader: Workers: hashing: 0 | get upload mode: 0 | pre-uploading: 62 | committing: 0 | waiting: 64
chaiml-gspo-glm47-chai-41286-v1-uploader: ---------------------------------------------------
2026-03-19T18:47:03.637404+00:00 monitor updated for chaiml-gspo-glm47-combin_7698_v1
2026-03-19T18:47:41.930240+00:00 monitor updated for chaiml-gspo-glm47-chai-_41286_v1
chaiml-gspo-glm47-chai-41286-v1-uploader:                              







chaiml-gspo-glm47-chai-41286-v1-uploader: ---------- 2026-03-19 11:47:45 (0:04:00) ----------
chaiml-gspo-glm47-chai-41286-v1-uploader: Files:   hashed 80/80 (354.7G/354.7G) | pre-uploaded: 19/73 (85.0G/354.7G) | committed: 0/80 (0.0/354.7G) | ignored: 0
chaiml-gspo-glm47-chai-41286-v1-uploader: Workers: hashing: 0 | get upload mode: 0 | pre-uploading: 54 | committing: 0 | waiting: 72
chaiml-gspo-glm47-chai-41286-v1-uploader: ---------------------------------------------------
2026-03-19T18:48:03.838622+00:00 monitor updated for chaiml-gspo-glm47-combin_7698_v1
2026-03-19T18:48:42.128388+00:00 monitor updated for chaiml-gspo-glm47-chai-_41286_v1
chaiml-gspo-glm47-chai-41286-v1-uploader:                              







chaiml-gspo-glm47-chai-41286-v1-uploader: ---------- 2026-03-19 11:48:45 (0:05:00) ----------
chaiml-gspo-glm47-chai-41286-v1-uploader: Files:   hashed 80/80 (354.7G/354.7G) | pre-uploaded: 24/73 (110.0G/354.7G) | committed: 0/80 (0.0/354.7G) | ignored: 0
chaiml-gspo-glm47-chai-41286-v1-uploader: Workers: hashing: 0 | get upload mode: 0 | pre-uploading: 49 | committing: 0 | waiting: 77
chaiml-gspo-glm47-chai-41286-v1-uploader: ---------------------------------------------------
2026-03-19T18:49:04.033836+00:00 monitor updated for chaiml-gspo-glm47-combin_7698_v1
2026-03-19T18:49:42.327656+00:00 monitor updated for chaiml-gspo-glm47-chai-_41286_v1
chaiml-gspo-glm47-chai-41286-v1-uploader:                              







chaiml-gspo-glm47-chai-41286-v1-uploader: ---------- 2026-03-19 11:49:45 (0:06:00) ----------
chaiml-gspo-glm47-chai-41286-v1-uploader: Files:   hashed 80/80 (354.7G/354.7G) | pre-uploaded: 27/73 (124.9G/354.7G) | committed: 0/80 (0.0/354.7G) | ignored: 0
chaiml-gspo-glm47-chai-41286-v1-uploader: Workers: hashing: 0 | get upload mode: 0 | pre-uploading: 46 | committing: 0 | waiting: 80
chaiml-gspo-glm47-chai-41286-v1-uploader: ---------------------------------------------------
2026-03-19T18:50:04.232511+00:00 monitor updated for chaiml-gspo-glm47-combin_7698_v1
2026-03-19T18:50:42.528913+00:00 monitor updated for chaiml-gspo-glm47-chai-_41286_v1
chaiml-gspo-glm47-chai-41286-v1-uploader:                              







chaiml-gspo-glm47-chai-41286-v1-uploader: ---------- 2026-03-19 11:50:45 (0:07:00) ----------
chaiml-gspo-glm47-chai-41286-v1-uploader: Files:   hashed 80/80 (354.7G/354.7G) | pre-uploaded: 35/73 (164.9G/354.7G) | committed: 0/80 (0.0/354.7G) | ignored: 0
chaiml-gspo-glm47-chai-41286-v1-uploader: Workers: hashing: 0 | get upload mode: 0 | pre-uploading: 38 | committing: 0 | waiting: 88
chaiml-gspo-glm47-chai-41286-v1-uploader: ---------------------------------------------------
2026-03-19T18:51:04.438591+00:00 monitor updated for chaiml-gspo-glm47-combin_7698_v1
2026-03-19T18:51:42.805661+00:00 monitor updated for chaiml-gspo-glm47-chai-_41286_v1
chaiml-gspo-glm47-chai-41286-v1-uploader:                              







chaiml-gspo-glm47-chai-41286-v1-uploader: ---------- 2026-03-19 11:51:45 (0:08:00) ----------
chaiml-gspo-glm47-chai-41286-v1-uploader: Files:   hashed 80/80 (354.7G/354.7G) | pre-uploaded: 43/73 (204.9G/354.7G) | committed: 0/80 (0.0/354.7G) | ignored: 0
chaiml-gspo-glm47-chai-41286-v1-uploader: Workers: hashing: 0 | get upload mode: 0 | pre-uploading: 30 | committing: 0 | waiting: 96
chaiml-gspo-glm47-chai-41286-v1-uploader: ---------------------------------------------------
2026-03-19T18:52:04.637952+00:00 monitor updated for chaiml-gspo-glm47-combin_7698_v1
2026-03-19T18:52:43.012102+00:00 monitor updated for chaiml-gspo-glm47-chai-_41286_v1
chaiml-gspo-glm47-chai-41286-v1-uploader:                              







chaiml-gspo-glm47-chai-41286-v1-uploader: ---------- 2026-03-19 11:52:45 (0:09:00) ----------
chaiml-gspo-glm47-chai-41286-v1-uploader: Files:   hashed 80/80 (354.7G/354.7G) | pre-uploaded: 49/73 (234.8G/354.7G) | committed: 0/80 (0.0/354.7G) | ignored: 0
chaiml-gspo-glm47-chai-41286-v1-uploader: Workers: hashing: 0 | get upload mode: 0 | pre-uploading: 24 | committing: 0 | waiting: 102
chaiml-gspo-glm47-chai-41286-v1-uploader: ---------------------------------------------------
2026-03-19T18:53:04.844530+00:00 monitor updated for chaiml-gspo-glm47-combin_7698_v1
2026-03-19T18:53:43.205165+00:00 monitor updated for chaiml-gspo-glm47-chai-_41286_v1
chaiml-gspo-glm47-chai-41286-v1-uploader:                              







chaiml-gspo-glm47-chai-41286-v1-uploader: ---------- 2026-03-19 11:53:45 (0:10:00) ----------
chaiml-gspo-glm47-chai-41286-v1-uploader: Files:   hashed 80/80 (354.7G/354.7G) | pre-uploaded: 52/73 (249.8G/354.7G) | committed: 0/80 (0.0/354.7G) | ignored: 0
chaiml-gspo-glm47-chai-41286-v1-uploader: Workers: hashing: 0 | get upload mode: 0 | pre-uploading: 21 | committing: 0 | waiting: 105
chaiml-gspo-glm47-chai-41286-v1-uploader: ---------------------------------------------------
2026-03-19T18:54:05.044801+00:00 monitor updated for chaiml-gspo-glm47-combin_7698_v1
2026-03-19T18:54:43.412626+00:00 monitor updated for chaiml-gspo-glm47-chai-_41286_v1
Retrying (%r) after connection broken by '%r': %s
chaiml-gspo-glm47-combin-7698-v1-uploader: Cleaning quantization config in /dev/shm/model_output
chaiml-gspo-glm47-combin-7698-v1-uploader: Pushing to ChaiML/gspo-glm47-combine-rm82-zy-data-step100-FP8
chaiml-gspo-glm47-combin-7698-v1-uploader: Checking if ChaiML/gspo-glm47-combine-rm82-zy-data-step100-FP8 already exists in ChaiML
chaiml-gspo-glm47-combin-7698-v1-uploader: Creating repo ChaiML/gspo-glm47-combine-rm82-zy-data-step100-FP8 and uploading /dev/shm/model_output to it
chaiml-gspo-glm47-chai-41286-v1-uploader:                              







chaiml-gspo-glm47-chai-41286-v1-uploader: ---------- 2026-03-19 11:54:45 (0:11:00) ----------
chaiml-gspo-glm47-chai-41286-v1-uploader: Files:   hashed 80/80 (354.7G/354.7G) | pre-uploaded: 59/73 (284.8G/354.7G) | committed: 0/80 (0.0/354.7G) | ignored: 0
chaiml-gspo-glm47-chai-41286-v1-uploader: Workers: hashing: 0 | get upload mode: 0 | pre-uploading: 14 | committing: 0 | waiting: 112
chaiml-gspo-glm47-chai-41286-v1-uploader: ---------------------------------------------------
2026-03-19T18:55:05.251876+00:00 monitor updated for chaiml-gspo-glm47-combin_7698_v1
2026-03-19T18:55:43.607007+00:00 monitor updated for chaiml-gspo-glm47-chai-_41286_v1
chaiml-gspo-glm47-chai-41286-v1-uploader:                              







chaiml-gspo-glm47-chai-41286-v1-uploader: ---------- 2026-03-19 11:55:45 (0:12:00) ----------
chaiml-gspo-glm47-chai-41286-v1-uploader: Files:   hashed 80/80 (354.7G/354.7G) | pre-uploaded: 67/73 (324.7G/354.7G) | committed: 0/80 (0.0/354.7G) | ignored: 0
chaiml-gspo-glm47-chai-41286-v1-uploader: Workers: hashing: 0 | get upload mode: 0 | pre-uploading: 6 | committing: 0 | waiting: 120
chaiml-gspo-glm47-chai-41286-v1-uploader: ---------------------------------------------------
chaiml-gspo-glm47-combin-7698-v1-uploader: 







chaiml-gspo-glm47-combin-7698-v1-uploader: ---------- 2026-03-19 11:55:46 (0:01:00) ----------
chaiml-gspo-glm47-combin-7698-v1-uploader: Files:   hashed 80/80 (354.7G/354.7G) | pre-uploaded: 3/73 (5.1G/354.7G) | committed: 0/80 (0.0/354.7G) | ignored: 0
chaiml-gspo-glm47-combin-7698-v1-uploader: Workers: hashing: 0 | get upload mode: 0 | pre-uploading: 70 | committing: 0 | waiting: 56
chaiml-gspo-glm47-combin-7698-v1-uploader: ---------------------------------------------------
2026-03-19T18:56:05.459097+00:00 monitor updated for chaiml-gspo-glm47-combin_7698_v1
2026-03-19T18:56:43.809586+00:00 monitor updated for chaiml-gspo-glm47-chai-_41286_v1
chaiml-gspo-glm47-chai-41286-v1-uploader:                              







chaiml-gspo-glm47-chai-41286-v1-uploader: ---------- 2026-03-19 11:56:45 (0:13:00) ----------
chaiml-gspo-glm47-chai-41286-v1-uploader: Files:   hashed 80/80 (354.7G/354.7G) | pre-uploaded: 73/73 (354.7G/354.7G) | committed: 0/80 (0.0/354.7G) | ignored: 0
chaiml-gspo-glm47-chai-41286-v1-uploader: Workers: hashing: 0 | get upload mode: 0 | pre-uploading: 0 | committing: 1 | waiting: 125
chaiml-gspo-glm47-chai-41286-v1-uploader: ---------------------------------------------------
chaiml-gspo-glm47-combin-7698-v1-uploader:                              







chaiml-gspo-glm47-combin-7698-v1-uploader: ---------- 2026-03-19 11:56:46 (0:02:00) ----------
chaiml-gspo-glm47-combin-7698-v1-uploader: Files:   hashed 80/80 (354.7G/354.7G) | pre-uploaded: 10/73 (40.0G/354.7G) | committed: 0/80 (0.0/354.7G) | ignored: 0
chaiml-gspo-glm47-combin-7698-v1-uploader: Workers: hashing: 0 | get upload mode: 0 | pre-uploading: 63 | committing: 0 | waiting: 63
chaiml-gspo-glm47-combin-7698-v1-uploader: ---------------------------------------------------
2026-03-19T18:57:05.671095+00:00 monitor updated for chaiml-gspo-glm47-combin_7698_v1
2026-03-19T18:57:44.015618+00:00 monitor updated for chaiml-gspo-glm47-chai-_41286_v1
chaiml-gspo-glm47-chai-41286-v1-uploader:                              







chaiml-gspo-glm47-chai-41286-v1-uploader: ---------- 2026-03-19 11:57:45 (0:14:00) ----------
chaiml-gspo-glm47-chai-41286-v1-uploader: Files:   hashed 80/80 (354.7G/354.7G) | pre-uploaded: 73/73 (354.7G/354.7G) | committed: 0/80 (0.0/354.7G) | ignored: 0
chaiml-gspo-glm47-chai-41286-v1-uploader: Workers: hashing: 0 | get upload mode: 0 | pre-uploading: 0 | committing: 1 | waiting: 125
chaiml-gspo-glm47-chai-41286-v1-uploader: ---------------------------------------------------
chaiml-gspo-glm47-combin-7698-v1-uploader:                              







chaiml-gspo-glm47-combin-7698-v1-uploader: ---------- 2026-03-19 11:57:46 (0:03:00) ----------
chaiml-gspo-glm47-combin-7698-v1-uploader: Files:   hashed 80/80 (354.7G/354.7G) | pre-uploaded: 12/73 (50.0G/354.7G) | committed: 0/80 (0.0/354.7G) | ignored: 0
chaiml-gspo-glm47-combin-7698-v1-uploader: Workers: hashing: 0 | get upload mode: 0 | pre-uploading: 61 | committing: 0 | waiting: 65
chaiml-gspo-glm47-combin-7698-v1-uploader: ---------------------------------------------------
2026-03-19T18:58:05.882876+00:00 monitor updated for chaiml-gspo-glm47-combin_7698_v1
2026-03-19T18:58:44.222428+00:00 monitor updated for chaiml-gspo-glm47-chai-_41286_v1
Retrying (%r) after connection broken by '%r': %s
chaiml-gspo-glm47-chai-41286-v1-uploader:                              Processed model ChaiML/gspo-glm47-chai-rm-zy-data-step100 in 4438.343s
chaiml-gspo-glm47-combin-7698-v1-uploader:                              







chaiml-gspo-glm47-combin-7698-v1-uploader: ---------- 2026-03-19 11:58:46 (0:04:00) ----------
chaiml-gspo-glm47-combin-7698-v1-uploader: Files:   hashed 80/80 (354.7G/354.7G) | pre-uploaded: 19/73 (85.0G/354.7G) | committed: 0/80 (0.0/354.7G) | ignored: 0
chaiml-gspo-glm47-combin-7698-v1-uploader: Workers: hashing: 0 | get upload mode: 0 | pre-uploading: 54 | committing: 0 | waiting: 72
chaiml-gspo-glm47-combin-7698-v1-uploader: ---------------------------------------------------
2026-03-19T18:59:06.104513+00:00 monitor updated for chaiml-gspo-glm47-combin_7698_v1
chaiml-gspo-glm47-chai-41286-v1-uploader: creating bucket guanaco-vllm-models
chaiml-gspo-glm47-chai-41286-v1-uploader: /usr/lib/python3/dist-packages/S3/BaseUtils.py:56: SyntaxWarning: invalid escape sequence '\.'
chaiml-gspo-glm47-chai-41286-v1-uploader:   RE_S3_DATESTRING = re.compile('\.[0-9]*(?:[Z\\-\\+]*?)')
chaiml-gspo-glm47-chai-41286-v1-uploader: /usr/lib/python3/dist-packages/S3/BaseUtils.py:57: SyntaxWarning: invalid escape sequence '\s'
chaiml-gspo-glm47-chai-41286-v1-uploader:   RE_XML_NAMESPACE = re.compile(b'^(<?[^>]+?>\s*|\s*)(<\w+) xmlns=[\'"](https?://[^\'"]+)[\'"]', re.MULTILINE)
chaiml-gspo-glm47-chai-41286-v1-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:240: SyntaxWarning: invalid escape sequence '\.'
chaiml-gspo-glm47-chai-41286-v1-uploader:   invalid = re.search("([^a-z0-9\.-])", bucket, re.UNICODE)
chaiml-gspo-glm47-chai-41286-v1-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:244: SyntaxWarning: invalid escape sequence '\.'
chaiml-gspo-glm47-chai-41286-v1-uploader:   invalid = re.search("([^A-Za-z0-9\._-])", bucket, re.UNICODE)
chaiml-gspo-glm47-chai-41286-v1-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:255: SyntaxWarning: invalid escape sequence '\.'
chaiml-gspo-glm47-chai-41286-v1-uploader:   if re.search("-\.", bucket, re.UNICODE):
chaiml-gspo-glm47-chai-41286-v1-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:257: SyntaxWarning: invalid escape sequence '\.'
chaiml-gspo-glm47-chai-41286-v1-uploader:   if re.search("\.\.", bucket, re.UNICODE):
chaiml-gspo-glm47-chai-41286-v1-uploader: /usr/lib/python3/dist-packages/S3/S3Uri.py:155: SyntaxWarning: invalid escape sequence '\w'
chaiml-gspo-glm47-chai-41286-v1-uploader:   _re = re.compile("^(\w+://)?(.*)", re.UNICODE)
chaiml-gspo-glm47-chai-41286-v1-uploader: /usr/lib/python3/dist-packages/S3/FileLists.py:480: SyntaxWarning: invalid escape sequence '\*'
chaiml-gspo-glm47-chai-41286-v1-uploader:   wildcard_split_result = re.split("\*|\?", uri_str, maxsplit=1)
chaiml-gspo-glm47-chai-41286-v1-uploader: Bucket 's3://guanaco-vllm-models/' created
chaiml-gspo-glm47-chai-41286-v1-uploader: uploading /dev/shm/model_output to s3://guanaco-vllm-models/chaiml-gspo-glm47-chai-41286-v1/default
chaiml-gspo-glm47-chai-41286-v1-uploader: cp /dev/shm/model_output/chat_template.jinja s3://guanaco-vllm-models/chaiml-gspo-glm47-chai-41286-v1/default/chat_template.jinja
chaiml-gspo-glm47-chai-41286-v1-uploader: cp /dev/shm/model_output/generation_config.json s3://guanaco-vllm-models/chaiml-gspo-glm47-chai-41286-v1/default/generation_config.json
chaiml-gspo-glm47-chai-41286-v1-uploader: cp /dev/shm/model_output/recipe.yaml s3://guanaco-vllm-models/chaiml-gspo-glm47-chai-41286-v1/default/recipe.yaml
chaiml-gspo-glm47-chai-41286-v1-uploader: cp /dev/shm/model_output/special_tokens_map.json s3://guanaco-vllm-models/chaiml-gspo-glm47-chai-41286-v1/default/special_tokens_map.json
chaiml-gspo-glm47-chai-41286-v1-uploader: cp /dev/shm/model_output/config.json s3://guanaco-vllm-models/chaiml-gspo-glm47-chai-41286-v1/default/config.json
chaiml-gspo-glm47-chai-41286-v1-uploader: cp /dev/shm/model_output/model.safetensors.index.json s3://guanaco-vllm-models/chaiml-gspo-glm47-chai-41286-v1/default/model.safetensors.index.json
chaiml-gspo-glm47-chai-41286-v1-uploader: cp /dev/shm/model_output/tokenizer_config.json s3://guanaco-vllm-models/chaiml-gspo-glm47-chai-41286-v1/default/tokenizer_config.json
chaiml-gspo-glm47-chai-41286-v1-uploader: cp /dev/shm/model_output/tokenizer.json s3://guanaco-vllm-models/chaiml-gspo-glm47-chai-41286-v1/default/tokenizer.json
chaiml-gspo-glm47-chai-41286-v1-uploader: cp /dev/shm/model_output/model-00072-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm47-chai-41286-v1/default/model-00072-of-00072.safetensors
2026-03-19T18:59:44.521490+00:00 monitor updated for chaiml-gspo-glm47-chai-_41286_v1
chaiml-gspo-glm47-combin-7698-v1-uploader:                              







chaiml-gspo-glm47-combin-7698-v1-uploader: ---------- 2026-03-19 11:59:46 (0:05:00) ----------
chaiml-gspo-glm47-combin-7698-v1-uploader: Files:   hashed 80/80 (354.7G/354.7G) | pre-uploaded: 27/73 (125.0G/354.7G) | committed: 0/80 (0.0/354.7G) | ignored: 0
chaiml-gspo-glm47-combin-7698-v1-uploader: Workers: hashing: 0 | get upload mode: 0 | pre-uploading: 46 | committing: 0 | waiting: 80
chaiml-gspo-glm47-combin-7698-v1-uploader: ---------------------------------------------------
2026-03-19T19:00:06.427265+00:00 monitor updated for chaiml-gspo-glm47-combin_7698_v1
chaiml-gspo-glm47-chai-41286-v1-uploader: cp /dev/shm/model_output/model-00071-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm47-chai-41286-v1/default/model-00071-of-00072.safetensors
chaiml-gspo-glm47-chai-41286-v1-uploader: cp /dev/shm/model_output/model-00002-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm47-chai-41286-v1/default/model-00002-of-00072.safetensors
chaiml-gspo-glm47-chai-41286-v1-uploader: cp /dev/shm/model_output/model-00057-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm47-chai-41286-v1/default/model-00057-of-00072.safetensors
chaiml-gspo-glm47-chai-41286-v1-uploader: cp /dev/shm/model_output/model-00030-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm47-chai-41286-v1/default/model-00030-of-00072.safetensors
chaiml-gspo-glm47-chai-41286-v1-uploader: cp /dev/shm/model_output/model-00007-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm47-chai-41286-v1/default/model-00007-of-00072.safetensors
chaiml-gspo-glm47-chai-41286-v1-uploader: cp /dev/shm/model_output/model-00016-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm47-chai-41286-v1/default/model-00016-of-00072.safetensors
chaiml-gspo-glm47-chai-41286-v1-uploader: cp /dev/shm/model_output/model-00004-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm47-chai-41286-v1/default/model-00004-of-00072.safetensors
chaiml-gspo-glm47-chai-41286-v1-uploader: cp /dev/shm/model_output/model-00041-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm47-chai-41286-v1/default/model-00041-of-00072.safetensors
chaiml-gspo-glm47-chai-41286-v1-uploader: cp /dev/shm/model_output/model-00065-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm47-chai-41286-v1/default/model-00065-of-00072.safetensors
chaiml-gspo-glm47-chai-41286-v1-uploader: cp /dev/shm/model_output/model-00066-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm47-chai-41286-v1/default/model-00066-of-00072.safetensors
chaiml-gspo-glm47-chai-41286-v1-uploader: cp /dev/shm/model_output/model-00054-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm47-chai-41286-v1/default/model-00054-of-00072.safetensors
chaiml-gspo-glm47-chai-41286-v1-uploader: cp /dev/shm/model_output/model-00013-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm47-chai-41286-v1/default/model-00013-of-00072.safetensors
chaiml-gspo-glm47-chai-41286-v1-uploader: cp /dev/shm/model_output/model-00024-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm47-chai-41286-v1/default/model-00024-of-00072.safetensors
chaiml-gspo-glm47-chai-41286-v1-uploader: cp /dev/shm/model_output/model-00017-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm47-chai-41286-v1/default/model-00017-of-00072.safetensors
chaiml-gspo-glm47-chai-41286-v1-uploader: cp /dev/shm/model_output/model-00018-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm47-chai-41286-v1/default/model-00018-of-00072.safetensors
chaiml-gspo-glm47-chai-41286-v1-uploader: cp /dev/shm/model_output/model-00019-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm47-chai-41286-v1/default/model-00019-of-00072.safetensors
chaiml-gspo-glm47-chai-41286-v1-uploader: cp /dev/shm/model_output/model-00052-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm47-chai-41286-v1/default/model-00052-of-00072.safetensors
chaiml-gspo-glm47-chai-41286-v1-uploader: cp /dev/shm/model_output/model-00025-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm47-chai-41286-v1/default/model-00025-of-00072.safetensors
chaiml-gspo-glm47-chai-41286-v1-uploader: cp /dev/shm/model_output/model-00015-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm47-chai-41286-v1/default/model-00015-of-00072.safetensors
chaiml-gspo-glm47-chai-41286-v1-uploader: cp /dev/shm/model_output/model-00045-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm47-chai-41286-v1/default/model-00045-of-00072.safetensors
chaiml-gspo-glm47-chai-41286-v1-uploader: cp /dev/shm/model_output/model-00044-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm47-chai-41286-v1/default/model-00044-of-00072.safetensors
chaiml-gspo-glm47-chai-41286-v1-uploader: cp /dev/shm/model_output/model-00034-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm47-chai-41286-v1/default/model-00034-of-00072.safetensors
chaiml-gspo-glm47-chai-41286-v1-uploader: cp /dev/shm/model_output/model-00068-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm47-chai-41286-v1/default/model-00068-of-00072.safetensors
chaiml-gspo-glm47-chai-41286-v1-uploader: cp /dev/shm/model_output/model-00046-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm47-chai-41286-v1/default/model-00046-of-00072.safetensors
chaiml-gspo-glm47-chai-41286-v1-uploader: cp /dev/shm/model_output/model-00058-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm47-chai-41286-v1/default/model-00058-of-00072.safetensors
chaiml-gspo-glm47-chai-41286-v1-uploader: cp /dev/shm/model_output/model-00006-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm47-chai-41286-v1/default/model-00006-of-00072.safetensors
chaiml-gspo-glm47-chai-41286-v1-uploader: cp /dev/shm/model_output/model-00008-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm47-chai-41286-v1/default/model-00008-of-00072.safetensors
chaiml-gspo-glm47-chai-41286-v1-uploader: cp /dev/shm/model_output/model-00020-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm47-chai-41286-v1/default/model-00020-of-00072.safetensors
chaiml-gspo-glm47-chai-41286-v1-uploader: cp /dev/shm/model_output/model-00067-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm47-chai-41286-v1/default/model-00067-of-00072.safetensors
chaiml-gspo-glm47-chai-41286-v1-uploader: cp /dev/shm/model_output/model-00063-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm47-chai-41286-v1/default/model-00063-of-00072.safetensors
chaiml-gspo-glm47-chai-41286-v1-uploader: cp /dev/shm/model_output/model-00032-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm47-chai-41286-v1/default/model-00032-of-00072.safetensors
chaiml-gspo-glm47-chai-41286-v1-uploader: cp /dev/shm/model_output/model-00022-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm47-chai-41286-v1/default/model-00022-of-00072.safetensors
chaiml-gspo-glm47-chai-41286-v1-uploader: cp /dev/shm/model_output/model-00012-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm47-chai-41286-v1/default/model-00012-of-00072.safetensors
chaiml-gspo-glm47-chai-41286-v1-uploader: cp /dev/shm/model_output/model-00026-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm47-chai-41286-v1/default/model-00026-of-00072.safetensors
chaiml-gspo-glm47-chai-41286-v1-uploader: cp /dev/shm/model_output/model-00049-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm47-chai-41286-v1/default/model-00049-of-00072.safetensors
chaiml-gspo-glm47-chai-41286-v1-uploader: cp /dev/shm/model_output/model-00037-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm47-chai-41286-v1/default/model-00037-of-00072.safetensors
chaiml-gspo-glm47-chai-41286-v1-uploader: cp /dev/shm/model_output/model-00056-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm47-chai-41286-v1/default/model-00056-of-00072.safetensors
chaiml-gspo-glm47-chai-41286-v1-uploader: cp /dev/shm/model_output/model-00035-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm47-chai-41286-v1/default/model-00035-of-00072.safetensors
chaiml-gspo-glm47-chai-41286-v1-uploader: cp /dev/shm/model_output/model-00055-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm47-chai-41286-v1/default/model-00055-of-00072.safetensors
chaiml-gspo-glm47-chai-41286-v1-uploader: cp /dev/shm/model_output/model-00005-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm47-chai-41286-v1/default/model-00005-of-00072.safetensors
chaiml-gspo-glm47-chai-41286-v1-uploader: cp /dev/shm/model_output/model-00011-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm47-chai-41286-v1/default/model-00011-of-00072.safetensors
chaiml-gspo-glm47-chai-41286-v1-uploader: cp /dev/shm/model_output/model-00060-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm47-chai-41286-v1/default/model-00060-of-00072.safetensors
chaiml-gspo-glm47-chai-41286-v1-uploader: cp /dev/shm/model_output/model-00014-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm47-chai-41286-v1/default/model-00014-of-00072.safetensors
chaiml-gspo-glm47-chai-41286-v1-uploader: cp /dev/shm/model_output/model-00040-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm47-chai-41286-v1/default/model-00040-of-00072.safetensors
chaiml-gspo-glm47-chai-41286-v1-uploader: cp /dev/shm/model_output/model-00047-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm47-chai-41286-v1/default/model-00047-of-00072.safetensors
chaiml-gspo-glm47-chai-41286-v1-uploader: cp /dev/shm/model_output/model-00001-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm47-chai-41286-v1/default/model-00001-of-00072.safetensors
chaiml-gspo-glm47-chai-41286-v1-uploader: cp /dev/shm/model_output/model-00009-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm47-chai-41286-v1/default/model-00009-of-00072.safetensors
chaiml-gspo-glm47-chai-41286-v1-uploader: cp /dev/shm/model_output/model-00070-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm47-chai-41286-v1/default/model-00070-of-00072.safetensors
chaiml-gspo-glm47-chai-41286-v1-uploader: cp /dev/shm/model_output/model-00061-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm47-chai-41286-v1/default/model-00061-of-00072.safetensors
chaiml-gspo-glm47-chai-41286-v1-uploader: cp /dev/shm/model_output/model-00028-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm47-chai-41286-v1/default/model-00028-of-00072.safetensors
chaiml-gspo-glm47-chai-41286-v1-uploader: cp /dev/shm/model_output/model-00033-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm47-chai-41286-v1/default/model-00033-of-00072.safetensors
chaiml-gspo-glm47-chai-41286-v1-uploader: cp /dev/shm/model_output/model-00021-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm47-chai-41286-v1/default/model-00021-of-00072.safetensors
chaiml-gspo-glm47-chai-41286-v1-uploader: cp /dev/shm/model_output/model-00069-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm47-chai-41286-v1/default/model-00069-of-00072.safetensors
chaiml-gspo-glm47-chai-41286-v1-uploader: cp /dev/shm/model_output/model-00039-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm47-chai-41286-v1/default/model-00039-of-00072.safetensors
chaiml-gspo-glm47-chai-41286-v1-uploader: cp /dev/shm/model_output/model-00048-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm47-chai-41286-v1/default/model-00048-of-00072.safetensors
chaiml-gspo-glm47-chai-41286-v1-uploader: cp /dev/shm/model_output/model-00043-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm47-chai-41286-v1/default/model-00043-of-00072.safetensors
chaiml-gspo-glm47-chai-41286-v1-uploader: cp /dev/shm/model_output/model-00010-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm47-chai-41286-v1/default/model-00010-of-00072.safetensors
chaiml-gspo-glm47-chai-41286-v1-uploader: cp /dev/shm/model_output/model-00031-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm47-chai-41286-v1/default/model-00031-of-00072.safetensors
chaiml-gspo-glm47-chai-41286-v1-uploader: cp /dev/shm/model_output/model-00003-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm47-chai-41286-v1/default/model-00003-of-00072.safetensors
2026-03-19T19:00:44.822748+00:00 monitor updated for chaiml-gspo-glm47-chai-_41286_v1
Job chaiml-gspo-glm47-chai-41286-v1-uploader completed after 4597.82s with status: succeeded
Stopping job with name chaiml-gspo-glm47-chai-41286-v1-uploader
Pipeline stage VLLMUploader completed in 4598.80s
run pipeline stage %s
Running pipeline stage VLLMTemplater
Pipeline stage VLLMTemplater completed in 0.72s
run pipeline stage %s
Running pipeline stage VLLMDeployer
Creating inference service chaiml-gspo-glm47-chai-41286-v1
Waiting for inference service chaiml-gspo-glm47-chai-41286-v1 to be ready
chaiml-gspo-glm47-combin-7698-v1-uploader:                              







chaiml-gspo-glm47-combin-7698-v1-uploader: ---------- 2026-03-19 12:00:46 (0:06:00) ----------
chaiml-gspo-glm47-combin-7698-v1-uploader: Files:   hashed 80/80 (354.7G/354.7G) | pre-uploaded: 35/73 (164.9G/354.7G) | committed: 0/80 (0.0/354.7G) | ignored: 0
chaiml-gspo-glm47-combin-7698-v1-uploader: Workers: hashing: 0 | get upload mode: 0 | pre-uploading: 38 | committing: 0 | waiting: 88
chaiml-gspo-glm47-combin-7698-v1-uploader: ---------------------------------------------------
2026-03-19T19:01:06.648137+00:00 monitor updated for chaiml-gspo-glm47-combin_7698_v1
2026-03-19T19:01:45.032958+00:00 monitor updated for chaiml-gspo-glm47-chai-_41286_v1
chaiml-gspo-glm47-combin-7698-v1-uploader:                              







chaiml-gspo-glm47-combin-7698-v1-uploader: ---------- 2026-03-19 12:01:46 (0:07:00) ----------
chaiml-gspo-glm47-combin-7698-v1-uploader: Files:   hashed 80/80 (354.7G/354.7G) | pre-uploaded: 37/73 (174.9G/354.7G) | committed: 0/80 (0.0/354.7G) | ignored: 0
chaiml-gspo-glm47-combin-7698-v1-uploader: Workers: hashing: 0 | get upload mode: 0 | pre-uploading: 36 | committing: 0 | waiting: 90
chaiml-gspo-glm47-combin-7698-v1-uploader: ---------------------------------------------------
2026-03-19T19:02:12.615221+00:00 monitor updated for chaiml-gspo-glm47-combin_7698_v1
2026-03-19T19:02:45.250803+00:00 monitor updated for chaiml-gspo-glm47-chai-_41286_v1
chaiml-gspo-glm47-combin-7698-v1-uploader:                              







chaiml-gspo-glm47-combin-7698-v1-uploader: ---------- 2026-03-19 12:02:46 (0:08:00) ----------
chaiml-gspo-glm47-combin-7698-v1-uploader: Files:   hashed 80/80 (354.7G/354.7G) | pre-uploaded: 43/73 (204.8G/354.7G) | committed: 0/80 (0.0/354.7G) | ignored: 0
chaiml-gspo-glm47-combin-7698-v1-uploader: Workers: hashing: 0 | get upload mode: 0 | pre-uploading: 30 | committing: 0 | waiting: 96
chaiml-gspo-glm47-combin-7698-v1-uploader: ---------------------------------------------------
Retrying (%r) after connection broken by '%r': %s
2026-03-19T19:03:12.923252+00:00 monitor updated for chaiml-gspo-glm47-combin_7698_v1
2026-03-19T19:03:45.501837+00:00 monitor updated for chaiml-gspo-glm47-chai-_41286_v1
chaiml-gspo-glm47-combin-7698-v1-uploader:                              







chaiml-gspo-glm47-combin-7698-v1-uploader: ---------- 2026-03-19 12:03:46 (0:09:00) ----------
chaiml-gspo-glm47-combin-7698-v1-uploader: Files:   hashed 80/80 (354.7G/354.7G) | pre-uploaded: 51/73 (244.8G/354.7G) | committed: 0/80 (0.0/354.7G) | ignored: 0
chaiml-gspo-glm47-combin-7698-v1-uploader: Workers: hashing: 0 | get upload mode: 0 | pre-uploading: 22 | committing: 0 | waiting: 104
chaiml-gspo-glm47-combin-7698-v1-uploader: ---------------------------------------------------
2026-03-19T19:04:13.146910+00:00 monitor updated for chaiml-gspo-glm47-combin_7698_v1
2026-03-19T19:04:45.720242+00:00 monitor updated for chaiml-gspo-glm47-chai-_41286_v1
chaiml-gspo-glm47-combin-7698-v1-uploader:                              







chaiml-gspo-glm47-combin-7698-v1-uploader: ---------- 2026-03-19 12:04:46 (0:10:00) ----------
chaiml-gspo-glm47-combin-7698-v1-uploader: Files:   hashed 80/80 (354.7G/354.7G) | pre-uploaded: 58/73 (279.8G/354.7G) | committed: 0/80 (0.0/354.7G) | ignored: 0
chaiml-gspo-glm47-combin-7698-v1-uploader: Workers: hashing: 0 | get upload mode: 0 | pre-uploading: 15 | committing: 0 | waiting: 111
chaiml-gspo-glm47-combin-7698-v1-uploader: ---------------------------------------------------
2026-03-19T19:05:13.380423+00:00 monitor updated for chaiml-gspo-glm47-combin_7698_v1
2026-03-19T19:05:45.941171+00:00 monitor updated for chaiml-gspo-glm47-chai-_41286_v1
chaiml-gspo-glm47-combin-7698-v1-uploader:                              







chaiml-gspo-glm47-combin-7698-v1-uploader: ---------- 2026-03-19 12:05:46 (0:11:00) ----------
chaiml-gspo-glm47-combin-7698-v1-uploader: Files:   hashed 80/80 (354.7G/354.7G) | pre-uploaded: 62/73 (299.8G/354.7G) | committed: 0/80 (0.0/354.7G) | ignored: 0
chaiml-gspo-glm47-combin-7698-v1-uploader: Workers: hashing: 0 | get upload mode: 0 | pre-uploading: 11 | committing: 0 | waiting: 115
chaiml-gspo-glm47-combin-7698-v1-uploader: ---------------------------------------------------
2026-03-19T19:06:13.618963+00:00 monitor updated for chaiml-gspo-glm47-combin_7698_v1
2026-03-19T19:06:46.167826+00:00 monitor updated for chaiml-gspo-glm47-chai-_41286_v1
chaiml-gspo-glm47-combin-7698-v1-uploader:                              







chaiml-gspo-glm47-combin-7698-v1-uploader: ---------- 2026-03-19 12:06:46 (0:12:00) ----------
chaiml-gspo-glm47-combin-7698-v1-uploader: Files:   hashed 80/80 (354.7G/354.7G) | pre-uploaded: 73/73 (354.7G/354.7G) | committed: 0/80 (0.0/354.7G) | ignored: 0
chaiml-gspo-glm47-combin-7698-v1-uploader: Workers: hashing: 0 | get upload mode: 0 | pre-uploading: 0 | committing: 1 | waiting: 125
chaiml-gspo-glm47-combin-7698-v1-uploader: ---------------------------------------------------
2026-03-19T19:07:13.929106+00:00 monitor updated for chaiml-gspo-glm47-combin_7698_v1
2026-03-19T19:07:46.416968+00:00 monitor updated for chaiml-gspo-glm47-chai-_41286_v1
chaiml-gspo-glm47-combin-7698-v1-uploader:                              







chaiml-gspo-glm47-combin-7698-v1-uploader: ---------- 2026-03-19 12:07:46 (0:13:00) ----------
chaiml-gspo-glm47-combin-7698-v1-uploader: Files:   hashed 80/80 (354.7G/354.7G) | pre-uploaded: 73/73 (354.7G/354.7G) | committed: 0/80 (0.0/354.7G) | ignored: 0
chaiml-gspo-glm47-combin-7698-v1-uploader: Workers: hashing: 0 | get upload mode: 0 | pre-uploading: 0 | committing: 1 | waiting: 125
chaiml-gspo-glm47-combin-7698-v1-uploader: ---------------------------------------------------
2026-03-19T19:08:14.152435+00:00 monitor updated for chaiml-gspo-glm47-combin_7698_v1
2026-03-19T19:08:46.638466+00:00 monitor updated for chaiml-gspo-glm47-chai-_41286_v1
chaiml-gspo-glm47-combin-7698-v1-uploader:                              







chaiml-gspo-glm47-combin-7698-v1-uploader: ---------- 2026-03-19 12:08:46 (0:14:00) ----------
chaiml-gspo-glm47-combin-7698-v1-uploader: Files:   hashed 80/80 (354.7G/354.7G) | pre-uploaded: 73/73 (354.7G/354.7G) | committed: 50/80 (204.9G/354.7G) | ignored: 0
chaiml-gspo-glm47-combin-7698-v1-uploader: Workers: hashing: 0 | get upload mode: 0 | pre-uploading: 0 | committing: 1 | waiting: 125
chaiml-gspo-glm47-combin-7698-v1-uploader: ---------------------------------------------------
chaiml-gspo-glm47-combin-7698-v1-uploader:                              Processed model ChaiML/gspo-glm47-combine-rm82-zy-data-step100 in 5077.676s
2026-03-19T19:09:14.377784+00:00 monitor updated for chaiml-gspo-glm47-combin_7698_v1
chaiml-gspo-glm47-combin-7698-v1-uploader: creating bucket guanaco-vllm-models
chaiml-gspo-glm47-combin-7698-v1-uploader: /usr/lib/python3/dist-packages/S3/BaseUtils.py:56: SyntaxWarning: invalid escape sequence '\.'
chaiml-gspo-glm47-combin-7698-v1-uploader:   RE_S3_DATESTRING = re.compile('\.[0-9]*(?:[Z\\-\\+]*?)')
chaiml-gspo-glm47-combin-7698-v1-uploader: /usr/lib/python3/dist-packages/S3/BaseUtils.py:57: SyntaxWarning: invalid escape sequence '\s'
chaiml-gspo-glm47-combin-7698-v1-uploader:   RE_XML_NAMESPACE = re.compile(b'^(<?[^>]+?>\s*|\s*)(<\w+) xmlns=[\'"](https?://[^\'"]+)[\'"]', re.MULTILINE)
chaiml-gspo-glm47-combin-7698-v1-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:240: SyntaxWarning: invalid escape sequence '\.'
chaiml-gspo-glm47-combin-7698-v1-uploader:   invalid = re.search("([^a-z0-9\.-])", bucket, re.UNICODE)
chaiml-gspo-glm47-combin-7698-v1-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:244: SyntaxWarning: invalid escape sequence '\.'
chaiml-gspo-glm47-combin-7698-v1-uploader:   invalid = re.search("([^A-Za-z0-9\._-])", bucket, re.UNICODE)
chaiml-gspo-glm47-combin-7698-v1-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:255: SyntaxWarning: invalid escape sequence '\.'
chaiml-gspo-glm47-combin-7698-v1-uploader:   if re.search("-\.", bucket, re.UNICODE):
chaiml-gspo-glm47-combin-7698-v1-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:257: SyntaxWarning: invalid escape sequence '\.'
chaiml-gspo-glm47-combin-7698-v1-uploader:   if re.search("\.\.", bucket, re.UNICODE):
chaiml-gspo-glm47-combin-7698-v1-uploader: /usr/lib/python3/dist-packages/S3/S3Uri.py:155: SyntaxWarning: invalid escape sequence '\w'
chaiml-gspo-glm47-combin-7698-v1-uploader:   _re = re.compile("^(\w+://)?(.*)", re.UNICODE)
chaiml-gspo-glm47-combin-7698-v1-uploader: /usr/lib/python3/dist-packages/S3/FileLists.py:480: SyntaxWarning: invalid escape sequence '\*'
chaiml-gspo-glm47-combin-7698-v1-uploader:   wildcard_split_result = re.split("\*|\?", uri_str, maxsplit=1)
Retrying (%r) after connection broken by '%r': %s
2026-03-19T19:09:46.907542+00:00 monitor updated for chaiml-gspo-glm47-chai-_41286_v1
chaiml-gspo-glm47-combin-7698-v1-uploader: cp /dev/shm/model_output/model-00072-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm47-combin-7698-v1/default/model-00072-of-00072.safetensors
2026-03-19T19:10:14.607634+00:00 monitor updated for chaiml-gspo-glm47-combin_7698_v1
chaiml-gspo-glm47-combin-7698-v1-uploader: cp /dev/shm/model_output/model-00071-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm47-combin-7698-v1/default/model-00071-of-00072.safetensors
chaiml-gspo-glm47-combin-7698-v1-uploader: cp /dev/shm/model_output/model-00070-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm47-combin-7698-v1/default/model-00070-of-00072.safetensors
chaiml-gspo-glm47-combin-7698-v1-uploader: cp /dev/shm/model_output/model-00021-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm47-combin-7698-v1/default/model-00021-of-00072.safetensors
chaiml-gspo-glm47-combin-7698-v1-uploader: cp /dev/shm/model_output/model-00058-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm47-combin-7698-v1/default/model-00058-of-00072.safetensors
chaiml-gspo-glm47-combin-7698-v1-uploader: cp /dev/shm/model_output/model-00041-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm47-combin-7698-v1/default/model-00041-of-00072.safetensors
chaiml-gspo-glm47-combin-7698-v1-uploader: cp /dev/shm/model_output/model-00010-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm47-combin-7698-v1/default/model-00010-of-00072.safetensors
chaiml-gspo-glm47-combin-7698-v1-uploader: cp /dev/shm/model_output/model-00064-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm47-combin-7698-v1/default/model-00064-of-00072.safetensors
chaiml-gspo-glm47-combin-7698-v1-uploader: cp /dev/shm/model_output/model-00005-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm47-combin-7698-v1/default/model-00005-of-00072.safetensors
chaiml-gspo-glm47-combin-7698-v1-uploader: cp /dev/shm/model_output/model-00065-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm47-combin-7698-v1/default/model-00065-of-00072.safetensors
chaiml-gspo-glm47-combin-7698-v1-uploader: cp /dev/shm/model_output/model-00016-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm47-combin-7698-v1/default/model-00016-of-00072.safetensors
2026-03-19T19:10:47.162894+00:00 monitor updated for chaiml-gspo-glm47-chai-_41286_v1
chaiml-gspo-glm47-combin-7698-v1-uploader: cp /dev/shm/model_output/model-00006-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm47-combin-7698-v1/default/model-00006-of-00072.safetensors
chaiml-gspo-glm47-combin-7698-v1-uploader: cp /dev/shm/model_output/model-00031-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm47-combin-7698-v1/default/model-00031-of-00072.safetensors
chaiml-gspo-glm47-combin-7698-v1-uploader: cp /dev/shm/model_output/model-00013-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm47-combin-7698-v1/default/model-00013-of-00072.safetensors
chaiml-gspo-glm47-combin-7698-v1-uploader: cp /dev/shm/model_output/model-00033-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm47-combin-7698-v1/default/model-00033-of-00072.safetensors
chaiml-gspo-glm47-combin-7698-v1-uploader: cp /dev/shm/model_output/model-00034-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm47-combin-7698-v1/default/model-00034-of-00072.safetensors
chaiml-gspo-glm47-combin-7698-v1-uploader: cp /dev/shm/model_output/model-00056-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm47-combin-7698-v1/default/model-00056-of-00072.safetensors
chaiml-gspo-glm47-combin-7698-v1-uploader: cp /dev/shm/model_output/model-00050-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm47-combin-7698-v1/default/model-00050-of-00072.safetensors
chaiml-gspo-glm47-combin-7698-v1-uploader: cp /dev/shm/model_output/model-00057-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm47-combin-7698-v1/default/model-00057-of-00072.safetensors
chaiml-gspo-glm47-combin-7698-v1-uploader: cp /dev/shm/model_output/model-00051-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm47-combin-7698-v1/default/model-00051-of-00072.safetensors
chaiml-gspo-glm47-combin-7698-v1-uploader: cp /dev/shm/model_output/model-00048-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm47-combin-7698-v1/default/model-00048-of-00072.safetensors
chaiml-gspo-glm47-combin-7698-v1-uploader: cp /dev/shm/model_output/model-00020-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm47-combin-7698-v1/default/model-00020-of-00072.safetensors
chaiml-gspo-glm47-combin-7698-v1-uploader: cp /dev/shm/model_output/model-00060-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm47-combin-7698-v1/default/model-00060-of-00072.safetensors
chaiml-gspo-glm47-combin-7698-v1-uploader: cp /dev/shm/model_output/model-00019-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm47-combin-7698-v1/default/model-00019-of-00072.safetensors
chaiml-gspo-glm47-combin-7698-v1-uploader: cp /dev/shm/model_output/model-00045-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm47-combin-7698-v1/default/model-00045-of-00072.safetensors
chaiml-gspo-glm47-combin-7698-v1-uploader: cp /dev/shm/model_output/model-00039-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm47-combin-7698-v1/default/model-00039-of-00072.safetensors
chaiml-gspo-glm47-combin-7698-v1-uploader: cp /dev/shm/model_output/model-00055-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm47-combin-7698-v1/default/model-00055-of-00072.safetensors
chaiml-gspo-glm47-combin-7698-v1-uploader: cp /dev/shm/model_output/model-00025-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm47-combin-7698-v1/default/model-00025-of-00072.safetensors
chaiml-gspo-glm47-combin-7698-v1-uploader: cp /dev/shm/model_output/model-00030-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm47-combin-7698-v1/default/model-00030-of-00072.safetensors
chaiml-gspo-glm47-combin-7698-v1-uploader: cp /dev/shm/model_output/model-00023-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm47-combin-7698-v1/default/model-00023-of-00072.safetensors
chaiml-gspo-glm47-combin-7698-v1-uploader: cp /dev/shm/model_output/model-00053-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm47-combin-7698-v1/default/model-00053-of-00072.safetensors
chaiml-gspo-glm47-combin-7698-v1-uploader: cp /dev/shm/model_output/model-00008-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm47-combin-7698-v1/default/model-00008-of-00072.safetensors
chaiml-gspo-glm47-combin-7698-v1-uploader: cp /dev/shm/model_output/model-00022-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm47-combin-7698-v1/default/model-00022-of-00072.safetensors
chaiml-gspo-glm47-combin-7698-v1-uploader: cp /dev/shm/model_output/model-00040-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm47-combin-7698-v1/default/model-00040-of-00072.safetensors
chaiml-gspo-glm47-combin-7698-v1-uploader: cp /dev/shm/model_output/model-00024-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm47-combin-7698-v1/default/model-00024-of-00072.safetensors
chaiml-gspo-glm47-combin-7698-v1-uploader: cp /dev/shm/model_output/model-00066-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm47-combin-7698-v1/default/model-00066-of-00072.safetensors
chaiml-gspo-glm47-combin-7698-v1-uploader: cp /dev/shm/model_output/model-00002-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm47-combin-7698-v1/default/model-00002-of-00072.safetensors
chaiml-gspo-glm47-combin-7698-v1-uploader: cp /dev/shm/model_output/model-00009-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm47-combin-7698-v1/default/model-00009-of-00072.safetensors
chaiml-gspo-glm47-combin-7698-v1-uploader: cp /dev/shm/model_output/model-00018-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm47-combin-7698-v1/default/model-00018-of-00072.safetensors
chaiml-gspo-glm47-combin-7698-v1-uploader: cp /dev/shm/model_output/model-00004-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm47-combin-7698-v1/default/model-00004-of-00072.safetensors
chaiml-gspo-glm47-combin-7698-v1-uploader: cp /dev/shm/model_output/model-00032-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm47-combin-7698-v1/default/model-00032-of-00072.safetensors
chaiml-gspo-glm47-combin-7698-v1-uploader: cp /dev/shm/model_output/model-00014-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm47-combin-7698-v1/default/model-00014-of-00072.safetensors
chaiml-gspo-glm47-combin-7698-v1-uploader: cp /dev/shm/model_output/model-00063-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm47-combin-7698-v1/default/model-00063-of-00072.safetensors
chaiml-gspo-glm47-combin-7698-v1-uploader: cp /dev/shm/model_output/model-00067-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm47-combin-7698-v1/default/model-00067-of-00072.safetensors
chaiml-gspo-glm47-combin-7698-v1-uploader: cp /dev/shm/model_output/model-00043-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm47-combin-7698-v1/default/model-00043-of-00072.safetensors
chaiml-gspo-glm47-combin-7698-v1-uploader: cp /dev/shm/model_output/model-00007-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm47-combin-7698-v1/default/model-00007-of-00072.safetensors
chaiml-gspo-glm47-combin-7698-v1-uploader: cp /dev/shm/model_output/model-00028-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm47-combin-7698-v1/default/model-00028-of-00072.safetensors
chaiml-gspo-glm47-combin-7698-v1-uploader: cp /dev/shm/model_output/model-00059-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm47-combin-7698-v1/default/model-00059-of-00072.safetensors
chaiml-gspo-glm47-combin-7698-v1-uploader: cp /dev/shm/model_output/model-00027-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm47-combin-7698-v1/default/model-00027-of-00072.safetensors
chaiml-gspo-glm47-combin-7698-v1-uploader: cp /dev/shm/model_output/model-00038-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm47-combin-7698-v1/default/model-00038-of-00072.safetensors
chaiml-gspo-glm47-combin-7698-v1-uploader: cp /dev/shm/model_output/model-00052-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm47-combin-7698-v1/default/model-00052-of-00072.safetensors
chaiml-gspo-glm47-combin-7698-v1-uploader: cp /dev/shm/model_output/model-00061-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm47-combin-7698-v1/default/model-00061-of-00072.safetensors
chaiml-gspo-glm47-combin-7698-v1-uploader: cp /dev/shm/model_output/model-00029-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm47-combin-7698-v1/default/model-00029-of-00072.safetensors
chaiml-gspo-glm47-combin-7698-v1-uploader: cp /dev/shm/model_output/model-00037-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm47-combin-7698-v1/default/model-00037-of-00072.safetensors
chaiml-gspo-glm47-combin-7698-v1-uploader: cp /dev/shm/model_output/model-00069-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm47-combin-7698-v1/default/model-00069-of-00072.safetensors
chaiml-gspo-glm47-combin-7698-v1-uploader: cp /dev/shm/model_output/model-00049-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm47-combin-7698-v1/default/model-00049-of-00072.safetensors
chaiml-gspo-glm47-combin-7698-v1-uploader: cp /dev/shm/model_output/model-00062-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm47-combin-7698-v1/default/model-00062-of-00072.safetensors
chaiml-gspo-glm47-combin-7698-v1-uploader: cp /dev/shm/model_output/model-00054-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm47-combin-7698-v1/default/model-00054-of-00072.safetensors
chaiml-gspo-glm47-combin-7698-v1-uploader: cp /dev/shm/model_output/model-00003-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm47-combin-7698-v1/default/model-00003-of-00072.safetensors
chaiml-gspo-glm47-combin-7698-v1-uploader: cp /dev/shm/model_output/model-00046-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm47-combin-7698-v1/default/model-00046-of-00072.safetensors
chaiml-gspo-glm47-combin-7698-v1-uploader: cp /dev/shm/model_output/model-00026-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm47-combin-7698-v1/default/model-00026-of-00072.safetensors
chaiml-gspo-glm47-combin-7698-v1-uploader: cp /dev/shm/model_output/model-00044-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm47-combin-7698-v1/default/model-00044-of-00072.safetensors
chaiml-gspo-glm47-combin-7698-v1-uploader: cp /dev/shm/model_output/model-00011-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm47-combin-7698-v1/default/model-00011-of-00072.safetensors
chaiml-gspo-glm47-combin-7698-v1-uploader: cp /dev/shm/model_output/model-00035-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm47-combin-7698-v1/default/model-00035-of-00072.safetensors
chaiml-gspo-glm47-combin-7698-v1-uploader: cp /dev/shm/model_output/model-00001-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm47-combin-7698-v1/default/model-00001-of-00072.safetensors
chaiml-gspo-glm47-combin-7698-v1-uploader: cp /dev/shm/model_output/model-00042-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm47-combin-7698-v1/default/model-00042-of-00072.safetensors
chaiml-gspo-glm47-combin-7698-v1-uploader: cp /dev/shm/model_output/model-00036-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm47-combin-7698-v1/default/model-00036-of-00072.safetensors
chaiml-gspo-glm47-combin-7698-v1-uploader: cp /dev/shm/model_output/model-00015-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm47-combin-7698-v1/default/model-00015-of-00072.safetensors
chaiml-gspo-glm47-combin-7698-v1-uploader: cp /dev/shm/model_output/model-00012-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm47-combin-7698-v1/default/model-00012-of-00072.safetensors
chaiml-gspo-glm47-combin-7698-v1-uploader: cp /dev/shm/model_output/model-00047-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm47-combin-7698-v1/default/model-00047-of-00072.safetensors
chaiml-gspo-glm47-combin-7698-v1-uploader: cp /dev/shm/model_output/model-00068-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm47-combin-7698-v1/default/model-00068-of-00072.safetensors
chaiml-gspo-glm47-combin-7698-v1-uploader: cp /dev/shm/model_output/model-00017-of-00072.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm47-combin-7698-v1/default/model-00017-of-00072.safetensors
Job chaiml-gspo-glm47-combin-7698-v1-uploader completed after 5242.08s with status: succeeded
Stopping job with name chaiml-gspo-glm47-combin-7698-v1-uploader
Pipeline stage VLLMUploader completed in 5242.93s
run pipeline stage %s
Running pipeline stage VLLMTemplater
Pipeline stage VLLMTemplater completed in 0.60s
run pipeline stage %s
Running pipeline stage VLLMDeployer
Creating inference service chaiml-gspo-glm47-combin-7698-v1
Waiting for inference service chaiml-gspo-glm47-combin-7698-v1 to be ready
2026-03-19T19:11:14.863406+00:00 monitor updated for chaiml-gspo-glm47-combin_7698_v1
2026-03-19T19:11:47.647570+00:00 monitor updated for chaiml-gspo-glm47-chai-_41286_v1
2026-03-19T19:12:15.120491+00:00 monitor updated for chaiml-gspo-glm47-combin_7698_v1
2026-03-19T19:12:47.898295+00:00 monitor updated for chaiml-gspo-glm47-chai-_41286_v1
2026-03-19T19:13:15.381167+00:00 monitor updated for chaiml-gspo-glm47-combin_7698_v1
2026-03-19T19:13:48.145754+00:00 monitor updated for chaiml-gspo-glm47-chai-_41286_v1
2026-03-19T19:14:15.732499+00:00 monitor updated for chaiml-gspo-glm47-combin_7698_v1
2026-03-19T19:14:48.371676+00:00 monitor updated for chaiml-gspo-glm47-chai-_41286_v1
2026-03-19T19:15:15.972300+00:00 monitor updated for chaiml-gspo-glm47-combin_7698_v1
2026-03-19T19:15:48.593602+00:00 monitor updated for chaiml-gspo-glm47-chai-_41286_v1
2026-03-19T19:16:16.207952+00:00 monitor updated for chaiml-gspo-glm47-combin_7698_v1
2026-03-19T19:16:48.813295+00:00 monitor updated for chaiml-gspo-glm47-chai-_41286_v1
2026-03-19T19:17:16.438264+00:00 monitor updated for chaiml-gspo-glm47-combin_7698_v1
2026-03-19T19:17:49.036636+00:00 monitor updated for chaiml-gspo-glm47-chai-_41286_v1
2026-03-19T19:18:16.674915+00:00 monitor updated for chaiml-gspo-glm47-combin_7698_v1
2026-03-19T19:18:49.269167+00:00 monitor updated for chaiml-gspo-glm47-chai-_41286_v1
2026-03-19T19:19:17.209577+00:00 monitor updated for chaiml-gspo-glm47-combin_7698_v1
2026-03-19T19:19:49.492616+00:00 monitor updated for chaiml-gspo-glm47-chai-_41286_v1
2026-03-19T19:20:17.448860+00:00 monitor updated for chaiml-gspo-glm47-combin_7698_v1
2026-03-19T19:20:49.729066+00:00 monitor updated for chaiml-gspo-glm47-chai-_41286_v1
2026-03-19T19:21:17.684015+00:00 monitor updated for chaiml-gspo-glm47-combin_7698_v1
2026-03-19T19:21:49.955473+00:00 monitor updated for chaiml-gspo-glm47-chai-_41286_v1
2026-03-19T19:22:17.895011+00:00 monitor updated for chaiml-gspo-glm47-combin_7698_v1
2026-03-19T19:22:50.155264+00:00 monitor updated for chaiml-gspo-glm47-chai-_41286_v1
2026-03-19T19:23:18.117829+00:00 monitor updated for chaiml-gspo-glm47-combin_7698_v1
2026-03-19T19:23:50.642114+00:00 monitor updated for chaiml-gspo-glm47-chai-_41286_v1
2026-03-19T19:24:18.333915+00:00 monitor updated for chaiml-gspo-glm47-combin_7698_v1
2026-03-19T19:24:50.866281+00:00 monitor updated for chaiml-gspo-glm47-chai-_41286_v1
2026-03-19T19:25:18.567639+00:00 monitor updated for chaiml-gspo-glm47-combin_7698_v1
2026-03-19T19:25:51.093924+00:00 monitor updated for chaiml-gspo-glm47-chai-_41286_v1
2026-03-19T19:26:18.801483+00:00 monitor updated for chaiml-gspo-glm47-combin_7698_v1
2026-03-19T19:26:51.312918+00:00 monitor updated for chaiml-gspo-glm47-chai-_41286_v1
2026-03-19T19:27:19.045133+00:00 monitor updated for chaiml-gspo-glm47-combin_7698_v1
2026-03-19T19:27:51.541519+00:00 monitor updated for chaiml-gspo-glm47-chai-_41286_v1
Inference service chaiml-gspo-glm47-combin-7698-v1 ready after 1003.8598146438599s
Pipeline stage VLLMDeployer completed in 1005.54s
run pipeline stage %s
Running pipeline stage StressChecker
Received healthy response to inference request in 9.122750520706177s
Received healthy response to inference request in 2.0565497875213623s
Received healthy response to inference request in 2.1879475116729736s
Received healthy response to inference request in 2.0656776428222656s
Received healthy response to inference request in 2.0741522312164307s
2026-03-19T19:28:19.289255+00:00 monitor updated for chaiml-gspo-glm47-combin_7698_v1
Received healthy response to inference request in 2.0158438682556152s
Received healthy response to inference request in 1.980372428894043s
Received healthy response to inference request in 2.101872444152832s
Received healthy response to inference request in 2.0713765621185303s
Received healthy response to inference request in 2.0166425704956055s
Received healthy response to inference request in 2.0397861003875732s
Received healthy response to inference request in 2.113605499267578s
Received healthy response to inference request in 2.000307083129883s
Received healthy response to inference request in 2.0629513263702393s
Received healthy response to inference request in 1.9933092594146729s
Received healthy response to inference request in 2.400967836380005s
Received healthy response to inference request in 2.0178380012512207s
Received healthy response to inference request in 2.1933019161224365s
Received healthy response to inference request in 2.0981667041778564s
2026-03-19T19:28:51.764485+00:00 monitor updated for chaiml-gspo-glm47-chai-_41286_v1
Received healthy response to inference request in 2.031200885772705s
Received healthy response to inference request in 2.029702663421631s
Received healthy response to inference request in 2.253248453140259s
Received healthy response to inference request in 1.9756948947906494s
Received healthy response to inference request in 2.0623223781585693s
Received healthy response to inference request in 2.026646852493286s
Received healthy response to inference request in 3.602764844894409s
Received healthy response to inference request in 2.1248860359191895s
Received healthy response to inference request in 2.0065836906433105s
Received healthy response to inference request in 1.9943881034851074s
Received healthy response to inference request in 2.1801958084106445s
30 requests
0 failed requests
5th percentile: 1.9861940026283265
10th percentile: 1.994280219078064
20th percentile: 2.0139918327331543
30th percentile: 2.0240041971206666
40th percentile: 2.036352014541626
50th percentile: 2.0626368522644043
60th percentile: 2.0724868297576906
70th percentile: 2.1053923606872558
80th percentile: 2.1817461490631103
90th percentile: 2.2680203914642334
95th percentile: 3.061956191062924
2026-03-19T19:29:19.536708+00:00 monitor updated for chaiml-gspo-glm47-combin_7698_v1
99th percentile: 7.521954674720769
mean time: 2.3633684635162355
Pipeline stage StressChecker completed in 81.67s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyTriggerPipeline
run_pipeline:run_in_cloud %s
starting trigger_guanaco_pipeline args=%s
triggered trigger_guanaco_pipeline args=%s
Pipeline stage OfflineFamilyFriendlyTriggerPipeline completed in 1.47s
Shutdown handler de-registered
chaiml-gspo-glm47-combin_7698_v1 status is now deployed due to DeploymentManager action
chaiml-gspo-glm47-combin_7698_v1 status is now inactive due to auto deactivation removed underperforming models
chaiml-gspo-glm47-combin_7698_v1 status is now torndown due to DeploymentManager action