developer_uid: chai_backend_admin
submission_id: chaiml-gspo-glm47-kimi-_59551_v2
model_name: chaiml-gspo-glm47-kimi-_59551_v2
model_group: ChaiML/gspo-glm47-kimi-r
status: inactive
timestamp: 2026-04-08T20:28:31+00:00
num_battles: 10512
num_wins: 6027
celo_rating: 1355.56
family_friendly_score: 0.0
family_friendly_standard_error: 0.0
submission_type: basic
model_repo: ChaiML/gspo-glm47-kimi-rm-pony-data-apr02-step400
model_architecture: Glm4MoeForCausalLM
model_num_parameters: 24110003200.0
best_of: 8
max_input_tokens: 1500
max_output_tokens: 80
reward_model: default
display_name: chaiml-gspo-glm47-kimi-_59551_v2
ineligible_reason: max_output_tokens!=64
is_internal_developer: True
language_model: ChaiML/gspo-glm47-kimi-rm-pony-data-apr02-step400
model_size: 24B
ranking_group: single
us_pacific_date: 2026-04-08
win_ratio: 0.5733447488584474
generation_params: {'temperature': 1.0, 'top_p': 0.95, 'min_p': 0.0, 'top_k': 60, 'presence_penalty': 0.0, 'frequency_penalty': 0.0, 'stopping_words': ['<|im_end|>', '####', '<|assistant|>', '</s>', '<|user|>'], 'max_input_tokens': 1500, 'best_of': 8, 'max_output_tokens': 80}
formatter: {'memory_template': "[gMASK]<sop><|system|>{bot_name}'s persona: {memory}", 'prompt_template': '', 'bot_template': '<|assistant|>{message}', 'user_template': '<|user|>{message}', 'response_template': '<|assistant|></think>', 'truncate_by_message': True}
Resubmit model
Shutdown handler not registered because Python interpreter is not running in the main thread
run pipeline %s
run pipeline stage %s
Running pipeline stage VLLMUploader
Starting job with name chaiml-gspo-glm47-kimi-59551-v2-uploader
Waiting for job on chaiml-gspo-glm47-kimi-59551-v2-uploader to finish
Failed to get response for submission chaiml-gspo-glm47-kimi-r_8813_v1: ('http://chaiml-gspo-glm47-kimi-r-8813-v1-predictor.tenant-chaiml-guanaco.kchai-coreweave-us-east-04a.chaiverse.com/v1/completions', 'activator request timeout')
chaiml-gspo-glm47-kimi-59551-v2-uploader: Using quantization_mode: fp8
chaiml-gspo-glm47-kimi-59551-v2-uploader: Checking if ChaiML/gspo-glm47-kimi-rm-pony-data-apr02-step400-FP8 already exists in ChaiML
chaiml-gspo-glm47-kimi-59551-v2-uploader: Model already exists. Downloading to /tmp/model_output...
chaiml-gspo-glm47-kimi-59551-v2-uploader: Downloading snapshot of ChaiML/gspo-glm47-kimi-rm-pony-data-apr02-step400-FP8...
2026-04-08T15:03:49.211871+00:00 monitor updated for chaiml-gspo-glm47-kimi-_59551_v2
2026-04-08T15:04:49.321993+00:00 monitor updated for chaiml-gspo-glm47-kimi-_59551_v2
2026-04-08T15:05:49.425194+00:00 monitor updated for chaiml-gspo-glm47-kimi-_59551_v2
2026-04-08T15:06:49.526873+00:00 monitor updated for chaiml-gspo-glm47-kimi-_59551_v2
chaiml-gspo-glm47-kimi-59551-v2-uploader: Downloaded in 204.836s
chaiml-gspo-glm47-kimi-59551-v2-uploader: Processed model ChaiML/gspo-glm47-kimi-rm-pony-data-apr02-step400 in 207.871s
2026-04-08T15:07:49.641247+00:00 monitor updated for chaiml-gspo-glm47-kimi-_59551_v2
chaiml-gspo-glm47-kimi-59551-v2-uploader: creating bucket guanaco-vllm-models
chaiml-gspo-glm47-kimi-59551-v2-uploader: /usr/lib/python3/dist-packages/S3/BaseUtils.py:56: SyntaxWarning: invalid escape sequence '\.'
chaiml-gspo-glm47-kimi-59551-v2-uploader:   RE_S3_DATESTRING = re.compile('\.[0-9]*(?:[Z\\-\\+]*?)')
chaiml-gspo-glm47-kimi-59551-v2-uploader: /usr/lib/python3/dist-packages/S3/BaseUtils.py:57: SyntaxWarning: invalid escape sequence '\s'
chaiml-gspo-glm47-kimi-59551-v2-uploader:   RE_XML_NAMESPACE = re.compile(b'^(<?[^>]+?>\s*|\s*)(<\w+) xmlns=[\'"](https?://[^\'"]+)[\'"]', re.MULTILINE)
chaiml-gspo-glm47-kimi-59551-v2-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:240: SyntaxWarning: invalid escape sequence '\.'
chaiml-gspo-glm47-kimi-59551-v2-uploader:   invalid = re.search("([^a-z0-9\.-])", bucket, re.UNICODE)
chaiml-gspo-glm47-kimi-59551-v2-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:244: SyntaxWarning: invalid escape sequence '\.'
chaiml-gspo-glm47-kimi-59551-v2-uploader:   invalid = re.search("([^A-Za-z0-9\._-])", bucket, re.UNICODE)
chaiml-gspo-glm47-kimi-59551-v2-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:255: SyntaxWarning: invalid escape sequence '\.'
chaiml-gspo-glm47-kimi-59551-v2-uploader:   if re.search("-\.", bucket, re.UNICODE):
chaiml-gspo-glm47-kimi-59551-v2-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:257: SyntaxWarning: invalid escape sequence '\.'
chaiml-gspo-glm47-kimi-59551-v2-uploader:   if re.search("\.\.", bucket, re.UNICODE):
chaiml-gspo-glm47-kimi-59551-v2-uploader: /usr/lib/python3/dist-packages/S3/S3Uri.py:155: SyntaxWarning: invalid escape sequence '\w'
chaiml-gspo-glm47-kimi-59551-v2-uploader:   _re = re.compile("^(\w+://)?(.*)", re.UNICODE)
chaiml-gspo-glm47-kimi-59551-v2-uploader: /usr/lib/python3/dist-packages/S3/FileLists.py:480: SyntaxWarning: invalid escape sequence '\*'
chaiml-gspo-glm47-kimi-59551-v2-uploader:   wildcard_split_result = re.split("\*|\?", uri_str, maxsplit=1)
chaiml-gspo-glm47-kimi-59551-v2-uploader: Bucket 's3://guanaco-vllm-models/' created
chaiml-gspo-glm47-kimi-59551-v2-uploader: uploading /tmp/model_output to s3://guanaco-vllm-models/chaiml-gspo-glm47-kimi-59551-v2/default
chaiml-gspo-glm47-kimi-59551-v2-uploader: cp /tmp/model_output/chat_template.jinja s3://guanaco-vllm-models/chaiml-gspo-glm47-kimi-59551-v2/default/chat_template.jinja
chaiml-gspo-glm47-kimi-59551-v2-uploader: cp /tmp/model_output/.gitattributes s3://guanaco-vllm-models/chaiml-gspo-glm47-kimi-59551-v2/default/.gitattributes
chaiml-gspo-glm47-kimi-59551-v2-uploader: cp /tmp/model_output/generation_config.json s3://guanaco-vllm-models/chaiml-gspo-glm47-kimi-59551-v2/default/generation_config.json
chaiml-gspo-glm47-kimi-59551-v2-uploader: cp /tmp/model_output/recipe.yaml s3://guanaco-vllm-models/chaiml-gspo-glm47-kimi-59551-v2/default/recipe.yaml
chaiml-gspo-glm47-kimi-59551-v2-uploader: cp /tmp/model_output/tokenizer_config.json s3://guanaco-vllm-models/chaiml-gspo-glm47-kimi-59551-v2/default/tokenizer_config.json
chaiml-gspo-glm47-kimi-59551-v2-uploader: cp /tmp/model_output/config.json s3://guanaco-vllm-models/chaiml-gspo-glm47-kimi-59551-v2/default/config.json
chaiml-gspo-glm47-kimi-59551-v2-uploader: cp /tmp/model_output/model.safetensors.index.json s3://guanaco-vllm-models/chaiml-gspo-glm47-kimi-59551-v2/default/model.safetensors.index.json
chaiml-gspo-glm47-kimi-59551-v2-uploader: cp /tmp/model_output/tokenizer.json s3://guanaco-vllm-models/chaiml-gspo-glm47-kimi-59551-v2/default/tokenizer.json
2026-04-08T15:08:49.730818+00:00 monitor updated for chaiml-gspo-glm47-kimi-_59551_v2
Failed to get response for submission chaiml-gspo-glm47-kimi-_44097_v1: HTTPConnectionPool(host='chaiml-gspo-glm47-kimi-44097-v1-predictor.tenant-chaiml-guanaco.kchai-coreweave-us-east-04a.chaiverse.com', port=80): Read timed out. (read timeout=20.0)
chaiml-gspo-glm47-kimi-59551-v2-uploader: cp /tmp/model_output/model-00015-of-00015.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm47-kimi-59551-v2/default/model-00015-of-00015.safetensors
2026-04-08T15:09:49.817423+00:00 monitor updated for chaiml-gspo-glm47-kimi-_59551_v2
chaiml-gspo-glm47-kimi-59551-v2-uploader: cp /tmp/model_output/model-00012-of-00015.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm47-kimi-59551-v2/default/model-00012-of-00015.safetensors
chaiml-gspo-glm47-kimi-59551-v2-uploader: cp /tmp/model_output/model-00010-of-00015.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm47-kimi-59551-v2/default/model-00010-of-00015.safetensors
chaiml-gspo-glm47-kimi-59551-v2-uploader: cp /tmp/model_output/model-00014-of-00015.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm47-kimi-59551-v2/default/model-00014-of-00015.safetensors
chaiml-gspo-glm47-kimi-59551-v2-uploader: cp /tmp/model_output/model-00011-of-00015.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm47-kimi-59551-v2/default/model-00011-of-00015.safetensors
chaiml-gspo-glm47-kimi-59551-v2-uploader: cp /tmp/model_output/model-00013-of-00015.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm47-kimi-59551-v2/default/model-00013-of-00015.safetensors
chaiml-gspo-glm47-kimi-59551-v2-uploader: cp /tmp/model_output/model-00009-of-00015.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm47-kimi-59551-v2/default/model-00009-of-00015.safetensors
chaiml-gspo-glm47-kimi-59551-v2-uploader: cp /tmp/model_output/model-00008-of-00015.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm47-kimi-59551-v2/default/model-00008-of-00015.safetensors
chaiml-gspo-glm47-kimi-59551-v2-uploader: cp /tmp/model_output/model-00007-of-00015.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm47-kimi-59551-v2/default/model-00007-of-00015.safetensors
chaiml-gspo-glm47-kimi-59551-v2-uploader: cp /tmp/model_output/model-00005-of-00015.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm47-kimi-59551-v2/default/model-00005-of-00015.safetensors
chaiml-gspo-glm47-kimi-59551-v2-uploader: cp /tmp/model_output/model-00004-of-00015.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm47-kimi-59551-v2/default/model-00004-of-00015.safetensors
chaiml-gspo-glm47-kimi-59551-v2-uploader: cp /tmp/model_output/model-00003-of-00015.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm47-kimi-59551-v2/default/model-00003-of-00015.safetensors
chaiml-gspo-glm47-kimi-59551-v2-uploader: cp /tmp/model_output/model-00006-of-00015.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm47-kimi-59551-v2/default/model-00006-of-00015.safetensors
2026-04-08T15:10:49.930684+00:00 monitor updated for chaiml-gspo-glm47-kimi-_59551_v2
chaiml-gspo-glm47-kimi-59551-v2-uploader: cp /tmp/model_output/model-00002-of-00015.safetensors s3://guanaco-vllm-models/chaiml-gspo-glm47-kimi-59551-v2/default/model-00002-of-00015.safetensors
Connection pool is full, discarding connection: %s. Connection pool size: %s
Job chaiml-gspo-glm47-kimi-59551-v2-uploader completed after 512.09s with status: succeeded
Stopping job with name chaiml-gspo-glm47-kimi-59551-v2-uploader
Pipeline stage VLLMUploader completed in 512.59s
run pipeline stage %s
Running pipeline stage VLLMUploaderAMD
Pipeline stage vllm_upload_amd skipped, reason=not amd cluster
Pipeline stage VLLMUploaderAMD completed in 0.09s
run pipeline stage %s
Running pipeline stage VLLMTemplater
Pipeline stage VLLMTemplater completed in 2.07s
run pipeline stage %s
Running pipeline stage VLLMDeployer
Creating inference service chaiml-gspo-glm47-kimi-59551-v2
Waiting for inference service chaiml-gspo-glm47-kimi-59551-v2 to be ready
2026-04-08T15:11:50.021507+00:00 monitor updated for chaiml-gspo-glm47-kimi-_59551_v2
2026-04-08T15:12:50.107418+00:00 monitor updated for chaiml-gspo-glm47-kimi-_59551_v2
Failed to get response for submission chaiml-gspo-glm47-kimi-_44097_v1: ('http://chaiml-gspo-glm47-kimi-44097-v1-predictor.tenant-chaiml-guanaco.kchai-coreweave-us-east-04a.chaiverse.com/v1/completions', 'activator request timeout')
2026-04-08T15:13:50.227289+00:00 monitor updated for chaiml-gspo-glm47-kimi-_59551_v2
2026-04-08T15:14:50.324913+00:00 monitor updated for chaiml-gspo-glm47-kimi-_59551_v2
2026-04-08T15:15:50.472072+00:00 monitor updated for chaiml-gspo-glm47-kimi-_59551_v2
2026-04-08T15:16:50.598019+00:00 monitor updated for chaiml-gspo-glm47-kimi-_59551_v2
2026-04-08T15:17:50.696401+00:00 monitor updated for chaiml-gspo-glm47-kimi-_59551_v2
2026-04-08T15:18:51.213971+00:00 monitor updated for chaiml-gspo-glm47-kimi-_59551_v2
Failed to get response for submission chaiml-gspo-glm47-kimi-r_8813_v1: ('http://chaiml-gspo-glm47-kimi-r-8813-v1-predictor.tenant-chaiml-guanaco.kchai-coreweave-us-east-04a.chaiverse.com/v1/completions', 'activator request timeout')
2026-04-08T15:19:51.368315+00:00 monitor updated for chaiml-gspo-glm47-kimi-_59551_v2
Inference service chaiml-gspo-glm47-kimi-59551-v2 ready after 563.7614259719849s
Pipeline stage VLLMDeployer completed in 564.21s
run pipeline stage %s
Running pipeline stage StressChecker
2026-04-08T15:20:51.493671+00:00 monitor updated for chaiml-gspo-glm47-kimi-_59551_v2
Received healthy response to inference request in 10.625190258026123s
Received healthy response to inference request in 2.1761298179626465s
Received healthy response to inference request in 2.189481496810913s
Received healthy response to inference request in 2.1471807956695557s
Received healthy response to inference request in 2.0992116928100586s
Received healthy response to inference request in 2.392634391784668s
Received healthy response to inference request in 2.136986017227173s
Received healthy response to inference request in 1.9831297397613525s
Received healthy response to inference request in 2.1178128719329834s
Received healthy response to inference request in 1.9973938465118408s
Received healthy response to inference request in 2.2092316150665283s
Received healthy response to inference request in 2.3341898918151855s
Received healthy response to inference request in 2.24546217918396s
Received healthy response to inference request in 3.859163999557495s
Received healthy response to inference request in 2.1177449226379395s
Received healthy response to inference request in 2.1070642471313477s
Received healthy response to inference request in 2.086684465408325s
Received healthy response to inference request in 2.3266210556030273s
Received healthy response to inference request in 1.9910991191864014s
Received healthy response to inference request in 1.9878027439117432s
Received healthy response to inference request in 1.992441177368164s
Received healthy response to inference request in 2.4913926124572754s
Received healthy response to inference request in 2.108767032623291s
2026-04-08T15:21:51.739770+00:00 monitor updated for chaiml-gspo-glm47-kimi-_59551_v2
Received healthy response to inference request in 2.282670736312866s
Received healthy response to inference request in 2.0990517139434814s
Received healthy response to inference request in 2.017181634902954s
Received healthy response to inference request in 2.0223755836486816s
Received healthy response to inference request in 2.059812307357788s
Received healthy response to inference request in 2.18280029296875s
Received healthy response to inference request in 2.1199464797973633s
30 requests
0 failed requests
5th percentile: 1.9892861127853394
10th percentile: 1.9923069715499877
20th percentile: 2.021336793899536
30th percentile: 2.0953415393829347
40th percentile: 2.1080859184265135
50th percentile: 2.1188796758651733
60th percentile: 2.158760404586792
70th percentile: 2.1954065322875977
80th percentile: 2.2914608001708987
90th percentile: 2.402510213851929
95th percentile: 3.2436668753623925
99th percentile: 8.663042643070227
mean time: 2.4835551579793296
Pipeline stage StressChecker completed in 77.47s
Shutdown handler de-registered
chaiml-gspo-glm47-kimi-_59551_v2 status is now deployed due to DeploymentManager action
chaiml-gspo-glm47-kimi-_59551_v2 status is now inactive due to auto deactivation removed underperforming models