developer_uid: acehao-chai
submission_id: chaiml-grpo-q235b-opusd_34252_v2
model_name: chaiml-grpo-q235b-opusd_34252_v2
model_group: ChaiML/grpo-q235b-opusd-
status: torndown
timestamp: 2026-03-11T17:33:27+00:00
num_battles: 10484
num_wins: 5596
celo_rating: 1322.07
family_friendly_score: 0.0
family_friendly_standard_error: 0.0
submission_type: basic
model_repo: ChaiML/grpo-q235b-opusd-v1-merged-chai-rm-step-200
model_architecture: Qwen3MoeForCausalLM
model_num_parameters: 18790207488.0
best_of: 8
max_input_tokens: 2048
max_output_tokens: 80
reward_model: default
display_name: chaiml-grpo-q235b-opusd_34252_v2
ineligible_reason: max_output_tokens!=64
is_internal_developer: False
language_model: ChaiML/grpo-q235b-opusd-v1-merged-chai-rm-step-200
model_size: 19B
ranking_group: single
us_pacific_date: 2026-02-20
win_ratio: 0.5337657382678367
generation_params: {'temperature': 1.0, 'top_p': 1.0, 'min_p': 0.0, 'top_k': 40, 'presence_penalty': 0.0, 'frequency_penalty': 0.0, 'stopping_words': ['<|assistant|>', '<|im_end|>', '####', '<|user|>', '</think>', '</s>'], 'max_input_tokens': 2048, 'best_of': 8, 'max_output_tokens': 80}
formatter: {'memory_template': "<|im_start|>system\n{bot_name}'s persona: {memory}<|im_end|>\n", 'prompt_template': '', 'bot_template': '<|im_start|>assistant\n{bot_name}: {message}<|im_end|>\n', 'user_template': '<|im_start|>user\n{message}<|im_end|>\n', 'response_template': '<|im_start|>assistant\n{bot_name}:', 'truncate_by_message': True}
Resubmit model
Shutdown handler not registered because Python interpreter is not running in the main thread
run pipeline %s
run pipeline stage %s
Running pipeline stage VLLMUploader
Starting job with name chaiml-grpo-q235b-opusd-34252-v2-uploader
Waiting for job on chaiml-grpo-q235b-opusd-34252-v2-uploader to finish
chaiml-grpo-q235b-opusd-34252-v2-uploader: Using quantization_mode: w4a16
chaiml-grpo-q235b-opusd-34252-v2-uploader: Checking if ChaiML/grpo-q235b-opusd-v1-merged-chai-rm-step-200-W4A16 already exists in ChaiML
chaiml-grpo-q235b-opusd-34252-v2-uploader: Model already exists. Downloading to /dev/shm/model_output...
chaiml-grpo-q235b-opusd-34252-v2-uploader: Downloading snapshot of ChaiML/grpo-q235b-opusd-v1-merged-chai-rm-step-200-W4A16...
chaiml-grpo-q235b-opusd-34252-v2-uploader: Downloaded in 53.606s
chaiml-grpo-q235b-opusd-34252-v2-uploader: Processed model ChaiML/grpo-q235b-opusd-v1-merged-chai-rm-step-200 in 54.226s
chaiml-grpo-q235b-opusd-34252-v2-uploader: creating bucket guanaco-vllm-models
chaiml-grpo-q235b-opusd-34252-v2-uploader: /usr/lib/python3/dist-packages/S3/BaseUtils.py:56: SyntaxWarning: invalid escape sequence '\.'
chaiml-grpo-q235b-opusd-34252-v2-uploader:   RE_S3_DATESTRING = re.compile('\.[0-9]*(?:[Z\\-\\+]*?)')
chaiml-grpo-q235b-opusd-34252-v2-uploader: /usr/lib/python3/dist-packages/S3/BaseUtils.py:57: SyntaxWarning: invalid escape sequence '\s'
chaiml-grpo-q235b-opusd-34252-v2-uploader:   RE_XML_NAMESPACE = re.compile(b'^(<?[^>]+?>\s*|\s*)(<\w+) xmlns=[\'"](https?://[^\'"]+)[\'"]', re.MULTILINE)
chaiml-grpo-q235b-opusd-34252-v2-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:240: SyntaxWarning: invalid escape sequence '\.'
chaiml-grpo-q235b-opusd-34252-v2-uploader:   invalid = re.search("([^a-z0-9\.-])", bucket, re.UNICODE)
chaiml-grpo-q235b-opusd-34252-v2-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:244: SyntaxWarning: invalid escape sequence '\.'
chaiml-grpo-q235b-opusd-34252-v2-uploader:   invalid = re.search("([^A-Za-z0-9\._-])", bucket, re.UNICODE)
chaiml-grpo-q235b-opusd-34252-v2-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:255: SyntaxWarning: invalid escape sequence '\.'
chaiml-grpo-q235b-opusd-34252-v2-uploader:   if re.search("-\.", bucket, re.UNICODE):
chaiml-grpo-q235b-opusd-34252-v2-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:257: SyntaxWarning: invalid escape sequence '\.'
chaiml-grpo-q235b-opusd-34252-v2-uploader:   if re.search("\.\.", bucket, re.UNICODE):
chaiml-grpo-q235b-opusd-34252-v2-uploader: /usr/lib/python3/dist-packages/S3/S3Uri.py:155: SyntaxWarning: invalid escape sequence '\w'
chaiml-grpo-q235b-opusd-34252-v2-uploader:   _re = re.compile("^(\w+://)?(.*)", re.UNICODE)
chaiml-grpo-q235b-opusd-34252-v2-uploader: /usr/lib/python3/dist-packages/S3/FileLists.py:480: SyntaxWarning: invalid escape sequence '\*'
chaiml-grpo-q235b-opusd-34252-v2-uploader:   wildcard_split_result = re.split("\*|\?", uri_str, maxsplit=1)
chaiml-grpo-q235b-opusd-34252-v2-uploader: Bucket 's3://guanaco-vllm-models/' created
chaiml-grpo-q235b-opusd-34252-v2-uploader: uploading /dev/shm/model_output to s3://guanaco-vllm-models/chaiml-grpo-q235b-opusd-34252-v2/default
chaiml-grpo-q235b-opusd-34252-v2-uploader: cp /dev/shm/model_output/.gitattributes s3://guanaco-vllm-models/chaiml-grpo-q235b-opusd-34252-v2/default/.gitattributes
chaiml-grpo-q235b-opusd-34252-v2-uploader: cp /dev/shm/model_output/added_tokens.json s3://guanaco-vllm-models/chaiml-grpo-q235b-opusd-34252-v2/default/added_tokens.json
chaiml-grpo-q235b-opusd-34252-v2-uploader: cp /dev/shm/model_output/tokenizer_config.json s3://guanaco-vllm-models/chaiml-grpo-q235b-opusd-34252-v2/default/tokenizer_config.json
chaiml-grpo-q235b-opusd-34252-v2-uploader: cp /dev/shm/model_output/chat_template.jinja s3://guanaco-vllm-models/chaiml-grpo-q235b-opusd-34252-v2/default/chat_template.jinja
chaiml-grpo-q235b-opusd-34252-v2-uploader: cp /dev/shm/model_output/quantization_config.json s3://guanaco-vllm-models/chaiml-grpo-q235b-opusd-34252-v2/default/quantization_config.json
chaiml-grpo-q235b-opusd-34252-v2-uploader: cp /dev/shm/model_output/config.json s3://guanaco-vllm-models/chaiml-grpo-q235b-opusd-34252-v2/default/config.json
chaiml-grpo-q235b-opusd-34252-v2-uploader: cp /dev/shm/model_output/generation_config.json s3://guanaco-vllm-models/chaiml-grpo-q235b-opusd-34252-v2/default/generation_config.json
chaiml-grpo-q235b-opusd-34252-v2-uploader: cp /dev/shm/model_output/special_tokens_map.json s3://guanaco-vllm-models/chaiml-grpo-q235b-opusd-34252-v2/default/special_tokens_map.json
chaiml-grpo-q235b-opusd-34252-v2-uploader: cp /dev/shm/model_output/merges.txt s3://guanaco-vllm-models/chaiml-grpo-q235b-opusd-34252-v2/default/merges.txt
chaiml-grpo-q235b-opusd-34252-v2-uploader: cp /dev/shm/model_output/vocab.json s3://guanaco-vllm-models/chaiml-grpo-q235b-opusd-34252-v2/default/vocab.json
chaiml-grpo-q235b-opusd-34252-v2-uploader: cp /dev/shm/model_output/model.safetensors.index.json s3://guanaco-vllm-models/chaiml-grpo-q235b-opusd-34252-v2/default/model.safetensors.index.json
chaiml-grpo-q235b-opusd-34252-v2-uploader: cp /dev/shm/model_output/tokenizer.json s3://guanaco-vllm-models/chaiml-grpo-q235b-opusd-34252-v2/default/tokenizer.json
chaiml-grpo-q235b-opusd-34252-v2-uploader: cp /dev/shm/model_output/model-00027-of-00027.safetensors s3://guanaco-vllm-models/chaiml-grpo-q235b-opusd-34252-v2/default/model-00027-of-00027.safetensors
chaiml-grpo-q235b-opusd-34252-v2-uploader: cp /dev/shm/model_output/model-00005-of-00027.safetensors s3://guanaco-vllm-models/chaiml-grpo-q235b-opusd-34252-v2/default/model-00005-of-00027.safetensors
chaiml-grpo-q235b-opusd-34252-v2-uploader: cp /dev/shm/model_output/model-00007-of-00027.safetensors s3://guanaco-vllm-models/chaiml-grpo-q235b-opusd-34252-v2/default/model-00007-of-00027.safetensors
chaiml-grpo-q235b-opusd-34252-v2-uploader: cp /dev/shm/model_output/model-00001-of-00027.safetensors s3://guanaco-vllm-models/chaiml-grpo-q235b-opusd-34252-v2/default/model-00001-of-00027.safetensors
chaiml-grpo-q235b-opusd-34252-v2-uploader: cp /dev/shm/model_output/model-00020-of-00027.safetensors s3://guanaco-vllm-models/chaiml-grpo-q235b-opusd-34252-v2/default/model-00020-of-00027.safetensors
chaiml-grpo-q235b-opusd-34252-v2-uploader: cp /dev/shm/model_output/model-00016-of-00027.safetensors s3://guanaco-vllm-models/chaiml-grpo-q235b-opusd-34252-v2/default/model-00016-of-00027.safetensors
chaiml-grpo-q235b-opusd-34252-v2-uploader: cp /dev/shm/model_output/model-00010-of-00027.safetensors s3://guanaco-vllm-models/chaiml-grpo-q235b-opusd-34252-v2/default/model-00010-of-00027.safetensors
chaiml-grpo-q235b-opusd-34252-v2-uploader: cp /dev/shm/model_output/model-00014-of-00027.safetensors s3://guanaco-vllm-models/chaiml-grpo-q235b-opusd-34252-v2/default/model-00014-of-00027.safetensors
chaiml-grpo-q235b-opusd-34252-v2-uploader: cp /dev/shm/model_output/model-00006-of-00027.safetensors s3://guanaco-vllm-models/chaiml-grpo-q235b-opusd-34252-v2/default/model-00006-of-00027.safetensors
chaiml-grpo-q235b-opusd-34252-v2-uploader: cp /dev/shm/model_output/model-00018-of-00027.safetensors s3://guanaco-vllm-models/chaiml-grpo-q235b-opusd-34252-v2/default/model-00018-of-00027.safetensors
chaiml-grpo-q235b-opusd-34252-v2-uploader: cp /dev/shm/model_output/model-00025-of-00027.safetensors s3://guanaco-vllm-models/chaiml-grpo-q235b-opusd-34252-v2/default/model-00025-of-00027.safetensors
chaiml-grpo-q235b-opusd-34252-v2-uploader: cp /dev/shm/model_output/model-00015-of-00027.safetensors s3://guanaco-vllm-models/chaiml-grpo-q235b-opusd-34252-v2/default/model-00015-of-00027.safetensors
chaiml-grpo-q235b-opusd-34252-v2-uploader: cp /dev/shm/model_output/model-00011-of-00027.safetensors s3://guanaco-vllm-models/chaiml-grpo-q235b-opusd-34252-v2/default/model-00011-of-00027.safetensors
chaiml-grpo-q235b-opusd-34252-v2-uploader: cp /dev/shm/model_output/model-00009-of-00027.safetensors s3://guanaco-vllm-models/chaiml-grpo-q235b-opusd-34252-v2/default/model-00009-of-00027.safetensors
chaiml-grpo-q235b-opusd-34252-v2-uploader: cp /dev/shm/model_output/model-00017-of-00027.safetensors s3://guanaco-vllm-models/chaiml-grpo-q235b-opusd-34252-v2/default/model-00017-of-00027.safetensors
chaiml-grpo-q235b-opusd-34252-v2-uploader: cp /dev/shm/model_output/model-00004-of-00027.safetensors s3://guanaco-vllm-models/chaiml-grpo-q235b-opusd-34252-v2/default/model-00004-of-00027.safetensors
chaiml-grpo-q235b-opusd-34252-v2-uploader: cp /dev/shm/model_output/model-00008-of-00027.safetensors s3://guanaco-vllm-models/chaiml-grpo-q235b-opusd-34252-v2/default/model-00008-of-00027.safetensors
chaiml-grpo-q235b-opusd-34252-v2-uploader: cp /dev/shm/model_output/model-00024-of-00027.safetensors s3://guanaco-vllm-models/chaiml-grpo-q235b-opusd-34252-v2/default/model-00024-of-00027.safetensors
chaiml-grpo-q235b-opusd-34252-v2-uploader: cp /dev/shm/model_output/model-00022-of-00027.safetensors s3://guanaco-vllm-models/chaiml-grpo-q235b-opusd-34252-v2/default/model-00022-of-00027.safetensors
chaiml-grpo-q235b-opusd-34252-v2-uploader: cp /dev/shm/model_output/model-00019-of-00027.safetensors s3://guanaco-vllm-models/chaiml-grpo-q235b-opusd-34252-v2/default/model-00019-of-00027.safetensors
chaiml-grpo-q235b-opusd-34252-v2-uploader: cp /dev/shm/model_output/model-00013-of-00027.safetensors s3://guanaco-vllm-models/chaiml-grpo-q235b-opusd-34252-v2/default/model-00013-of-00027.safetensors
chaiml-grpo-q235b-opusd-34252-v2-uploader: cp /dev/shm/model_output/model-00003-of-00027.safetensors s3://guanaco-vllm-models/chaiml-grpo-q235b-opusd-34252-v2/default/model-00003-of-00027.safetensors
chaiml-grpo-q235b-opusd-34252-v2-uploader: cp /dev/shm/model_output/model-00026-of-00027.safetensors s3://guanaco-vllm-models/chaiml-grpo-q235b-opusd-34252-v2/default/model-00026-of-00027.safetensors
chaiml-grpo-q235b-opusd-34252-v2-uploader: cp /dev/shm/model_output/model-00012-of-00027.safetensors s3://guanaco-vllm-models/chaiml-grpo-q235b-opusd-34252-v2/default/model-00012-of-00027.safetensors
chaiml-grpo-q235b-opusd-34252-v2-uploader: cp /dev/shm/model_output/model-00021-of-00027.safetensors s3://guanaco-vllm-models/chaiml-grpo-q235b-opusd-34252-v2/default/model-00021-of-00027.safetensors
chaiml-grpo-q235b-opusd-34252-v2-uploader: cp /dev/shm/model_output/model-00002-of-00027.safetensors s3://guanaco-vllm-models/chaiml-grpo-q235b-opusd-34252-v2/default/model-00002-of-00027.safetensors
chaiml-grpo-q235b-opusd-34252-v2-uploader: cp /dev/shm/model_output/model-00023-of-00027.safetensors s3://guanaco-vllm-models/chaiml-grpo-q235b-opusd-34252-v2/default/model-00023-of-00027.safetensors
Job chaiml-grpo-q235b-opusd-34252-v2-uploader completed after 151.36s with status: succeeded
Stopping job with name chaiml-grpo-q235b-opusd-34252-v2-uploader
Pipeline stage VLLMUploader completed in 152.28s
run pipeline stage %s
Running pipeline stage VLLMTemplater
Pipeline stage VLLMTemplater completed in 0.15s
run pipeline stage %s
Running pipeline stage VLLMDeployer
Creating inference service chaiml-grpo-q235b-opusd-34252-v2
Waiting for inference service chaiml-grpo-q235b-opusd-34252-v2 to be ready
Failed to get response for submission chaiml-mistral-24b-2048_15988_v1: ('http://chaiml-mistral-24b-2048-15988-v1-predictor.tenant-chaiml-guanaco.kchai-coreweave-us-east-04a.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', '')
Failed to get response for submission chaiml-mistral-24b-2048-_2678_v3: ('http://chaiml-mistral-24b-2048-2678-v3-predictor.tenant-chaiml-guanaco.kchai-coreweave-us-east-04a.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', '')
Failed to get response for submission chaiml-mistral-24b-2048_15988_v1: ('http://chaiml-mistral-24b-2048-15988-v1-predictor.tenant-chaiml-guanaco.kchai-coreweave-us-east-04a.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', '')
Inference service chaiml-grpo-q235b-opusd-34252-v2 ready after 422.69436025619507s
Pipeline stage VLLMDeployer completed in 423.35s
run pipeline stage %s
Running pipeline stage StressChecker
Received healthy response to inference request in 1.9741716384887695s
Received healthy response to inference request in 2.2879080772399902s
Received healthy response to inference request in 1.958270788192749s
Received healthy response to inference request in 2.1806249618530273s
Received healthy response to inference request in 2.0085482597351074s
Received healthy response to inference request in 2.131683111190796s
Received healthy response to inference request in 1.9694645404815674s
Received healthy response to inference request in 1.8972525596618652s
Received healthy response to inference request in 1.9759864807128906s
Received healthy response to inference request in 2.739025354385376s
Received healthy response to inference request in 1.9738850593566895s
Received healthy response to inference request in 1.9716496467590332s
Received healthy response to inference request in 1.9424760341644287s
Received healthy response to inference request in 2.1435606479644775s
Received healthy response to inference request in 2.092071294784546s
Received healthy response to inference request in 1.9922118186950684s
Received healthy response to inference request in 2.0096840858459473s
Received healthy response to inference request in 2.046116590499878s
Received healthy response to inference request in 2.4462640285491943s
Received healthy response to inference request in 2.452512264251709s
Received healthy response to inference request in 2.7434146404266357s
Received healthy response to inference request in 2.802152156829834s
Received healthy response to inference request in 3.0100302696228027s
Received healthy response to inference request in 2.0667412281036377s
Failed to get response for submission chaiml-mistral-24b-2048_54327_v6: ('http://chaiml-mistral-24b-2048-54327-v6-predictor.tenant-chaiml-guanaco.kchai-coreweave-us-east-04a.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', '')
Received healthy response to inference request in 2.222909688949585s
Received healthy response to inference request in 2.231018543243408s
Received healthy response to inference request in 2.308882474899292s
Received healthy response to inference request in 2.3193068504333496s
Received healthy response to inference request in 1.8648719787597656s
Received healthy response to inference request in 1.9976580142974854s
30 requests
0 failed requests
5th percentile: 1.9176031231880188
10th percentile: 1.956691312789917
20th percentile: 1.9734379768371582
30th percentile: 1.987344217300415
40th percentile: 2.0092297554016114
50th percentile: 2.079406261444092
60th percentile: 2.1583863735198974
70th percentile: 2.2480854034423827
80th percentile: 2.344698286056519
90th percentile: 2.739464282989502
95th percentile: 2.7757202744483944
99th percentile: 2.9497456169128418
mean time: 2.19201176961263
Pipeline stage StressChecker completed in 74.01s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyTriggerPipeline
run_pipeline:run_in_cloud %s
starting trigger_guanaco_pipeline args=%s
triggered trigger_guanaco_pipeline args=%s
Pipeline stage OfflineFamilyFriendlyTriggerPipeline completed in 0.66s
Shutdown handler de-registered
chaiml-grpo-q235b-opusd_34252_v2 status is now deployed due to DeploymentManager action
chaiml-grpo-q235b-opusd_34252_v2 status is now inactive due to auto deactivation removed underperforming models
chaiml-grpo-q235b-opusd_34252_v2 status is now inactive due to Froze recruitment for AB test 0220_feynman
chaiml-grpo-q235b-opusd_34252_v2 status is now torndown due to DeploymentManager action