chaiml-grpo-q235b-opusd_23459

developer_uid: acehao-chai
submission_id: chaiml-grpo-q235b-opusd_23459_v2
model_name: chaiml-grpo-q235b-opusd_23459_v2
model_group: ChaiML/grpo-q235b-opusd-
status: torndown
timestamp: 2026-02-22T19:42:02+00:00
num_battles: 11325
num_wins: 6243
celo_rating: 1333.41
family_friendly_score: 0.0
family_friendly_standard_error: 0.0
submission_type: basic
model_repo: ChaiML/grpo-q235b-opusd-v1-merged-chai-rm-high-kl-averaged-loras
model_architecture: Qwen3MoeForCausalLM
model_num_parameters: 18790207488.0
best_of: 8
max_input_tokens: 2048
max_output_tokens: 80
reward_model: default
display_name: chaiml-grpo-q235b-opusd_23459_v2
ineligible_reason: max_output_tokens!=64
is_internal_developer: False
language_model: ChaiML/grpo-q235b-opusd-v1-merged-chai-rm-high-kl-averaged-loras
model_size: 19B
ranking_group: single
us_pacific_date: 2026-02-19
win_ratio: 0.5512582781456954
generation_params: {'temperature': 1.0, 'top_p': 1.0, 'min_p': 0.0, 'top_k': 40, 'presence_penalty': 0.0, 'frequency_penalty': 0.0, 'stopping_words': ['<|assistant|>', '<|im_end|>', '####', '<|user|>', '</think>', '</s>'], 'max_input_tokens': 2048, 'best_of': 8, 'max_output_tokens': 80}
formatter: {'memory_template': "<|im_start|>system\n{bot_name}'s persona: {memory}<|im_end|>\n", 'prompt_template': '', 'bot_template': '<|im_start|>assistant\n{bot_name}: {message}<|im_end|>\n', 'user_template': '<|im_start|>user\n{message}<|im_end|>\n', 'response_template': '<|im_start|>assistant\n{bot_name}:', 'truncate_by_message': True}
Resubmit model
Shutdown handler not registered because Python interpreter is not running in the main thread
run pipeline %s
run pipeline stage %s
Running pipeline stage VLLMUploader
Starting job with name chaiml-grpo-q235b-opusd-23459-v2-uploader
Waiting for job on chaiml-grpo-q235b-opusd-23459-v2-uploader to finish
Failed to get response for submission chaiml-mistral-24b-2048_54327_v6: ('http://chaiml-mistral-24b-2048-54327-v6-predictor.tenant-chaiml-guanaco.kchai-coreweave-us-east-04a.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', '')
chaiml-grpo-q235b-opusd-23459-v2-uploader: Using quantization_mode: w4a16
chaiml-grpo-q235b-opusd-23459-v2-uploader: Checking if ChaiML/grpo-q235b-opusd-v1-merged-chai-rm-high-kl-averaged-loras-W4A16 already exists in ChaiML
chaiml-grpo-q235b-opusd-23459-v2-uploader: Model already exists. Downloading to /dev/shm/model_output...
chaiml-grpo-q235b-opusd-23459-v2-uploader: Downloading snapshot of ChaiML/grpo-q235b-opusd-v1-merged-chai-rm-high-kl-averaged-loras-W4A16...
chaiml-grpo-q235b-opusd-23459-v2-uploader: Downloaded in 57.288s
chaiml-grpo-q235b-opusd-23459-v2-uploader: Processed model ChaiML/grpo-q235b-opusd-v1-merged-chai-rm-high-kl-averaged-loras in 57.929s
chaiml-grpo-q235b-opusd-23459-v2-uploader: creating bucket guanaco-vllm-models
chaiml-grpo-q235b-opusd-23459-v2-uploader: /usr/lib/python3/dist-packages/S3/BaseUtils.py:56: SyntaxWarning: invalid escape sequence '\.'
chaiml-grpo-q235b-opusd-23459-v2-uploader:   RE_S3_DATESTRING = re.compile('\.[0-9]*(?:[Z\\-\\+]*?)')
chaiml-grpo-q235b-opusd-23459-v2-uploader: /usr/lib/python3/dist-packages/S3/BaseUtils.py:57: SyntaxWarning: invalid escape sequence '\s'
chaiml-grpo-q235b-opusd-23459-v2-uploader:   RE_XML_NAMESPACE = re.compile(b'^(<?[^>]+?>\s*|\s*)(<\w+) xmlns=[\'"](https?://[^\'"]+)[\'"]', re.MULTILINE)
chaiml-grpo-q235b-opusd-23459-v2-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:240: SyntaxWarning: invalid escape sequence '\.'
chaiml-grpo-q235b-opusd-23459-v2-uploader:   invalid = re.search("([^a-z0-9\.-])", bucket, re.UNICODE)
chaiml-grpo-q235b-opusd-23459-v2-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:244: SyntaxWarning: invalid escape sequence '\.'
chaiml-grpo-q235b-opusd-23459-v2-uploader:   invalid = re.search("([^A-Za-z0-9\._-])", bucket, re.UNICODE)
chaiml-grpo-q235b-opusd-23459-v2-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:255: SyntaxWarning: invalid escape sequence '\.'
chaiml-grpo-q235b-opusd-23459-v2-uploader:   if re.search("-\.", bucket, re.UNICODE):
chaiml-grpo-q235b-opusd-23459-v2-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:257: SyntaxWarning: invalid escape sequence '\.'
chaiml-grpo-q235b-opusd-23459-v2-uploader:   if re.search("\.\.", bucket, re.UNICODE):
chaiml-grpo-q235b-opusd-23459-v2-uploader: /usr/lib/python3/dist-packages/S3/S3Uri.py:155: SyntaxWarning: invalid escape sequence '\w'
chaiml-grpo-q235b-opusd-23459-v2-uploader:   _re = re.compile("^(\w+://)?(.*)", re.UNICODE)
chaiml-grpo-q235b-opusd-23459-v2-uploader: /usr/lib/python3/dist-packages/S3/FileLists.py:480: SyntaxWarning: invalid escape sequence '\*'
chaiml-grpo-q235b-opusd-23459-v2-uploader:   wildcard_split_result = re.split("\*|\?", uri_str, maxsplit=1)
chaiml-grpo-q235b-opusd-23459-v2-uploader: Bucket 's3://guanaco-vllm-models/' created
chaiml-grpo-q235b-opusd-23459-v2-uploader: uploading /dev/shm/model_output to s3://guanaco-vllm-models/chaiml-grpo-q235b-opusd-23459-v2/default
chaiml-grpo-q235b-opusd-23459-v2-uploader: cp /dev/shm/model_output/chat_template.jinja s3://guanaco-vllm-models/chaiml-grpo-q235b-opusd-23459-v2/default/chat_template.jinja
chaiml-grpo-q235b-opusd-23459-v2-uploader: cp /dev/shm/model_output/generation_config.json s3://guanaco-vllm-models/chaiml-grpo-q235b-opusd-23459-v2/default/generation_config.json
chaiml-grpo-q235b-opusd-23459-v2-uploader: cp /dev/shm/model_output/.gitattributes s3://guanaco-vllm-models/chaiml-grpo-q235b-opusd-23459-v2/default/.gitattributes
chaiml-grpo-q235b-opusd-23459-v2-uploader: cp /dev/shm/model_output/added_tokens.json s3://guanaco-vllm-models/chaiml-grpo-q235b-opusd-23459-v2/default/added_tokens.json
chaiml-grpo-q235b-opusd-23459-v2-uploader: cp /dev/shm/model_output/config.json s3://guanaco-vllm-models/chaiml-grpo-q235b-opusd-23459-v2/default/config.json
chaiml-grpo-q235b-opusd-23459-v2-uploader: cp /dev/shm/model_output/special_tokens_map.json s3://guanaco-vllm-models/chaiml-grpo-q235b-opusd-23459-v2/default/special_tokens_map.json
chaiml-grpo-q235b-opusd-23459-v2-uploader: cp /dev/shm/model_output/quantization_config.json s3://guanaco-vllm-models/chaiml-grpo-q235b-opusd-23459-v2/default/quantization_config.json
chaiml-grpo-q235b-opusd-23459-v2-uploader: cp /dev/shm/model_output/model.safetensors.index.json s3://guanaco-vllm-models/chaiml-grpo-q235b-opusd-23459-v2/default/model.safetensors.index.json
chaiml-grpo-q235b-opusd-23459-v2-uploader: cp /dev/shm/model_output/tokenizer_config.json s3://guanaco-vllm-models/chaiml-grpo-q235b-opusd-23459-v2/default/tokenizer_config.json
chaiml-grpo-q235b-opusd-23459-v2-uploader: cp /dev/shm/model_output/tokenizer.json s3://guanaco-vllm-models/chaiml-grpo-q235b-opusd-23459-v2/default/tokenizer.json
chaiml-grpo-q235b-opusd-23459-v2-uploader: cp /dev/shm/model_output/vocab.json s3://guanaco-vllm-models/chaiml-grpo-q235b-opusd-23459-v2/default/vocab.json
chaiml-grpo-q235b-opusd-23459-v2-uploader: cp /dev/shm/model_output/merges.txt s3://guanaco-vllm-models/chaiml-grpo-q235b-opusd-23459-v2/default/merges.txt
chaiml-grpo-q235b-opusd-23459-v2-uploader: cp /dev/shm/model_output/model-00027-of-00027.safetensors s3://guanaco-vllm-models/chaiml-grpo-q235b-opusd-23459-v2/default/model-00027-of-00027.safetensors
chaiml-grpo-q235b-opusd-23459-v2-uploader: cp /dev/shm/model_output/model-00016-of-00027.safetensors s3://guanaco-vllm-models/chaiml-grpo-q235b-opusd-23459-v2/default/model-00016-of-00027.safetensors
chaiml-grpo-q235b-opusd-23459-v2-uploader: cp /dev/shm/model_output/model-00004-of-00027.safetensors s3://guanaco-vllm-models/chaiml-grpo-q235b-opusd-23459-v2/default/model-00004-of-00027.safetensors
chaiml-grpo-q235b-opusd-23459-v2-uploader: cp /dev/shm/model_output/model-00025-of-00027.safetensors s3://guanaco-vllm-models/chaiml-grpo-q235b-opusd-23459-v2/default/model-00025-of-00027.safetensors
chaiml-grpo-q235b-opusd-23459-v2-uploader: cp /dev/shm/model_output/model-00003-of-00027.safetensors s3://guanaco-vllm-models/chaiml-grpo-q235b-opusd-23459-v2/default/model-00003-of-00027.safetensors
chaiml-grpo-q235b-opusd-23459-v2-uploader: cp /dev/shm/model_output/model-00002-of-00027.safetensors s3://guanaco-vllm-models/chaiml-grpo-q235b-opusd-23459-v2/default/model-00002-of-00027.safetensors
chaiml-grpo-q235b-opusd-23459-v2-uploader: cp /dev/shm/model_output/model-00008-of-00027.safetensors s3://guanaco-vllm-models/chaiml-grpo-q235b-opusd-23459-v2/default/model-00008-of-00027.safetensors
chaiml-grpo-q235b-opusd-23459-v2-uploader: cp /dev/shm/model_output/model-00014-of-00027.safetensors s3://guanaco-vllm-models/chaiml-grpo-q235b-opusd-23459-v2/default/model-00014-of-00027.safetensors
chaiml-grpo-q235b-opusd-23459-v2-uploader: cp /dev/shm/model_output/model-00022-of-00027.safetensors s3://guanaco-vllm-models/chaiml-grpo-q235b-opusd-23459-v2/default/model-00022-of-00027.safetensors
chaiml-grpo-q235b-opusd-23459-v2-uploader: cp /dev/shm/model_output/model-00011-of-00027.safetensors s3://guanaco-vllm-models/chaiml-grpo-q235b-opusd-23459-v2/default/model-00011-of-00027.safetensors
chaiml-grpo-q235b-opusd-23459-v2-uploader: cp /dev/shm/model_output/model-00021-of-00027.safetensors s3://guanaco-vllm-models/chaiml-grpo-q235b-opusd-23459-v2/default/model-00021-of-00027.safetensors
chaiml-grpo-q235b-opusd-23459-v2-uploader: cp /dev/shm/model_output/model-00006-of-00027.safetensors s3://guanaco-vllm-models/chaiml-grpo-q235b-opusd-23459-v2/default/model-00006-of-00027.safetensors
chaiml-grpo-q235b-opusd-23459-v2-uploader: cp /dev/shm/model_output/model-00026-of-00027.safetensors s3://guanaco-vllm-models/chaiml-grpo-q235b-opusd-23459-v2/default/model-00026-of-00027.safetensors
chaiml-grpo-q235b-opusd-23459-v2-uploader: cp /dev/shm/model_output/model-00023-of-00027.safetensors s3://guanaco-vllm-models/chaiml-grpo-q235b-opusd-23459-v2/default/model-00023-of-00027.safetensors
chaiml-grpo-q235b-opusd-23459-v2-uploader: cp /dev/shm/model_output/model-00019-of-00027.safetensors s3://guanaco-vllm-models/chaiml-grpo-q235b-opusd-23459-v2/default/model-00019-of-00027.safetensors
chaiml-grpo-q235b-opusd-23459-v2-uploader: cp /dev/shm/model_output/model-00020-of-00027.safetensors s3://guanaco-vllm-models/chaiml-grpo-q235b-opusd-23459-v2/default/model-00020-of-00027.safetensors
chaiml-grpo-q235b-opusd-23459-v2-uploader: cp /dev/shm/model_output/model-00013-of-00027.safetensors s3://guanaco-vllm-models/chaiml-grpo-q235b-opusd-23459-v2/default/model-00013-of-00027.safetensors
chaiml-grpo-q235b-opusd-23459-v2-uploader: cp /dev/shm/model_output/model-00018-of-00027.safetensors s3://guanaco-vllm-models/chaiml-grpo-q235b-opusd-23459-v2/default/model-00018-of-00027.safetensors
chaiml-grpo-q235b-opusd-23459-v2-uploader: cp /dev/shm/model_output/model-00010-of-00027.safetensors s3://guanaco-vllm-models/chaiml-grpo-q235b-opusd-23459-v2/default/model-00010-of-00027.safetensors
chaiml-grpo-q235b-opusd-23459-v2-uploader: cp /dev/shm/model_output/model-00007-of-00027.safetensors s3://guanaco-vllm-models/chaiml-grpo-q235b-opusd-23459-v2/default/model-00007-of-00027.safetensors
chaiml-grpo-q235b-opusd-23459-v2-uploader: cp /dev/shm/model_output/model-00009-of-00027.safetensors s3://guanaco-vllm-models/chaiml-grpo-q235b-opusd-23459-v2/default/model-00009-of-00027.safetensors
chaiml-grpo-q235b-opusd-23459-v2-uploader: cp /dev/shm/model_output/model-00005-of-00027.safetensors s3://guanaco-vllm-models/chaiml-grpo-q235b-opusd-23459-v2/default/model-00005-of-00027.safetensors
chaiml-grpo-q235b-opusd-23459-v2-uploader: cp /dev/shm/model_output/model-00024-of-00027.safetensors s3://guanaco-vllm-models/chaiml-grpo-q235b-opusd-23459-v2/default/model-00024-of-00027.safetensors
chaiml-grpo-q235b-opusd-23459-v2-uploader: cp /dev/shm/model_output/model-00017-of-00027.safetensors s3://guanaco-vllm-models/chaiml-grpo-q235b-opusd-23459-v2/default/model-00017-of-00027.safetensors
chaiml-grpo-q235b-opusd-23459-v2-uploader: cp /dev/shm/model_output/model-00012-of-00027.safetensors s3://guanaco-vllm-models/chaiml-grpo-q235b-opusd-23459-v2/default/model-00012-of-00027.safetensors
chaiml-grpo-q235b-opusd-23459-v2-uploader: cp /dev/shm/model_output/model-00001-of-00027.safetensors s3://guanaco-vllm-models/chaiml-grpo-q235b-opusd-23459-v2/default/model-00001-of-00027.safetensors
chaiml-grpo-q235b-opusd-23459-v2-uploader: cp /dev/shm/model_output/model-00015-of-00027.safetensors s3://guanaco-vllm-models/chaiml-grpo-q235b-opusd-23459-v2/default/model-00015-of-00027.safetensors
Job chaiml-grpo-q235b-opusd-23459-v2-uploader completed after 145.38s with status: succeeded
Stopping job with name chaiml-grpo-q235b-opusd-23459-v2-uploader
Pipeline stage VLLMUploader completed in 145.91s
run pipeline stage %s
Running pipeline stage VLLMTemplater
Pipeline stage VLLMTemplater completed in 0.15s
run pipeline stage %s
Running pipeline stage VLLMDeployer
Creating inference service chaiml-grpo-q235b-opusd-23459-v2
Waiting for inference service chaiml-grpo-q235b-opusd-23459-v2 to be ready
Failed to get response for submission chaiml-grpo-q235b-kimid_37540_v1: HTTPConnectionPool(host='chaiml-grpo-q235b-kimid-37540-v1-predictor.tenant-chaiml-guanaco.kchai-coreweave-us-east-04a.chaiverse.com', port=80): Read timed out. (read timeout=12.0)
Failed to get response for submission chaiml-mistral-24b-2048-_2678_v3: ('http://chaiml-mistral-24b-2048-2678-v3-predictor.tenant-chaiml-guanaco.kchai-coreweave-us-east-04a.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', '')
HTTP Request: %s %s "%s %d %s"
Failed to get response for submission chaiml-mistral-24b-2048_54327_v6: ('http://chaiml-mistral-24b-2048-54327-v6-predictor.tenant-chaiml-guanaco.kchai-coreweave-us-east-04a.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', '')
Failed to get response for submission chaiml-mistral-24b-2048-_2678_v3: ('http://chaiml-mistral-24b-2048-2678-v3-predictor.tenant-chaiml-guanaco.kchai-coreweave-us-east-04a.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', '')
HTTP Request: %s %s "%s %d %s"
Failed to get response for submission chaiml-mistral-24b-2048-_2678_v3: ('http://chaiml-mistral-24b-2048-2678-v3-predictor.tenant-chaiml-guanaco.kchai-coreweave-us-east-04a.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', '')
Inference service chaiml-grpo-q235b-opusd-23459-v2 ready after 422.38555550575256s
Pipeline stage VLLMDeployer completed in 422.94s
run pipeline stage %s
Running pipeline stage StressChecker
Received healthy response to inference request in 1.7087881565093994s
Received healthy response to inference request in 1.5967633724212646s
Received healthy response to inference request in 1.9696099758148193s
Received healthy response to inference request in 1.6010186672210693s
Received healthy response to inference request in 1.7142088413238525s
Received healthy response to inference request in 1.7152767181396484s
Received healthy response to inference request in 1.5502140522003174s
Received healthy response to inference request in 1.7949559688568115s
Received healthy response to inference request in 1.6880054473876953s
Received healthy response to inference request in 2.037193775177002s
Received healthy response to inference request in 1.9162218570709229s
Received healthy response to inference request in 1.6833012104034424s
Received healthy response to inference request in 1.797999620437622s
Received healthy response to inference request in 1.760568380355835s
Received healthy response to inference request in 1.7466495037078857s
Received healthy response to inference request in 1.5521597862243652s
Received healthy response to inference request in 1.9103760719299316s
Received healthy response to inference request in 1.7292814254760742s
Received healthy response to inference request in 1.7818517684936523s
Received healthy response to inference request in 1.7629940509796143s
Received healthy response to inference request in 1.7902567386627197s
Received healthy response to inference request in 1.5845403671264648s
Received healthy response to inference request in 1.820042610168457s
Failed to get response for submission chaiml-mistral-24b-2048_54327_v6: ('http://chaiml-mistral-24b-2048-54327-v6-predictor.tenant-chaiml-guanaco.kchai-coreweave-us-east-04a.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', '')
Received healthy response to inference request in 1.6329128742218018s
Received healthy response to inference request in 1.835559606552124s
Received healthy response to inference request in 1.7119522094726562s
Received healthy response to inference request in 1.9090826511383057s
Received healthy response to inference request in 1.6531672477722168s
Received healthy response to inference request in 1.7305638790130615s
Received healthy response to inference request in 1.7019548416137695s
30 requests
0 failed requests
5th percentile: 1.56673104763031
10th percentile: 1.5955410718917846
20th percentile: 1.6491163730621339
30th percentile: 1.6977700233459472
40th percentile: 1.713306188583374
50th percentile: 1.7299226522445679
60th percentile: 1.7615386486053466
70th percentile: 1.7916665077209473
80th percentile: 1.8231460094451906
90th percentile: 1.9109606504440309
95th percentile: 1.9455853223800659
99th percentile: 2.017594473361969
mean time: 1.7462490558624268
Pipeline stage StressChecker completed in 55.37s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyTriggerPipeline
run_pipeline:run_in_cloud %s
starting trigger_guanaco_pipeline args=%s
triggered trigger_guanaco_pipeline args=%s
Pipeline stage OfflineFamilyFriendlyTriggerPipeline completed in 0.66s
Shutdown handler de-registered
chaiml-grpo-q235b-opusd_23459_v2 status is now deployed due to DeploymentManager action
chaiml-grpo-q235b-opusd_23459_v2 status is now inactive due to auto deactivation removed underperforming models
chaiml-grpo-q235b-opusd_23459_v2 status is now torndown due to DeploymentManager action