developer_uid: acehao-chai
submission_id: chaiml-grpo-q235b-kimid_83709_v2
model_name: chaiml-grpo-q235b-kimid_83709_v2
model_group: ChaiML/grpo-q235b-kimid-
status: torndown
timestamp: 2026-03-11T17:33:25+00:00
num_battles: 10632
num_wins: 5688
celo_rating: 1323.14
family_friendly_score: 0.0
family_friendly_standard_error: 0.0
submission_type: basic
model_repo: ChaiML/grpo-q235b-kimid-v5a-merged-chai-rm-step-450
model_architecture: Qwen3MoeForCausalLM
model_num_parameters: 18790207488.0
best_of: 8
max_input_tokens: 2048
max_output_tokens: 80
reward_model: default
display_name: chaiml-grpo-q235b-kimid_83709_v2
ineligible_reason: max_output_tokens!=64
is_internal_developer: False
language_model: ChaiML/grpo-q235b-kimid-v5a-merged-chai-rm-step-450
model_size: 19B
ranking_group: single
us_pacific_date: 2026-02-20
win_ratio: 0.5349887133182845
generation_params: {'temperature': 1.0, 'top_p': 1.0, 'min_p': 0.0, 'top_k': 40, 'presence_penalty': 0.0, 'frequency_penalty': 0.0, 'stopping_words': ['<|assistant|>', '<|im_end|>', '####', '<|user|>', '</think>', '</s>'], 'max_input_tokens': 2048, 'best_of': 8, 'max_output_tokens': 80}
formatter: {'memory_template': "<|im_start|>system\n{bot_name}'s persona: {memory}<|im_end|>\n", 'prompt_template': '', 'bot_template': '<|im_start|>assistant\n{bot_name}: {message}<|im_end|>\n', 'user_template': '<|im_start|>user\n{message}<|im_end|>\n', 'response_template': '<|im_start|>assistant\n{bot_name}:', 'truncate_by_message': True}
Resubmit model
Shutdown handler not registered because Python interpreter is not running in the main thread
run pipeline %s
run pipeline stage %s
Running pipeline stage VLLMUploader
Starting job with name chaiml-grpo-q235b-kimid-83709-v2-uploader
Waiting for job on chaiml-grpo-q235b-kimid-83709-v2-uploader to finish
Failed to get response for submission chaiml-grpo-q235b-kimid_37540_v1: HTTPConnectionPool(host='chaiml-grpo-q235b-kimid-37540-v1-predictor.tenant-chaiml-guanaco.kchai-coreweave-us-east-04a.chaiverse.com', port=80): Read timed out. (read timeout=12.0)
Failed to get response for submission chaiml-mistral-24b-2048-_2678_v3: ('http://chaiml-mistral-24b-2048-2678-v3-predictor.tenant-chaiml-guanaco.kchai-coreweave-us-east-04a.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', '')
chaiml-grpo-q235b-kimid-83709-v2-uploader: Using quantization_mode: w4a16
chaiml-grpo-q235b-kimid-83709-v2-uploader: Checking if ChaiML/grpo-q235b-kimid-v5a-merged-chai-rm-step-450-W4A16 already exists in ChaiML
chaiml-grpo-q235b-kimid-83709-v2-uploader: Model already exists. Downloading to /dev/shm/model_output...
chaiml-grpo-q235b-kimid-83709-v2-uploader: Downloading snapshot of ChaiML/grpo-q235b-kimid-v5a-merged-chai-rm-step-450-W4A16...
Failed to get response for submission chaiml-mistral-24b-2048_15988_v1: ('http://chaiml-mistral-24b-2048-15988-v1-predictor.tenant-chaiml-guanaco.kchai-coreweave-us-east-04a.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', '')
Failed to get response for submission chaiml-mistral-24b-2048_15988_v1: ('http://chaiml-mistral-24b-2048-15988-v1-predictor.tenant-chaiml-guanaco.kchai-coreweave-us-east-04a.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', '')
Failed to get response for submission chaiml-mistral-24b-2048-_2678_v3: ('http://chaiml-mistral-24b-2048-2678-v3-predictor.tenant-chaiml-guanaco.kchai-coreweave-us-east-04a.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', '')
chaiml-grpo-q235b-kimid-83709-v2-uploader: Downloaded in 54.341s
chaiml-grpo-q235b-kimid-83709-v2-uploader: Processed model ChaiML/grpo-q235b-kimid-v5a-merged-chai-rm-step-450 in 54.898s
chaiml-grpo-q235b-kimid-83709-v2-uploader: creating bucket guanaco-vllm-models
chaiml-grpo-q235b-kimid-83709-v2-uploader: /usr/lib/python3/dist-packages/S3/BaseUtils.py:56: SyntaxWarning: invalid escape sequence '\.'
chaiml-grpo-q235b-kimid-83709-v2-uploader:   RE_S3_DATESTRING = re.compile('\.[0-9]*(?:[Z\\-\\+]*?)')
chaiml-grpo-q235b-kimid-83709-v2-uploader: /usr/lib/python3/dist-packages/S3/BaseUtils.py:57: SyntaxWarning: invalid escape sequence '\s'
chaiml-grpo-q235b-kimid-83709-v2-uploader:   RE_XML_NAMESPACE = re.compile(b'^(<?[^>]+?>\s*|\s*)(<\w+) xmlns=[\'"](https?://[^\'"]+)[\'"]', re.MULTILINE)
chaiml-grpo-q235b-kimid-83709-v2-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:240: SyntaxWarning: invalid escape sequence '\.'
chaiml-grpo-q235b-kimid-83709-v2-uploader:   invalid = re.search("([^a-z0-9\.-])", bucket, re.UNICODE)
chaiml-grpo-q235b-kimid-83709-v2-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:244: SyntaxWarning: invalid escape sequence '\.'
chaiml-grpo-q235b-kimid-83709-v2-uploader:   invalid = re.search("([^A-Za-z0-9\._-])", bucket, re.UNICODE)
chaiml-grpo-q235b-kimid-83709-v2-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:255: SyntaxWarning: invalid escape sequence '\.'
chaiml-grpo-q235b-kimid-83709-v2-uploader:   if re.search("-\.", bucket, re.UNICODE):
chaiml-grpo-q235b-kimid-83709-v2-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:257: SyntaxWarning: invalid escape sequence '\.'
chaiml-grpo-q235b-kimid-83709-v2-uploader:   if re.search("\.\.", bucket, re.UNICODE):
chaiml-grpo-q235b-kimid-83709-v2-uploader: /usr/lib/python3/dist-packages/S3/S3Uri.py:155: SyntaxWarning: invalid escape sequence '\w'
chaiml-grpo-q235b-kimid-83709-v2-uploader:   _re = re.compile("^(\w+://)?(.*)", re.UNICODE)
chaiml-grpo-q235b-kimid-83709-v2-uploader: /usr/lib/python3/dist-packages/S3/FileLists.py:480: SyntaxWarning: invalid escape sequence '\*'
chaiml-grpo-q235b-kimid-83709-v2-uploader:   wildcard_split_result = re.split("\*|\?", uri_str, maxsplit=1)
chaiml-grpo-q235b-kimid-83709-v2-uploader: Bucket 's3://guanaco-vllm-models/' created
chaiml-grpo-q235b-kimid-83709-v2-uploader: uploading /dev/shm/model_output to s3://guanaco-vllm-models/chaiml-grpo-q235b-kimid-83709-v2/default
chaiml-grpo-q235b-kimid-83709-v2-uploader: cp /dev/shm/model_output/added_tokens.json s3://guanaco-vllm-models/chaiml-grpo-q235b-kimid-83709-v2/default/added_tokens.json
chaiml-grpo-q235b-kimid-83709-v2-uploader: cp /dev/shm/model_output/.gitattributes s3://guanaco-vllm-models/chaiml-grpo-q235b-kimid-83709-v2/default/.gitattributes
chaiml-grpo-q235b-kimid-83709-v2-uploader: cp /dev/shm/model_output/quantization_config.json s3://guanaco-vllm-models/chaiml-grpo-q235b-kimid-83709-v2/default/quantization_config.json
chaiml-grpo-q235b-kimid-83709-v2-uploader: cp /dev/shm/model_output/merges.txt s3://guanaco-vllm-models/chaiml-grpo-q235b-kimid-83709-v2/default/merges.txt
chaiml-grpo-q235b-kimid-83709-v2-uploader: cp /dev/shm/model_output/config.json s3://guanaco-vllm-models/chaiml-grpo-q235b-kimid-83709-v2/default/config.json
chaiml-grpo-q235b-kimid-83709-v2-uploader: cp /dev/shm/model_output/chat_template.jinja s3://guanaco-vllm-models/chaiml-grpo-q235b-kimid-83709-v2/default/chat_template.jinja
chaiml-grpo-q235b-kimid-83709-v2-uploader: cp /dev/shm/model_output/generation_config.json s3://guanaco-vllm-models/chaiml-grpo-q235b-kimid-83709-v2/default/generation_config.json
chaiml-grpo-q235b-kimid-83709-v2-uploader: cp /dev/shm/model_output/tokenizer_config.json s3://guanaco-vllm-models/chaiml-grpo-q235b-kimid-83709-v2/default/tokenizer_config.json
chaiml-grpo-q235b-kimid-83709-v2-uploader: cp /dev/shm/model_output/special_tokens_map.json s3://guanaco-vllm-models/chaiml-grpo-q235b-kimid-83709-v2/default/special_tokens_map.json
chaiml-grpo-q235b-kimid-83709-v2-uploader: cp /dev/shm/model_output/model.safetensors.index.json s3://guanaco-vllm-models/chaiml-grpo-q235b-kimid-83709-v2/default/model.safetensors.index.json
chaiml-grpo-q235b-kimid-83709-v2-uploader: cp /dev/shm/model_output/tokenizer.json s3://guanaco-vllm-models/chaiml-grpo-q235b-kimid-83709-v2/default/tokenizer.json
chaiml-grpo-q235b-kimid-83709-v2-uploader: cp /dev/shm/model_output/vocab.json s3://guanaco-vllm-models/chaiml-grpo-q235b-kimid-83709-v2/default/vocab.json
Failed to get response for submission chaiml-mistral-24b-2048-_2678_v3: ('http://chaiml-mistral-24b-2048-2678-v3-predictor.tenant-chaiml-guanaco.kchai-coreweave-us-east-04a.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', '')
chaiml-grpo-q235b-kimid-83709-v2-uploader: cp /dev/shm/model_output/model-00027-of-00027.safetensors s3://guanaco-vllm-models/chaiml-grpo-q235b-kimid-83709-v2/default/model-00027-of-00027.safetensors
chaiml-grpo-q235b-kimid-83709-v2-uploader: cp /dev/shm/model_output/model-00002-of-00027.safetensors s3://guanaco-vllm-models/chaiml-grpo-q235b-kimid-83709-v2/default/model-00002-of-00027.safetensors
chaiml-grpo-q235b-kimid-83709-v2-uploader: cp /dev/shm/model_output/model-00020-of-00027.safetensors s3://guanaco-vllm-models/chaiml-grpo-q235b-kimid-83709-v2/default/model-00020-of-00027.safetensors
chaiml-grpo-q235b-kimid-83709-v2-uploader: cp /dev/shm/model_output/model-00011-of-00027.safetensors s3://guanaco-vllm-models/chaiml-grpo-q235b-kimid-83709-v2/default/model-00011-of-00027.safetensors
chaiml-grpo-q235b-kimid-83709-v2-uploader: cp /dev/shm/model_output/model-00017-of-00027.safetensors s3://guanaco-vllm-models/chaiml-grpo-q235b-kimid-83709-v2/default/model-00017-of-00027.safetensors
chaiml-grpo-q235b-kimid-83709-v2-uploader: cp /dev/shm/model_output/model-00015-of-00027.safetensors s3://guanaco-vllm-models/chaiml-grpo-q235b-kimid-83709-v2/default/model-00015-of-00027.safetensors
chaiml-grpo-q235b-kimid-83709-v2-uploader: cp /dev/shm/model_output/model-00022-of-00027.safetensors s3://guanaco-vllm-models/chaiml-grpo-q235b-kimid-83709-v2/default/model-00022-of-00027.safetensors
chaiml-grpo-q235b-kimid-83709-v2-uploader: cp /dev/shm/model_output/model-00018-of-00027.safetensors s3://guanaco-vllm-models/chaiml-grpo-q235b-kimid-83709-v2/default/model-00018-of-00027.safetensors
chaiml-grpo-q235b-kimid-83709-v2-uploader: cp /dev/shm/model_output/model-00005-of-00027.safetensors s3://guanaco-vllm-models/chaiml-grpo-q235b-kimid-83709-v2/default/model-00005-of-00027.safetensors
chaiml-grpo-q235b-kimid-83709-v2-uploader: cp /dev/shm/model_output/model-00024-of-00027.safetensors s3://guanaco-vllm-models/chaiml-grpo-q235b-kimid-83709-v2/default/model-00024-of-00027.safetensors
chaiml-grpo-q235b-kimid-83709-v2-uploader: cp /dev/shm/model_output/model-00021-of-00027.safetensors s3://guanaco-vllm-models/chaiml-grpo-q235b-kimid-83709-v2/default/model-00021-of-00027.safetensors
chaiml-grpo-q235b-kimid-83709-v2-uploader: cp /dev/shm/model_output/model-00012-of-00027.safetensors s3://guanaco-vllm-models/chaiml-grpo-q235b-kimid-83709-v2/default/model-00012-of-00027.safetensors
chaiml-grpo-q235b-kimid-83709-v2-uploader: cp /dev/shm/model_output/model-00010-of-00027.safetensors s3://guanaco-vllm-models/chaiml-grpo-q235b-kimid-83709-v2/default/model-00010-of-00027.safetensors
chaiml-grpo-q235b-kimid-83709-v2-uploader: cp /dev/shm/model_output/model-00026-of-00027.safetensors s3://guanaco-vllm-models/chaiml-grpo-q235b-kimid-83709-v2/default/model-00026-of-00027.safetensors
chaiml-grpo-q235b-kimid-83709-v2-uploader: cp /dev/shm/model_output/model-00019-of-00027.safetensors s3://guanaco-vllm-models/chaiml-grpo-q235b-kimid-83709-v2/default/model-00019-of-00027.safetensors
chaiml-grpo-q235b-kimid-83709-v2-uploader: cp /dev/shm/model_output/model-00001-of-00027.safetensors s3://guanaco-vllm-models/chaiml-grpo-q235b-kimid-83709-v2/default/model-00001-of-00027.safetensors
chaiml-grpo-q235b-kimid-83709-v2-uploader: cp /dev/shm/model_output/model-00009-of-00027.safetensors s3://guanaco-vllm-models/chaiml-grpo-q235b-kimid-83709-v2/default/model-00009-of-00027.safetensors
chaiml-grpo-q235b-kimid-83709-v2-uploader: cp /dev/shm/model_output/model-00007-of-00027.safetensors s3://guanaco-vllm-models/chaiml-grpo-q235b-kimid-83709-v2/default/model-00007-of-00027.safetensors
chaiml-grpo-q235b-kimid-83709-v2-uploader: cp /dev/shm/model_output/model-00006-of-00027.safetensors s3://guanaco-vllm-models/chaiml-grpo-q235b-kimid-83709-v2/default/model-00006-of-00027.safetensors
chaiml-grpo-q235b-kimid-83709-v2-uploader: cp /dev/shm/model_output/model-00004-of-00027.safetensors s3://guanaco-vllm-models/chaiml-grpo-q235b-kimid-83709-v2/default/model-00004-of-00027.safetensors
chaiml-grpo-q235b-kimid-83709-v2-uploader: cp /dev/shm/model_output/model-00008-of-00027.safetensors s3://guanaco-vllm-models/chaiml-grpo-q235b-kimid-83709-v2/default/model-00008-of-00027.safetensors
chaiml-grpo-q235b-kimid-83709-v2-uploader: cp /dev/shm/model_output/model-00023-of-00027.safetensors s3://guanaco-vllm-models/chaiml-grpo-q235b-kimid-83709-v2/default/model-00023-of-00027.safetensors
chaiml-grpo-q235b-kimid-83709-v2-uploader: cp /dev/shm/model_output/model-00016-of-00027.safetensors s3://guanaco-vllm-models/chaiml-grpo-q235b-kimid-83709-v2/default/model-00016-of-00027.safetensors
chaiml-grpo-q235b-kimid-83709-v2-uploader: cp /dev/shm/model_output/model-00013-of-00027.safetensors s3://guanaco-vllm-models/chaiml-grpo-q235b-kimid-83709-v2/default/model-00013-of-00027.safetensors
chaiml-grpo-q235b-kimid-83709-v2-uploader: cp /dev/shm/model_output/model-00014-of-00027.safetensors s3://guanaco-vllm-models/chaiml-grpo-q235b-kimid-83709-v2/default/model-00014-of-00027.safetensors
chaiml-grpo-q235b-kimid-83709-v2-uploader: cp /dev/shm/model_output/model-00025-of-00027.safetensors s3://guanaco-vllm-models/chaiml-grpo-q235b-kimid-83709-v2/default/model-00025-of-00027.safetensors
chaiml-grpo-q235b-kimid-83709-v2-uploader: cp /dev/shm/model_output/model-00003-of-00027.safetensors s3://guanaco-vllm-models/chaiml-grpo-q235b-kimid-83709-v2/default/model-00003-of-00027.safetensors
Job chaiml-grpo-q235b-kimid-83709-v2-uploader completed after 135.93s with status: succeeded
Stopping job with name chaiml-grpo-q235b-kimid-83709-v2-uploader
Pipeline stage VLLMUploader completed in 136.52s
run pipeline stage %s
Running pipeline stage VLLMTemplater
Pipeline stage VLLMTemplater completed in 0.17s
run pipeline stage %s
Running pipeline stage VLLMDeployer
Creating inference service chaiml-grpo-q235b-kimid-83709-v2
Waiting for inference service chaiml-grpo-q235b-kimid-83709-v2 to be ready
Failed to get response for submission chaiml-mistral-24b-2048_54327_v6: ('http://chaiml-mistral-24b-2048-54327-v6-predictor.tenant-chaiml-guanaco.kchai-coreweave-us-east-04a.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', '')
Failed to get response for submission chaiml-mistral-24b-2048-_2678_v3: ('http://chaiml-mistral-24b-2048-2678-v3-predictor.tenant-chaiml-guanaco.kchai-coreweave-us-east-04a.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', '')
Failed to get response for submission chaiml-grpo-q235b-kimid_37540_v1: HTTPConnectionPool(host='chaiml-grpo-q235b-kimid-37540-v1-predictor.tenant-chaiml-guanaco.kchai-coreweave-us-east-04a.chaiverse.com', port=80): Read timed out. (read timeout=12.0)
Failed to get response for submission chaiml-mistral-24b-2048_54327_v6: ('http://chaiml-mistral-24b-2048-54327-v6-predictor.tenant-chaiml-guanaco.kchai-coreweave-us-east-04a.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', '')
Inference service chaiml-grpo-q235b-kimid-83709-v2 ready after 452.2906925678253s
Pipeline stage VLLMDeployer completed in 453.44s
run pipeline stage %s
Running pipeline stage StressChecker
Received healthy response to inference request in 2.1862289905548096s
Received healthy response to inference request in 2.234222888946533s
Received healthy response to inference request in 2.0361642837524414s
Received healthy response to inference request in 1.986107587814331s
Received healthy response to inference request in 1.9710946083068848s
Received healthy response to inference request in 1.9540367126464844s
Received healthy response to inference request in 2.084864854812622s
Received healthy response to inference request in 2.1403746604919434s
Received healthy response to inference request in 1.9959452152252197s
Received healthy response to inference request in 1.9793071746826172s
Received healthy response to inference request in 2.297027826309204s
Received healthy response to inference request in 2.0743393898010254s
Received healthy response to inference request in 2.3597829341888428s
Received healthy response to inference request in 1.9178526401519775s
Received healthy response to inference request in 2.1379942893981934s
Received healthy response to inference request in 2.0280113220214844s
Received healthy response to inference request in 1.9443318843841553s
Received healthy response to inference request in 2.1558265686035156s
Received healthy response to inference request in 2.200977325439453s
Received healthy response to inference request in 2.3416736125946045s
Received healthy response to inference request in 2.313014030456543s
Received healthy response to inference request in 2.085465908050537s
Received healthy response to inference request in 2.046260356903076s
Received healthy response to inference request in 2.053713798522949s
Received healthy response to inference request in 2.0990042686462402s
Received healthy response to inference request in 2.1963164806365967s
Received healthy response to inference request in 2.4077908992767334s
Received healthy response to inference request in 2.2479209899902344s
Received healthy response to inference request in 2.0413148403167725s
Received healthy response to inference request in 2.110374689102173s
30 requests
0 failed requests
5th percentile: 1.9486990571022034
10th percentile: 1.9693888187408448
20th percentile: 1.993977689743042
30th percentile: 2.039769673347473
40th percentile: 2.066089153289795
50th percentile: 2.0922350883483887
60th percentile: 2.1389464378356933
70th percentile: 2.1892552375793457
80th percentile: 2.2369625091552736
90th percentile: 2.315879988670349
95th percentile: 2.3516337394714353
99th percentile: 2.393868589401245
mean time: 2.1209113677342732
Pipeline stage StressChecker completed in 67.84s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyTriggerPipeline
run_pipeline:run_in_cloud %s
starting trigger_guanaco_pipeline args=%s
triggered trigger_guanaco_pipeline args=%s
Pipeline stage OfflineFamilyFriendlyTriggerPipeline completed in 0.65s
Shutdown handler de-registered
chaiml-grpo-q235b-kimid_83709_v2 status is now deployed due to DeploymentManager action
chaiml-grpo-q235b-kimid_83709_v2 status is now inactive due to auto deactivation removed underperforming models
chaiml-grpo-q235b-kimid_83709_v2 status is now inactive due to Froze recruitment for AB test 0220_feynman
Deleting key chaiml-grpo-q235b-opusd-34252-v2/default/tokenizer_config.json from bucket guanaco-vllm-models
chaiml-grpo-q235b-kimid_35892_v2 status is now torndown due to DeploymentManager action
Deleting key chaiml-grpo-q235b-opusd-35623-v1/default/vocab.json from bucket guanaco-vllm-models