chaiml-reward-dpo-1c36-_72316

developer_uid: rirv938
submission_id: chaiml-reward-dpo-1c36-_72316_v1
model_name: chaiml-reward-dpo-1c36-_72316_v1
model_group: ChaiML/reward-dpo-1c36-c
status: torndown
timestamp: 2026-02-22T10:31:43+00:00
num_battles: 10952
num_wins: 6382
celo_rating: 1355.71
family_friendly_score: 0.0
family_friendly_standard_error: 0.0
submission_type: basic
model_repo: ChaiML/reward-dpo-1c36-chaiml-235b-sft-new-rm-_42002_v1-W4A16
model_architecture: Qwen3MoeForCausalLM
model_num_parameters: 18790207488.0
best_of: 8
max_input_tokens: 2048
max_output_tokens: 80
reward_model: default
display_name: chaiml-reward-dpo-1c36-_72316_v1
ineligible_reason: max_output_tokens!=64
is_internal_developer: True
language_model: ChaiML/reward-dpo-1c36-chaiml-235b-sft-new-rm-_42002_v1-W4A16
model_size: 19B
ranking_group: single
us_pacific_date: 2026-02-19
win_ratio: 0.5827246165084002
generation_params: {'temperature': 1.0, 'top_p': 0.95, 'min_p': 0.05, 'top_k': 60, 'presence_penalty': 0.1, 'frequency_penalty': 0.0, 'stopping_words': ['<|im_end|>', '###', 'You:', '<|im_start|>', '</s>'], 'max_input_tokens': 2048, 'best_of': 8, 'max_output_tokens': 80}
formatter: {'memory_template': '<|im_start|>system\nRespond as a high quality storyteller.<|im_end|>\n<|im_start|>user\n', 'prompt_template': '', 'bot_template': '{bot_name}: {message}\n', 'user_template': '{user_name}: {message}\n', 'response_template': '<|im_end|>\n<|im_start|>assistant\n{bot_name}:', 'truncate_by_message': False}
Resubmit model
Shutdown handler not registered because Python interpreter is not running in the main thread
run pipeline %s
run pipeline stage %s
Running pipeline stage VLLMUploader
Starting job with name chaiml-reward-dpo-1c36-72316-v1-uploader
Waiting for job on chaiml-reward-dpo-1c36-72316-v1-uploader to finish
chaiml-reward-dpo-1c36-72316-v1-uploader: Using quantization_mode: w4a16
chaiml-reward-dpo-1c36-72316-v1-uploader: Repo ChaiML/reward-dpo-1c36-chaiml-235b-sft-new-rm-_42002_v1-W4A16 already ends in W4A16. Skipping...
chaiml-reward-dpo-1c36-72316-v1-uploader: Checking if ChaiML/reward-dpo-1c36-chaiml-235b-sft-new-rm-_42002_v1-W4A16 already exists in ChaiML
chaiml-reward-dpo-1c36-72316-v1-uploader: Model already exists. Downloading to /dev/shm/model_output...
chaiml-reward-dpo-1c36-72316-v1-uploader: Downloading snapshot of ChaiML/reward-dpo-1c36-chaiml-235b-sft-new-rm-_42002_v1-W4A16...
Failed to get response for submission chaiml-grpo-q235b-kimid_37540_v1: HTTPConnectionPool(host='chaiml-grpo-q235b-kimid-37540-v1-predictor.tenant-chaiml-guanaco.kchai-coreweave-us-east-04a.chaiverse.com', port=80): Read timed out. (read timeout=12.0)
chaiml-reward-dpo-1c36-72316-v1-uploader: Downloaded in 64.633s
chaiml-reward-dpo-1c36-72316-v1-uploader: Processed model ChaiML/reward-dpo-1c36-chaiml-235b-sft-new-rm-_42002_v1-W4A16 in 65.267s
chaiml-reward-dpo-1c36-72316-v1-uploader: creating bucket guanaco-vllm-models
chaiml-reward-dpo-1c36-72316-v1-uploader: /usr/lib/python3/dist-packages/S3/BaseUtils.py:56: SyntaxWarning: invalid escape sequence '\.'
chaiml-reward-dpo-1c36-72316-v1-uploader:   RE_S3_DATESTRING = re.compile('\.[0-9]*(?:[Z\\-\\+]*?)')
chaiml-reward-dpo-1c36-72316-v1-uploader: /usr/lib/python3/dist-packages/S3/BaseUtils.py:57: SyntaxWarning: invalid escape sequence '\s'
chaiml-reward-dpo-1c36-72316-v1-uploader:   RE_XML_NAMESPACE = re.compile(b'^(<?[^>]+?>\s*|\s*)(<\w+) xmlns=[\'"](https?://[^\'"]+)[\'"]', re.MULTILINE)
chaiml-reward-dpo-1c36-72316-v1-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:240: SyntaxWarning: invalid escape sequence '\.'
chaiml-reward-dpo-1c36-72316-v1-uploader:   invalid = re.search("([^a-z0-9\.-])", bucket, re.UNICODE)
chaiml-reward-dpo-1c36-72316-v1-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:244: SyntaxWarning: invalid escape sequence '\.'
chaiml-reward-dpo-1c36-72316-v1-uploader:   invalid = re.search("([^A-Za-z0-9\._-])", bucket, re.UNICODE)
chaiml-reward-dpo-1c36-72316-v1-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:255: SyntaxWarning: invalid escape sequence '\.'
chaiml-reward-dpo-1c36-72316-v1-uploader:   if re.search("-\.", bucket, re.UNICODE):
chaiml-reward-dpo-1c36-72316-v1-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:257: SyntaxWarning: invalid escape sequence '\.'
chaiml-reward-dpo-1c36-72316-v1-uploader:   if re.search("\.\.", bucket, re.UNICODE):
chaiml-reward-dpo-1c36-72316-v1-uploader: /usr/lib/python3/dist-packages/S3/S3Uri.py:155: SyntaxWarning: invalid escape sequence '\w'
chaiml-reward-dpo-1c36-72316-v1-uploader:   _re = re.compile("^(\w+://)?(.*)", re.UNICODE)
chaiml-reward-dpo-1c36-72316-v1-uploader: /usr/lib/python3/dist-packages/S3/FileLists.py:480: SyntaxWarning: invalid escape sequence '\*'
chaiml-reward-dpo-1c36-72316-v1-uploader:   wildcard_split_result = re.split("\*|\?", uri_str, maxsplit=1)
chaiml-reward-dpo-1c36-72316-v1-uploader: Bucket 's3://guanaco-vllm-models/' created
chaiml-reward-dpo-1c36-72316-v1-uploader: uploading /dev/shm/model_output to s3://guanaco-vllm-models/chaiml-reward-dpo-1c36-72316-v1/default
chaiml-reward-dpo-1c36-72316-v1-uploader: cp /dev/shm/model_output/.gitattributes s3://guanaco-vllm-models/chaiml-reward-dpo-1c36-72316-v1/default/.gitattributes
chaiml-reward-dpo-1c36-72316-v1-uploader: cp /dev/shm/model_output/chat_template.jinja s3://guanaco-vllm-models/chaiml-reward-dpo-1c36-72316-v1/default/chat_template.jinja
chaiml-reward-dpo-1c36-72316-v1-uploader: cp /dev/shm/model_output/tokenizer_config.json s3://guanaco-vllm-models/chaiml-reward-dpo-1c36-72316-v1/default/tokenizer_config.json
chaiml-reward-dpo-1c36-72316-v1-uploader: cp /dev/shm/model_output/generation_config.json s3://guanaco-vllm-models/chaiml-reward-dpo-1c36-72316-v1/default/generation_config.json
chaiml-reward-dpo-1c36-72316-v1-uploader: cp /dev/shm/model_output/added_tokens.json s3://guanaco-vllm-models/chaiml-reward-dpo-1c36-72316-v1/default/added_tokens.json
chaiml-reward-dpo-1c36-72316-v1-uploader: cp /dev/shm/model_output/special_tokens_map.json s3://guanaco-vllm-models/chaiml-reward-dpo-1c36-72316-v1/default/special_tokens_map.json
chaiml-reward-dpo-1c36-72316-v1-uploader: cp /dev/shm/model_output/config.json s3://guanaco-vllm-models/chaiml-reward-dpo-1c36-72316-v1/default/config.json
chaiml-reward-dpo-1c36-72316-v1-uploader: cp /dev/shm/model_output/quantization_config.json s3://guanaco-vllm-models/chaiml-reward-dpo-1c36-72316-v1/default/quantization_config.json
chaiml-reward-dpo-1c36-72316-v1-uploader: cp /dev/shm/model_output/vocab.json s3://guanaco-vllm-models/chaiml-reward-dpo-1c36-72316-v1/default/vocab.json
chaiml-reward-dpo-1c36-72316-v1-uploader: cp /dev/shm/model_output/merges.txt s3://guanaco-vllm-models/chaiml-reward-dpo-1c36-72316-v1/default/merges.txt
chaiml-reward-dpo-1c36-72316-v1-uploader: cp /dev/shm/model_output/model.safetensors.index.json s3://guanaco-vllm-models/chaiml-reward-dpo-1c36-72316-v1/default/model.safetensors.index.json
chaiml-reward-dpo-1c36-72316-v1-uploader: cp /dev/shm/model_output/tokenizer.json s3://guanaco-vllm-models/chaiml-reward-dpo-1c36-72316-v1/default/tokenizer.json
Failed to get response for submission chaiml-mistral-24b-2048_54327_v6: ('http://chaiml-mistral-24b-2048-54327-v6-predictor.tenant-chaiml-guanaco.kchai-coreweave-us-east-04a.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', '')
Failed to get response for submission chaiml-mistral-24b-2048-_2678_v3: ('http://chaiml-mistral-24b-2048-2678-v3-predictor.tenant-chaiml-guanaco.kchai-coreweave-us-east-04a.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', '')
chaiml-reward-dpo-1c36-72316-v1-uploader: cp /dev/shm/model_output/model-00027-of-00027.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-1c36-72316-v1/default/model-00027-of-00027.safetensors
Failed to get response for submission chaiml-grpo-q235b-kimid_37540_v1: HTTPConnectionPool(host='chaiml-grpo-q235b-kimid-37540-v1-predictor.tenant-chaiml-guanaco.kchai-coreweave-us-east-04a.chaiverse.com', port=80): Read timed out. (read timeout=12.0)
Failed to get response for submission chaiml-mistral-24b-2048_15988_v1: ('http://chaiml-mistral-24b-2048-15988-v1-predictor.tenant-chaiml-guanaco.kchai-coreweave-us-east-04a.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', '')
Failed to get response for submission chaiml-grpo-q235b-kimid_37540_v1: HTTPConnectionPool(host='chaiml-grpo-q235b-kimid-37540-v1-predictor.tenant-chaiml-guanaco.kchai-coreweave-us-east-04a.chaiverse.com', port=80): Read timed out. (read timeout=12.0)
chaiml-reward-dpo-1c36-72316-v1-uploader: cp /dev/shm/model_output/model-00019-of-00027.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-1c36-72316-v1/default/model-00019-of-00027.safetensors
chaiml-reward-dpo-1c36-72316-v1-uploader: cp /dev/shm/model_output/model-00015-of-00027.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-1c36-72316-v1/default/model-00015-of-00027.safetensors
chaiml-reward-dpo-1c36-72316-v1-uploader: cp /dev/shm/model_output/model-00024-of-00027.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-1c36-72316-v1/default/model-00024-of-00027.safetensors
chaiml-reward-dpo-1c36-72316-v1-uploader: cp /dev/shm/model_output/model-00012-of-00027.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-1c36-72316-v1/default/model-00012-of-00027.safetensors
chaiml-reward-dpo-1c36-72316-v1-uploader: cp /dev/shm/model_output/model-00023-of-00027.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-1c36-72316-v1/default/model-00023-of-00027.safetensors
chaiml-reward-dpo-1c36-72316-v1-uploader: cp /dev/shm/model_output/model-00011-of-00027.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-1c36-72316-v1/default/model-00011-of-00027.safetensors
chaiml-reward-dpo-1c36-72316-v1-uploader: cp /dev/shm/model_output/model-00025-of-00027.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-1c36-72316-v1/default/model-00025-of-00027.safetensors
chaiml-reward-dpo-1c36-72316-v1-uploader: cp /dev/shm/model_output/model-00022-of-00027.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-1c36-72316-v1/default/model-00022-of-00027.safetensors
chaiml-reward-dpo-1c36-72316-v1-uploader: cp /dev/shm/model_output/model-00026-of-00027.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-1c36-72316-v1/default/model-00026-of-00027.safetensors
chaiml-reward-dpo-1c36-72316-v1-uploader: cp /dev/shm/model_output/model-00020-of-00027.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-1c36-72316-v1/default/model-00020-of-00027.safetensors
chaiml-reward-dpo-1c36-72316-v1-uploader: cp /dev/shm/model_output/model-00018-of-00027.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-1c36-72316-v1/default/model-00018-of-00027.safetensors
chaiml-reward-dpo-1c36-72316-v1-uploader: cp /dev/shm/model_output/model-00016-of-00027.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-1c36-72316-v1/default/model-00016-of-00027.safetensors
chaiml-reward-dpo-1c36-72316-v1-uploader: cp /dev/shm/model_output/model-00002-of-00027.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-1c36-72316-v1/default/model-00002-of-00027.safetensors
chaiml-reward-dpo-1c36-72316-v1-uploader: cp /dev/shm/model_output/model-00003-of-00027.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-1c36-72316-v1/default/model-00003-of-00027.safetensors
chaiml-reward-dpo-1c36-72316-v1-uploader: cp /dev/shm/model_output/model-00017-of-00027.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-1c36-72316-v1/default/model-00017-of-00027.safetensors
chaiml-reward-dpo-1c36-72316-v1-uploader: cp /dev/shm/model_output/model-00009-of-00027.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-1c36-72316-v1/default/model-00009-of-00027.safetensors
chaiml-reward-dpo-1c36-72316-v1-uploader: cp /dev/shm/model_output/model-00001-of-00027.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-1c36-72316-v1/default/model-00001-of-00027.safetensors
chaiml-reward-dpo-1c36-72316-v1-uploader: cp /dev/shm/model_output/model-00007-of-00027.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-1c36-72316-v1/default/model-00007-of-00027.safetensors
chaiml-reward-dpo-1c36-72316-v1-uploader: cp /dev/shm/model_output/model-00014-of-00027.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-1c36-72316-v1/default/model-00014-of-00027.safetensors
chaiml-reward-dpo-1c36-72316-v1-uploader: cp /dev/shm/model_output/model-00008-of-00027.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-1c36-72316-v1/default/model-00008-of-00027.safetensors
chaiml-reward-dpo-1c36-72316-v1-uploader: cp /dev/shm/model_output/model-00005-of-00027.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-1c36-72316-v1/default/model-00005-of-00027.safetensors
chaiml-reward-dpo-1c36-72316-v1-uploader: cp /dev/shm/model_output/model-00006-of-00027.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-1c36-72316-v1/default/model-00006-of-00027.safetensors
chaiml-reward-dpo-1c36-72316-v1-uploader: cp /dev/shm/model_output/model-00010-of-00027.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-1c36-72316-v1/default/model-00010-of-00027.safetensors
chaiml-reward-dpo-1c36-72316-v1-uploader: cp /dev/shm/model_output/model-00021-of-00027.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-1c36-72316-v1/default/model-00021-of-00027.safetensors
chaiml-reward-dpo-1c36-72316-v1-uploader: cp /dev/shm/model_output/model-00004-of-00027.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-1c36-72316-v1/default/model-00004-of-00027.safetensors
Job chaiml-reward-dpo-1c36-72316-v1-uploader completed after 995.29s with status: succeeded
Stopping job with name chaiml-reward-dpo-1c36-72316-v1-uploader
Pipeline stage VLLMUploader completed in 995.85s
run pipeline stage %s
Running pipeline stage VLLMTemplater
Pipeline stage VLLMTemplater completed in 0.16s
run pipeline stage %s
Running pipeline stage VLLMDeployer
Creating inference service chaiml-reward-dpo-1c36-72316-v1
Waiting for inference service chaiml-reward-dpo-1c36-72316-v1 to be ready
Failed to get response for submission chaiml-mistral-24b-2048_15988_v1: ('http://chaiml-mistral-24b-2048-15988-v1-predictor.tenant-chaiml-guanaco.kchai-coreweave-us-east-04a.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', '')
Failed to get response for submission chaiml-mistral-24b-2048-_2678_v3: ('http://chaiml-mistral-24b-2048-2678-v3-predictor.tenant-chaiml-guanaco.kchai-coreweave-us-east-04a.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', '')
Failed to get response for submission chaiml-mistral-24b-2048_15988_v1: ('http://chaiml-mistral-24b-2048-15988-v1-predictor.tenant-chaiml-guanaco.kchai-coreweave-us-east-04a.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', '')
Failed to get response for submission chaiml-mistral-24b-2048_54327_v6: ('http://chaiml-mistral-24b-2048-54327-v6-predictor.tenant-chaiml-guanaco.kchai-coreweave-us-east-04a.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', '')
HTTP Request: %s %s "%s %d %s"
Failed to get response for submission chaiml-mistral-24b-2048-_2678_v3: ('http://chaiml-mistral-24b-2048-2678-v3-predictor.tenant-chaiml-guanaco.kchai-coreweave-us-east-04a.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', '')
Failed to get response for submission chaiml-mistral-24b-2048-_2678_v3: ('http://chaiml-mistral-24b-2048-2678-v3-predictor.tenant-chaiml-guanaco.kchai-coreweave-us-east-04a.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', '')
Failed to get response for submission chaiml-mistral-24b-2048-_2678_v3: ('http://chaiml-mistral-24b-2048-2678-v3-predictor.tenant-chaiml-guanaco.kchai-coreweave-us-east-04a.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', '')
Inference service chaiml-reward-dpo-1c36-72316-v1 ready after 1064.7678124904633s
Pipeline stage VLLMDeployer completed in 1065.36s
run pipeline stage %s
Running pipeline stage StressChecker
Received healthy response to inference request in 2.1934444904327393s
Received healthy response to inference request in 1.969421625137329s
Received healthy response to inference request in 1.9321064949035645s
Received healthy response to inference request in 2.443761110305786s
Received healthy response to inference request in 1.9081439971923828s
Received healthy response to inference request in 1.9365572929382324s
Received healthy response to inference request in 2.0729198455810547s
Received healthy response to inference request in 1.89988112449646s
Received healthy response to inference request in 2.0796942710876465s
Received healthy response to inference request in 1.9233689308166504s
Received healthy response to inference request in 2.0053439140319824s
Received healthy response to inference request in 1.9873073101043701s
Received healthy response to inference request in 2.120572805404663s
Received healthy response to inference request in 1.9777603149414062s
Received healthy response to inference request in 1.932210922241211s
Received healthy response to inference request in 1.9318439960479736s
Received healthy response to inference request in 2.0094282627105713s
HTTP Request: %s %s "%s %d %s"
Received healthy response to inference request in 2.0446932315826416s
Received healthy response to inference request in 1.951197624206543s
Received healthy response to inference request in 2.0329818725585938s
Received healthy response to inference request in 1.9893765449523926s
Received healthy response to inference request in 2.0374624729156494s
Received healthy response to inference request in 2.020937919616699s
Received healthy response to inference request in 2.243905782699585s
Received healthy response to inference request in 1.9731297492980957s
Received healthy response to inference request in 2.1130495071411133s
Received healthy response to inference request in 2.0140702724456787s
Received healthy response to inference request in 2.127493381500244s
Received healthy response to inference request in 2.197164535522461s
Received healthy response to inference request in 1.9930901527404785s
30 requests
0 failed requests
5th percentile: 1.9149952173233031
10th percentile: 1.9309964895248413
20th percentile: 1.9356880187988281
30th percentile: 1.9720173120498656
40th percentile: 1.9885488510131837
50th percentile: 2.007386088371277
60th percentile: 2.025755500793457
70th percentile: 2.0531612157821653
80th percentile: 2.1145541667938232
90th percentile: 2.1938164949417116
95th percentile: 2.222872221469879
99th percentile: 2.3858030652999878
mean time: 2.0354106585184732
Pipeline stage StressChecker completed in 64.78s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyTriggerPipeline
run_pipeline:run_in_cloud %s
starting trigger_guanaco_pipeline args=%s
triggered trigger_guanaco_pipeline args=%s
Pipeline stage OfflineFamilyFriendlyTriggerPipeline completed in 1.15s
Shutdown handler de-registered
chaiml-reward-dpo-1c36-_72316_v1 status is now deployed due to DeploymentManager action
chaiml-reward-dpo-1c36-_72316_v1 status is now inactive due to auto deactivation removed underperforming models
chaiml-reward-dpo-1c36-_72316_v1 status is now torndown due to DeploymentManager action