chaiml-reward-dpo-52f9-_47866

developer_uid: rirv938
submission_id: chaiml-reward-dpo-52f9-_47866_v3
model_name: chaiml-reward-dpo-52f9-_47866_v3
model_group: ChaiML/reward-dpo-52f9-c
status: inactive
timestamp: 2026-02-20T04:26:56+00:00
num_battles: 10774
num_wins: 6183
celo_rating: 1350.41
family_friendly_score: 0.0
family_friendly_standard_error: 0.0
submission_type: basic
model_repo: ChaiML/reward-dpo-52f9-chaiml-235b-sft-prod-rm_38783_v1-W4A16
model_architecture: Qwen3MoeForCausalLM
model_num_parameters: 18790207488.0
best_of: 8
max_input_tokens: 2048
max_output_tokens: 80
reward_model: default
display_name: chaiml-reward-dpo-52f9-_47866_v3
ineligible_reason: max_output_tokens!=64
is_internal_developer: True
language_model: ChaiML/reward-dpo-52f9-chaiml-235b-sft-prod-rm_38783_v1-W4A16
model_size: 19B
ranking_group: single
us_pacific_date: 2026-02-19
win_ratio: 0.5738815667347318
generation_params: {'temperature': 1.0, 'top_p': 0.95, 'min_p': 0.05, 'top_k': 60, 'presence_penalty': 0.1, 'frequency_penalty': 0.0, 'stopping_words': ['###', '</s>', '<|im_end|>', '<|im_start|>', 'You:'], 'max_input_tokens': 2048, 'best_of': 8, 'max_output_tokens': 80}
formatter: {'memory_template': '<|im_start|>system\nRespond as a high quality storyteller.<|im_end|>\n<|im_start|>user\n', 'prompt_template': '', 'bot_template': '{bot_name}: {message}\n', 'user_template': '{user_name}: {message}\n', 'response_template': '<|im_end|>\n<|im_start|>assistant\n{bot_name}:', 'truncate_by_message': False}
Resubmit model
Shutdown handler not registered because Python interpreter is not running in the main thread
run pipeline %s
run pipeline stage %s
Running pipeline stage VLLMUploader
Starting job with name chaiml-reward-dpo-52f9-47866-v3-uploader
Waiting for job on chaiml-reward-dpo-52f9-47866-v3-uploader to finish
chaiml-reward-dpo-52f9-47866-v3-uploader: Using quantization_mode: w4a16
chaiml-reward-dpo-52f9-47866-v3-uploader: Repo ChaiML/reward-dpo-52f9-chaiml-235b-sft-prod-rm_38783_v1-W4A16 already ends in W4A16. Skipping...
chaiml-reward-dpo-52f9-47866-v3-uploader: Checking if ChaiML/reward-dpo-52f9-chaiml-235b-sft-prod-rm_38783_v1-W4A16 already exists in ChaiML
chaiml-reward-dpo-52f9-47866-v3-uploader: Model already exists. Downloading to /dev/shm/model_output...
chaiml-reward-dpo-52f9-47866-v3-uploader: Downloading snapshot of ChaiML/reward-dpo-52f9-chaiml-235b-sft-prod-rm_38783_v1-W4A16...
chaiml-reward-dpo-52f9-47866-v3-uploader: Downloaded in 67.984s
chaiml-reward-dpo-52f9-47866-v3-uploader: Processed model ChaiML/reward-dpo-52f9-chaiml-235b-sft-prod-rm_38783_v1-W4A16 in 68.634s
chaiml-reward-dpo-52f9-47866-v3-uploader: creating bucket guanaco-vllm-models
chaiml-reward-dpo-52f9-47866-v3-uploader: /usr/lib/python3/dist-packages/S3/BaseUtils.py:56: SyntaxWarning: invalid escape sequence '\.'
chaiml-reward-dpo-52f9-47866-v3-uploader:   RE_S3_DATESTRING = re.compile('\.[0-9]*(?:[Z\\-\\+]*?)')
chaiml-reward-dpo-52f9-47866-v3-uploader: /usr/lib/python3/dist-packages/S3/BaseUtils.py:57: SyntaxWarning: invalid escape sequence '\s'
chaiml-reward-dpo-52f9-47866-v3-uploader:   RE_XML_NAMESPACE = re.compile(b'^(<?[^>]+?>\s*|\s*)(<\w+) xmlns=[\'"](https?://[^\'"]+)[\'"]', re.MULTILINE)
chaiml-reward-dpo-52f9-47866-v3-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:240: SyntaxWarning: invalid escape sequence '\.'
chaiml-reward-dpo-52f9-47866-v3-uploader:   invalid = re.search("([^a-z0-9\.-])", bucket, re.UNICODE)
chaiml-reward-dpo-52f9-47866-v3-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:244: SyntaxWarning: invalid escape sequence '\.'
chaiml-reward-dpo-52f9-47866-v3-uploader:   invalid = re.search("([^A-Za-z0-9\._-])", bucket, re.UNICODE)
chaiml-reward-dpo-52f9-47866-v3-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:255: SyntaxWarning: invalid escape sequence '\.'
chaiml-reward-dpo-52f9-47866-v3-uploader:   if re.search("-\.", bucket, re.UNICODE):
chaiml-reward-dpo-52f9-47866-v3-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:257: SyntaxWarning: invalid escape sequence '\.'
chaiml-reward-dpo-52f9-47866-v3-uploader:   if re.search("\.\.", bucket, re.UNICODE):
chaiml-reward-dpo-52f9-47866-v3-uploader: /usr/lib/python3/dist-packages/S3/S3Uri.py:155: SyntaxWarning: invalid escape sequence '\w'
chaiml-reward-dpo-52f9-47866-v3-uploader:   _re = re.compile("^(\w+://)?(.*)", re.UNICODE)
chaiml-reward-dpo-52f9-47866-v3-uploader: /usr/lib/python3/dist-packages/S3/FileLists.py:480: SyntaxWarning: invalid escape sequence '\*'
chaiml-reward-dpo-52f9-47866-v3-uploader:   wildcard_split_result = re.split("\*|\?", uri_str, maxsplit=1)
chaiml-reward-dpo-52f9-47866-v3-uploader: Bucket 's3://guanaco-vllm-models/' created
chaiml-reward-dpo-52f9-47866-v3-uploader: uploading /dev/shm/model_output to s3://guanaco-vllm-models/chaiml-reward-dpo-52f9-47866-v3/default
chaiml-reward-dpo-52f9-47866-v3-uploader: cp /dev/shm/model_output/config.json s3://guanaco-vllm-models/chaiml-reward-dpo-52f9-47866-v3/default/config.json
chaiml-reward-dpo-52f9-47866-v3-uploader: cp /dev/shm/model_output/merges.txt s3://guanaco-vllm-models/chaiml-reward-dpo-52f9-47866-v3/default/merges.txt
chaiml-reward-dpo-52f9-47866-v3-uploader: cp /dev/shm/model_output/added_tokens.json s3://guanaco-vllm-models/chaiml-reward-dpo-52f9-47866-v3/default/added_tokens.json
chaiml-reward-dpo-52f9-47866-v3-uploader: cp /dev/shm/model_output/special_tokens_map.json s3://guanaco-vllm-models/chaiml-reward-dpo-52f9-47866-v3/default/special_tokens_map.json
chaiml-reward-dpo-52f9-47866-v3-uploader: cp /dev/shm/model_output/.gitattributes s3://guanaco-vllm-models/chaiml-reward-dpo-52f9-47866-v3/default/.gitattributes
chaiml-reward-dpo-52f9-47866-v3-uploader: cp /dev/shm/model_output/quantization_config.json s3://guanaco-vllm-models/chaiml-reward-dpo-52f9-47866-v3/default/quantization_config.json
chaiml-reward-dpo-52f9-47866-v3-uploader: cp /dev/shm/model_output/chat_template.jinja s3://guanaco-vllm-models/chaiml-reward-dpo-52f9-47866-v3/default/chat_template.jinja
chaiml-reward-dpo-52f9-47866-v3-uploader: cp /dev/shm/model_output/generation_config.json s3://guanaco-vllm-models/chaiml-reward-dpo-52f9-47866-v3/default/generation_config.json
chaiml-reward-dpo-52f9-47866-v3-uploader: cp /dev/shm/model_output/tokenizer_config.json s3://guanaco-vllm-models/chaiml-reward-dpo-52f9-47866-v3/default/tokenizer_config.json
chaiml-reward-dpo-52f9-47866-v3-uploader: cp /dev/shm/model_output/model.safetensors.index.json s3://guanaco-vllm-models/chaiml-reward-dpo-52f9-47866-v3/default/model.safetensors.index.json
chaiml-reward-dpo-52f9-47866-v3-uploader: cp /dev/shm/model_output/vocab.json s3://guanaco-vllm-models/chaiml-reward-dpo-52f9-47866-v3/default/vocab.json
chaiml-reward-dpo-52f9-47866-v3-uploader: cp /dev/shm/model_output/tokenizer.json s3://guanaco-vllm-models/chaiml-reward-dpo-52f9-47866-v3/default/tokenizer.json
chaiml-reward-dpo-52f9-47866-v3-uploader: cp /dev/shm/model_output/model-00027-of-00027.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-52f9-47866-v3/default/model-00027-of-00027.safetensors
Failed to get response for submission chaiml-mistral-24b-2048_15988_v1: ('http://chaiml-mistral-24b-2048-15988-v1-predictor.tenant-chaiml-guanaco.kchai-coreweave-us-east-04a.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', '')
chaiml-reward-dpo-52f9-47866-v3-uploader: cp /dev/shm/model_output/model-00001-of-00027.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-52f9-47866-v3/default/model-00001-of-00027.safetensors
chaiml-reward-dpo-52f9-47866-v3-uploader: cp /dev/shm/model_output/model-00008-of-00027.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-52f9-47866-v3/default/model-00008-of-00027.safetensors
chaiml-reward-dpo-52f9-47866-v3-uploader: cp /dev/shm/model_output/model-00019-of-00027.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-52f9-47866-v3/default/model-00019-of-00027.safetensors
chaiml-reward-dpo-52f9-47866-v3-uploader: cp /dev/shm/model_output/model-00026-of-00027.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-52f9-47866-v3/default/model-00026-of-00027.safetensors
chaiml-reward-dpo-52f9-47866-v3-uploader: cp /dev/shm/model_output/model-00002-of-00027.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-52f9-47866-v3/default/model-00002-of-00027.safetensors
chaiml-reward-dpo-52f9-47866-v3-uploader: cp /dev/shm/model_output/model-00023-of-00027.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-52f9-47866-v3/default/model-00023-of-00027.safetensors
chaiml-reward-dpo-52f9-47866-v3-uploader: cp /dev/shm/model_output/model-00015-of-00027.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-52f9-47866-v3/default/model-00015-of-00027.safetensors
chaiml-reward-dpo-52f9-47866-v3-uploader: cp /dev/shm/model_output/model-00025-of-00027.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-52f9-47866-v3/default/model-00025-of-00027.safetensors
chaiml-reward-dpo-52f9-47866-v3-uploader: cp /dev/shm/model_output/model-00003-of-00027.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-52f9-47866-v3/default/model-00003-of-00027.safetensors
chaiml-reward-dpo-52f9-47866-v3-uploader: cp /dev/shm/model_output/model-00005-of-00027.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-52f9-47866-v3/default/model-00005-of-00027.safetensors
chaiml-reward-dpo-52f9-47866-v3-uploader: cp /dev/shm/model_output/model-00009-of-00027.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-52f9-47866-v3/default/model-00009-of-00027.safetensors
chaiml-reward-dpo-52f9-47866-v3-uploader: cp /dev/shm/model_output/model-00011-of-00027.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-52f9-47866-v3/default/model-00011-of-00027.safetensors
chaiml-reward-dpo-52f9-47866-v3-uploader: cp /dev/shm/model_output/model-00016-of-00027.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-52f9-47866-v3/default/model-00016-of-00027.safetensors
chaiml-reward-dpo-52f9-47866-v3-uploader: cp /dev/shm/model_output/model-00007-of-00027.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-52f9-47866-v3/default/model-00007-of-00027.safetensors
chaiml-reward-dpo-52f9-47866-v3-uploader: cp /dev/shm/model_output/model-00022-of-00027.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-52f9-47866-v3/default/model-00022-of-00027.safetensors
chaiml-reward-dpo-52f9-47866-v3-uploader: cp /dev/shm/model_output/model-00018-of-00027.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-52f9-47866-v3/default/model-00018-of-00027.safetensors
chaiml-reward-dpo-52f9-47866-v3-uploader: cp /dev/shm/model_output/model-00024-of-00027.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-52f9-47866-v3/default/model-00024-of-00027.safetensors
chaiml-reward-dpo-52f9-47866-v3-uploader: cp /dev/shm/model_output/model-00010-of-00027.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-52f9-47866-v3/default/model-00010-of-00027.safetensors
chaiml-reward-dpo-52f9-47866-v3-uploader: cp /dev/shm/model_output/model-00006-of-00027.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-52f9-47866-v3/default/model-00006-of-00027.safetensors
chaiml-reward-dpo-52f9-47866-v3-uploader: cp /dev/shm/model_output/model-00013-of-00027.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-52f9-47866-v3/default/model-00013-of-00027.safetensors
chaiml-reward-dpo-52f9-47866-v3-uploader: cp /dev/shm/model_output/model-00012-of-00027.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-52f9-47866-v3/default/model-00012-of-00027.safetensors
chaiml-reward-dpo-52f9-47866-v3-uploader: cp /dev/shm/model_output/model-00021-of-00027.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-52f9-47866-v3/default/model-00021-of-00027.safetensors
chaiml-reward-dpo-52f9-47866-v3-uploader: cp /dev/shm/model_output/model-00014-of-00027.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-52f9-47866-v3/default/model-00014-of-00027.safetensors
chaiml-reward-dpo-52f9-47866-v3-uploader: cp /dev/shm/model_output/model-00017-of-00027.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-52f9-47866-v3/default/model-00017-of-00027.safetensors
chaiml-reward-dpo-52f9-47866-v3-uploader: cp /dev/shm/model_output/model-00020-of-00027.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-52f9-47866-v3/default/model-00020-of-00027.safetensors
chaiml-reward-dpo-52f9-47866-v3-uploader: cp /dev/shm/model_output/model-00004-of-00027.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-52f9-47866-v3/default/model-00004-of-00027.safetensors
Job chaiml-reward-dpo-52f9-47866-v3-uploader completed after 237.57s with status: succeeded
Stopping job with name chaiml-reward-dpo-52f9-47866-v3-uploader
Pipeline stage VLLMUploader completed in 238.05s
run pipeline stage %s
Running pipeline stage VLLMTemplater
Pipeline stage VLLMTemplater completed in 0.14s
run pipeline stage %s
Running pipeline stage VLLMDeployer
Creating inference service chaiml-reward-dpo-52f9-47866-v3
Waiting for inference service chaiml-reward-dpo-52f9-47866-v3 to be ready
Failed to get response for submission chaiml-grpo-q235b-kimid_37540_v1: HTTPConnectionPool(host='chaiml-grpo-q235b-kimid-37540-v1-predictor.tenant-chaiml-guanaco.kchai-coreweave-us-east-04a.chaiverse.com', port=80): Read timed out. (read timeout=12.0)
Failed to get response for submission chaiml-mistral-24b-2048-_2678_v3: ('http://chaiml-mistral-24b-2048-2678-v3-predictor.tenant-chaiml-guanaco.kchai-coreweave-us-east-04a.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', '')
HTTP Request: %s %s "%s %d %s"
Failed to get response for submission chaiml-mistral-24b-2048_54327_v6: ('http://chaiml-mistral-24b-2048-54327-v6-predictor.tenant-chaiml-guanaco.kchai-coreweave-us-east-04a.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', '')
Failed to get response for submission chaiml-mistral-24b-2048_15988_v1: ('http://chaiml-mistral-24b-2048-15988-v1-predictor.tenant-chaiml-guanaco.kchai-coreweave-us-east-04a.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', '')
Failed to get response for submission chaiml-mistral-24b-2048_54327_v6: ('http://chaiml-mistral-24b-2048-54327-v6-predictor.tenant-chaiml-guanaco.kchai-coreweave-us-east-04a.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', '')
HTTP Request: %s %s "%s %d %s"
Failed to get response for submission chaiml-mistral-24b-2048_15988_v1: ('http://chaiml-mistral-24b-2048-15988-v1-predictor.tenant-chaiml-guanaco.kchai-coreweave-us-east-04a.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', '')
Inference service chaiml-reward-dpo-52f9-47866-v3 ready after 523.0012645721436s
Pipeline stage VLLMDeployer completed in 523.51s
run pipeline stage %s
Running pipeline stage StressChecker
Received healthy response to inference request in 2.2051689624786377s
Received healthy response to inference request in 2.017897129058838s
Received healthy response to inference request in 2.188236713409424s
Received healthy response to inference request in 2.0260443687438965s
Received healthy response to inference request in 2.060530185699463s
Received healthy response to inference request in 2.4392495155334473s
Received healthy response to inference request in 1.9075696468353271s
Received healthy response to inference request in 1.9208271503448486s
Received healthy response to inference request in 2.025421380996704s
Received healthy response to inference request in 2.056739330291748s
Received healthy response to inference request in 1.9719877243041992s
Received healthy response to inference request in 2.0458037853240967s
Received healthy response to inference request in 2.2404654026031494s
Received healthy response to inference request in 2.1353816986083984s
Received healthy response to inference request in 2.1058101654052734s
Received healthy response to inference request in 1.9473495483398438s
Received healthy response to inference request in 1.9979190826416016s
Received healthy response to inference request in 1.9991893768310547s
Received healthy response to inference request in 2.087541103363037s
Received healthy response to inference request in 2.2011020183563232s
Received healthy response to inference request in 2.063324451446533s
Received healthy response to inference request in 1.9818215370178223s
Received healthy response to inference request in 2.2767932415008545s
Received healthy response to inference request in 2.1077535152435303s
Received healthy response to inference request in 1.9740045070648193s
Received healthy response to inference request in 1.9487473964691162s
Received healthy response to inference request in 2.202235460281372s
Received healthy response to inference request in 2.0549044609069824s
Received healthy response to inference request in 2.00588059425354s
Received healthy response to inference request in 2.321544647216797s
30 requests
0 failed requests
5th percentile: 1.9327622294425963
10th percentile: 1.948607611656189
20th percentile: 1.9802581310272216
30th percentile: 2.0038732290267944
40th percentile: 2.0257951736450197
50th percentile: 2.0558218955993652
60th percentile: 2.073011112213135
70th percentile: 2.1160419702529905
80th percentile: 2.201328706741333
90th percentile: 2.24409818649292
95th percentile: 2.3014065146446225
99th percentile: 2.4051151037216187
mean time: 2.083908136685689
Pipeline stage StressChecker completed in 66.97s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyTriggerPipeline
run_pipeline:run_in_cloud %s
starting trigger_guanaco_pipeline args=%s
triggered trigger_guanaco_pipeline args=%s
Pipeline stage OfflineFamilyFriendlyTriggerPipeline completed in 1.34s
Shutdown handler de-registered
chaiml-reward-dpo-52f9-_47866_v3 status is now deployed due to DeploymentManager action
chaiml-reward-dpo-52f9-_47866_v3 status is now inactive due to auto deactivation removed underperforming models