chaiml-reward-dpo-79ab-_18815

developer_uid: rirv938
submission_id: chaiml-reward-dpo-79ab-_18815_v1
model_name: chaiml-reward-dpo-79ab-_18815_v1
model_group: ChaiML/reward-dpo-79ab-c
status: torndown
timestamp: 2026-02-22T10:31:44+00:00
num_battles: 10878
num_wins: 6196
celo_rating: 1348.39
family_friendly_score: 0.0
family_friendly_standard_error: 0.0
submission_type: basic
model_repo: ChaiML/reward-dpo-79ab-chaiml-235b-sft-new-rm-_42002_v1-W4A16
model_architecture: Qwen3MoeForCausalLM
model_num_parameters: 18790207488.0
best_of: 8
max_input_tokens: 2048
max_output_tokens: 80
reward_model: default
display_name: chaiml-reward-dpo-79ab-_18815_v1
ineligible_reason: max_output_tokens!=64
is_internal_developer: True
language_model: ChaiML/reward-dpo-79ab-chaiml-235b-sft-new-rm-_42002_v1-W4A16
model_size: 19B
ranking_group: single
us_pacific_date: 2026-02-19
win_ratio: 0.5695899981614267
generation_params: {'temperature': 1.0, 'top_p': 0.95, 'min_p': 0.05, 'top_k': 60, 'presence_penalty': 0.1, 'frequency_penalty': 0.0, 'stopping_words': ['<|im_start|>', 'You:', '<|im_end|>', '###', '</s>'], 'max_input_tokens': 2048, 'best_of': 8, 'max_output_tokens': 80}
formatter: {'memory_template': '<|im_start|>system\nRespond as a high quality storyteller.<|im_end|>\n<|im_start|>user\n', 'prompt_template': '', 'bot_template': '{bot_name}: {message}\n', 'user_template': '{user_name}: {message}\n', 'response_template': '<|im_end|>\n<|im_start|>assistant\n{bot_name}:', 'truncate_by_message': False}
Resubmit model
Shutdown handler not registered because Python interpreter is not running in the main thread
run pipeline %s
run pipeline stage %s
Running pipeline stage VLLMUploader
Starting job with name chaiml-reward-dpo-79ab-18815-v1-uploader
Waiting for job on chaiml-reward-dpo-79ab-18815-v1-uploader to finish
chaiml-reward-dpo-79ab-18815-v1-uploader: Using quantization_mode: w4a16
chaiml-reward-dpo-79ab-18815-v1-uploader: Repo ChaiML/reward-dpo-79ab-chaiml-235b-sft-new-rm-_42002_v1-W4A16 already ends in W4A16. Skipping...
chaiml-reward-dpo-79ab-18815-v1-uploader: Checking if ChaiML/reward-dpo-79ab-chaiml-235b-sft-new-rm-_42002_v1-W4A16 already exists in ChaiML
chaiml-reward-dpo-79ab-18815-v1-uploader: Model already exists. Downloading to /dev/shm/model_output...
chaiml-reward-dpo-79ab-18815-v1-uploader: Downloading snapshot of ChaiML/reward-dpo-79ab-chaiml-235b-sft-new-rm-_42002_v1-W4A16...
Failed to get response for submission chaiml-mistral-24b-2048-_2678_v3: ('http://chaiml-mistral-24b-2048-2678-v3-predictor.tenant-chaiml-guanaco.kchai-coreweave-us-east-04a.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', '')
Failed to get response for submission chaiml-mistral-24b-2048_54327_v6: ('http://chaiml-mistral-24b-2048-54327-v6-predictor.tenant-chaiml-guanaco.kchai-coreweave-us-east-04a.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', '')
chaiml-reward-dpo-79ab-18815-v1-uploader: Downloaded in 72.676s
chaiml-reward-dpo-79ab-18815-v1-uploader: Processed model ChaiML/reward-dpo-79ab-chaiml-235b-sft-new-rm-_42002_v1-W4A16 in 73.204s
chaiml-reward-dpo-79ab-18815-v1-uploader: creating bucket guanaco-vllm-models
chaiml-reward-dpo-79ab-18815-v1-uploader: /usr/lib/python3/dist-packages/S3/BaseUtils.py:56: SyntaxWarning: invalid escape sequence '\.'
chaiml-reward-dpo-79ab-18815-v1-uploader:   RE_S3_DATESTRING = re.compile('\.[0-9]*(?:[Z\\-\\+]*?)')
chaiml-reward-dpo-79ab-18815-v1-uploader: /usr/lib/python3/dist-packages/S3/BaseUtils.py:57: SyntaxWarning: invalid escape sequence '\s'
chaiml-reward-dpo-79ab-18815-v1-uploader:   RE_XML_NAMESPACE = re.compile(b'^(<?[^>]+?>\s*|\s*)(<\w+) xmlns=[\'"](https?://[^\'"]+)[\'"]', re.MULTILINE)
chaiml-reward-dpo-79ab-18815-v1-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:240: SyntaxWarning: invalid escape sequence '\.'
chaiml-reward-dpo-79ab-18815-v1-uploader:   invalid = re.search("([^a-z0-9\.-])", bucket, re.UNICODE)
chaiml-reward-dpo-79ab-18815-v1-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:244: SyntaxWarning: invalid escape sequence '\.'
chaiml-reward-dpo-79ab-18815-v1-uploader:   invalid = re.search("([^A-Za-z0-9\._-])", bucket, re.UNICODE)
chaiml-reward-dpo-79ab-18815-v1-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:255: SyntaxWarning: invalid escape sequence '\.'
chaiml-reward-dpo-79ab-18815-v1-uploader:   if re.search("-\.", bucket, re.UNICODE):
chaiml-reward-dpo-79ab-18815-v1-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:257: SyntaxWarning: invalid escape sequence '\.'
chaiml-reward-dpo-79ab-18815-v1-uploader:   if re.search("\.\.", bucket, re.UNICODE):
chaiml-reward-dpo-79ab-18815-v1-uploader: /usr/lib/python3/dist-packages/S3/S3Uri.py:155: SyntaxWarning: invalid escape sequence '\w'
chaiml-reward-dpo-79ab-18815-v1-uploader:   _re = re.compile("^(\w+://)?(.*)", re.UNICODE)
chaiml-reward-dpo-79ab-18815-v1-uploader: cp /dev/shm/model_output/special_tokens_map.json s3://guanaco-vllm-models/chaiml-reward-dpo-79ab-18815-v1/default/special_tokens_map.json
chaiml-reward-dpo-79ab-18815-v1-uploader: cp /dev/shm/model_output/added_tokens.json s3://guanaco-vllm-models/chaiml-reward-dpo-79ab-18815-v1/default/added_tokens.json
chaiml-reward-dpo-79ab-18815-v1-uploader: cp /dev/shm/model_output/tokenizer_config.json s3://guanaco-vllm-models/chaiml-reward-dpo-79ab-18815-v1/default/tokenizer_config.json
chaiml-reward-dpo-79ab-18815-v1-uploader: cp /dev/shm/model_output/generation_config.json s3://guanaco-vllm-models/chaiml-reward-dpo-79ab-18815-v1/default/generation_config.json
chaiml-reward-dpo-79ab-18815-v1-uploader: cp /dev/shm/model_output/chat_template.jinja s3://guanaco-vllm-models/chaiml-reward-dpo-79ab-18815-v1/default/chat_template.jinja
chaiml-reward-dpo-79ab-18815-v1-uploader: cp /dev/shm/model_output/.gitattributes s3://guanaco-vllm-models/chaiml-reward-dpo-79ab-18815-v1/default/.gitattributes
chaiml-reward-dpo-79ab-18815-v1-uploader: cp /dev/shm/model_output/quantization_config.json s3://guanaco-vllm-models/chaiml-reward-dpo-79ab-18815-v1/default/quantization_config.json
chaiml-reward-dpo-79ab-18815-v1-uploader: cp /dev/shm/model_output/config.json s3://guanaco-vllm-models/chaiml-reward-dpo-79ab-18815-v1/default/config.json
chaiml-reward-dpo-79ab-18815-v1-uploader: cp /dev/shm/model_output/merges.txt s3://guanaco-vllm-models/chaiml-reward-dpo-79ab-18815-v1/default/merges.txt
chaiml-reward-dpo-79ab-18815-v1-uploader: cp /dev/shm/model_output/vocab.json s3://guanaco-vllm-models/chaiml-reward-dpo-79ab-18815-v1/default/vocab.json
chaiml-reward-dpo-79ab-18815-v1-uploader: cp /dev/shm/model_output/model.safetensors.index.json s3://guanaco-vllm-models/chaiml-reward-dpo-79ab-18815-v1/default/model.safetensors.index.json
chaiml-reward-dpo-79ab-18815-v1-uploader: cp /dev/shm/model_output/tokenizer.json s3://guanaco-vllm-models/chaiml-reward-dpo-79ab-18815-v1/default/tokenizer.json
Failed to get response for submission chaiml-mistral-24b-2048-_2678_v3: ('http://chaiml-mistral-24b-2048-2678-v3-predictor.tenant-chaiml-guanaco.kchai-coreweave-us-east-04a.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', '')
Failed to get response for submission chaiml-mistral-24b-2048_54327_v6: ('http://chaiml-mistral-24b-2048-54327-v6-predictor.tenant-chaiml-guanaco.kchai-coreweave-us-east-04a.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', '')
chaiml-reward-dpo-79ab-18815-v1-uploader: cp /dev/shm/model_output/model-00027-of-00027.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-79ab-18815-v1/default/model-00027-of-00027.safetensors
Failed to get response for submission chaiml-mistral-24b-2048-_2678_v3: ('http://chaiml-mistral-24b-2048-2678-v3-predictor.tenant-chaiml-guanaco.kchai-coreweave-us-east-04a.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', '')
chaiml-reward-dpo-79ab-18815-v1-uploader: cp /dev/shm/model_output/model-00013-of-00027.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-79ab-18815-v1/default/model-00013-of-00027.safetensors
chaiml-reward-dpo-79ab-18815-v1-uploader: cp /dev/shm/model_output/model-00021-of-00027.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-79ab-18815-v1/default/model-00021-of-00027.safetensors
chaiml-reward-dpo-79ab-18815-v1-uploader: cp /dev/shm/model_output/model-00007-of-00027.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-79ab-18815-v1/default/model-00007-of-00027.safetensors
Failed to get response for submission chaiml-mistral-24b-2048_54327_v6: ('http://chaiml-mistral-24b-2048-54327-v6-predictor.tenant-chaiml-guanaco.kchai-coreweave-us-east-04a.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', '')
chaiml-reward-dpo-79ab-18815-v1-uploader: cp /dev/shm/model_output/model-00017-of-00027.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-79ab-18815-v1/default/model-00017-of-00027.safetensors
chaiml-reward-dpo-79ab-18815-v1-uploader: cp /dev/shm/model_output/model-00010-of-00027.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-79ab-18815-v1/default/model-00010-of-00027.safetensors
chaiml-reward-dpo-79ab-18815-v1-uploader: cp /dev/shm/model_output/model-00024-of-00027.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-79ab-18815-v1/default/model-00024-of-00027.safetensors
chaiml-reward-dpo-79ab-18815-v1-uploader: cp /dev/shm/model_output/model-00015-of-00027.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-79ab-18815-v1/default/model-00015-of-00027.safetensors
chaiml-reward-dpo-79ab-18815-v1-uploader: cp /dev/shm/model_output/model-00011-of-00027.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-79ab-18815-v1/default/model-00011-of-00027.safetensors
chaiml-reward-dpo-79ab-18815-v1-uploader: cp /dev/shm/model_output/model-00023-of-00027.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-79ab-18815-v1/default/model-00023-of-00027.safetensors
chaiml-reward-dpo-79ab-18815-v1-uploader: cp /dev/shm/model_output/model-00016-of-00027.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-79ab-18815-v1/default/model-00016-of-00027.safetensors
chaiml-reward-dpo-79ab-18815-v1-uploader: cp /dev/shm/model_output/model-00012-of-00027.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-79ab-18815-v1/default/model-00012-of-00027.safetensors
chaiml-reward-dpo-79ab-18815-v1-uploader: cp /dev/shm/model_output/model-00020-of-00027.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-79ab-18815-v1/default/model-00020-of-00027.safetensors
chaiml-reward-dpo-79ab-18815-v1-uploader: cp /dev/shm/model_output/model-00018-of-00027.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-79ab-18815-v1/default/model-00018-of-00027.safetensors
chaiml-reward-dpo-79ab-18815-v1-uploader: cp /dev/shm/model_output/model-00004-of-00027.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-79ab-18815-v1/default/model-00004-of-00027.safetensors
chaiml-reward-dpo-79ab-18815-v1-uploader: cp /dev/shm/model_output/model-00025-of-00027.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-79ab-18815-v1/default/model-00025-of-00027.safetensors
chaiml-reward-dpo-79ab-18815-v1-uploader: cp /dev/shm/model_output/model-00001-of-00027.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-79ab-18815-v1/default/model-00001-of-00027.safetensors
chaiml-reward-dpo-79ab-18815-v1-uploader: cp /dev/shm/model_output/model-00006-of-00027.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-79ab-18815-v1/default/model-00006-of-00027.safetensors
chaiml-reward-dpo-79ab-18815-v1-uploader: cp /dev/shm/model_output/model-00008-of-00027.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-79ab-18815-v1/default/model-00008-of-00027.safetensors
chaiml-reward-dpo-79ab-18815-v1-uploader: cp /dev/shm/model_output/model-00014-of-00027.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-79ab-18815-v1/default/model-00014-of-00027.safetensors
Failed to get response for submission chaiml-mistral-24b-2048_15988_v1: ('http://chaiml-mistral-24b-2048-15988-v1-predictor.tenant-chaiml-guanaco.kchai-coreweave-us-east-04a.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', '')
chaiml-reward-dpo-79ab-18815-v1-uploader: cp /dev/shm/model_output/model-00005-of-00027.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-79ab-18815-v1/default/model-00005-of-00027.safetensors
chaiml-reward-dpo-79ab-18815-v1-uploader: cp /dev/shm/model_output/model-00002-of-00027.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-79ab-18815-v1/default/model-00002-of-00027.safetensors
chaiml-reward-dpo-79ab-18815-v1-uploader: cp /dev/shm/model_output/model-00009-of-00027.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-79ab-18815-v1/default/model-00009-of-00027.safetensors
chaiml-reward-dpo-79ab-18815-v1-uploader: cp /dev/shm/model_output/model-00026-of-00027.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-79ab-18815-v1/default/model-00026-of-00027.safetensors
chaiml-reward-dpo-79ab-18815-v1-uploader: cp /dev/shm/model_output/model-00019-of-00027.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-79ab-18815-v1/default/model-00019-of-00027.safetensors
chaiml-reward-dpo-79ab-18815-v1-uploader: cp /dev/shm/model_output/model-00003-of-00027.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-79ab-18815-v1/default/model-00003-of-00027.safetensors
Job chaiml-reward-dpo-79ab-18815-v1-uploader completed after 993.1s with status: succeeded
Stopping job with name chaiml-reward-dpo-79ab-18815-v1-uploader
Pipeline stage VLLMUploader completed in 993.60s
run pipeline stage %s
Running pipeline stage VLLMTemplater
Pipeline stage VLLMTemplater completed in 0.15s
run pipeline stage %s
Running pipeline stage VLLMDeployer
Creating inference service chaiml-reward-dpo-79ab-18815-v1
Waiting for inference service chaiml-reward-dpo-79ab-18815-v1 to be ready
Failed to get response for submission chaiml-mistral-24b-2048-_2678_v3: ('http://chaiml-mistral-24b-2048-2678-v3-predictor.tenant-chaiml-guanaco.kchai-coreweave-us-east-04a.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', '')
Failed to get response for submission chaiml-mistral-24b-2048_54327_v6: ('http://chaiml-mistral-24b-2048-54327-v6-predictor.tenant-chaiml-guanaco.kchai-coreweave-us-east-04a.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', '')
HTTP Request: %s %s "%s %d %s"
Failed to get response for submission chaiml-mistral-24b-2048-_2678_v3: ('http://chaiml-mistral-24b-2048-2678-v3-predictor.tenant-chaiml-guanaco.kchai-coreweave-us-east-04a.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', '')
Failed to get response for submission chaiml-grpo-q235b-kimid_37540_v1: HTTPConnectionPool(host='chaiml-grpo-q235b-kimid-37540-v1-predictor.tenant-chaiml-guanaco.kchai-coreweave-us-east-04a.chaiverse.com', port=80): Read timed out. (read timeout=12.0)
Failed to get response for submission chaiml-mistral-24b-2048_54327_v6: ('http://chaiml-mistral-24b-2048-54327-v6-predictor.tenant-chaiml-guanaco.kchai-coreweave-us-east-04a.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', '')
Failed to get response for submission chaiml-mistral-24b-2048_15988_v1: ('http://chaiml-mistral-24b-2048-15988-v1-predictor.tenant-chaiml-guanaco.kchai-coreweave-us-east-04a.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', '')
Failed to get response for submission chaiml-mistral-24b-2048-_2678_v3: ('http://chaiml-mistral-24b-2048-2678-v3-predictor.tenant-chaiml-guanaco.kchai-coreweave-us-east-04a.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', '')
Failed to get response for submission chaiml-mistral-24b-2048-_2678_v3: ('http://chaiml-mistral-24b-2048-2678-v3-predictor.tenant-chaiml-guanaco.kchai-coreweave-us-east-04a.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', '')
HTTP Request: %s %s "%s %d %s"
HTTP Request: %s %s "%s %d %s"
Failed to get response for submission chaiml-mistral-24b-2048_54327_v6: ('http://chaiml-mistral-24b-2048-54327-v6-predictor.tenant-chaiml-guanaco.kchai-coreweave-us-east-04a.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', '')
Inference service chaiml-reward-dpo-79ab-18815-v1 ready after 973.6072442531586s
Pipeline stage VLLMDeployer completed in 974.13s
run pipeline stage %s
Running pipeline stage StressChecker
Received healthy response to inference request in 1.9858765602111816s
Received healthy response to inference request in 1.913177490234375s
Received healthy response to inference request in 1.9431025981903076s
Received healthy response to inference request in 2.071401357650757s
Received healthy response to inference request in 2.0489866733551025s
Received healthy response to inference request in 1.9553775787353516s
Received healthy response to inference request in 2.027005195617676s
Received healthy response to inference request in 1.970944881439209s
Received healthy response to inference request in 2.0593924522399902s
Received healthy response to inference request in 2.0380125045776367s
Received healthy response to inference request in 1.9316582679748535s
Received healthy response to inference request in 2.1331112384796143s
Received healthy response to inference request in 2.0394132137298584s
Received healthy response to inference request in 1.9160854816436768s
Received healthy response to inference request in 1.9392950534820557s
Received healthy response to inference request in 1.9414951801300049s
Received healthy response to inference request in 1.9559848308563232s
Received healthy response to inference request in 1.93514084815979s
Received healthy response to inference request in 2.160327434539795s
Received healthy response to inference request in 2.053464651107788s
Received healthy response to inference request in 2.224792003631592s
Received healthy response to inference request in 1.9773058891296387s
Received healthy response to inference request in 1.9413974285125732s
Received healthy response to inference request in 1.8976707458496094s
Received healthy response to inference request in 1.917022466659546s
Received healthy response to inference request in 1.959094524383545s
Received healthy response to inference request in 2.032930374145508s
Received healthy response to inference request in 2.0704779624938965s
Received healthy response to inference request in 1.96085786819458s
Received healthy response to inference request in 1.9409470558166504s
30 requests
0 failed requests
5th percentile: 1.9144860863685609
10th percentile: 1.916928768157959
20th percentile: 1.9384642124176026
30th percentile: 1.9414658546447754
40th percentile: 1.9557419300079346
50th percentile: 1.9659013748168945
60th percentile: 2.0023280143737794
70th percentile: 2.0384327173233032
80th percentile: 2.0546502113342284
90th percentile: 2.0775723457336426
95th percentile: 2.1480801463127137
99th percentile: 2.206097278594971
mean time: 1.9980583270390828
Pipeline stage StressChecker completed in 63.11s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyTriggerPipeline
run_pipeline:run_in_cloud %s
starting trigger_guanaco_pipeline args=%s
triggered trigger_guanaco_pipeline args=%s
Pipeline stage OfflineFamilyFriendlyTriggerPipeline completed in 1.40s
Shutdown handler de-registered
chaiml-reward-dpo-79ab-_18815_v1 status is now deployed due to DeploymentManager action
chaiml-reward-dpo-79ab-_18815_v1 status is now inactive due to auto deactivation removed underperforming models
chaiml-reward-dpo-79ab-_18815_v1 status is now torndown due to DeploymentManager action