chaiml-ssnew-v5-dpo-lr_19068

developer_uid: Meliodia
submission_id: chaiml-ssnew-v5-dpo-lr_19068_v15
model_name: chaiml-ssnew-v5-dpo-lr5_19068_v5
model_group: ChaiML/ssnew-v5-dpo-lr5e
status: torndown
timestamp: 2026-03-28T14:18:48+00:00
num_battles: 11995
num_wins: 5817
celo_rating: 8469.92
family_friendly_score: 0.0
family_friendly_standard_error: 0.0
submission_type: basic
model_repo: ChaiML/ssnew-v5-dpo-lr5e6b01-lora-FP8
model_architecture: MistralForCausalLM
model_num_parameters: 24096691200.0
best_of: 8
max_input_tokens: 2048
max_output_tokens: 64
reward_model: default
display_name: chaiml-ssnew-v5-dpo-lr5_19068_v5
is_internal_developer: True
language_model: ChaiML/ssnew-v5-dpo-lr5e6b01-lora-FP8
model_size: 24B
ranking_group: single
us_pacific_date: 2026-03-25
win_ratio: 0.48495206335973323
generation_params: {'temperature': 1.0, 'top_p': 1.0, 'min_p': 0.0, 'top_k': 80, 'presence_penalty': 0.0, 'frequency_penalty': 0.0, 'stopping_words': ['\n'], 'max_input_tokens': 2048, 'best_of': 8, 'max_output_tokens': 64}
formatter: {'memory_template': '<|im_start|>system\n{memory}<|im_end|>\n', 'prompt_template': '', 'bot_template': '{bot_name}: {message}\n', 'user_template': '<|im_start|>user\n{user_name}: {message}<|im_end|>\n', 'response_template': '<|im_start|>assistant\n{bot_name}:', 'truncate_by_message': True}
Resubmit model
Shutdown handler not registered because Python interpreter is not running in the main thread
run pipeline %s
run pipeline stage %s
Running pipeline stage VLLMUploader
Starting job with name chaiml-ssnew-v5-dpo-lr-19068-v15-uploader
Waiting for job on chaiml-ssnew-v5-dpo-lr-19068-v15-uploader to finish
chaiml-ssnew-v5-dpo-lr-19068-v15-uploader: Using quantization_mode: fp8
chaiml-ssnew-v5-dpo-lr-19068-v15-uploader: Repo ChaiML/ssnew-v5-dpo-lr5e6b01-lora-FP8 already ends in FP8. Skipping...
chaiml-ssnew-v5-dpo-lr-19068-v15-uploader: Checking if ChaiML/ssnew-v5-dpo-lr5e6b01-lora-FP8 already exists in ChaiML
chaiml-ssnew-v5-dpo-lr-19068-v15-uploader: Model already exists. Downloading to /dev/shm/model_output...
chaiml-ssnew-v5-dpo-lr-19068-v15-uploader: Downloading snapshot of ChaiML/ssnew-v5-dpo-lr5e6b01-lora-FP8...
chaiml-ssnew-v5-dpo-lr-19068-v15-uploader: Downloaded in 11.050s
chaiml-ssnew-v5-dpo-lr-19068-v15-uploader: Processed model ChaiML/ssnew-v5-dpo-lr5e6b01-lora-FP8 in 14.593s
chaiml-ssnew-v5-dpo-lr-19068-v15-uploader: creating bucket guanaco-vllm-models
chaiml-ssnew-v5-dpo-lr-19068-v15-uploader: /usr/lib/python3/dist-packages/S3/BaseUtils.py:56: SyntaxWarning: invalid escape sequence '\.'
chaiml-ssnew-v5-dpo-lr-19068-v15-uploader:   RE_S3_DATESTRING = re.compile('\.[0-9]*(?:[Z\\-\\+]*?)')
chaiml-ssnew-v5-dpo-lr-19068-v15-uploader: /usr/lib/python3/dist-packages/S3/BaseUtils.py:57: SyntaxWarning: invalid escape sequence '\s'
chaiml-ssnew-v5-dpo-lr-19068-v15-uploader:   RE_XML_NAMESPACE = re.compile(b'^(<?[^>]+?>\s*|\s*)(<\w+) xmlns=[\'"](https?://[^\'"]+)[\'"]', re.MULTILINE)
chaiml-ssnew-v5-dpo-lr-19068-v15-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:240: SyntaxWarning: invalid escape sequence '\.'
chaiml-ssnew-v5-dpo-lr-19068-v15-uploader:   invalid = re.search("([^a-z0-9\.-])", bucket, re.UNICODE)
chaiml-ssnew-v5-dpo-lr-19068-v15-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:244: SyntaxWarning: invalid escape sequence '\.'
chaiml-ssnew-v5-dpo-lr-19068-v15-uploader:   invalid = re.search("([^A-Za-z0-9\._-])", bucket, re.UNICODE)
chaiml-ssnew-v5-dpo-lr-19068-v15-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:255: SyntaxWarning: invalid escape sequence '\.'
chaiml-ssnew-v5-dpo-lr-19068-v15-uploader:   if re.search("-\.", bucket, re.UNICODE):
chaiml-ssnew-v5-dpo-lr-19068-v15-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:257: SyntaxWarning: invalid escape sequence '\.'
chaiml-ssnew-v5-dpo-lr-19068-v15-uploader:   if re.search("\.\.", bucket, re.UNICODE):
chaiml-ssnew-v5-dpo-lr-19068-v15-uploader: /usr/lib/python3/dist-packages/S3/S3Uri.py:155: SyntaxWarning: invalid escape sequence '\w'
chaiml-ssnew-v5-dpo-lr-19068-v15-uploader:   _re = re.compile("^(\w+://)?(.*)", re.UNICODE)
chaiml-ssnew-v5-dpo-lr-19068-v15-uploader: /usr/lib/python3/dist-packages/S3/FileLists.py:480: SyntaxWarning: invalid escape sequence '\*'
chaiml-ssnew-v5-dpo-lr-19068-v15-uploader:   wildcard_split_result = re.split("\*|\?", uri_str, maxsplit=1)
chaiml-ssnew-v5-dpo-lr-19068-v15-uploader: Bucket 's3://guanaco-vllm-models/' created
chaiml-ssnew-v5-dpo-lr-19068-v15-uploader: uploading /dev/shm/model_output to s3://guanaco-vllm-models/chaiml-ssnew-v5-dpo-lr-19068-v15/default
chaiml-ssnew-v5-dpo-lr-19068-v15-uploader: cp /dev/shm/model_output/special_tokens_map.json s3://guanaco-vllm-models/chaiml-ssnew-v5-dpo-lr-19068-v15/default/special_tokens_map.json
chaiml-ssnew-v5-dpo-lr-19068-v15-uploader: cp /dev/shm/model_output/recipe.yaml s3://guanaco-vllm-models/chaiml-ssnew-v5-dpo-lr-19068-v15/default/recipe.yaml
chaiml-ssnew-v5-dpo-lr-19068-v15-uploader: cp /dev/shm/model_output/tokenizer_config.json s3://guanaco-vllm-models/chaiml-ssnew-v5-dpo-lr-19068-v15/default/tokenizer_config.json
chaiml-ssnew-v5-dpo-lr-19068-v15-uploader: cp /dev/shm/model_output/README.md s3://guanaco-vllm-models/chaiml-ssnew-v5-dpo-lr-19068-v15/default/README.md
chaiml-ssnew-v5-dpo-lr-19068-v15-uploader: cp /dev/shm/model_output/config.json s3://guanaco-vllm-models/chaiml-ssnew-v5-dpo-lr-19068-v15/default/config.json
chaiml-ssnew-v5-dpo-lr-19068-v15-uploader: cp /dev/shm/model_output/generation_config.json s3://guanaco-vllm-models/chaiml-ssnew-v5-dpo-lr-19068-v15/default/generation_config.json
chaiml-ssnew-v5-dpo-lr-19068-v15-uploader: cp /dev/shm/model_output/model.safetensors.index.json s3://guanaco-vllm-models/chaiml-ssnew-v5-dpo-lr-19068-v15/default/model.safetensors.index.json
chaiml-ssnew-v5-dpo-lr-19068-v15-uploader: cp /dev/shm/model_output/.gitattributes s3://guanaco-vllm-models/chaiml-ssnew-v5-dpo-lr-19068-v15/default/.gitattributes
chaiml-ssnew-v5-dpo-lr-19068-v15-uploader: cp /dev/shm/model_output/tokenizer.json s3://guanaco-vllm-models/chaiml-ssnew-v5-dpo-lr-19068-v15/default/tokenizer.json
chaiml-ssnew-v5-dpo-lr-19068-v15-uploader: cp /dev/shm/model_output/model-00006-of-00006.safetensors s3://guanaco-vllm-models/chaiml-ssnew-v5-dpo-lr-19068-v15/default/model-00006-of-00006.safetensors
chaiml-ssnew-v5-dpo-lr-19068-v15-uploader: cp /dev/shm/model_output/model-00005-of-00006.safetensors s3://guanaco-vllm-models/chaiml-ssnew-v5-dpo-lr-19068-v15/default/model-00005-of-00006.safetensors
chaiml-ssnew-v5-dpo-lr-19068-v15-uploader: cp /dev/shm/model_output/model-00002-of-00006.safetensors s3://guanaco-vllm-models/chaiml-ssnew-v5-dpo-lr-19068-v15/default/model-00002-of-00006.safetensors
2026-03-25T12:26:49.651027+00:00 monitor updated for chaiml-ssnew-v5-dpo-lr_19068_v15
Job chaiml-ssnew-v5-dpo-lr-19068-v15-uploader completed after 62.43s with status: succeeded
Stopping job with name chaiml-ssnew-v5-dpo-lr-19068-v15-uploader
Pipeline stage VLLMUploader completed in 62.97s
run pipeline stage %s
Running pipeline stage VLLMTemplater
Pipeline stage VLLMTemplater completed in 3.60s
run pipeline stage %s
Running pipeline stage VLLMDeployer
Creating inference service chaiml-ssnew-v5-dpo-lr-19068-v15
Waiting for inference service chaiml-ssnew-v5-dpo-lr-19068-v15 to be ready
2026-03-25T12:27:49.783036+00:00 monitor updated for chaiml-ssnew-v5-dpo-lr_19068_v15
Shutdown handler not registered because Python interpreter is not running in the main thread
run pipeline %s
run pipeline stage %s
Running pipeline stage VLLMUploader
Starting job with name chaiml-csfs-v3-3-dpo-lr5-937-v10-uploader
Waiting for job on chaiml-csfs-v3-3-dpo-lr5-937-v10-uploader to finish
2026-03-25T12:28:49.880326+00:00 monitor updated for chaiml-ssnew-v5-dpo-lr_19068_v15
chaiml-csfs-v3-3-dpo-lr5-937-v10-uploader: Using quantization_mode: fp8
chaiml-csfs-v3-3-dpo-lr5-937-v10-uploader: Repo ChaiML/csfs-v3-3-dpo-lr5e6b01-lora-FP8 already ends in FP8. Skipping...
chaiml-csfs-v3-3-dpo-lr5-937-v10-uploader: Checking if ChaiML/csfs-v3-3-dpo-lr5e6b01-lora-FP8 already exists in ChaiML
chaiml-csfs-v3-3-dpo-lr5-937-v10-uploader: Model already exists. Downloading to /dev/shm/model_output...
chaiml-csfs-v3-3-dpo-lr5-937-v10-uploader: Downloading snapshot of ChaiML/csfs-v3-3-dpo-lr5e6b01-lora-FP8...
chaiml-csfs-v3-3-dpo-lr5-937-v10-uploader: Downloaded in 10.686s
chaiml-csfs-v3-3-dpo-lr5-937-v10-uploader: Processed model ChaiML/csfs-v3-3-dpo-lr5e6b01-lora-FP8 in 14.308s
chaiml-csfs-v3-3-dpo-lr5-937-v10-uploader: creating bucket guanaco-vllm-models
chaiml-csfs-v3-3-dpo-lr5-937-v10-uploader: /usr/lib/python3/dist-packages/S3/BaseUtils.py:56: SyntaxWarning: invalid escape sequence '\.'
chaiml-csfs-v3-3-dpo-lr5-937-v10-uploader:   RE_S3_DATESTRING = re.compile('\.[0-9]*(?:[Z\\-\\+]*?)')
chaiml-csfs-v3-3-dpo-lr5-937-v10-uploader: /usr/lib/python3/dist-packages/S3/BaseUtils.py:57: SyntaxWarning: invalid escape sequence '\s'
chaiml-csfs-v3-3-dpo-lr5-937-v10-uploader:   RE_XML_NAMESPACE = re.compile(b'^(<?[^>]+?>\s*|\s*)(<\w+) xmlns=[\'"](https?://[^\'"]+)[\'"]', re.MULTILINE)
chaiml-csfs-v3-3-dpo-lr5-937-v10-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:240: SyntaxWarning: invalid escape sequence '\.'
chaiml-csfs-v3-3-dpo-lr5-937-v10-uploader:   invalid = re.search("([^a-z0-9\.-])", bucket, re.UNICODE)
chaiml-csfs-v3-3-dpo-lr5-937-v10-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:244: SyntaxWarning: invalid escape sequence '\.'
chaiml-csfs-v3-3-dpo-lr5-937-v10-uploader:   invalid = re.search("([^A-Za-z0-9\._-])", bucket, re.UNICODE)
chaiml-csfs-v3-3-dpo-lr5-937-v10-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:255: SyntaxWarning: invalid escape sequence '\.'
chaiml-csfs-v3-3-dpo-lr5-937-v10-uploader:   if re.search("-\.", bucket, re.UNICODE):
chaiml-csfs-v3-3-dpo-lr5-937-v10-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:257: SyntaxWarning: invalid escape sequence '\.'
chaiml-csfs-v3-3-dpo-lr5-937-v10-uploader:   if re.search("\.\.", bucket, re.UNICODE):
chaiml-csfs-v3-3-dpo-lr5-937-v10-uploader: /usr/lib/python3/dist-packages/S3/S3Uri.py:155: SyntaxWarning: invalid escape sequence '\w'
chaiml-csfs-v3-3-dpo-lr5-937-v10-uploader:   _re = re.compile("^(\w+://)?(.*)", re.UNICODE)
chaiml-csfs-v3-3-dpo-lr5-937-v10-uploader: /usr/lib/python3/dist-packages/S3/FileLists.py:480: SyntaxWarning: invalid escape sequence '\*'
chaiml-csfs-v3-3-dpo-lr5-937-v10-uploader:   wildcard_split_result = re.split("\*|\?", uri_str, maxsplit=1)
chaiml-csfs-v3-3-dpo-lr5-937-v10-uploader: Bucket 's3://guanaco-vllm-models/' created
chaiml-csfs-v3-3-dpo-lr5-937-v10-uploader: uploading /dev/shm/model_output to s3://guanaco-vllm-models/chaiml-csfs-v3-3-dpo-lr5-937-v10/default
chaiml-csfs-v3-3-dpo-lr5-937-v10-uploader: cp /dev/shm/model_output/model.safetensors.index.json s3://guanaco-vllm-models/chaiml-csfs-v3-3-dpo-lr5-937-v10/default/model.safetensors.index.json
chaiml-csfs-v3-3-dpo-lr5-937-v10-uploader: cp /dev/shm/model_output/README.md s3://guanaco-vllm-models/chaiml-csfs-v3-3-dpo-lr5-937-v10/default/README.md
chaiml-csfs-v3-3-dpo-lr5-937-v10-uploader: cp /dev/shm/model_output/.gitattributes s3://guanaco-vllm-models/chaiml-csfs-v3-3-dpo-lr5-937-v10/default/.gitattributes
chaiml-csfs-v3-3-dpo-lr5-937-v10-uploader: cp /dev/shm/model_output/special_tokens_map.json s3://guanaco-vllm-models/chaiml-csfs-v3-3-dpo-lr5-937-v10/default/special_tokens_map.json
chaiml-csfs-v3-3-dpo-lr5-937-v10-uploader: cp /dev/shm/model_output/config.json s3://guanaco-vllm-models/chaiml-csfs-v3-3-dpo-lr5-937-v10/default/config.json
chaiml-csfs-v3-3-dpo-lr5-937-v10-uploader: cp /dev/shm/model_output/tokenizer_config.json s3://guanaco-vllm-models/chaiml-csfs-v3-3-dpo-lr5-937-v10/default/tokenizer_config.json
chaiml-csfs-v3-3-dpo-lr5-937-v10-uploader: cp /dev/shm/model_output/generation_config.json s3://guanaco-vllm-models/chaiml-csfs-v3-3-dpo-lr5-937-v10/default/generation_config.json
chaiml-csfs-v3-3-dpo-lr5-937-v10-uploader: cp /dev/shm/model_output/recipe.yaml s3://guanaco-vllm-models/chaiml-csfs-v3-3-dpo-lr5-937-v10/default/recipe.yaml
chaiml-csfs-v3-3-dpo-lr5-937-v10-uploader: cp /dev/shm/model_output/tokenizer.json s3://guanaco-vllm-models/chaiml-csfs-v3-3-dpo-lr5-937-v10/default/tokenizer.json
chaiml-csfs-v3-3-dpo-lr5-937-v10-uploader: cp /dev/shm/model_output/model-00006-of-00006.safetensors s3://guanaco-vllm-models/chaiml-csfs-v3-3-dpo-lr5-937-v10/default/model-00006-of-00006.safetensors
chaiml-csfs-v3-3-dpo-lr5-937-v10-uploader: cp /dev/shm/model_output/model-00005-of-00006.safetensors s3://guanaco-vllm-models/chaiml-csfs-v3-3-dpo-lr5-937-v10/default/model-00005-of-00006.safetensors
chaiml-csfs-v3-3-dpo-lr5-937-v10-uploader: cp /dev/shm/model_output/model-00003-of-00006.safetensors s3://guanaco-vllm-models/chaiml-csfs-v3-3-dpo-lr5-937-v10/default/model-00003-of-00006.safetensors
chaiml-csfs-v3-3-dpo-lr5-937-v10-uploader: cp /dev/shm/model_output/model-00002-of-00006.safetensors s3://guanaco-vllm-models/chaiml-csfs-v3-3-dpo-lr5-937-v10/default/model-00002-of-00006.safetensors
chaiml-csfs-v3-3-dpo-lr5-937-v10-uploader: cp /dev/shm/model_output/model-00004-of-00006.safetensors s3://guanaco-vllm-models/chaiml-csfs-v3-3-dpo-lr5-937-v10/default/model-00004-of-00006.safetensors
chaiml-csfs-v3-3-dpo-lr5-937-v10-uploader: cp /dev/shm/model_output/model-00001-of-00006.safetensors s3://guanaco-vllm-models/chaiml-csfs-v3-3-dpo-lr5-937-v10/default/model-00001-of-00006.safetensors
Job chaiml-csfs-v3-3-dpo-lr5-937-v10-uploader completed after 44.94s with status: succeeded
Stopping job with name chaiml-csfs-v3-3-dpo-lr5-937-v10-uploader
Pipeline stage VLLMUploader completed in 45.63s
run pipeline stage %s
Running pipeline stage VLLMTemplater
Pipeline stage VLLMTemplater completed in 0.45s
run pipeline stage %s
Running pipeline stage VLLMDeployer
Creating inference service chaiml-csfs-v3-3-dpo-lr5-937-v10
Waiting for inference service chaiml-csfs-v3-3-dpo-lr5-937-v10 to be ready
2026-03-25T12:29:34.408331+00:00 monitor updated for chaiml-csfs-v3-3-dpo-lr5_937_v10
Inference service chaiml-ssnew-v5-dpo-lr-19068-v15 ready after 160.3820993900299s
Pipeline stage VLLMDeployer completed in 161.03s
run pipeline stage %s
Running pipeline stage StressChecker
Received healthy response to inference request in 4.398024559020996s
Received healthy response to inference request in 4.5714874267578125s
2026-03-25T12:29:50.301336+00:00 monitor updated for chaiml-ssnew-v5-dpo-lr_19068_v15
Received healthy response to inference request in 4.192166566848755s
Received healthy response to inference request in 2.6582190990448s
Received healthy response to inference request in 2.703934907913208s
Received healthy response to inference request in 2.6619629859924316s
Received healthy response to inference request in 2.724956750869751s
Received healthy response to inference request in 2.737095594406128s
Received healthy response to inference request in 2.795311450958252s
Received healthy response to inference request in 4.2797462940216064s
Received healthy response to inference request in 2.7763922214508057s
Received healthy response to inference request in 2.6876420974731445s
Received healthy response to inference request in 2.6869559288024902s
Received healthy response to inference request in 2.669828414916992s
Received healthy response to inference request in 2.7208542823791504s
Received healthy response to inference request in 2.6431548595428467s
Received healthy response to inference request in 2.723360300064087s
2026-03-25T12:30:34.559458+00:00 monitor updated for chaiml-csfs-v3-3-dpo-lr5_937_v10
Received healthy response to inference request in 2.6743898391723633s
Received healthy response to inference request in 2.6841237545013428s
Received healthy response to inference request in 2.6775670051574707s
Received healthy response to inference request in 2.752131700515747s
Received healthy response to inference request in 2.684474229812622s
Received healthy response to inference request in 2.677527904510498s
2026-03-25T12:30:50.468886+00:00 monitor updated for chaiml-ssnew-v5-dpo-lr_19068_v15
Received healthy response to inference request in 4.31263542175293s
Received healthy response to inference request in 2.77150297164917s
Received healthy response to inference request in 2.6898088455200195s
Received healthy response to inference request in 2.6948516368865967s
Received healthy response to inference request in 2.757887840270996s
Received healthy response to inference request in 2.876107931137085s
Received healthy response to inference request in 2.7837047576904297s
30 requests
0 failed requests
5th percentile: 2.659903848171234
10th percentile: 2.669041872024536
20th percentile: 2.677559185028076
30th percentile: 2.68621141910553
40th percentile: 2.6928345203399657
50th percentile: 2.7221072912216187
60th percentile: 2.7431100368499757
70th percentile: 2.7729697465896606
80th percentile: 2.811470746994019
90th percentile: 4.283035206794739
95th percentile: 4.359599447250366
99th percentile: 4.521183195114136
mean time: 2.988926919301351
Pipeline stage StressChecker completed in 95.25s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyTriggerPipeline
run_pipeline:run_in_cloud %s
starting trigger_guanaco_pipeline args=%s
triggered trigger_guanaco_pipeline args=%s
Pipeline stage OfflineFamilyFriendlyTriggerPipeline completed in 1.43s
Shutdown handler de-registered
chaiml-ssnew-v5-dpo-lr_19068_v15 status is now deployed due to DeploymentManager action
chaiml-ssnew-v5-dpo-lr_19068_v15 status is now inactive due to auto deactivation removed underperforming models
chaiml-ssnew-v5-dpo-lr_19068_v15 status is now torndown due to DeploymentManager action