developer_uid: chai_backend_admin
submission_id: chaiml-ssnew-v5-dpo-lr5_19068_v3
model_name: chaiml-ssnew-v5-dpo-lr5_19068_v3
model_group: ChaiML/ssnew-v5-dpo-lr5e
status: torndown
timestamp: 2026-02-10T03:21:33+00:00
num_battles: 11259
num_wins: 5464
celo_rating: 1317.81
family_friendly_score: 0.0
family_friendly_standard_error: 0.0
submission_type: basic
model_repo: ChaiML/ssnew-v5-dpo-lr5e6b01-lora-FP8
model_architecture: MistralForCausalLM
model_num_parameters: 24096691200.0
best_of: 8
max_input_tokens: 2048
max_output_tokens: 64
reward_model: default
display_name: chaiml-ssnew-v5-dpo-lr5_19068_v3
is_internal_developer: True
language_model: ChaiML/ssnew-v5-dpo-lr5e6b01-lora-FP8
model_size: 24B
ranking_group: single
us_pacific_date: 2026-02-06
win_ratio: 0.4853006483701927
generation_params: {'temperature': 1.0, 'top_p': 1.0, 'min_p': 0.0, 'top_k': 80, 'presence_penalty': 0.0, 'frequency_penalty': 0.0, 'stopping_words': ['####', '\n', '</s>', 'You:'], 'max_input_tokens': 2048, 'best_of': 8, 'max_output_tokens': 64}
formatter: {'memory_template': "{bot_name}'s persona: {memory}", 'prompt_template': '', 'bot_template': '{bot_name}: {message}\n', 'user_template': 'You: {message}\n', 'response_template': '####\n{bot_name}:', 'truncate_by_message': True}
Resubmit model
Shutdown handler not registered because Python interpreter is not running in the main thread
run pipeline %s
run pipeline stage %s
Running pipeline stage VLLMUploader
Starting job with name chaiml-ssnew-v5-dpo-lr5-19068-v3-uploader
Waiting for job on chaiml-ssnew-v5-dpo-lr5-19068-v3-uploader to finish
chaiml-ssnew-v5-dpo-lr5-19068-v3-uploader: Using quantization_mode: none
chaiml-ssnew-v5-dpo-lr5-19068-v3-uploader: Downloading snapshot of ChaiML/ssnew-v5-dpo-lr5e6b01-lora-FP8...
chaiml-ssnew-v5-dpo-lr5-19068-v3-uploader: 
Fetching 14 files:   0%|          | 0/14 [00:00<?, ?it/s]
Fetching 14 files:   7%|▋         | 1/14 [00:00<00:03,  3.63it/s]
Fetching 14 files:  29%|██▊       | 4/14 [00:10<00:28,  2.87s/it]
Fetching 14 files:  36%|███▌      | 5/14 [00:11<00:21,  2.38s/it]
Fetching 14 files: 100%|██████████| 14/14 [00:11<00:00,  1.19it/s]
chaiml-ssnew-v5-dpo-lr5-19068-v3-uploader: Downloaded in 11.959s
chaiml-ssnew-v5-dpo-lr5-19068-v3-uploader: Processed model ChaiML/ssnew-v5-dpo-lr5e6b01-lora-FP8 in 21.044s
chaiml-ssnew-v5-dpo-lr5-19068-v3-uploader: creating bucket guanaco-vllm-models
chaiml-ssnew-v5-dpo-lr5-19068-v3-uploader: /usr/lib/python3/dist-packages/S3/BaseUtils.py:56: SyntaxWarning: invalid escape sequence '\.'
chaiml-ssnew-v5-dpo-lr5-19068-v3-uploader:   RE_S3_DATESTRING = re.compile('\.[0-9]*(?:[Z\\-\\+]*?)')
chaiml-ssnew-v5-dpo-lr5-19068-v3-uploader: /usr/lib/python3/dist-packages/S3/BaseUtils.py:57: SyntaxWarning: invalid escape sequence '\s'
chaiml-ssnew-v5-dpo-lr5-19068-v3-uploader:   RE_XML_NAMESPACE = re.compile(b'^(<?[^>]+?>\s*|\s*)(<\w+) xmlns=[\'"](https?://[^\'"]+)[\'"]', re.MULTILINE)
chaiml-ssnew-v5-dpo-lr5-19068-v3-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:240: SyntaxWarning: invalid escape sequence '\.'
chaiml-ssnew-v5-dpo-lr5-19068-v3-uploader:   invalid = re.search("([^a-z0-9\.-])", bucket, re.UNICODE)
chaiml-ssnew-v5-dpo-lr5-19068-v3-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:244: SyntaxWarning: invalid escape sequence '\.'
chaiml-ssnew-v5-dpo-lr5-19068-v3-uploader:   invalid = re.search("([^A-Za-z0-9\._-])", bucket, re.UNICODE)
chaiml-ssnew-v5-dpo-lr5-19068-v3-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:255: SyntaxWarning: invalid escape sequence '\.'
chaiml-ssnew-v5-dpo-lr5-19068-v3-uploader:   if re.search("-\.", bucket, re.UNICODE):
chaiml-ssnew-v5-dpo-lr5-19068-v3-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:257: SyntaxWarning: invalid escape sequence '\.'
chaiml-ssnew-v5-dpo-lr5-19068-v3-uploader:   if re.search("\.\.", bucket, re.UNICODE):
chaiml-ssnew-v5-dpo-lr5-19068-v3-uploader: /usr/lib/python3/dist-packages/S3/S3Uri.py:155: SyntaxWarning: invalid escape sequence '\w'
chaiml-ssnew-v5-dpo-lr5-19068-v3-uploader:   _re = re.compile("^(\w+://)?(.*)", re.UNICODE)
chaiml-ssnew-v5-dpo-lr5-19068-v3-uploader: /usr/lib/python3/dist-packages/S3/FileLists.py:480: SyntaxWarning: invalid escape sequence '\*'
chaiml-ssnew-v5-dpo-lr5-19068-v3-uploader:   wildcard_split_result = re.split("\*|\?", uri_str, maxsplit=1)
chaiml-ssnew-v5-dpo-lr5-19068-v3-uploader: Bucket 's3://guanaco-vllm-models/' created
chaiml-ssnew-v5-dpo-lr5-19068-v3-uploader: uploading /dev/shm/model_output to s3://guanaco-vllm-models/chaiml-ssnew-v5-dpo-lr5-19068-v3
chaiml-ssnew-v5-dpo-lr5-19068-v3-uploader: cp /dev/shm/model_output/.gitattributes s3://guanaco-vllm-models/chaiml-ssnew-v5-dpo-lr5-19068-v3/.gitattributes
chaiml-ssnew-v5-dpo-lr5-19068-v3-uploader: cp /dev/shm/model_output/config.json s3://guanaco-vllm-models/chaiml-ssnew-v5-dpo-lr5-19068-v3/config.json
chaiml-ssnew-v5-dpo-lr5-19068-v3-uploader: cp /dev/shm/model_output/generation_config.json s3://guanaco-vllm-models/chaiml-ssnew-v5-dpo-lr5-19068-v3/generation_config.json
chaiml-ssnew-v5-dpo-lr5-19068-v3-uploader: cp /dev/shm/model_output/recipe.yaml s3://guanaco-vllm-models/chaiml-ssnew-v5-dpo-lr5-19068-v3/recipe.yaml
chaiml-ssnew-v5-dpo-lr5-19068-v3-uploader: cp /dev/shm/model_output/special_tokens_map.json s3://guanaco-vllm-models/chaiml-ssnew-v5-dpo-lr5-19068-v3/special_tokens_map.json
chaiml-ssnew-v5-dpo-lr5-19068-v3-uploader: cp /dev/shm/model_output/model.safetensors.index.json s3://guanaco-vllm-models/chaiml-ssnew-v5-dpo-lr5-19068-v3/model.safetensors.index.json
chaiml-ssnew-v5-dpo-lr5-19068-v3-uploader: cp /dev/shm/model_output/tokenizer_config.json s3://guanaco-vllm-models/chaiml-ssnew-v5-dpo-lr5-19068-v3/tokenizer_config.json
chaiml-ssnew-v5-dpo-lr5-19068-v3-uploader: cp /dev/shm/model_output/tokenizer.json s3://guanaco-vllm-models/chaiml-ssnew-v5-dpo-lr5-19068-v3/tokenizer.json
admin requested tearing down of chaiml-muster-v0b-lr5e6_21126_v2
Shutdown handler not registered because Python interpreter is not running in the main thread
run pipeline %s
run pipeline stage %s
Running pipeline stage VLLMDeleter
Pipeline stage %s skipped, reason=%s
Pipeline stage VLLMDeleter completed in 1.03s
run pipeline stage %s
Running pipeline stage VLLMModelDeleter
Pipeline stage %s skipped, reason=%s
Pipeline stage VLLMModelDeleter completed in 0.45s
Shutdown handler de-registered
chaiml-muster-v0b-lr5e6_21126_v2 status is now torndown due to DeploymentManager action
chaiml-ssnew-v5-dpo-lr5-19068-v3-uploader: cp /dev/shm/model_output/model-00006-of-00006.safetensors s3://guanaco-vllm-models/chaiml-ssnew-v5-dpo-lr5-19068-v3/model-00006-of-00006.safetensors
chaiml-ssnew-v5-dpo-lr5-19068-v3-uploader: cp /dev/shm/model_output/model-00005-of-00006.safetensors s3://guanaco-vllm-models/chaiml-ssnew-v5-dpo-lr5-19068-v3/model-00005-of-00006.safetensors
chaiml-ssnew-v5-dpo-lr5-19068-v3-uploader: cp /dev/shm/model_output/model-00001-of-00006.safetensors s3://guanaco-vllm-models/chaiml-ssnew-v5-dpo-lr5-19068-v3/model-00001-of-00006.safetensors
chaiml-ssnew-v5-dpo-lr5-19068-v3-uploader: cp /dev/shm/model_output/model-00002-of-00006.safetensors s3://guanaco-vllm-models/chaiml-ssnew-v5-dpo-lr5-19068-v3/model-00002-of-00006.safetensors
chaiml-ssnew-v5-dpo-lr5-19068-v3-uploader: cp /dev/shm/model_output/model-00004-of-00006.safetensors s3://guanaco-vllm-models/chaiml-ssnew-v5-dpo-lr5-19068-v3/model-00004-of-00006.safetensors
chaiml-ssnew-v5-dpo-lr5-19068-v3-uploader: cp /dev/shm/model_output/model-00003-of-00006.safetensors s3://guanaco-vllm-models/chaiml-ssnew-v5-dpo-lr5-19068-v3/model-00003-of-00006.safetensors
Job chaiml-ssnew-v5-dpo-lr5-19068-v3-uploader completed after 276.97s with status: succeeded
Stopping job with name chaiml-ssnew-v5-dpo-lr5-19068-v3-uploader
Pipeline stage VLLMUploader completed in 277.43s
run pipeline stage %s
Running pipeline stage VLLMTemplater
Pipeline stage VLLMTemplater completed in 0.14s
run pipeline stage %s
Running pipeline stage VLLMDeployer
Creating inference service chaiml-ssnew-v5-dpo-lr5-19068-v3
Waiting for inference service chaiml-ssnew-v5-dpo-lr5-19068-v3 to be ready
HTTP Request: %s %s "%s %d %s"
Inference service chaiml-ssnew-v5-dpo-lr5-19068-v3 ready after 190.88146591186523s
Pipeline stage VLLMDeployer completed in 191.35s
run pipeline stage %s
Running pipeline stage StressChecker
Received healthy response to inference request in 1.5303442478179932s
Received healthy response to inference request in 1.471311092376709s
Received healthy response to inference request in 1.5036590099334717s
Received healthy response to inference request in 1.4082767963409424s
Received healthy response to inference request in 1.3328821659088135s
Received healthy response to inference request in 1.7340655326843262s
Received healthy response to inference request in 1.3917217254638672s
Received healthy response to inference request in 1.4978370666503906s
Received healthy response to inference request in 1.3502063751220703s
Received healthy response to inference request in 1.335036039352417s
Received healthy response to inference request in 1.3548498153686523s
Received healthy response to inference request in 1.3796849250793457s
Received healthy response to inference request in 1.316861867904663s
Received healthy response to inference request in 1.4980244636535645s
Received healthy response to inference request in 1.6336417198181152s
Received healthy response to inference request in 1.5930802822113037s
Received healthy response to inference request in 1.4337458610534668s
Received healthy response to inference request in 1.4762794971466064s
Received healthy response to inference request in 1.6403019428253174s
Received healthy response to inference request in 1.5295913219451904s
Received healthy response to inference request in 1.4655892848968506s
Received healthy response to inference request in 1.6448321342468262s
Received healthy response to inference request in 1.4673840999603271s
Received healthy response to inference request in 1.4452121257781982s
Received healthy response to inference request in 1.4687163829803467s
Received healthy response to inference request in 1.379103660583496s
Received healthy response to inference request in 1.464336633682251s
Received healthy response to inference request in 1.4801061153411865s
Received healthy response to inference request in 1.3377797603607178s
Received healthy response to inference request in 1.6958894729614258s
30 requests
0 failed requests
5th percentile: 1.333851408958435
10th percentile: 1.3375053882598877
20th percentile: 1.3742528915405274
30th percentile: 1.40331027507782
40th percentile: 1.45668683052063
50th percentile: 1.468050241470337
60th percentile: 1.4778101444244385
70th percentile: 1.4997148275375367
80th percentile: 1.5428914546966555
90th percentile: 1.6407549619674682
95th percentile: 1.6729136705398557
99th percentile: 1.722994475364685
mean time: 1.4753450473149619
Pipeline stage StressChecker completed in 47.19s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyTriggerPipeline
run_pipeline:run_in_cloud %s
starting trigger_guanaco_pipeline args=%s
triggered trigger_guanaco_pipeline args=%s
Pipeline stage OfflineFamilyFriendlyTriggerPipeline completed in 0.61s
Shutdown handler de-registered
chaiml-ssnew-v5-dpo-lr5_19068_v3 status is now deployed due to DeploymentManager action
chaiml-ssnew-v5-dpo-lr5_19068_v3 status is now inactive due to auto deactivation removed underperforming models
chaiml-ssnew-v5-dpo-lr5_19068_v3 status is now torndown due to DeploymentManager action