developer_uid: chai_backend_admin
submission_id: chaiml-csfs-v3-3-dpo-lr_86819_v3
model_name: chaiml-csfs-v3-3-dpo-lr_86819_v3
model_group: ChaiML/csfs-v3-3-dpo-lr5
status: torndown
timestamp: 2026-02-10T00:48:21+00:00
num_battles: 10559
num_wins: 5078
celo_rating: 1299.35
family_friendly_score: 0.0
family_friendly_standard_error: 0.0
submission_type: basic
model_repo: ChaiML/csfs-v3-3-dpo-lr5e6b01-lora-W4A16-G128-AutoRound
model_architecture: MistralForCausalLM
model_num_parameters: 24096691200.0
best_of: 8
max_input_tokens: 2048
max_output_tokens: 64
reward_model: default
display_name: chaiml-csfs-v3-3-dpo-lr_86819_v3
is_internal_developer: True
language_model: ChaiML/csfs-v3-3-dpo-lr5e6b01-lora-W4A16-G128-AutoRound
model_size: 24B
ranking_group: single
us_pacific_date: 2026-02-06
win_ratio: 0.4809167534804432
generation_params: {'temperature': 1.0, 'top_p': 1.0, 'min_p': 0.0, 'top_k': 80, 'presence_penalty': 0.0, 'frequency_penalty': 0.0, 'stopping_words': ['</s>', 'You:', '####', '\n'], 'max_input_tokens': 2048, 'best_of': 8, 'max_output_tokens': 64}
formatter: {'memory_template': "{bot_name}'s persona: {memory}", 'prompt_template': '', 'bot_template': '{bot_name}: {message}\n', 'user_template': 'You: {message}\n', 'response_template': '####\n{bot_name}:', 'truncate_by_message': True}
Resubmit model
Shutdown handler not registered because Python interpreter is not running in the main thread
run pipeline %s
run pipeline stage %s
Running pipeline stage VLLMUploader
Starting job with name chaiml-csfs-v3-3-dpo-lr-86819-v3-uploader
Waiting for job on chaiml-csfs-v3-3-dpo-lr-86819-v3-uploader to finish
chaiml-csfs-v3-3-dpo-lr-86819-v3-uploader: Using quantization_mode: none
chaiml-csfs-v3-3-dpo-lr-86819-v3-uploader: Downloading snapshot of ChaiML/csfs-v3-3-dpo-lr5e6b01-lora-W4A16-G128-AutoRound...
chaiml-csfs-v3-3-dpo-lr-86819-v3-uploader: 
Fetching 12 files:   0%|          | 0/12 [00:00<?, ?it/s]
Fetching 12 files:   8%|▊         | 1/12 [00:00<00:03,  3.37it/s]
Fetching 12 files:  42%|████▏     | 5/12 [00:08<00:12,  1.80s/it]
Fetching 12 files: 100%|██████████| 12/12 [00:08<00:00,  1.41it/s]
chaiml-csfs-v3-3-dpo-lr-86819-v3-uploader: Downloaded in 8.649s
chaiml-csfs-v3-3-dpo-lr-86819-v3-uploader: Processed model ChaiML/csfs-v3-3-dpo-lr5e6b01-lora-W4A16-G128-AutoRound in 14.071s
chaiml-csfs-v3-3-dpo-lr-86819-v3-uploader: creating bucket guanaco-vllm-models
chaiml-csfs-v3-3-dpo-lr-86819-v3-uploader: /usr/lib/python3/dist-packages/S3/BaseUtils.py:56: SyntaxWarning: invalid escape sequence '\.'
chaiml-csfs-v3-3-dpo-lr-86819-v3-uploader:   RE_S3_DATESTRING = re.compile('\.[0-9]*(?:[Z\\-\\+]*?)')
chaiml-csfs-v3-3-dpo-lr-86819-v3-uploader: /usr/lib/python3/dist-packages/S3/BaseUtils.py:57: SyntaxWarning: invalid escape sequence '\s'
chaiml-csfs-v3-3-dpo-lr-86819-v3-uploader:   RE_XML_NAMESPACE = re.compile(b'^(<?[^>]+?>\s*|\s*)(<\w+) xmlns=[\'"](https?://[^\'"]+)[\'"]', re.MULTILINE)
chaiml-csfs-v3-3-dpo-lr-86819-v3-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:240: SyntaxWarning: invalid escape sequence '\.'
chaiml-csfs-v3-3-dpo-lr-86819-v3-uploader:   invalid = re.search("([^a-z0-9\.-])", bucket, re.UNICODE)
chaiml-csfs-v3-3-dpo-lr-86819-v3-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:244: SyntaxWarning: invalid escape sequence '\.'
chaiml-csfs-v3-3-dpo-lr-86819-v3-uploader:   invalid = re.search("([^A-Za-z0-9\._-])", bucket, re.UNICODE)
chaiml-csfs-v3-3-dpo-lr-86819-v3-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:255: SyntaxWarning: invalid escape sequence '\.'
chaiml-csfs-v3-3-dpo-lr-86819-v3-uploader:   if re.search("-\.", bucket, re.UNICODE):
chaiml-csfs-v3-3-dpo-lr-86819-v3-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:257: SyntaxWarning: invalid escape sequence '\.'
chaiml-csfs-v3-3-dpo-lr-86819-v3-uploader:   if re.search("\.\.", bucket, re.UNICODE):
chaiml-csfs-v3-3-dpo-lr-86819-v3-uploader: /usr/lib/python3/dist-packages/S3/S3Uri.py:155: SyntaxWarning: invalid escape sequence '\w'
chaiml-csfs-v3-3-dpo-lr-86819-v3-uploader:   _re = re.compile("^(\w+://)?(.*)", re.UNICODE)
chaiml-csfs-v3-3-dpo-lr-86819-v3-uploader: /usr/lib/python3/dist-packages/S3/FileLists.py:480: SyntaxWarning: invalid escape sequence '\*'
chaiml-csfs-v3-3-dpo-lr-86819-v3-uploader:   wildcard_split_result = re.split("\*|\?", uri_str, maxsplit=1)
chaiml-csfs-v3-3-dpo-lr-86819-v3-uploader: Bucket 's3://guanaco-vllm-models/' created
chaiml-csfs-v3-3-dpo-lr-86819-v3-uploader: uploading /dev/shm/model_output to s3://guanaco-vllm-models/chaiml-csfs-v3-3-dpo-lr-86819-v3
chaiml-csfs-v3-3-dpo-lr-86819-v3-uploader: cp /dev/shm/model_output/.gitattributes s3://guanaco-vllm-models/chaiml-csfs-v3-3-dpo-lr-86819-v3/.gitattributes
chaiml-csfs-v3-3-dpo-lr-86819-v3-uploader: cp /dev/shm/model_output/generation_config.json s3://guanaco-vllm-models/chaiml-csfs-v3-3-dpo-lr-86819-v3/generation_config.json
chaiml-csfs-v3-3-dpo-lr-86819-v3-uploader: cp /dev/shm/model_output/config.json s3://guanaco-vllm-models/chaiml-csfs-v3-3-dpo-lr-86819-v3/config.json
chaiml-csfs-v3-3-dpo-lr-86819-v3-uploader: cp /dev/shm/model_output/recipe.yaml s3://guanaco-vllm-models/chaiml-csfs-v3-3-dpo-lr-86819-v3/recipe.yaml
chaiml-csfs-v3-3-dpo-lr-86819-v3-uploader: cp /dev/shm/model_output/special_tokens_map.json s3://guanaco-vllm-models/chaiml-csfs-v3-3-dpo-lr-86819-v3/special_tokens_map.json
chaiml-csfs-v3-3-dpo-lr-86819-v3-uploader: cp /dev/shm/model_output/tokenizer_config.json s3://guanaco-vllm-models/chaiml-csfs-v3-3-dpo-lr-86819-v3/tokenizer_config.json
chaiml-csfs-v3-3-dpo-lr-86819-v3-uploader: cp /dev/shm/model_output/model.safetensors.index.json s3://guanaco-vllm-models/chaiml-csfs-v3-3-dpo-lr-86819-v3/model.safetensors.index.json
chaiml-csfs-v3-3-dpo-lr-86819-v3-uploader: cp /dev/shm/model_output/README.md s3://guanaco-vllm-models/chaiml-csfs-v3-3-dpo-lr-86819-v3/README.md
chaiml-csfs-v3-3-dpo-lr-86819-v3-uploader: cp /dev/shm/model_output/tokenizer.json s3://guanaco-vllm-models/chaiml-csfs-v3-3-dpo-lr-86819-v3/tokenizer.json
chaiml-csfs-v3-3-dpo-lr-86819-v3-uploader: cp /dev/shm/model_output/model-00003-of-00003.safetensors s3://guanaco-vllm-models/chaiml-csfs-v3-3-dpo-lr-86819-v3/model-00003-of-00003.safetensors
chaiml-csfs-v3-3-dpo-lr-86819-v3-uploader: cp /dev/shm/model_output/model-00001-of-00003.safetensors s3://guanaco-vllm-models/chaiml-csfs-v3-3-dpo-lr-86819-v3/model-00001-of-00003.safetensors
chaiml-csfs-v3-3-dpo-lr-86819-v3-uploader: cp /dev/shm/model_output/model-00002-of-00003.safetensors s3://guanaco-vllm-models/chaiml-csfs-v3-3-dpo-lr-86819-v3/model-00002-of-00003.safetensors
Job chaiml-csfs-v3-3-dpo-lr-86819-v3-uploader completed after 268.33s with status: succeeded
Stopping job with name chaiml-csfs-v3-3-dpo-lr-86819-v3-uploader
Pipeline stage VLLMUploader completed in 268.82s
run pipeline stage %s
Running pipeline stage VLLMTemplater
Pipeline stage VLLMTemplater completed in 0.15s
run pipeline stage %s
Running pipeline stage VLLMDeployer
Creating inference service chaiml-csfs-v3-3-dpo-lr-86819-v3
Waiting for inference service chaiml-csfs-v3-3-dpo-lr-86819-v3 to be ready
HTTP Request: %s %s "%s %d %s"
Inference service chaiml-csfs-v3-3-dpo-lr-86819-v3 ready after 443.3449788093567s
Pipeline stage VLLMDeployer completed in 444.03s
run pipeline stage %s
Running pipeline stage StressChecker
Received healthy response to inference request in 1.1198244094848633s
Received healthy response to inference request in 1.1259405612945557s
Received healthy response to inference request in 1.1360013484954834s
Received healthy response to inference request in 1.1862037181854248s
Received healthy response to inference request in 1.4419851303100586s
Received healthy response to inference request in 0.9608640670776367s
Received healthy response to inference request in 0.9649941921234131s
Received healthy response to inference request in 1.2063770294189453s
Received healthy response to inference request in 1.0625905990600586s
Received healthy response to inference request in 0.9450654983520508s
Received healthy response to inference request in 1.1208531856536865s
Received healthy response to inference request in 1.0107240676879883s
Received healthy response to inference request in 1.0350701808929443s
Received healthy response to inference request in 1.3041906356811523s
Received healthy response to inference request in 1.4203641414642334s
Received healthy response to inference request in 0.9861791133880615s
Received healthy response to inference request in 1.22003173828125s
Received healthy response to inference request in 1.8737709522247314s
Received healthy response to inference request in 0.9446709156036377s
Received healthy response to inference request in 1.2306432723999023s
Received healthy response to inference request in 0.9494128227233887s
Received healthy response to inference request in 1.7181057929992676s
Received healthy response to inference request in 1.4289851188659668s
Received healthy response to inference request in 1.2350196838378906s
Received healthy response to inference request in 1.473372459411621s
{"detail":"HTTPConnectionPool(host='chaiml-csfs-v3-3-dpo-lr-86819-v3-predictor.tenant-chaiml-guanaco.kchai-coreweave-us-east-04a.chaiverse.com', port=80): Max retries exceeded with url: /v1/completions (Caused by ConnectTimeoutError(<urllib3.connection.HTTPConnection object at 0x7b8143cad690>, 'Connection to chaiml-csfs-v3-3-dpo-lr-86819-v3-predictor.tenant-chaiml-guanaco.kchai-coreweave-us-east-04a.chaiverse.com timed out. (connect timeout=12.0)'))"}
Received unhealthy response to inference request!
Received healthy response to inference request in 1.017993688583374s
Received healthy response to inference request in 0.9600915908813477s
Received healthy response to inference request in 1.118469476699829s
Received healthy response to inference request in 1.4621522426605225s
30 requests
1 failed requests
5th percentile: 0.9470217943191528
10th percentile: 0.9590237140655518
20th percentile: 0.9819421291351319
30th percentile: 1.0299472332000732
40th percentile: 1.1192824363708496
50th percentile: 1.1309709548950195
60th percentile: 1.2118389129638671
70th percentile: 1.255770969390869
80th percentile: 1.4315851211547852
90th percentile: 1.497845792770386
95th percentile: 1.8037216305732722
99th percentile: 9.21068707704545
mean time: 1.5622467756271363
%s, retrying in %s seconds...
Received healthy response to inference request in 0.939246416091919s
Received healthy response to inference request in 1.0496156215667725s
Received healthy response to inference request in 1.0543913841247559s
Received healthy response to inference request in 1.0507714748382568s
Received healthy response to inference request in 0.9331402778625488s
Received healthy response to inference request in 1.2305738925933838s
Received healthy response to inference request in 1.194897174835205s
Received healthy response to inference request in 0.9551825523376465s
Received healthy response to inference request in 0.9808745384216309s
Received healthy response to inference request in 1.3068387508392334s
Received healthy response to inference request in 1.1959245204925537s
Received healthy response to inference request in 1.8002793788909912s
Received healthy response to inference request in 0.9125864505767822s
Received healthy response to inference request in 1.2066006660461426s
Received healthy response to inference request in 1.00535249710083s
Received healthy response to inference request in 1.2564358711242676s
Received healthy response to inference request in 1.019286870956421s
Received healthy response to inference request in 1.1066458225250244s
Received healthy response to inference request in 1.2809069156646729s
Received healthy response to inference request in 1.0353312492370605s
Received healthy response to inference request in 1.6842265129089355s
Received healthy response to inference request in 0.9351809024810791s
Received healthy response to inference request in 1.0734994411468506s
Received healthy response to inference request in 1.283738374710083s
Received healthy response to inference request in 1.2668704986572266s
Received healthy response to inference request in 1.8925166130065918s
Received healthy response to inference request in 1.2010369300842285s
Received healthy response to inference request in 1.278959035873413s
Received healthy response to inference request in 1.3602466583251953s
Received healthy response to inference request in 0.9551968574523926s
30 requests
0 failed requests
5th percentile: 0.9340585589408874
10th percentile: 0.9388398647308349
20th percentile: 0.9757390022277832
30th percentile: 1.0305179357528687
40th percentile: 1.0529434204101562
50th percentile: 1.1507714986801147
60th percentile: 1.2032624244689942
70th percentile: 1.2595662593841552
80th percentile: 1.281473207473755
90th percentile: 1.3926446437835698
95th percentile: 1.748055589199066
99th percentile: 1.8657678151130677
mean time: 1.1815451383590698
Pipeline stage StressChecker completed in 88.10s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyTriggerPipeline
run_pipeline:run_in_cloud %s
starting trigger_guanaco_pipeline args=%s
triggered trigger_guanaco_pipeline args=%s
Pipeline stage OfflineFamilyFriendlyTriggerPipeline completed in 0.66s
Shutdown handler de-registered
chaiml-csfs-v3-3-dpo-lr_86819_v3 status is now deployed due to DeploymentManager action
chaiml-csfs-v3-3-dpo-lr_86819_v3 status is now inactive due to auto deactivation removed underperforming models
chaiml-csfs-v3-3-dpo-lr_86819_v3 status is now torndown due to DeploymentManager action