chaiml-merged-qwen-35-d_39140

developer_uid: chai_backend_admin

submission_id: chaiml-merged-qwen-35-d_39140_v2

model_name: chaiml-merged-qwen-35-d_39140_v2

model_group: ChaiML/merged_qwen_35_dp

status: torndown

timestamp: 2026-03-28T23:02:01+00:00

num_battles: 10900

num_wins: 6161

celo_rating: 1338.47

family_friendly_score: 0.0

family_friendly_standard_error: 0.0

submission_type: basic

model_repo: ChaiML/merged_qwen_35_dpo_lower_lr_v

model_architecture: Qwen3_5ForConditionalGeneration

model_num_parameters: 23564784640.0

best_of: 8

max_input_tokens: 2048

max_output_tokens: 80

reward_model: default

display_name: chaiml-merged-qwen-35-d_39140_v2

ineligible_reason: max_output_tokens!=64

is_internal_developer: True

language_model: ChaiML/merged_qwen_35_dpo_lower_lr_v

model_size: 24B

ranking_group: single

us_pacific_date: 2026-03-25

win_ratio: 0.5652293577981652

generation_params: {'temperature': 1.0, 'top_p': 0.95, 'min_p': 0.05, 'top_k': 60, 'presence_penalty': 0.1, 'frequency_penalty': 0.0, 'stopping_words': ['###', 'You:', '<|im_end|>', '</s>', '<|im_start|>'], 'max_input_tokens': 2048, 'best_of': 8, 'max_output_tokens': 80}

formatter: {'memory_template': '<|im_start|>system\nRespond as a high quality storyteller.<|im_end|>\n<|im_start|>user\n', 'prompt_template': '', 'bot_template': '{bot_name}: {message}\n', 'user_template': '{user_name}: {message}\n', 'response_template': '<|im_end|>\n<|im_start|>assistant\n{bot_name}:', 'truncate_by_message': False}

Resubmit model

Shutdown handler not registered because Python interpreter is not running in the main thread
run pipeline %s
run pipeline stage %s
Running pipeline stage VLLMUploader
Starting job with name chaiml-merged-qwen-35-d-39140-v2-uploader
Waiting for job on chaiml-merged-qwen-35-d-39140-v2-uploader to finish
chaiml-merged-qwen-35-d-39140-v2-uploader: Using quantization_mode: none
chaiml-merged-qwen-35-d-39140-v2-uploader: Downloading snapshot of ChaiML/merged_qwen_35_dpo_lower_lr_v...
chaiml-merged-qwen-35-d-39140-v2-uploader: Downloaded in 31.670s
2026-03-25T21:06:05.268121+00:00 monitor updated for chaiml-merged-qwen-35-d_39140_v2
chaiml-merged-qwen-35-d-39140-v2-uploader: Processed model ChaiML/merged_qwen_35_dpo_lower_lr_v in 52.626s
chaiml-merged-qwen-35-d-39140-v2-uploader: creating bucket guanaco-vllm-models
chaiml-merged-qwen-35-d-39140-v2-uploader: /usr/lib/python3/dist-packages/S3/BaseUtils.py:56: SyntaxWarning: invalid escape sequence '\.'
chaiml-merged-qwen-35-d-39140-v2-uploader:   RE_S3_DATESTRING = re.compile('\.[0-9]*(?:[Z\\-\\+]*?)')
chaiml-merged-qwen-35-d-39140-v2-uploader: /usr/lib/python3/dist-packages/S3/BaseUtils.py:57: SyntaxWarning: invalid escape sequence '\s'
chaiml-merged-qwen-35-d-39140-v2-uploader:   RE_XML_NAMESPACE = re.compile(b'^(<?[^>]+?>\s*|\s*)(<\w+) xmlns=[\'"](https?://[^\'"]+)[\'"]', re.MULTILINE)
chaiml-merged-qwen-35-d-39140-v2-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:240: SyntaxWarning: invalid escape sequence '\.'
chaiml-merged-qwen-35-d-39140-v2-uploader:   invalid = re.search("([^a-z0-9\.-])", bucket, re.UNICODE)
chaiml-merged-qwen-35-d-39140-v2-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:244: SyntaxWarning: invalid escape sequence '\.'
chaiml-merged-qwen-35-d-39140-v2-uploader:   invalid = re.search("([^A-Za-z0-9\._-])", bucket, re.UNICODE)
chaiml-merged-qwen-35-d-39140-v2-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:255: SyntaxWarning: invalid escape sequence '\.'
chaiml-merged-qwen-35-d-39140-v2-uploader:   if re.search("-\.", bucket, re.UNICODE):
chaiml-merged-qwen-35-d-39140-v2-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:257: SyntaxWarning: invalid escape sequence '\.'
chaiml-merged-qwen-35-d-39140-v2-uploader:   if re.search("\.\.", bucket, re.UNICODE):
chaiml-merged-qwen-35-d-39140-v2-uploader: /usr/lib/python3/dist-packages/S3/S3Uri.py:155: SyntaxWarning: invalid escape sequence '\w'
chaiml-merged-qwen-35-d-39140-v2-uploader:   _re = re.compile("^(\w+://)?(.*)", re.UNICODE)
chaiml-merged-qwen-35-d-39140-v2-uploader: /usr/lib/python3/dist-packages/S3/FileLists.py:480: SyntaxWarning: invalid escape sequence '\*'
chaiml-merged-qwen-35-d-39140-v2-uploader:   wildcard_split_result = re.split("\*|\?", uri_str, maxsplit=1)
chaiml-merged-qwen-35-d-39140-v2-uploader: Bucket 's3://guanaco-vllm-models/' created
chaiml-merged-qwen-35-d-39140-v2-uploader: uploading /dev/shm/model_output to s3://guanaco-vllm-models/chaiml-merged-qwen-35-d-39140-v2/default
chaiml-merged-qwen-35-d-39140-v2-uploader: cp /dev/shm/model_output/README.md s3://guanaco-vllm-models/chaiml-merged-qwen-35-d-39140-v2/default/README.md
chaiml-merged-qwen-35-d-39140-v2-uploader: cp /dev/shm/model_output/generation_config.json s3://guanaco-vllm-models/chaiml-merged-qwen-35-d-39140-v2/default/generation_config.json
chaiml-merged-qwen-35-d-39140-v2-uploader: cp /dev/shm/model_output/config.json s3://guanaco-vllm-models/chaiml-merged-qwen-35-d-39140-v2/default/config.json
chaiml-merged-qwen-35-d-39140-v2-uploader: cp /dev/shm/model_output/model.safetensors.index.json s3://guanaco-vllm-models/chaiml-merged-qwen-35-d-39140-v2/default/model.safetensors.index.json
chaiml-merged-qwen-35-d-39140-v2-uploader: cp /dev/shm/model_output/.gitattributes s3://guanaco-vllm-models/chaiml-merged-qwen-35-d-39140-v2/default/.gitattributes
chaiml-merged-qwen-35-d-39140-v2-uploader: cp /dev/shm/model_output/tokenizer_config.json s3://guanaco-vllm-models/chaiml-merged-qwen-35-d-39140-v2/default/tokenizer_config.json
chaiml-merged-qwen-35-d-39140-v2-uploader: cp /dev/shm/model_output/tokenizer.json s3://guanaco-vllm-models/chaiml-merged-qwen-35-d-39140-v2/default/tokenizer.json
chaiml-merged-qwen-35-d-39140-v2-uploader: cp /dev/shm/model_output/model-00002-of-00002.safetensors s3://guanaco-vllm-models/chaiml-merged-qwen-35-d-39140-v2/default/model-00002-of-00002.safetensors
2026-03-25T21:07:05.371475+00:00 monitor updated for chaiml-merged-qwen-35-d_39140_v2
chaiml-merged-qwen-35-d-39140-v2-uploader: cp /dev/shm/model_output/model-00001-of-00002.safetensors s3://guanaco-vllm-models/chaiml-merged-qwen-35-d-39140-v2/default/model-00001-of-00002.safetensors
Job chaiml-merged-qwen-35-d-39140-v2-uploader completed after 143.67s with status: succeeded
Stopping job with name chaiml-merged-qwen-35-d-39140-v2-uploader
Pipeline stage VLLMUploader completed in 144.21s
run pipeline stage %s
Running pipeline stage VLLMTemplater
Pipeline stage VLLMTemplater completed in 3.18s
run pipeline stage %s
Running pipeline stage VLLMDeployer
Creating inference service chaiml-merged-qwen-35-d-39140-v2
Waiting for inference service chaiml-merged-qwen-35-d-39140-v2 to be ready
2026-03-25T21:08:05.478450+00:00 monitor updated for chaiml-merged-qwen-35-d_39140_v2
2026-03-25T21:09:05.570196+00:00 monitor updated for chaiml-merged-qwen-35-d_39140_v2
2026-03-25T21:10:05.659789+00:00 monitor updated for chaiml-merged-qwen-35-d_39140_v2
Inference service chaiml-merged-qwen-35-d-39140-v2 ready after 180.5371494293213s
Pipeline stage VLLMDeployer completed in 181.09s
run pipeline stage %s
Running pipeline stage StressChecker
HTTPConnectionPool(host='guanaco-submitter-v2.guanaco-backend.kchai-google-us-east4.chaiverse.com', port=80): Read timed out. (read timeout=20)
Received unhealthy response to inference request!
2026-03-25T21:11:05.749678+00:00 monitor updated for chaiml-merged-qwen-35-d_39140_v2
HTTPConnectionPool(host='guanaco-submitter-v2.guanaco-backend.kchai-google-us-east4.chaiverse.com', port=80): Read timed out. (read timeout=20)
Received unhealthy response to inference request!
HTTPConnectionPool(host='guanaco-submitter-v2.guanaco-backend.kchai-google-us-east4.chaiverse.com', port=80): Read timed out. (read timeout=20)
Received unhealthy response to inference request!
HTTPConnectionPool(host='guanaco-submitter-v2.guanaco-backend.kchai-google-us-east4.chaiverse.com', port=80): Read timed out. (read timeout=20)
Received unhealthy response to inference request!
2026-03-25T21:12:05.838911+00:00 monitor updated for chaiml-merged-qwen-35-d_39140_v2
HTTPConnectionPool(host='guanaco-submitter-v2.guanaco-backend.kchai-google-us-east4.chaiverse.com', port=80): Read timed out. (read timeout=20)
Received unhealthy response to inference request!
Received healthy response to inference request in 10.860724210739136s
Received healthy response to inference request in 10.928340911865234s
Received healthy response to inference request in 3.7726876735687256s
HTTPConnectionPool(host='guanaco-submitter-v2.guanaco-backend.kchai-google-us-east4.chaiverse.com', port=80): Read timed out. (read timeout=20)
Received unhealthy response to inference request!
2026-03-25T21:13:05.928914+00:00 monitor updated for chaiml-merged-qwen-35-d_39140_v2
HTTPConnectionPool(host='guanaco-submitter-v2.guanaco-backend.kchai-google-us-east4.chaiverse.com', port=80): Read timed out. (read timeout=20)
Received unhealthy response to inference request!
Received healthy response to inference request in 3.893080234527588s
Received healthy response to inference request in 10.959409952163696s
Received healthy response to inference request in 4.425200939178467s
HTTPConnectionPool(host='guanaco-submitter-v2.guanaco-backend.kchai-google-us-east4.chaiverse.com', port=80): Read timed out. (read timeout=20)
Received unhealthy response to inference request!
Received healthy response to inference request in 4.297606468200684s
2026-03-25T21:14:06.022531+00:00 monitor updated for chaiml-merged-qwen-35-d_39140_v2
Received healthy response to inference request in 4.1480467319488525s
Received healthy response to inference request in 4.286984443664551s
Received healthy response to inference request in 3.951817274093628s
Received healthy response to inference request in 4.00217342376709s
Received healthy response to inference request in 3.895796775817871s
Received healthy response to inference request in 4.027745723724365s
Received healthy response to inference request in 3.9653515815734863s
Received healthy response to inference request in 4.084394454956055s
Received healthy response to inference request in 3.9409656524658203s
Received healthy response to inference request in 4.093670129776001s
Received healthy response to inference request in 4.003840684890747s
Received healthy response to inference request in 3.973865032196045s
Received healthy response to inference request in 4.071971893310547s
Received healthy response to inference request in 4.074754476547241s
2026-03-25T21:15:06.137464+00:00 monitor updated for chaiml-merged-qwen-35-d_39140_v2
Received healthy response to inference request in 4.075298070907593s
30 requests
8 failed requests
5th percentile: 3.8943026781082155
10th percentile: 3.9364487648010256
20th percentile: 3.9721623420715333
30th percentile: 4.02057421207428
40th percentile: 4.075080633163452
50th percentile: 4.120858430862427
60th percentile: 4.348644256591797
70th percentile: 10.937661623954773
80th percentile: 20.116388988494872
90th percentile: 20.136811208724975
95th percentile: 20.14676411151886
99th percentile: 20.38924783706665
mean time: 9.037636431058248
%s, retrying in %s seconds...
Received healthy response to inference request in 3.7674598693847656s
Received healthy response to inference request in 3.8915092945098877s
Received healthy response to inference request in 5.635619640350342s
Received healthy response to inference request in 3.754075050354004s
Received healthy response to inference request in 4.051485776901245s
Received healthy response to inference request in 3.834411859512329s
Received healthy response to inference request in 4.068678379058838s
Received healthy response to inference request in 3.7879953384399414s
Received healthy response to inference request in 4.21508264541626s
Received healthy response to inference request in 3.807668924331665s
Received healthy response to inference request in 3.909403085708618s
Received healthy response to inference request in 3.9484925270080566s
Received healthy response to inference request in 4.098727226257324s
Received healthy response to inference request in 3.931988000869751s
2026-03-25T21:16:06.362485+00:00 monitor updated for chaiml-merged-qwen-35-d_39140_v2
Received healthy response to inference request in 3.7987377643585205s
Received healthy response to inference request in 4.722065210342407s
Received healthy response to inference request in 3.9365081787109375s
Received healthy response to inference request in 4.012132406234741s
Received healthy response to inference request in 3.9404101371765137s
Received healthy response to inference request in 4.081709384918213s
Received healthy response to inference request in 4.266304969787598s
Received healthy response to inference request in 4.1984312534332275s
Received healthy response to inference request in 4.10934591293335s
Received healthy response to inference request in 4.3329596519470215s
Received healthy response to inference request in 4.369241237640381s
Received healthy response to inference request in 4.005509614944458s
Received healthy response to inference request in 3.9548757076263428s
Received healthy response to inference request in 4.038480997085571s
2026-03-25T21:17:06.461958+00:00 monitor updated for chaiml-merged-qwen-35-d_39140_v2
Received healthy response to inference request in 4.2628583908081055s
Received healthy response to inference request in 4.296729326248169s
30 requests
0 failed requests
5th percentile: 3.776700830459595
10th percentile: 3.7976635217666628
20th percentile: 3.880089807510376
30th percentile: 3.9351521253585817
40th percentile: 3.9523224353790285
50th percentile: 4.025306701660156
60th percentile: 4.0738907814025875
70th percentile: 4.136071515083313
80th percentile: 4.263547706604004
90th percentile: 4.336587810516358
95th percentile: 4.563294422626495
99th percentile: 5.370688855648042
mean time: 4.1009632587432865
Pipeline stage StressChecker completed in 399.23s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyTriggerPipeline
run_pipeline:run_in_cloud %s
starting trigger_guanaco_pipeline args=%s
triggered trigger_guanaco_pipeline args=%s
Pipeline stage OfflineFamilyFriendlyTriggerPipeline completed in 2.30s
Shutdown handler de-registered
chaiml-merged-qwen-35-d_39140_v2 status is now deployed due to DeploymentManager action
chaiml-merged-qwen-35-d_39140_v2 status is now inactive due to auto deactivation removed underperforming models
chaiml-merged-qwen-35-d_39140_v2 status is now torndown due to DeploymentManager action