chaiml-merged-qwen-35-_39140

developer_uid: chai_backend_admin
submission_id: chaiml-merged-qwen-35-_39140_v10
model_name: chaiml-merged-qwen-35-_39140_v10
model_group: ChaiML/merged_qwen_35_dp
status: torndown
timestamp: 2026-03-30T20:51:57+00:00
num_battles: 14966
num_wins: 8135
celo_rating: 1325.96
family_friendly_score: 0.0
family_friendly_standard_error: 0.0
submission_type: basic
model_repo: ChaiML/merged_qwen_35_dpo_lower_lr_v
model_architecture: Qwen3_5ForConditionalGeneration
model_num_parameters: 23564784640.0
best_of: 8
max_input_tokens: 2048
max_output_tokens: 80
reward_model: default
display_name: chaiml-merged-qwen-35-_39140_v10
ineligible_reason: max_output_tokens!=64
is_internal_developer: True
language_model: ChaiML/merged_qwen_35_dpo_lower_lr_v
model_size: 24B
ranking_group: single
us_pacific_date: 2026-03-27
win_ratio: 0.5435654149405319
generation_params: {'temperature': 1.0, 'top_p': 0.95, 'min_p': 0.05, 'top_k': 60, 'presence_penalty': 0.1, 'frequency_penalty': 0.0, 'stopping_words': ['<|im_end|>', '<|im_start|>', 'You:', '###', '</s>'], 'max_input_tokens': 2048, 'best_of': 8, 'max_output_tokens': 80}
formatter: {'memory_template': '<|im_start|>system\nRespond as a high quality storyteller.<|im_end|>\n<|im_start|>user\n', 'prompt_template': '', 'bot_template': '{bot_name}: {message}\n', 'user_template': '{user_name}: {message}\n', 'response_template': '<|im_end|>\n<|im_start|>assistant\n{bot_name}:', 'truncate_by_message': False}
Resubmit model
Shutdown handler not registered because Python interpreter is not running in the main thread
run pipeline %s
run pipeline stage %s
Running pipeline stage VLLMUploader
Starting job with name chaiml-merged-qwen-35-39140-v10-uploader
Waiting for job on chaiml-merged-qwen-35-39140-v10-uploader to finish
chaiml-merged-qwen-35-39140-v10-uploader: Using quantization_mode: none
chaiml-merged-qwen-35-39140-v10-uploader: Downloading snapshot of ChaiML/merged_qwen_35_dpo_lower_lr_v...
chaiml-merged-qwen-35-39140-v10-uploader: Downloaded in 32.368s
2026-03-27T19:32:44.255867+00:00 monitor updated for chaiml-merged-qwen-35-_39140_v10
chaiml-merged-qwen-35-39140-v10-uploader: Processed model ChaiML/merged_qwen_35_dpo_lower_lr_v in 53.518s
chaiml-merged-qwen-35-39140-v10-uploader: creating bucket guanaco-vllm-models
chaiml-merged-qwen-35-39140-v10-uploader: /usr/lib/python3/dist-packages/S3/BaseUtils.py:56: SyntaxWarning: invalid escape sequence '\.'
chaiml-merged-qwen-35-39140-v10-uploader:   RE_S3_DATESTRING = re.compile('\.[0-9]*(?:[Z\\-\\+]*?)')
chaiml-merged-qwen-35-39140-v10-uploader: /usr/lib/python3/dist-packages/S3/BaseUtils.py:57: SyntaxWarning: invalid escape sequence '\s'
chaiml-merged-qwen-35-39140-v10-uploader:   RE_XML_NAMESPACE = re.compile(b'^(<?[^>]+?>\s*|\s*)(<\w+) xmlns=[\'"](https?://[^\'"]+)[\'"]', re.MULTILINE)
chaiml-merged-qwen-35-39140-v10-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:240: SyntaxWarning: invalid escape sequence '\.'
chaiml-merged-qwen-35-39140-v10-uploader:   invalid = re.search("([^a-z0-9\.-])", bucket, re.UNICODE)
chaiml-merged-qwen-35-39140-v10-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:244: SyntaxWarning: invalid escape sequence '\.'
chaiml-merged-qwen-35-39140-v10-uploader:   invalid = re.search("([^A-Za-z0-9\._-])", bucket, re.UNICODE)
chaiml-merged-qwen-35-39140-v10-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:255: SyntaxWarning: invalid escape sequence '\.'
chaiml-merged-qwen-35-39140-v10-uploader:   if re.search("-\.", bucket, re.UNICODE):
chaiml-merged-qwen-35-39140-v10-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:257: SyntaxWarning: invalid escape sequence '\.'
chaiml-merged-qwen-35-39140-v10-uploader:   if re.search("\.\.", bucket, re.UNICODE):
chaiml-merged-qwen-35-39140-v10-uploader: /usr/lib/python3/dist-packages/S3/S3Uri.py:155: SyntaxWarning: invalid escape sequence '\w'
chaiml-merged-qwen-35-39140-v10-uploader:   _re = re.compile("^(\w+://)?(.*)", re.UNICODE)
chaiml-merged-qwen-35-39140-v10-uploader: /usr/lib/python3/dist-packages/S3/FileLists.py:480: SyntaxWarning: invalid escape sequence '\*'
chaiml-merged-qwen-35-39140-v10-uploader:   wildcard_split_result = re.split("\*|\?", uri_str, maxsplit=1)
chaiml-merged-qwen-35-39140-v10-uploader: Bucket 's3://guanaco-vllm-models/' created
chaiml-merged-qwen-35-39140-v10-uploader: uploading /dev/shm/model_output to s3://guanaco-vllm-models/chaiml-merged-qwen-35-39140-v10/default
chaiml-merged-qwen-35-39140-v10-uploader: cp /dev/shm/model_output/.gitattributes s3://guanaco-vllm-models/chaiml-merged-qwen-35-39140-v10/default/.gitattributes
chaiml-merged-qwen-35-39140-v10-uploader: cp /dev/shm/model_output/tokenizer_config.json s3://guanaco-vllm-models/chaiml-merged-qwen-35-39140-v10/default/tokenizer_config.json
chaiml-merged-qwen-35-39140-v10-uploader: cp /dev/shm/model_output/config.json s3://guanaco-vllm-models/chaiml-merged-qwen-35-39140-v10/default/config.json
chaiml-merged-qwen-35-39140-v10-uploader: cp /dev/shm/model_output/generation_config.json s3://guanaco-vllm-models/chaiml-merged-qwen-35-39140-v10/default/generation_config.json
chaiml-merged-qwen-35-39140-v10-uploader: cp /dev/shm/model_output/README.md s3://guanaco-vllm-models/chaiml-merged-qwen-35-39140-v10/default/README.md
chaiml-merged-qwen-35-39140-v10-uploader: cp /dev/shm/model_output/model.safetensors.index.json s3://guanaco-vllm-models/chaiml-merged-qwen-35-39140-v10/default/model.safetensors.index.json
chaiml-merged-qwen-35-39140-v10-uploader: cp /dev/shm/model_output/tokenizer.json s3://guanaco-vllm-models/chaiml-merged-qwen-35-39140-v10/default/tokenizer.json
chaiml-merged-qwen-35-39140-v10-uploader: cp /dev/shm/model_output/model-00002-of-00002.safetensors s3://guanaco-vllm-models/chaiml-merged-qwen-35-39140-v10/default/model-00002-of-00002.safetensors
2026-03-27T19:33:44.344863+00:00 monitor updated for chaiml-merged-qwen-35-_39140_v10
chaiml-merged-qwen-35-39140-v10-uploader: cp /dev/shm/model_output/model-00001-of-00002.safetensors s3://guanaco-vllm-models/chaiml-merged-qwen-35-39140-v10/default/model-00001-of-00002.safetensors
Job chaiml-merged-qwen-35-39140-v10-uploader completed after 152.99s with status: succeeded
Stopping job with name chaiml-merged-qwen-35-39140-v10-uploader
Pipeline stage VLLMUploader completed in 153.48s
run pipeline stage %s
Running pipeline stage VLLMTemplater
Pipeline stage VLLMTemplater completed in 2.13s
run pipeline stage %s
Running pipeline stage VLLMDeployer
Creating inference service chaiml-merged-qwen-35-39140-v10
Waiting for inference service chaiml-merged-qwen-35-39140-v10 to be ready
2026-03-27T19:34:44.448726+00:00 monitor updated for chaiml-merged-qwen-35-_39140_v10
2026-03-27T19:35:44.552656+00:00 monitor updated for chaiml-merged-qwen-35-_39140_v10
2026-03-27T19:36:44.649974+00:00 monitor updated for chaiml-merged-qwen-35-_39140_v10
Inference service chaiml-merged-qwen-35-39140-v10 ready after 180.57799196243286s
Pipeline stage VLLMDeployer completed in 181.09s
run pipeline stage %s
Running pipeline stage StressChecker
HTTPConnectionPool(host='guanaco-submitter.guanaco-backend.kchai-google-us-east4.chaiverse.com', port=80): Read timed out. (read timeout=20)
Received unhealthy response to inference request!
2026-03-27T19:37:44.739407+00:00 monitor updated for chaiml-merged-qwen-35-_39140_v10
HTTPConnectionPool(host='guanaco-submitter.guanaco-backend.kchai-google-us-east4.chaiverse.com', port=80): Read timed out. (read timeout=20)
Received unhealthy response to inference request!
HTTPConnectionPool(host='guanaco-submitter.guanaco-backend.kchai-google-us-east4.chaiverse.com', port=80): Read timed out. (read timeout=20)
Received unhealthy response to inference request!
HTTPConnectionPool(host='guanaco-submitter.guanaco-backend.kchai-google-us-east4.chaiverse.com', port=80): Read timed out. (read timeout=20)
Received unhealthy response to inference request!
2026-03-27T19:38:44.832794+00:00 monitor updated for chaiml-merged-qwen-35-_39140_v10
HTTPConnectionPool(host='guanaco-submitter.guanaco-backend.kchai-google-us-east4.chaiverse.com', port=80): Read timed out. (read timeout=20)
Received unhealthy response to inference request!
HTTPConnectionPool(host='guanaco-submitter.guanaco-backend.kchai-google-us-east4.chaiverse.com', port=80): Read timed out. (read timeout=20)
Received unhealthy response to inference request!
HTTPConnectionPool(host='guanaco-submitter.guanaco-backend.kchai-google-us-east4.chaiverse.com', port=80): Read timed out. (read timeout=20)
Received unhealthy response to inference request!
2026-03-27T19:39:44.931363+00:00 monitor updated for chaiml-merged-qwen-35-_39140_v10
HTTPConnectionPool(host='guanaco-submitter.guanaco-backend.kchai-google-us-east4.chaiverse.com', port=80): Read timed out. (read timeout=20)
Received unhealthy response to inference request!
HTTPConnectionPool(host='guanaco-submitter.guanaco-backend.kchai-google-us-east4.chaiverse.com', port=80): Read timed out. (read timeout=20)
Received unhealthy response to inference request!
HTTPConnectionPool(host='guanaco-submitter.guanaco-backend.kchai-google-us-east4.chaiverse.com', port=80): Read timed out. (read timeout=20)
Received unhealthy response to inference request!
2026-03-27T19:40:45.036164+00:00 monitor updated for chaiml-merged-qwen-35-_39140_v10
Received healthy response to inference request in 4.015596866607666s
Received healthy response to inference request in 10.33107328414917s
Received healthy response to inference request in 10.530271768569946s
Received healthy response to inference request in 10.310223817825317s
Received healthy response to inference request in 3.707411527633667s
Received healthy response to inference request in 3.8954992294311523s
Received healthy response to inference request in 10.3325514793396s
Received healthy response to inference request in 3.819653034210205s
Received healthy response to inference request in 3.6357736587524414s
2026-03-27T19:41:45.138608+00:00 monitor updated for chaiml-merged-qwen-35-_39140_v10
Received healthy response to inference request in 3.682938814163208s
Received healthy response to inference request in 3.6427013874053955s
Received healthy response to inference request in 3.7330827713012695s
Received healthy response to inference request in 3.6854424476623535s
Received healthy response to inference request in 3.653923749923706s
Received healthy response to inference request in 4.174396514892578s
Received healthy response to inference request in 3.683255910873413s
Received healthy response to inference request in 3.8984861373901367s
Received healthy response to inference request in 3.7086617946624756s
Received healthy response to inference request in 3.7304584980010986s
Received healthy response to inference request in 3.7233691215515137s
30 requests
10 failed requests
5th percentile: 3.6477514505386353
10th percentile: 3.6800373077392576
20th percentile: 3.7030177116394043
30th percentile: 3.728331685066223
40th percentile: 3.8651607513427737
50th percentile: 4.094996690750122
60th percentile: 10.331664562225342
70th percentile: 20.10252151489258
80th percentile: 20.117742204666136
90th percentile: 20.124517560005188
95th percentile: 20.136465525627138
99th percentile: 20.141341114044188
mean time: 10.103398338953655
%s, retrying in %s seconds...
Received healthy response to inference request in 3.6584293842315674s
Received healthy response to inference request in 3.710052728652954s
Received healthy response to inference request in 3.7176740169525146s
Received healthy response to inference request in 3.7045161724090576s
Received healthy response to inference request in 3.720886468887329s
2026-03-27T19:42:45.239827+00:00 monitor updated for chaiml-merged-qwen-35-_39140_v10
Received healthy response to inference request in 3.6565616130828857s
Received healthy response to inference request in 3.606623649597168s
Received healthy response to inference request in 3.8023970127105713s
Received healthy response to inference request in 3.7735462188720703s
Received healthy response to inference request in 3.649217128753662s
Received healthy response to inference request in 3.8734593391418457s
Received healthy response to inference request in 3.7939114570617676s
Received healthy response to inference request in 3.6013314723968506s
Received healthy response to inference request in 3.6790072917938232s
Received healthy response to inference request in 3.66778564453125s
Received healthy response to inference request in 4.086525201797485s
Received healthy response to inference request in 3.7173638343811035s
Received healthy response to inference request in 3.7173731327056885s
Received healthy response to inference request in 3.7181882858276367s
Received healthy response to inference request in 3.6915206909179688s
2026-03-27T19:43:45.336017+00:00 monitor updated for chaiml-merged-qwen-35-_39140_v10
Received healthy response to inference request in 3.8342678546905518s
Received healthy response to inference request in 3.736711263656616s
Received healthy response to inference request in 3.569862127304077s
Received healthy response to inference request in 3.7084450721740723s
Received healthy response to inference request in 3.697300434112549s
Received healthy response to inference request in 3.8408470153808594s
Received healthy response to inference request in 3.6954779624938965s
Received healthy response to inference request in 3.7234506607055664s
Received healthy response to inference request in 3.7858128547668457s
Received healthy response to inference request in 3.728081464767456s
30 requests
0 failed requests
5th percentile: 3.6037129521369935
10th percentile: 3.6449577808380127
20th percentile: 3.6659143924713136
30th percentile: 3.694290781021118
40th percentile: 3.7068735122680665
50th percentile: 3.717368483543396
60th percentile: 3.719267559051514
70th percentile: 3.7306704044342043
80th percentile: 3.78743257522583
90th percentile: 3.8349257707595825
95th percentile: 3.8587837934494016
99th percentile: 4.02473610162735
mean time: 3.7288875818252563
Pipeline stage StressChecker completed in 419.36s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyTriggerPipeline
run_pipeline:run_in_cloud %s
starting trigger_guanaco_pipeline args=%s
triggered trigger_guanaco_pipeline args=%s
Pipeline stage OfflineFamilyFriendlyTriggerPipeline completed in 0.66s
Shutdown handler de-registered
chaiml-merged-qwen-35-_39140_v10 status is now deployed due to DeploymentManager action
chaiml-merged-qwen-35-_39140_v10 status is now inactive due to auto deactivation removed underperforming models
chaiml-merged-qwen-35-_39140_v10 status is now torndown due to DeploymentManager action