chaiml-qwen-bobo-dpo-jud_2070

developer_uid: rirv938
submission_id: chaiml-qwen-bobo-dpo-jud_2070_v1
model_name: chaiml-qwen-bobo-dpo-jud_2070_v1
model_group: ChaiML/qwen_bobo_dpo_jud
status: torndown
timestamp: 2026-03-30T18:31:41+00:00
num_battles: 10475
num_wins: 5983
celo_rating: 1343.67
family_friendly_score: 0.0
family_friendly_standard_error: 0.0
submission_type: basic
model_repo: ChaiML/qwen_bobo_dpo_judge_plus_reward_375_merged
model_architecture: Qwen3_5ForConditionalGeneration
model_num_parameters: 23564784640.0
best_of: 8
max_input_tokens: 2048
max_output_tokens: 80
reward_model: default
display_name: chaiml-qwen-bobo-dpo-jud_2070_v1
ineligible_reason: max_output_tokens!=64
is_internal_developer: True
language_model: ChaiML/qwen_bobo_dpo_judge_plus_reward_375_merged
model_size: 24B
ranking_group: single
us_pacific_date: 2026-03-27
win_ratio: 0.5711694510739856
generation_params: {'temperature': 1.0, 'top_p': 0.95, 'min_p': 0.05, 'top_k': 60, 'presence_penalty': 0.1, 'frequency_penalty': 0.0, 'stopping_words': ['</s>', '###', '<|im_end|>', 'You:', '<|im_start|>'], 'max_input_tokens': 2048, 'best_of': 8, 'max_output_tokens': 80}
formatter: {'memory_template': '<|im_start|>system\nRespond as a high quality storyteller.<|im_end|>\n<|im_start|>user\n', 'prompt_template': '', 'bot_template': '{bot_name}: {message}\n', 'user_template': '{user_name}: {message}\n', 'response_template': '<|im_end|>\n<|im_start|>assistant\n{bot_name}:', 'truncate_by_message': False}
Resubmit model
Shutdown handler not registered because Python interpreter is not running in the main thread
run pipeline %s
run pipeline stage %s
Running pipeline stage VLLMUploader
Starting job with name chaiml-qwen-bobo-dpo-jud-2070-v1-uploader
Waiting for job on chaiml-qwen-bobo-dpo-jud-2070-v1-uploader to finish
Shutdown handler not registered because Python interpreter is not running in the main thread
run pipeline %s
run pipeline stage %s
Running pipeline stage VLLMUploader
Starting job with name chaiml-qwen-bobo-dpo-ju-49011-v1-uploader
Waiting for job on chaiml-qwen-bobo-dpo-ju-49011-v1-uploader to finish
chaiml-qwen-bobo-dpo-jud-2070-v1-uploader: Using quantization_mode: none
chaiml-qwen-bobo-dpo-jud-2070-v1-uploader: Downloading snapshot of ChaiML/qwen_bobo_dpo_judge_plus_reward_375_merged...
chaiml-qwen-bobo-dpo-ju-49011-v1-uploader: Using quantization_mode: none
chaiml-qwen-bobo-dpo-ju-49011-v1-uploader: Downloading snapshot of ChaiML/qwen_bobo_dpo_judge_plus_reward_750_merged...
chaiml-qwen-bobo-dpo-jud-2070-v1-uploader: Downloaded in 30.902s
chaiml-qwen-bobo-dpo-ju-49011-v1-uploader: Downloaded in 31.850s
2026-03-27T15:20:45.081825+00:00 monitor updated for chaiml-qwen-bobo-dpo-jud_2070_v1
chaiml-qwen-bobo-dpo-jud-2070-v1-uploader: Processed model ChaiML/qwen_bobo_dpo_judge_plus_reward_375_merged in 51.135s
chaiml-qwen-bobo-dpo-jud-2070-v1-uploader: creating bucket guanaco-vllm-models
chaiml-qwen-bobo-dpo-jud-2070-v1-uploader: /usr/lib/python3/dist-packages/S3/BaseUtils.py:56: SyntaxWarning: invalid escape sequence '\.'
chaiml-qwen-bobo-dpo-jud-2070-v1-uploader:   RE_S3_DATESTRING = re.compile('\.[0-9]*(?:[Z\\-\\+]*?)')
chaiml-qwen-bobo-dpo-jud-2070-v1-uploader: /usr/lib/python3/dist-packages/S3/BaseUtils.py:57: SyntaxWarning: invalid escape sequence '\s'
chaiml-qwen-bobo-dpo-jud-2070-v1-uploader:   RE_XML_NAMESPACE = re.compile(b'^(<?[^>]+?>\s*|\s*)(<\w+) xmlns=[\'"](https?://[^\'"]+)[\'"]', re.MULTILINE)
chaiml-qwen-bobo-dpo-jud-2070-v1-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:240: SyntaxWarning: invalid escape sequence '\.'
chaiml-qwen-bobo-dpo-jud-2070-v1-uploader:   invalid = re.search("([^a-z0-9\.-])", bucket, re.UNICODE)
chaiml-qwen-bobo-dpo-jud-2070-v1-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:244: SyntaxWarning: invalid escape sequence '\.'
chaiml-qwen-bobo-dpo-jud-2070-v1-uploader:   invalid = re.search("([^A-Za-z0-9\._-])", bucket, re.UNICODE)
chaiml-qwen-bobo-dpo-jud-2070-v1-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:255: SyntaxWarning: invalid escape sequence '\.'
chaiml-qwen-bobo-dpo-jud-2070-v1-uploader:   if re.search("-\.", bucket, re.UNICODE):
chaiml-qwen-bobo-dpo-jud-2070-v1-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:257: SyntaxWarning: invalid escape sequence '\.'
chaiml-qwen-bobo-dpo-jud-2070-v1-uploader:   if re.search("\.\.", bucket, re.UNICODE):
chaiml-qwen-bobo-dpo-jud-2070-v1-uploader: /usr/lib/python3/dist-packages/S3/S3Uri.py:155: SyntaxWarning: invalid escape sequence '\w'
chaiml-qwen-bobo-dpo-jud-2070-v1-uploader:   _re = re.compile("^(\w+://)?(.*)", re.UNICODE)
chaiml-qwen-bobo-dpo-jud-2070-v1-uploader: /usr/lib/python3/dist-packages/S3/FileLists.py:480: SyntaxWarning: invalid escape sequence '\*'
chaiml-qwen-bobo-dpo-jud-2070-v1-uploader:   wildcard_split_result = re.split("\*|\?", uri_str, maxsplit=1)
chaiml-qwen-bobo-dpo-jud-2070-v1-uploader: Bucket 's3://guanaco-vllm-models/' created
chaiml-qwen-bobo-dpo-jud-2070-v1-uploader: uploading /dev/shm/model_output to s3://guanaco-vllm-models/chaiml-qwen-bobo-dpo-jud-2070-v1/default
chaiml-qwen-bobo-dpo-jud-2070-v1-uploader: cp /dev/shm/model_output/.gitattributes s3://guanaco-vllm-models/chaiml-qwen-bobo-dpo-jud-2070-v1/default/.gitattributes
chaiml-qwen-bobo-dpo-jud-2070-v1-uploader: cp /dev/shm/model_output/model.safetensors.index.json s3://guanaco-vllm-models/chaiml-qwen-bobo-dpo-jud-2070-v1/default/model.safetensors.index.json
chaiml-qwen-bobo-dpo-jud-2070-v1-uploader: cp /dev/shm/model_output/generation_config.json s3://guanaco-vllm-models/chaiml-qwen-bobo-dpo-jud-2070-v1/default/generation_config.json
chaiml-qwen-bobo-dpo-jud-2070-v1-uploader: cp /dev/shm/model_output/config.json s3://guanaco-vllm-models/chaiml-qwen-bobo-dpo-jud-2070-v1/default/config.json
2026-03-27T15:20:48.925384+00:00 monitor updated for chaiml-qwen-bobo-dpo-ju_49011_v1
chaiml-qwen-bobo-dpo-jud-2070-v1-uploader: cp /dev/shm/model_output/tokenizer_config.json s3://guanaco-vllm-models/chaiml-qwen-bobo-dpo-jud-2070-v1/default/tokenizer_config.json
chaiml-qwen-bobo-dpo-jud-2070-v1-uploader: cp /dev/shm/model_output/README.md s3://guanaco-vllm-models/chaiml-qwen-bobo-dpo-jud-2070-v1/default/README.md
chaiml-qwen-bobo-dpo-jud-2070-v1-uploader: cp /dev/shm/model_output/chat_template.jinja s3://guanaco-vllm-models/chaiml-qwen-bobo-dpo-jud-2070-v1/default/chat_template.jinja
chaiml-qwen-bobo-dpo-jud-2070-v1-uploader: cp /dev/shm/model_output/tokenizer.json s3://guanaco-vllm-models/chaiml-qwen-bobo-dpo-jud-2070-v1/default/tokenizer.json
chaiml-qwen-bobo-dpo-ju-49011-v1-uploader: Processed model ChaiML/qwen_bobo_dpo_judge_plus_reward_750_merged in 51.831s
chaiml-qwen-bobo-dpo-ju-49011-v1-uploader: creating bucket guanaco-vllm-models
chaiml-qwen-bobo-dpo-ju-49011-v1-uploader: /usr/lib/python3/dist-packages/S3/BaseUtils.py:56: SyntaxWarning: invalid escape sequence '\.'
chaiml-qwen-bobo-dpo-ju-49011-v1-uploader:   RE_S3_DATESTRING = re.compile('\.[0-9]*(?:[Z\\-\\+]*?)')
chaiml-qwen-bobo-dpo-ju-49011-v1-uploader: /usr/lib/python3/dist-packages/S3/BaseUtils.py:57: SyntaxWarning: invalid escape sequence '\s'
chaiml-qwen-bobo-dpo-ju-49011-v1-uploader:   RE_XML_NAMESPACE = re.compile(b'^(<?[^>]+?>\s*|\s*)(<\w+) xmlns=[\'"](https?://[^\'"]+)[\'"]', re.MULTILINE)
chaiml-qwen-bobo-dpo-ju-49011-v1-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:240: SyntaxWarning: invalid escape sequence '\.'
chaiml-qwen-bobo-dpo-ju-49011-v1-uploader:   invalid = re.search("([^a-z0-9\.-])", bucket, re.UNICODE)
chaiml-qwen-bobo-dpo-ju-49011-v1-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:244: SyntaxWarning: invalid escape sequence '\.'
chaiml-qwen-bobo-dpo-ju-49011-v1-uploader:   invalid = re.search("([^A-Za-z0-9\._-])", bucket, re.UNICODE)
chaiml-qwen-bobo-dpo-ju-49011-v1-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:255: SyntaxWarning: invalid escape sequence '\.'
chaiml-qwen-bobo-dpo-ju-49011-v1-uploader:   if re.search("-\.", bucket, re.UNICODE):
chaiml-qwen-bobo-dpo-ju-49011-v1-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:257: SyntaxWarning: invalid escape sequence '\.'
chaiml-qwen-bobo-dpo-ju-49011-v1-uploader:   if re.search("\.\.", bucket, re.UNICODE):
chaiml-qwen-bobo-dpo-ju-49011-v1-uploader: /usr/lib/python3/dist-packages/S3/S3Uri.py:155: SyntaxWarning: invalid escape sequence '\w'
chaiml-qwen-bobo-dpo-ju-49011-v1-uploader:   _re = re.compile("^(\w+://)?(.*)", re.UNICODE)
chaiml-qwen-bobo-dpo-ju-49011-v1-uploader: /usr/lib/python3/dist-packages/S3/FileLists.py:480: SyntaxWarning: invalid escape sequence '\*'
chaiml-qwen-bobo-dpo-ju-49011-v1-uploader:   wildcard_split_result = re.split("\*|\?", uri_str, maxsplit=1)
chaiml-qwen-bobo-dpo-ju-49011-v1-uploader: Bucket 's3://guanaco-vllm-models/' created
chaiml-qwen-bobo-dpo-ju-49011-v1-uploader: uploading /dev/shm/model_output to s3://guanaco-vllm-models/chaiml-qwen-bobo-dpo-ju-49011-v1/default
chaiml-qwen-bobo-dpo-ju-49011-v1-uploader: cp /dev/shm/model_output/README.md s3://guanaco-vllm-models/chaiml-qwen-bobo-dpo-ju-49011-v1/default/README.md
chaiml-qwen-bobo-dpo-ju-49011-v1-uploader: cp /dev/shm/model_output/tokenizer_config.json s3://guanaco-vllm-models/chaiml-qwen-bobo-dpo-ju-49011-v1/default/tokenizer_config.json
chaiml-qwen-bobo-dpo-ju-49011-v1-uploader: cp /dev/shm/model_output/config.json s3://guanaco-vllm-models/chaiml-qwen-bobo-dpo-ju-49011-v1/default/config.json
chaiml-qwen-bobo-dpo-ju-49011-v1-uploader: cp /dev/shm/model_output/chat_template.jinja s3://guanaco-vllm-models/chaiml-qwen-bobo-dpo-ju-49011-v1/default/chat_template.jinja
chaiml-qwen-bobo-dpo-ju-49011-v1-uploader: cp /dev/shm/model_output/generation_config.json s3://guanaco-vllm-models/chaiml-qwen-bobo-dpo-ju-49011-v1/default/generation_config.json
chaiml-qwen-bobo-dpo-ju-49011-v1-uploader: cp /dev/shm/model_output/.gitattributes s3://guanaco-vllm-models/chaiml-qwen-bobo-dpo-ju-49011-v1/default/.gitattributes
chaiml-qwen-bobo-dpo-ju-49011-v1-uploader: cp /dev/shm/model_output/model.safetensors.index.json s3://guanaco-vllm-models/chaiml-qwen-bobo-dpo-ju-49011-v1/default/model.safetensors.index.json
chaiml-qwen-bobo-dpo-ju-49011-v1-uploader: cp /dev/shm/model_output/tokenizer.json s3://guanaco-vllm-models/chaiml-qwen-bobo-dpo-ju-49011-v1/default/tokenizer.json
chaiml-qwen-bobo-dpo-jud-2070-v1-uploader: cp /dev/shm/model_output/model-00002-of-00002.safetensors s3://guanaco-vllm-models/chaiml-qwen-bobo-dpo-jud-2070-v1/default/model-00002-of-00002.safetensors
chaiml-qwen-bobo-dpo-ju-49011-v1-uploader: cp /dev/shm/model_output/model-00002-of-00002.safetensors s3://guanaco-vllm-models/chaiml-qwen-bobo-dpo-ju-49011-v1/default/model-00002-of-00002.safetensors
2026-03-27T15:21:45.303613+00:00 monitor updated for chaiml-qwen-bobo-dpo-jud_2070_v1
2026-03-27T15:21:49.098636+00:00 monitor updated for chaiml-qwen-bobo-dpo-ju_49011_v1
chaiml-qwen-bobo-dpo-jud-2070-v1-uploader: cp /dev/shm/model_output/model-00001-of-00002.safetensors s3://guanaco-vllm-models/chaiml-qwen-bobo-dpo-jud-2070-v1/default/model-00001-of-00002.safetensors
chaiml-qwen-bobo-dpo-ju-49011-v1-uploader: cp /dev/shm/model_output/model-00001-of-00002.safetensors s3://guanaco-vllm-models/chaiml-qwen-bobo-dpo-ju-49011-v1/default/model-00001-of-00002.safetensors
Job chaiml-qwen-bobo-dpo-ju-49011-v1-uploader completed after 155.04s with status: succeeded
Stopping job with name chaiml-qwen-bobo-dpo-ju-49011-v1-uploader
Pipeline stage VLLMUploader completed in 155.78s
run pipeline stage %s
Running pipeline stage VLLMTemplater
Pipeline stage VLLMTemplater completed in 0.37s
run pipeline stage %s
Running pipeline stage VLLMDeployer
Creating inference service chaiml-qwen-bobo-dpo-ju-49011-v1
Waiting for inference service chaiml-qwen-bobo-dpo-ju-49011-v1 to be ready
Job chaiml-qwen-bobo-dpo-jud-2070-v1-uploader completed after 165.19s with status: succeeded
Stopping job with name chaiml-qwen-bobo-dpo-jud-2070-v1-uploader
Pipeline stage VLLMUploader completed in 165.82s
run pipeline stage %s
Running pipeline stage VLLMTemplater
Pipeline stage VLLMTemplater completed in 0.33s
run pipeline stage %s
Running pipeline stage VLLMDeployer
Creating inference service chaiml-qwen-bobo-dpo-jud-2070-v1
Waiting for inference service chaiml-qwen-bobo-dpo-jud-2070-v1 to be ready
2026-03-27T15:22:45.473535+00:00 monitor updated for chaiml-qwen-bobo-dpo-jud_2070_v1
2026-03-27T15:22:49.245429+00:00 monitor updated for chaiml-qwen-bobo-dpo-ju_49011_v1
2026-03-27T15:23:45.631923+00:00 monitor updated for chaiml-qwen-bobo-dpo-jud_2070_v1
2026-03-27T15:23:49.399284+00:00 monitor updated for chaiml-qwen-bobo-dpo-ju_49011_v1
2026-03-27T15:24:45.786813+00:00 monitor updated for chaiml-qwen-bobo-dpo-jud_2070_v1
2026-03-27T15:24:49.549528+00:00 monitor updated for chaiml-qwen-bobo-dpo-ju_49011_v1
Inference service chaiml-qwen-bobo-dpo-ju-49011-v1 ready after 160.43340802192688s
Pipeline stage VLLMDeployer completed in 161.26s
run pipeline stage %s
Running pipeline stage StressChecker
Inference service chaiml-qwen-bobo-dpo-jud-2070-v1 ready after 170.53378438949585s
HTTPConnectionPool(host='guanaco-submitter.guanaco-backend.kchai-google-us-east4.chaiverse.com', port=80): Read timed out. (read timeout=20)
Received unhealthy response to inference request!
Pipeline stage VLLMDeployer completed in 187.48s
run pipeline stage %s
Running pipeline stage StressChecker
2026-03-27T15:25:46.031522+00:00 monitor updated for chaiml-qwen-bobo-dpo-jud_2070_v1
2026-03-27T15:25:49.707153+00:00 monitor updated for chaiml-qwen-bobo-dpo-ju_49011_v1
HTTPConnectionPool(host='guanaco-submitter.guanaco-backend.kchai-google-us-east4.chaiverse.com', port=80): Read timed out. (read timeout=20)
Received unhealthy response to inference request!
HTTPConnectionPool(host='guanaco-submitter.guanaco-backend.kchai-google-us-east4.chaiverse.com', port=80): Read timed out. (read timeout=20)
Received unhealthy response to inference request!
HTTPConnectionPool(host='guanaco-submitter.guanaco-backend.kchai-google-us-east4.chaiverse.com', port=80): Read timed out. (read timeout=20)
Received unhealthy response to inference request!
HTTPConnectionPool(host='guanaco-submitter.guanaco-backend.kchai-google-us-east4.chaiverse.com', port=80): Read timed out. (read timeout=20)
Received unhealthy response to inference request!
HTTPConnectionPool(host='guanaco-submitter.guanaco-backend.kchai-google-us-east4.chaiverse.com', port=80): Read timed out. (read timeout=20)
Received unhealthy response to inference request!
HTTPConnectionPool(host='guanaco-submitter.guanaco-backend.kchai-google-us-east4.chaiverse.com', port=80): Read timed out. (read timeout=20)
Received unhealthy response to inference request!
2026-03-27T15:26:46.200755+00:00 monitor updated for chaiml-qwen-bobo-dpo-jud_2070_v1
2026-03-27T15:26:49.869056+00:00 monitor updated for chaiml-qwen-bobo-dpo-ju_49011_v1
HTTPConnectionPool(host='guanaco-submitter.guanaco-backend.kchai-google-us-east4.chaiverse.com', port=80): Read timed out. (read timeout=20)
Received unhealthy response to inference request!
HTTPConnectionPool(host='guanaco-submitter.guanaco-backend.kchai-google-us-east4.chaiverse.com', port=80): Read timed out. (read timeout=20)
Received unhealthy response to inference request!
Received healthy response to inference request in 9.772594690322876s
HTTPConnectionPool(host='guanaco-submitter.guanaco-backend.kchai-google-us-east4.chaiverse.com', port=80): Read timed out. (read timeout=20)
Received unhealthy response to inference request!
HTTPConnectionPool(host='guanaco-submitter.guanaco-backend.kchai-google-us-east4.chaiverse.com', port=80): Read timed out. (read timeout=20)
Received unhealthy response to inference request!
Received healthy response to inference request in 16.319266080856323s
2026-03-27T15:27:46.481221+00:00 monitor updated for chaiml-qwen-bobo-dpo-jud_2070_v1
2026-03-27T15:27:50.038201+00:00 monitor updated for chaiml-qwen-bobo-dpo-ju_49011_v1
HTTPConnectionPool(host='guanaco-submitter.guanaco-backend.kchai-google-us-east4.chaiverse.com', port=80): Read timed out. (read timeout=20)
Received unhealthy response to inference request!
HTTPConnectionPool(host='guanaco-submitter.guanaco-backend.kchai-google-us-east4.chaiverse.com', port=80): Read timed out. (read timeout=20)
Received unhealthy response to inference request!
Received healthy response to inference request in 2.647864580154419s
Received healthy response to inference request in 9.274441719055176s
HTTPConnectionPool(host='guanaco-submitter.guanaco-backend.kchai-google-us-east4.chaiverse.com', port=80): Read timed out. (read timeout=20)
Received unhealthy response to inference request!
Received healthy response to inference request in 2.5333142280578613s
Received healthy response to inference request in 2.4847559928894043s
Received healthy response to inference request in 15.521679639816284s
HTTPConnectionPool(host='guanaco-submitter.guanaco-backend.kchai-google-us-east4.chaiverse.com', port=80): Read timed out. (read timeout=20)
Received unhealthy response to inference request!
Received healthy response to inference request in 9.431349515914917s
Received healthy response to inference request in 2.5041210651397705s
Received healthy response to inference request in 2.62117075920105s
2026-03-27T15:28:46.690019+00:00 monitor updated for chaiml-qwen-bobo-dpo-jud_2070_v1
Received healthy response to inference request in 2.6502315998077393s
2026-03-27T15:28:50.210301+00:00 monitor updated for chaiml-qwen-bobo-dpo-ju_49011_v1
Received healthy response to inference request in 2.538382053375244s
HTTPConnectionPool(host='guanaco-submitter.guanaco-backend.kchai-google-us-east4.chaiverse.com', port=80): Read timed out. (read timeout=20)
Received unhealthy response to inference request!
Received healthy response to inference request in 2.5877065658569336s
Received healthy response to inference request in 2.9294488430023193s
HTTPConnectionPool(host='guanaco-submitter.guanaco-backend.kchai-google-us-east4.chaiverse.com', port=80): Read timed out. (read timeout=20)
Received unhealthy response to inference request!
Received healthy response to inference request in 2.55161190032959s
HTTPConnectionPool(host='guanaco-submitter.guanaco-backend.kchai-google-us-east4.chaiverse.com', port=80): Read timed out. (read timeout=20)
Received unhealthy response to inference request!
Received healthy response to inference request in 2.839338541030884s
Received healthy response to inference request in 2.5978705883026123s
Received healthy response to inference request in 2.6725850105285645s
Received healthy response to inference request in 2.5909957885742188s
Received healthy response to inference request in 2.573949098587036s
Received healthy response to inference request in 2.5905067920684814s
Received healthy response to inference request in 2.679887294769287s
Received healthy response to inference request in 2.611205816268921s
Received healthy response to inference request in 9.591144800186157s
Received healthy response to inference request in 2.577834367752075s
Received healthy response to inference request in 2.583381414413452s
HTTPConnectionPool(host='guanaco-submitter.guanaco-backend.kchai-google-us-east4.chaiverse.com', port=80): Read timed out. (read timeout=20)
Received unhealthy response to inference request!
2026-03-27T15:29:46.877514+00:00 monitor updated for chaiml-qwen-bobo-dpo-jud_2070_v1
Received healthy response to inference request in 2.6425023078918457s
2026-03-27T15:29:50.404722+00:00 monitor updated for chaiml-qwen-bobo-dpo-ju_49011_v1
Received healthy response to inference request in 2.6478192806243896s
Received healthy response to inference request in 2.5112454891204834s
Received healthy response to inference request in 9.451468229293823s
Received healthy response to inference request in 2.639399528503418s
Received healthy response to inference request in 2.761462450027466s
Received healthy response to inference request in 2.7464137077331543s
Received healthy response to inference request in 2.5992002487182617s
Received healthy response to inference request in 2.5991194248199463s
Received healthy response to inference request in 2.6796114444732666s
Received healthy response to inference request in 2.7299160957336426s
Received healthy response to inference request in 2.7610440254211426s
Received healthy response to inference request in 2.6360604763031006s
30 requests
9 failed requests
5th percentile: 2.5073270559310914
10th percentile: 2.535668396949768
20th percentile: 2.5908979892730715
30th percentile: 2.6145553588867188
40th percentile: 2.6412611961364747
50th percentile: 2.690073847770691
Received healthy response to inference request in 2.925809144973755s
60th percentile: 5.530209112167349
70th percentile: 16.939229965209947
80th percentile: 20.27625365257263
90th percentile: 20.296421480178832
95th percentile: 20.414889025688172
99th percentile: 20.774800729751586
mean time: 8.84443077246348
%s, retrying in %s seconds...
Received healthy response to inference request in 2.6721365451812744s
30 requests
10 failed requests
5th percentile: 2.561663639545441
10th percentile: 2.5774458408355714
20th percentile: 2.608804702758789
30th percentile: 2.6724504709243773
40th percentile: 2.7285813331604003
50th percentile: 2.8825738430023193
60th percentile: 9.507338857650756
70th percentile: 20.258756232261657
80th percentile: 20.280653715133667
90th percentile: 20.308494114875792
95th percentile: 20.312193620204926
99th percentile: 28.601778290271767
mean time: 10.06333930492401
%s, retrying in %s seconds...
Received healthy response to inference request in 2.6172897815704346s
Received healthy response to inference request in 2.524393081665039s
Received healthy response to inference request in 9.383843183517456s
Received healthy response to inference request in 2.5136706829071045s
Received healthy response to inference request in 2.784226179122925s
Received healthy response to inference request in 2.584750175476074s
Received healthy response to inference request in 2.5502078533172607s
Received healthy response to inference request in 2.4920499324798584s
Received healthy response to inference request in 2.5557777881622314s
Received healthy response to inference request in 2.520256519317627s
Received healthy response to inference request in 2.5603768825531006s
Received healthy response to inference request in 2.4908249378204346s
Received healthy response to inference request in 2.733280658721924s
Received healthy response to inference request in 2.516566753387451s
Received healthy response to inference request in 2.44136381149292s
Received healthy response to inference request in 2.4718689918518066s
Received healthy response to inference request in 2.706082344055176s
Received healthy response to inference request in 2.5327303409576416s
Received healthy response to inference request in 2.477259397506714s
Received healthy response to inference request in 2.5875868797302246s
Received healthy response to inference request in 2.519946336746216s
Received healthy response to inference request in 2.4761946201324463s
Received healthy response to inference request in 2.5798935890197754s
2026-03-27T15:30:47.067832+00:00 monitor updated for chaiml-qwen-bobo-dpo-jud_2070_v1
Received healthy response to inference request in 2.632201671600342s
Received healthy response to inference request in 2.508769989013672s
2026-03-27T15:30:50.581179+00:00 monitor updated for chaiml-qwen-bobo-dpo-ju_49011_v1
Received healthy response to inference request in 2.674485445022583s
Received healthy response to inference request in 2.5867228507995605s
Received healthy response to inference request in 2.5927574634552s
Received healthy response to inference request in 2.6882734298706055s
Received healthy response to inference request in 2.5783045291900635s
Received healthy response to inference request in 2.476433753967285s
Received healthy response to inference request in 2.6250195503234863s
Received healthy response to inference request in 2.6971895694732666s
Received healthy response to inference request in 2.6797282695770264s
Received healthy response to inference request in 2.5885987281799316s
Received healthy response to inference request in 2.6305904388427734s
Received healthy response to inference request in 2.570803642272949s
Received healthy response to inference request in 2.6679282188415527s
Received healthy response to inference request in 2.774118423461914s
Received healthy response to inference request in 2.7083959579467773s
Received healthy response to inference request in 2.662559747695923s
Received healthy response to inference request in 2.562493324279785s
Received healthy response to inference request in 2.7048392295837402s
Received healthy response to inference request in 2.6734509468078613s
Received healthy response to inference request in 2.6034905910491943s
Received healthy response to inference request in 2.5136260986328125s
Received healthy response to inference request in 2.6277732849121094s
Received healthy response to inference request in 2.550750970840454s
Received healthy response to inference request in 2.593698501586914s
Received healthy response to inference request in 2.637319326400757s
Received healthy response to inference request in 2.6641180515289307s
Received healthy response to inference request in 2.6720311641693115s
Received healthy response to inference request in 2.6144890785217285s
Received healthy response to inference request in 2.6135025024414062s
Received healthy response to inference request in 2.7721898555755615s
Received healthy response to inference request in 2.6102328300476074s
Received healthy response to inference request in 2.716033935546875s
Received healthy response to inference request in 2.7886087894439697s
Received healthy response to inference request in 2.776137113571167s
30 requests
0 failed requests
5th percentile: 2.482778263092041
10th percentile: 2.5139925718307494
20th percentile: 2.546712350845337
30th percentile: 2.572926235198975
40th percentile: 2.5933220863342283
50th percentile: 2.6158894300460815
60th percentile: 2.628900146484375
70th percentile: 2.6652611017227175
80th percentile: 2.684750461578369
90th percentile: 2.7216495275497437
95th percentile: 2.7743608474731447
99th percentile: 2.7818803501129152
Received healthy response to inference request in 2.6523687839508057s
mean time: 2.6176741758982343
30 requests
Pipeline stage StressChecker completed in 393.28s
0 failed requests
run pipeline stage %s
5th percentile: 2.476805293560028
Running pipeline stage OfflineFamilyFriendlyTriggerPipeline
10th percentile: 2.490570878982544
run_pipeline:run_in_cloud %s
20th percentile: 2.513661766052246
starting trigger_guanaco_pipeline args=%s
30th percentile: 2.541602635383606
40th percentile: 2.576257610321045
50th percentile: 2.587660789489746
triggered trigger_guanaco_pipeline args=%s
60th percentile: 2.6230292320251465
Pipeline stage OfflineFamilyFriendlyTriggerPipeline completed in 1.64s
70th percentile: 2.6654011726379396
Shutdown handler de-registered
80th percentile: 2.690056657791138
chaiml-qwen-bobo-dpo-ju_49011_v1 status is now deployed due to DeploymentManager action
90th percentile: 2.7373644351959228
95th percentile: 2.7820881247520446
99th percentile: 7.471225209236151
mean time: 2.8260083834330243
Pipeline stage StressChecker completed in 364.13s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyTriggerPipeline
run_pipeline:run_in_cloud %s
starting trigger_guanaco_pipeline args=%s
triggered trigger_guanaco_pipeline args=%s
Pipeline stage OfflineFamilyFriendlyTriggerPipeline completed in 0.78s
Shutdown handler de-registered
chaiml-qwen-bobo-dpo-jud_2070_v1 status is now deployed due to DeploymentManager action
chaiml-qwen-bobo-dpo-jud_2070_v1 status is now inactive due to auto deactivation removed underperforming models
chaiml-qwen-bobo-dpo-jud_2070_v1 status is now torndown due to DeploymentManager action