chaiml-bol-v3-opusdv1b-_38416

developer_uid: zonemercy
submission_id: chaiml-bol-v3-opusdv1b-_38416_v3
model_name: chaiml-bol-v3-opusdv1b-_38416_v3
model_group: ChaiML/bol-v3-opusdv1b-l
status: torndown
timestamp: 2026-02-04T15:21:01+00:00
num_battles: 12053
num_wins: 6568
celo_rating: 1335.73
family_friendly_score: 0.5678000000000001
family_friendly_standard_error: 0.007005757061160485
submission_type: basic
model_repo: ChaiML/bol-v3-opusdv1b-lr1e5ep2r64g4b01-int4-mixed
model_architecture: Qwen3MoeForCausalLM
model_num_parameters: 18790207488.0
best_of: 8
max_input_tokens: 2048
max_output_tokens: 72
reward_model: chaiml-prm-kimi-v1-300k_92220_v7
display_name: chaiml-bol-v3-opusdv1b-_38416_v3
ineligible_reason: max_output_tokens!=64
is_internal_developer: True
language_model: ChaiML/bol-v3-opusdv1b-lr1e5ep2r64g4b01-int4-mixed
model_size: 19B
ranking_group: single
us_pacific_date: 2026-02-01
win_ratio: 0.5449265742968555
generation_params: {'temperature': 1.0, 'top_p': 1.0, 'min_p': 0.0, 'top_k': 40, 'presence_penalty': 0.0, 'frequency_penalty': 0.0, 'stopping_words': ['####', '</s>', '<|user|>', '<|assistant|>', '</think>', '<|im_end|>'], 'max_input_tokens': 2048, 'best_of': 8, 'max_output_tokens': 72}
formatter: {'memory_template': "<|im_start|>system\n{bot_name}'s persona: {memory}<|im_end|>\n", 'prompt_template': '', 'bot_template': '<|im_start|>assistant\n{bot_name}: {message}<|im_end|>\n', 'user_template': '<|im_start|>user\n{message}<|im_end|>\n', 'response_template': '<|im_start|>assistant\n{bot_name}:', 'truncate_by_message': True}
Resubmit model
Shutdown handler not registered because Python interpreter is not running in the main thread
run pipeline %s
run pipeline stage %s
Running pipeline stage VLLMUploader
Starting job with name chaiml-bol-v3-opusdv1b-38416-v3-uploader
Waiting for job on chaiml-bol-v3-opusdv1b-38416-v3-uploader to finish
chaiml-bol-v3-opusdv1b-38416-v3-uploader: bash: cannot set terminal process group (-1): Inappropriate ioctl for device
chaiml-bol-v3-opusdv1b-38416-v3-uploader: bash: no job control in this shell
chaiml-bol-v3-opusdv1b-38416-v3-uploader: /root/miniconda3/envs/nvidia/lib/python3.11/site-packages/mk1/__init__.py:1: UserWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html. The pkg_resources package is slated for removal as early as 2025-11-30. Refrain from using this package or pin to Setuptools<81.
chaiml-bol-v3-opusdv1b-38416-v3-uploader:   __import__('pkg_resources').declare_namespace(__name__)
chaiml-bol-v3-opusdv1b-38416-v3-uploader: ╔═════════════════════════════════════════════════════════════════════╗
chaiml-bol-v3-opusdv1b-38416-v3-uploader: ║                                                                     ║
chaiml-bol-v3-opusdv1b-38416-v3-uploader: ║   ██████   ██████  █████   ████  ████                               ║
chaiml-bol-v3-opusdv1b-38416-v3-uploader: ║  ░░██████ ██████  ░░███   ███░  ░░███                               ║
chaiml-bol-v3-opusdv1b-38416-v3-uploader: ║   ░███░█████░███   ░███  ███     ░███                               ║
chaiml-bol-v3-opusdv1b-38416-v3-uploader: ║   ░███░░███ ░███   ░███████      ░███                               ║
chaiml-bol-v3-opusdv1b-38416-v3-uploader: ║   ░███ ░░░  ░███   ░███░░███     ░███                               ║
chaiml-bol-v3-opusdv1b-38416-v3-uploader: ║   ░███      ░███   ░███ ░░███    ░███                               ║
chaiml-bol-v3-opusdv1b-38416-v3-uploader: ║   █████     █████  █████ ░░████  █████                              ║
chaiml-bol-v3-opusdv1b-38416-v3-uploader: ║  ░░░░░     ░░░░░  ░░░░░   ░░░░  ░░░░░                               ║
chaiml-bol-v3-opusdv1b-38416-v3-uploader: ║                                                                     ║
chaiml-bol-v3-opusdv1b-38416-v3-uploader: ║  Version: 0.30.6+torch280                                           ║
chaiml-bol-v3-opusdv1b-38416-v3-uploader: ║  Features: FLYWHEEL, CUDA                                           ║
chaiml-bol-v3-opusdv1b-38416-v3-uploader: ║  Copyright 2023-2025 MK ONE TECHNOLOGIES Inc.                       ║
chaiml-bol-v3-opusdv1b-38416-v3-uploader: ║  https://mk1.ai                                                     ║
chaiml-bol-v3-opusdv1b-38416-v3-uploader: ║                                                                     ║
chaiml-bol-v3-opusdv1b-38416-v3-uploader: ║  The license key for the current software has been verified as      ║
chaiml-bol-v3-opusdv1b-38416-v3-uploader: ║  belonging to:                                                      ║
chaiml-bol-v3-opusdv1b-38416-v3-uploader: ║                                                                     ║
chaiml-bol-v3-opusdv1b-38416-v3-uploader: ║  Chai Research Corp.                                                ║
chaiml-bol-v3-opusdv1b-38416-v3-uploader: ║  Account ID: 7997a29f-0ceb-4cc7-9adf-840c57b4ae6f                   ║
chaiml-bol-v3-opusdv1b-38416-v3-uploader: ║  Expiration: 2028-03-31 23:59:59                                    ║
chaiml-bol-v3-opusdv1b-38416-v3-uploader: ║                                                                     ║
chaiml-bol-v3-opusdv1b-38416-v3-uploader: ╚═════════════════════════════════════════════════════════════════════╝
HTTP Request: %s %s "%s %d %s"
chaiml-bol-v3-opusdv1b-38416-v3-uploader: Downloaded to shared memory in 93.489s
chaiml-bol-v3-opusdv1b-38416-v3-uploader: Processed model ChaiML/bol-v3-opusdv1b-lr1e5ep2r64g4b01-int4-mixed in 158.321s
chaiml-bol-v3-opusdv1b-38416-v3-uploader: creating bucket guanaco-vllm-models
chaiml-bol-v3-opusdv1b-38416-v3-uploader: Bucket 's3://guanaco-vllm-models/' created
chaiml-bol-v3-opusdv1b-38416-v3-uploader: uploading /dev/shm/model_cache to s3://guanaco-vllm-models/chaiml-bol-v3-opusdv1b-38416-v3
chaiml-bol-v3-opusdv1b-38416-v3-uploader: cp /dev/shm/model_cache/.gitattributes s3://guanaco-vllm-models/chaiml-bol-v3-opusdv1b-38416-v3/.gitattributes
chaiml-bol-v3-opusdv1b-38416-v3-uploader: cp /dev/shm/model_cache/quantization_config.json s3://guanaco-vllm-models/chaiml-bol-v3-opusdv1b-38416-v3/quantization_config.json
chaiml-bol-v3-opusdv1b-38416-v3-uploader: cp /dev/shm/model_cache/tokenizer_config.json s3://guanaco-vllm-models/chaiml-bol-v3-opusdv1b-38416-v3/tokenizer_config.json
chaiml-bol-v3-opusdv1b-38416-v3-uploader: cp /dev/shm/model_cache/special_tokens_map.json s3://guanaco-vllm-models/chaiml-bol-v3-opusdv1b-38416-v3/special_tokens_map.json
chaiml-bol-v3-opusdv1b-38416-v3-uploader: cp /dev/shm/model_cache/generation_config.json s3://guanaco-vllm-models/chaiml-bol-v3-opusdv1b-38416-v3/generation_config.json
chaiml-bol-v3-opusdv1b-38416-v3-uploader: cp /dev/shm/model_cache/added_tokens.json s3://guanaco-vllm-models/chaiml-bol-v3-opusdv1b-38416-v3/added_tokens.json
chaiml-bol-v3-opusdv1b-38416-v3-uploader: cp /dev/shm/model_cache/config.json s3://guanaco-vllm-models/chaiml-bol-v3-opusdv1b-38416-v3/config.json
chaiml-bol-v3-opusdv1b-38416-v3-uploader: cp /dev/shm/model_cache/merges.txt s3://guanaco-vllm-models/chaiml-bol-v3-opusdv1b-38416-v3/merges.txt
chaiml-bol-v3-opusdv1b-38416-v3-uploader: cp /dev/shm/model_cache/chat_template.jinja s3://guanaco-vllm-models/chaiml-bol-v3-opusdv1b-38416-v3/chat_template.jinja
chaiml-bol-v3-opusdv1b-38416-v3-uploader: cp /dev/shm/model_cache/vocab.json s3://guanaco-vllm-models/chaiml-bol-v3-opusdv1b-38416-v3/vocab.json
chaiml-bol-v3-opusdv1b-38416-v3-uploader: cp /dev/shm/model_cache/model.safetensors.index.json s3://guanaco-vllm-models/chaiml-bol-v3-opusdv1b-38416-v3/model.safetensors.index.json
chaiml-bol-v3-opusdv1b-38416-v3-uploader: cp /dev/shm/model_cache/tokenizer.json s3://guanaco-vllm-models/chaiml-bol-v3-opusdv1b-38416-v3/tokenizer.json
HTTP Request: %s %s "%s %d %s"
chaiml-bol-v3-opusdv1b-38416-v3-uploader: DEBUG retryable error: RequestError: send request failed
chaiml-bol-v3-opusdv1b-38416-v3-uploader: caused by: Put "https://object.ord1.coreweave.com/guanaco-vllm-models/chaiml-bol-v3-opusdv1b-38416-v3/model-00024-of-00027.safetensors?partNumber=17&uploadId=2~Y-tBzwpppZszZmERJXIUXzZf1l3BQYM": write tcp 10.2.1.99:41888->216.153.53.63:443: write: connection reset by peer
HTTP Request: %s %s "%s %d %s"
chaiml-bol-v3-opusdv1b-38416-v3-uploader: ERROR "cp /dev/shm/model_cache/model-00024-of-00027.safetensors s3://guanaco-vllm-models/chaiml-bol-v3-opusdv1b-38416-v3/model-00024-of-00027.safetensors": MultipartUpload: upload multipart failed upload id: 2~Y-tBzwpppZszZmERJXIUXzZf1l3BQYM caused by: SignatureDoesNotMatch:  status code: 403, request id: tx00000d4d0d0715f09c017-00697f6915-14a26732ad-default, host id:
chaiml-bol-v3-opusdv1b-38416-v3-uploader: cp /dev/shm/model_cache/model-00027-of-00027.safetensors s3://guanaco-vllm-models/chaiml-bol-v3-opusdv1b-38416-v3/model-00027-of-00027.safetensors
HTTP Request: %s %s "%s %d %s"
HTTP Request: %s %s "%s %d %s"
chaiml-bol-v3-opusdv1b-38416-v3-uploader: cp /dev/shm/model_cache/model-00015-of-00027.safetensors s3://guanaco-vllm-models/chaiml-bol-v3-opusdv1b-38416-v3/model-00015-of-00027.safetensors
chaiml-bol-v3-opusdv1b-38416-v3-uploader: cp /dev/shm/model_cache/model-00025-of-00027.safetensors s3://guanaco-vllm-models/chaiml-bol-v3-opusdv1b-38416-v3/model-00025-of-00027.safetensors
chaiml-bol-v3-opusdv1b-38416-v3-uploader: cp /dev/shm/model_cache/model-00023-of-00027.safetensors s3://guanaco-vllm-models/chaiml-bol-v3-opusdv1b-38416-v3/model-00023-of-00027.safetensors
chaiml-bol-v3-opusdv1b-38416-v3-uploader: cp /dev/shm/model_cache/model-00020-of-00027.safetensors s3://guanaco-vllm-models/chaiml-bol-v3-opusdv1b-38416-v3/model-00020-of-00027.safetensors
chaiml-bol-v3-opusdv1b-38416-v3-uploader: cp /dev/shm/model_cache/model-00008-of-00027.safetensors s3://guanaco-vllm-models/chaiml-bol-v3-opusdv1b-38416-v3/model-00008-of-00027.safetensors
HTTP Request: %s %s "%s %d %s"
chaiml-bol-v3-opusdv1b-38416-v3-uploader: cp /dev/shm/model_cache/model-00011-of-00027.safetensors s3://guanaco-vllm-models/chaiml-bol-v3-opusdv1b-38416-v3/model-00011-of-00027.safetensors
chaiml-bol-v3-opusdv1b-38416-v3-uploader: cp /dev/shm/model_cache/model-00003-of-00027.safetensors s3://guanaco-vllm-models/chaiml-bol-v3-opusdv1b-38416-v3/model-00003-of-00027.safetensors
HTTP Request: %s %s "%s %d %s"
chaiml-bol-v3-opusdv1b-38416-v3-uploader: cp /dev/shm/model_cache/model-00022-of-00027.safetensors s3://guanaco-vllm-models/chaiml-bol-v3-opusdv1b-38416-v3/model-00022-of-00027.safetensors
chaiml-bol-v3-opusdv1b-38416-v3-uploader: cp /dev/shm/model_cache/model-00014-of-00027.safetensors s3://guanaco-vllm-models/chaiml-bol-v3-opusdv1b-38416-v3/model-00014-of-00027.safetensors
chaiml-bol-v3-opusdv1b-38416-v3-uploader: cp /dev/shm/model_cache/model-00012-of-00027.safetensors s3://guanaco-vllm-models/chaiml-bol-v3-opusdv1b-38416-v3/model-00012-of-00027.safetensors
chaiml-bol-v3-opusdv1b-38416-v3-uploader: cp /dev/shm/model_cache/model-00009-of-00027.safetensors s3://guanaco-vllm-models/chaiml-bol-v3-opusdv1b-38416-v3/model-00009-of-00027.safetensors
chaiml-bol-v3-opusdv1b-38416-v3-uploader: cp /dev/shm/model_cache/model-00007-of-00027.safetensors s3://guanaco-vllm-models/chaiml-bol-v3-opusdv1b-38416-v3/model-00007-of-00027.safetensors
chaiml-bol-v3-opusdv1b-38416-v3-uploader: cp /dev/shm/model_cache/model-00010-of-00027.safetensors s3://guanaco-vllm-models/chaiml-bol-v3-opusdv1b-38416-v3/model-00010-of-00027.safetensors
chaiml-bol-v3-opusdv1b-38416-v3-uploader: cp /dev/shm/model_cache/model-00004-of-00027.safetensors s3://guanaco-vllm-models/chaiml-bol-v3-opusdv1b-38416-v3/model-00004-of-00027.safetensors
chaiml-bol-v3-opusdv1b-38416-v3-uploader: cp /dev/shm/model_cache/model-00019-of-00027.safetensors s3://guanaco-vllm-models/chaiml-bol-v3-opusdv1b-38416-v3/model-00019-of-00027.safetensors
chaiml-bol-v3-opusdv1b-38416-v3-uploader: cp /dev/shm/model_cache/model-00026-of-00027.safetensors s3://guanaco-vllm-models/chaiml-bol-v3-opusdv1b-38416-v3/model-00026-of-00027.safetensors
chaiml-bol-v3-opusdv1b-38416-v3-uploader: cp /dev/shm/model_cache/model-00016-of-00027.safetensors s3://guanaco-vllm-models/chaiml-bol-v3-opusdv1b-38416-v3/model-00016-of-00027.safetensors
chaiml-bol-v3-opusdv1b-38416-v3-uploader: cp /dev/shm/model_cache/model-00013-of-00027.safetensors s3://guanaco-vllm-models/chaiml-bol-v3-opusdv1b-38416-v3/model-00013-of-00027.safetensors
chaiml-bol-v3-opusdv1b-38416-v3-uploader: cp /dev/shm/model_cache/model-00006-of-00027.safetensors s3://guanaco-vllm-models/chaiml-bol-v3-opusdv1b-38416-v3/model-00006-of-00027.safetensors
chaiml-bol-v3-opusdv1b-38416-v3-uploader: cp /dev/shm/model_cache/model-00017-of-00027.safetensors s3://guanaco-vllm-models/chaiml-bol-v3-opusdv1b-38416-v3/model-00017-of-00027.safetensors
chaiml-bol-v3-opusdv1b-38416-v3-uploader: cp /dev/shm/model_cache/model-00002-of-00027.safetensors s3://guanaco-vllm-models/chaiml-bol-v3-opusdv1b-38416-v3/model-00002-of-00027.safetensors
chaiml-bol-v3-opusdv1b-38416-v3-uploader: cp /dev/shm/model_cache/model-00021-of-00027.safetensors s3://guanaco-vllm-models/chaiml-bol-v3-opusdv1b-38416-v3/model-00021-of-00027.safetensors
chaiml-bol-v3-opusdv1b-38416-v3-uploader: cp /dev/shm/model_cache/model-00018-of-00027.safetensors s3://guanaco-vllm-models/chaiml-bol-v3-opusdv1b-38416-v3/model-00018-of-00027.safetensors
chaiml-bol-v3-opusdv1b-38416-v3-uploader: cp /dev/shm/model_cache/model-00001-of-00027.safetensors s3://guanaco-vllm-models/chaiml-bol-v3-opusdv1b-38416-v3/model-00001-of-00027.safetensors
Connection pool is full, discarding connection: %s. Connection pool size: %s
Job chaiml-bol-v3-opusdv1b-38416-v3-uploader completed after 800.1s with status: failed
Stopping job with name chaiml-bol-v3-opusdv1b-38416-v3-uploader
%s, retrying in %s seconds...
Starting job with name chaiml-bol-v3-opusdv1b-38416-v3-uploader
Waiting for job on chaiml-bol-v3-opusdv1b-38416-v3-uploader to finish
chaiml-bol-v3-opusdv1b-38416-v3-uploader: bash: cannot set terminal process group (-1): Inappropriate ioctl for device
chaiml-bol-v3-opusdv1b-38416-v3-uploader: bash: no job control in this shell
chaiml-bol-v3-opusdv1b-38416-v3-uploader: /root/miniconda3/envs/nvidia/lib/python3.11/site-packages/mk1/__init__.py:1: UserWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html. The pkg_resources package is slated for removal as early as 2025-11-30. Refrain from using this package or pin to Setuptools<81.
chaiml-bol-v3-opusdv1b-38416-v3-uploader:   __import__('pkg_resources').declare_namespace(__name__)
chaiml-bol-v3-opusdv1b-38416-v3-uploader: ╔═════════════════════════════════════════════════════════════════════╗
chaiml-bol-v3-opusdv1b-38416-v3-uploader: ║                                                                     ║
chaiml-bol-v3-opusdv1b-38416-v3-uploader: ║   ██████   ██████  █████   ████  ████                               ║
chaiml-bol-v3-opusdv1b-38416-v3-uploader: ║  ░░██████ ██████  ░░███   ███░  ░░███                               ║
chaiml-bol-v3-opusdv1b-38416-v3-uploader: ║   ░███░█████░███   ░███  ███     ░███                               ║
chaiml-bol-v3-opusdv1b-38416-v3-uploader: ║   ░███░░███ ░███   ░███████      ░███                               ║
chaiml-bol-v3-opusdv1b-38416-v3-uploader: ║   ░███ ░░░  ░███   ░███░░███     ░███                               ║
chaiml-bol-v3-opusdv1b-38416-v3-uploader: ║   ░███      ░███   ░███ ░░███    ░███                               ║
chaiml-bol-v3-opusdv1b-38416-v3-uploader: ║   █████     █████  █████ ░░████  █████                              ║
chaiml-bol-v3-opusdv1b-38416-v3-uploader: ║  ░░░░░     ░░░░░  ░░░░░   ░░░░  ░░░░░                               ║
chaiml-bol-v3-opusdv1b-38416-v3-uploader: ║                                                                     ║
chaiml-bol-v3-opusdv1b-38416-v3-uploader: ║  Version: 0.30.6+torch280                                           ║
chaiml-bol-v3-opusdv1b-38416-v3-uploader: ║  Features: FLYWHEEL, CUDA                                           ║
chaiml-bol-v3-opusdv1b-38416-v3-uploader: ║  Copyright 2023-2025 MK ONE TECHNOLOGIES Inc.                       ║
chaiml-bol-v3-opusdv1b-38416-v3-uploader: ║  https://mk1.ai                                                     ║
chaiml-bol-v3-opusdv1b-38416-v3-uploader: ║                                                                     ║
chaiml-bol-v3-opusdv1b-38416-v3-uploader: ║  The license key for the current software has been verified as      ║
chaiml-bol-v3-opusdv1b-38416-v3-uploader: ║  belonging to:                                                      ║
chaiml-bol-v3-opusdv1b-38416-v3-uploader: ║                                                                     ║
chaiml-bol-v3-opusdv1b-38416-v3-uploader: ║  Chai Research Corp.                                                ║
chaiml-bol-v3-opusdv1b-38416-v3-uploader: ║  Account ID: 7997a29f-0ceb-4cc7-9adf-840c57b4ae6f                   ║
chaiml-bol-v3-opusdv1b-38416-v3-uploader: ║  Expiration: 2028-03-31 23:59:59                                    ║
chaiml-bol-v3-opusdv1b-38416-v3-uploader: ║                                                                     ║
chaiml-bol-v3-opusdv1b-38416-v3-uploader: ╚═════════════════════════════════════════════════════════════════════╝
chaiml-bol-v3-opusdv1b-38416-v3-uploader: Downloaded to shared memory in 95.983s
HTTP Request: %s %s "%s %d %s"
HTTP Request: %s %s "%s %d %s"
chaiml-bol-v3-opusdv1b-38416-v3-uploader: Processed model ChaiML/bol-v3-opusdv1b-lr1e5ep2r64g4b01-int4-mixed in 161.768s
chaiml-bol-v3-opusdv1b-38416-v3-uploader: creating bucket guanaco-vllm-models
chaiml-bol-v3-opusdv1b-38416-v3-uploader: Bucket 's3://guanaco-vllm-models/' created
chaiml-bol-v3-opusdv1b-38416-v3-uploader: uploading /dev/shm/model_cache to s3://guanaco-vllm-models/chaiml-bol-v3-opusdv1b-38416-v3
chaiml-bol-v3-opusdv1b-38416-v3-uploader: DEBUG "sync /dev/shm/model_cache/.gitattributes s3://guanaco-vllm-models/chaiml-bol-v3-opusdv1b-38416-v3/.gitattributes": object size matches
chaiml-bol-v3-opusdv1b-38416-v3-uploader: DEBUG "sync /dev/shm/model_cache/added_tokens.json s3://guanaco-vllm-models/chaiml-bol-v3-opusdv1b-38416-v3/added_tokens.json": object size matches
chaiml-bol-v3-opusdv1b-38416-v3-uploader: DEBUG "sync /dev/shm/model_cache/chat_template.jinja s3://guanaco-vllm-models/chaiml-bol-v3-opusdv1b-38416-v3/chat_template.jinja": object size matches
chaiml-bol-v3-opusdv1b-38416-v3-uploader: DEBUG "sync /dev/shm/model_cache/config.json s3://guanaco-vllm-models/chaiml-bol-v3-opusdv1b-38416-v3/config.json": object size matches
chaiml-bol-v3-opusdv1b-38416-v3-uploader: DEBUG "sync /dev/shm/model_cache/generation_config.json s3://guanaco-vllm-models/chaiml-bol-v3-opusdv1b-38416-v3/generation_config.json": object size matches
chaiml-bol-v3-opusdv1b-38416-v3-uploader: DEBUG "sync /dev/shm/model_cache/merges.txt s3://guanaco-vllm-models/chaiml-bol-v3-opusdv1b-38416-v3/merges.txt": object size matches
chaiml-bol-v3-opusdv1b-38416-v3-uploader: DEBUG "sync /dev/shm/model_cache/model-00001-of-00027.safetensors s3://guanaco-vllm-models/chaiml-bol-v3-opusdv1b-38416-v3/model-00001-of-00027.safetensors": object size matches
chaiml-bol-v3-opusdv1b-38416-v3-uploader: DEBUG "sync /dev/shm/model_cache/model-00002-of-00027.safetensors s3://guanaco-vllm-models/chaiml-bol-v3-opusdv1b-38416-v3/model-00002-of-00027.safetensors": object size matches
chaiml-bol-v3-opusdv1b-38416-v3-uploader: DEBUG "sync /dev/shm/model_cache/model-00003-of-00027.safetensors s3://guanaco-vllm-models/chaiml-bol-v3-opusdv1b-38416-v3/model-00003-of-00027.safetensors": object size matches
chaiml-bol-v3-opusdv1b-38416-v3-uploader: DEBUG "sync /dev/shm/model_cache/model-00004-of-00027.safetensors s3://guanaco-vllm-models/chaiml-bol-v3-opusdv1b-38416-v3/model-00004-of-00027.safetensors": object size matches
chaiml-bol-v3-opusdv1b-38416-v3-uploader: DEBUG "sync /dev/shm/model_cache/model-00005-of-00027.safetensors s3://guanaco-vllm-models/chaiml-bol-v3-opusdv1b-38416-v3/model-00005-of-00027.safetensors": object size matches
chaiml-bol-v3-opusdv1b-38416-v3-uploader: DEBUG "sync /dev/shm/model_cache/model-00006-of-00027.safetensors s3://guanaco-vllm-models/chaiml-bol-v3-opusdv1b-38416-v3/model-00006-of-00027.safetensors": object size matches
chaiml-bol-v3-opusdv1b-38416-v3-uploader: DEBUG "sync /dev/shm/model_cache/model-00007-of-00027.safetensors s3://guanaco-vllm-models/chaiml-bol-v3-opusdv1b-38416-v3/model-00007-of-00027.safetensors": object size matches
chaiml-bol-v3-opusdv1b-38416-v3-uploader: DEBUG "sync /dev/shm/model_cache/model-00008-of-00027.safetensors s3://guanaco-vllm-models/chaiml-bol-v3-opusdv1b-38416-v3/model-00008-of-00027.safetensors": object size matches
chaiml-bol-v3-opusdv1b-38416-v3-uploader: DEBUG "sync /dev/shm/model_cache/model-00009-of-00027.safetensors s3://guanaco-vllm-models/chaiml-bol-v3-opusdv1b-38416-v3/model-00009-of-00027.safetensors": object size matches
chaiml-bol-v3-opusdv1b-38416-v3-uploader: DEBUG "sync /dev/shm/model_cache/model-00010-of-00027.safetensors s3://guanaco-vllm-models/chaiml-bol-v3-opusdv1b-38416-v3/model-00010-of-00027.safetensors": object size matches
chaiml-bol-v3-opusdv1b-38416-v3-uploader: DEBUG "sync /dev/shm/model_cache/model-00011-of-00027.safetensors s3://guanaco-vllm-models/chaiml-bol-v3-opusdv1b-38416-v3/model-00011-of-00027.safetensors": object size matches
chaiml-bol-v3-opusdv1b-38416-v3-uploader: DEBUG "sync /dev/shm/model_cache/model-00012-of-00027.safetensors s3://guanaco-vllm-models/chaiml-bol-v3-opusdv1b-38416-v3/model-00012-of-00027.safetensors": object size matches
chaiml-bol-v3-opusdv1b-38416-v3-uploader: DEBUG "sync /dev/shm/model_cache/model-00013-of-00027.safetensors s3://guanaco-vllm-models/chaiml-bol-v3-opusdv1b-38416-v3/model-00013-of-00027.safetensors": object size matches
chaiml-bol-v3-opusdv1b-38416-v3-uploader: DEBUG "sync /dev/shm/model_cache/model-00014-of-00027.safetensors s3://guanaco-vllm-models/chaiml-bol-v3-opusdv1b-38416-v3/model-00014-of-00027.safetensors": object size matches
chaiml-bol-v3-opusdv1b-38416-v3-uploader: DEBUG "sync /dev/shm/model_cache/model-00015-of-00027.safetensors s3://guanaco-vllm-models/chaiml-bol-v3-opusdv1b-38416-v3/model-00015-of-00027.safetensors": object size matches
chaiml-bol-v3-opusdv1b-38416-v3-uploader: DEBUG "sync /dev/shm/model_cache/model-00016-of-00027.safetensors s3://guanaco-vllm-models/chaiml-bol-v3-opusdv1b-38416-v3/model-00016-of-00027.safetensors": object size matches
chaiml-bol-v3-opusdv1b-38416-v3-uploader: DEBUG "sync /dev/shm/model_cache/model-00017-of-00027.safetensors s3://guanaco-vllm-models/chaiml-bol-v3-opusdv1b-38416-v3/model-00017-of-00027.safetensors": object size matches
chaiml-bol-v3-opusdv1b-38416-v3-uploader: DEBUG "sync /dev/shm/model_cache/model-00018-of-00027.safetensors s3://guanaco-vllm-models/chaiml-bol-v3-opusdv1b-38416-v3/model-00018-of-00027.safetensors": object size matches
chaiml-bol-v3-opusdv1b-38416-v3-uploader: DEBUG "sync /dev/shm/model_cache/model-00019-of-00027.safetensors s3://guanaco-vllm-models/chaiml-bol-v3-opusdv1b-38416-v3/model-00019-of-00027.safetensors": object size matches
chaiml-bol-v3-opusdv1b-38416-v3-uploader: DEBUG "sync /dev/shm/model_cache/model-00020-of-00027.safetensors s3://guanaco-vllm-models/chaiml-bol-v3-opusdv1b-38416-v3/model-00020-of-00027.safetensors": object size matches
chaiml-bol-v3-opusdv1b-38416-v3-uploader: DEBUG "sync /dev/shm/model_cache/model-00021-of-00027.safetensors s3://guanaco-vllm-models/chaiml-bol-v3-opusdv1b-38416-v3/model-00021-of-00027.safetensors": object size matches
chaiml-bol-v3-opusdv1b-38416-v3-uploader: DEBUG "sync /dev/shm/model_cache/model-00022-of-00027.safetensors s3://guanaco-vllm-models/chaiml-bol-v3-opusdv1b-38416-v3/model-00022-of-00027.safetensors": object size matches
chaiml-bol-v3-opusdv1b-38416-v3-uploader: DEBUG "sync /dev/shm/model_cache/model-00023-of-00027.safetensors s3://guanaco-vllm-models/chaiml-bol-v3-opusdv1b-38416-v3/model-00023-of-00027.safetensors": object size matches
chaiml-bol-v3-opusdv1b-38416-v3-uploader: DEBUG "sync /dev/shm/model_cache/model-00025-of-00027.safetensors s3://guanaco-vllm-models/chaiml-bol-v3-opusdv1b-38416-v3/model-00025-of-00027.safetensors": object size matches
chaiml-bol-v3-opusdv1b-38416-v3-uploader: DEBUG "sync /dev/shm/model_cache/model-00026-of-00027.safetensors s3://guanaco-vllm-models/chaiml-bol-v3-opusdv1b-38416-v3/model-00026-of-00027.safetensors": object size matches
chaiml-bol-v3-opusdv1b-38416-v3-uploader: DEBUG "sync /dev/shm/model_cache/model-00027-of-00027.safetensors s3://guanaco-vllm-models/chaiml-bol-v3-opusdv1b-38416-v3/model-00027-of-00027.safetensors": object size matches
chaiml-bol-v3-opusdv1b-38416-v3-uploader: DEBUG "sync /dev/shm/model_cache/model.safetensors.index.json s3://guanaco-vllm-models/chaiml-bol-v3-opusdv1b-38416-v3/model.safetensors.index.json": object size matches
chaiml-bol-v3-opusdv1b-38416-v3-uploader: DEBUG "sync /dev/shm/model_cache/quantization_config.json s3://guanaco-vllm-models/chaiml-bol-v3-opusdv1b-38416-v3/quantization_config.json": object size matches
chaiml-bol-v3-opusdv1b-38416-v3-uploader: DEBUG "sync /dev/shm/model_cache/special_tokens_map.json s3://guanaco-vllm-models/chaiml-bol-v3-opusdv1b-38416-v3/special_tokens_map.json": object size matches
chaiml-bol-v3-opusdv1b-38416-v3-uploader: DEBUG "sync /dev/shm/model_cache/tokenizer.json s3://guanaco-vllm-models/chaiml-bol-v3-opusdv1b-38416-v3/tokenizer.json": object size matches
chaiml-bol-v3-opusdv1b-38416-v3-uploader: DEBUG "sync /dev/shm/model_cache/tokenizer_config.json s3://guanaco-vllm-models/chaiml-bol-v3-opusdv1b-38416-v3/tokenizer_config.json": object size matches
chaiml-bol-v3-opusdv1b-38416-v3-uploader: DEBUG "sync /dev/shm/model_cache/vocab.json s3://guanaco-vllm-models/chaiml-bol-v3-opusdv1b-38416-v3/vocab.json": object size matches
chaiml-bol-v3-opusdv1b-38416-v3-uploader: cp /dev/shm/model_cache/model-00024-of-00027.safetensors s3://guanaco-vllm-models/chaiml-bol-v3-opusdv1b-38416-v3/model-00024-of-00027.safetensors
Job chaiml-bol-v3-opusdv1b-38416-v3-uploader completed after 230.67s with status: succeeded
Stopping job with name chaiml-bol-v3-opusdv1b-38416-v3-uploader
Pipeline stage VLLMUploader completed in 1031.51s
run pipeline stage %s
Running pipeline stage VLLMTemplater
Pipeline stage VLLMTemplater completed in 0.15s
run pipeline stage %s
Running pipeline stage VLLMDeployer
Creating inference service chaiml-bol-v3-opusdv1b-38416-v3
Waiting for inference service chaiml-bol-v3-opusdv1b-38416-v3 to be ready
HTTP Request: %s %s "%s %d %s"
HTTP Request: %s %s "%s %d %s"
Failed to get response for submission blend_hehus_2026-01-30: ('http://chaiml-csfs-v3-3-dpo-lr-86358-v1-predictor.creator-studio.kchai-coreweave-us-east-04a.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'read tcp 127.0.0.1:56442->127.0.0.1:8080: read: connection reset by peer\n')
HTTP Request: %s %s "%s %d %s"
HTTP Request: %s %s "%s %d %s"
Inference service chaiml-bol-v3-opusdv1b-38416-v3 ready after 521.5897309780121s
Pipeline stage VLLMDeployer completed in 522.04s
run pipeline stage %s
Running pipeline stage StressChecker
Received healthy response to inference request in 2.539266347885132s
Received healthy response to inference request in 1.7710764408111572s
Received healthy response to inference request in 1.8647921085357666s
Received healthy response to inference request in 1.832972764968872s
Received healthy response to inference request in 1.8364920616149902s
Received healthy response to inference request in 1.876678705215454s
Received healthy response to inference request in 1.776397705078125s
Received healthy response to inference request in 2.1057896614074707s
Received healthy response to inference request in 1.8031940460205078s
Received healthy response to inference request in 1.7761194705963135s
Received healthy response to inference request in 1.9427571296691895s
Received healthy response to inference request in 2.1243021488189697s
Received healthy response to inference request in 1.7707431316375732s
Received healthy response to inference request in 1.9397265911102295s
Received healthy response to inference request in 1.8481559753417969s
Received healthy response to inference request in 2.076263904571533s
Received healthy response to inference request in 1.8597314357757568s
Received healthy response to inference request in 1.9914841651916504s
Received healthy response to inference request in 1.741926908493042s
Received healthy response to inference request in 1.749054193496704s
Received healthy response to inference request in 1.916602611541748s
Received healthy response to inference request in 2.213923454284668s
Received healthy response to inference request in 1.7141237258911133s
Received healthy response to inference request in 1.9990928173065186s
Received healthy response to inference request in 2.2108707427978516s
Received healthy response to inference request in 2.060911178588867s
Received healthy response to inference request in 1.8526861667633057s
Received healthy response to inference request in 2.198488712310791s
Received healthy response to inference request in 2.032862424850464s
Received healthy response to inference request in 1.9786391258239746s
30 requests
0 failed requests
5th percentile: 1.74513418674469
10th percentile: 1.7685742378234863
20th percentile: 1.7763420581817626
30th percentile: 1.8354362726211548
40th percentile: 1.8569133281707764
50th percentile: 1.896640658378601
60th percentile: 1.9571099281311035
70th percentile: 2.009223699569702
80th percentile: 2.0821690559387207
90th percentile: 2.199726915359497
95th percentile: 2.2125497341156004
99th percentile: 2.4449169087409977
mean time: 1.9468375285466513
Pipeline stage StressChecker completed in 61.57s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyTriggerPipeline
run_pipeline:run_in_cloud %s
starting trigger_guanaco_pipeline args=%s
triggered trigger_guanaco_pipeline args=%s
Pipeline stage OfflineFamilyFriendlyTriggerPipeline completed in 0.79s
Shutdown handler de-registered
chaiml-bol-v3-opusdv1b-_38416_v3 status is now deployed due to DeploymentManager action
Shutdown handler registered
run pipeline %s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyScorer
Evaluating %s Family Friendly Score with %s threads
%s, retrying in %s seconds...
Evaluating %s Family Friendly Score with %s threads
%s, retrying in %s seconds...
Evaluating %s Family Friendly Score with %s threads
Generating Leaderboard row for %s
Generated Leaderboard row for %s
Pipeline stage OfflineFamilyFriendlyScorer completed in 6164.99s
Shutdown handler de-registered
chaiml-bol-v3-opusdv1b-_38416_v3 status is now torndown due to DeploymentManager action