Shutdown handler not registered because Python interpreter is not running in the main thread
run pipeline %s
run pipeline stage %s
Running pipeline stage VLLMUploader
Starting job with name chaiml-bol-v4-opusdv1b-59944-v1-uploader
Waiting for job on chaiml-bol-v4-opusdv1b-59944-v1-uploader to finish
Stopping job with name chaiml-bol-v4-opusdv1b-59944-v1-uploader
%s, retrying in %s seconds...
Starting job with name chaiml-bol-v4-opusdv1b-59944-v1-uploader
Waiting for job on chaiml-bol-v4-opusdv1b-59944-v1-uploader to finish
HTTP Request: %s %s "%s %d %s"
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
chaiml-bol-v4-opusdv1b-59944-v1-uploader: bash: cannot set terminal process group (-1): Inappropriate ioctl for device
chaiml-bol-v4-opusdv1b-59944-v1-uploader: bash: no job control in this shell
chaiml-bol-v4-opusdv1b-59944-v1-uploader: /root/miniconda3/envs/nvidia/lib/python3.11/site-packages/mk1/__init__.py:1: UserWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html. The pkg_resources package is slated for removal as early as 2025-11-30. Refrain from using this package or pin to Setuptools<81.
chaiml-bol-v4-opusdv1b-59944-v1-uploader: __import__('pkg_resources').declare_namespace(__name__)
chaiml-bol-v4-opusdv1b-59944-v1-uploader: ╔═════════════════════════════════════════════════════════════════════╗
chaiml-bol-v4-opusdv1b-59944-v1-uploader: ║ ║
chaiml-bol-v4-opusdv1b-59944-v1-uploader: ║ ██████ ██████ █████ ████ ████ ║
chaiml-bol-v4-opusdv1b-59944-v1-uploader: ║ ░░██████ ██████ ░░███ ███░ ░░███ ║
chaiml-bol-v4-opusdv1b-59944-v1-uploader: ║ ░███░█████░███ ░███ ███ ░███ ║
chaiml-bol-v4-opusdv1b-59944-v1-uploader: ║ ░███░░███ ░███ ░███████ ░███ ║
chaiml-bol-v4-opusdv1b-59944-v1-uploader: ║ ░███ ░░░ ░███ ░███░░███ ░███ ║
chaiml-bol-v4-opusdv1b-59944-v1-uploader: ║ ░███ ░███ ░███ ░░███ ░███ ║
chaiml-bol-v4-opusdv1b-59944-v1-uploader: ║ █████ █████ █████ ░░████ █████ ║
chaiml-bol-v4-opusdv1b-59944-v1-uploader: ║ ░░░░░ ░░░░░ ░░░░░ ░░░░ ░░░░░ ║
chaiml-bol-v4-opusdv1b-59944-v1-uploader: ║ ║
chaiml-bol-v4-opusdv1b-59944-v1-uploader: ║ Version: 0.30.6+torch280 ║
chaiml-bol-v4-opusdv1b-59944-v1-uploader: ║ Features: FLYWHEEL, CUDA ║
chaiml-bol-v4-opusdv1b-59944-v1-uploader: ║ Copyright 2023-2025 MK ONE TECHNOLOGIES Inc. ║
chaiml-bol-v4-opusdv1b-59944-v1-uploader: ║ https://mk1.ai ║
chaiml-bol-v4-opusdv1b-59944-v1-uploader: ║ ║
chaiml-bol-v4-opusdv1b-59944-v1-uploader: ║ The license key for the current software has been verified as ║
chaiml-bol-v4-opusdv1b-59944-v1-uploader: ║ belonging to: ║
chaiml-bol-v4-opusdv1b-59944-v1-uploader: ║ ║
chaiml-bol-v4-opusdv1b-59944-v1-uploader: ║ Chai Research Corp. ║
chaiml-bol-v4-opusdv1b-59944-v1-uploader: ║ Account ID: 7997a29f-0ceb-4cc7-9adf-840c57b4ae6f ║
chaiml-bol-v4-opusdv1b-59944-v1-uploader: ║ Expiration: 2028-03-31 23:59:59 ║
chaiml-bol-v4-opusdv1b-59944-v1-uploader: ║ ║
chaiml-bol-v4-opusdv1b-59944-v1-uploader: ╚═════════════════════════════════════════════════════════════════════╝
chaiml-bol-v4-opusdv1b-59944-v1-uploader: Downloaded to shared memory in 111.623s
HTTP Request: %s %s "%s %d %s"
chaiml-bol-v4-opusdv1b-59944-v1-uploader: Processed model ChaiML/bol-v4-opusdv1b-lr5e6ep2r64g4b01-int4-mixed in 168.624s
chaiml-bol-v4-opusdv1b-59944-v1-uploader: creating bucket guanaco-vllm-models
chaiml-bol-v4-opusdv1b-59944-v1-uploader: Bucket 's3://guanaco-vllm-models/' created
chaiml-bol-v4-opusdv1b-59944-v1-uploader: uploading /dev/shm/model_cache to s3://guanaco-vllm-models/chaiml-bol-v4-opusdv1b-59944-v1
chaiml-bol-v4-opusdv1b-59944-v1-uploader: cp /dev/shm/model_cache/.gitattributes s3://guanaco-vllm-models/chaiml-bol-v4-opusdv1b-59944-v1/.gitattributes
chaiml-bol-v4-opusdv1b-59944-v1-uploader: cp /dev/shm/model_cache/added_tokens.json s3://guanaco-vllm-models/chaiml-bol-v4-opusdv1b-59944-v1/added_tokens.json
chaiml-bol-v4-opusdv1b-59944-v1-uploader: cp /dev/shm/model_cache/chat_template.jinja s3://guanaco-vllm-models/chaiml-bol-v4-opusdv1b-59944-v1/chat_template.jinja
chaiml-bol-v4-opusdv1b-59944-v1-uploader: cp /dev/shm/model_cache/special_tokens_map.json s3://guanaco-vllm-models/chaiml-bol-v4-opusdv1b-59944-v1/special_tokens_map.json
chaiml-bol-v4-opusdv1b-59944-v1-uploader: cp /dev/shm/model_cache/generation_config.json s3://guanaco-vllm-models/chaiml-bol-v4-opusdv1b-59944-v1/generation_config.json
chaiml-bol-v4-opusdv1b-59944-v1-uploader: cp /dev/shm/model_cache/tokenizer_config.json s3://guanaco-vllm-models/chaiml-bol-v4-opusdv1b-59944-v1/tokenizer_config.json
chaiml-bol-v4-opusdv1b-59944-v1-uploader: cp /dev/shm/model_cache/config.json s3://guanaco-vllm-models/chaiml-bol-v4-opusdv1b-59944-v1/config.json
chaiml-bol-v4-opusdv1b-59944-v1-uploader: cp /dev/shm/model_cache/merges.txt s3://guanaco-vllm-models/chaiml-bol-v4-opusdv1b-59944-v1/merges.txt
chaiml-bol-v4-opusdv1b-59944-v1-uploader: cp /dev/shm/model_cache/quantization_config.json s3://guanaco-vllm-models/chaiml-bol-v4-opusdv1b-59944-v1/quantization_config.json
chaiml-bol-v4-opusdv1b-59944-v1-uploader: cp /dev/shm/model_cache/vocab.json s3://guanaco-vllm-models/chaiml-bol-v4-opusdv1b-59944-v1/vocab.json
chaiml-bol-v4-opusdv1b-59944-v1-uploader: cp /dev/shm/model_cache/tokenizer.json s3://guanaco-vllm-models/chaiml-bol-v4-opusdv1b-59944-v1/tokenizer.json
chaiml-bol-v4-opusdv1b-59944-v1-uploader: cp /dev/shm/model_cache/model.safetensors.index.json s3://guanaco-vllm-models/chaiml-bol-v4-opusdv1b-59944-v1/model.safetensors.index.json
HTTP Request: %s %s "%s %d %s"
Unable to record family friendly update due to error: Invalid JSON input: Expecting value: line 1 column 1 (char 0)
chaiml-bol-v4-opusdv1b-59944-v1-uploader: cp /dev/shm/model_cache/model-00027-of-00027.safetensors s3://guanaco-vllm-models/chaiml-bol-v4-opusdv1b-59944-v1/model-00027-of-00027.safetensors
chaiml-bol-v4-opusdv1b-59944-v1-uploader: cp /dev/shm/model_cache/model-00015-of-00027.safetensors s3://guanaco-vllm-models/chaiml-bol-v4-opusdv1b-59944-v1/model-00015-of-00027.safetensors
chaiml-bol-v4-opusdv1b-59944-v1-uploader: cp /dev/shm/model_cache/model-00011-of-00027.safetensors s3://guanaco-vllm-models/chaiml-bol-v4-opusdv1b-59944-v1/model-00011-of-00027.safetensors
chaiml-bol-v4-opusdv1b-59944-v1-uploader: cp /dev/shm/model_cache/model-00021-of-00027.safetensors s3://guanaco-vllm-models/chaiml-bol-v4-opusdv1b-59944-v1/model-00021-of-00027.safetensors
chaiml-bol-v4-opusdv1b-59944-v1-uploader: cp /dev/shm/model_cache/model-00022-of-00027.safetensors s3://guanaco-vllm-models/chaiml-bol-v4-opusdv1b-59944-v1/model-00022-of-00027.safetensors
chaiml-bol-v4-opusdv1b-59944-v1-uploader: cp /dev/shm/model_cache/model-00024-of-00027.safetensors s3://guanaco-vllm-models/chaiml-bol-v4-opusdv1b-59944-v1/model-00024-of-00027.safetensors
chaiml-bol-v4-opusdv1b-59944-v1-uploader: cp /dev/shm/model_cache/model-00018-of-00027.safetensors s3://guanaco-vllm-models/chaiml-bol-v4-opusdv1b-59944-v1/model-00018-of-00027.safetensors
chaiml-bol-v4-opusdv1b-59944-v1-uploader: cp /dev/shm/model_cache/model-00007-of-00027.safetensors s3://guanaco-vllm-models/chaiml-bol-v4-opusdv1b-59944-v1/model-00007-of-00027.safetensors
chaiml-bol-v4-opusdv1b-59944-v1-uploader: cp /dev/shm/model_cache/model-00008-of-00027.safetensors s3://guanaco-vllm-models/chaiml-bol-v4-opusdv1b-59944-v1/model-00008-of-00027.safetensors
chaiml-bol-v4-opusdv1b-59944-v1-uploader: cp /dev/shm/model_cache/model-00013-of-00027.safetensors s3://guanaco-vllm-models/chaiml-bol-v4-opusdv1b-59944-v1/model-00013-of-00027.safetensors
chaiml-bol-v4-opusdv1b-59944-v1-uploader: cp /dev/shm/model_cache/model-00010-of-00027.safetensors s3://guanaco-vllm-models/chaiml-bol-v4-opusdv1b-59944-v1/model-00010-of-00027.safetensors
chaiml-bol-v4-opusdv1b-59944-v1-uploader: cp /dev/shm/model_cache/model-00026-of-00027.safetensors s3://guanaco-vllm-models/chaiml-bol-v4-opusdv1b-59944-v1/model-00026-of-00027.safetensors
chaiml-bol-v4-opusdv1b-59944-v1-uploader: cp /dev/shm/model_cache/model-00009-of-00027.safetensors s3://guanaco-vllm-models/chaiml-bol-v4-opusdv1b-59944-v1/model-00009-of-00027.safetensors
chaiml-bol-v4-opusdv1b-59944-v1-uploader: cp /dev/shm/model_cache/model-00023-of-00027.safetensors s3://guanaco-vllm-models/chaiml-bol-v4-opusdv1b-59944-v1/model-00023-of-00027.safetensors
chaiml-bol-v4-opusdv1b-59944-v1-uploader: cp /dev/shm/model_cache/model-00002-of-00027.safetensors s3://guanaco-vllm-models/chaiml-bol-v4-opusdv1b-59944-v1/model-00002-of-00027.safetensors
chaiml-bol-v4-opusdv1b-59944-v1-uploader: cp /dev/shm/model_cache/model-00012-of-00027.safetensors s3://guanaco-vllm-models/chaiml-bol-v4-opusdv1b-59944-v1/model-00012-of-00027.safetensors
chaiml-bol-v4-opusdv1b-59944-v1-uploader: cp /dev/shm/model_cache/model-00025-of-00027.safetensors s3://guanaco-vllm-models/chaiml-bol-v4-opusdv1b-59944-v1/model-00025-of-00027.safetensors
chaiml-bol-v4-opusdv1b-59944-v1-uploader: cp /dev/shm/model_cache/model-00004-of-00027.safetensors s3://guanaco-vllm-models/chaiml-bol-v4-opusdv1b-59944-v1/model-00004-of-00027.safetensors
chaiml-bol-v4-opusdv1b-59944-v1-uploader: cp /dev/shm/model_cache/model-00001-of-00027.safetensors s3://guanaco-vllm-models/chaiml-bol-v4-opusdv1b-59944-v1/model-00001-of-00027.safetensors
chaiml-bol-v4-opusdv1b-59944-v1-uploader: cp /dev/shm/model_cache/model-00016-of-00027.safetensors s3://guanaco-vllm-models/chaiml-bol-v4-opusdv1b-59944-v1/model-00016-of-00027.safetensors
chaiml-bol-v4-opusdv1b-59944-v1-uploader: cp /dev/shm/model_cache/model-00017-of-00027.safetensors s3://guanaco-vllm-models/chaiml-bol-v4-opusdv1b-59944-v1/model-00017-of-00027.safetensors
chaiml-bol-v4-opusdv1b-59944-v1-uploader: cp /dev/shm/model_cache/model-00020-of-00027.safetensors s3://guanaco-vllm-models/chaiml-bol-v4-opusdv1b-59944-v1/model-00020-of-00027.safetensors
chaiml-bol-v4-opusdv1b-59944-v1-uploader: cp /dev/shm/model_cache/model-00014-of-00027.safetensors s3://guanaco-vllm-models/chaiml-bol-v4-opusdv1b-59944-v1/model-00014-of-00027.safetensors
chaiml-bol-v4-opusdv1b-59944-v1-uploader: cp /dev/shm/model_cache/model-00019-of-00027.safetensors s3://guanaco-vllm-models/chaiml-bol-v4-opusdv1b-59944-v1/model-00019-of-00027.safetensors
chaiml-bol-v4-opusdv1b-59944-v1-uploader: cp /dev/shm/model_cache/model-00006-of-00027.safetensors s3://guanaco-vllm-models/chaiml-bol-v4-opusdv1b-59944-v1/model-00006-of-00027.safetensors
chaiml-bol-v4-opusdv1b-59944-v1-uploader: cp /dev/shm/model_cache/model-00005-of-00027.safetensors s3://guanaco-vllm-models/chaiml-bol-v4-opusdv1b-59944-v1/model-00005-of-00027.safetensors
Job chaiml-bol-v4-opusdv1b-59944-v1-uploader completed after 860.96s with status: succeeded
Stopping job with name chaiml-bol-v4-opusdv1b-59944-v1-uploader
Pipeline stage VLLMUploader completed in 861.79s
run pipeline stage %s
Running pipeline stage VLLMTemplater
Pipeline stage VLLMTemplater completed in 0.25s
run pipeline stage %s
Running pipeline stage VLLMDeployer
Creating inference service chaiml-bol-v4-opusdv1b-59944-v1
Waiting for inference service chaiml-bol-v4-opusdv1b-59944-v1 to be ready
HTTP Request: %s %s "%s %d %s"
HTTP Request: %s %s "%s %d %s"
Connection pool is full, discarding connection: %s. Connection pool size: %s
HTTP Request: %s %s "%s %d %s"
HTTP Request: %s %s "%s %d %s"
Inference service chaiml-bol-v4-opusdv1b-59944-v1 ready after 621.1327483654022s
Pipeline stage VLLMDeployer completed in 621.55s
run pipeline stage %s
Running pipeline stage StressChecker
Received healthy response to inference request in 2.6412811279296875s
Received healthy response to inference request in 1.7404284477233887s
Received healthy response to inference request in 1.8865060806274414s
Received healthy response to inference request in 1.7758922576904297s
Received healthy response to inference request in 1.8750498294830322s
Received healthy response to inference request in 2.042410135269165s
Received healthy response to inference request in 1.8684885501861572s
Received healthy response to inference request in 1.7883713245391846s
Received healthy response to inference request in 1.8026018142700195s
Received healthy response to inference request in 1.8607852458953857s
Received healthy response to inference request in 1.742957353591919s
Received healthy response to inference request in 2.117745876312256s
Received healthy response to inference request in 1.7816998958587646s
Received healthy response to inference request in 1.8105154037475586s
Received healthy response to inference request in 1.8213419914245605s
Received healthy response to inference request in 1.989112377166748s
Received healthy response to inference request in 1.7915661334991455s
Received healthy response to inference request in 1.9898083209991455s
Received healthy response to inference request in 1.719909906387329s
Received healthy response to inference request in 1.8767383098602295s
Received healthy response to inference request in 2.016273021697998s
Received healthy response to inference request in 1.759246826171875s
Received healthy response to inference request in 2.2750837802886963s
Received healthy response to inference request in 2.1000521183013916s
Received healthy response to inference request in 1.8143069744110107s
Received healthy response to inference request in 1.865621566772461s
Received healthy response to inference request in 2.5950441360473633s
Received healthy response to inference request in 1.9317376613616943s
HTTP Request: %s %s "%s %d %s"
Received healthy response to inference request in 1.7849526405334473s
Received healthy response to inference request in 1.901221513748169s
30 requests
0 failed requests
5th percentile: 1.7415664553642274
10th percentile: 1.7576178789138794
20th percentile: 1.7843020915985108
30th percentile: 1.7992911100387574
40th percentile: 1.8185279846191407
50th percentile: 1.867055058479309
60th percentile: 1.8806454181671142
70th percentile: 1.9489500761032104
80th percentile: 2.0215004444122315
90th percentile: 2.1334796667099
95th percentile: 2.4510619759559624
99th percentile: 2.6278724002838136
mean time: 1.932225020726522
Pipeline stage StressChecker completed in 62.49s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyTriggerPipeline
run_pipeline:run_in_cloud %s
starting trigger_guanaco_pipeline args=%s
triggered trigger_guanaco_pipeline args=%s
Pipeline stage OfflineFamilyFriendlyTriggerPipeline completed in 0.91s
Shutdown handler de-registered
chaiml-bol-v4-opusdv1b-_59944_v1 status is now deployed due to DeploymentManager action
Shutdown handler registered
run pipeline %s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyScorer
Evaluating %s Family Friendly Score with %s threads
%s, retrying in %s seconds...
Evaluating %s Family Friendly Score with %s threads
%s, retrying in %s seconds...
Evaluating %s Family Friendly Score with %s threads
Generating Leaderboard row for %s
Generated Leaderboard row for %s
Pipeline stage OfflineFamilyFriendlyScorer completed in 4977.12s
Shutdown handler de-registered
chaiml-bol-v4-opusdv1b-_59944_v1 status is now torndown due to DeploymentManager action