Shutdown handler not registered because Python interpreter is not running in the main thread
run pipeline %s
run pipeline stage %s
Running pipeline stage VLLMUploader
Starting job with name chaiml-opusd-v1b-q235-l-55309-v6-uploader
Waiting for job on chaiml-opusd-v1b-q235-l-55309-v6-uploader to finish
chaiml-opusd-v1b-q235-l-55309-v6-uploader: bash: cannot set terminal process group (-1): Inappropriate ioctl for device
chaiml-opusd-v1b-q235-l-55309-v6-uploader: bash: no job control in this shell
chaiml-opusd-v1b-q235-l-55309-v6-uploader: /root/miniconda3/envs/nvidia/lib/python3.11/site-packages/mk1/__init__.py:1: UserWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html. The pkg_resources package is slated for removal as early as 2025-11-30. Refrain from using this package or pin to Setuptools<81.
chaiml-opusd-v1b-q235-l-55309-v6-uploader: __import__('pkg_resources').declare_namespace(__name__)
chaiml-opusd-v1b-q235-l-55309-v6-uploader: ╔═════════════════════════════════════════════════════════════════════╗
chaiml-opusd-v1b-q235-l-55309-v6-uploader: ║ ║
chaiml-opusd-v1b-q235-l-55309-v6-uploader: ║ ██████ ██████ █████ ████ ████ ║
chaiml-opusd-v1b-q235-l-55309-v6-uploader: ║ ░░██████ ██████ ░░███ ███░ ░░███ ║
chaiml-opusd-v1b-q235-l-55309-v6-uploader: ║ ░███░█████░███ ░███ ███ ░███ ║
chaiml-opusd-v1b-q235-l-55309-v6-uploader: ║ ░███░░███ ░███ ░███████ ░███ ║
chaiml-opusd-v1b-q235-l-55309-v6-uploader: ║ ░███ ░░░ ░███ ░███░░███ ░███ ║
chaiml-opusd-v1b-q235-l-55309-v6-uploader: ║ ░███ ░███ ░███ ░░███ ░███ ║
chaiml-opusd-v1b-q235-l-55309-v6-uploader: ║ █████ █████ █████ ░░████ █████ ║
chaiml-opusd-v1b-q235-l-55309-v6-uploader: ║ ░░░░░ ░░░░░ ░░░░░ ░░░░ ░░░░░ ║
chaiml-opusd-v1b-q235-l-55309-v6-uploader: ║ ║
chaiml-opusd-v1b-q235-l-55309-v6-uploader: ║ Version: 0.30.6+torch280 ║
chaiml-opusd-v1b-q235-l-55309-v6-uploader: ║ Features: FLYWHEEL, CUDA ║
chaiml-opusd-v1b-q235-l-55309-v6-uploader: ║ Copyright 2023-2025 MK ONE TECHNOLOGIES Inc. ║
chaiml-opusd-v1b-q235-l-55309-v6-uploader: ║ https://mk1.ai ║
chaiml-opusd-v1b-q235-l-55309-v6-uploader: ║ ║
chaiml-opusd-v1b-q235-l-55309-v6-uploader: ║ The license key for the current software has been verified as ║
chaiml-opusd-v1b-q235-l-55309-v6-uploader: ║ belonging to: ║
chaiml-opusd-v1b-q235-l-55309-v6-uploader: ║ ║
chaiml-opusd-v1b-q235-l-55309-v6-uploader: ║ Chai Research Corp. ║
chaiml-opusd-v1b-q235-l-55309-v6-uploader: ║ Account ID: 7997a29f-0ceb-4cc7-9adf-840c57b4ae6f ║
chaiml-opusd-v1b-q235-l-55309-v6-uploader: ║ Expiration: 2028-03-31 23:59:59 ║
chaiml-opusd-v1b-q235-l-55309-v6-uploader: ║ ║
chaiml-opusd-v1b-q235-l-55309-v6-uploader: ╚═════════════════════════════════════════════════════════════════════╝
chaiml-opusd-v1b-q235-l-55309-v6-uploader: Downloaded to shared memory in 115.999s
chaiml-opusd-v1b-q235-l-55309-v6-uploader: Processed model ChaiML/opusd-v1b-q235-lr1e4ep1r64g8-int4-mixed in 181.810s
chaiml-opusd-v1b-q235-l-55309-v6-uploader: creating bucket guanaco-vllm-models
chaiml-opusd-v1b-q235-l-55309-v6-uploader: Bucket 's3://guanaco-vllm-models/' created
chaiml-opusd-v1b-q235-l-55309-v6-uploader: uploading /dev/shm/model_cache to s3://guanaco-vllm-models/chaiml-opusd-v1b-q235-l-55309-v6
chaiml-opusd-v1b-q235-l-55309-v6-uploader: cp /dev/shm/model_cache/chat_template.jinja s3://guanaco-vllm-models/chaiml-opusd-v1b-q235-l-55309-v6/chat_template.jinja
chaiml-opusd-v1b-q235-l-55309-v6-uploader: cp /dev/shm/model_cache/added_tokens.json s3://guanaco-vllm-models/chaiml-opusd-v1b-q235-l-55309-v6/added_tokens.json
chaiml-opusd-v1b-q235-l-55309-v6-uploader: cp /dev/shm/model_cache/README.md s3://guanaco-vllm-models/chaiml-opusd-v1b-q235-l-55309-v6/README.md
chaiml-opusd-v1b-q235-l-55309-v6-uploader: cp /dev/shm/model_cache/special_tokens_map.json s3://guanaco-vllm-models/chaiml-opusd-v1b-q235-l-55309-v6/special_tokens_map.json
chaiml-opusd-v1b-q235-l-55309-v6-uploader: cp /dev/shm/model_cache/generation_config.json s3://guanaco-vllm-models/chaiml-opusd-v1b-q235-l-55309-v6/generation_config.json
chaiml-opusd-v1b-q235-l-55309-v6-uploader: cp /dev/shm/model_cache/config.json s3://guanaco-vllm-models/chaiml-opusd-v1b-q235-l-55309-v6/config.json
chaiml-opusd-v1b-q235-l-55309-v6-uploader: cp /dev/shm/model_cache/quantization_config.json s3://guanaco-vllm-models/chaiml-opusd-v1b-q235-l-55309-v6/quantization_config.json
chaiml-opusd-v1b-q235-l-55309-v6-uploader: cp /dev/shm/model_cache/.gitattributes s3://guanaco-vllm-models/chaiml-opusd-v1b-q235-l-55309-v6/.gitattributes
chaiml-opusd-v1b-q235-l-55309-v6-uploader: cp /dev/shm/model_cache/tokenizer_config.json s3://guanaco-vllm-models/chaiml-opusd-v1b-q235-l-55309-v6/tokenizer_config.json
chaiml-opusd-v1b-q235-l-55309-v6-uploader: cp /dev/shm/model_cache/vocab.json s3://guanaco-vllm-models/chaiml-opusd-v1b-q235-l-55309-v6/vocab.json
chaiml-opusd-v1b-q235-l-55309-v6-uploader: cp /dev/shm/model_cache/merges.txt s3://guanaco-vllm-models/chaiml-opusd-v1b-q235-l-55309-v6/merges.txt
chaiml-opusd-v1b-q235-l-55309-v6-uploader: cp /dev/shm/model_cache/tokenizer.json s3://guanaco-vllm-models/chaiml-opusd-v1b-q235-l-55309-v6/tokenizer.json
chaiml-opusd-v1b-q235-l-55309-v6-uploader: cp /dev/shm/model_cache/model.safetensors.index.json s3://guanaco-vllm-models/chaiml-opusd-v1b-q235-l-55309-v6/model.safetensors.index.json
chaiml-opusd-v1b-q235-l-55309-v6-uploader: cp /dev/shm/model_cache/model-00027-of-00027.safetensors s3://guanaco-vllm-models/chaiml-opusd-v1b-q235-l-55309-v6/model-00027-of-00027.safetensors
chaiml-opusd-v1b-q235-l-55309-v6-uploader: cp /dev/shm/model_cache/model-00004-of-00027.safetensors s3://guanaco-vllm-models/chaiml-opusd-v1b-q235-l-55309-v6/model-00004-of-00027.safetensors
chaiml-opusd-v1b-q235-l-55309-v6-uploader: cp /dev/shm/model_cache/model-00018-of-00027.safetensors s3://guanaco-vllm-models/chaiml-opusd-v1b-q235-l-55309-v6/model-00018-of-00027.safetensors
chaiml-opusd-v1b-q235-l-55309-v6-uploader: cp /dev/shm/model_cache/model-00009-of-00027.safetensors s3://guanaco-vllm-models/chaiml-opusd-v1b-q235-l-55309-v6/model-00009-of-00027.safetensors
chaiml-opusd-v1b-q235-l-55309-v6-uploader: cp /dev/shm/model_cache/model-00002-of-00027.safetensors s3://guanaco-vllm-models/chaiml-opusd-v1b-q235-l-55309-v6/model-00002-of-00027.safetensors
chaiml-opusd-v1b-q235-l-55309-v6-uploader: cp /dev/shm/model_cache/model-00006-of-00027.safetensors s3://guanaco-vllm-models/chaiml-opusd-v1b-q235-l-55309-v6/model-00006-of-00027.safetensors
chaiml-opusd-v1b-q235-l-55309-v6-uploader: cp /dev/shm/model_cache/model-00024-of-00027.safetensors s3://guanaco-vllm-models/chaiml-opusd-v1b-q235-l-55309-v6/model-00024-of-00027.safetensors
chaiml-opusd-v1b-q235-l-55309-v6-uploader: cp /dev/shm/model_cache/model-00007-of-00027.safetensors s3://guanaco-vllm-models/chaiml-opusd-v1b-q235-l-55309-v6/model-00007-of-00027.safetensors
chaiml-opusd-v1b-q235-l-55309-v6-uploader: cp /dev/shm/model_cache/model-00026-of-00027.safetensors s3://guanaco-vllm-models/chaiml-opusd-v1b-q235-l-55309-v6/model-00026-of-00027.safetensors
chaiml-opusd-v1b-q235-l-55309-v6-uploader: cp /dev/shm/model_cache/model-00025-of-00027.safetensors s3://guanaco-vllm-models/chaiml-opusd-v1b-q235-l-55309-v6/model-00025-of-00027.safetensors
chaiml-opusd-v1b-q235-l-55309-v6-uploader: cp /dev/shm/model_cache/model-00023-of-00027.safetensors s3://guanaco-vllm-models/chaiml-opusd-v1b-q235-l-55309-v6/model-00023-of-00027.safetensors
chaiml-opusd-v1b-q235-l-55309-v6-uploader: cp /dev/shm/model_cache/model-00012-of-00027.safetensors s3://guanaco-vllm-models/chaiml-opusd-v1b-q235-l-55309-v6/model-00012-of-00027.safetensors
chaiml-opusd-v1b-q235-l-55309-v6-uploader: cp /dev/shm/model_cache/model-00020-of-00027.safetensors s3://guanaco-vllm-models/chaiml-opusd-v1b-q235-l-55309-v6/model-00020-of-00027.safetensors
chaiml-opusd-v1b-q235-l-55309-v6-uploader: cp /dev/shm/model_cache/model-00014-of-00027.safetensors s3://guanaco-vllm-models/chaiml-opusd-v1b-q235-l-55309-v6/model-00014-of-00027.safetensors
chaiml-opusd-v1b-q235-l-55309-v6-uploader: cp /dev/shm/model_cache/model-00022-of-00027.safetensors s3://guanaco-vllm-models/chaiml-opusd-v1b-q235-l-55309-v6/model-00022-of-00027.safetensors
chaiml-opusd-v1b-q235-l-55309-v6-uploader: cp /dev/shm/model_cache/model-00003-of-00027.safetensors s3://guanaco-vllm-models/chaiml-opusd-v1b-q235-l-55309-v6/model-00003-of-00027.safetensors
chaiml-opusd-v1b-q235-l-55309-v6-uploader: cp /dev/shm/model_cache/model-00008-of-00027.safetensors s3://guanaco-vllm-models/chaiml-opusd-v1b-q235-l-55309-v6/model-00008-of-00027.safetensors
chaiml-opusd-v1b-q235-l-55309-v6-uploader: cp /dev/shm/model_cache/model-00015-of-00027.safetensors s3://guanaco-vllm-models/chaiml-opusd-v1b-q235-l-55309-v6/model-00015-of-00027.safetensors
chaiml-opusd-v1b-q235-l-55309-v6-uploader: cp /dev/shm/model_cache/model-00005-of-00027.safetensors s3://guanaco-vllm-models/chaiml-opusd-v1b-q235-l-55309-v6/model-00005-of-00027.safetensors
chaiml-opusd-v1b-q235-l-55309-v6-uploader: cp /dev/shm/model_cache/model-00010-of-00027.safetensors s3://guanaco-vllm-models/chaiml-opusd-v1b-q235-l-55309-v6/model-00010-of-00027.safetensors
chaiml-opusd-v1b-q235-l-55309-v6-uploader: cp /dev/shm/model_cache/model-00011-of-00027.safetensors s3://guanaco-vllm-models/chaiml-opusd-v1b-q235-l-55309-v6/model-00011-of-00027.safetensors
chaiml-opusd-v1b-q235-l-55309-v6-uploader: cp /dev/shm/model_cache/model-00017-of-00027.safetensors s3://guanaco-vllm-models/chaiml-opusd-v1b-q235-l-55309-v6/model-00017-of-00027.safetensors
chaiml-opusd-v1b-q235-l-55309-v6-uploader: cp /dev/shm/model_cache/model-00016-of-00027.safetensors s3://guanaco-vllm-models/chaiml-opusd-v1b-q235-l-55309-v6/model-00016-of-00027.safetensors
chaiml-opusd-v1b-q235-l-55309-v6-uploader: cp /dev/shm/model_cache/model-00001-of-00027.safetensors s3://guanaco-vllm-models/chaiml-opusd-v1b-q235-l-55309-v6/model-00001-of-00027.safetensors
Job chaiml-opusd-v1b-q235-l-55309-v6-uploader completed after 718.57s with status: succeeded
Stopping job with name chaiml-opusd-v1b-q235-l-55309-v6-uploader
Pipeline stage VLLMUploader completed in 718.92s
run pipeline stage %s
Running pipeline stage VLLMTemplater
Pipeline stage VLLMTemplater completed in 0.19s
run pipeline stage %s
Running pipeline stage VLLMDeployer
Creating inference service chaiml-opusd-v1b-q235-l-55309-v6
Waiting for inference service chaiml-opusd-v1b-q235-l-55309-v6 to be ready
HTTP Request: %s %s "%s %d %s"
Inference service chaiml-opusd-v1b-q235-l-55309-v6 ready after 470.26659750938416s
Pipeline stage VLLMDeployer completed in 470.63s
run pipeline stage %s
Running pipeline stage StressChecker
Received healthy response to inference request in 1.8944640159606934s
Received healthy response to inference request in 1.8405125141143799s
Received healthy response to inference request in 1.7647440433502197s
Received healthy response to inference request in 1.7240095138549805s
Received healthy response to inference request in 1.8590781688690186s
Received healthy response to inference request in 1.8804500102996826s
Received healthy response to inference request in 1.7264223098754883s
Received healthy response to inference request in 2.2258758544921875s
Received healthy response to inference request in 1.7917232513427734s
Received healthy response to inference request in 1.8274893760681152s
Received healthy response to inference request in 1.834761619567871s
Received healthy response to inference request in 1.7750277519226074s
Received healthy response to inference request in 1.7444329261779785s
Received healthy response to inference request in 1.8549656867980957s
Received healthy response to inference request in 1.8424596786499023s
Received healthy response to inference request in 1.7813501358032227s
Received healthy response to inference request in 1.8041810989379883s
Received healthy response to inference request in 1.749964714050293s
Received healthy response to inference request in 1.795936107635498s
Received healthy response to inference request in 1.9996991157531738s
Received healthy response to inference request in 1.8889610767364502s
Received healthy response to inference request in 1.7277181148529053s
Received healthy response to inference request in 1.792468786239624s
Received healthy response to inference request in 1.7474355697631836s
Received healthy response to inference request in 2.054549217224121s
Received healthy response to inference request in 1.8245923519134521s
Received healthy response to inference request in 1.786820888519287s
Received healthy response to inference request in 1.832695484161377s
Received healthy response to inference request in 1.7481677532196045s
Received healthy response to inference request in 1.7547330856323242s
30 requests
0 failed requests
5th percentile: 1.727005422115326
10th percentile: 1.7427614450454711
20th percentile: 1.7496053218841552
30th percentile: 1.771942639350891
40th percentile: 1.789762306213379
50th percentile: 1.8000586032867432
60th percentile: 1.82957181930542
70th percentile: 1.8410966634750365
80th percentile: 1.8633525371551514
90th percentile: 1.9049875259399416
95th percentile: 2.0298666715621945
99th percentile: 2.1761911296844483
mean time: 1.82918967405955
Pipeline stage StressChecker completed in 57.62s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyTriggerPipeline
run_pipeline:run_in_cloud %s
starting trigger_guanaco_pipeline args=%s
triggered trigger_guanaco_pipeline args=%s
Pipeline stage OfflineFamilyFriendlyTriggerPipeline completed in 0.72s
Shutdown handler de-registered
chaiml-opusd-v1b-q235-l_55309_v6 status is now deployed due to DeploymentManager action
Shutdown handler registered
run pipeline %s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyScorer
Evaluating %s Family Friendly Score with %s threads
Generating Leaderboard row for %s
Generated Leaderboard row for %s
Pipeline stage OfflineFamilyFriendlyScorer completed in 3087.66s
Shutdown handler de-registered
chaiml-opusd-v1b-q235-l_55309_v6 status is now inactive due to auto deactivation removed underperforming models
chaiml-opusd-v1b-q235-l_55309_v6 status is now torndown due to DeploymentManager action