Shutdown handler not registered because Python interpreter is not running in the main thread
run pipeline %s
run pipeline stage %s
Running pipeline stage VLLMUploader
Starting job with name chaiml-bol-v1-opusdv1b-67993-v2-uploader
Waiting for job on chaiml-bol-v1-opusdv1b-67993-v2-uploader to finish
chaiml-bol-v1-opusdv1b-67993-v2-uploader: bash: cannot set terminal process group (-1): Inappropriate ioctl for device
chaiml-bol-v1-opusdv1b-67993-v2-uploader: bash: no job control in this shell
chaiml-bol-v1-opusdv1b-67993-v2-uploader: /root/miniconda3/envs/nvidia/lib/python3.11/site-packages/mk1/__init__.py:1: UserWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html. The pkg_resources package is slated for removal as early as 2025-11-30. Refrain from using this package or pin to Setuptools<81.
chaiml-bol-v1-opusdv1b-67993-v2-uploader: __import__('pkg_resources').declare_namespace(__name__)
chaiml-bol-v1-opusdv1b-67993-v2-uploader: ╔═════════════════════════════════════════════════════════════════════╗
chaiml-bol-v1-opusdv1b-67993-v2-uploader: ║ ║
chaiml-bol-v1-opusdv1b-67993-v2-uploader: ║ ██████ ██████ █████ ████ ████ ║
chaiml-bol-v1-opusdv1b-67993-v2-uploader: ║ ░░██████ ██████ ░░███ ███░ ░░███ ║
chaiml-bol-v1-opusdv1b-67993-v2-uploader: ║ ░███░█████░███ ░███ ███ ░███ ║
chaiml-bol-v1-opusdv1b-67993-v2-uploader: ║ ░███░░███ ░███ ░███████ ░███ ║
chaiml-bol-v1-opusdv1b-67993-v2-uploader: ║ ░███ ░░░ ░███ ░███░░███ ░███ ║
chaiml-bol-v1-opusdv1b-67993-v2-uploader: ║ ░███ ░███ ░███ ░░███ ░███ ║
chaiml-bol-v1-opusdv1b-67993-v2-uploader: ║ █████ █████ █████ ░░████ █████ ║
chaiml-bol-v1-opusdv1b-67993-v2-uploader: ║ ░░░░░ ░░░░░ ░░░░░ ░░░░ ░░░░░ ║
chaiml-bol-v1-opusdv1b-67993-v2-uploader: ║ ║
chaiml-bol-v1-opusdv1b-67993-v2-uploader: ║ Version: 0.30.6+torch280 ║
chaiml-bol-v1-opusdv1b-67993-v2-uploader: ║ Features: FLYWHEEL, CUDA ║
chaiml-bol-v1-opusdv1b-67993-v2-uploader: ║ Copyright 2023-2025 MK ONE TECHNOLOGIES Inc. ║
chaiml-bol-v1-opusdv1b-67993-v2-uploader: ║ https://mk1.ai ║
chaiml-bol-v1-opusdv1b-67993-v2-uploader: ║ ║
chaiml-bol-v1-opusdv1b-67993-v2-uploader: ║ The license key for the current software has been verified as ║
chaiml-bol-v1-opusdv1b-67993-v2-uploader: ║ belonging to: ║
chaiml-bol-v1-opusdv1b-67993-v2-uploader: ║ ║
chaiml-bol-v1-opusdv1b-67993-v2-uploader: ║ Chai Research Corp. ║
chaiml-bol-v1-opusdv1b-67993-v2-uploader: ║ Account ID: 7997a29f-0ceb-4cc7-9adf-840c57b4ae6f ║
chaiml-bol-v1-opusdv1b-67993-v2-uploader: ║ Expiration: 2028-03-31 23:59:59 ║
chaiml-bol-v1-opusdv1b-67993-v2-uploader: ║ ║
chaiml-bol-v1-opusdv1b-67993-v2-uploader: ╚═════════════════════════════════════════════════════════════════════╝
chaiml-bol-v1-opusdv1b-67993-v2-uploader: Downloaded to shared memory in 142.810s
HTTP Request: %s %s "%s %d %s"
chaiml-bol-v1-opusdv1b-67993-v2-uploader: Processed model ChaiML/bol-v1-opusdv1b-lr1e5ep2r64g4b01r02-int4-mixed in 207.146s
chaiml-bol-v1-opusdv1b-67993-v2-uploader: creating bucket guanaco-vllm-models
chaiml-bol-v1-opusdv1b-67993-v2-uploader: Bucket 's3://guanaco-vllm-models/' created
chaiml-bol-v1-opusdv1b-67993-v2-uploader: uploading /dev/shm/model_cache to s3://guanaco-vllm-models/chaiml-bol-v1-opusdv1b-67993-v2
chaiml-bol-v1-opusdv1b-67993-v2-uploader: cp /dev/shm/model_cache/added_tokens.json s3://guanaco-vllm-models/chaiml-bol-v1-opusdv1b-67993-v2/added_tokens.json
chaiml-bol-v1-opusdv1b-67993-v2-uploader: cp /dev/shm/model_cache/chat_template.jinja s3://guanaco-vllm-models/chaiml-bol-v1-opusdv1b-67993-v2/chat_template.jinja
chaiml-bol-v1-opusdv1b-67993-v2-uploader: cp /dev/shm/model_cache/tokenizer_config.json s3://guanaco-vllm-models/chaiml-bol-v1-opusdv1b-67993-v2/tokenizer_config.json
chaiml-bol-v1-opusdv1b-67993-v2-uploader: cp /dev/shm/model_cache/quantization_config.json s3://guanaco-vllm-models/chaiml-bol-v1-opusdv1b-67993-v2/quantization_config.json
chaiml-bol-v1-opusdv1b-67993-v2-uploader: cp /dev/shm/model_cache/generation_config.json s3://guanaco-vllm-models/chaiml-bol-v1-opusdv1b-67993-v2/generation_config.json
chaiml-bol-v1-opusdv1b-67993-v2-uploader: cp /dev/shm/model_cache/special_tokens_map.json s3://guanaco-vllm-models/chaiml-bol-v1-opusdv1b-67993-v2/special_tokens_map.json
chaiml-bol-v1-opusdv1b-67993-v2-uploader: cp /dev/shm/model_cache/.gitattributes s3://guanaco-vllm-models/chaiml-bol-v1-opusdv1b-67993-v2/.gitattributes
chaiml-bol-v1-opusdv1b-67993-v2-uploader: cp /dev/shm/model_cache/config.json s3://guanaco-vllm-models/chaiml-bol-v1-opusdv1b-67993-v2/config.json
chaiml-bol-v1-opusdv1b-67993-v2-uploader: cp /dev/shm/model_cache/vocab.json s3://guanaco-vllm-models/chaiml-bol-v1-opusdv1b-67993-v2/vocab.json
chaiml-bol-v1-opusdv1b-67993-v2-uploader: cp /dev/shm/model_cache/merges.txt s3://guanaco-vllm-models/chaiml-bol-v1-opusdv1b-67993-v2/merges.txt
chaiml-bol-v1-opusdv1b-67993-v2-uploader: cp /dev/shm/model_cache/model.safetensors.index.json s3://guanaco-vllm-models/chaiml-bol-v1-opusdv1b-67993-v2/model.safetensors.index.json
chaiml-bol-v1-opusdv1b-67993-v2-uploader: cp /dev/shm/model_cache/tokenizer.json s3://guanaco-vllm-models/chaiml-bol-v1-opusdv1b-67993-v2/tokenizer.json
chaiml-bol-v1-opusdv1b-67993-v2-uploader: cp /dev/shm/model_cache/model-00027-of-00027.safetensors s3://guanaco-vllm-models/chaiml-bol-v1-opusdv1b-67993-v2/model-00027-of-00027.safetensors
Failed to get response for submission chaiml-mistral-24b-2048_76638_v1: ('http://chaiml-mistral-24b-2048-76638-v1-predictor.tenant-chaiml-guanaco.kchai-coreweave-us-east-04a.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'read tcp 127.0.0.1:37620->127.0.0.1:8080: read: connection reset by peer\n')
chaiml-bol-v1-opusdv1b-67993-v2-uploader: cp /dev/shm/model_cache/model-00020-of-00027.safetensors s3://guanaco-vllm-models/chaiml-bol-v1-opusdv1b-67993-v2/model-00020-of-00027.safetensors
HTTP Request: %s %s "%s %d %s"
chaiml-bol-v1-opusdv1b-67993-v2-uploader: cp /dev/shm/model_cache/model-00025-of-00027.safetensors s3://guanaco-vllm-models/chaiml-bol-v1-opusdv1b-67993-v2/model-00025-of-00027.safetensors
chaiml-bol-v1-opusdv1b-67993-v2-uploader: cp /dev/shm/model_cache/model-00015-of-00027.safetensors s3://guanaco-vllm-models/chaiml-bol-v1-opusdv1b-67993-v2/model-00015-of-00027.safetensors
chaiml-bol-v1-opusdv1b-67993-v2-uploader: cp /dev/shm/model_cache/model-00003-of-00027.safetensors s3://guanaco-vllm-models/chaiml-bol-v1-opusdv1b-67993-v2/model-00003-of-00027.safetensors
chaiml-bol-v1-opusdv1b-67993-v2-uploader: cp /dev/shm/model_cache/model-00001-of-00027.safetensors s3://guanaco-vllm-models/chaiml-bol-v1-opusdv1b-67993-v2/model-00001-of-00027.safetensors
chaiml-bol-v1-opusdv1b-67993-v2-uploader: cp /dev/shm/model_cache/model-00008-of-00027.safetensors s3://guanaco-vllm-models/chaiml-bol-v1-opusdv1b-67993-v2/model-00008-of-00027.safetensors
chaiml-bol-v1-opusdv1b-67993-v2-uploader: cp /dev/shm/model_cache/model-00021-of-00027.safetensors s3://guanaco-vllm-models/chaiml-bol-v1-opusdv1b-67993-v2/model-00021-of-00027.safetensors
chaiml-bol-v1-opusdv1b-67993-v2-uploader: cp /dev/shm/model_cache/model-00009-of-00027.safetensors s3://guanaco-vllm-models/chaiml-bol-v1-opusdv1b-67993-v2/model-00009-of-00027.safetensors
chaiml-bol-v1-opusdv1b-67993-v2-uploader: cp /dev/shm/model_cache/model-00019-of-00027.safetensors s3://guanaco-vllm-models/chaiml-bol-v1-opusdv1b-67993-v2/model-00019-of-00027.safetensors
chaiml-bol-v1-opusdv1b-67993-v2-uploader: cp /dev/shm/model_cache/model-00007-of-00027.safetensors s3://guanaco-vllm-models/chaiml-bol-v1-opusdv1b-67993-v2/model-00007-of-00027.safetensors
chaiml-bol-v1-opusdv1b-67993-v2-uploader: cp /dev/shm/model_cache/model-00022-of-00027.safetensors s3://guanaco-vllm-models/chaiml-bol-v1-opusdv1b-67993-v2/model-00022-of-00027.safetensors
chaiml-bol-v1-opusdv1b-67993-v2-uploader: cp /dev/shm/model_cache/model-00005-of-00027.safetensors s3://guanaco-vllm-models/chaiml-bol-v1-opusdv1b-67993-v2/model-00005-of-00027.safetensors
chaiml-bol-v1-opusdv1b-67993-v2-uploader: cp /dev/shm/model_cache/model-00026-of-00027.safetensors s3://guanaco-vllm-models/chaiml-bol-v1-opusdv1b-67993-v2/model-00026-of-00027.safetensors
chaiml-bol-v1-opusdv1b-67993-v2-uploader: cp /dev/shm/model_cache/model-00018-of-00027.safetensors s3://guanaco-vllm-models/chaiml-bol-v1-opusdv1b-67993-v2/model-00018-of-00027.safetensors
chaiml-bol-v1-opusdv1b-67993-v2-uploader: cp /dev/shm/model_cache/model-00010-of-00027.safetensors s3://guanaco-vllm-models/chaiml-bol-v1-opusdv1b-67993-v2/model-00010-of-00027.safetensors
chaiml-bol-v1-opusdv1b-67993-v2-uploader: cp /dev/shm/model_cache/model-00004-of-00027.safetensors s3://guanaco-vllm-models/chaiml-bol-v1-opusdv1b-67993-v2/model-00004-of-00027.safetensors
chaiml-bol-v1-opusdv1b-67993-v2-uploader: cp /dev/shm/model_cache/model-00006-of-00027.safetensors s3://guanaco-vllm-models/chaiml-bol-v1-opusdv1b-67993-v2/model-00006-of-00027.safetensors
chaiml-bol-v1-opusdv1b-67993-v2-uploader: cp /dev/shm/model_cache/model-00016-of-00027.safetensors s3://guanaco-vllm-models/chaiml-bol-v1-opusdv1b-67993-v2/model-00016-of-00027.safetensors
chaiml-bol-v1-opusdv1b-67993-v2-uploader: cp /dev/shm/model_cache/model-00011-of-00027.safetensors s3://guanaco-vllm-models/chaiml-bol-v1-opusdv1b-67993-v2/model-00011-of-00027.safetensors
chaiml-bol-v1-opusdv1b-67993-v2-uploader: cp /dev/shm/model_cache/model-00017-of-00027.safetensors s3://guanaco-vllm-models/chaiml-bol-v1-opusdv1b-67993-v2/model-00017-of-00027.safetensors
chaiml-bol-v1-opusdv1b-67993-v2-uploader: cp /dev/shm/model_cache/model-00024-of-00027.safetensors s3://guanaco-vllm-models/chaiml-bol-v1-opusdv1b-67993-v2/model-00024-of-00027.safetensors
chaiml-bol-v1-opusdv1b-67993-v2-uploader: cp /dev/shm/model_cache/model-00002-of-00027.safetensors s3://guanaco-vllm-models/chaiml-bol-v1-opusdv1b-67993-v2/model-00002-of-00027.safetensors
chaiml-bol-v1-opusdv1b-67993-v2-uploader: cp /dev/shm/model_cache/model-00012-of-00027.safetensors s3://guanaco-vllm-models/chaiml-bol-v1-opusdv1b-67993-v2/model-00012-of-00027.safetensors
chaiml-bol-v1-opusdv1b-67993-v2-uploader: cp /dev/shm/model_cache/model-00014-of-00027.safetensors s3://guanaco-vllm-models/chaiml-bol-v1-opusdv1b-67993-v2/model-00014-of-00027.safetensors
chaiml-bol-v1-opusdv1b-67993-v2-uploader: cp /dev/shm/model_cache/model-00023-of-00027.safetensors s3://guanaco-vllm-models/chaiml-bol-v1-opusdv1b-67993-v2/model-00023-of-00027.safetensors
chaiml-bol-v1-opusdv1b-67993-v2-uploader: cp /dev/shm/model_cache/model-00013-of-00027.safetensors s3://guanaco-vllm-models/chaiml-bol-v1-opusdv1b-67993-v2/model-00013-of-00027.safetensors
Job chaiml-bol-v1-opusdv1b-67993-v2-uploader completed after 719.1s with status: succeeded
Stopping job with name chaiml-bol-v1-opusdv1b-67993-v2-uploader
Pipeline stage VLLMUploader completed in 719.47s
run pipeline stage %s
Running pipeline stage VLLMTemplater
Pipeline stage VLLMTemplater completed in 0.15s
run pipeline stage %s
Running pipeline stage VLLMDeployer
Creating inference service chaiml-bol-v1-opusdv1b-67993-v2
Waiting for inference service chaiml-bol-v1-opusdv1b-67993-v2 to be ready
HTTP Request: %s %s "%s %d %s"
HTTP Request: %s %s "%s %d %s"
Inference service chaiml-bol-v1-opusdv1b-67993-v2 ready after 480.2371275424957s
Pipeline stage VLLMDeployer completed in 480.79s
run pipeline stage %s
Running pipeline stage StressChecker
Received healthy response to inference request in 1.860461950302124s
Received healthy response to inference request in 1.755253553390503s
Received healthy response to inference request in 1.977715015411377s
Received healthy response to inference request in 2.0052285194396973s
Received healthy response to inference request in 1.7557446956634521s
Received healthy response to inference request in 1.7322072982788086s
Received healthy response to inference request in 1.8909459114074707s
Received healthy response to inference request in 1.896320104598999s
Received healthy response to inference request in 1.871532678604126s
Received healthy response to inference request in 1.7999804019927979s
Received healthy response to inference request in 1.7437982559204102s
Received healthy response to inference request in 1.7170886993408203s
Received healthy response to inference request in 1.8359546661376953s
Received healthy response to inference request in 1.7712743282318115s
Received healthy response to inference request in 1.7019317150115967s
Received healthy response to inference request in 1.7791438102722168s
Received healthy response to inference request in 1.7965636253356934s
Received healthy response to inference request in 1.7919151782989502s
Received healthy response to inference request in 1.787311315536499s
Received healthy response to inference request in 1.7974412441253662s
Received healthy response to inference request in 1.8844823837280273s
Received healthy response to inference request in 1.7458117008209229s
Received healthy response to inference request in 2.147956371307373s
Received healthy response to inference request in 1.7902414798736572s
Received healthy response to inference request in 1.8345832824707031s
Received healthy response to inference request in 1.7596380710601807s
Received healthy response to inference request in 1.772643804550171s
Received healthy response to inference request in 1.802586317062378s
Received healthy response to inference request in 2.1701767444610596s
Received healthy response to inference request in 1.7869977951049805s
30 requests
0 failed requests
5th percentile: 1.7238920688629151
10th percentile: 1.74263916015625
20th percentile: 1.7556464672088623
30th percentile: 1.772232961654663
40th percentile: 1.7871859073638916
50th percentile: 1.7942394018173218
60th percentile: 1.8010227680206299
70th percentile: 1.8433068513870239
80th percentile: 1.885775089263916
90th percentile: 1.980466365814209
95th percentile: 2.0837288379669188
99th percentile: 2.1637328362464907
mean time: 1.8320976972579956
Pipeline stage StressChecker completed in 58.19s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyTriggerPipeline
run_pipeline:run_in_cloud %s
starting trigger_guanaco_pipeline args=%s
triggered trigger_guanaco_pipeline args=%s
Pipeline stage OfflineFamilyFriendlyTriggerPipeline completed in 0.66s
Shutdown handler de-registered
chaiml-bol-v1-opusdv1b-_67993_v2 status is now deployed due to DeploymentManager action
Shutdown handler registered
run pipeline %s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyScorer
Evaluating %s Family Friendly Score with %s threads
%s, retrying in %s seconds...
Evaluating %s Family Friendly Score with %s threads
Generating Leaderboard row for %s
Generated Leaderboard row for %s
Pipeline stage OfflineFamilyFriendlyScorer completed in 5354.47s
Shutdown handler de-registered
chaiml-bol-v1-opusdv1b-_67993_v2 status is now inactive due to auto deactivation removed underperforming models
chaiml-bol-v1-opusdv1b-_67993_v2 status is now torndown due to DeploymentManager action