Shutdown handler not registered because Python interpreter is not running in the main thread
run pipeline %s
run pipeline stage %s
Running pipeline stage MKMLizer
Starting job with name sao10k-14b-qwen2-5-kunou-v1-v2-mkmlizer
Waiting for job on sao10k-14b-qwen2-5-kunou-v1-v2-mkmlizer to finish
sao10k-14b-qwen2-5-kunou-v1-v2-mkmlizer: ╔═════════════════════════════════════════════════════════════════════╗
sao10k-14b-qwen2-5-kunou-v1-v2-mkmlizer: ║ _____ __ __ ║
sao10k-14b-qwen2-5-kunou-v1-v2-mkmlizer: ║ / _/ /_ ___ __/ / ___ ___ / / ║
sao10k-14b-qwen2-5-kunou-v1-v2-mkmlizer: ║ / _/ / // / |/|/ / _ \/ -_) -_) / ║
sao10k-14b-qwen2-5-kunou-v1-v2-mkmlizer: ║ /_//_/\_, /|__,__/_//_/\__/\__/_/ ║
sao10k-14b-qwen2-5-kunou-v1-v2-mkmlizer: ║ /___/ ║
sao10k-14b-qwen2-5-kunou-v1-v2-mkmlizer: ║ ║
sao10k-14b-qwen2-5-kunou-v1-v2-mkmlizer: ║ Version: 0.11.12 ║
sao10k-14b-qwen2-5-kunou-v1-v2-mkmlizer: ║ Copyright 2023 MK ONE TECHNOLOGIES Inc. ║
sao10k-14b-qwen2-5-kunou-v1-v2-mkmlizer: ║ https://mk1.ai ║
sao10k-14b-qwen2-5-kunou-v1-v2-mkmlizer: ║ ║
sao10k-14b-qwen2-5-kunou-v1-v2-mkmlizer: ║ The license key for the current software has been verified as ║
sao10k-14b-qwen2-5-kunou-v1-v2-mkmlizer: ║ belonging to: ║
sao10k-14b-qwen2-5-kunou-v1-v2-mkmlizer: ║ ║
sao10k-14b-qwen2-5-kunou-v1-v2-mkmlizer: ║ Chai Research Corp. ║
sao10k-14b-qwen2-5-kunou-v1-v2-mkmlizer: ║ Account ID: 7997a29f-0ceb-4cc7-9adf-840c57b4ae6f ║
sao10k-14b-qwen2-5-kunou-v1-v2-mkmlizer: ║ Expiration: 2025-04-15 23:59:59 ║
sao10k-14b-qwen2-5-kunou-v1-v2-mkmlizer: ║ ║
sao10k-14b-qwen2-5-kunou-v1-v2-mkmlizer: ╚═════════════════════════════════════════════════════════════════════╝
sao10k-14b-qwen2-5-kunou-v1-v2-mkmlizer: Downloaded to shared memory in 36.708s
sao10k-14b-qwen2-5-kunou-v1-v2-mkmlizer: quantizing model to /dev/shm/model_cache, profile:s0, folder:/tmp/tmpfg8fbdqp, device:0
sao10k-14b-qwen2-5-kunou-v1-v2-mkmlizer: Saving flywheel model at /dev/shm/model_cache
sao10k-14b-qwen2-5-kunou-v1-v2-mkmlizer: quantized model in 35.795s
sao10k-14b-qwen2-5-kunou-v1-v2-mkmlizer: Processed model Sao10K/14B-Qwen2.5-Kunou-v1 in 72.504s
sao10k-14b-qwen2-5-kunou-v1-v2-mkmlizer: creating bucket guanaco-mkml-models
sao10k-14b-qwen2-5-kunou-v1-v2-mkmlizer: Bucket 's3://guanaco-mkml-models/' created
sao10k-14b-qwen2-5-kunou-v1-v2-mkmlizer: uploading /dev/shm/model_cache to s3://guanaco-mkml-models/sao10k-14b-qwen2-5-kunou-v1-v2
sao10k-14b-qwen2-5-kunou-v1-v2-mkmlizer: cp /dev/shm/model_cache/added_tokens.json s3://guanaco-mkml-models/sao10k-14b-qwen2-5-kunou-v1-v2/added_tokens.json
sao10k-14b-qwen2-5-kunou-v1-v2-mkmlizer: cp /dev/shm/model_cache/config.json s3://guanaco-mkml-models/sao10k-14b-qwen2-5-kunou-v1-v2/config.json
sao10k-14b-qwen2-5-kunou-v1-v2-mkmlizer: cp /dev/shm/model_cache/tokenizer_config.json s3://guanaco-mkml-models/sao10k-14b-qwen2-5-kunou-v1-v2/tokenizer_config.json
sao10k-14b-qwen2-5-kunou-v1-v2-mkmlizer: cp /dev/shm/model_cache/special_tokens_map.json s3://guanaco-mkml-models/sao10k-14b-qwen2-5-kunou-v1-v2/special_tokens_map.json
sao10k-14b-qwen2-5-kunou-v1-v2-mkmlizer: cp /dev/shm/model_cache/merges.txt s3://guanaco-mkml-models/sao10k-14b-qwen2-5-kunou-v1-v2/merges.txt
sao10k-14b-qwen2-5-kunou-v1-v2-mkmlizer: cp /dev/shm/model_cache/vocab.json s3://guanaco-mkml-models/sao10k-14b-qwen2-5-kunou-v1-v2/vocab.json
sao10k-14b-qwen2-5-kunou-v1-v2-mkmlizer: cp /dev/shm/model_cache/tokenizer.json s3://guanaco-mkml-models/sao10k-14b-qwen2-5-kunou-v1-v2/tokenizer.json
sao10k-14b-qwen2-5-kunou-v1-v2-mkmlizer: cp /dev/shm/model_cache/flywheel_model.1.safetensors s3://guanaco-mkml-models/sao10k-14b-qwen2-5-kunou-v1-v2/flywheel_model.1.safetensors
sao10k-14b-qwen2-5-kunou-v1-v2-mkmlizer: cp /dev/shm/model_cache/flywheel_model.0.safetensors s3://guanaco-mkml-models/sao10k-14b-qwen2-5-kunou-v1-v2/flywheel_model.0.safetensors
Job sao10k-14b-qwen2-5-kunou-v1-v2-mkmlizer completed after 103.99s with status: succeeded
Stopping job with name sao10k-14b-qwen2-5-kunou-v1-v2-mkmlizer
Pipeline stage MKMLizer completed in 104.53s
run pipeline stage %s
Running pipeline stage MKMLTemplater
Pipeline stage MKMLTemplater completed in 0.16s
run pipeline stage %s
Running pipeline stage MKMLDeployer
Creating inference service sao10k-14b-qwen2-5-kunou-v1-v2
Waiting for inference service sao10k-14b-qwen2-5-kunou-v1-v2 to be ready
Inference service sao10k-14b-qwen2-5-kunou-v1-v2 ready after 180.6860408782959s
Pipeline stage MKMLDeployer completed in 181.12s
run pipeline stage %s
Running pipeline stage StressChecker
{"detail":"HTTPConnectionPool(host='sao10k-14b-qwen2-5-kunou-v1-v2-predictor.tenant-chaiml-guanaco.k.chaiverse.com', port=80): Read timed out. (read timeout=12.0)"}
Received unhealthy response to inference request!
{"detail":"HTTPConnectionPool(host='sao10k-14b-qwen2-5-kunou-v1-v2-predictor.tenant-chaiml-guanaco.k.chaiverse.com', port=80): Read timed out. (read timeout=12.0)"}
Received unhealthy response to inference request!
{"detail":"HTTPConnectionPool(host='sao10k-14b-qwen2-5-kunou-v1-v2-predictor.tenant-chaiml-guanaco.k.chaiverse.com', port=80): Read timed out. (read timeout=12.0)"}
Received unhealthy response to inference request!
{"detail":"HTTPConnectionPool(host='sao10k-14b-qwen2-5-kunou-v1-v2-predictor.tenant-chaiml-guanaco.k.chaiverse.com', port=80): Read timed out. (read timeout=12.0)"}
Received unhealthy response to inference request!
{"detail":"HTTPConnectionPool(host='sao10k-14b-qwen2-5-kunou-v1-v2-predictor.tenant-chaiml-guanaco.k.chaiverse.com', port=80): Read timed out. (read timeout=12.0)"}
Received unhealthy response to inference request!
5 requests
5 failed requests
5th percentile: 12.20556583404541
10th percentile: 12.20617036819458
20th percentile: 12.20737943649292
30th percentile: 12.208252000808717
40th percentile: 12.208788061141968
50th percentile: 12.20932412147522
60th percentile: 12.209427976608277
70th percentile: 12.209531831741334
80th percentile: 12.257141399383546
90th percentile: 12.352256679534912
95th percentile: 12.399814319610595
99th percentile: 12.437860431671142
mean time: 12.255845022201537
%s, retrying in %s seconds...
{"detail":"HTTPConnectionPool(host='sao10k-14b-qwen2-5-kunou-v1-v2-predictor.tenant-chaiml-guanaco.k.chaiverse.com', port=80): Read timed out. (read timeout=12.0)"}
Received unhealthy response to inference request!
{"detail":"HTTPConnectionPool(host='sao10k-14b-qwen2-5-kunou-v1-v2-predictor.tenant-chaiml-guanaco.k.chaiverse.com', port=80): Read timed out. (read timeout=12.0)"}
Received unhealthy response to inference request!
{"detail":"HTTPConnectionPool(host='sao10k-14b-qwen2-5-kunou-v1-v2-predictor.tenant-chaiml-guanaco.k.chaiverse.com', port=80): Read timed out. (read timeout=12.0)"}
Received unhealthy response to inference request!
{"detail":"HTTPConnectionPool(host='sao10k-14b-qwen2-5-kunou-v1-v2-predictor.tenant-chaiml-guanaco.k.chaiverse.com', port=80): Read timed out. (read timeout=12.0)"}
Received unhealthy response to inference request!
{"detail":"HTTPConnectionPool(host='sao10k-14b-qwen2-5-kunou-v1-v2-predictor.tenant-chaiml-guanaco.k.chaiverse.com', port=80): Read timed out. (read timeout=12.0)"}
Received unhealthy response to inference request!
5 requests
5 failed requests
5th percentile: 12.215452432632446
10th percentile: 12.217527866363525
20th percentile: 12.221678733825684
30th percentile: 12.225258159637452
40th percentile: 12.228266143798828
50th percentile: 12.231274127960205
60th percentile: 12.240918254852295
70th percentile: 12.250562381744384
80th percentile: 12.29971890449524
90th percentile: 12.388387823104859
95th percentile: 12.432722282409667
99th percentile: 12.468189849853516
mean time: 12.28016929626465
%s, retrying in %s seconds...
{"detail":"HTTPConnectionPool(host='sao10k-14b-qwen2-5-kunou-v1-v2-predictor.tenant-chaiml-guanaco.k.chaiverse.com', port=80): Read timed out. (read timeout=12.0)"}
Received unhealthy response to inference request!
{"detail":"HTTPConnectionPool(host='sao10k-14b-qwen2-5-kunou-v1-v2-predictor.tenant-chaiml-guanaco.k.chaiverse.com', port=80): Read timed out. (read timeout=12.0)"}
Received unhealthy response to inference request!
{"detail":"HTTPConnectionPool(host='sao10k-14b-qwen2-5-kunou-v1-v2-predictor.tenant-chaiml-guanaco.k.chaiverse.com', port=80): Read timed out. (read timeout=12.0)"}
Received unhealthy response to inference request!
{"detail":"HTTPConnectionPool(host='sao10k-14b-qwen2-5-kunou-v1-v2-predictor.tenant-chaiml-guanaco.k.chaiverse.com', port=80): Read timed out. (read timeout=12.0)"}
Received unhealthy response to inference request!
{"detail":"HTTPConnectionPool(host='sao10k-14b-qwen2-5-kunou-v1-v2-predictor.tenant-chaiml-guanaco.k.chaiverse.com', port=80): Read timed out. (read timeout=12.0)"}
Received unhealthy response to inference request!
5 requests
5 failed requests
5th percentile: 12.200521516799927
10th percentile: 12.202814197540283
20th percentile: 12.207399559020995
30th percentile: 12.211534881591797
40th percentile: 12.215220165252685
50th percentile: 12.218905448913574
60th percentile: 12.219811534881591
70th percentile: 12.220717620849609
80th percentile: 12.232128858566284
90th percentile: 12.254045248031616
95th percentile: 12.265003442764282
99th percentile: 12.273769998550415
mean time: 12.224791765213013
clean up pipeline due to error=DeploymentChecksError('Unacceptable number of predict errors: 100.0%')
Shutdown handler de-registered
sao10k-14b-qwen2-5-kunou-v1_v2 status is now failed due to DeploymentManager action