Shutdown handler not registered because Python interpreter is not running in the main thread
run pipeline %s
run pipeline stage %s
Running pipeline stage MKMLizer
Starting job with name nitral-ai-captainerisne-56304-v1-mkmlizer
Waiting for job on nitral-ai-captainerisne-56304-v1-mkmlizer to finish
nitral-ai-captainerisne-56304-v1-mkmlizer: ╔═════════════════════════════════════════════════════════════════════╗
nitral-ai-captainerisne-56304-v1-mkmlizer: ║ ║
nitral-ai-captainerisne-56304-v1-mkmlizer: ║ ██████ ██████ █████ ████ ████ ║
nitral-ai-captainerisne-56304-v1-mkmlizer: ║ ░░██████ ██████ ░░███ ███░ ░░███ ║
nitral-ai-captainerisne-56304-v1-mkmlizer: ║ ░███░█████░███ ░███ ███ ░███ ║
nitral-ai-captainerisne-56304-v1-mkmlizer: ║ ░███░░███ ░███ ░███████ ░███ ║
nitral-ai-captainerisne-56304-v1-mkmlizer: ║ ░███ ░░░ ░███ ░███░░███ ░███ ║
nitral-ai-captainerisne-56304-v1-mkmlizer: ║ ░███ ░███ ░███ ░░███ ░███ ║
nitral-ai-captainerisne-56304-v1-mkmlizer: ║ █████ █████ █████ ░░████ █████ ║
nitral-ai-captainerisne-56304-v1-mkmlizer: ║ ░░░░░ ░░░░░ ░░░░░ ░░░░ ░░░░░ ║
nitral-ai-captainerisne-56304-v1-mkmlizer: ║ ║
nitral-ai-captainerisne-56304-v1-mkmlizer: ║ Version: 0.30.2 ║
nitral-ai-captainerisne-56304-v1-mkmlizer: ║ Features: FLYWHEEL, CUDA ║
nitral-ai-captainerisne-56304-v1-mkmlizer: ║ Copyright 2023-2025 MK ONE TECHNOLOGIES Inc. ║
nitral-ai-captainerisne-56304-v1-mkmlizer: ║ https://mk1.ai ║
nitral-ai-captainerisne-56304-v1-mkmlizer: ║ ║
nitral-ai-captainerisne-56304-v1-mkmlizer: ║ The license key for the current software has been verified as ║
nitral-ai-captainerisne-56304-v1-mkmlizer: ║ belonging to: ║
nitral-ai-captainerisne-56304-v1-mkmlizer: ║ ║
nitral-ai-captainerisne-56304-v1-mkmlizer: ║ Chai Research Corp. ║
nitral-ai-captainerisne-56304-v1-mkmlizer: ║ Account ID: 7997a29f-0ceb-4cc7-9adf-840c57b4ae6f ║
nitral-ai-captainerisne-56304-v1-mkmlizer: ║ Expiration: 2028-03-31 23:59:59 ║
nitral-ai-captainerisne-56304-v1-mkmlizer: ║ ║
nitral-ai-captainerisne-56304-v1-mkmlizer: ╚═════════════════════════════════════════════════════════════════════╝
nitral-ai-captainerisne-56304-v1-mkmlizer: Downloaded to shared memory in 25.865s
nitral-ai-captainerisne-56304-v1-mkmlizer: Checking if Nitral-AI/CaptainErisNebula-12B-Chimera-v1.1 already exists in ChaiML
nitral-ai-captainerisne-56304-v1-mkmlizer: Creating repo ChaiML/CaptainErisNebula-12B-Chimera-v1.1 and uploading /tmp/tmp5h9rr_pd to it
nitral-ai-captainerisne-56304-v1-mkmlizer:
0%| | 0/6 [00:00<?, ?it/s]
17%|█▋ | 1/6 [00:04<00:24, 4.81s/it]
33%|███▎ | 2/6 [00:08<00:16, 4.12s/it]
50%|█████ | 3/6 [00:12<00:12, 4.05s/it]
67%|██████▋ | 4/6 [00:16<00:07, 3.91s/it]
83%|████████▎ | 5/6 [00:23<00:05, 5.21s/it]
100%|██████████| 6/6 [00:24<00:00, 3.59s/it]
100%|██████████| 6/6 [00:24<00:00, 4.01s/it]
nitral-ai-captainerisne-56304-v1-mkmlizer: quantizing model to /dev/shm/model_cache, profile:q4, folder:/tmp/tmp5h9rr_pd, device:0
nitral-ai-captainerisne-56304-v1-mkmlizer: Saving flywheel model at /dev/shm/model_cache
HTTP Request: %s %s "%s %d %s"
HTTP Request: %s %s "%s %d %s"
HTTP Request: %s %s "%s %d %s"
nitral-ai-captainerisne-56304-v1-mkmlizer: quantized model in 144.670s
nitral-ai-captainerisne-56304-v1-mkmlizer: Processed model Nitral-AI/CaptainErisNebula-12B-Chimera-v1.1 in 213.943s
nitral-ai-captainerisne-56304-v1-mkmlizer: creating bucket guanaco-mkml-models
nitral-ai-captainerisne-56304-v1-mkmlizer: Bucket 's3://guanaco-mkml-models/' created
nitral-ai-captainerisne-56304-v1-mkmlizer: uploading /dev/shm/model_cache to s3://guanaco-mkml-models/nitral-ai-captainerisne-56304-v1/nvidia
nitral-ai-captainerisne-56304-v1-mkmlizer: cp /dev/shm/model_cache/config.json s3://guanaco-mkml-models/nitral-ai-captainerisne-56304-v1/nvidia/config.json
nitral-ai-captainerisne-56304-v1-mkmlizer: cp /dev/shm/model_cache/tokenizer_config.json s3://guanaco-mkml-models/nitral-ai-captainerisne-56304-v1/nvidia/tokenizer_config.json
nitral-ai-captainerisne-56304-v1-mkmlizer: cp /dev/shm/model_cache/special_tokens_map.json s3://guanaco-mkml-models/nitral-ai-captainerisne-56304-v1/nvidia/special_tokens_map.json
nitral-ai-captainerisne-56304-v1-mkmlizer: cp /dev/shm/model_cache/tokenizer.json s3://guanaco-mkml-models/nitral-ai-captainerisne-56304-v1/nvidia/tokenizer.json
nitral-ai-captainerisne-56304-v1-mkmlizer: cp /dev/shm/model_cache/flywheel_model.0.safetensors s3://guanaco-mkml-models/nitral-ai-captainerisne-56304-v1/nvidia/flywheel_model.0.safetensors
Job nitral-ai-captainerisne-56304-v1-mkmlizer completed after 246.6s with status: succeeded
Stopping job with name nitral-ai-captainerisne-56304-v1-mkmlizer
Pipeline stage MKMLizer completed in 246.99s
run pipeline stage %s
Running pipeline stage MKMLTemplater
Pipeline stage MKMLTemplater completed in 0.16s
run pipeline stage %s
Running pipeline stage MKMLDeployer
Creating inference service nitral-ai-captainerisne-56304-v1
Waiting for inference service nitral-ai-captainerisne-56304-v1 to be ready
Inference service nitral-ai-captainerisne-56304-v1 ready after 60.084835052490234s
Pipeline stage MKMLDeployer completed in 60.49s
run pipeline stage %s
Running pipeline stage StressChecker
Received healthy response to inference request in 1.5597994327545166s
Received healthy response to inference request in 1.3086206912994385s
Received healthy response to inference request in 1.2295536994934082s
Received healthy response to inference request in 1.229886531829834s
Received healthy response to inference request in 1.9828987121582031s
5 requests
0 failed requests
5th percentile: 1.2296202659606934
10th percentile: 1.2296868324279786
20th percentile: 1.2298199653625488
30th percentile: 1.2456333637237549
40th percentile: 1.2771270275115967
50th percentile: 1.3086206912994385
60th percentile: 1.4090921878814697
70th percentile: 1.509563684463501
80th percentile: 1.644419288635254
90th percentile: 1.8136590003967286
95th percentile: 1.8982788562774657
99th percentile: 1.9659747409820556
mean time: 1.4621518135070801
Pipeline stage StressChecker completed in 9.21s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyTriggerPipeline
run_pipeline:run_in_cloud %s
starting trigger_guanaco_pipeline args=%s
triggered trigger_guanaco_pipeline args=%s
Pipeline stage OfflineFamilyFriendlyTriggerPipeline completed in 0.99s
run pipeline stage %s
Running pipeline stage TriggerMKMLProfilingPipeline
run_pipeline:run_in_cloud %s
starting trigger_guanaco_pipeline args=%s
triggered trigger_guanaco_pipeline args=%s
Pipeline stage TriggerMKMLProfilingPipeline completed in 1.03s
Shutdown handler de-registered
nitral-ai-captainerisne_56304_v1 status is now deployed due to DeploymentManager action
Shutdown handler registered
run pipeline %s
run pipeline stage %s
Running pipeline stage MKMLProfilerDeleter
Skipping teardown as no inference service was successfully deployed
Pipeline stage MKMLProfilerDeleter completed in 0.09s
run pipeline stage %s
Running pipeline stage MKMLProfilerTemplater
Pipeline stage MKMLProfilerTemplater completed in 0.09s
run pipeline stage %s
Running pipeline stage MKMLProfilerDeployer
Creating inference service nitral-ai-captainerisne-56304-v1-profiler
Shutdown handler registered
run pipeline %s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyScorer
Evaluating %s Family Friendly Score with %s threads
Pipeline stage OfflineFamilyFriendlyScorer completed in 2705.30s
Shutdown handler de-registered
nitral-ai-captainerisne_56304_v1 status is now inactive due to auto deactivation removed underperforming models
nitral-ai-captainerisne_56304_v1 status is now torndown due to DeploymentManager action