Shutdown handler not registered because Python interpreter is not running in the main thread
run pipeline %s
run pipeline stage %s
Running pipeline stage MKMLizer
Starting job with name nitral-ai-captain-eris-59667-v18-mkmlizer
Waiting for job on nitral-ai-captain-eris-59667-v18-mkmlizer to finish
nitral-ai-captain-eris-59667-v18-mkmlizer: ╔═════════════════════════════════════════════════════════════════════╗
nitral-ai-captain-eris-59667-v18-mkmlizer: ║ _____ __ __ ║
nitral-ai-captain-eris-59667-v18-mkmlizer: ║ / _/ /_ ___ __/ / ___ ___ / / ║
nitral-ai-captain-eris-59667-v18-mkmlizer: ║ / _/ / // / |/|/ / _ \/ -_) -_) / ║
nitral-ai-captain-eris-59667-v18-mkmlizer: ║ /_//_/\_, /|__,__/_//_/\__/\__/_/ ║
nitral-ai-captain-eris-59667-v18-mkmlizer: ║ /___/ ║
nitral-ai-captain-eris-59667-v18-mkmlizer: ║ ║
nitral-ai-captain-eris-59667-v18-mkmlizer: ║ Version: 0.11.12 ║
nitral-ai-captain-eris-59667-v18-mkmlizer: ║ Copyright 2023 MK ONE TECHNOLOGIES Inc. ║
nitral-ai-captain-eris-59667-v18-mkmlizer: ║ https://mk1.ai ║
nitral-ai-captain-eris-59667-v18-mkmlizer: ║ ║
nitral-ai-captain-eris-59667-v18-mkmlizer: ║ The license key for the current software has been verified as ║
nitral-ai-captain-eris-59667-v18-mkmlizer: ║ belonging to: ║
nitral-ai-captain-eris-59667-v18-mkmlizer: ║ ║
nitral-ai-captain-eris-59667-v18-mkmlizer: ║ Chai Research Corp. ║
nitral-ai-captain-eris-59667-v18-mkmlizer: ║ Account ID: 7997a29f-0ceb-4cc7-9adf-840c57b4ae6f ║
nitral-ai-captain-eris-59667-v18-mkmlizer: ║ Expiration: 2025-04-15 23:59:59 ║
nitral-ai-captain-eris-59667-v18-mkmlizer: ║ ║
nitral-ai-captain-eris-59667-v18-mkmlizer: ╚═════════════════════════════════════════════════════════════════════╝
nitral-ai-captain-eris-59667-v18-mkmlizer: Downloaded to shared memory in 36.349s
nitral-ai-captain-eris-59667-v18-mkmlizer: quantizing model to /dev/shm/model_cache, profile:s0, folder:/tmp/tmp8dw8av3l, device:0
nitral-ai-captain-eris-59667-v18-mkmlizer: Saving flywheel model at /dev/shm/model_cache
nitral-ai-captain-eris-59667-v18-mkmlizer: quantized model in 37.080s
nitral-ai-captain-eris-59667-v18-mkmlizer: Processed model Nitral-AI/Captain-Eris_Violet-V0.420-12B in 73.430s
nitral-ai-captain-eris-59667-v18-mkmlizer: creating bucket guanaco-mkml-models
nitral-ai-captain-eris-59667-v18-mkmlizer: Bucket 's3://guanaco-mkml-models/' created
nitral-ai-captain-eris-59667-v18-mkmlizer: uploading /dev/shm/model_cache to s3://guanaco-mkml-models/nitral-ai-captain-eris-59667-v18
nitral-ai-captain-eris-59667-v18-mkmlizer: cp /dev/shm/model_cache/config.json s3://guanaco-mkml-models/nitral-ai-captain-eris-59667-v18/config.json
nitral-ai-captain-eris-59667-v18-mkmlizer: cp /dev/shm/model_cache/special_tokens_map.json s3://guanaco-mkml-models/nitral-ai-captain-eris-59667-v18/special_tokens_map.json
nitral-ai-captain-eris-59667-v18-mkmlizer: cp /dev/shm/model_cache/tokenizer_config.json s3://guanaco-mkml-models/nitral-ai-captain-eris-59667-v18/tokenizer_config.json
nitral-ai-captain-eris-59667-v18-mkmlizer: cp /dev/shm/model_cache/tokenizer.json s3://guanaco-mkml-models/nitral-ai-captain-eris-59667-v18/tokenizer.json
nitral-ai-captain-eris-59667-v18-mkmlizer: cp /dev/shm/model_cache/flywheel_model.0.safetensors s3://guanaco-mkml-models/nitral-ai-captain-eris-59667-v18/flywheel_model.0.safetensors
nitral-ai-captain-eris-59667-v18-mkmlizer:
Loading 0: 0%| | 0/363 [00:00<?, ?it/s]
Loading 0: 1%| | 2/363 [00:06<18:24, 3.06s/it]
Loading 0: 2%|▏ | 6/363 [00:06<04:52, 1.22it/s]
Loading 0: 4%|▎ | 13/363 [00:06<01:44, 3.36it/s]
Loading 0: 5%|▍ | 18/363 [00:06<01:04, 5.33it/s]
Loading 0: 6%|▋ | 23/363 [00:06<00:43, 7.89it/s]
Loading 0: 8%|▊ | 29/363 [00:06<00:29, 11.34it/s]
Loading 0: 9%|▉ | 34/363 [00:06<00:22, 14.72it/s]
Loading 0: 11%|█ | 40/363 [00:07<00:20, 15.44it/s]
Loading 0: 12%|█▏ | 44/363 [00:07<00:17, 17.97it/s]
Loading 0: 13%|█▎ | 49/363 [00:07<00:14, 22.13it/s]
Loading 0: 15%|█▍ | 53/363 [00:07<00:12, 24.14it/s]
Loading 0: 16%|█▌ | 58/363 [00:07<00:10, 28.23it/s]
Loading 0: 17%|█▋ | 62/363 [00:07<00:10, 29.87it/s]
Loading 0: 18%|█▊ | 67/363 [00:07<00:08, 34.05it/s]
Loading 0: 20%|█▉ | 72/363 [00:08<00:08, 36.08it/s]
Loading 0: 21%|██ | 77/363 [00:08<00:07, 38.24it/s]
Loading 0: 23%|██▎ | 82/363 [00:08<00:06, 40.56it/s]
Loading 0: 24%|██▍ | 87/363 [00:08<00:08, 34.18it/s]
Loading 0: 26%|██▌ | 94/363 [00:08<00:06, 40.04it/s]
Loading 0: 27%|██▋ | 99/363 [00:08<00:06, 39.70it/s]
Loading 0: 29%|██▊ | 104/363 [00:08<00:06, 40.32it/s]
Loading 0: 30%|███ | 109/363 [00:08<00:06, 41.22it/s]
Loading 0: 31%|███▏ | 114/363 [00:09<00:07, 32.65it/s]
Loading 0: 33%|███▎ | 121/363 [00:09<00:09, 25.21it/s]
Loading 0: 34%|███▍ | 125/363 [00:09<00:08, 26.84it/s]
Loading 0: 36%|███▌ | 130/363 [00:09<00:07, 30.49it/s]
Loading 0: 37%|███▋ | 134/363 [00:09<00:07, 31.21it/s]
Loading 0: 38%|███▊ | 139/363 [00:09<00:06, 33.95it/s]
Loading 0: 39%|███▉ | 143/363 [00:10<00:06, 33.91it/s]
Loading 0: 41%|████ | 148/363 [00:10<00:05, 36.30it/s]
Loading 0: 42%|████▏ | 152/363 [00:10<00:06, 34.94it/s]
Loading 0: 43%|████▎ | 157/363 [00:10<00:05, 36.68it/s]
Loading 0: 44%|████▍ | 161/363 [00:10<00:05, 35.82it/s]
Loading 0: 46%|████▌ | 166/363 [00:10<00:05, 37.94it/s]
Loading 0: 47%|████▋ | 170/363 [00:10<00:05, 37.32it/s]
Loading 0: 48%|████▊ | 175/363 [00:10<00:04, 37.85it/s]
Loading 0: 49%|████▉ | 179/363 [00:11<00:05, 36.08it/s]
Loading 0: 51%|█████ | 184/363 [00:11<00:04, 38.18it/s]
Loading 0: 52%|█████▏ | 188/363 [00:11<00:04, 36.61it/s]
Loading 0: 53%|█████▎ | 192/363 [00:11<00:04, 37.41it/s]
Loading 0: 54%|█████▍ | 196/363 [00:11<00:04, 33.96it/s]
Loading 0: 56%|█████▌ | 202/363 [00:11<00:06, 24.03it/s]
Loading 0: 56%|█████▋ | 205/363 [00:12<00:06, 24.70it/s]
Loading 0: 58%|█████▊ | 211/363 [00:12<00:04, 31.25it/s]
Loading 0: 59%|█████▉ | 215/363 [00:12<00:04, 32.73it/s]
Loading 0: 61%|██████ | 220/363 [00:12<00:03, 36.04it/s]
Loading 0: 62%|██████▏ | 224/363 [00:12<00:03, 35.53it/s]
Loading 0: 63%|██████▎ | 230/363 [00:12<00:03, 39.87it/s]
Loading 0: 65%|██████▌ | 236/363 [00:12<00:03, 39.10it/s]
Loading 0: 66%|██████▋ | 241/363 [00:12<00:03, 39.06it/s]
Loading 0: 68%|██████▊ | 247/363 [00:12<00:02, 44.14it/s]
Loading 0: 69%|██████▉ | 252/363 [00:13<00:02, 44.35it/s]
Loading 0: 71%|███████ | 257/363 [00:13<00:02, 43.95it/s]
Loading 0: 72%|███████▏ | 262/363 [00:13<00:02, 43.64it/s]
Loading 0: 74%|███████▎ | 267/363 [00:13<00:02, 34.47it/s]
Loading 0: 75%|███████▌ | 274/363 [00:13<00:02, 41.19it/s]
Loading 0: 77%|███████▋ | 279/363 [00:13<00:02, 40.84it/s]
Loading 0: 78%|███████▊ | 284/363 [00:14<00:03, 24.97it/s]
Loading 0: 79%|███████▉ | 288/363 [00:14<00:02, 26.69it/s]
Loading 0: 80%|████████ | 292/363 [00:14<00:02, 28.63it/s]
Loading 0: 82%|████████▏ | 296/363 [00:14<00:02, 29.66it/s]
Loading 0: 83%|████████▎ | 301/363 [00:14<00:01, 33.31it/s]
Loading 0: 84%|████████▍ | 305/363 [00:14<00:01, 33.45it/s]
Loading 0: 85%|████████▌ | 310/363 [00:14<00:01, 35.56it/s]
Loading 0: 87%|████████▋ | 314/363 [00:15<00:01, 34.99it/s]
Loading 0: 88%|████████▊ | 319/363 [00:15<00:01, 37.19it/s]
Loading 0: 89%|████████▉ | 323/363 [00:15<00:01, 35.80it/s]
Loading 0: 90%|█████████ | 328/363 [00:15<00:00, 37.62it/s]
Loading 0: 91%|█████████▏| 332/363 [00:15<00:00, 36.10it/s]
Loading 0: 93%|█████████▎| 337/363 [00:15<00:00, 38.60it/s]
Loading 0: 94%|█████████▍| 341/363 [00:15<00:00, 37.27it/s]
Loading 0: 95%|█████████▌| 346/363 [00:15<00:00, 37.48it/s]
Loading 0: 96%|█████████▋| 350/363 [00:15<00:00, 35.87it/s]
Loading 0: 98%|█████████▊| 355/363 [00:16<00:00, 37.60it/s]
Loading 0: 99%|█████████▉| 359/363 [00:16<00:00, 36.39it/s]
Job nitral-ai-captain-eris-59667-v18-mkmlizer completed after 114.51s with status: succeeded
Stopping job with name nitral-ai-captain-eris-59667-v18-mkmlizer
Pipeline stage MKMLizer completed in 115.00s
run pipeline stage %s
Running pipeline stage MKMLTemplater
Pipeline stage MKMLTemplater completed in 0.15s
run pipeline stage %s
Running pipeline stage MKMLDeployer
Creating inference service nitral-ai-captain-eris-59667-v18
Waiting for inference service nitral-ai-captain-eris-59667-v18 to be ready
Inference service nitral-ai-captain-eris-59667-v18 ready after 160.6068732738495s
Pipeline stage MKMLDeployer completed in 161.05s
run pipeline stage %s
Running pipeline stage StressChecker
HTTPConnectionPool(host='guanaco-submitter.guanaco-backend.k2.chaiverse.com', port=80): Read timed out. (read timeout=20)
Received unhealthy response to inference request!
Received healthy response to inference request in 2.0952868461608887s
Received healthy response to inference request in 1.6813552379608154s
Received healthy response to inference request in 0.9619557857513428s
Received healthy response to inference request in 1.7223381996154785s
5 requests
1 failed requests
5th percentile: 1.1058356761932373
10th percentile: 1.2497155666351318
20th percentile: 1.537475347518921
30th percentile: 1.689551830291748
40th percentile: 1.7059450149536133
50th percentile: 1.7223381996154785
60th percentile: 1.8715176582336426
70th percentile: 2.0206971168518066
80th percentile: 5.703500366210941
90th percentile: 12.919927406311036
95th percentile: 16.528140926361083
99th percentile: 19.414711742401124
mean time: 5.319458103179931
%s, retrying in %s seconds...
Received healthy response to inference request in 2.056062698364258s
Received healthy response to inference request in 1.6758832931518555s
Received healthy response to inference request in 2.0818333625793457s
Received healthy response to inference request in 1.7377703189849854s
Received healthy response to inference request in 1.758274793624878s
5 requests
0 failed requests
5th percentile: 1.6882606983184814
10th percentile: 1.7006381034851075
20th percentile: 1.7253929138183595
30th percentile: 1.741871213912964
40th percentile: 1.750073003768921
50th percentile: 1.758274793624878
60th percentile: 1.87738995552063
70th percentile: 1.9965051174163817
80th percentile: 2.0612168312072754
90th percentile: 2.0715250968933105
95th percentile: 2.076679229736328
99th percentile: 2.0808025360107423
mean time: 1.8619648933410644
Pipeline stage StressChecker completed in 38.49s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyTriggerPipeline
run_pipeline:run_in_cloud %s
starting trigger_guanaco_pipeline args=%s
triggered trigger_guanaco_pipeline args=%s
Pipeline stage OfflineFamilyFriendlyTriggerPipeline completed in 0.65s
run pipeline stage %s
Running pipeline stage TriggerMKMLProfilingPipeline
run_pipeline:run_in_cloud %s
starting trigger_guanaco_pipeline args=%s
triggered trigger_guanaco_pipeline args=%s
Pipeline stage TriggerMKMLProfilingPipeline completed in 0.67s
Shutdown handler de-registered
nitral-ai-captain-eris_59667_v18 status is now deployed due to DeploymentManager action
Shutdown handler registered
run pipeline %s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyScorer
Evaluating %s Family Friendly Score with %s threads
Pipeline stage OfflineFamilyFriendlyScorer completed in 3194.46s
Shutdown handler de-registered
nitral-ai-captain-eris_59667_v18 status is now inactive due to auto deactivation removed underperforming models
nitral-ai-captain-eris_59667_v18 status is now torndown due to DeploymentManager action