Shutdown handler not registered because Python interpreter is not running in the main thread
run pipeline %s
run pipeline stage %s
Running pipeline stage MKMLizer
Starting job with name vongolachouko-starcanno-61577-v6-mkmlizer
Waiting for job on vongolachouko-starcanno-61577-v6-mkmlizer to finish
vongolachouko-starcanno-61577-v6-mkmlizer: ╔═════════════════════════════════════════════════════════════════════╗
vongolachouko-starcanno-61577-v6-mkmlizer: ║ _____ __ __ ║
vongolachouko-starcanno-61577-v6-mkmlizer: ║ / _/ /_ ___ __/ / ___ ___ / / ║
vongolachouko-starcanno-61577-v6-mkmlizer: ║ / _/ / // / |/|/ / _ \/ -_) -_) / ║
vongolachouko-starcanno-61577-v6-mkmlizer: ║ /_//_/\_, /|__,__/_//_/\__/\__/_/ ║
vongolachouko-starcanno-61577-v6-mkmlizer: ║ /___/ ║
vongolachouko-starcanno-61577-v6-mkmlizer: ║ ║
vongolachouko-starcanno-61577-v6-mkmlizer: ║ Version: 0.11.12 ║
vongolachouko-starcanno-61577-v6-mkmlizer: ║ Copyright 2023 MK ONE TECHNOLOGIES Inc. ║
vongolachouko-starcanno-61577-v6-mkmlizer: ║ https://mk1.ai ║
vongolachouko-starcanno-61577-v6-mkmlizer: ║ ║
vongolachouko-starcanno-61577-v6-mkmlizer: ║ The license key for the current software has been verified as ║
vongolachouko-starcanno-61577-v6-mkmlizer: ║ belonging to: ║
vongolachouko-starcanno-61577-v6-mkmlizer: ║ ║
vongolachouko-starcanno-61577-v6-mkmlizer: ║ Chai Research Corp. ║
vongolachouko-starcanno-61577-v6-mkmlizer: ║ Account ID: 7997a29f-0ceb-4cc7-9adf-840c57b4ae6f ║
vongolachouko-starcanno-61577-v6-mkmlizer: ║ Expiration: 2025-04-15 23:59:59 ║
vongolachouko-starcanno-61577-v6-mkmlizer: ║ ║
vongolachouko-starcanno-61577-v6-mkmlizer: ╚═════════════════════════════════════════════════════════════════════╝
vongolachouko-starcanno-61577-v6-mkmlizer: Downloaded to shared memory in 32.474s
vongolachouko-starcanno-61577-v6-mkmlizer: quantizing model to /dev/shm/model_cache, profile:s0, folder:/tmp/tmprmtaicil, device:0
vongolachouko-starcanno-61577-v6-mkmlizer: Saving flywheel model at /dev/shm/model_cache
vongolachouko-starcanno-61577-v6-mkmlizer: quantized model in 35.442s
vongolachouko-starcanno-61577-v6-mkmlizer: Processed model VongolaChouko/Starcannon-Unleashed-12B-v1.0 in 67.916s
vongolachouko-starcanno-61577-v6-mkmlizer: creating bucket guanaco-mkml-models
vongolachouko-starcanno-61577-v6-mkmlizer: Bucket 's3://guanaco-mkml-models/' created
vongolachouko-starcanno-61577-v6-mkmlizer: uploading /dev/shm/model_cache to s3://guanaco-mkml-models/vongolachouko-starcanno-61577-v6
vongolachouko-starcanno-61577-v6-mkmlizer: cp /dev/shm/model_cache/config.json s3://guanaco-mkml-models/vongolachouko-starcanno-61577-v6/config.json
vongolachouko-starcanno-61577-v6-mkmlizer: cp /dev/shm/model_cache/special_tokens_map.json s3://guanaco-mkml-models/vongolachouko-starcanno-61577-v6/special_tokens_map.json
vongolachouko-starcanno-61577-v6-mkmlizer: cp /dev/shm/model_cache/tokenizer_config.json s3://guanaco-mkml-models/vongolachouko-starcanno-61577-v6/tokenizer_config.json
vongolachouko-starcanno-61577-v6-mkmlizer: cp /dev/shm/model_cache/tokenizer.json s3://guanaco-mkml-models/vongolachouko-starcanno-61577-v6/tokenizer.json
vongolachouko-starcanno-61577-v6-mkmlizer: cp /dev/shm/model_cache/flywheel_model.0.safetensors s3://guanaco-mkml-models/vongolachouko-starcanno-61577-v6/flywheel_model.0.safetensors
vongolachouko-starcanno-61577-v6-mkmlizer:
Loading 0: 0%| | 0/363 [00:00<?, ?it/s]
Loading 0: 1%| | 2/363 [00:06<18:09, 3.02s/it]
Loading 0: 2%|▏ | 6/363 [00:06<04:49, 1.23it/s]
Loading 0: 4%|▎ | 13/363 [00:06<01:43, 3.39it/s]
Loading 0: 5%|▍ | 18/363 [00:06<01:04, 5.38it/s]
Loading 0: 6%|▋ | 23/363 [00:06<00:42, 7.93it/s]
Loading 0: 8%|▊ | 29/363 [00:06<00:29, 11.50it/s]
Loading 0: 9%|▉ | 34/363 [00:06<00:22, 14.83it/s]
Loading 0: 11%|█ | 40/363 [00:07<00:20, 16.00it/s]
Loading 0: 12%|█▏ | 44/363 [00:07<00:17, 18.73it/s]
Loading 0: 14%|█▍ | 50/363 [00:07<00:12, 24.17it/s]
Loading 0: 15%|█▌ | 56/363 [00:07<00:10, 27.99it/s]
Loading 0: 17%|█▋ | 61/363 [00:07<00:09, 30.38it/s]
Loading 0: 18%|█▊ | 67/363 [00:07<00:08, 35.57it/s]
Loading 0: 20%|█▉ | 72/363 [00:07<00:07, 37.66it/s]
Loading 0: 21%|██▏ | 78/363 [00:07<00:07, 36.74it/s]
Loading 0: 24%|██▎ | 86/363 [00:08<00:06, 45.08it/s]
Loading 0: 25%|██▌ | 92/363 [00:08<00:06, 43.63it/s]
Loading 0: 27%|██▋ | 97/363 [00:08<00:06, 42.83it/s]
Loading 0: 29%|██▊ | 104/363 [00:08<00:05, 47.42it/s]
Loading 0: 30%|███ | 110/363 [00:08<00:05, 43.76it/s]
Loading 0: 32%|███▏ | 115/363 [00:08<00:06, 40.96it/s]
Loading 0: 33%|███▎ | 121/363 [00:09<00:08, 29.14it/s]
Loading 0: 34%|███▍ | 125/363 [00:09<00:07, 30.40it/s]
Loading 0: 36%|███▌ | 130/363 [00:09<00:06, 34.25it/s]
Loading 0: 37%|███▋ | 134/363 [00:09<00:06, 35.25it/s]
Loading 0: 39%|███▊ | 140/363 [00:09<00:05, 39.56it/s]
Loading 0: 40%|████ | 146/363 [00:09<00:05, 38.83it/s]
Loading 0: 42%|████▏ | 151/363 [00:09<00:05, 38.77it/s]
Loading 0: 43%|████▎ | 157/363 [00:09<00:04, 43.81it/s]
Loading 0: 45%|████▍ | 162/363 [00:10<00:04, 44.10it/s]
Loading 0: 46%|████▌ | 167/363 [00:10<00:04, 43.04it/s]
Loading 0: 48%|████▊ | 173/363 [00:10<00:04, 41.40it/s]
Loading 0: 49%|████▉ | 178/363 [00:10<00:04, 41.80it/s]
Loading 0: 51%|█████ | 185/363 [00:10<00:03, 46.88it/s]
Loading 0: 53%|█████▎ | 191/363 [00:10<00:03, 44.37it/s]
Loading 0: 54%|█████▍ | 196/363 [00:10<00:03, 43.38it/s]
Loading 0: 56%|█████▌ | 202/363 [00:11<00:05, 30.51it/s]
Loading 0: 57%|█████▋ | 206/363 [00:11<00:04, 31.79it/s]
Loading 0: 58%|█████▊ | 212/363 [00:11<00:04, 36.58it/s]
Loading 0: 60%|██████ | 218/363 [00:11<00:03, 37.06it/s]
Loading 0: 61%|██████▏ | 223/363 [00:11<00:03, 37.23it/s]
Loading 0: 63%|██████▎ | 229/363 [00:11<00:03, 41.71it/s]
Loading 0: 64%|██████▍ | 234/363 [00:11<00:03, 42.99it/s]
Loading 0: 66%|██████▌ | 239/363 [00:12<00:02, 43.90it/s]
Loading 0: 67%|██████▋ | 245/363 [00:12<00:02, 41.41it/s]
Loading 0: 69%|██████▉ | 250/363 [00:12<00:02, 41.43it/s]
Loading 0: 71%|███████ | 257/363 [00:12<00:02, 46.24it/s]
Loading 0: 72%|███████▏ | 263/363 [00:12<00:02, 44.61it/s]
Loading 0: 74%|███████▍ | 268/363 [00:12<00:02, 43.76it/s]
Loading 0: 75%|███████▌ | 274/363 [00:12<00:01, 46.51it/s]
Loading 0: 77%|███████▋ | 279/363 [00:12<00:01, 46.49it/s]
Loading 0: 78%|███████▊ | 284/363 [00:13<00:02, 29.57it/s]
Loading 0: 80%|███████▉ | 290/363 [00:13<00:02, 32.06it/s]
Loading 0: 81%|████████ | 294/363 [00:13<00:02, 32.26it/s]
Loading 0: 83%|████████▎ | 302/363 [00:13<00:01, 40.67it/s]
Loading 0: 85%|████████▍ | 307/363 [00:13<00:01, 42.23it/s]
Loading 0: 86%|████████▌ | 312/363 [00:13<00:01, 37.20it/s]
Loading 0: 88%|████████▊ | 319/363 [00:14<00:00, 44.58it/s]
Loading 0: 89%|████████▉ | 324/363 [00:14<00:00, 44.20it/s]
Loading 0: 91%|█████████ | 329/363 [00:14<00:00, 43.05it/s]
Loading 0: 92%|█████████▏| 335/363 [00:14<00:00, 41.70it/s]
Loading 0: 94%|█████████▎| 340/363 [00:14<00:00, 41.91it/s]
Loading 0: 95%|█████████▌| 346/363 [00:14<00:00, 45.83it/s]
Loading 0: 97%|█████████▋| 351/363 [00:14<00:00, 45.56it/s]
Loading 0: 98%|█████████▊| 356/363 [00:14<00:00, 46.19it/s]
Loading 0: 100%|█████████▉| 362/363 [00:14<00:00, 43.79it/s]
Job vongolachouko-starcanno-61577-v6-mkmlizer completed after 94.79s with status: succeeded
Stopping job with name vongolachouko-starcanno-61577-v6-mkmlizer
Pipeline stage MKMLizer completed in 95.24s
run pipeline stage %s
Running pipeline stage MKMLTemplater
Pipeline stage MKMLTemplater completed in 0.15s
run pipeline stage %s
Running pipeline stage MKMLDeployer
Creating inference service vongolachouko-starcanno-61577-v6
Waiting for inference service vongolachouko-starcanno-61577-v6 to be ready
Failed to get response for submission vongolachouko-starcanno_61577_v4: HTTPConnectionPool(host='vongolachouko-starcanno-61577-v4-predictor.tenant-chaiml-guanaco.k.chaiverse.com', port=80): Read timed out. (read timeout=12.0)
Failed to get response for submission function_lijit_2024-11-26: ('http://zonemercy-lexical-viral-7348-v17-predictor.tenant-chaiml-guanaco.k2.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'read tcp 127.0.0.1:44606->127.0.0.1:8080: read: connection reset by peer\n')
Inference service vongolachouko-starcanno-61577-v6 ready after 160.61371779441833s
Pipeline stage MKMLDeployer completed in 161.12s
run pipeline stage %s
Running pipeline stage StressChecker
Received healthy response to inference request in 2.5055298805236816s
Received healthy response to inference request in 1.5332109928131104s
Received healthy response to inference request in 1.4284334182739258s
Received healthy response to inference request in 1.236659049987793s
Received healthy response to inference request in 1.166475534439087s
5 requests
0 failed requests
5th percentile: 1.1805122375488282
10th percentile: 1.1945489406585694
20th percentile: 1.2226223468780517
30th percentile: 1.2750139236450195
40th percentile: 1.3517236709594727
50th percentile: 1.4284334182739258
60th percentile: 1.4703444480895995
70th percentile: 1.5122554779052735
80th percentile: 1.7276747703552249
90th percentile: 2.116602325439453
95th percentile: 2.311066102981567
99th percentile: 2.4666371250152586
mean time: 1.5740617752075194
Pipeline stage StressChecker completed in 9.04s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyTriggerPipeline
run_pipeline:run_in_cloud %s
starting trigger_guanaco_pipeline args=%s
triggered trigger_guanaco_pipeline args=%s
Pipeline stage OfflineFamilyFriendlyTriggerPipeline completed in 0.62s
run pipeline stage %s
Running pipeline stage TriggerMKMLProfilingPipeline
run_pipeline:run_in_cloud %s
starting trigger_guanaco_pipeline args=%s
triggered trigger_guanaco_pipeline args=%s
Pipeline stage TriggerMKMLProfilingPipeline completed in 0.63s
Shutdown handler de-registered
vongolachouko-starcanno_61577_v6 status is now deployed due to DeploymentManager action
Shutdown handler registered
run pipeline %s
run pipeline stage %s
Running pipeline stage MKMLProfilerDeleter
Skipping teardown as no inference service was successfully deployed
Pipeline stage MKMLProfilerDeleter completed in 0.09s
run pipeline stage %s
Running pipeline stage MKMLProfilerTemplater
Pipeline stage MKMLProfilerTemplater completed in 0.08s
run pipeline stage %s
Running pipeline stage MKMLProfilerDeployer
Creating inference service vongolachouko-starcanno-61577-v6-profiler
Waiting for inference service vongolachouko-starcanno-61577-v6-profiler to be ready
Shutdown handler registered
run pipeline %s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyScorer
Evaluating %s Family Friendly Score with %s threads
Pipeline stage OfflineFamilyFriendlyScorer completed in 3288.39s
Shutdown handler de-registered
vongolachouko-starcanno_61577_v6 status is now inactive due to auto deactivation removed underperforming models
vongolachouko-starcanno_61577_v6 status is now torndown due to DeploymentManager action