Shutdown handler not registered because Python interpreter is not running in the main thread
run pipeline %s
run pipeline stage %s
Running pipeline stage MKMLizer
Starting job with name vongolachouko-starcann-61577-v14-mkmlizer
Waiting for job on vongolachouko-starcann-61577-v14-mkmlizer to finish
vongolachouko-starcann-61577-v14-mkmlizer: ╔═════════════════════════════════════════════════════════════════════╗
vongolachouko-starcann-61577-v14-mkmlizer: ║ _____ __ __ ║
vongolachouko-starcann-61577-v14-mkmlizer: ║ / _/ /_ ___ __/ / ___ ___ / / ║
vongolachouko-starcann-61577-v14-mkmlizer: ║ / _/ / // / |/|/ / _ \/ -_) -_) / ║
vongolachouko-starcann-61577-v14-mkmlizer: ║ /_//_/\_, /|__,__/_//_/\__/\__/_/ ║
vongolachouko-starcann-61577-v14-mkmlizer: ║ /___/ ║
vongolachouko-starcann-61577-v14-mkmlizer: ║ ║
vongolachouko-starcann-61577-v14-mkmlizer: ║ Version: 0.11.12 ║
vongolachouko-starcann-61577-v14-mkmlizer: ║ Copyright 2023 MK ONE TECHNOLOGIES Inc. ║
vongolachouko-starcann-61577-v14-mkmlizer: ║ https://mk1.ai ║
vongolachouko-starcann-61577-v14-mkmlizer: ║ ║
vongolachouko-starcann-61577-v14-mkmlizer: ║ The license key for the current software has been verified as ║
vongolachouko-starcann-61577-v14-mkmlizer: ║ belonging to: ║
vongolachouko-starcann-61577-v14-mkmlizer: ║ ║
vongolachouko-starcann-61577-v14-mkmlizer: ║ Chai Research Corp. ║
vongolachouko-starcann-61577-v14-mkmlizer: ║ Account ID: 7997a29f-0ceb-4cc7-9adf-840c57b4ae6f ║
vongolachouko-starcann-61577-v14-mkmlizer: ║ Expiration: 2025-04-15 23:59:59 ║
vongolachouko-starcann-61577-v14-mkmlizer: ║ ║
vongolachouko-starcann-61577-v14-mkmlizer: ╚═════════════════════════════════════════════════════════════════════╝
vongolachouko-starcann-61577-v14-mkmlizer: Downloaded to shared memory in 30.985s
vongolachouko-starcann-61577-v14-mkmlizer: quantizing model to /dev/shm/model_cache, profile:s0, folder:/tmp/tmpqt2wva_b, device:0
vongolachouko-starcann-61577-v14-mkmlizer: Saving flywheel model at /dev/shm/model_cache
vongolachouko-starcann-61577-v14-mkmlizer: quantized model in 36.842s
vongolachouko-starcann-61577-v14-mkmlizer: Processed model VongolaChouko/Starcannon-Unleashed-12B-v1.0 in 67.827s
vongolachouko-starcann-61577-v14-mkmlizer: creating bucket guanaco-mkml-models
vongolachouko-starcann-61577-v14-mkmlizer: Bucket 's3://guanaco-mkml-models/' created
vongolachouko-starcann-61577-v14-mkmlizer: uploading /dev/shm/model_cache to s3://guanaco-mkml-models/vongolachouko-starcann-61577-v14
vongolachouko-starcann-61577-v14-mkmlizer: cp /dev/shm/model_cache/config.json s3://guanaco-mkml-models/vongolachouko-starcann-61577-v14/config.json
vongolachouko-starcann-61577-v14-mkmlizer: cp /dev/shm/model_cache/special_tokens_map.json s3://guanaco-mkml-models/vongolachouko-starcann-61577-v14/special_tokens_map.json
vongolachouko-starcann-61577-v14-mkmlizer: cp /dev/shm/model_cache/tokenizer_config.json s3://guanaco-mkml-models/vongolachouko-starcann-61577-v14/tokenizer_config.json
vongolachouko-starcann-61577-v14-mkmlizer: cp /dev/shm/model_cache/tokenizer.json s3://guanaco-mkml-models/vongolachouko-starcann-61577-v14/tokenizer.json
vongolachouko-starcann-61577-v14-mkmlizer: cp /dev/shm/model_cache/flywheel_model.0.safetensors s3://guanaco-mkml-models/vongolachouko-starcann-61577-v14/flywheel_model.0.safetensors
vongolachouko-starcann-61577-v14-mkmlizer:
Loading 0: 0%| | 0/363 [00:00<?, ?it/s]
Loading 0: 1%| | 2/363 [00:06<18:29, 3.07s/it]
Loading 0: 2%|▏ | 6/363 [00:06<04:53, 1.22it/s]
Loading 0: 3%|▎ | 11/363 [00:06<02:08, 2.73it/s]
Loading 0: 4%|▍ | 15/363 [00:06<01:21, 4.28it/s]
Loading 0: 6%|▋ | 23/363 [00:06<00:40, 8.48it/s]
Loading 0: 8%|▊ | 29/363 [00:06<00:28, 11.86it/s]
Loading 0: 9%|▉ | 34/363 [00:06<00:21, 15.04it/s]
Loading 0: 11%|█ | 40/363 [00:07<00:20, 15.72it/s]
Loading 0: 12%|█▏ | 44/363 [00:07<00:17, 18.13it/s]
Loading 0: 13%|█▎ | 49/363 [00:07<00:14, 22.29it/s]
Loading 0: 15%|█▍ | 53/363 [00:07<00:12, 24.57it/s]
Loading 0: 16%|█▌ | 58/363 [00:07<00:10, 29.07it/s]
Loading 0: 17%|█▋ | 63/363 [00:07<00:09, 31.93it/s]
Loading 0: 19%|█▊ | 68/363 [00:07<00:08, 34.52it/s]
Loading 0: 20%|██ | 73/363 [00:08<00:07, 37.38it/s]
Loading 0: 21%|██▏ | 78/363 [00:08<00:08, 33.25it/s]
Loading 0: 23%|██▎ | 85/363 [00:08<00:06, 41.34it/s]
Loading 0: 25%|██▍ | 90/363 [00:08<00:06, 41.43it/s]
Loading 0: 26%|██▌ | 95/363 [00:08<00:06, 41.20it/s]
Loading 0: 28%|██▊ | 100/363 [00:08<00:06, 43.10it/s]
Loading 0: 29%|██▉ | 105/363 [00:08<00:07, 35.89it/s]
Loading 0: 31%|███ | 112/363 [00:08<00:05, 42.22it/s]
Loading 0: 32%|███▏ | 117/363 [00:09<00:05, 42.76it/s]
Loading 0: 34%|███▎ | 122/363 [00:09<00:08, 27.56it/s]
Loading 0: 35%|███▍ | 126/363 [00:09<00:07, 29.73it/s]
Loading 0: 36%|███▌ | 131/363 [00:09<00:07, 32.81it/s]
Loading 0: 37%|███▋ | 136/363 [00:09<00:06, 36.57it/s]
Loading 0: 39%|███▉ | 141/363 [00:09<00:06, 32.98it/s]
Loading 0: 41%|████ | 148/363 [00:10<00:05, 40.23it/s]
Loading 0: 42%|████▏ | 153/363 [00:10<00:05, 41.33it/s]
Loading 0: 44%|████▎ | 158/363 [00:10<00:04, 42.30it/s]
Loading 0: 45%|████▌ | 164/363 [00:10<00:04, 40.11it/s]
Loading 0: 47%|████▋ | 169/363 [00:10<00:04, 39.23it/s]
Loading 0: 48%|████▊ | 175/363 [00:10<00:04, 43.42it/s]
Loading 0: 50%|████▉ | 180/363 [00:10<00:04, 43.46it/s]
Loading 0: 51%|█████ | 185/363 [00:10<00:04, 42.84it/s]
Loading 0: 52%|█████▏ | 190/363 [00:11<00:03, 44.02it/s]
Loading 0: 54%|█████▎ | 195/363 [00:11<00:04, 36.21it/s]
Loading 0: 56%|█████▌ | 202/363 [00:11<00:05, 27.62it/s]
Loading 0: 57%|█████▋ | 206/363 [00:11<00:05, 29.24it/s]
Loading 0: 58%|█████▊ | 211/363 [00:11<00:04, 33.24it/s]
Loading 0: 59%|█████▉ | 215/363 [00:11<00:04, 33.89it/s]
Loading 0: 61%|██████ | 220/363 [00:12<00:03, 37.09it/s]
Loading 0: 62%|██████▏ | 225/363 [00:12<00:03, 38.91it/s]
Loading 0: 63%|██████▎ | 230/363 [00:12<00:03, 40.33it/s]
Loading 0: 65%|██████▍ | 235/363 [00:12<00:03, 42.53it/s]
Loading 0: 66%|██████▌ | 240/363 [00:12<00:03, 34.86it/s]
Loading 0: 68%|██████▊ | 247/363 [00:12<00:02, 41.92it/s]
Loading 0: 69%|██████▉ | 252/363 [00:12<00:02, 41.91it/s]
Loading 0: 71%|███████ | 257/363 [00:12<00:02, 40.70it/s]
Loading 0: 72%|███████▏ | 262/363 [00:13<00:02, 42.45it/s]
Loading 0: 74%|███████▎ | 267/363 [00:13<00:02, 34.55it/s]
Loading 0: 75%|███████▌ | 274/363 [00:13<00:02, 41.22it/s]
Loading 0: 77%|███████▋ | 279/363 [00:13<00:02, 41.39it/s]
Loading 0: 78%|███████▊ | 284/363 [00:13<00:02, 27.65it/s]
Loading 0: 79%|███████▉ | 288/363 [00:13<00:02, 29.66it/s]
Loading 0: 81%|████████ | 293/363 [00:14<00:02, 32.47it/s]
Loading 0: 82%|████████▏ | 297/363 [00:14<00:01, 33.98it/s]
Loading 0: 83%|████████▎ | 302/363 [00:14<00:01, 36.85it/s]
Loading 0: 85%|████████▍ | 308/363 [00:14<00:01, 36.47it/s]
Loading 0: 86%|████████▌ | 312/363 [00:14<00:01, 33.87it/s]
Loading 0: 88%|████████▊ | 319/363 [00:14<00:01, 40.53it/s]
Loading 0: 89%|████████▉ | 324/363 [00:14<00:00, 40.10it/s]
Loading 0: 91%|█████████ | 329/363 [00:14<00:00, 40.59it/s]
Loading 0: 92%|█████████▏| 334/363 [00:15<00:00, 42.09it/s]
Loading 0: 93%|█████████▎| 339/363 [00:15<00:00, 34.96it/s]
Loading 0: 95%|█████████▌| 346/363 [00:15<00:00, 41.42it/s]
Loading 0: 97%|█████████▋| 351/363 [00:15<00:00, 41.54it/s]
Loading 0: 98%|█████████▊| 356/363 [00:15<00:00, 41.82it/s]
Loading 0: 99%|█████████▉| 361/363 [00:15<00:00, 43.32it/s]
Job vongolachouko-starcann-61577-v14-mkmlizer completed after 93.94s with status: succeeded
Stopping job with name vongolachouko-starcann-61577-v14-mkmlizer
Pipeline stage MKMLizer completed in 94.40s
run pipeline stage %s
Running pipeline stage MKMLTemplater
Pipeline stage MKMLTemplater completed in 0.20s
run pipeline stage %s
Running pipeline stage MKMLDeployer
Creating inference service vongolachouko-starcann-61577-v14
Waiting for inference service vongolachouko-starcann-61577-v14 to be ready
Inference service vongolachouko-starcann-61577-v14 ready after 170.5850908756256s
Pipeline stage MKMLDeployer completed in 171.02s
run pipeline stage %s
Running pipeline stage StressChecker
Received healthy response to inference request in 2.0445237159729004s
Received healthy response to inference request in 1.6351203918457031s
Received healthy response to inference request in 1.693312644958496s
Received healthy response to inference request in 1.336371660232544s
Received healthy response to inference request in 1.8850655555725098s
5 requests
0 failed requests
5th percentile: 1.3961214065551757
10th percentile: 1.4558711528778077
20th percentile: 1.5753706455230714
30th percentile: 1.6467588424682618
40th percentile: 1.6700357437133788
50th percentile: 1.693312644958496
60th percentile: 1.7700138092041016
70th percentile: 1.846714973449707
80th percentile: 1.916957187652588
90th percentile: 1.980740451812744
95th percentile: 2.0126320838928224
99th percentile: 2.0381453895568846
mean time: 1.7188787937164307
Pipeline stage StressChecker completed in 9.86s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyTriggerPipeline
run_pipeline:run_in_cloud %s
starting trigger_guanaco_pipeline args=%s
triggered trigger_guanaco_pipeline args=%s
Pipeline stage OfflineFamilyFriendlyTriggerPipeline completed in 0.63s
run pipeline stage %s
Running pipeline stage TriggerMKMLProfilingPipeline
run_pipeline:run_in_cloud %s
starting trigger_guanaco_pipeline args=%s
triggered trigger_guanaco_pipeline args=%s
Pipeline stage TriggerMKMLProfilingPipeline completed in 0.65s
Shutdown handler de-registered
vongolachouko-starcann_61577_v14 status is now deployed due to DeploymentManager action
Shutdown handler registered
run pipeline %s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyScorer
Evaluating %s Family Friendly Score with %s threads
Pipeline stage OfflineFamilyFriendlyScorer completed in 3295.50s
Shutdown handler de-registered
vongolachouko-starcann_61577_v14 status is now inactive due to auto deactivation removed underperforming models
vongolachouko-starcann_61577_v14 status is now torndown due to DeploymentManager action