Running pipeline stage MKMLizer
Starting job with name senseable-trillama-8b-v1-mkmlizer
Waiting for job on senseable-trillama-8b-v1-mkmlizer to finish
senseable-trillama-8b-v1-mkmlizer: ╔═════════════════════════════════════════════════════════════════════╗
senseable-trillama-8b-v1-mkmlizer: ║ _____ __ __ ║
senseable-trillama-8b-v1-mkmlizer: ║ / _/ /_ ___ __/ / ___ ___ / / ║
senseable-trillama-8b-v1-mkmlizer: ║ / _/ / // / |/|/ / _ \/ -_) -_) / ║
senseable-trillama-8b-v1-mkmlizer: ║ /_//_/\_, /|__,__/_//_/\__/\__/_/ ║
senseable-trillama-8b-v1-mkmlizer: ║ /___/ ║
senseable-trillama-8b-v1-mkmlizer: ║ ║
senseable-trillama-8b-v1-mkmlizer: ║ Version: 0.6.11 ║
senseable-trillama-8b-v1-mkmlizer: ║ Copyright 2023 MK ONE TECHNOLOGIES Inc. ║
senseable-trillama-8b-v1-mkmlizer: ║ ║
senseable-trillama-8b-v1-mkmlizer: ║ The license key for the current software has been verified as ║
senseable-trillama-8b-v1-mkmlizer: ║ belonging to: ║
senseable-trillama-8b-v1-mkmlizer: ║ ║
senseable-trillama-8b-v1-mkmlizer: ║ Chai Research Corp. ║
senseable-trillama-8b-v1-mkmlizer: ║ Account ID: 7997a29f-0ceb-4cc7-9adf-840c57b4ae6f ║
senseable-trillama-8b-v1-mkmlizer: ║ Expiration: 2024-07-15 23:59:59 ║
senseable-trillama-8b-v1-mkmlizer: ║ ║
senseable-trillama-8b-v1-mkmlizer: ╚═════════════════════════════════════════════════════════════════════╝
senseable-trillama-8b-v1-mkmlizer:
.gitattributes: 0%| | 0.00/1.52k [00:00<?, ?B/s]
.gitattributes: 100%|██████████| 1.52k/1.52k [00:00<00:00, 15.7MB/s]
senseable-trillama-8b-v1-mkmlizer:
README.md: 0%| | 0.00/567 [00:00<?, ?B/s]
README.md: 100%|██████████| 567/567 [00:00<00:00, 4.75MB/s]
senseable-trillama-8b-v1-mkmlizer:
config.json: 0%| | 0.00/654 [00:00<?, ?B/s]
config.json: 100%|██████████| 654/654 [00:00<00:00, 5.48MB/s]
senseable-trillama-8b-v1-mkmlizer:
model-00001-of-00004.safetensors: 0%| | 0.00/4.98G [00:00<?, ?B/s]
model-00001-of-00004.safetensors: 0%| | 10.5M/4.98G [00:00<04:02, 20.5MB/s]
model-00001-of-00004.safetensors: 0%| | 21.0M/4.98G [00:00<02:12, 37.3MB/s]
model-00001-of-00004.safetensors: 1%| | 52.4M/4.98G [00:00<00:52, 94.1MB/s]
model-00001-of-00004.safetensors: 1%|▏ | 73.4M/4.98G [00:00<00:40, 120MB/s]
model-00001-of-00004.safetensors: 2%|▏ | 115M/4.98G [00:00<00:26, 186MB/s]
model-00001-of-00004.safetensors: 3%|▎ | 147M/4.98G [00:01<00:31, 151MB/s]
model-00001-of-00004.safetensors: 4%|▍ | 199M/4.98G [00:01<00:25, 188MB/s]
model-00001-of-00004.safetensors: 5%|▌ | 252M/4.98G [00:01<00:19, 245MB/s]
model-00001-of-00004.safetensors: 6%|▌ | 294M/4.98G [00:01<00:16, 278MB/s]
model-00001-of-00004.safetensors: 8%|▊ | 377M/4.98G [00:01<00:11, 397MB/s]
model-00001-of-00004.safetensors: 9%|▊ | 430M/4.98G [00:01<00:10, 423MB/s]
model-00001-of-00004.safetensors: 11%|█ | 556M/4.98G [00:02<00:07, 624MB/s]
model-00001-of-00004.safetensors: 13%|█▎ | 629M/4.98G [00:02<00:07, 549MB/s]
model-00001-of-00004.safetensors: 15%|█▌ | 765M/4.98G [00:02<00:05, 742MB/s]
model-00001-of-00004.safetensors: 17%|█▋ | 849M/4.98G [00:02<00:05, 721MB/s]
model-00001-of-00004.safetensors: 19%|█▉ | 933M/4.98G [00:02<00:05, 694MB/s]
model-00001-of-00004.safetensors: 21%|██▏ | 1.07G/4.98G [00:02<00:04, 819MB/s]
model-00001-of-00004.safetensors: 24%|██▎ | 1.17G/4.98G [00:02<00:04, 867MB/s]
model-00001-of-00004.safetensors: 28%|██▊ | 1.38G/4.98G [00:02<00:03, 1.19GB/s]
model-00001-of-00004.safetensors: 34%|███▍ | 1.69G/4.98G [00:02<00:01, 1.67GB/s]
model-00001-of-00004.safetensors: 38%|███▊ | 1.87G/4.98G [00:03<00:02, 1.12GB/s]
model-00001-of-00004.safetensors: 40%|████ | 2.01G/4.98G [00:03<00:02, 1.05GB/s]
model-00001-of-00004.safetensors: 43%|████▎ | 2.14G/4.98G [00:03<00:03, 827MB/s]
model-00001-of-00004.safetensors: 45%|████▌ | 2.24G/4.98G [00:03<00:03, 684MB/s]
model-00001-of-00004.safetensors: 48%|████▊ | 2.39G/4.98G [00:04<00:03, 800MB/s]
model-00001-of-00004.safetensors: 50%|█████ | 2.50G/4.98G [00:04<00:03, 739MB/s]
model-00001-of-00004.safetensors: 52%|█████▏ | 2.59G/4.98G [00:04<00:03, 716MB/s]
model-00001-of-00004.safetensors: 55%|█████▍ | 2.72G/4.98G [00:04<00:02, 771MB/s]
model-00001-of-00004.safetensors: 56%|█████▋ | 2.81G/4.98G [00:04<00:03, 702MB/s]
model-00001-of-00004.safetensors: 58%|█████▊ | 2.89G/4.98G [00:04<00:03, 685MB/s]
model-00001-of-00004.safetensors: 60%|█████▉ | 2.97G/4.98G [00:04<00:02, 694MB/s]
model-00001-of-00004.safetensors: 61%|██████▏ | 3.05G/4.98G [00:05<00:02, 703MB/s]
model-00001-of-00004.safetensors: 64%|██████▍ | 3.19G/4.98G [00:05<00:02, 854MB/s]
model-00001-of-00004.safetensors: 66%|██████▌ | 3.28G/4.98G [00:05<00:02, 820MB/s]
model-00001-of-00004.safetensors: 68%|██████▊ | 3.40G/4.98G [00:05<00:01, 791MB/s]
model-00001-of-00004.safetensors: 70%|███████ | 3.50G/4.98G [00:05<00:01, 818MB/s]
model-00001-of-00004.safetensors: 72%|███████▏ | 3.60G/4.98G [00:05<00:01, 847MB/s]
model-00001-of-00004.safetensors: 75%|███████▍ | 3.72G/4.98G [00:05<00:01, 951MB/s]
model-00001-of-00004.safetensors: 77%|███████▋ | 3.83G/4.98G [00:06<00:01, 662MB/s]
model-00001-of-00004.safetensors: 79%|███████▉ | 3.93G/4.98G [00:06<00:01, 660MB/s]
model-00001-of-00004.safetensors: 81%|████████ | 4.04G/4.98G [00:06<00:01, 732MB/s]
model-00001-of-00004.safetensors: 83%|████████▎ | 4.12G/4.98G [00:06<00:01, 738MB/s]
model-00001-of-00004.safetensors: 85%|████████▍ | 4.23G/4.98G [00:06<00:00, 791MB/s]
model-00001-of-00004.safetensors: 87%|████████▋ | 4.32G/4.98G [00:06<00:00, 765MB/s]
model-00001-of-00004.safetensors: 91%|█████████ | 4.51G/4.98G [00:06<00:00, 1.04GB/s]
model-00001-of-00004.safetensors: 93%|█████████▎| 4.62G/4.98G [00:06<00:00, 1.06GB/s]
model-00001-of-00004.safetensors: 97%|█████████▋| 4.84G/4.98G [00:06<00:00, 1.29GB/s]
model-00001-of-00004.safetensors: 100%|█████████▉| 4.98G/4.98G [00:07<00:00, 707MB/s]
senseable-trillama-8b-v1-mkmlizer:
model-00002-of-00004.safetensors: 0%| | 0.00/5.00G [00:00<?, ?B/s]
model-00002-of-00004.safetensors: 0%| | 10.5M/5.00G [00:00<04:56, 16.8MB/s]
model-00002-of-00004.safetensors: 0%| | 21.0M/5.00G [00:00<03:13, 25.7MB/s]
model-00002-of-00004.safetensors: 1%| | 52.4M/5.00G [00:01<01:08, 72.3MB/s]
model-00002-of-00004.safetensors: 1%|▏ | 73.4M/5.00G [00:01<00:53, 92.6MB/s]
model-00002-of-00004.safetensors: 3%|▎ | 126M/5.00G [00:01<00:51, 93.8MB/s]
model-00002-of-00004.safetensors: 3%|▎ | 147M/5.00G [00:01<00:44, 109MB/s]
model-00002-of-00004.safetensors: 5%|▌ | 262M/5.00G [00:01<00:17, 271MB/s]
model-00002-of-00004.safetensors: 6%|▋ | 315M/5.00G [00:02<00:19, 240MB/s]
model-00002-of-00004.safetensors: 7%|▋ | 357M/5.00G [00:02<00:21, 221MB/s]
model-00002-of-00004.safetensors: 9%|▉ | 472M/5.00G [00:02<00:12, 351MB/s]
model-00002-of-00004.safetensors: 11%|█▏ | 566M/5.00G [00:02<00:09, 451MB/s]
model-00002-of-00004.safetensors: 13%|█▎ | 671M/5.00G [00:02<00:07, 542MB/s]
model-00002-of-00004.safetensors: 16%|█▌ | 807M/5.00G [00:02<00:06, 682MB/s]
model-00002-of-00004.safetensors: 18%|█▊ | 891M/5.00G [00:03<00:06, 616MB/s]
model-00002-of-00004.safetensors: 24%|██▍ | 1.22G/5.00G [00:03<00:03, 1.19GB/s]
model-00002-of-00004.safetensors: 29%|██▉ | 1.45G/5.00G [00:03<00:03, 1.08GB/s]
model-00002-of-00004.safetensors: 32%|███▏ | 1.58G/5.00G [00:03<00:04, 715MB/s]
model-00002-of-00004.safetensors: 34%|███▍ | 1.69G/5.00G [00:03<00:04, 679MB/s]
model-00002-of-00004.safetensors: 36%|███▋ | 1.82G/5.00G [00:04<00:04, 789MB/s]
model-00002-of-00004.safetensors: 39%|███▊ | 1.93G/5.00G [00:04<00:03, 801MB/s]
model-00002-of-00004.safetensors: 41%|████ | 2.03G/5.00G [00:04<00:03, 819MB/s]
model-00002-of-00004.safetensors: 43%|████▎ | 2.17G/5.00G [00:04<00:03, 935MB/s]
model-00002-of-00004.safetensors: 46%|████▌ | 2.29G/5.00G [00:04<00:03, 817MB/s]
model-00002-of-00004.safetensors: 48%|████▊ | 2.38G/5.00G [00:04<00:03, 700MB/s]
model-00002-of-00004.safetensors: 50%|█████ | 2.51G/5.00G [00:04<00:03, 799MB/s]
model-00002-of-00004.safetensors: 53%|█████▎ | 2.64G/5.00G [00:05<00:02, 876MB/s]
model-00002-of-00004.safetensors: 57%|█████▋ | 2.86G/5.00G [00:05<00:01, 1.18GB/s]
model-00002-of-00004.safetensors: 62%|██████▏ | 3.09G/5.00G [00:05<00:01, 1.36GB/s]
model-00002-of-00004.safetensors: 65%|██████▍ | 3.24G/5.00G [00:05<00:01, 1.01GB/s]
model-00002-of-00004.safetensors: 67%|██████▋ | 3.37G/5.00G [00:05<00:01, 864MB/s]
model-00002-of-00004.safetensors: 69%|██████▉ | 3.47G/5.00G [00:05<00:01, 859MB/s]
model-00002-of-00004.safetensors: 72%|███████▏ | 3.58G/5.00G [00:06<00:01, 856MB/s]
model-00002-of-00004.safetensors: 73%|███████▎ | 3.67G/5.00G [00:06<00:01, 839MB/s]
model-00002-of-00004.safetensors: 75%|███████▌ | 3.76G/5.00G [00:06<00:01, 848MB/s]
model-00002-of-00004.safetensors: 77%|███████▋ | 3.86G/5.00G [00:06<00:01, 835MB/s]
model-00002-of-00004.safetensors: 80%|████████ | 4.01G/5.00G [00:06<00:01, 991MB/s]
model-00002-of-00004.safetensors: 82%|████████▏ | 4.12G/5.00G [00:06<00:00, 958MB/s]
model-00002-of-00004.safetensors: 85%|████████▍ | 4.23G/5.00G [00:06<00:00, 960MB/s]
model-00002-of-00004.safetensors: 87%|████████▋ | 4.33G/5.00G [00:06<00:00, 901MB/s]
model-00002-of-00004.safetensors: 89%|████████▉ | 4.46G/5.00G [00:06<00:00, 978MB/s]
model-00002-of-00004.safetensors: 92%|█████████▏| 4.62G/5.00G [00:07<00:00, 1.16GB/s]
model-00002-of-00004.safetensors: 95%|█████████▌| 4.77G/5.00G [00:07<00:00, 1.22GB/s]
model-00002-of-00004.safetensors: 98%|█████████▊| 4.90G/5.00G [00:07<00:00, 1.22GB/s]
model-00002-of-00004.safetensors: 100%|█████████▉| 5.00G/5.00G [00:07<00:00, 635MB/s]
senseable-trillama-8b-v1-mkmlizer:
model-00003-of-00004.safetensors: 0%| | 0.00/4.92G [00:00<?, ?B/s]
model-00003-of-00004.safetensors: 0%| | 10.5M/4.92G [00:00<04:42, 17.4MB/s]
model-00003-of-00004.safetensors: 1%| | 41.9M/4.92G [00:00<01:15, 64.3MB/s]
model-00003-of-00004.safetensors: 1%|▏ | 62.9M/4.92G [00:00<01:01, 79.2MB/s]
model-00003-of-00004.safetensors: 2%|▏ | 83.9M/4.92G [00:01<00:57, 84.0MB/s]
model-00003-of-00004.safetensors: 2%|▏ | 105M/4.92G [00:01<00:46, 103MB/s]
model-00003-of-00004.safetensors: 3%|▎ | 126M/4.92G [00:01<00:45, 105MB/s]
model-00003-of-00004.safetensors: 3%|▎ | 168M/4.92G [00:01<00:30, 158MB/s]
model-00003-of-00004.safetensors: 4%|▍ | 199M/4.92G [00:01<00:38, 123MB/s]
model-00003-of-00004.safetensors: 4%|▍ | 220M/4.92G [00:02<00:39, 120MB/s]
model-00003-of-00004.safetensors: 6%|▌ | 294M/4.92G [00:02<00:20, 222MB/s]
model-00003-of-00004.safetensors: 8%|▊ | 388M/4.92G [00:02<00:15, 288MB/s]
model-00003-of-00004.safetensors: 9%|▉ | 440M/4.92G [00:02<00:13, 330MB/s]
model-00003-of-00004.safetensors: 10%|█ | 503M/4.92G [00:02<00:11, 380MB/s]
model-00003-of-00004.safetensors: 13%|█▎ | 629M/4.92G [00:02<00:08, 534MB/s]
model-00003-of-00004.safetensors: 15%|█▌ | 744M/4.92G [00:02<00:06, 659MB/s]
model-00003-of-00004.safetensors: 17%|█▋ | 818M/4.92G [00:03<00:07, 558MB/s]
model-00003-of-00004.safetensors: 19%|█▉ | 933M/4.92G [00:03<00:05, 688MB/s]
model-00003-of-00004.safetensors: 27%|██▋ | 1.33G/4.92G [00:03<00:02, 1.48GB/s]
model-00003-of-00004.safetensors: 31%|███ | 1.51G/4.92G [00:03<00:02, 1.36GB/s]
model-00003-of-00004.safetensors: 34%|███▍ | 1.67G/4.92G [00:03<00:02, 1.13GB/s]
model-00003-of-00004.safetensors: 37%|███▋ | 1.80G/4.92G [00:04<00:04, 676MB/s]
model-00003-of-00004.safetensors: 39%|███▉ | 1.91G/4.92G [00:04<00:06, 488MB/s]
model-00003-of-00004.safetensors: 41%|████ | 1.99G/4.92G [00:04<00:05, 521MB/s]
model-00003-of-00004.safetensors: 42%|████▏ | 2.09G/4.92G [00:04<00:05, 561MB/s]
model-00003-of-00004.safetensors: 45%|████▍ | 2.19G/4.92G [00:04<00:04, 629MB/s]
model-00003-of-00004.safetensors: 46%|████▋ | 2.28G/4.92G [00:05<00:04, 643MB/s]
model-00003-of-00004.safetensors: 49%|████▉ | 2.42G/4.92G [00:05<00:03, 811MB/s]
model-00003-of-00004.safetensors: 51%|█████▏ | 2.53G/4.92G [00:05<00:03, 721MB/s]
model-00003-of-00004.safetensors: 55%|█████▍ | 2.69G/4.92G [00:05<00:02, 894MB/s]
model-00003-of-00004.safetensors: 57%|█████▋ | 2.80G/4.92G [00:05<00:02, 885MB/s]
model-00003-of-00004.safetensors: 59%|█████▉ | 2.90G/4.92G [00:05<00:03, 652MB/s]
model-00003-of-00004.safetensors: 61%|██████ | 2.99G/4.92G [00:06<00:04, 447MB/s]
model-00003-of-00004.safetensors: 63%|██████▎ | 3.09G/4.92G [00:06<00:03, 524MB/s]
model-00003-of-00004.safetensors: 64%|██████▍ | 3.17G/4.92G [00:06<00:03, 483MB/s]
model-00003-of-00004.safetensors: 66%|██████▌ | 3.23G/4.92G [00:06<00:03, 508MB/s]
model-00003-of-00004.safetensors: 67%|██████▋ | 3.29G/4.92G [00:06<00:03, 511MB/s]
model-00003-of-00004.safetensors: 69%|██████▊ | 3.38G/4.92G [00:06<00:02, 568MB/s]
model-00003-of-00004.safetensors: 72%|███████▏ | 3.52G/4.92G [00:07<00:02, 643MB/s]
model-00003-of-00004.safetensors: 74%|███████▍ | 3.66G/4.92G [00:07<00:01, 797MB/s]
model-00003-of-00004.safetensors: 77%|███████▋ | 3.79G/4.92G [00:07<00:01, 862MB/s]
model-00003-of-00004.safetensors: 79%|███████▉ | 3.88G/4.92G [00:07<00:01, 861MB/s]
model-00003-of-00004.safetensors: 81%|████████ | 3.97G/4.92G [00:07<00:01, 628MB/s]
model-00003-of-00004.safetensors: 83%|████████▎ | 4.06G/4.92G [00:07<00:01, 503MB/s]
model-00003-of-00004.safetensors: 84%|████████▍ | 4.15G/4.92G [00:08<00:01, 559MB/s]
model-00003-of-00004.safetensors: 86%|████████▋ | 4.25G/4.92G [00:08<00:01, 600MB/s]
model-00003-of-00004.safetensors: 88%|████████▊ | 4.32G/4.92G [00:08<00:00, 620MB/s]
model-00003-of-00004.safetensors: 90%|█████████ | 4.44G/4.92G [00:08<00:00, 688MB/s]
model-00003-of-00004.safetensors: 92%|█████████▏| 4.54G/4.92G [00:08<00:00, 746MB/s]
model-00003-of-00004.safetensors: 96%|█████████▌| 4.71G/4.92G [00:08<00:00, 958MB/s]
model-00003-of-00004.safetensors: 100%|█████████▉| 4.92G/4.92G [00:08<00:00, 559MB/s]
senseable-trillama-8b-v1-mkmlizer:
model-00004-of-00004.safetensors: 0%| | 0.00/1.17G [00:00<?, ?B/s]
model-00004-of-00004.safetensors: 1%| | 10.5M/1.17G [00:00<01:24, 13.7MB/s]
model-00004-of-00004.safetensors: 6%|▋ | 73.4M/1.17G [00:00<00:10, 109MB/s]
model-00004-of-00004.safetensors: 10%|▉ | 115M/1.17G [00:01<00:10, 104MB/s]
model-00004-of-00004.safetensors: 13%|█▎ | 157M/1.17G [00:01<00:07, 140MB/s]
model-00004-of-00004.safetensors: 16%|█▌ | 189M/1.17G [00:01<00:07, 124MB/s]
model-00004-of-00004.safetensors: 23%|██▎ | 266M/1.17G [00:01<00:04, 215MB/s]
model-00004-of-00004.safetensors: 28%|██▊ | 329M/1.17G [00:02<00:03, 231MB/s]
model-00004-of-00004.safetensors: 32%|███▏ | 371M/1.17G [00:02<00:03, 214MB/s]
model-00004-of-00004.safetensors: 34%|███▍ | 403M/1.17G [00:02<00:03, 208MB/s]
model-00004-of-00004.safetensors: 37%|███▋ | 434M/1.17G [00:02<00:03, 195MB/s]
model-00004-of-00004.safetensors: 44%|████▍ | 518M/1.17G [00:02<00:02, 305MB/s]
model-00004-of-00004.safetensors: 53%|█████▎ | 623M/1.17G [00:02<00:01, 451MB/s]
model-00004-of-00004.safetensors: 59%|█████▊ | 686M/1.17G [00:03<00:01, 416MB/s]
model-00004-of-00004.safetensors: 100%|█████████▉| 1.17G/1.17G [00:03<00:00, 359MB/s]
senseable-trillama-8b-v1-mkmlizer:
model.safetensors.index.json: 0%| | 0.00/23.9k [00:00<?, ?B/s]
model.safetensors.index.json: 100%|██████████| 23.9k/23.9k [00:00<00:00, 175MB/s]
senseable-trillama-8b-v1-mkmlizer:
params.json: 0%| | 0.00/211 [00:00<?, ?B/s]
params.json: 100%|██████████| 211/211 [00:00<00:00, 3.31MB/s]
senseable-trillama-8b-v1-mkmlizer:
special_tokens_map.json: 0%| | 0.00/73.0 [00:00<?, ?B/s]
special_tokens_map.json: 100%|██████████| 73.0/73.0 [00:00<00:00, 1.13MB/s]
senseable-trillama-8b-v1-mkmlizer:
tokenizer.json: 0%| | 0.00/9.08M [00:00<?, ?B/s]
tokenizer.json: 100%|██████████| 9.08M/9.08M [00:00<00:00, 27.3MB/s]
tokenizer.json: 100%|██████████| 9.08M/9.08M [00:00<00:00, 27.0MB/s]
senseable-trillama-8b-v1-mkmlizer:
tokenizer_config.json: 0%| | 0.00/50.9k [00:00<?, ?B/s]
tokenizer_config.json: 100%|██████████| 50.9k/50.9k [00:00<00:00, 204MB/s]
senseable-trillama-8b-v1-mkmlizer: Downloaded to shared memory in 29.400s
senseable-trillama-8b-v1-mkmlizer: quantizing model to /dev/shm/model_cache
senseable-trillama-8b-v1-mkmlizer: Saving mkml model at /dev/shm/model_cache
senseable-trillama-8b-v1-mkmlizer: Reading /tmp/tmpcv_orqxh/model.safetensors.index.json
senseable-trillama-8b-v1-mkmlizer:
Profiling: 0%| | 0/291 [00:00<?, ?it/s]
Profiling: 0%| | 1/291 [00:05<24:46, 5.13s/it]
Profiling: 4%|▍ | 13/291 [00:05<01:20, 3.44it/s]
Profiling: 11%|█ | 31/291 [00:05<00:26, 9.98it/s]
Profiling: 17%|█▋ | 49/291 [00:05<00:13, 18.46it/s]
Profiling: 23%|██▎ | 67/291 [00:05<00:07, 29.11it/s]
Profiling: 29%|██▊ | 83/291 [00:05<00:06, 32.44it/s]
Profiling: 34%|███▎ | 98/291 [00:06<00:04, 42.96it/s]
Profiling: 38%|███▊ | 111/291 [00:06<00:03, 52.02it/s]
Profiling: 42%|████▏ | 123/291 [00:06<00:02, 58.64it/s]
Profiling: 47%|████▋ | 138/291 [00:06<00:02, 69.78it/s]
Profiling: 54%|█████▎ | 156/291 [00:06<00:01, 88.20it/s]
Profiling: 59%|█████▉ | 173/291 [00:06<00:01, 102.72it/s]
Profiling: 64%|██████▍ | 187/291 [00:06<00:01, 71.62it/s]
Profiling: 69%|██████▉ | 201/291 [00:07<00:01, 82.92it/s]
Profiling: 75%|███████▌ | 219/291 [00:07<00:00, 100.40it/s]
Profiling: 81%|████████▏ | 237/291 [00:07<00:00, 115.60it/s]
Profiling: 88%|████████▊ | 255/291 [00:07<00:00, 126.60it/s]
Profiling: 94%|█████████▍| 273/291 [00:07<00:00, 137.56it/s]
Profiling: 99%|█████████▉| 289/291 [00:13<00:00, 9.18it/s]
Profiling: 100%|██████████| 291/291 [00:13<00:00, 21.66it/s]
senseable-trillama-8b-v1-mkmlizer: Special tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained.
senseable-trillama-8b-v1-mkmlizer: quantized model in 27.627s
senseable-trillama-8b-v1-mkmlizer: Processed model senseable/Trillama-8B in 58.561s
senseable-trillama-8b-v1-mkmlizer: creating bucket guanaco-mkml-models
senseable-trillama-8b-v1-mkmlizer: Bucket 's3://guanaco-mkml-models/' created
senseable-trillama-8b-v1-mkmlizer: uploading /dev/shm/model_cache to s3://guanaco-mkml-models/senseable-trillama-8b-v1
senseable-trillama-8b-v1-mkmlizer: cp /dev/shm/model_cache/special_tokens_map.json s3://guanaco-mkml-models/senseable-trillama-8b-v1/special_tokens_map.json
senseable-trillama-8b-v1-mkmlizer: cp /dev/shm/model_cache/config.json s3://guanaco-mkml-models/senseable-trillama-8b-v1/config.json
senseable-trillama-8b-v1-mkmlizer: cp /dev/shm/model_cache/tokenizer_config.json s3://guanaco-mkml-models/senseable-trillama-8b-v1/tokenizer_config.json
senseable-trillama-8b-v1-mkmlizer: cp /dev/shm/model_cache/tokenizer.json s3://guanaco-mkml-models/senseable-trillama-8b-v1/tokenizer.json
senseable-trillama-8b-v1-mkmlizer: cp /dev/shm/model_cache/mkml_model.tensors s3://guanaco-mkml-models/senseable-trillama-8b-v1/mkml_model.tensors
senseable-trillama-8b-v1-mkmlizer: loading reward model from ChaiML/reward_gpt2_medium_preference_24m_e2
senseable-trillama-8b-v1-mkmlizer: /opt/conda/lib/python3.10/site-packages/transformers/models/auto/configuration_auto.py:1067: FutureWarning: The `use_auth_token` argument is deprecated and will be removed in v5 of Transformers. Please use `token` instead.
senseable-trillama-8b-v1-mkmlizer: warnings.warn(
senseable-trillama-8b-v1-mkmlizer:
config.json: 0%| | 0.00/1.05k [00:00<?, ?B/s]
config.json: 100%|██████████| 1.05k/1.05k [00:00<00:00, 11.6MB/s]
senseable-trillama-8b-v1-mkmlizer: /opt/conda/lib/python3.10/site-packages/transformers/models/auto/tokenization_auto.py:690: FutureWarning: The `use_auth_token` argument is deprecated and will be removed in v5 of Transformers. Please use `token` instead.
senseable-trillama-8b-v1-mkmlizer: warnings.warn(
senseable-trillama-8b-v1-mkmlizer:
tokenizer_config.json: 0%| | 0.00/234 [00:00<?, ?B/s]
tokenizer_config.json: 100%|██████████| 234/234 [00:00<00:00, 2.93MB/s]
senseable-trillama-8b-v1-mkmlizer:
vocab.json: 0%| | 0.00/1.04M [00:00<?, ?B/s]
vocab.json: 100%|██████████| 1.04M/1.04M [00:00<00:00, 14.3MB/s]
senseable-trillama-8b-v1-mkmlizer:
tokenizer.json: 0%| | 0.00/2.11M [00:00<?, ?B/s]
tokenizer.json: 100%|██████████| 2.11M/2.11M [00:00<00:00, 17.3MB/s]
tokenizer.json: 100%|██████████| 2.11M/2.11M [00:00<00:00, 17.2MB/s]
senseable-trillama-8b-v1-mkmlizer: /opt/conda/lib/python3.10/site-packages/transformers/models/auto/auto_factory.py:472: FutureWarning: The `use_auth_token` argument is deprecated and will be removed in v5 of Transformers. Please use `token` instead.
senseable-trillama-8b-v1-mkmlizer: warnings.warn(
senseable-trillama-8b-v1-mkmlizer:
pytorch_model.bin: 0%| | 0.00/1.44G [00:00<?, ?B/s]
pytorch_model.bin: 1%|▏ | 21.0M/1.44G [00:00<00:12, 112MB/s]
pytorch_model.bin: 3%|▎ | 41.9M/1.44G [00:00<00:14, 98.1MB/s]
pytorch_model.bin: 15%|█▍ | 210M/1.44G [00:01<00:05, 220MB/s]
pytorch_model.bin: 28%|██▊ | 409M/1.44G [00:01<00:02, 467MB/s]
pytorch_model.bin: 52%|█████▏ | 755M/1.44G [00:01<00:00, 960MB/s]
pytorch_model.bin: 69%|██████▉ | 996M/1.44G [00:01<00:00, 1.01GB/s]
pytorch_model.bin: 84%|████████▍ | 1.21G/1.44G [00:01<00:00, 1.21GB/s]
pytorch_model.bin: 100%|█████████▉| 1.44G/1.44G [00:01<00:00, 875MB/s]
senseable-trillama-8b-v1-mkmlizer: Saving model to /tmp/reward_cache/reward.tensors
senseable-trillama-8b-v1-mkmlizer: Saving duration: 0.330s
senseable-trillama-8b-v1-mkmlizer: Processed model ChaiML/reward_gpt2_medium_preference_24m_e2 in 6.086s
senseable-trillama-8b-v1-mkmlizer: cp /tmp/reward_cache/reward.tensors s3://guanaco-reward-models/senseable-trillama-8b-v1_reward/reward.tensors
Job senseable-trillama-8b-v1-mkmlizer completed after 94.66s with status: succeeded
Stopping job with name senseable-trillama-8b-v1-mkmlizer
Pipeline stage MKMLizer completed in 98.01s
Running pipeline stage MKMLKubeTemplater
Pipeline stage MKMLKubeTemplater completed in 0.10s
Running pipeline stage ISVCDeployer
Creating inference service senseable-trillama-8b-v1
Waiting for inference service senseable-trillama-8b-v1 to be ready
Inference service senseable-trillama-8b-v1 ready after 40.25101327896118s
Pipeline stage ISVCDeployer completed in 47.24s
Running pipeline stage StressChecker
Received healthy response to inference request in 0.9059233665466309s
Received healthy response to inference request in 0.4433135986328125s
Received healthy response to inference request in 0.7596330642700195s
Received healthy response to inference request in 0.7819831371307373s
Received healthy response to inference request in 0.743391752243042s
5 requests
0 failed requests
5th percentile: 0.5033292293548584
10th percentile: 0.5633448600769043
20th percentile: 0.6833761215209961
30th percentile: 0.7466400146484375
40th percentile: 0.7531365394592285
50th percentile: 0.7596330642700195
60th percentile: 0.7685730934143067
70th percentile: 0.7775131225585937
80th percentile: 0.806771183013916
90th percentile: 0.8563472747802734
95th percentile: 0.8811353206634521
99th percentile: 0.9009657573699951
mean time: 0.7268489837646485
Pipeline stage StressChecker completed in 5.55s
Running pipeline stage DaemonicModelEvalScorer
Pipeline stage DaemonicModelEvalScorer completed in 0.04s
Running pipeline stage DaemonicSafetyScorer
Running M-Eval for topic stay_in_character
Pipeline stage DaemonicSafetyScorer completed in 0.04s
M-Eval Dataset for topic stay_in_character is loaded
senseable-trillama-8b_v1 status is now deployed due to DeploymentManager action
senseable-trillama-8b_v1 status is now inactive due to auto deactivation removed underperforming models
admin requested tearing down of senseable-trillama-8b_v1
Running pipeline stage ISVCDeleter
Checking if service senseable-trillama-8b-v1 is running
Tearing down inference service senseable-trillama-8b-v1
Toredown service senseable-trillama-8b-v1
Pipeline stage ISVCDeleter completed in 3.57s
Running pipeline stage MKMLModelDeleter
Cleaning model data from S3
Cleaning model data from model cache
Deleting key senseable-trillama-8b-v1/config.json from bucket guanaco-mkml-models
Deleting key senseable-trillama-8b-v1/mkml_model.tensors from bucket guanaco-mkml-models
Deleting key senseable-trillama-8b-v1/special_tokens_map.json from bucket guanaco-mkml-models
Deleting key senseable-trillama-8b-v1/tokenizer.json from bucket guanaco-mkml-models
Deleting key senseable-trillama-8b-v1/tokenizer_config.json from bucket guanaco-mkml-models
Cleaning model data from model cache
Deleting key senseable-trillama-8b-v1_reward/config.json from bucket guanaco-reward-models
Deleting key senseable-trillama-8b-v1_reward/merges.txt from bucket guanaco-reward-models
Deleting key senseable-trillama-8b-v1_reward/reward.tensors from bucket guanaco-reward-models
Deleting key senseable-trillama-8b-v1_reward/special_tokens_map.json from bucket guanaco-reward-models
Deleting key senseable-trillama-8b-v1_reward/tokenizer.json from bucket guanaco-reward-models
Deleting key senseable-trillama-8b-v1_reward/tokenizer_config.json from bucket guanaco-reward-models
Deleting key senseable-trillama-8b-v1_reward/vocab.json from bucket guanaco-reward-models
Pipeline stage MKMLModelDeleter completed in 2.30s
senseable-trillama-8b_v1 status is now torndown due to DeploymentManager action