Shutdown handler not registered because Python interpreter is not running in the main thread
run pipeline %s
run pipeline stage %s
Running pipeline stage MKMLizer
Starting job with name mistralai-mixtral-8x22b-4383-v2-mkmlizer
Waiting for job on mistralai-mixtral-8x22b-4383-v2-mkmlizer to finish
mistralai-mixtral-8x22b-4383-v2-mkmlizer: ╔═════════════════════════════════════════════════════════════════════╗
mistralai-mixtral-8x22b-4383-v2-mkmlizer: ║ _____ __ __ ║
mistralai-mixtral-8x22b-4383-v2-mkmlizer: ║ / _/ /_ ___ __/ / ___ ___ / / ║
mistralai-mixtral-8x22b-4383-v2-mkmlizer: ║ / _/ / // / |/|/ / _ \/ -_) -_) / ║
mistralai-mixtral-8x22b-4383-v2-mkmlizer: ║ /_//_/\_, /|__,__/_//_/\__/\__/_/ ║
mistralai-mixtral-8x22b-4383-v2-mkmlizer: ║ /___/ ║
mistralai-mixtral-8x22b-4383-v2-mkmlizer: ║ ║
mistralai-mixtral-8x22b-4383-v2-mkmlizer: ║ Version: 0.11.12 ║
mistralai-mixtral-8x22b-4383-v2-mkmlizer: ║ Copyright 2023 MK ONE TECHNOLOGIES Inc. ║
mistralai-mixtral-8x22b-4383-v2-mkmlizer: ║ https://mk1.ai ║
mistralai-mixtral-8x22b-4383-v2-mkmlizer: ║ ║
mistralai-mixtral-8x22b-4383-v2-mkmlizer: ║ The license key for the current software has been verified as ║
mistralai-mixtral-8x22b-4383-v2-mkmlizer: ║ belonging to: ║
mistralai-mixtral-8x22b-4383-v2-mkmlizer: ║ ║
mistralai-mixtral-8x22b-4383-v2-mkmlizer: ║ Chai Research Corp. ║
mistralai-mixtral-8x22b-4383-v2-mkmlizer: ║ Account ID: 7997a29f-0ceb-4cc7-9adf-840c57b4ae6f ║
mistralai-mixtral-8x22b-4383-v2-mkmlizer: ║ Expiration: 2025-01-15 23:59:59 ║
mistralai-mixtral-8x22b-4383-v2-mkmlizer: ║ ║
mistralai-mixtral-8x22b-4383-v2-mkmlizer: ╚═════════════════════════════════════════════════════════════════════╝
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Failed to get response for submission zonemercy-viral-ss-v0-1e5ep2_v66: ('http://zonemercy-viral-ss-v0-1e5ep2-v66-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'read tcp 127.0.0.1:53506->127.0.0.1:8080: read: connection reset by peer\n')
Connection pool is full, discarding connection: %s. Connection pool size: %s
mistralai-mixtral-8x22b-4383-v2-mkmlizer: Downloaded to shared memory in 490.662s
mistralai-mixtral-8x22b-4383-v2-mkmlizer: quantizing model to /dev/shm/model_cache, profile:s0, folder:/tmp/tmpoh1sw8d6, device:0
mistralai-mixtral-8x22b-4383-v2-mkmlizer: Saving flywheel model at /dev/shm/model_cache
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
mistralai-mixtral-8x22b-4383-v2-mkmlizer:
Loading 0: 0%| | 0/1739 [00:00<?, ?it/s]
Loading 0: 0%| | 4/1739 [00:00<01:31, 18.92it/s]
Loading 0: 0%| | 7/1739 [00:00<01:34, 18.32it/s]
Loading 0: 1%| | 10/1739 [00:00<01:37, 17.82it/s]
Loading 0: 1%| | 13/1739 [00:00<01:40, 17.24it/s]
Loading 0: 1%| | 16/1739 [00:00<01:43, 16.57it/s]
Loading 0: 1%| | 19/1739 [00:01<01:46, 16.13it/s]
Loading 0: 1%|▏ | 22/1739 [00:01<01:47, 16.02it/s]
Loading 0: 2%|▏ | 27/1739 [00:01<01:16, 22.39it/s]
Loading 0: 2%|▏ | 30/1739 [00:01<02:04, 13.74it/s]
Loading 0: 2%|▏ | 35/1739 [00:02<01:41, 16.72it/s]
Loading 0: 2%|▏ | 38/1739 [00:02<01:42, 16.54it/s]
Loading 0: 2%|▏ | 41/1739 [00:02<01:41, 16.79it/s]
Loading 0: 3%|▎ | 44/1739 [00:02<01:41, 16.63it/s]
Loading 0: 3%|▎ | 47/1739 [00:02<01:43, 16.34it/s]
Loading 0: 3%|▎ | 50/1739 [00:02<01:41, 16.70it/s]
Loading 0: 3%|▎ | 56/1739 [00:03<01:08, 24.64it/s]
Loading 0: 3%|▎ | 59/1739 [00:03<01:50, 15.15it/s]
Loading 0: 4%|▎ | 62/1739 [00:03<01:43, 16.24it/s]
Loading 0: 4%|▍ | 66/1739 [00:03<01:34, 17.73it/s]
Loading 0: 4%|▍ | 69/1739 [00:04<01:34, 17.67it/s]
Loading 0: 4%|▍ | 72/1739 [00:04<01:35, 17.39it/s]
Loading 0: 4%|▍ | 75/1739 [00:04<01:35, 17.36it/s]
Loading 0: 4%|▍ | 78/1739 [00:04<01:36, 17.16it/s]
Loading 0: 5%|▍ | 81/1739 [00:04<01:36, 17.12it/s]
Loading 0: 5%|▌ | 87/1739 [00:04<01:10, 23.46it/s]
Loading 0: 5%|▌ | 90/1739 [00:05<01:43, 15.93it/s]
Loading 0: 5%|▌ | 93/1739 [00:05<01:36, 17.05it/s]
Loading 0: 6%|▌ | 97/1739 [00:05<01:27, 18.69it/s]
Loading 0: 6%|▌ | 100/1739 [00:05<01:30, 18.04it/s]
Loading 0: 6%|▌ | 103/1739 [00:05<01:32, 17.76it/s]
Loading 0: 6%|▌ | 106/1739 [00:06<01:33, 17.51it/s]
Loading 0: 6%|▋ | 109/1739 [00:06<01:35, 17.11it/s]
Loading 0: 6%|▋ | 112/1739 [00:06<01:34, 17.16it/s]
Loading 0: 7%|▋ | 118/1739 [00:06<01:32, 17.58it/s]
Loading 0: 7%|▋ | 120/1739 [00:06<01:36, 16.85it/s]
Loading 0: 7%|▋ | 123/1739 [00:07<01:34, 17.04it/s]
Loading 0: 7%|▋ | 128/1739 [00:07<01:18, 20.42it/s]
Loading 0: 8%|▊ | 131/1739 [00:07<01:22, 19.59it/s]
Loading 0: 8%|▊ | 134/1739 [00:07<01:25, 18.71it/s]
Loading 0: 8%|▊ | 137/1739 [00:07<01:30, 17.73it/s]
Loading 0: 8%|▊ | 140/1739 [00:08<01:35, 16.74it/s]
Loading 0: 8%|▊ | 146/1739 [00:08<01:05, 24.48it/s]
Loading 0: 9%|▊ | 149/1739 [00:08<01:41, 15.60it/s]
Loading 0: 9%|▊ | 152/1739 [00:08<01:38, 16.19it/s]
Loading 0: 9%|▉ | 155/1739 [00:08<01:29, 17.77it/s]
Loading 0: 9%|▉ | 159/1739 [00:08<01:20, 19.55it/s]
Loading 0: 9%|▉ | 161/1739 [00:23<01:20, 19.55it/s]
Loading 0: 9%|▉ | 162/1739 [00:24<36:14, 1.38s/it]
Loading 0: 9%|▉ | 163/1739 [00:24<32:28, 1.24s/it]
Loading 0: 9%|▉ | 165/1739 [00:24<25:10, 1.04it/s]
Loading 0: 10%|▉ | 168/1739 [00:24<17:02, 1.54it/s]
Loading 0: 10%|▉ | 171/1739 [00:24<11:54, 2.19it/s]
Loading 0: 10%|█ | 177/1739 [00:24<06:23, 4.08it/s]
Loading 0: 10%|█ | 180/1739 [00:25<05:33, 4.68it/s]
Loading 0: 11%|█ | 183/1739 [00:25<04:26, 5.83it/s]
Loading 0: 11%|█ | 185/1739 [00:25<03:52, 6.67it/s]
Loading 0: 11%|█ | 190/1739 [00:25<02:36, 9.87it/s]
Loading 0: 11%|█ | 193/1739 [00:25<02:20, 11.02it/s]
Loading 0: 11%|█▏ | 196/1739 [00:26<02:05, 12.29it/s]
Loading 0: 11%|█▏ | 199/1739 [00:26<01:56, 13.27it/s]
Loading 0: 12%|█▏ | 202/1739 [00:26<01:47, 14.33it/s]
Loading 0: 12%|█▏ | 208/1739 [00:26<01:34, 16.24it/s]
Loading 0: 12%|█▏ | 210/1739 [00:26<01:36, 15.90it/s]
Loading 0: 12%|█▏ | 213/1739 [00:27<01:32, 16.56it/s]
Loading 0: 12%|█▏ | 216/1739 [00:27<01:30, 16.90it/s]
Loading 0: 13%|█▎ | 221/1739 [00:27<01:16, 19.80it/s]
Loading 0: 13%|█▎ | 224/1739 [00:27<01:19, 19.14it/s]
Loading 0: 13%|█▎ | 227/1739 [00:27<01:22, 18.38it/s]
Loading 0: 13%|█▎ | 230/1739 [00:27<01:24, 17.89it/s]
Loading 0: 14%|█▎ | 236/1739 [00:28<00:58, 25.88it/s]
Loading 0: 14%|█▍ | 240/1739 [00:28<01:28, 16.85it/s]
Loading 0: 14%|█▍ | 243/1739 [00:28<01:28, 16.98it/s]
Loading 0: 14%|█▍ | 246/1739 [00:28<01:27, 17.15it/s]
Loading 0: 14%|█▍ | 250/1739 [00:28<01:15, 19.78it/s]
Loading 0: 15%|█▍ | 253/1739 [00:29<01:17, 19.22it/s]
Loading 0: 15%|█▍ | 256/1739 [00:29<01:19, 18.73it/s]
Loading 0: 15%|█▍ | 259/1739 [00:29<01:20, 18.43it/s]
Loading 0: 15%|█▌ | 261/1739 [00:29<01:23, 17.81it/s]
Loading 0: 15%|█▌ | 267/1739 [00:29<00:59, 24.57it/s]
Loading 0: 16%|█▌ | 270/1739 [00:30<01:30, 16.26it/s]
Loading 0: 16%|█▌ | 273/1739 [00:30<01:29, 16.42it/s]
Loading 0: 16%|█▌ | 275/1739 [00:30<01:29, 16.31it/s]
Loading 0: 16%|█▌ | 278/1739 [00:30<01:27, 16.77it/s]
Loading 0: 16%|█▋ | 283/1739 [00:30<01:12, 20.16it/s]
Loading 0: 16%|█▋ | 286/1739 [00:30<01:17, 18.85it/s]
Loading 0: 17%|█▋ | 289/1739 [00:31<01:20, 18.12it/s]
Loading 0: 17%|█▋ | 292/1739 [00:31<01:24, 17.16it/s]
Loading 0: 17%|█▋ | 297/1739 [00:31<01:03, 22.62it/s]
Loading 0: 17%|█▋ | 300/1739 [00:31<01:34, 15.20it/s]
Loading 0: 17%|█▋ | 303/1739 [00:31<01:32, 15.51it/s]
Loading 0: 18%|█▊ | 306/1739 [00:32<01:30, 15.87it/s]
Loading 0: 18%|█▊ | 312/1739 [00:32<01:01, 23.11it/s]
Loading 0: 18%|█▊ | 315/1739 [00:32<01:03, 22.60it/s]
Loading 0: 18%|█▊ | 318/1739 [00:32<01:07, 21.09it/s]
Loading 0: 18%|█▊ | 321/1739 [00:32<01:13, 19.27it/s]
Loading 0: 19%|█▊ | 324/1739 [00:32<01:16, 18.58it/s]
Loading 0: 19%|█▉ | 327/1739 [00:43<01:16, 18.58it/s]
Loading 0: 19%|█▉ | 328/1739 [00:48<31:19, 1.33s/it]
Loading 0: 19%|█▉ | 329/1739 [00:48<28:09, 1.20s/it]
Loading 0: 19%|█▉ | 331/1739 [00:48<22:14, 1.06it/s]
Loading 0: 19%|█▉ | 334/1739 [00:48<15:09, 1.54it/s]
Loading 0: 19%|█▉ | 337/1739 [00:48<10:37, 2.20it/s]
Loading 0: 20%|█▉ | 340/1739 [00:48<07:39, 3.04it/s]
Loading 0: 20%|█▉ | 345/1739 [00:49<04:40, 4.98it/s]
Loading 0: 20%|██ | 348/1739 [00:49<03:44, 6.20it/s]
Loading 0: 20%|██ | 351/1739 [00:49<03:02, 7.60it/s]
Loading 0: 21%|██ | 357/1739 [00:49<01:54, 12.10it/s]
Loading 0: 21%|██ | 360/1739 [00:49<02:04, 11.06it/s]
Loading 0: 21%|██ | 363/1739 [00:50<01:51, 12.35it/s]
Loading 0: 21%|██ | 366/1739 [00:50<01:40, 13.65it/s]
Loading 0: 21%|██ | 369/1739 [00:50<01:32, 14.80it/s]
Loading 0: 21%|██▏ | 372/1739 [00:50<01:21, 16.87it/s]
Loading 0: 22%|██▏ | 376/1739 [00:50<01:12, 18.93it/s]
Loading 0: 22%|██▏ | 379/1739 [00:50<01:12, 18.64it/s]
Loading 0: 22%|██▏ | 382/1739 [00:51<01:17, 17.58it/s]
Loading 0: 22%|██▏ | 388/1739 [00:51<01:12, 18.61it/s]
Loading 0: 22%|██▏ | 390/1739 [00:51<01:15, 17.85it/s]
Loading 0: 23%|██▎ | 393/1739 [00:51<01:15, 17.93it/s]
Loading 0: 23%|██▎ | 396/1739 [00:51<01:13, 18.17it/s]
Loading 0: 23%|██▎ | 399/1739 [00:51<01:12, 18.48it/s]
Loading 0: 23%|██▎ | 402/1739 [00:52<01:12, 18.55it/s]
Loading 0: 23%|██▎ | 407/1739 [00:52<01:01, 21.51it/s]
Loading 0: 24%|██▎ | 410/1739 [00:52<01:05, 20.21it/s]
Loading 0: 24%|██▍ | 417/1739 [00:52<00:50, 26.27it/s]
Loading 0: 24%|██▍ | 420/1739 [00:52<01:14, 17.66it/s]
Loading 0: 24%|██▍ | 423/1739 [00:53<01:16, 17.23it/s]
Loading 0: 24%|██▍ | 425/1739 [00:53<01:20, 16.27it/s]
Loading 0: 25%|██▍ | 427/1739 [00:53<01:20, 16.22it/s]
Loading 0: 25%|██▍ | 430/1739 [00:53<01:18, 16.76it/s]
Loading 0: 25%|██▍ | 433/1739 [00:53<01:18, 16.68it/s]
Loading 0: 25%|██▌ | 438/1739 [00:53<01:03, 20.41it/s]
Loading 0: 25%|██▌ | 441/1739 [00:54<01:07, 19.10it/s]
Loading 0: 26%|██▌ | 447/1739 [00:54<00:51, 25.21it/s]
Loading 0: 26%|██▌ | 450/1739 [00:54<01:15, 17.05it/s]
Loading 0: 26%|██▌ | 453/1739 [00:54<01:15, 17.05it/s]
Loading 0: 26%|██▌ | 455/1739 [00:54<01:15, 16.93it/s]
Loading 0: 26%|██▋ | 458/1739 [00:55<01:13, 17.36it/s]
Loading 0: 27%|██▋ | 461/1739 [00:55<01:14, 17.21it/s]
Loading 0: 27%|██▋ | 464/1739 [00:55<01:14, 17.19it/s]
Loading 0: 27%|██▋ | 469/1739 [00:55<01:00, 20.90it/s]
Loading 0: 27%|██▋ | 472/1739 [00:55<01:04, 19.60it/s]
Loading 0: 27%|██▋ | 477/1739 [00:55<00:49, 25.49it/s]
Loading 0: 28%|██▊ | 480/1739 [00:56<01:16, 16.56it/s]
Loading 0: 28%|██▊ | 483/1739 [00:56<01:15, 16.55it/s]
Loading 0: 28%|██▊ | 486/1739 [00:56<01:14, 16.81it/s]
Loading 0: 28%|██▊ | 489/1739 [01:11<29:29, 1.42s/it]
Loading 0: 28%|██▊ | 490/1739 [01:11<26:16, 1.26s/it]
Loading 0: 28%|██▊ | 492/1739 [01:11<20:25, 1.02it/s]
Loading 0: 28%|██▊ | 495/1739 [01:11<13:38, 1.52it/s]
Loading 0: 29%|██▉ | 500/1739 [01:12<07:43, 2.67it/s]
Loading 0: 29%|██▉ | 506/1739 [01:12<04:28, 4.59it/s]
Loading 0: 29%|██▉ | 510/1739 [01:12<03:52, 5.29it/s]
Loading 0: 29%|██▉ | 513/1739 [01:12<03:13, 6.34it/s]
Loading 0: 30%|██▉ | 516/1739 [01:13<02:41, 7.56it/s]
Loading 0: 30%|██▉ | 519/1739 [01:13<02:17, 8.88it/s]
Loading 0: 30%|██▉ | 521/1739 [01:13<02:06, 9.61it/s]
Loading 0: 30%|███ | 523/1739 [01:13<01:57, 10.32it/s]
Loading 0: 30%|███ | 526/1739 [01:13<01:42, 11.89it/s]
Loading 0: 31%|███ | 531/1739 [01:13<01:15, 16.03it/s]
Loading 0: 31%|███ | 537/1739 [01:13<00:55, 21.67it/s]
Loading 0: 31%|███ | 540/1739 [01:14<01:17, 15.48it/s]
Loading 0: 31%|███ | 543/1739 [01:14<01:16, 15.69it/s]
Loading 0: 31%|███▏ | 545/1739 [01:14<01:17, 15.39it/s]
Loading 0: 32%|███▏ | 548/1739 [01:14<01:14, 15.97it/s]
Loading 0: 32%|███▏ | 551/1739 [01:15<01:12, 16.45it/s]
Loading 0: 32%|███▏ | 554/1739 [01:15<01:10, 16.75it/s]
Loading 0: 32%|███▏ | 557/1739 [01:15<01:08, 17.35it/s]
Loading 0: 32%|███▏ | 562/1739 [01:15<00:55, 21.20it/s]
Loading 0: 33%|███▎ | 567/1739 [01:15<00:43, 26.83it/s]
Loading 0: 33%|███▎ | 570/1739 [01:15<01:04, 17.99it/s]
Loading 0: 33%|███▎ | 573/1739 [01:16<01:04, 18.05it/s]
Loading 0: 33%|███▎ | 576/1739 [01:16<01:03, 18.33it/s]
Loading 0: 33%|███▎ | 579/1739 [01:16<01:02, 18.54it/s]
Loading 0: 33%|███▎ | 582/1739 [01:16<01:05, 17.77it/s]
Loading 0: 34%|███▎ | 585/1739 [01:16<01:03, 18.06it/s]
Loading 0: 34%|███▍ | 588/1739 [01:16<01:02, 18.34it/s]
Loading 0: 34%|███▍ | 597/1739 [01:17<00:38, 29.52it/s]
Loading 0: 35%|███▍ | 601/1739 [01:17<01:00, 18.66it/s]
Loading 0: 35%|███▍ | 604/1739 [01:17<01:00, 18.62it/s]
Loading 0: 35%|███▍ | 607/1739 [01:17<01:00, 18.64it/s]
Loading 0: 35%|███▌ | 610/1739 [01:18<01:00, 18.55it/s]
Loading 0: 35%|███▌ | 613/1739 [01:18<01:02, 17.92it/s]
Loading 0: 35%|███▌ | 616/1739 [01:18<01:03, 17.70it/s]
Loading 0: 36%|███▌ | 619/1739 [01:18<01:03, 17.51it/s]
Loading 0: 36%|███▌ | 627/1739 [01:18<00:40, 27.75it/s]
Loading 0: 36%|███▋ | 631/1739 [01:19<00:57, 19.28it/s]
Loading 0: 36%|███▋ | 634/1739 [01:19<00:57, 19.10it/s]
Loading 0: 37%|███▋ | 637/1739 [01:19<00:57, 19.17it/s]
Loading 0: 37%|███▋ | 640/1739 [01:19<00:56, 19.33it/s]
Loading 0: 37%|███▋ | 643/1739 [01:19<00:59, 18.49it/s]
Loading 0: 37%|███▋ | 645/1739 [01:19<01:01, 17.74it/s]
Loading 0: 37%|███▋ | 647/1739 [01:19<01:03, 17.28it/s]
Loading 0: 37%|███▋ | 649/1739 [01:34<01:03, 17.28it/s]
Loading 0: 37%|███▋ | 650/1739 [01:34<29:49, 1.64s/it]
Loading 0: 38%|███▊ | 657/1739 [01:34<14:25, 1.25it/s]
Loading 0: 38%|███▊ | 660/1739 [01:35<11:25, 1.57it/s]
Loading 0: 38%|███▊ | 663/1739 [01:35<08:44, 2.05it/s]
Loading 0: 38%|███▊ | 666/1739 [01:35<06:39, 2.69it/s]
Loading 0: 38%|███▊ | 669/1739 [01:35<05:04, 3.52it/s]
Loading 0: 39%|███▊ | 672/1739 [01:35<03:54, 4.54it/s]
Loading 0: 39%|███▉ | 675/1739 [01:36<03:06, 5.71it/s]
Loading 0: 39%|███▉ | 678/1739 [01:36<02:29, 7.10it/s]
Loading 0: 39%|███▉ | 681/1739 [01:36<02:02, 8.64it/s]
Loading 0: 40%|███▉ | 687/1739 [01:36<01:28, 11.86it/s]
Loading 0: 40%|███▉ | 689/1739 [01:36<01:25, 12.32it/s]
Loading 0: 40%|███▉ | 692/1739 [01:37<01:18, 13.42it/s]
Loading 0: 40%|███▉ | 695/1739 [01:37<01:12, 14.33it/s]
Loading 0: 40%|████ | 698/1739 [01:37<01:07, 15.39it/s]
Loading 0: 40%|████ | 701/1739 [01:37<01:04, 16.05it/s]
Loading 0: 40%|████ | 704/1739 [01:37<01:04, 16.09it/s]
Loading 0: 41%|████ | 707/1739 [01:37<01:04, 15.92it/s]
Loading 0: 41%|████ | 710/1739 [01:38<01:04, 15.86it/s]
Loading 0: 41%|████ | 717/1739 [01:38<00:48, 21.02it/s]
Loading 0: 41%|████▏ | 720/1739 [01:38<00:51, 19.92it/s]
Loading 0: 42%|████▏ | 722/1739 [01:38<00:53, 18.84it/s]
Loading 0: 42%|████▏ | 725/1739 [01:38<00:54, 18.59it/s]
Loading 0: 42%|████▏ | 728/1739 [01:38<00:54, 18.47it/s]
Loading 0: 42%|████▏ | 731/1739 [01:39<00:55, 18.17it/s]
Loading 0: 42%|████▏ | 734/1739 [01:39<00:56, 17.75it/s]
Loading 0: 42%|████▏ | 737/1739 [01:39<00:56, 17.86it/s]
Loading 0: 43%|████▎ | 740/1739 [01:39<00:56, 17.65it/s]
Loading 0: 43%|████▎ | 747/1739 [01:39<00:49, 20.10it/s]
Loading 0: 43%|████▎ | 749/1739 [01:40<00:52, 18.99it/s]
Loading 0: 43%|████▎ | 752/1739 [01:40<00:52, 18.75it/s]
Loading 0: 43%|████▎ | 755/1739 [01:40<00:51, 18.94it/s]
Loading 0: 44%|████▎ | 758/1739 [01:40<00:52, 18.54it/s]
Loading 0: 44%|████▍ | 761/1739 [01:40<00:53, 18.21it/s]
Loading 0: 44%|████▍ | 764/1739 [01:40<00:53, 18.27it/s]
Loading 0: 44%|████▍ | 767/1739 [01:41<00:54, 17.97it/s]
Loading 0: 44%|████▍ | 770/1739 [01:41<00:55, 17.31it/s]
Loading 0: 45%|████▍ | 776/1739 [01:41<00:37, 25.63it/s]
Loading 0: 45%|████▍ | 779/1739 [01:41<00:56, 17.01it/s]
Loading 0: 45%|████▍ | 782/1739 [01:41<00:56, 16.86it/s]
Loading 0: 45%|████▌ | 785/1739 [01:42<00:55, 17.08it/s]
Loading 0: 45%|████▌ | 788/1739 [01:42<00:55, 17.03it/s]
Loading 0: 45%|████▌ | 791/1739 [01:42<00:55, 17.06it/s]
Loading 0: 46%|████▌ | 794/1739 [01:42<00:56, 16.84it/s]
Loading 0: 46%|████▌ | 797/1739 [01:42<00:56, 16.62it/s]
Loading 0: 46%|████▌ | 804/1739 [01:43<00:40, 23.35it/s]
Loading 0: 46%|████▋ | 807/1739 [01:43<00:50, 18.34it/s]
Loading 0: 47%|████▋ | 810/1739 [01:43<00:51, 18.07it/s]
Loading 0: 47%|████▋ | 812/1739 [01:58<00:51, 18.07it/s]
Loading 0: 47%|████▋ |
Connection pool is full, discarding connection: %s. Connection pool size: %s
mistralai-mixtral-8x22b-4383-v2-mkmlizer: 813/1739 [01:58<20:33, 1.33s/it]
Loading 0: 47%|████▋ | 814/1739 [01:58<18:26, 1.20s/it]
Loading 0: 47%|████▋ | 816/1739 [01:58<14:31, 1.06it/s]
Loading 0: 47%|████▋ | 819/1739 [01:58<09:51, 1.55it/s]
Loading 0: 47%|████▋ | 822/1739 [01:58<06:54, 2.21it/s]
Loading 0: 47%|████▋ | 825/1739 [01:59<04:57, 3.08it/s]
Loading 0: 48%|████▊ | 828/1739 [01:59<03:38, 4.16it/s]
Loading 0: 48%|████▊ | 834/1739 [01:59<02:03, 7.30it/s]
Loading 0: 48%|████▊ | 837/1739 [01:59<01:55, 7.81it/s]
Loading 0: 48%|████▊ | 841/1739 [01:59<01:29, 10.03it/s]
Loading 0: 49%|████▊ | 844/1739 [02:00<01:19, 11.31it/s]
Loading 0: 49%|████▊ | 847/1739 [02:00<01:09, 12.80it/s]
Loading 0: 49%|████▉ | 850/1739 [02:00<01:03, 14.05it/s]
Loading 0: 49%|████▉ | 853/1739 [02:00<01:00, 14.62it/s]
Loading 0: 49%|████▉ | 856/1739 [02:00<00:58, 15.20it/s]
Loading 0: 49%|████▉ | 859/1739 [02:00<00:56, 15.70it/s]
Loading 0: 50%|████▉ | 864/1739 [02:01<00:40, 21.50it/s]
Loading 0: 50%|████▉ | 867/1739 [02:01<00:57, 15.29it/s]
Loading 0: 50%|█████ | 872/1739 [02:01<00:47, 18.14it/s]
Loading 0: 50%|█████ | 875/1739 [02:01<00:48, 17.68it/s]
Loading 0: 50%|█████ | 878/1739 [02:01<00:48, 17.62it/s]
Loading 0: 51%|█████ | 881/1739 [02:02<00:49, 17.38it/s]
Loading 0: 51%|█████ | 884/1739 [02:02<00:48, 17.47it/s]
Loading 0: 51%|█████ | 887/1739 [02:02<00:48, 17.51it/s]
Loading 0: 51%|█████▏ | 894/1739 [02:02<00:34, 24.22it/s]
Loading 0: 52%|█████▏ | 897/1739 [02:02<00:49, 17.12it/s]
Loading 0: 52%|█████▏ | 900/1739 [02:03<00:44, 19.01it/s]
Loading 0: 52%|█████▏ | 903/1739 [02:03<00:46, 18.04it/s]
Loading 0: 52%|█████▏ | 906/1739 [02:03<00:47, 17.55it/s]
Loading 0: 52%|█████▏ | 909/1739 [02:03<00:48, 17.24it/s]
Loading 0: 52%|█████▏ | 912/1739 [02:03<00:48, 16.98it/s]
Loading 0: 53%|█████▎ | 915/1739 [02:03<00:48, 17.11it/s]
Loading 0: 53%|█████▎ | 918/1739 [02:04<00:47, 17.13it/s]
Loading 0: 53%|█████▎ | 924/1739 [02:04<00:33, 24.26it/s]
Loading 0: 53%|█████▎ | 927/1739 [02:04<00:48, 16.71it/s]
Loading 0: 53%|█████▎ | 930/1739 [02:04<00:45, 17.71it/s]
Loading 0: 54%|█████▎ | 934/1739 [02:04<00:42, 19.00it/s]
Loading 0: 54%|█████▍ | 937/1739 [02:05<00:42, 18.68it/s]
Loading 0: 54%|█████▍ | 940/1739 [02:05<00:42, 18.66it/s]
Loading 0: 54%|█████▍ | 943/1739 [02:05<00:42, 18.93it/s]
Loading 0: 54%|█████▍ | 946/1739 [02:05<00:41, 19.21it/s]
Loading 0: 55%|█████▍ | 949/1739 [02:05<00:42, 18.80it/s]
Loading 0: 55%|█████▍ | 955/1739 [02:06<00:39, 19.80it/s]
Loading 0: 55%|█████▌ | 957/1739 [02:06<00:41, 18.68it/s]
Loading 0: 55%|█████▌ | 960/1739 [02:06<00:42, 18.26it/s]
Loading 0: 55%|█████▌ | 965/1739 [02:06<00:35, 21.59it/s]
Loading 0: 56%|█████▌ | 968/1739 [02:06<00:36, 21.01it/s]
Loading 0: 56%|█████▌ | 971/1739 [02:06<00:37, 20.46it/s]
Loading 0: 56%|█████▌ | 973/1739 [02:21<00:37, 20.46it/s]
Loading 0: 56%|█████▌ | 974/1739 [02:21<17:45, 1.39s/it]
Loading 0: 56%|█████▌ | 975/1739 [02:21<15:51, 1.24s/it]
Loading 0: 56%|█████▌ | 977/1739 [02:22<12:21, 1.03it/s]
Loading 0: 57%|█████▋ | 983/1739 [02:22<06:02, 2.08it/s]
Loading 0: 57%|█████▋ | 986/1739 [02:22<04:49, 2.60it/s]
Loading 0: 57%|█████▋ | 989/1739 [02:22<03:39, 3.41it/s]
Loading 0: 57%|█████▋ | 992/1739 [02:22<02:45, 4.52it/s]
Loading 0: 57%|█████▋ | 996/1739 [02:22<01:58, 6.29it/s]
Loading 0: 57%|█████▋ | 999/1739 [02:23<01:36, 7.66it/s]
Loading 0: 58%|█████▊ | 1002/1739 [02:23<01:20, 9.10it/s]
Loading 0: 58%|█████▊ | 1005/1739 [02:23<01:09, 10.50it/s]
Loading 0: 58%|█████▊ | 1008/1739 [02:23<01:00, 12.04it/s]
Loading 0: 58%|█████▊ | 1014/1739 [02:23<00:39, 18.15it/s]
Loading 0: 58%|█████▊ | 1017/1739 [02:24<00:50, 14.23it/s]
Loading 0: 59%|█████▊ | 1020/1739 [02:24<00:47, 15.08it/s]
Loading 0: 59%|█████▉ | 1023/1739 [02:24<00:42, 16.81it/s]
Loading 0: 59%|█████▉ | 1027/1739 [02:24<00:38, 18.29it/s]
Loading 0: 59%|█████▉ | 1030/1739 [02:24<00:38, 18.51it/s]
Loading 0: 59%|█████▉ | 1033/1739 [02:24<00:39, 18.09it/s]
Loading 0: 60%|█████▉ | 1036/1739 [02:25<00:39, 17.60it/s]
Loading 0: 60%|█████▉ | 1039/1739 [02:25<00:39, 17.77it/s]
Loading 0: 60%|██████ | 1044/1739 [02:25<00:28, 24.00it/s]
Loading 0: 60%|██████ | 1047/1739 [02:25<00:41, 16.73it/s]
Loading 0: 60%|██████ | 1050/1739 [02:25<00:40, 17.14it/s]
Loading 0: 61%|██████ | 1053/1739 [02:26<00:40, 16.88it/s]
Loading 0: 61%|██████ | 1058/1739 [02:26<00:32, 20.79it/s]
Loading 0: 61%|██████ | 1061/1739 [02:26<00:33, 20.14it/s]
Loading 0: 61%|██████ | 1064/1739 [02:26<00:34, 19.43it/s]
Loading 0: 61%|██████▏ | 1067/1739 [02:26<00:34, 19.28it/s]
Loading 0: 62%|██████▏ | 1074/1739 [02:26<00:25, 25.94it/s]
Loading 0: 62%|██████▏ | 1077/1739 [02:27<00:36, 18.04it/s]
Loading 0: 62%|██████▏ | 1080/1739 [02:27<00:37, 17.79it/s]
Loading 0: 62%|██████▏ | 1082/1739 [02:27<00:39, 16.52it/s]
Loading 0: 62%|██████▏ | 1084/1739 [02:27<00:40, 16.23it/s]
Loading 0: 63%|██████▎ | 1089/1739 [02:27<00:32, 20.22it/s]
Loading 0: 63%|██████▎ | 1092/1739 [02:28<00:33, 19.37it/s]
Loading 0: 63%|██████▎ | 1095/1739 [02:28<00:33, 19.03it/s]
Loading 0: 63%|██████▎ | 1098/1739 [02:28<00:34, 18.75it/s]
Loading 0: 63%|██████▎ | 1104/1739 [02:28<00:24, 25.46it/s]
Loading 0: 64%|██████▎ | 1107/1739 [02:28<00:36, 17.39it/s]
Loading 0: 64%|██████▍ | 1110/1739 [02:29<00:36, 17.41it/s]
Loading 0: 64%|██████▍ | 1113/1739 [02:29<00:35, 17.69it/s]
Loading 0: 64%|██████▍ | 1115/1739 [02:29<00:35, 17.71it/s]
Loading 0: 64%|██████▍ | 1118/1739 [02:29<00:31, 19.82it/s]
Loading 0: 64%|██████▍ | 1121/1739 [02:29<00:32, 19.27it/s]
Loading 0: 65%|██████▍ | 1124/1739 [02:29<00:33, 18.57it/s]
Loading 0: 65%|██████▍ | 1126/1739 [02:29<00:34, 17.92it/s]
Loading 0: 65%|██████▍ | 1129/1739 [02:30<00:34, 17.72it/s]
Loading 0: 65%|██████▌ | 1135/1739 [02:30<00:31, 19.43it/s]
Loading 0: 65%|██████▌ | 1137/1739 [02:30<00:32, 18.29it/s]
Loading 0: 65%|██████▌ | 1139/1739 [02:45<00:32, 18.29it/s]
Loading 0: 66%|██████▌ | 1140/1739 [02:45<14:54, 1.49s/it]
Loading 0: 66%|██████▌ | 1141/1739 [02:45<13:14, 1.33s/it]
Loading 0: 66%|██████▌ | 1143/1739 [02:46<10:11, 1.03s/it]
Loading 0: 66%|██████▌ | 1146/1739 [02:46<06:44, 1.46it/s]
Loading 0: 66%|██████▌ | 1151/1739 [02:46<03:46, 2.59it/s]
Loading 0: 66%|██████▋ | 1154/1739 [02:46<02:52, 3.39it/s]
Loading 0: 67%|██████▋ | 1157/1739 [02:46<02:11, 4.43it/s]
Loading 0: 67%|██████▋ | 1164/1739 [02:46<01:13, 7.86it/s]
Loading 0: 67%|██████▋ | 1167/1739 [02:47<01:10, 8.14it/s]
Loading 0: 67%|██████▋ | 1170/1739 [02:47<01:01, 9.30it/s]
Loading 0: 67%|██████▋ | 1172/1739 [02:47<00:57, 9.93it/s]
Loading 0: 68%|██████▊ | 1174/1739 [02:47<00:53, 10.65it/s]
Loading 0: 68%|██████▊ | 1177/1739 [02:47<00:46, 12.09it/s]
Loading 0: 68%|██████▊ | 1182/1739 [02:48<00:35, 15.87it/s]
Loading 0: 68%|██████▊ | 1185/1739 [02:48<00:34, 16.18it/s]
Loading 0: 68%|██████▊ | 1188/1739 [02:48<00:33, 16.26it/s]
Loading 0: 69%|██████▊ | 1194/1739 [02:48<00:24, 22.15it/s]
Loading 0: 69%|██████▉ | 1197/1739 [02:48<00:33, 16.14it/s]
Loading 0: 69%|██████▉ | 1200/1739 [02:49<00:33, 16.16it/s]
Loading 0: 69%|██████▉ | 1202/1739 [02:49<00:33, 15.88it/s]
Loading 0: 69%|██████▉ | 1205/1739 [02:49<00:32, 16.42it/s]
Loading 0: 69%|██████▉ | 1208/1739 [02:49<00:31, 16.78it/s]
Loading 0: 70%|██████▉ | 1213/1739 [02:49<00:26, 20.15it/s]
Loading 0: 70%|██████▉ | 1216/1739 [02:49<00:26, 19.77it/s]
Loading 0: 70%|███████ | 1219/1739 [02:50<00:27, 19.24it/s]
Loading 0: 70%|███████ | 1225/1739 [02:50<00:26, 19.55it/s]
Loading 0: 71%|███████ | 1227/1739 [02:50<00:27, 18.33it/s]
Loading 0: 71%|███████ | 1230/1739 [02:50<00:28, 17.63it/s]
Loading 0: 71%|███████ | 1233/1739 [02:50<00:29, 17.41it/s]
Loading 0: 71%|███████ | 1236/1739 [02:51<00:29, 17.08it/s]
Loading 0: 71%|███████ | 1239/1739 [02:51<00:29, 16.91it/s]
Loading 0: 72%|███████▏ | 1244/1739 [02:51<00:24, 20.17it/s]
Loading 0: 72%|███████▏ | 1247/1739 [02:51<00:26, 18.72it/s]
Loading 0: 72%|███████▏ | 1253/1739 [02:51<00:18, 26.28it/s]
Loading 0: 72%|███████▏ | 1257/1739 [02:52<00:28, 17.12it/s]
Loading 0: 72%|███████▏ | 1260/1739 [02:52<00:27, 17.26it/s]
Loading 0: 73%|███████▎ | 1263/1739 [02:52<00:27, 17.27it/s]
Loading 0: 73%|███████▎ | 1266/1739 [02:52<00:27, 17.21it/s]
Loading 0: 73%|███████▎ | 1268/1739 [02:52<00:28, 16.65it/s]
Loading 0: 73%|███████▎ | 1270/1739 [02:53<00:28, 16.47it/s]
Loading 0: 73%|███████▎ | 1275/1739 [02:53<00:22, 20.33it/s]
Loading 0: 73%|███████▎ | 1278/1739 [02:53<00:23, 19.45it/s]
Loading 0: 74%|███████▍ | 1284/1739 [02:53<00:17, 25.78it/s]
Loading 0: 74%|███████▍ | 1287/1739 [02:53<00:26, 16.85it/s]
Loading 0: 74%|███████▍ | 1290/1739 [02:54<00:26, 16.90it/s]
Loading 0: 74%|███████▍ | 1293/1739 [02:54<00:26, 16.94it/s]
Loading 0: 74%|███████▍ | 1295/1739 [02:54<00:26, 16.76it/s]
Loading 0: 75%|███████▍ | 1298/1739 [02:54<00:25, 17.13it/s]
Loading 0: 75%|███████▍ | 1300/1739 [03:09<00:25, 17.13it/s]
Loading 0: 75%|███████▍ | 1301/1739 [03:09<11:26, 1.57s/it]
Loading 0: 75%|███████▍ | 1304/1739 [03:09<08:00, 1.11s/it]
Loading 0: 75%|███████▌ | 1307/1739 [03:10<05:47, 1.24it/s]
Loading 0: 75%|███████▌ | 1309/1739 [03:10<04:34, 1.57it/s]
Loading 0: 76%|███████▌ | 1314/1739 [03:10<02:35, 2.74it/s]
Loading 0: 76%|███████▌ | 1317/1739 [03:10<02:05, 3.37it/s]
Loading 0: 76%|███████▌ | 1320/1739 [03:10<01:36, 4.36it/s]
Loading 0: 76%|███████▌ | 1323/1739 [03:11<01:14, 5.55it/s]
Loading 0: 76%|███████▋ | 1326/1739 [03:11<00:59, 6.92it/s]
Loading 0: 76%|███████▋ | 1329/1739 [03:11<00:48, 8.44it/s]
Loading 0: 77%|███████▋ | 1332/1739 [03:11<00:40, 9.97it/s]
Loading 0: 77%|███████▋ | 1337/1739 [03:11<00:29, 13.59it/s]
Loading 0: 77%|███████▋ | 1344/1739 [03:11<00:20, 19.55it/s]
Loading 0: 77%|███████▋ | 1347/1739 [03:12<00:25, 15.30it/s]
Loading 0: 78%|███████▊ | 1350/1739 [03:12<00:24, 15.74it/s]
Loading 0: 78%|███████▊ | 1353/1739 [03:12<00:23, 16.34it/s]
Loading 0: 78%|███████▊ | 1355/1739 [03:12<00:24, 15.93it/s]
Loading 0: 78%|███████▊ | 1357/1739 [03:12<00:24, 15.81it/s]
Loading 0: 78%|███████▊ | 1360/1739 [03:12<00:22, 16.52it/s]
Loading 0: 78%|███████▊ | 1363/1739 [03:13<00:22, 16.75it/s]
Loading 0: 79%|███████▊ | 1368/1739 [03:13<00:18, 20.36it/s]
Loading 0: 79%|███████▉ | 1374/1739 [03:13<00:13, 26.51it/s]
Loading 0: 79%|███████▉ | 1377/1739 [03:13<00:20, 17.85it/s]
Loading 0: 79%|███████▉ | 1380/1739 [03:14<00:20, 17.73it/s]
Loading 0: 80%|███████▉ | 1383/1739 [03:14<00:20, 17.06it/s]
Loading 0: 80%|███████▉ | 1385/1739 [03:14<00:21, 16.27it/s]
Loading 0: 80%|███████▉ | 1388/1739 [03:14<00:20, 16.72it/s]
Loading 0: 80%|███████▉ | 1391/1739 [03:14<00:20, 16.94it/s]
Loading 0: 80%|████████ | 1394/1739 [03:14<00:19, 17.40it/s]
Loading 0: 80%|████████ | 1399/1739 [03:15<00:15, 21.41it/s]
Loading 0: 81%|████████ | 1405/1739 [03:15<00:15, 21.11it/s]
Loading 0: 81%|████████ | 1408/1739 [03:15<00:16, 20.07it/s]
Loading 0: 81%|████████ | 1411/1739 [03:15<00:17, 19.01it/s]
Loading 0: 81%|████████▏ | 1413/1739 [03:15<00:17, 18.13it/s]
Loading 0: 81%|████████▏ | 1416/1739 [03:15<00:18, 17.44it/s]
Loading 0: 82%|████████▏ | 1419/1739 [03:16<00:18, 17.20it/s]
Loading 0: 82%|████████▏ | 1422/1739 [03:16<00:18, 17.29it/s]
Loading 0: 82%|████████▏ | 1425/1739 [03:16<00:17, 17.71it/s]
Loading 0: 82%|████████▏ | 1434/1739 [03:16<00:10, 28.79it/s]
Loading 0: 83%|████████▎ | 1437/1739 [03:16<00:15, 20.12it/s]
Loading 0: 83%|████████▎ | 1440/1739 [03:17<00:15, 19.10it/s]
Loading 0: 83%|████████▎ | 1443/1739 [03:17<00:15, 18.82it/s]
Loading 0: 83%|████████▎ | 1446/1739 [03:17<00:16, 18.31it/s]
Loading 0: 83%|████████▎ | 1448/1739 [03:17<00:16, 17.70it/s]
Loading 0: 83%|████████▎ | 1450/1739 [03:17<00:16, 17.04it/s]
Loading 0: 84%|████████▎ | 1453/1739 [03:17<00:16, 17.05it/s]
Loading 0: 84%|████████▎ | 1456/1739 [03:18<00:16, 16.82it/s]
Loading 0: 84%|████████▍ | 1464/1739 [03:18<00:09, 27.60it/s]
Loading 0: 84%|████████▍ | 1467/1739 [03:18<00:14, 18.96it/s]
Loading 0: 84%|████████▍ | 1468/1739 [03:29<00:14, 18.96it/s]
Loading 0: 84%|████████▍ | 1469/1739 [03:34<06:24, 1.43s/it]
Loading 0: 85%|████████▍ | 1470/1739 [03:34<05:43, 1.28s/it]
Loading 0: 85%|████████▍ | 1472/1739 [03:34<04:27, 1.00s/it]
Loading 0: 85%|████████▍ | 1475/1739 [03:34<02:59, 1.47it/s]
Loading 0: 85%|████████▍ | 1478/1739 [03:34<02:04, 2.10it/s]
Loading 0: 85%|████████▌ | 1481/1739 [03:35<01:29, 2.89it/s]
Loading 0: 85%|████████▌ | 1484/1739 [03:35<01:05, 3.88it/s]
Loading 0: 86%|████████▌ | 1487/1739 [03:35<00:49, 5.06it/s]
Loading 0: 86%|████████▌ | 1494/1739 [03:35<00:26, 9.41it/s]
Loading 0: 86%|████████▌ | 1497/1739 [03:36<00:26, 9.11it/s]
Loading 0: 86%|████████▋ | 1500/1739 [03:36<00:24, 9.94it/s]
Loading 0: 86%|████████▋ | 1503/1739 [03:36<00:21, 11.03it/s]
Loading 0: 87%|████████▋ | 1506/1739 [03:36<00:19, 11.95it/s]
Loading 0: 87%|████████▋ | 1509/1739 [03:36<00:17, 12.83it/s]
Loading 0: 87%|████████▋ | 1512/173
Failed to get response for submission jic062-dpo-v3-0-nemo_v1: ('http://jic062-dpo-v3-0-nemo-v1-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', '')
mistralai-mixtral-8x22b-4383-v2-mkmlizer: quantized model in 260.437s
mistralai-mixtral-8x22b-4383-v2-mkmlizer: Processed model mistralai/Mixtral-8x22B-Instruct-v0.1 in 751.099s
mistralai-mixtral-8x22b-4383-v2-mkmlizer: creating bucket guanaco-mkml-models
mistralai-mixtral-8x22b-4383-v2-mkmlizer: Bucket 's3://guanaco-mkml-models/' created
mistralai-mixtral-8x22b-4383-v2-mkmlizer: uploading /dev/shm/model_cache to s3://guanaco-mkml-models/mistralai-mixtral-8x22b-4383-v2
mistralai-mixtral-8x22b-4383-v2-mkmlizer: cp /dev/shm/model_cache/config.json s3://guanaco-mkml-models/mistralai-mixtral-8x22b-4383-v2/config.json
mistralai-mixtral-8x22b-4383-v2-mkmlizer: cp /dev/shm/model_cache/special_tokens_map.json s3://guanaco-mkml-models/mistralai-mixtral-8x22b-4383-v2/special_tokens_map.json
mistralai-mixtral-8x22b-4383-v2-mkmlizer: cp /dev/shm/model_cache/tokenizer_config.json s3://guanaco-mkml-models/mistralai-mixtral-8x22b-4383-v2/tokenizer_config.json
mistralai-mixtral-8x22b-4383-v2-mkmlizer: cp /dev/shm/model_cache/tokenizer.model s3://guanaco-mkml-models/mistralai-mixtral-8x22b-4383-v2/tokenizer.model
mistralai-mixtral-8x22b-4383-v2-mkmlizer: cp /dev/shm/model_cache/flywheel_model.10.safetensors s3://guanaco-mkml-models/mistralai-mixtral-8x22b-4383-v2/flywheel_model.10.safetensors
mistralai-mixtral-8x22b-4383-v2-mkmlizer: cp /dev/shm/model_cache/flywheel_model.0.safetensors s3://guanaco-mkml-models/mistralai-mixtral-8x22b-4383-v2/flywheel_model.0.safetensors
mistralai-mixtral-8x22b-4383-v2-mkmlizer: cp /dev/shm/model_cache/flywheel_model.1.safetensors s3://guanaco-mkml-models/mistralai-mixtral-8x22b-4383-v2/flywheel_model.1.safetensors
mistralai-mixtral-8x22b-4383-v2-mkmlizer: cp /dev/shm/model_cache/flywheel_model.5.safetensors s3://guanaco-mkml-models/mistralai-mixtral-8x22b-4383-v2/flywheel_model.5.safetensors
mistralai-mixtral-8x22b-4383-v2-mkmlizer: cp /dev/shm/model_cache/flywheel_model.9.safetensors s3://guanaco-mkml-models/mistralai-mixtral-8x22b-4383-v2/flywheel_model.9.safetensors
mistralai-mixtral-8x22b-4383-v2-mkmlizer: cp /dev/shm/model_cache/flywheel_model.8.safetensors s3://guanaco-mkml-models/mistralai-mixtral-8x22b-4383-v2/flywheel_model.8.safetensors
mistralai-mixtral-8x22b-4383-v2-mkmlizer: cp /dev/shm/model_cache/flywheel_model.6.safetensors s3://guanaco-mkml-models/mistralai-mixtral-8x22b-4383-v2/flywheel_model.6.safetensors
mistralai-mixtral-8x22b-4383-v2-mkmlizer: cp /dev/shm/model_cache/flywheel_model.3.safetensors s3://guanaco-mkml-models/mistralai-mixtral-8x22b-4383-v2/flywheel_model.3.safetensors
mistralai-mixtral-8x22b-4383-v2-mkmlizer: cp /dev/shm/model_cache/flywheel_model.2.safetensors s3://guanaco-mkml-models/mistralai-mixtral-8x22b-4383-v2/flywheel_model.2.safetensors
mistralai-mixtral-8x22b-4383-v2-mkmlizer: cp /dev/shm/model_cache/flywheel_model.7.safetensors s3://guanaco-mkml-models/mistralai-mixtral-8x22b-4383-v2/flywheel_model.7.safetensors
mistralai-mixtral-8x22b-4383-v2-mkmlizer: cp /dev/shm/model_cache/flywheel_model.4.safetensors s3://guanaco-mkml-models/mistralai-mixtral-8x22b-4383-v2/flywheel_model.4.safetensors
Job mistralai-mixtral-8x22b-4383-v2-mkmlizer completed after 842.52s with status: succeeded
Stopping job with name mistralai-mixtral-8x22b-4383-v2-mkmlizer
Pipeline stage MKMLizer completed in 843.31s
run pipeline stage %s
Running pipeline stage MKMLTemplater
Pipeline stage MKMLTemplater completed in 0.22s
run pipeline stage %s
Running pipeline stage MKMLDeployer
Creating inference service mistralai-mixtral-8x22b-4383-v2
Waiting for inference service mistralai-mixtral-8x22b-4383-v2 to be ready
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Inference service mistralai-mixtral-8x22b-4383-v2 ready after 191.1494038105011s
Pipeline stage MKMLDeployer completed in 191.80s
run pipeline stage %s
Running pipeline stage StressChecker
Received healthy response to inference request in 4.165751695632935s
Received healthy response to inference request in 4.07743501663208s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Received healthy response to inference request in 3.907766819000244s
Received healthy response to inference request in 3.667011022567749s
Received healthy response to inference request in 2.832801580429077s
5 requests
0 failed requests
5th percentile: 2.9996434688568114
10th percentile: 3.1664853572845457
20th percentile: 3.5001691341400147
30th percentile: 3.715162181854248
40th percentile: 3.8114645004272463
50th percentile: 3.907766819000244
60th percentile: 3.9756340980529785
70th percentile: 4.043501377105713
80th percentile: 4.095098352432251
90th percentile: 4.130425024032593
95th percentile: 4.148088359832764
99th percentile: 4.162219028472901
mean time: 3.730153226852417
%s, retrying in %s seconds...
Received healthy response to inference request in 4.188339710235596s
Received healthy response to inference request in 3.357820510864258s
Received healthy response to inference request in 4.254093885421753s
Received healthy response to inference request in 2.617422342300415s
Received healthy response to inference request in 4.048181533813477s
5 requests
0 failed requests
5th percentile: 2.7655019760131836
10th percentile: 2.913581609725952
20th percentile: 3.2097408771514893
30th percentile: 3.4958927154541017
40th percentile: 3.772037124633789
50th percentile: 4.048181533813477
60th percentile: 4.104244804382324
70th percentile: 4.160308074951172
80th percentile: 4.2014905452728275
90th percentile: 4.22779221534729
95th percentile: 4.240943050384521
99th percentile: 4.251463718414307
mean time: 3.6931715965270997
%s, retrying in %s seconds...
Received healthy response to inference request in 3.2277674674987793s
Received healthy response to inference request in 4.028099298477173s
Received healthy response to inference request in 3.5421078205108643s
Received healthy response to inference request in 2.457923412322998s
Received healthy response to inference request in 4.361262321472168s
5 requests
0 failed requests
5th percentile: 2.6118922233581543
10th percentile: 2.7658610343933105
20th percentile: 3.073798656463623
30th percentile: 3.2906355381011965
40th percentile: 3.4163716793060304
50th percentile: 3.5421078205108643
60th percentile: 3.736504411697388
70th percentile: 3.930901002883911
80th percentile: 4.0947319030761715
90th percentile: 4.22799711227417
95th percentile: 4.294629716873169
99th percentile: 4.347935800552368
mean time: 3.5234320640563963
clean up pipeline due to error=DeploymentChecksError('Unacceptable 70th percentile latency 3.930901002883911s')
Shutdown handler de-registered
mistralai-mixtral-8x22b-_4383_v2 status is now failed due to DeploymentManager action
mistralai-mixtral-8x22b-_4383_v2 status is now torndown due to DeploymentManager action