Shutdown handler not registered because Python interpreter is not running in the main thread
Starting job with name chaiml-98p-2ff-chaiml-m-39704-v1-mkmlizer
run pipeline %s
run pipeline stage %s
Running pipeline stage MKMLizer
Waiting for job on chaiml-98p-2ff-chaiml-m-39704-v1-mkmlizer to finish
Starting job with name chaiml-mistral-24b-2048-83002-v4-mkmlizer
Waiting for job on chaiml-mistral-24b-2048-83002-v4-mkmlizer to finish
chaiml-98p-2ff-chaiml-m-67425-v1-mkmlizer: bash: cannot set terminal process group (-1): Inappropriate ioctl for device
chaiml-98p-2ff-chaiml-m-67425-v1-mkmlizer: bash: no job control in this shell
chaiml-98p-2ff-chaiml-m-67425-v1-mkmlizer: /root/miniconda3/envs/nvidia/lib/python3.11/site-packages/mk1/__init__.py:1: UserWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html. The pkg_resources package is slated for removal as early as 2025-11-30. Refrain from using this package or pin to Setuptools<81.
chaiml-98p-2ff-chaiml-m-67425-v1-mkmlizer: __import__('pkg_resources').declare_namespace(__name__)
chaiml-98p-2ff-chaiml-m-67425-v1-mkmlizer: ╔═════════════════════════════════════════════════════════════════════╗
chaiml-98p-2ff-chaiml-m-67425-v1-mkmlizer: ║ ║
chaiml-98p-2ff-chaiml-m-67425-v1-mkmlizer: ║ ██████ ██████ █████ ████ ████ ║
chaiml-98p-2ff-chaiml-m-67425-v1-mkmlizer: ║ ░░██████ ██████ ░░███ ███░ ░░███ ║
chaiml-98p-2ff-chaiml-m-67425-v1-mkmlizer: ║ ░███░█████░███ ░███ ███ ░███ ║
chaiml-98p-2ff-chaiml-m-67425-v1-mkmlizer: ║ ░███░░███ ░███ ░███████ ░███ ║
chaiml-98p-2ff-chaiml-m-67425-v1-mkmlizer: ║ ░███ ░░░ ░███ ░███░░███ ░███ ║
chaiml-98p-2ff-chaiml-m-67425-v1-mkmlizer: ║ ░███ ░███ ░███ ░░███ ░███ ║
chaiml-98p-2ff-chaiml-m-67425-v1-mkmlizer: ║ █████ █████ █████ ░░████ █████ ║
chaiml-98p-2ff-chaiml-m-67425-v1-mkmlizer: ║ ░░░░░ ░░░░░ ░░░░░ ░░░░ ░░░░░ ║
chaiml-98p-2ff-chaiml-m-67425-v1-mkmlizer: ║ ║
chaiml-98p-2ff-chaiml-m-67425-v1-mkmlizer: ║ Version: 0.30.6+torch280 ║
chaiml-98p-2ff-chaiml-m-67425-v1-mkmlizer: ║ Features: FLYWHEEL, CUDA ║
chaiml-98p-2ff-chaiml-m-67425-v1-mkmlizer: ║ Copyright 2023-2025 MK ONE TECHNOLOGIES Inc. ║
chaiml-98p-2ff-chaiml-m-67425-v1-mkmlizer: ║ https://mk1.ai ║
chaiml-98p-2ff-chaiml-m-67425-v1-mkmlizer: ║ ║
chaiml-98p-2ff-chaiml-m-67425-v1-mkmlizer: ║ The license key for the current software has been verified as ║
chaiml-98p-2ff-chaiml-m-67425-v1-mkmlizer: ║ belonging to: ║
chaiml-98p-2ff-chaiml-m-67425-v1-mkmlizer: ║ ║
chaiml-98p-2ff-chaiml-m-67425-v1-mkmlizer: ║ Chai Research Corp. ║
chaiml-98p-2ff-chaiml-m-67425-v1-mkmlizer: ║ Account ID: 7997a29f-0ceb-4cc7-9adf-840c57b4ae6f ║
chaiml-98p-2ff-chaiml-m-67425-v1-mkmlizer: ║ Expiration: 2028-03-31 23:59:59 ║
chaiml-98p-2ff-chaiml-m-67425-v1-mkmlizer: ║ ║
chaiml-98p-2ff-chaiml-m-67425-v1-mkmlizer: ╚═════════════════════════════════════════════════════════════════════╝
chaiml-98p-2ff-chaiml-m-39704-v1-mkmlizer: bash: cannot set terminal process group (-1): Inappropriate ioctl for device
chaiml-98p-2ff-chaiml-m-39704-v1-mkmlizer: bash: no job control in this shell
chaiml-98p-2ff-chaiml-m-39704-v1-mkmlizer: /root/miniconda3/envs/nvidia/lib/python3.11/site-packages/mk1/__init__.py:1: UserWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html. The pkg_resources package is slated for removal as early as 2025-11-30. Refrain from using this package or pin to Setuptools<81.
chaiml-98p-2ff-chaiml-m-39704-v1-mkmlizer: __import__('pkg_resources').declare_namespace(__name__)
chaiml-98p-2ff-chaiml-m-39704-v1-mkmlizer: ╔═════════════════════════════════════════════════════════════════════╗
chaiml-98p-2ff-chaiml-m-39704-v1-mkmlizer: ║ ║
chaiml-98p-2ff-chaiml-m-39704-v1-mkmlizer: ║ ██████ ██████ █████ ████ ████ ║
chaiml-98p-2ff-chaiml-m-39704-v1-mkmlizer: ║ ░░██████ ██████ ░░███ ███░ ░░███ ║
chaiml-98p-2ff-chaiml-m-39704-v1-mkmlizer: ║ ░███░█████░███ ░███ ███ ░███ ║
chaiml-98p-2ff-chaiml-m-39704-v1-mkmlizer: ║ ░███░░███ ░███ ░███████ ░███ ║
chaiml-98p-2ff-chaiml-m-39704-v1-mkmlizer: ║ ░███ ░░░ ░███ ░███░░███ ░███ ║
chaiml-98p-2ff-chaiml-m-39704-v1-mkmlizer: ║ ░███ ░███ ░███ ░░███ ░███ ║
chaiml-98p-2ff-chaiml-m-39704-v1-mkmlizer: ║ █████ █████ █████ ░░████ █████ ║
chaiml-98p-2ff-chaiml-m-39704-v1-mkmlizer: ║ ░░░░░ ░░░░░ ░░░░░ ░░░░ ░░░░░ ║
chaiml-98p-2ff-chaiml-m-39704-v1-mkmlizer: ║ ║
chaiml-98p-2ff-chaiml-m-39704-v1-mkmlizer: ║ Version: 0.30.6+torch280 ║
chaiml-98p-2ff-chaiml-m-39704-v1-mkmlizer: ║ Features: FLYWHEEL, CUDA ║
chaiml-98p-2ff-chaiml-m-39704-v1-mkmlizer: ║ Copyright 2023-2025 MK ONE TECHNOLOGIES Inc. ║
chaiml-98p-2ff-chaiml-m-39704-v1-mkmlizer: ║ https://mk1.ai ║
chaiml-98p-2ff-chaiml-m-39704-v1-mkmlizer: ║ ║
chaiml-98p-2ff-chaiml-m-39704-v1-mkmlizer: ║ The license key for the current software has been verified as ║
chaiml-98p-2ff-chaiml-m-39704-v1-mkmlizer: ║ belonging to: ║
chaiml-98p-2ff-chaiml-m-39704-v1-mkmlizer: ║ ║
chaiml-98p-2ff-chaiml-m-39704-v1-mkmlizer: ║ Chai Research Corp. ║
chaiml-98p-2ff-chaiml-m-39704-v1-mkmlizer: ║ Account ID: 7997a29f-0ceb-4cc7-9adf-840c57b4ae6f ║
chaiml-98p-2ff-chaiml-m-39704-v1-mkmlizer: ║ Expiration: 2028-03-31 23:59:59 ║
chaiml-98p-2ff-chaiml-m-39704-v1-mkmlizer: ║ ║
chaiml-98p-2ff-chaiml-m-39704-v1-mkmlizer: ╚═════════════════════════════════════════════════════════════════════╝
chaiml-mistral-24b-2048-83002-v4-mkmlizer: /root/miniconda3/envs/nvidia/lib/python3.11/site-packages/mk1/__init__.py:1: UserWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html. The pkg_resources package is slated for removal as early as 2025-11-30. Refrain from using this package or pin to Setuptools<81.
chaiml-mistral-24b-2048-83002-v4-mkmlizer: __import__('pkg_resources').declare_namespace(__name__)
chaiml-mistral-24b-2048-83002-v4-mkmlizer: ╔═════════════════════════════════════════════════════════════════════╗
chaiml-mistral-24b-2048-83002-v4-mkmlizer: ║ ║
chaiml-mistral-24b-2048-83002-v4-mkmlizer: ║ ██████ ██████ █████ ████ ████ ║
chaiml-mistral-24b-2048-83002-v4-mkmlizer: ║ ░░██████ ██████ ░░███ ███░ ░░███ ║
chaiml-mistral-24b-2048-83002-v4-mkmlizer: ║ ░███░█████░███ ░███ ███ ░███ ║
chaiml-mistral-24b-2048-83002-v4-mkmlizer: ║ ░███░░███ ░███ ░███████ ░███ ║
chaiml-mistral-24b-2048-83002-v4-mkmlizer: ║ ░███ ░░░ ░███ ░███░░███ ░███ ║
chaiml-mistral-24b-2048-83002-v4-mkmlizer: ║ ░███ ░███ ░███ ░░███ ░███ ║
chaiml-mistral-24b-2048-83002-v4-mkmlizer: ║ █████ █████ █████ ░░████ █████ ║
chaiml-mistral-24b-2048-83002-v4-mkmlizer: ║ ░░░░░ ░░░░░ ░░░░░ ░░░░ ░░░░░ ║
chaiml-mistral-24b-2048-83002-v4-mkmlizer: ║ ║
chaiml-mistral-24b-2048-83002-v4-mkmlizer: ║ Version: 0.30.6+torch280 ║
chaiml-mistral-24b-2048-83002-v4-mkmlizer: ║ Features: FLYWHEEL, CUDA ║
chaiml-mistral-24b-2048-83002-v4-mkmlizer: ║ Copyright 2023-2025 MK ONE TECHNOLOGIES Inc. ║
chaiml-mistral-24b-2048-83002-v4-mkmlizer: ║ https://mk1.ai ║
chaiml-mistral-24b-2048-83002-v4-mkmlizer: ║ ║
chaiml-mistral-24b-2048-83002-v4-mkmlizer: ║ The license key for the current software has been verified as ║
chaiml-mistral-24b-2048-83002-v4-mkmlizer: ║ belonging to: ║
chaiml-mistral-24b-2048-83002-v4-mkmlizer: ║ ║
chaiml-mistral-24b-2048-83002-v4-mkmlizer: ║ Chai Research Corp. ║
chaiml-mistral-24b-2048-83002-v4-mkmlizer: ║ Account ID: 7997a29f-0ceb-4cc7-9adf-840c57b4ae6f ║
chaiml-mistral-24b-2048-83002-v4-mkmlizer: ║ Expiration: 2028-03-31 23:59:59 ║
chaiml-mistral-24b-2048-83002-v4-mkmlizer: ║ ║
chaiml-mistral-24b-2048-83002-v4-mkmlizer: ╚═════════════════════════════════════════════════════════════════════╝
chaiml-98p-2ff-chaiml-m-67425-v1-mkmlizer: Downloaded to shared memory in 95.137s
chaiml-98p-2ff-chaiml-m-67425-v1-mkmlizer: Checking if ChaiML/98p_2ff_chaiml_mistral_24b_2048_83002_v1_cp468_prod_rm_merged already exists in ChaiML
chaiml-98p-2ff-chaiml-m-67425-v1-mkmlizer: quantizing model to /dev/shm/model_cache, profile:q4, folder:/tmp/tmpx7vf7yk9, device:0
chaiml-98p-2ff-chaiml-m-67425-v1-mkmlizer: Saving flywheel model at /dev/shm/model_cache
chaiml-98p-2ff-chaiml-m-39704-v1-mkmlizer: Downloaded to shared memory in 88.541s
chaiml-98p-2ff-chaiml-m-39704-v1-mkmlizer: Checking if ChaiML/98p_2ff_chaiml_mistral_24b_2048_83002_v1_cp936_prod_rm_merged already exists in ChaiML
chaiml-98p-2ff-chaiml-m-39704-v1-mkmlizer: quantizing model to /dev/shm/model_cache, profile:q4, folder:/tmp/tmpg1p4j0gs, device:0
chaiml-98p-2ff-chaiml-m-39704-v1-mkmlizer: Saving flywheel model at /dev/shm/model_cache
chaiml-mistral-24b-2048-83002-v4-mkmlizer: Downloaded to shared memory in 92.684s
chaiml-mistral-24b-2048-83002-v4-mkmlizer: Checking if ChaiML/mistral_24b_2048_gemini_opus_ds_v10_843_merged already exists in ChaiML
chaiml-mistral-24b-2048-83002-v4-mkmlizer: quantizing model to /dev/shm/model_cache, profile:q4, folder:/tmp/tmpilq4q5kg, device:0
chaiml-mistral-24b-2048-83002-v4-mkmlizer: Saving flywheel model at /dev/shm/model_cache
chaiml-98p-2ff-chaiml-m-39704-v1-mkmlizer:
Loading 0: 0%| | 0.00/363 [00:00<?, ?it/s]
Loading 0: 1%| | 3.00/363 [00:01<03:53, 1.54it/s]
Loading 0: 1%| | 3.00/363 [00:01<03:53, 1.54it/s]
Loading 0: 1%| | 4.00/363 [00:03<06:17, 1.05s/it]
Loading 0: 1%| | 4.00/363 [00:03<06:17, 1.05s/it]
Loading 0: 1%|▏ | 5.00/363 [00:05<08:18, 1.39s/it]
Loading 0: 1%|▏ | 5.00/363 [00:05<08:18, 1.39s/it]
Loading 0: 3%|▎ | 12.0/363 [00:07<02:44, 2.14it/s]
Loading 0: 3%|▎ | 12.0/363 [00:07<02:44, 2.14it/s]
Loading 0: 4%|▎ | 13.0/363 [00:09<03:51, 1.51it/s]
Loading 0: 4%|▎ | 13.0/363 [00:09<03:51, 1.51it/s]
Loading 0: 4%|▍ | 14.0/363 [00:11<05:00, 1.16it/s]
Loading 0: 4%|▍ | 14.0/363 [00:11<05:00, 1.16it/s]
Loading 0: 4%|▍ | 15.0/363 [00:13<06:14, 1.08s/it]
Loading 0: 4%|▍ | 15.0/363 [00:13<06:14, 1.08s/it]
Loading 0: 6%|▌ | 21.0/363 [00:15<03:44, 1.52it/s]
Loading 0: 6%|▌ | 21.0/363 [00:15<03:44, 1.52it/s]
Loading 0: 6%|▌ | 22.0/363 [00:17<04:38, 1.23it/s]
Loading 0: 6%|▌ | 22.0/363 [00:17<04:38, 1.23it/s]
Loading 0: 6%|▋ | 23.0/363 [00:19<05:40, 1.00s/it]
Loading 0: 6%|▋ | 23.0/363 [00:19<05:40, 1.00s/it]
Loading 0: 9%|▊ | 31.0/363 [00:20<02:32, 2.17it/s]
Loading 0: 9%|▊ | 31.0/363 [00:20<02:32, 2.17it/s]
Loading 0: 9%|▉ | 34.0/363 [00:22<02:54, 1.88it/s]
Loading 0: 9%|▉ | 34.0/363 [00:22<02:54, 1.88it/s]
Loading 0: 10%|▉ | 35.0/363 [00:24<03:42, 1.47it/s]
Loading 0: 10%|▉ | 35.0/363 [00:24<03:42, 1.47it/s]
Loading 0: 10%|▉ | 36.0/363 [00:26<04:41, 1.16it/s]
Loading 0: 10%|▉ | 36.0/363 [00:26<04:41, 1.16it/s]
Loading 0: 11%|█ | 39.0/363 [00:28<04:12, 1.28it/s]
Loading 0: 11%|█ | 39.0/363 [00:28<04:12, 1.28it/s]
Loading 0: 11%|█ | 40.0/363 [00:30<05:04, 1.06it/s]
Loading 0: 11%|█ | 40.0/363 [00:30<05:04, 1.06it/s]
Loading 0: 11%|█▏ | 41.0/363 [00:32<06:04, 1.13s/it]
Loading 0: 11%|█▏ | 41.0/363 [00:32<06:04, 1.13s/it]
Loading 0: 13%|█▎ | 49.0/363 [00:33<02:29, 2.10it/s]
Loading 0: 13%|█▎ | 49.0/363 [00:33<02:29, 2.10it/s]
Loading 0: 14%|█▍ | 52.0/363 [00:35<02:48, 1.84it/s]
Loading 0: 14%|█▍ | 52.0/363 [00:35<02:48, 1.84it/s]
Loading 0: 15%|█▍ | 53.0/363 [00:37<03:34, 1.44it/s]
Loading 0: 15%|█▍ | 53.0/363 [00:37<03:34, 1.44it/s]
Loading 0: 15%|█▍ | 54.0/363 [00:39<04:30, 1.14it/s]
Loading 0: 15%|█▍ | 54.0/363 [00:39<04:30, 1.14it/s]
Loading 0: 16%|█▌ | 57.0/363 [00:41<04:01, 1.27it/s]
Loading 0: 16%|█▌ | 57.0/363 [00:41<04:01, 1.27it/s]
Loading 0: 16%|█▌ | 58.0/363 [00:43<04:51, 1.05it/s]
Loading 0: 16%|█▌ | 58.0/363 [00:43<04:51, 1.05it/s]
Loading 0: 16%|█▋ | 59.0/363 [00:45<05:47, 1.14s/it]
Loading 0: 16%|█▋ | 59.0/363 [00:45<05:47, 1.14s/it]
Loading 0: 18%|█▊ | 67.0/363 [00:46<02:21, 2.09it/s]
Loading 0: 18%|█▊ | 67.0/363 [00:46<02:21, 2.09it/s]
Loading 0: 19%|█▉ | 70.0/363 [00:49<02:40, 1.83it/s]
Loading 0: 19%|█▉ | 70.0/363 [00:49<02:40, 1.83it/s]
Loading 0: 20%|█▉ | 71.0/363 [00:50<03:23, 1.43it/s]
Loading 0: 20%|█▉ | 71.0/363 [00:50<03:23, 1.43it/s]
Loading 0: 20%|█▉ | 72.0/363 [00:52<04:15, 1.14it/s]
Loading 0: 20%|█▉ | 72.0/363 [00:52<04:15, 1.14it/s]
Loading 0: 21%|██ | 75.0/363 [00:54<03:47, 1.27it/s]
Loading 0: 21%|██ | 75.0/363 [00:54<03:47, 1.27it/s]
Loading 0: 21%|██ | 76.0/363 [00:56<04:33, 1.05it/s]
Loading 0: 21%|██ | 76.0/363 [00:56<04:33, 1.05it/s]
Loading 0: 21%|██ | 77.0/363 [00:58<05:26, 1.14s/it]
Loading 0: 21%|██ | 77.0/363 [00:58<05:26, 1.14s/it]
Loading 0: 23%|██▎ | 85.0/363 [00:59<02:13, 2.09it/s]
Loading 0: 23%|██▎ | 85.0/363 [00:59<02:13, 2.09it/s]
Loading 0: 24%|██▍ | 88.0/363 [01:02<02:29, 1.84it/s]
Loading 0: 24%|██▍ | 88.0/363 [01:02<02:29, 1.84it/s]
Loading 0: 25%|██▍ | 89.0/363 [01:04<03:10, 1.44it/s]
Loading 0: 25%|██▍ | 89.0/363 [01:04<03:10, 1.44it/s]
Loading 0: 25%|██▍ | 90.0/363 [01:06<03:59, 1.14it/s]
Loading 0: 25%|██▍ | 90.0/363 [01:06<03:59, 1.14it/s]
Loading 0: 27%|██▋ | 98.0/363 [01:07<01:54, 2.30it/s]
Loading 0: 27%|██▋ | 98.0/363 [01:07<01:54, 2.30it/s]
Loading 0: 28%|██▊ | 101/363 [01:09<02:10, 2.01it/s]
Loading 0: 28%|██▊ | 101/363 [01:09<02:10, 2.01it/s]
Loading 0: 28%|██▊ | 102/363 [01:11<02:47, 1.55it/s]
Loading 0: 28%|██▊ | 102/363 [01:11<02:47, 1.55it/s]
Loading 0: 28%|██▊ | 103/363 [01:13<03:33, 1.22it/s]
Loading 0: 28%|██▊ | 103/363 [01:13<03:33, 1.22it/s]
Loading 0: 29%|██▉ | 106/363 [01:15<03:18, 1.30it/s]
Loading 0: 29%|██▉ | 106/363 [01:15<03:18, 1.30it/s]
Loading 0: 29%|██▉ | 107/363 [01:17<03:59, 1.07it/s]
Loading 0: 29%|██▉ | 107/363 [01:17<03:59, 1.07it/s]
Loading 0: 30%|██▉ | 108/363 [01:19<04:45, 1.12s/it]
Loading 0: 30%|██▉ | 108/363 [01:19<04:45, 1.12s/it]
Loading 0: 31%|███ | 111/363 [01:21<03:50, 1.09it/s]
Loading 0: 31%|███ | 111/363 [01:21<03:50, 1.09it/s]
Loading 0: 31%|███ | 112/363 [01:23<04:31, 1.08s/it]
Loading 0: 31%|███ | 112/363 [01:23<04:31, 1.08s/it]
Loading 0: 31%|███ | 113/363 [01:25<05:16, 1.26s/it]
Loading 0: 31%|███ | 113/363 [01:25<05:16, 1.26s/it]
Loading 0: 33%|███▎ | 121/363 [01:26<01:59, 2.02it/s]
Loading 0: 33%|███▎ | 121/363 [01:26<01:59, 2.02it/s]
Loading 0: 34%|███▍ | 124/363 [01:28<02:13, 1.79it/s]
Loading 0: 34%|███▍ | 124/363 [01:28<02:13, 1.79it/s]
Loading 0: 34%|███▍ | 125/363 [01:30<02:49, 1.40it/s]
Loading 0: 34%|███▍ | 125/363 [01:30<02:49, 1.40it/s]
Loading 0: 35%|███▍ | 126/363 [01:32<03:31, 1.12it/s]
Loading 0: 35%|███▍ | 126/363 [01:32<03:31, 1.12it/s]
Loading 0: 36%|███▌ | 129/363 [01:34<03:06, 1.25it/s]
Loading 0: 36%|███▌ | 129/363 [01:34<03:06, 1.25it/s]
Loading 0: 36%|███▌ | 130/363 [01:36<03:44, 1.04it/s]
Loading 0: 36%|███▌ | 130/363 [01:36<03:44, 1.04it/s]
Loading 0: 36%|███▌ | 131/363 [01:38<04:27, 1.15s/it]
Loading 0: 36%|███▌ | 131/363 [01:38<04:27, 1.15s/it]
Loading 0: 38%|███▊ | 139/363 [01:39<01:48, 2.07it/s]
Loading 0: 38%|███▊ | 139/363 [01:39<01:48, 2.07it/s]
Loading 0: 39%|███▉ | 142/363 [01:41<02:01, 1.82it/s]
Loading 0: 39%|███▉ | 142/363 [01:41<02:01, 1.82it/s]
Loading 0: 39%|███▉ | 143/363 [01:43<02:34, 1.43it/s]
Loading 0: 39%|███▉ | 143/363 [01:43<02:34, 1.43it/s]
Loading 0: 40%|███▉ | 144/363 [01:45<03:13, 1.13it/s]
Loading 0: 40%|███▉ | 144/363 [01:45<03:13, 1.13it/s]
Loading 0: 40%|████ | 147/363 [01:47<02:51, 1.26it/s]
Loading 0: 40%|████ | 147/363 [01:47<02:51, 1.26it/s]
Loading 0: 41%|████ | 148/363 [01:49<03:26, 1.04it/s]
Loading 0: 41%|████ | 148/363 [01:49<03:26, 1.04it/s]
Loading 0: 41%|████ | 149/363 [01:51<04:05, 1.15s/it]
Loading 0: 41%|████ | 149/363 [01:51<04:05, 1.15s/it]
Loading 0: 43%|████▎ | 157/363 [01:52<01:39, 2.07it/s]
Loading 0: 43%|████▎ | 157/363 [01:52<01:39, 2.07it/s]
Loading 0: 44%|████▍ | 160/363 [01:54<01:51, 1.82it/s]
Loading 0: 44%|████▍ | 160/363 [01:54<01:51, 1.82it/s]
Loading 0: 44%|████▍ | 161/363 [01:56<02:22, 1.42it/s]
Loading 0: 44%|████▍ | 161/363 [01:56<02:22, 1.42it/s]
Loading 0: 45%|████▍ | 162/363 [01:58<02:57, 1.13it/s]
Loading 0: 45%|████▍ | 162/363 [01:58<02:57, 1.13it/s]
Loading 0: 45%|████▌ | 165/363 [02:00<02:37, 1.26it/s]
Loading 0: 45%|████▌ | 165/363 [02:00<02:37, 1.26it/s]
Loading 0: 46%|████▌ | 166/363 [02:02<03:09, 1.04it/s]
Loading 0: 46%|████▌ | 166/363 [02:02<03:09, 1.04it/s]
Loading 0: 46%|████▌ | 167/363 [02:04<03:45, 1.15s/it]
Loading 0: 46%|████▌ | 167/363 [02:04<03:45, 1.15s/it]
Loading 0: 48%|████▊ | 175/363 [02:05<01:30, 2.07it/s]
Loading 0: 48%|████▊ | 175/363 [02:05<01:30, 2.07it/s]
Loading 0: 49%|████▉ | 178/363 [02:08<01:41, 1.82it/s]
Loading 0: 49%|████▉ | 178/363 [02:08<01:41, 1.82it/s]
Loading 0: 49%|████▉ | 179/363 [02:09<02:09, 1.43it/s]
Loading 0: 49%|████▉ | 179/363 [02:09<02:09, 1.43it/s]
Loading 0: 50%|████▉ | 180/363 [02:11<02:41, 1.13it/s]
Loading 0: 50%|████▉ | 180/363 [02:11<02:41, 1.13it/s]
Loading 0: 50%|█████ | 183/363 [02:13<02:22, 1.26it/s]
Loading 0: 50%|█████ | 183/363 [02:13<02:22, 1.26it/s]
Loading 0: 51%|█████ | 184/363 [02:15<02:51, 1.04it/s]
Loading 0: 51%|█████ | 184/363 [02:15<02:51, 1.04it/s]
Loading 0: 51%|█████ | 185/363 [02:17<03:24, 1.15s/it]
Loading 0: 51%|█████ | 185/363 [02:17<03:24, 1.15s/it]
Loading 0: 53%|█████▎ | 193/363 [02:19<01:22, 2.06it/s]
Loading 0: 53%|█████▎ | 193/363 [02:19<01:22, 2.06it/s]
Loading 0: 54%|█████▍ | 196/363 [02:21<01:32, 1.81it/s]
Loading 0: 54%|█████▍ | 196/363 [02:21<01:32, 1.81it/s]
Loading 0: 54%|█████▍ | 197/363 [02:23<01:56, 1.42it/s]
Loading 0: 54%|█████▍ | 197/363 [02:23<01:56, 1.42it/s]
Loading 0: 55%|█████▍ | 198/363 [02:25<02:25, 1.13it/s]
Loading 0: 55%|█████▍ | 198/363 [02:25<02:25, 1.13it/s]
Loading 0: 55%|█████▌ | 201/363 [02:27<02:08, 1.26it/s]
Loading 0: 55%|█████▌ | 201/363 [02:27<02:08, 1.26it/s]
Loading 0: 56%|█████▌ | 202/363 [02:29<02:34, 1.04it/s]
Loading 0: 56%|█████▌ | 202/363 [02:29<02:34, 1.04it/s]
Loading 0: 56%|█████▌ | 203/363 [02:31<03:03, 1.15s/it]
Loading 0: 56%|█████▌ | 203/363 [02:31<03:03, 1.15s/it]
Loading 0: 58%|█████▊ | 211/363 [02:32<01:13, 2.07it/s]
Loading 0: 58%|█████▊ | 211/363 [02:32<01:13, 2.07it/s]
Loading 0: 59%|█████▉ | 214/363 [02:34<01:21, 1.82it/s]
Loading 0: 59%|█████▉ | 214/363 [02:34<01:21, 1.82it/s]
Loading 0: 59%|█████▉ | 215/363 [02:36<01:43, 1.43it/s]
Loading 0: 59%|█████▉ | 215/363 [02:36<01:43, 1.43it/s]
Loading 0: 60%|█████▉ | 216/363 [02:38<02:09, 1.14it/s]
Loading 0: 60%|█████▉ | 216/363 [02:38<02:09, 1.14it/s]
Loading 0: 60%|██████ | 219/363 [02:40<01:53, 1.26it/s]
Loading 0: 60%|██████ | 219/363 [02:40<01:53, 1.26it/s]
Loading 0: 61%|██████ | 220/363 [02:42<02:17, 1.04it/s]
Loading 0: 61%|██████ | 220/363 [02:42<02:17, 1.04it/s]
Loading 0: 61%|██████ | 221/363 [02:44<02:42, 1.15s/it]
Loading 0: 61%|██████ | 221/363 [02:44<02:42, 1.15s/it]
Loading 0: 63%|██████▎ | 228/363 [02:45<01:08, 1.97it/s]
Loading 0: 63%|██████▎ | 228/363 [02:45<01:08, 1.97it/s]
Loading 0: 64%|██████▍ | 232/363 [02:47<01:11, 1.84it/s]
Loading 0: 64%|██████▍ | 232/363 [02:47<01:11, 1.84it/s]
Loading 0: 64%|██████▍ | 233/363 [02:49<01:29, 1.45it/s]
Loading 0: 64%|██████▍ | 233/363 [02:49<01:29, 1.45it/s]
Loading 0: 64%|██████▍ | 234/363 [02:51<01:51, 1.15it/s]
Loading 0: 64%|██████▍ | 234/363 [02:51<01:51, 1.15it/s]
Loading 0: 65%|██████▌ | 237/363 [02:53<01:38, 1.28it/s]
Loading 0: 65%|██████▌ | 237/363 [02:53<01:38, 1.28it/s]
Loading 0: 66%|██████▌ | 238/363 [02:55<01:58, 1.05it/s]
Loading 0: 66%|██████▌ | 238/363 [02:55<01:58, 1.05it/s]
Loading 0: 66%|██████▌ | 239/363 [02:57<02:20, 1.14s/it]
Loading 0: 66%|██████▌ | 239/363 [02:57<02:20, 1.14s/it]
Loading 0: 68%|██████▊ | 247/363 [02:58<00:55, 2.08it/s]
Loading 0: 68%|██████▊ | 247/363 [02:58<00:55, 2.08it/s]
Loading 0: 69%|██████▉ | 250/363 [03:00<01:01, 1.82it/s]
Loading 0: 69%|██████▉ | 250/363 [03:00<01:01, 1.82it/s]
Loading 0: 69%|██████▉ | 251/363 [03:02<01:18, 1.43it/s]
Loading 0: 69%|██████▉ | 251/363 [03:02<01:18, 1.43it/s]
Loading 0: 69%|██████▉ | 252/363 [03:04<01:37, 1.13it/s]
Loading 0: 69%|██████▉ | 252/363 [03:04<01:37, 1.13it/s]
Loading 0: 70%|███████ | 255/363 [03:06<01:25, 1.26it/s]
Loading 0: 70%|███████ | 255/363 [03:06<01:25, 1.26it/s]
Loading 0: 71%|███████ | 256/363 [03:08<01:42, 1.04it/s]
Loading 0: 71%|███████ | 256/363 [03:08<01:42, 1.04it/s]
Loading 0: 71%|███████ | 257/363 [03:10<02:01, 1.15s/it]
Loading 0: 71%|███████ | 257/363 [03:10<02:01, 1.15s/it]
Loading 0: 73%|███████▎ | 265/363 [03:11<00:47, 2.07it/s]
Loading 0: 73%|███████▎ | 265/363 [03:11<00:47, 2.07it/s]
Loading 0: 74%|███████▍ | 268/363 [03:14<00:52, 1.82it/s]
Loading 0: 74%|███████▍ | 268/363 [03:14<00:52, 1.82it/s]
Loading 0: 74%|███████▍ | 269/363 [03:16<01:05, 1.43it/s]
Loading 0: 74%|███████▍ | 269/363 [03:16<01:05, 1.43it/s]
Loading 0: 74%|███████▍ | 270/363 [03:18<01:21, 1.14it/s]
Loading 0: 74%|███████▍ | 270/363 [03:18<01:21, 1.14it/s]
Loading 0: 75%|███████▌ | 273/363 [03:33<03:36, 2.41s/it]
Loading 0: 75%|███████▌ | 273/363 [03:33<03:36, 2.41s/it]
Loading 0: 75%|███████▌ | 274/363 [03:34<03:27, 2.33s/it]
Loading 0: 75%|███████▌ | 274/363 [03:34<03:27, 2.33s/it]
Loading 0: 76%|███████▌ | 275/363 [03:36<03:21, 2.29s/it]
Loading 0: 76%|███████▌ | 275/363 [03:36<03:21, 2.29s/it]
Loading 0: 78%|███████▊ | 283/363 [03:38<01:08, 1.17it/s]
Loading 0: 78%|███████▊ | 283/363 [03:38<01:08, 1.17it/s]
Loading 0: 79%|███████▉ | 286/363 [03:40<01:02, 1.23it/s]
Loading 0: 79%|███████▉ | 286/363 [03:40<01:02, 1.23it/s]
Loading 0: 79%|███████▉ | 287/363 [03:42<01:09, 1.09it/s]
Loading 0: 79%|███████▉ | 287/363 [03:42<01:09, 1.09it/s]
Loading 0: 79%|███████▉ | 288/363 [03:43<01:19, 1.06s/it]
Loading 0: 79%|███████▉ | 288/363 [03:43<01:19, 1.06s/it]
Loading 0: 80%|████████ | 291/363 [03:45<01:03, 1.13it/s]
Loading 0: 80%|████████ | 291/363 [03:45<01:03, 1.13it/s]
Loading 0: 80%|████████ | 292/363 [03:47<01:13, 1.04s/it]
Loading 0: 80%|████████ | 292/363 [03:47<01:13, 1.04s/it]
Loading 0: 81%|████████ | 293/363 [03:49<01:24, 1.21s/it]
Loading 0: 81%|████████ | 293/363 [03:49<01:24, 1.21s/it]
Loading 0: 83%|████████▎ | 301/363 [03:50<00:30, 2.02it/s]
Loading 0: 83%|████████▎ | 301/363 [03:50<00:30, 2.02it/s]
Loading 0: 84%|████████▎ | 304/363 [03:53<00:33, 1.77it/s]
Loading 0: 84%|████████▎ | 304/363 [03:53<00:33, 1.77it/s]
Loading 0: 84%|████████▍ | 305/363 [03:54<00:41, 1.40it/s]
Loading 0: 84%|████████▍ | 305/363 [03:54<00:41, 1.40it/s]
Loading 0: 84%|████████▍ | 306/363 [03:56<00:50, 1.12it/s]
Loading 0: 84%|████████▍ | 306/363 [03:56<00:50, 1.12it/s]
Loading 0: 85%|████████▌ | 309/363 [03:58<00:43, 1.25it/s]
Loading 0: 85%|████████▌ | 309/363 [03:58<00:43, 1.25it/s]
Loading 0: 85%|████████▌ | 310/363 [04:00<00:51, 1.04it/s]
Loading 0: 85%|████████▌ | 310/363 [04:00<00:51, 1.04it/s]
Loading 0: 86%|████████▌ | 311/363 [04:02<00:59, 1.15s/it]
Loading 0: 86%|████████▌ | 311/363 [04:02<00:59, 1.15s/it]
Loading 0: 88%|████████▊ | 319/363 [04:03<00:20, 2.10it/s]
Loading 0: 88%|████████▊ | 319/363 [04:03<00:20, 2.10it/s]
Loading 0: 89%|████████▊ | 322/363 [04:06<00:22, 1.83it/s]
Loading 0: 89%|████████▊ | 322/363 [04:06<00:22, 1.83it/s]
Loading 0: 89%|████████▉ | 323/363 [04:08<00:27, 1.44it/s]
Loading 0: 89%|████████▉ | 323/363 [04:08<00:27, 1.44it/s]
Loading 0: 89%|████████▉ | 324/363 [04:10<00:34, 1.14it/s]
Loading 0: 89%|████████▉ | 324/363 [04:10<00:34, 1.14it/s]
Loading 0: 90%|█████████ | 327/363 [04:12<00:28, 1.27it/s]
Loading 0: 90%|█████████ | 327/363 [04:12<00:28, 1.27it/s]
Loading 0: 90%|█████████ | 328/363 [04:13<00:33, 1.05it/s]
Loading 0: 90%|█████████ | 328/363 [04:13<00:33, 1.05it/s]
Loading 0: 91%|█████████ | 329/363 [04:15<00:38, 1.15s/it]
Loading 0: 91%|█████████ | 329/363 [04:15<00:38, 1.15s/it]
Loading 0: 93%|█████████▎| 337/363 [04:17<00:12, 2.11it/s]
Loading 0: 93%|█████████▎| 337/363 [04:17<00:12, 2.11it/s]
Loading 0: 94%|█████████▎| 340/363 [04:19<00:12, 1.84it/s]
Loading 0: 94%|█████████▎| 340/363 [04:19<00:12, 1.84it/s]
Loading 0: 94%|█████████▍| 341/363 [04:21<00:15, 1.44it/s]
Loading 0: 94%|█████████▍| 341/363 [04:21<00:15, 1.44it/s]
Loading 0: 94%|█████████▍| 342/363 [04:23<00:18, 1.13it/s]
Loading 0: 94%|█████████▍| 342/363 [04:23<00:18, 1.13it/s]
Loading 0: 95%|█████████▌| 345/363 [04:25<00:14, 1.26it/s]
Loading 0: 95%|█████████▌| 345/363 [04:25<00:14, 1.26it/s]
Loading 0: 95%|█████████▌| 346/363 [04:27<00:16, 1.04it/s]
Loading 0: 95%|█████████▌| 346/363 [04:27<00:16, 1.04it/s]
Loading 0: 96%|█████████▌| 347/363 [04:29<00:18, 1.15s/it]
Loading 0: 96%|█████████▌| 347/363 [04:29<00:18, 1.15s/it]
Loading 0: 98%|█████████▊| 355/363 [04:30<00:03, 2.09it/s]
Loading 0: 98%|█████████▊| 355/363 [04:30<00:03, 2.09it/s]
Loading 0: 99%|█████████▉| 359/363 [04:32<00:02, 1.87it/s]
Loading 0: 99%|█████████▉| 359/363 [04:32<00:02, 1.87it/s]
Loading 0: 99%|█████████▉| 360/363 [04:34<00:02, 1.48it/s]
Loading 0: 99%|█████████▉| 360/363 [04:34<00:02, 1.48it/s]
Loading 0: 99%|█████████▉| 361/363 [04:36<00:01, 1.18it/s]
Loading 0: 99%|█████████▉| 361/363 [04:36<00:01, 1.18it/s]
Loading 0: 100%|██████████| 363/363 [04:36<00:00, 1.18it/s]
Loading 0: 100%|██████████| 363/363 [04:36<00:00, 1.31it/s]
chaiml-98p-2ff-chaiml-m-39704-v1-mkmlizer: The tokenizer you are loading from '/tmp/tmpg1p4j0gs' with an incorrect regex pattern: https://huggingface.co/mistralai/Mistral-Small-3.1-24B-Instruct-2503/discussions/84#69121093e8b480e709447d5e. This will lead to incorrect tokenization. You should set the `fix_mistral_regex=True` flag when loading this tokenizer to fix this issue.
chaiml-98p-2ff-chaiml-m-39704-v1-mkmlizer: quantized model in 283.075s
chaiml-98p-2ff-chaiml-m-39704-v1-mkmlizer: Processed model ChaiML/98p_2ff_chaiml_mistral_24b_2048_83002_v1_cp936_prod_rm_merged in 371.617s
chaiml-98p-2ff-chaiml-m-39704-v1-mkmlizer: creating bucket guanaco-mkml-models
chaiml-98p-2ff-chaiml-m-39704-v1-mkmlizer: Bucket 's3://guanaco-mkml-models/' created
chaiml-98p-2ff-chaiml-m-39704-v1-mkmlizer: uploading /dev/shm/model_cache to s3://guanaco-mkml-models/chaiml-98p-2ff-chaiml-m-39704-v1/nvidia
chaiml-98p-2ff-chaiml-m-39704-v1-mkmlizer: cp /dev/shm/model_cache/config.json s3://guanaco-mkml-models/chaiml-98p-2ff-chaiml-m-39704-v1/nvidia/config.json
chaiml-98p-2ff-chaiml-m-39704-v1-mkmlizer: cp /dev/shm/model_cache/special_tokens_map.json s3://guanaco-mkml-models/chaiml-98p-2ff-chaiml-m-39704-v1/nvidia/special_tokens_map.json
chaiml-98p-2ff-chaiml-m-67425-v1-mkmlizer:
Loading 0: 0%| | 0.00/363 [00:00<?, ?it/s]
Loading 0: 1%| | 3.00/363 [00:02<04:14, 1.41it/s]
Loading 0: 1%| | 3.00/363 [00:02<04:14, 1.41it/s]
Loading 0: 1%| | 4.00/363 [00:04<06:52, 1.15s/it]
Loading 0: 1%| | 4.00/363 [00:04<06:52, 1.15s/it]
Loading 0: 1%|▏ | 5.00/363 [00:06<08:49, 1.48s/it]
Loading 0: 1%|▏ | 5.00/363 [00:06<08:49, 1.48s/it]
Loading 0: 3%|▎ | 12.0/363 [00:07<02:49, 2.08it/s]
Loading 0: 3%|▎ | 12.0/363 [00:07<02:49, 2.08it/s]
Loading 0: 4%|▎ | 13.0/363 [00:09<04:05, 1.43it/s]
Loading 0: 4%|▎ | 13.0/363 [00:09<04:05, 1.43it/s]
Loading 0: 4%|▍ | 14.0/363 [00:11<05:23, 1.08it/s]
Loading 0: 4%|▍ | 14.0/363 [00:11<05:23, 1.08it/s]
Loading 0: 4%|▍ | 15.0/363 [00:13<06:46, 1.17s/it]
Loading 0: 4%|▍ | 15.0/363 [00:13<06:46, 1.17s/it]
Loading 0: 6%|▌ | 21.0/363 [00:16<04:05, 1.40it/s]
Loading 0: 6%|▌ | 21.0/363 [00:16<04:05, 1.40it/s]
Loading 0: 6%|▌ | 22.0/363 [00:18<05:05, 1.12it/s]
Loading 0: 6%|▌ | 22.0/363 [00:18<05:05, 1.12it/s]
Loading 0: 6%|▋ | 23.0/363 [00:20<06:14, 1.10s/it]
Loading 0: 6%|▋ | 23.0/363 [00:20<06:14, 1.10s/it]
Loading 0: 8%|▊ | 30.0/363 [00:21<02:53, 1.92it/s]
Loading 0: 8%|▊ | 30.0/363 [00:21<02:53, 1.92it/s]
Loading 0: 9%|▉ | 34.0/363 [00:24<03:07, 1.75it/s]
Loading 0: 9%|▉ | 34.0/363 [00:24<03:07, 1.75it/s]
Loading 0: 10%|▉ | 35.0/363 [00:26<03:55, 1.39it/s]
Loading 0: 10%|▉ | 35.0/363 [00:26<03:55, 1.39it/s]
Loading 0: 10%|▉ | 36.0/363 [00:28<04:55, 1.11it/s]
Loading 0: 10%|▉ | 36.0/363 [00:28<04:55, 1.11it/s]
Loading 0: 11%|█ | 39.0/363 [00:30<04:24, 1.23it/s]
Loading 0: 11%|█ | 39.0/363 [00:30<04:24, 1.23it/s]
Loading 0: 11%|█ | 40.0/363 [00:32<05:17, 1.02it/s]
Loading 0: 11%|█ | 40.0/363 [00:32<05:17, 1.02it/s]
Loading 0: 11%|█▏ | 41.0/363 [00:34<06:17, 1.17s/it]
Loading 0: 11%|█▏ | 41.0/363 [00:34<06:17, 1.17s/it]
Loading 0: 13%|█▎ | 49.0/363 [00:35<02:34, 2.03it/s]
Loading 0: 13%|█▎ | 49.0/363 [00:35<02:34, 2.03it/s]
Loading 0: 14%|█▍ | 52.0/363 [00:38<02:55, 1.77it/s]
Loading 0: 14%|█▍ | 52.0/363 [00:38<02:55, 1.77it/s]
Loading 0: 15%|█▍ | 53.0/363 [00:40<03:43, 1.38it/s]
Loading 0: 15%|█▍ | 53.0/363 [00:40<03:43, 1.38it/s]
Loading 0: 15%|█▍ | 54.0/363 [00:42<04:41, 1.10it/s]
Loading 0: 15%|█▍ | 54.0/363 [00:42<04:41, 1.10it/s]
Loading 0: 16%|█▌ | 57.0/363 [00:44<04:11, 1.22it/s]
Loading 0: 16%|█▌ | 57.0/363 [00:44<04:11, 1.22it/s]
Loading 0: 16%|█▌ | 58.0/363 [00:46<05:03, 1.00it/s]
Loading 0: 16%|█▌ | 58.0/363 [00:46<05:03, 1.00it/s]
Loading 0: 16%|█▋ | 59.0/363 [00:48<06:01, 1.19s/it]
Loading 0: 16%|█▋ | 59.0/363 [00:48<06:01, 1.19s/it]
Loading 0: 18%|█▊ | 67.0/363 [00:49<02:27, 2.01it/s]
Loading 0: 18%|█▊ | 67.0/363 [00:49<02:27, 2.01it/s]
Loading 0: 19%|█▉ | 70.0/363 [00:51<02:47, 1.75it/s]
Loading 0: 19%|█▉ | 70.0/363 [00:51<02:47, 1.75it/s]
Loading 0: 20%|█▉ | 71.0/363 [00:53<03:32, 1.37it/s]
Loading 0: 20%|█▉ | 71.0/363 [00:53<03:32, 1.37it/s]
Loading 0: 20%|█▉ | 72.0/363 [00:56<04:27, 1.09it/s]
Loading 0: 20%|█▉ | 72.0/363 [00:56<04:27, 1.09it/s]
Loading 0: 21%|██ | 75.0/363 [00:58<03:58, 1.21it/s]
Loading 0: 21%|██ | 75.0/363 [00:58<03:58, 1.21it/s]
Loading 0: 21%|██ | 76.0/363 [01:00<04:47, 1.00s/it]
Loading 0: 21%|██ | 76.0/363 [01:00<04:47, 1.00s/it]
Loading 0: 21%|██ | 77.0/363 [01:02<05:41, 1.20s/it]
Loading 0: 21%|██ | 77.0/363 [01:02<05:41, 1.20s/it]
Loading 0: 23%|██▎ | 85.0/363 [01:03<02:18, 2.01it/s]
Loading 0: 23%|██▎ | 85.0/363 [01:03<02:18, 2.01it/s]
Loading 0: 24%|██▍ | 88.0/363 [01:05<02:36, 1.75it/s]
Loading 0: 24%|██▍ | 88.0/363 [01:05<02:36, 1.75it/s]
Loading 0: 25%|██▍ | 89.0/363 [01:07<03:19, 1.37it/s]
Loading 0: 25%|██▍ | 89.0/363 [01:07<03:19, 1.37it/s]
Loading 0: 25%|██▍ | 90.0/363 [01:09<04:11, 1.09it/s]
Loading 0: 25%|██▍ | 90.0/363 [01:09<04:11, 1.09it/s]
Loading 0: 27%|██▋ | 98.0/363 [01:11<01:59, 2.21it/s]
Loading 0: 27%|██▋ | 98.0/363 [01:11<01:59, 2.21it/s]
Loading 0: 28%|██▊ | 101/363 [01:13<02:16, 1.92it/s]
Loading 0: 28%|██▊ | 101/363 [01:13<02:16, 1.92it/s]
Loading 0: 28%|██▊ | 102/363 [01:15<02:56, 1.48it/s]
Loading 0: 28%|██▊ | 102/363 [01:15<02:56, 1.48it/s]
Loading 0: 28%|██▊ | 103/363 [01:17<03:45, 1.15it/s]
Loading 0: 28%|██▊ | 103/363 [01:17<03:45, 1.15it/s]
Loading 0: 29%|██▉ | 106/363 [01:19<03:29, 1.23it/s]
Loading 0: 29%|██▉ | 106/363 [01:19<03:29, 1.23it/s]
Loading 0: 29%|██▉ | 107/363 [01:21<04:13, 1.01it/s]
Loading 0: 29%|██▉ | 107/363 [01:21<04:13, 1.01it/s]
Loading 0: 30%|██▉ | 108/363 [01:23<05:02, 1.19s/it]
Loading 0: 30%|██▉ | 108/363 [01:23<05:02, 1.19s/it]
Loading 0: 31%|███ | 111/363 [01:25<04:04, 1.03it/s]
Loading 0: 31%|███ | 111/363 [01:25<04:04, 1.03it/s]
Loading 0: 31%|███ | 112/363 [01:27<04:47, 1.14s/it]
Loading 0: 31%|███ | 112/363 [01:27<04:47, 1.14s/it]
Loading 0: 31%|███ | 113/363 [01:29<05:33, 1.33s/it]
Loading 0: 31%|███ | 113/363 [01:29<05:33, 1.33s/it]
Loading 0: 33%|███▎ | 120/363 [01:30<02:13, 1.82it/s]
Loading 0: 33%|███▎ | 120/363 [01:30<02:13, 1.82it/s]
Loading 0: 34%|███▍ | 124/363 [01:33<02:19, 1.71it/s]
Loading 0: 34%|███▍ | 124/363 [01:33<02:19, 1.71it/s]
Loading 0: 34%|███▍ | 125/363 [01:35<02:56, 1.35it/s]
Loading 0: 34%|███▍ | 125/363 [01:35<02:56, 1.35it/s]
Loading 0: 35%|███▍ | 126/363 [01:37<03:41, 1.07it/s]
Loading 0: 35%|███▍ | 126/363 [01:37<03:41, 1.07it/s]
Loading 0: 36%|███▌ | 129/363 [01:39<03:16, 1.19it/s]
Loading 0: 36%|███▌ | 129/363 [01:39<03:16, 1.19it/s]
Loading 0: 36%|███▌ | 130/363 [01:41<03:56, 1.01s/it]
Loading 0: 36%|███▌ | 130/363 [01:41<03:56, 1.01s/it]
Loading 0: 36%|███▌ | 131/363 [01:43<04:41, 1.21s/it]
Loading 0: 36%|███▌ | 131/363 [01:43<04:41, 1.21s/it]
Loading 0: 38%|███▊ | 138/363 [01:44<01:59, 1.89it/s]
Loading 0: 38%|███▊ | 138/363 [01:44<01:59, 1.89it/s]
Loading 0: 39%|███▉ | 142/363 [01:47<02:06, 1.75it/s]
Loading 0: 39%|███▉ | 142/363 [01:47<02:06, 1.75it/s]
Loading 0: 39%|███▉ | 143/363 [01:49<02:40, 1.37it/s]
Loading 0: 39%|███▉ | 143/363 [01:49<02:40, 1.37it/s]
Loading 0: 40%|███▉ | 144/363 [01:51<03:20, 1.09it/s]
Loading 0: 40%|███▉ | 144/363 [01:51<03:20, 1.09it/s]
Loading 0: 40%|████ | 147/363 [01:53<02:59, 1.21it/s]
Loading 0: 40%|████ | 147/363 [01:53<02:59, 1.21it/s]
Loading 0: 41%|████ | 148/363 [01:55<03:36, 1.01s/it]
Loading 0: 41%|████ | 148/363 [01:55<03:36, 1.01s/it]
Loading 0: 41%|████ | 149/363 [01:57<04:17, 1.21s/it]
Loading 0: 41%|████ | 149/363 [01:57<04:17, 1.21s/it]
Loading 0: 43%|████▎ | 156/363 [01:58<01:49, 1.89it/s]
Loading 0: 43%|████▎ | 156/363 [01:58<01:49, 1.89it/s]
Loading 0: 44%|████▍ | 160/363 [02:01<01:55, 1.76it/s]
Loading 0: 44%|████▍ | 160/363 [02:01<01:55, 1.76it/s]
Loading 0: 44%|████▍ | 161/363 [02:03<02:26, 1.38it/s]
Loading 0: 44%|████▍ | 161/363 [02:03<02:26, 1.38it/s]
Loading 0: 45%|████▍ | 162/363 [02:05<03:03, 1.09it/s]
Loading 0: 45%|████▍ | 162/363 [02:05<03:03, 1.09it/s]
Loading 0: 45%|████▌ | 165/363 [02:07<02:44, 1.20it/s]
Loading 0: 45%|████▌ | 165/363 [02:07<02:44, 1.20it/s]
Loading 0: 46%|████▌ | 166/363 [02:09<03:18, 1.01s/it]
Loading 0: 46%|████▌ | 166/363 [02:09<03:18, 1.01s/it]
Loading 0: 46%|████▌ | 167/363 [02:11<03:57, 1.21s/it]
Loading 0: 46%|████▌ | 167/363 [02:11<03:57, 1.21s/it]
Loading 0: 48%|████▊ | 175/363 [02:12<01:34, 1.98it/s]
Loading 0: 48%|████▊ | 175/363 [02:12<01:34, 1.98it/s]
Loading 0: 49%|████▉ | 178/363 [02:15<01:47, 1.73it/s]
Loading 0: 49%|████▉ | 178/363 [02:15<01:47, 1.73it/s]
Loading 0: 49%|████▉ | 179/363 [02:17<02:16, 1.35it/s]
Loading 0: 49%|████▉ | 179/363 [02:17<02:16, 1.35it/s]
Loading 0: 50%|████▉ | 180/363 [02:19<02:50, 1.07it/s]
Loading 0: 50%|████▉ | 180/363 [02:19<02:50, 1.07it/s]
Loading 0: 50%|█████ | 183/363 [02:21<02:31, 1.19it/s]
Loading 0: 50%|█████ | 183/363 [02:21<02:31, 1.19it/s]
Loading 0: 51%|█████ | 184/363 [02:23<03:02, 1.02s/it]
Loading 0: 51%|█████ | 184/363 [02:23<03:02, 1.02s/it]
Loading 0: 51%|█████ | 185/363 [02:25<03:37, 1.22s/it]
Loading 0: 51%|█████ | 185/363 [02:25<03:37, 1.22s/it]
Loading 0: 53%|█████▎ | 192/363 [02:26<01:31, 1.87it/s]
Loading 0: 53%|█████▎ | 192/363 [02:26<01:31, 1.87it/s]
Loading 0: 54%|█████▍ | 196/363 [02:29<01:35, 1.75it/s]
Loading 0: 54%|█████▍ | 196/363 [02:29<01:35, 1.75it/s]
Loading 0: 54%|█████▍ | 197/363 [02:31<02:01, 1.37it/s]
Loading 0: 54%|█████▍ | 197/363 [02:31<02:01, 1.37it/s]
Loading 0: 55%|█████▍ | 198/363 [02:33<02:31, 1.09it/s]
Loading 0: 55%|█████▍ | 198/363 [02:33<02:31, 1.09it/s]
Loading 0: 55%|█████▌ | 201/363 [02:35<02:15, 1.20it/s]
Loading 0: 55%|█████▌ | 201/363 [02:35<02:15, 1.20it/s]
Loading 0: 56%|█████▌ | 202/363 [02:37<02:42, 1.01s/it]
Loading 0: 56%|█████▌ | 202/363 [02:37<02:42, 1.01s/it]
Loading 0: 56%|█████▌ | 203/363 [02:39<03:14, 1.21s/it]
Loading 0: 56%|█████▌ | 203/363 [02:39<03:14, 1.21s/it]
Loading 0: 58%|█████▊ | 211/363 [02:40<01:16, 1.98it/s]
Loading 0: 58%|█████▊ | 211/363 [02:40<01:16, 1.98it/s]
Loading 0: 59%|█████▉ | 214/363 [02:43<01:26, 1.73it/s]
Loading 0: 59%|█████▉ | 214/363 [02:43<01:26, 1.73it/s]
Loading 0: 59%|█████▉ | 215/363 [02:45<01:49, 1.35it/s]
Loading 0: 59%|█████▉ | 215/363 [02:45<01:49, 1.35it/s]
Loading 0: 60%|█████▉ | 216/363 [02:47<02:17, 1.07it/s]
Loading 0: 60%|█████▉ | 216/363 [02:47<02:17, 1.07it/s]
Loading 0: 60%|██████ | 219/363 [02:49<02:01, 1.19it/s]
Loading 0: 60%|██████ | 219/363 [02:49<02:01, 1.19it/s]
Loading 0: 61%|██████ | 220/363 [02:51<02:26, 1.02s/it]
Loading 0: 61%|██████ | 220/363 [02:51<02:26, 1.02s/it]
Loading 0: 61%|██████ | 221/363 [02:53<02:53, 1.22s/it]
Loading 0: 61%|██████ | 221/363 [02:53<02:53, 1.22s/it]
Loading 0: 63%|██████▎ | 229/363 [02:54<01:07, 1.98it/s]
Loading 0: 63%|██████▎ | 229/363 [02:54<01:07, 1.98it/s]
Loading 0: 64%|██████▍ | 232/363 [02:57<01:16, 1.72it/s]
Loading 0: 64%|██████▍ | 232/363 [02:57<01:16, 1.72it/s]
Loading 0: 64%|██████▍ | 233/363 [02:59<01:36, 1.34it/s]
Loading 0: 64%|██████▍ | 233/363 [02:59<01:36, 1.34it/s]
Loading 0: 64%|██████▍ | 234/363 [03:01<02:01, 1.07it/s]
Loading 0: 64%|██████▍ | 234/363 [03:01<02:01, 1.07it/s]
Loading 0: 65%|██████▌ | 237/363 [03:03<01:46, 1.18it/s]
Loading 0: 65%|██████▌ | 237/363 [03:03<01:46, 1.18it/s]
Loading 0: 66%|██████▌ | 238/363 [03:05<02:08, 1.03s/it]
Loading 0: 66%|██████▌ | 238/363 [03:05<02:08, 1.03s/it]
Loading 0: 66%|██████▌ | 239/363 [03:07<02:31, 1.22s/it]
Loading 0: 66%|██████▌ | 239/363 [03:07<02:31, 1.22s/it]
Loading 0: 68%|██████▊ | 247/363 [03:08<00:59, 1.97it/s]
Loading 0: 68%|██████▊ | 247/363 [03:08<00:59, 1.97it/s]
Loading 0: 69%|██████▉ | 250/363 [03:11<01:05, 1.72it/s]
Loading 0: 69%|██████▉ | 250/363 [03:11<01:05, 1.72it/s]
Loading 0: 69%|██████▉ | 251/363 [03:13<01:23, 1.34it/s]
Loading 0: 69%|██████▉ | 251/363 [03:13<01:23, 1.34it/s]
Loading 0: 69%|██████▉ | 252/363 [03:15<01:44, 1.07it/s]
Loading 0: 69%|██████▉ | 252/363 [03:15<01:44, 1.07it/s]
Loading 0: 70%|███████ | 255/363 [03:17<01:31, 1.18it/s]
Loading 0: 70%|███████ | 255/363 [03:17<01:31, 1.18it/s]
Loading 0: 71%|███████ | 256/363 [03:19<01:49, 1.02s/it]
Loading 0: 71%|███████ | 256/363 [03:19<01:49, 1.02s/it]
Loading 0: 71%|███████ | 257/363 [03:21<02:09, 1.22s/it]
Loading 0: 71%|███████ | 257/363 [03:21<02:09, 1.22s/it]
Loading 0: 73%|███████▎ | 264/363 [03:22<00:52, 1.88it/s]
Loading 0: 73%|███████▎ | 264/363 [03:22<00:52, 1.88it/s]
Loading 0: 74%|███████▍ | 268/363 [03:25<00:54, 1.75it/s]
Loading 0: 74%|███████▍ | 268/363 [03:25<00:54, 1.75it/s]
Loading 0: 74%|███████▍ | 269/363 [03:27<01:08, 1.37it/s]
Loading 0: 74%|███████▍ | 269/363 [03:27<01:08, 1.37it/s]
Loading 0: 74%|███████▍ | 270/363 [03:29<01:25, 1.09it/s]
Loading 0: 74%|███████▍ | 270/363 [03:29<01:25, 1.09it/s]
Loading 0: 75%|███████▌ | 273/363 [03:50<04:38, 3.09s/it]
Loading 0: 75%|███████▌ | 273/363 [03:50<04:38, 3.09s/it]
Loading 0: 75%|███████▌ | 274/363 [03:52<04:21, 2.94s/it]
Loading 0: 75%|███████▌ | 274/363 [03:52<04:21, 2.94s/it]
Loading 0: 76%|███████▌ | 275/363 [03:54<04:07, 2.81s/it]
Loading 0: 76%|███████▌ | 275/363 [03:54<04:07, 2.81s/it]
Loading 0: 78%|███████▊ | 283/363 [03:55<01:22, 1.04s/it]
Loading 0: 78%|███████▊ | 283/363 [03:55<01:22, 1.04s/it]
Loading 0: 79%|███████▉ | 286/363 [03:57<01:14, 1.04it/s]
Loading 0: 79%|███████▉ | 286/363 [03:57<01:14, 1.04it/s]
Loading 0: 79%|███████▉ | 287/363 [03:59<01:20, 1.06s/it]
Loading 0: 79%|███████▉ | 287/363 [03:59<01:20, 1.06s/it]
Loading 0: 79%|███████▉ | 288/363 [04:01<01:30, 1.21s/it]
Loading 0: 79%|███████▉ | 288/363 [04:01<01:30, 1.21s/it]
Loading 0: 80%|████████ | 291/363 [04:03<01:13, 1.02s/it]
Loading 0: 80%|████████ | 291/363 [04:03<01:13, 1.02s/it]
Loading 0: 80%|████████ | 292/363 [04:05<01:23, 1.17s/it]
Loading 0: 80%|████████ | 292/363 [04:05<01:23, 1.17s/it]
Loading 0: 81%|████████ | 293/363 [04:08<01:34, 1.34s/it]
Loading 0: 81%|████████ | 293/363 [04:08<01:34, 1.34s/it]
Loading 0: 83%|████████▎ | 301/363 [04:09<00:33, 1.84it/s]
Loading 0: 83%|████████▎ | 301/363 [04:09<00:33, 1.84it/s]
Loading 0: 84%|████████▎ | 304/363 [04:11<00:35, 1.64it/s]
Loading 0: 84%|████████▎ | 304/363 [04:11<00:35, 1.64it/s]
Loading 0: 84%|████████▍ | 305/363 [04:13<00:44, 1.30it/s]
Loading 0: 84%|████████▍ | 305/363 [04:13<00:44, 1.30it/s]
Loading 0: 84%|████████▍ | 306/363 [04:15<00:54, 1.04it/s]
Loading 0: 84%|████████▍ | 306/363 [04:15<00:54, 1.04it/s]
Loading 0: 85%|████████▌ | 309/363 [04:17<00:46, 1.16it/s]
Loading 0: 85%|████████▌ | 309/363 [04:17<00:46, 1.16it/s]
Loading 0: 85%|████████▌ | 310/363 [04:19<00:54, 1.04s/it]
Loading 0: 85%|████████▌ | 310/363 [04:19<00:54, 1.04s/it]
Loading 0: 86%|████████▌ | 311/363 [04:22<01:04, 1.24s/it]
Loading 0: 86%|████████▌ | 311/363 [04:22<01:04, 1.24s/it]
Loading 0: 88%|████████▊ | 319/363 [04:23<00:22, 1.98it/s]
Loading 0: 88%|████████▊ | 319/363 [04:23<00:22, 1.98it/s]
Loading 0: 89%|████████▊ | 322/363 [04:25<00:23, 1.73it/s]
Loading 0: 89%|████████▊ | 322/363 [04:25<00:23, 1.73it/s]
Loading 0: 89%|████████▉ | 323/363 [04:27<00:29, 1.35it/s]
Loading 0: 89%|████████▉ | 323/363 [04:27<00:29, 1.35it/s]
Loading 0: 89%|████████▉ | 324/363 [04:29<00:36, 1.07it/s]
Loading 0: 89%|████████▉ | 324/363 [04:29<00:36, 1.07it/s]
Loading 0: 90%|█████████ | 327/363 [04:31<00:30, 1.18it/s]
Loading 0: 90%|█████████ | 327/363 [04:31<00:30, 1.18it/s]
Loading 0: 90%|█████████ | 328/363 [04:33<00:35, 1.03s/it]
Loading 0: 90%|█████████ | 328/363 [04:33<00:35, 1.03s/it]
Loading 0: 91%|█████████ | 329/363 [04:36<00:41, 1.23s/it]
Loading 0: 91%|█████████ | 329/363 [04:36<00:41, 1.23s/it]
Loading 0: 93%|█████████▎| 337/363 [04:37<00:13, 1.98it/s]
Loading 0: 93%|█████████▎| 337/363 [04:37<00:13, 1.98it/s]
Loading 0: 94%|█████████▎| 340/363 [04:39<00:13, 1.73it/s]
Loading 0: 94%|█████████▎| 340/363 [04:39<00:13, 1.73it/s]
Loading 0: 94%|█████████▍| 341/363 [04:41<00:16, 1.35it/s]
Loading 0: 94%|█████████▍| 341/363 [04:41<00:16, 1.35it/s]
Loading 0: 94%|█████████▍| 342/363 [04:43<00:19, 1.07it/s]
Loading 0: 94%|█████████▍| 342/363 [04:43<00:19, 1.07it/s]
Loading 0: 95%|█████████▌| 345/363 [04:45<00:15, 1.18it/s]
Loading 0: 95%|█████████▌| 345/363 [04:45<00:15, 1.18it/s]
Loading 0: 95%|█████████▌| 346/363 [04:47<00:17, 1.03s/it]
Loading 0: 95%|█████████▌| 346/363 [04:47<00:17, 1.03s/it]
Loading 0: 96%|█████████▌| 347/363 [04:50<00:19, 1.23s/it]
Loading 0: 96%|█████████▌| 347/363 [04:50<00:19, 1.23s/it]
Loading 0: 98%|█████████▊| 355/363 [04:51<00:04, 1.97it/s]
Loading 0: 98%|█████████▊| 355/363 [04:51<00:04, 1.97it/s]
Loading 0: 99%|█████████▉| 359/363 [04:54<00:02, 1.77it/s]
Loading 0: 99%|█████████▉| 359/363 [04:54<00:02, 1.77it/s]
Loading 0: 99%|█████████▉| 360/363 [04:56<00:02, 1.39it/s]
Loading 0: 99%|█████████▉| 360/363 [04:56<00:02, 1.39it/s]
Loading 0: 99%|█████████▉| 361/363 [04:58<00:01, 1.10it/s]
Loading 0: 99%|█████████▉| 361/363 [04:58<00:01, 1.10it/s]
Loading 0: 100%|██████████| 363/363 [04:58<00:00, 1.10it/s]
Loading 0: 100%|██████████| 363/363 [04:58<00:00, 1.22it/s]
chaiml-98p-2ff-chaiml-m-67425-v1-mkmlizer: The tokenizer you are loading from '/tmp/tmpx7vf7yk9' with an incorrect regex pattern: https://huggingface.co/mistralai/Mistral-Small-3.1-24B-Instruct-2503/discussions/84#69121093e8b480e709447d5e. This will lead to incorrect tokenization. You should set the `fix_mistral_regex=True` flag when loading this tokenizer to fix this issue.
chaiml-98p-2ff-chaiml-m-67425-v1-mkmlizer: quantized model in 307.814s
chaiml-98p-2ff-chaiml-m-67425-v1-mkmlizer: Processed model ChaiML/98p_2ff_chaiml_mistral_24b_2048_83002_v1_cp468_prod_rm_merged in 402.951s
chaiml-98p-2ff-chaiml-m-67425-v1-mkmlizer: creating bucket guanaco-mkml-models
chaiml-98p-2ff-chaiml-m-67425-v1-mkmlizer: Bucket 's3://guanaco-mkml-models/' created
chaiml-98p-2ff-chaiml-m-67425-v1-mkmlizer: uploading /dev/shm/model_cache to s3://guanaco-mkml-models/chaiml-98p-2ff-chaiml-m-67425-v1/nvidia
chaiml-98p-2ff-chaiml-m-67425-v1-mkmlizer: cp /dev/shm/model_cache/config.json s3://guanaco-mkml-models/chaiml-98p-2ff-chaiml-m-67425-v1/nvidia/config.json
chaiml-98p-2ff-chaiml-m-67425-v1-mkmlizer: cp /dev/shm/model_cache/special_tokens_map.json s3://guanaco-mkml-models/chaiml-98p-2ff-chaiml-m-67425-v1/nvidia/special_tokens_map.json
chaiml-98p-2ff-chaiml-m-67425-v1-mkmlizer: cp /dev/shm/model_cache/tokenizer_config.json s3://guanaco-mkml-models/chaiml-98p-2ff-chaiml-m-67425-v1/nvidia/tokenizer_config.json
chaiml-98p-2ff-chaiml-m-67425-v1-mkmlizer: cp /dev/shm/model_cache/tokenizer.json s3://guanaco-mkml-models/chaiml-98p-2ff-chaiml-m-67425-v1/nvidia/tokenizer.json
chaiml-mistral-24b-2048-83002-v4-mkmlizer:
Loading 0: 0%| | 0.00/363 [00:00<?, ?it/s]
Loading 0: 1%| | 3.00/363 [00:02<04:26, 1.35it/s]
Loading 0: 1%| | 3.00/363 [00:02<04:26, 1.35it/s]
Loading 0: 1%| | 4.00/363 [00:04<07:10, 1.20s/it]
Loading 0: 1%| | 4.00/363 [00:04<07:10, 1.20s/it]
Loading 0: 1%|▏ | 5.00/363 [00:06<09:12, 1.54s/it]
Loading 0: 1%|▏ | 5.00/363 [00:06<09:12, 1.54s/it]
Loading 0: 3%|▎ | 11.0/363 [00:07<03:13, 1.82it/s]
Loading 0: 3%|▎ | 11.0/363 [00:07<03:13, 1.82it/s]
Loading 0: 4%|▎ | 13.0/363 [00:10<04:11, 1.39it/s]
Loading 0: 4%|▎ | 13.0/363 [00:10<04:11, 1.39it/s]
Loading 0: 4%|▍ | 14.0/363 [00:12<05:28, 1.06it/s]
Loading 0: 4%|▍ | 14.0/363 [00:12<05:28, 1.06it/s]
Loading 0: 4%|▍ | 15.0/363 [00:14<06:53, 1.19s/it]
Loading 0: 4%|▍ | 15.0/363 [00:14<06:53, 1.19s/it]
Loading 0: 6%|▌ | 21.0/363 [00:17<04:11, 1.36it/s]
Loading 0: 6%|▌ | 21.0/363 [00:17<04:11, 1.36it/s]
Loading 0: 6%|▌ | 22.0/363 [00:19<05:12, 1.09it/s]
Loading 0: 6%|▌ | 22.0/363 [00:19<05:12, 1.09it/s]
Loading 0: 6%|▋ | 23.0/363 [00:21<06:22, 1.13s/it]
Loading 0: 6%|▋ | 23.0/363 [00:21<06:22, 1.13s/it]
Loading 0: 8%|▊ | 30.0/363 [00:22<02:59, 1.85it/s]
Loading 0: 8%|▊ | 30.0/363 [00:22<02:59, 1.85it/s]
Loading 0: 9%|▉ | 34.0/363 [00:25<03:14, 1.69it/s]
Loading 0: 9%|▉ | 34.0/363 [00:25<03:14, 1.69it/s]
Loading 0: 10%|▉ | 35.0/363 [00:27<04:08, 1.32it/s]
Loading 0: 10%|▉ | 35.0/363 [00:27<04:08, 1.32it/s]
Loading 0: 10%|▉ | 36.0/363 [00:30<05:14, 1.04it/s]
Loading 0: 10%|▉ | 36.0/363 [00:30<05:14, 1.04it/s]
Loading 0: 11%|█ | 39.0/363 [00:32<04:43, 1.14it/s]
Loading 0: 11%|█ | 39.0/363 [00:32<04:43, 1.14it/s]
Loading 0: 11%|█ | 40.0/363 [00:34<05:43, 1.06s/it]
Loading 0: 11%|█ | 40.0/363 [00:34<05:43, 1.06s/it]
Loading 0: 11%|█▏ | 41.0/363 [00:36<06:51, 1.28s/it]
Loading 0: 11%|█▏ | 41.0/363 [00:36<06:51, 1.28s/it]
Loading 0: 13%|█▎ | 48.0/363 [00:37<02:59, 1.75it/s]
Loading 0: 13%|█▎ | 48.0/363 [00:37<02:59, 1.75it/s]
Loading 0: 14%|█▍ | 52.0/363 [00:40<03:11, 1.62it/s]
Loading 0: 14%|█▍ | 52.0/363 [00:40<03:11, 1.62it/s]
Loading 0: 15%|█▍ | 53.0/363 [00:42<04:05, 1.26it/s]
Loading 0: 15%|█▍ | 53.0/363 [00:42<04:05, 1.26it/s]
Loading 0: 15%|█▍ | 54.0/363 [00:45<05:07, 1.00it/s]
Loading 0: 15%|█▍ | 54.0/363 [00:45<05:07, 1.00it/s]
Loading 0: 16%|█▌ | 57.0/363 [00:47<04:36, 1.11it/s]
Loading 0: 16%|█▌ | 57.0/363 [00:47<04:36, 1.11it/s]
Loading 0: 16%|█▌ | 58.0/363 [00:49<05:34, 1.10s/it]
Loading 0: 16%|█▌ | 58.0/363 [00:49<05:34, 1.10s/it]
Loading 0: 16%|█▋ | 59.0/363 [00:52<06:40, 1.32s/it]
Loading 0: 16%|█▋ | 59.0/363 [00:52<06:40, 1.32s/it]
Loading 0: 18%|█▊ | 66.0/363 [00:53<02:52, 1.72it/s]
Loading 0: 18%|█▊ | 66.0/363 [00:53<02:52, 1.72it/s]
Loading 0: 19%|█▉ | 70.0/363 [00:55<03:02, 1.60it/s]
Loading 0: 19%|█▉ | 70.0/363 [00:55<03:02, 1.60it/s]
Loading 0: 20%|█▉ | 71.0/363 [00:58<03:52, 1.26it/s]
Loading 0: 20%|█▉ | 71.0/363 [00:58<03:52, 1.26it/s]
Loading 0: 20%|█▉ | 72.0/363 [01:00<04:51, 1.00s/it]
Loading 0: 20%|█▉ | 72.0/363 [01:00<04:51, 1.00s/it]
Loading 0: 21%|██ | 75.0/363 [01:02<04:20, 1.10it/s]
Loading 0: 21%|██ | 75.0/363 [01:02<04:20, 1.10it/s]
Loading 0: 21%|██ | 76.0/363 [01:05<05:27, 1.14s/it]
Loading 0: 21%|██ | 76.0/363 [01:05<05:27, 1.14s/it]
Loading 0: 21%|██ | 77.0/363 [01:07<06:26, 1.35s/it]
Loading 0: 21%|██ | 77.0/363 [01:07<06:26, 1.35s/it]
Loading 0: 23%|██▎ | 84.0/363 [01:08<02:47, 1.67it/s]
Loading 0: 23%|██▎ | 84.0/363 [01:08<02:47, 1.67it/s]
Loading 0: 24%|██▍ | 88.0/363 [01:11<02:55, 1.57it/s]
Loading 0: 24%|██▍ | 88.0/363 [01:11<02:55, 1.57it/s]
Loading 0: 25%|██▍ | 89.0/363 [01:13<03:41, 1.24it/s]
Loading 0: 25%|██▍ | 89.0/363 [01:13<03:41, 1.24it/s]
Loading 0: 25%|██▍ | 90.0/363 [01:16<04:35, 1.01s/it]
Loading 0: 25%|██▍ | 90.0/363 [01:16<04:35, 1.01s/it]
Loading 0: 27%|██▋ | 97.0/363 [01:17<02:18, 1.93it/s]
Loading 0: 27%|██▋ | 97.0/363 [01:17<02:18, 1.93it/s]
Loading 0: 28%|██▊ | 101/363 [01:19<02:27, 1.77it/s]
Loading 0: 28%|██▊ | 101/363 [01:19<02:27, 1.77it/s]
Loading 0: 28%|██▊ | 102/363 [01:22<03:10, 1.37it/s]
Loading 0: 28%|██▊ | 102/363 [01:22<03:10, 1.37it/s]
Loading 0: 28%|██▊ | 103/363 [01:24<04:02, 1.07it/s]
Loading 0: 28%|██▊ | 103/363 [01:24<04:02, 1.07it/s]
Loading 0: 29%|██▉ | 106/363 [01:26<03:47, 1.13it/s]
Loading 0: 29%|██▉ | 106/363 [01:26<03:47, 1.13it/s]
Loading 0: 29%|██▉ | 107/363 [01:28<04:34, 1.07s/it]
Loading 0: 29%|██▉ | 107/363 [01:28<04:34, 1.07s/it]
Loading 0: 30%|██▉ | 108/363 [01:31<05:27, 1.28s/it]
Loading 0: 30%|██▉ | 108/363 [01:31<05:27, 1.28s/it]
Loading 0: 31%|███ | 111/363 [01:33<04:25, 1.05s/it]
Loading 0: 31%|███ | 111/363 [01:33<04:25, 1.05s/it]
Loading 0: 31%|███ | 112/363 [01:35<05:11, 1.24s/it]
Loading 0: 31%|███ | 112/363 [01:35<05:11, 1.24s/it]
Loading 0: 31%|███ | 113/363 [01:37<06:02, 1.45s/it]
Loading 0: 31%|███ | 113/363 [01:37<06:02, 1.45s/it]
Loading 0: 33%|███▎ | 120/363 [01:39<02:25, 1.67it/s]
Loading 0: 33%|███▎ | 120/363 [01:39<02:25, 1.67it/s]
Loading 0: 34%|███▍ | 124/363 [01:41<02:31, 1.58it/s]
Loading 0: 34%|███▍ | 124/363 [01:41<02:31, 1.58it/s]
Loading 0: 34%|███▍ | 125/363 [01:44<03:12, 1.24it/s]
Loading 0: 34%|███▍ | 125/363 [01:44<03:12, 1.24it/s]
Loading 0: 35%|███▍ | 126/363 [01:46<04:01, 1.02s/it]
Loading 0: 35%|███▍ | 126/363 [01:46<04:01, 1.02s/it]
Loading 0: 36%|███▌ | 129/363 [01:48<03:34, 1.09it/s]
Loading 0: 36%|███▌ | 129/363 [01:48<03:34, 1.09it/s]
Loading 0: 36%|███▌ | 130/363 [01:50<04:17, 1.11s/it]
Loading 0: 36%|███▌ | 130/363 [01:50<04:17, 1.11s/it]
Loading 0: 36%|███▌ | 131/363 [01:53<05:05, 1.32s/it]
Loading 0: 36%|███▌ | 131/363 [01:53<05:05, 1.32s/it]
Loading 0: 38%|███▊ | 138/363 [01:54<02:10, 1.72it/s]
Loading 0: 38%|███▊ | 138/363 [01:54<02:10, 1.72it/s]
Loading 0: 39%|███▉ | 142/363 [01:57<02:17, 1.60it/s]
Loading 0: 39%|███▉ | 142/363 [01:57<02:17, 1.60it/s]
Loading 0: 39%|███▉ | 143/363 [01:59<02:54, 1.26it/s]
Loading 0: 39%|███▉ | 143/363 [01:59<02:54, 1.26it/s]
Loading 0: 40%|███▉ | 144/363 [02:01<03:38, 1.00it/s]
Loading 0: 40%|███▉ | 144/363 [02:01<03:38, 1.00it/s]
Loading 0: 40%|████ | 147/363 [02:04<03:22, 1.07it/s]
Loading 0: 40%|████ | 147/363 [02:04<03:22, 1.07it/s]
Loading 0: 41%|████ | 148/363 [02:06<04:01, 1.12s/it]
Loading 0: 41%|████ | 148/363 [02:06<04:01, 1.12s/it]
Loading 0: 41%|████ | 149/363 [02:08<04:44, 1.33s/it]
Loading 0: 41%|████ | 149/363 [02:08<04:44, 1.33s/it]
Loading 0: 43%|████▎ | 156/363 [02:09<02:01, 1.70it/s]
Loading 0: 43%|████▎ | 156/363 [02:09<02:01, 1.70it/s]
Loading 0: 44%|████▍ | 160/363 [02:12<02:07, 1.59it/s]
Loading 0: 44%|████▍ | 160/363 [02:12<02:07, 1.59it/s]
Loading 0: 44%|████▍ | 161/363 [02:14<02:42, 1.25it/s]
Loading 0: 44%|████▍ | 161/363 [02:14<02:42, 1.25it/s]
Loading 0: 45%|████▍ | 162/363 [02:17<03:21, 1.00s/it]
Loading 0: 45%|████▍ | 162/363 [02:17<03:21, 1.00s/it]
Loading 0: 45%|████▌ | 165/363 [02:19<03:01, 1.09it/s]
Loading 0: 45%|████▌ | 165/363 [02:19<03:01, 1.09it/s]
Loading 0: 46%|████▌ | 166/363 [02:21<03:37, 1.10s/it]
Loading 0: 46%|████▌ | 166/363 [02:21<03:37, 1.10s/it]
Loading 0: 46%|████▌ | 167/363 [02:23<04:17, 1.32s/it]
Loading 0: 46%|████▌ | 167/363 [02:23<04:17, 1.32s/it]
Loading 0: 48%|████▊ | 174/363 [02:24<01:49, 1.72it/s]
Loading 0: 48%|████▊ | 174/363 [02:24<01:49, 1.72it/s]
Loading 0: 49%|████▉ | 178/363 [02:27<01:55, 1.61it/s]
Loading 0: 49%|████▉ | 178/363 [02:27<01:55, 1.61it/s]
Loading 0: 49%|████▉ | 179/363 [02:29<02:25, 1.26it/s]
Loading 0: 49%|████▉ | 179/363 [02:29<02:25, 1.26it/s]
Loading 0: 50%|████▉ | 180/363 [02:32<03:02, 1.00it/s]
Loading 0: 50%|████▉ | 180/363 [02:32<03:02, 1.00it/s]
Loading 0: 50%|█████ | 183/363 [02:34<02:46, 1.08it/s]
Loading 0: 50%|█████ | 183/363 [02:34<02:46, 1.08it/s]
Loading 0: 51%|█████ | 184/363 [02:36<03:18, 1.11s/it]
Loading 0: 51%|█████ | 184/363 [02:36<03:18, 1.11s/it]
Loading 0: 51%|█████ | 185/363 [02:39<03:55, 1.32s/it]
Loading 0: 51%|█████ | 185/363 [02:39<03:55, 1.32s/it]
Loading 0: 53%|█████▎ | 192/363 [02:40<01:39, 1.71it/s]
Loading 0: 53%|█████▎ | 192/363 [02:40<01:39, 1.71it/s]
Loading 0: 54%|█████▍ | 196/363 [02:43<01:44, 1.60it/s]
Loading 0: 54%|█████▍ | 196/363 [02:43<01:44, 1.60it/s]
Loading 0: 54%|█████▍ | 197/363 [02:45<02:12, 1.26it/s]
Loading 0: 54%|█████▍ | 197/363 [02:45<02:12, 1.26it/s]
Loading 0: 55%|█████▍ | 198/363 [02:47<02:44, 1.00it/s]
Loading 0: 55%|█████▍ | 198/363 [02:47<02:44, 1.00it/s]
Loading 0: 55%|█████▌ | 201/363 [02:49<02:25, 1.11it/s]
Loading 0: 55%|█████▌ | 201/363 [02:49<02:25, 1.11it/s]
Loading 0: 56%|█████▌ | 202/363 [02:51<02:55, 1.09s/it]
Loading 0: 56%|█████▌ | 202/363 [02:51<02:55, 1.09s/it]
Loading 0: 56%|█████▌ | 203/363 [02:54<03:28, 1.31s/it]
Loading 0: 56%|█████▌ | 203/363 [02:54<03:28, 1.31s/it]
Loading 0: 58%|█████▊ | 210/363 [02:55<01:27, 1.74it/s]
Loading 0: 58%|█████▊ | 210/363 [02:55<01:27, 1.74it/s]
Loading 0: 59%|█████▉ | 214/363 [02:58<01:31, 1.62it/s]
Loading 0: 59%|█████▉ | 214/363 [02:58<01:31, 1.62it/s]
Loading 0: 59%|█████▉ | 215/363 [03:00<01:56, 1.27it/s]
Loading 0: 59%|█████▉ | 215/363 [03:00<01:56, 1.27it/s]
Loading 0: 60%|█████▉ | 216/363 [03:02<02:25, 1.01it/s]
Loading 0: 60%|█████▉ | 216/363 [03:02<02:25, 1.01it/s]
Loading 0: 60%|██████ | 219/363 [03:04<02:09, 1.12it/s]
Loading 0: 60%|██████ | 219/363 [03:04<02:09, 1.12it/s]
Loading 0: 61%|██████ | 220/363 [03:07<02:35, 1.09s/it]
Loading 0: 61%|██████ | 220/363 [03:07<02:35, 1.09s/it]
Loading 0: 61%|██████ | 221/363 [03:09<03:04, 1.30s/it]
Loading 0: 61%|██████ | 221/363 [03:09<03:04, 1.30s/it]
Loading 0: 63%|██████▎ | 228/363 [03:10<01:17, 1.74it/s]
Loading 0: 63%|██████▎ | 228/363 [03:10<01:17, 1.74it/s]
Loading 0: 64%|██████▍ | 232/363 [03:13<01:21, 1.62it/s]
Loading 0: 64%|██████▍ | 232/363 [03:13<01:21, 1.62it/s]
Loading 0: 64%|██████▍ | 233/363 [03:15<01:42, 1.27it/s]
Loading 0: 64%|██████▍ | 233/363 [03:15<01:42, 1.27it/s]
Loading 0: 64%|██████▍ | 234/363 [03:17<02:07, 1.01it/s]
Loading 0: 64%|██████▍ | 234/363 [03:17<02:07, 1.01it/s]
Loading 0: 65%|██████▌ | 237/363 [03:19<01:53, 1.11it/s]
Loading 0: 65%|██████▌ | 237/363 [03:19<01:53, 1.11it/s]
Loading 0: 66%|██████▌ | 238/363 [03:22<02:15, 1.09s/it]
Loading 0: 66%|██████▌ | 238/363 [03:22<02:15, 1.09s/it]
Loading 0: 66%|██████▌ | 239/363 [03:24<02:41, 1.30s/it]
Loading 0: 66%|██████▌ | 239/363 [03:24<02:41, 1.30s/it]
Loading 0: 68%|██████▊ | 246/363 [03:25<01:07, 1.74it/s]
Loading 0: 68%|██████▊ | 246/363 [03:25<01:07, 1.74it/s]
Loading 0: 69%|██████▉ | 250/363 [03:28<01:09, 1.62it/s]
Loading 0: 69%|██████▉ | 250/363 [03:28<01:09, 1.62it/s]
Loading 0: 69%|██████▉ | 251/363 [03:30<01:28, 1.27it/s]
Loading 0: 69%|██████▉ | 251/363 [03:30<01:28, 1.27it/s]
Loading 0: 69%|██████▉ | 252/363 [03:32<01:49, 1.01it/s]
Loading 0: 69%|██████▉ | 252/363 [03:32<01:49, 1.01it/s]
Loading 0: 70%|███████ | 255/363 [03:35<01:36, 1.11it/s]
Loading 0: 70%|███████ | 255/363 [03:35<01:36, 1.11it/s]
Loading 0: 71%|███████ | 256/363 [03:37<01:56, 1.09s/it]
Loading 0: 71%|███████ | 256/363 [03:37<01:56, 1.09s/it]
Loading 0: 71%|███████ | 257/363 [03:39<02:17, 1.30s/it]
Loading 0: 71%|███████ | 257/363 [03:39<02:17, 1.30s/it]
Loading 0: 73%|███████▎ | 264/363 [03:40<00:56, 1.75it/s]
Loading 0: 73%|███████▎ | 264/363 [03:40<00:56, 1.75it/s]
Loading 0: 74%|███████▍ | 268/363 [03:43<00:58, 1.63it/s]
Loading 0: 74%|███████▍ | 268/363 [03:43<00:58, 1.63it/s]
Loading 0: 74%|███████▍ | 269/363 [03:45<01:13, 1.27it/s]
Loading 0: 74%|███████▍ | 269/363 [03:45<01:13, 1.27it/s]
Loading 0: 74%|███████▍ | 270/363 [03:47<01:32, 1.01it/s]
Loading 0: 74%|███████▍ | 270/363 [03:47<01:32, 1.01it/s]
Loading 0: 75%|███████▌ | 273/363 [04:04<03:56, 2.63s/it]
Loading 0: 75%|███████▌ | 273/363 [04:04<03:56, 2.63s/it]
Loading 0: 75%|███████▌ | 274/363 [04:06<03:47, 2.56s/it]
Loading 0: 75%|███████▌ | 274/363 [04:06<03:47, 2.56s/it]
Loading 0: 76%|███████▌ | 275/363 [04:08<03:41, 2.52s/it]
Loading 0: 76%|███████▌ | 275/363 [04:08<03:41, 2.52s/it]
Loading 0: 78%|███████▊ | 282/363 [04:09<01:22, 1.02s/it]
Loading 0: 78%|███████▊ | 282/363 [04:09<01:22, 1.02s/it]
Loading 0: 79%|███████▉ | 286/363 [04:12<01:09, 1.11it/s]
Loading 0: 79%|███████▉ | 286/363 [04:12<01:09, 1.11it/s]
Loading 0: 79%|███████▉ | 287/363 [04:14<01:17, 1.02s/it]
Loading 0: 79%|███████▉ | 287/363 [04:14<01:17, 1.02s/it]
Loading 0: 79%|███████▉ | 288/363 [04:16<01:29, 1.19s/it]
Loading 0: 79%|███████▉ | 288/363 [04:16<01:29, 1.19s/it]
Loading 0: 80%|████████ | 291/363 [04:19<01:13, 1.03s/it]
Loading 0: 80%|████████ | 291/363 [04:19<01:13, 1.03s/it]
Loading 0: 80%|████████ | 292/363 [04:21<01:25, 1.20s/it]
Loading 0: 80%|████████ | 292/363 [04:21<01:25, 1.20s/it]
Loading 0: 81%|████████ | 293/363 [04:23<01:37, 1.40s/it]
Loading 0: 81%|████████ | 293/363 [04:23<01:37, 1.40s/it]
Loading 0: 83%|████████▎ | 300/363 [04:24<00:38, 1.65it/s]
Loading 0: 83%|████████▎ | 300/363 [04:24<00:38, 1.65it/s]
Loading 0: 84%|████████▎ | 304/363 [04:27<00:37, 1.57it/s]
Loading 0: 84%|████████▎ | 304/363 [04:27<00:37, 1.57it/s]
Loading 0: 84%|████████▍ | 305/363 [04:29<00:46, 1.24it/s]
Loading 0: 84%|████████▍ | 305/363 [04:29<00:46, 1.24it/s]
Loading 0: 84%|████████▍ | 306/363 [04:31<00:57, 1.01s/it]
Loading 0: 84%|████████▍ | 306/363 [04:31<00:57, 1.01s/it]
Loading 0: 85%|████████▌ | 309/363 [04:34<00:49, 1.10it/s]
Loading 0: 85%|████████▌ | 309/363 [04:34<00:49, 1.10it/s]
Loading 0: 85%|████████▌ | 310/363 [04:36<00:58, 1.10s/it]
Loading 0: 85%|████████▌ | 310/363 [04:36<00:58, 1.10s/it]
Loading 0: 86%|████████▌ | 311/363 [04:38<01:08, 1.31s/it]
Loading 0: 86%|████████▌ | 311/363 [04:38<01:08, 1.31s/it]
Loading 0: 88%|████████▊ | 318/363 [04:39<00:25, 1.74it/s]
Loading 0: 88%|████████▊ | 318/363 [04:39<00:25, 1.74it/s]
Loading 0: 89%|████████▊ | 322/363 [04:42<00:25, 1.62it/s]
Loading 0: 89%|████████▊ | 322/363 [04:42<00:25, 1.62it/s]
Loading 0: 89%|████████▉ | 323/363 [04:44<00:31, 1.27it/s]
Loading 0: 89%|████████▉ | 323/363 [04:44<00:31, 1.27it/s]
Loading 0: 89%|████████▉ | 324/363 [04:47<00:38, 1.01it/s]
Loading 0: 89%|████████▉ | 324/363 [04:47<00:38, 1.01it/s]
Loading 0: 90%|█████████ | 327/363 [04:49<00:32, 1.11it/s]
Loading 0: 90%|█████████ | 327/363 [04:49<00:32, 1.11it/s]
Loading 0: 90%|█████████ | 328/363 [04:51<00:38, 1.09s/it]
Loading 0: 90%|█████████ | 328/363 [04:51<00:38, 1.09s/it]
Loading 0: 91%|█████████ | 329/363 [04:53<00:44, 1.30s/it]
Loading 0: 91%|█████████ | 329/363 [04:53<00:44, 1.30s/it]
Loading 0: 93%|█████████▎| 336/363 [04:54<00:15, 1.75it/s]
Loading 0: 93%|█████████▎| 336/363 [04:54<00:15, 1.75it/s]
Loading 0: 94%|█████████▎| 340/363 [04:57<00:14, 1.62it/s]
Loading 0: 94%|█████████▎| 340/363 [04:57<00:14, 1.62it/s]
Loading 0: 94%|█████████▍| 341/363 [04:59<00:17, 1.27it/s]
Loading 0: 94%|█████████▍| 341/363 [04:59<00:17, 1.27it/s]
Loading 0: 94%|█████████▍| 342/363 [05:02<00:20, 1.01it/s]
Loading 0: 94%|█████████▍| 342/363 [05:02<00:20, 1.01it/s]
Loading 0: 95%|█████████▌| 345/363 [05:04<00:16, 1.11it/s]
Loading 0: 95%|█████████▌| 345/363 [05:04<00:16, 1.11it/s]
Loading 0: 95%|█████████▌| 346/363 [05:06<00:18, 1.09s/it]
Loading 0: 95%|█████████▌| 346/363 [05:06<00:18, 1.09s/it]
Loading 0: 96%|█████████▌| 347/363 [05:08<00:20, 1.30s/it]
Loading 0: 96%|█████████▌| 347/363 [05:08<00:20, 1.30s/it]
Loading 0: 98%|█████████▊| 354/363 [05:09<00:05, 1.74it/s]
Loading 0: 98%|█████████▊| 354/363 [05:09<00:05, 1.74it/s]
Loading 0: 99%|█████████▉| 359/363 [05:13<00:02, 1.66it/s]
Loading 0: 99%|█████████▉| 359/363 [05:13<00:02, 1.66it/s]
Loading 0: 99%|█████████▉| 360/363 [05:15<00:02, 1.31it/s]
Loading 0: 99%|█████████▉| 360/363 [05:15<00:02, 1.31it/s]
Loading 0: 99%|█████████▉| 361/363 [05:17<00:01, 1.05it/s]
Loading 0: 99%|█████████▉| 361/363 [05:17<00:01, 1.05it/s]
Loading 0: 100%|██████████| 363/363 [05:17<00:00, 1.40it/s]
Loading 0: 100%|██████████| 363/363 [05:17<00:00, 1.40it/s]
Loading 0: 100%|██████████| 363/363 [05:17<00:00, 1.14it/s]
chaiml-mistral-24b-2048-83002-v4-mkmlizer: The tokenizer you are loading from '/tmp/tmpilq4q5kg' with an incorrect regex pattern: https://huggingface.co/mistralai/Mistral-Small-3.1-24B-Instruct-2503/discussions/84#69121093e8b480e709447d5e. This will lead to incorrect tokenization. You should set the `fix_mistral_regex=True` flag when loading this tokenizer to fix this issue.
chaiml-mistral-24b-2048-83002-v4-mkmlizer: quantized model in 325.019s
chaiml-mistral-24b-2048-83002-v4-mkmlizer: Processed model ChaiML/mistral_24b_2048_gemini_opus_ds_v10_843_merged in 417.703s
chaiml-mistral-24b-2048-83002-v4-mkmlizer: creating bucket guanaco-mkml-models
chaiml-mistral-24b-2048-83002-v4-mkmlizer: Bucket 's3://guanaco-mkml-models/' created
chaiml-mistral-24b-2048-83002-v4-mkmlizer: uploading /dev/shm/model_cache to s3://guanaco-mkml-models/chaiml-mistral-24b-2048-83002-v4/nvidia
chaiml-mistral-24b-2048-83002-v4-mkmlizer: cp /dev/shm/model_cache/config.json s3://guanaco-mkml-models/chaiml-mistral-24b-2048-83002-v4/nvidia/config.json
chaiml-mistral-24b-2048-83002-v4-mkmlizer: cp /dev/shm/model_cache/special_tokens_map.json s3://guanaco-mkml-models/chaiml-mistral-24b-2048-83002-v4/nvidia/special_tokens_map.json
chaiml-mistral-24b-2048-83002-v4-mkmlizer: cp /dev/shm/model_cache/tokenizer_config.json s3://guanaco-mkml-models/chaiml-mistral-24b-2048-83002-v4/nvidia/tokenizer_config.json
chaiml-mistral-24b-2048-83002-v4-mkmlizer: cp /dev/shm/model_cache/tokenizer.json s3://guanaco-mkml-models/chaiml-mistral-24b-2048-83002-v4/nvidia/tokenizer.json
chaiml-98p-2ff-chaiml-m-39704-v1-mkmlizer: cp /dev/shm/model_cache/flywheel_model.1.safetensors s3://guanaco-mkml-models/chaiml-98p-2ff-chaiml-m-39704-v1/nvidia/flywheel_model.1.safetensors
chaiml-98p-2ff-chaiml-m-67425-v1-mkmlizer: cp /dev/shm/model_cache/flywheel_model.1.safetensors s3://guanaco-mkml-models/chaiml-98p-2ff-chaiml-m-67425-v1/nvidia/flywheel_model.1.safetensors
Job chaiml-mistral-24b-2048-83002-v4-mkmlizer completed after 707.54s with status: failed
Job failed chaiml-mistral-24b-2048-83002-v4-mkmlizer:
Stopping job with name chaiml-mistral-24b-2048-83002-v4-mkmlizer
%s, retrying in %s seconds...
Starting job with name chaiml-mistral-24b-2048-83002-v4-mkmlizer
Waiting for job on chaiml-mistral-24b-2048-83002-v4-mkmlizer to finish
chaiml-98p-2ff-chaiml-m-39704-v1-mkmlizer: cp /dev/shm/model_cache/flywheel_model.0.safetensors s3://guanaco-mkml-models/chaiml-98p-2ff-chaiml-m-39704-v1/nvidia/flywheel_model.0.safetensors
Job chaiml-98p-2ff-chaiml-m-39704-v1-mkmlizer completed after 716.28s with status: succeeded
Stopping job with name chaiml-98p-2ff-chaiml-m-39704-v1-mkmlizer
Pipeline stage MKMLizer completed in 718.55s
run pipeline stage %s
Running pipeline stage MKMLTemplater
Pipeline stage MKMLTemplater completed in 0.61s
run pipeline stage %s
Running pipeline stage MKMLDeployer
Creating inference service chaiml-98p-2ff-chaiml-m-39704-v1
Waiting for inference service chaiml-98p-2ff-chaiml-m-39704-v1 to be ready
chaiml-mistral-24b-2048-83002-v4-mkmlizer: bash: cannot set terminal process group (-1): Inappropriate ioctl for device
chaiml-mistral-24b-2048-83002-v4-mkmlizer: bash: no job control in this shell
chaiml-98p-2ff-chaiml-m-67425-v1-mkmlizer: cp /dev/shm/model_cache/flywheel_model.0.safetensors s3://guanaco-mkml-models/chaiml-98p-2ff-chaiml-m-67425-v1/nvidia/flywheel_model.0.safetensors
chaiml-mistral-24b-2048-83002-v4-mkmlizer: /root/miniconda3/envs/nvidia/lib/python3.11/site-packages/mk1/__init__.py:1: UserWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html. The pkg_resources package is slated for removal as early as 2025-11-30. Refrain from using this package or pin to Setuptools<81.
chaiml-mistral-24b-2048-83002-v4-mkmlizer: __import__('pkg_resources').declare_namespace(__name__)
chaiml-mistral-24b-2048-83002-v4-mkmlizer: ╔═════════════════════════════════════════════════════════════════════╗
chaiml-mistral-24b-2048-83002-v4-mkmlizer: ║ ║
chaiml-mistral-24b-2048-83002-v4-mkmlizer: ║ ██████ ██████ █████ ████ ████ ║
chaiml-mistral-24b-2048-83002-v4-mkmlizer: ║ ░░██████ ██████ ░░███ ███░ ░░███ ║
chaiml-mistral-24b-2048-83002-v4-mkmlizer: ║ ░███░█████░███ ░███ ███ ░███ ║
chaiml-mistral-24b-2048-83002-v4-mkmlizer: ║ ░███░░███ ░███ ░███████ ░███ ║
chaiml-mistral-24b-2048-83002-v4-mkmlizer: ║ ░███ ░░░ ░███ ░███░░███ ░███ ║
chaiml-mistral-24b-2048-83002-v4-mkmlizer: ║ ░███ ░███ ░███ ░░███ ░███ ║
chaiml-mistral-24b-2048-83002-v4-mkmlizer: ║ █████ █████ █████ ░░████ █████ ║
chaiml-mistral-24b-2048-83002-v4-mkmlizer: ║ ░░░░░ ░░░░░ ░░░░░ ░░░░ ░░░░░ ║
chaiml-mistral-24b-2048-83002-v4-mkmlizer: ║ ║
chaiml-mistral-24b-2048-83002-v4-mkmlizer: ║ Version: 0.30.6+torch280 ║
chaiml-mistral-24b-2048-83002-v4-mkmlizer: ║ Features: FLYWHEEL, CUDA ║
chaiml-mistral-24b-2048-83002-v4-mkmlizer: ║ Copyright 2023-2025 MK ONE TECHNOLOGIES Inc. ║
chaiml-mistral-24b-2048-83002-v4-mkmlizer: ║ https://mk1.ai ║
chaiml-mistral-24b-2048-83002-v4-mkmlizer: ║ ║
chaiml-mistral-24b-2048-83002-v4-mkmlizer: ║ The license key for the current software has been verified as ║
chaiml-mistral-24b-2048-83002-v4-mkmlizer: ║ belonging to: ║
chaiml-mistral-24b-2048-83002-v4-mkmlizer: ║ ║
chaiml-mistral-24b-2048-83002-v4-mkmlizer: ║ Chai Research Corp. ║
chaiml-mistral-24b-2048-83002-v4-mkmlizer: ║ Account ID: 7997a29f-0ceb-4cc7-9adf-840c57b4ae6f ║
chaiml-mistral-24b-2048-83002-v4-mkmlizer: ║ Expiration: 2028-03-31 23:59:59 ║
Job chaiml-98p-2ff-chaiml-m-67425-v1-mkmlizer completed after 766.26s with status: succeeded
chaiml-mistral-24b-2048-83002-v4-mkmlizer: ║ ║
Stopping job with name chaiml-98p-2ff-chaiml-m-67425-v1-mkmlizer
chaiml-mistral-24b-2048-83002-v4-mkmlizer: ╚═════════════════════════════════════════════════════════════════════╝
Pipeline stage MKMLizer completed in 767.80s
run pipeline stage %s
Running pipeline stage MKMLTemplater
Pipeline stage MKMLTemplater completed in 0.54s
run pipeline stage %s
Running pipeline stage MKMLDeployer
Creating inference service chaiml-98p-2ff-chaiml-m-67425-v1
Waiting for inference service chaiml-98p-2ff-chaiml-m-67425-v1 to be ready
chaiml-mistral-24b-2048-83002-v4-mkmlizer: Downloaded to shared memory in 87.727s
chaiml-mistral-24b-2048-83002-v4-mkmlizer: Checking if ChaiML/mistral_24b_2048_gemini_opus_ds_v10_843_merged already exists in ChaiML
chaiml-mistral-24b-2048-83002-v4-mkmlizer: quantizing model to /dev/shm/model_cache, profile:q4, folder:/tmp/tmpmye32o0r, device:0
chaiml-mistral-24b-2048-83002-v4-mkmlizer: Saving flywheel model at /dev/shm/model_cache
Inference service chaiml-98p-2ff-chaiml-m-39704-v1 ready after 162.30749988555908s
Pipeline stage MKMLDeployer completed in 164.34s
run pipeline stage %s
Running pipeline stage StressChecker
Received healthy response to inference request in 3.9097981452941895s
Received healthy response to inference request in 4.020081281661987s
Received healthy response to inference request in 3.2096967697143555s
Received healthy response to inference request in 3.367011070251465s
Received healthy response to inference request in 3.155485153198242s
5 requests
0 failed requests
5th percentile: 3.166327476501465
10th percentile: 3.1771697998046875
20th percentile: 3.198854446411133
30th percentile: 3.2411596298217775
40th percentile: 3.304085350036621
50th percentile: 3.367011070251465
60th percentile: 3.5841259002685546
70th percentile: 3.8012407302856444
80th percentile: 3.931854772567749
90th percentile: 3.975968027114868
95th percentile: 3.998024654388428
99th percentile: 4.015669956207275
mean time: 3.5324144840240477
Pipeline stage StressChecker completed in 27.96s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyTriggerPipeline
run_pipeline:run_in_cloud %s
starting trigger_guanaco_pipeline args=%s
triggered trigger_guanaco_pipeline args=%s
Pipeline stage OfflineFamilyFriendlyTriggerPipeline completed in 2.24s
run pipeline stage %s
Running pipeline stage TriggerMKMLProfilingPipeline
run_pipeline:run_in_cloud %s
starting trigger_guanaco_pipeline args=%s
triggered trigger_guanaco_pipeline args=%s
Pipeline stage TriggerMKMLProfilingPipeline completed in 2.29s
Shutdown handler de-registered
chaiml-98p-2ff-chaiml-m_39704_v1 status is now deployed due to DeploymentManager action
Inference service chaiml-98p-2ff-chaiml-m-67425-v1 ready after 162.16064596176147s
Pipeline stage MKMLDeployer completed in 164.02s
run pipeline stage %s
Running pipeline stage StressChecker
Received healthy response to inference request in 3.1867356300354004s
Received healthy response to inference request in 3.401787757873535s
Received healthy response to inference request in 3.3211402893066406s
Received healthy response to inference request in 3.024822950363159s
Received healthy response to inference request in 3.4616172313690186s
5 requests
0 failed requests
5th percentile: 3.0572054862976072
10th percentile: 3.0895880222320558
20th percentile: 3.1543530941009523
30th percentile: 3.2136165618896486
40th percentile: 3.2673784255981446
50th percentile: 3.3211402893066406
60th percentile: 3.3533992767333984
70th percentile: 3.3856582641601562
80th percentile: 3.4137536525726317
90th percentile: 3.4376854419708254
95th percentile: 3.449651336669922
99th percentile: 3.459224052429199
mean time: 3.279220771789551
Pipeline stage StressChecker completed in 22.39s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyTriggerPipeline
run_pipeline:run_in_cloud %s
starting trigger_guanaco_pipeline args=%s
triggered trigger_guanaco_pipeline args=%s
Pipeline stage OfflineFamilyFriendlyTriggerPipeline completed in 1.69s
run pipeline stage %s
Running pipeline stage TriggerMKMLProfilingPipeline
run_pipeline:run_in_cloud %s
starting trigger_guanaco_pipeline args=%s
triggered trigger_guanaco_pipeline args=%s
Pipeline stage TriggerMKMLProfilingPipeline completed in 1.83s
Shutdown handler de-registered
chaiml-98p-2ff-chaiml-m_67425_v1 status is now deployed due to DeploymentManager action
chaiml-mistral-24b-2048-83002-v4-mkmlizer:
Loading 0: 0%| | 0.00/363 [00:00<?, ?it/s]
Loading 0: 1%| | 3.00/363 [00:01<03:54, 1.54it/s]
Loading 0: 1%| | 3.00/363 [00:01<03:54, 1.54it/s]
Loading 0: 1%| | 4.00/363 [00:03<06:19, 1.06s/it]
Loading 0: 1%| | 4.00/363 [00:03<06:19, 1.06s/it]
Loading 0: 1%|▏ | 5.00/363 [00:05<08:07, 1.36s/it]
Loading 0: 1%|▏ | 5.00/363 [00:05<08:07, 1.36s/it]
Loading 0: 3%|▎ | 12.0/363 [00:07<02:39, 2.20it/s]
Loading 0: 3%|▎ | 12.0/363 [00:07<02:39, 2.20it/s]
Loading 0: 4%|▎ | 13.0/363 [00:08<03:48, 1.53it/s]
Loading 0: 4%|▎ | 13.0/363 [00:08<03:48, 1.53it/s]
Loading 0: 4%|▍ | 14.0/363 [00:10<04:59, 1.16it/s]
Loading 0: 4%|▍ | 14.0/363 [00:10<04:59, 1.16it/s]
Loading 0: 4%|▍ | 15.0/363 [00:12<06:16, 1.08s/it]
Loading 0: 4%|▍ | 15.0/363 [00:12<06:16, 1.08s/it]
Loading 0: 6%|▌ | 21.0/363 [00:15<03:46, 1.51it/s]
Loading 0: 6%|▌ | 21.0/363 [00:15<03:46, 1.51it/s]
Loading 0: 6%|▌ | 22.0/363 [00:17<04:40, 1.21it/s]
Loading 0: 6%|▌ | 22.0/363 [00:17<04:40, 1.21it/s]
Loading 0: 6%|▋ | 23.0/363 [00:19<05:44, 1.01s/it]
Loading 0: 6%|▋ | 23.0/363 [00:19<05:44, 1.01s/it]
Loading 0: 9%|▊ | 31.0/363 [00:20<02:34, 2.15it/s]
Loading 0: 9%|▊ | 31.0/363 [00:20<02:34, 2.15it/s]
Loading 0: 9%|▉ | 34.0/363 [00:22<02:56, 1.87it/s]
Loading 0: 9%|▉ | 34.0/363 [00:22<02:56, 1.87it/s]
Loading 0: 10%|▉ | 35.0/363 [00:24<03:45, 1.45it/s]
Loading 0: 10%|▉ | 35.0/363 [00:24<03:45, 1.45it/s]
Loading 0: 10%|▉ | 36.0/363 [00:26<04:44, 1.15it/s]
Loading 0: 10%|▉ | 36.0/363 [00:26<04:44, 1.15it/s]
Loading 0: 11%|█ | 39.0/363 [00:28<04:15, 1.27it/s]
Loading 0: 11%|█ | 39.0/363 [00:28<04:15, 1.27it/s]
Loading 0: 11%|█ | 40.0/363 [00:30<05:08, 1.05it/s]
Loading 0: 11%|█ | 40.0/363 [00:30<05:08, 1.05it/s]
Loading 0: 11%|█▏ | 41.0/363 [00:32<06:08, 1.14s/it]
Loading 0: 11%|█▏ | 41.0/363 [00:32<06:08, 1.14s/it]
Loading 0: 13%|█▎ | 49.0/363 [00:33<02:32, 2.06it/s]
Loading 0: 13%|█▎ | 49.0/363 [00:33<02:32, 2.06it/s]
Loading 0: 14%|█▍ | 52.0/363 [00:36<02:52, 1.81it/s]
Loading 0: 14%|█▍ | 52.0/363 [00:36<02:52, 1.81it/s]
Loading 0: 15%|█▍ | 53.0/363 [00:38<03:38, 1.42it/s]
Loading 0: 15%|█▍ | 53.0/363 [00:38<03:38, 1.42it/s]
Loading 0: 15%|█▍ | 54.0/363 [00:40<04:34, 1.13it/s]
Loading 0: 15%|█▍ | 54.0/363 [00:40<04:34, 1.13it/s]
Loading 0: 16%|█▌ | 57.0/363 [00:42<04:04, 1.25it/s]
Loading 0: 16%|█▌ | 57.0/363 [00:42<04:04, 1.25it/s]
Loading 0: 16%|█▌ | 58.0/363 [00:43<04:55, 1.03it/s]
Loading 0: 16%|█▌ | 58.0/363 [00:43<04:55, 1.03it/s]
Loading 0: 16%|█▋ | 59.0/363 [00:46<05:52, 1.16s/it]
Loading 0: 16%|█▋ | 59.0/363 [00:46<05:52, 1.16s/it]
Loading 0: 18%|█▊ | 67.0/363 [00:47<02:23, 2.06it/s]
Loading 0: 18%|█▊ | 67.0/363 [00:47<02:23, 2.06it/s]
Loading 0: 19%|█▉ | 70.0/363 [00:49<02:42, 1.80it/s]
Loading 0: 19%|█▉ | 70.0/363 [00:49<02:42, 1.80it/s]
Loading 0: 20%|█▉ | 71.0/363 [00:51<03:26, 1.41it/s]
Loading 0: 20%|█▉ | 71.0/363 [00:51<03:26, 1.41it/s]
Loading 0: 20%|█▉ | 72.0/363 [00:53<04:19, 1.12it/s]
Loading 0: 20%|█▉ | 72.0/363 [00:53<04:19, 1.12it/s]
Loading 0: 21%|██ | 75.0/363 [00:55<03:51, 1.25it/s]
Loading 0: 21%|██ | 75.0/363 [00:55<03:51, 1.25it/s]
Loading 0: 21%|██ | 76.0/363 [00:57<04:38, 1.03it/s]
Loading 0: 21%|██ | 76.0/363 [00:57<04:38, 1.03it/s]
Loading 0: 21%|██ | 77.0/363 [00:59<05:31, 1.16s/it]
Loading 0: 21%|██ | 77.0/363 [00:59<05:31, 1.16s/it]
Loading 0: 23%|██▎ | 85.0/363 [01:00<02:15, 2.06it/s]
Loading 0: 23%|██▎ | 85.0/363 [01:00<02:15, 2.06it/s]
Loading 0: 24%|██▍ | 88.0/363 [01:02<02:32, 1.80it/s]
Loading 0: 24%|██▍ | 88.0/363 [01:02<02:32, 1.80it/s]
Loading 0: 25%|██▍ | 89.0/363 [01:04<03:13, 1.41it/s]
Loading 0: 25%|██▍ | 89.0/363 [01:04<03:13, 1.41it/s]
Loading 0: 25%|██▍ | 90.0/363 [01:06<04:03, 1.12it/s]
Loading 0: 25%|██▍ | 90.0/363 [01:06<04:03, 1.12it/s]
Loading 0: 27%|██▋ | 98.0/363 [01:07<01:55, 2.29it/s]
Loading 0: 27%|██▋ | 98.0/363 [01:07<01:55, 2.29it/s]
Loading 0: 28%|██▊ | 101/363 [01:10<02:11, 1.99it/s]
Loading 0: 28%|██▊ | 101/363 [01:10<02:11, 1.99it/s]
Loading 0: 28%|██▊ | 102/363 [01:12<02:50, 1.54it/s]
Loading 0: 28%|██▊ | 102/363 [01:12<02:50, 1.54it/s]
Loading 0: 28%|██▊ | 103/363 [01:14<03:36, 1.20it/s]
Loading 0: 28%|██▊ | 103/363 [01:14<03:36, 1.20it/s]
Loading 0: 29%|██▉ | 106/363 [01:16<03:21, 1.28it/s]
Loading 0: 29%|██▉ | 106/363 [01:16<03:21, 1.28it/s]
Loading 0: 29%|██▉ | 107/363 [01:18<04:03, 1.05it/s]
Loading 0: 29%|██▉ | 107/363 [01:18<04:03, 1.05it/s]
Loading 0: 30%|██▉ | 108/363 [01:20<04:50, 1.14s/it]
Loading 0: 30%|██▉ | 108/363 [01:20<04:50, 1.14s/it]
Loading 0: 31%|███ | 111/363 [01:22<03:54, 1.07it/s]
Loading 0: 31%|███ | 111/363 [01:22<03:54, 1.07it/s]
Loading 0: 31%|███ | 112/363 [01:23<04:35, 1.10s/it]
Loading 0: 31%|███ | 112/363 [01:23<04:35, 1.10s/it]
Loading 0: 31%|███ | 113/363 [01:26<05:19, 1.28s/it]
Loading 0: 31%|███ | 113/363 [01:26<05:19, 1.28s/it]
Loading 0: 33%|███▎ | 120/363 [01:27<02:08, 1.89it/s]
Loading 0: 33%|███▎ | 120/363 [01:27<02:08, 1.89it/s]
Loading 0: 34%|███▍ | 124/363 [01:29<02:13, 1.78it/s]
Loading 0: 34%|███▍ | 124/363 [01:29<02:13, 1.78it/s]
Loading 0: 34%|███▍ | 125/363 [01:31<02:49, 1.41it/s]
Loading 0: 34%|███▍ | 125/363 [01:31<02:49, 1.41it/s]
Loading 0: 35%|███▍ | 126/363 [01:33<03:32, 1.12it/s]
Loading 0: 35%|███▍ | 126/363 [01:33<03:32, 1.12it/s]
Loading 0: 36%|███▌ | 129/363 [01:35<03:07, 1.25it/s]
Loading 0: 36%|███▌ | 129/363 [01:35<03:07, 1.25it/s]
Loading 0: 36%|███▌ | 130/363 [01:37<03:45, 1.03it/s]
Loading 0: 36%|███▌ | 130/363 [01:37<03:45, 1.03it/s]
Loading 0: 36%|███▌ | 131/363 [01:39<04:28, 1.16s/it]
Loading 0: 36%|███▌ | 131/363 [01:39<04:28, 1.16s/it]
Loading 0: 38%|███▊ | 138/363 [01:40<01:54, 1.96it/s]
Loading 0: 38%|███▊ | 138/363 [01:40<01:54, 1.96it/s]
Loading 0: 39%|███▉ | 142/363 [01:42<02:00, 1.83it/s]
Loading 0: 39%|███▉ | 142/363 [01:42<02:00, 1.83it/s]
Loading 0: 39%|███▉ | 143/363 [01:44<02:33, 1.44it/s]
Loading 0: 39%|███▉ | 143/363 [01:44<02:33, 1.44it/s]
Loading 0: 40%|███▉ | 144/363 [01:46<03:12, 1.14it/s]
Loading 0: 40%|███▉ | 144/363 [01:46<03:12, 1.14it/s]
Loading 0: 40%|████ | 147/363 [01:48<02:51, 1.26it/s]
Loading 0: 40%|████ | 147/363 [01:48<02:51, 1.26it/s]
Loading 0: 41%|████ | 148/363 [01:50<03:26, 1.04it/s]
Loading 0: 41%|████ | 148/363 [01:50<03:26, 1.04it/s]
Loading 0: 41%|████ | 149/363 [01:52<04:05, 1.15s/it]
Loading 0: 41%|████ | 149/363 [01:52<04:05, 1.15s/it]
Loading 0: 43%|████▎ | 157/363 [01:53<01:39, 2.06it/s]
Loading 0: 43%|████▎ | 157/363 [01:53<01:39, 2.06it/s]
Loading 0: 44%|████▍ | 160/363 [01:56<01:51, 1.81it/s]
Loading 0: 44%|████▍ | 160/363 [01:56<01:51, 1.81it/s]
Loading 0: 44%|████▍ | 161/363 [01:58<02:22, 1.42it/s]
Loading 0: 44%|████▍ | 161/363 [01:58<02:22, 1.42it/s]
Loading 0: 45%|████▍ | 162/363 [02:00<02:57, 1.13it/s]
Loading 0: 45%|████▍ | 162/363 [02:00<02:57, 1.13it/s]
Loading 0: 45%|████▌ | 165/363 [02:01<02:37, 1.25it/s]
Loading 0: 45%|████▌ | 165/363 [02:01<02:37, 1.25it/s]
Loading 0: 46%|████▌ | 166/363 [02:03<03:09, 1.04it/s]
Loading 0: 46%|████▌ | 166/363 [02:03<03:09, 1.04it/s]
Loading 0: 46%|████▌ | 167/363 [02:05<03:45, 1.15s/it]
Loading 0: 46%|████▌ | 167/363 [02:05<03:45, 1.15s/it]
Loading 0: 48%|████▊ | 175/363 [02:07<01:30, 2.07it/s]
Loading 0: 48%|████▊ | 175/363 [02:07<01:30, 2.07it/s]
Loading 0: 49%|████▉ | 178/363 [02:09<01:41, 1.82it/s]
Loading 0: 49%|████▉ | 178/363 [02:09<01:41, 1.82it/s]
Loading 0: 49%|████▉ | 179/363 [02:11<02:09, 1.42it/s]
Loading 0: 49%|████▉ | 179/363 [02:11<02:09, 1.42it/s]
Loading 0: 50%|████▉ | 180/363 [02:13<02:41, 1.13it/s]
Loading 0: 50%|████▉ | 180/363 [02:13<02:41, 1.13it/s]
Loading 0: 50%|█████ | 183/363 [02:15<02:22, 1.26it/s]
Loading 0: 50%|█████ | 183/363 [02:15<02:22, 1.26it/s]
Loading 0: 51%|█████ | 184/363 [02:17<02:52, 1.04it/s]
Loading 0: 51%|█████ | 184/363 [02:17<02:52, 1.04it/s]
Loading 0: 51%|█████ | 185/363 [02:19<03:25, 1.15s/it]
Loading 0: 51%|█████ | 185/363 [02:19<03:25, 1.15s/it]
Loading 0: 53%|█████▎ | 193/363 [02:20<01:22, 2.07it/s]
Loading 0: 53%|█████▎ | 193/363 [02:20<01:22, 2.07it/s]
Loading 0: 54%|█████▍ | 196/363 [02:22<01:31, 1.82it/s]
Loading 0: 54%|█████▍ | 196/363 [02:22<01:31, 1.82it/s]
Loading 0: 54%|█████▍ | 197/363 [02:24<01:56, 1.43it/s]
Loading 0: 54%|█████▍ | 197/363 [02:24<01:56, 1.43it/s]
Loading 0: 55%|█████▍ | 198/363 [02:26<02:25, 1.13it/s]
Loading 0: 55%|█████▍ | 198/363 [02:26<02:25, 1.13it/s]
Loading 0: 55%|█████▌ | 201/363 [02:28<02:08, 1.26it/s]
Loading 0: 55%|█████▌ | 201/363 [02:28<02:08, 1.26it/s]
Loading 0: 56%|█████▌ | 202/363 [02:30<02:34, 1.04it/s]
Loading 0: 56%|█████▌ | 202/363 [02:30<02:34, 1.04it/s]
Loading 0: 56%|█████▌ | 203/363 [02:32<03:03, 1.15s/it]
Loading 0: 56%|█████▌ | 203/363 [02:32<03:03, 1.15s/it]
Loading 0: 58%|█████▊ | 211/363 [02:33<01:12, 2.09it/s]
Loading 0: 58%|█████▊ | 211/363 [02:33<01:12, 2.09it/s]
Loading 0: 59%|█████▉ | 214/363 [02:35<01:21, 1.83it/s]
Loading 0: 59%|█████▉ | 214/363 [02:35<01:21, 1.83it/s]
Loading 0: 59%|█████▉ | 215/363 [02:37<01:42, 1.44it/s]
Loading 0: 59%|█████▉ | 215/363 [02:37<01:42, 1.44it/s]
Loading 0: 60%|█████▉ | 216/363 [02:39<02:08, 1.14it/s]
Loading 0: 60%|█████▉ | 216/363 [02:39<02:08, 1.14it/s]
Loading 0: 60%|██████ | 219/363 [02:41<01:53, 1.27it/s]
Loading 0: 60%|██████ | 219/363 [02:41<01:53, 1.27it/s]
Loading 0: 61%|██████ | 220/363 [02:43<02:16, 1.05it/s]
Loading 0: 61%|██████ | 220/363 [02:43<02:16, 1.05it/s]
Loading 0: 61%|██████ | 221/363 [02:45<02:41, 1.14s/it]
Loading 0: 61%|██████ | 221/363 [02:45<02:41, 1.14s/it]
Loading 0: 63%|██████▎ | 229/363 [02:46<01:03, 2.10it/s]
Loading 0: 63%|██████▎ | 229/363 [02:46<01:03, 2.10it/s]
Loading 0: 64%|██████▍ | 232/363 [02:48<01:11, 1.84it/s]
Loading 0: 64%|██████▍ | 232/363 [02:48<01:11, 1.84it/s]
Loading 0: 64%|██████▍ | 233/363 [02:50<01:30, 1.44it/s]
Loading 0: 64%|██████▍ | 233/363 [02:50<01:30, 1.44it/s]
Loading 0: 64%|██████▍ | 234/363 [02:52<01:52, 1.14it/s]
Loading 0: 64%|██████▍ | 234/363 [02:52<01:52, 1.14it/s]
Loading 0: 65%|██████▌ | 237/363 [02:54<01:39, 1.27it/s]
Loading 0: 65%|██████▌ | 237/363 [02:54<01:39, 1.27it/s]
Loading 0: 66%|██████▌ | 238/363 [02:56<01:59, 1.05it/s]
Loading 0: 66%|██████▌ | 238/363 [02:56<01:59, 1.05it/s]
Loading 0: 66%|██████▌ | 239/363 [02:58<02:21, 1.14s/it]
Loading 0: 66%|██████▌ | 239/363 [02:58<02:21, 1.14s/it]
Loading 0: 68%|██████▊ | 247/363 [02:59<00:55, 2.09it/s]
Loading 0: 68%|██████▊ | 247/363 [02:59<00:55, 2.09it/s]
Loading 0: 69%|██████▉ | 250/363 [03:01<01:01, 1.83it/s]
Loading 0: 69%|██████▉ | 250/363 [03:01<01:01, 1.83it/s]
Loading 0: 69%|██████▉ | 251/363 [03:03<01:17, 1.44it/s]
Loading 0: 69%|██████▉ | 251/363 [03:03<01:17, 1.44it/s]
Loading 0: 69%|██████▉ | 252/363 [03:05<01:37, 1.14it/s]
Loading 0: 69%|██████▉ | 252/363 [03:05<01:37, 1.14it/s]
Loading 0: 70%|███████ | 255/363 [03:07<01:25, 1.27it/s]
Loading 0: 70%|███████ | 255/363 [03:07<01:25, 1.27it/s]
Loading 0: 71%|███████ | 256/363 [03:09<01:42, 1.05it/s]
Loading 0: 71%|███████ | 256/363 [03:09<01:42, 1.05it/s]
Loading 0: 71%|███████ | 257/363 [03:11<02:01, 1.14s/it]
Loading 0: 71%|███████ | 257/363 [03:11<02:01, 1.14s/it]
Loading 0: 73%|███████▎ | 265/363 [03:12<00:47, 2.08it/s]
Loading 0: 73%|███████▎ | 265/363 [03:12<00:47, 2.08it/s]
Loading 0: 74%|███████▍ | 268/363 [03:15<00:51, 1.83it/s]
Loading 0: 74%|███████▍ | 268/363 [03:15<00:51, 1.83it/s]
Loading 0: 74%|███████▍ | 269/363 [03:16<01:05, 1.43it/s]
Loading 0: 74%|███████▍ | 269/363 [03:16<01:05, 1.43it/s]
Loading 0: 74%|███████▍ | 270/363 [03:18<01:21, 1.14it/s]
Loading 0: 74%|███████▍ | 270/363 [03:18<01:21, 1.14it/s]
Loading 0: 75%|███████▌ | 273/363 [03:34<03:37, 2.41s/it]
Loading 0: 75%|███████▌ | 273/363 [03:34<03:37, 2.41s/it]
Loading 0: 75%|███████▌ | 274/363 [03:35<03:28, 2.34s/it]
Loading 0: 75%|███████▌ | 274/363 [03:35<03:28, 2.34s/it]
Loading 0: 76%|███████▌ | 275/363 [03:38<03:21, 2.29s/it]
Loading 0: 76%|███████▌ | 275/363 [03:38<03:21, 2.29s/it]
Loading 0: 78%|███████▊ | 283/363 [03:39<01:08, 1.16it/s]
Loading 0: 78%|███████▊ | 283/363 [03:39<01:08, 1.16it/s]
Loading 0: 79%|███████▉ | 286/363 [03:41<01:03, 1.22it/s]
Loading 0: 79%|███████▉ | 286/363 [03:41<01:03, 1.22it/s]
Loading 0: 79%|███████▉ | 287/363 [03:43<01:10, 1.08it/s]
Loading 0: 79%|███████▉ | 287/363 [03:43<01:10, 1.08it/s]
Loading 0: 79%|███████▉ | 288/363 [03:45<01:21, 1.08s/it]
Loading 0: 79%|███████▉ | 288/363 [03:45<01:21, 1.08s/it]
Loading 0: 80%|████████ | 291/363 [03:47<01:06, 1.09it/s]
Loading 0: 80%|████████ | 291/363 [03:47<01:06, 1.09it/s]
Loading 0: 80%|████████ | 292/363 [03:49<01:15, 1.07s/it]
Loading 0: 80%|████████ | 292/363 [03:49<01:15, 1.07s/it]
Loading 0: 81%|████████ | 293/363 [03:51<01:26, 1.24s/it]
Loading 0: 81%|████████ | 293/363 [03:51<01:26, 1.24s/it]
Loading 0: 83%|████████▎ | 301/363 [03:52<00:31, 1.98it/s]
Loading 0: 83%|████████▎ | 301/363 [03:52<00:31, 1.98it/s]
Loading 0: 84%|████████▎ | 304/363 [03:54<00:33, 1.76it/s]
Loading 0: 84%|████████▎ | 304/363 [03:54<00:33, 1.76it/s]
Loading 0: 84%|████████▍ | 305/363 [03:56<00:41, 1.39it/s]
Loading 0: 84%|████████▍ | 305/363 [03:56<00:41, 1.39it/s]
Loading 0: 84%|████████▍ | 306/363 [03:58<00:51, 1.11it/s]
Loading 0: 84%|████████▍ | 306/363 [03:58<00:51, 1.11it/s]
Loading 0: 85%|████████▌ | 309/363 [04:00<00:43, 1.24it/s]
Loading 0: 85%|████████▌ | 309/363 [04:00<00:43, 1.24it/s]
Loading 0: 85%|████████▌ | 310/363 [04:02<00:51, 1.02it/s]
Loading 0: 85%|████████▌ | 310/363 [04:02<00:51, 1.02it/s]
Loading 0: 86%|████████▌ | 311/363 [04:04<01:00, 1.16s/it]
Loading 0: 86%|████████▌ | 311/363 [04:04<01:00, 1.16s/it]
Loading 0: 88%|████████▊ | 319/363 [04:05<00:21, 2.08it/s]
Loading 0: 88%|████████▊ | 319/363 [04:05<00:21, 2.08it/s]
Loading 0: 89%|████████▊ | 322/363 [04:07<00:22, 1.81it/s]
Loading 0: 89%|████████▊ | 322/363 [04:07<00:22, 1.81it/s]
Loading 0: 89%|████████▉ | 323/363 [04:09<00:28, 1.42it/s]
Loading 0: 89%|████████▉ | 323/363 [04:09<00:28, 1.42it/s]
Loading 0: 89%|████████▉ | 324/363 [04:11<00:34, 1.12it/s]
Loading 0: 89%|████████▉ | 324/363 [04:11<00:34, 1.12it/s]
Loading 0: 90%|█████████ | 327/363 [04:13<00:28, 1.25it/s]
Loading 0: 90%|█████████ | 327/363 [04:13<00:28, 1.25it/s]
Loading 0: 90%|█████████ | 328/363 [04:15<00:33, 1.03it/s]
Loading 0: 90%|█████████ | 328/363 [04:15<00:33, 1.03it/s]
Loading 0: 91%|█████████ | 329/363 [04:17<00:39, 1.16s/it]
Loading 0: 91%|█████████ | 329/363 [04:17<00:39, 1.16s/it]
Loading 0: 93%|█████████▎| 337/363 [04:18<00:12, 2.08it/s]
Loading 0: 93%|█████████▎| 337/363 [04:18<00:12, 2.08it/s]
Loading 0: 94%|█████████▎| 340/363 [04:21<00:12, 1.81it/s]
Loading 0: 94%|█████████▎| 340/363 [04:21<00:12, 1.81it/s]
Loading 0: 94%|█████████▍| 341/363 [04:22<00:15, 1.42it/s]
Loading 0: 94%|█████████▍| 341/363 [04:22<00:15, 1.42it/s]
Loading 0: 94%|█████████▍| 342/363 [04:24<00:18, 1.13it/s]
Loading 0: 94%|█████████▍| 342/363 [04:24<00:18, 1.13it/s]
Loading 0: 95%|█████████▌| 345/363 [04:26<00:14, 1.25it/s]
Loading 0: 95%|█████████▌| 345/363 [04:26<00:14, 1.25it/s]
Loading 0: 95%|█████████▌| 346/363 [04:28<00:16, 1.03it/s]
Loading 0: 95%|█████████▌| 346/363 [04:28<00:16, 1.03it/s]
Loading 0: 96%|█████████▌| 347/363 [04:30<00:18, 1.17s/it]
Loading 0: 96%|█████████▌| 347/363 [04:30<00:18, 1.17s/it]
Loading 0: 98%|█████████▊| 355/363 [04:32<00:03, 2.07it/s]
Loading 0: 98%|█████████▊| 355/363 [04:32<00:03, 2.07it/s]
Loading 0: 99%|█████████▉| 359/363 [04:34<00:02, 1.85it/s]
Loading 0: 99%|█████████▉| 359/363 [04:34<00:02, 1.85it/s]
Loading 0: 99%|█████████▉| 360/363 [04:36<00:02, 1.46it/s]
Loading 0: 99%|█████████▉| 360/363 [04:36<00:02, 1.46it/s]
Loading 0: 99%|█████████▉| 361/363 [04:38<00:01, 1.17it/s]
Loading 0: 99%|█████████▉| 361/363 [04:38<00:01, 1.17it/s]
Loading 0: 100%|██████████| 363/363 [04:38<00:00, 1.17it/s]
Loading 0: 100%|██████████| 363/363 [04:38<00:00, 1.30it/s]
chaiml-mistral-24b-2048-83002-v4-mkmlizer: The tokenizer you are loading from '/tmp/tmpmye32o0r' with an incorrect regex pattern: https://huggingface.co/mistralai/Mistral-Small-3.1-24B-Instruct-2503/discussions/84#69121093e8b480e709447d5e. This will lead to incorrect tokenization. You should set the `fix_mistral_regex=True` flag when loading this tokenizer to fix this issue.
chaiml-mistral-24b-2048-83002-v4-mkmlizer: quantized model in 288.657s
chaiml-mistral-24b-2048-83002-v4-mkmlizer: Processed model ChaiML/mistral_24b_2048_gemini_opus_ds_v10_843_merged in 376.385s
chaiml-mistral-24b-2048-83002-v4-mkmlizer: creating bucket guanaco-mkml-models
chaiml-mistral-24b-2048-83002-v4-mkmlizer: Bucket 's3://guanaco-mkml-models/' created
chaiml-mistral-24b-2048-83002-v4-mkmlizer: uploading /dev/shm/model_cache to s3://guanaco-mkml-models/chaiml-mistral-24b-2048-83002-v4/nvidia
chaiml-mistral-24b-2048-83002-v4-mkmlizer: DEBUG "sync /dev/shm/model_cache/config.json s3://guanaco-mkml-models/chaiml-mistral-24b-2048-83002-v4/nvidia/config.json": object size matches
chaiml-mistral-24b-2048-83002-v4-mkmlizer: DEBUG "sync /dev/shm/model_cache/flywheel_model.1.safetensors s3://guanaco-mkml-models/chaiml-mistral-24b-2048-83002-v4/nvidia/flywheel_model.1.safetensors": object size matches
chaiml-mistral-24b-2048-83002-v4-mkmlizer: DEBUG "sync /dev/shm/model_cache/special_tokens_map.json s3://guanaco-mkml-models/chaiml-mistral-24b-2048-83002-v4/nvidia/special_tokens_map.json": object size matches
chaiml-mistral-24b-2048-83002-v4-mkmlizer: DEBUG "sync /dev/shm/model_cache/tokenizer.json s3://guanaco-mkml-models/chaiml-mistral-24b-2048-83002-v4/nvidia/tokenizer.json": object size matches
chaiml-mistral-24b-2048-83002-v4-mkmlizer: DEBUG "sync /dev/shm/model_cache/tokenizer_config.json s3://guanaco-mkml-models/chaiml-mistral-24b-2048-83002-v4/nvidia/tokenizer_config.json": object size matches
chaiml-mistral-24b-2048-83002-v4-mkmlizer: cp /dev/shm/model_cache/flywheel_model.0.safetensors s3://guanaco-mkml-models/chaiml-mistral-24b-2048-83002-v4/nvidia/flywheel_model.0.safetensors
Job chaiml-mistral-24b-2048-83002-v4-mkmlizer completed after 643.51s with status: succeeded
Stopping job with name chaiml-mistral-24b-2048-83002-v4-mkmlizer
Pipeline stage MKMLizer completed in 1355.02s
run pipeline stage %s
Running pipeline stage MKMLTemplater
Pipeline stage MKMLTemplater completed in 0.35s
run pipeline stage %s
Running pipeline stage MKMLDeployer
Creating inference service chaiml-mistral-24b-2048-83002-v4
Waiting for inference service chaiml-mistral-24b-2048-83002-v4 to be ready
Inference service chaiml-mistral-24b-2048-83002-v4 ready after 233.4919023513794s
Pipeline stage MKMLDeployer completed in 234.71s
run pipeline stage %s
Running pipeline stage StressChecker
Received healthy response to inference request in 3.445882797241211s
Received healthy response to inference request in 3.333137035369873s
Received healthy response to inference request in 2.9493322372436523s
Received healthy response to inference request in 3.4849045276641846s
Received healthy response to inference request in 3.2419633865356445s
5 requests
0 failed requests
5th percentile: 3.007858467102051
10th percentile: 3.066384696960449
20th percentile: 3.183437156677246
30th percentile: 3.2601981163024902
40th percentile: 3.2966675758361816
50th percentile: 3.333137035369873
60th percentile: 3.378235340118408
70th percentile: 3.4233336448669434
80th percentile: 3.453687143325806
90th percentile: 3.469295835494995
95th percentile: 3.4771001815795897
99th percentile: 3.4833436584472657
mean time: 3.291043996810913
Pipeline stage StressChecker completed in 19.84s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyTriggerPipeline
run_pipeline:run_in_cloud %s
starting trigger_guanaco_pipeline args=%s
triggered trigger_guanaco_pipeline args=%s
Pipeline stage OfflineFamilyFriendlyTriggerPipeline completed in 1.31s
run pipeline stage %s
Running pipeline stage TriggerMKMLProfilingPipeline
run_pipeline:run_in_cloud %s
starting trigger_guanaco_pipeline args=%s
triggered trigger_guanaco_pipeline args=%s
Pipeline stage TriggerMKMLProfilingPipeline completed in 1.17s
Shutdown handler de-registered
chaiml-mistral-24b-2048_83002_v4 status is now deployed due to DeploymentManager action