Shutdown handler not registered because Python interpreter is not running in the main thread
run pipeline %s
run pipeline stage %s
Running pipeline stage MKMLizer
Starting job with name chaiml-mistral-24b-2048-4101-v1-mkmlizer
Waiting for job on chaiml-mistral-24b-2048-4101-v1-mkmlizer to finish
Shutdown handler not registered because Python interpreter is not running in the main thread
run pipeline %s
run pipeline stage %s
Running pipeline stage MKMLizer
Starting job with name chaiml-mistral-24b-2048-90209-v1-mkmlizer
Waiting for job on chaiml-mistral-24b-2048-90209-v1-mkmlizer to finish
Shutdown handler not registered because Python interpreter is not running in the main thread
run pipeline %s
run pipeline stage %s
Running pipeline stage MKMLizer
Starting job with name chaiml-mistral-24b-2048-46891-v1-mkmlizer
Waiting for job on chaiml-mistral-24b-2048-46891-v1-mkmlizer to finish
Shutdown handler not registered because Python interpreter is not running in the main thread
run pipeline %s
run pipeline stage %s
Running pipeline stage MKMLizer
Starting job with name chaiml-mistral-24b-2048-82993-v1-mkmlizer
Waiting for job on chaiml-mistral-24b-2048-82993-v1-mkmlizer to finish
chaiml-mistral-24b-2048-4101-v1-mkmlizer: bash: cannot set terminal process group (-1): Inappropriate ioctl for device
chaiml-mistral-24b-2048-4101-v1-mkmlizer: bash: no job control in this shell
chaiml-mistral-24b-2048-4101-v1-mkmlizer: /root/miniconda3/envs/nvidia/lib/python3.11/site-packages/mk1/__init__.py:1: UserWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html. The pkg_resources package is slated for removal as early as 2025-11-30. Refrain from using this package or pin to Setuptools<81.
chaiml-mistral-24b-2048-4101-v1-mkmlizer: __import__('pkg_resources').declare_namespace(__name__)
chaiml-mistral-24b-2048-4101-v1-mkmlizer: ╔═════════════════════════════════════════════════════════════════════╗
chaiml-mistral-24b-2048-4101-v1-mkmlizer: ║ ║
chaiml-mistral-24b-2048-4101-v1-mkmlizer: ║ ██████ ██████ █████ ████ ████ ║
chaiml-mistral-24b-2048-4101-v1-mkmlizer: ║ ░░██████ ██████ ░░███ ███░ ░░███ ║
chaiml-mistral-24b-2048-4101-v1-mkmlizer: ║ ░███░█████░███ ░███ ███ ░███ ║
chaiml-mistral-24b-2048-4101-v1-mkmlizer: ║ ░███░░███ ░███ ░███████ ░███ ║
chaiml-mistral-24b-2048-4101-v1-mkmlizer: ║ ░███ ░░░ ░███ ░███░░███ ░███ ║
chaiml-mistral-24b-2048-4101-v1-mkmlizer: ║ ░███ ░███ ░███ ░░███ ░███ ║
chaiml-mistral-24b-2048-4101-v1-mkmlizer: ║ █████ █████ █████ ░░████ █████ ║
chaiml-mistral-24b-2048-4101-v1-mkmlizer: ║ ░░░░░ ░░░░░ ░░░░░ ░░░░ ░░░░░ ║
chaiml-mistral-24b-2048-4101-v1-mkmlizer: ║ ║
chaiml-mistral-24b-2048-4101-v1-mkmlizer: ║ Version: 0.30.6+torch280 ║
chaiml-mistral-24b-2048-4101-v1-mkmlizer: ║ Features: FLYWHEEL, CUDA ║
chaiml-mistral-24b-2048-90209-v1-mkmlizer: /root/miniconda3/envs/nvidia/lib/python3.11/site-packages/mk1/__init__.py:1: UserWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html. The pkg_resources package is slated for removal as early as 2025-11-30. Refrain from using this package or pin to Setuptools<81.
chaiml-mistral-24b-2048-4101-v1-mkmlizer: ║ Copyright 2023-2025 MK ONE TECHNOLOGIES Inc. ║
chaiml-mistral-24b-2048-90209-v1-mkmlizer: __import__('pkg_resources').declare_namespace(__name__)
chaiml-mistral-24b-2048-4101-v1-mkmlizer: ║ https://mk1.ai ║
chaiml-mistral-24b-2048-90209-v1-mkmlizer: ╔═════════════════════════════════════════════════════════════════════╗
chaiml-mistral-24b-2048-4101-v1-mkmlizer: ║ ║
chaiml-mistral-24b-2048-90209-v1-mkmlizer: ║ ║
chaiml-mistral-24b-2048-4101-v1-mkmlizer: ║ The license key for the current software has been verified as ║
chaiml-mistral-24b-2048-90209-v1-mkmlizer: ║ ██████ ██████ █████ ████ ████ ║
chaiml-mistral-24b-2048-4101-v1-mkmlizer: ║ belonging to: ║
chaiml-mistral-24b-2048-90209-v1-mkmlizer: ║ ░░██████ ██████ ░░███ ███░ ░░███ ║
chaiml-mistral-24b-2048-4101-v1-mkmlizer: ║ ║
chaiml-mistral-24b-2048-90209-v1-mkmlizer: ║ ░███░█████░███ ░███ ███ ░███ ║
chaiml-mistral-24b-2048-4101-v1-mkmlizer: ║ Chai Research Corp. ║
chaiml-mistral-24b-2048-90209-v1-mkmlizer: ║ ░███░░███ ░███ ░███████ ░███ ║
chaiml-mistral-24b-2048-4101-v1-mkmlizer: ║ Account ID: 7997a29f-0ceb-4cc7-9adf-840c57b4ae6f ║
chaiml-mistral-24b-2048-90209-v1-mkmlizer: ║ ░███ ░░░ ░███ ░███░░███ ░███ ║
chaiml-mistral-24b-2048-4101-v1-mkmlizer: ║ Expiration: 2028-03-31 23:59:59 ║
chaiml-mistral-24b-2048-90209-v1-mkmlizer: ║ ░███ ░███ ░███ ░░███ ░███ ║
chaiml-mistral-24b-2048-4101-v1-mkmlizer: ║ ║
chaiml-mistral-24b-2048-90209-v1-mkmlizer: ║ █████ █████ █████ ░░████ █████ ║
chaiml-mistral-24b-2048-4101-v1-mkmlizer: ╚═════════════════════════════════════════════════════════════════════╝
chaiml-mistral-24b-2048-90209-v1-mkmlizer: ║ ░░░░░ ░░░░░ ░░░░░ ░░░░ ░░░░░ ║
chaiml-mistral-24b-2048-90209-v1-mkmlizer: ║ ║
chaiml-mistral-24b-2048-90209-v1-mkmlizer: ║ Version: 0.30.6+torch280 ║
chaiml-mistral-24b-2048-90209-v1-mkmlizer: ║ Features: FLYWHEEL, CUDA ║
chaiml-mistral-24b-2048-46891-v1-mkmlizer: /root/miniconda3/envs/nvidia/lib/python3.11/site-packages/mk1/__init__.py:1: UserWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html. The pkg_resources package is slated for removal as early as 2025-11-30. Refrain from using this package or pin to Setuptools<81.
chaiml-mistral-24b-2048-90209-v1-mkmlizer: ║ Copyright 2023-2025 MK ONE TECHNOLOGIES Inc. ║
chaiml-mistral-24b-2048-46891-v1-mkmlizer: __import__('pkg_resources').declare_namespace(__name__)
chaiml-mistral-24b-2048-90209-v1-mkmlizer: ║ https://mk1.ai ║
chaiml-mistral-24b-2048-46891-v1-mkmlizer: ╔═════════════════════════════════════════════════════════════════════╗
chaiml-mistral-24b-2048-90209-v1-mkmlizer: ║ ║
chaiml-mistral-24b-2048-46891-v1-mkmlizer: ║ ║
chaiml-mistral-24b-2048-90209-v1-mkmlizer: ║ The license key for the current software has been verified as ║
chaiml-mistral-24b-2048-46891-v1-mkmlizer: ║ ██████ ██████ █████ ████ ████ ║
chaiml-mistral-24b-2048-90209-v1-mkmlizer: ║ belonging to: ║
chaiml-mistral-24b-2048-46891-v1-mkmlizer: ║ ░░██████ ██████ ░░███ ███░ ░░███ ║
chaiml-mistral-24b-2048-90209-v1-mkmlizer: ║ ║
chaiml-mistral-24b-2048-46891-v1-mkmlizer: ║ ░███░█████░███ ░███ ███ ░███ ║
chaiml-mistral-24b-2048-90209-v1-mkmlizer: ║ Chai Research Corp. ║
chaiml-mistral-24b-2048-46891-v1-mkmlizer: ║ ░███░░███ ░███ ░███████ ░███ ║
chaiml-mistral-24b-2048-90209-v1-mkmlizer: ║ Account ID: 7997a29f-0ceb-4cc7-9adf-840c57b4ae6f ║
chaiml-mistral-24b-2048-46891-v1-mkmlizer: ║ ░███ ░░░ ░███ ░███░░███ ░███ ║
chaiml-mistral-24b-2048-90209-v1-mkmlizer: ║ Expiration: 2028-03-31 23:59:59 ║
chaiml-mistral-24b-2048-46891-v1-mkmlizer: ║ ░███ ░███ ░███ ░░███ ░███ ║
chaiml-mistral-24b-2048-90209-v1-mkmlizer: ║ ║
chaiml-mistral-24b-2048-46891-v1-mkmlizer: ║ █████ █████ █████ ░░████ █████ ║
chaiml-mistral-24b-2048-82993-v1-mkmlizer: /root/miniconda3/envs/nvidia/lib/python3.11/site-packages/mk1/__init__.py:1: UserWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html. The pkg_resources package is slated for removal as early as 2025-11-30. Refrain from using this package or pin to Setuptools<81.
chaiml-mistral-24b-2048-90209-v1-mkmlizer: ╚═════════════════════════════════════════════════════════════════════╝
chaiml-mistral-24b-2048-46891-v1-mkmlizer: ║ ░░░░░ ░░░░░ ░░░░░ ░░░░ ░░░░░ ║
chaiml-mistral-24b-2048-82993-v1-mkmlizer: __import__('pkg_resources').declare_namespace(__name__)
chaiml-mistral-24b-2048-46891-v1-mkmlizer: ║ ║
chaiml-mistral-24b-2048-82993-v1-mkmlizer: ╔═════════════════════════════════════════════════════════════════════╗
chaiml-mistral-24b-2048-46891-v1-mkmlizer: ║ Version: 0.30.6+torch280 ║
chaiml-mistral-24b-2048-82993-v1-mkmlizer: ║ ║
chaiml-mistral-24b-2048-46891-v1-mkmlizer: ║ Features: FLYWHEEL, CUDA ║
chaiml-mistral-24b-2048-82993-v1-mkmlizer: ║ ██████ ██████ █████ ████ ████ ║
chaiml-mistral-24b-2048-46891-v1-mkmlizer: ║ Copyright 2023-2025 MK ONE TECHNOLOGIES Inc. ║
chaiml-mistral-24b-2048-82993-v1-mkmlizer: ║ ░░██████ ██████ ░░███ ███░ ░░███ ║
chaiml-mistral-24b-2048-46891-v1-mkmlizer: ║ https://mk1.ai ║
chaiml-mistral-24b-2048-82993-v1-mkmlizer: ║ ░███░█████░███ ░███ ███ ░███ ║
chaiml-mistral-24b-2048-46891-v1-mkmlizer: ║ ║
chaiml-mistral-24b-2048-82993-v1-mkmlizer: ║ ░███░░███ ░███ ░███████ ░███ ║
chaiml-mistral-24b-2048-46891-v1-mkmlizer: ║ The license key for the current software has been verified as ║
chaiml-mistral-24b-2048-82993-v1-mkmlizer: ║ ░███ ░░░ ░███ ░███░░███ ░███ ║
chaiml-mistral-24b-2048-46891-v1-mkmlizer: ║ belonging to: ║
chaiml-mistral-24b-2048-82993-v1-mkmlizer: ║ ░███ ░███ ░███ ░░███ ░███ ║
chaiml-mistral-24b-2048-46891-v1-mkmlizer: ║ ║
chaiml-mistral-24b-2048-82993-v1-mkmlizer: ║ █████ █████ █████ ░░████ █████ ║
chaiml-mistral-24b-2048-46891-v1-mkmlizer: ║ Chai Research Corp. ║
chaiml-mistral-24b-2048-82993-v1-mkmlizer: ║ ░░░░░ ░░░░░ ░░░░░ ░░░░ ░░░░░ ║
chaiml-mistral-24b-2048-46891-v1-mkmlizer: ║ Account ID: 7997a29f-0ceb-4cc7-9adf-840c57b4ae6f ║
chaiml-mistral-24b-2048-82993-v1-mkmlizer: ║ ║
chaiml-mistral-24b-2048-46891-v1-mkmlizer: ║ Expiration: 2028-03-31 23:59:59 ║
chaiml-mistral-24b-2048-82993-v1-mkmlizer: ║ Version: 0.30.6+torch280 ║
chaiml-mistral-24b-2048-46891-v1-mkmlizer: ║ ║
chaiml-mistral-24b-2048-82993-v1-mkmlizer: ║ Features: FLYWHEEL, CUDA ║
chaiml-mistral-24b-2048-46891-v1-mkmlizer: ╚═════════════════════════════════════════════════════════════════════╝
chaiml-mistral-24b-2048-82993-v1-mkmlizer: ║ Copyright 2023-2025 MK ONE TECHNOLOGIES Inc. ║
chaiml-mistral-24b-2048-82993-v1-mkmlizer: ║ https://mk1.ai ║
chaiml-mistral-24b-2048-82993-v1-mkmlizer: ║ ║
chaiml-mistral-24b-2048-82993-v1-mkmlizer: ║ The license key for the current software has been verified as ║
chaiml-mistral-24b-2048-82993-v1-mkmlizer: ║ belonging to: ║
chaiml-mistral-24b-2048-82993-v1-mkmlizer: ║ ║
chaiml-mistral-24b-2048-82993-v1-mkmlizer: ║ Chai Research Corp. ║
chaiml-mistral-24b-2048-82993-v1-mkmlizer: ║ Account ID: 7997a29f-0ceb-4cc7-9adf-840c57b4ae6f ║
chaiml-mistral-24b-2048-82993-v1-mkmlizer: ║ Expiration: 2028-03-31 23:59:59 ║
chaiml-mistral-24b-2048-82993-v1-mkmlizer: ║ ║
chaiml-mistral-24b-2048-82993-v1-mkmlizer: ╚═════════════════════════════════════════════════════════════════════╝
chaiml-mistral-24b-2048-4101-v1-mkmlizer: Downloaded to shared memory in 89.911s
chaiml-mistral-24b-2048-46891-v1-mkmlizer: Downloaded to shared memory in 82.403s
chaiml-mistral-24b-2048-4101-v1-mkmlizer: Checking if ChaiML/mistral_24b_2048_gemini_ds_v7_4372_merged already exists in ChaiML
chaiml-mistral-24b-2048-46891-v1-mkmlizer: Checking if ChaiML/mistral_24b_2048_gemini_ds_v7_3279_merged already exists in ChaiML
chaiml-mistral-24b-2048-4101-v1-mkmlizer: quantizing model to /dev/shm/model_cache, profile:q4, folder:/tmp/tmppkcws2yz, device:0
chaiml-mistral-24b-2048-46891-v1-mkmlizer: quantizing model to /dev/shm/model_cache, profile:q4, folder:/tmp/tmpa30ubyxi, device:0
chaiml-mistral-24b-2048-4101-v1-mkmlizer: Saving flywheel model at /dev/shm/model_cache
chaiml-mistral-24b-2048-46891-v1-mkmlizer: Saving flywheel model at /dev/shm/model_cache
chaiml-mistral-24b-2048-82993-v1-mkmlizer: Downloaded to shared memory in 91.241s
chaiml-mistral-24b-2048-82993-v1-mkmlizer: Checking if ChaiML/mistral_24b_2048_gemini_ds_v6_3279_merged already exists in ChaiML
chaiml-mistral-24b-2048-82993-v1-mkmlizer: quantizing model to /dev/shm/model_cache, profile:q4, folder:/tmp/tmptga7wpba, device:0
chaiml-mistral-24b-2048-90209-v1-mkmlizer: Downloaded to shared memory in 104.300s
chaiml-mistral-24b-2048-82993-v1-mkmlizer: Saving flywheel model at /dev/shm/model_cache
chaiml-mistral-24b-2048-90209-v1-mkmlizer: Checking if ChaiML/mistral_24b_2048_gemini_ds_v6_4372_merged already exists in ChaiML
chaiml-mistral-24b-2048-90209-v1-mkmlizer: quantizing model to /dev/shm/model_cache, profile:q4, folder:/tmp/tmpcm5lelcq, device:0
chaiml-mistral-24b-2048-90209-v1-mkmlizer: Saving flywheel model at /dev/shm/model_cache
Stopping job with name chaiml-mistral-24b-2048-46891-v1-mkmlizer
%s, retrying in %s seconds...
Starting job with name chaiml-mistral-24b-2048-46891-v1-mkmlizer
Waiting for job on chaiml-mistral-24b-2048-46891-v1-mkmlizer to finish
chaiml-mistral-24b-2048-46891-v1-mkmlizer: bash: cannot set terminal process group (-1): Inappropriate ioctl for device
chaiml-mistral-24b-2048-46891-v1-mkmlizer: bash: no job control in this shell
chaiml-mistral-24b-2048-46891-v1-mkmlizer: /root/miniconda3/envs/nvidia/lib/python3.11/site-packages/mk1/__init__.py:1: UserWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html. The pkg_resources package is slated for removal as early as 2025-11-30. Refrain from using this package or pin to Setuptools<81.
chaiml-mistral-24b-2048-46891-v1-mkmlizer: __import__('pkg_resources').declare_namespace(__name__)
chaiml-mistral-24b-2048-46891-v1-mkmlizer: ╔═════════════════════════════════════════════════════════════════════╗
chaiml-mistral-24b-2048-46891-v1-mkmlizer: ║ ║
chaiml-mistral-24b-2048-46891-v1-mkmlizer: ║ ██████ ██████ █████ ████ ████ ║
chaiml-mistral-24b-2048-46891-v1-mkmlizer: ║ ░░██████ ██████ ░░███ ███░ ░░███ ║
chaiml-mistral-24b-2048-46891-v1-mkmlizer: ║ ░███░█████░███ ░███ ███ ░███ ║
chaiml-mistral-24b-2048-46891-v1-mkmlizer: ║ ░███░░███ ░███ ░███████ ░███ ║
chaiml-mistral-24b-2048-46891-v1-mkmlizer: ║ ░███ ░░░ ░███ ░███░░███ ░███ ║
chaiml-mistral-24b-2048-46891-v1-mkmlizer: ║ ░███ ░███ ░███ ░░███ ░███ ║
chaiml-mistral-24b-2048-46891-v1-mkmlizer: ║ █████ █████ █████ ░░████ █████ ║
chaiml-mistral-24b-2048-46891-v1-mkmlizer: ║ ░░░░░ ░░░░░ ░░░░░ ░░░░ ░░░░░ ║
chaiml-mistral-24b-2048-46891-v1-mkmlizer: ║ ║
chaiml-mistral-24b-2048-46891-v1-mkmlizer: ║ Version: 0.30.6+torch280 ║
chaiml-mistral-24b-2048-46891-v1-mkmlizer: ║ Features: FLYWHEEL, CUDA ║
chaiml-mistral-24b-2048-46891-v1-mkmlizer: ║ Copyright 2023-2025 MK ONE TECHNOLOGIES Inc. ║
chaiml-mistral-24b-2048-46891-v1-mkmlizer: ║ https://mk1.ai ║
chaiml-mistral-24b-2048-46891-v1-mkmlizer: ║ ║
chaiml-mistral-24b-2048-46891-v1-mkmlizer: ║ The license key for the current software has been verified as ║
chaiml-mistral-24b-2048-46891-v1-mkmlizer: ║ belonging to: ║
chaiml-mistral-24b-2048-46891-v1-mkmlizer: ║ ║
chaiml-mistral-24b-2048-46891-v1-mkmlizer: ║ Chai Research Corp. ║
chaiml-mistral-24b-2048-46891-v1-mkmlizer: ║ Account ID: 7997a29f-0ceb-4cc7-9adf-840c57b4ae6f ║
chaiml-mistral-24b-2048-46891-v1-mkmlizer: ║ Expiration: 2028-03-31 23:59:59 ║
chaiml-mistral-24b-2048-46891-v1-mkmlizer: ║ ║
chaiml-mistral-24b-2048-46891-v1-mkmlizer: ╚═════════════════════════════════════════════════════════════════════╝
Stopping job with name chaiml-mistral-24b-2048-46891-v1-mkmlizer
%s, retrying in %s seconds...
Starting job with name chaiml-mistral-24b-2048-46891-v1-mkmlizer
Waiting for job on chaiml-mistral-24b-2048-46891-v1-mkmlizer to finish
chaiml-mistral-24b-2048-46891-v1-mkmlizer: /root/miniconda3/envs/nvidia/lib/python3.11/site-packages/mk1/__init__.py:1: UserWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html. The pkg_resources package is slated for removal as early as 2025-11-30. Refrain from using this package or pin to Setuptools<81.
chaiml-mistral-24b-2048-46891-v1-mkmlizer: __import__('pkg_resources').declare_namespace(__name__)
chaiml-mistral-24b-2048-46891-v1-mkmlizer: ╔═════════════════════════════════════════════════════════════════════╗
chaiml-mistral-24b-2048-46891-v1-mkmlizer: ║ ║
chaiml-mistral-24b-2048-46891-v1-mkmlizer: ║ ██████ ██████ █████ ████ ████ ║
chaiml-mistral-24b-2048-46891-v1-mkmlizer: ║ ░░██████ ██████ ░░███ ███░ ░░███ ║
chaiml-mistral-24b-2048-46891-v1-mkmlizer: ║ ░███░█████░███ ░███ ███ ░███ ║
chaiml-mistral-24b-2048-46891-v1-mkmlizer: ║ ░███░░███ ░███ ░███████ ░███ ║
chaiml-mistral-24b-2048-46891-v1-mkmlizer: ║ ░███ ░░░ ░███ ░███░░███ ░███ ║
chaiml-mistral-24b-2048-46891-v1-mkmlizer: ║ ░███ ░███ ░███ ░░███ ░███ ║
chaiml-mistral-24b-2048-46891-v1-mkmlizer: ║ █████ █████ █████ ░░████ █████ ║
chaiml-mistral-24b-2048-46891-v1-mkmlizer: ║ ░░░░░ ░░░░░ ░░░░░ ░░░░ ░░░░░ ║
chaiml-mistral-24b-2048-46891-v1-mkmlizer: ║ ║
chaiml-mistral-24b-2048-46891-v1-mkmlizer: ║ Version: 0.30.6+torch280 ║
chaiml-mistral-24b-2048-46891-v1-mkmlizer: ║ Features: FLYWHEEL, CUDA ║
chaiml-mistral-24b-2048-46891-v1-mkmlizer: ║ Copyright 2023-2025 MK ONE TECHNOLOGIES Inc. ║
chaiml-mistral-24b-2048-46891-v1-mkmlizer: ║ https://mk1.ai ║
chaiml-mistral-24b-2048-46891-v1-mkmlizer: ║ ║
chaiml-mistral-24b-2048-46891-v1-mkmlizer: ║ The license key for the current software has been verified as ║
chaiml-mistral-24b-2048-46891-v1-mkmlizer: ║ belonging to: ║
chaiml-mistral-24b-2048-46891-v1-mkmlizer: ║ ║
chaiml-mistral-24b-2048-46891-v1-mkmlizer: ║ Chai Research Corp. ║
chaiml-mistral-24b-2048-46891-v1-mkmlizer: ║ Account ID: 7997a29f-0ceb-4cc7-9adf-840c57b4ae6f ║
chaiml-mistral-24b-2048-46891-v1-mkmlizer: ║ Expiration: 2028-03-31 23:59:59 ║
chaiml-mistral-24b-2048-46891-v1-mkmlizer: ║ ║
chaiml-mistral-24b-2048-46891-v1-mkmlizer: ╚═════════════════════════════════════════════════════════════════════╝
chaiml-mistral-24b-2048-82993-v1-mkmlizer:
Loading 0: 0%| | 0.00/363 [00:00<?, ?it/s]
Loading 0: 1%| | 3.00/363 [00:02<04:20, 1.38it/s]
Loading 0: 1%| | 3.00/363 [00:02<04:20, 1.38it/s]
Loading 0: 1%| | 4.00/363 [00:04<06:43, 1.13s/it]
Loading 0: 1%| | 4.00/363 [00:04<06:43, 1.13s/it]
Loading 0: 1%|▏ | 5.00/363 [00:06<08:26, 1.42s/it]
Loading 0: 1%|▏ | 5.00/363 [00:06<08:26, 1.42s/it]
Loading 0: 3%|▎ | 12.0/363 [00:07<02:46, 2.11it/s]
Loading 0: 3%|▎ | 12.0/363 [00:07<02:46, 2.11it/s]
Loading 0: 4%|▎ | 13.0/363 [00:09<03:53, 1.50it/s]
Loading 0: 4%|▎ | 13.0/363 [00:09<03:53, 1.50it/s]
Loading 0: 4%|▍ | 14.0/363 [00:11<05:03, 1.15it/s]
Loading 0: 4%|▍ | 14.0/363 [00:11<05:03, 1.15it/s]
Loading 0: 4%|▍ | 15.0/363 [00:13<06:18, 1.09s/it]
Loading 0: 4%|▍ | 15.0/363 [00:13<06:18, 1.09s/it]
Loading 0: 6%|▌ | 21.0/363 [00:15<03:46, 1.51it/s]
Loading 0: 6%|▌ | 21.0/363 [00:15<03:46, 1.51it/s]
Loading 0: 6%|▌ | 22.0/363 [00:17<04:39, 1.22it/s]
Loading 0: 6%|▌ | 22.0/363 [00:17<04:39, 1.22it/s]
Loading 0: 6%|▋ | 23.0/363 [00:19<05:42, 1.01s/it]
Loading 0: 6%|▋ | 23.0/363 [00:19<05:42, 1.01s/it]
Loading 0: 9%|▊ | 31.0/363 [00:20<02:33, 2.17it/s]
Loading 0: 9%|▊ | 31.0/363 [00:20<02:33, 2.17it/s]
Loading 0: 9%|▉ | 34.0/363 [00:23<02:56, 1.87it/s]
Loading 0: 9%|▉ | 34.0/363 [00:23<02:56, 1.87it/s]
Loading 0: 10%|▉ | 35.0/363 [00:24<03:44, 1.46it/s]
Loading 0: 10%|▉ | 35.0/363 [00:24<03:44, 1.46it/s]
Loading 0: 10%|▉ | 36.0/363 [00:27<04:42, 1.16it/s]
Loading 0: 10%|▉ | 36.0/363 [00:27<04:42, 1.16it/s]
Loading 0: 11%|█ | 39.0/363 [00:28<04:14, 1.27it/s]
Loading 0: 11%|█ | 39.0/363 [00:28<04:14, 1.27it/s]
Loading 0: 11%|█ | 40.0/363 [00:30<05:07, 1.05it/s]
Loading 0: 11%|█ | 40.0/363 [00:30<05:07, 1.05it/s]
Loading 0: 11%|█▏ | 41.0/363 [00:32<06:06, 1.14s/it]
Loading 0: 11%|█▏ | 41.0/363 [00:32<06:06, 1.14s/it]
Loading 0: 13%|█▎ | 49.0/363 [00:34<02:31, 2.08it/s]
Loading 0: 13%|█▎ | 49.0/363 [00:34<02:31, 2.08it/s]
Loading 0: 14%|█▍ | 52.0/363 [00:36<02:51, 1.81it/s]
Loading 0: 14%|█▍ | 52.0/363 [00:36<02:51, 1.81it/s]
Loading 0: 15%|█▍ | 53.0/363 [00:38<03:37, 1.42it/s]
Loading 0: 15%|█▍ | 53.0/363 [00:38<03:37, 1.42it/s]
Loading 0: 15%|█▍ | 54.0/363 [00:40<04:32, 1.13it/s]
Loading 0: 15%|█▍ | 54.0/363 [00:40<04:32, 1.13it/s]
Loading 0: 16%|█▌ | 57.0/363 [00:42<04:02, 1.26it/s]
Loading 0: 16%|█▌ | 57.0/363 [00:42<04:02, 1.26it/s]
Loading 0: 16%|█▌ | 58.0/363 [00:44<04:52, 1.04it/s]
Loading 0: 16%|█▌ | 58.0/363 [00:44<04:52, 1.04it/s]
Loading 0: 16%|█▋ | 59.0/363 [00:46<05:48, 1.15s/it]
Loading 0: 16%|█▋ | 59.0/363 [00:46<05:48, 1.15s/it]
Loading 0: 18%|█▊ | 67.0/363 [00:47<02:23, 2.07it/s]
Loading 0: 18%|█▊ | 67.0/363 [00:47<02:23, 2.07it/s]
Loading 0: 19%|█▉ | 70.0/363 [00:49<02:42, 1.81it/s]
Loading 0: 19%|█▉ | 70.0/363 [00:49<02:42, 1.81it/s]
Loading 0: 20%|█▉ | 71.0/363 [00:51<03:25, 1.42it/s]
Loading 0: 20%|█▉ | 71.0/363 [00:51<03:25, 1.42it/s]
Loading 0: 20%|█▉ | 72.0/363 [00:53<04:18, 1.13it/s]
Loading 0: 20%|█▉ | 72.0/363 [00:53<04:18, 1.13it/s]
Loading 0: 21%|██ | 75.0/363 [00:55<03:49, 1.26it/s]
Loading 0: 21%|██ | 75.0/363 [00:55<03:49, 1.26it/s]
Loading 0: 21%|██ | 76.0/363 [00:57<04:36, 1.04it/s]
Loading 0: 21%|██ | 76.0/363 [00:57<04:36, 1.04it/s]
Loading 0: 21%|██ | 77.0/363 [00:59<05:28, 1.15s/it]
Loading 0: 21%|██ | 77.0/363 [00:59<05:28, 1.15s/it]
Loading 0: 23%|██▎ | 85.0/363 [01:00<02:14, 2.07it/s]
Loading 0: 23%|██▎ | 85.0/363 [01:00<02:14, 2.07it/s]
Loading 0: 24%|██▍ | 88.0/363 [01:02<02:32, 1.80it/s]
Loading 0: 24%|██▍ | 88.0/363 [01:02<02:32, 1.80it/s]
Loading 0: 25%|██▍ | 89.0/363 [01:04<03:13, 1.42it/s]
Loading 0: 25%|██▍ | 89.0/363 [01:04<03:13, 1.42it/s]
Loading 0: 25%|██▍ | 90.0/363 [01:06<04:01, 1.13it/s]
Loading 0: 25%|██▍ | 90.0/363 [01:06<04:01, 1.13it/s]
Loading 0: 27%|██▋ | 97.0/363 [01:07<02:01, 2.20it/s]
Loading 0: 27%|██▋ | 97.0/363 [01:07<02:01, 2.20it/s]
Loading 0: 28%|██▊ | 101/363 [01:10<02:09, 2.02it/s]
Loading 0: 28%|██▊ | 101/363 [01:10<02:09, 2.02it/s]
Loading 0: 28%|██▊ | 102/363 [01:12<02:47, 1.56it/s]
Loading 0: 28%|██▊ | 102/363 [01:12<02:47, 1.56it/s]
Loading 0: 28%|██▊ | 103/363 [01:14<03:38, 1.19it/s]
Loading 0: 28%|██▊ | 103/363 [01:14<03:38, 1.19it/s]
Loading 0: 29%|██▉ | 106/363 [01:16<03:23, 1.26it/s]
Loading 0: 29%|██▉ | 106/363 [01:16<03:23, 1.26it/s]
Loading 0: 29%|██▉ | 107/363 [01:18<04:03, 1.05it/s]
Loading 0: 29%|██▉ | 107/363 [01:18<04:03, 1.05it/s]
Loading 0: 30%|██▉ | 108/363 [01:20<04:49, 1.14s/it]
Loading 0: 30%|██▉ | 108/363 [01:20<04:49, 1.14s/it]
Loading 0: 31%|███ | 111/363 [01:22<03:53, 1.08it/s]
Loading 0: 31%|███ | 111/363 [01:22<03:53, 1.08it/s]
Loading 0: 31%|███ | 112/363 [01:24<04:33, 1.09s/it]
Loading 0: 31%|███ | 112/363 [01:24<04:33, 1.09s/it]
Loading 0: 31%|███ | 113/363 [01:26<05:19, 1.28s/it]
Loading 0: 31%|███ | 113/363 [01:26<05:19, 1.28s/it]
Loading 0: 33%|███▎ | 121/363 [01:27<02:00, 2.01it/s]
Loading 0: 33%|███▎ | 121/363 [01:27<02:00, 2.01it/s]
Loading 0: 34%|███▍ | 124/363 [01:29<02:14, 1.77it/s]
Loading 0: 34%|███▍ | 124/363 [01:29<02:14, 1.77it/s]
Loading 0: 34%|███▍ | 125/363 [01:31<02:50, 1.39it/s]
Loading 0: 34%|███▍ | 125/363 [01:31<02:50, 1.39it/s]
Loading 0: 35%|███▍ | 126/363 [01:33<03:33, 1.11it/s]
Loading 0: 35%|███▍ | 126/363 [01:33<03:33, 1.11it/s]
Loading 0: 36%|███▌ | 129/363 [01:35<03:07, 1.25it/s]
Loading 0: 36%|███▌ | 129/363 [01:35<03:07, 1.25it/s]
Loading 0: 36%|███▌ | 130/363 [01:37<03:45, 1.03it/s]
Loading 0: 36%|███▌ | 130/363 [01:37<03:45, 1.03it/s]
Loading 0: 36%|███▌ | 131/363 [01:39<04:28, 1.16s/it]
Loading 0: 36%|███▌ | 131/363 [01:39<04:28, 1.16s/it]
Loading 0: 38%|███▊ | 139/363 [01:40<01:48, 2.06it/s]
Loading 0: 38%|███▊ | 139/363 [01:40<01:48, 2.06it/s]
Loading 0: 39%|███▉ | 142/363 [01:42<02:02, 1.80it/s]
Loading 0: 39%|███▉ | 142/363 [01:42<02:02, 1.80it/s]
Loading 0: 39%|███▉ | 143/363 [01:44<02:35, 1.42it/s]
Loading 0: 39%|███▉ | 143/363 [01:44<02:35, 1.42it/s]
Loading 0: 40%|███▉ | 144/363 [01:46<03:14, 1.13it/s]
Loading 0: 40%|███▉ | 144/363 [01:46<03:14, 1.13it/s]
Loading 0: 40%|████ | 147/363 [01:48<02:51, 1.26it/s]
Loading 0: 40%|████ | 147/363 [01:48<02:51, 1.26it/s]
Loading 0: 41%|████ | 148/363 [01:50<03:27, 1.04it/s]
Loading 0: 41%|████ | 148/363 [01:50<03:27, 1.04it/s]
Loading 0: 41%|████ | 149/363 [01:52<04:06, 1.15s/it]
Loading 0: 41%|████ | 149/363 [01:52<04:06, 1.15s/it]
Loading 0: 43%|████▎ | 157/363 [01:53<01:39, 2.07it/s]
Loading 0: 43%|████▎ | 157/363 [01:53<01:39, 2.07it/s]
Loading 0: 44%|████▍ | 160/363 [01:56<01:52, 1.81it/s]
Loading 0: 44%|████▍ | 160/363 [01:56<01:52, 1.81it/s]
Loading 0: 44%|████▍ | 161/363 [01:57<02:22, 1.42it/s]
Loading 0: 44%|████▍ | 161/363 [01:57<02:22, 1.42it/s]
Loading 0: 45%|████▍ | 162/363 [01:59<02:57, 1.13it/s]
Loading 0: 45%|████▍ | 162/363 [01:59<02:57, 1.13it/s]
Loading 0: 45%|████▌ | 165/363 [02:01<02:37, 1.26it/s]
Loading 0: 45%|████▌ | 165/363 [02:01<02:37, 1.26it/s]
Loading 0: 46%|████▌ | 166/363 [02:03<03:09, 1.04it/s]
Loading 0: 46%|████▌ | 166/363 [02:03<03:09, 1.04it/s]
Loading 0: 46%|████▌ | 167/363 [02:05<03:45, 1.15s/it]
Loading 0: 46%|████▌ | 167/363 [02:05<03:45, 1.15s/it]
Loading 0: 48%|████▊ | 175/363 [02:07<01:30, 2.07it/s]
Loading 0: 48%|████▊ | 175/363 [02:07<01:30, 2.07it/s]
Loading 0: 49%|████▉ | 178/363 [02:09<01:42, 1.81it/s]
Loading 0: 49%|████▉ | 178/363 [02:09<01:42, 1.81it/s]
Loading 0: 49%|████▉ | 179/363 [02:11<02:09, 1.42it/s]
Loading 0: 49%|████▉ | 179/363 [02:11<02:09, 1.42it/s]
Loading 0: 50%|████▉ | 180/363 [02:13<02:41, 1.13it/s]
Loading 0: 50%|████▉ | 180/363 [02:13<02:41, 1.13it/s]
Loading 0: 50%|█████ | 183/363 [02:15<02:22, 1.26it/s]
Loading 0: 50%|█████ | 183/363 [02:15<02:22, 1.26it/s]
Loading 0: 51%|█████ | 184/363 [02:17<02:51, 1.04it/s]
Loading 0: 51%|█████ | 184/363 [02:17<02:51, 1.04it/s]
Loading 0: 51%|█████ | 185/363 [02:19<03:24, 1.15s/it]
Loading 0: 51%|█████ | 185/363 [02:19<03:24, 1.15s/it]
Loading 0: 53%|█████▎ | 193/363 [02:20<01:22, 2.07it/s]
Loading 0: 53%|█████▎ | 193/363 [02:20<01:22, 2.07it/s]
Loading 0: 54%|█████▍ | 196/363 [02:22<01:32, 1.81it/s]
Loading 0: 54%|█████▍ | 196/363 [02:22<01:32, 1.81it/s]
Loading 0: 54%|█████▍ | 197/363 [02:24<01:56, 1.42it/s]
Loading 0: 54%|█████▍ | 197/363 [02:24<01:56, 1.42it/s]
Loading 0: 55%|█████▍ | 198/363 [02:26<02:25, 1.13it/s]
Loading 0: 55%|█████▍ | 198/363 [02:26<02:25, 1.13it/s]
Loading 0: 55%|█████▌ | 201/363 [02:28<02:08, 1.26it/s]
Loading 0: 55%|█████▌ | 201/363 [02:28<02:08, 1.26it/s]
Loading 0: 56%|█████▌ | 202/363 [02:30<02:34, 1.04it/s]
Loading 0: 56%|█████▌ | 202/363 [02:30<02:34, 1.04it/s]
Loading 0: 56%|█████▌ | 203/363 [02:32<03:03, 1.15s/it]
Loading 0: 56%|█████▌ | 203/363 [02:32<03:03, 1.15s/it]
Loading 0: 58%|█████▊ | 211/363 [02:33<01:13, 2.07it/s]
Loading 0: 58%|█████▊ | 211/363 [02:33<01:13, 2.07it/s]
Loading 0: 59%|█████▉ | 214/363 [02:35<01:22, 1.81it/s]
Loading 0: 59%|█████▉ | 214/363 [02:35<01:22, 1.81it/s]
Loading 0: 59%|█████▉ | 215/363 [02:37<01:43, 1.43it/s]
Loading 0: 59%|█████▉ | 215/363 [02:37<01:43, 1.43it/s]
Loading 0: 60%|█████▉ | 216/363 [02:39<02:09, 1.13it/s]
Loading 0: 60%|█████▉ | 216/363 [02:39<02:09, 1.13it/s]
Loading 0: 60%|██████ | 219/363 [02:41<01:54, 1.26it/s]
Loading 0: 60%|██████ | 219/363 [02:41<01:54, 1.26it/s]
Loading 0: 61%|██████ | 220/363 [02:43<02:17, 1.04it/s]
Loading 0: 61%|██████ | 220/363 [02:43<02:17, 1.04it/s]
Loading 0: 61%|██████ | 221/363 [02:45<02:42, 1.15s/it]
Loading 0: 61%|██████ | 221/363 [02:45<02:42, 1.15s/it]
Loading 0: 63%|██████▎ | 229/363 [02:46<01:04, 2.07it/s]
Loading 0: 63%|██████▎ | 229/363 [02:46<01:04, 2.07it/s]
Loading 0: 64%|██████▍ | 232/363 [02:48<01:12, 1.81it/s]
Loading 0: 64%|██████▍ | 232/363 [02:48<01:12, 1.81it/s]
Loading 0: 64%|██████▍ | 233/363 [02:50<01:31, 1.42it/s]
Loading 0: 64%|██████▍ | 233/363 [02:50<01:31, 1.42it/s]
Loading 0: 64%|██████▍ | 234/363 [02:52<01:54, 1.13it/s]
Loading 0: 64%|██████▍ | 234/363 [02:52<01:54, 1.13it/s]
Loading 0: 65%|██████▌ | 237/363 [02:54<01:39, 1.26it/s]
Loading 0: 65%|██████▌ | 237/363 [02:54<01:39, 1.26it/s]
Loading 0: 66%|██████▌ | 238/363 [02:56<01:59, 1.04it/s]
Loading 0: 66%|██████▌ | 238/363 [02:56<01:59, 1.04it/s]
Loading 0: 66%|██████▌ | 239/363 [02:58<02:22, 1.15s/it]
Loading 0: 66%|██████▌ | 239/363 [02:58<02:22, 1.15s/it]
Loading 0: 68%|██████▊ | 247/363 [02:59<00:56, 2.07it/s]
Loading 0: 68%|██████▊ | 247/363 [02:59<00:56, 2.07it/s]
Loading 0: 69%|██████▉ | 250/363 [03:02<01:02, 1.82it/s]
Loading 0: 69%|██████▉ | 250/363 [03:02<01:02, 1.82it/s]
Loading 0: 69%|██████▉ | 251/363 [03:04<01:18, 1.43it/s]
Loading 0: 69%|██████▉ | 251/363 [03:04<01:18, 1.43it/s]
Loading 0: 69%|██████▉ | 252/363 [03:06<01:38, 1.13it/s]
Loading 0: 69%|██████▉ | 252/363 [03:06<01:38, 1.13it/s]
Loading 0: 70%|███████ | 255/363 [03:08<01:25, 1.26it/s]
Loading 0: 70%|███████ | 255/363 [03:08<01:25, 1.26it/s]
Loading 0: 71%|███████ | 256/363 [03:09<01:42, 1.04it/s]
Loading 0: 71%|███████ | 256/363 [03:09<01:42, 1.04it/s]
Loading 0: 71%|███████ | 257/363 [03:11<02:01, 1.14s/it]
Loading 0: 71%|███████ | 257/363 [03:11<02:01, 1.14s/it]
Loading 0: 73%|███████▎ | 265/363 [03:13<00:47, 2.08it/s]
Loading 0: 73%|███████▎ | 265/363 [03:13<00:47, 2.08it/s]
Loading 0: 74%|███████▍ | 268/363 [03:15<00:52, 1.82it/s]
Loading 0: 74%|███████▍ | 268/363 [03:15<00:52, 1.82it/s]
Loading 0: 74%|███████▍ | 269/363 [03:17<01:05, 1.43it/s]
Loading 0: 74%|███████▍ | 269/363 [03:17<01:05, 1.43it/s]
Loading 0: 74%|███████▍ | 270/363 [03:19<01:22, 1.13it/s]
Loading 0: 74%|███████▍ | 270/363 [03:19<01:22, 1.13it/s]
Loading 0: 75%|███████▌ | 273/363 [03:34<03:36, 2.41s/it]
Loading 0: 75%|███████▌ | 273/363 [03:34<03:36, 2.41s/it]
Loading 0: 75%|███████▌ | 274/363 [03:36<03:27, 2.33s/it]
Loading 0: 75%|███████▌ | 274/363 [03:36<03:27, 2.33s/it]
Loading 0: 76%|███████▌ | 275/363 [03:38<03:20, 2.28s/it]
Loading 0: 76%|███████▌ | 275/363 [03:38<03:20, 2.28s/it]
Loading 0: 78%|███████▊ | 283/363 [03:39<01:08, 1.17it/s]
Loading 0: 78%|███████▊ | 283/363 [03:39<01:08, 1.17it/s]
Loading 0: 79%|███████▉ | 286/363 [03:41<01:03, 1.21it/s]
Loading 0: 79%|███████▉ | 286/363 [03:41<01:03, 1.21it/s]
Loading 0: 79%|███████▉ | 287/363 [03:43<01:11, 1.06it/s]
Loading 0: 79%|███████▉ | 287/363 [03:43<01:11, 1.06it/s]
Loading 0: 79%|███████▉ | 288/363 [03:45<01:22, 1.09s/it]
Loading 0: 79%|███████▉ | 288/363 [03:45<01:22, 1.09s/it]
Loading 0: 80%|████████ | 291/363 [03:47<01:06, 1.08it/s]
Loading 0: 80%|████████ | 291/363 [03:47<01:06, 1.08it/s]
Loading 0: 80%|████████ | 292/363 [03:49<01:15, 1.07s/it]
Loading 0: 80%|████████ | 292/363 [03:49<01:15, 1.07s/it]
Loading 0: 81%|████████ | 293/363 [03:51<01:26, 1.24s/it]
Loading 0: 81%|████████ | 293/363 [03:51<01:26, 1.24s/it]
Loading 0: 83%|████████▎ | 301/363 [03:52<00:31, 1.98it/s]
Loading 0: 83%|████████▎ | 301/363 [03:52<00:31, 1.98it/s]
Loading 0: 84%|████████▎ | 304/363 [03:54<00:33, 1.77it/s]
Loading 0: 84%|████████▎ | 304/363 [03:54<00:33, 1.77it/s]
Loading 0: 84%|████████▍ | 305/363 [03:56<00:41, 1.40it/s]
Loading 0: 84%|████████▍ | 305/363 [03:56<00:41, 1.40it/s]
Loading 0: 84%|████████▍ | 306/363 [03:58<00:50, 1.12it/s]
Loading 0: 84%|████████▍ | 306/363 [03:58<00:50, 1.12it/s]
Loading 0: 85%|████████▌ | 309/363 [04:00<00:43, 1.25it/s]
Loading 0: 85%|████████▌ | 309/363 [04:00<00:43, 1.25it/s]
Loading 0: 85%|████████▌ | 310/363 [04:02<00:51, 1.04it/s]
Loading 0: 85%|████████▌ | 310/363 [04:02<00:51, 1.04it/s]
Loading 0: 86%|████████▌ | 311/363 [04:04<00:59, 1.15s/it]
Loading 0: 86%|████████▌ | 311/363 [04:04<00:59, 1.15s/it]
Loading 0: 88%|████████▊ | 319/363 [04:05<00:20, 2.10it/s]
Loading 0: 88%|████████▊ | 319/363 [04:05<00:20, 2.10it/s]
Loading 0: 89%|████████▊ | 322/363 [04:07<00:22, 1.83it/s]
Loading 0: 89%|████████▊ | 322/363 [04:07<00:22, 1.83it/s]
Loading 0: 89%|████████▉ | 323/363 [04:09<00:27, 1.44it/s]
Loading 0: 89%|████████▉ | 323/363 [04:09<00:27, 1.44it/s]
Loading 0: 89%|████████▉ | 324/363 [04:11<00:34, 1.14it/s]
Loading 0: 89%|████████▉ | 324/363 [04:11<00:34, 1.14it/s]
Loading 0: 90%|█████████ | 327/363 [04:13<00:28, 1.27it/s]
Loading 0: 90%|█████████ | 327/363 [04:13<00:28, 1.27it/s]
Loading 0: 90%|█████████ | 328/363 [04:15<00:33, 1.05it/s]
Loading 0: 90%|█████████ | 328/363 [04:15<00:33, 1.05it/s]
Loading 0: 91%|█████████ | 329/363 [04:17<00:38, 1.14s/it]
Loading 0: 91%|█████████ | 329/363 [04:17<00:38, 1.14s/it]
Loading 0: 93%|█████████▎| 337/363 [04:18<00:12, 2.11it/s]
Loading 0: 93%|█████████▎| 337/363 [04:18<00:12, 2.11it/s]
Loading 0: 94%|█████████▎| 340/363 [04:20<00:12, 1.84it/s]
Loading 0: 94%|█████████▎| 340/363 [04:20<00:12, 1.84it/s]
Loading 0: 94%|█████████▍| 341/363 [04:22<00:15, 1.44it/s]
Loading 0: 94%|█████████▍| 341/363 [04:22<00:15, 1.44it/s]
Loading 0: 94%|█████████▍| 342/363 [04:24<00:18, 1.14it/s]
Loading 0: 94%|█████████▍| 342/363 [04:24<00:18, 1.14it/s]
Loading 0: 95%|█████████▌| 345/363 [04:26<00:14, 1.26it/s]
Loading 0: 95%|█████████▌| 345/363 [04:26<00:14, 1.26it/s]
Loading 0: 95%|█████████▌| 346/363 [04:28<00:16, 1.04it/s]
Loading 0: 95%|█████████▌| 346/363 [04:28<00:16, 1.04it/s]
Loading 0: 96%|█████████▌| 347/363 [04:30<00:18, 1.15s/it]
Loading 0: 96%|█████████▌| 347/363 [04:30<00:18, 1.15s/it]
Loading 0: 98%|█████████▊| 355/363 [04:31<00:03, 2.10it/s]
Loading 0: 98%|█████████▊| 355/363 [04:31<00:03, 2.10it/s]
Loading 0: 99%|█████████▉| 359/363 [04:34<00:02, 1.89it/s]
Loading 0: 99%|█████████▉| 359/363 [04:34<00:02, 1.89it/s]
Loading 0: 99%|█████████▉| 360/363 [04:36<00:02, 1.49it/s]
Loading 0: 99%|█████████▉| 360/363 [04:36<00:02, 1.49it/s]
Loading 0: 99%|█████████▉| 361/363 [04:38<00:01, 1.18it/s]
Loading 0: 99%|█████████▉| 361/363 [04:38<00:01, 1.18it/s]
Loading 0: 100%|██████████| 363/363 [04:38<00:00, 1.18it/s]
Loading 0: 100%|██████████| 363/363 [04:38<00:00, 1.30it/s]
chaiml-mistral-24b-2048-90209-v1-mkmlizer:
Loading 0: 0%| | 0.00/363 [00:00<?, ?it/s]
Loading 0: 1%| | 3.00/363 [00:01<03:52, 1.55it/s]
Loading 0: 1%| | 3.00/363 [00:01<03:52, 1.55it/s]
Loading 0: 1%| | 4.00/363 [00:04<06:40, 1.11s/it]
Loading 0: 1%| | 4.00/363 [00:04<06:40, 1.11s/it]
Loading 0: 1%|▏ | 5.00/363 [00:06<08:20, 1.40s/it]
Loading 0: 1%|▏ | 5.00/363 [00:06<08:20, 1.40s/it]
Loading 0: 3%|▎ | 12.0/363 [00:07<02:44, 2.14it/s]
Loading 0: 3%|▎ | 12.0/363 [00:07<02:44, 2.14it/s]
Loading 0: 4%|▎ | 13.0/363 [00:09<03:51, 1.51it/s]
Loading 0: 4%|▎ | 13.0/363 [00:09<03:51, 1.51it/s]
Loading 0: 4%|▍ | 14.0/363 [00:11<04:59, 1.16it/s]
Loading 0: 4%|▍ | 14.0/363 [00:11<04:59, 1.16it/s]
Loading 0: 4%|▍ | 15.0/363 [00:13<06:14, 1.08s/it]
Loading 0: 4%|▍ | 15.0/363 [00:13<06:14, 1.08s/it]
Loading 0: 6%|▌ | 21.0/363 [00:15<03:44, 1.53it/s]
Loading 0: 6%|▌ | 21.0/363 [00:15<03:44, 1.53it/s]
Loading 0: 6%|▌ | 22.0/363 [00:17<04:37, 1.23it/s]
Loading 0: 6%|▌ | 22.0/363 [00:17<04:37, 1.23it/s]
Loading 0: 6%|▋ | 23.0/363 [00:19<05:39, 1.00it/s]
Loading 0: 6%|▋ | 23.0/363 [00:19<05:39, 1.00it/s]
Loading 0: 9%|▊ | 31.0/363 [00:20<02:32, 2.18it/s]
Loading 0: 9%|▊ | 31.0/363 [00:20<02:32, 2.18it/s]
Loading 0: 9%|▉ | 34.0/363 [00:22<02:54, 1.88it/s]
Loading 0: 9%|▉ | 34.0/363 [00:22<02:54, 1.88it/s]
Loading 0: 10%|▉ | 35.0/363 [00:24<03:42, 1.47it/s]
Loading 0: 10%|▉ | 35.0/363 [00:24<03:42, 1.47it/s]
Loading 0: 10%|▉ | 36.0/363 [00:26<04:40, 1.17it/s]
Loading 0: 10%|▉ | 36.0/363 [00:26<04:40, 1.17it/s]
Loading 0: 11%|█ | 39.0/363 [00:28<04:11, 1.29it/s]
Loading 0: 11%|█ | 39.0/363 [00:28<04:11, 1.29it/s]
Loading 0: 11%|█ | 40.0/363 [00:30<05:04, 1.06it/s]
Loading 0: 11%|█ | 40.0/363 [00:30<05:04, 1.06it/s]
Loading 0: 11%|█▏ | 41.0/363 [00:32<06:02, 1.13s/it]
Loading 0: 11%|█▏ | 41.0/363 [00:32<06:02, 1.13s/it]
Loading 0: 13%|█▎ | 49.0/363 [00:33<02:29, 2.10it/s]
Loading 0: 13%|█▎ | 49.0/363 [00:33<02:29, 2.10it/s]
Loading 0: 14%|█▍ | 52.0/363 [00:35<02:49, 1.83it/s]
Loading 0: 14%|█▍ | 52.0/363 [00:35<02:49, 1.83it/s]
Loading 0: 15%|█▍ | 53.0/363 [00:37<03:35, 1.44it/s]
Loading 0: 15%|█▍ | 53.0/363 [00:37<03:35, 1.44it/s]
Loading 0: 15%|█▍ | 54.0/363 [00:39<04:31, 1.14it/s]
Loading 0: 15%|█▍ | 54.0/363 [00:39<04:31, 1.14it/s]
Loading 0: 16%|█▌ | 57.0/363 [00:41<04:01, 1.27it/s]
Loading 0: 16%|█▌ | 57.0/363 [00:41<04:01, 1.27it/s]
Loading 0: 16%|█▌ | 58.0/363 [00:43<04:51, 1.05it/s]
Loading 0: 16%|█▌ | 58.0/363 [00:43<04:51, 1.05it/s]
Loading 0: 16%|█▋ | 59.0/363 [00:45<05:47, 1.14s/it]
Loading 0: 16%|█▋ | 59.0/363 [00:45<05:47, 1.14s/it]
Loading 0: 18%|█▊ | 67.0/363 [00:46<02:22, 2.08it/s]
Loading 0: 18%|█▊ | 67.0/363 [00:46<02:22, 2.08it/s]
Loading 0: 19%|█▉ | 70.0/363 [00:49<02:41, 1.82it/s]
Loading 0: 19%|█▉ | 70.0/363 [00:49<02:41, 1.82it/s]
Loading 0: 20%|█▉ | 71.0/363 [00:51<03:24, 1.43it/s]
Loading 0: 20%|█▉ | 71.0/363 [00:51<03:24, 1.43it/s]
Loading 0: 20%|█▉ | 72.0/363 [00:53<04:16, 1.13it/s]
Loading 0: 20%|█▉ | 72.0/363 [00:53<04:16, 1.13it/s]
Loading 0: 21%|██ | 75.0/363 [00:54<03:47, 1.26it/s]
Loading 0: 21%|██ | 75.0/363 [00:54<03:47, 1.26it/s]
Loading 0: 21%|██ | 76.0/363 [00:56<04:34, 1.05it/s]
Loading 0: 21%|██ | 76.0/363 [00:56<04:34, 1.05it/s]
Loading 0: 21%|██ | 77.0/363 [00:58<05:27, 1.14s/it]
Loading 0: 21%|██ | 77.0/363 [00:58<05:27, 1.14s/it]
Loading 0: 23%|██▎ | 85.0/363 [01:00<02:13, 2.08it/s]
Loading 0: 23%|██▎ | 85.0/363 [01:00<02:13, 2.08it/s]
Loading 0: 24%|██▍ | 88.0/363 [01:02<02:30, 1.82it/s]
Loading 0: 24%|██▍ | 88.0/363 [01:02<02:30, 1.82it/s]
Loading 0: 25%|██▍ | 89.0/363 [01:04<03:11, 1.43it/s]
Loading 0: 25%|██▍ | 89.0/363 [01:04<03:11, 1.43it/s]
Loading 0: 25%|██▍ | 90.0/363 [01:06<04:00, 1.13it/s]
Loading 0: 25%|██▍ | 90.0/363 [01:06<04:00, 1.13it/s]
Loading 0: 27%|██▋ | 98.0/363 [01:07<01:54, 2.32it/s]
Loading 0: 27%|██▋ | 98.0/363 [01:07<01:54, 2.32it/s]
Loading 0: 28%|██▊ | 101/363 [01:09<02:09, 2.02it/s]
Loading 0: 28%|██▊ | 101/363 [01:09<02:09, 2.02it/s]
Loading 0: 28%|██▊ | 102/363 [01:11<02:47, 1.56it/s]
Loading 0: 28%|██▊ | 102/363 [01:11<02:47, 1.56it/s]
Loading 0: 28%|██▊ | 103/363 [01:13<03:32, 1.22it/s]
Loading 0: 28%|██▊ | 103/363 [01:13<03:32, 1.22it/s]
Loading 0: 29%|██▉ | 106/363 [01:15<03:17, 1.30it/s]
Loading 0: 29%|██▉ | 106/363 [01:15<03:17, 1.30it/s]
Loading 0: 29%|██▉ | 107/363 [01:17<03:59, 1.07it/s]
Loading 0: 29%|██▉ | 107/363 [01:17<03:59, 1.07it/s]
Loading 0: 30%|██▉ | 108/363 [01:19<04:46, 1.12s/it]
Loading 0: 30%|██▉ | 108/363 [01:19<04:46, 1.12s/it]
Loading 0: 31%|███ | 111/363 [01:21<03:50, 1.09it/s]
Loading 0: 31%|███ | 111/363 [01:21<03:50, 1.09it/s]
Loading 0: 31%|███ | 112/363 [01:23<04:31, 1.08s/it]
Loading 0: 31%|███ | 112/363 [01:23<04:31, 1.08s/it]
Loading 0: 31%|███ | 113/363 [01:25<05:15, 1.26s/it]
Loading 0: 31%|███ | 113/363 [01:25<05:15, 1.26s/it]
Loading 0: 33%|███▎ | 121/363 [01:26<01:59, 2.02it/s]
Loading 0: 33%|███▎ | 121/363 [01:26<01:59, 2.02it/s]
Loading 0: 34%|███▍ | 124/363 [01:28<02:13, 1.78it/s]
Loading 0: 34%|███▍ | 124/363 [01:28<02:13, 1.78it/s]
Loading 0: 34%|███▍ | 125/363 [01:30<02:49, 1.40it/s]
Loading 0: 34%|███▍ | 125/363 [01:30<02:49, 1.40it/s]
Loading 0: 35%|███▍ | 126/363 [01:32<03:32, 1.12it/s]
Loading 0: 35%|███▍ | 126/363 [01:32<03:32, 1.12it/s]
Loading 0: 36%|███▌ | 129/363 [01:34<03:06, 1.25it/s]
Loading 0: 36%|███▌ | 129/363 [01:34<03:06, 1.25it/s]
Loading 0: 36%|███▌ | 130/363 [01:36<03:45, 1.04it/s]
Loading 0: 36%|███▌ | 130/363 [01:36<03:45, 1.04it/s]
Loading 0: 36%|███▌ | 131/363 [01:38<04:27, 1.15s/it]
Loading 0: 36%|███▌ | 131/363 [01:38<04:27, 1.15s/it]
Loading 0: 38%|███▊ | 139/363 [01:39<01:48, 2.07it/s]
Loading 0: 38%|███▊ | 139/363 [01:39<01:48, 2.07it/s]
Loading 0: 39%|███▉ | 142/363 [01:41<02:01, 1.81it/s]
Loading 0: 39%|███▉ | 142/363 [01:41<02:01, 1.81it/s]
Loading 0: 39%|███▉ | 143/363 [01:43<02:34, 1.42it/s]
Loading 0: 39%|███▉ | 143/363 [01:43<02:34, 1.42it/s]
Loading 0: 40%|███▉ | 144/363 [01:45<03:13, 1.13it/s]
Loading 0: 40%|███▉ | 144/363 [01:45<03:13, 1.13it/s]
Loading 0: 40%|████ | 147/363 [01:47<02:51, 1.26it/s]
Loading 0: 40%|████ | 147/363 [01:47<02:51, 1.26it/s]
Loading 0: 41%|████ | 148/363 [01:49<03:26, 1.04it/s]
Loading 0: 41%|████ | 148/363 [01:49<03:26, 1.04it/s]
Loading 0: 41%|████ | 149/363 [01:51<04:05, 1.15s/it]
Loading 0: 41%|████ | 149/363 [01:51<04:05, 1.15s/it]
Loading 0: 43%|████▎ | 157/363 [01:52<01:38, 2.08it/s]
Loading 0: 43%|████▎ | 157/363 [01:52<01:38, 2.08it/s]
Loading 0: 44%|████▍ | 160/363 [01:54<01:51, 1.83it/s]
Loading 0: 44%|████▍ | 160/363 [01:54<01:51, 1.83it/s]
Loading 0: 44%|████▍ | 161/363 [01:56<02:21, 1.43it/s]
Loading 0: 44%|████▍ | 161/363 [01:56<02:21, 1.43it/s]
Loading 0: 45%|████▍ | 162/363 [01:58<02:56, 1.14it/s]
Loading 0: 45%|████▍ | 162/363 [01:58<02:56, 1.14it/s]
Loading 0: 45%|████▌ | 165/363 [02:00<02:36, 1.27it/s]
Loading 0: 45%|████▌ | 165/363 [02:00<02:36, 1.27it/s]
Loading 0: 46%|████▌ | 166/363 [02:02<03:08, 1.05it/s]
Loading 0: 46%|████▌ | 166/363 [02:02<03:08, 1.05it/s]
Loading 0: 46%|████▌ | 167/363 [02:04<03:44, 1.15s/it]
Loading 0: 46%|████▌ | 167/363 [02:04<03:44, 1.15s/it]
Loading 0: 48%|████▊ | 175/363 [02:05<01:30, 2.08it/s]
Loading 0: 48%|████▊ | 175/363 [02:05<01:30, 2.08it/s]
Loading 0: 49%|████▉ | 178/363 [02:08<01:41, 1.83it/s]
Loading 0: 49%|████▉ | 178/363 [02:08<01:41, 1.83it/s]
Loading 0: 49%|████▉ | 179/363 [02:09<02:08, 1.43it/s]
Loading 0: 49%|████▉ | 179/363 [02:09<02:08, 1.43it/s]
Loading 0: 50%|████▉ | 180/363 [02:11<02:41, 1.14it/s]
Loading 0: 50%|████▉ | 180/363 [02:11<02:41, 1.14it/s]
Loading 0: 50%|█████ | 183/363 [02:13<02:22, 1.27it/s]
Loading 0: 50%|█████ | 183/363 [02:13<02:22, 1.27it/s]
Loading 0: 51%|█████ | 184/363 [02:15<02:51, 1.05it/s]
Loading 0: 51%|█████ | 184/363 [02:15<02:51, 1.05it/s]
Loading 0: 51%|█████ | 185/363 [02:17<03:23, 1.14s/it]
Loading 0: 51%|█████ | 185/363 [02:17<03:23, 1.14s/it]
Loading 0: 53%|█████▎ | 193/363 [02:19<01:21, 2.08it/s]
Loading 0: 53%|█████▎ | 193/363 [02:19<01:21, 2.08it/s]
Loading 0: 54%|█████▍ | 196/363 [02:21<01:31, 1.83it/s]
Loading 0: 54%|█████▍ | 196/363 [02:21<01:31, 1.83it/s]
Loading 0: 54%|█████▍ | 197/363 [02:23<01:55, 1.43it/s]
Loading 0: 54%|█████▍ | 197/363 [02:23<01:55, 1.43it/s]
Loading 0: 55%|█████▍ | 198/363 [02:25<02:24, 1.14it/s]
Loading 0: 55%|█████▍ | 198/363 [02:25<02:24, 1.14it/s]
Loading 0: 55%|█████▌ | 201/363 [02:27<02:07, 1.27it/s]
Loading 0: 55%|█████▌ | 201/363 [02:27<02:07, 1.27it/s]
Loading 0: 56%|█████▌ | 202/363 [02:28<02:33, 1.05it/s]
Loading 0: 56%|█████▌ | 202/363 [02:28<02:33, 1.05it/s]
Loading 0: 56%|█████▌ | 203/363 [02:30<03:02, 1.14s/it]
Loading 0: 56%|█████▌ | 203/363 [02:30<03:02, 1.14s/it]
Loading 0: 58%|█████▊ | 211/363 [02:32<01:12, 2.10it/s]
Loading 0: 58%|█████▊ | 211/363 [02:32<01:12, 2.10it/s]
Loading 0: 59%|█████▉ | 214/363 [02:34<01:20, 1.84it/s]
Loading 0: 59%|█████▉ | 214/363 [02:34<01:20, 1.84it/s]
Loading 0: 59%|█████▉ | 215/363 [02:36<01:42, 1.44it/s]
Loading 0: 59%|█████▉ | 215/363 [02:36<01:42, 1.44it/s]
Loading 0: 60%|█████▉ | 216/363 [02:38<02:08, 1.14it/s]
Loading 0: 60%|█████▉ | 216/363 [02:38<02:08, 1.14it/s]
Loading 0: 60%|██████ | 219/363 [02:40<01:53, 1.27it/s]
Loading 0: 60%|██████ | 219/363 [02:40<01:53, 1.27it/s]
Loading 0: 61%|██████ | 220/363 [02:42<02:16, 1.05it/s]
Loading 0: 61%|██████ | 220/363 [02:42<02:16, 1.05it/s]
Loading 0: 61%|██████ | 221/363 [02:44<02:42, 1.14s/it]
Loading 0: 61%|██████ | 221/363 [02:44<02:42, 1.14s/it]
Loading 0: 63%|██████▎ | 229/363 [02:45<01:04, 2.08it/s]
Loading 0: 63%|██████▎ | 229/363 [02:45<01:04, 2.08it/s]
Loading 0: 64%|██████▍ | 232/363 [02:47<01:11, 1.82it/s]
Loading 0: 64%|██████▍ | 232/363 [02:47<01:11, 1.82it/s]
Loading 0: 64%|██████▍ | 233/363 [02:49<01:30, 1.43it/s]
Loading 0: 64%|██████▍ | 233/363 [02:49<01:30, 1.43it/s]
Loading 0: 64%|██████▍ | 234/363 [02:51<01:53, 1.14it/s]
Loading 0: 64%|██████▍ | 234/363 [02:51<01:53, 1.14it/s]
Loading 0: 65%|██████▌ | 237/363 [02:53<01:39, 1.27it/s]
Loading 0: 65%|██████▌ | 237/363 [02:53<01:39, 1.27it/s]
Loading 0: 66%|██████▌ | 238/363 [02:55<01:59, 1.05it/s]
Loading 0: 66%|██████▌ | 238/363 [02:55<01:59, 1.05it/s]
Loading 0: 66%|██████▌ | 239/363 [02:57<02:21, 1.14s/it]
Loading 0: 66%|██████▌ | 239/363 [02:57<02:21, 1.14s/it]
Loading 0: 68%|██████▊ | 247/363 [02:58<00:55, 2.08it/s]
Loading 0: 68%|██████▊ | 247/363 [02:58<00:55, 2.08it/s]
Loading 0: 69%|██████▉ | 250/363 [03:00<01:02, 1.82it/s]
Loading 0: 69%|██████▉ | 250/363 [03:00<01:02, 1.82it/s]
Loading 0: 69%|██████▉ | 251/363 [03:02<01:18, 1.43it/s]
Loading 0: 69%|██████▉ | 251/363 [03:02<01:18, 1.43it/s]
Loading 0: 69%|██████▉ | 252/363 [03:04<01:37, 1.14it/s]
Loading 0: 69%|██████▉ | 252/363 [03:04<01:37, 1.14it/s]
Loading 0: 70%|███████ | 255/363 [03:06<01:25, 1.27it/s]
Loading 0: 70%|███████ | 255/363 [03:06<01:25, 1.27it/s]
Loading 0: 71%|███████ | 256/363 [03:08<01:42, 1.05it/s]
Loading 0: 71%|███████ | 256/363 [03:08<01:42, 1.05it/s]
Loading 0: 71%|███████ | 257/363 [03:10<02:00, 1.14s/it]
Loading 0: 71%|███████ | 257/363 [03:10<02:00, 1.14s/it]
Loading 0: 73%|███████▎ | 265/363 [03:11<00:47, 2.08it/s]
Loading 0: 73%|███████▎ | 265/363 [03:11<00:47, 2.08it/s]
Loading 0: 74%|███████▍ | 268/363 [03:13<00:51, 1.83it/s]
Loading 0: 74%|███████▍ | 268/363 [03:13<00:51, 1.83it/s]
Loading 0: 74%|███████▍ | 269/363 [03:15<01:05, 1.43it/s]
Loading 0: 74%|███████▍ | 269/363 [03:15<01:05, 1.43it/s]
Loading 0: 74%|███████▍ | 270/363 [03:17<01:21, 1.14it/s]
Loading 0: 74%|███████▍ | 270/363 [03:17<01:21, 1.14it/s]
Loading 0: 75%|███████▌ | 273/363 [03:32<03:35, 2.39s/it]
Loading 0: 75%|███████▌ | 273/363 [03:32<03:35, 2.39s/it]
Loading 0: 75%|███████▌ | 274/363 [03:34<03:26, 2.32s/it]
Loading 0: 75%|███████▌ | 274/363 [03:34<03:26, 2.32s/it]
Loading 0: 76%|███████▌ | 275/363 [03:36<03:19, 2.27s/it]
Loading 0: 76%|███████▌ | 275/363 [03:36<03:19, 2.27s/it]
Loading 0: 78%|███████▊ | 283/363 [03:37<01:08, 1.17it/s]
Loading 0: 78%|███████▊ | 283/363 [03:37<01:08, 1.17it/s]
Loading 0: 79%|███████▉ | 286/363 [03:39<01:02, 1.23it/s]
Loading 0: 79%|███████▉ | 286/363 [03:39<01:02, 1.23it/s]
Loading 0: 79%|███████▉ | 287/363 [03:41<01:11, 1.07it/s]
Loading 0: 79%|███████▉ | 287/363 [03:41<01:11, 1.07it/s]
Loading 0: 79%|███████▉ | 288/363 [03:43<01:21, 1.08s/it]
Loading 0: 79%|███████▉ | 288/363 [03:43<01:21, 1.08s/it]
Loading 0: 80%|████████ | 291/363 [03:45<01:06, 1.09it/s]
Loading 0: 80%|████████ | 291/363 [03:45<01:06, 1.09it/s]
Loading 0: 80%|████████ | 292/363 [03:47<01:15, 1.07s/it]
Loading 0: 80%|████████ | 292/363 [03:47<01:15, 1.07s/it]
Loading 0: 81%|████████ | 293/363 [03:49<01:26, 1.23s/it]
Loading 0: 81%|████████ | 293/363 [03:49<01:26, 1.23s/it]
Loading 0: 83%|████████▎ | 301/363 [03:50<00:31, 1.99it/s]
Loading 0: 83%|████████▎ | 301/363 [03:50<00:31, 1.99it/s]
Loading 0: 84%|████████▎ | 304/363 [03:52<00:33, 1.77it/s]
Loading 0: 84%|████████▎ | 304/363 [03:52<00:33, 1.77it/s]
Loading 0: 84%|████████▍ | 305/363 [03:54<00:41, 1.40it/s]
Loading 0: 84%|████████▍ | 305/363 [03:54<00:41, 1.40it/s]
Loading 0: 84%|████████▍ | 306/363 [03:56<00:50, 1.12it/s]
Loading 0: 84%|████████▍ | 306/363 [03:56<00:50, 1.12it/s]
Loading 0: 85%|████████▌ | 309/363 [03:58<00:42, 1.26it/s]
Loading 0: 85%|████████▌ | 309/363 [03:58<00:42, 1.26it/s]
Loading 0: 85%|████████▌ | 310/363 [04:00<00:50, 1.04it/s]
Loading 0: 85%|████████▌ | 310/363 [04:00<00:50, 1.04it/s]
Loading 0: 86%|████████▌ | 311/363 [04:02<00:59, 1.14s/it]
Loading 0: 86%|████████▌ | 311/363 [04:02<00:59, 1.14s/it]
Loading 0: 88%|████████▊ | 319/363 [04:03<00:20, 2.11it/s]
Loading 0: 88%|████████▊ | 319/363 [04:03<00:20, 2.11it/s]
Loading 0: 89%|████████▊ | 322/363 [04:05<00:22, 1.83it/s]
Loading 0: 89%|████████▊ | 322/363 [04:05<00:22, 1.83it/s]
Loading 0: 89%|████████▉ | 323/363 [04:07<00:27, 1.43it/s]
Loading 0: 89%|████████▉ | 323/363 [04:07<00:27, 1.43it/s]
Loading 0: 89%|████████▉ | 324/363 [04:09<00:34, 1.14it/s]
Loading 0: 89%|████████▉ | 324/363 [04:09<00:34, 1.14it/s]
Loading 0: 90%|█████████ | 327/363 [04:11<00:28, 1.27it/s]
Loading 0: 90%|█████████ | 327/363 [04:11<00:28, 1.27it/s]
Loading 0: 90%|█████████ | 328/363 [04:13<00:33, 1.05it/s]
Loading 0: 90%|█████████ | 328/363 [04:13<00:33, 1.05it/s]
Loading 0: 91%|█████████ | 329/363 [04:15<00:38, 1.14s/it]
Loading 0: 91%|█████████ | 329/363 [04:15<00:38, 1.14s/it]
Loading 0: 93%|█████████▎| 337/363 [04:16<00:12, 2.11it/s]
Loading 0: 93%|█████████▎| 337/363 [04:16<00:12, 2.11it/s]
Loading 0: 94%|█████████▎| 340/363 [04:19<00:12, 1.84it/s]
Loading 0: 94%|█████████▎| 340/363 [04:19<00:12, 1.84it/s]
Loading 0: 94%|█████████▍| 341/363 [04:20<00:15, 1.44it/s]
Loading 0: 94%|█████████▍| 341/363 [04:20<00:15, 1.44it/s]
Loading 0: 94%|█████████▍| 342/363 [04:22<00:18, 1.14it/s]
Loading 0: 94%|█████████▍| 342/363 [04:22<00:18, 1.14it/s]
Loading 0: 95%|█████████▌| 345/363 [04:24<00:14, 1.27it/s]
Loading 0: 95%|█████████▌| 345/363 [04:24<00:14, 1.27it/s]
Loading 0: 95%|█████████▌| 346/363 [04:26<00:16, 1.05it/s]
Loading 0: 95%|█████████▌| 346/363 [04:26<00:16, 1.05it/s]
Loading 0: 96%|█████████▌| 347/363 [04:28<00:18, 1.14s/it]
Loading 0: 96%|█████████▌| 347/363 [04:28<00:18, 1.14s/it]
Loading 0: 98%|█████████▊| 355/363 [04:30<00:03, 2.10it/s]
Loading 0: 98%|█████████▊| 355/363 [04:30<00:03, 2.10it/s]
Loading 0: 99%|█████████▉| 359/363 [04:32<00:02, 1.89it/s]
Loading 0: 99%|█████████▉| 359/363 [04:32<00:02, 1.89it/s]
Loading 0: 99%|█████████▉| 360/363 [04:34<00:02, 1.49it/s]
Loading 0: 99%|█████████▉| 360/363 [04:34<00:02, 1.49it/s]
Loading 0: 99%|█████████▉| 361/363 [04:36<00:01, 1.19it/s]
Loading 0: 99%|█████████▉| 361/363 [04:36<00:01, 1.19it/s]
Loading 0: 100%|██████████| 363/363 [04:36<00:00, 1.19it/s]
Loading 0: 100%|██████████| 363/363 [04:36<00:00, 1.31it/s]
chaiml-mistral-24b-2048-82993-v1-mkmlizer: The tokenizer you are loading from '/tmp/tmptga7wpba' with an incorrect regex pattern: https://huggingface.co/mistralai/Mistral-Small-3.1-24B-Instruct-2503/discussions/84#69121093e8b480e709447d5e. This will lead to incorrect tokenization. You should set the `fix_mistral_regex=True` flag when loading this tokenizer to fix this issue.
chaiml-mistral-24b-2048-82993-v1-mkmlizer: quantized model in 285.027s
chaiml-mistral-24b-2048-90209-v1-mkmlizer: The tokenizer you are loading from '/tmp/tmpcm5lelcq' with an incorrect regex pattern: https://huggingface.co/mistralai/Mistral-Small-3.1-24B-Instruct-2503/discussions/84#69121093e8b480e709447d5e. This will lead to incorrect tokenization. You should set the `fix_mistral_regex=True` flag when loading this tokenizer to fix this issue.
chaiml-mistral-24b-2048-82993-v1-mkmlizer: Processed model ChaiML/mistral_24b_2048_gemini_ds_v6_3279_merged in 376.269s
chaiml-mistral-24b-2048-90209-v1-mkmlizer: quantized model in 282.998s
chaiml-mistral-24b-2048-82993-v1-mkmlizer: creating bucket guanaco-mkml-models
chaiml-mistral-24b-2048-90209-v1-mkmlizer: Processed model ChaiML/mistral_24b_2048_gemini_ds_v6_4372_merged in 387.299s
chaiml-mistral-24b-2048-82993-v1-mkmlizer: Bucket 's3://guanaco-mkml-models/' created
chaiml-mistral-24b-2048-90209-v1-mkmlizer: creating bucket guanaco-mkml-models
chaiml-mistral-24b-2048-82993-v1-mkmlizer: uploading /dev/shm/model_cache to s3://guanaco-mkml-models/chaiml-mistral-24b-2048-82993-v1/nvidia
chaiml-mistral-24b-2048-90209-v1-mkmlizer: Bucket 's3://guanaco-mkml-models/' created
chaiml-mistral-24b-2048-82993-v1-mkmlizer: cp /dev/shm/model_cache/config.json s3://guanaco-mkml-models/chaiml-mistral-24b-2048-82993-v1/nvidia/config.json
chaiml-mistral-24b-2048-90209-v1-mkmlizer: uploading /dev/shm/model_cache to s3://guanaco-mkml-models/chaiml-mistral-24b-2048-90209-v1/nvidia
chaiml-mistral-24b-2048-82993-v1-mkmlizer: cp /dev/shm/model_cache/special_tokens_map.json s3://guanaco-mkml-models/chaiml-mistral-24b-2048-82993-v1/nvidia/special_tokens_map.json
chaiml-mistral-24b-2048-90209-v1-mkmlizer: cp /dev/shm/model_cache/config.json s3://guanaco-mkml-models/chaiml-mistral-24b-2048-90209-v1/nvidia/config.json
chaiml-mistral-24b-2048-82993-v1-mkmlizer: cp /dev/shm/model_cache/tokenizer_config.json s3://guanaco-mkml-models/chaiml-mistral-24b-2048-82993-v1/nvidia/tokenizer_config.json
chaiml-mistral-24b-2048-90209-v1-mkmlizer: cp /dev/shm/model_cache/special_tokens_map.json s3://guanaco-mkml-models/chaiml-mistral-24b-2048-90209-v1/nvidia/special_tokens_map.json
chaiml-mistral-24b-2048-82993-v1-mkmlizer: cp /dev/shm/model_cache/tokenizer.json s3://guanaco-mkml-models/chaiml-mistral-24b-2048-82993-v1/nvidia/tokenizer.json
chaiml-mistral-24b-2048-90209-v1-mkmlizer: cp /dev/shm/model_cache/tokenizer_config.json s3://guanaco-mkml-models/chaiml-mistral-24b-2048-90209-v1/nvidia/tokenizer_config.json
chaiml-mistral-24b-2048-90209-v1-mkmlizer: cp /dev/shm/model_cache/tokenizer.json s3://guanaco-mkml-models/chaiml-mistral-24b-2048-90209-v1/nvidia/tokenizer.json
chaiml-mistral-24b-2048-4101-v1-mkmlizer:
Loading 0: 0%| | 0.00/363 [00:00<?, ?it/s]
Loading 0: 1%| | 3.00/363 [00:02<04:24, 1.36it/s]
Loading 0: 1%| | 3.00/363 [00:02<04:24, 1.36it/s]
Loading 0: 1%| | 4.00/363 [00:04<07:10, 1.20s/it]
Loading 0: 1%| | 4.00/363 [00:04<07:10, 1.20s/it]
Loading 0: 1%|▏ | 5.00/363 [00:06<09:15, 1.55s/it]
Loading 0: 1%|▏ | 5.00/363 [00:06<09:15, 1.55s/it]
Loading 0: 3%|▎ | 11.0/363 [00:07<03:14, 1.81it/s]
Loading 0: 3%|▎ | 11.0/363 [00:07<03:14, 1.81it/s]
Loading 0: 4%|▎ | 13.0/363 [00:10<04:12, 1.39it/s]
Loading 0: 4%|▎ | 13.0/363 [00:10<04:12, 1.39it/s]
Loading 0: 4%|▍ | 14.0/363 [00:12<05:30, 1.06it/s]
Loading 0: 4%|▍ | 14.0/363 [00:12<05:30, 1.06it/s]
Loading 0: 4%|▍ | 15.0/363 [00:14<06:55, 1.19s/it]
Loading 0: 4%|▍ | 15.0/363 [00:14<06:55, 1.19s/it]
Loading 0: 6%|▌ | 21.0/363 [00:17<04:13, 1.35it/s]
Loading 0: 6%|▌ | 21.0/363 [00:17<04:13, 1.35it/s]
Loading 0: 6%|▌ | 22.0/363 [00:19<05:13, 1.09it/s]
Loading 0: 6%|▌ | 22.0/363 [00:19<05:13, 1.09it/s]
Loading 0: 6%|▋ | 23.0/363 [00:21<06:25, 1.13s/it]
Loading 0: 6%|▋ | 23.0/363 [00:21<06:25, 1.13s/it]
Loading 0: 8%|▊ | 30.0/363 [00:22<03:00, 1.84it/s]
Loading 0: 8%|▊ | 30.0/363 [00:22<03:00, 1.84it/s]
Loading 0: 9%|▉ | 34.0/363 [00:25<03:15, 1.69it/s]
Loading 0: 9%|▉ | 34.0/363 [00:25<03:15, 1.69it/s]
Loading 0: 10%|▉ | 35.0/363 [00:27<04:09, 1.32it/s]
Loading 0: 10%|▉ | 35.0/363 [00:27<04:09, 1.32it/s]
Loading 0: 10%|▉ | 36.0/363 [00:30<05:14, 1.04it/s]
Loading 0: 10%|▉ | 36.0/363 [00:30<05:14, 1.04it/s]
Loading 0: 11%|█ | 39.0/363 [00:32<04:44, 1.14it/s]
Loading 0: 11%|█ | 39.0/363 [00:32<04:44, 1.14it/s]
Loading 0: 11%|█ | 40.0/363 [00:34<05:43, 1.06s/it]
Loading 0: 11%|█ | 40.0/363 [00:34<05:43, 1.06s/it]
Loading 0: 11%|█▏ | 41.0/363 [00:36<06:53, 1.28s/it]
Loading 0: 11%|█▏ | 41.0/363 [00:36<06:53, 1.28s/it]
Loading 0: 13%|█▎ | 48.0/363 [00:37<02:58, 1.76it/s]
Loading 0: 13%|█▎ | 48.0/363 [00:37<02:58, 1.76it/s]
Loading 0: 14%|█▍ | 52.0/363 [00:40<03:10, 1.63it/s]
Loading 0: 14%|█▍ | 52.0/363 [00:40<03:10, 1.63it/s]
Loading 0: 15%|█▍ | 53.0/363 [00:42<04:02, 1.28it/s]
Loading 0: 15%|█▍ | 53.0/363 [00:42<04:02, 1.28it/s]
Loading 0: 15%|█▍ | 54.0/363 [00:45<05:05, 1.01it/s]
Loading 0: 15%|█▍ | 54.0/363 [00:45<05:05, 1.01it/s]
Loading 0: 16%|█▌ | 57.0/363 [00:47<04:33, 1.12it/s]
Loading 0: 16%|█▌ | 57.0/363 [00:47<04:33, 1.12it/s]
Loading 0: 16%|█▌ | 58.0/363 [00:49<05:31, 1.09s/it]
Loading 0: 16%|█▌ | 58.0/363 [00:49<05:31, 1.09s/it]
Loading 0: 16%|█▋ | 59.0/363 [00:52<06:43, 1.33s/it]
Loading 0: 16%|█▋ | 59.0/363 [00:52<06:43, 1.33s/it]
Loading 0: 18%|█▊ | 66.0/363 [00:53<02:53, 1.71it/s]
Loading 0: 18%|█▊ | 66.0/363 [00:53<02:53, 1.71it/s]
Loading 0: 19%|█▉ | 70.0/363 [00:56<03:03, 1.60it/s]
Loading 0: 19%|█▉ | 70.0/363 [00:56<03:03, 1.60it/s]
Loading 0: 20%|█▉ | 71.0/363 [00:58<03:51, 1.26it/s]
Loading 0: 20%|█▉ | 71.0/363 [00:58<03:51, 1.26it/s]
Loading 0: 20%|█▉ | 72.0/363 [01:00<04:49, 1.01it/s]
Loading 0: 20%|█▉ | 72.0/363 [01:00<04:49, 1.01it/s]
Loading 0: 21%|██ | 75.0/363 [01:02<04:18, 1.11it/s]
Loading 0: 21%|██ | 75.0/363 [01:02<04:18, 1.11it/s]
Loading 0: 21%|██ | 76.0/363 [01:04<05:11, 1.09s/it]
Loading 0: 21%|██ | 76.0/363 [01:04<05:11, 1.09s/it]
Loading 0: 21%|██ | 77.0/363 [01:07<06:12, 1.30s/it]
Loading 0: 21%|██ | 77.0/363 [01:07<06:12, 1.30s/it]
Loading 0: 23%|██▎ | 84.0/363 [01:08<02:41, 1.72it/s]
Loading 0: 23%|██▎ | 84.0/363 [01:08<02:41, 1.72it/s]
Loading 0: 24%|██▍ | 88.0/363 [01:11<02:51, 1.61it/s]
Loading 0: 24%|██▍ | 88.0/363 [01:11<02:51, 1.61it/s]
Loading 0: 25%|██▍ | 89.0/363 [01:13<03:36, 1.27it/s]
Loading 0: 25%|██▍ | 89.0/363 [01:13<03:36, 1.27it/s]
Loading 0: 25%|██▍ | 90.0/363 [01:15<04:32, 1.00it/s]
Loading 0: 25%|██▍ | 90.0/363 [01:15<04:32, 1.00it/s]
Loading 0: 27%|██▋ | 97.0/363 [01:16<02:17, 1.93it/s]
Loading 0: 27%|██▋ | 97.0/363 [01:16<02:17, 1.93it/s]
Loading 0: 28%|██▊ | 101/363 [01:19<02:27, 1.78it/s]
Loading 0: 28%|██▊ | 101/363 [01:19<02:27, 1.78it/s]
Loading 0: 28%|██▊ | 102/363 [01:21<03:10, 1.37it/s]
Loading 0: 28%|██▊ | 102/363 [01:21<03:10, 1.37it/s]
Loading 0: 28%|██▊ | 103/363 [01:23<04:02, 1.07it/s]
Loading 0: 28%|██▊ | 103/363 [01:23<04:02, 1.07it/s]
Loading 0: 29%|██▉ | 106/363 [01:26<03:46, 1.13it/s]
Loading 0: 29%|██▉ | 106/363 [01:26<03:46, 1.13it/s]
Loading 0: 29%|██▉ | 107/363 [01:28<04:33, 1.07s/it]
Loading 0: 29%|██▉ | 107/363 [01:28<04:33, 1.07s/it]
Loading 0: 30%|██▉ | 108/363 [01:30<05:26, 1.28s/it]
Loading 0: 30%|██▉ | 108/363 [01:30<05:26, 1.28s/it]
Loading 0: 31%|███ | 111/363 [01:32<04:23, 1.05s/it]
Loading 0: 31%|███ | 111/363 [01:32<04:23, 1.05s/it]
Loading 0: 31%|███ | 112/363 [01:35<05:10, 1.24s/it]
Loading 0: 31%|███ | 112/363 [01:35<05:10, 1.24s/it]
Loading 0: 31%|███ | 113/363 [01:37<06:00, 1.44s/it]
Loading 0: 31%|███ | 113/363 [01:37<06:00, 1.44s/it]
Loading 0: 33%|███▎ | 120/363 [01:38<02:25, 1.67it/s]
Loading 0: 33%|███▎ | 120/363 [01:38<02:25, 1.67it/s]
Loading 0: 34%|███▍ | 124/363 [01:41<02:31, 1.57it/s]
Loading 0: 34%|███▍ | 124/363 [01:41<02:31, 1.57it/s]
Loading 0: 34%|███▍ | 125/363 [01:43<03:11, 1.24it/s]
Loading 0: 34%|███▍ | 125/363 [01:43<03:11, 1.24it/s]
Loading 0: 35%|███▍ | 126/363 [01:45<03:59, 1.01s/it]
Loading 0: 35%|███▍ | 126/363 [01:45<03:59, 1.01s/it]
Loading 0: 36%|███▌ | 129/363 [01:48<03:32, 1.10it/s]
Loading 0: 36%|███▌ | 129/363 [01:48<03:32, 1.10it/s]
Loading 0: 36%|███▌ | 130/363 [01:50<04:15, 1.10s/it]
Loading 0: 36%|███▌ | 130/363 [01:50<04:15, 1.10s/it]
Loading 0: 36%|███▌ | 131/363 [01:52<05:05, 1.32s/it]
Loading 0: 36%|███▌ | 131/363 [01:52<05:05, 1.32s/it]
Loading 0: 38%|███▊ | 138/363 [01:53<02:09, 1.74it/s]
Loading 0: 38%|███▊ | 138/363 [01:53<02:09, 1.74it/s]
Loading 0: 39%|███▉ | 142/363 [01:56<02:16, 1.62it/s]
Loading 0: 39%|███▉ | 142/363 [01:56<02:16, 1.62it/s]
Loading 0: 39%|███▉ | 143/363 [01:58<02:52, 1.27it/s]
Loading 0: 39%|███▉ | 143/363 [01:58<02:52, 1.27it/s]
Loading 0: 40%|███▉ | 144/363 [02:00<03:37, 1.01it/s]
Loading 0: 40%|███▉ | 144/363 [02:00<03:37, 1.01it/s]
Loading 0: 40%|████ | 147/363 [02:03<03:13, 1.12it/s]
Loading 0: 40%|████ | 147/363 [02:03<03:13, 1.12it/s]
Loading 0: 41%|████ | 148/363 [02:05<03:53, 1.09s/it]
Loading 0: 41%|████ | 148/363 [02:05<03:53, 1.09s/it]
Loading 0: 41%|████ | 149/363 [02:07<04:37, 1.30s/it]
Loading 0: 41%|████ | 149/363 [02:07<04:37, 1.30s/it]
Loading 0: 43%|████▎ | 156/363 [02:08<01:58, 1.75it/s]
Loading 0: 43%|████▎ | 156/363 [02:08<01:58, 1.75it/s]
Loading 0: 44%|████▍ | 160/363 [02:11<02:04, 1.62it/s]
Loading 0: 44%|████▍ | 160/363 [02:11<02:04, 1.62it/s]
Loading 0: 44%|████▍ | 161/363 [02:13<02:38, 1.27it/s]
Loading 0: 44%|████▍ | 161/363 [02:13<02:38, 1.27it/s]
Loading 0: 45%|████▍ | 162/363 [02:15<03:18, 1.01it/s]
Loading 0: 45%|████▍ | 162/363 [02:15<03:18, 1.01it/s]
Loading 0: 45%|████▌ | 165/363 [02:18<02:57, 1.12it/s]
Loading 0: 45%|████▌ | 165/363 [02:18<02:57, 1.12it/s]
Loading 0: 46%|████▌ | 166/363 [02:20<03:33, 1.08s/it]
Loading 0: 46%|████▌ | 166/363 [02:20<03:33, 1.08s/it]
Loading 0: 46%|████▌ | 167/363 [02:22<04:15, 1.30s/it]
Loading 0: 46%|████▌ | 167/363 [02:22<04:15, 1.30s/it]
Loading 0: 48%|████▊ | 174/363 [02:23<01:47, 1.75it/s]
Loading 0: 48%|████▊ | 174/363 [02:23<01:47, 1.75it/s]
Loading 0: 49%|████▉ | 178/363 [02:26<01:53, 1.63it/s]
Loading 0: 49%|████▉ | 178/363 [02:26<01:53, 1.63it/s]
Loading 0: 49%|████▉ | 179/363 [02:28<02:24, 1.27it/s]
Loading 0: 49%|████▉ | 179/363 [02:28<02:24, 1.27it/s]
Loading 0: 50%|████▉ | 180/363 [02:30<03:00, 1.01it/s]
Loading 0: 50%|████▉ | 180/363 [02:30<03:00, 1.01it/s]
Loading 0: 50%|█████ | 183/363 [02:33<02:40, 1.12it/s]
Loading 0: 50%|█████ | 183/363 [02:33<02:40, 1.12it/s]
Loading 0: 51%|█████ | 184/363 [02:35<03:13, 1.08s/it]
Loading 0: 51%|█████ | 184/363 [02:35<03:13, 1.08s/it]
Loading 0: 51%|█████ | 185/363 [02:37<03:50, 1.30s/it]
Loading 0: 51%|█████ | 185/363 [02:37<03:50, 1.30s/it]
Loading 0: 53%|█████▎ | 192/363 [02:38<01:37, 1.75it/s]
Loading 0: 53%|█████▎ | 192/363 [02:38<01:37, 1.75it/s]
Loading 0: 54%|█████▍ | 196/363 [02:41<01:42, 1.63it/s]
Loading 0: 54%|█████▍ | 196/363 [02:41<01:42, 1.63it/s]
Loading 0: 54%|█████▍ | 197/363 [02:43<02:10, 1.28it/s]
Loading 0: 54%|█████▍ | 197/363 [02:43<02:10, 1.28it/s]
Loading 0: 55%|█████▍ | 198/363 [02:45<02:43, 1.01it/s]
Loading 0: 55%|█████▍ | 198/363 [02:45<02:43, 1.01it/s]
Loading 0: 55%|█████▌ | 201/363 [02:48<02:24, 1.12it/s]
Loading 0: 55%|█████▌ | 201/363 [02:48<02:24, 1.12it/s]
Loading 0: 56%|█████▌ | 202/363 [02:50<02:54, 1.08s/it]
Loading 0: 56%|█████▌ | 202/363 [02:50<02:54, 1.08s/it]
Loading 0: 56%|█████▌ | 203/363 [02:53<03:39, 1.37s/it]
Loading 0: 56%|█████▌ | 203/363 [02:53<03:39, 1.37s/it]
Loading 0: 58%|█████▊ | 210/363 [02:54<01:32, 1.66it/s]
Loading 0: 58%|█████▊ | 210/363 [02:54<01:32, 1.66it/s]
Loading 0: 59%|█████▉ | 214/363 [02:57<01:35, 1.57it/s]
Loading 0: 59%|█████▉ | 214/363 [02:57<01:35, 1.57it/s]
Loading 0: 59%|█████▉ | 215/363 [02:59<01:59, 1.24it/s]
Loading 0: 59%|█████▉ | 215/363 [02:59<01:59, 1.24it/s]
Loading 0: 60%|█████▉ | 216/363 [03:01<02:28, 1.01s/it]
Loading 0: 60%|█████▉ | 216/363 [03:01<02:28, 1.01s/it]
Loading 0: 60%|██████ | 219/363 [03:03<02:10, 1.10it/s]
Loading 0: 60%|██████ | 219/363 [03:03<02:10, 1.10it/s]
Loading 0: 61%|██████ | 220/363 [03:05<02:36, 1.09s/it]
Loading 0: 61%|██████ | 220/363 [03:05<02:36, 1.09s/it]
Loading 0: 61%|██████ | 221/363 [03:08<03:17, 1.39s/it]
Loading 0: 61%|██████ | 221/363 [03:08<03:17, 1.39s/it]
Loading 0: 63%|██████▎ | 228/363 [03:09<01:21, 1.65it/s]
Loading 0: 63%|██████▎ | 228/363 [03:09<01:21, 1.65it/s]
Loading 0: 64%|██████▍ | 232/363 [03:12<01:23, 1.57it/s]
Loading 0: 64%|██████▍ | 232/363 [03:12<01:23, 1.57it/s]
Loading 0: 64%|██████▍ | 233/363 [03:14<01:44, 1.24it/s]
Loading 0: 64%|██████▍ | 233/363 [03:14<01:44, 1.24it/s]
Loading 0: 64%|██████▍ | 234/363 [03:17<02:10, 1.01s/it]
Loading 0: 64%|██████▍ | 234/363 [03:17<02:10, 1.01s/it]
Loading 0: 65%|██████▌ | 237/363 [03:19<01:54, 1.10it/s]
Loading 0: 65%|██████▌ | 237/363 [03:19<01:54, 1.10it/s]
Loading 0: 66%|██████▌ | 238/363 [03:21<02:16, 1.09s/it]
Loading 0: 66%|██████▌ | 238/363 [03:21<02:16, 1.09s/it]
Loading 0: 66%|██████▌ | 239/363 [03:23<02:41, 1.30s/it]
Loading 0: 66%|██████▌ | 239/363 [03:23<02:41, 1.30s/it]
Loading 0: 68%|██████▊ | 246/363 [03:24<01:07, 1.73it/s]
Loading 0: 68%|██████▊ | 246/363 [03:24<01:07, 1.73it/s]
Loading 0: 69%|██████▉ | 250/363 [03:27<01:10, 1.61it/s]
Loading 0: 69%|██████▉ | 250/363 [03:27<01:10, 1.61it/s]
Loading 0: 69%|██████▉ | 251/363 [03:29<01:28, 1.27it/s]
Loading 0: 69%|██████▉ | 251/363 [03:29<01:28, 1.27it/s]
Loading 0: 69%|██████▉ | 252/363 [03:32<01:50, 1.01it/s]
Loading 0: 69%|██████▉ | 252/363 [03:32<01:50, 1.01it/s]
Loading 0: 70%|███████ | 255/363 [03:34<01:36, 1.12it/s]
Loading 0: 70%|███████ | 255/363 [03:34<01:36, 1.12it/s]
Loading 0: 71%|███████ | 256/363 [03:36<01:56, 1.09s/it]
Loading 0: 71%|███████ | 256/363 [03:36<01:56, 1.09s/it]
Loading 0: 71%|███████ | 257/363 [03:38<02:17, 1.30s/it]
Loading 0: 71%|███████ | 257/363 [03:38<02:17, 1.30s/it]
Loading 0: 73%|███████▎ | 264/363 [03:39<00:56, 1.75it/s]
Loading 0: 73%|███████▎ | 264/363 [03:39<00:56, 1.75it/s]
Loading 0: 74%|███████▍ | 268/363 [03:42<00:58, 1.63it/s]
Loading 0: 74%|███████▍ | 268/363 [03:42<00:58, 1.63it/s]
Loading 0: 74%|███████▍ | 269/363 [03:44<01:13, 1.28it/s]
Loading 0: 74%|███████▍ | 269/363 [03:44<01:13, 1.28it/s]
Loading 0: 74%|███████▍ | 270/363 [03:47<01:31, 1.01it/s]
Loading 0: 74%|███████▍ | 270/363 [03:47<01:31, 1.01it/s]
Loading 0: 75%|███████▌ | 273/363 [04:03<03:56, 2.62s/it]
Loading 0: 75%|███████▌ | 273/363 [04:03<03:56, 2.62s/it]
Loading 0: 75%|███████▌ | 274/363 [04:05<03:46, 2.54s/it]
Loading 0: 75%|███████▌ | 274/363 [04:05<03:46, 2.54s/it]
Loading 0: 76%|███████▌ | 275/363 [04:08<03:42, 2.53s/it]
Loading 0: 76%|███████▌ | 275/363 [04:08<03:42, 2.53s/it]
Loading 0: 78%|███████▊ | 282/363 [04:09<01:22, 1.02s/it]
Loading 0: 78%|███████▊ | 282/363 [04:09<01:22, 1.02s/it]
Loading 0: 79%|███████▉ | 286/363 [04:11<01:10, 1.10it/s]
Loading 0: 79%|███████▉ | 286/363 [04:11<01:10, 1.10it/s]
Loading 0: 79%|███████▉ | 287/363 [04:14<01:19, 1.05s/it]
Loading 0: 79%|███████▉ | 287/363 [04:14<01:19, 1.05s/it]
Loading 0: 79%|███████▉ | 288/363 [04:16<01:31, 1.22s/it]
Loading 0: 79%|███████▉ | 288/363 [04:16<01:31, 1.22s/it]
Loading 0: 80%|████████ | 291/363 [04:18<01:14, 1.04s/it]
Loading 0: 80%|████████ | 291/363 [04:18<01:14, 1.04s/it]
Loading 0: 80%|████████ | 292/363 [04:20<01:25, 1.21s/it]
Loading 0: 80%|████████ | 292/363 [04:20<01:25, 1.21s/it]
Loading 0: 81%|████████ | 293/363 [04:23<01:38, 1.40s/it]
Loading 0: 81%|████████ | 293/363 [04:23<01:38, 1.40s/it]
Loading 0: 83%|████████▎ | 300/363 [04:24<00:38, 1.65it/s]
Loading 0: 83%|████████▎ | 300/363 [04:24<00:38, 1.65it/s]
Loading 0: 84%|████████▎ | 304/363 [04:26<00:37, 1.57it/s]
Loading 0: 84%|████████▎ | 304/363 [04:26<00:37, 1.57it/s]
Loading 0: 84%|████████▍ | 305/363 [04:29<00:46, 1.25it/s]
Loading 0: 84%|████████▍ | 305/363 [04:29<00:46, 1.25it/s]
Loading 0: 84%|████████▍ | 306/363 [04:31<00:57, 1.00s/it]
Loading 0: 84%|████████▍ | 306/363 [04:31<00:57, 1.00s/it]
Loading 0: 85%|████████▌ | 309/363 [04:33<00:48, 1.10it/s]
Loading 0: 85%|████████▌ | 309/363 [04:33<00:48, 1.10it/s]
Loading 0: 85%|████████▌ | 310/363 [04:35<00:58, 1.09s/it]
Loading 0: 85%|████████▌ | 310/363 [04:35<00:58, 1.09s/it]
Loading 0: 86%|████████▌ | 311/363 [04:38<01:09, 1.33s/it]
Loading 0: 86%|████████▌ | 311/363 [04:38<01:09, 1.33s/it]
Loading 0: 88%|████████▊ | 318/363 [04:39<00:26, 1.72it/s]
Loading 0: 88%|████████▊ | 318/363 [04:39<00:26, 1.72it/s]
Loading 0: 89%|████████▊ | 322/363 [04:42<00:25, 1.61it/s]
Loading 0: 89%|████████▊ | 322/363 [04:42<00:25, 1.61it/s]
Loading 0: 89%|████████▉ | 323/363 [04:44<00:31, 1.27it/s]
Loading 0: 89%|████████▉ | 323/363 [04:44<00:31, 1.27it/s]
Loading 0: 89%|████████▉ | 324/363 [04:46<00:38, 1.01it/s]
Loading 0: 89%|████████▉ | 324/363 [04:46<00:38, 1.01it/s]
Loading 0: 90%|█████████ | 327/363 [04:48<00:32, 1.11it/s]
Loading 0: 90%|█████████ | 327/363 [04:48<00:32, 1.11it/s]
Loading 0: 90%|█████████ | 328/363 [04:50<00:38, 1.09s/it]
Loading 0: 90%|█████████ | 328/363 [04:50<00:38, 1.09s/it]
Loading 0: 91%|█████████ | 329/363 [04:53<00:44, 1.30s/it]
Loading 0: 91%|█████████ | 329/363 [04:53<00:44, 1.30s/it]
Loading 0: 93%|█████████▎| 336/363 [04:54<00:15, 1.75it/s]
Loading 0: 93%|█████████▎| 336/363 [04:54<00:15, 1.75it/s]
Loading 0: 94%|█████████▎| 340/363 [04:57<00:14, 1.62it/s]
Loading 0: 94%|█████████▎| 340/363 [04:57<00:14, 1.62it/s]
Loading 0: 94%|█████████▍| 341/363 [04:59<00:17, 1.27it/s]
Loading 0: 94%|█████████▍| 341/363 [04:59<00:17, 1.27it/s]
Loading 0: 94%|█████████▍| 342/363 [05:01<00:20, 1.00it/s]
Loading 0: 94%|█████████▍| 342/363 [05:01<00:20, 1.00it/s]
Loading 0: 95%|█████████▌| 345/363 [05:03<00:16, 1.11it/s]
Loading 0: 95%|█████████▌| 345/363 [05:03<00:16, 1.11it/s]
Loading 0: 95%|█████████▌| 346/363 [05:06<00:18, 1.09s/it]
Loading 0: 95%|█████████▌| 346/363 [05:06<00:18, 1.09s/it]
Loading 0: 96%|█████████▌| 347/363 [05:08<00:20, 1.31s/it]
Loading 0: 96%|█████████▌| 347/363 [05:08<00:20, 1.31s/it]
Loading 0: 98%|█████████▊| 354/363 [05:09<00:05, 1.73it/s]
Loading 0: 98%|█████████▊| 354/363 [05:09<00:05, 1.73it/s]
Loading 0: 98%|█████████▊| 357/363 [05:10<00:03, 1.96it/s]
Loading 0: 98%|█████████▊| 357/363 [05:10<00:03, 1.96it/s]
Loading 0: 99%|█████████▉| 359/363 [05:12<00:02, 1.57it/s]
Loading 0: 99%|█████████▉| 359/363 [05:12<00:02, 1.57it/s]
Loading 0: 99%|█████████▉| 360/363 [05:15<00:02, 1.18it/s]
Loading 0: 99%|█████████▉| 360/363 [05:15<00:02, 1.18it/s]
Loading 0: 99%|█████████▉| 361/363 [05:17<00:02, 1.08s/it]
Loading 0: 99%|█████████▉| 361/363 [05:17<00:02, 1.08s/it]
Loading 0: 100%|██████████| 363/363 [05:17<00:00, 1.31it/s]
Loading 0: 100%|██████████| 363/363 [05:17<00:00, 1.31it/s]
Loading 0: 100%|██████████| 363/363 [05:17<00:00, 1.14it/s]
chaiml-mistral-24b-2048-46891-v1-mkmlizer: Downloaded to shared memory in 65.177s
chaiml-mistral-24b-2048-46891-v1-mkmlizer: Checking if ChaiML/mistral_24b_2048_gemini_ds_v7_3279_merged already exists in ChaiML
chaiml-mistral-24b-2048-46891-v1-mkmlizer: quantizing model to /dev/shm/model_cache, profile:q4, folder:/tmp/tmp374nfmpc, device:0
chaiml-mistral-24b-2048-46891-v1-mkmlizer: Saving flywheel model at /dev/shm/model_cache
chaiml-mistral-24b-2048-4101-v1-mkmlizer: The tokenizer you are loading from '/tmp/tmppkcws2yz' with an incorrect regex pattern: https://huggingface.co/mistralai/Mistral-Small-3.1-24B-Instruct-2503/discussions/84#69121093e8b480e709447d5e. This will lead to incorrect tokenization. You should set the `fix_mistral_regex=True` flag when loading this tokenizer to fix this issue.
chaiml-mistral-24b-2048-4101-v1-mkmlizer: quantized model in 324.820s
chaiml-mistral-24b-2048-4101-v1-mkmlizer: Processed model ChaiML/mistral_24b_2048_gemini_ds_v7_4372_merged in 414.731s
chaiml-mistral-24b-2048-4101-v1-mkmlizer: creating bucket guanaco-mkml-models
chaiml-mistral-24b-2048-4101-v1-mkmlizer: Bucket 's3://guanaco-mkml-models/' created
chaiml-mistral-24b-2048-4101-v1-mkmlizer: uploading /dev/shm/model_cache to s3://guanaco-mkml-models/chaiml-mistral-24b-2048-4101-v1/nvidia
chaiml-mistral-24b-2048-4101-v1-mkmlizer: cp /dev/shm/model_cache/config.json s3://guanaco-mkml-models/chaiml-mistral-24b-2048-4101-v1/nvidia/config.json
chaiml-mistral-24b-2048-4101-v1-mkmlizer: cp /dev/shm/model_cache/special_tokens_map.json s3://guanaco-mkml-models/chaiml-mistral-24b-2048-4101-v1/nvidia/special_tokens_map.json
chaiml-mistral-24b-2048-4101-v1-mkmlizer: cp /dev/shm/model_cache/tokenizer_config.json s3://guanaco-mkml-models/chaiml-mistral-24b-2048-4101-v1/nvidia/tokenizer_config.json
chaiml-mistral-24b-2048-4101-v1-mkmlizer: cp /dev/shm/model_cache/tokenizer.json s3://guanaco-mkml-models/chaiml-mistral-24b-2048-4101-v1/nvidia/tokenizer.json
chaiml-mistral-24b-2048-82993-v1-mkmlizer: cp /dev/shm/model_cache/flywheel_model.1.safetensors s3://guanaco-mkml-models/chaiml-mistral-24b-2048-82993-v1/nvidia/flywheel_model.1.safetensors
chaiml-mistral-24b-2048-90209-v1-mkmlizer: cp /dev/shm/model_cache/flywheel_model.1.safetensors s3://guanaco-mkml-models/chaiml-mistral-24b-2048-90209-v1/nvidia/flywheel_model.1.safetensors
chaiml-mistral-24b-2048-4101-v1-mkmlizer: cp /dev/shm/model_cache/flywheel_model.1.safetensors s3://guanaco-mkml-models/chaiml-mistral-24b-2048-4101-v1/nvidia/flywheel_model.1.safetensors
chaiml-mistral-24b-2048-82993-v1-mkmlizer: cp /dev/shm/model_cache/flywheel_model.0.safetensors s3://guanaco-mkml-models/chaiml-mistral-24b-2048-82993-v1/nvidia/flywheel_model.0.safetensors
chaiml-mistral-24b-2048-90209-v1-mkmlizer: cp /dev/shm/model_cache/flywheel_model.0.safetensors s3://guanaco-mkml-models/chaiml-mistral-24b-2048-90209-v1/nvidia/flywheel_model.0.safetensors
Job chaiml-mistral-24b-2048-82993-v1-mkmlizer completed after 500.83s with status: succeeded
Job chaiml-mistral-24b-2048-90209-v1-mkmlizer completed after 510.58s with status: succeeded
Stopping job with name chaiml-mistral-24b-2048-82993-v1-mkmlizer
Stopping job with name chaiml-mistral-24b-2048-90209-v1-mkmlizer
Pipeline stage MKMLizer completed in 503.60s
Pipeline stage MKMLizer completed in 512.94s
run pipeline stage %s
run pipeline stage %s
Running pipeline stage MKMLTemplater
Running pipeline stage MKMLTemplater
Pipeline stage MKMLTemplater completed in 0.79s
run pipeline stage %s
Pipeline stage MKMLTemplater completed in 0.95s
Running pipeline stage MKMLDeployer
run pipeline stage %s
Creating inference service chaiml-mistral-24b-2048-82993-v1
Running pipeline stage MKMLDeployer
Waiting for inference service chaiml-mistral-24b-2048-82993-v1 to be ready
Creating inference service chaiml-mistral-24b-2048-90209-v1
Waiting for inference service chaiml-mistral-24b-2048-90209-v1 to be ready
chaiml-mistral-24b-2048-4101-v1-mkmlizer: cp /dev/shm/model_cache/flywheel_model.0.safetensors s3://guanaco-mkml-models/chaiml-mistral-24b-2048-4101-v1/nvidia/flywheel_model.0.safetensors
Job chaiml-mistral-24b-2048-4101-v1-mkmlizer completed after 543.3s with status: succeeded
Stopping job with name chaiml-mistral-24b-2048-4101-v1-mkmlizer
Pipeline stage MKMLizer completed in 545.76s
run pipeline stage %s
Running pipeline stage MKMLTemplater
Pipeline stage MKMLTemplater completed in 0.70s
run pipeline stage %s
Running pipeline stage MKMLDeployer
Creating inference service chaiml-mistral-24b-2048-4101-v1
Waiting for inference service chaiml-mistral-24b-2048-4101-v1 to be ready
Inference service chaiml-mistral-24b-2048-82993-v1 ready after 152.0974416732788s
Pipeline stage MKMLDeployer completed in 154.76s
run pipeline stage %s
Inference service chaiml-mistral-24b-2048-90209-v1 ready after 152.30769038200378s
Running pipeline stage StressChecker
Pipeline stage MKMLDeployer completed in 155.06s
run pipeline stage %s
Running pipeline stage StressChecker
Received healthy response to inference request in 4.425960302352905s
Received healthy response to inference request in 3.8707096576690674s
Received healthy response to inference request in 3.713087320327759s
Received healthy response to inference request in 4.110254764556885s
Received healthy response to inference request in 3.2727792263031006s
Received healthy response to inference request in 4.138288259506226s
Received healthy response to inference request in 3.443653106689453s
Received healthy response to inference request in 3.6324329376220703s
Received healthy response to inference request in 3.2204997539520264s
5 requests
Received healthy response to inference request in 3.7962965965270996s
0 failed requests
5 requests
5th percentile: 3.230955648422241
0 failed requests
10th percentile: 3.241411542892456
5th percentile: 3.648563814163208
20th percentile: 3.262323331832886
10th percentile: 3.6646946907043456
30th percentile: 3.306954002380371
20th percentile: 3.696956443786621
40th percentile: 3.375303554534912
30th percentile: 3.729729175567627
Inference service chaiml-mistral-24b-2048-4101-v1 ready after 152.14980149269104s
50th percentile: 3.443653106689453
40th percentile: 3.7630128860473633
Pipeline stage MKMLDeployer completed in 154.89s
60th percentile: 3.614475727081299
50th percentile: 3.7962965965270996
run pipeline stage %s
70th percentile: 3.7852983474731445
60th percentile: 3.93309326171875
Running pipeline stage StressChecker
80th percentile: 3.918618679046631
70th percentile: 4.069889926910401
90th percentile: 4.014436721801758
80th percentile: 4.195822668075562
95th percentile: 4.062345743179321
90th percentile: 4.310891485214233
99th percentile: 4.100672960281372
95th percentile: 4.368425893783569
mean time: 3.5835793018341064
99th percentile: 4.414453420639038
Pipeline stage StressChecker completed in 31.05s
mean time: 3.941213083267212
Received healthy response to inference request in 3.7713592052459717s
run pipeline stage %s
%s, retrying in %s seconds...
Running pipeline stage OfflineFamilyFriendlyTriggerPipeline
run_pipeline:run_in_cloud %s
starting trigger_guanaco_pipeline args=%s
Received healthy response to inference request in 3.4062376022338867s
triggered trigger_guanaco_pipeline args=%s
Received healthy response to inference request in 3.699669361114502s
Pipeline stage OfflineFamilyFriendlyTriggerPipeline completed in 4.71s
run pipeline stage %s
Running pipeline stage TriggerMKMLProfilingPipeline
run_pipeline:run_in_cloud %s
starting trigger_guanaco_pipeline args=%s
Received healthy response to inference request in 3.711951494216919s
Received healthy response to inference request in 3.3120851516723633s
triggered trigger_guanaco_pipeline args=%s
Pipeline stage TriggerMKMLProfilingPipeline completed in 2.95s
Shutdown handler de-registered
chaiml-mistral-24b-2048_90209_v1 status is now deployed due to DeploymentManager action
Received healthy response to inference request in 3.573206663131714s
Received healthy response to inference request in 3.35355806350708s
Received healthy response to inference request in 3.6945862770080566s
chaiml-mistral-24b-2048-46891-v1-mkmlizer:
Loading 0: 0%| | 0.00/363 [00:00<?, ?it/s]
Loading 0: 1%| | 3.00/363 [00:01<03:50, 1.56it/s]
Loading 0: 1%| | 3.00/363 [00:01<03:50, 1.56it/s]
Loading 0: 1%| | 4.00/363 [00:03<06:19, 1.06s/it]
Loading 0: 1%| | 4.00/363 [00:03<06:19, 1.06s/it]
Loading 0: 1%|▏ | 5.00/363 [00:05<08:04, 1.35s/it]
Loading 0: 1%|▏ | 5.00/363 [00:05<08:04, 1.35s/it]
Loading 0: 3%|▎ | 12.0/363 [00:06<02:36, 2.24it/s]
Loading 0: 3%|▎ | 12.0/363 [00:06<02:36, 2.24it/s]
Loading 0: 4%|▎ | 13.0/363 [00:08<03:44, 1.56it/s]
Loading 0: 4%|▎ | 13.0/363 [00:08<03:44, 1.56it/s]
Loading 0: 4%|▍ | 14.0/363 [00:10<04:53, 1.19it/s]
Loading 0: 4%|▍ | 14.0/363 [00:10<04:53, 1.19it/s]
Loading 0: 4%|▍ | 15.0/363 [00:12<06:08, 1.06s/it]
Loading 0: 4%|▍ | 15.0/363 [00:12<06:08, 1.06s/it]
Loading 0: 6%|▌ | 21.0/363 [00:15<03:41, 1.55it/s]
Loading 0: 6%|▌ | 21.0/363 [00:15<03:41, 1.55it/s]
Loading 0: 6%|▌ | 22.0/363 [00:16<04:34, 1.24it/s]
Loading 0: 6%|▌ | 22.0/363 [00:16<04:34, 1.24it/s]
Loading 0: 6%|▋ | 23.0/363 [00:18<05:37, 1.01it/s]
Loading 0: 6%|▋ | 23.0/363 [00:18<05:37, 1.01it/s]
Loading 0: 9%|▊ | 31.0/363 [00:20<02:30, 2.20it/s]
Loading 0: 9%|▊ | 31.0/363 [00:20<02:30, 2.20it/s]
Loading 0: 9%|▉ | 34.0/363 [00:22<02:52, 1.91it/s]
Loading 0: 9%|▉ | 34.0/363 [00:22<02:52, 1.91it/s]
Loading 0: 10%|▉ | 35.0/363 [00:24<03:40, 1.49it/s]
Loading 0: 10%|▉ | 35.0/363 [00:24<03:40, 1.49it/s]
Loading 0: 10%|▉ | 36.0/363 [00:26<04:37, 1.18it/s]
Loading 0: 10%|▉ | 36.0/363 [00:26<04:37, 1.18it/s]
Loading 0: 11%|█ | 39.0/363 [00:28<04:09, 1.30it/s]
Loading 0: 11%|█ | 39.0/363 [00:28<04:09, 1.30it/s]
Loading 0: 11%|█ | 40.0/363 [00:30<05:02, 1.07it/s]
Loading 0: 11%|█ | 40.0/363 [00:30<05:02, 1.07it/s]
Loading 0: 11%|█▏ | 41.0/363 [00:32<06:01, 1.12s/it]
Loading 0: 11%|█▏ | 41.0/363 [00:32<06:01, 1.12s/it]
Loading 0: 13%|█▎ | 49.0/363 [00:33<02:28, 2.11it/s]
Loading 0: 13%|█▎ | 49.0/363 [00:33<02:28, 2.11it/s]
Loading 0: 14%|█▍ | 52.0/363 [00:35<02:48, 1.85it/s]
Loading 0: 14%|█▍ | 52.0/363 [00:35<02:48, 1.85it/s]
Loading 0: 15%|█▍ | 53.0/363 [00:37<03:34, 1.45it/s]
Loading 0: 15%|█▍ | 53.0/363 [00:37<03:34, 1.45it/s]
Loading 0: 15%|█▍ | 54.0/363 [00:39<04:29, 1.15it/s]
Loading 0: 15%|█▍ | 54.0/363 [00:39<04:29, 1.15it/s]
Loading 0: 16%|█▌ | 57.0/363 [00:41<04:00, 1.27it/s]
Loading 0: 16%|█▌ | 57.0/363 [00:41<04:00, 1.27it/s]
Loading 0: 16%|█▌ | 58.0/363 [00:43<04:50, 1.05it/s]
Loading 0: 16%|█▌ | 58.0/363 [00:43<04:50, 1.05it/s]
Loading 0: 16%|█▋ | 59.0/363 [00:45<05:45, 1.14s/it]
Loading 0: 16%|█▋ | 59.0/363 [00:45<05:45, 1.14s/it]
Loading 0: 18%|█▊ | 67.0/363 [00:46<02:21, 2.10it/s]
Loading 0: 18%|█▊ | 67.0/363 [00:46<02:21, 2.10it/s]
Loading 0: 19%|█▉ | 70.0/363 [00:48<02:39, 1.84it/s]
Loading 0: 19%|█▉ | 70.0/363 [00:48<02:39, 1.84it/s]
Loading 0: 20%|█▉ | 71.0/363 [00:50<03:22, 1.44it/s]
Loading 0: 20%|█▉ | 71.0/363 [00:50<03:22, 1.44it/s]
Loading 0: 20%|█▉ | 72.0/363 [00:52<04:14, 1.14it/s]
Loading 0: 20%|█▉ | 72.0/363 [00:52<04:14, 1.14it/s]
Loading 0: 21%|██ | 75.0/363 [00:54<03:46, 1.27it/s]
Loading 0: 21%|██ | 75.0/363 [00:54<03:46, 1.27it/s]
Loading 0: 21%|██ | 76.0/363 [00:56<04:33, 1.05it/s]
Loading 0: 21%|██ | 76.0/363 [00:56<04:33, 1.05it/s]
Loading 0: 21%|██ | 77.0/363 [00:58<05:26, 1.14s/it]
Loading 0: 21%|██ | 77.0/363 [00:58<05:26, 1.14s/it]
Loading 0: 23%|██▎ | 85.0/363 [00:59<02:12, 2.09it/s]
Loading 0: 23%|██▎ | 85.0/363 [00:59<02:12, 2.09it/s]
Loading 0: 24%|██▍ | 88.0/363 [01:01<02:29, 1.84it/s]
Loading 0: 24%|██▍ | 88.0/363 [01:01<02:29, 1.84it/s]
Loading 0: 25%|██▍ | 89.0/363 [01:03<03:10, 1.44it/s]
Loading 0: 25%|██▍ | 89.0/363 [01:03<03:10, 1.44it/s]
Loading 0: 25%|██▍ | 90.0/363 [01:05<03:59, 1.14it/s]
Loading 0: 25%|██▍ | 90.0/363 [01:05<03:59, 1.14it/s]
Loading 0: 27%|██▋ | 98.0/363 [01:06<01:54, 2.32it/s]
Loading 0: 27%|██▋ | 98.0/363 [01:06<01:54, 2.32it/s]
Loading 0: 28%|██▊ | 101/363 [01:08<02:09, 2.02it/s]
Loading 0: 28%|██▊ | 101/363 [01:08<02:09, 2.02it/s]
Loading 0: 28%|██▊ | 102/363 [01:10<02:47, 1.56it/s]
Loading 0: 28%|██▊ | 102/363 [01:10<02:47, 1.56it/s]
Loading 0: 28%|██▊ | 103/363 [01:12<03:32, 1.22it/s]
Loading 0: 28%|██▊ | 103/363 [01:12<03:32, 1.22it/s]
Loading 0: 29%|██▉ | 106/363 [01:14<03:17, 1.30it/s]
Loading 0: 29%|██▉ | 106/363 [01:14<03:17, 1.30it/s]
Loading 0: 29%|██▉ | 107/363 [01:16<03:58, 1.07it/s]
Loading 0: 29%|██▉ | 107/363 [01:16<03:58, 1.07it/s]
Loading 0: 30%|██▉ | 108/363 [01:18<04:45, 1.12s/it]
Loading 0: 30%|██▉ | 108/363 [01:18<04:45, 1.12s/it]
Loading 0: 31%|███ | 111/363 [01:20<03:50, 1.09it/s]
Loading 0: 31%|███ | 111/363 [01:20<03:50, 1.09it/s]
Loading 0: 31%|███ | 112/363 [01:22<04:30, 1.08s/it]
Loading 0: 31%|███ | 112/363 [01:22<04:30, 1.08s/it]
Loading 0: 31%|███ | 113/363 [01:24<05:14, 1.26s/it]
Loading 0: 31%|███ | 113/363 [01:24<05:14, 1.26s/it]
Loading 0: 33%|███▎ | 121/363 [01:25<01:58, 2.04it/s]
Loading 0: 33%|███▎ | 121/363 [01:25<01:58, 2.04it/s]
Loading 0: 34%|███▍ | 124/363 [01:27<02:12, 1.80it/s]
Loading 0: 34%|███▍ | 124/363 [01:27<02:12, 1.80it/s]
Loading 0: 34%|███▍ | 125/363 [01:29<02:48, 1.41it/s]
Loading 0: 34%|███▍ | 125/363 [01:29<02:48, 1.41it/s]
Loading 0: 35%|███▍ | 126/363 [01:31<03:30, 1.12it/s]
Loading 0: 35%|███▍ | 126/363 [01:31<03:30, 1.12it/s]
Loading 0: 36%|███▌ | 129/363 [01:33<03:05, 1.26it/s]
Loading 0: 36%|███▌ | 129/363 [01:33<03:05, 1.26it/s]
Loading 0: 36%|███▌ | 130/363 [01:35<03:43, 1.04it/s]
Loading 0: 36%|███▌ | 130/363 [01:35<03:43, 1.04it/s]
Loading 0: 36%|███▌ | 131/363 [01:37<04:26, 1.15s/it]
Loading 0: 36%|███▌ | 131/363 [01:37<04:26, 1.15s/it]
Loading 0: 38%|███▊ | 139/363 [01:38<01:47, 2.08it/s]
Loading 0: 38%|███▊ | 139/363 [01:38<01:47, 2.08it/s]
Loading 0: 39%|███▉ | 142/363 [01:40<02:01, 1.83it/s]
Loading 0: 39%|███▉ | 142/363 [01:40<02:01, 1.83it/s]
Loading 0: 39%|███▉ | 143/363 [01:42<02:33, 1.43it/s]
Loading 0: 39%|███▉ | 143/363 [01:42<02:33, 1.43it/s]
Loading 0: 40%|███▉ | 144/363 [01:44<03:12, 1.14it/s]
Loading 0: 40%|███▉ | 144/363 [01:44<03:12, 1.14it/s]
Loading 0: 40%|████ | 147/363 [01:46<02:50, 1.27it/s]
Loading 0: 40%|████ | 147/363 [01:46<02:50, 1.27it/s]
Loading 0: 41%|████ | 148/363 [01:48<03:25, 1.05it/s]
Loading 0: 41%|████ | 148/363 [01:48<03:25, 1.05it/s]
Loading 0: 41%|████ | 149/363 [01:50<04:04, 1.14s/it]
Loading 0: 41%|████ | 149/363 [01:50<04:04, 1.14s/it]
Loading 0: 43%|████▎ | 157/363 [01:51<01:39, 2.08it/s]
Loading 0: 43%|████▎ | 157/363 [01:51<01:39, 2.08it/s]
Loading 0: 44%|████▍ | 160/363 [01:54<01:51, 1.82it/s]
Loading 0: 44%|████▍ | 160/363 [01:54<01:51, 1.82it/s]
Loading 0: 44%|████▍ | 161/363 [01:55<02:21, 1.43it/s]
Loading 0: 44%|████▍ | 161/363 [01:55<02:21, 1.43it/s]
Loading 0: 45%|████▍ | 162/363 [01:57<02:56, 1.14it/s]
Loading 0: 45%|████▍ | 162/363 [01:57<02:56, 1.14it/s]
Loading 0: 45%|████▌ | 165/363 [01:59<02:36, 1.27it/s]
Loading 0: 45%|████▌ | 165/363 [01:59<02:36, 1.27it/s]
Loading 0: 46%|████▌ | 166/363 [02:01<03:07, 1.05it/s]
Loading 0: 46%|████▌ | 166/363 [02:01<03:07, 1.05it/s]
Loading 0: 46%|████▌ | 167/363 [02:03<03:43, 1.14s/it]
Loading 0: 46%|████▌ | 167/363 [02:03<03:43, 1.14s/it]
Loading 0: 48%|████▊ | 175/363 [02:04<01:30, 2.08it/s]
Loading 0: 48%|████▊ | 175/363 [02:04<01:30, 2.08it/s]
Loading 0: 49%|████▉ | 178/363 [02:07<01:41, 1.83it/s]
Loading 0: 49%|████▉ | 178/363 [02:07<01:41, 1.83it/s]
Loading 0: 49%|████▉ | 179/363 [02:09<02:08, 1.44it/s]
Loading 0: 49%|████▉ | 179/363 [02:09<02:08, 1.44it/s]
Loading 0: 50%|████▉ | 180/363 [02:11<02:40, 1.14it/s]
Loading 0: 50%|████▉ | 180/363 [02:11<02:40, 1.14it/s]
Loading 0: 50%|█████ | 183/363 [02:12<02:21, 1.27it/s]
Loading 0: 50%|█████ | 183/363 [02:12<02:21, 1.27it/s]
Loading 0: 51%|█████ | 184/363 [02:14<02:50, 1.05it/s]
Loading 0: 51%|█████ | 184/363 [02:14<02:50, 1.05it/s]
Loading 0: 51%|█████ | 185/363 [02:16<03:22, 1.14s/it]
Loading 0: 51%|█████ | 185/363 [02:16<03:22, 1.14s/it]
Loading 0: 53%|█████▎ | 192/363 [02:17<01:26, 1.98it/s]
Loading 0: 53%|█████▎ | 192/363 [02:17<01:26, 1.98it/s]
Loading 0: 54%|█████▍ | 196/363 [02:20<01:30, 1.85it/s]
Loading 0: 54%|█████▍ | 196/363 [02:20<01:30, 1.85it/s]
Loading 0: 54%|█████▍ | 197/363 [02:22<01:53, 1.46it/s]
Loading 0: 54%|█████▍ | 197/363 [02:22<01:53, 1.46it/s]
Loading 0: 55%|█████▍ | 198/363 [02:24<02:22, 1.16it/s]
Loading 0: 55%|█████▍ | 198/363 [02:24<02:22, 1.16it/s]
Loading 0: 55%|█████▌ | 201/363 [02:26<02:06, 1.28it/s]
Loading 0: 55%|█████▌ | 201/363 [02:26<02:06, 1.28it/s]
Loading 0: 56%|█████▌ | 202/363 [02:27<02:32, 1.06it/s]
Loading 0: 56%|█████▌ | 202/363 [02:27<02:32, 1.06it/s]
Loading 0: 56%|█████▌ | 203/363 [02:29<03:00, 1.13s/it]
Loading 0: 56%|█████▌ | 203/363 [02:29<03:00, 1.13s/it]
Loading 0: 58%|█████▊ | 211/363 [02:31<01:12, 2.09it/s]
Loading 0: 58%|█████▊ | 211/363 [02:31<01:12, 2.09it/s]
Loading 0: 59%|█████▉ | 214/363 [02:33<01:21, 1.83it/s]
Loading 0: 59%|█████▉ | 214/363 [02:33<01:21, 1.83it/s]
Loading 0: 59%|█████▉ | 215/363 [02:35<01:42, 1.44it/s]
Loading 0: 59%|█████▉ | 215/363 [02:35<01:42, 1.44it/s]
Loading 0: 60%|█████▉ | 216/363 [02:37<02:08, 1.14it/s]
Loading 0: 60%|█████▉ | 216/363 [02:37<02:08, 1.14it/s]
Loading 0: 60%|██████ | 219/363 [02:39<01:53, 1.27it/s]
Loading 0: 60%|██████ | 219/363 [02:39<01:53, 1.27it/s]
Loading 0: 61%|██████ | 220/363 [02:41<02:16, 1.05it/s]
Loading 0: 61%|██████ | 220/363 [02:41<02:16, 1.05it/s]
Loading 0: 61%|██████ | 221/363 [02:43<02:41, 1.14s/it]
Loading 0: 61%|██████ | 221/363 [02:43<02:41, 1.14s/it]
Loading 0: 63%|██████▎ | 229/363 [02:44<01:04, 2.08it/s]
Loading 0: 63%|██████▎ | 229/363 [02:44<01:04, 2.08it/s]
Loading 0: 64%|██████▍ | 232/363 [02:46<01:11, 1.83it/s]
Loading 0: 64%|██████▍ | 232/363 [02:46<01:11, 1.83it/s]
Loading 0: 64%|██████▍ | 233/363 [02:48<01:30, 1.44it/s]
Loading 0: 64%|██████▍ | 233/363 [02:48<01:30, 1.44it/s]
Loading 0: 64%|██████▍ | 234/363 [02:50<01:52, 1.14it/s]
Loading 0: 64%|██████▍ | 234/363 [02:50<01:52, 1.14it/s]
Loading 0: 65%|██████▌ | 237/363 [02:52<01:39, 1.27it/s]
Loading 0: 65%|██████▌ | 237/363 [02:52<01:39, 1.27it/s]
Loading 0: 66%|██████▌ | 238/363 [02:54<01:58, 1.05it/s]
Loading 0: 66%|██████▌ | 238/363 [02:54<01:58, 1.05it/s]
Loading 0: 66%|██████▌ | 239/363 [02:56<02:21, 1.14s/it]
Loading 0: 66%|██████▌ | 239/363 [02:56<02:21, 1.14s/it]
Loading 0: 68%|██████▊ | 247/363 [02:57<00:55, 2.08it/s]
Loading 0: 68%|██████▊ | 247/363 [02:57<00:55, 2.08it/s]
Loading 0: 69%|██████▉ | 250/363 [02:59<01:01, 1.83it/s]
Loading 0: 69%|██████▉ | 250/363 [02:59<01:01, 1.83it/s]
Loading 0: 69%|██████▉ | 251/363 [03:01<01:18, 1.44it/s]
Loading 0: 69%|██████▉ | 251/363 [03:01<01:18, 1.44it/s]
Loading 0: 69%|██████▉ | 252/363 [03:03<01:37, 1.14it/s]
Loading 0: 69%|██████▉ | 252/363 [03:03<01:37, 1.14it/s]
Loading 0: 70%|███████ | 255/363 [03:05<01:25, 1.27it/s]
Loading 0: 70%|███████ | 255/363 [03:05<01:25, 1.27it/s]
Loading 0: 71%|███████ | 256/363 [03:07<01:41, 1.05it/s]
Loading 0: 71%|███████ | 256/363 [03:07<01:41, 1.05it/s]
Loading 0: 71%|███████ | 257/363 [03:09<02:00, 1.14s/it]
Loading 0: 71%|███████ | 257/363 [03:09<02:00, 1.14s/it]
Loading 0: 73%|███████▎ | 265/363 [03:10<00:46, 2.09it/s]
Loading 0: 73%|███████▎ | 265/363 [03:10<00:46, 2.09it/s]
Loading 0: 74%|███████▍ | 268/363 [03:12<00:51, 1.84it/s]
Loading 0: 74%|███████▍ | 268/363 [03:12<00:51, 1.84it/s]
Loading 0: 74%|███████▍ | 269/363 [03:14<01:05, 1.44it/s]
Loading 0: 74%|███████▍ | 269/363 [03:14<01:05, 1.44it/s]
Loading 0: 74%|███████▍ | 270/363 [03:16<01:21, 1.15it/s]
Loading 0: 74%|███████▍ | 270/363 [03:16<01:21, 1.15it/s]
Loading 0: 75%|███████▌ | 273/363 [03:31<03:33, 2.37s/it]
Loading 0: 75%|███████▌ | 273/363 [03:31<03:33, 2.37s/it]
Loading 0: 75%|███████▌ | 274/363 [03:33<03:24, 2.30s/it]
Loading 0: 75%|███████▌ | 274/363 [03:33<03:24, 2.30s/it]
Loading 0: 76%|███████▌ | 275/363 [03:35<03:18, 2.25s/it]
Loading 0: 76%|███████▌ | 275/363 [03:35<03:18, 2.25s/it]
Loading 0: 78%|███████▊ | 283/363 [03:36<01:07, 1.18it/s]
Loading 0: 78%|███████▊ | 283/363 [03:36<01:07, 1.18it/s]
Loading 0: 79%|███████▉ | 286/363 [03:38<01:02, 1.24it/s]
Loading 0: 79%|███████▉ | 286/363 [03:38<01:02, 1.24it/s]
Loading 0: 79%|███████▉ | 287/363 [03:40<01:09, 1.10it/s]
Loading 0: 79%|███████▉ | 287/363 [03:40<01:09, 1.10it/s]
Loading 0: 79%|███████▉ | 288/363 [03:42<01:19, 1.06s/it]
Loading 0: 79%|███████▉ | 288/363 [03:42<01:19, 1.06s/it]
Loading 0: 80%|████████ | 291/363 [03:44<01:05, 1.11it/s]
Loading 0: 80%|████████ | 291/363 [03:44<01:05, 1.11it/s]
Loading 0: 80%|████████ | 292/363 [03:46<01:13, 1.03s/it]
Loading 0: 80%|████████ | 292/363 [03:46<01:13, 1.03s/it]
Loading 0: 81%|████████ | 293/363 [03:47<01:24, 1.20s/it]
Loading 0: 81%|████████ | 293/363 [03:47<01:24, 1.20s/it]
Loading 0: 83%|████████▎ | 301/363 [03:49<00:30, 2.03it/s]
Loading 0: 83%|████████▎ | 301/363 [03:49<00:30, 2.03it/s]
Loading 0: 84%|████████▎ | 304/363 [03:51<00:32, 1.82it/s]
Loading 0: 84%|████████▎ | 304/363 [03:51<00:32, 1.82it/s]
Loading 0: 84%|████████▍ | 305/363 [03:53<00:40, 1.43it/s]
Loading 0: 84%|████████▍ | 305/363 [03:53<00:40, 1.43it/s]
Loading 0: 84%|████████▍ | 306/363 [03:55<00:50, 1.14it/s]
Loading 0: 84%|████████▍ | 306/363 [03:55<00:50, 1.14it/s]
Loading 0: 85%|████████▌ | 309/363 [03:57<00:42, 1.27it/s]
Loading 0: 85%|████████▌ | 309/363 [03:57<00:42, 1.27it/s]
Loading 0: 85%|████████▌ | 310/363 [03:58<00:50, 1.05it/s]
Loading 0: 85%|████████▌ | 310/363 [03:58<00:50, 1.05it/s]
Loading 0: 86%|████████▌ | 311/363 [04:00<00:59, 1.14s/it]
Loading 0: 86%|████████▌ | 311/363 [04:00<00:59, 1.14s/it]
Loading 0: 88%|████████▊ | 319/363 [04:02<00:20, 2.12it/s]
Loading 0: 88%|████████▊ | 319/363 [04:02<00:20, 2.12it/s]
Loading 0: 89%|████████▊ | 322/363 [04:04<00:22, 1.85it/s]
Loading 0: 89%|████████▊ | 322/363 [04:04<00:22, 1.85it/s]
Loading 0: 89%|████████▉ | 323/363 [04:06<00:27, 1.45it/s]
Loading 0: 89%|████████▉ | 323/363 [04:06<00:27, 1.45it/s]
Loading 0: 89%|████████▉ | 324/363 [04:08<00:33, 1.15it/s]
Loading 0: 89%|████████▉ | 324/363 [04:08<00:33, 1.15it/s]
Loading 0: 90%|█████████ | 327/363 [04:10<00:28, 1.28it/s]
Loading 0: 90%|█████████ | 327/363 [04:10<00:28, 1.28it/s]
Loading 0: 90%|█████████ | 328/363 [04:11<00:33, 1.05it/s]
Loading 0: 90%|█████████ | 328/363 [04:11<00:33, 1.05it/s]
Loading 0: 91%|█████████ | 329/363 [04:13<00:38, 1.13s/it]
Loading 0: 91%|█████████ | 329/363 [04:13<00:38, 1.13s/it]
Loading 0: 93%|█████████▎| 337/363 [04:15<00:12, 2.12it/s]
Loading 0: 93%|█████████▎| 337/363 [04:15<00:12, 2.12it/s]
Loading 0: 94%|█████████▎| 340/363 [04:17<00:12, 1.85it/s]
Loading 0: 94%|█████████▎| 340/363 [04:17<00:12, 1.85it/s]
Loading 0: 94%|█████████▍| 341/363 [04:19<00:15, 1.45it/s]
Loading 0: 94%|█████████▍| 341/363 [04:19<00:15, 1.45it/s]
Loading 0: 94%|█████████▍| 342/363 [04:21<00:18, 1.15it/s]
Loading 0: 94%|█████████▍| 342/363 [04:21<00:18, 1.15it/s]
Loading 0: 95%|█████████▌| 345/363 [04:23<00:14, 1.27it/s]
Loading 0: 95%|█████████▌| 345/363 [04:23<00:14, 1.27it/s]
Loading 0: 95%|█████████▌| 346/363 [04:24<00:16, 1.05it/s]
Loading 0: 95%|█████████▌| 346/363 [04:24<00:16, 1.05it/s]
Loading 0: 96%|█████████▌| 347/363 [04:26<00:18, 1.14s/it]
Loading 0: 96%|█████████▌| 347/363 [04:26<00:18, 1.14s/it]
Loading 0: 98%|█████████▊| 355/363 [04:28<00:03, 2.11it/s]
Loading 0: 98%|█████████▊| 355/363 [04:28<00:03, 2.11it/s]
Loading 0: 99%|█████████▉| 359/363 [04:30<00:02, 1.90it/s]
Loading 0: 99%|█████████▉| 359/363 [04:30<00:02, 1.90it/s]
Loading 0: 99%|█████████▉| 360/363 [04:32<00:02, 1.50it/s]
Loading 0: 99%|█████████▉| 360/363 [04:32<00:02, 1.50it/s]
Loading 0: 99%|█████████▉| 361/363 [04:34<00:01, 1.19it/s]
Loading 0: 99%|█████████▉| 361/363 [04:34<00:01, 1.19it/s]
Loading 0: 100%|██████████| 363/363 [04:34<00:00, 1.19it/s]
Loading 0: 100%|██████████| 363/363 [04:34<00:00, 1.32it/s]
Received healthy response to inference request in 3.5232081413269043s
5 requests
0 failed requests
5th percentile: 3.4396314144134523
10th percentile: 3.4730252265930175
20th percentile: 3.5398128509521483
30th percentile: 3.5974825859069823
40th percentile: 3.6460344314575197
50th percentile: 3.6945862770080566
Received healthy response to inference request in 3.4492075443267822s
60th percentile: 3.7015323638916016
5 requests
70th percentile: 3.7084784507751465
0 failed requests
80th percentile: 3.7238330364227297
5th percentile: 3.3203797340393066
90th percentile: 3.7475961208343507
10th percentile: 3.32867431640625
95th percentile: 3.759477663040161
20th percentile: 3.3452634811401367
99th percentile: 3.7689828968048094
30th percentile: 3.3726879596710204
mean time: 3.6314682483673097
40th percentile: 3.4109477519989015
Pipeline stage StressChecker completed in 31.79s
50th percentile: 3.4492075443267822
run pipeline stage %s
60th percentile: 3.478807783126831
Running pipeline stage OfflineFamilyFriendlyTriggerPipeline
70th percentile: 3.50840802192688
run_pipeline:run_in_cloud %s
80th percentile: 3.558500385284424
starting trigger_guanaco_pipeline args=%s
90th percentile: 3.6290848731994627
chaiml-mistral-24b-2048-46891-v1-mkmlizer: The tokenizer you are loading from '/tmp/tmp374nfmpc' with an incorrect regex pattern: https://huggingface.co/mistralai/Mistral-Small-3.1-24B-Instruct-2503/discussions/84#69121093e8b480e709447d5e. This will lead to incorrect tokenization. You should set the `fix_mistral_regex=True` flag when loading this tokenizer to fix this issue.
triggered trigger_guanaco_pipeline args=%s
95th percentile: 3.6643771171569823
chaiml-mistral-24b-2048-46891-v1-mkmlizer: quantized model in 280.653s
Pipeline stage OfflineFamilyFriendlyTriggerPipeline completed in 4.07s
99th percentile: 3.692610912322998
chaiml-mistral-24b-2048-46891-v1-mkmlizer: Processed model ChaiML/mistral_24b_2048_gemini_ds_v7_3279_merged in 345.831s
run pipeline stage %s
mean time: 3.4675456523895263
chaiml-mistral-24b-2048-46891-v1-mkmlizer: creating bucket guanaco-mkml-models
Running pipeline stage TriggerMKMLProfilingPipeline
Pipeline stage StressChecker completed in 67.32s
chaiml-mistral-24b-2048-46891-v1-mkmlizer: Bucket 's3://guanaco-mkml-models/' created
run_pipeline:run_in_cloud %s
run pipeline stage %s
chaiml-mistral-24b-2048-46891-v1-mkmlizer: uploading /dev/shm/model_cache to s3://guanaco-mkml-models/chaiml-mistral-24b-2048-46891-v1/nvidia
starting trigger_guanaco_pipeline args=%s
Running pipeline stage OfflineFamilyFriendlyTriggerPipeline
chaiml-mistral-24b-2048-46891-v1-mkmlizer: cp /dev/shm/model_cache/config.json s3://guanaco-mkml-models/chaiml-mistral-24b-2048-46891-v1/nvidia/config.json
run_pipeline:run_in_cloud %s
chaiml-mistral-24b-2048-46891-v1-mkmlizer: cp /dev/shm/model_cache/special_tokens_map.json s3://guanaco-mkml-models/chaiml-mistral-24b-2048-46891-v1/nvidia/special_tokens_map.json
triggered trigger_guanaco_pipeline args=%s
starting trigger_guanaco_pipeline args=%s
chaiml-mistral-24b-2048-46891-v1-mkmlizer: cp /dev/shm/model_cache/tokenizer_config.json s3://guanaco-mkml-models/chaiml-mistral-24b-2048-46891-v1/nvidia/tokenizer_config.json
Pipeline stage TriggerMKMLProfilingPipeline completed in 3.56s
chaiml-mistral-24b-2048-46891-v1-mkmlizer: cp /dev/shm/model_cache/tokenizer.json s3://guanaco-mkml-models/chaiml-mistral-24b-2048-46891-v1/nvidia/tokenizer.json
Shutdown handler de-registered
triggered trigger_guanaco_pipeline args=%s
chaiml-mistral-24b-2048-_4101_v1 status is now deployed due to DeploymentManager action
Pipeline stage OfflineFamilyFriendlyTriggerPipeline completed in 3.31s
Shutdown handler registered
run pipeline %s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyScorer
Evaluating %s Family Friendly Score with %s threads
Generating Leaderboard row for %s
Generated Leaderboard row for %s
Pipeline stage OfflineFamilyFriendlyScorer completed in 2988.84s
Shutdown handler de-registered
chaiml-mistral-24b-2048-_4101_v1 status is now inactive due to auto deactivation removed underperforming models