chaiml-mistral-24b-2048_87648

developer_uid: rirv938
submission_id: chaiml-mistral-24b-2048_87648_v1
model_name: chaiml-mistral-24b-2048_87648_v1
model_group: ChaiML/mistral_24b_2048_
status: torndown
timestamp: 2026-02-01T05:18:44+00:00
num_battles: 10933
num_wins: 5711
celo_rating: 1320.42
family_friendly_score: 0.49260000000000004
family_friendly_standard_error: 0.007070293346106652
submission_type: basic
model_repo: ChaiML/mistral_24b_2048_gemini_ds_v3_4374_merged
model_architecture: MistralForCausalLM
model_num_parameters: 24096691200.0
best_of: 8
max_input_tokens: 2048
max_output_tokens: 112
reward_model: default
latencies: [{'batch_size': 1, 'throughput': 0.3112487308619326, 'latency_mean': 3.2127951860427855, 'latency_p50': 3.2192790508270264, 'latency_p90': 3.42740478515625}, {'batch_size': 2, 'throughput': 0.5004546421398348, 'latency_mean': 3.9913334465026855, 'latency_p50': 3.9817742109298706, 'latency_p90': 4.212646555900574}, {'batch_size': 3, 'throughput': 0.6352502751021974, 'latency_mean': 4.70894569516182, 'latency_p50': 4.703855037689209, 'latency_p90': 4.958653926849365}, {'batch_size': 4, 'throughput': 0.7202666508640787, 'latency_mean': 5.525526180267334, 'latency_p50': 5.512428522109985, 'latency_p90': 5.810662770271302}, {'batch_size': 5, 'throughput': 0.7936374733100084, 'latency_mean': 6.266703525781631, 'latency_p50': 6.2671321630477905, 'latency_p90': 6.809777784347534}]
gpu_counts: {'NVIDIA A100-SXM4-80GB': 1}
display_name: chaiml-mistral-24b-2048_87648_v1
ineligible_reason: max_output_tokens!=64
is_internal_developer: True
language_model: ChaiML/mistral_24b_2048_gemini_ds_v3_4374_merged
model_size: 24B
ranking_group: single
throughput_3p7s: 0.44
us_pacific_date: 2026-01-28
win_ratio: 0.5223634866916674
generation_params: {'temperature': 1.0, 'top_p': 0.95, 'min_p': 0.05, 'top_k': 60, 'presence_penalty': 0.1, 'frequency_penalty': 0.0, 'stopping_words': ['</s>', '<|im_start|>', '###', 'You:', '<|im_end|>'], 'max_input_tokens': 2048, 'best_of': 8, 'max_output_tokens': 112}
formatter: {'memory_template': '', 'prompt_template': '', 'bot_template': '{bot_name}: {message}\n', 'user_template': '{user_name}: {message}\n', 'response_template': '{bot_name}:', 'truncate_by_message': False}
Resubmit model
Shutdown handler not registered because Python interpreter is not running in the main thread
run pipeline %s
run pipeline stage %s
Running pipeline stage MKMLizer
Starting job with name chaiml-mistral-24b-2048-87648-v1-mkmlizer
Waiting for job on chaiml-mistral-24b-2048-87648-v1-mkmlizer to finish
mistralai-mistral-smal-88026-v68-mkmlizer: /root/miniconda3/envs/nvidia/lib/python3.11/site-packages/mk1/__init__.py:1: UserWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html. The pkg_resources package is slated for removal as early as 2025-11-30. Refrain from using this package or pin to Setuptools<81.
mistralai-mistral-smal-88026-v68-mkmlizer:   __import__('pkg_resources').declare_namespace(__name__)
mistralai-mistral-smal-88026-v68-mkmlizer: ╔═════════════════════════════════════════════════════════════════════╗
mistralai-mistral-smal-88026-v68-mkmlizer: ║                                                                     ║
mistralai-mistral-smal-88026-v68-mkmlizer: ║   ██████   ██████  █████   ████  ████                               ║
mistralai-mistral-smal-88026-v68-mkmlizer: ║  ░░██████ ██████  ░░███   ███░  ░░███                               ║
mistralai-mistral-smal-88026-v68-mkmlizer: ║   ░███░█████░███   ░███  ███     ░███                               ║
mistralai-mistral-smal-88026-v68-mkmlizer: ║   ░███░░███ ░███   ░███████      ░███                               ║
mistralai-mistral-smal-88026-v68-mkmlizer: ║   ░███ ░░░  ░███   ░███░░███     ░███                               ║
mistralai-mistral-smal-88026-v68-mkmlizer: ║   ░███      ░███   ░███ ░░███    ░███                               ║
mistralai-mistral-smal-88026-v68-mkmlizer: ║   █████     █████  █████ ░░████  █████                              ║
mistralai-mistral-smal-88026-v68-mkmlizer: ║  ░░░░░     ░░░░░  ░░░░░   ░░░░  ░░░░░                               ║
mistralai-mistral-smal-88026-v68-mkmlizer: ║                                                                     ║
mistralai-mistral-smal-88026-v68-mkmlizer: ║  Version: 0.30.6+torch280                                           ║
mistralai-mistral-smal-88026-v68-mkmlizer: ║  Features: FLYWHEEL, CUDA                                           ║
chaiml-mistral-24b-2048-76638-v1-mkmlizer: /root/miniconda3/envs/nvidia/lib/python3.11/site-packages/mk1/__init__.py:1: UserWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html. The pkg_resources package is slated for removal as early as 2025-11-30. Refrain from using this package or pin to Setuptools<81.
mistralai-mistral-smal-88026-v68-mkmlizer: ║  Copyright 2023-2025 MK ONE TECHNOLOGIES Inc.                       ║
chaiml-mistral-24b-2048-76638-v1-mkmlizer:   __import__('pkg_resources').declare_namespace(__name__)
mistralai-mistral-smal-88026-v68-mkmlizer: ║  https://mk1.ai                                                     ║
chaiml-mistral-24b-2048-76638-v1-mkmlizer: ╔═════════════════════════════════════════════════════════════════════╗
mistralai-mistral-smal-88026-v68-mkmlizer: ║                                                                     ║
chaiml-mistral-24b-2048-76638-v1-mkmlizer: ║                                                                     ║
mistralai-mistral-smal-88026-v68-mkmlizer: ║  The license key for the current software has been verified as      ║
chaiml-mistral-24b-2048-76638-v1-mkmlizer: ║   ██████   ██████  █████   ████  ████                               ║
mistralai-mistral-smal-88026-v68-mkmlizer: ║  belonging to:                                                      ║
chaiml-mistral-24b-2048-76638-v1-mkmlizer: ║  ░░██████ ██████  ░░███   ███░  ░░███                               ║
mistralai-mistral-smal-88026-v68-mkmlizer: ║                                                                     ║
chaiml-mistral-24b-2048-76638-v1-mkmlizer: ║   ░███░█████░███   ░███  ███     ░███                               ║
mistralai-mistral-smal-88026-v68-mkmlizer: ║  Chai Research Corp.                                                ║
chaiml-mistral-24b-2048-76638-v1-mkmlizer: ║   ░███░░███ ░███   ░███████      ░███                               ║
mistralai-mistral-smal-88026-v68-mkmlizer: ║  Account ID: 7997a29f-0ceb-4cc7-9adf-840c57b4ae6f                   ║
chaiml-mistral-24b-2048-76638-v1-mkmlizer: ║   ░███ ░░░  ░███   ░███░░███     ░███                               ║
mistralai-mistral-smal-88026-v68-mkmlizer: ║  Expiration: 2028-03-31 23:59:59                                    ║
chaiml-mistral-24b-2048-76638-v1-mkmlizer: ║   ░███      ░███   ░███ ░░███    ░███                               ║
mistralai-mistral-smal-88026-v68-mkmlizer: ║                                                                     ║
chaiml-mistral-24b-2048-76638-v1-mkmlizer: ║   █████     █████  █████ ░░████  █████                              ║
mistralai-mistral-smal-88026-v68-mkmlizer: ╚═════════════════════════════════════════════════════════════════════╝
chaiml-mistral-24b-2048-76638-v1-mkmlizer: ║  ░░░░░     ░░░░░  ░░░░░   ░░░░  ░░░░░                               ║
chaiml-mistral-24b-2048-76638-v1-mkmlizer: ║                                                                     ║
chaiml-mistral-24b-2048-76638-v1-mkmlizer: ║  Version: 0.30.6+torch280                                           ║
chaiml-mistral-24b-2048-76638-v1-mkmlizer: ║  Features: FLYWHEEL, CUDA                                           ║
chaiml-mistral-24b-2048-41286-v1-mkmlizer: /root/miniconda3/envs/nvidia/lib/python3.11/site-packages/mk1/__init__.py:1: UserWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html. The pkg_resources package is slated for removal as early as 2025-11-30. Refrain from using this package or pin to Setuptools<81.
chaiml-mistral-24b-2048-76638-v1-mkmlizer: ║  Copyright 2023-2025 MK ONE TECHNOLOGIES Inc.                       ║
chaiml-mistral-24b-2048-41286-v1-mkmlizer:   __import__('pkg_resources').declare_namespace(__name__)
chaiml-mistral-24b-2048-76638-v1-mkmlizer: ║  https://mk1.ai                                                     ║
chaiml-mistral-24b-2048-41286-v1-mkmlizer: ╔═════════════════════════════════════════════════════════════════════╗
chaiml-mistral-24b-2048-76638-v1-mkmlizer: ║                                                                     ║
chaiml-mistral-24b-2048-41286-v1-mkmlizer: ║                                                                     ║
chaiml-mistral-24b-2048-76638-v1-mkmlizer: ║  The license key for the current software has been verified as      ║
chaiml-mistral-24b-2048-41286-v1-mkmlizer: ║   ██████   ██████  █████   ████  ████                               ║
chaiml-mistral-24b-2048-76638-v1-mkmlizer: ║  belonging to:                                                      ║
chaiml-mistral-24b-2048-41286-v1-mkmlizer: ║  ░░██████ ██████  ░░███   ███░  ░░███                               ║
chaiml-mistral-24b-2048-76638-v1-mkmlizer: ║                                                                     ║
chaiml-mistral-24b-2048-41286-v1-mkmlizer: ║   ░███░█████░███   ░███  ███     ░███                               ║
chaiml-mistral-24b-2048-76638-v1-mkmlizer: ║  Chai Research Corp.                                                ║
chaiml-mistral-24b-2048-41286-v1-mkmlizer: ║   ░███░░███ ░███   ░███████      ░███                               ║
chaiml-mistral-24b-2048-76638-v1-mkmlizer: ║  Account ID: 7997a29f-0ceb-4cc7-9adf-840c57b4ae6f                   ║
chaiml-mistral-24b-2048-41286-v1-mkmlizer: ║   ░███ ░░░  ░███   ░███░░███     ░███                               ║
chaiml-mistral-24b-2048-76638-v1-mkmlizer: ║  Expiration: 2028-03-31 23:59:59                                    ║
chaiml-mistral-24b-2048-41286-v1-mkmlizer: ║   ░███      ░███   ░███ ░░███    ░███                               ║
chaiml-mistral-24b-2048-76638-v1-mkmlizer: ║                                                                     ║
chaiml-mistral-24b-2048-41286-v1-mkmlizer: ║   █████     █████  █████ ░░████  █████                              ║
chaiml-mistral-24b-2048-76638-v1-mkmlizer: ╚═════════════════════════════════════════════════════════════════════╝
chaiml-mistral-24b-2048-41286-v1-mkmlizer: ║  ░░░░░     ░░░░░  ░░░░░   ░░░░  ░░░░░                               ║
chaiml-mistral-24b-2048-41286-v1-mkmlizer: ║                                                                     ║
chaiml-mistral-24b-2048-41286-v1-mkmlizer: ║  Version: 0.30.6+torch280                                           ║
chaiml-mistral-24b-2048-41286-v1-mkmlizer: ║  Features: FLYWHEEL, CUDA                                           ║
chaiml-mistral-24b-2048-41286-v1-mkmlizer: ║  Copyright 2023-2025 MK ONE TECHNOLOGIES Inc.                       ║
chaiml-mistral-24b-2048-41286-v1-mkmlizer: ║  https://mk1.ai                                                     ║
chaiml-mistral-24b-2048-87648-v1-mkmlizer: bash: cannot set terminal process group (-1): Inappropriate ioctl for device
chaiml-mistral-24b-2048-41286-v1-mkmlizer: ║                                                                     ║
chaiml-mistral-24b-2048-87648-v1-mkmlizer: bash: no job control in this shell
chaiml-mistral-24b-2048-41286-v1-mkmlizer: ║  The license key for the current software has been verified as      ║
chaiml-mistral-24b-2048-87648-v1-mkmlizer: /root/miniconda3/envs/nvidia/lib/python3.11/site-packages/mk1/__init__.py:1: UserWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html. The pkg_resources package is slated for removal as early as 2025-11-30. Refrain from using this package or pin to Setuptools<81.
chaiml-mistral-24b-2048-41286-v1-mkmlizer: ║  belonging to:                                                      ║
chaiml-mistral-24b-2048-87648-v1-mkmlizer:   __import__('pkg_resources').declare_namespace(__name__)
chaiml-mistral-24b-2048-41286-v1-mkmlizer: ║                                                                     ║
chaiml-mistral-24b-2048-87648-v1-mkmlizer: ╔═════════════════════════════════════════════════════════════════════╗
chaiml-mistral-24b-2048-41286-v1-mkmlizer: ║  Chai Research Corp.                                                ║
chaiml-mistral-24b-2048-87648-v1-mkmlizer: ║                                                                     ║
chaiml-mistral-24b-2048-41286-v1-mkmlizer: ║  Account ID: 7997a29f-0ceb-4cc7-9adf-840c57b4ae6f                   ║
chaiml-mistral-24b-2048-87648-v1-mkmlizer: ║   ██████   ██████  █████   ████  ████                               ║
chaiml-mistral-24b-2048-41286-v1-mkmlizer: ║  Expiration: 2028-03-31 23:59:59                                    ║
chaiml-mistral-24b-2048-87648-v1-mkmlizer: ║  ░░██████ ██████  ░░███   ███░  ░░███                               ║
chaiml-mistral-24b-2048-41286-v1-mkmlizer: ║                                                                     ║
chaiml-mistral-24b-2048-87648-v1-mkmlizer: ║   ░███░█████░███   ░███  ███     ░███                               ║
chaiml-mistral-24b-2048-41286-v1-mkmlizer: ╚═════════════════════════════════════════════════════════════════════╝
chaiml-mistral-24b-2048-87648-v1-mkmlizer: ║   ░███░░███ ░███   ░███████      ░███                               ║
chaiml-mistral-24b-2048-87648-v1-mkmlizer: ║   ░███ ░░░  ░███   ░███░░███     ░███                               ║
chaiml-mistral-24b-2048-87648-v1-mkmlizer: ║   ░███      ░███   ░███ ░░███    ░███                               ║
chaiml-mistral-24b-2048-87648-v1-mkmlizer: ║   █████     █████  █████ ░░████  █████                              ║
chaiml-mistral-24b-2048-87648-v1-mkmlizer: ║  ░░░░░     ░░░░░  ░░░░░   ░░░░  ░░░░░                               ║
chaiml-mistral-24b-2048-87648-v1-mkmlizer: ║                                                                     ║
chaiml-mistral-24b-2048-87648-v1-mkmlizer: ║  Version: 0.30.6+torch280                                           ║
chaiml-mistral-24b-2048-87648-v1-mkmlizer: ║  Features: FLYWHEEL, CUDA                                           ║
chaiml-mistral-24b-2048-87648-v1-mkmlizer: ║  Copyright 2023-2025 MK ONE TECHNOLOGIES Inc.                       ║
chaiml-mistral-24b-2048-87648-v1-mkmlizer: ║  https://mk1.ai                                                     ║
chaiml-mistral-24b-2048-87648-v1-mkmlizer: ║                                                                     ║
chaiml-mistral-24b-2048-87648-v1-mkmlizer: ║  The license key for the current software has been verified as      ║
chaiml-mistral-24b-2048-87648-v1-mkmlizer: ║  belonging to:                                                      ║
chaiml-mistral-24b-2048-87648-v1-mkmlizer: ║                                                                     ║
chaiml-mistral-24b-2048-87648-v1-mkmlizer: ║  Chai Research Corp.                                                ║
chaiml-mistral-24b-2048-87648-v1-mkmlizer: ║  Account ID: 7997a29f-0ceb-4cc7-9adf-840c57b4ae6f                   ║
chaiml-mistral-24b-2048-87648-v1-mkmlizer: ║  Expiration: 2028-03-31 23:59:59                                    ║
chaiml-mistral-24b-2048-87648-v1-mkmlizer: ║                                                                     ║
chaiml-mistral-24b-2048-87648-v1-mkmlizer: ╚═════════════════════════════════════════════════════════════════════╝
mistralai-mistral-smal-88026-v68-mkmlizer: Downloaded to shared memory in 73.552s
mistralai-mistral-smal-88026-v68-mkmlizer: Checking if mistralai/Mistral-Small-24B-Base-2501 already exists in ChaiML
mistralai-mistral-smal-88026-v68-mkmlizer: quantizing model to /dev/shm/model_cache, profile:q4, folder:/tmp/tmpjbd6w5x8, device:0
mistralai-mistral-smal-88026-v68-mkmlizer: Saving flywheel model at /dev/shm/model_cache
chaiml-mistral-24b-2048-41286-v1-mkmlizer: Downloaded to shared memory in 92.540s
chaiml-mistral-24b-2048-41286-v1-mkmlizer: Checking if ChaiML/mistral_24b_2048_gemini_ds_v3_2187_merged already exists in ChaiML
chaiml-mistral-24b-2048-41286-v1-mkmlizer: quantizing model to /dev/shm/model_cache, profile:q4, folder:/tmp/tmpyhgbbh9k, device:0
chaiml-mistral-24b-2048-41286-v1-mkmlizer: Saving flywheel model at /dev/shm/model_cache
chaiml-mistral-24b-2048-76638-v1-mkmlizer: Downloaded to shared memory in 96.517s
chaiml-mistral-24b-2048-76638-v1-mkmlizer: Checking if ChaiML/mistral_24b_2048_gemini_ds_v3_6561_merged already exists in ChaiML
chaiml-mistral-24b-2048-76638-v1-mkmlizer: quantizing model to /dev/shm/model_cache, profile:q4, folder:/tmp/tmphcu9_fue, device:0
chaiml-mistral-24b-2048-76638-v1-mkmlizer: Saving flywheel model at /dev/shm/model_cache
chaiml-mistral-24b-2048-87648-v1-mkmlizer: Downloaded to shared memory in 89.345s
chaiml-mistral-24b-2048-87648-v1-mkmlizer: Checking if ChaiML/mistral_24b_2048_gemini_ds_v3_4374_merged already exists in ChaiML
chaiml-mistral-24b-2048-87648-v1-mkmlizer: quantizing model to /dev/shm/model_cache, profile:q4, folder:/tmp/tmpynetxqq5, device:0
chaiml-mistral-24b-2048-87648-v1-mkmlizer: Saving flywheel model at /dev/shm/model_cache
mistralai-mistral-smal-88026-v68-mkmlizer: 
Loading 0:   0%|          | 0.00/363 [00:00<?, ?it/s]
Loading 0:   1%|          | 3.00/363 [00:02<04:06, 1.46it/s]
Loading 0:   1%|          | 3.00/363 [00:02<04:06, 1.46it/s]
Loading 0:   1%|          | 4.00/363 [00:03<06:23, 1.07s/it]
Loading 0:   1%|          | 4.00/363 [00:03<06:23, 1.07s/it]
Loading 0:   1%|▏         | 5.00/363 [00:05<08:08, 1.36s/it]
Loading 0:   1%|▏         | 5.00/363 [00:05<08:08, 1.36s/it]
Loading 0:   3%|▎         | 12.0/363 [00:08<03:34, 1.64it/s]
Loading 0:   3%|▎         | 12.0/363 [00:08<03:34, 1.64it/s]
Loading 0:   4%|▎         | 13.0/363 [00:10<04:33, 1.28it/s]
Loading 0:   4%|▎         | 13.0/363 [00:10<04:33, 1.28it/s]
Loading 0:   4%|▍         | 14.0/363 [00:12<05:40, 1.03it/s]
Loading 0:   4%|▍         | 14.0/363 [00:12<05:40, 1.03it/s]
Loading 0:   6%|▌         | 21.0/363 [00:14<03:24, 1.67it/s]
Loading 0:   6%|▌         | 21.0/363 [00:14<03:24, 1.67it/s]
Loading 0:   6%|▌         | 22.0/363 [00:16<04:12, 1.35it/s]
Loading 0:   6%|▌         | 22.0/363 [00:16<04:12, 1.35it/s]
Loading 0:   6%|▋         | 23.0/363 [00:18<05:11, 1.09it/s]
Loading 0:   6%|▋         | 23.0/363 [00:18<05:11, 1.09it/s]
Loading 0:   9%|▊         | 31.0/363 [00:20<02:28, 2.24it/s]
Loading 0:   9%|▊         | 31.0/363 [00:20<02:28, 2.24it/s]
Loading 0:   9%|▉         | 34.0/363 [00:22<02:49, 1.94it/s]
Loading 0:   9%|▉         | 34.0/363 [00:22<02:49, 1.94it/s]
Loading 0:  10%|▉         | 35.0/363 [00:24<03:35, 1.52it/s]
Loading 0:  10%|▉         | 35.0/363 [00:24<03:35, 1.52it/s]
Loading 0:  10%|▉         | 36.0/363 [00:26<04:34, 1.19it/s]
Loading 0:  10%|▉         | 36.0/363 [00:26<04:34, 1.19it/s]
Loading 0:  11%|█         | 39.0/363 [00:28<04:07, 1.31it/s]
Loading 0:  11%|█         | 39.0/363 [00:28<04:07, 1.31it/s]
Loading 0:  11%|█         | 40.0/363 [00:29<04:59, 1.08it/s]
Loading 0:  11%|█         | 40.0/363 [00:29<04:59, 1.08it/s]
Loading 0:  11%|█▏        | 41.0/363 [00:31<05:57, 1.11s/it]
Loading 0:  11%|█▏        | 41.0/363 [00:31<05:57, 1.11s/it]
Loading 0:  13%|█▎        | 48.0/363 [00:34<03:21, 1.56it/s]
Loading 0:  13%|█▎        | 48.0/363 [00:34<03:21, 1.56it/s]
Loading 0:  13%|█▎        | 49.0/363 [00:36<04:05, 1.28it/s]
Loading 0:  13%|█▎        | 49.0/363 [00:36<04:05, 1.28it/s]
Loading 0:  14%|█▍        | 50.0/363 [00:38<04:58, 1.05it/s]
Loading 0:  14%|█▍        | 50.0/363 [00:38<04:58, 1.05it/s]
Loading 0:  16%|█▌        | 57.0/363 [00:40<03:06, 1.65it/s]
Loading 0:  16%|█▌        | 57.0/363 [00:40<03:06, 1.65it/s]
Loading 0:  16%|█▌        | 58.0/363 [00:42<03:46, 1.35it/s]
Loading 0:  16%|█▌        | 58.0/363 [00:42<03:46, 1.35it/s]
Loading 0:  16%|█▋        | 59.0/363 [00:44<04:35, 1.10it/s]
Loading 0:  16%|█▋        | 59.0/363 [00:44<04:35, 1.10it/s]
Loading 0:  18%|█▊        | 65.0/363 [00:47<03:13, 1.54it/s]
Loading 0:  18%|█▊        | 65.0/363 [00:47<03:13, 1.54it/s]
Loading 0:  20%|█▉        | 71.0/363 [00:49<02:44, 1.77it/s]
Loading 0:  20%|█▉        | 71.0/363 [00:49<02:44, 1.77it/s]
Loading 0:  20%|█▉        | 72.0/363 [00:51<03:19, 1.46it/s]
Loading 0:  20%|█▉        | 72.0/363 [00:51<03:19, 1.46it/s]
Loading 0:  20%|██        | 73.0/363 [00:53<04:03, 1.19it/s]
Loading 0:  20%|██        | 73.0/363 [00:53<04:03, 1.19it/s]
Loading 0:  22%|██▏       | 79.0/363 [00:56<02:59, 1.58it/s]
Loading 0:  22%|██▏       | 79.0/363 [00:56<02:59, 1.58it/s]
Loading 0:  22%|██▏       | 80.0/363 [00:58<03:39, 1.29it/s]
Loading 0:  22%|██▏       | 80.0/363 [00:58<03:39, 1.29it/s]
Loading 0:  24%|██▎       | 86.0/363 [01:00<02:49, 1.64it/s]
Loading 0:  24%|██▎       | 86.0/363 [01:00<02:49, 1.64it/s]
Loading 0:  24%|██▍       | 87.0/363 [01:02<03:27, 1.33it/s]
Loading 0:  24%|██▍       | 87.0/363 [01:02<03:27, 1.33it/s]
Loading 0:  25%|██▍       | 90.0/363 [01:04<03:14, 1.40it/s]
Loading 0:  25%|██▍       | 90.0/363 [01:04<03:14, 1.40it/s]
Loading 0:  25%|██▌       | 91.0/363 [01:06<03:54, 1.16it/s]
Loading 0:  25%|██▌       | 91.0/363 [01:06<03:54, 1.16it/s]
Loading 0:  25%|██▌       | 92.0/363 [01:08<04:40, 1.03s/it]
Loading 0:  25%|██▌       | 92.0/363 [01:08<04:40, 1.03s/it]
Loading 0:  27%|██▋       | 99.0/363 [01:11<02:45, 1.59it/s]
Loading 0:  27%|██▋       | 99.0/363 [01:11<02:45, 1.59it/s]
Loading 0:  28%|██▊       | 100/363 [01:13<03:21, 1.31it/s] 
Loading 0:  28%|██▊       | 100/363 [01:13<03:21, 1.31it/s]
Loading 0:  28%|██▊       | 101/363 [01:15<04:03, 1.07it/s]
Loading 0:  28%|██▊       | 101/363 [01:15<04:03, 1.07it/s]
Loading 0:  30%|██▉       | 108/363 [01:17<02:36, 1.63it/s]
Loading 0:  30%|██▉       | 108/363 [01:17<02:36, 1.63it/s]
Loading 0:  31%|███       | 111/363 [01:19<02:35, 1.62it/s]
Loading 0:  31%|███       | 111/363 [01:19<02:35, 1.62it/s]
Loading 0:  31%|███       | 112/363 [01:21<03:09, 1.32it/s]
Loading 0:  31%|███       | 112/363 [01:21<03:09, 1.32it/s]
Loading 0:  31%|███       | 113/363 [01:23<03:51, 1.08it/s]
Loading 0:  31%|███       | 113/363 [01:23<03:51, 1.08it/s]
Loading 0:  33%|███▎      | 120/363 [01:26<02:26, 1.66it/s]
Loading 0:  33%|███▎      | 120/363 [01:26<02:26, 1.66it/s]
Loading 0:  33%|███▎      | 121/363 [01:27<02:58, 1.36it/s]
Loading 0:  33%|███▎      | 121/363 [01:27<02:58, 1.36it/s]
Loading 0:  34%|███▎      | 122/363 [01:29<03:37, 1.11it/s]
Loading 0:  34%|███▎      | 122/363 [01:29<03:37, 1.11it/s]
Loading 0:  36%|███▌      | 129/363 [01:32<02:19, 1.68it/s]
Loading 0:  36%|███▌      | 129/363 [01:32<02:19, 1.68it/s]
Loading 0:  36%|███▌      | 130/363 [01:34<02:49, 1.37it/s]
Loading 0:  36%|███▌      | 130/363 [01:34<02:49, 1.37it/s]
Loading 0:  36%|███▌      | 131/363 [01:36<03:26, 1.12it/s]
Loading 0:  36%|███▌      | 131/363 [01:36<03:26, 1.12it/s]
Loading 0:  38%|███▊      | 138/363 [01:38<02:13, 1.69it/s]
Loading 0:  38%|███▊      | 138/363 [01:38<02:13, 1.69it/s]
Loading 0:  38%|███▊      | 139/363 [01:40<02:42, 1.38it/s]
Loading 0:  38%|███▊      | 139/363 [01:40<02:42, 1.38it/s]
Loading 0:  39%|███▊      | 140/363 [01:42<03:18, 1.12it/s]
Loading 0:  39%|███▊      | 140/363 [01:42<03:18, 1.12it/s]
Loading 0:  41%|████      | 148/363 [01:43<01:35, 2.24it/s]
Loading 0:  41%|████      | 148/363 [01:43<01:35, 2.24it/s]
Loading 0:  42%|████▏     | 151/363 [01:46<01:49, 1.94it/s]
Loading 0:  42%|████▏     | 151/363 [01:46<01:49, 1.94it/s]
Loading 0:  42%|████▏     | 152/363 [01:47<02:18, 1.53it/s]
Loading 0:  42%|████▏     | 152/363 [01:47<02:18, 1.53it/s]
Loading 0:  42%|████▏     | 153/363 [01:49<02:53, 1.21it/s]
Loading 0:  42%|████▏     | 153/363 [01:49<02:53, 1.21it/s]
Loading 0:  43%|████▎     | 156/363 [01:51<02:36, 1.32it/s]
Loading 0:  43%|████▎     | 156/363 [01:51<02:36, 1.32it/s]
Loading 0:  43%|████▎     | 157/363 [01:53<03:09, 1.09it/s]
Loading 0:  43%|████▎     | 157/363 [01:53<03:09, 1.09it/s]
Loading 0:  44%|████▎     | 158/363 [01:55<03:45, 1.10s/it]
Loading 0:  44%|████▎     | 158/363 [01:55<03:45, 1.10s/it]
Loading 0:  45%|████▌     | 165/363 [01:58<02:05, 1.57it/s]
Loading 0:  45%|████▌     | 165/363 [01:58<02:05, 1.57it/s]
Loading 0:  46%|████▌     | 166/363 [02:00<02:32, 1.29it/s]
Loading 0:  46%|████▌     | 166/363 [02:00<02:32, 1.29it/s]
Loading 0:  46%|████▌     | 167/363 [02:02<03:05, 1.06it/s]
Loading 0:  46%|████▌     | 167/363 [02:02<03:05, 1.06it/s]
Loading 0:  48%|████▊     | 174/363 [02:04<01:54, 1.65it/s]
Loading 0:  48%|████▊     | 174/363 [02:04<01:54, 1.65it/s]
Loading 0:  48%|████▊     | 175/363 [02:06<02:18, 1.35it/s]
Loading 0:  48%|████▊     | 175/363 [02:06<02:18, 1.35it/s]
Loading 0:  48%|████▊     | 176/363 [02:08<02:49, 1.10it/s]
Loading 0:  48%|████▊     | 176/363 [02:08<02:49, 1.10it/s]
Loading 0:  50%|█████     | 182/363 [02:11<01:57, 1.54it/s]
Loading 0:  50%|█████     | 182/363 [02:11<01:57, 1.54it/s]
Loading 0:  52%|█████▏    | 188/363 [02:13<01:38, 1.78it/s]
Loading 0:  52%|█████▏    | 188/363 [02:13<01:38, 1.78it/s]
Loading 0:  52%|█████▏    | 189/363 [02:15<02:00, 1.44it/s]
Loading 0:  52%|█████▏    | 189/363 [02:15<02:00, 1.44it/s]
Loading 0:  53%|█████▎    | 192/363 [02:17<01:55, 1.48it/s]
Loading 0:  53%|█████▎    | 192/363 [02:17<01:55, 1.48it/s]
Loading 0:  53%|█████▎    | 193/363 [02:19<02:19, 1.22it/s]
Loading 0:  53%|█████▎    | 193/363 [02:19<02:19, 1.22it/s]
Loading 0:  53%|█████▎    | 194/363 [02:21<02:47, 1.01it/s]
Loading 0:  53%|█████▎    | 194/363 [02:21<02:47, 1.01it/s]
Loading 0:  55%|█████▌    | 201/363 [02:23<01:40, 1.62it/s]
Loading 0:  55%|█████▌    | 201/363 [02:23<01:40, 1.62it/s]
Loading 0:  56%|█████▌    | 202/363 [02:25<02:01, 1.33it/s]
Loading 0:  56%|█████▌    | 202/363 [02:25<02:01, 1.33it/s]
Loading 0:  56%|█████▌    | 203/363 [02:27<02:26, 1.09it/s]
Loading 0:  56%|█████▌    | 203/363 [02:27<02:26, 1.09it/s]
Loading 0:  58%|█████▊    | 210/363 [02:30<01:31, 1.67it/s]
Loading 0:  58%|█████▊    | 210/363 [02:30<01:31, 1.67it/s]
Loading 0:  58%|█████▊    | 211/363 [02:32<01:51, 1.37it/s]
Loading 0:  58%|█████▊    | 211/363 [02:32<01:51, 1.37it/s]
Loading 0:  58%|█████▊    | 212/363 [02:34<02:14, 1.12it/s]
Loading 0:  58%|█████▊    | 212/363 [02:34<02:14, 1.12it/s]
Loading 0:  60%|██████    | 218/363 [02:36<01:33, 1.55it/s]
Loading 0:  60%|██████    | 218/363 [02:36<01:33, 1.55it/s]
Loading 0:  60%|██████    | 219/363 [02:38<01:54, 1.26it/s]
Loading 0:  60%|██████    | 219/363 [02:38<01:54, 1.26it/s]
Loading 0:  62%|██████▏   | 225/363 [02:41<01:25, 1.61it/s]
Loading 0:  62%|██████▏   | 225/363 [02:41<01:25, 1.61it/s]
Loading 0:  63%|██████▎   | 228/363 [02:43<01:24, 1.60it/s]
Loading 0:  63%|██████▎   | 228/363 [02:43<01:24, 1.60it/s]
Loading 0:  63%|██████▎   | 229/363 [02:45<01:41, 1.32it/s]
Loading 0:  63%|██████▎   | 229/363 [02:45<01:41, 1.32it/s]
Loading 0:  63%|██████▎   | 230/363 [02:47<02:02, 1.08it/s]
Loading 0:  63%|██████▎   | 230/363 [02:47<02:02, 1.08it/s]
Loading 0:  65%|██████▌   | 237/363 [02:49<01:15, 1.66it/s]
Loading 0:  65%|██████▌   | 237/363 [02:49<01:15, 1.66it/s]
Loading 0:  66%|██████▌   | 238/363 [02:51<01:31, 1.36it/s]
Loading 0:  66%|██████▌   | 238/363 [02:51<01:31, 1.36it/s]
Loading 0:  66%|██████▌   | 239/363 [02:53<01:51, 1.11it/s]
Loading 0:  66%|██████▌   | 239/363 [02:53<01:51, 1.11it/s]
Loading 0:  68%|██████▊   | 246/363 [02:56<01:09, 1.69it/s]
Loading 0:  68%|██████▊   | 246/363 [02:56<01:09, 1.69it/s]
Loading 0:  68%|██████▊   | 247/363 [02:57<01:24, 1.38it/s]
Loading 0:  68%|██████▊   | 247/363 [02:57<01:24, 1.38it/s]
Loading 0:  68%|██████▊   | 248/363 [02:59<01:41, 1.13it/s]
Loading 0:  68%|██████▊   | 248/363 [02:59<01:41, 1.13it/s]
Loading 0:  70%|███████   | 255/363 [03:02<01:03, 1.69it/s]
Loading 0:  70%|███████   | 255/363 [03:02<01:03, 1.69it/s]
Loading 0:  71%|███████   | 256/363 [03:04<01:17, 1.38it/s]
Loading 0:  71%|███████   | 256/363 [03:04<01:17, 1.38it/s]
Loading 0:  71%|███████   | 257/363 [03:06<01:33, 1.13it/s]
Loading 0:  71%|███████   | 257/363 [03:06<01:33, 1.13it/s]
Loading 0:  73%|███████▎  | 265/363 [03:07<00:43, 2.23it/s]
Loading 0:  73%|███████▎  | 265/363 [03:07<00:43, 2.23it/s]
Loading 0:  74%|███████▍  | 268/363 [03:09<00:49, 1.92it/s]
Loading 0:  74%|███████▍  | 268/363 [03:09<00:49, 1.92it/s]
Loading 0:  74%|███████▍  | 269/363 [03:11<01:02, 1.51it/s]
Loading 0:  74%|███████▍  | 269/363 [03:11<01:02, 1.51it/s]
Loading 0:  74%|███████▍  | 270/363 [03:13<01:17, 1.21it/s]
Loading 0:  74%|███████▍  | 270/363 [03:13<01:17, 1.21it/s]
Loading 0:  75%|███████▌  | 273/363 [03:28<03:26, 2.29s/it]
Loading 0:  75%|███████▌  | 273/363 [03:28<03:26, 2.29s/it]
Loading 0:  75%|███████▌  | 274/363 [03:30<03:18, 2.23s/it]
Loading 0:  75%|███████▌  | 274/363 [03:30<03:18, 2.23s/it]
Loading 0:  76%|███████▌  | 275/363 [03:32<03:13, 2.20s/it]
Loading 0:  76%|███████▌  | 275/363 [03:32<03:13, 2.20s/it]
Loading 0:  78%|███████▊  | 282/363 [03:34<01:23, 1.03s/it]
Loading 0:  78%|███████▊  | 282/363 [03:34<01:23, 1.03s/it]
Loading 0:  78%|███████▊  | 283/363 [03:36<01:28, 1.11s/it]
Loading 0:  78%|███████▊  | 283/363 [03:36<01:28, 1.11s/it]
Loading 0:  78%|███████▊  | 284/363 [03:38<01:36, 1.22s/it]
Loading 0:  78%|███████▊  | 284/363 [03:38<01:36, 1.22s/it]
Loading 0:  80%|████████  | 291/363 [03:40<00:51, 1.39it/s]
Loading 0:  80%|████████  | 291/363 [03:40<00:51, 1.39it/s]
Loading 0:  80%|████████  | 292/363 [03:42<00:59, 1.19it/s]
Loading 0:  80%|████████  | 292/363 [03:42<00:59, 1.19it/s]
Loading 0:  81%|████████  | 293/363 [03:44<01:09, 1.01it/s]
Loading 0:  81%|████████  | 293/363 [03:44<01:09, 1.01it/s]
Loading 0:  82%|████████▏ | 299/363 [03:47<00:43, 1.46it/s]
Loading 0:  82%|████████▏ | 299/363 [03:47<00:43, 1.46it/s]
Loading 0:  84%|████████▍ | 305/363 [03:49<00:33, 1.72it/s]
Loading 0:  84%|████████▍ | 305/363 [03:49<00:33, 1.72it/s]
Loading 0:  84%|████████▍ | 306/363 [03:51<00:40, 1.40it/s]
Loading 0:  84%|████████▍ | 306/363 [03:51<00:40, 1.40it/s]
Loading 0:  85%|████████▌ | 309/363 [03:53<00:37, 1.45it/s]
Loading 0:  85%|████████▌ | 309/363 [03:53<00:37, 1.45it/s]
Loading 0:  85%|████████▌ | 310/363 [03:55<00:43, 1.21it/s]
Loading 0:  85%|████████▌ | 310/363 [03:55<00:43, 1.21it/s]
Loading 0:  86%|████████▌ | 311/363 [03:57<00:52, 1.00s/it]
Loading 0:  86%|████████▌ | 311/363 [03:57<00:52, 1.00s/it]
Loading 0:  88%|████████▊ | 318/363 [04:00<00:27, 1.61it/s]
Loading 0:  88%|████████▊ | 318/363 [04:00<00:27, 1.61it/s]
Loading 0:  88%|████████▊ | 319/363 [04:02<00:33, 1.32it/s]
Loading 0:  88%|████████▊ | 319/363 [04:02<00:33, 1.32it/s]
Loading 0:  88%|████████▊ | 320/363 [04:04<00:39, 1.09it/s]
Loading 0:  88%|████████▊ | 320/363 [04:04<00:39, 1.09it/s]
Loading 0:  90%|█████████ | 327/363 [04:06<00:21, 1.67it/s]
Loading 0:  90%|█████████ | 327/363 [04:06<00:21, 1.67it/s]
Loading 0:  90%|█████████ | 328/363 [04:08<00:25, 1.36it/s]
Loading 0:  90%|█████████ | 328/363 [04:08<00:25, 1.36it/s]
Loading 0:  91%|█████████ | 329/363 [04:10<00:30, 1.12it/s]
Loading 0:  91%|█████████ | 329/363 [04:10<00:30, 1.12it/s]
Loading 0:  92%|█████████▏| 335/363 [04:12<00:18, 1.55it/s]
Loading 0:  92%|█████████▏| 335/363 [04:12<00:18, 1.55it/s]
Loading 0:  93%|█████████▎| 336/363 [04:14<00:21, 1.25it/s]
Loading 0:  93%|█████████▎| 336/363 [04:14<00:21, 1.25it/s]
Loading 0:  94%|█████████▍| 343/363 [04:17<00:11, 1.69it/s]
Loading 0:  94%|█████████▍| 343/363 [04:17<00:11, 1.69it/s]
Loading 0:  95%|█████████▌| 346/363 [04:19<00:10, 1.67it/s]
Loading 0:  95%|█████████▌| 346/363 [04:19<00:10, 1.67it/s]
Loading 0:  96%|█████████▌| 347/363 [04:21<00:11, 1.36it/s]
Loading 0:  96%|█████████▌| 347/363 [04:21<00:11, 1.36it/s]
Loading 0:  96%|█████████▌| 348/363 [04:23<00:13, 1.12it/s]
Loading 0:  96%|█████████▌| 348/363 [04:23<00:13, 1.12it/s]
Loading 0:  98%|█████████▊| 355/363 [04:26<00:04, 1.68it/s]
Loading 0:  98%|█████████▊| 355/363 [04:26<00:04, 1.68it/s]
Loading 0:  98%|█████████▊| 356/363 [04:27<00:05, 1.37it/s]
Loading 0:  98%|█████████▊| 356/363 [04:27<00:05, 1.37it/s]
Loading 0:  98%|█████████▊| 357/363 [04:29<00:05, 1.12it/s]
Loading 0:  98%|█████████▊| 357/363 [04:29<00:05, 1.12it/s]
Loading 0: 100%|██████████| 363/363 [04:30<00:00, 2.05it/s]
Loading 0: 100%|██████████| 363/363 [04:30<00:00, 2.05it/s]
                                                           

Loading 0: 100%|██████████| 363/363 [04:30<00:00, 1.34it/s]
mistralai-mistral-smal-88026-v68-mkmlizer: The tokenizer you are loading from '/tmp/tmpjbd6w5x8' with an incorrect regex pattern: https://huggingface.co/mistralai/Mistral-Small-3.1-24B-Instruct-2503/discussions/84#69121093e8b480e709447d5e. This will lead to incorrect tokenization. You should set the `fix_mistral_regex=True` flag when loading this tokenizer to fix this issue.
mistralai-mistral-smal-88026-v68-mkmlizer: quantized model in 277.117s
mistralai-mistral-smal-88026-v68-mkmlizer: Processed model mistralai/Mistral-Small-24B-Base-2501 in 350.696s
mistralai-mistral-smal-88026-v68-mkmlizer: creating bucket guanaco-mkml-models
mistralai-mistral-smal-88026-v68-mkmlizer: Bucket 's3://guanaco-mkml-models/' created
mistralai-mistral-smal-88026-v68-mkmlizer: uploading /dev/shm/model_cache to s3://guanaco-mkml-models/mistralai-mistral-smal-88026-v68/nvidia
mistralai-mistral-smal-88026-v68-mkmlizer: cp /dev/shm/model_cache/config.json s3://guanaco-mkml-models/mistralai-mistral-smal-88026-v68/nvidia/config.json
mistralai-mistral-smal-88026-v68-mkmlizer: cp /dev/shm/model_cache/special_tokens_map.json s3://guanaco-mkml-models/mistralai-mistral-smal-88026-v68/nvidia/special_tokens_map.json
mistralai-mistral-smal-88026-v68-mkmlizer: cp /dev/shm/model_cache/tokenizer_config.json s3://guanaco-mkml-models/mistralai-mistral-smal-88026-v68/nvidia/tokenizer_config.json
mistralai-mistral-smal-88026-v68-mkmlizer: cp /dev/shm/model_cache/tokenizer.json s3://guanaco-mkml-models/mistralai-mistral-smal-88026-v68/nvidia/tokenizer.json
mistralai-mistral-smal-88026-v68-mkmlizer: cp /dev/shm/model_cache/flywheel_model.1.safetensors s3://guanaco-mkml-models/mistralai-mistral-smal-88026-v68/nvidia/flywheel_model.1.safetensors
chaiml-mistral-24b-2048-41286-v1-mkmlizer: 
Loading 0:   0%|          | 0.00/363 [00:00<?, ?it/s]
Loading 0:   1%|          | 3.00/363 [00:01<03:54, 1.53it/s]
Loading 0:   1%|          | 3.00/363 [00:01<03:54, 1.53it/s]
Loading 0:   1%|          | 4.00/363 [00:03<06:19, 1.06s/it]
Loading 0:   1%|          | 4.00/363 [00:03<06:19, 1.06s/it]
Loading 0:   1%|▏         | 5.00/363 [00:05<08:08, 1.37s/it]
Loading 0:   1%|▏         | 5.00/363 [00:05<08:08, 1.37s/it]
Loading 0:   3%|▎         | 12.0/363 [00:07<02:40, 2.19it/s]
Loading 0:   3%|▎         | 12.0/363 [00:07<02:40, 2.19it/s]
Loading 0:   4%|▎         | 13.0/363 [00:08<03:48, 1.53it/s]
Loading 0:   4%|▎         | 13.0/363 [00:08<03:48, 1.53it/s]
Loading 0:   4%|▍         | 14.0/363 [00:10<04:57, 1.17it/s]
Loading 0:   4%|▍         | 14.0/363 [00:10<04:57, 1.17it/s]
Loading 0:   4%|▍         | 15.0/363 [00:12<06:12, 1.07s/it]
Loading 0:   4%|▍         | 15.0/363 [00:12<06:12, 1.07s/it]
Loading 0:   6%|▌         | 21.0/363 [00:15<03:43, 1.53it/s]
Loading 0:   6%|▌         | 21.0/363 [00:15<03:43, 1.53it/s]
Loading 0:   6%|▌         | 22.0/363 [00:17<04:37, 1.23it/s]
Loading 0:   6%|▌         | 22.0/363 [00:17<04:37, 1.23it/s]
Loading 0:   6%|▋         | 23.0/363 [00:19<05:39, 1.00it/s]
Loading 0:   6%|▋         | 23.0/363 [00:19<05:39, 1.00it/s]
Loading 0:   9%|▊         | 31.0/363 [00:20<02:31, 2.19it/s]
Loading 0:   9%|▊         | 31.0/363 [00:20<02:31, 2.19it/s]
Loading 0:   9%|▉         | 34.0/363 [00:22<02:54, 1.89it/s]
Loading 0:   9%|▉         | 34.0/363 [00:22<02:54, 1.89it/s]
Loading 0:  10%|▉         | 35.0/363 [00:24<03:42, 1.47it/s]
Loading 0:  10%|▉         | 35.0/363 [00:24<03:42, 1.47it/s]
Loading 0:  10%|▉         | 36.0/363 [00:26<04:43, 1.15it/s]
Loading 0:  10%|▉         | 36.0/363 [00:26<04:43, 1.15it/s]
Loading 0:  11%|█         | 39.0/363 [00:28<04:14, 1.27it/s]
Loading 0:  11%|█         | 39.0/363 [00:28<04:14, 1.27it/s]
Loading 0:  11%|█         | 40.0/363 [00:30<05:06, 1.05it/s]
Loading 0:  11%|█         | 40.0/363 [00:30<05:06, 1.05it/s]
Loading 0:  11%|█▏        | 41.0/363 [00:32<06:05, 1.14s/it]
Loading 0:  11%|█▏        | 41.0/363 [00:32<06:05, 1.14s/it]
Loading 0:  13%|█▎        | 49.0/363 [00:33<02:29, 2.10it/s]
Loading 0:  13%|█▎        | 49.0/363 [00:33<02:29, 2.10it/s]
Loading 0:  14%|█▍        | 52.0/363 [00:35<02:50, 1.83it/s]
Loading 0:  14%|█▍        | 52.0/363 [00:35<02:50, 1.83it/s]
Loading 0:  15%|█▍        | 53.0/363 [00:37<03:36, 1.43it/s]
Loading 0:  15%|█▍        | 53.0/363 [00:37<03:36, 1.43it/s]
Loading 0:  15%|█▍        | 54.0/363 [00:39<04:31, 1.14it/s]
Loading 0:  15%|█▍        | 54.0/363 [00:39<04:31, 1.14it/s]
Loading 0:  16%|█▌        | 57.0/363 [00:41<04:01, 1.27it/s]
Loading 0:  16%|█▌        | 57.0/363 [00:41<04:01, 1.27it/s]
Loading 0:  16%|█▌        | 58.0/363 [00:43<04:51, 1.05it/s]
Loading 0:  16%|█▌        | 58.0/363 [00:43<04:51, 1.05it/s]
Loading 0:  16%|█▋        | 59.0/363 [00:45<05:48, 1.15s/it]
Loading 0:  16%|█▋        | 59.0/363 [00:45<05:48, 1.15s/it]
Loading 0:  18%|█▊        | 67.0/363 [00:46<02:22, 2.08it/s]
Loading 0:  18%|█▊        | 67.0/363 [00:46<02:22, 2.08it/s]
Loading 0:  19%|█▉        | 70.0/363 [00:49<02:40, 1.82it/s]
Loading 0:  19%|█▉        | 70.0/363 [00:49<02:40, 1.82it/s]
Loading 0:  20%|█▉        | 71.0/363 [00:50<03:23, 1.43it/s]
Loading 0:  20%|█▉        | 71.0/363 [00:50<03:23, 1.43it/s]
Loading 0:  20%|█▉        | 72.0/363 [00:52<04:16, 1.13it/s]
Loading 0:  20%|█▉        | 72.0/363 [00:52<04:16, 1.13it/s]
Loading 0:  21%|██        | 75.0/363 [00:54<03:47, 1.26it/s]
Loading 0:  21%|██        | 75.0/363 [00:54<03:47, 1.26it/s]
Loading 0:  21%|██        | 76.0/363 [00:56<04:34, 1.04it/s]
Loading 0:  21%|██        | 76.0/363 [00:56<04:34, 1.04it/s]
Loading 0:  21%|██        | 77.0/363 [00:58<05:26, 1.14s/it]
Loading 0:  21%|██        | 77.0/363 [00:58<05:26, 1.14s/it]
Loading 0:  23%|██▎       | 85.0/363 [00:59<02:13, 2.09it/s]
Loading 0:  23%|██▎       | 85.0/363 [00:59<02:13, 2.09it/s]
Loading 0:  24%|██▍       | 88.0/363 [01:02<02:30, 1.83it/s]
Loading 0:  24%|██▍       | 88.0/363 [01:02<02:30, 1.83it/s]
Loading 0:  25%|██▍       | 89.0/363 [01:04<03:11, 1.43it/s]
Loading 0:  25%|██▍       | 89.0/363 [01:04<03:11, 1.43it/s]
Loading 0:  25%|██▍       | 90.0/363 [01:06<04:00, 1.14it/s]
Loading 0:  25%|██▍       | 90.0/363 [01:06<04:00, 1.14it/s]
Loading 0:  27%|██▋       | 98.0/363 [01:07<01:55, 2.30it/s]
Loading 0:  27%|██▋       | 98.0/363 [01:07<01:55, 2.30it/s]
Loading 0:  28%|██▊       | 101/363 [01:09<02:10, 2.01it/s] 
Loading 0:  28%|██▊       | 101/363 [01:09<02:10, 2.01it/s]
Loading 0:  28%|██▊       | 102/363 [01:11<02:48, 1.55it/s]
Loading 0:  28%|██▊       | 102/363 [01:11<02:48, 1.55it/s]
Loading 0:  28%|██▊       | 103/363 [01:13<03:34, 1.21it/s]
Loading 0:  28%|██▊       | 103/363 [01:13<03:34, 1.21it/s]
Loading 0:  29%|██▉       | 106/363 [01:15<03:18, 1.29it/s]
Loading 0:  29%|██▉       | 106/363 [01:15<03:18, 1.29it/s]
Loading 0:  29%|██▉       | 107/363 [01:17<04:00, 1.07it/s]
Loading 0:  29%|██▉       | 107/363 [01:17<04:00, 1.07it/s]
Loading 0:  30%|██▉       | 108/363 [01:19<04:46, 1.12s/it]
Loading 0:  30%|██▉       | 108/363 [01:19<04:46, 1.12s/it]
Loading 0:  31%|███       | 111/363 [01:21<03:51, 1.09it/s]
Loading 0:  31%|███       | 111/363 [01:21<03:51, 1.09it/s]
Loading 0:  31%|███       | 112/363 [01:23<04:32, 1.08s/it]
Loading 0:  31%|███       | 112/363 [01:23<04:32, 1.08s/it]
Loading 0:  31%|███       | 113/363 [01:25<05:16, 1.27s/it]
Loading 0:  31%|███       | 113/363 [01:25<05:16, 1.27s/it]
Loading 0:  33%|███▎      | 121/363 [01:26<01:59, 2.02it/s]
Loading 0:  33%|███▎      | 121/363 [01:26<01:59, 2.02it/s]
Loading 0:  34%|███▍      | 124/363 [01:28<02:13, 1.79it/s]
Loading 0:  34%|███▍      | 124/363 [01:28<02:13, 1.79it/s]
Loading 0:  34%|███▍      | 125/363 [01:30<02:49, 1.40it/s]
Loading 0:  34%|███▍      | 125/363 [01:30<02:49, 1.40it/s]
Loading 0:  35%|███▍      | 126/363 [01:32<03:31, 1.12it/s]
Loading 0:  35%|███▍      | 126/363 [01:32<03:31, 1.12it/s]
Loading 0:  36%|███▌      | 129/363 [01:34<03:06, 1.25it/s]
Loading 0:  36%|███▌      | 129/363 [01:34<03:06, 1.25it/s]
Loading 0:  36%|███▌      | 130/363 [01:36<03:44, 1.04it/s]
Loading 0:  36%|███▌      | 130/363 [01:36<03:44, 1.04it/s]
Loading 0:  36%|███▌      | 131/363 [01:38<04:27, 1.15s/it]
Loading 0:  36%|███▌      | 131/363 [01:38<04:27, 1.15s/it]
Loading 0:  38%|███▊      | 139/363 [01:39<01:47, 2.08it/s]
Loading 0:  38%|███▊      | 139/363 [01:39<01:47, 2.08it/s]
Loading 0:  39%|███▉      | 142/363 [01:41<02:01, 1.82it/s]
Loading 0:  39%|███▉      | 142/363 [01:41<02:01, 1.82it/s]
Loading 0:  39%|███▉      | 143/363 [01:43<02:34, 1.43it/s]
Loading 0:  39%|███▉      | 143/363 [01:43<02:34, 1.43it/s]
Loading 0:  40%|███▉      | 144/363 [01:45<03:12, 1.14it/s]
Loading 0:  40%|███▉      | 144/363 [01:45<03:12, 1.14it/s]
Loading 0:  40%|████      | 147/363 [01:47<02:50, 1.26it/s]
Loading 0:  40%|████      | 147/363 [01:47<02:50, 1.26it/s]
Loading 0:  41%|████      | 148/363 [01:49<03:25, 1.04it/s]
Loading 0:  41%|████      | 148/363 [01:49<03:25, 1.04it/s]
Loading 0:  41%|████      | 149/363 [01:51<04:04, 1.14s/it]
Loading 0:  41%|████      | 149/363 [01:51<04:04, 1.14s/it]
Loading 0:  43%|████▎     | 157/363 [01:52<01:38, 2.10it/s]
Loading 0:  43%|████▎     | 157/363 [01:52<01:38, 2.10it/s]
Loading 0:  44%|████▍     | 160/363 [01:54<01:50, 1.83it/s]
Loading 0:  44%|████▍     | 160/363 [01:54<01:50, 1.83it/s]
Loading 0:  44%|████▍     | 161/363 [01:56<02:20, 1.43it/s]
Loading 0:  44%|████▍     | 161/363 [01:56<02:20, 1.43it/s]
Loading 0:  45%|████▍     | 162/363 [01:58<02:56, 1.14it/s]
Loading 0:  45%|████▍     | 162/363 [01:58<02:56, 1.14it/s]
Loading 0:  45%|████▌     | 165/363 [02:00<02:36, 1.27it/s]
Loading 0:  45%|████▌     | 165/363 [02:00<02:36, 1.27it/s]
Loading 0:  46%|████▌     | 166/363 [02:02<03:08, 1.05it/s]
Loading 0:  46%|████▌     | 166/363 [02:02<03:08, 1.05it/s]
Loading 0:  46%|████▌     | 167/363 [02:04<03:43, 1.14s/it]
Loading 0:  46%|████▌     | 167/363 [02:04<03:43, 1.14s/it]
Loading 0:  48%|████▊     | 175/363 [02:05<01:30, 2.08it/s]
Loading 0:  48%|████▊     | 175/363 [02:05<01:30, 2.08it/s]
Loading 0:  49%|████▉     | 178/363 [02:07<01:41, 1.83it/s]
Loading 0:  49%|████▉     | 178/363 [02:07<01:41, 1.83it/s]
Loading 0:  49%|████▉     | 179/363 [02:09<02:08, 1.43it/s]
Loading 0:  49%|████▉     | 179/363 [02:09<02:08, 1.43it/s]
Loading 0:  50%|████▉     | 180/363 [02:11<02:41, 1.14it/s]
Loading 0:  50%|████▉     | 180/363 [02:11<02:41, 1.14it/s]
Loading 0:  50%|█████     | 183/363 [02:13<02:22, 1.27it/s]
Loading 0:  50%|█████     | 183/363 [02:13<02:22, 1.27it/s]
Loading 0:  51%|█████     | 184/363 [02:15<02:51, 1.05it/s]
Loading 0:  51%|█████     | 184/363 [02:15<02:51, 1.05it/s]
Loading 0:  51%|█████     | 185/363 [02:17<03:23, 1.14s/it]
Loading 0:  51%|█████     | 185/363 [02:17<03:23, 1.14s/it]
Loading 0:  53%|█████▎    | 193/363 [02:18<01:21, 2.08it/s]
Loading 0:  53%|█████▎    | 193/363 [02:18<01:21, 2.08it/s]
Loading 0:  54%|█████▍    | 196/363 [02:21<01:31, 1.83it/s]
Loading 0:  54%|█████▍    | 196/363 [02:21<01:31, 1.83it/s]
Loading 0:  54%|█████▍    | 197/363 [02:22<01:55, 1.43it/s]
Loading 0:  54%|█████▍    | 197/363 [02:22<01:55, 1.43it/s]
Loading 0:  55%|█████▍    | 198/363 [02:25<02:25, 1.14it/s]
Loading 0:  55%|█████▍    | 198/363 [02:25<02:25, 1.14it/s]
Loading 0:  55%|█████▌    | 201/363 [02:26<02:07, 1.27it/s]
Loading 0:  55%|█████▌    | 201/363 [02:26<02:07, 1.27it/s]
Loading 0:  56%|█████▌    | 202/363 [02:28<02:33, 1.05it/s]
Loading 0:  56%|█████▌    | 202/363 [02:28<02:33, 1.05it/s]
Loading 0:  56%|█████▌    | 203/363 [02:30<03:02, 1.14s/it]
Loading 0:  56%|█████▌    | 203/363 [02:30<03:02, 1.14s/it]
Loading 0:  58%|█████▊    | 211/363 [02:32<01:13, 2.08it/s]
Loading 0:  58%|█████▊    | 211/363 [02:32<01:13, 2.08it/s]
Loading 0:  59%|█████▉    | 214/363 [02:34<01:21, 1.83it/s]
Loading 0:  59%|█████▉    | 214/363 [02:34<01:21, 1.83it/s]
Loading 0:  59%|█████▉    | 215/363 [02:36<01:43, 1.43it/s]
Loading 0:  59%|█████▉    | 215/363 [02:36<01:43, 1.43it/s]
Loading 0:  60%|█████▉    | 216/363 [02:38<02:09, 1.14it/s]
Loading 0:  60%|█████▉    | 216/363 [02:38<02:09, 1.14it/s]
Loading 0:  60%|██████    | 219/363 [02:40<01:53, 1.27it/s]
Loading 0:  60%|██████    | 219/363 [02:40<01:53, 1.27it/s]
Loading 0:  61%|██████    | 220/363 [02:41<02:16, 1.05it/s]
Loading 0:  61%|██████    | 220/363 [02:41<02:16, 1.05it/s]
Loading 0:  61%|██████    | 221/363 [02:43<02:42, 1.14s/it]
Loading 0:  61%|██████    | 221/363 [02:43<02:42, 1.14s/it]
Loading 0:  63%|██████▎   | 229/363 [02:45<01:03, 2.10it/s]
Loading 0:  63%|██████▎   | 229/363 [02:45<01:03, 2.10it/s]
Loading 0:  64%|██████▍   | 232/363 [02:47<01:11, 1.84it/s]
Loading 0:  64%|██████▍   | 232/363 [02:47<01:11, 1.84it/s]
Loading 0:  64%|██████▍   | 233/363 [02:49<01:30, 1.44it/s]
Loading 0:  64%|██████▍   | 233/363 [02:49<01:30, 1.44it/s]
Loading 0:  64%|██████▍   | 234/363 [02:51<01:52, 1.14it/s]
Loading 0:  64%|██████▍   | 234/363 [02:51<01:52, 1.14it/s]
Loading 0:  65%|██████▌   | 237/363 [02:53<01:39, 1.27it/s]
Loading 0:  65%|██████▌   | 237/363 [02:53<01:39, 1.27it/s]
Loading 0:  66%|██████▌   | 238/363 [02:55<01:59, 1.05it/s]
Loading 0:  66%|██████▌   | 238/363 [02:55<01:59, 1.05it/s]
Loading 0:  66%|██████▌   | 239/363 [02:57<02:21, 1.14s/it]
Loading 0:  66%|██████▌   | 239/363 [02:57<02:21, 1.14s/it]
Loading 0:  68%|██████▊   | 247/363 [02:58<00:55, 2.09it/s]
Loading 0:  68%|██████▊   | 247/363 [02:58<00:55, 2.09it/s]
Loading 0:  69%|██████▉   | 250/363 [03:00<01:01, 1.83it/s]
Loading 0:  69%|██████▉   | 250/363 [03:00<01:01, 1.83it/s]
Loading 0:  69%|██████▉   | 251/363 [03:02<01:18, 1.44it/s]
Loading 0:  69%|██████▉   | 251/363 [03:02<01:18, 1.44it/s]
Loading 0:  69%|██████▉   | 252/363 [03:04<01:37, 1.14it/s]
Loading 0:  69%|██████▉   | 252/363 [03:04<01:37, 1.14it/s]
Loading 0:  70%|███████   | 255/363 [03:06<01:25, 1.27it/s]
Loading 0:  70%|███████   | 255/363 [03:06<01:25, 1.27it/s]
Loading 0:  71%|███████   | 256/363 [03:08<01:42, 1.05it/s]
Loading 0:  71%|███████   | 256/363 [03:08<01:42, 1.05it/s]
Loading 0:  71%|███████   | 257/363 [03:10<02:01, 1.14s/it]
Loading 0:  71%|███████   | 257/363 [03:10<02:01, 1.14s/it]
Loading 0:  73%|███████▎  | 265/363 [03:11<00:46, 2.10it/s]
Loading 0:  73%|███████▎  | 265/363 [03:11<00:46, 2.10it/s]
Loading 0:  74%|███████▍  | 268/363 [03:13<00:51, 1.84it/s]
Loading 0:  74%|███████▍  | 268/363 [03:13<00:51, 1.84it/s]
Loading 0:  74%|███████▍  | 269/363 [03:15<01:05, 1.44it/s]
Loading 0:  74%|███████▍  | 269/363 [03:15<01:05, 1.44it/s]
Loading 0:  74%|███████▍  | 270/363 [03:17<01:21, 1.14it/s]
Loading 0:  74%|███████▍  | 270/363 [03:17<01:21, 1.14it/s]
Loading 0:  75%|███████▌  | 273/363 [03:32<03:36, 2.41s/it]
Loading 0:  75%|███████▌  | 273/363 [03:32<03:36, 2.41s/it]
Loading 0:  75%|███████▌  | 274/363 [03:34<03:27, 2.33s/it]
Loading 0:  75%|███████▌  | 274/363 [03:34<03:27, 2.33s/it]
Loading 0:  76%|███████▌  | 275/363 [03:36<03:20, 2.28s/it]
Loading 0:  76%|███████▌  | 275/363 [03:36<03:20, 2.28s/it]
Loading 0:  78%|███████▊  | 283/363 [03:37<01:08, 1.17it/s]
Loading 0:  78%|███████▊  | 283/363 [03:37<01:08, 1.17it/s]
Loading 0:  79%|███████▉  | 286/363 [03:39<01:03, 1.22it/s]
Loading 0:  79%|███████▉  | 286/363 [03:39<01:03, 1.22it/s]
Loading 0:  79%|███████▉  | 287/363 [03:41<01:11, 1.06it/s]
Loading 0:  79%|███████▉  | 287/363 [03:41<01:11, 1.06it/s]
Loading 0:  79%|███████▉  | 288/363 [03:43<01:21, 1.09s/it]
Loading 0:  79%|███████▉  | 288/363 [03:43<01:21, 1.09s/it]
Loading 0:  80%|████████  | 291/363 [03:45<01:06, 1.08it/s]
Loading 0:  80%|████████  | 291/363 [03:45<01:06, 1.08it/s]
Loading 0:  80%|████████  | 292/363 [03:47<01:15, 1.07s/it]
Loading 0:  80%|████████  | 292/363 [03:47<01:15, 1.07s/it]
Loading 0:  81%|████████  | 293/363 [03:49<01:26, 1.23s/it]
Loading 0:  81%|████████  | 293/363 [03:49<01:26, 1.23s/it]
Loading 0:  83%|████████▎ | 301/363 [03:50<00:31, 1.99it/s]
Loading 0:  83%|████████▎ | 301/363 [03:50<00:31, 1.99it/s]
Loading 0:  84%|████████▎ | 304/363 [03:52<00:33, 1.77it/s]
Loading 0:  84%|████████▎ | 304/363 [03:52<00:33, 1.77it/s]
Loading 0:  84%|████████▍ | 305/363 [03:54<00:41, 1.39it/s]
Loading 0:  84%|████████▍ | 305/363 [03:54<00:41, 1.39it/s]
Loading 0:  84%|████████▍ | 306/363 [03:56<00:51, 1.11it/s]
Loading 0:  84%|████████▍ | 306/363 [03:56<00:51, 1.11it/s]
Loading 0:  85%|████████▌ | 309/363 [03:58<00:43, 1.25it/s]
Loading 0:  85%|████████▌ | 309/363 [03:58<00:43, 1.25it/s]
Loading 0:  85%|████████▌ | 310/363 [04:00<00:51, 1.04it/s]
Loading 0:  85%|████████▌ | 310/363 [04:00<00:51, 1.04it/s]
Loading 0:  86%|████████▌ | 311/363 [04:02<00:59, 1.15s/it]
Loading 0:  86%|████████▌ | 311/363 [04:02<00:59, 1.15s/it]
Loading 0:  88%|████████▊ | 319/363 [04:03<00:20, 2.10it/s]
Loading 0:  88%|████████▊ | 319/363 [04:03<00:20, 2.10it/s]
Loading 0:  89%|████████▊ | 322/363 [04:06<00:22, 1.84it/s]
Loading 0:  89%|████████▊ | 322/363 [04:06<00:22, 1.84it/s]
Loading 0:  89%|████████▉ | 323/363 [04:07<00:27, 1.44it/s]
Loading 0:  89%|████████▉ | 323/363 [04:07<00:27, 1.44it/s]
Loading 0:  89%|████████▉ | 324/363 [04:09<00:34, 1.14it/s]
Loading 0:  89%|████████▉ | 324/363 [04:09<00:34, 1.14it/s]
Loading 0:  90%|█████████ | 327/363 [04:11<00:28, 1.27it/s]
Loading 0:  90%|█████████ | 327/363 [04:11<00:28, 1.27it/s]
Loading 0:  90%|█████████ | 328/363 [04:13<00:33, 1.05it/s]
Loading 0:  90%|█████████ | 328/363 [04:13<00:33, 1.05it/s]
Loading 0:  91%|█████████ | 329/363 [04:15<00:38, 1.14s/it]
Loading 0:  91%|█████████ | 329/363 [04:15<00:38, 1.14s/it]
Loading 0:  93%|█████████▎| 337/363 [04:16<00:12, 2.11it/s]
Loading 0:  93%|█████████▎| 337/363 [04:16<00:12, 2.11it/s]
Loading 0:  94%|█████████▎| 340/363 [04:19<00:12, 1.84it/s]
Loading 0:  94%|█████████▎| 340/363 [04:19<00:12, 1.84it/s]
Loading 0:  94%|█████████▍| 341/363 [04:20<00:15, 1.44it/s]
Loading 0:  94%|█████████▍| 341/363 [04:20<00:15, 1.44it/s]
Loading 0:  94%|█████████▍| 342/363 [04:23<00:18, 1.14it/s]
Loading 0:  94%|█████████▍| 342/363 [04:23<00:18, 1.14it/s]
Loading 0:  95%|█████████▌| 345/363 [04:24<00:14, 1.27it/s]
Loading 0:  95%|█████████▌| 345/363 [04:24<00:14, 1.27it/s]
Loading 0:  95%|█████████▌| 346/363 [04:26<00:16, 1.05it/s]
Loading 0:  95%|█████████▌| 346/363 [04:26<00:16, 1.05it/s]
Loading 0:  96%|█████████▌| 347/363 [04:28<00:18, 1.14s/it]
Loading 0:  96%|█████████▌| 347/363 [04:28<00:18, 1.14s/it]
Loading 0:  98%|█████████▊| 355/363 [04:29<00:03, 2.10it/s]
Loading 0:  98%|█████████▊| 355/363 [04:29<00:03, 2.10it/s]
Loading 0:  99%|█████████▉| 359/363 [04:32<00:02, 1.89it/s]
Loading 0:  99%|█████████▉| 359/363 [04:32<00:02, 1.89it/s]
Loading 0:  99%|█████████▉| 360/363 [04:34<00:02, 1.49it/s]
Loading 0:  99%|█████████▉| 360/363 [04:34<00:02, 1.49it/s]
Loading 0:  99%|█████████▉| 361/363 [04:36<00:01, 1.18it/s]
Loading 0:  99%|█████████▉| 361/363 [04:36<00:01, 1.18it/s]
Loading 0: 100%|██████████| 363/363 [04:36<00:00, 1.18it/s]
                                                           

Loading 0: 100%|██████████| 363/363 [04:36<00:00, 1.31it/s]
mistralai-mistral-smal-88026-v68-mkmlizer: cp /dev/shm/model_cache/flywheel_model.0.safetensors s3://guanaco-mkml-models/mistralai-mistral-smal-88026-v68/nvidia/flywheel_model.0.safetensors
Job mistralai-mistral-smal-88026-v68-mkmlizer completed after 424.08s with status: succeeded
Stopping job with name mistralai-mistral-smal-88026-v68-mkmlizer
Pipeline stage MKMLizer completed in 426.11s
run pipeline stage %s
Running pipeline stage MKMLTemplater
Pipeline stage MKMLTemplater completed in 0.71s
run pipeline stage %s
Running pipeline stage MKMLDeployer
Creating inference service mistralai-mistral-smal-88026-v68
chaiml-mistral-24b-2048-41286-v1-mkmlizer: The tokenizer you are loading from '/tmp/tmpyhgbbh9k' with an incorrect regex pattern: https://huggingface.co/mistralai/Mistral-Small-3.1-24B-Instruct-2503/discussions/84#69121093e8b480e709447d5e. This will lead to incorrect tokenization. You should set the `fix_mistral_regex=True` flag when loading this tokenizer to fix this issue.
Waiting for inference service mistralai-mistral-smal-88026-v68 to be ready
chaiml-mistral-24b-2048-41286-v1-mkmlizer: quantized model in 283.300s
chaiml-mistral-24b-2048-41286-v1-mkmlizer: Processed model ChaiML/mistral_24b_2048_gemini_ds_v3_2187_merged in 375.840s
chaiml-mistral-24b-2048-41286-v1-mkmlizer: creating bucket guanaco-mkml-models
chaiml-mistral-24b-2048-41286-v1-mkmlizer: Bucket 's3://guanaco-mkml-models/' created
chaiml-mistral-24b-2048-41286-v1-mkmlizer: uploading /dev/shm/model_cache to s3://guanaco-mkml-models/chaiml-mistral-24b-2048-41286-v1/nvidia
chaiml-mistral-24b-2048-41286-v1-mkmlizer: cp /dev/shm/model_cache/config.json s3://guanaco-mkml-models/chaiml-mistral-24b-2048-41286-v1/nvidia/config.json
chaiml-mistral-24b-2048-41286-v1-mkmlizer: cp /dev/shm/model_cache/special_tokens_map.json s3://guanaco-mkml-models/chaiml-mistral-24b-2048-41286-v1/nvidia/special_tokens_map.json
chaiml-mistral-24b-2048-41286-v1-mkmlizer: cp /dev/shm/model_cache/tokenizer_config.json s3://guanaco-mkml-models/chaiml-mistral-24b-2048-41286-v1/nvidia/tokenizer_config.json
chaiml-mistral-24b-2048-41286-v1-mkmlizer: cp /dev/shm/model_cache/tokenizer.json s3://guanaco-mkml-models/chaiml-mistral-24b-2048-41286-v1/nvidia/tokenizer.json
chaiml-mistral-24b-2048-87648-v1-mkmlizer: 
Loading 0:   0%|          | 0.00/363 [00:00<?, ?it/s]
Loading 0:   1%|          | 3.00/363 [00:01<03:59, 1.50it/s]
Loading 0:   1%|          | 3.00/363 [00:01<03:59, 1.50it/s]
Loading 0:   1%|          | 4.00/363 [00:03<06:31, 1.09s/it]
Loading 0:   1%|          | 4.00/363 [00:03<06:31, 1.09s/it]
Loading 0:   1%|▏         | 5.00/363 [00:06<08:19, 1.40s/it]
Loading 0:   1%|▏         | 5.00/363 [00:06<08:19, 1.40s/it]
Loading 0:   3%|▎         | 11.0/363 [00:07<02:56, 1.99it/s]
Loading 0:   3%|▎         | 11.0/363 [00:07<02:56, 1.99it/s]
Loading 0:   4%|▎         | 13.0/363 [00:09<03:45, 1.55it/s]
Loading 0:   4%|▎         | 13.0/363 [00:09<03:45, 1.55it/s]
Loading 0:   4%|▍         | 14.0/363 [00:11<04:53, 1.19it/s]
Loading 0:   4%|▍         | 14.0/363 [00:11<04:53, 1.19it/s]
Loading 0:   4%|▍         | 15.0/363 [00:13<06:07, 1.06s/it]
Loading 0:   4%|▍         | 15.0/363 [00:13<06:07, 1.06s/it]
Loading 0:   6%|▌         | 21.0/363 [00:15<03:43, 1.53it/s]
Loading 0:   6%|▌         | 21.0/363 [00:15<03:43, 1.53it/s]
Loading 0:   6%|▌         | 22.0/363 [00:17<04:37, 1.23it/s]
Loading 0:   6%|▌         | 22.0/363 [00:17<04:37, 1.23it/s]
Loading 0:   6%|▋         | 23.0/363 [00:19<05:40, 1.00s/it]
Loading 0:   6%|▋         | 23.0/363 [00:19<05:40, 1.00s/it]
Loading 0:   9%|▊         | 31.0/363 [00:20<02:33, 2.17it/s]
Loading 0:   9%|▊         | 31.0/363 [00:20<02:33, 2.17it/s]
Loading 0:   9%|▉         | 34.0/363 [00:22<02:56, 1.87it/s]
Loading 0:   9%|▉         | 34.0/363 [00:22<02:56, 1.87it/s]
Loading 0:  10%|▉         | 35.0/363 [00:24<03:44, 1.46it/s]
Loading 0:  10%|▉         | 35.0/363 [00:24<03:44, 1.46it/s]
Loading 0:  10%|▉         | 36.0/363 [00:26<04:43, 1.15it/s]
Loading 0:  10%|▉         | 36.0/363 [00:26<04:43, 1.15it/s]
Loading 0:  11%|█         | 39.0/363 [00:28<04:16, 1.26it/s]
Loading 0:  11%|█         | 39.0/363 [00:28<04:16, 1.26it/s]
Loading 0:  11%|█         | 40.0/363 [00:30<05:09, 1.04it/s]
Loading 0:  11%|█         | 40.0/363 [00:30<05:09, 1.04it/s]
Loading 0:  11%|█▏        | 41.0/363 [00:32<06:08, 1.14s/it]
Loading 0:  11%|█▏        | 41.0/363 [00:32<06:08, 1.14s/it]
Loading 0:  13%|█▎        | 49.0/363 [00:34<02:31, 2.07it/s]
Loading 0:  13%|█▎        | 49.0/363 [00:34<02:31, 2.07it/s]
Loading 0:  14%|█▍        | 52.0/363 [00:36<02:51, 1.81it/s]
Loading 0:  14%|█▍        | 52.0/363 [00:36<02:51, 1.81it/s]
Loading 0:  15%|█▍        | 53.0/363 [00:38<03:38, 1.42it/s]
Loading 0:  15%|█▍        | 53.0/363 [00:38<03:38, 1.42it/s]
Loading 0:  15%|█▍        | 54.0/363 [00:40<04:34, 1.13it/s]
Loading 0:  15%|█▍        | 54.0/363 [00:40<04:34, 1.13it/s]
Loading 0:  16%|█▌        | 57.0/363 [00:42<04:04, 1.25it/s]
Loading 0:  16%|█▌        | 57.0/363 [00:42<04:04, 1.25it/s]
Loading 0:  16%|█▌        | 58.0/363 [00:44<04:54, 1.03it/s]
Loading 0:  16%|█▌        | 58.0/363 [00:44<04:54, 1.03it/s]
Loading 0:  16%|█▋        | 59.0/363 [00:46<05:51, 1.16s/it]
Loading 0:  16%|█▋        | 59.0/363 [00:46<05:51, 1.16s/it]
Loading 0:  18%|█▊        | 66.0/363 [00:47<02:31, 1.96it/s]
Loading 0:  18%|█▊        | 66.0/363 [00:47<02:31, 1.96it/s]
Loading 0:  19%|█▉        | 70.0/363 [00:49<02:40, 1.82it/s]
Loading 0:  19%|█▉        | 70.0/363 [00:49<02:40, 1.82it/s]
Loading 0:  20%|█▉        | 71.0/363 [00:51<03:23, 1.43it/s]
Loading 0:  20%|█▉        | 71.0/363 [00:51<03:23, 1.43it/s]
Loading 0:  20%|█▉        | 72.0/363 [00:53<04:16, 1.14it/s]
Loading 0:  20%|█▉        | 72.0/363 [00:53<04:16, 1.14it/s]
Loading 0:  21%|██        | 75.0/363 [00:55<03:48, 1.26it/s]
Loading 0:  21%|██        | 75.0/363 [00:55<03:48, 1.26it/s]
Loading 0:  21%|██        | 76.0/363 [00:57<04:36, 1.04it/s]
Loading 0:  21%|██        | 76.0/363 [00:57<04:36, 1.04it/s]
Loading 0:  21%|██        | 77.0/363 [00:59<05:29, 1.15s/it]
Loading 0:  21%|██        | 77.0/363 [00:59<05:29, 1.15s/it]
Loading 0:  23%|██▎       | 85.0/363 [01:00<02:14, 2.07it/s]
Loading 0:  23%|██▎       | 85.0/363 [01:00<02:14, 2.07it/s]
Loading 0:  24%|██▍       | 88.0/363 [01:02<02:31, 1.81it/s]
Loading 0:  24%|██▍       | 88.0/363 [01:02<02:31, 1.81it/s]
Loading 0:  25%|██▍       | 89.0/363 [01:04<03:13, 1.42it/s]
Loading 0:  25%|██▍       | 89.0/363 [01:04<03:13, 1.42it/s]
Loading 0:  25%|██▍       | 90.0/363 [01:06<04:02, 1.13it/s]
Loading 0:  25%|██▍       | 90.0/363 [01:06<04:02, 1.13it/s]
Loading 0:  27%|██▋       | 98.0/363 [01:08<01:55, 2.29it/s]
Loading 0:  27%|██▋       | 98.0/363 [01:08<01:55, 2.29it/s]
Loading 0:  28%|██▊       | 101/363 [01:10<02:11, 2.00it/s] 
Loading 0:  28%|██▊       | 101/363 [01:10<02:11, 2.00it/s]
Loading 0:  28%|██▊       | 102/363 [01:12<02:49, 1.54it/s]
Loading 0:  28%|██▊       | 102/363 [01:12<02:49, 1.54it/s]
Loading 0:  28%|██▊       | 103/363 [01:14<03:36, 1.20it/s]
Loading 0:  28%|██▊       | 103/363 [01:14<03:36, 1.20it/s]
Loading 0:  29%|██▉       | 106/363 [01:16<03:20, 1.28it/s]
Loading 0:  29%|██▉       | 106/363 [01:16<03:20, 1.28it/s]
Loading 0:  29%|██▉       | 107/363 [01:18<04:02, 1.05it/s]
Loading 0:  29%|██▉       | 107/363 [01:18<04:02, 1.05it/s]
Loading 0:  30%|██▉       | 108/363 [01:20<04:50, 1.14s/it]
Loading 0:  30%|██▉       | 108/363 [01:20<04:50, 1.14s/it]
Loading 0:  31%|███       | 111/363 [01:22<03:54, 1.08it/s]
Loading 0:  31%|███       | 111/363 [01:22<03:54, 1.08it/s]
Loading 0:  31%|███       | 112/363 [01:24<04:35, 1.10s/it]
Loading 0:  31%|███       | 112/363 [01:24<04:35, 1.10s/it]
Loading 0:  31%|███       | 113/363 [01:26<05:20, 1.28s/it]
Loading 0:  31%|███       | 113/363 [01:26<05:20, 1.28s/it]
Loading 0:  33%|███▎      | 121/363 [01:27<02:01, 1.99it/s]
Loading 0:  33%|███▎      | 121/363 [01:27<02:01, 1.99it/s]
Loading 0:  34%|███▍      | 124/363 [01:29<02:15, 1.76it/s]
Loading 0:  34%|███▍      | 124/363 [01:29<02:15, 1.76it/s]
Loading 0:  34%|███▍      | 125/363 [01:31<02:52, 1.38it/s]
Loading 0:  34%|███▍      | 125/363 [01:31<02:52, 1.38it/s]
Loading 0:  35%|███▍      | 126/363 [01:33<03:35, 1.10it/s]
Loading 0:  35%|███▍      | 126/363 [01:33<03:35, 1.10it/s]
Loading 0:  36%|███▌      | 129/363 [01:35<03:10, 1.23it/s]
Loading 0:  36%|███▌      | 129/363 [01:35<03:10, 1.23it/s]
Loading 0:  36%|███▌      | 130/363 [01:37<03:48, 1.02it/s]
Loading 0:  36%|███▌      | 130/363 [01:37<03:48, 1.02it/s]
Loading 0:  36%|███▌      | 131/363 [01:39<04:32, 1.17s/it]
Loading 0:  36%|███▌      | 131/363 [01:39<04:32, 1.17s/it]
Loading 0:  38%|███▊      | 138/363 [01:40<01:55, 1.95it/s]
Loading 0:  38%|███▊      | 138/363 [01:40<01:55, 1.95it/s]
Loading 0:  39%|███▉      | 142/363 [01:42<02:01, 1.82it/s]
Loading 0:  39%|███▉      | 142/363 [01:42<02:01, 1.82it/s]
Loading 0:  39%|███▉      | 143/363 [01:44<02:34, 1.42it/s]
Loading 0:  39%|███▉      | 143/363 [01:44<02:34, 1.42it/s]
Loading 0:  40%|███▉      | 144/363 [01:46<03:13, 1.13it/s]
Loading 0:  40%|███▉      | 144/363 [01:46<03:13, 1.13it/s]
Loading 0:  40%|████      | 147/363 [01:48<02:52, 1.25it/s]
Loading 0:  40%|████      | 147/363 [01:48<02:52, 1.25it/s]
Loading 0:  41%|████      | 148/363 [01:50<03:28, 1.03it/s]
Loading 0:  41%|████      | 148/363 [01:50<03:28, 1.03it/s]
Loading 0:  41%|████      | 149/363 [01:52<04:07, 1.16s/it]
Loading 0:  41%|████      | 149/363 [01:52<04:07, 1.16s/it]
Loading 0:  43%|████▎     | 157/363 [01:54<01:40, 2.05it/s]
Loading 0:  43%|████▎     | 157/363 [01:54<01:40, 2.05it/s]
Loading 0:  44%|████▍     | 160/363 [01:56<01:52, 1.80it/s]
Loading 0:  44%|████▍     | 160/363 [01:56<01:52, 1.80it/s]
Loading 0:  44%|████▍     | 161/363 [01:58<02:23, 1.41it/s]
Loading 0:  44%|████▍     | 161/363 [01:58<02:23, 1.41it/s]
Loading 0:  45%|████▍     | 162/363 [02:00<02:59, 1.12it/s]
Loading 0:  45%|████▍     | 162/363 [02:00<02:59, 1.12it/s]
Loading 0:  45%|████▌     | 165/363 [02:02<02:39, 1.24it/s]
Loading 0:  45%|████▌     | 165/363 [02:02<02:39, 1.24it/s]
Loading 0:  46%|████▌     | 166/363 [02:04<03:12, 1.03it/s]
Loading 0:  46%|████▌     | 166/363 [02:04<03:12, 1.03it/s]
Loading 0:  46%|████▌     | 167/363 [02:06<03:48, 1.17s/it]
Loading 0:  46%|████▌     | 167/363 [02:06<03:48, 1.17s/it]
Loading 0:  48%|████▊     | 175/363 [02:07<01:31, 2.05it/s]
Loading 0:  48%|████▊     | 175/363 [02:07<01:31, 2.05it/s]
Loading 0:  49%|████▉     | 178/363 [02:09<01:43, 1.80it/s]
Loading 0:  49%|████▉     | 178/363 [02:09<01:43, 1.80it/s]
Loading 0:  49%|████▉     | 179/363 [02:11<02:10, 1.41it/s]
Loading 0:  49%|████▉     | 179/363 [02:11<02:10, 1.41it/s]
Loading 0:  50%|████▉     | 180/363 [02:13<02:43, 1.12it/s]
Loading 0:  50%|████▉     | 180/363 [02:13<02:43, 1.12it/s]
Loading 0:  50%|█████     | 183/363 [02:15<02:24, 1.24it/s]
Loading 0:  50%|█████     | 183/363 [02:15<02:24, 1.24it/s]
Loading 0:  51%|█████     | 184/363 [02:17<02:54, 1.03it/s]
Loading 0:  51%|█████     | 184/363 [02:17<02:54, 1.03it/s]
Loading 0:  51%|█████     | 185/363 [02:19<03:28, 1.17s/it]
Loading 0:  51%|█████     | 185/363 [02:19<03:28, 1.17s/it]
Loading 0:  53%|█████▎    | 193/363 [02:20<01:23, 2.04it/s]
Loading 0:  53%|█████▎    | 193/363 [02:20<01:23, 2.04it/s]
Loading 0:  54%|█████▍    | 196/363 [02:23<01:33, 1.79it/s]
Loading 0:  54%|█████▍    | 196/363 [02:23<01:33, 1.79it/s]
Loading 0:  54%|█████▍    | 197/363 [02:25<01:58, 1.40it/s]
Loading 0:  54%|█████▍    | 197/363 [02:25<01:58, 1.40it/s]
Loading 0:  55%|█████▍    | 198/363 [02:27<02:28, 1.11it/s]
Loading 0:  55%|█████▍    | 198/363 [02:27<02:28, 1.11it/s]
Loading 0:  55%|█████▌    | 201/363 [02:29<02:10, 1.24it/s]
Loading 0:  55%|█████▌    | 201/363 [02:29<02:10, 1.24it/s]
Loading 0:  56%|█████▌    | 202/363 [02:31<02:37, 1.02it/s]
Loading 0:  56%|█████▌    | 202/363 [02:31<02:37, 1.02it/s]
Loading 0:  56%|█████▌    | 203/363 [02:33<03:07, 1.17s/it]
Loading 0:  56%|█████▌    | 203/363 [02:33<03:07, 1.17s/it]
Loading 0:  58%|█████▊    | 211/363 [02:34<01:14, 2.04it/s]
Loading 0:  58%|█████▊    | 211/363 [02:34<01:14, 2.04it/s]
Loading 0:  59%|█████▉    | 214/363 [02:36<01:23, 1.78it/s]
Loading 0:  59%|█████▉    | 214/363 [02:36<01:23, 1.78it/s]
Loading 0:  59%|█████▉    | 215/363 [02:38<01:46, 1.40it/s]
Loading 0:  59%|█████▉    | 215/363 [02:38<01:46, 1.40it/s]
Loading 0:  60%|█████▉    | 216/363 [02:40<02:12, 1.11it/s]
Loading 0:  60%|█████▉    | 216/363 [02:40<02:12, 1.11it/s]
Loading 0:  60%|██████    | 219/363 [02:42<01:56, 1.23it/s]
Loading 0:  60%|██████    | 219/363 [02:42<01:56, 1.23it/s]
Loading 0:  61%|██████    | 220/363 [02:44<02:20, 1.02it/s]
Loading 0:  61%|██████    | 220/363 [02:44<02:20, 1.02it/s]
Loading 0:  61%|██████    | 221/363 [02:46<02:46, 1.17s/it]
Loading 0:  61%|██████    | 221/363 [02:46<02:46, 1.17s/it]
Loading 0:  63%|██████▎   | 229/363 [02:47<01:05, 2.03it/s]
Loading 0:  63%|██████▎   | 229/363 [02:47<01:05, 2.03it/s]
Loading 0:  64%|██████▍   | 232/363 [02:50<01:13, 1.78it/s]
Loading 0:  64%|██████▍   | 232/363 [02:50<01:13, 1.78it/s]
Loading 0:  64%|██████▍   | 233/363 [02:52<01:33, 1.39it/s]
Loading 0:  64%|██████▍   | 233/363 [02:52<01:33, 1.39it/s]
Loading 0:  64%|██████▍   | 234/363 [02:54<01:57, 1.10it/s]
Loading 0:  64%|██████▍   | 234/363 [02:54<01:57, 1.10it/s]
Loading 0:  65%|██████▌   | 237/363 [02:56<01:44, 1.21it/s]
Loading 0:  65%|██████▌   | 237/363 [02:56<01:44, 1.21it/s]
Loading 0:  66%|██████▌   | 238/363 [02:58<02:04, 1.00it/s]
Loading 0:  66%|██████▌   | 238/363 [02:58<02:04, 1.00it/s]
Loading 0:  66%|██████▌   | 239/363 [03:00<02:27, 1.19s/it]
Loading 0:  66%|██████▌   | 239/363 [03:00<02:27, 1.19s/it]
Loading 0:  68%|██████▊   | 247/363 [03:01<00:57, 2.02it/s]
Loading 0:  68%|██████▊   | 247/363 [03:01<00:57, 2.02it/s]
Loading 0:  69%|██████▉   | 250/363 [03:03<01:03, 1.77it/s]
Loading 0:  69%|██████▉   | 250/363 [03:03<01:03, 1.77it/s]
Loading 0:  69%|██████▉   | 251/363 [03:05<01:20, 1.39it/s]
Loading 0:  69%|██████▉   | 251/363 [03:05<01:20, 1.39it/s]
Loading 0:  69%|██████▉   | 252/363 [03:07<01:41, 1.10it/s]
Loading 0:  69%|██████▉   | 252/363 [03:07<01:41, 1.10it/s]
Loading 0:  70%|███████   | 255/363 [03:09<01:28, 1.22it/s]
Loading 0:  70%|███████   | 255/363 [03:09<01:28, 1.22it/s]
Loading 0:  71%|███████   | 256/363 [03:11<01:46, 1.01it/s]
Loading 0:  71%|███████   | 256/363 [03:11<01:46, 1.01it/s]
Loading 0:  71%|███████   | 257/363 [03:13<02:05, 1.18s/it]
Loading 0:  71%|███████   | 257/363 [03:13<02:05, 1.18s/it]
Loading 0:  73%|███████▎  | 265/363 [03:15<00:48, 2.02it/s]
Loading 0:  73%|███████▎  | 265/363 [03:15<00:48, 2.02it/s]
Loading 0:  74%|███████▍  | 268/363 [03:17<00:53, 1.77it/s]
Loading 0:  74%|███████▍  | 268/363 [03:17<00:53, 1.77it/s]
Loading 0:  74%|███████▍  | 269/363 [03:19<01:07, 1.38it/s]
Loading 0:  74%|███████▍  | 269/363 [03:19<01:07, 1.38it/s]
Loading 0:  74%|███████▍  | 270/363 [03:21<01:24, 1.10it/s]
Loading 0:  74%|███████▍  | 270/363 [03:21<01:24, 1.10it/s]
Loading 0:  75%|███████▌  | 273/363 [03:42<04:44, 3.16s/it]
Loading 0:  75%|███████▌  | 273/363 [03:42<04:44, 3.16s/it]
Loading 0:  75%|███████▌  | 274/363 [03:44<04:25, 2.99s/it]
Loading 0:  75%|███████▌  | 274/363 [03:44<04:25, 2.99s/it]
Loading 0:  76%|███████▌  | 275/363 [03:46<04:10, 2.84s/it]
Loading 0:  76%|███████▌  | 275/363 [03:46<04:10, 2.84s/it]
Loading 0:  78%|███████▊  | 282/363 [03:47<01:30, 1.12s/it]
Loading 0:  78%|███████▊  | 282/363 [03:47<01:30, 1.12s/it]
Loading 0:  79%|███████▉  | 286/363 [03:50<01:12, 1.06it/s]
Loading 0:  79%|███████▉  | 286/363 [03:50<01:12, 1.06it/s]
Loading 0:  79%|███████▉  | 287/363 [03:51<01:18, 1.04s/it]
Loading 0:  79%|███████▉  | 287/363 [03:51<01:18, 1.04s/it]
Loading 0:  79%|███████▉  | 288/363 [03:54<01:28, 1.18s/it]
Loading 0:  79%|███████▉  | 288/363 [03:54<01:28, 1.18s/it]
Loading 0:  80%|████████  | 291/363 [03:56<01:11, 1.01it/s]
Loading 0:  80%|████████  | 291/363 [03:56<01:11, 1.01it/s]
Loading 0:  80%|████████  | 292/363 [03:58<01:21, 1.14s/it]
Loading 0:  80%|████████  | 292/363 [03:58<01:21, 1.14s/it]
Loading 0:  81%|████████  | 293/363 [04:00<01:31, 1.31s/it]
Loading 0:  81%|████████  | 293/363 [04:00<01:31, 1.31s/it]
Loading 0:  83%|████████▎ | 301/363 [04:01<00:33, 1.88it/s]
Loading 0:  83%|████████▎ | 301/363 [04:01<00:33, 1.88it/s]
Loading 0:  84%|████████▎ | 304/363 [04:03<00:35, 1.68it/s]
Loading 0:  84%|████████▎ | 304/363 [04:03<00:35, 1.68it/s]
Loading 0:  84%|████████▍ | 305/363 [04:05<00:43, 1.33it/s]
Loading 0:  84%|████████▍ | 305/363 [04:05<00:43, 1.33it/s]
Loading 0:  84%|████████▍ | 306/363 [04:07<00:53, 1.06it/s]
Loading 0:  84%|████████▍ | 306/363 [04:07<00:53, 1.06it/s]
Loading 0:  85%|████████▌ | 309/363 [04:09<00:45, 1.18it/s]
Loading 0:  85%|████████▌ | 309/363 [04:09<00:45, 1.18it/s]
Loading 0:  85%|████████▌ | 310/363 [04:11<00:54, 1.02s/it]
Loading 0:  85%|████████▌ | 310/363 [04:11<00:54, 1.02s/it]
Loading 0:  86%|████████▌ | 311/363 [04:13<01:03, 1.22s/it]
Loading 0:  86%|████████▌ | 311/363 [04:13<01:03, 1.22s/it]
Loading 0:  88%|████████▊ | 319/363 [04:15<00:22, 2.00it/s]
Loading 0:  88%|████████▊ | 319/363 [04:15<00:22, 2.00it/s]
Loading 0:  89%|████████▊ | 322/363 [04:17<00:23, 1.75it/s]
Loading 0:  89%|████████▊ | 322/363 [04:17<00:23, 1.75it/s]
Loading 0:  89%|████████▉ | 323/363 [04:19<00:29, 1.37it/s]
Loading 0:  89%|████████▉ | 323/363 [04:19<00:29, 1.37it/s]
Loading 0:  89%|████████▉ | 324/363 [04:21<00:36, 1.08it/s]
Loading 0:  89%|████████▉ | 324/363 [04:21<00:36, 1.08it/s]
Loading 0:  90%|█████████ | 327/363 [04:23<00:29, 1.20it/s]
Loading 0:  90%|█████████ | 327/363 [04:23<00:29, 1.20it/s]
Loading 0:  90%|█████████ | 328/363 [04:25<00:35, 1.01s/it]
Loading 0:  90%|█████████ | 328/363 [04:25<00:35, 1.01s/it]
Loading 0:  91%|█████████ | 329/363 [04:27<00:41, 1.21s/it]
Loading 0:  91%|█████████ | 329/363 [04:27<00:41, 1.21s/it]
Loading 0:  93%|█████████▎| 337/363 [04:28<00:12, 2.01it/s]
Loading 0:  93%|█████████▎| 337/363 [04:28<00:12, 2.01it/s]
Loading 0:  94%|█████████▎| 340/363 [04:31<00:13, 1.75it/s]
Loading 0:  94%|█████████▎| 340/363 [04:31<00:13, 1.75it/s]
Loading 0:  94%|█████████▍| 341/363 [04:33<00:16, 1.37it/s]
Loading 0:  94%|█████████▍| 341/363 [04:33<00:16, 1.37it/s]
Loading 0:  94%|█████████▍| 342/363 [04:35<00:19, 1.08it/s]
Loading 0:  94%|█████████▍| 342/363 [04:35<00:19, 1.08it/s]
Loading 0:  95%|█████████▌| 345/363 [04:37<00:15, 1.20it/s]
Loading 0:  95%|█████████▌| 345/363 [04:37<00:15, 1.20it/s]
Loading 0:  95%|█████████▌| 346/363 [04:39<00:17, 1.01s/it]
Loading 0:  95%|█████████▌| 346/363 [04:39<00:17, 1.01s/it]
Loading 0:  96%|█████████▌| 347/363 [04:41<00:19, 1.21s/it]
Loading 0:  96%|█████████▌| 347/363 [04:41<00:19, 1.21s/it]
Loading 0:  98%|█████████▊| 355/363 [04:42<00:04, 1.99it/s]
Loading 0:  98%|█████████▊| 355/363 [04:42<00:04, 1.99it/s]
Loading 0:  99%|█████████▉| 359/363 [04:45<00:02, 1.79it/s]
Loading 0:  99%|█████████▉| 359/363 [04:45<00:02, 1.79it/s]
Loading 0:  99%|█████████▉| 360/363 [04:47<00:02, 1.41it/s]
Loading 0:  99%|█████████▉| 360/363 [04:47<00:02, 1.41it/s]
Loading 0:  99%|█████████▉| 361/363 [04:49<00:01, 1.12it/s]
Loading 0:  99%|█████████▉| 361/363 [04:49<00:01, 1.12it/s]
Loading 0: 100%|██████████| 363/363 [04:49<00:00, 1.12it/s]
                                                           

Loading 0: 100%|██████████| 363/363 [04:49<00:00, 1.25it/s]
chaiml-mistral-24b-2048-87648-v1-mkmlizer: The tokenizer you are loading from '/tmp/tmpynetxqq5' with an incorrect regex pattern: https://huggingface.co/mistralai/Mistral-Small-3.1-24B-Instruct-2503/discussions/84#69121093e8b480e709447d5e. This will lead to incorrect tokenization. You should set the `fix_mistral_regex=True` flag when loading this tokenizer to fix this issue.
chaiml-mistral-24b-2048-87648-v1-mkmlizer: quantized model in 299.178s
chaiml-mistral-24b-2048-87648-v1-mkmlizer: Processed model ChaiML/mistral_24b_2048_gemini_ds_v3_4374_merged in 388.523s
chaiml-mistral-24b-2048-87648-v1-mkmlizer: creating bucket guanaco-mkml-models
chaiml-mistral-24b-2048-87648-v1-mkmlizer: Bucket 's3://guanaco-mkml-models/' created
chaiml-mistral-24b-2048-87648-v1-mkmlizer: uploading /dev/shm/model_cache to s3://guanaco-mkml-models/chaiml-mistral-24b-2048-87648-v1/nvidia
chaiml-mistral-24b-2048-87648-v1-mkmlizer: cp /dev/shm/model_cache/config.json s3://guanaco-mkml-models/chaiml-mistral-24b-2048-87648-v1/nvidia/config.json
chaiml-mistral-24b-2048-87648-v1-mkmlizer: cp /dev/shm/model_cache/special_tokens_map.json s3://guanaco-mkml-models/chaiml-mistral-24b-2048-87648-v1/nvidia/special_tokens_map.json
chaiml-mistral-24b-2048-87648-v1-mkmlizer: cp /dev/shm/model_cache/tokenizer_config.json s3://guanaco-mkml-models/chaiml-mistral-24b-2048-87648-v1/nvidia/tokenizer_config.json
chaiml-mistral-24b-2048-87648-v1-mkmlizer: cp /dev/shm/model_cache/tokenizer.json s3://guanaco-mkml-models/chaiml-mistral-24b-2048-87648-v1/nvidia/tokenizer.json
chaiml-mistral-24b-2048-41286-v1-mkmlizer: cp /dev/shm/model_cache/flywheel_model.1.safetensors s3://guanaco-mkml-models/chaiml-mistral-24b-2048-41286-v1/nvidia/flywheel_model.1.safetensors
chaiml-mistral-24b-2048-76638-v1-mkmlizer: 
Loading 0:   0%|          | 0.00/363 [00:00<?, ?it/s]
Loading 0:   1%|          | 3.00/363 [00:02<04:26, 1.35it/s]
Loading 0:   1%|          | 3.00/363 [00:02<04:26, 1.35it/s]
Loading 0:   1%|          | 4.00/363 [00:04<07:14, 1.21s/it]
Loading 0:   1%|          | 4.00/363 [00:04<07:14, 1.21s/it]
Loading 0:   1%|▏         | 5.00/363 [00:06<09:26, 1.58s/it]
Loading 0:   1%|▏         | 5.00/363 [00:06<09:26, 1.58s/it]
Loading 0:   3%|▎         | 11.0/363 [00:08<03:21, 1.75it/s]
Loading 0:   3%|▎         | 11.0/363 [00:08<03:21, 1.75it/s]
Loading 0:   4%|▎         | 13.0/363 [00:10<04:23, 1.33it/s]
Loading 0:   4%|▎         | 13.0/363 [00:10<04:23, 1.33it/s]
Loading 0:   4%|▍         | 14.0/363 [00:12<05:46, 1.01it/s]
Loading 0:   4%|▍         | 14.0/363 [00:12<05:46, 1.01it/s]
Loading 0:   4%|▍         | 15.0/363 [00:15<07:15, 1.25s/it]
Loading 0:   4%|▍         | 15.0/363 [00:15<07:15, 1.25s/it]
Loading 0:   6%|▌         | 21.0/363 [00:18<04:25, 1.29it/s]
Loading 0:   6%|▌         | 21.0/363 [00:18<04:25, 1.29it/s]
Loading 0:   6%|▌         | 22.0/363 [00:20<05:29, 1.03it/s]
Loading 0:   6%|▌         | 22.0/363 [00:20<05:29, 1.03it/s]
Loading 0:   6%|▋         | 23.0/363 [00:22<06:44, 1.19s/it]
Loading 0:   6%|▋         | 23.0/363 [00:22<06:44, 1.19s/it]
Loading 0:   8%|▊         | 30.0/363 [00:23<03:08, 1.76it/s]
Loading 0:   8%|▊         | 30.0/363 [00:23<03:08, 1.76it/s]
Loading 0:   9%|▉         | 34.0/363 [00:26<03:24, 1.61it/s]
Loading 0:   9%|▉         | 34.0/363 [00:26<03:24, 1.61it/s]
Loading 0:  10%|▉         | 35.0/363 [00:29<04:21, 1.25it/s]
Loading 0:  10%|▉         | 35.0/363 [00:29<04:21, 1.25it/s]
Loading 0:  10%|▉         | 36.0/363 [00:31<05:30, 1.01s/it]
Loading 0:  10%|▉         | 36.0/363 [00:31<05:30, 1.01s/it]
Loading 0:  11%|█         | 39.0/363 [00:33<04:58, 1.09it/s]
Loading 0:  11%|█         | 39.0/363 [00:33<04:58, 1.09it/s]
Loading 0:  11%|█         | 40.0/363 [00:36<06:01, 1.12s/it]
Loading 0:  11%|█         | 40.0/363 [00:36<06:01, 1.12s/it]
Loading 0:  11%|█▏        | 41.0/363 [00:38<07:11, 1.34s/it]
Loading 0:  11%|█▏        | 41.0/363 [00:38<07:11, 1.34s/it]
Loading 0:  13%|█▎        | 48.0/363 [00:39<03:07, 1.68it/s]
Loading 0:  13%|█▎        | 48.0/363 [00:39<03:07, 1.68it/s]
Loading 0:  14%|█▍        | 52.0/363 [00:42<03:19, 1.56it/s]
Loading 0:  14%|█▍        | 52.0/363 [00:42<03:19, 1.56it/s]
Loading 0:  15%|█▍        | 53.0/363 [00:44<04:13, 1.22it/s]
Loading 0:  15%|█▍        | 53.0/363 [00:44<04:13, 1.22it/s]
Loading 0:  15%|█▍        | 54.0/363 [00:47<05:19, 1.03s/it]
Loading 0:  15%|█▍        | 54.0/363 [00:47<05:19, 1.03s/it]
Loading 0:  16%|█▌        | 57.0/363 [00:49<04:45, 1.07it/s]
Loading 0:  16%|█▌        | 57.0/363 [00:49<04:45, 1.07it/s]
Loading 0:  16%|█▌        | 58.0/363 [00:51<05:45, 1.13s/it]
Loading 0:  16%|█▌        | 58.0/363 [00:51<05:45, 1.13s/it]
Loading 0:  16%|█▋        | 59.0/363 [00:54<06:52, 1.36s/it]
Loading 0:  16%|█▋        | 59.0/363 [00:54<06:52, 1.36s/it]
Loading 0:  18%|█▊        | 66.0/363 [00:55<02:58, 1.67it/s]
Loading 0:  18%|█▊        | 66.0/363 [00:55<02:58, 1.67it/s]
Loading 0:  19%|█▉        | 70.0/363 [00:58<03:11, 1.53it/s]
Loading 0:  19%|█▉        | 70.0/363 [00:58<03:11, 1.53it/s]
Loading 0:  20%|█▉        | 71.0/363 [01:00<04:04, 1.20it/s]
Loading 0:  20%|█▉        | 71.0/363 [01:00<04:04, 1.20it/s]
Loading 0:  20%|█▉        | 72.0/363 [01:03<05:15, 1.09s/it]
Loading 0:  20%|█▉        | 72.0/363 [01:03<05:15, 1.09s/it]
Loading 0:  21%|██        | 75.0/363 [01:05<04:42, 1.02it/s]
Loading 0:  21%|██        | 75.0/363 [01:05<04:42, 1.02it/s]
Loading 0:  21%|██        | 76.0/363 [01:08<05:38, 1.18s/it]
Loading 0:  21%|██        | 76.0/363 [01:08<05:38, 1.18s/it]
Loading 0:  21%|██        | 77.0/363 [01:10<06:42, 1.41s/it]
Loading 0:  21%|██        | 77.0/363 [01:10<06:42, 1.41s/it]
Loading 0:  23%|██▎       | 84.0/363 [01:11<02:53, 1.61it/s]
Loading 0:  23%|██▎       | 84.0/363 [01:11<02:53, 1.61it/s]
Loading 0:  24%|██▍       | 88.0/363 [01:14<03:03, 1.50it/s]
Loading 0:  24%|██▍       | 88.0/363 [01:14<03:03, 1.50it/s]
Loading 0:  25%|██▍       | 89.0/363 [01:17<03:52, 1.18it/s]
Loading 0:  25%|██▍       | 89.0/363 [01:17<03:52, 1.18it/s]
Loading 0:  25%|██▍       | 90.0/363 [01:19<04:50, 1.06s/it]
Loading 0:  25%|██▍       | 90.0/363 [01:19<04:50, 1.06s/it]
Loading 0:  27%|██▋       | 97.0/363 [01:20<02:26, 1.81it/s]
Loading 0:  27%|██▋       | 97.0/363 [01:20<02:26, 1.81it/s]
Loading 0:  28%|██▊       | 101/363 [01:23<02:38, 1.65it/s] 
Loading 0:  28%|██▊       | 101/363 [01:23<02:38, 1.65it/s]
Loading 0:  28%|██▊       | 102/363 [01:26<03:24, 1.28it/s]
Loading 0:  28%|██▊       | 102/363 [01:26<03:24, 1.28it/s]
Loading 0:  28%|██▊       | 103/363 [01:28<04:18, 1.01it/s]
Loading 0:  28%|██▊       | 103/363 [01:28<04:18, 1.01it/s]
Loading 0:  29%|██▉       | 106/363 [01:30<03:59, 1.07it/s]
Loading 0:  29%|██▉       | 106/363 [01:30<03:59, 1.07it/s]
Loading 0:  29%|██▉       | 107/363 [01:33<04:48, 1.13s/it]
Loading 0:  29%|██▉       | 107/363 [01:33<04:48, 1.13s/it]
Loading 0:  30%|██▉       | 108/363 [01:35<05:47, 1.36s/it]
Loading 0:  30%|██▉       | 108/363 [01:35<05:47, 1.36s/it]
Loading 0:  31%|███       | 111/363 [01:38<04:40, 1.11s/it]
Loading 0:  31%|███       | 111/363 [01:38<04:40, 1.11s/it]
Loading 0:  31%|███       | 112/363 [01:40<05:28, 1.31s/it]
Loading 0:  31%|███       | 112/363 [01:40<05:28, 1.31s/it]
Loading 0:  31%|███       | 113/363 [01:42<06:20, 1.52s/it]
Loading 0:  31%|███       | 113/363 [01:42<06:20, 1.52s/it]
Loading 0:  33%|███▎      | 120/363 [01:43<02:32, 1.59it/s]
Loading 0:  33%|███▎      | 120/363 [01:43<02:32, 1.59it/s]
Loading 0:  34%|███▍      | 124/363 [01:46<02:39, 1.50it/s]
Loading 0:  34%|███▍      | 124/363 [01:46<02:39, 1.50it/s]
Loading 0:  34%|███▍      | 125/363 [01:49<03:21, 1.18it/s]
Loading 0:  34%|███▍      | 125/363 [01:49<03:21, 1.18it/s]
Loading 0:  35%|███▍      | 126/363 [01:51<04:11, 1.06s/it]
Loading 0:  35%|███▍      | 126/363 [01:51<04:11, 1.06s/it]
Loading 0:  36%|███▌      | 129/363 [01:53<03:42, 1.05it/s]
Loading 0:  36%|███▌      | 129/363 [01:53<03:42, 1.05it/s]
Loading 0:  36%|███▌      | 130/363 [01:56<04:28, 1.15s/it]
Loading 0:  36%|███▌      | 130/363 [01:56<04:28, 1.15s/it]
Loading 0:  36%|███▌      | 131/363 [01:58<05:18, 1.37s/it]
Loading 0:  36%|███▌      | 131/363 [01:58<05:18, 1.37s/it]
Loading 0:  38%|███▊      | 138/363 [01:59<02:15, 1.66it/s]
Loading 0:  38%|███▊      | 138/363 [01:59<02:15, 1.66it/s]
Loading 0:  39%|███▉      | 142/363 [02:02<02:23, 1.54it/s]
Loading 0:  39%|███▉      | 142/363 [02:02<02:23, 1.54it/s]
Loading 0:  39%|███▉      | 143/363 [02:04<03:02, 1.21it/s]
Loading 0:  39%|███▉      | 143/363 [02:04<03:02, 1.21it/s]
Loading 0:  40%|███▉      | 144/363 [02:07<03:47, 1.04s/it]
Loading 0:  40%|███▉      | 144/363 [02:07<03:47, 1.04s/it]
Loading 0:  40%|████      | 147/363 [02:09<03:23, 1.06it/s]
Loading 0:  40%|████      | 147/363 [02:09<03:23, 1.06it/s]
Loading 0:  41%|████      | 148/363 [02:12<04:05, 1.14s/it]
Loading 0:  41%|████      | 148/363 [02:12<04:05, 1.14s/it]
Loading 0:  41%|████      | 149/363 [02:14<04:51, 1.36s/it]
Loading 0:  41%|████      | 149/363 [02:14<04:51, 1.36s/it]
Loading 0:  43%|████▎     | 156/363 [02:15<02:03, 1.68it/s]
Loading 0:  43%|████▎     | 156/363 [02:15<02:03, 1.68it/s]
Loading 0:  44%|████▍     | 160/363 [02:18<02:10, 1.56it/s]
Loading 0:  44%|████▍     | 160/363 [02:18<02:10, 1.56it/s]
Loading 0:  44%|████▍     | 161/363 [02:20<02:45, 1.22it/s]
Loading 0:  44%|████▍     | 161/363 [02:20<02:45, 1.22it/s]
Loading 0:  45%|████▍     | 162/363 [02:23<03:27, 1.03s/it]
Loading 0:  45%|████▍     | 162/363 [02:23<03:27, 1.03s/it]
Loading 0:  45%|████▌     | 165/363 [02:25<03:05, 1.07it/s]
Loading 0:  45%|████▌     | 165/363 [02:25<03:05, 1.07it/s]
Loading 0:  46%|████▌     | 166/363 [02:27<03:43, 1.14s/it]
Loading 0:  46%|████▌     | 166/363 [02:27<03:43, 1.14s/it]
Loading 0:  46%|████▌     | 167/363 [02:30<04:26, 1.36s/it]
Loading 0:  46%|████▌     | 167/363 [02:30<04:26, 1.36s/it]
Loading 0:  48%|████▊     | 174/363 [02:31<01:52, 1.68it/s]
Loading 0:  48%|████▊     | 174/363 [02:31<01:52, 1.68it/s]
Loading 0:  49%|████▉     | 178/363 [02:34<02:00, 1.54it/s]
Loading 0:  49%|████▉     | 178/363 [02:34<02:00, 1.54it/s]
Loading 0:  49%|████▉     | 179/363 [02:36<02:32, 1.21it/s]
Loading 0:  49%|████▉     | 179/363 [02:36<02:32, 1.21it/s]
Loading 0:  50%|████▉     | 180/363 [02:38<03:10, 1.04s/it]
Loading 0:  50%|████▉     | 180/363 [02:38<03:10, 1.04s/it]
Loading 0:  50%|█████     | 183/363 [02:41<02:49, 1.06it/s]
Loading 0:  50%|█████     | 183/363 [02:41<02:49, 1.06it/s]
Loading 0:  51%|█████     | 184/363 [02:43<03:23, 1.14s/it]
Loading 0:  51%|█████     | 184/363 [02:43<03:23, 1.14s/it]
Loading 0:  51%|█████     | 185/363 [02:45<04:02, 1.36s/it]
Loading 0:  51%|█████     | 185/363 [02:45<04:02, 1.36s/it]
Loading 0:  53%|█████▎    | 192/363 [02:47<01:41, 1.68it/s]
Loading 0:  53%|█████▎    | 192/363 [02:47<01:41, 1.68it/s]
Loading 0:  54%|█████▍    | 196/363 [02:49<01:46, 1.56it/s]
Loading 0:  54%|█████▍    | 196/363 [02:49<01:46, 1.56it/s]
Loading 0:  54%|█████▍    | 197/363 [02:52<02:15, 1.22it/s]
Loading 0:  54%|█████▍    | 197/363 [02:52<02:15, 1.22it/s]
Loading 0:  55%|█████▍    | 198/363 [02:54<02:50, 1.03s/it]
Loading 0:  55%|█████▍    | 198/363 [02:54<02:50, 1.03s/it]
Loading 0:  55%|█████▌    | 201/363 [02:56<02:31, 1.07it/s]
Loading 0:  55%|█████▌    | 201/363 [02:56<02:31, 1.07it/s]
Loading 0:  56%|█████▌    | 202/363 [02:59<03:02, 1.13s/it]
Loading 0:  56%|█████▌    | 202/363 [02:59<03:02, 1.13s/it]
Loading 0:  56%|█████▌    | 203/363 [03:01<03:42, 1.39s/it]
Loading 0:  56%|█████▌    | 203/363 [03:01<03:42, 1.39s/it]
Loading 0:  58%|█████▊    | 210/363 [03:02<01:33, 1.64it/s]
Loading 0:  58%|█████▊    | 210/363 [03:02<01:33, 1.64it/s]
Loading 0:  59%|█████▉    | 214/363 [03:05<01:36, 1.55it/s]
Loading 0:  59%|█████▉    | 214/363 [03:05<01:36, 1.55it/s]
Loading 0:  59%|█████▉    | 215/363 [03:08<02:01, 1.22it/s]
Loading 0:  59%|█████▉    | 215/363 [03:08<02:01, 1.22it/s]
Loading 0:  60%|█████▉    | 216/363 [03:10<02:32, 1.04s/it]
Loading 0:  60%|█████▉    | 216/363 [03:10<02:32, 1.04s/it]
Loading 0:  60%|██████    | 219/363 [03:12<02:14, 1.07it/s]
Loading 0:  60%|██████    | 219/363 [03:12<02:14, 1.07it/s]
Loading 0:  61%|██████    | 220/363 [03:15<02:41, 1.13s/it]
Loading 0:  61%|██████    | 220/363 [03:15<02:41, 1.13s/it]
Loading 0:  61%|██████    | 221/363 [03:17<03:12, 1.35s/it]
Loading 0:  61%|██████    | 221/363 [03:17<03:12, 1.35s/it]
Loading 0:  63%|██████▎   | 228/363 [03:18<01:20, 1.68it/s]
Loading 0:  63%|██████▎   | 228/363 [03:18<01:20, 1.68it/s]
Loading 0:  64%|██████▍   | 232/363 [03:21<01:24, 1.56it/s]
Loading 0:  64%|██████▍   | 232/363 [03:21<01:24, 1.56it/s]
Loading 0:  64%|██████▍   | 233/363 [03:23<01:46, 1.22it/s]
Loading 0:  64%|██████▍   | 233/363 [03:23<01:46, 1.22it/s]
Loading 0:  64%|██████▍   | 234/363 [03:26<02:13, 1.03s/it]
Loading 0:  64%|██████▍   | 234/363 [03:26<02:13, 1.03s/it]
Loading 0:  65%|██████▌   | 237/363 [03:28<01:57, 1.07it/s]
Loading 0:  65%|██████▌   | 237/363 [03:28<01:57, 1.07it/s]
Loading 0:  66%|██████▌   | 238/363 [03:30<02:21, 1.13s/it]
Loading 0:  66%|██████▌   | 238/363 [03:30<02:21, 1.13s/it]
Loading 0:  66%|██████▌   | 239/363 [03:33<02:47, 1.35s/it]
Loading 0:  66%|██████▌   | 239/363 [03:33<02:47, 1.35s/it]
Loading 0:  68%|██████▊   | 246/363 [03:34<01:09, 1.69it/s]
Loading 0:  68%|██████▊   | 246/363 [03:34<01:09, 1.69it/s]
Loading 0:  69%|██████▉   | 250/363 [03:37<01:12, 1.56it/s]
Loading 0:  69%|██████▉   | 250/363 [03:37<01:12, 1.56it/s]
Loading 0:  69%|██████▉   | 251/363 [03:39<01:31, 1.22it/s]
Loading 0:  69%|██████▉   | 251/363 [03:39<01:31, 1.22it/s]
Loading 0:  69%|██████▉   | 252/363 [03:41<01:54, 1.03s/it]
Loading 0:  69%|██████▉   | 252/363 [03:41<01:54, 1.03s/it]
Loading 0:  70%|███████   | 255/363 [03:44<01:40, 1.07it/s]
Loading 0:  70%|███████   | 255/363 [03:44<01:40, 1.07it/s]
Loading 0:  71%|███████   | 256/363 [03:46<02:00, 1.13s/it]
Loading 0:  71%|███████   | 256/363 [03:46<02:00, 1.13s/it]
Loading 0:  71%|███████   | 257/363 [03:48<02:22, 1.35s/it]
Loading 0:  71%|███████   | 257/363 [03:48<02:22, 1.35s/it]
Loading 0:  73%|███████▎  | 264/363 [03:49<00:58, 1.69it/s]
Loading 0:  73%|███████▎  | 264/363 [03:49<00:58, 1.69it/s]
Loading 0:  74%|███████▍  | 268/363 [03:52<01:00, 1.57it/s]
Loading 0:  74%|███████▍  | 268/363 [03:52<01:00, 1.57it/s]
Loading 0:  74%|███████▍  | 269/363 [03:54<01:16, 1.23it/s]
Loading 0:  74%|███████▍  | 269/363 [03:54<01:16, 1.23it/s]
Loading 0:  74%|███████▍  | 270/363 [03:57<01:35, 1.03s/it]
Loading 0:  74%|███████▍  | 270/363 [03:57<01:35, 1.03s/it]
Loading 0:  75%|███████▌  | 273/363 [04:14<04:04, 2.72s/it]
Loading 0:  75%|███████▌  | 273/363 [04:14<04:04, 2.72s/it]
Loading 0:  75%|███████▌  | 274/363 [04:16<03:55, 2.65s/it]
Loading 0:  75%|███████▌  | 274/363 [04:16<03:55, 2.65s/it]
Loading 0:  76%|███████▌  | 275/363 [04:19<03:49, 2.61s/it]
Loading 0:  76%|███████▌  | 275/363 [04:19<03:49, 2.61s/it]
Loading 0:  78%|███████▊  | 282/363 [04:20<01:25, 1.05s/it]
Loading 0:  78%|███████▊  | 282/363 [04:20<01:25, 1.05s/it]
Loading 0:  79%|███████▉  | 286/363 [04:22<01:12, 1.07it/s]
Loading 0:  79%|███████▉  | 286/363 [04:22<01:12, 1.07it/s]
Loading 0:  79%|███████▉  | 287/363 [04:25<01:22, 1.08s/it]
Loading 0:  79%|███████▉  | 287/363 [04:25<01:22, 1.08s/it]
Loading 0:  79%|███████▉  | 288/363 [04:27<01:34, 1.25s/it]
Loading 0:  79%|███████▉  | 288/363 [04:27<01:34, 1.25s/it]
Loading 0:  80%|████████  | 291/363 [04:29<01:17, 1.07s/it]
Loading 0:  80%|████████  | 291/363 [04:29<01:17, 1.07s/it]
Loading 0:  80%|████████  | 292/363 [04:32<01:28, 1.25s/it]
Loading 0:  80%|████████  | 292/363 [04:32<01:28, 1.25s/it]
Loading 0:  81%|████████  | 293/363 [04:34<01:41, 1.45s/it]
Loading 0:  81%|████████  | 293/363 [04:34<01:41, 1.45s/it]
Loading 0:  83%|████████▎ | 300/363 [04:35<00:39, 1.60it/s]
Loading 0:  83%|████████▎ | 300/363 [04:35<00:39, 1.60it/s]
Loading 0:  84%|████████▎ | 304/363 [04:38<00:38, 1.52it/s]
Loading 0:  84%|████████▎ | 304/363 [04:38<00:38, 1.52it/s]
Loading 0:  84%|████████▍ | 305/363 [04:40<00:48, 1.20it/s]
Loading 0:  84%|████████▍ | 305/363 [04:40<00:48, 1.20it/s]
Loading 0:  84%|████████▍ | 306/363 [04:43<00:59, 1.04s/it]
Loading 0:  84%|████████▍ | 306/363 [04:43<00:59, 1.04s/it]
Loading 0:  85%|████████▌ | 309/363 [04:45<00:50, 1.07it/s]
Loading 0:  85%|████████▌ | 309/363 [04:45<00:50, 1.07it/s]
Loading 0:  85%|████████▌ | 310/363 [04:47<00:59, 1.13s/it]
Loading 0:  85%|████████▌ | 310/363 [04:47<00:59, 1.13s/it]
Loading 0:  86%|████████▌ | 311/363 [04:50<01:10, 1.36s/it]
Loading 0:  86%|████████▌ | 311/363 [04:50<01:10, 1.36s/it]
Loading 0:  88%|████████▊ | 318/363 [04:51<00:26, 1.69it/s]
Loading 0:  88%|████████▊ | 318/363 [04:51<00:26, 1.69it/s]
Loading 0:  89%|████████▊ | 322/363 [04:54<00:26, 1.57it/s]
Loading 0:  89%|████████▊ | 322/363 [04:54<00:26, 1.57it/s]
Loading 0:  89%|████████▉ | 323/363 [04:56<00:32, 1.23it/s]
Loading 0:  89%|████████▉ | 323/363 [04:56<00:32, 1.23it/s]
Loading 0:  89%|████████▉ | 324/363 [04:58<00:39, 1.02s/it]
Loading 0:  89%|████████▉ | 324/363 [04:58<00:39, 1.02s/it]
Loading 0:  90%|█████████ | 327/363 [05:00<00:33, 1.08it/s]
Loading 0:  90%|█████████ | 327/363 [05:00<00:33, 1.08it/s]
Loading 0:  90%|█████████ | 328/363 [05:03<00:39, 1.13s/it]
Loading 0:  90%|█████████ | 328/363 [05:03<00:39, 1.13s/it]
Loading 0:  91%|█████████ | 329/363 [05:05<00:45, 1.35s/it]
Loading 0:  91%|█████████ | 329/363 [05:05<00:45, 1.35s/it]
Loading 0:  93%|█████████▎| 336/363 [05:06<00:15, 1.70it/s]
Loading 0:  93%|█████████▎| 336/363 [05:06<00:15, 1.70it/s]
Loading 0:  94%|█████████▎| 340/363 [05:09<00:14, 1.57it/s]
Loading 0:  94%|█████████▎| 340/363 [05:09<00:14, 1.57it/s]
Loading 0:  94%|█████████▍| 341/363 [05:11<00:17, 1.23it/s]
Loading 0:  94%|█████████▍| 341/363 [05:11<00:17, 1.23it/s]
Loading 0:  94%|█████████▍| 342/363 [05:14<00:21, 1.03s/it]
Loading 0:  94%|█████████▍| 342/363 [05:14<00:21, 1.03s/it]
Loading 0:  95%|█████████▌| 345/363 [05:16<00:16, 1.07it/s]
Loading 0:  95%|█████████▌| 345/363 [05:16<00:16, 1.07it/s]
Loading 0:  95%|█████████▌| 346/363 [05:18<00:19, 1.14s/it]
Loading 0:  95%|█████████▌| 346/363 [05:18<00:19, 1.14s/it]
Loading 0:  96%|█████████▌| 347/363 [05:21<00:21, 1.36s/it]
Loading 0:  96%|█████████▌| 347/363 [05:21<00:21, 1.36s/it]
Loading 0:  98%|█████████▊| 354/363 [05:22<00:05, 1.67it/s]
Loading 0:  98%|█████████▊| 354/363 [05:22<00:05, 1.67it/s]
Loading 0:  98%|█████████▊| 357/363 [05:23<00:03, 1.89it/s]
Loading 0:  98%|█████████▊| 357/363 [05:23<00:03, 1.89it/s]
Loading 0:  99%|█████████▉| 359/363 [05:25<00:02, 1.50it/s]
Loading 0:  99%|█████████▉| 359/363 [05:25<00:02, 1.50it/s]
Loading 0:  99%|█████████▉| 360/363 [05:28<00:02, 1.14it/s]
Loading 0:  99%|█████████▉| 360/363 [05:28<00:02, 1.14it/s]
Loading 0:  99%|█████████▉| 361/363 [05:30<00:02, 1.12s/it]
Loading 0:  99%|█████████▉| 361/363 [05:30<00:02, 1.12s/it]
Loading 0: 100%|██████████| 363/363 [05:30<00:00, 1.26it/s]
Loading 0: 100%|██████████| 363/363 [05:30<00:00, 1.26it/s]
                                                           

Loading 0: 100%|██████████| 363/363 [05:30<00:00, 1.10it/s]
chaiml-mistral-24b-2048-87648-v1-mkmlizer: cp /dev/shm/model_cache/flywheel_model.1.safetensors s3://guanaco-mkml-models/chaiml-mistral-24b-2048-87648-v1/nvidia/flywheel_model.1.safetensors
chaiml-mistral-24b-2048-76638-v1-mkmlizer: The tokenizer you are loading from '/tmp/tmphcu9_fue' with an incorrect regex pattern: https://huggingface.co/mistralai/Mistral-Small-3.1-24B-Instruct-2503/discussions/84#69121093e8b480e709447d5e. This will lead to incorrect tokenization. You should set the `fix_mistral_regex=True` flag when loading this tokenizer to fix this issue.
chaiml-mistral-24b-2048-76638-v1-mkmlizer: quantized model in 338.328s
chaiml-mistral-24b-2048-76638-v1-mkmlizer: Processed model ChaiML/mistral_24b_2048_gemini_ds_v3_6561_merged in 434.845s
chaiml-mistral-24b-2048-76638-v1-mkmlizer: creating bucket guanaco-mkml-models
chaiml-mistral-24b-2048-76638-v1-mkmlizer: Bucket 's3://guanaco-mkml-models/' created
chaiml-mistral-24b-2048-76638-v1-mkmlizer: uploading /dev/shm/model_cache to s3://guanaco-mkml-models/chaiml-mistral-24b-2048-76638-v1/nvidia
chaiml-mistral-24b-2048-76638-v1-mkmlizer: cp /dev/shm/model_cache/config.json s3://guanaco-mkml-models/chaiml-mistral-24b-2048-76638-v1/nvidia/config.json
chaiml-mistral-24b-2048-76638-v1-mkmlizer: cp /dev/shm/model_cache/special_tokens_map.json s3://guanaco-mkml-models/chaiml-mistral-24b-2048-76638-v1/nvidia/special_tokens_map.json
chaiml-mistral-24b-2048-76638-v1-mkmlizer: cp /dev/shm/model_cache/tokenizer_config.json s3://guanaco-mkml-models/chaiml-mistral-24b-2048-76638-v1/nvidia/tokenizer_config.json
chaiml-mistral-24b-2048-76638-v1-mkmlizer: cp /dev/shm/model_cache/tokenizer.json s3://guanaco-mkml-models/chaiml-mistral-24b-2048-76638-v1/nvidia/tokenizer.json
chaiml-mistral-24b-2048-41286-v1-mkmlizer: cp /dev/shm/model_cache/flywheel_model.0.safetensors s3://guanaco-mkml-models/chaiml-mistral-24b-2048-41286-v1/nvidia/flywheel_model.0.safetensors
Job chaiml-mistral-24b-2048-41286-v1-mkmlizer completed after 498.9s with status: succeeded
Stopping job with name chaiml-mistral-24b-2048-41286-v1-mkmlizer
Pipeline stage MKMLizer completed in 501.43s
run pipeline stage %s
Running pipeline stage MKMLTemplater
Pipeline stage MKMLTemplater completed in 0.66s
run pipeline stage %s
Running pipeline stage MKMLDeployer
Creating inference service chaiml-mistral-24b-2048-41286-v1
Waiting for inference service chaiml-mistral-24b-2048-41286-v1 to be ready
chaiml-mistral-24b-2048-76638-v1-mkmlizer: cp /dev/shm/model_cache/flywheel_model.1.safetensors s3://guanaco-mkml-models/chaiml-mistral-24b-2048-76638-v1/nvidia/flywheel_model.1.safetensors
chaiml-mistral-24b-2048-87648-v1-mkmlizer: cp /dev/shm/model_cache/flywheel_model.0.safetensors s3://guanaco-mkml-models/chaiml-mistral-24b-2048-87648-v1/nvidia/flywheel_model.0.safetensors
Job chaiml-mistral-24b-2048-87648-v1-mkmlizer completed after 511.94s with status: succeeded
Stopping job with name chaiml-mistral-24b-2048-87648-v1-mkmlizer
Pipeline stage MKMLizer completed in 515.47s
run pipeline stage %s
Running pipeline stage MKMLTemplater
Pipeline stage MKMLTemplater completed in 0.79s
run pipeline stage %s
Running pipeline stage MKMLDeployer
Creating inference service chaiml-mistral-24b-2048-87648-v1
Waiting for inference service chaiml-mistral-24b-2048-87648-v1 to be ready
chaiml-mistral-24b-2048-76638-v1-mkmlizer: cp /dev/shm/model_cache/flywheel_model.0.safetensors s3://guanaco-mkml-models/chaiml-mistral-24b-2048-76638-v1/nvidia/flywheel_model.0.safetensors
Job chaiml-mistral-24b-2048-76638-v1-mkmlizer completed after 561.05s with status: succeeded
Stopping job with name chaiml-mistral-24b-2048-76638-v1-mkmlizer
Pipeline stage MKMLizer completed in 563.09s
run pipeline stage %s
Running pipeline stage MKMLTemplater
Pipeline stage MKMLTemplater completed in 0.93s
run pipeline stage %s
Running pipeline stage MKMLDeployer
Creating inference service chaiml-mistral-24b-2048-76638-v1
Waiting for inference service chaiml-mistral-24b-2048-76638-v1 to be ready
Inference service mistralai-mistral-smal-88026-v68 ready after 151.93807172775269s
Pipeline stage MKMLDeployer completed in 154.92s
run pipeline stage %s
Running pipeline stage StressChecker
Received healthy response to inference request in 2.5273895263671875s
Received healthy response to inference request in 2.530282735824585s
Received healthy response to inference request in 2.149505376815796s
Received healthy response to inference request in 2.5875275135040283s
Received healthy response to inference request in 4.358465909957886s
5 requests
0 failed requests
5th percentile: 2.225082206726074
10th percentile: 2.3006590366363526
20th percentile: 2.4518126964569094
30th percentile: 2.527968168258667
40th percentile: 2.529125452041626
50th percentile: 2.530282735824585
60th percentile: 2.5531806468963625
70th percentile: 2.5760785579681396
80th percentile: 2.9417151927948
90th percentile: 3.650090551376343
95th percentile: 4.004278230667114
99th percentile: 4.287628374099731
mean time: 2.8306342124938966
Pipeline stage StressChecker completed in 27.80s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyTriggerPipeline
run_pipeline:run_in_cloud %s
starting trigger_guanaco_pipeline args=%s
triggered trigger_guanaco_pipeline args=%s
Pipeline stage OfflineFamilyFriendlyTriggerPipeline completed in 3.61s
run pipeline stage %s
Running pipeline stage TriggerMKMLProfilingPipeline
run_pipeline:run_in_cloud %s
starting trigger_guanaco_pipeline args=%s
triggered trigger_guanaco_pipeline args=%s
Pipeline stage TriggerMKMLProfilingPipeline completed in 3.42s
Shutdown handler de-registered
mistralai-mistral-smal_88026_v68 status is now deployed due to DeploymentManager action
Inference service chaiml-mistral-24b-2048-41286-v1 ready after 151.93670964241028s
Pipeline stage MKMLDeployer completed in 154.41s
run pipeline stage %s
Running pipeline stage StressChecker
Received healthy response to inference request in 3.380140781402588s
Received healthy response to inference request in 3.284905433654785s
Received healthy response to inference request in 3.4388692378997803s
Received healthy response to inference request in 3.2603344917297363s
Received healthy response to inference request in 3.5439951419830322s
5 requests
0 failed requests
5th percentile: 3.265248680114746
10th percentile: 3.2701628684997557
20th percentile: 3.2799912452697755
30th percentile: 3.3039525032043455
40th percentile: 3.3420466423034667
50th percentile: 3.380140781402588
60th percentile: 3.403632164001465
70th percentile: 3.427123546600342
80th percentile: 3.4598944187164307
90th percentile: 3.5019447803497314
95th percentile: 3.522969961166382
99th percentile: 3.5397901058197023
mean time: 3.3816490173339844
Pipeline stage StressChecker completed in 27.86s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyTriggerPipeline
run_pipeline:run_in_cloud %s
Inference service chaiml-mistral-24b-2048-87648-v1 ready after 162.25142073631287s
starting trigger_guanaco_pipeline args=%s
Pipeline stage MKMLDeployer completed in 165.09s
run pipeline stage %s
triggered trigger_guanaco_pipeline args=%s
Running pipeline stage StressChecker
Pipeline stage OfflineFamilyFriendlyTriggerPipeline completed in 2.65s
run pipeline stage %s
Running pipeline stage TriggerMKMLProfilingPipeline
run_pipeline:run_in_cloud %s
starting trigger_guanaco_pipeline args=%s
triggered trigger_guanaco_pipeline args=%s
Pipeline stage TriggerMKMLProfilingPipeline completed in 2.40s
Shutdown handler de-registered
chaiml-mistral-24b-2048_41286_v1 status is now deployed due to DeploymentManager action
Received healthy response to inference request in 3.2944087982177734s
Received healthy response to inference request in 2.9384870529174805s
Received healthy response to inference request in 2.9797377586364746s
Received healthy response to inference request in 3.3583693504333496s
5 requests
0 failed requests
5th percentile: 2.946737194061279
10th percentile: 2.954987335205078
20th percentile: 2.971487617492676
30th percentile: 3.042671966552734
40th percentile: 3.168540382385254
50th percentile: 3.2944087982177734
60th percentile: 3.319993019104004
70th percentile: 3.3455772399902344
80th percentile: 3.3905468940734864
90th percentile: 3.4549019813537596
95th percentile: 3.4870795249938964
99th percentile: 3.5128215599060058
mean time: 3.2180520057678224
Pipeline stage StressChecker completed in 23.41s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyTriggerPipeline
run_pipeline:run_in_cloud %s
starting trigger_guanaco_pipeline args=%s
Inference service chaiml-mistral-24b-2048-76638-v1 ready after 152.13292050361633s
Pipeline stage MKMLDeployer completed in 154.88s
triggered trigger_guanaco_pipeline args=%s
run pipeline stage %s
Pipeline stage OfflineFamilyFriendlyTriggerPipeline completed in 2.14s
Running pipeline stage StressChecker
run pipeline stage %s
Running pipeline stage TriggerMKMLProfilingPipeline
run_pipeline:run_in_cloud %s
starting trigger_guanaco_pipeline args=%s
triggered trigger_guanaco_pipeline args=%s
Pipeline stage TriggerMKMLProfilingPipeline completed in 2.64s
Received healthy response to inference request in 3.0888514518737793s
Shutdown handler de-registered
chaiml-mistral-24b-2048_87648_v1 status is now deployed due to DeploymentManager action
Shutdown handler registered
run pipeline %s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyScorer
Evaluating %s Family Friendly Score with %s threads
Generating Leaderboard row for %s
Generated Leaderboard row for %s
Pipeline stage OfflineFamilyFriendlyScorer completed in 2675.63s
Shutdown handler de-registered
chaiml-mistral-24b-2048_87648_v1 status is now inactive due to auto deactivation removed underperforming models
chaiml-mistral-24b-2048_87648_v1 status is now torndown due to DeploymentManager action