developer_uid: chai_backend_admin
submission_id: chaiml-02f4-69d4-linear-w01_v42
model_name: chaiml-02f4-69d4-linear-w01_v42
model_group: ChaiML/02f4-69d4-linear-
status: inactive
timestamp: 2025-12-01T17:15:24+00:00
num_battles: 6860
num_wins: 3642
celo_rating: 1312.41
family_friendly_score: 0.0
family_friendly_standard_error: 0.0
submission_type: basic
model_repo: ChaiML/02f4-69d4-linear-w01
model_architecture: MistralForCausalLM
model_num_parameters: 24096691200.0
best_of: 8
max_input_tokens: 1024
max_output_tokens: 64
reward_model: default
display_name: chaiml-02f4-69d4-linear-w01_v42
is_internal_developer: True
language_model: ChaiML/02f4-69d4-linear-w01
model_size: 24B
ranking_group: single
us_pacific_date: 2025-12-01
win_ratio: 0.5309037900874636
generation_params: {'temperature': 0.7, 'top_p': 0.95, 'min_p': 0.025, 'top_k': 80, 'presence_penalty': 0.4, 'frequency_penalty': 0.4, 'stopping_words': ['\n'], 'max_input_tokens': 1024, 'best_of': 8, 'max_output_tokens': 64}
formatter: {'memory_template': '<|im_start|>system\n{memory}<|im_end|>\n', 'prompt_template': '', 'bot_template': '<|im_start|>assistant\n{bot_name}: {message}<|im_end|>\n', 'user_template': '<|im_start|>user\n{user_name}: {message}<|im_end|>\n', 'response_template': '<|im_start|>assistant\n{bot_name}:', 'truncate_by_message': True}
Resubmit model
Shutdown handler not registered because Python interpreter is not running in the main thread
run pipeline %s
run pipeline stage %s
Running pipeline stage MKMLizer
Starting job with name chaiml-02f4-69d4-linear-w01-v42-mkmlizer
Waiting for job on chaiml-02f4-69d4-linear-w01-v42-mkmlizer to finish
Shutdown handler not registered because Python interpreter is not running in the main thread
run pipeline %s
run pipeline stage %s
Running pipeline stage MKMLizer
Starting job with name chaiml-7b07-69d4-linear-w01-v25-mkmlizer
Waiting for job on chaiml-7b07-69d4-linear-w01-v25-mkmlizer to finish
HTTP Request: %s %s "%s %d %s"
Shutdown handler not registered because Python interpreter is not running in the main thread
run pipeline %s
run pipeline stage %s
Running pipeline stage MKMLizer
Starting job with name chaiml-2fe5-c13f-linear-w01-v30-mkmlizer
Waiting for job on chaiml-2fe5-c13f-linear-w01-v30-mkmlizer to finish
chaiml-02f4-69d4-linear-w01-v42-mkmlizer: /root/miniconda3/envs/nvidia/lib/python3.11/site-packages/mk1/__init__.py:1: UserWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html. The pkg_resources package is slated for removal as early as 2025-11-30. Refrain from using this package or pin to Setuptools<81.
chaiml-02f4-69d4-linear-w01-v42-mkmlizer:   __import__('pkg_resources').declare_namespace(__name__)
chaiml-02f4-69d4-linear-w01-v42-mkmlizer: ╔═════════════════════════════════════════════════════════════════════╗
chaiml-02f4-69d4-linear-w01-v42-mkmlizer: ║                                                                     ║
chaiml-02f4-69d4-linear-w01-v42-mkmlizer: ║   ██████   ██████  █████   ████  ████                               ║
chaiml-02f4-69d4-linear-w01-v42-mkmlizer: ║  ░░██████ ██████  ░░███   ███░  ░░███                               ║
chaiml-02f4-69d4-linear-w01-v42-mkmlizer: ║   ░███░█████░███   ░███  ███     ░███                               ║
chaiml-02f4-69d4-linear-w01-v42-mkmlizer: ║   ░███░░███ ░███   ░███████      ░███                               ║
chaiml-02f4-69d4-linear-w01-v42-mkmlizer: ║   ░███ ░░░  ░███   ░███░░███     ░███                               ║
chaiml-02f4-69d4-linear-w01-v42-mkmlizer: ║   ░███      ░███   ░███ ░░███    ░███                               ║
chaiml-02f4-69d4-linear-w01-v42-mkmlizer: ║   █████     █████  █████ ░░████  █████                              ║
chaiml-02f4-69d4-linear-w01-v42-mkmlizer: ║  ░░░░░     ░░░░░  ░░░░░   ░░░░  ░░░░░                               ║
chaiml-02f4-69d4-linear-w01-v42-mkmlizer: ║                                                                     ║
chaiml-02f4-69d4-linear-w01-v42-mkmlizer: ║  Version: 0.30.6+torch280                                           ║
chaiml-02f4-69d4-linear-w01-v42-mkmlizer: ║  Features: FLYWHEEL, CUDA                                           ║
chaiml-02f4-69d4-linear-w01-v42-mkmlizer: ║  Copyright 2023-2025 MK ONE TECHNOLOGIES Inc.                       ║
chaiml-02f4-69d4-linear-w01-v42-mkmlizer: ║  https://mk1.ai                                                     ║
chaiml-02f4-69d4-linear-w01-v42-mkmlizer: ║                                                                     ║
chaiml-02f4-69d4-linear-w01-v42-mkmlizer: ║  The license key for the current software has been verified as      ║
chaiml-02f4-69d4-linear-w01-v42-mkmlizer: ║  belonging to:                                                      ║
chaiml-02f4-69d4-linear-w01-v42-mkmlizer: ║                                                                     ║
chaiml-02f4-69d4-linear-w01-v42-mkmlizer: ║  Chai Research Corp.                                                ║
chaiml-02f4-69d4-linear-w01-v42-mkmlizer: ║  Account ID: 7997a29f-0ceb-4cc7-9adf-840c57b4ae6f                   ║
chaiml-7b07-69d4-linear-w01-v25-mkmlizer: bash: cannot set terminal process group (-1): Inappropriate ioctl for device
chaiml-02f4-69d4-linear-w01-v42-mkmlizer: ║  Expiration: 2028-03-31 23:59:59                                    ║
chaiml-7b07-69d4-linear-w01-v25-mkmlizer: bash: no job control in this shell
chaiml-02f4-69d4-linear-w01-v42-mkmlizer: ║                                                                     ║
chaiml-7b07-69d4-linear-w01-v25-mkmlizer: /root/miniconda3/envs/nvidia/lib/python3.11/site-packages/mk1/__init__.py:1: UserWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html. The pkg_resources package is slated for removal as early as 2025-11-30. Refrain from using this package or pin to Setuptools<81.
chaiml-02f4-69d4-linear-w01-v42-mkmlizer: ╚═════════════════════════════════════════════════════════════════════╝
chaiml-7b07-69d4-linear-w01-v25-mkmlizer:   __import__('pkg_resources').declare_namespace(__name__)
chaiml-7b07-69d4-linear-w01-v25-mkmlizer: ╔═════════════════════════════════════════════════════════════════════╗
chaiml-7b07-69d4-linear-w01-v25-mkmlizer: ║                                                                     ║
chaiml-7b07-69d4-linear-w01-v25-mkmlizer: ║   ██████   ██████  █████   ████  ████                               ║
chaiml-7b07-69d4-linear-w01-v25-mkmlizer: ║  ░░██████ ██████  ░░███   ███░  ░░███                               ║
chaiml-7b07-69d4-linear-w01-v25-mkmlizer: ║   ░███░█████░███   ░███  ███     ░███                               ║
chaiml-7b07-69d4-linear-w01-v25-mkmlizer: ║   ░███░░███ ░███   ░███████      ░███                               ║
chaiml-7b07-69d4-linear-w01-v25-mkmlizer: ║   ░███ ░░░  ░███   ░███░░███     ░███                               ║
chaiml-7b07-69d4-linear-w01-v25-mkmlizer: ║   ░███      ░███   ░███ ░░███    ░███                               ║
chaiml-7b07-69d4-linear-w01-v25-mkmlizer: ║   █████     █████  █████ ░░████  █████                              ║
chaiml-7b07-69d4-linear-w01-v25-mkmlizer: ║  ░░░░░     ░░░░░  ░░░░░   ░░░░  ░░░░░                               ║
chaiml-7b07-69d4-linear-w01-v25-mkmlizer: ║                                                                     ║
chaiml-7b07-69d4-linear-w01-v25-mkmlizer: ║  Version: 0.30.6+torch280                                           ║
chaiml-7b07-69d4-linear-w01-v25-mkmlizer: ║  Features: FLYWHEEL, CUDA                                           ║
chaiml-7b07-69d4-linear-w01-v25-mkmlizer: ║  Copyright 2023-2025 MK ONE TECHNOLOGIES Inc.                       ║
chaiml-7b07-69d4-linear-w01-v25-mkmlizer: ║  https://mk1.ai                                                     ║
chaiml-7b07-69d4-linear-w01-v25-mkmlizer: ║                                                                     ║
chaiml-7b07-69d4-linear-w01-v25-mkmlizer: ║  The license key for the current software has been verified as      ║
chaiml-7b07-69d4-linear-w01-v25-mkmlizer: ║  belonging to:                                                      ║
chaiml-7b07-69d4-linear-w01-v25-mkmlizer: ║                                                                     ║
chaiml-7b07-69d4-linear-w01-v25-mkmlizer: ║  Chai Research Corp.                                                ║
chaiml-7b07-69d4-linear-w01-v25-mkmlizer: ║  Account ID: 7997a29f-0ceb-4cc7-9adf-840c57b4ae6f                   ║
chaiml-7b07-69d4-linear-w01-v25-mkmlizer: ║  Expiration: 2028-03-31 23:59:59                                    ║
chaiml-7b07-69d4-linear-w01-v25-mkmlizer: ║                                                                     ║
chaiml-7b07-69d4-linear-w01-v25-mkmlizer: ╚═════════════════════════════════════════════════════════════════════╝
chaiml-2fe5-c13f-linear-w01-v30-mkmlizer: bash: cannot set terminal process group (-1): Inappropriate ioctl for device
chaiml-2fe5-c13f-linear-w01-v30-mkmlizer: bash: no job control in this shell
chaiml-2fe5-c13f-linear-w01-v30-mkmlizer: /root/miniconda3/envs/nvidia/lib/python3.11/site-packages/mk1/__init__.py:1: UserWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html. The pkg_resources package is slated for removal as early as 2025-11-30. Refrain from using this package or pin to Setuptools<81.
chaiml-2fe5-c13f-linear-w01-v30-mkmlizer:   __import__('pkg_resources').declare_namespace(__name__)
chaiml-2fe5-c13f-linear-w01-v30-mkmlizer: ╔═════════════════════════════════════════════════════════════════════╗
chaiml-2fe5-c13f-linear-w01-v30-mkmlizer: ║                                                                     ║
chaiml-2fe5-c13f-linear-w01-v30-mkmlizer: ║   ██████   ██████  █████   ████  ████                               ║
chaiml-02f4-69d4-linear-w01-v42-mkmlizer: Downloaded to shared memory in 47.001s
chaiml-2fe5-c13f-linear-w01-v30-mkmlizer: ║  ░░██████ ██████  ░░███   ███░  ░░███                               ║
chaiml-02f4-69d4-linear-w01-v42-mkmlizer: Checking if ChaiML/02f4-69d4-linear-w01 already exists in ChaiML
chaiml-2fe5-c13f-linear-w01-v30-mkmlizer: ║   ░███░█████░███   ░███  ███     ░███                               ║
chaiml-02f4-69d4-linear-w01-v42-mkmlizer: quantizing model to /dev/shm/model_cache, profile:q4, folder:/tmp/tmp84l6pd2j, device:0
chaiml-2fe5-c13f-linear-w01-v30-mkmlizer: ║   ░███░░███ ░███   ░███████      ░███                               ║
chaiml-02f4-69d4-linear-w01-v42-mkmlizer: Saving flywheel model at /dev/shm/model_cache
chaiml-2fe5-c13f-linear-w01-v30-mkmlizer: ║   ░███ ░░░  ░███   ░███░░███     ░███                               ║
chaiml-2fe5-c13f-linear-w01-v30-mkmlizer: ║   ░███      ░███   ░███ ░░███    ░███                               ║
chaiml-2fe5-c13f-linear-w01-v30-mkmlizer: ║   █████     █████  █████ ░░████  █████                              ║
chaiml-2fe5-c13f-linear-w01-v30-mkmlizer: ║  ░░░░░     ░░░░░  ░░░░░   ░░░░  ░░░░░                               ║
chaiml-2fe5-c13f-linear-w01-v30-mkmlizer: ║                                                                     ║
chaiml-2fe5-c13f-linear-w01-v30-mkmlizer: ║  Version: 0.30.6+torch280                                           ║
chaiml-2fe5-c13f-linear-w01-v30-mkmlizer: ║  Features: FLYWHEEL, CUDA                                           ║
chaiml-2fe5-c13f-linear-w01-v30-mkmlizer: ║  Copyright 2023-2025 MK ONE TECHNOLOGIES Inc.                       ║
chaiml-2fe5-c13f-linear-w01-v30-mkmlizer: ║  https://mk1.ai                                                     ║
chaiml-2fe5-c13f-linear-w01-v30-mkmlizer: ║                                                                     ║
chaiml-2fe5-c13f-linear-w01-v30-mkmlizer: ║  The license key for the current software has been verified as      ║
chaiml-2fe5-c13f-linear-w01-v30-mkmlizer: ║  belonging to:                                                      ║
chaiml-2fe5-c13f-linear-w01-v30-mkmlizer: ║                                                                     ║
chaiml-2fe5-c13f-linear-w01-v30-mkmlizer: ║  Chai Research Corp.                                                ║
chaiml-2fe5-c13f-linear-w01-v30-mkmlizer: ║  Account ID: 7997a29f-0ceb-4cc7-9adf-840c57b4ae6f                   ║
chaiml-2fe5-c13f-linear-w01-v30-mkmlizer: ║  Expiration: 2028-03-31 23:59:59                                    ║
chaiml-2fe5-c13f-linear-w01-v30-mkmlizer: ║                                                                     ║
chaiml-2fe5-c13f-linear-w01-v30-mkmlizer: ╚═════════════════════════════════════════════════════════════════════╝
chaiml-7b07-69d4-linear-w01-v25-mkmlizer: Downloaded to shared memory in 50.271s
chaiml-7b07-69d4-linear-w01-v25-mkmlizer: Checking if ChaiML/7b07-69d4-linear-w01 already exists in ChaiML
chaiml-7b07-69d4-linear-w01-v25-mkmlizer: quantizing model to /dev/shm/model_cache, profile:q4, folder:/tmp/tmpqhwkwy5l, device:0
chaiml-7b07-69d4-linear-w01-v25-mkmlizer: Saving flywheel model at /dev/shm/model_cache
chaiml-2fe5-c13f-linear-w01-v30-mkmlizer: Downloaded to shared memory in 25.987s
chaiml-2fe5-c13f-linear-w01-v30-mkmlizer: Checking if ChaiML/2fe5-c13f-linear-w01 already exists in ChaiML
chaiml-2fe5-c13f-linear-w01-v30-mkmlizer: quantizing model to /dev/shm/model_cache, profile:q4, folder:/tmp/tmphyie7jp8, device:0
chaiml-2fe5-c13f-linear-w01-v30-mkmlizer: Saving flywheel model at /dev/shm/model_cache
chaiml-2fe5-c13f-linear-w01-v30-mkmlizer: 
Loading 0:   0%|          | 0.00/363 [00:00<?, ?it/s]
Loading 0:   1%|▏         | 5.00/363 [00:01<02:09, 2.76it/s]
Loading 0:   1%|▏         | 5.00/363 [00:01<02:09, 2.76it/s]
Loading 0:   2%|▏         | 8.00/363 [00:02<02:04, 2.85it/s]
Loading 0:   2%|▏         | 8.00/363 [00:02<02:04, 2.85it/s]
Loading 0:   4%|▎         | 13.0/363 [00:04<01:56, 2.99it/s]
Loading 0:   4%|▎         | 13.0/363 [00:04<01:56, 2.99it/s]
Loading 0:   4%|▍         | 15.0/363 [00:06<02:39, 2.19it/s]
Loading 0:   4%|▍         | 15.0/363 [00:06<02:39, 2.19it/s]
Loading 0:   6%|▌         | 22.0/363 [00:07<01:51, 3.06it/s]
Loading 0:   6%|▌         | 22.0/363 [00:07<01:51, 3.06it/s]
Loading 0:   7%|▋         | 24.0/363 [00:09<02:23, 2.36it/s]
Loading 0:   7%|▋         | 24.0/363 [00:09<02:23, 2.36it/s]
Loading 0:   9%|▊         | 31.0/363 [00:10<01:48, 3.07it/s]
Loading 0:   9%|▊         | 31.0/363 [00:10<01:48, 3.07it/s]
Loading 0:   9%|▉         | 33.0/363 [00:12<02:16, 2.42it/s]
Loading 0:   9%|▉         | 33.0/363 [00:12<02:16, 2.42it/s]
Loading 0:  11%|█         | 40.0/363 [00:14<01:47, 3.01it/s]
Loading 0:  11%|█         | 40.0/363 [00:14<01:47, 3.01it/s]
Loading 0:  12%|█▏        | 42.0/363 [00:16<02:13, 2.41it/s]
Loading 0:  12%|█▏        | 42.0/363 [00:16<02:13, 2.41it/s]
Loading 0:  13%|█▎        | 49.0/363 [00:17<01:42, 3.05it/s]
Loading 0:  13%|█▎        | 49.0/363 [00:17<01:42, 3.05it/s]
Loading 0:  14%|█▍        | 51.0/363 [00:19<02:07, 2.45it/s]
Loading 0:  14%|█▍        | 51.0/363 [00:19<02:07, 2.45it/s]
Loading 0:  16%|█▌        | 58.0/363 [00:20<01:38, 3.08it/s]
Loading 0:  16%|█▌        | 58.0/363 [00:20<01:38, 3.08it/s]
Loading 0:  17%|█▋        | 60.0/363 [00:22<02:02, 2.46it/s]
Loading 0:  17%|█▋        | 60.0/363 [00:22<02:02, 2.46it/s]
Loading 0:  18%|█▊        | 67.0/363 [00:23<01:35, 3.08it/s]
Loading 0:  18%|█▊        | 67.0/363 [00:23<01:35, 3.08it/s]
Loading 0:  19%|█▉        | 69.0/363 [00:25<01:59, 2.47it/s]
Loading 0:  19%|█▉        | 69.0/363 [00:25<01:59, 2.47it/s]
Loading 0:  21%|██        | 76.0/363 [00:27<01:32, 3.09it/s]
Loading 0:  21%|██        | 76.0/363 [00:27<01:32, 3.09it/s]
Loading 0:  21%|██▏       | 78.0/363 [00:28<01:55, 2.48it/s]
Loading 0:  21%|██▏       | 78.0/363 [00:28<01:55, 2.48it/s]
Loading 0:  23%|██▎       | 85.0/363 [00:30<01:29, 3.09it/s]
Loading 0:  23%|██▎       | 85.0/363 [00:30<01:29, 3.09it/s]
Loading 0:  24%|██▍       | 87.0/363 [00:32<01:51, 2.48it/s]
Loading 0:  24%|██▍       | 87.0/363 [00:32<01:51, 2.48it/s]
Loading 0:  26%|██▌       | 94.0/363 [00:33<01:26, 3.09it/s]
Loading 0:  26%|██▌       | 94.0/363 [00:33<01:26, 3.09it/s]
Loading 0:  26%|██▋       | 96.0/363 [00:35<01:47, 2.48it/s]
Loading 0:  26%|██▋       | 96.0/363 [00:35<01:47, 2.48it/s]
Loading 0:  28%|██▊       | 103/363 [00:36<01:23, 3.10it/s] 
Loading 0:  28%|██▊       | 103/363 [00:36<01:23, 3.10it/s]
Loading 0:  29%|██▉       | 105/363 [00:38<01:44, 2.48it/s]
Loading 0:  29%|██▉       | 105/363 [00:38<01:44, 2.48it/s]
Loading 0:  31%|███       | 112/363 [00:40<01:20, 3.10it/s]
Loading 0:  31%|███       | 112/363 [00:40<01:20, 3.10it/s]
Loading 0:  31%|███▏      | 114/363 [00:41<01:40, 2.48it/s]
Loading 0:  31%|███▏      | 114/363 [00:41<01:40, 2.48it/s]
Loading 0:  33%|███▎      | 121/363 [00:43<01:20, 3.02it/s]
Loading 0:  33%|███▎      | 121/363 [00:43<01:20, 3.02it/s]
Loading 0:  34%|███▍      | 123/363 [00:45<01:38, 2.44it/s]
Loading 0:  34%|███▍      | 123/363 [00:45<01:38, 2.44it/s]
Loading 0:  36%|███▌      | 130/363 [00:46<01:16, 3.06it/s]
Loading 0:  36%|███▌      | 130/363 [00:46<01:16, 3.06it/s]
Loading 0:  36%|███▋      | 132/363 [00:48<01:33, 2.46it/s]
Loading 0:  36%|███▋      | 132/363 [00:48<01:33, 2.46it/s]
Loading 0:  38%|███▊      | 139/363 [00:49<01:12, 3.10it/s]
Loading 0:  38%|███▊      | 139/363 [00:49<01:12, 3.10it/s]
Loading 0:  39%|███▉      | 141/363 [00:51<01:29, 2.48it/s]
Loading 0:  39%|███▉      | 141/363 [00:51<01:29, 2.48it/s]
Loading 0:  41%|████      | 148/363 [00:53<01:09, 3.11it/s]
Loading 0:  41%|████      | 148/363 [00:53<01:09, 3.11it/s]
Loading 0:  41%|████▏     | 150/363 [00:54<01:25, 2.48it/s]
Loading 0:  41%|████▏     | 150/363 [00:54<01:25, 2.48it/s]
Loading 0:  43%|████▎     | 157/363 [00:56<01:06, 3.11it/s]
Loading 0:  43%|████▎     | 157/363 [00:56<01:06, 3.11it/s]
Loading 0:  44%|████▍     | 159/363 [00:58<01:21, 2.49it/s]
Loading 0:  44%|████▍     | 159/363 [00:58<01:21, 2.49it/s]
Loading 0:  46%|████▌     | 166/363 [00:59<01:02, 3.13it/s]
Loading 0:  46%|████▌     | 166/363 [00:59<01:02, 3.13it/s]
Loading 0:  46%|████▋     | 168/363 [01:01<01:17, 2.50it/s]
Loading 0:  46%|████▋     | 168/363 [01:01<01:17, 2.50it/s]
Loading 0:  48%|████▊     | 175/363 [01:02<01:00, 3.13it/s]
Loading 0:  48%|████▊     | 175/363 [01:02<01:00, 3.13it/s]
Loading 0:  49%|████▉     | 177/363 [01:04<01:14, 2.49it/s]
Loading 0:  49%|████▉     | 177/363 [01:04<01:14, 2.49it/s]
Loading 0:  51%|█████     | 184/363 [01:05<00:57, 3.11it/s]
Loading 0:  51%|█████     | 184/363 [01:05<00:57, 3.11it/s]
Loading 0:  51%|█████     | 186/363 [01:07<01:11, 2.49it/s]
Loading 0:  51%|█████     | 186/363 [01:07<01:11, 2.49it/s]
Loading 0:  53%|█████▎    | 193/363 [01:09<00:54, 3.11it/s]
Loading 0:  53%|█████▎    | 193/363 [01:09<00:54, 3.11it/s]
Loading 0:  54%|█████▎    | 195/363 [01:10<01:07, 2.49it/s]
Loading 0:  54%|█████▎    | 195/363 [01:10<01:07, 2.49it/s]
Loading 0:  56%|█████▌    | 202/363 [01:12<00:53, 3.03it/s]
Loading 0:  56%|█████▌    | 202/363 [01:12<00:53, 3.03it/s]
Loading 0:  56%|█████▌    | 204/363 [01:14<01:04, 2.45it/s]
Loading 0:  56%|█████▌    | 204/363 [01:14<01:04, 2.45it/s]
Loading 0:  58%|█████▊    | 211/363 [01:15<00:49, 3.08it/s]
Loading 0:  58%|█████▊    | 211/363 [01:15<00:49, 3.08it/s]
Loading 0:  59%|█████▊    | 213/363 [01:17<01:00, 2.47it/s]
Loading 0:  59%|█████▊    | 213/363 [01:17<01:00, 2.47it/s]
Loading 0:  61%|██████    | 220/363 [01:18<00:46, 3.10it/s]
Loading 0:  61%|██████    | 220/363 [01:18<00:46, 3.10it/s]
Loading 0:  61%|██████    | 222/363 [01:20<00:56, 2.48it/s]
Loading 0:  61%|██████    | 222/363 [01:20<00:56, 2.48it/s]
Loading 0:  63%|██████▎   | 229/363 [01:22<00:43, 3.11it/s]
Loading 0:  63%|██████▎   | 229/363 [01:22<00:43, 3.11it/s]
Loading 0:  64%|██████▎   | 231/363 [01:23<00:52, 2.50it/s]
Loading 0:  64%|██████▎   | 231/363 [01:23<00:52, 2.50it/s]
Loading 0:  66%|██████▌   | 238/363 [01:25<00:40, 3.12it/s]
Loading 0:  66%|██████▌   | 238/363 [01:25<00:40, 3.12it/s]
Loading 0:  66%|██████▌   | 240/363 [01:27<00:49, 2.50it/s]
Loading 0:  66%|██████▌   | 240/363 [01:27<00:49, 2.50it/s]
Loading 0:  68%|██████▊   | 247/363 [01:28<00:37, 3.12it/s]
Loading 0:  68%|██████▊   | 247/363 [01:28<00:37, 3.12it/s]
Loading 0:  69%|██████▊   | 249/363 [01:30<00:45, 2.50it/s]
Loading 0:  69%|██████▊   | 249/363 [01:30<00:45, 2.50it/s]
Loading 0:  71%|███████   | 256/363 [01:31<00:34, 3.11it/s]
Loading 0:  71%|███████   | 256/363 [01:31<00:34, 3.11it/s]
Loading 0:  71%|███████   | 258/363 [01:33<00:42, 2.49it/s]
Loading 0:  71%|███████   | 258/363 [01:33<00:42, 2.49it/s]
Loading 0:  73%|███████▎  | 265/363 [01:35<00:31, 3.12it/s]
Loading 0:  73%|███████▎  | 265/363 [01:35<00:31, 3.12it/s]
Loading 0:  74%|███████▎  | 267/363 [01:36<00:38, 2.49it/s]
Loading 0:  74%|███████▎  | 267/363 [01:36<00:38, 2.49it/s]
Loading 0:  75%|███████▌  | 274/363 [01:38<00:28, 3.11it/s]
Loading 0:  75%|███████▌  | 274/363 [01:38<00:28, 3.11it/s]
Loading 0:  76%|███████▌  | 276/363 [01:39<00:34, 2.49it/s]
Loading 0:  76%|███████▌  | 276/363 [01:39<00:34, 2.49it/s]
Loading 0:  78%|███████▊  | 283/363 [01:41<00:26, 3.03it/s]
Loading 0:  78%|███████▊  | 283/363 [01:41<00:26, 3.03it/s]
Loading 0:  79%|███████▊  | 285/363 [01:43<00:31, 2.45it/s]
Loading 0:  79%|███████▊  | 285/363 [01:43<00:31, 2.45it/s]
Loading 0:  80%|████████  | 292/363 [01:44<00:23, 3.07it/s]
Loading 0:  80%|████████  | 292/363 [01:44<00:23, 3.07it/s]
Loading 0:  81%|████████  | 294/363 [01:46<00:27, 2.47it/s]
Loading 0:  81%|████████  | 294/363 [01:46<00:27, 2.47it/s]
Loading 0:  83%|████████▎ | 301/363 [01:48<00:20, 3.08it/s]
Loading 0:  83%|████████▎ | 301/363 [01:48<00:20, 3.08it/s]
Loading 0:  83%|████████▎ | 303/363 [01:49<00:24, 2.46it/s]
Loading 0:  83%|████████▎ | 303/363 [01:49<00:24, 2.46it/s]
Loading 0:  85%|████████▌ | 310/363 [01:51<00:17, 3.08it/s]
Loading 0:  85%|████████▌ | 310/363 [01:51<00:17, 3.08it/s]
Loading 0:  86%|████████▌ | 312/363 [01:53<00:20, 2.47it/s]
Loading 0:  86%|████████▌ | 312/363 [01:53<00:20, 2.47it/s]
Loading 0:  88%|████████▊ | 319/363 [01:54<00:14, 3.09it/s]
Loading 0:  88%|████████▊ | 319/363 [01:54<00:14, 3.09it/s]
Loading 0:  88%|████████▊ | 321/363 [01:56<00:16, 2.47it/s]
Loading 0:  88%|████████▊ | 321/363 [01:56<00:16, 2.47it/s]
Loading 0:  90%|█████████ | 328/363 [01:57<00:11, 3.09it/s]
Loading 0:  90%|█████████ | 328/363 [01:57<00:11, 3.09it/s]
Loading 0:  91%|█████████ | 330/363 [01:59<00:13, 2.46it/s]
Loading 0:  91%|█████████ | 330/363 [01:59<00:13, 2.46it/s]
Loading 0:  93%|█████████▎| 337/363 [02:01<00:08, 3.09it/s]
Loading 0:  93%|█████████▎| 337/363 [02:01<00:08, 3.09it/s]
Loading 0:  93%|█████████▎| 339/363 [02:02<00:09, 2.47it/s]
Loading 0:  93%|█████████▎| 339/363 [02:02<00:09, 2.47it/s]
Loading 0:  95%|█████████▌| 346/363 [02:04<00:05, 3.09it/s]
Loading 0:  95%|█████████▌| 346/363 [02:04<00:05, 3.09it/s]
Loading 0:  96%|█████████▌| 348/363 [02:05<00:06, 2.47it/s]
Loading 0:  96%|█████████▌| 348/363 [02:05<00:06, 2.47it/s]
Loading 0:  98%|█████████▊| 355/363 [02:07<00:02, 3.10it/s]
Loading 0:  98%|█████████▊| 355/363 [02:07<00:02, 3.10it/s]
Loading 0:  98%|█████████▊| 357/363 [02:09<00:02, 2.47it/s]
Loading 0:  98%|█████████▊| 357/363 [02:09<00:02, 2.47it/s]
Loading 0: 100%|██████████| 363/363 [02:09<00:00, 3.40it/s]
Loading 0: 100%|██████████| 363/363 [02:09<00:00, 3.40it/s]
                                                           

Loading 0: 100%|██████████| 363/363 [02:09<00:00, 2.79it/s]
chaiml-2fe5-c13f-linear-w01-v30-mkmlizer: The tokenizer you are loading from '/tmp/tmphyie7jp8' with an incorrect regex pattern: https://huggingface.co/mistralai/Mistral-Small-3.1-24B-Instruct-2503/discussions/84#69121093e8b480e709447d5e. This will lead to incorrect tokenization. You should set the `fix_mistral_regex=True` flag when loading this tokenizer to fix this issue.
chaiml-2fe5-c13f-linear-w01-v30-mkmlizer: quantized model in 141.568s
chaiml-2fe5-c13f-linear-w01-v30-mkmlizer: Processed model ChaiML/2fe5-c13f-linear-w01 in 167.556s
chaiml-2fe5-c13f-linear-w01-v30-mkmlizer: creating bucket guanaco-mkml-models
chaiml-2fe5-c13f-linear-w01-v30-mkmlizer: Bucket 's3://guanaco-mkml-models/' created
chaiml-2fe5-c13f-linear-w01-v30-mkmlizer: uploading /dev/shm/model_cache to s3://guanaco-mkml-models/chaiml-2fe5-c13f-linear-w01-v30/nvidia
chaiml-2fe5-c13f-linear-w01-v30-mkmlizer: cp /dev/shm/model_cache/special_tokens_map.json s3://guanaco-mkml-models/chaiml-2fe5-c13f-linear-w01-v30/nvidia/special_tokens_map.json
chaiml-2fe5-c13f-linear-w01-v30-mkmlizer: cp /dev/shm/model_cache/config.json s3://guanaco-mkml-models/chaiml-2fe5-c13f-linear-w01-v30/nvidia/config.json
chaiml-2fe5-c13f-linear-w01-v30-mkmlizer: cp /dev/shm/model_cache/chat_template.jinja s3://guanaco-mkml-models/chaiml-2fe5-c13f-linear-w01-v30/nvidia/chat_template.jinja
chaiml-2fe5-c13f-linear-w01-v30-mkmlizer: cp /dev/shm/model_cache/tokenizer_config.json s3://guanaco-mkml-models/chaiml-2fe5-c13f-linear-w01-v30/nvidia/tokenizer_config.json
chaiml-2fe5-c13f-linear-w01-v30-mkmlizer: cp /dev/shm/model_cache/tokenizer.json s3://guanaco-mkml-models/chaiml-2fe5-c13f-linear-w01-v30/nvidia/tokenizer.json
chaiml-2fe5-c13f-linear-w01-v30-mkmlizer: cp /dev/shm/model_cache/flywheel_model.0.safetensors s3://guanaco-mkml-models/chaiml-2fe5-c13f-linear-w01-v30/nvidia/flywheel_model.0.safetensors
Job chaiml-2fe5-c13f-linear-w01-v30-mkmlizer completed after 256.86s with status: succeeded
Stopping job with name chaiml-2fe5-c13f-linear-w01-v30-mkmlizer
Pipeline stage MKMLizer completed in 259.01s
run pipeline stage %s
Running pipeline stage MKMLTemplater
Pipeline stage MKMLTemplater completed in 0.49s
run pipeline stage %s
Running pipeline stage MKMLDeployer
Creating inference service chaiml-2fe5-c13f-linear-w01-v30
Waiting for inference service chaiml-2fe5-c13f-linear-w01-v30 to be ready
HTTP Request: %s %s "%s %d %s"
HTTP Request: %s %s "%s %d %s"
Shutdown handler not registered because Python interpreter is not running in the main thread
run pipeline %s
run pipeline stage %s
Running pipeline stage MKMLizer
Starting job with name chaiml-ca18-c13f-linear-w01-v27-mkmlizer
Waiting for job on chaiml-ca18-c13f-linear-w01-v27-mkmlizer to finish
chaiml-02f4-69d4-linear-w01-v42-mkmlizer: 
Loading 0:   0%|          | 0.00/363 [00:00<?, ?it/s]
Loading 0:   1%|          | 4.00/363 [00:02<03:18, 1.81it/s]
Loading 0:   1%|          | 4.00/363 [00:02<03:18, 1.81it/s]
Loading 0:   1%|▏         | 5.00/363 [00:04<05:19, 1.12it/s]
Loading 0:   1%|▏         | 5.00/363 [00:04<05:19, 1.12it/s]
Loading 0:   2%|▏         | 6.00/363 [00:06<07:04, 1.19s/it]
Loading 0:   2%|▏         | 6.00/363 [00:06<07:04, 1.19s/it]
Loading 0:   4%|▎         | 13.0/363 [00:08<03:26, 1.70it/s]
Loading 0:   4%|▎         | 13.0/363 [00:08<03:26, 1.70it/s]
Loading 0:   4%|▍         | 14.0/363 [00:10<04:22, 1.33it/s]
Loading 0:   4%|▍         | 14.0/363 [00:10<04:22, 1.33it/s]
Loading 0:   4%|▍         | 15.0/363 [00:12<05:27, 1.06it/s]
Loading 0:   4%|▍         | 15.0/363 [00:12<05:27, 1.06it/s]
Loading 0:   6%|▌         | 22.0/363 [00:15<03:25, 1.66it/s]
Loading 0:   6%|▌         | 22.0/363 [00:15<03:25, 1.66it/s]
Loading 0:   6%|▋         | 23.0/363 [00:16<04:11, 1.35it/s]
Loading 0:   6%|▋         | 23.0/363 [00:16<04:11, 1.35it/s]
Loading 0:   7%|▋         | 24.0/363 [00:18<05:07, 1.10it/s]
Loading 0:   7%|▋         | 24.0/363 [00:18<05:07, 1.10it/s]
Loading 0:   9%|▊         | 31.0/363 [00:21<03:16, 1.69it/s]
Loading 0:   9%|▊         | 31.0/363 [00:21<03:16, 1.69it/s]
Loading 0:   9%|▉         | 32.0/363 [00:23<03:59, 1.38it/s]
Loading 0:   9%|▉         | 32.0/363 [00:23<03:59, 1.38it/s]
Loading 0:   9%|▉         | 33.0/363 [00:25<04:52, 1.13it/s]
Loading 0:   9%|▉         | 33.0/363 [00:25<04:52, 1.13it/s]
Loading 0:  11%|█         | 40.0/363 [00:27<03:09, 1.70it/s]
Loading 0:  11%|█         | 40.0/363 [00:27<03:09, 1.70it/s]
Loading 0:  11%|█▏        | 41.0/363 [00:29<03:51, 1.39it/s]
Loading 0:  11%|█▏        | 41.0/363 [00:29<03:51, 1.39it/s]
Loading 0:  12%|█▏        | 42.0/363 [00:31<04:42, 1.14it/s]
Loading 0:  12%|█▏        | 42.0/363 [00:31<04:42, 1.14it/s]
Loading 0:  13%|█▎        | 49.0/363 [00:34<03:04, 1.70it/s]
Loading 0:  13%|█▎        | 49.0/363 [00:34<03:04, 1.70it/s]
Loading 0:  14%|█▍        | 50.0/363 [00:35<03:44, 1.39it/s]
Loading 0:  14%|█▍        | 50.0/363 [00:35<03:44, 1.39it/s]
Loading 0:  14%|█▍        | 51.0/363 [00:37<04:33, 1.14it/s]
Loading 0:  14%|█▍        | 51.0/363 [00:37<04:33, 1.14it/s]
Loading 0:  16%|█▌        | 58.0/363 [00:40<02:59, 1.70it/s]
Loading 0:  16%|█▌        | 58.0/363 [00:40<02:59, 1.70it/s]
Loading 0:  16%|█▋        | 59.0/363 [00:42<03:44, 1.36it/s]
Loading 0:  16%|█▋        | 59.0/363 [00:42<03:44, 1.36it/s]
Loading 0:  17%|█▋        | 60.0/363 [00:44<04:30, 1.12it/s]
Loading 0:  17%|█▋        | 60.0/363 [00:44<04:30, 1.12it/s]
Loading 0:  18%|█▊        | 67.0/363 [00:46<02:55, 1.68it/s]
Loading 0:  18%|█▊        | 67.0/363 [00:46<02:55, 1.68it/s]
Loading 0:  19%|█▊        | 68.0/363 [00:48<03:33, 1.38it/s]
Loading 0:  19%|█▊        | 68.0/363 [00:48<03:33, 1.38it/s]
Loading 0:  19%|█▉        | 69.0/363 [00:50<04:19, 1.13it/s]
Loading 0:  19%|█▉        | 69.0/363 [00:50<04:19, 1.13it/s]
Loading 0:  21%|██        | 76.0/363 [00:53<02:48, 1.70it/s]
Loading 0:  21%|██        | 76.0/363 [00:53<02:48, 1.70it/s]
Loading 0:  21%|██        | 77.0/363 [00:55<03:25, 1.39it/s]
Loading 0:  21%|██        | 77.0/363 [00:55<03:25, 1.39it/s]
Loading 0:  21%|██▏       | 78.0/363 [00:57<04:09, 1.14it/s]
Loading 0:  21%|██▏       | 78.0/363 [00:57<04:09, 1.14it/s]
Loading 0:  23%|██▎       | 85.0/363 [00:59<02:42, 1.71it/s]
Loading 0:  23%|██▎       | 85.0/363 [00:59<02:42, 1.71it/s]
Loading 0:  24%|██▎       | 86.0/363 [01:01<03:18, 1.40it/s]
Loading 0:  24%|██▎       | 86.0/363 [01:01<03:18, 1.40it/s]
Loading 0:  24%|██▍       | 87.0/363 [01:03<04:01, 1.14it/s]
Loading 0:  24%|██▍       | 87.0/363 [01:03<04:01, 1.14it/s]
Loading 0:  26%|██▌       | 94.0/363 [01:05<02:37, 1.71it/s]
Loading 0:  26%|██▌       | 94.0/363 [01:05<02:37, 1.71it/s]
Loading 0:  26%|██▌       | 95.0/363 [01:07<03:11, 1.40it/s]
Loading 0:  26%|██▌       | 95.0/363 [01:07<03:11, 1.40it/s]
Loading 0:  26%|██▋       | 96.0/363 [01:09<03:59, 1.12it/s]
Loading 0:  26%|██▋       | 96.0/363 [01:09<03:59, 1.12it/s]
Loading 0:  28%|██▊       | 103/363 [01:12<02:34, 1.68it/s] 
Loading 0:  28%|██▊       | 103/363 [01:12<02:34, 1.68it/s]
Loading 0:  29%|██▊       | 104/363 [01:14<03:07, 1.38it/s]
Loading 0:  29%|██▊       | 104/363 [01:14<03:07, 1.38it/s]
Loading 0:  29%|██▉       | 105/363 [01:16<03:47, 1.14it/s]
Loading 0:  29%|██▉       | 105/363 [01:16<03:47, 1.14it/s]
Loading 0:  31%|███       | 112/363 [01:18<02:27, 1.70it/s]
Loading 0:  31%|███       | 112/363 [01:18<02:27, 1.70it/s]
Loading 0:  31%|███       | 113/363 [01:20<02:59, 1.39it/s]
Loading 0:  31%|███       | 113/363 [01:20<02:59, 1.39it/s]
Loading 0:  31%|███▏      | 114/363 [01:22<03:38, 1.14it/s]
Loading 0:  31%|███▏      | 114/363 [01:22<03:38, 1.14it/s]
Loading 0:  33%|███▎      | 121/363 [01:25<02:21, 1.71it/s]
Loading 0:  33%|███▎      | 121/363 [01:25<02:21, 1.71it/s]
Loading 0:  34%|███▎      | 122/363 [01:26<02:52, 1.39it/s]
Loading 0:  34%|███▎      | 122/363 [01:26<02:52, 1.39it/s]
Loading 0:  34%|███▍      | 123/363 [01:28<03:30, 1.14it/s]
Loading 0:  34%|███▍      | 123/363 [01:28<03:30, 1.14it/s]
Loading 0:  36%|███▌      | 130/363 [01:31<02:16, 1.70it/s]
Loading 0:  36%|███▌      | 130/363 [01:31<02:16, 1.70it/s]
Loading 0:  36%|███▌      | 131/363 [01:33<02:46, 1.40it/s]
Loading 0:  36%|███▌      | 131/363 [01:33<02:46, 1.40it/s]
Loading 0:  36%|███▋      | 132/363 [01:35<03:22, 1.14it/s]
Loading 0:  36%|███▋      | 132/363 [01:35<03:22, 1.14it/s]
Loading 0:  38%|███▊      | 139/363 [01:37<02:13, 1.68it/s]
Loading 0:  38%|███▊      | 139/363 [01:37<02:13, 1.68it/s]
Loading 0:  39%|███▊      | 140/363 [01:39<02:41, 1.38it/s]
Loading 0:  39%|███▊      | 140/363 [01:39<02:41, 1.38it/s]
Loading 0:  39%|███▉      | 141/363 [01:41<03:15, 1.13it/s]
Loading 0:  39%|███▉      | 141/363 [01:41<03:15, 1.13it/s]
Loading 0:  41%|████      | 148/363 [01:44<02:06, 1.70it/s]
Loading 0:  41%|████      | 148/363 [01:44<02:06, 1.70it/s]
Loading 0:  41%|████      | 149/363 [01:45<02:33, 1.40it/s]
Loading 0:  41%|████      | 149/363 [01:45<02:33, 1.40it/s]
Loading 0:  41%|████▏     | 150/363 [01:47<03:06, 1.14it/s]
Loading 0:  41%|████▏     | 150/363 [01:47<03:06, 1.14it/s]
Loading 0:  43%|████▎     | 157/363 [01:50<02:00, 1.71it/s]
Loading 0:  43%|████▎     | 157/363 [01:50<02:00, 1.71it/s]
Loading 0:  44%|████▎     | 158/363 [01:52<02:26, 1.40it/s]
Loading 0:  44%|████▎     | 158/363 [01:52<02:26, 1.40it/s]
Loading 0:  44%|████▍     | 159/363 [01:54<02:57, 1.15it/s]
Loading 0:  44%|████▍     | 159/363 [01:54<02:57, 1.15it/s]
Loading 0:  46%|████▌     | 166/363 [01:56<01:54, 1.72it/s]
Loading 0:  46%|████▌     | 166/363 [01:56<01:54, 1.72it/s]
Loading 0:  46%|████▌     | 167/363 [01:58<02:19, 1.40it/s]
Loading 0:  46%|████▌     | 167/363 [01:58<02:19, 1.40it/s]
Loading 0:  46%|████▋     | 168/363 [02:00<02:49, 1.15it/s]
Loading 0:  46%|████▋     | 168/363 [02:00<02:49, 1.15it/s]
Loading 0:  48%|████▊     | 175/363 [02:03<01:49, 1.71it/s]
Loading 0:  48%|████▊     | 175/363 [02:03<01:49, 1.71it/s]
Loading 0:  48%|████▊     | 176/363 [02:05<02:16, 1.37it/s]
Loading 0:  48%|████▊     | 176/363 [02:05<02:16, 1.37it/s]
Loading 0:  49%|████▉     | 177/363 [02:07<02:45, 1.13it/s]
Loading 0:  49%|████▉     | 177/363 [02:07<02:45, 1.13it/s]
Loading 0:  51%|█████     | 184/363 [02:09<01:45, 1.69it/s]
Loading 0:  51%|█████     | 184/363 [02:09<01:45, 1.69it/s]
Loading 0:  51%|█████     | 185/363 [02:11<02:08, 1.39it/s]
Loading 0:  51%|█████     | 185/363 [02:11<02:08, 1.39it/s]
Loading 0:  51%|█████     | 186/363 [02:13<02:35, 1.14it/s]
Loading 0:  51%|█████     | 186/363 [02:13<02:35, 1.14it/s]
Loading 0:  53%|█████▎    | 193/363 [02:15<01:39, 1.70it/s]
Loading 0:  53%|█████▎    | 193/363 [02:15<01:39, 1.70it/s]
Loading 0:  53%|█████▎    | 194/363 [02:17<02:01, 1.39it/s]
Loading 0:  53%|█████▎    | 194/363 [02:17<02:01, 1.39it/s]
Loading 0:  54%|█████▎    | 195/363 [02:19<02:26, 1.14it/s]
Loading 0:  54%|█████▎    | 195/363 [02:19<02:26, 1.14it/s]
Loading 0:  56%|█████▌    | 202/363 [02:22<01:34, 1.71it/s]
Loading 0:  56%|█████▌    | 202/363 [02:22<01:34, 1.71it/s]
Loading 0:  56%|█████▌    | 203/363 [02:23<01:54, 1.40it/s]
Loading 0:  56%|█████▌    | 203/363 [02:23<01:54, 1.40it/s]
Loading 0:  56%|█████▌    | 204/363 [02:25<02:18, 1.15it/s]
Loading 0:  56%|█████▌    | 204/363 [02:25<02:18, 1.15it/s]
Loading 0:  58%|█████▊    | 211/363 [02:28<01:28, 1.71it/s]
Loading 0:  58%|█████▊    | 211/363 [02:28<01:28, 1.71it/s]
Loading 0:  58%|█████▊    | 212/363 [02:30<01:47, 1.40it/s]
Loading 0:  58%|█████▊    | 212/363 [02:30<01:47, 1.40it/s]
Loading 0:  59%|█████▊    | 213/363 [02:32<02:14, 1.12it/s]
Loading 0:  59%|█████▊    | 213/363 [02:32<02:14, 1.12it/s]
Loading 0:  61%|██████    | 220/363 [02:34<01:24, 1.68it/s]
Loading 0:  61%|██████    | 220/363 [02:34<01:24, 1.68it/s]
Loading 0:  61%|██████    | 221/363 [02:36<01:42, 1.38it/s]
Loading 0:  61%|██████    | 221/363 [02:36<01:42, 1.38it/s]
Loading 0:  61%|██████    | 222/363 [02:38<02:03, 1.14it/s]
Loading 0:  61%|██████    | 222/363 [02:38<02:03, 1.14it/s]
Loading 0:  63%|██████▎   | 229/363 [02:41<01:18, 1.71it/s]
Loading 0:  63%|██████▎   | 229/363 [02:41<01:18, 1.71it/s]
Loading 0:  63%|██████▎   | 230/363 [02:43<01:35, 1.40it/s]
Loading 0:  63%|██████▎   | 230/363 [02:43<01:35, 1.40it/s]
Loading 0:  64%|██████▎   | 231/363 [02:57<05:29, 2.49s/it]
Loading 0:  64%|██████▎   | 231/363 [02:57<05:29, 2.49s/it]
Loading 0:  66%|██████▌   | 238/363 [03:00<02:41, 1.29s/it]
Loading 0:  66%|██████▌   | 238/363 [03:00<02:41, 1.29s/it]
Loading 0:  66%|██████▌   | 239/363 [03:02<02:47, 1.35s/it]
Loading 0:  66%|██████▌   | 239/363 [03:02<02:47, 1.35s/it]
Loading 0:  66%|██████▌   | 240/363 [03:03<02:54, 1.42s/it]
Loading 0:  66%|██████▌   | 240/363 [03:03<02:54, 1.42s/it]
Loading 0:  68%|██████▊   | 247/363 [03:06<01:33, 1.23it/s]
Loading 0:  68%|██████▊   | 247/363 [03:06<01:33, 1.23it/s]
Loading 0:  68%|██████▊   | 248/363 [03:08<01:45, 1.09it/s]
Loading 0:  68%|██████▊   | 248/363 [03:08<01:45, 1.09it/s]
Loading 0:  69%|██████▊   | 249/363 [03:10<01:59, 1.05s/it]
Loading 0:  69%|██████▊   | 249/363 [03:10<01:59, 1.05s/it]
Loading 0:  71%|███████   | 256/363 [03:12<01:11, 1.49it/s]
Loading 0:  71%|███████   | 256/363 [03:12<01:11, 1.49it/s]
Loading 0:  71%|███████   | 257/363 [03:14<01:23, 1.27it/s]
Loading 0:  71%|███████   | 257/363 [03:14<01:23, 1.27it/s]
Loading 0:  71%|███████   | 258/363 [03:16<01:38, 1.06it/s]
Loading 0:  71%|███████   | 258/363 [03:16<01:38, 1.06it/s]
Loading 0:  73%|███████▎  | 265/363 [03:18<01:00, 1.63it/s]
Loading 0:  73%|███████▎  | 265/363 [03:18<01:00, 1.63it/s]
Loading 0:  73%|███████▎  | 266/363 [03:20<01:11, 1.35it/s]
Loading 0:  73%|███████▎  | 266/363 [03:20<01:11, 1.35it/s]
Loading 0:  74%|███████▎  | 267/363 [03:22<01:26, 1.12it/s]
Loading 0:  74%|███████▎  | 267/363 [03:22<01:26, 1.12it/s]
Loading 0:  75%|███████▌  | 274/363 [03:25<00:52, 1.69it/s]
Loading 0:  75%|███████▌  | 274/363 [03:25<00:52, 1.69it/s]
Loading 0:  76%|███████▌  | 275/363 [03:27<01:03, 1.39it/s]
Loading 0:  76%|███████▌  | 275/363 [03:27<01:03, 1.39it/s]
Loading 0:  76%|███████▌  | 276/363 [03:29<01:16, 1.13it/s]
Loading 0:  76%|███████▌  | 276/363 [03:29<01:16, 1.13it/s]
Loading 0:  78%|███████▊  | 283/363 [03:31<00:46, 1.71it/s]
Loading 0:  78%|███████▊  | 283/363 [03:31<00:46, 1.71it/s]
Loading 0:  78%|███████▊  | 284/363 [03:33<00:56, 1.40it/s]
Loading 0:  78%|███████▊  | 284/363 [03:33<00:56, 1.40it/s]
Loading 0:  79%|███████▊  | 285/363 [03:35<01:08, 1.14it/s]
Loading 0:  79%|███████▊  | 285/363 [03:35<01:08, 1.14it/s]
Loading 0:  80%|████████  | 292/363 [03:37<00:41, 1.72it/s]
Loading 0:  80%|████████  | 292/363 [03:37<00:41, 1.72it/s]
Loading 0:  81%|████████  | 293/363 [03:39<00:51, 1.37it/s]
Loading 0:  81%|████████  | 293/363 [03:39<00:51, 1.37it/s]
Loading 0:  81%|████████  | 294/363 [03:41<01:01, 1.13it/s]
Loading 0:  81%|████████  | 294/363 [03:41<01:01, 1.13it/s]
Loading 0:  83%|████████▎ | 301/363 [03:44<00:36, 1.69it/s]
Loading 0:  83%|████████▎ | 301/363 [03:44<00:36, 1.69it/s]
Loading 0:  83%|████████▎ | 302/363 [03:46<00:43, 1.39it/s]
Loading 0:  83%|████████▎ | 302/363 [03:46<00:43, 1.39it/s]
Loading 0:  83%|████████▎ | 303/363 [03:48<00:52, 1.14it/s]
Loading 0:  83%|████████▎ | 303/363 [03:48<00:52, 1.14it/s]
Loading 0:  85%|████████▌ | 310/363 [03:50<00:31, 1.71it/s]
Loading 0:  85%|████████▌ | 310/363 [03:50<00:31, 1.71it/s]
Loading 0:  86%|████████▌ | 311/363 [03:52<00:37, 1.40it/s]
Loading 0:  86%|████████▌ | 311/363 [03:52<00:37, 1.40it/s]
Loading 0:  86%|████████▌ | 312/363 [03:54<00:44, 1.14it/s]
Loading 0:  86%|████████▌ | 312/363 [03:54<00:44, 1.14it/s]
Loading 0:  88%|████████▊ | 319/363 [03:56<00:25, 1.72it/s]
Loading 0:  88%|████████▊ | 319/363 [03:56<00:25, 1.72it/s]
Loading 0:  88%|████████▊ | 320/363 [03:58<00:30, 1.40it/s]
Loading 0:  88%|████████▊ | 320/363 [03:58<00:30, 1.40it/s]
Loading 0:  88%|████████▊ | 321/363 [04:00<00:36, 1.15it/s]
Loading 0:  88%|████████▊ | 321/363 [04:00<00:36, 1.15it/s]
Loading 0:  90%|█████████ | 328/363 [04:03<00:20, 1.72it/s]
Loading 0:  90%|█████████ | 328/363 [04:03<00:20, 1.72it/s]
Loading 0:  91%|█████████ | 329/363 [04:05<00:24, 1.40it/s]
Loading 0:  91%|█████████ | 329/363 [04:05<00:24, 1.40it/s]
Loading 0:  91%|█████████ | 330/363 [04:07<00:29, 1.12it/s]
Loading 0:  91%|█████████ | 330/363 [04:07<00:29, 1.12it/s]
Loading 0:  93%|█████████▎| 337/363 [04:09<00:15, 1.69it/s]
Loading 0:  93%|█████████▎| 337/363 [04:09<00:15, 1.69it/s]
Loading 0:  93%|█████████▎| 338/363 [04:11<00:18, 1.39it/s]
Loading 0:  93%|█████████▎| 338/363 [04:11<00:18, 1.39it/s]
Loading 0:  93%|█████████▎| 339/363 [04:13<00:21, 1.14it/s]
Loading 0:  93%|█████████▎| 339/363 [04:13<00:21, 1.14it/s]
Loading 0:  95%|█████████▌| 346/363 [04:15<00:09, 1.71it/s]
Loading 0:  95%|█████████▌| 346/363 [04:15<00:09, 1.71it/s]
Loading 0:  96%|█████████▌| 347/363 [04:17<00:11, 1.40it/s]
Loading 0:  96%|█████████▌| 347/363 [04:17<00:11, 1.40it/s]
Loading 0:  96%|█████████▌| 348/363 [04:19<00:13, 1.14it/s]
Loading 0:  96%|█████████▌| 348/363 [04:19<00:13, 1.14it/s]
Loading 0:  98%|█████████▊| 355/363 [04:22<00:04, 1.72it/s]
Loading 0:  98%|█████████▊| 355/363 [04:22<00:04, 1.72it/s]
Loading 0:  98%|█████████▊| 356/363 [04:24<00:04, 1.40it/s]
Loading 0:  98%|█████████▊| 356/363 [04:24<00:04, 1.40it/s]
Loading 0:  98%|█████████▊| 357/363 [04:26<00:05, 1.15it/s]
Loading 0:  98%|█████████▊| 357/363 [04:26<00:05, 1.15it/s]
Loading 0: 100%|██████████| 363/363 [04:26<00:00, 2.10it/s]
Loading 0: 100%|██████████| 363/363 [04:26<00:00, 2.10it/s]
                                                           

Loading 0: 100%|██████████| 363/363 [04:26<00:00, 1.36it/s]
chaiml-02f4-69d4-linear-w01-v42-mkmlizer: The tokenizer you are loading from '/tmp/tmp84l6pd2j' with an incorrect regex pattern: https://huggingface.co/mistralai/Mistral-Small-3.1-24B-Instruct-2503/discussions/84#69121093e8b480e709447d5e. This will lead to incorrect tokenization. You should set the `fix_mistral_regex=True` flag when loading this tokenizer to fix this issue.
chaiml-02f4-69d4-linear-w01-v42-mkmlizer: quantized model in 273.155s
chaiml-02f4-69d4-linear-w01-v42-mkmlizer: Processed model ChaiML/02f4-69d4-linear-w01 in 320.157s
chaiml-7b07-69d4-linear-w01-v25-mkmlizer: 
Loading 0:   0%|          | 0.00/363 [00:00<?, ?it/s]
Loading 0:   1%|          | 4.00/363 [00:02<03:04, 1.95it/s]
Loading 0:   1%|          | 4.00/363 [00:02<03:04, 1.95it/s]
Loading 0:   1%|▏         | 5.00/363 [00:03<05:10, 1.15it/s]
Loading 0:   1%|▏         | 5.00/363 [00:03<05:10, 1.15it/s]
Loading 0:   2%|▏         | 6.00/363 [00:05<06:58, 1.17s/it]
Loading 0:   2%|▏         | 6.00/363 [00:05<06:58, 1.17s/it]
Loading 0:   4%|▎         | 13.0/363 [00:08<03:26, 1.69it/s]
Loading 0:   4%|▎         | 13.0/363 [00:08<03:26, 1.69it/s]
Loading 0:   4%|▍         | 14.0/363 [00:10<04:23, 1.32it/s]
Loading 0:   4%|▍         | 14.0/363 [00:10<04:23, 1.32it/s]
Loading 0:   4%|▍         | 15.0/363 [00:12<05:29, 1.06it/s]
Loading 0:   4%|▍         | 15.0/363 [00:12<05:29, 1.06it/s]
Loading 0:   6%|▌         | 22.0/363 [00:15<03:26, 1.65it/s]
Loading 0:   6%|▌         | 22.0/363 [00:15<03:26, 1.65it/s]
Loading 0:   6%|▋         | 23.0/363 [00:16<04:12, 1.35it/s]
Loading 0:   6%|▋         | 23.0/363 [00:16<04:12, 1.35it/s]
Loading 0:   7%|▋         | 24.0/363 [00:18<05:08, 1.10it/s]
Loading 0:   7%|▋         | 24.0/363 [00:18<05:08, 1.10it/s]
Loading 0:   9%|▊         | 31.0/363 [00:21<03:17, 1.68it/s]
Loading 0:   9%|▊         | 31.0/363 [00:21<03:17, 1.68it/s]
Loading 0:   9%|▉         | 32.0/363 [00:23<04:01, 1.37it/s]
Loading 0:   9%|▉         | 32.0/363 [00:23<04:01, 1.37it/s]
Loading 0:   9%|▉         | 33.0/363 [00:25<04:54, 1.12it/s]
Loading 0:   9%|▉         | 33.0/363 [00:25<04:54, 1.12it/s]
Loading 0:  11%|█         | 40.0/363 [00:27<03:10, 1.69it/s]
Loading 0:  11%|█         | 40.0/363 [00:27<03:10, 1.69it/s]
Loading 0:  11%|█▏        | 41.0/363 [00:29<03:52, 1.38it/s]
Loading 0:  11%|█▏        | 41.0/363 [00:29<03:52, 1.38it/s]
Loading 0:  12%|█▏        | 42.0/363 [00:31<04:43, 1.13it/s]
Loading 0:  12%|█▏        | 42.0/363 [00:31<04:43, 1.13it/s]
Loading 0:  13%|█▎        | 49.0/363 [00:34<03:05, 1.69it/s]
Loading 0:  13%|█▎        | 49.0/363 [00:34<03:05, 1.69it/s]
Loading 0:  14%|█▍        | 50.0/363 [00:36<03:46, 1.38it/s]
Loading 0:  14%|█▍        | 50.0/363 [00:36<03:46, 1.38it/s]
Loading 0:  14%|█▍        | 51.0/363 [00:38<04:35, 1.13it/s]
Loading 0:  14%|█▍        | 51.0/363 [00:38<04:35, 1.13it/s]
Loading 0:  16%|█▌        | 58.0/363 [00:40<03:00, 1.69it/s]
Loading 0:  16%|█▌        | 58.0/363 [00:40<03:00, 1.69it/s]
Loading 0:  16%|█▋        | 59.0/363 [00:42<03:45, 1.35it/s]
Loading 0:  16%|█▋        | 59.0/363 [00:42<03:45, 1.35it/s]
Loading 0:  17%|█▋        | 60.0/363 [00:44<04:32, 1.11it/s]
Loading 0:  17%|█▋        | 60.0/363 [00:44<04:32, 1.11it/s]
Loading 0:  18%|█▊        | 67.0/363 [00:47<02:56, 1.68it/s]
Loading 0:  18%|█▊        | 67.0/363 [00:47<02:56, 1.68it/s]
Loading 0:  19%|█▊        | 68.0/363 [00:48<03:34, 1.38it/s]
Loading 0:  19%|█▊        | 68.0/363 [00:48<03:34, 1.38it/s]
Loading 0:  19%|█▉        | 69.0/363 [00:50<04:20, 1.13it/s]
Loading 0:  19%|█▉        | 69.0/363 [00:50<04:20, 1.13it/s]
Loading 0:  21%|██        | 76.0/363 [00:53<02:49, 1.69it/s]
Loading 0:  21%|██        | 76.0/363 [00:53<02:49, 1.69it/s]
Loading 0:  21%|██        | 77.0/363 [00:55<03:26, 1.39it/s]
Loading 0:  21%|██        | 77.0/363 [00:55<03:26, 1.39it/s]
Loading 0:  21%|██▏       | 78.0/363 [00:57<04:11, 1.14it/s]
Loading 0:  21%|██▏       | 78.0/363 [00:57<04:11, 1.14it/s]
Loading 0:  23%|██▎       | 85.0/363 [00:59<02:44, 1.69it/s]
Loading 0:  23%|██▎       | 85.0/363 [00:59<02:44, 1.69it/s]
Loading 0:  24%|██▎       | 86.0/363 [01:01<03:19, 1.39it/s]
Loading 0:  24%|██▎       | 86.0/363 [01:01<03:19, 1.39it/s]
Loading 0:  24%|██▍       | 87.0/363 [01:03<04:03, 1.13it/s]
Loading 0:  24%|██▍       | 87.0/363 [01:03<04:03, 1.13it/s]
Loading 0:  26%|██▌       | 94.0/363 [01:06<02:38, 1.70it/s]
Loading 0:  26%|██▌       | 94.0/363 [01:06<02:38, 1.70it/s]
Loading 0:  26%|██▌       | 95.0/363 [01:08<03:12, 1.39it/s]
Loading 0:  26%|██▌       | 95.0/363 [01:08<03:12, 1.39it/s]
Loading 0:  26%|██▋       | 96.0/363 [01:10<04:00, 1.11it/s]
Loading 0:  26%|██▋       | 96.0/363 [01:10<04:00, 1.11it/s]
Loading 0:  28%|██▊       | 103/363 [01:12<02:35, 1.68it/s] 
Loading 0:  28%|██▊       | 103/363 [01:12<02:35, 1.68it/s]
Loading 0:  29%|██▊       | 104/363 [01:14<03:08, 1.38it/s]
Loading 0:  29%|██▊       | 104/363 [01:14<03:08, 1.38it/s]
Loading 0:  29%|██▉       | 105/363 [01:16<03:48, 1.13it/s]
Loading 0:  29%|██▉       | 105/363 [01:16<03:48, 1.13it/s]
Loading 0:  31%|███       | 112/363 [01:19<02:28, 1.70it/s]
Loading 0:  31%|███       | 112/363 [01:19<02:28, 1.70it/s]
Loading 0:  31%|███       | 113/363 [01:20<03:00, 1.39it/s]
Loading 0:  31%|███       | 113/363 [01:20<03:00, 1.39it/s]
Loading 0:  31%|███▏      | 114/363 [01:22<03:38, 1.14it/s]
Loading 0:  31%|███▏      | 114/363 [01:22<03:38, 1.14it/s]
Loading 0:  33%|███▎      | 121/363 [01:25<02:22, 1.70it/s]
Loading 0:  33%|███▎      | 121/363 [01:25<02:22, 1.70it/s]
Loading 0:  34%|███▎      | 122/363 [01:27<02:53, 1.39it/s]
Loading 0:  34%|███▎      | 122/363 [01:27<02:53, 1.39it/s]
Loading 0:  34%|███▍      | 123/363 [01:29<03:30, 1.14it/s]
Loading 0:  34%|███▍      | 123/363 [01:29<03:30, 1.14it/s]
Loading 0:  36%|███▌      | 130/363 [01:31<02:17, 1.70it/s]
Loading 0:  36%|███▌      | 130/363 [01:31<02:17, 1.70it/s]
Loading 0:  36%|███▌      | 131/363 [01:33<02:46, 1.39it/s]
Loading 0:  36%|███▌      | 131/363 [01:33<02:46, 1.39it/s]
Loading 0:  36%|███▋      | 132/363 [01:35<03:22, 1.14it/s]
Loading 0:  36%|███▋      | 132/363 [01:35<03:22, 1.14it/s]
Loading 0:  38%|███▊      | 139/363 [01:38<02:14, 1.67it/s]
Loading 0:  38%|███▊      | 139/363 [01:38<02:14, 1.67it/s]
Loading 0:  39%|███▊      | 140/363 [01:40<02:42, 1.37it/s]
Loading 0:  39%|███▊      | 140/363 [01:40<02:42, 1.37it/s]
Loading 0:  39%|███▉      | 141/363 [01:42<03:16, 1.13it/s]
Loading 0:  39%|███▉      | 141/363 [01:42<03:16, 1.13it/s]
Loading 0:  41%|████      | 148/363 [01:44<02:07, 1.69it/s]
Loading 0:  41%|████      | 148/363 [01:44<02:07, 1.69it/s]
Loading 0:  41%|████      | 149/363 [01:46<02:34, 1.39it/s]
Loading 0:  41%|████      | 149/363 [01:46<02:34, 1.39it/s]
Loading 0:  41%|████▏     | 150/363 [01:48<03:07, 1.13it/s]
Loading 0:  41%|████▏     | 150/363 [01:48<03:07, 1.13it/s]
Loading 0:  43%|████▎     | 157/363 [01:50<02:01, 1.70it/s]
Loading 0:  43%|████▎     | 157/363 [01:50<02:01, 1.70it/s]
Loading 0:  44%|████▎     | 158/363 [01:52<02:27, 1.39it/s]
Loading 0:  44%|████▎     | 158/363 [01:52<02:27, 1.39it/s]
Loading 0:  44%|████▍     | 159/363 [01:54<02:58, 1.14it/s]
Loading 0:  44%|████▍     | 159/363 [01:54<02:58, 1.14it/s]
Loading 0:  46%|████▌     | 166/363 [01:57<01:55, 1.70it/s]
Loading 0:  46%|████▌     | 166/363 [01:57<01:55, 1.70it/s]
Loading 0:  46%|████▌     | 167/363 [01:59<02:20, 1.39it/s]
Loading 0:  46%|████▌     | 167/363 [01:59<02:20, 1.39it/s]
Loading 0:  46%|████▋     | 168/363 [02:01<02:51, 1.14it/s]
Loading 0:  46%|████▋     | 168/363 [02:01<02:51, 1.14it/s]
Loading 0:  48%|████▊     | 175/363 [02:03<01:50, 1.71it/s]
Loading 0:  48%|████▊     | 175/363 [02:03<01:50, 1.71it/s]
Loading 0:  48%|████▊     | 176/363 [02:05<02:17, 1.36it/s]
Loading 0:  48%|████▊     | 176/363 [02:05<02:17, 1.36it/s]
Loading 0:  49%|████▉     | 177/363 [02:07<02:45, 1.12it/s]
Loading 0:  49%|████▉     | 177/363 [02:07<02:45, 1.12it/s]
Loading 0:  51%|█████     | 184/363 [02:10<01:46, 1.69it/s]
Loading 0:  51%|█████     | 184/363 [02:10<01:46, 1.69it/s]
Loading 0:  51%|█████     | 185/363 [02:11<02:08, 1.38it/s]
Loading 0:  51%|█████     | 185/363 [02:11<02:08, 1.38it/s]
Loading 0:  51%|█████     | 186/363 [02:13<02:35, 1.14it/s]
Loading 0:  51%|█████     | 186/363 [02:13<02:35, 1.14it/s]
Loading 0:  53%|█████▎    | 193/363 [02:16<01:40, 1.70it/s]
Loading 0:  53%|█████▎    | 193/363 [02:16<01:40, 1.70it/s]
Loading 0:  53%|█████▎    | 194/363 [02:18<02:01, 1.39it/s]
Loading 0:  53%|█████▎    | 194/363 [02:18<02:01, 1.39it/s]
Loading 0:  54%|█████▎    | 195/363 [02:20<02:27, 1.14it/s]
Loading 0:  54%|█████▎    | 195/363 [02:20<02:27, 1.14it/s]
Loading 0:  56%|█████▌    | 202/363 [02:22<01:34, 1.70it/s]
Loading 0:  56%|█████▌    | 202/363 [02:22<01:34, 1.70it/s]
Loading 0:  56%|█████▌    | 203/363 [02:24<01:55, 1.39it/s]
Loading 0:  56%|█████▌    | 203/363 [02:24<01:55, 1.39it/s]
Loading 0:  56%|█████▌    | 204/363 [02:26<02:19, 1.14it/s]
Loading 0:  56%|█████▌    | 204/363 [02:26<02:19, 1.14it/s]
Loading 0:  58%|█████▊    | 211/363 [02:29<01:29, 1.70it/s]
Loading 0:  58%|█████▊    | 211/363 [02:29<01:29, 1.70it/s]
Loading 0:  58%|█████▊    | 212/363 [02:31<01:48, 1.39it/s]
Loading 0:  58%|█████▊    | 212/363 [02:31<01:48, 1.39it/s]
Loading 0:  59%|█████▊    | 213/363 [02:33<02:15, 1.11it/s]
Loading 0:  59%|█████▊    | 213/363 [02:33<02:15, 1.11it/s]
Loading 0:  61%|██████    | 220/363 [02:35<01:25, 1.67it/s]
Loading 0:  61%|██████    | 220/363 [02:35<01:25, 1.67it/s]
Loading 0:  61%|██████    | 221/363 [02:37<01:43, 1.38it/s]
Loading 0:  61%|██████    | 221/363 [02:37<01:43, 1.38it/s]
Loading 0:  61%|██████    | 222/363 [02:39<02:04, 1.13it/s]
Loading 0:  61%|██████    | 222/363 [02:39<02:04, 1.13it/s]
Loading 0:  63%|██████▎   | 229/363 [02:42<01:19, 1.70it/s]
Loading 0:  63%|██████▎   | 229/363 [02:42<01:19, 1.70it/s]
Loading 0:  63%|██████▎   | 230/363 [02:43<01:35, 1.39it/s]
Loading 0:  63%|██████▎   | 230/363 [02:43<01:35, 1.39it/s]
Loading 0:  64%|██████▎   | 231/363 [02:58<05:34, 2.54s/it]
Loading 0:  64%|██████▎   | 231/363 [02:58<05:34, 2.54s/it]
Loading 0:  66%|██████▌   | 238/363 [03:01<02:43, 1.31s/it]
Loading 0:  66%|██████▌   | 238/363 [03:01<02:43, 1.31s/it]
Loading 0:  66%|██████▌   | 239/363 [03:03<02:49, 1.36s/it]
Loading 0:  66%|██████▌   | 239/363 [03:03<02:49, 1.36s/it]
Loading 0:  66%|██████▌   | 240/363 [03:05<02:57, 1.44s/it]
Loading 0:  66%|██████▌   | 240/363 [03:05<02:57, 1.44s/it]
Loading 0:  68%|██████▊   | 247/363 [03:07<01:36, 1.20it/s]
Loading 0:  68%|██████▊   | 247/363 [03:07<01:36, 1.20it/s]
Loading 0:  68%|██████▊   | 248/363 [03:09<01:48, 1.06it/s]
Loading 0:  68%|██████▊   | 248/363 [03:09<01:48, 1.06it/s]
Loading 0:  69%|██████▊   | 249/363 [03:11<02:02, 1.07s/it]
Loading 0:  69%|██████▊   | 249/363 [03:11<02:02, 1.07s/it]
Loading 0:  71%|███████   | 256/363 [03:14<01:12, 1.47it/s]
Loading 0:  71%|███████   | 256/363 [03:14<01:12, 1.47it/s]
Loading 0:  71%|███████   | 257/363 [03:15<01:25, 1.24it/s]
Loading 0:  71%|███████   | 257/363 [03:15<01:25, 1.24it/s]
Loading 0:  71%|███████   | 258/363 [03:17<01:40, 1.05it/s]
Loading 0:  71%|███████   | 258/363 [03:17<01:40, 1.05it/s]
Loading 0:  73%|███████▎  | 265/363 [03:20<01:00, 1.62it/s]
Loading 0:  73%|███████▎  | 265/363 [03:20<01:00, 1.62it/s]
Loading 0:  73%|███████▎  | 266/363 [03:22<01:12, 1.34it/s]
Loading 0:  73%|███████▎  | 266/363 [03:22<01:12, 1.34it/s]
Loading 0:  74%|███████▎  | 267/363 [03:24<01:26, 1.11it/s]
Loading 0:  74%|███████▎  | 267/363 [03:24<01:26, 1.11it/s]
Loading 0:  75%|███████▌  | 274/363 [03:26<00:53, 1.67it/s]
Loading 0:  75%|███████▌  | 274/363 [03:26<00:53, 1.67it/s]
Loading 0:  76%|███████▌  | 275/363 [03:28<01:04, 1.37it/s]
Loading 0:  76%|███████▌  | 275/363 [03:28<01:04, 1.37it/s]
Loading 0:  76%|███████▌  | 276/363 [03:30<01:17, 1.13it/s]
Loading 0:  76%|███████▌  | 276/363 [03:30<01:17, 1.13it/s]
Loading 0:  78%|███████▊  | 283/363 [03:33<00:47, 1.69it/s]
Loading 0:  78%|███████▊  | 283/363 [03:33<00:47, 1.69it/s]
Loading 0:  78%|███████▊  | 284/363 [03:34<00:57, 1.39it/s]
Loading 0:  78%|███████▊  | 284/363 [03:34<00:57, 1.39it/s]
Loading 0:  79%|███████▊  | 285/363 [03:36<01:09, 1.13it/s]
Loading 0:  79%|███████▊  | 285/363 [03:36<01:09, 1.13it/s]
Loading 0:  80%|████████  | 292/363 [03:39<00:41, 1.69it/s]
Loading 0:  80%|████████  | 292/363 [03:39<00:41, 1.69it/s]
Loading 0:  81%|████████  | 293/363 [03:41<00:52, 1.35it/s]
Loading 0:  81%|████████  | 293/363 [03:41<00:52, 1.35it/s]
Loading 0:  81%|████████  | 294/363 [03:43<01:02, 1.11it/s]
Loading 0:  81%|████████  | 294/363 [03:43<01:02, 1.11it/s]
Loading 0:  83%|████████▎ | 301/363 [03:46<00:37, 1.67it/s]
Loading 0:  83%|████████▎ | 301/363 [03:46<00:37, 1.67it/s]
Loading 0:  83%|████████▎ | 302/363 [03:47<00:44, 1.37it/s]
Loading 0:  83%|████████▎ | 302/363 [03:47<00:44, 1.37it/s]
Loading 0:  83%|████████▎ | 303/363 [03:49<00:53, 1.13it/s]
Loading 0:  83%|████████▎ | 303/363 [03:49<00:53, 1.13it/s]
Loading 0:  85%|████████▌ | 310/363 [03:52<00:31, 1.69it/s]
Loading 0:  85%|████████▌ | 310/363 [03:52<00:31, 1.69it/s]
Loading 0:  86%|████████▌ | 311/363 [03:54<00:37, 1.39it/s]
Loading 0:  86%|████████▌ | 311/363 [03:54<00:37, 1.39it/s]
Loading 0:  86%|████████▌ | 312/363 [03:56<00:45, 1.13it/s]
Loading 0:  86%|████████▌ | 312/363 [03:56<00:45, 1.13it/s]
Loading 0:  88%|████████▊ | 319/363 [03:58<00:25, 1.70it/s]
Loading 0:  88%|████████▊ | 319/363 [03:58<00:25, 1.70it/s]
Loading 0:  88%|████████▊ | 320/363 [04:00<00:30, 1.39it/s]
Loading 0:  88%|████████▊ | 320/363 [04:00<00:30, 1.39it/s]
Loading 0:  88%|████████▊ | 321/363 [04:02<00:36, 1.14it/s]
Loading 0:  88%|████████▊ | 321/363 [04:02<00:36, 1.14it/s]
Loading 0:  90%|█████████ | 328/363 [04:05<00:20, 1.71it/s]
Loading 0:  90%|█████████ | 328/363 [04:05<00:20, 1.71it/s]
Loading 0:  91%|█████████ | 329/363 [04:06<00:24, 1.39it/s]
Loading 0:  91%|█████████ | 329/363 [04:06<00:24, 1.39it/s]
Loading 0:  91%|█████████ | 330/363 [04:09<00:29, 1.11it/s]
Loading 0:  91%|█████████ | 330/363 [04:09<00:29, 1.11it/s]
Loading 0:  93%|█████████▎| 337/363 [04:11<00:15, 1.68it/s]
Loading 0:  93%|█████████▎| 337/363 [04:11<00:15, 1.68it/s]
Loading 0:  93%|█████████▎| 338/363 [04:13<00:18, 1.37it/s]
Loading 0:  93%|█████████▎| 338/363 [04:13<00:18, 1.37it/s]
Loading 0:  93%|█████████▎| 339/363 [04:15<00:21, 1.13it/s]
Loading 0:  93%|█████████▎| 339/363 [04:15<00:21, 1.13it/s]
Loading 0:  95%|█████████▌| 346/363 [04:17<00:10, 1.70it/s]
Loading 0:  95%|█████████▌| 346/363 [04:17<00:10, 1.70it/s]
Loading 0:  96%|█████████▌| 347/363 [04:19<00:11, 1.39it/s]
Loading 0:  96%|█████████▌| 347/363 [04:19<00:11, 1.39it/s]
Loading 0:  96%|█████████▌| 348/363 [04:21<00:13, 1.14it/s]
Loading 0:  96%|█████████▌| 348/363 [04:21<00:13, 1.14it/s]
Loading 0:  98%|█████████▊| 355/363 [04:24<00:04, 1.70it/s]
Loading 0:  98%|█████████▊| 355/363 [04:24<00:04, 1.70it/s]
Loading 0:  98%|█████████▊| 356/363 [04:26<00:05, 1.39it/s]
Loading 0:  98%|█████████▊| 356/363 [04:26<00:05, 1.39it/s]
Loading 0:  98%|█████████▊| 357/363 [04:28<00:05, 1.14it/s]
Loading 0:  98%|█████████▊| 357/363 [04:28<00:05, 1.14it/s]
Loading 0: 100%|██████████| 363/363 [04:28<00:00, 2.08it/s]
Loading 0: 100%|██████████| 363/363 [04:28<00:00, 2.08it/s]
                                                           

Loading 0: 100%|██████████| 363/363 [04:28<00:00, 1.35it/s]
chaiml-7b07-69d4-linear-w01-v25-mkmlizer: The tokenizer you are loading from '/tmp/tmpqhwkwy5l' with an incorrect regex pattern: https://huggingface.co/mistralai/Mistral-Small-3.1-24B-Instruct-2503/discussions/84#69121093e8b480e709447d5e. This will lead to incorrect tokenization. You should set the `fix_mistral_regex=True` flag when loading this tokenizer to fix this issue.
chaiml-7b07-69d4-linear-w01-v25-mkmlizer: quantized model in 275.119s
chaiml-7b07-69d4-linear-w01-v25-mkmlizer: Processed model ChaiML/7b07-69d4-linear-w01 in 325.390s
chaiml-7b07-69d4-linear-w01-v25-mkmlizer: creating bucket guanaco-mkml-models
chaiml-7b07-69d4-linear-w01-v25-mkmlizer: Bucket 's3://guanaco-mkml-models/' created
chaiml-7b07-69d4-linear-w01-v25-mkmlizer: uploading /dev/shm/model_cache to s3://guanaco-mkml-models/chaiml-7b07-69d4-linear-w01-v25/nvidia
chaiml-7b07-69d4-linear-w01-v25-mkmlizer: cp /dev/shm/model_cache/config.json s3://guanaco-mkml-models/chaiml-7b07-69d4-linear-w01-v25/nvidia/config.json
chaiml-02f4-69d4-linear-w01-v42-mkmlizer: cp /dev/shm/model_cache/flywheel_model.1.safetensors s3://guanaco-mkml-models/chaiml-02f4-69d4-linear-w01-v42/nvidia/flywheel_model.1.safetensors
chaiml-7b07-69d4-linear-w01-v25-mkmlizer: cp /dev/shm/model_cache/flywheel_model.1.safetensors s3://guanaco-mkml-models/chaiml-7b07-69d4-linear-w01-v25/nvidia/flywheel_model.1.safetensors
chaiml-02f4-69d4-linear-w01-v42-mkmlizer: cp /dev/shm/model_cache/flywheel_model.0.safetensors s3://guanaco-mkml-models/chaiml-02f4-69d4-linear-w01-v42/nvidia/flywheel_model.0.safetensors
Job chaiml-02f4-69d4-linear-w01-v42-mkmlizer completed after 401.31s with status: succeeded
Stopping job with name chaiml-02f4-69d4-linear-w01-v42-mkmlizer
Pipeline stage MKMLizer completed in 403.34s
run pipeline stage %s
Running pipeline stage MKMLTemplater
Pipeline stage MKMLTemplater completed in 0.64s
run pipeline stage %s
Running pipeline stage MKMLDeployer
Creating inference service chaiml-02f4-69d4-linear-w01-v42
Waiting for inference service chaiml-02f4-69d4-linear-w01-v42 to be ready
chaiml-7b07-69d4-linear-w01-v25-mkmlizer: cp /dev/shm/model_cache/flywheel_model.0.safetensors s3://guanaco-mkml-models/chaiml-7b07-69d4-linear-w01-v25/nvidia/flywheel_model.0.safetensors
Job chaiml-7b07-69d4-linear-w01-v25-mkmlizer completed after 415.27s with status: succeeded
Stopping job with name chaiml-7b07-69d4-linear-w01-v25-mkmlizer
Pipeline stage MKMLizer completed in 417.43s
run pipeline stage %s
Running pipeline stage MKMLTemplater
Pipeline stage MKMLTemplater completed in 0.77s
run pipeline stage %s
Running pipeline stage MKMLDeployer
Creating inference service chaiml-7b07-69d4-linear-w01-v25
Waiting for inference service chaiml-7b07-69d4-linear-w01-v25 to be ready
Inference service chaiml-2fe5-c13f-linear-w01-v30 ready after 172.55593919754028s
Pipeline stage MKMLDeployer completed in 174.65s
run pipeline stage %s
Running pipeline stage StressChecker
HTTP Request: %s %s "%s %d %s"
Received healthy response to inference request in 3.10451602935791s
HTTP Request: %s %s "%s %d %s"
Received healthy response to inference request in 2.827838897705078s
HTTP Request: %s %s "%s %d %s"
Received healthy response to inference request in 2.942458152770996s
HTTP Request: %s %s "%s %d %s"
Received healthy response to inference request in 2.1563029289245605s
HTTP Request: %s %s "%s %d %s"
Received healthy response to inference request in 2.8585612773895264s
5 requests
0 failed requests
5th percentile: 2.290610122680664
10th percentile: 2.4249173164367677
20th percentile: 2.693531703948975
30th percentile: 2.833983373641968
40th percentile: 2.846272325515747
50th percentile: 2.8585612773895264
60th percentile: 2.892120027542114
70th percentile: 2.925678777694702
80th percentile: 2.974869728088379
90th percentile: 3.0396928787231445
95th percentile: 3.0721044540405273
99th percentile: 3.0980337142944334
mean time: 2.7779354572296144
Pipeline stage StressChecker completed in 24.78s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyTriggerPipeline
Pipeline stage OfflineFamilyFriendlyTriggerPipeline completed in 0.59s
run pipeline stage %s
Running pipeline stage TriggerMKMLProfilingPipeline
Pipeline stage TriggerMKMLProfilingPipeline completed in 0.65s
Shutdown handler de-registered
chaiml-2fe5-c13f-linear-w01_v30 status is now deployed due to DeploymentManager action
HTTP Request: %s %s "%s %d %s"
HTTP Request: %s %s "%s %d %s"
chaiml-ca18-c13f-linear-w01-v27-mkmlizer: bash: cannot set terminal process group (-1): Inappropriate ioctl for device
chaiml-ca18-c13f-linear-w01-v27-mkmlizer: bash: no job control in this shell
chaiml-ca18-c13f-linear-w01-v27-mkmlizer: /root/miniconda3/envs/nvidia/lib/python3.11/site-packages/mk1/__init__.py:1: UserWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html. The pkg_resources package is slated for removal as early as 2025-11-30. Refrain from using this package or pin to Setuptools<81.
chaiml-ca18-c13f-linear-w01-v27-mkmlizer:   __import__('pkg_resources').declare_namespace(__name__)
chaiml-ca18-c13f-linear-w01-v27-mkmlizer: ╔═════════════════════════════════════════════════════════════════════╗
chaiml-ca18-c13f-linear-w01-v27-mkmlizer: ║                                                                     ║
chaiml-ca18-c13f-linear-w01-v27-mkmlizer: ║   ██████   ██████  █████   ████  ████                               ║
chaiml-ca18-c13f-linear-w01-v27-mkmlizer: ║  ░░██████ ██████  ░░███   ███░  ░░███                               ║
chaiml-ca18-c13f-linear-w01-v27-mkmlizer: ║   ░███░█████░███   ░███  ███     ░███                               ║
chaiml-ca18-c13f-linear-w01-v27-mkmlizer: ║   ░███░░███ ░███   ░███████      ░███                               ║
chaiml-ca18-c13f-linear-w01-v27-mkmlizer: ║   ░███ ░░░  ░███   ░███░░███     ░███                               ║
chaiml-ca18-c13f-linear-w01-v27-mkmlizer: ║   ░███      ░███   ░███ ░░███    ░███                               ║
chaiml-ca18-c13f-linear-w01-v27-mkmlizer: ║   █████     █████  █████ ░░████  █████                              ║
chaiml-ca18-c13f-linear-w01-v27-mkmlizer: ║  ░░░░░     ░░░░░  ░░░░░   ░░░░  ░░░░░                               ║
chaiml-ca18-c13f-linear-w01-v27-mkmlizer: ║                                                                     ║
chaiml-ca18-c13f-linear-w01-v27-mkmlizer: ║  Version: 0.30.6+torch280                                           ║
chaiml-ca18-c13f-linear-w01-v27-mkmlizer: ║  Features: FLYWHEEL, CUDA                                           ║
chaiml-ca18-c13f-linear-w01-v27-mkmlizer: ║  Copyright 2023-2025 MK ONE TECHNOLOGIES Inc.                       ║
chaiml-ca18-c13f-linear-w01-v27-mkmlizer: ║  https://mk1.ai                                                     ║
chaiml-ca18-c13f-linear-w01-v27-mkmlizer: ║                                                                     ║
chaiml-ca18-c13f-linear-w01-v27-mkmlizer: ║  The license key for the current software has been verified as      ║
chaiml-ca18-c13f-linear-w01-v27-mkmlizer: ║  belonging to:                                                      ║
chaiml-ca18-c13f-linear-w01-v27-mkmlizer: ║                                                                     ║
chaiml-ca18-c13f-linear-w01-v27-mkmlizer: ║  Chai Research Corp.                                                ║
chaiml-ca18-c13f-linear-w01-v27-mkmlizer: ║  Account ID: 7997a29f-0ceb-4cc7-9adf-840c57b4ae6f                   ║
chaiml-ca18-c13f-linear-w01-v27-mkmlizer: ║  Expiration: 2028-03-31 23:59:59                                    ║
chaiml-ca18-c13f-linear-w01-v27-mkmlizer: ║                                                                     ║
chaiml-ca18-c13f-linear-w01-v27-mkmlizer: ╚═════════════════════════════════════════════════════════════════════╝
chaiml-ca18-c13f-linear-w01-v27-mkmlizer: Downloaded to shared memory in 27.170s
chaiml-ca18-c13f-linear-w01-v27-mkmlizer: Checking if ChaiML/ca18-c13f-linear-w01 already exists in ChaiML
chaiml-ca18-c13f-linear-w01-v27-mkmlizer: quantizing model to /dev/shm/model_cache, profile:q4, folder:/tmp/tmpzo6u27wv, device:0
chaiml-ca18-c13f-linear-w01-v27-mkmlizer: Saving flywheel model at /dev/shm/model_cache
Inference service chaiml-02f4-69d4-linear-w01-v42 ready after 162.58064579963684s
Pipeline stage MKMLDeployer completed in 165.37s
run pipeline stage %s
Running pipeline stage StressChecker
HTTP Request: %s %s "%s %d %s"
Received healthy response to inference request in 3.1125481128692627s
HTTP Request: %s %s "%s %d %s"
Received healthy response to inference request in 2.786238431930542s
HTTP Request: %s %s "%s %d %s"
Received healthy response to inference request in 3.2542905807495117s
HTTP Request: %s %s "%s %d %s"
Received healthy response to inference request in 3.186695098876953s
HTTP Request: %s %s "%s %d %s"
Received healthy response to inference request in 2.6164839267730713s
5 requests
0 failed requests
5th percentile: 2.6504348278045655
10th percentile: 2.6843857288360597
20th percentile: 2.7522875308990478
30th percentile: 2.851500368118286
40th percentile: 2.9820242404937742
50th percentile: 3.1125481128692627
60th percentile: 3.142206907272339
70th percentile: 3.171865701675415
80th percentile: 3.2002141952514647
90th percentile: 3.227252388000488
95th percentile: 3.240771484375
99th percentile: 3.2515867614746092
mean time: 2.991251230239868
Pipeline stage StressChecker completed in 23.64s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyTriggerPipeline
Pipeline stage OfflineFamilyFriendlyTriggerPipeline completed in 0.53s
run pipeline stage %s
Running pipeline stage TriggerMKMLProfilingPipeline
Pipeline stage TriggerMKMLProfilingPipeline completed in 0.56s
Shutdown handler de-registered
chaiml-02f4-69d4-linear-w01_v42 status is now deployed due to DeploymentManager action
chaiml-02f4-69d4-linear-w01_v42 status is now inactive due to auto deactivation removed underperforming models
Shutdown handler registered
run pipeline %s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyScorer
Evaluating %s Family Friendly Score with %s threads
%s, retrying in %s seconds...
Evaluating %s Family Friendly Score with %s threads
Pipeline stage OfflineFamilyFriendlyScorer completed in 4096.19s
Shutdown handler de-registered