mistralai-mistral-nem_93303

submission_id: mistralai-mistral-nem_93303_v612
developer_uid: richhx
status: torndown
model_repo: mistralai/Mistral-Nemo-Instruct-2407
generation_params: {'temperature': 1.0, 'top_p': 1.0, 'min_p': 0.0, 'top_k': 40, 'presence_penalty': 0.0, 'frequency_penalty': 0.0, 'stopping_words': ['####\n', '</s>', 'You:', '####', '\n'], 'max_input_tokens': 1024, 'best_of': 8, 'max_output_tokens': 64}
formatter: {'memory_template': '', 'prompt_template': '', 'bot_template': '{bot_name}: {message}</s>\n', 'user_template': 'You: {message}\n', 'response_template': '{bot_name}:', 'truncate_by_message': True}
timestamp: 2026-03-28T17:30:56+00:00
model_name: mistralai-mistral-nem_93303_v612
Resubmit model
Shutdown handler not registered because Python interpreter is not running in the main thread
run pipeline %s
run pipeline stage %s
Running pipeline stage VLLMUploader
Starting job with name mistralai-mistral-nem-93303-v612-uploader
Waiting for job on mistralai-mistral-nem-93303-v612-uploader to finish
mistralai-mistral-nem-93303-v612-uploader: Using quantization_mode: fp8
mistralai-mistral-nem-93303-v612-uploader: Checking if ChaiML/Mistral-Nemo-Instruct-2407-FP8 already exists in ChaiML
mistralai-mistral-nem-93303-v612-uploader: Downloading snapshot of mistralai/Mistral-Nemo-Instruct-2407...
2026-03-28T17:26:31.880164+00:00 monitor updated for mistralai-mistral-nem_93303_v612
mistralai-mistral-nem-93303-v612-uploader: Downloaded in 23.774s
mistralai-mistral-nem-93303-v612-uploader: Loading /tmp/model_input...
mistralai-mistral-nem-93303-v612-uploader: The tokenizer you are loading from '/tmp/model_input' with an incorrect regex pattern: https://huggingface.co/mistralai/Mistral-Small-3.1-24B-Instruct-2503/discussions/84#69121093e8b480e709447d5e. This will lead to incorrect tokenization. You should set the `fix_mistral_regex=True` flag when loading this tokenizer to fix this issue.
mistralai-mistral-nem-93303-v612-uploader: Applying quantization...
mistralai-mistral-nem-93303-v612-uploader: 2026-03-28T17:26:38.024350+0000 | __init__ | WARNING - Disabling tokenizer parallelism due to threading conflict between FastTokenizer and Datasets. Set TOKENIZERS_PARALLELISM=false to suppress this warning.
mistralai-mistral-nem-93303-v612-uploader: The tokenizer you are loading from '/tmp/model_input' with an incorrect regex pattern: https://huggingface.co/mistralai/Mistral-Small-3.1-24B-Instruct-2503/discussions/84#69121093e8b480e709447d5e. This will lead to incorrect tokenization. You should set the `fix_mistral_regex=True` flag when loading this tokenizer to fix this issue.
mistralai-mistral-nem-93303-v612-uploader: 2026-03-28T17:26:38.719693+0000 | reset | INFO - Compression lifecycle reset
mistralai-mistral-nem-93303-v612-uploader: 2026-03-28T17:26:38.722966+0000 | from_modifiers | INFO - Creating recipe from modifiers
mistralai-mistral-nem-93303-v612-uploader: 2026-03-28T17:26:38.751667+0000 | initialize | INFO - Compression lifecycle initialized for 1 modifiers
mistralai-mistral-nem-93303-v612-uploader: 2026-03-28T17:26:38.751913+0000 | IndependentPipeline | INFO - Inferred `DataFreePipeline` for `QuantizationModifier`
mistralai-mistral-nem-93303-v612-uploader: 2026-03-28T17:26:42.767601+0000 | finalize | INFO - Compression lifecycle finalized for 1 modifiers
mistralai-mistral-nem-93303-v612-uploader: 2026-03-28T17:26:42.767767+0000 | post_process | WARNING - Optimized model is not saved. To save, please provide`output_dir` as input arg.Ex. `oneshot(..., output_dir=...)`
mistralai-mistral-nem-93303-v612-uploader: Saving to /dev/shm/model_output...
mistralai-mistral-nem-93303-v612-uploader: /usr/local/lib/python3.12/dist-packages/transformers/modeling_utils.py:3344: UserWarning: Attempting to save a model with offloaded modules. Ensure that unallocated cpu memory exceeds the `shard_size` (50GB default)
mistralai-mistral-nem-93303-v612-uploader:   warnings.warn(
mistralai-mistral-nem-93303-v612-uploader: Updating config in /dev/shm/model_output
mistralai-mistral-nem-93303-v612-uploader: Traceback (most recent call last):
mistralai-mistral-nem-93303-v612-uploader:   File "/code/uploading/compress.py", line 344, in <module>
mistralai-mistral-nem-93303-v612-uploader:     cli()
mistralai-mistral-nem-93303-v612-uploader:   File "/usr/local/lib/python3.12/dist-packages/click/core.py", line 1485, in __call__
mistralai-mistral-nem-93303-v612-uploader:     return self.main(*args, **kwargs)
mistralai-mistral-nem-93303-v612-uploader:            ^^^^^^^^^^^^^^^^^^^^^^^^^^
mistralai-mistral-nem-93303-v612-uploader:   File "/usr/local/lib/python3.12/dist-packages/click/core.py", line 1406, in main
mistralai-mistral-nem-93303-v612-uploader:     rv = self.invoke(ctx)
mistralai-mistral-nem-93303-v612-uploader:          ^^^^^^^^^^^^^^^^
mistralai-mistral-nem-93303-v612-uploader:   File "/usr/local/lib/python3.12/dist-packages/click/core.py", line 1873, in invoke
mistralai-mistral-nem-93303-v612-uploader:     return _process_result(sub_ctx.command.invoke(sub_ctx))
mistralai-mistral-nem-93303-v612-uploader:                            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
mistralai-mistral-nem-93303-v612-uploader:   File "/usr/local/lib/python3.12/dist-packages/click/core.py", line 1269, in invoke
mistralai-mistral-nem-93303-v612-uploader:     return ctx.invoke(self.callback, **ctx.params)
mistralai-mistral-nem-93303-v612-uploader:            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
mistralai-mistral-nem-93303-v612-uploader:   File "/usr/local/lib/python3.12/dist-packages/click/core.py", line 824, in invoke
mistralai-mistral-nem-93303-v612-uploader:     return callback(*args, **kwargs)
mistralai-mistral-nem-93303-v612-uploader:            ^^^^^^^^^^^^^^^^^^^^^^^^^
mistralai-mistral-nem-93303-v612-uploader:   File "/code/uploading/compress.py", line 54, in process
mistralai-mistral-nem-93303-v612-uploader:     quantize_fp8(repo_id, download_path, output_path, revision, hf_auth_token)
mistralai-mistral-nem-93303-v612-uploader:   File "/code/uploading/compress.py", line 223, in quantize_fp8
mistralai-mistral-nem-93303-v612-uploader:     update_fp8_config(model_arch, output_path, ignore_layers)
mistralai-mistral-nem-93303-v612-uploader: TypeError: update_fp8_config() takes 2 positional arguments but 3 were given
Job mistralai-mistral-nem-93303-v612-uploader completed after 98.53s with status: failed
Job failed mistralai-mistral-nem-93303-v612-uploader: 
Stopping job with name mistralai-mistral-nem-93303-v612-uploader
%s, retrying in %s seconds...
Starting job with name mistralai-mistral-nem-93303-v612-uploader
Waiting for job on mistralai-mistral-nem-93303-v612-uploader to finish
mistralai-mistral-nem-93303-v612-uploader: Using quantization_mode: fp8
2026-03-28T17:27:32.087133+00:00 monitor updated for mistralai-mistral-nem_93303_v612
mistralai-mistral-nem-93303-v612-uploader: Checking if ChaiML/Mistral-Nemo-Instruct-2407-FP8 already exists in ChaiML
mistralai-mistral-nem-93303-v612-uploader: Downloading snapshot of mistralai/Mistral-Nemo-Instruct-2407...
mistralai-mistral-nem-93303-v612-uploader: Downloaded in 24.203s
mistralai-mistral-nem-93303-v612-uploader: Loading /tmp/model_input...
mistralai-mistral-nem-93303-v612-uploader: The tokenizer you are loading from '/tmp/model_input' with an incorrect regex pattern: https://huggingface.co/mistralai/Mistral-Small-3.1-24B-Instruct-2503/discussions/84#69121093e8b480e709447d5e. This will lead to incorrect tokenization. You should set the `fix_mistral_regex=True` flag when loading this tokenizer to fix this issue.
mistralai-mistral-nem-93303-v612-uploader: Applying quantization...
mistralai-mistral-nem-93303-v612-uploader: 2026-03-28T17:27:49.618453+0000 | __init__ | WARNING - Disabling tokenizer parallelism due to threading conflict between FastTokenizer and Datasets. Set TOKENIZERS_PARALLELISM=false to suppress this warning.
mistralai-mistral-nem-93303-v612-uploader: The tokenizer you are loading from '/tmp/model_input' with an incorrect regex pattern: https://huggingface.co/mistralai/Mistral-Small-3.1-24B-Instruct-2503/discussions/84#69121093e8b480e709447d5e. This will lead to incorrect tokenization. You should set the `fix_mistral_regex=True` flag when loading this tokenizer to fix this issue.
mistralai-mistral-nem-93303-v612-uploader: 2026-03-28T17:27:50.325584+0000 | reset | INFO - Compression lifecycle reset
mistralai-mistral-nem-93303-v612-uploader: 2026-03-28T17:27:50.329087+0000 | from_modifiers | INFO - Creating recipe from modifiers
mistralai-mistral-nem-93303-v612-uploader: 2026-03-28T17:27:50.357587+0000 | initialize | INFO - Compression lifecycle initialized for 1 modifiers
mistralai-mistral-nem-93303-v612-uploader: 2026-03-28T17:27:50.357836+0000 | IndependentPipeline | INFO - Inferred `DataFreePipeline` for `QuantizationModifier`
mistralai-mistral-nem-93303-v612-uploader: 2026-03-28T17:27:54.522643+0000 | finalize | INFO - Compression lifecycle finalized for 1 modifiers
mistralai-mistral-nem-93303-v612-uploader: 2026-03-28T17:27:54.522804+0000 | post_process | WARNING - Optimized model is not saved. To save, please provide`output_dir` as input arg.Ex. `oneshot(..., output_dir=...)`
mistralai-mistral-nem-93303-v612-uploader: Saving to /dev/shm/model_output...
mistralai-mistral-nem-93303-v612-uploader: /usr/local/lib/python3.12/dist-packages/transformers/modeling_utils.py:3344: UserWarning: Attempting to save a model with offloaded modules. Ensure that unallocated cpu memory exceeds the `shard_size` (50GB default)
mistralai-mistral-nem-93303-v612-uploader:   warnings.warn(
mistralai-mistral-nem-93303-v612-uploader: Updating config in /dev/shm/model_output
mistralai-mistral-nem-93303-v612-uploader: Traceback (most recent call last):
mistralai-mistral-nem-93303-v612-uploader:   File "/code/uploading/compress.py", line 344, in <module>
mistralai-mistral-nem-93303-v612-uploader:     cli()
mistralai-mistral-nem-93303-v612-uploader:   File "/usr/local/lib/python3.12/dist-packages/click/core.py", line 1485, in __call__
mistralai-mistral-nem-93303-v612-uploader:     return self.main(*args, **kwargs)
mistralai-mistral-nem-93303-v612-uploader:            ^^^^^^^^^^^^^^^^^^^^^^^^^^
mistralai-mistral-nem-93303-v612-uploader:   File "/usr/local/lib/python3.12/dist-packages/click/core.py", line 1406, in main
mistralai-mistral-nem-93303-v612-uploader:     rv = self.invoke(ctx)
mistralai-mistral-nem-93303-v612-uploader:          ^^^^^^^^^^^^^^^^
mistralai-mistral-nem-93303-v612-uploader:   File "/usr/local/lib/python3.12/dist-packages/click/core.py", line 1873, in invoke
mistralai-mistral-nem-93303-v612-uploader:     return _process_result(sub_ctx.command.invoke(sub_ctx))
mistralai-mistral-nem-93303-v612-uploader:                            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
mistralai-mistral-nem-93303-v612-uploader:   File "/usr/local/lib/python3.12/dist-packages/click/core.py", line 1269, in invoke
mistralai-mistral-nem-93303-v612-uploader:     return ctx.invoke(self.callback, **ctx.params)
mistralai-mistral-nem-93303-v612-uploader:            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
mistralai-mistral-nem-93303-v612-uploader:   File "/usr/local/lib/python3.12/dist-packages/click/core.py", line 824, in invoke
mistralai-mistral-nem-93303-v612-uploader:     return callback(*args, **kwargs)
mistralai-mistral-nem-93303-v612-uploader:            ^^^^^^^^^^^^^^^^^^^^^^^^^
mistralai-mistral-nem-93303-v612-uploader:   File "/code/uploading/compress.py", line 54, in process
mistralai-mistral-nem-93303-v612-uploader:     quantize_fp8(repo_id, download_path, output_path, revision, hf_auth_token)
mistralai-mistral-nem-93303-v612-uploader:   File "/code/uploading/compress.py", line 223, in quantize_fp8
mistralai-mistral-nem-93303-v612-uploader:     update_fp8_config(model_arch, output_path, ignore_layers)
mistralai-mistral-nem-93303-v612-uploader: TypeError: update_fp8_config() takes 2 positional arguments but 3 were given
Job mistralai-mistral-nem-93303-v612-uploader completed after 66.97s with status: failed
Job failed mistralai-mistral-nem-93303-v612-uploader: 
Stopping job with name mistralai-mistral-nem-93303-v612-uploader
%s, retrying in %s seconds...
Starting job with name mistralai-mistral-nem-93303-v612-uploader
Waiting for job on mistralai-mistral-nem-93303-v612-uploader to finish
2026-03-28T17:28:32.365532+00:00 monitor updated for mistralai-mistral-nem_93303_v612
mistralai-mistral-nem-93303-v612-uploader: Using quantization_mode: fp8
mistralai-mistral-nem-93303-v612-uploader: Checking if ChaiML/Mistral-Nemo-Instruct-2407-FP8 already exists in ChaiML
mistralai-mistral-nem-93303-v612-uploader: Downloading snapshot of mistralai/Mistral-Nemo-Instruct-2407...
mistralai-mistral-nem-93303-v612-uploader: Downloaded in 21.342s
mistralai-mistral-nem-93303-v612-uploader: Loading /tmp/model_input...
mistralai-mistral-nem-93303-v612-uploader: The tokenizer you are loading from '/tmp/model_input' with an incorrect regex pattern: https://huggingface.co/mistralai/Mistral-Small-3.1-24B-Instruct-2503/discussions/84#69121093e8b480e709447d5e. This will lead to incorrect tokenization. You should set the `fix_mistral_regex=True` flag when loading this tokenizer to fix this issue.
mistralai-mistral-nem-93303-v612-uploader: Applying quantization...
mistralai-mistral-nem-93303-v612-uploader: 2026-03-28T17:29:23.517378+0000 | __init__ | WARNING - Disabling tokenizer parallelism due to threading conflict between FastTokenizer and Datasets. Set TOKENIZERS_PARALLELISM=false to suppress this warning.
2026-03-28T17:29:32.582691+00:00 monitor updated for mistralai-mistral-nem_93303_v612
mistralai-mistral-nem-93303-v612-uploader: 2026-03-28T17:29:28.480237+0000 | finalize | INFO - Compression lifecycle finalized for 1 modifiers
mistralai-mistral-nem-93303-v612-uploader: 2026-03-28T17:29:28.480415+0000 | post_process | WARNING - Optimized model is not saved. To save, please provide`output_dir` as input arg.Ex. `oneshot(..., output_dir=...)`
mistralai-mistral-nem-93303-v612-uploader: Saving to /dev/shm/model_output...
mistralai-mistral-nem-93303-v612-uploader: /usr/local/lib/python3.12/dist-packages/transformers/modeling_utils.py:3344: UserWarning: Attempting to save a model with offloaded modules. Ensure that unallocated cpu memory exceeds the `shard_size` (50GB default)
mistralai-mistral-nem-93303-v612-uploader:   warnings.warn(
mistralai-mistral-nem-93303-v612-uploader: Updating config in /dev/shm/model_output
mistralai-mistral-nem-93303-v612-uploader: Traceback (most recent call last):
mistralai-mistral-nem-93303-v612-uploader:   File "/code/uploading/compress.py", line 344, in <module>
mistralai-mistral-nem-93303-v612-uploader:     cli()
mistralai-mistral-nem-93303-v612-uploader:   File "/usr/local/lib/python3.12/dist-packages/click/core.py", line 1485, in __call__
mistralai-mistral-nem-93303-v612-uploader:     return self.main(*args, **kwargs)
mistralai-mistral-nem-93303-v612-uploader:            ^^^^^^^^^^^^^^^^^^^^^^^^^^
mistralai-mistral-nem-93303-v612-uploader:   File "/usr/local/lib/python3.12/dist-packages/click/core.py", line 1406, in main
mistralai-mistral-nem-93303-v612-uploader:     rv = self.invoke(ctx)
mistralai-mistral-nem-93303-v612-uploader:          ^^^^^^^^^^^^^^^^
mistralai-mistral-nem-93303-v612-uploader:   File "/usr/local/lib/python3.12/dist-packages/click/core.py", line 1873, in invoke
mistralai-mistral-nem-93303-v612-uploader:     return _process_result(sub_ctx.command.invoke(sub_ctx))
mistralai-mistral-nem-93303-v612-uploader:                            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
mistralai-mistral-nem-93303-v612-uploader:   File "/usr/local/lib/python3.12/dist-packages/click/core.py", line 1269, in invoke
mistralai-mistral-nem-93303-v612-uploader:     return ctx.invoke(self.callback, **ctx.params)
mistralai-mistral-nem-93303-v612-uploader:            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
mistralai-mistral-nem-93303-v612-uploader:   File "/usr/local/lib/python3.12/dist-packages/click/core.py", line 824, in invoke
mistralai-mistral-nem-93303-v612-uploader:     return callback(*args, **kwargs)
mistralai-mistral-nem-93303-v612-uploader:            ^^^^^^^^^^^^^^^^^^^^^^^^^
mistralai-mistral-nem-93303-v612-uploader:   File "/code/uploading/compress.py", line 54, in process
mistralai-mistral-nem-93303-v612-uploader:     quantize_fp8(repo_id, download_path, output_path, revision, hf_auth_token)
mistralai-mistral-nem-93303-v612-uploader:   File "/code/uploading/compress.py", line 223, in quantize_fp8
mistralai-mistral-nem-93303-v612-uploader:     update_fp8_config(model_arch, output_path, ignore_layers)
mistralai-mistral-nem-93303-v612-uploader: TypeError: update_fp8_config() takes 2 positional arguments but 3 were given
Job mistralai-mistral-nem-93303-v612-uploader completed after 98.19s with status: failed
Job failed mistralai-mistral-nem-93303-v612-uploader: 
Stopping job with name mistralai-mistral-nem-93303-v612-uploader
clean up pipeline due to error=VLLMUploaderError('')
run pipeline stage %s
Running pipeline stage VLLMDeleter
Skipping teardown as no inference service was successfully deployed
Pipeline stage VLLMDeleter completed in 0.44s
run pipeline stage %s
Running pipeline stage VLLMModelDeleter
Cleaning model data from S3
Pipeline stage VLLMModelDeleter completed in 0.43s
Shutdown handler de-registered
mistralai-mistral-nem_93303_v612 status is now failed due to DeploymentManager action
mistralai-mistral-nem_93303_v612 status is now torndown due to DeploymentManager action