chaiml-mistral-24b-2048_74171

developer_uid: rirv938
submission_id: chaiml-mistral-24b-2048_74171_v1
model_name: chaiml-mistral-24b-2048_74171_v1
model_group: ChaiML/mistral_24b_2048_
status: inactive
timestamp: 2026-02-25T08:37:21+00:00
num_battles: 10127
num_wins: 5142
celo_rating: 1302.98
family_friendly_score: 0.0
family_friendly_standard_error: 0.0
submission_type: basic
model_repo: ChaiML/mistral_24b_2048_110_max_top_10k_comb_rm_309_merged
model_architecture: MistralForCausalLM
model_num_parameters: 24096691200.0
best_of: 8
max_input_tokens: 2048
max_output_tokens: 80
reward_model: default
display_name: chaiml-mistral-24b-2048_74171_v1
ineligible_reason: max_output_tokens!=64
is_internal_developer: True
language_model: ChaiML/mistral_24b_2048_110_max_top_10k_comb_rm_309_merged
model_size: 24B
ranking_group: single
us_pacific_date: 2026-02-25
win_ratio: 0.507751555248346
generation_params: {'temperature': 1.0, 'top_p': 0.95, 'min_p': 0.05, 'top_k': 60, 'presence_penalty': 0.1, 'frequency_penalty': 0.0, 'stopping_words': ['You:', '<|im_end|>', '</s>', '###', '<|im_start|>'], 'max_input_tokens': 2048, 'best_of': 8, 'max_output_tokens': 80}
formatter: {'memory_template': '[SYSTEM_PROMPT]Respond as a high quality storyteller.[/SYSTEM_PROMPT][INST]', 'prompt_template': '', 'bot_template': '{bot_name}: {message}\n', 'user_template': '{user_name}: {message}\n', 'response_template': '[/INST]{bot_name}:', 'truncate_by_message': False}
Resubmit model
Shutdown handler not registered because Python interpreter is not running in the main thread
run pipeline %s
run pipeline stage %s
Running pipeline stage VLLMUploader
Starting job with name chaiml-mistral-24b-2048-74171-v1-uploader
Waiting for job on chaiml-mistral-24b-2048-74171-v1-uploader to finish
Shutdown handler not registered because Python interpreter is not running in the main thread
run pipeline %s
run pipeline stage %s
Running pipeline stage VLLMUploader
Starting job with name chaiml-mistral-24b-2048-48145-v1-uploader
Waiting for job on chaiml-mistral-24b-2048-48145-v1-uploader to finish
chaiml-mistral-24b-2048-54828-v1-uploader: Using quantization_mode: fp8
chaiml-mistral-24b-2048-54828-v1-uploader: Checking if ChaiML/mistral_24b_2048_110_max_top_10k_comb_rm_1236_merged-FP8 already exists in ChaiML
chaiml-mistral-24b-2048-54828-v1-uploader: Downloading snapshot of ChaiML/mistral_24b_2048_110_max_top_10k_comb_rm_1236_merged...
chaiml-mistral-24b-2048-74171-v1-uploader: Using quantization_mode: fp8
chaiml-mistral-24b-2048-74171-v1-uploader: Checking if ChaiML/mistral_24b_2048_110_max_top_10k_comb_rm_309_merged-FP8 already exists in ChaiML
chaiml-mistral-24b-2048-74171-v1-uploader: Downloading snapshot of ChaiML/mistral_24b_2048_110_max_top_10k_comb_rm_309_merged...
chaiml-mistral-24b-2048-48145-v1-uploader: Using quantization_mode: fp8
chaiml-mistral-24b-2048-48145-v1-uploader: Checking if ChaiML/mistral_24b_2048_110_max_top_10k_comb_rm_618_merged-FP8 already exists in ChaiML
chaiml-mistral-24b-2048-48145-v1-uploader: Downloading snapshot of ChaiML/mistral_24b_2048_110_max_top_10k_comb_rm_618_merged...
chaiml-mistral-24b-2048-74171-v1-uploader: Downloaded in 42.149s
chaiml-mistral-24b-2048-74171-v1-uploader: Loading /tmp/model_input...
chaiml-mistral-24b-2048-74171-v1-uploader: The tokenizer you are loading from '/tmp/model_input' with an incorrect regex pattern: https://huggingface.co/mistralai/Mistral-Small-3.1-24B-Instruct-2503/discussions/84#69121093e8b480e709447d5e. This will lead to incorrect tokenization. You should set the `fix_mistral_regex=True` flag when loading this tokenizer to fix this issue.
chaiml-mistral-24b-2048-74171-v1-uploader: `torch_dtype` is deprecated! Use `dtype` instead!
chaiml-mistral-24b-2048-54828-v1-uploader: Downloaded in 45.250s
chaiml-mistral-24b-2048-54828-v1-uploader: Loading /tmp/model_input...
chaiml-mistral-24b-2048-54828-v1-uploader: The tokenizer you are loading from '/tmp/model_input' with an incorrect regex pattern: https://huggingface.co/mistralai/Mistral-Small-3.1-24B-Instruct-2503/discussions/84#69121093e8b480e709447d5e. This will lead to incorrect tokenization. You should set the `fix_mistral_regex=True` flag when loading this tokenizer to fix this issue.
chaiml-mistral-24b-2048-54828-v1-uploader: `torch_dtype` is deprecated! Use `dtype` instead!
chaiml-mistral-24b-2048-74171-v1-uploader: Some parameters are on the meta device because they were offloaded to the cpu.
chaiml-mistral-24b-2048-74171-v1-uploader: Applying quantization...
chaiml-mistral-24b-2048-74171-v1-uploader: The tokenizer you are loading from '/tmp/model_input' with an incorrect regex pattern: https://huggingface.co/mistralai/Mistral-Small-3.1-24B-Instruct-2503/discussions/84#69121093e8b480e709447d5e. This will lead to incorrect tokenization. You should set the `fix_mistral_regex=True` flag when loading this tokenizer to fix this issue.
chaiml-mistral-24b-2048-74171-v1-uploader: 2026-02-24T19:41:16.134810-0800 | reset | INFO - Compression lifecycle reset
chaiml-mistral-24b-2048-74171-v1-uploader: 2026-02-24T19:41:16.135780-0800 | from_modifiers | INFO - Creating recipe from modifiers
chaiml-mistral-24b-2048-74171-v1-uploader: 2026-02-24T19:41:16.226355-0800 | initialize | INFO - Compression lifecycle initialized for 1 modifiers
chaiml-mistral-24b-2048-74171-v1-uploader: 2026-02-24T19:41:16.226619-0800 | IndependentPipeline | INFO - Inferred `DataFreePipeline` for `QuantizationModifier`
chaiml-mistral-24b-2048-48145-v1-uploader: Downloaded in 44.523s
chaiml-mistral-24b-2048-48145-v1-uploader: Loading /tmp/model_input...
chaiml-mistral-24b-2048-48145-v1-uploader: The tokenizer you are loading from '/tmp/model_input' with an incorrect regex pattern: https://huggingface.co/mistralai/Mistral-Small-3.1-24B-Instruct-2503/discussions/84#69121093e8b480e709447d5e. This will lead to incorrect tokenization. You should set the `fix_mistral_regex=True` flag when loading this tokenizer to fix this issue.
chaiml-mistral-24b-2048-48145-v1-uploader: `torch_dtype` is deprecated! Use `dtype` instead!
chaiml-mistral-24b-2048-54828-v1-uploader: Some parameters are on the meta device because they were offloaded to the cpu.
chaiml-mistral-24b-2048-54828-v1-uploader: Applying quantization...
chaiml-mistral-24b-2048-54828-v1-uploader: The tokenizer you are loading from '/tmp/model_input' with an incorrect regex pattern: https://huggingface.co/mistralai/Mistral-Small-3.1-24B-Instruct-2503/discussions/84#69121093e8b480e709447d5e. This will lead to incorrect tokenization. You should set the `fix_mistral_regex=True` flag when loading this tokenizer to fix this issue.
chaiml-mistral-24b-2048-54828-v1-uploader: 2026-02-24T19:41:17.122334-0800 | reset | INFO - Compression lifecycle reset
chaiml-mistral-24b-2048-54828-v1-uploader: 2026-02-24T19:41:17.123295-0800 | from_modifiers | INFO - Creating recipe from modifiers
chaiml-mistral-24b-2048-54828-v1-uploader: 2026-02-24T19:41:17.218876-0800 | initialize | INFO - Compression lifecycle initialized for 1 modifiers
chaiml-mistral-24b-2048-54828-v1-uploader: 2026-02-24T19:41:17.219160-0800 | IndependentPipeline | INFO - Inferred `DataFreePipeline` for `QuantizationModifier`
chaiml-mistral-24b-2048-74171-v1-uploader: Some parameters are on the meta device because they were offloaded to the cpu.
chaiml-mistral-24b-2048-54828-v1-uploader: Some parameters are on the meta device because they were offloaded to the cpu.
chaiml-mistral-24b-2048-48145-v1-uploader: Some parameters are on the meta device because they were offloaded to the cpu.
chaiml-mistral-24b-2048-54828-v1-uploader: 2026-02-24T19:42:09.667079-0800 | finalize | INFO - Compression lifecycle finalized for 1 modifiers
chaiml-mistral-24b-2048-74171-v1-uploader: 2026-02-24T19:42:08.993198-0800 | finalize | INFO - Compression lifecycle finalized for 1 modifiers
chaiml-mistral-24b-2048-74171-v1-uploader: 2026-02-24T19:42:12.230703-0800 | post_process | WARNING - Optimized model is not saved. To save, please provide`output_dir` as input arg.Ex. `oneshot(..., output_dir=...)`
chaiml-mistral-24b-2048-74171-v1-uploader: Saving to /dev/shm/model_output...
chaiml-mistral-24b-2048-74171-v1-uploader: 2026-02-24T19:42:12.259193-0800 | get_model_compressor | INFO - skip_sparsity_compression_stats set to True. Skipping sparsity compression statistic calculations. No sparsity compressor will be applied.
chaiml-mistral-24b-2048-48145-v1-uploader: 2026-02-24T19:42:18.017785-0800 | finalize | INFO - Compression lifecycle finalized for 1 modifiers
chaiml-mistral-24b-2048-54828-v1-uploader: 2026-02-24T19:42:12.961206-0800 | post_process | WARNING - Optimized model is not saved. To save, please provide`output_dir` as input arg.Ex. `oneshot(..., output_dir=...)`
chaiml-mistral-24b-2048-54828-v1-uploader: Saving to /dev/shm/model_output...
chaiml-mistral-24b-2048-54828-v1-uploader: 2026-02-24T19:42:12.988365-0800 | get_model_compressor | INFO - skip_sparsity_compression_stats set to True. Skipping sparsity compression statistic calculations. No sparsity compressor will be applied.
chaiml-mistral-24b-2048-48145-v1-uploader: 2026-02-24T19:42:21.161360-0800 | post_process | WARNING - Optimized model is not saved. To save, please provide`output_dir` as input arg.Ex. `oneshot(..., output_dir=...)`
chaiml-mistral-24b-2048-48145-v1-uploader: Saving to /dev/shm/model_output...
chaiml-mistral-24b-2048-48145-v1-uploader: 2026-02-24T19:42:21.189364-0800 | get_model_compressor | INFO - skip_sparsity_compression_stats set to True. Skipping sparsity compression statistic calculations. No sparsity compressor will be applied.
chaiml-mistral-24b-2048-74171-v1-uploader: Cleaning quantization config in /dev/shm/model_output
chaiml-mistral-24b-2048-74171-v1-uploader: Pushing to ChaiML/mistral_24b_2048_110_max_top_10k_comb_rm_309_merged-FP8
chaiml-mistral-24b-2048-74171-v1-uploader: Checking if ChaiML/mistral_24b_2048_110_max_top_10k_comb_rm_309_merged-FP8 already exists in ChaiML
chaiml-mistral-24b-2048-74171-v1-uploader: Creating repo ChaiML/mistral_24b_2048_110_max_top_10k_comb_rm_309_merged-FP8 and uploading /dev/shm/model_output to it
chaiml-mistral-24b-2048-74171-v1-uploader: ---------- 2026-02-24 19:43:18 (0:00:00) ----------
chaiml-mistral-24b-2048-74171-v1-uploader: Files:   hashed 8/14 (17.4M/27.6G) | pre-uploaded: 0/0 (0.0/27.6G) (+14 unsure) | committed: 0/14 (0.0/27.6G) | ignored: 0
chaiml-mistral-24b-2048-74171-v1-uploader: Workers: hashing: 7 | get upload mode: 6 | pre-uploading: 0 | committing: 0 | waiting: 113
chaiml-mistral-24b-2048-74171-v1-uploader: ---------------------------------------------------
chaiml-mistral-24b-2048-48145-v1-uploader: Cleaning quantization config in /dev/shm/model_output
chaiml-mistral-24b-2048-48145-v1-uploader: Pushing to ChaiML/mistral_24b_2048_110_max_top_10k_comb_rm_618_merged-FP8
chaiml-mistral-24b-2048-48145-v1-uploader: Checking if ChaiML/mistral_24b_2048_110_max_top_10k_comb_rm_618_merged-FP8 already exists in ChaiML
chaiml-mistral-24b-2048-48145-v1-uploader: Creating repo ChaiML/mistral_24b_2048_110_max_top_10k_comb_rm_618_merged-FP8 and uploading /dev/shm/model_output to it
chaiml-mistral-24b-2048-48145-v1-uploader: ---------- 2026-02-24 19:43:26 (0:00:00) ----------
chaiml-mistral-24b-2048-54828-v1-uploader: Cleaning quantization config in /dev/shm/model_output
chaiml-mistral-24b-2048-48145-v1-uploader: Files:   hashed 7/14 (277.6K/27.6G) | pre-uploaded: 0/0 (0.0/27.6G) (+14 unsure) | committed: 0/14 (0.0/27.6G) | ignored: 0
chaiml-mistral-24b-2048-54828-v1-uploader: Pushing to ChaiML/mistral_24b_2048_110_max_top_10k_comb_rm_1236_merged-FP8
chaiml-mistral-24b-2048-48145-v1-uploader: Workers: hashing: 7 | get upload mode: 6 | pre-uploading: 0 | committing: 0 | waiting: 113
chaiml-mistral-24b-2048-54828-v1-uploader: Checking if ChaiML/mistral_24b_2048_110_max_top_10k_comb_rm_1236_merged-FP8 already exists in ChaiML
chaiml-mistral-24b-2048-48145-v1-uploader: ---------------------------------------------------
chaiml-mistral-24b-2048-54828-v1-uploader: Creating repo ChaiML/mistral_24b_2048_110_max_top_10k_comb_rm_1236_merged-FP8 and uploading /dev/shm/model_output to it
chaiml-mistral-24b-2048-54828-v1-uploader: ---------- 2026-02-24 19:43:25 (0:00:00) ----------
chaiml-mistral-24b-2048-54828-v1-uploader: Files:   hashed 7/14 (277.6K/27.6G) | pre-uploaded: 0/0 (0.0/27.6G) (+14 unsure) | committed: 0/14 (0.0/27.6G) | ignored: 0
chaiml-mistral-24b-2048-54828-v1-uploader: Workers: hashing: 7 | get upload mode: 5 | pre-uploading: 0 | committing: 0 | waiting: 114
chaiml-mistral-24b-2048-54828-v1-uploader: ---------------------------------------------------
chaiml-mistral-24b-2048-74171-v1-uploader: 
[K[F
[K[F
[K[F
[K[F
[K[F
[K[F
[K[F
chaiml-mistral-24b-2048-74171-v1-uploader: ---------- 2026-02-24 19:44:18 (0:01:00) ----------
chaiml-mistral-24b-2048-74171-v1-uploader: Files:   hashed 14/14 (27.6G/27.6G) | pre-uploaded: 7/7 (27.6G/27.6G) | committed: 0/14 (0.0/27.6G) | ignored: 0
chaiml-mistral-24b-2048-74171-v1-uploader: Workers: hashing: 0 | get upload mode: 0 | pre-uploading: 0 | committing: 1 | waiting: 125
chaiml-mistral-24b-2048-74171-v1-uploader: ---------------------------------------------------
chaiml-mistral-24b-2048-48145-v1-uploader: 
[K[F
[K[F
[K[F
[K[F
[K[F
[K[F
[K[F
chaiml-mistral-24b-2048-48145-v1-uploader: ---------- 2026-02-24 19:44:26 (0:01:00) ----------
chaiml-mistral-24b-2048-48145-v1-uploader: Files:   hashed 14/14 (27.6G/27.6G) | pre-uploaded: 7/7 (27.6G/27.6G) | committed: 0/14 (0.0/27.6G) | ignored: 0
chaiml-mistral-24b-2048-48145-v1-uploader: Workers: hashing: 0 | get upload mode: 0 | pre-uploading: 0 | committing: 1 | waiting: 125
chaiml-mistral-24b-2048-48145-v1-uploader: ---------------------------------------------------
chaiml-mistral-24b-2048-54828-v1-uploader: 
[K[F
[K[F
[K[F
[K[F
[K[F
[K[F
[K[F
chaiml-mistral-24b-2048-54828-v1-uploader: ---------- 2026-02-24 19:44:25 (0:01:00) ----------
chaiml-mistral-24b-2048-54828-v1-uploader: Files:   hashed 14/14 (27.6G/27.6G) | pre-uploaded: 7/7 (27.6G/27.6G) | committed: 0/14 (0.0/27.6G) | ignored: 0
chaiml-mistral-24b-2048-54828-v1-uploader: Workers: hashing: 0 | get upload mode: 0 | pre-uploading: 0 | committing: 1 | waiting: 125
chaiml-mistral-24b-2048-54828-v1-uploader: ---------------------------------------------------
chaiml-mistral-24b-2048-74171-v1-uploader:                              Processed model ChaiML/mistral_24b_2048_110_max_top_10k_comb_rm_309_merged in 248.494s
chaiml-mistral-24b-2048-74171-v1-uploader: creating bucket guanaco-vllm-models
chaiml-mistral-24b-2048-74171-v1-uploader: /usr/lib/python3/dist-packages/S3/BaseUtils.py:56: SyntaxWarning: invalid escape sequence '\.'
chaiml-mistral-24b-2048-74171-v1-uploader:   RE_S3_DATESTRING = re.compile('\.[0-9]*(?:[Z\\-\\+]*?)')
chaiml-mistral-24b-2048-74171-v1-uploader: /usr/lib/python3/dist-packages/S3/BaseUtils.py:57: SyntaxWarning: invalid escape sequence '\s'
chaiml-mistral-24b-2048-74171-v1-uploader:   RE_XML_NAMESPACE = re.compile(b'^(<?[^>]+?>\s*|\s*)(<\w+) xmlns=[\'"](https?://[^\'"]+)[\'"]', re.MULTILINE)
chaiml-mistral-24b-2048-74171-v1-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:240: SyntaxWarning: invalid escape sequence '\.'
chaiml-mistral-24b-2048-74171-v1-uploader:   invalid = re.search("([^a-z0-9\.-])", bucket, re.UNICODE)
chaiml-mistral-24b-2048-74171-v1-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:244: SyntaxWarning: invalid escape sequence '\.'
chaiml-mistral-24b-2048-48145-v1-uploader:                              Processed model ChaiML/mistral_24b_2048_110_max_top_10k_comb_rm_618_merged in 251.568s
chaiml-mistral-24b-2048-74171-v1-uploader:   invalid = re.search("([^A-Za-z0-9\._-])", bucket, re.UNICODE)
chaiml-mistral-24b-2048-74171-v1-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:255: SyntaxWarning: invalid escape sequence '\.'
chaiml-mistral-24b-2048-54828-v1-uploader:                              Processed model ChaiML/mistral_24b_2048_110_max_top_10k_comb_rm_1236_merged in 258.571s
chaiml-mistral-24b-2048-74171-v1-uploader:   if re.search("-\.", bucket, re.UNICODE):
chaiml-mistral-24b-2048-54828-v1-uploader: creating bucket guanaco-vllm-models
chaiml-mistral-24b-2048-74171-v1-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:257: SyntaxWarning: invalid escape sequence '\.'
chaiml-mistral-24b-2048-54828-v1-uploader: /usr/lib/python3/dist-packages/S3/BaseUtils.py:56: SyntaxWarning: invalid escape sequence '\.'
chaiml-mistral-24b-2048-74171-v1-uploader:   if re.search("\.\.", bucket, re.UNICODE):
chaiml-mistral-24b-2048-54828-v1-uploader:   RE_S3_DATESTRING = re.compile('\.[0-9]*(?:[Z\\-\\+]*?)')
chaiml-mistral-24b-2048-74171-v1-uploader: /usr/lib/python3/dist-packages/S3/S3Uri.py:155: SyntaxWarning: invalid escape sequence '\w'
chaiml-mistral-24b-2048-54828-v1-uploader: /usr/lib/python3/dist-packages/S3/BaseUtils.py:57: SyntaxWarning: invalid escape sequence '\s'
chaiml-mistral-24b-2048-74171-v1-uploader:   _re = re.compile("^(\w+://)?(.*)", re.UNICODE)
chaiml-mistral-24b-2048-54828-v1-uploader:   RE_XML_NAMESPACE = re.compile(b'^(<?[^>]+?>\s*|\s*)(<\w+) xmlns=[\'"](https?://[^\'"]+)[\'"]', re.MULTILINE)
chaiml-mistral-24b-2048-74171-v1-uploader: /usr/lib/python3/dist-packages/S3/FileLists.py:480: SyntaxWarning: invalid escape sequence '\*'
chaiml-mistral-24b-2048-54828-v1-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:240: SyntaxWarning: invalid escape sequence '\.'
chaiml-mistral-24b-2048-74171-v1-uploader:   wildcard_split_result = re.split("\*|\?", uri_str, maxsplit=1)
chaiml-mistral-24b-2048-54828-v1-uploader:   invalid = re.search("([^a-z0-9\.-])", bucket, re.UNICODE)
chaiml-mistral-24b-2048-74171-v1-uploader: Bucket 's3://guanaco-vllm-models/' created
chaiml-mistral-24b-2048-54828-v1-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:244: SyntaxWarning: invalid escape sequence '\.'
chaiml-mistral-24b-2048-74171-v1-uploader: uploading /dev/shm/model_output to s3://guanaco-vllm-models/chaiml-mistral-24b-2048-74171-v1/default
chaiml-mistral-24b-2048-54828-v1-uploader:   invalid = re.search("([^A-Za-z0-9\._-])", bucket, re.UNICODE)
chaiml-mistral-24b-2048-74171-v1-uploader: cp /dev/shm/model_output/generation_config.json s3://guanaco-vllm-models/chaiml-mistral-24b-2048-74171-v1/default/generation_config.json
chaiml-mistral-24b-2048-54828-v1-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:255: SyntaxWarning: invalid escape sequence '\.'
chaiml-mistral-24b-2048-74171-v1-uploader: cp /dev/shm/model_output/chat_template.jinja s3://guanaco-vllm-models/chaiml-mistral-24b-2048-74171-v1/default/chat_template.jinja
chaiml-mistral-24b-2048-54828-v1-uploader:   if re.search("-\.", bucket, re.UNICODE):
chaiml-mistral-24b-2048-74171-v1-uploader: cp /dev/shm/model_output/config.json s3://guanaco-vllm-models/chaiml-mistral-24b-2048-74171-v1/default/config.json
chaiml-mistral-24b-2048-54828-v1-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:257: SyntaxWarning: invalid escape sequence '\.'
chaiml-mistral-24b-2048-74171-v1-uploader: cp /dev/shm/model_output/special_tokens_map.json s3://guanaco-vllm-models/chaiml-mistral-24b-2048-74171-v1/default/special_tokens_map.json
chaiml-mistral-24b-2048-54828-v1-uploader:   if re.search("\.\.", bucket, re.UNICODE):
chaiml-mistral-24b-2048-74171-v1-uploader: cp /dev/shm/model_output/recipe.yaml s3://guanaco-vllm-models/chaiml-mistral-24b-2048-74171-v1/default/recipe.yaml
chaiml-mistral-24b-2048-54828-v1-uploader: /usr/lib/python3/dist-packages/S3/S3Uri.py:155: SyntaxWarning: invalid escape sequence '\w'
chaiml-mistral-24b-2048-74171-v1-uploader: cp /dev/shm/model_output/model.safetensors.index.json s3://guanaco-vllm-models/chaiml-mistral-24b-2048-74171-v1/default/model.safetensors.index.json
chaiml-mistral-24b-2048-54828-v1-uploader:   _re = re.compile("^(\w+://)?(.*)", re.UNICODE)
chaiml-mistral-24b-2048-74171-v1-uploader: cp /dev/shm/model_output/tokenizer_config.json s3://guanaco-vllm-models/chaiml-mistral-24b-2048-74171-v1/default/tokenizer_config.json
chaiml-mistral-24b-2048-54828-v1-uploader: /usr/lib/python3/dist-packages/S3/FileLists.py:480: SyntaxWarning: invalid escape sequence '\*'
chaiml-mistral-24b-2048-74171-v1-uploader: cp /dev/shm/model_output/tokenizer.json s3://guanaco-vllm-models/chaiml-mistral-24b-2048-74171-v1/default/tokenizer.json
chaiml-mistral-24b-2048-54828-v1-uploader:   wildcard_split_result = re.split("\*|\?", uri_str, maxsplit=1)
chaiml-mistral-24b-2048-54828-v1-uploader: Bucket 's3://guanaco-vllm-models/' created
chaiml-mistral-24b-2048-54828-v1-uploader: uploading /dev/shm/model_output to s3://guanaco-vllm-models/chaiml-mistral-24b-2048-54828-v1/default
chaiml-mistral-24b-2048-54828-v1-uploader: cp /dev/shm/model_output/chat_template.jinja s3://guanaco-vllm-models/chaiml-mistral-24b-2048-54828-v1/default/chat_template.jinja
chaiml-mistral-24b-2048-48145-v1-uploader: creating bucket guanaco-vllm-models
chaiml-mistral-24b-2048-54828-v1-uploader: cp /dev/shm/model_output/generation_config.json s3://guanaco-vllm-models/chaiml-mistral-24b-2048-54828-v1/default/generation_config.json
chaiml-mistral-24b-2048-48145-v1-uploader: /usr/lib/python3/dist-packages/S3/BaseUtils.py:56: SyntaxWarning: invalid escape sequence '\.'
chaiml-mistral-24b-2048-54828-v1-uploader: cp /dev/shm/model_output/model.safetensors.index.json s3://guanaco-vllm-models/chaiml-mistral-24b-2048-54828-v1/default/model.safetensors.index.json
chaiml-mistral-24b-2048-48145-v1-uploader:   RE_S3_DATESTRING = re.compile('\.[0-9]*(?:[Z\\-\\+]*?)')
chaiml-mistral-24b-2048-54828-v1-uploader: cp /dev/shm/model_output/config.json s3://guanaco-vllm-models/chaiml-mistral-24b-2048-54828-v1/default/config.json
chaiml-mistral-24b-2048-48145-v1-uploader: /usr/lib/python3/dist-packages/S3/BaseUtils.py:57: SyntaxWarning: invalid escape sequence '\s'
chaiml-mistral-24b-2048-54828-v1-uploader: cp /dev/shm/model_output/tokenizer_config.json s3://guanaco-vllm-models/chaiml-mistral-24b-2048-54828-v1/default/tokenizer_config.json
chaiml-mistral-24b-2048-48145-v1-uploader:   RE_XML_NAMESPACE = re.compile(b'^(<?[^>]+?>\s*|\s*)(<\w+) xmlns=[\'"](https?://[^\'"]+)[\'"]', re.MULTILINE)
chaiml-mistral-24b-2048-54828-v1-uploader: cp /dev/shm/model_output/recipe.yaml s3://guanaco-vllm-models/chaiml-mistral-24b-2048-54828-v1/default/recipe.yaml
chaiml-mistral-24b-2048-48145-v1-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:240: SyntaxWarning: invalid escape sequence '\.'
chaiml-mistral-24b-2048-54828-v1-uploader: cp /dev/shm/model_output/special_tokens_map.json s3://guanaco-vllm-models/chaiml-mistral-24b-2048-54828-v1/default/special_tokens_map.json
chaiml-mistral-24b-2048-48145-v1-uploader:   invalid = re.search("([^a-z0-9\.-])", bucket, re.UNICODE)
chaiml-mistral-24b-2048-54828-v1-uploader: cp /dev/shm/model_output/tokenizer.json s3://guanaco-vllm-models/chaiml-mistral-24b-2048-54828-v1/default/tokenizer.json
chaiml-mistral-24b-2048-48145-v1-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:244: SyntaxWarning: invalid escape sequence '\.'
chaiml-mistral-24b-2048-48145-v1-uploader:   invalid = re.search("([^A-Za-z0-9\._-])", bucket, re.UNICODE)
chaiml-mistral-24b-2048-48145-v1-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:255: SyntaxWarning: invalid escape sequence '\.'
chaiml-mistral-24b-2048-48145-v1-uploader:   if re.search("-\.", bucket, re.UNICODE):
chaiml-mistral-24b-2048-48145-v1-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:257: SyntaxWarning: invalid escape sequence '\.'
chaiml-mistral-24b-2048-48145-v1-uploader:   if re.search("\.\.", bucket, re.UNICODE):
chaiml-mistral-24b-2048-48145-v1-uploader: /usr/lib/python3/dist-packages/S3/S3Uri.py:155: SyntaxWarning: invalid escape sequence '\w'
chaiml-mistral-24b-2048-48145-v1-uploader:   _re = re.compile("^(\w+://)?(.*)", re.UNICODE)
chaiml-mistral-24b-2048-48145-v1-uploader: /usr/lib/python3/dist-packages/S3/FileLists.py:480: SyntaxWarning: invalid escape sequence '\*'
chaiml-mistral-24b-2048-48145-v1-uploader:   wildcard_split_result = re.split("\*|\?", uri_str, maxsplit=1)
chaiml-mistral-24b-2048-48145-v1-uploader: Bucket 's3://guanaco-vllm-models/' created
chaiml-mistral-24b-2048-48145-v1-uploader: uploading /dev/shm/model_output to s3://guanaco-vllm-models/chaiml-mistral-24b-2048-48145-v1/default
chaiml-mistral-24b-2048-48145-v1-uploader: cp /dev/shm/model_output/generation_config.json s3://guanaco-vllm-models/chaiml-mistral-24b-2048-48145-v1/default/generation_config.json
Job chaiml-mistral-24b-2048-74171-v1-uploader completed after 315.05s with status: succeeded
chaiml-mistral-24b-2048-48145-v1-uploader: cp /dev/shm/model_output/config.json s3://guanaco-vllm-models/chaiml-mistral-24b-2048-48145-v1/default/config.json
Stopping job with name chaiml-mistral-24b-2048-74171-v1-uploader
chaiml-mistral-24b-2048-48145-v1-uploader: cp /dev/shm/model_output/model.safetensors.index.json s3://guanaco-vllm-models/chaiml-mistral-24b-2048-48145-v1/default/model.safetensors.index.json
Pipeline stage VLLMUploader completed in 316.97s
chaiml-mistral-24b-2048-48145-v1-uploader: cp /dev/shm/model_output/special_tokens_map.json s3://guanaco-vllm-models/chaiml-mistral-24b-2048-48145-v1/default/special_tokens_map.json
run pipeline stage %s
chaiml-mistral-24b-2048-48145-v1-uploader: cp /dev/shm/model_output/recipe.yaml s3://guanaco-vllm-models/chaiml-mistral-24b-2048-48145-v1/default/recipe.yaml
Running pipeline stage VLLMTemplater
chaiml-mistral-24b-2048-48145-v1-uploader: cp /dev/shm/model_output/chat_template.jinja s3://guanaco-vllm-models/chaiml-mistral-24b-2048-48145-v1/default/chat_template.jinja
chaiml-mistral-24b-2048-48145-v1-uploader: cp /dev/shm/model_output/tokenizer_config.json s3://guanaco-vllm-models/chaiml-mistral-24b-2048-48145-v1/default/tokenizer_config.json
Pipeline stage VLLMTemplater completed in 1.82s
chaiml-mistral-24b-2048-48145-v1-uploader: cp /dev/shm/model_output/tokenizer.json s3://guanaco-vllm-models/chaiml-mistral-24b-2048-48145-v1/default/tokenizer.json
run pipeline stage %s
chaiml-mistral-24b-2048-48145-v1-uploader: cp /dev/shm/model_output/model-00006-of-00006.safetensors s3://guanaco-vllm-models/chaiml-mistral-24b-2048-48145-v1/default/model-00006-of-00006.safetensors
Running pipeline stage VLLMDeployer
chaiml-mistral-24b-2048-48145-v1-uploader: cp /dev/shm/model_output/model-00001-of-00006.safetensors s3://guanaco-vllm-models/chaiml-mistral-24b-2048-48145-v1/default/model-00001-of-00006.safetensors
Job chaiml-mistral-24b-2048-54828-v1-uploader completed after 325.03s with status: succeeded
Creating inference service chaiml-mistral-24b-2048-74171-v1
Stopping job with name chaiml-mistral-24b-2048-54828-v1-uploader
Waiting for inference service chaiml-mistral-24b-2048-74171-v1 to be ready
Pipeline stage VLLMUploader completed in 327.08s
run pipeline stage %s
Running pipeline stage VLLMTemplater
Pipeline stage VLLMTemplater completed in 0.62s
run pipeline stage %s
Running pipeline stage VLLMDeployer
Creating inference service chaiml-mistral-24b-2048-54828-v1
Waiting for inference service chaiml-mistral-24b-2048-54828-v1 to be ready
Job chaiml-mistral-24b-2048-48145-v1-uploader completed after 325.54s with status: succeeded
Stopping job with name chaiml-mistral-24b-2048-48145-v1-uploader
Pipeline stage VLLMUploader completed in 328.03s
run pipeline stage %s
Running pipeline stage VLLMTemplater
Pipeline stage VLLMTemplater completed in 0.63s
run pipeline stage %s
Running pipeline stage VLLMDeployer
Creating inference service chaiml-mistral-24b-2048-48145-v1
Waiting for inference service chaiml-mistral-24b-2048-48145-v1 to be ready
Inference service chaiml-mistral-24b-2048-74171-v1 ready after 152.09994769096375s
Pipeline stage VLLMDeployer completed in 154.59s
run pipeline stage %s
Running pipeline stage StressChecker
Inference service chaiml-mistral-24b-2048-54828-v1 ready after 152.04836249351501s
Pipeline stage VLLMDeployer completed in 154.44s
run pipeline stage %s
Received healthy response to inference request in 3.1095733642578125s
Running pipeline stage StressChecker
Received healthy response to inference request in 3.3050708770751953s
Received healthy response to inference request in 1.8970415592193604s
Received healthy response to inference request in 1.5897958278656006s
Received healthy response to inference request in 2.701477527618408s
Received healthy response to inference request in 2.344989061355591s
Received healthy response to inference request in 1.751725196838379s
Received healthy response to inference request in 1.8635766506195068s
Received healthy response to inference request in 1.9775762557983398s
Received healthy response to inference request in 1.769789218902588s
Inference service chaiml-mistral-24b-2048-48145-v1 ready after 162.25080275535583s
Pipeline stage VLLMDeployer completed in 164.66s
Received healthy response to inference request in 1.6209816932678223s
run pipeline stage %s
Received healthy response to inference request in 2.0956578254699707s
Running pipeline stage StressChecker
Received healthy response to inference request in 1.9855561256408691s
Received healthy response to inference request in 1.7215731143951416s
Received healthy response to inference request in 1.7404775619506836s
Received healthy response to inference request in 1.5390911102294922s
Received healthy response to inference request in 1.9935355186462402s
Received healthy response to inference request in 1.8430721759796143s
Received healthy response to inference request in 1.7443642616271973s
Received healthy response to inference request in 1.6230194568634033s
Received healthy response to inference request in 1.6414895057678223s
Received healthy response to inference request in 1.6702632904052734s
Received healthy response to inference request in 1.6701347827911377s
Received healthy response to inference request in 1.6712608337402344s
Received healthy response to inference request in 1.734403133392334s
Received healthy response to inference request in 1.6317195892333984s
Received healthy response to inference request in 1.731830358505249s
Received healthy response to inference request in 2.095813751220703s
Received healthy response to inference request in 1.727579116821289s
Received healthy response to inference request in 1.855262279510498s
Received healthy response to inference request in 1.7769343852996826s
Received healthy response to inference request in 1.6906943321228027s
Received healthy response to inference request in 2.015777349472046s
Received healthy response to inference request in 1.6715054512023926s
Received healthy response to inference request in 1.743746042251587s
Received healthy response to inference request in 1.6974079608917236s
Received healthy response to inference request in 1.6460456848144531s
Received healthy response to inference request in 1.6540982723236084s
Received healthy response to inference request in 1.6984434127807617s
Received healthy response to inference request in 1.679671049118042s
Received healthy response to inference request in 1.6680996417999268s
Received healthy response to inference request in 1.5742502212524414s
Received healthy response to inference request in 1.5399143695831299s
Received healthy response to inference request in 1.7305221557617188s
Received healthy response to inference request in 1.9506511688232422s
Received healthy response to inference request in 1.7334051132202148s
Received healthy response to inference request in 1.6268086433410645s
Received healthy response to inference request in 1.5581591129302979s
Received healthy response to inference request in 1.620826244354248s
Received healthy response to inference request in 1.6549453735351562s
Received healthy response to inference request in 1.6298060417175293s
Received healthy response to inference request in 1.871164321899414s
Received healthy response to inference request in 1.8859548568725586s
Received healthy response to inference request in 1.9044175148010254s
Received healthy response to inference request in 1.843815565109253s
Received healthy response to inference request in 2.6322133541107178s
Received healthy response to inference request in 2.0248332023620605s
Received healthy response to inference request in 2.72233247756958s
Received healthy response to inference request in 1.5403053760528564s
Received healthy response to inference request in 1.7405619621276855s
Received healthy response to inference request in 1.553553581237793s
Received healthy response to inference request in 1.5423283576965332s
Received healthy response to inference request in 1.6592438220977783s
Received healthy response to inference request in 1.6467230319976807s
Received healthy response to inference request in 1.558824062347412s
Received healthy response to inference request in 1.6617026329040527s
Received healthy response to inference request in 1.5234184265136719s
Received healthy response to inference request in 1.5432236194610596s
Received healthy response to inference request in 1.7383527755737305s
Received healthy response to inference request in 2.214874029159546s
Received healthy response to inference request in 1.8518483638763428s
Received healthy response to inference request in 1.665281057357788s
Received healthy response to inference request in 1.540450096130371s
Received healthy response to inference request in 1.6115384101867676s
Received healthy response to inference request in 1.7629711627960205s
Received healthy response to inference request in 1.6554245948791504s
Received healthy response to inference request in 1.6510930061340332s
Received healthy response to inference request in 1.6619620323181152s
Received healthy response to inference request in 1.6427326202392578s
Received healthy response to inference request in 1.708552360534668s
Received healthy response to inference request in 1.5888347625732422s
30 requests
Received healthy response to inference request in 1.6219522953033447s
0 failed requests
Received healthy response to inference request in 1.5914885997772217s
5th percentile: 1.5497514247894286
10th percentile: 1.5727076053619384
20th percentile: 1.614744520187378
30th percentile: 1.6302463054656982
Received healthy response to inference request in 1.811372995376587s
Received healthy response to inference request in 1.537226915359497s
40th percentile: 1.6639534473419189
30 requests
50th percentile: 1.6979256868362427
0 failed requests
60th percentile: 1.7336544036865233
5th percentile: 1.5499445915222168
70th percentile: 1.7979254484176632
Received healthy response to inference request in 1.7011477947235107s
10th percentile: 1.5881556510925292
80th percentile: 1.9871520042419435
20th percentile: 1.6391528129577637
90th percentile: 2.373711490631104
30th percentile: 1.6577001571655274
95th percentile: 2.8947613596916186
40th percentile: 1.6699963569641114
Received healthy response to inference request in 1.6540212631225586s
99th percentile: 3.2483765983581545
50th percentile: 1.7150627374649048
mean time: 1.8603501399358113
60th percentile: 1.741835594177246
Pipeline stage StressChecker completed in 91.55s
70th percentile: 1.8457050323486328
run pipeline stage %s
80th percentile: 1.8985167503356934
Received healthy response to inference request in 2.7125020027160645s
Running pipeline stage OfflineFamilyFriendlyTriggerPipeline
90th percentile: 1.982301950454712
run_pipeline:run_in_cloud %s
95th percentile: 2.063786745071411
starting trigger_guanaco_pipeline args=%s
99th percentile: 2.525789813995362
Received healthy response to inference request in 1.6454503536224365s
mean time: 1.7756317218144735
triggered trigger_guanaco_pipeline args=%s
Pipeline stage StressChecker completed in 91.80s
Pipeline stage OfflineFamilyFriendlyTriggerPipeline completed in 3.33s
run pipeline stage %s
Shutdown handler de-registered
Running pipeline stage OfflineFamilyFriendlyTriggerPipeline
Running pipeline stage OfflineFamilyFriendlyTriggerPipeline
Received healthy response to inference request in 1.9281189441680908s
chaiml-mistral-24b-2048_74171_v1 status is now deployed due to DeploymentManager action
run_pipeline:run_in_cloud %s
chaiml-mistral-24b-2048_74171_v1 status is now inactive due to auto deactivation removed underperforming models