chaiml-98p-2ff-chaiml-m_45461

developer_uid: rirv938
submission_id: chaiml-98p-2ff-chaiml-m_45461_v1
model_name: chaiml-98p-2ff-chaiml-m_45461_v1
model_group: ChaiML/98p_2ff_chaiml_mi
status: inactive
timestamp: 2026-02-17T22:27:19+00:00
num_battles: 10331
num_wins: 5667
celo_rating: 1331.01
family_friendly_score: 0.0
family_friendly_standard_error: 0.0
submission_type: basic
model_repo: ChaiML/98p_2ff_chaiml_mistral_24b_2048_90555_v2_cp624_merged
model_architecture: MistralForCausalLM
model_num_parameters: 24096691200.0
best_of: 8
max_input_tokens: 2048
max_output_tokens: 80
reward_model: default
display_name: chaiml-98p-2ff-chaiml-m_45461_v1
ineligible_reason: max_output_tokens!=64
is_internal_developer: True
language_model: ChaiML/98p_2ff_chaiml_mistral_24b_2048_90555_v2_cp624_merged
model_size: 24B
ranking_group: single
us_pacific_date: 2026-02-17
win_ratio: 0.548543219436647
generation_params: {'temperature': 1.0, 'top_p': 0.95, 'min_p': 0.05, 'top_k': 60, 'presence_penalty': 0.1, 'frequency_penalty': 0.0, 'stopping_words': ['<|im_end|>', 'You:', '<|im_start|>', '###', '</s>'], 'max_input_tokens': 2048, 'best_of': 8, 'max_output_tokens': 80}
formatter: {'memory_template': '[SYSTEM_PROMPT]Respond as a high quality storyteller.[/SYSTEM_PROMPT][INST]', 'prompt_template': '', 'bot_template': '{bot_name}: {message}\n', 'user_template': '{user_name}: {message}\n', 'response_template': '[/INST]{bot_name}:', 'truncate_by_message': False}
Resubmit model
Shutdown handler not registered because Python interpreter is not running in the main thread
run pipeline %s
run pipeline stage %s
Running pipeline stage VLLMUploader
Starting job with name chaiml-98p-2ff-chaiml-m-45461-v1-uploader
Waiting for job on chaiml-98p-2ff-chaiml-m-45461-v1-uploader to finish
chaiml-98p-2ff-chaiml-m-45461-v1-uploader: Using quantization_mode: fp8
chaiml-98p-2ff-chaiml-m-45461-v1-uploader: Checking if ChaiML/98p_2ff_chaiml_mistral_24b_2048_90555_v2_cp624_merged-FP8 already exists in ChaiML
chaiml-98p-2ff-chaiml-m-45461-v1-uploader: Downloading snapshot of ChaiML/98p_2ff_chaiml_mistral_24b_2048_90555_v2_cp624_merged...
chaiml-98p-2ff-chaiml-m-45461-v1-uploader: Downloaded in 43.243s
chaiml-98p-2ff-chaiml-m-45461-v1-uploader: Loading /tmp/model_input...
chaiml-98p-2ff-chaiml-m-45461-v1-uploader: The tokenizer you are loading from '/tmp/model_input' with an incorrect regex pattern: https://huggingface.co/mistralai/Mistral-Small-3.1-24B-Instruct-2503/discussions/84#69121093e8b480e709447d5e. This will lead to incorrect tokenization. You should set the `fix_mistral_regex=True` flag when loading this tokenizer to fix this issue.
chaiml-98p-2ff-chaiml-m-45461-v1-uploader: `torch_dtype` is deprecated! Use `dtype` instead!
chaiml-98p-2ff-chaiml-m-45461-v1-uploader: Some parameters are on the meta device because they were offloaded to the cpu.
chaiml-98p-2ff-chaiml-m-45461-v1-uploader: Applying quantization...
chaiml-98p-2ff-chaiml-m-45461-v1-uploader: The tokenizer you are loading from '/tmp/model_input' with an incorrect regex pattern: https://huggingface.co/mistralai/Mistral-Small-3.1-24B-Instruct-2503/discussions/84#69121093e8b480e709447d5e. This will lead to incorrect tokenization. You should set the `fix_mistral_regex=True` flag when loading this tokenizer to fix this issue.
chaiml-98p-2ff-chaiml-m-45461-v1-uploader: 2026-02-17T11:15:14.764579-0800 | reset | INFO - Compression lifecycle reset
chaiml-98p-2ff-chaiml-m-45461-v1-uploader: 2026-02-17T11:15:14.765479-0800 | from_modifiers | INFO - Creating recipe from modifiers
chaiml-98p-2ff-chaiml-m-45461-v1-uploader: 2026-02-17T11:15:14.858888-0800 | initialize | INFO - Compression lifecycle initialized for 1 modifiers
chaiml-98p-2ff-chaiml-m-45461-v1-uploader: 2026-02-17T11:15:14.859173-0800 | IndependentPipeline | INFO - Inferred `DataFreePipeline` for `QuantizationModifier`
chaiml-98p-2ff-chaiml-m-45461-v1-uploader: Some parameters are on the meta device because they were offloaded to the cpu.
chaiml-98p-2ff-chaiml-m-45461-v1-uploader: 2026-02-17T11:15:57.305788-0800 | finalize | INFO - Compression lifecycle finalized for 1 modifiers
chaiml-98p-2ff-chaiml-m-45461-v1-uploader: 2026-02-17T11:15:59.512399-0800 | post_process | WARNING - Optimized model is not saved. To save, please provide`output_dir` as input arg.Ex. `oneshot(..., output_dir=...)`
chaiml-98p-2ff-chaiml-m-45461-v1-uploader: Saving to /dev/shm/model_output...
chaiml-98p-2ff-chaiml-m-45461-v1-uploader: 2026-02-17T11:15:59.539204-0800 | get_model_compressor | INFO - skip_sparsity_compression_stats set to True. Skipping sparsity compression statistic calculations. No sparsity compressor will be applied.
Failed to get response for submission chaiml-mistral-24b-2048-_2678_v3: ('http://chaiml-mistral-24b-2048-2678-v3-predictor.tenant-chaiml-guanaco.kchai-coreweave-us-east-04a.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', '')
chaiml-98p-2ff-chaiml-m-45461-v1-uploader: Cleaning quantization config in /dev/shm/model_output
chaiml-98p-2ff-chaiml-m-45461-v1-uploader: Pushing to ChaiML/98p_2ff_chaiml_mistral_24b_2048_90555_v2_cp624_merged-FP8
chaiml-98p-2ff-chaiml-m-45461-v1-uploader: Checking if ChaiML/98p_2ff_chaiml_mistral_24b_2048_90555_v2_cp624_merged-FP8 already exists in ChaiML
chaiml-98p-2ff-chaiml-m-45461-v1-uploader: Creating repo ChaiML/98p_2ff_chaiml_mistral_24b_2048_90555_v2_cp624_merged-FP8 and uploading /dev/shm/model_output to it
chaiml-98p-2ff-chaiml-m-45461-v1-uploader: ---------- 2026-02-17 11:16:54 (0:00:00) ----------
chaiml-98p-2ff-chaiml-m-45461-v1-uploader: Files:   hashed 6/13 (276.0K/27.6G) | pre-uploaded: 0/0 (0.0/27.6G) (+13 unsure) | committed: 0/13 (0.0/27.6G) | ignored: 0
chaiml-98p-2ff-chaiml-m-45461-v1-uploader: Workers: hashing: 7 | get upload mode: 5 | pre-uploading: 0 | committing: 0 | waiting: 114
chaiml-98p-2ff-chaiml-m-45461-v1-uploader: ---------------------------------------------------
chaiml-98p-2ff-chaiml-m-45461-v1-uploader: 
[K[F
[K[F
[K[F
[K[F
[K[F
[K[F
[K[F
chaiml-98p-2ff-chaiml-m-45461-v1-uploader: ---------- 2026-02-17 11:17:54 (0:01:00) ----------
chaiml-98p-2ff-chaiml-m-45461-v1-uploader: Files:   hashed 13/13 (27.6G/27.6G) | pre-uploaded: 7/7 (27.6G/27.6G) | committed: 0/13 (0.0/27.6G) | ignored: 0
chaiml-98p-2ff-chaiml-m-45461-v1-uploader: Workers: hashing: 0 | get upload mode: 0 | pre-uploading: 0 | committing: 1 | waiting: 125
chaiml-98p-2ff-chaiml-m-45461-v1-uploader: ---------------------------------------------------
chaiml-98p-2ff-chaiml-m-45461-v1-uploader:                              Processed model ChaiML/98p_2ff_chaiml_mistral_24b_2048_90555_v2_cp624_merged in 231.049s
chaiml-98p-2ff-chaiml-m-45461-v1-uploader: creating bucket guanaco-vllm-models
chaiml-98p-2ff-chaiml-m-45461-v1-uploader: /usr/lib/python3/dist-packages/S3/BaseUtils.py:56: SyntaxWarning: invalid escape sequence '\.'
chaiml-98p-2ff-chaiml-m-45461-v1-uploader:   RE_S3_DATESTRING = re.compile('\.[0-9]*(?:[Z\\-\\+]*?)')
chaiml-98p-2ff-chaiml-m-45461-v1-uploader: /usr/lib/python3/dist-packages/S3/BaseUtils.py:57: SyntaxWarning: invalid escape sequence '\s'
chaiml-98p-2ff-chaiml-m-45461-v1-uploader:   RE_XML_NAMESPACE = re.compile(b'^(<?[^>]+?>\s*|\s*)(<\w+) xmlns=[\'"](https?://[^\'"]+)[\'"]', re.MULTILINE)
chaiml-98p-2ff-chaiml-m-45461-v1-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:240: SyntaxWarning: invalid escape sequence '\.'
chaiml-98p-2ff-chaiml-m-45461-v1-uploader:   invalid = re.search("([^a-z0-9\.-])", bucket, re.UNICODE)
chaiml-98p-2ff-chaiml-m-45461-v1-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:244: SyntaxWarning: invalid escape sequence '\.'
chaiml-98p-2ff-chaiml-m-45461-v1-uploader:   invalid = re.search("([^A-Za-z0-9\._-])", bucket, re.UNICODE)
chaiml-98p-2ff-chaiml-m-45461-v1-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:255: SyntaxWarning: invalid escape sequence '\.'
chaiml-98p-2ff-chaiml-m-45461-v1-uploader:   if re.search("-\.", bucket, re.UNICODE):
chaiml-98p-2ff-chaiml-m-45461-v1-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:257: SyntaxWarning: invalid escape sequence '\.'
chaiml-98p-2ff-chaiml-m-45461-v1-uploader:   if re.search("\.\.", bucket, re.UNICODE):
chaiml-98p-2ff-chaiml-m-45461-v1-uploader: /usr/lib/python3/dist-packages/S3/S3Uri.py:155: SyntaxWarning: invalid escape sequence '\w'
chaiml-98p-2ff-chaiml-m-45461-v1-uploader:   _re = re.compile("^(\w+://)?(.*)", re.UNICODE)
chaiml-98p-2ff-chaiml-m-45461-v1-uploader: /usr/lib/python3/dist-packages/S3/FileLists.py:480: SyntaxWarning: invalid escape sequence '\*'
chaiml-98p-2ff-chaiml-m-45461-v1-uploader:   wildcard_split_result = re.split("\*|\?", uri_str, maxsplit=1)
chaiml-98p-2ff-chaiml-m-45461-v1-uploader: Bucket 's3://guanaco-vllm-models/' created
chaiml-98p-2ff-chaiml-m-45461-v1-uploader: uploading /dev/shm/model_output to s3://guanaco-vllm-models/chaiml-98p-2ff-chaiml-m-45461-v1/default
chaiml-98p-2ff-chaiml-m-45461-v1-uploader: cp /dev/shm/model_output/generation_config.json s3://guanaco-vllm-models/chaiml-98p-2ff-chaiml-m-45461-v1/default/generation_config.json
chaiml-98p-2ff-chaiml-m-45461-v1-uploader: cp /dev/shm/model_output/config.json s3://guanaco-vllm-models/chaiml-98p-2ff-chaiml-m-45461-v1/default/config.json
chaiml-98p-2ff-chaiml-m-45461-v1-uploader: cp /dev/shm/model_output/recipe.yaml s3://guanaco-vllm-models/chaiml-98p-2ff-chaiml-m-45461-v1/default/recipe.yaml
chaiml-98p-2ff-chaiml-m-45461-v1-uploader: cp /dev/shm/model_output/special_tokens_map.json s3://guanaco-vllm-models/chaiml-98p-2ff-chaiml-m-45461-v1/default/special_tokens_map.json
chaiml-98p-2ff-chaiml-m-45461-v1-uploader: cp /dev/shm/model_output/model.safetensors.index.json s3://guanaco-vllm-models/chaiml-98p-2ff-chaiml-m-45461-v1/default/model.safetensors.index.json
chaiml-98p-2ff-chaiml-m-45461-v1-uploader: cp /dev/shm/model_output/tokenizer_config.json s3://guanaco-vllm-models/chaiml-98p-2ff-chaiml-m-45461-v1/default/tokenizer_config.json
chaiml-98p-2ff-chaiml-m-45461-v1-uploader: cp /dev/shm/model_output/tokenizer.json s3://guanaco-vllm-models/chaiml-98p-2ff-chaiml-m-45461-v1/default/tokenizer.json
Failed to get response for submission chaiml-mistral-24b-2048_54327_v6: ('http://chaiml-mistral-24b-2048-54327-v6-predictor.tenant-chaiml-guanaco.kchai-coreweave-us-east-04a.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', '')
chaiml-98p-2ff-chaiml-m-45461-v1-uploader: cp /dev/shm/model_output/model-00006-of-00006.safetensors s3://guanaco-vllm-models/chaiml-98p-2ff-chaiml-m-45461-v1/default/model-00006-of-00006.safetensors
chaiml-98p-2ff-chaiml-m-45461-v1-uploader: cp /dev/shm/model_output/model-00001-of-00006.safetensors s3://guanaco-vllm-models/chaiml-98p-2ff-chaiml-m-45461-v1/default/model-00001-of-00006.safetensors
chaiml-98p-2ff-chaiml-m-45461-v1-uploader: cp /dev/shm/model_output/model-00003-of-00006.safetensors s3://guanaco-vllm-models/chaiml-98p-2ff-chaiml-m-45461-v1/default/model-00003-of-00006.safetensors
chaiml-98p-2ff-chaiml-m-45461-v1-uploader: cp /dev/shm/model_output/model-00005-of-00006.safetensors s3://guanaco-vllm-models/chaiml-98p-2ff-chaiml-m-45461-v1/default/model-00005-of-00006.safetensors
chaiml-98p-2ff-chaiml-m-45461-v1-uploader: cp /dev/shm/model_output/model-00004-of-00006.safetensors s3://guanaco-vllm-models/chaiml-98p-2ff-chaiml-m-45461-v1/default/model-00004-of-00006.safetensors
chaiml-98p-2ff-chaiml-m-45461-v1-uploader: cp /dev/shm/model_output/model-00002-of-00006.safetensors s3://guanaco-vllm-models/chaiml-98p-2ff-chaiml-m-45461-v1/default/model-00002-of-00006.safetensors
Job chaiml-98p-2ff-chaiml-m-45461-v1-uploader completed after 401.93s with status: succeeded
Stopping job with name chaiml-98p-2ff-chaiml-m-45461-v1-uploader
Pipeline stage VLLMUploader completed in 402.48s
run pipeline stage %s
Running pipeline stage VLLMTemplater
Pipeline stage VLLMTemplater completed in 0.15s
run pipeline stage %s
Running pipeline stage VLLMDeployer
Creating inference service chaiml-98p-2ff-chaiml-m-45461-v1
Waiting for inference service chaiml-98p-2ff-chaiml-m-45461-v1 to be ready
Inference service chaiml-98p-2ff-chaiml-m-45461-v1 ready after 191.26285767555237s
Pipeline stage VLLMDeployer completed in 191.87s
run pipeline stage %s
Running pipeline stage StressChecker
Received healthy response to inference request in 1.2536156177520752s
Received healthy response to inference request in 1.3515546321868896s
Received healthy response to inference request in 1.452624797821045s
Received healthy response to inference request in 1.5224571228027344s
Received healthy response to inference request in 1.142937421798706s
Received healthy response to inference request in 1.457373857498169s
Received healthy response to inference request in 1.213186264038086s
Received healthy response to inference request in 1.1507642269134521s
Received healthy response to inference request in 1.1601135730743408s
Received healthy response to inference request in 1.5099546909332275s
Received healthy response to inference request in 1.1384999752044678s
Received healthy response to inference request in 1.225421667098999s
Received healthy response to inference request in 1.6968567371368408s
Received healthy response to inference request in 1.4420161247253418s
Received healthy response to inference request in 1.3522329330444336s
Received healthy response to inference request in 1.1635987758636475s
Received healthy response to inference request in 1.2713255882263184s
Received healthy response to inference request in 1.1737942695617676s
Received healthy response to inference request in 1.3326611518859863s
Received healthy response to inference request in 1.1555249691009521s
Received healthy response to inference request in 1.246959924697876s
Received healthy response to inference request in 1.5577147006988525s
Received healthy response to inference request in 1.491452932357788s
Received healthy response to inference request in 1.4325990676879883s
Received healthy response to inference request in 1.403806447982788s
Received healthy response to inference request in 1.4398248195648193s
Received healthy response to inference request in 1.1939198970794678s
Received healthy response to inference request in 1.248302698135376s
Received healthy response to inference request in 1.2339298725128174s
Received healthy response to inference request in 1.2320356369018555s
30 requests
0 failed requests
5th percentile: 1.1464594841003417
10th percentile: 1.1550488948822022
20th percentile: 1.1717551708221436
30th percentile: 1.221751046180725
40th percentile: 1.2417479038238526
50th percentile: 1.2624706029891968
60th percentile: 1.3518259525299072
70th percentile: 1.4347667932510375
80th percentile: 1.4535746097564697
90th percentile: 1.5112049341201783
95th percentile: 1.5418487906455993
99th percentile: 1.6565055465698244
mean time: 1.3215686798095703
Pipeline stage StressChecker completed in 42.93s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyTriggerPipeline
run_pipeline:run_in_cloud %s
starting trigger_guanaco_pipeline args=%s
triggered trigger_guanaco_pipeline args=%s
Pipeline stage OfflineFamilyFriendlyTriggerPipeline completed in 0.68s
Shutdown handler de-registered
chaiml-98p-2ff-chaiml-m_45461_v1 status is now deployed due to DeploymentManager action
chaiml-98p-2ff-chaiml-m_45461_v1 status is now inactive due to auto deactivation removed underperforming models