chaiml-98p-2ff-chaiml-m_29161

developer_uid: rirv938
submission_id: chaiml-98p-2ff-chaiml-m_29161_v1
model_name: chaiml-98p-2ff-chaiml-m_29161_v1
model_group: ChaiML/98p_2ff_chaiml_mi
status: inactive
timestamp: 2026-02-17T22:27:20+00:00
num_battles: 10354
num_wins: 5603
celo_rating: 1325.19
family_friendly_score: 0.0
family_friendly_standard_error: 0.0
submission_type: basic
model_repo: ChaiML/98p_2ff_chaiml_mistral_24b_2048_90555_v2_cp312_merged
model_architecture: MistralForCausalLM
model_num_parameters: 24096691200.0
best_of: 8
max_input_tokens: 2048
max_output_tokens: 80
reward_model: default
display_name: chaiml-98p-2ff-chaiml-m_29161_v1
ineligible_reason: max_output_tokens!=64
is_internal_developer: True
language_model: ChaiML/98p_2ff_chaiml_mistral_24b_2048_90555_v2_cp312_merged
model_size: 24B
ranking_group: single
us_pacific_date: 2026-02-17
win_ratio: 0.5411435194127874
generation_params: {'temperature': 1.0, 'top_p': 0.95, 'min_p': 0.05, 'top_k': 60, 'presence_penalty': 0.1, 'frequency_penalty': 0.0, 'stopping_words': ['</s>', '###', 'You:', '<|im_start|>', '<|im_end|>'], 'max_input_tokens': 2048, 'best_of': 8, 'max_output_tokens': 80}
formatter: {'memory_template': '[SYSTEM_PROMPT]Respond as a high quality storyteller.[/SYSTEM_PROMPT][INST]', 'prompt_template': '', 'bot_template': '{bot_name}: {message}\n', 'user_template': '{user_name}: {message}\n', 'response_template': '[/INST]{bot_name}:', 'truncate_by_message': False}
Resubmit model
Shutdown handler not registered because Python interpreter is not running in the main thread
run pipeline %s
run pipeline stage %s
Running pipeline stage VLLMUploader
Starting job with name chaiml-98p-2ff-chaiml-m-29161-v1-uploader
Waiting for job on chaiml-98p-2ff-chaiml-m-29161-v1-uploader to finish
Failed to get response for submission chaiml-mistral-24b-2048-_2678_v3: ('http://chaiml-mistral-24b-2048-2678-v3-predictor.tenant-chaiml-guanaco.kchai-coreweave-us-east-04a.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', '')
chaiml-98p-2ff-chaiml-m-29161-v1-uploader: Using quantization_mode: fp8
chaiml-98p-2ff-chaiml-m-29161-v1-uploader: Checking if ChaiML/98p_2ff_chaiml_mistral_24b_2048_90555_v2_cp312_merged-FP8 already exists in ChaiML
chaiml-98p-2ff-chaiml-m-29161-v1-uploader: Downloading snapshot of ChaiML/98p_2ff_chaiml_mistral_24b_2048_90555_v2_cp312_merged...
chaiml-98p-2ff-chaiml-m-29161-v1-uploader: Downloaded in 38.063s
chaiml-98p-2ff-chaiml-m-29161-v1-uploader: Loading /tmp/model_input...
chaiml-98p-2ff-chaiml-m-29161-v1-uploader: The tokenizer you are loading from '/tmp/model_input' with an incorrect regex pattern: https://huggingface.co/mistralai/Mistral-Small-3.1-24B-Instruct-2503/discussions/84#69121093e8b480e709447d5e. This will lead to incorrect tokenization. You should set the `fix_mistral_regex=True` flag when loading this tokenizer to fix this issue.
chaiml-98p-2ff-chaiml-m-29161-v1-uploader: `torch_dtype` is deprecated! Use `dtype` instead!
chaiml-98p-2ff-chaiml-m-29161-v1-uploader: Some parameters are on the meta device because they were offloaded to the cpu.
chaiml-98p-2ff-chaiml-m-29161-v1-uploader: Applying quantization...
chaiml-98p-2ff-chaiml-m-29161-v1-uploader: The tokenizer you are loading from '/tmp/model_input' with an incorrect regex pattern: https://huggingface.co/mistralai/Mistral-Small-3.1-24B-Instruct-2503/discussions/84#69121093e8b480e709447d5e. This will lead to incorrect tokenization. You should set the `fix_mistral_regex=True` flag when loading this tokenizer to fix this issue.
chaiml-98p-2ff-chaiml-m-29161-v1-uploader: 2026-02-17T11:15:32.286611-0800 | reset | INFO - Compression lifecycle reset
chaiml-98p-2ff-chaiml-m-29161-v1-uploader: 2026-02-17T11:15:32.287550-0800 | from_modifiers | INFO - Creating recipe from modifiers
chaiml-98p-2ff-chaiml-m-29161-v1-uploader: 2026-02-17T11:15:32.380706-0800 | initialize | INFO - Compression lifecycle initialized for 1 modifiers
chaiml-98p-2ff-chaiml-m-29161-v1-uploader: 2026-02-17T11:15:32.380973-0800 | IndependentPipeline | INFO - Inferred `DataFreePipeline` for `QuantizationModifier`
chaiml-98p-2ff-chaiml-m-29161-v1-uploader: Some parameters are on the meta device because they were offloaded to the cpu.
chaiml-98p-2ff-chaiml-m-29161-v1-uploader: 2026-02-17T11:16:14.402323-0800 | finalize | INFO - Compression lifecycle finalized for 1 modifiers
chaiml-98p-2ff-chaiml-m-29161-v1-uploader: 2026-02-17T11:16:16.598253-0800 | post_process | WARNING - Optimized model is not saved. To save, please provide`output_dir` as input arg.Ex. `oneshot(..., output_dir=...)`
chaiml-98p-2ff-chaiml-m-29161-v1-uploader: Saving to /dev/shm/model_output...
chaiml-98p-2ff-chaiml-m-29161-v1-uploader: 2026-02-17T11:16:16.625283-0800 | get_model_compressor | INFO - skip_sparsity_compression_stats set to True. Skipping sparsity compression statistic calculations. No sparsity compressor will be applied.
chaiml-98p-2ff-chaiml-m-29161-v1-uploader: Cleaning quantization config in /dev/shm/model_output
chaiml-98p-2ff-chaiml-m-29161-v1-uploader: Pushing to ChaiML/98p_2ff_chaiml_mistral_24b_2048_90555_v2_cp312_merged-FP8
chaiml-98p-2ff-chaiml-m-29161-v1-uploader: Checking if ChaiML/98p_2ff_chaiml_mistral_24b_2048_90555_v2_cp312_merged-FP8 already exists in ChaiML
chaiml-98p-2ff-chaiml-m-29161-v1-uploader: Creating repo ChaiML/98p_2ff_chaiml_mistral_24b_2048_90555_v2_cp312_merged-FP8 and uploading /dev/shm/model_output to it
chaiml-98p-2ff-chaiml-m-29161-v1-uploader: ---------- 2026-02-17 11:17:10 (0:00:00) ----------
chaiml-98p-2ff-chaiml-m-29161-v1-uploader: Files:   hashed 6/13 (276.0K/27.6G) | pre-uploaded: 0/0 (0.0/27.6G) (+13 unsure) | committed: 0/13 (0.0/27.6G) | ignored: 0
chaiml-98p-2ff-chaiml-m-29161-v1-uploader: Workers: hashing: 7 | get upload mode: 4 | pre-uploading: 0 | committing: 0 | waiting: 115
chaiml-98p-2ff-chaiml-m-29161-v1-uploader: ---------------------------------------------------
chaiml-98p-2ff-chaiml-m-29161-v1-uploader: 
[K[F
[K[F
[K[F
[K[F
[K[F
[K[F
[K[F
chaiml-98p-2ff-chaiml-m-29161-v1-uploader: ---------- 2026-02-17 11:18:10 (0:01:00) ----------
chaiml-98p-2ff-chaiml-m-29161-v1-uploader: Files:   hashed 13/13 (27.6G/27.6G) | pre-uploaded: 7/7 (27.6G/27.6G) | committed: 0/13 (0.0/27.6G) | ignored: 0
chaiml-98p-2ff-chaiml-m-29161-v1-uploader: Workers: hashing: 0 | get upload mode: 0 | pre-uploading: 0 | committing: 1 | waiting: 125
chaiml-98p-2ff-chaiml-m-29161-v1-uploader: ---------------------------------------------------
chaiml-98p-2ff-chaiml-m-29161-v1-uploader:                              Processed model ChaiML/98p_2ff_chaiml_mistral_24b_2048_90555_v2_cp312_merged in 217.814s
chaiml-98p-2ff-chaiml-m-29161-v1-uploader: creating bucket guanaco-vllm-models
chaiml-98p-2ff-chaiml-m-29161-v1-uploader: /usr/lib/python3/dist-packages/S3/BaseUtils.py:56: SyntaxWarning: invalid escape sequence '\.'
chaiml-98p-2ff-chaiml-m-29161-v1-uploader:   RE_S3_DATESTRING = re.compile('\.[0-9]*(?:[Z\\-\\+]*?)')
chaiml-98p-2ff-chaiml-m-29161-v1-uploader: /usr/lib/python3/dist-packages/S3/BaseUtils.py:57: SyntaxWarning: invalid escape sequence '\s'
chaiml-98p-2ff-chaiml-m-29161-v1-uploader:   RE_XML_NAMESPACE = re.compile(b'^(<?[^>]+?>\s*|\s*)(<\w+) xmlns=[\'"](https?://[^\'"]+)[\'"]', re.MULTILINE)
chaiml-98p-2ff-chaiml-m-29161-v1-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:240: SyntaxWarning: invalid escape sequence '\.'
chaiml-98p-2ff-chaiml-m-29161-v1-uploader:   invalid = re.search("([^a-z0-9\.-])", bucket, re.UNICODE)
chaiml-98p-2ff-chaiml-m-29161-v1-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:244: SyntaxWarning: invalid escape sequence '\.'
chaiml-98p-2ff-chaiml-m-29161-v1-uploader:   invalid = re.search("([^A-Za-z0-9\._-])", bucket, re.UNICODE)
chaiml-98p-2ff-chaiml-m-29161-v1-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:255: SyntaxWarning: invalid escape sequence '\.'
chaiml-98p-2ff-chaiml-m-29161-v1-uploader:   if re.search("-\.", bucket, re.UNICODE):
chaiml-98p-2ff-chaiml-m-29161-v1-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:257: SyntaxWarning: invalid escape sequence '\.'
chaiml-98p-2ff-chaiml-m-29161-v1-uploader:   if re.search("\.\.", bucket, re.UNICODE):
chaiml-98p-2ff-chaiml-m-29161-v1-uploader: /usr/lib/python3/dist-packages/S3/S3Uri.py:155: SyntaxWarning: invalid escape sequence '\w'
chaiml-98p-2ff-chaiml-m-29161-v1-uploader:   _re = re.compile("^(\w+://)?(.*)", re.UNICODE)
chaiml-98p-2ff-chaiml-m-29161-v1-uploader: /usr/lib/python3/dist-packages/S3/FileLists.py:480: SyntaxWarning: invalid escape sequence '\*'
chaiml-98p-2ff-chaiml-m-29161-v1-uploader:   wildcard_split_result = re.split("\*|\?", uri_str, maxsplit=1)
chaiml-98p-2ff-chaiml-m-29161-v1-uploader: Bucket 's3://guanaco-vllm-models/' created
chaiml-98p-2ff-chaiml-m-29161-v1-uploader: uploading /dev/shm/model_output to s3://guanaco-vllm-models/chaiml-98p-2ff-chaiml-m-29161-v1/default
chaiml-98p-2ff-chaiml-m-29161-v1-uploader: cp /dev/shm/model_output/config.json s3://guanaco-vllm-models/chaiml-98p-2ff-chaiml-m-29161-v1/default/config.json
chaiml-98p-2ff-chaiml-m-29161-v1-uploader: cp /dev/shm/model_output/generation_config.json s3://guanaco-vllm-models/chaiml-98p-2ff-chaiml-m-29161-v1/default/generation_config.json
chaiml-98p-2ff-chaiml-m-29161-v1-uploader: cp /dev/shm/model_output/recipe.yaml s3://guanaco-vllm-models/chaiml-98p-2ff-chaiml-m-29161-v1/default/recipe.yaml
chaiml-98p-2ff-chaiml-m-29161-v1-uploader: cp /dev/shm/model_output/special_tokens_map.json s3://guanaco-vllm-models/chaiml-98p-2ff-chaiml-m-29161-v1/default/special_tokens_map.json
chaiml-98p-2ff-chaiml-m-29161-v1-uploader: cp /dev/shm/model_output/model.safetensors.index.json s3://guanaco-vllm-models/chaiml-98p-2ff-chaiml-m-29161-v1/default/model.safetensors.index.json
chaiml-98p-2ff-chaiml-m-29161-v1-uploader: cp /dev/shm/model_output/tokenizer_config.json s3://guanaco-vllm-models/chaiml-98p-2ff-chaiml-m-29161-v1/default/tokenizer_config.json
chaiml-98p-2ff-chaiml-m-29161-v1-uploader: cp /dev/shm/model_output/tokenizer.json s3://guanaco-vllm-models/chaiml-98p-2ff-chaiml-m-29161-v1/default/tokenizer.json
chaiml-98p-2ff-chaiml-m-29161-v1-uploader: cp /dev/shm/model_output/model-00006-of-00006.safetensors s3://guanaco-vllm-models/chaiml-98p-2ff-chaiml-m-29161-v1/default/model-00006-of-00006.safetensors
chaiml-98p-2ff-chaiml-m-29161-v1-uploader: cp /dev/shm/model_output/model-00005-of-00006.safetensors s3://guanaco-vllm-models/chaiml-98p-2ff-chaiml-m-29161-v1/default/model-00005-of-00006.safetensors
chaiml-98p-2ff-chaiml-m-29161-v1-uploader: cp /dev/shm/model_output/model-00004-of-00006.safetensors s3://guanaco-vllm-models/chaiml-98p-2ff-chaiml-m-29161-v1/default/model-00004-of-00006.safetensors
chaiml-98p-2ff-chaiml-m-29161-v1-uploader: cp /dev/shm/model_output/model-00001-of-00006.safetensors s3://guanaco-vllm-models/chaiml-98p-2ff-chaiml-m-29161-v1/default/model-00001-of-00006.safetensors
chaiml-98p-2ff-chaiml-m-29161-v1-uploader: cp /dev/shm/model_output/model-00003-of-00006.safetensors s3://guanaco-vllm-models/chaiml-98p-2ff-chaiml-m-29161-v1/default/model-00003-of-00006.safetensors
chaiml-98p-2ff-chaiml-m-29161-v1-uploader: cp /dev/shm/model_output/model-00002-of-00006.safetensors s3://guanaco-vllm-models/chaiml-98p-2ff-chaiml-m-29161-v1/default/model-00002-of-00006.safetensors
Job chaiml-98p-2ff-chaiml-m-29161-v1-uploader completed after 414.91s with status: succeeded
Stopping job with name chaiml-98p-2ff-chaiml-m-29161-v1-uploader
Pipeline stage VLLMUploader completed in 415.53s
run pipeline stage %s
Running pipeline stage VLLMTemplater
Pipeline stage VLLMTemplater completed in 0.15s
run pipeline stage %s
Running pipeline stage VLLMDeployer
Creating inference service chaiml-98p-2ff-chaiml-m-29161-v1
Waiting for inference service chaiml-98p-2ff-chaiml-m-29161-v1 to be ready
Inference service chaiml-98p-2ff-chaiml-m-29161-v1 ready after 191.51415038108826s
Pipeline stage VLLMDeployer completed in 192.09s
run pipeline stage %s
Running pipeline stage StressChecker
Received healthy response to inference request in 1.224484920501709s
Received healthy response to inference request in 1.1894361972808838s
Received healthy response to inference request in 1.5046885013580322s
Received healthy response to inference request in 1.393388032913208s
Received healthy response to inference request in 1.6118237972259521s
Received healthy response to inference request in 1.6012465953826904s
Received healthy response to inference request in 1.3732521533966064s
Received healthy response to inference request in 1.1715655326843262s
Received healthy response to inference request in 1.3863670825958252s
Received healthy response to inference request in 1.2726528644561768s
Received healthy response to inference request in 1.5056591033935547s
Received healthy response to inference request in 1.1864290237426758s
Received healthy response to inference request in 1.2939424514770508s
Received healthy response to inference request in 1.640803575515747s
Received healthy response to inference request in 1.1835060119628906s
Received healthy response to inference request in 1.1968145370483398s
Received healthy response to inference request in 1.5479795932769775s
Received healthy response to inference request in 1.24204421043396s
Received healthy response to inference request in 1.2832815647125244s
Received healthy response to inference request in 1.1853671073913574s
Received healthy response to inference request in 1.466562032699585s
Received healthy response to inference request in 1.1946368217468262s
Received healthy response to inference request in 1.3246345520019531s
Received healthy response to inference request in 1.2655856609344482s
Received healthy response to inference request in 1.1603584289550781s
Received healthy response to inference request in 1.426445722579956s
Received healthy response to inference request in 1.2142884731292725s
Received healthy response to inference request in 2.2798409461975098s
Received healthy response to inference request in 1.5119132995605469s
Received healthy response to inference request in 2.271026611328125s
30 requests
0 failed requests
5th percentile: 1.1769387483596803
10th percentile: 1.1851809978485108
20th percentile: 1.1935966968536378
30th percentile: 1.2214259862899781
40th percentile: 1.2698259830474854
50th percentile: 1.309288501739502
60th percentile: 1.3891754627227784
70th percentile: 1.477999973297119
80th percentile: 1.5191265583038331
90th percentile: 1.6147217750549316
95th percentile: 1.987426245212553
99th percentile: 2.277284789085388
mean time: 1.4036675135294596
Pipeline stage StressChecker completed in 45.51s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyTriggerPipeline
run_pipeline:run_in_cloud %s
starting trigger_guanaco_pipeline args=%s
triggered trigger_guanaco_pipeline args=%s
Pipeline stage OfflineFamilyFriendlyTriggerPipeline completed in 0.81s
Shutdown handler de-registered
chaiml-98p-2ff-chaiml-m_29161_v1 status is now deployed due to DeploymentManager action
chaiml-98p-2ff-chaiml-m_29161_v1 status is now inactive due to auto deactivation removed underperforming models