chaiml-mega-v1-plc-q27b_57593

developer_uid: zonemercy
submission_id: chaiml-mega-v1-plc-q27b_57593_v1
model_name: chaiml-mega-v1-plc-q27b_57593_v1
model_group: ChaiML/mega-v1-plc-q27b-
status: inactive
timestamp: 2026-03-28T10:17:37+00:00
num_battles: 10738
num_wins: 5603
celo_rating: 1309.44
family_friendly_score: 0.0
family_friendly_standard_error: 0.0
submission_type: basic
model_repo: ChaiML/mega-v1-plc-q27b-lr5e6ep2g8
model_architecture: Qwen3_5ForConditionalGeneration
model_num_parameters: 23564784640.0
best_of: 8
max_input_tokens: 2048
max_output_tokens: 80
reward_model: default
display_name: chaiml-mega-v1-plc-q27b_57593_v1
ineligible_reason: max_output_tokens!=64
is_internal_developer: True
language_model: ChaiML/mega-v1-plc-q27b-lr5e6ep2g8
model_size: 24B
ranking_group: single
us_pacific_date: 2026-03-28
win_ratio: 0.5217917675544794
generation_params: {'temperature': 1.0, 'top_p': 1.0, 'min_p': 0.0, 'top_k': 40, 'presence_penalty': 0.8, 'frequency_penalty': 0.0, 'stopping_words': ['<|assistant|>', '<|user|>', '</s>', '####', '<|im_end|>'], 'max_input_tokens': 2048, 'best_of': 8, 'max_output_tokens': 80}
formatter: {'memory_template': "<|im_start|>system\n{bot_name}'s persona: {memory}<|im_end|>\n", 'prompt_template': '', 'bot_template': '<|im_start|>assistant\n{bot_name}: {message}<|im_end|>\n', 'user_template': '<|im_start|>user\n{message}<|im_end|>\n', 'response_template': '<|im_start|>assistant\n{bot_name}:', 'truncate_by_message': True}
Resubmit model
Shutdown handler not registered because Python interpreter is not running in the main thread
run pipeline %s
run pipeline stage %s
Running pipeline stage VLLMUploader
Starting job with name chaiml-mega-v1-plc-q27b-57593-v1-uploader
Waiting for job on chaiml-mega-v1-plc-q27b-57593-v1-uploader to finish
chaiml-mega-v1-plc-q27b-57593-v1-uploader: Using quantization_mode: fp8
chaiml-mega-v1-plc-q27b-57593-v1-uploader: Checking if ChaiML/mega-v1-plc-q27b-lr5e6ep2g8-FP8 already exists in ChaiML
chaiml-mega-v1-plc-q27b-57593-v1-uploader: Downloading snapshot of ChaiML/mega-v1-plc-q27b-lr5e6ep2g8...
2026-03-28T07:07:01.637107+00:00 monitor updated for chaiml-mega-v1-plc-q27b_57593_v1
chaiml-mega-v1-plc-q27b-57593-v1-uploader: Downloaded in 51.856s
chaiml-mega-v1-plc-q27b-57593-v1-uploader: Loading /tmp/model_input...
chaiml-mega-v1-plc-q27b-57593-v1-uploader: The fast path is not available because one of the required library is not installed. Falling back to torch implementation. To install follow https://github.com/fla-org/flash-linear-attention#installation and https://github.com/Dao-AILab/causal-conv1d
chaiml-mega-v1-plc-q27b-57593-v1-uploader: 2026-03-28T07:07:45.528741+0000 | reset | INFO - Compression lifecycle reset
chaiml-mega-v1-plc-q27b-57593-v1-uploader: 2026-03-28T07:07:45.530938+0000 | from_modifiers | INFO - Creating recipe from modifiers
chaiml-mega-v1-plc-q27b-57593-v1-uploader: 2026-03-28T07:07:45.577339+0000 | initialize | INFO - Compression lifecycle initialized for 1 modifiers
chaiml-mega-v1-plc-q27b-57593-v1-uploader: 2026-03-28T07:07:45.577590+0000 | IndependentPipeline | INFO - Inferred `DataFreePipeline` for `QuantizationModifier`
chaiml-mega-v1-plc-q27b-57593-v1-uploader: 2026-03-28T07:07:45.590224+0000 | dispatch_model | WARNING - Forced to offload modules due to insufficient gpu resources
chaiml-mega-v1-plc-q27b-57593-v1-uploader: 2026-03-28T07:07:52.600199+0000 | finalize | INFO - Compression lifecycle finalized for 1 modifiers
chaiml-mega-v1-plc-q27b-57593-v1-uploader: 2026-03-28T07:07:52.600410+0000 | post_process | WARNING - Optimized model is not saved. To save, please provide`output_dir` as input arg.Ex. `oneshot(..., output_dir=...)`
chaiml-mega-v1-plc-q27b-57593-v1-uploader: Saving to /dev/shm/model_output...
2026-03-28T07:08:01.736630+00:00 monitor updated for chaiml-mega-v1-plc-q27b_57593_v1
chaiml-mega-v1-plc-q27b-57593-v1-uploader: /usr/local/lib/python3.12/dist-packages/transformers/modeling_utils.py:3344: UserWarning: Attempting to save a model with offloaded modules. Ensure that unallocated cpu memory exceeds the `shard_size` (50GB default)
chaiml-mega-v1-plc-q27b-57593-v1-uploader:   warnings.warn(
chaiml-mega-v1-plc-q27b-57593-v1-uploader: Cleaning quantization config in /dev/shm/model_output
chaiml-mega-v1-plc-q27b-57593-v1-uploader: Pushing to ChaiML/mega-v1-plc-q27b-lr5e6ep2g8-FP8
chaiml-mega-v1-plc-q27b-57593-v1-uploader: Checking if ChaiML/mega-v1-plc-q27b-lr5e6ep2g8-FP8 already exists in ChaiML
chaiml-mega-v1-plc-q27b-57593-v1-uploader: Creating repo ChaiML/mega-v1-plc-q27b-lr5e6ep2g8-FP8 and uploading /dev/shm/model_output to it
chaiml-mega-v1-plc-q27b-57593-v1-uploader: Found 1 files larger than 20GB (recommended limit):
chaiml-mega-v1-plc-q27b-57593-v1-uploader:   - model.safetensors: 35.9GB
chaiml-mega-v1-plc-q27b-57593-v1-uploader: Large files may slow down loading and processing.
chaiml-mega-v1-plc-q27b-57593-v1-uploader: ---------- 2026-03-28 07:08:43 (0:00:00) ----------
chaiml-mega-v1-plc-q27b-57593-v1-uploader: Files:   hashed 5/7 (34.1K/35.9G) | pre-uploaded: 0/0 (0.0/35.9G) (+7 unsure) | committed: 0/7 (0.0/35.9G) | ignored: 0
chaiml-mega-v1-plc-q27b-57593-v1-uploader: Workers: hashing: 2 | get upload mode: 5 | pre-uploading: 0 | committing: 0 | waiting: 57
chaiml-mega-v1-plc-q27b-57593-v1-uploader: ---------------------------------------------------
2026-03-28T07:09:01.831570+00:00 monitor updated for chaiml-mega-v1-plc-q27b_57593_v1
chaiml-mega-v1-plc-q27b-57593-v1-uploader: 
[K[F
[K[F
[K[F
[K[F
[K[F
[K[F
[K[F
chaiml-mega-v1-plc-q27b-57593-v1-uploader: ---------- 2026-03-28 07:09:43 (0:01:00) ----------
chaiml-mega-v1-plc-q27b-57593-v1-uploader: Files:   hashed 7/7 (35.9G/35.9G) | pre-uploaded: 1/2 (20.0M/35.9G) | committed: 0/7 (0.0/35.9G) | ignored: 0
chaiml-mega-v1-plc-q27b-57593-v1-uploader: Workers: hashing: 0 | get upload mode: 0 | pre-uploading: 1 | committing: 0 | waiting: 63
chaiml-mega-v1-plc-q27b-57593-v1-uploader: ---------------------------------------------------
2026-03-28T07:10:01.914893+00:00 monitor updated for chaiml-mega-v1-plc-q27b_57593_v1
chaiml-mega-v1-plc-q27b-57593-v1-uploader:                              Processed model ChaiML/mega-v1-plc-q27b-lr5e6ep2g8 in 226.905s
chaiml-mega-v1-plc-q27b-57593-v1-uploader: creating bucket guanaco-vllm-models
chaiml-mega-v1-plc-q27b-57593-v1-uploader: /usr/lib/python3/dist-packages/S3/BaseUtils.py:56: SyntaxWarning: invalid escape sequence '\.'
chaiml-mega-v1-plc-q27b-57593-v1-uploader:   RE_S3_DATESTRING = re.compile('\.[0-9]*(?:[Z\\-\\+]*?)')
chaiml-mega-v1-plc-q27b-57593-v1-uploader: /usr/lib/python3/dist-packages/S3/BaseUtils.py:57: SyntaxWarning: invalid escape sequence '\s'
chaiml-mega-v1-plc-q27b-57593-v1-uploader:   RE_XML_NAMESPACE = re.compile(b'^(<?[^>]+?>\s*|\s*)(<\w+) xmlns=[\'"](https?://[^\'"]+)[\'"]', re.MULTILINE)
chaiml-mega-v1-plc-q27b-57593-v1-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:240: SyntaxWarning: invalid escape sequence '\.'
chaiml-mega-v1-plc-q27b-57593-v1-uploader:   invalid = re.search("([^a-z0-9\.-])", bucket, re.UNICODE)
chaiml-mega-v1-plc-q27b-57593-v1-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:244: SyntaxWarning: invalid escape sequence '\.'
chaiml-mega-v1-plc-q27b-57593-v1-uploader:   invalid = re.search("([^A-Za-z0-9\._-])", bucket, re.UNICODE)
chaiml-mega-v1-plc-q27b-57593-v1-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:255: SyntaxWarning: invalid escape sequence '\.'
chaiml-mega-v1-plc-q27b-57593-v1-uploader:   if re.search("-\.", bucket, re.UNICODE):
chaiml-mega-v1-plc-q27b-57593-v1-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:257: SyntaxWarning: invalid escape sequence '\.'
chaiml-mega-v1-plc-q27b-57593-v1-uploader:   if re.search("\.\.", bucket, re.UNICODE):
chaiml-mega-v1-plc-q27b-57593-v1-uploader: /usr/lib/python3/dist-packages/S3/S3Uri.py:155: SyntaxWarning: invalid escape sequence '\w'
chaiml-mega-v1-plc-q27b-57593-v1-uploader:   _re = re.compile("^(\w+://)?(.*)", re.UNICODE)
chaiml-mega-v1-plc-q27b-57593-v1-uploader: /usr/lib/python3/dist-packages/S3/FileLists.py:480: SyntaxWarning: invalid escape sequence '\*'
chaiml-mega-v1-plc-q27b-57593-v1-uploader:   wildcard_split_result = re.split("\*|\?", uri_str, maxsplit=1)
chaiml-mega-v1-plc-q27b-57593-v1-uploader: Bucket 's3://guanaco-vllm-models/' created
chaiml-mega-v1-plc-q27b-57593-v1-uploader: uploading /dev/shm/model_output to s3://guanaco-vllm-models/chaiml-mega-v1-plc-q27b-57593-v1/default
chaiml-mega-v1-plc-q27b-57593-v1-uploader: cp /dev/shm/model_output/generation_config.json s3://guanaco-vllm-models/chaiml-mega-v1-plc-q27b-57593-v1/default/generation_config.json
chaiml-mega-v1-plc-q27b-57593-v1-uploader: cp /dev/shm/model_output/recipe.yaml s3://guanaco-vllm-models/chaiml-mega-v1-plc-q27b-57593-v1/default/recipe.yaml
chaiml-mega-v1-plc-q27b-57593-v1-uploader: cp /dev/shm/model_output/config.json s3://guanaco-vllm-models/chaiml-mega-v1-plc-q27b-57593-v1/default/config.json
chaiml-mega-v1-plc-q27b-57593-v1-uploader: cp /dev/shm/model_output/chat_template.jinja s3://guanaco-vllm-models/chaiml-mega-v1-plc-q27b-57593-v1/default/chat_template.jinja
chaiml-mega-v1-plc-q27b-57593-v1-uploader: cp /dev/shm/model_output/tokenizer_config.json s3://guanaco-vllm-models/chaiml-mega-v1-plc-q27b-57593-v1/default/tokenizer_config.json
chaiml-mega-v1-plc-q27b-57593-v1-uploader: cp /dev/shm/model_output/tokenizer.json s3://guanaco-vllm-models/chaiml-mega-v1-plc-q27b-57593-v1/default/tokenizer.json
2026-03-28T07:11:01.999640+00:00 monitor updated for chaiml-mega-v1-plc-q27b_57593_v1
chaiml-mega-v1-plc-q27b-57593-v1-uploader: cp /dev/shm/model_output/model.safetensors s3://guanaco-vllm-models/chaiml-mega-v1-plc-q27b-57593-v1/default/model.safetensors
Job chaiml-mega-v1-plc-q27b-57593-v1-uploader completed after 338.26s with status: succeeded
Stopping job with name chaiml-mega-v1-plc-q27b-57593-v1-uploader
Pipeline stage VLLMUploader completed in 338.76s
run pipeline stage %s
Running pipeline stage VLLMUploaderAMD
Pipeline stage vllm_upload_amd skipped, reason=not amd cluster
Pipeline stage VLLMUploaderAMD completed in 0.10s
run pipeline stage %s
Running pipeline stage VLLMTemplater
Pipeline stage VLLMTemplater completed in 3.52s
run pipeline stage %s
Running pipeline stage VLLMDeployer
Creating inference service chaiml-mega-v1-plc-q27b-57593-v1
Waiting for inference service chaiml-mega-v1-plc-q27b-57593-v1 to be ready
2026-03-28T07:12:02.505600+00:00 monitor updated for chaiml-mega-v1-plc-q27b_57593_v1
2026-03-28T07:13:02.636793+00:00 monitor updated for chaiml-mega-v1-plc-q27b_57593_v1
2026-03-28T07:14:03.246659+00:00 monitor updated for chaiml-mega-v1-plc-q27b_57593_v1
Inference service chaiml-mega-v1-plc-q27b-57593-v1 ready after 170.60162162780762s
Pipeline stage VLLMDeployer completed in 171.05s
run pipeline stage %s
Running pipeline stage StressChecker
HTTPConnectionPool(host='guanaco-submitter-v2.guanaco-backend.kchai-google-us-east4.chaiverse.com', port=80): Read timed out. (read timeout=20)
Received unhealthy response to inference request!
2026-03-28T07:15:03.362583+00:00 monitor updated for chaiml-mega-v1-plc-q27b_57593_v1
HTTPConnectionPool(host='guanaco-submitter-v2.guanaco-backend.kchai-google-us-east4.chaiverse.com', port=80): Read timed out. (read timeout=20)
Received unhealthy response to inference request!
HTTPConnectionPool(host='guanaco-submitter-v2.guanaco-backend.kchai-google-us-east4.chaiverse.com', port=80): Read timed out. (read timeout=20)
Received unhealthy response to inference request!
HTTPConnectionPool(host='guanaco-submitter-v2.guanaco-backend.kchai-google-us-east4.chaiverse.com', port=80): Read timed out. (read timeout=20)
Received unhealthy response to inference request!
2026-03-28T07:16:03.487537+00:00 monitor updated for chaiml-mega-v1-plc-q27b_57593_v1
Received healthy response to inference request in 11.75775408744812s
Received healthy response to inference request in 5.180154800415039s
Received healthy response to inference request in 2.094433069229126s
HTTPConnectionPool(host='guanaco-submitter-v2.guanaco-backend.kchai-google-us-east4.chaiverse.com', port=80): Read timed out. (read timeout=20)
Received unhealthy response to inference request!
Received healthy response to inference request in 1.8045661449432373s
Received healthy response to inference request in 1.767749309539795s
Received healthy response to inference request in 1.992945671081543s
Received healthy response to inference request in 2.020677089691162s
Received healthy response to inference request in 1.940087080001831s
2026-03-28T07:17:03.659567+00:00 monitor updated for chaiml-mega-v1-plc-q27b_57593_v1
HTTPConnectionPool(host='guanaco-submitter-v2.guanaco-backend.kchai-google-us-east4.chaiverse.com', port=80): Read timed out. (read timeout=20)
Received unhealthy response to inference request!
Received healthy response to inference request in 4.188542604446411s
HTTPConnectionPool(host='guanaco-submitter-v2.guanaco-backend.kchai-google-us-east4.chaiverse.com', port=80): Read timed out. (read timeout=20)
Received unhealthy response to inference request!
Received healthy response to inference request in 1.8581516742706299s
HTTPConnectionPool(host='guanaco-submitter-v2.guanaco-backend.kchai-google-us-east4.chaiverse.com', port=80): Read timed out. (read timeout=20)
Received unhealthy response to inference request!
Received healthy response to inference request in 1.9751510620117188s
Received healthy response to inference request in 2.0704524517059326s
Received healthy response to inference request in 2.0976178646087646s
Received healthy response to inference request in 4.421795606613159s
2026-03-28T07:18:03.762438+00:00 monitor updated for chaiml-mega-v1-plc-q27b_57593_v1
Received healthy response to inference request in 2.161879539489746s
Received healthy response to inference request in 1.9379215240478516s
Received healthy response to inference request in 2.1221721172332764s
Received healthy response to inference request in 2.0430965423583984s
Received healthy response to inference request in 2.1951146125793457s
Received healthy response to inference request in 2.134125232696533s
Received healthy response to inference request in 2.1096246242523193s
Received healthy response to inference request in 1.8644113540649414s
30 requests
8 failed requests
5th percentile: 1.828679633140564
10th percentile: 1.8637853860855103
20th percentile: 1.9681382656097413
30th percentile: 2.0363707065582277
40th percentile: 2.0963439464569094
50th percentile: 2.128148674964905
60th percentile: 2.992485809326169
70th percentile: 7.153434586524945
80th percentile: 20.13339309692383
90th percentile: 20.16819758415222
95th percentile: 20.214889144897462
99th percentile: 20.537996029853822
mean time: 7.451288874944051
%s, retrying in %s seconds...
Received healthy response to inference request in 1.6765589714050293s
Received healthy response to inference request in 1.8307609558105469s
Received healthy response to inference request in 1.6251318454742432s
Received healthy response to inference request in 2.525308132171631s
Received healthy response to inference request in 1.779815912246704s
Received healthy response to inference request in 1.9467592239379883s
Received healthy response to inference request in 1.8352746963500977s
Received healthy response to inference request in 1.7920174598693848s
Received healthy response to inference request in 1.587655782699585s
Received healthy response to inference request in 1.856109857559204s
Received healthy response to inference request in 1.6388487815856934s
Received healthy response to inference request in 2.007472515106201s
Received healthy response to inference request in 1.882007122039795s
Received healthy response to inference request in 1.9667515754699707s
Received healthy response to inference request in 1.8370416164398193s
Received healthy response to inference request in 2.203482151031494s
Received healthy response to inference request in 1.8207805156707764s
Received healthy response to inference request in 1.922576904296875s
Received healthy response to inference request in 1.9408655166625977s
Received healthy response to inference request in 1.9347965717315674s
2026-03-28T07:19:03.890220+00:00 monitor updated for chaiml-mega-v1-plc-q27b_57593_v1
Received healthy response to inference request in 1.9494082927703857s
Received healthy response to inference request in 2.008561134338379s
Received healthy response to inference request in 1.8996496200561523s
Received healthy response to inference request in 1.9704158306121826s
Received healthy response to inference request in 1.9480361938476562s
Received healthy response to inference request in 1.982882022857666s
Received healthy response to inference request in 2.269390106201172s
Received healthy response to inference request in 2.1659135818481445s
Received healthy response to inference request in 2.266937017440796s
Received healthy response to inference request in 2.0141780376434326s
30 requests
0 failed requests
5th percentile: 1.6313044667243957
10th percentile: 1.6727879524230957
20th percentile: 1.815027904510498
30th percentile: 1.8365115404129029
40th percentile: 1.8925926208496093
50th percentile: 1.9378310441970825
60th percentile: 1.948585033416748
70th percentile: 1.9741556882858275
80th percentile: 2.00968451499939
90th percentile: 2.2098276376724244
95th percentile: 2.2682862162590025
99th percentile: 2.4510919046401978
mean time: 1.9361795981725056
Pipeline stage StressChecker completed in 289.40s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyTriggerPipeline
run_pipeline:run_in_cloud %s
starting trigger_guanaco_pipeline args=%s
triggered trigger_guanaco_pipeline args=%s
Pipeline stage OfflineFamilyFriendlyTriggerPipeline completed in 0.79s
Shutdown handler de-registered
chaiml-mega-v1-plc-q27b_57593_v1 status is now deployed due to DeploymentManager action
chaiml-mega-v1-plc-q27b_57593_v1 status is now inactive due to auto deactivation removed underperforming models