chaiml-muster-v0e-lr1e5_71769

developer_uid: zonemercy
submission_id: chaiml-muster-v0e-lr1e5_71769_v2
model_name: chaiml-muster-v0e-lr1e5_71769_v2
model_group: ChaiML/muster-v0e-lr1e5e
status: torndown
timestamp: 2026-03-04T18:01:29+00:00
num_battles: 12185
num_wins: 7016
celo_rating: 1358.12
family_friendly_score: 0.0
family_friendly_standard_error: 0.0
submission_type: basic
model_repo: ChaiML/muster-v0e-lr1e5ep2r64g4b01
model_architecture: Qwen3MoeForCausalLM
model_num_parameters: 18790207488.0
best_of: 8
max_input_tokens: 2048
max_output_tokens: 80
reward_model: default
display_name: chaiml-muster-v0e-lr1e5_71769_v2
ineligible_reason: max_output_tokens!=64
is_internal_developer: True
language_model: ChaiML/muster-v0e-lr1e5ep2r64g4b01
model_size: 19B
ranking_group: single
us_pacific_date: 2026-02-09
win_ratio: 0.575789905621666
generation_params: {'temperature': 1.0, 'top_p': 1.0, 'min_p': 0.0, 'top_k': 40, 'presence_penalty': 0.0, 'frequency_penalty': 0.0, 'stopping_words': ['<|assistant|>', '<|user|>', '<|im_end|>', '</s>', '####', '</think>'], 'max_input_tokens': 2048, 'best_of': 8, 'max_output_tokens': 80}
formatter: {'memory_template': "<|im_start|>system\n{bot_name}'s persona: {memory}<|im_end|>\n", 'prompt_template': '', 'bot_template': '<|im_start|>assistant\n{bot_name}: {message}<|im_end|>\n', 'user_template': '<|im_start|>user\n{message}<|im_end|>\n', 'response_template': '<|im_start|>assistant\n{bot_name}:', 'truncate_by_message': True}
Resubmit model
Shutdown handler not registered because Python interpreter is not running in the main thread
run pipeline %s
run pipeline stage %s
Running pipeline stage VLLMUploader
Starting job with name chaiml-muster-v0e-lr1e5-71769-v2-uploader
Waiting for job on chaiml-muster-v0e-lr1e5-71769-v2-uploader to finish
chaiml-muster-v0e-lr1e5-71769-v2-uploader: Using quantization_mode: w4a16
chaiml-muster-v0e-lr1e5-71769-v2-uploader: Checking if ChaiML/muster-v0e-lr1e5ep2r64g4b01-W4A16 already exists in ChaiML
chaiml-muster-v0e-lr1e5-71769-v2-uploader: Model already exists. Downloading to /dev/shm/model_output...
chaiml-muster-v0e-lr1e5-71769-v2-uploader: Downloading snapshot of ChaiML/muster-v0e-lr1e5ep2r64g4b01-W4A16...
chaiml-muster-v0e-lr1e5-71769-v2-uploader: 
Fetching 39 files:   0%|          | 0/39 [00:00<?, ?it/s]
Fetching 39 files:   3%|▎         | 1/39 [00:00<00:14,  2.55it/s]
Fetching 39 files:  18%|█▊        | 7/39 [00:16<01:19,  2.47s/it]
Fetching 39 files:  21%|██        | 8/39 [00:18<01:14,  2.42s/it]
Fetching 39 files:  38%|███▊      | 15/39 [00:29<00:44,  1.86s/it]
Fetching 39 files:  41%|████      | 16/39 [00:32<00:45,  2.00s/it]
Fetching 39 files:  46%|████▌     | 18/39 [00:33<00:34,  1.63s/it]
Fetching 39 files:  49%|████▊     | 19/39 [00:35<00:34,  1.73s/it]
Fetching 39 files:  51%|█████▏    | 20/39 [00:36<00:30,  1.59s/it]
Fetching 39 files:  54%|█████▍    | 21/39 [00:37<00:24,  1.34s/it]
Fetching 39 files:  59%|█████▉    | 23/39 [00:45<00:38,  2.42s/it]
Fetching 39 files:  62%|██████▏   | 24/39 [00:50<00:42,  2.84s/it]
Fetching 39 files:  69%|██████▉   | 27/39 [00:52<00:21,  1.79s/it]
Fetching 39 files:  72%|███████▏  | 28/39 [00:54<00:20,  1.87s/it]
Fetching 39 files:  77%|███████▋  | 30/39 [00:54<00:11,  1.26s/it]
Fetching 39 files:  79%|███████▉  | 31/39 [00:56<00:11,  1.38s/it]
Fetching 39 files:  82%|████████▏ | 32/39 [00:57<00:09,  1.39s/it]
Fetching 39 files: 100%|██████████| 39/39 [00:57<00:00,  1.48s/it]
chaiml-muster-v0e-lr1e5-71769-v2-uploader: Downloaded in 57.899s
chaiml-muster-v0e-lr1e5-71769-v2-uploader: Processed model ChaiML/muster-v0e-lr1e5ep2r64g4b01 in 58.442s
chaiml-muster-v0e-lr1e5-71769-v2-uploader: creating bucket guanaco-vllm-models
chaiml-muster-v0e-lr1e5-71769-v2-uploader: /usr/lib/python3/dist-packages/S3/BaseUtils.py:56: SyntaxWarning: invalid escape sequence '\.'
chaiml-muster-v0e-lr1e5-71769-v2-uploader:   RE_S3_DATESTRING = re.compile('\.[0-9]*(?:[Z\\-\\+]*?)')
chaiml-muster-v0e-lr1e5-71769-v2-uploader: /usr/lib/python3/dist-packages/S3/BaseUtils.py:57: SyntaxWarning: invalid escape sequence '\s'
chaiml-muster-v0e-lr1e5-71769-v2-uploader:   RE_XML_NAMESPACE = re.compile(b'^(<?[^>]+?>\s*|\s*)(<\w+) xmlns=[\'"](https?://[^\'"]+)[\'"]', re.MULTILINE)
chaiml-muster-v0e-lr1e5-71769-v2-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:240: SyntaxWarning: invalid escape sequence '\.'
chaiml-muster-v0e-lr1e5-71769-v2-uploader:   invalid = re.search("([^a-z0-9\.-])", bucket, re.UNICODE)
chaiml-muster-v0e-lr1e5-71769-v2-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:244: SyntaxWarning: invalid escape sequence '\.'
chaiml-muster-v0e-lr1e5-71769-v2-uploader:   invalid = re.search("([^A-Za-z0-9\._-])", bucket, re.UNICODE)
chaiml-muster-v0e-lr1e5-71769-v2-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:255: SyntaxWarning: invalid escape sequence '\.'
chaiml-muster-v0e-lr1e5-71769-v2-uploader:   if re.search("-\.", bucket, re.UNICODE):
chaiml-muster-v0e-lr1e5-71769-v2-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:257: SyntaxWarning: invalid escape sequence '\.'
chaiml-muster-v0e-lr1e5-71769-v2-uploader:   if re.search("\.\.", bucket, re.UNICODE):
chaiml-muster-v0e-lr1e5-71769-v2-uploader: /usr/lib/python3/dist-packages/S3/S3Uri.py:155: SyntaxWarning: invalid escape sequence '\w'
chaiml-muster-v0e-lr1e5-71769-v2-uploader:   _re = re.compile("^(\w+://)?(.*)", re.UNICODE)
chaiml-muster-v0e-lr1e5-71769-v2-uploader: /usr/lib/python3/dist-packages/S3/FileLists.py:480: SyntaxWarning: invalid escape sequence '\*'
chaiml-muster-v0e-lr1e5-71769-v2-uploader:   wildcard_split_result = re.split("\*|\?", uri_str, maxsplit=1)
chaiml-muster-v0e-lr1e5-71769-v2-uploader: Bucket 's3://guanaco-vllm-models/' created
chaiml-muster-v0e-lr1e5-71769-v2-uploader: uploading /dev/shm/model_output to s3://guanaco-vllm-models/chaiml-muster-v0e-lr1e5-71769-v2
chaiml-muster-v0e-lr1e5-71769-v2-uploader: cp /dev/shm/model_output/added_tokens.json s3://guanaco-vllm-models/chaiml-muster-v0e-lr1e5-71769-v2/added_tokens.json
chaiml-muster-v0e-lr1e5-71769-v2-uploader: cp /dev/shm/model_output/generation_config.json s3://guanaco-vllm-models/chaiml-muster-v0e-lr1e5-71769-v2/generation_config.json
chaiml-muster-v0e-lr1e5-71769-v2-uploader: cp /dev/shm/model_output/chat_template.jinja s3://guanaco-vllm-models/chaiml-muster-v0e-lr1e5-71769-v2/chat_template.jinja
chaiml-muster-v0e-lr1e5-71769-v2-uploader: cp /dev/shm/model_output/.gitattributes s3://guanaco-vllm-models/chaiml-muster-v0e-lr1e5-71769-v2/.gitattributes
chaiml-muster-v0e-lr1e5-71769-v2-uploader: cp /dev/shm/model_output/special_tokens_map.json s3://guanaco-vllm-models/chaiml-muster-v0e-lr1e5-71769-v2/special_tokens_map.json
chaiml-muster-v0e-lr1e5-71769-v2-uploader: cp /dev/shm/model_output/tokenizer_config.json s3://guanaco-vllm-models/chaiml-muster-v0e-lr1e5-71769-v2/tokenizer_config.json
chaiml-muster-v0e-lr1e5-71769-v2-uploader: cp /dev/shm/model_output/config.json s3://guanaco-vllm-models/chaiml-muster-v0e-lr1e5-71769-v2/config.json
chaiml-muster-v0e-lr1e5-71769-v2-uploader: cp /dev/shm/model_output/quantization_config.json s3://guanaco-vllm-models/chaiml-muster-v0e-lr1e5-71769-v2/quantization_config.json
chaiml-muster-v0e-lr1e5-71769-v2-uploader: cp /dev/shm/model_output/merges.txt s3://guanaco-vllm-models/chaiml-muster-v0e-lr1e5-71769-v2/merges.txt
chaiml-muster-v0e-lr1e5-71769-v2-uploader: cp /dev/shm/model_output/vocab.json s3://guanaco-vllm-models/chaiml-muster-v0e-lr1e5-71769-v2/vocab.json
chaiml-muster-v0e-lr1e5-71769-v2-uploader: cp /dev/shm/model_output/model.safetensors.index.json s3://guanaco-vllm-models/chaiml-muster-v0e-lr1e5-71769-v2/model.safetensors.index.json
chaiml-muster-v0e-lr1e5-71769-v2-uploader: cp /dev/shm/model_output/tokenizer.json s3://guanaco-vllm-models/chaiml-muster-v0e-lr1e5-71769-v2/tokenizer.json
HTTP Request: %s %s "%s %d %s"
chaiml-muster-v0e-lr1e5-71769-v2-uploader: cp /dev/shm/model_output/model-00027-of-00027.safetensors s3://guanaco-vllm-models/chaiml-muster-v0e-lr1e5-71769-v2/model-00027-of-00027.safetensors
chaiml-muster-v0e-lr1e5-71769-v2-uploader: cp /dev/shm/model_output/model-00005-of-00027.safetensors s3://guanaco-vllm-models/chaiml-muster-v0e-lr1e5-71769-v2/model-00005-of-00027.safetensors
chaiml-muster-v0e-lr1e5-71769-v2-uploader: cp /dev/shm/model_output/model-00010-of-00027.safetensors s3://guanaco-vllm-models/chaiml-muster-v0e-lr1e5-71769-v2/model-00010-of-00027.safetensors
chaiml-muster-v0e-lr1e5-71769-v2-uploader: cp /dev/shm/model_output/model-00017-of-00027.safetensors s3://guanaco-vllm-models/chaiml-muster-v0e-lr1e5-71769-v2/model-00017-of-00027.safetensors
chaiml-muster-v0e-lr1e5-71769-v2-uploader: cp /dev/shm/model_output/model-00023-of-00027.safetensors s3://guanaco-vllm-models/chaiml-muster-v0e-lr1e5-71769-v2/model-00023-of-00027.safetensors
chaiml-muster-v0e-lr1e5-71769-v2-uploader: cp /dev/shm/model_output/model-00013-of-00027.safetensors s3://guanaco-vllm-models/chaiml-muster-v0e-lr1e5-71769-v2/model-00013-of-00027.safetensors
chaiml-muster-v0e-lr1e5-71769-v2-uploader: cp /dev/shm/model_output/model-00009-of-00027.safetensors s3://guanaco-vllm-models/chaiml-muster-v0e-lr1e5-71769-v2/model-00009-of-00027.safetensors
chaiml-muster-v0e-lr1e5-71769-v2-uploader: cp /dev/shm/model_output/model-00011-of-00027.safetensors s3://guanaco-vllm-models/chaiml-muster-v0e-lr1e5-71769-v2/model-00011-of-00027.safetensors
chaiml-muster-v0e-lr1e5-71769-v2-uploader: cp /dev/shm/model_output/model-00008-of-00027.safetensors s3://guanaco-vllm-models/chaiml-muster-v0e-lr1e5-71769-v2/model-00008-of-00027.safetensors
chaiml-muster-v0e-lr1e5-71769-v2-uploader: cp /dev/shm/model_output/model-00016-of-00027.safetensors s3://guanaco-vllm-models/chaiml-muster-v0e-lr1e5-71769-v2/model-00016-of-00027.safetensors
chaiml-muster-v0e-lr1e5-71769-v2-uploader: cp /dev/shm/model_output/model-00018-of-00027.safetensors s3://guanaco-vllm-models/chaiml-muster-v0e-lr1e5-71769-v2/model-00018-of-00027.safetensors
chaiml-muster-v0e-lr1e5-71769-v2-uploader: cp /dev/shm/model_output/model-00001-of-00027.safetensors s3://guanaco-vllm-models/chaiml-muster-v0e-lr1e5-71769-v2/model-00001-of-00027.safetensors
chaiml-muster-v0e-lr1e5-71769-v2-uploader: cp /dev/shm/model_output/model-00012-of-00027.safetensors s3://guanaco-vllm-models/chaiml-muster-v0e-lr1e5-71769-v2/model-00012-of-00027.safetensors
chaiml-muster-v0e-lr1e5-71769-v2-uploader: cp /dev/shm/model_output/model-00003-of-00027.safetensors s3://guanaco-vllm-models/chaiml-muster-v0e-lr1e5-71769-v2/model-00003-of-00027.safetensors
chaiml-muster-v0e-lr1e5-71769-v2-uploader: cp /dev/shm/model_output/model-00026-of-00027.safetensors s3://guanaco-vllm-models/chaiml-muster-v0e-lr1e5-71769-v2/model-00026-of-00027.safetensors
chaiml-muster-v0e-lr1e5-71769-v2-uploader: cp /dev/shm/model_output/model-00015-of-00027.safetensors s3://guanaco-vllm-models/chaiml-muster-v0e-lr1e5-71769-v2/model-00015-of-00027.safetensors
chaiml-muster-v0e-lr1e5-71769-v2-uploader: cp /dev/shm/model_output/model-00022-of-00027.safetensors s3://guanaco-vllm-models/chaiml-muster-v0e-lr1e5-71769-v2/model-00022-of-00027.safetensors
chaiml-muster-v0e-lr1e5-71769-v2-uploader: cp /dev/shm/model_output/model-00007-of-00027.safetensors s3://guanaco-vllm-models/chaiml-muster-v0e-lr1e5-71769-v2/model-00007-of-00027.safetensors
chaiml-muster-v0e-lr1e5-71769-v2-uploader: cp /dev/shm/model_output/model-00004-of-00027.safetensors s3://guanaco-vllm-models/chaiml-muster-v0e-lr1e5-71769-v2/model-00004-of-00027.safetensors
chaiml-muster-v0e-lr1e5-71769-v2-uploader: cp /dev/shm/model_output/model-00019-of-00027.safetensors s3://guanaco-vllm-models/chaiml-muster-v0e-lr1e5-71769-v2/model-00019-of-00027.safetensors
chaiml-muster-v0e-lr1e5-71769-v2-uploader: cp /dev/shm/model_output/model-00006-of-00027.safetensors s3://guanaco-vllm-models/chaiml-muster-v0e-lr1e5-71769-v2/model-00006-of-00027.safetensors
chaiml-muster-v0e-lr1e5-71769-v2-uploader: cp /dev/shm/model_output/model-00024-of-00027.safetensors s3://guanaco-vllm-models/chaiml-muster-v0e-lr1e5-71769-v2/model-00024-of-00027.safetensors
chaiml-muster-v0e-lr1e5-71769-v2-uploader: cp /dev/shm/model_output/model-00002-of-00027.safetensors s3://guanaco-vllm-models/chaiml-muster-v0e-lr1e5-71769-v2/model-00002-of-00027.safetensors
chaiml-muster-v0e-lr1e5-71769-v2-uploader: cp /dev/shm/model_output/model-00020-of-00027.safetensors s3://guanaco-vllm-models/chaiml-muster-v0e-lr1e5-71769-v2/model-00020-of-00027.safetensors
chaiml-muster-v0e-lr1e5-71769-v2-uploader: cp /dev/shm/model_output/model-00021-of-00027.safetensors s3://guanaco-vllm-models/chaiml-muster-v0e-lr1e5-71769-v2/model-00021-of-00027.safetensors
chaiml-muster-v0e-lr1e5-71769-v2-uploader: cp /dev/shm/model_output/model-00014-of-00027.safetensors s3://guanaco-vllm-models/chaiml-muster-v0e-lr1e5-71769-v2/model-00014-of-00027.safetensors
chaiml-muster-v0e-lr1e5-71769-v2-uploader: cp /dev/shm/model_output/model-00025-of-00027.safetensors s3://guanaco-vllm-models/chaiml-muster-v0e-lr1e5-71769-v2/model-00025-of-00027.safetensors
Job chaiml-muster-v0e-lr1e5-71769-v2-uploader completed after 855.63s with status: succeeded
Stopping job with name chaiml-muster-v0e-lr1e5-71769-v2-uploader
Pipeline stage VLLMUploader completed in 856.03s
run pipeline stage %s
Running pipeline stage VLLMTemplater
Pipeline stage VLLMTemplater completed in 0.15s
run pipeline stage %s
Running pipeline stage VLLMDeployer
Creating inference service chaiml-muster-v0e-lr1e5-71769-v2
Waiting for inference service chaiml-muster-v0e-lr1e5-71769-v2 to be ready
HTTP Request: %s %s "%s %d %s"
HTTP Request: %s %s "%s %d %s"
HTTP Request: %s %s "%s %d %s"
HTTP Request: %s %s "%s %d %s"
HTTP Request: %s %s "%s %d %s"
HTTP Request: %s %s "%s %d %s"
HTTP Request: %s %s "%s %d %s"
HTTP Request: %s %s "%s %d %s"
HTTP Request: %s %s "%s %d %s"
Inference service chaiml-muster-v0e-lr1e5-71769-v2 ready after 941.6249334812164s
Pipeline stage VLLMDeployer completed in 942.25s
run pipeline stage %s
Running pipeline stage StressChecker
Received healthy response to inference request in 2.1741864681243896s
Received healthy response to inference request in 2.464689016342163s
Received healthy response to inference request in 1.916062831878662s
Received healthy response to inference request in 1.9669628143310547s
Received healthy response to inference request in 2.215820789337158s
Received healthy response to inference request in 2.040661334991455s
Received healthy response to inference request in 2.0184946060180664s
Received healthy response to inference request in 1.9627838134765625s
Received healthy response to inference request in 2.1017134189605713s
Received healthy response to inference request in 1.9645276069641113s
Received healthy response to inference request in 1.9425263404846191s
Received healthy response to inference request in 1.963984489440918s
Received healthy response to inference request in 1.9606971740722656s
Received healthy response to inference request in 1.956573247909546s
Received healthy response to inference request in 2.203355312347412s
Received healthy response to inference request in 2.0608811378479004s
Received healthy response to inference request in 2.2464957237243652s
Received healthy response to inference request in 1.9105713367462158s
Received healthy response to inference request in 2.1516032218933105s
Received healthy response to inference request in 2.036818027496338s
Received healthy response to inference request in 1.9937667846679688s
Received healthy response to inference request in 1.9270362854003906s
Received healthy response to inference request in 1.8943254947662354s
Received healthy response to inference request in 2.415818929672241s
Received healthy response to inference request in 2.3268051147460938s
Received healthy response to inference request in 2.1059703826904297s
Received healthy response to inference request in 2.0890772342681885s
Received healthy response to inference request in 1.9131953716278076s
Received healthy response to inference request in 1.9161996841430664s
Received healthy response to inference request in 2.0444533824920654s
30 requests
0 failed requests
5th percentile: 1.9117521524429322
10th percentile: 1.9157760858535766
20th percentile: 1.9394283294677734
30th percentile: 1.9621578216552735
40th percentile: 1.9659887313842774
50th percentile: 2.027656316757202
60th percentile: 2.0510244846343992
70th percentile: 2.102990508079529
80th percentile: 2.180020236968994
90th percentile: 2.2545266628265384
95th percentile: 2.3757627129554746
99th percentile: 2.450516691207886
mean time: 2.062868579228719
Pipeline stage StressChecker completed in 65.87s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyTriggerPipeline
run_pipeline:run_in_cloud %s
starting trigger_guanaco_pipeline args=%s
triggered trigger_guanaco_pipeline args=%s
Pipeline stage OfflineFamilyFriendlyTriggerPipeline completed in 0.68s
Shutdown handler de-registered
chaiml-muster-v0e-lr1e5_71769_v2 status is now deployed due to DeploymentManager action
chaiml-muster-v0e-lr1e5_71769_v2 status is now inactive due to auto deactivation removed underperforming models
chaiml-muster-v0e-lr1e5_71769_v2 status is now inactive due to Froze recruitment for AB test 0211_euclid
chaiml-muster-v0e-lr1e5_71769_v2 status is now torndown due to DeploymentManager action