developer_uid: rirv938
submission_id: chaiml-llama-8b-202503_16869_v41
model_name: chaiml-llama-8b-202503_16869_v41
model_group: ChaiML/llama_8b_202503_1
status: inactive
timestamp: 2026-02-24T22:37:39+00:00
num_battles: 10319
num_wins: 4530
celo_rating: 1253.48
family_friendly_score: 0.0
family_friendly_standard_error: 0.0
submission_type: basic
model_repo: ChaiML/llama_8b_202503_1m_nemo_safety
model_architecture: LlamaForSequenceClassification
model_num_parameters: 8030261248.0
best_of: 1
max_input_tokens: 512
max_output_tokens: 1
reward_model: default
display_name: chaiml-llama-8b-202503_16869_v41
ineligible_reason: max_output_tokens!=64
is_internal_developer: True
language_model: ChaiML/llama_8b_202503_1m_nemo_safety
model_size: 8B
ranking_group: single
us_pacific_date: 2026-02-24
win_ratio: 0.4389960267467778
generation_params: {'temperature': 1.0, 'top_p': 1.0, 'min_p': 0.0, 'top_k': 40, 'presence_penalty': 0.0, 'frequency_penalty': 0.0, 'stopping_words': ['\n'], 'max_input_tokens': 512, 'best_of': 1, 'max_output_tokens': 1}
formatter: {'memory_template': '', 'prompt_template': '', 'bot_template': '{bot_name}: {message}\n', 'user_template': '{user_name}: {message}\n', 'response_template': '', 'truncate_by_message': True}
Resubmit model
Shutdown handler not registered because Python interpreter is not running in the main thread
run pipeline %s
run pipeline stage %s
Running pipeline stage VLLMUploader
Starting job with name chaiml-llama-8b-202503-16869-v41-uploader
Waiting for job on chaiml-llama-8b-202503-16869-v41-uploader to finish
chaiml-llama-8b-202503-16869-v41-uploader: Using quantization_mode: none
chaiml-llama-8b-202503-16869-v41-uploader: Downloading snapshot of ChaiML/llama_8b_202503_1m_nemo_safety...
Retrying (%r) after connection broken by '%r': %s
chaiml-llama-8b-202503-16869-v41-uploader: Downloaded in 9.685s
HTTP Request: %s %s "%s %d %s"
chaiml-llama-8b-202503-16869-v41-uploader: Processed model ChaiML/llama_8b_202503_1m_nemo_safety in 15.330s
chaiml-llama-8b-202503-16869-v41-uploader: creating bucket guanaco-vllm-models
chaiml-llama-8b-202503-16869-v41-uploader: /usr/lib/python3/dist-packages/S3/BaseUtils.py:56: SyntaxWarning: invalid escape sequence '\.'
chaiml-llama-8b-202503-16869-v41-uploader:   RE_S3_DATESTRING = re.compile('\.[0-9]*(?:[Z\\-\\+]*?)')
chaiml-llama-8b-202503-16869-v41-uploader: /usr/lib/python3/dist-packages/S3/BaseUtils.py:57: SyntaxWarning: invalid escape sequence '\s'
chaiml-llama-8b-202503-16869-v41-uploader:   RE_XML_NAMESPACE = re.compile(b'^(<?[^>]+?>\s*|\s*)(<\w+) xmlns=[\'"](https?://[^\'"]+)[\'"]', re.MULTILINE)
chaiml-llama-8b-202503-16869-v41-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:240: SyntaxWarning: invalid escape sequence '\.'
chaiml-llama-8b-202503-16869-v41-uploader:   invalid = re.search("([^a-z0-9\.-])", bucket, re.UNICODE)
chaiml-llama-8b-202503-16869-v41-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:244: SyntaxWarning: invalid escape sequence '\.'
chaiml-llama-8b-202503-16869-v41-uploader:   invalid = re.search("([^A-Za-z0-9\._-])", bucket, re.UNICODE)
chaiml-llama-8b-202503-16869-v41-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:255: SyntaxWarning: invalid escape sequence '\.'
chaiml-llama-8b-202503-16869-v41-uploader:   if re.search("-\.", bucket, re.UNICODE):
chaiml-llama-8b-202503-16869-v41-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:257: SyntaxWarning: invalid escape sequence '\.'
chaiml-llama-8b-202503-16869-v41-uploader:   if re.search("\.\.", bucket, re.UNICODE):
chaiml-llama-8b-202503-16869-v41-uploader: /usr/lib/python3/dist-packages/S3/S3Uri.py:155: SyntaxWarning: invalid escape sequence '\w'
chaiml-llama-8b-202503-16869-v41-uploader:   _re = re.compile("^(\w+://)?(.*)", re.UNICODE)
chaiml-llama-8b-202503-16869-v41-uploader: /usr/lib/python3/dist-packages/S3/FileLists.py:480: SyntaxWarning: invalid escape sequence '\*'
chaiml-llama-8b-202503-16869-v41-uploader:   wildcard_split_result = re.split("\*|\?", uri_str, maxsplit=1)
chaiml-llama-8b-202503-16869-v41-uploader: Bucket 's3://guanaco-vllm-models/' created
chaiml-llama-8b-202503-16869-v41-uploader: uploading /dev/shm/model_output to s3://guanaco-vllm-models/chaiml-llama-8b-202503-16869-v41/default
chaiml-llama-8b-202503-16869-v41-uploader: cp /dev/shm/model_output/.gitattributes s3://guanaco-vllm-models/chaiml-llama-8b-202503-16869-v41/default/.gitattributes
chaiml-llama-8b-202503-16869-v41-uploader: cp /dev/shm/model_output/config.json s3://guanaco-vllm-models/chaiml-llama-8b-202503-16869-v41/default/config.json
chaiml-llama-8b-202503-16869-v41-uploader: cp /dev/shm/model_output/README.md s3://guanaco-vllm-models/chaiml-llama-8b-202503-16869-v41/default/README.md
chaiml-llama-8b-202503-16869-v41-uploader: cp /dev/shm/model_output/tokenizer_config.json s3://guanaco-vllm-models/chaiml-llama-8b-202503-16869-v41/default/tokenizer_config.json
chaiml-llama-8b-202503-16869-v41-uploader: cp /dev/shm/model_output/special_tokens_map.json s3://guanaco-vllm-models/chaiml-llama-8b-202503-16869-v41/default/special_tokens_map.json
chaiml-llama-8b-202503-16869-v41-uploader: cp /dev/shm/model_output/model.safetensors.index.json s3://guanaco-vllm-models/chaiml-llama-8b-202503-16869-v41/default/model.safetensors.index.json
chaiml-llama-8b-202503-16869-v41-uploader: cp /dev/shm/model_output/tokenizer.json s3://guanaco-vllm-models/chaiml-llama-8b-202503-16869-v41/default/tokenizer.json
chaiml-llama-8b-202503-16869-v41-uploader: cp /dev/shm/model_output/model-00004-of-00004.safetensors s3://guanaco-vllm-models/chaiml-llama-8b-202503-16869-v41/default/model-00004-of-00004.safetensors
Job chaiml-llama-8b-202503-16869-v41-uploader completed after 45.01s with status: succeeded
Stopping job with name chaiml-llama-8b-202503-16869-v41-uploader
Pipeline stage VLLMUploader completed in 47.00s
run pipeline stage %s
Running pipeline stage VLLMTemplater
Pipeline stage VLLMTemplater completed in 0.60s
run pipeline stage %s
Running pipeline stage VLLMDeployer
Creating inference service chaiml-llama-8b-202503-16869-v41
Waiting for inference service chaiml-llama-8b-202503-16869-v41 to be ready
Inference service chaiml-llama-8b-202503-16869-v41 ready after 150.7962203025818s
Pipeline stage VLLMDeployer completed in 151.50s
run pipeline stage %s
Running pipeline stage StressChecker
Received healthy response to inference request in 4.124415397644043s
Received healthy response to inference request in 4.478309631347656s
Received healthy response to inference request in 5.242760181427002s
Received healthy response to inference request in 3.5362560749053955s
Received healthy response to inference request in 3.534776210784912s
5 requests
0 failed requests
5th percentile: 3.535072183609009
10th percentile: 3.5353681564331056
20th percentile: 3.5359601020812987
30th percentile: 3.653887939453125
40th percentile: 3.889151668548584
50th percentile: 4.124415397644043
60th percentile: 4.265973091125488
70th percentile: 4.407530784606934
80th percentile: 4.631199741363526
90th percentile: 4.936979961395264
95th percentile: 5.0898700714111325
99th percentile: 5.212182159423828
mean time: 4.183303499221802
Pipeline stage StressChecker completed in 22.23s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyTriggerPipeline
run_pipeline:run_in_cloud %s
starting trigger_guanaco_pipeline args=%s
triggered trigger_guanaco_pipeline args=%s
Pipeline stage OfflineFamilyFriendlyTriggerPipeline completed in 0.70s
Shutdown handler de-registered
chaiml-llama-8b-202503_16869_v41 status is now deployed due to DeploymentManager action
chaiml-llama-8b-202503_16869_v41 status is now inactive due to auto deactivation removed underperforming models