chaiml-llama-8b-202503_16869

developer_uid: rirv938

submission_id: chaiml-llama-8b-202503_16869_v43

model_name: chaiml-llama-8b-202503_16869_v43

model_group: ChaiML/llama_8b_202503_1

status: inactive

timestamp: 2026-03-04T18:26:46+00:00

num_battles: 12274

num_wins: 5376

celo_rating: 9051.47

family_friendly_score: 0.0

family_friendly_standard_error: 0.0

submission_type: basic

model_repo: ChaiML/llama_8b_202503_1m_nemo_safety

model_architecture: LlamaForSequenceClassification

model_num_parameters: 8030261248.0

best_of: 1

max_input_tokens: 512

max_output_tokens: 1

reward_model: default

display_name: chaiml-llama-8b-202503_16869_v43

ineligible_reason: max_output_tokens!=64

is_internal_developer: True

language_model: ChaiML/llama_8b_202503_1m_nemo_safety

model_size: 8B

ranking_group: single

us_pacific_date: 2026-03-04

win_ratio: 0.4379990223236109

generation_params: {'temperature': 1.0, 'top_p': 1.0, 'min_p': 0.0, 'top_k': 40, 'presence_penalty': 0.0, 'frequency_penalty': 0.0, 'stopping_words': ['\n'], 'max_input_tokens': 512, 'best_of': 1, 'max_output_tokens': 1}

formatter: {'memory_template': '', 'prompt_template': '', 'bot_template': '{bot_name}: {message}\n', 'user_template': '{user_name}: {message}\n', 'response_template': '', 'truncate_by_message': True}

Resubmit model

admin requested tearing down of chaiml-4d70-fd43-linear_51732_v6
run pipeline %s
Shutdown handler not registered because Python interpreter is not running in the main thread
run pipeline stage %s
run pipeline %s
Running pipeline stage VLLMUploader
run pipeline stage %s
admin requested tearing down of chaiml-ssnew-v5-dpo-lr5_19068_v8
Starting job with name chaiml-llama-8b-202503-16869-v43-uploader
Running pipeline stage VLLMDeleter
Shutdown handler not registered because Python interpreter is not running in the main thread
Waiting for job on chaiml-llama-8b-202503-16869-v43-uploader to finish
Checking if service chaiml-4d70-fd43-linear-51732-v6 is running
run pipeline %s
run pipeline stage %s
Tearing down inference service chaiml-4d70-fd43-linear-51732-v6
Running pipeline stage VLLMDeleter
Service chaiml-4d70-fd43-linear-51732-v6 has been torndown
Checking if service chaiml-ssnew-v5-dpo-lr5-19068-v8 is running
Pipeline stage VLLMDeleter completed in 2.51s
run pipeline stage %s
Tearing down inference service chaiml-ssnew-v5-dpo-lr5-19068-v8
Running pipeline stage VLLMModelDeleter
Service chaiml-ssnew-v5-dpo-lr5-19068-v8 has been torndown
Cleaning model data from S3
Pipeline stage VLLMDeleter completed in 2.01s
Cleaning model data from model cache
run pipeline stage %s
Deleting key chaiml-4d70-fd43-linear-51732-v6/default/.gitattributes from bucket guanaco-vllm-models
Running pipeline stage VLLMModelDeleter
Deleting key chaiml-4d70-fd43-linear-51732-v6/default/chat_template.jinja from bucket guanaco-vllm-models
Cleaning model data from S3
Deleting key chaiml-4d70-fd43-linear-51732-v6/default/config.json from bucket guanaco-vllm-models
Cleaning model data from model cache
Deleting key chaiml-4d70-fd43-linear-51732-v6/default/generation_config.json from bucket guanaco-vllm-models
Deleting key chaiml-ssnew-v5-dpo-lr5-19068-v8/default/.gitattributes from bucket guanaco-vllm-models
Deleting key chaiml-4d70-fd43-linear-51732-v6/default/model-00001-of-00003.safetensors from bucket guanaco-vllm-models
Deleting key chaiml-ssnew-v5-dpo-lr5-19068-v8/default/config.json from bucket guanaco-vllm-models
Deleting key chaiml-ssnew-v5-dpo-lr5-19068-v8/default/generation_config.json from bucket guanaco-vllm-models
Deleting key chaiml-4d70-fd43-linear-51732-v6/default/model-00002-of-00003.safetensors from bucket guanaco-vllm-models
Deleting key chaiml-ssnew-v5-dpo-lr5-19068-v8/default/model-00001-of-00006.safetensors from bucket guanaco-vllm-models
Deleting key chaiml-4d70-fd43-linear-51732-v6/default/model-00003-of-00003.safetensors from bucket guanaco-vllm-models
Deleting key chaiml-ssnew-v5-dpo-lr5-19068-v8/default/model-00002-of-00006.safetensors from bucket guanaco-vllm-models
Deleting key chaiml-4d70-fd43-linear-51732-v6/default/model.safetensors.index.json from bucket guanaco-vllm-models
Deleting key chaiml-ssnew-v5-dpo-lr5-19068-v8/default/model-00003-of-00006.safetensors from bucket guanaco-vllm-models
Deleting key chaiml-4d70-fd43-linear-51732-v6/default/recipe.yaml from bucket guanaco-vllm-models
Deleting key chaiml-4d70-fd43-linear-51732-v6/default/special_tokens_map.json from bucket guanaco-vllm-models
Deleting key chaiml-ssnew-v5-dpo-lr5-19068-v8/default/model-00004-of-00006.safetensors from bucket guanaco-vllm-models
Deleting key chaiml-4d70-fd43-linear-51732-v6/default/tokenizer.json from bucket guanaco-vllm-models
chaiml-llama-8b-202503-16869-v43-uploader: Using quantization_mode: none
Deleting key chaiml-4d70-fd43-linear-51732-v6/default/tokenizer_config.json from bucket guanaco-vllm-models
chaiml-llama-8b-202503-16869-v43-uploader: Downloading snapshot of ChaiML/llama_8b_202503_1m_nemo_safety...
Deleting key chaiml-ssnew-v5-dpo-lr5-19068-v8/default/model-00005-of-00006.safetensors from bucket guanaco-vllm-models
Pipeline stage VLLMModelDeleter completed in 8.49s
Shutdown handler de-registered
Deleting key chaiml-ssnew-v5-dpo-lr5-19068-v8/default/model-00006-of-00006.safetensors from bucket guanaco-vllm-models
chaiml-4d70-fd43-linear_51732_v6 status is now torndown due to DeploymentManager action
Deleting key chaiml-ssnew-v5-dpo-lr5-19068-v8/default/model.safetensors.index.json from bucket guanaco-vllm-models
Deleting key chaiml-ssnew-v5-dpo-lr5-19068-v8/default/recipe.yaml from bucket guanaco-vllm-models
Deleting key chaiml-ssnew-v5-dpo-lr5-19068-v8/default/special_tokens_map.json from bucket guanaco-vllm-models
Deleting key chaiml-ssnew-v5-dpo-lr5-19068-v8/default/tokenizer.json from bucket guanaco-vllm-models
Deleting key chaiml-ssnew-v5-dpo-lr5-19068-v8/default/tokenizer_config.json from bucket guanaco-vllm-models
Pipeline stage VLLMModelDeleter completed in 11.12s
Shutdown handler de-registered
chaiml-ssnew-v5-dpo-lr5_19068_v8 status is now torndown due to DeploymentManager action
chaiml-llama-8b-202503-16869-v43-uploader: Downloaded in 8.505s
chaiml-llama-8b-202503-16869-v43-uploader: creating bucket guanaco-vllm-models
chaiml-llama-8b-202503-16869-v43-uploader: /usr/lib/python3/dist-packages/S3/BaseUtils.py:56: SyntaxWarning: invalid escape sequence '\.'
chaiml-llama-8b-202503-16869-v43-uploader:   RE_S3_DATESTRING = re.compile('\.[0-9]*(?:[Z\\-\\+]*?)')
chaiml-llama-8b-202503-16869-v43-uploader: /usr/lib/python3/dist-packages/S3/BaseUtils.py:57: SyntaxWarning: invalid escape sequence '\s'
chaiml-llama-8b-202503-16869-v43-uploader:   RE_XML_NAMESPACE = re.compile(b'^(<?[^>]+?>\s*|\s*)(<\w+) xmlns=[\'"](https?://[^\'"]+)[\'"]', re.MULTILINE)
chaiml-llama-8b-202503-16869-v43-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:240: SyntaxWarning: invalid escape sequence '\.'
chaiml-llama-8b-202503-16869-v43-uploader:   invalid = re.search("([^a-z0-9\.-])", bucket, re.UNICODE)
chaiml-llama-8b-202503-16869-v43-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:244: SyntaxWarning: invalid escape sequence '\.'
chaiml-llama-8b-202503-16869-v43-uploader:   invalid = re.search("([^A-Za-z0-9\._-])", bucket, re.UNICODE)
chaiml-llama-8b-202503-16869-v43-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:255: SyntaxWarning: invalid escape sequence '\.'
chaiml-llama-8b-202503-16869-v43-uploader:   if re.search("-\.", bucket, re.UNICODE):
chaiml-llama-8b-202503-16869-v43-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:257: SyntaxWarning: invalid escape sequence '\.'
chaiml-llama-8b-202503-16869-v43-uploader:   if re.search("\.\.", bucket, re.UNICODE):
chaiml-llama-8b-202503-16869-v43-uploader: /usr/lib/python3/dist-packages/S3/S3Uri.py:155: SyntaxWarning: invalid escape sequence '\w'
chaiml-llama-8b-202503-16869-v43-uploader:   _re = re.compile("^(\w+://)?(.*)", re.UNICODE)
chaiml-llama-8b-202503-16869-v43-uploader: /usr/lib/python3/dist-packages/S3/FileLists.py:480: SyntaxWarning: invalid escape sequence '\*'
chaiml-llama-8b-202503-16869-v43-uploader:   wildcard_split_result = re.split("\*|\?", uri_str, maxsplit=1)
chaiml-llama-8b-202503-16869-v43-uploader: Bucket 's3://guanaco-vllm-models/' created
chaiml-llama-8b-202503-16869-v43-uploader: uploading /dev/shm/model_output to s3://guanaco-vllm-models/chaiml-llama-8b-202503-16869-v43/default
chaiml-llama-8b-202503-16869-v43-uploader: cp /dev/shm/model_output/README.md s3://guanaco-vllm-models/chaiml-llama-8b-202503-16869-v43/default/README.md
chaiml-llama-8b-202503-16869-v43-uploader: cp /dev/shm/model_output/tokenizer_config.json s3://guanaco-vllm-models/chaiml-llama-8b-202503-16869-v43/default/tokenizer_config.json
chaiml-llama-8b-202503-16869-v43-uploader: cp /dev/shm/model_output/.gitattributes s3://guanaco-vllm-models/chaiml-llama-8b-202503-16869-v43/default/.gitattributes
chaiml-llama-8b-202503-16869-v43-uploader: cp /dev/shm/model_output/special_tokens_map.json s3://guanaco-vllm-models/chaiml-llama-8b-202503-16869-v43/default/special_tokens_map.json
chaiml-llama-8b-202503-16869-v43-uploader: cp /dev/shm/model_output/model.safetensors.index.json s3://guanaco-vllm-models/chaiml-llama-8b-202503-16869-v43/default/model.safetensors.index.json
chaiml-llama-8b-202503-16869-v43-uploader: cp /dev/shm/model_output/config.json s3://guanaco-vllm-models/chaiml-llama-8b-202503-16869-v43/default/config.json
chaiml-llama-8b-202503-16869-v43-uploader: cp /dev/shm/model_output/tokenizer.json s3://guanaco-vllm-models/chaiml-llama-8b-202503-16869-v43/default/tokenizer.json
chaiml-llama-8b-202503-16869-v43-uploader: cp /dev/shm/model_output/model-00004-of-00004.safetensors s3://guanaco-vllm-models/chaiml-llama-8b-202503-16869-v43/default/model-00004-of-00004.safetensors
chaiml-llama-8b-202503-16869-v43-uploader: cp /dev/shm/model_output/model-00003-of-00004.safetensors s3://guanaco-vllm-models/chaiml-llama-8b-202503-16869-v43/default/model-00003-of-00004.safetensors
chaiml-llama-8b-202503-16869-v43-uploader: cp /dev/shm/model_output/model-00001-of-00004.safetensors s3://guanaco-vllm-models/chaiml-llama-8b-202503-16869-v43/default/model-00001-of-00004.safetensors
chaiml-llama-8b-202503-16869-v43-uploader: cp /dev/shm/model_output/model-00002-of-00004.safetensors s3://guanaco-vllm-models/chaiml-llama-8b-202503-16869-v43/default/model-00002-of-00004.safetensors
Job chaiml-llama-8b-202503-16869-v43-uploader completed after 48.76s with status: succeeded
Stopping job with name chaiml-llama-8b-202503-16869-v43-uploader
Pipeline stage VLLMUploader completed in 50.43s
run pipeline stage %s
Running pipeline stage VLLMTemplater
Pipeline stage VLLMTemplater completed in 0.37s
run pipeline stage %s
Running pipeline stage VLLMDeployer
Creating inference service chaiml-llama-8b-202503-16869-v43
Waiting for inference service chaiml-llama-8b-202503-16869-v43 to be ready
Inference service chaiml-llama-8b-202503-16869-v43 ready after 160.47140789031982s
Pipeline stage VLLMDeployer completed in 161.94s
run pipeline stage %s
Running pipeline stage StressChecker
Received healthy response to inference request in 3.142616033554077s
Received healthy response to inference request in 6.704697847366333s
Received healthy response to inference request in 4.200588226318359s
Received healthy response to inference request in 5.631057500839233s
Received healthy response to inference request in 3.793672800064087s
5 requests
0 failed requests
5th percentile: 3.272827386856079
10th percentile: 3.403038740158081
20th percentile: 3.663461446762085
30th percentile: 3.8750558853149415
40th percentile: 4.03782205581665
50th percentile: 4.200588226318359
60th percentile: 4.772775936126709
70th percentile: 5.344963645935058
80th percentile: 5.8457855701446535
90th percentile: 6.275241708755493
95th percentile: 6.489969778060913
99th percentile: 6.661752233505249
mean time: 4.694526481628418
Pipeline stage StressChecker completed in 29.60s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyTriggerPipeline
run_pipeline:run_in_cloud %s
starting trigger_guanaco_pipeline args=%s
triggered trigger_guanaco_pipeline args=%s
Pipeline stage OfflineFamilyFriendlyTriggerPipeline completed in 1.53s
Shutdown handler de-registered
chaiml-llama-8b-202503_16869_v43 status is now deployed due to DeploymentManager action
chaiml-llama-8b-202503_16869_v43 status is now inactive due to auto deactivation removed underperforming models