developer_uid: rirv938
submission_id: chaiml-live-rm-20260216-_8340_v4
model_name: chaiml-live-rm-20260216-_8340_v4
model_group: ChaiML/live_rm_20260216_
status: inactive
timestamp: 2026-02-20T20:27:53+00:00
num_battles: 11319
num_wins: 6036
celo_rating: 1317.58
family_friendly_score: 0.0
family_friendly_standard_error: 0.0
submission_type: basic
model_repo: ChaiML/live_rm_20260216_1024_cp390_from_prev
model_architecture: LlamaForSequenceClassification
model_num_parameters: 8030261248.0
best_of: 1
max_input_tokens: 1024
max_output_tokens: 1
reward_model: default
display_name: chaiml-live-rm-20260216-_8340_v4
ineligible_reason: max_output_tokens!=64
is_internal_developer: True
language_model: ChaiML/live_rm_20260216_1024_cp390_from_prev
model_size: 8B
ranking_group: single
us_pacific_date: 2026-02-20
win_ratio: 0.5332626557116353
generation_params: {'temperature': 1.0, 'top_p': 1.0, 'min_p': 0.0, 'top_k': 40, 'presence_penalty': 0.0, 'frequency_penalty': 0.0, 'stopping_words': ['\n'], 'max_input_tokens': 1024, 'best_of': 1, 'max_output_tokens': 1}
formatter: {'memory_template': '', 'prompt_template': '', 'bot_template': '{bot_name}: {message}\n', 'user_template': '{user_name}: {message}\n', 'response_template': '', 'truncate_by_message': True}
Resubmit model
Shutdown handler not registered because Python interpreter is not running in the main thread
run pipeline %s
run pipeline stage %s
Running pipeline stage VLLMUploader
Starting job with name chaiml-live-rm-20260216-8340-v4-uploader
Waiting for job on chaiml-live-rm-20260216-8340-v4-uploader to finish
chaiml-live-rm-20260216-8340-v4-uploader: Using quantization_mode: none
chaiml-live-rm-20260216-8340-v4-uploader: Downloading snapshot of ChaiML/live_rm_20260216_1024_cp390_from_prev...
chaiml-live-rm-20260216-8340-v4-uploader: Downloaded in 11.352s
chaiml-live-rm-20260216-8340-v4-uploader: Processed model ChaiML/live_rm_20260216_1024_cp390_from_prev in 18.284s
chaiml-live-rm-20260216-8340-v4-uploader: creating bucket guanaco-vllm-models
chaiml-live-rm-20260216-8340-v4-uploader: /usr/lib/python3/dist-packages/S3/BaseUtils.py:56: SyntaxWarning: invalid escape sequence '\.'
chaiml-live-rm-20260216-8340-v4-uploader:   RE_S3_DATESTRING = re.compile('\.[0-9]*(?:[Z\\-\\+]*?)')
chaiml-live-rm-20260216-8340-v4-uploader: /usr/lib/python3/dist-packages/S3/BaseUtils.py:57: SyntaxWarning: invalid escape sequence '\s'
chaiml-live-rm-20260216-8340-v4-uploader:   RE_XML_NAMESPACE = re.compile(b'^(<?[^>]+?>\s*|\s*)(<\w+) xmlns=[\'"](https?://[^\'"]+)[\'"]', re.MULTILINE)
chaiml-live-rm-20260216-8340-v4-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:240: SyntaxWarning: invalid escape sequence '\.'
chaiml-live-rm-20260216-8340-v4-uploader:   invalid = re.search("([^a-z0-9\.-])", bucket, re.UNICODE)
chaiml-live-rm-20260216-8340-v4-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:244: SyntaxWarning: invalid escape sequence '\.'
chaiml-live-rm-20260216-8340-v4-uploader:   invalid = re.search("([^A-Za-z0-9\._-])", bucket, re.UNICODE)
chaiml-live-rm-20260216-8340-v4-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:255: SyntaxWarning: invalid escape sequence '\.'
chaiml-live-rm-20260216-8340-v4-uploader:   if re.search("-\.", bucket, re.UNICODE):
chaiml-live-rm-20260216-8340-v4-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:257: SyntaxWarning: invalid escape sequence '\.'
chaiml-live-rm-20260216-8340-v4-uploader:   if re.search("\.\.", bucket, re.UNICODE):
chaiml-live-rm-20260216-8340-v4-uploader: /usr/lib/python3/dist-packages/S3/S3Uri.py:155: SyntaxWarning: invalid escape sequence '\w'
chaiml-live-rm-20260216-8340-v4-uploader:   _re = re.compile("^(\w+://)?(.*)", re.UNICODE)
chaiml-live-rm-20260216-8340-v4-uploader: /usr/lib/python3/dist-packages/S3/FileLists.py:480: SyntaxWarning: invalid escape sequence '\*'
chaiml-live-rm-20260216-8340-v4-uploader:   wildcard_split_result = re.split("\*|\?", uri_str, maxsplit=1)
chaiml-live-rm-20260216-8340-v4-uploader: Bucket 's3://guanaco-vllm-models/' created
chaiml-live-rm-20260216-8340-v4-uploader: uploading /dev/shm/model_output to s3://guanaco-vllm-models/chaiml-live-rm-20260216-8340-v4/default
chaiml-live-rm-20260216-8340-v4-uploader: cp /dev/shm/model_output/special_tokens_map.json s3://guanaco-vllm-models/chaiml-live-rm-20260216-8340-v4/default/special_tokens_map.json
chaiml-live-rm-20260216-8340-v4-uploader: cp /dev/shm/model_output/tokenizer_config.json s3://guanaco-vllm-models/chaiml-live-rm-20260216-8340-v4/default/tokenizer_config.json
chaiml-live-rm-20260216-8340-v4-uploader: cp /dev/shm/model_output/README.md s3://guanaco-vllm-models/chaiml-live-rm-20260216-8340-v4/default/README.md
chaiml-live-rm-20260216-8340-v4-uploader: cp /dev/shm/model_output/model.safetensors.index.json s3://guanaco-vllm-models/chaiml-live-rm-20260216-8340-v4/default/model.safetensors.index.json
chaiml-live-rm-20260216-8340-v4-uploader: cp /dev/shm/model_output/config.json s3://guanaco-vllm-models/chaiml-live-rm-20260216-8340-v4/default/config.json
chaiml-live-rm-20260216-8340-v4-uploader: cp /dev/shm/model_output/.gitattributes s3://guanaco-vllm-models/chaiml-live-rm-20260216-8340-v4/default/.gitattributes
chaiml-live-rm-20260216-8340-v4-uploader: cp /dev/shm/model_output/tokenizer.json s3://guanaco-vllm-models/chaiml-live-rm-20260216-8340-v4/default/tokenizer.json
chaiml-live-rm-20260216-8340-v4-uploader: cp /dev/shm/model_output/model-00004-of-00004.safetensors s3://guanaco-vllm-models/chaiml-live-rm-20260216-8340-v4/default/model-00004-of-00004.safetensors
HTTP Request: %s %s "%s %d %s"
chaiml-live-rm-20260216-8340-v4-uploader: cp /dev/shm/model_output/model-00003-of-00004.safetensors s3://guanaco-vllm-models/chaiml-live-rm-20260216-8340-v4/default/model-00003-of-00004.safetensors
chaiml-live-rm-20260216-8340-v4-uploader: cp /dev/shm/model_output/model-00001-of-00004.safetensors s3://guanaco-vllm-models/chaiml-live-rm-20260216-8340-v4/default/model-00001-of-00004.safetensors
chaiml-live-rm-20260216-8340-v4-uploader: cp /dev/shm/model_output/model-00002-of-00004.safetensors s3://guanaco-vllm-models/chaiml-live-rm-20260216-8340-v4/default/model-00002-of-00004.safetensors
Job chaiml-live-rm-20260216-8340-v4-uploader completed after 53.12s with status: succeeded
Stopping job with name chaiml-live-rm-20260216-8340-v4-uploader
Pipeline stage VLLMUploader completed in 54.73s
run pipeline stage %s
Running pipeline stage VLLMTemplater
Pipeline stage VLLMTemplater completed in 0.15s
run pipeline stage %s
Running pipeline stage VLLMDeployer
Creating inference service chaiml-live-rm-20260216-8340-v4
Waiting for inference service chaiml-live-rm-20260216-8340-v4 to be ready
Failed to get response for submission chaiml-mistral-24b-2048_15988_v1: ('http://chaiml-mistral-24b-2048-15988-v1-predictor.tenant-chaiml-guanaco.kchai-coreweave-us-east-04a.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', '')
Failed to get response for submission chaiml-grpo-q235b-kimid_37540_v1: HTTPConnectionPool(host='chaiml-grpo-q235b-kimid-37540-v1-predictor.tenant-chaiml-guanaco.kchai-coreweave-us-east-04a.chaiverse.com', port=80): Read timed out. (read timeout=12.0)
Failed to get response for submission chaiml-mistral-24b-2048_15988_v1: ('http://chaiml-mistral-24b-2048-15988-v1-predictor.tenant-chaiml-guanaco.kchai-coreweave-us-east-04a.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', '')
Inference service chaiml-live-rm-20260216-8340-v4 ready after 170.9651734828949s
Pipeline stage VLLMDeployer completed in 173.30s
run pipeline stage %s
Running pipeline stage StressChecker
Received healthy response to inference request in 2.574314832687378s
Failed to get response for submission chaiml-grpo-q235b-kimid_37540_v1: HTTPConnectionPool(host='chaiml-grpo-q235b-kimid-37540-v1-predictor.tenant-chaiml-guanaco.kchai-coreweave-us-east-04a.chaiverse.com', port=80): Read timed out. (read timeout=12.0)
Received healthy response to inference request in 3.7473819255828857s
Received healthy response to inference request in 4.4776201248168945s
Failed to get response for submission chaiml-mistral-24b-2048_15988_v1: ('http://chaiml-mistral-24b-2048-15988-v1-predictor.tenant-chaiml-guanaco.kchai-coreweave-us-east-04a.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', '')
Received healthy response to inference request in 3.5198769569396973s
Received healthy response to inference request in 3.3142855167388916s
5 requests
0 failed requests
5th percentile: 2.7223089694976808
10th percentile: 2.8703031063079836
20th percentile: 3.166291379928589
30th percentile: 3.3554038047790526
40th percentile: 3.437640380859375
50th percentile: 3.5198769569396973
60th percentile: 3.6108789443969727
70th percentile: 3.701880931854248
80th percentile: 3.8934295654296878
90th percentile: 4.185524845123291
95th percentile: 4.331572484970093
99th percentile: 4.448410596847534
mean time: 3.5266958713531493
Pipeline stage StressChecker completed in 18.88s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyTriggerPipeline
run_pipeline:run_in_cloud %s
starting trigger_guanaco_pipeline args=%s
Failed to get response for submission chaiml-grpo-q235b-kimid_37540_v1: HTTPConnectionPool(host='chaiml-grpo-q235b-kimid-37540-v1-predictor.tenant-chaiml-guanaco.kchai-coreweave-us-east-04a.chaiverse.com', port=80): Read timed out. (read timeout=12.0)
triggered trigger_guanaco_pipeline args=%s
Pipeline stage OfflineFamilyFriendlyTriggerPipeline completed in 0.93s
Shutdown handler de-registered
chaiml-live-rm-20260216-_8340_v4 status is now deployed due to DeploymentManager action
chaiml-live-rm-20260216-_8340_v4 status is now inactive due to auto deactivation removed underperforming models