developer_uid: chai_evaluation_service
submission_id: evelyn777-chai-sft-3b_v4
model_name: evelyn777-chai-sft-3b_v4
model_group: evelyn777/chai-sft-3b
status: inactive
timestamp: 2026-02-07T23:18:59+00:00
num_battles: 12390
num_wins: 4036
celo_rating: 1177.79
family_friendly_score: 0.0
family_friendly_standard_error: 0.0
submission_type: basic
model_repo: evelyn777/chai-sft-3b
model_architecture: Qwen2ForCausalLM
model_num_parameters: 3397011456.0
best_of: 8
max_input_tokens: 2048
max_output_tokens: 64
reward_model: default
display_name: evelyn777-chai-sft-3b_v4
is_internal_developer: True
language_model: evelyn777/chai-sft-3b
model_size: 3B
ranking_group: single
us_pacific_date: 2026-02-07
win_ratio: 0.32574656981436645
generation_params: {'temperature': 0.85, 'top_p': 0.9, 'min_p': 0.0, 'top_k': 40, 'presence_penalty': 0.2, 'frequency_penalty': 0.3, 'stopping_words': ['\n'], 'max_input_tokens': 2048, 'best_of': 8, 'max_output_tokens': 64}
formatter: {'memory_template': '<|im_start|>system\n{memory}<|im_end|>\n', 'prompt_template': '<|im_start|>user\n{prompt}<|im_end|>\n', 'bot_template': '<|im_start|>assistant\n{bot_name}: {message}<|im_end|>\n', 'user_template': '<|im_start|>user\n{user_name}: {message}<|im_end|>\n', 'response_template': '<|im_start|>assistant\n{bot_name}:', 'truncate_by_message': True}
Resubmit model
Shutdown handler not registered because Python interpreter is not running in the main thread
run pipeline %s
run pipeline stage %s
Running pipeline stage VLLMUploader
Starting job with name evelyn777-chai-sft-3b-v4-uploader
Waiting for job on evelyn777-chai-sft-3b-v4-uploader to finish
evelyn777-chai-sft-3b-v4-uploader: Using quantization_mode: none
evelyn777-chai-sft-3b-v4-uploader: Downloading snapshot of evelyn777/chai-sft-3b...
evelyn777-chai-sft-3b-v4-uploader: Processed model evelyn777/chai-sft-3b in 6.787s
evelyn777-chai-sft-3b-v4-uploader: creating bucket guanaco-vllm-models
evelyn777-chai-sft-3b-v4-uploader: /usr/lib/python3/dist-packages/S3/BaseUtils.py:56: SyntaxWarning: invalid escape sequence '\.'
evelyn777-chai-sft-3b-v4-uploader:   RE_S3_DATESTRING = re.compile('\.[0-9]*(?:[Z\\-\\+]*?)')
evelyn777-chai-sft-3b-v4-uploader: /usr/lib/python3/dist-packages/S3/BaseUtils.py:57: SyntaxWarning: invalid escape sequence '\s'
evelyn777-chai-sft-3b-v4-uploader:   RE_XML_NAMESPACE = re.compile(b'^(<?[^>]+?>\s*|\s*)(<\w+) xmlns=[\'"](https?://[^\'"]+)[\'"]', re.MULTILINE)
evelyn777-chai-sft-3b-v4-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:240: SyntaxWarning: invalid escape sequence '\.'
evelyn777-chai-sft-3b-v4-uploader:   invalid = re.search("([^a-z0-9\.-])", bucket, re.UNICODE)
evelyn777-chai-sft-3b-v4-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:244: SyntaxWarning: invalid escape sequence '\.'
evelyn777-chai-sft-3b-v4-uploader:   invalid = re.search("([^A-Za-z0-9\._-])", bucket, re.UNICODE)
evelyn777-chai-sft-3b-v4-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:255: SyntaxWarning: invalid escape sequence '\.'
evelyn777-chai-sft-3b-v4-uploader:   if re.search("-\.", bucket, re.UNICODE):
evelyn777-chai-sft-3b-v4-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:257: SyntaxWarning: invalid escape sequence '\.'
evelyn777-chai-sft-3b-v4-uploader:   if re.search("\.\.", bucket, re.UNICODE):
evelyn777-chai-sft-3b-v4-uploader: /usr/lib/python3/dist-packages/S3/S3Uri.py:155: SyntaxWarning: invalid escape sequence '\w'
evelyn777-chai-sft-3b-v4-uploader:   _re = re.compile("^(\w+://)?(.*)", re.UNICODE)
evelyn777-chai-sft-3b-v4-uploader: /usr/lib/python3/dist-packages/S3/FileLists.py:480: SyntaxWarning: invalid escape sequence '\*'
evelyn777-chai-sft-3b-v4-uploader:   wildcard_split_result = re.split("\*|\?", uri_str, maxsplit=1)
evelyn777-chai-sft-3b-v4-uploader: Bucket 's3://guanaco-vllm-models/' created
evelyn777-chai-sft-3b-v4-uploader: uploading /dev/shm/model_output to s3://guanaco-vllm-models/evelyn777-chai-sft-3b-v4
evelyn777-chai-sft-3b-v4-uploader: cp /dev/shm/model_output/added_tokens.json s3://guanaco-vllm-models/evelyn777-chai-sft-3b-v4/added_tokens.json
evelyn777-chai-sft-3b-v4-uploader: cp /dev/shm/model_output/.gitattributes s3://guanaco-vllm-models/evelyn777-chai-sft-3b-v4/.gitattributes
evelyn777-chai-sft-3b-v4-uploader: cp /dev/shm/model_output/generation_config.json s3://guanaco-vllm-models/evelyn777-chai-sft-3b-v4/generation_config.json
evelyn777-chai-sft-3b-v4-uploader: cp /dev/shm/model_output/tokenizer_config.json s3://guanaco-vllm-models/evelyn777-chai-sft-3b-v4/tokenizer_config.json
evelyn777-chai-sft-3b-v4-uploader: cp /dev/shm/model_output/chat_template.jinja s3://guanaco-vllm-models/evelyn777-chai-sft-3b-v4/chat_template.jinja
evelyn777-chai-sft-3b-v4-uploader: cp /dev/shm/model_output/special_tokens_map.json s3://guanaco-vllm-models/evelyn777-chai-sft-3b-v4/special_tokens_map.json
evelyn777-chai-sft-3b-v4-uploader: cp /dev/shm/model_output/config.json s3://guanaco-vllm-models/evelyn777-chai-sft-3b-v4/config.json
evelyn777-chai-sft-3b-v4-uploader: cp /dev/shm/model_output/model.safetensors.index.json s3://guanaco-vllm-models/evelyn777-chai-sft-3b-v4/model.safetensors.index.json
evelyn777-chai-sft-3b-v4-uploader: cp /dev/shm/model_output/merges.txt s3://guanaco-vllm-models/evelyn777-chai-sft-3b-v4/merges.txt
evelyn777-chai-sft-3b-v4-uploader: cp /dev/shm/model_output/vocab.json s3://guanaco-vllm-models/evelyn777-chai-sft-3b-v4/vocab.json
evelyn777-chai-sft-3b-v4-uploader: cp /dev/shm/model_output/tokenizer.json s3://guanaco-vllm-models/evelyn777-chai-sft-3b-v4/tokenizer.json
evelyn777-chai-sft-3b-v4-uploader: cp /dev/shm/model_output/model-00001-of-00002.safetensors s3://guanaco-vllm-models/evelyn777-chai-sft-3b-v4/model-00001-of-00002.safetensors
Job evelyn777-chai-sft-3b-v4-uploader completed after 106.03s with status: succeeded
Stopping job with name evelyn777-chai-sft-3b-v4-uploader
Pipeline stage VLLMUploader completed in 106.71s
run pipeline stage %s
Running pipeline stage VLLMTemplater
Pipeline stage VLLMTemplater completed in 0.14s
run pipeline stage %s
Running pipeline stage VLLMDeployer
Creating inference service evelyn777-chai-sft-3b-v4
Waiting for inference service evelyn777-chai-sft-3b-v4 to be ready
HTTP Request: %s %s "%s %d %s"
Inference service evelyn777-chai-sft-3b-v4 ready after 171.19520020484924s
Pipeline stage VLLMDeployer completed in 172.40s
run pipeline stage %s
Running pipeline stage StressChecker
Received healthy response to inference request in 1.015228509902954s
Received healthy response to inference request in 0.9198646545410156s
Received healthy response to inference request in 1.0667023658752441s
Received healthy response to inference request in 1.708796501159668s
Received healthy response to inference request in 1.6504294872283936s
Received healthy response to inference request in 1.288942813873291s
Received healthy response to inference request in 0.8520092964172363s
Received healthy response to inference request in 1.1644032001495361s
Received healthy response to inference request in 1.7773191928863525s
Received healthy response to inference request in 0.9648890495300293s
Received healthy response to inference request in 0.9287407398223877s
Received healthy response to inference request in 1.2248058319091797s
Received healthy response to inference request in 1.1580708026885986s
Received healthy response to inference request in 0.8740711212158203s
Received healthy response to inference request in 0.9309296607971191s
Received healthy response to inference request in 1.510446310043335s
Received healthy response to inference request in 0.5031018257141113s
Received healthy response to inference request in 0.8804049491882324s
Received healthy response to inference request in 0.9677574634552002s
Received healthy response to inference request in 0.7087423801422119s
Received healthy response to inference request in 0.6098582744598389s
Received healthy response to inference request in 0.730067253112793s
Received healthy response to inference request in 0.991523027420044s
Received healthy response to inference request in 1.3809692859649658s
Received healthy response to inference request in 1.1942827701568604s
Received healthy response to inference request in 1.3324358463287354s
Received healthy response to inference request in 2.492919445037842s
Received healthy response to inference request in 1.0931627750396729s
Received healthy response to inference request in 1.3669395446777344s
Received healthy response to inference request in 1.3063511848449707s
30 requests
0 failed requests
5th percentile: 0.6543561220169067
10th percentile: 0.7279347658157349
20th percentile: 0.87913818359375
30th percentile: 0.9302729845046998
40th percentile: 0.9820168018341064
50th percentile: 1.0799325704574585
60th percentile: 1.1763550281524657
70th percentile: 1.2941653251647949
80th percentile: 1.3697454929351807
90th percentile: 1.656266188621521
95th percentile: 1.7464839816093443
99th percentile: 2.2853953719139106
mean time: 1.153138852119446
Pipeline stage StressChecker completed in 62.11s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyTriggerPipeline
run_pipeline:run_in_cloud %s
starting trigger_guanaco_pipeline args=%s
triggered trigger_guanaco_pipeline args=%s
Pipeline stage OfflineFamilyFriendlyTriggerPipeline completed in 3.58s
Shutdown handler de-registered
evelyn777-chai-sft-3b_v4 status is now deployed due to DeploymentManager action
evelyn777-chai-sft-3b_v4 status is now inactive due to auto deactivation removed underperforming models