developer_uid: chai_evaluation_service
submission_id: evelyn777-chai-sft-3b-v4_v1
model_name: evelyn777-chai-sft-3b-v4_v1
model_group: evelyn777/chai-sft-3b-v4
status: inactive
timestamp: 2026-02-08T06:46:37+00:00
num_battles: 12120
num_wins: 4110
celo_rating: 1188.07
family_friendly_score: 0.0
family_friendly_standard_error: 0.0
submission_type: basic
model_repo: evelyn777/chai-sft-3b-v4
model_architecture: Qwen2ForCausalLM
model_num_parameters: 3397011456.0
best_of: 8
max_input_tokens: 2048
max_output_tokens: 64
reward_model: default
display_name: evelyn777-chai-sft-3b-v4_v1
is_internal_developer: True
language_model: evelyn777/chai-sft-3b-v4
model_size: 3B
ranking_group: single
us_pacific_date: 2026-02-07
win_ratio: 0.33910891089108913
generation_params: {'temperature': 0.85, 'top_p': 0.9, 'min_p': 0.0, 'top_k': 40, 'presence_penalty': 0.0, 'frequency_penalty': 0.0, 'stopping_words': ['\n'], 'max_input_tokens': 2048, 'best_of': 8, 'max_output_tokens': 64}
formatter: {'memory_template': '<|im_start|>system\n{memory}<|im_end|>\n', 'prompt_template': '<|im_start|>user\n{prompt}<|im_end|>\n', 'bot_template': '<|im_start|>assistant\n{bot_name}: {message}<|im_end|>\n', 'user_template': '<|im_start|>user\n{user_name}: {message}<|im_end|>\n', 'response_template': '<|im_start|>assistant\n{bot_name}:', 'truncate_by_message': True}
Resubmit model
Shutdown handler not registered because Python interpreter is not running in the main thread
run pipeline %s
run pipeline stage %s
Running pipeline stage VLLMUploader
Starting job with name evelyn777-chai-sft-3b-v4-v1-uploader
Waiting for job on evelyn777-chai-sft-3b-v4-v1-uploader to finish
evelyn777-chai-sft-3b-v4-v1-uploader: Using quantization_mode: none
evelyn777-chai-sft-3b-v4-v1-uploader: Downloading snapshot of evelyn777/chai-sft-3b-v4...
evelyn777-chai-sft-3b-v4-v1-uploader: 
Fetching 13 files:   0%|          | 0/13 [00:00<?, ?it/s]
Fetching 13 files:   8%|▊         | 1/13 [00:00<00:03,  3.40it/s]
Fetching 13 files:  23%|██▎       | 3/13 [00:01<00:04,  2.36it/s]
Fetching 13 files:  54%|█████▍    | 7/13 [00:05<00:05,  1.20it/s]
Fetching 13 files: 100%|██████████| 13/13 [00:05<00:00,  2.43it/s]
evelyn777-chai-sft-3b-v4-v1-uploader: Downloaded in 5.498s
evelyn777-chai-sft-3b-v4-v1-uploader: creating bucket guanaco-vllm-models
evelyn777-chai-sft-3b-v4-v1-uploader: /usr/lib/python3/dist-packages/S3/BaseUtils.py:56: SyntaxWarning: invalid escape sequence '\.'
evelyn777-chai-sft-3b-v4-v1-uploader:   RE_S3_DATESTRING = re.compile('\.[0-9]*(?:[Z\\-\\+]*?)')
evelyn777-chai-sft-3b-v4-v1-uploader: /usr/lib/python3/dist-packages/S3/BaseUtils.py:57: SyntaxWarning: invalid escape sequence '\s'
evelyn777-chai-sft-3b-v4-v1-uploader:   RE_XML_NAMESPACE = re.compile(b'^(<?[^>]+?>\s*|\s*)(<\w+) xmlns=[\'"](https?://[^\'"]+)[\'"]', re.MULTILINE)
evelyn777-chai-sft-3b-v4-v1-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:240: SyntaxWarning: invalid escape sequence '\.'
evelyn777-chai-sft-3b-v4-v1-uploader:   invalid = re.search("([^a-z0-9\.-])", bucket, re.UNICODE)
evelyn777-chai-sft-3b-v4-v1-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:244: SyntaxWarning: invalid escape sequence '\.'
evelyn777-chai-sft-3b-v4-v1-uploader:   invalid = re.search("([^A-Za-z0-9\._-])", bucket, re.UNICODE)
evelyn777-chai-sft-3b-v4-v1-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:255: SyntaxWarning: invalid escape sequence '\.'
evelyn777-chai-sft-3b-v4-v1-uploader:   if re.search("-\.", bucket, re.UNICODE):
evelyn777-chai-sft-3b-v4-v1-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:257: SyntaxWarning: invalid escape sequence '\.'
evelyn777-chai-sft-3b-v4-v1-uploader:   if re.search("\.\.", bucket, re.UNICODE):
evelyn777-chai-sft-3b-v4-v1-uploader: /usr/lib/python3/dist-packages/S3/S3Uri.py:155: SyntaxWarning: invalid escape sequence '\w'
evelyn777-chai-sft-3b-v4-v1-uploader:   _re = re.compile("^(\w+://)?(.*)", re.UNICODE)
evelyn777-chai-sft-3b-v4-v1-uploader: /usr/lib/python3/dist-packages/S3/FileLists.py:480: SyntaxWarning: invalid escape sequence '\*'
evelyn777-chai-sft-3b-v4-v1-uploader:   wildcard_split_result = re.split("\*|\?", uri_str, maxsplit=1)
evelyn777-chai-sft-3b-v4-v1-uploader: Bucket 's3://guanaco-vllm-models/' created
evelyn777-chai-sft-3b-v4-v1-uploader: uploading /dev/shm/model_output to s3://guanaco-vllm-models/evelyn777-chai-sft-3b-v4-v1
evelyn777-chai-sft-3b-v4-v1-uploader: cp /dev/shm/model_output/added_tokens.json s3://guanaco-vllm-models/evelyn777-chai-sft-3b-v4-v1/added_tokens.json
evelyn777-chai-sft-3b-v4-v1-uploader: cp /dev/shm/model_output/config.json s3://guanaco-vllm-models/evelyn777-chai-sft-3b-v4-v1/config.json
evelyn777-chai-sft-3b-v4-v1-uploader: cp /dev/shm/model_output/generation_config.json s3://guanaco-vllm-models/evelyn777-chai-sft-3b-v4-v1/generation_config.json
evelyn777-chai-sft-3b-v4-v1-uploader: cp /dev/shm/model_output/.gitattributes s3://guanaco-vllm-models/evelyn777-chai-sft-3b-v4-v1/.gitattributes
evelyn777-chai-sft-3b-v4-v1-uploader: cp /dev/shm/model_output/chat_template.jinja s3://guanaco-vllm-models/evelyn777-chai-sft-3b-v4-v1/chat_template.jinja
evelyn777-chai-sft-3b-v4-v1-uploader: cp /dev/shm/model_output/special_tokens_map.json s3://guanaco-vllm-models/evelyn777-chai-sft-3b-v4-v1/special_tokens_map.json
evelyn777-chai-sft-3b-v4-v1-uploader: cp /dev/shm/model_output/tokenizer_config.json s3://guanaco-vllm-models/evelyn777-chai-sft-3b-v4-v1/tokenizer_config.json
evelyn777-chai-sft-3b-v4-v1-uploader: cp /dev/shm/model_output/model.safetensors.index.json s3://guanaco-vllm-models/evelyn777-chai-sft-3b-v4-v1/model.safetensors.index.json
evelyn777-chai-sft-3b-v4-v1-uploader: cp /dev/shm/model_output/vocab.json s3://guanaco-vllm-models/evelyn777-chai-sft-3b-v4-v1/vocab.json
evelyn777-chai-sft-3b-v4-v1-uploader: cp /dev/shm/model_output/merges.txt s3://guanaco-vllm-models/evelyn777-chai-sft-3b-v4-v1/merges.txt
evelyn777-chai-sft-3b-v4-v1-uploader: cp /dev/shm/model_output/tokenizer.json s3://guanaco-vllm-models/evelyn777-chai-sft-3b-v4-v1/tokenizer.json
evelyn777-chai-sft-3b-v4-v1-uploader: cp /dev/shm/model_output/model-00002-of-00002.safetensors s3://guanaco-vllm-models/evelyn777-chai-sft-3b-v4-v1/model-00002-of-00002.safetensors
HTTP Request: %s %s "%s %d %s"
evelyn777-chai-sft-3b-v4-v1-uploader: cp /dev/shm/model_output/model-00001-of-00002.safetensors s3://guanaco-vllm-models/evelyn777-chai-sft-3b-v4-v1/model-00001-of-00002.safetensors
Job evelyn777-chai-sft-3b-v4-v1-uploader completed after 82.71s with status: succeeded
Stopping job with name evelyn777-chai-sft-3b-v4-v1-uploader
Pipeline stage VLLMUploader completed in 83.52s
run pipeline stage %s
Running pipeline stage VLLMTemplater
Pipeline stage VLLMTemplater completed in 0.19s
run pipeline stage %s
Running pipeline stage VLLMDeployer
Creating inference service evelyn777-chai-sft-3b-v4-v1
Waiting for inference service evelyn777-chai-sft-3b-v4-v1 to be ready
Inference service evelyn777-chai-sft-3b-v4-v1 ready after 170.65225505828857s
Pipeline stage VLLMDeployer completed in 171.15s
run pipeline stage %s
Running pipeline stage StressChecker
Received healthy response to inference request in 0.5186254978179932s
Received healthy response to inference request in 0.7555444240570068s
Received healthy response to inference request in 0.5807886123657227s
Received healthy response to inference request in 0.6802060604095459s
Received healthy response to inference request in 0.8216593265533447s
Received healthy response to inference request in 0.8906214237213135s
Received healthy response to inference request in 0.90447998046875s
Received healthy response to inference request in 0.48292112350463867s
Received healthy response to inference request in 0.6959810256958008s
Received healthy response to inference request in 0.6284341812133789s
Received healthy response to inference request in 0.61814284324646s
Received healthy response to inference request in 0.7532618045806885s
Received healthy response to inference request in 0.7523393630981445s
Received healthy response to inference request in 0.7341828346252441s
Received healthy response to inference request in 0.8121635913848877s
Received healthy response to inference request in 1.1884100437164307s
Received healthy response to inference request in 0.6183760166168213s
Received healthy response to inference request in 0.8192169666290283s
Received healthy response to inference request in 0.8161294460296631s
Received healthy response to inference request in 0.6912734508514404s
Received healthy response to inference request in 0.7629477977752686s
Received healthy response to inference request in 0.9509081840515137s
Received healthy response to inference request in 0.576716423034668s
Received healthy response to inference request in 0.8463039398193359s
Received healthy response to inference request in 0.6536300182342529s
Received healthy response to inference request in 0.8488199710845947s
Received healthy response to inference request in 0.6883955001831055s
Received healthy response to inference request in 0.7123394012451172s
Received healthy response to inference request in 0.63700270652771s
Received healthy response to inference request in 0.5602076053619385s
30 requests
0 failed requests
5th percentile: 0.5373374462127686
10th percentile: 0.5750655412673951
20th percentile: 0.618329381942749
30th percentile: 0.64864182472229
40th percentile: 0.6901222705841065
50th percentile: 0.7232611179351807
60th percentile: 0.7541748523712158
70th percentile: 0.8133533477783202
80th percentile: 0.8265882492065431
90th percentile: 0.8920072793960572
95th percentile: 0.9300154924392698
99th percentile: 1.119534504413605
mean time: 0.7333343187967937
Pipeline stage StressChecker completed in 24.86s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyTriggerPipeline
run_pipeline:run_in_cloud %s
starting trigger_guanaco_pipeline args=%s
triggered trigger_guanaco_pipeline args=%s
Pipeline stage OfflineFamilyFriendlyTriggerPipeline completed in 0.60s
Shutdown handler de-registered
evelyn777-chai-sft-3b-v4_v1 status is now deployed due to DeploymentManager action
evelyn777-chai-sft-3b-v4_v1 status is now inactive due to auto deactivation removed underperforming models