developer_uid: chai_evaluation_service
submission_id: evelyn777-chai-sft-3b-v4_v2
model_name: evelyn777-chai-sft-3b-v4_v2
model_group: evelyn777/chai-sft-3b-v4
status: inactive
timestamp: 2026-02-08T07:47:00+00:00
num_battles: 11119
num_wins: 3992
celo_rating: 1202.15
family_friendly_score: 0.0
family_friendly_standard_error: 0.0
submission_type: basic
model_repo: evelyn777/chai-sft-3b-v4
model_architecture: Qwen2ForCausalLM
model_num_parameters: 3397011456.0
best_of: 8
max_input_tokens: 2048
max_output_tokens: 64
reward_model: default
display_name: evelyn777-chai-sft-3b-v4_v2
is_internal_developer: True
language_model: evelyn777/chai-sft-3b-v4
model_size: 3B
ranking_group: single
us_pacific_date: 2026-02-07
win_ratio: 0.359025092184549
generation_params: {'temperature': 0.7, 'top_p': 0.9, 'min_p': 0.0, 'top_k': 40, 'presence_penalty': 0.0, 'frequency_penalty': 0.0, 'stopping_words': ['\n'], 'max_input_tokens': 2048, 'best_of': 8, 'max_output_tokens': 64}
formatter: {'memory_template': '<|im_start|>system\n{memory}<|im_end|>\n', 'prompt_template': '<|im_start|>user\n{prompt}<|im_end|>\n', 'bot_template': '<|im_start|>assistant\n{bot_name}: {message}<|im_end|>\n', 'user_template': '<|im_start|>user\n{user_name}: {message}<|im_end|>\n', 'response_template': '<|im_start|>assistant\n{bot_name}:', 'truncate_by_message': True}
Resubmit model
Shutdown handler not registered because Python interpreter is not running in the main thread
run pipeline %s
run pipeline stage %s
Running pipeline stage VLLMUploader
Starting job with name evelyn777-chai-sft-3b-v4-v2-uploader
Waiting for job on evelyn777-chai-sft-3b-v4-v2-uploader to finish
HTTP Request: %s %s "%s %d %s"
evelyn777-chai-sft-3b-v4-v2-uploader: Using quantization_mode: none
evelyn777-chai-sft-3b-v4-v2-uploader: Downloading snapshot of evelyn777/chai-sft-3b-v4...
evelyn777-chai-sft-3b-v4-v2-uploader: 
Fetching 13 files:   0%|          | 0/13 [00:00<?, ?it/s]
Fetching 13 files:   8%|▊         | 1/13 [00:00<00:03,  3.51it/s]
Fetching 13 files:  54%|█████▍    | 7/13 [00:04<00:03,  1.53it/s]
Fetching 13 files: 100%|██████████| 13/13 [00:04<00:00,  2.91it/s]
evelyn777-chai-sft-3b-v4-v2-uploader: Downloaded in 4.590s
evelyn777-chai-sft-3b-v4-v2-uploader: Processed model evelyn777/chai-sft-3b-v4 in 6.915s
evelyn777-chai-sft-3b-v4-v2-uploader: creating bucket guanaco-vllm-models
evelyn777-chai-sft-3b-v4-v2-uploader: /usr/lib/python3/dist-packages/S3/BaseUtils.py:56: SyntaxWarning: invalid escape sequence '\.'
evelyn777-chai-sft-3b-v4-v2-uploader:   RE_S3_DATESTRING = re.compile('\.[0-9]*(?:[Z\\-\\+]*?)')
evelyn777-chai-sft-3b-v4-v2-uploader: /usr/lib/python3/dist-packages/S3/BaseUtils.py:57: SyntaxWarning: invalid escape sequence '\s'
evelyn777-chai-sft-3b-v4-v2-uploader:   RE_XML_NAMESPACE = re.compile(b'^(<?[^>]+?>\s*|\s*)(<\w+) xmlns=[\'"](https?://[^\'"]+)[\'"]', re.MULTILINE)
evelyn777-chai-sft-3b-v4-v2-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:240: SyntaxWarning: invalid escape sequence '\.'
evelyn777-chai-sft-3b-v4-v2-uploader:   invalid = re.search("([^a-z0-9\.-])", bucket, re.UNICODE)
evelyn777-chai-sft-3b-v4-v2-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:244: SyntaxWarning: invalid escape sequence '\.'
evelyn777-chai-sft-3b-v4-v2-uploader:   invalid = re.search("([^A-Za-z0-9\._-])", bucket, re.UNICODE)
evelyn777-chai-sft-3b-v4-v2-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:255: SyntaxWarning: invalid escape sequence '\.'
evelyn777-chai-sft-3b-v4-v2-uploader:   if re.search("-\.", bucket, re.UNICODE):
evelyn777-chai-sft-3b-v4-v2-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:257: SyntaxWarning: invalid escape sequence '\.'
evelyn777-chai-sft-3b-v4-v2-uploader:   if re.search("\.\.", bucket, re.UNICODE):
evelyn777-chai-sft-3b-v4-v2-uploader: /usr/lib/python3/dist-packages/S3/S3Uri.py:155: SyntaxWarning: invalid escape sequence '\w'
evelyn777-chai-sft-3b-v4-v2-uploader:   _re = re.compile("^(\w+://)?(.*)", re.UNICODE)
evelyn777-chai-sft-3b-v4-v2-uploader: /usr/lib/python3/dist-packages/S3/FileLists.py:480: SyntaxWarning: invalid escape sequence '\*'
evelyn777-chai-sft-3b-v4-v2-uploader:   wildcard_split_result = re.split("\*|\?", uri_str, maxsplit=1)
evelyn777-chai-sft-3b-v4-v2-uploader: Bucket 's3://guanaco-vllm-models/' created
evelyn777-chai-sft-3b-v4-v2-uploader: uploading /dev/shm/model_output to s3://guanaco-vllm-models/evelyn777-chai-sft-3b-v4-v2
evelyn777-chai-sft-3b-v4-v2-uploader: cp /dev/shm/model_output/.gitattributes s3://guanaco-vllm-models/evelyn777-chai-sft-3b-v4-v2/.gitattributes
evelyn777-chai-sft-3b-v4-v2-uploader: cp /dev/shm/model_output/added_tokens.json s3://guanaco-vllm-models/evelyn777-chai-sft-3b-v4-v2/added_tokens.json
evelyn777-chai-sft-3b-v4-v2-uploader: cp /dev/shm/model_output/special_tokens_map.json s3://guanaco-vllm-models/evelyn777-chai-sft-3b-v4-v2/special_tokens_map.json
evelyn777-chai-sft-3b-v4-v2-uploader: cp /dev/shm/model_output/chat_template.jinja s3://guanaco-vllm-models/evelyn777-chai-sft-3b-v4-v2/chat_template.jinja
evelyn777-chai-sft-3b-v4-v2-uploader: cp /dev/shm/model_output/model-00002-of-00002.safetensors s3://guanaco-vllm-models/evelyn777-chai-sft-3b-v4-v2/model-00002-of-00002.safetensors
evelyn777-chai-sft-3b-v4-v2-uploader: cp /dev/shm/model_output/model-00001-of-00002.safetensors s3://guanaco-vllm-models/evelyn777-chai-sft-3b-v4-v2/model-00001-of-00002.safetensors
Job evelyn777-chai-sft-3b-v4-v2-uploader completed after 82.9s with status: succeeded
Stopping job with name evelyn777-chai-sft-3b-v4-v2-uploader
Pipeline stage VLLMUploader completed in 83.36s
run pipeline stage %s
Running pipeline stage VLLMTemplater
Pipeline stage VLLMTemplater completed in 0.15s
run pipeline stage %s
Running pipeline stage VLLMDeployer
Creating inference service evelyn777-chai-sft-3b-v4-v2
Waiting for inference service evelyn777-chai-sft-3b-v4-v2 to be ready
HTTP Request: %s %s "%s %d %s"
Inference service evelyn777-chai-sft-3b-v4-v2 ready after 171.01225566864014s
Pipeline stage VLLMDeployer completed in 171.49s
run pipeline stage %s
Running pipeline stage StressChecker
Received healthy response to inference request in 0.9602057933807373s
Received healthy response to inference request in 1.0291204452514648s
Received healthy response to inference request in 0.8971555233001709s
Received healthy response to inference request in 0.7289533615112305s
Received healthy response to inference request in 0.6958334445953369s
Received healthy response to inference request in 0.6308276653289795s
Received healthy response to inference request in 0.7356302738189697s
Received healthy response to inference request in 0.724247932434082s
Received healthy response to inference request in 0.8101773262023926s
Received healthy response to inference request in 1.4220561981201172s
Received healthy response to inference request in 0.4966440200805664s
Received healthy response to inference request in 1.29837965965271s
Received healthy response to inference request in 0.8521590232849121s
Received healthy response to inference request in 0.7087531089782715s
Received healthy response to inference request in 1.0864324569702148s
Received healthy response to inference request in 0.7987871170043945s
Received healthy response to inference request in 0.5917394161224365s
Received healthy response to inference request in 0.6482923030853271s
Received healthy response to inference request in 0.6198761463165283s
Received healthy response to inference request in 0.7820534706115723s
Received healthy response to inference request in 0.5450835227966309s
Received healthy response to inference request in 0.7124719619750977s
Received healthy response to inference request in 0.5146927833557129s
Received healthy response to inference request in 0.5821154117584229s
Received healthy response to inference request in 0.5918927192687988s
Received healthy response to inference request in 0.6268715858459473s
Received healthy response to inference request in 0.44579148292541504s
Received healthy response to inference request in 0.8758807182312012s
Received healthy response to inference request in 0.9810261726379395s
Received healthy response to inference request in 0.9781739711761475s
30 requests
0 failed requests
5th percentile: 0.5047659635543823
10th percentile: 0.5420444488525391
20th percentile: 0.5918620586395263
30th percentile: 0.6296408414840698
40th percentile: 0.7035852432250976
50th percentile: 0.7266006469726562
60th percentile: 0.7887469291687011
70th percentile: 0.8592755317687988
80th percentile: 0.9637994289398194
90th percentile: 1.03485164642334
95th percentile: 1.2030034184455864
99th percentile: 1.3861900019645692
mean time: 0.7790441672007243
Pipeline stage StressChecker completed in 26.50s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyTriggerPipeline
run_pipeline:run_in_cloud %s
starting trigger_guanaco_pipeline args=%s
triggered trigger_guanaco_pipeline args=%s
Pipeline stage OfflineFamilyFriendlyTriggerPipeline completed in 0.61s
Shutdown handler de-registered
evelyn777-chai-sft-3b-v4_v2 status is now deployed due to DeploymentManager action
evelyn777-chai-sft-3b-v4_v2 status is now inactive due to auto deactivation removed underperforming models