developer_uid: chai_evaluation_service
submission_id: evelyn777-chai-sft-3b-v3_v2
model_name: evelyn777-chai-sft-3b-v3_v2
model_group: evelyn777/chai-sft-3b-v3
status: inactive
timestamp: 2026-02-08T07:26:31+00:00
num_battles: 12086
num_wins: 4406
celo_rating: 1206.91
family_friendly_score: 0.0
family_friendly_standard_error: 0.0
submission_type: basic
model_repo: evelyn777/chai-sft-3b-v3
model_architecture: Qwen2ForCausalLM
model_num_parameters: 3397011456.0
best_of: 8
max_input_tokens: 2048
max_output_tokens: 64
reward_model: default
display_name: evelyn777-chai-sft-3b-v3_v2
is_internal_developer: True
language_model: evelyn777/chai-sft-3b-v3
model_size: 3B
ranking_group: single
us_pacific_date: 2026-02-07
win_ratio: 0.3645540294555684
generation_params: {'temperature': 0.7, 'top_p': 0.9, 'min_p': 0.0, 'top_k': 40, 'presence_penalty': 0.0, 'frequency_penalty': 0.0, 'stopping_words': ['\n'], 'max_input_tokens': 2048, 'best_of': 8, 'max_output_tokens': 64}
formatter: {'memory_template': '<|im_start|>system\n{memory}<|im_end|>\n', 'prompt_template': '<|im_start|>user\n{prompt}<|im_end|>\n', 'bot_template': '<|im_start|>assistant\n{bot_name}: {message}<|im_end|>\n', 'user_template': '<|im_start|>user\n{user_name}: {message}<|im_end|>\n', 'response_template': '<|im_start|>assistant\n{bot_name}:', 'truncate_by_message': True}
Resubmit model
Shutdown handler not registered because Python interpreter is not running in the main thread
run pipeline %s
run pipeline stage %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Running pipeline stage VLLMUploader
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Starting job with name evelyn777-chai-sft-3b-v3-v2-uploader
Waiting for job on evelyn777-chai-sft-3b-v3-v2-uploader to finish
evelyn777-chai-sft-3b-v3-v2-uploader: Using quantization_mode: none
evelyn777-chai-sft-3b-v3-v2-uploader: Downloading snapshot of evelyn777/chai-sft-3b-v3...
evelyn777-chai-sft-3b-v3-v2-uploader: 
Fetching 13 files:   0%|          | 0/13 [00:00<?, ?it/s]
Fetching 13 files:   8%|▊         | 1/13 [00:00<00:03,  3.52it/s]
Fetching 13 files:  54%|█████▍    | 7/13 [00:04<00:03,  1.58it/s]
Fetching 13 files: 100%|██████████| 13/13 [00:04<00:00,  3.00it/s]
evelyn777-chai-sft-3b-v3-v2-uploader: Downloaded in 4.450s
evelyn777-chai-sft-3b-v3-v2-uploader: Processed model evelyn777/chai-sft-3b-v3 in 6.939s
evelyn777-chai-sft-3b-v3-v2-uploader: creating bucket guanaco-vllm-models
evelyn777-chai-sft-3b-v3-v2-uploader: /usr/lib/python3/dist-packages/S3/BaseUtils.py:56: SyntaxWarning: invalid escape sequence '\.'
evelyn777-chai-sft-3b-v3-v2-uploader:   RE_S3_DATESTRING = re.compile('\.[0-9]*(?:[Z\\-\\+]*?)')
evelyn777-chai-sft-3b-v3-v2-uploader: /usr/lib/python3/dist-packages/S3/BaseUtils.py:57: SyntaxWarning: invalid escape sequence '\s'
evelyn777-chai-sft-3b-v3-v2-uploader:   RE_XML_NAMESPACE = re.compile(b'^(<?[^>]+?>\s*|\s*)(<\w+) xmlns=[\'"](https?://[^\'"]+)[\'"]', re.MULTILINE)
evelyn777-chai-sft-3b-v3-v2-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:240: SyntaxWarning: invalid escape sequence '\.'
evelyn777-chai-sft-3b-v3-v2-uploader:   invalid = re.search("([^a-z0-9\.-])", bucket, re.UNICODE)
evelyn777-chai-sft-3b-v3-v2-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:244: SyntaxWarning: invalid escape sequence '\.'
evelyn777-chai-sft-3b-v3-v2-uploader:   invalid = re.search("([^A-Za-z0-9\._-])", bucket, re.UNICODE)
evelyn777-chai-sft-3b-v3-v2-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:255: SyntaxWarning: invalid escape sequence '\.'
evelyn777-chai-sft-3b-v3-v2-uploader:   if re.search("-\.", bucket, re.UNICODE):
evelyn777-chai-sft-3b-v3-v2-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:257: SyntaxWarning: invalid escape sequence '\.'
evelyn777-chai-sft-3b-v3-v2-uploader:   if re.search("\.\.", bucket, re.UNICODE):
evelyn777-chai-sft-3b-v3-v2-uploader: /usr/lib/python3/dist-packages/S3/S3Uri.py:155: SyntaxWarning: invalid escape sequence '\w'
evelyn777-chai-sft-3b-v3-v2-uploader:   _re = re.compile("^(\w+://)?(.*)", re.UNICODE)
evelyn777-chai-sft-3b-v3-v2-uploader: /usr/lib/python3/dist-packages/S3/FileLists.py:480: SyntaxWarning: invalid escape sequence '\*'
evelyn777-chai-sft-3b-v3-v2-uploader:   wildcard_split_result = re.split("\*|\?", uri_str, maxsplit=1)
evelyn777-chai-sft-3b-v3-v2-uploader: Bucket 's3://guanaco-vllm-models/' created
evelyn777-chai-sft-3b-v3-v2-uploader: uploading /dev/shm/model_output to s3://guanaco-vllm-models/evelyn777-chai-sft-3b-v3-v2
evelyn777-chai-sft-3b-v3-v2-uploader: cp /dev/shm/model_output/added_tokens.json s3://guanaco-vllm-models/evelyn777-chai-sft-3b-v3-v2/added_tokens.json
evelyn777-chai-sft-3b-v3-v2-uploader: cp /dev/shm/model_output/config.json s3://guanaco-vllm-models/evelyn777-chai-sft-3b-v3-v2/config.json
evelyn777-chai-sft-3b-v3-v2-uploader: cp /dev/shm/model_output/.gitattributes s3://guanaco-vllm-models/evelyn777-chai-sft-3b-v3-v2/.gitattributes
evelyn777-chai-sft-3b-v3-v2-uploader: cp /dev/shm/model_output/tokenizer_config.json s3://guanaco-vllm-models/evelyn777-chai-sft-3b-v3-v2/tokenizer_config.json
evelyn777-chai-sft-3b-v3-v2-uploader: cp /dev/shm/model_output/special_tokens_map.json s3://guanaco-vllm-models/evelyn777-chai-sft-3b-v3-v2/special_tokens_map.json
evelyn777-chai-sft-3b-v3-v2-uploader: cp /dev/shm/model_output/chat_template.jinja s3://guanaco-vllm-models/evelyn777-chai-sft-3b-v3-v2/chat_template.jinja
evelyn777-chai-sft-3b-v3-v2-uploader: cp /dev/shm/model_output/generation_config.json s3://guanaco-vllm-models/evelyn777-chai-sft-3b-v3-v2/generation_config.json
evelyn777-chai-sft-3b-v3-v2-uploader: cp /dev/shm/model_output/model.safetensors.index.json s3://guanaco-vllm-models/evelyn777-chai-sft-3b-v3-v2/model.safetensors.index.json
evelyn777-chai-sft-3b-v3-v2-uploader: cp /dev/shm/model_output/vocab.json s3://guanaco-vllm-models/evelyn777-chai-sft-3b-v3-v2/vocab.json
evelyn777-chai-sft-3b-v3-v2-uploader: cp /dev/shm/model_output/merges.txt s3://guanaco-vllm-models/evelyn777-chai-sft-3b-v3-v2/merges.txt
evelyn777-chai-sft-3b-v3-v2-uploader: cp /dev/shm/model_output/tokenizer.json s3://guanaco-vllm-models/evelyn777-chai-sft-3b-v3-v2/tokenizer.json
evelyn777-chai-sft-3b-v3-v2-uploader: cp /dev/shm/model_output/model-00002-of-00002.safetensors s3://guanaco-vllm-models/evelyn777-chai-sft-3b-v3-v2/model-00002-of-00002.safetensors
evelyn777-chai-sft-3b-v3-v2-uploader: cp /dev/shm/model_output/model-00001-of-00002.safetensors s3://guanaco-vllm-models/evelyn777-chai-sft-3b-v3-v2/model-00001-of-00002.safetensors
Job evelyn777-chai-sft-3b-v3-v2-uploader completed after 83.63s with status: succeeded
Stopping job with name evelyn777-chai-sft-3b-v3-v2-uploader
Pipeline stage VLLMUploader completed in 84.24s
run pipeline stage %s
Running pipeline stage VLLMTemplater
Pipeline stage VLLMTemplater completed in 0.15s
run pipeline stage %s
Running pipeline stage VLLMDeployer
Creating inference service evelyn777-chai-sft-3b-v3-v2
Waiting for inference service evelyn777-chai-sft-3b-v3-v2 to be ready
HTTP Request: %s %s "%s %d %s"
Inference service evelyn777-chai-sft-3b-v3-v2 ready after 170.59794425964355s
Pipeline stage VLLMDeployer completed in 171.08s
run pipeline stage %s
Running pipeline stage StressChecker
Received healthy response to inference request in 0.8680040836334229s
Received healthy response to inference request in 0.8234279155731201s
Received healthy response to inference request in 0.8295426368713379s
Received healthy response to inference request in 0.735344648361206s
Received healthy response to inference request in 0.6953530311584473s
Received healthy response to inference request in 0.9752049446105957s
Received healthy response to inference request in 0.6495046615600586s
Received healthy response to inference request in 0.7007544040679932s
Received healthy response to inference request in 0.8149600028991699s
Received healthy response to inference request in 0.8120384216308594s
Received healthy response to inference request in 1.5230066776275635s
Received healthy response to inference request in 0.9424326419830322s
Received healthy response to inference request in 0.640587329864502s
Received healthy response to inference request in 0.5046617984771729s
Received healthy response to inference request in 0.8796498775482178s
Received healthy response to inference request in 0.4326496124267578s
Received healthy response to inference request in 0.8506271839141846s
Received healthy response to inference request in 0.6262102127075195s
Received healthy response to inference request in 1.0363831520080566s
Received healthy response to inference request in 0.6333651542663574s
Received healthy response to inference request in 0.7248950004577637s
Received healthy response to inference request in 0.627084493637085s
Received healthy response to inference request in 0.8267953395843506s
Received healthy response to inference request in 0.564023494720459s
Received healthy response to inference request in 0.7152664661407471s
Received healthy response to inference request in 0.7968082427978516s
Received healthy response to inference request in 0.6294443607330322s
Received healthy response to inference request in 1.075110673904419s
Received healthy response to inference request in 0.7922048568725586s
Received healthy response to inference request in 1.103569746017456s
30 requests
0 failed requests
5th percentile: 0.5313745617866517
10th percentile: 0.6199915409088135
20th percentile: 0.6325809955596924
30th percentile: 0.6815985202789306
40th percentile: 0.721043586730957
50th percentile: 0.7945065498352051
60th percentile: 0.81834716796875
70th percentile: 0.8358680009841918
80th percentile: 0.8922064304351809
90th percentile: 1.0402559041976929
95th percentile: 1.0907631635665893
99th percentile: 1.4013699674606326
mean time: 0.7942970355351766
Pipeline stage StressChecker completed in 26.98s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyTriggerPipeline
run_pipeline:run_in_cloud %s
starting trigger_guanaco_pipeline args=%s
triggered trigger_guanaco_pipeline args=%s
Pipeline stage OfflineFamilyFriendlyTriggerPipeline completed in 0.59s
Shutdown handler de-registered
evelyn777-chai-sft-3b-v3_v2 status is now deployed due to DeploymentManager action
evelyn777-chai-sft-3b-v3_v2 status is now inactive due to auto deactivation removed underperforming models