developer_uid: chai_evaluation_service
submission_id: evelyn777-chai-sft-3b-v1_v1
model_name: evelyn777-chai-sft-3b-v1_v1
model_group: evelyn777/chai-sft-3b-v1
status: inactive
timestamp: 2026-02-08T00:16:51+00:00
num_battles: 14951
num_wins: 4954
celo_rating: 1179.86
family_friendly_score: 0.0
family_friendly_standard_error: 0.0
submission_type: basic
model_repo: evelyn777/chai-sft-3b-v1
model_architecture: Qwen2ForCausalLM
model_num_parameters: 3397011456.0
best_of: 8
max_input_tokens: 2048
max_output_tokens: 64
reward_model: default
display_name: evelyn777-chai-sft-3b-v1_v1
is_internal_developer: True
language_model: evelyn777/chai-sft-3b-v1
model_size: 3B
ranking_group: single
us_pacific_date: 2026-02-07
win_ratio: 0.33134907364055916
generation_params: {'temperature': 0.85, 'top_p': 0.9, 'min_p': 0.0, 'top_k': 40, 'presence_penalty': 0.2, 'frequency_penalty': 0.3, 'stopping_words': ['\n'], 'max_input_tokens': 2048, 'best_of': 8, 'max_output_tokens': 64}
formatter: {'memory_template': '<|im_start|>system\n{memory}<|im_end|>\n', 'prompt_template': '<|im_start|>user\n{prompt}<|im_end|>\n', 'bot_template': '<|im_start|>assistant\n{bot_name}: {message}<|im_end|>\n', 'user_template': '<|im_start|>user\n{user_name}: {message}<|im_end|>\n', 'response_template': '<|im_start|>assistant\n{bot_name}:', 'truncate_by_message': True}
Resubmit model
Shutdown handler not registered because Python interpreter is not running in the main thread
run pipeline %s
run pipeline stage %s
Running pipeline stage VLLMUploader
Starting job with name evelyn777-chai-sft-3b-v1-v1-uploader
Waiting for job on evelyn777-chai-sft-3b-v1-v1-uploader to finish
evelyn777-chai-sft-3b-v1-v1-uploader: Using quantization_mode: none
evelyn777-chai-sft-3b-v1-v1-uploader: Downloading snapshot of evelyn777/chai-sft-3b-v1...
evelyn777-chai-sft-3b-v1-v1-uploader: 
Fetching 13 files:   0%|          | 0/13 [00:00<?, ?it/s]
Fetching 13 files:   8%|▊         | 1/13 [00:00<00:03,  3.70it/s]
Fetching 13 files:  46%|████▌     | 6/13 [00:00<00:00, 17.14it/s]
Fetching 13 files:  69%|██████▉   | 9/13 [00:04<00:02,  1.65it/s]
Fetching 13 files: 100%|██████████| 13/13 [00:04<00:00,  2.95it/s]
evelyn777-chai-sft-3b-v1-v1-uploader: Downloaded in 4.550s
evelyn777-chai-sft-3b-v1-v1-uploader: Processed model evelyn777/chai-sft-3b-v1 in 7.016s
evelyn777-chai-sft-3b-v1-v1-uploader: creating bucket guanaco-vllm-models
evelyn777-chai-sft-3b-v1-v1-uploader: /usr/lib/python3/dist-packages/S3/BaseUtils.py:56: SyntaxWarning: invalid escape sequence '\.'
evelyn777-chai-sft-3b-v1-v1-uploader:   RE_S3_DATESTRING = re.compile('\.[0-9]*(?:[Z\\-\\+]*?)')
evelyn777-chai-sft-3b-v1-v1-uploader: /usr/lib/python3/dist-packages/S3/BaseUtils.py:57: SyntaxWarning: invalid escape sequence '\s'
evelyn777-chai-sft-3b-v1-v1-uploader:   RE_XML_NAMESPACE = re.compile(b'^(<?[^>]+?>\s*|\s*)(<\w+) xmlns=[\'"](https?://[^\'"]+)[\'"]', re.MULTILINE)
evelyn777-chai-sft-3b-v1-v1-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:240: SyntaxWarning: invalid escape sequence '\.'
evelyn777-chai-sft-3b-v1-v1-uploader:   invalid = re.search("([^a-z0-9\.-])", bucket, re.UNICODE)
evelyn777-chai-sft-3b-v1-v1-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:244: SyntaxWarning: invalid escape sequence '\.'
evelyn777-chai-sft-3b-v1-v1-uploader:   invalid = re.search("([^A-Za-z0-9\._-])", bucket, re.UNICODE)
evelyn777-chai-sft-3b-v1-v1-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:255: SyntaxWarning: invalid escape sequence '\.'
evelyn777-chai-sft-3b-v1-v1-uploader:   if re.search("-\.", bucket, re.UNICODE):
evelyn777-chai-sft-3b-v1-v1-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:257: SyntaxWarning: invalid escape sequence '\.'
evelyn777-chai-sft-3b-v1-v1-uploader:   if re.search("\.\.", bucket, re.UNICODE):
evelyn777-chai-sft-3b-v1-v1-uploader: /usr/lib/python3/dist-packages/S3/S3Uri.py:155: SyntaxWarning: invalid escape sequence '\w'
evelyn777-chai-sft-3b-v1-v1-uploader:   _re = re.compile("^(\w+://)?(.*)", re.UNICODE)
evelyn777-chai-sft-3b-v1-v1-uploader: /usr/lib/python3/dist-packages/S3/FileLists.py:480: SyntaxWarning: invalid escape sequence '\*'
evelyn777-chai-sft-3b-v1-v1-uploader:   wildcard_split_result = re.split("\*|\?", uri_str, maxsplit=1)
evelyn777-chai-sft-3b-v1-v1-uploader: Bucket 's3://guanaco-vllm-models/' created
evelyn777-chai-sft-3b-v1-v1-uploader: uploading /dev/shm/model_output to s3://guanaco-vllm-models/evelyn777-chai-sft-3b-v1-v1
evelyn777-chai-sft-3b-v1-v1-uploader: cp /dev/shm/model_output/.gitattributes s3://guanaco-vllm-models/evelyn777-chai-sft-3b-v1-v1/.gitattributes
evelyn777-chai-sft-3b-v1-v1-uploader: cp /dev/shm/model_output/generation_config.json s3://guanaco-vllm-models/evelyn777-chai-sft-3b-v1-v1/generation_config.json
evelyn777-chai-sft-3b-v1-v1-uploader: cp /dev/shm/model_output/tokenizer_config.json s3://guanaco-vllm-models/evelyn777-chai-sft-3b-v1-v1/tokenizer_config.json
evelyn777-chai-sft-3b-v1-v1-uploader: cp /dev/shm/model_output/special_tokens_map.json s3://guanaco-vllm-models/evelyn777-chai-sft-3b-v1-v1/special_tokens_map.json
evelyn777-chai-sft-3b-v1-v1-uploader: cp /dev/shm/model_output/added_tokens.json s3://guanaco-vllm-models/evelyn777-chai-sft-3b-v1-v1/added_tokens.json
evelyn777-chai-sft-3b-v1-v1-uploader: cp /dev/shm/model_output/config.json s3://guanaco-vllm-models/evelyn777-chai-sft-3b-v1-v1/config.json
evelyn777-chai-sft-3b-v1-v1-uploader: cp /dev/shm/model_output/chat_template.jinja s3://guanaco-vllm-models/evelyn777-chai-sft-3b-v1-v1/chat_template.jinja
evelyn777-chai-sft-3b-v1-v1-uploader: cp /dev/shm/model_output/model.safetensors.index.json s3://guanaco-vllm-models/evelyn777-chai-sft-3b-v1-v1/model.safetensors.index.json
evelyn777-chai-sft-3b-v1-v1-uploader: cp /dev/shm/model_output/merges.txt s3://guanaco-vllm-models/evelyn777-chai-sft-3b-v1-v1/merges.txt
evelyn777-chai-sft-3b-v1-v1-uploader: cp /dev/shm/model_output/vocab.json s3://guanaco-vllm-models/evelyn777-chai-sft-3b-v1-v1/vocab.json
evelyn777-chai-sft-3b-v1-v1-uploader: cp /dev/shm/model_output/tokenizer.json s3://guanaco-vllm-models/evelyn777-chai-sft-3b-v1-v1/tokenizer.json
evelyn777-chai-sft-3b-v1-v1-uploader: cp /dev/shm/model_output/model-00002-of-00002.safetensors s3://guanaco-vllm-models/evelyn777-chai-sft-3b-v1-v1/model-00002-of-00002.safetensors
evelyn777-chai-sft-3b-v1-v1-uploader: cp /dev/shm/model_output/model-00001-of-00002.safetensors s3://guanaco-vllm-models/evelyn777-chai-sft-3b-v1-v1/model-00001-of-00002.safetensors
Job evelyn777-chai-sft-3b-v1-v1-uploader completed after 84.02s with status: succeeded
Stopping job with name evelyn777-chai-sft-3b-v1-v1-uploader
Pipeline stage VLLMUploader completed in 95.04s
run pipeline stage %s
Running pipeline stage VLLMTemplater
Pipeline stage VLLMTemplater completed in 1.25s
run pipeline stage %s
Running pipeline stage VLLMDeployer
Creating inference service evelyn777-chai-sft-3b-v1-v1
Waiting for inference service evelyn777-chai-sft-3b-v1-v1 to be ready
Inference service evelyn777-chai-sft-3b-v1-v1 ready after 170.69929003715515s
Pipeline stage VLLMDeployer completed in 176.15s
run pipeline stage %s
Running pipeline stage StressChecker
Received healthy response to inference request in 1.9504179954528809s
Received healthy response to inference request in 1.033876657485962s
Received healthy response to inference request in 1.2058637142181396s
Received healthy response to inference request in 1.7100348472595215s
Received healthy response to inference request in 2.089837074279785s
Received healthy response to inference request in 2.144584894180298s
Received healthy response to inference request in 3.029435157775879s
Received healthy response to inference request in 2.741237163543701s
Received healthy response to inference request in 1.6425864696502686s
Received healthy response to inference request in 2.0662190914154053s
Received healthy response to inference request in 1.9925987720489502s
Received healthy response to inference request in 1.0076611042022705s
Received healthy response to inference request in 1.4737255573272705s
Received healthy response to inference request in 1.6674258708953857s
Unable to record family friendly update due to error: Invalid JSON input: Expecting value: line 1 column 1 (char 0)
Received healthy response to inference request in 0.742250919342041s
Received healthy response to inference request in 0.806020975112915s
Received healthy response to inference request in 1.6739411354064941s
Received healthy response to inference request in 0.6905453205108643s
Received healthy response to inference request in 1.1452667713165283s
Received healthy response to inference request in 0.996443510055542s
Received healthy response to inference request in 1.1951522827148438s
Received healthy response to inference request in 0.7605102062225342s
Received healthy response to inference request in 0.976478099822998s
Received healthy response to inference request in 1.2966883182525635s
Received healthy response to inference request in 0.6055254936218262s
Received healthy response to inference request in 0.7221732139587402s
Received healthy response to inference request in 0.6649551391601562s
Received healthy response to inference request in 0.5910332202911377s
Received healthy response to inference request in 0.7217741012573242s
Received healthy response to inference request in 0.48783135414123535s
30 requests
0 failed requests
5th percentile: 0.5975547432899475
10th percentile: 0.6590121746063232
20th percentile: 0.7220933914184571
30th percentile: 0.7923677444458007
40th percentile: 1.003174066543579
50th percentile: 1.170209527015686
60th percentile: 1.3675032138824461
70th percentile: 1.6693804502487182
80th percentile: 1.958854150772095
90th percentile: 2.0953118562698365
95th percentile: 2.472743642330168
99th percentile: 2.9458577394485475
mean time: 1.3277364810307821
Pipeline stage StressChecker completed in 72.52s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyTriggerPipeline
run_pipeline:run_in_cloud %s
starting trigger_guanaco_pipeline args=%s
triggered trigger_guanaco_pipeline args=%s
Pipeline stage OfflineFamilyFriendlyTriggerPipeline completed in 0.58s
Shutdown handler de-registered
evelyn777-chai-sft-3b-v1_v1 status is now deployed due to DeploymentManager action
evelyn777-chai-sft-3b-v1_v1 status is now inactive due to auto deactivation removed underperforming models