chaiml-grpo-qwen25-3b-l_65726

developer_uid: acehao-chai

submission_id: chaiml-grpo-qwen25-3b-l_65726_v1

model_name: chaiml-grpo-qwen25-3b-l_65726_v1

model_group: ChaiML/grpo-qwen25-3b-lo

status: inactive

timestamp: 2026-02-14T20:37:31+00:00

num_battles: 10728

num_wins: 2912

celo_rating: 1127.33

family_friendly_score: 0.0

family_friendly_standard_error: 0.0

submission_type: basic

model_repo: ChaiML/grpo-qwen25-3b-lora-opus14k-chai-rm-max64-step-1591

model_architecture: Qwen2ForCausalLM

model_num_parameters: 3397011456.0

best_of: 1

max_input_tokens: 2048

max_output_tokens: 64

reward_model: default

display_name: chaiml-grpo-qwen25-3b-l_65726_v1

is_internal_developer: False

language_model: ChaiML/grpo-qwen25-3b-lora-opus14k-chai-rm-max64-step-1591

model_size: 3B

ranking_group: single

us_pacific_date: 2026-02-14

win_ratio: 0.2714392244593587

generation_params: {'temperature': 1.0, 'top_p': 1.0, 'min_p': 0.0, 'top_k': 40, 'presence_penalty': 0.0, 'frequency_penalty': 0.0, 'stopping_words': ['\n'], 'max_input_tokens': 2048, 'best_of': 1, 'max_output_tokens': 64}

formatter: {'memory_template': "{bot_name}'s Persona: {memory}\n####\n", 'prompt_template': '{prompt}\n<START>\n', 'bot_template': '{bot_name}: {message}\n', 'user_template': '{user_name}: {message}\n', 'response_template': '{bot_name}:', 'truncate_by_message': False}

Resubmit model

Shutdown handler not registered because Python interpreter is not running in the main thread
run pipeline %s
run pipeline stage %s
Running pipeline stage VLLMUploader
Starting job with name chaiml-grpo-qwen25-3b-l-65726-v1-uploader
Waiting for job on chaiml-grpo-qwen25-3b-l-65726-v1-uploader to finish
chaiml-grpo-qwen25-3b-l-65726-v1-uploader: Using quantization_mode: none
chaiml-grpo-qwen25-3b-l-65726-v1-uploader: Downloading snapshot of ChaiML/grpo-qwen25-3b-lora-opus14k-chai-rm-max64-step-1591...
chaiml-grpo-qwen25-3b-l-65726-v1-uploader: Downloaded in 4.682s
chaiml-grpo-qwen25-3b-l-65726-v1-uploader: Processed model ChaiML/grpo-qwen25-3b-lora-opus14k-chai-rm-max64-step-1591 in 7.164s
chaiml-grpo-qwen25-3b-l-65726-v1-uploader: creating bucket guanaco-vllm-models
chaiml-grpo-qwen25-3b-l-65726-v1-uploader: /usr/lib/python3/dist-packages/S3/BaseUtils.py:56: SyntaxWarning: invalid escape sequence '\.'
chaiml-grpo-qwen25-3b-l-65726-v1-uploader:   RE_S3_DATESTRING = re.compile('\.[0-9]*(?:[Z\\-\\+]*?)')
chaiml-grpo-qwen25-3b-l-65726-v1-uploader: /usr/lib/python3/dist-packages/S3/BaseUtils.py:57: SyntaxWarning: invalid escape sequence '\s'
chaiml-grpo-qwen25-3b-l-65726-v1-uploader:   RE_XML_NAMESPACE = re.compile(b'^(<?[^>]+?>\s*|\s*)(<\w+) xmlns=[\'"](https?://[^\'"]+)[\'"]', re.MULTILINE)
chaiml-grpo-qwen25-3b-l-65726-v1-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:240: SyntaxWarning: invalid escape sequence '\.'
chaiml-grpo-qwen25-3b-l-65726-v1-uploader:   invalid = re.search("([^a-z0-9\.-])", bucket, re.UNICODE)
chaiml-grpo-qwen25-3b-l-65726-v1-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:244: SyntaxWarning: invalid escape sequence '\.'
chaiml-grpo-qwen25-3b-l-65726-v1-uploader:   invalid = re.search("([^A-Za-z0-9\._-])", bucket, re.UNICODE)
chaiml-grpo-qwen25-3b-l-65726-v1-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:255: SyntaxWarning: invalid escape sequence '\.'
chaiml-grpo-qwen25-3b-l-65726-v1-uploader:   if re.search("-\.", bucket, re.UNICODE):
chaiml-grpo-qwen25-3b-l-65726-v1-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:257: SyntaxWarning: invalid escape sequence '\.'
chaiml-grpo-qwen25-3b-l-65726-v1-uploader:   if re.search("\.\.", bucket, re.UNICODE):
chaiml-grpo-qwen25-3b-l-65726-v1-uploader: /usr/lib/python3/dist-packages/S3/S3Uri.py:155: SyntaxWarning: invalid escape sequence '\w'
chaiml-grpo-qwen25-3b-l-65726-v1-uploader:   _re = re.compile("^(\w+://)?(.*)", re.UNICODE)
chaiml-grpo-qwen25-3b-l-65726-v1-uploader: /usr/lib/python3/dist-packages/S3/FileLists.py:480: SyntaxWarning: invalid escape sequence '\*'
chaiml-grpo-qwen25-3b-l-65726-v1-uploader:   wildcard_split_result = re.split("\*|\?", uri_str, maxsplit=1)
chaiml-grpo-qwen25-3b-l-65726-v1-uploader: Bucket 's3://guanaco-vllm-models/' created
chaiml-grpo-qwen25-3b-l-65726-v1-uploader: uploading /dev/shm/model_output to s3://guanaco-vllm-models/chaiml-grpo-qwen25-3b-l-65726-v1/default
chaiml-grpo-qwen25-3b-l-65726-v1-uploader: cp /dev/shm/model_output/.gitattributes s3://guanaco-vllm-models/chaiml-grpo-qwen25-3b-l-65726-v1/default/.gitattributes
chaiml-grpo-qwen25-3b-l-65726-v1-uploader: cp /dev/shm/model_output/added_tokens.json s3://guanaco-vllm-models/chaiml-grpo-qwen25-3b-l-65726-v1/default/added_tokens.json
chaiml-grpo-qwen25-3b-l-65726-v1-uploader: cp /dev/shm/model_output/generation_config.json s3://guanaco-vllm-models/chaiml-grpo-qwen25-3b-l-65726-v1/default/generation_config.json
chaiml-grpo-qwen25-3b-l-65726-v1-uploader: cp /dev/shm/model_output/config.json s3://guanaco-vllm-models/chaiml-grpo-qwen25-3b-l-65726-v1/default/config.json
chaiml-grpo-qwen25-3b-l-65726-v1-uploader: cp /dev/shm/model_output/chat_template.jinja s3://guanaco-vllm-models/chaiml-grpo-qwen25-3b-l-65726-v1/default/chat_template.jinja
chaiml-grpo-qwen25-3b-l-65726-v1-uploader: cp /dev/shm/model_output/tokenizer_config.json s3://guanaco-vllm-models/chaiml-grpo-qwen25-3b-l-65726-v1/default/tokenizer_config.json
chaiml-grpo-qwen25-3b-l-65726-v1-uploader: cp /dev/shm/model_output/model.safetensors.index.json s3://guanaco-vllm-models/chaiml-grpo-qwen25-3b-l-65726-v1/default/model.safetensors.index.json
chaiml-grpo-qwen25-3b-l-65726-v1-uploader: cp /dev/shm/model_output/special_tokens_map.json s3://guanaco-vllm-models/chaiml-grpo-qwen25-3b-l-65726-v1/default/special_tokens_map.json
chaiml-grpo-qwen25-3b-l-65726-v1-uploader: cp /dev/shm/model_output/args.json s3://guanaco-vllm-models/chaiml-grpo-qwen25-3b-l-65726-v1/default/args.json
chaiml-grpo-qwen25-3b-l-65726-v1-uploader: cp /dev/shm/model_output/vocab.json s3://guanaco-vllm-models/chaiml-grpo-qwen25-3b-l-65726-v1/default/vocab.json
chaiml-grpo-qwen25-3b-l-65726-v1-uploader: cp /dev/shm/model_output/merges.txt s3://guanaco-vllm-models/chaiml-grpo-qwen25-3b-l-65726-v1/default/merges.txt
chaiml-grpo-qwen25-3b-l-65726-v1-uploader: cp /dev/shm/model_output/tokenizer.json s3://guanaco-vllm-models/chaiml-grpo-qwen25-3b-l-65726-v1/default/tokenizer.json
Failed to get response for submission chaiml-mistral-24b-2048_15988_v1: ('http://chaiml-mistral-24b-2048-15988-v1-predictor.tenant-chaiml-guanaco.kchai-coreweave-us-east-04a.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', '')
chaiml-grpo-qwen25-3b-l-65726-v1-uploader: cp /dev/shm/model_output/model-00002-of-00002.safetensors s3://guanaco-vllm-models/chaiml-grpo-qwen25-3b-l-65726-v1/default/model-00002-of-00002.safetensors
chaiml-grpo-qwen25-3b-l-65726-v1-uploader: cp /dev/shm/model_output/model-00001-of-00002.safetensors s3://guanaco-vllm-models/chaiml-grpo-qwen25-3b-l-65726-v1/default/model-00001-of-00002.safetensors
Job chaiml-grpo-qwen25-3b-l-65726-v1-uploader completed after 63.17s with status: succeeded
Stopping job with name chaiml-grpo-qwen25-3b-l-65726-v1-uploader
Pipeline stage VLLMUploader completed in 63.66s
run pipeline stage %s
Running pipeline stage VLLMTemplater
Pipeline stage VLLMTemplater completed in 0.15s
run pipeline stage %s
Running pipeline stage VLLMDeployer
Creating inference service chaiml-grpo-qwen25-3b-l-65726-v1
Waiting for inference service chaiml-grpo-qwen25-3b-l-65726-v1 to be ready
Failed to get response for submission chaiml-mistral-24b-2048_54327_v6: ('http://chaiml-mistral-24b-2048-54327-v6-predictor.tenant-chaiml-guanaco.kchai-coreweave-us-east-04a.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', '')
Failed to get response for submission chaiml-mistral-24b-2048-_2678_v3: ('http://chaiml-mistral-24b-2048-2678-v3-predictor.tenant-chaiml-guanaco.kchai-coreweave-us-east-04a.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', '')
Inference service chaiml-grpo-qwen25-3b-l-65726-v1 ready after 160.84521508216858s
Pipeline stage VLLMDeployer completed in 161.87s
run pipeline stage %s
Running pipeline stage StressChecker
Received healthy response to inference request in 0.3938126564025879s
Received healthy response to inference request in 0.4377148151397705s
Received healthy response to inference request in 0.31160402297973633s
Received healthy response to inference request in 0.6979491710662842s
Received healthy response to inference request in 0.26468992233276367s
Received healthy response to inference request in 0.17251205444335938s
Received healthy response to inference request in 0.4508209228515625s
Received healthy response to inference request in 0.6034982204437256s
Received healthy response to inference request in 0.4790463447570801s
Received healthy response to inference request in 0.2825772762298584s
Received healthy response to inference request in 0.29592084884643555s
Received healthy response to inference request in 0.18682479858398438s
Received healthy response to inference request in 0.18647217750549316s
Received healthy response to inference request in 0.3391845226287842s
Received healthy response to inference request in 0.8204345703125s
Received healthy response to inference request in 0.15273737907409668s
Received healthy response to inference request in 0.5563411712646484s
Received healthy response to inference request in 0.6172189712524414s
Received healthy response to inference request in 0.15111160278320312s
Received healthy response to inference request in 0.2433943748474121s
Received healthy response to inference request in 0.39995288848876953s
Received healthy response to inference request in 0.4096238613128662s
Received healthy response to inference request in 0.2359762191772461s
Received healthy response to inference request in 0.35961246490478516s
Received healthy response to inference request in 0.28491663932800293s
Received healthy response to inference request in 0.41710877418518066s
Received healthy response to inference request in 0.47290539741516113s
Received healthy response to inference request in 0.27785277366638184s
Received healthy response to inference request in 0.1149594783782959s
Received healthy response to inference request in 0.38341283798217773s
30 requests
0 failed requests
5th percentile: 0.15184320211410524
10th percentile: 0.1705345869064331
20th percentile: 0.2261459350585938
30th percentile: 0.2739039182662964
40th percentile: 0.2915191650390625
50th percentile: 0.34939849376678467
60th percentile: 0.3962687492370605
70th percentile: 0.42329058647155754
80th percentile: 0.47413358688354496
90th percentile: 0.6048702955245971
95th percentile: 0.6616205811500547
99th percentile: 0.7849138045310975
mean time: 0.36667290528615315
Pipeline stage StressChecker completed in 13.97s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyTriggerPipeline
run_pipeline:run_in_cloud %s
starting trigger_guanaco_pipeline args=%s
triggered trigger_guanaco_pipeline args=%s
Pipeline stage OfflineFamilyFriendlyTriggerPipeline completed in 0.65s
Shutdown handler de-registered
chaiml-grpo-qwen25-3b-l_65726_v1 status is now deployed due to DeploymentManager action
chaiml-grpo-qwen25-3b-l_65726_v1 status is now inactive due to auto deactivation removed underperforming models