chaiml-grpo-q3b-merged-_26723

developer_uid: acehao-chai

submission_id: chaiml-grpo-q3b-merged-_26723_v3

model_name: chaiml-grpo-q3b-merged-_26723_v3

model_group: ChaiML/grpo-q3b-merged-n

status: inactive

timestamp: 2026-02-14T20:27:30+00:00

num_battles: 11564

num_wins: 1696

celo_rating: 989.79

family_friendly_score: 0.0

family_friendly_standard_error: 0.0

submission_type: basic

model_repo: ChaiML/grpo-q3b-merged-nemo32b-step-900

model_architecture: Qwen2ForCausalLM

model_num_parameters: 3397011456.0

best_of: 1

max_input_tokens: 2048

max_output_tokens: 64

reward_model: default

display_name: chaiml-grpo-q3b-merged-_26723_v3

is_internal_developer: False

language_model: ChaiML/grpo-q3b-merged-nemo32b-step-900

model_size: 3B

ranking_group: single

us_pacific_date: 2026-02-14

win_ratio: 0.14666205465236942

generation_params: {'temperature': 1.0, 'top_p': 1.0, 'min_p': 0.0, 'top_k': 40, 'presence_penalty': 0.0, 'frequency_penalty': 0.0, 'stopping_words': ['\n'], 'max_input_tokens': 2048, 'best_of': 1, 'max_output_tokens': 64}

formatter: {'memory_template': "{bot_name}'s Persona: {memory}\n####\n", 'prompt_template': '{prompt}\n<START>\n', 'bot_template': '{bot_name}: {message}\n', 'user_template': '{user_name}: {message}\n', 'response_template': '{bot_name}:', 'truncate_by_message': False}

Resubmit model

Shutdown handler not registered because Python interpreter is not running in the main thread
run pipeline %s
run pipeline stage %s
Running pipeline stage VLLMUploader
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Starting job with name chaiml-grpo-q3b-merged-26723-v3-uploader
Waiting for job on chaiml-grpo-q3b-merged-26723-v3-uploader to finish
chaiml-grpo-q3b-merged-26723-v3-uploader: Using quantization_mode: none
chaiml-grpo-q3b-merged-26723-v3-uploader: Downloading snapshot of ChaiML/grpo-q3b-merged-nemo32b-step-900...
chaiml-grpo-q3b-merged-26723-v3-uploader: Downloaded in 5.245s
chaiml-grpo-q3b-merged-26723-v3-uploader: Processed model ChaiML/grpo-q3b-merged-nemo32b-step-900 in 8.404s
chaiml-grpo-q3b-merged-26723-v3-uploader: creating bucket guanaco-vllm-models
chaiml-grpo-q3b-merged-26723-v3-uploader: /usr/lib/python3/dist-packages/S3/BaseUtils.py:56: SyntaxWarning: invalid escape sequence '\.'
chaiml-grpo-q3b-merged-26723-v3-uploader:   RE_S3_DATESTRING = re.compile('\.[0-9]*(?:[Z\\-\\+]*?)')
chaiml-grpo-q3b-merged-26723-v3-uploader: /usr/lib/python3/dist-packages/S3/BaseUtils.py:57: SyntaxWarning: invalid escape sequence '\s'
chaiml-grpo-q3b-merged-26723-v3-uploader:   RE_XML_NAMESPACE = re.compile(b'^(<?[^>]+?>\s*|\s*)(<\w+) xmlns=[\'"](https?://[^\'"]+)[\'"]', re.MULTILINE)
chaiml-grpo-q3b-merged-26723-v3-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:240: SyntaxWarning: invalid escape sequence '\.'
chaiml-grpo-q3b-merged-26723-v3-uploader:   invalid = re.search("([^a-z0-9\.-])", bucket, re.UNICODE)
chaiml-grpo-q3b-merged-26723-v3-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:244: SyntaxWarning: invalid escape sequence '\.'
chaiml-grpo-q3b-merged-26723-v3-uploader:   invalid = re.search("([^A-Za-z0-9\._-])", bucket, re.UNICODE)
chaiml-grpo-q3b-merged-26723-v3-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:255: SyntaxWarning: invalid escape sequence '\.'
chaiml-grpo-q3b-merged-26723-v3-uploader:   if re.search("-\.", bucket, re.UNICODE):
chaiml-grpo-q3b-merged-26723-v3-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:257: SyntaxWarning: invalid escape sequence '\.'
chaiml-grpo-q3b-merged-26723-v3-uploader:   if re.search("\.\.", bucket, re.UNICODE):
chaiml-grpo-q3b-merged-26723-v3-uploader: /usr/lib/python3/dist-packages/S3/S3Uri.py:155: SyntaxWarning: invalid escape sequence '\w'
chaiml-grpo-q3b-merged-26723-v3-uploader:   _re = re.compile("^(\w+://)?(.*)", re.UNICODE)
chaiml-grpo-q3b-merged-26723-v3-uploader: /usr/lib/python3/dist-packages/S3/FileLists.py:480: SyntaxWarning: invalid escape sequence '\*'
chaiml-grpo-q3b-merged-26723-v3-uploader:   wildcard_split_result = re.split("\*|\?", uri_str, maxsplit=1)
chaiml-grpo-q3b-merged-26723-v3-uploader: Bucket 's3://guanaco-vllm-models/' created
chaiml-grpo-q3b-merged-26723-v3-uploader: uploading /dev/shm/model_output to s3://guanaco-vllm-models/chaiml-grpo-q3b-merged-26723-v3/default
chaiml-grpo-q3b-merged-26723-v3-uploader: cp /dev/shm/model_output/.gitattributes s3://guanaco-vllm-models/chaiml-grpo-q3b-merged-26723-v3/default/.gitattributes
chaiml-grpo-q3b-merged-26723-v3-uploader: cp /dev/shm/model_output/special_tokens_map.json s3://guanaco-vllm-models/chaiml-grpo-q3b-merged-26723-v3/default/special_tokens_map.json
chaiml-grpo-q3b-merged-26723-v3-uploader: cp /dev/shm/model_output/added_tokens.json s3://guanaco-vllm-models/chaiml-grpo-q3b-merged-26723-v3/default/added_tokens.json
chaiml-grpo-q3b-merged-26723-v3-uploader: cp /dev/shm/model_output/tokenizer_config.json s3://guanaco-vllm-models/chaiml-grpo-q3b-merged-26723-v3/default/tokenizer_config.json
chaiml-grpo-q3b-merged-26723-v3-uploader: cp /dev/shm/model_output/chat_template.jinja s3://guanaco-vllm-models/chaiml-grpo-q3b-merged-26723-v3/default/chat_template.jinja
chaiml-grpo-q3b-merged-26723-v3-uploader: cp /dev/shm/model_output/generation_config.json s3://guanaco-vllm-models/chaiml-grpo-q3b-merged-26723-v3/default/generation_config.json
chaiml-grpo-q3b-merged-26723-v3-uploader: cp /dev/shm/model_output/config.json s3://guanaco-vllm-models/chaiml-grpo-q3b-merged-26723-v3/default/config.json
chaiml-grpo-q3b-merged-26723-v3-uploader: cp /dev/shm/model_output/args.json s3://guanaco-vllm-models/chaiml-grpo-q3b-merged-26723-v3/default/args.json
chaiml-grpo-q3b-merged-26723-v3-uploader: cp /dev/shm/model_output/model.safetensors.index.json s3://guanaco-vllm-models/chaiml-grpo-q3b-merged-26723-v3/default/model.safetensors.index.json
chaiml-grpo-q3b-merged-26723-v3-uploader: cp /dev/shm/model_output/merges.txt s3://guanaco-vllm-models/chaiml-grpo-q3b-merged-26723-v3/default/merges.txt
chaiml-grpo-q3b-merged-26723-v3-uploader: cp /dev/shm/model_output/vocab.json s3://guanaco-vllm-models/chaiml-grpo-q3b-merged-26723-v3/default/vocab.json
chaiml-grpo-q3b-merged-26723-v3-uploader: cp /dev/shm/model_output/tokenizer.json s3://guanaco-vllm-models/chaiml-grpo-q3b-merged-26723-v3/default/tokenizer.json
chaiml-grpo-q3b-merged-26723-v3-uploader: cp /dev/shm/model_output/model-00002-of-00002.safetensors s3://guanaco-vllm-models/chaiml-grpo-q3b-merged-26723-v3/default/model-00002-of-00002.safetensors
chaiml-grpo-q3b-merged-26723-v3-uploader: cp /dev/shm/model_output/model-00001-of-00002.safetensors s3://guanaco-vllm-models/chaiml-grpo-q3b-merged-26723-v3/default/model-00001-of-00002.safetensors
Job chaiml-grpo-q3b-merged-26723-v3-uploader completed after 63.13s with status: succeeded
Stopping job with name chaiml-grpo-q3b-merged-26723-v3-uploader
Pipeline stage VLLMUploader completed in 66.21s
run pipeline stage %s
Running pipeline stage VLLMTemplater
Pipeline stage VLLMTemplater completed in 0.14s
run pipeline stage %s
Running pipeline stage VLLMDeployer
Creating inference service chaiml-grpo-q3b-merged-26723-v3
Waiting for inference service chaiml-grpo-q3b-merged-26723-v3 to be ready
Failed to get response for submission chaiml-mistral-24b-2048_54327_v6: ('http://chaiml-mistral-24b-2048-54327-v6-predictor.tenant-chaiml-guanaco.kchai-coreweave-us-east-04a.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', '')
Failed to get response for submission chaiml-mistral-24b-2048_15988_v1: ('http://chaiml-mistral-24b-2048-15988-v1-predictor.tenant-chaiml-guanaco.kchai-coreweave-us-east-04a.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', '')
HTTP Request: %s %s "%s %d %s"
Failed to get response for submission chaiml-mistral-24b-2048-_2678_v3: ('http://chaiml-mistral-24b-2048-2678-v3-predictor.tenant-chaiml-guanaco.kchai-coreweave-us-east-04a.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', '')
Failed to get response for submission chaiml-mistral-24b-2048_15988_v1: ('http://chaiml-mistral-24b-2048-15988-v1-predictor.tenant-chaiml-guanaco.kchai-coreweave-us-east-04a.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', '')
Failed to get response for submission chaiml-mistral-24b-2048_54327_v6: ('http://chaiml-mistral-24b-2048-54327-v6-predictor.tenant-chaiml-guanaco.kchai-coreweave-us-east-04a.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', '')
Inference service chaiml-grpo-q3b-merged-26723-v3 ready after 573.2986822128296s
Pipeline stage VLLMDeployer completed in 573.89s
run pipeline stage %s
Running pipeline stage StressChecker
Received healthy response to inference request in 0.37617969512939453s
Received healthy response to inference request in 0.9108388423919678s
Received healthy response to inference request in 0.4743790626525879s
Received healthy response to inference request in 0.7729980945587158s
Received healthy response to inference request in 0.2745175361633301s
Received healthy response to inference request in 0.5122084617614746s
Received healthy response to inference request in 0.1672375202178955s
Received healthy response to inference request in 0.2649416923522949s
Failed to get response for submission chaiml-mistral-24b-2048_15988_v1: ('http://chaiml-mistral-24b-2048-15988-v1-predictor.tenant-chaiml-guanaco.kchai-coreweave-us-east-04a.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', '')
Received healthy response to inference request in 0.5179064273834229s
Received healthy response to inference request in 0.2795064449310303s
Received healthy response to inference request in 0.17560315132141113s
Received healthy response to inference request in 0.2696385383605957s
Received healthy response to inference request in 0.12631607055664062s
Received healthy response to inference request in 0.296581506729126s
HTTP Request: %s %s "%s %d %s"
Received healthy response to inference request in 0.5687301158905029s
Received healthy response to inference request in 0.3083770275115967s
Received healthy response to inference request in 1.0182931423187256s
Received healthy response to inference request in 0.5692942142486572s
Received healthy response to inference request in 0.24155521392822266s
Received healthy response to inference request in 0.4679274559020996s
Received healthy response to inference request in 0.5653409957885742s
Received healthy response to inference request in 0.18782949447631836s
Received healthy response to inference request in 0.2390880584716797s
Received healthy response to inference request in 0.7521262168884277s
Received healthy response to inference request in 0.2683258056640625s
Received healthy response to inference request in 0.5255496501922607s
Received healthy response to inference request in 0.5605654716491699s
Received healthy response to inference request in 0.17854928970336914s
Received healthy response to inference request in 0.2451479434967041s
Received healthy response to inference request in 0.5486748218536377s
30 requests
0 failed requests
5th percentile: 0.17100205421447753
10th percentile: 0.17825467586517335
20th percentile: 0.24106178283691407
30th percentile: 0.26731057167053224
40th percentile: 0.2775108814239502
50th percentile: 0.3422783613204956
60th percentile: 0.4895108222961425
70th percentile: 0.5324872016906738
80th percentile: 0.56601881980896
90th percentile: 0.7542134046554566
95th percentile: 0.848810505867004
99th percentile: 0.9871313953399659
mean time: 0.42214093208312986
Pipeline stage StressChecker completed in 15.75s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyTriggerPipeline
run_pipeline:run_in_cloud %s
starting trigger_guanaco_pipeline args=%s
triggered trigger_guanaco_pipeline args=%s
Pipeline stage OfflineFamilyFriendlyTriggerPipeline completed in 0.65s
Shutdown handler de-registered
chaiml-grpo-q3b-merged-_26723_v3 status is now deployed due to DeploymentManager action
chaiml-grpo-q3b-merged-_26723_v3 status is now inactive due to auto deactivation removed underperforming models