chaiml-ca18-c13f-linear-w01

developer_uid: chai_backend_admin

submission_id: chaiml-ca18-c13f-linear-w01_v40

model_name: chaiml-ca18-c13f-linear-w01_v40

model_group: ChaiML/ca18-c13f-linear-

status: torndown

timestamp: 2026-02-10T17:11:20+00:00

num_battles: 2915

num_wins: 1409

celo_rating: 9999.0

family_friendly_score: 0.0

family_friendly_standard_error: 0.0

submission_type: basic

model_repo: ChaiML/ca18-c13f-linear-w01

model_architecture: MistralForCausalLM

model_num_parameters: 12772070400.0

best_of: 7

max_input_tokens: 1280

max_output_tokens: 60

reward_model: default

display_name: chaiml-ca18-c13f-linear-w01_v40

ineligible_reason: max_output_tokens!=64

is_internal_developer: True

language_model: ChaiML/ca18-c13f-linear-w01

model_size: 13B

ranking_group: single

us_pacific_date: 2026-02-07

win_ratio: 0.48336192109777015

generation_params: {'temperature': 1.0, 'top_p': 0.95, 'min_p': 0.05, 'top_k': 60, 'presence_penalty': 0.15, 'frequency_penalty': 0.15, 'stopping_words': ['</s>', 'You:', '###', '\n'], 'max_input_tokens': 1280, 'best_of': 7, 'max_output_tokens': 60}

formatter: {'memory_template': '', 'prompt_template': '', 'bot_template': '{bot_name}: {message}\n', 'user_template': '{user_name}: {message}\n', 'response_template': '{bot_name}:', 'truncate_by_message': True}

Resubmit model

Shutdown handler not registered because Python interpreter is not running in the main thread
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
run pipeline %s
run pipeline stage %s
Running pipeline stage VLLMUploader
Starting job with name chaiml-ca18-c13f-linear-w01-v40-uploader
Waiting for job on chaiml-ca18-c13f-linear-w01-v40-uploader to finish
chaiml-ca18-c13f-linear-w01-v40-uploader: Using quantization_mode: none
chaiml-ca18-c13f-linear-w01-v40-uploader: Downloading snapshot of ChaiML/ca18-c13f-linear-w01...
chaiml-ca18-c13f-linear-w01-v40-uploader: Processed model ChaiML/ca18-c13f-linear-w01 in 19.854s
chaiml-ca18-c13f-linear-w01-v40-uploader: creating bucket guanaco-vllm-models
chaiml-ca18-c13f-linear-w01-v40-uploader: /usr/lib/python3/dist-packages/S3/BaseUtils.py:56: SyntaxWarning: invalid escape sequence '\.'
chaiml-ca18-c13f-linear-w01-v40-uploader:   RE_S3_DATESTRING = re.compile('\.[0-9]*(?:[Z\\-\\+]*?)')
chaiml-ca18-c13f-linear-w01-v40-uploader: /usr/lib/python3/dist-packages/S3/BaseUtils.py:57: SyntaxWarning: invalid escape sequence '\s'
chaiml-ca18-c13f-linear-w01-v40-uploader:   RE_XML_NAMESPACE = re.compile(b'^(<?[^>]+?>\s*|\s*)(<\w+) xmlns=[\'"](https?://[^\'"]+)[\'"]', re.MULTILINE)
chaiml-ca18-c13f-linear-w01-v40-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:240: SyntaxWarning: invalid escape sequence '\.'
chaiml-ca18-c13f-linear-w01-v40-uploader:   invalid = re.search("([^a-z0-9\.-])", bucket, re.UNICODE)
chaiml-ca18-c13f-linear-w01-v40-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:244: SyntaxWarning: invalid escape sequence '\.'
chaiml-ca18-c13f-linear-w01-v40-uploader:   invalid = re.search("([^A-Za-z0-9\._-])", bucket, re.UNICODE)
chaiml-ca18-c13f-linear-w01-v40-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:255: SyntaxWarning: invalid escape sequence '\.'
chaiml-ca18-c13f-linear-w01-v40-uploader:   if re.search("-\.", bucket, re.UNICODE):
chaiml-ca18-c13f-linear-w01-v40-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:257: SyntaxWarning: invalid escape sequence '\.'
chaiml-ca18-c13f-linear-w01-v40-uploader:   if re.search("\.\.", bucket, re.UNICODE):
chaiml-ca18-c13f-linear-w01-v40-uploader: /usr/lib/python3/dist-packages/S3/S3Uri.py:155: SyntaxWarning: invalid escape sequence '\w'
chaiml-ca18-c13f-linear-w01-v40-uploader:   _re = re.compile("^(\w+://)?(.*)", re.UNICODE)
chaiml-ca18-c13f-linear-w01-v40-uploader: /usr/lib/python3/dist-packages/S3/FileLists.py:480: SyntaxWarning: invalid escape sequence '\*'
chaiml-ca18-c13f-linear-w01-v40-uploader:   wildcard_split_result = re.split("\*|\?", uri_str, maxsplit=1)
chaiml-ca18-c13f-linear-w01-v40-uploader: Bucket 's3://guanaco-vllm-models/' created
chaiml-ca18-c13f-linear-w01-v40-uploader: uploading /dev/shm/model_output to s3://guanaco-vllm-models/chaiml-ca18-c13f-linear-w01-v40
chaiml-ca18-c13f-linear-w01-v40-uploader: cp /dev/shm/model_output/README.md s3://guanaco-vllm-models/chaiml-ca18-c13f-linear-w01-v40/README.md
chaiml-ca18-c13f-linear-w01-v40-uploader: cp /dev/shm/model_output/config.json s3://guanaco-vllm-models/chaiml-ca18-c13f-linear-w01-v40/config.json
chaiml-ca18-c13f-linear-w01-v40-uploader: cp /dev/shm/model_output/special_tokens_map.json s3://guanaco-vllm-models/chaiml-ca18-c13f-linear-w01-v40/special_tokens_map.json
chaiml-ca18-c13f-linear-w01-v40-uploader: cp /dev/shm/model_output/mergekit_config.yaml s3://guanaco-vllm-models/chaiml-ca18-c13f-linear-w01-v40/mergekit_config.yaml
chaiml-ca18-c13f-linear-w01-v40-uploader: cp /dev/shm/model_output/.gitattributes s3://guanaco-vllm-models/chaiml-ca18-c13f-linear-w01-v40/.gitattributes
chaiml-ca18-c13f-linear-w01-v40-uploader: cp /dev/shm/model_output/mergekit_config.yml s3://guanaco-vllm-models/chaiml-ca18-c13f-linear-w01-v40/mergekit_config.yml
chaiml-ca18-c13f-linear-w01-v40-uploader: cp /dev/shm/model_output/model.safetensors.index.json s3://guanaco-vllm-models/chaiml-ca18-c13f-linear-w01-v40/model.safetensors.index.json
chaiml-ca18-c13f-linear-w01-v40-uploader: cp /dev/shm/model_output/tokenizer_config.json s3://guanaco-vllm-models/chaiml-ca18-c13f-linear-w01-v40/tokenizer_config.json
chaiml-ca18-c13f-linear-w01-v40-uploader: cp /dev/shm/model_output/model-00001-of-00005.safetensors s3://guanaco-vllm-models/chaiml-ca18-c13f-linear-w01-v40/model-00001-of-00005.safetensors
chaiml-ca18-c13f-linear-w01-v40-uploader: cp /dev/shm/model_output/model-00005-of-00005.safetensors s3://guanaco-vllm-models/chaiml-ca18-c13f-linear-w01-v40/model-00005-of-00005.safetensors
chaiml-ca18-c13f-linear-w01-v40-uploader: cp /dev/shm/model_output/model-00004-of-00005.safetensors s3://guanaco-vllm-models/chaiml-ca18-c13f-linear-w01-v40/model-00004-of-00005.safetensors
chaiml-ca18-c13f-linear-w01-v40-uploader: cp /dev/shm/model_output/model-00003-of-00005.safetensors s3://guanaco-vllm-models/chaiml-ca18-c13f-linear-w01-v40/model-00003-of-00005.safetensors
chaiml-ca18-c13f-linear-w01-v40-uploader: cp /dev/shm/model_output/model-00002-of-00005.safetensors s3://guanaco-vllm-models/chaiml-ca18-c13f-linear-w01-v40/model-00002-of-00005.safetensors
Job chaiml-ca18-c13f-linear-w01-v40-uploader completed after 141.0s with status: succeeded
Stopping job with name chaiml-ca18-c13f-linear-w01-v40-uploader
Pipeline stage VLLMUploader completed in 145.17s
run pipeline stage %s
Running pipeline stage VLLMTemplater
Pipeline stage VLLMTemplater completed in 1.24s
run pipeline stage %s
Running pipeline stage VLLMDeployer
Creating inference service chaiml-ca18-c13f-linear-w01-v40
Waiting for inference service chaiml-ca18-c13f-linear-w01-v40 to be ready
Failed to get response for submission chaiml-llama31-mer-v2-_44570_v66: ('http://guanaco-model-mesh-load-balancer.model-mesh.k2.chaiverse.com/models/chaiml-csfs-v3-3-dpo-lr_86358_v2/predict', '{"detail":"1 validation error for RuntimeResponse\\npredictions\\n  Field required [type=missing, input_value={\'detail\': \\"503, message=...o-lr_86358_v2/predict\'\\"}, input_type=dict]\\n    For further information visit https://errors.pydantic.dev/2.12/v/missing"}')
HTTP Request: %s %s "%s %d %s"
Inference service chaiml-ca18-c13f-linear-w01-v40 ready after 311.49856877326965s
Pipeline stage VLLMDeployer completed in 317.39s
run pipeline stage %s
Running pipeline stage StressChecker
Received healthy response to inference request in 2.3569531440734863s
Received healthy response to inference request in 1.9561996459960938s
Received healthy response to inference request in 1.2647547721862793s
Received healthy response to inference request in 1.5823912620544434s
Received healthy response to inference request in 1.428206205368042s
Received healthy response to inference request in 1.2277874946594238s
Received healthy response to inference request in 1.394850730895996s
Received healthy response to inference request in 1.2701854705810547s
Received healthy response to inference request in 1.2728605270385742s
Received healthy response to inference request in 1.407989501953125s
Received healthy response to inference request in 1.5371694564819336s
Received healthy response to inference request in 1.4879019260406494s
Received healthy response to inference request in 1.7556073665618896s
Received healthy response to inference request in 1.4338386058807373s
Received healthy response to inference request in 1.4657666683197021s
Received healthy response to inference request in 1.9629356861114502s
Received healthy response to inference request in 1.2582285404205322s
Received healthy response to inference request in 1.58677339553833s
Received healthy response to inference request in 1.2541558742523193s
Received healthy response to inference request in 1.6831181049346924s
Received healthy response to inference request in 1.306185007095337s
Received healthy response to inference request in 1.234365701675415s
Received healthy response to inference request in 1.233142375946045s
Received healthy response to inference request in 1.6079325675964355s
Received healthy response to inference request in 1.2790906429290771s
Received healthy response to inference request in 1.2237529754638672s
Received healthy response to inference request in 1.4149093627929688s
Received healthy response to inference request in 1.381983995437622s
Received healthy response to inference request in 1.47210693359375s
Received healthy response to inference request in 1.3414990901947021s
30 requests
0 failed requests
5th percentile: 1.2301971912384033
10th percentile: 1.2342433691024781
20th percentile: 1.26344952583313
30th percentile: 1.2772216081619263
40th percentile: 1.3657900333404542
50th percentile: 1.4114494323730469
60th percentile: 1.446609830856323
70th percentile: 1.5026821851730345
80th percentile: 1.5910052299499513
90th percentile: 1.7756665945053103
95th percentile: 1.9599044680595399
99th percentile: 2.2426880812644963
mean time: 1.4694214344024659
Pipeline stage StressChecker completed in 52.55s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyTriggerPipeline
run_pipeline:run_in_cloud %s
starting trigger_guanaco_pipeline args=%s
triggered trigger_guanaco_pipeline args=%s
Pipeline stage OfflineFamilyFriendlyTriggerPipeline completed in 0.62s
Shutdown handler de-registered
chaiml-ca18-c13f-linear-w01_v40 status is now deployed due to DeploymentManager action
chaiml-ca18-c13f-linear-w01_v40 status is now inactive due to system request
chaiml-ca18-c13f-linear-w01_v40 status is now torndown due to DeploymentManager action