submission_id: chaiml-ca18-c13f-linear_65808_v9
developer_uid: richhx
status: inactive
model_repo: ChaiML/ca18-c13f-linear-w01-FP8
generation_params: {'temperature': 1.0, 'top_p': 0.95, 'min_p': 0.05, 'top_k': 60, 'presence_penalty': 0.15, 'frequency_penalty': 0.15, 'stopping_words': ['###', '\n', '</s>', 'You:'], 'max_input_tokens': 1280, 'best_of': 7, 'max_output_tokens': 60}
formatter: {'memory_template': '', 'prompt_template': '', 'bot_template': '{bot_name}: {message}\n', 'user_template': '{user_name}: {message}\n', 'response_template': '{bot_name}:', 'truncate_by_message': True}
timestamp: 2026-03-06T00:02:01+00:00
model_name: chaiml-ca18-c13f-linear_65808_v9
Resubmit model
Shutdown handler not registered because Python interpreter is not running in the main thread
run pipeline %s
run pipeline stage %s
Running pipeline stage VLLMUploader
Starting job with name chaiml-ca18-c13f-linear-65808-v9-uploader
Waiting for job on chaiml-ca18-c13f-linear-65808-v9-uploader to finish
chaiml-ca18-c13f-linear-65808-v9-uploader: Using quantization_mode: fp8
chaiml-ca18-c13f-linear-65808-v9-uploader: Repo ChaiML/ca18-c13f-linear-w01-FP8 already ends in FP8. Skipping...
chaiml-ca18-c13f-linear-65808-v9-uploader: Checking if ChaiML/ca18-c13f-linear-w01-FP8 already exists in ChaiML
chaiml-ca18-c13f-linear-65808-v9-uploader: Model already exists. Downloading to /dev/shm/model_output...
chaiml-ca18-c13f-linear-65808-v9-uploader: Downloading snapshot of ChaiML/ca18-c13f-linear-w01-FP8...
chaiml-ca18-c13f-linear-65808-v9-uploader: Downloaded in 7.550s
chaiml-ca18-c13f-linear-65808-v9-uploader: Processed model ChaiML/ca18-c13f-linear-w01-FP8 in 11.071s
chaiml-ca18-c13f-linear-65808-v9-uploader: creating bucket guanaco-vllm-models
chaiml-ca18-c13f-linear-65808-v9-uploader: /usr/lib/python3/dist-packages/S3/BaseUtils.py:56: SyntaxWarning: invalid escape sequence '\.'
chaiml-ca18-c13f-linear-65808-v9-uploader:   RE_S3_DATESTRING = re.compile('\.[0-9]*(?:[Z\\-\\+]*?)')
chaiml-ca18-c13f-linear-65808-v9-uploader: /usr/lib/python3/dist-packages/S3/BaseUtils.py:57: SyntaxWarning: invalid escape sequence '\s'
chaiml-ca18-c13f-linear-65808-v9-uploader:   RE_XML_NAMESPACE = re.compile(b'^(<?[^>]+?>\s*|\s*)(<\w+) xmlns=[\'"](https?://[^\'"]+)[\'"]', re.MULTILINE)
chaiml-ca18-c13f-linear-65808-v9-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:240: SyntaxWarning: invalid escape sequence '\.'
chaiml-ca18-c13f-linear-65808-v9-uploader:   invalid = re.search("([^a-z0-9\.-])", bucket, re.UNICODE)
chaiml-ca18-c13f-linear-65808-v9-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:244: SyntaxWarning: invalid escape sequence '\.'
chaiml-ca18-c13f-linear-65808-v9-uploader:   invalid = re.search("([^A-Za-z0-9\._-])", bucket, re.UNICODE)
chaiml-ca18-c13f-linear-65808-v9-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:255: SyntaxWarning: invalid escape sequence '\.'
chaiml-ca18-c13f-linear-65808-v9-uploader:   if re.search("-\.", bucket, re.UNICODE):
chaiml-ca18-c13f-linear-65808-v9-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:257: SyntaxWarning: invalid escape sequence '\.'
chaiml-ca18-c13f-linear-65808-v9-uploader:   if re.search("\.\.", bucket, re.UNICODE):
chaiml-ca18-c13f-linear-65808-v9-uploader: /usr/lib/python3/dist-packages/S3/S3Uri.py:155: SyntaxWarning: invalid escape sequence '\w'
chaiml-ca18-c13f-linear-65808-v9-uploader:   _re = re.compile("^(\w+://)?(.*)", re.UNICODE)
chaiml-ca18-c13f-linear-65808-v9-uploader: /usr/lib/python3/dist-packages/S3/FileLists.py:480: SyntaxWarning: invalid escape sequence '\*'
chaiml-ca18-c13f-linear-65808-v9-uploader:   wildcard_split_result = re.split("\*|\?", uri_str, maxsplit=1)
chaiml-ca18-c13f-linear-65808-v9-uploader: Bucket 's3://guanaco-vllm-models/' created
chaiml-ca18-c13f-linear-65808-v9-uploader: uploading /dev/shm/model_output to s3://guanaco-vllm-models/chaiml-ca18-c13f-linear-65808-v9/default
chaiml-ca18-c13f-linear-65808-v9-uploader: cp /dev/shm/model_output/chat_template.jinja s3://guanaco-vllm-models/chaiml-ca18-c13f-linear-65808-v9/default/chat_template.jinja
chaiml-ca18-c13f-linear-65808-v9-uploader: cp /dev/shm/model_output/config.json s3://guanaco-vllm-models/chaiml-ca18-c13f-linear-65808-v9/default/config.json
chaiml-ca18-c13f-linear-65808-v9-uploader: cp /dev/shm/model_output/tokenizer_config.json s3://guanaco-vllm-models/chaiml-ca18-c13f-linear-65808-v9/default/tokenizer_config.json
chaiml-ca18-c13f-linear-65808-v9-uploader: cp /dev/shm/model_output/model.safetensors.index.json s3://guanaco-vllm-models/chaiml-ca18-c13f-linear-65808-v9/default/model.safetensors.index.json
chaiml-ca18-c13f-linear-65808-v9-uploader: cp /dev/shm/model_output/special_tokens_map.json s3://guanaco-vllm-models/chaiml-ca18-c13f-linear-65808-v9/default/special_tokens_map.json
chaiml-ca18-c13f-linear-65808-v9-uploader: cp /dev/shm/model_output/recipe.yaml s3://guanaco-vllm-models/chaiml-ca18-c13f-linear-65808-v9/default/recipe.yaml
chaiml-ca18-c13f-linear-65808-v9-uploader: cp /dev/shm/model_output/.gitattributes s3://guanaco-vllm-models/chaiml-ca18-c13f-linear-65808-v9/default/.gitattributes
chaiml-ca18-c13f-linear-65808-v9-uploader: cp /dev/shm/model_output/generation_config.json s3://guanaco-vllm-models/chaiml-ca18-c13f-linear-65808-v9/default/generation_config.json
chaiml-ca18-c13f-linear-65808-v9-uploader: cp /dev/shm/model_output/model-00003-of-00003.safetensors s3://guanaco-vllm-models/chaiml-ca18-c13f-linear-65808-v9/default/model-00003-of-00003.safetensors
chaiml-ca18-c13f-linear-65808-v9-uploader: cp /dev/shm/model_output/model-00001-of-00003.safetensors s3://guanaco-vllm-models/chaiml-ca18-c13f-linear-65808-v9/default/model-00001-of-00003.safetensors
chaiml-ca18-c13f-linear-65808-v9-uploader: cp /dev/shm/model_output/model-00002-of-00003.safetensors s3://guanaco-vllm-models/chaiml-ca18-c13f-linear-65808-v9/default/model-00002-of-00003.safetensors
Job chaiml-ca18-c13f-linear-65808-v9-uploader completed after 72.62s with status: succeeded
Stopping job with name chaiml-ca18-c13f-linear-65808-v9-uploader
Pipeline stage VLLMUploader completed in 73.07s
run pipeline stage %s
Running pipeline stage VLLMTemplater
Pipeline stage VLLMTemplater completed in 1.34s
run pipeline stage %s
Running pipeline stage VLLMDeployer
Creating inference service chaiml-ca18-c13f-linear-65808-v9
Waiting for inference service chaiml-ca18-c13f-linear-65808-v9 to be ready
Inference service chaiml-ca18-c13f-linear-65808-v9 ready after 161.0955526828766s
Pipeline stage VLLMDeployer completed in 163.92s
run pipeline stage %s
Running pipeline stage StressChecker
Received healthy response to inference request in 1.7287635803222656s
Received healthy response to inference request in 1.627284288406372s
Received healthy response to inference request in 2.3300161361694336s
Received healthy response to inference request in 1.537872314453125s
Received healthy response to inference request in 1.8384487628936768s
Received healthy response to inference request in 1.7040274143218994s
Received healthy response to inference request in 1.736027717590332s
Received healthy response to inference request in 1.7184028625488281s
Received healthy response to inference request in 1.613447666168213s
Received healthy response to inference request in 1.7256677150726318s
Received healthy response to inference request in 2.0624561309814453s
Received healthy response to inference request in 2.1777749061584473s
Received healthy response to inference request in 1.7977089881896973s
Received healthy response to inference request in 1.7824060916900635s
Received healthy response to inference request in 1.6227576732635498s
Received healthy response to inference request in 1.6445858478546143s
Received healthy response to inference request in 1.5576262474060059s
Received healthy response to inference request in 2.1162431240081787s
Received healthy response to inference request in 1.9551682472229004s
Received healthy response to inference request in 1.5549192428588867s
Received healthy response to inference request in 1.9704527854919434s
Received healthy response to inference request in 2.0074284076690674s
Received healthy response to inference request in 1.8205456733703613s
Received healthy response to inference request in 1.6604359149932861s
Received healthy response to inference request in 1.7871246337890625s
Received healthy response to inference request in 1.8424112796783447s
Received healthy response to inference request in 2.1600341796875s
Received healthy response to inference request in 1.7985942363739014s
Received healthy response to inference request in 1.8598699569702148s
Received healthy response to inference request in 1.8668010234832764s
30 requests
0 failed requests
5th percentile: 1.5561373949050903
10th percentile: 1.6078655242919921
20th percentile: 1.641125535964966
30th percentile: 1.7140902280807495
40th percentile: 1.7331220626831054
50th percentile: 1.7924168109893799
60th percentile: 1.8277069091796874
70th percentile: 1.8619492769241333
80th percentile: 1.9778479099273683
90th percentile: 2.120622229576111
95th percentile: 2.169791579246521
99th percentile: 2.2858661794662476
mean time: 1.8201767683029175
Pipeline stage StressChecker completed in 60.86s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyTriggerPipeline
run_pipeline:run_in_cloud %s
starting trigger_guanaco_pipeline args=%s
triggered trigger_guanaco_pipeline args=%s
Pipeline stage OfflineFamilyFriendlyTriggerPipeline completed in 0.77s
Shutdown handler de-registered
chaiml-ca18-c13f-linear_65808_v9 status is now deployed due to DeploymentManager action
chaiml-ca18-c13f-linear_65808_v9 status is now inactive due to admin request