chaiml-02f4-69d4-linea_30131

submission_id: chaiml-02f4-69d4-linea_30131_v10
developer_uid: richhx
status: inactive
model_repo: ChaiML/02f4-69d4-linear-w01-FP8
generation_params: {'temperature': 0.7, 'top_p': 0.95, 'min_p': 0.025, 'top_k': 80, 'presence_penalty': 0.4, 'frequency_penalty': 0.4, 'stopping_words': ['\n'], 'max_input_tokens': 1024, 'best_of': 8, 'max_output_tokens': 64}
formatter: {'memory_template': '<|im_start|>system\n{memory}<|im_end|>\n', 'prompt_template': '', 'bot_template': '<|im_start|>assistant\n{bot_name}: {message}<|im_end|>\n', 'user_template': '<|im_start|>user\n{user_name}: {message}<|im_end|>\n', 'response_template': '<|im_start|>assistant\n{bot_name}:', 'truncate_by_message': True}
timestamp: 2026-03-05T23:44:22+00:00
model_name: chaiml-02f4-69d4-linea_30131_v10
Resubmit model
Shutdown handler not registered because Python interpreter is not running in the main thread
run pipeline %s
run pipeline stage %s
Running pipeline stage VLLMUploader
Starting job with name chaiml-02f4-69d4-linea-30131-v10-uploader
Waiting for job on chaiml-02f4-69d4-linea-30131-v10-uploader to finish
chaiml-02f4-69d4-linea-30131-v10-uploader: Using quantization_mode: fp8
chaiml-02f4-69d4-linea-30131-v10-uploader: Repo ChaiML/02f4-69d4-linear-w01-FP8 already ends in FP8. Skipping...
chaiml-02f4-69d4-linea-30131-v10-uploader: Checking if ChaiML/02f4-69d4-linear-w01-FP8 already exists in ChaiML
chaiml-02f4-69d4-linea-30131-v10-uploader: Model already exists. Downloading to /dev/shm/model_output...
chaiml-02f4-69d4-linea-30131-v10-uploader: Downloading snapshot of ChaiML/02f4-69d4-linear-w01-FP8...
chaiml-02f4-69d4-linea-30131-v10-uploader: Downloaded in 11.757s
chaiml-02f4-69d4-linea-30131-v10-uploader: Processed model ChaiML/02f4-69d4-linear-w01-FP8 in 15.270s
chaiml-02f4-69d4-linea-30131-v10-uploader: creating bucket guanaco-vllm-models
chaiml-02f4-69d4-linea-30131-v10-uploader: /usr/lib/python3/dist-packages/S3/BaseUtils.py:56: SyntaxWarning: invalid escape sequence '\.'
chaiml-02f4-69d4-linea-30131-v10-uploader:   RE_S3_DATESTRING = re.compile('\.[0-9]*(?:[Z\\-\\+]*?)')
chaiml-02f4-69d4-linea-30131-v10-uploader: /usr/lib/python3/dist-packages/S3/BaseUtils.py:57: SyntaxWarning: invalid escape sequence '\s'
chaiml-02f4-69d4-linea-30131-v10-uploader:   RE_XML_NAMESPACE = re.compile(b'^(<?[^>]+?>\s*|\s*)(<\w+) xmlns=[\'"](https?://[^\'"]+)[\'"]', re.MULTILINE)
chaiml-02f4-69d4-linea-30131-v10-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:240: SyntaxWarning: invalid escape sequence '\.'
chaiml-02f4-69d4-linea-30131-v10-uploader:   invalid = re.search("([^a-z0-9\.-])", bucket, re.UNICODE)
chaiml-02f4-69d4-linea-30131-v10-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:244: SyntaxWarning: invalid escape sequence '\.'
chaiml-02f4-69d4-linea-30131-v10-uploader:   invalid = re.search("([^A-Za-z0-9\._-])", bucket, re.UNICODE)
chaiml-02f4-69d4-linea-30131-v10-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:255: SyntaxWarning: invalid escape sequence '\.'
chaiml-02f4-69d4-linea-30131-v10-uploader:   if re.search("-\.", bucket, re.UNICODE):
chaiml-02f4-69d4-linea-30131-v10-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:257: SyntaxWarning: invalid escape sequence '\.'
chaiml-02f4-69d4-linea-30131-v10-uploader:   if re.search("\.\.", bucket, re.UNICODE):
chaiml-02f4-69d4-linea-30131-v10-uploader: /usr/lib/python3/dist-packages/S3/S3Uri.py:155: SyntaxWarning: invalid escape sequence '\w'
chaiml-02f4-69d4-linea-30131-v10-uploader:   _re = re.compile("^(\w+://)?(.*)", re.UNICODE)
chaiml-02f4-69d4-linea-30131-v10-uploader: /usr/lib/python3/dist-packages/S3/FileLists.py:480: SyntaxWarning: invalid escape sequence '\*'
chaiml-02f4-69d4-linea-30131-v10-uploader:   wildcard_split_result = re.split("\*|\?", uri_str, maxsplit=1)
chaiml-02f4-69d4-linea-30131-v10-uploader: Bucket 's3://guanaco-vllm-models/' created
chaiml-02f4-69d4-linea-30131-v10-uploader: uploading /dev/shm/model_output to s3://guanaco-vllm-models/chaiml-02f4-69d4-linea-30131-v10/default
chaiml-02f4-69d4-linea-30131-v10-uploader: cp /dev/shm/model_output/config.json s3://guanaco-vllm-models/chaiml-02f4-69d4-linea-30131-v10/default/config.json
chaiml-02f4-69d4-linea-30131-v10-uploader: cp /dev/shm/model_output/special_tokens_map.json s3://guanaco-vllm-models/chaiml-02f4-69d4-linea-30131-v10/default/special_tokens_map.json
chaiml-02f4-69d4-linea-30131-v10-uploader: cp /dev/shm/model_output/generation_config.json s3://guanaco-vllm-models/chaiml-02f4-69d4-linea-30131-v10/default/generation_config.json
chaiml-02f4-69d4-linea-30131-v10-uploader: cp /dev/shm/model_output/recipe.yaml s3://guanaco-vllm-models/chaiml-02f4-69d4-linea-30131-v10/default/recipe.yaml
chaiml-02f4-69d4-linea-30131-v10-uploader: cp /dev/shm/model_output/tokenizer_config.json s3://guanaco-vllm-models/chaiml-02f4-69d4-linea-30131-v10/default/tokenizer_config.json
chaiml-02f4-69d4-linea-30131-v10-uploader: cp /dev/shm/model_output/tokenizer.json s3://guanaco-vllm-models/chaiml-02f4-69d4-linea-30131-v10/default/tokenizer.json
chaiml-02f4-69d4-linea-30131-v10-uploader: cp /dev/shm/model_output/model.safetensors.index.json s3://guanaco-vllm-models/chaiml-02f4-69d4-linea-30131-v10/default/model.safetensors.index.json
chaiml-02f4-69d4-linea-30131-v10-uploader: cp /dev/shm/model_output/.gitattributes s3://guanaco-vllm-models/chaiml-02f4-69d4-linea-30131-v10/default/.gitattributes
chaiml-02f4-69d4-linea-30131-v10-uploader: cp /dev/shm/model_output/model-00006-of-00006.safetensors s3://guanaco-vllm-models/chaiml-02f4-69d4-linea-30131-v10/default/model-00006-of-00006.safetensors
chaiml-02f4-69d4-linea-30131-v10-uploader: cp /dev/shm/model_output/model-00005-of-00006.safetensors s3://guanaco-vllm-models/chaiml-02f4-69d4-linea-30131-v10/default/model-00005-of-00006.safetensors
chaiml-02f4-69d4-linea-30131-v10-uploader: cp /dev/shm/model_output/model-00004-of-00006.safetensors s3://guanaco-vllm-models/chaiml-02f4-69d4-linea-30131-v10/default/model-00004-of-00006.safetensors
chaiml-02f4-69d4-linea-30131-v10-uploader: cp /dev/shm/model_output/model-00001-of-00006.safetensors s3://guanaco-vllm-models/chaiml-02f4-69d4-linea-30131-v10/default/model-00001-of-00006.safetensors
chaiml-02f4-69d4-linea-30131-v10-uploader: cp /dev/shm/model_output/model-00002-of-00006.safetensors s3://guanaco-vllm-models/chaiml-02f4-69d4-linea-30131-v10/default/model-00002-of-00006.safetensors
chaiml-02f4-69d4-linea-30131-v10-uploader: cp /dev/shm/model_output/model-00003-of-00006.safetensors s3://guanaco-vllm-models/chaiml-02f4-69d4-linea-30131-v10/default/model-00003-of-00006.safetensors
Job chaiml-02f4-69d4-linea-30131-v10-uploader completed after 92.73s with status: succeeded
Stopping job with name chaiml-02f4-69d4-linea-30131-v10-uploader
Pipeline stage VLLMUploader completed in 93.22s
run pipeline stage %s
Running pipeline stage VLLMTemplater
Pipeline stage VLLMTemplater completed in 1.59s
run pipeline stage %s
Running pipeline stage VLLMDeployer
Creating inference service chaiml-02f4-69d4-linea-30131-v10
Waiting for inference service chaiml-02f4-69d4-linea-30131-v10 to be ready
Inference service chaiml-02f4-69d4-linea-30131-v10 ready after 150.67785263061523s
Pipeline stage VLLMDeployer completed in 152.38s
run pipeline stage %s
Running pipeline stage StressChecker
Received healthy response to inference request in 3.2251882553100586s
Received healthy response to inference request in 3.1041100025177s
Received healthy response to inference request in 2.720433473587036s
Received healthy response to inference request in 2.749112129211426s
Received healthy response to inference request in 3.433933734893799s
Received healthy response to inference request in 2.8069732189178467s
Received healthy response to inference request in 2.814570188522339s
Received healthy response to inference request in 3.1377551555633545s
Received healthy response to inference request in 3.354301929473877s
Received healthy response to inference request in 3.499281167984009s
Received healthy response to inference request in 2.9765121936798096s
Received healthy response to inference request in 3.2364611625671387s
Received healthy response to inference request in 2.8267946243286133s
Received healthy response to inference request in 3.6265456676483154s
Received healthy response to inference request in 2.985720157623291s
Received healthy response to inference request in 3.169945478439331s
Received healthy response to inference request in 3.387913942337036s
Received healthy response to inference request in 2.843907594680786s
Received healthy response to inference request in 3.1634557247161865s
Received healthy response to inference request in 2.799969434738159s
Received healthy response to inference request in 3.000277280807495s
Received healthy response to inference request in 2.7138333320617676s
Received healthy response to inference request in 2.8621487617492676s
Received healthy response to inference request in 2.7941417694091797s
Received healthy response to inference request in 2.95330810546875s
Received healthy response to inference request in 2.7288625240325928s
Received healthy response to inference request in 3.327193260192871s
Received healthy response to inference request in 2.906208038330078s
Received healthy response to inference request in 3.4966278076171875s
Received healthy response to inference request in 3.041501045227051s
30 requests
0 failed requests
5th percentile: 2.724226546287537
10th percentile: 2.7470871686935423
20th percentile: 2.8055724620819094
30th percentile: 2.8387737035751344
40th percentile: 2.9344680786132815
50th percentile: 2.992998719215393
60th percentile: 3.1175680637359617
70th percentile: 3.1865183115005493
80th percentile: 3.3326149940490724
90th percentile: 3.440203142166138
95th percentile: 3.498087155818939
99th percentile: 3.5896389627456666
mean time: 3.0562329053878785
Pipeline stage StressChecker completed in 96.23s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyTriggerPipeline
run_pipeline:run_in_cloud %s
starting trigger_guanaco_pipeline args=%s
triggered trigger_guanaco_pipeline args=%s
Pipeline stage OfflineFamilyFriendlyTriggerPipeline completed in 0.79s
Shutdown handler de-registered
chaiml-02f4-69d4-linea_30131_v10 status is now deployed due to DeploymentManager action
chaiml-02f4-69d4-linea_30131_v10 status is now inactive due to admin request