submission_id: chaiml-2a6f-69d4-linear_43777_v8
developer_uid: richhx
status: inactive
model_repo: ChaiML/2a6f-69d4-linear-w01-FP8
generation_params: {'temperature': 0.9, 'top_p': 0.95, 'min_p': 0.05, 'top_k': 80, 'presence_penalty': 0.45, 'frequency_penalty': 0.45, 'stopping_words': ['\n'], 'max_input_tokens': 1280, 'best_of': 7, 'max_output_tokens': 64}
formatter: {'memory_template': '', 'prompt_template': '', 'bot_template': '{bot_name}: {message}\n', 'user_template': '{user_name}: {message}\n', 'response_template': '{bot_name}:', 'truncate_by_message': False}
timestamp: 2026-03-06T00:14:47+00:00
model_name: chaiml-2a6f-69d4-linear_43777_v8
Resubmit model
Shutdown handler not registered because Python interpreter is not running in the main thread
run pipeline %s
run pipeline stage %s
Running pipeline stage VLLMUploader
Starting job with name chaiml-2a6f-69d4-linear-43777-v8-uploader
Waiting for job on chaiml-2a6f-69d4-linear-43777-v8-uploader to finish
chaiml-2a6f-69d4-linear-43777-v8-uploader: Using quantization_mode: fp8
chaiml-2a6f-69d4-linear-43777-v8-uploader: Repo ChaiML/2a6f-69d4-linear-w01-FP8 already ends in FP8. Skipping...
chaiml-2a6f-69d4-linear-43777-v8-uploader: Checking if ChaiML/2a6f-69d4-linear-w01-FP8 already exists in ChaiML
chaiml-2a6f-69d4-linear-43777-v8-uploader: Model already exists. Downloading to /dev/shm/model_output...
chaiml-2a6f-69d4-linear-43777-v8-uploader: Downloading snapshot of ChaiML/2a6f-69d4-linear-w01-FP8...
chaiml-2a6f-69d4-linear-43777-v8-uploader: Downloaded in 18.759s
chaiml-2a6f-69d4-linear-43777-v8-uploader: Processed model ChaiML/2a6f-69d4-linear-w01-FP8 in 22.241s
chaiml-2a6f-69d4-linear-43777-v8-uploader: creating bucket guanaco-vllm-models
chaiml-2a6f-69d4-linear-43777-v8-uploader: /usr/lib/python3/dist-packages/S3/BaseUtils.py:56: SyntaxWarning: invalid escape sequence '\.'
chaiml-2a6f-69d4-linear-43777-v8-uploader:   RE_S3_DATESTRING = re.compile('\.[0-9]*(?:[Z\\-\\+]*?)')
chaiml-2a6f-69d4-linear-43777-v8-uploader: /usr/lib/python3/dist-packages/S3/BaseUtils.py:57: SyntaxWarning: invalid escape sequence '\s'
chaiml-2a6f-69d4-linear-43777-v8-uploader:   RE_XML_NAMESPACE = re.compile(b'^(<?[^>]+?>\s*|\s*)(<\w+) xmlns=[\'"](https?://[^\'"]+)[\'"]', re.MULTILINE)
chaiml-2a6f-69d4-linear-43777-v8-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:240: SyntaxWarning: invalid escape sequence '\.'
chaiml-2a6f-69d4-linear-43777-v8-uploader:   invalid = re.search("([^a-z0-9\.-])", bucket, re.UNICODE)
chaiml-2a6f-69d4-linear-43777-v8-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:244: SyntaxWarning: invalid escape sequence '\.'
chaiml-2a6f-69d4-linear-43777-v8-uploader:   invalid = re.search("([^A-Za-z0-9\._-])", bucket, re.UNICODE)
chaiml-2a6f-69d4-linear-43777-v8-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:255: SyntaxWarning: invalid escape sequence '\.'
chaiml-2a6f-69d4-linear-43777-v8-uploader:   if re.search("-\.", bucket, re.UNICODE):
chaiml-2a6f-69d4-linear-43777-v8-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:257: SyntaxWarning: invalid escape sequence '\.'
chaiml-2a6f-69d4-linear-43777-v8-uploader:   if re.search("\.\.", bucket, re.UNICODE):
chaiml-2a6f-69d4-linear-43777-v8-uploader: /usr/lib/python3/dist-packages/S3/S3Uri.py:155: SyntaxWarning: invalid escape sequence '\w'
chaiml-2a6f-69d4-linear-43777-v8-uploader:   _re = re.compile("^(\w+://)?(.*)", re.UNICODE)
chaiml-2a6f-69d4-linear-43777-v8-uploader: /usr/lib/python3/dist-packages/S3/FileLists.py:480: SyntaxWarning: invalid escape sequence '\*'
chaiml-2a6f-69d4-linear-43777-v8-uploader:   wildcard_split_result = re.split("\*|\?", uri_str, maxsplit=1)
chaiml-2a6f-69d4-linear-43777-v8-uploader: Bucket 's3://guanaco-vllm-models/' created
chaiml-2a6f-69d4-linear-43777-v8-uploader: uploading /dev/shm/model_output to s3://guanaco-vllm-models/chaiml-2a6f-69d4-linear-43777-v8/default
chaiml-2a6f-69d4-linear-43777-v8-uploader: cp /dev/shm/model_output/generation_config.json s3://guanaco-vllm-models/chaiml-2a6f-69d4-linear-43777-v8/default/generation_config.json
chaiml-2a6f-69d4-linear-43777-v8-uploader: cp /dev/shm/model_output/model.safetensors.index.json s3://guanaco-vllm-models/chaiml-2a6f-69d4-linear-43777-v8/default/model.safetensors.index.json
chaiml-2a6f-69d4-linear-43777-v8-uploader: cp /dev/shm/model_output/chat_template.jinja s3://guanaco-vllm-models/chaiml-2a6f-69d4-linear-43777-v8/default/chat_template.jinja
chaiml-2a6f-69d4-linear-43777-v8-uploader: cp /dev/shm/model_output/.gitattributes s3://guanaco-vllm-models/chaiml-2a6f-69d4-linear-43777-v8/default/.gitattributes
chaiml-2a6f-69d4-linear-43777-v8-uploader: cp /dev/shm/model_output/recipe.yaml s3://guanaco-vllm-models/chaiml-2a6f-69d4-linear-43777-v8/default/recipe.yaml
chaiml-2a6f-69d4-linear-43777-v8-uploader: cp /dev/shm/model_output/special_tokens_map.json s3://guanaco-vllm-models/chaiml-2a6f-69d4-linear-43777-v8/default/special_tokens_map.json
chaiml-2a6f-69d4-linear-43777-v8-uploader: cp /dev/shm/model_output/config.json s3://guanaco-vllm-models/chaiml-2a6f-69d4-linear-43777-v8/default/config.json
chaiml-2a6f-69d4-linear-43777-v8-uploader: cp /dev/shm/model_output/tokenizer_config.json s3://guanaco-vllm-models/chaiml-2a6f-69d4-linear-43777-v8/default/tokenizer_config.json
chaiml-2a6f-69d4-linear-43777-v8-uploader: cp /dev/shm/model_output/tokenizer.json s3://guanaco-vllm-models/chaiml-2a6f-69d4-linear-43777-v8/default/tokenizer.json
chaiml-2a6f-69d4-linear-43777-v8-uploader: cp /dev/shm/model_output/model-00006-of-00006.safetensors s3://guanaco-vllm-models/chaiml-2a6f-69d4-linear-43777-v8/default/model-00006-of-00006.safetensors
chaiml-2a6f-69d4-linear-43777-v8-uploader: cp /dev/shm/model_output/model-00005-of-00006.safetensors s3://guanaco-vllm-models/chaiml-2a6f-69d4-linear-43777-v8/default/model-00005-of-00006.safetensors
chaiml-2a6f-69d4-linear-43777-v8-uploader: cp /dev/shm/model_output/model-00001-of-00006.safetensors s3://guanaco-vllm-models/chaiml-2a6f-69d4-linear-43777-v8/default/model-00001-of-00006.safetensors
Job chaiml-2a6f-69d4-linear-43777-v8-uploader completed after 83.05s with status: succeeded
Stopping job with name chaiml-2a6f-69d4-linear-43777-v8-uploader
Pipeline stage VLLMUploader completed in 83.67s
run pipeline stage %s
Running pipeline stage VLLMTemplater
Pipeline stage VLLMTemplater completed in 2.17s
run pipeline stage %s
Running pipeline stage VLLMDeployer
Creating inference service chaiml-2a6f-69d4-linear-43777-v8
Waiting for inference service chaiml-2a6f-69d4-linear-43777-v8 to be ready
Inference service chaiml-2a6f-69d4-linear-43777-v8 ready after 160.42645692825317s
Pipeline stage VLLMDeployer completed in 161.36s
run pipeline stage %s
Running pipeline stage StressChecker
read tcp 127.0.0.1:52404->127.0.0.1:8080: read: connection reset by peer

Received unhealthy response to inference request!
Received healthy response to inference request in 2.8018105030059814s
Received healthy response to inference request in 2.7580173015594482s
Received healthy response to inference request in 2.8926310539245605s
Received healthy response to inference request in 3.170475721359253s
Received healthy response to inference request in 2.827962875366211s
Received healthy response to inference request in 2.866819143295288s
Received healthy response to inference request in 2.7986621856689453s
Received healthy response to inference request in 2.765568733215332s
Received healthy response to inference request in 3.23797869682312s
Received healthy response to inference request in 2.9370219707489014s
Received healthy response to inference request in 3.2578461170196533s
Received healthy response to inference request in 2.7127645015716553s
Received healthy response to inference request in 3.0872418880462646s
Received healthy response to inference request in 2.8268485069274902s
Received healthy response to inference request in 4.139420032501221s
Received healthy response to inference request in 3.016815423965454s
Received healthy response to inference request in 2.7434048652648926s
Received healthy response to inference request in 3.4418749809265137s
Received healthy response to inference request in 2.8264245986938477s
Received healthy response to inference request in 2.6888933181762695s
Received healthy response to inference request in 2.6934814453125s
Received healthy response to inference request in 3.7205355167388916s
Received healthy response to inference request in 2.774259328842163s
Received healthy response to inference request in 3.137120246887207s
Received healthy response to inference request in 3.1748244762420654s
Received healthy response to inference request in 2.6900527477264404s
Received healthy response to inference request in 2.833456039428711s
Received healthy response to inference request in 3.1589949131011963s
Received healthy response to inference request in 3.9920077323913574s
30 requests
1 failed requests
5th percentile: 2.6894150614738463
10th percentile: 2.693138575553894
20th percentile: 2.755094814300537
30th percentile: 2.7913413286209106
40th percentile: 2.8266789436340334
50th percentile: 2.8501375913619995
60th percentile: 2.9689393520355223
70th percentile: 3.1436826467514036
80th percentile: 3.1874553203582767
90th percentile: 3.4697410345077517
95th percentile: 3.869845235347747
99th percentile: 4.09667046546936
mean time: 2.9374939997990928
%s, retrying in %s seconds...
Received healthy response to inference request in 3.7055611610412598s
Received healthy response to inference request in 2.7792704105377197s
Received healthy response to inference request in 3.2776050567626953s
Received healthy response to inference request in 3.4519479274749756s
Received healthy response to inference request in 2.739112377166748s
Received healthy response to inference request in 2.8798999786376953s
Received healthy response to inference request in 2.6898128986358643s
Received healthy response to inference request in 2.683298349380493s
Received healthy response to inference request in 2.794684886932373s
Received healthy response to inference request in 3.3529937267303467s
Received healthy response to inference request in 2.788388252258301s
Received healthy response to inference request in 2.6968300342559814s
Received healthy response to inference request in 3.5805013179779053s
Received healthy response to inference request in 3.114651918411255s
Received healthy response to inference request in 2.724242687225342s
Received healthy response to inference request in 3.2474191188812256s
Received healthy response to inference request in 3.169607639312744s
Received healthy response to inference request in 3.2123708724975586s
Received healthy response to inference request in 2.7783334255218506s
Received healthy response to inference request in 2.6985981464385986s
Received healthy response to inference request in 3.3497116565704346s
Received healthy response to inference request in 2.834545612335205s
Received healthy response to inference request in 3.587554931640625s
Received healthy response to inference request in 2.872728109359741s
Received healthy response to inference request in 3.0012094974517822s
Received healthy response to inference request in 2.9721951484680176s
Received healthy response to inference request in 2.7882237434387207s
Received healthy response to inference request in 3.4931728839874268s
Received healthy response to inference request in 2.77162766456604s
Received healthy response to inference request in 3.1340739727020264s
30 requests
0 failed requests
5th percentile: 2.692970609664917
10th percentile: 2.698421335220337
20th percentile: 2.7651246070861815
30th percentile: 2.7855377435684203
40th percentile: 2.8186013221740724
50th percentile: 2.9260475635528564
60th percentile: 3.1224207401275637
70th percentile: 3.222885346412659
80th percentile: 3.350368070602417
90th percentile: 3.501905727386475
95th percentile: 3.584380805492401
99th percentile: 3.6713393545150756
mean time: 3.0390057802200316
Pipeline stage StressChecker completed in 186.80s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyTriggerPipeline
run_pipeline:run_in_cloud %s
starting trigger_guanaco_pipeline args=%s
triggered trigger_guanaco_pipeline args=%s
Pipeline stage OfflineFamilyFriendlyTriggerPipeline completed in 0.79s
Shutdown handler de-registered
chaiml-2a6f-69d4-linear_43777_v8 status is now deployed due to DeploymentManager action
chaiml-2a6f-69d4-linear_43777_v8 status is now inactive due to admin request