submission_id: chaiml-2fe5-c13f-linear_57126_v8
developer_uid: richhx
status: inactive
model_repo: ChaiML/2fe5-c13f-linear-w01-FP8
generation_params: {'temperature': 0.9, 'top_p': 1.0, 'min_p': 0.05, 'top_k': 80, 'presence_penalty': 0.0, 'frequency_penalty': 0.0, 'stopping_words': ['####', 'You:', '</s>', '<|im_end|>', '\n', '<|eot_id|>', 'User:', 'Bot:'], 'max_input_tokens': 1024, 'best_of': 10, 'max_output_tokens': 64}
formatter: {'memory_template': "{bot_name}'s Persona: {memory}\n####\n", 'prompt_template': '', 'bot_template': '{bot_name}: {message}\n', 'user_template': '{user_name}: {message}\n', 'response_template': '{bot_name}:', 'truncate_by_message': True}
timestamp: 2026-03-05T23:56:50+00:00
model_name: chaiml-2fe5-c13f-linear_57126_v8
Resubmit model
Shutdown handler not registered because Python interpreter is not running in the main thread
run pipeline %s
run pipeline stage %s
Running pipeline stage VLLMUploader
Starting job with name chaiml-2fe5-c13f-linear-57126-v8-uploader
Waiting for job on chaiml-2fe5-c13f-linear-57126-v8-uploader to finish
chaiml-2fe5-c13f-linear-57126-v8-uploader: Using quantization_mode: fp8
chaiml-2fe5-c13f-linear-57126-v8-uploader: Repo ChaiML/2fe5-c13f-linear-w01-FP8 already ends in FP8. Skipping...
chaiml-2fe5-c13f-linear-57126-v8-uploader: Checking if ChaiML/2fe5-c13f-linear-w01-FP8 already exists in ChaiML
chaiml-2fe5-c13f-linear-57126-v8-uploader: Model already exists. Downloading to /dev/shm/model_output...
chaiml-2fe5-c13f-linear-57126-v8-uploader: Downloading snapshot of ChaiML/2fe5-c13f-linear-w01-FP8...
chaiml-2fe5-c13f-linear-57126-v8-uploader: Downloaded in 6.802s
chaiml-2fe5-c13f-linear-57126-v8-uploader: Processed model ChaiML/2fe5-c13f-linear-w01-FP8 in 10.330s
chaiml-2fe5-c13f-linear-57126-v8-uploader: creating bucket guanaco-vllm-models
chaiml-2fe5-c13f-linear-57126-v8-uploader: /usr/lib/python3/dist-packages/S3/BaseUtils.py:56: SyntaxWarning: invalid escape sequence '\.'
chaiml-2fe5-c13f-linear-57126-v8-uploader:   RE_S3_DATESTRING = re.compile('\.[0-9]*(?:[Z\\-\\+]*?)')
chaiml-2fe5-c13f-linear-57126-v8-uploader: /usr/lib/python3/dist-packages/S3/BaseUtils.py:57: SyntaxWarning: invalid escape sequence '\s'
chaiml-2fe5-c13f-linear-57126-v8-uploader:   RE_XML_NAMESPACE = re.compile(b'^(<?[^>]+?>\s*|\s*)(<\w+) xmlns=[\'"](https?://[^\'"]+)[\'"]', re.MULTILINE)
chaiml-2fe5-c13f-linear-57126-v8-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:240: SyntaxWarning: invalid escape sequence '\.'
chaiml-2fe5-c13f-linear-57126-v8-uploader:   invalid = re.search("([^a-z0-9\.-])", bucket, re.UNICODE)
chaiml-2fe5-c13f-linear-57126-v8-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:244: SyntaxWarning: invalid escape sequence '\.'
chaiml-2fe5-c13f-linear-57126-v8-uploader:   invalid = re.search("([^A-Za-z0-9\._-])", bucket, re.UNICODE)
chaiml-2fe5-c13f-linear-57126-v8-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:255: SyntaxWarning: invalid escape sequence '\.'
chaiml-2fe5-c13f-linear-57126-v8-uploader:   if re.search("-\.", bucket, re.UNICODE):
chaiml-2fe5-c13f-linear-57126-v8-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:257: SyntaxWarning: invalid escape sequence '\.'
chaiml-2fe5-c13f-linear-57126-v8-uploader:   if re.search("\.\.", bucket, re.UNICODE):
chaiml-2fe5-c13f-linear-57126-v8-uploader: /usr/lib/python3/dist-packages/S3/S3Uri.py:155: SyntaxWarning: invalid escape sequence '\w'
chaiml-2fe5-c13f-linear-57126-v8-uploader:   _re = re.compile("^(\w+://)?(.*)", re.UNICODE)
chaiml-2fe5-c13f-linear-57126-v8-uploader: /usr/lib/python3/dist-packages/S3/FileLists.py:480: SyntaxWarning: invalid escape sequence '\*'
chaiml-2fe5-c13f-linear-57126-v8-uploader:   wildcard_split_result = re.split("\*|\?", uri_str, maxsplit=1)
chaiml-2fe5-c13f-linear-57126-v8-uploader: Bucket 's3://guanaco-vllm-models/' created
chaiml-2fe5-c13f-linear-57126-v8-uploader: uploading /dev/shm/model_output to s3://guanaco-vllm-models/chaiml-2fe5-c13f-linear-57126-v8/default
chaiml-2fe5-c13f-linear-57126-v8-uploader: cp /dev/shm/model_output/.gitattributes s3://guanaco-vllm-models/chaiml-2fe5-c13f-linear-57126-v8/default/.gitattributes
chaiml-2fe5-c13f-linear-57126-v8-uploader: cp /dev/shm/model_output/chat_template.jinja s3://guanaco-vllm-models/chaiml-2fe5-c13f-linear-57126-v8/default/chat_template.jinja
chaiml-2fe5-c13f-linear-57126-v8-uploader: cp /dev/shm/model_output/special_tokens_map.json s3://guanaco-vllm-models/chaiml-2fe5-c13f-linear-57126-v8/default/special_tokens_map.json
chaiml-2fe5-c13f-linear-57126-v8-uploader: cp /dev/shm/model_output/config.json s3://guanaco-vllm-models/chaiml-2fe5-c13f-linear-57126-v8/default/config.json
chaiml-2fe5-c13f-linear-57126-v8-uploader: cp /dev/shm/model_output/model.safetensors.index.json s3://guanaco-vllm-models/chaiml-2fe5-c13f-linear-57126-v8/default/model.safetensors.index.json
chaiml-2fe5-c13f-linear-57126-v8-uploader: cp /dev/shm/model_output/recipe.yaml s3://guanaco-vllm-models/chaiml-2fe5-c13f-linear-57126-v8/default/recipe.yaml
chaiml-2fe5-c13f-linear-57126-v8-uploader: cp /dev/shm/model_output/generation_config.json s3://guanaco-vllm-models/chaiml-2fe5-c13f-linear-57126-v8/default/generation_config.json
chaiml-2fe5-c13f-linear-57126-v8-uploader: cp /dev/shm/model_output/tokenizer_config.json s3://guanaco-vllm-models/chaiml-2fe5-c13f-linear-57126-v8/default/tokenizer_config.json
chaiml-2fe5-c13f-linear-57126-v8-uploader: cp /dev/shm/model_output/tokenizer.json s3://guanaco-vllm-models/chaiml-2fe5-c13f-linear-57126-v8/default/tokenizer.json
chaiml-2fe5-c13f-linear-57126-v8-uploader: cp /dev/shm/model_output/model-00003-of-00003.safetensors s3://guanaco-vllm-models/chaiml-2fe5-c13f-linear-57126-v8/default/model-00003-of-00003.safetensors
chaiml-2fe5-c13f-linear-57126-v8-uploader: cp /dev/shm/model_output/model-00002-of-00003.safetensors s3://guanaco-vllm-models/chaiml-2fe5-c13f-linear-57126-v8/default/model-00002-of-00003.safetensors
chaiml-2fe5-c13f-linear-57126-v8-uploader: cp /dev/shm/model_output/model-00001-of-00003.safetensors s3://guanaco-vllm-models/chaiml-2fe5-c13f-linear-57126-v8/default/model-00001-of-00003.safetensors
Job chaiml-2fe5-c13f-linear-57126-v8-uploader completed after 62.37s with status: succeeded
Stopping job with name chaiml-2fe5-c13f-linear-57126-v8-uploader
Pipeline stage VLLMUploader completed in 62.83s
run pipeline stage %s
Running pipeline stage VLLMTemplater
Pipeline stage VLLMTemplater completed in 0.64s
run pipeline stage %s
Running pipeline stage VLLMDeployer
Creating inference service chaiml-2fe5-c13f-linear-57126-v8
Waiting for inference service chaiml-2fe5-c13f-linear-57126-v8 to be ready
Inference service chaiml-2fe5-c13f-linear-57126-v8 ready after 160.35631442070007s
Pipeline stage VLLMDeployer completed in 160.91s
run pipeline stage %s
Running pipeline stage StressChecker
Received healthy response to inference request in 2.4125282764434814s
Received healthy response to inference request in 2.9148902893066406s
Received healthy response to inference request in 1.7745420932769775s
Received healthy response to inference request in 1.612253189086914s
Received healthy response to inference request in 1.6945216655731201s
Received healthy response to inference request in 1.6195182800292969s
Received healthy response to inference request in 2.3913393020629883s
Received healthy response to inference request in 1.9933300018310547s
Received healthy response to inference request in 1.908296823501587s
Received healthy response to inference request in 1.6447525024414062s
Received healthy response to inference request in 1.7216110229492188s
Received healthy response to inference request in 1.9595632553100586s
Received healthy response to inference request in 1.6475918292999268s
Received healthy response to inference request in 1.7436461448669434s
Received healthy response to inference request in 1.6799719333648682s
Received healthy response to inference request in 1.9146955013275146s
Received healthy response to inference request in 1.895418643951416s
Received healthy response to inference request in 1.701145887374878s
Received healthy response to inference request in 1.5914347171783447s
Received healthy response to inference request in 1.7379419803619385s
Received healthy response to inference request in 1.6193490028381348s
Received healthy response to inference request in 1.6594269275665283s
Received healthy response to inference request in 2.279320001602173s
Received healthy response to inference request in 1.9320952892303467s
Received healthy response to inference request in 1.8389418125152588s
Received healthy response to inference request in 1.8823010921478271s
Received healthy response to inference request in 2.1023173332214355s
Received healthy response to inference request in 1.865976095199585s
Received healthy response to inference request in 1.9003188610076904s
Received healthy response to inference request in 1.675851583480835s
30 requests
0 failed requests
5th percentile: 1.6154463052749635
10th percentile: 1.6195013523101807
20th percentile: 1.657059907913208
30th percentile: 1.6901567459106446
40th percentile: 1.7314095973968506
50th percentile: 1.8067419528961182
60th percentile: 1.8875481128692626
70th percentile: 1.9102164268493653
80th percentile: 1.966316604614258
90th percentile: 2.2905219316482546
95th percentile: 2.4029932379722596
99th percentile: 2.769205305576325
mean time: 1.877163044611613
Pipeline stage StressChecker completed in 62.09s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyTriggerPipeline
run_pipeline:run_in_cloud %s
starting trigger_guanaco_pipeline args=%s
triggered trigger_guanaco_pipeline args=%s
Pipeline stage OfflineFamilyFriendlyTriggerPipeline completed in 0.70s
Shutdown handler de-registered
chaiml-2fe5-c13f-linear_57126_v8 status is now deployed due to DeploymentManager action
chaiml-2fe5-c13f-linear_57126_v8 status is now inactive due to admin request