submission_id: function_lihum_2024-11-04
developer_uid: chai_backend_admin
status: torndown
generation_params: {'temperature': 1.5, 'top_p': 0.99, 'min_p': 0.1, 'top_k': 40, 'presence_penalty': 0.0, 'frequency_penalty': 0.0, 'stopping_words': ['\n', '<｜end▁of▁sentence｜>'], 'max_input_tokens': 512, 'best_of': 1, 'max_output_tokens': 64}
formatter: {'memory_template': "{bot_name}'s Persona: {memory}\n####\n", 'prompt_template': '{prompt}\n<START>\n', 'bot_template': '{bot_name}: {message}\n', 'user_template': '{user_name}: {message}\n', 'response_template': '{bot_name}:', 'truncate_by_message': False}
timestamp: 2024-11-04T02:15:42+00:00
model_name: function_lihum_2024-11-04
Resubmit model
Shutdown handler not registered because Python interpreter is not running in the main thread
run pipeline %s
run pipeline stage %s
Running pipeline stage StressChecker
Received healthy response to inference request in 4.821281433105469s
Received healthy response to inference request in 5.1286609172821045s
Received healthy response to inference request in 4.845617771148682s
Received healthy response to inference request in 4.27593207359314s
Received healthy response to inference request in 3.6281309127807617s
5 requests
0 failed requests
5th percentile: 3.7576911449432373
10th percentile: 3.887251377105713
20th percentile: 4.146371841430664
30th percentile: 4.385001945495605
40th percentile: 4.6031416893005375
50th percentile: 4.821281433105469
60th percentile: 4.831015968322754
70th percentile: 4.840750503540039
80th percentile: 4.902226400375366
90th percentile: 5.015443658828735
95th percentile: 5.07205228805542
99th percentile: 5.117339191436767
mean time: 4.539924621582031
%s, retrying in %s seconds...
Connection pool is full, discarding connection: %s. Connection pool size: %s
Received healthy response to inference request in 5.027175664901733s
function_lihum_2024-11-04 status is now torndown due to DeploymentManager action