submission_id: function_darol_2025-04-05
developer_uid: chai_backend_admin
status: failed
generation_params: {'temperature': 1.0, 'top_p': 1.0, 'min_p': 0.0, 'top_k': 40, 'presence_penalty': 0.0, 'frequency_penalty': 0.0, 'stopping_words': ['\n'], 'max_input_tokens': 1024, 'best_of': 8, 'max_output_tokens': 64}
formatter: {'memory_template': '### Instruction:\n{memory}\n', 'prompt_template': '### Input:\n{prompt}\n', 'bot_template': '{bot_name}: {message}\n', 'user_template': '{user_name}: {message}\n', 'response_template': '### Response:\n{bot_name}:', 'truncate_by_message': False}
timestamp: 2025-04-05T00:59:12+00:00
model_name: function_darol_2025-04-05
Resubmit model
Shutdown handler not registered because Python interpreter is not running in the main thread
run pipeline %s
run pipeline stage %s
Running pipeline stage StressChecker
Received healthy response to inference request in 4.064776420593262s
Received healthy response to inference request in 7.357038736343384s
Received healthy response to inference request in 4.728713035583496s
Received healthy response to inference request in 3.463029146194458s
Received healthy response to inference request in 2.832191228866577s
5 requests
0 failed requests
5th percentile: 2.958358812332153
10th percentile: 3.0845263957977296
20th percentile: 3.336861562728882
30th percentile: 3.5833786010742186
40th percentile: 3.82407751083374
50th percentile: 4.064776420593262
60th percentile: 4.330351066589356
70th percentile: 4.595925712585449
80th percentile: 5.254378175735474
90th percentile: 6.305708456039429
95th percentile: 6.831373596191406
99th percentile: 7.2519057083129885
mean time: 4.489149713516236
%s, retrying in %s seconds...
Received healthy response to inference request in 3.5752320289611816s
Received healthy response to inference request in 4.939149618148804s
Received healthy response to inference request in 6.2083470821380615s
Received healthy response to inference request in 14.700847148895264s
Received healthy response to inference request in 19.369189977645874s
5 requests
0 failed requests
5th percentile: 3.848015546798706
10th percentile: 4.1207990646362305
20th percentile: 4.666366100311279
30th percentile: 5.192989110946655
40th percentile: 5.700668096542358
50th percentile: 6.2083470821380615
60th percentile: 9.605347108840942
70th percentile: 13.002347135543822
80th percentile: 15.634515714645387
90th percentile: 17.50185284614563
95th percentile: 18.435521411895753
99th percentile: 19.18245626449585
mean time: 9.758553171157837
%s, retrying in %s seconds...
Received healthy response to inference request in 19.89353656768799s
Received healthy response to inference request in 6.233957767486572s
Received healthy response to inference request in 13.177154302597046s
Received healthy response to inference request in 15.932105541229248s
Received healthy response to inference request in 14.931747913360596s
5 requests
0 failed requests
5th percentile: 7.622597074508667
10th percentile: 9.011236381530761
20th percentile: 11.788514995574952
30th percentile: 13.528073024749755
40th percentile: 14.229910469055175
50th percentile: 14.931747913360596
60th percentile: 15.331890964508057
70th percentile: 15.732034015655518
80th percentile: 16.724391746520997
90th percentile: 18.30896415710449
95th percentile: 19.10125036239624
99th percentile: 19.735079326629638
mean time: 14.03370041847229
clean up pipeline due to error=DeploymentChecksError('Unacceptable 70th percentile latency 15.732034015655518s')
Shutdown handler de-registered
function_darol_2025-04-05 status is now failed due to DeploymentManager action
ChatRequest
Generation Params
Prompt Formatter
Chat History
ChatMessage 1