submission_id: function_bitil_2025-05-22
developer_uid: chai_backend_admin
status: failed
generation_params: {'temperature': 1.0, 'top_p': 1.0, 'min_p': 0.0, 'top_k': 40, 'presence_penalty': 0.0, 'frequency_penalty': 0.0, 'stopping_words': ['\n'], 'max_input_tokens': 1024, 'best_of': 8, 'max_output_tokens': 64}
formatter: {'memory_template': '### Instruction:\n{memory}\n', 'prompt_template': '### Input:\n{prompt}\n', 'bot_template': '{bot_name}: {message}\n', 'user_template': '{user_name}: {message}\n', 'response_template': '### Response:\n{bot_name}:', 'truncate_by_message': False}
timestamp: 2025-05-22T21:10:01+00:00
model_name: function_bitil_2025-05-22
Resubmit model
Shutdown handler not registered because Python interpreter is not running in the main thread
run pipeline %s
run pipeline stage %s
Running pipeline stage StressChecker
Received healthy response to inference request in 7.422162771224976s
Received healthy response to inference request in 12.560023069381714s
Received healthy response to inference request in 12.667434453964233s
Received healthy response to inference request in 7.968038320541382s
Received healthy response to inference request in 11.977625370025635s
5 requests
0 failed requests
5th percentile: 7.531337881088257
10th percentile: 7.640512990951538
20th percentile: 7.8588632106781
30th percentile: 8.769955730438232
40th percentile: 10.373790550231934
50th percentile: 11.977625370025635
60th percentile: 12.210584449768067
70th percentile: 12.443543529510498
80th percentile: 12.581505346298218
90th percentile: 12.624469900131226
95th percentile: 12.645952177047729
99th percentile: 12.663137998580932
mean time: 10.519056797027588
%s, retrying in %s seconds...
Received healthy response to inference request in 11.572575092315674s
Received healthy response to inference request in 9.817960023880005s
Received healthy response to inference request in 8.723673820495605s
Received healthy response to inference request in 12.46035885810852s
Received healthy response to inference request in 8.181693315505981s
5 requests
0 failed requests
5th percentile: 8.290089416503907
10th percentile: 8.39848551750183
20th percentile: 8.61527771949768
30th percentile: 8.942531061172485
40th percentile: 9.380245542526245
50th percentile: 9.817960023880005
60th percentile: 10.519806051254273
70th percentile: 11.221652078628539
80th percentile: 11.750131845474243
90th percentile: 12.105245351791382
95th percentile: 12.282802104949951
99th percentile: 12.424847507476807
mean time: 10.151252222061157
%s, retrying in %s seconds...
Received healthy response to inference request in 12.246783256530762s
Received healthy response to inference request in 6.529048681259155s
Received healthy response to inference request in 7.9782280921936035s
Received healthy response to inference request in 11.594842195510864s
Received healthy response to inference request in 7.4302332401275635s
5 requests
0 failed requests
5th percentile: 6.709285593032837
10th percentile: 6.889522504806519
20th percentile: 7.249996328353882
30th percentile: 7.539832210540771
40th percentile: 7.759030151367187
50th percentile: 7.9782280921936035
60th percentile: 9.424873733520508
70th percentile: 10.871519374847411
80th percentile: 11.725230407714843
90th percentile: 11.986006832122802
95th percentile: 12.116395044326783
99th percentile: 12.220705614089965
mean time: 9.15582709312439
clean up pipeline due to error=DeploymentChecksError('Unacceptable 70th percentile latency 10.871519374847411s')
Shutdown handler de-registered
function_bitil_2025-05-22 status is now failed due to DeploymentManager action