submission_id: function_gijot_2024-11-06
developer_uid: chai_backend_admin
formatter: {'memory_template': "{bot_name}'s Persona: {memory}\n####\n", 'prompt_template': '{prompt}\n<START>\n', 'bot_template': '{bot_name}: {message}\n', 'user_template': '{user_name}: {message}\n', 'response_template': '{bot_name}:', 'truncate_by_message': False}
generation_params: {'temperature': 0.9, 'top_p': 0.9, 'min_p': 0.05, 'top_k': 80, 'presence_penalty': 0.5, 'frequency_penalty': 0.5, 'stopping_words': ['\n', '</s>'], 'max_input_tokens': 1024, 'best_of': 8, 'max_output_tokens': 64}
model_name: retune_with_base
status: failed
timestamp: 2024-11-06T22:21:52+00:00
Resubmit model
Shutdown handler not registered because Python interpreter is not running in the main thread
run pipeline %s
run pipeline stage %s
Running pipeline stage StressChecker
Received healthy response to inference request in 9.383114576339722s
Received healthy response to inference request in 7.9003496170043945s
Received healthy response to inference request in 10.66761565208435s
Received healthy response to inference request in 10.416965246200562s
Received healthy response to inference request in 8.691878080368042s
5 requests
0 failed requests
5th percentile: 8.058655309677125
10th percentile: 8.216961002349853
20th percentile: 8.533572387695312
30th percentile: 8.830125379562379
40th percentile: 9.10661997795105
50th percentile: 9.383114576339722
60th percentile: 9.796654844284058
70th percentile: 10.210195112228394
80th percentile: 10.467095327377319
90th percentile: 10.567355489730835
95th percentile: 10.617485570907593
99th percentile: 10.657589635849
mean time: 9.411984634399413
%s, retrying in %s seconds...
Received healthy response to inference request in 9.877833604812622s
Received healthy response to inference request in 8.933451414108276s
Received healthy response to inference request in 10.49258017539978s
Received healthy response to inference request in 7.439162731170654s
Received healthy response to inference request in 12.89778470993042s
5 requests
0 failed requests
5th percentile: 7.7380204677581785
10th percentile: 8.036878204345703
20th percentile: 8.634593677520751
30th percentile: 9.122327852249146
40th percentile: 9.500080728530884
50th percentile: 9.877833604812622
60th percentile: 10.123732233047486
70th percentile: 10.36963086128235
80th percentile: 10.97362108230591
90th percentile: 11.935702896118164
95th percentile: 12.416743803024291
99th percentile: 12.801576528549194
mean time: 9.92816252708435
%s, retrying in %s seconds...
Received healthy response to inference request in 10.417154788970947s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Received healthy response to inference request in 9.358238935470581s
Received healthy response to inference request in 10.15946340560913s
Received healthy response to inference request in 3.807389736175537s
Received healthy response to inference request in 8.223499774932861s
5 requests
0 failed requests
5th percentile: 4.690611743927002
10th percentile: 5.573833751678467
20th percentile: 7.340277767181397
30th percentile: 8.450447607040406
40th percentile: 8.904343271255494
50th percentile: 9.358238935470581
60th percentile: 9.678728723526001
70th percentile: 9.999218511581422
80th percentile: 10.211001682281495
90th percentile: 10.314078235626221
95th percentile: 10.365616512298583
99th percentile: 10.406847133636475
mean time: 8.39314932823181
clean up pipeline due to error=DeploymentChecksError('Unacceptable 70th percentile latency 9.999218511581422s')
Shutdown handler de-registered
function_gijot_2024-11-06 status is now failed due to DeploymentManager action