submission_id: function_lafas_2024-11-06
developer_uid: chai_backend_admin
formatter: {'memory_template': "{bot_name}'s Persona: {memory}\n####\n", 'prompt_template': '{prompt}\n<START>\n', 'bot_template': '{bot_name}: {message}\n', 'user_template': '{user_name}: {message}\n', 'response_template': '{bot_name}:', 'truncate_by_message': False}
generation_params: {'temperature': 0.9, 'top_p': 0.9, 'min_p': 0.05, 'top_k': 80, 'presence_penalty': 0.5, 'frequency_penalty': 0.5, 'stopping_words': ['\n', '</s>'], 'max_input_tokens': 1024, 'best_of': 8, 'max_output_tokens': 64}
model_name: retune_with_base
status: failed
timestamp: 2024-11-06T22:19:32+00:00
Resubmit model
Shutdown handler not registered because Python interpreter is not running in the main thread
run pipeline %s
run pipeline stage %s
Running pipeline stage StressChecker
Received healthy response to inference request in 6.395528316497803s
Received healthy response to inference request in 3.0769431591033936s
Received healthy response to inference request in 2.527992010116577s
Received healthy response to inference request in 4.84425163269043s
Received healthy response to inference request in 3.4889490604400635s
5 requests
0 failed requests
5th percentile: 2.6377822399139403
10th percentile: 2.7475724697113035
20th percentile: 2.9671529293060304
30th percentile: 3.1593443393707275
40th percentile: 3.3241466999053957
50th percentile: 3.4889490604400635
60th percentile: 4.03107008934021
70th percentile: 4.573191118240356
80th percentile: 5.154506969451905
90th percentile: 5.775017642974854
95th percentile: 6.085272979736327
99th percentile: 6.333477249145508
mean time: 4.0667328357696535
%s, retrying in %s seconds...
Received healthy response to inference request in 3.4391825199127197s
Received healthy response to inference request in 3.027082920074463s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Received healthy response to inference request in 4.760708808898926s
Received healthy response to inference request in 4.44622278213501s
Received healthy response to inference request in 6.008378982543945s
5 requests
0 failed requests
5th percentile: 3.109502840042114
10th percentile: 3.1919227600097657
20th percentile: 3.3567625999450685
30th percentile: 3.6405905723571776
40th percentile: 4.043406677246094
50th percentile: 4.44622278213501
60th percentile: 4.572017192840576
70th percentile: 4.697811603546143
80th percentile: 5.01024284362793
90th percentile: 5.509310913085938
95th percentile: 5.758844947814941
99th percentile: 5.9584721755981445
mean time: 4.3363152027130125
%s, retrying in %s seconds...
Received healthy response to inference request in 3.6734960079193115s
Received healthy response to inference request in 5.181747674942017s
Received healthy response to inference request in 4.4400811195373535s
Received healthy response to inference request in 7.520281791687012s
Received healthy response to inference request in 6.963424205780029s
5 requests
0 failed requests
5th percentile: 3.8268130302429197
10th percentile: 3.9801300525665284
20th percentile: 4.286764097213745
30th percentile: 4.588414430618286
40th percentile: 4.885081052780151
50th percentile: 5.181747674942017
60th percentile: 5.8944182872772215
70th percentile: 6.607088899612426
80th percentile: 7.074795722961426
90th percentile: 7.297538757324219
95th percentile: 7.408910274505615
99th percentile: 7.498007488250733
mean time: 5.5558061599731445
clean up pipeline due to error=DeploymentChecksError('Unacceptable 70th percentile latency 6.607088899612426s')
Shutdown handler de-registered
function_lafas_2024-11-06 status is now failed due to DeploymentManager action