submission_id: function_pagik_2024-10-29
developer_uid: valentin
formatter: {'memory_template': "{bot_name}'s Persona: {memory}\n####\n", 'prompt_template': '{prompt}\n<START>\n', 'bot_template': '{bot_name}: {message}\n', 'user_template': '{user_name}: {message}\n', 'response_template': '{bot_name}:', 'truncate_by_message': False}
generation_params: {'temperature': 1.0, 'top_p': 1.0, 'min_p': 0.0, 'top_k': 40, 'presence_penalty': 0.0, 'frequency_penalty': 0.0, 'stopping_words': ['\n'], 'max_input_tokens': 1024, 'best_of': 8, 'max_output_tokens': 64}
model_name: guard_ensemble_w0pt1_bo16
status: torndown
timestamp: 2024-10-29T14:29:53+00:00
Resubmit model
Shutdown handler not registered because Python interpreter is not running in the main thread
run pipeline %s
run pipeline stage %s
Running pipeline stage StressChecker
HTTPConnectionPool(host='0.0.0.0', port=8080): Read timed out. (read timeout=20)
Received unhealthy response to inference request!
Received healthy response to inference request in 19.804829120635986s
HTTPConnectionPool(host='0.0.0.0', port=8080): Read timed out. (read timeout=20)
Received unhealthy response to inference request!
HTTPConnectionPool(host='0.0.0.0', port=8080): Read timed out. (read timeout=20)
Received unhealthy response to inference request!
HTTPConnectionPool(host='0.0.0.0', port=8080): Read timed out. (read timeout=20)
Received unhealthy response to inference request!
5 requests
4 failed requests
5th percentile: 19.900110149383544
10th percentile: 19.995391178131104
20th percentile: 20.185953235626222
30th percentile: 20.28305563926697
40th percentile: 20.286698389053345
50th percentile: 20.29034113883972
60th percentile: 20.30375781059265
70th percentile: 20.31717448234558
80th percentile: 20.33252959251404
90th percentile: 20.34982314109802
95th percentile: 20.358469915390014
99th percentile: 20.36538733482361
mean time: 20.21348080635071
%s, retrying in %s seconds...
HTTPConnectionPool(host='0.0.0.0', port=8080): Read timed out. (read timeout=20)
Received unhealthy response to inference request!
HTTPConnectionPool(host='0.0.0.0', port=8080): Read timed out. (read timeout=20)
Received unhealthy response to inference request!
HTTPConnectionPool(host='0.0.0.0', port=8080): Read timed out. (read timeout=20)
Received unhealthy response to inference request!
HTTPConnectionPool(host='0.0.0.0', port=8080): Read timed out. (read timeout=20)
Received unhealthy response to inference request!
Received healthy response to inference request in 16.370203018188477s
5 requests
4 failed requests
5th percentile: 17.1615168094635
10th percentile: 17.952830600738526
20th percentile: 19.535458183288576
30th percentile: 20.326879739761353
40th percentile: 20.32709527015686
50th percentile: 20.327310800552368
60th percentile: 20.328522062301637
70th percentile: 20.329733324050903
80th percentile: 20.33661518096924
90th percentile: 20.34916763305664
95th percentile: 20.35544385910034
99th percentile: 20.360464839935304
mean time: 19.543268966674805
%s, retrying in %s seconds...
Received healthy response to inference request in 16.455142974853516s
HTTPConnectionPool(host='0.0.0.0', port=8080): Read timed out. (read timeout=20)
Received unhealthy response to inference request!
Received healthy response to inference request in 17.145631074905396s
Received healthy response to inference request in 11.900092840194702s
Received healthy response to inference request in 13.735123872756958s
5 requests
1 failed requests
5th percentile: 12.267099046707154
10th percentile: 12.634105253219605
20th percentile: 13.368117666244506
30th percentile: 14.27912769317627
40th percentile: 15.367135334014893
50th percentile: 16.455142974853516
60th percentile: 16.731338214874267
70th percentile: 17.007533454895018
80th percentile: 17.785577440261843
90th percentile: 19.06547017097473
95th percentile: 19.705416536331175
99th percentile: 20.217373628616333
mean time: 15.916270732879639
clean up pipeline due to error=DeploymentChecksError('Unacceptable number of predict errors: 20.0%')
Shutdown handler de-registered
function_pagik_2024-10-29 status is now failed due to DeploymentManager action
function_pagik_2024-10-29 status is now torndown due to DeploymentManager action