submission_id: function_rosar_2025-05-20
developer_uid: chai_backend_admin
status: failed
generation_params: {'temperature': 1.0, 'top_p': 1.0, 'min_p': 0.0, 'top_k': 40, 'presence_penalty': 0.0, 'frequency_penalty': 0.0, 'stopping_words': ['\n'], 'max_input_tokens': 1024, 'best_of': 8, 'max_output_tokens': 64}
formatter: {'memory_template': '### Instruction:\n{memory}\n', 'prompt_template': '### Input:\n{prompt}\n', 'bot_template': '{bot_name}: {message}\n', 'user_template': '{user_name}: {message}\n', 'response_template': '### Response:\n{bot_name}:', 'truncate_by_message': False}
timestamp: 2025-05-20T21:05:03+00:00
model_name: function_rosar_2025-05-20
Resubmit model
Shutdown handler not registered because Python interpreter is not running in the main thread
run pipeline %s
run pipeline stage %s
Running pipeline stage StressChecker
Received healthy response to inference request in 4.279171466827393s
Received healthy response to inference request in 6.817078351974487s
HTTPConnectionPool(host='guanaco-submitter.guanaco-backend.k2.chaiverse.com', port=80): Read timed out. (read timeout=20)
Received unhealthy response to inference request!
Received healthy response to inference request in 5.8421471118927s
Received healthy response to inference request in 6.109342575073242s
5 requests
1 failed requests
5th percentile: 4.591766595840454
10th percentile: 4.904361724853516
20th percentile: 5.529551982879639
30th percentile: 5.895586204528809
40th percentile: 6.002464389801025
50th percentile: 6.109342575073242
60th percentile: 6.392436885833741
70th percentile: 6.675531196594238
80th percentile: 9.476243114471437
90th percentile: 14.794572639465333
95th percentile: 17.45373740196228
99th percentile: 19.58106921195984
mean time: 8.63212833404541
%s, retrying in %s seconds...
Received healthy response to inference request in 6.427734375s
Received healthy response to inference request in 5.811851739883423s
Received healthy response to inference request in 6.533172369003296s
Received healthy response to inference request in 5.096508741378784s
Received healthy response to inference request in 6.622439861297607s
5 requests
0 failed requests
5th percentile: 5.239577341079712
10th percentile: 5.382645940780639
20th percentile: 5.668783140182495
30th percentile: 5.9350282669067385
40th percentile: 6.181381320953369
50th percentile: 6.427734375
60th percentile: 6.469909572601319
70th percentile: 6.5120847702026365
80th percentile: 6.551025867462158
90th percentile: 6.586732864379883
95th percentile: 6.604586362838745
99th percentile: 6.618869161605835
mean time: 6.098341417312622
%s, retrying in %s seconds...
Received healthy response to inference request in 5.724660634994507s
Received healthy response to inference request in 7.311784267425537s
Received healthy response to inference request in 5.603486061096191s
Received healthy response to inference request in 14.513137340545654s
Received healthy response to inference request in 7.1497838497161865s
5 requests
0 failed requests
5th percentile: 5.627720975875855
10th percentile: 5.651955890655517
20th percentile: 5.700425720214843
30th percentile: 6.009685277938843
40th percentile: 6.579734563827515
50th percentile: 7.1497838497161865
60th percentile: 7.214584016799927
70th percentile: 7.279384183883667
80th percentile: 8.752054882049562
90th percentile: 11.632596111297609
95th percentile: 13.07286672592163
99th percentile: 14.225083217620849
mean time: 8.060570430755615
clean up pipeline due to error=DeploymentChecksError('Unacceptable 70th percentile latency 7.279384183883667s')
Shutdown handler de-registered
function_rosar_2025-05-20 status is now failed due to DeploymentManager action