submission_id: function_nubel_2024-12-31
developer_uid: jxlu90
status: torndown
generation_params: {'temperature': 0.9, 'top_p': 0.95, 'min_p': 0.0, 'top_k': 50, 'presence_penalty': 0.0, 'frequency_penalty': 0.1, 'stopping_words': ['\n', '</s>', 'You:'], 'max_input_tokens': 1024, 'best_of': 4, 'max_output_tokens': 64}
formatter: {'memory_template': "{bot_name}'s Persona: {memory}\n####\n", 'prompt_template': '{prompt}\n<START>\n', 'bot_template': '{bot_name}: {message}\n', 'user_template': '{user_name}: {message}\n', 'response_template': '{bot_name}:', 'truncate_by_message': False}
timestamp: 2025-12-20T00:31:09+00:00
model_name: llama_405b_bo4_ctx1k
Resubmit model
Shutdown handler not registered because Python interpreter is not running in the main thread
run pipeline %s
run pipeline stage %s
Running pipeline stage StressChecker
Received healthy response to inference request in 4.608228445053101s
Received healthy response to inference request in 1.9677715301513672s
Received healthy response to inference request in 4.529925346374512s
Received healthy response to inference request in 4.0134193897247314s
Received healthy response to inference request in 4.031874418258667s
5 requests
0 failed requests
5th percentile: 2.37690110206604
10th percentile: 2.786030673980713
20th percentile: 3.604289817810059
30th percentile: 4.017110395431518
40th percentile: 4.024492406845093
50th percentile: 4.031874418258667
60th percentile: 4.231094789505005
70th percentile: 4.430315160751343
80th percentile: 4.545585966110229
90th percentile: 4.576907205581665
95th percentile: 4.592567825317383
99th percentile: 4.6050963211059575
mean time: 3.8302438259124756
%s, retrying in %s seconds...
Received healthy response to inference request in 3.4559004306793213s
Received healthy response to inference request in 5.0579447746276855s
Received healthy response to inference request in 3.4773032665252686s
Received healthy response to inference request in 3.238736152648926s
Received healthy response to inference request in 4.035742521286011s
5 requests
0 failed requests
5th percentile: 3.282169008255005
10th percentile: 3.325601863861084
20th percentile: 3.4124675750732423
30th percentile: 3.4601809978485107
40th percentile: 3.4687421321868896
50th percentile: 3.4773032665252686
60th percentile: 3.7006789684295653
70th percentile: 3.924054670333862
80th percentile: 4.240182971954346
90th percentile: 4.6490638732910154
95th percentile: 4.85350432395935
99th percentile: 5.017056684494018
mean time: 3.8531254291534425
%s, retrying in %s seconds...
Received healthy response to inference request in 2.414055109024048s
Received healthy response to inference request in 4.1274073123931885s
Received healthy response to inference request in 3.7921290397644043s
Received healthy response to inference request in 4.5474326610565186s
Received healthy response to inference request in 2.76377010345459s
5 requests
0 failed requests
5th percentile: 2.483998107910156
10th percentile: 2.5539411067962647
20th percentile: 2.6938271045684816
30th percentile: 2.9694418907165527
40th percentile: 3.3807854652404785
50th percentile: 3.7921290397644043
60th percentile: 3.926240348815918
70th percentile: 4.0603516578674315
80th percentile: 4.211412382125855
90th percentile: 4.379422521591186
95th percentile: 4.463427591323852
99th percentile: 4.530631647109986
mean time: 3.5289588451385496
clean up pipeline due to error=DeploymentChecksError('Unacceptable 70th percentile latency 4.0603516578674315s')
Shutdown handler de-registered
function_nubel_2024-12-31 status is now failed due to DeploymentManager action
function_nubel_2024-12-31 status is now torndown due to DeploymentManager action