submission_id: function_datuk_2025-07-27
developer_uid: chai_backend_admin
status: failed
generation_params: {'temperature': 1.0, 'top_p': 1.0, 'min_p': 0.0, 'top_k': 40, 'presence_penalty': 0.0, 'frequency_penalty': 0.0, 'stopping_words': ['\n'], 'max_input_tokens': 1024, 'best_of': 8, 'max_output_tokens': 64}
formatter: {'memory_template': '### Instruction:\n{memory}\n', 'prompt_template': '### Input:\n{prompt}\n', 'bot_template': '{bot_name}: {message}\n', 'user_template': '{user_name}: {message}\n', 'response_template': '### Response:\n{bot_name}:', 'truncate_by_message': False}
timestamp: 2025-07-27T07:53:03+00:00
model_name: function_datuk_2025-07-27
Resubmit model
Shutdown handler not registered because Python interpreter is not running in the main thread
run pipeline %s
run pipeline stage %s
Running pipeline stage StressChecker
Received healthy response to inference request in 2.409639835357666s
Received healthy response to inference request in 4.47514009475708s
Received healthy response to inference request in 2.8090131282806396s
Received healthy response to inference request in 4.636168479919434s
Received healthy response to inference request in 3.0969443321228027s
5 requests
0 failed requests
5th percentile: 2.4895144939422607
10th percentile: 2.5693891525268553
20th percentile: 2.729138469696045
30th percentile: 2.866599369049072
40th percentile: 2.9817718505859374
50th percentile: 3.0969443321228027
60th percentile: 3.6482226371765134
70th percentile: 4.199500942230224
80th percentile: 4.50734577178955
90th percentile: 4.571757125854492
95th percentile: 4.603962802886963
99th percentile: 4.62972734451294
mean time: 3.4853811740875242
%s, retrying in %s seconds...
Received healthy response to inference request in 6.620399236679077s
Received healthy response to inference request in 3.2763185501098633s
Received healthy response to inference request in 4.969214677810669s
Received healthy response to inference request in 2.8436734676361084s
Received healthy response to inference request in 5.139024257659912s
5 requests
0 failed requests
5th percentile: 2.9302024841308594
10th percentile: 3.0167315006256104
20th percentile: 3.1897895336151123
30th percentile: 3.6148977756500242
40th percentile: 4.292056226730347
50th percentile: 4.969214677810669
60th percentile: 5.037138509750366
70th percentile: 5.105062341690063
80th percentile: 5.435299253463746
90th percentile: 6.027849245071411
95th percentile: 6.324124240875244
99th percentile: 6.56114423751831
mean time: 4.569726037979126
%s, retrying in %s seconds...
Received healthy response to inference request in 3.35813570022583s
Received healthy response to inference request in 4.007593393325806s
Received healthy response to inference request in 2.2894787788391113s
Received healthy response to inference request in 2.2566494941711426s
Received healthy response to inference request in 6.785814523696899s
5 requests
0 failed requests
5th percentile: 2.263215351104736
10th percentile: 2.26978120803833
20th percentile: 2.2829129219055178
30th percentile: 2.503210163116455
40th percentile: 2.930672931671143
50th percentile: 3.35813570022583
60th percentile: 3.61791877746582
70th percentile: 3.8777018547058106
80th percentile: 4.563237619400025
90th percentile: 5.6745260715484624
95th percentile: 6.2301702976226805
99th percentile: 6.674685678482056
mean time: 3.739534378051758
clean up pipeline due to error=DeploymentChecksError('Unacceptable 70th percentile latency 3.8777018547058106s')
Shutdown handler de-registered
function_datuk_2025-07-27 status is now failed due to DeploymentManager action