submission_id: function_rilim_2024-12-19
developer_uid: jxlu90
status: failed
generation_params: {'temperature': 1.0, 'top_p': 1.0, 'min_p': 0.0, 'top_k': 100, 'presence_penalty': 0.0, 'frequency_penalty': 0.0, 'stopping_words': ['\n', '</s>', 'You:'], 'max_input_tokens': 2048, 'best_of': 4, 'max_output_tokens': 64}
formatter: {'memory_template': "{bot_name}'s Persona: {memory}\n####\n", 'prompt_template': '{prompt}\n<START>\n', 'bot_template': '{bot_name}: {message}\n', 'user_template': '{user_name}: {message}\n', 'response_template': '{bot_name}:', 'truncate_by_message': False}
timestamp: 2024-12-19T00:06:18+00:00
model_name: run_3_v65_moonshine
Resubmit model
Shutdown handler not registered because Python interpreter is not running in the main thread
run pipeline %s
run pipeline stage %s
Running pipeline stage StressChecker
Received healthy response to inference request in 3.8800246715545654s
Received healthy response to inference request in 4.375488519668579s
Received healthy response to inference request in 3.9682796001434326s
Received healthy response to inference request in 2.3769125938415527s
Received healthy response to inference request in 3.9542295932769775s
5 requests
0 failed requests
5th percentile: 2.6775350093841555
10th percentile: 2.9781574249267577
20th percentile: 3.5794022560119627
30th percentile: 3.8948656558990478
40th percentile: 3.924547624588013
50th percentile: 3.9542295932769775
60th percentile: 3.9598495960235596
70th percentile: 3.9654695987701416
80th percentile: 4.049721384048462
90th percentile: 4.212604951858521
95th percentile: 4.2940467357635494
99th percentile: 4.359200162887573
mean time: 3.7109869956970214
%s, retrying in %s seconds...
Received healthy response to inference request in 3.154470205307007s
Received healthy response to inference request in 2.2834622859954834s
Received healthy response to inference request in 3.6710128784179688s
Received healthy response to inference request in 4.137973785400391s
Received healthy response to inference request in 5.302254915237427s
5 requests
0 failed requests
5th percentile: 2.457663869857788
10th percentile: 2.631865453720093
20th percentile: 2.980268621444702
30th percentile: 3.2577787399291993
40th percentile: 3.464395809173584
50th percentile: 3.6710128784179688
60th percentile: 3.8577972412109376
70th percentile: 4.044581604003906
80th percentile: 4.370830011367798
90th percentile: 4.836542463302612
95th percentile: 5.069398689270019
99th percentile: 5.255683670043945
mean time: 3.7098348140716553
%s, retrying in %s seconds...
Received healthy response to inference request in 7.499354600906372s
Received healthy response to inference request in 7.204309463500977s
Received healthy response to inference request in 2.656010866165161s
Received healthy response to inference request in 3.2530016899108887s
Received healthy response to inference request in 4.903976917266846s
5 requests
0 failed requests
5th percentile: 2.7754090309143065
10th percentile: 2.8948071956634522
20th percentile: 3.1336035251617433
30th percentile: 3.58319673538208
40th percentile: 4.243586826324463
50th percentile: 4.903976917266846
60th percentile: 5.8241099357604975
70th percentile: 6.74424295425415
80th percentile: 7.263318490982056
90th percentile: 7.3813365459442135
95th percentile: 7.440345573425293
99th percentile: 7.487552795410156
mean time: 5.1033307075500485
clean up pipeline due to error=DeploymentChecksError('Unacceptable 70th percentile latency 6.74424295425415s')
Shutdown handler de-registered
function_rilim_2024-12-19 status is now failed due to DeploymentManager action