submission_id: function_gulen_2024-12-19
developer_uid: jxlu90
status: failed
generation_params: {'temperature': 1.0, 'top_p': 1.0, 'min_p': 0.0, 'top_k': 100, 'presence_penalty': 0.0, 'frequency_penalty': 0.0, 'stopping_words': ['\n', '</s>', 'You:'], 'max_input_tokens': 2048, 'best_of': 4, 'max_output_tokens': 64}
formatter: {'memory_template': "{bot_name}'s Persona: {memory}\n####\n", 'prompt_template': '{prompt}\n<START>\n', 'bot_template': '{bot_name}: {message}\n', 'user_template': '{user_name}: {message}\n', 'response_template': '{bot_name}:', 'truncate_by_message': False}
timestamp: 2024-12-19T16:50:37+00:00
model_name: run_3_v65_ranked
Resubmit model
Shutdown handler not registered because Python interpreter is not running in the main thread
run pipeline %s
run pipeline stage %s
Running pipeline stage StressChecker
Received healthy response to inference request in 5.433512449264526s
Received healthy response to inference request in 4.8165717124938965s
Received healthy response to inference request in 4.336465358734131s
Received healthy response to inference request in 4.500162839889526s
Received healthy response to inference request in 3.6307873725891113s
5 requests
0 failed requests
5th percentile: 3.771922969818115
10th percentile: 3.913058567047119
20th percentile: 4.195329761505127
30th percentile: 4.36920485496521
40th percentile: 4.434683847427368
50th percentile: 4.500162839889526
60th percentile: 4.626726388931274
70th percentile: 4.7532899379730225
80th percentile: 4.939959859848023
90th percentile: 5.186736154556274
95th percentile: 5.3101243019104
99th percentile: 5.408834819793701
mean time: 4.543499946594238
%s, retrying in %s seconds...
Received healthy response to inference request in 3.3720591068267822s
Received healthy response to inference request in 6.87788724899292s
Received healthy response to inference request in 4.742891073226929s
Received healthy response to inference request in 5.056391716003418s
Received healthy response to inference request in 3.830643653869629s
5 requests
0 failed requests
5th percentile: 3.4637760162353515
10th percentile: 3.5554929256439207
20th percentile: 3.7389267444610597
30th percentile: 4.013093137741089
40th percentile: 4.377992105484009
50th percentile: 4.742891073226929
60th percentile: 4.868291330337525
70th percentile: 4.9936915874481205
80th percentile: 5.420690822601319
90th percentile: 6.1492890357971195
95th percentile: 6.513588142395019
99th percentile: 6.805027427673339
mean time: 4.775974559783935
%s, retrying in %s seconds...
Received healthy response to inference request in 4.620378732681274s
Received healthy response to inference request in 4.8020830154418945s
Received healthy response to inference request in 5.723395824432373s
Received healthy response to inference request in 7.029055833816528s
Received healthy response to inference request in 6.925957441329956s
5 requests
0 failed requests
5th percentile: 4.656719589233399
10th percentile: 4.693060445785522
20th percentile: 4.76574215888977
30th percentile: 4.98634557723999
40th percentile: 5.354870700836182
50th percentile: 5.723395824432373
60th percentile: 6.204420471191407
70th percentile: 6.685445117950439
80th percentile: 6.94657711982727
90th percentile: 6.9878164768219
95th percentile: 7.008436155319214
99th percentile: 7.024931898117066
mean time: 5.8201741695404055
clean up pipeline due to error=DeploymentChecksError('Unacceptable 70th percentile latency 6.685445117950439s')
Shutdown handler de-registered
function_gulen_2024-12-19 status is now failed due to DeploymentManager action