submission_id: function_holet_2024-12-09
developer_uid: jxlu90
status: failed
generation_params: {'temperature': 1.0, 'top_p': 1.0, 'min_p': 0.0, 'top_k': 100, 'presence_penalty': 0.0, 'frequency_penalty': 0.0, 'stopping_words': ['\n', '</s>', 'You:'], 'max_input_tokens': 1024, 'best_of': 4, 'max_output_tokens': 68}
formatter: {'memory_template': "{bot_name}'s Persona: {memory}\n####\n", 'prompt_template': '{prompt}\n<START>\n', 'bot_template': '{bot_name}: {message}\n', 'user_template': '{user_name}: {message}\n', 'response_template': '{bot_name}:', 'truncate_by_message': False}
timestamp: 2024-12-09T20:49:20+00:00
model_name: moonshine_nemo70B_150
Resubmit model
Shutdown handler not registered because Python interpreter is not running in the main thread
run pipeline %s
run pipeline stage %s
Running pipeline stage StressChecker
Received healthy response to inference request in 6.466424942016602s
Received healthy response to inference request in 3.917201042175293s
Received healthy response to inference request in 4.228259801864624s
Received healthy response to inference request in 3.756992816925049s
Failed to get response for submission zonemercy-lexical-viral-_8730_v4: ('http://zonemercy-lexical-viral-8730-v4-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'read tcp 127.0.0.1:37844->127.0.0.1:8080: read: connection reset by peer\n')
Received healthy response to inference request in 5.2025439739227295s
5 requests
0 failed requests
5th percentile: 3.7890344619750977
10th percentile: 3.8210761070251467
20th percentile: 3.885159397125244
30th percentile: 3.979412794113159
40th percentile: 4.103836297988892
50th percentile: 4.228259801864624
60th percentile: 4.617973470687867
70th percentile: 5.007687139511108
80th percentile: 5.455320167541504
90th percentile: 5.960872554779053
95th percentile: 6.213648748397827
99th percentile: 6.415869703292847
mean time: 4.714284515380859
%s, retrying in %s seconds...
Received healthy response to inference request in 3.973193645477295s
Received healthy response to inference request in 3.988110303878784s
Received healthy response to inference request in 3.4816529750823975s
Received healthy response to inference request in 2.518831968307495s
Received healthy response to inference request in 2.774225950241089s
5 requests
0 failed requests
5th percentile: 2.569910764694214
10th percentile: 2.6209895610809326
20th percentile: 2.72314715385437
30th percentile: 2.9157113552093508
40th percentile: 3.198682165145874
50th percentile: 3.4816529750823975
60th percentile: 3.6782692432403565
70th percentile: 3.874885511398315
80th percentile: 3.9761769771575928
90th percentile: 3.9821436405181885
95th percentile: 3.9851269721984863
99th percentile: 3.9875136375427247
mean time: 3.3472029685974123
%s, retrying in %s seconds...
Received healthy response to inference request in 3.2253262996673584s
Received healthy response to inference request in 4.288435220718384s
Received healthy response to inference request in 4.950358867645264s
Received healthy response to inference request in 4.0171308517456055s
Received healthy response to inference request in 2.7825565338134766s
5 requests
0 failed requests
5th percentile: 2.8711104869842528
10th percentile: 2.9596644401550294
20th percentile: 3.136772346496582
30th percentile: 3.383687210083008
40th percentile: 3.7004090309143067
50th percentile: 4.0171308517456055
60th percentile: 4.125652599334717
70th percentile: 4.234174346923828
80th percentile: 4.42081995010376
90th percentile: 4.685589408874511
95th percentile: 4.8179741382598875
99th percentile: 4.923881921768189
mean time: 3.8527615547180174
clean up pipeline due to error=DeploymentChecksError('Unacceptable 70th percentile latency 4.234174346923828s')
Shutdown handler de-registered
function_holet_2024-12-09 status is now failed due to DeploymentManager action