developer_uid: chai_backend_admin
submission_id: function_sulat_2025-05-12
model_name: function_sulat_2025-05-12
model_group:
status: torndown
timestamp: 2025-05-12T21:06:00+00:00
num_battles: 8471
num_wins: 4310
celo_rating: 1293.83
family_friendly_score: 0.0
family_friendly_standard_error: 0.0
submission_type: function
display_name: function_sulat_2025-05-12
is_internal_developer: True
ranking_group: single
us_pacific_date: 2025-05-12
win_ratio: 0.5087947113681974
generation_params: {'temperature': 1.0, 'top_p': 1.0, 'min_p': 0.0, 'top_k': 40, 'presence_penalty': 0.0, 'frequency_penalty': 0.0, 'stopping_words': ['\n'], 'max_input_tokens': 1024, 'best_of': 8, 'max_output_tokens': 64}
formatter: {'memory_template': '### Instruction:\n{memory}\n', 'prompt_template': '### Input:\n{prompt}\n', 'bot_template': '{bot_name}: {message}\n', 'user_template': '{user_name}: {message}\n', 'response_template': '### Response:\n{bot_name}:', 'truncate_by_message': False}
Resubmit model
Shutdown handler not registered because Python interpreter is not running in the main thread
run pipeline %s
run pipeline stage %s
Running pipeline stage StressChecker
Received healthy response to inference request in 6.765953302383423s
Received healthy response to inference request in 11.203210592269897s
Received healthy response to inference request in 20.018983364105225s
Received healthy response to inference request in 8.197057008743286s
Received healthy response to inference request in 16.639889240264893s
5 requests
0 failed requests
5th percentile: 7.052174043655396
10th percentile: 7.3383947849273685
20th percentile: 7.910836267471313
30th percentile: 8.798287725448608
40th percentile: 10.000749158859254
50th percentile: 11.203210592269897
60th percentile: 13.377882051467894
70th percentile: 15.552553510665893
80th percentile: 17.31570806503296
90th percentile: 18.66734571456909
95th percentile: 19.343164539337156
99th percentile: 19.88381959915161
mean time: 12.565018701553345
%s, retrying in %s seconds...
Received healthy response to inference request in 4.716709613800049s
HTTPConnectionPool(host='guanaco-submitter.guanaco-backend.k2.chaiverse.com', port=80): Read timed out. (read timeout=20)
Received unhealthy response to inference request!
Received healthy response to inference request in 5.668703317642212s
Received healthy response to inference request in 6.78268027305603s
Received healthy response to inference request in 7.203331470489502s
5 requests
1 failed requests
5th percentile: 4.907108354568481
10th percentile: 5.097507095336914
20th percentile: 5.4783045768737795
30th percentile: 5.891498708724976
40th percentile: 6.337089490890503
50th percentile: 6.78268027305603
60th percentile: 6.950940752029419
70th percentile: 7.119201231002807
80th percentile: 9.785927200317385
90th percentile: 14.951118659973146
95th percentile: 17.533714389801023
99th percentile: 19.59979097366333
mean time: 8.89754695892334
%s, retrying in %s seconds...
HTTPConnectionPool(host='guanaco-submitter.guanaco-backend.k2.chaiverse.com', port=80): Read timed out. (read timeout=20)
Received unhealthy response to inference request!
HTTPConnectionPool(host='guanaco-submitter.guanaco-backend.k2.chaiverse.com', port=80): Read timed out. (read timeout=20)
Received unhealthy response to inference request!
HTTPConnectionPool(host='guanaco-submitter.guanaco-backend.k2.chaiverse.com', port=80): Read timed out. (read timeout=20)
Received unhealthy response to inference request!
Received healthy response to inference request in 6.790820837020874s
Received healthy response to inference request in 8.47529411315918s
5 requests
3 failed requests
5th percentile: 7.1277154922485355
10th percentile: 7.464610147476196
20th percentile: 8.138399457931518
30th percentile: 10.80508303642273
40th percentile: 15.464660882949829
50th percentile: 20.12423872947693
60th percentile: 20.126976823806764
70th percentile: 20.129714918136596
80th percentile: 20.131117391586304
90th percentile: 20.131184244155882
95th percentile: 20.131217670440673
99th percentile: 20.131244411468504
mean time: 15.130537748336792
clean up pipeline due to error=DeploymentChecksError('Unacceptable number of predict errors: 60.0%')
Shutdown handler de-registered
function_sulat_2025-05-12 status is now failed due to DeploymentManager action
function_sulat_2025-05-12 status is now inactive due to auto deactivation removed underperforming models
function_sulat_2025-05-12 status is now torndown due to DeploymentManager action