submission_id: function_gabif_2024-10-18
developer_uid: chai_backend_admin
celo_rating: 1277.08
display_name: reward_blend_default_full_bon
family_friendly_score: 0.5804886374719594
family_friendly_standard_error: 0.004846528818083632
formatter: {'memory_template': "{bot_name}'s Persona: {memory}\n####\n", 'prompt_template': '{prompt}\n<START>\n', 'bot_template': '{bot_name}: {message}\n', 'user_template': '{user_name}: {message}\n', 'response_template': '{bot_name}:', 'truncate_by_message': False}
generation_params: {'temperature': 0.9, 'top_p': 1.0, 'min_p': 0.0, 'top_k': 50, 'presence_penalty': 0.5, 'frequency_penalty': 0.5, 'stopping_words': ['\n', '</s>', '<|user|>', '###'], 'max_input_tokens': 512, 'best_of': 4, 'max_output_tokens': 64}
is_internal_developer: True
model_group:
model_name: reward_blend_default_full_bon
num_battles: 13917
num_wins: 7295
ranking_group: single
status: torndown
submission_type: function
timestamp: 2024-10-18T19:40:01+00:00
us_pacific_date: 2024-10-18
win_ratio: 0.5241790615793633
Resubmit model
Shutdown handler not registered because Python interpreter is not running in the main thread
run pipeline %s
run pipeline stage %s
Running pipeline stage StressChecker
HTTPConnectionPool(host='guanaco-submitter.guanaco-backend.k2.chaiverse.com', port=80): Read timed out. (read timeout=20)
Received unhealthy response to inference request!
HTTPConnectionPool(host='guanaco-submitter.guanaco-backend.k2.chaiverse.com', port=80): Read timed out. (read timeout=20)
Received unhealthy response to inference request!
HTTPConnectionPool(host='guanaco-submitter.guanaco-backend.k2.chaiverse.com', port=80): Read timed out. (read timeout=20)
Received unhealthy response to inference request!
Received healthy response to inference request in 4.335912227630615s
Received healthy response to inference request in 5.0079944133758545s
5 requests
3 failed requests
5th percentile: 4.470328664779663
10th percentile: 4.604745101928711
20th percentile: 4.873577976226807
30th percentile: 8.029494190216063
40th percentile: 14.072493743896485
50th percentile: 20.115493297576904
60th percentile: 20.11699447631836
70th percentile: 20.118495655059814
80th percentile: 20.19317317008972
90th percentile: 20.34102702140808
95th percentile: 20.41495394706726
99th percentile: 20.474095487594603
mean time: 14.01350541114807
%s, retrying in %s seconds...
Failed to get response for submission jic062-dpo-v3-0-nemo_v1: ('http://jic062-dpo-v3-0-nemo-v1-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'activator request timeout')
HTTPConnectionPool(host='guanaco-submitter.guanaco-backend.k2.chaiverse.com', port=80): Read timed out. (read timeout=20)
Received unhealthy response to inference request!
Received healthy response to inference request in 3.8311593532562256s
Received healthy response to inference request in 2.253251791000366s
Received healthy response to inference request in 5.24559760093689s
Received healthy response to inference request in 2.6731884479522705s
5 requests
1 failed requests
5th percentile: 2.337239122390747
10th percentile: 2.421226453781128
20th percentile: 2.5892011165618896
30th percentile: 2.9047826290130616
40th percentile: 3.367970991134644
50th percentile: 3.8311593532562256
60th percentile: 4.396934652328492
70th percentile: 4.962709951400757
80th percentile: 8.220286464691165
90th percentile: 14.169664192199708
95th percentile: 17.144353055953978
99th percentile: 19.5241041469574
mean time: 6.824447822570801
%s, retrying in %s seconds...
Received healthy response to inference request in 3.1607606410980225s
Received healthy response to inference request in 3.2872040271759033s
Received healthy response to inference request in 3.3460211753845215s
Received healthy response to inference request in 2.592740297317505s
Received healthy response to inference request in 2.2479910850524902s
5 requests
0 failed requests
5th percentile: 2.316940927505493
10th percentile: 2.385890769958496
20th percentile: 2.523790454864502
30th percentile: 2.7063443660736084
40th percentile: 2.9335525035858154
50th percentile: 3.1607606410980225
60th percentile: 3.211337995529175
70th percentile: 3.261915349960327
80th percentile: 3.298967456817627
90th percentile: 3.322494316101074
95th percentile: 3.3342577457427978
99th percentile: 3.3436684894561766
mean time: 2.9269434452056884
Pipeline stage StressChecker completed in 122.21s
Shutdown handler de-registered
function_gabif_2024-10-18 status is now deployed due to DeploymentManager action
function_gabif_2024-10-18 status is now inactive due to auto deactivation removed underperforming models
function_gabif_2024-10-18 status is now torndown due to DeploymentManager action