developer_uid: chai_backend_admin
submission_id: function_sugin_2024-11-22
model_name: retune_with_base
model_group:
status: inactive
timestamp: 2024-11-22T05:45:50+00:00
num_battles: 11149
num_wins: 6008
celo_rating: 1279.52
family_friendly_score: 0.5840000000000001
family_friendly_standard_error: 0.006970566691453429
submission_type: function
display_name: retune_with_base
is_internal_developer: True
ranking_group: single
us_pacific_date: 2024-11-21
win_ratio: 0.5388824109785632
generation_params: {'temperature': 0.9, 'top_p': 0.9, 'min_p': 0.05, 'top_k': 80, 'presence_penalty': 0.5, 'frequency_penalty': 0.5, 'stopping_words': ['\n', '</s>'], 'max_input_tokens': 1024, 'best_of': 8, 'max_output_tokens': 64}
formatter: {'memory_template': "{bot_name}'s Persona: {memory}\n####\n", 'prompt_template': '{prompt}\n<START>\n', 'bot_template': '{bot_name}: {message}\n', 'user_template': '{user_name}: {message}\n', 'response_template': '{bot_name}:', 'truncate_by_message': False}
Resubmit model
Shutdown handler not registered because Python interpreter is not running in the main thread
run pipeline %s
run pipeline stage %s
Running pipeline stage StressChecker
HTTPConnectionPool(host='guanaco-submitter.guanaco-backend.k2.chaiverse.com', port=80): Read timed out. (read timeout=20)
Received unhealthy response to inference request!
Received healthy response to inference request in 3.3844950199127197s
Received healthy response to inference request in 3.501094102859497s
Received healthy response to inference request in 2.1791839599609375s
Received healthy response to inference request in 4.442118167877197s
5 requests
1 failed requests
5th percentile: 2.4202461719512938
10th percentile: 2.6613083839416505
20th percentile: 3.1434328079223635
30th percentile: 3.4078148365020753
40th percentile: 3.454454469680786
50th percentile: 3.501094102859497
60th percentile: 3.8775037288665772
70th percentile: 4.253913354873657
80th percentile: 7.576238393783572
90th percentile: 13.844478845596315
95th percentile: 16.978599071502682
99th percentile: 19.48589525222778
mean time: 6.7239221096038815
%s, retrying in %s seconds...
Received healthy response to inference request in 3.013716697692871s
Received healthy response to inference request in 2.5656206607818604s
Received healthy response to inference request in 3.075192928314209s
Received healthy response to inference request in 5.830901622772217s
Received healthy response to inference request in 3.6512937545776367s
5 requests
0 failed requests
5th percentile: 2.6552398681640623
10th percentile: 2.7448590755462647
20th percentile: 2.924097490310669
30th percentile: 3.0260119438171387
40th percentile: 3.050602436065674
50th percentile: 3.075192928314209
60th percentile: 3.3056332588195803
70th percentile: 3.536073589324951
80th percentile: 4.087215328216553
90th percentile: 4.959058475494385
95th percentile: 5.3949800491333
99th percentile: 5.743717308044434
mean time: 3.6273451328277586
Pipeline stage StressChecker completed in 54.12s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyTriggerPipeline
run_pipeline:run_in_cloud %s
starting trigger_guanaco_pipeline args=%s
triggered trigger_guanaco_pipeline args=%s
Pipeline stage OfflineFamilyFriendlyTriggerPipeline completed in 2.69s
Shutdown handler de-registered
function_sugin_2024-11-22 status is now deployed due to DeploymentManager action
Shutdown handler registered
run pipeline %s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyScorer
Evaluating %s Family Friendly Score with %s threads
Pipeline stage OfflineFamilyFriendlyScorer completed in 3562.32s
Shutdown handler de-registered
function_sugin_2024-11-22 status is now inactive due to auto deactivation removed underperforming models