developer_uid: chai_backend_admin
submission_id: function_tapif_2024-11-21
model_name: retune_with_base
model_group:
status: inactive
timestamp: 2024-11-21T20:13:34+00:00
num_battles: 13905
num_wins: 7328
celo_rating: 1268.44
family_friendly_score: 0.5878
family_friendly_standard_error: 0.006961194725045407
submission_type: function
display_name: retune_with_base
is_internal_developer: True
ranking_group: single
us_pacific_date: 2024-11-21
win_ratio: 0.5270046745774901
generation_params: {'temperature': 0.9, 'top_p': 0.9, 'min_p': 0.05, 'top_k': 80, 'presence_penalty': 0.5, 'frequency_penalty': 0.5, 'stopping_words': ['\n', '</s>'], 'max_input_tokens': 1024, 'best_of': 8, 'max_output_tokens': 64}
formatter: {'memory_template': "{bot_name}'s Persona: {memory}\n####\n", 'prompt_template': '{prompt}\n<START>\n', 'bot_template': '{bot_name}: {message}\n', 'user_template': '{user_name}: {message}\n', 'response_template': '{bot_name}:', 'truncate_by_message': False}
Resubmit model
Shutdown handler not registered because Python interpreter is not running in the main thread
run pipeline %s
run pipeline stage %s
Running pipeline stage StressChecker
HTTPConnectionPool(host='guanaco-submitter.guanaco-backend.k2.chaiverse.com', port=80): Read timed out. (read timeout=20)
Received unhealthy response to inference request!
Received healthy response to inference request in 3.1575262546539307s
Received healthy response to inference request in 4.070690870285034s
Received healthy response to inference request in 5.081867218017578s
Received healthy response to inference request in 4.193402290344238s
5 requests
1 failed requests
5th percentile: 3.3401591777801514
10th percentile: 3.522792100906372
20th percentile: 3.8880579471588135
30th percentile: 4.095233154296875
40th percentile: 4.144317722320556
50th percentile: 4.193402290344238
60th percentile: 4.548788261413574
70th percentile: 4.90417423248291
80th percentile: 8.104970359802248
90th percentile: 14.151176643371583
95th percentile: 17.174279785156248
99th percentile: 19.592762298583985
mean time: 7.34017391204834
%s, retrying in %s seconds...
Received healthy response to inference request in 4.1198413372039795s
Received healthy response to inference request in 2.7278645038604736s
Received healthy response to inference request in 3.165118455886841s
Received healthy response to inference request in 2.9225213527679443s
Received healthy response to inference request in 3.678835153579712s
5 requests
0 failed requests
5th percentile: 2.7667958736419678
10th percentile: 2.805727243423462
20th percentile: 2.88358998298645
30th percentile: 2.9710407733917235
40th percentile: 3.068079614639282
50th percentile: 3.165118455886841
60th percentile: 3.370605134963989
70th percentile: 3.5760918140411375
80th percentile: 3.7670363903045656
90th percentile: 3.9434388637542725
95th percentile: 4.031640100479126
99th percentile: 4.102201089859009
mean time: 3.32283616065979
Pipeline stage StressChecker completed in 55.64s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyTriggerPipeline
run_pipeline:run_in_cloud %s
starting trigger_guanaco_pipeline args=%s
triggered trigger_guanaco_pipeline args=%s
Pipeline stage OfflineFamilyFriendlyTriggerPipeline completed in 3.52s
Shutdown handler de-registered
function_tapif_2024-11-21 status is now deployed due to DeploymentManager action
Shutdown handler registered
run pipeline %s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyScorer
Evaluating %s Family Friendly Score with %s threads
Pipeline stage OfflineFamilyFriendlyScorer completed in 3702.98s
Shutdown handler de-registered
function_tapif_2024-11-21 status is now inactive due to auto deactivation removed underperforming models