function_jutem_2024-12-17

developer_uid: rirv938

submission_id: function_jutem_2024-12-17

model_name: retune_with_base

model_group:

status: inactive

timestamp: 2024-12-17T17:17:42+00:00

num_battles: 10684

num_wins: 4891

celo_rating: 1222.44

family_friendly_score: 0.5953999999999999

family_friendly_standard_error: 0.006941164743758788

submission_type: function

display_name: retune_with_base

is_internal_developer: True

ranking_group: single

us_pacific_date: 2024-12-17

win_ratio: 0.4577873455634594

generation_params: {'temperature': 0.9, 'top_p': 0.9, 'min_p': 0.05, 'top_k': 80, 'presence_penalty': 0.5, 'frequency_penalty': 0.5, 'stopping_words': ['\n', '</s>'], 'max_input_tokens': 1024, 'best_of': 8, 'max_output_tokens': 64}

formatter: {'memory_template': "{bot_name}'s Persona: {memory}\n####\n", 'prompt_template': '{prompt}\n<START>\n', 'bot_template': '{bot_name}: {message}\n', 'user_template': '{user_name}: {message}\n', 'response_template': '{bot_name}:', 'truncate_by_message': False}

Resubmit model

Shutdown handler not registered because Python interpreter is not running in the main thread
run pipeline %s
run pipeline stage %s
Running pipeline stage StressChecker
Received healthy response to inference request in 2.1388967037200928s
Received healthy response to inference request in 2.345123767852783s
read tcp 127.0.0.1:44048->127.0.0.1:8080: read: connection reset by peer
Received unhealthy response to inference request!
Received healthy response to inference request in 2.215404987335205s
Received healthy response to inference request in 3.413469076156616s
5 requests
1 failed requests
5th percentile: 0.664073896408081
10th percentile: 1.032779598236084
20th percentile: 1.7701910018920899
30th percentile: 2.1541983604431154
40th percentile: 2.1848016738891602
50th percentile: 2.215404987335205
60th percentile: 2.2672924995422363
70th percentile: 2.3191800117492676
80th percentile: 2.55879282951355
90th percentile: 2.986130952835083
95th percentile: 3.1998000144958496
99th percentile: 3.370735263824463
mean time: 2.081652545928955
%s, retrying in %s seconds...
Received healthy response to inference request in 2.393042802810669s
Received healthy response to inference request in 2.775573492050171s
Received healthy response to inference request in 2.309277296066284s
Received healthy response to inference request in 3.0701634883880615s
Received healthy response to inference request in 2.539536714553833s
5 requests
0 failed requests
5th percentile: 2.326030397415161
10th percentile: 2.342783498764038
20th percentile: 2.376289701461792
30th percentile: 2.4223415851593018
40th percentile: 2.4809391498565674
50th percentile: 2.539536714553833
60th percentile: 2.633951425552368
70th percentile: 2.7283661365509033
80th percentile: 2.834491491317749
90th percentile: 2.9523274898529053
95th percentile: 3.0112454891204834
99th percentile: 3.058379888534546
mean time: 2.617518758773804
Pipeline stage StressChecker completed in 26.38s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyTriggerPipeline
run_pipeline:run_in_cloud %s
starting trigger_guanaco_pipeline args=%s
triggered trigger_guanaco_pipeline args=%s
Pipeline stage OfflineFamilyFriendlyTriggerPipeline completed in 2.13s
Shutdown handler de-registered
function_jutem_2024-12-17 status is now deployed due to DeploymentManager action
Shutdown handler registered
run pipeline %s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyScorer
Evaluating %s Family Friendly Score with %s threads
Pipeline stage OfflineFamilyFriendlyScorer completed in 2867.80s
Shutdown handler de-registered
function_jutem_2024-12-17 status is now inactive due to auto deactivation removed underperforming models