function_famat_2024-11-18

submission_id: function_famat_2024-11-18

developer_uid: chai_backend_admin

celo_rating: 1249.68

display_name: retune_with_base

family_friendly_score: 0.6215999999999999

family_friendly_standard_error: 0.006858767236172984

formatter: {'memory_template': "{bot_name}'s Persona: {memory}\n####\n", 'prompt_template': '{prompt}\n<START>\n', 'bot_template': '{bot_name}: {message}\n', 'user_template': '{user_name}: {message}\n', 'response_template': '{bot_name}:', 'truncate_by_message': False}

generation_params: {'temperature': 0.9, 'top_p': 0.9, 'min_p': 0.05, 'top_k': 80, 'presence_penalty': 0.5, 'frequency_penalty': 0.5, 'stopping_words': ['\n', '</s>'], 'max_input_tokens': 1024, 'best_of': 8, 'max_output_tokens': 64}

is_internal_developer: True

model_group:

model_name: retune_with_base

num_battles: 14537

num_wins: 7048

ranking_group: single

status: inactive

submission_type: function

timestamp: 2024-11-18T18:37:38+00:00

us_pacific_date: 2024-11-18

win_ratio: 0.48483180848868407

Resubmit model

Shutdown handler not registered because Python interpreter is not running in the main thread
run pipeline %s
run pipeline stage %s
Running pipeline stage StressChecker
Received healthy response to inference request in 5.042393445968628s
Received healthy response to inference request in 6.361820220947266s
Received healthy response to inference request in 5.940228700637817s
Received healthy response to inference request in 5.1999192237854s
Received healthy response to inference request in 4.607640743255615s
5 requests
0 failed requests
5th percentile: 4.694591283798218
10th percentile: 4.78154182434082
20th percentile: 4.955442905426025
30th percentile: 5.073898601531982
40th percentile: 5.136908912658692
50th percentile: 5.1999192237854
60th percentile: 5.496043014526367
70th percentile: 5.792166805267334
80th percentile: 6.024547004699707
90th percentile: 6.1931836128234865
95th percentile: 6.277501916885376
99th percentile: 6.344956560134888
mean time: 5.430400466918945
%s, retrying in %s seconds...
Received healthy response to inference request in 4.391415119171143s
Received healthy response to inference request in 3.5093588829040527s
Received healthy response to inference request in 3.414618968963623s
Received healthy response to inference request in 3.411642551422119s
Received healthy response to inference request in 3.868427276611328s
5 requests
0 failed requests
5th percentile: 3.41223783493042
10th percentile: 3.4128331184387206
20th percentile: 3.414023685455322
30th percentile: 3.433566951751709
40th percentile: 3.471462917327881
50th percentile: 3.5093588829040527
60th percentile: 3.652986240386963
70th percentile: 3.796613597869873
80th percentile: 3.973024845123291
90th percentile: 4.182219982147217
95th percentile: 4.28681755065918
99th percentile: 4.37049560546875
mean time: 3.7190925598144533
%s, retrying in %s seconds...
Received healthy response to inference request in 3.9237146377563477s
Received healthy response to inference request in 3.44266414642334s
Received healthy response to inference request in 3.6761701107025146s
Received healthy response to inference request in 3.454418182373047s
Received healthy response to inference request in 3.494368314743042s
5 requests
0 failed requests
5th percentile: 3.4450149536132812
10th percentile: 3.4473657608032227
20th percentile: 3.4520673751831055
30th percentile: 3.462408208847046
40th percentile: 3.478388261795044
50th percentile: 3.494368314743042
60th percentile: 3.5670890331268312
70th percentile: 3.63980975151062
80th percentile: 3.7256790161132813
90th percentile: 3.8246968269348143
95th percentile: 3.874205732345581
99th percentile: 3.9138128566741943
mean time: 3.598267078399658
Pipeline stage StressChecker completed in 67.66s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyTriggerPipeline
run_pipeline:run_in_cloud %s
starting trigger_guanaco_pipeline args=%s
triggered trigger_guanaco_pipeline args=%s
Pipeline stage OfflineFamilyFriendlyTriggerPipeline completed in 2.30s
Shutdown handler de-registered
function_famat_2024-11-18 status is now deployed due to DeploymentManager action
Shutdown handler registered
run pipeline %s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyScorer
Evaluating %s Family Friendly Score with %s threads
Pipeline stage OfflineFamilyFriendlyScorer completed in 3435.84s
Shutdown handler de-registered
function_famat_2024-11-18 status is now inactive due to auto deactivation removed underperforming models