function_tenor_2024-10-27

submission_id: function_tenor_2024-10-27

developer_uid: chai_backend_admin

celo_rating: 1248.05

display_name: retune_with_base

family_friendly_score: 0.5546

family_friendly_standard_error: 0.007028781402206217

formatter: {'memory_template': "{bot_name}'s Persona: {memory}\n####\n", 'prompt_template': '{prompt}\n<START>\n', 'bot_template': '{bot_name}: {message}\n', 'user_template': '{user_name}: {message}\n', 'response_template': '{bot_name}:', 'truncate_by_message': False}

generation_params: {'temperature': 0.9, 'top_p': 1.0, 'min_p': 0.05, 'top_k': 80, 'presence_penalty': 0.0, 'frequency_penalty': 0.0, 'stopping_words': ['\n', '</s>', '###', 'Bot:', 'User:', 'You:', '<|im_end|>'], 'max_input_tokens': 512, 'best_of': 8, 'max_output_tokens': 64}

is_internal_developer: True

model_group:

model_name: retune_with_base

num_battles: 18180

num_wins: 10610

ranking_group: single

status: inactive

submission_type: function

timestamp: 2024-10-27T03:30:38+00:00

us_pacific_date: 2024-10-26

win_ratio: 0.5836083608360836

Resubmit model

Shutdown handler not registered because Python interpreter is not running in the main thread
run pipeline %s
run pipeline stage %s
Running pipeline stage StressChecker
Received healthy response to inference request in 3.929205894470215s
Received healthy response to inference request in 4.023500680923462s
Received healthy response to inference request in 2.303680658340454s
Received healthy response to inference request in 3.595947504043579s
Received healthy response to inference request in 3.239128828048706s
5 requests
0 failed requests
5th percentile: 2.4907702922821047
10th percentile: 2.677859926223755
20th percentile: 3.0520391941070555
30th percentile: 3.3104925632476805
40th percentile: 3.45322003364563
50th percentile: 3.595947504043579
60th percentile: 3.7292508602142336
70th percentile: 3.8625542163848876
80th percentile: 3.948064851760864
90th percentile: 3.985782766342163
95th percentile: 4.004641723632813
99th percentile: 4.019728889465332
mean time: 3.4182927131652834
%s, retrying in %s seconds...
Received healthy response to inference request in 3.419588565826416s
Received healthy response to inference request in 5.535949230194092s
Received healthy response to inference request in 4.66251015663147s
Received healthy response to inference request in 3.7958998680114746s
Received healthy response to inference request in 3.5162465572357178s
5 requests
0 failed requests
5th percentile: 3.4389201641082763
10th percentile: 3.4582517623901365
20th percentile: 3.4969149589538575
30th percentile: 3.572177219390869
40th percentile: 3.684038543701172
50th percentile: 3.7958998680114746
60th percentile: 4.142543983459473
70th percentile: 4.48918809890747
80th percentile: 4.837197971343994
90th percentile: 5.186573600769043
95th percentile: 5.361261415481567
99th percentile: 5.5010116672515865
mean time: 4.186038875579834
%s, retrying in %s seconds...
Received healthy response to inference request in 4.596503496170044s
Received healthy response to inference request in 1.6942873001098633s
Received healthy response to inference request in 2.3955304622650146s
Received healthy response to inference request in 2.647782325744629s
Received healthy response to inference request in 3.2986273765563965s
5 requests
0 failed requests
5th percentile: 1.8345359325408936
10th percentile: 1.9747845649719238
20th percentile: 2.2552818298339843
30th percentile: 2.4459808349609373
40th percentile: 2.546881580352783
50th percentile: 2.647782325744629
60th percentile: 2.908120346069336
70th percentile: 3.168458366394043
80th percentile: 3.558202600479126
90th percentile: 4.077353048324585
95th percentile: 4.336928272247314
99th percentile: 4.544588451385498
mean time: 2.9265461921691895
Pipeline stage StressChecker completed in 56.81s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyTriggerPipeline
run_pipeline:run_in_cloud %s
starting trigger_guanaco_pipeline args=%s
triggered trigger_guanaco_pipeline args=%s
Pipeline stage OfflineFamilyFriendlyTriggerPipeline completed in 2.91s
Shutdown handler de-registered
function_tenor_2024-10-27 status is now deployed due to DeploymentManager action
Shutdown handler registered
run pipeline %s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyScorer
Evaluating %s Family Friendly Score with %s threads
Pipeline stage OfflineFamilyFriendlyScorer completed in 3115.77s
Shutdown handler de-registered
function_tenor_2024-10-27 status is now inactive due to auto deactivation removed underperforming models