submission_id: function_lilon_2024-11-12
developer_uid: chai_backend_admin
celo_rating: 1252.47
display_name: retune_with_base
family_friendly_score: 0.5842
family_friendly_standard_error: 0.006970084074098389
formatter: {'memory_template': "{bot_name}'s Persona: {memory}\n####\n", 'prompt_template': '{prompt}\n<START>\n', 'bot_template': '{bot_name}: {message}\n', 'user_template': '{user_name}: {message}\n', 'response_template': '{bot_name}:', 'truncate_by_message': False}
generation_params: {'temperature': 0.9, 'top_p': 0.9, 'min_p': 0.05, 'top_k': 80, 'presence_penalty': 0.5, 'frequency_penalty': 0.5, 'stopping_words': ['\n', '</s>'], 'max_input_tokens': 1024, 'best_of': 8, 'max_output_tokens': 64}
is_internal_developer: True
model_group:
model_name: retune_with_base
num_battles: 12860
num_wins: 6588
ranking_group: single
status: inactive
submission_type: function
timestamp: 2024-11-12T19:47:56+00:00
us_pacific_date: 2024-11-12
win_ratio: 0.5122861586314152
Resubmit model
Shutdown handler not registered because Python interpreter is not running in the main thread
run pipeline %s
run pipeline stage %s
Running pipeline stage StressChecker
Received healthy response to inference request in 3.3833837509155273s
Received healthy response to inference request in 2.29692006111145s
Failed to get response for submission bbchicago-nana-nemo-12b-v1-0_v8: ('http://bbchicago-nana-nemo-12b-v1-0-v8-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', '')
Received healthy response to inference request in 3.145259380340576s
Received healthy response to inference request in 2.1401147842407227s
Received healthy response to inference request in 7.7518227100372314s
5 requests
0 failed requests
5th percentile: 2.1714758396148683
10th percentile: 2.2028368949890136
20th percentile: 2.2655590057373045
30th percentile: 2.4665879249572753
40th percentile: 2.805923652648926
50th percentile: 3.145259380340576
60th percentile: 3.2405091285705567
70th percentile: 3.3357588768005373
80th percentile: 4.257071542739869
90th percentile: 6.00444712638855
95th percentile: 6.87813491821289
99th percentile: 7.577085151672363
mean time: 3.7435001373291015
Pipeline stage StressChecker completed in 20.03s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyTriggerPipeline
run_pipeline:run_in_cloud %s
starting trigger_guanaco_pipeline args=%s
triggered trigger_guanaco_pipeline args=%s
Pipeline stage OfflineFamilyFriendlyTriggerPipeline completed in 5.27s
Shutdown handler de-registered
function_lilon_2024-11-12 status is now deployed due to DeploymentManager action
Shutdown handler registered
run pipeline %s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyScorer
Evaluating %s Family Friendly Score with %s threads
Pipeline stage OfflineFamilyFriendlyScorer completed in 3268.41s
Shutdown handler de-registered
function_lilon_2024-11-12 status is now inactive due to auto deactivation removed underperforming models