function_bejil_2024-11-14

submission_id: function_bejil_2024-11-14

developer_uid: chai_backend_admin

celo_rating: 1252.95

display_name: retune_with_base

family_friendly_score: 0.5993999999999999

family_friendly_standard_error: 0.006929929869775018

formatter: {'memory_template': "{bot_name}'s Persona: {memory}\n####\n", 'prompt_template': '{prompt}\n<START>\n', 'bot_template': '{bot_name}: {message}\n', 'user_template': '{user_name}: {message}\n', 'response_template': '{bot_name}:', 'truncate_by_message': False}

generation_params: {'temperature': 0.9, 'top_p': 0.9, 'min_p': 0.05, 'top_k': 80, 'presence_penalty': 0.5, 'frequency_penalty': 0.5, 'stopping_words': ['\n', '</s>'], 'max_input_tokens': 1024, 'best_of': 8, 'max_output_tokens': 64}

is_internal_developer: True

model_group:

model_name: retune_with_base

num_battles: 12792

num_wins: 6287

ranking_group: single

status: inactive

submission_type: function

timestamp: 2024-11-14T21:44:59+00:00

us_pacific_date: 2024-11-14

win_ratio: 0.49147904940587867

Resubmit model

Shutdown handler not registered because Python interpreter is not running in the main thread
run pipeline %s
run pipeline stage %s
Running pipeline stage StressChecker
Received healthy response to inference request in 3.8073339462280273s
Failed to get response for submission bbchicago-nana-nemo-12b-v1-0_v8: ('http://bbchicago-nana-nemo-12b-v1-0-v8-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', '')
Received healthy response to inference request in 3.542929172515869s
Received healthy response to inference request in 3.276420831680298s
Received healthy response to inference request in 3.6117382049560547s
Received healthy response to inference request in 4.358875513076782s
5 requests
0 failed requests
5th percentile: 3.329722499847412
10th percentile: 3.3830241680145265
20th percentile: 3.489627504348755
30th percentile: 3.556690979003906
40th percentile: 3.5842145919799804
50th percentile: 3.6117382049560547
60th percentile: 3.6899765014648436
70th percentile: 3.768214797973633
80th percentile: 3.9176422595977782
90th percentile: 4.1382588863372805
95th percentile: 4.248567199707031
99th percentile: 4.336813850402832
mean time: 3.7194595336914062
%s, retrying in %s seconds...
Received healthy response to inference request in 3.538930892944336s
Received healthy response to inference request in 4.369417905807495s
Received healthy response to inference request in 3.0824663639068604s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Received healthy response to inference request in 3.483809471130371s
Received healthy response to inference request in 3.684868097305298s
5 requests
0 failed requests
5th percentile: 3.1627349853515625
10th percentile: 3.2430036067962646
20th percentile: 3.403540849685669
30th percentile: 3.494833755493164
40th percentile: 3.51688232421875
50th percentile: 3.538930892944336
60th percentile: 3.5973057746887207
70th percentile: 3.6556806564331055
80th percentile: 3.8217780590057373
90th percentile: 4.095597982406616
95th percentile: 4.232507944107056
99th percentile: 4.342035913467408
mean time: 3.631898546218872
Pipeline stage StressChecker completed in 39.69s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyTriggerPipeline
run_pipeline:run_in_cloud %s
starting trigger_guanaco_pipeline args=%s
triggered trigger_guanaco_pipeline args=%s
Pipeline stage OfflineFamilyFriendlyTriggerPipeline completed in 2.66s
Shutdown handler de-registered
function_bejil_2024-11-14 status is now deployed due to DeploymentManager action
Shutdown handler registered
run pipeline %s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyScorer
Evaluating %s Family Friendly Score with %s threads
Pipeline stage OfflineFamilyFriendlyScorer completed in 3343.02s
Shutdown handler de-registered
function_bejil_2024-11-14 status is now inactive due to auto deactivation removed underperforming models