developer_uid: chai_backend_admin
submission_id: function_kilaf_2024-11-13
model_name: function_kilaf_2024-11-13
model_group:
status: inactive
timestamp: 2024-11-13T18:18:06+00:00
num_battles: 11874
num_wins: 6088
celo_rating: 1253.8
family_friendly_score: 0.5804
family_friendly_standard_error: 0.006979052084631551
submission_type: function
display_name: function_kilaf_2024-11-13
is_internal_developer: True
ranking_group: single
us_pacific_date: 2024-11-13
win_ratio: 0.5127168603671888
generation_params: {'temperature': 1.0, 'top_p': 1.0, 'min_p': 0.0, 'top_k': 40, 'presence_penalty': 0.0, 'frequency_penalty': 0.0, 'stopping_words': ['\n'], 'max_input_tokens': 1024, 'best_of': 8, 'max_output_tokens': 64}
formatter: {'memory_template': "{bot_name}'s Persona: {memory}\n####\n", 'prompt_template': '{prompt}\n<START>\n', 'bot_template': '{bot_name}: {message}\n', 'user_template': '{user_name}: {message}\n', 'response_template': '{bot_name}:', 'truncate_by_message': False}
Resubmit model
Shutdown handler not registered because Python interpreter is not running in the main thread
run pipeline %s
run pipeline stage %s
Running pipeline stage StressChecker
Received healthy response to inference request in 3.4122819900512695s
Received healthy response to inference request in 3.7007622718811035s
Received healthy response to inference request in 3.14066481590271s
Received healthy response to inference request in 6.155905485153198s
Received healthy response to inference request in 3.8630619049072266s
5 requests
0 failed requests
5th percentile: 3.194988250732422
10th percentile: 3.249311685562134
20th percentile: 3.3579585552215576
30th percentile: 3.4699780464172365
40th percentile: 3.58537015914917
50th percentile: 3.7007622718811035
60th percentile: 3.7656821250915526
70th percentile: 3.8306019783020018
80th percentile: 4.321630620956421
90th percentile: 5.23876805305481
95th percentile: 5.6973367691040036
99th percentile: 6.064191741943359
mean time: 4.054535293579102
%s, retrying in %s seconds...
Received healthy response to inference request in 2.949423313140869s
Received healthy response to inference request in 1.7783851623535156s
Received healthy response to inference request in 1.7358593940734863s
Received healthy response to inference request in 4.113810300827026s
Received healthy response to inference request in 2.3047635555267334s
5 requests
0 failed requests
5th percentile: 1.7443645477294922
10th percentile: 1.7528697013854981
20th percentile: 1.7698800086975097
30th percentile: 1.8836608409881592
40th percentile: 2.094212198257446
50th percentile: 2.3047635555267334
60th percentile: 2.5626274585723876
70th percentile: 2.820491361618042
80th percentile: 3.182300710678101
90th percentile: 3.6480555057525637
95th percentile: 3.8809329032897946
99th percentile: 4.06723482131958
mean time: 2.576448345184326
Pipeline stage StressChecker completed in 36.04s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyTriggerPipeline
run_pipeline:run_in_cloud %s
starting trigger_guanaco_pipeline args=%s
triggered trigger_guanaco_pipeline args=%s
Pipeline stage OfflineFamilyFriendlyTriggerPipeline completed in 2.63s
Shutdown handler de-registered
function_kilaf_2024-11-13 status is now deployed due to DeploymentManager action
Shutdown handler registered
run pipeline %s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyScorer
Evaluating %s Family Friendly Score with %s threads
Pipeline stage OfflineFamilyFriendlyScorer completed in 3237.36s
Shutdown handler de-registered
function_kilaf_2024-11-13 status is now inactive due to auto deactivation removed underperforming models