developer_uid: chai_backend_admin
submission_id: function_gumob_2024-11-01
model_name: llm-judge
model_group:
status: torndown
timestamp: 2024-11-01T10:46:59+00:00
num_battles: 9968
num_wins: 4820
celo_rating: 1229.88
family_friendly_score: 0.5908
family_friendly_standard_error: 0.006953493510459329
submission_type: function
display_name: llm-judge
is_internal_developer: True
ranking_group: single
us_pacific_date: 2024-11-01
win_ratio: 0.4835473515248796
generation_params: {'temperature': 1.0, 'top_p': 1.0, 'min_p': 0.1, 'top_k': 100, 'presence_penalty': 0.0, 'frequency_penalty': 0.0, 'stopping_words': ['\n', 'You:'], 'max_input_tokens': 512, 'best_of': 8, 'max_output_tokens': 64}
formatter: {'memory_template': "{bot_name}'s Persona: {memory}\n####\n", 'prompt_template': '{prompt}\n<START>\n', 'bot_template': '{bot_name}: {message}\n', 'user_template': '{user_name}: {message}\n', 'response_template': '{bot_name}:', 'truncate_by_message': False}
Resubmit model
Shutdown handler not registered because Python interpreter is not running in the main thread
run pipeline %s
run pipeline stage %s
Running pipeline stage StressChecker
Received healthy response to inference request in 2.0969736576080322s
Received healthy response to inference request in 2.1201369762420654s
Received healthy response to inference request in 1.606156349182129s
Received healthy response to inference request in 1.2688438892364502s
Received healthy response to inference request in 1.8398373126983643s
5 requests
0 failed requests
5th percentile: 1.336306381225586
10th percentile: 1.4037688732147218
20th percentile: 1.5386938571929931
30th percentile: 1.652892541885376
40th percentile: 1.7463649272918702
50th percentile: 1.8398373126983643
60th percentile: 1.9426918506622315
70th percentile: 2.0455463886260987
80th percentile: 2.101606321334839
90th percentile: 2.1108716487884522
95th percentile: 2.1155043125152586
99th percentile: 2.119210443496704
mean time: 1.7863896369934082
Pipeline stage StressChecker completed in 10.46s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyTriggerPipeline
run_pipeline:run_in_cloud %s
starting trigger_guanaco_pipeline args=%s
triggered trigger_guanaco_pipeline args=%s
Pipeline stage OfflineFamilyFriendlyTriggerPipeline completed in 2.61s
Shutdown handler de-registered
function_gumob_2024-11-01 status is now deployed due to DeploymentManager action
Shutdown handler registered
run pipeline %s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyScorer
Evaluating %s Family Friendly Score with %s threads
Pipeline stage OfflineFamilyFriendlyScorer completed in 1828.61s
Shutdown handler de-registered
function_gumob_2024-11-01 status is now inactive due to auto deactivation removed underperforming models
function_gumob_2024-11-01 status is now torndown due to DeploymentManager action