submission_id: function_hehen_2024-09-14
developer_uid: chai_backend_admin
celo_rating: 1260.42
display_name: dpo_with_ava_reward_100k_v1
family_friendly_score: 0.0
formatter: {'memory_template': "{bot_name}'s Persona: {memory}\n####\n", 'prompt_template': '{prompt}\n<START>\n', 'bot_template': '{bot_name}: {message}\n', 'user_template': '{user_name}: {message}\n', 'response_template': '{bot_name}:', 'truncate_by_message': False}
generation_params: {'temperature': 0.95, 'top_p': 1.0, 'min_p': 0.08, 'top_k': 40, 'presence_penalty': 0.0, 'frequency_penalty': 0.0, 'stopping_words': ['\n', '<|eot_id|>'], 'max_input_tokens': 512, 'best_of': 16, 'max_output_tokens': 64}
is_internal_developer: True
model_group:
model_name: dpo_with_ava_reward_100k_v1
num_battles: 11119
num_wins: 5650
ranking_group: single
status: torndown
submission_type: function
timestamp: 2024-09-14T21:06:49+00:00
us_pacific_date: 2024-09-14
win_ratio: 0.5081392211529814
Resubmit model
Shutdown handler not registered because Python interpreter is not running in the main thread
run pipeline %s
run pipeline stage %s
Running pipeline stage StressChecker
HTTPSConnectionPool(host='guanaco-submitter.chai-research.com', port=443): Read timed out. (read timeout=20)
Received unhealthy response to inference request!
HTTPSConnectionPool(host='guanaco-submitter.chai-research.com', port=443): Read timed out. (read timeout=20)
Received unhealthy response to inference request!
Received healthy response to inference request in 6.337852239608765s
Received healthy response to inference request in 5.026349067687988s
Received healthy response to inference request in 5.280754804611206s
5 requests
2 failed requests
5th percentile: 5.077230215072632
10th percentile: 5.128111362457275
20th percentile: 5.2298736572265625
30th percentile: 5.492174291610718
40th percentile: 5.915013265609741
50th percentile: 6.337852239608765
60th percentile: 11.827074193954466
70th percentile: 17.316296148300168
80th percentile: 20.069629001617432
90th percentile: 20.08707275390625
95th percentile: 20.09579463005066
99th percentile: 20.102772130966187
mean time: 11.36207594871521
%s, retrying in %s seconds...
Received healthy response to inference request in 3.638936996459961s
Received healthy response to inference request in 3.1309735774993896s
Received healthy response to inference request in 5.978461265563965s
Received healthy response to inference request in 3.381563425064087s
Received healthy response to inference request in 3.255084276199341s
5 requests
0 failed requests
5th percentile: 3.15579571723938
10th percentile: 3.18061785697937
20th percentile: 3.2302621364593507
30th percentile: 3.28038010597229
40th percentile: 3.3309717655181883
50th percentile: 3.381563425064087
60th percentile: 3.4845128536224363
70th percentile: 3.587462282180786
80th percentile: 4.106841850280762
90th percentile: 5.042651557922364
95th percentile: 5.510556411743163
99th percentile: 5.884880294799805
mean time: 3.8770039081573486
Pipeline stage StressChecker completed in 77.64s
Shutdown handler de-registered
function_hehen_2024-09-14 status is now deployed due to DeploymentManager action
function_hehen_2024-09-14 status is now inactive due to auto deactivation removed underperforming models
function_hehen_2024-09-14 status is now torndown due to DeploymentManager action