developer_uid: chai_backend_admin
submission_id: function_degul_2024-09-14
model_name: dpo_with_ava_reward_100k_v1
model_group:
status: torndown
timestamp: 2024-09-14T21:08:30+00:00
num_battles: 17568
num_wins: 8834
celo_rating: 1260.96
family_friendly_score: 0.0
submission_type: function
display_name: dpo_with_ava_reward_100k_v1
is_internal_developer: True
ranking_group: single
us_pacific_date: 2024-09-14
win_ratio: 0.5028460837887068
generation_params: {'temperature': 0.95, 'top_p': 1.0, 'min_p': 0.08, 'top_k': 40, 'presence_penalty': 0.0, 'frequency_penalty': 0.0, 'stopping_words': ['\n', '<|eot_id|>'], 'max_input_tokens': 512, 'best_of': 16, 'max_output_tokens': 64}
formatter: {'memory_template': "{bot_name}'s Persona: {memory}\n####\n", 'prompt_template': '{prompt}\n<START>\n', 'bot_template': '{bot_name}: {message}\n', 'user_template': '{user_name}: {message}\n', 'response_template': '{bot_name}:', 'truncate_by_message': False}
Resubmit model
Shutdown handler not registered because Python interpreter is not running in the main thread
run pipeline %s
run pipeline stage %s
Running pipeline stage StressChecker
Received healthy response to inference request in 4.481548547744751s
Received healthy response to inference request in 4.298039674758911s
Received healthy response to inference request in 4.189704895019531s
Failed to get response for submission blend_hokok_2024-09-09: ('http://neversleep-noromaid-v0-8068-v150-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', '')
Received healthy response to inference request in 4.650141000747681s
Received healthy response to inference request in 4.927019119262695s
5 requests
0 failed requests
5th percentile: 4.211371850967407
10th percentile: 4.233038806915284
20th percentile: 4.276372718811035
30th percentile: 4.334741449356079
40th percentile: 4.408144998550415
50th percentile: 4.481548547744751
60th percentile: 4.548985528945923
70th percentile: 4.616422510147094
80th percentile: 4.705516624450683
90th percentile: 4.81626787185669
95th percentile: 4.871643495559693
99th percentile: 4.915943994522094
mean time: 4.509290647506714
Pipeline stage StressChecker completed in 23.06s
Shutdown handler de-registered
function_degul_2024-09-14 status is now deployed due to DeploymentManager action
function_degul_2024-09-14 status is now inactive due to auto deactivation removed underperforming models
function_degul_2024-09-14 status is now torndown due to DeploymentManager action