submission_id: function_soref_2024-08-09
developer_uid: chai_backend_admin
alignment_samples: 1565
alignment_score: 3.7198951899925268
celo_rating: 1094.96
display_name: gpt-3-5
formatter: {'memory_template': "{bot_name}'s Persona: {memory}\n####\n", 'prompt_template': '{prompt}\n<START>\n', 'bot_template': '{bot_name}: {message}\n', 'user_template': '{user_name}: {message}\n', 'response_template': '{bot_name}:', 'truncate_by_message': False}
generation_params: {'temperature': 1.0, 'top_p': 1.0, 'min_p': 0.0, 'top_k': 40, 'presence_penalty': 0.0, 'frequency_penalty': 0.0, 'stopping_words': ['\n'], 'max_input_tokens': 512, 'best_of': 1, 'max_output_tokens': 64, 'reward_max_token_input': 256}
is_internal_developer: True
model_group:
model_name: gpt-3-5
num_battles: 14756
num_wins: 5509
propriety_score: 0.81226626776365
propriety_total_count: 1337.0
ranking_group: single
reward_repo: ChaiML/gpt2_xl_pairwise_89m_step_347634
status: torndown
submission_type: function
timestamp: 2024-08-09T18:27:50+00:00
us_pacific_date: 2024-08-09
win_ratio: 0.3733396584440228
Download Preference Data
Resubmit model
Running pipeline stage StressChecker
Received healthy response to inference request in 0.7293956279754639s
Received healthy response to inference request in 0.5735042095184326s
Received healthy response to inference request in 0.733830451965332s
Received healthy response to inference request in 0.3967742919921875s
Received healthy response to inference request in 0.6259980201721191s
5 requests
0 failed requests
5th percentile: 0.43212027549743653
10th percentile: 0.46746625900268557
20th percentile: 0.5381582260131836
30th percentile: 0.5840029716491699
40th percentile: 0.6050004959106445
50th percentile: 0.6259980201721191
60th percentile: 0.667357063293457
70th percentile: 0.7087161064147949
80th percentile: 0.7302825927734375
90th percentile: 0.7320565223693848
95th percentile: 0.7329434871673584
99th percentile: 0.7336530590057373
mean time: 0.611900520324707
Pipeline stage StressChecker completed in 3.70s
function_soref_2024-08-09 status is now deployed due to DeploymentManager action
function_soref_2024-08-09 status is now deployed due to admin request
function_soref_2024-08-09 status is now inactive due to auto deactivation removed underperforming models
function_soref_2024-08-09 status is now torndown due to DeploymentManager action
function_soref_2024-08-09 status is now torndown due to DeploymentManager action

Usage Metrics

Latency Metrics