function_pomof_2024-09-14

developer_uid: chai_backend_admin

submission_id: function_pomof_2024-09-14

model_name: dpo_baseline_reward

model_group:

status: torndown

timestamp: 2024-09-14T21:18:22+00:00

num_battles: 17299

num_wins: 8604

celo_rating: 1257.69

family_friendly_score: 0.0

submission_type: function

display_name: dpo_baseline_reward

is_internal_developer: True

ranking_group: single

us_pacific_date: 2024-09-14

win_ratio: 0.497369790161281

generation_params: {'temperature': 0.95, 'top_p': 1.0, 'min_p': 0.08, 'top_k': 40, 'presence_penalty': 0.0, 'frequency_penalty': 0.0, 'stopping_words': ['\n', '<|eot_id|>'], 'max_input_tokens': 512, 'best_of': 16, 'max_output_tokens': 64}

formatter: {'memory_template': "{bot_name}'s Persona: {memory}\n####\n", 'prompt_template': '{prompt}\n<START>\n', 'bot_template': '{bot_name}: {message}\n', 'user_template': '{user_name}: {message}\n', 'response_template': '{bot_name}:', 'truncate_by_message': False}

Resubmit model

Shutdown handler not registered because Python interpreter is not running in the main thread
run pipeline %s
run pipeline stage %s
Running pipeline stage StressChecker
Received healthy response to inference request in 6.0487060546875s
Received healthy response to inference request in 7.085926294326782s
Received healthy response to inference request in 3.4601821899414062s
Failed to get response for submission blend_hokok_2024-09-09: ('http://neversleep-noromaid-v0-8068-v150-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', '')
Received healthy response to inference request in 5.21300482749939s
Received healthy response to inference request in 4.590288162231445s
5 requests
0 failed requests
5th percentile: 3.686203384399414
10th percentile: 3.9122245788574217
20th percentile: 4.364266967773437
30th percentile: 4.7148314952850345
40th percentile: 4.963918161392212
50th percentile: 5.21300482749939
60th percentile: 5.547285318374634
70th percentile: 5.881565809249878
80th percentile: 6.256150102615357
90th percentile: 6.671038198471069
95th percentile: 6.878482246398925
99th percentile: 7.044437484741211
mean time: 5.279621505737305
%s, retrying in %s seconds...
Received healthy response to inference request in 3.396207332611084s
Failed to get response for submission blend_hokok_2024-09-09: ('http://neversleep-noromaid-v0-8068-v150-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', '')
Received healthy response to inference request in 3.29217529296875s
Received healthy response to inference request in 4.343332290649414s
Received healthy response to inference request in 4.286344528198242s
Received healthy response to inference request in 4.4755048751831055s
5 requests
0 failed requests
5th percentile: 3.312981700897217
10th percentile: 3.3337881088256838
20th percentile: 3.375400924682617
30th percentile: 3.5742347717285154
40th percentile: 3.930289649963379
50th percentile: 4.286344528198242
60th percentile: 4.309139633178711
70th percentile: 4.3319347381591795
80th percentile: 4.369766807556152
90th percentile: 4.422635841369629
95th percentile: 4.4490703582763675
99th percentile: 4.470217971801758
mean time: 3.958712863922119
Pipeline stage StressChecker completed in 48.40s
Shutdown handler de-registered
function_pomof_2024-09-14 status is now deployed due to DeploymentManager action
function_pomof_2024-09-14 status is now inactive due to auto deactivation removed underperforming models
function_pomof_2024-09-14 status is now torndown due to DeploymentManager action