function_kirum_2024-10-01

submission_id: function_kirum_2024-10-01

developer_uid: chai_backend_admin

celo_rating: 1253.22

display_name: retune_with_base

family_friendly_score: 0.5624880474278064

family_friendly_standard_error: 0.008379464842072307

formatter: {'memory_template': "{bot_name}'s Persona: {memory}\n####\n", 'prompt_template': '{prompt}\n<START>\n', 'bot_template': '{bot_name}: {message}\n', 'user_template': '{user_name}: {message}\n', 'response_template': '{bot_name}:', 'truncate_by_message': False}

generation_params: {'temperature': 0.9, 'top_p': 1.0, 'min_p': 0.05, 'top_k': 80, 'presence_penalty': 0.0, 'frequency_penalty': 0.0, 'stopping_words': ['\n', '</s>', '###', 'Bot:', 'User:', 'You:', '<|im_end|>'], 'max_input_tokens': 1024, 'best_of': 8, 'max_output_tokens': 64}

is_internal_developer: True

model_group:

model_name: retune_with_base

num_battles: 3592

num_wins: 1784

ranking_group: single

status: torndown

submission_type: function

timestamp: 2024-10-01T03:04:35+00:00

us_pacific_date: 2024-09-30

win_ratio: 0.49665924276169265

Download Preference Data

Resubmit model

Shutdown handler not registered because Python interpreter is not running in the main thread
run pipeline %s
run pipeline stage %s
Running pipeline stage StressChecker
Received healthy response to inference request in 5.685590982437134s
Received healthy response to inference request in 7.251749038696289s
Received healthy response to inference request in 8.077745199203491s
{"detail":"module 'guanaco_model_service.chat_api' has no attribute 'get_default_reward'"}
Received unhealthy response to inference request!
Received healthy response to inference request in 3.7819674015045166s
5 requests
1 failed requests
5th percentile: 2.0825024127960203
10th percentile: 2.5073686599731446
20th percentile: 3.3571011543273928
30th percentile: 4.16269211769104
40th percentile: 4.924141550064087
50th percentile: 5.685590982437134
60th percentile: 6.312054204940796
70th percentile: 6.938517427444458
80th percentile: 7.41694827079773
90th percentile: 7.74734673500061
95th percentile: 7.912545967102051
99th percentile: 8.044705352783204
mean time: 5.290937757492065
%s, retrying in %s seconds...
Shutdown handler not registered because Python interpreter is not running in the main thread
run pipeline %s
run pipeline stage %s
Running pipeline stage StressChecker
Received healthy response to inference request in 3.7732372283935547s
Received healthy response to inference request in 3.1145591735839844s
Received healthy response to inference request in 4.3076324462890625s
Received healthy response to inference request in 3.621340036392212s
Received healthy response to inference request in 3.794473171234131s
Received healthy response to inference request in 3.8285200595855713s
Received healthy response to inference request in 3.988645076751709s
Received healthy response to inference request in 3.80531644821167s
Received healthy response to inference request in 3.702613115310669s
5 requests
0 failed requests
5th percentile: 3.2462947845458983
10th percentile: 3.3780303955078126
20th percentile: 3.641501617431641
30th percentile: 3.77748441696167
40th percentile: 3.7859787940979004
50th percentile: 3.794473171234131
60th percentile: 3.7988104820251465
70th percentile: 3.803147792816162
80th percentile: 3.80995717048645
90th percentile: 3.819238615036011
95th percentile: 3.823879337310791
99th percentile: 3.8275919151306153
mean time: 3.6632212162017823
Pipeline stage StressChecker completed in 51.16s
Shutdown handler de-registered
function_kirum_2024-10-01 status is now deployed due to DeploymentManager action
function_kirum_2024-10-01 status is now inactive due to auto deactivation removed underperforming models
function_kirum_2024-10-01 status is now torndown due to DeploymentManager action