developer_uid: chai_backend_admin
submission_id: function_kubes_2024-10-13
model_name: retune_with_base
model_group:
status: torndown
timestamp: 2024-10-13T18:24:42+00:00
num_battles: 8453
num_wins: 4681
celo_rating: 1270.53
family_friendly_score: 0.6139772915630698
family_friendly_standard_error: 0.005407460949522884
submission_type: function
display_name: retune_with_base
is_internal_developer: True
ranking_group: single
us_pacific_date: 2024-10-13
win_ratio: 0.5537678930557198
generation_params: {'temperature': 0.75, 'top_p': 1.0, 'min_p': 0.1, 'top_k': 40, 'presence_penalty': 0.0, 'frequency_penalty': 0.0, 'stopping_words': ['\n', '\n\n', '\nYou:', '[/INST]', '<|im_end|>', '</s>'], 'max_input_tokens': 1024, 'best_of': 8, 'max_output_tokens': 64}
formatter: {'memory_template': "{bot_name}'s Persona: {memory}\n####\n", 'prompt_template': '{prompt}\n<START>\n', 'bot_template': '{bot_name}: {message}\n', 'user_template': '{user_name}: {message}\n', 'response_template': '{bot_name}:', 'truncate_by_message': False}
Resubmit model
Shutdown handler not registered because Python interpreter is not running in the main thread
run pipeline %s
run pipeline stage %s
Running pipeline stage StressChecker
Received healthy response to inference request in 2.0638365745544434s
Received healthy response to inference request in 1.4059505462646484s
Received healthy response to inference request in 2.3859541416168213s
Received healthy response to inference request in 1.601248025894165s
Received healthy response to inference request in 2.248983383178711s
5 requests
0 failed requests
5th percentile: 1.4450100421905518
10th percentile: 1.484069538116455
20th percentile: 1.5621885299682616
30th percentile: 1.6937657356262208
40th percentile: 1.878801155090332
50th percentile: 2.0638365745544434
60th percentile: 2.1378952980041506
70th percentile: 2.2119540214538573
80th percentile: 2.276377534866333
90th percentile: 2.331165838241577
95th percentile: 2.358559989929199
99th percentile: 2.3804753112792967
mean time: 1.9411945343017578
Pipeline stage StressChecker completed in 12.45s
Shutdown handler de-registered
function_kubes_2024-10-13 status is now deployed due to DeploymentManager action
function_kubes_2024-10-13 status is now inactive due to auto deactivation removed underperforming models
function_kubes_2024-10-13 status is now torndown due to DeploymentManager action