function_purun_2024-09-25

submission_id: function_purun_2024-09-25

developer_uid: chai_backend_admin

celo_rating: 1255.3

display_name: retune_with_base

family_friendly_score: 0.0

formatter: {'memory_template': "{bot_name}'s Persona: {memory}\n####\n", 'prompt_template': '{prompt}\n<START>\n', 'bot_template': '{bot_name}: {message}\n', 'user_template': '{user_name}: {message}\n', 'response_template': '{bot_name}:', 'truncate_by_message': False}

generation_params: {'temperature': 0.9, 'top_p': 1.0, 'min_p': 0.0, 'top_k': 50, 'presence_penalty': 0.5, 'frequency_penalty': 0.5, 'stopping_words': ['\n', '</s>', '<|user|>', '###'], 'max_input_tokens': 512, 'best_of': 4, 'max_output_tokens': 64}

is_internal_developer: True

model_group:

model_name: retune_with_base

num_battles: 3325

num_wins: 1669

ranking_group: single

status: torndown

submission_type: function

timestamp: 2024-09-25T22:58:23+00:00

us_pacific_date: 2024-09-25

win_ratio: 0.5019548872180452

Resubmit model

Shutdown handler not registered because Python interpreter is not running in the main thread
run pipeline %s
run pipeline stage %s
Running pipeline stage StressChecker
HTTPSConnectionPool(host='guanaco-submitter.chai-research.com', port=443): Read timed out. (read timeout=20)
Received unhealthy response to inference request!
HTTPSConnectionPool(host='guanaco-submitter.chai-research.com', port=443): Read timed out. (read timeout=20)
Received unhealthy response to inference request!
HTTPSConnectionPool(host='guanaco-submitter.chai-research.com', port=443): Read timed out. (read timeout=20)
Received unhealthy response to inference request!
Received healthy response to inference request in 8.139005184173584s
Received healthy response to inference request in 6.462926864624023s
5 requests
3 failed requests
5th percentile: 6.798142528533935
10th percentile: 7.133358192443848
20th percentile: 7.803789520263672
30th percentile: 10.562208604812621
40th percentile: 15.4086154460907
50th percentile: 20.255022287368774
60th percentile: 20.266992616653443
70th percentile: 20.27896294593811
80th percentile: 20.29294557571411
90th percentile: 20.308940505981447
95th percentile: 20.316937971115113
99th percentile: 20.323335943222045
mean time: 15.093367576599121
%s, retrying in %s seconds...
Received healthy response to inference request in 9.516420841217041s
Received healthy response to inference request in 7.14113712310791s
Received healthy response to inference request in 7.366447925567627s
Received healthy response to inference request in 7.411723852157593s
Received healthy response to inference request in 9.826264142990112s
5 requests
0 failed requests
5th percentile: 7.186199283599853
10th percentile: 7.231261444091797
20th percentile: 7.321385765075684
30th percentile: 7.37550311088562
40th percentile: 7.393613481521607
50th percentile: 7.411723852157593
60th percentile: 8.253602647781372
70th percentile: 9.09548144340515
80th percentile: 9.578389501571655
90th percentile: 9.702326822280884
95th percentile: 9.764295482635498
99th percentile: 9.81387041091919
mean time: 8.252398777008057
%s, retrying in %s seconds...
Received healthy response to inference request in 7.587050437927246s
Received healthy response to inference request in 6.229907512664795s
Received healthy response to inference request in 4.2178497314453125s
Received healthy response to inference request in 3.3340377807617188s
Received healthy response to inference request in 3.417217493057251s
5 requests
0 failed requests
5th percentile: 3.350673723220825
10th percentile: 3.3673096656799317
20th percentile: 3.4005815505981447
30th percentile: 3.5773439407348633
40th percentile: 3.897596836090088
50th percentile: 4.2178497314453125
60th percentile: 5.022672843933106
70th percentile: 5.827495956420898
80th percentile: 6.501336097717285
90th percentile: 7.044193267822266
95th percentile: 7.315621852874756
99th percentile: 7.532764720916748
mean time: 4.957212591171265
Pipeline stage StressChecker completed in 147.11s
Shutdown handler de-registered
function_purun_2024-09-25 status is now deployed due to DeploymentManager action
function_purun_2024-09-25 status is now inactive due to auto deactivation removed underperforming models
Pipeline stage %s skipped, reason=%s
function_purun_2024-09-25 status is now torndown due to DeploymentManager action