submission_id: function_gahal_2024-09-14
developer_uid: chai_backend_admin
celo_rating: 1265.43
display_name: dpo_with_ava_reward_100k_v1
family_friendly_score: 0.0
formatter: {'memory_template': "{bot_name}'s Persona: {memory}\n####\n", 'prompt_template': '{prompt}\n<START>\n', 'bot_template': '{bot_name}: {message}\n', 'user_template': '{user_name}: {message}\n', 'response_template': '{bot_name}:', 'truncate_by_message': False}
generation_params: {'temperature': 0.95, 'top_p': 1.0, 'min_p': 0.08, 'top_k': 40, 'presence_penalty': 0.0, 'frequency_penalty': 0.0, 'stopping_words': ['\n', '<|eot_id|>'], 'max_input_tokens': 512, 'best_of': 16, 'max_output_tokens': 64}
is_internal_developer: True
model_group:
model_name: dpo_with_ava_reward_100k_v1
num_battles: 11237
num_wins: 5791
ranking_group: single
status: torndown
submission_type: function
timestamp: 2024-09-14T21:05:41+00:00
us_pacific_date: 2024-09-14
win_ratio: 0.5153510723502714
Resubmit model
Shutdown handler not registered because Python interpreter is not running in the main thread
run pipeline %s
run pipeline stage %s
Running pipeline stage StressChecker
Received healthy response to inference request in 19.00479245185852s
HTTPSConnectionPool(host='guanaco-submitter.chai-research.com', port=443): Read timed out. (read timeout=20)
Received unhealthy response to inference request!
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Failed to get response for submission blend_gamub_2024-09-09: ('http://zonemercy-lexical-nemo-1518-v18-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'read tcp 127.0.0.1:42582->127.0.0.1:8080: read: connection reset by peer\n')
HTTPSConnectionPool(host='guanaco-submitter.chai-research.com', port=443): Read timed out. (read timeout=20)
Received unhealthy response to inference request!
HTTPSConnectionPool(host='guanaco-submitter.chai-research.com', port=443): Read timed out. (read timeout=20)
Received unhealthy response to inference request!
Connection pool is full, discarding connection: %s. Connection pool size: %s
Failed to get response for submission blend_hokok_2024-09-09: ('http://neversleep-noromaid-v0-8068-v150-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', '')
HTTPSConnectionPool(host='guanaco-submitter.chai-research.com', port=443): Read timed out. (read timeout=20)
Received unhealthy response to inference request!
5 requests
4 failed requests
5th percentile: 19.216664409637453
10th percentile: 19.42853636741638
20th percentile: 19.85228028297424
30th percentile: 20.064383935928344
40th percentile: 20.064847326278688
50th percentile: 20.06531071662903
60th percentile: 20.084591817855834
70th percentile: 20.10387291908264
80th percentile: 20.122848796844483
90th percentile: 20.141519451141356
95th percentile: 20.150854778289794
99th percentile: 20.158323040008543
mean time: 19.881591796875
%s, retrying in %s seconds...
Received healthy response to inference request in 7.190906763076782s
Received healthy response to inference request in 6.603452444076538s
Received healthy response to inference request in 6.095597505569458s
Received healthy response to inference request in 3.9106252193450928s
Received healthy response to inference request in 5.269168138504028s
5 requests
0 failed requests
5th percentile: 4.1823338031768795
10th percentile: 4.454042387008667
20th percentile: 4.997459554672242
30th percentile: 5.434454011917114
40th percentile: 5.7650257587432865
50th percentile: 6.095597505569458
60th percentile: 6.29873948097229
70th percentile: 6.501881456375122
80th percentile: 6.7209433078765874
90th percentile: 6.955925035476684
95th percentile: 7.073415899276733
99th percentile: 7.167408590316772
mean time: 5.81395001411438
%s, retrying in %s seconds...
Received healthy response to inference request in 4.760691404342651s
Received healthy response to inference request in 5.048639297485352s
Received healthy response to inference request in 4.4886744022369385s
Received healthy response to inference request in 3.0940282344818115s
Received healthy response to inference request in 4.00799036026001s
5 requests
0 failed requests
5th percentile: 3.276820659637451
10th percentile: 3.459613084793091
20th percentile: 3.82519793510437
30th percentile: 4.1041271686553955
40th percentile: 4.296400785446167
50th percentile: 4.4886744022369385
60th percentile: 4.597481203079224
70th percentile: 4.706288003921509
80th percentile: 4.818280982971191
90th percentile: 4.933460140228272
95th percentile: 4.991049718856812
99th percentile: 5.037121381759643
mean time: 4.2800047397613525
Pipeline stage StressChecker completed in 153.82s
Shutdown handler de-registered
function_gahal_2024-09-14 status is now deployed due to DeploymentManager action
function_gahal_2024-09-14 status is now inactive due to auto deactivation removed underperforming models
function_gahal_2024-09-14 status is now torndown due to DeploymentManager action