submission_id: function_susok_2024-09-19
developer_uid: chai_backend_admin
celo_rating: 1273.87
display_name: reward_blend_default_full_bon
family_friendly_score: 0.0
formatter: {'memory_template': "{bot_name}'s Persona: {memory}\n####\n", 'prompt_template': '{prompt}\n<START>\n', 'bot_template': '{bot_name}: {message}\n', 'user_template': '{user_name}: {message}\n', 'response_template': '{bot_name}:', 'truncate_by_message': False}
generation_params: {'temperature': 0.9, 'top_p': 1.0, 'min_p': 0.0, 'top_k': 50, 'presence_penalty': 0.5, 'frequency_penalty': 0.5, 'stopping_words': ['\n', '</s>', '<|user|>', '###'], 'max_input_tokens': 512, 'best_of': 4, 'max_output_tokens': 64}
is_internal_developer: True
model_group:
model_name: reward_blend_default_full_bon
num_battles: 12168
num_wins: 6450
ranking_group: single
status: torndown
submission_type: function
timestamp: 2024-09-19T19:39:01+00:00
us_pacific_date: 2024-09-19
win_ratio: 0.5300788954635108
Resubmit model
Shutdown handler not registered because Python interpreter is not running in the main thread
run pipeline %s
run pipeline stage %s
Running pipeline stage StressChecker
Received healthy response to inference request in 4.6361870765686035s
Received healthy response to inference request in 4.041322708129883s
Received healthy response to inference request in 5.6410839557647705s
Received healthy response to inference request in 4.068118572235107s
Received healthy response to inference request in 6.552275896072388s
5 requests
0 failed requests
5th percentile: 4.046681880950928
10th percentile: 4.052041053771973
20th percentile: 4.0627593994140625
30th percentile: 4.181732273101806
40th percentile: 4.408959674835205
50th percentile: 4.6361870765686035
60th percentile: 5.03814582824707
70th percentile: 5.440104579925537
80th percentile: 5.823322343826294
90th percentile: 6.187799119949341
95th percentile: 6.370037508010864
99th percentile: 6.515828218460083
mean time: 4.987797641754151
%s, retrying in %s seconds...
Failed to get response for submission zonemercy-lexical-nemov8_5966_v9: ('http://zonemercy-lexical-nemov8-5966-v9-predictor.tenant-chaiml-guanaco.k2.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'read tcp 127.0.0.1:58608->127.0.0.1:8080: read: connection reset by peer\n')
Received healthy response to inference request in 6.860304832458496s
Received healthy response to inference request in 5.502288579940796s
Received healthy response to inference request in 3.7212061882019043s
Received healthy response to inference request in 7.372311115264893s
Received healthy response to inference request in 3.738227128982544s
5 requests
0 failed requests
5th percentile: 3.724610376358032
10th percentile: 3.7280145645141602
20th percentile: 3.734822940826416
30th percentile: 4.091039419174194
40th percentile: 4.796663999557495
50th percentile: 5.502288579940796
60th percentile: 6.045495080947876
70th percentile: 6.588701581954956
80th percentile: 6.962706089019775
90th percentile: 7.167508602142334
95th percentile: 7.269909858703613
99th percentile: 7.351830863952637
mean time: 5.438867568969727
%s, retrying in %s seconds...
Received healthy response to inference request in 4.207545042037964s
Received healthy response to inference request in 4.718783378601074s
Received healthy response to inference request in 4.587752342224121s
Received healthy response to inference request in 4.851686000823975s
Received healthy response to inference request in 6.640544414520264s
5 requests
0 failed requests
5th percentile: 4.283586502075195
10th percentile: 4.359627962112427
20th percentile: 4.51171088218689
30th percentile: 4.613958549499512
40th percentile: 4.666370964050293
50th percentile: 4.718783378601074
60th percentile: 4.771944427490235
70th percentile: 4.825105476379394
80th percentile: 5.209457683563233
90th percentile: 5.925001049041748
95th percentile: 6.282772731781005
99th percentile: 6.568990077972412
mean time: 5.001262235641479
Pipeline stage StressChecker completed in 79.17s
Shutdown handler de-registered
function_susok_2024-09-19 status is now deployed due to DeploymentManager action
function_susok_2024-09-19 status is now inactive due to auto deactivation removed underperforming models
function_susok_2024-09-19 status is now torndown due to DeploymentManager action