submission_id: function_napot_2024-11-18
developer_uid: chai_backend_admin
celo_rating: 1266.39
display_name: retune_with_base
family_friendly_score: 0.6092
family_friendly_standard_error: 0.006900367526443791
formatter: {'memory_template': "{bot_name}'s Persona: {memory}\n####\n", 'prompt_template': '{prompt}\n<START>\n', 'bot_template': '{bot_name}: {message}\n', 'user_template': '{user_name}: {message}\n', 'response_template': '{bot_name}:', 'truncate_by_message': False}
generation_params: {'temperature': 0.9, 'top_p': 0.9, 'min_p': 0.05, 'top_k': 80, 'presence_penalty': 0.5, 'frequency_penalty': 0.5, 'stopping_words': ['\n', '</s>'], 'max_input_tokens': 1024, 'best_of': 8, 'max_output_tokens': 64}
is_internal_developer: True
model_group:
model_name: retune_with_base
num_battles: 14488
num_wins: 7490
ranking_group: single
status: inactive
submission_type: function
timestamp: 2024-11-18T23:21:19+00:00
us_pacific_date: 2024-11-18
win_ratio: 0.51697956929873
Resubmit model
Shutdown handler not registered because Python interpreter is not running in the main thread
run pipeline %s
run pipeline stage %s
Running pipeline stage StressChecker
HTTPConnectionPool(host='guanaco-submitter.guanaco-backend.k2.chaiverse.com', port=80): Read timed out. (read timeout=20)
Received unhealthy response to inference request!
Received healthy response to inference request in 2.951486110687256s
Received healthy response to inference request in 3.359590530395508s
Received healthy response to inference request in 4.066055536270142s
Received healthy response to inference request in 5.948643207550049s
5 requests
1 failed requests
5th percentile: 3.0331069946289064
10th percentile: 3.1147278785705566
20th percentile: 3.2779696464538572
30th percentile: 3.5008835315704347
40th percentile: 3.783469533920288
50th percentile: 4.066055536270142
60th percentile: 4.8190906047821045
70th percentile: 5.572125673294067
80th percentile: 8.781972360610965
90th percentile: 14.44863066673279
95th percentile: 17.2819598197937
99th percentile: 19.54862314224243
mean time: 7.288212871551513
%s, retrying in %s seconds...
Received healthy response to inference request in 4.486309289932251s
Received healthy response to inference request in 3.762068033218384s
Received healthy response to inference request in 6.677282810211182s
Received healthy response to inference request in 3.7374985218048096s
Received healthy response to inference request in 3.552886962890625s
5 requests
0 failed requests
5th percentile: 3.589809274673462
10th percentile: 3.626731586456299
20th percentile: 3.7005762100219726
30th percentile: 3.7424124240875245
40th percentile: 3.752240228652954
50th percentile: 3.762068033218384
60th percentile: 4.0517645359039305
70th percentile: 4.341461038589477
80th percentile: 4.924503993988037
90th percentile: 5.80089340209961
95th percentile: 6.239088106155395
99th percentile: 6.589643869400025
mean time: 4.44320912361145
%s, retrying in %s seconds...
Received healthy response to inference request in 2.644940137863159s
Received healthy response to inference request in 2.437857151031494s
Received healthy response to inference request in 3.6751556396484375s
Received healthy response to inference request in 3.1599764823913574s
Received healthy response to inference request in 3.746302843093872s
5 requests
0 failed requests
5th percentile: 2.4792737483978273
10th percentile: 2.52069034576416
20th percentile: 2.603523540496826
30th percentile: 2.747947406768799
40th percentile: 2.953961944580078
50th percentile: 3.1599764823913574
60th percentile: 3.3660481452941893
70th percentile: 3.5721198081970216
80th percentile: 3.6893850803375243
90th percentile: 3.7178439617156984
95th percentile: 3.7320734024047852
99th percentile: 3.743456954956055
mean time: 3.132846450805664
Pipeline stage StressChecker completed in 79.25s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyTriggerPipeline
run_pipeline:run_in_cloud %s
starting trigger_guanaco_pipeline args=%s
Connection pool is full, discarding connection: %s. Connection pool size: %s
triggered trigger_guanaco_pipeline args=%s
Pipeline stage OfflineFamilyFriendlyTriggerPipeline completed in 6.08s
Shutdown handler de-registered
function_napot_2024-11-18 status is now deployed due to DeploymentManager action
Shutdown handler registered
run pipeline %s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyScorer
Evaluating %s Family Friendly Score with %s threads
Pipeline stage OfflineFamilyFriendlyScorer completed in 3332.52s
Shutdown handler de-registered
function_napot_2024-11-18 status is now inactive due to auto deactivation removed underperforming models