developer_uid: chai_backend_admin
submission_id: function_kotek_2025-05-08
model_name: function_kotek_2025-05-08
model_group:
status: torndown
timestamp: 2025-05-08T10:30:36+00:00
num_battles: 7123
num_wins: 3636
celo_rating: 1293.73
family_friendly_score: 0.5347999999999999
family_friendly_standard_error: 0.007053920328441483
submission_type: function
display_name: function_kotek_2025-05-08
is_internal_developer: True
ranking_group: single
us_pacific_date: 2025-05-08
win_ratio: 0.5104590762319248
generation_params: {'temperature': 1.0, 'top_p': 1.0, 'min_p': 0.0, 'top_k': 40, 'presence_penalty': 0.0, 'frequency_penalty': 0.0, 'stopping_words': ['\n'], 'max_input_tokens': 1024, 'best_of': 8, 'max_output_tokens': 64}
formatter: {'memory_template': '### Instruction:\n{memory}\n', 'prompt_template': '### Input:\n{prompt}\n', 'bot_template': '{bot_name}: {message}\n', 'user_template': '{user_name}: {message}\n', 'response_template': '### Response:\n{bot_name}:', 'truncate_by_message': False}
Resubmit model
Shutdown handler not registered because Python interpreter is not running in the main thread
run pipeline %s
run pipeline stage %s
Running pipeline stage StressChecker
Received healthy response to inference request in 2.463261127471924s
HTTPConnectionPool(host='guanaco-submitter.guanaco-backend.k2.chaiverse.com', port=80): Read timed out. (read timeout=20)
Received unhealthy response to inference request!
Received healthy response to inference request in 4.030458211898804s
Received healthy response to inference request in 2.595388412475586s
Received healthy response to inference request in 2.435830593109131s
5 requests
1 failed requests
5th percentile: 2.4413166999816895
10th percentile: 2.4468028068542482
20th percentile: 2.457775020599365
30th percentile: 2.4896865844726563
40th percentile: 2.542537498474121
50th percentile: 2.595388412475586
60th percentile: 3.169416332244873
70th percentile: 3.74344425201416
80th percentile: 7.2481371879577665
90th percentile: 13.683495140075685
95th percentile: 16.90117411613464
99th percentile: 19.47531729698181
mean time: 6.328758287429809
%s, retrying in %s seconds...
Received healthy response to inference request in 2.161334753036499s
Received healthy response to inference request in 2.4588537216186523s
Received healthy response to inference request in 3.415992498397827s
Received healthy response to inference request in 2.6809115409851074s
Received healthy response to inference request in 3.071727991104126s
5 requests
0 failed requests
5th percentile: 2.2208385467529297
10th percentile: 2.2803423404693604
20th percentile: 2.3993499279022217
30th percentile: 2.5032652854919433
40th percentile: 2.5920884132385256
50th percentile: 2.6809115409851074
60th percentile: 2.8372381210327147
70th percentile: 2.9935647010803224
80th percentile: 3.140580892562866
90th percentile: 3.2782866954803467
95th percentile: 3.347139596939087
99th percentile: 3.402221918106079
mean time: 2.7577641010284424
Pipeline stage StressChecker completed in 47.65s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyTriggerPipeline
run_pipeline:run_in_cloud %s
starting trigger_guanaco_pipeline args=%s
triggered trigger_guanaco_pipeline args=%s
Pipeline stage OfflineFamilyFriendlyTriggerPipeline completed in 0.75s
Shutdown handler de-registered
function_kotek_2025-05-08 status is now deployed due to DeploymentManager action
Shutdown handler registered
run pipeline %s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyScorer
Evaluating %s Family Friendly Score with %s threads
Pipeline stage OfflineFamilyFriendlyScorer completed in 3408.65s
Shutdown handler de-registered
function_kotek_2025-05-08 status is now inactive due to auto deactivation removed underperforming models
function_kotek_2025-05-08 status is now torndown due to DeploymentManager action