function_kehol_2025-04-14

developer_uid: rirv938

submission_id: function_kehol_2025-04-14

model_name: dpo_data_collection

model_group:

status: torndown

timestamp: 2025-04-14T19:06:35+00:00

num_battles: 1501619

num_wins: 802863

celo_rating: 1311.05

family_friendly_score: 0.5604

family_friendly_standard_error: 0.007019285433717595

submission_type: function

display_name: dpo_data_collection

is_internal_developer: True

ranking_group: single

us_pacific_date: 2025-04-14

win_ratio: 0.5346649183314809

generation_params: {'temperature': 0.9, 'top_p': 0.9, 'min_p': 0.05, 'top_k': 80, 'presence_penalty': 0.5, 'frequency_penalty': 0.5, 'stopping_words': ['\n', '</s>'], 'max_input_tokens': 1024, 'best_of': 8, 'max_output_tokens': 64}

formatter: {'memory_template': "{bot_name}'s Persona: {memory}\n####\n", 'prompt_template': '{prompt}\n<START>\n', 'bot_template': '{bot_name}: {message}\n', 'user_template': '{user_name}: {message}\n', 'response_template': '{bot_name}:', 'truncate_by_message': False}

Resubmit model

Shutdown handler not registered because Python interpreter is not running in the main thread
run pipeline %s
run pipeline stage %s
Running pipeline stage StressChecker
HTTPConnectionPool(host='guanaco-submitter.guanaco-backend.k2.chaiverse.com', port=80): Read timed out. (read timeout=20)
Received unhealthy response to inference request!
Received healthy response to inference request in 3.5743887424468994s
Received healthy response to inference request in 2.486790418624878s
Received healthy response to inference request in 3.270785331726074s
Received healthy response to inference request in 3.1736319065093994s
5 requests
1 failed requests
5th percentile: 2.6241587162017823
10th percentile: 2.7615270137786867
20th percentile: 3.036263608932495
30th percentile: 3.193062591552734
40th percentile: 3.231923961639404
50th percentile: 3.270785331726074
60th percentile: 3.3922266960144043
70th percentile: 3.5136680603027344
80th percentile: 6.880025291442874
90th percentile: 13.491298389434816
95th percentile: 16.796934938430784
99th percentile: 19.441444177627563
mean time: 6.521633577346802
%s, retrying in %s seconds...
Received healthy response to inference request in 4.079611301422119s
Received healthy response to inference request in 3.5416975021362305s
Received healthy response to inference request in 3.758496046066284s
Received healthy response to inference request in 3.188037157058716s
Received healthy response to inference request in 3.247265577316284s
5 requests
0 failed requests
5th percentile: 3.1998828411102296
10th percentile: 3.2117285251617433
20th percentile: 3.2354198932647704
30th percentile: 3.3061519622802735
40th percentile: 3.4239247322082518
50th percentile: 3.5416975021362305
60th percentile: 3.628416919708252
70th percentile: 3.7151363372802733
80th percentile: 3.822719097137451
90th percentile: 3.951165199279785
95th percentile: 4.015388250350952
99th percentile: 4.066766691207886
mean time: 3.563021516799927
%s, retrying in %s seconds...
Received healthy response to inference request in 4.610999345779419s
Received healthy response to inference request in 2.6083076000213623s
Received healthy response to inference request in 2.7933509349823s
Received healthy response to inference request in 1.6601877212524414s
Received healthy response to inference request in 2.263349771499634s
5 requests
0 failed requests
5th percentile: 1.7808201313018799
10th percentile: 1.9014525413513184
20th percentile: 2.1427173614501953
30th percentile: 2.3323413372039794
40th percentile: 2.470324468612671
50th percentile: 2.6083076000213623
60th percentile: 2.682324934005737
70th percentile: 2.7563422679901124
80th percentile: 3.156880617141724
90th percentile: 3.8839399814605713
95th percentile: 4.247469663619995
99th percentile: 4.538293409347534
mean time: 2.7872390747070312
Pipeline stage StressChecker completed in 67.51s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyTriggerPipeline
run_pipeline:run_in_cloud %s
starting trigger_guanaco_pipeline args=%s
triggered trigger_guanaco_pipeline args=%s
Pipeline stage OfflineFamilyFriendlyTriggerPipeline completed in 0.81s
Shutdown handler de-registered
function_kehol_2025-04-14 status is now deployed due to DeploymentManager action
Shutdown handler registered
run pipeline %s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyScorer
Evaluating %s Family Friendly Score with %s threads
Pipeline stage OfflineFamilyFriendlyScorer completed in 3413.84s
Shutdown handler de-registered
function_kehol_2025-04-14 status is now torndown due to DeploymentManager action