function_rebeb_2024-12-27

developer_uid: jxlu90

submission_id: function_rebeb_2024-12-27

model_name: llama_405b_bo4

model_group:

status: torndown

timestamp: 2024-12-27T19:19:44+00:00

num_battles: 9

num_wins: 4

family_friendly_score: 0.0

family_friendly_standard_error: 0.0

submission_type: function

display_name: llama_405b_bo4

is_internal_developer: False

ranking_group: single

us_pacific_date: 2024-12-27

win_ratio: 0.4444444444444444

generation_params: {'temperature': 1.0, 'top_p': 1.0, 'min_p': 0.0, 'top_k': 100, 'presence_penalty': 0.0, 'frequency_penalty': 0.0, 'stopping_words': ['</s>', '\n', 'You:'], 'max_input_tokens': 1024, 'best_of': 4, 'max_output_tokens': 64}

formatter: {'memory_template': "{bot_name}'s Persona: {memory}\n####\n", 'prompt_template': '{prompt}\n<START>\n', 'bot_template': '{bot_name}: {message}\n', 'user_template': '{user_name}: {message}\n', 'response_template': '{bot_name}:', 'truncate_by_message': False}

Resubmit model

Shutdown handler not registered because Python interpreter is not running in the main thread
run pipeline %s
run pipeline stage %s
Running pipeline stage StressChecker
Received healthy response to inference request in 3.8451459407806396s
Received healthy response to inference request in 2.7825303077697754s
Received healthy response to inference request in 4.866390943527222s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Received healthy response to inference request in 5.669459104537964s
Received healthy response to inference request in 4.340605735778809s
5 requests
0 failed requests
5th percentile: 2.995053434371948
10th percentile: 3.207576560974121
20th percentile: 3.632622814178467
30th percentile: 3.9442378997802736
40th percentile: 4.142421817779541
50th percentile: 4.340605735778809
60th percentile: 4.550919818878174
70th percentile: 4.761233901977539
80th percentile: 5.027004575729371
90th percentile: 5.348231840133667
95th percentile: 5.508845472335815
99th percentile: 5.637336378097534
mean time: 4.3008264064788815
%s, retrying in %s seconds...
Received healthy response to inference request in 4.310561180114746s
Received healthy response to inference request in 2.99660062789917s
Received healthy response to inference request in 3.8380041122436523s
Received healthy response to inference request in 2.056173086166382s
Received healthy response to inference request in 5.088371753692627s
5 requests
0 failed requests
5th percentile: 2.2442585945129396
10th percentile: 2.432344102859497
20th percentile: 2.808515119552612
30th percentile: 3.1648813247680665
40th percentile: 3.5014427185058596
50th percentile: 3.8380041122436523
60th percentile: 4.02702693939209
70th percentile: 4.216049766540527
80th percentile: 4.466123294830322
90th percentile: 4.777247524261474
95th percentile: 4.932809638977051
99th percentile: 5.0572593307495115
mean time: 3.6579421520233155
%s, retrying in %s seconds...
Received healthy response to inference request in 2.1169190406799316s
Received healthy response to inference request in 3.9014549255371094s
Received healthy response to inference request in 2.394179105758667s
Received healthy response to inference request in 2.1464040279388428s
Received healthy response to inference request in 4.726902484893799s
5 requests
0 failed requests
5th percentile: 2.122816038131714
10th percentile: 2.128713035583496
20th percentile: 2.1405070304870604
30th percentile: 2.195959043502808
40th percentile: 2.2950690746307374
50th percentile: 2.394179105758667
60th percentile: 2.9970894336700438
70th percentile: 3.5999997615814205
80th percentile: 4.066544437408448
90th percentile: 4.396723461151123
95th percentile: 4.561812973022461
99th percentile: 4.693884582519531
mean time: 3.05717191696167
Pipeline stage StressChecker completed in 58.44s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyTriggerPipeline
run_pipeline:run_in_cloud %s
starting trigger_guanaco_pipeline args=%s
triggered trigger_guanaco_pipeline args=%s
Pipeline stage OfflineFamilyFriendlyTriggerPipeline completed in 0.61s
Shutdown handler de-registered
function_rebeb_2024-12-27 status is now deployed due to DeploymentManager action
Shutdown handler registered
run pipeline %s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyScorer
Evaluating %s Family Friendly Score with %s threads
Received signal 15, running shutdown handler
Shutdown handler de-registered
function_rebeb_2024-12-27 status is now torndown due to DeploymentManager action
function_rebeb_2024-12-27 status is now torndown due to DeploymentManager action
function_rebeb_2024-12-27 status is now torndown due to DeploymentManager action