function_nebak_2024-12-27

developer_uid: jxlu90

submission_id: function_nebak_2024-12-27

model_name: llama_405b_bo4

model_group:

status: torndown

timestamp: 2024-12-27T21:31:13+00:00

num_battles: 12

num_wins: 4

family_friendly_score: 0.0

family_friendly_standard_error: 0.0

submission_type: function

display_name: llama_405b_bo4

is_internal_developer: False

ranking_group: single

us_pacific_date: 2024-12-27

win_ratio: 0.3333333333333333

generation_params: {'temperature': 1.0, 'top_p': 1.0, 'min_p': 0.0, 'top_k': 100, 'presence_penalty': 0.0, 'frequency_penalty': 0.0, 'stopping_words': ['</s>', 'You:', '\n'], 'max_input_tokens': 1024, 'best_of': 4, 'max_output_tokens': 64}

formatter: {'memory_template': "{bot_name}'s Persona: {memory}\n####\n", 'prompt_template': '{prompt}\n<START>\n', 'bot_template': '{bot_name}: {message}\n', 'user_template': '{user_name}: {message}\n', 'response_template': '{bot_name}:', 'truncate_by_message': False}

Resubmit model

Shutdown handler not registered because Python interpreter is not running in the main thread
run pipeline %s
run pipeline stage %s
Running pipeline stage StressChecker
Received healthy response to inference request in 3.355139970779419s
Received healthy response to inference request in 3.038296699523926s
Received healthy response to inference request in 2.867356061935425s
Received healthy response to inference request in 5.726884126663208s
Received healthy response to inference request in 4.704832077026367s
5 requests
0 failed requests
5th percentile: 2.901544189453125
10th percentile: 2.9357323169708254
20th percentile: 3.0041085720062255
30th percentile: 3.1016653537750245
40th percentile: 3.2284026622772215
50th percentile: 3.355139970779419
60th percentile: 3.895016813278198
70th percentile: 4.434893655776977
80th percentile: 4.909242486953736
90th percentile: 5.3180633068084715
95th percentile: 5.522473716735839
99th percentile: 5.686002044677735
mean time: 3.938501787185669
%s, retrying in %s seconds...
Received healthy response to inference request in 2.6721246242523193s
Received healthy response to inference request in 3.7487218379974365s
Received healthy response to inference request in 2.952946662902832s
Received healthy response to inference request in 2.3219594955444336s
Received healthy response to inference request in 3.9969911575317383s
5 requests
0 failed requests
5th percentile: 2.391992521286011
10th percentile: 2.462025547027588
20th percentile: 2.602091598510742
30th percentile: 2.728289031982422
40th percentile: 2.840617847442627
50th percentile: 2.952946662902832
60th percentile: 3.2712567329406737
70th percentile: 3.5895668029785153
80th percentile: 3.798375701904297
90th percentile: 3.8976834297180174
95th percentile: 3.947337293624878
99th percentile: 3.9870603847503663
mean time: 3.138548755645752
Pipeline stage StressChecker completed in 37.55s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyTriggerPipeline
run_pipeline:run_in_cloud %s
starting trigger_guanaco_pipeline args=%s
triggered trigger_guanaco_pipeline args=%s
Pipeline stage OfflineFamilyFriendlyTriggerPipeline completed in 0.73s
Shutdown handler de-registered
function_nebak_2024-12-27 status is now deployed due to DeploymentManager action
Shutdown handler registered
run pipeline %s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyScorer
Evaluating %s Family Friendly Score with %s threads
Received signal 15, running shutdown handler
Shutdown handler de-registered
function_nebak_2024-12-27 status is now inactive due to admin request
function_nebak_2024-12-27 status is now torndown due to DeploymentManager action
function_nebak_2024-12-27 status is now torndown due to DeploymentManager action
function_nebak_2024-12-27 status is now torndown due to DeploymentManager action