function_maraf_2024-09-25

submission_id: function_maraf_2024-09-25

developer_uid: chai_backend_admin

celo_rating: 1252.3

display_name: mixtral_with_ava_reward_base

family_friendly_score: 0.5459363957597173

family_friendly_standard_error: 0.02094619519147341

formatter: {'memory_template': "{bot_name}'s Persona: {memory}\n####\n", 'prompt_template': '{prompt}\n<START>\n', 'bot_template': '{bot_name}: {message}\n', 'user_template': '{user_name}: {message}\n', 'response_template': '{bot_name}:', 'truncate_by_message': False}

generation_params: {'temperature': 0.9, 'top_p': 1.0, 'min_p': 0.0, 'top_k': 50, 'presence_penalty': 0.5, 'frequency_penalty': 0.5, 'stopping_words': ['\n', '</s>', '<|user|>', '###'], 'max_input_tokens': 512, 'best_of': 4, 'max_output_tokens': 64}

is_internal_developer: True

model_group:

model_name: mixtral_with_ava_reward_base

num_battles: 2929

num_wins: 1463

ranking_group: single

status: torndown

submission_type: function

timestamp: 2024-09-25T18:09:41+00:00

us_pacific_date: 2024-09-25

win_ratio: 0.499487879822465

Resubmit model

Shutdown handler not registered because Python interpreter is not running in the main thread
run pipeline %s
run pipeline stage %s
Running pipeline stage StressChecker
Received healthy response to inference request in 4.834696292877197s
Received healthy response to inference request in 5.447193145751953s
Received healthy response to inference request in 5.719545841217041s
Received healthy response to inference request in 4.9345831871032715s
Received healthy response to inference request in 2.4730660915374756s
5 requests
0 failed requests
5th percentile: 2.94539213180542
10th percentile: 3.4177181720733643
20th percentile: 4.362370252609253
30th percentile: 4.854673671722412
40th percentile: 4.894628429412842
50th percentile: 4.9345831871032715
60th percentile: 5.139627170562744
70th percentile: 5.344671154022217
80th percentile: 5.501663684844971
90th percentile: 5.610604763031006
95th percentile: 5.665075302124023
99th percentile: 5.708651733398438
mean time: 4.681816911697387
%s, retrying in %s seconds...
Received healthy response to inference request in 6.5925612449646s
Received healthy response to inference request in 6.178802728652954s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Received healthy response to inference request in 7.627270936965942s
Received healthy response to inference request in 9.07994294166565s
Received healthy response to inference request in 6.739941596984863s
5 requests
0 failed requests
5th percentile: 6.261554431915283
10th percentile: 6.344306135177613
20th percentile: 6.50980954170227
30th percentile: 6.622037315368653
40th percentile: 6.680989456176758
50th percentile: 6.739941596984863
60th percentile: 7.094873332977295
70th percentile: 7.449805068969726
80th percentile: 7.917805337905884
90th percentile: 8.498874139785766
95th percentile: 8.789408540725708
99th percentile: 9.02183606147766
mean time: 7.243703889846802
%s, retrying in %s seconds...
Received healthy response to inference request in 7.294291019439697s
Received healthy response to inference request in 7.9503843784332275s
Received healthy response to inference request in 7.139250755310059s
Failed to get response for submission rirv938-llama-8b-big-ret_4805_v2: ('http://rirv938-llama-8b-big-ret-4805-v2-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', '')
Received healthy response to inference request in 6.574814796447754s
Received healthy response to inference request in 6.940608501434326s
5 requests
0 failed requests
5th percentile: 6.647973537445068
10th percentile: 6.721132278442383
20th percentile: 6.867449760437012
30th percentile: 6.9803369522094725
40th percentile: 7.059793853759766
50th percentile: 7.139250755310059
60th percentile: 7.201266860961914
70th percentile: 7.26328296661377
80th percentile: 7.425509691238403
90th percentile: 7.687947034835815
95th percentile: 7.8191657066345215
99th percentile: 7.9241406440734865
mean time: 7.179869890213013
clean up pipeline due to error=%s
Shutdown handler de-registered
function_maraf_2024-09-25 status is now failed due to DeploymentManager action
function_maraf_2024-09-25 status is now torndown due to DeploymentManager action
function_maraf_2024-09-25 status is now inactive due to auto deactivation removed underperforming models
run pipeline %s
Pipeline stage ProductionBlendMKMLTemplater completed in 25.90s
function_maraf_2024-09-25 status is now torndown due to DeploymentManager action