developer_uid: chai_backend_admin
submission_id: function_nabem_2024-08-16
model_name: gpt4-tl
model_group:
status: torndown
timestamp: 2024-08-16T05:14:30+00:00
num_battles: 1250
num_wins: 642
celo_rating: 1239.23
family_friendly_score: 0.0
submission_type: function
display_name: gpt4-tl
is_internal_developer: True
ranking_group: single
us_pacific_date: 2024-08-15
win_ratio: 0.5136
generation_params: {'temperature': 1.0, 'top_p': 1.0, 'min_p': 0.1, 'top_k': 100, 'presence_penalty': 0.0, 'frequency_penalty': 0.0, 'stopping_words': ['\n', 'You:'], 'max_input_tokens': 512, 'best_of': 16, 'max_output_tokens': 64}
formatter: {'memory_template': "{bot_name}'s Persona: {memory}\n####\n", 'prompt_template': '{prompt}\n<START>\n', 'bot_template': '{bot_name}: {message}\n', 'user_template': '{user_name}: {message}\n', 'response_template': '{bot_name}:', 'truncate_by_message': False}
Resubmit model
Running pipeline stage StressChecker
HTTP Request: %s %s "%s %d %s"
Received healthy response to inference request in 5.30699896812439s
HTTP Request: %s %s "%s %d %s"
Received healthy response to inference request in 2.309319257736206s
HTTP Request: %s %s "%s %d %s"
Received healthy response to inference request in 3.0632200241088867s
HTTP Request: %s %s "%s %d %s"
Received healthy response to inference request in 2.4687280654907227s
('http://chaiml-llama-8b-pairwise-8189-v4-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'read tcp 127.0.0.1:52326->127.0.0.1:8080: read: connection reset by peer\n')
Received unhealthy response to inference request!
5 requests
1 failed requests
5th percentile: 2.3412010192871096
10th percentile: 2.3730827808380126
20th percentile: 2.436846303939819
30th percentile: 2.5876264572143555
40th percentile: 2.825423240661621
50th percentile: 3.0632200241088867
60th percentile: 3.516214656829834
70th percentile: 3.969209289550781
80th percentile: 4.417965078353882
90th percentile: 4.862482023239136
95th percentile: 5.084740495681762
99th percentile: 5.262547273635864
mean time: 3.468794584274292
%s, retrying in %s seconds...
HTTP Request: %s %s "%s %d %s"
Received healthy response to inference request in 3.4264135360717773s
HTTP Request: %s %s "%s %d %s"
Received healthy response to inference request in 3.8803346157073975s
HTTP Request: %s %s "%s %d %s"
Received healthy response to inference request in 3.2756197452545166s
HTTP Request: %s %s "%s %d %s"
Received healthy response to inference request in 2.6960511207580566s
HTTP Request: %s %s "%s %d %s"
Received healthy response to inference request in 2.8564748764038086s
5 requests
0 failed requests
5th percentile: 2.728135871887207
10th percentile: 2.7602206230163575
20th percentile: 2.8243901252746584
30th percentile: 2.94030385017395
40th percentile: 3.1079617977142333
50th percentile: 3.2756197452545166
60th percentile: 3.335937261581421
70th percentile: 3.396254777908325
80th percentile: 3.5171977519989013
90th percentile: 3.6987661838531496
95th percentile: 3.7895503997802735
99th percentile: 3.8621777725219726
mean time: 3.2269787788391113
Pipeline stage StressChecker completed in 39.71s
function_nabem_2024-08-16 status is now deployed due to DeploymentManager action
function_nabem_2024-08-16 status is now inactive due to admin request
function_nabem_2024-08-16 status is now torndown due to DeploymentManager action