submission_id: function_pakus_2024-08-17
developer_uid: chai_backend_admin
alignment_samples: 41
display_name: gpt4-tl
formatter: {'memory_template': "{bot_name}'s Persona: {memory}\n####\n", 'prompt_template': '{prompt}\n<START>\n', 'bot_template': '{bot_name}: {message}\n', 'user_template': '{user_name}: {message}\n', 'response_template': '{bot_name}:', 'truncate_by_message': False}
generation_params: {'temperature': 1.0, 'top_p': 1.0, 'min_p': 0.1, 'top_k': 100, 'presence_penalty': 0.0, 'frequency_penalty': 0.0, 'stopping_words': ['\n', 'You:'], 'max_input_tokens': 512, 'best_of': 8, 'max_output_tokens': 64}
is_internal_developer: True
model_group:
model_name: gpt4-tl
num_battles: 41
num_wins: 17
propriety_score: 1.0
propriety_total_count: 3.0
ranking_group: single
status: torndown
submission_type: function
timestamp: 2024-08-17T05:41:48+00:00
us_pacific_date: 2024-08-16
win_ratio: 0.4146341463414634
Download Preference Data
Resubmit model
Running pipeline stage StressChecker
Received healthy response to inference request in 2.2361996173858643s
Failed to get response for submission undi95-meta-llama-3-70b_6209_v19: ('http://undi95-meta-llama-3-70b-6209-v19-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'request timeout')
Received healthy response to inference request in 3.153369426727295s
Received healthy response to inference request in 1.5259113311767578s
Received healthy response to inference request in 1.7110633850097656s
Received healthy response to inference request in 9.46511197090149s
5 requests
0 failed requests
5th percentile: 1.5629417419433593
10th percentile: 1.5999721527099608
20th percentile: 1.674032974243164
30th percentile: 1.8160906314849854
40th percentile: 2.0261451244354247
50th percentile: 2.2361996173858643
60th percentile: 2.6030675411224364
70th percentile: 2.9699354648590086
80th percentile: 4.415717935562135
90th percentile: 6.940414953231812
95th percentile: 8.202763462066649
99th percentile: 9.212642269134522
mean time: 3.6183311462402346
Pipeline stage StressChecker completed in 19.05s
function_pakus_2024-08-17 status is now deployed due to DeploymentManager action
function_pakus_2024-08-17 status is now inactive due to admin request
function_pakus_2024-08-17 status is now torndown due to DeploymentManager action

Usage Metrics

Latency Metrics