submission_id: function_hamem_2024-10-04
developer_uid: valentin
celo_rating: 1252.36
display_name: llama-405b-together
family_friendly_score: 0.6398193451260821
family_friendly_standard_error: 0.009299602880642074
formatter: {'memory_template': "{bot_name}'s Persona: {memory}\n####\n", 'prompt_template': '{prompt}\n<START>\n', 'bot_template': '{bot_name}: {message}\n', 'user_template': '{user_name}: {message}\n', 'response_template': '{bot_name}:', 'truncate_by_message': False}
generation_params: {'temperature': 1.0, 'top_p': 1.0, 'min_p': 0.0, 'top_k': 80, 'presence_penalty': 0.0, 'frequency_penalty': 0.0, 'stopping_words': ['<|eot_id|>'], 'max_input_tokens': 1024, 'best_of': 8, 'max_output_tokens': 64}
is_internal_developer: True
model_group:
model_name: llama-405b-together
num_battles: 2712
num_wins: 1362
ranking_group: single
status: torndown
submission_type: function
timestamp: 2024-10-04T16:11:10+00:00
us_pacific_date: 2024-10-04
win_ratio: 0.5022123893805309
Download Preference Data
Resubmit model
Shutdown handler not registered because Python interpreter is not running in the main thread
run pipeline %s
run pipeline stage %s
Running pipeline stage StressChecker
HTTPSConnectionPool(host='guanaco-submitter.chai-research.com', port=443): Read timed out. (read timeout=20)
Received unhealthy response to inference request!
Received healthy response to inference request in 5.299854278564453s
Received healthy response to inference request in 3.7192399501800537s
Received healthy response to inference request in 6.19193434715271s
Received healthy response to inference request in 8.798734903335571s
5 requests
1 failed requests
5th percentile: 4.035362815856933
10th percentile: 4.351485681533814
20th percentile: 4.983731412887574
30th percentile: 5.4782702922821045
40th percentile: 5.835102319717407
50th percentile: 6.19193434715271
60th percentile: 7.234654569625854
70th percentile: 8.277374792098998
80th percentile: 11.322857570648196
90th percentile: 16.37110290527344
95th percentile: 18.895225572586057
99th percentile: 20.914523706436157
mean time: 9.085822343826294
%s, retrying in %s seconds...
HTTPSConnectionPool(host='guanaco-submitter.chai-research.com', port=443): Read timed out. (read timeout=20)
Received unhealthy response to inference request!
Received healthy response to inference request in 5.555740833282471s
HTTPSConnectionPool(host='guanaco-submitter.chai-research.com', port=443): Read timed out. (read timeout=20)
Received unhealthy response to inference request!
Received healthy response to inference request in 3.8202505111694336s
Received healthy response to inference request in 3.3533222675323486s
5 requests
2 failed requests
5th percentile: 3.446707916259766
10th percentile: 3.5400935649871825
20th percentile: 3.7268648624420164
30th percentile: 4.167348575592041
40th percentile: 4.861544704437256
50th percentile: 5.555740833282471
60th percentile: 11.636946010589599
70th percentile: 17.718151187896726
80th percentile: 20.789814949035645
90th percentile: 20.851937294006348
95th percentile: 20.8829984664917
99th percentile: 20.90784740447998
mean time: 10.880425405502319
%s, retrying in %s seconds...
Received healthy response to inference request in 3.5201363563537598s
Received healthy response to inference request in 4.3139753341674805s
Received healthy response to inference request in 5.245691776275635s
HTTPSConnectionPool(host='guanaco-submitter.chai-research.com', port=443): Read timed out. (read timeout=20)
Received unhealthy response to inference request!
Received healthy response to inference request in 4.889268636703491s
5 requests
1 failed requests
5th percentile: 3.678904151916504
10th percentile: 3.8376719474792482
20th percentile: 4.155207538604737
30th percentile: 4.429033994674683
40th percentile: 4.659151315689087
50th percentile: 4.889268636703491
60th percentile: 5.031837892532349
70th percentile: 5.174407148361206
80th percentile: 8.210135126113894
90th percentile: 14.139021825790406
95th percentile: 17.10346517562866
99th percentile: 19.475019855499266
mean time: 7.607396125793457
clean up pipeline due to error=DeploymentChecksError('Unacceptable number of predict errors: 20.0%')
Shutdown handler de-registered
function_hamem_2024-10-04 status is now failed due to DeploymentManager action
Shutdown handler not registered because Python interpreter is not running in the main thread
run pipeline stage %s
admin requested tearing down of blend_fedub_2024-09-19
run pipeline stage %s
blend_batat_2024-09-20 status is now torndown due to DeploymentManager action
function_hamem_2024-10-04 status is now failed due to DeploymentManager action
function_hamem_2024-10-04 status is now deployed due to DeploymentManager action
Shutdown handler de-registered
run pipeline %s
admin requested tearing down of blend_pikis_2024-09-14
admin requested tearing down of blend_dimam_2024-09-20
Shutdown handler not registered because Python interpreter is not running in the main thread
Shutdown handler de-registered
Shutdown handler not registered because Python interpreter is not running in the main thread
run pipeline %s
admin requested tearing down of blend_pudib_2024-09-20
run pipeline %s
Shutdown handler not registered because Python interpreter is not running in the main thread
admin requested tearing down of blend_rofur_2024-10-03
blend_nifob_2024-09-19 status is now torndown due to DeploymentManager action
Shutdown handler de-registered
run pipeline %s
Shutdown handler de-registered
Shutdown handler not registered because Python interpreter is not running in the main thread
run pipeline stage %s
run pipeline %s
admin requested tearing down of blend_rofur_2024-10-03
run pipeline %s
Shutdown handler not registered because Python interpreter is not running in the main thread
Shutdown handler de-registered
blend_notum_2024-09-20 status is now torndown due to DeploymentManager action
blend_batat_2024-09-20 status is now torndown due to DeploymentManager action
run pipeline %s
Running pipeline stage ProductionBlendMKMLTemplater
Shutdown handler de-registered
Connection pool is full, discarding connection: %s. Connection pool size: %s
function_hamem_2024-10-04 status is now inactive due to auto deactivation removed underperforming models
function_hamem_2024-10-04 status is now torndown due to DeploymentManager action