developer_uid: chai_backend_admin
submission_id: function_sikif_2024-11-27
model_name: retune_with_base
model_group:
status: inactive
timestamp: 2024-11-27T18:22:13+00:00
num_battles: 16335
num_wins: 8162
celo_rating: 1253.99
family_friendly_score: 0.5569999999999999
family_friendly_standard_error: 0.007024969750824553
submission_type: function
display_name: retune_with_base
is_internal_developer: True
ranking_group: single
us_pacific_date: 2024-11-27
win_ratio: 0.49966329966329964
generation_params: {'temperature': 0.9, 'top_p': 0.9, 'min_p': 0.05, 'top_k': 80, 'presence_penalty': 0.5, 'frequency_penalty': 0.5, 'stopping_words': ['\n', '</s>'], 'max_input_tokens': 1024, 'best_of': 8, 'max_output_tokens': 64}
formatter: {'memory_template': "{bot_name}'s Persona: {memory}\n####\n", 'prompt_template': '{prompt}\n<START>\n', 'bot_template': '{bot_name}: {message}\n', 'user_template': '{user_name}: {message}\n', 'response_template': '{bot_name}:', 'truncate_by_message': False}
Resubmit model
Shutdown handler not registered because Python interpreter is not running in the main thread
run pipeline %s
run pipeline stage %s
Running pipeline stage StressChecker
Received healthy response to inference request in 2.4171595573425293s
Received healthy response to inference request in 2.5041606426239014s
{"detail":"('http://chaiml-llama-8b-multihea-7810-v3-predictor.tenant-chaiml-guanaco.k2.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'read tcp 127.0.0.1:54400->127.0.0.1:8080: read: connection reset by peer\\n')"}
Received unhealthy response to inference request!
Received healthy response to inference request in 2.4783551692962646s
Received healthy response to inference request in 1.9241774082183838s
5 requests
1 failed requests
5th percentile: 2.022773838043213
10th percentile: 2.121370267868042
20th percentile: 2.3185631275177
30th percentile: 2.4293986797332763
40th percentile: 2.4538769245147707
50th percentile: 2.4783551692962646
60th percentile: 2.4886773586273194
70th percentile: 2.498999547958374
80th percentile: 2.5826026439666747
90th percentile: 2.739486646652222
95th percentile: 2.817928647994995
99th percentile: 2.880682249069214
mean time: 2.4440446853637696
%s, retrying in %s seconds...
Received healthy response to inference request in 2.3737926483154297s
Received healthy response to inference request in 2.7931997776031494s
Received healthy response to inference request in 2.472583055496216s
Received healthy response to inference request in 2.4418811798095703s
Received healthy response to inference request in 2.9476258754730225s
5 requests
0 failed requests
5th percentile: 2.387410354614258
10th percentile: 2.401028060913086
20th percentile: 2.428263473510742
30th percentile: 2.4480215549468993
40th percentile: 2.460302305221558
50th percentile: 2.472583055496216
60th percentile: 2.600829744338989
70th percentile: 2.7290764331817625
80th percentile: 2.824084997177124
90th percentile: 2.8858554363250732
95th percentile: 2.916740655899048
99th percentile: 2.9414488315582275
mean time: 2.6058165073394775
Pipeline stage StressChecker completed in 28.10s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyTriggerPipeline
run_pipeline:run_in_cloud %s
starting trigger_guanaco_pipeline args=%s
triggered trigger_guanaco_pipeline args=%s
Pipeline stage OfflineFamilyFriendlyTriggerPipeline completed in 2.53s
Shutdown handler de-registered
function_sikif_2024-11-27 status is now deployed due to DeploymentManager action
Shutdown handler registered
run pipeline %s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyScorer
Evaluating %s Family Friendly Score with %s threads
Pipeline stage OfflineFamilyFriendlyScorer completed in 3728.69s
Shutdown handler de-registered
function_sikif_2024-11-27 status is now inactive due to auto deactivation removed underperforming models