developer_uid: chai_backend_admin
submission_id: function_meguf_2024-11-12
model_name: retune_with_base
model_group:
status: inactive
timestamp: 2024-11-12T13:22:28+00:00
num_battles: 10507
num_wins: 5335
celo_rating: 1258.16
family_friendly_score: 0.5738
family_friendly_standard_error: 0.006993619377689924
submission_type: function
display_name: retune_with_base
is_internal_developer: True
ranking_group: single
us_pacific_date: 2024-11-12
win_ratio: 0.5077567336061674
generation_params: {'temperature': 0.9, 'top_p': 0.9, 'min_p': 0.05, 'top_k': 80, 'presence_penalty': 0.5, 'frequency_penalty': 0.5, 'stopping_words': ['\n', '</s>'], 'max_input_tokens': 1024, 'best_of': 8, 'max_output_tokens': 64}
formatter: {'memory_template': "{bot_name}'s Persona: {memory}\n####\n", 'prompt_template': '{prompt}\n<START>\n', 'bot_template': '{bot_name}: {message}\n', 'user_template': '{user_name}: {message}\n', 'response_template': '{bot_name}:', 'truncate_by_message': False}
Resubmit model
Shutdown handler not registered because Python interpreter is not running in the main thread
run pipeline %s
run pipeline stage %s
Running pipeline stage StressChecker
Received healthy response to inference request in 2.794231414794922s
Received healthy response to inference request in 1.930924892425537s
Received healthy response to inference request in 2.443497657775879s
Received healthy response to inference request in 3.328244686126709s
Failed to get response for submission jic062-dpo-v3-0-nemo_v1: ('http://jic062-dpo-v3-0-nemo-v1-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', '')
Received healthy response to inference request in 3.726226806640625s
5 requests
0 failed requests
5th percentile: 2.0334394454956053
10th percentile: 2.135953998565674
20th percentile: 2.3409831047058107
30th percentile: 2.5136444091796877
40th percentile: 2.6539379119873048
50th percentile: 2.794231414794922
60th percentile: 3.0078367233276366
70th percentile: 3.2214420318603514
80th percentile: 3.407841110229492
90th percentile: 3.5670339584350588
95th percentile: 3.646630382537842
99th percentile: 3.7103075218200683
mean time: 2.8446250915527345
Pipeline stage StressChecker completed in 15.70s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyTriggerPipeline
run_pipeline:run_in_cloud %s
starting trigger_guanaco_pipeline args=%s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
triggered trigger_guanaco_pipeline args=%s
Pipeline stage OfflineFamilyFriendlyTriggerPipeline completed in 5.56s
Shutdown handler de-registered
function_meguf_2024-11-12 status is now deployed due to DeploymentManager action
Shutdown handler registered
run pipeline %s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyScorer
Evaluating %s Family Friendly Score with %s threads
Pipeline stage OfflineFamilyFriendlyScorer completed in 3603.59s
Shutdown handler de-registered
function_meguf_2024-11-12 status is now inactive due to auto deactivation removed underperforming models