developer_uid: jxlu90
submission_id: function_furik_2024-12-13
model_name: nemo70B_600_avgctx1k_ranked
model_group:
status: inactive
timestamp: 2024-12-13T04:15:49+00:00
num_battles: 15594
num_wins: 7142
celo_rating: 1230.44
family_friendly_score: 0.6262
family_friendly_standard_error: 0.006842127739234339
submission_type: function
display_name: nemo70B_600_avgctx1k_ranked
is_internal_developer: True
ranking_group: single
us_pacific_date: 2024-12-12
win_ratio: 0.4579966653841221
generation_params: {'temperature': 1.0, 'top_p': 1.0, 'min_p': 0.0, 'top_k': 100, 'presence_penalty': 0.0, 'frequency_penalty': 0.0, 'stopping_words': ['\n', '</s>', 'You:'], 'max_input_tokens': 1024, 'best_of': 4, 'max_output_tokens': 68}
formatter: {'memory_template': "{bot_name}'s Persona: {memory}\n####\n", 'prompt_template': '{prompt}\n<START>\n', 'bot_template': '{bot_name}: {message}\n', 'user_template': '{user_name}: {message}\n', 'response_template': '{bot_name}:', 'truncate_by_message': False}
Resubmit model
Shutdown handler not registered because Python interpreter is not running in the main thread
run pipeline %s
run pipeline stage %s
Running pipeline stage StressChecker
HTTPConnectionPool(host='guanaco-submitter.guanaco-backend.k2.chaiverse.com', port=80): Read timed out. (read timeout=20)
Received unhealthy response to inference request!
Received healthy response to inference request in 2.3038699626922607s
Received healthy response to inference request in 2.835052251815796s
Received healthy response to inference request in 3.6752431392669678s
Received healthy response to inference request in 5.287317752838135s
5 requests
1 failed requests
5th percentile: 2.4101064205169678
10th percentile: 2.516342878341675
20th percentile: 2.728815793991089
30th percentile: 3.00309042930603
40th percentile: 3.339166784286499
50th percentile: 3.6752431392669678
60th percentile: 4.320072984695434
70th percentile: 4.964902830123901
80th percentile: 8.252216148376467
90th percentile: 14.182012939453127
95th percentile: 17.14691133499145
99th percentile: 19.518830051422118
mean time: 6.842658567428589
%s, retrying in %s seconds...
Received healthy response to inference request in 2.770421266555786s
Received healthy response to inference request in 2.735801935195923s
Received healthy response to inference request in 3.6842894554138184s
Received healthy response to inference request in 3.509971857070923s
Received healthy response to inference request in 3.0660574436187744s
5 requests
0 failed requests
5th percentile: 2.7427258014678957
10th percentile: 2.749649667739868
20th percentile: 2.7634974002838133
30th percentile: 2.8295485019683837
40th percentile: 2.9478029727935793
50th percentile: 3.0660574436187744
60th percentile: 3.243623208999634
70th percentile: 3.421188974380493
80th percentile: 3.5448353767395018
90th percentile: 3.61456241607666
95th percentile: 3.6494259357452394
99th percentile: 3.6773167514801024
mean time: 3.153308391571045
Pipeline stage StressChecker completed in 52.10s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyTriggerPipeline
run_pipeline:run_in_cloud %s
starting trigger_guanaco_pipeline args=%s
triggered trigger_guanaco_pipeline args=%s
Pipeline stage OfflineFamilyFriendlyTriggerPipeline completed in 2.31s
Shutdown handler de-registered
function_furik_2024-12-13 status is now deployed due to DeploymentManager action
Shutdown handler registered
run pipeline %s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyScorer
Evaluating %s Family Friendly Score with %s threads
Pipeline stage OfflineFamilyFriendlyScorer completed in 2771.93s
Shutdown handler de-registered
function_furik_2024-12-13 status is now inactive due to auto deactivation removed underperforming models