submission_id: function_todun_2024-09-26
developer_uid: chai_backend_admin
celo_rating: 1260.11
display_name: retune_with_base
family_friendly_score: 0.5787476280834914
family_friendly_standard_error: 0.00961323642245384
formatter: {'memory_template': "{bot_name}'s Persona: {memory}\n####\n", 'prompt_template': '{prompt}\n<START>\n', 'bot_template': '{bot_name}: {message}\n', 'user_template': '{user_name}: {message}\n', 'response_template': '{bot_name}:', 'truncate_by_message': False}
generation_params: {'temperature': 0.9, 'top_p': 1.0, 'min_p': 0.05, 'top_k': 80, 'presence_penalty': 0.0, 'frequency_penalty': 0.0, 'stopping_words': ['\n', '</s>', '###', 'Bot:', 'User:', 'You:', '<|im_end|>'], 'max_input_tokens': 1024, 'best_of': 8, 'max_output_tokens': 64}
is_internal_developer: True
model_group:
model_name: retune_with_base
num_battles: 2748
num_wins: 1409
ranking_group: single
status: torndown
submission_type: function
timestamp: 2024-09-26T16:57:54+00:00
us_pacific_date: 2024-09-26
win_ratio: 0.5127365356622998
Resubmit model
Shutdown handler not registered because Python interpreter is not running in the main thread
run pipeline %s
run pipeline stage %s
Running pipeline stage StressChecker
Received healthy response to inference request in 1.834529161453247s
Received healthy response to inference request in 3.493927240371704s
Received healthy response to inference request in 3.9724833965301514s
Received healthy response to inference request in 3.3730809688568115s
Received healthy response to inference request in 2.9999895095825195s
5 requests
0 failed requests
5th percentile: 2.0676212310791016
10th percentile: 2.300713300704956
20th percentile: 2.766897439956665
30th percentile: 3.074607801437378
40th percentile: 3.223844385147095
50th percentile: 3.3730809688568115
60th percentile: 3.4214194774627686
70th percentile: 3.4697579860687258
80th percentile: 3.5896384716033936
90th percentile: 3.7810609340667725
95th percentile: 3.876772165298462
99th percentile: 3.9533411502838134
mean time: 3.134802055358887
Pipeline stage StressChecker completed in 17.51s
Shutdown handler de-registered
function_todun_2024-09-26 status is now deployed due to DeploymentManager action
function_todun_2024-09-26 status is now inactive due to auto deactivation removed underperforming models
run pipeline %s
Shutdown handler not registered because Python interpreter is not running in the main thread
admin requested tearing down of nousresearch-hermes-2-pr_1418_v5
Deleting key chaiml-albert-dpo-0912-v-9673-v1/config.json from bucket guanaco-mkml-models
Deleting key chaiml-albert-dpo-0912-v-8296-v2/flywheel_model.0.safetensors from bucket guanaco-mkml-models
function_todun_2024-09-26 status is now torndown due to DeploymentManager action
run pipeline %s
Shutdown handler not registered because Python interpreter is not running in the main thread
admin requested tearing down of nousresearch-hermes-2-pr_1418_v5
Deleting key nousresearch-meta-llama-4939-v67/flywheel_model.0.safetensors from bucket guanaco-mkml-models
Deleting key nousresearch-meta-llama-4939-v68/config.json from bucket guanaco-mkml-models
Deleting key nousresearch-meta-llama-4939-v66/special_tokens_map.json from bucket guanaco-mkml-models
chaiml-albert-dpo-0912-v_8296_v2 status is now torndown due to DeploymentManager action
Shutdown handler de-registered
function_todun_2024-09-26 status is now torndown due to DeploymentManager action
run pipeline %s
Shutdown handler not registered because Python interpreter is not running in the main thread
admin requested tearing down of nousresearch-hermes-2-pr_1418_v5
chaiml-albert-dpo-0912-v_8296_v2 status is now torndown due to DeploymentManager action
Shutdown handler de-registered
function_todun_2024-09-26 status is now torndown due to DeploymentManager action
run pipeline %s
Shutdown handler not registered because Python interpreter is not running in the main thread
admin requested tearing down of nousresearch-hermes-2-pr_1418_v5
Deleting key nousresearch-meta-llama-4939-v71/special_tokens_map.json from bucket guanaco-mkml-models
chaiml-albert-dpo-0912-v_8296_v2 status is now torndown due to DeploymentManager action
Shutdown handler de-registered
function_todun_2024-09-26 status is now torndown due to DeploymentManager action
Shutdown handler not registered because Python interpreter is not running in the main thread
admin requested tearing down of nousresearch-hermes-2-pr_1418_v5
mistralai-mistral-nemo_9330_v138 status is now torndown due to DeploymentManager action
Pipeline stage MKMLModelDeleter completed in 54.45s
Running pipeline stage MKMLModelDeleter
run pipeline stage %s
admin requested tearing down of chaiml-0916-intent-suppo_6584_v3
admin requested tearing down of blend_rofur_2024-10-03
chaiml-albert-dpo-0912-v_8296_v2 status is now torndown due to DeploymentManager action
Shutdown handler de-registered
function_todun_2024-09-26 status is now torndown due to DeploymentManager action
run pipeline stage %s
run pipeline %s
Shutdown handler not registered because Python interpreter is not running in the main thread
admin requested tearing down of rirv938-llama-8b-big-ret_5210_v1
mistralai-mistral-nemo_9330_v140 status is now torndown due to DeploymentManager action
Shutdown handler de-registered
Pipeline stage MKMLModelDeleter completed in 72.83s
Shutdown handler not registered because Python interpreter is not running in the main thread
admin requested tearing down of nousresearch-hermes-2-pr_1418_v5
Shutdown handler de-registered
chaiml-albert-dpo-0912-v_8296_v2 status is now torndown due to DeploymentManager action
function_todun_2024-09-26 status is now torndown due to DeploymentManager action
run pipeline stage %s
run pipeline %s
Shutdown handler not registered because Python interpreter is not running in the main thread
run pipeline %s
Shutdown handler not registered because Python interpreter is not running in the main thread
admin requested tearing down of nousresearch-hermes-2-pr_1418_v5
chaiml-albert-dpo-0912-v_8296_v2 status is now torndown due to DeploymentManager action
Shutdown handler de-registered
function_todun_2024-09-26 status is now torndown due to DeploymentManager action