bbchicago-llama3-8b-lora_3778

submission_id: bbchicago-llama3-8b-lora_3778_v2

developer_uid: Bbbrun0

best_of: 16

celo_rating: 1170.61

display_name: bbchicago-llama3-8b-lora_3778_v2

family_friendly_score: 0.0

formatter: {'memory_template': "{bot_name}'s Persona: {memory}\n####\n", 'prompt_template': '{prompt}\n<START>\n', 'bot_template': '{bot_name}: {message}\n', 'user_template': '{user_name}: {message}\n', 'response_template': '{bot_name}:', 'truncate_by_message': False}

generation_params: {'temperature': 1.0, 'top_p': 1.0, 'min_p': 0.0, 'top_k': 40, 'presence_penalty': 0.0, 'frequency_penalty': 0.0, 'stopping_words': ['\n'], 'max_input_tokens': 512, 'best_of': 16, 'max_output_tokens': 64}

is_internal_developer: False

language_model: BBChicago/llama3-8b-lora-v0.0.0

max_input_tokens: 512

max_output_tokens: 64

model_architecture: LlamaForCausalLM

model_eval_status: success

model_group: BBChicago/llama3-8b-lora

model_name: bbchicago-llama3-8b-lora_3778_v2

model_num_parameters: 8030261248.0

model_repo: BBChicago/llama3-8b-lora-v0.0.0

model_size: 8B

num_battles: 9376

num_wins: 4681

ranking_group: single

reward_formatter: {'bot_template': '{bot_name}: {message}\n', 'memory_template': "{bot_name}'s Persona: {memory}\n####\n", 'prompt_template': '{prompt}\n<START>\n', 'response_template': '{bot_name}:', 'truncate_by_message': False, 'user_template': '{user_name}: {message}\n'}

reward_repo: ChaiML/reward_gpt2_medium_preference_24m_e2

status: torndown

submission_type: basic

timestamp: 2024-05-30T08:44:11+00:00

us_pacific_date: 2024-05-30

win_ratio: 0.4992534129692833

Resubmit model

Running pipeline stage MKMLizer
Starting job with name bbchicago-llama3-8b-lora-3778-v2-mkmlizer
Waiting for job on bbchicago-llama3-8b-lora-3778-v2-mkmlizer to finish
bbchicago-llama3-8b-lora-3778-v2-mkmlizer: ╔═════════════════════════════════════════════════════════════════════╗
bbchicago-llama3-8b-lora-3778-v2-mkmlizer: ║     _____            __           __                                ║
bbchicago-llama3-8b-lora-3778-v2-mkmlizer: ║    / _/ /_ ___    __/ /  ___ ___ / /                                ║
bbchicago-llama3-8b-lora-3778-v2-mkmlizer: ║   / _/ / // / |/|/ / _ \/ -_) -_) /                                 ║
bbchicago-llama3-8b-lora-3778-v2-mkmlizer: ║  /_//_/\_, /|__,__/_//_/\__/\__/_/                                  ║
bbchicago-llama3-8b-lora-3778-v2-mkmlizer: ║       /___/                                                         ║
bbchicago-llama3-8b-lora-3778-v2-mkmlizer: ║                                                                     ║
bbchicago-llama3-8b-lora-3778-v2-mkmlizer: ║  Version: 0.8.14                                                    ║
bbchicago-llama3-8b-lora-3778-v2-mkmlizer: ║  Copyright 2023 MK ONE TECHNOLOGIES Inc.                            ║
bbchicago-llama3-8b-lora-3778-v2-mkmlizer: ║  https://mk1.ai                                                     ║
bbchicago-llama3-8b-lora-3778-v2-mkmlizer: ║                                                                     ║
bbchicago-llama3-8b-lora-3778-v2-mkmlizer: ║  The license key for the current software has been verified as      ║
bbchicago-llama3-8b-lora-3778-v2-mkmlizer: ║  belonging to:                                                      ║
bbchicago-llama3-8b-lora-3778-v2-mkmlizer: ║                                                                     ║
bbchicago-llama3-8b-lora-3778-v2-mkmlizer: ║  Chai Research Corp.                                                ║
bbchicago-llama3-8b-lora-3778-v2-mkmlizer: ║  Account ID: 7997a29f-0ceb-4cc7-9adf-840c57b4ae6f                   ║
bbchicago-llama3-8b-lora-3778-v2-mkmlizer: ║  Expiration: 2024-07-15 23:59:59                                    ║
bbchicago-llama3-8b-lora-3778-v2-mkmlizer: ║                                                                     ║
bbchicago-llama3-8b-lora-3778-v2-mkmlizer: ╚═════════════════════════════════════════════════════════════════════╝
bbchicago-llama3-8b-lora-3778-v2-mkmlizer: /opt/conda/lib/python3.10/site-packages/huggingface_hub/utils/_deprecation.py:131: FutureWarning: 'list_files_info' (from 'huggingface_hub.hf_api') is deprecated and will be removed from version '0.23'. Use `list_repo_tree` and `get_paths_info` instead.
bbchicago-llama3-8b-lora-3778-v2-mkmlizer:   warnings.warn(warning_message, FutureWarning)
Job bbchicago-llama3-8b-lora-3778-v2-mkmlizer completed after 122.89s with status: failed
Stopping job with name bbchicago-llama3-8b-lora-3778-v2-mkmlizer
%s, retrying in %s seconds...
Starting job with name bbchicago-llama3-8b-lora-3778-v2-mkmlizer
Waiting for job on bbchicago-llama3-8b-lora-3778-v2-mkmlizer to finish
bbchicago-llama3-8b-lora-3778-v2-mkmlizer: ╔═════════════════════════════════════════════════════════════════════╗
bbchicago-llama3-8b-lora-3778-v2-mkmlizer: ║     _____            __           __                                ║
bbchicago-llama3-8b-lora-3778-v2-mkmlizer: ║    / _/ /_ ___    __/ /  ___ ___ / /                                ║
bbchicago-llama3-8b-lora-3778-v2-mkmlizer: ║   / _/ / // / |/|/ / _ \/ -_) -_) /                                 ║
bbchicago-llama3-8b-lora-3778-v2-mkmlizer: ║  /_//_/\_, /|__,__/_//_/\__/\__/_/                                  ║
bbchicago-llama3-8b-lora-3778-v2-mkmlizer: ║       /___/                                                         ║
bbchicago-llama3-8b-lora-3778-v2-mkmlizer: ║                                                                     ║
bbchicago-llama3-8b-lora-3778-v2-mkmlizer: ║  Version: 0.8.14                                                    ║
bbchicago-llama3-8b-lora-3778-v2-mkmlizer: ║  Copyright 2023 MK ONE TECHNOLOGIES Inc.                            ║
bbchicago-llama3-8b-lora-3778-v2-mkmlizer: ║  https://mk1.ai                                                     ║
bbchicago-llama3-8b-lora-3778-v2-mkmlizer: ║                                                                     ║
bbchicago-llama3-8b-lora-3778-v2-mkmlizer: ║  The license key for the current software has been verified as      ║
bbchicago-llama3-8b-lora-3778-v2-mkmlizer: ║  belonging to:                                                      ║
bbchicago-llama3-8b-lora-3778-v2-mkmlizer: ║                                                                     ║
bbchicago-llama3-8b-lora-3778-v2-mkmlizer: ║  Chai Research Corp.                                                ║
bbchicago-llama3-8b-lora-3778-v2-mkmlizer: ║  Account ID: 7997a29f-0ceb-4cc7-9adf-840c57b4ae6f                   ║
bbchicago-llama3-8b-lora-3778-v2-mkmlizer: ║  Expiration: 2024-07-15 23:59:59                                    ║
bbchicago-llama3-8b-lora-3778-v2-mkmlizer: ║                                                                     ║
bbchicago-llama3-8b-lora-3778-v2-mkmlizer: ╚═════════════════════════════════════════════════════════════════════╝
bbchicago-llama3-8b-lora-3778-v2-mkmlizer: /opt/conda/lib/python3.10/site-packages/huggingface_hub/utils/_deprecation.py:131: FutureWarning: 'list_files_info' (from 'huggingface_hub.hf_api') is deprecated and will be removed from version '0.23'. Use `list_repo_tree` and `get_paths_info` instead.
bbchicago-llama3-8b-lora-3778-v2-mkmlizer:   warnings.warn(warning_message, FutureWarning)
bbchicago-llama3-8b-lora-3778-v2-mkmlizer: unable to retrieve container logs for containerd://9740d6f5d2f3e6bb349a8
Job bbchicago-llama3-8b-lora-3778-v2-mkmlizer completed after 104.54s with status: succeeded
Stopping job with name bbchicago-llama3-8b-lora-3778-v2-mkmlizer
Pipeline stage MKMLizer completed in 232.22s
Running pipeline stage MKMLKubeTemplater
Pipeline stage MKMLKubeTemplater completed in 0.09s
Running pipeline stage ISVCDeployer
Creating inference service bbchicago-llama3-8b-lora-3778-v2
Waiting for inference service bbchicago-llama3-8b-lora-3778-v2 to be ready
Tearing down inference service bbchicago-llama3-8b-lora-3778-v2
%s, retrying in %s seconds...
Creating inference service bbchicago-llama3-8b-lora-3778-v2
Waiting for inference service bbchicago-llama3-8b-lora-3778-v2 to be ready
Inference service bbchicago-llama3-8b-lora-3778-v2 ready after 40.23518490791321s
Pipeline stage ISVCDeployer completed in 656.48s
Running pipeline stage StressChecker
Received healthy response to inference request in 2.217672109603882s
Received healthy response to inference request in 1.2778449058532715s
Received healthy response to inference request in 1.270805835723877s
Received healthy response to inference request in 1.2707548141479492s
Received healthy response to inference request in 1.2916693687438965s
5 requests
0 failed requests
5th percentile: 1.2707650184631347
10th percentile: 1.2707752227783202
20th percentile: 1.2707956314086915
30th percentile: 1.2722136497497558
40th percentile: 1.2750292778015138
50th percentile: 1.2778449058532715
60th percentile: 1.2833746910095214
70th percentile: 1.2889044761657715
80th percentile: 1.4768699169158936
90th percentile: 1.8472710132598877
95th percentile: 2.0324715614318847
99th percentile: 2.1806319999694823
mean time: 1.465749406814575
Pipeline stage StressChecker completed in 7.90s
Running pipeline stage DaemonicModelEvalScorer
Pipeline stage DaemonicModelEvalScorer completed in 0.03s
Running pipeline stage DaemonicSafetyScorer
Running M-Eval for topic stay_in_character
Pipeline stage DaemonicSafetyScorer completed in 0.04s
M-Eval Dataset for topic stay_in_character is loaded
bbchicago-llama3-8b-lora_3778_v2 status is now deployed due to DeploymentManager action
bbchicago-llama3-8b-lora_3778_v2 status is now inactive due to auto deactivation removed underperforming models
admin requested tearing down of bbchicago-llama3-8b-lora_3778_v2
Running pipeline stage ISVCDeleter
Checking if service bbchicago-llama3-8b-lora-3778-v2 is running
Skipping teardown as no inference service was found
Pipeline stage ISVCDeleter completed in 2.23s
Running pipeline stage MKMLModelDeleter
Cleaning model data from S3
Cleaning model data from model cache
Deleting key bbchicago-llama3-8b-lora-3778-v2/config.json from bucket guanaco-mkml-models
Deleting key bbchicago-llama3-8b-lora-3778-v2/flywheel_model.0.safetensors from bucket guanaco-mkml-models
Deleting key bbchicago-llama3-8b-lora-3778-v2/special_tokens_map.json from bucket guanaco-mkml-models
Deleting key bbchicago-llama3-8b-lora-3778-v2/tokenizer.json from bucket guanaco-mkml-models
Deleting key bbchicago-llama3-8b-lora-3778-v2/tokenizer_config.json from bucket guanaco-mkml-models
Cleaning model data from model cache
Deleting key bbchicago-llama3-8b-lora-3778-v2_reward/config.json from bucket guanaco-reward-models
Deleting key bbchicago-llama3-8b-lora-3778-v2_reward/merges.txt from bucket guanaco-reward-models
Deleting key bbchicago-llama3-8b-lora-3778-v2_reward/reward.tensors from bucket guanaco-reward-models
Deleting key bbchicago-llama3-8b-lora-3778-v2_reward/special_tokens_map.json from bucket guanaco-reward-models
Deleting key bbchicago-llama3-8b-lora-3778-v2_reward/tokenizer.json from bucket guanaco-reward-models
Deleting key bbchicago-llama3-8b-lora-3778-v2_reward/tokenizer_config.json from bucket guanaco-reward-models
Deleting key bbchicago-llama3-8b-lora-3778-v2_reward/vocab.json from bucket guanaco-reward-models
Pipeline stage MKMLModelDeleter completed in 3.45s
bbchicago-llama3-8b-lora_3778_v2 status is now torndown due to DeploymentManager action