bbchicago-brt-8b-l3-v1-9_3086

submission_id: bbchicago-brt-8b-l3-v1-9_3086_v1
developer_uid: Bbbrun0
best_of: 16
celo_rating: 1239.84
display_name: bbchicago-brt-v1_9-dpo-s5000
family_friendly_score: 0.0
formatter: {'memory_template': "{bot_name}'s Persona: {memory}\n####\n", 'prompt_template': '{prompt}\n<START>\n', 'bot_template': '{bot_name}: {message}\n', 'user_template': '{user_name}: {message}\n', 'response_template': '{bot_name}:', 'truncate_by_message': False}
generation_params: {'temperature': 0.95, 'top_p': 0.95, 'min_p': 0.05, 'top_k': 80, 'presence_penalty': 0.0, 'frequency_penalty': 0.0, 'stopping_words': ['\n', '<|end_header_id|>', '<|eot_id|>', '\n\n{user_name}'], 'max_input_tokens': 512, 'best_of': 16, 'max_output_tokens': 64, 'reward_max_token_input': 256}
is_internal_developer: False
language_model: BBChicago/Brt-8B-L3_v1.9_with_dpo_lora_s5000
max_input_tokens: 512
max_output_tokens: 64
model_architecture: LlamaForCausalLM
model_group: BBChicago/Brt-8B-L3_v1.9
model_name: bbchicago-brt-v1_9-dpo-s5000
model_num_parameters: 8030261248.0
model_repo: BBChicago/Brt-8B-L3_v1.9_with_dpo_lora_s5000
model_size: 8B
num_battles: 21202
num_wins: 11564
ranking_group: single
reward_formatter: {'bot_template': '{bot_name}: {message}\n', 'memory_template': "{bot_name}'s Persona: {memory}\n####\n", 'prompt_template': '{prompt}\n<START>\n', 'response_template': '{bot_name}:', 'truncate_by_message': False, 'user_template': '{user_name}: {message}\n'}
reward_repo: Jellywibble/gpt2_xl_pairwise_89m_step_347634
status: torndown
submission_type: basic
timestamp: 2024-07-24T08:06:11+00:00
us_pacific_date: 2024-07-24
win_ratio: 0.5454202433732667
Resubmit model
Running pipeline stage MKMLizer
Starting job with name bbchicago-brt-8b-l3-v1-9-3086-v1-mkmlizer
Waiting for job on bbchicago-brt-8b-l3-v1-9-3086-v1-mkmlizer to finish
Failed to get response for submission undi95-meta-llama-3-70b_6209_v18: ('http://undi95-meta-llama-3-70b-6209-v18-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', '{"error":"TypeError : SamplingParameters.__init__() got an unexpected keyword argument \'reward_max_tokens\'"}')
bbchicago-brt-8b-l3-v1-9-3086-v1-mkmlizer: ╔═════════════════════════════════════════════════════════════════════╗
bbchicago-brt-8b-l3-v1-9-3086-v1-mkmlizer: ║     _____            __           __                                ║
bbchicago-brt-8b-l3-v1-9-3086-v1-mkmlizer: ║    / _/ /_ ___    __/ /  ___ ___ / /                                ║
bbchicago-brt-8b-l3-v1-9-3086-v1-mkmlizer: ║   / _/ / // / |/|/ / _ \/ -_) -_) /                                 ║
bbchicago-brt-8b-l3-v1-9-3086-v1-mkmlizer: ║  /_//_/\_, /|__,__/_//_/\__/\__/_/                                  ║
bbchicago-brt-8b-l3-v1-9-3086-v1-mkmlizer: ║       /___/                                                         ║
bbchicago-brt-8b-l3-v1-9-3086-v1-mkmlizer: ║                                                                     ║
bbchicago-brt-8b-l3-v1-9-3086-v1-mkmlizer: ║  Version: 0.9.6                                                     ║
bbchicago-brt-8b-l3-v1-9-3086-v1-mkmlizer: ║  Copyright 2023 MK ONE TECHNOLOGIES Inc.                            ║
bbchicago-brt-8b-l3-v1-9-3086-v1-mkmlizer: ║  https://mk1.ai                                                     ║
bbchicago-brt-8b-l3-v1-9-3086-v1-mkmlizer: ║                                                                     ║
bbchicago-brt-8b-l3-v1-9-3086-v1-mkmlizer: ║  The license key for the current software has been verified as      ║
bbchicago-brt-8b-l3-v1-9-3086-v1-mkmlizer: ║  belonging to:                                                      ║
bbchicago-brt-8b-l3-v1-9-3086-v1-mkmlizer: ║                                                                     ║
bbchicago-brt-8b-l3-v1-9-3086-v1-mkmlizer: ║  Chai Research Corp.                                                ║
bbchicago-brt-8b-l3-v1-9-3086-v1-mkmlizer: ║  Account ID: 7997a29f-0ceb-4cc7-9adf-840c57b4ae6f                   ║
bbchicago-brt-8b-l3-v1-9-3086-v1-mkmlizer: ║  Expiration: 2024-10-15 23:59:59                                    ║
bbchicago-brt-8b-l3-v1-9-3086-v1-mkmlizer: ║                                                                     ║
bbchicago-brt-8b-l3-v1-9-3086-v1-mkmlizer: ╚═════════════════════════════════════════════════════════════════════╝
Failed to get response for submission undi95-meta-llama-3-70b_6209_v18: ('http://undi95-meta-llama-3-70b-6209-v18-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', '{"error":"TypeError : SamplingParameters.__init__() got an unexpected keyword argument \'reward_max_tokens\'"}')
bbchicago-brt-8b-l3-v1-9-3086-v1-mkmlizer: Downloaded to shared memory in 54.498s
bbchicago-brt-8b-l3-v1-9-3086-v1-mkmlizer: quantizing model to /dev/shm/model_cache, profile:s0, folder:/tmp/tmp0_10mrne, device:0
bbchicago-brt-8b-l3-v1-9-3086-v1-mkmlizer: Saving flywheel model at /dev/shm/model_cache
bbchicago-brt-8b-l3-v1-9-3086-v1-mkmlizer: 
Loading 0:   0%|          | 0/291 [00:00<?, ?it/s]
Loading 0:   2%|▏         | 5/291 [00:00<00:10, 27.61it/s]
Loading 0:   4%|▍         | 12/291 [00:00<00:07, 36.57it/s]
Loading 0:   5%|▌         | 16/291 [00:00<00:08, 34.12it/s]
Loading 0:   7%|▋         | 21/291 [00:00<00:07, 37.59it/s]
Loading 0:   9%|▊         | 25/291 [00:00<00:07, 34.53it/s]
Loading 0:  10%|█         | 30/291 [00:00<00:06, 38.26it/s]
Loading 0:  12%|█▏        | 34/291 [00:01<00:09, 26.01it/s]
Loading 0:  13%|█▎        | 38/291 [00:01<00:09, 27.55it/s]
Loading 0:  14%|█▍        | 42/291 [00:01<00:09, 26.82it/s]
Loading 0:  16%|█▋        | 48/291 [00:01<00:07, 32.62it/s]
Loading 0:  18%|█▊        | 52/291 [00:01<00:07, 32.15it/s]
Loading 0:  20%|█▉        | 57/291 [00:01<00:06, 35.18it/s]
Loading 0:  21%|██        | 61/291 [00:01<00:06, 34.07it/s]
Loading 0:  23%|██▎       | 66/291 [00:01<00:06, 35.99it/s]
Loading 0:  24%|██▍       | 70/291 [00:02<00:06, 34.64it/s]
Loading 0:  25%|██▌       | 74/291 [00:02<00:06, 34.85it/s]
Loading 0:  27%|██▋       | 78/291 [00:02<00:06, 35.12it/s]
Loading 0:  28%|██▊       | 82/291 [00:02<00:08, 24.36it/s]
Loading 0:  30%|██▉       | 86/291 [00:02<00:07, 27.27it/s]
Loading 0:  31%|███       | 90/291 [00:02<00:06, 29.65it/s]
Loading 0:  32%|███▏      | 94/291 [00:02<00:06, 29.46it/s]
Loading 0:  34%|███▍      | 99/291 [00:03<00:05, 33.28it/s]
Loading 0:  35%|███▌      | 103/291 [00:03<00:05, 32.69it/s]
Loading 0:  37%|███▋      | 108/291 [00:03<00:05, 35.03it/s]
Loading 0:  38%|███▊      | 112/291 [00:03<00:05, 33.69it/s]
Loading 0:  40%|███▉      | 116/291 [00:03<00:05, 34.15it/s]
Loading 0:  42%|████▏     | 122/291 [00:03<00:04, 38.83it/s]
Loading 0:  44%|████▎     | 127/291 [00:03<00:04, 36.11it/s]
Loading 0:  46%|████▌     | 133/291 [00:04<00:05, 30.27it/s]
Loading 0:  47%|████▋     | 137/291 [00:04<00:05, 30.47it/s]
Loading 0:  48%|████▊     | 141/291 [00:04<00:05, 28.61it/s]
Loading 0:  51%|█████     | 147/291 [00:04<00:04, 33.46it/s]
Loading 0:  52%|█████▏    | 151/291 [00:04<00:04, 32.78it/s]
Loading 0:  54%|█████▎    | 156/291 [00:04<00:03, 35.51it/s]
Loading 0:  55%|█████▍    | 160/291 [00:04<00:03, 33.98it/s]
Loading 0:  57%|█████▋    | 165/291 [00:05<00:03, 36.45it/s]
Loading 0:  58%|█████▊    | 169/291 [00:05<00:03, 34.57it/s]
Loading 0:  60%|█████▉    | 174/291 [00:05<00:03, 36.14it/s]
Loading 0:  61%|██████    | 178/291 [00:05<00:03, 34.67it/s]
Loading 0:  63%|██████▎   | 184/291 [00:05<00:02, 40.72it/s]
Loading 0:  65%|██████▍   | 189/291 [00:05<00:04, 24.82it/s]
Loading 0:  67%|██████▋   | 194/291 [00:06<00:03, 26.56it/s]
Loading 0:  69%|██████▉   | 201/291 [00:06<00:02, 33.43it/s]
Loading 0:  71%|███████   | 206/291 [00:06<00:02, 34.28it/s]
Loading 0:  72%|███████▏  | 210/291 [00:06<00:02, 35.10it/s]
Loading 0:  74%|███████▎  | 214/291 [00:06<00:02, 34.07it/s]
Loading 0:  75%|███████▌  | 219/291 [00:06<00:01, 36.92it/s]
Loading 0:  77%|███████▋  | 223/291 [00:06<00:01, 35.54it/s]
Loading 0:  78%|███████▊  | 227/291 [00:06<00:01, 35.41it/s]
Loading 0:  79%|███████▉  | 231/291 [00:07<00:01, 35.55it/s]
Loading 0:  81%|████████  | 235/291 [00:07<00:02, 26.53it/s]
Loading 0:  82%|████████▏ | 239/291 [00:07<00:01, 26.48it/s]
Loading 0:  85%|████████▍ | 246/291 [00:07<00:01, 34.65it/s]
Loading 0:  86%|████████▌ | 250/291 [00:07<00:01, 33.75it/s]
Loading 0:  88%|████████▊ | 255/291 [00:07<00:00, 36.51it/s]
Loading 0:  89%|████████▉ | 259/291 [00:07<00:00, 35.33it/s]
Loading 0:  91%|█████████ | 264/291 [00:08<00:00, 38.07it/s]
Loading 0:  92%|█████████▏| 268/291 [00:08<00:00, 36.53it/s]
Loading 0:  94%|█████████▍| 273/291 [00:08<00:00, 38.73it/s]
Loading 0:  95%|█████████▌| 277/291 [00:08<00:00, 36.01it/s]
Loading 0:  97%|█████████▋| 281/291 [00:08<00:00, 36.02it/s]
Loading 0:  98%|█████████▊| 286/291 [00:14<00:01,  2.58it/s]
Loading 0:  99%|█████████▉| 289/291 [00:14<00:00,  3.21it/s]
                                                            
Special tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained.
bbchicago-brt-8b-l3-v1-9-3086-v1-mkmlizer: quantized model in 28.524s
bbchicago-brt-8b-l3-v1-9-3086-v1-mkmlizer: Processed model BBChicago/Brt-8B-L3_v1.9_with_dpo_lora_s5000 in 83.022s
bbchicago-brt-8b-l3-v1-9-3086-v1-mkmlizer: creating bucket guanaco-mkml-models
bbchicago-brt-8b-l3-v1-9-3086-v1-mkmlizer: Bucket 's3://guanaco-mkml-models/' created
bbchicago-brt-8b-l3-v1-9-3086-v1-mkmlizer: uploading /dev/shm/model_cache to s3://guanaco-mkml-models/bbchicago-brt-8b-l3-v1-9-3086-v1
bbchicago-brt-8b-l3-v1-9-3086-v1-mkmlizer: cp /dev/shm/model_cache/special_tokens_map.json s3://guanaco-mkml-models/bbchicago-brt-8b-l3-v1-9-3086-v1/special_tokens_map.json
bbchicago-brt-8b-l3-v1-9-3086-v1-mkmlizer: cp /dev/shm/model_cache/config.json s3://guanaco-mkml-models/bbchicago-brt-8b-l3-v1-9-3086-v1/config.json
bbchicago-brt-8b-l3-v1-9-3086-v1-mkmlizer: cp /dev/shm/model_cache/tokenizer_config.json s3://guanaco-mkml-models/bbchicago-brt-8b-l3-v1-9-3086-v1/tokenizer_config.json
bbchicago-brt-8b-l3-v1-9-3086-v1-mkmlizer: cp /dev/shm/model_cache/tokenizer.json s3://guanaco-mkml-models/bbchicago-brt-8b-l3-v1-9-3086-v1/tokenizer.json
bbchicago-brt-8b-l3-v1-9-3086-v1-mkmlizer: loading reward model from Jellywibble/gpt2_xl_pairwise_89m_step_347634
bbchicago-brt-8b-l3-v1-9-3086-v1-mkmlizer: /opt/conda/lib/python3.10/site-packages/transformers/models/auto/configuration_auto.py:950: FutureWarning: The `use_auth_token` argument is deprecated and will be removed in v5 of Transformers. Please use `token` instead.
bbchicago-brt-8b-l3-v1-9-3086-v1-mkmlizer:   warnings.warn(
bbchicago-brt-8b-l3-v1-9-3086-v1-mkmlizer: /opt/conda/lib/python3.10/site-packages/transformers/models/auto/tokenization_auto.py:778: FutureWarning: The `use_auth_token` argument is deprecated and will be removed in v5 of Transformers. Please use `token` instead.
bbchicago-brt-8b-l3-v1-9-3086-v1-mkmlizer:   warnings.warn(
bbchicago-brt-8b-l3-v1-9-3086-v1-mkmlizer: /opt/conda/lib/python3.10/site-packages/transformers/models/auto/auto_factory.py:469: FutureWarning: The `use_auth_token` argument is deprecated and will be removed in v5 of Transformers. Please use `token` instead.
bbchicago-brt-8b-l3-v1-9-3086-v1-mkmlizer:   warnings.warn(
bbchicago-brt-8b-l3-v1-9-3086-v1-mkmlizer: Saving model to /tmp/reward_cache/reward.tensors
bbchicago-brt-8b-l3-v1-9-3086-v1-mkmlizer: Saving duration: 1.367s
bbchicago-brt-8b-l3-v1-9-3086-v1-mkmlizer: Processed model Jellywibble/gpt2_xl_pairwise_89m_step_347634 in 10.688s
bbchicago-brt-8b-l3-v1-9-3086-v1-mkmlizer: creating bucket guanaco-reward-models
bbchicago-brt-8b-l3-v1-9-3086-v1-mkmlizer: Bucket 's3://guanaco-reward-models/' created
bbchicago-brt-8b-l3-v1-9-3086-v1-mkmlizer: uploading /tmp/reward_cache to s3://guanaco-reward-models/bbchicago-brt-8b-l3-v1-9-3086-v1_reward
bbchicago-brt-8b-l3-v1-9-3086-v1-mkmlizer: cp /tmp/reward_cache/special_tokens_map.json s3://guanaco-reward-models/bbchicago-brt-8b-l3-v1-9-3086-v1_reward/special_tokens_map.json
bbchicago-brt-8b-l3-v1-9-3086-v1-mkmlizer: cp /tmp/reward_cache/tokenizer_config.json s3://guanaco-reward-models/bbchicago-brt-8b-l3-v1-9-3086-v1_reward/tokenizer_config.json
bbchicago-brt-8b-l3-v1-9-3086-v1-mkmlizer: cp /tmp/reward_cache/config.json s3://guanaco-reward-models/bbchicago-brt-8b-l3-v1-9-3086-v1_reward/config.json
bbchicago-brt-8b-l3-v1-9-3086-v1-mkmlizer: cp /tmp/reward_cache/merges.txt s3://guanaco-reward-models/bbchicago-brt-8b-l3-v1-9-3086-v1_reward/merges.txt
bbchicago-brt-8b-l3-v1-9-3086-v1-mkmlizer: cp /tmp/reward_cache/vocab.json s3://guanaco-reward-models/bbchicago-brt-8b-l3-v1-9-3086-v1_reward/vocab.json
bbchicago-brt-8b-l3-v1-9-3086-v1-mkmlizer: cp /tmp/reward_cache/tokenizer.json s3://guanaco-reward-models/bbchicago-brt-8b-l3-v1-9-3086-v1_reward/tokenizer.json
bbchicago-brt-8b-l3-v1-9-3086-v1-mkmlizer: cp /tmp/reward_cache/reward.tensors s3://guanaco-reward-models/bbchicago-brt-8b-l3-v1-9-3086-v1_reward/reward.tensors
Job bbchicago-brt-8b-l3-v1-9-3086-v1-mkmlizer completed after 177.32s with status: succeeded
Stopping job with name bbchicago-brt-8b-l3-v1-9-3086-v1-mkmlizer
Pipeline stage MKMLizer completed in 178.56s
Running pipeline stage MKMLKubeTemplater
Pipeline stage MKMLKubeTemplater completed in 0.10s
Running pipeline stage ISVCDeployer
Creating inference service bbchicago-brt-8b-l3-v1-9-3086-v1
Waiting for inference service bbchicago-brt-8b-l3-v1-9-3086-v1 to be ready
Inference service bbchicago-brt-8b-l3-v1-9-3086-v1 ready after 60.595181703567505s
Pipeline stage ISVCDeployer completed in 62.02s
Running pipeline stage StressChecker
Received healthy response to inference request in 2.321272850036621s
Received healthy response to inference request in 1.3807666301727295s
Received healthy response to inference request in 1.3575034141540527s
Received healthy response to inference request in 1.3920724391937256s
Received healthy response to inference request in 1.3800623416900635s
5 requests
0 failed requests
5th percentile: 1.3620151996612548
10th percentile: 1.366526985168457
20th percentile: 1.3755505561828614
30th percentile: 1.3802031993865966
40th percentile: 1.3804849147796632
50th percentile: 1.3807666301727295
60th percentile: 1.385288953781128
70th percentile: 1.3898112773895264
80th percentile: 1.5779125213623049
90th percentile: 1.949592685699463
95th percentile: 2.135432767868042
99th percentile: 2.284104833602905
mean time: 1.5663355350494386
Pipeline stage StressChecker completed in 8.73s
bbchicago-brt-8b-l3-v1-9_3086_v1 status is now deployed due to DeploymentManager action
bbchicago-brt-8b-l3-v1-9_3086_v1 status is now inactive due to auto deactivation removed underperforming models
admin requested tearing down of bbchicago-brt-8b-l3-v1-9_3086_v1
Running pipeline stage ISVCDeleter
Checking if service bbchicago-brt-8b-l3-v1-9-3086-v1 is running
Tearing down inference service bbchicago-brt-8b-l3-v1-9-3086-v1
Service bbchicago-brt-8b-l3-v1-9-3086-v1 has been torndown
Pipeline stage ISVCDeleter completed in 5.25s
Running pipeline stage MKMLModelDeleter
Cleaning model data from S3
Cleaning model data from model cache
Deleting key bbchicago-brt-8b-l3-v1-9-3086-v1/config.json from bucket guanaco-mkml-models
Deleting key bbchicago-brt-8b-l3-v1-9-3086-v1/flywheel_model.0.safetensors from bucket guanaco-mkml-models
Deleting key bbchicago-brt-8b-l3-v1-9-3086-v1/special_tokens_map.json from bucket guanaco-mkml-models
Deleting key bbchicago-brt-8b-l3-v1-9-3086-v1/tokenizer.json from bucket guanaco-mkml-models
Deleting key bbchicago-brt-8b-l3-v1-9-3086-v1/tokenizer_config.json from bucket guanaco-mkml-models
Cleaning model data from model cache
Deleting key bbchicago-brt-8b-l3-v1-9-3086-v1_reward/config.json from bucket guanaco-reward-models
Deleting key bbchicago-brt-8b-l3-v1-9-3086-v1_reward/merges.txt from bucket guanaco-reward-models
Deleting key bbchicago-brt-8b-l3-v1-9-3086-v1_reward/reward.tensors from bucket guanaco-reward-models
Deleting key bbchicago-brt-8b-l3-v1-9-3086-v1_reward/special_tokens_map.json from bucket guanaco-reward-models
Deleting key bbchicago-brt-8b-l3-v1-9-3086-v1_reward/tokenizer.json from bucket guanaco-reward-models
Deleting key bbchicago-brt-8b-l3-v1-9-3086-v1_reward/tokenizer_config.json from bucket guanaco-reward-models
Deleting key bbchicago-brt-8b-l3-v1-9-3086-v1_reward/vocab.json from bucket guanaco-reward-models
Pipeline stage MKMLModelDeleter completed in 6.33s
bbchicago-brt-8b-l3-v1-9_3086_v1 status is now torndown due to DeploymentManager action