bbchicago-brt-8b-l3-v1-_4895

submission_id: bbchicago-brt-8b-l3-v1-_4895_v22
developer_uid: Bbbrun0
best_of: 16
celo_rating: 1244.75
display_name: brt-v1_7-dpo-s2500-pygmalion
family_friendly_score: 0.0
formatter: {'memory_template': "{bot_name}'s Persona: {memory}\n####\n", 'prompt_template': '{prompt}\n<START>\n', 'bot_template': '{bot_name}: {message}\n', 'user_template': '{user_name}: {message}\n', 'response_template': '{bot_name}:', 'truncate_by_message': False}
generation_params: {'temperature': 0.95, 'top_p': 0.95, 'min_p': 0.05, 'top_k': 80, 'presence_penalty': 0.0, 'frequency_penalty': 0.0, 'stopping_words': ['\n', '<|end_header_id|>', '<|eot_id|>', '\n\n{user_name}'], 'max_input_tokens': 512, 'best_of': 16, 'max_output_tokens': 64, 'reward_max_token_input': 256}
is_internal_developer: False
language_model: BBChicago/Brt-8B-L3_v1.7_with_dpo_lora_s2500
max_input_tokens: 512
max_output_tokens: 64
model_architecture: LlamaForCausalLM
model_group: BBChicago/Brt-8B-L3_v1.7
model_name: brt-v1_7-dpo-s2500-pygmalion
model_num_parameters: 8030261248.0
model_repo: BBChicago/Brt-8B-L3_v1.7_with_dpo_lora_s2500
model_size: 8B
num_battles: 9893
num_wins: 5599
ranking_group: single
reward_formatter: {'bot_template': '{bot_name}: {message}\n', 'memory_template': "{bot_name}'s Persona: {memory}\n####\n", 'prompt_template': '{prompt}\n<START>\n', 'response_template': '{bot_name}:', 'truncate_by_message': False, 'user_template': '{user_name}: {message}\n'}
reward_repo: Jellywibble/gpt2_xl_pairwise_89m_step_347634
status: torndown
submission_type: basic
timestamp: 2024-07-25T06:16:13+00:00
us_pacific_date: 2024-07-24
win_ratio: 0.5659557262711008
Resubmit model
Running pipeline stage MKMLizer
Starting job with name bbchicago-brt-8b-l3-v1-4895-v22-mkmlizer
Waiting for job on bbchicago-brt-8b-l3-v1-4895-v22-mkmlizer to finish
bbchicago-brt-8b-l3-v1-4895-v22-mkmlizer: ╔═════════════════════════════════════════════════════════════════════╗
bbchicago-brt-8b-l3-v1-4895-v22-mkmlizer: ║     _____            __           __                                ║
bbchicago-brt-8b-l3-v1-4895-v22-mkmlizer: ║    / _/ /_ ___    __/ /  ___ ___ / /                                ║
bbchicago-brt-8b-l3-v1-4895-v22-mkmlizer: ║   / _/ / // / |/|/ / _ \/ -_) -_) /                                 ║
bbchicago-brt-8b-l3-v1-4895-v22-mkmlizer: ║  /_//_/\_, /|__,__/_//_/\__/\__/_/                                  ║
bbchicago-brt-8b-l3-v1-4895-v22-mkmlizer: ║       /___/                                                         ║
bbchicago-brt-8b-l3-v1-4895-v22-mkmlizer: ║                                                                     ║
bbchicago-brt-8b-l3-v1-4895-v22-mkmlizer: ║  Version: 0.9.6                                                     ║
bbchicago-brt-8b-l3-v1-4895-v22-mkmlizer: ║  Copyright 2023 MK ONE TECHNOLOGIES Inc.                            ║
bbchicago-brt-8b-l3-v1-4895-v22-mkmlizer: ║  https://mk1.ai                                                     ║
bbchicago-brt-8b-l3-v1-4895-v22-mkmlizer: ║                                                                     ║
bbchicago-brt-8b-l3-v1-4895-v22-mkmlizer: ║  The license key for the current software has been verified as      ║
bbchicago-brt-8b-l3-v1-4895-v22-mkmlizer: ║  belonging to:                                                      ║
bbchicago-brt-8b-l3-v1-4895-v22-mkmlizer: ║                                                                     ║
bbchicago-brt-8b-l3-v1-4895-v22-mkmlizer: ║  Chai Research Corp.                                                ║
bbchicago-brt-8b-l3-v1-4895-v22-mkmlizer: ║  Account ID: 7997a29f-0ceb-4cc7-9adf-840c57b4ae6f                   ║
bbchicago-brt-8b-l3-v1-4895-v22-mkmlizer: ║  Expiration: 2024-10-15 23:59:59                                    ║
bbchicago-brt-8b-l3-v1-4895-v22-mkmlizer: ║                                                                     ║
bbchicago-brt-8b-l3-v1-4895-v22-mkmlizer: ╚═════════════════════════════════════════════════════════════════════╝
Failed to get response for submission neversleep-lumimaid-mist_1884_v1: ('http://neversleep-lumimaid-mist-1884-v1-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'request timeout')
bbchicago-brt-8b-l3-v1-4895-v22-mkmlizer: Downloaded to shared memory in 37.592s
bbchicago-brt-8b-l3-v1-4895-v22-mkmlizer: quantizing model to /dev/shm/model_cache, profile:s0, folder:/tmp/tmp2xi6qfdy, device:0
bbchicago-brt-8b-l3-v1-4895-v22-mkmlizer: Saving flywheel model at /dev/shm/model_cache
Failed to get response for submission neversleep-lumimaid-mist_1884_v1: ('http://neversleep-lumimaid-mist-1884-v1-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'request timeout')
Failed to get response for submission neversleep-lumimaid-mist_1893_v1: ('http://neversleep-lumimaid-mist-1893-v1-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'activator request timeout')
bbchicago-brt-8b-l3-v1-4895-v22-mkmlizer: 
Loading 0:   0%|          | 0/291 [00:00<?, ?it/s]
Loading 0:   2%|▏         | 5/291 [00:00<00:10, 26.52it/s]
Loading 0:   4%|▍         | 12/291 [00:00<00:06, 41.43it/s]
Loading 0:   6%|▌         | 17/291 [00:00<00:07, 38.74it/s]
Loading 0:   8%|▊         | 22/291 [00:00<00:07, 37.92it/s]
Loading 0:   9%|▉         | 26/291 [00:00<00:06, 38.01it/s]
Loading 0:  11%|█         | 32/291 [00:00<00:06, 38.83it/s]
Loading 0:  12%|█▏        | 36/291 [00:01<00:11, 22.64it/s]
Loading 0:  14%|█▍        | 41/291 [00:01<00:10, 24.88it/s]
Loading 0:  16%|█▋        | 48/291 [00:01<00:07, 32.45it/s]
Loading 0:  18%|█▊        | 52/291 [00:01<00:07, 32.11it/s]
Loading 0:  20%|█▉        | 57/291 [00:01<00:06, 35.21it/s]
Loading 0:  21%|██        | 61/291 [00:01<00:06, 34.12it/s]
Loading 0:  23%|██▎       | 66/291 [00:01<00:06, 35.98it/s]
Loading 0:  24%|██▍       | 70/291 [00:02<00:06, 33.97it/s]
Loading 0:  25%|██▌       | 74/291 [00:02<00:06, 34.32it/s]
Loading 0:  27%|██▋       | 78/291 [00:02<00:06, 34.56it/s]
Loading 0:  28%|██▊       | 82/291 [00:02<00:08, 23.43it/s]
Loading 0:  30%|██▉       | 86/291 [00:02<00:07, 26.36it/s]
Loading 0:  31%|███       | 90/291 [00:02<00:06, 29.05it/s]
Loading 0:  32%|███▏      | 94/291 [00:02<00:06, 29.34it/s]
Loading 0:  34%|███▍      | 99/291 [00:03<00:05, 32.20it/s]
Loading 0:  35%|███▌      | 103/291 [00:03<00:05, 31.48it/s]
Loading 0:  37%|███▋      | 108/291 [00:03<00:05, 34.83it/s]
Loading 0:  38%|███▊      | 112/291 [00:03<00:05, 33.38it/s]
Loading 0:  40%|███▉      | 116/291 [00:03<00:05, 34.09it/s]
Loading 0:  42%|████▏     | 122/291 [00:03<00:04, 38.99it/s]
Loading 0:  44%|████▎     | 127/291 [00:03<00:04, 36.21it/s]
Loading 0:  46%|████▌     | 133/291 [00:04<00:05, 30.14it/s]
Loading 0:  47%|████▋     | 137/291 [00:04<00:05, 29.90it/s]
Loading 0:  48%|████▊     | 141/291 [00:04<00:05, 28.34it/s]
Loading 0:  51%|█████     | 147/291 [00:04<00:04, 33.37it/s]
Loading 0:  52%|█████▏    | 151/291 [00:04<00:04, 32.60it/s]
Loading 0:  54%|█████▎    | 156/291 [00:04<00:03, 35.15it/s]
Loading 0:  55%|█████▍    | 160/291 [00:04<00:03, 33.79it/s]
Loading 0:  57%|█████▋    | 165/291 [00:05<00:03, 36.26it/s]
Loading 0:  58%|█████▊    | 169/291 [00:05<00:03, 33.75it/s]
Loading 0:  60%|█████▉    | 174/291 [00:05<00:03, 34.47it/s]
Loading 0:  61%|██████    | 178/291 [00:05<00:03, 32.81it/s]
Loading 0:  63%|██████▎   | 183/291 [00:05<00:02, 36.85it/s]
Loading 0:  64%|██████▍   | 187/291 [00:05<00:04, 25.93it/s]
Loading 0:  66%|██████▌   | 191/291 [00:05<00:03, 27.39it/s]
Loading 0:  67%|██████▋   | 195/291 [00:06<00:03, 26.37it/s]
Loading 0:  69%|██████▉   | 201/291 [00:06<00:02, 31.60it/s]
Loading 0:  70%|███████   | 205/291 [00:06<00:02, 31.34it/s]
Loading 0:  72%|███████▏  | 210/291 [00:06<00:02, 33.86it/s]
Loading 0:  74%|███████▎  | 214/291 [00:06<00:02, 32.90it/s]
Loading 0:  75%|███████▍  | 218/291 [00:06<00:02, 34.41it/s]
Loading 0:  76%|███████▋  | 222/291 [00:06<00:02, 30.75it/s]
Loading 0:  78%|███████▊  | 227/291 [00:07<00:01, 33.03it/s]
Loading 0:  79%|███████▉  | 231/291 [00:07<00:01, 32.92it/s]
Loading 0:  81%|████████  | 235/291 [00:07<00:02, 24.65it/s]
Loading 0:  82%|████████▏ | 239/291 [00:07<00:02, 24.73it/s]
Loading 0:  85%|████████▍ | 246/291 [00:07<00:01, 32.18it/s]
Loading 0:  86%|████████▌ | 250/291 [00:07<00:01, 31.28it/s]
Loading 0:  88%|████████▊ | 255/291 [00:08<00:01, 33.70it/s]
Loading 0:  89%|████████▉ | 259/291 [00:08<00:00, 32.65it/s]
Loading 0:  91%|█████████ | 264/291 [00:08<00:00, 34.96it/s]
Loading 0:  92%|█████████▏| 268/291 [00:08<00:00, 33.20it/s]
Loading 0:  94%|█████████▍| 273/291 [00:08<00:00, 35.01it/s]
Loading 0:  95%|█████████▌| 277/291 [00:08<00:00, 32.85it/s]
Loading 0:  97%|█████████▋| 281/291 [00:08<00:00, 32.64it/s]
Loading 0:  98%|█████████▊| 286/291 [00:14<00:01,  2.52it/s]
Loading 0:  99%|█████████▉| 289/291 [00:14<00:00,  3.13it/s]
                                                            
Special tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained.
bbchicago-brt-8b-l3-v1-4895-v22-mkmlizer: quantized model in 29.292s
bbchicago-brt-8b-l3-v1-4895-v22-mkmlizer: Processed model BBChicago/Brt-8B-L3_v1.7_with_dpo_lora_s2500 in 66.885s
bbchicago-brt-8b-l3-v1-4895-v22-mkmlizer: creating bucket guanaco-mkml-models
Failed to get response for submission neversleep-lumimaid-mist_7954_v1: ('http://neversleep-lumimaid-mist-7954-v1-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'request timeout')
Failed to get response for submission neversleep-lumimaid-mist_1884_v1: ('http://neversleep-lumimaid-mist-1884-v1-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'request timeout')
bbchicago-brt-8b-l3-v1-4895-v22-mkmlizer: cp /dev/shm/model_cache/config.json s3://guanaco-mkml-models/bbchicago-brt-8b-l3-v1-4895-v22/config.json
bbchicago-brt-8b-l3-v1-4895-v22-mkmlizer: cp /dev/shm/model_cache/special_tokens_map.json s3://guanaco-mkml-models/bbchicago-brt-8b-l3-v1-4895-v22/special_tokens_map.json
bbchicago-brt-8b-l3-v1-4895-v22-mkmlizer: cp /dev/shm/model_cache/tokenizer_config.json s3://guanaco-mkml-models/bbchicago-brt-8b-l3-v1-4895-v22/tokenizer_config.json
bbchicago-brt-8b-l3-v1-4895-v22-mkmlizer: cp /dev/shm/model_cache/tokenizer.json s3://guanaco-mkml-models/bbchicago-brt-8b-l3-v1-4895-v22/tokenizer.json
bbchicago-brt-8b-l3-v1-4895-v22-mkmlizer: cp /dev/shm/model_cache/flywheel_model.0.safetensors s3://guanaco-mkml-models/bbchicago-brt-8b-l3-v1-4895-v22/flywheel_model.0.safetensors
bbchicago-brt-8b-l3-v1-4895-v22-mkmlizer: loading reward model from Jellywibble/gpt2_xl_pairwise_89m_step_347634
bbchicago-brt-8b-l3-v1-4895-v22-mkmlizer: /opt/conda/lib/python3.10/site-packages/transformers/models/auto/configuration_auto.py:950: FutureWarning: The `use_auth_token` argument is deprecated and will be removed in v5 of Transformers. Please use `token` instead.
bbchicago-brt-8b-l3-v1-4895-v22-mkmlizer:   warnings.warn(
bbchicago-brt-8b-l3-v1-4895-v22-mkmlizer: /opt/conda/lib/python3.10/site-packages/transformers/models/auto/tokenization_auto.py:778: FutureWarning: The `use_auth_token` argument is deprecated and will be removed in v5 of Transformers. Please use `token` instead.
bbchicago-brt-8b-l3-v1-4895-v22-mkmlizer:   warnings.warn(
bbchicago-brt-8b-l3-v1-4895-v22-mkmlizer: /opt/conda/lib/python3.10/site-packages/transformers/models/auto/auto_factory.py:469: FutureWarning: The `use_auth_token` argument is deprecated and will be removed in v5 of Transformers. Please use `token` instead.
bbchicago-brt-8b-l3-v1-4895-v22-mkmlizer:   warnings.warn(
Failed to get response for submission neversleep-lumimaid-mist_1893_v1: ('http://neversleep-lumimaid-mist-1893-v1-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'request timeout')
Failed to get response for submission neversleep-lumimaid-mist_1884_v1: ('http://neversleep-lumimaid-mist-1884-v1-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'request timeout')
bbchicago-brt-8b-l3-v1-4895-v22-mkmlizer: Saving model to /tmp/reward_cache/reward.tensors
bbchicago-brt-8b-l3-v1-4895-v22-mkmlizer: Saving duration: 1.384s
bbchicago-brt-8b-l3-v1-4895-v22-mkmlizer: Processed model Jellywibble/gpt2_xl_pairwise_89m_step_347634 in 10.956s
bbchicago-brt-8b-l3-v1-4895-v22-mkmlizer: creating bucket guanaco-reward-models
bbchicago-brt-8b-l3-v1-4895-v22-mkmlizer: Bucket 's3://guanaco-reward-models/' created
bbchicago-brt-8b-l3-v1-4895-v22-mkmlizer: uploading /tmp/reward_cache to s3://guanaco-reward-models/bbchicago-brt-8b-l3-v1-4895-v22_reward
bbchicago-brt-8b-l3-v1-4895-v22-mkmlizer: cp /tmp/reward_cache/config.json s3://guanaco-reward-models/bbchicago-brt-8b-l3-v1-4895-v22_reward/config.json
bbchicago-brt-8b-l3-v1-4895-v22-mkmlizer: cp /tmp/reward_cache/special_tokens_map.json s3://guanaco-reward-models/bbchicago-brt-8b-l3-v1-4895-v22_reward/special_tokens_map.json
bbchicago-brt-8b-l3-v1-4895-v22-mkmlizer: cp /tmp/reward_cache/tokenizer_config.json s3://guanaco-reward-models/bbchicago-brt-8b-l3-v1-4895-v22_reward/tokenizer_config.json
bbchicago-brt-8b-l3-v1-4895-v22-mkmlizer: cp /tmp/reward_cache/merges.txt s3://guanaco-reward-models/bbchicago-brt-8b-l3-v1-4895-v22_reward/merges.txt
bbchicago-brt-8b-l3-v1-4895-v22-mkmlizer: cp /tmp/reward_cache/vocab.json s3://guanaco-reward-models/bbchicago-brt-8b-l3-v1-4895-v22_reward/vocab.json
Failed to get response for submission neversleep-lumimaid-mist_1884_v1: ('http://neversleep-lumimaid-mist-1884-v1-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'request timeout')
Failed to get response for submission neversleep-lumimaid-mist_1893_v1: ('http://neversleep-lumimaid-mist-1893-v1-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'request timeout')
Failed to get response for submission neversleep-lumimaid-mist_1893_v1: ('http://neversleep-lumimaid-mist-1893-v1-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'request timeout')
bbchicago-brt-8b-l3-v1-4895-v22-mkmlizer: cp /tmp/reward_cache/reward.tensors s3://guanaco-reward-models/bbchicago-brt-8b-l3-v1-4895-v22_reward/reward.tensors
Job bbchicago-brt-8b-l3-v1-4895-v22-mkmlizer completed after 106.63s with status: succeeded
Stopping job with name bbchicago-brt-8b-l3-v1-4895-v22-mkmlizer
Pipeline stage MKMLizer completed in 107.97s
Running pipeline stage MKMLKubeTemplater
Pipeline stage MKMLKubeTemplater completed in 0.11s
Running pipeline stage ISVCDeployer
Creating inference service bbchicago-brt-8b-l3-v1-4895-v22
Waiting for inference service bbchicago-brt-8b-l3-v1-4895-v22 to be ready
Failed to get response for submission neversleep-lumimaid-mist_1893_v1: ('http://neversleep-lumimaid-mist-1893-v1-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'request timeout')
Failed to get response for submission neversleep-lumimaid-mist_7954_v1: ('http://neversleep-lumimaid-mist-7954-v1-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'request timeout')
Failed to get response for submission undi95-meta-llama-3-70b_6209_v18: ('http://undi95-meta-llama-3-70b-6209-v18-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', '{"error":"TypeError : SamplingParameters.__init__() got an unexpected keyword argument \'reward_max_tokens\'"}')
Inference service bbchicago-brt-8b-l3-v1-4895-v22 ready after 60.57190155982971s
Pipeline stage ISVCDeployer completed in 62.82s
Running pipeline stage StressChecker
Failed to get response for submission neversleep-lumimaid-mist_7954_v1: ('http://neversleep-lumimaid-mist-7954-v1-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'request timeout')
Failed to get response for submission neversleep-lumimaid-mist_1893_v1: ('http://neversleep-lumimaid-mist-1893-v1-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'request timeout')
Received healthy response to inference request in 2.268312454223633s
Received healthy response to inference request in 1.3999719619750977s
Received healthy response to inference request in 1.3869445323944092s
Received healthy response to inference request in 1.392282485961914s
Received healthy response to inference request in 1.3837745189666748s
5 requests
0 failed requests
5th percentile: 1.3844085216522217
10th percentile: 1.3850425243377686
20th percentile: 1.3863105297088623
30th percentile: 1.3880121231079101
40th percentile: 1.3901473045349122
50th percentile: 1.392282485961914
60th percentile: 1.3953582763671875
70th percentile: 1.3984340667724608
80th percentile: 1.5736400604248049
90th percentile: 1.9209762573242188
95th percentile: 2.0946443557739256
99th percentile: 2.2335788345336915
mean time: 1.5662571907043457
Pipeline stage StressChecker completed in 8.64s
bbchicago-brt-8b-l3-v1-_4895_v22 status is now deployed due to DeploymentManager action
bbchicago-brt-8b-l3-v1-_4895_v22 status is now inactive due to auto deactivation removed underperforming models
admin requested tearing down of bbchicago-brt-8b-l3-v1-_4895_v22
Running pipeline stage ISVCDeleter
Checking if service bbchicago-brt-8b-l3-v1-4895-v22 is running
Tearing down inference service bbchicago-brt-8b-l3-v1-4895-v22
Service bbchicago-brt-8b-l3-v1-4895-v22 has been torndown
Pipeline stage ISVCDeleter completed in 5.20s
Running pipeline stage MKMLModelDeleter
Cleaning model data from S3
Cleaning model data from model cache
Deleting key bbchicago-brt-8b-l3-v1-4895-v22/config.json from bucket guanaco-mkml-models
Deleting key bbchicago-brt-8b-l3-v1-4895-v22/flywheel_model.0.safetensors from bucket guanaco-mkml-models
Deleting key bbchicago-brt-8b-l3-v1-4895-v22/special_tokens_map.json from bucket guanaco-mkml-models
Deleting key bbchicago-brt-8b-l3-v1-4895-v22/tokenizer.json from bucket guanaco-mkml-models
Deleting key bbchicago-brt-8b-l3-v1-4895-v22/tokenizer_config.json from bucket guanaco-mkml-models
Cleaning model data from model cache
Deleting key bbchicago-brt-8b-l3-v1-4895-v22_reward/config.json from bucket guanaco-reward-models
Deleting key bbchicago-brt-8b-l3-v1-4895-v22_reward/merges.txt from bucket guanaco-reward-models
Deleting key bbchicago-brt-8b-l3-v1-4895-v22_reward/reward.tensors from bucket guanaco-reward-models
Deleting key bbchicago-brt-8b-l3-v1-4895-v22_reward/special_tokens_map.json from bucket guanaco-reward-models
Deleting key bbchicago-brt-8b-l3-v1-4895-v22_reward/tokenizer.json from bucket guanaco-reward-models
Deleting key bbchicago-brt-8b-l3-v1-4895-v22_reward/tokenizer_config.json from bucket guanaco-reward-models
Deleting key bbchicago-brt-8b-l3-v1-4895-v22_reward/vocab.json from bucket guanaco-reward-models
Pipeline stage MKMLModelDeleter completed in 5.66s
bbchicago-brt-8b-l3-v1-_4895_v22 status is now torndown due to DeploymentManager action