bbchicago-brt-v1-13-with_9716

developer_uid: Bbbrun0

submission_id: bbchicago-brt-v1-13-with_9716_v2

model_name: auto

model_group: BBChicago/Brt_v1.13_with

status: torndown

timestamp: 2024-08-28T03:12:33+00:00

num_battles: 10821

num_wins: 5964

celo_rating: 1270.42

family_friendly_score: 0.0

submission_type: basic

model_repo: BBChicago/Brt_v1.13_with_156k_DPO_s5900

model_architecture: LlamaForCausalLM

model_num_parameters: 8030261248.0

best_of: 16

max_input_tokens: 512

max_output_tokens: 64

display_name: auto

is_internal_developer: False

language_model: BBChicago/Brt_v1.13_with_156k_DPO_s5900

model_size: 8B

ranking_group: single

us_pacific_date: 2024-08-27

win_ratio: 0.55115054061547

generation_params: {'temperature': 1.4, 'top_p': 1.0, 'min_p': 0.2, 'top_k': 50, 'presence_penalty': 0.0, 'frequency_penalty': 0.0, 'stopping_words': ['\n', '<|end_header_id|>', '<|eot_id|>', '\n\n{user_name}'], 'max_input_tokens': 512, 'best_of': 16, 'max_output_tokens': 64}

formatter: {'memory_template': "<|begin_of_text|><|start_header_id|>system<|end_header_id|>\n\n{bot_name}'s Persona: {memory}\n\n", 'prompt_template': '{prompt}<|eot_id|>', 'bot_template': '<|start_header_id|>assistant<|end_header_id|>\n\n{bot_name}: {message}<|eot_id|>', 'user_template': '<|start_header_id|>user<|end_header_id|>\n\n{user_name}: {message}<|eot_id|>', 'response_template': '<|start_header_id|>assistant<|end_header_id|>\n\n{bot_name}:', 'truncate_by_message': False}

Resubmit model

Running pipeline stage MKMLizer
Starting job with name bbchicago-brt-v1-13-with-9716-v2-mkmlizer
Waiting for job on bbchicago-brt-v1-13-with-9716-v2-mkmlizer to finish
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
bbchicago-brt-v1-13-with-9716-v2-mkmlizer: ╔═════════════════════════════════════════════════════════════════════╗
bbchicago-brt-v1-13-with-9716-v2-mkmlizer: ║     _____            __           __                                ║
bbchicago-brt-v1-13-with-9716-v2-mkmlizer: ║    / _/ /_ ___    __/ /  ___ ___ / /                                ║
bbchicago-brt-v1-13-with-9716-v2-mkmlizer: ║   / _/ / // / |/|/ / _ \/ -_) -_) /                                 ║
bbchicago-brt-v1-13-with-9716-v2-mkmlizer: ║  /_//_/\_, /|__,__/_//_/\__/\__/_/                                  ║
bbchicago-brt-v1-13-with-9716-v2-mkmlizer: ║       /___/                                                         ║
bbchicago-brt-v1-13-with-9716-v2-mkmlizer: ║                                                                     ║
bbchicago-brt-v1-13-with-9716-v2-mkmlizer: ║  Version: 0.10.1                                                    ║
bbchicago-brt-v1-13-with-9716-v2-mkmlizer: ║  Copyright 2023 MK ONE TECHNOLOGIES Inc.                            ║
bbchicago-brt-v1-13-with-9716-v2-mkmlizer: ║  https://mk1.ai                                                     ║
bbchicago-brt-v1-13-with-9716-v2-mkmlizer: ║                                                                     ║
bbchicago-brt-v1-13-with-9716-v2-mkmlizer: ║  The license key for the current software has been verified as      ║
bbchicago-brt-v1-13-with-9716-v2-mkmlizer: ║  belonging to:                                                      ║
bbchicago-brt-v1-13-with-9716-v2-mkmlizer: ║                                                                     ║
bbchicago-brt-v1-13-with-9716-v2-mkmlizer: ║  Chai Research Corp.                                                ║
bbchicago-brt-v1-13-with-9716-v2-mkmlizer: ║  Account ID: 7997a29f-0ceb-4cc7-9adf-840c57b4ae6f                   ║
bbchicago-brt-v1-13-with-9716-v2-mkmlizer: ║  Expiration: 2024-10-15 23:59:59                                    ║
bbchicago-brt-v1-13-with-9716-v2-mkmlizer: ║                                                                     ║
bbchicago-brt-v1-13-with-9716-v2-mkmlizer: ╚═════════════════════════════════════════════════════════════════════╝
bbchicago-brt-v1-13-with-9716-v2-mkmlizer: Downloaded to shared memory in 76.236s
bbchicago-brt-v1-13-with-9716-v2-mkmlizer: quantizing model to /dev/shm/model_cache, profile:s0, folder:/tmp/tmps224hfbr, device:0
bbchicago-brt-v1-13-with-9716-v2-mkmlizer: Saving flywheel model at /dev/shm/model_cache
bbchicago-brt-v1-13-with-9716-v2-mkmlizer: quantized model in 29.151s
bbchicago-brt-v1-13-with-9716-v2-mkmlizer: Processed model BBChicago/Brt_v1.13_with_156k_DPO_s5900 in 105.387s
bbchicago-brt-v1-13-with-9716-v2-mkmlizer: creating bucket guanaco-mkml-models
bbchicago-brt-v1-13-with-9716-v2-mkmlizer: Bucket 's3://guanaco-mkml-models/' created
bbchicago-brt-v1-13-with-9716-v2-mkmlizer: uploading /dev/shm/model_cache to s3://guanaco-mkml-models/bbchicago-brt-v1-13-with-9716-v2
bbchicago-brt-v1-13-with-9716-v2-mkmlizer: cp /dev/shm/model_cache/config.json s3://guanaco-mkml-models/bbchicago-brt-v1-13-with-9716-v2/config.json
bbchicago-brt-v1-13-with-9716-v2-mkmlizer: cp /dev/shm/model_cache/special_tokens_map.json s3://guanaco-mkml-models/bbchicago-brt-v1-13-with-9716-v2/special_tokens_map.json
bbchicago-brt-v1-13-with-9716-v2-mkmlizer: cp /dev/shm/model_cache/tokenizer_config.json s3://guanaco-mkml-models/bbchicago-brt-v1-13-with-9716-v2/tokenizer_config.json
bbchicago-brt-v1-13-with-9716-v2-mkmlizer: cp /dev/shm/model_cache/tokenizer.json s3://guanaco-mkml-models/bbchicago-brt-v1-13-with-9716-v2/tokenizer.json
bbchicago-brt-v1-13-with-9716-v2-mkmlizer: cp /dev/shm/model_cache/flywheel_model.0.safetensors s3://guanaco-mkml-models/bbchicago-brt-v1-13-with-9716-v2/flywheel_model.0.safetensors
bbchicago-brt-v1-13-with-9716-v2-mkmlizer: 
Loading 0:   0%|          | 0/291 [00:00<?, ?it/s]
Loading 0:   2%|▏         | 5/291 [00:00<00:11, 25.20it/s]
Loading 0:   4%|▍         | 12/291 [00:00<00:08, 34.45it/s]
Loading 0:   5%|▌         | 16/291 [00:00<00:08, 32.48it/s]
Loading 0:   7%|▋         | 21/291 [00:00<00:07, 35.07it/s]
Loading 0:   9%|▊         | 25/291 [00:00<00:08, 32.93it/s]
Loading 0:  10%|█         | 30/291 [00:00<00:07, 36.96it/s]
Loading 0:  12%|█▏        | 34/291 [00:01<00:10, 25.25it/s]
Loading 0:  13%|█▎        | 38/291 [00:01<00:09, 26.48it/s]
Loading 0:  14%|█▍        | 42/291 [00:01<00:09, 25.29it/s]
Loading 0:  16%|█▋        | 48/291 [00:01<00:07, 30.99it/s]
Loading 0:  18%|█▊        | 52/291 [00:01<00:07, 30.58it/s]
Loading 0:  20%|█▉        | 57/291 [00:01<00:06, 34.61it/s]
Loading 0:  21%|██        | 61/291 [00:01<00:06, 33.21it/s]
Loading 0:  23%|██▎       | 66/291 [00:02<00:06, 36.76it/s]
Loading 0:  24%|██▍       | 70/291 [00:02<00:06, 33.96it/s]
Loading 0:  25%|██▌       | 74/291 [00:02<00:06, 33.93it/s]
Loading 0:  27%|██▋       | 78/291 [00:02<00:06, 34.47it/s]
Loading 0:  28%|██▊       | 82/291 [00:02<00:08, 23.89it/s]
Loading 0:  29%|██▉       | 85/291 [00:02<00:08, 24.71it/s]
Loading 0:  31%|███       | 90/291 [00:02<00:06, 29.56it/s]
Loading 0:  32%|███▏      | 94/291 [00:03<00:06, 30.29it/s]
Loading 0:  34%|███▍      | 99/291 [00:03<00:05, 32.78it/s]
Loading 0:  35%|███▌      | 103/291 [00:03<00:06, 31.21it/s]
Loading 0:  37%|███▋      | 108/291 [00:03<00:05, 34.28it/s]
Loading 0:  38%|███▊      | 112/291 [00:03<00:05, 32.31it/s]
Loading 0:  40%|███▉      | 116/291 [00:03<00:05, 32.29it/s]
Loading 0:  42%|████▏     | 122/291 [00:03<00:04, 36.03it/s]
Loading 0:  44%|████▎     | 127/291 [00:04<00:04, 33.97it/s]
Loading 0:  46%|████▌     | 133/291 [00:04<00:05, 30.56it/s]
Loading 0:  47%|████▋     | 137/291 [00:04<00:05, 30.74it/s]
Loading 0:  48%|████▊     | 141/291 [00:04<00:05, 28.54it/s]
Loading 0:  51%|█████     | 147/291 [00:04<00:04, 33.44it/s]
Loading 0:  52%|█████▏    | 151/291 [00:04<00:04, 31.97it/s]
Loading 0:  54%|█████▎    | 156/291 [00:04<00:03, 34.29it/s]
Loading 0:  55%|█████▍    | 160/291 [00:05<00:03, 33.15it/s]
Loading 0:  57%|█████▋    | 165/291 [00:05<00:03, 36.14it/s]
Loading 0:  58%|█████▊    | 169/291 [00:05<00:03, 34.27it/s]
Loading 0:  60%|█████▉    | 174/291 [00:05<00:03, 34.38it/s]
Loading 0:  61%|██████    | 178/291 [00:05<00:03, 32.72it/s]
Loading 0:  63%|██████▎   | 183/291 [00:05<00:02, 36.80it/s]
Loading 0:  64%|██████▍   | 187/291 [00:05<00:03, 26.60it/s]
Loading 0:  66%|██████▌   | 191/291 [00:06<00:03, 27.74it/s]
Loading 0:  67%|██████▋   | 195/291 [00:06<00:03, 26.83it/s]
Loading 0:  69%|██████▉   | 201/291 [00:06<00:02, 32.18it/s]
Loading 0:  70%|███████   | 205/291 [00:06<00:02, 30.49it/s]
Loading 0:  72%|███████▏  | 210/291 [00:06<00:02, 32.92it/s]
Loading 0:  74%|███████▎  | 214/291 [00:06<00:02, 32.25it/s]
Loading 0:  75%|███████▌  | 219/291 [00:06<00:02, 35.10it/s]
Loading 0:  77%|███████▋  | 223/291 [00:07<00:02, 33.47it/s]
Loading 0:  78%|███████▊  | 227/291 [00:07<00:01, 34.22it/s]
Loading 0:  79%|███████▉  | 231/291 [00:07<00:01, 32.92it/s]
Loading 0:  81%|████████  | 235/291 [00:07<00:02, 25.03it/s]
Loading 0:  82%|████████▏ | 239/291 [00:07<00:02, 24.63it/s]
Loading 0:  85%|████████▍ | 246/291 [00:07<00:01, 31.41it/s]
Loading 0:  86%|████████▌ | 250/291 [00:07<00:01, 30.60it/s]
Loading 0:  88%|████████▊ | 255/291 [00:08<00:01, 32.92it/s]
Loading 0:  89%|████████▉ | 259/291 [00:08<00:01, 31.90it/s]
Loading 0:  91%|█████████ | 264/291 [00:08<00:00, 34.58it/s]
Loading 0:  92%|█████████▏| 268/291 [00:08<00:00, 32.59it/s]
Loading 0:  94%|█████████▍| 273/291 [00:08<00:00, 35.58it/s]
Loading 0:  95%|█████████▌| 277/291 [00:08<00:00, 33.80it/s]
Loading 0:  97%|█████████▋| 281/291 [00:08<00:00, 33.16it/s]
Loading 0:  98%|█████████▊| 286/291 [00:14<00:01,  2.56it/s]
Loading 0:  99%|█████████▉| 289/291 [00:14<00:00,  3.17it/s]
                                                            
Job bbchicago-brt-v1-13-with-9716-v2-mkmlizer completed after 125.66s with status: succeeded
Stopping job with name bbchicago-brt-v1-13-with-9716-v2-mkmlizer
Pipeline stage MKMLizer completed in 127.31s
Running pipeline stage MKMLKubeTemplater
Pipeline stage MKMLKubeTemplater completed in 0.09s
Running pipeline stage ISVCDeployer
Creating inference service bbchicago-brt-v1-13-with-9716-v2
Waiting for inference service bbchicago-brt-v1-13-with-9716-v2 to be ready
Inference service bbchicago-brt-v1-13-with-9716-v2 ready after 160.41459941864014s
Pipeline stage ISVCDeployer completed in 160.93s
Running pipeline stage StressChecker
Received healthy response to inference request in 2.3869071006774902s
Received healthy response to inference request in 1.815925121307373s
Received healthy response to inference request in 1.733513593673706s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Received healthy response to inference request in 2.4824724197387695s
Received healthy response to inference request in 1.7126760482788086s
5 requests
0 failed requests
5th percentile: 1.716843557357788
10th percentile: 1.7210110664367675
20th percentile: 1.7293460845947266
30th percentile: 1.7499958992004394
40th percentile: 1.7829605102539063
50th percentile: 1.815925121307373
60th percentile: 2.04431791305542
70th percentile: 2.2727107048034667
80th percentile: 2.406020164489746
90th percentile: 2.444246292114258
95th percentile: 2.4633593559265137
99th percentile: 2.4786498069763185
mean time: 2.0262988567352296
Pipeline stage StressChecker completed in 11.32s
bbchicago-brt-v1-13-with_9716_v2 status is now deployed due to DeploymentManager action
bbchicago-brt-v1-13-with_9716_v2 status is now inactive due to auto deactivation removed underperforming models
bbchicago-brt-v1-13-with_9716_v2 status is now torndown due to DeploymentManager action
bbchicago-brt-v1-13-with_9716_v2 status is now torndown due to DeploymentManager action