submission_id: bbchicago-brt-v1-13-with_9716_v2
developer_uid: Bbbrun0
alignment_samples: 10821
alignment_score: -1.3087814579514494
best_of: 16
celo_rating: 1270.34
display_name: auto
formatter: {'memory_template': "<|begin_of_text|><|start_header_id|>system<|end_header_id|>\n\n{bot_name}'s Persona: {memory}\n\n", 'prompt_template': '{prompt}<|eot_id|>', 'bot_template': '<|start_header_id|>assistant<|end_header_id|>\n\n{bot_name}: {message}<|eot_id|>', 'user_template': '<|start_header_id|>user<|end_header_id|>\n\n{user_name}: {message}<|eot_id|>', 'response_template': '<|start_header_id|>assistant<|end_header_id|>\n\n{bot_name}:', 'truncate_by_message': False}
generation_params: {'temperature': 1.4, 'top_p': 1.0, 'min_p': 0.2, 'top_k': 50, 'presence_penalty': 0.0, 'frequency_penalty': 0.0, 'stopping_words': ['\n', '<|end_header_id|>', '<|eot_id|>', '\n\n{user_name}'], 'max_input_tokens': 512, 'best_of': 16, 'max_output_tokens': 64}
is_internal_developer: False
language_model: BBChicago/Brt_v1.13_with_156k_DPO_s5900
max_input_tokens: 512
max_output_tokens: 64
model_architecture: LlamaForCausalLM
model_group: BBChicago/Brt_v1.13_with
model_name: auto
model_num_parameters: 8030261248.0
model_repo: BBChicago/Brt_v1.13_with_156k_DPO_s5900
model_size: 8B
num_battles: 10821
num_wins: 5964
propriety_score: 0.7181136120042872
propriety_total_count: 933.0
ranking_group: single
status: inactive
submission_type: basic
timestamp: 2024-08-28T03:12:33+00:00
us_pacific_date: 2024-08-27
win_ratio: 0.55115054061547
Download Preference Data
Resubmit model
Running pipeline stage MKMLizer
Starting job with name bbchicago-brt-v1-13-with-9716-v2-mkmlizer
Waiting for job on bbchicago-brt-v1-13-with-9716-v2-mkmlizer to finish
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
bbchicago-brt-v1-13-with-9716-v2-mkmlizer: ╔═════════════════════════════════════════════════════════════════════╗
bbchicago-brt-v1-13-with-9716-v2-mkmlizer: ║     _____            __           __                                ║
bbchicago-brt-v1-13-with-9716-v2-mkmlizer: ║    / _/ /_ ___    __/ /  ___ ___ / /                                ║
bbchicago-brt-v1-13-with-9716-v2-mkmlizer: ║   / _/ / // / |/|/ / _ \/ -_) -_) /                                 ║
bbchicago-brt-v1-13-with-9716-v2-mkmlizer: ║  /_//_/\_, /|__,__/_//_/\__/\__/_/                                  ║
bbchicago-brt-v1-13-with-9716-v2-mkmlizer: ║       /___/                                                         ║
bbchicago-brt-v1-13-with-9716-v2-mkmlizer: ║                                                                     ║
bbchicago-brt-v1-13-with-9716-v2-mkmlizer: ║  Version: 0.10.1                                                    ║
bbchicago-brt-v1-13-with-9716-v2-mkmlizer: ║  Copyright 2023 MK ONE TECHNOLOGIES Inc.                            ║
bbchicago-brt-v1-13-with-9716-v2-mkmlizer: ║  https://mk1.ai                                                     ║
bbchicago-brt-v1-13-with-9716-v2-mkmlizer: ║                                                                     ║
bbchicago-brt-v1-13-with-9716-v2-mkmlizer: ║  The license key for the current software has been verified as      ║
bbchicago-brt-v1-13-with-9716-v2-mkmlizer: ║  belonging to:                                                      ║
bbchicago-brt-v1-13-with-9716-v2-mkmlizer: ║                                                                     ║
bbchicago-brt-v1-13-with-9716-v2-mkmlizer: ║  Chai Research Corp.                                                ║
bbchicago-brt-v1-13-with-9716-v2-mkmlizer: ║  Account ID: 7997a29f-0ceb-4cc7-9adf-840c57b4ae6f                   ║
bbchicago-brt-v1-13-with-9716-v2-mkmlizer: ║  Expiration: 2024-10-15 23:59:59                                    ║
bbchicago-brt-v1-13-with-9716-v2-mkmlizer: ║                                                                     ║
bbchicago-brt-v1-13-with-9716-v2-mkmlizer: ╚═════════════════════════════════════════════════════════════════════╝
bbchicago-brt-v1-13-with-9716-v2-mkmlizer: Downloaded to shared memory in 76.236s
bbchicago-brt-v1-13-with-9716-v2-mkmlizer: quantizing model to /dev/shm/model_cache, profile:s0, folder:/tmp/tmps224hfbr, device:0
bbchicago-brt-v1-13-with-9716-v2-mkmlizer: Saving flywheel model at /dev/shm/model_cache
bbchicago-brt-v1-13-with-9716-v2-mkmlizer: quantized model in 29.151s
bbchicago-brt-v1-13-with-9716-v2-mkmlizer: Processed model BBChicago/Brt_v1.13_with_156k_DPO_s5900 in 105.387s
bbchicago-brt-v1-13-with-9716-v2-mkmlizer: creating bucket guanaco-mkml-models
bbchicago-brt-v1-13-with-9716-v2-mkmlizer: Bucket 's3://guanaco-mkml-models/' created
bbchicago-brt-v1-13-with-9716-v2-mkmlizer: uploading /dev/shm/model_cache to s3://guanaco-mkml-models/bbchicago-brt-v1-13-with-9716-v2
bbchicago-brt-v1-13-with-9716-v2-mkmlizer: cp /dev/shm/model_cache/config.json s3://guanaco-mkml-models/bbchicago-brt-v1-13-with-9716-v2/config.json
bbchicago-brt-v1-13-with-9716-v2-mkmlizer: cp /dev/shm/model_cache/special_tokens_map.json s3://guanaco-mkml-models/bbchicago-brt-v1-13-with-9716-v2/special_tokens_map.json
bbchicago-brt-v1-13-with-9716-v2-mkmlizer: cp /dev/shm/model_cache/tokenizer_config.json s3://guanaco-mkml-models/bbchicago-brt-v1-13-with-9716-v2/tokenizer_config.json
bbchicago-brt-v1-13-with-9716-v2-mkmlizer: cp /dev/shm/model_cache/tokenizer.json s3://guanaco-mkml-models/bbchicago-brt-v1-13-with-9716-v2/tokenizer.json
bbchicago-brt-v1-13-with-9716-v2-mkmlizer: cp /dev/shm/model_cache/flywheel_model.0.safetensors s3://guanaco-mkml-models/bbchicago-brt-v1-13-with-9716-v2/flywheel_model.0.safetensors
bbchicago-brt-v1-13-with-9716-v2-mkmlizer: 
Loading 0:   0%|          | 0/291 [00:00<?, ?it/s]
Loading 0:   2%|▏         | 5/291 [00:00<00:11, 25.20it/s]
Loading 0:   4%|▍         | 12/291 [00:00<00:08, 34.45it/s]
Loading 0:   5%|▌         | 16/291 [00:00<00:08, 32.48it/s]
Loading 0:   7%|▋         | 21/291 [00:00<00:07, 35.07it/s]
Loading 0:   9%|▊         | 25/291 [00:00<00:08, 32.93it/s]
Loading 0:  10%|█         | 30/291 [00:00<00:07, 36.96it/s]
Loading 0:  12%|█▏        | 34/291 [00:01<00:10, 25.25it/s]
Loading 0:  13%|█▎        | 38/291 [00:01<00:09, 26.48it/s]
Loading 0:  14%|█▍        | 42/291 [00:01<00:09, 25.29it/s]
Loading 0:  16%|█▋        | 48/291 [00:01<00:07, 30.99it/s]
Loading 0:  18%|█▊        | 52/291 [00:01<00:07, 30.58it/s]
Loading 0:  20%|█▉        | 57/291 [00:01<00:06, 34.61it/s]
Loading 0:  21%|██        | 61/291 [00:01<00:06, 33.21it/s]
Loading 0:  23%|██▎       | 66/291 [00:02<00:06, 36.76it/s]
Loading 0:  24%|██▍       | 70/291 [00:02<00:06, 33.96it/s]
Loading 0:  25%|██▌       | 74/291 [00:02<00:06, 33.93it/s]
Loading 0:  27%|██▋       | 78/291 [00:02<00:06, 34.47it/s]
Loading 0:  28%|██▊       | 82/291 [00:02<00:08, 23.89it/s]
Loading 0:  29%|██▉       | 85/291 [00:02<00:08, 24.71it/s]
Loading 0:  31%|███       | 90/291 [00:02<00:06, 29.56it/s]
Loading 0:  32%|███▏      | 94/291 [00:03<00:06, 30.29it/s]
Loading 0:  34%|███▍      | 99/291 [00:03<00:05, 32.78it/s]
Loading 0:  35%|███▌      | 103/291 [00:03<00:06, 31.21it/s]
Loading 0:  37%|███▋      | 108/291 [00:03<00:05, 34.28it/s]
Loading 0:  38%|███▊      | 112/291 [00:03<00:05, 32.31it/s]
Loading 0:  40%|███▉      | 116/291 [00:03<00:05, 32.29it/s]
Loading 0:  42%|████▏     | 122/291 [00:03<00:04, 36.03it/s]
Loading 0:  44%|████▎     | 127/291 [00:04<00:04, 33.97it/s]
Loading 0:  46%|████▌     | 133/291 [00:04<00:05, 30.56it/s]
Loading 0:  47%|████▋     | 137/291 [00:04<00:05, 30.74it/s]
Loading 0:  48%|████▊     | 141/291 [00:04<00:05, 28.54it/s]
Loading 0:  51%|█████     | 147/291 [00:04<00:04, 33.44it/s]
Loading 0:  52%|█████▏    | 151/291 [00:04<00:04, 31.97it/s]
Loading 0:  54%|█████▎    | 156/291 [00:04<00:03, 34.29it/s]
Loading 0:  55%|█████▍    | 160/291 [00:05<00:03, 33.15it/s]
Loading 0:  57%|█████▋    | 165/291 [00:05<00:03, 36.14it/s]
Loading 0:  58%|█████▊    | 169/291 [00:05<00:03, 34.27it/s]
Loading 0:  60%|█████▉    | 174/291 [00:05<00:03, 34.38it/s]
Loading 0:  61%|██████    | 178/291 [00:05<00:03, 32.72it/s]
Loading 0:  63%|██████▎   | 183/291 [00:05<00:02, 36.80it/s]
Loading 0:  64%|██████▍   | 187/291 [00:05<00:03, 26.60it/s]
Loading 0:  66%|██████▌   | 191/291 [00:06<00:03, 27.74it/s]
Loading 0:  67%|██████▋   | 195/291 [00:06<00:03, 26.83it/s]
Loading 0:  69%|██████▉   | 201/291 [00:06<00:02, 32.18it/s]
Loading 0:  70%|███████   | 205/291 [00:06<00:02, 30.49it/s]
Loading 0:  72%|███████▏  | 210/291 [00:06<00:02, 32.92it/s]
Loading 0:  74%|███████▎  | 214/291 [00:06<00:02, 32.25it/s]
Loading 0:  75%|███████▌  | 219/291 [00:06<00:02, 35.10it/s]
Loading 0:  77%|███████▋  | 223/291 [00:07<00:02, 33.47it/s]
Loading 0:  78%|███████▊  | 227/291 [00:07<00:01, 34.22it/s]
Loading 0:  79%|███████▉  | 231/291 [00:07<00:01, 32.92it/s]
Loading 0:  81%|████████  | 235/291 [00:07<00:02, 25.03it/s]
Loading 0:  82%|████████▏ | 239/291 [00:07<00:02, 24.63it/s]
Loading 0:  85%|████████▍ | 246/291 [00:07<00:01, 31.41it/s]
Loading 0:  86%|████████▌ | 250/291 [00:07<00:01, 30.60it/s]
Loading 0:  88%|████████▊ | 255/291 [00:08<00:01, 32.92it/s]
Loading 0:  89%|████████▉ | 259/291 [00:08<00:01, 31.90it/s]
Loading 0:  91%|█████████ | 264/291 [00:08<00:00, 34.58it/s]
Loading 0:  92%|█████████▏| 268/291 [00:08<00:00, 32.59it/s]
Loading 0:  94%|█████████▍| 273/291 [00:08<00:00, 35.58it/s]
Loading 0:  95%|█████████▌| 277/291 [00:08<00:00, 33.80it/s]
Loading 0:  97%|█████████▋| 281/291 [00:08<00:00, 33.16it/s]
Loading 0:  98%|█████████▊| 286/291 [00:14<00:01,  2.56it/s]
Loading 0:  99%|█████████▉| 289/291 [00:14<00:00,  3.17it/s]
                                                            

Job bbchicago-brt-v1-13-with-9716-v2-mkmlizer completed after 125.66s with status: succeeded
Stopping job with name bbchicago-brt-v1-13-with-9716-v2-mkmlizer
Pipeline stage MKMLizer completed in 127.31s
Running pipeline stage MKMLKubeTemplater
Pipeline stage MKMLKubeTemplater completed in 0.09s
Running pipeline stage ISVCDeployer
Creating inference service bbchicago-brt-v1-13-with-9716-v2
Waiting for inference service bbchicago-brt-v1-13-with-9716-v2 to be ready
Inference service bbchicago-brt-v1-13-with-9716-v2 ready after 160.41459941864014s
Pipeline stage ISVCDeployer completed in 160.93s
Running pipeline stage StressChecker
Received healthy response to inference request in 2.3869071006774902s
Received healthy response to inference request in 1.815925121307373s
Received healthy response to inference request in 1.733513593673706s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Received healthy response to inference request in 2.4824724197387695s
Received healthy response to inference request in 1.7126760482788086s
5 requests
0 failed requests
5th percentile: 1.716843557357788
10th percentile: 1.7210110664367675
20th percentile: 1.7293460845947266
30th percentile: 1.7499958992004394
40th percentile: 1.7829605102539063
50th percentile: 1.815925121307373
60th percentile: 2.04431791305542
70th percentile: 2.2727107048034667
80th percentile: 2.406020164489746
90th percentile: 2.444246292114258
95th percentile: 2.4633593559265137
99th percentile: 2.4786498069763185
mean time: 2.0262988567352296
Pipeline stage StressChecker completed in 11.32s
bbchicago-brt-v1-13-with_9716_v2 status is now deployed due to DeploymentManager action
bbchicago-brt-v1-13-with_9716_v2 status is now inactive due to auto deactivation removed underperforming models

Usage Metrics

Latency Metrics