rirv938-multihead-mcl-sa_7055

developer_uid: robert_irvine
submission_id: rirv938-multihead-mcl-sa_7055_v1
model_name: rirv938-multihead-mcl-sa_7055_v1
model_group: rirv938/multihead_mcl_sa
status: torndown
timestamp: 2024-11-06T21:47:08+00:00
num_battles: 12106
num_wins: 5895
celo_rating: 1229.7
family_friendly_score: 0.5866
family_friendly_standard_error: 0.0069642004566209895
submission_type: basic
model_repo: rirv938/multihead_mcl_samples_26m
model_architecture: LlamaForSequenceClassification
model_num_parameters: 8030261248.0
best_of: 1
max_input_tokens: 256
max_output_tokens: 1
display_name: rirv938-multihead-mcl-sa_7055_v1
ineligible_reason: max_output_tokens!=64
is_internal_developer: True
language_model: rirv938/multihead_mcl_samples_26m
model_size: 8B
ranking_group: single
us_pacific_date: 2024-11-06
win_ratio: 0.48694862051875104
generation_params: {'temperature': 1.0, 'top_p': 1.0, 'min_p': 0.0, 'top_k': 40, 'presence_penalty': 0.0, 'frequency_penalty': 0.0, 'stopping_words': ['\n'], 'max_input_tokens': 256, 'best_of': 1, 'max_output_tokens': 1}
formatter: {'memory_template': '', 'prompt_template': '', 'bot_template': '{bot_name}: {message}\n', 'user_template': '{user_name}: {message}\n', 'response_template': '', 'truncate_by_message': False}
Resubmit model
Shutdown handler not registered because Python interpreter is not running in the main thread
run pipeline %s
run pipeline stage %s
Running pipeline stage MKMLizer
Starting job with name rirv938-multihead-mcl-sa-7055-v1-mkmlizer
Waiting for job on rirv938-multihead-mcl-sa-7055-v1-mkmlizer to finish
rirv938-multihead-mcl-sa-7055-v1-mkmlizer: ╔═════════════════════════════════════════════════════════════════════╗
rirv938-multihead-mcl-sa-7055-v1-mkmlizer: ║     _____            __           __                                ║
rirv938-multihead-mcl-sa-7055-v1-mkmlizer: ║    / _/ /_ ___    __/ /  ___ ___ / /                                ║
rirv938-multihead-mcl-sa-7055-v1-mkmlizer: ║   / _/ / // / |/|/ / _ \/ -_) -_) /                                 ║
rirv938-multihead-mcl-sa-7055-v1-mkmlizer: ║  /_//_/\_, /|__,__/_//_/\__/\__/_/                                  ║
rirv938-multihead-mcl-sa-7055-v1-mkmlizer: ║       /___/                                                         ║
rirv938-multihead-mcl-sa-7055-v1-mkmlizer: ║                                                                     ║
rirv938-multihead-mcl-sa-7055-v1-mkmlizer: ║  Version: 0.11.28                                                   ║
rirv938-multihead-mcl-sa-7055-v1-mkmlizer: ║  Copyright 2023 MK ONE TECHNOLOGIES Inc.                            ║
rirv938-multihead-mcl-sa-7055-v1-mkmlizer: ║  https://mk1.ai                                                     ║
rirv938-multihead-mcl-sa-7055-v1-mkmlizer: ║                                                                     ║
rirv938-multihead-mcl-sa-7055-v1-mkmlizer: ║  The license key for the current software has been verified as      ║
rirv938-multihead-mcl-sa-7055-v1-mkmlizer: ║  belonging to:                                                      ║
rirv938-multihead-mcl-sa-7055-v1-mkmlizer: ║                                                                     ║
rirv938-multihead-mcl-sa-7055-v1-mkmlizer: ║  Chai Research Corp.                                                ║
rirv938-multihead-mcl-sa-7055-v1-mkmlizer: ║  Account ID: 7997a29f-0ceb-4cc7-9adf-840c57b4ae6f                   ║
rirv938-multihead-mcl-sa-7055-v1-mkmlizer: ║  Expiration: 2025-01-15 23:59:59                                    ║
rirv938-multihead-mcl-sa-7055-v1-mkmlizer: ║                                                                     ║
rirv938-multihead-mcl-sa-7055-v1-mkmlizer: ╚═════════════════════════════════════════════════════════════════════╝
Connection pool is full, discarding connection: %s. Connection pool size: %s
rirv938-multihead-mcl-sa-7055-v1-mkmlizer: Downloaded to shared memory in 128.582s
rirv938-multihead-mcl-sa-7055-v1-mkmlizer: quantizing model to /dev/shm/model_cache, profile:t0, folder:/tmp/tmpur76ifm7, device:0
rirv938-multihead-mcl-sa-7055-v1-mkmlizer: Saving flywheel model at /dev/shm/model_cache
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
rirv938-multihead-mcl-sa-7055-v1-mkmlizer: quantized model in 89.816s
rirv938-multihead-mcl-sa-7055-v1-mkmlizer: Processed model rirv938/multihead_mcl_samples_26m in 218.398s
rirv938-multihead-mcl-sa-7055-v1-mkmlizer: creating bucket guanaco-mkml-models
rirv938-multihead-mcl-sa-7055-v1-mkmlizer: Bucket 's3://guanaco-mkml-models/' created
rirv938-multihead-mcl-sa-7055-v1-mkmlizer: uploading /dev/shm/model_cache to s3://guanaco-mkml-models/rirv938-multihead-mcl-sa-7055-v1
rirv938-multihead-mcl-sa-7055-v1-mkmlizer: cp /dev/shm/model_cache/special_tokens_map.json s3://guanaco-mkml-models/rirv938-multihead-mcl-sa-7055-v1/special_tokens_map.json
rirv938-multihead-mcl-sa-7055-v1-mkmlizer: cp /dev/shm/model_cache/tokenizer_config.json s3://guanaco-mkml-models/rirv938-multihead-mcl-sa-7055-v1/tokenizer_config.json
rirv938-multihead-mcl-sa-7055-v1-mkmlizer: cp /dev/shm/model_cache/config.json s3://guanaco-mkml-models/rirv938-multihead-mcl-sa-7055-v1/config.json
rirv938-multihead-mcl-sa-7055-v1-mkmlizer: cp /dev/shm/model_cache/tokenizer.json s3://guanaco-mkml-models/rirv938-multihead-mcl-sa-7055-v1/tokenizer.json
rirv938-multihead-mcl-sa-7055-v1-mkmlizer: 
Loading 0:   0%|          | 0/291 [00:00<?, ?it/s]
Loading 0:   1%|          | 3/291 [00:00<00:59,  4.84it/s]
Loading 0:   1%|▏         | 4/291 [00:01<01:36,  2.97it/s]
Loading 0:   2%|▏         | 5/291 [00:01<02:09,  2.21it/s]
Loading 0:   3%|▎         | 8/291 [00:02<01:06,  4.24it/s]
Loading 0:   3%|▎         | 9/291 [00:02<01:05,  4.32it/s]
Loading 0:   3%|▎         | 10/291 [00:02<00:57,  4.92it/s]
Loading 0:   4%|▍         | 12/291 [00:03<01:08,  4.10it/s]
Loading 0:   4%|▍         | 13/291 [00:03<01:30,  3.08it/s]
Loading 0:   5%|▍         | 14/291 [00:04<01:55,  2.40it/s]
Loading 0:   6%|▌         | 17/291 [00:04<01:06,  4.12it/s]
Loading 0:   6%|▌         | 18/291 [00:04<01:02,  4.35it/s]
Loading 0:   7%|▋         | 19/291 [00:04<00:55,  4.93it/s]
Loading 0:   7%|▋         | 21/291 [00:05<01:05,  4.10it/s]
Loading 0:   8%|▊         | 22/291 [00:06<01:26,  3.11it/s]
Loading 0:   8%|▊         | 23/291 [00:06<01:48,  2.47it/s]
Loading 0:   9%|▉         | 26/291 [00:07<01:02,  4.21it/s]
Loading 0:   9%|▉         | 27/291 [00:07<00:59,  4.43it/s]
Loading 0:  10%|▉         | 28/291 [00:07<00:52,  5.01it/s]
Loading 0:  10%|█         | 30/291 [00:07<00:42,  6.07it/s]
Loading 0:  11%|█         | 31/291 [00:07<00:43,  5.99it/s]
Loading 0:  11%|█         | 32/291 [00:07<00:40,  6.44it/s]
Loading 0:  11%|█▏        | 33/291 [00:08<00:45,  5.62it/s]
Loading 0:  12%|█▏        | 34/291 [00:08<01:14,  3.44it/s]
Loading 0:  12%|█▏        | 35/291 [00:09<01:36,  2.64it/s]
Loading 0:  12%|█▏        | 36/291 [00:09<01:59,  2.14it/s]
Loading 0:  13%|█▎        | 39/291 [00:10<01:21,  3.09it/s]
Loading 0:  14%|█▎        | 40/291 [00:11<01:35,  2.63it/s]
Loading 0:  14%|█▍        | 41/291 [00:11<01:52,  2.23it/s]
Loading 0:  15%|█▌        | 44/291 [00:12<01:04,  3.85it/s]
Loading 0:  15%|█▌        | 45/291 [00:12<00:59,  4.10it/s]
Loading 0:  16%|█▌        | 46/291 [00:12<00:53,  4.61it/s]
Loading 0:  16%|█▋        | 48/291 [00:12<01:00,  4.00it/s]
Loading 0:  17%|█▋        | 49/291 [00:13<01:18,  3.07it/s]
Loading 0:  17%|█▋        | 50/291 [00:14<01:37,  2.46it/s]
Loading 0:  18%|█▊        | 53/291 [00:14<00:56,  4.19it/s]
Loading 0:  19%|█▊        | 54/291 [00:14<00:52,  4.49it/s]
Loading 0:  19%|█▉        | 55/291 [00:14<00:46,  5.07it/s]
Loading 0:  20%|█▉        | 57/291 [00:15<00:55,  4.19it/s]
Loading 0:  20%|█▉        | 58/291 [00:15<01:13,  3.15it/s]
Loading 0:  20%|██        | 59/291 [00:16<01:33,  2.49it/s]
Loading 0:  21%|██▏       | 62/291 [00:16<00:53,  4.30it/s]
Loading 0:  22%|██▏       | 63/291 [00:16<00:50,  4.51it/s]
Loading 0:  22%|██▏       | 64/291 [00:17<00:44,  5.09it/s]
Loading 0:  23%|██▎       | 66/291 [00:17<00:53,  4.23it/s]
Loading 0:  23%|██▎       | 67/291 [00:18<01:10,  3.17it/s]
Loading 0:  23%|██▎       | 68/291 [00:18<01:29,  2.49it/s]
Loading 0:  24%|██▍       | 71/291 [00:19<00:51,  4.28it/s]
Loading 0:  25%|██▍       | 72/291 [00:19<00:48,  4.50it/s]
Loading 0:  25%|██▌       | 73/291 [00:19<00:43,  5.07it/s]
Loading 0:  25%|██▌       | 74/291 [00:20<01:03,  3.41it/s]
Loading 0:  26%|██▌       | 75/291 [00:20<01:23,  2.58it/s]
Loading 0:  26%|██▋       | 77/291 [00:20<00:57,  3.70it/s]
Loading 0:  27%|██▋       | 78/291 [00:21<00:53,  4.01it/s]
Loading 0:  27%|██▋       | 79/291 [00:21<00:46,  4.57it/s]
Loading 0:  27%|██▋       | 80/291 [00:21<00:45,  4.65it/s]
Loading 0:  28%|██▊       | 81/291 [00:22<01:07,  3.12it/s]
Loading 0:  28%|██▊       | 82/291 [00:22<01:23,  2.50it/s]
Loading 0:  29%|██▊       | 83/291 [00:23<01:39,  2.10it/s]
Loading 0:  30%|██▉       | 86/291 [00:23<00:51,  3.98it/s]
Loading 0:  30%|██▉       | 87/291 [00:23<00:47,  4.32it/s]
Loading 0:  30%|███       | 88/291 [00:23<00:41,  4.93it/s]
Loading 0:  31%|███       | 90/291 [00:24<00:48,  4.11it/s]
Loading 0:  31%|███▏      | 91/291 [00:25<01:04,  3.11it/s]
Loading 0:  32%|███▏      | 92/291 [00:25<01:20,  2.46it/s]
Loading 0:  33%|███▎      | 95/291 [00:25<00:46,  4.23it/s]
Loading 0:  33%|███▎      | 96/291 [00:26<00:43,  4.45it/s]
Loading 0:  33%|███▎      | 97/291 [00:26<00:39,  4.97it/s]
Loading 0:  34%|███▍      | 99/291 [00:26<00:46,  4.16it/s]
Loading 0:  34%|███▍      | 100/291 [00:27<01:00,  3.15it/s]
Loading 0:  35%|███▍      | 101/291 [00:28<01:15,  2.50it/s]
Loading 0:  36%|███▌      | 104/291 [00:28<00:43,  4.25it/s]
Loading 0:  36%|███▌      | 105/291 [00:28<00:41,  4.46it/s]
Loading 0:  36%|███▋      | 106/291 [00:28<00:37,  4.95it/s]
Loading 0:  37%|███▋      | 108/291 [00:29<00:44,  4.16it/s]
Loading 0:  37%|███▋      | 109/291 [00:29<00:57,  3.14it/s]
Loading 0:  38%|███▊      | 110/291 [00:30<01:12,  2.50it/s]
Loading 0:  39%|███▉      | 113/291 [00:30<00:41,  4.31it/s]
Loading 0:  39%|███▉      | 114/291 [00:30<00:39,  4.53it/s]
Loading 0:  40%|███▉      | 115/291 [00:30<00:34,  5.06it/s]
Loading 0:  40%|███▉      | 116/291 [00:31<00:51,  3.40it/s]
Loading 0:  41%|████      | 118/291 [00:31<00:37,  4.57it/s]
Loading 0:  41%|████      | 119/291 [00:31<00:35,  4.79it/s]
Loading 0:  41%|████      | 120/291 [00:32<00:31,  5.36it/s]
Loading 0:  42%|████▏     | 122/291 [00:32<00:39,  4.27it/s]
Loading 0:  43%|████▎     | 125/291 [00:33<00:36,  4.60it/s]
Loading 0:  43%|████▎     | 126/291 [00:33<00:47,  3.49it/s]
Loading 0:  44%|████▎     | 127/291 [00:34<01:00,  2.73it/s]
Loading 0:  45%|████▍     | 130/291 [00:34<00:36,  4.44it/s]
Loading 0:  45%|████▌     | 131/291 [00:34<00:34,  4.70it/s]
Loading 0:  45%|████▌     | 132/291 [00:34<00:30,  5.25it/s]
Loading 0:  46%|████▌     | 133/291 [00:35<00:30,  5.21it/s]
Loading 0:  46%|████▌     | 134/291 [00:35<00:45,  3.42it/s]
Loading 0:  46%|████▋     | 135/291 [00:36<01:00,  2.56it/s]
Loading 0:  47%|████▋     | 138/291 [00:37<00:44,  3.43it/s]
Loading 0:  48%|████▊     | 139/291 [00:37<00:53,  2.84it/s]
Loading 0:  48%|████▊     | 140/291 [00:38<01:04,  2.36it/s]
Loading 0:  49%|████▉     | 143/291 [00:38<00:36,  4.02it/s]
Loading 0:  49%|████▉     | 144/291 [00:38<00:34,  4.27it/s]
Loading 0:  50%|████▉     | 145/291 [00:38<00:30,  4.83it/s]
Loading 0:  51%|█████     | 147/291 [00:39<00:34,  4.12it/s]
Loading 0:  51%|█████     | 148/291 [00:39<00:45,  3.16it/s]
Loading 0:  51%|█████     | 149/291 [00:40<00:56,  2.51it/s]
Loading 0:  52%|█████▏    | 152/291 [00:40<00:32,  4.27it/s]
Loading 0:  53%|█████▎    | 153/291 [00:41<00:30,  4.51it/s]
Loading 0:  53%|█████▎    | 154/291 [00:41<00:27,  5.06it/s]
Loading 0:  54%|█████▎    | 156/291 [00:41<00:32,  4.21it/s]
Loading 0:  54%|█████▍    | 157/291 [00:42<00:42,  3.18it/s]
Loading 0:  54%|█████▍    | 158/291 [00:43<00:52,  2.53it/s]
Loading 0:  55%|█████▌    | 161/291 [00:43<00:30,  4.29it/s]
Loading 0:  56%|█████▌    | 162/291 [00:43<00:28,  4.53it/s]
Loading 0:  56%|█████▌    | 163/291 [00:43<00:25,  5.05it/s]
Loading 0:  57%|█████▋    | 165/291 [00:44<00:29,  4.21it/s]
Loading 0:  57%|█████▋    | 166/291 [00:44<00:39,  3.17it/s]
Loading 0:  57%|█████▋    | 167/291 [00:45<00:49,  2.49it/s]
Loading 0:  58%|█████▊    | 170/291 [00:45<00:28,  4.24it/s]
Loading 0:  59%|█████▉    | 171/291 [00:45<00:26,  4.47it/s]
Loading 0:  59%|█████▉    | 172/291 [00:45<00:23,  5.03it/s]
Loading 0:  60%|█████▉    | 174/291 [00:46<00:27,  4.21it/s]
Loading 0:  60%|██████    | 175/291 [00:47<00:36,  3.18it/s]
Loading 0:  60%|██████    | 176/291 [00:47<00:45,  2.51it/s]
Loading 0:  62%|██████▏   | 179/291 [00:47<00:26,  4.27it/s]
Loading 0:  62%|██████▏   | 180/291 [00:48<00:24,  4.49it/s]
Loading 0:  62%|██████▏   | 181/291 [00:48<00:21,  5.04it/s]
Loading 0:  63%|██████▎   | 183/291 [00:48<00:17,  6.11it/s]
Loading 0:  63%|██████▎   | 184/291 [00:48<00:17,  6.04it/s]
Loading 0:  64%|██████▎   | 185/291 [00:48<00:16,  6.51it/s]
Loading 0:  64%|██████▍   | 186/291 [00:48<00:17,  5.98it/s]
Loading 0:  64%|██████▍   | 187/291 [00:49<00:29,  3.57it/s]
Loading 0:  65%|██████▍   | 188/291 [00:50<00:37,  2.72it/s]
Loading 0:  65%|██████▍   | 189/291 [00:50<00:45,  2.22it/s]
Loading 0:  66%|██████▌   | 192/291 [00:51<00:30,  3.20it/s]
Loading 0:  66%|██████▋   | 193/291 [00:52<00:36,  2.70it/s]
Loading 0:  67%|██████▋   | 194/291 [00:52<00:42,  2.27it/s]
Loading 0:  68%|██████▊   | 197/291 [00:52<00:24,  3.87it/s]
Loading 0:  68%|██████▊   | 198/291 [00:53<00:22,  4.14it/s]
Loading 0:  68%|██████▊   | 199/291 [00:53<00:19,  4.67it/s]
Loading 0:  69%|██████▉   | 201/291 [00:53<00:22,  4.05it/s]
Loading 0:  69%|██████▉   | 202/291 [00:54<00:28,  3.12it/s]
Loading 0:  70%|██████▉   | 203/291 [00:55<00:35,  2.48it/s]
Loading 0:  71%|███████   | 206/291 [00:55<00:20,  4.21it/s]
Loading 0:  71%|███████   | 207/291 [00:55<00:18,  4.45it/s]
Loading 0:  71%|███████▏  | 208/291 [00:55<00:16,  4.99it/s]
Loading 0:  72%|███████▏  | 210/291 [00:56<00:19,  4.19it/s]
Loading 0:  73%|███████▎  | 211/291 [00:56<00:25,  3.17it/s]
Loading 0:  73%|███████▎  | 212/291 [00:57<00:31,  2.52it/s]
Loading 0:  74%|███████▍  | 215/291 [00:57<00:17,  4.29it/s]
Loading 0:  74%|███████▍  | 216/291 [00:57<00:16,  4.52it/s]
Loading 0:  75%|███████▍  | 217/291 [00:57<00:14,  5.10it/s]
Loading 0:  75%|███████▌  | 219/291 [00:58<00:17,  4.23it/s]
Loading 0:  76%|███████▌  | 220/291 [00:59<00:22,  3.20it/s]
Loading 0:  76%|███████▌  | 221/291 [00:59<00:27,  2.53it/s]
Loading 0:  77%|███████▋  | 224/291 [00:59<00:15,  4.37it/s]
Loading 0:  77%|███████▋  | 225/291 [01:00<00:14,  4.59it/s]
Loading 0:  78%|███████▊  | 226/291 [01:00<00:12,  5.12it/s]
Loading 0:  78%|███████▊  | 227/291 [01:00<00:18,  3.45it/s]
Loading 0:  78%|███████▊  | 228/291 [01:01<00:24,  2.60it/s]
Loading 0:  79%|███████▉  | 230/291 [01:01<00:16,  3.78it/s]
Loading 0:  79%|███████▉  | 231/291 [01:01<00:14,  4.12it/s]
Loading 0:  80%|███████▉  | 232/291 [01:01<00:12,  4.79it/s]
Loading 0:  80%|████████  | 233/291 [01:02<00:11,  4.95it/s]
Loading 0:  80%|████████  | 234/291 [01:02<00:17,  3.25it/s]
Loading 0:  81%|████████▏ | 237/291 [01:03<00:13,  4.01it/s]
Loading 0:  82%|████████▏ | 238/291 [01:03<00:16,  3.14it/s]
Loading 0:  82%|████████▏ | 239/291 [01:04<00:20,  2.52it/s]
Loading 0:  83%|████████▎ | 242/291 [01:04<00:11,  4.23it/s]
Loading 0:  84%|████████▎ | 243/291 [01:04<00:10,  4.46it/s]
Loading 0:  84%|████████▍ | 244/291 [01:05<00:09,  4.97it/s]
Loading 0:  85%|████████▍ | 246/291 [01:05<00:10,  4.20it/s]
Loading 0:  85%|████████▍ | 247/291 [01:06<00:13,  3.19it/s]
Loading 0:  85%|████████▌ | 248/291 [01:06<00:17,  2.52it/s]
Loading 0:  86%|████████▋ | 251/291 [01:07<00:09,  4.27it/s]
Loading 0:  87%|████████▋ | 252/291 [01:07<00:08,  4.50it/s]
Loading 0:  87%|████████▋ | 253/291 [01:07<00:07,  5.01it/s]
Loading 0:  88%|████████▊ | 255/291 [01:08<00:08,  4.21it/s]
Loading 0:  88%|████████▊ | 256/291 [01:08<00:10,  3.19it/s]
Loading 0:  88%|████████▊ | 257/291 [01:09<00:13,  2.54it/s]
Loading 0:  89%|████████▉ | 260/291 [01:09<00:07,  4.29it/s]
Loading 0:  90%|████████▉ | 261/291 [01:09<00:06,  4.53it/s]
Loading 0:  90%|█████████ | 262/291 [01:09<00:05,  5.08it/s]
Loading 0:  91%|█████████ | 264/291 [01:10<00:06,  4.23it/s]
Loading 0:  91%|█████████ | 265/291 [01:10<00:08,  3.20it/s]
Loading 0:  91%|█████████▏| 266/291 [01:11<00:09,  2.53it/s]
Loading 0:  92%|█████████▏| 269/291 [01:11<00:05,  4.29it/s]
Loading 0:  93%|█████████▎| 270/291 [01:12<00:04,  4.53it/s]
Loading 0:  93%|█████████▎| 271/291 [01:12<00:03,  5.08it/s]
Loading 0:  94%|█████████▍| 273/291 [01:12<00:04,  4.24it/s]
Loading 0:  94%|█████████▍| 274/291 [01:13<00:05,  3.20it/s]
Loading 0:  95%|█████████▍| 275/291 [01:14<00:06,  2.53it/s]
Loading 0:  96%|█████████▌| 278/291 [01:14<00:03,  4.29it/s]
Loading 0:  96%|█████████▌| 279/291 [01:14<00:02,  4.52it/s]
Loading 0:  96%|█████████▌| 280/291 [01:14<00:02,  5.04it/s]
Loading 0:  97%|█████████▋| 281/291 [01:15<00:02,  3.42it/s]
Loading 0:  97%|█████████▋| 283/291 [01:15<00:01,  4.59it/s]
Loading 0:  98%|█████████▊| 284/291 [01:15<00:01,  4.82it/s]
Loading 0:  98%|█████████▊| 285/291 [01:15<00:01,  5.45it/s]
Loading 0:  98%|█████████▊| 286/291 [01:15<00:00,  5.47it/s]
Loading 0:  99%|█████████▊| 287/291 [01:16<00:01,  3.41it/s]
Loading 0:  99%|█████████▉| 288/291 [01:17<00:01,  2.53it/s]
                                                            
Job rirv938-multihead-mcl-sa-7055-v1-mkmlizer completed after 246.95s with status: succeeded
Stopping job with name rirv938-multihead-mcl-sa-7055-v1-mkmlizer
Pipeline stage MKMLizer completed in 247.49s
run pipeline stage %s
Running pipeline stage MKMLTemplater
Pipeline stage MKMLTemplater completed in 0.16s
run pipeline stage %s
Running pipeline stage MKMLDeployer
Creating inference service rirv938-multihead-mcl-sa-7055-v1
Waiting for inference service rirv938-multihead-mcl-sa-7055-v1 to be ready
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Inference service rirv938-multihead-mcl-sa-7055-v1 ready after 170.59440183639526s
Pipeline stage MKMLDeployer completed in 171.05s
run pipeline stage %s
Running pipeline stage StressChecker
Received healthy response to inference request in 3.074420213699341s
Received healthy response to inference request in 2.209084987640381s
Received healthy response to inference request in 1.9576711654663086s
Received healthy response to inference request in 4.397072792053223s
Received healthy response to inference request in 5.080556392669678s
5 requests
0 failed requests
5th percentile: 2.007953929901123
10th percentile: 2.0582366943359376
20th percentile: 2.1588022232055666
30th percentile: 2.382152032852173
40th percentile: 2.728286123275757
50th percentile: 3.074420213699341
60th percentile: 3.6034812450408933
70th percentile: 4.132542276382446
80th percentile: 4.533769512176514
90th percentile: 4.807162952423096
95th percentile: 4.943859672546386
99th percentile: 5.05321704864502
mean time: 3.343761110305786
%s, retrying in %s seconds...
Received healthy response to inference request in 3.744812488555908s
Received healthy response to inference request in 2.047011375427246s
Received healthy response to inference request in 3.4057743549346924s
Received healthy response to inference request in 3.3814303874969482s
Received healthy response to inference request in 3.8940796852111816s
5 requests
0 failed requests
5th percentile: 2.3138951778411867
10th percentile: 2.580778980255127
20th percentile: 3.1145465850830076
30th percentile: 3.386299180984497
40th percentile: 3.3960367679595946
50th percentile: 3.4057743549346924
60th percentile: 3.5413896083831786
70th percentile: 3.677004861831665
80th percentile: 3.774665927886963
90th percentile: 3.834372806549072
95th percentile: 3.864226245880127
99th percentile: 3.8881089973449705
mean time: 3.2946216583251955
Pipeline stage StressChecker completed in 36.57s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyTriggerPipeline
run_pipeline:run_in_cloud %s
starting trigger_guanaco_pipeline args=%s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
triggered trigger_guanaco_pipeline args=%s
Pipeline stage OfflineFamilyFriendlyTriggerPipeline completed in 5.25s
Shutdown handler de-registered
rirv938-multihead-mcl-sa_7055_v1 status is now deployed due to DeploymentManager action
Shutdown handler registered
run pipeline %s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyScorer
Evaluating %s Family Friendly Score with %s threads
Pipeline stage OfflineFamilyFriendlyScorer completed in 3614.20s
Shutdown handler de-registered
rirv938-multihead-mcl-sa_7055_v1 status is now inactive due to auto deactivation removed underperforming models
rirv938-multihead-mcl-sa_7055_v1 status is now torndown due to DeploymentManager action