deverdever-heavenly-tigger-v1

developer_uid: clover0103
submission_id: deverdever-heavenly-tigger-v1_v2
model_name: deverdever-heavenly-tigger-v1_v2
model_group: DeverDever/heavenly-tigg
status: torndown
timestamp: 2024-08-28T11:43:23+00:00
num_battles: 12144
num_wins: 5839
celo_rating: 1222.92
family_friendly_score: 0.0
submission_type: basic
model_repo: DeverDever/heavenly-tigger-v1
model_architecture: LlamaForCausalLM
model_num_parameters: 70553706496.0
best_of: 4
max_input_tokens: 512
max_output_tokens: 64
latencies: [{'batch_size': 1, 'throughput': 0.25305906948302603, 'latency_mean': 3.9515911257267, 'latency_p50': 3.9793012142181396, 'latency_p90': 4.348193669319153}, {'batch_size': 2, 'throughput': 0.42775525208917853, 'latency_mean': 4.654891849756241, 'latency_p50': 4.6773130893707275, 'latency_p90': 5.138329982757568}, {'batch_size': 3, 'throughput': 0.586685159365861, 'latency_mean': 5.097483319044113, 'latency_p50': 5.124902725219727, 'latency_p90': 5.698004150390625}, {'batch_size': 5, 'throughput': 0.8120805956859802, 'latency_mean': 6.133992123603821, 'latency_p50': 6.14186155796051, 'latency_p90': 6.815071702003479}]
gpu_counts: {'NVIDIA A100-SXM4-80GB': 1}
display_name: deverdever-heavenly-tigger-v1_v2
is_internal_developer: False
language_model: DeverDever/heavenly-tigger-v1
model_size: 71B
ranking_group: single
us_pacific_date: 2024-08-28
win_ratio: 0.4808135704874835
generation_params: {'temperature': 1.12, 'top_p': 1.0, 'min_p': 0.02, 'top_k': 200, 'presence_penalty': 0.0, 'frequency_penalty': 0.0, 'stopping_words': ['\n', '<|eot_id|>', '<|end_header_id|>'], 'max_input_tokens': 512, 'best_of': 4, 'max_output_tokens': 64}
formatter: {'memory_template': "<|begin_of_text|><|start_header_id|>system<|end_header_id|>\n\n{bot_name}'s Persona: {memory}\n\n", 'prompt_template': '{prompt}<|eot_id|>', 'bot_template': '<|start_header_id|>assistant<|end_header_id|>\n\n{bot_name}: {message}<|eot_id|>', 'user_template': '<|start_header_id|>user<|end_header_id|>\n\n{user_name}: {message}<|eot_id|>', 'response_template': '<|start_header_id|>assistant<|end_header_id|>\n\n{bot_name}:', 'truncate_by_message': False}
Resubmit model
Running pipeline stage MKMLizer
Starting job with name deverdever-heavenly-tigger-v1-v2-mkmlizer
Waiting for job on deverdever-heavenly-tigger-v1-v2-mkmlizer to finish
Stopping job with name deverdever-heavenly-tigger-v1-v2-mkmlizer
%s, retrying in %s seconds...
Starting job with name deverdever-heavenly-tigger-v1-v2-mkmlizer
Waiting for job on deverdever-heavenly-tigger-v1-v2-mkmlizer to finish
deverdever-heavenly-tigger-v1-v2-mkmlizer: ╔═════════════════════════════════════════════════════════════════════╗
deverdever-heavenly-tigger-v1-v2-mkmlizer: ║     _____            __           __                                ║
deverdever-heavenly-tigger-v1-v2-mkmlizer: ║    / _/ /_ ___    __/ /  ___ ___ / /                                ║
deverdever-heavenly-tigger-v1-v2-mkmlizer: ║   / _/ / // / |/|/ / _ \/ -_) -_) /                                 ║
deverdever-heavenly-tigger-v1-v2-mkmlizer: ║  /_//_/\_, /|__,__/_//_/\__/\__/_/                                  ║
deverdever-heavenly-tigger-v1-v2-mkmlizer: ║       /___/                                                         ║
deverdever-heavenly-tigger-v1-v2-mkmlizer: ║                                                                     ║
deverdever-heavenly-tigger-v1-v2-mkmlizer: ║  Version: 0.10.1                                                    ║
deverdever-heavenly-tigger-v1-v2-mkmlizer: ║  Copyright 2023 MK ONE TECHNOLOGIES Inc.                            ║
deverdever-heavenly-tigger-v1-v2-mkmlizer: ║  https://mk1.ai                                                     ║
deverdever-heavenly-tigger-v1-v2-mkmlizer: ║                                                                     ║
deverdever-heavenly-tigger-v1-v2-mkmlizer: ║  The license key for the current software has been verified as      ║
deverdever-heavenly-tigger-v1-v2-mkmlizer: ║  belonging to:                                                      ║
deverdever-heavenly-tigger-v1-v2-mkmlizer: ║                                                                     ║
deverdever-heavenly-tigger-v1-v2-mkmlizer: ║  Chai Research Corp.                                                ║
deverdever-heavenly-tigger-v1-v2-mkmlizer: ║  Account ID: 7997a29f-0ceb-4cc7-9adf-840c57b4ae6f                   ║
deverdever-heavenly-tigger-v1-v2-mkmlizer: ║  Expiration: 2024-10-15 23:59:59                                    ║
deverdever-heavenly-tigger-v1-v2-mkmlizer: ║                                                                     ║
deverdever-heavenly-tigger-v1-v2-mkmlizer: ╚═════════════════════════════════════════════════════════════════════╝
Failed to get response for submission blend_berib_2024-08-16: ('http://zonemercy-lexical-nemo-1518-v18-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'read tcp 127.0.0.1:42264->127.0.0.1:8080: read: connection reset by peer\n')
deverdever-heavenly-tigger-v1-v2-mkmlizer: Downloaded to shared memory in 321.884s
deverdever-heavenly-tigger-v1-v2-mkmlizer: quantizing model to /dev/shm/model_cache, profile:s0, folder:/tmp/tmpgghzgiw4, device:0
deverdever-heavenly-tigger-v1-v2-mkmlizer: Saving flywheel model at /dev/shm/model_cache
deverdever-heavenly-tigger-v1-v2-mkmlizer: quantized model in 170.334s
deverdever-heavenly-tigger-v1-v2-mkmlizer: Processed model DeverDever/heavenly-tigger-v1 in 492.219s
deverdever-heavenly-tigger-v1-v2-mkmlizer: creating bucket guanaco-mkml-models
deverdever-heavenly-tigger-v1-v2-mkmlizer: Bucket 's3://guanaco-mkml-models/' created
deverdever-heavenly-tigger-v1-v2-mkmlizer: uploading /dev/shm/model_cache to s3://guanaco-mkml-models/deverdever-heavenly-tigger-v1-v2
deverdever-heavenly-tigger-v1-v2-mkmlizer: cp /dev/shm/model_cache/config.json s3://guanaco-mkml-models/deverdever-heavenly-tigger-v1-v2/config.json
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
deverdever-heavenly-tigger-v1-v2-mkmlizer: cp /dev/shm/model_cache/tokenizer.json s3://guanaco-mkml-models/deverdever-heavenly-tigger-v1-v2/tokenizer.json
deverdever-heavenly-tigger-v1-v2-mkmlizer: cp /dev/shm/model_cache/flywheel_model.5.safetensors s3://guanaco-mkml-models/deverdever-heavenly-tigger-v1-v2/flywheel_model.5.safetensors
deverdever-heavenly-tigger-v1-v2-mkmlizer: cp /dev/shm/model_cache/flywheel_model.0.safetensors s3://guanaco-mkml-models/deverdever-heavenly-tigger-v1-v2/flywheel_model.0.safetensors
deverdever-heavenly-tigger-v1-v2-mkmlizer: cp /dev/shm/model_cache/flywheel_model.4.safetensors s3://guanaco-mkml-models/deverdever-heavenly-tigger-v1-v2/flywheel_model.4.safetensors
deverdever-heavenly-tigger-v1-v2-mkmlizer: cp /dev/shm/model_cache/flywheel_model.3.safetensors s3://guanaco-mkml-models/deverdever-heavenly-tigger-v1-v2/flywheel_model.3.safetensors
deverdever-heavenly-tigger-v1-v2-mkmlizer: cp /dev/shm/model_cache/flywheel_model.1.safetensors s3://guanaco-mkml-models/deverdever-heavenly-tigger-v1-v2/flywheel_model.1.safetensors
deverdever-heavenly-tigger-v1-v2-mkmlizer: cp /dev/shm/model_cache/flywheel_model.2.safetensors s3://guanaco-mkml-models/deverdever-heavenly-tigger-v1-v2/flywheel_model.2.safetensors
deverdever-heavenly-tigger-v1-v2-mkmlizer: 
Loading 0:   0%|          | 0/723 [00:00<?, ?it/s]
Loading 0:   0%|          | 3/723 [00:00<00:38, 18.82it/s]
Loading 0:   1%|          | 5/723 [00:00<00:59, 12.12it/s]
Loading 0:   2%|▏         | 11/723 [00:00<00:32, 22.19it/s]
Loading 0:   2%|▏         | 15/723 [00:00<00:26, 26.48it/s]
Loading 0:   2%|▏         | 18/723 [00:00<00:42, 16.75it/s]
Loading 0:   3%|▎         | 22/723 [00:01<00:36, 19.10it/s]
Loading 0:   3%|▎         | 25/723 [00:01<00:35, 19.71it/s]
Loading 0:   4%|▍         | 28/723 [00:01<00:32, 21.29it/s]
Loading 0:   4%|▍         | 31/723 [00:01<00:32, 21.13it/s]
Loading 0:   5%|▍         | 34/723 [00:01<00:32, 21.20it/s]
Loading 0:   5%|▌         | 37/723 [00:01<00:31, 22.03it/s]
Loading 0:   6%|▌         | 41/723 [00:01<00:26, 25.27it/s]
Loading 0:   6%|▌         | 44/723 [00:02<00:38, 17.86it/s]
Loading 0:   7%|▋         | 47/723 [00:02<00:36, 18.64it/s]
Loading 0:   7%|▋         | 50/723 [00:02<00:44, 15.24it/s]
Loading 0:   8%|▊         | 55/723 [00:02<00:32, 20.61it/s]
Loading 0:   8%|▊         | 58/723 [00:02<00:32, 20.30it/s]
Loading 0:   8%|▊         | 61/723 [00:03<00:32, 20.60it/s]
Loading 0:   9%|▉         | 64/723 [00:03<00:30, 21.80it/s]
Loading 0:   9%|▉         | 68/723 [00:03<00:34, 18.81it/s]
Loading 0:  10%|▉         | 71/723 [00:03<00:41, 15.81it/s]
Loading 0:  10%|█         | 75/723 [00:03<00:32, 19.66it/s]
Loading 0:  11%|█         | 78/723 [00:04<00:36, 17.79it/s]
Loading 0:  11%|█▏        | 82/723 [00:04<00:31, 20.63it/s]
Loading 0:  12%|█▏        | 85/723 [00:04<00:30, 20.87it/s]
Loading 0:  12%|█▏        | 88/723 [00:04<00:30, 21.08it/s]
Loading 0:  13%|█▎        | 91/723 [00:04<00:28, 22.41it/s]
Loading 0:  13%|█▎        | 94/723 [00:04<00:38, 16.43it/s]
Loading 0:  13%|█▎        | 96/723 [00:05<00:39, 16.02it/s]
Loading 0:  14%|█▍        | 100/723 [00:05<00:31, 20.06it/s]
Loading 0:  14%|█▍        | 103/723 [00:05<00:30, 20.03it/s]
Loading 0:  15%|█▍        | 106/723 [00:05<00:30, 20.10it/s]
Loading 0:  15%|█▌        | 109/723 [00:05<00:28, 21.34it/s]
Loading 0:  15%|█▌        | 112/723 [00:05<00:33, 18.09it/s]
Loading 0:  16%|█▌        | 115/723 [00:05<00:30, 19.75it/s]
Loading 0:  16%|█▋        | 118/723 [00:06<00:35, 16.82it/s]
Loading 0:  17%|█▋        | 121/723 [00:06<00:33, 18.16it/s]
Loading 0:  17%|█▋        | 123/723 [00:06<00:34, 17.40it/s]
Loading 0:  17%|█▋        | 124/723 [00:20<00:34, 17.40it/s]
Loading 0:  17%|█▋        | 125/723 [00:26<24:25,  2.45s/it]
Loading 0:  18%|█▊        | 127/723 [00:26<18:15,  1.84s/it]
Loading 0:  18%|█▊        | 129/723 [00:26<13:40,  1.38s/it]
Loading 0:  18%|█▊        | 131/723 [00:26<10:07,  1.03s/it]
Loading 0:  19%|█▉        | 137/723 [00:27<04:45,  2.05it/s]
Loading 0:  20%|█▉        | 141/723 [00:27<03:12,  3.03it/s]
Loading 0:  20%|█▉        | 144/723 [00:27<02:38,  3.66it/s]
Loading 0:  20%|██        | 148/723 [00:27<01:51,  5.15it/s]
Loading 0:  21%|██        | 151/723 [00:27<01:28,  6.43it/s]
Loading 0:  21%|██▏       | 154/723 [00:28<01:09,  8.16it/s]
Loading 0:  22%|██▏       | 157/723 [00:28<00:56,  9.94it/s]
Loading 0:  22%|██▏       | 160/723 [00:28<00:47, 11.79it/s]
Loading 0:  23%|██▎       | 163/723 [00:28<00:39, 14.02it/s]
Loading 0:  23%|██▎       | 167/723 [00:28<00:31, 17.63it/s]
Loading 0:  24%|██▎       | 170/723 [00:28<00:36, 15.01it/s]
Loading 0:  24%|██▍       | 173/723 [00:28<00:33, 16.39it/s]
Loading 0:  24%|██▍       | 176/723 [00:29<00:38, 14.16it/s]
Loading 0:  25%|██▌       | 181/723 [00:29<00:28, 19.21it/s]
Loading 0:  25%|██▌       | 184/723 [00:29<00:28, 19.20it/s]
Loading 0:  26%|██▌       | 187/723 [00:29<00:27, 19.83it/s]
Loading 0:  26%|██▋       | 190/723 [00:29<00:24, 21.68it/s]
Loading 0:  27%|██▋       | 194/723 [00:30<00:27, 19.00it/s]
Loading 0:  27%|██▋       | 197/723 [00:30<00:33, 15.70it/s]
Loading 0:  28%|██▊       | 201/723 [00:30<00:26, 19.74it/s]
Loading 0:  28%|██▊       | 204/723 [00:30<00:28, 18.36it/s]
Loading 0:  29%|██▉       | 208/723 [00:30<00:23, 21.68it/s]
Loading 0:  29%|██▉       | 211/723 [00:30<00:24, 21.07it/s]
Loading 0:  30%|██▉       | 214/723 [00:31<00:23, 21.42it/s]
Loading 0:  30%|███       | 217/723 [00:31<00:21, 23.01it/s]
Loading 0:  30%|███       | 220/723 [00:31<00:29, 17.14it/s]
Loading 0:  31%|███       | 223/723 [00:31<00:27, 17.87it/s]
Loading 0:  31%|███▏      | 226/723 [00:31<00:25, 19.26it/s]
Loading 0:  32%|███▏      | 229/723 [00:31<00:25, 19.07it/s]
Loading 0:  32%|███▏      | 232/723 [00:31<00:25, 19.51it/s]
Loading 0:  33%|███▎      | 235/723 [00:32<00:22, 21.45it/s]
Loading 0:  33%|███▎      | 238/723 [00:32<00:27, 17.86it/s]
Loading 0:  33%|███▎      | 241/723 [00:32<00:24, 19.67it/s]
Loading 0:  34%|███▎      | 244/723 [00:32<00:27, 17.16it/s]
Loading 0:  34%|███▍      | 247/723 [00:32<00:25, 18.34it/s]
Loading 0:  34%|███▍      | 249/723 [00:32<00:27, 17.46it/s]
Loading 0:  35%|███▍      | 253/723 [00:33<00:21, 21.66it/s]
Loading 0:  35%|███▌      | 256/723 [00:33<00:22, 20.95it/s]
Loading 0:  36%|███▌      | 259/723 [00:33<00:21, 21.19it/s]
Loading 0:  36%|███▌      | 262/723 [00:33<00:20, 22.85it/s]
Loading 0:  37%|███▋      | 265/723 [00:33<00:19, 23.77it/s]
Loading 0:  37%|███▋      | 268/723 [00:33<00:24, 18.85it/s]
Loading 0:  37%|███▋      | 268/723 [00:53<00:24, 18.85it/s]
Loading 0:  37%|███▋      | 269/723 [00:53<19:03,  2.52s/it]
Loading 0:  37%|███▋      | 270/723 [00:53<16:19,  2.16s/it]
Loading 0:  38%|███▊      | 273/723 [00:54<10:06,  1.35s/it]
Loading 0:  38%|███▊      | 275/723 [00:54<07:34,  1.01s/it]
Loading 0:  39%|███▊      | 280/723 [00:54<03:53,  1.90it/s]
Loading 0:  39%|███▉      | 283/723 [00:54<02:49,  2.59it/s]
Loading 0:  40%|███▉      | 286/723 [00:54<02:04,  3.51it/s]
Loading 0:  40%|███▉      | 289/723 [00:54<01:32,  4.70it/s]
Loading 0:  41%|████      | 293/723 [00:55<01:03,  6.73it/s]
Loading 0:  41%|████      | 296/723 [00:55<00:57,  7.42it/s]
Loading 0:  41%|████      | 298/723 [00:55<00:51,  8.26it/s]
Loading 0:  42%|████▏     | 301/723 [00:55<00:41, 10.07it/s]
Loading 0:  42%|████▏     | 303/723 [00:55<00:38, 10.93it/s]
Loading 0:  42%|████▏     | 307/723 [00:55<00:28, 14.74it/s]
Loading 0:  43%|████▎     | 310/723 [00:56<00:26, 15.81it/s]
Loading 0:  43%|████▎     | 313/723 [00:56<00:23, 17.14it/s]
Loading 0:  44%|████▎     | 316/723 [00:56<00:20, 19.42it/s]
Loading 0:  44%|████▍     | 319/723 [00:56<00:18, 21.28it/s]
Loading 0:  45%|████▍     | 322/723 [00:56<00:25, 15.89it/s]
Loading 0:  45%|████▍     | 325/723 [00:56<00:24, 16.42it/s]
Loading 0:  45%|████▌     | 328/723 [00:57<00:22, 17.81it/s]
Loading 0:  46%|████▌     | 331/723 [00:57<00:20, 18.71it/s]
Loading 0:  46%|████▌     | 334/723 [00:57<00:18, 20.48it/s]
Loading 0:  47%|████▋     | 337/723 [00:57<00:19, 20.01it/s]
Loading 0:  47%|████▋     | 340/723 [00:57<00:19, 19.98it/s]
Loading 0:  47%|████▋     | 343/723 [00:57<00:17, 21.69it/s]
Loading 0:  48%|████▊     | 346/723 [00:57<00:23, 16.02it/s]
Loading 0:  48%|████▊     | 348/723 [00:58<00:23, 15.86it/s]
Loading 0:  49%|████▊     | 352/723 [00:58<00:18, 19.99it/s]
Loading 0:  49%|████▉     | 355/723 [00:58<00:18, 19.66it/s]
Loading 0:  50%|████▉     | 358/723 [00:58<00:18, 20.12it/s]
Loading 0:  50%|████▉     | 361/723 [00:58<00:16, 21.54it/s]
Loading 0:  50%|█████     | 364/723 [00:58<00:20, 17.80it/s]
Loading 0:  51%|█████     | 367/723 [00:58<00:18, 19.64it/s]
Loading 0:  51%|█████     | 370/723 [00:59<00:20, 17.18it/s]
Loading 0:  52%|█████▏    | 373/723 [00:59<00:18, 18.52it/s]
Loading 0:  52%|█████▏    | 376/723 [00:59<00:17, 19.30it/s]
Loading 0:  52%|█████▏    | 379/723 [00:59<00:16, 21.26it/s]
Loading 0:  53%|█████▎    | 382/723 [00:59<00:16, 20.43it/s]
Loading 0:  53%|█████▎    | 385/723 [00:59<00:16, 20.73it/s]
Loading 0:  54%|█████▎    | 388/723 [01:00<00:14, 22.35it/s]
Loading 0:  54%|█████▍    | 391/723 [01:00<00:14, 23.42it/s]
Loading 0:  54%|█████▍    | 394/723 [01:00<00:17, 18.42it/s]
Loading 0:  55%|█████▍    | 397/723 [01:00<00:18, 17.27it/s]
Loading 0:  55%|█████▌    | 400/723 [01:00<00:18, 17.91it/s]
Loading 0:  55%|█████▌    | 400/723 [01:13<00:18, 17.91it/s]
Loading 0:  55%|█████▌    | 401/723 [01:20<13:32,  2.52s/it]
Loading 0:  56%|█████▌    | 405/723 [01:20<07:53,  1.49s/it]
Loading 0:  56%|█████▋    | 408/723 [01:20<05:31,  1.05s/it]
Loading 0:  57%|█████▋    | 410/723 [01:21<04:22,  1.19it/s]
Loading 0:  57%|█████▋    | 415/723 [01:21<02:26,  2.11it/s]
Loading 0:  58%|█████▊    | 419/723 [01:21<01:38,  3.09it/s]
Loading 0:  58%|█████▊    | 422/723 [01:21<01:19,  3.80it/s]
Loading 0:  59%|█████▉    | 425/723 [01:21<01:00,  4.93it/s]
Loading 0:  59%|█████▉    | 428/723 [01:22<00:50,  5.80it/s]
Loading 0:  60%|█████▉    | 433/723 [01:22<00:32,  8.83it/s]
Loading 0:  60%|██████    | 436/723 [01:22<00:27, 10.30it/s]
Loading 0:  61%|██████    | 439/723 [01:22<00:23, 11.96it/s]
Loading 0:  61%|██████    | 442/723 [01:22<00:19, 14.24it/s]
Loading 0:  62%|██████▏   | 446/723 [01:22<00:18, 14.96it/s]
Loading 0:  62%|██████▏   | 449/723 [01:23<00:20, 13.69it/s]
Loading 0:  63%|██████▎   | 453/723 [01:23<00:15, 17.46it/s]
Loading 0:  63%|██████▎   | 456/723 [01:23<00:15, 16.72it/s]
Loading 0:  64%|██████▎   | 460/723 [01:23<00:13, 19.69it/s]
Loading 0:  64%|██████▍   | 463/723 [01:23<00:13, 19.68it/s]
Loading 0:  64%|██████▍   | 466/723 [01:23<00:12, 20.20it/s]
Loading 0:  65%|██████▍   | 469/723 [01:23<00:11, 21.68it/s]
Loading 0:  65%|██████▌   | 472/723 [01:24<00:15, 16.09it/s]
Loading 0:  66%|██████▌   | 474/723 [01:24<00:15, 15.66it/s]
Loading 0:  66%|██████▌   | 478/723 [01:24<00:12, 19.20it/s]
Loading 0:  67%|██████▋   | 481/723 [01:24<00:12, 19.28it/s]
Loading 0:  67%|██████▋   | 484/723 [01:24<00:12, 19.40it/s]
Loading 0:  67%|██████▋   | 487/723 [01:24<00:11, 20.53it/s]
Loading 0:  68%|██████▊   | 490/723 [01:25<00:13, 17.19it/s]
Loading 0:  68%|██████▊   | 493/723 [01:25<00:12, 19.05it/s]
Loading 0:  69%|██████▊   | 496/723 [01:25<00:13, 17.30it/s]
Loading 0:  69%|██████▉   | 499/723 [01:25<00:12, 18.46it/s]
Loading 0:  69%|██████▉   | 501/723 [01:25<00:12, 17.54it/s]
Loading 0:  70%|██████▉   | 505/723 [01:25<00:09, 21.82it/s]
Loading 0:  70%|███████   | 508/723 [01:26<00:10, 21.25it/s]
Loading 0:  71%|███████   | 511/723 [01:26<00:09, 21.52it/s]
Loading 0:  71%|███████   | 514/723 [01:26<00:09, 23.07it/s]
Loading 0:  72%|███████▏  | 517/723 [01:26<00:08, 24.20it/s]
Loading 0:  72%|███████▏  | 520/723 [01:26<00:10, 19.44it/s]
Loading 0:  72%|███████▏  | 523/723 [01:26<00:11, 17.88it/s]
Loading 0:  73%|███████▎  | 526/723 [01:27<00:10, 18.16it/s]
Loading 0:  73%|███████▎  | 528/723 [01:27<00:11, 17.37it/s]
Loading 0:  74%|███████▎  | 532/723 [01:27<00:08, 21.22it/s]
Loading 0:  74%|███████▍  | 535/723 [01:27<00:09, 20.47it/s]
Loading 0:  74%|███████▍  | 535/723 [01:46<00:09, 20.47it/s]
Loading 0:  74%|███████▍  | 536/723 [01:47<07:37,  2.45s/it]
Loading 0:  75%|███████▍  | 540/723 [01:47<04:26,  1.45s/it]
Loading 0:  75%|███████▌  | 543/723 [01:47<03:04,  1.02s/it]
Loading 0:  76%|███████▌  | 546/723 [01:47<02:10,  1.35it/s]
Loading 0:  76%|███████▌  | 548/723 [01:47<01:43,  1.70it/s]
Loading 0:  76%|███████▌  | 550/723 [01:47<01:19,  2.16it/s]
Loading 0:  76%|███████▋  | 553/723 [01:48<00:54,  3.11it/s]
Loading 0:  77%|███████▋  | 555/723 [01:48<00:43,  3.86it/s]
Loading 0:  77%|███████▋  | 559/723 [01:48<00:27,  6.07it/s]
Loading 0:  78%|███████▊  | 562/723 [01:48<00:21,  7.66it/s]
Loading 0:  78%|███████▊  | 565/723 [01:48<00:16,  9.47it/s]
Loading 0:  79%|███████▊  | 568/723 [01:48<00:13, 11.70it/s]
Loading 0:  79%|███████▉  | 571/723 [01:48<00:10, 14.26it/s]
Loading 0:  79%|███████▉  | 574/723 [01:49<00:11, 12.43it/s]
Loading 0:  80%|███████▉  | 576/723 [01:49<00:11, 12.88it/s]
Loading 0:  80%|████████  | 579/723 [01:49<00:09, 15.66it/s]
Loading 0:  80%|████████  | 582/723 [01:49<00:09, 15.37it/s]
Loading 0:  81%|████████  | 586/723 [01:49<00:07, 19.11it/s]
Loading 0:  81%|████████▏ | 589/723 [01:49<00:07, 19.04it/s]
Loading 0:  82%|████████▏ | 592/723 [01:49<00:06, 19.62it/s]
Loading 0:  82%|████████▏ | 595/723 [01:50<00:05, 21.49it/s]
Loading 0:  83%|████████▎ | 598/723 [01:50<00:07, 15.99it/s]
Loading 0:  83%|████████▎ | 600/723 [01:50<00:07, 15.83it/s]
Loading 0:  84%|████████▎ | 604/723 [01:50<00:05, 19.97it/s]
Loading 0:  84%|████████▍ | 607/723 [01:50<00:05, 20.19it/s]
Loading 0:  84%|████████▍ | 610/723 [01:50<00:05, 20.77it/s]
Loading 0:  85%|████████▍ | 613/723 [01:51<00:04, 22.64it/s]
Loading 0:  85%|████████▌ | 616/723 [01:51<00:05, 18.69it/s]
Loading 0:  86%|████████▌ | 619/723 [01:51<00:05, 20.45it/s]
Loading 0:  86%|████████▌ | 622/723 [01:51<00:05, 18.14it/s]
Loading 0:  86%|████████▋ | 625/723 [01:51<00:05, 19.12it/s]
Loading 0:  87%|████████▋ | 628/723 [01:51<00:04, 19.21it/s]
Loading 0:  87%|████████▋ | 631/723 [01:51<00:04, 20.87it/s]
Loading 0:  88%|████████▊ | 634/723 [01:52<00:04, 20.39it/s]
Loading 0:  88%|████████▊ | 637/723 [01:52<00:04, 20.50it/s]
Loading 0:  89%|████████▊ | 640/723 [01:52<00:03, 21.18it/s]
Loading 0:  89%|████████▉ | 643/723 [01:52<00:03, 22.43it/s]
Loading 0:  89%|████████▉ | 646/723 [01:52<00:04, 17.84it/s]
Loading 0:  90%|████████▉ | 648/723 [01:52<00:04, 15.04it/s]
Loading 0:  90%|█████████ | 652/723 [01:53<00:03, 18.03it/s]
Loading 0:  90%|█████████ | 654/723 [01:53<00:03, 17.33it/s]
Loading 0:  91%|█████████ | 658/723 [01:53<00:03, 20.96it/s]
Loading 0:  91%|█████████▏| 661/723 [01:53<00:03, 20.40it/s]
Loading 0:  92%|█████████▏| 664/723 [01:53<00:02, 20.57it/s]
Loading 0:  92%|█████████▏| 667/723 [01:53<00:02, 21.82it/s]
Loading 0:  93%|█████████▎| 671/723 [01:53<00:02, 23.91it/s]
Loading 0:  93%|█████████▎| 674/723 [01:54<00:02, 17.06it/s]
Loading 0:  93%|█████████▎| 674/723 [02:06<00:02, 17.06it/s]
Loading 0:  93%|█████████▎| 675/723 [02:14<01:54,  2.38s/it]
Loading 0:  94%|█████████▍| 678/723 [02:14<01:12,  1.62s/it]
Loading 0:  94%|█████████▍| 680/723 [02:14<00:53,  1.26s/it]
Loading 0:  95%|█████████▍| 685/723 [02:14<00:26,  1.46it/s]
Loading 0:  95%|█████████▌| 688/723 [02:14<00:17,  1.98it/s]
Loading 0:  96%|█████████▌| 691/723 [02:15<00:11,  2.67it/s]
Loading 0:  96%|█████████▌| 694/723 [02:15<00:07,  3.63it/s]
Loading 0:  96%|█████████▋| 697/723 [02:15<00:05,  4.86it/s]
Loading 0:  97%|█████████▋| 700/723 [02:15<00:04,  5.67it/s]
Loading 0:  97%|█████████▋| 702/723 [02:15<00:03,  6.56it/s]
Loading 0:  98%|█████████▊| 705/723 [02:15<00:02,  8.69it/s]
Loading 0:  98%|█████████▊| 708/723 [02:16<00:01,  9.96it/s]
Loading 0:  98%|█████████▊| 712/723 [02:16<00:00, 13.38it/s]
Loading 0:  99%|█████████▉| 715/723 [02:16<00:00, 14.58it/s]
Loading 0:  99%|█████████▉| 718/723 [02:16<00:00, 15.82it/s]
Loading 0: 100%|█████████▉| 721/723 [02:16<00:00, 17.77it/s]
Loading 0: 100%|█████████▉| 722/723 [02:27<00:00, 17.77it/s]
Loading 0: 100%|██████████| 723/723 [02:27<00:00,  1.28s/it]
                                                            
Job deverdever-heavenly-tigger-v1-v2-mkmlizer completed after 544.18s with status: succeeded
Stopping job with name deverdever-heavenly-tigger-v1-v2-mkmlizer
Pipeline stage MKMLizer completed in 546.06s
Running pipeline stage MKMLKubeTemplater
Pipeline stage MKMLKubeTemplater completed in 0.08s
Running pipeline stage ISVCDeployer
Creating inference service deverdever-heavenly-tigger-v1-v2
Waiting for inference service deverdever-heavenly-tigger-v1-v2 to be ready
Inference service deverdever-heavenly-tigger-v1-v2 ready after 170.8564817905426s
Pipeline stage ISVCDeployer completed in 172.63s
Running pipeline stage StressChecker
Received healthy response to inference request in 5.536344766616821s
Received healthy response to inference request in 5.299520015716553s
Received healthy response to inference request in 4.273136377334595s
Received healthy response to inference request in 3.399688720703125s
Received healthy response to inference request in 4.315879821777344s
5 requests
0 failed requests
5th percentile: 3.574378252029419
10th percentile: 3.749067783355713
20th percentile: 4.098446846008301
30th percentile: 4.281685066223145
40th percentile: 4.298782444000244
50th percentile: 4.315879821777344
60th percentile: 4.709335899353027
70th percentile: 5.102791976928711
80th percentile: 5.3468849658966064
90th percentile: 5.441614866256714
95th percentile: 5.488979816436768
99th percentile: 5.52687177658081
mean time: 4.564913940429688
%s, retrying in %s seconds...
Received healthy response to inference request in 3.7984044551849365s
Received healthy response to inference request in 4.559218406677246s
Received healthy response to inference request in 4.491179704666138s
Received healthy response to inference request in 4.268683433532715s
Received healthy response to inference request in 4.352477073669434s
5 requests
0 failed requests
5th percentile: 3.892460250854492
10th percentile: 3.9865160465240477
20th percentile: 4.174627637863159
30th percentile: 4.285442161560058
40th percentile: 4.3189596176147464
50th percentile: 4.352477073669434
60th percentile: 4.407958126068115
70th percentile: 4.463439178466797
80th percentile: 4.504787445068359
90th percentile: 4.532002925872803
95th percentile: 4.545610666275024
99th percentile: 4.556496858596802
mean time: 4.293992614746093
Pipeline stage StressChecker completed in 47.04s
deverdever-heavenly-tigger-v1_v2 status is now deployed due to DeploymentManager action
deverdever-heavenly-tigger-v1_v2 status is now inactive due to auto deactivation removed underperforming models
Shutdown handler registered
run pipeline %s
run pipeline stage %s
Running pipeline stage MKMLProfilerDeleter
Skipping teardown as no inference service was successfully deployed
Pipeline stage MKMLProfilerDeleter completed in 0.15s
run pipeline stage %s
Running pipeline stage MKMLProfilerTemplater
Pipeline stage MKMLProfilerTemplater completed in 0.15s
run pipeline stage %s
Running pipeline stage MKMLProfilerDeployer
Creating inference service deverdever-heavenly-tigger-v1-v2-profiler
Waiting for inference service deverdever-heavenly-tigger-v1-v2-profiler to be ready
Inference service deverdever-heavenly-tigger-v1-v2-profiler ready after 240.5307891368866s
Pipeline stage MKMLProfilerDeployer completed in 240.95s
run pipeline stage %s
Running pipeline stage MKMLProfilerRunner
kubectl cp /code/guanaco/guanaco_inference_services/src/inference_scripts tenant-chaiml-guanaco/deverdever-heavenly-a21efceb1fb1cc2961a5282882e5159c-deploqtxf7:/code/chaiverse_profiler_1725514945 --namespace tenant-chaiml-guanaco
kubectl exec -it deverdever-heavenly-a21efceb1fb1cc2961a5282882e5159c-deploqtxf7 --namespace tenant-chaiml-guanaco -- sh -c 'cd /code/chaiverse_profiler_1725514945 && python profiles.py profile --best_of_n 4 --auto_batch 5 --batches 1,5,10,15,20,25,30,35,40,45,50,55,60,65,70,75,80,85,90,95,100,105,110,115,120,125,130,135,140,145,150,155,160,165,170,175,180,185,190,195 --samples 200 --input_tokens 512 --output_tokens 64 --summary /code/chaiverse_profiler_1725514945/summary.json'
kubectl exec -it deverdever-heavenly-a21efceb1fb1cc2961a5282882e5159c-deploqtxf7 --namespace tenant-chaiml-guanaco -- bash -c 'cat /code/chaiverse_profiler_1725514945/summary.json'
Pipeline stage MKMLProfilerRunner completed in 1852.70s
run pipeline stage %s
Running pipeline stage MKMLProfilerDeleter
Checking if service deverdever-heavenly-tigger-v1-v2-profiler is running
Tearing down inference service deverdever-heavenly-tigger-v1-v2-profiler
Service deverdever-heavenly-tigger-v1-v2-profiler has been torndown
Pipeline stage MKMLProfilerDeleter completed in 1.74s
Shutdown handler de-registered
deverdever-heavenly-tigger-v1_v2 status is now torndown due to DeploymentManager action
deverdever-heavenly-tigger-v1_v2 status is now torndown due to DeploymentManager action