zonemercy-vingt-deux-v2-1e5

submission_id: zonemercy-vingt-deux-v2-1e5_v1
developer_uid: chai_backend_admin
best_of: 8
celo_rating: 1251.97
display_name: temp-4
family_friendly_score: 0.0
formatter: {'memory_template': "{bot_name}'s Persona: {memory}\n####\n", 'prompt_template': '{prompt}\n<START>\n', 'bot_template': '{bot_name}: {message}\n', 'user_template': '{user_name}: {message}\n', 'response_template': '{bot_name}:', 'truncate_by_message': False}
generation_params: {'temperature': 0.9, 'top_p': 1.0, 'min_p': 0.05, 'top_k': 80, 'presence_penalty': 0.0, 'frequency_penalty': 0.0, 'stopping_words': ['\n', '</s>', '####', 'Bot:', 'User:', 'You:', '<|im_end|>', '<|eot_id|>'], 'max_input_tokens': 1024, 'best_of': 8, 'max_output_tokens': 64}
gpu_counts: {'NVIDIA RTX A6000': 1}
ineligible_reason: num_battles<5000
is_internal_developer: True
language_model: zonemercy/Vingt-Deux-v2-1e5
latencies: [{'batch_size': 1, 'throughput': 0.37852815915269183, 'latency_mean': 2.6417468011379244, 'latency_p50': 2.621500611305237, 'latency_p90': 2.9163576126098634}, {'batch_size': 2, 'throughput': 0.5922403324593986, 'latency_mean': 3.371749222278595, 'latency_p50': 3.3472946882247925, 'latency_p90': 3.7068390369415285}, {'batch_size': 3, 'throughput': 0.7439963326583818, 'latency_mean': 4.015046926736832, 'latency_p50': 4.018017053604126, 'latency_p90': 4.398577880859375}, {'batch_size': 4, 'throughput': 0.8612203363283656, 'latency_mean': 4.618188242912293, 'latency_p50': 4.597997426986694, 'latency_p90': 5.199566459655761}, {'batch_size': 5, 'throughput': 0.9525268555434196, 'latency_mean': 5.2366658008098605, 'latency_p50': 5.250544428825378, 'latency_p90': 5.870875263214112}]
max_input_tokens: 1024
max_output_tokens: 64
model_architecture: MistralForCausalLM
model_group: zonemercy/Vingt-Deux-v2-
model_name: temp-4
model_num_parameters: 22247282688.0
model_repo: zonemercy/Vingt-Deux-v2-1e5
model_size: 22B
num_battles: 4342
num_wins: 2161
ranking_group: single
status: torndown
submission_type: basic
throughput_3p7s: 0.68
timestamp: 2024-09-24T15:39:12+00:00
us_pacific_date: 2024-09-24
win_ratio: 0.49769691386457854
Download Preference Data
Resubmit model
Shutdown handler not registered because Python interpreter is not running in the main thread
run pipeline %s
run pipeline stage %s
Running pipeline stage MKMLizer
Starting job with name zonemercy-vingt-deux-v2-1e5-v1-mkmlizer
Waiting for job on zonemercy-vingt-deux-v2-1e5-v1-mkmlizer to finish
zonemercy-vingt-deux-v2-1e5-v1-mkmlizer: ╔═════════════════════════════════════════════════════════════════════╗
zonemercy-vingt-deux-v2-1e5-v1-mkmlizer: ║     _____            __           __                                ║
zonemercy-vingt-deux-v2-1e5-v1-mkmlizer: ║    / _/ /_ ___    __/ /  ___ ___ / /                                ║
zonemercy-vingt-deux-v2-1e5-v1-mkmlizer: ║   / _/ / // / |/|/ / _ \/ -_) -_) /                                 ║
zonemercy-vingt-deux-v2-1e5-v1-mkmlizer: ║  /_//_/\_, /|__,__/_//_/\__/\__/_/                                  ║
zonemercy-vingt-deux-v2-1e5-v1-mkmlizer: ║       /___/                                                         ║
zonemercy-vingt-deux-v2-1e5-v1-mkmlizer: ║                                                                     ║
zonemercy-vingt-deux-v2-1e5-v1-mkmlizer: ║  Version: 0.10.1                                                    ║
zonemercy-vingt-deux-v2-1e5-v1-mkmlizer: ║  Copyright 2023 MK ONE TECHNOLOGIES Inc.                            ║
zonemercy-vingt-deux-v2-1e5-v1-mkmlizer: ║  https://mk1.ai                                                     ║
zonemercy-vingt-deux-v2-1e5-v1-mkmlizer: ║                                                                     ║
zonemercy-vingt-deux-v2-1e5-v1-mkmlizer: ║  The license key for the current software has been verified as      ║
zonemercy-vingt-deux-v2-1e5-v1-mkmlizer: ║  belonging to:                                                      ║
zonemercy-vingt-deux-v2-1e5-v1-mkmlizer: ║                                                                     ║
zonemercy-vingt-deux-v2-1e5-v1-mkmlizer: ║  Chai Research Corp.                                                ║
zonemercy-vingt-deux-v2-1e5-v1-mkmlizer: ║  Account ID: 7997a29f-0ceb-4cc7-9adf-840c57b4ae6f                   ║
zonemercy-vingt-deux-v2-1e5-v1-mkmlizer: ║  Expiration: 2024-10-15 23:59:59                                    ║
zonemercy-vingt-deux-v2-1e5-v1-mkmlizer: ║                                                                     ║
zonemercy-vingt-deux-v2-1e5-v1-mkmlizer: ╚═════════════════════════════════════════════════════════════════════╝
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
zonemercy-vingt-deux-v2-1e5-v1-mkmlizer: Downloaded to shared memory in 100.616s
zonemercy-vingt-deux-v2-1e5-v1-mkmlizer: quantizing model to /dev/shm/model_cache, profile:s0, folder:/tmp/tmpebor0d36, device:0
zonemercy-vingt-deux-v2-1e5-v1-mkmlizer: Saving flywheel model at /dev/shm/model_cache
zonemercy-vingt-deux-v2-1e5-v1-mkmlizer: quantized model in 45.670s
zonemercy-vingt-deux-v2-1e5-v1-mkmlizer: Processed model zonemercy/Vingt-Deux-v2-1e5 in 146.285s
zonemercy-vingt-deux-v2-1e5-v1-mkmlizer: creating bucket guanaco-mkml-models
zonemercy-vingt-deux-v2-1e5-v1-mkmlizer: Bucket 's3://guanaco-mkml-models/' created
zonemercy-vingt-deux-v2-1e5-v1-mkmlizer: uploading /dev/shm/model_cache to s3://guanaco-mkml-models/zonemercy-vingt-deux-v2-1e5-v1
zonemercy-vingt-deux-v2-1e5-v1-mkmlizer: cp /dev/shm/model_cache/config.json s3://guanaco-mkml-models/zonemercy-vingt-deux-v2-1e5-v1/config.json
zonemercy-vingt-deux-v2-1e5-v1-mkmlizer: cp /dev/shm/model_cache/special_tokens_map.json s3://guanaco-mkml-models/zonemercy-vingt-deux-v2-1e5-v1/special_tokens_map.json
zonemercy-vingt-deux-v2-1e5-v1-mkmlizer: cp /dev/shm/model_cache/tokenizer_config.json s3://guanaco-mkml-models/zonemercy-vingt-deux-v2-1e5-v1/tokenizer_config.json
zonemercy-vingt-deux-v2-1e5-v1-mkmlizer: cp /dev/shm/model_cache/tokenizer.json s3://guanaco-mkml-models/zonemercy-vingt-deux-v2-1e5-v1/tokenizer.json
zonemercy-vingt-deux-v2-1e5-v1-mkmlizer: cp /dev/shm/model_cache/flywheel_model.1.safetensors s3://guanaco-mkml-models/zonemercy-vingt-deux-v2-1e5-v1/flywheel_model.1.safetensors
zonemercy-vingt-deux-v2-1e5-v1-mkmlizer: cp /dev/shm/model_cache/flywheel_model.0.safetensors s3://guanaco-mkml-models/zonemercy-vingt-deux-v2-1e5-v1/flywheel_model.0.safetensors
zonemercy-vingt-deux-v2-1e5-v1-mkmlizer: 
Loading 0:   0%|          | 0/507 [00:00<?, ?it/s]
Loading 0:   1%|          | 5/507 [00:00<00:19, 25.42it/s]
Loading 0:   2%|▏         | 10/507 [00:00<00:14, 35.48it/s]
Loading 0:   3%|▎         | 14/507 [00:00<00:16, 29.84it/s]
Loading 0:   4%|▎         | 19/507 [00:00<00:13, 35.12it/s]
Loading 0:   5%|▍         | 23/507 [00:00<00:15, 30.56it/s]
Loading 0:   6%|▌         | 28/507 [00:00<00:13, 34.95it/s]
Loading 0:   6%|▋         | 32/507 [00:01<00:15, 30.85it/s]
Loading 0:   7%|▋         | 37/507 [00:01<00:13, 35.12it/s]
Loading 0:   8%|▊         | 41/507 [00:01<00:14, 31.70it/s]
Loading 0:   9%|▉         | 46/507 [00:01<00:12, 35.87it/s]
Loading 0:  10%|▉         | 50/507 [00:01<00:13, 32.90it/s]
Loading 0:  11%|█         | 54/507 [00:01<00:19, 23.07it/s]
Loading 0:  11%|█         | 57/507 [00:01<00:19, 23.04it/s]
Loading 0:  12%|█▏        | 61/507 [00:02<00:16, 26.48it/s]
Loading 0:  13%|█▎        | 65/507 [00:02<00:17, 25.91it/s]
Loading 0:  14%|█▍        | 70/507 [00:02<00:14, 31.06it/s]
Loading 0:  15%|█▍        | 75/507 [00:02<00:12, 34.09it/s]
Loading 0:  16%|█▌        | 80/507 [00:02<00:12, 33.85it/s]
Loading 0:  17%|█▋        | 87/507 [00:02<00:10, 39.37it/s]
Loading 0:  18%|█▊        | 92/507 [00:02<00:10, 37.94it/s]
Loading 0:  19%|█▉        | 96/507 [00:02<00:11, 37.35it/s]
Loading 0:  20%|█▉        | 100/507 [00:03<00:11, 35.58it/s]
Loading 0:  21%|██        | 105/507 [00:03<00:10, 36.86it/s]
Loading 0:  21%|██▏       | 109/507 [00:03<00:11, 34.70it/s]
Loading 0:  22%|██▏       | 113/507 [00:03<00:15, 26.02it/s]
Loading 0:  23%|██▎       | 116/507 [00:03<00:16, 24.14it/s]
Loading 0:  24%|██▍       | 122/507 [00:03<00:14, 27.09it/s]
Loading 0:  25%|██▌       | 127/507 [00:04<00:12, 31.59it/s]
Loading 0:  26%|██▌       | 131/507 [00:04<00:12, 29.40it/s]
Loading 0:  27%|██▋       | 136/507 [00:04<00:11, 33.67it/s]
Loading 0:  28%|██▊       | 140/507 [00:04<00:11, 31.28it/s]
Loading 0:  29%|██▊       | 145/507 [00:04<00:10, 35.25it/s]
Loading 0:  29%|██▉       | 149/507 [00:04<00:11, 32.11it/s]
Loading 0:  30%|███       | 154/507 [00:04<00:09, 36.13it/s]
Loading 0:  31%|███       | 158/507 [00:05<00:10, 32.16it/s]
Loading 0:  32%|███▏      | 164/507 [00:05<00:09, 37.02it/s]
Loading 0:  33%|███▎      | 169/507 [00:05<00:11, 28.59it/s]
Loading 0:  34%|███▍      | 173/507 [00:05<00:11, 29.38it/s]
Loading 0:  35%|███▍      | 177/507 [00:05<00:11, 28.00it/s]
Loading 0:  36%|███▌      | 181/507 [00:05<00:10, 30.18it/s]
Loading 0:  36%|███▋      | 185/507 [00:05<00:11, 28.44it/s]
Loading 0:  37%|███▋      | 190/507 [00:06<00:09, 32.82it/s]
Loading 0:  38%|███▊      | 194/507 [00:06<00:10, 29.87it/s]
Loading 0:  39%|███▉      | 199/507 [00:06<00:09, 34.21it/s]
Loading 0:  40%|████      | 203/507 [00:06<00:09, 31.66it/s]
Loading 0:  41%|████▏     | 210/507 [00:06<00:07, 38.70it/s]
Loading 0:  42%|████▏     | 215/507 [00:06<00:07, 37.98it/s]
Loading 0:  43%|████▎     | 219/507 [00:06<00:09, 30.57it/s]
Loading 0:  44%|████▍     | 223/507 [00:07<00:08, 32.14it/s]
Loading 0:  45%|████▍     | 227/507 [00:07<00:10, 27.53it/s]
Loading 0:  46%|████▌     | 231/507 [00:07<00:10, 26.83it/s]
Loading 0:  46%|████▋     | 235/507 [00:07<00:09, 29.28it/s]
Loading 0:  47%|████▋     | 239/507 [00:07<00:09, 27.33it/s]
Loading 0:  48%|████▊     | 244/507 [00:07<00:08, 31.71it/s]
Loading 0:  49%|████▉     | 248/507 [00:07<00:08, 29.01it/s]
Loading 0:  50%|████▉     | 253/507 [00:08<00:07, 33.48it/s]
Loading 0:  51%|█████     | 257/507 [00:08<00:08, 30.17it/s]
Loading 0:  52%|█████▏    | 262/507 [00:08<00:07, 34.51it/s]
Loading 0:  52%|█████▏    | 266/507 [00:08<00:07, 31.27it/s]
Loading 0:  53%|█████▎    | 271/507 [00:08<00:06, 34.97it/s]
Loading 0:  54%|█████▍    | 275/507 [00:08<00:07, 31.47it/s]
Loading 0:  55%|█████▌    | 280/507 [00:08<00:06, 35.64it/s]
Loading 0:  56%|█████▌    | 284/507 [00:09<00:08, 26.61it/s]
Loading 0:  57%|█████▋    | 288/507 [00:09<00:08, 25.89it/s]
Loading 0:  58%|█████▊    | 293/507 [00:09<00:08, 26.46it/s]
Loading 0:  59%|█████▉    | 298/507 [00:09<00:06, 30.87it/s]
Loading 0:  59%|█████▉    | 299/507 [00:20<00:06, 30.87it/s]
Loading 0:  59%|█████▉    | 300/507 [00:24<04:04,  1.18s/it]
Loading 0:  60%|█████▉    | 302/507 [00:24<03:20,  1.02it/s]
Loading 0:  61%|██████    | 307/507 [00:24<02:01,  1.65it/s]
Loading 0:  61%|██████    | 310/507 [00:24<01:31,  2.15it/s]
Loading 0:  62%|██████▏   | 313/507 [00:24<01:08,  2.85it/s]
Loading 0:  63%|██████▎   | 318/507 [00:25<00:42,  4.45it/s]
Loading 0:  64%|██████▎   | 322/507 [00:25<00:30,  6.03it/s]
Loading 0:  64%|██████▍   | 327/507 [00:25<00:20,  8.62it/s]
Loading 0:  65%|██████▌   | 331/507 [00:25<00:16, 10.85it/s]
Loading 0:  66%|██████▌   | 335/507 [00:25<00:12, 13.41it/s]
Loading 0:  67%|██████▋   | 340/507 [00:25<00:11, 14.77it/s]
Loading 0:  68%|██████▊   | 343/507 [00:25<00:10, 16.10it/s]
Loading 0:  68%|██████▊   | 347/507 [00:26<00:09, 17.54it/s]
Loading 0:  69%|██████▉   | 352/507 [00:26<00:06, 22.35it/s]
Loading 0:  70%|███████   | 356/507 [00:26<00:06, 22.97it/s]
Loading 0:  71%|███████   | 361/507 [00:26<00:05, 27.75it/s]
Loading 0:  72%|███████▏  | 365/507 [00:26<00:05, 26.78it/s]
Loading 0:  73%|███████▎  | 370/507 [00:26<00:04, 31.42it/s]
Loading 0:  74%|███████▍  | 374/507 [00:26<00:04, 29.71it/s]
Loading 0:  75%|███████▌  | 381/507 [00:27<00:03, 36.80it/s]
Loading 0:  76%|███████▌  | 386/507 [00:27<00:03, 36.54it/s]
Loading 0:  77%|███████▋  | 390/507 [00:27<00:03, 29.53it/s]
Loading 0:  78%|███████▊  | 394/507 [00:27<00:03, 31.23it/s]
Loading 0:  79%|███████▊  | 398/507 [00:27<00:04, 26.67it/s]
Loading 0:  79%|███████▉  | 401/507 [00:27<00:04, 24.41it/s]
Loading 0:  80%|████████  | 406/507 [00:28<00:03, 29.47it/s]
Loading 0:  81%|████████  | 410/507 [00:28<00:03, 27.66it/s]
Loading 0:  82%|████████▏ | 415/507 [00:28<00:02, 31.90it/s]
Loading 0:  83%|████████▎ | 419/507 [00:28<00:02, 29.91it/s]
Loading 0:  84%|████████▍ | 426/507 [00:28<00:02, 36.91it/s]
Loading 0:  85%|████████▍ | 430/507 [00:28<00:02, 34.81it/s]
Loading 0:  86%|████████▌ | 435/507 [00:28<00:01, 36.69it/s]
Loading 0:  87%|████████▋ | 439/507 [00:28<00:01, 35.26it/s]
Loading 0:  88%|████████▊ | 444/507 [00:29<00:01, 37.21it/s]
Loading 0:  88%|████████▊ | 448/507 [00:29<00:01, 35.62it/s]
Loading 0:  89%|████████▉ | 453/507 [00:29<00:01, 38.93it/s]
Loading 0:  90%|█████████ | 457/507 [00:31<00:08,  5.81it/s]
Loading 0:  91%|█████████ | 460/507 [00:31<00:06,  6.98it/s]
Loading 0:  92%|█████████▏| 465/507 [00:31<00:04,  9.44it/s]
Loading 0:  93%|█████████▎| 470/507 [00:31<00:02, 12.90it/s]
Loading 0:  93%|█████████▎| 474/507 [00:32<00:02, 14.58it/s]
Loading 0:  94%|█████████▍| 479/507 [00:32<00:01, 18.81it/s]
Loading 0:  95%|█████████▌| 483/507 [00:32<00:01, 20.20it/s]
Loading 0:  96%|█████████▋| 488/507 [00:32<00:00, 24.85it/s]
Loading 0:  97%|█████████▋| 492/507 [00:32<00:00, 24.75it/s]
Loading 0:  98%|█████████▊| 497/507 [00:32<00:00, 29.19it/s]
Loading 0:  99%|█████████▉| 501/507 [00:32<00:00, 27.58it/s]
Loading 0: 100%|█████████▉| 506/507 [00:33<00:00, 31.94it/s]
                                                            
Job zonemercy-vingt-deux-v2-1e5-v1-mkmlizer completed after 200.3s with status: succeeded
Stopping job with name zonemercy-vingt-deux-v2-1e5-v1-mkmlizer
Pipeline stage MKMLizer completed in 204.23s
run pipeline stage %s
Running pipeline stage MKMLTemplater
Pipeline stage MKMLTemplater completed in 0.20s
run pipeline stage %s
Running pipeline stage MKMLDeployer
Creating inference service zonemercy-vingt-deux-v2-1e5-v1
Waiting for inference service zonemercy-vingt-deux-v2-1e5-v1 to be ready
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Inference service zonemercy-vingt-deux-v2-1e5-v1 ready after 213.79531621932983s
Pipeline stage MKMLDeployer completed in 214.52s
run pipeline stage %s
Running pipeline stage StressChecker
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Received healthy response to inference request in 5.553677082061768s
Received healthy response to inference request in 3.732356309890747s
Received healthy response to inference request in 3.9748764038085938s
Received healthy response to inference request in 6.35319709777832s
Received healthy response to inference request in 9.885231018066406s
5 requests
0 failed requests
5th percentile: 3.7808603286743163
10th percentile: 3.8293643474578856
20th percentile: 3.9263723850250245
30th percentile: 4.2906365394592285
40th percentile: 4.922156810760498
50th percentile: 5.553677082061768
60th percentile: 5.873485088348389
70th percentile: 6.19329309463501
80th percentile: 7.059603881835939
90th percentile: 8.472417449951172
95th percentile: 9.178824234008788
99th percentile: 9.743949661254883
mean time: 5.899867582321167
%s, retrying in %s seconds...
Received healthy response to inference request in 4.077255725860596s
Received healthy response to inference request in 3.1228508949279785s
Received healthy response to inference request in 2.9199042320251465s
Received healthy response to inference request in 5.377828598022461s
Received healthy response to inference request in 2.863351345062256s
5 requests
0 failed requests
5th percentile: 2.874661922454834
10th percentile: 2.885972499847412
20th percentile: 2.9085936546325684
30th percentile: 2.960493564605713
40th percentile: 3.0416722297668457
50th percentile: 3.1228508949279785
60th percentile: 3.5046128273010253
70th percentile: 3.886374759674072
80th percentile: 4.337370300292969
90th percentile: 4.857599449157715
95th percentile: 5.1177140235900875
99th percentile: 5.325805683135986
mean time: 3.6722381591796873
%s, retrying in %s seconds...
Received healthy response to inference request in 2.9852182865142822s
Received healthy response to inference request in 3.018904447555542s
Received healthy response to inference request in 3.0266709327697754s
Received healthy response to inference request in 4.541728496551514s
Received healthy response to inference request in 3.4469218254089355s
5 requests
0 failed requests
5th percentile: 2.991955518722534
10th percentile: 2.998692750930786
20th percentile: 3.01216721534729
30th percentile: 3.0204577445983887
40th percentile: 3.023564338684082
50th percentile: 3.0266709327697754
60th percentile: 3.1947712898254395
70th percentile: 3.3628716468811035
80th percentile: 3.6658831596374513
90th percentile: 4.103805828094482
95th percentile: 4.3227671623229975
99th percentile: 4.49793622970581
mean time: 3.40388879776001
Pipeline stage StressChecker completed in 70.30s
run pipeline stage %s
Running pipeline stage TriggerMKMLProfilingPipeline
run_pipeline:run_in_cloud %s
starting trigger_guanaco_pipeline args=%s
Pipeline stage TriggerMKMLProfilingPipeline completed in 4.35s
Shutdown handler de-registered
zonemercy-vingt-deux-v2-1e5_v1 status is now deployed due to DeploymentManager action
Shutdown handler registered
run pipeline %s
run pipeline stage %s
Running pipeline stage MKMLProfilerDeleter
Skipping teardown as no inference service was successfully deployed
Pipeline stage MKMLProfilerDeleter completed in 0.17s
run pipeline stage %s
Running pipeline stage MKMLProfilerTemplater
Pipeline stage MKMLProfilerTemplater completed in 0.14s
run pipeline stage %s
Running pipeline stage MKMLProfilerDeployer
Creating inference service zonemercy-vingt-deux-v2-1e5-v1-profiler
Waiting for inference service zonemercy-vingt-deux-v2-1e5-v1-profiler to be ready
Inference service zonemercy-vingt-deux-v2-1e5-v1-profiler ready after 310.72975182533264s
Pipeline stage MKMLProfilerDeployer completed in 311.15s
run pipeline stage %s
Running pipeline stage MKMLProfilerRunner
kubectl cp /code/guanaco/guanaco_inference_services/src/inference_scripts tenant-chaiml-guanaco/zonemercy-vingt-deux9ac6f203126b195fd5b1361eb4336a81-deplolf9t7:/code/chaiverse_profiler_1727193211 --namespace tenant-chaiml-guanaco
kubectl exec -it zonemercy-vingt-deux9ac6f203126b195fd5b1361eb4336a81-deplolf9t7 --namespace tenant-chaiml-guanaco -- sh -c 'cd /code/chaiverse_profiler_1727193211 && python profiles.py profile --best_of_n 8 --auto_batch 5 --batches 1,5,10,15,20,25,30,35,40,45,50,55,60,65,70,75,80,85,90,95,100,105,110,115,120,125,130,135,140,145,150,155,160,165,170,175,180,185,190,195 --samples 200 --input_tokens 1024 --output_tokens 64 --summary /code/chaiverse_profiler_1727193211/summary.json'
kubectl exec -it zonemercy-vingt-deux9ac6f203126b195fd5b1361eb4336a81-deplolf9t7 --namespace tenant-chaiml-guanaco -- bash -c 'cat /code/chaiverse_profiler_1727193211/summary.json'
Pipeline stage MKMLProfilerRunner completed in 1584.31s
run pipeline stage %s
Running pipeline stage MKMLProfilerDeleter
Checking if service zonemercy-vingt-deux-v2-1e5-v1-profiler is running
Tearing down inference service zonemercy-vingt-deux-v2-1e5-v1-profiler
Service zonemercy-vingt-deux-v2-1e5-v1-profiler has been torndown
Pipeline stage MKMLProfilerDeleter completed in 2.09s
Shutdown handler de-registered
zonemercy-vingt-deux-v2-1e5_v1 status is now inactive due to auto deactivation removed underperforming models
Deleting key zonemercy-vingt-deux-v2-1e5-v2/tokenizer_config.json from bucket guanaco-mkml-models
Deleting key zonemercy-vingt-deux-gfv-3432-v1/tokenizer_config.json from bucket guanaco-mkml-models
Pipeline stage MKMLModelDeleter completed in 9.17s
zonemercy-vingt-deux-v2-1e5_v1 status is now torndown due to DeploymentManager action