developer_uid: zonemercy
submission_id: zonemercy-virgo-edit-v4-1e5_v2
model_name: zonemercy-virgo-edit-v4-1e5_v2
model_group: zonemercy/Virgo-Edit-v4-
status: torndown
timestamp: 2024-09-19T10:07:52+00:00
num_battles: 14773
num_wins: 7192
celo_rating: 1234.41
family_friendly_score: 0.0
submission_type: basic
model_repo: zonemercy/Virgo-Edit-v4-1e5
model_architecture: MistralForCausalLM
model_num_parameters: 12772070400.0
best_of: 8
max_input_tokens: 1024
max_output_tokens: 64
latencies: [{'batch_size': 1, 'throughput': 0.6129577143888878, 'latency_mean': 1.631334582567215, 'latency_p50': 1.6418107748031616, 'latency_p90': 1.7962204694747925}, {'batch_size': 3, 'throughput': 1.0718445103588992, 'latency_mean': 2.7874943768978118, 'latency_p50': 2.7828088998794556, 'latency_p90': 3.1129243850708006}, {'batch_size': 5, 'throughput': 1.2101236971406695, 'latency_mean': 4.111450810432434, 'latency_p50': 4.113201022148132, 'latency_p90': 4.601292157173157}, {'batch_size': 6, 'throughput': 1.2537929101287537, 'latency_mean': 4.764010162353515, 'latency_p50': 4.780732274055481, 'latency_p90': 5.438910579681396}, {'batch_size': 8, 'throughput': 1.2320258051043984, 'latency_mean': 6.463915491104126, 'latency_p50': 6.499725818634033, 'latency_p90': 7.2685138463974}, {'batch_size': 10, 'throughput': 1.1949980897300083, 'latency_mean': 8.321032410860061, 'latency_p50': 8.381908535957336, 'latency_p90': 9.387161827087402}]
gpu_counts: {'NVIDIA RTX A5000': 1}
display_name: zonemercy-virgo-edit-v4-1e5_v2
is_internal_developer: True
language_model: zonemercy/Virgo-Edit-v4-1e5
model_size: 13B
ranking_group: single
throughput_3p7s: 1.18
us_pacific_date: 2024-09-19
win_ratio: 0.4868340892168145
generation_params: {'temperature': 0.9, 'top_p': 1.0, 'min_p': 0.05, 'top_k': 80, 'presence_penalty': 0.0, 'frequency_penalty': 0.0, 'stopping_words': ['</s>', '###', 'Bot:', 'User:', 'You:', '<|im_end|>'], 'max_input_tokens': 1024, 'best_of': 8, 'max_output_tokens': 64}
formatter: {'memory_template': '', 'prompt_template': '', 'bot_template': '{bot_name}: {message}\n', 'user_template': '{user_name}: {message}\n', 'response_template': '{bot_name}:', 'truncate_by_message': False}
Resubmit model
Shutdown handler not registered because Python interpreter is not running in the main thread
run pipeline %s
run pipeline stage %s
Running pipeline stage MKMLizer
Failed to get response for submission zonemercy-virgo-edit-v5-1e5_v1: ('http://zonemercy-virgo-edit-v5-1e5-v1-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'activator request timeout')
Starting job with name zonemercy-virgo-edit-v4-1e5-v2-mkmlizer
Waiting for job on zonemercy-virgo-edit-v4-1e5-v2-mkmlizer to finish
Failed to get response for submission cycy233-nemo-p-v3-c4_v1: ('http://cycy233-nemo-p-v3-c4-v1-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'activator request timeout')
zonemercy-virgo-edit-v4-1e5-v2-mkmlizer: ╔═════════════════════════════════════════════════════════════════════╗
zonemercy-virgo-edit-v4-1e5-v2-mkmlizer: ║     _____            __           __                                ║
zonemercy-virgo-edit-v4-1e5-v2-mkmlizer: ║    / _/ /_ ___    __/ /  ___ ___ / /                                ║
Failed to get response for submission cycy233-nemo-p-v3-c2_v1: ('http://cycy233-nemo-p-v3-c2-v1-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'activator request timeout')
Failed to get response for submission chaiml-0918-horror-retor_3693_v2: ('http://chaiml-0918-horror-retor-3693-v2-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'activator request timeout')
zonemercy-virgo-edit-v4-1e5-v2-mkmlizer: ║   / _/ / // / |/|/ / _ \/ -_) -_) /                                 ║
zonemercy-virgo-edit-v4-1e5-v2-mkmlizer: ║  /_//_/\_, /|__,__/_//_/\__/\__/_/                                  ║
zonemercy-virgo-edit-v4-1e5-v2-mkmlizer: ║       /___/                                                         ║
zonemercy-virgo-edit-v4-1e5-v2-mkmlizer: ║                                                                     ║
zonemercy-virgo-edit-v4-1e5-v2-mkmlizer: ║  Version: 0.10.1                                                    ║
zonemercy-virgo-edit-v4-1e5-v2-mkmlizer: ║  Copyright 2023 MK ONE TECHNOLOGIES Inc.                            ║
zonemercy-virgo-edit-v4-1e5-v2-mkmlizer: ║  https://mk1.ai                                                     ║
zonemercy-virgo-edit-v4-1e5-v2-mkmlizer: ║                                                                     ║
zonemercy-virgo-edit-v4-1e5-v2-mkmlizer: ║  The license key for the current software has been verified as      ║
zonemercy-virgo-edit-v4-1e5-v2-mkmlizer: ║  belonging to:                                                      ║
zonemercy-virgo-edit-v4-1e5-v2-mkmlizer: ║                                                                     ║
zonemercy-virgo-edit-v4-1e5-v2-mkmlizer: ║  Chai Research Corp.                                                ║
zonemercy-virgo-edit-v4-1e5-v2-mkmlizer: ║  Account ID: 7997a29f-0ceb-4cc7-9adf-840c57b4ae6f                   ║
zonemercy-virgo-edit-v4-1e5-v2-mkmlizer: ║  Expiration: 2024-10-15 23:59:59                                    ║
zonemercy-virgo-edit-v4-1e5-v2-mkmlizer: ║                                                                     ║
zonemercy-virgo-edit-v4-1e5-v2-mkmlizer: ╚═════════════════════════════════════════════════════════════════════╝
Failed to get response for submission zonemercy-virgo-edit-v5-1e5_v1: ('http://zonemercy-virgo-edit-v5-1e5-v1-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'activator request timeout')
Failed to get response for submission zonemercy-virgo-edit-v3-1e5_v1: ('http://zonemercy-virgo-edit-v3-1e5-v1-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'request timeout')
Failed to get response for submission zonemercy-virgo-edit-v5-_5534_v1: ('http://zonemercy-virgo-edit-v5-5534-v1-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'request timeout')
zonemercy-virgo-edit-v4-1e5-v2-mkmlizer: Downloaded to shared memory in 54.808s
zonemercy-virgo-edit-v4-1e5-v2-mkmlizer: quantizing model to /dev/shm/model_cache, profile:s0, folder:/tmp/tmpxp8a_isg, device:0
zonemercy-virgo-edit-v4-1e5-v2-mkmlizer: Saving flywheel model at /dev/shm/model_cache
Failed to get response for submission chaiml-0919-quad-dataset_3078_v2: ('http://chaiml-0919-quad-dataset-3078-v2-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'request timeout')
Failed to get response for submission cycy233-nemo-p-v3-c4_v1: ('http://cycy233-nemo-p-v3-c4-v1-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'request timeout')
zonemercy-virgo-edit-v4-1e5-v2-mkmlizer: quantized model in 40.949s
Failed to get response for submission cycy233-nemo-p-v3-c4_v1: ('http://cycy233-nemo-p-v3-c4-v1-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'request timeout')
zonemercy-virgo-edit-v4-1e5-v2-mkmlizer: Processed model zonemercy/Virgo-Edit-v4-1e5 in 95.758s
zonemercy-virgo-edit-v4-1e5-v2-mkmlizer: creating bucket guanaco-mkml-models
Failed to get response for submission zonemercy-virgo-edit-v5-1e5_v1: ('http://zonemercy-virgo-edit-v5-1e5-v1-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'request timeout')
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Failed to get response for submission zonemercy-virgo-edit-v5-_5534_v1: ('http://zonemercy-virgo-edit-v5-5534-v1-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'request timeout')
Failed to get response for submission cycy233-nemo-p-v3-c4_v1: ('http://cycy233-nemo-p-v3-c4-v1-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'request timeout')
zonemercy-virgo-edit-v4-1e5-v2-mkmlizer: cp /dev/shm/model_cache/tokenizer.json s3://guanaco-mkml-models/zonemercy-virgo-edit-v4-1e5-v2/tokenizer.json
zonemercy-virgo-edit-v4-1e5-v2-mkmlizer: cp /dev/shm/model_cache/flywheel_model.0.safetensors s3://guanaco-mkml-models/zonemercy-virgo-edit-v4-1e5-v2/flywheel_model.0.safetensors
zonemercy-virgo-edit-v4-1e5-v2-mkmlizer: 
Loading 0:   0%|          | 0/363 [00:00<?, ?it/s]
Loading 0:   1%|▏         | 5/363 [00:00<00:15, 22.95it/s]
Loading 0:   3%|▎         | 10/363 [00:00<00:11, 29.49it/s]
Loading 0:   4%|▍         | 14/363 [00:00<00:13, 25.83it/s]
Loading 0:   6%|▌         | 21/363 [00:00<00:09, 37.49it/s]
Loading 0:   7%|▋         | 26/363 [00:01<00:15, 21.71it/s]
Loading 0:   9%|▊         | 31/363 [00:01<00:12, 26.50it/s]
Loading 0:  10%|▉         | 35/363 [00:01<00:11, 28.16it/s]
Loading 0:  11%|█         | 39/363 [00:01<00:10, 29.50it/s]
Loading 0:  12%|█▏        | 43/363 [00:01<00:11, 28.56it/s]
Loading 0:  13%|█▎        | 48/363 [00:01<00:10, 30.76it/s]
Loading 0:  14%|█▍        | 52/363 [00:01<00:10, 28.50it/s]
Loading 0:  15%|█▌        | 56/363 [00:01<00:10, 29.14it/s]
Loading 0:  17%|█▋        | 61/363 [00:02<00:11, 25.32it/s]
Loading 0:  18%|█▊        | 64/363 [00:02<00:13, 22.19it/s]
Loading 0:  20%|█▉        | 71/363 [00:02<00:09, 29.41it/s]
Loading 0:  21%|██        | 75/363 [00:02<00:10, 28.77it/s]
Loading 0:  22%|██▏       | 79/363 [00:02<00:10, 28.16it/s]
Loading 0:  23%|██▎       | 84/363 [00:03<00:09, 30.15it/s]
Loading 0:  24%|██▍       | 88/363 [00:03<00:09, 29.19it/s]
Loading 0:  26%|██▌       | 93/363 [00:03<00:08, 31.85it/s]
Loading 0:  27%|██▋       | 97/363 [00:03<00:08, 30.53it/s]
Loading 0:  28%|██▊       | 101/363 [00:03<00:10, 24.73it/s]
Loading 0:  29%|██▊       | 104/363 [00:03<00:11, 21.65it/s]
Loading 0:  31%|███       | 111/363 [00:04<00:08, 28.29it/s]
Loading 0:  32%|███▏      | 115/363 [00:04<00:09, 27.39it/s]
Loading 0:  33%|███▎      | 120/363 [00:04<00:08, 30.22it/s]
Loading 0:  34%|███▍      | 124/363 [00:04<00:08, 29.17it/s]
Loading 0:  36%|███▌      | 129/363 [00:04<00:07, 31.09it/s]
Loading 0:  37%|███▋      | 133/363 [00:04<00:07, 28.83it/s]
Loading 0:  38%|███▊      | 137/363 [00:04<00:07, 28.71it/s]
Loading 0:  39%|███▉      | 142/363 [00:05<00:08, 24.65it/s]
Loading 0:  40%|███▉      | 145/363 [00:05<00:09, 23.32it/s]
Loading 0:  41%|████      | 149/363 [00:05<00:09, 22.39it/s]
Loading 0:  43%|████▎     | 156/363 [00:05<00:07, 29.27it/s]
Loading 0:  44%|████▍     | 160/363 [00:05<00:07, 28.73it/s]
Loading 0:  45%|████▌     | 165/363 [00:05<00:06, 30.66it/s]
Loading 0:  47%|████▋     | 169/363 [00:06<00:06, 29.26it/s]
Loading 0:  48%|████▊     | 174/363 [00:06<00:06, 30.90it/s]
Loading 0:  49%|████▉     | 178/363 [00:06<00:06, 29.53it/s]
Loading 0:  50%|█████     | 182/363 [00:06<00:07, 25.41it/s]
Loading 0:  51%|█████     | 185/363 [00:06<00:08, 22.01it/s]
Loading 0:  52%|█████▏    | 190/363 [00:06<00:06, 27.38it/s]
Loading 0:  53%|█████▎    | 194/363 [00:07<00:06, 24.56it/s]
Loading 0:  55%|█████▌    | 201/363 [00:07<00:05, 30.77it/s]
Loading 0:  56%|█████▋    | 205/363 [00:07<00:05, 29.54it/s]
Loading 0:  58%|█████▊    | 210/363 [00:07<00:04, 31.77it/s]
Loading 0:  59%|█████▉    | 214/363 [00:07<00:04, 30.13it/s]
Loading 0:  60%|██████    | 218/363 [00:07<00:04, 30.04it/s]
Loading 0:  61%|██████▏   | 223/363 [00:08<00:05, 25.26it/s]
Loading 0:  62%|██████▏   | 226/363 [00:08<00:05, 23.71it/s]
Loading 0:  63%|██████▎   | 230/363 [00:08<00:05, 22.73it/s]
Loading 0:  65%|██████▌   | 237/363 [00:08<00:04, 29.36it/s]
Loading 0:  66%|██████▋   | 241/363 [00:08<00:04, 28.44it/s]
Loading 0:  68%|██████▊   | 246/363 [00:08<00:03, 30.98it/s]
Loading 0:  69%|██████▉   | 250/363 [00:09<00:03, 29.33it/s]
Loading 0:  70%|███████   | 255/363 [00:09<00:03, 31.65it/s]
Loading 0:  71%|███████▏  | 259/363 [00:09<00:03, 29.64it/s]
Loading 0:  72%|███████▏  | 263/363 [00:09<00:04, 24.37it/s]
Loading 0:  73%|███████▎  | 266/363 [00:09<00:04, 21.79it/s]
Loading 0:  75%|███████▌  | 273/363 [00:09<00:03, 28.86it/s]
Loading 0:  76%|███████▋  | 277/363 [00:10<00:03, 28.02it/s]
Loading 0:  78%|███████▊  | 282/363 [00:10<00:02, 30.21it/s]
Loading 0:  79%|███████▉  | 286/363 [00:10<00:02, 29.19it/s]
Loading 0:  80%|████████  | 291/363 [00:10<00:02, 31.71it/s]
Loading 0:  81%|████████▏ | 295/363 [00:10<00:02, 29.83it/s]
Loading 0:  82%|████████▏ | 299/363 [00:10<00:02, 29.81it/s]
Loading 0:  84%|████████▎ | 304/363 [00:10<00:02, 24.99it/s]
Loading 0:  85%|████████▍ | 307/363 [00:11<00:02, 23.81it/s]
Loading 0:  86%|████████▌ | 311/363 [00:11<00:02, 22.78it/s]
Loading 0:  88%|████████▊ | 318/363 [00:11<00:01, 29.84it/s]
Loading 0:  89%|████████▊ | 322/363 [00:11<00:01, 28.45it/s]
Loading 0:  90%|█████████ | 327/363 [00:11<00:01, 30.55it/s]
Loading 0:  91%|█████████ | 331/363 [00:11<00:01, 29.30it/s]
Loading 0:  93%|█████████▎| 336/363 [00:12<00:00, 31.69it/s]
Loading 0:  94%|█████████▎| 340/363 [00:12<00:00, 29.51it/s]
Loading 0:  95%|█████████▍| 344/363 [00:19<00:09,  1.97it/s]
Loading 0:  96%|█████████▌| 348/363 [00:19<00:05,  2.65it/s]
Loading 0:  97%|█████████▋| 353/363 [00:19<00:02,  3.84it/s]
Loading 0:  98%|█████████▊| 357/363 [00:19<00:01,  4.97it/s]
                                                            

Job zonemercy-virgo-edit-v4-1e5-v2-mkmlizer completed after 114.31s with status: succeeded
Stopping job with name zonemercy-virgo-edit-v4-1e5-v2-mkmlizer
Pipeline stage MKMLizer completed in 115.89s
run pipeline stage %s
Running pipeline stage MKMLTemplater
Pipeline stage MKMLTemplater completed in 0.08s
run pipeline stage %s
Running pipeline stage MKMLDeployer
Creating inference service zonemercy-virgo-edit-v4-1e5-v2
Waiting for inference service zonemercy-virgo-edit-v4-1e5-v2 to be ready
Failed to get response for submission zonemercy-virgo-edit-v5-_5534_v1: ('http://zonemercy-virgo-edit-v5-5534-v1-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'EOF\n')
Failed to get response for submission chaiml-0919-quad-dataset_3078_v2: ('http://chaiml-0919-quad-dataset-3078-v2-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'request timeout')
Failed to get response for submission zonemercy-virgo-edit-v5-1e5b1_v1: ('http://zonemercy-virgo-edit-v5-1e5b1-v1-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'request timeout')
Failed to get response for submission cycy233-nemo-p-v3-c4_v1: ('http://cycy233-nemo-p-v3-c4-v1-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'request timeout')
Failed to get response for submission cycy233-nemo-p-v3-c2_v1: ('http://cycy233-nemo-p-v3-c2-v1-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'request timeout')
Failed to get response for submission cycy233-nemo-p-v3-c1_v1: ('http://cycy233-nemo-p-v3-c1-v1-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'EOF\n')
Failed to get response for submission cycy233-nemo-p-v3-c1_v1: ('http://cycy233-nemo-p-v3-c1-v1-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'EOF\n')
Failed to get response for submission cycy233-nemo-p-v3-c2_v1: ('http://cycy233-nemo-p-v3-c2-v1-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'EOF\n')
Failed to get response for submission chaiml-0918-horror-retor_3693_v2: ('http://chaiml-0918-horror-retor-3693-v2-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'request timeout')
Failed to get response for submission zonemercy-virgo-edit-v5-1e5_v1: ('http://zonemercy-virgo-edit-v5-1e5-v1-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'EOF\n')
Failed to get response for submission mistralai-mistral-nemo-_9330_v92: ('http://mistralai-mistral-nemo-9330-v92-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'request timeout')
Failed to get response for submission chaiml-0918-horror-retor_3693_v2: ('http://chaiml-0918-horror-retor-3693-v2-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'request timeout')
Failed to get response for submission cycy233-nemo-p-v3-c4_v1: ('http://cycy233-nemo-p-v3-c4-v1-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'EOF\n')
Connection pool is full, discarding connection: %s. Connection pool size: %s
Failed to get response for submission chaiml-0918-horror-retor_3693_v2: ('http://chaiml-0918-horror-retor-3693-v2-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'request timeout')
Failed to get response for submission zonemercy-virgo-edit-v5-_5534_v1: ('http://zonemercy-virgo-edit-v5-5534-v1-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'request timeout')
Failed to get response for submission chaiml-0918-horror-retor_3693_v2: ('http://chaiml-0918-horror-retor-3693-v2-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'request timeout')
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Failed to get response for submission chaiml-0918-horror-retor_3693_v2: ('http://chaiml-0918-horror-retor-3693-v2-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'request timeout')
Failed to get response for submission chaiml-0918-null-sft-albert_v2: ('http://chaiml-0918-null-sft-albert-v2-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'request timeout')
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Failed to get response for submission zonemercy-virgo-edit-v5-1e5_v1: ('http://zonemercy-virgo-edit-v5-1e5-v1-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'request timeout')
Inference service zonemercy-virgo-edit-v4-1e5-v2 ready after 181.61027693748474s
Pipeline stage MKMLDeployer completed in 182.20s
run pipeline stage %s
Running pipeline stage StressChecker
Received healthy response to inference request in 2.635577917098999s
Received healthy response to inference request in 1.826678991317749s
Failed to get response for submission zonemercy-virgo-edit-v5-_5534_v1: ('http://zonemercy-virgo-edit-v5-5534-v1-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'EOF\n')
Received healthy response to inference request in 1.9571974277496338s
Received healthy response to inference request in 1.7974720001220703s
Received healthy response to inference request in 2.6186347007751465s
5 requests
0 failed requests
Failed to get response for submission cycy233-nemo-p-v3-c4_v1: ('http://cycy233-nemo-p-v3-c4-v1-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'request timeout')
5th percentile: 1.803313398361206
10th percentile: 1.8091547966003418
20th percentile: 1.8208375930786134
30th percentile: 1.852782678604126
40th percentile: 1.90499005317688
50th percentile: 1.9571974277496338
60th percentile: 2.221772336959839
70th percentile: 2.4863472461700438
80th percentile: 2.622023344039917
90th percentile: 2.628800630569458
95th percentile: 2.6321892738342285
99th percentile: 2.634900188446045
mean time: 2.16711220741272
Pipeline stage StressChecker completed in 12.94s
run pipeline stage %s
Running pipeline stage TriggerMKMLProfilingPipeline
run_pipeline:run_in_cloud %s
starting trigger_guanaco_pipeline args=%s
Pipeline stage TriggerMKMLProfilingPipeline completed in 5.67s
Shutdown handler de-registered
zonemercy-virgo-edit-v4-1e5_v2 status is now deployed due to DeploymentManager action
Shutdown handler registered
run pipeline %s
run pipeline stage %s
Running pipeline stage MKMLProfilerDeleter
Skipping teardown as no inference service was successfully deployed
Pipeline stage MKMLProfilerDeleter completed in 0.16s
run pipeline stage %s
Running pipeline stage MKMLProfilerTemplater
Pipeline stage MKMLProfilerTemplater completed in 0.15s
run pipeline stage %s
Running pipeline stage MKMLProfilerDeployer
Creating inference service zonemercy-virgo-edit-v4-1e5-v2-profiler
Waiting for inference service zonemercy-virgo-edit-v4-1e5-v2-profiler to be ready
Inference service zonemercy-virgo-edit-v4-1e5-v2-profiler ready after 190.42550015449524s
Pipeline stage MKMLProfilerDeployer completed in 190.83s
run pipeline stage %s
Running pipeline stage MKMLProfilerRunner
kubectl cp /code/guanaco/guanaco_inference_services/src/inference_scripts tenant-chaiml-guanaco/zonemercy-virgo-edit20dca1af889c6feda697883f469a1a1b-deplorw2pd:/code/chaiverse_profiler_1726741024 --namespace tenant-chaiml-guanaco
kubectl exec -it zonemercy-virgo-edit20dca1af889c6feda697883f469a1a1b-deplorw2pd --namespace tenant-chaiml-guanaco -- sh -c 'cd /code/chaiverse_profiler_1726741024 && python profiles.py profile --best_of_n 8 --auto_batch 5 --batches 1,5,10,15,20,25,30,35,40,45,50,55,60,65,70,75,80,85,90,95,100,105,110,115,120,125,130,135,140,145,150,155,160,165,170,175,180,185,190,195 --samples 200 --input_tokens 1024 --output_tokens 64 --summary /code/chaiverse_profiler_1726741024/summary.json'
kubectl exec -it zonemercy-virgo-edit20dca1af889c6feda697883f469a1a1b-deplorw2pd --namespace tenant-chaiml-guanaco -- bash -c 'cat /code/chaiverse_profiler_1726741024/summary.json'
Pipeline stage MKMLProfilerRunner completed in 1174.22s
run pipeline stage %s
Running pipeline stage MKMLProfilerDeleter
Checking if service zonemercy-virgo-edit-v4-1e5-v2-profiler is running
Tearing down inference service zonemercy-virgo-edit-v4-1e5-v2-profiler
Service zonemercy-virgo-edit-v4-1e5-v2-profiler has been torndown
Pipeline stage MKMLProfilerDeleter completed in 2.14s
Shutdown handler de-registered
zonemercy-virgo-edit-v4-1e5_v2 status is now inactive due to auto deactivation removed underperforming models
zonemercy-virgo-edit-v4-1e5_v2 status is now torndown due to DeploymentManager action