zonemercy-acute-nemo-v1-_4488

developer_uid: zonemercy
submission_id: zonemercy-acute-nemo-v1-_4488_v5
model_name: 0815v1-2
model_group: zonemercy/Acute-Nemo-v1-
status: torndown
timestamp: 2024-08-15T18:19:18+00:00
num_battles: 152943
num_wins: 77724
celo_rating: 1279.7
family_friendly_score: 0.0
submission_type: basic
model_repo: zonemercy/Acute-Nemo-v1-1e5ep1
model_architecture: MistralForCausalLM
model_num_parameters: 12772070400.0
best_of: 16
max_input_tokens: 512
max_output_tokens: 64
display_name: 0815v1-2
is_internal_developer: True
language_model: zonemercy/Acute-Nemo-v1-1e5ep1
model_size: 13B
ranking_group: single
us_pacific_date: 2024-08-15
win_ratio: 0.508189325434966
generation_params: {'temperature': 1.0, 'top_p': 1.0, 'min_p': 0.08, 'top_k': 80, 'presence_penalty': 0.0, 'frequency_penalty': 0.0, 'stopping_words': ['\n', '</s>', '###', 'Bot:', 'User:', 'You:', '<|im_end|>'], 'max_input_tokens': 512, 'best_of': 16, 'max_output_tokens': 64}
formatter: {'memory_template': "{bot_name}'s Persona: {memory}\n####\n", 'prompt_template': '{prompt}\n<START>\n', 'bot_template': '{bot_name}: {message}\n', 'user_template': '{user_name}: {message}\n', 'response_template': '{bot_name}:', 'truncate_by_message': False}
Resubmit model
Running pipeline stage MKMLizer
Starting job with name zonemercy-acute-nemo-v1-4488-v5-mkmlizer
Waiting for job on zonemercy-acute-nemo-v1-4488-v5-mkmlizer to finish
zonemercy-acute-nemo-v1-4488-v5-mkmlizer: ╔═════════════════════════════════════════════════════════════════════╗
zonemercy-acute-nemo-v1-4488-v5-mkmlizer: ║     _____            __           __                                ║
zonemercy-acute-nemo-v1-4488-v5-mkmlizer: ║    / _/ /_ ___    __/ /  ___ ___ / /                                ║
zonemercy-acute-nemo-v1-4488-v5-mkmlizer: ║   / _/ / // / |/|/ / _ \/ -_) -_) /                                 ║
zonemercy-acute-nemo-v1-4488-v5-mkmlizer: ║  /_//_/\_, /|__,__/_//_/\__/\__/_/                                  ║
zonemercy-acute-nemo-v1-4488-v5-mkmlizer: ║       /___/                                                         ║
zonemercy-acute-nemo-v1-4488-v5-mkmlizer: ║                                                                     ║
zonemercy-acute-nemo-v1-4488-v5-mkmlizer: ║  Version: 0.9.11                                                    ║
zonemercy-acute-nemo-v1-4488-v5-mkmlizer: ║  Copyright 2023 MK ONE TECHNOLOGIES Inc.                            ║
zonemercy-acute-nemo-v1-4488-v5-mkmlizer: ║  https://mk1.ai                                                     ║
zonemercy-acute-nemo-v1-4488-v5-mkmlizer: ║                                                                     ║
zonemercy-acute-nemo-v1-4488-v5-mkmlizer: ║  The license key for the current software has been verified as      ║
zonemercy-acute-nemo-v1-4488-v5-mkmlizer: ║  belonging to:                                                      ║
zonemercy-acute-nemo-v1-4488-v5-mkmlizer: ║                                                                     ║
zonemercy-acute-nemo-v1-4488-v5-mkmlizer: ║  Chai Research Corp.                                                ║
zonemercy-acute-nemo-v1-4488-v5-mkmlizer: ║  Account ID: 7997a29f-0ceb-4cc7-9adf-840c57b4ae6f                   ║
zonemercy-acute-nemo-v1-4488-v5-mkmlizer: ║  Expiration: 2024-10-15 23:59:59                                    ║
zonemercy-acute-nemo-v1-4488-v5-mkmlizer: ║                                                                     ║
zonemercy-acute-nemo-v1-4488-v5-mkmlizer: ╚═════════════════════════════════════════════════════════════════════╝
zonemercy-acute-nemo-v1-4488-v5-mkmlizer: Downloaded to shared memory in 52.027s
zonemercy-acute-nemo-v1-4488-v5-mkmlizer: quantizing model to /dev/shm/model_cache, profile:s0, folder:/tmp/tmp2ow2y1wq, device:0
zonemercy-acute-nemo-v1-4488-v5-mkmlizer: Saving flywheel model at /dev/shm/model_cache
zonemercy-acute-nemo-v1-4488-v5-mkmlizer: quantized model in 41.366s
zonemercy-acute-nemo-v1-4488-v5-mkmlizer: Processed model zonemercy/Acute-Nemo-v1-1e5ep1 in 93.393s
zonemercy-acute-nemo-v1-4488-v5-mkmlizer: creating bucket guanaco-mkml-models
zonemercy-acute-nemo-v1-4488-v5-mkmlizer: Bucket 's3://guanaco-mkml-models/' created
zonemercy-acute-nemo-v1-4488-v5-mkmlizer: uploading /dev/shm/model_cache to s3://guanaco-mkml-models/zonemercy-acute-nemo-v1-4488-v5
zonemercy-acute-nemo-v1-4488-v5-mkmlizer: cp /dev/shm/model_cache/special_tokens_map.json s3://guanaco-mkml-models/zonemercy-acute-nemo-v1-4488-v5/special_tokens_map.json
zonemercy-acute-nemo-v1-4488-v5-mkmlizer: cp /dev/shm/model_cache/config.json s3://guanaco-mkml-models/zonemercy-acute-nemo-v1-4488-v5/config.json
zonemercy-acute-nemo-v1-4488-v5-mkmlizer: cp /dev/shm/model_cache/tokenizer_config.json s3://guanaco-mkml-models/zonemercy-acute-nemo-v1-4488-v5/tokenizer_config.json
zonemercy-acute-nemo-v1-4488-v5-mkmlizer: cp /dev/shm/model_cache/tokenizer.json s3://guanaco-mkml-models/zonemercy-acute-nemo-v1-4488-v5/tokenizer.json
Failed to get response for submission function_fahor_2024-08-15: no entry with id "function_fahor_2024-08-15" found on database!
zonemercy-acute-nemo-v1-4488-v5-mkmlizer: cp /dev/shm/model_cache/flywheel_model.0.safetensors s3://guanaco-mkml-models/zonemercy-acute-nemo-v1-4488-v5/flywheel_model.0.safetensors
zonemercy-acute-nemo-v1-4488-v5-mkmlizer: 
Loading 0:   0%|          | 0/363 [00:00<?, ?it/s]
Loading 0:   1%|▏         | 5/363 [00:00<00:16, 22.14it/s]
Loading 0:   3%|▎         | 10/363 [00:00<00:12, 27.47it/s]
Loading 0:   4%|▍         | 14/363 [00:00<00:14, 24.27it/s]
Loading 0:   6%|▌         | 20/363 [00:00<00:10, 32.77it/s]
Loading 0:   7%|▋         | 24/363 [00:00<00:15, 22.52it/s]
Loading 0:   7%|▋         | 27/363 [00:01<00:15, 21.70it/s]
Loading 0:   9%|▊         | 31/363 [00:01<00:13, 25.35it/s]
Loading 0:   9%|▉         | 34/363 [00:01<00:13, 25.24it/s]
Loading 0:  11%|█         | 39/363 [00:01<00:11, 28.10it/s]
Loading 0:  12%|█▏        | 43/363 [00:01<00:11, 26.83it/s]
Loading 0:  13%|█▎        | 48/363 [00:01<00:10, 28.97it/s]
Loading 0:  14%|█▍        | 52/363 [00:01<00:11, 27.47it/s]
Loading 0:  15%|█▌        | 56/363 [00:02<00:11, 27.28it/s]
Loading 0:  17%|█▋        | 60/363 [00:02<00:10, 30.00it/s]
Loading 0:  18%|█▊        | 64/363 [00:02<00:15, 19.56it/s]
Loading 0:  19%|█▉        | 69/363 [00:02<00:12, 24.47it/s]
Loading 0:  20%|██        | 73/363 [00:02<00:11, 24.62it/s]
Loading 0:  21%|██        | 77/363 [00:03<00:12, 22.88it/s]
Loading 0:  23%|██▎       | 82/363 [00:03<00:10, 27.86it/s]
Loading 0:  24%|██▎       | 86/363 [00:03<00:11, 24.79it/s]
Loading 0:  25%|██▌       | 91/363 [00:03<00:09, 29.60it/s]
Loading 0:  26%|██▌       | 95/363 [00:03<00:10, 25.97it/s]
Loading 0:  28%|██▊       | 100/363 [00:03<00:08, 30.67it/s]
Loading 0:  29%|██▊       | 104/363 [00:04<00:12, 20.15it/s]
Loading 0:  31%|███       | 111/363 [00:04<00:09, 26.28it/s]
Loading 0:  32%|███▏      | 115/363 [00:04<00:09, 25.88it/s]
Loading 0:  33%|███▎      | 120/363 [00:04<00:08, 28.08it/s]
Loading 0:  34%|███▍      | 124/363 [00:04<00:08, 27.18it/s]
Loading 0:  36%|███▌      | 129/363 [00:04<00:07, 29.43it/s]
Loading 0:  37%|███▋      | 133/363 [00:05<00:08, 27.72it/s]
Loading 0:  38%|███▊      | 137/363 [00:05<00:08, 27.50it/s]
Loading 0:  39%|███▉      | 141/363 [00:05<00:07, 29.86it/s]
Loading 0:  40%|███▉      | 145/363 [00:05<00:10, 20.93it/s]
Loading 0:  41%|████      | 149/363 [00:05<00:10, 20.58it/s]
Loading 0:  42%|████▏     | 154/363 [00:05<00:08, 25.20it/s]
Loading 0:  44%|████▎     | 158/363 [00:06<00:08, 23.32it/s]
Loading 0:  45%|████▍     | 163/363 [00:06<00:07, 28.10it/s]
Loading 0:  46%|████▌     | 167/363 [00:06<00:07, 25.21it/s]
Loading 0:  47%|████▋     | 172/363 [00:06<00:06, 30.00it/s]
Loading 0:  48%|████▊     | 176/363 [00:06<00:07, 25.94it/s]
Loading 0:  50%|████▉     | 181/363 [00:06<00:05, 30.57it/s]
Loading 0:  51%|█████     | 185/363 [00:07<00:08, 20.19it/s]
Loading 0:  53%|█████▎    | 192/363 [00:07<00:06, 26.21it/s]
Loading 0:  54%|█████▍    | 196/363 [00:07<00:06, 25.59it/s]
Loading 0:  55%|█████▌    | 201/363 [00:07<00:05, 27.79it/s]
Loading 0:  56%|█████▋    | 205/363 [00:07<00:05, 26.86it/s]
Loading 0:  58%|█████▊    | 210/363 [00:08<00:05, 28.71it/s]
Loading 0:  59%|█████▉    | 214/363 [00:08<00:05, 27.50it/s]
Loading 0:  60%|██████    | 218/363 [00:08<00:05, 27.60it/s]
Loading 0:  61%|██████    | 222/363 [00:08<00:04, 29.67it/s]
Loading 0:  62%|██████▏   | 226/363 [00:08<00:06, 20.97it/s]
Loading 0:  63%|██████▎   | 230/363 [00:09<00:06, 20.58it/s]
Loading 0:  65%|██████▍   | 235/363 [00:09<00:05, 25.54it/s]
Loading 0:  66%|██████▌   | 239/363 [00:09<00:05, 23.43it/s]
Loading 0:  67%|██████▋   | 244/363 [00:09<00:04, 28.39it/s]
Loading 0:  68%|██████▊   | 248/363 [00:09<00:04, 25.15it/s]
Loading 0:  70%|██████▉   | 253/363 [00:09<00:03, 29.62it/s]
Loading 0:  71%|███████   | 257/363 [00:09<00:04, 25.93it/s]
Loading 0:  72%|███████▏  | 262/363 [00:10<00:03, 30.76it/s]
Loading 0:  73%|███████▎  | 266/363 [00:10<00:04, 20.20it/s]
Loading 0:  75%|███████▌  | 273/363 [00:10<00:03, 26.18it/s]
Loading 0:  76%|███████▋  | 277/363 [00:10<00:03, 25.78it/s]
Loading 0:  78%|███████▊  | 282/363 [00:10<00:02, 28.36it/s]
Loading 0:  79%|███████▉  | 286/363 [00:11<00:02, 27.62it/s]
Loading 0:  80%|████████  | 291/363 [00:11<00:02, 29.83it/s]
Loading 0:  81%|████████▏ | 295/363 [00:11<00:02, 28.79it/s]
Loading 0:  82%|████████▏ | 299/363 [00:11<00:02, 28.34it/s]
Loading 0:  84%|████████▎ | 304/363 [00:11<00:02, 24.67it/s]
Loading 0:  85%|████████▍ | 307/363 [00:11<00:02, 23.34it/s]
Loading 0:  86%|████████▌ | 311/363 [00:12<00:02, 22.21it/s]
Loading 0:  87%|████████▋ | 316/363 [00:12<00:01, 27.46it/s]
Loading 0:  88%|████████▊ | 320/363 [00:12<00:01, 24.63it/s]
Loading 0:  90%|████████▉ | 325/363 [00:12<00:01, 29.59it/s]
Loading 0:  91%|█████████ | 329/363 [00:12<00:01, 26.01it/s]
Loading 0:  92%|█████████▏| 334/363 [00:12<00:00, 30.64it/s]
Loading 0:  93%|█████████▎| 338/363 [00:13<00:00, 26.38it/s]
Loading 0:  94%|█████████▍| 343/363 [00:13<00:00, 30.95it/s]
Loading 0:  96%|█████████▌| 347/363 [00:20<00:07,  2.00it/s]
Loading 0:  96%|█████████▋| 350/363 [00:20<00:05,  2.52it/s]
Loading 0:  97%|█████████▋| 353/363 [00:20<00:03,  3.21it/s]
Loading 0:  98%|█████████▊| 357/363 [00:20<00:01,  4.39it/s]
                                                            
Job zonemercy-acute-nemo-v1-4488-v5-mkmlizer completed after 114.28s with status: succeeded
Stopping job with name zonemercy-acute-nemo-v1-4488-v5-mkmlizer
Pipeline stage MKMLizer completed in 115.26s
Running pipeline stage MKMLKubeTemplater
Pipeline stage MKMLKubeTemplater completed in 0.13s
Running pipeline stage ISVCDeployer
Creating inference service zonemercy-acute-nemo-v1-4488-v5
Waiting for inference service zonemercy-acute-nemo-v1-4488-v5 to be ready
Failed to get response for submission function_fahor_2024-08-15: no entry with id "function_fahor_2024-08-15" found on database!
Failed to get response for submission function_fahor_2024-08-15: no entry with id "function_fahor_2024-08-15" found on database!
Failed to get response for submission function_fahor_2024-08-15: no entry with id "function_fahor_2024-08-15" found on database!
Failed to get response for submission function_fahor_2024-08-15: no entry with id "function_fahor_2024-08-15" found on database!
Failed to get response for submission function_fahor_2024-08-15: no entry with id "function_fahor_2024-08-15" found on database!
Failed to get response for submission function_fahor_2024-08-15: no entry with id "function_fahor_2024-08-15" found on database!
Failed to get response for submission function_fahor_2024-08-15: no entry with id "function_fahor_2024-08-15" found on database!
Inference service zonemercy-acute-nemo-v1-4488-v5 ready after 221.42639923095703s
Pipeline stage ISVCDeployer completed in 223.41s
Running pipeline stage StressChecker
Received healthy response to inference request in 2.625415563583374s
{"detail":"('http://chaiml-llama-8b-pairwise-8189-v4-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', '{\"error\":\"ValueError : [TypeError(\\\\\"\\'numpy.int64\\' object is not iterable\\\\\"), TypeError(\\'vars() argument must have __dict__ attribute\\')]\"}')"}
Received unhealthy response to inference request!
Received healthy response to inference request in 2.0944831371307373s
Received healthy response to inference request in 1.9779491424560547s
Received healthy response to inference request in 2.0836713314056396s
5 requests
1 failed requests
5th percentile: 1.9990935802459717
10th percentile: 2.0202380180358888
20th percentile: 2.062526893615723
30th percentile: 2.085833692550659
40th percentile: 2.0901584148406984
50th percentile: 2.0944831371307373
60th percentile: 2.1964763164520265
70th percentile: 2.2984694957733156
80th percentile: 2.404655981063843
90th percentile: 2.5150357723236083
95th percentile: 2.570225667953491
99th percentile: 2.6143775844573973
mean time: 2.226197052001953
%s, retrying in %s seconds...
Received healthy response to inference request in 2.180675506591797s
Received healthy response to inference request in 2.3099584579467773s
Received healthy response to inference request in 2.4734270572662354s
Received healthy response to inference request in 2.6105117797851562s
Received healthy response to inference request in 1.8715004920959473s
5 requests
0 failed requests
5th percentile: 1.9333354949951171
10th percentile: 1.995170497894287
20th percentile: 2.1188405036926268
30th percentile: 2.206532096862793
40th percentile: 2.258245277404785
50th percentile: 2.3099584579467773
60th percentile: 2.3753458976745607
70th percentile: 2.4407333374023437
80th percentile: 2.5008440017700195
90th percentile: 2.555677890777588
95th percentile: 2.583094835281372
99th percentile: 2.6050283908843994
mean time: 2.2892146587371824
Pipeline stage StressChecker completed in 24.15s
zonemercy-acute-nemo-v1-_4488_v5 status is now deployed due to DeploymentManager action
zonemercy-acute-nemo-v1-_4488_v5 status is now inactive due to admin request
zonemercy-acute-nemo-v1-_4488_v5 status is now torndown due to DeploymentManager action