zonemercy-acute-nemo-v0-_5087

developer_uid: zonemercy
submission_id: zonemercy-acute-nemo-v0-_5087_v5
model_name: 0815v3-1
model_group: zonemercy/Acute-Nemo-v0-
status: torndown
timestamp: 2024-08-17T06:08:41+00:00
num_battles: 121773
num_wins: 61430
celo_rating: 1287.49
family_friendly_score: 0.0
submission_type: basic
model_repo: zonemercy/Acute-Nemo-v0-5e6ep1
model_architecture: MistralForCausalLM
model_num_parameters: 12772070400.0
best_of: 32
max_input_tokens: 512
max_output_tokens: 64
display_name: 0815v3-1
is_internal_developer: True
language_model: zonemercy/Acute-Nemo-v0-5e6ep1
model_size: 13B
ranking_group: single
us_pacific_date: 2024-08-16
win_ratio: 0.5044632225534396
generation_params: {'temperature': 1.0, 'top_p': 1.0, 'min_p': 0.08, 'top_k': 80, 'presence_penalty': 0.0, 'frequency_penalty': 0.0, 'stopping_words': ['\n', '</s>', '###', 'Bot:', 'User:', 'You:', '<|im_end|>'], 'max_input_tokens': 512, 'best_of': 32, 'max_output_tokens': 64}
formatter: {'memory_template': "{bot_name}'s Persona: {memory}\n####\n", 'prompt_template': '{prompt}\n<START>\n', 'bot_template': '{bot_name}: {message}\n', 'user_template': '{user_name}: {message}\n', 'response_template': '{bot_name}:', 'truncate_by_message': False}
Resubmit model
Running pipeline stage MKMLizer
Starting job with name zonemercy-acute-nemo-v0-5087-v5-mkmlizer
Waiting for job on zonemercy-acute-nemo-v0-5087-v5-mkmlizer to finish
zonemercy-acute-nemo-v0-5087-v5-mkmlizer: ╔═════════════════════════════════════════════════════════════════════╗
zonemercy-acute-nemo-v0-5087-v5-mkmlizer: ║     _____            __           __                                ║
zonemercy-acute-nemo-v0-5087-v5-mkmlizer: ║    / _/ /_ ___    __/ /  ___ ___ / /                                ║
zonemercy-acute-nemo-v0-5087-v5-mkmlizer: ║   / _/ / // / |/|/ / _ \/ -_) -_) /                                 ║
zonemercy-acute-nemo-v0-5087-v5-mkmlizer: ║  /_//_/\_, /|__,__/_//_/\__/\__/_/                                  ║
zonemercy-acute-nemo-v0-5087-v5-mkmlizer: ║       /___/                                                         ║
zonemercy-acute-nemo-v0-5087-v5-mkmlizer: ║                                                                     ║
zonemercy-acute-nemo-v0-5087-v5-mkmlizer: ║  Version: 0.9.11                                                    ║
zonemercy-acute-nemo-v0-5087-v5-mkmlizer: ║  Copyright 2023 MK ONE TECHNOLOGIES Inc.                            ║
zonemercy-acute-nemo-v0-5087-v5-mkmlizer: ║  https://mk1.ai                                                     ║
zonemercy-acute-nemo-v0-5087-v5-mkmlizer: ║                                                                     ║
zonemercy-acute-nemo-v0-5087-v5-mkmlizer: ║  The license key for the current software has been verified as      ║
zonemercy-acute-nemo-v0-5087-v5-mkmlizer: ║  belonging to:                                                      ║
zonemercy-acute-nemo-v0-5087-v5-mkmlizer: ║                                                                     ║
zonemercy-acute-nemo-v0-5087-v5-mkmlizer: ║  Chai Research Corp.                                                ║
zonemercy-acute-nemo-v0-5087-v5-mkmlizer: ║  Account ID: 7997a29f-0ceb-4cc7-9adf-840c57b4ae6f                   ║
zonemercy-acute-nemo-v0-5087-v5-mkmlizer: ║  Expiration: 2024-10-15 23:59:59                                    ║
zonemercy-acute-nemo-v0-5087-v5-mkmlizer: ║                                                                     ║
zonemercy-acute-nemo-v0-5087-v5-mkmlizer: ╚═════════════════════════════════════════════════════════════════════╝
Failed to get response for submission undi95-meta-llama-3-70b_6209_v19: ('http://undi95-meta-llama-3-70b-6209-v19-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'request timeout')
Failed to get response for submission undi95-meta-llama-3-70b_6209_v19: ('http://undi95-meta-llama-3-70b-6209-v19-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', '{"error":"TypeError : SamplingParameters.__init__() got an unexpected keyword argument \'reward_max_tokens\'"}')
Failed to get response for submission undi95-meta-llama-3-70b_6209_v19: ('http://undi95-meta-llama-3-70b-6209-v19-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'request timeout')
zonemercy-acute-nemo-v0-5087-v5-mkmlizer: Downloaded to shared memory in 71.562s
zonemercy-acute-nemo-v0-5087-v5-mkmlizer: quantizing model to /dev/shm/model_cache, profile:s0, folder:/tmp/tmp87lh_adz, device:0
zonemercy-acute-nemo-v0-5087-v5-mkmlizer: Saving flywheel model at /dev/shm/model_cache
Failed to get response for submission undi95-meta-llama-3-70b_6209_v19: ('http://undi95-meta-llama-3-70b-6209-v19-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'request timeout')
zonemercy-acute-nemo-v0-5087-v5-mkmlizer: quantized model in 41.274s
zonemercy-acute-nemo-v0-5087-v5-mkmlizer: Processed model zonemercy/Acute-Nemo-v0-5e6ep1 in 112.837s
zonemercy-acute-nemo-v0-5087-v5-mkmlizer: creating bucket guanaco-mkml-models
zonemercy-acute-nemo-v0-5087-v5-mkmlizer: Bucket 's3://guanaco-mkml-models/' created
zonemercy-acute-nemo-v0-5087-v5-mkmlizer: uploading /dev/shm/model_cache to s3://guanaco-mkml-models/zonemercy-acute-nemo-v0-5087-v5
zonemercy-acute-nemo-v0-5087-v5-mkmlizer: cp /dev/shm/model_cache/config.json s3://guanaco-mkml-models/zonemercy-acute-nemo-v0-5087-v5/config.json
zonemercy-acute-nemo-v0-5087-v5-mkmlizer: cp /dev/shm/model_cache/special_tokens_map.json s3://guanaco-mkml-models/zonemercy-acute-nemo-v0-5087-v5/special_tokens_map.json
zonemercy-acute-nemo-v0-5087-v5-mkmlizer: cp /dev/shm/model_cache/tokenizer_config.json s3://guanaco-mkml-models/zonemercy-acute-nemo-v0-5087-v5/tokenizer_config.json
zonemercy-acute-nemo-v0-5087-v5-mkmlizer: cp /dev/shm/model_cache/tokenizer.json s3://guanaco-mkml-models/zonemercy-acute-nemo-v0-5087-v5/tokenizer.json
zonemercy-acute-nemo-v0-5087-v5-mkmlizer: cp /dev/shm/model_cache/flywheel_model.0.safetensors s3://guanaco-mkml-models/zonemercy-acute-nemo-v0-5087-v5/flywheel_model.0.safetensors
zonemercy-acute-nemo-v0-5087-v5-mkmlizer: 
Loading 0:   0%|          | 0/363 [00:00<?, ?it/s]
Loading 0:   1%|▏         | 5/363 [00:00<00:16, 22.26it/s]
Loading 0:   3%|▎         | 10/363 [00:00<00:13, 25.88it/s]
Loading 0:   4%|▍         | 14/363 [00:00<00:14, 23.54it/s]
Loading 0:   5%|▌         | 19/363 [00:00<00:11, 30.08it/s]
Loading 0:   6%|▋         | 23/363 [00:00<00:15, 22.45it/s]
Loading 0:   7%|▋         | 26/363 [00:01<00:16, 20.05it/s]
Loading 0:   9%|▊         | 31/363 [00:01<00:12, 25.87it/s]
Loading 0:  10%|▉         | 35/363 [00:01<00:12, 26.49it/s]
Loading 0:  11%|█         | 39/363 [00:01<00:11, 27.64it/s]
Loading 0:  12%|█▏        | 43/363 [00:01<00:11, 26.73it/s]
Loading 0:  13%|█▎        | 48/363 [00:01<00:10, 29.12it/s]
Loading 0:  14%|█▍        | 52/363 [00:01<00:11, 27.73it/s]
Loading 0:  15%|█▌        | 56/363 [00:02<00:11, 27.73it/s]
Loading 0:  17%|█▋        | 60/363 [00:02<00:10, 30.28it/s]
Loading 0:  18%|█▊        | 64/363 [00:02<00:15, 19.76it/s]
Loading 0:  19%|█▉        | 69/363 [00:02<00:11, 24.96it/s]
Loading 0:  20%|██        | 73/363 [00:02<00:11, 25.01it/s]
Loading 0:  21%|██        | 77/363 [00:03<00:12, 23.58it/s]
Loading 0:  23%|██▎       | 82/363 [00:03<00:10, 27.93it/s]
Loading 0:  24%|██▎       | 86/363 [00:03<00:11, 25.05it/s]
Loading 0:  25%|██▌       | 91/363 [00:03<00:09, 29.96it/s]
Loading 0:  26%|██▌       | 95/363 [00:03<00:10, 26.14it/s]
Loading 0:  28%|██▊       | 100/363 [00:03<00:08, 30.74it/s]
Loading 0:  29%|██▊       | 104/363 [00:04<00:12, 19.97it/s]
Loading 0:  30%|███       | 109/363 [00:04<00:10, 24.79it/s]
Loading 0:  31%|███       | 113/363 [00:04<00:10, 23.63it/s]
Loading 0:  33%|███▎      | 120/363 [00:04<00:08, 29.68it/s]
Loading 0:  34%|███▍      | 124/363 [00:04<00:08, 28.48it/s]
Loading 0:  36%|███▌      | 129/363 [00:04<00:07, 30.04it/s]
Loading 0:  37%|███▋      | 133/363 [00:05<00:08, 28.54it/s]
Loading 0:  38%|███▊      | 137/363 [00:05<00:07, 28.28it/s]
Loading 0:  39%|███▉      | 141/363 [00:05<00:07, 30.65it/s]
Loading 0:  40%|███▉      | 145/363 [00:05<00:10, 21.42it/s]
Loading 0:  41%|████      | 149/363 [00:05<00:10, 21.07it/s]
Loading 0:  42%|████▏     | 154/363 [00:05<00:08, 25.87it/s]
Loading 0:  44%|████▎     | 158/363 [00:06<00:08, 23.90it/s]
Loading 0:  45%|████▍     | 163/363 [00:06<00:06, 28.87it/s]
Loading 0:  46%|████▌     | 167/363 [00:06<00:07, 25.63it/s]
Loading 0:  47%|████▋     | 172/363 [00:06<00:06, 30.49it/s]
Loading 0:  48%|████▊     | 176/363 [00:06<00:06, 26.75it/s]
Loading 0:  50%|████▉     | 181/363 [00:06<00:05, 31.40it/s]
Loading 0:  51%|█████     | 185/363 [00:07<00:08, 20.68it/s]
Loading 0:  53%|█████▎    | 192/363 [00:07<00:06, 26.82it/s]
Loading 0:  54%|█████▍    | 196/363 [00:07<00:06, 26.42it/s]
Loading 0:  55%|█████▌    | 201/363 [00:07<00:05, 29.00it/s]
Loading 0:  56%|█████▋    | 205/363 [00:07<00:05, 27.41it/s]
Loading 0:  58%|█████▊    | 210/363 [00:07<00:05, 29.26it/s]
Loading 0:  59%|█████▉    | 214/363 [00:08<00:05, 28.12it/s]
Loading 0:  60%|██████    | 218/363 [00:08<00:05, 28.03it/s]
Loading 0:  61%|██████    | 222/363 [00:08<00:04, 30.60it/s]
Loading 0:  62%|██████▏   | 226/363 [00:08<00:06, 21.70it/s]
Loading 0:  63%|██████▎   | 230/363 [00:08<00:06, 21.10it/s]
Loading 0:  65%|██████▍   | 235/363 [00:09<00:04, 26.17it/s]
Loading 0:  66%|██████▌   | 239/363 [00:09<00:05, 24.37it/s]
Loading 0:  67%|██████▋   | 244/363 [00:09<00:04, 28.99it/s]
Loading 0:  68%|██████▊   | 248/363 [00:09<00:04, 25.99it/s]
Loading 0:  70%|██████▉   | 253/363 [00:09<00:03, 30.88it/s]
Loading 0:  71%|███████   | 257/363 [00:09<00:03, 26.77it/s]
Loading 0:  72%|███████▏  | 262/363 [00:09<00:03, 31.24it/s]
Loading 0:  73%|███████▎  | 266/363 [00:10<00:04, 21.06it/s]
Loading 0:  75%|███████▍  | 271/363 [00:10<00:03, 25.97it/s]
Loading 0:  76%|███████▌  | 275/363 [00:10<00:03, 24.22it/s]
Loading 0:  78%|███████▊  | 282/363 [00:10<00:02, 30.49it/s]
Loading 0:  79%|███████▉  | 286/363 [00:10<00:02, 29.15it/s]
Loading 0:  80%|████████  | 291/363 [00:11<00:02, 30.80it/s]
Loading 0:  81%|████████▏ | 295/363 [00:11<00:02, 29.20it/s]
Loading 0:  82%|████████▏ | 299/363 [00:11<00:02, 29.12it/s]
Loading 0:  84%|████████▎ | 304/363 [00:11<00:02, 24.97it/s]
Loading 0:  85%|████████▍ | 307/363 [00:11<00:02, 23.64it/s]
Loading 0:  86%|████████▌ | 311/363 [00:11<00:02, 22.64it/s]
Loading 0:  87%|████████▋ | 316/363 [00:12<00:01, 27.71it/s]
Loading 0:  88%|████████▊ | 320/363 [00:12<00:01, 25.59it/s]
Loading 0:  90%|█████████ | 327/363 [00:12<00:01, 31.46it/s]
Loading 0:  91%|█████████ | 331/363 [00:12<00:01, 29.63it/s]
Loading 0:  93%|█████████▎| 336/363 [00:12<00:00, 31.19it/s]
Loading 0:  94%|█████████▎| 340/363 [00:12<00:00, 29.45it/s]
Loading 0:  95%|█████████▍| 344/363 [00:19<00:09,  2.01it/s]
Loading 0:  96%|█████████▌| 348/363 [00:19<00:05,  2.70it/s]
Loading 0:  97%|█████████▋| 353/363 [00:20<00:02,  3.89it/s]
Loading 0:  98%|█████████▊| 357/363 [00:20<00:01,  5.02it/s]
                                                            
Job zonemercy-acute-nemo-v0-5087-v5-mkmlizer completed after 137.08s with status: succeeded
Stopping job with name zonemercy-acute-nemo-v0-5087-v5-mkmlizer
Pipeline stage MKMLizer completed in 138.23s
Running pipeline stage MKMLKubeTemplater
Pipeline stage MKMLKubeTemplater completed in 0.09s
Running pipeline stage ISVCDeployer
Creating inference service zonemercy-acute-nemo-v0-5087-v5
Waiting for inference service zonemercy-acute-nemo-v0-5087-v5 to be ready
Failed to get response for submission zonemercy-acute-nemo-v1_4488_v11: ('http://zonemercy-acute-nemo-v1-4488-v11-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'request timeout')
Failed to get response for submission undi95-meta-llama-3-70b_6209_v19: ('http://undi95-meta-llama-3-70b-6209-v19-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'request timeout')
Failed to get response for submission zonemercy-acute-nemo-v1_4488_v11: ('http://zonemercy-acute-nemo-v1-4488-v11-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'request timeout')
Failed to get response for submission zonemercy-acute-nemo-v1_4488_v11: ('http://zonemercy-acute-nemo-v1-4488-v11-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'EOF\n')
Failed to get response for submission zonemercy-acute-nemo-v1_4488_v11: ('http://zonemercy-acute-nemo-v1-4488-v11-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'EOF\n')
Failed to get response for submission zonemercy-acute-nemo-v1_4488_v11: ('http://zonemercy-acute-nemo-v1-4488-v11-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'EOF\n')
Failed to get response for submission zonemercy-acute-nemo-v1_4488_v11: ('http://zonemercy-acute-nemo-v1-4488-v11-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'EOF\n')
Failed to get response for submission zonemercy-acute-nemo-v1_4488_v10: ('http://zonemercy-acute-nemo-v1-4488-v10-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'request timeout')
Failed to get response for submission zonemercy-acute-nemo-v1_4488_v10: ('http://zonemercy-acute-nemo-v1-4488-v10-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'request timeout')
Failed to get response for submission zonemercy-acute-nemo-v1_4488_v10: ('http://zonemercy-acute-nemo-v1-4488-v10-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'read tcp 127.0.0.1:52490->127.0.0.1:8080: read: connection reset by peer\n')
Failed to get response for submission zonemercy-acute-nemo-v1_4488_v10: ('http://zonemercy-acute-nemo-v1-4488-v10-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'EOF\n')
Failed to get response for submission zonemercy-acute-nemo-v1_4488_v10: ('http://zonemercy-acute-nemo-v1-4488-v10-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'EOF\n')
Failed to get response for submission undi95-meta-llama-3-70b_6209_v19: ('http://undi95-meta-llama-3-70b-6209-v19-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'request timeout')
Inference service zonemercy-acute-nemo-v0-5087-v5 ready after 241.49882698059082s
Pipeline stage ISVCDeployer completed in 242.77s
Running pipeline stage StressChecker
Failed to get response for submission undi95-meta-llama-3-70b_6209_v19: ('http://undi95-meta-llama-3-70b-6209-v19-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'request timeout')
Received healthy response to inference request in 3.332423210144043s
Received healthy response to inference request in 3.2243340015411377s
Failed to get response for submission undi95-meta-llama-3-70b_6209_v19: ('http://undi95-meta-llama-3-70b-6209-v19-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'request timeout')
Received healthy response to inference request in 2.387697219848633s
Received healthy response to inference request in 2.449303388595581s
Received healthy response to inference request in 2.384058713912964s
5 requests
0 failed requests
5th percentile: 2.3847864151000975
10th percentile: 2.3855141162872315
20th percentile: 2.386969518661499
30th percentile: 2.4000184535980225
40th percentile: 2.4246609210968018
50th percentile: 2.449303388595581
60th percentile: 2.7593156337738036
70th percentile: 3.069327878952026
80th percentile: 3.245951843261719
90th percentile: 3.289187526702881
95th percentile: 3.3108053684234617
99th percentile: 3.328099641799927
mean time: 2.7555633068084715
Pipeline stage StressChecker completed in 14.75s
zonemercy-acute-nemo-v0-_5087_v5 status is now deployed due to DeploymentManager action
zonemercy-acute-nemo-v0-_5087_v5 status is now inactive due to auto deactivation removed underperforming models
zonemercy-acute-nemo-v0-_5087_v5 status is now torndown due to DeploymentManager action