developer_uid: zonemercy
submission_id: zonemercy-graft-cogent-n_9815_v5
model_name: 0815v1-3
model_group: zonemercy/Graft-Cogent-N
status: torndown
timestamp: 2024-08-15T18:39:40+00:00
num_battles: 28821
num_wins: 15882
celo_rating: 1284.39
family_friendly_score: 0.0
submission_type: basic
model_repo: zonemercy/Graft-Cogent-Nemo-v2.1-5e6
model_architecture: MistralForCausalLM
model_num_parameters: 12772070400.0
best_of: 16
max_input_tokens: 512
max_output_tokens: 64
display_name: 0815v1-3
is_internal_developer: True
language_model: zonemercy/Graft-Cogent-Nemo-v2.1-5e6
model_size: 13B
ranking_group: single
us_pacific_date: 2024-08-15
win_ratio: 0.5510565212865619
generation_params: {'temperature': 1.0, 'top_p': 1.0, 'min_p': 0.08, 'top_k': 80, 'presence_penalty': 0.0, 'frequency_penalty': 0.0, 'stopping_words': ['\n', '</s>', '###', 'Bot:', 'User:', 'You:', '<|im_end|>'], 'max_input_tokens': 512, 'best_of': 16, 'max_output_tokens': 64}
formatter: {'memory_template': "{bot_name}'s Persona: {memory}\n####\n", 'prompt_template': '{prompt}\n<START>\n', 'bot_template': '{bot_name}: {message}\n', 'user_template': '{user_name}: {message}\n', 'response_template': '{bot_name}:', 'truncate_by_message': False}
Resubmit model
Running pipeline stage MKMLizer
Starting job with name zonemercy-graft-cogent-n-9815-v5-mkmlizer
Waiting for job on zonemercy-graft-cogent-n-9815-v5-mkmlizer to finish
Stopping job with name zonemercy-graft-cogent-n-9815-v5-mkmlizer
%s, retrying in %s seconds...
Starting job with name zonemercy-graft-cogent-n-9815-v5-mkmlizer
Waiting for job on zonemercy-graft-cogent-n-9815-v5-mkmlizer to finish
zonemercy-graft-cogent-n-9815-v5-mkmlizer: ╔═════════════════════════════════════════════════════════════════════╗
zonemercy-graft-cogent-n-9815-v5-mkmlizer: ║     _____            __           __                                ║
zonemercy-graft-cogent-n-9815-v5-mkmlizer: ║    / _/ /_ ___    __/ /  ___ ___ / /                                ║
zonemercy-graft-cogent-n-9815-v5-mkmlizer: ║   / _/ / // / |/|/ / _ \/ -_) -_) /                                 ║
zonemercy-graft-cogent-n-9815-v5-mkmlizer: ║  /_//_/\_, /|__,__/_//_/\__/\__/_/                                  ║
zonemercy-graft-cogent-n-9815-v5-mkmlizer: ║       /___/                                                         ║
zonemercy-graft-cogent-n-9815-v5-mkmlizer: ║                                                                     ║
zonemercy-graft-cogent-n-9815-v5-mkmlizer: ║  Version: 0.9.11                                                    ║
zonemercy-graft-cogent-n-9815-v5-mkmlizer: ║  Copyright 2023 MK ONE TECHNOLOGIES Inc.                            ║
zonemercy-graft-cogent-n-9815-v5-mkmlizer: ║  https://mk1.ai                                                     ║
zonemercy-graft-cogent-n-9815-v5-mkmlizer: ║                                                                     ║
zonemercy-graft-cogent-n-9815-v5-mkmlizer: ║  The license key for the current software has been verified as      ║
zonemercy-graft-cogent-n-9815-v5-mkmlizer: ║  belonging to:                                                      ║
zonemercy-graft-cogent-n-9815-v5-mkmlizer: ║                                                                     ║
zonemercy-graft-cogent-n-9815-v5-mkmlizer: ║  Chai Research Corp.                                                ║
zonemercy-graft-cogent-n-9815-v5-mkmlizer: ║  Account ID: 7997a29f-0ceb-4cc7-9adf-840c57b4ae6f                   ║
zonemercy-graft-cogent-n-9815-v5-mkmlizer: ║  Expiration: 2024-10-15 23:59:59                                    ║
zonemercy-graft-cogent-n-9815-v5-mkmlizer: ║                                                                     ║
zonemercy-graft-cogent-n-9815-v5-mkmlizer: ╚═════════════════════════════════════════════════════════════════════╝
Failed to get response for submission function_fahor_2024-08-15: no entry with id "function_fahor_2024-08-15" found on database!
Failed to get response for submission function_fahor_2024-08-15: no entry with id "function_fahor_2024-08-15" found on database!
zonemercy-graft-cogent-n-9815-v5-mkmlizer: Downloaded to shared memory in 59.046s
zonemercy-graft-cogent-n-9815-v5-mkmlizer: quantizing model to /dev/shm/model_cache, profile:s0, folder:/tmp/tmpptt_3kon, device:0
zonemercy-graft-cogent-n-9815-v5-mkmlizer: Saving flywheel model at /dev/shm/model_cache
Failed to get response for submission function_fahor_2024-08-15: no entry with id "function_fahor_2024-08-15" found on database!
zonemercy-graft-cogent-n-9815-v5-mkmlizer: quantized model in 42.531s
zonemercy-graft-cogent-n-9815-v5-mkmlizer: Processed model zonemercy/Graft-Cogent-Nemo-v2.1-5e6 in 101.577s
zonemercy-graft-cogent-n-9815-v5-mkmlizer: creating bucket guanaco-mkml-models
zonemercy-graft-cogent-n-9815-v5-mkmlizer: Bucket 's3://guanaco-mkml-models/' created
zonemercy-graft-cogent-n-9815-v5-mkmlizer: uploading /dev/shm/model_cache to s3://guanaco-mkml-models/zonemercy-graft-cogent-n-9815-v5
zonemercy-graft-cogent-n-9815-v5-mkmlizer: cp /dev/shm/model_cache/config.json s3://guanaco-mkml-models/zonemercy-graft-cogent-n-9815-v5/config.json
zonemercy-graft-cogent-n-9815-v5-mkmlizer: cp /dev/shm/model_cache/special_tokens_map.json s3://guanaco-mkml-models/zonemercy-graft-cogent-n-9815-v5/special_tokens_map.json
zonemercy-graft-cogent-n-9815-v5-mkmlizer: cp /dev/shm/model_cache/tokenizer_config.json s3://guanaco-mkml-models/zonemercy-graft-cogent-n-9815-v5/tokenizer_config.json
zonemercy-graft-cogent-n-9815-v5-mkmlizer: cp /dev/shm/model_cache/tokenizer.json s3://guanaco-mkml-models/zonemercy-graft-cogent-n-9815-v5/tokenizer.json
zonemercy-graft-cogent-n-9815-v5-mkmlizer: cp /dev/shm/model_cache/flywheel_model.0.safetensors s3://guanaco-mkml-models/zonemercy-graft-cogent-n-9815-v5/flywheel_model.0.safetensors
zonemercy-graft-cogent-n-9815-v5-mkmlizer: 
Loading 0:   0%|          | 0/363 [00:00<?, ?it/s]
Loading 0:   1%|▏         | 5/363 [00:00<00:15, 23.54it/s]
Loading 0:   3%|▎         | 10/363 [00:00<00:11, 29.83it/s]
Loading 0:   4%|▍         | 14/363 [00:00<00:14, 23.76it/s]
Loading 0:   5%|▌         | 19/363 [00:00<00:11, 30.12it/s]
Loading 0:   6%|▋         | 23/363 [00:00<00:15, 22.44it/s]
Loading 0:   7%|▋         | 26/363 [00:01<00:16, 19.90it/s]
Loading 0:   9%|▊         | 31/363 [00:01<00:13, 25.50it/s]
Loading 0:  10%|▉         | 35/363 [00:01<00:12, 26.55it/s]
Loading 0:  11%|█         | 39/363 [00:01<00:11, 27.19it/s]
Loading 0:  12%|█▏        | 42/363 [00:01<00:13, 24.63it/s]
Loading 0:  13%|█▎        | 46/363 [00:01<00:11, 27.08it/s]
Loading 0:  13%|█▎        | 49/363 [00:01<00:11, 26.97it/s]
Loading 0:  14%|█▍        | 52/363 [00:02<00:11, 26.75it/s]
Loading 0:  15%|█▌        | 56/363 [00:02<00:11, 27.83it/s]
Loading 0:  17%|█▋        | 61/363 [00:02<00:12, 24.76it/s]
Loading 0:  18%|█▊        | 64/363 [00:02<00:14, 21.13it/s]
Loading 0:  20%|█▉        | 71/363 [00:02<00:10, 28.47it/s]
Loading 0:  21%|██        | 75/363 [00:02<00:10, 27.91it/s]
Loading 0:  21%|██▏       | 78/363 [00:03<00:10, 26.27it/s]
Loading 0:  23%|██▎       | 84/363 [00:03<00:09, 30.71it/s]
Loading 0:  24%|██▍       | 88/363 [00:03<00:09, 29.57it/s]
Loading 0:  26%|██▌       | 93/363 [00:03<00:08, 31.64it/s]
Loading 0:  27%|██▋       | 97/363 [00:03<00:08, 29.94it/s]
Loading 0:  28%|██▊       | 101/363 [00:03<00:10, 24.13it/s]
Loading 0:  29%|██▊       | 104/363 [00:04<00:12, 21.34it/s]
Loading 0:  31%|███       | 111/363 [00:04<00:08, 28.64it/s]
Loading 0:  32%|███▏      | 115/363 [00:04<00:08, 28.01it/s]
Loading 0:  33%|███▎      | 120/363 [00:04<00:08, 30.29it/s]
Loading 0:  34%|███▍      | 124/363 [00:04<00:08, 28.93it/s]
Loading 0:  36%|███▌      | 129/363 [00:04<00:07, 31.50it/s]
Loading 0:  37%|███▋      | 133/363 [00:04<00:07, 29.22it/s]
Loading 0:  38%|███▊      | 137/363 [00:05<00:07, 28.80it/s]
Loading 0:  39%|███▉      | 142/363 [00:05<00:08, 25.31it/s]
Loading 0:  40%|███▉      | 145/363 [00:05<00:09, 23.97it/s]
Loading 0:  41%|████      | 149/363 [00:05<00:09, 23.02it/s]
Loading 0:  43%|████▎     | 156/363 [00:05<00:06, 29.72it/s]
Loading 0:  44%|████▍     | 160/363 [00:05<00:07, 28.19it/s]
Loading 0:  45%|████▌     | 165/363 [00:06<00:06, 30.17it/s]
Loading 0:  47%|████▋     | 169/363 [00:06<00:06, 28.40it/s]
Loading 0:  48%|████▊     | 174/363 [00:06<00:06, 30.20it/s]
Loading 0:  49%|████▉     | 178/363 [00:06<00:06, 27.76it/s]
Loading 0:  50%|█████     | 182/363 [00:06<00:08, 22.62it/s]
Loading 0:  51%|█████     | 185/363 [00:07<00:08, 20.16it/s]
Loading 0:  53%|█████▎    | 192/363 [00:07<00:06, 26.88it/s]
Loading 0:  54%|█████▎    | 195/363 [00:07<00:06, 24.92it/s]
Loading 0:  55%|█████▌    | 201/363 [00:07<00:05, 29.31it/s]
Loading 0:  56%|█████▋    | 205/363 [00:07<00:05, 27.66it/s]
Loading 0:  58%|█████▊    | 210/363 [00:07<00:05, 28.95it/s]
Loading 0:  59%|█████▊    | 213/363 [00:07<00:05, 26.09it/s]
Loading 0:  60%|██████    | 218/363 [00:08<00:05, 28.29it/s]
Loading 0:  61%|██████▏   | 223/363 [00:08<00:05, 24.20it/s]
Loading 0:  62%|██████▏   | 226/363 [00:08<00:05, 23.04it/s]
Loading 0:  63%|██████▎   | 230/363 [00:08<00:06, 21.94it/s]
Loading 0:  65%|██████▌   | 237/363 [00:08<00:04, 28.17it/s]
Loading 0:  66%|██████▌   | 240/363 [00:09<00:04, 25.62it/s]
Loading 0:  67%|██████▋   | 244/363 [00:09<00:04, 28.35it/s]
Loading 0:  68%|██████▊   | 248/363 [00:09<00:04, 24.79it/s]
Loading 0:  70%|██████▉   | 253/363 [00:09<00:03, 29.84it/s]
Loading 0:  71%|███████   | 257/363 [00:09<00:04, 26.18it/s]
Loading 0:  72%|███████▏  | 263/363 [00:09<00:04, 24.74it/s]
Loading 0:  73%|███████▎  | 266/363 [00:10<00:04, 21.20it/s]
Loading 0:  75%|███████▌  | 273/363 [00:10<00:03, 27.40it/s]
Loading 0:  76%|███████▋  | 277/363 [00:10<00:03, 26.64it/s]
Loading 0:  78%|███████▊  | 282/363 [00:10<00:02, 28.76it/s]
Loading 0:  79%|███████▉  | 286/363 [00:10<00:02, 27.34it/s]
Loading 0:  80%|████████  | 291/363 [00:10<00:02, 29.49it/s]
Loading 0:  81%|████████▏ | 295/363 [00:11<00:02, 28.01it/s]
Loading 0:  82%|████████▏ | 298/363 [00:11<00:02, 27.86it/s]
Loading 0:  83%|████████▎ | 303/363 [00:11<00:02, 28.95it/s]
Loading 0:  84%|████████▍ | 306/363 [00:11<00:03, 18.92it/s]
Loading 0:  85%|████████▌ | 310/363 [00:11<00:02, 22.53it/s]
Loading 0:  86%|████████▌ | 313/363 [00:11<00:02, 22.90it/s]
Loading 0:  88%|████████▊ | 318/363 [00:12<00:01, 25.75it/s]
Loading 0:  88%|████████▊ | 321/363 [00:12<00:01, 23.60it/s]
Loading 0:  90%|████████▉ | 325/363 [00:12<00:01, 26.95it/s]
Loading 0:  90%|█████████ | 328/363 [00:12<00:01, 27.53it/s]
Loading 0:  91%|█████████ | 331/363 [00:12<00:01, 25.59it/s]
Loading 0:  93%|█████████▎| 336/363 [00:12<00:00, 28.33it/s]
Loading 0:  93%|█████████▎| 339/363 [00:12<00:00, 25.00it/s]
Loading 0:  94%|█████████▍| 343/363 [00:12<00:00, 27.96it/s]
Loading 0:  95%|█████████▌| 346/363 [00:19<00:10,  1.62it/s]
Loading 0:  96%|█████████▌| 349/363 [00:20<00:06,  2.13it/s]
Loading 0:  97%|█████████▋| 353/363 [00:20<00:03,  3.09it/s]
Loading 0:  98%|█████████▊| 357/363 [00:20<00:01,  4.24it/s]
Loading 0: 100%|█████████▉| 362/363 [00:20<00:00,  6.31it/s]
                                                            

Job zonemercy-graft-cogent-n-9815-v5-mkmlizer completed after 160.65s with status: succeeded
Stopping job with name zonemercy-graft-cogent-n-9815-v5-mkmlizer
Pipeline stage MKMLizer completed in 162.58s
Running pipeline stage MKMLKubeTemplater
Pipeline stage MKMLKubeTemplater completed in 0.39s
Running pipeline stage ISVCDeployer
Creating inference service zonemercy-graft-cogent-n-9815-v5
Waiting for inference service zonemercy-graft-cogent-n-9815-v5 to be ready
Failed to get response for submission function_fahor_2024-08-15: no entry with id "function_fahor_2024-08-15" found on database!
Failed to get response for submission function_fahor_2024-08-15: no entry with id "function_fahor_2024-08-15" found on database!
Failed to get response for submission function_fahor_2024-08-15: no entry with id "function_fahor_2024-08-15" found on database!
Failed to get response for submission function_fahor_2024-08-15: no entry with id "function_fahor_2024-08-15" found on database!
Inference service zonemercy-graft-cogent-n-9815-v5 ready after 231.68720126152039s
Pipeline stage ISVCDeployer completed in 233.31s
Running pipeline stage StressChecker
Received healthy response to inference request in 13.326469898223877s
Failed to get response for submission function_fahor_2024-08-15: no entry with id "function_fahor_2024-08-15" found on database!
Received healthy response to inference request in 9.15270447731018s
Received healthy response to inference request in 8.782462120056152s
Received healthy response to inference request in 4.681170463562012s
Received healthy response to inference request in 5.437134265899658s
5 requests
0 failed requests
5th percentile: 4.832363224029541
10th percentile: 4.9835559844970705
20th percentile: 5.285941505432129
30th percentile: 6.106199836730957
40th percentile: 7.444330978393555
50th percentile: 8.782462120056152
60th percentile: 8.930559062957764
70th percentile: 9.078656005859376
80th percentile: 9.98745756149292
90th percentile: 11.656963729858399
95th percentile: 12.491716814041137
99th percentile: 13.159519281387329
mean time: 8.275988245010376
%s, retrying in %s seconds...
Received healthy response to inference request in 1.964191198348999s
Received healthy response to inference request in 6.190676927566528s
Received healthy response to inference request in 4.549346208572388s
Received healthy response to inference request in 1.9217374324798584s
Failed to get response for submission blend_renor_2024-08-15: ('http://chaiml-elo-alignment-run-3-v19-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'read tcp 127.0.0.1:36138->127.0.0.1:8080: read: connection reset by peer\n')
Received healthy response to inference request in 1.945383071899414s
5 requests
0 failed requests
5th percentile: 1.9264665603637696
10th percentile: 1.9311956882476806
20th percentile: 1.9406539440155028
30th percentile: 1.9491446971893311
40th percentile: 1.956667947769165
50th percentile: 1.964191198348999
60th percentile: 2.998253202438354
70th percentile: 4.032315206527709
80th percentile: 4.877612352371216
90th percentile: 5.534144639968872
95th percentile: 5.8624107837677
99th percentile: 6.125023698806762
mean time: 3.3142669677734373
Pipeline stage StressChecker completed in 59.54s
zonemercy-graft-cogent-n_9815_v5 status is now deployed due to DeploymentManager action
zonemercy-graft-cogent-n_9815_v5 status is now inactive due to admin request
zonemercy-graft-cogent-n_9815_v5 status is now torndown due to DeploymentManager action