zonemercy-graft-cogent-n_9815

developer_uid: zonemercy
submission_id: zonemercy-graft-cogent-n_9815_v9
model_name: 0815v1-3
model_group: zonemercy/Graft-Cogent-N
status: torndown
timestamp: 2024-08-15T20:53:15+00:00
num_battles: 16913
num_wins: 9074
celo_rating: 1271.85
family_friendly_score: 0.0
submission_type: basic
model_repo: zonemercy/Graft-Cogent-Nemo-v2.1-5e6
model_architecture: MistralForCausalLM
model_num_parameters: 12772070400.0
best_of: 8
max_input_tokens: 1024
max_output_tokens: 64
reward_model: default
display_name: 0815v1-3
is_internal_developer: True
language_model: zonemercy/Graft-Cogent-Nemo-v2.1-5e6
model_size: 13B
ranking_group: single
us_pacific_date: 2024-08-15
win_ratio: 0.536510376633359
generation_params: {'temperature': 0.9, 'top_p': 1.0, 'min_p': 0.05, 'top_k': 80, 'presence_penalty': 0.0, 'frequency_penalty': 0.0, 'stopping_words': ['You:', '\n', 'Bot:', '###', 'User:', '</s>', '<|im_end|>'], 'max_input_tokens': 1024, 'best_of': 8, 'max_output_tokens': 64}
formatter: {'memory_template': "{bot_name}'s Persona: {memory}\n####\n", 'prompt_template': '{prompt}\n<START>\n', 'bot_template': '{bot_name}: {message}\n', 'user_template': '{user_name}: {message}\n', 'response_template': '{bot_name}:', 'truncate_by_message': False}
Resubmit model
Running pipeline stage MKMLizer
Starting job with name zonemercy-graft-cogent-n-9815-v9-mkmlizer
Waiting for job on zonemercy-graft-cogent-n-9815-v9-mkmlizer to finish
Stopping job with name zonemercy-graft-cogent-n-9815-v9-mkmlizer
%s, retrying in %s seconds...
Starting job with name zonemercy-graft-cogent-n-9815-v9-mkmlizer
Waiting for job on zonemercy-graft-cogent-n-9815-v9-mkmlizer to finish
zonemercy-graft-cogent-n-9815-v9-mkmlizer: ╔═════════════════════════════════════════════════════════════════════╗
zonemercy-graft-cogent-n-9815-v9-mkmlizer: ║     _____            __           __                                ║
zonemercy-graft-cogent-n-9815-v9-mkmlizer: ║    / _/ /_ ___    __/ /  ___ ___ / /                                ║
zonemercy-graft-cogent-n-9815-v9-mkmlizer: ║   / _/ / // / |/|/ / _ \/ -_) -_) /                                 ║
zonemercy-graft-cogent-n-9815-v9-mkmlizer: ║  /_//_/\_, /|__,__/_//_/\__/\__/_/                                  ║
zonemercy-graft-cogent-n-9815-v9-mkmlizer: ║       /___/                                                         ║
zonemercy-graft-cogent-n-9815-v9-mkmlizer: ║                                                                     ║
zonemercy-graft-cogent-n-9815-v9-mkmlizer: ║  Version: 0.9.11                                                    ║
zonemercy-graft-cogent-n-9815-v9-mkmlizer: ║  Copyright 2023 MK ONE TECHNOLOGIES Inc.                            ║
zonemercy-graft-cogent-n-9815-v9-mkmlizer: ║  https://mk1.ai                                                     ║
zonemercy-graft-cogent-n-9815-v9-mkmlizer: ║                                                                     ║
zonemercy-graft-cogent-n-9815-v9-mkmlizer: ║  The license key for the current software has been verified as      ║
zonemercy-graft-cogent-n-9815-v9-mkmlizer: ║  belonging to:                                                      ║
zonemercy-graft-cogent-n-9815-v9-mkmlizer: ║                                                                     ║
zonemercy-graft-cogent-n-9815-v9-mkmlizer: ║  Chai Research Corp.                                                ║
zonemercy-graft-cogent-n-9815-v9-mkmlizer: ║  Account ID: 7997a29f-0ceb-4cc7-9adf-840c57b4ae6f                   ║
zonemercy-graft-cogent-n-9815-v9-mkmlizer: ║  Expiration: 2024-10-15 23:59:59                                    ║
zonemercy-graft-cogent-n-9815-v9-mkmlizer: ║                                                                     ║
zonemercy-graft-cogent-n-9815-v9-mkmlizer: ╚═════════════════════════════════════════════════════════════════════╝
Failed to get response for submission function_fahor_2024-08-15: no entry with id "function_fahor_2024-08-15" found on database!
zonemercy-graft-cogent-n-9815-v9-mkmlizer: Downloaded to shared memory in 73.615s
zonemercy-graft-cogent-n-9815-v9-mkmlizer: quantizing model to /dev/shm/model_cache, profile:s0, folder:/tmp/tmpp8s7xif2, device:0
zonemercy-graft-cogent-n-9815-v9-mkmlizer: Saving flywheel model at /dev/shm/model_cache
Failed to get response for submission v000000-l3-8b-poppy-moon_1166_v4: ('http://v000000-l3-8b-poppy-moon-1166-v4-predictor-default.tenant-chaiml-guanaco.knative.ord1.coreweave.cloud/v1/models/GPT-J-6B-lit-v2:predict', 'request timeout')
zonemercy-graft-cogent-n-9815-v9-mkmlizer: quantized model in 42.577s
zonemercy-graft-cogent-n-9815-v9-mkmlizer: Processed model zonemercy/Graft-Cogent-Nemo-v2.1-5e6 in 116.192s
zonemercy-graft-cogent-n-9815-v9-mkmlizer: creating bucket guanaco-mkml-models
zonemercy-graft-cogent-n-9815-v9-mkmlizer: Bucket 's3://guanaco-mkml-models/' created
zonemercy-graft-cogent-n-9815-v9-mkmlizer: uploading /dev/shm/model_cache to s3://guanaco-mkml-models/zonemercy-graft-cogent-n-9815-v9
zonemercy-graft-cogent-n-9815-v9-mkmlizer: cp /dev/shm/model_cache/config.json s3://guanaco-mkml-models/zonemercy-graft-cogent-n-9815-v9/config.json
zonemercy-graft-cogent-n-9815-v9-mkmlizer: cp /dev/shm/model_cache/special_tokens_map.json s3://guanaco-mkml-models/zonemercy-graft-cogent-n-9815-v9/special_tokens_map.json
zonemercy-graft-cogent-n-9815-v9-mkmlizer: cp /dev/shm/model_cache/tokenizer_config.json s3://guanaco-mkml-models/zonemercy-graft-cogent-n-9815-v9/tokenizer_config.json
zonemercy-graft-cogent-n-9815-v9-mkmlizer: cp /dev/shm/model_cache/tokenizer.json s3://guanaco-mkml-models/zonemercy-graft-cogent-n-9815-v9/tokenizer.json
zonemercy-graft-cogent-n-9815-v9-mkmlizer: cp /dev/shm/model_cache/flywheel_model.0.safetensors s3://guanaco-mkml-models/zonemercy-graft-cogent-n-9815-v9/flywheel_model.0.safetensors
zonemercy-graft-cogent-n-9815-v9-mkmlizer: 
Loading 0:   0%|          | 0/363 [00:00<?, ?it/s]
Loading 0:   1%|▏         | 5/363 [00:00<00:17, 21.05it/s]
Loading 0:   3%|▎         | 10/363 [00:00<00:13, 26.97it/s]
Loading 0:   4%|▍         | 14/363 [00:00<00:14, 23.34it/s]
Loading 0:   5%|▌         | 19/363 [00:00<00:11, 29.60it/s]
Loading 0:   6%|▋         | 23/363 [00:00<00:15, 21.67it/s]
Loading 0:   7%|▋         | 26/363 [00:01<00:17, 19.41it/s]
Loading 0:   9%|▊         | 31/363 [00:01<00:13, 24.92it/s]
Loading 0:  10%|▉         | 35/363 [00:01<00:12, 26.23it/s]
Loading 0:  11%|█         | 39/363 [00:01<00:11, 27.40it/s]
Loading 0:  12%|█▏        | 43/363 [00:01<00:12, 25.68it/s]
Loading 0:  13%|█▎        | 46/363 [00:01<00:11, 26.53it/s]
Loading 0:  14%|█▍        | 50/363 [00:02<00:13, 23.59it/s]
Loading 0:  15%|█▌        | 56/363 [00:02<00:10, 28.26it/s]
Loading 0:  17%|█▋        | 60/363 [00:02<00:09, 30.46it/s]
Loading 0:  18%|█▊        | 64/363 [00:02<00:15, 19.57it/s]
Loading 0:  19%|█▉        | 69/363 [00:02<00:12, 24.27it/s]
Loading 0:  20%|██        | 73/363 [00:02<00:11, 24.50it/s]
Loading 0:  21%|██        | 77/363 [00:03<00:12, 22.44it/s]
Loading 0:  23%|██▎       | 82/363 [00:03<00:10, 26.37it/s]
Loading 0:  24%|██▎       | 86/363 [00:03<00:11, 23.70it/s]
Loading 0:  25%|██▌       | 91/363 [00:03<00:09, 28.04it/s]
Loading 0:  26%|██▌       | 95/363 [00:03<00:10, 24.59it/s]
Loading 0:  28%|██▊       | 100/363 [00:03<00:09, 28.69it/s]
Loading 0:  29%|██▊       | 104/363 [00:04<00:13, 19.37it/s]
Loading 0:  30%|███       | 109/363 [00:04<00:10, 23.90it/s]
Loading 0:  31%|███       | 113/363 [00:04<00:11, 22.04it/s]
Loading 0:  33%|███▎      | 118/363 [00:04<00:09, 26.95it/s]
Loading 0:  34%|███▎      | 122/363 [00:04<00:10, 24.08it/s]
Loading 0:  36%|███▌      | 129/363 [00:05<00:07, 29.98it/s]
Loading 0:  37%|███▋      | 133/363 [00:05<00:08, 27.56it/s]
Loading 0:  38%|███▊      | 137/363 [00:05<00:08, 26.68it/s]
Loading 0:  39%|███▉      | 142/363 [00:05<00:09, 23.15it/s]
Loading 0:  40%|███▉      | 145/363 [00:05<00:09, 22.03it/s]
Loading 0:  41%|████      | 149/363 [00:06<00:10, 20.94it/s]
Loading 0:  42%|████▏     | 154/363 [00:06<00:08, 26.07it/s]
Loading 0:  44%|████▎     | 158/363 [00:06<00:08, 23.30it/s]
Loading 0:  45%|████▍     | 163/363 [00:06<00:07, 28.34it/s]
Loading 0:  46%|████▌     | 167/363 [00:06<00:07, 24.95it/s]
Loading 0:  47%|████▋     | 172/363 [00:06<00:06, 29.33it/s]
Loading 0:  48%|████▊     | 176/363 [00:07<00:07, 25.26it/s]
Loading 0:  50%|████▉     | 181/363 [00:07<00:06, 30.01it/s]
Loading 0:  51%|█████     | 185/363 [00:07<00:10, 17.27it/s]
Loading 0:  52%|█████▏    | 190/363 [00:07<00:07, 21.75it/s]
Loading 0:  53%|█████▎    | 194/363 [00:07<00:08, 21.03it/s]
Loading 0:  55%|█████▍    | 199/363 [00:08<00:06, 25.69it/s]
Loading 0:  56%|█████▌    | 203/363 [00:08<00:06, 22.89it/s]
Loading 0:  57%|█████▋    | 208/363 [00:08<00:05, 26.94it/s]
Loading 0:  58%|█████▊    | 212/363 [00:08<00:06, 23.93it/s]
Loading 0:  60%|█████▉    | 217/363 [00:08<00:05, 28.57it/s]
Loading 0:  61%|██████    | 222/363 [00:08<00:04, 29.48it/s]
Loading 0:  62%|██████▏   | 226/363 [00:09<00:06, 20.56it/s]
Loading 0:  63%|██████▎   | 230/363 [00:09<00:06, 20.24it/s]
Loading 0:  65%|██████▍   | 235/363 [00:09<00:05, 24.70it/s]
Loading 0:  66%|██████▌   | 239/363 [00:09<00:05, 22.44it/s]
Loading 0:  67%|██████▋   | 244/363 [00:09<00:04, 26.87it/s]
Loading 0:  68%|██████▊   | 248/363 [00:10<00:04, 23.76it/s]
Loading 0:  70%|██████▉   | 253/363 [00:10<00:03, 27.94it/s]
Loading 0:  71%|███████   | 257/363 [00:10<00:04, 24.25it/s]
Loading 0:  72%|███████▏  | 262/363 [00:10<00:03, 28.39it/s]
Loading 0:  73%|███████▎  | 266/363 [00:11<00:05, 18.97it/s]
Loading 0:  75%|███████▍  | 271/363 [00:11<00:03, 23.19it/s]
Loading 0:  76%|███████▌  | 275/363 [00:11<00:04, 21.79it/s]
Loading 0:  78%|███████▊  | 282/363 [00:11<00:02, 27.85it/s]
Loading 0:  79%|███████▉  | 286/363 [00:11<00:02, 26.70it/s]
Loading 0:  80%|███████▉  | 290/363 [00:11<00:02, 28.43it/s]
Loading 0:  81%|████████  | 294/363 [00:11<00:02, 25.07it/s]
Loading 0:  82%|████████▏ | 298/363 [00:12<00:02, 27.44it/s]
Loading 0:  83%|████████▎ | 303/363 [00:12<00:02, 28.84it/s]
Loading 0:  85%|████████▍ | 307/363 [00:12<00:02, 20.35it/s]
Loading 0:  86%|████████▌ | 311/363 [00:12<00:02, 19.96it/s]
Loading 0:  87%|████████▋ | 316/363 [00:12<00:01, 24.44it/s]
Loading 0:  88%|████████▊ | 320/363 [00:13<00:01, 22.20it/s]
Loading 0:  90%|████████▉ | 325/363 [00:13<00:01, 26.69it/s]
Loading 0:  91%|█████████ | 329/363 [00:13<00:01, 23.34it/s]
Loading 0:  92%|█████████▏| 334/363 [00:13<00:01, 27.69it/s]
Loading 0:  93%|█████████▎| 338/363 [00:13<00:01, 24.41it/s]
Loading 0:  94%|█████████▍| 343/363 [00:13<00:00, 29.06it/s]
Loading 0:  96%|█████████▌| 347/363 [00:21<00:08,  1.95it/s]
Loading 0:  96%|█████████▋| 350/363 [00:21<00:05,  2.46it/s]
Loading 0:  97%|█████████▋| 353/363 [00:21<00:03,  3.13it/s]
Loading 0:  98%|█████████▊| 356/363 [00:21<00:01,  4.06it/s]
Loading 0:  99%|█████████▉| 359/363 [00:21<00:00,  5.16it/s]
                                                            
Job zonemercy-graft-cogent-n-9815-v9-mkmlizer completed after 146.57s with status: succeeded
Stopping job with name zonemercy-graft-cogent-n-9815-v9-mkmlizer
Pipeline stage MKMLizer completed in 147.91s
Running pipeline stage MKMLKubeTemplater
Pipeline stage MKMLKubeTemplater completed in 0.09s
Running pipeline stage ISVCDeployer
Creating inference service zonemercy-graft-cogent-n-9815-v9
Waiting for inference service zonemercy-graft-cogent-n-9815-v9 to be ready
Failed to get response for submission function_fahor_2024-08-15: no entry with id "function_fahor_2024-08-15" found on database!
Failed to get response for submission function_fahor_2024-08-15: no entry with id "function_fahor_2024-08-15" found on database!
Failed to get response for submission function_fahor_2024-08-15: no entry with id "function_fahor_2024-08-15" found on database!
Failed to get response for submission function_fahor_2024-08-15: no entry with id "function_fahor_2024-08-15" found on database!
Inference service zonemercy-graft-cogent-n-9815-v9 ready after 231.5531828403473s
Pipeline stage ISVCDeployer completed in 232.96s
Running pipeline stage StressChecker
Received healthy response to inference request in 3.373588800430298s
Received healthy response to inference request in 2.0728604793548584s
Received healthy response to inference request in 4.062062740325928s
Received healthy response to inference request in 2.8494656085968018s
Received healthy response to inference request in 1.8894460201263428s
5 requests
0 failed requests
5th percentile: 1.926128911972046
10th percentile: 1.9628118038177491
20th percentile: 2.0361775875091555
30th percentile: 2.228181505203247
40th percentile: 2.5388235569000246
50th percentile: 2.8494656085968018
60th percentile: 3.0591148853302004
70th percentile: 3.2687641620635985
80th percentile: 3.511283588409424
90th percentile: 3.786673164367676
95th percentile: 3.9243679523468016
99th percentile: 4.034523782730102
mean time: 2.8494847297668455
Pipeline stage StressChecker completed in 14.97s
zonemercy-graft-cogent-n_9815_v9 status is now deployed due to DeploymentManager action
zonemercy-graft-cogent-n_9815_v9 status is now inactive due to auto deactivation removed underperforming models
zonemercy-graft-cogent-n_9815_v9 status is now torndown due to DeploymentManager action