zonemercy-cogent-nemo-v2-5e6

developer_uid: zonemercy
submission_id: zonemercy-cogent-nemo-v2-5e6_v17
model_name: 0815v3-2
model_group: zonemercy/Cogent-Nemo-v2
status: torndown
timestamp: 2024-08-17T05:26:38+00:00
num_battles: 108706
num_wins: 54759
celo_rating: 1291.26
family_friendly_score: 0.0
submission_type: basic
model_repo: zonemercy/Cogent-Nemo-v2-5e6
model_architecture: MistralForCausalLM
model_num_parameters: 12772070400.0
best_of: 16
max_input_tokens: 1024
max_output_tokens: 64
display_name: 0815v3-2
is_internal_developer: True
language_model: zonemercy/Cogent-Nemo-v2-5e6
model_size: 13B
ranking_group: single
us_pacific_date: 2024-08-16
win_ratio: 0.503734844442809
generation_params: {'temperature': 0.9, 'top_p': 1.0, 'min_p': 0.05, 'top_k': 80, 'presence_penalty': 0.0, 'frequency_penalty': 0.0, 'stopping_words': ['\n', '</s>', '###', 'Bot:', 'User:', 'You:', '<|im_end|>'], 'max_input_tokens': 1024, 'best_of': 16, 'max_output_tokens': 64}
formatter: {'memory_template': "{bot_name}'s Persona: {memory}\n####\n", 'prompt_template': '{prompt}\n<START>\n', 'bot_template': '{bot_name}: {message}\n', 'user_template': '{user_name}: {message}\n', 'response_template': '{bot_name}:', 'truncate_by_message': False}
Resubmit model
Running pipeline stage MKMLizer
Starting job with name zonemercy-cogent-nemo-v2-5e6-v17-mkmlizer
Waiting for job on zonemercy-cogent-nemo-v2-5e6-v17-mkmlizer to finish
zonemercy-cogent-nemo-v2-5e6-v17-mkmlizer: ╔═════════════════════════════════════════════════════════════════════╗
zonemercy-cogent-nemo-v2-5e6-v17-mkmlizer: ║     _____            __           __                                ║
zonemercy-cogent-nemo-v2-5e6-v17-mkmlizer: ║    / _/ /_ ___    __/ /  ___ ___ / /                                ║
zonemercy-cogent-nemo-v2-5e6-v17-mkmlizer: ║   / _/ / // / |/|/ / _ \/ -_) -_) /                                 ║
zonemercy-cogent-nemo-v2-5e6-v17-mkmlizer: ║  /_//_/\_, /|__,__/_//_/\__/\__/_/                                  ║
zonemercy-cogent-nemo-v2-5e6-v17-mkmlizer: ║       /___/                                                         ║
zonemercy-cogent-nemo-v2-5e6-v17-mkmlizer: ║                                                                     ║
zonemercy-cogent-nemo-v2-5e6-v17-mkmlizer: ║  Version: 0.9.11                                                    ║
zonemercy-cogent-nemo-v2-5e6-v17-mkmlizer: ║  Copyright 2023 MK ONE TECHNOLOGIES Inc.                            ║
zonemercy-cogent-nemo-v2-5e6-v17-mkmlizer: ║  https://mk1.ai                                                     ║
zonemercy-cogent-nemo-v2-5e6-v17-mkmlizer: ║                                                                     ║
zonemercy-cogent-nemo-v2-5e6-v17-mkmlizer: ║  The license key for the current software has been verified as      ║
zonemercy-cogent-nemo-v2-5e6-v17-mkmlizer: ║  belonging to:                                                      ║
zonemercy-cogent-nemo-v2-5e6-v17-mkmlizer: ║                                                                     ║
zonemercy-cogent-nemo-v2-5e6-v17-mkmlizer: ║  Chai Research Corp.                                                ║
zonemercy-cogent-nemo-v2-5e6-v17-mkmlizer: ║  Account ID: 7997a29f-0ceb-4cc7-9adf-840c57b4ae6f                   ║
zonemercy-cogent-nemo-v2-5e6-v17-mkmlizer: ║  Expiration: 2024-10-15 23:59:59                                    ║
zonemercy-cogent-nemo-v2-5e6-v17-mkmlizer: ║                                                                     ║
zonemercy-cogent-nemo-v2-5e6-v17-mkmlizer: ╚═════════════════════════════════════════════════════════════════════╝
Failed to get response for submission undi95-meta-llama-3-70b_6209_v19: ('http://undi95-meta-llama-3-70b-6209-v19-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'activator request timeout')
Failed to get response for submission undi95-meta-llama-3-70b_6209_v19: ('http://undi95-meta-llama-3-70b-6209-v19-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'activator request timeout')
zonemercy-cogent-nemo-v2-5e6-v17-mkmlizer: Downloaded to shared memory in 84.693s
zonemercy-cogent-nemo-v2-5e6-v17-mkmlizer: quantizing model to /dev/shm/model_cache, profile:s0, folder:/tmp/tmpwimapn5m, device:0
zonemercy-cogent-nemo-v2-5e6-v17-mkmlizer: Saving flywheel model at /dev/shm/model_cache
zonemercy-cogent-nemo-v2-5e6-v17-mkmlizer: quantized model in 42.000s
zonemercy-cogent-nemo-v2-5e6-v17-mkmlizer: Processed model zonemercy/Cogent-Nemo-v2-5e6 in 126.693s
zonemercy-cogent-nemo-v2-5e6-v17-mkmlizer: creating bucket guanaco-mkml-models
zonemercy-cogent-nemo-v2-5e6-v17-mkmlizer: Bucket 's3://guanaco-mkml-models/' created
zonemercy-cogent-nemo-v2-5e6-v17-mkmlizer: uploading /dev/shm/model_cache to s3://guanaco-mkml-models/zonemercy-cogent-nemo-v2-5e6-v17
zonemercy-cogent-nemo-v2-5e6-v17-mkmlizer: cp /dev/shm/model_cache/config.json s3://guanaco-mkml-models/zonemercy-cogent-nemo-v2-5e6-v17/config.json
zonemercy-cogent-nemo-v2-5e6-v17-mkmlizer: cp /dev/shm/model_cache/special_tokens_map.json s3://guanaco-mkml-models/zonemercy-cogent-nemo-v2-5e6-v17/special_tokens_map.json
zonemercy-cogent-nemo-v2-5e6-v17-mkmlizer: cp /dev/shm/model_cache/flywheel_model.0.safetensors s3://guanaco-mkml-models/zonemercy-cogent-nemo-v2-5e6-v17/flywheel_model.0.safetensors
zonemercy-cogent-nemo-v2-5e6-v17-mkmlizer: 
Loading 0:   0%|          | 0/363 [00:00<?, ?it/s]
Loading 0:   1%|          | 4/363 [00:00<00:09, 36.36it/s]
Loading 0:   2%|▏         | 8/363 [00:00<00:14, 25.01it/s]
Loading 0:   3%|▎         | 12/363 [00:00<00:13, 26.33it/s]
Loading 0:   4%|▍         | 15/363 [00:00<00:14, 23.44it/s]
Loading 0:   6%|▌         | 20/363 [00:00<00:11, 30.36it/s]
Loading 0:   7%|▋         | 24/363 [00:01<00:16, 20.92it/s]
Loading 0:   7%|▋         | 27/363 [00:01<00:16, 20.29it/s]
Loading 0:   9%|▊         | 31/363 [00:01<00:13, 24.04it/s]
Loading 0:   9%|▉         | 34/363 [00:01<00:13, 23.88it/s]
Loading 0:  10%|█         | 37/363 [00:01<00:13, 25.03it/s]
Loading 0:  11%|█▏        | 41/363 [00:01<00:14, 22.56it/s]
Loading 0:  13%|█▎        | 48/363 [00:01<00:10, 29.76it/s]
Loading 0:  14%|█▍        | 52/363 [00:02<00:11, 28.23it/s]
Loading 0:  15%|█▌        | 56/363 [00:02<00:10, 28.31it/s]
Loading 0:  17%|█▋        | 61/363 [00:02<00:12, 24.18it/s]
Loading 0:  18%|█▊        | 64/363 [00:02<00:13, 21.36it/s]
Loading 0:  20%|█▉        | 71/363 [00:02<00:10, 27.81it/s]
Loading 0:  21%|██        | 75/363 [00:02<00:10, 26.80it/s]
Loading 0:  21%|██▏       | 78/363 [00:03<00:11, 24.37it/s]
Loading 0:  23%|██▎       | 82/363 [00:03<00:10, 27.46it/s]
Loading 0:  24%|██▎       | 86/363 [00:03<00:11, 24.37it/s]
Loading 0:  25%|██▌       | 92/363 [00:03<00:08, 31.62it/s]
Loading 0:  26%|██▋       | 96/363 [00:03<00:09, 27.11it/s]
Loading 0:  28%|██▊       | 101/363 [00:03<00:11, 23.72it/s]
Loading 0:  29%|██▊       | 104/363 [00:04<00:12, 20.90it/s]
Loading 0:  30%|███       | 109/363 [00:04<00:09, 25.98it/s]
Loading 0:  31%|███       | 113/363 [00:04<00:10, 23.72it/s]
Loading 0:  33%|███▎      | 118/363 [00:04<00:08, 28.67it/s]
Loading 0:  34%|███▎      | 122/363 [00:04<00:09, 25.47it/s]
Loading 0:  36%|███▌      | 129/363 [00:04<00:07, 31.51it/s]
Loading 0:  37%|███▋      | 133/363 [00:05<00:07, 29.04it/s]
Loading 0:  38%|███▊      | 137/363 [00:05<00:07, 28.81it/s]
Loading 0:  39%|███▉      | 141/363 [00:05<00:07, 31.19it/s]
Loading 0:  40%|███▉      | 145/363 [00:05<00:10, 21.23it/s]
Loading 0:  41%|████      | 149/363 [00:05<00:10, 20.79it/s]
Loading 0:  43%|████▎     | 156/363 [00:06<00:07, 27.02it/s]
Loading 0:  44%|████▍     | 160/363 [00:06<00:07, 26.02it/s]
Loading 0:  45%|████▌     | 165/363 [00:06<00:07, 27.91it/s]
Loading 0:  47%|████▋     | 169/363 [00:06<00:07, 26.89it/s]
Loading 0:  48%|████▊     | 174/363 [00:06<00:06, 29.01it/s]
Loading 0:  49%|████▉     | 178/363 [00:06<00:06, 27.44it/s]
Loading 0:  50%|█████     | 182/363 [00:07<00:08, 22.30it/s]
Loading 0:  51%|█████     | 185/363 [00:07<00:08, 20.07it/s]
Loading 0:  53%|█████▎    | 192/363 [00:07<00:06, 26.89it/s]
Loading 0:  54%|█████▎    | 195/363 [00:07<00:06, 25.44it/s]
Loading 0:  55%|█████▌    | 201/363 [00:07<00:05, 30.00it/s]
Loading 0:  56%|█████▋    | 205/363 [00:07<00:05, 28.99it/s]
Loading 0:  58%|█████▊    | 210/363 [00:08<00:04, 31.22it/s]
Loading 0:  59%|█████▉    | 214/363 [00:08<00:05, 28.53it/s]
Loading 0:  60%|██████    | 218/363 [00:08<00:05, 28.44it/s]
Loading 0:  61%|██████▏   | 223/363 [00:08<00:05, 25.17it/s]
Loading 0:  62%|██████▏   | 226/363 [00:08<00:05, 23.78it/s]
Loading 0:  63%|██████▎   | 230/363 [00:08<00:05, 22.65it/s]
Loading 0:  65%|██████▌   | 237/363 [00:09<00:04, 29.10it/s]
Loading 0:  66%|██████▋   | 241/363 [00:09<00:04, 27.85it/s]
Loading 0:  68%|██████▊   | 246/363 [00:09<00:03, 29.89it/s]
Loading 0:  69%|██████▉   | 250/363 [00:09<00:03, 28.52it/s]
Loading 0:  70%|███████   | 255/363 [00:09<00:03, 30.97it/s]
Loading 0:  71%|███████▏  | 259/363 [00:09<00:03, 29.43it/s]
Loading 0:  72%|███████▏  | 263/363 [00:10<00:04, 23.65it/s]
Loading 0:  73%|███████▎  | 266/363 [00:10<00:04, 20.87it/s]
Loading 0:  75%|███████▍  | 271/363 [00:10<00:03, 26.11it/s]
Loading 0:  76%|███████▌  | 275/363 [00:10<00:03, 24.33it/s]
Loading 0:  78%|███████▊  | 282/363 [00:10<00:02, 31.21it/s]
Loading 0:  79%|███████▉  | 286/363 [00:10<00:02, 29.69it/s]
Loading 0:  80%|████████  | 291/363 [00:11<00:02, 31.91it/s]
Loading 0:  81%|████████▏ | 295/363 [00:11<00:02, 29.79it/s]
Loading 0:  82%|████████▏ | 299/363 [00:11<00:02, 29.92it/s]
Loading 0:  84%|████████▎ | 304/363 [00:11<00:02, 25.77it/s]
Loading 0:  85%|████████▍ | 307/363 [00:11<00:02, 23.73it/s]
Loading 0:  86%|████████▌ | 311/363 [00:11<00:02, 22.57it/s]
Loading 0:  87%|████████▋ | 316/363 [00:12<00:01, 27.48it/s]
Loading 0:  88%|████████▊ | 320/363 [00:12<00:01, 24.48it/s]
Loading 0:  90%|████████▉ | 325/363 [00:12<00:01, 29.27it/s]
Loading 0:  91%|█████████ | 329/363 [00:12<00:01, 25.72it/s]
Loading 0:  92%|█████████▏| 334/363 [00:12<00:00, 30.40it/s]
Loading 0:  93%|█████████▎| 338/363 [00:12<00:00, 25.94it/s]
Loading 0:  94%|█████████▍| 343/363 [00:12<00:00, 30.11it/s]
Loading 0:  96%|█████████▌| 347/363 [00:20<00:08,  1.98it/s]
Loading 0:  96%|█████████▋| 350/363 [00:20<00:05,  2.50it/s]
Loading 0:  97%|█████████▋| 353/363 [00:20<00:03,  3.18it/s]
Loading 0:  98%|█████████▊| 357/363 [00:20<00:01,  4.34it/s]
                                                            
Job zonemercy-cogent-nemo-v2-5e6-v17-mkmlizer completed after 156.63s with status: succeeded
Stopping job with name zonemercy-cogent-nemo-v2-5e6-v17-mkmlizer
Pipeline stage MKMLizer completed in 157.62s
Running pipeline stage MKMLKubeTemplater
Pipeline stage MKMLKubeTemplater completed in 0.11s
Running pipeline stage ISVCDeployer
Creating inference service zonemercy-cogent-nemo-v2-5e6-v17
Waiting for inference service zonemercy-cogent-nemo-v2-5e6-v17 to be ready
Failed to get response for submission undi95-meta-llama-3-70b_6209_v19: ('http://undi95-meta-llama-3-70b-6209-v19-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'request timeout')
Failed to get response for submission chaiml-llama-8b-pairwis_8189_v14: ('http://chaiml-llama-8b-pairwis-8189-v14-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'request timeout')
Failed to get response for submission chaiml-llama-8b-pairwis_8189_v14: ('http://chaiml-llama-8b-pairwis-8189-v14-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'request timeout')
Failed to get response for submission chaiml-llama-8b-pairwis_8189_v14: ('http://chaiml-llama-8b-pairwis-8189-v14-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'request timeout')
Failed to get response for submission chaiml-llama-8b-pairwis_8189_v14: ('http://chaiml-llama-8b-pairwis-8189-v14-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'request timeout')
Failed to get response for submission blend_popob_2024-08-16: ('http://mistralai-mixtral-8x7b-3473-v106-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'read tcp 127.0.0.1:57292->127.0.0.1:8080: read: connection reset by peer\n')
Failed to get response for submission undi95-meta-llama-3-70b_6209_v19: ('http://undi95-meta-llama-3-70b-6209-v19-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'request timeout')
Failed to get response for submission chaiml-llama-8b-pairwis_8189_v14: ('http://chaiml-llama-8b-pairwis-8189-v14-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'request timeout')
Failed to get response for submission undi95-meta-llama-3-70b_6209_v19: ('http://undi95-meta-llama-3-70b-6209-v19-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'activator request timeout')
Failed to get response for submission undi95-meta-llama-3-70b_6209_v19: ('http://undi95-meta-llama-3-70b-6209-v19-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'activator request timeout')
Inference service zonemercy-cogent-nemo-v2-5e6-v17 ready after 241.66942763328552s
Pipeline stage ISVCDeployer completed in 242.88s
Running pipeline stage StressChecker
Received healthy response to inference request in 2.663248062133789s
Received healthy response to inference request in 2.1586761474609375s
Received healthy response to inference request in 2.0633575916290283s
Failed to get response for submission undi95-meta-llama-3-70b_6209_v19: ('http://undi95-meta-llama-3-70b-6209-v19-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'request timeout')
Received healthy response to inference request in 2.1805837154388428s
Received healthy response to inference request in 2.1551663875579834s
5 requests
0 failed requests
5th percentile: 2.0817193508148195
10th percentile: 2.1000811100006103
20th percentile: 2.136804628372192
30th percentile: 2.155868339538574
40th percentile: 2.1572722434997558
50th percentile: 2.1586761474609375
60th percentile: 2.1674391746521
70th percentile: 2.1762022018432616
80th percentile: 2.2771165847778323
90th percentile: 2.4701823234558105
95th percentile: 2.5667151927947995
99th percentile: 2.643941488265991
mean time: 2.2442063808441164
Pipeline stage StressChecker completed in 12.09s
zonemercy-cogent-nemo-v2-5e6_v17 status is now deployed due to DeploymentManager action
zonemercy-cogent-nemo-v2-5e6_v17 status is now inactive due to auto deactivation removed underperforming models
zonemercy-cogent-nemo-v2-5e6_v17 status is now torndown due to DeploymentManager action