zonemercy-lexical-nemo-_1518

developer_uid: zonemercy
submission_id: zonemercy-lexical-nemo-_1518_v18
model_name: 0815v2-0
model_group: zonemercy/Lexical-Nemo-v
status: torndown
timestamp: 2024-08-15T23:08:49+00:00
num_battles: 3405821
num_wins: 1690096
celo_rating: 1251.48
family_friendly_score: 0.5274776577297586
family_friendly_standard_error: 0.004961463701823655
submission_type: basic
model_repo: zonemercy/Lexical-Nemo-v4-1k1e5
model_architecture: MistralForCausalLM
model_num_parameters: 12772070400.0
best_of: 8
max_input_tokens: 1024
max_output_tokens: 64
display_name: 0815v2-0
is_internal_developer: True
language_model: zonemercy/Lexical-Nemo-v4-1k1e5
model_size: 13B
ranking_group: single
us_pacific_date: 2024-08-15
win_ratio: 0.4962374710825965
generation_params: {'temperature': 0.9, 'top_p': 1.0, 'min_p': 0.05, 'top_k': 80, 'presence_penalty': 0.0, 'frequency_penalty': 0.0, 'stopping_words': ['\n', '</s>', '###', 'Bot:', 'User:', 'You:', '<|im_end|>'], 'max_input_tokens': 1024, 'best_of': 8, 'max_output_tokens': 64}
formatter: {'memory_template': "{bot_name}'s Persona: {memory}\n####\n", 'prompt_template': '{prompt}\n<START>\n', 'bot_template': '{bot_name}: {message}\n', 'user_template': '{user_name}: {message}\n', 'response_template': '{bot_name}:', 'truncate_by_message': False}
Resubmit model
Running pipeline stage MKMLizer
Starting job with name zonemercy-lexical-nemo-1518-v18-mkmlizer
Waiting for job on zonemercy-lexical-nemo-1518-v18-mkmlizer to finish
Stopping job with name zonemercy-lexical-nemo-1518-v18-mkmlizer
%s, retrying in %s seconds...
Starting job with name zonemercy-lexical-nemo-1518-v18-mkmlizer
Waiting for job on zonemercy-lexical-nemo-1518-v18-mkmlizer to finish
Stopping job with name zonemercy-lexical-nemo-1518-v18-mkmlizer
%s, retrying in %s seconds...
Starting job with name zonemercy-lexical-nemo-1518-v18-mkmlizer
Waiting for job on zonemercy-lexical-nemo-1518-v18-mkmlizer to finish
zonemercy-lexical-nemo-1518-v18-mkmlizer: ╔═════════════════════════════════════════════════════════════════════╗
zonemercy-lexical-nemo-1518-v18-mkmlizer: ║     _____            __           __                                ║
zonemercy-lexical-nemo-1518-v18-mkmlizer: ║    / _/ /_ ___    __/ /  ___ ___ / /                                ║
zonemercy-lexical-nemo-1518-v18-mkmlizer: ║   / _/ / // / |/|/ / _ \/ -_) -_) /                                 ║
zonemercy-lexical-nemo-1518-v18-mkmlizer: ║  /_//_/\_, /|__,__/_//_/\__/\__/_/                                  ║
zonemercy-lexical-nemo-1518-v18-mkmlizer: ║       /___/                                                         ║
zonemercy-lexical-nemo-1518-v18-mkmlizer: ║                                                                     ║
zonemercy-lexical-nemo-1518-v18-mkmlizer: ║  Version: 0.9.11                                                    ║
zonemercy-lexical-nemo-1518-v18-mkmlizer: ║  Copyright 2023 MK ONE TECHNOLOGIES Inc.                            ║
zonemercy-lexical-nemo-1518-v18-mkmlizer: ║  https://mk1.ai                                                     ║
zonemercy-lexical-nemo-1518-v18-mkmlizer: ║                                                                     ║
zonemercy-lexical-nemo-1518-v18-mkmlizer: ║  The license key for the current software has been verified as      ║
Failed to get response for submission function_fahor_2024-08-15: no entry with id "function_fahor_2024-08-15" found on database!
zonemercy-lexical-nemo-1518-v18-mkmlizer: ║  belonging to:                                                      ║
zonemercy-lexical-nemo-1518-v18-mkmlizer: ║                                                                     ║
zonemercy-lexical-nemo-1518-v18-mkmlizer: ║  Chai Research Corp.                                                ║
zonemercy-lexical-nemo-1518-v18-mkmlizer: ║  Account ID: 7997a29f-0ceb-4cc7-9adf-840c57b4ae6f                   ║
zonemercy-lexical-nemo-1518-v18-mkmlizer: ║  Expiration: 2024-10-15 23:59:59                                    ║
zonemercy-lexical-nemo-1518-v18-mkmlizer: ║                                                                     ║
zonemercy-lexical-nemo-1518-v18-mkmlizer: ╚═════════════════════════════════════════════════════════════════════╝
Failed to get response for submission function_fahor_2024-08-15: no entry with id "function_fahor_2024-08-15" found on database!
zonemercy-lexical-nemo-1518-v18-mkmlizer: Downloaded to shared memory in 93.965s
zonemercy-lexical-nemo-1518-v18-mkmlizer: quantizing model to /dev/shm/model_cache, profile:s0, folder:/tmp/tmp0r1fi4mv, device:0
zonemercy-lexical-nemo-1518-v18-mkmlizer: Saving flywheel model at /dev/shm/model_cache
Failed to get response for submission zonemercy-cogent-nemo-v2-5e6_v14: ('http://zonemercy-cogent-nemo-v2-5e6-v14-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'request timeout')
Failed to get response for submission zonemercy-cogent-nemo-v2-5e6_v14: ('http://zonemercy-cogent-nemo-v2-5e6-v14-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'request timeout')
Failed to get response for submission function_fahor_2024-08-15: no entry with id "function_fahor_2024-08-15" found on database!
zonemercy-lexical-nemo-1518-v18-mkmlizer: quantized model in 41.273s
zonemercy-lexical-nemo-1518-v18-mkmlizer: Processed model zonemercy/Lexical-Nemo-v4-1k1e5 in 135.239s
zonemercy-lexical-nemo-1518-v18-mkmlizer: creating bucket guanaco-mkml-models
zonemercy-lexical-nemo-1518-v18-mkmlizer: Bucket 's3://guanaco-mkml-models/' created
zonemercy-lexical-nemo-1518-v18-mkmlizer: uploading /dev/shm/model_cache to s3://guanaco-mkml-models/zonemercy-lexical-nemo-1518-v18
zonemercy-lexical-nemo-1518-v18-mkmlizer: cp /dev/shm/model_cache/special_tokens_map.json s3://guanaco-mkml-models/zonemercy-lexical-nemo-1518-v18/special_tokens_map.json
zonemercy-lexical-nemo-1518-v18-mkmlizer: cp /dev/shm/model_cache/config.json s3://guanaco-mkml-models/zonemercy-lexical-nemo-1518-v18/config.json
zonemercy-lexical-nemo-1518-v18-mkmlizer: cp /dev/shm/model_cache/tokenizer_config.json s3://guanaco-mkml-models/zonemercy-lexical-nemo-1518-v18/tokenizer_config.json
zonemercy-lexical-nemo-1518-v18-mkmlizer: cp /dev/shm/model_cache/tokenizer.json s3://guanaco-mkml-models/zonemercy-lexical-nemo-1518-v18/tokenizer.json
Failed to get response for submission function_fahor_2024-08-15: no entry with id "function_fahor_2024-08-15" found on database!
Failed to get response for submission zonemercy-cogent-nemo-v2-5e6_v14: ('http://zonemercy-cogent-nemo-v2-5e6-v14-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'request timeout')
zonemercy-lexical-nemo-1518-v18-mkmlizer: cp /dev/shm/model_cache/flywheel_model.0.safetensors s3://guanaco-mkml-models/zonemercy-lexical-nemo-1518-v18/flywheel_model.0.safetensors
zonemercy-lexical-nemo-1518-v18-mkmlizer: 
Loading 0:   0%|          | 0/363 [00:00<?, ?it/s]
Loading 0:   1%|▏         | 5/363 [00:00<00:15, 22.76it/s]
Loading 0:   3%|▎         | 10/363 [00:00<00:12, 27.74it/s]
Loading 0:   4%|▍         | 14/363 [00:00<00:14, 24.81it/s]
Loading 0:   6%|▌         | 21/363 [00:00<00:09, 35.57it/s]
Loading 0:   7%|▋         | 26/363 [00:01<00:15, 21.94it/s]
Loading 0:   9%|▊         | 31/363 [00:01<00:12, 26.75it/s]
Loading 0:  10%|▉         | 35/363 [00:01<00:11, 28.37it/s]
Loading 0:  11%|█         | 39/363 [00:01<00:10, 30.12it/s]
Loading 0:  12%|█▏        | 43/363 [00:01<00:11, 28.16it/s]
Loading 0:  13%|█▎        | 48/363 [00:01<00:10, 30.19it/s]
Loading 0:  14%|█▍        | 52/363 [00:01<00:11, 27.91it/s]
Loading 0:  15%|█▌        | 56/363 [00:02<00:11, 27.70it/s]
Loading 0:  17%|█▋        | 61/363 [00:02<00:12, 24.54it/s]
Loading 0:  18%|█▊        | 64/363 [00:02<00:13, 21.77it/s]
Loading 0:  19%|█▉        | 69/363 [00:02<00:10, 26.97it/s]
Loading 0:  20%|██        | 73/363 [00:02<00:10, 26.44it/s]
Loading 0:  21%|██        | 77/363 [00:02<00:11, 24.34it/s]
Loading 0:  23%|██▎       | 82/363 [00:03<00:09, 28.93it/s]
Loading 0:  24%|██▎       | 86/363 [00:03<00:11, 25.10it/s]
Loading 0:  26%|██▌       | 93/363 [00:03<00:08, 31.11it/s]
Loading 0:  27%|██▋       | 97/363 [00:03<00:09, 29.39it/s]
Loading 0:  28%|██▊       | 101/363 [00:03<00:10, 24.28it/s]
Loading 0:  29%|██▊       | 104/363 [00:03<00:11, 21.60it/s]
Loading 0:  30%|███       | 109/363 [00:04<00:09, 26.72it/s]
Loading 0:  31%|███       | 113/363 [00:04<00:10, 24.67it/s]
Loading 0:  33%|███▎      | 120/363 [00:04<00:07, 30.73it/s]
Loading 0:  34%|███▍      | 124/363 [00:04<00:08, 29.26it/s]
Loading 0:  36%|███▌      | 129/363 [00:04<00:07, 31.12it/s]
Loading 0:  37%|███▋      | 133/363 [00:04<00:07, 29.76it/s]
Loading 0:  38%|███▊      | 137/363 [00:05<00:07, 29.71it/s]
Loading 0:  39%|███▉      | 142/363 [00:05<00:08, 26.78it/s]
Loading 0:  40%|███▉      | 145/363 [00:05<00:08, 24.84it/s]
Loading 0:  41%|████      | 149/363 [00:05<00:09, 22.68it/s]
Loading 0:  42%|████▏     | 154/363 [00:05<00:07, 27.84it/s]
Loading 0:  44%|████▎     | 158/363 [00:05<00:08, 25.48it/s]
Loading 0:  45%|████▌     | 165/363 [00:06<00:06, 31.76it/s]
Loading 0:  47%|████▋     | 169/363 [00:06<00:06, 29.95it/s]
Loading 0:  48%|████▊     | 174/363 [00:06<00:06, 31.34it/s]
Loading 0:  49%|████▉     | 178/363 [00:06<00:06, 29.24it/s]
Loading 0:  50%|█████     | 182/363 [00:06<00:07, 23.75it/s]
Loading 0:  51%|█████     | 185/363 [00:06<00:08, 21.22it/s]
Loading 0:  53%|█████▎    | 192/363 [00:07<00:06, 28.41it/s]
Loading 0:  54%|█████▍    | 196/363 [00:07<00:06, 27.82it/s]
Loading 0:  55%|█████▌    | 201/363 [00:07<00:05, 30.29it/s]
Loading 0:  56%|█████▋    | 205/363 [00:07<00:05, 28.58it/s]
Loading 0:  58%|█████▊    | 210/363 [00:07<00:05, 30.10it/s]
Loading 0:  59%|█████▉    | 214/363 [00:07<00:05, 28.65it/s]
Loading 0:  60%|██████    | 218/363 [00:07<00:04, 29.30it/s]
Loading 0:  61%|██████▏   | 223/363 [00:08<00:05, 25.77it/s]
Loading 0:  62%|██████▏   | 226/363 [00:08<00:05, 24.30it/s]
Loading 0:  63%|██████▎   | 230/363 [00:08<00:05, 23.07it/s]
Loading 0:  65%|██████▌   | 237/363 [00:08<00:04, 29.37it/s]
Loading 0:  66%|██████▋   | 241/363 [00:08<00:04, 28.18it/s]
Loading 0:  68%|██████▊   | 246/363 [00:08<00:03, 30.46it/s]
Loading 0:  69%|██████▉   | 250/363 [00:09<00:03, 29.23it/s]
Loading 0:  70%|███████   | 255/363 [00:09<00:03, 31.17it/s]
Loading 0:  71%|███████▏  | 259/363 [00:09<00:03, 29.62it/s]
Loading 0:  72%|███████▏  | 263/363 [00:09<00:04, 23.67it/s]
Loading 0:  73%|███████▎  | 266/363 [00:09<00:04, 20.39it/s]
Loading 0:  75%|███████▌  | 273/363 [00:10<00:03, 27.14it/s]
Loading 0:  76%|███████▌  | 276/363 [00:10<00:03, 25.43it/s]
Loading 0:  78%|███████▊  | 282/363 [00:10<00:02, 30.13it/s]
Loading 0:  79%|███████▉  | 286/363 [00:10<00:02, 28.73it/s]
Loading 0:  80%|████████  | 291/363 [00:10<00:02, 30.52it/s]
Loading 0:  81%|████████▏ | 295/363 [00:10<00:02, 28.75it/s]
Loading 0:  82%|████████▏ | 299/363 [00:10<00:02, 28.47it/s]
Loading 0:  84%|████████▎ | 304/363 [00:11<00:02, 24.17it/s]
Loading 0:  85%|████████▍ | 307/363 [00:11<00:02, 23.16it/s]
Loading 0:  86%|████████▌ | 311/363 [00:11<00:02, 22.19it/s]
Loading 0:  87%|████████▋ | 316/363 [00:11<00:01, 27.45it/s]
Loading 0:  88%|████████▊ | 320/363 [00:11<00:01, 24.68it/s]
Loading 0:  90%|████████▉ | 325/363 [00:11<00:01, 29.61it/s]
Loading 0:  91%|█████████ | 329/363 [00:12<00:01, 25.72it/s]
Loading 0:  92%|█████████▏| 334/363 [00:12<00:00, 30.53it/s]
Loading 0:  93%|█████████▎| 338/363 [00:12<00:00, 26.30it/s]
Loading 0:  94%|█████████▍| 343/363 [00:12<00:00, 31.04it/s]
Loading 0:  96%|█████████▌| 347/363 [00:19<00:07,  2.01it/s]
Loading 0:  96%|█████████▋| 350/363 [00:19<00:05,  2.53it/s]
Loading 0:  97%|█████████▋| 353/363 [00:19<00:03,  3.24it/s]
Loading 0:  98%|█████████▊| 357/363 [00:19<00:01,  4.44it/s]
                                                            
Job zonemercy-lexical-nemo-1518-v18-mkmlizer completed after 157.43s with status: succeeded
Stopping job with name zonemercy-lexical-nemo-1518-v18-mkmlizer
Pipeline stage MKMLizer completed in 170.88s
Running pipeline stage MKMLKubeTemplater
Pipeline stage MKMLKubeTemplater completed in 0.12s
Running pipeline stage ISVCDeployer
Creating inference service zonemercy-lexical-nemo-1518-v18
Waiting for inference service zonemercy-lexical-nemo-1518-v18 to be ready
Failed to get response for submission zonemercy-cogent-nemo-v2-5e6_v14: ('http://zonemercy-cogent-nemo-v2-5e6-v14-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'request timeout')
Failed to get response for submission function_fahor_2024-08-15: no entry with id "function_fahor_2024-08-15" found on database!
Failed to get response for submission function_fahor_2024-08-15: no entry with id "function_fahor_2024-08-15" found on database!
Failed to get response for submission function_fahor_2024-08-15: no entry with id "function_fahor_2024-08-15" found on database!
Failed to get response for submission zonemercy-cogent-nemo-v2-5e6_v14: ('http://zonemercy-cogent-nemo-v2-5e6-v14-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'request timeout')
Failed to get response for submission function_fahor_2024-08-15: no entry with id "function_fahor_2024-08-15" found on database!
Failed to get response for submission function_fahor_2024-08-15: no entry with id "function_fahor_2024-08-15" found on database!
Inference service zonemercy-lexical-nemo-1518-v18 ready after 231.47024416923523s
Pipeline stage ISVCDeployer completed in 233.50s
Running pipeline stage StressChecker
Received healthy response to inference request in 2.256101369857788s
Received healthy response to inference request in 1.7645783424377441s
Received healthy response to inference request in 2.1408698558807373s
Failed to get response for submission zonemercy-cogent-nemo-v2-5e6_v14: ('http://zonemercy-cogent-nemo-v2-5e6-v14-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'request timeout')
Received healthy response to inference request in 1.7600743770599365s
Received healthy response to inference request in 2.7043440341949463s
5 requests
0 failed requests
5th percentile: 1.760975170135498
10th percentile: 1.7618759632110597
20th percentile: 1.7636775493621826
30th percentile: 1.8398366451263428
40th percentile: 1.99035325050354
50th percentile: 2.1408698558807373
60th percentile: 2.1869624614715577
70th percentile: 2.233055067062378
80th percentile: 2.34574990272522
90th percentile: 2.525046968460083
95th percentile: 2.6146955013275144
99th percentile: 2.68641432762146
mean time: 2.1251935958862305
Pipeline stage StressChecker completed in 11.47s
zonemercy-lexical-nemo-_1518_v18 status is now deployed due to DeploymentManager action
zonemercy-lexical-nemo-_1518_v18 status is now inactive due to auto deactivation removed underperforming models
zonemercy-lexical-nemo-_1518_v18 status is now deployed due to admin request
zonemercy-lexical-nemo-_1518_v18 status is now torndown due to DeploymentManager action