submission_id: zonemercy-lexical-nemo-_1518_v22
developer_uid: zonemercy
alignment_samples: 10445
alignment_score: 3.735376870505241
best_of: 16
celo_rating: 1277.43
display_name: 0815v2-2
formatter: {'memory_template': "{bot_name}'s Persona: {memory}\n####\n", 'prompt_template': '{prompt}\n<START>\n', 'bot_template': '{bot_name}: {message}\n', 'user_template': '{user_name}: {message}\n', 'response_template': '{bot_name}:', 'truncate_by_message': False}
generation_params: {'temperature': 0.9, 'top_p': 1.0, 'min_p': 0.05, 'top_k': 80, 'presence_penalty': 0.0, 'frequency_penalty': 0.0, 'stopping_words': ['\n', '</s>', '###', 'Bot:', 'User:', 'You:', '<|im_end|>'], 'max_input_tokens': 512, 'best_of': 16, 'max_output_tokens': 64}
is_internal_developer: True
language_model: zonemercy/Lexical-Nemo-v4-1k1e5
max_input_tokens: 512
max_output_tokens: 64
model_architecture: MistralForCausalLM
model_group: zonemercy/Lexical-Nemo-v
model_name: 0815v2-2
model_num_parameters: 12772070400.0
model_repo: zonemercy/Lexical-Nemo-v4-1k1e5
model_size: 13B
num_battles: 10445
num_wins: 5854
propriety_score: 0.7574421168687983
propriety_total_count: 907.0
ranking_group: single
status: torndown
submission_type: basic
timestamp: 2024-08-16T17:29:58+00:00
us_pacific_date: 2024-08-16
win_ratio: 0.5604595500239349
Download Preferencedata
Resubmit model
Running pipeline stage MKMLizer
Starting job with name zonemercy-lexical-nemo-1518-v22-mkmlizer
Waiting for job on zonemercy-lexical-nemo-1518-v22-mkmlizer to finish
zonemercy-lexical-nemo-1518-v22-mkmlizer: ╔═════════════════════════════════════════════════════════════════════╗
zonemercy-lexical-nemo-1518-v22-mkmlizer: ║     _____            __           __                                ║
zonemercy-lexical-nemo-1518-v22-mkmlizer: ║    / _/ /_ ___    __/ /  ___ ___ / /                                ║
zonemercy-lexical-nemo-1518-v22-mkmlizer: ║   / _/ / // / |/|/ / _ \/ -_) -_) /                                 ║
zonemercy-lexical-nemo-1518-v22-mkmlizer: ║  /_//_/\_, /|__,__/_//_/\__/\__/_/                                  ║
zonemercy-lexical-nemo-1518-v22-mkmlizer: ║       /___/                                                         ║
zonemercy-lexical-nemo-1518-v22-mkmlizer: ║                                                                     ║
zonemercy-lexical-nemo-1518-v22-mkmlizer: ║  Version: 0.9.11                                                    ║
zonemercy-lexical-nemo-1518-v22-mkmlizer: ║  Copyright 2023 MK ONE TECHNOLOGIES Inc.                            ║
zonemercy-lexical-nemo-1518-v22-mkmlizer: ║  https://mk1.ai                                                     ║
zonemercy-lexical-nemo-1518-v22-mkmlizer: ║                                                                     ║
zonemercy-lexical-nemo-1518-v22-mkmlizer: ║  The license key for the current software has been verified as      ║
zonemercy-lexical-nemo-1518-v22-mkmlizer: ║  belonging to:                                                      ║
zonemercy-lexical-nemo-1518-v22-mkmlizer: ║                                                                     ║
zonemercy-lexical-nemo-1518-v22-mkmlizer: ║  Chai Research Corp.                                                ║
zonemercy-lexical-nemo-1518-v22-mkmlizer: ║  Account ID: 7997a29f-0ceb-4cc7-9adf-840c57b4ae6f                   ║
zonemercy-lexical-nemo-1518-v22-mkmlizer: ║  Expiration: 2024-10-15 23:59:59                                    ║
zonemercy-lexical-nemo-1518-v22-mkmlizer: ║                                                                     ║
zonemercy-lexical-nemo-1518-v22-mkmlizer: ╚═════════════════════════════════════════════════════════════════════╝
Failed to get response for submission blend_filor_2024-08-16: ('http://zonemercy-cogent-nemo-v2-5e6-v13-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'request timeout')
zonemercy-lexical-nemo-1518-v22-mkmlizer: Downloaded to shared memory in 54.332s
zonemercy-lexical-nemo-1518-v22-mkmlizer: quantizing model to /dev/shm/model_cache, profile:s0, folder:/tmp/tmpfyl_3ww0, device:0
zonemercy-lexical-nemo-1518-v22-mkmlizer: Saving flywheel model at /dev/shm/model_cache
zonemercy-lexical-nemo-1518-v22-mkmlizer: quantized model in 40.732s
zonemercy-lexical-nemo-1518-v22-mkmlizer: Processed model zonemercy/Lexical-Nemo-v4-1k1e5 in 95.064s
zonemercy-lexical-nemo-1518-v22-mkmlizer: creating bucket guanaco-mkml-models
zonemercy-lexical-nemo-1518-v22-mkmlizer: Bucket 's3://guanaco-mkml-models/' created
zonemercy-lexical-nemo-1518-v22-mkmlizer: uploading /dev/shm/model_cache to s3://guanaco-mkml-models/zonemercy-lexical-nemo-1518-v22
zonemercy-lexical-nemo-1518-v22-mkmlizer: cp /dev/shm/model_cache/config.json s3://guanaco-mkml-models/zonemercy-lexical-nemo-1518-v22/config.json
zonemercy-lexical-nemo-1518-v22-mkmlizer: cp /dev/shm/model_cache/special_tokens_map.json s3://guanaco-mkml-models/zonemercy-lexical-nemo-1518-v22/special_tokens_map.json
zonemercy-lexical-nemo-1518-v22-mkmlizer: cp /dev/shm/model_cache/tokenizer_config.json s3://guanaco-mkml-models/zonemercy-lexical-nemo-1518-v22/tokenizer_config.json
zonemercy-lexical-nemo-1518-v22-mkmlizer: cp /dev/shm/model_cache/tokenizer.json s3://guanaco-mkml-models/zonemercy-lexical-nemo-1518-v22/tokenizer.json
zonemercy-lexical-nemo-1518-v22-mkmlizer: cp /dev/shm/model_cache/flywheel_model.0.safetensors s3://guanaco-mkml-models/zonemercy-lexical-nemo-1518-v22/flywheel_model.0.safetensors
zonemercy-lexical-nemo-1518-v22-mkmlizer: 
Loading 0:   0%|          | 0/363 [00:00<?, ?it/s]
Loading 0:   1%|▏         | 5/363 [00:00<00:15, 23.53it/s]
Loading 0:   3%|▎         | 10/363 [00:00<00:12, 29.30it/s]
Loading 0:   4%|▍         | 14/363 [00:00<00:13, 25.82it/s]
Loading 0:   6%|▌         | 20/363 [00:00<00:09, 34.77it/s]
Loading 0:   7%|▋         | 24/363 [00:00<00:14, 23.47it/s]
Loading 0:   7%|▋         | 27/363 [00:01<00:14, 22.67it/s]
Loading 0:   9%|▊         | 31/363 [00:01<00:12, 26.31it/s]
Loading 0:  10%|▉         | 35/363 [00:01<00:11, 27.99it/s]
Loading 0:  11%|█         | 39/363 [00:01<00:11, 29.38it/s]
Loading 0:  12%|█▏        | 43/363 [00:01<00:11, 28.25it/s]
Loading 0:  13%|█▎        | 48/363 [00:01<00:10, 30.56it/s]
Loading 0:  14%|█▍        | 52/363 [00:01<00:10, 29.34it/s]
Loading 0:  15%|█▌        | 56/363 [00:02<00:10, 29.09it/s]
Loading 0:  17%|█▋        | 61/363 [00:02<00:12, 24.79it/s]
Loading 0:  18%|█▊        | 64/363 [00:02<00:13, 22.17it/s]
Loading 0:  20%|█▉        | 71/363 [00:02<00:10, 29.18it/s]
Loading 0:  21%|██        | 75/363 [00:02<00:10, 28.48it/s]
Loading 0:  22%|██▏       | 79/363 [00:02<00:10, 27.49it/s]
Loading 0:  23%|██▎       | 84/363 [00:03<00:09, 30.18it/s]
Loading 0:  24%|██▍       | 88/363 [00:03<00:09, 29.22it/s]
Loading 0:  26%|██▌       | 93/363 [00:03<00:08, 31.55it/s]
Loading 0:  27%|██▋       | 97/363 [00:03<00:09, 29.39it/s]
Loading 0:  28%|██▊       | 101/363 [00:03<00:11, 23.02it/s]
Loading 0:  29%|██▊       | 104/363 [00:03<00:12, 20.80it/s]
Loading 0:  30%|███       | 109/363 [00:04<00:09, 26.07it/s]
Loading 0:  31%|███       | 113/363 [00:04<00:10, 24.27it/s]
Loading 0:  33%|███▎      | 120/363 [00:04<00:07, 31.03it/s]
Loading 0:  34%|███▍      | 124/363 [00:04<00:08, 29.64it/s]
Loading 0:  36%|███▌      | 129/363 [00:04<00:07, 31.68it/s]
Loading 0:  37%|███▋      | 133/363 [00:04<00:07, 29.98it/s]
Loading 0:  38%|███▊      | 137/363 [00:04<00:07, 29.97it/s]
Loading 0:  39%|███▉      | 142/363 [00:05<00:08, 24.70it/s]
Loading 0:  40%|███▉      | 145/363 [00:05<00:09, 23.51it/s]
Loading 0:  41%|████      | 149/363 [00:05<00:09, 22.07it/s]
Loading 0:  43%|████▎     | 156/363 [00:05<00:07, 28.55it/s]
Loading 0:  44%|████▍     | 160/363 [00:05<00:07, 27.59it/s]
Loading 0:  45%|████▌     | 165/363 [00:06<00:06, 30.04it/s]
Loading 0:  47%|████▋     | 169/363 [00:06<00:06, 28.79it/s]
Loading 0:  48%|████▊     | 174/363 [00:06<00:06, 30.95it/s]
Loading 0:  49%|████▉     | 178/363 [00:06<00:06, 29.73it/s]
Loading 0:  50%|█████     | 182/363 [00:06<00:07, 23.93it/s]
Loading 0:  51%|█████     | 185/363 [00:06<00:08, 21.50it/s]
Loading 0:  53%|█████▎    | 192/363 [00:07<00:06, 28.34it/s]
Loading 0:  54%|█████▍    | 196/363 [00:07<00:06, 27.44it/s]
Loading 0:  55%|█████▌    | 201/363 [00:07<00:05, 29.84it/s]
Loading 0:  56%|█████▋    | 205/363 [00:07<00:05, 28.84it/s]
Loading 0:  58%|█████▊    | 210/363 [00:07<00:05, 30.59it/s]
Loading 0:  59%|█████▉    | 214/363 [00:07<00:05, 29.38it/s]
Loading 0:  60%|██████    | 218/363 [00:07<00:04, 29.09it/s]
Loading 0:  61%|██████▏   | 223/363 [00:08<00:05, 24.28it/s]
Loading 0:  62%|██████▏   | 226/363 [00:08<00:05, 23.52it/s]
Loading 0:  63%|██████▎   | 230/363 [00:08<00:05, 22.90it/s]
Loading 0:  65%|██████▍   | 235/363 [00:08<00:04, 28.02it/s]
Loading 0:  66%|██████▌   | 239/363 [00:08<00:04, 25.90it/s]
Loading 0:  68%|██████▊   | 246/363 [00:08<00:03, 32.27it/s]
Loading 0:  69%|██████▉   | 250/363 [00:09<00:03, 31.12it/s]
Loading 0:  70%|███████   | 255/363 [00:09<00:03, 33.41it/s]
Loading 0:  71%|███████▏  | 259/363 [00:09<00:03, 31.27it/s]
Loading 0:  72%|███████▏  | 263/363 [00:09<00:04, 24.26it/s]
Loading 0:  73%|███████▎  | 266/363 [00:09<00:04, 22.00it/s]
Loading 0:  75%|███████▌  | 273/363 [00:09<00:03, 28.92it/s]
Loading 0:  76%|███████▋  | 277/363 [00:10<00:03, 28.61it/s]
Loading 0:  78%|███████▊  | 282/363 [00:10<00:02, 31.10it/s]
Loading 0:  79%|███████▉  | 286/363 [00:10<00:02, 29.44it/s]
Loading 0:  80%|████████  | 291/363 [00:10<00:02, 31.64it/s]
Loading 0:  81%|████████▏ | 295/363 [00:10<00:02, 30.31it/s]
Loading 0:  82%|████████▏ | 299/363 [00:10<00:02, 30.65it/s]
Loading 0:  84%|████████▎ | 304/363 [00:11<00:02, 25.77it/s]
Loading 0:  85%|████████▍ | 307/363 [00:11<00:02, 24.36it/s]
Loading 0:  86%|████████▌ | 311/363 [00:11<00:02, 23.35it/s]
Loading 0:  88%|████████▊ | 318/363 [00:11<00:01, 29.71it/s]
Loading 0:  89%|████████▊ | 322/363 [00:11<00:01, 28.94it/s]
Loading 0:  90%|█████████ | 327/363 [00:11<00:01, 31.57it/s]
Loading 0:  91%|█████████ | 331/363 [00:11<00:01, 30.56it/s]
Loading 0:  93%|█████████▎| 336/363 [00:12<00:00, 32.92it/s]
Loading 0:  94%|█████████▎| 340/363 [00:12<00:00, 30.92it/s]
Loading 0:  95%|█████████▍| 344/363 [00:19<00:09,  2.00it/s]
Loading 0:  96%|█████████▌| 348/363 [00:19<00:05,  2.70it/s]
Loading 0:  97%|█████████▋| 353/363 [00:19<00:02,  3.91it/s]
Loading 0:  98%|█████████▊| 357/363 [00:19<00:01,  5.07it/s]
                                                            

Job zonemercy-lexical-nemo-1518-v22-mkmlizer completed after 115.28s with status: succeeded
Stopping job with name zonemercy-lexical-nemo-1518-v22-mkmlizer
Pipeline stage MKMLizer completed in 116.07s
Running pipeline stage MKMLKubeTemplater
Pipeline stage MKMLKubeTemplater completed in 0.09s
Running pipeline stage ISVCDeployer
Creating inference service zonemercy-lexical-nemo-1518-v22
Waiting for inference service zonemercy-lexical-nemo-1518-v22 to be ready
Failed to get response for submission blend_filor_2024-08-16: ('http://zonemercy-cogent-nemo-v2-5e6-v13-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'request timeout')
Inference service zonemercy-lexical-nemo-1518-v22 ready after 231.4866499900818s
Pipeline stage ISVCDeployer completed in 232.76s
Running pipeline stage StressChecker
Received healthy response to inference request in 2.3935508728027344s
Received healthy response to inference request in 2.340174436569214s
Received healthy response to inference request in 1.9425396919250488s
Received healthy response to inference request in 1.8703479766845703s
Received healthy response to inference request in 1.8934130668640137s
5 requests
0 failed requests
5th percentile: 1.874960994720459
10th percentile: 1.8795740127563476
20th percentile: 1.888800048828125
30th percentile: 1.9032383918762208
40th percentile: 1.9228890419006348
50th percentile: 1.9425396919250488
60th percentile: 2.101593589782715
70th percentile: 2.2606474876403806
80th percentile: 2.350849723815918
90th percentile: 2.372200298309326
95th percentile: 2.38287558555603
99th percentile: 2.3914158153533935
mean time: 2.088005208969116
Pipeline stage StressChecker completed in 11.14s
zonemercy-lexical-nemo-_1518_v22 status is now deployed due to DeploymentManager action
zonemercy-lexical-nemo-_1518_v22 status is now inactive due to auto deactivation removed underperforming models
run pipeline %s
admin requested tearing down of zonemercy-lexical-nemo-_1518_v22
run pipeline stage %s
Shutdown handler not registered because Python interpreter is not running in the main thread
Running pipeline stage MKMLDeleter
run pipeline %s
%s, retrying in %s seconds...
run pipeline stage %s
%s, retrying in %s seconds...
Running pipeline stage MKMLDeleter
clean up pipeline due to error=%s
%s, retrying in %s seconds...
Shutdown handler de-registered
%s, retrying in %s seconds...
clean up pipeline due to error=%s
Shutdown handler de-registered
zonemercy-lexical-nemo-_1518_v22 status is now torndown due to DeploymentManager action

Usage Metrics

Latency Metrics