submission_id: zonemercy-lexical-nemo-_1518_v18
developer_uid: zonemercy
alignment_samples: 526627
alignment_score: 2.3519770690250503
best_of: 8
celo_rating: 1265.08
display_name: 0815v2-0
formatter: {'memory_template': "{bot_name}'s Persona: {memory}\n####\n", 'prompt_template': '{prompt}\n<START>\n', 'bot_template': '{bot_name}: {message}\n', 'user_template': '{user_name}: {message}\n', 'response_template': '{bot_name}:', 'truncate_by_message': False}
generation_params: {'temperature': 0.9, 'top_p': 1.0, 'min_p': 0.05, 'top_k': 80, 'presence_penalty': 0.0, 'frequency_penalty': 0.0, 'stopping_words': ['\n', '</s>', '###', 'Bot:', 'User:', 'You:', '<|im_end|>'], 'max_input_tokens': 1024, 'best_of': 8, 'max_output_tokens': 64}
is_internal_developer: True
language_model: zonemercy/Lexical-Nemo-v4-1k1e5
max_input_tokens: 1024
max_output_tokens: 64
model_architecture: MistralForCausalLM
model_group: zonemercy/Lexical-Nemo-v
model_name: 0815v2-0
model_num_parameters: 12772070400.0
model_repo: zonemercy/Lexical-Nemo-v4-1k1e5
model_size: 13B
num_battles: 525849
num_wins: 266011
propriety_score: 0.7364596246740478
propriety_total_count: 46786.0
ranking_group: single
status: inactive
submission_type: basic
timestamp: 2024-08-15T23:08:49+00:00
us_pacific_date: 2024-08-15
win_ratio: 0.5058695557089583
Download Preferencedata
Resubmit model
Running pipeline stage MKMLizer
Starting job with name zonemercy-lexical-nemo-1518-v18-mkmlizer
Waiting for job on zonemercy-lexical-nemo-1518-v18-mkmlizer to finish
Stopping job with name zonemercy-lexical-nemo-1518-v18-mkmlizer
%s, retrying in %s seconds...
Starting job with name zonemercy-lexical-nemo-1518-v18-mkmlizer
Waiting for job on zonemercy-lexical-nemo-1518-v18-mkmlizer to finish
Stopping job with name zonemercy-lexical-nemo-1518-v18-mkmlizer
%s, retrying in %s seconds...
Starting job with name zonemercy-lexical-nemo-1518-v18-mkmlizer
Waiting for job on zonemercy-lexical-nemo-1518-v18-mkmlizer to finish
zonemercy-lexical-nemo-1518-v18-mkmlizer: ╔═════════════════════════════════════════════════════════════════════╗
zonemercy-lexical-nemo-1518-v18-mkmlizer: ║ _____ __ __ ║
zonemercy-lexical-nemo-1518-v18-mkmlizer: ║ / _/ /_ ___ __/ / ___ ___ / / ║
zonemercy-lexical-nemo-1518-v18-mkmlizer: ║ / _/ / // / |/|/ / _ \/ -_) -_) / ║
zonemercy-lexical-nemo-1518-v18-mkmlizer: ║ /_//_/\_, /|__,__/_//_/\__/\__/_/ ║
zonemercy-lexical-nemo-1518-v18-mkmlizer: ║ /___/ ║
zonemercy-lexical-nemo-1518-v18-mkmlizer: ║ ║
zonemercy-lexical-nemo-1518-v18-mkmlizer: ║ Version: 0.9.11 ║
zonemercy-lexical-nemo-1518-v18-mkmlizer: ║ Copyright 2023 MK ONE TECHNOLOGIES Inc. ║
zonemercy-lexical-nemo-1518-v18-mkmlizer: ║ https://mk1.ai ║
zonemercy-lexical-nemo-1518-v18-mkmlizer: ║ ║
zonemercy-lexical-nemo-1518-v18-mkmlizer: ║ The license key for the current software has been verified as ║
Failed to get response for submission function_fahor_2024-08-15: no entry with id "function_fahor_2024-08-15" found on database!
zonemercy-lexical-nemo-1518-v18-mkmlizer: ║ belonging to: ║
zonemercy-lexical-nemo-1518-v18-mkmlizer: ║ ║
zonemercy-lexical-nemo-1518-v18-mkmlizer: ║ Chai Research Corp. ║
zonemercy-lexical-nemo-1518-v18-mkmlizer: ║ Account ID: 7997a29f-0ceb-4cc7-9adf-840c57b4ae6f ║
zonemercy-lexical-nemo-1518-v18-mkmlizer: ║ Expiration: 2024-10-15 23:59:59 ║
zonemercy-lexical-nemo-1518-v18-mkmlizer: ║ ║
zonemercy-lexical-nemo-1518-v18-mkmlizer: ╚═════════════════════════════════════════════════════════════════════╝
Failed to get response for submission function_fahor_2024-08-15: no entry with id "function_fahor_2024-08-15" found on database!
zonemercy-lexical-nemo-1518-v18-mkmlizer: Downloaded to shared memory in 93.965s
zonemercy-lexical-nemo-1518-v18-mkmlizer: quantizing model to /dev/shm/model_cache, profile:s0, folder:/tmp/tmp0r1fi4mv, device:0
zonemercy-lexical-nemo-1518-v18-mkmlizer: Saving flywheel model at /dev/shm/model_cache
Failed to get response for submission zonemercy-cogent-nemo-v2-5e6_v14: ('http://zonemercy-cogent-nemo-v2-5e6-v14-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'request timeout')
Failed to get response for submission zonemercy-cogent-nemo-v2-5e6_v14: ('http://zonemercy-cogent-nemo-v2-5e6-v14-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'request timeout')
Failed to get response for submission function_fahor_2024-08-15: no entry with id "function_fahor_2024-08-15" found on database!
zonemercy-lexical-nemo-1518-v18-mkmlizer: quantized model in 41.273s
zonemercy-lexical-nemo-1518-v18-mkmlizer: Processed model zonemercy/Lexical-Nemo-v4-1k1e5 in 135.239s
zonemercy-lexical-nemo-1518-v18-mkmlizer: creating bucket guanaco-mkml-models
zonemercy-lexical-nemo-1518-v18-mkmlizer: Bucket 's3://guanaco-mkml-models/' created
zonemercy-lexical-nemo-1518-v18-mkmlizer: uploading /dev/shm/model_cache to s3://guanaco-mkml-models/zonemercy-lexical-nemo-1518-v18
zonemercy-lexical-nemo-1518-v18-mkmlizer: cp /dev/shm/model_cache/special_tokens_map.json s3://guanaco-mkml-models/zonemercy-lexical-nemo-1518-v18/special_tokens_map.json
zonemercy-lexical-nemo-1518-v18-mkmlizer: cp /dev/shm/model_cache/config.json s3://guanaco-mkml-models/zonemercy-lexical-nemo-1518-v18/config.json
zonemercy-lexical-nemo-1518-v18-mkmlizer: cp /dev/shm/model_cache/tokenizer_config.json s3://guanaco-mkml-models/zonemercy-lexical-nemo-1518-v18/tokenizer_config.json
zonemercy-lexical-nemo-1518-v18-mkmlizer: cp /dev/shm/model_cache/tokenizer.json s3://guanaco-mkml-models/zonemercy-lexical-nemo-1518-v18/tokenizer.json
Failed to get response for submission function_fahor_2024-08-15: no entry with id "function_fahor_2024-08-15" found on database!
Failed to get response for submission zonemercy-cogent-nemo-v2-5e6_v14: ('http://zonemercy-cogent-nemo-v2-5e6-v14-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'request timeout')
zonemercy-lexical-nemo-1518-v18-mkmlizer: cp /dev/shm/model_cache/flywheel_model.0.safetensors s3://guanaco-mkml-models/zonemercy-lexical-nemo-1518-v18/flywheel_model.0.safetensors
zonemercy-lexical-nemo-1518-v18-mkmlizer: Loading 0: 0%| | 0/363 [00:00<?, ?it/s] Loading 0: 1%|▏ | 5/363 [00:00<00:15, 22.76it/s] Loading 0: 3%|▎ | 10/363 [00:00<00:12, 27.74it/s] Loading 0: 4%|▍ | 14/363 [00:00<00:14, 24.81it/s] Loading 0: 6%|▌ | 21/363 [00:00<00:09, 35.57it/s] Loading 0: 7%|▋ | 26/363 [00:01<00:15, 21.94it/s] Loading 0: 9%|▊ | 31/363 [00:01<00:12, 26.75it/s] Loading 0: 10%|▉ | 35/363 [00:01<00:11, 28.37it/s] Loading 0: 11%|█ | 39/363 [00:01<00:10, 30.12it/s] Loading 0: 12%|█▏ | 43/363 [00:01<00:11, 28.16it/s] Loading 0: 13%|█▎ | 48/363 [00:01<00:10, 30.19it/s] Loading 0: 14%|█▍ | 52/363 [00:01<00:11, 27.91it/s] Loading 0: 15%|█▌ | 56/363 [00:02<00:11, 27.70it/s] Loading 0: 17%|█▋ | 61/363 [00:02<00:12, 24.54it/s] Loading 0: 18%|█▊ | 64/363 [00:02<00:13, 21.77it/s] Loading 0: 19%|█▉ | 69/363 [00:02<00:10, 26.97it/s] Loading 0: 20%|██ | 73/363 [00:02<00:10, 26.44it/s] Loading 0: 21%|██ | 77/363 [00:02<00:11, 24.34it/s] Loading 0: 23%|██▎ | 82/363 [00:03<00:09, 28.93it/s] Loading 0: 24%|██▎ | 86/363 [00:03<00:11, 25.10it/s] Loading 0: 26%|██▌ | 93/363 [00:03<00:08, 31.11it/s] Loading 0: 27%|██▋ | 97/363 [00:03<00:09, 29.39it/s] Loading 0: 28%|██▊ | 101/363 [00:03<00:10, 24.28it/s] Loading 0: 29%|██▊ | 104/363 [00:03<00:11, 21.60it/s] Loading 0: 30%|███ | 109/363 [00:04<00:09, 26.72it/s] Loading 0: 31%|███ | 113/363 [00:04<00:10, 24.67it/s] Loading 0: 33%|███▎ | 120/363 [00:04<00:07, 30.73it/s] Loading 0: 34%|███▍ | 124/363 [00:04<00:08, 29.26it/s] Loading 0: 36%|███▌ | 129/363 [00:04<00:07, 31.12it/s] Loading 0: 37%|███▋ | 133/363 [00:04<00:07, 29.76it/s] Loading 0: 38%|███▊ | 137/363 [00:05<00:07, 29.71it/s] Loading 0: 39%|███▉ | 142/363 [00:05<00:08, 26.78it/s] Loading 0: 40%|███▉ | 145/363 [00:05<00:08, 24.84it/s] Loading 0: 41%|████ | 149/363 [00:05<00:09, 22.68it/s] Loading 0: 42%|████▏ | 154/363 [00:05<00:07, 27.84it/s] Loading 0: 44%|████▎ | 158/363 [00:05<00:08, 25.48it/s] Loading 0: 45%|████▌ | 165/363 [00:06<00:06, 31.76it/s] Loading 0: 47%|████▋ | 169/363 [00:06<00:06, 29.95it/s] Loading 0: 48%|████▊ | 174/363 [00:06<00:06, 31.34it/s] Loading 0: 49%|████▉ | 178/363 [00:06<00:06, 29.24it/s] Loading 0: 50%|█████ | 182/363 [00:06<00:07, 23.75it/s] Loading 0: 51%|█████ | 185/363 [00:06<00:08, 21.22it/s] Loading 0: 53%|█████▎ | 192/363 [00:07<00:06, 28.41it/s] Loading 0: 54%|█████▍ | 196/363 [00:07<00:06, 27.82it/s] Loading 0: 55%|█████▌ | 201/363 [00:07<00:05, 30.29it/s] Loading 0: 56%|█████▋ | 205/363 [00:07<00:05, 28.58it/s] Loading 0: 58%|█████▊ | 210/363 [00:07<00:05, 30.10it/s] Loading 0: 59%|█████▉ | 214/363 [00:07<00:05, 28.65it/s] Loading 0: 60%|██████ | 218/363 [00:07<00:04, 29.30it/s] Loading 0: 61%|██████▏ | 223/363 [00:08<00:05, 25.77it/s] Loading 0: 62%|██████▏ | 226/363 [00:08<00:05, 24.30it/s] Loading 0: 63%|██████▎ | 230/363 [00:08<00:05, 23.07it/s] Loading 0: 65%|██████▌ | 237/363 [00:08<00:04, 29.37it/s] Loading 0: 66%|██████▋ | 241/363 [00:08<00:04, 28.18it/s] Loading 0: 68%|██████▊ | 246/363 [00:08<00:03, 30.46it/s] Loading 0: 69%|██████▉ | 250/363 [00:09<00:03, 29.23it/s] Loading 0: 70%|███████ | 255/363 [00:09<00:03, 31.17it/s] Loading 0: 71%|███████▏ | 259/363 [00:09<00:03, 29.62it/s] Loading 0: 72%|███████▏ | 263/363 [00:09<00:04, 23.67it/s] Loading 0: 73%|███████▎ | 266/363 [00:09<00:04, 20.39it/s] Loading 0: 75%|███████▌ | 273/363 [00:10<00:03, 27.14it/s] Loading 0: 76%|███████▌ | 276/363 [00:10<00:03, 25.43it/s] Loading 0: 78%|███████▊ | 282/363 [00:10<00:02, 30.13it/s] Loading 0: 79%|███████▉ | 286/363 [00:10<00:02, 28.73it/s] Loading 0: 80%|████████ | 291/363 [00:10<00:02, 30.52it/s] Loading 0: 81%|████████▏ | 295/363 [00:10<00:02, 28.75it/s] Loading 0: 82%|████████▏ | 299/363 [00:10<00:02, 28.47it/s] Loading 0: 84%|████████▎ | 304/363 [00:11<00:02, 24.17it/s] Loading 0: 85%|████████▍ | 307/363 [00:11<00:02, 23.16it/s] Loading 0: 86%|████████▌ | 311/363 [00:11<00:02, 22.19it/s] Loading 0: 87%|████████▋ | 316/363 [00:11<00:01, 27.45it/s] Loading 0: 88%|████████▊ | 320/363 [00:11<00:01, 24.68it/s] Loading 0: 90%|████████▉ | 325/363 [00:11<00:01, 29.61it/s] Loading 0: 91%|█████████ | 329/363 [00:12<00:01, 25.72it/s] Loading 0: 92%|█████████▏| 334/363 [00:12<00:00, 30.53it/s] Loading 0: 93%|█████████▎| 338/363 [00:12<00:00, 26.30it/s] Loading 0: 94%|█████████▍| 343/363 [00:12<00:00, 31.04it/s] Loading 0: 96%|█████████▌| 347/363 [00:19<00:07, 2.01it/s] Loading 0: 96%|█████████▋| 350/363 [00:19<00:05, 2.53it/s] Loading 0: 97%|█████████▋| 353/363 [00:19<00:03, 3.24it/s] Loading 0: 98%|█████████▊| 357/363 [00:19<00:01, 4.44it/s]
Job zonemercy-lexical-nemo-1518-v18-mkmlizer completed after 157.43s with status: succeeded
Stopping job with name zonemercy-lexical-nemo-1518-v18-mkmlizer
Pipeline stage MKMLizer completed in 170.88s
Running pipeline stage MKMLKubeTemplater
Pipeline stage MKMLKubeTemplater completed in 0.12s
Running pipeline stage ISVCDeployer
Creating inference service zonemercy-lexical-nemo-1518-v18
Waiting for inference service zonemercy-lexical-nemo-1518-v18 to be ready
Failed to get response for submission zonemercy-cogent-nemo-v2-5e6_v14: ('http://zonemercy-cogent-nemo-v2-5e6-v14-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'request timeout')
Failed to get response for submission function_fahor_2024-08-15: no entry with id "function_fahor_2024-08-15" found on database!
Failed to get response for submission function_fahor_2024-08-15: no entry with id "function_fahor_2024-08-15" found on database!
Failed to get response for submission function_fahor_2024-08-15: no entry with id "function_fahor_2024-08-15" found on database!
Failed to get response for submission zonemercy-cogent-nemo-v2-5e6_v14: ('http://zonemercy-cogent-nemo-v2-5e6-v14-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'request timeout')
Failed to get response for submission function_fahor_2024-08-15: no entry with id "function_fahor_2024-08-15" found on database!
Failed to get response for submission function_fahor_2024-08-15: no entry with id "function_fahor_2024-08-15" found on database!
Inference service zonemercy-lexical-nemo-1518-v18 ready after 231.47024416923523s
Pipeline stage ISVCDeployer completed in 233.50s
Running pipeline stage StressChecker
Received healthy response to inference request in 2.256101369857788s
Received healthy response to inference request in 1.7645783424377441s
Received healthy response to inference request in 2.1408698558807373s
Failed to get response for submission zonemercy-cogent-nemo-v2-5e6_v14: ('http://zonemercy-cogent-nemo-v2-5e6-v14-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'request timeout')
Received healthy response to inference request in 1.7600743770599365s
Received healthy response to inference request in 2.7043440341949463s
5 requests
0 failed requests
5th percentile: 1.760975170135498
10th percentile: 1.7618759632110597
20th percentile: 1.7636775493621826
30th percentile: 1.8398366451263428
40th percentile: 1.99035325050354
50th percentile: 2.1408698558807373
60th percentile: 2.1869624614715577
70th percentile: 2.233055067062378
80th percentile: 2.34574990272522
90th percentile: 2.525046968460083
95th percentile: 2.6146955013275144
99th percentile: 2.68641432762146
mean time: 2.1251935958862305
Pipeline stage StressChecker completed in 11.47s
zonemercy-lexical-nemo-_1518_v18 status is now deployed due to DeploymentManager action
zonemercy-lexical-nemo-_1518_v18 status is now inactive due to auto deactivation removed underperforming models

Usage Metrics

Latency Metrics