zonemercy-lexical-nemo-_1518

developer_uid: zonemercy
submission_id: zonemercy-lexical-nemo-_1518_v16
model_name: zonemercy-lexical-nemo-_1518_v16
model_group: zonemercy/Lexical-Nemo-v
status: torndown
timestamp: 2024-08-12T20:02:00+00:00
num_battles: 11555
num_wins: 6036
celo_rating: 1240.93
family_friendly_score: 0.0
submission_type: basic
model_repo: zonemercy/Lexical-Nemo-v4-1k1e5
model_architecture: MistralForCausalLM
reward_repo: ChaiML/gpt2_xl_pairwise_89m_step_347634
model_num_parameters: 12772070400.0
best_of: 4
max_input_tokens: 1024
max_output_tokens: 64
display_name: zonemercy-lexical-nemo-_1518_v16
is_internal_developer: True
language_model: zonemercy/Lexical-Nemo-v4-1k1e5
model_size: 13B
ranking_group: single
us_pacific_date: 2024-08-12
win_ratio: 0.5223712678494158
generation_params: {'temperature': 0.9, 'top_p': 1.0, 'min_p': 0.05, 'top_k': 80, 'presence_penalty': 0.0, 'frequency_penalty': 0.0, 'stopping_words': ['\n', '</s>', '###', 'Bot:', 'User:', 'You:', '<|im_end|>'], 'max_input_tokens': 1024, 'best_of': 4, 'max_output_tokens': 64, 'reward_max_token_input': 256}
formatter: {'memory_template': "{bot_name}'s Persona: {memory}\n####\n", 'prompt_template': '{prompt}\n<START>\n', 'bot_template': '{bot_name}: {message}\n', 'user_template': '{user_name}: {message}\n', 'response_template': '{bot_name}:', 'truncate_by_message': False}
reward_formatter: {'bot_template': '{bot_name}: {message}\n', 'memory_template': '', 'prompt_template': '', 'response_template': '{bot_name}:', 'truncate_by_message': False, 'user_template': '{user_name}: {message}\n'}
Resubmit model
Running pipeline stage MKMLizer
Starting job with name zonemercy-lexical-nemo-1518-v16-mkmlizer
Waiting for job on zonemercy-lexical-nemo-1518-v16-mkmlizer to finish
zonemercy-lexical-nemo-1518-v16-mkmlizer: ╔═════════════════════════════════════════════════════════════════════╗
zonemercy-lexical-nemo-1518-v16-mkmlizer: ║     _____            __           __                                ║
zonemercy-lexical-nemo-1518-v16-mkmlizer: ║    / _/ /_ ___    __/ /  ___ ___ / /                                ║
zonemercy-lexical-nemo-1518-v16-mkmlizer: ║   / _/ / // / |/|/ / _ \/ -_) -_) /                                 ║
zonemercy-lexical-nemo-1518-v16-mkmlizer: ║  /_//_/\_, /|__,__/_//_/\__/\__/_/                                  ║
zonemercy-lexical-nemo-1518-v16-mkmlizer: ║       /___/                                                         ║
zonemercy-lexical-nemo-1518-v16-mkmlizer: ║                                                                     ║
zonemercy-lexical-nemo-1518-v16-mkmlizer: ║  Version: 0.9.9                                                     ║
zonemercy-lexical-nemo-1518-v16-mkmlizer: ║  Copyright 2023 MK ONE TECHNOLOGIES Inc.                            ║
zonemercy-lexical-nemo-1518-v16-mkmlizer: ║  https://mk1.ai                                                     ║
zonemercy-lexical-nemo-1518-v16-mkmlizer: ║                                                                     ║
zonemercy-lexical-nemo-1518-v16-mkmlizer: ║  The license key for the current software has been verified as      ║
zonemercy-lexical-nemo-1518-v16-mkmlizer: ║  belonging to:                                                      ║
zonemercy-lexical-nemo-1518-v16-mkmlizer: ║                                                                     ║
zonemercy-lexical-nemo-1518-v16-mkmlizer: ║  Chai Research Corp.                                                ║
zonemercy-lexical-nemo-1518-v16-mkmlizer: ║  Account ID: 7997a29f-0ceb-4cc7-9adf-840c57b4ae6f                   ║
zonemercy-lexical-nemo-1518-v16-mkmlizer: ║  Expiration: 2024-10-15 23:59:59                                    ║
zonemercy-lexical-nemo-1518-v16-mkmlizer: ║                                                                     ║
zonemercy-lexical-nemo-1518-v16-mkmlizer: ╚═════════════════════════════════════════════════════════════════════╝
zonemercy-lexical-nemo-1518-v16-mkmlizer: Downloaded to shared memory in 87.553s
zonemercy-lexical-nemo-1518-v16-mkmlizer: quantizing model to /dev/shm/model_cache, profile:s0, folder:/tmp/tmp42ynnyhy, device:0
zonemercy-lexical-nemo-1518-v16-mkmlizer: Saving flywheel model at /dev/shm/model_cache
zonemercy-lexical-nemo-1518-v16-mkmlizer: quantized model in 39.617s
zonemercy-lexical-nemo-1518-v16-mkmlizer: Processed model zonemercy/Lexical-Nemo-v4-1k1e5 in 127.171s
zonemercy-lexical-nemo-1518-v16-mkmlizer: creating bucket guanaco-mkml-models
zonemercy-lexical-nemo-1518-v16-mkmlizer: cp /dev/shm/model_cache/config.json s3://guanaco-mkml-models/zonemercy-lexical-nemo-1518-v16/config.json
zonemercy-lexical-nemo-1518-v16-mkmlizer: cp /dev/shm/model_cache/special_tokens_map.json s3://guanaco-mkml-models/zonemercy-lexical-nemo-1518-v16/special_tokens_map.json
zonemercy-lexical-nemo-1518-v16-mkmlizer: cp /dev/shm/model_cache/tokenizer_config.json s3://guanaco-mkml-models/zonemercy-lexical-nemo-1518-v16/tokenizer_config.json
zonemercy-lexical-nemo-1518-v16-mkmlizer: cp /dev/shm/model_cache/tokenizer.json s3://guanaco-mkml-models/zonemercy-lexical-nemo-1518-v16/tokenizer.json
zonemercy-lexical-nemo-1518-v16-mkmlizer: cp /dev/shm/model_cache/flywheel_model.0.safetensors s3://guanaco-mkml-models/zonemercy-lexical-nemo-1518-v16/flywheel_model.0.safetensors
zonemercy-lexical-nemo-1518-v16-mkmlizer: loading reward model from ChaiML/gpt2_xl_pairwise_89m_step_347634
zonemercy-lexical-nemo-1518-v16-mkmlizer: 
Loading 0:   0%|          | 0/363 [00:00<?, ?it/s]
Loading 0:   1%|▏         | 5/363 [00:00<00:14, 25.15it/s]
Loading 0:   3%|▎         | 10/363 [00:00<00:11, 30.81it/s]
Loading 0:   4%|▍         | 14/363 [00:00<00:12, 26.90it/s]
Loading 0:   6%|▌         | 21/363 [00:00<00:09, 37.68it/s]
Loading 0:   7%|▋         | 26/363 [00:01<00:15, 22.30it/s]
Loading 0:   9%|▉         | 32/363 [00:01<00:13, 24.88it/s]
Loading 0:  11%|█         | 39/363 [00:01<00:10, 30.89it/s]
Loading 0:  12%|█▏        | 43/363 [00:01<00:10, 30.09it/s]
Loading 0:  13%|█▎        | 48/363 [00:01<00:09, 32.39it/s]
Loading 0:  14%|█▍        | 52/363 [00:01<00:09, 31.16it/s]
Loading 0:  15%|█▌        | 56/363 [00:01<00:10, 30.31it/s]
Loading 0:  17%|█▋        | 61/363 [00:02<00:11, 26.56it/s]
Loading 0:  18%|█▊        | 64/363 [00:02<00:12, 23.48it/s]
Loading 0:  20%|█▉        | 71/363 [00:02<00:09, 30.40it/s]
Loading 0:  21%|██        | 75/363 [00:02<00:09, 30.10it/s]
Loading 0:  22%|██▏       | 79/363 [00:02<00:09, 29.41it/s]
Loading 0:  23%|██▎       | 84/363 [00:02<00:08, 31.59it/s]
Loading 0:  24%|██▍       | 88/363 [00:03<00:09, 30.30it/s]
Loading 0:  26%|██▌       | 93/363 [00:03<00:08, 32.48it/s]
Loading 0:  27%|██▋       | 97/363 [00:03<00:08, 30.85it/s]
Loading 0:  28%|██▊       | 101/363 [00:03<00:10, 25.56it/s]
Loading 0:  29%|██▊       | 104/363 [00:03<00:11, 22.79it/s]
Loading 0:  31%|███       | 111/363 [00:03<00:08, 29.87it/s]
Loading 0:  32%|███▏      | 115/363 [00:03<00:08, 29.41it/s]
Loading 0:  33%|███▎      | 120/363 [00:04<00:07, 31.91it/s]
Loading 0:  34%|███▍      | 124/363 [00:04<00:07, 30.49it/s]
Loading 0:  36%|███▌      | 129/363 [00:04<00:07, 32.28it/s]
Loading 0:  37%|███▋      | 133/363 [00:04<00:07, 30.60it/s]
Loading 0:  38%|███▊      | 137/363 [00:04<00:07, 31.01it/s]
Loading 0:  39%|███▉      | 142/363 [00:04<00:08, 27.04it/s]
Loading 0:  40%|███▉      | 145/363 [00:05<00:08, 25.56it/s]
Loading 0:  41%|████      | 149/363 [00:05<00:08, 24.60it/s]
Loading 0:  43%|████▎     | 156/363 [00:05<00:06, 31.67it/s]
Loading 0:  44%|████▍     | 160/363 [00:05<00:06, 30.47it/s]
Loading 0:  45%|████▌     | 165/363 [00:05<00:06, 32.43it/s]
Loading 0:  47%|████▋     | 169/363 [00:05<00:06, 30.36it/s]
Loading 0:  48%|████▊     | 174/363 [00:05<00:05, 31.78it/s]
Loading 0:  49%|████▉     | 178/363 [00:06<00:06, 30.15it/s]
Loading 0:  50%|█████     | 182/363 [00:06<00:07, 25.01it/s]
Loading 0:  51%|█████     | 185/363 [00:06<00:08, 22.16it/s]
Loading 0:  53%|█████▎    | 192/363 [00:06<00:05, 29.45it/s]
Loading 0:  54%|█████▍    | 196/363 [00:06<00:05, 29.07it/s]
Loading 0:  55%|█████▌    | 201/363 [00:06<00:05, 31.86it/s]
Loading 0:  56%|█████▋    | 205/363 [00:07<00:05, 30.74it/s]
Loading 0:  58%|█████▊    | 210/363 [00:07<00:04, 32.98it/s]
Loading 0:  59%|█████▉    | 214/363 [00:07<00:04, 31.61it/s]
Loading 0:  60%|██████    | 218/363 [00:07<00:04, 31.74it/s]
Loading 0:  61%|██████▏   | 223/363 [00:07<00:05, 26.97it/s]
Loading 0:  62%|██████▏   | 226/363 [00:07<00:05, 25.58it/s]
Loading 0:  63%|██████▎   | 230/363 [00:08<00:05, 24.53it/s]
Loading 0:  65%|██████▌   | 237/363 [00:08<00:03, 31.56it/s]
Loading 0:  66%|██████▋   | 241/363 [00:08<00:03, 30.58it/s]
Loading 0:  68%|██████▊   | 246/363 [00:08<00:03, 32.04it/s]
Loading 0:  69%|██████▉   | 250/363 [00:08<00:03, 30.44it/s]
Loading 0:  70%|███████   | 255/363 [00:08<00:03, 32.78it/s]
Loading 0:  71%|███████▏  | 259/363 [00:08<00:03, 30.42it/s]
Loading 0:  72%|███████▏  | 263/363 [00:09<00:03, 25.22it/s]
Loading 0:  73%|███████▎  | 266/363 [00:09<00:04, 22.68it/s]
Loading 0:  75%|███████▌  | 273/363 [00:09<00:03, 29.53it/s]
Loading 0:  76%|███████▋  | 277/363 [00:09<00:02, 29.05it/s]
Loading 0:  78%|███████▊  | 282/363 [00:09<00:02, 31.94it/s]
Loading 0:  79%|███████▉  | 286/363 [00:09<00:02, 30.90it/s]
Loading 0:  80%|████████  | 291/363 [00:09<00:02, 33.12it/s]
Loading 0:  81%|████████▏ | 295/363 [00:10<00:02, 31.75it/s]
Loading 0:  82%|████████▏ | 299/363 [00:10<00:02, 31.51it/s]
Loading 0:  84%|████████▎ | 304/363 [00:10<00:02, 27.02it/s]
Loading 0:  85%|████████▍ | 307/363 [00:10<00:02, 25.59it/s]
Loading 0:  86%|████████▌ | 311/363 [00:10<00:02, 24.56it/s]
Loading 0:  88%|████████▊ | 318/363 [00:10<00:01, 31.66it/s]
Loading 0:  89%|████████▊ | 322/363 [00:11<00:01, 30.76it/s]
Loading 0:  90%|█████████ | 327/363 [00:11<00:01, 33.22it/s]
Loading 0:  91%|█████████ | 331/363 [00:11<00:01, 31.39it/s]
Loading 0:  93%|█████████▎| 336/363 [00:11<00:00, 32.67it/s]
Loading 0:  94%|█████████▎| 340/363 [00:11<00:00, 30.53it/s]
Loading 0:  95%|█████████▍| 344/363 [00:18<00:09,  1.99it/s]
Loading 0:  96%|█████████▌| 348/363 [00:18<00:05,  2.69it/s]
Loading 0:  97%|█████████▋| 353/363 [00:18<00:02,  3.91it/s]
Loading 0:  98%|█████████▊| 357/363 [00:19<00:01,  5.06it/s]
                                                            
/opt/conda/lib/python3.10/site-packages/transformers/models/auto/configuration_auto.py:957: FutureWarning: The `use_auth_token` argument is deprecated and will be removed in v5 of Transformers. Please use `token` instead.
zonemercy-lexical-nemo-1518-v16-mkmlizer:   warnings.warn(
zonemercy-lexical-nemo-1518-v16-mkmlizer: /opt/conda/lib/python3.10/site-packages/transformers/models/auto/tokenization_auto.py:785: FutureWarning: The `use_auth_token` argument is deprecated and will be removed in v5 of Transformers. Please use `token` instead.
zonemercy-lexical-nemo-1518-v16-mkmlizer:   warnings.warn(
zonemercy-lexical-nemo-1518-v16-mkmlizer: /opt/conda/lib/python3.10/site-packages/transformers/models/auto/auto_factory.py:469: FutureWarning: The `use_auth_token` argument is deprecated and will be removed in v5 of Transformers. Please use `token` instead.
zonemercy-lexical-nemo-1518-v16-mkmlizer:   warnings.warn(
zonemercy-lexical-nemo-1518-v16-mkmlizer: 
Downloading shards:   0%|          | 0/2 [00:00<?, ?it/s]
Downloading shards:  50%|█████     | 1/2 [00:05<00:05,  5.56s/it]
Downloading shards: 100%|██████████| 2/2 [00:08<00:00,  3.93s/it]
Downloading shards: 100%|██████████| 2/2 [00:08<00:00,  4.17s/it]
zonemercy-lexical-nemo-1518-v16-mkmlizer: 
Loading checkpoint shards:   0%|          | 0/2 [00:00<?, ?it/s]
Loading checkpoint shards:  50%|█████     | 1/2 [00:00<00:00,  2.33it/s]
Loading checkpoint shards: 100%|██████████| 2/2 [00:00<00:00,  3.88it/s]
Loading checkpoint shards: 100%|██████████| 2/2 [00:00<00:00,  3.53it/s]
zonemercy-lexical-nemo-1518-v16-mkmlizer: Saving model to /tmp/reward_cache/reward.tensors
zonemercy-lexical-nemo-1518-v16-mkmlizer: Saving duration: 1.436s
zonemercy-lexical-nemo-1518-v16-mkmlizer: Processed model ChaiML/gpt2_xl_pairwise_89m_step_347634 in 13.801s
zonemercy-lexical-nemo-1518-v16-mkmlizer: creating bucket guanaco-reward-models
zonemercy-lexical-nemo-1518-v16-mkmlizer: Bucket 's3://guanaco-reward-models/' created
zonemercy-lexical-nemo-1518-v16-mkmlizer: uploading /tmp/reward_cache to s3://guanaco-reward-models/zonemercy-lexical-nemo-1518-v16_reward
zonemercy-lexical-nemo-1518-v16-mkmlizer: cp /tmp/reward_cache/special_tokens_map.json s3://guanaco-reward-models/zonemercy-lexical-nemo-1518-v16_reward/special_tokens_map.json
zonemercy-lexical-nemo-1518-v16-mkmlizer: cp /tmp/reward_cache/config.json s3://guanaco-reward-models/zonemercy-lexical-nemo-1518-v16_reward/config.json
zonemercy-lexical-nemo-1518-v16-mkmlizer: cp /tmp/reward_cache/tokenizer_config.json s3://guanaco-reward-models/zonemercy-lexical-nemo-1518-v16_reward/tokenizer_config.json
zonemercy-lexical-nemo-1518-v16-mkmlizer: cp /tmp/reward_cache/merges.txt s3://guanaco-reward-models/zonemercy-lexical-nemo-1518-v16_reward/merges.txt
zonemercy-lexical-nemo-1518-v16-mkmlizer: cp /tmp/reward_cache/vocab.json s3://guanaco-reward-models/zonemercy-lexical-nemo-1518-v16_reward/vocab.json
zonemercy-lexical-nemo-1518-v16-mkmlizer: cp /tmp/reward_cache/tokenizer.json s3://guanaco-reward-models/zonemercy-lexical-nemo-1518-v16_reward/tokenizer.json
zonemercy-lexical-nemo-1518-v16-mkmlizer: cp /tmp/reward_cache/reward.tensors s3://guanaco-reward-models/zonemercy-lexical-nemo-1518-v16_reward/reward.tensors
Job zonemercy-lexical-nemo-1518-v16-mkmlizer completed after 177.05s with status: succeeded
Stopping job with name zonemercy-lexical-nemo-1518-v16-mkmlizer
Pipeline stage MKMLizer completed in 178.08s
Running pipeline stage MKMLKubeTemplater
Pipeline stage MKMLKubeTemplater completed in 0.11s
Running pipeline stage ISVCDeployer
Creating inference service zonemercy-lexical-nemo-1518-v16
Waiting for inference service zonemercy-lexical-nemo-1518-v16 to be ready
Inference service zonemercy-lexical-nemo-1518-v16 ready after 201.20889258384705s
Pipeline stage ISVCDeployer completed in 203.07s
Running pipeline stage StressChecker
Received healthy response to inference request in 2.3914237022399902s
Received healthy response to inference request in 1.5958425998687744s
Received healthy response to inference request in 1.5877866744995117s
Received healthy response to inference request in 1.6119215488433838s
Received healthy response to inference request in 1.5541224479675293s
5 requests
0 failed requests
5th percentile: 1.5608552932739257
10th percentile: 1.5675881385803223
20th percentile: 1.5810538291931153
30th percentile: 1.5893978595733642
40th percentile: 1.5926202297210694
50th percentile: 1.5958425998687744
60th percentile: 1.6022741794586182
70th percentile: 1.608705759048462
80th percentile: 1.7678219795227053
90th percentile: 2.0796228408813477
95th percentile: 2.2355232715606688
99th percentile: 2.360243616104126
mean time: 1.7482193946838378
Pipeline stage StressChecker completed in 9.75s
zonemercy-lexical-nemo-_1518_v16 status is now deployed due to DeploymentManager action
zonemercy-lexical-nemo-_1518_v16 status is now inactive due to auto deactivation removed underperforming models
zonemercy-lexical-nemo-_1518_v16 status is now torndown due to DeploymentManager action