submission_id: zonemercy-lexical-nemov4_7345_v3
developer_uid: zonemercy
alignment_samples: 0
best_of: 4
celo_rating: 1237.13
display_name: zonemercy-lexical-nemov4_7345_v3
formatter: {'memory_template': "{bot_name}'s Persona: {memory}\n####\n", 'prompt_template': '{prompt}\n<START>\n', 'bot_template': '{bot_name}: {message}\n', 'user_template': '{user_name}: {message}\n', 'response_template': '{bot_name}:', 'truncate_by_message': False}
generation_params: {'temperature': 0.9, 'top_p': 1.0, 'min_p': 0.05, 'top_k': 80, 'presence_penalty': 0.0, 'frequency_penalty': 0.0, 'stopping_words': ['\n', '</s>', '###', 'Bot:', 'User:', 'You:', '<|im_end|>'], 'max_input_tokens': 1024, 'best_of': 4, 'max_output_tokens': 64, 'reward_max_token_input': 256}
is_internal_developer: True
language_model: zonemercy/Lexical-Nemov4-1k5e6
max_input_tokens: 1024
max_output_tokens: 64
model_architecture: MistralForCausalLM
model_group: zonemercy/Lexical-Nemov4
model_name: zonemercy-lexical-nemov4_7345_v3
model_num_parameters: 12772070400.0
model_repo: zonemercy/Lexical-Nemov4-1k5e6
model_size: 13B
num_battles: 20395
num_wins: 10690
propriety_score: 0.6936008676789588
propriety_total_count: 1844.0
ranking_group: single
reward_formatter: {'bot_template': '{bot_name}: {message}\n', 'memory_template': '', 'prompt_template': '', 'response_template': '{bot_name}:', 'truncate_by_message': False, 'user_template': '{user_name}: {message}\n'}
reward_repo: ChaiML/gpt2_xl_pairwise_89m_step_347634
status: torndown
submission_type: basic
timestamp: 2024-08-04T05:25:15+00:00
us_pacific_date: 2024-08-03
win_ratio: 0.5241480755087031
Download Preference Data
Resubmit model
Running pipeline stage MKMLizer
Starting job with name zonemercy-lexical-nemov4-7345-v3-mkmlizer
Waiting for job on zonemercy-lexical-nemov4-7345-v3-mkmlizer to finish
Stopping job with name zonemercy-lexical-nemov4-7345-v3-mkmlizer
%s, retrying in %s seconds...
Starting job with name zonemercy-lexical-nemov4-7345-v3-mkmlizer
Waiting for job on zonemercy-lexical-nemov4-7345-v3-mkmlizer to finish
zonemercy-lexical-nemov4-7345-v3-mkmlizer: ╔═════════════════════════════════════════════════════════════════════╗
zonemercy-lexical-nemov4-7345-v3-mkmlizer: ║ _____ __ __ ║
zonemercy-lexical-nemov4-7345-v3-mkmlizer: ║ / _/ /_ ___ __/ / ___ ___ / / ║
zonemercy-lexical-nemov4-7345-v3-mkmlizer: ║ / _/ / // / |/|/ / _ \/ -_) -_) / ║
zonemercy-lexical-nemov4-7345-v3-mkmlizer: ║ /_//_/\_, /|__,__/_//_/\__/\__/_/ ║
zonemercy-lexical-nemov4-7345-v3-mkmlizer: ║ /___/ ║
zonemercy-lexical-nemov4-7345-v3-mkmlizer: ║ ║
zonemercy-lexical-nemov4-7345-v3-mkmlizer: ║ Version: 0.9.9 ║
zonemercy-lexical-nemov4-7345-v3-mkmlizer: ║ Copyright 2023 MK ONE TECHNOLOGIES Inc. ║
zonemercy-lexical-nemov4-7345-v3-mkmlizer: ║ https://mk1.ai ║
zonemercy-lexical-nemov4-7345-v3-mkmlizer: ║ ║
zonemercy-lexical-nemov4-7345-v3-mkmlizer: ║ The license key for the current software has been verified as ║
zonemercy-lexical-nemov4-7345-v3-mkmlizer: ║ belonging to: ║
zonemercy-lexical-nemov4-7345-v3-mkmlizer: ║ ║
zonemercy-lexical-nemov4-7345-v3-mkmlizer: ║ Chai Research Corp. ║
zonemercy-lexical-nemov4-7345-v3-mkmlizer: ║ Account ID: 7997a29f-0ceb-4cc7-9adf-840c57b4ae6f ║
zonemercy-lexical-nemov4-7345-v3-mkmlizer: ║ Expiration: 2024-10-15 23:59:59 ║
zonemercy-lexical-nemov4-7345-v3-mkmlizer: ║ ║
zonemercy-lexical-nemov4-7345-v3-mkmlizer: ╚═════════════════════════════════════════════════════════════════════╝
Retrying (%r) after connection broken by '%r': %s
zonemercy-lexical-nemov4-7345-v3-mkmlizer: Downloaded to shared memory in 61.610s
zonemercy-lexical-nemov4-7345-v3-mkmlizer: quantizing model to /dev/shm/model_cache, profile:s0, folder:/tmp/tmpez0vk_ce, device:0
zonemercy-lexical-nemov4-7345-v3-mkmlizer: Saving flywheel model at /dev/shm/model_cache
zonemercy-lexical-nemov4-7345-v3-mkmlizer: quantized model in 41.398s
zonemercy-lexical-nemov4-7345-v3-mkmlizer: Processed model zonemercy/Lexical-Nemov4-1k5e6 in 103.008s
zonemercy-lexical-nemov4-7345-v3-mkmlizer: creating bucket guanaco-mkml-models
zonemercy-lexical-nemov4-7345-v3-mkmlizer: Bucket 's3://guanaco-mkml-models/' created
zonemercy-lexical-nemov4-7345-v3-mkmlizer: uploading /dev/shm/model_cache to s3://guanaco-mkml-models/zonemercy-lexical-nemov4-7345-v3
zonemercy-lexical-nemov4-7345-v3-mkmlizer: cp /dev/shm/model_cache/config.json s3://guanaco-mkml-models/zonemercy-lexical-nemov4-7345-v3/config.json
zonemercy-lexical-nemov4-7345-v3-mkmlizer: cp /dev/shm/model_cache/special_tokens_map.json s3://guanaco-mkml-models/zonemercy-lexical-nemov4-7345-v3/special_tokens_map.json
zonemercy-lexical-nemov4-7345-v3-mkmlizer: cp /dev/shm/model_cache/tokenizer_config.json s3://guanaco-mkml-models/zonemercy-lexical-nemov4-7345-v3/tokenizer_config.json
zonemercy-lexical-nemov4-7345-v3-mkmlizer: cp /dev/shm/model_cache/tokenizer.json s3://guanaco-mkml-models/zonemercy-lexical-nemov4-7345-v3/tokenizer.json
zonemercy-lexical-nemov4-7345-v3-mkmlizer: cp /dev/shm/model_cache/flywheel_model.0.safetensors s3://guanaco-mkml-models/zonemercy-lexical-nemov4-7345-v3/flywheel_model.0.safetensors
zonemercy-lexical-nemov4-7345-v3-mkmlizer: loading reward model from ChaiML/gpt2_xl_pairwise_89m_step_347634
zonemercy-lexical-nemov4-7345-v3-mkmlizer: Loading 0: 0%| | 0/363 [00:00<?, ?it/s] Loading 0: 1%|▏ | 5/363 [00:00<00:15, 23.09it/s] Loading 0: 3%|▎ | 10/363 [00:00<00:12, 29.05it/s] Loading 0: 4%|▍ | 14/363 [00:00<00:13, 25.17it/s] Loading 0: 6%|▌ | 20/363 [00:00<00:10, 34.22it/s] Loading 0: 7%|▋ | 24/363 [00:00<00:15, 21.76it/s] Loading 0: 7%|▋ | 27/363 [00:01<00:15, 21.42it/s] Loading 0: 9%|▊ | 31/363 [00:01<00:13, 24.98it/s] Loading 0: 10%|▉ | 35/363 [00:01<00:12, 26.72it/s] Loading 0: 11%|█ | 39/363 [00:01<00:11, 27.78it/s] Loading 0: 12%|█▏ | 43/363 [00:01<00:11, 27.22it/s] Loading 0: 13%|█▎ | 48/363 [00:01<00:10, 30.08it/s] Loading 0: 14%|█▍ | 52/363 [00:01<00:10, 28.75it/s] Loading 0: 15%|█▌ | 56/363 [00:02<00:10, 28.82it/s] Loading 0: 17%|█▋ | 61/363 [00:02<00:12, 23.29it/s] Loading 0: 18%|█▊ | 64/363 [00:02<00:14, 20.68it/s] Loading 0: 20%|█▉ | 71/363 [00:02<00:10, 27.64it/s] Loading 0: 21%|██ | 75/363 [00:02<00:10, 27.12it/s] Loading 0: 21%|██▏ | 78/363 [00:03<00:11, 25.57it/s] Loading 0: 23%|██▎ | 84/363 [00:03<00:09, 29.87it/s] Loading 0: 24%|██▍ | 88/363 [00:03<00:09, 28.49it/s] Loading 0: 26%|██▌ | 93/363 [00:03<00:08, 30.32it/s] Loading 0: 27%|██▋ | 97/363 [00:03<00:09, 29.01it/s] Loading 0: 28%|██▊ | 101/363 [00:03<00:11, 22.96it/s] Loading 0: 29%|██▊ | 104/363 [00:04<00:12, 20.90it/s] Loading 0: 31%|███ | 111/363 [00:04<00:09, 27.72it/s] Loading 0: 32%|███▏ | 115/363 [00:04<00:09, 27.44it/s] Loading 0: 33%|███▎ | 120/363 [00:04<00:08, 30.10it/s] Loading 0: 34%|███▍ | 124/363 [00:04<00:08, 29.20it/s] Loading 0: 35%|███▌ | 128/363 [00:04<00:07, 31.37it/s] Loading 0: 36%|███▋ | 132/363 [00:04<00:08, 27.65it/s] Loading 0: 38%|███▊ | 137/363 [00:05<00:07, 29.82it/s] Loading 0: 39%|███▉ | 142/363 [00:05<00:08, 25.35it/s] Loading 0: 40%|███▉ | 145/363 [00:05<00:08, 24.31it/s] Loading 0: 41%|████ | 149/363 [00:05<00:09, 23.22it/s] Loading 0: 43%|████▎ | 156/363 [00:05<00:06, 29.99it/s] Loading 0: 44%|████▍ | 160/363 [00:05<00:06, 29.09it/s] Loading 0: 45%|████▌ | 165/363 [00:06<00:06, 30.56it/s] Loading 0: 47%|████▋ | 169/363 [00:06<00:06, 29.01it/s] Loading 0: 48%|████▊ | 174/363 [00:06<00:06, 31.22it/s] Loading 0: 49%|████▉ | 178/363 [00:06<00:06, 29.91it/s] Loading 0: 50%|█████ | 182/363 [00:06<00:07, 24.22it/s] Loading 0: 51%|█████ | 185/363 [00:06<00:08, 21.73it/s] Loading 0: 53%|█████▎ | 192/363 [00:07<00:05, 28.69it/s] Loading 0: 54%|█████▍ | 196/363 [00:07<00:06, 27.79it/s] Loading 0: 55%|█████▌ | 201/363 [00:07<00:05, 29.50it/s] Loading 0: 56%|█████▋ | 205/363 [00:07<00:05, 28.68it/s] Loading 0: 58%|█████▊ | 210/363 [00:07<00:04, 31.21it/s] Loading 0: 59%|█████▉ | 214/363 [00:07<00:04, 30.10it/s] Loading 0: 60%|██████ | 218/363 [00:07<00:04, 30.14it/s] Loading 0: 61%|██████▏ | 223/363 [00:08<00:05, 25.97it/s] Loading 0: 62%|██████▏ | 226/363 [00:08<00:05, 24.65it/s] Loading 0: 63%|██████▎ | 230/363 [00:08<00:05, 23.30it/s] Loading 0: 65%|██████▌ | 237/363 [00:08<00:04, 29.80it/s] Loading 0: 66%|██████▋ | 241/363 [00:08<00:04, 28.97it/s] Loading 0: 68%|██████▊ | 246/363 [00:08<00:03, 31.37it/s] Loading 0: 69%|██████▉ | 250/363 [00:09<00:03, 30.05it/s] Loading 0: 70%|███████ | 255/363 [00:09<00:03, 32.00it/s] Loading 0: 71%|███████▏ | 259/363 [00:09<00:03, 30.15it/s] Loading 0: 72%|███████▏ | 263/363 [00:09<00:04, 23.67it/s] Loading 0: 73%|███████▎ | 266/363 [00:09<00:04, 21.05it/s] Loading 0: 75%|███████▌ | 273/363 [00:10<00:03, 27.83it/s] Loading 0: 76%|███████▋ | 277/363 [00:10<00:03, 27.62it/s] Loading 0: 78%|███████▊ | 282/363 [00:10<00:02, 30.18it/s] Loading 0: 79%|███████▉ | 286/363 [00:10<00:02, 29.08it/s] Loading 0: 80%|████████ | 291/363 [00:10<00:02, 31.41it/s] Loading 0: 81%|████████▏ | 295/363 [00:10<00:02, 29.84it/s] Loading 0: 82%|████████▏ | 299/363 [00:10<00:02, 30.13it/s] Loading 0: 84%|████████▎ | 304/363 [00:11<00:02, 25.70it/s] Loading 0: 85%|████████▍ | 307/363 [00:11<00:02, 24.25it/s] Loading 0: 86%|████████▌ | 311/363 [00:11<00:02, 23.52it/s] Loading 0: 88%|████████▊ | 318/363 [00:11<00:01, 30.10it/s] Loading 0: 89%|████████▊ | 322/363 [00:11<00:01, 29.23it/s] Loading 0: 90%|█████████ | 327/363 [00:11<00:01, 31.57it/s] Loading 0: 91%|█████████ | 331/363 [00:12<00:01, 30.22it/s] Loading 0: 93%|█████████▎| 336/363 [00:12<00:00, 31.79it/s] Loading 0: 94%|█████████▎| 340/363 [00:12<00:00, 30.27it/s] Loading 0: 95%|█████████▍| 344/363 [00:19<00:09, 1.98it/s] Loading 0: 96%|█████████▌| 348/363 [00:19<00:05, 2.66it/s] Loading 0: 97%|█████████▋| 353/363 [00:19<00:02, 3.86it/s] Loading 0: 98%|█████████▊| 357/363 [00:19<00:01, 5.01it/s] /opt/conda/lib/python3.10/site-packages/transformers/models/auto/configuration_auto.py:957: FutureWarning: The `use_auth_token` argument is deprecated and will be removed in v5 of Transformers. Please use `token` instead.
zonemercy-lexical-nemov4-7345-v3-mkmlizer: warnings.warn(
zonemercy-lexical-nemov4-7345-v3-mkmlizer: /opt/conda/lib/python3.10/site-packages/transformers/models/auto/tokenization_auto.py:785: FutureWarning: The `use_auth_token` argument is deprecated and will be removed in v5 of Transformers. Please use `token` instead.
zonemercy-lexical-nemov4-7345-v3-mkmlizer: warnings.warn(
zonemercy-lexical-nemov4-7345-v3-mkmlizer: /opt/conda/lib/python3.10/site-packages/transformers/models/auto/auto_factory.py:469: FutureWarning: The `use_auth_token` argument is deprecated and will be removed in v5 of Transformers. Please use `token` instead.
zonemercy-lexical-nemov4-7345-v3-mkmlizer: warnings.warn(
zonemercy-lexical-nemov4-7345-v3-mkmlizer: Downloading shards: 0%| | 0/2 [00:00<?, ?it/s] Downloading shards: 50%|█████ | 1/2 [00:05<00:05, 5.43s/it] Downloading shards: 100%|██████████| 2/2 [00:08<00:00, 4.11s/it] Downloading shards: 100%|██████████| 2/2 [00:08<00:00, 4.31s/it]
zonemercy-lexical-nemov4-7345-v3-mkmlizer: Loading checkpoint shards: 0%| | 0/2 [00:00<?, ?it/s] Loading checkpoint shards: 50%|█████ | 1/2 [00:00<00:00, 2.29it/s] Loading checkpoint shards: 100%|██████████| 2/2 [00:00<00:00, 3.78it/s] Loading checkpoint shards: 100%|██████████| 2/2 [00:00<00:00, 3.44it/s]
zonemercy-lexical-nemov4-7345-v3-mkmlizer: Saving model to /tmp/reward_cache/reward.tensors
zonemercy-lexical-nemov4-7345-v3-mkmlizer: Saving duration: 1.319s
zonemercy-lexical-nemov4-7345-v3-mkmlizer: Processed model ChaiML/gpt2_xl_pairwise_89m_step_347634 in 13.614s
zonemercy-lexical-nemov4-7345-v3-mkmlizer: creating bucket guanaco-reward-models
zonemercy-lexical-nemov4-7345-v3-mkmlizer: Bucket 's3://guanaco-reward-models/' created
zonemercy-lexical-nemov4-7345-v3-mkmlizer: uploading /tmp/reward_cache to s3://guanaco-reward-models/zonemercy-lexical-nemov4-7345-v3_reward
zonemercy-lexical-nemov4-7345-v3-mkmlizer: cp /tmp/reward_cache/config.json s3://guanaco-reward-models/zonemercy-lexical-nemov4-7345-v3_reward/config.json
zonemercy-lexical-nemov4-7345-v3-mkmlizer: cp /tmp/reward_cache/tokenizer_config.json s3://guanaco-reward-models/zonemercy-lexical-nemov4-7345-v3_reward/tokenizer_config.json
zonemercy-lexical-nemov4-7345-v3-mkmlizer: cp /tmp/reward_cache/special_tokens_map.json s3://guanaco-reward-models/zonemercy-lexical-nemov4-7345-v3_reward/special_tokens_map.json
zonemercy-lexical-nemov4-7345-v3-mkmlizer: cp /tmp/reward_cache/merges.txt s3://guanaco-reward-models/zonemercy-lexical-nemov4-7345-v3_reward/merges.txt
zonemercy-lexical-nemov4-7345-v3-mkmlizer: cp /tmp/reward_cache/vocab.json s3://guanaco-reward-models/zonemercy-lexical-nemov4-7345-v3_reward/vocab.json
zonemercy-lexical-nemov4-7345-v3-mkmlizer: cp /tmp/reward_cache/tokenizer.json s3://guanaco-reward-models/zonemercy-lexical-nemov4-7345-v3_reward/tokenizer.json
zonemercy-lexical-nemov4-7345-v3-mkmlizer: cp /tmp/reward_cache/reward.tensors s3://guanaco-reward-models/zonemercy-lexical-nemov4-7345-v3_reward/reward.tensors
Job zonemercy-lexical-nemov4-7345-v3-mkmlizer completed after 146.82s with status: succeeded
Stopping job with name zonemercy-lexical-nemov4-7345-v3-mkmlizer
Pipeline stage MKMLizer completed in 148.42s
Running pipeline stage MKMLKubeTemplater
Pipeline stage MKMLKubeTemplater completed in 0.10s
Running pipeline stage ISVCDeployer
Creating inference service zonemercy-lexical-nemov4-7345-v3
Waiting for inference service zonemercy-lexical-nemov4-7345-v3 to be ready
Inference service zonemercy-lexical-nemov4-7345-v3 ready after 160.9973316192627s
Pipeline stage ISVCDeployer completed in 162.98s
Running pipeline stage StressChecker
Received healthy response to inference request in 2.5612242221832275s
Received healthy response to inference request in 1.5998585224151611s
Received healthy response to inference request in 1.6812431812286377s
Received healthy response to inference request in 1.5847764015197754s
Received healthy response to inference request in 1.5907225608825684s
5 requests
0 failed requests
5th percentile: 1.585965633392334
10th percentile: 1.5871548652648926
20th percentile: 1.5895333290100098
30th percentile: 1.5925497531890869
40th percentile: 1.5962041378021241
50th percentile: 1.5998585224151611
60th percentile: 1.6324123859405517
70th percentile: 1.6649662494659423
80th percentile: 1.8572393894195558
90th percentile: 2.209231805801392
95th percentile: 2.385228013992309
99th percentile: 2.526024980545044
mean time: 1.803564977645874
Pipeline stage StressChecker completed in 9.76s
zonemercy-lexical-nemov4_7345_v3 status is now deployed due to DeploymentManager action
zonemercy-lexical-nemov4_7345_v3 status is now inactive due to auto deactivation removed underperforming models
zonemercy-lexical-nemov4_7345_v3 status is now torndown due to DeploymentManager action

Usage Metrics

Latency Metrics