aetherwiing-mn-12b-starc_2085

submission_id: aetherwiing-mn-12b-starc_2085_v1
developer_uid: aetherwiing
best_of: 4
celo_rating: 1215.45
display_name: MN-12B-Starcannon-v2
family_friendly_score: 0.0
formatter: {'memory_template': "{bot_name}'s Persona: {memory}\n####\n", 'prompt_template': '{prompt}\n<START>\n', 'bot_template': '{bot_name}: {message}\n', 'user_template': '{user_name}: {message}\n', 'response_template': '{bot_name}:', 'truncate_by_message': False}
generation_params: {'temperature': 0.87, 'top_p': 0.81, 'min_p': 0.0, 'top_k': 60, 'presence_penalty': 0.15, 'frequency_penalty': 0.0, 'stopping_words': ['\n'], 'max_input_tokens': 512, 'best_of': 4, 'max_output_tokens': 64, 'reward_max_token_input': 512}
is_internal_developer: False
language_model: aetherwiing/MN-12B-Starcannon-v2
max_input_tokens: 512
max_output_tokens: 64
model_architecture: MistralForCausalLM
model_group: aetherwiing/MN-12B-Starc
model_name: MN-12B-Starcannon-v2
model_num_parameters: 12772070400.0
model_repo: aetherwiing/MN-12B-Starcannon-v2
model_size: 13B
num_battles: 16523
num_wins: 8479
ranking_group: single
reward_formatter: {'bot_template': '{bot_name}: {message}\n', 'memory_template': "{bot_name}'s Persona: {memory}\n####\n", 'prompt_template': '{prompt}\n<START>\n', 'response_template': '{bot_name}:', 'truncate_by_message': False, 'user_template': '{user_name}: {message}\n'}
reward_repo: ChaiML/gpt2_xl_pairwise_89m_step_347634
status: torndown
submission_type: basic
timestamp: 2024-08-01T13:50:35+00:00
us_pacific_date: 2024-08-01
win_ratio: 0.5131634691036737
Download Preference Data
Resubmit model
Running pipeline stage MKMLizer
Starting job with name aetherwiing-mn-12b-starc-2085-v1-mkmlizer
Waiting for job on aetherwiing-mn-12b-starc-2085-v1-mkmlizer to finish
aetherwiing-mn-12b-starc-2085-v1-mkmlizer: ╔═════════════════════════════════════════════════════════════════════╗
aetherwiing-mn-12b-starc-2085-v1-mkmlizer: ║     _____            __           __                                ║
aetherwiing-mn-12b-starc-2085-v1-mkmlizer: ║    / _/ /_ ___    __/ /  ___ ___ / /                                ║
aetherwiing-mn-12b-starc-2085-v1-mkmlizer: ║   / _/ / // / |/|/ / _ \/ -_) -_) /                                 ║
aetherwiing-mn-12b-starc-2085-v1-mkmlizer: ║  /_//_/\_, /|__,__/_//_/\__/\__/_/                                  ║
aetherwiing-mn-12b-starc-2085-v1-mkmlizer: ║       /___/                                                         ║
aetherwiing-mn-12b-starc-2085-v1-mkmlizer: ║                                                                     ║
aetherwiing-mn-12b-starc-2085-v1-mkmlizer: ║  Version: 0.9.7                                                     ║
aetherwiing-mn-12b-starc-2085-v1-mkmlizer: ║  Copyright 2023 MK ONE TECHNOLOGIES Inc.                            ║
aetherwiing-mn-12b-starc-2085-v1-mkmlizer: ║  https://mk1.ai                                                     ║
aetherwiing-mn-12b-starc-2085-v1-mkmlizer: ║                                                                     ║
aetherwiing-mn-12b-starc-2085-v1-mkmlizer: ║  The license key for the current software has been verified as      ║
aetherwiing-mn-12b-starc-2085-v1-mkmlizer: ║  belonging to:                                                      ║
aetherwiing-mn-12b-starc-2085-v1-mkmlizer: ║                                                                     ║
aetherwiing-mn-12b-starc-2085-v1-mkmlizer: ║  Chai Research Corp.                                                ║
aetherwiing-mn-12b-starc-2085-v1-mkmlizer: ║  Account ID: 7997a29f-0ceb-4cc7-9adf-840c57b4ae6f                   ║
aetherwiing-mn-12b-starc-2085-v1-mkmlizer: ║  Expiration: 2024-10-15 23:59:59                                    ║
aetherwiing-mn-12b-starc-2085-v1-mkmlizer: ║                                                                     ║
aetherwiing-mn-12b-starc-2085-v1-mkmlizer: ╚═════════════════════════════════════════════════════════════════════╝
aetherwiing-mn-12b-starc-2085-v1-mkmlizer: Downloaded to shared memory in 44.277s
aetherwiing-mn-12b-starc-2085-v1-mkmlizer: quantizing model to /dev/shm/model_cache, profile:s0, folder:/tmp/tmpzg70qjki, device:0
aetherwiing-mn-12b-starc-2085-v1-mkmlizer: Saving flywheel model at /dev/shm/model_cache
aetherwiing-mn-12b-starc-2085-v1-mkmlizer: quantized model in 36.322s
aetherwiing-mn-12b-starc-2085-v1-mkmlizer: Processed model aetherwiing/MN-12B-Starcannon-v2 in 80.599s
aetherwiing-mn-12b-starc-2085-v1-mkmlizer: creating bucket guanaco-mkml-models
aetherwiing-mn-12b-starc-2085-v1-mkmlizer: Bucket 's3://guanaco-mkml-models/' created
aetherwiing-mn-12b-starc-2085-v1-mkmlizer: uploading /dev/shm/model_cache to s3://guanaco-mkml-models/aetherwiing-mn-12b-starc-2085-v1
aetherwiing-mn-12b-starc-2085-v1-mkmlizer: cp /dev/shm/model_cache/special_tokens_map.json s3://guanaco-mkml-models/aetherwiing-mn-12b-starc-2085-v1/special_tokens_map.json
aetherwiing-mn-12b-starc-2085-v1-mkmlizer: cp /dev/shm/model_cache/config.json s3://guanaco-mkml-models/aetherwiing-mn-12b-starc-2085-v1/config.json
aetherwiing-mn-12b-starc-2085-v1-mkmlizer: cp /dev/shm/model_cache/tokenizer_config.json s3://guanaco-mkml-models/aetherwiing-mn-12b-starc-2085-v1/tokenizer_config.json
aetherwiing-mn-12b-starc-2085-v1-mkmlizer: cp /dev/shm/model_cache/vocab.json s3://guanaco-mkml-models/aetherwiing-mn-12b-starc-2085-v1/vocab.json
aetherwiing-mn-12b-starc-2085-v1-mkmlizer: cp /dev/shm/model_cache/merges.txt s3://guanaco-mkml-models/aetherwiing-mn-12b-starc-2085-v1/merges.txt
aetherwiing-mn-12b-starc-2085-v1-mkmlizer: cp /dev/shm/model_cache/tokenizer.json s3://guanaco-mkml-models/aetherwiing-mn-12b-starc-2085-v1/tokenizer.json
aetherwiing-mn-12b-starc-2085-v1-mkmlizer: cp /dev/shm/model_cache/flywheel_model.0.safetensors s3://guanaco-mkml-models/aetherwiing-mn-12b-starc-2085-v1/flywheel_model.0.safetensors
aetherwiing-mn-12b-starc-2085-v1-mkmlizer: loading reward model from ChaiML/gpt2_xl_pairwise_89m_step_347634
aetherwiing-mn-12b-starc-2085-v1-mkmlizer: 
Loading 0:   0%|          | 0/363 [00:00<?, ?it/s]
Loading 0:   1%|          | 2/363 [00:06<18:13,  3.03s/it]
Loading 0:   2%|▏         | 6/363 [00:06<04:50,  1.23it/s]
Loading 0:   4%|▎         | 13/363 [00:06<01:43,  3.38it/s]
Loading 0:   5%|▍         | 17/363 [00:06<01:10,  4.93it/s]
Loading 0:   6%|▋         | 23/363 [00:06<00:42,  8.05it/s]
Loading 0:   8%|▊         | 29/363 [00:06<00:28, 11.52it/s]
Loading 0:   9%|▉         | 34/363 [00:06<00:22, 14.71it/s]
Loading 0:  11%|█         | 40/363 [00:07<00:19, 16.33it/s]
Loading 0:  12%|█▏        | 44/363 [00:07<00:16, 18.91it/s]
Loading 0:  13%|█▎        | 49/363 [00:07<00:13, 23.35it/s]
Loading 0:  15%|█▍        | 53/363 [00:07<00:12, 25.70it/s]
Loading 0:  16%|█▋        | 59/363 [00:07<00:09, 31.05it/s]
Loading 0:  18%|█▊        | 64/363 [00:07<00:08, 34.79it/s]
Loading 0:  19%|█▉        | 69/363 [00:07<00:09, 31.43it/s]
Loading 0:  21%|██        | 76/363 [00:07<00:07, 39.10it/s]
Loading 0:  22%|██▏       | 81/363 [00:08<00:06, 40.30it/s]
Loading 0:  24%|██▎       | 86/363 [00:08<00:06, 41.71it/s]
Loading 0:  25%|██▌       | 91/363 [00:08<00:06, 42.82it/s]
Loading 0:  26%|██▋       | 96/363 [00:08<00:07, 36.73it/s]
Loading 0:  28%|██▊       | 103/363 [00:08<00:05, 44.27it/s]
Loading 0:  30%|██▉       | 108/363 [00:08<00:05, 44.10it/s]
Loading 0:  31%|███       | 113/363 [00:08<00:05, 43.37it/s]
Loading 0:  33%|███▎      | 118/363 [00:08<00:05, 44.65it/s]
Loading 0:  34%|███▍      | 123/363 [00:09<00:08, 27.16it/s]
Loading 0:  36%|███▌      | 130/363 [00:09<00:06, 34.13it/s]
Loading 0:  37%|███▋      | 135/363 [00:09<00:06, 36.11it/s]
Loading 0:  39%|███▊      | 140/363 [00:09<00:05, 37.72it/s]
Loading 0:  40%|████      | 146/363 [00:09<00:05, 36.68it/s]
Loading 0:  42%|████▏     | 151/363 [00:09<00:05, 36.86it/s]
Loading 0:  43%|████▎     | 157/363 [00:10<00:05, 41.10it/s]
Loading 0:  45%|████▍     | 162/363 [00:10<00:04, 41.44it/s]
Loading 0:  46%|████▌     | 167/363 [00:10<00:04, 42.03it/s]
Loading 0:  47%|████▋     | 172/363 [00:10<00:04, 43.73it/s]
Loading 0:  49%|████▉     | 177/363 [00:10<00:05, 36.07it/s]
Loading 0:  51%|█████     | 184/363 [00:10<00:04, 42.95it/s]
Loading 0:  52%|█████▏    | 189/363 [00:10<00:04, 42.45it/s]
Loading 0:  53%|█████▎    | 194/363 [00:10<00:04, 42.01it/s]
Loading 0:  55%|█████▍    | 199/363 [00:11<00:03, 43.83it/s]
Loading 0:  56%|█████▌    | 204/363 [00:11<00:06, 26.21it/s]
Loading 0:  58%|█████▊    | 211/363 [00:11<00:04, 33.37it/s]
Loading 0:  60%|█████▉    | 216/363 [00:11<00:04, 35.19it/s]
Loading 0:  61%|██████    | 221/363 [00:11<00:03, 36.48it/s]
Loading 0:  62%|██████▏   | 226/363 [00:11<00:03, 38.72it/s]
Loading 0:  64%|██████▎   | 231/363 [00:12<00:03, 33.30it/s]
Loading 0:  66%|██████▌   | 238/363 [00:12<00:03, 40.30it/s]
Loading 0:  67%|██████▋   | 243/363 [00:12<00:02, 40.43it/s]
Loading 0:  68%|██████▊   | 248/363 [00:12<00:02, 41.05it/s]
Loading 0:  70%|██████▉   | 253/363 [00:12<00:02, 42.81it/s]
Loading 0:  71%|███████   | 258/363 [00:12<00:02, 35.26it/s]
Loading 0:  73%|███████▎  | 265/363 [00:12<00:02, 41.85it/s]
Loading 0:  74%|███████▍  | 270/363 [00:12<00:02, 41.85it/s]
Loading 0:  76%|███████▌  | 275/363 [00:13<00:02, 41.88it/s]
Loading 0:  77%|███████▋  | 280/363 [00:13<00:01, 43.34it/s]
Loading 0:  79%|███████▊  | 285/363 [00:13<00:02, 26.45it/s]
Loading 0:  80%|████████  | 292/363 [00:13<00:02, 33.59it/s]
Loading 0:  82%|████████▏ | 297/363 [00:13<00:01, 35.76it/s]
Loading 0:  83%|████████▎ | 302/363 [00:13<00:01, 37.83it/s]
Loading 0:  85%|████████▍ | 308/363 [00:14<00:01, 37.59it/s]
Loading 0:  86%|████████▌ | 313/363 [00:14<00:01, 37.85it/s]
Loading 0:  88%|████████▊ | 319/363 [00:14<00:01, 42.26it/s]
Loading 0:  89%|████████▉ | 324/363 [00:14<00:00, 42.86it/s]
Loading 0:  91%|█████████ | 329/363 [00:14<00:00, 43.08it/s]
Loading 0:  92%|█████████▏| 335/363 [00:14<00:00, 41.26it/s]
Loading 0:  94%|█████████▎| 340/363 [00:14<00:00, 41.12it/s]
Loading 0:  95%|█████████▌| 346/363 [00:14<00:00, 45.20it/s]
Loading 0:  97%|█████████▋| 351/363 [00:15<00:00, 44.98it/s]
Loading 0:  98%|█████████▊| 356/363 [00:15<00:00, 45.23it/s]
Loading 0:  99%|█████████▉| 361/363 [00:15<00:00, 46.36it/s]
                                                            
/opt/conda/lib/python3.10/site-packages/transformers/models/auto/configuration_auto.py:957: FutureWarning: The `use_auth_token` argument is deprecated and will be removed in v5 of Transformers. Please use `token` instead.
aetherwiing-mn-12b-starc-2085-v1-mkmlizer:   warnings.warn(
aetherwiing-mn-12b-starc-2085-v1-mkmlizer: /opt/conda/lib/python3.10/site-packages/transformers/models/auto/tokenization_auto.py:785: FutureWarning: The `use_auth_token` argument is deprecated and will be removed in v5 of Transformers. Please use `token` instead.
aetherwiing-mn-12b-starc-2085-v1-mkmlizer:   warnings.warn(
aetherwiing-mn-12b-starc-2085-v1-mkmlizer: /opt/conda/lib/python3.10/site-packages/transformers/models/auto/auto_factory.py:469: FutureWarning: The `use_auth_token` argument is deprecated and will be removed in v5 of Transformers. Please use `token` instead.
aetherwiing-mn-12b-starc-2085-v1-mkmlizer:   warnings.warn(
aetherwiing-mn-12b-starc-2085-v1-mkmlizer: 
Downloading shards:   0%|          | 0/2 [00:00<?, ?it/s]
Downloading shards:  50%|█████     | 1/2 [00:05<00:05,  5.73s/it]
Downloading shards: 100%|██████████| 2/2 [00:08<00:00,  4.06s/it]
Downloading shards: 100%|██████████| 2/2 [00:08<00:00,  4.31s/it]
aetherwiing-mn-12b-starc-2085-v1-mkmlizer: 
Loading checkpoint shards:   0%|          | 0/2 [00:00<?, ?it/s]
Loading checkpoint shards:  50%|█████     | 1/2 [00:00<00:00,  2.40it/s]
Loading checkpoint shards: 100%|██████████| 2/2 [00:00<00:00,  3.89it/s]
Loading checkpoint shards: 100%|██████████| 2/2 [00:00<00:00,  3.56it/s]
aetherwiing-mn-12b-starc-2085-v1-mkmlizer: Saving model to /tmp/reward_cache/reward.tensors
aetherwiing-mn-12b-starc-2085-v1-mkmlizer: Saving duration: 1.377s
aetherwiing-mn-12b-starc-2085-v1-mkmlizer: Processed model ChaiML/gpt2_xl_pairwise_89m_step_347634 in 13.645s
aetherwiing-mn-12b-starc-2085-v1-mkmlizer: creating bucket guanaco-reward-models
aetherwiing-mn-12b-starc-2085-v1-mkmlizer: Bucket 's3://guanaco-reward-models/' created
aetherwiing-mn-12b-starc-2085-v1-mkmlizer: uploading /tmp/reward_cache to s3://guanaco-reward-models/aetherwiing-mn-12b-starc-2085-v1_reward
aetherwiing-mn-12b-starc-2085-v1-mkmlizer: cp /tmp/reward_cache/tokenizer_config.json s3://guanaco-reward-models/aetherwiing-mn-12b-starc-2085-v1_reward/tokenizer_config.json
aetherwiing-mn-12b-starc-2085-v1-mkmlizer: cp /tmp/reward_cache/merges.txt s3://guanaco-reward-models/aetherwiing-mn-12b-starc-2085-v1_reward/merges.txt
aetherwiing-mn-12b-starc-2085-v1-mkmlizer: cp /tmp/reward_cache/vocab.json s3://guanaco-reward-models/aetherwiing-mn-12b-starc-2085-v1_reward/vocab.json
aetherwiing-mn-12b-starc-2085-v1-mkmlizer: cp /tmp/reward_cache/tokenizer.json s3://guanaco-reward-models/aetherwiing-mn-12b-starc-2085-v1_reward/tokenizer.json
aetherwiing-mn-12b-starc-2085-v1-mkmlizer: cp /tmp/reward_cache/reward.tensors s3://guanaco-reward-models/aetherwiing-mn-12b-starc-2085-v1_reward/reward.tensors
Job aetherwiing-mn-12b-starc-2085-v1-mkmlizer completed after 126.05s with status: succeeded
Stopping job with name aetherwiing-mn-12b-starc-2085-v1-mkmlizer
Pipeline stage MKMLizer completed in 127.04s
Running pipeline stage MKMLKubeTemplater
Pipeline stage MKMLKubeTemplater completed in 0.11s
Running pipeline stage ISVCDeployer
Creating inference service aetherwiing-mn-12b-starc-2085-v1
Waiting for inference service aetherwiing-mn-12b-starc-2085-v1 to be ready
Inference service aetherwiing-mn-12b-starc-2085-v1 ready after 130.96150517463684s
Pipeline stage ISVCDeployer completed in 132.52s
Running pipeline stage StressChecker
Received healthy response to inference request in 2.426522731781006s
Received healthy response to inference request in 1.5728049278259277s
Received healthy response to inference request in 1.4998629093170166s
Received healthy response to inference request in 1.4549837112426758s
Received healthy response to inference request in 1.588564395904541s
5 requests
0 failed requests
5th percentile: 1.463959550857544
10th percentile: 1.472935390472412
20th percentile: 1.4908870697021483
30th percentile: 1.5144513130187989
40th percentile: 1.5436281204223632
50th percentile: 1.5728049278259277
60th percentile: 1.579108715057373
70th percentile: 1.5854125022888184
80th percentile: 1.756156063079834
90th percentile: 2.09133939743042
95th percentile: 2.258931064605713
99th percentile: 2.3930043983459472
mean time: 1.7085477352142333
Pipeline stage StressChecker completed in 9.82s
aetherwiing-mn-12b-starc_2085_v1 status is now deployed due to DeploymentManager action
aetherwiing-mn-12b-starc_2085_v1 status is now inactive due to auto deactivation removed underperforming models
admin requested tearing down of aetherwiing-mn-12b-starc_2085_v1
Running pipeline stage ISVCDeleter
Checking if service aetherwiing-mn-12b-starc-2085-v1 is running
Tearing down inference service aetherwiing-mn-12b-starc-2085-v1
Service aetherwiing-mn-12b-starc-2085-v1 has been torndown
Pipeline stage ISVCDeleter completed in 5.23s
Running pipeline stage MKMLModelDeleter
Cleaning model data from S3
Cleaning model data from model cache
Deleting key aetherwiing-mn-12b-starc-2085-v1/config.json from bucket guanaco-mkml-models
Deleting key aetherwiing-mn-12b-starc-2085-v1/flywheel_model.0.safetensors from bucket guanaco-mkml-models
Deleting key aetherwiing-mn-12b-starc-2085-v1/merges.txt from bucket guanaco-mkml-models
Deleting key aetherwiing-mn-12b-starc-2085-v1/special_tokens_map.json from bucket guanaco-mkml-models
Deleting key aetherwiing-mn-12b-starc-2085-v1/tokenizer.json from bucket guanaco-mkml-models
Deleting key aetherwiing-mn-12b-starc-2085-v1/tokenizer_config.json from bucket guanaco-mkml-models
Deleting key aetherwiing-mn-12b-starc-2085-v1/vocab.json from bucket guanaco-mkml-models
Cleaning model data from model cache
Deleting key aetherwiing-mn-12b-starc-2085-v1_reward/config.json from bucket guanaco-reward-models
Deleting key aetherwiing-mn-12b-starc-2085-v1_reward/merges.txt from bucket guanaco-reward-models
Deleting key aetherwiing-mn-12b-starc-2085-v1_reward/reward.tensors from bucket guanaco-reward-models
Deleting key aetherwiing-mn-12b-starc-2085-v1_reward/special_tokens_map.json from bucket guanaco-reward-models
Deleting key aetherwiing-mn-12b-starc-2085-v1_reward/tokenizer.json from bucket guanaco-reward-models
Deleting key aetherwiing-mn-12b-starc-2085-v1_reward/tokenizer_config.json from bucket guanaco-reward-models
Deleting key aetherwiing-mn-12b-starc-2085-v1_reward/vocab.json from bucket guanaco-reward-models
Pipeline stage MKMLModelDeleter completed in 6.86s
aetherwiing-mn-12b-starc_2085_v1 status is now torndown due to DeploymentManager action