submission_id: aetherwiing-mn-12b-stars_6598_v1
developer_uid: aetherwiing
alignment_samples: 2160
alignment_score: 0.09557510482995649
best_of: 4
celo_rating: 1221.51
display_name: MN-12B-Starsong-v1
formatter: {'memory_template': '<|im_start|>system\n{memory}<|im_end|>\n', 'prompt_template': '<|im_start|>user\n{prompt}<|im_end|>\n', 'bot_template': '<|im_start|>assistant\n{bot_name}: {message}<|im_end|>\n', 'user_template': '<|im_start|>user\n{user_name}: {message}<|im_end|>\n', 'response_template': '<|im_start|>assistant\n{bot_name}:', 'truncate_by_message': True}
generation_params: {'temperature': 0.81, 'top_p': 0.87, 'min_p': 0.0, 'top_k': 40, 'presence_penalty': 0.1, 'frequency_penalty': 0.0, 'stopping_words': ['\n'], 'max_input_tokens': 512, 'best_of': 4, 'max_output_tokens': 64, 'reward_max_token_input': 512}
is_internal_developer: False
language_model: aetherwiing/MN-12B-Starsong-v1
max_input_tokens: 512
max_output_tokens: 64
model_architecture: MistralForCausalLM
model_group: aetherwiing/MN-12B-Stars
model_name: MN-12B-Starsong-v1
model_num_parameters: 12772070400.0
model_repo: aetherwiing/MN-12B-Starsong-v1
model_size: 13B
num_battles: 14444
num_wins: 7261
propriety_score: 0.7103825136612022
propriety_total_count: 1281.0
ranking_group: single
reward_formatter: {'bot_template': '{bot_name}: {message}\n', 'memory_template': "{bot_name}'s Persona: {memory}\n####\n", 'prompt_template': '{prompt}\n<START>\n', 'response_template': '{bot_name}:', 'truncate_by_message': False, 'user_template': '{user_name}: {message}\n'}
reward_repo: ChaiML/gpt2_xl_pairwise_89m_step_347634
status: torndown
submission_type: basic
timestamp: 2024-08-04T02:07:12+00:00
us_pacific_date: 2024-08-03
win_ratio: 0.5027000830794793
Download Preference Data
Resubmit model
Running pipeline stage MKMLizer
Starting job with name aetherwiing-mn-12b-stars-6598-v1-mkmlizer
Waiting for job on aetherwiing-mn-12b-stars-6598-v1-mkmlizer to finish
aetherwiing-mn-12b-stars-6598-v1-mkmlizer: ╔═════════════════════════════════════════════════════════════════════╗
aetherwiing-mn-12b-stars-6598-v1-mkmlizer: ║ _____ __ __ ║
aetherwiing-mn-12b-stars-6598-v1-mkmlizer: ║ / _/ /_ ___ __/ / ___ ___ / / ║
aetherwiing-mn-12b-stars-6598-v1-mkmlizer: ║ / _/ / // / |/|/ / _ \/ -_) -_) / ║
aetherwiing-mn-12b-stars-6598-v1-mkmlizer: ║ /_//_/\_, /|__,__/_//_/\__/\__/_/ ║
aetherwiing-mn-12b-stars-6598-v1-mkmlizer: ║ /___/ ║
aetherwiing-mn-12b-stars-6598-v1-mkmlizer: ║ ║
aetherwiing-mn-12b-stars-6598-v1-mkmlizer: ║ Version: 0.9.9 ║
aetherwiing-mn-12b-stars-6598-v1-mkmlizer: ║ Copyright 2023 MK ONE TECHNOLOGIES Inc. ║
aetherwiing-mn-12b-stars-6598-v1-mkmlizer: ║ https://mk1.ai ║
aetherwiing-mn-12b-stars-6598-v1-mkmlizer: ║ ║
aetherwiing-mn-12b-stars-6598-v1-mkmlizer: ║ The license key for the current software has been verified as ║
aetherwiing-mn-12b-stars-6598-v1-mkmlizer: ║ belonging to: ║
aetherwiing-mn-12b-stars-6598-v1-mkmlizer: ║ ║
aetherwiing-mn-12b-stars-6598-v1-mkmlizer: ║ Chai Research Corp. ║
aetherwiing-mn-12b-stars-6598-v1-mkmlizer: ║ Account ID: 7997a29f-0ceb-4cc7-9adf-840c57b4ae6f ║
aetherwiing-mn-12b-stars-6598-v1-mkmlizer: ║ Expiration: 2024-10-15 23:59:59 ║
aetherwiing-mn-12b-stars-6598-v1-mkmlizer: ║ ║
aetherwiing-mn-12b-stars-6598-v1-mkmlizer: ╚═════════════════════════════════════════════════════════════════════╝
aetherwiing-mn-12b-stars-6598-v1-mkmlizer: Downloaded to shared memory in 45.744s
aetherwiing-mn-12b-stars-6598-v1-mkmlizer: quantizing model to /dev/shm/model_cache, profile:s0, folder:/tmp/tmp_cylvkgh, device:0
aetherwiing-mn-12b-stars-6598-v1-mkmlizer: Saving flywheel model at /dev/shm/model_cache
aetherwiing-mn-12b-stars-6598-v1-mkmlizer: quantized model in 36.487s
aetherwiing-mn-12b-stars-6598-v1-mkmlizer: Processed model aetherwiing/MN-12B-Starsong-v1 in 82.231s
aetherwiing-mn-12b-stars-6598-v1-mkmlizer: creating bucket guanaco-mkml-models
aetherwiing-mn-12b-stars-6598-v1-mkmlizer: Bucket 's3://guanaco-mkml-models/' created
aetherwiing-mn-12b-stars-6598-v1-mkmlizer: uploading /dev/shm/model_cache to s3://guanaco-mkml-models/aetherwiing-mn-12b-stars-6598-v1
aetherwiing-mn-12b-stars-6598-v1-mkmlizer: cp /dev/shm/model_cache/config.json s3://guanaco-mkml-models/aetherwiing-mn-12b-stars-6598-v1/config.json
aetherwiing-mn-12b-stars-6598-v1-mkmlizer: cp /dev/shm/model_cache/special_tokens_map.json s3://guanaco-mkml-models/aetherwiing-mn-12b-stars-6598-v1/special_tokens_map.json
aetherwiing-mn-12b-stars-6598-v1-mkmlizer: cp /dev/shm/model_cache/tokenizer_config.json s3://guanaco-mkml-models/aetherwiing-mn-12b-stars-6598-v1/tokenizer_config.json
aetherwiing-mn-12b-stars-6598-v1-mkmlizer: cp /dev/shm/model_cache/vocab.json s3://guanaco-mkml-models/aetherwiing-mn-12b-stars-6598-v1/vocab.json
aetherwiing-mn-12b-stars-6598-v1-mkmlizer: cp /dev/shm/model_cache/merges.txt s3://guanaco-mkml-models/aetherwiing-mn-12b-stars-6598-v1/merges.txt
aetherwiing-mn-12b-stars-6598-v1-mkmlizer: cp /dev/shm/model_cache/tokenizer.json s3://guanaco-mkml-models/aetherwiing-mn-12b-stars-6598-v1/tokenizer.json
aetherwiing-mn-12b-stars-6598-v1-mkmlizer: cp /dev/shm/model_cache/flywheel_model.0.safetensors s3://guanaco-mkml-models/aetherwiing-mn-12b-stars-6598-v1/flywheel_model.0.safetensors
aetherwiing-mn-12b-stars-6598-v1-mkmlizer: loading reward model from ChaiML/gpt2_xl_pairwise_89m_step_347634
aetherwiing-mn-12b-stars-6598-v1-mkmlizer: Loading 0: 0%| | 0/363 [00:00<?, ?it/s] Loading 0: 1%| | 2/363 [00:06<18:20, 3.05s/it] Loading 0: 2%|▏ | 6/363 [00:06<04:52, 1.22it/s] Loading 0: 3%|▎ | 11/363 [00:06<02:08, 2.75it/s] Loading 0: 4%|▍ | 15/363 [00:06<01:21, 4.29it/s] Loading 0: 6%|▋ | 23/363 [00:06<00:39, 8.53it/s] Loading 0: 8%|▊ | 28/363 [00:06<00:28, 11.56it/s] Loading 0: 9%|▉ | 33/363 [00:06<00:23, 13.94it/s] Loading 0: 11%|█ | 40/363 [00:07<00:19, 16.62it/s] Loading 0: 12%|█▏ | 44/363 [00:07<00:16, 19.15it/s] Loading 0: 13%|█▎ | 49/363 [00:07<00:13, 23.30it/s] Loading 0: 15%|█▍ | 53/363 [00:07<00:12, 25.76it/s] Loading 0: 16%|█▌ | 58/363 [00:07<00:10, 30.09it/s] Loading 0: 17%|█▋ | 63/363 [00:07<00:09, 33.10it/s] Loading 0: 19%|█▊ | 68/363 [00:07<00:08, 36.08it/s] Loading 0: 20%|██ | 74/363 [00:08<00:07, 36.60it/s] Loading 0: 22%|██▏ | 79/363 [00:08<00:07, 36.84it/s] Loading 0: 23%|██▎ | 85/363 [00:08<00:06, 40.92it/s] Loading 0: 25%|██▍ | 90/363 [00:08<00:06, 41.53it/s] Loading 0: 26%|██▌ | 95/363 [00:08<00:06, 42.59it/s] Loading 0: 28%|██▊ | 100/363 [00:08<00:05, 44.38it/s] Loading 0: 29%|██▉ | 105/363 [00:08<00:06, 37.15it/s] Loading 0: 31%|███ | 112/363 [00:08<00:05, 44.49it/s] Loading 0: 32%|███▏ | 117/363 [00:08<00:05, 44.26it/s] Loading 0: 34%|███▎ | 122/363 [00:09<00:08, 29.82it/s] Loading 0: 35%|███▍ | 127/363 [00:09<00:07, 33.70it/s] Loading 0: 36%|███▋ | 132/363 [00:09<00:07, 30.39it/s] Loading 0: 38%|███▊ | 139/363 [00:09<00:05, 37.87it/s] Loading 0: 40%|███▉ | 144/363 [00:09<00:05, 38.34it/s] Loading 0: 41%|████ | 149/363 [00:09<00:05, 39.95it/s] Loading 0: 42%|████▏ | 154/363 [00:10<00:05, 41.74it/s] Loading 0: 44%|████▍ | 159/363 [00:10<00:05, 35.15it/s] Loading 0: 46%|████▌ | 166/363 [00:10<00:04, 41.55it/s] Loading 0: 47%|████▋ | 171/363 [00:10<00:04, 41.70it/s] Loading 0: 48%|████▊ | 176/363 [00:10<00:04, 42.43it/s] Loading 0: 50%|████▉ | 181/363 [00:10<00:04, 43.95it/s] Loading 0: 51%|█████ | 186/363 [00:10<00:04, 36.56it/s] Loading 0: 53%|█████▎ | 193/363 [00:10<00:03, 43.39it/s] Loading 0: 55%|█████▍ | 198/363 [00:11<00:03, 43.34it/s] Loading 0: 56%|█████▌ | 203/363 [00:11<00:05, 29.84it/s] Loading 0: 57%|█████▋ | 208/363 [00:11<00:04, 33.39it/s] Loading 0: 59%|█████▊ | 213/363 [00:11<00:04, 30.92it/s] Loading 0: 61%|██████ | 220/363 [00:11<00:03, 38.28it/s] Loading 0: 62%|██████▏ | 225/363 [00:11<00:03, 39.59it/s] Loading 0: 63%|██████▎ | 230/363 [00:12<00:03, 41.11it/s] Loading 0: 65%|██████▍ | 235/363 [00:12<00:02, 43.05it/s] Loading 0: 66%|██████▌ | 240/363 [00:12<00:03, 35.89it/s] Loading 0: 68%|██████▊ | 247/363 [00:12<00:02, 42.95it/s] Loading 0: 69%|██████▉ | 252/363 [00:12<00:02, 43.33it/s] Loading 0: 71%|███████ | 257/363 [00:12<00:02, 42.62it/s] Loading 0: 72%|███████▏ | 263/363 [00:12<00:02, 41.13it/s] Loading 0: 74%|███████▍ | 268/363 [00:12<00:02, 40.27it/s] Loading 0: 75%|███████▌ | 274/363 [00:13<00:02, 44.13it/s] Loading 0: 77%|███████▋ | 279/363 [00:13<00:01, 44.35it/s] Loading 0: 78%|███████▊ | 284/363 [00:13<00:02, 31.29it/s] Loading 0: 80%|███████▉ | 289/363 [00:13<00:02, 34.70it/s] Loading 0: 81%|████████ | 294/363 [00:13<00:02, 31.53it/s] Loading 0: 83%|████████▎ | 301/363 [00:13<00:01, 38.89it/s] Loading 0: 84%|████████▍ | 306/363 [00:13<00:01, 40.31it/s] Loading 0: 86%|████████▌ | 311/363 [00:14<00:01, 40.71it/s] Loading 0: 87%|████████▋ | 316/363 [00:14<00:01, 42.62it/s] Loading 0: 88%|████████▊ | 321/363 [00:14<00:01, 35.64it/s] Loading 0: 90%|█████████ | 328/363 [00:14<00:00, 42.32it/s] Loading 0: 92%|█████████▏| 333/363 [00:14<00:00, 42.31it/s] Loading 0: 93%|█████████▎| 338/363 [00:14<00:00, 42.23it/s] Loading 0: 95%|█████████▍| 344/363 [00:14<00:00, 41.47it/s] Loading 0: 96%|█████████▌| 349/363 [00:15<00:00, 41.34it/s] Loading 0: 98%|█████████▊| 355/363 [00:15<00:00, 45.68it/s] Loading 0: 99%|█████████▉| 360/363 [00:15<00:00, 45.28it/s] /opt/conda/lib/python3.10/site-packages/transformers/models/auto/configuration_auto.py:957: FutureWarning: The `use_auth_token` argument is deprecated and will be removed in v5 of Transformers. Please use `token` instead.
aetherwiing-mn-12b-stars-6598-v1-mkmlizer: warnings.warn(
aetherwiing-mn-12b-stars-6598-v1-mkmlizer: /opt/conda/lib/python3.10/site-packages/transformers/models/auto/tokenization_auto.py:785: FutureWarning: The `use_auth_token` argument is deprecated and will be removed in v5 of Transformers. Please use `token` instead.
aetherwiing-mn-12b-stars-6598-v1-mkmlizer: warnings.warn(
aetherwiing-mn-12b-stars-6598-v1-mkmlizer: /opt/conda/lib/python3.10/site-packages/transformers/models/auto/auto_factory.py:469: FutureWarning: The `use_auth_token` argument is deprecated and will be removed in v5 of Transformers. Please use `token` instead.
aetherwiing-mn-12b-stars-6598-v1-mkmlizer: warnings.warn(
aetherwiing-mn-12b-stars-6598-v1-mkmlizer: Downloading shards: 0%| | 0/2 [00:00<?, ?it/s] Downloading shards: 50%|█████ | 1/2 [00:05<00:05, 5.61s/it] Downloading shards: 100%|██████████| 2/2 [00:09<00:00, 4.33s/it] Downloading shards: 100%|██████████| 2/2 [00:09<00:00, 4.52s/it]
aetherwiing-mn-12b-stars-6598-v1-mkmlizer: Loading checkpoint shards: 0%| | 0/2 [00:00<?, ?it/s] Loading checkpoint shards: 50%|█████ | 1/2 [00:00<00:00, 2.41it/s] Loading checkpoint shards: 100%|██████████| 2/2 [00:00<00:00, 3.93it/s] Loading checkpoint shards: 100%|██████████| 2/2 [00:00<00:00, 3.59it/s]
aetherwiing-mn-12b-stars-6598-v1-mkmlizer: Saving model to /tmp/reward_cache/reward.tensors
aetherwiing-mn-12b-stars-6598-v1-mkmlizer: Saving duration: 1.315s
aetherwiing-mn-12b-stars-6598-v1-mkmlizer: Processed model ChaiML/gpt2_xl_pairwise_89m_step_347634 in 14.061s
aetherwiing-mn-12b-stars-6598-v1-mkmlizer: creating bucket guanaco-reward-models
aetherwiing-mn-12b-stars-6598-v1-mkmlizer: Bucket 's3://guanaco-reward-models/' created
aetherwiing-mn-12b-stars-6598-v1-mkmlizer: uploading /tmp/reward_cache to s3://guanaco-reward-models/aetherwiing-mn-12b-stars-6598-v1_reward
aetherwiing-mn-12b-stars-6598-v1-mkmlizer: cp /tmp/reward_cache/config.json s3://guanaco-reward-models/aetherwiing-mn-12b-stars-6598-v1_reward/config.json
aetherwiing-mn-12b-stars-6598-v1-mkmlizer: cp /tmp/reward_cache/tokenizer_config.json s3://guanaco-reward-models/aetherwiing-mn-12b-stars-6598-v1_reward/tokenizer_config.json
aetherwiing-mn-12b-stars-6598-v1-mkmlizer: cp /tmp/reward_cache/special_tokens_map.json s3://guanaco-reward-models/aetherwiing-mn-12b-stars-6598-v1_reward/special_tokens_map.json
aetherwiing-mn-12b-stars-6598-v1-mkmlizer: cp /tmp/reward_cache/merges.txt s3://guanaco-reward-models/aetherwiing-mn-12b-stars-6598-v1_reward/merges.txt
aetherwiing-mn-12b-stars-6598-v1-mkmlizer: cp /tmp/reward_cache/vocab.json s3://guanaco-reward-models/aetherwiing-mn-12b-stars-6598-v1_reward/vocab.json
aetherwiing-mn-12b-stars-6598-v1-mkmlizer: cp /tmp/reward_cache/tokenizer.json s3://guanaco-reward-models/aetherwiing-mn-12b-stars-6598-v1_reward/tokenizer.json
aetherwiing-mn-12b-stars-6598-v1-mkmlizer: cp /tmp/reward_cache/reward.tensors s3://guanaco-reward-models/aetherwiing-mn-12b-stars-6598-v1_reward/reward.tensors
Job aetherwiing-mn-12b-stars-6598-v1-mkmlizer completed after 125.16s with status: succeeded
Stopping job with name aetherwiing-mn-12b-stars-6598-v1-mkmlizer
Pipeline stage MKMLizer completed in 126.29s
Running pipeline stage MKMLKubeTemplater
Pipeline stage MKMLKubeTemplater completed in 0.11s
Running pipeline stage ISVCDeployer
Creating inference service aetherwiing-mn-12b-stars-6598-v1
Waiting for inference service aetherwiing-mn-12b-stars-6598-v1 to be ready
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Inference service aetherwiing-mn-12b-stars-6598-v1 ready after 160.95907545089722s
Pipeline stage ISVCDeployer completed in 163.30s
Running pipeline stage StressChecker
Received healthy response to inference request in 2.6102828979492188s
Received healthy response to inference request in 1.5947277545928955s
Received healthy response to inference request in 1.5927700996398926s
Received healthy response to inference request in 1.6080543994903564s
Received healthy response to inference request in 1.2931945323944092s
5 requests
0 failed requests
5th percentile: 1.353109645843506
10th percentile: 1.4130247592926026
20th percentile: 1.5328549861907959
30th percentile: 1.5931616306304932
40th percentile: 1.5939446926116942
50th percentile: 1.5947277545928955
60th percentile: 1.60005841255188
70th percentile: 1.6053890705108642
80th percentile: 1.8085000991821292
90th percentile: 2.2093914985656737
95th percentile: 2.409837198257446
99th percentile: 2.570193758010864
mean time: 1.7398059368133545
Pipeline stage StressChecker completed in 9.42s
aetherwiing-mn-12b-stars_6598_v1 status is now deployed due to DeploymentManager action
aetherwiing-mn-12b-stars_6598_v1 status is now inactive due to auto deactivation removed underperforming models
aetherwiing-mn-12b-stars_6598_v1 status is now deployed due to admin request
aetherwiing-mn-12b-stars_6598_v1 status is now inactive due to auto deactivation removed underperforming models
admin requested tearing down of aetherwiing-mn-12b-stars_6598_v1
Running pipeline stage ISVCDeleter
Checking if service aetherwiing-mn-12b-stars-6598-v1 is running
Tearing down inference service aetherwiing-mn-12b-stars-6598-v1
Service aetherwiing-mn-12b-stars-6598-v1 has been torndown
Pipeline stage ISVCDeleter completed in 5.85s
Running pipeline stage MKMLModelDeleter
Cleaning model data from S3
Cleaning model data from model cache
Deleting key aetherwiing-mn-12b-stars-6598-v1/config.json from bucket guanaco-mkml-models
Deleting key aetherwiing-mn-12b-stars-6598-v1/flywheel_model.0.safetensors from bucket guanaco-mkml-models
Deleting key aetherwiing-mn-12b-stars-6598-v1/merges.txt from bucket guanaco-mkml-models
Deleting key aetherwiing-mn-12b-stars-6598-v1/special_tokens_map.json from bucket guanaco-mkml-models
Deleting key aetherwiing-mn-12b-stars-6598-v1/tokenizer.json from bucket guanaco-mkml-models
Deleting key aetherwiing-mn-12b-stars-6598-v1/tokenizer_config.json from bucket guanaco-mkml-models
Deleting key aetherwiing-mn-12b-stars-6598-v1/vocab.json from bucket guanaco-mkml-models
Cleaning model data from model cache
Deleting key aetherwiing-mn-12b-stars-6598-v1_reward/config.json from bucket guanaco-reward-models
Deleting key aetherwiing-mn-12b-stars-6598-v1_reward/merges.txt from bucket guanaco-reward-models
Deleting key aetherwiing-mn-12b-stars-6598-v1_reward/reward.tensors from bucket guanaco-reward-models
Deleting key aetherwiing-mn-12b-stars-6598-v1_reward/special_tokens_map.json from bucket guanaco-reward-models
Deleting key aetherwiing-mn-12b-stars-6598-v1_reward/tokenizer.json from bucket guanaco-reward-models
Deleting key aetherwiing-mn-12b-stars-6598-v1_reward/tokenizer_config.json from bucket guanaco-reward-models
Deleting key aetherwiing-mn-12b-stars-6598-v1_reward/vocab.json from bucket guanaco-reward-models
Pipeline stage MKMLModelDeleter completed in 6.94s
aetherwiing-mn-12b-stars_6598_v1 status is now torndown due to DeploymentManager action

Usage Metrics

Latency Metrics