anthracite-org-magnum-12b-v2

submission_id: anthracite-org-magnum-12b-v2_v1
developer_uid: Azazelle
best_of: 8
celo_rating: 1233.98
display_name: magnum-12b-v2
family_friendly_score: 0.0
formatter: {'memory_template': '<|im_start|>system\n{memory}<|im_end|>\n', 'prompt_template': '<|im_start|>user\n{prompt}<|im_end|>\n', 'bot_template': '<|im_start|>assistant\n{bot_name}: {message}<|im_end|>\n', 'user_template': '<|im_start|>user\n{user_name}: {message}<|im_end|>\n', 'response_template': '<|im_start|>assistant\n{bot_name}:', 'truncate_by_message': False}
generation_params: {'temperature': 1.05, 'top_p': 0.95, 'min_p': 0.05, 'top_k': 80, 'presence_penalty': 0.0, 'frequency_penalty': 0.0, 'stopping_words': ['\n', '<', '<|'], 'max_input_tokens': 512, 'best_of': 8, 'max_output_tokens': 64, 'reward_max_token_input': 256}
is_internal_developer: False
language_model: anthracite-org/magnum-12b-v2
max_input_tokens: 512
max_output_tokens: 64
model_architecture: MistralForCausalLM
model_group: anthracite-org/magnum-12
model_name: magnum-12b-v2
model_num_parameters: 12772070400.0
model_repo: anthracite-org/magnum-12b-v2
model_size: 13B
num_battles: 15837
num_wins: 8281
ranking_group: single
reward_formatter: {'bot_template': '{bot_name}: {message}\n', 'memory_template': '', 'prompt_template': '', 'response_template': '{bot_name}:', 'truncate_by_message': False, 'user_template': '{user_name}: {message}\n'}
reward_repo: ChaiML/gpt2_xl_pairwise_89m_step_347634
status: torndown
submission_type: basic
timestamp: 2024-08-06T19:44:33+00:00
us_pacific_date: 2024-08-06
win_ratio: 0.5228894361305803
Resubmit model
Running pipeline stage MKMLizer
Starting job with name anthracite-org-magnum-12b-v2-v1-mkmlizer
Waiting for job on anthracite-org-magnum-12b-v2-v1-mkmlizer to finish
Stopping job with name anthracite-org-magnum-12b-v2-v1-mkmlizer
%s, retrying in %s seconds...
Starting job with name anthracite-org-magnum-12b-v2-v1-mkmlizer
Waiting for job on anthracite-org-magnum-12b-v2-v1-mkmlizer to finish
anthracite-org-magnum-12b-v2-v1-mkmlizer: ╔═════════════════════════════════════════════════════════════════════╗
anthracite-org-magnum-12b-v2-v1-mkmlizer: ║     _____            __           __                                ║
anthracite-org-magnum-12b-v2-v1-mkmlizer: ║    / _/ /_ ___    __/ /  ___ ___ / /                                ║
anthracite-org-magnum-12b-v2-v1-mkmlizer: ║   / _/ / // / |/|/ / _ \/ -_) -_) /                                 ║
anthracite-org-magnum-12b-v2-v1-mkmlizer: ║  /_//_/\_, /|__,__/_//_/\__/\__/_/                                  ║
anthracite-org-magnum-12b-v2-v1-mkmlizer: ║       /___/                                                         ║
anthracite-org-magnum-12b-v2-v1-mkmlizer: ║                                                                     ║
anthracite-org-magnum-12b-v2-v1-mkmlizer: ║  Version: 0.9.9                                                     ║
anthracite-org-magnum-12b-v2-v1-mkmlizer: ║  Copyright 2023 MK ONE TECHNOLOGIES Inc.                            ║
anthracite-org-magnum-12b-v2-v1-mkmlizer: ║  https://mk1.ai                                                     ║
anthracite-org-magnum-12b-v2-v1-mkmlizer: ║                                                                     ║
anthracite-org-magnum-12b-v2-v1-mkmlizer: ║  The license key for the current software has been verified as      ║
anthracite-org-magnum-12b-v2-v1-mkmlizer: ║  belonging to:                                                      ║
anthracite-org-magnum-12b-v2-v1-mkmlizer: ║                                                                     ║
anthracite-org-magnum-12b-v2-v1-mkmlizer: ║  Chai Research Corp.                                                ║
anthracite-org-magnum-12b-v2-v1-mkmlizer: ║  Account ID: 7997a29f-0ceb-4cc7-9adf-840c57b4ae6f                   ║
anthracite-org-magnum-12b-v2-v1-mkmlizer: ║  Expiration: 2024-10-15 23:59:59                                    ║
anthracite-org-magnum-12b-v2-v1-mkmlizer: ║                                                                     ║
anthracite-org-magnum-12b-v2-v1-mkmlizer: ╚═════════════════════════════════════════════════════════════════════╝
anthracite-org-magnum-12b-v2-v1-mkmlizer: Downloaded to shared memory in 34.617s
anthracite-org-magnum-12b-v2-v1-mkmlizer: quantizing model to /dev/shm/model_cache, profile:s0, folder:/tmp/tmp563z7k6t, device:0
anthracite-org-magnum-12b-v2-v1-mkmlizer: Saving flywheel model at /dev/shm/model_cache
anthracite-org-magnum-12b-v2-v1-mkmlizer: quantized model in 35.991s
anthracite-org-magnum-12b-v2-v1-mkmlizer: Processed model anthracite-org/magnum-12b-v2 in 70.608s
anthracite-org-magnum-12b-v2-v1-mkmlizer: creating bucket guanaco-mkml-models
anthracite-org-magnum-12b-v2-v1-mkmlizer: Bucket 's3://guanaco-mkml-models/' created
anthracite-org-magnum-12b-v2-v1-mkmlizer: uploading /dev/shm/model_cache to s3://guanaco-mkml-models/anthracite-org-magnum-12b-v2-v1
anthracite-org-magnum-12b-v2-v1-mkmlizer: cp /dev/shm/model_cache/config.json s3://guanaco-mkml-models/anthracite-org-magnum-12b-v2-v1/config.json
anthracite-org-magnum-12b-v2-v1-mkmlizer: cp /dev/shm/model_cache/special_tokens_map.json s3://guanaco-mkml-models/anthracite-org-magnum-12b-v2-v1/special_tokens_map.json
anthracite-org-magnum-12b-v2-v1-mkmlizer: cp /dev/shm/model_cache/tokenizer_config.json s3://guanaco-mkml-models/anthracite-org-magnum-12b-v2-v1/tokenizer_config.json
anthracite-org-magnum-12b-v2-v1-mkmlizer: cp /dev/shm/model_cache/tokenizer.json s3://guanaco-mkml-models/anthracite-org-magnum-12b-v2-v1/tokenizer.json
anthracite-org-magnum-12b-v2-v1-mkmlizer: cp /dev/shm/model_cache/flywheel_model.0.safetensors s3://guanaco-mkml-models/anthracite-org-magnum-12b-v2-v1/flywheel_model.0.safetensors
anthracite-org-magnum-12b-v2-v1-mkmlizer: loading reward model from ChaiML/gpt2_xl_pairwise_89m_step_347634
anthracite-org-magnum-12b-v2-v1-mkmlizer: 
Loading 0:   0%|          | 0/363 [00:00<?, ?it/s]
Loading 0:   1%|▏         | 5/363 [00:00<00:10, 33.31it/s]
Loading 0:   4%|▎         | 13/363 [00:00<00:06, 52.70it/s]
Loading 0:   5%|▌         | 19/363 [00:00<00:07, 46.38it/s]
Loading 0:   7%|▋         | 24/363 [00:00<00:07, 43.45it/s]
Loading 0:   9%|▊         | 31/363 [00:00<00:06, 48.23it/s]
Loading 0:  10%|█         | 37/363 [00:00<00:07, 44.98it/s]
Loading 0:  12%|█▏        | 42/363 [00:00<00:07, 43.97it/s]
Loading 0:  13%|█▎        | 49/363 [00:01<00:06, 48.78it/s]
Loading 0:  15%|█▌        | 55/363 [00:01<00:06, 45.90it/s]
Loading 0:  17%|█▋        | 61/363 [00:01<00:08, 33.83it/s]
Loading 0:  18%|█▊        | 65/363 [00:01<00:08, 33.44it/s]
Loading 0:  20%|█▉        | 72/363 [00:01<00:07, 39.65it/s]
Loading 0:  21%|██        | 77/363 [00:01<00:06, 41.47it/s]
Loading 0:  23%|██▎       | 82/363 [00:02<00:07, 36.18it/s]
Loading 0:  25%|██▍       | 89/363 [00:02<00:06, 43.40it/s]
Loading 0:  26%|██▌       | 94/363 [00:02<00:06, 43.79it/s]
Loading 0:  27%|██▋       | 99/363 [00:02<00:06, 43.83it/s]
Loading 0:  29%|██▊       | 104/363 [00:02<00:05, 45.18it/s]
Loading 0:  30%|███       | 109/363 [00:02<00:05, 46.13it/s]
Loading 0:  31%|███▏      | 114/363 [00:02<00:06, 39.48it/s]
Loading 0:  33%|███▎      | 119/363 [00:02<00:06, 39.24it/s]
Loading 0:  34%|███▍      | 125/363 [00:02<00:05, 43.56it/s]
Loading 0:  36%|███▌      | 130/363 [00:03<00:05, 43.47it/s]
Loading 0:  37%|███▋      | 135/363 [00:03<00:05, 43.29it/s]
Loading 0:  39%|███▊      | 140/363 [00:03<00:04, 45.06it/s]
Loading 0:  40%|███▉      | 145/363 [00:03<00:07, 27.60it/s]
Loading 0:  41%|████      | 149/363 [00:03<00:07, 27.81it/s]
Loading 0:  43%|████▎     | 156/363 [00:03<00:05, 35.26it/s]
Loading 0:  44%|████▍     | 161/363 [00:04<00:05, 37.23it/s]
Loading 0:  46%|████▌     | 166/363 [00:04<00:05, 38.90it/s]
Loading 0:  47%|████▋     | 172/363 [00:04<00:04, 38.27it/s]
Loading 0:  49%|████▉     | 177/363 [00:04<00:04, 37.54it/s]
Loading 0:  50%|█████     | 183/363 [00:04<00:04, 42.15it/s]
Loading 0:  52%|█████▏    | 188/363 [00:04<00:04, 42.07it/s]
Loading 0:  53%|█████▎    | 193/363 [00:04<00:04, 41.93it/s]
Loading 0:  55%|█████▍    | 199/363 [00:04<00:04, 40.95it/s]
Loading 0:  56%|█████▌    | 204/363 [00:05<00:03, 40.47it/s]
Loading 0:  58%|█████▊    | 210/363 [00:05<00:03, 43.71it/s]
Loading 0:  59%|█████▉    | 215/363 [00:05<00:03, 43.37it/s]
Loading 0:  61%|██████    | 221/363 [00:05<00:03, 47.01it/s]
Loading 0:  62%|██████▏   | 226/363 [00:05<00:04, 27.81it/s]
Loading 0:  63%|██████▎   | 230/363 [00:05<00:04, 28.42it/s]
Loading 0:  65%|██████▌   | 237/363 [00:05<00:03, 36.20it/s]
Loading 0:  67%|██████▋   | 242/363 [00:06<00:03, 38.07it/s]
Loading 0:  68%|██████▊   | 247/363 [00:06<00:02, 39.98it/s]
Loading 0:  70%|██████▉   | 253/363 [00:06<00:02, 39.44it/s]
Loading 0:  71%|███████   | 258/363 [00:06<00:02, 39.02it/s]
Loading 0:  73%|███████▎  | 264/363 [00:06<00:02, 43.67it/s]
Loading 0:  74%|███████▍  | 269/363 [00:06<00:02, 43.84it/s]
Loading 0:  75%|███████▌  | 274/363 [00:06<00:02, 43.45it/s]
Loading 0:  77%|███████▋  | 280/363 [00:06<00:01, 41.54it/s]
Loading 0:  79%|███████▊  | 285/363 [00:07<00:01, 40.83it/s]
Loading 0:  80%|████████  | 291/363 [00:07<00:01, 44.97it/s]
Loading 0:  82%|████████▏ | 296/363 [00:07<00:01, 44.57it/s]
Loading 0:  83%|████████▎ | 301/363 [00:07<00:01, 45.98it/s]
Loading 0:  84%|████████▍ | 306/363 [00:14<00:23,  2.45it/s]
Loading 0:  85%|████████▌ | 310/363 [00:14<00:16,  3.19it/s]
Loading 0:  87%|████████▋ | 315/363 [00:14<00:10,  4.47it/s]
Loading 0:  88%|████████▊ | 320/363 [00:14<00:06,  6.19it/s]
Loading 0:  90%|████████▉ | 326/363 [00:14<00:04,  8.73it/s]
Loading 0:  91%|█████████ | 331/363 [00:14<00:02, 11.34it/s]
Loading 0:  93%|█████████▎| 338/363 [00:14<00:01, 16.19it/s]
Loading 0:  95%|█████████▍| 344/363 [00:15<00:00, 19.82it/s]
Loading 0:  96%|█████████▌| 349/363 [00:15<00:00, 23.04it/s]
Loading 0:  98%|█████████▊| 356/363 [00:15<00:00, 29.70it/s]
Loading 0: 100%|█████████▉| 362/363 [00:15<00:00, 32.32it/s]
                                                            
/opt/conda/lib/python3.10/site-packages/transformers/models/auto/configuration_auto.py:957: FutureWarning: The `use_auth_token` argument is deprecated and will be removed in v5 of Transformers. Please use `token` instead.
anthracite-org-magnum-12b-v2-v1-mkmlizer:   warnings.warn(
anthracite-org-magnum-12b-v2-v1-mkmlizer: /opt/conda/lib/python3.10/site-packages/transformers/models/auto/tokenization_auto.py:785: FutureWarning: The `use_auth_token` argument is deprecated and will be removed in v5 of Transformers. Please use `token` instead.
anthracite-org-magnum-12b-v2-v1-mkmlizer:   warnings.warn(
anthracite-org-magnum-12b-v2-v1-mkmlizer: /opt/conda/lib/python3.10/site-packages/transformers/models/auto/auto_factory.py:469: FutureWarning: The `use_auth_token` argument is deprecated and will be removed in v5 of Transformers. Please use `token` instead.
anthracite-org-magnum-12b-v2-v1-mkmlizer:   warnings.warn(
anthracite-org-magnum-12b-v2-v1-mkmlizer: 
Downloading shards:   0%|          | 0/2 [00:00<?, ?it/s]
Downloading shards:  50%|█████     | 1/2 [00:05<00:05,  5.23s/it]
Downloading shards: 100%|██████████| 2/2 [00:08<00:00,  3.83s/it]
Downloading shards: 100%|██████████| 2/2 [00:08<00:00,  4.04s/it]
anthracite-org-magnum-12b-v2-v1-mkmlizer: Saving duration: 1.374s
anthracite-org-magnum-12b-v2-v1-mkmlizer: Processed model ChaiML/gpt2_xl_pairwise_89m_step_347634 in 13.432s
anthracite-org-magnum-12b-v2-v1-mkmlizer: creating bucket guanaco-reward-models
anthracite-org-magnum-12b-v2-v1-mkmlizer: Bucket 's3://guanaco-reward-models/' created
anthracite-org-magnum-12b-v2-v1-mkmlizer: uploading /tmp/reward_cache to s3://guanaco-reward-models/anthracite-org-magnum-12b-v2-v1_reward
anthracite-org-magnum-12b-v2-v1-mkmlizer: cp /tmp/reward_cache/config.json s3://guanaco-reward-models/anthracite-org-magnum-12b-v2-v1_reward/config.json
anthracite-org-magnum-12b-v2-v1-mkmlizer: cp /tmp/reward_cache/special_tokens_map.json s3://guanaco-reward-models/anthracite-org-magnum-12b-v2-v1_reward/special_tokens_map.json
anthracite-org-magnum-12b-v2-v1-mkmlizer: cp /tmp/reward_cache/tokenizer_config.json s3://guanaco-reward-models/anthracite-org-magnum-12b-v2-v1_reward/tokenizer_config.json
anthracite-org-magnum-12b-v2-v1-mkmlizer: cp /tmp/reward_cache/merges.txt s3://guanaco-reward-models/anthracite-org-magnum-12b-v2-v1_reward/merges.txt
anthracite-org-magnum-12b-v2-v1-mkmlizer: cp /tmp/reward_cache/vocab.json s3://guanaco-reward-models/anthracite-org-magnum-12b-v2-v1_reward/vocab.json
anthracite-org-magnum-12b-v2-v1-mkmlizer: cp /tmp/reward_cache/tokenizer.json s3://guanaco-reward-models/anthracite-org-magnum-12b-v2-v1_reward/tokenizer.json
anthracite-org-magnum-12b-v2-v1-mkmlizer: cp /tmp/reward_cache/reward.tensors s3://guanaco-reward-models/anthracite-org-magnum-12b-v2-v1_reward/reward.tensors
Job anthracite-org-magnum-12b-v2-v1-mkmlizer completed after 125.86s with status: succeeded
Stopping job with name anthracite-org-magnum-12b-v2-v1-mkmlizer
Pipeline stage MKMLizer completed in 127.78s
Running pipeline stage MKMLKubeTemplater
Pipeline stage MKMLKubeTemplater completed in 0.10s
Running pipeline stage ISVCDeployer
Creating inference service anthracite-org-magnum-12b-v2-v1
Waiting for inference service anthracite-org-magnum-12b-v2-v1 to be ready
Inference service anthracite-org-magnum-12b-v2-v1 ready after 181.3906798362732s
Pipeline stage ISVCDeployer completed in 183.60s
Running pipeline stage StressChecker
Received healthy response to inference request in 2.618595600128174s
Received healthy response to inference request in 1.593982458114624s
Received healthy response to inference request in 1.5864958763122559s
Received healthy response to inference request in 1.4197258949279785s
Received healthy response to inference request in 1.5708677768707275s
5 requests
0 failed requests
5th percentile: 1.4499542713165283
10th percentile: 1.4801826477050781
20th percentile: 1.5406394004821777
30th percentile: 1.5739933967590332
40th percentile: 1.5802446365356446
50th percentile: 1.5864958763122559
60th percentile: 1.5894905090332032
70th percentile: 1.5924851417541503
80th percentile: 1.7989050865173342
90th percentile: 2.208750343322754
95th percentile: 2.4136729717254637
99th percentile: 2.5776110744476317
mean time: 1.7579335212707519
Pipeline stage StressChecker completed in 9.49s
anthracite-org-magnum-12b-v2_v1 status is now deployed due to DeploymentManager action
anthracite-org-magnum-12b-v2_v1 status is now inactive due to auto deactivation removed underperforming models
anthracite-org-magnum-12b-v2_v1 status is now deployed due to admin request
anthracite-org-magnum-12b-v2_v1 status is now inactive due to auto deactivation removed underperforming models
admin requested tearing down of anthracite-org-magnum-12b-v2_v1
Running pipeline stage ISVCDeleter
Checking if service anthracite-org-magnum-12b-v2-v1 is running
Tearing down inference service anthracite-org-magnum-12b-v2-v1
Service anthracite-org-magnum-12b-v2-v1 has been torndown
Pipeline stage ISVCDeleter completed in 12.51s
Running pipeline stage MKMLModelDeleter
Cleaning model data from S3
Cleaning model data from model cache
Deleting key anthracite-org-magnum-12b-v2-v1/config.json from bucket guanaco-mkml-models
Deleting key anthracite-org-magnum-12b-v2-v1/flywheel_model.0.safetensors from bucket guanaco-mkml-models
Deleting key anthracite-org-magnum-12b-v2-v1/special_tokens_map.json from bucket guanaco-mkml-models
Deleting key anthracite-org-magnum-12b-v2-v1/tokenizer.json from bucket guanaco-mkml-models
Deleting key anthracite-org-magnum-12b-v2-v1/tokenizer_config.json from bucket guanaco-mkml-models
Cleaning model data from model cache
Deleting key anthracite-org-magnum-12b-v2-v1_reward/config.json from bucket guanaco-reward-models
Deleting key anthracite-org-magnum-12b-v2-v1_reward/merges.txt from bucket guanaco-reward-models
Deleting key anthracite-org-magnum-12b-v2-v1_reward/reward.tensors from bucket guanaco-reward-models
Deleting key anthracite-org-magnum-12b-v2-v1_reward/special_tokens_map.json from bucket guanaco-reward-models
Deleting key anthracite-org-magnum-12b-v2-v1_reward/tokenizer.json from bucket guanaco-reward-models
Deleting key anthracite-org-magnum-12b-v2-v1_reward/tokenizer_config.json from bucket guanaco-reward-models
Deleting key anthracite-org-magnum-12b-v2-v1_reward/vocab.json from bucket guanaco-reward-models
Pipeline stage MKMLModelDeleter completed in 7.01s
anthracite-org-magnum-12b-v2_v1 status is now torndown due to DeploymentManager action