anthracite-org-magnum-v2-12b

developer_uid: zonemercy
submission_id: anthracite-org-magnum-v2-12b_v2
model_name: anthracite-org-magnum-v2-12b_v2
model_group: anthracite-org/magnum-v2
status: torndown
timestamp: 2024-10-21T23:54:15+00:00
num_battles: 6825
num_wins: 3552
celo_rating: 1267.03
family_friendly_score: 0.5539623805118717
family_friendly_standard_error: 0.00613733881164611
submission_type: basic
model_repo: anthracite-org/magnum-v2-12b
model_architecture: MistralForCausalLM
model_num_parameters: 12772070400.0
best_of: 8
max_input_tokens: 1024
max_output_tokens: 64
display_name: anthracite-org-magnum-v2-12b_v2
is_internal_developer: True
language_model: anthracite-org/magnum-v2-12b
model_size: 13B
ranking_group: single
us_pacific_date: 2024-10-21
win_ratio: 0.5204395604395604
generation_params: {'temperature': 1.0, 'top_p': 1.0, 'min_p': 0.0, 'top_k': 40, 'presence_penalty': 0.0, 'frequency_penalty': 0.0, 'stopping_words': ['\n', '</s>', '####', 'Bot:', 'User:', 'You:', '<|im_end|>', '<|eot_id|>'], 'max_input_tokens': 1024, 'best_of': 8, 'max_output_tokens': 64}
formatter: {'memory_template': '', 'prompt_template': '', 'bot_template': '{bot_name}: {message}\n', 'user_template': '{user_name}: {message}\n', 'response_template': '{bot_name}:', 'truncate_by_message': True}
Resubmit model
Shutdown handler not registered because Python interpreter is not running in the main thread
run pipeline %s
run pipeline stage %s
Running pipeline stage MKMLizer
Starting job with name anthracite-org-magnum-v2-12b-v2-mkmlizer
Waiting for job on anthracite-org-magnum-v2-12b-v2-mkmlizer to finish
anthracite-org-magnum-v2-12b-v2-mkmlizer: ╔═════════════════════════════════════════════════════════════════════╗
anthracite-org-magnum-v2-12b-v2-mkmlizer: ║     _____            __           __                                ║
anthracite-org-magnum-v2-12b-v2-mkmlizer: ║    / _/ /_ ___    __/ /  ___ ___ / /                                ║
anthracite-org-magnum-v2-12b-v2-mkmlizer: ║   / _/ / // / |/|/ / _ \/ -_) -_) /                                 ║
anthracite-org-magnum-v2-12b-v2-mkmlizer: ║  /_//_/\_, /|__,__/_//_/\__/\__/_/                                  ║
anthracite-org-magnum-v2-12b-v2-mkmlizer: ║       /___/                                                         ║
anthracite-org-magnum-v2-12b-v2-mkmlizer: ║                                                                     ║
anthracite-org-magnum-v2-12b-v2-mkmlizer: ║  Version: 0.11.12                                                   ║
anthracite-org-magnum-v2-12b-v2-mkmlizer: ║  Copyright 2023 MK ONE TECHNOLOGIES Inc.                            ║
anthracite-org-magnum-v2-12b-v2-mkmlizer: ║  https://mk1.ai                                                     ║
anthracite-org-magnum-v2-12b-v2-mkmlizer: ║                                                                     ║
anthracite-org-magnum-v2-12b-v2-mkmlizer: ║  The license key for the current software has been verified as      ║
anthracite-org-magnum-v2-12b-v2-mkmlizer: ║  belonging to:                                                      ║
anthracite-org-magnum-v2-12b-v2-mkmlizer: ║                                                                     ║
anthracite-org-magnum-v2-12b-v2-mkmlizer: ║  Chai Research Corp.                                                ║
anthracite-org-magnum-v2-12b-v2-mkmlizer: ║  Account ID: 7997a29f-0ceb-4cc7-9adf-840c57b4ae6f                   ║
anthracite-org-magnum-v2-12b-v2-mkmlizer: ║  Expiration: 2025-01-15 23:59:59                                    ║
anthracite-org-magnum-v2-12b-v2-mkmlizer: ║                                                                     ║
anthracite-org-magnum-v2-12b-v2-mkmlizer: ╚═════════════════════════════════════════════════════════════════════╝
anthracite-org-magnum-v2-12b-v2-mkmlizer: Downloaded to shared memory in 36.467s
anthracite-org-magnum-v2-12b-v2-mkmlizer: quantizing model to /dev/shm/model_cache, profile:s0, folder:/tmp/tmp2e_jzopx, device:0
anthracite-org-magnum-v2-12b-v2-mkmlizer: Saving flywheel model at /dev/shm/model_cache
anthracite-org-magnum-v2-12b-v2-mkmlizer: quantized model in 38.902s
anthracite-org-magnum-v2-12b-v2-mkmlizer: Processed model anthracite-org/magnum-v2-12b in 75.369s
anthracite-org-magnum-v2-12b-v2-mkmlizer: Bucket 's3://guanaco-mkml-models/' created
anthracite-org-magnum-v2-12b-v2-mkmlizer: uploading /dev/shm/model_cache to s3://guanaco-mkml-models/anthracite-org-magnum-v2-12b-v2
anthracite-org-magnum-v2-12b-v2-mkmlizer: cp /dev/shm/model_cache/config.json s3://guanaco-mkml-models/anthracite-org-magnum-v2-12b-v2/config.json
anthracite-org-magnum-v2-12b-v2-mkmlizer: cp /dev/shm/model_cache/special_tokens_map.json s3://guanaco-mkml-models/anthracite-org-magnum-v2-12b-v2/special_tokens_map.json
anthracite-org-magnum-v2-12b-v2-mkmlizer: cp /dev/shm/model_cache/tokenizer_config.json s3://guanaco-mkml-models/anthracite-org-magnum-v2-12b-v2/tokenizer_config.json
anthracite-org-magnum-v2-12b-v2-mkmlizer: cp /dev/shm/model_cache/tokenizer.json s3://guanaco-mkml-models/anthracite-org-magnum-v2-12b-v2/tokenizer.json
anthracite-org-magnum-v2-12b-v2-mkmlizer: cp /dev/shm/model_cache/flywheel_model.0.safetensors s3://guanaco-mkml-models/anthracite-org-magnum-v2-12b-v2/flywheel_model.0.safetensors
anthracite-org-magnum-v2-12b-v2-mkmlizer: 
Loading 0:   0%|          | 0/363 [00:00<?, ?it/s]
Loading 0:   1%|▏         | 5/363 [00:00<00:12, 29.54it/s]
Loading 0:   3%|▎         | 12/363 [00:00<00:07, 47.72it/s]
Loading 0:   5%|▍         | 18/363 [00:00<00:07, 46.80it/s]
Loading 0:   6%|▋         | 23/363 [00:00<00:09, 36.10it/s]
Loading 0:   9%|▊         | 31/363 [00:00<00:07, 45.48it/s]
Loading 0:  10%|█         | 37/363 [00:00<00:07, 42.82it/s]
Loading 0:  12%|█▏        | 42/363 [00:01<00:07, 41.10it/s]
Loading 0:  13%|█▎        | 49/363 [00:01<00:06, 46.53it/s]
Loading 0:  15%|█▍        | 54/363 [00:01<00:06, 46.29it/s]
Loading 0:  16%|█▋        | 59/363 [00:01<00:06, 45.28it/s]
Loading 0:  18%|█▊        | 64/363 [00:01<00:11, 25.65it/s]
Loading 0:  20%|█▉        | 71/363 [00:01<00:08, 33.19it/s]
Loading 0:  21%|██        | 76/363 [00:01<00:08, 34.45it/s]
Loading 0:  22%|██▏       | 81/363 [00:02<00:07, 36.01it/s]
Loading 0:  24%|██▎       | 86/363 [00:02<00:07, 37.89it/s]
Loading 0:  25%|██▌       | 91/363 [00:02<00:09, 29.96it/s]
Loading 0:  26%|██▋       | 96/363 [00:02<00:07, 33.63it/s]
Loading 0:  28%|██▊       | 100/363 [00:02<00:08, 31.00it/s]
Loading 0:  29%|██▉       | 105/363 [00:02<00:07, 33.98it/s]
Loading 0:  30%|███       | 110/363 [00:02<00:06, 36.19it/s]
Loading 0:  31%|███▏      | 114/363 [00:03<00:07, 34.36it/s]
Loading 0:  33%|███▎      | 118/363 [00:03<00:07, 31.48it/s]
Loading 0:  34%|███▍      | 125/363 [00:03<00:06, 37.80it/s]
Loading 0:  36%|███▌      | 129/363 [00:03<00:06, 35.44it/s]
Loading 0:  37%|███▋      | 134/363 [00:03<00:06, 37.20it/s]
Loading 0:  38%|███▊      | 138/363 [00:03<00:06, 36.14it/s]
Loading 0:  39%|███▉      | 142/363 [00:04<00:08, 26.10it/s]
Loading 0:  40%|████      | 146/363 [00:04<00:07, 27.89it/s]
Loading 0:  41%|████▏     | 150/363 [00:04<00:07, 27.62it/s]
Loading 0:  43%|████▎     | 156/363 [00:04<00:06, 32.73it/s]
Loading 0:  44%|████▍     | 160/363 [00:04<00:06, 32.84it/s]
Loading 0:  45%|████▌     | 165/363 [00:04<00:05, 36.80it/s]
Loading 0:  47%|████▋     | 169/363 [00:04<00:05, 34.82it/s]
Loading 0:  48%|████▊     | 174/363 [00:04<00:05, 37.52it/s]
Loading 0:  49%|████▉     | 178/363 [00:05<00:05, 35.58it/s]
Loading 0:  50%|█████     | 183/363 [00:05<00:04, 37.12it/s]
Loading 0:  52%|█████▏    | 187/363 [00:05<00:04, 35.85it/s]
Loading 0:  53%|█████▎    | 192/363 [00:05<00:04, 37.31it/s]
Loading 0:  54%|█████▍    | 196/363 [00:05<00:04, 35.93it/s]
Loading 0:  55%|█████▌    | 200/363 [00:05<00:06, 23.67it/s]
Loading 0:  56%|█████▌    | 203/363 [00:05<00:07, 22.71it/s]
Loading 0:  58%|█████▊    | 210/363 [00:06<00:04, 30.66it/s]
Loading 0:  59%|█████▉    | 214/363 [00:06<00:04, 31.08it/s]
Loading 0:  60%|██████    | 218/363 [00:06<00:04, 32.60it/s]
Loading 0:  61%|██████▏   | 223/363 [00:06<00:05, 26.70it/s]
Loading 0:  63%|██████▎   | 227/363 [00:06<00:04, 27.84it/s]
Loading 0:  64%|██████▎   | 231/363 [00:06<00:04, 27.05it/s]
Loading 0:  65%|██████▍   | 235/363 [00:06<00:04, 29.42it/s]
Loading 0:  66%|██████▌   | 239/363 [00:07<00:04, 28.68it/s]
Loading 0:  67%|██████▋   | 244/363 [00:07<00:03, 32.56it/s]
Loading 0:  68%|██████▊   | 248/363 [00:07<00:03, 29.95it/s]
Loading 0:  70%|██████▉   | 253/363 [00:07<00:03, 34.57it/s]
Loading 0:  71%|███████   | 257/363 [00:07<00:03, 31.52it/s]
Loading 0:  72%|███████▏  | 262/363 [00:07<00:02, 35.68it/s]
Loading 0:  73%|███████▎  | 266/363 [00:07<00:02, 33.05it/s]
Loading 0:  75%|███████▌  | 273/363 [00:08<00:02, 39.58it/s]
Loading 0:  77%|███████▋  | 278/363 [00:08<00:02, 39.20it/s]
Loading 0:  78%|███████▊  | 283/363 [00:08<00:02, 38.83it/s]
Loading 0:  79%|███████▉  | 287/363 [00:08<00:02, 37.47it/s]
Loading 0:  80%|████████  | 291/363 [00:08<00:01, 37.28it/s]
Loading 0:  81%|████████▏ | 295/363 [00:08<00:01, 35.90it/s]
Loading 0:  82%|████████▏ | 299/363 [00:08<00:01, 35.18it/s]
Loading 0:  84%|████████▎ | 304/363 [00:15<00:28,  2.04it/s]
Loading 0:  85%|████████▍ | 307/363 [00:15<00:21,  2.55it/s]
Loading 0:  86%|████████▌ | 312/363 [00:16<00:13,  3.75it/s]
Loading 0:  87%|████████▋ | 317/363 [00:16<00:08,  5.39it/s]
Loading 0:  88%|████████▊ | 321/363 [00:16<00:06,  6.90it/s]
Loading 0:  90%|█████████ | 328/363 [00:16<00:03, 10.77it/s]
Loading 0:  91%|█████████▏| 332/363 [00:16<00:02, 12.90it/s]
Loading 0:  93%|█████████▎| 337/363 [00:16<00:01, 16.36it/s]
Loading 0:  94%|█████████▍| 341/363 [00:16<00:01, 18.58it/s]
Loading 0:  95%|█████████▌| 345/363 [00:16<00:00, 21.62it/s]
Loading 0:  96%|█████████▌| 349/363 [00:17<00:00, 22.21it/s]
Loading 0:  97%|█████████▋| 353/363 [00:17<00:00, 24.94it/s]
Loading 0:  98%|█████████▊| 357/363 [00:17<00:00, 24.51it/s]
Loading 0: 100%|█████████▉| 362/363 [00:17<00:00, 29.21it/s]
                                                            
Job anthracite-org-magnum-v2-12b-v2-mkmlizer completed after 121.25s with status: succeeded
Stopping job with name anthracite-org-magnum-v2-12b-v2-mkmlizer
Pipeline stage MKMLizer completed in 121.74s
run pipeline stage %s
Running pipeline stage MKMLTemplater
Pipeline stage MKMLTemplater completed in 0.15s
run pipeline stage %s
Running pipeline stage MKMLDeployer
Creating inference service anthracite-org-magnum-v2-12b-v2
Waiting for inference service anthracite-org-magnum-v2-12b-v2 to be ready
Inference service anthracite-org-magnum-v2-12b-v2 ready after 150.5524868965149s
Pipeline stage MKMLDeployer completed in 151.12s
run pipeline stage %s
Running pipeline stage StressChecker
Received healthy response to inference request in 2.0375256538391113s
Received healthy response to inference request in 1.6165871620178223s
Received healthy response to inference request in 1.795816421508789s
Received healthy response to inference request in 1.4720373153686523s
Received healthy response to inference request in 1.4424333572387695s
5 requests
0 failed requests
5th percentile: 1.448354148864746
10th percentile: 1.4542749404907227
20th percentile: 1.4661165237426759
30th percentile: 1.5009472846984864
40th percentile: 1.5587672233581542
50th percentile: 1.6165871620178223
60th percentile: 1.688278865814209
70th percentile: 1.7599705696105956
80th percentile: 1.8441582679748536
90th percentile: 1.9408419609069825
95th percentile: 1.9891838073730468
99th percentile: 2.0278572845458984
mean time: 1.672879981994629
Pipeline stage StressChecker completed in 9.68s
Shutdown handler de-registered
anthracite-org-magnum-v2-12b_v2 status is now deployed due to DeploymentManager action
anthracite-org-magnum-v2-12b_v2 status is now inactive due to auto deactivation removed underperforming models
anthracite-org-magnum-v2-12b_v2 status is now torndown due to DeploymentManager action