anthracite-org-magnum-v2-12b

submission_id: anthracite-org-magnum-v2-12b_v2

developer_uid: zonemercy

best_of: 8

celo_rating: 1267.03

display_name: anthracite-org-magnum-v2-12b_v2

family_friendly_score: 0.5539623805118717

family_friendly_standard_error: 0.00613733881164611

formatter: {'memory_template': '', 'prompt_template': '', 'bot_template': '{bot_name}: {message}\n', 'user_template': '{user_name}: {message}\n', 'response_template': '{bot_name}:', 'truncate_by_message': True}

generation_params: {'temperature': 1.0, 'top_p': 1.0, 'min_p': 0.0, 'top_k': 40, 'presence_penalty': 0.0, 'frequency_penalty': 0.0, 'stopping_words': ['\n', '</s>', '####', 'Bot:', 'User:', 'You:', '<|im_end|>', '<|eot_id|>'], 'max_input_tokens': 1024, 'best_of': 8, 'max_output_tokens': 64}

is_internal_developer: True

language_model: anthracite-org/magnum-v2-12b

max_input_tokens: 1024

max_output_tokens: 64

model_architecture: MistralForCausalLM

model_group: anthracite-org/magnum-v2

model_name: anthracite-org-magnum-v2-12b_v2

model_num_parameters: 12772070400.0

model_repo: anthracite-org/magnum-v2-12b

model_size: 13B

num_battles: 6825

num_wins: 3552

ranking_group: single

status: inactive

submission_type: basic

timestamp: 2024-10-21T23:54:15+00:00

us_pacific_date: 2024-10-21

win_ratio: 0.5204395604395604

Download Preference Data

Resubmit model

Shutdown handler not registered because Python interpreter is not running in the main thread
run pipeline %s
run pipeline stage %s
Running pipeline stage MKMLizer
Starting job with name anthracite-org-magnum-v2-12b-v2-mkmlizer
Waiting for job on anthracite-org-magnum-v2-12b-v2-mkmlizer to finish
anthracite-org-magnum-v2-12b-v2-mkmlizer: ╔═════════════════════════════════════════════════════════════════════╗
anthracite-org-magnum-v2-12b-v2-mkmlizer: ║     _____            __           __                                ║
anthracite-org-magnum-v2-12b-v2-mkmlizer: ║    / _/ /_ ___    __/ /  ___ ___ / /                                ║
anthracite-org-magnum-v2-12b-v2-mkmlizer: ║   / _/ / // / |/|/ / _ \/ -_) -_) /                                 ║
anthracite-org-magnum-v2-12b-v2-mkmlizer: ║  /_//_/\_, /|__,__/_//_/\__/\__/_/                                  ║
anthracite-org-magnum-v2-12b-v2-mkmlizer: ║       /___/                                                         ║
anthracite-org-magnum-v2-12b-v2-mkmlizer: ║                                                                     ║
anthracite-org-magnum-v2-12b-v2-mkmlizer: ║  Version: 0.11.12                                                   ║
anthracite-org-magnum-v2-12b-v2-mkmlizer: ║  Copyright 2023 MK ONE TECHNOLOGIES Inc.                            ║
anthracite-org-magnum-v2-12b-v2-mkmlizer: ║  https://mk1.ai                                                     ║
anthracite-org-magnum-v2-12b-v2-mkmlizer: ║                                                                     ║
anthracite-org-magnum-v2-12b-v2-mkmlizer: ║  The license key for the current software has been verified as      ║
anthracite-org-magnum-v2-12b-v2-mkmlizer: ║  belonging to:                                                      ║
anthracite-org-magnum-v2-12b-v2-mkmlizer: ║                                                                     ║
anthracite-org-magnum-v2-12b-v2-mkmlizer: ║  Chai Research Corp.                                                ║
anthracite-org-magnum-v2-12b-v2-mkmlizer: ║  Account ID: 7997a29f-0ceb-4cc7-9adf-840c57b4ae6f                   ║
anthracite-org-magnum-v2-12b-v2-mkmlizer: ║  Expiration: 2025-01-15 23:59:59                                    ║
anthracite-org-magnum-v2-12b-v2-mkmlizer: ║                                                                     ║
anthracite-org-magnum-v2-12b-v2-mkmlizer: ╚═════════════════════════════════════════════════════════════════════╝
anthracite-org-magnum-v2-12b-v2-mkmlizer: Downloaded to shared memory in 36.467s
anthracite-org-magnum-v2-12b-v2-mkmlizer: quantizing model to /dev/shm/model_cache, profile:s0, folder:/tmp/tmp2e_jzopx, device:0
anthracite-org-magnum-v2-12b-v2-mkmlizer: Saving flywheel model at /dev/shm/model_cache
anthracite-org-magnum-v2-12b-v2-mkmlizer: quantized model in 38.902s
anthracite-org-magnum-v2-12b-v2-mkmlizer: Processed model anthracite-org/magnum-v2-12b in 75.369s
anthracite-org-magnum-v2-12b-v2-mkmlizer: Bucket 's3://guanaco-mkml-models/' created
anthracite-org-magnum-v2-12b-v2-mkmlizer: uploading /dev/shm/model_cache to s3://guanaco-mkml-models/anthracite-org-magnum-v2-12b-v2
anthracite-org-magnum-v2-12b-v2-mkmlizer: cp /dev/shm/model_cache/config.json s3://guanaco-mkml-models/anthracite-org-magnum-v2-12b-v2/config.json
anthracite-org-magnum-v2-12b-v2-mkmlizer: cp /dev/shm/model_cache/special_tokens_map.json s3://guanaco-mkml-models/anthracite-org-magnum-v2-12b-v2/special_tokens_map.json
anthracite-org-magnum-v2-12b-v2-mkmlizer: cp /dev/shm/model_cache/tokenizer_config.json s3://guanaco-mkml-models/anthracite-org-magnum-v2-12b-v2/tokenizer_config.json
anthracite-org-magnum-v2-12b-v2-mkmlizer: cp /dev/shm/model_cache/tokenizer.json s3://guanaco-mkml-models/anthracite-org-magnum-v2-12b-v2/tokenizer.json
anthracite-org-magnum-v2-12b-v2-mkmlizer: cp /dev/shm/model_cache/flywheel_model.0.safetensors s3://guanaco-mkml-models/anthracite-org-magnum-v2-12b-v2/flywheel_model.0.safetensors
anthracite-org-magnum-v2-12b-v2-mkmlizer: 
Loading 0:   0%|          | 0/363 [00:00<?, ?it/s]
Loading 0:   1%|▏         | 5/363 [00:00<00:12, 29.54it/s]
Loading 0:   3%|▎         | 12/363 [00:00<00:07, 47.72it/s]
Loading 0:   5%|▍         | 18/363 [00:00<00:07, 46.80it/s]
Loading 0:   6%|▋         | 23/363 [00:00<00:09, 36.10it/s]
Loading 0:   9%|▊         | 31/363 [00:00<00:07, 45.48it/s]
Loading 0:  10%|█         | 37/363 [00:00<00:07, 42.82it/s]
Loading 0:  12%|█▏        | 42/363 [00:01<00:07, 41.10it/s]
Loading 0:  13%|█▎        | 49/363 [00:01<00:06, 46.53it/s]
Loading 0:  15%|█▍        | 54/363 [00:01<00:06, 46.29it/s]
Loading 0:  16%|█▋        | 59/363 [00:01<00:06, 45.28it/s]
Loading 0:  18%|█▊        | 64/363 [00:01<00:11, 25.65it/s]
Loading 0:  20%|█▉        | 71/363 [00:01<00:08, 33.19it/s]
Loading 0:  21%|██        | 76/363 [00:01<00:08, 34.45it/s]
Loading 0:  22%|██▏       | 81/363 [00:02<00:07, 36.01it/s]
Loading 0:  24%|██▎       | 86/363 [00:02<00:07, 37.89it/s]
Loading 0:  25%|██▌       | 91/363 [00:02<00:09, 29.96it/s]
Loading 0:  26%|██▋       | 96/363 [00:02<00:07, 33.63it/s]
Loading 0:  28%|██▊       | 100/363 [00:02<00:08, 31.00it/s]
Loading 0:  29%|██▉       | 105/363 [00:02<00:07, 33.98it/s]
Loading 0:  30%|███       | 110/363 [00:02<00:06, 36.19it/s]
Loading 0:  31%|███▏      | 114/363 [00:03<00:07, 34.36it/s]
Loading 0:  33%|███▎      | 118/363 [00:03<00:07, 31.48it/s]
Loading 0:  34%|███▍      | 125/363 [00:03<00:06, 37.80it/s]
Loading 0:  36%|███▌      | 129/363 [00:03<00:06, 35.44it/s]
Loading 0:  37%|███▋      | 134/363 [00:03<00:06, 37.20it/s]
Loading 0:  38%|███▊      | 138/363 [00:03<00:06, 36.14it/s]
Loading 0:  39%|███▉      | 142/363 [00:04<00:08, 26.10it/s]
Loading 0:  40%|████      | 146/363 [00:04<00:07, 27.89it/s]
Loading 0:  41%|████▏     | 150/363 [00:04<00:07, 27.62it/s]
Loading 0:  43%|████▎     | 156/363 [00:04<00:06, 32.73it/s]
Loading 0:  44%|████▍     | 160/363 [00:04<00:06, 32.84it/s]
Loading 0:  45%|████▌     | 165/363 [00:04<00:05, 36.80it/s]
Loading 0:  47%|████▋     | 169/363 [00:04<00:05, 34.82it/s]
Loading 0:  48%|████▊     | 174/363 [00:04<00:05, 37.52it/s]
Loading 0:  49%|████▉     | 178/363 [00:05<00:05, 35.58it/s]
Loading 0:  50%|█████     | 183/363 [00:05<00:04, 37.12it/s]
Loading 0:  52%|█████▏    | 187/363 [00:05<00:04, 35.85it/s]
Loading 0:  53%|█████▎    | 192/363 [00:05<00:04, 37.31it/s]
Loading 0:  54%|█████▍    | 196/363 [00:05<00:04, 35.93it/s]
Loading 0:  55%|█████▌    | 200/363 [00:05<00:06, 23.67it/s]
Loading 0:  56%|█████▌    | 203/363 [00:05<00:07, 22.71it/s]
Loading 0:  58%|█████▊    | 210/363 [00:06<00:04, 30.66it/s]
Loading 0:  59%|█████▉    | 214/363 [00:06<00:04, 31.08it/s]
Loading 0:  60%|██████    | 218/363 [00:06<00:04, 32.60it/s]
Loading 0:  61%|██████▏   | 223/363 [00:06<00:05, 26.70it/s]
Loading 0:  63%|██████▎   | 227/363 [00:06<00:04, 27.84it/s]
Loading 0:  64%|██████▎   | 231/363 [00:06<00:04, 27.05it/s]
Loading 0:  65%|██████▍   | 235/363 [00:06<00:04, 29.42it/s]
Loading 0:  66%|██████▌   | 239/363 [00:07<00:04, 28.68it/s]
Loading 0:  67%|██████▋   | 244/363 [00:07<00:03, 32.56it/s]
Loading 0:  68%|██████▊   | 248/363 [00:07<00:03, 29.95it/s]
Loading 0:  70%|██████▉   | 253/363 [00:07<00:03, 34.57it/s]
Loading 0:  71%|███████   | 257/363 [00:07<00:03, 31.52it/s]
Loading 0:  72%|███████▏  | 262/363 [00:07<00:02, 35.68it/s]
Loading 0:  73%|███████▎  | 266/363 [00:07<00:02, 33.05it/s]
Loading 0:  75%|███████▌  | 273/363 [00:08<00:02, 39.58it/s]
Loading 0:  77%|███████▋  | 278/363 [00:08<00:02, 39.20it/s]
Loading 0:  78%|███████▊  | 283/363 [00:08<00:02, 38.83it/s]
Loading 0:  79%|███████▉  | 287/363 [00:08<00:02, 37.47it/s]
Loading 0:  80%|████████  | 291/363 [00:08<00:01, 37.28it/s]
Loading 0:  81%|████████▏ | 295/363 [00:08<00:01, 35.90it/s]
Loading 0:  82%|████████▏ | 299/363 [00:08<00:01, 35.18it/s]
Loading 0:  84%|████████▎ | 304/363 [00:15<00:28,  2.04it/s]
Loading 0:  85%|████████▍ | 307/363 [00:15<00:21,  2.55it/s]
Loading 0:  86%|████████▌ | 312/363 [00:16<00:13,  3.75it/s]
Loading 0:  87%|████████▋ | 317/363 [00:16<00:08,  5.39it/s]
Loading 0:  88%|████████▊ | 321/363 [00:16<00:06,  6.90it/s]
Loading 0:  90%|█████████ | 328/363 [00:16<00:03, 10.77it/s]
Loading 0:  91%|█████████▏| 332/363 [00:16<00:02, 12.90it/s]
Loading 0:  93%|█████████▎| 337/363 [00:16<00:01, 16.36it/s]
Loading 0:  94%|█████████▍| 341/363 [00:16<00:01, 18.58it/s]
Loading 0:  95%|█████████▌| 345/363 [00:16<00:00, 21.62it/s]
Loading 0:  96%|█████████▌| 349/363 [00:17<00:00, 22.21it/s]
Loading 0:  97%|█████████▋| 353/363 [00:17<00:00, 24.94it/s]
Loading 0:  98%|█████████▊| 357/363 [00:17<00:00, 24.51it/s]
Loading 0: 100%|█████████▉| 362/363 [00:17<00:00, 29.21it/s]
                                                            
Job anthracite-org-magnum-v2-12b-v2-mkmlizer completed after 121.25s with status: succeeded
Stopping job with name anthracite-org-magnum-v2-12b-v2-mkmlizer
Pipeline stage MKMLizer completed in 121.74s
run pipeline stage %s
Running pipeline stage MKMLTemplater
Pipeline stage MKMLTemplater completed in 0.15s
run pipeline stage %s
Running pipeline stage MKMLDeployer
Creating inference service anthracite-org-magnum-v2-12b-v2
Waiting for inference service anthracite-org-magnum-v2-12b-v2 to be ready
Inference service anthracite-org-magnum-v2-12b-v2 ready after 150.5524868965149s
Pipeline stage MKMLDeployer completed in 151.12s
run pipeline stage %s
Running pipeline stage StressChecker
Received healthy response to inference request in 2.0375256538391113s
Received healthy response to inference request in 1.6165871620178223s
Received healthy response to inference request in 1.795816421508789s
Received healthy response to inference request in 1.4720373153686523s
Received healthy response to inference request in 1.4424333572387695s
5 requests
0 failed requests
5th percentile: 1.448354148864746
10th percentile: 1.4542749404907227
20th percentile: 1.4661165237426759
30th percentile: 1.5009472846984864
40th percentile: 1.5587672233581542
50th percentile: 1.6165871620178223
60th percentile: 1.688278865814209
70th percentile: 1.7599705696105956
80th percentile: 1.8441582679748536
90th percentile: 1.9408419609069825
95th percentile: 1.9891838073730468
99th percentile: 2.0278572845458984
mean time: 1.672879981994629
Pipeline stage StressChecker completed in 9.68s
Shutdown handler de-registered
anthracite-org-magnum-v2-12b_v2 status is now deployed due to DeploymentManager action
anthracite-org-magnum-v2-12b_v2 status is now inactive due to auto deactivation removed underperforming models