anthracite-org-magnum-v4-12b

developer_uid: Luna1040

submission_id: anthracite-org-magnum-v4-12b_v7

model_name: anthracite-org-magnum-v4-12b_v7

model_group: anthracite-org/magnum-v4

status: torndown

timestamp: 2025-02-13T11:51:47+00:00

num_battles: 6402

num_wins: 3181

celo_rating: 1263.96

family_friendly_score: 0.5546

family_friendly_standard_error: 0.007028781402206217

submission_type: basic

model_repo: anthracite-org/magnum-v4-12b

model_architecture: MistralForCausalLM

model_num_parameters: 12772070400.0

best_of: 8

max_input_tokens: 1024

max_output_tokens: 64

display_name: anthracite-org-magnum-v4-12b_v7

is_internal_developer: False

language_model: anthracite-org/magnum-v4-12b

model_size: 13B

ranking_group: single

us_pacific_date: 2025-02-13

win_ratio: 0.4968759762574196

generation_params: {'temperature': 0.72, 'top_p': 0.82, 'min_p': 0.08, 'top_k': 105, 'presence_penalty': 0.1, 'frequency_penalty': 0.12, 'stopping_words': ['\n'], 'max_input_tokens': 1024, 'best_of': 8, 'max_output_tokens': 64}

formatter: {'memory_template': "{bot_name}'s Persona: {memory}\n####\n", 'prompt_template': '{prompt}\n<START>\n', 'bot_template': '{bot_name}: {message}\n', 'user_template': '{user_name}: {message}\n', 'response_template': '{bot_name}:', 'truncate_by_message': False}

Resubmit model

Shutdown handler not registered because Python interpreter is not running in the main thread
run pipeline %s
run pipeline stage %s
Running pipeline stage MKMLizer
Starting job with name anthracite-org-magnum-v4-12b-v7-mkmlizer
Waiting for job on anthracite-org-magnum-v4-12b-v7-mkmlizer to finish
Failed to get response for submission dtnewman-20250212-c-4ep_25732_v1: HTTPConnectionPool(host='dtnewman-20250212-c-4ep-25732-v1-predictor.tenant-chaiml-guanaco.k.chaiverse.com', port=80): Read timed out. (read timeout=12.0)
anthracite-org-magnum-v4-12b-v7-mkmlizer: Downloaded to shared memory in 41.132s
anthracite-org-magnum-v4-12b-v7-mkmlizer: quantizing model to /dev/shm/model_cache, profile:s0, folder:/tmp/tmpgbwnxk8y, device:0
anthracite-org-magnum-v4-12b-v7-mkmlizer: Saving flywheel model at /dev/shm/model_cache
anthracite-org-magnum-v4-12b-v7-mkmlizer: quantized model in 37.448s
anthracite-org-magnum-v4-12b-v7-mkmlizer: Processed model anthracite-org/magnum-v4-12b in 78.581s
anthracite-org-magnum-v4-12b-v7-mkmlizer: creating bucket guanaco-mkml-models
anthracite-org-magnum-v4-12b-v7-mkmlizer: Bucket 's3://guanaco-mkml-models/' created
anthracite-org-magnum-v4-12b-v7-mkmlizer: uploading /dev/shm/model_cache to s3://guanaco-mkml-models/anthracite-org-magnum-v4-12b-v7
Failed to get response for submission dtnewman-20250212-c-4ep_25732_v1: HTTPConnectionPool(host='dtnewman-20250212-c-4ep-25732-v1-predictor.tenant-chaiml-guanaco.k.chaiverse.com', port=80): Read timed out. (read timeout=12.0)
anthracite-org-magnum-v4-12b-v7-mkmlizer: 
Loading 0:   0%|          | 0/363 [00:00<?, ?it/s]
Loading 0:   1%|▏         | 5/363 [00:00<00:10, 32.60it/s]
Loading 0:   3%|▎         | 12/363 [00:00<00:07, 50.00it/s]
Loading 0:   5%|▍         | 18/363 [00:00<00:07, 49.10it/s]
Loading 0:   7%|▋         | 24/363 [00:00<00:08, 39.24it/s]
Loading 0:   8%|▊         | 30/363 [00:00<00:07, 43.76it/s]
Loading 0:  10%|▉         | 35/363 [00:00<00:07, 43.20it/s]
Loading 0:  11%|█         | 40/363 [00:00<00:07, 43.23it/s]
Loading 0:  13%|█▎        | 46/363 [00:01<00:07, 41.16it/s]
Loading 0:  14%|█▍        | 51/363 [00:01<00:07, 40.38it/s]
Loading 0:  15%|█▌        | 56/363 [00:01<00:07, 42.51it/s]
Loading 0:  17%|█▋        | 61/363 [00:01<00:10, 27.80it/s]
Loading 0:  18%|█▊        | 65/363 [00:01<00:10, 28.24it/s]
Loading 0:  20%|█▉        | 71/363 [00:01<00:08, 34.46it/s]
Loading 0:  21%|██        | 76/363 [00:02<00:07, 36.21it/s]
Loading 0:  22%|██▏       | 81/363 [00:02<00:07, 37.71it/s]
Loading 0:  24%|██▎       | 86/363 [00:02<00:06, 39.83it/s]
Loading 0:  25%|██▌       | 91/363 [00:02<00:08, 33.66it/s]
Loading 0:  27%|██▋       | 98/363 [00:02<00:06, 40.30it/s]
Loading 0:  28%|██▊       | 103/363 [00:02<00:06, 40.03it/s]
Loading 0:  30%|██▉       | 108/363 [00:02<00:06, 40.81it/s]
Loading 0:  31%|███       | 113/363 [00:02<00:07, 34.39it/s]
Loading 0:  33%|███▎      | 118/363 [00:03<00:06, 35.15it/s]
Loading 0:  34%|███▍      | 125/363 [00:03<00:05, 41.29it/s]
Loading 0:  36%|███▌      | 130/363 [00:03<00:05, 41.85it/s]
Loading 0:  37%|███▋      | 135/363 [00:03<00:05, 41.78it/s]
Loading 0:  39%|███▊      | 140/363 [00:03<00:05, 42.65it/s]
Loading 0:  40%|███▉      | 145/363 [00:04<00:09, 23.44it/s]
Loading 0:  41%|████      | 149/363 [00:04<00:08, 24.46it/s]
Loading 0:  43%|████▎     | 155/363 [00:04<00:06, 30.80it/s]
Loading 0:  44%|████▍     | 160/363 [00:04<00:06, 31.62it/s]
Loading 0:  45%|████▌     | 165/363 [00:04<00:05, 34.57it/s]
Loading 0:  47%|████▋     | 170/363 [00:04<00:05, 36.27it/s]
Loading 0:  48%|████▊     | 175/363 [00:04<00:05, 36.74it/s]
Loading 0:  49%|████▉     | 179/363 [00:04<00:04, 36.85it/s]
Loading 0:  50%|█████     | 183/363 [00:05<00:04, 37.19it/s]
Loading 0:  52%|█████▏    | 187/363 [00:05<00:04, 36.38it/s]
Loading 0:  53%|█████▎    | 192/363 [00:05<00:04, 37.81it/s]
Loading 0:  54%|█████▍    | 196/363 [00:05<00:04, 36.30it/s]
Loading 0:  55%|█████▌    | 201/363 [00:05<00:04, 38.68it/s]
Loading 0:  56%|█████▋    | 205/363 [00:05<00:05, 28.47it/s]
Loading 0:  58%|█████▊    | 210/363 [00:05<00:04, 31.79it/s]
Loading 0:  59%|█████▉    | 214/363 [00:05<00:04, 31.98it/s]
Loading 0:  61%|██████    | 220/363 [00:06<00:03, 38.17it/s]
Loading 0:  62%|██████▏   | 225/363 [00:06<00:06, 22.89it/s]
Loading 0:  63%|██████▎   | 230/363 [00:06<00:05, 24.90it/s]
Loading 0:  65%|██████▍   | 235/363 [00:06<00:04, 28.56it/s]
Loading 0:  66%|██████▌   | 239/363 [00:06<00:04, 29.39it/s]
Loading 0:  68%|██████▊   | 246/363 [00:06<00:03, 37.96it/s]
Loading 0:  69%|██████▉   | 251/363 [00:07<00:02, 39.24it/s]
Loading 0:  71%|███████   | 256/363 [00:07<00:02, 38.82it/s]
Loading 0:  72%|███████▏  | 261/363 [00:07<00:02, 41.38it/s]
Loading 0:  73%|███████▎  | 266/363 [00:07<00:02, 36.43it/s]
Loading 0:  75%|███████▌  | 273/363 [00:07<00:02, 43.55it/s]
Loading 0:  77%|███████▋  | 278/363 [00:07<00:01, 43.59it/s]
Loading 0:  78%|███████▊  | 283/363 [00:07<00:01, 44.52it/s]
Loading 0:  80%|███████▉  | 289/363 [00:07<00:01, 42.82it/s]
Loading 0:  81%|████████  | 294/363 [00:08<00:01, 42.10it/s]
Loading 0:  82%|████████▏ | 299/363 [00:08<00:01, 43.12it/s]
Loading 0:  84%|████████▎ | 304/363 [00:15<00:24,  2.37it/s]
Loading 0:  85%|████████▍ | 308/363 [00:15<00:17,  3.08it/s]
Loading 0:  86%|████████▌ | 312/363 [00:15<00:12,  4.00it/s]
Loading 0:  87%|████████▋ | 317/363 [00:15<00:08,  5.65it/s]
Loading 0:  88%|████████▊ | 321/363 [00:15<00:05,  7.22it/s]
Loading 0:  90%|█████████ | 328/363 [00:15<00:03, 11.23it/s]
Loading 0:  92%|█████████▏| 333/363 [00:15<00:02, 14.24it/s]
Loading 0:  93%|█████████▎| 338/363 [00:16<00:01, 17.57it/s]
Loading 0:  94%|█████████▍| 343/363 [00:16<00:00, 21.52it/s]
Loading 0:  96%|█████████▌| 348/363 [00:16<00:00, 20.94it/s]
Loading 0:  98%|█████████▊| 355/363 [00:16<00:00, 28.15it/s]
Loading 0:  99%|█████████▉| 360/363 [00:16<00:00, 31.08it/s]
                                                            
Job anthracite-org-magnum-v4-12b-v7-mkmlizer completed after 120.94s with status: succeeded
Stopping job with name anthracite-org-magnum-v4-12b-v7-mkmlizer
Pipeline stage MKMLizer completed in 121.40s
run pipeline stage %s
Running pipeline stage MKMLTemplater
Pipeline stage MKMLTemplater completed in 0.14s
run pipeline stage %s
Running pipeline stage MKMLDeployer
Creating inference service anthracite-org-magnum-v4-12b-v7
Waiting for inference service anthracite-org-magnum-v4-12b-v7 to be ready
Inference service anthracite-org-magnum-v4-12b-v7 ready after 170.59176301956177s
Pipeline stage MKMLDeployer completed in 171.07s
run pipeline stage %s
Running pipeline stage StressChecker
Received healthy response to inference request in 2.070528268814087s
Received healthy response to inference request in 1.688530683517456s
Received healthy response to inference request in 1.4869017601013184s
Received healthy response to inference request in 1.8840367794036865s
Received healthy response to inference request in 1.4634788036346436s
5 requests
0 failed requests
5th percentile: 1.4681633949279784
10th percentile: 1.4728479862213135
20th percentile: 1.4822171688079835
30th percentile: 1.5272275447845458
40th percentile: 1.607879114151001
50th percentile: 1.688530683517456
60th percentile: 1.7667331218719482
70th percentile: 1.8449355602264403
80th percentile: 1.9213350772857667
90th percentile: 1.9959316730499268
95th percentile: 2.0332299709320067
99th percentile: 2.063068609237671
mean time: 1.7186952590942384
Pipeline stage StressChecker completed in 9.93s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyTriggerPipeline
run_pipeline:run_in_cloud %s
starting trigger_guanaco_pipeline args=%s
triggered trigger_guanaco_pipeline args=%s
Pipeline stage OfflineFamilyFriendlyTriggerPipeline completed in 0.78s
run pipeline stage %s
Running pipeline stage TriggerMKMLProfilingPipeline
run_pipeline:run_in_cloud %s
starting trigger_guanaco_pipeline args=%s
triggered trigger_guanaco_pipeline args=%s
Pipeline stage TriggerMKMLProfilingPipeline completed in 0.66s
Shutdown handler de-registered
anthracite-org-magnum-v4-12b_v7 status is now deployed due to DeploymentManager action
Shutdown handler registered
run pipeline %s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyScorer
Evaluating %s Family Friendly Score with %s threads
Pipeline stage OfflineFamilyFriendlyScorer completed in 2645.07s
Shutdown handler de-registered
anthracite-org-magnum-v4-12b_v7 status is now inactive due to auto deactivation removed underperforming models
anthracite-org-magnum-v4-12b_v7 status is now torndown due to DeploymentManager action
anthracite-org-magnum-v4-12b_v7 status is now torndown due to DeploymentManager action
anthracite-org-magnum-v4-12b_v7 status is now torndown due to DeploymentManager action