anthracite-org-magnum-v2_6820

developer_uid: azuruce

submission_id: anthracite-org-magnum-v2_6820_v4

model_name: anthracite-org-magnum-v2_6820_v4

model_group: anthracite-org/magnum-v2

status: torndown

timestamp: 2024-10-10T18:36:50+00:00

num_battles: 6325

num_wins: 3328

celo_rating: 1261.29

family_friendly_score: 0.5792930958663149

family_friendly_standard_error: 0.006313277271819234

submission_type: basic

model_repo: anthracite-org/magnum-v2.5-12b-kto

model_architecture: MistralForCausalLM

model_num_parameters: 12772070400.0

best_of: 8

max_input_tokens: 1024

max_output_tokens: 64

display_name: anthracite-org-magnum-v2_6820_v4

is_internal_developer: True

language_model: anthracite-org/magnum-v2.5-12b-kto

model_size: 13B

ranking_group: single

us_pacific_date: 2024-10-10

win_ratio: 0.5261660079051383

generation_params: {'temperature': 1.0, 'top_p': 1.0, 'min_p': 0.05, 'top_k': 1000, 'presence_penalty': 0.0, 'frequency_penalty': 0.0, 'stopping_words': ['\n', 'You:'], 'max_input_tokens': 1024, 'best_of': 8, 'max_output_tokens': 64}

formatter: {'memory_template': "{bot_name}'s Persona: {memory}\n####\n", 'prompt_template': '{prompt}\n<START>\n', 'bot_template': '{bot_name}: {message}\n', 'user_template': '{user_name}: {message}\n', 'response_template': '{bot_name}:', 'truncate_by_message': False}

Resubmit model

Shutdown handler not registered because Python interpreter is not running in the main thread
run pipeline %s
run pipeline stage %s
Running pipeline stage MKMLizer
Starting job with name anthracite-org-magnum-v2-6820-v4-mkmlizer
Waiting for job on anthracite-org-magnum-v2-6820-v4-mkmlizer to finish
anthracite-org-magnum-v2-6820-v4-mkmlizer: Downloaded to shared memory in 30.850s
anthracite-org-magnum-v2-6820-v4-mkmlizer: quantizing model to /dev/shm/model_cache, profile:s0, folder:/tmp/tmpa8_u2knd, device:0
anthracite-org-magnum-v2-6820-v4-mkmlizer: Saving flywheel model at /dev/shm/model_cache
anthracite-org-magnum-v2-6820-v4-mkmlizer: quantized model in 36.538s
anthracite-org-magnum-v2-6820-v4-mkmlizer: Processed model anthracite-org/magnum-v2.5-12b-kto in 67.389s
anthracite-org-magnum-v2-6820-v4-mkmlizer: creating bucket guanaco-mkml-models
anthracite-org-magnum-v2-6820-v4-mkmlizer: Bucket 's3://guanaco-mkml-models/' created
anthracite-org-magnum-v2-6820-v4-mkmlizer: uploading /dev/shm/model_cache to s3://guanaco-mkml-models/anthracite-org-magnum-v2-6820-v4
anthracite-org-magnum-v2-6820-v4-mkmlizer: cp /dev/shm/model_cache/config.json s3://guanaco-mkml-models/anthracite-org-magnum-v2-6820-v4/config.json
anthracite-org-magnum-v2-6820-v4-mkmlizer: cp /dev/shm/model_cache/special_tokens_map.json s3://guanaco-mkml-models/anthracite-org-magnum-v2-6820-v4/special_tokens_map.json
anthracite-org-magnum-v2-6820-v4-mkmlizer: cp /dev/shm/model_cache/tokenizer_config.json s3://guanaco-mkml-models/anthracite-org-magnum-v2-6820-v4/tokenizer_config.json
anthracite-org-magnum-v2-6820-v4-mkmlizer: cp /dev/shm/model_cache/tokenizer.json s3://guanaco-mkml-models/anthracite-org-magnum-v2-6820-v4/tokenizer.json
anthracite-org-magnum-v2-6820-v4-mkmlizer: cp /dev/shm/model_cache/flywheel_model.0.safetensors s3://guanaco-mkml-models/anthracite-org-magnum-v2-6820-v4/flywheel_model.0.safetensors
anthracite-org-magnum-v2-6820-v4-mkmlizer: 
Loading 0:   0%|          | 0/363 [00:00<?, ?it/s]
Loading 0:   1%|▏         | 5/363 [00:00<00:13, 26.75it/s]
Loading 0:   3%|▎         | 12/363 [00:00<00:07, 44.72it/s]
Loading 0:   5%|▍         | 18/363 [00:00<00:07, 47.83it/s]
Loading 0:   7%|▋         | 24/363 [00:00<00:07, 42.89it/s]
Loading 0:   9%|▊         | 31/363 [00:00<00:06, 49.60it/s]
Loading 0:  10%|█         | 37/363 [00:00<00:06, 47.58it/s]
Loading 0:  12%|█▏        | 42/363 [00:00<00:07, 44.94it/s]
Loading 0:  13%|█▎        | 49/363 [00:01<00:06, 49.25it/s]
Loading 0:  15%|█▌        | 55/363 [00:01<00:06, 46.18it/s]
Loading 0:  17%|█▋        | 61/363 [00:01<00:09, 32.82it/s]
Loading 0:  18%|█▊        | 65/363 [00:01<00:09, 32.26it/s]
Loading 0:  20%|█▉        | 72/363 [00:01<00:07, 38.91it/s]
Loading 0:  21%|██▏       | 78/363 [00:01<00:07, 39.52it/s]
Loading 0:  23%|██▎       | 83/363 [00:02<00:06, 40.16it/s]
Loading 0:  25%|██▍       | 90/363 [00:02<00:05, 45.76it/s]
Loading 0:  26%|██▋       | 96/363 [00:02<00:06, 43.83it/s]
Loading 0:  28%|██▊       | 101/363 [00:02<00:06, 41.31it/s]
Loading 0:  29%|██▉       | 106/363 [00:02<00:05, 43.17it/s]
Loading 0:  31%|███       | 113/363 [00:02<00:05, 42.29it/s]
Loading 0:  33%|███▎      | 118/363 [00:02<00:06, 40.04it/s]
Loading 0:  34%|███▍      | 125/363 [00:02<00:05, 46.78it/s]
Loading 0:  36%|███▌      | 130/363 [00:03<00:05, 45.67it/s]
Loading 0:  37%|███▋      | 135/363 [00:03<00:04, 46.23it/s]
Loading 0:  39%|███▉      | 141/363 [00:03<00:04, 44.40it/s]
Loading 0:  40%|████      | 146/363 [00:03<00:07, 29.62it/s]
Loading 0:  41%|████▏     | 150/363 [00:03<00:07, 29.99it/s]
Loading 0:  43%|████▎     | 156/363 [00:03<00:05, 35.85it/s]
Loading 0:  44%|████▍     | 161/363 [00:03<00:05, 37.07it/s]
Loading 0:  46%|████▌     | 166/363 [00:04<00:04, 39.43it/s]
Loading 0:  47%|████▋     | 171/363 [00:04<00:04, 40.88it/s]
Loading 0:  48%|████▊     | 176/363 [00:04<00:05, 35.80it/s]
Loading 0:  50%|█████     | 183/363 [00:04<00:04, 43.03it/s]
Loading 0:  52%|█████▏    | 188/363 [00:04<00:03, 43.78it/s]
Loading 0:  53%|█████▎    | 193/363 [00:04<00:03, 45.14it/s]
Loading 0:  55%|█████▍    | 198/363 [00:04<00:03, 46.11it/s]
Loading 0:  56%|█████▌    | 203/363 [00:04<00:04, 37.33it/s]
Loading 0:  58%|█████▊    | 211/363 [00:05<00:03, 45.89it/s]
Loading 0:  60%|█████▉    | 217/363 [00:05<00:03, 42.27it/s]
Loading 0:  61%|██████    | 222/363 [00:05<00:03, 42.29it/s]
Loading 0:  63%|██████▎   | 227/363 [00:05<00:04, 29.75it/s]
Loading 0:  64%|██████▎   | 231/363 [00:05<00:04, 29.27it/s]
Loading 0:  66%|██████▌   | 238/363 [00:05<00:03, 35.91it/s]
Loading 0:  67%|██████▋   | 243/363 [00:06<00:03, 38.86it/s]
Loading 0:  68%|██████▊   | 248/363 [00:06<00:03, 34.95it/s]
Loading 0:  70%|███████   | 255/363 [00:06<00:02, 42.44it/s]
Loading 0:  72%|███████▏  | 260/363 [00:06<00:02, 42.67it/s]
Loading 0:  73%|███████▎  | 265/363 [00:06<00:02, 43.97it/s]
Loading 0:  75%|███████▍  | 271/363 [00:06<00:02, 42.10it/s]
Loading 0:  76%|███████▌  | 276/363 [00:06<00:02, 41.96it/s]
Loading 0:  78%|███████▊  | 283/363 [00:06<00:01, 46.28it/s]
Loading 0:  80%|███████▉  | 289/363 [00:07<00:01, 45.48it/s]
Loading 0:  81%|████████  | 294/363 [00:07<00:01, 43.45it/s]
Loading 0:  83%|████████▎ | 301/363 [00:07<00:01, 49.33it/s]
Loading 0:  85%|████████▍ | 307/363 [00:14<00:20,  2.76it/s]
Loading 0:  86%|████████▌ | 312/363 [00:14<00:13,  3.66it/s]
Loading 0:  88%|████████▊ | 320/363 [00:14<00:07,  5.68it/s]
Loading 0:  90%|████████▉ | 325/363 [00:14<00:05,  7.30it/s]
Loading 0:  91%|█████████ | 330/363 [00:14<00:03,  9.12it/s]
Loading 0:  93%|█████████▎| 338/363 [00:14<00:01, 13.59it/s]
Loading 0:  95%|█████████▍| 344/363 [00:15<00:01, 16.94it/s]
Loading 0:  96%|█████████▌| 349/363 [00:15<00:00, 20.13it/s]
Loading 0:  98%|█████████▊| 356/363 [00:15<00:00, 26.17it/s]
Loading 0: 100%|█████████▉| 362/363 [00:15<00:00, 29.79it/s]
                                                            
Job anthracite-org-magnum-v2-6820-v4-mkmlizer completed after 93.01s with status: succeeded
Stopping job with name anthracite-org-magnum-v2-6820-v4-mkmlizer
Pipeline stage MKMLizer completed in 93.49s
run pipeline stage %s
Running pipeline stage MKMLTemplater
Pipeline stage MKMLTemplater completed in 0.17s
run pipeline stage %s
Running pipeline stage MKMLDeployer
Creating inference service anthracite-org-magnum-v2-6820-v4
Waiting for inference service anthracite-org-magnum-v2-6820-v4 to be ready
Inference service anthracite-org-magnum-v2-6820-v4 ready after 140.58008289337158s
Pipeline stage MKMLDeployer completed in 141.03s
run pipeline stage %s
Running pipeline stage StressChecker
HTTPConnectionPool(host='guanaco-submitter.guanaco-backend.k2.chaiverse.com', port=80): Read timed out. (read timeout=20)
Received unhealthy response to inference request!
Received healthy response to inference request in 2.2229087352752686s
Received healthy response to inference request in 1.9871532917022705s
Received healthy response to inference request in 1.7085468769073486s
Received healthy response to inference request in 1.9237439632415771s
5 requests
1 failed requests
5th percentile: 1.7515862941741944
10th percentile: 1.79462571144104
20th percentile: 1.8807045459747314
30th percentile: 1.9364258289337157
40th percentile: 1.9617895603179931
50th percentile: 1.9871532917022705
60th percentile: 2.0814554691314697
70th percentile: 2.175757646560669
80th percentile: 5.807711076736453
90th percentile: 12.977315759658815
95th percentile: 16.562118101119992
99th percentile: 19.42995997428894
mean time: 5.597854661941528
%s, retrying in %s seconds...
Received healthy response to inference request in 2.1016757488250732s
Received healthy response to inference request in 1.8571100234985352s
Received healthy response to inference request in 1.703141689300537s
Received healthy response to inference request in 1.6693079471588135s
Received healthy response to inference request in 1.6546485424041748s
5 requests
0 failed requests
5th percentile: 1.6575804233551026
10th percentile: 1.6605123043060304
20th percentile: 1.6663760662078857
30th percentile: 1.6760746955871582
40th percentile: 1.6896081924438477
50th percentile: 1.703141689300537
60th percentile: 1.7647290229797363
70th percentile: 1.8263163566589355
80th percentile: 1.906023168563843
90th percentile: 2.003849458694458
95th percentile: 2.0527626037597657
99th percentile: 2.0918931198120116
mean time: 1.7971767902374267
Pipeline stage StressChecker completed in 40.02s
Shutdown handler de-registered
anthracite-org-magnum-v2_6820_v4 status is now deployed due to DeploymentManager action
anthracite-org-magnum-v2_6820_v4 status is now inactive due to auto deactivation removed underperforming models
anthracite-org-magnum-v2_6820_v4 status is now torndown due to DeploymentManager action