sao10k-mn-12b-spicier

developer_uid: sao10k

submission_id: sao10k-mn-12b-spicier_v2

model_name: spice-12

model_group: Sao10K/MN-12B-Spicier

status: torndown

timestamp: 2024-10-13T11:01:53+00:00

num_battles: 7840

num_wins: 4100

celo_rating: 1270.43

family_friendly_score: 0.5962157809983897

family_friendly_standard_error: 0.0055064182724203665

submission_type: basic

model_repo: Sao10K/MN-12B-Spicier

model_architecture: MistralForCausalLM

model_num_parameters: 12772070400.0

best_of: 8

max_input_tokens: 512

max_output_tokens: 64

display_name: spice-12

is_internal_developer: False

language_model: Sao10K/MN-12B-Spicier

model_size: 13B

ranking_group: single

us_pacific_date: 2024-10-13

win_ratio: 0.5229591836734694

generation_params: {'temperature': 0.9, 'top_p': 1.0, 'min_p': 0.1, 'top_k': 40, 'presence_penalty': 0.0, 'frequency_penalty': 0.0, 'stopping_words': ['\n', '\n\n', '\nYou:', '[/INST]', '<|im_end|>', '</s>'], 'max_input_tokens': 512, 'best_of': 8, 'max_output_tokens': 64}

formatter: {'memory_template': '<|im_start|>system\n{memory}<|im_end|>\n', 'prompt_template': '<|im_start|>user\n{prompt}<|im_end|>\n', 'bot_template': '<|im_start|>assistant\n{bot_name}: {message}<|im_end|>\n', 'user_template': '<|im_start|>user\n{user_name}: {message}<|im_end|>\n', 'response_template': '<|im_start|>assistant\n{bot_name}:', 'truncate_by_message': False}

Resubmit model

Shutdown handler not registered because Python interpreter is not running in the main thread
run pipeline %s
run pipeline stage %s
Running pipeline stage MKMLizer
Starting job with name sao10k-mn-12b-spicier-v2-mkmlizer
Waiting for job on sao10k-mn-12b-spicier-v2-mkmlizer to finish
sao10k-mn-12b-spicier-v2-mkmlizer: ╔═════════════════════════════════════════════════════════════════════╗
sao10k-mn-12b-spicier-v2-mkmlizer: ║     _____            __           __                                ║
sao10k-mn-12b-spicier-v2-mkmlizer: ║    / _/ /_ ___    __/ /  ___ ___ / /                                ║
sao10k-mn-12b-spicier-v2-mkmlizer: ║   / _/ / // / |/|/ / _ \/ -_) -_) /                                 ║
sao10k-mn-12b-spicier-v2-mkmlizer: ║  /_//_/\_, /|__,__/_//_/\__/\__/_/                                  ║
sao10k-mn-12b-spicier-v2-mkmlizer: ║       /___/                                                         ║
sao10k-mn-12b-spicier-v2-mkmlizer: ║                                                                     ║
sao10k-mn-12b-spicier-v2-mkmlizer: ║  Version: 0.11.12                                                   ║
sao10k-mn-12b-spicier-v2-mkmlizer: ║  Copyright 2023 MK ONE TECHNOLOGIES Inc.                            ║
sao10k-mn-12b-spicier-v2-mkmlizer: ║  https://mk1.ai                                                     ║
sao10k-mn-12b-spicier-v2-mkmlizer: ║                                                                     ║
sao10k-mn-12b-spicier-v2-mkmlizer: ║  The license key for the current software has been verified as      ║
sao10k-mn-12b-spicier-v2-mkmlizer: ║  belonging to:                                                      ║
sao10k-mn-12b-spicier-v2-mkmlizer: ║                                                                     ║
sao10k-mn-12b-spicier-v2-mkmlizer: ║  Chai Research Corp.                                                ║
sao10k-mn-12b-spicier-v2-mkmlizer: ║  Account ID: 7997a29f-0ceb-4cc7-9adf-840c57b4ae6f                   ║
sao10k-mn-12b-spicier-v2-mkmlizer: ║  Expiration: 2024-10-15 23:59:59                                    ║
sao10k-mn-12b-spicier-v2-mkmlizer: ║                                                                     ║
sao10k-mn-12b-spicier-v2-mkmlizer: ╚═════════════════════════════════════════════════════════════════════╝
sao10k-mn-12b-spicier-v2-mkmlizer: Downloaded to shared memory in 31.974s
sao10k-mn-12b-spicier-v2-mkmlizer: quantizing model to /dev/shm/model_cache, profile:s0, folder:/tmp/tmpxrae4yjh, device:0
sao10k-mn-12b-spicier-v2-mkmlizer: Saving flywheel model at /dev/shm/model_cache
sao10k-mn-12b-spicier-v2-mkmlizer: quantized model in 35.865s
sao10k-mn-12b-spicier-v2-mkmlizer: Processed model Sao10K/MN-12B-Spicier in 67.839s
sao10k-mn-12b-spicier-v2-mkmlizer: creating bucket guanaco-mkml-models
sao10k-mn-12b-spicier-v2-mkmlizer: Bucket 's3://guanaco-mkml-models/' created
sao10k-mn-12b-spicier-v2-mkmlizer: uploading /dev/shm/model_cache to s3://guanaco-mkml-models/sao10k-mn-12b-spicier-v2
sao10k-mn-12b-spicier-v2-mkmlizer: cp /dev/shm/model_cache/special_tokens_map.json s3://guanaco-mkml-models/sao10k-mn-12b-spicier-v2/special_tokens_map.json
sao10k-mn-12b-spicier-v2-mkmlizer: cp /dev/shm/model_cache/config.json s3://guanaco-mkml-models/sao10k-mn-12b-spicier-v2/config.json
sao10k-mn-12b-spicier-v2-mkmlizer: cp /dev/shm/model_cache/tokenizer_config.json s3://guanaco-mkml-models/sao10k-mn-12b-spicier-v2/tokenizer_config.json
sao10k-mn-12b-spicier-v2-mkmlizer: cp /dev/shm/model_cache/tokenizer.json s3://guanaco-mkml-models/sao10k-mn-12b-spicier-v2/tokenizer.json
sao10k-mn-12b-spicier-v2-mkmlizer: 
Loading 0:   0%|          | 0/363 [00:00<?, ?it/s]
Loading 0:   1%|          | 2/363 [00:06<18:07,  3.01s/it]
Loading 0:   2%|▏         | 6/363 [00:06<04:49,  1.23it/s]
Loading 0:   3%|▎         | 11/363 [00:06<02:07,  2.76it/s]
Loading 0:   4%|▍         | 15/363 [00:06<01:20,  4.31it/s]
Loading 0:   6%|▋         | 23/363 [00:06<00:43,  7.74it/s]
Loading 0:   8%|▊         | 29/363 [00:06<00:30, 10.88it/s]
Loading 0:   9%|▉         | 33/363 [00:07<00:25, 13.04it/s]
Loading 0:  11%|█         | 40/363 [00:07<00:16, 19.04it/s]
Loading 0:  12%|█▏        | 45/363 [00:07<00:13, 22.73it/s]
Loading 0:  14%|█▍        | 50/363 [00:07<00:11, 26.70it/s]
Loading 0:  15%|█▌        | 56/363 [00:07<00:10, 29.70it/s]
Loading 0:  17%|█▋        | 61/363 [00:07<00:09, 32.11it/s]
Loading 0:  19%|█▊        | 68/363 [00:07<00:07, 38.49it/s]
Loading 0:  20%|██        | 74/363 [00:07<00:07, 38.75it/s]
Loading 0:  22%|██▏       | 79/363 [00:08<00:07, 38.74it/s]
Loading 0:  24%|██▎       | 86/363 [00:08<00:06, 44.29it/s]
Loading 0:  25%|██▌       | 91/363 [00:08<00:08, 32.08it/s]
Loading 0:  26%|██▌       | 95/363 [00:08<00:08, 33.31it/s]
Loading 0:  28%|██▊       | 101/363 [00:08<00:07, 34.93it/s]
Loading 0:  29%|██▉       | 105/363 [00:08<00:07, 34.25it/s]
Loading 0:  31%|███       | 112/363 [00:08<00:05, 42.29it/s]
Loading 0:  32%|███▏      | 117/363 [00:09<00:05, 42.86it/s]
Loading 0:  34%|███▎      | 122/363 [00:09<00:05, 44.11it/s]
Loading 0:  35%|███▌      | 128/363 [00:09<00:05, 42.04it/s]
Loading 0:  37%|███▋      | 133/363 [00:09<00:05, 41.63it/s]
Loading 0:  38%|███▊      | 139/363 [00:09<00:04, 46.07it/s]
Loading 0:  40%|███▉      | 144/363 [00:09<00:04, 45.23it/s]
Loading 0:  41%|████      | 149/363 [00:09<00:04, 45.18it/s]
Loading 0:  42%|████▏     | 154/363 [00:09<00:06, 33.29it/s]
Loading 0:  44%|████▎     | 158/363 [00:10<00:06, 33.85it/s]
Loading 0:  45%|████▌     | 164/363 [00:10<00:05, 35.83it/s]
Loading 0:  46%|████▋     | 168/363 [00:10<00:05, 33.62it/s]
Loading 0:  48%|████▊     | 175/363 [00:10<00:04, 41.16it/s]
Loading 0:  50%|████▉     | 180/363 [00:10<00:04, 42.52it/s]
Loading 0:  51%|█████     | 185/363 [00:10<00:04, 43.70it/s]
Loading 0:  53%|█████▎    | 191/363 [00:10<00:04, 42.27it/s]
Loading 0:  54%|█████▍    | 196/363 [00:10<00:04, 41.31it/s]
Loading 0:  56%|█████▌    | 202/363 [00:11<00:03, 45.67it/s]
Loading 0:  57%|█████▋    | 207/363 [00:11<00:03, 43.81it/s]
Loading 0:  58%|█████▊    | 212/363 [00:11<00:03, 43.62it/s]
Loading 0:  60%|██████    | 218/363 [00:11<00:03, 41.94it/s]
Loading 0:  61%|██████▏   | 223/363 [00:11<00:04, 31.17it/s]
Loading 0:  63%|██████▎   | 229/363 [00:11<00:03, 36.51it/s]
Loading 0:  64%|██████▍   | 234/363 [00:11<00:03, 38.43it/s]
Loading 0:  66%|██████▌   | 239/363 [00:12<00:03, 40.23it/s]
Loading 0:  67%|██████▋   | 245/363 [00:12<00:02, 39.71it/s]
Loading 0:  69%|██████▉   | 250/363 [00:12<00:02, 39.79it/s]
Loading 0:  71%|███████   | 256/363 [00:12<00:02, 42.58it/s]
Loading 0:  72%|███████▏  | 261/363 [00:12<00:02, 42.82it/s]
Loading 0:  73%|███████▎  | 266/363 [00:12<00:02, 43.35it/s]
Loading 0:  75%|███████▍  | 272/363 [00:12<00:02, 41.59it/s]
Loading 0:  76%|███████▋  | 277/363 [00:12<00:02, 40.91it/s]
Loading 0:  78%|███████▊  | 283/363 [00:13<00:01, 45.12it/s]
Loading 0:  79%|███████▉  | 288/363 [00:13<00:02, 31.28it/s]
Loading 0:  81%|████████  | 293/363 [00:13<00:02, 34.01it/s]
Loading 0:  82%|████████▏ | 298/363 [00:13<00:01, 37.26it/s]
Loading 0:  83%|████████▎ | 303/363 [00:13<00:01, 33.15it/s]
Loading 0:  85%|████████▌ | 310/363 [00:13<00:01, 40.89it/s]
Loading 0:  87%|████████▋ | 315/363 [00:13<00:01, 41.95it/s]
Loading 0:  88%|████████▊ | 320/363 [00:14<00:01, 42.39it/s]
Loading 0:  90%|████████▉ | 326/363 [00:14<00:00, 40.94it/s]
Loading 0:  91%|█████████ | 331/363 [00:14<00:00, 40.22it/s]
Loading 0:  93%|█████████▎| 337/363 [00:14<00:00, 43.88it/s]
Loading 0:  94%|█████████▍| 342/363 [00:14<00:00, 43.90it/s]
Loading 0:  96%|█████████▌| 347/363 [00:14<00:00, 44.25it/s]
Loading 0:  97%|█████████▋| 352/363 [00:14<00:00, 32.41it/s]
Loading 0:  98%|█████████▊| 356/363 [00:15<00:00, 33.70it/s]
Loading 0: 100%|█████████▉| 362/363 [00:15<00:00, 35.45it/s]
                                                            
Job sao10k-mn-12b-spicier-v2-mkmlizer completed after 94.01s with status: succeeded
Stopping job with name sao10k-mn-12b-spicier-v2-mkmlizer
Pipeline stage MKMLizer completed in 94.48s
run pipeline stage %s
Running pipeline stage MKMLTemplater
Pipeline stage MKMLTemplater completed in 0.16s
run pipeline stage %s
Running pipeline stage MKMLDeployer
Creating inference service sao10k-mn-12b-spicier-v2
Waiting for inference service sao10k-mn-12b-spicier-v2 to be ready
Inference service sao10k-mn-12b-spicier-v2 ready after 170.9993600845337s
Pipeline stage MKMLDeployer completed in 171.48s
run pipeline stage %s
Running pipeline stage StressChecker
Received healthy response to inference request in 2.255520820617676s
Received healthy response to inference request in 1.7457339763641357s
Received healthy response to inference request in 1.7638800144195557s
Received healthy response to inference request in 1.8178069591522217s
Received healthy response to inference request in 1.5908539295196533s
5 requests
0 failed requests
5th percentile: 1.6218299388885498
10th percentile: 1.6528059482574462
20th percentile: 1.7147579669952393
30th percentile: 1.7493631839752197
40th percentile: 1.7566215991973877
50th percentile: 1.7638800144195557
60th percentile: 1.7854507923126222
70th percentile: 1.8070215702056884
80th percentile: 1.9053497314453125
90th percentile: 2.080435276031494
95th percentile: 2.167978048324585
99th percentile: 2.238012266159058
mean time: 1.8347591400146483
Pipeline stage StressChecker completed in 10.98s
Shutdown handler de-registered
sao10k-mn-12b-spicier_v2 status is now deployed due to DeploymentManager action
sao10k-mn-12b-spicier_v2 status is now inactive due to auto deactivation removed underperforming models
sao10k-mn-12b-spicier_v2 status is now torndown due to DeploymentManager action