nothingiisreal-mn-12b-s_51657

developer_uid: azuruce

submission_id: nothingiisreal-mn-12b-s_51657_v6

model_name: nothingiisreal-mn-12b-s_51657_v6

model_group: nothingiisreal/MN-12B-St

status: torndown

timestamp: 2025-01-09T23:39:33+00:00

num_battles: 6228

num_wins: 3161

celo_rating: 1243.41

family_friendly_score: 0.5820000000000001

family_friendly_standard_error: 0.006975327949279518

submission_type: basic

model_repo: nothingiisreal/MN-12B-Starcannon-v2

model_architecture: MistralForCausalLM

model_num_parameters: 12772070400.0

best_of: 4

max_input_tokens: 1024

max_output_tokens: 64

reward_model: default

latencies: [{'batch_size': 1, 'throughput': 0.6369600668712821, 'latency_mean': 1.5698666715621947, 'latency_p50': 1.5603418350219727, 'latency_p90': 1.724781084060669}, {'batch_size': 3, 'throughput': 1.2563073349360265, 'latency_mean': 2.3810177600383757, 'latency_p50': 2.385901689529419, 'latency_p90': 2.630889368057251}, {'batch_size': 5, 'throughput': 1.5942341588891238, 'latency_mean': 3.1230314898490907, 'latency_p50': 3.11310613155365, 'latency_p90': 3.5176047325134276}, {'batch_size': 6, 'throughput': 1.7013976611692403, 'latency_mean': 3.5010161137580873, 'latency_p50': 3.5035804510116577, 'latency_p90': 3.9745349407196042}, {'batch_size': 8, 'throughput': 1.8558740342773452, 'latency_mean': 4.2803916585445405, 'latency_p50': 4.254267334938049, 'latency_p90': 4.868307781219483}, {'batch_size': 10, 'throughput': 1.940472663168084, 'latency_mean': 5.109787758588791, 'latency_p50': 5.073079466819763, 'latency_p90': 5.736332583427429}]

gpu_counts: {'NVIDIA RTX A5000': 1}

display_name: nothingiisreal-mn-12b-s_51657_v6

is_internal_developer: False

language_model: nothingiisreal/MN-12B-Starcannon-v2

model_size: 13B

ranking_group: single

throughput_3p7s: 1.76

us_pacific_date: 2025-01-09

win_ratio: 0.5075465639049455

generation_params: {'temperature': 1.0, 'top_p': 1.0, 'min_p': 0.001, 'top_k': 80, 'presence_penalty': 0.0, 'frequency_penalty': 0.0, 'stopping_words': ['\n', 'You:', '</s>', '####', 'User:', '<|eot_id|>', '<|im_end|>', 'Me:', 'Bot:'], 'max_input_tokens': 1024, 'best_of': 4, 'max_output_tokens': 64}

formatter: {'memory_template': '', 'prompt_template': '', 'bot_template': '{bot_name}: {message}\n', 'user_template': '{user_name}: {message}\n', 'response_template': '{bot_name}:', 'truncate_by_message': False}

Resubmit model

Shutdown handler not registered because Python interpreter is not running in the main thread
run pipeline %s
run pipeline stage %s
Running pipeline stage MKMLizer
Starting job with name nothingiisreal-mn-12b-s-51657-v6-mkmlizer
Waiting for job on nothingiisreal-mn-12b-s-51657-v6-mkmlizer to finish
nothingiisreal-mn-12b-s-51657-v6-mkmlizer: ╔═════════════════════════════════════════════════════════════════════╗
nothingiisreal-mn-12b-s-51657-v6-mkmlizer: ║     _____            __           __                                ║
nothingiisreal-mn-12b-s-51657-v6-mkmlizer: ║    / _/ /_ ___    __/ /  ___ ___ / /                                ║
nothingiisreal-mn-12b-s-51657-v6-mkmlizer: ║   / _/ / // / |/|/ / _ \/ -_) -_) /                                 ║
nothingiisreal-mn-12b-s-51657-v6-mkmlizer: ║  /_//_/\_, /|__,__/_//_/\__/\__/_/                                  ║
nothingiisreal-mn-12b-s-51657-v6-mkmlizer: ║       /___/                                                         ║
nothingiisreal-mn-12b-s-51657-v6-mkmlizer: ║                                                                     ║
nothingiisreal-mn-12b-s-51657-v6-mkmlizer: ║  Version: 0.11.12                                                   ║
nothingiisreal-mn-12b-s-51657-v6-mkmlizer: ║  Copyright 2023 MK ONE TECHNOLOGIES Inc.                            ║
nothingiisreal-mn-12b-s-51657-v6-mkmlizer: ║  https://mk1.ai                                                     ║
nothingiisreal-mn-12b-s-51657-v6-mkmlizer: ║                                                                     ║
nothingiisreal-mn-12b-s-51657-v6-mkmlizer: ║  The license key for the current software has been verified as      ║
nothingiisreal-mn-12b-s-51657-v6-mkmlizer: ║  belonging to:                                                      ║
nothingiisreal-mn-12b-s-51657-v6-mkmlizer: ║                                                                     ║
nothingiisreal-mn-12b-s-51657-v6-mkmlizer: ║  Chai Research Corp.                                                ║
nothingiisreal-mn-12b-s-51657-v6-mkmlizer: ║  Account ID: 7997a29f-0ceb-4cc7-9adf-840c57b4ae6f                   ║
nothingiisreal-mn-12b-s-51657-v6-mkmlizer: ║  Expiration: 2025-04-15 23:59:59                                    ║
nothingiisreal-mn-12b-s-51657-v6-mkmlizer: ║                                                                     ║
nothingiisreal-mn-12b-s-51657-v6-mkmlizer: ╚═════════════════════════════════════════════════════════════════════╝
nothingiisreal-mn-12b-s-51657-v6-mkmlizer: Downloaded to shared memory in 55.771s
nothingiisreal-mn-12b-s-51657-v6-mkmlizer: quantizing model to /dev/shm/model_cache, profile:s0, folder:/tmp/tmpt90n333t, device:0
nothingiisreal-mn-12b-s-51657-v6-mkmlizer: Saving flywheel model at /dev/shm/model_cache
nothingiisreal-mn-12b-s-51657-v6-mkmlizer: quantized model in 37.249s
nothingiisreal-mn-12b-s-51657-v6-mkmlizer: Processed model nothingiisreal/MN-12B-Starcannon-v2 in 93.020s
nothingiisreal-mn-12b-s-51657-v6-mkmlizer: creating bucket guanaco-mkml-models
nothingiisreal-mn-12b-s-51657-v6-mkmlizer: Bucket 's3://guanaco-mkml-models/' created
nothingiisreal-mn-12b-s-51657-v6-mkmlizer: uploading /dev/shm/model_cache to s3://guanaco-mkml-models/nothingiisreal-mn-12b-s-51657-v6
nothingiisreal-mn-12b-s-51657-v6-mkmlizer: cp /dev/shm/model_cache/config.json s3://guanaco-mkml-models/nothingiisreal-mn-12b-s-51657-v6/config.json
nothingiisreal-mn-12b-s-51657-v6-mkmlizer: cp /dev/shm/model_cache/special_tokens_map.json s3://guanaco-mkml-models/nothingiisreal-mn-12b-s-51657-v6/special_tokens_map.json
nothingiisreal-mn-12b-s-51657-v6-mkmlizer: cp /dev/shm/model_cache/tokenizer_config.json s3://guanaco-mkml-models/nothingiisreal-mn-12b-s-51657-v6/tokenizer_config.json
nothingiisreal-mn-12b-s-51657-v6-mkmlizer: cp /dev/shm/model_cache/vocab.json s3://guanaco-mkml-models/nothingiisreal-mn-12b-s-51657-v6/vocab.json
nothingiisreal-mn-12b-s-51657-v6-mkmlizer: cp /dev/shm/model_cache/merges.txt s3://guanaco-mkml-models/nothingiisreal-mn-12b-s-51657-v6/merges.txt
nothingiisreal-mn-12b-s-51657-v6-mkmlizer: cp /dev/shm/model_cache/tokenizer.json s3://guanaco-mkml-models/nothingiisreal-mn-12b-s-51657-v6/tokenizer.json
nothingiisreal-mn-12b-s-51657-v6-mkmlizer: cp /dev/shm/model_cache/flywheel_model.0.safetensors s3://guanaco-mkml-models/nothingiisreal-mn-12b-s-51657-v6/flywheel_model.0.safetensors
nothingiisreal-mn-12b-s-51657-v6-mkmlizer: 
Loading 0:   0%|          | 0/363 [00:00<?, ?it/s]
Loading 0:   1%|          | 2/363 [00:06<18:31,  3.08s/it]
Loading 0:   2%|▏         | 6/363 [00:06<04:56,  1.20it/s]
Loading 0:   3%|▎         | 11/363 [00:06<02:10,  2.69it/s]
Loading 0:   4%|▍         | 15/363 [00:06<01:23,  4.17it/s]
Loading 0:   6%|▌         | 22/363 [00:06<00:43,  7.79it/s]
Loading 0:   7%|▋         | 27/363 [00:06<00:31, 10.71it/s]
Loading 0:   9%|▉         | 32/363 [00:06<00:23, 13.80it/s]
Loading 0:  10%|▉         | 36/363 [00:07<00:19, 16.61it/s]
Loading 0:  11%|█         | 40/363 [00:07<00:20, 16.02it/s]
Loading 0:  12%|█▏        | 44/363 [00:07<00:16, 18.90it/s]
Loading 0:  13%|█▎        | 49/363 [00:07<00:13, 23.55it/s]
Loading 0:  15%|█▍        | 53/363 [00:07<00:12, 25.66it/s]
Loading 0:  16%|█▌        | 58/363 [00:07<00:10, 28.79it/s]
Loading 0:  17%|█▋        | 62/363 [00:07<00:10, 28.82it/s]
Loading 0:  18%|█▊        | 67/363 [00:08<00:09, 32.86it/s]
Loading 0:  20%|█▉        | 71/363 [00:08<00:08, 32.91it/s]
Loading 0:  21%|██        | 76/363 [00:08<00:07, 36.50it/s]
Loading 0:  22%|██▏       | 80/363 [00:08<00:08, 35.28it/s]
Loading 0:  23%|██▎       | 85/363 [00:08<00:07, 38.33it/s]
Loading 0:  25%|██▍       | 90/363 [00:08<00:07, 37.74it/s]
Loading 0:  26%|██▌       | 94/363 [00:08<00:07, 38.17it/s]
Loading 0:  27%|██▋       | 98/363 [00:08<00:07, 35.28it/s]
Loading 0:  28%|██▊       | 103/363 [00:09<00:07, 37.05it/s]
Loading 0:  29%|██▉       | 107/363 [00:09<00:07, 35.52it/s]
Loading 0:  31%|███       | 112/363 [00:09<00:06, 38.35it/s]
Loading 0:  32%|███▏      | 116/363 [00:09<00:06, 35.49it/s]
Loading 0:  33%|███▎      | 120/363 [00:09<00:06, 36.59it/s]
Loading 0:  34%|███▍      | 124/363 [00:09<00:10, 22.39it/s]
Loading 0:  36%|███▌      | 130/363 [00:09<00:08, 28.58it/s]
Loading 0:  37%|███▋      | 134/363 [00:10<00:07, 30.09it/s]
Loading 0:  38%|███▊      | 139/363 [00:10<00:06, 33.77it/s]
Loading 0:  39%|███▉      | 143/363 [00:10<00:06, 33.45it/s]
Loading 0:  41%|████      | 148/363 [00:10<00:06, 35.73it/s]
Loading 0:  42%|████▏     | 152/363 [00:10<00:06, 34.95it/s]
Loading 0:  43%|████▎     | 157/363 [00:10<00:05, 37.99it/s]
Loading 0:  44%|████▍     | 161/363 [00:10<00:05, 35.94it/s]
Loading 0:  46%|████▌     | 166/363 [00:10<00:05, 37.12it/s]
Loading 0:  47%|████▋     | 170/363 [00:11<00:05, 35.68it/s]
Loading 0:  48%|████▊     | 175/363 [00:11<00:04, 38.39it/s]
Loading 0:  49%|████▉     | 179/363 [00:11<00:05, 35.99it/s]
Loading 0:  51%|█████     | 184/363 [00:11<00:04, 37.52it/s]
Loading 0:  52%|█████▏    | 188/363 [00:11<00:04, 35.95it/s]
Loading 0:  53%|█████▎    | 193/363 [00:11<00:04, 38.29it/s]
Loading 0:  54%|█████▍    | 197/363 [00:11<00:04, 37.34it/s]
Loading 0:  56%|█████▌    | 202/363 [00:12<00:05, 27.87it/s]
Loading 0:  57%|█████▋    | 206/363 [00:12<00:05, 29.19it/s]
Loading 0:  58%|█████▊    | 211/363 [00:12<00:04, 33.19it/s]
Loading 0:  59%|█████▉    | 215/363 [00:12<00:04, 33.09it/s]
Loading 0:  61%|██████    | 220/363 [00:12<00:03, 36.41it/s]
Loading 0:  62%|██████▏   | 224/363 [00:12<00:03, 35.33it/s]
Loading 0:  63%|██████▎   | 229/363 [00:12<00:03, 37.94it/s]
Loading 0:  64%|██████▍   | 233/363 [00:12<00:03, 36.26it/s]
Loading 0:  66%|██████▌   | 238/363 [00:12<00:03, 39.17it/s]
Loading 0:  67%|██████▋   | 243/363 [00:13<00:03, 39.96it/s]
Loading 0:  68%|██████▊   | 248/363 [00:13<00:02, 40.25it/s]
Loading 0:  70%|██████▉   | 253/363 [00:13<00:02, 41.10it/s]
Loading 0:  71%|███████   | 258/363 [00:13<00:03, 33.64it/s]
Loading 0:  73%|███████▎  | 265/363 [00:13<00:02, 40.30it/s]
Loading 0:  74%|███████▍  | 270/363 [00:13<00:02, 39.83it/s]
Loading 0:  76%|███████▌  | 275/363 [00:13<00:02, 39.87it/s]
Loading 0:  77%|███████▋  | 280/363 [00:14<00:02, 41.32it/s]
Loading 0:  79%|███████▊  | 285/363 [00:14<00:03, 25.76it/s]
Loading 0:  80%|███████▉  | 290/363 [00:14<00:02, 29.15it/s]
Loading 0:  81%|████████  | 294/363 [00:14<00:02, 30.29it/s]
Loading 0:  83%|████████▎ | 301/363 [00:14<00:01, 38.78it/s]
Loading 0:  85%|████████▍ | 307/363 [00:14<00:01, 43.19it/s]
Loading 0:  86%|████████▌ | 312/363 [00:14<00:01, 39.12it/s]
Loading 0:  88%|████████▊ | 320/363 [00:15<00:00, 47.58it/s]
Loading 0:  90%|████████▉ | 326/363 [00:15<00:00, 46.14it/s]
Loading 0:  91%|█████████ | 331/363 [00:15<00:00, 44.10it/s]
Loading 0:  93%|█████████▎| 338/363 [00:15<00:00, 48.60it/s]
Loading 0:  95%|█████████▍| 344/363 [00:15<00:00, 45.99it/s]
Loading 0:  96%|█████████▌| 349/363 [00:15<00:00, 44.43it/s]
Loading 0:  98%|█████████▊| 356/363 [00:15<00:00, 48.47it/s]
Loading 0: 100%|█████████▉| 362/363 [00:16<00:00, 46.03it/s]
                                                            
Job nothingiisreal-mn-12b-s-51657-v6-mkmlizer completed after 115.88s with status: succeeded
Stopping job with name nothingiisreal-mn-12b-s-51657-v6-mkmlizer
Pipeline stage MKMLizer completed in 116.42s
run pipeline stage %s
Running pipeline stage MKMLTemplater
Pipeline stage MKMLTemplater completed in 0.16s
run pipeline stage %s
Running pipeline stage MKMLDeployer
Creating inference service nothingiisreal-mn-12b-s-51657-v6
Waiting for inference service nothingiisreal-mn-12b-s-51657-v6 to be ready
Failed to get response for submission pocketdoc-dans-personal_66216_v1: HTTPConnectionPool(host='pocketdoc-dans-personal-66216-v1-predictor.tenant-chaiml-guanaco.k.chaiverse.com', port=80): Read timed out. (read timeout=12.0)
Failed to get response for submission pocketdoc-dans-creepings_2540_v1: HTTPConnectionPool(host='pocketdoc-dans-creepings-2540-v1-predictor.tenant-chaiml-guanaco.k.chaiverse.com', port=80): Read timed out. (read timeout=12.0)
Failed to get response for submission pocketdoc-dans-creepings_2540_v1: HTTPConnectionPool(host='pocketdoc-dans-creepings-2540-v1-predictor.tenant-chaiml-guanaco.k.chaiverse.com', port=80): Read timed out. (read timeout=12.0)
Failed to get response for submission pocketdoc-dans-creepings_2540_v1: HTTPConnectionPool(host='pocketdoc-dans-creepings-2540-v1-predictor.tenant-chaiml-guanaco.k.chaiverse.com', port=80): Read timed out. (read timeout=12.0)
Failed to get response for submission pocketdoc-dans-creepings_2540_v1: HTTPConnectionPool(host='pocketdoc-dans-creepings-2540-v1-predictor.tenant-chaiml-guanaco.k.chaiverse.com', port=80): Read timed out. (read timeout=12.0)
Retrying (%r) after connection broken by '%r': %s
Failed to get response for submission pocketdoc-dans-creepings_2540_v1: HTTPConnectionPool(host='pocketdoc-dans-creepings-2540-v1-predictor.tenant-chaiml-guanaco.k.chaiverse.com', port=80): Read timed out. (read timeout=12.0)
Inference service nothingiisreal-mn-12b-s-51657-v6 ready after 372.2625563144684s
Pipeline stage MKMLDeployer completed in 372.79s
run pipeline stage %s
Running pipeline stage StressChecker
Received healthy response to inference request in 2.0972330570220947s
Received healthy response to inference request in 1.3618505001068115s
Received healthy response to inference request in 1.071817398071289s
Received healthy response to inference request in 1.4353935718536377s
Received healthy response to inference request in 1.3508472442626953s
5 requests
0 failed requests
5th percentile: 1.1276233673095704
10th percentile: 1.1834293365478517
20th percentile: 1.295041275024414
30th percentile: 1.3530478954315186
40th percentile: 1.357449197769165
50th percentile: 1.3618505001068115
60th percentile: 1.391267728805542
70th percentile: 1.4206849575042724
80th percentile: 1.5677614688873291
90th percentile: 1.832497262954712
95th percentile: 1.9648651599884033
99th percentile: 2.0707594776153564
mean time: 1.4634283542633058
Pipeline stage StressChecker completed in 8.51s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyTriggerPipeline
run_pipeline:run_in_cloud %s
starting trigger_guanaco_pipeline args=%s
triggered trigger_guanaco_pipeline args=%s
Pipeline stage OfflineFamilyFriendlyTriggerPipeline completed in 0.67s
run pipeline stage %s
Running pipeline stage TriggerMKMLProfilingPipeline
run_pipeline:run_in_cloud %s
starting trigger_guanaco_pipeline args=%s
triggered trigger_guanaco_pipeline args=%s
Pipeline stage TriggerMKMLProfilingPipeline completed in 1.03s
Shutdown handler de-registered
nothingiisreal-mn-12b-s_51657_v6 status is now deployed due to DeploymentManager action
Shutdown handler registered
run pipeline %s
run pipeline stage %s
Running pipeline stage MKMLProfilerDeleter
Skipping teardown as no inference service was successfully deployed
Pipeline stage MKMLProfilerDeleter completed in 0.11s
run pipeline stage %s
Running pipeline stage MKMLProfilerTemplater
Pipeline stage MKMLProfilerTemplater completed in 0.10s
run pipeline stage %s
Running pipeline stage MKMLProfilerDeployer
Creating inference service nothingiisreal-mn-12b-s-51657-v6-profiler
Waiting for inference service nothingiisreal-mn-12b-s-51657-v6-profiler to be ready
Shutdown handler registered
run pipeline %s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyScorer
Evaluating %s Family Friendly Score with %s threads
Pipeline stage OfflineFamilyFriendlyScorer completed in 2501.08s
Shutdown handler de-registered
nothingiisreal-mn-12b-s_51657_v6 status is now inactive due to auto deactivation removed underperforming models
nothingiisreal-mn-12b-s_51657_v6 status is now torndown due to DeploymentManager action