inflatebot-mn-12b-mag-mell-r1

developer_uid: huohuo12

submission_id: inflatebot-mn-12b-mag-mell-r1_v4

model_name: inflatebot-mn-12b-mag-mell-r1_v4

model_group: inflatebot/MN-12B-Mag-Me

status: torndown

timestamp: 2025-02-26T09:57:08+00:00

num_battles: 6205

num_wins: 3026

celo_rating: 1258.38

family_friendly_score: 0.5586

family_friendly_standard_error: 0.0070223363633480276

submission_type: basic

model_repo: inflatebot/MN-12B-Mag-Mell-R1

model_architecture: MistralForCausalLM

model_num_parameters: 12772070400.0

best_of: 8

max_input_tokens: 1024

max_output_tokens: 64

latencies: [{'batch_size': 1, 'throughput': 0.6117853606574495, 'latency_mean': 1.6344944858551025, 'latency_p50': 1.6256296634674072, 'latency_p90': 1.8187246561050414}, {'batch_size': 3, 'throughput': 1.1145147286782073, 'latency_mean': 2.6805205178260803, 'latency_p50': 2.6872284412384033, 'latency_p90': 2.960520219802856}, {'batch_size': 5, 'throughput': 1.3459584066549115, 'latency_mean': 3.6896441543102263, 'latency_p50': 3.683987259864807, 'latency_p90': 4.15767228603363}, {'batch_size': 6, 'throughput': 1.415150593609694, 'latency_mean': 4.220040199756622, 'latency_p50': 4.234951734542847, 'latency_p90': 4.681730604171753}, {'batch_size': 8, 'throughput': 1.4710408981018257, 'latency_mean': 5.399426463842392, 'latency_p50': 5.442536354064941, 'latency_p90': 6.062582302093506}, {'batch_size': 10, 'throughput': 1.5117299752173006, 'latency_mean': 6.559950416088104, 'latency_p50': 6.563505411148071, 'latency_p90': 7.352943110466003}]

gpu_counts: {'NVIDIA RTX A5000': 1}

display_name: inflatebot-mn-12b-mag-mell-r1_v4

is_internal_developer: False

language_model: inflatebot/MN-12B-Mag-Mell-R1

model_size: 13B

ranking_group: single

throughput_3p7s: 1.36

us_pacific_date: 2025-02-26

win_ratio: 0.4876712328767123

generation_params: {'temperature': 1.25, 'top_p': 0.94, 'min_p': 0.24, 'top_k': 40, 'presence_penalty': 0.3, 'frequency_penalty': 0.2, 'stopping_words': ['\n'], 'max_input_tokens': 1024, 'best_of': 8, 'max_output_tokens': 64}

formatter: {'memory_template': "{bot_name}'s Persona: {memory}\n####\n", 'prompt_template': '{prompt}\n<START>\n', 'bot_template': '{bot_name}: {message}\n', 'user_template': '{user_name}: {message}\n', 'response_template': '{bot_name}:', 'truncate_by_message': False}

Resubmit model

Shutdown handler not registered because Python interpreter is not running in the main thread
run pipeline %s
run pipeline stage %s
Running pipeline stage MKMLizer
Starting job with name inflatebot-mn-12b-mag-mell-r1-v4-mkmlizer
Waiting for job on inflatebot-mn-12b-mag-mell-r1-v4-mkmlizer to finish
inflatebot-mn-12b-mag-mell-r1-v4-mkmlizer: ╔═════════════════════════════════════════════════════════════════════╗
inflatebot-mn-12b-mag-mell-r1-v4-mkmlizer: ║     _____            __           __                                ║
inflatebot-mn-12b-mag-mell-r1-v4-mkmlizer: ║    / _/ /_ ___    __/ /  ___ ___ / /                                ║
inflatebot-mn-12b-mag-mell-r1-v4-mkmlizer: ║   / _/ / // / |/|/ / _ \/ -_) -_) /                                 ║
inflatebot-mn-12b-mag-mell-r1-v4-mkmlizer: ║  /_//_/\_, /|__,__/_//_/\__/\__/_/                                  ║
inflatebot-mn-12b-mag-mell-r1-v4-mkmlizer: ║       /___/                                                         ║
inflatebot-mn-12b-mag-mell-r1-v4-mkmlizer: ║                                                                     ║
inflatebot-mn-12b-mag-mell-r1-v4-mkmlizer: ║  Version: 0.12.8                                                    ║
inflatebot-mn-12b-mag-mell-r1-v4-mkmlizer: ║  Copyright 2023 MK ONE TECHNOLOGIES Inc.                            ║
inflatebot-mn-12b-mag-mell-r1-v4-mkmlizer: ║  https://mk1.ai                                                     ║
inflatebot-mn-12b-mag-mell-r1-v4-mkmlizer: ║                                                                     ║
inflatebot-mn-12b-mag-mell-r1-v4-mkmlizer: ║  The license key for the current software has been verified as      ║
inflatebot-mn-12b-mag-mell-r1-v4-mkmlizer: ║  belonging to:                                                      ║
inflatebot-mn-12b-mag-mell-r1-v4-mkmlizer: ║                                                                     ║
inflatebot-mn-12b-mag-mell-r1-v4-mkmlizer: ║  Chai Research Corp.                                                ║
inflatebot-mn-12b-mag-mell-r1-v4-mkmlizer: ║  Account ID: 7997a29f-0ceb-4cc7-9adf-840c57b4ae6f                   ║
inflatebot-mn-12b-mag-mell-r1-v4-mkmlizer: ║  Expiration: 2025-04-15 23:59:59                                    ║
inflatebot-mn-12b-mag-mell-r1-v4-mkmlizer: ║                                                                     ║
inflatebot-mn-12b-mag-mell-r1-v4-mkmlizer: ╚═════════════════════════════════════════════════════════════════════╝
inflatebot-mn-12b-mag-mell-r1-v4-mkmlizer: Downloaded to shared memory in 30.465s
inflatebot-mn-12b-mag-mell-r1-v4-mkmlizer: quantizing model to /dev/shm/model_cache, profile:s0, folder:/tmp/tmpt4146pyn, device:0
inflatebot-mn-12b-mag-mell-r1-v4-mkmlizer: Saving flywheel model at /dev/shm/model_cache
inflatebot-mn-12b-mag-mell-r1-v4-mkmlizer: quantized model in 35.640s
inflatebot-mn-12b-mag-mell-r1-v4-mkmlizer: Processed model inflatebot/MN-12B-Mag-Mell-R1 in 66.125s
inflatebot-mn-12b-mag-mell-r1-v4-mkmlizer: creating bucket guanaco-mkml-models
inflatebot-mn-12b-mag-mell-r1-v4-mkmlizer: Bucket 's3://guanaco-mkml-models/' created
inflatebot-mn-12b-mag-mell-r1-v4-mkmlizer: uploading /dev/shm/model_cache to s3://guanaco-mkml-models/inflatebot-mn-12b-mag-mell-r1-v4
inflatebot-mn-12b-mag-mell-r1-v4-mkmlizer: cp /dev/shm/model_cache/config.json s3://guanaco-mkml-models/inflatebot-mn-12b-mag-mell-r1-v4/config.json
inflatebot-mn-12b-mag-mell-r1-v4-mkmlizer: cp /dev/shm/model_cache/special_tokens_map.json s3://guanaco-mkml-models/inflatebot-mn-12b-mag-mell-r1-v4/special_tokens_map.json
inflatebot-mn-12b-mag-mell-r1-v4-mkmlizer: cp /dev/shm/model_cache/tokenizer_config.json s3://guanaco-mkml-models/inflatebot-mn-12b-mag-mell-r1-v4/tokenizer_config.json
inflatebot-mn-12b-mag-mell-r1-v4-mkmlizer: cp /dev/shm/model_cache/tokenizer.json s3://guanaco-mkml-models/inflatebot-mn-12b-mag-mell-r1-v4/tokenizer.json
inflatebot-mn-12b-mag-mell-r1-v4-mkmlizer: cp /dev/shm/model_cache/flywheel_model.0.safetensors s3://guanaco-mkml-models/inflatebot-mn-12b-mag-mell-r1-v4/flywheel_model.0.safetensors
inflatebot-mn-12b-mag-mell-r1-v4-mkmlizer: 
Loading 0:   0%|          | 0/363 [00:00<?, ?it/s]
Loading 0:   1%|          | 2/363 [00:06<18:09,  3.02s/it]
Loading 0:   2%|▏         | 6/363 [00:06<04:50,  1.23it/s]
Loading 0:   3%|▎         | 12/363 [00:06<01:53,  3.08it/s]
Loading 0:   4%|▍         | 16/363 [00:06<01:14,  4.65it/s]
Loading 0:   6%|▋         | 23/363 [00:06<00:40,  8.38it/s]
Loading 0:   8%|▊         | 29/363 [00:06<00:28, 11.80it/s]
Loading 0:   9%|▉         | 34/363 [00:06<00:21, 15.09it/s]
Loading 0:  11%|█         | 40/363 [00:07<00:18, 17.42it/s]
Loading 0:  12%|█▏        | 44/363 [00:07<00:16, 19.66it/s]
Loading 0:  14%|█▍        | 50/363 [00:07<00:12, 25.16it/s]
Loading 0:  15%|█▌        | 56/363 [00:07<00:10, 28.65it/s]
Loading 0:  17%|█▋        | 61/363 [00:07<00:09, 30.78it/s]
Loading 0:  19%|█▊        | 68/363 [00:07<00:07, 37.63it/s]
Loading 0:  20%|██        | 74/363 [00:07<00:07, 39.41it/s]
Loading 0:  22%|██▏       | 79/363 [00:07<00:07, 40.18it/s]
Loading 0:  24%|██▎       | 86/363 [00:08<00:06, 45.49it/s]
Loading 0:  25%|██▌       | 92/363 [00:08<00:06, 44.97it/s]
Loading 0:  27%|██▋       | 97/363 [00:08<00:05, 45.13it/s]
Loading 0:  29%|██▊       | 104/363 [00:08<00:05, 50.24it/s]
Loading 0:  30%|███       | 110/363 [00:08<00:05, 47.78it/s]
Loading 0:  32%|███▏      | 115/363 [00:08<00:05, 45.08it/s]
Loading 0:  33%|███▎      | 121/363 [00:08<00:06, 37.01it/s]
Loading 0:  35%|███▍      | 126/363 [00:09<00:06, 37.77it/s]
Loading 0:  36%|███▌      | 131/363 [00:09<00:05, 39.43it/s]
Loading 0:  37%|███▋      | 136/363 [00:09<00:05, 41.69it/s]
Loading 0:  39%|███▉      | 141/363 [00:09<00:06, 35.95it/s]
Loading 0:  41%|████      | 148/363 [00:09<00:04, 43.50it/s]
Loading 0:  42%|████▏     | 153/363 [00:09<00:04, 44.68it/s]
Loading 0:  44%|████▍     | 159/363 [00:09<00:04, 41.49it/s]
Loading 0:  46%|████▌     | 167/363 [00:09<00:04, 48.74it/s]
Loading 0:  48%|████▊     | 173/363 [00:10<00:04, 46.43it/s]
Loading 0:  49%|████▉     | 178/363 [00:10<00:04, 45.73it/s]
Loading 0:  51%|█████     | 185/363 [00:10<00:03, 49.95it/s]
Loading 0:  53%|█████▎    | 191/363 [00:10<00:03, 47.86it/s]
Loading 0:  54%|█████▍    | 196/363 [00:10<00:03, 45.46it/s]
Loading 0:  56%|█████▌    | 202/363 [00:10<00:04, 37.38it/s]
Loading 0:  57%|█████▋    | 207/363 [00:10<00:04, 37.72it/s]
Loading 0:  58%|█████▊    | 212/363 [00:11<00:03, 38.55it/s]
Loading 0:  60%|█████▉    | 217/363 [00:11<00:03, 40.59it/s]
Loading 0:  61%|██████    | 222/363 [00:11<00:03, 35.75it/s]
Loading 0:  63%|██████▎   | 230/363 [00:11<00:03, 43.73it/s]
Loading 0:  65%|██████▍   | 235/363 [00:11<00:02, 44.84it/s]
Loading 0:  66%|██████▌   | 240/363 [00:11<00:03, 38.72it/s]
Loading 0:  68%|██████▊   | 248/363 [00:11<00:02, 46.65it/s]
Loading 0:  70%|██████▉   | 254/363 [00:11<00:02, 45.01it/s]
Loading 0:  71%|███████▏  | 259/363 [00:12<00:02, 43.58it/s]
Loading 0:  73%|███████▎  | 266/363 [00:12<00:02, 47.59it/s]
Loading 0:  75%|███████▍  | 272/363 [00:12<00:01, 45.66it/s]
Loading 0:  76%|███████▋  | 277/363 [00:12<00:01, 44.07it/s]
Loading 0:  78%|███████▊  | 283/363 [00:12<00:02, 36.11it/s]
Loading 0:  79%|███████▉  | 287/363 [00:12<00:02, 36.20it/s]
Loading 0:  80%|████████  | 292/363 [00:12<00:01, 39.20it/s]
Loading 0:  82%|████████▏ | 297/363 [00:13<00:01, 41.48it/s]
Loading 0:  83%|████████▎ | 302/363 [00:13<00:01, 43.34it/s]
Loading 0:  85%|████████▍ | 308/363 [00:13<00:01, 42.07it/s]
Loading 0:  86%|████████▌ | 313/363 [00:13<00:01, 41.19it/s]
Loading 0:  88%|████████▊ | 320/363 [00:13<00:00, 46.40it/s]
Loading 0:  90%|████████▉ | 326/363 [00:13<00:00, 44.29it/s]
Loading 0:  91%|█████████ | 331/363 [00:13<00:00, 43.84it/s]
Loading 0:  93%|█████████▎| 337/363 [00:13<00:00, 46.87it/s]
Loading 0:  94%|█████████▍| 342/363 [00:14<00:00, 45.15it/s]
Loading 0:  96%|█████████▌| 347/363 [00:14<00:00, 45.54it/s]
Loading 0:  97%|█████████▋| 353/363 [00:14<00:00, 43.97it/s]
Loading 0:  99%|█████████▊| 358/363 [00:14<00:00, 43.09it/s]
                                                            
Job inflatebot-mn-12b-mag-mell-r1-v4-mkmlizer completed after 94.05s with status: succeeded
Stopping job with name inflatebot-mn-12b-mag-mell-r1-v4-mkmlizer
Pipeline stage MKMLizer completed in 94.48s
run pipeline stage %s
Running pipeline stage MKMLTemplater
Pipeline stage MKMLTemplater completed in 0.14s
run pipeline stage %s
Running pipeline stage MKMLDeployer
Creating inference service inflatebot-mn-12b-mag-mell-r1-v4
Waiting for inference service inflatebot-mn-12b-mag-mell-r1-v4 to be ready
Failed to get response for submission nischaydnk-exp16-mistra_31861_v2: HTTPConnectionPool(host='nischaydnk-exp16-mistra-31861-v2-predictor.tenant-chaiml-guanaco.k.chaiverse.com', port=80): Read timed out. (read timeout=12.0)
Inference service inflatebot-mn-12b-mag-mell-r1-v4 ready after 270.95089626312256s
Pipeline stage MKMLDeployer completed in 271.44s
run pipeline stage %s
Running pipeline stage StressChecker
Received healthy response to inference request in 2.170407772064209s
Received healthy response to inference request in 1.550656795501709s
Received healthy response to inference request in 1.7078030109405518s
Received healthy response to inference request in 1.5005707740783691s
Received healthy response to inference request in 1.6775572299957275s
5 requests
0 failed requests
5th percentile: 1.5105879783630372
10th percentile: 1.5206051826477052
20th percentile: 1.540639591217041
30th percentile: 1.5760368824005127
40th percentile: 1.6267970561981202
50th percentile: 1.6775572299957275
60th percentile: 1.6896555423736572
70th percentile: 1.701753854751587
80th percentile: 1.8003239631652832
90th percentile: 1.985365867614746
95th percentile: 2.0778868198394775
99th percentile: 2.151903581619263
mean time: 1.7213991165161133
Pipeline stage StressChecker completed in 9.67s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyTriggerPipeline
run_pipeline:run_in_cloud %s
starting trigger_guanaco_pipeline args=%s
triggered trigger_guanaco_pipeline args=%s
Pipeline stage OfflineFamilyFriendlyTriggerPipeline completed in 0.62s
run pipeline stage %s
Running pipeline stage TriggerMKMLProfilingPipeline
run_pipeline:run_in_cloud %s
starting trigger_guanaco_pipeline args=%s
triggered trigger_guanaco_pipeline args=%s
Pipeline stage TriggerMKMLProfilingPipeline completed in 0.59s
Shutdown handler de-registered
inflatebot-mn-12b-mag-mell-r1_v4 status is now deployed due to DeploymentManager action
Shutdown handler registered
run pipeline %s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyScorer
Evaluating %s Family Friendly Score with %s threads
Pipeline stage OfflineFamilyFriendlyScorer completed in 2631.98s
Shutdown handler de-registered
inflatebot-mn-12b-mag-mell-r1_v4 status is now inactive due to auto deactivation removed underperforming models
inflatebot-mn-12b-mag-mell-r1_v4 status is now torndown due to DeploymentManager action