developer_uid: Luna1040
submission_id: intervitens-mini-magnum_51806_v9
model_name: intervitens-mini-magnum_51806_v9
model_group: intervitens/mini-magnum-
status: torndown
timestamp: 2025-02-19T07:51:05+00:00
num_battles: 5544
num_wins: 2670
celo_rating: 1250.31
family_friendly_score: 0.5516
family_friendly_standard_error: 0.007033312732987209
submission_type: basic
model_repo: intervitens/mini-magnum-12b-v1.1
model_architecture: MistralForCausalLM
model_num_parameters: 12772080640.0
best_of: 8
max_input_tokens: 1024
max_output_tokens: 64
latencies: [{'batch_size': 1, 'throughput': 0.6057340050842861, 'latency_mean': 1.6508254075050355, 'latency_p50': 1.6388869285583496, 'latency_p90': 1.8229972124099731}, {'batch_size': 3, 'throughput': 1.100323513377924, 'latency_mean': 2.724132649898529, 'latency_p50': 2.7137032747268677, 'latency_p90': 3.012637972831726}, {'batch_size': 5, 'throughput': 1.3172135499631, 'latency_mean': 3.77267422914505, 'latency_p50': 3.7592180967330933, 'latency_p90': 4.233663582801818}, {'batch_size': 6, 'throughput': 1.3726701996966184, 'latency_mean': 4.328186658620834, 'latency_p50': 4.305877089500427, 'latency_p90': 4.9220504522323605}, {'batch_size': 8, 'throughput': 1.443922182297489, 'latency_mean': 5.515030206441879, 'latency_p50': 5.5061211585998535, 'latency_p90': 6.261465764045715}, {'batch_size': 10, 'throughput': 1.4825702520747537, 'latency_mean': 6.703565453290939, 'latency_p50': 6.685311913490295, 'latency_p90': 7.642417645454406}]
gpu_counts: {'NVIDIA RTX A5000': 1}
display_name: intervitens-mini-magnum_51806_v9
is_internal_developer: False
language_model: intervitens/mini-magnum-12b-v1.1
model_size: 13B
ranking_group: single
throughput_3p7s: 1.31
us_pacific_date: 2025-02-18
win_ratio: 0.4816017316017316
generation_params: {'temperature': 1.0, 'top_p': 1.0, 'min_p': 0.0, 'top_k': 40, 'presence_penalty': 0.0, 'frequency_penalty': 0.0, 'stopping_words': ['\n'], 'max_input_tokens': 1024, 'best_of': 8, 'max_output_tokens': 64}
formatter: {'memory_template': '<|im_start|>system\n{memory}<|im_end|>\n', 'prompt_template': '<|im_start|>user\n{prompt}<|im_end|>\n', 'bot_template': '<|im_start|>assistant\n{bot_name}: {message}<|im_end|>\n', 'user_template': '<|im_start|>user\n{user_name}: {message}<|im_end|>\n', 'response_template': '<|im_start|>assistant\n{bot_name}:', 'truncate_by_message': False}
Resubmit model
Shutdown handler not registered because Python interpreter is not running in the main thread
run pipeline %s
run pipeline stage %s
Running pipeline stage MKMLizer
Starting job with name intervitens-mini-magnum-51806-v9-mkmlizer
Waiting for job on intervitens-mini-magnum-51806-v9-mkmlizer to finish
intervitens-mini-magnum-51806-v9-mkmlizer: ╔═════════════════════════════════════════════════════════════════════╗
intervitens-mini-magnum-51806-v9-mkmlizer: ║     _____            __           __                                ║
intervitens-mini-magnum-51806-v9-mkmlizer: ║    / _/ /_ ___    __/ /  ___ ___ / /                                ║
intervitens-mini-magnum-51806-v9-mkmlizer: ║   / _/ / // / |/|/ / _ \/ -_) -_) /                                 ║
intervitens-mini-magnum-51806-v9-mkmlizer: ║  /_//_/\_, /|__,__/_//_/\__/\__/_/                                  ║
intervitens-mini-magnum-51806-v9-mkmlizer: ║       /___/                                                         ║
intervitens-mini-magnum-51806-v9-mkmlizer: ║                                                                     ║
intervitens-mini-magnum-51806-v9-mkmlizer: ║  Version: 0.12.8                                                    ║
intervitens-mini-magnum-51806-v9-mkmlizer: ║  Copyright 2023 MK ONE TECHNOLOGIES Inc.                            ║
intervitens-mini-magnum-51806-v9-mkmlizer: ║  https://mk1.ai                                                     ║
intervitens-mini-magnum-51806-v9-mkmlizer: ║                                                                     ║
intervitens-mini-magnum-51806-v9-mkmlizer: ║  The license key for the current software has been verified as      ║
intervitens-mini-magnum-51806-v9-mkmlizer: ║  belonging to:                                                      ║
intervitens-mini-magnum-51806-v9-mkmlizer: ║                                                                     ║
intervitens-mini-magnum-51806-v9-mkmlizer: ║  Chai Research Corp.                                                ║
intervitens-mini-magnum-51806-v9-mkmlizer: ║  Account ID: 7997a29f-0ceb-4cc7-9adf-840c57b4ae6f                   ║
intervitens-mini-magnum-51806-v9-mkmlizer: ║  Expiration: 2025-04-15 23:59:59                                    ║
intervitens-mini-magnum-51806-v9-mkmlizer: ║                                                                     ║
intervitens-mini-magnum-51806-v9-mkmlizer: ╚═════════════════════════════════════════════════════════════════════╝
intervitens-mini-magnum-51806-v9-mkmlizer: Downloaded to shared memory in 43.427s
intervitens-mini-magnum-51806-v9-mkmlizer: quantizing model to /dev/shm/model_cache, profile:s0, folder:/tmp/tmprz__8uew, device:0
intervitens-mini-magnum-51806-v9-mkmlizer: Saving flywheel model at /dev/shm/model_cache
intervitens-mini-magnum-51806-v9-mkmlizer: quantized model in 39.062s
intervitens-mini-magnum-51806-v9-mkmlizer: Processed model intervitens/mini-magnum-12b-v1.1 in 82.489s
intervitens-mini-magnum-51806-v9-mkmlizer: creating bucket guanaco-mkml-models
intervitens-mini-magnum-51806-v9-mkmlizer: cp /dev/shm/model_cache/config.json s3://guanaco-mkml-models/intervitens-mini-magnum-51806-v9/config.json
intervitens-mini-magnum-51806-v9-mkmlizer: cp /dev/shm/model_cache/special_tokens_map.json s3://guanaco-mkml-models/intervitens-mini-magnum-51806-v9/special_tokens_map.json
intervitens-mini-magnum-51806-v9-mkmlizer: cp /dev/shm/model_cache/tokenizer_config.json s3://guanaco-mkml-models/intervitens-mini-magnum-51806-v9/tokenizer_config.json
intervitens-mini-magnum-51806-v9-mkmlizer: cp /dev/shm/model_cache/tokenizer.json s3://guanaco-mkml-models/intervitens-mini-magnum-51806-v9/tokenizer.json
intervitens-mini-magnum-51806-v9-mkmlizer: cp /dev/shm/model_cache/flywheel_model.0.safetensors s3://guanaco-mkml-models/intervitens-mini-magnum-51806-v9/flywheel_model.0.safetensors
intervitens-mini-magnum-51806-v9-mkmlizer: 
Loading 0:   0%|          | 0/363 [00:00<?, ?it/s]
Loading 0:   1%|▏         | 5/363 [00:00<00:13, 27.53it/s]
Loading 0:   3%|▎         | 10/363 [00:00<00:09, 37.40it/s]
Loading 0:   4%|▍         | 15/363 [00:00<00:09, 35.23it/s]
Loading 0:   6%|▌         | 21/363 [00:00<00:08, 40.29it/s]
Loading 0:   7%|▋         | 26/363 [00:00<00:08, 39.17it/s]
Loading 0:   9%|▊         | 31/363 [00:00<00:08, 38.19it/s]
Loading 0:  10%|▉         | 35/363 [00:00<00:08, 37.14it/s]
Loading 0:  11%|█         | 39/363 [00:01<00:08, 36.63it/s]
Loading 0:  12%|█▏        | 43/363 [00:01<00:09, 34.80it/s]
Loading 0:  13%|█▎        | 48/363 [00:01<00:08, 36.79it/s]
Loading 0:  14%|█▍        | 52/363 [00:01<00:08, 34.97it/s]
Loading 0:  15%|█▌        | 56/363 [00:01<00:08, 34.41it/s]
Loading 0:  17%|█▋        | 60/363 [00:01<00:08, 35.85it/s]
Loading 0:  18%|█▊        | 64/363 [00:02<00:14, 21.22it/s]
Loading 0:  19%|█▉        | 69/363 [00:02<00:11, 26.26it/s]
Loading 0:  20%|██        | 73/363 [00:02<00:11, 25.93it/s]
Loading 0:  21%|██▏       | 78/363 [00:02<00:09, 30.27it/s]
Loading 0:  23%|██▎       | 82/363 [00:02<00:09, 28.50it/s]
Loading 0:  24%|██▍       | 87/363 [00:02<00:08, 32.32it/s]
Loading 0:  25%|██▌       | 91/363 [00:02<00:08, 30.28it/s]
Loading 0:  26%|██▋       | 96/363 [00:02<00:07, 34.06it/s]
Loading 0:  28%|██▊       | 100/363 [00:03<00:08, 31.05it/s]
Loading 0:  29%|██▉       | 105/363 [00:03<00:07, 35.01it/s]
Loading 0:  30%|███       | 110/363 [00:03<00:06, 36.18it/s]
Loading 0:  31%|███▏      | 114/363 [00:03<00:07, 34.13it/s]
Loading 0:  33%|███▎      | 118/363 [00:03<00:07, 31.47it/s]
Loading 0:  34%|███▍      | 123/363 [00:03<00:06, 35.48it/s]
Loading 0:  35%|███▍      | 127/363 [00:03<00:07, 32.36it/s]
Loading 0:  36%|███▋      | 132/363 [00:03<00:06, 35.60it/s]
Loading 0:  37%|███▋      | 136/363 [00:04<00:07, 30.99it/s]
Loading 0:  39%|███▉      | 141/363 [00:04<00:06, 34.85it/s]
Loading 0:  40%|███▉      | 145/363 [00:04<00:09, 23.31it/s]
Loading 0:  41%|████      | 149/363 [00:04<00:08, 23.78it/s]
Loading 0:  42%|████▏     | 154/363 [00:04<00:07, 28.45it/s]
Loading 0:  44%|████▎     | 158/363 [00:05<00:07, 27.31it/s]
Loading 0:  45%|████▍     | 163/363 [00:05<00:06, 31.42it/s]
Loading 0:  46%|████▌     | 167/363 [00:05<00:06, 29.75it/s]
Loading 0:  47%|████▋     | 172/363 [00:05<00:05, 33.84it/s]
Loading 0:  48%|████▊     | 176/363 [00:05<00:06, 30.16it/s]
Loading 0:  50%|█████     | 183/363 [00:05<00:04, 37.13it/s]
Loading 0:  52%|█████▏    | 187/363 [00:05<00:05, 34.88it/s]
Loading 0:  53%|█████▎    | 192/363 [00:05<00:04, 36.31it/s]
Loading 0:  54%|█████▍    | 196/363 [00:06<00:04, 34.26it/s]
Loading 0:  55%|█████▌    | 201/363 [00:06<00:04, 36.23it/s]
Loading 0:  56%|█████▋    | 205/363 [00:06<00:04, 33.98it/s]
Loading 0:  58%|█████▊    | 209/363 [00:06<00:04, 35.16it/s]
Loading 0:  59%|█████▊    | 213/363 [00:06<00:04, 31.82it/s]
Loading 0:  60%|█████▉    | 217/363 [00:06<00:04, 33.22it/s]
Loading 0:  61%|██████    | 222/363 [00:06<00:04, 35.22it/s]
Loading 0:  62%|██████▏   | 226/363 [00:07<00:05, 23.06it/s]
Loading 0:  63%|██████▎   | 230/363 [00:07<00:05, 23.59it/s]
Loading 0:  65%|██████▍   | 235/363 [00:07<00:04, 28.32it/s]
Loading 0:  66%|██████▌   | 239/363 [00:07<00:04, 27.22it/s]
Loading 0:  67%|██████▋   | 244/363 [00:07<00:03, 31.99it/s]
Loading 0:  68%|██████▊   | 248/363 [00:07<00:03, 30.35it/s]
Loading 0:  70%|██████▉   | 253/363 [00:07<00:03, 34.05it/s]
Loading 0:  71%|███████   | 257/363 [00:08<00:03, 31.51it/s]
Loading 0:  72%|███████▏  | 262/363 [00:08<00:02, 35.37it/s]
Loading 0:  73%|███████▎  | 266/363 [00:08<00:03, 31.91it/s]
Loading 0:  75%|███████▍  | 271/363 [00:08<00:02, 35.70it/s]
Loading 0:  76%|███████▌  | 275/363 [00:08<00:02, 33.03it/s]
Loading 0:  77%|███████▋  | 281/363 [00:08<00:02, 39.57it/s]
Loading 0:  79%|███████▉  | 286/363 [00:08<00:02, 36.54it/s]
Loading 0:  80%|████████  | 291/363 [00:09<00:01, 37.86it/s]
Loading 0:  81%|████████▏ | 295/363 [00:09<00:01, 35.63it/s]
Loading 0:  82%|████████▏ | 299/363 [00:09<00:01, 34.35it/s]
Loading 0:  84%|████████▎ | 304/363 [00:16<00:27,  2.12it/s]
Loading 0:  85%|████████▍ | 307/363 [00:16<00:21,  2.64it/s]
Loading 0:  86%|████████▌ | 311/363 [00:16<00:14,  3.63it/s]
Loading 0:  87%|████████▋ | 314/363 [00:16<00:10,  4.58it/s]
Loading 0:  88%|████████▊ | 319/363 [00:16<00:06,  6.77it/s]
Loading 0:  89%|████████▉ | 323/363 [00:16<00:04,  8.77it/s]
Loading 0:  90%|█████████ | 327/363 [00:16<00:03, 11.37it/s]
Loading 0:  91%|█████████ | 331/363 [00:17<00:02, 13.65it/s]
Loading 0:  92%|█████████▏| 335/363 [00:17<00:01, 16.83it/s]
Loading 0:  93%|█████████▎| 339/363 [00:17<00:01, 18.93it/s]
Loading 0:  95%|█████████▍| 344/363 [00:17<00:00, 24.04it/s]
Loading 0:  96%|█████████▌| 348/363 [00:17<00:00, 24.90it/s]
Loading 0:  97%|█████████▋| 353/363 [00:17<00:00, 29.94it/s]
Loading 0:  98%|█████████▊| 357/363 [00:17<00:00, 28.90it/s]
Loading 0: 100%|█████████▉| 362/363 [00:17<00:00, 33.32it/s]
                                                            

Job intervitens-mini-magnum-51806-v9-mkmlizer completed after 104.09s with status: succeeded
Stopping job with name intervitens-mini-magnum-51806-v9-mkmlizer
Pipeline stage MKMLizer completed in 104.56s
run pipeline stage %s
Running pipeline stage MKMLTemplater
Pipeline stage MKMLTemplater completed in 0.15s
run pipeline stage %s
Running pipeline stage MKMLDeployer
Creating inference service intervitens-mini-magnum-51806-v9
Waiting for inference service intervitens-mini-magnum-51806-v9 to be ready
Failed to get response for submission chaiml-20250218-c-4epoc_55567_v2: HTTPConnectionPool(host='chaiml-20250218-c-4epoc-55567-v2-predictor.tenant-chaiml-guanaco.k.chaiverse.com', port=80): Read timed out. (read timeout=12.0)
Inference service intervitens-mini-magnum-51806-v9 ready after 160.53144907951355s
Pipeline stage MKMLDeployer completed in 161.04s
run pipeline stage %s
Running pipeline stage StressChecker
Received healthy response to inference request in 2.289088249206543s
Received healthy response to inference request in 1.532757043838501s
Failed to get response for submission chaiml-20250218-c-4epoc_55567_v1: HTTPConnectionPool(host='chaiml-20250218-c-4epoc-55567-v1-predictor.tenant-chaiml-guanaco.k.chaiverse.com', port=80): Read timed out. (read timeout=12.0)
Received healthy response to inference request in 1.5075461864471436s
Received healthy response to inference request in 1.69647216796875s
Received healthy response to inference request in 1.646230936050415s
5 requests
0 failed requests
5th percentile: 1.5125883579254151
10th percentile: 1.5176305294036865
20th percentile: 1.5277148723602294
30th percentile: 1.5554518222808837
40th percentile: 1.6008413791656495
50th percentile: 1.646230936050415
60th percentile: 1.666327428817749
70th percentile: 1.686423921585083
80th percentile: 1.8149953842163087
90th percentile: 2.0520418167114256
95th percentile: 2.1705650329589843
99th percentile: 2.265383605957031
mean time: 1.7344189167022706
Pipeline stage StressChecker completed in 9.91s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyTriggerPipeline
run_pipeline:run_in_cloud %s
starting trigger_guanaco_pipeline args=%s
triggered trigger_guanaco_pipeline args=%s
Pipeline stage OfflineFamilyFriendlyTriggerPipeline completed in 0.61s
run pipeline stage %s
Running pipeline stage TriggerMKMLProfilingPipeline
run_pipeline:run_in_cloud %s
starting trigger_guanaco_pipeline args=%s
triggered trigger_guanaco_pipeline args=%s
Pipeline stage TriggerMKMLProfilingPipeline completed in 0.66s
Shutdown handler de-registered
intervitens-mini-magnum_51806_v9 status is now deployed due to DeploymentManager action
Shutdown handler registered
run pipeline %s
run pipeline stage %s
Running pipeline stage MKMLProfilerDeleter
Skipping teardown as no inference service was successfully deployed
Pipeline stage MKMLProfilerDeleter completed in 0.10s
run pipeline stage %s
Running pipeline stage MKMLProfilerTemplater
Pipeline stage MKMLProfilerTemplater completed in 0.08s
run pipeline stage %s
Running pipeline stage MKMLProfilerDeployer
Creating inference service intervitens-mini-magnum-51806-v9-profiler
Waiting for inference service intervitens-mini-magnum-51806-v9-profiler to be ready
Shutdown handler registered
run pipeline %s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyScorer
Evaluating %s Family Friendly Score with %s threads
Pipeline stage OfflineFamilyFriendlyScorer completed in 2589.63s
Shutdown handler de-registered
intervitens-mini-magnum_51806_v9 status is now inactive due to auto deactivation removed underperforming models
intervitens-mini-magnum_51806_v9 status is now torndown due to DeploymentManager action
admin requested tearing down of intervitens-mini-magnum_51806_v9
Checking if service leosheng-ft-dpo-219-v7 is running
Shutdown handler not registered because Python interpreter is not running in the main thread
run pipeline %s
run pipeline stage %s
Running pipeline stage MKMLDeleter
Pipeline stage %s skipped, reason=%s
Pipeline stage MKMLDeleter completed in 0.48s
run pipeline stage %s
Tearing down inference service leosheng-ft-dpo-219-v6
Running pipeline stage MKMLModelDeleter
Service leosheng-ft-dpo-219-v6 has been torndown
Pipeline stage %s skipped, reason=%s
Pipeline stage MKMLDeleter completed in 3.92s
Tearing down inference service leosheng-ft-dpo-219-v7
Pipeline stage MKMLModelDeleter completed in 0.81s
run pipeline stage %s
Service leosheng-ft-dpo-219-v7 has been torndown
Shutdown handler de-registered
Running pipeline stage MKMLModelDeleter
Pipeline stage MKMLDeleter completed in 4.17s
intervitens-mini-magnum_51806_v9 status is now torndown due to DeploymentManager action
intervitens-mini-magnum_51806_v9 status is now torndown due to DeploymentManager action