developer_uid: rirv938
submission_id: rirv938-mistral-12b-ant_70830_v2
model_name: rirv938-mistral-12b-ant_70830_v2
model_group: rirv938/mistral_12b_anth
status: torndown
timestamp: 2025-02-20T19:17:08+00:00
num_battles: 6483
num_wins: 3296
celo_rating: 1274.74
family_friendly_score: 0.5724
family_friendly_standard_error: 0.006996545433283485
submission_type: basic
model_repo: rirv938/mistral_12b_anthropic_1120_300_seed_20_v2
model_architecture: MistralForCausalLM
model_num_parameters: 12772070400.0
best_of: 8
max_input_tokens: 1024
max_output_tokens: 64
latencies: [{'batch_size': 1, 'throughput': 0.5996518579867631, 'latency_mean': 1.6675467205047607, 'latency_p50': 1.6722034215927124, 'latency_p90': 1.8178396224975586}, {'batch_size': 3, 'throughput': 1.082501239747022, 'latency_mean': 2.763930319547653, 'latency_p50': 2.7702311277389526, 'latency_p90': 3.005135750770569}, {'batch_size': 5, 'throughput': 1.3109781397893379, 'latency_mean': 3.799493978023529, 'latency_p50': 3.796163558959961, 'latency_p90': 4.31948983669281}, {'batch_size': 6, 'throughput': 1.374877716990177, 'latency_mean': 4.343628431558609, 'latency_p50': 4.302603721618652, 'latency_p90': 4.969324660301209}, {'batch_size': 8, 'throughput': 1.4459167769562253, 'latency_mean': 5.489816703796387, 'latency_p50': 5.45887553691864, 'latency_p90': 6.113070130348206}, {'batch_size': 10, 'throughput': 1.4493541734343933, 'latency_mean': 6.847855634689331, 'latency_p50': 6.849611520767212, 'latency_p90': 7.650791120529175}]
gpu_counts: {'NVIDIA RTX A5000': 1}
display_name: rirv938-mistral-12b-ant_70830_v2
is_internal_developer: True
language_model: rirv938/mistral_12b_anthropic_1120_300_seed_20_v2
model_size: 13B
ranking_group: single
throughput_3p7s: 1.3
us_pacific_date: 2025-02-20
win_ratio: 0.5084066018818448
generation_params: {'temperature': 0.9, 'top_p': 0.9, 'min_p': 0.6, 'top_k': 80, 'presence_penalty': 0.5, 'frequency_penalty': 0.5, 'stopping_words': ['\n', '</s>', '###', 'You:'], 'max_input_tokens': 1024, 'best_of': 8, 'max_output_tokens': 64}
formatter: {'memory_template': '', 'prompt_template': '', 'bot_template': '{bot_name}: {message}\n', 'user_template': '{user_name}: {message}\n', 'response_template': '{bot_name}:', 'truncate_by_message': False}
Resubmit model
Shutdown handler not registered because Python interpreter is not running in the main thread
run pipeline %s
run pipeline stage %s
Running pipeline stage MKMLizer
Starting job with name rirv938-mistral-12b-ant-70830-v2-mkmlizer
Waiting for job on rirv938-mistral-12b-ant-70830-v2-mkmlizer to finish
rirv938-mistral-12b-ant-70830-v2-mkmlizer: ╔═════════════════════════════════════════════════════════════════════╗
rirv938-mistral-12b-ant-70830-v2-mkmlizer: ║     _____            __           __                                ║
rirv938-mistral-12b-ant-70830-v2-mkmlizer: ║    / _/ /_ ___    __/ /  ___ ___ / /                                ║
rirv938-mistral-12b-ant-70830-v2-mkmlizer: ║   / _/ / // / |/|/ / _ \/ -_) -_) /                                 ║
rirv938-mistral-12b-ant-70830-v2-mkmlizer: ║  /_//_/\_, /|__,__/_//_/\__/\__/_/                                  ║
rirv938-mistral-12b-ant-70830-v2-mkmlizer: ║       /___/                                                         ║
rirv938-mistral-12b-ant-70830-v2-mkmlizer: ║                                                                     ║
rirv938-mistral-12b-ant-70830-v2-mkmlizer: ║  Version: 0.12.8                                                    ║
rirv938-mistral-12b-ant-70830-v2-mkmlizer: ║  Copyright 2023 MK ONE TECHNOLOGIES Inc.                            ║
rirv938-mistral-12b-ant-70830-v2-mkmlizer: ║  https://mk1.ai                                                     ║
rirv938-mistral-12b-ant-70830-v2-mkmlizer: ║                                                                     ║
rirv938-mistral-12b-ant-70830-v2-mkmlizer: ║  The license key for the current software has been verified as      ║
rirv938-mistral-12b-ant-70830-v2-mkmlizer: ║  belonging to:                                                      ║
rirv938-mistral-12b-ant-70830-v2-mkmlizer: ║                                                                     ║
rirv938-mistral-12b-ant-70830-v2-mkmlizer: ║  Chai Research Corp.                                                ║
rirv938-mistral-12b-ant-70830-v2-mkmlizer: ║  Account ID: 7997a29f-0ceb-4cc7-9adf-840c57b4ae6f                   ║
rirv938-mistral-12b-ant-70830-v2-mkmlizer: ║  Expiration: 2025-04-15 23:59:59                                    ║
rirv938-mistral-12b-ant-70830-v2-mkmlizer: ║                                                                     ║
rirv938-mistral-12b-ant-70830-v2-mkmlizer: ╚═════════════════════════════════════════════════════════════════════╝
Failed to get response for submission rirv938-mistral-12b-ant_46303_v1: HTTPConnectionPool(host='rirv938-mistral-12b-ant-46303-v1-predictor.tenant-chaiml-guanaco.k.chaiverse.com', port=80): Read timed out. (read timeout=12.0)
Failed to get response for submission rirv938-mistral-12b-ant_46303_v1: HTTPConnectionPool(host='rirv938-mistral-12b-ant-46303-v1-predictor.tenant-chaiml-guanaco.k.chaiverse.com', port=80): Read timed out. (read timeout=12.0)
rirv938-mistral-12b-ant-70830-v2-mkmlizer: Downloaded to shared memory in 90.508s
rirv938-mistral-12b-ant-70830-v2-mkmlizer: quantizing model to /dev/shm/model_cache, profile:s0, folder:/tmp/tmpjwcg50f7, device:0
rirv938-mistral-12b-ant-70830-v2-mkmlizer: Saving flywheel model at /dev/shm/model_cache
Failed to get response for submission rirv938-mistral-12b-ant_41569_v1: HTTPConnectionPool(host='rirv938-mistral-12b-ant-41569-v1-predictor.tenant-chaiml-guanaco.k.chaiverse.com', port=80): Read timed out. (read timeout=12.0)
rirv938-mistral-12b-ant-70830-v2-mkmlizer: quantized model in 43.376s
rirv938-mistral-12b-ant-70830-v2-mkmlizer: Processed model rirv938/mistral_12b_anthropic_1120_300_seed_20_v2 in 133.884s
rirv938-mistral-12b-ant-70830-v2-mkmlizer: creating bucket guanaco-mkml-models
rirv938-mistral-12b-ant-70830-v2-mkmlizer: cp /dev/shm/model_cache/config.json s3://guanaco-mkml-models/rirv938-mistral-12b-ant-70830-v2/config.json
rirv938-mistral-12b-ant-70830-v2-mkmlizer: cp /dev/shm/model_cache/special_tokens_map.json s3://guanaco-mkml-models/rirv938-mistral-12b-ant-70830-v2/special_tokens_map.json
rirv938-mistral-12b-ant-70830-v2-mkmlizer: cp /dev/shm/model_cache/tokenizer_config.json s3://guanaco-mkml-models/rirv938-mistral-12b-ant-70830-v2/tokenizer_config.json
rirv938-mistral-12b-ant-70830-v2-mkmlizer: cp /dev/shm/model_cache/tokenizer.json s3://guanaco-mkml-models/rirv938-mistral-12b-ant-70830-v2/tokenizer.json
rirv938-mistral-12b-ant-70830-v2-mkmlizer: cp /dev/shm/model_cache/flywheel_model.0.safetensors s3://guanaco-mkml-models/rirv938-mistral-12b-ant-70830-v2/flywheel_model.0.safetensors
rirv938-mistral-12b-ant-70830-v2-mkmlizer: 
Loading 0:   0%|          | 0/363 [00:00<?, ?it/s]
Loading 0:   1%|▏         | 5/363 [00:00<00:16, 21.25it/s]
Loading 0:   3%|▎         | 10/363 [00:00<00:12, 27.32it/s]
Loading 0:   4%|▎         | 13/363 [00:00<00:12, 27.50it/s]
Loading 0:   4%|▍         | 16/363 [00:00<00:14, 24.47it/s]
Loading 0:   6%|▌         | 20/363 [00:00<00:12, 28.17it/s]
Loading 0:   6%|▋         | 23/363 [00:01<00:17, 18.92it/s]
Loading 0:   7%|▋         | 26/363 [00:01<00:19, 17.56it/s]
Loading 0:   9%|▊         | 31/363 [00:01<00:14, 23.61it/s]
Loading 0:   9%|▉         | 34/363 [00:01<00:13, 24.26it/s]
Loading 0:  11%|█         | 39/363 [00:01<00:12, 26.92it/s]
Loading 0:  12%|█▏        | 42/363 [00:01<00:13, 24.29it/s]
Loading 0:  13%|█▎        | 48/363 [00:01<00:10, 29.43it/s]
Loading 0:  14%|█▍        | 52/363 [00:02<00:10, 28.79it/s]
Loading 0:  15%|█▌        | 56/363 [00:02<00:10, 29.50it/s]
Loading 0:  17%|█▋        | 61/363 [00:02<00:11, 25.68it/s]
Loading 0:  18%|█▊        | 64/363 [00:02<00:13, 22.38it/s]
Loading 0:  20%|█▉        | 71/363 [00:02<00:09, 29.60it/s]
Loading 0:  21%|██        | 75/363 [00:02<00:09, 29.40it/s]
Loading 0:  22%|██▏       | 79/363 [00:03<00:09, 28.53it/s]
Loading 0:  23%|██▎       | 84/363 [00:03<00:08, 31.49it/s]
Loading 0:  24%|██▍       | 88/363 [00:03<00:09, 29.95it/s]
Loading 0:  26%|██▌       | 93/363 [00:03<00:08, 32.63it/s]
Loading 0:  27%|██▋       | 97/363 [00:03<00:08, 31.45it/s]
Loading 0:  28%|██▊       | 101/363 [00:03<00:10, 25.95it/s]
Loading 0:  29%|██▊       | 104/363 [00:04<00:11, 22.50it/s]
Loading 0:  31%|███       | 111/363 [00:04<00:08, 29.36it/s]
Loading 0:  32%|███▏      | 115/363 [00:04<00:08, 29.11it/s]
Loading 0:  33%|███▎      | 120/363 [00:04<00:07, 32.04it/s]
Loading 0:  34%|███▍      | 124/363 [00:04<00:08, 29.41it/s]
Loading 0:  36%|███▌      | 129/363 [00:04<00:07, 30.98it/s]
Loading 0:  37%|███▋      | 133/363 [00:04<00:07, 29.94it/s]
Loading 0:  38%|███▊      | 137/363 [00:04<00:07, 30.26it/s]
Loading 0:  39%|███▉      | 141/363 [00:05<00:06, 32.12it/s]
Loading 0:  40%|███▉      | 145/363 [00:05<00:10, 21.20it/s]
Loading 0:  41%|████      | 149/363 [00:05<00:10, 20.42it/s]
Loading 0:  42%|████▏     | 154/363 [00:05<00:08, 25.56it/s]
Loading 0:  44%|████▎     | 158/363 [00:05<00:08, 23.29it/s]
Loading 0:  45%|████▍     | 163/363 [00:06<00:07, 27.87it/s]
Loading 0:  46%|████▌     | 167/363 [00:06<00:08, 24.32it/s]
Loading 0:  47%|████▋     | 172/363 [00:06<00:06, 27.38it/s]
Loading 0:  48%|████▊     | 176/363 [00:06<00:07, 24.67it/s]
Loading 0:  50%|████▉     | 181/363 [00:06<00:06, 29.18it/s]
Loading 0:  51%|█████     | 185/363 [00:07<00:09, 19.37it/s]
Loading 0:  52%|█████▏    | 190/363 [00:07<00:07, 23.57it/s]
Loading 0:  53%|█████▎    | 194/363 [00:07<00:07, 21.78it/s]
Loading 0:  55%|█████▍    | 199/363 [00:07<00:06, 25.78it/s]
Loading 0:  56%|█████▌    | 203/363 [00:07<00:06, 23.18it/s]
Loading 0:  57%|█████▋    | 208/363 [00:07<00:05, 27.28it/s]
Loading 0:  58%|█████▊    | 212/363 [00:08<00:06, 24.08it/s]
Loading 0:  60%|█████▉    | 217/363 [00:08<00:05, 28.30it/s]
Loading 0:  61%|██████    | 222/363 [00:08<00:04, 29.14it/s]
Loading 0:  62%|██████▏   | 226/363 [00:08<00:06, 20.80it/s]
Loading 0:  63%|██████▎   | 230/363 [00:08<00:06, 20.15it/s]
Loading 0:  65%|██████▍   | 235/363 [00:09<00:05, 24.36it/s]
Loading 0:  66%|██████▌   | 239/363 [00:09<00:05, 22.30it/s]
Loading 0:  67%|██████▋   | 244/363 [00:09<00:04, 26.62it/s]
Loading 0:  68%|██████▊   | 248/363 [00:09<00:04, 23.35it/s]
Loading 0:  70%|██████▉   | 253/363 [00:09<00:03, 27.53it/s]
Loading 0:  71%|███████   | 257/363 [00:10<00:04, 24.03it/s]
Loading 0:  72%|███████▏  | 262/363 [00:10<00:03, 28.19it/s]
Loading 0:  73%|███████▎  | 266/363 [00:10<00:05, 18.76it/s]
Loading 0:  75%|███████▍  | 271/363 [00:10<00:03, 23.14it/s]
Loading 0:  76%|███████▌  | 275/363 [00:10<00:04, 21.76it/s]
Loading 0:  77%|███████▋  | 280/363 [00:10<00:03, 26.13it/s]
Loading 0:  78%|███████▊  | 284/363 [00:11<00:03, 23.61it/s]
Loading 0:  80%|███████▉  | 289/363 [00:11<00:02, 27.82it/s]
Loading 0:  81%|████████  | 293/363 [00:11<00:02, 23.91it/s]
Loading 0:  82%|████████▏ | 298/363 [00:11<00:02, 28.13it/s]
Loading 0:  83%|████████▎ | 303/363 [00:11<00:02, 28.94it/s]
Loading 0:  85%|████████▍ | 307/363 [00:12<00:02, 20.69it/s]
Loading 0:  86%|████████▌ | 311/363 [00:12<00:02, 20.01it/s]
Loading 0:  87%|████████▋ | 316/363 [00:12<00:01, 24.29it/s]
Loading 0:  88%|████████▊ | 320/363 [00:12<00:01, 22.23it/s]
Loading 0:  90%|████████▉ | 325/363 [00:12<00:01, 26.42it/s]
Loading 0:  91%|█████████ | 329/363 [00:13<00:01, 23.02it/s]
Loading 0:  92%|█████████▏| 334/363 [00:13<00:01, 27.12it/s]
Loading 0:  93%|█████████▎| 338/363 [00:13<00:01, 23.26it/s]
Loading 0:  94%|█████████▍| 343/363 [00:13<00:00, 27.39it/s]
Loading 0:  96%|█████████▌| 347/363 [00:20<00:08,  1.96it/s]
Loading 0:  96%|█████████▋| 350/363 [00:20<00:05,  2.45it/s]
Loading 0:  97%|█████████▋| 353/363 [00:20<00:03,  3.12it/s]
Loading 0:  98%|█████████▊| 357/363 [00:21<00:01,  4.25it/s]
                                                            

Job rirv938-mistral-12b-ant-70830-v2-mkmlizer completed after 165.4s with status: succeeded
Stopping job with name rirv938-mistral-12b-ant-70830-v2-mkmlizer
Pipeline stage MKMLizer completed in 165.95s
run pipeline stage %s
Running pipeline stage MKMLTemplater
Pipeline stage MKMLTemplater completed in 0.17s
run pipeline stage %s
Running pipeline stage MKMLDeployer
Creating inference service rirv938-mistral-12b-ant-70830-v2
Waiting for inference service rirv938-mistral-12b-ant-70830-v2 to be ready
Failed to get response for submission rirv938-mistral-12b-ant_46303_v1: HTTPConnectionPool(host='rirv938-mistral-12b-ant-46303-v1-predictor.tenant-chaiml-guanaco.k.chaiverse.com', port=80): Read timed out. (read timeout=12.0)
Failed to get response for submission rirv938-mistral-12b-ant_70830_v1: HTTPConnectionPool(host='rirv938-mistral-12b-ant-70830-v1-predictor.tenant-chaiml-guanaco.k.chaiverse.com', port=80): Read timed out. (read timeout=12.0)
Failed to get response for submission rirv938-mistral-12b-ant_70830_v1: HTTPConnectionPool(host='rirv938-mistral-12b-ant-70830-v1-predictor.tenant-chaiml-guanaco.k.chaiverse.com', port=80): Read timed out. (read timeout=12.0)
Inference service rirv938-mistral-12b-ant-70830-v2 ready after 200.7239933013916s
Pipeline stage MKMLDeployer completed in 201.28s
run pipeline stage %s
Running pipeline stage StressChecker
Received healthy response to inference request in 2.3354148864746094s
Received healthy response to inference request in 1.6305406093597412s
Failed to get response for submission rirv938-mistral-12b-ant_46303_v1: HTTPConnectionPool(host='rirv938-mistral-12b-ant-46303-v1-predictor.tenant-chaiml-guanaco.k.chaiverse.com', port=80): Read timed out. (read timeout=12.0)
Received healthy response to inference request in 1.9837796688079834s
Received healthy response to inference request in 1.5899884700775146s
Received healthy response to inference request in 1.5076384544372559s
5 requests
0 failed requests
5th percentile: 1.5241084575653077
10th percentile: 1.5405784606933595
20th percentile: 1.5735184669494628
30th percentile: 1.5980988979339599
40th percentile: 1.6143197536468505
50th percentile: 1.6305406093597412
60th percentile: 1.771836233139038
70th percentile: 1.9131318569183349
80th percentile: 2.0541067123413086
90th percentile: 2.194760799407959
95th percentile: 2.265087842941284
99th percentile: 2.3213494777679444
mean time: 1.809472417831421
Pipeline stage StressChecker completed in 10.37s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyTriggerPipeline
run_pipeline:run_in_cloud %s
starting trigger_guanaco_pipeline args=%s
triggered trigger_guanaco_pipeline args=%s
Pipeline stage OfflineFamilyFriendlyTriggerPipeline completed in 0.64s
run pipeline stage %s
Running pipeline stage TriggerMKMLProfilingPipeline
run_pipeline:run_in_cloud %s
starting trigger_guanaco_pipeline args=%s
triggered trigger_guanaco_pipeline args=%s
Pipeline stage TriggerMKMLProfilingPipeline completed in 0.66s
Shutdown handler de-registered
rirv938-mistral-12b-ant_70830_v2 status is now deployed due to DeploymentManager action
Shutdown handler registered
run pipeline %s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyScorer
Evaluating %s Family Friendly Score with %s threads
Pipeline stage OfflineFamilyFriendlyScorer completed in 2634.16s
Shutdown handler de-registered
rirv938-mistral-12b-ant_70830_v2 status is now inactive due to auto deactivation removed underperforming models
rirv938-mistral-12b-ant_70830_v2 status is now torndown due to DeploymentManager action
ChatRequest
Generation Params
Prompt Formatter
Chat History
ChatMessage 1