developer_uid: zmeeks
submission_id: zmeeks-capitanito-53_v13
model_name: capitanito_t10__z53__ppmmp
model_group: zmeeks/capitanito__53
status: torndown
timestamp: 2025-07-17T15:40:17+00:00
num_battles: 5722
num_wins: 2760
celo_rating: 1251.66
family_friendly_score: 0.5804
family_friendly_standard_error: 0.006979052084631551
submission_type: basic
model_repo: zmeeks/capitanito__53
model_architecture: MistralForCausalLM
model_num_parameters: 12772070400.0
best_of: 8
max_input_tokens: 1024
max_output_tokens: 64
reward_model: default
latencies: [{'batch_size': 1, 'throughput': 0.5904199857550317, 'latency_mean': 1.6935946333408356, 'latency_p50': 1.6960028409957886, 'latency_p90': 1.8676511287689208}, {'batch_size': 3, 'throughput': 1.0610007256564657, 'latency_mean': 2.8153246676921846, 'latency_p50': 2.793480634689331, 'latency_p90': 3.1483308553695677}, {'batch_size': 5, 'throughput': 1.2720819035201572, 'latency_mean': 3.91403666973114, 'latency_p50': 3.9065037965774536, 'latency_p90': 4.393639659881591}, {'batch_size': 6, 'throughput': 1.3325670029287409, 'latency_mean': 4.478916736841202, 'latency_p50': 4.453136324882507, 'latency_p90': 5.009203243255615}, {'batch_size': 8, 'throughput': 1.3747309416928288, 'latency_mean': 5.77757760643959, 'latency_p50': 5.781431317329407, 'latency_p90': 6.471193480491638}, {'batch_size': 10, 'throughput': 1.4250692245835843, 'latency_mean': 6.970256663560868, 'latency_p50': 7.023592710494995, 'latency_p90': 7.786097455024719}]
gpu_counts: {'NVIDIA RTX A5000': 1}
display_name: capitanito_t10__z53__ppmmp
is_internal_developer: False
language_model: zmeeks/capitanito__53
model_size: 13B
ranking_group: single
throughput_3p7s: 1.25
us_pacific_date: 2025-07-17
win_ratio: 0.482348829080741
generation_params: {'temperature': 1.0, 'top_p': 1.0, 'min_p': 0.0, 'top_k': 25, 'presence_penalty': 0.0, 'frequency_penalty': 0.0, 'stopping_words': ['\n'], 'max_input_tokens': 1024, 'best_of': 8, 'max_output_tokens': 64}
formatter: {'memory_template': "{bot_name}'s Persona: {memory}\n####\n", 'prompt_template': '{prompt}\n<START>\n', 'bot_template': '{bot_name}: {message}\n', 'user_template': '{user_name}: {message}\n', 'response_template': '{bot_name}:', 'truncate_by_message': True}
Resubmit model
Shutdown handler not registered because Python interpreter is not running in the main thread
run pipeline %s
run pipeline stage %s
Running pipeline stage MKMLizer
Starting job with name zmeeks-capitanito-53-v13-mkmlizer
Waiting for job on zmeeks-capitanito-53-v13-mkmlizer to finish
zmeeks-capitanito-53-v13-mkmlizer: ╔═════════════════════════════════════════════════════════════════════╗
zmeeks-capitanito-53-v13-mkmlizer: ║                                                                     ║
zmeeks-capitanito-53-v13-mkmlizer: ║   ██████   ██████  █████   ████  ████                               ║
zmeeks-capitanito-53-v13-mkmlizer: ║  ░░██████ ██████  ░░███   ███░  ░░███                               ║
zmeeks-capitanito-53-v13-mkmlizer: ║   ░███░█████░███   ░███  ███     ░███                               ║
zmeeks-capitanito-53-v13-mkmlizer: ║   ░███░░███ ░███   ░███████      ░███                               ║
zmeeks-capitanito-53-v13-mkmlizer: ║   ░███ ░░░  ░███   ░███░░███     ░███                               ║
zmeeks-capitanito-53-v13-mkmlizer: ║   ░███      ░███   ░███ ░░███    ░███                               ║
zmeeks-capitanito-53-v13-mkmlizer: ║   █████     █████  █████ ░░████  █████                              ║
zmeeks-capitanito-53-v13-mkmlizer: ║  ░░░░░     ░░░░░  ░░░░░   ░░░░  ░░░░░                               ║
zmeeks-capitanito-53-v13-mkmlizer: ║                                                                     ║
zmeeks-capitanito-53-v13-mkmlizer: ║  Version: 0.29.15                                                   ║
zmeeks-capitanito-53-v13-mkmlizer: ║  Features: FLYWHEEL, CUDA                                           ║
zmeeks-capitanito-53-v13-mkmlizer: ║  Copyright 2023-2025 MK ONE TECHNOLOGIES Inc.                       ║
zmeeks-capitanito-53-v13-mkmlizer: ║  https://mk1.ai                                                     ║
zmeeks-capitanito-53-v13-mkmlizer: ║                                                                     ║
zmeeks-capitanito-53-v13-mkmlizer: ║  The license key for the current software has been verified as      ║
zmeeks-capitanito-53-v13-mkmlizer: ║  belonging to:                                                      ║
zmeeks-capitanito-53-v13-mkmlizer: ║                                                                     ║
zmeeks-capitanito-53-v13-mkmlizer: ║  Chai Research Corp.                                                ║
zmeeks-capitanito-53-v13-mkmlizer: ║  Account ID: 7997a29f-0ceb-4cc7-9adf-840c57b4ae6f                   ║
zmeeks-capitanito-53-v13-mkmlizer: ║  Expiration: 2028-03-31 23:59:59                                    ║
zmeeks-capitanito-53-v13-mkmlizer: ║                                                                     ║
zmeeks-capitanito-53-v13-mkmlizer: ╚═════════════════════════════════════════════════════════════════════╝
Failed to get response for submission blend_fader_2025-07-10: ('http://chaiml-slerpv5-mistral-24540-v21-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'read tcp 127.0.0.1:34720->127.0.0.1:8080: read: connection reset by peer\n')
zmeeks-capitanito-53-v13-mkmlizer: Downloaded to shared memory in 27.462s
zmeeks-capitanito-53-v13-mkmlizer: Checking if zmeeks/capitanito__53 already exists in ChaiML
zmeeks-capitanito-53-v13-mkmlizer: quantizing model to /dev/shm/model_cache, profile:s0, folder:/tmp/tmp4j2agf8x, device:0
zmeeks-capitanito-53-v13-mkmlizer: Saving flywheel model at /dev/shm/model_cache
Failed to get response for submission chaiml-nis-8b-v1-llama3_64493_v3: HTTPConnectionPool(host='chaiml-nis-8b-v1-llama3-64493-v3-predictor.tenant-chaiml-guanaco.k.chaiverse.com', port=80): Read timed out. (read timeout=12.0)
zmeeks-capitanito-53-v13-mkmlizer: creating bucket guanaco-mkml-models
zmeeks-capitanito-53-v13-mkmlizer: Bucket 's3://guanaco-mkml-models/' created
zmeeks-capitanito-53-v13-mkmlizer: uploading /dev/shm/model_cache to s3://guanaco-mkml-models/zmeeks-capitanito-53-v13/nvidia
zmeeks-capitanito-53-v13-mkmlizer: cp /dev/shm/model_cache/config.json s3://guanaco-mkml-models/zmeeks-capitanito-53-v13/nvidia/config.json
zmeeks-capitanito-53-v13-mkmlizer: cp /dev/shm/model_cache/special_tokens_map.json s3://guanaco-mkml-models/zmeeks-capitanito-53-v13/nvidia/special_tokens_map.json
zmeeks-capitanito-53-v13-mkmlizer: cp /dev/shm/model_cache/tokenizer_config.json s3://guanaco-mkml-models/zmeeks-capitanito-53-v13/nvidia/tokenizer_config.json
zmeeks-capitanito-53-v13-mkmlizer: cp /dev/shm/model_cache/tokenizer.json s3://guanaco-mkml-models/zmeeks-capitanito-53-v13/nvidia/tokenizer.json
zmeeks-capitanito-53-v13-mkmlizer: cp /dev/shm/model_cache/flywheel_model.0.safetensors s3://guanaco-mkml-models/zmeeks-capitanito-53-v13/nvidia/flywheel_model.0.safetensors
zmeeks-capitanito-53-v13-mkmlizer: 
Loading 0:   0%|          | 0/363 [00:00<?, ?it/s]
Loading 0:   1%|▏         | 5/363 [00:00<00:11, 30.47it/s]
Loading 0:   4%|▎         | 13/363 [00:00<00:07, 48.82it/s]
Loading 0:   5%|▌         | 19/363 [00:00<00:07, 43.49it/s]
Loading 0:   7%|▋         | 24/363 [00:00<00:08, 42.02it/s]
Loading 0:   9%|▊         | 31/363 [00:00<00:06, 47.91it/s]
Loading 0:  10%|▉         | 36/363 [00:00<00:06, 47.30it/s]
Loading 0:  11%|█▏        | 41/363 [00:00<00:08, 38.03it/s]
Loading 0:  13%|█▎        | 49/363 [00:01<00:06, 46.35it/s]
Loading 0:  15%|█▍        | 54/363 [00:01<00:06, 47.06it/s]
Loading 0:  16%|█▋        | 59/363 [00:01<00:06, 47.60it/s]
Loading 0:  18%|█▊        | 64/363 [00:01<00:10, 27.50it/s]
Loading 0:  20%|█▉        | 71/363 [00:01<00:08, 34.73it/s]
Loading 0:  21%|██        | 76/363 [00:01<00:08, 35.81it/s]
Loading 0:  22%|██▏       | 81/363 [00:02<00:07, 37.59it/s]
Loading 0:  24%|██▎       | 86/363 [00:02<00:07, 39.37it/s]
Loading 0:  25%|██▌       | 91/363 [00:02<00:08, 32.37it/s]
Loading 0:  27%|██▋       | 98/363 [00:02<00:06, 39.57it/s]
Loading 0:  28%|██▊       | 103/363 [00:02<00:06, 40.15it/s]
Loading 0:  30%|███       | 109/363 [00:02<00:05, 44.72it/s]
Loading 0:  31%|███▏      | 114/363 [00:02<00:06, 39.64it/s]
Loading 0:  33%|███▎      | 119/363 [00:02<00:06, 39.70it/s]
Loading 0:  35%|███▍      | 126/363 [00:03<00:05, 44.84it/s]
Loading 0:  36%|███▋      | 132/363 [00:03<00:05, 42.93it/s]
Loading 0:  38%|███▊      | 137/363 [00:03<00:05, 41.32it/s]
Loading 0:  39%|███▉      | 142/363 [00:03<00:06, 31.67it/s]
Loading 0:  40%|████      | 146/363 [00:03<00:06, 32.40it/s]
Loading 0:  41%|████▏     | 150/363 [00:03<00:06, 32.26it/s]
Loading 0:  43%|████▎     | 156/363 [00:03<00:05, 38.26it/s]
Loading 0:  44%|████▍     | 161/363 [00:04<00:05, 39.48it/s]
Loading 0:  46%|████▌     | 166/363 [00:04<00:04, 40.77it/s]
Loading 0:  47%|████▋     | 171/363 [00:04<00:04, 42.02it/s]
Loading 0:  48%|████▊     | 176/363 [00:04<00:05, 36.03it/s]
Loading 0:  50%|█████     | 183/363 [00:04<00:04, 42.73it/s]
Loading 0:  52%|█████▏    | 188/363 [00:04<00:04, 42.13it/s]
Loading 0:  53%|█████▎    | 193/363 [00:04<00:03, 42.93it/s]
Loading 0:  55%|█████▍    | 198/363 [00:04<00:03, 44.68it/s]
Loading 0:  56%|█████▌    | 203/363 [00:05<00:04, 36.41it/s]
Loading 0:  58%|█████▊    | 210/363 [00:05<00:03, 43.18it/s]
Loading 0:  59%|█████▉    | 215/363 [00:05<00:03, 43.28it/s]
Loading 0:  61%|██████    | 220/363 [00:05<00:03, 44.80it/s]
Loading 0:  62%|██████▏   | 225/363 [00:05<00:05, 27.45it/s]
Loading 0:  63%|██████▎   | 230/363 [00:05<00:04, 29.69it/s]
Loading 0:  65%|██████▌   | 237/363 [00:06<00:03, 37.06it/s]
Loading 0:  67%|██████▋   | 242/363 [00:06<00:03, 38.68it/s]
Loading 0:  68%|██████▊   | 247/363 [00:06<00:02, 39.89it/s]
Loading 0:  69%|██████▉   | 252/363 [00:06<00:02, 40.49it/s]
Loading 0:  71%|███████   | 257/363 [00:06<00:03, 34.16it/s]
Loading 0:  73%|███████▎  | 264/363 [00:06<00:02, 40.68it/s]
Loading 0:  74%|███████▍  | 269/363 [00:06<00:02, 40.61it/s]
Loading 0:  75%|███████▌  | 274/363 [00:07<00:02, 40.82it/s]
Loading 0:  77%|███████▋  | 279/363 [00:07<00:02, 41.54it/s]
Loading 0:  78%|███████▊  | 284/363 [00:07<00:02, 34.61it/s]
Loading 0:  80%|████████  | 291/363 [00:07<00:01, 41.49it/s]
Loading 0:  82%|████████▏ | 296/363 [00:07<00:01, 41.02it/s]
Loading 0:  83%|████████▎ | 301/363 [00:07<00:01, 42.19it/s]
Loading 0:  84%|████████▍ | 306/363 [00:08<00:02, 23.27it/s]
Loading 0:  85%|████████▌ | 310/363 [00:08<00:02, 24.44it/s]
Loading 0:  87%|████████▋ | 314/363 [00:08<00:01, 27.04it/s]
Loading 0:  88%|████████▊ | 320/363 [00:08<00:01, 33.15it/s]
Loading 0:  90%|████████▉ | 325/363 [00:08<00:01, 36.34it/s]
Loading 0:  91%|█████████ | 330/363 [00:08<00:00, 33.53it/s]
Loading 0:  93%|█████████▎| 338/363 [00:08<00:00, 41.97it/s]
Loading 0:  94%|█████████▍| 343/363 [00:08<00:00, 43.15it/s]
Loading 0:  96%|█████████▌| 348/363 [00:09<00:00, 37.32it/s]
Loading 0:  98%|█████████▊| 356/363 [00:09<00:00, 45.13it/s]
Loading 0:  99%|█████████▉| 361/363 [00:09<00:00, 45.57it/s]
                                                            

Job zmeeks-capitanito-53-v13-mkmlizer completed after 86.91s with status: succeeded
Stopping job with name zmeeks-capitanito-53-v13-mkmlizer
Pipeline stage MKMLizer completed in 87.49s
run pipeline stage %s
Running pipeline stage MKMLTemplater
Pipeline stage MKMLTemplater completed in 0.17s
run pipeline stage %s
Running pipeline stage MKMLDeployer
Creating inference service zmeeks-capitanito-53-v13
Waiting for inference service zmeeks-capitanito-53-v13 to be ready
Failed to get response for submission blend_hunen_2025-06-23: HTTPConnectionPool(host='guanaco-model-mesh.k2.chaiverse.com', port=80): Read timed out. (read timeout=12.0)
Inference service zmeeks-capitanito-53-v13 ready after 311.2971646785736s
Pipeline stage MKMLDeployer completed in 311.85s
run pipeline stage %s
Running pipeline stage StressChecker
Received healthy response to inference request in 17.252584218978882s
Received healthy response to inference request in 9.05972957611084s
Received healthy response to inference request in 9.161383867263794s
Received healthy response to inference request in 5.82740592956543s
Received healthy response to inference request in 4.000182390213013s
5 requests
0 failed requests
5th percentile: 4.365627098083496
10th percentile: 4.731071805953979
20th percentile: 5.4619612216949465
Failed to get response for submission chaiml-nis-8b-v1-llama3_64493_v3: HTTPConnectionPool(host='chaiml-nis-8b-v1-llama3-64493-v3-predictor.tenant-chaiml-guanaco.k.chaiverse.com', port=80): Read timed out. (read timeout=12.0)
30th percentile: 6.473870658874512
40th percentile: 7.766800117492676
50th percentile: 9.05972957611084
60th percentile: 9.100391292572022
70th percentile: 9.141053009033204
80th percentile: 10.779623937606813
90th percentile: 14.016104078292848
95th percentile: 15.634344148635863
99th percentile: 16.92893620491028
mean time: 9.060257196426392
%s, retrying in %s seconds...
Received healthy response to inference request in 1.7159137725830078s
Received healthy response to inference request in 1.430255651473999s
Failed to get response for submission chaiml-nis-8b-v1-llama3_64493_v3: HTTPConnectionPool(host='chaiml-nis-8b-v1-llama3-64493-v3-predictor.tenant-chaiml-guanaco.k.chaiverse.com', port=80): Read timed out. (read timeout=12.0)
HTTPConnectionPool(host='guanaco-submitter.guanaco-backend.k2.chaiverse.com', port=80): Read timed out. (read timeout=20)
Received unhealthy response to inference request!
Received healthy response to inference request in 6.666293382644653s
Failed to get response for submission chaiml-nis-8b-v1-llama3_64493_v3: HTTPConnectionPool(host='chaiml-nis-8b-v1-llama3-64493-v3-predictor.tenant-chaiml-guanaco.k.chaiverse.com', port=80): Read timed out. (read timeout=12.0)
Received healthy response to inference request in 2.1482415199279785s
5 requests
1 failed requests
5th percentile: 1.4873872756958009
10th percentile: 1.5445188999176025
20th percentile: 1.658782148361206
30th percentile: 1.802379322052002
40th percentile: 1.9753104209899903
50th percentile: 2.1482415199279785
60th percentile: 3.9554622650146483
70th percentile: 5.762683010101318
80th percentile: 9.3666494846344
90th percentile: 14.767361688613892
95th percentile: 17.467717790603636
99th percentile: 19.628002672195436
mean time: 6.425755643844605
%s, retrying in %s seconds...
Received healthy response to inference request in 1.4508705139160156s
Received healthy response to inference request in 11.678948640823364s
Received healthy response to inference request in 1.750986099243164s
Received healthy response to inference request in 1.6740343570709229s
Received healthy response to inference request in 1.6552047729492188s
5 requests
0 failed requests
5th percentile: 1.4917373657226562
10th percentile: 1.5326042175292969
20th percentile: 1.6143379211425781
30th percentile: 1.6589706897735597
40th percentile: 1.6665025234222413
50th percentile: 1.6740343570709229
60th percentile: 1.7048150539398192
70th percentile: 1.7355957508087159
80th percentile: 3.736578607559206
90th percentile: 7.707763624191285
95th percentile: 9.693356132507322
99th percentile: 11.281830139160157
mean time: 3.642008876800537
Pipeline stage StressChecker completed in 99.65s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyTriggerPipeline
run_pipeline:run_in_cloud %s
starting trigger_guanaco_pipeline args=%s
triggered trigger_guanaco_pipeline args=%s
Pipeline stage OfflineFamilyFriendlyTriggerPipeline completed in 0.77s
run pipeline stage %s
Running pipeline stage TriggerMKMLProfilingPipeline
run_pipeline:run_in_cloud %s
starting trigger_guanaco_pipeline args=%s
triggered trigger_guanaco_pipeline args=%s
Pipeline stage TriggerMKMLProfilingPipeline completed in 0.78s
Shutdown handler de-registered
zmeeks-capitanito-53_v13 status is now deployed due to DeploymentManager action
Shutdown handler registered
run pipeline %s
run pipeline stage %s
Running pipeline stage MKMLProfilerDeleter
Skipping teardown as no inference service was successfully deployed
Pipeline stage MKMLProfilerDeleter completed in 0.14s
run pipeline stage %s
Running pipeline stage MKMLProfilerTemplater
Pipeline stage MKMLProfilerTemplater completed in 0.12s
run pipeline stage %s
Running pipeline stage MKMLProfilerDeployer
Creating inference service zmeeks-capitanito-53-v13-profiler
Waiting for inference service zmeeks-capitanito-53-v13-profiler to be ready
Inference service zmeeks-capitanito-53-v13-profiler ready after 303.64046716690063s
Pipeline stage MKMLProfilerDeployer completed in 304.47s
run pipeline stage %s
Running pipeline stage MKMLProfilerRunner
kubectl cp /code/guanaco/guanaco_inference_services/src/inference_scripts tenant-chaiml-guanaco/zmeeks-capitanito-53-v13-profiler-predictor-00001-deploymetlrx7:/code/chaiverse_profiler_1752767690 --namespace tenant-chaiml-guanaco
kubectl exec -it zmeeks-capitanito-53-v13-profiler-predictor-00001-deploymetlrx7 --namespace tenant-chaiml-guanaco -- sh -c 'cd /code/chaiverse_profiler_1752767690 && python profiles.py profile --best_of_n 8 --auto_batch 5 --batches 1,5,10,15,20,25,30,35,40,45,50,55,60,65,70,75,80,85,90,95,100,105,110,115,120,125,130,135,140,145,150,155,160,165,170,175,180,185,190,195 --samples 200 --input_tokens 1024 --output_tokens 64 --summary /code/chaiverse_profiler_1752767690/summary.json'
kubectl exec -it zmeeks-capitanito-53-v13-profiler-predictor-00001-deploymetlrx7 --namespace tenant-chaiml-guanaco -- bash -c 'cat /code/chaiverse_profiler_1752767690/summary.json'
Pipeline stage MKMLProfilerRunner completed in 1132.65s
run pipeline stage %s
Running pipeline stage MKMLProfilerDeleter
Checking if service zmeeks-capitanito-53-v13-profiler is running
Tearing down inference service zmeeks-capitanito-53-v13-profiler
Service zmeeks-capitanito-53-v13-profiler has been torndown
Pipeline stage MKMLProfilerDeleter completed in 4.55s
Shutdown handler de-registered
Shutdown handler registered
run pipeline %s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyScorer
Evaluating %s Family Friendly Score with %s threads
%s, retrying in %s seconds...
Evaluating %s Family Friendly Score with %s threads
%s, retrying in %s seconds...
Evaluating %s Family Friendly Score with %s threads
clean up pipeline due to error=DeploymentChecksError('None: None')
Shutdown handler de-registered
Shutdown handler registered
run pipeline %s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyScorer
Evaluating %s Family Friendly Score with %s threads
Pipeline stage OfflineFamilyFriendlyScorer completed in 2685.45s
Shutdown handler de-registered
zmeeks-capitanito-53_v13 status is now inactive due to auto deactivation removed underperforming models
zmeeks-capitanito-53_v13 status is now torndown due to DeploymentManager action
zmeeks-capitanito-53_v13 status is now torndown due to DeploymentManager action