developer_uid: zmeeks
submission_id: zmeeks-capitanito-36_v3
model_name: capitanito_t10__z36__ppmmp
model_group: zmeeks/capitanito__36
status: torndown
timestamp: 2025-07-14T00:03:47+00:00
num_battles: 9465
num_wins: 4452
celo_rating: 1258.48
family_friendly_score: 0.5862
family_friendly_standard_error: 0.006965192890365636
submission_type: basic
model_repo: zmeeks/capitanito__36
model_architecture: MistralForCausalLM
model_num_parameters: 12772070400.0
best_of: 8
max_input_tokens: 1024
max_output_tokens: 64
reward_model: default
latencies: [{'batch_size': 1, 'throughput': 0.5933727265975501, 'latency_mean': 1.6851833581924438, 'latency_p50': 1.6933470964431763, 'latency_p90': 1.8473134994506835}, {'batch_size': 3, 'throughput': 1.0669280425474512, 'latency_mean': 2.808384039402008, 'latency_p50': 2.805607557296753, 'latency_p90': 3.089896488189697}, {'batch_size': 5, 'throughput': 1.2775391950976802, 'latency_mean': 3.8909519398212433, 'latency_p50': 3.897659182548523, 'latency_p90': 4.345761609077454}, {'batch_size': 6, 'throughput': 1.3404529085101446, 'latency_mean': 4.449920868873596, 'latency_p50': 4.46831488609314, 'latency_p90': 4.976318454742431}, {'batch_size': 8, 'throughput': 1.3974720982299036, 'latency_mean': 5.689772678613663, 'latency_p50': 5.697243690490723, 'latency_p90': 6.334841156005859}, {'batch_size': 10, 'throughput': 1.426400217391579, 'latency_mean': 6.9586252617836, 'latency_p50': 7.017991423606873, 'latency_p90': 7.861917519569397}]
gpu_counts: {'NVIDIA RTX A5000': 1}
display_name: capitanito_t10__z36__ppmmp
is_internal_developer: False
language_model: zmeeks/capitanito__36
model_size: 13B
ranking_group: single
throughput_3p7s: 1.26
us_pacific_date: 2025-07-13
win_ratio: 0.470364500792393
generation_params: {'temperature': 1.0, 'top_p': 1.0, 'min_p': 0.0, 'top_k': 25, 'presence_penalty': 0.0, 'frequency_penalty': 0.0, 'stopping_words': ['\n'], 'max_input_tokens': 1024, 'best_of': 8, 'max_output_tokens': 64}
formatter: {'memory_template': "{bot_name}'s Persona: {memory}\n####\n", 'prompt_template': '{prompt}\n<START>\n', 'bot_template': '{bot_name}: {message}\n', 'user_template': '{user_name}: {message}\n', 'response_template': '{bot_name}:', 'truncate_by_message': True}
Resubmit model
Shutdown handler not registered because Python interpreter is not running in the main thread
run pipeline %s
run pipeline stage %s
Running pipeline stage MKMLizer
Starting job with name zmeeks-capitanito-36-v3-mkmlizer
Waiting for job on zmeeks-capitanito-36-v3-mkmlizer to finish
zmeeks-capitanito-36-v3-mkmlizer: ╔═════════════════════════════════════════════════════════════════════╗
zmeeks-capitanito-36-v3-mkmlizer: ║                                                                     ║
zmeeks-capitanito-36-v3-mkmlizer: ║   ██████   ██████  █████   ████  ████                               ║
zmeeks-capitanito-36-v3-mkmlizer: ║  ░░██████ ██████  ░░███   ███░  ░░███                               ║
zmeeks-capitanito-36-v3-mkmlizer: ║   ░███░█████░███   ░███  ███     ░███                               ║
zmeeks-capitanito-36-v3-mkmlizer: ║   ░███░░███ ░███   ░███████      ░███                               ║
zmeeks-capitanito-36-v3-mkmlizer: ║   ░███ ░░░  ░███   ░███░░███     ░███                               ║
zmeeks-capitanito-36-v3-mkmlizer: ║   ░███      ░███   ░███ ░░███    ░███                               ║
zmeeks-capitanito-36-v3-mkmlizer: ║   █████     █████  █████ ░░████  █████                              ║
zmeeks-capitanito-36-v3-mkmlizer: ║  ░░░░░     ░░░░░  ░░░░░   ░░░░  ░░░░░                               ║
zmeeks-capitanito-36-v3-mkmlizer: ║                                                                     ║
zmeeks-capitanito-36-v3-mkmlizer: ║  Version: 0.29.15                                                   ║
zmeeks-capitanito-36-v3-mkmlizer: ║  Features: FLYWHEEL, CUDA                                           ║
zmeeks-capitanito-36-v3-mkmlizer: ║  Copyright 2023-2025 MK ONE TECHNOLOGIES Inc.                       ║
zmeeks-capitanito-36-v3-mkmlizer: ║  https://mk1.ai                                                     ║
zmeeks-capitanito-36-v3-mkmlizer: ║                                                                     ║
zmeeks-capitanito-36-v3-mkmlizer: ║  The license key for the current software has been verified as      ║
zmeeks-capitanito-36-v3-mkmlizer: ║  belonging to:                                                      ║
zmeeks-capitanito-36-v3-mkmlizer: ║                                                                     ║
zmeeks-capitanito-36-v3-mkmlizer: ║  Chai Research Corp.                                                ║
zmeeks-capitanito-36-v3-mkmlizer: ║  Account ID: 7997a29f-0ceb-4cc7-9adf-840c57b4ae6f                   ║
zmeeks-capitanito-36-v3-mkmlizer: ║  Expiration: 2028-03-31 23:59:59                                    ║
zmeeks-capitanito-36-v3-mkmlizer: ║                                                                     ║
zmeeks-capitanito-36-v3-mkmlizer: ╚═════════════════════════════════════════════════════════════════════╝
zmeeks-capitanito-36-v3-mkmlizer: Downloaded to shared memory in 36.114s
zmeeks-capitanito-36-v3-mkmlizer: Checking if zmeeks/capitanito__36 already exists in ChaiML
zmeeks-capitanito-36-v3-mkmlizer: quantizing model to /dev/shm/model_cache, profile:s0, folder:/tmp/tmp0v0pc7ut, device:0
zmeeks-capitanito-36-v3-mkmlizer: Saving flywheel model at /dev/shm/model_cache
zmeeks-capitanito-36-v3-mkmlizer: quantized model in 30.296s
zmeeks-capitanito-36-v3-mkmlizer: Processed model zmeeks/capitanito__36 in 66.491s
zmeeks-capitanito-36-v3-mkmlizer: creating bucket guanaco-mkml-models
zmeeks-capitanito-36-v3-mkmlizer: Bucket 's3://guanaco-mkml-models/' created
zmeeks-capitanito-36-v3-mkmlizer: uploading /dev/shm/model_cache to s3://guanaco-mkml-models/zmeeks-capitanito-36-v3/nvidia
zmeeks-capitanito-36-v3-mkmlizer: cp /dev/shm/model_cache/config.json s3://guanaco-mkml-models/zmeeks-capitanito-36-v3/nvidia/config.json
zmeeks-capitanito-36-v3-mkmlizer: cp /dev/shm/model_cache/special_tokens_map.json s3://guanaco-mkml-models/zmeeks-capitanito-36-v3/nvidia/special_tokens_map.json
zmeeks-capitanito-36-v3-mkmlizer: cp /dev/shm/model_cache/tokenizer_config.json s3://guanaco-mkml-models/zmeeks-capitanito-36-v3/nvidia/tokenizer_config.json
zmeeks-capitanito-36-v3-mkmlizer: cp /dev/shm/model_cache/tokenizer.json s3://guanaco-mkml-models/zmeeks-capitanito-36-v3/nvidia/tokenizer.json
zmeeks-capitanito-36-v3-mkmlizer: cp /dev/shm/model_cache/flywheel_model.0.safetensors s3://guanaco-mkml-models/zmeeks-capitanito-36-v3/nvidia/flywheel_model.0.safetensors
zmeeks-capitanito-36-v3-mkmlizer: 
Loading 0:   0%|          | 0/363 [00:00<?, ?it/s]
Loading 0:   1%|▏         | 5/363 [00:00<00:11, 30.86it/s]
Loading 0:   4%|▎         | 13/363 [00:00<00:07, 49.55it/s]
Loading 0:   5%|▌         | 19/363 [00:00<00:08, 42.90it/s]
Loading 0:   7%|▋         | 24/363 [00:00<00:08, 41.51it/s]
Loading 0:   9%|▊         | 31/363 [00:00<00:07, 47.18it/s]
Loading 0:  10%|█         | 37/363 [00:00<00:07, 44.02it/s]
Loading 0:  12%|█▏        | 42/363 [00:00<00:07, 42.78it/s]
Loading 0:  13%|█▎        | 49/363 [00:01<00:06, 47.68it/s]
Loading 0:  15%|█▌        | 55/363 [00:01<00:06, 44.62it/s]
Loading 0:  17%|█▋        | 61/363 [00:01<00:09, 32.71it/s]
Loading 0:  18%|█▊        | 65/363 [00:01<00:09, 32.18it/s]
Loading 0:  20%|█▉        | 72/363 [00:01<00:07, 38.36it/s]
Loading 0:  21%|██        | 77/363 [00:01<00:07, 40.70it/s]
Loading 0:  23%|██▎       | 82/363 [00:02<00:08, 35.11it/s]
Loading 0:  25%|██▍       | 89/363 [00:02<00:06, 41.78it/s]
Loading 0:  26%|██▌       | 94/363 [00:02<00:06, 41.11it/s]
Loading 0:  27%|██▋       | 99/363 [00:02<00:06, 41.09it/s]
Loading 0:  29%|██▊       | 104/363 [00:02<00:06, 41.40it/s]
Loading 0:  30%|███       | 109/363 [00:02<00:05, 43.51it/s]
Loading 0:  31%|███▏      | 114/363 [00:02<00:06, 37.92it/s]
Loading 0:  33%|███▎      | 119/363 [00:02<00:06, 37.69it/s]
Loading 0:  34%|███▍      | 125/363 [00:03<00:05, 41.81it/s]
Loading 0:  36%|███▌      | 130/363 [00:03<00:05, 41.49it/s]
Loading 0:  37%|███▋      | 135/363 [00:03<00:05, 41.43it/s]
Loading 0:  39%|███▊      | 140/363 [00:03<00:05, 43.38it/s]
Loading 0:  40%|███▉      | 145/363 [00:03<00:08, 26.37it/s]
Loading 0:  41%|████      | 149/363 [00:03<00:07, 27.45it/s]
Loading 0:  43%|████▎     | 156/363 [00:04<00:05, 35.25it/s]
Loading 0:  44%|████▍     | 161/363 [00:04<00:05, 37.18it/s]
Loading 0:  46%|████▌     | 166/363 [00:04<00:05, 39.04it/s]
Loading 0:  47%|████▋     | 172/363 [00:04<00:04, 38.84it/s]
Loading 0:  49%|████▉     | 177/363 [00:04<00:04, 38.99it/s]
Loading 0:  50%|█████     | 183/363 [00:04<00:04, 42.96it/s]
Loading 0:  52%|█████▏    | 188/363 [00:04<00:04, 43.45it/s]
Loading 0:  53%|█████▎    | 193/363 [00:04<00:03, 43.57it/s]
Loading 0:  55%|█████▍    | 199/363 [00:05<00:03, 41.04it/s]
Loading 0:  56%|█████▌    | 204/363 [00:05<00:03, 40.33it/s]
Loading 0:  58%|█████▊    | 210/363 [00:05<00:03, 44.52it/s]
Loading 0:  59%|█████▉    | 215/363 [00:05<00:03, 44.02it/s]
Loading 0:  61%|██████    | 220/363 [00:05<00:03, 45.26it/s]
Loading 0:  62%|██████▏   | 225/363 [00:05<00:05, 27.55it/s]
Loading 0:  63%|██████▎   | 230/363 [00:05<00:04, 29.86it/s]
Loading 0:  65%|██████▌   | 237/363 [00:06<00:03, 37.08it/s]
Loading 0:  67%|██████▋   | 242/363 [00:06<00:03, 38.58it/s]
Loading 0:  68%|██████▊   | 247/363 [00:06<00:02, 39.77it/s]
Loading 0:  70%|██████▉   | 253/363 [00:06<00:02, 38.72it/s]
Loading 0:  71%|███████   | 258/363 [00:06<00:02, 38.82it/s]
Loading 0:  73%|███████▎  | 264/363 [00:06<00:02, 42.86it/s]
Loading 0:  74%|███████▍  | 269/363 [00:06<00:02, 43.21it/s]
Loading 0:  75%|███████▌  | 274/363 [00:06<00:02, 43.45it/s]
Loading 0:  77%|███████▋  | 280/363 [00:07<00:02, 41.33it/s]
Loading 0:  79%|███████▊  | 285/363 [00:07<00:01, 40.25it/s]
Loading 0:  80%|████████  | 291/363 [00:07<00:01, 43.30it/s]
Loading 0:  82%|████████▏ | 296/363 [00:07<00:01, 43.42it/s]
Loading 0:  83%|████████▎ | 301/363 [00:07<00:01, 44.25it/s]
Loading 0:  84%|████████▍ | 306/363 [00:08<00:02, 23.50it/s]
Loading 0:  85%|████████▌ | 310/363 [00:08<00:02, 24.80it/s]
Loading 0:  87%|████████▋ | 314/363 [00:08<00:01, 26.37it/s]
Loading 0:  88%|████████▊ | 320/363 [00:08<00:01, 32.02it/s]
Loading 0:  90%|████████▉ | 326/363 [00:08<00:01, 33.66it/s]
Loading 0:  91%|█████████ | 330/363 [00:08<00:00, 33.30it/s]
Loading 0:  93%|█████████▎| 337/363 [00:08<00:00, 40.91it/s]
Loading 0:  94%|█████████▍| 342/363 [00:08<00:00, 41.70it/s]
Loading 0:  96%|█████████▌| 347/363 [00:09<00:00, 43.06it/s]
Loading 0:  97%|█████████▋| 353/363 [00:09<00:00, 40.94it/s]
Loading 0:  99%|█████████▊| 358/363 [00:09<00:00, 40.69it/s]
                                                            

Job zmeeks-capitanito-36-v3-mkmlizer completed after 94.56s with status: succeeded
Stopping job with name zmeeks-capitanito-36-v3-mkmlizer
Pipeline stage MKMLizer completed in 95.23s
run pipeline stage %s
Running pipeline stage MKMLTemplater
Pipeline stage MKMLTemplater completed in 0.17s
run pipeline stage %s
Running pipeline stage MKMLDeployer
Creating inference service zmeeks-capitanito-36-v3
Waiting for inference service zmeeks-capitanito-36-v3 to be ready
Unable to record family friendly update due to error: HTTPConnectionPool(host='chaiml-nemo-guard-merged-v2-predictor.tenant-chaiml-guanaco.k.chaiverse.com', port=80): Max retries exceeded with url: /v1/models/GPT-J-6B-lit-v2:predict (Caused by ConnectTimeoutError(<urllib3.connection.HTTPConnection object at 0x77016823f050>, 'Connection to chaiml-nemo-guard-merged-v2-predictor.tenant-chaiml-guanaco.k.chaiverse.com timed out. (connect timeout=12.0)'))
Inference service zmeeks-capitanito-36-v3 ready after 220.78587341308594s
Pipeline stage MKMLDeployer completed in 221.22s
run pipeline stage %s
Running pipeline stage StressChecker
HTTPConnectionPool(host='guanaco-submitter.guanaco-backend.k2.chaiverse.com', port=80): Read timed out. (read timeout=20)
Received unhealthy response to inference request!
Received healthy response to inference request in 2.0665292739868164s
Received healthy response to inference request in 2.0830929279327393s
Received healthy response to inference request in 1.7194068431854248s
Received healthy response to inference request in 1.7393360137939453s
5 requests
1 failed requests
5th percentile: 1.7233926773071289
10th percentile: 1.727378511428833
20th percentile: 1.7353501796722413
30th percentile: 1.8047746658325194
40th percentile: 1.935651969909668
50th percentile: 2.0665292739868164
60th percentile: 2.0731547355651854
70th percentile: 2.0797801971435548
80th percentile: 5.702853155136111
90th percentile: 12.942373609542848
95th percentile: 16.562133836746213
99th percentile: 19.45794201850891
mean time: 5.558051824569702
%s, retrying in %s seconds...
Received healthy response to inference request in 2.4054410457611084s
Received healthy response to inference request in 1.7422840595245361s
Received healthy response to inference request in 1.5494332313537598s
Received healthy response to inference request in 1.3623640537261963s
Received healthy response to inference request in 1.5797388553619385s
5 requests
0 failed requests
5th percentile: 1.399777889251709
10th percentile: 1.4371917247772217
20th percentile: 1.512019395828247
30th percentile: 1.5554943561553956
40th percentile: 1.567616605758667
50th percentile: 1.5797388553619385
60th percentile: 1.6447569370269775
70th percentile: 1.7097750186920166
80th percentile: 1.8749154567718507
90th percentile: 2.1401782512664793
95th percentile: 2.272809648513794
99th percentile: 2.3789147663116457
mean time: 1.7278522491455077
Pipeline stage StressChecker completed in 38.95s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyTriggerPipeline
run_pipeline:run_in_cloud %s
starting trigger_guanaco_pipeline args=%s
triggered trigger_guanaco_pipeline args=%s
Pipeline stage OfflineFamilyFriendlyTriggerPipeline completed in 0.77s
run pipeline stage %s
Running pipeline stage TriggerMKMLProfilingPipeline
run_pipeline:run_in_cloud %s
starting trigger_guanaco_pipeline args=%s
triggered trigger_guanaco_pipeline args=%s
Pipeline stage TriggerMKMLProfilingPipeline completed in 0.77s
Shutdown handler de-registered
zmeeks-capitanito-36_v3 status is now deployed due to DeploymentManager action
Shutdown handler registered
run pipeline %s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyScorer
Evaluating %s Family Friendly Score with %s threads
Pipeline stage OfflineFamilyFriendlyScorer completed in 2899.71s
Shutdown handler de-registered
zmeeks-capitanito-36_v3 status is now inactive due to auto deactivation removed underperforming models
zmeeks-capitanito-36_v3 status is now torndown due to DeploymentManager action