zmeeks-capitanito-49-4140

developer_uid: zmeeks

submission_id: zmeeks-capitanito-49-4140_v9

model_name: capitanito_t10__49-4140__ppmmp

model_group: zmeeks/capitanito__49-41

status: torndown

timestamp: 2025-07-15T19:53:10+00:00

num_battles: 7739

num_wins: 3763

celo_rating: 1263.21

family_friendly_score: 0.5734

family_friendly_standard_error: 0.006994461237293406

submission_type: basic

model_repo: zmeeks/capitanito__49-4140

model_architecture: MistralForCausalLM

model_num_parameters: 12772070400.0

best_of: 8

max_input_tokens: 1024

max_output_tokens: 64

reward_model: default

latencies: [{'batch_size': 1, 'throughput': 0.5999092899629496, 'latency_mean': 1.66676717877388, 'latency_p50': 1.6690659523010254, 'latency_p90': 1.822678565979004}, {'batch_size': 3, 'throughput': 1.0713182751136296, 'latency_mean': 2.795667154788971, 'latency_p50': 2.798408031463623, 'latency_p90': 3.0755108833312987}, {'batch_size': 5, 'throughput': 1.3032227893928965, 'latency_mean': 3.821730918884277, 'latency_p50': 3.832084894180298, 'latency_p90': 4.29605655670166}, {'batch_size': 6, 'throughput': 1.356223033137102, 'latency_mean': 4.392674289941787, 'latency_p50': 4.3909255266189575, 'latency_p90': 5.044388008117676}, {'batch_size': 8, 'throughput': 1.4111735964749634, 'latency_mean': 5.628281025886536, 'latency_p50': 5.6254225969314575, 'latency_p90': 6.282089471817017}, {'batch_size': 10, 'throughput': 1.4676773562766257, 'latency_mean': 6.776477584838867, 'latency_p50': 6.7507102489471436, 'latency_p90': 7.722760200500488}]

gpu_counts: {'NVIDIA RTX A5000': 1}

display_name: capitanito_t10__49-4140__ppmmp

is_internal_developer: False

language_model: zmeeks/capitanito__49-4140

model_size: 13B

ranking_group: single

throughput_3p7s: 1.29

us_pacific_date: 2025-07-15

win_ratio: 0.48623853211009177

generation_params: {'temperature': 1.0, 'top_p': 1.0, 'min_p': 0.0, 'top_k': 25, 'presence_penalty': 0.0, 'frequency_penalty': 0.0, 'stopping_words': ['\n'], 'max_input_tokens': 1024, 'best_of': 8, 'max_output_tokens': 64}

formatter: {'memory_template': "{bot_name}'s Persona: {memory}\n####\n", 'prompt_template': '{prompt}\n<START>\n', 'bot_template': '{bot_name}: {message}\n', 'user_template': '{user_name}: {message}\n', 'response_template': '{bot_name}:', 'truncate_by_message': False}

Resubmit model

Shutdown handler not registered because Python interpreter is not running in the main thread
run pipeline %s
run pipeline stage %s
Running pipeline stage MKMLizer
Starting job with name zmeeks-capitanito-49-4140-v9-mkmlizer
Waiting for job on zmeeks-capitanito-49-4140-v9-mkmlizer to finish
zmeeks-capitanito-49-4140-v9-mkmlizer: ╔═════════════════════════════════════════════════════════════════════╗
zmeeks-capitanito-49-4140-v9-mkmlizer: ║                                                                     ║
zmeeks-capitanito-49-4140-v9-mkmlizer: ║   ██████   ██████  █████   ████  ████                               ║
zmeeks-capitanito-49-4140-v9-mkmlizer: ║  ░░██████ ██████  ░░███   ███░  ░░███                               ║
zmeeks-capitanito-49-4140-v9-mkmlizer: ║   ░███░█████░███   ░███  ███     ░███                               ║
zmeeks-capitanito-49-4140-v9-mkmlizer: ║   ░███░░███ ░███   ░███████      ░███                               ║
zmeeks-capitanito-49-4140-v9-mkmlizer: ║   ░███ ░░░  ░███   ░███░░███     ░███                               ║
zmeeks-capitanito-49-4140-v9-mkmlizer: ║   ░███      ░███   ░███ ░░███    ░███                               ║
zmeeks-capitanito-49-4140-v9-mkmlizer: ║   █████     █████  █████ ░░████  █████                              ║
zmeeks-capitanito-49-4140-v9-mkmlizer: ║  ░░░░░     ░░░░░  ░░░░░   ░░░░  ░░░░░                               ║
zmeeks-capitanito-49-4140-v9-mkmlizer: ║                                                                     ║
zmeeks-capitanito-49-4140-v9-mkmlizer: ║  Version: 0.29.15                                                   ║
zmeeks-capitanito-49-4140-v9-mkmlizer: ║  Features: FLYWHEEL, CUDA                                           ║
zmeeks-capitanito-49-4140-v9-mkmlizer: ║  Copyright 2023-2025 MK ONE TECHNOLOGIES Inc.                       ║
zmeeks-capitanito-49-4140-v9-mkmlizer: ║  https://mk1.ai                                                     ║
zmeeks-capitanito-49-4140-v9-mkmlizer: ║                                                                     ║
zmeeks-capitanito-49-4140-v9-mkmlizer: ║  The license key for the current software has been verified as      ║
zmeeks-capitanito-49-4140-v9-mkmlizer: ║  belonging to:                                                      ║
zmeeks-capitanito-49-4140-v9-mkmlizer: ║                                                                     ║
zmeeks-capitanito-49-4140-v9-mkmlizer: ║  Chai Research Corp.                                                ║
zmeeks-capitanito-49-4140-v9-mkmlizer: ║  Account ID: 7997a29f-0ceb-4cc7-9adf-840c57b4ae6f                   ║
zmeeks-capitanito-49-4140-v9-mkmlizer: ║  Expiration: 2028-03-31 23:59:59                                    ║
zmeeks-capitanito-49-4140-v9-mkmlizer: ║                                                                     ║
zmeeks-capitanito-49-4140-v9-mkmlizer: ╚═════════════════════════════════════════════════════════════════════╝
zmeeks-capitanito-49-4140-v9-mkmlizer: Downloaded to shared memory in 31.124s
zmeeks-capitanito-49-4140-v9-mkmlizer: Checking if zmeeks/capitanito__49-4140 already exists in ChaiML
zmeeks-capitanito-49-4140-v9-mkmlizer: quantizing model to /dev/shm/model_cache, profile:s0, folder:/tmp/tmpo70z9z7a, device:0
zmeeks-capitanito-49-4140-v9-mkmlizer: Saving flywheel model at /dev/shm/model_cache
zmeeks-capitanito-49-4140-v9-mkmlizer: quantized model in 30.850s
zmeeks-capitanito-49-4140-v9-mkmlizer: Processed model zmeeks/capitanito__49-4140 in 62.060s
zmeeks-capitanito-49-4140-v9-mkmlizer: creating bucket guanaco-mkml-models
zmeeks-capitanito-49-4140-v9-mkmlizer: Bucket 's3://guanaco-mkml-models/' created
zmeeks-capitanito-49-4140-v9-mkmlizer: uploading /dev/shm/model_cache to s3://guanaco-mkml-models/zmeeks-capitanito-49-4140-v9/nvidia
zmeeks-capitanito-49-4140-v9-mkmlizer: cp /dev/shm/model_cache/config.json s3://guanaco-mkml-models/zmeeks-capitanito-49-4140-v9/nvidia/config.json
zmeeks-capitanito-49-4140-v9-mkmlizer: cp /dev/shm/model_cache/special_tokens_map.json s3://guanaco-mkml-models/zmeeks-capitanito-49-4140-v9/nvidia/special_tokens_map.json
zmeeks-capitanito-49-4140-v9-mkmlizer: cp /dev/shm/model_cache/tokenizer_config.json s3://guanaco-mkml-models/zmeeks-capitanito-49-4140-v9/nvidia/tokenizer_config.json
zmeeks-capitanito-49-4140-v9-mkmlizer: cp /dev/shm/model_cache/tokenizer.json s3://guanaco-mkml-models/zmeeks-capitanito-49-4140-v9/nvidia/tokenizer.json
zmeeks-capitanito-49-4140-v9-mkmlizer: cp /dev/shm/model_cache/flywheel_model.0.safetensors s3://guanaco-mkml-models/zmeeks-capitanito-49-4140-v9/nvidia/flywheel_model.0.safetensors
zmeeks-capitanito-49-4140-v9-mkmlizer: 
Loading 0:   0%|          | 0/363 [00:00<?, ?it/s]
Loading 0:   1%|▏         | 5/363 [00:00<00:12, 29.39it/s]
Loading 0:   3%|▎         | 12/363 [00:00<00:07, 47.00it/s]
Loading 0:   5%|▍         | 18/363 [00:00<00:07, 47.59it/s]
Loading 0:   7%|▋         | 24/363 [00:00<00:08, 39.27it/s]
Loading 0:   9%|▊         | 31/363 [00:00<00:07, 44.93it/s]
Loading 0:  10%|▉         | 36/363 [00:00<00:07, 45.53it/s]
Loading 0:  11%|█▏        | 41/363 [00:01<00:08, 37.01it/s]
Loading 0:  13%|█▎        | 48/363 [00:01<00:07, 44.31it/s]
Loading 0:  15%|█▍        | 53/363 [00:01<00:07, 43.95it/s]
Loading 0:  16%|█▌        | 58/363 [00:01<00:06, 45.38it/s]
Loading 0:  17%|█▋        | 63/363 [00:01<00:10, 29.16it/s]
Loading 0:  19%|█▊        | 68/363 [00:01<00:08, 33.00it/s]
Loading 0:  20%|██        | 73/363 [00:01<00:09, 30.22it/s]
Loading 0:  22%|██▏       | 80/363 [00:02<00:07, 37.06it/s]
Loading 0:  23%|██▎       | 85/363 [00:02<00:07, 37.08it/s]
Loading 0:  25%|██▍       | 90/363 [00:02<00:07, 37.84it/s]
Loading 0:  26%|██▌       | 95/363 [00:02<00:06, 39.44it/s]
Loading 0:  28%|██▊       | 100/363 [00:02<00:07, 33.33it/s]
Loading 0:  29%|██▉       | 106/363 [00:02<00:06, 37.69it/s]
Loading 0:  31%|███       | 112/363 [00:02<00:06, 41.46it/s]
Loading 0:  32%|███▏      | 117/363 [00:03<00:06, 39.28it/s]
Loading 0:  34%|███▎      | 122/363 [00:03<00:05, 41.08it/s]
Loading 0:  35%|███▍      | 127/363 [00:03<00:06, 34.73it/s]
Loading 0:  37%|███▋      | 134/363 [00:03<00:05, 41.12it/s]
Loading 0:  38%|███▊      | 139/363 [00:03<00:05, 41.12it/s]
Loading 0:  40%|███▉      | 144/363 [00:03<00:08, 26.09it/s]
Loading 0:  41%|████      | 149/363 [00:04<00:07, 28.31it/s]
Loading 0:  43%|████▎     | 156/363 [00:04<00:05, 34.95it/s]
Loading 0:  44%|████▍     | 161/363 [00:04<00:05, 36.12it/s]
Loading 0:  46%|████▌     | 166/363 [00:04<00:05, 37.42it/s]
Loading 0:  47%|████▋     | 171/363 [00:04<00:04, 39.64it/s]
Loading 0:  48%|████▊     | 176/363 [00:04<00:05, 33.66it/s]
Loading 0:  50%|█████     | 183/363 [00:04<00:04, 40.91it/s]
Loading 0:  52%|█████▏    | 188/363 [00:04<00:04, 41.47it/s]
Loading 0:  53%|█████▎    | 193/363 [00:05<00:04, 41.67it/s]
Loading 0:  55%|█████▍    | 198/363 [00:05<00:03, 42.87it/s]
Loading 0:  56%|█████▌    | 203/363 [00:05<00:04, 35.54it/s]
Loading 0:  58%|█████▊    | 210/363 [00:05<00:03, 42.14it/s]
Loading 0:  59%|█████▉    | 215/363 [00:05<00:03, 41.51it/s]
Loading 0:  61%|██████    | 220/363 [00:05<00:03, 42.73it/s]
Loading 0:  62%|██████▏   | 225/363 [00:06<00:05, 27.33it/s]
Loading 0:  63%|██████▎   | 230/363 [00:06<00:04, 29.38it/s]
Loading 0:  65%|██████▍   | 235/363 [00:06<00:03, 33.37it/s]
Loading 0:  66%|██████▌   | 239/363 [00:06<00:03, 32.33it/s]
Loading 0:  68%|██████▊   | 246/363 [00:06<00:02, 39.46it/s]
Loading 0:  69%|██████▉   | 251/363 [00:06<00:02, 40.03it/s]
Loading 0:  71%|███████   | 256/363 [00:06<00:02, 39.43it/s]
Loading 0:  72%|███████▏  | 261/363 [00:06<00:02, 40.68it/s]
Loading 0:  73%|███████▎  | 266/363 [00:07<00:02, 33.48it/s]
Loading 0:  75%|███████▌  | 273/363 [00:07<00:02, 39.91it/s]
Loading 0:  77%|███████▋  | 278/363 [00:07<00:02, 40.09it/s]
Loading 0:  78%|███████▊  | 283/363 [00:07<00:01, 40.17it/s]
Loading 0:  79%|███████▉  | 288/363 [00:07<00:01, 41.51it/s]
Loading 0:  81%|████████  | 293/363 [00:07<00:02, 34.51it/s]
Loading 0:  82%|████████▏ | 299/363 [00:07<00:01, 38.87it/s]
Loading 0:  84%|████████▎ | 304/363 [00:08<00:02, 22.89it/s]
Loading 0:  85%|████████▍ | 308/363 [00:08<00:02, 25.20it/s]
Loading 0:  86%|████████▌ | 312/363 [00:08<00:02, 24.92it/s]
Loading 0:  88%|████████▊ | 320/363 [00:08<00:01, 33.99it/s]
Loading 0:  90%|████████▉ | 325/363 [00:08<00:01, 37.03it/s]
Loading 0:  91%|█████████ | 330/363 [00:09<00:01, 32.93it/s]
Loading 0:  93%|█████████▎| 337/363 [00:09<00:00, 39.74it/s]
Loading 0:  94%|█████████▍| 342/363 [00:09<00:00, 39.72it/s]
Loading 0:  96%|█████████▌| 347/363 [00:09<00:00, 40.33it/s]
Loading 0:  97%|█████████▋| 352/363 [00:09<00:00, 42.01it/s]
Loading 0:  98%|█████████▊| 357/363 [00:09<00:00, 35.38it/s]
                                                            
Job zmeeks-capitanito-49-4140-v9-mkmlizer completed after 96.06s with status: succeeded
Stopping job with name zmeeks-capitanito-49-4140-v9-mkmlizer
Pipeline stage MKMLizer completed in 96.77s
run pipeline stage %s
Running pipeline stage MKMLTemplater
Pipeline stage MKMLTemplater completed in 0.17s
run pipeline stage %s
Running pipeline stage MKMLDeployer
Creating inference service zmeeks-capitanito-49-4140-v9
Waiting for inference service zmeeks-capitanito-49-4140-v9 to be ready
Inference service zmeeks-capitanito-49-4140-v9 ready after 261.3792088031769s
Pipeline stage MKMLDeployer completed in 261.91s
run pipeline stage %s
Running pipeline stage StressChecker
HTTPConnectionPool(host='guanaco-submitter.guanaco-backend.k2.chaiverse.com', port=80): Read timed out. (read timeout=20)
Received unhealthy response to inference request!
Received healthy response to inference request in 2.816237211227417s
Received healthy response to inference request in 1.8428668975830078s
Received healthy response to inference request in 1.6508517265319824s
Received healthy response to inference request in 1.5556762218475342s
5 requests
1 failed requests
5th percentile: 1.5747113227844238
10th percentile: 1.5937464237213135
20th percentile: 1.6318166255950928
30th percentile: 1.6892547607421875
40th percentile: 1.7660608291625977
50th percentile: 1.8428668975830078
60th percentile: 2.2322150230407716
70th percentile: 2.621563148498535
80th percentile: 6.281170701980594
90th percentile: 13.21103768348694
95th percentile: 16.67597117424011
99th percentile: 19.447917966842653
mean time: 5.601307344436646
%s, retrying in %s seconds...
Received healthy response to inference request in 1.6573889255523682s
Received healthy response to inference request in 1.50803804397583s
Received healthy response to inference request in 1.5296976566314697s
Received healthy response to inference request in 1.5378773212432861s
Received healthy response to inference request in 1.995722770690918s
5 requests
0 failed requests
5th percentile: 1.512369966506958
10th percentile: 1.516701889038086
20th percentile: 1.5253657341003417
30th percentile: 1.531333589553833
40th percentile: 1.5346054553985595
50th percentile: 1.5378773212432861
60th percentile: 1.585681962966919
70th percentile: 1.6334866046905518
80th percentile: 1.7250556945800781
90th percentile: 1.860389232635498
95th percentile: 1.928056001663208
99th percentile: 1.982189416885376
mean time: 1.6457449436187743
Pipeline stage StressChecker completed in 39.42s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyTriggerPipeline
run_pipeline:run_in_cloud %s
starting trigger_guanaco_pipeline args=%s
triggered trigger_guanaco_pipeline args=%s
Pipeline stage OfflineFamilyFriendlyTriggerPipeline completed in 1.06s
run pipeline stage %s
Running pipeline stage TriggerMKMLProfilingPipeline
run_pipeline:run_in_cloud %s
starting trigger_guanaco_pipeline args=%s
triggered trigger_guanaco_pipeline args=%s
Pipeline stage TriggerMKMLProfilingPipeline completed in 0.72s
Shutdown handler de-registered
zmeeks-capitanito-49-4140_v9 status is now deployed due to DeploymentManager action
Shutdown handler registered
run pipeline %s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyScorer
Evaluating %s Family Friendly Score with %s threads
Pipeline stage OfflineFamilyFriendlyScorer completed in 5268.63s
Shutdown handler de-registered
zmeeks-capitanito-49-4140_v9 status is now inactive due to auto deactivation removed underperforming models
zmeeks-capitanito-49-4140_v9 status is now torndown due to DeploymentManager action
zmeeks-capitanito-49-4140_v9 status is now torndown due to DeploymentManager action