developer_uid: RandomForest1024
submission_id: albertwang8192-2025-07-07-0_v1
model_name: 2025-07-07_0
model_group: AlbertWang8192/2025-07-0
status: torndown
timestamp: 2025-07-07T19:04:43+00:00
num_battles: 6590
num_wins: 2982
celo_rating: 1257.96
family_friendly_score: 0.0
family_friendly_standard_error: 0.0
submission_type: basic
model_repo: AlbertWang8192/2025-07-07_0
model_architecture: MistralForCausalLM
model_num_parameters: 12772070400.0
best_of: 8
max_input_tokens: 1024
max_output_tokens: 64
reward_model: default
latencies: [{'batch_size': 1, 'throughput': 0.5943648422666357, 'latency_mean': 1.6823519158363343, 'latency_p50': 1.6858010292053223, 'latency_p90': 1.8503978729248047}, {'batch_size': 3, 'throughput': 1.0584018682250103, 'latency_mean': 2.82925714969635, 'latency_p50': 2.825644373893738, 'latency_p90': 3.1369449138641357}, {'batch_size': 5, 'throughput': 1.281893519625356, 'latency_mean': 3.884375808238983, 'latency_p50': 3.8488930463790894, 'latency_p90': 4.359500241279602}, {'batch_size': 6, 'throughput': 1.349487779176868, 'latency_mean': 4.42312989115715, 'latency_p50': 4.38543176651001, 'latency_p90': 4.951997470855713}, {'batch_size': 8, 'throughput': 1.3966994195692644, 'latency_mean': 5.695517897605896, 'latency_p50': 5.716572880744934, 'latency_p90': 6.342927360534668}, {'batch_size': 10, 'throughput': 1.4328086586584412, 'latency_mean': 6.92684693813324, 'latency_p50': 6.9256967306137085, 'latency_p90': 7.7792102813720705}]
gpu_counts: {'NVIDIA RTX A5000': 1}
display_name: 2025-07-07_0
ineligible_reason: num_battles<10000
is_internal_developer: False
language_model: AlbertWang8192/2025-07-07_0
model_size: 13B
ranking_group: single
throughput_3p7s: 1.26
us_pacific_date: 2025-07-07
win_ratio: 0.4525037936267071
generation_params: {'temperature': 0.6, 'top_p': 0.9, 'min_p': 0.025, 'top_k': 60, 'presence_penalty': 0.4, 'frequency_penalty': 0.4, 'stopping_words': ['\n', '<|im_start|>', '<|im_end|>'], 'max_input_tokens': 1024, 'best_of': 8, 'max_output_tokens': 64}
formatter: {'memory_template': "{bot_name}'s Persona: {memory}\n####\n", 'prompt_template': '{prompt}\n<START>\n', 'bot_template': '{bot_name}: {message}\n', 'user_template': '{user_name}: {message}\n', 'response_template': '{bot_name}:', 'truncate_by_message': False}
Resubmit model
Shutdown handler not registered because Python interpreter is not running in the main thread
run pipeline %s
run pipeline stage %s
Running pipeline stage MKMLizer
Starting job with name albertwang8192-2025-07-07-0-v1-mkmlizer
Waiting for job on albertwang8192-2025-07-07-0-v1-mkmlizer to finish
albertwang8192-2025-07-07-0-v1-mkmlizer: ╔═════════════════════════════════════════════════════════════════════╗
albertwang8192-2025-07-07-0-v1-mkmlizer: ║                                                                     ║
albertwang8192-2025-07-07-0-v1-mkmlizer: ║   ██████   ██████  █████   ████  ████                               ║
albertwang8192-2025-07-07-0-v1-mkmlizer: ║  ░░██████ ██████  ░░███   ███░  ░░███                               ║
albertwang8192-2025-07-07-0-v1-mkmlizer: ║   ░███░█████░███   ░███  ███     ░███                               ║
albertwang8192-2025-07-07-0-v1-mkmlizer: ║   ░███░░███ ░███   ░███████      ░███                               ║
albertwang8192-2025-07-07-0-v1-mkmlizer: ║   ░███ ░░░  ░███   ░███░░███     ░███                               ║
albertwang8192-2025-07-07-0-v1-mkmlizer: ║   ░███      ░███   ░███ ░░███    ░███                               ║
albertwang8192-2025-07-07-0-v1-mkmlizer: ║   █████     █████  █████ ░░████  █████                              ║
albertwang8192-2025-07-07-0-v1-mkmlizer: ║  ░░░░░     ░░░░░  ░░░░░   ░░░░  ░░░░░                               ║
albertwang8192-2025-07-07-0-v1-mkmlizer: ║                                                                     ║
albertwang8192-2025-07-07-0-v1-mkmlizer: ║  Version: 0.29.15                                                   ║
albertwang8192-2025-07-07-0-v1-mkmlizer: ║  Features: FLYWHEEL, CUDA                                           ║
albertwang8192-2025-07-07-0-v1-mkmlizer: ║  Copyright 2023-2025 MK ONE TECHNOLOGIES Inc.                       ║
albertwang8192-2025-07-07-0-v1-mkmlizer: ║  https://mk1.ai                                                     ║
albertwang8192-2025-07-07-0-v1-mkmlizer: ║                                                                     ║
albertwang8192-2025-07-07-0-v1-mkmlizer: ║  The license key for the current software has been verified as      ║
albertwang8192-2025-07-07-0-v1-mkmlizer: ║  belonging to:                                                      ║
albertwang8192-2025-07-07-0-v1-mkmlizer: ║                                                                     ║
albertwang8192-2025-07-07-0-v1-mkmlizer: ║  Chai Research Corp.                                                ║
albertwang8192-2025-07-07-0-v1-mkmlizer: ║  Account ID: 7997a29f-0ceb-4cc7-9adf-840c57b4ae6f                   ║
albertwang8192-2025-07-07-0-v1-mkmlizer: ║  Expiration: 2028-03-31 23:59:59                                    ║
albertwang8192-2025-07-07-0-v1-mkmlizer: ║                                                                     ║
albertwang8192-2025-07-07-0-v1-mkmlizer: ╚═════════════════════════════════════════════════════════════════════╝
albertwang8192-2025-07-07-0-v1-mkmlizer: Downloaded to shared memory in 47.094s
albertwang8192-2025-07-07-0-v1-mkmlizer: Checking if AlbertWang8192/2025-07-07_0 already exists in ChaiML
albertwang8192-2025-07-07-0-v1-mkmlizer: Creating repo ChaiML/2025-07-07_0 and uploading /tmp/tmp93ow_ik0 to it
albertwang8192-2025-07-07-0-v1-mkmlizer: 
  0%|          | 0/6 [00:00<?, ?it/s]
 17%|█▋        | 1/6 [00:06<00:34,  6.87s/it]
 33%|███▎      | 2/6 [00:10<00:20,  5.06s/it]
 50%|█████     | 3/6 [00:16<00:16,  5.63s/it]
 67%|██████▋   | 4/6 [00:22<00:11,  5.52s/it]
 83%|████████▎ | 5/6 [00:29<00:06,  6.01s/it]
100%|██████████| 6/6 [00:30<00:00,  4.37s/it]
100%|██████████| 6/6 [00:30<00:00,  5.06s/it]
albertwang8192-2025-07-07-0-v1-mkmlizer: quantizing model to /dev/shm/model_cache, profile:s0, folder:/tmp/tmp93ow_ik0, device:0
albertwang8192-2025-07-07-0-v1-mkmlizer: Saving flywheel model at /dev/shm/model_cache
albertwang8192-2025-07-07-0-v1-mkmlizer: quantized model in 31.420s
albertwang8192-2025-07-07-0-v1-mkmlizer: Processed model AlbertWang8192/2025-07-07_0 in 134.561s
albertwang8192-2025-07-07-0-v1-mkmlizer: creating bucket guanaco-mkml-models
albertwang8192-2025-07-07-0-v1-mkmlizer: cp /dev/shm/model_cache/config.json s3://guanaco-mkml-models/albertwang8192-2025-07-07-0-v1/nvidia/config.json
albertwang8192-2025-07-07-0-v1-mkmlizer: cp /dev/shm/model_cache/special_tokens_map.json s3://guanaco-mkml-models/albertwang8192-2025-07-07-0-v1/nvidia/special_tokens_map.json
albertwang8192-2025-07-07-0-v1-mkmlizer: cp /dev/shm/model_cache/tokenizer_config.json s3://guanaco-mkml-models/albertwang8192-2025-07-07-0-v1/nvidia/tokenizer_config.json
albertwang8192-2025-07-07-0-v1-mkmlizer: cp /dev/shm/model_cache/tokenizer.json s3://guanaco-mkml-models/albertwang8192-2025-07-07-0-v1/nvidia/tokenizer.json
albertwang8192-2025-07-07-0-v1-mkmlizer: cp /dev/shm/model_cache/flywheel_model.0.safetensors s3://guanaco-mkml-models/albertwang8192-2025-07-07-0-v1/nvidia/flywheel_model.0.safetensors
albertwang8192-2025-07-07-0-v1-mkmlizer: 
Loading 0:   0%|          | 0/363 [00:00<?, ?it/s]
Loading 0:   1%|▏         | 5/363 [00:00<00:11, 32.31it/s]
Loading 0:   4%|▎         | 13/363 [00:00<00:06, 51.09it/s]
Loading 0:   5%|▌         | 19/363 [00:00<00:07, 44.82it/s]
Loading 0:   7%|▋         | 24/363 [00:00<00:07, 43.46it/s]
Loading 0:   9%|▊         | 31/363 [00:00<00:06, 49.11it/s]
Loading 0:  10%|█         | 37/363 [00:00<00:07, 45.08it/s]
Loading 0:  12%|█▏        | 42/363 [00:00<00:07, 43.83it/s]
Loading 0:  13%|█▎        | 49/363 [00:01<00:06, 48.61it/s]
Loading 0:  15%|█▌        | 55/363 [00:01<00:06, 44.62it/s]
Loading 0:  17%|█▋        | 61/363 [00:01<00:08, 33.74it/s]
Loading 0:  18%|█▊        | 65/363 [00:01<00:08, 33.31it/s]
Loading 0:  20%|█▉        | 72/363 [00:01<00:07, 39.41it/s]
Loading 0:  21%|██▏       | 78/363 [00:01<00:07, 38.89it/s]
Loading 0:  23%|██▎       | 83/363 [00:02<00:07, 39.16it/s]
Loading 0:  25%|██▍       | 89/363 [00:02<00:06, 43.83it/s]
Loading 0:  26%|██▌       | 94/363 [00:02<00:06, 43.62it/s]
Loading 0:  27%|██▋       | 99/363 [00:02<00:06, 43.29it/s]
Loading 0:  29%|██▉       | 105/363 [00:02<00:06, 40.43it/s]
Loading 0:  30%|███       | 110/363 [00:02<00:05, 42.55it/s]
Loading 0:  32%|███▏      | 115/363 [00:02<00:05, 42.66it/s]
Loading 0:  33%|███▎      | 120/363 [00:02<00:06, 40.49it/s]
Loading 0:  34%|███▍      | 125/363 [00:02<00:05, 42.54it/s]
Loading 0:  36%|███▌      | 130/363 [00:03<00:05, 42.43it/s]
Loading 0:  37%|███▋      | 135/363 [00:03<00:05, 41.43it/s]
Loading 0:  39%|███▊      | 140/363 [00:03<00:05, 42.86it/s]
Loading 0:  40%|███▉      | 145/363 [00:03<00:08, 26.66it/s]
Loading 0:  41%|████      | 149/363 [00:03<00:07, 27.59it/s]
Loading 0:  43%|████▎     | 156/363 [00:03<00:05, 35.35it/s]
Loading 0:  44%|████▍     | 161/363 [00:04<00:05, 37.12it/s]
Loading 0:  46%|████▌     | 166/363 [00:04<00:05, 38.26it/s]
Loading 0:  47%|████▋     | 171/363 [00:04<00:04, 40.78it/s]
Loading 0:  48%|████▊     | 176/363 [00:04<00:05, 33.79it/s]
Loading 0:  50%|█████     | 183/363 [00:04<00:04, 39.47it/s]
Loading 0:  52%|█████▏    | 188/363 [00:04<00:04, 39.81it/s]
Loading 0:  53%|█████▎    | 193/363 [00:04<00:04, 40.38it/s]
Loading 0:  55%|█████▍    | 198/363 [00:04<00:03, 42.33it/s]
Loading 0:  56%|█████▌    | 203/363 [00:05<00:04, 34.55it/s]
Loading 0:  57%|█████▋    | 208/363 [00:05<00:04, 37.64it/s]
Loading 0:  59%|█████▊    | 213/363 [00:05<00:04, 36.88it/s]
Loading 0:  60%|██████    | 218/363 [00:05<00:03, 38.20it/s]
Loading 0:  61%|██████▏   | 223/363 [00:05<00:04, 29.43it/s]
Loading 0:  63%|██████▎   | 227/363 [00:05<00:04, 30.60it/s]
Loading 0:  64%|██████▎   | 231/363 [00:06<00:04, 30.08it/s]
Loading 0:  65%|██████▌   | 237/363 [00:06<00:03, 35.46it/s]
Loading 0:  66%|██████▋   | 241/363 [00:06<00:03, 35.42it/s]
Loading 0:  68%|██████▊   | 246/363 [00:06<00:03, 38.27it/s]
Loading 0:  69%|██████▉   | 251/363 [00:06<00:02, 38.65it/s]
Loading 0:  70%|███████   | 255/363 [00:06<00:02, 38.98it/s]
Loading 0:  71%|███████▏  | 259/363 [00:06<00:02, 38.25it/s]
Loading 0:  73%|███████▎  | 264/363 [00:06<00:02, 41.18it/s]
Loading 0:  74%|███████▍  | 269/363 [00:06<00:02, 41.19it/s]
Loading 0:  75%|███████▌  | 274/363 [00:07<00:02, 41.61it/s]
Loading 0:  77%|███████▋  | 280/363 [00:07<00:02, 39.95it/s]
Loading 0:  79%|███████▊  | 285/363 [00:07<00:01, 39.73it/s]
Loading 0:  80%|████████  | 291/363 [00:07<00:01, 43.37it/s]
Loading 0:  82%|████████▏ | 296/363 [00:07<00:01, 42.74it/s]
Loading 0:  83%|████████▎ | 301/363 [00:07<00:01, 44.01it/s]
Loading 0:  84%|████████▍ | 306/363 [00:08<00:02, 23.68it/s]
Loading 0:  85%|████████▌ | 310/363 [00:08<00:02, 25.01it/s]
Loading 0:  87%|████████▋ | 314/363 [00:08<00:01, 27.47it/s]
Loading 0:  88%|████████▊ | 320/363 [00:08<00:01, 33.32it/s]
Loading 0:  90%|████████▉ | 326/363 [00:08<00:01, 34.95it/s]
Loading 0:  91%|█████████ | 330/363 [00:08<00:00, 34.22it/s]
Loading 0:  93%|█████████▎| 337/363 [00:08<00:00, 42.29it/s]
Loading 0:  94%|█████████▍| 342/363 [00:08<00:00, 41.69it/s]
Loading 0:  96%|█████████▌| 347/363 [00:09<00:00, 42.22it/s]
Loading 0:  97%|█████████▋| 353/363 [00:09<00:00, 40.59it/s]
Loading 0:  99%|█████████▊| 358/363 [00:09<00:00, 39.89it/s]
                                                            

Job albertwang8192-2025-07-07-0-v1-mkmlizer completed after 156.88s with status: succeeded
Stopping job with name albertwang8192-2025-07-07-0-v1-mkmlizer
Pipeline stage MKMLizer completed in 157.49s
run pipeline stage %s
Running pipeline stage MKMLTemplater
Pipeline stage MKMLTemplater completed in 0.15s
run pipeline stage %s
Running pipeline stage MKMLDeployer
Creating inference service albertwang8192-2025-07-07-0-v1
Waiting for inference service albertwang8192-2025-07-07-0-v1 to be ready
Failed to get response for submission blend_hunen_2025-06-23: HTTPConnectionPool(host='guanaco-model-mesh.k2.chaiverse.com', port=80): Read timed out. (read timeout=12.0)
Inference service albertwang8192-2025-07-07-0-v1 ready after 351.814670085907s
Pipeline stage MKMLDeployer completed in 352.62s
run pipeline stage %s
Running pipeline stage StressChecker
HTTPConnectionPool(host='guanaco-submitter.guanaco-backend.k2.chaiverse.com', port=80): Read timed out. (read timeout=20)
Received unhealthy response to inference request!
Received healthy response to inference request in 2.442296028137207s
Received healthy response to inference request in 1.7220580577850342s
Received healthy response to inference request in 1.8853416442871094s
Received healthy response to inference request in 1.6827049255371094s
5 requests
1 failed requests
5th percentile: 1.6905755519866943
10th percentile: 1.6984461784362792
20th percentile: 1.7141874313354493
30th percentile: 1.7547147750854493
40th percentile: 1.8200282096862792
50th percentile: 1.8853416442871094
60th percentile: 2.1081233978271485
70th percentile: 2.3309051513671872
80th percentile: 5.981736993789676
90th percentile: 13.060618925094605
95th percentile: 16.60005989074707
99th percentile: 19.431612663269043
mean time: 5.5743803024292
%s, retrying in %s seconds...
Received healthy response to inference request in 1.76346755027771s
Received healthy response to inference request in 1.9438459873199463s
Received healthy response to inference request in 1.434072732925415s
Received healthy response to inference request in 1.7997548580169678s
Received healthy response to inference request in 2.200230121612549s
5 requests
0 failed requests
5th percentile: 1.4999516963958741
10th percentile: 1.565830659866333
20th percentile: 1.6975885868072509
30th percentile: 1.7707250118255615
40th percentile: 1.7852399349212646
50th percentile: 1.7997548580169678
60th percentile: 1.8573913097381591
70th percentile: 1.9150277614593505
80th percentile: 1.995122814178467
90th percentile: 2.097676467895508
95th percentile: 2.1489532947540284
99th percentile: 2.189974756240845
mean time: 1.8282742500305176
Pipeline stage StressChecker completed in 39.45s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyTriggerPipeline
run_pipeline:run_in_cloud %s
starting trigger_guanaco_pipeline args=%s
triggered trigger_guanaco_pipeline args=%s
Pipeline stage OfflineFamilyFriendlyTriggerPipeline completed in 0.66s
run pipeline stage %s
Running pipeline stage TriggerMKMLProfilingPipeline
run_pipeline:run_in_cloud %s
starting trigger_guanaco_pipeline args=%s
triggered trigger_guanaco_pipeline args=%s
Pipeline stage TriggerMKMLProfilingPipeline completed in 0.69s
Shutdown handler de-registered
albertwang8192-2025-07-07-0_v1 status is now deployed due to DeploymentManager action
Shutdown handler registered
run pipeline %s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyScorer
Evaluating %s Family Friendly Score with %s threads
%s, retrying in %s seconds...
Evaluating %s Family Friendly Score with %s threads
%s, retrying in %s seconds...
%s, retrying in %s seconds...
%s, retrying in %s seconds...
Received signal 15, running shutdown handler
Shutdown handler registered
run pipeline %s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyScorer
Evaluating %s Family Friendly Score with %s threads
Pipeline stage OfflineFamilyFriendlyScorer completed in 3395.35s
Shutdown handler de-registered
albertwang8192-2025-07-07-0_v1 status is now torndown due to DeploymentManager action