developer_uid: RandomForest1024
submission_id: albertwang8192-2025-07-11-6_v1
model_name: 2025-07-11_6
model_group: AlbertWang8192/2025-07-1
status: torndown
timestamp: 2025-07-11T22:29:25+00:00
num_battles: 8563
num_wins: 4019
celo_rating: 1267.12
family_friendly_score: 0.47840000000000005
family_friendly_standard_error: 0.007064466575757862
submission_type: basic
model_repo: AlbertWang8192/2025-07-11_6
model_architecture: MistralForCausalLM
model_num_parameters: 12772070400.0
best_of: 8
max_input_tokens: 1024
max_output_tokens: 64
reward_model: default
latencies: [{'batch_size': 1, 'throughput': 0.5965810104757281, 'latency_mean': 1.6760420191287995, 'latency_p50': 1.6678179502487183, 'latency_p90': 1.8458763837814331}, {'batch_size': 3, 'throughput': 1.0769040388166138, 'latency_mean': 2.7783901154994965, 'latency_p50': 2.7754764556884766, 'latency_p90': 3.051630425453186}, {'batch_size': 5, 'throughput': 1.286690231617703, 'latency_mean': 3.868350361585617, 'latency_p50': 3.881420612335205, 'latency_p90': 4.287459588050842}, {'batch_size': 6, 'throughput': 1.3236899171197105, 'latency_mean': 4.497195862531662, 'latency_p50': 4.5319578647613525, 'latency_p90': 5.019230628013611}, {'batch_size': 8, 'throughput': 1.4001481959795539, 'latency_mean': 5.666003634929657, 'latency_p50': 5.664803385734558, 'latency_p90': 6.3565489768981935}, {'batch_size': 10, 'throughput': 1.4307031133141812, 'latency_mean': 6.9380909907817845, 'latency_p50': 6.9524043798446655, 'latency_p90': 7.820628023147583}]
gpu_counts: {'NVIDIA RTX A5000': 1}
display_name: 2025-07-11_6
is_internal_developer: False
language_model: AlbertWang8192/2025-07-11_6
model_size: 13B
ranking_group: single
throughput_3p7s: 1.27
us_pacific_date: 2025-07-11
win_ratio: 0.46934485577484525
generation_params: {'temperature': 0.6, 'top_p': 0.95, 'min_p': 0.025, 'top_k': 60, 'presence_penalty': 0.4, 'frequency_penalty': 0.4, 'stopping_words': ['\n', '<|im_end|>', '<|im_start|>'], 'max_input_tokens': 1024, 'best_of': 8, 'max_output_tokens': 64}
formatter: {'memory_template': "{bot_name}'s Persona: {memory}\n####\n", 'prompt_template': '{prompt}\n<START>\n', 'bot_template': '{bot_name}: {message}\n', 'user_template': '{user_name}: {message}\n', 'response_template': '{bot_name}:', 'truncate_by_message': True}
Resubmit model
Shutdown handler not registered because Python interpreter is not running in the main thread
run pipeline %s
run pipeline stage %s
Running pipeline stage MKMLizer
Starting job with name albertwang8192-2025-07-11-6-v1-mkmlizer
Waiting for job on albertwang8192-2025-07-11-6-v1-mkmlizer to finish
albertwang8192-2025-07-11-6-v1-mkmlizer: ╔═════════════════════════════════════════════════════════════════════╗
albertwang8192-2025-07-11-6-v1-mkmlizer: ║                                                                     ║
albertwang8192-2025-07-11-6-v1-mkmlizer: ║   ██████   ██████  █████   ████  ████                               ║
albertwang8192-2025-07-11-6-v1-mkmlizer: ║  ░░██████ ██████  ░░███   ███░  ░░███                               ║
albertwang8192-2025-07-11-6-v1-mkmlizer: ║   ░███░█████░███   ░███  ███     ░███                               ║
albertwang8192-2025-07-11-6-v1-mkmlizer: ║   ░███░░███ ░███   ░███████      ░███                               ║
albertwang8192-2025-07-11-6-v1-mkmlizer: ║   ░███ ░░░  ░███   ░███░░███     ░███                               ║
albertwang8192-2025-07-11-6-v1-mkmlizer: ║   ░███      ░███   ░███ ░░███    ░███                               ║
albertwang8192-2025-07-11-6-v1-mkmlizer: ║   █████     █████  █████ ░░████  █████                              ║
albertwang8192-2025-07-11-6-v1-mkmlizer: ║  ░░░░░     ░░░░░  ░░░░░   ░░░░  ░░░░░                               ║
albertwang8192-2025-07-11-6-v1-mkmlizer: ║                                                                     ║
albertwang8192-2025-07-11-6-v1-mkmlizer: ║  Version: 0.29.15                                                   ║
albertwang8192-2025-07-11-6-v1-mkmlizer: ║  Features: FLYWHEEL, CUDA                                           ║
albertwang8192-2025-07-11-6-v1-mkmlizer: ║  Copyright 2023-2025 MK ONE TECHNOLOGIES Inc.                       ║
albertwang8192-2025-07-11-6-v1-mkmlizer: ║  https://mk1.ai                                                     ║
albertwang8192-2025-07-11-6-v1-mkmlizer: ║                                                                     ║
albertwang8192-2025-07-11-6-v1-mkmlizer: ║  The license key for the current software has been verified as      ║
albertwang8192-2025-07-11-6-v1-mkmlizer: ║  belonging to:                                                      ║
albertwang8192-2025-07-11-6-v1-mkmlizer: ║                                                                     ║
albertwang8192-2025-07-11-6-v1-mkmlizer: ║  Chai Research Corp.                                                ║
albertwang8192-2025-07-11-6-v1-mkmlizer: ║  Account ID: 7997a29f-0ceb-4cc7-9adf-840c57b4ae6f                   ║
albertwang8192-2025-07-11-6-v1-mkmlizer: ║  Expiration: 2028-03-31 23:59:59                                    ║
albertwang8192-2025-07-11-6-v1-mkmlizer: ║                                                                     ║
albertwang8192-2025-07-11-6-v1-mkmlizer: ╚═════════════════════════════════════════════════════════════════════╝
albertwang8192-2025-07-11-6-v1-mkmlizer: Xet Storage is enabled for this repo, but the 'hf_xet' package is not installed. Falling back to regular HTTP download. For better performance, install the package with: `pip install huggingface_hub[hf_xet]` or `pip install hf_xet`
Failed to get response for submission blend_hunen_2025-06-23: HTTPConnectionPool(host='guanaco-model-mesh.k2.chaiverse.com', port=80): Read timed out. (read timeout=12.0)
albertwang8192-2025-07-11-6-v1-mkmlizer: Xet Storage is enabled for this repo, but the 'hf_xet' package is not installed. Falling back to regular HTTP download. For better performance, install the package with: `pip install huggingface_hub[hf_xet]` or `pip install hf_xet`
albertwang8192-2025-07-11-6-v1-mkmlizer: Xet Storage is enabled for this repo, but the 'hf_xet' package is not installed. Falling back to regular HTTP download. For better performance, install the package with: `pip install huggingface_hub[hf_xet]` or `pip install hf_xet`
albertwang8192-2025-07-11-6-v1-mkmlizer: Xet Storage is enabled for this repo, but the 'hf_xet' package is not installed. Falling back to regular HTTP download. For better performance, install the package with: `pip install huggingface_hub[hf_xet]` or `pip install hf_xet`
albertwang8192-2025-07-11-6-v1-mkmlizer: Xet Storage is enabled for this repo, but the 'hf_xet' package is not installed. Falling back to regular HTTP download. For better performance, install the package with: `pip install huggingface_hub[hf_xet]` or `pip install hf_xet`
albertwang8192-2025-07-11-6-v1-mkmlizer: Xet Storage is enabled for this repo, but the 'hf_xet' package is not installed. Falling back to regular HTTP download. For better performance, install the package with: `pip install huggingface_hub[hf_xet]` or `pip install hf_xet`
albertwang8192-2025-07-11-6-v1-mkmlizer: Downloaded to shared memory in 52.957s
albertwang8192-2025-07-11-6-v1-mkmlizer: Checking if AlbertWang8192/2025-07-11_6 already exists in ChaiML
albertwang8192-2025-07-11-6-v1-mkmlizer: Creating repo ChaiML/2025-07-11_6 and uploading /tmp/tmp_bd2wqzz to it
albertwang8192-2025-07-11-6-v1-mkmlizer: 
  0%|          | 0/6 [00:00<?, ?it/s]
 17%|█▋        | 1/6 [00:03<00:17,  3.46s/it]
 33%|███▎      | 2/6 [00:11<00:24,  6.06s/it]
 50%|█████     | 3/6 [00:16<00:16,  5.49s/it]
 67%|██████▋   | 4/6 [00:19<00:09,  4.68s/it]
 83%|████████▎ | 5/6 [00:23<00:04,  4.25s/it]
100%|██████████| 6/6 [00:24<00:00,  3.14s/it]
100%|██████████| 6/6 [00:24<00:00,  4.01s/it]
albertwang8192-2025-07-11-6-v1-mkmlizer: quantizing model to /dev/shm/model_cache, profile:s0, folder:/tmp/tmp_bd2wqzz, device:0
albertwang8192-2025-07-11-6-v1-mkmlizer: Saving flywheel model at /dev/shm/model_cache
albertwang8192-2025-07-11-6-v1-mkmlizer: quantized model in 29.722s
albertwang8192-2025-07-11-6-v1-mkmlizer: Processed model AlbertWang8192/2025-07-11_6 in 131.759s
albertwang8192-2025-07-11-6-v1-mkmlizer: creating bucket guanaco-mkml-models
albertwang8192-2025-07-11-6-v1-mkmlizer: Bucket 's3://guanaco-mkml-models/' created
albertwang8192-2025-07-11-6-v1-mkmlizer: uploading /dev/shm/model_cache to s3://guanaco-mkml-models/albertwang8192-2025-07-11-6-v1/nvidia
albertwang8192-2025-07-11-6-v1-mkmlizer: cp /dev/shm/model_cache/config.json s3://guanaco-mkml-models/albertwang8192-2025-07-11-6-v1/nvidia/config.json
albertwang8192-2025-07-11-6-v1-mkmlizer: cp /dev/shm/model_cache/special_tokens_map.json s3://guanaco-mkml-models/albertwang8192-2025-07-11-6-v1/nvidia/special_tokens_map.json
albertwang8192-2025-07-11-6-v1-mkmlizer: cp /dev/shm/model_cache/tokenizer_config.json s3://guanaco-mkml-models/albertwang8192-2025-07-11-6-v1/nvidia/tokenizer_config.json
albertwang8192-2025-07-11-6-v1-mkmlizer: cp /dev/shm/model_cache/tokenizer.json s3://guanaco-mkml-models/albertwang8192-2025-07-11-6-v1/nvidia/tokenizer.json
albertwang8192-2025-07-11-6-v1-mkmlizer: cp /dev/shm/model_cache/flywheel_model.0.safetensors s3://guanaco-mkml-models/albertwang8192-2025-07-11-6-v1/nvidia/flywheel_model.0.safetensors
albertwang8192-2025-07-11-6-v1-mkmlizer: 
Loading 0:   0%|          | 0/363 [00:00<?, ?it/s]
Loading 0:   1%|▏         | 5/363 [00:00<00:11, 31.21it/s]
Loading 0:   4%|▎         | 13/363 [00:00<00:06, 50.03it/s]
Loading 0:   5%|▌         | 19/363 [00:00<00:07, 44.78it/s]
Loading 0:   7%|▋         | 24/363 [00:00<00:07, 43.50it/s]
Loading 0:   9%|▊         | 31/363 [00:00<00:06, 49.31it/s]
Loading 0:  10%|█         | 37/363 [00:00<00:07, 45.45it/s]
Loading 0:  12%|█▏        | 42/363 [00:00<00:07, 44.35it/s]
Loading 0:  13%|█▎        | 49/363 [00:01<00:06, 49.30it/s]
Loading 0:  15%|█▌        | 55/363 [00:01<00:06, 45.60it/s]
Loading 0:  17%|█▋        | 61/363 [00:01<00:09, 32.88it/s]
Loading 0:  18%|█▊        | 65/363 [00:01<00:09, 32.50it/s]
Loading 0:  20%|█▉        | 72/363 [00:01<00:07, 39.03it/s]
Loading 0:  21%|██        | 77/363 [00:01<00:06, 41.43it/s]
Loading 0:  23%|██▎       | 82/363 [00:02<00:07, 36.48it/s]
Loading 0:  25%|██▍       | 90/363 [00:02<00:06, 44.31it/s]
Loading 0:  26%|██▋       | 96/363 [00:02<00:06, 41.87it/s]
Loading 0:  28%|██▊       | 101/363 [00:02<00:06, 41.57it/s]
Loading 0:  29%|██▉       | 106/363 [00:02<00:05, 43.35it/s]
Loading 0:  31%|███       | 112/363 [00:02<00:05, 46.59it/s]
Loading 0:  32%|███▏      | 117/363 [00:02<00:05, 44.24it/s]
Loading 0:  34%|███▎      | 122/363 [00:02<00:05, 45.41it/s]
Loading 0:  35%|███▍      | 127/363 [00:03<00:06, 38.00it/s]
Loading 0:  37%|███▋      | 134/363 [00:03<00:05, 44.27it/s]
Loading 0:  38%|███▊      | 139/363 [00:03<00:05, 43.85it/s]
Loading 0:  40%|███▉      | 144/363 [00:03<00:07, 27.60it/s]
Loading 0:  41%|████      | 149/363 [00:03<00:07, 30.01it/s]
Loading 0:  43%|████▎     | 156/363 [00:03<00:05, 36.60it/s]
Loading 0:  44%|████▍     | 161/363 [00:04<00:05, 37.52it/s]
Loading 0:  46%|████▌     | 166/363 [00:04<00:05, 38.43it/s]
Loading 0:  47%|████▋     | 171/363 [00:04<00:04, 40.80it/s]
Loading 0:  48%|████▊     | 176/363 [00:04<00:05, 34.79it/s]
Loading 0:  50%|█████     | 183/363 [00:04<00:04, 42.18it/s]
Loading 0:  52%|█████▏    | 188/363 [00:04<00:04, 42.39it/s]
Loading 0:  53%|█████▎    | 193/363 [00:04<00:03, 42.84it/s]
Loading 0:  55%|█████▍    | 199/363 [00:04<00:04, 40.84it/s]
Loading 0:  56%|█████▌    | 204/363 [00:05<00:03, 40.35it/s]
Loading 0:  58%|█████▊    | 210/363 [00:05<00:03, 44.26it/s]
Loading 0:  59%|█████▉    | 215/363 [00:05<00:03, 44.29it/s]
Loading 0:  61%|██████    | 221/363 [00:05<00:02, 47.79it/s]
Loading 0:  62%|██████▏   | 226/363 [00:05<00:04, 29.41it/s]
Loading 0:  63%|██████▎   | 230/363 [00:05<00:04, 30.02it/s]
Loading 0:  65%|██████▌   | 237/363 [00:05<00:03, 37.22it/s]
Loading 0:  67%|██████▋   | 242/363 [00:06<00:03, 37.69it/s]
Loading 0:  68%|██████▊   | 247/363 [00:06<00:03, 38.34it/s]
Loading 0:  69%|██████▉   | 252/363 [00:06<00:02, 41.04it/s]
Loading 0:  71%|███████   | 257/363 [00:06<00:03, 34.68it/s]
Loading 0:  73%|███████▎  | 264/363 [00:06<00:02, 41.59it/s]
Loading 0:  74%|███████▍  | 269/363 [00:06<00:02, 42.13it/s]
Loading 0:  75%|███████▌  | 274/363 [00:06<00:02, 41.78it/s]
Loading 0:  77%|███████▋  | 279/363 [00:06<00:01, 43.46it/s]
Loading 0:  78%|███████▊  | 284/363 [00:07<00:02, 36.18it/s]
Loading 0:  80%|████████  | 291/363 [00:07<00:01, 42.86it/s]
Loading 0:  82%|████████▏ | 296/363 [00:07<00:01, 42.82it/s]
Loading 0:  83%|████████▎ | 301/363 [00:07<00:01, 44.01it/s]
Loading 0:  84%|████████▍ | 306/363 [00:07<00:02, 23.94it/s]
Loading 0:  85%|████████▌ | 310/363 [00:08<00:02, 25.13it/s]
Loading 0:  87%|████████▋ | 314/363 [00:08<00:01, 27.47it/s]
Loading 0:  88%|████████▊ | 320/363 [00:08<00:01, 33.12it/s]
Loading 0:  90%|████████▉ | 326/363 [00:08<00:01, 34.80it/s]
Loading 0:  91%|█████████ | 330/363 [00:08<00:00, 34.24it/s]
Loading 0:  93%|█████████▎| 337/363 [00:08<00:00, 41.92it/s]
Loading 0:  94%|█████████▍| 342/363 [00:08<00:00, 42.34it/s]
Loading 0:  96%|█████████▌| 347/363 [00:08<00:00, 43.74it/s]
Loading 0:  97%|█████████▋| 353/363 [00:09<00:00, 41.50it/s]
Loading 0:  99%|█████████▊| 358/363 [00:09<00:00, 40.14it/s]
                                                            

Job albertwang8192-2025-07-11-6-v1-mkmlizer completed after 157.09s with status: succeeded
Stopping job with name albertwang8192-2025-07-11-6-v1-mkmlizer
Pipeline stage MKMLizer completed in 157.69s
run pipeline stage %s
Running pipeline stage MKMLTemplater
Pipeline stage MKMLTemplater completed in 0.16s
run pipeline stage %s
Running pipeline stage MKMLDeployer
Creating inference service albertwang8192-2025-07-11-6-v1
Waiting for inference service albertwang8192-2025-07-11-6-v1 to be ready
Shutdown handler not registered because Python interpreter is not running in the main thread
run pipeline %s
run pipeline stage %s
Running pipeline stage StressChecker
Received healthy response to inference request in 3.3435933589935303s
Received healthy response to inference request in 3.0581703186035156s
Received healthy response to inference request in 4.8162689208984375s
Received healthy response to inference request in 2.988140344619751s
Received healthy response to inference request in 2.7996060848236084s
5 requests
0 failed requests
5th percentile: 2.837312936782837
10th percentile: 2.8750197887420654
20th percentile: 2.9504334926605225
30th percentile: 3.002146339416504
40th percentile: 3.0301583290100096
50th percentile: 3.0581703186035156
60th percentile: 3.1723395347595216
70th percentile: 3.286508750915527
80th percentile: 3.638128471374512
90th percentile: 4.227198696136474
95th percentile: 4.521733808517456
99th percentile: 4.757361898422241
mean time: 3.4011558055877686
Pipeline stage StressChecker completed in 19.58s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyTriggerPipeline
run_pipeline:run_in_cloud %s
starting trigger_guanaco_pipeline args=%s
triggered trigger_guanaco_pipeline args=%s
Pipeline stage OfflineFamilyFriendlyTriggerPipeline completed in 0.95s
Shutdown handler de-registered
function_hiraf_2025-07-11 status is now deployed due to DeploymentManager action
Inference service albertwang8192-2025-07-11-6-v1 ready after 210.98935747146606s
Pipeline stage MKMLDeployer completed in 211.57s
run pipeline stage %s
Running pipeline stage StressChecker
Received healthy response to inference request in 2.3922548294067383s
Received healthy response to inference request in 1.814701795578003s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Received healthy response to inference request in 1.576136589050293s
Received healthy response to inference request in 1.6094608306884766s
Received healthy response to inference request in 1.6495819091796875s
5 requests
0 failed requests
5th percentile: 1.5828014373779298
10th percentile: 1.5894662857055664
20th percentile: 1.6027959823608398
30th percentile: 1.6174850463867188
40th percentile: 1.6335334777832031
50th percentile: 1.6495819091796875
60th percentile: 1.7156298637390137
70th percentile: 1.7816778182983397
80th percentile: 1.9302124023437501
90th percentile: 2.1612336158752443
95th percentile: 2.2767442226409913
99th percentile: 2.369152708053589
mean time: 1.8084271907806397
Pipeline stage StressChecker completed in 10.66s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyTriggerPipeline
run_pipeline:run_in_cloud %s
starting trigger_guanaco_pipeline args=%s
triggered trigger_guanaco_pipeline args=%s
Pipeline stage OfflineFamilyFriendlyTriggerPipeline completed in 0.77s
run pipeline stage %s
Running pipeline stage TriggerMKMLProfilingPipeline
run_pipeline:run_in_cloud %s
starting trigger_guanaco_pipeline args=%s
triggered trigger_guanaco_pipeline args=%s
Pipeline stage TriggerMKMLProfilingPipeline completed in 0.70s
Shutdown handler de-registered
albertwang8192-2025-07-11-6_v1 status is now deployed due to DeploymentManager action
Shutdown handler registered
run pipeline %s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyScorer
Evaluating %s Family Friendly Score with %s threads
Pipeline stage OfflineFamilyFriendlyScorer completed in 3171.70s
Shutdown handler de-registered
albertwang8192-2025-07-11-6_v1 status is now inactive due to auto deactivation removed underperforming models
albertwang8192-2025-07-11-6_v1 status is now torndown due to DeploymentManager action