albertwang8192-2025-07-10-2

developer_uid: RandomForest1024

submission_id: albertwang8192-2025-07-10-2_v1

model_name: 2025-07-10_2

model_group: AlbertWang8192/2025-07-1

status: torndown

timestamp: 2025-07-11T01:05:23+00:00

num_battles: 7317

num_wins: 3494

celo_rating: 1263.61

family_friendly_score: 0.5952

family_friendly_standard_error: 0.006941713909403066

submission_type: basic

model_repo: AlbertWang8192/2025-07-10_2

model_architecture: MistralForCausalLM

model_num_parameters: 12772070400.0

best_of: 8

max_input_tokens: 1024

max_output_tokens: 64

reward_model: default

latencies: [{'batch_size': 1, 'throughput': 0.6021615988630405, 'latency_mean': 1.66057532787323, 'latency_p50': 1.6613566875457764, 'latency_p90': 1.8119866609573365}, {'batch_size': 3, 'throughput': 1.080651062182067, 'latency_mean': 2.7675601649284363, 'latency_p50': 2.7742615938186646, 'latency_p90': 3.077749752998352}, {'batch_size': 5, 'throughput': 1.2939205563274967, 'latency_mean': 3.8446250891685487, 'latency_p50': 3.845948576927185, 'latency_p90': 4.2506498336792}, {'batch_size': 6, 'throughput': 1.3486715825076772, 'latency_mean': 4.4189450287818905, 'latency_p50': 4.411712646484375, 'latency_p90': 5.004455494880676}, {'batch_size': 8, 'throughput': 1.4162906885528521, 'latency_mean': 5.616814261674881, 'latency_p50': 5.629956007003784, 'latency_p90': 6.34860942363739}, {'batch_size': 10, 'throughput': 1.4390431617877015, 'latency_mean': 6.885648242235184, 'latency_p50': 6.875382423400879, 'latency_p90': 7.73653347492218}]

gpu_counts: {'NVIDIA RTX A5000': 1}

display_name: 2025-07-10_2

ineligible_reason: num_battles<10000

is_internal_developer: False

language_model: AlbertWang8192/2025-07-10_2

model_size: 13B

ranking_group: single

throughput_3p7s: 1.28

us_pacific_date: 2025-07-10

win_ratio: 0.4775181085144185

generation_params: {'temperature': 0.6, 'top_p': 0.95, 'min_p': 0.025, 'top_k': 60, 'presence_penalty': 0.4, 'frequency_penalty': 0.4, 'stopping_words': ['<|im_end|>', '\n', '<|im_start|>'], 'max_input_tokens': 1024, 'best_of': 8, 'max_output_tokens': 64}

formatter: {'memory_template': "{bot_name}'s Persona: {memory}\n####\n", 'prompt_template': '{prompt}\n<START>\n', 'bot_template': '{bot_name}: {message}\n', 'user_template': '{user_name}: {message}\n', 'response_template': '{bot_name}:', 'truncate_by_message': False}

Resubmit model

Shutdown handler not registered because Python interpreter is not running in the main thread
run pipeline %s
run pipeline stage %s
Running pipeline stage MKMLizer
Starting job with name albertwang8192-2025-07-10-2-v1-mkmlizer
Waiting for job on albertwang8192-2025-07-10-2-v1-mkmlizer to finish
albertwang8192-2025-07-10-2-v1-mkmlizer: ╔═════════════════════════════════════════════════════════════════════╗
albertwang8192-2025-07-10-2-v1-mkmlizer: ║                                                                     ║
albertwang8192-2025-07-10-2-v1-mkmlizer: ║   ██████   ██████  █████   ████  ████                               ║
albertwang8192-2025-07-10-2-v1-mkmlizer: ║  ░░██████ ██████  ░░███   ███░  ░░███                               ║
albertwang8192-2025-07-10-2-v1-mkmlizer: ║   ░███░█████░███   ░███  ███     ░███                               ║
albertwang8192-2025-07-10-2-v1-mkmlizer: ║   ░███░░███ ░███   ░███████      ░███                               ║
albertwang8192-2025-07-10-2-v1-mkmlizer: ║   ░███ ░░░  ░███   ░███░░███     ░███                               ║
albertwang8192-2025-07-10-2-v1-mkmlizer: ║   ░███      ░███   ░███ ░░███    ░███                               ║
albertwang8192-2025-07-10-2-v1-mkmlizer: ║   █████     █████  █████ ░░████  █████                              ║
albertwang8192-2025-07-10-2-v1-mkmlizer: ║  ░░░░░     ░░░░░  ░░░░░   ░░░░  ░░░░░                               ║
albertwang8192-2025-07-10-2-v1-mkmlizer: ║                                                                     ║
albertwang8192-2025-07-10-2-v1-mkmlizer: ║  Version: 0.29.15                                                   ║
albertwang8192-2025-07-10-2-v1-mkmlizer: ║  Features: FLYWHEEL, CUDA                                           ║
albertwang8192-2025-07-10-2-v1-mkmlizer: ║  Copyright 2023-2025 MK ONE TECHNOLOGIES Inc.                       ║
albertwang8192-2025-07-10-2-v1-mkmlizer: ║  https://mk1.ai                                                     ║
albertwang8192-2025-07-10-2-v1-mkmlizer: ║                                                                     ║
albertwang8192-2025-07-10-2-v1-mkmlizer: ║  The license key for the current software has been verified as      ║
albertwang8192-2025-07-10-2-v1-mkmlizer: ║  belonging to:                                                      ║
albertwang8192-2025-07-10-2-v1-mkmlizer: ║                                                                     ║
albertwang8192-2025-07-10-2-v1-mkmlizer: ║  Chai Research Corp.                                                ║
albertwang8192-2025-07-10-2-v1-mkmlizer: ║  Account ID: 7997a29f-0ceb-4cc7-9adf-840c57b4ae6f                   ║
albertwang8192-2025-07-10-2-v1-mkmlizer: ║  Expiration: 2028-03-31 23:59:59                                    ║
albertwang8192-2025-07-10-2-v1-mkmlizer: ║                                                                     ║
albertwang8192-2025-07-10-2-v1-mkmlizer: ╚═════════════════════════════════════════════════════════════════════╝
albertwang8192-2025-07-10-2-v1-mkmlizer: Xet Storage is enabled for this repo, but the 'hf_xet' package is not installed. Falling back to regular HTTP download. For better performance, install the package with: `pip install huggingface_hub[hf_xet]` or `pip install hf_xet`
albertwang8192-2025-07-10-2-v1-mkmlizer: Xet Storage is enabled for this repo, but the 'hf_xet' package is not installed. Falling back to regular HTTP download. For better performance, install the package with: `pip install huggingface_hub[hf_xet]` or `pip install hf_xet`
albertwang8192-2025-07-10-2-v1-mkmlizer: Xet Storage is enabled for this repo, but the 'hf_xet' package is not installed. Falling back to regular HTTP download. For better performance, install the package with: `pip install huggingface_hub[hf_xet]` or `pip install hf_xet`
albertwang8192-2025-07-10-2-v1-mkmlizer: Xet Storage is enabled for this repo, but the 'hf_xet' package is not installed. Falling back to regular HTTP download. For better performance, install the package with: `pip install huggingface_hub[hf_xet]` or `pip install hf_xet`
albertwang8192-2025-07-10-2-v1-mkmlizer: Xet Storage is enabled for this repo, but the 'hf_xet' package is not installed. Falling back to regular HTTP download. For better performance, install the package with: `pip install huggingface_hub[hf_xet]` or `pip install hf_xet`
albertwang8192-2025-07-10-2-v1-mkmlizer: Downloaded to shared memory in 64.355s
albertwang8192-2025-07-10-2-v1-mkmlizer: Checking if AlbertWang8192/2025-07-10_2 already exists in ChaiML
albertwang8192-2025-07-10-2-v1-mkmlizer: Creating repo ChaiML/2025-07-10_2 and uploading /tmp/tmpxbyjzc23 to it
albertwang8192-2025-07-10-2-v1-mkmlizer: 
  0%|          | 0/6 [00:00<?, ?it/s]
 17%|█▋        | 1/6 [00:04<00:20,  4.07s/it]
 33%|███▎      | 2/6 [00:11<00:25,  6.28s/it]
 50%|█████     | 3/6 [00:20<00:21,  7.25s/it]
 67%|██████▋   | 4/6 [00:27<00:14,  7.10s/it]
 83%|████████▎ | 5/6 [00:34<00:07,  7.03s/it]
100%|██████████| 6/6 [00:35<00:00,  5.00s/it]
100%|██████████| 6/6 [00:35<00:00,  5.86s/it]
albertwang8192-2025-07-10-2-v1-mkmlizer: quantizing model to /dev/shm/model_cache, profile:s0, folder:/tmp/tmpxbyjzc23, device:0
albertwang8192-2025-07-10-2-v1-mkmlizer: Saving flywheel model at /dev/shm/model_cache
albertwang8192-2025-07-10-2-v1-mkmlizer: quantized model in 30.938s
albertwang8192-2025-07-10-2-v1-mkmlizer: Processed model AlbertWang8192/2025-07-10_2 in 156.756s
albertwang8192-2025-07-10-2-v1-mkmlizer: creating bucket guanaco-mkml-models
albertwang8192-2025-07-10-2-v1-mkmlizer: Bucket 's3://guanaco-mkml-models/' created
albertwang8192-2025-07-10-2-v1-mkmlizer: uploading /dev/shm/model_cache to s3://guanaco-mkml-models/albertwang8192-2025-07-10-2-v1/nvidia
albertwang8192-2025-07-10-2-v1-mkmlizer: cp /dev/shm/model_cache/special_tokens_map.json s3://guanaco-mkml-models/albertwang8192-2025-07-10-2-v1/nvidia/special_tokens_map.json
albertwang8192-2025-07-10-2-v1-mkmlizer: cp /dev/shm/model_cache/config.json s3://guanaco-mkml-models/albertwang8192-2025-07-10-2-v1/nvidia/config.json
albertwang8192-2025-07-10-2-v1-mkmlizer: cp /dev/shm/model_cache/tokenizer_config.json s3://guanaco-mkml-models/albertwang8192-2025-07-10-2-v1/nvidia/tokenizer_config.json
albertwang8192-2025-07-10-2-v1-mkmlizer: cp /dev/shm/model_cache/tokenizer.json s3://guanaco-mkml-models/albertwang8192-2025-07-10-2-v1/nvidia/tokenizer.json
albertwang8192-2025-07-10-2-v1-mkmlizer: cp /dev/shm/model_cache/flywheel_model.0.safetensors s3://guanaco-mkml-models/albertwang8192-2025-07-10-2-v1/nvidia/flywheel_model.0.safetensors
albertwang8192-2025-07-10-2-v1-mkmlizer: 
Loading 0:   0%|          | 0/363 [00:00<?, ?it/s]
Loading 0:   1%|▏         | 5/363 [00:00<00:12, 29.76it/s]
Loading 0:   4%|▎         | 13/363 [00:00<00:07, 48.73it/s]
Loading 0:   5%|▌         | 19/363 [00:00<00:08, 42.73it/s]
Loading 0:   7%|▋         | 24/363 [00:00<00:08, 40.67it/s]
Loading 0:   9%|▊         | 31/363 [00:00<00:07, 46.06it/s]
Loading 0:  10%|▉         | 36/363 [00:00<00:06, 46.98it/s]
Loading 0:  11%|█▏        | 41/363 [00:01<00:08, 37.55it/s]
Loading 0:  13%|█▎        | 48/363 [00:01<00:06, 45.01it/s]
Loading 0:  15%|█▍        | 53/363 [00:01<00:06, 44.37it/s]
Loading 0:  16%|█▌        | 58/363 [00:01<00:06, 45.46it/s]
Loading 0:  17%|█▋        | 63/363 [00:01<00:10, 28.78it/s]
Loading 0:  18%|█▊        | 67/363 [00:01<00:09, 30.49it/s]
Loading 0:  20%|█▉        | 72/363 [00:01<00:08, 33.22it/s]
Loading 0:  21%|██        | 76/363 [00:01<00:08, 34.48it/s]
Loading 0:  22%|██▏       | 81/363 [00:02<00:07, 36.93it/s]
Loading 0:  24%|██▎       | 86/363 [00:02<00:06, 39.65it/s]
Loading 0:  25%|██▌       | 91/363 [00:02<00:08, 33.75it/s]
Loading 0:  27%|██▋       | 98/363 [00:02<00:06, 41.30it/s]
Loading 0:  28%|██▊       | 103/363 [00:02<00:06, 41.56it/s]
Loading 0:  30%|███       | 109/363 [00:02<00:05, 45.50it/s]
Loading 0:  31%|███▏      | 114/363 [00:02<00:06, 38.31it/s]
Loading 0:  33%|███▎      | 119/363 [00:03<00:06, 37.85it/s]
Loading 0:  34%|███▍      | 125/363 [00:03<00:05, 42.22it/s]
Loading 0:  36%|███▌      | 130/363 [00:03<00:05, 41.90it/s]
Loading 0:  37%|███▋      | 135/363 [00:03<00:05, 42.00it/s]
Loading 0:  39%|███▊      | 140/363 [00:03<00:05, 43.37it/s]
Loading 0:  40%|███▉      | 145/363 [00:03<00:08, 26.20it/s]
Loading 0:  41%|████      | 149/363 [00:04<00:07, 27.01it/s]
Loading 0:  43%|████▎     | 156/363 [00:04<00:06, 33.87it/s]
Loading 0:  44%|████▍     | 161/363 [00:04<00:05, 35.92it/s]
Loading 0:  46%|████▌     | 166/363 [00:04<00:05, 36.75it/s]
Loading 0:  47%|████▋     | 171/363 [00:04<00:04, 39.27it/s]
Loading 0:  48%|████▊     | 176/363 [00:04<00:05, 32.77it/s]
Loading 0:  50%|█████     | 183/363 [00:04<00:04, 40.13it/s]
Loading 0:  52%|█████▏    | 188/363 [00:04<00:04, 40.93it/s]
Loading 0:  53%|█████▎    | 193/363 [00:05<00:04, 40.40it/s]
Loading 0:  55%|█████▍    | 198/363 [00:05<00:03, 42.26it/s]
Loading 0:  56%|█████▌    | 203/363 [00:05<00:04, 35.11it/s]
Loading 0:  58%|█████▊    | 210/363 [00:05<00:03, 41.42it/s]
Loading 0:  59%|█████▉    | 215/363 [00:05<00:03, 41.08it/s]
Loading 0:  61%|██████    | 220/363 [00:05<00:03, 42.06it/s]
Loading 0:  62%|██████▏   | 225/363 [00:06<00:05, 25.39it/s]
Loading 0:  63%|██████▎   | 230/363 [00:06<00:04, 27.93it/s]
Loading 0:  65%|██████▌   | 237/363 [00:06<00:03, 34.57it/s]
Loading 0:  67%|██████▋   | 242/363 [00:06<00:03, 36.29it/s]
Loading 0:  68%|██████▊   | 247/363 [00:06<00:03, 37.79it/s]
Loading 0:  69%|██████▉   | 252/363 [00:06<00:02, 39.80it/s]
Loading 0:  71%|███████   | 257/363 [00:06<00:03, 33.61it/s]
Loading 0:  73%|███████▎  | 264/363 [00:07<00:02, 40.15it/s]
Loading 0:  74%|███████▍  | 269/363 [00:07<00:02, 39.82it/s]
Loading 0:  75%|███████▌  | 274/363 [00:07<00:02, 39.54it/s]
Loading 0:  77%|███████▋  | 279/363 [00:07<00:02, 41.55it/s]
Loading 0:  78%|███████▊  | 284/363 [00:07<00:02, 35.25it/s]
Loading 0:  80%|████████  | 291/363 [00:07<00:01, 41.39it/s]
Loading 0:  82%|████████▏ | 296/363 [00:07<00:01, 41.69it/s]
Loading 0:  83%|████████▎ | 301/363 [00:07<00:01, 43.37it/s]
Loading 0:  84%|████████▍ | 306/363 [00:08<00:02, 23.01it/s]
Loading 0:  85%|████████▌ | 310/363 [00:08<00:02, 24.32it/s]
Loading 0:  87%|████████▋ | 314/363 [00:08<00:01, 26.44it/s]
Loading 0:  88%|████████▊ | 319/363 [00:08<00:01, 30.49it/s]
Loading 0:  89%|████████▉ | 323/363 [00:08<00:01, 32.00it/s]
Loading 0:  91%|█████████ | 329/363 [00:08<00:00, 36.74it/s]
Loading 0:  92%|█████████▏| 334/363 [00:09<00:00, 39.54it/s]
Loading 0:  93%|█████████▎| 339/363 [00:09<00:00, 33.70it/s]
Loading 0:  95%|█████████▌| 346/363 [00:09<00:00, 41.22it/s]
Loading 0:  97%|█████████▋| 351/363 [00:09<00:00, 41.62it/s]
Loading 0:  98%|█████████▊| 356/363 [00:09<00:00, 42.40it/s]
Loading 0:  99%|█████████▉| 361/363 [00:09<00:00, 43.99it/s]
                                                            
Job albertwang8192-2025-07-10-2-v1-mkmlizer completed after 180.92s with status: succeeded
Stopping job with name albertwang8192-2025-07-10-2-v1-mkmlizer
Pipeline stage MKMLizer completed in 181.57s
run pipeline stage %s
Running pipeline stage MKMLTemplater
Pipeline stage MKMLTemplater completed in 0.18s
run pipeline stage %s
Running pipeline stage MKMLDeployer
Creating inference service albertwang8192-2025-07-10-2-v1
Waiting for inference service albertwang8192-2025-07-10-2-v1 to be ready
Inference service albertwang8192-2025-07-10-2-v1 ready after 191.0301558971405s
Pipeline stage MKMLDeployer completed in 191.80s
run pipeline stage %s
Running pipeline stage StressChecker
{"detail":"HTTPConnectionPool(host='albertwang8192-2025-07-10-2-v1-predictor.tenant-chaiml-guanaco.k.chaiverse.com', port=80): Max retries exceeded with url: /v1/models/GPT-J-6B-lit-v2:predict (Caused by ConnectTimeoutError(<urllib3.connection.HTTPConnection object at 0x7874dc37eb90>, 'Connection to albertwang8192-2025-07-10-2-v1-predictor.tenant-chaiml-guanaco.k.chaiverse.com timed out. (connect timeout=12.0)'))"}
Received unhealthy response to inference request!
Received healthy response to inference request in 2.2368619441986084s
Received healthy response to inference request in 1.4697411060333252s
Received healthy response to inference request in 1.5785529613494873s
Received healthy response to inference request in 2.3269054889678955s
5 requests
1 failed requests
5th percentile: 1.4915034770965576
10th percentile: 1.51326584815979
20th percentile: 1.5567905902862549
30th percentile: 1.7102147579193114
40th percentile: 1.9735383510589601
50th percentile: 2.2368619441986084
60th percentile: 2.2728793621063232
70th percentile: 2.308896780014038
80th percentile: 4.340798902511598
90th percentile: 8.368585729599001
95th percentile: 10.3824791431427
99th percentile: 11.99359387397766
mean time: 4.0016868114471436
%s, retrying in %s seconds...
Received healthy response to inference request in 1.751814842224121s
Received healthy response to inference request in 1.7244868278503418s
Received healthy response to inference request in 1.4080939292907715s
Received healthy response to inference request in 1.3946022987365723s
Received healthy response to inference request in 1.4586832523345947s
5 requests
0 failed requests
5th percentile: 1.397300624847412
10th percentile: 1.399998950958252
20th percentile: 1.4053956031799317
30th percentile: 1.4182117938995362
40th percentile: 1.4384475231170655
50th percentile: 1.4586832523345947
60th percentile: 1.5650046825408936
70th percentile: 1.6713261127471923
80th percentile: 1.7299524307250977
90th percentile: 1.7408836364746094
95th percentile: 1.7463492393493651
99th percentile: 1.75072172164917
mean time: 1.5475362300872804
Pipeline stage StressChecker completed in 30.63s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyTriggerPipeline
run_pipeline:run_in_cloud %s
starting trigger_guanaco_pipeline args=%s
triggered trigger_guanaco_pipeline args=%s
Pipeline stage OfflineFamilyFriendlyTriggerPipeline completed in 0.76s
run pipeline stage %s
Running pipeline stage TriggerMKMLProfilingPipeline
run_pipeline:run_in_cloud %s
starting trigger_guanaco_pipeline args=%s
triggered trigger_guanaco_pipeline args=%s
Pipeline stage TriggerMKMLProfilingPipeline completed in 0.80s
Shutdown handler de-registered
albertwang8192-2025-07-10-2_v1 status is now deployed due to DeploymentManager action
Shutdown handler registered
run pipeline %s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyScorer
Evaluating %s Family Friendly Score with %s threads
Pipeline stage OfflineFamilyFriendlyScorer completed in 3072.59s
Shutdown handler de-registered
albertwang8192-2025-07-10-2_v1 status is now inactive due to auto deactivation removed underperforming models
albertwang8192-2025-07-10-2_v1 status is now torndown due to DeploymentManager action