chaiml-severus-snape2507_3897

developer_uid: chai_backend_admin

submission_id: chaiml-severus-snape2507_3897_v1

model_name: chaiml-severus-snape2507_3897_v1

model_group: ChaiML/Severus-snape2507

status: torndown

timestamp: 2025-07-07T22:21:01+00:00

num_battles: 6704

num_wins: 3478

celo_rating: 1300.0

family_friendly_score: 0.5266

family_friendly_standard_error: 0.007061054312211456

submission_type: basic

model_repo: ChaiML/Severus-snape250707134922_sft

model_architecture: MistralForCausalLM

model_num_parameters: 24096691200.0

best_of: 8

max_input_tokens: 1024

max_output_tokens: 64

reward_model: default

latencies: [{'batch_size': 1, 'throughput': 0.5324657872713702, 'latency_mean': 1.8779544258117675, 'latency_p50': 1.8701660633087158, 'latency_p90': 2.0916460275650026}, {'batch_size': 3, 'throughput': 1.0616255933587735, 'latency_mean': 2.8186044776439667, 'latency_p50': 2.8009732961654663, 'latency_p90': 3.113379383087158}, {'batch_size': 5, 'throughput': 1.3496174380561887, 'latency_mean': 3.6807274055480956, 'latency_p50': 3.6681580543518066, 'latency_p90': 4.160017561912537}, {'batch_size': 6, 'throughput': 1.4549351422858212, 'latency_mean': 4.103307659626007, 'latency_p50': 4.089199185371399, 'latency_p90': 4.511265802383423}, {'batch_size': 8, 'throughput': 1.5731646858490018, 'latency_mean': 5.041178141832352, 'latency_p50': 5.0427093505859375, 'latency_p90': 5.59889280796051}, {'batch_size': 10, 'throughput': 1.664262364094992, 'latency_mean': 5.964943890571594, 'latency_p50': 5.972887754440308, 'latency_p90': 6.6662856340408325}]

gpu_counts: {'NVIDIA A100-SXM4-80GB': 1}

display_name: chaiml-severus-snape2507_3897_v1

is_internal_developer: True

language_model: ChaiML/Severus-snape250707134922_sft

model_size: 24B

ranking_group: single

throughput_3p7s: 1.36

us_pacific_date: 2025-07-07

win_ratio: 0.5187947494033412

generation_params: {'temperature': 1.0, 'top_p': 1.0, 'min_p': 0.0, 'top_k': 40, 'presence_penalty': 0.0, 'frequency_penalty': 0.0, 'stopping_words': ['\n', '####', '</s>', '####\n', 'You:'], 'max_input_tokens': 1024, 'best_of': 8, 'max_output_tokens': 64}

formatter: {'memory_template': '', 'prompt_template': '', 'bot_template': '{bot_name}: {message}</s>\n', 'user_template': 'You: {message}\n', 'response_template': '{bot_name}:', 'truncate_by_message': True}

Resubmit model

Shutdown handler not registered because Python interpreter is not running in the main thread
run pipeline %s
run pipeline stage %s
Running pipeline stage MKMLizer
Starting job with name chaiml-severus-snape2507-3897-v1-mkmlizer
Waiting for job on chaiml-severus-snape2507-3897-v1-mkmlizer to finish
chaiml-severus-snape2507-3897-v1-mkmlizer: ╔═════════════════════════════════════════════════════════════════════╗
chaiml-severus-snape2507-3897-v1-mkmlizer: ║                                                                     ║
chaiml-severus-snape2507-3897-v1-mkmlizer: ║   ██████   ██████  █████   ████  ████                               ║
chaiml-severus-snape2507-3897-v1-mkmlizer: ║  ░░██████ ██████  ░░███   ███░  ░░███                               ║
chaiml-severus-snape2507-3897-v1-mkmlizer: ║   ░███░█████░███   ░███  ███     ░███                               ║
chaiml-severus-snape2507-3897-v1-mkmlizer: ║   ░███░░███ ░███   ░███████      ░███                               ║
chaiml-severus-snape2507-3897-v1-mkmlizer: ║   ░███ ░░░  ░███   ░███░░███     ░███                               ║
chaiml-severus-snape2507-3897-v1-mkmlizer: ║   ░███      ░███   ░███ ░░███    ░███                               ║
chaiml-severus-snape2507-3897-v1-mkmlizer: ║   █████     █████  █████ ░░████  █████                              ║
chaiml-severus-snape2507-3897-v1-mkmlizer: ║  ░░░░░     ░░░░░  ░░░░░   ░░░░  ░░░░░                               ║
chaiml-severus-snape2507-3897-v1-mkmlizer: ║                                                                     ║
chaiml-severus-snape2507-3897-v1-mkmlizer: ║  Version: 0.29.15                                                   ║
chaiml-severus-snape2507-3897-v1-mkmlizer: ║  Features: FLYWHEEL, CUDA                                           ║
chaiml-severus-snape2507-3897-v1-mkmlizer: ║  Copyright 2023-2025 MK ONE TECHNOLOGIES Inc.                       ║
chaiml-severus-snape2507-3897-v1-mkmlizer: ║  https://mk1.ai                                                     ║
chaiml-severus-snape2507-3897-v1-mkmlizer: ║                                                                     ║
chaiml-severus-snape2507-3897-v1-mkmlizer: ║  The license key for the current software has been verified as      ║
chaiml-severus-snape2507-3897-v1-mkmlizer: ║  belonging to:                                                      ║
chaiml-severus-snape2507-3897-v1-mkmlizer: ║                                                                     ║
chaiml-severus-snape2507-3897-v1-mkmlizer: ║  Chai Research Corp.                                                ║
chaiml-severus-snape2507-3897-v1-mkmlizer: ║  Account ID: 7997a29f-0ceb-4cc7-9adf-840c57b4ae6f                   ║
chaiml-severus-snape2507-3897-v1-mkmlizer: ║  Expiration: 2028-03-31 23:59:59                                    ║
chaiml-severus-snape2507-3897-v1-mkmlizer: ║                                                                     ║
chaiml-severus-snape2507-3897-v1-mkmlizer: ╚═════════════════════════════════════════════════════════════════════╝
chaiml-severus-snape2507-3897-v1-mkmlizer: Downloaded to shared memory in 108.012s
chaiml-severus-snape2507-3897-v1-mkmlizer: Checking if ChaiML/Severus-snape250707134922_sft already exists in ChaiML
chaiml-severus-snape2507-3897-v1-mkmlizer: quantizing model to /dev/shm/model_cache, profile:s0, folder:/tmp/tmpaiboh99t, device:0
chaiml-severus-snape2507-3897-v1-mkmlizer: Saving flywheel model at /dev/shm/model_cache
chaiml-severus-snape2507-3897-v1-mkmlizer: quantized model in 66.028s
chaiml-severus-snape2507-3897-v1-mkmlizer: Processed model ChaiML/Severus-snape250707134922_sft in 174.041s
chaiml-severus-snape2507-3897-v1-mkmlizer: creating bucket guanaco-mkml-models
chaiml-severus-snape2507-3897-v1-mkmlizer: Bucket 's3://guanaco-mkml-models/' created
chaiml-severus-snape2507-3897-v1-mkmlizer: uploading /dev/shm/model_cache to s3://guanaco-mkml-models/chaiml-severus-snape2507-3897-v1/nvidia
chaiml-severus-snape2507-3897-v1-mkmlizer: cp /dev/shm/model_cache/config.json s3://guanaco-mkml-models/chaiml-severus-snape2507-3897-v1/nvidia/config.json
chaiml-severus-snape2507-3897-v1-mkmlizer: cp /dev/shm/model_cache/special_tokens_map.json s3://guanaco-mkml-models/chaiml-severus-snape2507-3897-v1/nvidia/special_tokens_map.json
chaiml-severus-snape2507-3897-v1-mkmlizer: cp /dev/shm/model_cache/tokenizer_config.json s3://guanaco-mkml-models/chaiml-severus-snape2507-3897-v1/nvidia/tokenizer_config.json
chaiml-severus-snape2507-3897-v1-mkmlizer: cp /dev/shm/model_cache/tokenizer.json s3://guanaco-mkml-models/chaiml-severus-snape2507-3897-v1/nvidia/tokenizer.json
chaiml-severus-snape2507-3897-v1-mkmlizer: cp /dev/shm/model_cache/flywheel_model.1.safetensors s3://guanaco-mkml-models/chaiml-severus-snape2507-3897-v1/nvidia/flywheel_model.1.safetensors
chaiml-severus-snape2507-3897-v1-mkmlizer: cp /dev/shm/model_cache/flywheel_model.0.safetensors s3://guanaco-mkml-models/chaiml-severus-snape2507-3897-v1/nvidia/flywheel_model.0.safetensors
chaiml-severus-snape2507-3897-v1-mkmlizer: 
Loading 0:   0%|          | 0/363 [00:00<?, ?it/s]
Loading 0:   1%|          | 4/363 [00:00<00:12, 27.78it/s]
Loading 0:   2%|▏         | 7/363 [00:00<00:17, 20.67it/s]
Loading 0:   3%|▎         | 10/363 [00:00<00:14, 23.61it/s]
Loading 0:   4%|▎         | 13/363 [00:00<00:14, 23.70it/s]
Loading 0:   4%|▍         | 16/363 [00:00<00:16, 21.58it/s]
Loading 0:   5%|▌         | 19/363 [00:00<00:14, 23.72it/s]
Loading 0:   6%|▌         | 22/363 [00:00<00:14, 24.16it/s]
Loading 0:   7%|▋         | 25/363 [00:01<00:14, 22.70it/s]
Loading 0:   8%|▊         | 29/363 [00:01<00:12, 26.83it/s]
Loading 0:   9%|▉         | 33/363 [00:01<00:17, 18.87it/s]
Loading 0:  10%|▉         | 36/363 [00:01<00:20, 15.60it/s]
Loading 0:  11%|█         | 40/363 [00:01<00:17, 18.46it/s]
Loading 0:  12%|█▏        | 43/363 [00:02<00:17, 18.76it/s]
Loading 0:  13%|█▎        | 48/363 [00:02<00:14, 22.04it/s]
Loading 0:  14%|█▍        | 51/363 [00:02<00:16, 19.32it/s]
Loading 0:  15%|█▌        | 55/363 [00:02<00:13, 22.33it/s]
Loading 0:  16%|█▌        | 58/363 [00:02<00:13, 22.52it/s]
Loading 0:  17%|█▋        | 61/363 [00:02<00:14, 21.48it/s]
Loading 0:  18%|█▊        | 64/363 [00:02<00:12, 23.03it/s]
Loading 0:  19%|█▉        | 69/363 [00:03<00:11, 24.94it/s]
Loading 0:  20%|█▉        | 72/363 [00:03<00:17, 16.43it/s]
Loading 0:  21%|██        | 75/363 [00:03<00:16, 17.03it/s]
Loading 0:  21%|██▏       | 78/363 [00:03<00:14, 19.14it/s]
Loading 0:  22%|██▏       | 81/363 [00:04<00:16, 17.32it/s]
Loading 0:  23%|██▎       | 84/363 [00:04<00:14, 19.30it/s]
Loading 0:  24%|██▍       | 87/363 [00:04<00:16, 17.10it/s]
Loading 0:  25%|██▌       | 91/363 [00:04<00:13, 20.14it/s]
Loading 0:  26%|██▌       | 94/363 [00:04<00:13, 19.77it/s]
Loading 0:  27%|██▋       | 97/363 [00:04<00:12, 21.26it/s]
Loading 0:  28%|██▊       | 100/363 [00:04<00:12, 21.46it/s]
Loading 0:  28%|██▊       | 103/363 [00:05<00:12, 20.36it/s]
Loading 0:  29%|██▉       | 106/363 [00:05<00:11, 21.68it/s]
Loading 0:  30%|███       | 109/363 [00:05<00:15, 15.96it/s]
Loading 0:  31%|███       | 112/363 [00:05<00:14, 17.27it/s]
Loading 0:  31%|███▏      | 114/363 [00:05<00:15, 16.14it/s]
Loading 0:  33%|███▎      | 118/363 [00:05<00:12, 20.26it/s]
Loading 0:  33%|███▎      | 121/363 [00:06<00:11, 20.65it/s]
Loading 0:  34%|███▍      | 124/363 [00:06<00:12, 19.62it/s]
Loading 0:  36%|███▌      | 129/363 [00:06<00:10, 22.73it/s]
Loading 0:  36%|███▋      | 132/363 [00:06<00:12, 19.21it/s]
Loading 0:  37%|███▋      | 136/363 [00:06<00:10, 22.55it/s]
Loading 0:  38%|███▊      | 139/363 [00:06<00:10, 22.31it/s]
Loading 0:  39%|███▉      | 142/363 [00:06<00:10, 21.41it/s]
Loading 0:  40%|███▉      | 145/363 [00:07<00:09, 22.34it/s]
Loading 0:  41%|████      | 149/363 [00:07<00:08, 25.23it/s]
Loading 0:  42%|████▏     | 152/363 [00:07<00:13, 15.26it/s]
Loading 0:  43%|████▎     | 155/363 [00:07<00:12, 16.28it/s]
Loading 0:  44%|████▎     | 158/363 [00:08<00:14, 13.95it/s]
Loading 0:  45%|████▍     | 163/363 [00:08<00:10, 19.12it/s]
Loading 0:  46%|████▌     | 166/363 [00:08<00:09, 19.89it/s]
Loading 0:  47%|████▋     | 169/363 [00:08<00:09, 19.67it/s]
Loading 0:  47%|████▋     | 172/363 [00:08<00:08, 21.29it/s]
Loading 0:  48%|████▊     | 175/363 [00:08<00:08, 21.31it/s]
Loading 0:  49%|████▉     | 178/363 [00:08<00:08, 20.83it/s]
Loading 0:  50%|████▉     | 181/363 [00:09<00:08, 22.47it/s]
Loading 0:  51%|█████     | 185/363 [00:09<00:06, 26.56it/s]
Loading 0:  52%|█████▏    | 188/363 [00:09<00:10, 16.07it/s]
Loading 0:  53%|█████▎    | 191/363 [00:09<00:10, 16.96it/s]
Loading 0:  53%|█████▎    | 194/363 [00:09<00:11, 14.32it/s]
Loading 0:  55%|█████▍    | 199/363 [00:10<00:08, 18.95it/s]
Loading 0:  55%|█████▌    | 200/363 [00:20<00:08, 18.95it/s]
Loading 0:  55%|█████▌    | 201/363 [00:30<05:12,  1.93s/it]
Loading 0:  56%|█████▌    | 203/363 [00:30<04:07,  1.55s/it]
Loading 0:  57%|█████▋    | 208/363 [00:30<02:18,  1.12it/s]
Loading 0:  58%|█████▊    | 211/363 [00:30<01:42,  1.48it/s]
Loading 0:  59%|█████▊    | 213/363 [00:30<01:23,  1.81it/s]
Loading 0:  60%|█████▉    | 217/363 [00:31<00:52,  2.77it/s]
Loading 0:  61%|██████    | 220/363 [00:31<00:40,  3.56it/s]
Loading 0:  61%|██████▏   | 223/363 [00:31<00:29,  4.76it/s]
Loading 0:  62%|██████▏   | 226/363 [00:31<00:24,  5.50it/s]
Loading 0:  63%|██████▎   | 229/363 [00:31<00:18,  7.08it/s]
Loading 0:  64%|██████▎   | 231/363 [00:31<00:16,  7.77it/s]
Loading 0:  65%|██████▍   | 235/363 [00:32<00:11, 11.01it/s]
Loading 0:  66%|██████▌   | 238/363 [00:32<00:09, 12.90it/s]
Loading 0:  66%|██████▋   | 241/363 [00:32<00:08, 14.33it/s]
Loading 0:  67%|██████▋   | 244/363 [00:32<00:07, 16.60it/s]
Loading 0:  68%|██████▊   | 247/363 [00:32<00:06, 17.74it/s]
Loading 0:  69%|██████▉   | 250/363 [00:32<00:06, 18.03it/s]
Loading 0:  70%|██████▉   | 253/363 [00:32<00:05, 19.27it/s]
Loading 0:  71%|███████   | 256/363 [00:33<00:05, 19.59it/s]
Loading 0:  71%|███████▏  | 259/363 [00:33<00:05, 19.39it/s]
Loading 0:  72%|███████▏  | 262/363 [00:33<00:04, 21.52it/s]
Loading 0:  73%|███████▎  | 266/363 [00:33<00:03, 25.09it/s]
Loading 0:  74%|███████▍  | 269/363 [00:33<00:06, 15.48it/s]
Loading 0:  75%|███████▍  | 272/363 [00:34<00:05, 15.89it/s]
Loading 0:  75%|███████▌  | 274/363 [00:34<00:05, 15.28it/s]
Loading 0:  76%|███████▌  | 276/363 [00:34<00:05, 14.55it/s]
Loading 0:  77%|███████▋  | 280/363 [00:34<00:04, 18.12it/s]
Loading 0:  78%|███████▊  | 283/363 [00:34<00:04, 18.81it/s]
Loading 0:  79%|███████▉  | 286/363 [00:34<00:04, 19.13it/s]
Loading 0:  80%|███████▉  | 289/363 [00:34<00:03, 21.35it/s]
Loading 0:  80%|████████  | 292/363 [00:35<00:03, 21.20it/s]
Loading 0:  81%|████████▏ | 295/363 [00:35<00:03, 20.28it/s]
Loading 0:  82%|████████▏ | 298/363 [00:35<00:02, 21.94it/s]
Loading 0:  83%|████████▎ | 303/363 [00:35<00:02, 24.13it/s]
Loading 0:  84%|████████▍ | 306/363 [00:35<00:03, 14.55it/s]
Loading 0:  85%|████████▌ | 310/363 [00:36<00:03, 17.49it/s]
Loading 0:  86%|████████▌ | 313/363 [00:36<00:02, 17.74it/s]
Loading 0:  87%|████████▋ | 316/363 [00:36<00:02, 19.33it/s]
Loading 0:  88%|████████▊ | 319/363 [00:36<00:02, 19.97it/s]
Loading 0:  89%|████████▊ | 322/363 [00:36<00:02, 19.38it/s]
Loading 0:  90%|████████▉ | 325/363 [00:36<00:01, 20.86it/s]
Loading 0:  90%|█████████ | 328/363 [00:36<00:01, 21.02it/s]
Loading 0:  91%|█████████ | 331/363 [00:37<00:01, 20.55it/s]
Loading 0:  92%|█████████▏| 334/363 [00:37<00:01, 22.49it/s]
Loading 0:  93%|█████████▎| 337/363 [00:37<00:01, 19.15it/s]
Loading 0:  94%|█████████▎| 340/363 [00:37<00:01, 20.64it/s]
Loading 0:  94%|█████████▍| 343/363 [00:37<00:01, 11.62it/s]
Loading 0:  96%|█████████▌| 347/363 [00:38<00:01, 14.76it/s]
Loading 0:  96%|█████████▋| 350/363 [00:38<00:00, 15.80it/s]
Loading 0:  97%|█████████▋| 353/363 [00:38<00:00, 17.73it/s]
Loading 0:  98%|█████████▊| 356/363 [00:38<00:00, 19.10it/s]
Loading 0:  99%|█████████▉| 359/363 [00:38<00:00, 19.16it/s]
Loading 0: 100%|█████████▉| 362/363 [00:38<00:00, 21.25it/s]
                                                            
Job chaiml-severus-snape2507-3897-v1-mkmlizer completed after 219.16s with status: succeeded
Stopping job with name chaiml-severus-snape2507-3897-v1-mkmlizer
Pipeline stage MKMLizer completed in 220.23s
run pipeline stage %s
Running pipeline stage MKMLTemplater
Pipeline stage MKMLTemplater completed in 0.18s
run pipeline stage %s
Running pipeline stage MKMLDeployer
Creating inference service chaiml-severus-snape2507-3897-v1
Waiting for inference service chaiml-severus-snape2507-3897-v1 to be ready
Failed to get response for submission blend_hunen_2025-06-23: HTTPConnectionPool(host='guanaco-model-mesh.k2.chaiverse.com', port=80): Read timed out. (read timeout=12.0)
Inference service chaiml-severus-snape2507-3897-v1 ready after 191.22908449172974s
Pipeline stage MKMLDeployer completed in 192.10s
run pipeline stage %s
Running pipeline stage StressChecker
HTTPConnectionPool(host='guanaco-submitter.guanaco-backend.k2.chaiverse.com', port=80): Read timed out. (read timeout=20)
Received unhealthy response to inference request!
Received healthy response to inference request in 3.681335926055908s
Received healthy response to inference request in 2.0188026428222656s
Received healthy response to inference request in 2.129697799682617s
Received healthy response to inference request in 2.524879217147827s
5 requests
1 failed requests
5th percentile: 2.040981674194336
10th percentile: 2.063160705566406
20th percentile: 2.107518768310547
30th percentile: 2.208734083175659
40th percentile: 2.366806650161743
50th percentile: 2.524879217147827
60th percentile: 2.9874619007110597
70th percentile: 3.4500445842742917
80th percentile: 6.978257989883426
90th percentile: 13.572102117538453
95th percentile: 16.869024181365965
99th percentile: 19.506561832427977
mean time: 6.10413236618042
%s, retrying in %s seconds...
Received healthy response to inference request in 2.1961660385131836s
Received healthy response to inference request in 2.4846036434173584s
Received healthy response to inference request in 2.4059767723083496s
Received healthy response to inference request in 2.1175873279571533s
Received healthy response to inference request in 2.417222738265991s
5 requests
0 failed requests
5th percentile: 2.1333030700683593
10th percentile: 2.1490188121795653
20th percentile: 2.1804502964019776
30th percentile: 2.238128185272217
40th percentile: 2.322052478790283
50th percentile: 2.4059767723083496
60th percentile: 2.410475158691406
70th percentile: 2.4149735450744627
80th percentile: 2.4306989192962645
90th percentile: 2.4576512813568114
95th percentile: 2.471127462387085
99th percentile: 2.4819084072113036
mean time: 2.324311304092407
Pipeline stage StressChecker completed in 45.01s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyTriggerPipeline
run_pipeline:run_in_cloud %s
starting trigger_guanaco_pipeline args=%s
triggered trigger_guanaco_pipeline args=%s
Pipeline stage OfflineFamilyFriendlyTriggerPipeline completed in 0.73s
run pipeline stage %s
Running pipeline stage TriggerMKMLProfilingPipeline
run_pipeline:run_in_cloud %s
starting trigger_guanaco_pipeline args=%s
triggered trigger_guanaco_pipeline args=%s
Pipeline stage TriggerMKMLProfilingPipeline completed in 0.70s
Shutdown handler de-registered
chaiml-severus-snape2507_3897_v1 status is now deployed due to DeploymentManager action
Shutdown handler registered
run pipeline %s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyScorer
Evaluating %s Family Friendly Score with %s threads
Pipeline stage OfflineFamilyFriendlyScorer completed in 6659.20s
Shutdown handler de-registered
chaiml-severus-snape2507_3897_v1 status is now inactive due to auto deactivation removed underperforming models
chaiml-severus-snape2507_3897_v1 status is now protected
chaiml-severus-snape2507_3897_v1 status is now torndown due to DeploymentManager action
chaiml-severus-snape2507_3897_v1 status is now torndown due to DeploymentManager action
chaiml-severus-snape2507_3897_v1 status is now torndown due to DeploymentManager action