chaiml-felix-mafia-gang_99878

developer_uid: richhx

submission_id: chaiml-felix-mafia-gang_99878_v2

model_name: chaiml-felix-mafia-gang_99878_v2

model_group: ChaiML/Felix-Mafia-gang-

status: torndown

timestamp: 2025-06-26T18:30:20+00:00

num_battles: 7253

num_wins: 3334

celo_rating: 1255.06

family_friendly_score: 0.5658000000000001

family_friendly_standard_error: 0.007009570029609519

submission_type: basic

model_repo: ChaiML/Felix-Mafia-gang-leader_Felix-Mafia-ga250518164255_preference_dpo

model_architecture: MistralForCausalLM

model_num_parameters: 24096691200.0

best_of: 8

max_input_tokens: 1024

max_output_tokens: 64

reward_model: default

latencies: [{'batch_size': 1, 'throughput': 0.5216800578112445, 'latency_mean': 1.9167616224288941, 'latency_p50': 1.913264274597168, 'latency_p90': 2.125426459312439}, {'batch_size': 3, 'throughput': 1.0541219490335603, 'latency_mean': 2.830116435289383, 'latency_p50': 2.839446187019348, 'latency_p90': 3.1879714727401733}, {'batch_size': 5, 'throughput': 1.3415498274865258, 'latency_mean': 3.7088707447052003, 'latency_p50': 3.7139538526535034, 'latency_p90': 4.146053981781006}, {'batch_size': 6, 'throughput': 1.4370289014680333, 'latency_mean': 4.133045698404312, 'latency_p50': 4.167832612991333, 'latency_p90': 4.536510586738586}, {'batch_size': 8, 'throughput': 1.6006543385348526, 'latency_mean': 4.966330374479294, 'latency_p50': 4.926282525062561, 'latency_p90': 5.596068525314331}, {'batch_size': 10, 'throughput': 1.6881348904582196, 'latency_mean': 5.887141302824021, 'latency_p50': 5.9188573360443115, 'latency_p90': 6.693570756912232}]

gpu_counts: {'NVIDIA A100-SXM4-80GB': 1}

display_name: chaiml-felix-mafia-gang_99878_v2

ineligible_reason: num_battles<10000

is_internal_developer: True

language_model: ChaiML/Felix-Mafia-gang-leader_Felix-Mafia-ga250518164255_preference_dpo

model_size: 24B

ranking_group: single

throughput_3p7s: 1.35

us_pacific_date: 2025-06-26

win_ratio: 0.45967185992003307

generation_params: {'temperature': 0.5, 'top_p': 0.95, 'min_p': 0.02, 'top_k': 60, 'presence_penalty': 0.2, 'frequency_penalty': 0.2, 'stopping_words': ['####\n', 'You:', '####', '\n', '</s>'], 'max_input_tokens': 1024, 'best_of': 8, 'max_output_tokens': 64}

formatter: {'memory_template': '', 'prompt_template': '', 'bot_template': '{bot_name}: {message}</s>\n', 'user_template': 'You: {message}\n', 'response_template': '{bot_name}:', 'truncate_by_message': True}

Resubmit model

Shutdown handler not registered because Python interpreter is not running in the main thread
run pipeline %s
run pipeline stage %s
Running pipeline stage MKMLizer
Starting job with name chaiml-felix-mafia-gang-99878-v2-mkmlizer
Waiting for job on chaiml-felix-mafia-gang-99878-v2-mkmlizer to finish
chaiml-felix-mafia-gang-99878-v2-mkmlizer: ╔═════════════════════════════════════════════════════════════════════╗
chaiml-felix-mafia-gang-99878-v2-mkmlizer: ║                                                                     ║
chaiml-felix-mafia-gang-99878-v2-mkmlizer: ║   ██████   ██████  █████   ████  ████                               ║
chaiml-felix-mafia-gang-99878-v2-mkmlizer: ║  ░░██████ ██████  ░░███   ███░  ░░███                               ║
chaiml-felix-mafia-gang-99878-v2-mkmlizer: ║   ░███░█████░███   ░███  ███     ░███                               ║
chaiml-felix-mafia-gang-99878-v2-mkmlizer: ║   ░███░░███ ░███   ░███████      ░███                               ║
chaiml-felix-mafia-gang-99878-v2-mkmlizer: ║   ░███ ░░░  ░███   ░███░░███     ░███                               ║
chaiml-felix-mafia-gang-99878-v2-mkmlizer: ║   ░███      ░███   ░███ ░░███    ░███                               ║
chaiml-felix-mafia-gang-99878-v2-mkmlizer: ║   █████     █████  █████ ░░████  █████                              ║
chaiml-felix-mafia-gang-99878-v2-mkmlizer: ║  ░░░░░     ░░░░░  ░░░░░   ░░░░  ░░░░░                               ║
chaiml-felix-mafia-gang-99878-v2-mkmlizer: ║                                                                     ║
chaiml-felix-mafia-gang-99878-v2-mkmlizer: ║  Version: 0.29.3                                                    ║
chaiml-felix-mafia-gang-99878-v2-mkmlizer: ║  Features: FLYWHEEL, CUDA                                           ║
chaiml-felix-mafia-gang-99878-v2-mkmlizer: ║  Copyright 2023-2025 MK ONE TECHNOLOGIES Inc.                       ║
chaiml-felix-mafia-gang-99878-v2-mkmlizer: ║  https://mk1.ai                                                     ║
chaiml-felix-mafia-gang-99878-v2-mkmlizer: ║                                                                     ║
chaiml-felix-mafia-gang-99878-v2-mkmlizer: ║  The license key for the current software has been verified as      ║
chaiml-felix-mafia-gang-99878-v2-mkmlizer: ║  belonging to:                                                      ║
chaiml-felix-mafia-gang-99878-v2-mkmlizer: ║                                                                     ║
chaiml-felix-mafia-gang-99878-v2-mkmlizer: ║  Chai Research Corp.                                                ║
chaiml-felix-mafia-gang-99878-v2-mkmlizer: ║  Account ID: 7997a29f-0ceb-4cc7-9adf-840c57b4ae6f                   ║
chaiml-felix-mafia-gang-99878-v2-mkmlizer: ║  Expiration: 2028-03-31 23:59:59                                    ║
chaiml-felix-mafia-gang-99878-v2-mkmlizer: ║                                                                     ║
chaiml-felix-mafia-gang-99878-v2-mkmlizer: ╚═════════════════════════════════════════════════════════════════════╝
chaiml-felix-mafia-gang-99878-v2-mkmlizer: Downloaded to shared memory in 115.323s
chaiml-felix-mafia-gang-99878-v2-mkmlizer: Checking if ChaiML/Felix-Mafia-gang-leader_Felix-Mafia-ga250518164255_preference_dpo already exists in ChaiML
chaiml-felix-mafia-gang-99878-v2-mkmlizer: quantizing model to /dev/shm/model_cache, profile:s0, folder:/tmp/tmpcmx9d_qw, device:0
chaiml-felix-mafia-gang-99878-v2-mkmlizer: Saving flywheel model at /dev/shm/model_cache
chaiml-felix-mafia-gang-99878-v2-mkmlizer: quantized model in 48.590s
chaiml-felix-mafia-gang-99878-v2-mkmlizer: Processed model ChaiML/Felix-Mafia-gang-leader_Felix-Mafia-ga250518164255_preference_dpo in 163.914s
chaiml-felix-mafia-gang-99878-v2-mkmlizer: creating bucket guanaco-mkml-models
chaiml-felix-mafia-gang-99878-v2-mkmlizer: cp /dev/shm/model_cache/special_tokens_map.json s3://guanaco-mkml-models/chaiml-felix-mafia-gang-99878-v2/special_tokens_map.json
chaiml-felix-mafia-gang-99878-v2-mkmlizer: cp /dev/shm/model_cache/config.json s3://guanaco-mkml-models/chaiml-felix-mafia-gang-99878-v2/config.json
chaiml-felix-mafia-gang-99878-v2-mkmlizer: cp /dev/shm/model_cache/tokenizer_config.json s3://guanaco-mkml-models/chaiml-felix-mafia-gang-99878-v2/tokenizer_config.json
chaiml-felix-mafia-gang-99878-v2-mkmlizer: cp /dev/shm/model_cache/tokenizer.json s3://guanaco-mkml-models/chaiml-felix-mafia-gang-99878-v2/tokenizer.json
chaiml-felix-mafia-gang-99878-v2-mkmlizer: cp /dev/shm/model_cache/flywheel_model.1.safetensors s3://guanaco-mkml-models/chaiml-felix-mafia-gang-99878-v2/flywheel_model.1.safetensors
chaiml-felix-mafia-gang-99878-v2-mkmlizer: cp /dev/shm/model_cache/flywheel_model.0.safetensors s3://guanaco-mkml-models/chaiml-felix-mafia-gang-99878-v2/flywheel_model.0.safetensors
chaiml-felix-mafia-gang-99878-v2-mkmlizer: 
Loading 0:   0%|          | 0/363 [00:00<?, ?it/s]
Loading 0:   1%|▏         | 5/363 [00:00<00:17, 20.76it/s]
Loading 0:   3%|▎         | 12/363 [00:00<00:09, 36.12it/s]
Loading 0:   5%|▍         | 17/363 [00:00<00:10, 32.56it/s]
Loading 0:   6%|▌         | 21/363 [00:00<00:10, 33.02it/s]
Loading 0:   7%|▋         | 25/363 [00:00<00:11, 29.82it/s]
Loading 0:   9%|▉         | 32/363 [00:00<00:09, 35.55it/s]
Loading 0:  10%|▉         | 36/363 [00:01<00:15, 21.53it/s]
Loading 0:  11%|█         | 40/363 [00:01<00:13, 24.16it/s]
Loading 0:  12%|█▏        | 44/363 [00:01<00:12, 25.61it/s]
Loading 0:  13%|█▎        | 48/363 [00:01<00:11, 27.61it/s]
Loading 0:  14%|█▍        | 52/363 [00:01<00:11, 26.45it/s]
Loading 0:  16%|█▌        | 57/363 [00:02<00:10, 29.50it/s]
Loading 0:  17%|█▋        | 61/363 [00:02<00:10, 28.02it/s]
Loading 0:  18%|█▊        | 65/363 [00:02<00:10, 28.44it/s]
Loading 0:  19%|█▉        | 70/363 [00:02<00:11, 24.56it/s]
Loading 0:  20%|██        | 73/363 [00:02<00:13, 21.26it/s]
Loading 0:  22%|██▏       | 79/363 [00:02<00:10, 26.07it/s]
Loading 0:  23%|██▎       | 82/363 [00:03<00:11, 24.51it/s]
Loading 0:  24%|██▎       | 86/363 [00:03<00:10, 26.33it/s]
Loading 0:  25%|██▍       | 89/363 [00:03<00:10, 25.68it/s]
Loading 0:  25%|██▌       | 92/363 [00:03<00:13, 20.57it/s]
Loading 0:  27%|██▋       | 99/363 [00:03<00:10, 26.34it/s]
Loading 0:  28%|██▊       | 102/363 [00:03<00:10, 23.89it/s]
Loading 0:  29%|██▉       | 107/363 [00:04<00:11, 21.66it/s]
Loading 0:  31%|███       | 111/363 [00:04<00:10, 23.41it/s]
Loading 0:  31%|███▏      | 114/363 [00:04<00:12, 20.70it/s]
Loading 0:  33%|███▎      | 120/363 [00:04<00:09, 25.55it/s]
Loading 0:  34%|███▍      | 123/363 [00:04<00:10, 23.75it/s]
Loading 0:  36%|███▌      | 129/363 [00:04<00:08, 27.64it/s]
Loading 0:  36%|███▋      | 132/363 [00:05<00:09, 24.97it/s]
Loading 0:  38%|███▊      | 138/363 [00:05<00:07, 29.05it/s]
Loading 0:  39%|███▉      | 141/363 [00:05<00:08, 25.10it/s]
Loading 0:  40%|███▉      | 145/363 [00:05<00:07, 28.10it/s]
Loading 0:  41%|████▏     | 150/363 [00:05<00:08, 24.18it/s]
Loading 0:  42%|████▏     | 153/363 [00:06<00:10, 19.29it/s]
Loading 0:  43%|████▎     | 157/363 [00:06<00:09, 21.17it/s]
Loading 0:  44%|████▍     | 160/363 [00:06<00:09, 21.85it/s]
Loading 0:  45%|████▍     | 163/363 [00:06<00:09, 22.13it/s]
Loading 0:  46%|████▌     | 166/363 [00:06<00:08, 22.13it/s]
Loading 0:  47%|████▋     | 169/363 [00:06<00:08, 22.29it/s]
Loading 0:  47%|████▋     | 172/363 [00:06<00:08, 23.75it/s]
Loading 0:  48%|████▊     | 175/363 [00:07<00:08, 23.45it/s]
Loading 0:  49%|████▉     | 178/363 [00:07<00:07, 23.53it/s]
Loading 0:  50%|█████     | 182/363 [00:07<00:07, 25.01it/s]
Loading 0:  52%|█████▏    | 187/363 [00:07<00:07, 22.60it/s]
Loading 0:  52%|█████▏    | 190/363 [00:07<00:08, 21.47it/s]
Loading 0:  53%|█████▎    | 193/363 [00:07<00:07, 22.93it/s]
Loading 0:  54%|█████▍    | 196/363 [00:07<00:07, 23.74it/s]
Loading 0:  55%|█████▌    | 200/363 [00:22<00:06, 23.74it/s]
Loading 0:  55%|█████▌    | 201/363 [00:22<03:10,  1.18s/it]
Loading 0:  56%|█████▌    | 203/363 [00:22<02:36,  1.02it/s]
Loading 0:  57%|█████▋    | 208/363 [00:22<01:33,  1.66it/s]
Loading 0:  58%|█████▊    | 211/363 [00:23<01:11,  2.13it/s]
Loading 0:  59%|█████▉    | 214/363 [00:23<00:52,  2.82it/s]
Loading 0:  60%|██████    | 218/363 [00:23<00:35,  4.03it/s]
Loading 0:  61%|██████    | 221/363 [00:23<00:27,  5.15it/s]
Loading 0:  62%|██████▏   | 224/363 [00:23<00:22,  6.12it/s]
Loading 0:  63%|██████▎   | 229/363 [00:23<00:14,  9.06it/s]
Loading 0:  64%|██████▍   | 232/363 [00:24<00:12, 10.70it/s]
Loading 0:  65%|██████▌   | 237/363 [00:24<00:08, 14.27it/s]
Loading 0:  66%|██████▌   | 240/363 [00:24<00:08, 15.02it/s]
Loading 0:  68%|██████▊   | 246/363 [00:24<00:05, 20.10it/s]
Loading 0:  69%|██████▊   | 249/363 [00:24<00:05, 19.88it/s]
Loading 0:  70%|███████   | 255/363 [00:24<00:04, 24.68it/s]
Loading 0:  71%|███████   | 258/363 [00:24<00:04, 23.14it/s]
Loading 0:  73%|███████▎  | 264/363 [00:25<00:03, 29.78it/s]
Loading 0:  74%|███████▍  | 268/363 [00:25<00:04, 22.23it/s]
Loading 0:  75%|███████▍  | 271/363 [00:25<00:04, 20.92it/s]
Loading 0:  75%|███████▌  | 274/363 [00:25<00:03, 22.29it/s]
Loading 0:  76%|███████▋  | 277/363 [00:25<00:03, 22.70it/s]
Loading 0:  77%|███████▋  | 281/363 [00:25<00:03, 26.44it/s]
Loading 0:  78%|███████▊  | 284/363 [00:26<00:03, 21.56it/s]
Loading 0:  80%|████████  | 291/363 [00:26<00:02, 28.93it/s]
Loading 0:  81%|████████▏ | 295/363 [00:26<00:02, 27.42it/s]
Loading 0:  82%|████████▏ | 299/363 [00:26<00:02, 27.57it/s]
Loading 0:  84%|████████▎ | 304/363 [00:26<00:02, 24.00it/s]
Loading 0:  85%|████████▍ | 307/363 [00:26<00:02, 22.08it/s]
Loading 0:  85%|████████▌ | 310/363 [00:27<00:02, 23.35it/s]
Loading 0:  86%|████████▌ | 313/363 [00:27<00:02, 23.37it/s]
Loading 0:  87%|████████▋ | 316/363 [00:27<00:01, 24.72it/s]
Loading 0:  88%|████████▊ | 319/363 [00:27<00:01, 25.63it/s]
Loading 0:  89%|████████▊ | 322/363 [00:27<00:01, 25.46it/s]
Loading 0:  90%|█████████ | 327/363 [00:27<00:01, 28.34it/s]
Loading 0:  91%|█████████ | 330/363 [00:27<00:01, 24.91it/s]
Loading 0:  92%|█████████▏| 335/363 [00:27<00:01, 27.75it/s]
Loading 0:  93%|█████████▎| 338/363 [00:28<00:00, 26.37it/s]
Loading 0:  94%|█████████▍| 341/363 [00:28<00:01, 14.83it/s]
Loading 0:  96%|█████████▌| 347/363 [00:28<00:00, 20.48it/s]
Loading 0:  96%|█████████▋| 350/363 [00:28<00:00, 21.13it/s]
Loading 0:  97%|█████████▋| 353/363 [00:28<00:00, 22.73it/s]
Loading 0:  98%|█████████▊| 357/363 [00:29<00:00, 21.52it/s]
                                                            
Job chaiml-felix-mafia-gang-99878-v2-mkmlizer completed after 196.3s with status: succeeded
Stopping job with name chaiml-felix-mafia-gang-99878-v2-mkmlizer
Pipeline stage MKMLizer completed in 196.81s
run pipeline stage %s
Running pipeline stage MKMLTemplater
Pipeline stage MKMLTemplater completed in 0.15s
run pipeline stage %s
Running pipeline stage MKMLDeployer
Creating inference service chaiml-felix-mafia-gang-99878-v2
Waiting for inference service chaiml-felix-mafia-gang-99878-v2 to be ready
Inference service chaiml-felix-mafia-gang-99878-v2 ready after 150.6078760623932s
Pipeline stage MKMLDeployer completed in 151.10s
run pipeline stage %s
Running pipeline stage StressChecker
HTTPConnectionPool(host='guanaco-submitter.guanaco-backend.k2.chaiverse.com', port=80): Read timed out. (read timeout=20)
Received unhealthy response to inference request!
Received healthy response to inference request in 3.0827271938323975s
Received healthy response to inference request in 1.8553352355957031s
Received healthy response to inference request in 1.9870433807373047s
Received healthy response to inference request in 1.7005608081817627s
5 requests
1 failed requests
5th percentile: 1.7315156936645508
10th percentile: 1.762470579147339
20th percentile: 1.824380350112915
30th percentile: 1.8816768646240234
40th percentile: 1.9343601226806642
50th percentile: 1.9870433807373047
60th percentile: 2.425316905975342
70th percentile: 2.8635904312133786
80th percentile: 6.4920728683471705
90th percentile: 13.310764217376711
95th percentile: 16.720109891891475
99th percentile: 19.447586431503296
mean time: 5.751024436950684
%s, retrying in %s seconds...
Received healthy response to inference request in 1.5789871215820312s
Received healthy response to inference request in 2.035428762435913s
Received healthy response to inference request in 1.554931402206421s
Received healthy response to inference request in 1.7222392559051514s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Received healthy response to inference request in 1.8977878093719482s
5 requests
0 failed requests
5th percentile: 1.559742546081543
10th percentile: 1.564553689956665
20th percentile: 1.574175977706909
30th percentile: 1.6076375484466552
40th percentile: 1.6649384021759033
50th percentile: 1.7222392559051514
60th percentile: 1.79245867729187
70th percentile: 1.862678098678589
80th percentile: 1.9253159999847413
90th percentile: 1.980372381210327
95th percentile: 2.00790057182312
99th percentile: 2.0299231243133544
mean time: 1.7578748703002929
Pipeline stage StressChecker completed in 40.05s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyTriggerPipeline
run_pipeline:run_in_cloud %s
starting trigger_guanaco_pipeline args=%s
triggered trigger_guanaco_pipeline args=%s
Pipeline stage OfflineFamilyFriendlyTriggerPipeline completed in 0.72s
run pipeline stage %s
Running pipeline stage TriggerMKMLProfilingPipeline
run_pipeline:run_in_cloud %s
starting trigger_guanaco_pipeline args=%s
triggered trigger_guanaco_pipeline args=%s
Pipeline stage TriggerMKMLProfilingPipeline completed in 0.69s
Shutdown handler de-registered
chaiml-felix-mafia-gang_99878_v2 status is now deployed due to DeploymentManager action
Shutdown handler registered
run pipeline %s
run pipeline stage %s
Running pipeline stage MKMLProfilerDeleter
Skipping teardown as no inference service was successfully deployed
Pipeline stage MKMLProfilerDeleter completed in 0.14s
run pipeline stage %s
Running pipeline stage MKMLProfilerTemplater
Pipeline stage MKMLProfilerTemplater completed in 0.14s
run pipeline stage %s
Running pipeline stage MKMLProfilerDeployer
Creating inference service chaiml-felix-mafia-gang-99878-v2-profiler
Waiting for inference service chaiml-felix-mafia-gang-99878-v2-profiler to be ready
Shutdown handler registered
run pipeline %s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyScorer
Evaluating %s Family Friendly Score with %s threads
Pipeline stage OfflineFamilyFriendlyScorer completed in 3402.28s
Shutdown handler de-registered
chaiml-felix-mafia-gang_99878_v2 status is now inactive due to auto deactivation removed underperforming models
chaiml-felix-mafia-gang_99878_v2 status is now torndown due to DeploymentManager action