chaiml-1111-quang-ir-mi_73541

developer_uid: chai_backend_admin

submission_id: chaiml-1111-quang-ir-mi_73541_v1

model_name: training123

model_group: ChaiML/1111-quang-ir-mix

status: torndown

timestamp: 2025-11-14T20:31:26+00:00

num_battles: 5163

num_wins: 2630

celo_rating: 1292.05

family_friendly_score: 0.0

family_friendly_standard_error: 0.0

submission_type: basic

model_repo: ChaiML/1111-quang-ir-mixall-cld-i0a0-freev2-f10000-dpo_freev22_0164641ep_2e6ct

model_architecture: MistralForCausalLM

model_num_parameters: 24096691200.0

best_of: 8

max_input_tokens: 2048

max_output_tokens: 64

reward_model: default

latencies: [{'batch_size': 1, 'throughput': 0.37971987836981763, 'latency_mean': 2.6334165596961974, 'latency_p50': 2.6249266862869263, 'latency_p90': 2.891482353210449}, {'batch_size': 2, 'throughput': 0.5698213511234425, 'latency_mean': 3.5019192600250246, 'latency_p50': 3.508625030517578, 'latency_p90': 3.7429733514785766}, {'batch_size': 3, 'throughput': 0.7048385826070753, 'latency_mean': 4.244010131359101, 'latency_p50': 4.230258584022522, 'latency_p90': 4.634623908996581}, {'batch_size': 4, 'throughput': 0.7979401979996071, 'latency_mean': 4.990027806758881, 'latency_p50': 4.975666880607605, 'latency_p90': 5.758975505828857}, {'batch_size': 5, 'throughput': 0.8587836857991561, 'latency_mean': 5.794435861110688, 'latency_p50': 5.775079369544983, 'latency_p90': 6.512025690078735}]

gpu_counts: {'NVIDIA L40S': 1}

display_name: training123

is_internal_developer: True

language_model: ChaiML/1111-quang-ir-mixall-cld-i0a0-freev2-f10000-dpo_freev22_0164641ep_2e6ct

model_size: 24B

ranking_group: single

throughput_3p7s: 0.61

us_pacific_date: 2025-11-11

win_ratio: 0.5093937633159016

generation_params: {'temperature': 1.0, 'top_p': 1.0, 'min_p': 0.0, 'top_k': 80, 'presence_penalty': 0.0, 'frequency_penalty': 0.0, 'stopping_words': ['You:', '####', '\n', '</s>'], 'max_input_tokens': 2048, 'best_of': 8, 'max_output_tokens': 64}

formatter: {'memory_template': "{bot_name}'s persona: {memory}", 'prompt_template': '', 'bot_template': '{bot_name}: {message}\n', 'user_template': 'You: {message}\n', 'response_template': '####\n{bot_name}:', 'truncate_by_message': True}

Resubmit model

Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Shutdown handler not registered because Python interpreter is not running in the main thread
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
run pipeline %s
run pipeline stage %s
Running pipeline stage MKMLizer
Starting job with name chaiml-1111-quang-ir-mi-73541-v1-mkmlizer
Waiting for job on chaiml-1111-quang-ir-mi-73541-v1-mkmlizer to finish
chaiml-1111-quang-ir-mi-73541-v1-mkmlizer: ╔═════════════════════════════════════════════════════════════════════╗
chaiml-1111-quang-ir-mi-73541-v1-mkmlizer: ║                                                                     ║
chaiml-1111-quang-ir-mi-73541-v1-mkmlizer: ║   ██████   ██████  █████   ████  ████                               ║
chaiml-1111-quang-ir-mi-73541-v1-mkmlizer: ║  ░░██████ ██████  ░░███   ███░  ░░███                               ║
chaiml-1111-quang-ir-mi-73541-v1-mkmlizer: ║   ░███░█████░███   ░███  ███     ░███                               ║
chaiml-1111-quang-ir-mi-73541-v1-mkmlizer: ║   ░███░░███ ░███   ░███████      ░███                               ║
chaiml-1111-quang-ir-mi-73541-v1-mkmlizer: ║   ░███ ░░░  ░███   ░███░░███     ░███                               ║
chaiml-1111-quang-ir-mi-73541-v1-mkmlizer: ║   ░███      ░███   ░███ ░░███    ░███                               ║
chaiml-1111-quang-ir-mi-73541-v1-mkmlizer: ║   █████     █████  █████ ░░████  █████                              ║
chaiml-1111-quang-ir-mi-73541-v1-mkmlizer: ║  ░░░░░     ░░░░░  ░░░░░   ░░░░  ░░░░░                               ║
chaiml-1111-quang-ir-mi-73541-v1-mkmlizer: ║                                                                     ║
chaiml-1111-quang-ir-mi-73541-v1-mkmlizer: ║  Version: 0.30.2                                                    ║
chaiml-1111-quang-ir-mi-73541-v1-mkmlizer: ║  Features: FLYWHEEL, CUDA                                           ║
chaiml-1111-quang-ir-mi-73541-v1-mkmlizer: ║  Copyright 2023-2025 MK ONE TECHNOLOGIES Inc.                       ║
chaiml-1111-quang-ir-mi-73541-v1-mkmlizer: ║  https://mk1.ai                                                     ║
chaiml-1111-quang-ir-mi-73541-v1-mkmlizer: ║                                                                     ║
chaiml-1111-quang-ir-mi-73541-v1-mkmlizer: ║  The license key for the current software has been verified as      ║
chaiml-1111-quang-ir-mi-73541-v1-mkmlizer: ║  belonging to:                                                      ║
chaiml-1111-quang-ir-mi-73541-v1-mkmlizer: ║                                                                     ║
chaiml-1111-quang-ir-mi-73541-v1-mkmlizer: ║  Chai Research Corp.                                                ║
chaiml-1111-quang-ir-mi-73541-v1-mkmlizer: ║  Account ID: 7997a29f-0ceb-4cc7-9adf-840c57b4ae6f                   ║
chaiml-1111-quang-ir-mi-73541-v1-mkmlizer: ║  Expiration: 2028-03-31 23:59:59                                    ║
chaiml-1111-quang-ir-mi-73541-v1-mkmlizer: ║                                                                     ║
chaiml-1111-quang-ir-mi-73541-v1-mkmlizer: ╚═════════════════════════════════════════════════════════════════════╝
chaiml-1111-quang-ir-mi-73541-v1-mkmlizer: Downloaded to shared memory in 48.342s
chaiml-1111-quang-ir-mi-73541-v1-mkmlizer: Checking if ChaiML/1111-quang-ir-mixall-cld-i0a0-freev2-f10000-dpo_freev22_0164641ep_2e6ct already exists in ChaiML
chaiml-1111-quang-ir-mi-73541-v1-mkmlizer: quantizing model to /dev/shm/model_cache, profile:s0, folder:/tmp/tmpwwkifpag, device:0
chaiml-1111-quang-ir-mi-73541-v1-mkmlizer: Saving flywheel model at /dev/shm/model_cache
Failed to get response for submission chaiml-7b07-69d4-linear-w01_v7: ('http://guanaco-model-mesh-load-balancer.model-mesh.k2.chaiverse.com/models/chaiml-7b07-69d4-linear-w01_v7/predict', '{"detail":"1 validation error for RuntimeResponse\\npredictions\\n  Field required [type=missing, input_value={\'detail\': \\"503, message=...linear-w01_v7/predict\'\\"}, input_type=dict]\\n    For further information visit https://errors.pydantic.dev/2.11/v/missing"}')
chaiml-1111-quang-ir-mi-73541-v1-mkmlizer: quantized model in 42.555s
chaiml-1111-quang-ir-mi-73541-v1-mkmlizer: Processed model ChaiML/1111-quang-ir-mixall-cld-i0a0-freev2-f10000-dpo_freev22_0164641ep_2e6ct in 90.898s
chaiml-1111-quang-ir-mi-73541-v1-mkmlizer: creating bucket guanaco-mkml-models
chaiml-1111-quang-ir-mi-73541-v1-mkmlizer: Bucket 's3://guanaco-mkml-models/' created
chaiml-1111-quang-ir-mi-73541-v1-mkmlizer: uploading /dev/shm/model_cache to s3://guanaco-mkml-models/chaiml-1111-quang-ir-mi-73541-v1/nvidia
chaiml-1111-quang-ir-mi-73541-v1-mkmlizer: cp /dev/shm/model_cache/config.json s3://guanaco-mkml-models/chaiml-1111-quang-ir-mi-73541-v1/nvidia/config.json
chaiml-1111-quang-ir-mi-73541-v1-mkmlizer: cp /dev/shm/model_cache/special_tokens_map.json s3://guanaco-mkml-models/chaiml-1111-quang-ir-mi-73541-v1/nvidia/special_tokens_map.json
chaiml-1111-quang-ir-mi-73541-v1-mkmlizer: cp /dev/shm/model_cache/tokenizer_config.json s3://guanaco-mkml-models/chaiml-1111-quang-ir-mi-73541-v1/nvidia/tokenizer_config.json
chaiml-1111-quang-ir-mi-73541-v1-mkmlizer: cp /dev/shm/model_cache/tokenizer.json s3://guanaco-mkml-models/chaiml-1111-quang-ir-mi-73541-v1/nvidia/tokenizer.json
Failed to get response for submission mistralai-mistral-nem_93303_v569: ('http://mistralai-mistral-nem-93303-v569-predictor.tenant-chaiml-guanaco.k2.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', '')
chaiml-1111-quang-ir-mi-73541-v1-mkmlizer: cp /dev/shm/model_cache/flywheel_model.1.safetensors s3://guanaco-mkml-models/chaiml-1111-quang-ir-mi-73541-v1/nvidia/flywheel_model.1.safetensors
chaiml-1111-quang-ir-mi-73541-v1-mkmlizer: cp /dev/shm/model_cache/flywheel_model.0.safetensors s3://guanaco-mkml-models/chaiml-1111-quang-ir-mi-73541-v1/nvidia/flywheel_model.0.safetensors
chaiml-1111-quang-ir-mi-73541-v1-mkmlizer: 
Loading 0:   0%|          | 0/363 [00:00<?, ?it/s]
Loading 0:   1%|▏         | 5/363 [00:00<00:16, 22.19it/s]
Loading 0:   4%|▎         | 13/363 [00:00<00:09, 37.99it/s]
Loading 0:   5%|▍         | 18/363 [00:00<00:08, 41.10it/s]
Loading 0:   6%|▋         | 23/363 [00:00<00:10, 31.79it/s]
Loading 0:   9%|▉         | 32/363 [00:00<00:07, 44.09it/s]
Loading 0:  10%|█         | 38/363 [00:01<00:10, 30.05it/s]
Loading 0:  12%|█▏        | 42/363 [00:01<00:11, 27.19it/s]
Loading 0:  13%|█▎        | 48/363 [00:01<00:09, 32.42it/s]
Loading 0:  14%|█▍        | 52/363 [00:01<00:10, 31.07it/s]
Loading 0:  16%|█▌        | 57/363 [00:01<00:08, 34.44it/s]
Loading 0:  17%|█▋        | 61/363 [00:01<00:09, 31.81it/s]
Loading 0:  18%|█▊        | 65/363 [00:01<00:09, 33.01it/s]
Loading 0:  19%|█▉        | 70/363 [00:02<00:09, 29.45it/s]
Loading 0:  20%|██        | 74/363 [00:02<00:10, 26.40it/s]
Loading 0:  22%|██▏       | 79/363 [00:02<00:09, 30.45it/s]
Loading 0:  23%|██▎       | 83/363 [00:02<00:08, 31.92it/s]
Loading 0:  24%|██▍       | 87/363 [00:02<00:09, 28.10it/s]
Loading 0:  25%|██▌       | 92/363 [00:02<00:09, 27.50it/s]
Loading 0:  27%|██▋       | 99/363 [00:03<00:07, 34.89it/s]
Loading 0:  28%|██▊       | 103/363 [00:03<00:08, 32.45it/s]
Loading 0:  29%|██▉       | 107/363 [00:03<00:09, 28.41it/s]
Loading 0:  31%|███       | 112/363 [00:03<00:08, 31.08it/s]
Loading 0:  32%|███▏      | 116/363 [00:03<00:08, 30.08it/s]
Loading 0:  33%|███▎      | 120/363 [00:03<00:07, 32.18it/s]
Loading 0:  34%|███▍      | 124/363 [00:03<00:07, 30.91it/s]
Loading 0:  36%|███▌      | 129/363 [00:04<00:06, 34.37it/s]
Loading 0:  37%|███▋      | 133/363 [00:04<00:07, 32.46it/s]
Loading 0:  38%|███▊      | 138/363 [00:04<00:06, 35.68it/s]
Loading 0:  39%|███▉      | 142/363 [00:04<00:06, 33.33it/s]
Loading 0:  41%|████      | 149/363 [00:04<00:05, 39.63it/s]
Loading 0:  42%|████▏     | 154/363 [00:04<00:07, 27.73it/s]
Loading 0:  44%|████▎     | 158/363 [00:05<00:08, 25.47it/s]
Loading 0:  45%|████▌     | 165/363 [00:05<00:06, 32.67it/s]
Loading 0:  47%|████▋     | 169/363 [00:05<00:06, 31.57it/s]
Loading 0:  48%|████▊     | 174/363 [00:05<00:05, 34.18it/s]
Loading 0:  49%|████▉     | 178/363 [00:05<00:05, 32.18it/s]
Loading 0:  50%|█████     | 182/363 [00:05<00:05, 33.05it/s]
Loading 0:  52%|█████▏    | 187/363 [00:05<00:05, 30.54it/s]
Loading 0:  53%|█████▎    | 191/363 [00:06<00:05, 29.73it/s]
Loading 0:  54%|█████▎    | 195/363 [00:06<00:06, 26.67it/s]
Loading 0:  55%|█████▌    | 201/363 [00:19<02:15,  1.20it/s]
Loading 0:  56%|█████▌    | 203/363 [00:19<01:55,  1.38it/s]
Loading 0:  57%|█████▋    | 208/363 [00:19<01:14,  2.09it/s]
Loading 0:  58%|█████▊    | 211/363 [00:19<00:57,  2.64it/s]
Loading 0:  59%|█████▉    | 214/363 [00:19<00:43,  3.41it/s]
Loading 0:  60%|██████    | 218/363 [00:19<00:30,  4.81it/s]
Loading 0:  61%|██████    | 222/363 [00:20<00:21,  6.59it/s]
Loading 0:  62%|██████▏   | 226/363 [00:20<00:16,  8.23it/s]
Loading 0:  63%|██████▎   | 230/363 [00:20<00:13, 10.13it/s]
Loading 0:  65%|██████▌   | 237/363 [00:20<00:07, 15.80it/s]
Loading 0:  66%|██████▋   | 241/363 [00:20<00:06, 17.79it/s]
Loading 0:  68%|██████▊   | 246/363 [00:20<00:05, 21.90it/s]
Loading 0:  69%|██████▉   | 250/363 [00:21<00:04, 23.19it/s]
Loading 0:  70%|███████   | 255/363 [00:21<00:03, 27.18it/s]
Loading 0:  71%|███████▏  | 259/363 [00:21<00:03, 27.31it/s]
Loading 0:  73%|███████▎  | 266/363 [00:21<00:02, 34.22it/s]
Loading 0:  75%|███████▍  | 271/363 [00:21<00:03, 24.77it/s]
Loading 0:  76%|███████▌  | 275/363 [00:21<00:03, 23.51it/s]
Loading 0:  78%|███████▊  | 282/363 [00:22<00:02, 30.58it/s]
Loading 0:  79%|███████▉  | 286/363 [00:22<00:02, 29.79it/s]
Loading 0:  80%|████████  | 291/363 [00:22<00:02, 33.07it/s]
Loading 0:  81%|████████▏ | 295/363 [00:22<00:02, 30.84it/s]
Loading 0:  82%|████████▏ | 299/363 [00:22<00:02, 31.88it/s]
Loading 0:  84%|████████▎ | 304/363 [00:22<00:02, 29.34it/s]
Loading 0:  85%|████████▍ | 308/363 [00:23<00:02, 22.32it/s]
Loading 0:  86%|████████▌ | 311/363 [00:23<00:02, 19.20it/s]
Loading 0:  87%|████████▋ | 316/363 [00:23<00:02, 23.18it/s]
Loading 0:  88%|████████▊ | 320/363 [00:23<00:02, 20.46it/s]
Loading 0:  90%|████████▉ | 325/363 [00:23<00:01, 23.03it/s]
Loading 0:  90%|█████████ | 328/363 [00:24<00:01, 22.07it/s]
Loading 0:  91%|█████████ | 331/363 [00:24<00:01, 22.90it/s]
Loading 0:  92%|█████████▏| 335/363 [00:24<00:01, 25.59it/s]
Loading 0:  93%|█████████▎| 338/363 [00:24<00:00, 26.03it/s]
Loading 0:  94%|█████████▍| 341/363 [00:24<00:01, 16.33it/s]
Loading 0:  96%|█████████▌| 347/363 [00:24<00:00, 22.44it/s]
Loading 0:  96%|█████████▋| 350/363 [00:24<00:00, 23.37it/s]
Loading 0:  98%|█████████▊| 355/363 [00:25<00:00, 28.08it/s]
Loading 0:  99%|█████████▉| 359/363 [00:25<00:00, 27.83it/s]
                                                            
Job chaiml-1111-quang-ir-mi-73541-v1-mkmlizer completed after 165.26s with status: succeeded
Stopping job with name chaiml-1111-quang-ir-mi-73541-v1-mkmlizer
Pipeline stage MKMLizer completed in 166.91s
run pipeline stage %s
Running pipeline stage MKMLTemplater
Pipeline stage MKMLTemplater completed in 0.17s
run pipeline stage %s
Running pipeline stage MKMLDeployer
Creating inference service chaiml-1111-quang-ir-mi-73541-v1
Waiting for inference service chaiml-1111-quang-ir-mi-73541-v1 to be ready
Inference service chaiml-1111-quang-ir-mi-73541-v1 ready after 141.5829496383667s
Pipeline stage MKMLDeployer completed in 142.08s
run pipeline stage %s
Running pipeline stage StressChecker
HTTPConnectionPool(host='guanaco-submitter.guanaco-backend.k2.chaiverse.com', port=80): Read timed out. (read timeout=20)
Received unhealthy response to inference request!
Received healthy response to inference request in 2.7031466960906982s
Received healthy response to inference request in 2.134535074234009s
Received healthy response to inference request in 2.4244742393493652s
Received healthy response to inference request in 2.235393762588501s
5 requests
1 failed requests
5th percentile: 2.1547068119049073
10th percentile: 2.174878549575806
20th percentile: 2.2152220249176025
30th percentile: 2.2732098579406737
40th percentile: 2.3488420486450194
50th percentile: 2.4244742393493652
60th percentile: 2.5359432220458986
70th percentile: 2.6474122047424316
80th percentile: 6.268863105773929
90th percentile: 13.400295925140384
95th percentile: 16.966012334823606
99th percentile: 19.81858546257019
mean time: 6.005855703353882
%s, retrying in %s seconds...
Received healthy response to inference request in 2.2270781993865967s
Received healthy response to inference request in 2.4221715927124023s
Received healthy response to inference request in 2.1223034858703613s
Received healthy response to inference request in 3.8972811698913574s
Received healthy response to inference request in 2.0729658603668213s
5 requests
0 failed requests
5th percentile: 2.0828333854675294
10th percentile: 2.0927009105682375
20th percentile: 2.1124359607696532
30th percentile: 2.1432584285736085
40th percentile: 2.1851683139801024
50th percentile: 2.2270781993865967
60th percentile: 2.305115556716919
70th percentile: 2.3831529140472414
80th percentile: 2.7171935081481937
90th percentile: 3.3072373390197756
95th percentile: 3.602259254455566
99th percentile: 3.838276786804199
mean time: 2.548360061645508
Pipeline stage StressChecker completed in 45.68s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyTriggerPipeline
run_pipeline:run_in_cloud %s
starting trigger_guanaco_pipeline args=%s
triggered trigger_guanaco_pipeline args=%s
Pipeline stage OfflineFamilyFriendlyTriggerPipeline completed in 0.75s
run pipeline stage %s
Running pipeline stage TriggerMKMLProfilingPipeline
run_pipeline:run_in_cloud %s
starting trigger_guanaco_pipeline args=%s
triggered trigger_guanaco_pipeline args=%s
Pipeline stage TriggerMKMLProfilingPipeline completed in 0.79s
Shutdown handler de-registered
chaiml-1111-quang-ir-mi_73541_v1 status is now deployed due to DeploymentManager action
Shutdown handler registered
run pipeline %s
run pipeline stage %s
Running pipeline stage MKMLProfilerDeleter
Skipping teardown as no inference service was successfully deployed
Pipeline stage MKMLProfilerDeleter completed in 0.10s
run pipeline stage %s
Running pipeline stage MKMLProfilerTemplater
Pipeline stage MKMLProfilerTemplater completed in 0.09s
run pipeline stage %s
Running pipeline stage MKMLProfilerDeployer
Creating inference service chaiml-1111-quang-ir-mi-73541-v1-profiler
Waiting for inference service chaiml-1111-quang-ir-mi-73541-v1-profiler to be ready
Shutdown handler registered
run pipeline %s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyScorer
Evaluating %s Family Friendly Score with %s threads
%s, retrying in %s seconds...
Evaluating %s Family Friendly Score with %s threads
Pipeline stage OfflineFamilyFriendlyScorer completed in 4812.91s
Shutdown handler de-registered
chaiml-1111-quang-ir-mi_73541_v1 status is now torndown due to DeploymentManager action