chaiml-1031-quang-ir-mix_9634

developer_uid: chai_backend_admin

submission_id: chaiml-1031-quang-ir-mix_9634_v1

model_name: training123

model_group: ChaiML/1031-quang-ir-mix

status: inactive

timestamp: 2025-11-01T09:29:56+00:00

num_battles: 6561

num_wins: 3368

celo_rating: 1297.23

family_friendly_score: 0.0

family_friendly_standard_error: 0.0

submission_type: basic

model_repo: ChaiML/1031-quang-ir-mixoct31all-cld-iq8-adh8_5e62e

model_architecture: MistralForCausalLM

model_num_parameters: 24096691200.0

best_of: 8

max_input_tokens: 2048

max_output_tokens: 64

reward_model: default

latencies: [{'batch_size': 1, 'throughput': 0.38342610655377113, 'latency_mean': 2.607942250967026, 'latency_p50': 2.6174904108047485, 'latency_p90': 2.84794909954071}, {'batch_size': 2, 'throughput': 0.573414306279292, 'latency_mean': 3.4854488134384156, 'latency_p50': 3.4747735261917114, 'latency_p90': 3.822439932823181}, {'batch_size': 3, 'throughput': 0.7087061591077941, 'latency_mean': 4.225306440591812, 'latency_p50': 4.226983547210693, 'latency_p90': 4.7394164323806764}, {'batch_size': 4, 'throughput': 0.8033990410888119, 'latency_mean': 4.9754256653785705, 'latency_p50': 4.951920390129089, 'latency_p90': 5.69354944229126}, {'batch_size': 5, 'throughput': 0.8599446582684717, 'latency_mean': 5.804179784059524, 'latency_p50': 5.821462035179138, 'latency_p90': 6.619794225692749}]

gpu_counts: {'NVIDIA L40S': 1}

display_name: training123

is_internal_developer: True

language_model: ChaiML/1031-quang-ir-mixoct31all-cld-iq8-adh8_5e62e

model_size: 24B

ranking_group: single

throughput_3p7s: 0.62

us_pacific_date: 2025-11-01

win_ratio: 0.5133363816491389

generation_params: {'temperature': 1.0, 'top_p': 1.0, 'min_p': 0.0, 'top_k': 80, 'presence_penalty': 0.0, 'frequency_penalty': 0.0, 'stopping_words': ['####\n', '\n', '####', 'You:', '</s>'], 'max_input_tokens': 2048, 'best_of': 8, 'max_output_tokens': 64}

formatter: {'memory_template': "{bot_name}'s persona: {memory}", 'prompt_template': '', 'bot_template': '{bot_name}: {message}\n', 'user_template': 'You: {message}\n', 'response_template': '####\n{bot_name}:', 'truncate_by_message': True}

Resubmit model

Shutdown handler not registered because Python interpreter is not running in the main thread
run pipeline %s
run pipeline stage %s
Running pipeline stage MKMLizer
Starting job with name chaiml-1031-quang-ir-mix-9634-v1-mkmlizer
Waiting for job on chaiml-1031-quang-ir-mix-9634-v1-mkmlizer to finish
chaiml-1031-quang-ir-mix-9634-v1-mkmlizer: ╔═════════════════════════════════════════════════════════════════════╗
chaiml-1031-quang-ir-mix-9634-v1-mkmlizer: ║                                                                     ║
chaiml-1031-quang-ir-mix-9634-v1-mkmlizer: ║   ██████   ██████  █████   ████  ████                               ║
chaiml-1031-quang-ir-mix-9634-v1-mkmlizer: ║  ░░██████ ██████  ░░███   ███░  ░░███                               ║
chaiml-1031-quang-ir-mix-9634-v1-mkmlizer: ║   ░███░█████░███   ░███  ███     ░███                               ║
chaiml-1031-quang-ir-mix-9634-v1-mkmlizer: ║   ░███░░███ ░███   ░███████      ░███                               ║
chaiml-1031-quang-ir-mix-9634-v1-mkmlizer: ║   ░███ ░░░  ░███   ░███░░███     ░███                               ║
chaiml-1031-quang-ir-mix-9634-v1-mkmlizer: ║   ░███      ░███   ░███ ░░███    ░███                               ║
chaiml-1031-quang-ir-mix-9634-v1-mkmlizer: ║   █████     █████  █████ ░░████  █████                              ║
chaiml-1031-quang-ir-mix-9634-v1-mkmlizer: ║  ░░░░░     ░░░░░  ░░░░░   ░░░░  ░░░░░                               ║
chaiml-1031-quang-ir-mix-9634-v1-mkmlizer: ║                                                                     ║
chaiml-1031-quang-ir-mix-9634-v1-mkmlizer: ║  Version: 0.30.2                                                    ║
chaiml-1031-quang-ir-mix-9634-v1-mkmlizer: ║  Features: FLYWHEEL, CUDA                                           ║
chaiml-1031-quang-ir-mix-9634-v1-mkmlizer: ║  Copyright 2023-2025 MK ONE TECHNOLOGIES Inc.                       ║
chaiml-1031-quang-ir-mix-9634-v1-mkmlizer: ║  https://mk1.ai                                                     ║
chaiml-1031-quang-ir-mix-9634-v1-mkmlizer: ║                                                                     ║
chaiml-1031-quang-ir-mix-9634-v1-mkmlizer: ║  The license key for the current software has been verified as      ║
chaiml-1031-quang-ir-mix-9634-v1-mkmlizer: ║  belonging to:                                                      ║
chaiml-1031-quang-ir-mix-9634-v1-mkmlizer: ║                                                                     ║
chaiml-1031-quang-ir-mix-9634-v1-mkmlizer: ║  Chai Research Corp.                                                ║
chaiml-1031-quang-ir-mix-9634-v1-mkmlizer: ║  Account ID: 7997a29f-0ceb-4cc7-9adf-840c57b4ae6f                   ║
chaiml-1031-quang-ir-mix-9634-v1-mkmlizer: ║  Expiration: 2028-03-31 23:59:59                                    ║
chaiml-1031-quang-ir-mix-9634-v1-mkmlizer: ║                                                                     ║
chaiml-1031-quang-ir-mix-9634-v1-mkmlizer: ╚═════════════════════════════════════════════════════════════════════╝
chaiml-1031-quang-ir-mix-9634-v1-mkmlizer: Downloaded to shared memory in 53.404s
chaiml-1031-quang-ir-mix-9634-v1-mkmlizer: Checking if ChaiML/1031-quang-ir-mixoct31all-cld-iq8-adh8_5e62e already exists in ChaiML
chaiml-1031-quang-ir-mix-9634-v1-mkmlizer: quantizing model to /dev/shm/model_cache, profile:s0, folder:/tmp/tmpc2illcdx, device:0
chaiml-1031-quang-ir-mix-9634-v1-mkmlizer: Saving flywheel model at /dev/shm/model_cache
chaiml-1031-quang-ir-mix-9634-v1-mkmlizer: quantized model in 42.724s
chaiml-1031-quang-ir-mix-9634-v1-mkmlizer: Processed model ChaiML/1031-quang-ir-mixoct31all-cld-iq8-adh8_5e62e in 96.129s
chaiml-1031-quang-ir-mix-9634-v1-mkmlizer: creating bucket guanaco-mkml-models
chaiml-1031-quang-ir-mix-9634-v1-mkmlizer: Bucket 's3://guanaco-mkml-models/' created
chaiml-1031-quang-ir-mix-9634-v1-mkmlizer: uploading /dev/shm/model_cache to s3://guanaco-mkml-models/chaiml-1031-quang-ir-mix-9634-v1/nvidia
chaiml-1031-quang-ir-mix-9634-v1-mkmlizer: cp /dev/shm/model_cache/config.json s3://guanaco-mkml-models/chaiml-1031-quang-ir-mix-9634-v1/nvidia/config.json
chaiml-1031-quang-ir-mix-9634-v1-mkmlizer: cp /dev/shm/model_cache/special_tokens_map.json s3://guanaco-mkml-models/chaiml-1031-quang-ir-mix-9634-v1/nvidia/special_tokens_map.json
chaiml-1031-quang-ir-mix-9634-v1-mkmlizer: cp /dev/shm/model_cache/tokenizer_config.json s3://guanaco-mkml-models/chaiml-1031-quang-ir-mix-9634-v1/nvidia/tokenizer_config.json
chaiml-1031-quang-ir-mix-9634-v1-mkmlizer: cp /dev/shm/model_cache/tokenizer.json s3://guanaco-mkml-models/chaiml-1031-quang-ir-mix-9634-v1/nvidia/tokenizer.json
chaiml-1031-quang-ir-mix-9634-v1-mkmlizer: cp /dev/shm/model_cache/flywheel_model.0.safetensors s3://guanaco-mkml-models/chaiml-1031-quang-ir-mix-9634-v1/nvidia/flywheel_model.0.safetensors
chaiml-1031-quang-ir-mix-9634-v1-mkmlizer: cp /dev/shm/model_cache/flywheel_model.1.safetensors s3://guanaco-mkml-models/chaiml-1031-quang-ir-mix-9634-v1/nvidia/flywheel_model.1.safetensors
chaiml-1031-quang-ir-mix-9634-v1-mkmlizer: 
Loading 0:   0%|          | 0/363 [00:00<?, ?it/s]
Loading 0:   1%|▏         | 5/363 [00:00<00:16, 21.55it/s]
Loading 0:   3%|▎         | 12/363 [00:00<00:08, 39.77it/s]
Loading 0:   5%|▍         | 17/363 [00:00<00:09, 35.68it/s]
Loading 0:   6%|▌         | 22/363 [00:00<00:09, 35.60it/s]
Loading 0:   7%|▋         | 26/363 [00:00<00:09, 34.96it/s]
Loading 0:   9%|▉         | 32/363 [00:00<00:08, 40.40it/s]
Loading 0:  10%|█         | 37/363 [00:01<00:12, 25.31it/s]
Loading 0:  11%|█▏        | 41/363 [00:01<00:13, 23.73it/s]
Loading 0:  13%|█▎        | 48/363 [00:01<00:10, 31.24it/s]
Loading 0:  14%|█▍        | 52/363 [00:01<00:10, 29.49it/s]
Loading 0:  16%|█▌        | 57/363 [00:01<00:09, 32.97it/s]
Loading 0:  17%|█▋        | 61/363 [00:01<00:09, 31.31it/s]
Loading 0:  18%|█▊        | 65/363 [00:02<00:09, 32.45it/s]
Loading 0:  19%|█▉        | 70/363 [00:02<00:10, 28.01it/s]
Loading 0:  20%|██        | 74/363 [00:02<00:11, 25.12it/s]
Loading 0:  22%|██▏       | 79/363 [00:02<00:09, 29.16it/s]
Loading 0:  23%|██▎       | 83/363 [00:02<00:09, 30.01it/s]
Loading 0:  24%|██▍       | 87/363 [00:02<00:10, 26.36it/s]
Loading 0:  25%|██▌       | 92/363 [00:03<00:10, 25.83it/s]
Loading 0:  27%|██▋       | 99/363 [00:03<00:07, 33.19it/s]
Loading 0:  28%|██▊       | 103/363 [00:03<00:08, 31.01it/s]
Loading 0:  29%|██▉       | 107/363 [00:03<00:09, 26.66it/s]
Loading 0:  31%|███       | 112/363 [00:03<00:08, 29.62it/s]
Loading 0:  32%|███▏      | 116/363 [00:03<00:08, 30.25it/s]
Loading 0:  33%|███▎      | 120/363 [00:03<00:07, 32.03it/s]
Loading 0:  34%|███▍      | 124/363 [00:04<00:07, 30.90it/s]
Loading 0:  36%|███▌      | 129/363 [00:04<00:06, 34.52it/s]
Loading 0:  37%|███▋      | 133/363 [00:04<00:07, 32.03it/s]
Loading 0:  38%|███▊      | 138/363 [00:04<00:06, 35.35it/s]
Loading 0:  39%|███▉      | 142/363 [00:04<00:06, 33.02it/s]
Loading 0:  41%|████      | 149/363 [00:04<00:05, 39.61it/s]
Loading 0:  42%|████▏     | 154/363 [00:05<00:07, 27.39it/s]
Loading 0:  44%|████▎     | 158/363 [00:05<00:08, 25.18it/s]
Loading 0:  45%|████▌     | 165/363 [00:05<00:06, 32.29it/s]
Loading 0:  47%|████▋     | 169/363 [00:05<00:06, 30.88it/s]
Loading 0:  48%|████▊     | 174/363 [00:05<00:05, 34.04it/s]
Loading 0:  49%|████▉     | 178/363 [00:05<00:05, 32.18it/s]
Loading 0:  50%|█████     | 182/363 [00:05<00:05, 33.06it/s]
Loading 0:  52%|█████▏    | 187/363 [00:06<00:05, 30.25it/s]
Loading 0:  53%|█████▎    | 191/363 [00:06<00:05, 29.45it/s]
Loading 0:  54%|█████▎    | 195/363 [00:06<00:06, 26.24it/s]
Loading 0:  55%|█████▌    | 201/363 [00:19<02:13,  1.21it/s]
Loading 0:  56%|█████▌    | 203/363 [00:19<01:54,  1.39it/s]
Loading 0:  57%|█████▋    | 208/363 [00:19<01:13,  2.10it/s]
Loading 0:  58%|█████▊    | 212/363 [00:19<00:53,  2.82it/s]
Loading 0:  60%|██████    | 218/363 [00:20<00:32,  4.40it/s]
Loading 0:  61%|██████    | 222/363 [00:20<00:24,  5.70it/s]
Loading 0:  62%|██████▏   | 226/363 [00:20<00:19,  6.99it/s]
Loading 0:  63%|██████▎   | 230/363 [00:20<00:15,  8.60it/s]
Loading 0:  65%|██████▌   | 237/363 [00:20<00:09, 13.22it/s]
Loading 0:  66%|██████▋   | 241/363 [00:20<00:08, 15.08it/s]
Loading 0:  68%|██████▊   | 246/363 [00:20<00:06, 18.91it/s]
Loading 0:  69%|██████▉   | 250/363 [00:21<00:05, 20.41it/s]
Loading 0:  70%|███████   | 255/363 [00:21<00:04, 24.59it/s]
Loading 0:  71%|███████▏  | 259/363 [00:21<00:04, 24.91it/s]
Loading 0:  73%|███████▎  | 266/363 [00:21<00:03, 31.72it/s]
Loading 0:  74%|███████▍  | 270/363 [00:21<00:04, 22.66it/s]
Loading 0:  76%|███████▌  | 275/363 [00:22<00:03, 22.66it/s]
Loading 0:  78%|███████▊  | 282/363 [00:22<00:02, 28.96it/s]
Loading 0:  79%|███████▉  | 286/363 [00:22<00:02, 28.25it/s]
Loading 0:  80%|████████  | 291/363 [00:22<00:02, 31.59it/s]
Loading 0:  81%|████████▏ | 295/363 [00:22<00:02, 30.13it/s]
Loading 0:  82%|████████▏ | 299/363 [00:22<00:02, 31.33it/s]
Loading 0:  84%|████████▎ | 304/363 [00:22<00:02, 28.60it/s]
Loading 0:  85%|████████▍ | 308/363 [00:23<00:01, 28.07it/s]
Loading 0:  86%|████████▌ | 311/363 [00:23<00:02, 23.29it/s]
Loading 0:  88%|████████▊ | 318/363 [00:23<00:01, 30.95it/s]
Loading 0:  89%|████████▊ | 322/363 [00:23<00:01, 29.43it/s]
Loading 0:  90%|█████████ | 327/363 [00:23<00:01, 32.86it/s]
Loading 0:  91%|█████████ | 331/363 [00:23<00:01, 30.16it/s]
Loading 0:  92%|█████████▏| 335/363 [00:23<00:00, 31.04it/s]
Loading 0:  93%|█████████▎| 339/363 [00:24<00:00, 30.86it/s]
Loading 0:  94%|█████████▍| 343/363 [00:24<00:01, 19.33it/s]
Loading 0:  96%|█████████▌| 348/363 [00:24<00:00, 21.04it/s]
Loading 0:  98%|█████████▊| 355/363 [00:24<00:00, 28.10it/s]
Loading 0:  99%|█████████▉| 359/363 [00:25<00:00, 25.91it/s]
Loading 0: 100%|██████████| 363/363 [00:25<00:00, 28.45it/s]
                                                            
Job chaiml-1031-quang-ir-mix-9634-v1-mkmlizer completed after 154.97s with status: succeeded
Stopping job with name chaiml-1031-quang-ir-mix-9634-v1-mkmlizer
Pipeline stage MKMLizer completed in 155.98s
run pipeline stage %s
Running pipeline stage MKMLTemplater
Pipeline stage MKMLTemplater completed in 0.17s
run pipeline stage %s
Running pipeline stage MKMLDeployer
Creating inference service chaiml-1031-quang-ir-mix-9634-v1
Waiting for inference service chaiml-1031-quang-ir-mix-9634-v1 to be ready
Inference service chaiml-1031-quang-ir-mix-9634-v1 ready after 241.02584028244019s
Pipeline stage MKMLDeployer completed in 241.74s
run pipeline stage %s
Running pipeline stage StressChecker
Received healthy response to inference request in 2.4628705978393555s
Received healthy response to inference request in 2.200317859649658s
Failed to get response for submission mistralai-mistral-nem_93303_v569: ('http://mistralai-mistral-nem-93303-v569-predictor.tenant-chaiml-guanaco.k2.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', '')
Received healthy response to inference request in 2.7083852291107178s
Received healthy response to inference request in 2.1470696926116943s
Received healthy response to inference request in 2.472473621368408s
5 requests
0 failed requests
5th percentile: 2.1577193260192873
10th percentile: 2.16836895942688
20th percentile: 2.1896682262420653
30th percentile: 2.2528284072875975
40th percentile: 2.3578495025634765
50th percentile: 2.4628705978393555
60th percentile: 2.4667118072509764
70th percentile: 2.4705530166625977
80th percentile: 2.51965594291687
90th percentile: 2.614020586013794
95th percentile: 2.661202907562256
99th percentile: 2.6989487648010253
mean time: 2.398223400115967
Pipeline stage StressChecker completed in 13.58s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyTriggerPipeline
run_pipeline:run_in_cloud %s
starting trigger_guanaco_pipeline args=%s
triggered trigger_guanaco_pipeline args=%s
Pipeline stage OfflineFamilyFriendlyTriggerPipeline completed in 1.01s
run pipeline stage %s
Running pipeline stage TriggerMKMLProfilingPipeline
run_pipeline:run_in_cloud %s
starting trigger_guanaco_pipeline args=%s
triggered trigger_guanaco_pipeline args=%s
Pipeline stage TriggerMKMLProfilingPipeline completed in 1.09s
Shutdown handler de-registered
chaiml-1031-quang-ir-mix_9634_v1 status is now deployed due to DeploymentManager action
Shutdown handler registered
run pipeline %s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyScorer
Evaluating %s Family Friendly Score with %s threads
%s, retrying in %s seconds...
Evaluating %s Family Friendly Score with %s threads
Pipeline stage OfflineFamilyFriendlyScorer completed in 5647.89s
Shutdown handler de-registered