qwen-qwq-32b

developer_uid: NischayDnk

submission_id: qwen-qwq-32b_v1

model_name: qwen-qwq-32b_v1

model_group: Qwen/QwQ-32B

status: torndown

timestamp: 2025-04-03T18:36:02+00:00

num_battles: 10

num_wins: 4

family_friendly_score: 0.0

family_friendly_standard_error: 0.0

submission_type: basic

model_repo: Qwen/QwQ-32B

model_architecture: Qwen2ForCausalLM

model_num_parameters: 32763417600.0

best_of: 8

max_input_tokens: 1024

max_output_tokens: 64

reward_model: default

latencies: [{'batch_size': 1, 'throughput': 0.25498132586984174, 'latency_mean': 3.9217920863628386, 'latency_p50': 3.89677357673645, 'latency_p90': 4.372114896774292}, {'batch_size': 2, 'throughput': 0.39765607777351725, 'latency_mean': 5.023171802759171, 'latency_p50': 5.024967193603516, 'latency_p90': 5.559881734848022}, {'batch_size': 5, 'throughput': 0.6346159062948862, 'latency_mean': 7.846997122764588, 'latency_p50': 7.857662081718445, 'latency_p90': 8.80043888092041}]

gpu_counts: {'NVIDIA RTX A6000': 1}

display_name: qwen-qwq-32b_v1

ineligible_reason: num_battles<5000

is_internal_developer: False

language_model: Qwen/QwQ-32B

model_size: 33B

ranking_group: single

us_pacific_date: 2025-04-03

win_ratio: 0.4

generation_params: {'temperature': 1.0, 'top_p': 1.0, 'min_p': 0.0, 'top_k': 40, 'presence_penalty': 0.0, 'frequency_penalty': 0.0, 'stopping_words': ['\n'], 'max_input_tokens': 1024, 'best_of': 8, 'max_output_tokens': 64}

formatter: {'memory_template': '<|im_start|>system\n{memory}<|im_end|>\n', 'prompt_template': '<|im_start|>user\n{prompt}<|im_end|>\n', 'bot_template': '<|im_start|>assistant\n{bot_name}: {message}<|im_end|>\n', 'user_template': '<|im_start|>user\n{user_name}: {message}<|im_end|>\n', 'response_template': '<|im_start|>assistant\n{bot_name}:', 'truncate_by_message': False}

Resubmit model

Shutdown handler not registered because Python interpreter is not running in the main thread
run pipeline %s
run pipeline stage %s
Running pipeline stage MKMLizer
Starting job with name qwen-qwq-32b-v1-mkmlizer
Waiting for job on qwen-qwq-32b-v1-mkmlizer to finish
qwen-qwq-32b-v1-mkmlizer: ╔═════════════════════════════════════════════════════════════════════╗
qwen-qwq-32b-v1-mkmlizer: ║     _____            __           __                                ║
qwen-qwq-32b-v1-mkmlizer: ║    / _/ /_ ___    __/ /  ___ ___ / /                                ║
qwen-qwq-32b-v1-mkmlizer: ║   / _/ / // / |/|/ / _ \/ -_) -_) /                                 ║
qwen-qwq-32b-v1-mkmlizer: ║  /_//_/\_, /|__,__/_//_/\__/\__/_/                                  ║
qwen-qwq-32b-v1-mkmlizer: ║       /___/                                                         ║
qwen-qwq-32b-v1-mkmlizer: ║                                                                     ║
qwen-qwq-32b-v1-mkmlizer: ║  Version: 0.12.8                                                    ║
qwen-qwq-32b-v1-mkmlizer: ║  Copyright 2023 MK ONE TECHNOLOGIES Inc.                            ║
qwen-qwq-32b-v1-mkmlizer: ║  https://mk1.ai                                                     ║
qwen-qwq-32b-v1-mkmlizer: ║                                                                     ║
qwen-qwq-32b-v1-mkmlizer: ║  The license key for the current software has been verified as      ║
qwen-qwq-32b-v1-mkmlizer: ║  belonging to:                                                      ║
qwen-qwq-32b-v1-mkmlizer: ║                                                                     ║
qwen-qwq-32b-v1-mkmlizer: ║  Chai Research Corp.                                                ║
qwen-qwq-32b-v1-mkmlizer: ║  Account ID: 7997a29f-0ceb-4cc7-9adf-840c57b4ae6f                   ║
qwen-qwq-32b-v1-mkmlizer: ║  Expiration: 2025-04-15 23:59:59                                    ║
qwen-qwq-32b-v1-mkmlizer: ║                                                                     ║
qwen-qwq-32b-v1-mkmlizer: ╚═════════════════════════════════════════════════════════════════════╝
Failed to get response for submission rirv938-96p-4ff-rirv938_38486_v1: HTTPConnectionPool(host='rirv938-96p-4ff-rirv938-38486-v1-predictor.tenant-chaiml-guanaco.k.chaiverse.com', port=80): Read timed out. (read timeout=12.0)
Failed to get response for submission chaiml-sft-gemma2-28b-v_83370_v5: HTTPConnectionPool(host='chaiml-sft-gemma2-28b-v-83370-v5-predictor.tenant-chaiml-guanaco.k.chaiverse.com', port=80): Read timed out. (read timeout=12.0)
qwen-qwq-32b-v1-mkmlizer: Downloaded to shared memory in 72.184s
qwen-qwq-32b-v1-mkmlizer: quantizing model to /dev/shm/model_cache, profile:s0, folder:/tmp/tmp7ucd__x_, device:0
qwen-qwq-32b-v1-mkmlizer: Saving flywheel model at /dev/shm/model_cache
qwen-qwq-32b-v1-mkmlizer: quantized model in 64.059s
qwen-qwq-32b-v1-mkmlizer: Processed model Qwen/QwQ-32B in 136.243s
qwen-qwq-32b-v1-mkmlizer: creating bucket guanaco-mkml-models
qwen-qwq-32b-v1-mkmlizer: Bucket 's3://guanaco-mkml-models/' created
qwen-qwq-32b-v1-mkmlizer: uploading /dev/shm/model_cache to s3://guanaco-mkml-models/qwen-qwq-32b-v1
qwen-qwq-32b-v1-mkmlizer: cp /dev/shm/model_cache/config.json s3://guanaco-mkml-models/qwen-qwq-32b-v1/config.json
qwen-qwq-32b-v1-mkmlizer: cp /dev/shm/model_cache/added_tokens.json s3://guanaco-mkml-models/qwen-qwq-32b-v1/added_tokens.json
qwen-qwq-32b-v1-mkmlizer: cp /dev/shm/model_cache/special_tokens_map.json s3://guanaco-mkml-models/qwen-qwq-32b-v1/special_tokens_map.json
qwen-qwq-32b-v1-mkmlizer: cp /dev/shm/model_cache/tokenizer_config.json s3://guanaco-mkml-models/qwen-qwq-32b-v1/tokenizer_config.json
qwen-qwq-32b-v1-mkmlizer: cp /dev/shm/model_cache/vocab.json s3://guanaco-mkml-models/qwen-qwq-32b-v1/vocab.json
qwen-qwq-32b-v1-mkmlizer: cp /dev/shm/model_cache/merges.txt s3://guanaco-mkml-models/qwen-qwq-32b-v1/merges.txt
qwen-qwq-32b-v1-mkmlizer: cp /dev/shm/model_cache/tokenizer.json s3://guanaco-mkml-models/qwen-qwq-32b-v1/tokenizer.json
qwen-qwq-32b-v1-mkmlizer: cp /dev/shm/model_cache/flywheel_model.2.safetensors s3://guanaco-mkml-models/qwen-qwq-32b-v1/flywheel_model.2.safetensors
qwen-qwq-32b-v1-mkmlizer: cp /dev/shm/model_cache/flywheel_model.1.safetensors s3://guanaco-mkml-models/qwen-qwq-32b-v1/flywheel_model.1.safetensors
qwen-qwq-32b-v1-mkmlizer: cp /dev/shm/model_cache/flywheel_model.0.safetensors s3://guanaco-mkml-models/qwen-qwq-32b-v1/flywheel_model.0.safetensors
qwen-qwq-32b-v1-mkmlizer: 
Loading 0:   0%|          | 0/771 [00:00<?, ?it/s]
Loading 0:   1%|          | 5/771 [00:00<00:20, 37.39it/s]
Loading 0:   2%|▏         | 16/771 [00:00<00:10, 71.83it/s]
Loading 0:   4%|▎         | 27/771 [00:00<00:08, 84.63it/s]
Loading 0:   5%|▍         | 36/771 [00:00<00:08, 84.69it/s]
Loading 0:   6%|▌         | 46/771 [00:00<00:13, 54.27it/s]
Loading 0:   7%|▋         | 53/771 [00:00<00:15, 46.24it/s]
Loading 0:   8%|▊         | 64/771 [00:01<00:12, 58.45it/s]
Loading 0:  10%|▉         | 75/771 [00:01<00:10, 69.11it/s]
Loading 0:  11%|█         | 84/771 [00:01<00:09, 73.32it/s]
Loading 0:  12%|█▏        | 93/771 [00:01<00:09, 70.76it/s]
Loading 0:  14%|█▎        | 106/771 [00:01<00:10, 61.14it/s]
Loading 0:  15%|█▍        | 113/771 [00:01<00:11, 57.99it/s]
Loading 0:  16%|█▌        | 121/771 [00:01<00:11, 58.77it/s]
Loading 0:  17%|█▋        | 132/771 [00:02<00:09, 69.45it/s]
Loading 0:  18%|█▊        | 142/771 [00:02<00:08, 75.88it/s]
Loading 0:  20%|█▉        | 152/771 [00:02<00:07, 80.99it/s]
Loading 0:  21%|██        | 161/771 [00:02<00:08, 76.14it/s]
Loading 0:  22%|██▏       | 169/771 [00:02<00:11, 52.39it/s]
Loading 0:  23%|██▎       | 176/771 [00:02<00:11, 52.99it/s]
Loading 0:  24%|██▍       | 185/771 [00:02<00:10, 55.92it/s]
Loading 0:  25%|██▌       | 196/771 [00:03<00:08, 67.04it/s]
Loading 0:  27%|██▋       | 207/771 [00:03<00:07, 76.29it/s]
Loading 0:  28%|██▊       | 216/771 [00:03<00:07, 79.14it/s]
Loading 0:  29%|██▉       | 226/771 [00:03<00:09, 60.51it/s]
Loading 0:  30%|███       | 234/771 [00:03<00:10, 52.07it/s]
Loading 0:  32%|███▏      | 244/771 [00:03<00:08, 61.11it/s]
Loading 0:  33%|███▎      | 255/771 [00:03<00:07, 70.99it/s]
Loading 0:  34%|███▍      | 264/771 [00:04<00:06, 74.57it/s]
Loading 0:  35%|███▌      | 273/771 [00:04<00:07, 70.80it/s]
Loading 0:  37%|███▋      | 286/771 [00:04<00:08, 59.39it/s]
Loading 0:  38%|███▊      | 293/771 [00:04<00:09, 49.45it/s]
Loading 0:  39%|███▉      | 303/771 [00:19<03:36,  2.16it/s]
Loading 0:  41%|████      | 313/771 [00:19<02:27,  3.10it/s]
Loading 0:  42%|████▏     | 320/771 [00:19<01:53,  3.98it/s]
Loading 0:  43%|████▎     | 328/771 [00:19<01:21,  5.41it/s]
Loading 0:  44%|████▍     | 338/771 [00:19<00:55,  7.87it/s]
Loading 0:  45%|████▍     | 346/771 [00:19<00:42, 10.00it/s]
Loading 0:  46%|████▌     | 353/771 [00:20<00:36, 11.55it/s]
Loading 0:  47%|████▋     | 364/771 [00:20<00:23, 17.10it/s]
Loading 0:  49%|████▊     | 375/771 [00:20<00:16, 23.97it/s]
Loading 0:  50%|████▉     | 383/771 [00:20<00:13, 29.30it/s]
Loading 0:  51%|█████     | 391/771 [00:20<00:11, 34.29it/s]
Loading 0:  53%|█████▎    | 406/771 [00:20<00:08, 40.62it/s]
Loading 0:  54%|█████▎    | 413/771 [00:21<00:09, 38.19it/s]
Loading 0:  55%|█████▍    | 424/771 [00:21<00:07, 48.10it/s]
Loading 0:  56%|█████▋    | 435/771 [00:21<00:05, 58.16it/s]
Loading 0:  58%|█████▊    | 444/771 [00:21<00:05, 63.93it/s]
Loading 0:  59%|█████▉    | 453/771 [00:21<00:04, 64.44it/s]
Loading 0:  60%|██████    | 466/771 [00:21<00:05, 58.41it/s]
Loading 0:  61%|██████▏   | 473/771 [00:21<00:05, 49.95it/s]
Loading 0:  63%|██████▎   | 484/771 [00:22<00:04, 60.75it/s]
Loading 0:  64%|██████▍   | 496/771 [00:22<00:04, 67.34it/s]
Loading 0:  66%|██████▌   | 507/771 [00:22<00:03, 75.83it/s]
Loading 0:  67%|██████▋   | 516/771 [00:22<00:03, 78.14it/s]
Loading 0:  68%|██████▊   | 526/771 [00:22<00:04, 60.28it/s]
Loading 0:  69%|██████▉   | 534/771 [00:22<00:04, 52.35it/s]
Loading 0:  71%|███████   | 544/771 [00:22<00:03, 61.29it/s]
Loading 0:  72%|███████▏  | 555/771 [00:23<00:03, 71.16it/s]
Loading 0:  73%|███████▎  | 564/771 [00:23<00:02, 74.90it/s]
Loading 0:  74%|███████▍  | 573/771 [00:23<00:02, 71.80it/s]
Loading 0:  76%|███████▌  | 586/771 [00:23<00:02, 62.23it/s]
Loading 0:  77%|███████▋  | 593/771 [00:23<00:03, 52.26it/s]
Loading 0:  78%|███████▊  | 604/771 [00:23<00:02, 62.76it/s]
Loading 0:  80%|███████▉  | 615/771 [00:23<00:02, 72.11it/s]
Loading 0:  81%|████████  | 624/771 [00:24<00:01, 75.23it/s]
Loading 0:  82%|████████▏ | 633/771 [00:38<01:04,  2.14it/s]
Loading 0:  84%|████████▍ | 646/771 [00:39<00:37,  3.29it/s]
Loading 0:  85%|████████▍ | 654/771 [00:39<00:28,  4.16it/s]
Loading 0:  86%|████████▌ | 664/771 [00:39<00:18,  5.87it/s]
Loading 0:  88%|████████▊ | 675/771 [00:39<00:11,  8.45it/s]
Loading 0:  89%|████████▊ | 684/771 [00:39<00:07, 11.21it/s]
Loading 0:  90%|████████▉ | 693/771 [00:39<00:05, 14.61it/s]
Loading 0:  92%|█████████▏| 706/771 [00:40<00:03, 19.81it/s]
Loading 0:  92%|█████████▏| 713/771 [00:40<00:02, 21.63it/s]
Loading 0:  94%|█████████▍| 724/771 [00:40<00:01, 29.38it/s]
Loading 0:  95%|█████████▌| 736/771 [00:40<00:00, 37.87it/s]
Loading 0:  97%|█████████▋| 747/771 [00:40<00:00, 47.23it/s]
Loading 0:  98%|█████████▊| 756/771 [00:40<00:00, 53.77it/s]
Loading 0:  99%|█████████▉| 766/771 [00:48<00:01,  4.08it/s]
                                                            
Job qwen-qwq-32b-v1-mkmlizer completed after 176.16s with status: succeeded
Stopping job with name qwen-qwq-32b-v1-mkmlizer
Pipeline stage MKMLizer completed in 176.63s
run pipeline stage %s
Running pipeline stage MKMLTemplater
Pipeline stage MKMLTemplater completed in 0.16s
run pipeline stage %s
Running pipeline stage MKMLDeployer
Creating inference service qwen-qwq-32b-v1
Waiting for inference service qwen-qwq-32b-v1 to be ready
Failed to get response for submission rirv938-96p-4ff-rirv938_38486_v1: HTTPConnectionPool(host='rirv938-96p-4ff-rirv938-38486-v1-predictor.tenant-chaiml-guanaco.k.chaiverse.com', port=80): Read timed out. (read timeout=12.0)
Failed to get response for submission chaiml-sft-gemma2-28b-v_83370_v6: HTTPConnectionPool(host='chaiml-sft-gemma2-28b-v-83370-v6-predictor.tenant-chaiml-guanaco.k.chaiverse.com', port=80): Read timed out. (read timeout=12.0)
Inference service qwen-qwq-32b-v1 ready after 120.59695482254028s
Pipeline stage MKMLDeployer completed in 121.11s
run pipeline stage %s
Running pipeline stage StressChecker
Received healthy response to inference request in 4.560375452041626s
Failed to get response for submission chaiml-sft-gemma2-28b-v_83370_v5: HTTPConnectionPool(host='chaiml-sft-gemma2-28b-v-83370-v5-predictor.tenant-chaiml-guanaco.k.chaiverse.com', port=80): Read timed out. (read timeout=12.0)
Received healthy response to inference request in 3.378200054168701s
Received healthy response to inference request in 3.728883743286133s
Received healthy response to inference request in 3.518868923187256s
Received healthy response to inference request in 3.714747667312622s
5 requests
0 failed requests
5th percentile: 3.406333827972412
10th percentile: 3.434467601776123
20th percentile: 3.490735149383545
30th percentile: 3.558044672012329
40th percentile: 3.6363961696624756
50th percentile: 3.714747667312622
60th percentile: 3.7204020977020265
70th percentile: 3.726056528091431
80th percentile: 3.8951820850372316
90th percentile: 4.227778768539428
95th percentile: 4.394077110290527
99th percentile: 4.527115783691406
mean time: 3.7802151679992675
%s, retrying in %s seconds...
Received healthy response to inference request in 3.5445666313171387s
Received healthy response to inference request in 3.5512185096740723s
Received healthy response to inference request in 3.6188242435455322s
Received healthy response to inference request in 3.5005712509155273s
Received healthy response to inference request in 3.553623914718628s
5 requests
0 failed requests
5th percentile: 3.5093703269958496
10th percentile: 3.518169403076172
20th percentile: 3.5357675552368164
30th percentile: 3.5458970069885254
40th percentile: 3.548557758331299
50th percentile: 3.5512185096740723
60th percentile: 3.5521806716918944
70th percentile: 3.5531428337097166
80th percentile: 3.566663980484009
90th percentile: 3.5927441120147705
95th percentile: 3.6057841777801514
99th percentile: 3.616216230392456
mean time: 3.5537609100341796
Pipeline stage StressChecker completed in 39.48s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyTriggerPipeline
run_pipeline:run_in_cloud %s
starting trigger_guanaco_pipeline args=%s
triggered trigger_guanaco_pipeline args=%s
Pipeline stage OfflineFamilyFriendlyTriggerPipeline completed in 0.73s
run pipeline stage %s
Running pipeline stage TriggerMKMLProfilingPipeline
run_pipeline:run_in_cloud %s
starting trigger_guanaco_pipeline args=%s
triggered trigger_guanaco_pipeline args=%s
Pipeline stage TriggerMKMLProfilingPipeline completed in 0.73s
Shutdown handler de-registered
qwen-qwq-32b_v1 status is now deployed due to DeploymentManager action
Shutdown handler registered
run pipeline %s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyScorer
Evaluating %s Family Friendly Score with %s threads
%s, retrying in %s seconds...
Evaluating %s Family Friendly Score with %s threads
%s, retrying in %s seconds...
Evaluating %s Family Friendly Score with %s threads
clean up pipeline due to error=DeploymentChecksError('None: None')
Shutdown handler de-registered
Shutdown handler registered
run pipeline %s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyScorer
Evaluating %s Family Friendly Score with %s threads
%s, retrying in %s seconds...
Evaluating %s Family Friendly Score with %s threads
%s, retrying in %s seconds...
Evaluating %s Family Friendly Score with %s threads
clean up pipeline due to error=DeploymentChecksError('None: None')
Shutdown handler de-registered
Shutdown handler registered
run pipeline %s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyScorer
Evaluating %s Family Friendly Score with %s threads
%s, retrying in %s seconds...
Evaluating %s Family Friendly Score with %s threads
%s, retrying in %s seconds...
Evaluating %s Family Friendly Score with %s threads
clean up pipeline due to error=DeploymentChecksError('None: None')
Shutdown handler de-registered
qwen-qwq-32b_v1 status is now torndown due to DeploymentManager action