rirv938-prefgrok-r2-cp62_5963

developer_uid: rirv938

submission_id: rirv938-prefgrok-r2-cp62_5963_v2

model_name: rirv938-prefgrok-r2-cp62_5963_v2

model_group: rirv938/prefgrok_r2_cp62

status: torndown

timestamp: 2025-07-09T14:22:09+00:00

num_battles: 8377

num_wins: 4228

celo_rating: 1298.81

family_friendly_score: 0.5589999999999999

family_friendly_standard_error: 0.007021666468866205

submission_type: basic

model_repo: rirv938/prefgrok_r2_cp624_98ff_b5_merged

model_architecture: MistralForCausalLM

model_num_parameters: 24096691200.0

best_of: 8

max_input_tokens: 1024

max_output_tokens: 64

reward_model: default

latencies: [{'batch_size': 1, 'throughput': 0.5238315982967993, 'latency_mean': 1.9089078044891357, 'latency_p50': 1.9269191026687622, 'latency_p90': 2.111790728569031}, {'batch_size': 3, 'throughput': 1.0516605598593385, 'latency_mean': 2.847246015071869, 'latency_p50': 2.857433557510376, 'latency_p90': 3.1424539566040037}, {'batch_size': 5, 'throughput': 1.3502666910186387, 'latency_mean': 3.68242422580719, 'latency_p50': 3.6406776905059814, 'latency_p90': 4.106873035430908}, {'batch_size': 6, 'throughput': 1.442641683492139, 'latency_mean': 4.114716469049454, 'latency_p50': 4.133740067481995, 'latency_p90': 4.628363990783692}, {'batch_size': 8, 'throughput': 1.5679604955827073, 'latency_mean': 5.067166380882263, 'latency_p50': 5.046223998069763, 'latency_p90': 5.692672824859619}, {'batch_size': 10, 'throughput': 1.6675978966610134, 'latency_mean': 5.946470972299576, 'latency_p50': 5.917827725410461, 'latency_p90': 6.69915280342102}]

gpu_counts: {'NVIDIA A100-SXM4-80GB': 1}

display_name: rirv938-prefgrok-r2-cp62_5963_v2

is_internal_developer: True

language_model: rirv938/prefgrok_r2_cp624_98ff_b5_merged

model_size: 24B

ranking_group: single

throughput_3p7s: 1.36

us_pacific_date: 2025-07-09

win_ratio: 0.504715291870598

generation_params: {'temperature': 1.0, 'top_p': 0.95, 'min_p': 0.05, 'top_k': 80, 'presence_penalty': 0.45, 'frequency_penalty': 0.45, 'stopping_words': ['You:', '</s>', '###', '\n'], 'max_input_tokens': 1024, 'best_of': 8, 'max_output_tokens': 64}

formatter: {'memory_template': '', 'prompt_template': '', 'bot_template': '<|im_start|>assistant\n{message}<|im_end|>\n', 'user_template': '<|im_start|>user\nYou:{message}<|im_end|>\n', 'response_template': '<|im_start|>assistant\n', 'truncate_by_message': False}

Resubmit model

Shutdown handler not registered because Python interpreter is not running in the main thread
run pipeline %s
run pipeline stage %s
Running pipeline stage MKMLizer
Starting job with name rirv938-prefgrok-r2-cp62-5963-v2-mkmlizer
Waiting for job on rirv938-prefgrok-r2-cp62-5963-v2-mkmlizer to finish
rirv938-prefgrok-r2-cp62-5963-v2-mkmlizer: ╔═════════════════════════════════════════════════════════════════════╗
rirv938-prefgrok-r2-cp62-5963-v2-mkmlizer: ║                                                                     ║
rirv938-prefgrok-r2-cp62-5963-v2-mkmlizer: ║   ██████   ██████  █████   ████  ████                               ║
rirv938-prefgrok-r2-cp62-5963-v2-mkmlizer: ║  ░░██████ ██████  ░░███   ███░  ░░███                               ║
rirv938-prefgrok-r2-cp62-5963-v2-mkmlizer: ║   ░███░█████░███   ░███  ███     ░███                               ║
rirv938-prefgrok-r2-cp62-5963-v2-mkmlizer: ║   ░███░░███ ░███   ░███████      ░███                               ║
rirv938-prefgrok-r2-cp62-5963-v2-mkmlizer: ║   ░███ ░░░  ░███   ░███░░███     ░███                               ║
rirv938-prefgrok-r2-cp62-5963-v2-mkmlizer: ║   ░███      ░███   ░███ ░░███    ░███                               ║
rirv938-prefgrok-r2-cp62-5963-v2-mkmlizer: ║   █████     █████  █████ ░░████  █████                              ║
rirv938-prefgrok-r2-cp62-5963-v2-mkmlizer: ║  ░░░░░     ░░░░░  ░░░░░   ░░░░  ░░░░░                               ║
rirv938-prefgrok-r2-cp62-5963-v2-mkmlizer: ║                                                                     ║
rirv938-prefgrok-r2-cp62-5963-v2-mkmlizer: ║  Version: 0.29.15                                                   ║
rirv938-prefgrok-r2-cp62-5963-v2-mkmlizer: ║  Features: FLYWHEEL, CUDA                                           ║
rirv938-prefgrok-r2-cp62-5963-v2-mkmlizer: ║  Copyright 2023-2025 MK ONE TECHNOLOGIES Inc.                       ║
rirv938-prefgrok-r2-cp62-5963-v2-mkmlizer: ║  https://mk1.ai                                                     ║
rirv938-prefgrok-r2-cp62-5963-v2-mkmlizer: ║                                                                     ║
rirv938-prefgrok-r2-cp62-5963-v2-mkmlizer: ║  The license key for the current software has been verified as      ║
rirv938-prefgrok-r2-cp62-5963-v2-mkmlizer: ║  belonging to:                                                      ║
rirv938-prefgrok-r2-cp62-5963-v2-mkmlizer: ║                                                                     ║
rirv938-prefgrok-r2-cp62-5963-v2-mkmlizer: ║  Chai Research Corp.                                                ║
rirv938-prefgrok-r2-cp62-5963-v2-mkmlizer: ║  Account ID: 7997a29f-0ceb-4cc7-9adf-840c57b4ae6f                   ║
rirv938-prefgrok-r2-cp62-5963-v2-mkmlizer: ║  Expiration: 2028-03-31 23:59:59                                    ║
rirv938-prefgrok-r2-cp62-5963-v2-mkmlizer: ║                                                                     ║
rirv938-prefgrok-r2-cp62-5963-v2-mkmlizer: ╚═════════════════════════════════════════════════════════════════════╝
rirv938-prefgrok-r2-cp62-5963-v2-mkmlizer: Downloaded to shared memory in 172.512s
rirv938-prefgrok-r2-cp62-5963-v2-mkmlizer: Checking if rirv938/prefgrok_r2_cp624_98ff_b5_merged already exists in ChaiML
rirv938-prefgrok-r2-cp62-5963-v2-mkmlizer: quantizing model to /dev/shm/model_cache, profile:s0, folder:/tmp/tmpif0sp6va, device:0
rirv938-prefgrok-r2-cp62-5963-v2-mkmlizer: Saving flywheel model at /dev/shm/model_cache
Failed to get response for submission chaiml-gy-exp129-dpo-ex_61642_v4: HTTPConnectionPool(host='chaiml-gy-exp129-dpo-ex-61642-v4-predictor.tenant-chaiml-guanaco.k.chaiverse.com', port=80): Read timed out. (read timeout=12.0)
rirv938-prefgrok-r2-cp62-5963-v2-mkmlizer: quantized model in 62.940s
rirv938-prefgrok-r2-cp62-5963-v2-mkmlizer: Processed model rirv938/prefgrok_r2_cp624_98ff_b5_merged in 235.597s
rirv938-prefgrok-r2-cp62-5963-v2-mkmlizer: creating bucket guanaco-mkml-models
rirv938-prefgrok-r2-cp62-5963-v2-mkmlizer: Bucket 's3://guanaco-mkml-models/' created
rirv938-prefgrok-r2-cp62-5963-v2-mkmlizer: uploading /dev/shm/model_cache to s3://guanaco-mkml-models/rirv938-prefgrok-r2-cp62-5963-v2/nvidia
rirv938-prefgrok-r2-cp62-5963-v2-mkmlizer: cp /dev/shm/model_cache/config.json s3://guanaco-mkml-models/rirv938-prefgrok-r2-cp62-5963-v2/nvidia/config.json
rirv938-prefgrok-r2-cp62-5963-v2-mkmlizer: cp /dev/shm/model_cache/special_tokens_map.json s3://guanaco-mkml-models/rirv938-prefgrok-r2-cp62-5963-v2/nvidia/special_tokens_map.json
rirv938-prefgrok-r2-cp62-5963-v2-mkmlizer: cp /dev/shm/model_cache/tokenizer_config.json s3://guanaco-mkml-models/rirv938-prefgrok-r2-cp62-5963-v2/nvidia/tokenizer_config.json
rirv938-prefgrok-r2-cp62-5963-v2-mkmlizer: cp /dev/shm/model_cache/tokenizer.json s3://guanaco-mkml-models/rirv938-prefgrok-r2-cp62-5963-v2/nvidia/tokenizer.json
rirv938-prefgrok-r2-cp62-5963-v2-mkmlizer: cp /dev/shm/model_cache/flywheel_model.0.safetensors s3://guanaco-mkml-models/rirv938-prefgrok-r2-cp62-5963-v2/nvidia/flywheel_model.0.safetensors
rirv938-prefgrok-r2-cp62-5963-v2-mkmlizer: cp /dev/shm/model_cache/flywheel_model.1.safetensors s3://guanaco-mkml-models/rirv938-prefgrok-r2-cp62-5963-v2/nvidia/flywheel_model.1.safetensors
rirv938-prefgrok-r2-cp62-5963-v2-mkmlizer: 
Loading 0:   0%|          | 0/363 [00:00<?, ?it/s]
Loading 0:   1%|          | 3/363 [00:00<00:12, 29.07it/s]
Loading 0:   2%|▏         | 6/363 [00:00<00:25, 13.84it/s]
Loading 0:   3%|▎         | 11/363 [00:00<00:16, 21.86it/s]
Loading 0:   4%|▍         | 14/363 [00:00<00:26, 13.33it/s]
Loading 0:   4%|▍         | 16/363 [00:01<00:26, 12.94it/s]
Loading 0:   6%|▌         | 21/363 [00:01<00:20, 16.54it/s]
Loading 0:   6%|▋         | 23/363 [00:01<00:25, 13.09it/s]
Loading 0:   8%|▊         | 28/363 [00:01<00:17, 18.98it/s]
Loading 0:   9%|▉         | 32/363 [00:01<00:14, 22.83it/s]
Loading 0:  10%|▉         | 35/363 [00:02<00:21, 15.38it/s]
Loading 0:  10%|█         | 38/363 [00:02<00:20, 15.93it/s]
Loading 0:  11%|█▏        | 41/363 [00:02<00:25, 12.44it/s]
Loading 0:  13%|█▎        | 46/363 [00:02<00:17, 17.67it/s]
Loading 0:  14%|█▍        | 50/363 [00:02<00:14, 21.27it/s]
Loading 0:  15%|█▍        | 54/363 [00:03<00:22, 13.51it/s]
Loading 0:  16%|█▌        | 57/363 [00:03<00:19, 15.41it/s]
Loading 0:  17%|█▋        | 60/363 [00:03<00:21, 13.93it/s]
Loading 0:  18%|█▊        | 64/363 [00:03<00:16, 17.79it/s]
Loading 0:  19%|█▉        | 69/363 [00:04<00:16, 18.24it/s]
Loading 0:  20%|█▉        | 72/363 [00:04<00:20, 14.12it/s]
Loading 0:  21%|██        | 75/363 [00:04<00:17, 16.09it/s]
Loading 0:  21%|██▏       | 78/363 [00:04<00:19, 14.46it/s]
Loading 0:  23%|██▎       | 82/363 [00:05<00:15, 18.25it/s]
Loading 0:  24%|██▍       | 87/363 [00:05<00:14, 18.53it/s]
Loading 0:  25%|██▍       | 90/363 [00:05<00:19, 14.13it/s]
Loading 0:  26%|██▋       | 96/363 [00:05<00:13, 20.40it/s]
Loading 0:  28%|██▊       | 101/363 [00:05<00:11, 22.22it/s]
Loading 0:  29%|██▊       | 104/363 [00:06<00:14, 17.89it/s]
Loading 0:  29%|██▉       | 107/363 [00:06<00:18, 13.85it/s]
Loading 0:  30%|███       | 109/363 [00:06<00:18, 13.52it/s]
Loading 0:  31%|███       | 111/363 [00:06<00:17, 14.22it/s]
Loading 0:  31%|███       | 113/363 [00:07<00:20, 12.03it/s]
Loading 0:  33%|███▎      | 118/363 [00:07<00:13, 18.28it/s]
Loading 0:  34%|███▍      | 123/363 [00:07<00:12, 18.77it/s]
Loading 0:  35%|███▍      | 126/363 [00:07<00:17, 13.93it/s]
Loading 0:  36%|███▌      | 129/363 [00:08<00:14, 15.93it/s]
Loading 0:  36%|███▋      | 132/363 [00:08<00:16, 14.13it/s]
Loading 0:  37%|███▋      | 136/363 [00:08<00:12, 18.03it/s]
Loading 0:  39%|███▊      | 140/363 [00:08<00:10, 21.53it/s]
Loading 0:  39%|███▉      | 143/363 [00:08<00:14, 15.26it/s]
Loading 0:  40%|████      | 146/363 [00:09<00:13, 15.75it/s]
Loading 0:  41%|████      | 149/363 [00:09<00:17, 12.43it/s]
Loading 0:  42%|████▏     | 154/363 [00:09<00:12, 17.27it/s]
Loading 0:  44%|████▎     | 158/363 [00:09<00:09, 20.58it/s]
Loading 0:  44%|████▍     | 161/363 [00:09<00:13, 15.07it/s]
Loading 0:  45%|████▌     | 164/363 [00:10<00:12, 15.74it/s]
Loading 0:  46%|████▌     | 167/363 [00:10<00:16, 12.24it/s]
Loading 0:  47%|████▋     | 172/363 [00:10<00:10, 17.38it/s]
Loading 0:  48%|████▊     | 176/363 [00:10<00:08, 20.81it/s]
Loading 0:  49%|████▉     | 179/363 [00:11<00:12, 15.10it/s]
Loading 0:  50%|█████     | 182/363 [00:11<00:11, 15.81it/s]
Loading 0:  51%|█████     | 185/363 [00:11<00:14, 12.65it/s]
Loading 0:  52%|█████▏    | 190/363 [00:11<00:09, 17.79it/s]
Loading 0:  54%|█████▎    | 195/363 [00:12<00:09, 18.36it/s]
Loading 0:  55%|█████▍    | 198/363 [00:12<00:11, 14.37it/s]
Loading 0:  55%|█████▌    | 201/363 [00:28<03:45,  1.39s/it]
Loading 0:  56%|█████▌    | 203/363 [00:28<03:04,  1.15s/it]
Loading 0:  57%|█████▋    | 207/363 [00:28<01:57,  1.33it/s]
Loading 0:  58%|█████▊    | 210/363 [00:28<01:24,  1.80it/s]
Loading 0:  59%|█████▊    | 213/363 [00:29<01:03,  2.36it/s]
Loading 0:  59%|█████▉    | 215/363 [00:29<00:52,  2.82it/s]
Loading 0:  60%|█████▉    | 217/363 [00:29<00:42,  3.43it/s]
Loading 0:  60%|██████    | 219/363 [00:29<00:33,  4.30it/s]
Loading 0:  61%|██████    | 221/363 [00:29<00:29,  4.81it/s]
Loading 0:  62%|██████▏   | 226/363 [00:29<00:16,  8.49it/s]
Loading 0:  63%|██████▎   | 230/363 [00:30<00:11, 11.67it/s]
Loading 0:  64%|██████▍   | 233/363 [00:30<00:12, 10.42it/s]
Loading 0:  65%|██████▌   | 236/363 [00:30<00:10, 11.77it/s]
Loading 0:  66%|██████▌   | 239/363 [00:31<00:12, 10.32it/s]
Loading 0:  67%|██████▋   | 244/363 [00:31<00:07, 14.98it/s]
Loading 0:  68%|██████▊   | 248/363 [00:31<00:06, 18.40it/s]
Loading 0:  69%|██████▉   | 251/363 [00:31<00:07, 14.12it/s]
Loading 0:  70%|██████▉   | 254/363 [00:31<00:07, 15.05it/s]
Loading 0:  71%|███████   | 257/363 [00:32<00:08, 12.23it/s]
Loading 0:  72%|███████▏  | 262/363 [00:32<00:05, 17.44it/s]
Loading 0:  73%|███████▎  | 266/363 [00:32<00:04, 20.87it/s]
Loading 0:  74%|███████▍  | 269/363 [00:32<00:06, 15.28it/s]
Loading 0:  75%|███████▍  | 272/363 [00:32<00:05, 15.98it/s]
Loading 0:  76%|███████▌  | 275/363 [00:33<00:07, 12.34it/s]
Loading 0:  77%|███████▋  | 280/363 [00:33<00:04, 17.36it/s]
Loading 0:  78%|███████▊  | 284/363 [00:33<00:03, 21.02it/s]
Loading 0:  79%|███████▉  | 288/363 [00:33<00:05, 13.63it/s]
Loading 0:  80%|████████  | 291/363 [00:34<00:04, 15.44it/s]
Loading 0:  81%|████████  | 294/363 [00:34<00:04, 14.18it/s]
Loading 0:  82%|████████▏ | 298/363 [00:34<00:03, 18.02it/s]
Loading 0:  83%|████████▎ | 302/363 [00:34<00:02, 21.69it/s]
Loading 0:  84%|████████▍ | 305/363 [00:34<00:03, 15.41it/s]
Loading 0:  85%|████████▍ | 308/363 [00:35<00:03, 16.10it/s]
Loading 0:  86%|████████▌ | 311/363 [00:35<00:04, 12.72it/s]
Loading 0:  87%|████████▋ | 316/363 [00:35<00:02, 17.21it/s]
Loading 0:  88%|████████▊ | 320/363 [00:35<00:02, 20.39it/s]
Loading 0:  89%|████████▉ | 323/363 [00:36<00:02, 14.83it/s]
Loading 0:  90%|████████▉ | 326/363 [00:36<00:02, 15.65it/s]
Loading 0:  91%|█████████ | 329/363 [00:36<00:02, 12.61it/s]
Loading 0:  92%|█████████▏| 334/363 [00:36<00:01, 17.88it/s]
Loading 0:  93%|█████████▎| 339/363 [00:36<00:01, 18.61it/s]
Loading 0:  94%|█████████▍| 342/363 [00:37<00:01, 14.42it/s]
Loading 0:  95%|█████████▌| 345/363 [00:37<00:01, 16.35it/s]
Loading 0:  96%|█████████▌| 348/363 [00:37<00:01, 14.94it/s]
Loading 0:  97%|█████████▋| 352/363 [00:37<00:00, 18.91it/s]
Loading 0:  98%|█████████▊| 356/363 [00:37<00:00, 22.61it/s]
Loading 0:  99%|█████████▉| 359/363 [00:38<00:00, 11.51it/s]
Loading 0: 100%|█████████▉| 362/363 [00:38<00:00, 11.27it/s]
                                                            
Job rirv938-prefgrok-r2-cp62-5963-v2-mkmlizer completed after 261.88s with status: succeeded
Stopping job with name rirv938-prefgrok-r2-cp62-5963-v2-mkmlizer
Pipeline stage MKMLizer completed in 262.51s
run pipeline stage %s
Running pipeline stage MKMLTemplater
Pipeline stage MKMLTemplater completed in 0.15s
run pipeline stage %s
Running pipeline stage MKMLDeployer
Creating inference service rirv938-prefgrok-r2-cp62-5963-v2
Waiting for inference service rirv938-prefgrok-r2-cp62-5963-v2 to be ready
Failed to get response for submission chaiml-gy-exp129-dpo-ex_61642_v4: HTTPConnectionPool(host='chaiml-gy-exp129-dpo-ex-61642-v4-predictor.tenant-chaiml-guanaco.k.chaiverse.com', port=80): Read timed out. (read timeout=12.0)
Inference service rirv938-prefgrok-r2-cp62-5963-v2 ready after 200.99679470062256s
Pipeline stage MKMLDeployer completed in 201.45s
run pipeline stage %s
Running pipeline stage StressChecker
HTTPConnectionPool(host='guanaco-submitter.guanaco-backend.k2.chaiverse.com', port=80): Read timed out. (read timeout=20)
Received unhealthy response to inference request!
Received healthy response to inference request in 3.21481990814209s
Received healthy response to inference request in 2.2248308658599854s
Received healthy response to inference request in 1.996795654296875s
Received healthy response to inference request in 2.2107417583465576s
5 requests
1 failed requests
5th percentile: 2.0395848751068115
10th percentile: 2.082374095916748
20th percentile: 2.167952537536621
30th percentile: 2.2135595798492433
40th percentile: 2.2191952228546143
50th percentile: 2.2248308658599854
60th percentile: 2.620826482772827
70th percentile: 3.016822099685669
80th percentile: 6.5963496685028105
90th percentile: 13.359409189224245
95th percentile: 16.740938949584958
99th percentile: 19.446162757873534
mean time: 5.953931379318237
%s, retrying in %s seconds...
Received healthy response to inference request in 1.8251328468322754s
Received healthy response to inference request in 1.9576330184936523s
Received healthy response to inference request in 1.9376928806304932s
Received healthy response to inference request in 1.873037576675415s
Received healthy response to inference request in 3.255596399307251s
5 requests
0 failed requests
5th percentile: 1.8347137928009034
10th percentile: 1.8442947387695312
20th percentile: 1.863456630706787
30th percentile: 1.8859686374664306
40th percentile: 1.911830759048462
50th percentile: 1.9376928806304932
60th percentile: 1.9456689357757568
70th percentile: 1.9536449909210205
80th percentile: 2.2172256946563724
90th percentile: 2.7364110469818117
95th percentile: 2.996003723144531
99th percentile: 3.203677864074707
mean time: 2.1698185443878173
Pipeline stage StressChecker completed in 43.29s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyTriggerPipeline
run_pipeline:run_in_cloud %s
starting trigger_guanaco_pipeline args=%s
triggered trigger_guanaco_pipeline args=%s
Pipeline stage OfflineFamilyFriendlyTriggerPipeline completed in 0.90s
run pipeline stage %s
Running pipeline stage TriggerMKMLProfilingPipeline
run_pipeline:run_in_cloud %s
starting trigger_guanaco_pipeline args=%s
triggered trigger_guanaco_pipeline args=%s
Pipeline stage TriggerMKMLProfilingPipeline completed in 0.74s
Shutdown handler de-registered
rirv938-prefgrok-r2-cp62_5963_v2 status is now deployed due to DeploymentManager action
Shutdown handler registered
run pipeline %s
run pipeline stage %s
Running pipeline stage MKMLProfilerDeleter
Skipping teardown as no inference service was successfully deployed
Pipeline stage MKMLProfilerDeleter completed in 0.14s
run pipeline stage %s
Running pipeline stage MKMLProfilerTemplater
Pipeline stage MKMLProfilerTemplater completed in 0.12s
run pipeline stage %s
Running pipeline stage MKMLProfilerDeployer
Creating inference service rirv938-prefgrok-r2-cp62-5963-v2-profiler
Waiting for inference service rirv938-prefgrok-r2-cp62-5963-v2-profiler to be ready
Shutdown handler registered
run pipeline %s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyScorer
Evaluating %s Family Friendly Score with %s threads
%s, retrying in %s seconds...
Evaluating %s Family Friendly Score with %s threads
%s, retrying in %s seconds...
Evaluating %s Family Friendly Score with %s threads
Pipeline stage OfflineFamilyFriendlyScorer completed in 4116.24s
Shutdown handler de-registered
rirv938-prefgrok-r2-cp62_5963_v2 status is now inactive due to auto deactivation removed underperforming models
rirv938-prefgrok-r2-cp62_5963_v2 status is now torndown due to DeploymentManager action