developer_uid: rirv938
submission_id: rirv938-98p-2ff-rirv938_42155_v3
model_name: rirv938-98p-2ff-rirv938_42155_v3
model_group: rirv938/98p_2ff_rirv938_
status: inactive
timestamp: 2025-03-29T04:52:31+00:00
num_battles: 7394
num_wins: 3908
celo_rating: 1303.37
family_friendly_score: 0.5256000000000001
family_friendly_standard_error: 0.007061793539887725
submission_type: basic
model_repo: rirv938/98p_2ff_rirv938_0sw_99p_1ff_rir_73127_v1_cp936_v3
model_architecture: MistralForCausalLM
model_num_parameters: 24096691200.0
best_of: 1
max_input_tokens: 768
max_output_tokens: 64
latencies: [{'batch_size': 1, 'throughput': 0.4168396297794799, 'latency_mean': 2.3989179265499114, 'latency_p50': 2.3805298805236816, 'latency_p90': 2.696007823944092}, {'batch_size': 3, 'throughput': 0.934193604779234, 'latency_mean': 3.208598337173462, 'latency_p50': 3.2020870447158813, 'latency_p90': 3.5457112789154053}, {'batch_size': 5, 'throughput': 1.303604193000364, 'latency_mean': 3.8018136501312254, 'latency_p50': 3.787344455718994, 'latency_p90': 4.258811807632446}, {'batch_size': 6, 'throughput': 1.4462690691746598, 'latency_mean': 4.1091834008693695, 'latency_p50': 4.093625783920288, 'latency_p90': 4.598357224464417}, {'batch_size': 8, 'throughput': 1.6745066777901638, 'latency_mean': 4.731905019283294, 'latency_p50': 4.727911710739136, 'latency_p90': 5.251450943946838}, {'batch_size': 10, 'throughput': 1.8678044544686805, 'latency_mean': 5.291351915597915, 'latency_p50': 5.268207669258118, 'latency_p90': 5.975078821182251}]
gpu_counts: {'NVIDIA RTX A6000': 1}
display_name: rirv938-98p-2ff-rirv938_42155_v3
is_internal_developer: True
language_model: rirv938/98p_2ff_rirv938_0sw_99p_1ff_rir_73127_v1_cp936_v3
model_size: 24B
ranking_group: single
throughput_3p7s: 1.26
us_pacific_date: 2025-03-28
win_ratio: 0.5285366513389235
generation_params: {'temperature': 0.9, 'top_p': 0.9, 'min_p': 0.6, 'top_k': 80, 'presence_penalty': 0.5, 'frequency_penalty': 0.5, 'stopping_words': ['\n', '</s>', '###', 'You:'], 'max_input_tokens': 768, 'best_of': 1, 'max_output_tokens': 64}
formatter: {'memory_template': '[INST]', 'prompt_template': '', 'bot_template': '{bot_name}: {message}\n', 'user_template': '{user_name}: {message}\n', 'response_template': '[/INST]{bot_name}:', 'truncate_by_message': False}
Resubmit model
Shutdown handler not registered because Python interpreter is not running in the main thread
run pipeline %s
run pipeline stage %s
Running pipeline stage MKMLizer
Starting job with name rirv938-98p-2ff-rirv938-42155-v3-mkmlizer
Waiting for job on rirv938-98p-2ff-rirv938-42155-v3-mkmlizer to finish
rirv938-98p-2ff-rirv938-42155-v3-mkmlizer: ╔═════════════════════════════════════════════════════════════════════╗
rirv938-98p-2ff-rirv938-42155-v3-mkmlizer: ║     _____            __           __                                ║
rirv938-98p-2ff-rirv938-42155-v3-mkmlizer: ║    / _/ /_ ___    __/ /  ___ ___ / /                                ║
rirv938-98p-2ff-rirv938-42155-v3-mkmlizer: ║   / _/ / // / |/|/ / _ \/ -_) -_) /                                 ║
rirv938-98p-2ff-rirv938-42155-v3-mkmlizer: ║  /_//_/\_, /|__,__/_//_/\__/\__/_/                                  ║
rirv938-98p-2ff-rirv938-42155-v3-mkmlizer: ║       /___/                                                         ║
rirv938-98p-2ff-rirv938-42155-v3-mkmlizer: ║                                                                     ║
rirv938-98p-2ff-rirv938-42155-v3-mkmlizer: ║  Version: 0.12.8                                                    ║
rirv938-98p-2ff-rirv938-42155-v3-mkmlizer: ║  Copyright 2023 MK ONE TECHNOLOGIES Inc.                            ║
rirv938-98p-2ff-rirv938-42155-v3-mkmlizer: ║  https://mk1.ai                                                     ║
rirv938-98p-2ff-rirv938-42155-v3-mkmlizer: ║                                                                     ║
rirv938-98p-2ff-rirv938-42155-v3-mkmlizer: ║  The license key for the current software has been verified as      ║
rirv938-98p-2ff-rirv938-42155-v3-mkmlizer: ║  belonging to:                                                      ║
rirv938-98p-2ff-rirv938-42155-v3-mkmlizer: ║                                                                     ║
rirv938-98p-2ff-rirv938-42155-v3-mkmlizer: ║  Chai Research Corp.                                                ║
rirv938-98p-2ff-rirv938-42155-v3-mkmlizer: ║  Account ID: 7997a29f-0ceb-4cc7-9adf-840c57b4ae6f                   ║
rirv938-98p-2ff-rirv938-42155-v3-mkmlizer: ║  Expiration: 2025-04-15 23:59:59                                    ║
rirv938-98p-2ff-rirv938-42155-v3-mkmlizer: ║                                                                     ║
rirv938-98p-2ff-rirv938-42155-v3-mkmlizer: ╚═════════════════════════════════════════════════════════════════════╝
rirv938-98p-2ff-rirv938-42155-v3-mkmlizer: Downloaded to shared memory in 102.849s
rirv938-98p-2ff-rirv938-42155-v3-mkmlizer: quantizing model to /dev/shm/model_cache, profile:s0, folder:/tmp/tmp3h718wb0, device:0
rirv938-98p-2ff-rirv938-42155-v3-mkmlizer: Saving flywheel model at /dev/shm/model_cache
rirv938-98p-2ff-rirv938-42155-v3-mkmlizer: quantized model in 61.351s
rirv938-98p-2ff-rirv938-42155-v3-mkmlizer: Processed model rirv938/98p_2ff_rirv938_0sw_99p_1ff_rir_73127_v1_cp936_v3 in 164.200s
rirv938-98p-2ff-rirv938-42155-v3-mkmlizer: creating bucket guanaco-mkml-models
rirv938-98p-2ff-rirv938-42155-v3-mkmlizer: cp /dev/shm/model_cache/tokenizer_config.json s3://guanaco-mkml-models/rirv938-98p-2ff-rirv938-42155-v3/tokenizer_config.json
rirv938-98p-2ff-rirv938-42155-v3-mkmlizer: cp /dev/shm/model_cache/tokenizer.json s3://guanaco-mkml-models/rirv938-98p-2ff-rirv938-42155-v3/tokenizer.json
rirv938-98p-2ff-rirv938-42155-v3-mkmlizer: cp /dev/shm/model_cache/flywheel_model.1.safetensors s3://guanaco-mkml-models/rirv938-98p-2ff-rirv938-42155-v3/flywheel_model.1.safetensors
rirv938-98p-2ff-rirv938-42155-v3-mkmlizer: cp /dev/shm/model_cache/flywheel_model.0.safetensors s3://guanaco-mkml-models/rirv938-98p-2ff-rirv938-42155-v3/flywheel_model.0.safetensors
rirv938-98p-2ff-rirv938-42155-v3-mkmlizer: 
Loading 0:   0%|          | 0/363 [00:00<?, ?it/s]
Loading 0:   1%|          | 4/363 [00:00<00:17, 20.55it/s]
Loading 0:   2%|▏         | 7/363 [00:00<00:20, 17.74it/s]
Loading 0:   3%|▎         | 11/363 [00:00<00:15, 22.79it/s]
Loading 0:   4%|▍         | 14/363 [00:00<00:25, 13.79it/s]
Loading 0:   4%|▍         | 16/363 [00:01<00:26, 13.32it/s]
Loading 0:   6%|▌         | 21/363 [00:01<00:19, 17.23it/s]
Loading 0:   6%|▋         | 23/363 [00:01<00:24, 13.68it/s]
Loading 0:   8%|▊         | 29/363 [00:01<00:15, 21.46it/s]
Loading 0:   9%|▉         | 33/363 [00:01<00:16, 19.56it/s]
Loading 0:  10%|▉         | 36/363 [00:02<00:22, 14.51it/s]
Loading 0:  11%|█         | 39/363 [00:02<00:19, 16.76it/s]
Loading 0:  12%|█▏        | 42/363 [00:02<00:21, 15.21it/s]
Loading 0:  13%|█▎        | 47/363 [00:02<00:15, 20.68it/s]
Loading 0:  14%|█▍        | 51/363 [00:02<00:15, 19.66it/s]
Loading 0:  15%|█▍        | 54/363 [00:03<00:20, 14.91it/s]
Loading 0:  16%|█▌        | 57/363 [00:03<00:18, 16.99it/s]
Loading 0:  17%|█▋        | 60/363 [00:03<00:19, 15.36it/s]
Loading 0:  18%|█▊        | 64/363 [00:03<00:15, 19.11it/s]
Loading 0:  19%|█▉        | 69/363 [00:03<00:14, 19.63it/s]
Loading 0:  20%|█▉        | 72/363 [00:04<00:19, 14.71it/s]
Loading 0:  21%|██        | 75/363 [00:04<00:17, 16.77it/s]
Loading 0:  21%|██▏       | 78/363 [00:04<00:18, 15.10it/s]
Loading 0:  23%|██▎       | 82/363 [00:04<00:14, 19.04it/s]
Loading 0:  24%|██▍       | 87/363 [00:05<00:14, 19.43it/s]
Loading 0:  25%|██▍       | 90/363 [00:05<00:18, 14.80it/s]
Loading 0:  26%|██▋       | 96/363 [00:05<00:12, 21.33it/s]
Loading 0:  28%|██▊       | 101/363 [00:05<00:11, 22.65it/s]
Loading 0:  29%|██▊       | 104/363 [00:05<00:14, 18.41it/s]
Loading 0:  29%|██▉       | 107/363 [00:06<00:17, 14.25it/s]
Loading 0:  30%|███       | 109/363 [00:06<00:18, 13.83it/s]
Loading 0:  31%|███       | 111/363 [00:06<00:17, 14.40it/s]
Loading 0:  31%|███       | 113/363 [00:06<00:20, 12.17it/s]
Loading 0:  33%|███▎      | 118/363 [00:06<00:13, 18.47it/s]
Loading 0:  34%|███▍      | 123/363 [00:07<00:12, 19.18it/s]
Loading 0:  35%|███▍      | 126/363 [00:07<00:16, 14.32it/s]
Loading 0:  36%|███▌      | 129/363 [00:07<00:14, 16.40it/s]
Loading 0:  36%|███▋      | 132/363 [00:07<00:15, 14.69it/s]
Loading 0:  37%|███▋      | 136/363 [00:08<00:12, 18.35it/s]
Loading 0:  39%|███▊      | 140/363 [00:08<00:10, 21.75it/s]
Loading 0:  39%|███▉      | 143/363 [00:08<00:14, 14.95it/s]
Loading 0:  40%|████      | 146/363 [00:08<00:13, 15.56it/s]
Loading 0:  41%|████      | 149/363 [00:09<00:17, 12.42it/s]
Loading 0:  43%|████▎     | 155/363 [00:09<00:11, 18.89it/s]
Loading 0:  44%|████▎     | 158/363 [00:09<00:09, 20.59it/s]
Loading 0:  44%|████▍     | 161/363 [00:09<00:13, 15.31it/s]
Loading 0:  45%|████▌     | 164/363 [00:09<00:12, 16.10it/s]
Loading 0:  46%|████▌     | 167/363 [00:10<00:15, 12.92it/s]
Loading 0:  48%|████▊     | 173/363 [00:10<00:09, 19.58it/s]
Loading 0:  49%|████▉     | 177/363 [00:10<00:09, 18.76it/s]
Loading 0:  50%|████▉     | 180/363 [00:10<00:12, 14.53it/s]
Loading 0:  50%|█████     | 183/363 [00:10<00:10, 16.43it/s]
Loading 0:  51%|█████     | 186/363 [00:11<00:11, 15.12it/s]
Loading 0:  53%|█████▎    | 191/363 [00:11<00:08, 20.36it/s]
Loading 0:  54%|█████▎    | 195/363 [00:11<00:08, 18.89it/s]
Loading 0:  55%|█████▍    | 198/363 [00:11<00:11, 14.36it/s]
Loading 0:  55%|█████▌    | 200/363 [00:26<00:11, 14.36it/s]
Loading 0:  55%|█████▌    | 201/363 [00:26<03:29,  1.29s/it]
Loading 0:  56%|█████▌    | 203/363 [00:26<02:49,  1.06s/it]
Loading 0:  57%|█████▋    | 207/363 [00:26<01:47,  1.46it/s]
Loading 0:  58%|█████▊    | 210/363 [00:26<01:17,  1.98it/s]
Loading 0:  59%|█████▊    | 213/363 [00:26<00:57,  2.59it/s]
Loading 0:  59%|█████▉    | 215/363 [00:27<00:47,  3.08it/s]
Loading 0:  60%|█████▉    | 217/363 [00:27<00:38,  3.75it/s]
Loading 0:  60%|██████    | 219/363 [00:27<00:30,  4.69it/s]
Loading 0:  61%|██████    | 221/363 [00:27<00:27,  5.22it/s]
Loading 0:  62%|██████▏   | 226/363 [00:27<00:14,  9.21it/s]
Loading 0:  64%|██████▎   | 231/363 [00:27<00:11, 11.86it/s]
Loading 0:  64%|██████▍   | 234/363 [00:28<00:11, 10.80it/s]
Loading 0:  65%|██████▌   | 237/363 [00:28<00:09, 12.94it/s]
Loading 0:  66%|██████▌   | 240/363 [00:28<00:09, 12.41it/s]
Loading 0:  67%|██████▋   | 245/363 [00:28<00:06, 17.59it/s]
Loading 0:  69%|██████▊   | 249/363 [00:29<00:06, 17.29it/s]
Loading 0:  69%|██████▉   | 252/363 [00:29<00:08, 13.50it/s]
Loading 0:  70%|███████   | 255/363 [00:29<00:06, 15.45it/s]
Loading 0:  71%|███████   | 258/363 [00:29<00:07, 14.07it/s]
Loading 0:  72%|███████▏  | 262/363 [00:29<00:05, 18.02it/s]
Loading 0:  73%|███████▎  | 266/363 [00:30<00:04, 21.80it/s]
Loading 0:  74%|███████▍  | 269/363 [00:30<00:05, 15.73it/s]
Loading 0:  75%|███████▍  | 272/363 [00:30<00:05, 16.32it/s]
Loading 0:  76%|███████▌  | 275/363 [00:30<00:06, 12.75it/s]
Loading 0:  77%|███████▋  | 280/363 [00:30<00:04, 17.93it/s]
Loading 0:  78%|███████▊  | 284/363 [00:31<00:03, 21.52it/s]
Loading 0:  79%|███████▉  | 287/363 [00:31<00:04, 15.79it/s]
Loading 0:  80%|███████▉  | 290/363 [00:31<00:04, 16.50it/s]
Loading 0:  81%|████████  | 293/363 [00:31<00:05, 12.83it/s]
Loading 0:  82%|████████▏ | 299/363 [00:32<00:03, 19.49it/s]
Loading 0:  83%|████████▎ | 303/363 [00:32<00:03, 17.92it/s]
Loading 0:  84%|████████▍ | 306/363 [00:32<00:04, 14.21it/s]
Loading 0:  85%|████████▌ | 309/363 [00:32<00:03, 16.13it/s]
Loading 0:  86%|████████▌ | 312/363 [00:33<00:03, 14.98it/s]
Loading 0:  87%|████████▋ | 317/363 [00:33<00:02, 20.41it/s]
Loading 0:  88%|████████▊ | 321/363 [00:33<00:02, 19.36it/s]
Loading 0:  89%|████████▉ | 324/363 [00:33<00:02, 14.83it/s]
Loading 0:  90%|█████████ | 327/363 [00:33<00:02, 16.79it/s]
Loading 0:  91%|█████████ | 330/363 [00:34<00:02, 15.38it/s]
Loading 0:  92%|█████████▏| 335/363 [00:34<00:01, 20.94it/s]
Loading 0:  93%|█████████▎| 339/363 [00:34<00:01, 19.12it/s]
Loading 0:  94%|█████████▍| 342/363 [00:34<00:01, 14.65it/s]
Loading 0:  95%|█████████▌| 345/363 [00:34<00:01, 16.76it/s]
Loading 0:  96%|█████████▌| 348/363 [00:35<00:00, 15.36it/s]
Loading 0:  97%|█████████▋| 353/363 [00:35<00:00, 20.75it/s]
Loading 0:  98%|█████████▊| 357/363 [00:42<00:03,  1.73it/s]
Loading 0:  99%|█████████▉| 360/363 [00:42<00:01,  2.23it/s]
Loading 0: 100%|█████████▉| 362/363 [00:42<00:00,  2.64it/s]
                                                            

Job rirv938-98p-2ff-rirv938-42155-v3-mkmlizer completed after 196.65s with status: succeeded
Stopping job with name rirv938-98p-2ff-rirv938-42155-v3-mkmlizer
Pipeline stage MKMLizer completed in 197.10s
run pipeline stage %s
Running pipeline stage MKMLTemplater
Pipeline stage MKMLTemplater completed in 0.14s
run pipeline stage %s
Running pipeline stage MKMLDeployer
Creating inference service rirv938-98p-2ff-rirv938-42155-v3
Waiting for inference service rirv938-98p-2ff-rirv938-42155-v3 to be ready
Inference service rirv938-98p-2ff-rirv938-42155-v3 ready after 120.39721322059631s
Pipeline stage MKMLDeployer completed in 120.90s
run pipeline stage %s
Running pipeline stage StressChecker
Received healthy response to inference request in 2.6695821285247803s
Received healthy response to inference request in 2.1591835021972656s
Received healthy response to inference request in 1.9976441860198975s
Received healthy response to inference request in 2.151780128479004s
Received healthy response to inference request in 2.1453676223754883s
5 requests
0 failed requests
5th percentile: 2.0271888732910157
10th percentile: 2.056733560562134
20th percentile: 2.11582293510437
30th percentile: 2.1466501235961912
40th percentile: 2.1492151260375976
50th percentile: 2.151780128479004
60th percentile: 2.1547414779663088
70th percentile: 2.157702827453613
80th percentile: 2.2612632274627686
90th percentile: 2.4654226779937742
95th percentile: 2.5675024032592773
99th percentile: 2.6491661834716798
mean time: 2.224711513519287
Pipeline stage StressChecker completed in 12.37s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyTriggerPipeline
run_pipeline:run_in_cloud %s
starting trigger_guanaco_pipeline args=%s
triggered trigger_guanaco_pipeline args=%s
Pipeline stage OfflineFamilyFriendlyTriggerPipeline completed in 0.66s
run pipeline stage %s
Running pipeline stage TriggerMKMLProfilingPipeline
run_pipeline:run_in_cloud %s
starting trigger_guanaco_pipeline args=%s
triggered trigger_guanaco_pipeline args=%s
Pipeline stage TriggerMKMLProfilingPipeline completed in 0.66s
Shutdown handler de-registered
rirv938-98p-2ff-rirv938_42155_v3 status is now deployed due to DeploymentManager action
Shutdown handler registered
run pipeline %s
Shutdown handler registered
run pipeline %s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyScorer
Evaluating %s Family Friendly Score with %s threads
Pipeline stage OfflineFamilyFriendlyScorer completed in 2655.82s
Shutdown handler de-registered
rirv938-98p-2ff-rirv938_42155_v3 status is now inactive due to auto deactivation removed underperforming models
ChatRequest
Generation Params
Prompt Formatter
Chat History
ChatMessage 1