rirv938-98p-2ff-rirv938_86941

developer_uid: rirv938

submission_id: rirv938-98p-2ff-rirv938_86941_v1

model_name: rirv938-98p-2ff-rirv938_86941_v1

model_group: rirv938/98p_2ff_rirv938_

status: inactive

timestamp: 2025-04-01T23:58:16+00:00

num_battles: 6405

num_wins: 3298

celo_rating: 1303.71

family_friendly_score: 0.5706

family_friendly_standard_error: 0.007000223425005805

submission_type: basic

model_repo: rirv938/98p_2ff_rirv938_mistral_24b_bon_31129_v6_cp625_v4

model_architecture: MistralForCausalLM

model_num_parameters: 24096691200.0

best_of: 8

max_input_tokens: 768

max_output_tokens: 64

latencies: [{'batch_size': 1, 'throughput': 0.3924291719070112, 'latency_mean': 2.5481374180316925, 'latency_p50': 2.5258824825286865, 'latency_p90': 2.85381178855896}, {'batch_size': 3, 'throughput': 0.8049406454715639, 'latency_mean': 3.7214417123794554, 'latency_p50': 3.755850911140442, 'latency_p90': 4.0794929504394535}, {'batch_size': 5, 'throughput': 1.071311755717225, 'latency_mean': 4.647084816694259, 'latency_p50': 4.628069639205933, 'latency_p90': 5.168895959854126}, {'batch_size': 6, 'throughput': 1.1508114542361054, 'latency_mean': 5.1834890186786655, 'latency_p50': 5.1497732400894165, 'latency_p90': 5.864309096336365}, {'batch_size': 10, 'throughput': 1.3388120441878288, 'latency_mean': 7.387464625835419, 'latency_p50': 7.415208220481873, 'latency_p90': 8.277838087081909}]

gpu_counts: {'NVIDIA RTX A6000': 1}

display_name: rirv938-98p-2ff-rirv938_86941_v1

is_internal_developer: True

language_model: rirv938/98p_2ff_rirv938_mistral_24b_bon_31129_v6_cp625_v4

model_size: 24B

ranking_group: single

throughput_3p7s: 0.8

us_pacific_date: 2025-04-01

win_ratio: 0.5149102263856362

generation_params: {'temperature': 0.9, 'top_p': 0.9, 'min_p': 0.6, 'top_k': 80, 'presence_penalty': 0.5, 'frequency_penalty': 0.5, 'stopping_words': ['\n', '</s>', '###', 'You:'], 'max_input_tokens': 768, 'best_of': 8, 'max_output_tokens': 64}

formatter: {'memory_template': '[INST]', 'prompt_template': '', 'bot_template': '{bot_name}: {message}\n', 'user_template': '{user_name}: {message}\n', 'response_template': '[/INST]{bot_name}:', 'truncate_by_message': False}

Resubmit model

Shutdown handler not registered because Python interpreter is not running in the main thread
run pipeline %s
run pipeline stage %s
Running pipeline stage MKMLizer
Starting job with name rirv938-98p-2ff-rirv938-86941-v1-mkmlizer
Waiting for job on rirv938-98p-2ff-rirv938-86941-v1-mkmlizer to finish
rirv938-98p-2ff-rirv938-86941-v1-mkmlizer: Downloaded to shared memory in 169.168s
rirv938-98p-2ff-rirv938-86941-v1-mkmlizer: quantizing model to /dev/shm/model_cache, profile:s0, folder:/tmp/tmpb61rx06n, device:0
rirv938-98p-2ff-rirv938-86941-v1-mkmlizer: Saving flywheel model at /dev/shm/model_cache
rirv938-98p-2ff-rirv938-86941-v1-mkmlizer: quantized model in 66.728s
rirv938-98p-2ff-rirv938-86941-v1-mkmlizer: Processed model rirv938/98p_2ff_rirv938_mistral_24b_bon_31129_v6_cp625_v4 in 235.897s
rirv938-98p-2ff-rirv938-86941-v1-mkmlizer: Bucket 's3://guanaco-mkml-models/' created
rirv938-98p-2ff-rirv938-86941-v1-mkmlizer: uploading /dev/shm/model_cache to s3://guanaco-mkml-models/rirv938-98p-2ff-rirv938-86941-v1
rirv938-98p-2ff-rirv938-86941-v1-mkmlizer: cp /dev/shm/model_cache/config.json s3://guanaco-mkml-models/rirv938-98p-2ff-rirv938-86941-v1/config.json
rirv938-98p-2ff-rirv938-86941-v1-mkmlizer: cp /dev/shm/model_cache/special_tokens_map.json s3://guanaco-mkml-models/rirv938-98p-2ff-rirv938-86941-v1/special_tokens_map.json
rirv938-98p-2ff-rirv938-86941-v1-mkmlizer: cp /dev/shm/model_cache/tokenizer_config.json s3://guanaco-mkml-models/rirv938-98p-2ff-rirv938-86941-v1/tokenizer_config.json
rirv938-98p-2ff-rirv938-86941-v1-mkmlizer: cp /dev/shm/model_cache/tokenizer.json s3://guanaco-mkml-models/rirv938-98p-2ff-rirv938-86941-v1/tokenizer.json
rirv938-98p-2ff-rirv938-86941-v1-mkmlizer: cp /dev/shm/model_cache/flywheel_model.1.safetensors s3://guanaco-mkml-models/rirv938-98p-2ff-rirv938-86941-v1/flywheel_model.1.safetensors
rirv938-98p-2ff-rirv938-86941-v1-mkmlizer: cp /dev/shm/model_cache/flywheel_model.0.safetensors s3://guanaco-mkml-models/rirv938-98p-2ff-rirv938-86941-v1/flywheel_model.0.safetensors
rirv938-98p-2ff-rirv938-86941-v1-mkmlizer: 
Loading 0:   0%|          | 0/363 [00:00<?, ?it/s]
Loading 0:   1%|          | 3/363 [00:00<00:13, 26.16it/s]
Loading 0:   2%|▏         | 6/363 [00:00<00:28, 12.73it/s]
Loading 0:   3%|▎         | 10/363 [00:00<00:18, 19.15it/s]
Loading 0:   4%|▎         | 13/363 [00:00<00:27, 12.81it/s]
Loading 0:   4%|▍         | 15/363 [00:01<00:33, 10.37it/s]
Loading 0:   5%|▌         | 19/363 [00:01<00:23, 14.45it/s]
Loading 0:   6%|▌         | 21/363 [00:01<00:23, 14.68it/s]
Loading 0:   6%|▋         | 23/363 [00:01<00:30, 11.07it/s]
Loading 0:   8%|▊         | 28/363 [00:01<00:20, 16.57it/s]
Loading 0:   9%|▉         | 32/363 [00:02<00:16, 19.68it/s]
Loading 0:  10%|▉         | 35/363 [00:02<00:23, 13.81it/s]
Loading 0:  10%|█         | 37/363 [00:02<00:24, 13.29it/s]
Loading 0:  11%|█         | 39/363 [00:02<00:22, 14.10it/s]
Loading 0:  11%|█▏        | 41/363 [00:02<00:27, 11.67it/s]
Loading 0:  13%|█▎        | 46/363 [00:03<00:17, 18.08it/s]
Loading 0:  14%|█▍        | 50/363 [00:03<00:14, 22.28it/s]
Loading 0:  15%|█▍        | 53/363 [00:03<00:19, 15.58it/s]
Loading 0:  15%|█▌        | 56/363 [00:03<00:18, 16.18it/s]
Loading 0:  16%|█▋        | 59/363 [00:04<00:24, 12.45it/s]
Loading 0:  18%|█▊        | 64/363 [00:04<00:16, 17.80it/s]
Loading 0:  19%|█▊        | 68/363 [00:04<00:13, 21.53it/s]
Loading 0:  20%|█▉        | 72/363 [00:04<00:21, 13.70it/s]
Loading 0:  21%|██        | 75/363 [00:04<00:18, 15.60it/s]
Loading 0:  21%|██▏       | 78/363 [00:05<00:20, 13.63it/s]
Loading 0:  23%|██▎       | 82/363 [00:05<00:16, 16.94it/s]
Loading 0:  24%|██▎       | 86/363 [00:05<00:13, 20.14it/s]
Loading 0:  25%|██▍       | 89/363 [00:05<00:18, 14.51it/s]
Loading 0:  25%|██▌       | 92/363 [00:06<00:18, 14.49it/s]
Loading 0:  26%|██▌       | 95/363 [00:06<00:16, 16.03it/s]
Loading 0:  27%|██▋       | 99/363 [00:06<00:13, 19.72it/s]
Loading 0:  28%|██▊       | 102/363 [00:06<00:15, 17.27it/s]
Loading 0:  29%|██▉       | 105/363 [00:06<00:18, 13.72it/s]
Loading 0:  29%|██▉       | 107/363 [00:07<00:20, 12.40it/s]
Loading 0:  30%|███       | 109/363 [00:07<00:20, 12.13it/s]
Loading 0:  31%|███       | 111/363 [00:07<00:19, 13.02it/s]
Loading 0:  31%|███       | 113/363 [00:07<00:22, 10.96it/s]
Loading 0:  33%|███▎      | 118/363 [00:07<00:14, 17.26it/s]
Loading 0:  34%|███▎      | 122/363 [00:07<00:11, 21.37it/s]
Loading 0:  34%|███▍      | 125/363 [00:08<00:15, 15.10it/s]
Loading 0:  35%|███▌      | 128/363 [00:08<00:14, 15.75it/s]
Loading 0:  36%|███▌      | 131/363 [00:08<00:18, 12.31it/s]
Loading 0:  37%|███▋      | 136/363 [00:08<00:13, 17.42it/s]
Loading 0:  39%|███▊      | 140/363 [00:08<00:10, 21.00it/s]
Loading 0:  39%|███▉      | 143/363 [00:09<00:14, 15.34it/s]
Loading 0:  40%|████      | 146/363 [00:09<00:13, 15.85it/s]
Loading 0:  41%|████      | 149/363 [00:09<00:17, 12.48it/s]
Loading 0:  42%|████▏     | 154/363 [00:09<00:11, 17.55it/s]
Loading 0:  44%|████▎     | 158/363 [00:10<00:09, 20.95it/s]
Loading 0:  44%|████▍     | 161/363 [00:10<00:13, 15.37it/s]
Loading 0:  45%|████▌     | 164/363 [00:10<00:12, 15.77it/s]
Loading 0:  46%|████▌     | 167/363 [00:11<00:16, 12.00it/s]
Loading 0:  47%|████▋     | 172/363 [00:11<00:11, 16.60it/s]
Loading 0:  48%|████▊     | 176/363 [00:11<00:09, 19.40it/s]
Loading 0:  49%|████▉     | 179/363 [00:11<00:13, 14.05it/s]
Loading 0:  50%|█████     | 182/363 [00:11<00:12, 14.35it/s]
Loading 0:  51%|█████     | 184/363 [00:12<00:14, 12.76it/s]
Loading 0:  51%|█████     | 186/363 [00:12<00:14, 12.01it/s]
Loading 0:  52%|█████▏    | 190/363 [00:12<00:10, 15.77it/s]
Loading 0:  53%|█████▎    | 194/363 [00:12<00:08, 18.93it/s]
Loading 0:  54%|█████▍    | 197/363 [00:12<00:12, 13.65it/s]
Loading 0:  55%|█████▍    | 199/363 [00:13<00:12, 12.93it/s]
Loading 0:  55%|█████▌    | 201/363 [00:28<04:51,  1.80s/it]
Loading 0:  56%|█████▌    | 203/363 [00:28<03:42,  1.39s/it]
Loading 0:  57%|█████▋    | 207/363 [00:28<02:08,  1.21it/s]
Loading 0:  58%|█████▊    | 210/363 [00:28<01:29,  1.70it/s]
Loading 0:  59%|█████▊    | 213/363 [00:29<01:05,  2.28it/s]
Loading 0:  59%|█████▉    | 215/363 [00:29<00:53,  2.77it/s]
Loading 0:  60%|█████▉    | 217/363 [00:29<00:43,  3.39it/s]
Loading 0:  60%|██████    | 219/363 [00:29<00:33,  4.29it/s]
Loading 0:  61%|██████    | 221/363 [00:29<00:29,  4.80it/s]
Loading 0:  62%|██████▏   | 226/363 [00:30<00:16,  8.45it/s]
Loading 0:  63%|██████▎   | 230/363 [00:30<00:11, 11.56it/s]
Loading 0:  64%|██████▍   | 233/363 [00:30<00:12, 10.41it/s]
Loading 0:  65%|██████▍   | 235/363 [00:30<00:12, 10.61it/s]
Loading 0:  65%|██████▌   | 237/363 [00:30<00:10, 11.69it/s]
Loading 0:  66%|██████▌   | 239/363 [00:31<00:12, 10.18it/s]
Loading 0:  67%|██████▋   | 244/363 [00:31<00:07, 15.78it/s]
Loading 0:  68%|██████▊   | 248/363 [00:31<00:05, 19.51it/s]
Loading 0:  69%|██████▉   | 251/363 [00:31<00:07, 14.06it/s]
Loading 0:  70%|██████▉   | 254/363 [00:31<00:07, 14.87it/s]
Loading 0:  71%|███████   | 256/363 [00:32<00:08, 13.24it/s]
Loading 0:  71%|███████   | 258/363 [00:32<00:08, 12.80it/s]
Loading 0:  72%|███████▏  | 262/363 [00:32<00:05, 16.93it/s]
Loading 0:  73%|███████▎  | 266/363 [00:32<00:04, 20.60it/s]
Loading 0:  74%|███████▍  | 269/363 [00:32<00:06, 14.24it/s]
Loading 0:  75%|███████▍  | 271/363 [00:33<00:06, 13.49it/s]
Loading 0:  75%|███████▌  | 273/363 [00:33<00:06, 14.34it/s]
Loading 0:  76%|███████▌  | 275/363 [00:33<00:07, 11.51it/s]
Loading 0:  77%|███████▋  | 280/363 [00:33<00:04, 17.34it/s]
Loading 0:  78%|███████▊  | 284/363 [00:33<00:03, 20.95it/s]
Loading 0:  79%|███████▉  | 287/363 [00:34<00:05, 14.77it/s]
Loading 0:  80%|███████▉  | 290/363 [00:34<00:04, 15.32it/s]
Loading 0:  80%|████████  | 292/363 [00:34<00:05, 13.50it/s]
Loading 0:  81%|████████  | 294/363 [00:34<00:05, 12.53it/s]
Loading 0:  82%|████████▏ | 298/363 [00:34<00:03, 16.63it/s]
Loading 0:  83%|████████▎ | 302/363 [00:34<00:03, 20.22it/s]
Loading 0:  84%|████████▍ | 305/363 [00:35<00:04, 14.30it/s]
Loading 0:  85%|████████▍ | 307/363 [00:35<00:04, 13.65it/s]
Loading 0:  85%|████████▌ | 309/363 [00:35<00:03, 14.47it/s]
Loading 0:  86%|████████▌ | 311/363 [00:35<00:04, 11.50it/s]
Loading 0:  87%|████████▋ | 316/363 [00:35<00:02, 17.11it/s]
Loading 0:  88%|████████▊ | 320/363 [00:36<00:02, 20.55it/s]
Loading 0:  89%|████████▉ | 323/363 [00:36<00:02, 14.71it/s]
Loading 0:  90%|████████▉ | 325/363 [00:36<00:02, 13.74it/s]
Loading 0:  90%|█████████ | 327/363 [00:36<00:02, 14.52it/s]
Loading 0:  91%|█████████ | 329/363 [00:36<00:02, 11.60it/s]
Loading 0:  92%|█████████▏| 334/363 [00:37<00:01, 17.41it/s]
Loading 0:  93%|█████████▎| 338/363 [00:37<00:01, 20.98it/s]
Loading 0:  94%|█████████▍| 341/363 [00:37<00:01, 14.43it/s]
Loading 0:  95%|█████████▍| 344/363 [00:37<00:01, 14.89it/s]
Loading 0:  95%|█████████▌| 346/363 [00:37<00:01, 13.22it/s]
Loading 0:  96%|█████████▌| 348/363 [00:38<00:01, 12.67it/s]
Loading 0:  97%|█████████▋| 352/363 [00:38<00:00, 16.70it/s]
Loading 0:  98%|█████████▊| 356/363 [00:38<00:00, 20.47it/s]
Loading 0:  99%|█████████▉| 359/363 [00:45<00:02,  1.37it/s]
Loading 0:  99%|█████████▉| 361/363 [00:46<00:01,  1.65it/s]
                                                            
Job rirv938-98p-2ff-rirv938-86941-v1-mkmlizer completed after 277.63s with status: succeeded
Stopping job with name rirv938-98p-2ff-rirv938-86941-v1-mkmlizer
Pipeline stage MKMLizer completed in 278.07s
run pipeline stage %s
Running pipeline stage MKMLTemplater
Pipeline stage MKMLTemplater completed in 0.16s
run pipeline stage %s
Running pipeline stage MKMLDeployer
Creating inference service rirv938-98p-2ff-rirv938-86941-v1
Waiting for inference service rirv938-98p-2ff-rirv938-86941-v1 to be ready
Inference service rirv938-98p-2ff-rirv938-86941-v1 ready after 110.69523334503174s
Pipeline stage MKMLDeployer completed in 111.17s
run pipeline stage %s
Running pipeline stage StressChecker
Received healthy response to inference request in 2.9053332805633545s
Received healthy response to inference request in 2.467643976211548s
Received healthy response to inference request in 2.5167222023010254s
Received healthy response to inference request in 2.429485559463501s
Received healthy response to inference request in 2.691779375076294s
5 requests
0 failed requests
5th percentile: 2.4371172428131103
10th percentile: 2.4447489261627195
20th percentile: 2.4600122928619386
30th percentile: 2.4774596214294435
40th percentile: 2.4970909118652345
50th percentile: 2.5167222023010254
60th percentile: 2.5867450714111326
70th percentile: 2.6567679405212403
80th percentile: 2.734490156173706
90th percentile: 2.8199117183685303
95th percentile: 2.8626224994659424
99th percentile: 2.896791124343872
mean time: 2.6021928787231445
Pipeline stage StressChecker completed in 14.11s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyTriggerPipeline
run_pipeline:run_in_cloud %s
starting trigger_guanaco_pipeline args=%s
triggered trigger_guanaco_pipeline args=%s
Pipeline stage OfflineFamilyFriendlyTriggerPipeline completed in 0.60s
run pipeline stage %s
Running pipeline stage TriggerMKMLProfilingPipeline
run_pipeline:run_in_cloud %s
starting trigger_guanaco_pipeline args=%s
triggered trigger_guanaco_pipeline args=%s
Pipeline stage TriggerMKMLProfilingPipeline completed in 0.79s
Shutdown handler de-registered
rirv938-98p-2ff-rirv938_86941_v1 status is now deployed due to DeploymentManager action
Shutdown handler registered
run pipeline %s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyScorer
Evaluating %s Family Friendly Score with %s threads
Pipeline stage OfflineFamilyFriendlyScorer completed in 2967.81s
Shutdown handler de-registered
rirv938-98p-2ff-rirv938_86941_v1 status is now inactive due to auto deactivation removed underperforming models

ChatRequest

Bot Name

Generation Params

Prompt Formatter

ChatMessage 1

Sender

Message

User Message