chaiml-oscar-piastri250_77993

developer_uid: chai_backend_admin

submission_id: chaiml-oscar-piastri250_77993_v1

model_name: chaiml-oscar-piastri250_77993_v1

model_group: ChaiML/oscar-piastri2507

status: torndown

timestamp: 2025-07-17T06:01:24+00:00

num_battles: 5559

num_wins: 2775

celo_rating: 1293.82

family_friendly_score: 0.48560000000000003

family_friendly_standard_error: 0.007068134690284276

submission_type: basic

model_repo: ChaiML/oscar-piastri250716224452_sft

model_architecture: MistralForCausalLM

model_num_parameters: 24096691200.0

best_of: 8

max_input_tokens: 1024

max_output_tokens: 64

reward_model: default

latencies: [{'batch_size': 1, 'throughput': 0.3602509315107666, 'latency_mean': 2.7757381880283356, 'latency_p50': 2.8153616189956665, 'latency_p90': 3.0520567655563355}, {'batch_size': 2, 'throughput': 0.564320497544452, 'latency_mean': 3.538478627204895, 'latency_p50': 3.534145951271057, 'latency_p90': 3.916126775741577}, {'batch_size': 3, 'throughput': 0.70902364854815, 'latency_mean': 4.2169785714149475, 'latency_p50': 4.219638824462891, 'latency_p90': 4.696030688285828}, {'batch_size': 4, 'throughput': 0.8284536214563354, 'latency_mean': 4.816140288114548, 'latency_p50': 4.826171517372131, 'latency_p90': 5.330316090583801}, {'batch_size': 5, 'throughput': 0.9071974432469518, 'latency_mean': 5.474710693359375, 'latency_p50': 5.487564206123352, 'latency_p90': 6.080595874786377}]

gpu_counts: {'NVIDIA RTX A6000': 1}

display_name: chaiml-oscar-piastri250_77993_v1

is_internal_developer: True

language_model: ChaiML/oscar-piastri250716224452_sft

model_size: 24B

ranking_group: single

throughput_3p7s: 0.6

us_pacific_date: 2025-07-16

win_ratio: 0.4991905018888289

generation_params: {'temperature': 1.0, 'top_p': 1.0, 'min_p': 0.0, 'top_k': 40, 'presence_penalty': 0.0, 'frequency_penalty': 0.0, 'stopping_words': ['</s>', '####\n', '\n', 'You:', '####'], 'max_input_tokens': 1024, 'best_of': 8, 'max_output_tokens': 64}

formatter: {'memory_template': '', 'prompt_template': '', 'bot_template': '{bot_name}: {message}</s>\n', 'user_template': 'You: {message}\n', 'response_template': '{bot_name}:', 'truncate_by_message': True}

Resubmit model

Shutdown handler not registered because Python interpreter is not running in the main thread
run pipeline %s
run pipeline stage %s
Running pipeline stage MKMLizer
Starting job with name chaiml-oscar-piastri250-77993-v1-mkmlizer
Waiting for job on chaiml-oscar-piastri250-77993-v1-mkmlizer to finish
Failed to get response for submission b2: no entry with id "b2" found on database!
Failed to get response for submission b1: no entry with id "b1" found on database!
Failed to get response for submission b1: no entry with id "b1" found on database!
Failed to get response for submission b2: no entry with id "b2" found on database!
chaiml-oscar-piastri250-77993-v1-mkmlizer: ╔═════════════════════════════════════════════════════════════════════╗
chaiml-oscar-piastri250-77993-v1-mkmlizer: ║                                                                     ║
chaiml-oscar-piastri250-77993-v1-mkmlizer: ║   ██████   ██████  █████   ████  ████                               ║
chaiml-oscar-piastri250-77993-v1-mkmlizer: ║  ░░██████ ██████  ░░███   ███░  ░░███                               ║
chaiml-oscar-piastri250-77993-v1-mkmlizer: ║   ░███░█████░███   ░███  ███     ░███                               ║
chaiml-oscar-piastri250-77993-v1-mkmlizer: ║   ░███░░███ ░███   ░███████      ░███                               ║
chaiml-oscar-piastri250-77993-v1-mkmlizer: ║   ░███ ░░░  ░███   ░███░░███     ░███                               ║
chaiml-oscar-piastri250-77993-v1-mkmlizer: ║   ░███      ░███   ░███ ░░███    ░███                               ║
chaiml-oscar-piastri250-77993-v1-mkmlizer: ║   █████     █████  █████ ░░████  █████                              ║
chaiml-oscar-piastri250-77993-v1-mkmlizer: ║  ░░░░░     ░░░░░  ░░░░░   ░░░░  ░░░░░                               ║
chaiml-oscar-piastri250-77993-v1-mkmlizer: ║                                                                     ║
chaiml-oscar-piastri250-77993-v1-mkmlizer: ║  Version: 0.29.15                                                   ║
chaiml-oscar-piastri250-77993-v1-mkmlizer: ║  Features: FLYWHEEL, CUDA                                           ║
chaiml-oscar-piastri250-77993-v1-mkmlizer: ║  Copyright 2023-2025 MK ONE TECHNOLOGIES Inc.                       ║
chaiml-oscar-piastri250-77993-v1-mkmlizer: ║  https://mk1.ai                                                     ║
chaiml-oscar-piastri250-77993-v1-mkmlizer: ║                                                                     ║
chaiml-oscar-piastri250-77993-v1-mkmlizer: ║  The license key for the current software has been verified as      ║
chaiml-oscar-piastri250-77993-v1-mkmlizer: ║  belonging to:                                                      ║
chaiml-oscar-piastri250-77993-v1-mkmlizer: ║                                                                     ║
chaiml-oscar-piastri250-77993-v1-mkmlizer: ║  Chai Research Corp.                                                ║
chaiml-oscar-piastri250-77993-v1-mkmlizer: ║  Account ID: 7997a29f-0ceb-4cc7-9adf-840c57b4ae6f                   ║
chaiml-oscar-piastri250-77993-v1-mkmlizer: ║  Expiration: 2028-03-31 23:59:59                                    ║
chaiml-oscar-piastri250-77993-v1-mkmlizer: ║                                                                     ║
chaiml-oscar-piastri250-77993-v1-mkmlizer: ╚═════════════════════════════════════════════════════════════════════╝
chaiml-oscar-piastri250-77993-v1-mkmlizer: Downloaded to shared memory in 69.128s
chaiml-oscar-piastri250-77993-v1-mkmlizer: Checking if ChaiML/oscar-piastri250716224452_sft already exists in ChaiML
chaiml-oscar-piastri250-77993-v1-mkmlizer: quantizing model to /dev/shm/model_cache, profile:s0, folder:/tmp/tmplq2mbola, device:0
chaiml-oscar-piastri250-77993-v1-mkmlizer: Saving flywheel model at /dev/shm/model_cache
Failed to get response for submission b1: no entry with id "b1" found on database!
Failed to get response for submission b2: no entry with id "b2" found on database!
chaiml-oscar-piastri250-77993-v1-mkmlizer: quantized model in 47.613s
chaiml-oscar-piastri250-77993-v1-mkmlizer: Processed model ChaiML/oscar-piastri250716224452_sft in 116.741s
chaiml-oscar-piastri250-77993-v1-mkmlizer: creating bucket guanaco-mkml-models
chaiml-oscar-piastri250-77993-v1-mkmlizer: Bucket 's3://guanaco-mkml-models/' created
chaiml-oscar-piastri250-77993-v1-mkmlizer: uploading /dev/shm/model_cache to s3://guanaco-mkml-models/chaiml-oscar-piastri250-77993-v1/nvidia
chaiml-oscar-piastri250-77993-v1-mkmlizer: cp /dev/shm/model_cache/config.json s3://guanaco-mkml-models/chaiml-oscar-piastri250-77993-v1/nvidia/config.json
chaiml-oscar-piastri250-77993-v1-mkmlizer: cp /dev/shm/model_cache/special_tokens_map.json s3://guanaco-mkml-models/chaiml-oscar-piastri250-77993-v1/nvidia/special_tokens_map.json
chaiml-oscar-piastri250-77993-v1-mkmlizer: cp /dev/shm/model_cache/tokenizer_config.json s3://guanaco-mkml-models/chaiml-oscar-piastri250-77993-v1/nvidia/tokenizer_config.json
chaiml-oscar-piastri250-77993-v1-mkmlizer: cp /dev/shm/model_cache/tokenizer.json s3://guanaco-mkml-models/chaiml-oscar-piastri250-77993-v1/nvidia/tokenizer.json
chaiml-oscar-piastri250-77993-v1-mkmlizer: cp /dev/shm/model_cache/flywheel_model.0.safetensors s3://guanaco-mkml-models/chaiml-oscar-piastri250-77993-v1/nvidia/flywheel_model.0.safetensors
chaiml-oscar-piastri250-77993-v1-mkmlizer: cp /dev/shm/model_cache/flywheel_model.1.safetensors s3://guanaco-mkml-models/chaiml-oscar-piastri250-77993-v1/nvidia/flywheel_model.1.safetensors
chaiml-oscar-piastri250-77993-v1-mkmlizer: 
Loading 0:   0%|          | 0/363 [00:00<?, ?it/s]
Loading 0:   1%|▏         | 5/363 [00:00<00:17, 20.89it/s]
Loading 0:   3%|▎         | 12/363 [00:00<00:09, 37.48it/s]
Loading 0:   5%|▍         | 17/363 [00:00<00:10, 34.40it/s]
Loading 0:   6%|▌         | 21/363 [00:00<00:09, 35.63it/s]
Loading 0:   7%|▋         | 25/363 [00:00<00:10, 31.85it/s]
Loading 0:   9%|▉         | 32/363 [00:00<00:08, 38.17it/s]
Loading 0:  10%|▉         | 36/363 [00:01<00:14, 22.78it/s]
Loading 0:  11%|█         | 40/363 [00:01<00:12, 25.38it/s]
Loading 0:  12%|█▏        | 44/363 [00:01<00:11, 26.66it/s]
Loading 0:  13%|█▎        | 48/363 [00:01<00:10, 28.91it/s]
Loading 0:  14%|█▍        | 52/363 [00:01<00:11, 27.95it/s]
Loading 0:  16%|█▌        | 57/363 [00:01<00:10, 30.52it/s]
Loading 0:  17%|█▋        | 61/363 [00:02<00:10, 29.05it/s]
Loading 0:  18%|█▊        | 65/363 [00:02<00:10, 29.30it/s]
Loading 0:  19%|█▉        | 70/363 [00:02<00:12, 23.70it/s]
Loading 0:  20%|██        | 73/363 [00:02<00:14, 19.92it/s]
Loading 0:  22%|██▏       | 79/363 [00:02<00:11, 25.11it/s]
Loading 0:  23%|██▎       | 82/363 [00:03<00:11, 24.62it/s]
Loading 0:  24%|██▎       | 86/363 [00:03<00:10, 25.72it/s]
Loading 0:  25%|██▍       | 89/363 [00:03<00:10, 25.97it/s]
Loading 0:  25%|██▌       | 92/363 [00:03<00:13, 20.58it/s]
Loading 0:  27%|██▋       | 99/363 [00:03<00:09, 27.22it/s]
Loading 0:  28%|██▊       | 102/363 [00:03<00:10, 24.05it/s]
Loading 0:  29%|██▉       | 107/363 [00:04<00:12, 21.27it/s]
Loading 0:  31%|███       | 111/363 [00:04<00:10, 23.65it/s]
Loading 0:  31%|███▏      | 114/363 [00:04<00:11, 21.53it/s]
Loading 0:  33%|███▎      | 120/363 [00:04<00:09, 26.77it/s]
Loading 0:  34%|███▍      | 123/363 [00:04<00:09, 24.54it/s]
Loading 0:  35%|███▍      | 127/363 [00:04<00:08, 27.44it/s]
Loading 0:  36%|███▌      | 130/363 [00:04<00:08, 27.21it/s]
Loading 0:  37%|███▋      | 133/363 [00:05<00:08, 25.77it/s]
Loading 0:  38%|███▊      | 138/363 [00:05<00:08, 28.01it/s]
Loading 0:  39%|███▉      | 141/363 [00:05<00:08, 25.01it/s]
Loading 0:  40%|████      | 147/363 [00:05<00:06, 32.89it/s]
Loading 0:  42%|████▏     | 151/363 [00:05<00:09, 23.49it/s]
Loading 0:  42%|████▏     | 154/363 [00:05<00:09, 21.70it/s]
Loading 0:  43%|████▎     | 157/363 [00:06<00:09, 22.73it/s]
Loading 0:  44%|████▍     | 160/363 [00:06<00:08, 22.88it/s]
Loading 0:  45%|████▌     | 165/363 [00:06<00:07, 25.79it/s]
Loading 0:  46%|████▋     | 168/363 [00:06<00:08, 23.62it/s]
Loading 0:  48%|████▊     | 174/363 [00:06<00:06, 28.91it/s]
Loading 0:  49%|████▉     | 178/363 [00:06<00:06, 27.02it/s]
Loading 0:  50%|█████     | 182/363 [00:06<00:06, 27.65it/s]
Loading 0:  52%|█████▏    | 187/363 [00:07<00:07, 24.52it/s]
Loading 0:  52%|█████▏    | 190/363 [00:07<00:07, 22.64it/s]
Loading 0:  53%|█████▎    | 193/363 [00:07<00:07, 23.46it/s]
Loading 0:  54%|█████▍    | 196/363 [00:07<00:06, 24.07it/s]
Loading 0:  55%|█████▌    | 200/363 [00:22<00:06, 24.07it/s]
Loading 0:  55%|█████▌    | 201/363 [00:22<03:03,  1.13s/it]
Loading 0:  56%|█████▌    | 203/363 [00:22<02:31,  1.06it/s]
Loading 0:  57%|█████▋    | 208/363 [00:22<01:31,  1.70it/s]
Loading 0:  58%|█████▊    | 211/363 [00:22<01:10,  2.17it/s]
Loading 0:  59%|█████▉    | 214/363 [00:22<00:52,  2.85it/s]
Loading 0:  60%|██████    | 218/363 [00:22<00:35,  4.07it/s]
Loading 0:  61%|██████    | 221/363 [00:23<00:27,  5.19it/s]
Loading 0:  62%|██████▏   | 224/363 [00:23<00:22,  6.20it/s]
Loading 0:  63%|██████▎   | 229/363 [00:23<00:14,  9.14it/s]
Loading 0:  64%|██████▍   | 232/363 [00:23<00:12, 10.82it/s]
Loading 0:  65%|██████▌   | 237/363 [00:23<00:08, 14.62it/s]
Loading 0:  66%|██████▌   | 240/363 [00:23<00:08, 15.35it/s]
Loading 0:  68%|██████▊   | 246/363 [00:24<00:05, 20.75it/s]
Loading 0:  69%|██████▊   | 249/363 [00:24<00:05, 20.45it/s]
Loading 0:  70%|███████   | 255/363 [00:24<00:04, 25.85it/s]
Loading 0:  71%|███████▏  | 259/363 [00:24<00:04, 25.62it/s]
Loading 0:  73%|███████▎  | 265/363 [00:24<00:03, 32.43it/s]
Loading 0:  74%|███████▍  | 269/363 [00:24<00:04, 23.07it/s]
Loading 0:  75%|███████▌  | 273/363 [00:25<00:03, 22.56it/s]
Loading 0:  76%|███████▌  | 276/363 [00:25<00:04, 21.40it/s]
Loading 0:  78%|███████▊  | 282/363 [00:25<00:03, 26.30it/s]
Loading 0:  79%|███████▊  | 285/363 [00:25<00:03, 24.77it/s]
Loading 0:  80%|████████  | 291/363 [00:25<00:02, 29.68it/s]
Loading 0:  81%|████████▏ | 295/363 [00:25<00:02, 28.26it/s]
Loading 0:  82%|████████▏ | 299/363 [00:25<00:02, 28.67it/s]
Loading 0:  84%|████████▎ | 304/363 [00:26<00:02, 25.55it/s]
Loading 0:  85%|████████▍ | 307/363 [00:26<00:02, 23.09it/s]
Loading 0:  85%|████████▌ | 310/363 [00:26<00:02, 23.97it/s]
Loading 0:  86%|████████▌ | 313/363 [00:26<00:02, 24.56it/s]
Loading 0:  88%|████████▊ | 318/363 [00:26<00:01, 27.64it/s]
Loading 0:  88%|████████▊ | 321/363 [00:26<00:01, 24.00it/s]
Loading 0:  90%|█████████ | 327/363 [00:27<00:01, 29.46it/s]
Loading 0:  91%|█████████ | 331/363 [00:27<00:01, 28.21it/s]
Loading 0:  92%|█████████▏| 335/363 [00:27<00:00, 28.56it/s]
Loading 0:  93%|█████████▎| 338/363 [00:27<00:00, 25.99it/s]
Loading 0:  94%|█████████▍| 341/363 [00:27<00:01, 15.85it/s]
Loading 0:  96%|█████████▌| 347/363 [00:28<00:00, 21.40it/s]
Loading 0:  96%|█████████▋| 350/363 [00:28<00:00, 22.03it/s]
Loading 0:  98%|█████████▊| 355/363 [00:28<00:00, 25.75it/s]
Loading 0:  99%|█████████▊| 358/363 [00:28<00:00, 23.94it/s]
                                                            
Job chaiml-oscar-piastri250-77993-v1-mkmlizer completed after 188.71s with status: succeeded
Stopping job with name chaiml-oscar-piastri250-77993-v1-mkmlizer
Pipeline stage MKMLizer completed in 189.32s
run pipeline stage %s
Running pipeline stage MKMLTemplater
Pipeline stage MKMLTemplater completed in 0.19s
run pipeline stage %s
Running pipeline stage MKMLDeployer
Creating inference service chaiml-oscar-piastri250-77993-v1
Waiting for inference service chaiml-oscar-piastri250-77993-v1 to be ready
Failed to get response for submission b1: no entry with id "b1" found on database!
Failed to get response for submission b2: no entry with id "b2" found on database!
Failed to get response for submission b1: no entry with id "b1" found on database!
Failed to get response for submission b2: no entry with id "b2" found on database!
Failed to get response for submission b2: no entry with id "b2" found on database!
Failed to get response for submission b1: no entry with id "b1" found on database!
Failed to get response for submission b1: no entry with id "b1" found on database!
Failed to get response for submission b2: no entry with id "b2" found on database!
Failed to get response for submission b2: no entry with id "b2" found on database!
Failed to get response for submission b1: no entry with id "b1" found on database!
Failed to get response for submission b2: no entry with id "b2" found on database!
Failed to get response for submission b1: no entry with id "b1" found on database!
Inference service chaiml-oscar-piastri250-77993-v1 ready after 281.18416023254395s
Pipeline stage MKMLDeployer completed in 282.04s
run pipeline stage %s
Running pipeline stage StressChecker
Failed to get response for submission b1: no entry with id "b1" found on database!
Failed to get response for submission b2: no entry with id "b2" found on database!
Received healthy response to inference request in 2.7216460704803467s
Received healthy response to inference request in 1.8752338886260986s
Received healthy response to inference request in 2.0842397212982178s
Received healthy response to inference request in 1.9959213733673096s
Received healthy response to inference request in 1.7734043598175049s
5 requests
0 failed requests
5th percentile: 1.7937702655792236
10th percentile: 1.8141361713409423
20th percentile: 1.85486798286438
30th percentile: 1.8993713855743408
40th percentile: 1.9476463794708252
50th percentile: 1.9959213733673096
60th percentile: 2.0312487125396728
70th percentile: 2.066576051712036
80th percentile: 2.211720991134644
90th percentile: 2.466683530807495
95th percentile: 2.5941648006439206
99th percentile: 2.6961498165130613
mean time: 2.0900890827178955
Pipeline stage StressChecker completed in 12.16s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyTriggerPipeline
run_pipeline:run_in_cloud %s
starting trigger_guanaco_pipeline args=%s
triggered trigger_guanaco_pipeline args=%s
Pipeline stage OfflineFamilyFriendlyTriggerPipeline completed in 0.74s
run pipeline stage %s
Running pipeline stage TriggerMKMLProfilingPipeline
run_pipeline:run_in_cloud %s
starting trigger_guanaco_pipeline args=%s
triggered trigger_guanaco_pipeline args=%s
Pipeline stage TriggerMKMLProfilingPipeline completed in 0.71s
Shutdown handler de-registered
chaiml-oscar-piastri250_77993_v1 status is now deployed due to DeploymentManager action
Shutdown handler registered
run pipeline %s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyScorer
Evaluating %s Family Friendly Score with %s threads
Pipeline stage OfflineFamilyFriendlyScorer completed in 4395.82s
Shutdown handler de-registered
chaiml-oscar-piastri250_77993_v1 status is now inactive due to auto deactivation removed underperforming models
chaiml-oscar-piastri250_77993_v1 status is now torndown due to DeploymentManager action
chaiml-oscar-piastri250_77993_v1 status is now torndown due to DeploymentManager action
chaiml-oscar-piastri250_77993_v1 status is now torndown due to DeploymentManager action