developer_uid: ChaiHarshitSheoran
submission_id: fizzarolli-mistral-smal_69249_v1
model_name: fizzarolli-mistral-smal_69249_v1
model_group: Fizzarolli/Mistral-Small
status: torndown
timestamp: 2025-09-12T23:43:58+00:00
num_battles: 6354
num_wins: 3183
celo_rating: 1243.5
family_friendly_score: 0.5844
family_friendly_standard_error: 0.006969600275482089
submission_type: basic
model_repo: Fizzarolli/Mistral-Small-3.2-24B-Instruct-2506-Text-Only
model_architecture: MistralForCausalLM
model_num_parameters: 24096691200.0
best_of: 8
max_input_tokens: 1024
max_output_tokens: 64
reward_model: default
latencies: [{'batch_size': 1, 'throughput': 0.549579161760067, 'latency_mean': 1.8194754207134247, 'latency_p50': 1.829514741897583, 'latency_p90': 2.0062103986740114}, {'batch_size': 3, 'throughput': 1.1113497354571902, 'latency_mean': 2.6862937426567077, 'latency_p50': 2.7125202417373657, 'latency_p90': 3.0125699043273926}, {'batch_size': 5, 'throughput': 1.4379903689090097, 'latency_mean': 3.4588872349262236, 'latency_p50': 3.456329107284546, 'latency_p90': 3.855745244026184}, {'batch_size': 6, 'throughput': 1.5508606842447534, 'latency_mean': 3.8400433921813963, 'latency_p50': 3.8567155599594116, 'latency_p90': 4.332662987709045}, {'batch_size': 8, 'throughput': 1.7178768230002235, 'latency_mean': 4.633077621459961, 'latency_p50': 4.618134379386902, 'latency_p90': 5.233885192871094}, {'batch_size': 10, 'throughput': 1.7986913376198337, 'latency_mean': 5.525531636476517, 'latency_p50': 5.519872307777405, 'latency_p90': 6.390372920036316}]
gpu_counts: {'NVIDIA L40S': 1}
display_name: fizzarolli-mistral-smal_69249_v1
is_internal_developer: False
language_model: Fizzarolli/Mistral-Small-3.2-24B-Instruct-2506-Text-Only
model_size: 24B
ranking_group: single
throughput_3p7s: 1.52
us_pacific_date: 2025-09-12
win_ratio: 0.5009442870632672
generation_params: {'temperature': 1.0, 'top_p': 1.0, 'min_p': 0.0, 'top_k': 40, 'presence_penalty': 0.0, 'frequency_penalty': 0.0, 'stopping_words': ['\n'], 'max_input_tokens': 1024, 'best_of': 8, 'max_output_tokens': 64}
formatter: {'memory_template': "{bot_name}'s Persona: {memory}\n####\n", 'prompt_template': '{prompt}\n<START>\n', 'bot_template': '{bot_name}: {message}\n', 'user_template': '{user_name}: {message}\n', 'response_template': '{bot_name}:', 'truncate_by_message': False}
Resubmit model
Shutdown handler not registered because Python interpreter is not running in the main thread
run pipeline %s
run pipeline stage %s
Running pipeline stage MKMLizer
Starting job with name fizzarolli-mistral-smal-69249-v1-mkmlizer
Waiting for job on fizzarolli-mistral-smal-69249-v1-mkmlizer to finish
Stopping job with name fizzarolli-mistral-smal-69249-v1-mkmlizer
%s, retrying in %s seconds...
Starting job with name fizzarolli-mistral-smal-69249-v1-mkmlizer
Waiting for job on fizzarolli-mistral-smal-69249-v1-mkmlizer to finish
fizzarolli-mistral-smal-69249-v1-mkmlizer: ╔═════════════════════════════════════════════════════════════════════╗
fizzarolli-mistral-smal-69249-v1-mkmlizer: ║                                                                     ║
fizzarolli-mistral-smal-69249-v1-mkmlizer: ║   ██████   ██████  █████   ████  ████                               ║
fizzarolli-mistral-smal-69249-v1-mkmlizer: ║  ░░██████ ██████  ░░███   ███░  ░░███                               ║
fizzarolli-mistral-smal-69249-v1-mkmlizer: ║   ░███░█████░███   ░███  ███     ░███                               ║
fizzarolli-mistral-smal-69249-v1-mkmlizer: ║   ░███░░███ ░███   ░███████      ░███                               ║
fizzarolli-mistral-smal-69249-v1-mkmlizer: ║   ░███ ░░░  ░███   ░███░░███     ░███                               ║
fizzarolli-mistral-smal-69249-v1-mkmlizer: ║   ░███      ░███   ░███ ░░███    ░███                               ║
fizzarolli-mistral-smal-69249-v1-mkmlizer: ║   █████     █████  █████ ░░████  █████                              ║
fizzarolli-mistral-smal-69249-v1-mkmlizer: ║  ░░░░░     ░░░░░  ░░░░░   ░░░░  ░░░░░                               ║
fizzarolli-mistral-smal-69249-v1-mkmlizer: ║                                                                     ║
fizzarolli-mistral-smal-69249-v1-mkmlizer: ║  Version: 0.30.2                                                    ║
fizzarolli-mistral-smal-69249-v1-mkmlizer: ║  Features: FLYWHEEL, CUDA                                           ║
fizzarolli-mistral-smal-69249-v1-mkmlizer: ║  Copyright 2023-2025 MK ONE TECHNOLOGIES Inc.                       ║
fizzarolli-mistral-smal-69249-v1-mkmlizer: ║  https://mk1.ai                                                     ║
fizzarolli-mistral-smal-69249-v1-mkmlizer: ║                                                                     ║
fizzarolli-mistral-smal-69249-v1-mkmlizer: ║  The license key for the current software has been verified as      ║
fizzarolli-mistral-smal-69249-v1-mkmlizer: ║  belonging to:                                                      ║
fizzarolli-mistral-smal-69249-v1-mkmlizer: ║                                                                     ║
fizzarolli-mistral-smal-69249-v1-mkmlizer: ║  Chai Research Corp.                                                ║
fizzarolli-mistral-smal-69249-v1-mkmlizer: ║  Account ID: 7997a29f-0ceb-4cc7-9adf-840c57b4ae6f                   ║
fizzarolli-mistral-smal-69249-v1-mkmlizer: ║  Expiration: 2028-03-31 23:59:59                                    ║
fizzarolli-mistral-smal-69249-v1-mkmlizer: ║                                                                     ║
fizzarolli-mistral-smal-69249-v1-mkmlizer: ╚═════════════════════════════════════════════════════════════════════╝
fizzarolli-mistral-smal-69249-v1-mkmlizer: Downloaded to shared memory in 45.103s
fizzarolli-mistral-smal-69249-v1-mkmlizer: Checking if Fizzarolli/Mistral-Small-3.2-24B-Instruct-2506-Text-Only already exists in ChaiML
fizzarolli-mistral-smal-69249-v1-mkmlizer: Creating repo ChaiML/Mistral-Small-3.2-24B-Instruct-2506-Text-Only and uploading /tmp/tmpob201k4o to it
fizzarolli-mistral-smal-69249-v1-mkmlizer: 
  0%|          | 0/12 [00:00<?, ?it/s]
  8%|▊         | 1/12 [00:05<01:04,  5.83s/it]
 17%|█▋        | 2/12 [00:13<01:08,  6.88s/it]
 25%|██▌       | 3/12 [00:16<00:46,  5.18s/it]
 33%|███▎      | 4/12 [00:23<00:46,  5.81s/it]
 42%|████▏     | 5/12 [00:28<00:39,  5.62s/it]
 50%|█████     | 6/12 [00:31<00:28,  4.74s/it]
 58%|█████▊    | 7/12 [00:36<00:23,  4.63s/it]
 67%|██████▋   | 8/12 [00:40<00:18,  4.63s/it]
 75%|███████▌  | 9/12 [00:45<00:14,  4.69s/it]
 83%|████████▎ | 10/12 [00:49<00:08,  4.41s/it]
 92%|█████████▏| 11/12 [00:49<00:03,  3.20s/it]
100%|██████████| 12/12 [00:50<00:00,  2.35s/it]
100%|██████████| 12/12 [00:50<00:00,  4.18s/it]
fizzarolli-mistral-smal-69249-v1-mkmlizer: quantizing model to /dev/shm/model_cache, profile:q4, folder:/tmp/tmpob201k4o, device:0
fizzarolli-mistral-smal-69249-v1-mkmlizer: Saving flywheel model at /dev/shm/model_cache
Failed to get response for submission mistralai-mistral-nem_93303_v569: ('http://mistralai-mistral-nem-93303-v569-predictor.tenant-chaiml-guanaco.k2.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', '')
Failed to get response for submission mistralai-mistral-nem_93303_v569: ('http://mistralai-mistral-nem-93303-v569-predictor.tenant-chaiml-guanaco.k2.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', '')
fizzarolli-mistral-smal-69249-v1-mkmlizer: quantized model in 288.992s
fizzarolli-mistral-smal-69249-v1-mkmlizer: Processed model Fizzarolli/Mistral-Small-3.2-24B-Instruct-2506-Text-Only in 418.757s
fizzarolli-mistral-smal-69249-v1-mkmlizer: creating bucket guanaco-mkml-models
fizzarolli-mistral-smal-69249-v1-mkmlizer: Bucket 's3://guanaco-mkml-models/' created
fizzarolli-mistral-smal-69249-v1-mkmlizer: uploading /dev/shm/model_cache to s3://guanaco-mkml-models/fizzarolli-mistral-smal-69249-v1/nvidia
fizzarolli-mistral-smal-69249-v1-mkmlizer: cp /dev/shm/model_cache/config.json s3://guanaco-mkml-models/fizzarolli-mistral-smal-69249-v1/nvidia/config.json
fizzarolli-mistral-smal-69249-v1-mkmlizer: cp /dev/shm/model_cache/special_tokens_map.json s3://guanaco-mkml-models/fizzarolli-mistral-smal-69249-v1/nvidia/special_tokens_map.json
fizzarolli-mistral-smal-69249-v1-mkmlizer: cp /dev/shm/model_cache/tokenizer_config.json s3://guanaco-mkml-models/fizzarolli-mistral-smal-69249-v1/nvidia/tokenizer_config.json
fizzarolli-mistral-smal-69249-v1-mkmlizer: cp /dev/shm/model_cache/tokenizer.json s3://guanaco-mkml-models/fizzarolli-mistral-smal-69249-v1/nvidia/tokenizer.json
fizzarolli-mistral-smal-69249-v1-mkmlizer: cp /dev/shm/model_cache/flywheel_model.1.safetensors s3://guanaco-mkml-models/fizzarolli-mistral-smal-69249-v1/nvidia/flywheel_model.1.safetensors
fizzarolli-mistral-smal-69249-v1-mkmlizer: cp /dev/shm/model_cache/flywheel_model.0.safetensors s3://guanaco-mkml-models/fizzarolli-mistral-smal-69249-v1/nvidia/flywheel_model.0.safetensors
Job fizzarolli-mistral-smal-69249-v1-mkmlizer completed after 488.79s with status: succeeded
Stopping job with name fizzarolli-mistral-smal-69249-v1-mkmlizer
Pipeline stage MKMLizer completed in 489.72s
run pipeline stage %s
Running pipeline stage MKMLTemplater
Pipeline stage MKMLTemplater completed in 0.17s
run pipeline stage %s
Running pipeline stage MKMLDeployer
Creating inference service fizzarolli-mistral-smal-69249-v1
Waiting for inference service fizzarolli-mistral-smal-69249-v1 to be ready
Inference service fizzarolli-mistral-smal-69249-v1 ready after 60.23374152183533s
Pipeline stage MKMLDeployer completed in 60.78s
run pipeline stage %s
Running pipeline stage StressChecker
Received healthy response to inference request in 1.9464709758758545s
Received healthy response to inference request in 1.7895288467407227s
Received healthy response to inference request in 1.685727834701538s
Received healthy response to inference request in 1.777249813079834s
Received healthy response to inference request in 1.8743507862091064s
5 requests
0 failed requests
5th percentile: 1.7040322303771973
10th percentile: 1.7223366260528565
20th percentile: 1.7589454174041748
30th percentile: 1.7797056198120118
40th percentile: 1.7846172332763672
50th percentile: 1.7895288467407227
60th percentile: 1.823457622528076
70th percentile: 1.8573863983154297
80th percentile: 1.888774824142456
90th percentile: 1.9176229000091554
95th percentile: 1.932046937942505
99th percentile: 1.9435861682891846
mean time: 1.8146656513214112
Pipeline stage StressChecker completed in 10.70s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyTriggerPipeline
run_pipeline:run_in_cloud %s
starting trigger_guanaco_pipeline args=%s
triggered trigger_guanaco_pipeline args=%s
Pipeline stage OfflineFamilyFriendlyTriggerPipeline completed in 0.64s
run pipeline stage %s
Running pipeline stage TriggerMKMLProfilingPipeline
run_pipeline:run_in_cloud %s
starting trigger_guanaco_pipeline args=%s
triggered trigger_guanaco_pipeline args=%s
Pipeline stage TriggerMKMLProfilingPipeline completed in 0.70s
Shutdown handler de-registered
fizzarolli-mistral-smal_69249_v1 status is now deployed due to DeploymentManager action
Shutdown handler registered
run pipeline %s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyScorer
Evaluating %s Family Friendly Score with %s threads
Pipeline stage OfflineFamilyFriendlyScorer completed in 2484.22s
Shutdown handler de-registered
fizzarolli-mistral-smal_69249_v1 status is now inactive due to auto deactivation removed underperforming models
fizzarolli-mistral-smal_69249_v1 status is now torndown due to DeploymentManager action