developer_uid: ChaiHarshitSheoran
submission_id: chaiml-mn24b-v2preferwi_52388_v1
model_name: chaiml-mn24b-v2preferwi_52388_v1
model_group: ChaiML/mn24b_v2preferwin
status: torndown
timestamp: 2025-09-25T10:23:45+00:00
num_battles: 5743
num_wins: 2831
celo_rating: 1283.5
family_friendly_score: 0.5362
family_friendly_standard_error: 0.007052511042174979
submission_type: basic
model_repo: ChaiML/mn24b_v2preferwin2_qwen_nonl_bo16_tune4
model_architecture: MistralForCausalLM
model_num_parameters: 24096691200.0
best_of: 8
max_input_tokens: 1024
max_output_tokens: 64
reward_model: default
latencies: [{'batch_size': 1, 'throughput': 0.43040387674211167, 'latency_mean': 2.323270683288574, 'latency_p50': 2.3179389238357544, 'latency_p90': 2.5338922500610352}, {'batch_size': 2, 'throughput': 0.6444436699935424, 'latency_mean': 3.0987834346294405, 'latency_p50': 3.1226407289505005, 'latency_p90': 3.464936375617981}, {'batch_size': 3, 'throughput': 0.7778115058442169, 'latency_mean': 3.837868688106537, 'latency_p50': 3.832575798034668, 'latency_p90': 4.389339137077331}, {'batch_size': 4, 'throughput': 0.8593786764479711, 'latency_mean': 4.633842262029648, 'latency_p50': 4.605474352836609, 'latency_p90': 5.222850942611695}, {'batch_size': 5, 'throughput': 0.8977723505134422, 'latency_mean': 5.5411804890632625, 'latency_p50': 5.572018265724182, 'latency_p90': 6.3249754190444945}]
gpu_counts: {'NVIDIA L40S': 1}
display_name: chaiml-mn24b-v2preferwi_52388_v1
is_internal_developer: False
language_model: ChaiML/mn24b_v2preferwin2_qwen_nonl_bo16_tune4
model_size: 24B
ranking_group: single
throughput_3p7s: 0.76
us_pacific_date: 2025-09-25
win_ratio: 0.49294793661849207
generation_params: {'temperature': 0.65, 'top_p': 1.0, 'min_p': 0.0, 'top_k': 40, 'presence_penalty': 0.0, 'frequency_penalty': 0.0, 'stopping_words': ['\n'], 'max_input_tokens': 1024, 'best_of': 8, 'max_output_tokens': 64}
formatter: {'memory_template': "{bot_name}'s Persona: {memory}\n####\n", 'prompt_template': '\n<START>\n', 'bot_template': '{bot_name}: {message}\n', 'user_template': '{user_name}: {message}\n', 'response_template': '{bot_name}:', 'truncate_by_message': True}
Resubmit model
Shutdown handler not registered because Python interpreter is not running in the main thread
run pipeline %s
run pipeline stage %s
Running pipeline stage MKMLizer
Starting job with name chaiml-mn24b-v2preferwi-52388-v1-mkmlizer
Shutdown handler not registered because Python interpreter is not running in the main thread
run pipeline %s
run pipeline stage %s
Running pipeline stage MKMLizer
Starting job with name chaiml-mn24b-v2preferwi-52388-v1-mkmlizer
Stopping job with name chaiml-mn24b-v2preferwi-52388-v1-mkmlizer
%s, retrying in %s seconds...
Starting job with name chaiml-mn24b-v2preferwi-52388-v1-mkmlizer
Waiting for job on chaiml-mn24b-v2preferwi-52388-v1-mkmlizer to finish
chaiml-mn24b-v2preferwi-52388-v1-mkmlizer: ╔═════════════════════════════════════════════════════════════════════╗
chaiml-mn24b-v2preferwi-52388-v1-mkmlizer: ║                                                                     ║
chaiml-mn24b-v2preferwi-52388-v1-mkmlizer: ║   ██████   ██████  █████   ████  ████                               ║
chaiml-mn24b-v2preferwi-52388-v1-mkmlizer: ║  ░░██████ ██████  ░░███   ███░  ░░███                               ║
chaiml-mn24b-v2preferwi-52388-v1-mkmlizer: ║   ░███░█████░███   ░███  ███     ░███                               ║
chaiml-mn24b-v2preferwi-52388-v1-mkmlizer: ║   ░███░░███ ░███   ░███████      ░███                               ║
chaiml-mn24b-v2preferwi-52388-v1-mkmlizer: ║   ░███ ░░░  ░███   ░███░░███     ░███                               ║
chaiml-mn24b-v2preferwi-52388-v1-mkmlizer: ║   ░███      ░███   ░███ ░░███    ░███                               ║
chaiml-mn24b-v2preferwi-52388-v1-mkmlizer: ║   █████     █████  █████ ░░████  █████                              ║
chaiml-mn24b-v2preferwi-52388-v1-mkmlizer: ║  ░░░░░     ░░░░░  ░░░░░   ░░░░  ░░░░░                               ║
chaiml-mn24b-v2preferwi-52388-v1-mkmlizer: ║                                                                     ║
chaiml-mn24b-v2preferwi-52388-v1-mkmlizer: ║  Version: 0.30.2                                                    ║
chaiml-mn24b-v2preferwi-52388-v1-mkmlizer: ║  Features: FLYWHEEL, CUDA                                           ║
chaiml-mn24b-v2preferwi-52388-v1-mkmlizer: ║  Copyright 2023-2025 MK ONE TECHNOLOGIES Inc.                       ║
chaiml-mn24b-v2preferwi-52388-v1-mkmlizer: ║  https://mk1.ai                                                     ║
chaiml-mn24b-v2preferwi-52388-v1-mkmlizer: ║                                                                     ║
chaiml-mn24b-v2preferwi-52388-v1-mkmlizer: ║  The license key for the current software has been verified as      ║
chaiml-mn24b-v2preferwi-52388-v1-mkmlizer: ║  belonging to:                                                      ║
chaiml-mn24b-v2preferwi-52388-v1-mkmlizer: ║                                                                     ║
chaiml-mn24b-v2preferwi-52388-v1-mkmlizer: ║  Chai Research Corp.                                                ║
chaiml-mn24b-v2preferwi-52388-v1-mkmlizer: ║  Account ID: 7997a29f-0ceb-4cc7-9adf-840c57b4ae6f                   ║
chaiml-mn24b-v2preferwi-52388-v1-mkmlizer: ║  Expiration: 2028-03-31 23:59:59                                    ║
chaiml-mn24b-v2preferwi-52388-v1-mkmlizer: ║                                                                     ║
chaiml-mn24b-v2preferwi-52388-v1-mkmlizer: ╚═════════════════════════════════════════════════════════════════════╝
chaiml-mn24b-v2preferwi-52388-v1-mkmlizer: Downloaded to shared memory in 60.214s
chaiml-mn24b-v2preferwi-52388-v1-mkmlizer: Checking if ChaiML/mn24b_v2preferwin2_qwen_nonl_bo16_tune4 already exists in ChaiML
chaiml-mn24b-v2preferwi-52388-v1-mkmlizer: quantizing model to /dev/shm/model_cache, profile:q4, folder:/tmp/tmprc5qfn9n, device:0
chaiml-mn24b-v2preferwi-52388-v1-mkmlizer: Saving flywheel model at /dev/shm/model_cache
chaiml-mn24b-v2preferwi-52388-v1-mkmlizer: quantized model in 283.848s
chaiml-mn24b-v2preferwi-52388-v1-mkmlizer: Processed model ChaiML/mn24b_v2preferwin2_qwen_nonl_bo16_tune4 in 344.063s
chaiml-mn24b-v2preferwi-52388-v1-mkmlizer: creating bucket guanaco-mkml-models
chaiml-mn24b-v2preferwi-52388-v1-mkmlizer: Bucket 's3://guanaco-mkml-models/' created
chaiml-mn24b-v2preferwi-52388-v1-mkmlizer: uploading /dev/shm/model_cache to s3://guanaco-mkml-models/chaiml-mn24b-v2preferwi-52388-v1/nvidia
chaiml-mn24b-v2preferwi-52388-v1-mkmlizer: cp /dev/shm/model_cache/config.json s3://guanaco-mkml-models/chaiml-mn24b-v2preferwi-52388-v1/nvidia/config.json
chaiml-mn24b-v2preferwi-52388-v1-mkmlizer: cp /dev/shm/model_cache/special_tokens_map.json s3://guanaco-mkml-models/chaiml-mn24b-v2preferwi-52388-v1/nvidia/special_tokens_map.json
chaiml-mn24b-v2preferwi-52388-v1-mkmlizer: cp /dev/shm/model_cache/tokenizer_config.json s3://guanaco-mkml-models/chaiml-mn24b-v2preferwi-52388-v1/nvidia/tokenizer_config.json
chaiml-mn24b-v2preferwi-52388-v1-mkmlizer: cp /dev/shm/model_cache/tokenizer.json s3://guanaco-mkml-models/chaiml-mn24b-v2preferwi-52388-v1/nvidia/tokenizer.json
chaiml-mn24b-v2preferwi-52388-v1-mkmlizer: cp /dev/shm/model_cache/flywheel_model.1.safetensors s3://guanaco-mkml-models/chaiml-mn24b-v2preferwi-52388-v1/nvidia/flywheel_model.1.safetensors
Stopping job with name chaiml-mn24b-v2preferwi-52388-v1-mkmlizer
%s, retrying in %s seconds...
Stopping job with name chaiml-mn24b-v2preferwi-52388-v1-mkmlizer
%s, retrying in %s seconds...
Stopping job with name chaiml-mn24b-v2preferwi-52388-v1-mkmlizer
%s, retrying in %s seconds...
Starting job with name chaiml-mn24b-v2preferwi-52388-v1-mkmlizer
Waiting for job on chaiml-mn24b-v2preferwi-52388-v1-mkmlizer to finish
chaiml-mn24b-v2preferwi-52388-v1-mkmlizer: ╔═════════════════════════════════════════════════════════════════════╗
chaiml-mn24b-v2preferwi-52388-v1-mkmlizer: ║                                                                     ║
chaiml-mn24b-v2preferwi-52388-v1-mkmlizer: ║   ██████   ██████  █████   ████  ████                               ║
chaiml-mn24b-v2preferwi-52388-v1-mkmlizer: ║  ░░██████ ██████  ░░███   ███░  ░░███                               ║
chaiml-mn24b-v2preferwi-52388-v1-mkmlizer: ║   ░███░█████░███   ░███  ███     ░███                               ║
chaiml-mn24b-v2preferwi-52388-v1-mkmlizer: ║   ░███░░███ ░███   ░███████      ░███                               ║
chaiml-mn24b-v2preferwi-52388-v1-mkmlizer: ║   ░███ ░░░  ░███   ░███░░███     ░███                               ║
chaiml-mn24b-v2preferwi-52388-v1-mkmlizer: ║   ░███      ░███   ░███ ░░███    ░███                               ║
chaiml-mn24b-v2preferwi-52388-v1-mkmlizer: ║   █████     █████  █████ ░░████  █████                              ║
chaiml-mn24b-v2preferwi-52388-v1-mkmlizer: ║  ░░░░░     ░░░░░  ░░░░░   ░░░░  ░░░░░                               ║
chaiml-mn24b-v2preferwi-52388-v1-mkmlizer: ║                                                                     ║
chaiml-mn24b-v2preferwi-52388-v1-mkmlizer: ║  Version: 0.30.2                                                    ║
chaiml-mn24b-v2preferwi-52388-v1-mkmlizer: ║  Features: FLYWHEEL, CUDA                                           ║
chaiml-mn24b-v2preferwi-52388-v1-mkmlizer: ║  Copyright 2023-2025 MK ONE TECHNOLOGIES Inc.                       ║
chaiml-mn24b-v2preferwi-52388-v1-mkmlizer: ║  https://mk1.ai                                                     ║
chaiml-mn24b-v2preferwi-52388-v1-mkmlizer: ║                                                                     ║
chaiml-mn24b-v2preferwi-52388-v1-mkmlizer: ║  The license key for the current software has been verified as      ║
chaiml-mn24b-v2preferwi-52388-v1-mkmlizer: ║  belonging to:                                                      ║
chaiml-mn24b-v2preferwi-52388-v1-mkmlizer: ║                                                                     ║
chaiml-mn24b-v2preferwi-52388-v1-mkmlizer: ║  Chai Research Corp.                                                ║
chaiml-mn24b-v2preferwi-52388-v1-mkmlizer: ║  Account ID: 7997a29f-0ceb-4cc7-9adf-840c57b4ae6f                   ║
chaiml-mn24b-v2preferwi-52388-v1-mkmlizer: ║  Expiration: 2028-03-31 23:59:59                                    ║
chaiml-mn24b-v2preferwi-52388-v1-mkmlizer: ║                                                                     ║
chaiml-mn24b-v2preferwi-52388-v1-mkmlizer: ╚═════════════════════════════════════════════════════════════════════╝
chaiml-mn24b-v2preferwi-52388-v1-mkmlizer: Downloaded to shared memory in 50.929s
chaiml-mn24b-v2preferwi-52388-v1-mkmlizer: Checking if ChaiML/mn24b_v2preferwin2_qwen_nonl_bo16_tune4 already exists in ChaiML
chaiml-mn24b-v2preferwi-52388-v1-mkmlizer: quantizing model to /dev/shm/model_cache, profile:q4, folder:/tmp/tmpsepmn25d, device:0
chaiml-mn24b-v2preferwi-52388-v1-mkmlizer: Saving flywheel model at /dev/shm/model_cache
Shutdown handler registered
run pipeline %s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyScorer
Evaluating %s Family Friendly Score with %s threads
%s, retrying in %s seconds...
Shutdown handler registered
run pipeline %s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyScorer
Evaluating %s Family Friendly Score with %s threads
Pipeline stage OfflineFamilyFriendlyScorer completed in 2592.58s
Shutdown handler de-registered
chaiml-mn24b-v2preferwi_52388_v1 status is now inactive due to auto deactivation removed underperforming models
chaiml-mn24b-v2preferwi_52388_v1 status is now torndown due to DeploymentManager action