developer_uid: rirv938
submission_id: rirv938-mistral-24b-dpo_73307_v1
model_name: rirv938-mistral-24b-dpo_73307_v1
model_group: rirv938/mistral_24b_dpo_
status: torndown
timestamp: 2025-03-25T18:22:04+00:00
num_battles: 8818
num_wins: 4531
celo_rating: 1280.98
family_friendly_score: 0.49260000000000004
family_friendly_standard_error: 0.007070293346106652
submission_type: basic
model_repo: rirv938/mistral_24b_dpo_kl_10k_beta2_624_v3
model_architecture: MistralForCausalLM
model_num_parameters: 24096691200.0
best_of: 8
max_input_tokens: 768
max_output_tokens: 64
latencies: [{'batch_size': 1, 'throughput': 0.3864889233235328, 'latency_mean': 2.587331488132477, 'latency_p50': 2.5817214250564575, 'latency_p90': 2.8667894124984743}, {'batch_size': 3, 'throughput': 0.795021540171908, 'latency_mean': 3.758441923856735, 'latency_p50': 3.7525914907455444, 'latency_p90': 4.141624546051025}, {'batch_size': 5, 'throughput': 1.047337502456595, 'latency_mean': 4.748900637626648, 'latency_p50': 4.707342863082886, 'latency_p90': 5.342441821098328}, {'batch_size': 6, 'throughput': 1.132957551114224, 'latency_mean': 5.2738969302177425, 'latency_p50': 5.238316893577576, 'latency_p90': 5.917677283287048}, {'batch_size': 10, 'throughput': 1.2851910257086527, 'latency_mean': 7.735967149734497, 'latency_p50': 7.673913836479187, 'latency_p90': 8.810107541084289}]
gpu_counts: {'NVIDIA RTX A6000': 1}
display_name: rirv938-mistral-24b-dpo_73307_v1
is_internal_developer: True
language_model: rirv938/mistral_24b_dpo_kl_10k_beta2_624_v3
model_size: 24B
ranking_group: single
throughput_3p7s: 0.78
us_pacific_date: 2025-03-25
win_ratio: 0.5138353368110683
generation_params: {'temperature': 0.9, 'top_p': 0.9, 'min_p': 0.6, 'top_k': 80, 'presence_penalty': 0.5, 'frequency_penalty': 0.5, 'stopping_words': ['\n', '</s>', '###', 'You:'], 'max_input_tokens': 768, 'best_of': 8, 'max_output_tokens': 64}
formatter: {'memory_template': '[INST]', 'prompt_template': '', 'bot_template': '{bot_name}: {message}\n', 'user_template': '{user_name}: {message}\n', 'response_template': '[/INST]{bot_name}:', 'truncate_by_message': False}
Resubmit model
Shutdown handler not registered because Python interpreter is not running in the main thread
run pipeline %s
run pipeline stage %s
Running pipeline stage MKMLizer
Starting job with name rirv938-mistral-24b-dpo-73307-v1-mkmlizer
Waiting for job on rirv938-mistral-24b-dpo-73307-v1-mkmlizer to finish
rirv938-mistral-24b-dpo-73307-v1-mkmlizer: ╔═════════════════════════════════════════════════════════════════════╗
rirv938-mistral-24b-dpo-73307-v1-mkmlizer: ║     _____            __           __                                ║
rirv938-mistral-24b-dpo-73307-v1-mkmlizer: ║    / _/ /_ ___    __/ /  ___ ___ / /                                ║
rirv938-mistral-24b-dpo-73307-v1-mkmlizer: ║   / _/ / // / |/|/ / _ \/ -_) -_) /                                 ║
rirv938-mistral-24b-dpo-73307-v1-mkmlizer: ║  /_//_/\_, /|__,__/_//_/\__/\__/_/                                  ║
rirv938-mistral-24b-dpo-73307-v1-mkmlizer: ║       /___/                                                         ║
rirv938-mistral-24b-dpo-73307-v1-mkmlizer: ║                                                                     ║
rirv938-mistral-24b-dpo-73307-v1-mkmlizer: ║  Version: 0.12.8                                                    ║
rirv938-mistral-24b-dpo-73307-v1-mkmlizer: ║  Copyright 2023 MK ONE TECHNOLOGIES Inc.                            ║
rirv938-mistral-24b-dpo-73307-v1-mkmlizer: ║  https://mk1.ai                                                     ║
rirv938-mistral-24b-dpo-73307-v1-mkmlizer: ║                                                                     ║
rirv938-mistral-24b-dpo-73307-v1-mkmlizer: ║  The license key for the current software has been verified as      ║
rirv938-mistral-24b-dpo-73307-v1-mkmlizer: ║  belonging to:                                                      ║
rirv938-mistral-24b-dpo-73307-v1-mkmlizer: ║                                                                     ║
rirv938-mistral-24b-dpo-73307-v1-mkmlizer: ║  Chai Research Corp.                                                ║
rirv938-mistral-24b-dpo-73307-v1-mkmlizer: ║  Account ID: 7997a29f-0ceb-4cc7-9adf-840c57b4ae6f                   ║
rirv938-mistral-24b-dpo-73307-v1-mkmlizer: ║  Expiration: 2025-04-15 23:59:59                                    ║
rirv938-mistral-24b-dpo-73307-v1-mkmlizer: ║                                                                     ║
rirv938-mistral-24b-dpo-73307-v1-mkmlizer: ╚═════════════════════════════════════════════════════════════════════╝
Unable to record family friendly update due to error: ('Connection aborted.', RemoteDisconnected('Remote end closed connection without response'))
rirv938-mistral-24b-dpo-73307-v1-mkmlizer: Downloaded to shared memory in 180.304s
rirv938-mistral-24b-dpo-73307-v1-mkmlizer: quantizing model to /dev/shm/model_cache, profile:s0, folder:/tmp/tmpwyib_7p7, device:0
rirv938-mistral-24b-dpo-73307-v1-mkmlizer: Saving flywheel model at /dev/shm/model_cache
rirv938-mistral-24b-dpo-73307-v1-mkmlizer: quantized model in 66.490s
rirv938-mistral-24b-dpo-73307-v1-mkmlizer: Processed model rirv938/mistral_24b_dpo_kl_10k_beta2_624_v3 in 246.795s
rirv938-mistral-24b-dpo-73307-v1-mkmlizer: creating bucket guanaco-mkml-models
rirv938-mistral-24b-dpo-73307-v1-mkmlizer: Bucket 's3://guanaco-mkml-models/' created
rirv938-mistral-24b-dpo-73307-v1-mkmlizer: uploading /dev/shm/model_cache to s3://guanaco-mkml-models/rirv938-mistral-24b-dpo-73307-v1
rirv938-mistral-24b-dpo-73307-v1-mkmlizer: cp /dev/shm/model_cache/config.json s3://guanaco-mkml-models/rirv938-mistral-24b-dpo-73307-v1/config.json
rirv938-mistral-24b-dpo-73307-v1-mkmlizer: cp /dev/shm/model_cache/special_tokens_map.json s3://guanaco-mkml-models/rirv938-mistral-24b-dpo-73307-v1/special_tokens_map.json
rirv938-mistral-24b-dpo-73307-v1-mkmlizer: cp /dev/shm/model_cache/tokenizer_config.json s3://guanaco-mkml-models/rirv938-mistral-24b-dpo-73307-v1/tokenizer_config.json
rirv938-mistral-24b-dpo-73307-v1-mkmlizer: cp /dev/shm/model_cache/tokenizer.json s3://guanaco-mkml-models/rirv938-mistral-24b-dpo-73307-v1/tokenizer.json
Job rirv938-mistral-24b-dpo-73307-v1-mkmlizer completed after 296.8s with status: succeeded
Stopping job with name rirv938-mistral-24b-dpo-73307-v1-mkmlizer
Pipeline stage MKMLizer completed in 297.26s
run pipeline stage %s
Running pipeline stage MKMLTemplater
Pipeline stage MKMLTemplater completed in 0.14s
run pipeline stage %s
Running pipeline stage MKMLDeployer
Creating inference service rirv938-mistral-24b-dpo-73307-v1
Waiting for inference service rirv938-mistral-24b-dpo-73307-v1 to be ready
Failed to get response for submission nitral-ai-captain-eris_45741_v27: HTTPConnectionPool(host='nitral-ai-captain-eris-45741-v27-predictor.tenant-chaiml-guanaco.k.chaiverse.com', port=80): Read timed out. (read timeout=12.0)
Inference service rirv938-mistral-24b-dpo-73307-v1 ready after 100.42663383483887s
Pipeline stage MKMLDeployer completed in 100.91s
run pipeline stage %s
Running pipeline stage StressChecker
Received healthy response to inference request in 3.239889621734619s
Received healthy response to inference request in 2.98870849609375s
Failed to get response for submission nitral-ai-captain-eris_45741_v27: HTTPConnectionPool(host='nitral-ai-captain-eris-45741-v27-predictor.tenant-chaiml-guanaco.k.chaiverse.com', port=80): Read timed out. (read timeout=12.0)
Received healthy response to inference request in 2.7816851139068604s
Received healthy response to inference request in 2.6321771144866943s
Received healthy response to inference request in 2.75858998298645s
5 requests
0 failed requests
5th percentile: 2.6574596881866457
10th percentile: 2.6827422618865966
20th percentile: 2.733307409286499
30th percentile: 2.763209009170532
40th percentile: 2.7724470615386965
50th percentile: 2.7816851139068604
60th percentile: 2.864494466781616
70th percentile: 2.947303819656372
80th percentile: 3.038944721221924
90th percentile: 3.1394171714782715
95th percentile: 3.1896533966064453
99th percentile: 3.2298423767089846
mean time: 2.8802100658416747
Pipeline stage StressChecker completed in 15.61s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyTriggerPipeline
run_pipeline:run_in_cloud %s
starting trigger_guanaco_pipeline args=%s
triggered trigger_guanaco_pipeline args=%s
Pipeline stage OfflineFamilyFriendlyTriggerPipeline completed in 0.63s
run pipeline stage %s
Running pipeline stage TriggerMKMLProfilingPipeline
run_pipeline:run_in_cloud %s
starting trigger_guanaco_pipeline args=%s
triggered trigger_guanaco_pipeline args=%s
Pipeline stage TriggerMKMLProfilingPipeline completed in 0.64s
Shutdown handler de-registered
rirv938-mistral-24b-dpo_73307_v1 status is now deployed due to DeploymentManager action
Shutdown handler registered
run pipeline %s
run pipeline stage %s
Running pipeline stage MKMLProfilerDeleter
Skipping teardown as no inference service was successfully deployed
Pipeline stage MKMLProfilerDeleter completed in 0.11s
run pipeline stage %s
Running pipeline stage MKMLProfilerTemplater
Pipeline stage MKMLProfilerTemplater completed in 0.10s
run pipeline stage %s
Running pipeline stage MKMLProfilerDeployer
Creating inference service rirv938-mistral-24b-dpo-73307-v1-profiler
Waiting for inference service rirv938-mistral-24b-dpo-73307-v1-profiler to be ready
Shutdown handler registered
run pipeline %s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyScorer
Evaluating %s Family Friendly Score with %s threads
%s, retrying in %s seconds...
Evaluating %s Family Friendly Score with %s threads
Pipeline stage OfflineFamilyFriendlyScorer completed in 5268.90s
Shutdown handler de-registered
rirv938-mistral-24b-dpo_73307_v1 status is now inactive due to auto deactivation removed underperforming models
rirv938-mistral-24b-dpo_73307_v1 status is now torndown due to DeploymentManager action
ChatRequest
Generation Params
Prompt Formatter
Chat History
ChatMessage 1