tang82-mv-lora-dpo

developer_uid: cycy233

submission_id: tang82-mv-lora-dpo_v1

model_name: 777

model_group: Tang82/mv_lora_dpo

status: torndown

timestamp: 2025-04-23T08:29:36+00:00

num_battles: 7459

num_wins: 3635

celo_rating: 1278.78

family_friendly_score: 0.5438000000000001

family_friendly_standard_error: 0.0070438847236450434

submission_type: basic

model_repo: Tang82/mv_lora_dpo

model_architecture: MistralForCausalLM

model_num_parameters: 12772070400.0

best_of: 8

max_input_tokens: 1024

max_output_tokens: 64

latencies: [{'batch_size': 1, 'throughput': 0.6054031201935808, 'latency_mean': 1.651730762720108, 'latency_p50': 1.6635557413101196, 'latency_p90': 1.8122236490249635}, {'batch_size': 3, 'throughput': 1.1071383472046714, 'latency_mean': 2.7013808035850526, 'latency_p50': 2.7041651010513306, 'latency_p90': 2.9831163406372068}, {'batch_size': 5, 'throughput': 1.323059297541384, 'latency_mean': 3.7594631123542785, 'latency_p50': 3.755621910095215, 'latency_p90': 4.141138672828674}, {'batch_size': 6, 'throughput': 1.3846924333726967, 'latency_mean': 4.310766832828522, 'latency_p50': 4.338842511177063, 'latency_p90': 4.8002824783325195}, {'batch_size': 8, 'throughput': 1.4405832522865065, 'latency_mean': 5.524466601610183, 'latency_p50': 5.504761099815369, 'latency_p90': 6.199666833877563}, {'batch_size': 10, 'throughput': 1.4702302820865263, 'latency_mean': 6.7520297062397, 'latency_p50': 6.813309907913208, 'latency_p90': 7.628601503372193}]

gpu_counts: {'NVIDIA RTX A5000': 1}

display_name: 777

is_internal_developer: False

language_model: Tang82/mv_lora_dpo

model_size: 13B

ranking_group: single

throughput_3p7s: 1.32

us_pacific_date: 2025-04-23

win_ratio: 0.4873307413862448

generation_params: {'temperature': 1.0, 'top_p': 1.0, 'min_p': 0.0, 'top_k': 40, 'presence_penalty': 0.0, 'frequency_penalty': 0.0, 'stopping_words': ['\n'], 'max_input_tokens': 1024, 'best_of': 8, 'max_output_tokens': 64}

formatter: {'memory_template': "{bot_name}'s Persona: {memory}\n####\n", 'prompt_template': '{prompt}\n<START>\n', 'bot_template': '{bot_name}: {message}\n', 'user_template': '{user_name}: {message}\n', 'response_template': '{bot_name}:', 'truncate_by_message': False}

Resubmit model

Shutdown handler not registered because Python interpreter is not running in the main thread
run pipeline %s
run pipeline stage %s
Running pipeline stage MKMLizer
Starting job with name tang82-mv-lora-dpo-v1-mkmlizer
Waiting for job on tang82-mv-lora-dpo-v1-mkmlizer to finish
tang82-mv-lora-dpo-v1-mkmlizer: ╔═════════════════════════════════════════════════════════════════════╗
tang82-mv-lora-dpo-v1-mkmlizer: ║     _____            __           __                                ║
tang82-mv-lora-dpo-v1-mkmlizer: ║    / _/ /_ ___    __/ /  ___ ___ / /                                ║
tang82-mv-lora-dpo-v1-mkmlizer: ║   / _/ / // / |/|/ / _ \/ -_) -_) /                                 ║
tang82-mv-lora-dpo-v1-mkmlizer: ║  /_//_/\_, /|__,__/_//_/\__/\__/_/                                  ║
tang82-mv-lora-dpo-v1-mkmlizer: ║       /___/                                                         ║
tang82-mv-lora-dpo-v1-mkmlizer: ║                                                                     ║
tang82-mv-lora-dpo-v1-mkmlizer: ║  Version: 0.12.8                                                    ║
tang82-mv-lora-dpo-v1-mkmlizer: ║  Copyright 2023 MK ONE TECHNOLOGIES Inc.                            ║
tang82-mv-lora-dpo-v1-mkmlizer: ║  https://mk1.ai                                                     ║
tang82-mv-lora-dpo-v1-mkmlizer: ║                                                                     ║
tang82-mv-lora-dpo-v1-mkmlizer: ║  The license key for the current software has been verified as      ║
tang82-mv-lora-dpo-v1-mkmlizer: ║  belonging to:                                                      ║
tang82-mv-lora-dpo-v1-mkmlizer: ║                                                                     ║
tang82-mv-lora-dpo-v1-mkmlizer: ║  Chai Research Corp.                                                ║
tang82-mv-lora-dpo-v1-mkmlizer: ║  Account ID: 7997a29f-0ceb-4cc7-9adf-840c57b4ae6f                   ║
tang82-mv-lora-dpo-v1-mkmlizer: ║  Expiration: 2028-03-31 23:59:59                                    ║
tang82-mv-lora-dpo-v1-mkmlizer: ║                                                                     ║
tang82-mv-lora-dpo-v1-mkmlizer: ╚═════════════════════════════════════════════════════════════════════╝
tang82-mv-lora-dpo-v1-mkmlizer: Downloaded to shared memory in 36.241s
tang82-mv-lora-dpo-v1-mkmlizer: quantizing model to /dev/shm/model_cache, profile:s0, folder:/tmp/tmpfq7_nf5w, device:0
tang82-mv-lora-dpo-v1-mkmlizer: Saving flywheel model at /dev/shm/model_cache
tang82-mv-lora-dpo-v1-mkmlizer: quantized model in 36.386s
tang82-mv-lora-dpo-v1-mkmlizer: Processed model Tang82/mv_lora_dpo in 72.628s
tang82-mv-lora-dpo-v1-mkmlizer: creating bucket guanaco-mkml-models
tang82-mv-lora-dpo-v1-mkmlizer: Bucket 's3://guanaco-mkml-models/' created
tang82-mv-lora-dpo-v1-mkmlizer: uploading /dev/shm/model_cache to s3://guanaco-mkml-models/tang82-mv-lora-dpo-v1
tang82-mv-lora-dpo-v1-mkmlizer: cp /dev/shm/model_cache/special_tokens_map.json s3://guanaco-mkml-models/tang82-mv-lora-dpo-v1/special_tokens_map.json
tang82-mv-lora-dpo-v1-mkmlizer: cp /dev/shm/model_cache/config.json s3://guanaco-mkml-models/tang82-mv-lora-dpo-v1/config.json
tang82-mv-lora-dpo-v1-mkmlizer: cp /dev/shm/model_cache/tokenizer_config.json s3://guanaco-mkml-models/tang82-mv-lora-dpo-v1/tokenizer_config.json
tang82-mv-lora-dpo-v1-mkmlizer: cp /dev/shm/model_cache/tokenizer.json s3://guanaco-mkml-models/tang82-mv-lora-dpo-v1/tokenizer.json
tang82-mv-lora-dpo-v1-mkmlizer: cp /dev/shm/model_cache/flywheel_model.0.safetensors s3://guanaco-mkml-models/tang82-mv-lora-dpo-v1/flywheel_model.0.safetensors
tang82-mv-lora-dpo-v1-mkmlizer: 
Loading 0:   0%|          | 0/363 [00:00<?, ?it/s]
Loading 0:   1%|▏         | 5/363 [00:00<00:12, 29.08it/s]
Loading 0:   3%|▎         | 12/363 [00:00<00:07, 47.05it/s]
Loading 0:   5%|▍         | 18/363 [00:00<00:07, 46.73it/s]
Loading 0:   6%|▋         | 23/363 [00:00<00:09, 36.22it/s]
Loading 0:   9%|▊         | 31/363 [00:00<00:07, 45.75it/s]
Loading 0:  10%|█         | 37/363 [00:00<00:07, 43.73it/s]
Loading 0:  12%|█▏        | 42/363 [00:00<00:07, 42.71it/s]
Loading 0:  13%|█▎        | 49/363 [00:01<00:06, 48.29it/s]
Loading 0:  15%|█▌        | 55/363 [00:01<00:06, 45.99it/s]
Loading 0:  17%|█▋        | 61/363 [00:01<00:08, 34.28it/s]
Loading 0:  18%|█▊        | 65/363 [00:01<00:08, 33.49it/s]
Loading 0:  20%|█▉        | 72/363 [00:01<00:07, 40.24it/s]
Loading 0:  21%|██        | 77/363 [00:01<00:06, 41.92it/s]
Loading 0:  23%|██▎       | 82/363 [00:02<00:07, 36.11it/s]
Loading 0:  25%|██▍       | 90/363 [00:02<00:06, 44.02it/s]
Loading 0:  26%|██▋       | 96/363 [00:02<00:06, 43.35it/s]
Loading 0:  28%|██▊       | 101/363 [00:02<00:06, 43.13it/s]
Loading 0:  30%|██▉       | 108/363 [00:02<00:05, 49.58it/s]
Loading 0:  31%|███▏      | 114/363 [00:02<00:05, 42.45it/s]
Loading 0:  33%|███▎      | 119/363 [00:02<00:06, 36.20it/s]
Loading 0:  34%|███▍      | 125/363 [00:03<00:05, 40.64it/s]
Loading 0:  36%|███▌      | 130/363 [00:03<00:05, 41.17it/s]
Loading 0:  37%|███▋      | 135/363 [00:03<00:05, 42.37it/s]
Loading 0:  39%|███▉      | 141/363 [00:03<00:05, 41.97it/s]
Loading 0:  40%|████      | 146/363 [00:03<00:07, 29.90it/s]
Loading 0:  41%|████▏     | 150/363 [00:03<00:07, 29.58it/s]
Loading 0:  43%|████▎     | 156/363 [00:03<00:05, 35.57it/s]
Loading 0:  44%|████▍     | 161/363 [00:04<00:05, 38.05it/s]
Loading 0:  46%|████▌     | 166/363 [00:04<00:04, 39.59it/s]
Loading 0:  47%|████▋     | 172/363 [00:04<00:04, 38.90it/s]
Loading 0:  49%|████▉     | 177/363 [00:04<00:04, 39.00it/s]
Loading 0:  51%|█████     | 184/363 [00:04<00:04, 44.15it/s]
Loading 0:  52%|█████▏    | 190/363 [00:04<00:04, 42.27it/s]
Loading 0:  54%|█████▎    | 195/363 [00:04<00:04, 41.78it/s]
Loading 0:  56%|█████▌    | 202/363 [00:04<00:03, 46.39it/s]
Loading 0:  57%|█████▋    | 208/363 [00:05<00:03, 43.38it/s]
Loading 0:  59%|█████▊    | 213/363 [00:05<00:03, 41.90it/s]
Loading 0:  61%|██████    | 220/363 [00:05<00:02, 48.03it/s]
Loading 0:  62%|██████▏   | 225/363 [00:05<00:04, 30.16it/s]
Loading 0:  63%|██████▎   | 230/363 [00:05<00:04, 32.57it/s]
Loading 0:  65%|██████▌   | 237/363 [00:05<00:03, 39.65it/s]
Loading 0:  67%|██████▋   | 242/363 [00:06<00:02, 41.18it/s]
Loading 0:  68%|██████▊   | 247/363 [00:06<00:02, 42.56it/s]
Loading 0:  70%|██████▉   | 253/363 [00:06<00:02, 41.80it/s]
Loading 0:  71%|███████   | 258/363 [00:06<00:02, 41.27it/s]
Loading 0:  73%|███████▎  | 265/363 [00:06<00:02, 46.29it/s]
Loading 0:  75%|███████▍  | 271/363 [00:06<00:02, 44.73it/s]
Loading 0:  76%|███████▌  | 276/363 [00:06<00:02, 41.77it/s]
Loading 0:  78%|███████▊  | 282/363 [00:06<00:01, 45.30it/s]
Loading 0:  79%|███████▉  | 287/363 [00:07<00:01, 45.30it/s]
Loading 0:  80%|████████  | 292/363 [00:07<00:01, 45.39it/s]
Loading 0:  82%|████████▏ | 297/363 [00:07<00:01, 46.05it/s]
Loading 0:  83%|████████▎ | 302/363 [00:07<00:01, 46.85it/s]
Loading 0:  85%|████████▍ | 307/363 [00:14<00:23,  2.35it/s]
Loading 0:  86%|████████▌ | 312/363 [00:14<00:15,  3.26it/s]
Loading 0:  88%|████████▊ | 320/363 [00:14<00:08,  5.30it/s]
Loading 0:  90%|████████▉ | 326/363 [00:14<00:05,  7.21it/s]
Loading 0:  91%|█████████ | 331/363 [00:14<00:03,  9.26it/s]
Loading 0:  93%|█████████▎| 338/363 [00:14<00:01, 13.18it/s]
Loading 0:  95%|█████████▍| 344/363 [00:15<00:01, 16.65it/s]
Loading 0:  96%|█████████▌| 349/363 [00:15<00:00, 19.89it/s]
Loading 0:  98%|█████████▊| 356/363 [00:15<00:00, 26.19it/s]
Loading 0: 100%|█████████▉| 362/363 [00:15<00:00, 29.82it/s]
                                                            
Job tang82-mv-lora-dpo-v1-mkmlizer completed after 94.01s with status: succeeded
Stopping job with name tang82-mv-lora-dpo-v1-mkmlizer
Pipeline stage MKMLizer completed in 94.52s
run pipeline stage %s
Running pipeline stage MKMLTemplater
Pipeline stage MKMLTemplater completed in 0.15s
run pipeline stage %s
Running pipeline stage MKMLDeployer
Creating inference service tang82-mv-lora-dpo-v1
Waiting for inference service tang82-mv-lora-dpo-v1 to be ready
Inference service tang82-mv-lora-dpo-v1 ready after 140.61470794677734s
Pipeline stage MKMLDeployer completed in 141.13s
run pipeline stage %s
Running pipeline stage StressChecker
Received healthy response to inference request in 2.2398157119750977s
Received healthy response to inference request in 1.679065227508545s
Received healthy response to inference request in 1.885134220123291s
Received healthy response to inference request in 1.4894123077392578s
Received healthy response to inference request in 1.7020354270935059s
5 requests
0 failed requests
5th percentile: 1.5273428916931153
10th percentile: 1.5652734756469726
20th percentile: 1.6411346435546874
30th percentile: 1.683659267425537
40th percentile: 1.6928473472595216
50th percentile: 1.7020354270935059
60th percentile: 1.7752749443054199
70th percentile: 1.848514461517334
80th percentile: 1.9560705184936524
90th percentile: 2.097943115234375
95th percentile: 2.1688794136047362
99th percentile: 2.2256284523010255
mean time: 1.7990925788879395
Pipeline stage StressChecker completed in 10.37s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyTriggerPipeline
run_pipeline:run_in_cloud %s
starting trigger_guanaco_pipeline args=%s
triggered trigger_guanaco_pipeline args=%s
Pipeline stage OfflineFamilyFriendlyTriggerPipeline completed in 0.70s
run pipeline stage %s
Running pipeline stage TriggerMKMLProfilingPipeline
run_pipeline:run_in_cloud %s
starting trigger_guanaco_pipeline args=%s
triggered trigger_guanaco_pipeline args=%s
Pipeline stage TriggerMKMLProfilingPipeline completed in 0.67s
Shutdown handler de-registered
tang82-mv-lora-dpo_v1 status is now deployed due to DeploymentManager action
Shutdown handler registered
run pipeline %s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyScorer
Evaluating %s Family Friendly Score with %s threads
Pipeline stage OfflineFamilyFriendlyScorer completed in 2770.21s
Shutdown handler de-registered
tang82-mv-lora-dpo_v1 status is now inactive due to auto deactivation removed underperforming models
tang82-mv-lora-dpo_v1 status is now torndown due to DeploymentManager action