bbchicago-tang82mv-dpo-ep1

developer_uid: BBChicago

submission_id: bbchicago-tang82mv-dpo-ep1_v1

model_name: tang82mv-dpo-ep1

model_group: BBChicago/tang82mv-dpo-e

status: torndown

timestamp: 2025-04-30T02:33:42+00:00

num_battles: 5921

num_wins: 2944

celo_rating: 1288.49

family_friendly_score: 0.512

family_friendly_standard_error: 0.007069031050999847

submission_type: basic

model_repo: BBChicago/tang82mv-dpo-ep1

model_architecture: MistralForCausalLM

model_num_parameters: 12772070400.0

best_of: 8

max_input_tokens: 1024

max_output_tokens: 64

latencies: [{'batch_size': 1, 'throughput': 0.6095259473395559, 'latency_mean': 1.640554963350296, 'latency_p50': 1.6269606351852417, 'latency_p90': 1.8241519927978516}, {'batch_size': 3, 'throughput': 1.1083775784364276, 'latency_mean': 2.7019489121437075, 'latency_p50': 2.7069003582000732, 'latency_p90': 2.9885871410369873}, {'batch_size': 5, 'throughput': 1.3290368291889023, 'latency_mean': 3.7430797731876373, 'latency_p50': 3.7507447004318237, 'latency_p90': 4.12868721485138}, {'batch_size': 6, 'throughput': 1.4049099625338608, 'latency_mean': 4.255060925483703, 'latency_p50': 4.28594446182251, 'latency_p90': 4.789736700057984}, {'batch_size': 8, 'throughput': 1.4656134131044933, 'latency_mean': 5.4282365572452544, 'latency_p50': 5.474196791648865, 'latency_p90': 6.021982216835022}, {'batch_size': 10, 'throughput': 1.4994528126782538, 'latency_mean': 6.62029226899147, 'latency_p50': 6.5817660093307495, 'latency_p90': 7.490721678733825}]

gpu_counts: {'NVIDIA RTX A5000': 1}

display_name: tang82mv-dpo-ep1

is_internal_developer: False

language_model: BBChicago/tang82mv-dpo-ep1

model_size: 13B

ranking_group: single

throughput_3p7s: 1.33

us_pacific_date: 2025-04-29

win_ratio: 0.49721330856274276

generation_params: {'temperature': 1.0, 'top_p': 1.0, 'min_p': 0.0, 'top_k': 40, 'presence_penalty': 0.0, 'frequency_penalty': 0.0, 'stopping_words': ['\n', '<', '>', '[INST]', '[\\INST]'], 'max_input_tokens': 1024, 'best_of': 8, 'max_output_tokens': 64}

formatter: {'memory_template': "{bot_name}'s Persona: {memory}\n####\n", 'prompt_template': '{prompt}\n<START>\n', 'bot_template': '{bot_name}: {message}\n', 'user_template': '{user_name}: {message}\n', 'response_template': '{bot_name}:', 'truncate_by_message': False}

Resubmit model

Shutdown handler not registered because Python interpreter is not running in the main thread
run pipeline %s
run pipeline stage %s
Running pipeline stage MKMLizer
Starting job with name bbchicago-tang82mv-dpo-ep1-v1-mkmlizer
Waiting for job on bbchicago-tang82mv-dpo-ep1-v1-mkmlizer to finish
bbchicago-tang82mv-dpo-ep1-v1-mkmlizer: ╔═════════════════════════════════════════════════════════════════════╗
bbchicago-tang82mv-dpo-ep1-v1-mkmlizer: ║     _____            __           __                                ║
bbchicago-tang82mv-dpo-ep1-v1-mkmlizer: ║    / _/ /_ ___    __/ /  ___ ___ / /                                ║
bbchicago-tang82mv-dpo-ep1-v1-mkmlizer: ║   / _/ / // / |/|/ / _ \/ -_) -_) /                                 ║
bbchicago-tang82mv-dpo-ep1-v1-mkmlizer: ║  /_//_/\_, /|__,__/_//_/\__/\__/_/                                  ║
bbchicago-tang82mv-dpo-ep1-v1-mkmlizer: ║       /___/                                                         ║
bbchicago-tang82mv-dpo-ep1-v1-mkmlizer: ║                                                                     ║
bbchicago-tang82mv-dpo-ep1-v1-mkmlizer: ║  Version: 0.12.8                                                    ║
bbchicago-tang82mv-dpo-ep1-v1-mkmlizer: ║  Copyright 2023 MK ONE TECHNOLOGIES Inc.                            ║
bbchicago-tang82mv-dpo-ep1-v1-mkmlizer: ║  https://mk1.ai                                                     ║
bbchicago-tang82mv-dpo-ep1-v1-mkmlizer: ║                                                                     ║
bbchicago-tang82mv-dpo-ep1-v1-mkmlizer: ║  The license key for the current software has been verified as      ║
bbchicago-tang82mv-dpo-ep1-v1-mkmlizer: ║  belonging to:                                                      ║
bbchicago-tang82mv-dpo-ep1-v1-mkmlizer: ║                                                                     ║
bbchicago-tang82mv-dpo-ep1-v1-mkmlizer: ║  Chai Research Corp.                                                ║
bbchicago-tang82mv-dpo-ep1-v1-mkmlizer: ║  Account ID: 7997a29f-0ceb-4cc7-9adf-840c57b4ae6f                   ║
bbchicago-tang82mv-dpo-ep1-v1-mkmlizer: ║  Expiration: 2028-03-31 23:59:59                                    ║
bbchicago-tang82mv-dpo-ep1-v1-mkmlizer: ║                                                                     ║
bbchicago-tang82mv-dpo-ep1-v1-mkmlizer: ╚═════════════════════════════════════════════════════════════════════╝
bbchicago-tang82mv-dpo-ep1-v1-mkmlizer: Downloaded to shared memory in 43.247s
bbchicago-tang82mv-dpo-ep1-v1-mkmlizer: quantizing model to /dev/shm/model_cache, profile:s0, folder:/tmp/tmptulnt33l, device:0
bbchicago-tang82mv-dpo-ep1-v1-mkmlizer: Saving flywheel model at /dev/shm/model_cache
bbchicago-tang82mv-dpo-ep1-v1-mkmlizer: quantized model in 36.313s
bbchicago-tang82mv-dpo-ep1-v1-mkmlizer: Processed model BBChicago/tang82mv-dpo-ep1 in 79.560s
bbchicago-tang82mv-dpo-ep1-v1-mkmlizer: creating bucket guanaco-mkml-models
bbchicago-tang82mv-dpo-ep1-v1-mkmlizer: Bucket 's3://guanaco-mkml-models/' created
bbchicago-tang82mv-dpo-ep1-v1-mkmlizer: uploading /dev/shm/model_cache to s3://guanaco-mkml-models/bbchicago-tang82mv-dpo-ep1-v1
bbchicago-tang82mv-dpo-ep1-v1-mkmlizer: cp /dev/shm/model_cache/config.json s3://guanaco-mkml-models/bbchicago-tang82mv-dpo-ep1-v1/config.json
bbchicago-tang82mv-dpo-ep1-v1-mkmlizer: cp /dev/shm/model_cache/special_tokens_map.json s3://guanaco-mkml-models/bbchicago-tang82mv-dpo-ep1-v1/special_tokens_map.json
bbchicago-tang82mv-dpo-ep1-v1-mkmlizer: cp /dev/shm/model_cache/tokenizer_config.json s3://guanaco-mkml-models/bbchicago-tang82mv-dpo-ep1-v1/tokenizer_config.json
bbchicago-tang82mv-dpo-ep1-v1-mkmlizer: cp /dev/shm/model_cache/tokenizer.json s3://guanaco-mkml-models/bbchicago-tang82mv-dpo-ep1-v1/tokenizer.json
bbchicago-tang82mv-dpo-ep1-v1-mkmlizer: cp /dev/shm/model_cache/flywheel_model.0.safetensors s3://guanaco-mkml-models/bbchicago-tang82mv-dpo-ep1-v1/flywheel_model.0.safetensors
bbchicago-tang82mv-dpo-ep1-v1-mkmlizer: 
Loading 0:   0%|          | 0/363 [00:00<?, ?it/s]
Loading 0:   1%|▏         | 5/363 [00:00<00:11, 30.25it/s]
Loading 0:   4%|▎         | 13/363 [00:00<00:07, 49.59it/s]
Loading 0:   5%|▌         | 19/363 [00:00<00:07, 47.20it/s]
Loading 0:   7%|▋         | 24/363 [00:00<00:07, 46.11it/s]
Loading 0:   9%|▊         | 31/363 [00:00<00:06, 52.68it/s]
Loading 0:  10%|█         | 37/363 [00:00<00:06, 48.36it/s]
Loading 0:  12%|█▏        | 42/363 [00:00<00:07, 44.73it/s]
Loading 0:  13%|█▎        | 49/363 [00:01<00:06, 50.51it/s]
Loading 0:  15%|█▌        | 55/363 [00:01<00:06, 48.47it/s]
Loading 0:  17%|█▋        | 61/363 [00:01<00:08, 35.69it/s]
Loading 0:  18%|█▊        | 66/363 [00:01<00:08, 36.01it/s]
Loading 0:  20%|█▉        | 71/363 [00:01<00:07, 38.93it/s]
Loading 0:  21%|██        | 76/363 [00:01<00:07, 39.20it/s]
Loading 0:  22%|██▏       | 81/363 [00:01<00:06, 40.95it/s]
Loading 0:  24%|██▎       | 86/363 [00:01<00:06, 43.06it/s]
Loading 0:  25%|██▌       | 91/363 [00:02<00:07, 37.46it/s]
Loading 0:  27%|██▋       | 99/363 [00:02<00:05, 46.20it/s]
Loading 0:  29%|██▉       | 105/363 [00:02<00:05, 44.24it/s]
Loading 0:  30%|███       | 110/363 [00:02<00:05, 44.90it/s]
Loading 0:  32%|███▏      | 115/363 [00:02<00:05, 44.90it/s]
Loading 0:  33%|███▎      | 120/363 [00:02<00:05, 41.40it/s]
Loading 0:  34%|███▍      | 125/363 [00:02<00:05, 41.67it/s]
Loading 0:  36%|███▌      | 130/363 [00:03<00:05, 40.92it/s]
Loading 0:  37%|███▋      | 135/363 [00:03<00:05, 41.23it/s]
Loading 0:  39%|███▊      | 140/363 [00:03<00:05, 42.43it/s]
Loading 0:  40%|███▉      | 145/363 [00:03<00:08, 25.99it/s]
Loading 0:  41%|████      | 149/363 [00:03<00:07, 27.13it/s]
Loading 0:  43%|████▎     | 156/363 [00:03<00:05, 34.68it/s]
Loading 0:  44%|████▍     | 161/363 [00:03<00:05, 36.93it/s]
Loading 0:  46%|████▌     | 166/363 [00:04<00:05, 38.59it/s]
Loading 0:  47%|████▋     | 171/363 [00:04<00:04, 40.04it/s]
Loading 0:  48%|████▊     | 176/363 [00:04<00:05, 32.84it/s]
Loading 0:  50%|█████     | 183/363 [00:04<00:04, 40.62it/s]
Loading 0:  52%|█████▏    | 188/363 [00:04<00:04, 40.91it/s]
Loading 0:  53%|█████▎    | 193/363 [00:04<00:04, 40.67it/s]
Loading 0:  55%|█████▍    | 198/363 [00:04<00:03, 42.66it/s]
Loading 0:  56%|█████▌    | 203/363 [00:05<00:04, 36.82it/s]
Loading 0:  58%|█████▊    | 210/363 [00:05<00:03, 43.58it/s]
Loading 0:  59%|█████▉    | 215/363 [00:05<00:03, 43.20it/s]
Loading 0:  61%|██████    | 222/363 [00:05<00:03, 44.38it/s]
Loading 0:  63%|██████▎   | 227/363 [00:05<00:04, 31.43it/s]
Loading 0:  64%|██████▎   | 231/363 [00:05<00:04, 31.22it/s]
Loading 0:  65%|██████▌   | 237/363 [00:05<00:03, 35.14it/s]
Loading 0:  66%|██████▋   | 241/363 [00:06<00:03, 35.62it/s]
Loading 0:  68%|██████▊   | 247/363 [00:06<00:02, 39.71it/s]
Loading 0:  70%|██████▉   | 253/363 [00:06<00:02, 39.10it/s]
Loading 0:  71%|███████   | 258/363 [00:06<00:02, 39.29it/s]
Loading 0:  73%|███████▎  | 264/363 [00:06<00:02, 42.58it/s]
Loading 0:  74%|███████▍  | 269/363 [00:06<00:02, 43.03it/s]
Loading 0:  75%|███████▌  | 274/363 [00:06<00:02, 41.64it/s]
Loading 0:  77%|███████▋  | 280/363 [00:07<00:02, 41.15it/s]
Loading 0:  79%|███████▊  | 285/363 [00:07<00:01, 40.39it/s]
Loading 0:  80%|████████  | 292/363 [00:07<00:01, 45.19it/s]
Loading 0:  82%|████████▏ | 298/363 [00:07<00:01, 43.97it/s]
Loading 0:  83%|████████▎ | 303/363 [00:07<00:01, 45.04it/s]
Loading 0:  85%|████████▍ | 308/363 [00:14<00:21,  2.51it/s]
Loading 0:  86%|████████▌ | 312/363 [00:14<00:15,  3.22it/s]
Loading 0:  88%|████████▊ | 320/363 [00:14<00:08,  5.26it/s]
Loading 0:  90%|████████▉ | 326/363 [00:14<00:05,  7.14it/s]
Loading 0:  91%|█████████ | 331/363 [00:14<00:03,  9.16it/s]
Loading 0:  93%|█████████▎| 338/363 [00:15<00:01, 13.01it/s]
Loading 0:  95%|█████████▍| 344/363 [00:15<00:01, 16.32it/s]
Loading 0:  96%|█████████▌| 349/363 [00:15<00:00, 19.36it/s]
Loading 0:  98%|█████████▊| 356/363 [00:15<00:00, 25.31it/s]
Loading 0: 100%|█████████▉| 362/363 [00:15<00:00, 28.47it/s]
                                                            
Job bbchicago-tang82mv-dpo-ep1-v1-mkmlizer completed after 104.34s with status: succeeded
Stopping job with name bbchicago-tang82mv-dpo-ep1-v1-mkmlizer
Pipeline stage MKMLizer completed in 104.81s
run pipeline stage %s
Running pipeline stage MKMLTemplater
Pipeline stage MKMLTemplater completed in 0.14s
run pipeline stage %s
Running pipeline stage MKMLDeployer
Creating inference service bbchicago-tang82mv-dpo-ep1-v1
Waiting for inference service bbchicago-tang82mv-dpo-ep1-v1 to be ready
Failed to get response for submission cycy233-modelv-chai-step7000_v1: ('http://cycy233-modelv-chai-step7000-v1-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'read tcp 127.0.0.1:34200->127.0.0.1:8080: read: connection reset by peer\n')
Inference service bbchicago-tang82mv-dpo-ep1-v1 ready after 140.4927203655243s
Pipeline stage MKMLDeployer completed in 140.90s
run pipeline stage %s
Running pipeline stage StressChecker
HTTPConnectionPool(host='guanaco-submitter.guanaco-backend.k2.chaiverse.com', port=80): Read timed out. (read timeout=20)
Received unhealthy response to inference request!
Received healthy response to inference request in 2.090158224105835s
Received healthy response to inference request in 1.4998853206634521s
Received healthy response to inference request in 1.4948742389678955s
Received healthy response to inference request in 1.877957820892334s
5 requests
1 failed requests
5th percentile: 1.4958764553070067
10th percentile: 1.4968786716461182
20th percentile: 1.498883104324341
30th percentile: 1.5754998207092286
40th percentile: 1.7267288208007812
50th percentile: 1.877957820892334
60th percentile: 1.9628379821777344
70th percentile: 2.047718143463135
80th percentile: 5.705225133895877
90th percentile: 12.935358953475955
95th percentile: 16.55042586326599
99th percentile: 19.442479391098022
mean time: 5.4256736755371096
%s, retrying in %s seconds...
Received healthy response to inference request in 1.6859405040740967s
Received healthy response to inference request in 1.8183293342590332s
Received healthy response to inference request in 1.7519805431365967s
Received healthy response to inference request in 1.7298271656036377s
Received healthy response to inference request in 0.8367211818695068s
5 requests
0 failed requests
5th percentile: 1.0065650463104248
10th percentile: 1.1764089107513427
20th percentile: 1.5160966396331788
30th percentile: 1.6947178363800048
40th percentile: 1.7122725009918214
50th percentile: 1.7298271656036377
60th percentile: 1.7386885166168213
70th percentile: 1.747549867630005
80th percentile: 1.765250301361084
90th percentile: 1.7917898178100586
95th percentile: 1.8050595760345458
99th percentile: 1.8156753826141356
mean time: 1.5645597457885743
Pipeline stage StressChecker completed in 37.46s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyTriggerPipeline
run_pipeline:run_in_cloud %s
starting trigger_guanaco_pipeline args=%s
triggered trigger_guanaco_pipeline args=%s
Pipeline stage OfflineFamilyFriendlyTriggerPipeline completed in 0.70s
run pipeline stage %s
Running pipeline stage TriggerMKMLProfilingPipeline
run_pipeline:run_in_cloud %s
starting trigger_guanaco_pipeline args=%s
triggered trigger_guanaco_pipeline args=%s
Pipeline stage TriggerMKMLProfilingPipeline completed in 0.70s
Shutdown handler de-registered
bbchicago-tang82mv-dpo-ep1_v1 status is now deployed due to DeploymentManager action
Shutdown handler registered
run pipeline %s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyScorer
Evaluating %s Family Friendly Score with %s threads
Pipeline stage OfflineFamilyFriendlyScorer completed in 2752.04s
Shutdown handler de-registered
bbchicago-tang82mv-dpo-ep1_v1 status is now inactive due to auto deactivation removed underperforming models
bbchicago-tang82mv-dpo-ep1_v1 status is now torndown due to DeploymentManager action