white-bird-1-sft-label1-200

developer_uid: zhenzhe

submission_id: white-bird-1-sft-label1-200_v2

model_name: white-bird-1-sft-label1-200_v2

model_group: white-bird/1_sft_label1_

status: torndown

timestamp: 2025-03-01T18:04:25+00:00

num_battles: 7026

num_wins: 3037

celo_rating: 1221.67

family_friendly_score: 0.6404000000000001

family_friendly_standard_error: 0.006786572625412625

submission_type: basic

model_repo: white-bird/1_sft_label1_200

model_architecture: MistralForCausalLM

model_num_parameters: 12772070400.0

best_of: 8

max_input_tokens: 1024

max_output_tokens: 64

reward_model: default

latencies: [{'batch_size': 1, 'throughput': 0.6027704691447059, 'latency_mean': 1.6589427423477172, 'latency_p50': 1.665208101272583, 'latency_p90': 1.8350449323654174}, {'batch_size': 3, 'throughput': 1.0973613453193014, 'latency_mean': 2.72782097697258, 'latency_p50': 2.7174845933914185, 'latency_p90': 3.0078254461288454}, {'batch_size': 5, 'throughput': 1.3206100943928802, 'latency_mean': 3.759845441579819, 'latency_p50': 3.763449549674988, 'latency_p90': 4.1439725160598755}, {'batch_size': 6, 'throughput': 1.3734976141842299, 'latency_mean': 4.337615633010865, 'latency_p50': 4.3788982629776, 'latency_p90': 4.899048018455505}, {'batch_size': 8, 'throughput': 1.4401457228445032, 'latency_mean': 5.527765642404557, 'latency_p50': 5.506275653839111, 'latency_p90': 6.229314827919006}, {'batch_size': 10, 'throughput': 1.488603364059745, 'latency_mean': 6.668109028339386, 'latency_p50': 6.636715650558472, 'latency_p90': 7.595463490486145}]

gpu_counts: {'NVIDIA RTX A5000': 1}

display_name: white-bird-1-sft-label1-200_v2

is_internal_developer: False

language_model: white-bird/1_sft_label1_200

model_size: 13B

ranking_group: single

throughput_3p7s: 1.32

us_pacific_date: 2025-03-01

win_ratio: 0.4322516367776829

generation_params: {'temperature': 0.4, 'top_p': 0.925, 'min_p': 0.075, 'top_k': 80, 'presence_penalty': 0.5, 'frequency_penalty': 0.5, 'stopping_words': ['\n', '####', 'User:', '</s>', 'Bot:', 'Me:', 'You:'], 'max_input_tokens': 1024, 'best_of': 8, 'max_output_tokens': 64}

formatter: {'memory_template': "{bot_name}'s Persona: {memory}\n####\n", 'prompt_template': '{prompt}\n<START>\n', 'bot_template': '{bot_name}: {message}\n', 'user_template': '{user_name}: {message}\n', 'response_template': '{bot_name}:', 'truncate_by_message': False}

Resubmit model

Shutdown handler not registered because Python interpreter is not running in the main thread
run pipeline %s
run pipeline stage %s
Running pipeline stage MKMLizer
Starting job with name white-bird-1-sft-label1-200-v2-mkmlizer
Waiting for job on white-bird-1-sft-label1-200-v2-mkmlizer to finish
white-bird-1-sft-label1-200-v2-mkmlizer: ╔═════════════════════════════════════════════════════════════════════╗
white-bird-1-sft-label1-200-v2-mkmlizer: ║     _____            __           __                                ║
white-bird-1-sft-label1-200-v2-mkmlizer: ║    / _/ /_ ___    __/ /  ___ ___ / /                                ║
white-bird-1-sft-label1-200-v2-mkmlizer: ║   / _/ / // / |/|/ / _ \/ -_) -_) /                                 ║
white-bird-1-sft-label1-200-v2-mkmlizer: ║  /_//_/\_, /|__,__/_//_/\__/\__/_/                                  ║
white-bird-1-sft-label1-200-v2-mkmlizer: ║       /___/                                                         ║
white-bird-1-sft-label1-200-v2-mkmlizer: ║                                                                     ║
white-bird-1-sft-label1-200-v2-mkmlizer: ║  Version: 0.12.8                                                    ║
white-bird-1-sft-label1-200-v2-mkmlizer: ║  Copyright 2023 MK ONE TECHNOLOGIES Inc.                            ║
white-bird-1-sft-label1-200-v2-mkmlizer: ║  https://mk1.ai                                                     ║
white-bird-1-sft-label1-200-v2-mkmlizer: ║                                                                     ║
white-bird-1-sft-label1-200-v2-mkmlizer: ║  The license key for the current software has been verified as      ║
white-bird-1-sft-label1-200-v2-mkmlizer: ║  belonging to:                                                      ║
white-bird-1-sft-label1-200-v2-mkmlizer: ║                                                                     ║
white-bird-1-sft-label1-200-v2-mkmlizer: ║  Chai Research Corp.                                                ║
white-bird-1-sft-label1-200-v2-mkmlizer: ║  Account ID: 7997a29f-0ceb-4cc7-9adf-840c57b4ae6f                   ║
white-bird-1-sft-label1-200-v2-mkmlizer: ║  Expiration: 2025-04-15 23:59:59                                    ║
white-bird-1-sft-label1-200-v2-mkmlizer: ║                                                                     ║
white-bird-1-sft-label1-200-v2-mkmlizer: ╚═════════════════════════════════════════════════════════════════════╝
white-bird-1-sft-label1-200-v2-mkmlizer: Downloaded to shared memory in 36.218s
white-bird-1-sft-label1-200-v2-mkmlizer: quantizing model to /dev/shm/model_cache, profile:s0, folder:/tmp/tmpwii159j_, device:0
white-bird-1-sft-label1-200-v2-mkmlizer: Saving flywheel model at /dev/shm/model_cache
white-bird-1-sft-label1-200-v2-mkmlizer: quantized model in 35.657s
white-bird-1-sft-label1-200-v2-mkmlizer: Processed model white-bird/1_sft_label1_200 in 71.876s
white-bird-1-sft-label1-200-v2-mkmlizer: creating bucket guanaco-mkml-models
white-bird-1-sft-label1-200-v2-mkmlizer: Bucket 's3://guanaco-mkml-models/' created
white-bird-1-sft-label1-200-v2-mkmlizer: uploading /dev/shm/model_cache to s3://guanaco-mkml-models/white-bird-1-sft-label1-200-v2
white-bird-1-sft-label1-200-v2-mkmlizer: cp /dev/shm/model_cache/config.json s3://guanaco-mkml-models/white-bird-1-sft-label1-200-v2/config.json
white-bird-1-sft-label1-200-v2-mkmlizer: cp /dev/shm/model_cache/special_tokens_map.json s3://guanaco-mkml-models/white-bird-1-sft-label1-200-v2/special_tokens_map.json
white-bird-1-sft-label1-200-v2-mkmlizer: cp /dev/shm/model_cache/tokenizer_config.json s3://guanaco-mkml-models/white-bird-1-sft-label1-200-v2/tokenizer_config.json
white-bird-1-sft-label1-200-v2-mkmlizer: cp /dev/shm/model_cache/tokenizer.json s3://guanaco-mkml-models/white-bird-1-sft-label1-200-v2/tokenizer.json
white-bird-1-sft-label1-200-v2-mkmlizer: cp /dev/shm/model_cache/flywheel_model.0.safetensors s3://guanaco-mkml-models/white-bird-1-sft-label1-200-v2/flywheel_model.0.safetensors
white-bird-1-sft-label1-200-v2-mkmlizer: 
Loading 0:   0%|          | 0/363 [00:00<?, ?it/s]
Loading 0:   1%|▏         | 5/363 [00:00<00:11, 30.84it/s]
Loading 0:   4%|▎         | 13/363 [00:00<00:06, 51.12it/s]
Loading 0:   5%|▌         | 19/363 [00:00<00:07, 45.45it/s]
Loading 0:   7%|▋         | 24/363 [00:00<00:07, 43.60it/s]
Loading 0:   9%|▊         | 31/363 [00:00<00:06, 48.93it/s]
Loading 0:  10%|█         | 37/363 [00:00<00:07, 45.68it/s]
Loading 0:  12%|█▏        | 42/363 [00:00<00:07, 44.47it/s]
Loading 0:  13%|█▎        | 49/363 [00:01<00:06, 49.87it/s]
Loading 0:  15%|█▌        | 55/363 [00:01<00:06, 46.96it/s]
Loading 0:  17%|█▋        | 60/363 [00:01<00:06, 46.74it/s]
Loading 0:  18%|█▊        | 65/363 [00:01<00:10, 29.76it/s]
Loading 0:  20%|█▉        | 72/363 [00:01<00:07, 36.87it/s]
Loading 0:  21%|██▏       | 78/363 [00:01<00:07, 38.34it/s]
Loading 0:  23%|██▎       | 83/363 [00:01<00:07, 39.60it/s]
Loading 0:  25%|██▍       | 90/363 [00:02<00:06, 45.02it/s]
Loading 0:  26%|██▋       | 96/363 [00:02<00:06, 43.20it/s]
Loading 0:  28%|██▊       | 101/363 [00:02<00:06, 41.86it/s]
Loading 0:  30%|███       | 109/363 [00:02<00:05, 50.26it/s]
Loading 0:  32%|███▏      | 115/363 [00:02<00:05, 46.79it/s]
Loading 0:  33%|███▎      | 120/363 [00:02<00:05, 43.33it/s]
Loading 0:  35%|███▍      | 126/363 [00:02<00:05, 45.81it/s]
Loading 0:  36%|███▋      | 132/363 [00:03<00:05, 44.89it/s]
Loading 0:  38%|███▊      | 137/363 [00:03<00:05, 43.84it/s]
Loading 0:  39%|███▉      | 142/363 [00:03<00:06, 34.07it/s]
Loading 0:  40%|████      | 146/363 [00:03<00:06, 34.70it/s]
Loading 0:  41%|████▏     | 150/363 [00:03<00:06, 33.81it/s]
Loading 0:  42%|████▏     | 154/363 [00:03<00:05, 34.93it/s]
Loading 0:  44%|████▎     | 158/363 [00:03<00:06, 31.80it/s]
Loading 0:  45%|████▌     | 165/363 [00:03<00:04, 40.61it/s]
Loading 0:  47%|████▋     | 170/363 [00:04<00:04, 41.34it/s]
Loading 0:  48%|████▊     | 175/363 [00:04<00:04, 42.19it/s]
Loading 0:  50%|████▉     | 181/363 [00:04<00:04, 41.46it/s]
Loading 0:  51%|█████     | 186/363 [00:04<00:04, 41.48it/s]
Loading 0:  53%|█████▎    | 193/363 [00:04<00:03, 46.25it/s]
Loading 0:  55%|█████▍    | 198/363 [00:04<00:03, 46.69it/s]
Loading 0:  56%|█████▌    | 203/363 [00:04<00:04, 39.24it/s]
Loading 0:  58%|█████▊    | 210/363 [00:05<00:03, 45.87it/s]
Loading 0:  59%|█████▉    | 215/363 [00:05<00:03, 45.96it/s]
Loading 0:  61%|██████    | 222/363 [00:05<00:03, 45.67it/s]
Loading 0:  63%|██████▎   | 227/363 [00:05<00:04, 32.88it/s]
Loading 0:  64%|██████▎   | 231/363 [00:05<00:03, 33.02it/s]
Loading 0:  65%|██████▌   | 237/363 [00:05<00:03, 38.25it/s]
Loading 0:  67%|██████▋   | 242/363 [00:05<00:03, 40.17it/s]
Loading 0:  68%|██████▊   | 247/363 [00:05<00:02, 42.29it/s]
Loading 0:  70%|██████▉   | 253/363 [00:06<00:02, 41.96it/s]
Loading 0:  71%|███████   | 258/363 [00:06<00:02, 41.98it/s]
Loading 0:  73%|███████▎  | 265/363 [00:06<00:02, 47.18it/s]
Loading 0:  75%|███████▍  | 271/363 [00:06<00:02, 45.21it/s]
Loading 0:  76%|███████▌  | 276/363 [00:06<00:01, 43.65it/s]
Loading 0:  78%|███████▊  | 283/363 [00:06<00:01, 48.14it/s]
Loading 0:  80%|███████▉  | 289/363 [00:06<00:01, 46.89it/s]
Loading 0:  81%|████████  | 294/363 [00:06<00:01, 45.56it/s]
Loading 0:  82%|████████▏ | 299/363 [00:07<00:01, 45.09it/s]
Loading 0:  84%|████████▎ | 304/363 [00:13<00:23,  2.52it/s]
Loading 0:  85%|████████▍ | 308/363 [00:14<00:16,  3.25it/s]
Loading 0:  86%|████████▌ | 312/363 [00:14<00:12,  4.25it/s]
Loading 0:  88%|████████▊ | 320/363 [00:14<00:06,  7.09it/s]
Loading 0:  90%|████████▉ | 326/363 [00:14<00:03,  9.65it/s]
Loading 0:  91%|█████████ | 331/363 [00:14<00:02, 12.29it/s]
Loading 0:  93%|█████████▎| 338/363 [00:14<00:01, 17.25it/s]
Loading 0:  95%|█████████▍| 344/363 [00:14<00:00, 21.28it/s]
Loading 0:  96%|█████████▋| 350/363 [00:14<00:00, 26.00it/s]
Loading 0:  98%|█████████▊| 356/363 [00:14<00:00, 31.17it/s]
Loading 0: 100%|█████████▉| 362/363 [00:15<00:00, 34.11it/s]
                                                            
Job white-bird-1-sft-label1-200-v2-mkmlizer completed after 94.1s with status: succeeded
Stopping job with name white-bird-1-sft-label1-200-v2-mkmlizer
Pipeline stage MKMLizer completed in 94.90s
run pipeline stage %s
Running pipeline stage MKMLTemplater
Pipeline stage MKMLTemplater completed in 0.17s
run pipeline stage %s
Running pipeline stage MKMLDeployer
Creating inference service white-bird-1-sft-label1-200-v2
Waiting for inference service white-bird-1-sft-label1-200-v2 to be ready
Failed to get response for submission nischaydnk-exp9dpo-chai_63105_v2: HTTPConnectionPool(host='nischaydnk-exp9dpo-chai-63105-v2-predictor.tenant-chaiml-guanaco.k.chaiverse.com', port=80): Read timed out. (read timeout=12.0)
Inference service white-bird-1-sft-label1-200-v2 ready after 140.48774552345276s
Pipeline stage MKMLDeployer completed in 141.04s
run pipeline stage %s
Running pipeline stage StressChecker
Received healthy response to inference request in 2.5544204711914062s
Received healthy response to inference request in 1.3913602828979492s
Failed to get response for submission nischaydnk-exp9dpo-chai_63105_v2: HTTPConnectionPool(host='nischaydnk-exp9dpo-chai-63105-v2-predictor.tenant-chaiml-guanaco.k.chaiverse.com', port=80): Read timed out. (read timeout=12.0)
Received healthy response to inference request in 1.4748904705047607s
Received healthy response to inference request in 1.3120510578155518s
Received healthy response to inference request in 1.4687063694000244s
5 requests
0 failed requests
5th percentile: 1.3279129028320313
10th percentile: 1.3437747478485107
20th percentile: 1.3754984378814696
30th percentile: 1.4068295001983642
40th percentile: 1.4377679347991943
50th percentile: 1.4687063694000244
60th percentile: 1.471180009841919
70th percentile: 1.4736536502838136
80th percentile: 1.69079647064209
90th percentile: 2.1226084709167483
95th percentile: 2.338514471054077
99th percentile: 2.5112392711639404
mean time: 1.6402857303619385
Pipeline stage StressChecker completed in 9.84s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyTriggerPipeline
run_pipeline:run_in_cloud %s
starting trigger_guanaco_pipeline args=%s
triggered trigger_guanaco_pipeline args=%s
Pipeline stage OfflineFamilyFriendlyTriggerPipeline completed in 0.90s
run pipeline stage %s
Running pipeline stage TriggerMKMLProfilingPipeline
run_pipeline:run_in_cloud %s
starting trigger_guanaco_pipeline args=%s
triggered trigger_guanaco_pipeline args=%s
Pipeline stage TriggerMKMLProfilingPipeline completed in 0.64s
Shutdown handler de-registered
white-bird-1-sft-label1-200_v2 status is now deployed due to DeploymentManager action
Shutdown handler registered
run pipeline %s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyScorer
Evaluating %s Family Friendly Score with %s threads
%s, retrying in %s seconds...
Evaluating %s Family Friendly Score with %s threads
Pipeline stage OfflineFamilyFriendlyScorer completed in 3863.37s
Shutdown handler de-registered
white-bird-1-sft-label1-200_v2 status is now inactive due to auto deactivation removed underperforming models
white-bird-1-sft-label1-200_v2 status is now torndown due to DeploymentManager action