chaiml-albert-dpo-0912-v_9673

developer_uid: Jellywibble
submission_id: chaiml-albert-dpo-0912-v_9673_v3
model_name: asd
model_group: ChaiML/albert_dpo_0912_v
status: torndown
timestamp: 2024-11-05T03:09:40+00:00
num_battles: 10209
num_wins: 5123
celo_rating: 1253.45
family_friendly_score: 0.5546
family_friendly_standard_error: 0.007028781402206217
submission_type: basic
model_repo: ChaiML/albert_dpo_0912_virgo_edit_variety_ai_disagree_with_user
model_architecture: MistralForCausalLM
model_num_parameters: 12772070400.0
best_of: 8
max_input_tokens: 1024
max_output_tokens: 64
display_name: asd
is_internal_developer: True
language_model: ChaiML/albert_dpo_0912_virgo_edit_variety_ai_disagree_with_user
model_size: 13B
ranking_group: single
us_pacific_date: 2024-11-04
win_ratio: 0.5018121265550005
generation_params: {'temperature': 0.9, 'top_p': 1.0, 'min_p': 0.05, 'top_k': 80, 'presence_penalty': 0.0, 'frequency_penalty': 0.0, 'stopping_words': ['\n', '</s>', '###', 'Bot:', 'User:', 'You:', '<|im_end|>'], 'max_input_tokens': 1024, 'best_of': 8, 'max_output_tokens': 64}
formatter: {'memory_template': '', 'prompt_template': '', 'bot_template': '{bot_name}: {message}\n', 'user_template': '{user_name}: {message}\n', 'response_template': '{bot_name}:', 'truncate_by_message': False}
Resubmit model
Shutdown handler not registered because Python interpreter is not running in the main thread
run pipeline %s
run pipeline stage %s
Running pipeline stage MKMLizer
Starting job with name chaiml-albert-dpo-0912-v-9673-v3-mkmlizer
Waiting for job on chaiml-albert-dpo-0912-v-9673-v3-mkmlizer to finish
chaiml-albert-dpo-0912-v-9673-v3-mkmlizer: ╔═════════════════════════════════════════════════════════════════════╗
chaiml-albert-dpo-0912-v-9673-v3-mkmlizer: ║     _____            __           __                                ║
chaiml-albert-dpo-0912-v-9673-v3-mkmlizer: ║    / _/ /_ ___    __/ /  ___ ___ / /                                ║
chaiml-albert-dpo-0912-v-9673-v3-mkmlizer: ║   / _/ / // / |/|/ / _ \/ -_) -_) /                                 ║
chaiml-albert-dpo-0912-v-9673-v3-mkmlizer: ║  /_//_/\_, /|__,__/_//_/\__/\__/_/                                  ║
chaiml-albert-dpo-0912-v-9673-v3-mkmlizer: ║       /___/                                                         ║
chaiml-albert-dpo-0912-v-9673-v3-mkmlizer: ║                                                                     ║
chaiml-albert-dpo-0912-v-9673-v3-mkmlizer: ║  Version: 0.11.12                                                   ║
chaiml-albert-dpo-0912-v-9673-v3-mkmlizer: ║  Copyright 2023 MK ONE TECHNOLOGIES Inc.                            ║
chaiml-albert-dpo-0912-v-9673-v3-mkmlizer: ║  https://mk1.ai                                                     ║
chaiml-albert-dpo-0912-v-9673-v3-mkmlizer: ║                                                                     ║
chaiml-albert-dpo-0912-v-9673-v3-mkmlizer: ║  The license key for the current software has been verified as      ║
chaiml-albert-dpo-0912-v-9673-v3-mkmlizer: ║  belonging to:                                                      ║
chaiml-albert-dpo-0912-v-9673-v3-mkmlizer: ║                                                                     ║
chaiml-albert-dpo-0912-v-9673-v3-mkmlizer: ║  Chai Research Corp.                                                ║
chaiml-albert-dpo-0912-v-9673-v3-mkmlizer: ║  Account ID: 7997a29f-0ceb-4cc7-9adf-840c57b4ae6f                   ║
chaiml-albert-dpo-0912-v-9673-v3-mkmlizer: ║  Expiration: 2025-01-15 23:59:59                                    ║
chaiml-albert-dpo-0912-v-9673-v3-mkmlizer: ║                                                                     ║
chaiml-albert-dpo-0912-v-9673-v3-mkmlizer: ╚═════════════════════════════════════════════════════════════════════╝
chaiml-albert-dpo-0912-v-9673-v3-mkmlizer: Downloaded to shared memory in 243.254s
chaiml-albert-dpo-0912-v-9673-v3-mkmlizer: quantizing model to /dev/shm/model_cache, profile:s0, folder:/tmp/tmp28ktwywv, device:0
chaiml-albert-dpo-0912-v-9673-v3-mkmlizer: Saving flywheel model at /dev/shm/model_cache
chaiml-albert-dpo-0912-v-9673-v3-mkmlizer: quantized model in 41.329s
chaiml-albert-dpo-0912-v-9673-v3-mkmlizer: Processed model ChaiML/albert_dpo_0912_virgo_edit_variety_ai_disagree_with_user in 284.584s
chaiml-albert-dpo-0912-v-9673-v3-mkmlizer: creating bucket guanaco-mkml-models
chaiml-albert-dpo-0912-v-9673-v3-mkmlizer: Bucket 's3://guanaco-mkml-models/' created
chaiml-albert-dpo-0912-v-9673-v3-mkmlizer: uploading /dev/shm/model_cache to s3://guanaco-mkml-models/chaiml-albert-dpo-0912-v-9673-v3
chaiml-albert-dpo-0912-v-9673-v3-mkmlizer: cp /dev/shm/model_cache/config.json s3://guanaco-mkml-models/chaiml-albert-dpo-0912-v-9673-v3/config.json
chaiml-albert-dpo-0912-v-9673-v3-mkmlizer: cp /dev/shm/model_cache/special_tokens_map.json s3://guanaco-mkml-models/chaiml-albert-dpo-0912-v-9673-v3/special_tokens_map.json
chaiml-albert-dpo-0912-v-9673-v3-mkmlizer: cp /dev/shm/model_cache/tokenizer_config.json s3://guanaco-mkml-models/chaiml-albert-dpo-0912-v-9673-v3/tokenizer_config.json
chaiml-albert-dpo-0912-v-9673-v3-mkmlizer: cp /dev/shm/model_cache/tokenizer.json s3://guanaco-mkml-models/chaiml-albert-dpo-0912-v-9673-v3/tokenizer.json
chaiml-albert-dpo-0912-v-9673-v3-mkmlizer: cp /dev/shm/model_cache/flywheel_model.0.safetensors s3://guanaco-mkml-models/chaiml-albert-dpo-0912-v-9673-v3/flywheel_model.0.safetensors
chaiml-albert-dpo-0912-v-9673-v3-mkmlizer: 
Loading 0:   0%|          | 0/363 [00:00<?, ?it/s]
Loading 0:   1%|▏         | 5/363 [00:00<00:15, 22.81it/s]
Loading 0:   3%|▎         | 10/363 [00:00<00:12, 28.54it/s]
Loading 0:   4%|▍         | 14/363 [00:00<00:13, 25.60it/s]
Loading 0:   6%|▌         | 20/363 [00:00<00:09, 34.53it/s]
Loading 0:   7%|▋         | 24/363 [00:00<00:14, 23.76it/s]
Loading 0:   7%|▋         | 27/363 [00:01<00:14, 22.97it/s]
Loading 0:   9%|▉         | 32/363 [00:01<00:13, 23.74it/s]
Loading 0:  11%|█         | 39/363 [00:01<00:10, 30.53it/s]
Loading 0:  12%|█▏        | 43/363 [00:01<00:10, 29.71it/s]
Loading 0:  13%|█▎        | 48/363 [00:01<00:09, 31.90it/s]
Loading 0:  14%|█▍        | 52/363 [00:01<00:10, 30.55it/s]
Loading 0:  15%|█▌        | 56/363 [00:01<00:10, 30.21it/s]
Loading 0:  17%|█▋        | 61/363 [00:02<00:11, 25.60it/s]
Loading 0:  18%|█▊        | 64/363 [00:02<00:13, 22.33it/s]
Loading 0:  20%|█▉        | 71/363 [00:02<00:10, 28.94it/s]
Loading 0:  21%|██        | 75/363 [00:02<00:10, 28.63it/s]
Loading 0:  22%|██▏       | 79/363 [00:02<00:10, 28.15it/s]
Loading 0:  23%|██▎       | 84/363 [00:03<00:09, 30.47it/s]
Loading 0:  24%|██▍       | 88/363 [00:03<00:09, 29.70it/s]
Loading 0:  26%|██▌       | 93/363 [00:03<00:08, 31.29it/s]
Loading 0:  27%|██▋       | 97/363 [00:03<00:09, 29.22it/s]
Loading 0:  28%|██▊       | 101/363 [00:03<00:10, 23.88it/s]
Loading 0:  29%|██▊       | 104/363 [00:03<00:12, 21.51it/s]
Loading 0:  31%|███       | 111/363 [00:04<00:08, 28.57it/s]
Loading 0:  32%|███▏      | 115/363 [00:04<00:08, 28.48it/s]
Loading 0:  33%|███▎      | 120/363 [00:04<00:08, 30.34it/s]
Loading 0:  34%|███▍      | 124/363 [00:04<00:08, 29.30it/s]
Loading 0:  36%|███▌      | 129/363 [00:04<00:07, 31.66it/s]
Loading 0:  37%|███▋      | 133/363 [00:04<00:07, 30.04it/s]
Loading 0:  38%|███▊      | 137/363 [00:04<00:07, 29.87it/s]
Loading 0:  39%|███▉      | 142/363 [00:05<00:08, 25.97it/s]
Loading 0:  40%|███▉      | 145/363 [00:05<00:08, 24.66it/s]
Loading 0:  41%|████      | 149/363 [00:05<00:09, 23.29it/s]
Loading 0:  43%|████▎     | 156/363 [00:05<00:06, 29.96it/s]
Loading 0:  44%|████▍     | 160/363 [00:05<00:06, 29.16it/s]
Loading 0:  45%|████▌     | 165/363 [00:05<00:06, 31.54it/s]
Loading 0:  47%|████▋     | 169/363 [00:06<00:06, 30.20it/s]
Loading 0:  48%|████▊     | 174/363 [00:06<00:05, 32.24it/s]
Loading 0:  49%|████▉     | 178/363 [00:06<00:06, 30.66it/s]
Loading 0:  50%|█████     | 182/363 [00:06<00:07, 24.73it/s]
Loading 0:  51%|█████     | 185/363 [00:06<00:08, 21.92it/s]
Loading 0:  52%|█████▏    | 190/363 [00:06<00:06, 26.61it/s]
Loading 0:  53%|█████▎    | 194/363 [00:07<00:07, 24.14it/s]
Loading 0:  55%|█████▌    | 201/363 [00:07<00:05, 29.85it/s]
Loading 0:  56%|█████▋    | 205/363 [00:07<00:05, 27.67it/s]
Loading 0:  58%|█████▊    | 210/363 [00:07<00:05, 30.15it/s]
Loading 0:  59%|█████▉    | 214/363 [00:07<00:05, 28.97it/s]
Loading 0:  60%|██████    | 218/363 [00:07<00:04, 29.45it/s]
Loading 0:  61%|██████▏   | 223/363 [00:08<00:05, 25.27it/s]
Loading 0:  62%|██████▏   | 226/363 [00:08<00:05, 23.86it/s]
Loading 0:  63%|██████▎   | 230/363 [00:08<00:05, 22.46it/s]
Loading 0:  65%|██████▍   | 235/363 [00:08<00:04, 27.50it/s]
Loading 0:  66%|██████▌   | 239/363 [00:08<00:05, 24.73it/s]
Loading 0:  68%|██████▊   | 246/363 [00:08<00:03, 31.33it/s]
Loading 0:  69%|██████▉   | 250/363 [00:09<00:03, 30.37it/s]
Loading 0:  70%|███████   | 255/363 [00:09<00:03, 32.41it/s]
Loading 0:  71%|███████▏  | 259/363 [00:09<00:03, 30.56it/s]
Loading 0:  72%|███████▏  | 263/363 [00:09<00:04, 23.23it/s]
Loading 0:  73%|███████▎  | 266/363 [00:09<00:04, 21.18it/s]
Loading 0:  75%|███████▌  | 273/363 [00:09<00:03, 28.05it/s]
Loading 0:  76%|███████▋  | 277/363 [00:10<00:03, 27.88it/s]
Loading 0:  78%|███████▊  | 282/363 [00:10<00:02, 30.65it/s]
Loading 0:  79%|███████▉  | 286/363 [00:10<00:02, 28.65it/s]
Loading 0:  80%|████████  | 291/363 [00:10<00:02, 31.09it/s]
Loading 0:  81%|████████▏ | 295/363 [00:10<00:02, 29.84it/s]
Loading 0:  82%|████████▏ | 299/363 [00:10<00:02, 29.73it/s]
Loading 0:  84%|████████▎ | 304/363 [00:10<00:02, 26.07it/s]
Loading 0:  85%|████████▍ | 307/363 [00:11<00:02, 24.13it/s]
Loading 0:  86%|████████▌ | 311/363 [00:11<00:02, 23.32it/s]
Loading 0:  88%|████████▊ | 318/363 [00:11<00:01, 30.22it/s]
Loading 0:  89%|████████▊ | 322/363 [00:11<00:01, 29.63it/s]
Loading 0:  90%|█████████ | 327/363 [00:11<00:01, 31.67it/s]
Loading 0:  91%|█████████ | 331/363 [00:11<00:01, 30.27it/s]
Loading 0:  92%|█████████▏| 335/363 [00:12<00:00, 32.15it/s]
Loading 0:  93%|█████████▎| 339/363 [00:12<00:00, 28.15it/s]
Loading 0:  94%|█████████▍| 343/363 [00:12<00:00, 30.68it/s]
Loading 0:  96%|█████████▌| 347/363 [00:19<00:08,  1.90it/s]
Loading 0:  96%|█████████▋| 350/363 [00:19<00:05,  2.43it/s]
Loading 0:  97%|█████████▋| 353/363 [00:19<00:03,  3.14it/s]
Loading 0:  98%|█████████▊| 357/363 [00:19<00:01,  4.35it/s]
                                                            
Job chaiml-albert-dpo-0912-v-9673-v3-mkmlizer completed after 308.05s with status: succeeded
Stopping job with name chaiml-albert-dpo-0912-v-9673-v3-mkmlizer
Pipeline stage MKMLizer completed in 308.58s
run pipeline stage %s
Running pipeline stage MKMLTemplater
Pipeline stage MKMLTemplater completed in 0.17s
run pipeline stage %s
Running pipeline stage MKMLDeployer
Creating inference service chaiml-albert-dpo-0912-v-9673-v3
Waiting for inference service chaiml-albert-dpo-0912-v-9673-v3 to be ready
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Inference service chaiml-albert-dpo-0912-v-9673-v3 ready after 160.5779767036438s
Pipeline stage MKMLDeployer completed in 161.37s
run pipeline stage %s
Running pipeline stage StressChecker
Received healthy response to inference request in 1.7291786670684814s
Received healthy response to inference request in 1.025944709777832s
Received healthy response to inference request in 0.9440450668334961s
Received healthy response to inference request in 1.4293222427368164s
Received healthy response to inference request in 1.3427588939666748s
5 requests
0 failed requests
5th percentile: 0.9604249954223633
10th percentile: 0.9768049240112304
20th percentile: 1.0095647811889648
30th percentile: 1.0893075466156006
40th percentile: 1.2160332202911377
50th percentile: 1.3427588939666748
60th percentile: 1.3773842334747315
70th percentile: 1.4120095729827882
80th percentile: 1.4892935276031494
90th percentile: 1.6092360973358155
95th percentile: 1.6692073822021485
99th percentile: 1.717184410095215
mean time: 1.29424991607666
Pipeline stage StressChecker completed in 7.81s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyTriggerPipeline
run_pipeline:run_in_cloud %s
starting trigger_guanaco_pipeline args=%s
triggered trigger_guanaco_pipeline args=%s
Pipeline stage OfflineFamilyFriendlyTriggerPipeline completed in 2.39s
Shutdown handler de-registered
chaiml-albert-dpo-0912-v_9673_v3 status is now deployed due to DeploymentManager action
Shutdown handler registered
run pipeline %s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyScorer
Evaluating %s Family Friendly Score with %s threads
Pipeline stage OfflineFamilyFriendlyScorer completed in 2477.96s
Shutdown handler de-registered
chaiml-albert-dpo-0912-v_9673_v3 status is now inactive due to auto deactivation removed underperforming models
chaiml-albert-dpo-0912-v_9673_v3 status is now torndown due to DeploymentManager action