trace2333-dpo-v9-v2-reprompt

developer_uid: Trace2333
submission_id: trace2333-dpo-v9-v2-reprompt_v4
model_name: trace2333-dpo-v9-v2-reprompt_v4
model_group: Trace2333/dpo_v9_v2_repr
status: torndown
timestamp: 2024-08-27T03:52:18+00:00
num_battles: 10905
num_wins: 5711
celo_rating: 1254.8
family_friendly_score: 0.0
submission_type: basic
model_repo: Trace2333/dpo_v9_v2_reprompt
model_architecture: LlamaForCausalLM
model_num_parameters: 8030261248.0
best_of: 16
max_input_tokens: 512
max_output_tokens: 64
display_name: trace2333-dpo-v9-v2-reprompt_v4
is_internal_developer: False
language_model: Trace2333/dpo_v9_v2_reprompt
model_size: 8B
ranking_group: single
us_pacific_date: 2024-08-26
win_ratio: 0.52370472260431
generation_params: {'temperature': 1.0, 'top_p': 1.0, 'min_p': 0.0, 'top_k': 40, 'presence_penalty': 0.0, 'frequency_penalty': 0.0, 'stopping_words': ['\n', '<|eot_id|>'], 'max_input_tokens': 512, 'best_of': 16, 'max_output_tokens': 64}
formatter: {'memory_template': "{bot_name}'s Persona: {memory}\n####\n", 'prompt_template': '{prompt}\n<START>\n', 'bot_template': '{bot_name}: {message}\n', 'user_template': '{user_name}: {message}\n', 'response_template': '{bot_name}:', 'truncate_by_message': False}
Resubmit model
Running pipeline stage MKMLizer
Starting job with name trace2333-dpo-v9-v2-reprompt-v4-mkmlizer
Waiting for job on trace2333-dpo-v9-v2-reprompt-v4-mkmlizer to finish
Stopping job with name trace2333-dpo-v9-v2-reprompt-v4-mkmlizer
%s, retrying in %s seconds...
Starting job with name trace2333-dpo-v9-v2-reprompt-v4-mkmlizer
Waiting for job on trace2333-dpo-v9-v2-reprompt-v4-mkmlizer to finish
trace2333-dpo-v9-v2-reprompt-v4-mkmlizer: ╔═════════════════════════════════════════════════════════════════════╗
trace2333-dpo-v9-v2-reprompt-v4-mkmlizer: ║     _____            __           __                                ║
trace2333-dpo-v9-v2-reprompt-v4-mkmlizer: ║    / _/ /_ ___    __/ /  ___ ___ / /                                ║
trace2333-dpo-v9-v2-reprompt-v4-mkmlizer: ║   / _/ / // / |/|/ / _ \/ -_) -_) /                                 ║
trace2333-dpo-v9-v2-reprompt-v4-mkmlizer: ║  /_//_/\_, /|__,__/_//_/\__/\__/_/                                  ║
trace2333-dpo-v9-v2-reprompt-v4-mkmlizer: ║       /___/                                                         ║
trace2333-dpo-v9-v2-reprompt-v4-mkmlizer: ║                                                                     ║
trace2333-dpo-v9-v2-reprompt-v4-mkmlizer: ║  Version: 0.10.1                                                    ║
trace2333-dpo-v9-v2-reprompt-v4-mkmlizer: ║  Copyright 2023 MK ONE TECHNOLOGIES Inc.                            ║
trace2333-dpo-v9-v2-reprompt-v4-mkmlizer: ║  https://mk1.ai                                                     ║
trace2333-dpo-v9-v2-reprompt-v4-mkmlizer: ║                                                                     ║
trace2333-dpo-v9-v2-reprompt-v4-mkmlizer: ║  The license key for the current software has been verified as      ║
trace2333-dpo-v9-v2-reprompt-v4-mkmlizer: ║  belonging to:                                                      ║
trace2333-dpo-v9-v2-reprompt-v4-mkmlizer: ║                                                                     ║
trace2333-dpo-v9-v2-reprompt-v4-mkmlizer: ║  Chai Research Corp.                                                ║
trace2333-dpo-v9-v2-reprompt-v4-mkmlizer: ║  Account ID: 7997a29f-0ceb-4cc7-9adf-840c57b4ae6f                   ║
trace2333-dpo-v9-v2-reprompt-v4-mkmlizer: ║  Expiration: 2024-10-15 23:59:59                                    ║
trace2333-dpo-v9-v2-reprompt-v4-mkmlizer: ║                                                                     ║
trace2333-dpo-v9-v2-reprompt-v4-mkmlizer: ╚═════════════════════════════════════════════════════════════════════╝
trace2333-dpo-v9-v2-reprompt-v4-mkmlizer: Downloaded to shared memory in 72.931s
trace2333-dpo-v9-v2-reprompt-v4-mkmlizer: quantizing model to /dev/shm/model_cache, profile:s0, folder:/tmp/tmpck0_u4yb, device:0
trace2333-dpo-v9-v2-reprompt-v4-mkmlizer: Saving flywheel model at /dev/shm/model_cache
trace2333-dpo-v9-v2-reprompt-v4-mkmlizer: creating bucket guanaco-mkml-models
trace2333-dpo-v9-v2-reprompt-v4-mkmlizer: Bucket 's3://guanaco-mkml-models/' created
trace2333-dpo-v9-v2-reprompt-v4-mkmlizer: uploading /dev/shm/model_cache to s3://guanaco-mkml-models/trace2333-dpo-v9-v2-reprompt-v4
trace2333-dpo-v9-v2-reprompt-v4-mkmlizer: cp /dev/shm/model_cache/config.json s3://guanaco-mkml-models/trace2333-dpo-v9-v2-reprompt-v4/config.json
trace2333-dpo-v9-v2-reprompt-v4-mkmlizer: cp /dev/shm/model_cache/special_tokens_map.json s3://guanaco-mkml-models/trace2333-dpo-v9-v2-reprompt-v4/special_tokens_map.json
trace2333-dpo-v9-v2-reprompt-v4-mkmlizer: cp /dev/shm/model_cache/tokenizer_config.json s3://guanaco-mkml-models/trace2333-dpo-v9-v2-reprompt-v4/tokenizer_config.json
trace2333-dpo-v9-v2-reprompt-v4-mkmlizer: cp /dev/shm/model_cache/tokenizer.json s3://guanaco-mkml-models/trace2333-dpo-v9-v2-reprompt-v4/tokenizer.json
trace2333-dpo-v9-v2-reprompt-v4-mkmlizer: cp /dev/shm/model_cache/flywheel_model.0.safetensors s3://guanaco-mkml-models/trace2333-dpo-v9-v2-reprompt-v4/flywheel_model.0.safetensors
trace2333-dpo-v9-v2-reprompt-v4-mkmlizer: 
Loading 0:   0%|          | 0/291 [00:00<?, ?it/s]
Loading 0:   2%|▏         | 5/291 [00:00<00:11, 24.76it/s]
Loading 0:   4%|▍         | 12/291 [00:00<00:07, 35.10it/s]
Loading 0:   5%|▌         | 16/291 [00:00<00:08, 32.74it/s]
Loading 0:   7%|▋         | 21/291 [00:00<00:07, 36.16it/s]
Loading 0:   9%|▊         | 25/291 [00:00<00:07, 34.32it/s]
Loading 0:  11%|█         | 32/291 [00:00<00:06, 39.13it/s]
Loading 0:  12%|█▏        | 36/291 [00:01<00:10, 24.54it/s]
Loading 0:  14%|█▍        | 41/291 [00:01<00:09, 26.00it/s]
Loading 0:  16%|█▋        | 48/291 [00:01<00:07, 33.06it/s]
Loading 0:  18%|█▊        | 52/291 [00:01<00:07, 32.32it/s]
Loading 0:  20%|█▉        | 57/291 [00:01<00:06, 35.67it/s]
Loading 0:  21%|██        | 61/291 [00:01<00:06, 34.32it/s]
Loading 0:  23%|██▎       | 66/291 [00:01<00:05, 37.62it/s]
Loading 0:  24%|██▍       | 71/291 [00:02<00:05, 38.08it/s]
Loading 0:  26%|██▌       | 75/291 [00:02<00:06, 32.19it/s]
Loading 0:  27%|██▋       | 80/291 [00:02<00:07, 28.37it/s]
Loading 0:  29%|██▉       | 84/291 [00:02<00:07, 26.39it/s]
Loading 0:  31%|███       | 90/291 [00:02<00:06, 31.72it/s]
Loading 0:  32%|███▏      | 94/291 [00:02<00:06, 31.02it/s]
Loading 0:  34%|███▍      | 99/291 [00:03<00:05, 34.63it/s]
Loading 0:  35%|███▌      | 103/291 [00:03<00:05, 33.20it/s]
Loading 0:  37%|███▋      | 108/291 [00:03<00:05, 36.23it/s]
Loading 0:  38%|███▊      | 112/291 [00:03<00:05, 35.07it/s]
Loading 0:  40%|███▉      | 116/291 [00:03<00:04, 35.18it/s]
Loading 0:  42%|████▏     | 122/291 [00:03<00:04, 39.80it/s]
Loading 0:  44%|████▎     | 127/291 [00:03<00:04, 37.18it/s]
Loading 0:  46%|████▌     | 133/291 [00:04<00:04, 32.30it/s]
Loading 0:  47%|████▋     | 137/291 [00:04<00:04, 31.35it/s]
Loading 0:  48%|████▊     | 141/291 [00:04<00:05, 28.81it/s]
Loading 0:  51%|█████     | 147/291 [00:04<00:04, 33.82it/s]
Loading 0:  52%|█████▏    | 151/291 [00:04<00:04, 32.88it/s]
Loading 0:  54%|█████▎    | 156/291 [00:04<00:03, 35.91it/s]
Loading 0:  55%|█████▍    | 160/291 [00:04<00:03, 33.55it/s]
Loading 0:  57%|█████▋    | 165/291 [00:04<00:03, 35.89it/s]
Loading 0:  58%|█████▊    | 169/291 [00:05<00:03, 33.95it/s]
Loading 0:  60%|█████▉    | 174/291 [00:05<00:03, 37.11it/s]
Loading 0:  61%|██████    | 178/291 [00:05<00:03, 34.44it/s]
Loading 0:  63%|██████▎   | 184/291 [00:05<00:02, 40.20it/s]
Loading 0:  65%|██████▍   | 189/291 [00:05<00:04, 25.09it/s]
Loading 0:  67%|██████▋   | 194/291 [00:06<00:03, 26.31it/s]
Loading 0:  69%|██████▉   | 201/291 [00:06<00:02, 32.79it/s]
Loading 0:  70%|███████   | 205/291 [00:06<00:02, 32.35it/s]
Loading 0:  72%|███████▏  | 210/291 [00:06<00:02, 35.72it/s]
Loading 0:  74%|███████▎  | 214/291 [00:06<00:02, 34.79it/s]
Loading 0:  75%|███████▌  | 219/291 [00:06<00:01, 37.10it/s]
Loading 0:  77%|███████▋  | 223/291 [00:06<00:01, 34.34it/s]
Loading 0:  78%|███████▊  | 227/291 [00:06<00:01, 34.15it/s]
Loading 0:  79%|███████▉  | 231/291 [00:07<00:02, 29.63it/s]
Loading 0:  81%|████████  | 235/291 [00:07<00:02, 22.63it/s]
Loading 0:  82%|████████▏ | 239/291 [00:07<00:02, 22.73it/s]
Loading 0:  85%|████████▍ | 246/291 [00:07<00:01, 29.02it/s]
Loading 0:  86%|████████▌ | 250/291 [00:07<00:01, 28.49it/s]
Loading 0:  88%|████████▊ | 255/291 [00:07<00:01, 31.02it/s]
Loading 0:  89%|████████▉ | 259/291 [00:08<00:01, 30.44it/s]
Loading 0:  91%|█████████ | 264/291 [00:08<00:00, 33.11it/s]
Loading 0:  92%|█████████▏| 268/291 [00:08<00:00, 31.76it/s]
Loading 0:  94%|█████████▍| 273/291 [00:08<00:00, 33.23it/s]
Loading 0:  95%|█████████▌| 277/291 [00:08<00:00, 31.79it/s]
Loading 0:  97%|█████████▋| 281/291 [00:08<00:00, 32.81it/s]
Loading 0:  98%|█████████▊| 286/291 [00:14<00:01,  2.59it/s]
Loading 0:  99%|█████████▉| 289/291 [00:14<00:00,  3.21it/s]
                                                            
Job trace2333-dpo-v9-v2-reprompt-v4-mkmlizer completed after 118.95s with status: succeeded
Stopping job with name trace2333-dpo-v9-v2-reprompt-v4-mkmlizer
Pipeline stage MKMLizer completed in 121.75s
Running pipeline stage MKMLKubeTemplater
Pipeline stage MKMLKubeTemplater completed in 0.13s
Running pipeline stage ISVCDeployer
Creating inference service trace2333-dpo-v9-v2-reprompt-v4
Waiting for inference service trace2333-dpo-v9-v2-reprompt-v4 to be ready
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Inference service trace2333-dpo-v9-v2-reprompt-v4 ready after 171.86177325248718s
Pipeline stage ISVCDeployer completed in 172.71s
Running pipeline stage StressChecker
Received healthy response to inference request in 2.547621250152588s
Received healthy response to inference request in 2.2602458000183105s
Received healthy response to inference request in 1.8397510051727295s
Received healthy response to inference request in 1.80845308303833s
Received healthy response to inference request in 1.5824244022369385s
5 requests
0 failed requests
5th percentile: 1.6276301383972167
10th percentile: 1.6728358745574952
20th percentile: 1.7632473468780518
30th percentile: 1.8147126674652099
40th percentile: 1.8272318363189697
50th percentile: 1.8397510051727295
60th percentile: 2.0079489231109617
70th percentile: 2.1761468410491944
80th percentile: 2.317720890045166
90th percentile: 2.432671070098877
95th percentile: 2.4901461601257324
99th percentile: 2.536126232147217
mean time: 2.0076991081237794
Pipeline stage StressChecker completed in 10.91s
trace2333-dpo-v9-v2-reprompt_v4 status is now deployed due to DeploymentManager action
trace2333-dpo-v9-v2-reprompt_v4 status is now inactive due to auto deactivation removed underperforming models
trace2333-dpo-v9-v2-reprompt_v4 status is now torndown due to DeploymentManager action