trace2333-dpo-v9-v3-reprompt

developer_uid: Trace2333

submission_id: trace2333-dpo-v9-v3-reprompt_v1

model_name: trace2333-dpo-v9-v3-reprompt_v1

model_group: Trace2333/dpo_v9_v3_repr

status: torndown

timestamp: 2024-08-27T04:12:47+00:00

num_battles: 10296

num_wins: 5372

celo_rating: 1252.73

family_friendly_score: 0.0

submission_type: basic

model_repo: Trace2333/dpo_v9_v3_reprompt

model_architecture: LlamaForCausalLM

model_num_parameters: 8030261248.0

best_of: 16

max_input_tokens: 512

max_output_tokens: 64

display_name: trace2333-dpo-v9-v3-reprompt_v1

is_internal_developer: False

language_model: Trace2333/dpo_v9_v3_reprompt

model_size: 8B

ranking_group: single

us_pacific_date: 2024-08-26

win_ratio: 0.5217560217560218

generation_params: {'temperature': 1.0, 'top_p': 1.0, 'min_p': 0.0, 'top_k': 40, 'presence_penalty': 0.0, 'frequency_penalty': 0.0, 'stopping_words': ['\n', '<|eot_id|>'], 'max_input_tokens': 512, 'best_of': 16, 'max_output_tokens': 64}

formatter: {'memory_template': "{bot_name}'s Persona: {memory}\n####\n", 'prompt_template': '{prompt}\n<START>\n', 'bot_template': '{bot_name}: {message}\n', 'user_template': '{user_name}: {message}\n', 'response_template': '{bot_name}:', 'truncate_by_message': False}

Resubmit model

Running pipeline stage MKMLizer
Starting job with name trace2333-dpo-v9-v3-reprompt-v1-mkmlizer
Waiting for job on trace2333-dpo-v9-v3-reprompt-v1-mkmlizer to finish
Stopping job with name trace2333-dpo-v9-v3-reprompt-v1-mkmlizer
%s, retrying in %s seconds...
Starting job with name trace2333-dpo-v9-v3-reprompt-v1-mkmlizer
Waiting for job on trace2333-dpo-v9-v3-reprompt-v1-mkmlizer to finish
trace2333-dpo-v9-v3-reprompt-v1-mkmlizer: ╔═════════════════════════════════════════════════════════════════════╗
trace2333-dpo-v9-v3-reprompt-v1-mkmlizer: ║     _____            __           __                                ║
trace2333-dpo-v9-v3-reprompt-v1-mkmlizer: ║    / _/ /_ ___    __/ /  ___ ___ / /                                ║
trace2333-dpo-v9-v3-reprompt-v1-mkmlizer: ║   / _/ / // / |/|/ / _ \/ -_) -_) /                                 ║
trace2333-dpo-v9-v3-reprompt-v1-mkmlizer: ║  /_//_/\_, /|__,__/_//_/\__/\__/_/                                  ║
trace2333-dpo-v9-v3-reprompt-v1-mkmlizer: ║       /___/                                                         ║
trace2333-dpo-v9-v3-reprompt-v1-mkmlizer: ║                                                                     ║
trace2333-dpo-v9-v3-reprompt-v1-mkmlizer: ║  Version: 0.10.1                                                    ║
trace2333-dpo-v9-v3-reprompt-v1-mkmlizer: ║  Copyright 2023 MK ONE TECHNOLOGIES Inc.                            ║
trace2333-dpo-v9-v3-reprompt-v1-mkmlizer: ║  https://mk1.ai                                                     ║
trace2333-dpo-v9-v3-reprompt-v1-mkmlizer: ║                                                                     ║
trace2333-dpo-v9-v3-reprompt-v1-mkmlizer: ║  The license key for the current software has been verified as      ║
trace2333-dpo-v9-v3-reprompt-v1-mkmlizer: ║  belonging to:                                                      ║
trace2333-dpo-v9-v3-reprompt-v1-mkmlizer: ║                                                                     ║
trace2333-dpo-v9-v3-reprompt-v1-mkmlizer: ║  Chai Research Corp.                                                ║
trace2333-dpo-v9-v3-reprompt-v1-mkmlizer: ║  Account ID: 7997a29f-0ceb-4cc7-9adf-840c57b4ae6f                   ║
trace2333-dpo-v9-v3-reprompt-v1-mkmlizer: ║  Expiration: 2024-10-15 23:59:59                                    ║
trace2333-dpo-v9-v3-reprompt-v1-mkmlizer: ║                                                                     ║
trace2333-dpo-v9-v3-reprompt-v1-mkmlizer: ╚═════════════════════════════════════════════════════════════════════╝
trace2333-dpo-v9-v3-reprompt-v1-mkmlizer: Downloaded to shared memory in 75.877s
trace2333-dpo-v9-v3-reprompt-v1-mkmlizer: quantizing model to /dev/shm/model_cache, profile:s0, folder:/tmp/tmpdna6pccl, device:0
trace2333-dpo-v9-v3-reprompt-v1-mkmlizer: Saving flywheel model at /dev/shm/model_cache
trace2333-dpo-v9-v3-reprompt-v1-mkmlizer: quantized model in 29.051s
trace2333-dpo-v9-v3-reprompt-v1-mkmlizer: Processed model Trace2333/dpo_v9_v3_reprompt in 104.928s
trace2333-dpo-v9-v3-reprompt-v1-mkmlizer: creating bucket guanaco-mkml-models
trace2333-dpo-v9-v3-reprompt-v1-mkmlizer: Bucket 's3://guanaco-mkml-models/' created
trace2333-dpo-v9-v3-reprompt-v1-mkmlizer: uploading /dev/shm/model_cache to s3://guanaco-mkml-models/trace2333-dpo-v9-v3-reprompt-v1
trace2333-dpo-v9-v3-reprompt-v1-mkmlizer: cp /dev/shm/model_cache/config.json s3://guanaco-mkml-models/trace2333-dpo-v9-v3-reprompt-v1/config.json
trace2333-dpo-v9-v3-reprompt-v1-mkmlizer: cp /dev/shm/model_cache/special_tokens_map.json s3://guanaco-mkml-models/trace2333-dpo-v9-v3-reprompt-v1/special_tokens_map.json
trace2333-dpo-v9-v3-reprompt-v1-mkmlizer: cp /dev/shm/model_cache/tokenizer_config.json s3://guanaco-mkml-models/trace2333-dpo-v9-v3-reprompt-v1/tokenizer_config.json
trace2333-dpo-v9-v3-reprompt-v1-mkmlizer: cp /dev/shm/model_cache/tokenizer.json s3://guanaco-mkml-models/trace2333-dpo-v9-v3-reprompt-v1/tokenizer.json
trace2333-dpo-v9-v3-reprompt-v1-mkmlizer: cp /dev/shm/model_cache/flywheel_model.0.safetensors s3://guanaco-mkml-models/trace2333-dpo-v9-v3-reprompt-v1/flywheel_model.0.safetensors
trace2333-dpo-v9-v3-reprompt-v1-mkmlizer: 
Loading 0:   0%|          | 0/291 [00:00<?, ?it/s]
Loading 0:   2%|▏         | 5/291 [00:00<00:11, 24.95it/s]
Loading 0:   4%|▍         | 12/291 [00:00<00:08, 34.52it/s]
Loading 0:   5%|▌         | 16/291 [00:00<00:08, 31.80it/s]
Loading 0:   7%|▋         | 21/291 [00:00<00:07, 35.13it/s]
Loading 0:   9%|▊         | 25/291 [00:00<00:08, 32.93it/s]
Loading 0:  10%|█         | 30/291 [00:00<00:06, 37.42it/s]
Loading 0:  12%|█▏        | 34/291 [00:01<00:10, 25.30it/s]
Loading 0:  13%|█▎        | 38/291 [00:01<00:09, 26.54it/s]
Loading 0:  14%|█▍        | 42/291 [00:01<00:09, 25.57it/s]
Loading 0:  16%|█▋        | 48/291 [00:01<00:07, 31.47it/s]
Loading 0:  18%|█▊        | 52/291 [00:01<00:07, 30.88it/s]
Loading 0:  20%|█▉        | 57/291 [00:01<00:06, 34.32it/s]
Loading 0:  21%|██        | 61/291 [00:01<00:07, 32.59it/s]
Loading 0:  23%|██▎       | 66/291 [00:02<00:06, 35.46it/s]
Loading 0:  24%|██▍       | 70/291 [00:02<00:06, 31.73it/s]
Loading 0:  25%|██▌       | 74/291 [00:02<00:06, 32.27it/s]
Loading 0:  27%|██▋       | 78/291 [00:02<00:06, 31.71it/s]
Loading 0:  28%|██▊       | 82/291 [00:02<00:09, 22.96it/s]
Loading 0:  29%|██▉       | 85/291 [00:02<00:08, 24.10it/s]
Loading 0:  31%|███       | 90/291 [00:03<00:07, 28.23it/s]
Loading 0:  32%|███▏      | 94/291 [00:03<00:06, 28.63it/s]
Loading 0:  34%|███▍      | 99/291 [00:03<00:05, 32.54it/s]
Loading 0:  35%|███▌      | 103/291 [00:03<00:05, 31.82it/s]
Loading 0:  37%|███▋      | 108/291 [00:03<00:05, 34.20it/s]
Loading 0:  38%|███▊      | 112/291 [00:03<00:05, 32.24it/s]
Loading 0:  40%|███▉      | 116/291 [00:03<00:05, 32.47it/s]
Loading 0:  42%|████▏     | 122/291 [00:03<00:04, 36.33it/s]
Loading 0:  44%|████▎     | 127/291 [00:04<00:04, 34.11it/s]
Loading 0:  46%|████▌     | 133/291 [00:04<00:05, 29.39it/s]
Loading 0:  47%|████▋     | 137/291 [00:04<00:05, 28.89it/s]
Loading 0:  48%|████▊     | 141/291 [00:04<00:05, 27.22it/s]
Loading 0:  51%|█████     | 147/291 [00:04<00:04, 32.15it/s]
Loading 0:  52%|█████▏    | 151/291 [00:04<00:04, 32.10it/s]
Loading 0:  54%|█████▎    | 156/291 [00:05<00:03, 35.72it/s]
Loading 0:  55%|█████▍    | 160/291 [00:05<00:03, 34.64it/s]
Loading 0:  57%|█████▋    | 165/291 [00:05<00:03, 37.78it/s]
Loading 0:  58%|█████▊    | 169/291 [00:05<00:03, 35.55it/s]
Loading 0:  60%|█████▉    | 174/291 [00:05<00:03, 36.60it/s]
Loading 0:  61%|██████    | 178/291 [00:05<00:03, 34.02it/s]
Loading 0:  63%|██████▎   | 184/291 [00:05<00:02, 40.33it/s]
Loading 0:  65%|██████▍   | 189/291 [00:06<00:04, 23.59it/s]
Loading 0:  67%|██████▋   | 194/291 [00:06<00:03, 25.19it/s]
Loading 0:  69%|██████▉   | 201/291 [00:06<00:02, 31.36it/s]
Loading 0:  70%|███████   | 205/291 [00:06<00:02, 30.84it/s]
Loading 0:  72%|███████▏  | 210/291 [00:06<00:02, 33.80it/s]
Loading 0:  74%|███████▎  | 214/291 [00:06<00:02, 31.90it/s]
Loading 0:  75%|███████▌  | 219/291 [00:06<00:02, 34.14it/s]
Loading 0:  77%|███████▋  | 223/291 [00:07<00:02, 33.03it/s]
Loading 0:  78%|███████▊  | 227/291 [00:07<00:01, 33.61it/s]
Loading 0:  79%|███████▉  | 231/291 [00:07<00:01, 32.95it/s]
Loading 0:  81%|████████  | 235/291 [00:07<00:02, 25.66it/s]
Loading 0:  82%|████████▏ | 239/291 [00:07<00:02, 25.36it/s]
Loading 0:  85%|████████▍ | 246/291 [00:07<00:01, 33.00it/s]
Loading 0:  86%|████████▌ | 250/291 [00:08<00:01, 31.17it/s]
Loading 0:  88%|████████▊ | 255/291 [00:08<00:01, 33.62it/s]
Loading 0:  89%|████████▉ | 259/291 [00:08<00:00, 32.04it/s]
Loading 0:  91%|█████████ | 264/291 [00:08<00:00, 34.82it/s]
Loading 0:  92%|█████████▏| 268/291 [00:08<00:00, 32.37it/s]
Loading 0:  94%|█████████▍| 273/291 [00:08<00:00, 35.56it/s]
Loading 0:  95%|█████████▌| 277/291 [00:08<00:00, 33.01it/s]
Loading 0:  97%|█████████▋| 281/291 [00:08<00:00, 33.00it/s]
Loading 0:  98%|█████████▊| 286/291 [00:14<00:01,  2.59it/s]
Loading 0:  99%|█████████▉| 289/291 [00:14<00:00,  3.23it/s]
                                                            
Job trace2333-dpo-v9-v3-reprompt-v1-mkmlizer completed after 126.52s with status: succeeded
Stopping job with name trace2333-dpo-v9-v3-reprompt-v1-mkmlizer
Pipeline stage MKMLizer completed in 129.12s
Running pipeline stage MKMLKubeTemplater
Pipeline stage MKMLKubeTemplater completed in 0.21s
Running pipeline stage ISVCDeployer
Creating inference service trace2333-dpo-v9-v3-reprompt-v1
Waiting for inference service trace2333-dpo-v9-v3-reprompt-v1 to be ready
Inference service trace2333-dpo-v9-v3-reprompt-v1 ready after 170.60793924331665s
Pipeline stage ISVCDeployer completed in 171.17s
Running pipeline stage StressChecker
Received healthy response to inference request in 2.778707981109619s
Received healthy response to inference request in 1.6073226928710938s
Received healthy response to inference request in 1.6748285293579102s
Received healthy response to inference request in 1.6644823551177979s
Received healthy response to inference request in 2.0961220264434814s
5 requests
0 failed requests
5th percentile: 1.6187546253204346
10th percentile: 1.6301865577697754
20th percentile: 1.653050422668457
30th percentile: 1.6665515899658203
40th percentile: 1.6706900596618652
50th percentile: 1.6748285293579102
60th percentile: 1.8433459281921387
70th percentile: 2.0118633270263673
80th percentile: 2.2326392173767093
90th percentile: 2.5056735992431642
95th percentile: 2.6421907901763912
99th percentile: 2.7514045429229737
mean time: 1.9642927169799804
Pipeline stage StressChecker completed in 10.81s
trace2333-dpo-v9-v3-reprompt_v1 status is now deployed due to DeploymentManager action
trace2333-dpo-v9-v3-reprompt_v1 status is now inactive due to auto deactivation removed underperforming models
trace2333-dpo-v9-v3-reprompt_v1 status is now torndown due to DeploymentManager action