jellywibble-wibblewobble

submission_id: jellywibble-wibblewobble_v3

developer_uid: Bbbrun0

best_of: 4

celo_rating: 1207.95

display_name: auto

family_friendly_score: 0.0

formatter: {'memory_template': "<|begin_of_text|><|start_header_id|>system<|end_header_id|>\n\n{bot_name}'s Persona: {memory}\n\n", 'prompt_template': '{prompt}<|eot_id|>', 'bot_template': '<|start_header_id|>assistant<|end_header_id|>\n\n{bot_name}: {message}<|eot_id|>', 'user_template': '<|start_header_id|>user<|end_header_id|>\n\n{user_name}: {message}<|eot_id|>', 'response_template': '<|start_header_id|>assistant<|end_header_id|>\n\n{bot_name}:', 'truncate_by_message': False}

generation_params: {'temperature': 1.0, 'top_p': 0.9, 'min_p': 0.05, 'top_k': 80, 'presence_penalty': 0.0, 'frequency_penalty': 0.0, 'stopping_words': ['\n', '<|end_header_id|>', '<|eot_id|>', '\n\n{user_name}'], 'max_input_tokens': 512, 'best_of': 4, 'max_output_tokens': 64, 'reward_max_token_input': 256}

is_internal_developer: False

language_model: Jellywibble/WibbleWobble

max_input_tokens: 512

max_output_tokens: 64

model_architecture: LlamaForCausalLM

model_group: Jellywibble/WibbleWobble

model_name: auto

model_num_parameters: 8030261248.0

model_repo: Jellywibble/WibbleWobble

model_size: 8B

num_battles: 9697

num_wins: 4725

ranking_group: single

reward_formatter: {'bot_template': '{bot_name}: {message}\n', 'memory_template': "{bot_name}'s Persona: {memory}\n####\n", 'prompt_template': '{prompt}\n<START>\n', 'response_template': '{bot_name}:', 'truncate_by_message': False, 'user_template': '{user_name}: {message}\n'}

reward_repo: rirv938/reward_gpt2_medium_preference_24m_e2

status: torndown

submission_type: basic

timestamp: 2024-08-05T06:07:31+00:00

us_pacific_date: 2024-08-04

win_ratio: 0.4872641022996803

Resubmit model

Running pipeline stage MKMLizer
Starting job with name jellywibble-wibblewobble-v3-mkmlizer
Waiting for job on jellywibble-wibblewobble-v3-mkmlizer to finish
jellywibble-wibblewobble-v3-mkmlizer: ╔═════════════════════════════════════════════════════════════════════╗
jellywibble-wibblewobble-v3-mkmlizer: ║     _____            __           __                                ║
jellywibble-wibblewobble-v3-mkmlizer: ║    / _/ /_ ___    __/ /  ___ ___ / /                                ║
jellywibble-wibblewobble-v3-mkmlizer: ║   / _/ / // / |/|/ / _ \/ -_) -_) /                                 ║
jellywibble-wibblewobble-v3-mkmlizer: ║  /_//_/\_, /|__,__/_//_/\__/\__/_/                                  ║
jellywibble-wibblewobble-v3-mkmlizer: ║       /___/                                                         ║
jellywibble-wibblewobble-v3-mkmlizer: ║                                                                     ║
jellywibble-wibblewobble-v3-mkmlizer: ║  Version: 0.9.9                                                     ║
jellywibble-wibblewobble-v3-mkmlizer: ║  Copyright 2023 MK ONE TECHNOLOGIES Inc.                            ║
jellywibble-wibblewobble-v3-mkmlizer: ║  https://mk1.ai                                                     ║
jellywibble-wibblewobble-v3-mkmlizer: ║                                                                     ║
jellywibble-wibblewobble-v3-mkmlizer: ║  The license key for the current software has been verified as      ║
jellywibble-wibblewobble-v3-mkmlizer: ║  belonging to:                                                      ║
jellywibble-wibblewobble-v3-mkmlizer: ║                                                                     ║
jellywibble-wibblewobble-v3-mkmlizer: ║  Chai Research Corp.                                                ║
jellywibble-wibblewobble-v3-mkmlizer: ║  Account ID: 7997a29f-0ceb-4cc7-9adf-840c57b4ae6f                   ║
jellywibble-wibblewobble-v3-mkmlizer: ║  Expiration: 2024-10-15 23:59:59                                    ║
jellywibble-wibblewobble-v3-mkmlizer: ║                                                                     ║
jellywibble-wibblewobble-v3-mkmlizer: ╚═════════════════════════════════════════════════════════════════════╝
jellywibble-wibblewobble-v3-mkmlizer: Downloaded to shared memory in 41.123s
jellywibble-wibblewobble-v3-mkmlizer: quantizing model to /dev/shm/model_cache, profile:s0, folder:/tmp/tmp5b7pe9xf, device:0
jellywibble-wibblewobble-v3-mkmlizer: Saving flywheel model at /dev/shm/model_cache
jellywibble-wibblewobble-v3-mkmlizer: quantized model in 29.016s
jellywibble-wibblewobble-v3-mkmlizer: Processed model Jellywibble/WibbleWobble in 70.139s
jellywibble-wibblewobble-v3-mkmlizer: creating bucket guanaco-mkml-models
jellywibble-wibblewobble-v3-mkmlizer: Bucket 's3://guanaco-mkml-models/' created
jellywibble-wibblewobble-v3-mkmlizer: uploading /dev/shm/model_cache to s3://guanaco-mkml-models/jellywibble-wibblewobble-v3
jellywibble-wibblewobble-v3-mkmlizer: cp /dev/shm/model_cache/config.json s3://guanaco-mkml-models/jellywibble-wibblewobble-v3/config.json
jellywibble-wibblewobble-v3-mkmlizer: cp /dev/shm/model_cache/special_tokens_map.json s3://guanaco-mkml-models/jellywibble-wibblewobble-v3/special_tokens_map.json
jellywibble-wibblewobble-v3-mkmlizer: cp /dev/shm/model_cache/tokenizer_config.json s3://guanaco-mkml-models/jellywibble-wibblewobble-v3/tokenizer_config.json
jellywibble-wibblewobble-v3-mkmlizer: cp /dev/shm/model_cache/tokenizer.json s3://guanaco-mkml-models/jellywibble-wibblewobble-v3/tokenizer.json
jellywibble-wibblewobble-v3-mkmlizer: cp /dev/shm/model_cache/flywheel_model.0.safetensors s3://guanaco-mkml-models/jellywibble-wibblewobble-v3/flywheel_model.0.safetensors
jellywibble-wibblewobble-v3-mkmlizer: loading reward model from rirv938/reward_gpt2_medium_preference_24m_e2
jellywibble-wibblewobble-v3-mkmlizer: 
Loading 0:   0%|          | 0/291 [00:00<?, ?it/s]
Loading 0:   2%|▏         | 5/291 [00:00<00:11, 25.98it/s]
Loading 0:   4%|▍         | 12/291 [00:00<00:06, 42.09it/s]
Loading 0:   6%|▌         | 17/291 [00:00<00:07, 38.61it/s]
Loading 0:   8%|▊         | 22/291 [00:00<00:07, 36.35it/s]
Loading 0:   9%|▉         | 26/291 [00:00<00:07, 36.33it/s]
Loading 0:  11%|█         | 32/291 [00:00<00:07, 36.99it/s]
Loading 0:  12%|█▏        | 36/291 [00:01<00:10, 23.20it/s]
Loading 0:  14%|█▍        | 41/291 [00:01<00:09, 25.21it/s]
Loading 0:  16%|█▋        | 48/291 [00:01<00:07, 32.63it/s]
Loading 0:  18%|█▊        | 52/291 [00:01<00:07, 31.95it/s]
Loading 0:  20%|█▉        | 57/291 [00:01<00:06, 34.55it/s]
Loading 0:  21%|██        | 61/291 [00:01<00:06, 32.95it/s]
Loading 0:  23%|██▎       | 66/291 [00:02<00:06, 35.77it/s]
Loading 0:  24%|██▍       | 70/291 [00:02<00:06, 34.45it/s]
Loading 0:  25%|██▌       | 74/291 [00:02<00:06, 34.75it/s]
Loading 0:  27%|██▋       | 78/291 [00:02<00:06, 34.14it/s]
Loading 0:  28%|██▊       | 82/291 [00:02<00:08, 23.36it/s]
Loading 0:  30%|██▉       | 86/291 [00:02<00:07, 26.27it/s]
Loading 0:  31%|███       | 90/291 [00:02<00:06, 28.92it/s]
Loading 0:  32%|███▏      | 94/291 [00:03<00:06, 29.85it/s]
Loading 0:  34%|███▍      | 99/291 [00:03<00:05, 34.01it/s]
Loading 0:  35%|███▌      | 103/291 [00:03<00:05, 33.10it/s]
Loading 0:  37%|███▋      | 108/291 [00:03<00:05, 36.23it/s]
Loading 0:  38%|███▊      | 112/291 [00:03<00:05, 34.22it/s]
Loading 0:  40%|███▉      | 116/291 [00:03<00:05, 34.27it/s]
Loading 0:  42%|████▏     | 122/291 [00:03<00:04, 38.96it/s]
Loading 0:  44%|████▎     | 127/291 [00:03<00:04, 36.54it/s]
Loading 0:  46%|████▌     | 133/291 [00:04<00:05, 30.52it/s]
Loading 0:  47%|████▋     | 137/291 [00:04<00:05, 30.32it/s]
Loading 0:  48%|████▊     | 141/291 [00:04<00:05, 28.25it/s]
Loading 0:  51%|█████     | 147/291 [00:04<00:04, 32.79it/s]
Loading 0:  52%|█████▏    | 151/291 [00:04<00:04, 32.18it/s]
Loading 0:  54%|█████▎    | 156/291 [00:04<00:03, 35.02it/s]
Loading 0:  55%|█████▍    | 160/291 [00:04<00:03, 33.77it/s]
Loading 0:  57%|█████▋    | 165/291 [00:05<00:03, 36.22it/s]
Loading 0:  58%|█████▊    | 169/291 [00:05<00:03, 33.61it/s]
Loading 0:  60%|█████▉    | 174/291 [00:05<00:03, 36.25it/s]
Loading 0:  61%|██████    | 178/291 [00:05<00:03, 34.36it/s]
Loading 0:  63%|██████▎   | 184/291 [00:05<00:02, 39.84it/s]
Loading 0:  65%|██████▍   | 189/291 [00:05<00:04, 23.90it/s]
Loading 0:  67%|██████▋   | 194/291 [00:06<00:03, 25.19it/s]
Loading 0:  69%|██████▉   | 201/291 [00:06<00:02, 31.55it/s]
Loading 0:  70%|███████   | 205/291 [00:06<00:02, 31.37it/s]
Loading 0:  72%|███████▏  | 210/291 [00:06<00:02, 33.98it/s]
Loading 0:  74%|███████▎  | 214/291 [00:06<00:02, 33.15it/s]
Loading 0:  75%|███████▌  | 219/291 [00:06<00:02, 35.51it/s]
Loading 0:  77%|███████▋  | 223/291 [00:06<00:01, 34.23it/s]
Loading 0:  78%|███████▊  | 227/291 [00:07<00:01, 34.04it/s]
Loading 0:  79%|███████▉  | 231/291 [00:07<00:01, 33.23it/s]
Loading 0:  81%|████████  | 235/291 [00:07<00:02, 24.05it/s]
Loading 0:  82%|████████▏ | 239/291 [00:07<00:02, 24.54it/s]
Loading 0:  85%|████████▍ | 246/291 [00:07<00:01, 31.95it/s]
Loading 0:  86%|████████▌ | 250/291 [00:07<00:01, 31.53it/s]
Loading 0:  88%|████████▊ | 255/291 [00:07<00:01, 33.91it/s]
Loading 0:  89%|████████▉ | 259/291 [00:08<00:00, 32.79it/s]
Loading 0:  91%|█████████ | 264/291 [00:08<00:00, 35.21it/s]
Loading 0:  92%|█████████▏| 268/291 [00:08<00:00, 33.53it/s]
Loading 0:  94%|█████████▍| 273/291 [00:08<00:00, 35.53it/s]
Loading 0:  95%|█████████▌| 277/291 [00:08<00:00, 33.65it/s]
Loading 0:  97%|█████████▋| 281/291 [00:08<00:00, 33.33it/s]
Loading 0:  98%|█████████▊| 286/291 [00:14<00:01,  2.55it/s]
Loading 0:  99%|█████████▉| 289/291 [00:14<00:00,  3.17it/s]
                                                            
/opt/conda/lib/python3.10/site-packages/transformers/models/auto/configuration_auto.py:957: FutureWarning: The `use_auth_token` argument is deprecated and will be removed in v5 of Transformers. Please use `token` instead.
jellywibble-wibblewobble-v3-mkmlizer:   warnings.warn(
jellywibble-wibblewobble-v3-mkmlizer: /opt/conda/lib/python3.10/site-packages/transformers/models/auto/tokenization_auto.py:785: FutureWarning: The `use_auth_token` argument is deprecated and will be removed in v5 of Transformers. Please use `token` instead.
jellywibble-wibblewobble-v3-mkmlizer:   warnings.warn(
jellywibble-wibblewobble-v3-mkmlizer: /opt/conda/lib/python3.10/site-packages/transformers/models/auto/auto_factory.py:469: FutureWarning: The `use_auth_token` argument is deprecated and will be removed in v5 of Transformers. Please use `token` instead.
jellywibble-wibblewobble-v3-mkmlizer:   warnings.warn(
jellywibble-wibblewobble-v3-mkmlizer: Saving model to /tmp/reward_cache/reward.tensors
jellywibble-wibblewobble-v3-mkmlizer: Saving duration: 0.343s
jellywibble-wibblewobble-v3-mkmlizer: Processed model rirv938/reward_gpt2_medium_preference_24m_e2 in 7.416s
jellywibble-wibblewobble-v3-mkmlizer: creating bucket guanaco-reward-models
jellywibble-wibblewobble-v3-mkmlizer: Bucket 's3://guanaco-reward-models/' created
jellywibble-wibblewobble-v3-mkmlizer: uploading /tmp/reward_cache to s3://guanaco-reward-models/jellywibble-wibblewobble-v3_reward
jellywibble-wibblewobble-v3-mkmlizer: cp /tmp/reward_cache/config.json s3://guanaco-reward-models/jellywibble-wibblewobble-v3_reward/config.json
jellywibble-wibblewobble-v3-mkmlizer: cp /tmp/reward_cache/special_tokens_map.json s3://guanaco-reward-models/jellywibble-wibblewobble-v3_reward/special_tokens_map.json
jellywibble-wibblewobble-v3-mkmlizer: cp /tmp/reward_cache/tokenizer_config.json s3://guanaco-reward-models/jellywibble-wibblewobble-v3_reward/tokenizer_config.json
jellywibble-wibblewobble-v3-mkmlizer: cp /tmp/reward_cache/merges.txt s3://guanaco-reward-models/jellywibble-wibblewobble-v3_reward/merges.txt
jellywibble-wibblewobble-v3-mkmlizer: cp /tmp/reward_cache/vocab.json s3://guanaco-reward-models/jellywibble-wibblewobble-v3_reward/vocab.json
jellywibble-wibblewobble-v3-mkmlizer: cp /tmp/reward_cache/tokenizer.json s3://guanaco-reward-models/jellywibble-wibblewobble-v3_reward/tokenizer.json
jellywibble-wibblewobble-v3-mkmlizer: cp /tmp/reward_cache/reward.tensors s3://guanaco-reward-models/jellywibble-wibblewobble-v3_reward/reward.tensors
Job jellywibble-wibblewobble-v3-mkmlizer completed after 104.85s with status: succeeded
Stopping job with name jellywibble-wibblewobble-v3-mkmlizer
Pipeline stage MKMLizer completed in 105.73s
Running pipeline stage MKMLKubeTemplater
Pipeline stage MKMLKubeTemplater completed in 0.10s
Running pipeline stage ISVCDeployer
Creating inference service jellywibble-wibblewobble-v3
Waiting for inference service jellywibble-wibblewobble-v3 to be ready
Inference service jellywibble-wibblewobble-v3 ready after 181.1525890827179s
Pipeline stage ISVCDeployer completed in 182.78s
Running pipeline stage StressChecker
Received healthy response to inference request in 1.96547532081604s
Received healthy response to inference request in 1.1140236854553223s
Received healthy response to inference request in 1.102936029434204s
Received healthy response to inference request in 1.06144380569458s
Received healthy response to inference request in 1.105583667755127s
5 requests
0 failed requests
5th percentile: 1.069742250442505
10th percentile: 1.0780406951904298
20th percentile: 1.0946375846862793
30th percentile: 1.1034655570983887
40th percentile: 1.1045246124267578
50th percentile: 1.105583667755127
60th percentile: 1.108959674835205
70th percentile: 1.1123356819152832
80th percentile: 1.284314012527466
90th percentile: 1.624894666671753
95th percentile: 1.7951849937438964
99th percentile: 1.9314172554016114
mean time: 1.2698925018310547
Pipeline stage StressChecker completed in 7.16s
jellywibble-wibblewobble_v3 status is now deployed due to DeploymentManager action
jellywibble-wibblewobble_v3 status is now inactive due to auto deactivation removed underperforming models
jellywibble-wibblewobble_v3 status is now torndown due to DeploymentManager action