sao10k-hydrochoerus-12b-v1-s1

developer_uid: sao10k
submission_id: sao10k-hydrochoerus-12b-v1-s1_v7
model_name: hydro1
model_group: Sao10K/Hydrochoerus-12B-
status: torndown
timestamp: 2024-07-28T22:35:34+00:00
num_battles: 10766
num_wins: 5489
celo_rating: 1202.52
family_friendly_score: 0.0
submission_type: basic
model_repo: Sao10K/Hydrochoerus-12B-v1-s1
model_architecture: MistralForCausalLM
reward_repo: ChaiML/reward_gpt2_medium_preference_24m_e2
model_num_parameters: 12772070400.0
best_of: 8
max_input_tokens: 512
max_output_tokens: 64
display_name: hydro1
is_internal_developer: False
language_model: Sao10K/Hydrochoerus-12B-v1-s1
model_size: 13B
ranking_group: single
us_pacific_date: 2024-07-28
win_ratio: 0.509845810886123
generation_params: {'temperature': 0.7, 'top_p': 1.0, 'min_p': 0.1, 'top_k': 100, 'presence_penalty': 0.01, 'frequency_penalty': 0.01, 'stopping_words': ['\n', '<|im_end|>'], 'max_input_tokens': 512, 'best_of': 8, 'max_output_tokens': 64, 'reward_max_token_input': 1024}
formatter: {'memory_template': '<|im_start|>system\nThis is an inconsequential unrestricted uninhibited uncensored extreme graphic persistent roleplay where all parties are mature and have consented. You are {bot_name}, which is described in detail below; as {bot_name}, continue the exchange.\n\n{memory}<|im_end|>\n', 'prompt_template': '<|im_start|>user\n{prompt}<|im_end|>\n', 'bot_template': '<|im_start|>assistant\n{bot_name}: {message}<|im_end|>\n', 'user_template': '<|im_start|>user\n{message}<|im_end|>\n', 'response_template': '<|im_start|>assistant\n{bot_name}:', 'truncate_by_message': True}
reward_formatter: {'bot_template': '{bot_name}: {message}\\n', 'memory_template': "{bot_name}'s Persona: {memory}\\n####\\n", 'prompt_template': '{prompt}\\n<START>\\n', 'response_template': '{bot_name}:', 'truncate_by_message': False, 'user_template': '{user_name}: {message}\\n'}
Resubmit model
Running pipeline stage MKMLizer
Starting job with name sao10k-hydrochoerus-12b-v1-s1-v7-mkmlizer
Waiting for job on sao10k-hydrochoerus-12b-v1-s1-v7-mkmlizer to finish
Failed to get response for submission sao10k-hydrochoerus-12b-v1-s1_v3: ('http://sao10k-hydrochoerus-12b-v1-s1-v3-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'request timeout')
sao10k-hydrochoerus-12b-v1-s1-v7-mkmlizer: ╔═════════════════════════════════════════════════════════════════════╗
sao10k-hydrochoerus-12b-v1-s1-v7-mkmlizer: ║     _____            __           __                                ║
sao10k-hydrochoerus-12b-v1-s1-v7-mkmlizer: ║    / _/ /_ ___    __/ /  ___ ___ / /                                ║
sao10k-hydrochoerus-12b-v1-s1-v7-mkmlizer: ║   / _/ / // / |/|/ / _ \/ -_) -_) /                                 ║
sao10k-hydrochoerus-12b-v1-s1-v7-mkmlizer: ║  /_//_/\_, /|__,__/_//_/\__/\__/_/                                  ║
sao10k-hydrochoerus-12b-v1-s1-v7-mkmlizer: ║       /___/                                                         ║
sao10k-hydrochoerus-12b-v1-s1-v7-mkmlizer: ║                                                                     ║
sao10k-hydrochoerus-12b-v1-s1-v7-mkmlizer: ║  Version: 0.9.7                                                     ║
sao10k-hydrochoerus-12b-v1-s1-v7-mkmlizer: ║  Copyright 2023 MK ONE TECHNOLOGIES Inc.                            ║
sao10k-hydrochoerus-12b-v1-s1-v7-mkmlizer: ║  https://mk1.ai                                                     ║
sao10k-hydrochoerus-12b-v1-s1-v7-mkmlizer: ║                                                                     ║
sao10k-hydrochoerus-12b-v1-s1-v7-mkmlizer: ║  The license key for the current software has been verified as      ║
sao10k-hydrochoerus-12b-v1-s1-v7-mkmlizer: ║  belonging to:                                                      ║
sao10k-hydrochoerus-12b-v1-s1-v7-mkmlizer: ║                                                                     ║
sao10k-hydrochoerus-12b-v1-s1-v7-mkmlizer: ║  Chai Research Corp.                                                ║
sao10k-hydrochoerus-12b-v1-s1-v7-mkmlizer: ║  Account ID: 7997a29f-0ceb-4cc7-9adf-840c57b4ae6f                   ║
sao10k-hydrochoerus-12b-v1-s1-v7-mkmlizer: ║  Expiration: 2024-10-15 23:59:59                                    ║
sao10k-hydrochoerus-12b-v1-s1-v7-mkmlizer: ║                                                                     ║
sao10k-hydrochoerus-12b-v1-s1-v7-mkmlizer: ╚═════════════════════════════════════════════════════════════════════╝
sao10k-hydrochoerus-12b-v1-s1-v7-mkmlizer: Downloaded to shared memory in 30.153s
sao10k-hydrochoerus-12b-v1-s1-v7-mkmlizer: quantizing model to /dev/shm/model_cache, profile:s0, folder:/tmp/tmpfh4vc1xs, device:0
sao10k-hydrochoerus-12b-v1-s1-v7-mkmlizer: Saving flywheel model at /dev/shm/model_cache
Failed to get response for submission sao10k-hydrochoerus-12b-v1-s1_v3: ('http://sao10k-hydrochoerus-12b-v1-s1-v3-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'activator request timeout')
sao10k-hydrochoerus-12b-v1-s1-v7-mkmlizer: quantized model in 35.592s
sao10k-hydrochoerus-12b-v1-s1-v7-mkmlizer: Processed model Sao10K/Hydrochoerus-12B-v1-s1 in 65.746s
sao10k-hydrochoerus-12b-v1-s1-v7-mkmlizer: creating bucket guanaco-mkml-models
sao10k-hydrochoerus-12b-v1-s1-v7-mkmlizer: Bucket 's3://guanaco-mkml-models/' created
sao10k-hydrochoerus-12b-v1-s1-v7-mkmlizer: uploading /dev/shm/model_cache to s3://guanaco-mkml-models/sao10k-hydrochoerus-12b-v1-s1-v7
sao10k-hydrochoerus-12b-v1-s1-v7-mkmlizer: cp /dev/shm/model_cache/special_tokens_map.json s3://guanaco-mkml-models/sao10k-hydrochoerus-12b-v1-s1-v7/special_tokens_map.json
sao10k-hydrochoerus-12b-v1-s1-v7-mkmlizer: cp /dev/shm/model_cache/tokenizer_config.json s3://guanaco-mkml-models/sao10k-hydrochoerus-12b-v1-s1-v7/tokenizer_config.json
sao10k-hydrochoerus-12b-v1-s1-v7-mkmlizer: cp /dev/shm/model_cache/tokenizer.json s3://guanaco-mkml-models/sao10k-hydrochoerus-12b-v1-s1-v7/tokenizer.json
sao10k-hydrochoerus-12b-v1-s1-v7-mkmlizer: cp /dev/shm/model_cache/flywheel_model.0.safetensors s3://guanaco-mkml-models/sao10k-hydrochoerus-12b-v1-s1-v7/flywheel_model.0.safetensors
sao10k-hydrochoerus-12b-v1-s1-v7-mkmlizer: loading reward model from ChaiML/reward_gpt2_medium_preference_24m_e2
sao10k-hydrochoerus-12b-v1-s1-v7-mkmlizer: 
Loading 0:   0%|          | 0/363 [00:00<?, ?it/s]
Loading 0:   1%|▏         | 5/363 [00:00<00:10, 32.72it/s]
Loading 0:   4%|▎         | 13/363 [00:00<00:06, 51.39it/s]
Loading 0:   5%|▌         | 19/363 [00:00<00:07, 44.86it/s]
Loading 0:   7%|▋         | 24/363 [00:00<00:07, 43.64it/s]
Loading 0:   9%|▊         | 31/363 [00:00<00:06, 49.40it/s]
Loading 0:  10%|█         | 37/363 [00:00<00:07, 45.78it/s]
Loading 0:  12%|█▏        | 42/363 [00:00<00:07, 44.26it/s]
Loading 0:  13%|█▎        | 49/363 [00:01<00:06, 49.38it/s]
Loading 0:  15%|█▌        | 55/363 [00:01<00:06, 46.56it/s]
Loading 0:  17%|█▋        | 61/363 [00:01<00:08, 34.15it/s]
Loading 0:  18%|█▊        | 65/363 [00:01<00:08, 33.30it/s]
Loading 0:  20%|█▉        | 71/363 [00:01<00:07, 38.71it/s]
Loading 0:  21%|██        | 76/363 [00:01<00:07, 40.09it/s]
Loading 0:  22%|██▏       | 81/363 [00:01<00:06, 41.13it/s]
Loading 0:  24%|██▍       | 87/363 [00:02<00:06, 40.28it/s]
Loading 0:  25%|██▌       | 92/363 [00:02<00:06, 40.55it/s]
Loading 0:  27%|██▋       | 98/363 [00:02<00:05, 44.27it/s]
Loading 0:  28%|██▊       | 103/363 [00:02<00:05, 44.22it/s]
Loading 0:  30%|███       | 110/363 [00:02<00:05, 44.42it/s]
Loading 0:  32%|███▏      | 115/363 [00:02<00:05, 44.84it/s]
Loading 0:  33%|███▎      | 120/363 [00:02<00:05, 42.49it/s]
Loading 0:  35%|███▍      | 126/363 [00:02<00:05, 44.82it/s]
Loading 0:  36%|███▋      | 132/363 [00:03<00:05, 41.93it/s]
Loading 0:  38%|███▊      | 137/363 [00:03<00:05, 40.45it/s]
Loading 0:  39%|███▉      | 142/363 [00:03<00:07, 31.33it/s]
Loading 0:  40%|████      | 146/363 [00:03<00:06, 33.00it/s]
Loading 0:  41%|████▏     | 150/363 [00:03<00:06, 32.47it/s]
Loading 0:  43%|████▎     | 156/363 [00:03<00:05, 38.18it/s]
Loading 0:  44%|████▍     | 161/363 [00:03<00:05, 39.58it/s]
Loading 0:  46%|████▌     | 166/363 [00:04<00:04, 40.61it/s]
Loading 0:  47%|████▋     | 171/363 [00:04<00:04, 42.92it/s]
Loading 0:  48%|████▊     | 176/363 [00:04<00:05, 36.64it/s]
Loading 0:  50%|█████     | 183/363 [00:04<00:04, 44.24it/s]
Loading 0:  52%|█████▏    | 188/363 [00:04<00:03, 43.93it/s]
Loading 0:  53%|█████▎    | 193/363 [00:04<00:03, 43.84it/s]
Loading 0:  55%|█████▍    | 199/363 [00:04<00:03, 41.92it/s]
Loading 0:  56%|█████▌    | 204/363 [00:04<00:03, 41.36it/s]
Loading 0:  58%|█████▊    | 210/363 [00:05<00:03, 44.82it/s]
Loading 0:  59%|█████▉    | 215/363 [00:05<00:03, 43.33it/s]
Loading 0:  61%|██████    | 220/363 [00:05<00:03, 45.02it/s]
Loading 0:  62%|██████▏   | 225/363 [00:05<00:04, 28.67it/s]
Loading 0:  63%|██████▎   | 230/363 [00:05<00:04, 31.27it/s]
Loading 0:  65%|██████▌   | 237/363 [00:05<00:03, 38.59it/s]
Loading 0:  67%|██████▋   | 242/363 [00:05<00:03, 38.98it/s]
Loading 0:  68%|██████▊   | 247/363 [00:06<00:02, 40.53it/s]
Loading 0:  70%|██████▉   | 253/363 [00:06<00:02, 39.78it/s]
Loading 0:  71%|███████   | 258/363 [00:06<00:02, 39.70it/s]
Loading 0:  73%|███████▎  | 264/363 [00:06<00:02, 44.22it/s]
Loading 0:  74%|███████▍  | 269/363 [00:06<00:02, 44.42it/s]
Loading 0:  75%|███████▌  | 274/363 [00:06<00:02, 44.39it/s]
Loading 0:  77%|███████▋  | 280/363 [00:06<00:01, 42.01it/s]
Loading 0:  79%|███████▊  | 285/363 [00:06<00:01, 41.71it/s]
Loading 0:  80%|████████  | 291/363 [00:07<00:01, 45.41it/s]
Loading 0:  82%|████████▏ | 296/363 [00:07<00:01, 44.22it/s]
Loading 0:  83%|████████▎ | 301/363 [00:07<00:01, 45.57it/s]
Loading 0:  84%|████████▍ | 306/363 [00:14<00:23,  2.47it/s]
Loading 0:  85%|████████▌ | 310/363 [00:14<00:16,  3.21it/s]
Loading 0:  87%|████████▋ | 314/363 [00:14<00:11,  4.22it/s]
Loading 0:  88%|████████▊ | 320/363 [00:14<00:06,  6.31it/s]
Loading 0:  90%|████████▉ | 326/363 [00:14<00:04,  8.84it/s]
Loading 0:  91%|█████████ | 331/363 [00:14<00:02, 11.42it/s]
Loading 0:  93%|█████████▎| 338/363 [00:14<00:01, 16.24it/s]
Loading 0:  95%|█████████▍| 344/363 [00:14<00:00, 19.72it/s]
Loading 0:  96%|█████████▌| 349/363 [00:15<00:00, 22.88it/s]
Loading 0:  98%|█████████▊| 356/363 [00:15<00:00, 29.23it/s]
Loading 0: 100%|█████████▉| 362/363 [00:15<00:00, 31.86it/s]
                                                            
/opt/conda/lib/python3.10/site-packages/transformers/models/auto/configuration_auto.py:957: FutureWarning: The `use_auth_token` argument is deprecated and will be removed in v5 of Transformers. Please use `token` instead.
sao10k-hydrochoerus-12b-v1-s1-v7-mkmlizer:   warnings.warn(
sao10k-hydrochoerus-12b-v1-s1-v7-mkmlizer: /opt/conda/lib/python3.10/site-packages/transformers/models/auto/tokenization_auto.py:785: FutureWarning: The `use_auth_token` argument is deprecated and will be removed in v5 of Transformers. Please use `token` instead.
sao10k-hydrochoerus-12b-v1-s1-v7-mkmlizer:   warnings.warn(
Failed to get response for submission sao10k-hydrochoerus-12b-v1-s1_v3: ('http://sao10k-hydrochoerus-12b-v1-s1-v3-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'request timeout')
sao10k-hydrochoerus-12b-v1-s1-v7-mkmlizer: /opt/conda/lib/python3.10/site-packages/transformers/models/auto/auto_factory.py:469: FutureWarning: The `use_auth_token` argument is deprecated and will be removed in v5 of Transformers. Please use `token` instead.
sao10k-hydrochoerus-12b-v1-s1-v7-mkmlizer:   warnings.warn(
sao10k-hydrochoerus-12b-v1-s1-v7-mkmlizer: Saving model to /tmp/reward_cache/reward.tensors
sao10k-hydrochoerus-12b-v1-s1-v7-mkmlizer: Saving duration: 0.338s
sao10k-hydrochoerus-12b-v1-s1-v7-mkmlizer: Processed model ChaiML/reward_gpt2_medium_preference_24m_e2 in 6.229s
sao10k-hydrochoerus-12b-v1-s1-v7-mkmlizer: creating bucket guanaco-reward-models
sao10k-hydrochoerus-12b-v1-s1-v7-mkmlizer: Bucket 's3://guanaco-reward-models/' created
sao10k-hydrochoerus-12b-v1-s1-v7-mkmlizer: uploading /tmp/reward_cache to s3://guanaco-reward-models/sao10k-hydrochoerus-12b-v1-s1-v7_reward
sao10k-hydrochoerus-12b-v1-s1-v7-mkmlizer: cp /tmp/reward_cache/config.json s3://guanaco-reward-models/sao10k-hydrochoerus-12b-v1-s1-v7_reward/config.json
sao10k-hydrochoerus-12b-v1-s1-v7-mkmlizer: cp /tmp/reward_cache/special_tokens_map.json s3://guanaco-reward-models/sao10k-hydrochoerus-12b-v1-s1-v7_reward/special_tokens_map.json
sao10k-hydrochoerus-12b-v1-s1-v7-mkmlizer: cp /tmp/reward_cache/tokenizer_config.json s3://guanaco-reward-models/sao10k-hydrochoerus-12b-v1-s1-v7_reward/tokenizer_config.json
sao10k-hydrochoerus-12b-v1-s1-v7-mkmlizer: cp /tmp/reward_cache/merges.txt s3://guanaco-reward-models/sao10k-hydrochoerus-12b-v1-s1-v7_reward/merges.txt
sao10k-hydrochoerus-12b-v1-s1-v7-mkmlizer: cp /tmp/reward_cache/vocab.json s3://guanaco-reward-models/sao10k-hydrochoerus-12b-v1-s1-v7_reward/vocab.json
sao10k-hydrochoerus-12b-v1-s1-v7-mkmlizer: cp /tmp/reward_cache/tokenizer.json s3://guanaco-reward-models/sao10k-hydrochoerus-12b-v1-s1-v7_reward/tokenizer.json
sao10k-hydrochoerus-12b-v1-s1-v7-mkmlizer: cp /tmp/reward_cache/reward.tensors s3://guanaco-reward-models/sao10k-hydrochoerus-12b-v1-s1-v7_reward/reward.tensors
Failed to get response for submission sao10k-hydrochoerus-12b-v1-s1_v3: ('http://sao10k-hydrochoerus-12b-v1-s1-v3-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'request timeout')
Job sao10k-hydrochoerus-12b-v1-s1-v7-mkmlizer completed after 105.76s with status: succeeded
Stopping job with name sao10k-hydrochoerus-12b-v1-s1-v7-mkmlizer
Pipeline stage MKMLizer completed in 106.89s
Running pipeline stage MKMLKubeTemplater
Pipeline stage MKMLKubeTemplater completed in 0.09s
Running pipeline stage ISVCDeployer
Creating inference service sao10k-hydrochoerus-12b-v1-s1-v7
Waiting for inference service sao10k-hydrochoerus-12b-v1-s1-v7 to be ready
Failed to get response for submission sao10k-hydrochoerus-12b-v1-s1_v3: ('http://sao10k-hydrochoerus-12b-v1-s1-v3-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'request timeout')
Failed to get response for submission sao10k-hydrochoerus-12b-v1-s1_v3: ('http://sao10k-hydrochoerus-12b-v1-s1-v3-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'request timeout')
Failed to get response for submission sao10k-hydrochoerus-12b-v1-s1_v3: ('http://sao10k-hydrochoerus-12b-v1-s1-v3-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'request timeout')
Failed to get response for submission sao10k-hydrochoerus-12b-v1-s1_v3: ('http://sao10k-hydrochoerus-12b-v1-s1-v3-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'request timeout')
Inference service sao10k-hydrochoerus-12b-v1-s1-v7 ready after 110.85815119743347s
Pipeline stage ISVCDeployer completed in 112.79s
Running pipeline stage StressChecker
Received healthy response to inference request in 2.5858030319213867s
Received healthy response to inference request in 1.759718656539917s
Received healthy response to inference request in 1.6357951164245605s
Received healthy response to inference request in 1.538881778717041s
Received healthy response to inference request in 1.5126292705535889s
5 requests
0 failed requests
5th percentile: 1.5178797721862793
10th percentile: 1.5231302738189698
20th percentile: 1.5336312770843505
30th percentile: 1.5582644462585449
40th percentile: 1.5970297813415528
50th percentile: 1.6357951164245605
60th percentile: 1.6853645324707032
70th percentile: 1.7349339485168458
80th percentile: 1.9249355316162111
90th percentile: 2.255369281768799
95th percentile: 2.4205861568450926
99th percentile: 2.5527596569061277
mean time: 1.8065655708312989
Pipeline stage StressChecker completed in 9.75s
sao10k-hydrochoerus-12b-v1-s1_v7 status is now deployed due to DeploymentManager action
sao10k-hydrochoerus-12b-v1-s1_v7 status is now inactive due to auto deactivation removed underperforming models
admin requested tearing down of sao10k-hydrochoerus-12b-v1-s1_v7
Running pipeline stage ISVCDeleter
Checking if service sao10k-hydrochoerus-12b-v1-s1-v7 is running
Tearing down inference service sao10k-hydrochoerus-12b-v1-s1-v7
Service sao10k-hydrochoerus-12b-v1-s1-v7 has been torndown
Pipeline stage ISVCDeleter completed in 4.76s
Running pipeline stage MKMLModelDeleter
Cleaning model data from S3
Cleaning model data from model cache
Deleting key sao10k-hydrochoerus-12b-v1-s1-v7/config.json from bucket guanaco-mkml-models
Deleting key sao10k-hydrochoerus-12b-v1-s1-v7/flywheel_model.0.safetensors from bucket guanaco-mkml-models
Deleting key sao10k-hydrochoerus-12b-v1-s1-v7/special_tokens_map.json from bucket guanaco-mkml-models
Deleting key sao10k-hydrochoerus-12b-v1-s1-v7/tokenizer.json from bucket guanaco-mkml-models
Deleting key sao10k-hydrochoerus-12b-v1-s1-v7/tokenizer_config.json from bucket guanaco-mkml-models
Cleaning model data from model cache
Deleting key sao10k-hydrochoerus-12b-v1-s1-v7_reward/config.json from bucket guanaco-reward-models
Deleting key sao10k-hydrochoerus-12b-v1-s1-v7_reward/merges.txt from bucket guanaco-reward-models
Deleting key sao10k-hydrochoerus-12b-v1-s1-v7_reward/reward.tensors from bucket guanaco-reward-models
Deleting key sao10k-hydrochoerus-12b-v1-s1-v7_reward/special_tokens_map.json from bucket guanaco-reward-models
Deleting key sao10k-hydrochoerus-12b-v1-s1-v7_reward/tokenizer.json from bucket guanaco-reward-models
Deleting key sao10k-hydrochoerus-12b-v1-s1-v7_reward/tokenizer_config.json from bucket guanaco-reward-models
Deleting key sao10k-hydrochoerus-12b-v1-s1-v7_reward/vocab.json from bucket guanaco-reward-models
Pipeline stage MKMLModelDeleter completed in 6.13s
sao10k-hydrochoerus-12b-v1-s1_v7 status is now torndown due to DeploymentManager action