azazelle-l3-hecate-8b-v1-0

submission_id: azazelle-l3-hecate-8b-v1-0_v5

developer_uid: Azazelle

best_of: 16

celo_rating: 1216.0

display_name: Azazelle-L3-Hecate-8B-v1_0

family_friendly_score: 0.0

formatter: {'memory_template': "<|begin_of_text|><|start_header_id|>system<|end_header_id|>\n\n{bot_name}'s Persona: {memory}\n\n", 'prompt_template': '{prompt}<|eot_id|>', 'bot_template': '<|start_header_id|>assistant<|end_header_id|>\n\n{bot_name}: {message}<|eot_id|>', 'user_template': '<|start_header_id|>user<|end_header_id|>\n\n{user_name}: {message}<|eot_id|>', 'response_template': '<|start_header_id|>assistant<|end_header_id|>\n\n{bot_name}:', 'truncate_by_message': False}

generation_params: {'temperature': 0.95, 'top_p': 1.0, 'min_p': 0.08, 'top_k': 50, 'presence_penalty': 0.0, 'frequency_penalty': 0.0, 'stopping_words': ['<|eot_id|>', '<|start_header_id|>system<|end_header_id|>', '<|start_header_id|>assistant<|end_header_id|>', '<|start_header_id|>user<|end_header_id|>', '\n{user_name}:'], 'max_input_tokens': 512, 'best_of': 16, 'max_output_tokens': 64}

is_internal_developer: False

language_model: Azazelle/L3-Hecate-8B-v1.0

max_input_tokens: 512

max_output_tokens: 64

model_architecture: LlamaForCausalLM

model_group: Azazelle/L3-Hecate-8B-v1

model_name: Azazelle-L3-Hecate-8B-v1_0

model_num_parameters: 8030261248.0

model_repo: Azazelle/L3-Hecate-8B-v1.0

model_size: 8B

num_battles: 46010

num_wins: 24638

ranking_group: single

reward_formatter: {'bot_template': '{bot_name}: {message}\n', 'memory_template': "{bot_name}'s Persona: {memory}\n####\n", 'prompt_template': '{prompt}\n<START>\n', 'response_template': '{bot_name}:', 'truncate_by_message': False, 'user_template': '{user_name}: {message}\n'}

reward_repo: ChaiML/reward_gpt2_medium_preference_24m_e2

status: torndown

submission_type: basic

timestamp: 2024-07-06T23:02:17+00:00

us_pacific_date: 2024-07-06

win_ratio: 0.5354922842860248

Resubmit model

Running pipeline stage MKMLizer
Starting job with name azazelle-l3-hecate-8b-v1-0-v5-mkmlizer
Waiting for job on azazelle-l3-hecate-8b-v1-0-v5-mkmlizer to finish
azazelle-l3-hecate-8b-v1-0-v5-mkmlizer: ╔═════════════════════════════════════════════════════════════════════╗
azazelle-l3-hecate-8b-v1-0-v5-mkmlizer: ║     _____            __           __                                ║
azazelle-l3-hecate-8b-v1-0-v5-mkmlizer: ║    / _/ /_ ___    __/ /  ___ ___ / /                                ║
azazelle-l3-hecate-8b-v1-0-v5-mkmlizer: ║   / _/ / // / |/|/ / _ \/ -_) -_) /                                 ║
azazelle-l3-hecate-8b-v1-0-v5-mkmlizer: ║  /_//_/\_, /|__,__/_//_/\__/\__/_/                                  ║
azazelle-l3-hecate-8b-v1-0-v5-mkmlizer: ║       /___/                                                         ║
azazelle-l3-hecate-8b-v1-0-v5-mkmlizer: ║                                                                     ║
azazelle-l3-hecate-8b-v1-0-v5-mkmlizer: ║  Version: 0.8.14                                                    ║
azazelle-l3-hecate-8b-v1-0-v5-mkmlizer: ║  Copyright 2023 MK ONE TECHNOLOGIES Inc.                            ║
azazelle-l3-hecate-8b-v1-0-v5-mkmlizer: ║  https://mk1.ai                                                     ║
azazelle-l3-hecate-8b-v1-0-v5-mkmlizer: ║                                                                     ║
azazelle-l3-hecate-8b-v1-0-v5-mkmlizer: ║  The license key for the current software has been verified as      ║
azazelle-l3-hecate-8b-v1-0-v5-mkmlizer: ║  belonging to:                                                      ║
azazelle-l3-hecate-8b-v1-0-v5-mkmlizer: ║                                                                     ║
azazelle-l3-hecate-8b-v1-0-v5-mkmlizer: ║  Chai Research Corp.                                                ║
azazelle-l3-hecate-8b-v1-0-v5-mkmlizer: ║  Account ID: 7997a29f-0ceb-4cc7-9adf-840c57b4ae6f                   ║
azazelle-l3-hecate-8b-v1-0-v5-mkmlizer: ║  Expiration: 2024-07-15 23:59:59                                    ║
azazelle-l3-hecate-8b-v1-0-v5-mkmlizer: ║                                                                     ║
azazelle-l3-hecate-8b-v1-0-v5-mkmlizer: ╚═════════════════════════════════════════════════════════════════════╝
Connection pool is full, discarding connection: %s
Connection pool is full, discarding connection: %s
Job azazelle-l3-hecate-8b-v1-0-v5-mkmlizer completed after 138.01s with status: failed
Stopping job with name azazelle-l3-hecate-8b-v1-0-v5-mkmlizer
%s, retrying in %s seconds...
Starting job with name azazelle-l3-hecate-8b-v1-0-v5-mkmlizer
Waiting for job on azazelle-l3-hecate-8b-v1-0-v5-mkmlizer to finish
azazelle-l3-hecate-8b-v1-0-v5-mkmlizer: ╔═════════════════════════════════════════════════════════════════════╗
azazelle-l3-hecate-8b-v1-0-v5-mkmlizer: ║     _____            __           __                                ║
azazelle-l3-hecate-8b-v1-0-v5-mkmlizer: ║    / _/ /_ ___    __/ /  ___ ___ / /                                ║
azazelle-l3-hecate-8b-v1-0-v5-mkmlizer: ║   / _/ / // / |/|/ / _ \/ -_) -_) /                                 ║
azazelle-l3-hecate-8b-v1-0-v5-mkmlizer: ║  /_//_/\_, /|__,__/_//_/\__/\__/_/                                  ║
azazelle-l3-hecate-8b-v1-0-v5-mkmlizer: ║       /___/                                                         ║
azazelle-l3-hecate-8b-v1-0-v5-mkmlizer: ║                                                                     ║
azazelle-l3-hecate-8b-v1-0-v5-mkmlizer: ║  Version: 0.8.14                                                    ║
azazelle-l3-hecate-8b-v1-0-v5-mkmlizer: ║  Copyright 2023 MK ONE TECHNOLOGIES Inc.                            ║
azazelle-l3-hecate-8b-v1-0-v5-mkmlizer: ║  https://mk1.ai                                                     ║
azazelle-l3-hecate-8b-v1-0-v5-mkmlizer: ║                                                                     ║
azazelle-l3-hecate-8b-v1-0-v5-mkmlizer: ║  The license key for the current software has been verified as      ║
azazelle-l3-hecate-8b-v1-0-v5-mkmlizer: ║  belonging to:                                                      ║
azazelle-l3-hecate-8b-v1-0-v5-mkmlizer: ║                                                                     ║
azazelle-l3-hecate-8b-v1-0-v5-mkmlizer: ║  Chai Research Corp.                                                ║
azazelle-l3-hecate-8b-v1-0-v5-mkmlizer: ║  Account ID: 7997a29f-0ceb-4cc7-9adf-840c57b4ae6f                   ║
azazelle-l3-hecate-8b-v1-0-v5-mkmlizer: ║  Expiration: 2024-07-15 23:59:59                                    ║
azazelle-l3-hecate-8b-v1-0-v5-mkmlizer: ║                                                                     ║
azazelle-l3-hecate-8b-v1-0-v5-mkmlizer: ╚═════════════════════════════════════════════════════════════════════╝
azazelle-l3-hecate-8b-v1-0-v5-mkmlizer: Downloaded to shared memory in 74.093s
azazelle-l3-hecate-8b-v1-0-v5-mkmlizer: quantizing model to /dev/shm/model_cache
azazelle-l3-hecate-8b-v1-0-v5-mkmlizer: Saving flywheel model at /dev/shm/model_cache
Connection pool is full, discarding connection: %s
azazelle-l3-hecate-8b-v1-0-v5-mkmlizer: 
Loading 0:   0%|          | 0/291 [00:00<?, ?it/s]
Loading 0:   1%|          | 2/291 [00:06<16:40,  3.46s/it]
Loading 0:   2%|▏         | 5/291 [00:07<05:21,  1.12s/it]
Loading 0:   3%|▎         | 10/291 [00:08<02:40,  1.75it/s]
Loading 0:   5%|▌         | 15/291 [00:08<01:27,  3.17it/s]
Loading 0:   8%|▊         | 23/291 [00:08<00:42,  6.25it/s]
Loading 0:  11%|█         | 31/291 [00:08<00:25, 10.10it/s]
Loading 0:  12%|█▏        | 36/291 [00:08<00:19, 12.94it/s]
Loading 0:  14%|█▍        | 42/291 [00:08<00:15, 16.55it/s]
Loading 0:  17%|█▋        | 50/291 [00:08<00:10, 23.26it/s]
Loading 0:  20%|█▉        | 58/291 [00:09<00:07, 29.29it/s]
Loading 0:  22%|██▏       | 64/291 [00:09<00:09, 22.87it/s]
Loading 0:  24%|██▎       | 69/291 [00:09<00:08, 25.07it/s]
Loading 0:  26%|██▋       | 77/291 [00:09<00:06, 32.51it/s]
Loading 0:  29%|██▉       | 85/291 [00:09<00:05, 39.36it/s]
Loading 0:  31%|███▏      | 91/291 [00:09<00:04, 41.21it/s]
Loading 0:  33%|███▎      | 97/291 [00:10<00:04, 41.86it/s]
Loading 0:  36%|███▌      | 104/291 [00:10<00:04, 46.63it/s]
Loading 0:  38%|███▊      | 112/291 [00:10<00:03, 51.64it/s]
Loading 0:  41%|████      | 118/291 [00:10<00:05, 31.21it/s]
Loading 0:  42%|████▏     | 123/291 [00:10<00:05, 32.49it/s]
Loading 0:  45%|████▌     | 131/291 [00:11<00:04, 39.16it/s]
Loading 0:  48%|████▊     | 139/291 [00:11<00:03, 44.48it/s]
Loading 0:  50%|████▉     | 145/291 [00:11<00:03, 46.27it/s]
Loading 0:  52%|█████▏    | 151/291 [00:11<00:03, 45.16it/s]
Loading 0:  54%|█████▍    | 158/291 [00:11<00:02, 49.27it/s]
Loading 0:  57%|█████▋    | 166/291 [00:11<00:03, 33.12it/s]
Loading 0:  59%|█████▉    | 171/291 [00:12<00:03, 35.78it/s]
Loading 0:  60%|██████    | 176/291 [00:12<00:03, 37.35it/s]
Loading 0:  63%|██████▎   | 184/291 [00:12<00:02, 45.13it/s]
Loading 0:  65%|██████▌   | 190/291 [00:12<00:02, 46.11it/s]
Loading 0:  67%|██████▋   | 196/291 [00:12<00:02, 44.60it/s]
Loading 0:  70%|██████▉   | 203/291 [00:12<00:01, 47.52it/s]
Loading 0:  73%|███████▎  | 211/291 [00:12<00:01, 53.39it/s]
Loading 0:  75%|███████▍  | 217/291 [00:13<00:02, 31.82it/s]
Loading 0:  76%|███████▋  | 222/291 [00:13<00:02, 33.39it/s]
Loading 0:  79%|███████▉  | 230/291 [00:13<00:01, 40.29it/s]
Loading 0:  82%|████████▏ | 238/291 [00:13<00:01, 46.83it/s]
Loading 0:  84%|████████▍ | 244/291 [00:13<00:00, 48.44it/s]
Loading 0:  86%|████████▌ | 250/291 [00:13<00:00, 46.52it/s]
Loading 0:  88%|████████▊ | 257/291 [00:13<00:00, 49.23it/s]
Loading 0:  91%|█████████ | 265/291 [00:14<00:00, 54.84it/s]
Loading 0:  93%|█████████▎| 271/291 [00:14<00:00, 33.80it/s]
Loading 0:  95%|█████████▍| 276/291 [00:14<00:00, 33.22it/s]
Loading 0:  98%|█████████▊| 284/291 [00:14<00:00, 40.30it/s]
                                                            
Special tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained.
azazelle-l3-hecate-8b-v1-0-v5-mkmlizer: quantized model in 62.291s
azazelle-l3-hecate-8b-v1-0-v5-mkmlizer: Processed model Azazelle/L3-Hecate-8B-v1.0 in 136.384s
azazelle-l3-hecate-8b-v1-0-v5-mkmlizer: creating bucket guanaco-mkml-models
azazelle-l3-hecate-8b-v1-0-v5-mkmlizer: Bucket 's3://guanaco-mkml-models/' created
azazelle-l3-hecate-8b-v1-0-v5-mkmlizer: uploading /dev/shm/model_cache to s3://guanaco-mkml-models/azazelle-l3-hecate-8b-v1-0-v5
azazelle-l3-hecate-8b-v1-0-v5-mkmlizer: cp /dev/shm/model_cache/tokenizer_config.json s3://guanaco-mkml-models/azazelle-l3-hecate-8b-v1-0-v5/tokenizer_config.json
azazelle-l3-hecate-8b-v1-0-v5-mkmlizer: cp /dev/shm/model_cache/config.json s3://guanaco-mkml-models/azazelle-l3-hecate-8b-v1-0-v5/config.json
azazelle-l3-hecate-8b-v1-0-v5-mkmlizer: cp /dev/shm/model_cache/tokenizer.json s3://guanaco-mkml-models/azazelle-l3-hecate-8b-v1-0-v5/tokenizer.json
azazelle-l3-hecate-8b-v1-0-v5-mkmlizer: cp /dev/shm/model_cache/special_tokens_map.json s3://guanaco-mkml-models/azazelle-l3-hecate-8b-v1-0-v5/special_tokens_map.json
azazelle-l3-hecate-8b-v1-0-v5-mkmlizer: cp /dev/shm/model_cache/flywheel_model.0.safetensors s3://guanaco-mkml-models/azazelle-l3-hecate-8b-v1-0-v5/flywheel_model.0.safetensors
azazelle-l3-hecate-8b-v1-0-v5-mkmlizer: loading reward model from ChaiML/reward_gpt2_medium_preference_24m_e2
azazelle-l3-hecate-8b-v1-0-v5-mkmlizer: /opt/conda/lib/python3.10/site-packages/transformers/models/auto/configuration_auto.py:919: FutureWarning: The `use_auth_token` argument is deprecated and will be removed in v5 of Transformers. Please use `token` instead.
azazelle-l3-hecate-8b-v1-0-v5-mkmlizer:   warnings.warn(
azazelle-l3-hecate-8b-v1-0-v5-mkmlizer: /opt/conda/lib/python3.10/site-packages/huggingface_hub/file_download.py:1132: FutureWarning: `resume_download` is deprecated and will be removed in version 1.0.0. Downloads always resume when possible. If you want to force a new download, use `force_download=True`.
azazelle-l3-hecate-8b-v1-0-v5-mkmlizer:   warnings.warn(
azazelle-l3-hecate-8b-v1-0-v5-mkmlizer: /opt/conda/lib/python3.10/site-packages/transformers/models/auto/tokenization_auto.py:769: FutureWarning: The `use_auth_token` argument is deprecated and will be removed in v5 of Transformers. Please use `token` instead.
azazelle-l3-hecate-8b-v1-0-v5-mkmlizer:   warnings.warn(
Job azazelle-l3-hecate-8b-v1-0-v5-mkmlizer completed after 257.28s with status: succeeded
Stopping job with name azazelle-l3-hecate-8b-v1-0-v5-mkmlizer
Pipeline stage MKMLizer completed in 396.51s
Running pipeline stage MKMLKubeTemplater
Pipeline stage MKMLKubeTemplater completed in 0.10s
Running pipeline stage ISVCDeployer
Creating inference service azazelle-l3-hecate-8b-v1-0-v5
Waiting for inference service azazelle-l3-hecate-8b-v1-0-v5 to be ready
Inference service azazelle-l3-hecate-8b-v1-0-v5 ready after 40.194581031799316s
Pipeline stage ISVCDeployer completed in 47.27s
Running pipeline stage StressChecker
Received healthy response to inference request in 2.084643602371216s
Received healthy response to inference request in 2.350501775741577s
Received healthy response to inference request in 1.3133869171142578s
Received healthy response to inference request in 1.2901384830474854s
Received healthy response to inference request in 1.354306936264038s
5 requests
0 failed requests
5th percentile: 1.2947881698608399
10th percentile: 1.2994378566741944
20th percentile: 1.3087372303009033
30th percentile: 1.321570920944214
40th percentile: 1.3379389286041259
50th percentile: 1.354306936264038
60th percentile: 1.646441602706909
70th percentile: 1.93857626914978
80th percentile: 2.137815237045288
90th percentile: 2.2441585063934326
95th percentile: 2.297330141067505
99th percentile: 2.339867448806763
mean time: 1.6785955429077148
Pipeline stage StressChecker completed in 9.11s
azazelle-l3-hecate-8b-v1-0_v5 status is now deployed due to DeploymentManager action
azazelle-l3-hecate-8b-v1-0_v5 status is now inactive due to auto deactivation removed underperforming models
admin requested tearing down of azazelle-l3-hecate-8b-v1-0_v5
Running pipeline stage ISVCDeleter
Checking if service azazelle-l3-hecate-8b-v1-0-v5 is running
Skipping teardown as no inference service was found
Pipeline stage ISVCDeleter completed in 5.22s
Running pipeline stage MKMLModelDeleter
Cleaning model data from S3
Cleaning model data from model cache
Deleting key azazelle-l3-hecate-8b-v1-0-v5/config.json from bucket guanaco-mkml-models
Deleting key azazelle-l3-hecate-8b-v1-0-v5/flywheel_model.0.safetensors from bucket guanaco-mkml-models
Deleting key azazelle-l3-hecate-8b-v1-0-v5/special_tokens_map.json from bucket guanaco-mkml-models
Deleting key azazelle-l3-hecate-8b-v1-0-v5/tokenizer.json from bucket guanaco-mkml-models
Deleting key azazelle-l3-hecate-8b-v1-0-v5/tokenizer_config.json from bucket guanaco-mkml-models
Cleaning model data from model cache
Deleting key azazelle-l3-hecate-8b-v1-0-v5_reward/config.json from bucket guanaco-reward-models
Deleting key azazelle-l3-hecate-8b-v1-0-v5_reward/merges.txt from bucket guanaco-reward-models
Deleting key azazelle-l3-hecate-8b-v1-0-v5_reward/reward.tensors from bucket guanaco-reward-models
Deleting key azazelle-l3-hecate-8b-v1-0-v5_reward/special_tokens_map.json from bucket guanaco-reward-models
Deleting key azazelle-l3-hecate-8b-v1-0-v5_reward/tokenizer.json from bucket guanaco-reward-models
Deleting key azazelle-l3-hecate-8b-v1-0-v5_reward/tokenizer_config.json from bucket guanaco-reward-models
Deleting key azazelle-l3-hecate-8b-v1-0-v5_reward/vocab.json from bucket guanaco-reward-models
Pipeline stage MKMLModelDeleter completed in 5.60s
azazelle-l3-hecate-8b-v1-0_v5 status is now torndown due to DeploymentManager action