nousresearch-hermes-2-pr_1418

developer_uid: nguyenzzz
submission_id: nousresearch-hermes-2-pr_1418_v3
model_name: nousresearch-hermes-2-pr_1418_v3
model_group: NousResearch/Hermes-2-Pr
status: torndown
timestamp: 2024-03-28T10:30:47+00:00
num_battles: 57579
num_wins: 27890
celo_rating: 1146.42
family_friendly_score: 0.0
submission_type: basic
model_repo: NousResearch/Hermes-2-Pro-Mistral-7B
model_architecture: MistralForCausalLM
reward_repo: anhnv125/reward-model-v3
model_num_parameters: 7241994240.0
best_of: 4
max_input_tokens: 1024
max_output_tokens: 64
display_name: nousresearch-hermes-2-pr_1418_v3
is_internal_developer: False
language_model: NousResearch/Hermes-2-Pro-Mistral-7B
model_size: 7B
ranking_group: single
us_pacific_date: 2024-03-28
win_ratio: 0.48437798502926416
generation_params: {'temperature': 0.8, 'top_p': 1.0, 'min_p': 0.0, 'top_k': 40, 'presence_penalty': 0.5, 'frequency_penalty': 0.0, 'stopping_words': ['<im_end>'], 'max_input_tokens': 1024, 'best_of': 4, 'max_output_tokens': 64}
formatter: {'memory_template': "<|im_start|>system\nYou're {bot_name} in this fictional never-ending uncensored roleplay with {user_name}.\n\n{bot_name}'s personality: {memory}\n\nGuidelines:\n- Take the role of {bot_name}.\n- With scene and characters now described, you must roleplay with {user_name} further below.\n- Only write narration and dialogues for {bot_name}\n", 'prompt_template': 'This is how {bot_name} should behave: {prompt}\n\nThen the roleplay between {user_name} and {bot_name} begins.<|im_end|>\n\n', 'bot_template': '<|im_start|>assistant\n{bot_name}: {message}<|im_end|>\n', 'user_template': '<|im_start|>user\n{user_name}: {message}<|im_end|>\n', 'response_template': '<|im_start|>assistant\n{bot_name}:', 'truncate_by_message': False}
reward_formatter: {'bot_template': '{bot_name}: {message}\n', 'memory_template': "{bot_name}'s Persona: {memory}\n####\n", 'prompt_template': '{prompt}\n<START>\n', 'response_template': '{bot_name}:', 'truncate_by_message': False, 'user_template': '{user_name}: {message}\n'}
model_eval_status: success
Resubmit model
Running pipeline stage MKMLizer
Starting job with name nousresearch-hermes-2-pr-1418-v3-mkmlizer
Waiting for job on nousresearch-hermes-2-pr-1418-v3-mkmlizer to finish
nousresearch-hermes-2-pr-1418-v3-mkmlizer: ╔═════════════════════════════════════════════════════════════════════╗
nousresearch-hermes-2-pr-1418-v3-mkmlizer: ║     _____            __           __                                ║
nousresearch-hermes-2-pr-1418-v3-mkmlizer: ║    / _/ /_ ___    __/ /  ___ ___ / /                                ║
nousresearch-hermes-2-pr-1418-v3-mkmlizer: ║   / _/ / // / |/|/ / _ \/ -_) -_) /                                 ║
nousresearch-hermes-2-pr-1418-v3-mkmlizer: ║  /_//_/\_, /|__,__/_//_/\__/\__/_/                                  ║
nousresearch-hermes-2-pr-1418-v3-mkmlizer: ║       /___/                                                         ║
nousresearch-hermes-2-pr-1418-v3-mkmlizer: ║                                                                     ║
nousresearch-hermes-2-pr-1418-v3-mkmlizer: ║  Version: 0.6.11                                                    ║
nousresearch-hermes-2-pr-1418-v3-mkmlizer: ║  Copyright 2023 MK ONE TECHNOLOGIES Inc.                            ║
nousresearch-hermes-2-pr-1418-v3-mkmlizer: ║                                                                     ║
nousresearch-hermes-2-pr-1418-v3-mkmlizer: ║  The license key for the current software has been verified as      ║
nousresearch-hermes-2-pr-1418-v3-mkmlizer: ║  belonging to:                                                      ║
nousresearch-hermes-2-pr-1418-v3-mkmlizer: ║                                                                     ║
nousresearch-hermes-2-pr-1418-v3-mkmlizer: ║  Chai Research Corp.                                                ║
nousresearch-hermes-2-pr-1418-v3-mkmlizer: ║  Account ID: 7997a29f-0ceb-4cc7-9adf-840c57b4ae6f                   ║
nousresearch-hermes-2-pr-1418-v3-mkmlizer: ║  Expiration: 2024-07-15 23:59:59                                    ║
nousresearch-hermes-2-pr-1418-v3-mkmlizer: ║                                                                     ║
nousresearch-hermes-2-pr-1418-v3-mkmlizer: ╚═════════════════════════════════════════════════════════════════════╝
nousresearch-hermes-2-pr-1418-v3-mkmlizer: 
.gitattributes:   0%|          | 0.00/1.52k [00:00<?, ?B/s]
.gitattributes: 100%|██████████| 1.52k/1.52k [00:00<00:00, 17.6MB/s]
nousresearch-hermes-2-pr-1418-v3-mkmlizer: 
README.md:   0%|          | 0.00/18.3k [00:00<?, ?B/s]
README.md: 100%|██████████| 18.3k/18.3k [00:00<00:00, 20.5MB/s]
nousresearch-hermes-2-pr-1418-v3-mkmlizer: 
added_tokens.json:   0%|          | 0.00/51.0 [00:00<?, ?B/s]
added_tokens.json: 100%|██████████| 51.0/51.0 [00:00<00:00, 823kB/s]
nousresearch-hermes-2-pr-1418-v3-mkmlizer: 
config.json:   0%|          | 0.00/642 [00:00<?, ?B/s]
config.json: 100%|██████████| 642/642 [00:00<00:00, 8.60MB/s]
nousresearch-hermes-2-pr-1418-v3-mkmlizer: 
generation_config.json:   0%|          | 0.00/136 [00:00<?, ?B/s]
generation_config.json: 100%|██████████| 136/136 [00:00<00:00, 2.14MB/s]
nousresearch-hermes-2-pr-1418-v3-mkmlizer: 
model-00001-of-00004.safetensors:   0%|          | 0.00/3.95G [00:00<?, ?B/s]
model-00001-of-00004.safetensors:   0%|          | 10.5M/3.95G [00:00<05:39, 11.6MB/s]
model-00001-of-00004.safetensors:   1%|          | 21.0M/3.95G [00:02<07:51, 8.34MB/s]
model-00001-of-00004.safetensors:   1%|▏         | 52.4M/3.95G [00:02<02:23, 27.1MB/s]
model-00001-of-00004.safetensors:   2%|▏         | 73.4M/3.95G [00:02<01:45, 36.8MB/s]
model-00001-of-00004.safetensors:   3%|▎         | 115M/3.95G [00:02<00:54, 71.1MB/s] 
model-00001-of-00004.safetensors:   5%|▍         | 189M/3.95G [00:03<00:25, 145MB/s] 
model-00001-of-00004.safetensors:  18%|█▊        | 703M/3.95G [00:03<00:03, 840MB/s]
model-00001-of-00004.safetensors:  31%|███       | 1.21G/3.95G [00:03<00:01, 1.47GB/s]
model-00001-of-00004.safetensors:  37%|███▋      | 1.47G/3.95G [00:03<00:02, 1.12GB/s]
model-00001-of-00004.safetensors:  42%|████▏     | 1.67G/3.95G [00:03<00:02, 1.06GB/s]
model-00001-of-00004.safetensors:  46%|████▋     | 1.84G/3.95G [00:04<00:02, 988MB/s] 
model-00001-of-00004.safetensors:  50%|█████     | 1.98G/3.95G [00:04<00:02, 966MB/s]
model-00001-of-00004.safetensors:  53%|█████▎    | 2.11G/3.95G [00:04<00:01, 1.01GB/s]
model-00001-of-00004.safetensors:  56%|█████▋    | 2.23G/3.95G [00:04<00:01, 1.02GB/s]
model-00001-of-00004.safetensors:  60%|█████▉    | 2.37G/3.95G [00:04<00:01, 1.06GB/s]
model-00001-of-00004.safetensors:  63%|██████▎   | 2.50G/3.95G [00:05<00:02, 604MB/s] 
model-00001-of-00004.safetensors:  66%|██████▌   | 2.60G/3.95G [00:05<00:02, 669MB/s]
model-00001-of-00004.safetensors:  72%|███████▏  | 2.83G/3.95G [00:05<00:01, 950MB/s]
model-00001-of-00004.safetensors:  81%|████████  | 3.21G/3.95G [00:05<00:00, 1.50GB/s]
model-00001-of-00004.safetensors:  89%|████████▉ | 3.52G/3.95G [00:05<00:00, 1.85GB/s]
model-00001-of-00004.safetensors:  97%|█████████▋| 3.82G/3.95G [00:05<00:00, 2.09GB/s]
model-00001-of-00004.safetensors: 100%|█████████▉| 3.95G/3.95G [00:05<00:00, 701MB/s] 
nousresearch-hermes-2-pr-1418-v3-mkmlizer: 
model-00002-of-00004.safetensors:   0%|          | 0.00/3.93G [00:00<?, ?B/s]
model-00002-of-00004.safetensors:   0%|          | 10.5M/3.93G [00:01<10:23, 6.28MB/s]
model-00002-of-00004.safetensors:   1%|          | 21.0M/3.93G [00:03<09:13, 7.05MB/s]
model-00002-of-00004.safetensors:   1%|          | 31.5M/3.93G [00:03<05:28, 11.9MB/s]
model-00002-of-00004.safetensors:   2%|▏         | 83.9M/3.93G [00:03<01:27, 44.0MB/s]
model-00002-of-00004.safetensors:   6%|▋         | 252M/3.93G [00:03<00:20, 181MB/s]  
model-00002-of-00004.safetensors:  13%|█▎        | 493M/3.93G [00:03<00:08, 420MB/s]
model-00002-of-00004.safetensors:  30%|██▉       | 1.16G/3.93G [00:03<00:02, 1.22GB/s]
model-00002-of-00004.safetensors:  37%|███▋      | 1.44G/3.93G [00:04<00:02, 876MB/s] 
model-00002-of-00004.safetensors:  42%|████▏     | 1.64G/3.93G [00:04<00:02, 971MB/s]
model-00002-of-00004.safetensors:  46%|████▋     | 1.82G/3.93G [00:04<00:01, 1.06GB/s]
model-00002-of-00004.safetensors:  51%|█████     | 2.00G/3.93G [00:04<00:01, 1.08GB/s]
model-00002-of-00004.safetensors:  55%|█████▌    | 2.16G/3.93G [00:04<00:01, 1.11GB/s]
model-00002-of-00004.safetensors:  60%|█████▉    | 2.35G/3.93G [00:04<00:01, 1.24GB/s]
model-00002-of-00004.safetensors:  64%|██████▍   | 2.51G/3.93G [00:05<00:01, 1.17GB/s]
model-00002-of-00004.safetensors:  68%|██████▊   | 2.65G/3.93G [00:05<00:01, 917MB/s] 
model-00002-of-00004.safetensors:  71%|███████▏  | 2.80G/3.93G [00:05<00:01, 1.01GB/s]
model-00002-of-00004.safetensors:  81%|████████  | 3.18G/3.93G [00:05<00:00, 1.56GB/s]
model-00002-of-00004.safetensors:  88%|████████▊ | 3.46G/3.93G [00:05<00:00, 1.83GB/s]
model-00002-of-00004.safetensors:  98%|█████████▊| 3.83G/3.93G [00:05<00:00, 2.26GB/s]
model-00002-of-00004.safetensors: 100%|█████████▉| 3.93G/3.93G [00:05<00:00, 668MB/s] 
nousresearch-hermes-2-pr-1418-v3-mkmlizer: 
model-00003-of-00004.safetensors:   0%|          | 0.00/3.93G [00:00<?, ?B/s]
model-00003-of-00004.safetensors:   0%|          | 10.5M/3.93G [00:01<10:11, 6.40MB/s]
model-00003-of-00004.safetensors:   1%|          | 21.0M/3.93G [00:02<08:05, 8.04MB/s]
model-00003-of-00004.safetensors:   1%|          | 31.5M/3.93G [00:02<05:07, 12.7MB/s]
model-00003-of-00004.safetensors:   3%|▎         | 136M/3.93G [00:03<00:42, 88.3MB/s] 
model-00003-of-00004.safetensors:   5%|▌         | 199M/3.93G [00:03<00:27, 133MB/s] 
model-00003-of-00004.safetensors:   6%|▋         | 252M/3.93G [00:03<00:21, 174MB/s]
model-00003-of-00004.safetensors:  15%|█▌        | 598M/3.93G [00:03<00:05, 623MB/s]
model-00003-of-00004.safetensors:  29%|██▉       | 1.14G/3.93G [00:03<00:01, 1.40GB/s]
model-00003-of-00004.safetensors:  36%|███▌      | 1.41G/3.93G [00:03<00:02, 1.20GB/s]
model-00003-of-00004.safetensors:  41%|████      | 1.61G/3.93G [00:03<00:01, 1.30GB/s]
model-00003-of-00004.safetensors:  46%|████▌     | 1.81G/3.93G [00:04<00:01, 1.38GB/s]
model-00003-of-00004.safetensors:  51%|█████     | 2.00G/3.93G [00:04<00:01, 1.37GB/s]
model-00003-of-00004.safetensors:  56%|█████▌    | 2.18G/3.93G [00:04<00:01, 1.36GB/s]
model-00003-of-00004.safetensors:  60%|█████▉    | 2.35G/3.93G [00:04<00:01, 1.30GB/s]
model-00003-of-00004.safetensors:  64%|██████▎   | 2.50G/3.93G [00:04<00:01, 1.27GB/s]
model-00003-of-00004.safetensors:  67%|██████▋   | 2.64G/3.93G [00:04<00:01, 1.08GB/s]
model-00003-of-00004.safetensors:  71%|███████   | 2.77G/3.93G [00:04<00:01, 1.02GB/s]
model-00003-of-00004.safetensors:  74%|███████▎  | 2.89G/3.93G [00:05<00:00, 1.06GB/s]
model-00003-of-00004.safetensors:  78%|███████▊  | 3.07G/3.93G [00:05<00:00, 1.23GB/s]
model-00003-of-00004.safetensors:  85%|████████▍ | 3.33G/3.93G [00:05<00:00, 1.56GB/s]
model-00003-of-00004.safetensors:  99%|█████████▉| 3.88G/3.93G [00:05<00:00, 2.58GB/s]
model-00003-of-00004.safetensors: 100%|█████████▉| 3.93G/3.93G [00:05<00:00, 722MB/s] 
nousresearch-hermes-2-pr-1418-v3-mkmlizer: 
model-00004-of-00004.safetensors:   0%|          | 0.00/2.68G [00:00<?, ?B/s]
model-00004-of-00004.safetensors:   0%|          | 10.5M/2.68G [00:01<06:31, 6.82MB/s]
model-00004-of-00004.safetensors:   1%|          | 21.0M/2.68G [00:02<06:10, 7.18MB/s]
model-00004-of-00004.safetensors:   9%|▊         | 231M/2.68G [00:03<00:20, 119MB/s]  
model-00004-of-00004.safetensors:  12%|█▏        | 315M/2.68G [00:03<00:16, 142MB/s]
model-00004-of-00004.safetensors:  23%|██▎       | 629M/2.68G [00:03<00:05, 375MB/s]
model-00004-of-00004.safetensors:  43%|████▎     | 1.14G/2.68G [00:03<00:01, 840MB/s]
model-00004-of-00004.safetensors:  53%|█████▎    | 1.43G/2.68G [00:03<00:01, 1.04GB/s]
model-00004-of-00004.safetensors:  62%|██████▏   | 1.67G/2.68G [00:04<00:01, 989MB/s] 
model-00004-of-00004.safetensors:  69%|██████▉   | 1.86G/2.68G [00:04<00:00, 951MB/s]
model-00004-of-00004.safetensors:  75%|███████▌  | 2.01G/2.68G [00:04<00:00, 1.01GB/s]
model-00004-of-00004.safetensors:  81%|████████  | 2.16G/2.68G [00:04<00:00, 829MB/s] 
model-00004-of-00004.safetensors:  86%|████████▌ | 2.30G/2.68G [00:04<00:00, 891MB/s]
model-00004-of-00004.safetensors:  91%|█████████ | 2.43G/2.68G [00:04<00:00, 955MB/s]
model-00004-of-00004.safetensors: 100%|█████████▉| 2.68G/2.68G [00:05<00:00, 532MB/s]
nousresearch-hermes-2-pr-1418-v3-mkmlizer: 
model.safetensors.index.json:   0%|          | 0.00/23.9k [00:00<?, ?B/s]
model.safetensors.index.json: 100%|██████████| 23.9k/23.9k [00:00<00:00, 6.06MB/s]
nousresearch-hermes-2-pr-1418-v3-mkmlizer: 
special_tokens_map.json:   0%|          | 0.00/557 [00:00<?, ?B/s]
special_tokens_map.json: 100%|██████████| 557/557 [00:00<00:00, 6.08MB/s]
nousresearch-hermes-2-pr-1418-v3-mkmlizer: 
tokenizer_config.json:   0%|          | 0.00/1.60k [00:00<?, ?B/s]
tokenizer_config.json: 100%|██████████| 1.60k/1.60k [00:00<00:00, 20.8MB/s]
nousresearch-hermes-2-pr-1418-v3-mkmlizer: Downloaded to shared memory in 26.839s
nousresearch-hermes-2-pr-1418-v3-mkmlizer: quantizing model to /dev/shm/model_cache
nousresearch-hermes-2-pr-1418-v3-mkmlizer: Saving mkml model at /dev/shm/model_cache
nousresearch-hermes-2-pr-1418-v3-mkmlizer: Reading /tmp/tmp4i8cdb0s/model.safetensors.index.json
nousresearch-hermes-2-pr-1418-v3-mkmlizer: 
Profiling:   0%|          | 0/291 [00:00<?, ?it/s]
Profiling:   0%|          | 1/291 [00:01<05:47,  1.20s/it]
Profiling:   5%|▍         | 14/291 [00:01<00:19, 14.52it/s]
Profiling:  11%|█         | 32/291 [00:01<00:07, 35.57it/s]
Profiling:  17%|█▋        | 50/291 [00:01<00:04, 56.83it/s]
Profiling:  23%|██▎       | 68/291 [00:01<00:02, 76.78it/s]
Profiling:  28%|██▊       | 82/291 [00:01<00:03, 67.26it/s]
Profiling:  34%|███▍      | 99/291 [00:02<00:02, 84.53it/s]
Profiling:  40%|███▉      | 116/291 [00:02<00:01, 100.62it/s]
Profiling:  45%|████▍     | 130/291 [00:02<00:01, 109.23it/s]
Profiling:  51%|█████     | 148/291 [00:02<00:01, 123.49it/s]
Profiling:  56%|█████▌    | 163/291 [00:02<00:01, 89.11it/s] 
Profiling:  60%|██████    | 176/291 [00:02<00:01, 94.80it/s]
Profiling:  66%|██████▋   | 193/291 [00:02<00:00, 109.05it/s]
Profiling:  72%|███████▏  | 210/291 [00:02<00:00, 120.80it/s]
Profiling:  77%|███████▋  | 224/291 [00:03<00:00, 124.54it/s]
Profiling:  83%|████████▎ | 241/291 [00:04<00:01, 29.80it/s] 
Profiling:  88%|████████▊ | 256/291 [00:04<00:00, 38.55it/s]
Profiling:  93%|█████████▎| 270/291 [00:04<00:00, 48.23it/s]
Profiling:  98%|█████████▊| 285/291 [00:04<00:00, 59.45it/s]
Profiling: 100%|██████████| 291/291 [00:04<00:00, 58.42it/s]
nousresearch-hermes-2-pr-1418-v3-mkmlizer: Special tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained.
nousresearch-hermes-2-pr-1418-v3-mkmlizer: Special tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained.
nousresearch-hermes-2-pr-1418-v3-mkmlizer: quantized model in 16.726s
nousresearch-hermes-2-pr-1418-v3-mkmlizer: Processed model NousResearch/Hermes-2-Pro-Mistral-7B in 44.448s
nousresearch-hermes-2-pr-1418-v3-mkmlizer: creating bucket guanaco-mkml-models
nousresearch-hermes-2-pr-1418-v3-mkmlizer: Bucket 's3://guanaco-mkml-models/' created
nousresearch-hermes-2-pr-1418-v3-mkmlizer: uploading /dev/shm/model_cache to s3://guanaco-mkml-models/nousresearch-hermes-2-pr-1418-v3
nousresearch-hermes-2-pr-1418-v3-mkmlizer: cp /dev/shm/model_cache/config.json s3://guanaco-mkml-models/nousresearch-hermes-2-pr-1418-v3/config.json
nousresearch-hermes-2-pr-1418-v3-mkmlizer: cp /dev/shm/model_cache/special_tokens_map.json s3://guanaco-mkml-models/nousresearch-hermes-2-pr-1418-v3/special_tokens_map.json
nousresearch-hermes-2-pr-1418-v3-mkmlizer: cp /dev/shm/model_cache/tokenizer_config.json s3://guanaco-mkml-models/nousresearch-hermes-2-pr-1418-v3/tokenizer_config.json
nousresearch-hermes-2-pr-1418-v3-mkmlizer: cp /dev/shm/model_cache/added_tokens.json s3://guanaco-mkml-models/nousresearch-hermes-2-pr-1418-v3/added_tokens.json
nousresearch-hermes-2-pr-1418-v3-mkmlizer: cp /dev/shm/model_cache/tokenizer.model s3://guanaco-mkml-models/nousresearch-hermes-2-pr-1418-v3/tokenizer.model
nousresearch-hermes-2-pr-1418-v3-mkmlizer: cp /dev/shm/model_cache/tokenizer.json s3://guanaco-mkml-models/nousresearch-hermes-2-pr-1418-v3/tokenizer.json
nousresearch-hermes-2-pr-1418-v3-mkmlizer: cp /dev/shm/model_cache/mkml_model.tensors s3://guanaco-mkml-models/nousresearch-hermes-2-pr-1418-v3/mkml_model.tensors
nousresearch-hermes-2-pr-1418-v3-mkmlizer: loading reward model from anhnv125/reward-model-v3
nousresearch-hermes-2-pr-1418-v3-mkmlizer: /opt/conda/lib/python3.10/site-packages/transformers/models/auto/configuration_auto.py:1067: FutureWarning: The `use_auth_token` argument is deprecated and will be removed in v5 of Transformers. Please use `token` instead.
nousresearch-hermes-2-pr-1418-v3-mkmlizer:   warnings.warn(
nousresearch-hermes-2-pr-1418-v3-mkmlizer: 
config.json:   0%|          | 0.00/1.04k [00:00<?, ?B/s]
config.json: 100%|██████████| 1.04k/1.04k [00:00<00:00, 12.8MB/s]
nousresearch-hermes-2-pr-1418-v3-mkmlizer: /opt/conda/lib/python3.10/site-packages/transformers/models/auto/tokenization_auto.py:690: FutureWarning: The `use_auth_token` argument is deprecated and will be removed in v5 of Transformers. Please use `token` instead.
nousresearch-hermes-2-pr-1418-v3-mkmlizer:   warnings.warn(
nousresearch-hermes-2-pr-1418-v3-mkmlizer: 
tokenizer_config.json:   0%|          | 0.00/477 [00:00<?, ?B/s]
tokenizer_config.json: 100%|██████████| 477/477 [00:00<00:00, 4.81MB/s]
nousresearch-hermes-2-pr-1418-v3-mkmlizer: 
vocab.json:   0%|          | 0.00/798k [00:00<?, ?B/s]
vocab.json: 100%|██████████| 798k/798k [00:00<00:00, 3.51MB/s]
vocab.json: 100%|██████████| 798k/798k [00:00<00:00, 3.50MB/s]
nousresearch-hermes-2-pr-1418-v3-mkmlizer: 
merges.txt:   0%|          | 0.00/456k [00:00<?, ?B/s]
merges.txt: 100%|██████████| 456k/456k [00:00<00:00, 4.35MB/s]
merges.txt: 100%|██████████| 456k/456k [00:00<00:00, 4.34MB/s]
nousresearch-hermes-2-pr-1418-v3-mkmlizer: 
tokenizer.json:   0%|          | 0.00/2.11M [00:00<?, ?B/s]
tokenizer.json: 100%|██████████| 2.11M/2.11M [00:00<00:00, 13.2MB/s]
tokenizer.json: 100%|██████████| 2.11M/2.11M [00:00<00:00, 13.1MB/s]
nousresearch-hermes-2-pr-1418-v3-mkmlizer: 
special_tokens_map.json:   0%|          | 0.00/131 [00:00<?, ?B/s]
special_tokens_map.json: 100%|██████████| 131/131 [00:00<00:00, 1.53MB/s]
nousresearch-hermes-2-pr-1418-v3-mkmlizer: /opt/conda/lib/python3.10/site-packages/transformers/models/auto/auto_factory.py:472: FutureWarning: The `use_auth_token` argument is deprecated and will be removed in v5 of Transformers. Please use `token` instead.
nousresearch-hermes-2-pr-1418-v3-mkmlizer:   warnings.warn(
nousresearch-hermes-2-pr-1418-v3-mkmlizer: creating bucket guanaco-reward-models
nousresearch-hermes-2-pr-1418-v3-mkmlizer: Bucket 's3://guanaco-reward-models/' created
nousresearch-hermes-2-pr-1418-v3-mkmlizer: uploading /tmp/reward_cache to s3://guanaco-reward-models/nousresearch-hermes-2-pr-1418-v3_reward
nousresearch-hermes-2-pr-1418-v3-mkmlizer: cp /tmp/reward_cache/config.json s3://guanaco-reward-models/nousresearch-hermes-2-pr-1418-v3_reward/config.json
nousresearch-hermes-2-pr-1418-v3-mkmlizer: cp /tmp/reward_cache/special_tokens_map.json s3://guanaco-reward-models/nousresearch-hermes-2-pr-1418-v3_reward/special_tokens_map.json
nousresearch-hermes-2-pr-1418-v3-mkmlizer: cp /tmp/reward_cache/tokenizer_config.json s3://guanaco-reward-models/nousresearch-hermes-2-pr-1418-v3_reward/tokenizer_config.json
nousresearch-hermes-2-pr-1418-v3-mkmlizer: cp /tmp/reward_cache/merges.txt s3://guanaco-reward-models/nousresearch-hermes-2-pr-1418-v3_reward/merges.txt
nousresearch-hermes-2-pr-1418-v3-mkmlizer: cp /tmp/reward_cache/vocab.json s3://guanaco-reward-models/nousresearch-hermes-2-pr-1418-v3_reward/vocab.json
nousresearch-hermes-2-pr-1418-v3-mkmlizer: cp /tmp/reward_cache/tokenizer.json s3://guanaco-reward-models/nousresearch-hermes-2-pr-1418-v3_reward/tokenizer.json
nousresearch-hermes-2-pr-1418-v3-mkmlizer: cp /tmp/reward_cache/reward.tensors s3://guanaco-reward-models/nousresearch-hermes-2-pr-1418-v3_reward/reward.tensors
Job nousresearch-hermes-2-pr-1418-v3-mkmlizer completed after 75.06s with status: succeeded
Stopping job with name nousresearch-hermes-2-pr-1418-v3-mkmlizer
Pipeline stage MKMLizer completed in 80.39s
Running pipeline stage MKMLKubeTemplater
Pipeline stage MKMLKubeTemplater completed in 0.12s
Running pipeline stage ISVCDeployer
Creating inference service nousresearch-hermes-2-pr-1418-v3
Waiting for inference service nousresearch-hermes-2-pr-1418-v3 to be ready
Inference service nousresearch-hermes-2-pr-1418-v3 ready after 40.22454476356506s
Pipeline stage ISVCDeployer completed in 47.75s
Running pipeline stage StressChecker
Received healthy response to inference request in 1.7418687343597412s
Received healthy response to inference request in 1.3183355331420898s
Received healthy response to inference request in 1.6334929466247559s
Received healthy response to inference request in 1.1532270908355713s
Received healthy response to inference request in 1.156118392944336s
5 requests
0 failed requests
5th percentile: 1.1538053512573243
10th percentile: 1.154383611679077
20th percentile: 1.155540132522583
30th percentile: 1.1885618209838866
40th percentile: 1.2534486770629882
50th percentile: 1.3183355331420898
60th percentile: 1.4443984985351563
70th percentile: 1.5704614639282226
80th percentile: 1.655168104171753
90th percentile: 1.698518419265747
95th percentile: 1.720193576812744
99th percentile: 1.7375337028503417
mean time: 1.4006085395812988
Pipeline stage StressChecker completed in 8.00s
Running pipeline stage DaemonicModelEvalScorer
Pipeline stage DaemonicModelEvalScorer completed in 0.04s
Running pipeline stage DaemonicSafetyScorer
Running M-Eval for topic stay_in_character
Pipeline stage DaemonicSafetyScorer completed in 0.04s
M-Eval Dataset for topic stay_in_character is loaded
nousresearch-hermes-2-pr_1418_v3 status is now deployed due to DeploymentManager action
nousresearch-hermes-2-pr_1418_v3 status is now inactive due to auto deactivation removed underperforming models
admin requested tearing down of nousresearch-hermes-2-pr_1418_v3
Running pipeline stage ISVCDeleter
Checking if service nousresearch-hermes-2-pr-1418-v3 is running
Tearing down inference service nousresearch-hermes-2-pr-1418-v3
Toredown service nousresearch-hermes-2-pr-1418-v3
Pipeline stage ISVCDeleter completed in 4.15s
Running pipeline stage MKMLModelDeleter
Cleaning model data from S3
Cleaning model data from model cache
Deleting key nousresearch-hermes-2-pr-1418-v3/added_tokens.json from bucket guanaco-mkml-models
Deleting key nousresearch-hermes-2-pr-1418-v3/config.json from bucket guanaco-mkml-models
Deleting key nousresearch-hermes-2-pr-1418-v3/mkml_model.tensors from bucket guanaco-mkml-models
Deleting key nousresearch-hermes-2-pr-1418-v3/special_tokens_map.json from bucket guanaco-mkml-models
Deleting key nousresearch-hermes-2-pr-1418-v3/tokenizer.json from bucket guanaco-mkml-models
Deleting key nousresearch-hermes-2-pr-1418-v3/tokenizer.model from bucket guanaco-mkml-models
Deleting key nousresearch-hermes-2-pr-1418-v3/tokenizer_config.json from bucket guanaco-mkml-models
Cleaning model data from model cache
Deleting key nousresearch-hermes-2-pr-1418-v3_reward/config.json from bucket guanaco-reward-models
Deleting key nousresearch-hermes-2-pr-1418-v3_reward/merges.txt from bucket guanaco-reward-models
Deleting key nousresearch-hermes-2-pr-1418-v3_reward/reward.tensors from bucket guanaco-reward-models
Deleting key nousresearch-hermes-2-pr-1418-v3_reward/special_tokens_map.json from bucket guanaco-reward-models
Deleting key nousresearch-hermes-2-pr-1418-v3_reward/tokenizer.json from bucket guanaco-reward-models
Deleting key nousresearch-hermes-2-pr-1418-v3_reward/tokenizer_config.json from bucket guanaco-reward-models
Deleting key nousresearch-hermes-2-pr-1418-v3_reward/vocab.json from bucket guanaco-reward-models
Pipeline stage MKMLModelDeleter completed in 2.15s
nousresearch-hermes-2-pr_1418_v3 status is now torndown due to DeploymentManager action