locutusque-hercules-3-1-_3515

developer_uid: Locutusque
submission_id: locutusque-hercules-3-1-_3515_v1
model_name: locutusque-hercules-3-1-_3515_v1
model_group: Locutusque/Hercules-3.1-
status: torndown
timestamp: 2024-03-31T23:10:38+00:00
num_battles: 10361
num_wins: 5041
celo_rating: 1147.82
family_friendly_score: 0.0
submission_type: basic
model_repo: Locutusque/Hercules-3.1-Mistral-7B
model_architecture: MistralForCausalLM
reward_repo: ChaiML/reward_gpt2_medium_preference_24m_e2
model_num_parameters: 7241732096.0
best_of: 4
max_input_tokens: 1536
max_output_tokens: 64
display_name: locutusque-hercules-3-1-_3515_v1
is_internal_developer: False
language_model: Locutusque/Hercules-3.1-Mistral-7B
model_size: 7B
ranking_group: single
us_pacific_date: 2024-03-31
win_ratio: 0.4865360486439533
generation_params: {'temperature': 0.8, 'top_p': 0.5, 'min_p': 0.0, 'top_k': 40, 'presence_penalty': 0.0, 'frequency_penalty': 0.15, 'stopping_words': ['<\\s>', '<|im_end|>'], 'max_input_tokens': 1536, 'best_of': 4, 'max_output_tokens': 64}
formatter: {'memory_template': '<|im_start|>system\nYou are now in roleplay mode. You are to behave as {bot_name}. Persona: {memory}<|im_end|>\n', 'prompt_template': '<|im_start|>user\n{prompt}<¦im_end|>\n', 'bot_template': '<|im_start|>assistant\n{bot_name}: {message}</s>', 'user_template': '<|im_start|>user\n{user_name}: {message}<¦im_end|>\n', 'response_template': '<|im_start|>assistant\n{bot_name}:', 'truncate_by_message': False}
reward_formatter: {'bot_template': '{bot_name}: {message}\n', 'memory_template': "{bot_name}'s Persona: {memory}\n####\n", 'prompt_template': '{prompt}\n<START>\n', 'response_template': '{bot_name}:', 'truncate_by_message': False, 'user_template': '{user_name}: {message}\n'}
model_eval_status: success
Resubmit model
Running pipeline stage MKMLizer
Starting job with name locutusque-hercules-3-1-3515-v1-mkmlizer
Waiting for job on locutusque-hercules-3-1-3515-v1-mkmlizer to finish
locutusque-hercules-3-1-3515-v1-mkmlizer: ╔═════════════════════════════════════════════════════════════════════╗
locutusque-hercules-3-1-3515-v1-mkmlizer: ║     _____            __           __                                ║
locutusque-hercules-3-1-3515-v1-mkmlizer: ║    / _/ /_ ___    __/ /  ___ ___ / /                                ║
locutusque-hercules-3-1-3515-v1-mkmlizer: ║   / _/ / // / |/|/ / _ \/ -_) -_) /                                 ║
locutusque-hercules-3-1-3515-v1-mkmlizer: ║  /_//_/\_, /|__,__/_//_/\__/\__/_/                                  ║
locutusque-hercules-3-1-3515-v1-mkmlizer: ║       /___/                                                         ║
locutusque-hercules-3-1-3515-v1-mkmlizer: ║                                                                     ║
locutusque-hercules-3-1-3515-v1-mkmlizer: ║  Version: 0.6.11                                                    ║
locutusque-hercules-3-1-3515-v1-mkmlizer: ║  Copyright 2023 MK ONE TECHNOLOGIES Inc.                            ║
locutusque-hercules-3-1-3515-v1-mkmlizer: ║                                                                     ║
locutusque-hercules-3-1-3515-v1-mkmlizer: ║  The license key for the current software has been verified as      ║
locutusque-hercules-3-1-3515-v1-mkmlizer: ║  belonging to:                                                      ║
locutusque-hercules-3-1-3515-v1-mkmlizer: ║                                                                     ║
locutusque-hercules-3-1-3515-v1-mkmlizer: ║  Chai Research Corp.                                                ║
locutusque-hercules-3-1-3515-v1-mkmlizer: ║  Account ID: 7997a29f-0ceb-4cc7-9adf-840c57b4ae6f                   ║
locutusque-hercules-3-1-3515-v1-mkmlizer: ║  Expiration: 2024-07-15 23:59:59                                    ║
locutusque-hercules-3-1-3515-v1-mkmlizer: ║                                                                     ║
locutusque-hercules-3-1-3515-v1-mkmlizer: ╚═════════════════════════════════════════════════════════════════════╝
locutusque-hercules-3-1-3515-v1-mkmlizer: 
.gitattributes:   0%|          | 0.00/1.52k [00:00<?, ?B/s]
.gitattributes: 100%|██████████| 1.52k/1.52k [00:00<00:00, 12.3MB/s]
locutusque-hercules-3-1-3515-v1-mkmlizer: 
README.md:   0%|          | 0.00/7.98k [00:00<?, ?B/s]
README.md: 100%|██████████| 7.98k/7.98k [00:00<00:00, 67.0MB/s]
locutusque-hercules-3-1-3515-v1-mkmlizer: 
config.json:   0%|          | 0.00/642 [00:00<?, ?B/s]
config.json: 100%|██████████| 642/642 [00:00<00:00, 6.35MB/s]
locutusque-hercules-3-1-3515-v1-mkmlizer: 
generation_config.json:   0%|          | 0.00/111 [00:00<?, ?B/s]
generation_config.json: 100%|██████████| 111/111 [00:00<00:00, 1.49MB/s]
locutusque-hercules-3-1-3515-v1-mkmlizer: 
model-00001-of-00008.safetensors:   0%|          | 0.00/1.89G [00:00<?, ?B/s]
model-00001-of-00008.safetensors:   1%|          | 10.5M/1.89G [00:01<03:45, 8.32MB/s]
model-00001-of-00008.safetensors:   1%|          | 21.0M/1.89G [00:01<02:33, 12.2MB/s]
model-00001-of-00008.safetensors:   2%|▏         | 41.9M/1.89G [00:02<01:16, 24.1MB/s]
model-00001-of-00008.safetensors:   3%|▎         | 62.9M/1.89G [00:02<00:46, 39.3MB/s]
model-00001-of-00008.safetensors:   4%|▍         | 73.4M/1.89G [00:02<00:52, 34.8MB/s]
model-00001-of-00008.safetensors:   5%|▍         | 94.4M/1.89G [00:02<00:34, 52.1MB/s]
model-00001-of-00008.safetensors:   6%|▌         | 115M/1.89G [00:03<00:27, 65.7MB/s] 
model-00001-of-00008.safetensors:   8%|▊         | 147M/1.89G [00:03<00:20, 85.8MB/s]
model-00001-of-00008.safetensors:  14%|█▍        | 262M/1.89G [00:03<00:06, 243MB/s] 
model-00001-of-00008.safetensors:  19%|█▉        | 367M/1.89G [00:03<00:04, 311MB/s]
model-00001-of-00008.safetensors:  23%|██▎       | 440M/1.89G [00:03<00:04, 362MB/s]
model-00001-of-00008.safetensors:  28%|██▊       | 535M/1.89G [00:03<00:02, 468MB/s]
model-00001-of-00008.safetensors:  62%|██████▏   | 1.16G/1.89G [00:03<00:00, 1.67GB/s]
model-00001-of-00008.safetensors:  74%|███████▍  | 1.40G/1.89G [00:04<00:00, 824MB/s] 
model-00001-of-00008.safetensors:  83%|████████▎ | 1.58G/1.89G [00:04<00:00, 706MB/s]
model-00001-of-00008.safetensors:  91%|█████████ | 1.71G/1.89G [00:05<00:00, 633MB/s]
model-00001-of-00008.safetensors: 100%|█████████▉| 1.89G/1.89G [00:05<00:00, 353MB/s]
locutusque-hercules-3-1-3515-v1-mkmlizer: 
model-00002-of-00008.safetensors:   0%|          | 0.00/1.95G [00:00<?, ?B/s]
model-00002-of-00008.safetensors:   1%|          | 10.5M/1.95G [00:00<02:32, 12.7MB/s]
model-00002-of-00008.safetensors:   1%|          | 21.0M/1.95G [00:01<01:26, 22.3MB/s]
model-00002-of-00008.safetensors:   2%|▏         | 31.5M/1.95G [00:01<01:06, 28.9MB/s]
model-00002-of-00008.safetensors:   3%|▎         | 62.9M/1.95G [00:01<00:26, 70.1MB/s]
model-00002-of-00008.safetensors:   5%|▌         | 105M/1.95G [00:01<00:14, 129MB/s]  
model-00002-of-00008.safetensors:   8%|▊         | 147M/1.95G [00:01<00:10, 173MB/s]
model-00002-of-00008.safetensors:  10%|▉         | 189M/1.95G [00:01<00:08, 217MB/s]
model-00002-of-00008.safetensors:  12%|█▏        | 231M/1.95G [00:01<00:06, 256MB/s]
model-00002-of-00008.safetensors:  14%|█▍        | 273M/1.95G [00:02<00:06, 275MB/s]
model-00002-of-00008.safetensors:  16%|█▌        | 315M/1.95G [00:02<00:05, 279MB/s]
model-00002-of-00008.safetensors:  19%|█▉        | 377M/1.95G [00:02<00:05, 307MB/s]
model-00002-of-00008.safetensors:  22%|██▏       | 419M/1.95G [00:02<00:04, 317MB/s]
model-00002-of-00008.safetensors:  24%|██▍       | 472M/1.95G [00:02<00:04, 340MB/s]
model-00002-of-00008.safetensors:  26%|██▋       | 514M/1.95G [00:02<00:04, 341MB/s]
model-00002-of-00008.safetensors:  30%|███       | 587M/1.95G [00:02<00:03, 432MB/s]
model-00002-of-00008.safetensors:  37%|███▋      | 724M/1.95G [00:02<00:01, 653MB/s]
model-00002-of-00008.safetensors:  41%|████      | 797M/1.95G [00:03<00:01, 647MB/s]
model-00002-of-00008.safetensors:  45%|████▍     | 870M/1.95G [00:03<00:01, 610MB/s]
model-00002-of-00008.safetensors:  70%|███████   | 1.36G/1.95G [00:03<00:00, 1.71GB/s]
model-00002-of-00008.safetensors:  80%|████████  | 1.56G/1.95G [00:03<00:00, 1.61GB/s]
model-00002-of-00008.safetensors:  90%|████████▉ | 1.75G/1.95G [00:03<00:00, 923MB/s] 
model-00002-of-00008.safetensors:  97%|█████████▋| 1.89G/1.95G [00:03<00:00, 1.00GB/s]
model-00002-of-00008.safetensors: 100%|█████████▉| 1.95G/1.95G [00:04<00:00, 483MB/s] 
locutusque-hercules-3-1-3515-v1-mkmlizer: 
model-00003-of-00008.safetensors:   0%|          | 0.00/1.98G [00:00<?, ?B/s]
model-00003-of-00008.safetensors:   1%|          | 10.5M/1.98G [00:00<02:37, 12.5MB/s]
model-00003-of-00008.safetensors:   1%|          | 21.0M/1.98G [00:01<02:05, 15.6MB/s]
model-00003-of-00008.safetensors:   2%|▏         | 41.9M/1.98G [00:01<00:56, 34.0MB/s]
model-00003-of-00008.safetensors:   4%|▍         | 83.9M/1.98G [00:01<00:23, 82.1MB/s]
model-00003-of-00008.safetensors:   7%|▋         | 136M/1.98G [00:01<00:14, 127MB/s]  
model-00003-of-00008.safetensors:   8%|▊         | 168M/1.98G [00:02<00:11, 156MB/s]
model-00003-of-00008.safetensors:  11%|█         | 210M/1.98G [00:02<00:08, 202MB/s]
model-00003-of-00008.safetensors:  13%|█▎        | 252M/1.98G [00:02<00:07, 236MB/s]
model-00003-of-00008.safetensors:  14%|█▍        | 283M/1.98G [00:02<00:08, 203MB/s]
model-00003-of-00008.safetensors:  17%|█▋        | 336M/1.98G [00:02<00:06, 265MB/s]
model-00003-of-00008.safetensors:  19%|█▉        | 377M/1.98G [00:02<00:06, 234MB/s]
model-00003-of-00008.safetensors:  21%|██        | 409M/1.98G [00:02<00:06, 249MB/s]
model-00003-of-00008.safetensors:  22%|██▏       | 440M/1.98G [00:02<00:05, 260MB/s]
model-00003-of-00008.safetensors:  25%|██▌       | 503M/1.98G [00:03<00:04, 327MB/s]
model-00003-of-00008.safetensors:  31%|███       | 608M/1.98G [00:03<00:02, 500MB/s]
model-00003-of-00008.safetensors:  34%|███▍      | 682M/1.98G [00:03<00:02, 507MB/s]
model-00003-of-00008.safetensors:  38%|███▊      | 744M/1.98G [00:03<00:02, 536MB/s]
model-00003-of-00008.safetensors:  43%|████▎     | 849M/1.98G [00:03<00:01, 668MB/s]
model-00003-of-00008.safetensors:  77%|███████▋  | 1.52G/1.98G [00:03<00:00, 2.29GB/s]
model-00003-of-00008.safetensors:  89%|████████▉ | 1.77G/1.98G [00:04<00:00, 1.22GB/s]
model-00003-of-00008.safetensors:  99%|█████████▉| 1.96G/1.98G [00:04<00:00, 1.08GB/s]
model-00003-of-00008.safetensors: 100%|█████████▉| 1.98G/1.98G [00:04<00:00, 425MB/s] 
locutusque-hercules-3-1-3515-v1-mkmlizer: 
model-00004-of-00008.safetensors:   0%|          | 0.00/1.95G [00:00<?, ?B/s]
model-00004-of-00008.safetensors:   1%|          | 10.5M/1.95G [00:00<01:31, 21.1MB/s]
model-00004-of-00008.safetensors:   1%|          | 21.0M/1.95G [00:01<01:33, 20.5MB/s]
model-00004-of-00008.safetensors:   2%|▏         | 41.9M/1.95G [00:01<00:41, 45.4MB/s]
model-00004-of-00008.safetensors:   3%|▎         | 52.4M/1.95G [00:01<00:41, 46.1MB/s]
model-00004-of-00008.safetensors:   4%|▍         | 73.4M/1.95G [00:01<00:25, 72.2MB/s]
model-00004-of-00008.safetensors:   6%|▌         | 115M/1.95G [00:01<00:14, 125MB/s]  
model-00004-of-00008.safetensors:   8%|▊         | 157M/1.95G [00:01<00:10, 177MB/s]
model-00004-of-00008.safetensors:  11%|█         | 210M/1.95G [00:01<00:07, 233MB/s]
model-00004-of-00008.safetensors:  12%|█▏        | 241M/1.95G [00:01<00:06, 251MB/s]
model-00004-of-00008.safetensors:  15%|█▌        | 294M/1.95G [00:02<00:05, 314MB/s]
model-00004-of-00008.safetensors:  17%|█▋        | 336M/1.95G [00:02<00:05, 299MB/s]
model-00004-of-00008.safetensors:  19%|█▉        | 377M/1.95G [00:02<00:05, 290MB/s]
model-00004-of-00008.safetensors:  22%|██▏       | 419M/1.95G [00:02<00:04, 310MB/s]
model-00004-of-00008.safetensors:  24%|██▎       | 461M/1.95G [00:02<00:04, 324MB/s]
model-00004-of-00008.safetensors:  26%|██▌       | 503M/1.95G [00:02<00:04, 318MB/s]
model-00004-of-00008.safetensors:  31%|███       | 608M/1.95G [00:02<00:02, 489MB/s]
model-00004-of-00008.safetensors:  35%|███▌      | 682M/1.95G [00:02<00:02, 533MB/s]
model-00004-of-00008.safetensors:  38%|███▊      | 744M/1.95G [00:03<00:02, 533MB/s]
model-00004-of-00008.safetensors:  41%|████▏     | 807M/1.95G [00:03<00:02, 552MB/s]
model-00004-of-00008.safetensors:  47%|████▋     | 912M/1.95G [00:03<00:01, 662MB/s]
model-00004-of-00008.safetensors:  79%|███████▊  | 1.53G/1.95G [00:03<00:00, 2.11GB/s]
model-00004-of-00008.safetensors:  90%|████████▉ | 1.75G/1.95G [00:03<00:00, 1.30GB/s]
model-00004-of-00008.safetensors: 100%|█████████▉| 1.95G/1.95G [00:03<00:00, 497MB/s] 
locutusque-hercules-3-1-3515-v1-mkmlizer: 
model-00006-of-00008.safetensors:   0%|          | 0.00/1.95G [00:00<?, ?B/s]
model-00006-of-00008.safetensors:   1%|          | 10.5M/1.95G [00:00<01:57, 16.5MB/s]
model-00006-of-00008.safetensors:   2%|▏         | 31.5M/1.95G [00:01<01:04, 29.7MB/s]
model-00006-of-00008.safetensors:   3%|▎         | 52.4M/1.95G [00:01<00:35, 52.9MB/s]
model-00006-of-00008.safetensors:   4%|▍         | 73.4M/1.95G [00:01<00:25, 73.6MB/s]
model-00006-of-00008.safetensors:   6%|▌         | 115M/1.95G [00:01<00:14, 125MB/s]  
model-00006-of-00008.safetensors:   8%|▊         | 157M/1.95G [00:01<00:10, 172MB/s]
model-00006-of-00008.safetensors:  10%|▉         | 189M/1.95G [00:01<00:08, 201MB/s]
model-00006-of-00008.safetensors:  11%|█▏        | 220M/1.95G [00:01<00:08, 202MB/s]
model-00006-of-00008.safetensors:  14%|█▍        | 273M/1.95G [00:02<00:06, 268MB/s]
model-00006-of-00008.safetensors:  16%|█▌        | 315M/1.95G [00:02<00:05, 302MB/s]
model-00006-of-00008.safetensors:  18%|█▊        | 357M/1.95G [00:02<00:06, 251MB/s]
model-00006-of-00008.safetensors:  20%|█▉        | 388M/1.95G [00:02<00:06, 251MB/s]
model-00006-of-00008.safetensors:  22%|██▏       | 430M/1.95G [00:02<00:05, 288MB/s]
model-00006-of-00008.safetensors:  25%|██▌       | 493M/1.95G [00:02<00:04, 349MB/s]
model-00006-of-00008.safetensors:  28%|██▊       | 545M/1.95G [00:02<00:03, 386MB/s]
model-00006-of-00008.safetensors:  32%|███▏      | 629M/1.95G [00:02<00:03, 422MB/s]
model-00006-of-00008.safetensors:  35%|███▌      | 682M/1.95G [00:03<00:02, 435MB/s]
model-00006-of-00008.safetensors:  39%|███▉      | 765M/1.95G [00:03<00:02, 534MB/s]
model-00006-of-00008.safetensors:  73%|███████▎  | 1.42G/1.95G [00:03<00:00, 2.09GB/s]
model-00006-of-00008.safetensors:  86%|████████▌ | 1.68G/1.95G [00:03<00:00, 1.90GB/s]
model-00006-of-00008.safetensors:  97%|█████████▋| 1.89G/1.95G [00:03<00:00, 1.62GB/s]
model-00006-of-00008.safetensors: 100%|█████████▉| 1.95G/1.95G [00:04<00:00, 468MB/s] 
locutusque-hercules-3-1-3515-v1-mkmlizer: 
model-00007-of-00008.safetensors:   0%|          | 0.00/1.98G [00:00<?, ?B/s]
model-00007-of-00008.safetensors:   1%|          | 10.5M/1.98G [00:01<03:21, 9.79MB/s]
model-00007-of-00008.safetensors:   1%|          | 21.0M/1.98G [00:01<01:47, 18.2MB/s]
model-00007-of-00008.safetensors:   4%|▍         | 83.9M/1.98G [00:01<00:19, 95.1MB/s]
model-00007-of-00008.safetensors:   6%|▌         | 115M/1.98G [00:01<00:15, 119MB/s]  
model-00007-of-00008.safetensors:   8%|▊         | 157M/1.98G [00:01<00:14, 123MB/s]
model-00007-of-00008.safetensors:  14%|█▍        | 283M/1.98G [00:01<00:05, 286MB/s]
model-00007-of-00008.safetensors:  17%|█▋        | 336M/1.98G [00:02<00:06, 243MB/s]
model-00007-of-00008.safetensors:  22%|██▏       | 430M/1.98G [00:02<00:04, 328MB/s]
model-00007-of-00008.safetensors:  24%|██▍       | 482M/1.98G [00:02<00:06, 236MB/s]
model-00007-of-00008.safetensors:  29%|██▊       | 566M/1.98G [00:02<00:04, 308MB/s]
model-00007-of-00008.safetensors:  32%|███▏      | 629M/1.98G [00:03<00:03, 353MB/s]
model-00007-of-00008.safetensors:  34%|███▍      | 682M/1.98G [00:03<00:03, 367MB/s]
model-00007-of-00008.safetensors:  41%|████▏     | 818M/1.98G [00:03<00:02, 563MB/s]
model-00007-of-00008.safetensors:  45%|████▌     | 891M/1.98G [00:03<00:01, 553MB/s]
model-00007-of-00008.safetensors:  50%|████▉     | 986M/1.98G [00:03<00:01, 636MB/s]
model-00007-of-00008.safetensors:  60%|█████▉    | 1.18G/1.98G [00:03<00:00, 957MB/s]
model-00007-of-00008.safetensors:  83%|████████▎ | 1.64G/1.98G [00:03<00:00, 1.86GB/s]
model-00007-of-00008.safetensors:  94%|█████████▎| 1.86G/1.98G [00:04<00:00, 959MB/s] 
model-00007-of-00008.safetensors: 100%|█████████▉| 1.98G/1.98G [00:04<00:00, 450MB/s]
locutusque-hercules-3-1-3515-v1-mkmlizer: 
model-00008-of-00008.safetensors:   0%|          | 0.00/816M [00:00<?, ?B/s]
model-00008-of-00008.safetensors:   1%|▏         | 10.5M/816M [00:00<01:08, 11.7MB/s]
model-00008-of-00008.safetensors:   3%|▎         | 21.0M/816M [00:01<00:33, 23.9MB/s]
model-00008-of-00008.safetensors:   5%|▌         | 41.9M/816M [00:01<00:15, 50.0MB/s]
model-00008-of-00008.safetensors:   9%|▉         | 73.4M/816M [00:01<00:08, 90.4MB/s]
model-00008-of-00008.safetensors:  19%|█▉        | 157M/816M [00:01<00:04, 164MB/s]  
model-00008-of-00008.safetensors:  22%|██▏       | 176M/816M [00:02<00:05, 116MB/s]
model-00008-of-00008.safetensors:  29%|██▉       | 239M/816M [00:02<00:03, 176MB/s]
model-00008-of-00008.safetensors:  46%|████▌     | 375M/816M [00:02<00:01, 342MB/s]
model-00008-of-00008.safetensors:  58%|█████▊    | 470M/816M [00:02<00:00, 442MB/s]
model-00008-of-00008.safetensors: 100%|█████████▉| 816M/816M [00:02<00:00, 330MB/s]
locutusque-hercules-3-1-3515-v1-mkmlizer: 
model.safetensors.index.json:   0%|          | 0.00/23.9k [00:00<?, ?B/s]
model.safetensors.index.json: 100%|██████████| 23.9k/23.9k [00:00<00:00, 37.6MB/s]
locutusque-hercules-3-1-3515-v1-mkmlizer: 
special_tokens_map.json:   0%|          | 0.00/437 [00:00<?, ?B/s]
special_tokens_map.json: 100%|██████████| 437/437 [00:00<00:00, 3.05MB/s]
locutusque-hercules-3-1-3515-v1-mkmlizer: 
tokenizer.json:   0%|          | 0.00/1.80M [00:00<?, ?B/s]
tokenizer.json: 100%|██████████| 1.80M/1.80M [00:00<00:00, 14.5MB/s]
tokenizer.json: 100%|██████████| 1.80M/1.80M [00:00<00:00, 14.4MB/s]
locutusque-hercules-3-1-3515-v1-mkmlizer: 
tokenizer.model:   0%|          | 0.00/493k [00:00<?, ?B/s]
tokenizer.model: 100%|██████████| 493k/493k [00:00<00:00, 29.0MB/s]
locutusque-hercules-3-1-3515-v1-mkmlizer: 
tokenizer_config.json:   0%|          | 0.00/969 [00:00<?, ?B/s]
tokenizer_config.json: 100%|██████████| 969/969 [00:00<00:00, 7.88MB/s]
locutusque-hercules-3-1-3515-v1-mkmlizer: Downloaded to shared memory in 43.799s
locutusque-hercules-3-1-3515-v1-mkmlizer: quantizing model to /dev/shm/model_cache
locutusque-hercules-3-1-3515-v1-mkmlizer: Saving mkml model at /dev/shm/model_cache
locutusque-hercules-3-1-3515-v1-mkmlizer: Reading /tmp/tmpzgod4whi/model.safetensors.index.json
locutusque-hercules-3-1-3515-v1-mkmlizer: 
Profiling:   0%|          | 0/291 [00:00<?, ?it/s]
Profiling:   0%|          | 1/291 [00:01<07:08,  1.48s/it]
Profiling:   4%|▍         | 13/291 [00:01<00:25, 11.05it/s]
Profiling:   8%|▊         | 24/291 [00:01<00:12, 21.96it/s]
Profiling:  12%|█▏        | 35/291 [00:01<00:08, 30.05it/s]
Profiling:  16%|█▋        | 48/291 [00:02<00:05, 43.75it/s]
Profiling:  20%|██        | 59/291 [00:02<00:04, 53.62it/s]
Profiling:  26%|██▌       | 76/291 [00:02<00:02, 75.45it/s]
Profiling:  30%|███       | 88/291 [00:02<00:02, 68.17it/s]
Profiling:  34%|███▎      | 98/291 [00:02<00:02, 74.36it/s]
Profiling:  37%|███▋      | 109/291 [00:02<00:02, 80.14it/s]
Profiling:  41%|████      | 119/291 [00:02<00:02, 72.82it/s]
Profiling:  45%|████▍     | 130/291 [00:02<00:02, 78.44it/s]
Profiling:  48%|████▊     | 140/291 [00:03<00:01, 83.43it/s]
Profiling:  54%|█████▎    | 156/291 [00:03<00:01, 102.63it/s]
Profiling:  58%|█████▊    | 168/291 [00:03<00:01, 77.20it/s] 
Profiling:  63%|██████▎   | 183/291 [00:03<00:01, 91.04it/s]
Profiling:  67%|██████▋   | 194/291 [00:03<00:01, 93.97it/s]
Profiling:  70%|███████   | 205/291 [00:03<00:01, 78.76it/s]
Profiling:  75%|███████▌  | 219/291 [00:03<00:00, 90.31it/s]
Profiling:  79%|███████▉  | 230/291 [00:04<00:00, 91.21it/s]
Profiling:  82%|████████▏ | 240/291 [00:04<00:00, 87.23it/s]
Profiling:  86%|████████▌ | 250/291 [00:04<00:00, 79.02it/s]
Profiling:  90%|████████▉ | 261/291 [00:04<00:00, 85.35it/s]
Profiling:  94%|█████████▍| 273/291 [00:04<00:00, 90.37it/s]
Profiling:  97%|█████████▋| 283/291 [00:06<00:00, 20.59it/s]
Profiling: 100%|██████████| 291/291 [00:06<00:00, 47.56it/s]
locutusque-hercules-3-1-3515-v1-mkmlizer: quantized model in 17.359s
locutusque-hercules-3-1-3515-v1-mkmlizer: Processed model Locutusque/Hercules-3.1-Mistral-7B in 62.215s
locutusque-hercules-3-1-3515-v1-mkmlizer: creating bucket guanaco-mkml-models
locutusque-hercules-3-1-3515-v1-mkmlizer: Bucket 's3://guanaco-mkml-models/' created
locutusque-hercules-3-1-3515-v1-mkmlizer: uploading /dev/shm/model_cache to s3://guanaco-mkml-models/locutusque-hercules-3-1-3515-v1
locutusque-hercules-3-1-3515-v1-mkmlizer: cp /dev/shm/model_cache/tokenizer_config.json s3://guanaco-mkml-models/locutusque-hercules-3-1-3515-v1/tokenizer_config.json
locutusque-hercules-3-1-3515-v1-mkmlizer: cp /dev/shm/model_cache/config.json s3://guanaco-mkml-models/locutusque-hercules-3-1-3515-v1/config.json
locutusque-hercules-3-1-3515-v1-mkmlizer: cp /dev/shm/model_cache/special_tokens_map.json s3://guanaco-mkml-models/locutusque-hercules-3-1-3515-v1/special_tokens_map.json
locutusque-hercules-3-1-3515-v1-mkmlizer: cp /dev/shm/model_cache/tokenizer.model s3://guanaco-mkml-models/locutusque-hercules-3-1-3515-v1/tokenizer.model
locutusque-hercules-3-1-3515-v1-mkmlizer: cp /dev/shm/model_cache/tokenizer.json s3://guanaco-mkml-models/locutusque-hercules-3-1-3515-v1/tokenizer.json
locutusque-hercules-3-1-3515-v1-mkmlizer: cp /dev/shm/model_cache/mkml_model.tensors s3://guanaco-mkml-models/locutusque-hercules-3-1-3515-v1/mkml_model.tensors
locutusque-hercules-3-1-3515-v1-mkmlizer: loading reward model from ChaiML/reward_gpt2_medium_preference_24m_e2
locutusque-hercules-3-1-3515-v1-mkmlizer: /opt/conda/lib/python3.10/site-packages/transformers/models/auto/configuration_auto.py:1067: FutureWarning: The `use_auth_token` argument is deprecated and will be removed in v5 of Transformers. Please use `token` instead.
locutusque-hercules-3-1-3515-v1-mkmlizer:   warnings.warn(
locutusque-hercules-3-1-3515-v1-mkmlizer: 
config.json:   0%|          | 0.00/1.05k [00:00<?, ?B/s]
config.json: 100%|██████████| 1.05k/1.05k [00:00<00:00, 7.24MB/s]
locutusque-hercules-3-1-3515-v1-mkmlizer: /opt/conda/lib/python3.10/site-packages/transformers/models/auto/tokenization_auto.py:690: FutureWarning: The `use_auth_token` argument is deprecated and will be removed in v5 of Transformers. Please use `token` instead.
locutusque-hercules-3-1-3515-v1-mkmlizer:   warnings.warn(
locutusque-hercules-3-1-3515-v1-mkmlizer: 
tokenizer_config.json:   0%|          | 0.00/234 [00:00<?, ?B/s]
tokenizer_config.json: 100%|██████████| 234/234 [00:00<00:00, 1.53MB/s]
locutusque-hercules-3-1-3515-v1-mkmlizer: 
vocab.json:   0%|          | 0.00/1.04M [00:00<?, ?B/s]
vocab.json: 100%|██████████| 1.04M/1.04M [00:00<00:00, 10.9MB/s]
locutusque-hercules-3-1-3515-v1-mkmlizer: 
tokenizer.json:   0%|          | 0.00/2.11M [00:00<?, ?B/s]
tokenizer.json: 100%|██████████| 2.11M/2.11M [00:00<00:00, 84.7MB/s]
locutusque-hercules-3-1-3515-v1-mkmlizer: /opt/conda/lib/python3.10/site-packages/transformers/models/auto/auto_factory.py:472: FutureWarning: The `use_auth_token` argument is deprecated and will be removed in v5 of Transformers. Please use `token` instead.
locutusque-hercules-3-1-3515-v1-mkmlizer:   warnings.warn(
locutusque-hercules-3-1-3515-v1-mkmlizer: 
pytorch_model.bin:   0%|          | 0.00/1.44G [00:00<?, ?B/s]
pytorch_model.bin:   1%|▏         | 21.0M/1.44G [00:00<00:07, 187MB/s]
pytorch_model.bin:   5%|▌         | 73.4M/1.44G [00:00<00:06, 225MB/s]
pytorch_model.bin:   9%|▊         | 126M/1.44G [00:00<00:04, 313MB/s] 
pytorch_model.bin:  15%|█▍        | 210M/1.44G [00:00<00:02, 472MB/s]
pytorch_model.bin:  23%|██▎       | 325M/1.44G [00:00<00:01, 679MB/s]
pytorch_model.bin:  28%|██▊       | 409M/1.44G [00:01<00:04, 251MB/s]
pytorch_model.bin:  38%|███▊      | 556M/1.44G [00:01<00:02, 405MB/s]
pytorch_model.bin:  60%|██████    | 870M/1.44G [00:01<00:00, 813MB/s]
pytorch_model.bin: 100%|█████████▉| 1.44G/1.44G [00:01<00:00, 851MB/s]
locutusque-hercules-3-1-3515-v1-mkmlizer: Saving model to /tmp/reward_cache/reward.tensors
locutusque-hercules-3-1-3515-v1-mkmlizer: Saving duration: 0.295s
locutusque-hercules-3-1-3515-v1-mkmlizer: Processed model ChaiML/reward_gpt2_medium_preference_24m_e2 in 5.690s
locutusque-hercules-3-1-3515-v1-mkmlizer: creating bucket guanaco-reward-models
locutusque-hercules-3-1-3515-v1-mkmlizer: Bucket 's3://guanaco-reward-models/' created
locutusque-hercules-3-1-3515-v1-mkmlizer: uploading /tmp/reward_cache to s3://guanaco-reward-models/locutusque-hercules-3-1-3515-v1_reward
locutusque-hercules-3-1-3515-v1-mkmlizer: cp /tmp/reward_cache/special_tokens_map.json s3://guanaco-reward-models/locutusque-hercules-3-1-3515-v1_reward/special_tokens_map.json
locutusque-hercules-3-1-3515-v1-mkmlizer: cp /tmp/reward_cache/config.json s3://guanaco-reward-models/locutusque-hercules-3-1-3515-v1_reward/config.json
locutusque-hercules-3-1-3515-v1-mkmlizer: cp /tmp/reward_cache/tokenizer_config.json s3://guanaco-reward-models/locutusque-hercules-3-1-3515-v1_reward/tokenizer_config.json
locutusque-hercules-3-1-3515-v1-mkmlizer: cp /tmp/reward_cache/merges.txt s3://guanaco-reward-models/locutusque-hercules-3-1-3515-v1_reward/merges.txt
locutusque-hercules-3-1-3515-v1-mkmlizer: cp /tmp/reward_cache/vocab.json s3://guanaco-reward-models/locutusque-hercules-3-1-3515-v1_reward/vocab.json
locutusque-hercules-3-1-3515-v1-mkmlizer: cp /tmp/reward_cache/tokenizer.json s3://guanaco-reward-models/locutusque-hercules-3-1-3515-v1_reward/tokenizer.json
locutusque-hercules-3-1-3515-v1-mkmlizer: cp /tmp/reward_cache/reward.tensors s3://guanaco-reward-models/locutusque-hercules-3-1-3515-v1_reward/reward.tensors
Job locutusque-hercules-3-1-3515-v1-mkmlizer completed after 95.73s with status: succeeded
Stopping job with name locutusque-hercules-3-1-3515-v1-mkmlizer
Pipeline stage MKMLizer completed in 101.09s
Running pipeline stage MKMLKubeTemplater
Pipeline stage MKMLKubeTemplater completed in 0.12s
Running pipeline stage ISVCDeployer
Creating inference service locutusque-hercules-3-1-3515-v1
Waiting for inference service locutusque-hercules-3-1-3515-v1 to be ready
Inference service locutusque-hercules-3-1-3515-v1 ready after 40.24492812156677s
Pipeline stage ISVCDeployer completed in 47.82s
Running pipeline stage StressChecker
Received healthy response to inference request in 1.7370760440826416s
Received healthy response to inference request in 0.9952054023742676s
Received healthy response to inference request in 1.1359400749206543s
Received healthy response to inference request in 1.0834143161773682s
Received healthy response to inference request in 1.1885943412780762s
5 requests
0 failed requests
5th percentile: 1.0128471851348877
10th percentile: 1.0304889678955078
20th percentile: 1.065772533416748
30th percentile: 1.0939194679260253
40th percentile: 1.11492977142334
50th percentile: 1.1359400749206543
60th percentile: 1.157001781463623
70th percentile: 1.1780634880065919
80th percentile: 1.2982906818389894
90th percentile: 1.5176833629608155
95th percentile: 1.6273797035217283
99th percentile: 1.715136775970459
mean time: 1.2280460357666017
Pipeline stage StressChecker completed in 7.00s
Running pipeline stage DaemonicModelEvalScorer
Pipeline stage DaemonicModelEvalScorer completed in 0.04s
Running pipeline stage DaemonicSafetyScorer
Running M-Eval for topic stay_in_character
Pipeline stage DaemonicSafetyScorer completed in 0.05s
M-Eval Dataset for topic stay_in_character is loaded
locutusque-hercules-3-1-_3515_v1 status is now deployed due to DeploymentManager action
locutusque-hercules-3-1-_3515_v1 status is now inactive due to auto deactivation removed underperforming models
admin requested tearing down of locutusque-hercules-3-1-_3515_v1
Running pipeline stage ISVCDeleter
Checking if service locutusque-hercules-3-1-3515-v1 is running
Tearing down inference service locutusque-hercules-3-1-3515-v1
Toredown service locutusque-hercules-3-1-3515-v1
Pipeline stage ISVCDeleter completed in 3.35s
Running pipeline stage MKMLModelDeleter
Cleaning model data from S3
Cleaning model data from model cache
Deleting key locutusque-hercules-3-1-3515-v1/config.json from bucket guanaco-mkml-models
Deleting key locutusque-hercules-3-1-3515-v1/mkml_model.tensors from bucket guanaco-mkml-models
Deleting key locutusque-hercules-3-1-3515-v1/special_tokens_map.json from bucket guanaco-mkml-models
Deleting key locutusque-hercules-3-1-3515-v1/tokenizer.json from bucket guanaco-mkml-models
Deleting key locutusque-hercules-3-1-3515-v1/tokenizer.model from bucket guanaco-mkml-models
Deleting key locutusque-hercules-3-1-3515-v1/tokenizer_config.json from bucket guanaco-mkml-models
Cleaning model data from model cache
Deleting key locutusque-hercules-3-1-3515-v1_reward/config.json from bucket guanaco-reward-models
Deleting key locutusque-hercules-3-1-3515-v1_reward/merges.txt from bucket guanaco-reward-models
Deleting key locutusque-hercules-3-1-3515-v1_reward/reward.tensors from bucket guanaco-reward-models
Deleting key locutusque-hercules-3-1-3515-v1_reward/special_tokens_map.json from bucket guanaco-reward-models
Deleting key locutusque-hercules-3-1-3515-v1_reward/tokenizer.json from bucket guanaco-reward-models
Deleting key locutusque-hercules-3-1-3515-v1_reward/tokenizer_config.json from bucket guanaco-reward-models
Deleting key locutusque-hercules-3-1-3515-v1_reward/vocab.json from bucket guanaco-reward-models
Pipeline stage MKMLModelDeleter completed in 1.98s
locutusque-hercules-3-1-_3515_v1 status is now torndown due to DeploymentManager action