submission_id: khoantap-missy-7b_v1
developer_uid: chai_backend_admin
status: torndown
model_repo: khoantap/missy-7b
reward_repo: ChaiML/reward_models_100_170000000_cp_498032
generation_params: {'temperature': 0.9, 'top_p': 0.9, 'min_p': 0.0, 'top_k': 70, 'presence_penalty': 0.9, 'frequency_penalty': 0.2, 'stopping_words': ['</s>', '###'], 'max_input_tokens': 512, 'best_of': 16, 'max_output_tokens': 64}
formatter: {'memory_template': "Below is an instruction that describes background information for a story-rich chat. Write an appropriate response for both the instruction and user input.\n\n### Instruction:\n\n{bot_name}'s Persona: {memory}.\n\n", 'prompt_template': '{prompt}\n\n', 'bot_template': '### Response:\n\n{bot_name}: {message}\n\n', 'user_template': '### Input:\n\n{user_name}: {message}\n\n', 'response_template': '### Response: (length = long)\n\n{bot_name}:', 'truncate_by_message': False}
timestamp: 2023-12-21T21:21:04+00:00
model_name: 7b-16
model_group: khoantap/missy-7b
num_battles: 86655
num_wins: 42369
celo_rating: 1148.09
propriety_score: 0.0
propriety_total_count: 0.0
submission_type: basic
model_architecture: None
model_num_parameters: None
best_of: 16
max_input_tokens: 512
max_output_tokens: 64
display_name: 7b-16
ineligible_reason: propriety_total_count < 800
language_model: khoantap/missy-7b
model_size: NoneB
reward_model: ChaiML/reward_models_100_170000000_cp_498032
us_pacific_date: 2023-12-21
win_ratio: 0.4889388956205643
preference_data_url: None
Resubmit model
Running pipeline stage MKMLizer
Starting job with name khoantap-missy-7b-v1-mkmlizer
Waiting for job on khoantap-missy-7b-v1-mkmlizer to finish
khoantap-missy-7b-v1-mkmlizer: ╔═════════════════════════════════════════════════════════════════════╗
khoantap-missy-7b-v1-mkmlizer: ║   _______ __  __ _______ _____                                      ║
khoantap-missy-7b-v1-mkmlizer: ║  |   |   |  |/  |   |   |     |_                                    ║
khoantap-missy-7b-v1-mkmlizer: ║  |       |     <|       |       |                                   ║
khoantap-missy-7b-v1-mkmlizer: ║  |__|_|__|__|\__|__|_|__|_______|                                   ║
khoantap-missy-7b-v1-mkmlizer: ║                                                                     ║
khoantap-missy-7b-v1-mkmlizer: ║  Copyright 2023 MK ONE TECHNOLOGIES Inc.                            ║
khoantap-missy-7b-v1-mkmlizer: ║                                                                     ║
khoantap-missy-7b-v1-mkmlizer: ║  The license key for the current software has been verified as      ║
khoantap-missy-7b-v1-mkmlizer: ║  belonging to:                                                      ║
khoantap-missy-7b-v1-mkmlizer: ║                                                                     ║
khoantap-missy-7b-v1-mkmlizer: ║  Chai Research Corp                                                 ║
khoantap-missy-7b-v1-mkmlizer: ║  Account ID: 7997a29f-0ceb-4cc7-9adf-840c57b4ae6f                   ║
khoantap-missy-7b-v1-mkmlizer: ║  Expiration: 2024-01-08 23:59:59                                    ║
khoantap-missy-7b-v1-mkmlizer: ║                                                                     ║
khoantap-missy-7b-v1-mkmlizer: ╚═════════════════════════════════════════════════════════════════════╝
khoantap-missy-7b-v1-mkmlizer: loading model from khoantap/missy-7b
khoantap-missy-7b-v1-mkmlizer: /opt/conda/lib/python3.10/site-packages/transformers/models/auto/configuration_auto.py:1067: FutureWarning: The `use_auth_token` argument is deprecated and will be removed in v5 of Transformers. Please use `token` instead.
khoantap-missy-7b-v1-mkmlizer:   warnings.warn(
khoantap-missy-7b-v1-mkmlizer: 
config.json:   0%|          | 0.00/646 [00:00<?, ?B/s]
config.json: 100%|██████████| 646/646 [00:00<00:00, 6.64MB/s]
khoantap-missy-7b-v1-mkmlizer: /opt/conda/lib/python3.10/site-packages/transformers/models/auto/tokenization_auto.py:690: FutureWarning: The `use_auth_token` argument is deprecated and will be removed in v5 of Transformers. Please use `token` instead.
khoantap-missy-7b-v1-mkmlizer:   warnings.warn(
khoantap-missy-7b-v1-mkmlizer: 
tokenizer_config.json:   0%|          | 0.00/966 [00:00<?, ?B/s]
tokenizer_config.json: 100%|██████████| 966/966 [00:00<00:00, 15.3MB/s]
khoantap-missy-7b-v1-mkmlizer: 
tokenizer.json:   0%|          | 0.00/1.80M [00:00<?, ?B/s]
tokenizer.json: 100%|██████████| 1.80M/1.80M [00:00<00:00, 55.5MB/s]
khoantap-missy-7b-v1-mkmlizer: 
special_tokens_map.json:   0%|          | 0.00/414 [00:00<?, ?B/s]
special_tokens_map.json: 100%|██████████| 414/414 [00:00<00:00, 4.89MB/s]
khoantap-missy-7b-v1-mkmlizer: /opt/conda/lib/python3.10/site-packages/transformers/models/auto/auto_factory.py:472: FutureWarning: The `use_auth_token` argument is deprecated and will be removed in v5 of Transformers. Please use `token` instead.
khoantap-missy-7b-v1-mkmlizer:   warnings.warn(
khoantap-missy-7b-v1-mkmlizer: 
model.safetensors.index.json:   0%|          | 0.00/23.9k [00:00<?, ?B/s]
model.safetensors.index.json: 100%|██████████| 23.9k/23.9k [00:00<00:00, 152MB/s]
khoantap-missy-7b-v1-mkmlizer: 
Downloading shards:   0%|          | 0/3 [00:00<?, ?it/s]
khoantap-missy-7b-v1-mkmlizer: 
model-00001-of-00003.safetensors:   0%|          | 0.00/4.94G [00:00<?, ?B/s]
khoantap-missy-7b-v1-mkmlizer: 
model-00001-of-00003.safetensors:   0%|          | 10.5M/4.94G [00:01<14:34, 5.64MB/s]
khoantap-missy-7b-v1-mkmlizer: 
model-00001-of-00003.safetensors:   0%|          | 21.0M/4.94G [00:02<08:28, 9.67MB/s]
khoantap-missy-7b-v1-mkmlizer: 
model-00001-of-00003.safetensors:  28%|██▊       | 1.37G/4.94G [00:04<00:06, 552MB/s]
khoantap-missy-7b-v1-mkmlizer: 
model-00001-of-00003.safetensors:  29%|██▉       | 1.46G/4.94G [00:05<00:06, 542MB/s]
khoantap-missy-7b-v1-mkmlizer: 
model-00001-of-00003.safetensors:  33%|███▎      | 1.64G/4.94G [00:05<00:04, 722MB/s]
khoantap-missy-7b-v1-mkmlizer: 
model-00001-of-00003.safetensors:  35%|███▌      | 1.74G/4.94G [00:05<00:04, 769MB/s]
khoantap-missy-7b-v1-mkmlizer: 
model-00001-of-00003.safetensors:  37%|███▋      | 1.85G/4.94G [00:05<00:03, 805MB/s]
khoantap-missy-7b-v1-mkmlizer: 
model-00001-of-00003.safetensors:  39%|███▉      | 1.95G/4.94G [00:05<00:04, 727MB/s]
khoantap-missy-7b-v1-mkmlizer: 
model-00001-of-00003.safetensors:  42%|████▏     | 2.09G/4.94G [00:05<00:03, 850MB/s]
khoantap-missy-7b-v1-mkmlizer: 
model-00001-of-00003.safetensors:  44%|████▍     | 2.19G/4.94G [00:05<00:03, 765MB/s]
khoantap-missy-7b-v1-mkmlizer: 
model-00001-of-00003.safetensors:  46%|████▌     | 2.29G/4.94G [00:05<00:03, 709MB/s]
khoantap-missy-7b-v1-mkmlizer: 
model-00001-of-00003.safetensors:  50%|█████     | 2.50G/4.94G [00:06<00:02, 1.00GB/s]
khoantap-missy-7b-v1-mkmlizer: 
model-00001-of-00003.safetensors:  56%|█████▌    | 2.78G/4.94G [00:06<00:01, 1.43GB/s]
khoantap-missy-7b-v1-mkmlizer: 
model-00001-of-00003.safetensors:  60%|█████▉    | 2.95G/4.94G [00:06<00:01, 1.21GB/s]
khoantap-missy-7b-v1-mkmlizer: 
model-00001-of-00003.safetensors:  63%|██████▎   | 3.09G/4.94G [00:06<00:02, 903MB/s] 
khoantap-missy-7b-v1-mkmlizer: 
model-00001-of-00003.safetensors:  65%|██████▌   | 3.22G/4.94G [00:07<00:02, 631MB/s]
khoantap-missy-7b-v1-mkmlizer: 
model-00001-of-00003.safetensors:  67%|██████▋   | 3.31G/4.94G [00:07<00:03, 530MB/s]
khoantap-missy-7b-v1-mkmlizer: 
model-00001-of-00003.safetensors:  69%|██████▉   | 3.42G/4.94G [00:07<00:02, 582MB/s]
khoantap-missy-7b-v1-mkmlizer: 
model-00001-of-00003.safetensors:  72%|███████▏  | 3.54G/4.94G [00:07<00:02, 692MB/s]
khoantap-missy-7b-v1-mkmlizer: 
model-00001-of-00003.safetensors:  74%|███████▎  | 3.64G/4.94G [00:07<00:01, 736MB/s]
khoantap-missy-7b-v1-mkmlizer: 
model-00001-of-00003.safetensors:  76%|███████▌  | 3.73G/4.94G [00:07<00:01, 761MB/s]
khoantap-missy-7b-v1-mkmlizer: 
model-00001-of-00003.safetensors:  79%|███████▉  | 3.90G/4.94G [00:07<00:01, 969MB/s]
khoantap-missy-7b-v1-mkmlizer: 
model-00001-of-00003.safetensors:  84%|████████▎ | 4.13G/4.94G [00:07<00:00, 1.29GB/s]
khoantap-missy-7b-v1-mkmlizer: 
model-00001-of-00003.safetensors:  87%|████████▋ | 4.28G/4.94G [00:08<00:00, 1.06GB/s]
khoantap-missy-7b-v1-mkmlizer: 
model-00001-of-00003.safetensors:  89%|████████▉ | 4.40G/4.94G [00:08<00:00, 927MB/s] 
khoantap-missy-7b-v1-mkmlizer: 
model-00001-of-00003.safetensors:  91%|█████████▏| 4.51G/4.94G [00:08<00:00, 799MB/s]
khoantap-missy-7b-v1-mkmlizer: 
model-00001-of-00003.safetensors:  94%|█████████▍| 4.65G/4.94G [00:08<00:00, 903MB/s]
khoantap-missy-7b-v1-mkmlizer: 
model-00001-of-00003.safetensors:  99%|█████████▉| 4.90G/4.94G [00:08<00:00, 1.26GB/s]
model-00001-of-00003.safetensors: 100%|█████████▉| 4.94G/4.94G [00:08<00:00, 562MB/s] 
khoantap-missy-7b-v1-mkmlizer: 
Downloading shards:  33%|███▎      | 1/3 [00:09<00:18,  9.26s/it]
khoantap-missy-7b-v1-mkmlizer: 
model-00002-of-00003.safetensors:   0%|          | 0.00/5.00G [00:00<?, ?B/s]
khoantap-missy-7b-v1-mkmlizer: 
model-00002-of-00003.safetensors:   0%|          | 10.5M/5.00G [00:01<15:20, 5.42MB/s]
khoantap-missy-7b-v1-mkmlizer: 
model-00002-of-00003.safetensors:   0%|          | 21.0M/5.00G [00:02<08:03, 10.3MB/s]
khoantap-missy-7b-v1-mkmlizer: 
model-00002-of-00003.safetensors:   1%|          | 31.5M/5.00G [00:02<04:53, 17.0MB/s]
khoantap-missy-7b-v1-mkmlizer: 
model-00002-of-00003.safetensors:   1%|          | 41.9M/5.00G [00:02<03:58, 20.8MB/s]
khoantap-missy-7b-v1-mkmlizer: 
model-00002-of-00003.safetensors:   1%|▏         | 73.4M/5.00G [00:02<01:46, 46.2MB/s]
khoantap-missy-7b-v1-mkmlizer: 
model-00002-of-00003.safetensors:   2%|▏         | 105M/5.00G [00:03<01:03, 76.7MB/s] 
khoantap-missy-7b-v1-mkmlizer: 
model-00002-of-00003.safetensors:   4%|▍         | 210M/5.00G [00:03<00:23, 205MB/s] 
khoantap-missy-7b-v1-mkmlizer: 
model-00002-of-00003.safetensors:   5%|▌         | 273M/5.00G [00:03<00:19, 237MB/s]
khoantap-missy-7b-v1-mkmlizer: 
model-00002-of-00003.safetensors:   6%|▋         | 315M/5.00G [00:03<00:19, 244MB/s]
khoantap-missy-7b-v1-mkmlizer: 
model-00002-of-00003.safetensors:   8%|▊         | 409M/5.00G [00:03<00:12, 369MB/s]
khoantap-missy-7b-v1-mkmlizer: 
model-00002-of-00003.safetensors:  14%|█▍        | 703M/5.00G [00:03<00:04, 891MB/s]
khoantap-missy-7b-v1-mkmlizer: 
model-00002-of-00003.safetensors:  24%|██▎       | 1.18G/5.00G [00:03<00:02, 1.58GB/s]
khoantap-missy-7b-v1-mkmlizer: 
model-00002-of-00003.safetensors:  27%|██▋       | 1.37G/5.00G [00:04<00:06, 587MB/s] 
khoantap-missy-7b-v1-mkmlizer: 
model-00002-of-00003.safetensors:  30%|███       | 1.51G/5.00G [00:05<00:05, 604MB/s]
khoantap-missy-7b-v1-mkmlizer: 
model-00002-of-00003.safetensors:  33%|███▎      | 1.63G/5.00G [00:05<00:05, 652MB/s]
khoantap-missy-7b-v1-mkmlizer: 
model-00002-of-00003.safetensors:  35%|███▌      | 1.76G/5.00G [00:05<00:04, 735MB/s]
khoantap-missy-7b-v1-mkmlizer: 
model-00002-of-00003.safetensors:  38%|███▊      | 1.88G/5.00G [00:05<00:04, 739MB/s]
khoantap-missy-7b-v1-mkmlizer: 
model-00002-of-00003.safetensors:  40%|███▉      | 1.98G/5.00G [00:05<00:04, 670MB/s]
khoantap-missy-7b-v1-mkmlizer: 
model-00002-of-00003.safetensors:  44%|████▍     | 2.20G/5.00G [00:05<00:02, 935MB/s]
khoantap-missy-7b-v1-mkmlizer: 
model-00002-of-00003.safetensors:  82%|████████▏ | 4.11G/5.00G [00:07<00:01, 735MB/s]
khoantap-missy-7b-v1-mkmlizer: 
model-00002-of-00003.safetensors:  84%|████████▍ | 4.19G/5.00G [00:07<00:01, 678MB/s]
khoantap-missy-7b-v1-mkmlizer: 
model-00002-of-00003.safetensors:  86%|████████▌ | 4.29G/5.00G [00:08<00:00, 713MB/s]
khoantap-missy-7b-v1-mkmlizer: 
model-00002-of-00003.safetensors:  89%|████████▉ | 4.47G/5.00G [00:08<00:00, 915MB/s]
khoantap-missy-7b-v1-mkmlizer: 
model-00002-of-00003.safetensors:  91%|█████████▏| 4.57G/5.00G [00:08<00:00, 938MB/s]
khoantap-missy-7b-v1-mkmlizer: 
model-00002-of-00003.safetensors:  94%|█████████▎| 4.68G/5.00G [00:08<00:00, 847MB/s]
khoantap-missy-7b-v1-mkmlizer: 
model-00002-of-00003.safetensors: 100%|█████████▉| 5.00G/5.00G [00:08<00:00, 1.26GB/s]
model-00002-of-00003.safetensors: 100%|█████████▉| 5.00G/5.00G [00:08<00:00, 577MB/s] 
khoantap-missy-7b-v1-mkmlizer: 
Downloading shards:  67%|██████▋   | 2/3 [00:18<00:09,  9.09s/it]
khoantap-missy-7b-v1-mkmlizer: 
model-00003-of-00003.safetensors:   0%|          | 0.00/4.54G [00:00<?, ?B/s]
khoantap-missy-7b-v1-mkmlizer: 
model-00003-of-00003.safetensors:   0%|          | 10.5M/4.54G [00:01<10:32, 7.16MB/s]
khoantap-missy-7b-v1-mkmlizer: 
model-00003-of-00003.safetensors:   0%|          | 21.0M/4.54G [00:01<06:12, 12.1MB/s]
khoantap-missy-7b-v1-mkmlizer: 
model-00003-of-00003.safetensors:   1%|          | 31.5M/4.54G [00:02<04:12, 17.9MB/s]
khoantap-missy-7b-v1-mkmlizer: 
model-00003-of-00003.safetensors:   1%|          | 41.9M/4.54G [00:03<05:05, 14.7MB/s]
khoantap-missy-7b-v1-mkmlizer: 
model-00003-of-00003.safetensors:   3%|▎         | 115M/4.54G [00:03<01:07, 65.7MB/s] 
khoantap-missy-7b-v1-mkmlizer: 
model-00003-of-00003.safetensors:   5%|▍         | 220M/4.54G [00:03<00:27, 156MB/s] 
khoantap-missy-7b-v1-mkmlizer: 
model-00003-of-00003.safetensors:   7%|▋         | 304M/4.54G [00:03<00:21, 196MB/s]
khoantap-missy-7b-v1-mkmlizer: 
model-00003-of-00003.safetensors:   8%|▊         | 357M/4.54G [00:03<00:19, 217MB/s]
khoantap-missy-7b-v1-mkmlizer: 
model-00003-of-00003.safetensors:  12%|█▏        | 535M/4.54G [00:03<00:09, 432MB/s]
khoantap-missy-7b-v1-mkmlizer: 
model-00003-of-00003.safetensors:  24%|██▍       | 1.11G/4.54G [00:03<00:02, 1.25GB/s]
khoantap-missy-7b-v1-mkmlizer: 
model-00003-of-00003.safetensors:  29%|██▉       | 1.31G/4.54G [00:04<00:04, 777MB/s] 
khoantap-missy-7b-v1-mkmlizer: 
model-00003-of-00003.safetensors:  32%|███▏      | 1.46G/4.54G [00:05<00:05, 570MB/s]
khoantap-missy-7b-v1-mkmlizer: 
model-00003-of-00003.safetensors:  35%|███▍      | 1.57G/4.54G [00:05<00:04, 606MB/s]
khoantap-missy-7b-v1-mkmlizer: 
model-00003-of-00003.safetensors:  38%|███▊      | 1.72G/4.54G [00:05<00:03, 708MB/s]
khoantap-missy-7b-v1-mkmlizer: 
model-00003-of-00003.safetensors:  43%|████▎     | 1.94G/4.54G [00:05<00:02, 930MB/s]
khoantap-missy-7b-v1-mkmlizer: 
model-00003-of-00003.safetensors:  46%|████▌     | 2.09G/4.54G [00:05<00:02, 1.03GB/s]
khoantap-missy-7b-v1-mkmlizer: 
model-00003-of-00003.safetensors:  49%|████▉     | 2.23G/4.54G [00:05<00:02, 870MB/s] 
khoantap-missy-7b-v1-mkmlizer: 
model-00003-of-00003.safetensors:  53%|█████▎    | 2.39G/4.54G [00:05<00:02, 986MB/s]
khoantap-missy-7b-v1-mkmlizer: 
model-00003-of-00003.safetensors:  58%|█████▊    | 2.61G/4.54G [00:05<00:01, 1.22GB/s]
khoantap-missy-7b-v1-mkmlizer: 
model-00003-of-00003.safetensors:  63%|██████▎   | 2.87G/4.54G [00:06<00:01, 1.48GB/s]
khoantap-missy-7b-v1-mkmlizer: 
model-00003-of-00003.safetensors:  67%|██████▋   | 3.05G/4.54G [00:06<00:01, 1.38GB/s]
khoantap-missy-7b-v1-mkmlizer: 
model-00003-of-00003.safetensors:  71%|███████   | 3.21G/4.54G [00:06<00:01, 917MB/s] 
khoantap-missy-7b-v1-mkmlizer: 
model-00003-of-00003.safetensors:  73%|███████▎  | 3.33G/4.54G [00:06<00:01, 803MB/s]
khoantap-missy-7b-v1-mkmlizer: 
model-00003-of-00003.safetensors:  76%|███████▌  | 3.44G/4.54G [00:06<00:01, 760MB/s]
khoantap-missy-7b-v1-mkmlizer: 
model-00003-of-00003.safetensors:  79%|███████▊  | 3.58G/4.54G [00:07<00:01, 854MB/s]
khoantap-missy-7b-v1-mkmlizer: 
model-00003-of-00003.safetensors:  81%|████████▏ | 3.69G/4.54G [00:07<00:00, 912MB/s]
khoantap-missy-7b-v1-mkmlizer: 
model-00003-of-00003.safetensors:  84%|████████▍ | 3.83G/4.54G [00:07<00:00, 1.01GB/s]
khoantap-missy-7b-v1-mkmlizer: 
model-00003-of-00003.safetensors:  87%|████████▋ | 3.95G/4.54G [00:07<00:00, 1.05GB/s]
khoantap-missy-7b-v1-mkmlizer: 
model-00003-of-00003.safetensors:  90%|█████████ | 4.09G/4.54G [00:07<00:00, 1.13GB/s]
khoantap-missy-7b-v1-mkmlizer: 
model-00003-of-00003.safetensors:  93%|█████████▎| 4.22G/4.54G [00:07<00:00, 1.15GB/s]
khoantap-missy-7b-v1-mkmlizer: 
model-00003-of-00003.safetensors:  96%|█████████▌| 4.34G/4.54G [00:07<00:00, 1.10GB/s]
model-00003-of-00003.safetensors: 100%|█████████▉| 4.54G/4.54G [00:07<00:00, 582MB/s] 
khoantap-missy-7b-v1-mkmlizer: 
Downloading shards: 100%|██████████| 3/3 [00:26<00:00,  8.63s/it]
Downloading shards: 100%|██████████| 3/3 [00:26<00:00,  8.77s/it]
khoantap-missy-7b-v1-mkmlizer: saved to disk in 22.683s
khoantap-missy-7b-v1-mkmlizer: quantizing model to /tmp/model_cache
khoantap-missy-7b-v1-mkmlizer: Saving mkml model at /tmp/model_cache
khoantap-missy-7b-v1-mkmlizer: Reading /tmp/tmpu3m76ie4/model.safetensors.index.json
khoantap-missy-7b-v1-mkmlizer: 
Profiling:   0%|          | 0/291 [00:00<?, ?it/s]
Profiling:   0%|          | 1/291 [00:00<04:42,  1.03it/s]
Profiling:   1%|          | 3/291 [00:01<02:09,  2.22it/s]
Profiling:   1%|▏         | 4/291 [00:02<02:17,  2.09it/s]
Profiling:   2%|▏         | 5/291 [00:02<02:22,  2.01it/s]
Profiling:   3%|▎         | 8/291 [00:02<01:11,  3.94it/s]
Profiling:   3%|▎         | 9/291 [00:03<01:12,  3.91it/s]
Profiling:   4%|▍         | 12/291 [00:03<01:03,  4.37it/s]
Profiling:   4%|▍         | 13/291 [00:04<01:19,  3.49it/s]
Profiling:   5%|▍         | 14/291 [00:04<01:35,  2.91it/s]
Profiling:   6%|▌         | 17/291 [00:04<00:58,  4.67it/s]
Profiling:   6%|▌         | 18/291 [00:05<00:57,  4.71it/s]
Profiling:   7%|▋         | 21/291 [00:05<00:55,  4.90it/s]
Profiling:   8%|▊         | 22/291 [00:06<01:10,  3.83it/s]
Profiling:   8%|▊         | 23/291 [00:06<01:23,  3.19it/s]
Profiling:   9%|▉         | 26/291 [00:07<00:53,  4.99it/s]
Profiling:   9%|▉         | 27/291 [00:07<00:50,  5.19it/s]
Profiling:  10%|█         | 30/291 [00:07<00:50,  5.21it/s]
Profiling:  11%|█         | 31/291 [00:08<01:05,  3.95it/s]
Profiling:  11%|█         | 32/291 [00:08<01:19,  3.24it/s]
Profiling:  12%|█▏        | 35/291 [00:09<00:50,  5.05it/s]
Profiling:  12%|█▏        | 36/291 [00:09<00:49,  5.20it/s]
Profiling:  13%|█▎        | 39/291 [00:09<00:48,  5.19it/s]
Profiling:  14%|█▎        | 40/291 [00:10<01:03,  3.98it/s]
Profiling:  14%|█▍        | 41/291 [00:10<01:17,  3.21it/s]
Profiling:  15%|█▌        | 44/291 [00:11<00:49,  5.00it/s]
Profiling:  15%|█▌        | 45/291 [00:11<00:48,  5.07it/s]
Profiling:  16%|█▋        | 48/291 [00:11<00:35,  6.79it/s]
Profiling:  17%|█▋        | 50/291 [00:12<00:49,  4.89it/s]
Profiling:  18%|█▊        | 51/291 [00:12<01:05,  3.68it/s]
Profiling:  18%|█▊        | 52/291 [00:12<00:57,  4.15it/s]
Profiling:  18%|█▊        | 53/291 [00:13<00:56,  4.20it/s]
Profiling:  19%|█▊        | 54/291 [00:13<00:53,  4.42it/s]
Profiling:  20%|█▉        | 57/291 [00:13<00:51,  4.54it/s]
Profiling:  20%|█▉        | 58/291 [00:14<01:08,  3.40it/s]
Profiling:  20%|██        | 59/291 [00:15<01:23,  2.77it/s]
Profiling:  21%|██        | 61/291 [00:15<01:01,  3.75it/s]
Profiling:  22%|██▏       | 63/291 [00:15<01:01,  3.72it/s]
Profiling:  22%|██▏       | 64/291 [00:16<01:12,  3.12it/s]
Profiling:  22%|██▏       | 65/291 [00:17<01:26,  2.61it/s]
Profiling:  23%|██▎       | 68/291 [00:17<00:51,  4.31it/s]
Profiling:  24%|██▎       | 69/291 [00:17<00:48,  4.55it/s]
Profiling:  25%|██▍       | 72/291 [00:18<00:45,  4.81it/s]
Profiling:  25%|██▌       | 73/291 [00:18<00:59,  3.65it/s]
Profiling:  25%|██▌       | 74/291 [00:19<01:12,  3.01it/s]
Profiling:  26%|██▌       | 76/291 [00:19<00:50,  4.29it/s]
Profiling:  26%|██▋       | 77/291 [00:19<00:47,  4.55it/s]
Profiling:  27%|██▋       | 78/291 [00:19<00:43,  4.88it/s]
Profiling:  28%|██▊       | 81/291 [00:20<00:43,  4.85it/s]
Profiling:  28%|██▊       | 82/291 [00:20<00:56,  3.72it/s]
Profiling:  29%|██▊       | 83/291 [00:21<01:08,  3.05it/s]
Profiling:  30%|██▉       | 86/291 [00:21<00:41,  4.93it/s]
Profiling:  30%|██▉       | 87/291 [00:21<00:39,  5.15it/s]
Profiling:  31%|███       | 90/291 [00:22<00:38,  5.18it/s]
Profiling:  31%|███▏      | 91/291 [00:22<00:50,  3.96it/s]
Profiling:  32%|███▏      | 92/291 [00:23<01:01,  3.21it/s]
Profiling:  33%|███▎      | 95/291 [00:23<00:38,  5.04it/s]
Profiling:  33%|███▎      | 96/291 [00:23<00:37,  5.24it/s]
Profiling:  34%|███▍      | 99/291 [00:24<00:38,  5.01it/s]
Profiling:  35%|███▌      | 102/291 [00:24<00:36,  5.17it/s]
Profiling:  35%|███▌      | 103/291 [00:25<00:49,  3.76it/s]
Profiling:  36%|███▌      | 104/291 [00:26<00:59,  3.15it/s]
Profiling:  37%|███▋      | 107/291 [00:26<00:38,  4.73it/s]
Profiling:  37%|███▋      | 108/291 [00:26<00:37,  4.83it/s]
Profiling:  38%|███▊      | 111/291 [00:27<00:36,  4.96it/s]
Profiling:  38%|███▊      | 112/291 [00:27<00:45,  3.89it/s]
Profiling:  39%|███▉      | 113/291 [00:28<00:55,  3.21it/s]
Profiling:  40%|███▉      | 116/291 [00:28<00:35,  4.99it/s]
Profiling:  40%|████      | 117/291 [00:28<00:33,  5.15it/s]
Profiling:  41%|████      | 120/291 [00:29<00:33,  5.16it/s]
Profiling:  42%|████▏     | 121/291 [00:29<00:42,  3.96it/s]
Profiling:  42%|████▏     | 122/291 [00:30<00:52,  3.25it/s]
Profiling:  43%|████▎     | 125/291 [00:30<00:33,  4.93it/s]
Profiling:  43%|████▎     | 126/291 [00:30<00:33,  4.97it/s]
Profiling:  44%|████▍     | 129/291 [00:31<00:33,  4.88it/s]
Profiling:  45%|████▍     | 130/291 [00:31<00:43,  3.73it/s]
Profiling:  45%|████▌     | 131/291 [00:32<00:52,  3.06it/s]
Profiling:  46%|████▌     | 134/291 [00:32<00:32,  4.82it/s]
Profiling:  46%|████▋     | 135/291 [00:32<00:30,  5.04it/s]
Profiling:  47%|████▋     | 138/291 [00:33<00:29,  5.12it/s]
Profiling:  48%|████▊     | 139/291 [00:33<00:38,  3.95it/s]
Profiling:  48%|████▊     | 140/291 [00:34<00:46,  3.24it/s]
Profiling:  49%|████▉     | 143/291 [00:34<00:29,  5.02it/s]
Profiling:  49%|████▉     | 144/291 [00:34<00:28,  5.20it/s]
Profiling:  50%|█████     | 146/291 [00:35<00:32,  4.50it/s]
Profiling:  51%|█████     | 148/291 [00:35<00:25,  5.51it/s]
Profiling:  51%|█████     | 149/291 [00:35<00:25,  5.67it/s]
Profiling:  52%|█████▏    | 151/291 [00:35<00:20,  6.86it/s]
Profiling:  52%|█████▏    | 152/291 [00:36<00:30,  4.53it/s]
Profiling:  53%|█████▎    | 153/291 [00:36<00:41,  3.29it/s]
Profiling:  54%|█████▎    | 156/291 [00:37<00:33,  3.98it/s]
Profiling:  54%|█████▍    | 157/291 [00:38<00:40,  3.29it/s]
Profiling:  54%|█████▍    | 158/291 [00:38<00:46,  2.85it/s]
Profiling:  55%|█████▌    | 161/291 [00:38<00:28,  4.62it/s]
Profiling:  56%|█████▌    | 162/291 [00:38<00:26,  4.87it/s]
Profiling:  57%|█████▋    | 165/291 [00:39<00:25,  5.04it/s]
Profiling:  57%|█████▋    | 166/291 [00:40<00:32,  3.89it/s]
Profiling:  57%|█████▋    | 167/291 [00:40<00:38,  3.19it/s]
Profiling:  58%|█████▊    | 170/291 [00:40<00:24,  4.99it/s]
Profiling:  59%|█████▉    | 171/291 [00:40<00:23,  5.08it/s]
Profiling:  59%|█████▉    | 172/291 [00:41<00:21,  5.55it/s]
Profiling:  60%|█████▉    | 174/291 [00:41<00:25,  4.52it/s]
Profiling:  60%|██████    | 175/291 [00:42<00:33,  3.50it/s]
Profiling:  60%|██████    | 176/291 [00:42<00:39,  2.94it/s]
Profiling:  62%|██████▏   | 179/291 [00:42<00:22,  4.89it/s]
Profiling:  62%|██████▏   | 180/291 [00:43<00:21,  5.11it/s]
Profiling:  63%|██████▎   | 183/291 [00:43<00:20,  5.18it/s]
Profiling:  63%|██████▎   | 184/291 [00:44<00:26,  3.98it/s]
Profiling:  64%|██████▎   | 185/291 [00:44<00:32,  3.25it/s]
Profiling:  65%|██████▍   | 188/291 [00:44<00:20,  5.09it/s]
Profiling:  65%|██████▍   | 189/291 [00:45<00:19,  5.28it/s]
Profiling:  66%|██████▌   | 192/291 [00:45<00:18,  5.23it/s]
Profiling:  66%|██████▋   | 193/291 [00:46<00:24,  3.92it/s]
Profiling:  67%|██████▋   | 194/291 [00:46<00:30,  3.21it/s]
Profiling:  67%|██████▋   | 196/291 [00:46<00:20,  4.53it/s]
Profiling:  68%|██████▊   | 197/291 [00:47<00:20,  4.58it/s]
Profiling:  68%|██████▊   | 198/291 [00:47<00:19,  4.85it/s]
Profiling:  69%|██████▉   | 201/291 [00:47<00:13,  6.83it/s]
Profiling:  69%|██████▉   | 202/291 [00:47<00:13,  6.72it/s]
Profiling:  70%|███████   | 204/291 [00:47<00:10,  8.01it/s]
Profiling:  70%|███████   | 205/291 [00:48<00:17,  4.84it/s]
Profiling:  71%|███████   | 206/291 [00:48<00:23,  3.58it/s]
Profiling:  71%|███████   | 207/291 [00:49<00:28,  2.96it/s]
Profiling:  72%|███████▏  | 210/291 [00:49<00:21,  3.81it/s]
Profiling:  73%|███████▎  | 211/291 [00:50<00:25,  3.10it/s]
Profiling:  73%|███████▎  | 212/291 [00:51<00:28,  2.73it/s]
Profiling:  74%|███████▍  | 215/291 [00:51<00:17,  4.47it/s]
Profiling:  74%|███████▍  | 216/291 [00:51<00:15,  4.71it/s]
Profiling:  75%|███████▌  | 219/291 [00:52<00:14,  4.98it/s]
Profiling:  76%|███████▌  | 220/291 [00:52<00:18,  3.90it/s]
Profiling:  76%|███████▌  | 221/291 [00:53<00:21,  3.22it/s]
Profiling:  77%|███████▋  | 224/291 [00:53<00:13,  5.01it/s]
Profiling:  77%|███████▋  | 225/291 [00:53<00:12,  5.20it/s]
Profiling:  78%|███████▊  | 228/291 [00:54<00:12,  5.12it/s]
Profiling:  79%|███████▊  | 229/291 [00:54<00:15,  3.95it/s]
Profiling:  79%|███████▉  | 230/291 [00:55<00:18,  3.25it/s]
Profiling:  80%|████████  | 233/291 [00:55<00:11,  5.04it/s]
Profiling:  80%|████████  | 234/291 [00:55<00:10,  5.23it/s]
Profiling:  81%|████████▏ | 237/291 [00:56<00:10,  5.25it/s]
Profiling:  82%|████████▏ | 238/291 [00:56<00:13,  4.04it/s]
Profiling:  82%|████████▏ | 239/291 [00:57<00:15,  3.30it/s]
Profiling:  83%|████████▎ | 242/291 [00:57<00:09,  5.08it/s]
Profiling:  84%|████████▎ | 243/291 [00:57<00:09,  5.26it/s]
Profiling:  84%|████████▍ | 245/291 [00:58<00:10,  4.52it/s]
Profiling:  85%|████████▍ | 246/291 [00:58<00:12,  3.56it/s]
Profiling:  85%|████████▌ | 248/291 [00:58<00:09,  4.66it/s]
Profiling:  86%|████████▌ | 249/291 [00:58<00:08,  4.94it/s]
Profiling:  86%|████████▋ | 251/291 [01:00<00:14,  2.81it/s]
Profiling:  87%|████████▋ | 253/291 [01:00<00:12,  3.10it/s]
Profiling:  88%|████████▊ | 256/291 [01:01<00:09,  3.80it/s]
Profiling:  88%|████████▊ | 257/291 [01:01<00:10,  3.25it/s]
Profiling:  89%|████████▊ | 258/291 [01:02<00:11,  2.77it/s]
Profiling:  89%|████████▉ | 260/291 [01:02<00:07,  3.93it/s]
Profiling:  90%|████████▉ | 261/291 [01:02<00:07,  3.99it/s]
Profiling:  90%|█████████ | 262/291 [01:02<00:07,  4.04it/s]
Profiling:  91%|█████████ | 265/291 [01:03<00:05,  4.52it/s]
Profiling:  91%|█████████▏| 266/291 [01:04<00:06,  3.59it/s]
Profiling:  92%|█████████▏| 267/291 [01:04<00:07,  3.02it/s]
Profiling:  93%|█████████▎| 270/291 [01:04<00:04,  4.86it/s]
Profiling:  93%|█████████▎| 271/291 [01:04<00:03,  5.06it/s]
Profiling:  94%|█████████▍| 274/291 [01:05<00:03,  5.15it/s]
Profiling:  95%|█████████▍| 275/291 [01:06<00:04,  3.96it/s]
Profiling:  95%|█████████▍| 276/291 [01:06<00:04,  3.25it/s]
Profiling:  96%|█████████▌| 279/291 [01:06<00:02,  5.05it/s]
Profiling:  96%|█████████▌| 280/291 [01:06<00:02,  5.24it/s]
Profiling:  97%|█████████▋| 283/291 [01:07<00:01,  5.24it/s]
Profiling:  98%|█████████▊| 284/291 [01:08<00:01,  4.00it/s]
Profiling:  98%|█████████▊| 285/291 [01:08<00:01,  3.27it/s]
Profiling:  99%|█████████▉| 288/291 [01:08<00:00,  5.10it/s]
Profiling:  99%|█████████▉| 289/291 [01:08<00:00,  5.21it/s]
Profiling: 100%|██████████| 291/291 [01:09<00:00,  4.22it/s]
khoantap-missy-7b-v1-mkmlizer: quantized model in 80.083s
khoantap-missy-7b-v1-mkmlizer: Processed model khoantap/missy-7b in 132.941s
khoantap-missy-7b-v1-mkmlizer: creating bucket guanaco-mkml-models
khoantap-missy-7b-v1-mkmlizer: Bucket 's3://guanaco-mkml-models/' created
khoantap-missy-7b-v1-mkmlizer: uploading /tmp/model_cache to s3://guanaco-mkml-models/khoantap-missy-7b-v1
khoantap-missy-7b-v1-mkmlizer: cp /tmp/model_cache/special_tokens_map.json s3://guanaco-mkml-models/khoantap-missy-7b-v1/special_tokens_map.json
khoantap-missy-7b-v1-mkmlizer: cp /tmp/model_cache/tokenizer_config.json s3://guanaco-mkml-models/khoantap-missy-7b-v1/tokenizer_config.json
khoantap-missy-7b-v1-mkmlizer: cp /tmp/model_cache/config.json s3://guanaco-mkml-models/khoantap-missy-7b-v1/config.json
khoantap-missy-7b-v1-mkmlizer: cp /tmp/model_cache/tokenizer.json s3://guanaco-mkml-models/khoantap-missy-7b-v1/tokenizer.json
khoantap-missy-7b-v1-mkmlizer: cp /tmp/model_cache/mkml_model.tensors s3://guanaco-mkml-models/khoantap-missy-7b-v1/mkml_model.tensors
khoantap-missy-7b-v1-mkmlizer: loading reward model from ChaiML/reward_models_100_170000000_cp_498032
khoantap-missy-7b-v1-mkmlizer: /opt/conda/lib/python3.10/site-packages/transformers/models/auto/configuration_auto.py:1067: FutureWarning: The `use_auth_token` argument is deprecated and will be removed in v5 of Transformers. Please use `token` instead.
khoantap-missy-7b-v1-mkmlizer:   warnings.warn(
khoantap-missy-7b-v1-mkmlizer: 
config.json:   0%|          | 0.00/1.03k [00:00<?, ?B/s]
config.json: 100%|██████████| 1.03k/1.03k [00:00<00:00, 9.78MB/s]
khoantap-missy-7b-v1-mkmlizer: /opt/conda/lib/python3.10/site-packages/transformers/models/auto/tokenization_auto.py:690: FutureWarning: The `use_auth_token` argument is deprecated and will be removed in v5 of Transformers. Please use `token` instead.
khoantap-missy-7b-v1-mkmlizer:   warnings.warn(
khoantap-missy-7b-v1-mkmlizer: 
tokenizer_config.json:   0%|          | 0.00/234 [00:00<?, ?B/s]
tokenizer_config.json: 100%|██████████| 234/234 [00:00<00:00, 1.70MB/s]
khoantap-missy-7b-v1-mkmlizer: 
vocab.json:   0%|          | 0.00/798k [00:00<?, ?B/s]
vocab.json: 100%|██████████| 798k/798k [00:00<00:00, 34.9MB/s]
khoantap-missy-7b-v1-mkmlizer: 
merges.txt:   0%|          | 0.00/456k [00:00<?, ?B/s]
merges.txt: 100%|██████████| 456k/456k [00:00<00:00, 59.0MB/s]
khoantap-missy-7b-v1-mkmlizer: 
tokenizer.json:   0%|          | 0.00/2.11M [00:00<?, ?B/s]
tokenizer.json: 100%|██████████| 2.11M/2.11M [00:00<00:00, 46.8MB/s]
khoantap-missy-7b-v1-mkmlizer: 
special_tokens_map.json:   0%|          | 0.00/99.0 [00:00<?, ?B/s]
special_tokens_map.json: 100%|██████████| 99.0/99.0 [00:00<00:00, 989kB/s]
khoantap-missy-7b-v1-mkmlizer: /opt/conda/lib/python3.10/site-packages/transformers/models/auto/auto_factory.py:472: FutureWarning: The `use_auth_token` argument is deprecated and will be removed in v5 of Transformers. Please use `token` instead.
khoantap-missy-7b-v1-mkmlizer:   warnings.warn(
khoantap-missy-7b-v1-mkmlizer: 
pytorch_model.bin:   0%|          | 0.00/510M [00:00<?, ?B/s]
pytorch_model.bin:   2%|▏         | 10.5M/510M [00:00<00:08, 59.6MB/s]
pytorch_model.bin:  10%|█         | 52.4M/510M [00:00<00:02, 196MB/s] 
pytorch_model.bin:  16%|█▋        | 83.9M/510M [00:00<00:01, 219MB/s]
pytorch_model.bin:  30%|███       | 154M/510M [00:00<00:00, 364MB/s] 
pytorch_model.bin:  59%|█████▉    | 301M/510M [00:00<00:00, 705MB/s]
pytorch_model.bin:  90%|████████▉ | 458M/510M [00:00<00:00, 919MB/s]
pytorch_model.bin: 100%|█████████▉| 510M/510M [00:00<00:00, 635MB/s]
khoantap-missy-7b-v1-mkmlizer: Saving model to /tmp/reward_cache/reward.tensors
khoantap-missy-7b-v1-mkmlizer: Saving duration: 0.099s
khoantap-missy-7b-v1-mkmlizer: Processed model ChaiML/reward_models_100_170000000_cp_498032 in 3.034s
khoantap-missy-7b-v1-mkmlizer: creating bucket guanaco-reward-models
khoantap-missy-7b-v1-mkmlizer: Bucket 's3://guanaco-reward-models/' created
khoantap-missy-7b-v1-mkmlizer: uploading /tmp/reward_cache to s3://guanaco-reward-models/khoantap-missy-7b-v1_reward
khoantap-missy-7b-v1-mkmlizer: cp /tmp/reward_cache/special_tokens_map.json s3://guanaco-reward-models/khoantap-missy-7b-v1_reward/special_tokens_map.json
khoantap-missy-7b-v1-mkmlizer: cp /tmp/reward_cache/config.json s3://guanaco-reward-models/khoantap-missy-7b-v1_reward/config.json
khoantap-missy-7b-v1-mkmlizer: cp /tmp/reward_cache/tokenizer_config.json s3://guanaco-reward-models/khoantap-missy-7b-v1_reward/tokenizer_config.json
khoantap-missy-7b-v1-mkmlizer: cp /tmp/reward_cache/merges.txt s3://guanaco-reward-models/khoantap-missy-7b-v1_reward/merges.txt
khoantap-missy-7b-v1-mkmlizer: cp /tmp/reward_cache/vocab.json s3://guanaco-reward-models/khoantap-missy-7b-v1_reward/vocab.json
khoantap-missy-7b-v1-mkmlizer: cp /tmp/reward_cache/tokenizer.json s3://guanaco-reward-models/khoantap-missy-7b-v1_reward/tokenizer.json
khoantap-missy-7b-v1-mkmlizer: cp /tmp/reward_cache/reward.tensors s3://guanaco-reward-models/khoantap-missy-7b-v1_reward/reward.tensors
Job khoantap-missy-7b-v1-mkmlizer completed after 177.58s with status: succeeded
Stopping job with name khoantap-missy-7b-v1-mkmlizer
Pipeline stage MKMLizer completed in 182.47s
Running pipeline stage MKMLKubeTemplater
Pipeline stage MKMLKubeTemplater completed in 0.15s
Running pipeline stage ISVCDeployer
Creating inference service khoantap-missy-7b-v1
Waiting for inference service khoantap-missy-7b-v1 to be ready
Inference service khoantap-missy-7b-v1 ready after 90.61301016807556s
Pipeline stage ISVCDeployer completed in 98.63s
Running pipeline stage StressChecker
Received healthy response to inference request with status code 200 in 8.626258850097656s
Received healthy response to inference request with status code 200 in 1.334493637084961s
Received healthy response to inference request with status code 200 in 1.3064932823181152s
Received healthy response to inference request with status code 200 in 1.3198270797729492s
Received healthy response to inference request with status code 200 in 1.339191198348999s
Received healthy response to inference request with status code 200 in 1.4255800247192383s
Received healthy response to inference request with status code 200 in 1.3112199306488037s
Received healthy response to inference request with status code 200 in 1.331115961074829s
Received healthy response to inference request with status code 200 in 1.302454948425293s
Received healthy response to inference request with status code 200 in 1.3083312511444092s
Received healthy response to inference request with status code 200 in 1.312140941619873s
Received healthy response to inference request with status code 200 in 1.3363642692565918s
Received healthy response to inference request with status code 200 in 1.3239614963531494s
Received healthy response to inference request with status code 200 in 1.3108270168304443s
Received healthy response to inference request with status code 200 in 1.3177545070648193s
Received healthy response to inference request with status code 200 in 1.272808313369751s
Received healthy response to inference request with status code 200 in 1.3109612464904785s
Received healthy response to inference request with status code 200 in 1.3438961505889893s
Received healthy response to inference request with status code 200 in 1.3309662342071533s
Received healthy response to inference request with status code 200 in 1.329064130783081s
Received healthy response to inference request with status code 200 in 1.346121072769165s
Received healthy response to inference request with status code 200 in 1.4598097801208496s
Received healthy response to inference request with status code 200 in 1.3562281131744385s
Received healthy response to inference request with status code 200 in 1.366891622543335s
Received healthy response to inference request with status code 200 in 1.3324551582336426s
Received healthy response to inference request with status code 200 in 1.3485805988311768s
Received healthy response to inference request with status code 200 in 1.325514554977417s
Received healthy response to inference request with status code 200 in 1.3281800746917725s
Received healthy response to inference request with status code 200 in 1.3807332515716553s
Received healthy response to inference request with status code 200 in 1.3801565170288086s
Received healthy response to inference request with status code 200 in 1.3509502410888672s
Received healthy response to inference request with status code 200 in 1.349583625793457s
Received healthy response to inference request with status code 200 in 1.3127496242523193s
Received healthy response to inference request with status code 200 in 1.3454804420471191s
Received healthy response to inference request with status code 200 in 1.3407607078552246s
Received healthy response to inference request with status code 200 in 1.3330543041229248s
Received healthy response to inference request with status code 200 in 1.3202471733093262s
Received healthy response to inference request with status code 200 in 1.3451409339904785s
Received healthy response to inference request with status code 200 in 1.3393597602844238s
Received healthy response to inference request with status code 200 in 1.3716928958892822s
Received healthy response to inference request with status code 200 in 1.3521976470947266s
Received healthy response to inference request with status code 200 in 1.4187500476837158s
Received healthy response to inference request with status code 200 in 1.4947412014007568s
Received healthy response to inference request with status code 200 in 1.3326444625854492s
Received healthy response to inference request with status code 200 in 1.3242912292480469s
Received healthy response to inference request with status code 200 in 1.4316513538360596s
Received healthy response to inference request with status code 200 in 1.3847332000732422s
Received healthy response to inference request with status code 200 in 1.3205127716064453s
Received healthy response to inference request with status code 200 in 1.3115811347961426s
Received healthy response to inference request with status code 200 in 1.342071771621704s
Received healthy response to inference request with status code 200 in 1.3214163780212402s
Received healthy response to inference request with status code 200 in 1.334606647491455s
Received healthy response to inference request with status code 200 in 1.3362786769866943s
Received healthy response to inference request with status code 200 in 1.3154165744781494s
Received healthy response to inference request with status code 200 in 1.3462796211242676s
Received healthy response to inference request with status code 200 in 1.326974868774414s
Received healthy response to inference request with status code 200 in 1.3386280536651611s
Received healthy response to inference request with status code 200 in 1.3311309814453125s
Received healthy response to inference request with status code 200 in 1.335524320602417s
Received healthy response to inference request with status code 200 in 1.5934195518493652s
Received healthy response to inference request with status code 200 in 1.3571083545684814s
Received healthy response to inference request with status code 200 in 1.3411610126495361s
Received healthy response to inference request with status code 200 in 1.356074571609497s
Received healthy response to inference request with status code 200 in 1.3502466678619385s
Received healthy response to inference request with status code 200 in 2.3753020763397217s
Received healthy response to inference request with status code 200 in 1.320587396621704s
Received healthy response to inference request with status code 200 in 1.346900463104248s
Received healthy response to inference request with status code 200 in 1.3146581649780273s
Received healthy response to inference request with status code 200 in 1.314150333404541s
Received healthy response to inference request with status code 200 in 1.3344571590423584s
Received healthy response to inference request with status code 200 in 1.3328173160552979s
Received healthy response to inference request with status code 200 in 1.36421537399292s
Received healthy response to inference request with status code 200 in 1.3428449630737305s
Received healthy response to inference request with status code 200 in 1.3504338264465332s
Received healthy response to inference request with status code 200 in 1.3188164234161377s
Received healthy response to inference request with status code 200 in 1.3291690349578857s
Received healthy response to inference request with status code 200 in 1.3612501621246338s
Received healthy response to inference request with status code 200 in 1.3150019645690918s
Received healthy response to inference request with status code 200 in 1.3466150760650635s
Received healthy response to inference request with status code 200 in 1.310260534286499s
Received healthy response to inference request with status code 200 in 1.3371286392211914s
Received healthy response to inference request with status code 200 in 1.3417770862579346s
Received healthy response to inference request with status code 200 in 1.3107452392578125s
Received healthy response to inference request with status code 200 in 1.331125020980835s
Received healthy response to inference request with status code 200 in 1.341487169265747s
Received healthy response to inference request with status code 200 in 1.34531831741333s
Received healthy response to inference request with status code 200 in 1.3314142227172852s
Received healthy response to inference request with status code 200 in 1.313619613647461s
Received healthy response to inference request with status code 200 in 1.3638880252838135s
Received healthy response to inference request with status code 200 in 1.3576605319976807s
Received healthy response to inference request with status code 200 in 1.3519623279571533s
Received healthy response to inference request with status code 200 in 1.3305740356445312s
Received healthy response to inference request with status code 200 in 1.3346762657165527s
Received healthy response to inference request with status code 200 in 1.3288860321044922s
Received healthy response to inference request with status code 200 in 1.3311705589294434s
Received healthy response to inference request with status code 200 in 1.3727948665618896s
Received healthy response to inference request with status code 200 in 1.4168498516082764s
Received healthy response to inference request with status code 200 in 1.356867790222168s
Received healthy response to inference request with status code 200 in 1.3356664180755615s
Received healthy response to inference request with status code 200 in 1.3265247344970703s
100 requests
0 failed requests
5th percentile: 1.3107210040092467
10th percentile: 1.3120849609375
20th percentile: 1.3201631546020507
30th percentile: 1.3286742448806763
40th percentile: 1.3320387840270995
50th percentile: 1.335972547531128
60th percentile: 1.3418949604034425
70th percentile: 1.3474045038223266
80th percentile: 1.3569159030914306
90th percentile: 1.381133246421814
95th percentile: 1.433059275150299
99th percentile: 2.437811644077333
mean time: 1.4276245903968812
Pipeline stage StressChecker completed in 151.51s
Running pipeline stage SafetyScorer
Pipeline stage SafetyScorer completed in 38.97s
Running pipeline stage MEvalScorer
Running M-Eval for topic stay_in_character
Scoring model output for bot Salazar
Received score 7 for bot Salazar
Scoring model output for bot Alise (Everlasting Summer)
Received score 9 for bot Alise (Everlasting Summer)
Scoring model output for bot Your bad boy bestfriend (Liam)
Received score 8 for bot Your bad boy bestfriend (Liam)
Scoring model output for bot Miguel O’Hara
Received score 8 for bot Miguel O’Hara
Scoring model output for bot Bakugo Katsuki
Received score 8 for bot Bakugo Katsuki
Scoring model output for bot Smoker (One Piece)
Received score 8 for bot Smoker (One Piece)
Scoring model output for bot khalid von riegan
Received score 9 for bot khalid von riegan
Scoring model output for bot Tanjiro Kamado
Received score 9 for bot Tanjiro Kamado
Scoring model output for bot Bodyguard Konig
Received score 9 for bot Bodyguard Konig
Scoring model output for bot Ghost
Received score 9 for bot Ghost
Scoring model output for bot United Kingdom
Received score 8 for bot United Kingdom
Scoring model output for bot ENA
Received score 9 for bot ENA
Scoring model output for bot Angel
Received score 9 for bot Angel
Scoring model output for bot yandere kaeya
Received score 8 for bot yandere kaeya
Scoring model output for bot Lee
Received score 8 for bot Lee
Scoring model output for bot Noia (Tomboy)
Received score 8 for bot Noia (Tomboy)
Scoring model output for bot Nikolai Gogol || Fyodor Pov
Received score 8 for bot Nikolai Gogol || Fyodor Pov
Scoring model output for bot Sanemi | Butler | ASDJ
Received score 8 for bot Sanemi | Butler | ASDJ
Scoring model output for bot Ruwa
Received score 9 for bot Ruwa
Scoring model output for bot Katsuki Bakugo
Received score 9 for bot Katsuki Bakugo
Scoring model output for bot Acera
Received score 9 for bot Acera
Scoring model output for bot ⁠♡Therapy dog♡
Received score 9 for bot ⁠♡Therapy dog♡
Scoring model output for bot Nagito Komaeda (angst,comfort)
Received score 9 for bot Nagito Komaeda (angst,comfort)
Scoring model output for bot Jimin (Mafia Boyfriend)
Received score 9 for bot Jimin (Mafia Boyfriend)
Scoring model output for bot Enid Sinclair
Received score 9 for bot Enid Sinclair
Scoring model output for bot Hannah (depressed girl)
Received score 9 for bot Hannah (depressed girl)
Scoring model output for bot Miles Morales (your roommate)
Received score 8 for bot Miles Morales (your roommate)
Scoring model output for bot Bully girl
Received score 8 for bot Bully girl
Scoring model output for bot Miguel O’Hara
Received score 9 for bot Miguel O’Hara
Scoring model output for bot Daemonique
Received score 9 for bot Daemonique
Scoring model output for bot Drayton
Received score 9 for bot Drayton
Scoring model output for bot Katsuki Bakugo
Received score 9 for bot Katsuki Bakugo
Scoring model output for bot Leon (Obsessive Bully)
Received score 8 for bot Leon (Obsessive Bully)
Scoring model output for bot Stacy
Received score 9 for bot Stacy
Scoring model output for bot miguel ohara (ñsfw)
Received score 9 for bot miguel ohara (ñsfw)
Scoring model output for bot Mafia boss (Jamie, ex bf)
Received score 8 for bot Mafia boss (Jamie, ex bf)
Scoring model output for bot Zoe - The class representative
Received score 9 for bot Zoe - The class representative
Scoring model output for bot Tim Wright
Received score 9 for bot Tim Wright
Scoring model output for bot Space Girl Nina-chan
Received score 8 for bot Space Girl Nina-chan
Scoring model output for bot Xavier  [ Demon King Husband ]
Received score 9 for bot Xavier  [ Demon King Husband ]
Scoring model output for bot Kirlia
Received score 9 for bot Kirlia
Scoring model output for bot Mia (Dumb Bimbo)
Received score 8 for bot Mia (Dumb Bimbo)
Scoring model output for bot Diluc (ur colleague)
Received score 9 for bot Diluc (ur colleague)
Scoring model output for bot Javier Peña
Received score 9 for bot Javier Peña
Scoring model output for bot Mia (Dumb Bimbo)
Received score 8 for bot Mia (Dumb Bimbo)
Scoring model output for bot Wednesday Addams
Received score 9 for bot Wednesday Addams
Scoring model output for bot Rocket Raccoon
Received score 8 for bot Rocket Raccoon
Scoring model output for bot Demon Bakugou
Received score 9 for bot Demon Bakugou
Pipeline stage MEvalScorer completed in 376.67s
khoantap-missy-7b_v1 status is now inactive due to auto deactivation removed underperforming models
khoantap-missy-7b_v1 status is now deployed due to admin request
khoantap-missy-7b_v1 status is now inactive due to auto deactivation removed underperforming models
admin requested tearing down of khoantap-missy-7b_v1
Running pipeline stage ISVCDeleter
Checking if service khoantap-missy-7b-v1 is running
Tearing down inference service khoantap-missy-7b-v1
Toredown service khoantap-missy-7b-v1
Pipeline stage ISVCDeleter completed in 4.47s
Running pipeline stage MKMLModelDeleter
Cleaning model data from S3
Cleaning model data from model cache
Deleting key khoantap-missy-7b-v1/config.json from bucket guanaco-mkml-models
Deleting key khoantap-missy-7b-v1/mkml_model.tensors from bucket guanaco-mkml-models
Deleting key khoantap-missy-7b-v1/special_tokens_map.json from bucket guanaco-mkml-models
Deleting key khoantap-missy-7b-v1/tokenizer.json from bucket guanaco-mkml-models
Deleting key khoantap-missy-7b-v1/tokenizer_config.json from bucket guanaco-mkml-models
Cleaning model data from model cache
Deleting key khoantap-missy-7b-v1_reward/config.json from bucket guanaco-reward-models
Deleting key khoantap-missy-7b-v1_reward/merges.txt from bucket guanaco-reward-models
Deleting key khoantap-missy-7b-v1_reward/reward.tensors from bucket guanaco-reward-models
Deleting key khoantap-missy-7b-v1_reward/special_tokens_map.json from bucket guanaco-reward-models
Deleting key khoantap-missy-7b-v1_reward/tokenizer.json from bucket guanaco-reward-models
Deleting key khoantap-missy-7b-v1_reward/tokenizer_config.json from bucket guanaco-reward-models
Deleting key khoantap-missy-7b-v1_reward/vocab.json from bucket guanaco-reward-models
Pipeline stage MKMLModelDeleter completed in 2.71s
khoantap-missy-7b_v1 status is now torndown due to DeploymentManager action

Usage Metrics

Latency Metrics