submission_id: anhnv125-elephant_v9
developer_uid: chai_backend_admin
status: inactive
model_repo: anhnv125/elephant
reward_repo: anhnv125/reward-model-v2
generation_params: {'temperature': 1.2, 'top_p': 1.0, 'top_k': 20, 'presence_penalty': 0.0, 'frequency_penalty': 0.0, 'stopping_words': ['\n', '</s>', '<|im_end|>'], 'max_input_tokens': 1024, 'best_of': 8, 'max_output_tokens': 64}
formatter: {'memory_template': "### Instruction:\nAs the assistant, your task is to fully embody the given character, creating immersive, captivating narratives. Stay true to the character's personality and background, generating responses that not only reflect their core traits but are also accurate to their character. Your responses should evoke emotion, suspense, and anticipation in the user. The more detailed and descriptive your response, the more vivid the narrative becomes. Aim to create a fertile environment for ongoing interaction – introduce new elements, offer choices, or ask questions to invite the user to participate more fully in the conversation. This conversation is a dance, always continuing, always evolving.\nYour character: {bot_name}.\nContext: {memory}\n\n", 'prompt_template': '### Input:\n# Example conversation:\n{prompt}\n# Actual conversation:\n<START>\n', 'bot_template': '{bot_name}: {message}\n', 'user_template': '{user_name}: {message}\n', 'response_template': '### Response:\n{bot_name}:'}
timestamp: 2024-01-01T13:25:02+00:00
model_name: anhnv125-elephant_v9
safety_score: 0.99
entertaining: None
stay_in_character: None
user_preference: None
double_thumbs_up: 2898
thumbs_up: 5194
thumbs_down: 2410
num_battles: 86368
num_wins: 41056
win_ratio: 0.47536124490552056
celo_rating: 1138.66
Resubmit model
Running pipeline stage MKMLizer
Starting job with name anhnv125-elephant-v9-mkmlizer
Waiting for job on anhnv125-elephant-v9-mkmlizer to finish
anhnv125-elephant-v9-mkmlizer: ╔═════════════════════════════════════════════════════════════════════╗
anhnv125-elephant-v9-mkmlizer: ║ _______ __ __ _______ _____ ║
anhnv125-elephant-v9-mkmlizer: ║ | | | |/ | | | |_ ║
anhnv125-elephant-v9-mkmlizer: ║ | | <| | | ║
anhnv125-elephant-v9-mkmlizer: ║ |__|_|__|__|\__|__|_|__|_______| ║
anhnv125-elephant-v9-mkmlizer: ║ ║
anhnv125-elephant-v9-mkmlizer: ║ Copyright 2023 MK ONE TECHNOLOGIES Inc. ║
anhnv125-elephant-v9-mkmlizer: ║ ║
anhnv125-elephant-v9-mkmlizer: ║ The license key for the current software has been verified as ║
anhnv125-elephant-v9-mkmlizer: ║ belonging to: ║
anhnv125-elephant-v9-mkmlizer: ║ ║
anhnv125-elephant-v9-mkmlizer: ║ Chai Research Corp. ║
anhnv125-elephant-v9-mkmlizer: ║ Account ID: 7997a29f-0ceb-4cc7-9adf-840c57b4ae6f ║
anhnv125-elephant-v9-mkmlizer: ║ Expiration: 2024-04-15 23:59:59 ║
anhnv125-elephant-v9-mkmlizer: ║ ║
anhnv125-elephant-v9-mkmlizer: ╚═════════════════════════════════════════════════════════════════════╝
anhnv125-elephant-v9-mkmlizer: loading model from anhnv125/elephant
anhnv125-elephant-v9-mkmlizer: /opt/conda/lib/python3.10/site-packages/transformers/models/auto/configuration_auto.py:1067: FutureWarning: The `use_auth_token` argument is deprecated and will be removed in v5 of Transformers. Please use `token` instead.
anhnv125-elephant-v9-mkmlizer: warnings.warn(
anhnv125-elephant-v9-mkmlizer: config.json: 0%| | 0.00/600 [00:00<?, ?B/s] config.json: 100%|██████████| 600/600 [00:00<00:00, 4.38MB/s]
anhnv125-elephant-v9-mkmlizer: /opt/conda/lib/python3.10/site-packages/transformers/models/auto/tokenization_auto.py:690: FutureWarning: The `use_auth_token` argument is deprecated and will be removed in v5 of Transformers. Please use `token` instead.
anhnv125-elephant-v9-mkmlizer: warnings.warn(
anhnv125-elephant-v9-mkmlizer: tokenizer_config.json: 0%| | 0.00/1.02k [00:00<?, ?B/s] tokenizer_config.json: 100%|██████████| 1.02k/1.02k [00:00<00:00, 7.69MB/s]
anhnv125-elephant-v9-mkmlizer: tokenizer.model: 0%| | 0.00/493k [00:00<?, ?B/s] tokenizer.model: 100%|██████████| 493k/493k [00:00<00:00, 29.9MB/s]
anhnv125-elephant-v9-mkmlizer: special_tokens_map.json: 0%| | 0.00/437 [00:00<?, ?B/s] special_tokens_map.json: 100%|██████████| 437/437 [00:00<00:00, 2.86MB/s]
anhnv125-elephant-v9-mkmlizer: /opt/conda/lib/python3.10/site-packages/transformers/models/auto/auto_factory.py:472: FutureWarning: The `use_auth_token` argument is deprecated and will be removed in v5 of Transformers. Please use `token` instead.
anhnv125-elephant-v9-mkmlizer: warnings.warn(
anhnv125-elephant-v9-mkmlizer: pytorch_model.bin.index.json: 0%| | 0.00/23.9k [00:00<?, ?B/s] pytorch_model.bin.index.json: 100%|██████████| 23.9k/23.9k [00:00<00:00, 147MB/s]
anhnv125-elephant-v9-mkmlizer: Downloading shards: 0%| | 0/3 [00:00<?, ?it/s]
anhnv125-elephant-v9-mkmlizer: pytorch_model-00001-of-00003.bin: 0%| | 0.00/4.94G [00:00<?, ?B/s]
anhnv125-elephant-v9-mkmlizer: pytorch_model-00001-of-00003.bin: 0%| | 10.5M/4.94G [00:00<02:55, 28.2MB/s]
anhnv125-elephant-v9-mkmlizer: pytorch_model-00001-of-00003.bin: 1%| | 31.5M/4.94G [00:00<01:04, 76.5MB/s]
anhnv125-elephant-v9-mkmlizer: pytorch_model-00001-of-00003.bin: 1%|▏ | 62.9M/4.94G [00:00<00:34, 143MB/s] 
anhnv125-elephant-v9-mkmlizer: pytorch_model-00001-of-00003.bin: 3%|▎ | 136M/4.94G [00:00<00:16, 288MB/s] 
anhnv125-elephant-v9-mkmlizer: pytorch_model-00001-of-00003.bin: 4%|▍ | 189M/4.94G [00:00<00:13, 340MB/s]
anhnv125-elephant-v9-mkmlizer: pytorch_model-00001-of-00003.bin: 5%|▍ | 231M/4.94G [00:00<00:13, 361MB/s]
anhnv125-elephant-v9-mkmlizer: pytorch_model-00001-of-00003.bin: 27%|██▋ | 1.33G/4.94G [00:02<00:04, 758MB/s]
anhnv125-elephant-v9-mkmlizer: pytorch_model-00001-of-00003.bin: 30%|███ | 1.50G/4.94G [00:02<00:03, 934MB/s]
anhnv125-elephant-v9-mkmlizer: pytorch_model-00001-of-00003.bin: 32%|███▏ | 1.59G/4.94G [00:02<00:03, 901MB/s]
anhnv125-elephant-v9-mkmlizer: pytorch_model-00001-of-00003.bin: 35%|███▌ | 1.74G/4.94G [00:02<00:03, 1.04GB/s]
anhnv125-elephant-v9-mkmlizer: pytorch_model-00001-of-00003.bin: 38%|███▊ | 1.89G/4.94G [00:02<00:02, 1.14GB/s]
anhnv125-elephant-v9-mkmlizer: pytorch_model-00001-of-00003.bin: 42%|████▏ | 2.08G/4.94G [00:03<00:02, 1.34GB/s]
anhnv125-elephant-v9-mkmlizer: pytorch_model-00001-of-00003.bin: 45%|████▍ | 2.22G/4.94G [00:03<00:02, 1.31GB/s]
anhnv125-elephant-v9-mkmlizer: pytorch_model-00001-of-00003.bin: 48%|████▊ | 2.38G/4.94G [00:03<00:01, 1.38GB/s]
anhnv125-elephant-v9-mkmlizer: pytorch_model-00001-of-00003.bin: 54%|█████▍ | 2.66G/4.94G [00:03<00:01, 1.75GB/s]
anhnv125-elephant-v9-mkmlizer: pytorch_model-00001-of-00003.bin: 57%|█████▋ | 2.84G/4.94G [00:03<00:01, 1.64GB/s]
anhnv125-elephant-v9-mkmlizer: pytorch_model-00001-of-00003.bin: 61%|██████ | 3.01G/4.94G [00:03<00:01, 1.15GB/s]
anhnv125-elephant-v9-mkmlizer: pytorch_model-00001-of-00003.bin: 64%|██████▎ | 3.15G/4.94G [00:03<00:01, 1.09GB/s]
anhnv125-elephant-v9-mkmlizer: pytorch_model-00001-of-00003.bin: 66%|██████▌ | 3.27G/4.94G [00:03<00:01, 1.07GB/s]
anhnv125-elephant-v9-mkmlizer: pytorch_model-00001-of-00003.bin: 69%|██████▊ | 3.40G/4.94G [00:04<00:01, 898MB/s] 
anhnv125-elephant-v9-mkmlizer: pytorch_model-00001-of-00003.bin: 71%|███████ | 3.50G/4.94G [00:04<00:01, 841MB/s]
anhnv125-elephant-v9-mkmlizer: pytorch_model-00001-of-00003.bin: 73%|███████▎ | 3.61G/4.94G [00:04<00:01, 843MB/s]
anhnv125-elephant-v9-mkmlizer: pytorch_model-00001-of-00003.bin: 75%|███████▍ | 3.70G/4.94G [00:04<00:01, 861MB/s]
anhnv125-elephant-v9-mkmlizer: pytorch_model-00001-of-00003.bin: 77%|███████▋ | 3.83G/4.94G [00:04<00:01, 931MB/s]
anhnv125-elephant-v9-mkmlizer: pytorch_model-00001-of-00003.bin: 80%|███████▉ | 3.93G/4.94G [00:04<00:01, 915MB/s]
anhnv125-elephant-v9-mkmlizer: pytorch_model-00001-of-00003.bin: 83%|████████▎ | 4.11G/4.94G [00:04<00:00, 1.12GB/s]
anhnv125-elephant-v9-mkmlizer: pytorch_model-00001-of-00003.bin: 86%|████████▌ | 4.23G/4.94G [00:05<00:00, 1.08GB/s]
anhnv125-elephant-v9-mkmlizer: pytorch_model-00001-of-00003.bin: 88%|████████▊ | 4.36G/4.94G [00:05<00:00, 1.11GB/s]
anhnv125-elephant-v9-mkmlizer: pytorch_model-00001-of-00003.bin: 90%|█████████ | 4.47G/4.94G [00:05<00:00, 1.11GB/s]
anhnv125-elephant-v9-mkmlizer: pytorch_model-00001-of-00003.bin: 93%|█████████▎| 4.61G/4.94G [00:05<00:00, 1.09GB/s]
anhnv125-elephant-v9-mkmlizer: pytorch_model-00001-of-00003.bin: 98%|█████████▊| 4.83G/4.94G [00:05<00:00, 1.39GB/s] pytorch_model-00001-of-00003.bin: 100%|█████████▉| 4.94G/4.94G [00:05<00:00, 856MB/s]
anhnv125-elephant-v9-mkmlizer: Downloading shards: 33%|███▎ | 1/3 [00:06<00:12, 6.09s/it]
anhnv125-elephant-v9-mkmlizer: pytorch_model-00002-of-00003.bin: 0%| | 0.00/5.00G [00:00<?, ?B/s]
anhnv125-elephant-v9-mkmlizer: pytorch_model-00002-of-00003.bin: 0%| | 10.5M/5.00G [00:00<01:33, 53.5MB/s]
anhnv125-elephant-v9-mkmlizer: pytorch_model-00002-of-00003.bin: 0%| | 21.0M/5.00G [00:00<01:10, 70.9MB/s]
anhnv125-elephant-v9-mkmlizer: pytorch_model-00002-of-00003.bin: 1%|▏ | 62.9M/5.00G [00:00<00:28, 172MB/s] 
anhnv125-elephant-v9-mkmlizer: pytorch_model-00002-of-00003.bin: 2%|▏ | 105M/5.00G [00:00<00:22, 221MB/s] 
anhnv125-elephant-v9-mkmlizer: pytorch_model-00002-of-00003.bin: 4%|▎ | 178M/5.00G [00:00<00:13, 353MB/s]
anhnv125-elephant-v9-mkmlizer: pytorch_model-00002-of-00003.bin: 5%|▌ | 262M/5.00G [00:00<00:10, 443MB/s]
anhnv125-elephant-v9-mkmlizer: pytorch_model-00002-of-00003.bin: 7%|▋ | 346M/5.00G [00:00<00:08, 539MB/s]
anhnv125-elephant-v9-mkmlizer: pytorch_model-00002-of-00003.bin: 9%|▊ | 430M/5.00G [00:01<00:07, 612MB/s]
anhnv125-elephant-v9-mkmlizer: pytorch_model-00002-of-00003.bin: 10%|█ | 503M/5.00G [00:01<00:07, 633MB/s]
anhnv125-elephant-v9-mkmlizer: pytorch_model-00002-of-00003.bin: 12%|█▏ | 598M/5.00G [00:01<00:06, 678MB/s]
anhnv125-elephant-v9-mkmlizer: pytorch_model-00002-of-00003.bin: 13%|█▎ | 671M/5.00G [00:01<00:07, 586MB/s]
anhnv125-elephant-v9-mkmlizer: pytorch_model-00002-of-00003.bin: 16%|█▌ | 786M/5.00G [00:01<00:06, 699MB/s]
anhnv125-elephant-v9-mkmlizer: pytorch_model-00002-of-00003.bin: 17%|█▋ | 860M/5.00G [00:01<00:06, 660MB/s]
anhnv125-elephant-v9-mkmlizer: pytorch_model-00002-of-00003.bin: 19%|█▉ | 954M/5.00G [00:01<00:06, 646MB/s]
anhnv125-elephant-v9-mkmlizer: pytorch_model-00002-of-00003.bin: 21%|██ | 1.03G/5.00G [00:02<00:07, 509MB/s]
anhnv125-elephant-v9-mkmlizer: pytorch_model-00002-of-00003.bin: 23%|██▎ | 1.15G/5.00G [00:02<00:06, 634MB/s]
anhnv125-elephant-v9-mkmlizer: pytorch_model-00002-of-00003.bin: 25%|██▍ | 1.23G/5.00G [00:02<00:05, 654MB/s]
anhnv125-elephant-v9-mkmlizer: pytorch_model-00002-of-00003.bin: 27%|██▋ | 1.33G/5.00G [00:02<00:05, 726MB/s]
anhnv125-elephant-v9-mkmlizer: pytorch_model-00002-of-00003.bin: 29%|██▉ | 1.47G/5.00G [00:02<00:04, 845MB/s]
anhnv125-elephant-v9-mkmlizer: pytorch_model-00002-of-00003.bin: 31%|███ | 1.56G/5.00G [00:02<00:04, 814MB/s]
anhnv125-elephant-v9-mkmlizer: pytorch_model-00002-of-00003.bin: 35%|███▍ | 1.73G/5.00G [00:02<00:03, 1.03GB/s]
anhnv125-elephant-v9-mkmlizer: pytorch_model-00002-of-00003.bin: 38%|███▊ | 1.90G/5.00G [00:02<00:02, 1.17GB/s]
anhnv125-elephant-v9-mkmlizer: pytorch_model-00002-of-00003.bin: 42%|████▏ | 2.08G/5.00G [00:03<00:02, 1.24GB/s]
anhnv125-elephant-v9-mkmlizer: pytorch_model-00002-of-00003.bin: 45%|████▍ | 2.24G/5.00G [00:03<00:02, 1.31GB/s]
anhnv125-elephant-v9-mkmlizer: pytorch_model-00002-of-00003.bin: 48%|████▊ | 2.40G/5.00G [00:03<00:01, 1.38GB/s]
anhnv125-elephant-v9-mkmlizer: pytorch_model-00002-of-00003.bin: 52%|█████▏ | 2.61G/5.00G [00:03<00:01, 1.57GB/s]
anhnv125-elephant-v9-mkmlizer: pytorch_model-00002-of-00003.bin: 58%|█████▊ | 2.88G/5.00G [00:03<00:01, 1.89GB/s]
anhnv125-elephant-v9-mkmlizer: pytorch_model-00002-of-00003.bin: 62%|██████▏ | 3.08G/5.00G [00:03<00:01, 1.68GB/s]
anhnv125-elephant-v9-mkmlizer: pytorch_model-00002-of-00003.bin: 65%|██████▌ | 3.26G/5.00G [00:03<00:01, 1.26GB/s]
anhnv125-elephant-v9-mkmlizer: pytorch_model-00002-of-00003.bin: 68%|██████▊ | 3.41G/5.00G [00:03<00:01, 1.12GB/s]
anhnv125-elephant-v9-mkmlizer: pytorch_model-00002-of-00003.bin: 71%|███████ | 3.54G/5.00G [00:04<00:01, 1.09GB/s]
anhnv125-elephant-v9-mkmlizer: pytorch_model-00002-of-00003.bin: 73%|███████▎ | 3.67G/5.00G [00:04<00:01, 1.09GB/s]
anhnv125-elephant-v9-mkmlizer: pytorch_model-00002-of-00003.bin: 76%|███████▌ | 3.79G/5.00G [00:04<00:01, 1.07GB/s]
anhnv125-elephant-v9-mkmlizer: pytorch_model-00002-of-00003.bin: 78%|███████▊ | 3.90G/5.00G [00:04<00:01, 1.04GB/s]
anhnv125-elephant-v9-mkmlizer: pytorch_model-00002-of-00003.bin: 80%|████████ | 4.02G/5.00G [00:04<00:00, 1.02GB/s]
anhnv125-elephant-v9-mkmlizer: pytorch_model-00002-of-00003.bin: 82%|████████▏ | 4.12G/5.00G [00:04<00:01, 831MB/s] 
anhnv125-elephant-v9-mkmlizer: pytorch_model-00002-of-00003.bin: 85%|████████▍ | 4.24G/5.00G [00:04<00:00, 919MB/s]
anhnv125-elephant-v9-mkmlizer: pytorch_model-00002-of-00003.bin: 88%|████████▊ | 4.38G/5.00G [00:04<00:00, 997MB/s]
anhnv125-elephant-v9-mkmlizer: pytorch_model-00002-of-00003.bin: 90%|████████▉ | 4.50G/5.00G [00:05<00:00, 1.01GB/s]
anhnv125-elephant-v9-mkmlizer: pytorch_model-00002-of-00003.bin: 92%|█████████▏| 4.61G/5.00G [00:05<00:00, 961MB/s] 
anhnv125-elephant-v9-mkmlizer: pytorch_model-00002-of-00003.bin: 95%|█████████▌| 4.77G/5.00G [00:05<00:00, 1.11GB/s]
anhnv125-elephant-v9-mkmlizer: pytorch_model-00002-of-00003.bin: 100%|█████████▉| 5.00G/5.00G [00:05<00:00, 1.21GB/s] pytorch_model-00002-of-00003.bin: 100%|█████████▉| 5.00G/5.00G [00:05<00:00, 902MB/s]
anhnv125-elephant-v9-mkmlizer: Downloading shards: 67%|██████▋ | 2/3 [00:11<00:05, 5.96s/it]
anhnv125-elephant-v9-mkmlizer: pytorch_model-00003-of-00003.bin: 0%| | 0.00/4.54G [00:00<?, ?B/s]
anhnv125-elephant-v9-mkmlizer: pytorch_model-00003-of-00003.bin: 0%| | 10.5M/4.54G [00:00<01:46, 42.4MB/s]
anhnv125-elephant-v9-mkmlizer: pytorch_model-00003-of-00003.bin: 55%|█████▌ | 2.51G/4.54G [00:03<00:01, 1.17GB/s]
anhnv125-elephant-v9-mkmlizer: pytorch_model-00003-of-00003.bin: 58%|█████▊ | 2.65G/4.54G [00:03<00:01, 1.02GB/s]
anhnv125-elephant-v9-mkmlizer: pytorch_model-00003-of-00003.bin: 61%|██████ | 2.78G/4.54G [00:03<00:01, 984MB/s] 
anhnv125-elephant-v9-mkmlizer: pytorch_model-00003-of-00003.bin: 64%|██████▎ | 2.89G/4.54G [00:03<00:01, 937MB/s]
anhnv125-elephant-v9-mkmlizer: pytorch_model-00003-of-00003.bin: 67%|██████▋ | 3.02G/4.54G [00:03<00:01, 999MB/s]
anhnv125-elephant-v9-mkmlizer: pytorch_model-00003-of-00003.bin: 69%|██████▉ | 3.14G/4.54G [00:03<00:01, 969MB/s]
anhnv125-elephant-v9-mkmlizer: pytorch_model-00003-of-00003.bin: 71%|███████▏ | 3.24G/4.54G [00:04<00:01, 913MB/s]
anhnv125-elephant-v9-mkmlizer: pytorch_model-00003-of-00003.bin: 74%|███████▎ | 3.34G/4.54G [00:04<00:01, 922MB/s]
anhnv125-elephant-v9-mkmlizer: pytorch_model-00003-of-00003.bin: 76%|███████▌ | 3.45G/4.54G [00:04<00:01, 857MB/s]
anhnv125-elephant-v9-mkmlizer: pytorch_model-00003-of-00003.bin: 79%|███████▉ | 3.58G/4.54G [00:04<00:01, 891MB/s]
anhnv125-elephant-v9-mkmlizer: pytorch_model-00003-of-00003.bin: 82%|████████▏ | 3.71G/4.54G [00:04<00:00, 981MB/s]
anhnv125-elephant-v9-mkmlizer: pytorch_model-00003-of-00003.bin: 84%|████████▍ | 3.82G/4.54G [00:04<00:00, 988MB/s]
anhnv125-elephant-v9-mkmlizer: pytorch_model-00003-of-00003.bin: 87%|████████▋ | 3.94G/4.54G [00:04<00:00, 1.06GB/s]
anhnv125-elephant-v9-mkmlizer: pytorch_model-00003-of-00003.bin: 89%|████████▉ | 4.06G/4.54G [00:04<00:00, 972MB/s] 
anhnv125-elephant-v9-mkmlizer: pytorch_model-00003-of-00003.bin: 92%|█████████▏| 4.17G/4.54G [00:05<00:00, 996MB/s]
anhnv125-elephant-v9-mkmlizer: pytorch_model-00003-of-00003.bin: 100%|█████████▉| 4.52G/4.54G [00:05<00:00, 1.63GB/s] pytorch_model-00003-of-00003.bin: 100%|█████████▉| 4.54G/4.54G [00:05<00:00, 871MB/s]
anhnv125-elephant-v9-mkmlizer: Downloading shards: 100%|██████████| 3/3 [00:17<00:00, 5.78s/it] Downloading shards: 100%|██████████| 3/3 [00:17<00:00, 5.85s/it]
anhnv125-elephant-v9-mkmlizer: Loading checkpoint shards: 0%| | 0/3 [00:00<?, ?it/s] Loading checkpoint shards: 33%|███▎ | 1/3 [00:03<00:06, 3.31s/it] Loading checkpoint shards: 67%|██████▋ | 2/3 [00:06<00:03, 3.15s/it] Loading checkpoint shards: 100%|██████████| 3/3 [00:09<00:00, 2.95s/it] Loading checkpoint shards: 100%|██████████| 3/3 [00:09<00:00, 3.02s/it]
anhnv125-elephant-v9-mkmlizer: generation_config.json: 0%| | 0.00/116 [00:00<?, ?B/s] generation_config.json: 100%|██████████| 116/116 [00:00<00:00, 999kB/s]
anhnv125-elephant-v9-mkmlizer: loaded model in 28.356s
anhnv125-elephant-v9-mkmlizer: saved to disk in 26.671s
anhnv125-elephant-v9-mkmlizer: quantizing model to /tmp/model_cache
anhnv125-elephant-v9-mkmlizer: Saving mkml model at /tmp/model_cache
anhnv125-elephant-v9-mkmlizer: Reading /tmp/tmp8wyp6r1v/model.safetensors.index.json
anhnv125-elephant-v9-mkmlizer: Profiling: 0%| | 0/291 [00:00<?, ?it/s] Profiling: 0%| | 1/291 [00:01<05:38, 1.17s/it] Profiling: 1%| | 3/291 [00:01<02:45, 1.74it/s] Profiling: 1%|▏ | 4/291 [00:02<02:50, 1.68it/s] Profiling: 2%|▏ | 5/291 [00:03<02:47, 1.70it/s] Profiling: 3%|▎ | 8/291 [00:03<01:23, 3.38it/s] Profiling: 3%|▎ | 9/291 [00:03<01:20, 3.50it/s] Profiling: 3%|▎ | 10/291 [00:03<01:09, 4.04it/s] Profiling: 4%|▍ | 12/291 [00:04<01:16, 3.66it/s] Profiling: 4%|▍ | 13/291 [00:05<01:38, 2.81it/s] Profiling: 5%|▍ | 14/291 [00:05<01:57, 2.36it/s] Profiling: 5%|▌ | 16/291 [00:05<01:15, 3.64it/s] Profiling: 6%|▌ | 17/291 [00:05<01:08, 3.98it/s] Profiling: 6%|▌ | 18/291 [00:06<01:06, 4.10it/s] Profiling: 7%|▋ | 21/291 [00:06<01:01, 4.41it/s] Profiling: 8%|▊ | 22/291 [00:07<01:34, 2.84it/s] Profiling: 8%|▊ | 23/291 [00:08<02:09, 2.08it/s] Profiling: 9%|▊ | 25/291 [00:08<01:24, 3.14it/s] Profiling: 9%|▉ | 26/291 [00:09<01:24, 3.13it/s] Profiling: 9%|▉ | 27/291 [00:09<01:24, 3.13it/s] Profiling: 10%|▉ | 28/291 [00:09<01:11, 3.66it/s] Profiling: 10%|█ | 30/291 [00:10<01:25, 3.06it/s] Profiling: 11%|█ | 31/291 [00:11<02:00, 2.15it/s] Profiling: 11%|█ | 32/291 [00:11<02:06, 2.04it/s] Profiling: 12%|█▏ | 35/291 [00:12<01:10, 3.65it/s] Profiling: 12%|█▏ | 36/291 [00:12<01:04, 3.93it/s] Profiling: 13%|█▎ | 39/291 [00:12<00:58, 4.34it/s] Profiling: 14%|█▎ | 40/291 [00:13<01:13, 3.43it/s] Profiling: 14%|█▍ | 41/291 [00:14<01:31, 2.74it/s] Profiling: 15%|█▌ | 44/291 [00:14<00:57, 4.32it/s] Profiling: 15%|█▌ | 45/291 [00:14<00:55, 4.46it/s] Profiling: 16%|█▌ | 47/291 [00:14<00:41, 5.84it/s] Profiling: 16%|█▋ | 48/291 [00:14<00:46, 5.25it/s] Profiling: 17%|█▋ | 49/291 [00:14<00:41, 5.83it/s] Profiling: 17%|█▋ | 50/291 [00:15<01:24, 2.84it/s] Profiling: 18%|█▊ | 51/291 [00:17<02:12, 1.81it/s] Profiling: 18%|█▊ | 52/291 [00:17<01:48, 2.20it/s] Profiling: 18%|█▊ | 53/291 [00:17<01:44, 2.27it/s] Profiling: 19%|█▊ | 54/291 [00:17<01:35, 2.48it/s] Profiling: 20%|█▉ | 57/291 [00:19<01:32, 2.53it/s] Profiling: 20%|█▉ | 58/291 [00:20<02:02, 1.90it/s] Profiling: 20%|██ | 59/291 [00:20<02:18, 1.68it/s] Profiling: 21%|██ | 61/291 [00:21<01:30, 2.54it/s] Profiling: 22%|██▏ | 63/291 [00:21<01:20, 2.84it/s] Profiling: 22%|██▏ | 64/291 [00:22<01:29, 2.52it/s] Profiling: 22%|██▏ | 65/291 [00:22<01:38, 2.30it/s] Profiling: 23%|██▎ | 68/291 [00:23<00:56, 3.91it/s] Profiling: 24%|██▎ | 69/291 [00:23<00:52, 4.20it/s] Profiling: 25%|██▍ | 72/291 [00:23<00:48, 4.50it/s] Profiling: 25%|██▌ | 73/291 [00:24<01:01, 3.55it/s] Profiling: 25%|██▌ | 74/291 [00:24<01:15, 2.87it/s] Profiling: 26%|██▋ | 77/291 [00:25<00:47, 4.49it/s] Profiling: 27%|██▋ | 78/291 [00:25<00:45, 4.67it/s] Profiling: 28%|██▊ | 81/291 [00:25<00:43, 4.79it/s] Profiling: 28%|██▊ | 82/291 [00:26<00:56, 3.69it/s] Profiling: 29%|██▊ | 83/291 [00:27<01:08, 3.04it/s] Profiling: 30%|██▉ | 86/291 [00:27<00:42, 4.77it/s] Profiling: 30%|██▉ | 87/291 [00:27<00:41, 4.94it/s] Profiling: 31%|███ | 90/291 [00:28<00:40, 4.93it/s] Profiling: 31%|███▏ | 91/291 [00:28<00:52, 3.82it/s] Profiling: 32%|███▏ | 92/291 [00:29<01:04, 3.11it/s] Profiling: 32%|███▏ | 94/291 [00:29<00:44, 4.44it/s] Profiling: 33%|███▎ | 95/291 [00:29<00:41, 4.71it/s] Profiling: 33%|███▎ | 96/291 [00:29<00:39, 4.96it/s] Profiling: 34%|███▎ | 98/291 [00:29<00:29, 6.56it/s] Profiling: 34%|███▍ | 99/291 [00:30<00:45, 4.20it/s] Profiling: 35%|███▌ | 102/291 [00:30<00:40, 4.71it/s] Profiling: 35%|███▌ | 103/291 [00:31<00:51, 3.63it/s] Profiling: 36%|███▌ | 104/291 [00:31<01:02, 2.97it/s] Profiling: 37%|███▋ | 107/291 [00:32<00:38, 4.78it/s] Profiling: 37%|███▋ | 108/291 [00:32<00:39, 4.66it/s] Profiling: 37%|███▋ | 109/291 [00:32<00:35, 5.13it/s] Profiling: 38%|███▊ | 111/291 [00:33<00:41, 4.30it/s] Profiling: 38%|███▊ | 112/291 [00:33<00:55, 3.22it/s] Profiling: 39%|███▉ | 113/291 [00:34<01:08, 2.61it/s] Profiling: 40%|███▉ | 115/291 [00:34<00:44, 3.96it/s] Profiling: 40%|███▉ | 116/291 [00:34<00:42, 4.08it/s] Profiling: 40%|████ | 117/291 [00:34<00:39, 4.42it/s] Profiling: 41%|████ | 120/291 [00:35<00:36, 4.74it/s] Profiling: 42%|████▏ | 121/291 [00:36<00:47, 3.61it/s] Profiling: 42%|████▏ | 122/291 [00:36<00:57, 2.96it/s] Profiling: 43%|████▎ | 125/291 [00:36<00:34, 4.83it/s] Profiling: 43%|████▎ | 126/291 [00:36<00:32, 5.03it/s] Profiling: 44%|████▍ | 129/291 [00:37<00:32, 5.05it/s] Profiling: 45%|████▍ | 130/291 [00:38<00:42, 3.83it/s] Profiling: 45%|████▌ | 131/291 [00:38<00:52, 3.04it/s] Profiling: 46%|████▌ | 133/291 [00:38<00:36, 4.31it/s] Profiling: 46%|████▌ | 134/291 [00:39<00:36, 4.27it/s] Profiling: 46%|████▋ | 135/291 [00:39<00:36, 4.25it/s] Profiling: 47%|████▋ | 136/291 [00:39<00:32, 4.84it/s] Profiling: 47%|████▋ | 138/291 [00:40<00:37, 4.04it/s] Profiling: 48%|████▊ | 139/291 [00:40<00:50, 3.02it/s] Profiling: 48%|████▊ | 140/291 [00:41<01:01, 2.47it/s] Profiling: 49%|████▉ | 142/291 [00:41<00:39, 3.80it/s] Profiling: 49%|████▉ | 143/291 [00:41<00:38, 3.89it/s] Profiling: 49%|████▉ | 144/291 [00:41<00:37, 3.96it/s] Profiling: 50%|████▉ | 145/291 [00:41<00:31, 4.57it/s] Profiling: 50%|█████ | 146/291 [00:42<00:46, 3.11it/s] Profiling: 51%|█████ | 148/291 [00:42<00:33, 4.23it/s] Profiling: 51%|█████ | 149/291 [00:43<00:31, 4.53it/s] Profiling: 52%|█████▏ | 150/291 [00:43<00:27, 5.19it/s] Profiling: 52%|█████▏ | 151/291 [00:43<00:23, 5.86it/s] Profiling: 52%|█████▏ | 152/291 [00:43<00:40, 3.43it/s] Profiling: 53%|█████▎ | 153/291 [00:44<00:52, 2.62it/s] Profiling: 54%|█████▎ | 156/291 [00:45<00:38, 3.49it/s] Profiling: 54%|█████▍ | 157/291 [00:45<00:47, 2.83it/s] Profiling: 54%|█████▍ | 158/291 [00:46<00:55, 2.41it/s] Profiling: 55%|█████▍ | 160/291 [00:46<00:35, 3.65it/s] Profiling: 55%|█████▌ | 161/291 [00:46<00:33, 3.88it/s] Profiling: 56%|█████▌ | 162/291 [00:46<00:30, 4.25it/s] Profiling: 57%|█████▋ | 165/291 [00:47<00:27, 4.59it/s] Profiling: 57%|█████▋ | 166/291 [00:47<00:36, 3.42it/s] Profiling: 57%|█████▋ | 167/291 [00:48<00:43, 2.84it/s] Profiling: 58%|█████▊ | 170/291 [00:48<00:26, 4.59it/s] Profiling: 59%|█████▉ | 171/291 [00:48<00:24, 4.81it/s] Profiling: 60%|█████▉ | 174/291 [00:49<00:23, 4.93it/s] Profiling: 60%|██████ | 175/291 [00:50<00:30, 3.79it/s] Profiling: 60%|██████ | 176/291 [00:50<00:36, 3.12it/s] Profiling: 62%|██████▏ | 179/291 [00:50<00:22, 4.91it/s] Profiling: 62%|██████▏ | 180/291 [00:51<00:21, 5.08it/s] Profiling: 63%|██████▎ | 183/291 [00:51<00:21, 5.07it/s] Profiling: 63%|██████▎ | 184/291 [00:52<00:27, 3.87it/s] Profiling: 64%|██████▎ | 185/291 [00:52<00:33, 3.17it/s] Profiling: 65%|██████▍ | 188/291 [00:52<00:20, 4.96it/s] Profiling: 65%|██████▍ | 189/291 [00:53<00:19, 5.13it/s] Profiling: 66%|██████▌ | 192/291 [00:53<00:21, 4.64it/s] Profiling: 66%|██████▋ | 193/291 [00:54<00:27, 3.61it/s] Profiling: 67%|██████▋ | 194/291 [00:54<00:32, 3.00it/s] Profiling: 68%|██████▊ | 197/291 [00:55<00:20, 4.67it/s] Profiling: 68%|██████▊ | 198/291 [00:55<00:19, 4.87it/s] Profiling: 69%|██████▊ | 200/291 [00:55<00:14, 6.41it/s] Profiling: 69%|██████▉ | 201/291 [00:55<00:14, 6.35it/s] Profiling: 69%|██████▉ | 202/291 [00:55<00:14, 6.30it/s] Profiling: 70%|███████ | 204/291 [00:55<00:11, 7.59it/s] Profiling: 70%|███████ | 205/291 [00:56<00:19, 4.42it/s] Profiling: 71%|███████ | 206/291 [00:57<00:25, 3.27it/s] Profiling: 71%|███████ | 207/291 [00:57<00:31, 2.67it/s] Profiling: 72%|███████▏ | 210/291 [00:58<00:22, 3.63it/s] Profiling: 73%|███████▎ | 211/291 [00:58<00:26, 3.02it/s] Profiling: 73%|███████▎ | 212/291 [00:59<00:30, 2.63it/s] Profiling: 74%|███████▍ | 215/291 [00:59<00:17, 4.36it/s] Profiling: 74%|███████▍ | 216/291 [00:59<00:16, 4.61it/s] Profiling: 75%|███████▌ | 219/291 [01:00<00:15, 4.67it/s] Profiling: 76%|███████▌ | 220/291 [01:00<00:20, 3.46it/s] Profiling: 76%|███████▌ | 221/291 [01:01<00:24, 2.81it/s] Profiling: 77%|███████▋ | 223/291 [01:01<00:16, 4.02it/s] Profiling: 77%|███████▋ | 224/291 [01:01<00:16, 4.03it/s] Profiling: 77%|███████▋ | 225/291 [01:02<00:16, 4.05it/s] Profiling: 78%|███████▊ | 226/291 [01:02<00:14, 4.62it/s] Profiling: 78%|███████▊ | 228/291 [01:02<00:16, 3.93it/s] Profiling: 79%|███████▊ | 229/291 [01:03<00:20, 2.97it/s] Profiling: 79%|███████▉ | 230/291 [01:04<00:24, 2.47it/s] Profiling: 80%|███████▉ | 232/291 [01:04<00:15, 3.79it/s] Profiling: 80%|████████ | 233/291 [01:04<00:15, 3.86it/s] Profiling: 80%|████████ | 234/291 [01:04<00:14, 3.86it/s] Profiling: 81%|████████ | 235/291 [01:04<00:12, 4.47it/s] Profiling: 81%|████████▏ | 237/291 [01:05<00:14, 3.85it/s] Profiling: 82%|████████▏ | 238/291 [01:06<00:18, 2.86it/s] Profiling: 82%|████████▏ | 239/291 [01:06<00:21, 2.40it/s] Profiling: 83%|████████▎ | 241/291 [01:06<00:13, 3.72it/s] Profiling: 83%|████████▎ | 242/291 [01:07<00:12, 3.80it/s] Profiling: 84%|████████▎ | 243/291 [01:07<00:12, 3.88it/s] Profiling: 84%|████████▍ | 244/291 [01:07<00:10, 4.52it/s] Profiling: 84%|████████▍ | 245/291 [01:08<00:14, 3.17it/s] Profiling: 85%|████████▍ | 246/291 [01:08<00:17, 2.50it/s] Profiling: 85%|████████▍ | 247/291 [01:08<00:14, 3.12it/s] Profiling: 85%|████████▌ | 248/291 [01:09<00:12, 3.37it/s] Profiling: 86%|████████▌ | 249/291 [01:09<00:11, 3.58it/s] Profiling: 86%|████████▌ | 250/291 [01:09<00:09, 4.31it/s] Profiling: 86%|████████▋ | 251/291 [01:10<00:22, 1.77it/s] Profiling: 87%|████████▋ | 253/291 [01:11<00:16, 2.25it/s] Profiling: 88%|████████▊ | 256/291 [01:11<00:11, 3.08it/s] Profiling: 88%|████████▊ | 257/291 [01:12<00:12, 2.64it/s] Profiling: 89%|████████▊ | 258/291 [01:13<00:14, 2.32it/s] Profiling: 89%|████████▉ | 260/291 [01:13<00:08, 3.46it/s] Profiling: 90%|████████▉ | 261/291 [01:13<00:08, 3.54it/s] Profiling: 90%|█████████ | 262/291 [01:13<00:07, 3.66it/s] Profiling: 90%|█████████ | 263/291 [01:13<00:06, 4.24it/s] Profiling: 91%|█████████ | 265/291 [01:14<00:06, 3.75it/s] Profiling: 91%|█████████▏| 266/291 [01:15<00:08, 2.87it/s] Profiling: 92%|█████████▏| 267/291 [01:15<00:09, 2.41it/s] Profiling: 92%|█████████▏| 269/291 [01:15<00:05, 3.71it/s] Profiling: 93%|█████████▎| 270/291 [01:16<00:05, 3.71it/s] Profiling: 93%|█████████▎| 271/291 [01:16<00:05, 3.76it/s] Profiling: 93%|█████████▎| 272/291 [01:16<00:04, 4.40it/s] Profiling: 94%|█████████▍| 274/291 [01:17<00:04, 3.84it/s] Profiling: 95%|█████████▍| 275/291 [01:17<00:05, 2.92it/s] Profiling: 95%|█████████▍| 276/291 [01:18<00:06, 2.43it/s] Profiling: 96%|█████████▌| 278/291 [01:18<00:03, 3.77it/s] Profiling: 96%|█████████▌| 279/291 [01:18<00:03, 3.85it/s] Profiling: 96%|█████████▌| 280/291 [01:19<00:02, 3.92it/s] Profiling: 97%|█████████▋| 281/291 [01:19<00:02, 4.53it/s] Profiling: 97%|█████████▋| 283/291 [01:19<00:02, 3.85it/s] Profiling: 98%|█████████▊| 284/291 [01:20<00:02, 2.87it/s] Profiling: 98%|█████████▊| 285/291 [01:21<00:02, 2.41it/s] Profiling: 99%|█████████▊| 287/291 [01:21<00:01, 3.73it/s] Profiling: 99%|█████████▉| 288/291 [01:21<00:00, 3.82it/s] Profiling: 99%|█████████▉| 289/291 [01:21<00:00, 3.90it/s] Profiling: 100%|█████████▉| 290/291 [01:21<00:00, 4.53it/s] Profiling: 100%|██████████| 291/291 [01:21<00:00, 3.56it/s]
anhnv125-elephant-v9-mkmlizer: quantized model in 92.881s
anhnv125-elephant-v9-mkmlizer: Processed model anhnv125/elephant in 147.912s
anhnv125-elephant-v9-mkmlizer: creating bucket guanaco-mkml-models
anhnv125-elephant-v9-mkmlizer: Bucket 's3://guanaco-mkml-models/' created
anhnv125-elephant-v9-mkmlizer: uploading /tmp/model_cache to s3://guanaco-mkml-models/anhnv125-elephant-v9
anhnv125-elephant-v9-mkmlizer: cp /tmp/model_cache/config.json s3://guanaco-mkml-models/anhnv125-elephant-v9/config.json
anhnv125-elephant-v9-mkmlizer: cp /tmp/model_cache/special_tokens_map.json s3://guanaco-mkml-models/anhnv125-elephant-v9/special_tokens_map.json
anhnv125-elephant-v9-mkmlizer: cp /tmp/model_cache/tokenizer_config.json s3://guanaco-mkml-models/anhnv125-elephant-v9/tokenizer_config.json
anhnv125-elephant-v9-mkmlizer: cp /tmp/model_cache/tokenizer.model s3://guanaco-mkml-models/anhnv125-elephant-v9/tokenizer.model
anhnv125-elephant-v9-mkmlizer: cp /tmp/model_cache/tokenizer.json s3://guanaco-mkml-models/anhnv125-elephant-v9/tokenizer.json
anhnv125-elephant-v9-mkmlizer: cp /tmp/model_cache/mkml_model.tensors s3://guanaco-mkml-models/anhnv125-elephant-v9/mkml_model.tensors
anhnv125-elephant-v9-mkmlizer: loading reward model from anhnv125/reward-model-v2
anhnv125-elephant-v9-mkmlizer: /opt/conda/lib/python3.10/site-packages/transformers/models/auto/configuration_auto.py:1067: FutureWarning: The `use_auth_token` argument is deprecated and will be removed in v5 of Transformers. Please use `token` instead.
anhnv125-elephant-v9-mkmlizer: warnings.warn(
anhnv125-elephant-v9-mkmlizer: config.json: 0%| | 0.00/1.04k [00:00<?, ?B/s] config.json: 100%|██████████| 1.04k/1.04k [00:00<00:00, 6.46MB/s]
anhnv125-elephant-v9-mkmlizer: /opt/conda/lib/python3.10/site-packages/transformers/models/auto/tokenization_auto.py:690: FutureWarning: The `use_auth_token` argument is deprecated and will be removed in v5 of Transformers. Please use `token` instead.
anhnv125-elephant-v9-mkmlizer: warnings.warn(
anhnv125-elephant-v9-mkmlizer: tokenizer_config.json: 0%| | 0.00/477 [00:00<?, ?B/s] tokenizer_config.json: 100%|██████████| 477/477 [00:00<00:00, 1.84MB/s]
anhnv125-elephant-v9-mkmlizer: vocab.json: 0%| | 0.00/798k [00:00<?, ?B/s] vocab.json: 100%|██████████| 798k/798k [00:00<00:00, 38.3MB/s]
anhnv125-elephant-v9-mkmlizer: merges.txt: 0%| | 0.00/456k [00:00<?, ?B/s] merges.txt: 100%|██████████| 456k/456k [00:00<00:00, 31.2MB/s]
anhnv125-elephant-v9-mkmlizer: tokenizer.json: 0%| | 0.00/2.11M [00:00<?, ?B/s] tokenizer.json: 100%|██████████| 2.11M/2.11M [00:00<00:00, 87.5MB/s]
anhnv125-elephant-v9-mkmlizer: special_tokens_map.json: 0%| | 0.00/131 [00:00<?, ?B/s] special_tokens_map.json: 100%|██████████| 131/131 [00:00<00:00, 635kB/s]
anhnv125-elephant-v9-mkmlizer: /opt/conda/lib/python3.10/site-packages/transformers/models/auto/auto_factory.py:472: FutureWarning: The `use_auth_token` argument is deprecated and will be removed in v5 of Transformers. Please use `token` instead.
anhnv125-elephant-v9-mkmlizer: warnings.warn(
anhnv125-elephant-v9-mkmlizer: model.safetensors: 0%| | 0.00/498M [00:00<?, ?B/s] model.safetensors: 1%| | 4.95M/498M [00:00<01:30, 5.47MB/s] model.safetensors: 5%|▌ | 25.9M/498M [00:01<00:23, 20.3MB/s] model.safetensors: 9%|▉ | 46.9M/498M [00:01<00:12, 36.8MB/s] model.safetensors: 22%|██▏ | 110M/498M [00:01<00:03, 108MB/s] model.safetensors: 100%|█████████▉| 498M/498M [00:01<00:00, 271MB/s]
anhnv125-elephant-v9-mkmlizer: Saving model to /tmp/reward_cache/reward.tensors
anhnv125-elephant-v9-mkmlizer: Saving duration: 0.115s
anhnv125-elephant-v9-mkmlizer: Processed model anhnv125/reward-model-v2 in 3.966s
anhnv125-elephant-v9-mkmlizer: creating bucket guanaco-reward-models
anhnv125-elephant-v9-mkmlizer: Bucket 's3://guanaco-reward-models/' created
anhnv125-elephant-v9-mkmlizer: uploading /tmp/reward_cache to s3://guanaco-reward-models/anhnv125-elephant-v9_reward
anhnv125-elephant-v9-mkmlizer: cp /tmp/reward_cache/config.json s3://guanaco-reward-models/anhnv125-elephant-v9_reward/config.json
anhnv125-elephant-v9-mkmlizer: cp /tmp/reward_cache/special_tokens_map.json s3://guanaco-reward-models/anhnv125-elephant-v9_reward/special_tokens_map.json
anhnv125-elephant-v9-mkmlizer: cp /tmp/reward_cache/merges.txt s3://guanaco-reward-models/anhnv125-elephant-v9_reward/merges.txt
anhnv125-elephant-v9-mkmlizer: cp /tmp/reward_cache/tokenizer_config.json s3://guanaco-reward-models/anhnv125-elephant-v9_reward/tokenizer_config.json
anhnv125-elephant-v9-mkmlizer: cp /tmp/reward_cache/vocab.json s3://guanaco-reward-models/anhnv125-elephant-v9_reward/vocab.json
anhnv125-elephant-v9-mkmlizer: cp /tmp/reward_cache/tokenizer.json s3://guanaco-reward-models/anhnv125-elephant-v9_reward/tokenizer.json
anhnv125-elephant-v9-mkmlizer: cp /tmp/reward_cache/reward.tensors s3://guanaco-reward-models/anhnv125-elephant-v9_reward/reward.tensors
Job anhnv125-elephant-v9-mkmlizer completed after 183.76s with status: succeeded
Stopping job with name anhnv125-elephant-v9-mkmlizer
Pipeline stage MKMLizer completed in 188.99s
Running pipeline stage MKMLKubeTemplater
Pipeline stage MKMLKubeTemplater completed in 0.16s
Running pipeline stage ISVCDeployer
Creating inference service anhnv125-elephant-v9
Waiting for inference service anhnv125-elephant-v9 to be ready
Inference service anhnv125-elephant-v9 ready after 110.69257926940918s
Pipeline stage ISVCDeployer completed in 118.51s
Running pipeline stage StressChecker
Received healthy response to inference request with status code 200 in 2.412285566329956s
Received healthy response to inference request with status code 200 in 1.507063627243042s
Received healthy response to inference request with status code 200 in 1.768779993057251s
Received healthy response to inference request with status code 200 in 1.517698049545288s
Received healthy response to inference request with status code 200 in 1.5429213047027588s
Received healthy response to inference request with status code 200 in 1.5307378768920898s
Received healthy response to inference request with status code 200 in 1.4785828590393066s
Received healthy response to inference request with status code 200 in 1.5578927993774414s
Received healthy response to inference request with status code 200 in 1.5669686794281006s
Received healthy response to inference request with status code 200 in 1.5750341415405273s
Received healthy response to inference request with status code 200 in 1.5295779705047607s
Received healthy response to inference request with status code 200 in 1.5471081733703613s
Received healthy response to inference request with status code 200 in 1.759537935256958s
Received healthy response to inference request with status code 200 in 1.5317792892456055s
Received healthy response to inference request with status code 200 in 1.5635795593261719s
Received healthy response to inference request with status code 200 in 1.705505132675171s
Received healthy response to inference request with status code 200 in 1.5598161220550537s
Received healthy response to inference request with status code 200 in 1.6276392936706543s
Received healthy response to inference request with status code 200 in 1.5844969749450684s
Received healthy response to inference request with status code 200 in 1.5115690231323242s
Received healthy response to inference request with status code 200 in 1.5657110214233398s
Received healthy response to inference request with status code 200 in 2.023475408554077s
Received healthy response to inference request with status code 200 in 1.562704086303711s
Received healthy response to inference request with status code 200 in 1.5588264465332031s
Received healthy response to inference request with status code 200 in 1.5629982948303223s
Received healthy response to inference request with status code 200 in 1.592177152633667s
Received healthy response to inference request with status code 200 in 1.5938467979431152s
Received healthy response to inference request with status code 200 in 1.7478854656219482s
Received healthy response to inference request with status code 200 in 1.5800063610076904s
Received healthy response to inference request with status code 200 in 1.5826327800750732s
Received healthy response to inference request with status code 200 in 1.6490435600280762s
Received healthy response to inference request with status code 200 in 1.729583501815796s
Received healthy response to inference request with status code 200 in 1.577014446258545s
Received healthy response to inference request with status code 200 in 1.6777796745300293s
Received healthy response to inference request with status code 200 in 1.6304585933685303s
Received healthy response to inference request with status code 200 in 1.5660808086395264s
Received healthy response to inference request with status code 200 in 1.5616588592529297s
Received healthy response to inference request with status code 200 in 1.5355815887451172s
Received healthy response to inference request with status code 200 in 1.5413405895233154s
Received healthy response to inference request with status code 200 in 1.5589230060577393s
Received healthy response to inference request with status code 200 in 1.5405361652374268s
Received healthy response to inference request with status code 200 in 1.528719425201416s
Received healthy response to inference request with status code 200 in 1.5664198398590088s
Received healthy response to inference request with status code 200 in 1.5652742385864258s
Received healthy response to inference request with status code 200 in 1.5417976379394531s
Received healthy response to inference request with status code 200 in 1.5456223487854004s
Received healthy response to inference request with status code 200 in 1.5593359470367432s
Received healthy response to inference request with status code 200 in 1.5662426948547363s
Received healthy response to inference request with status code 200 in 1.6381042003631592s
Received healthy response to inference request with status code 200 in 1.5509271621704102s
Received healthy response to inference request with status code 200 in 1.5821154117584229s
Received healthy response to inference request with status code 200 in 1.5923945903778076s
Received healthy response to inference request with status code 200 in 1.554398536682129s
Received healthy response to inference request with status code 200 in 1.5688502788543701s
Received healthy response to inference request with status code 200 in 1.745450735092163s
Received healthy response to inference request with status code 200 in 1.5880565643310547s
Received healthy response to inference request with status code 200 in 1.5508480072021484s
Received healthy response to inference request with status code 200 in 1.5573503971099854s
Received healthy response to inference request with status code 200 in 1.5440430641174316s
Received healthy response to inference request with status code 200 in 1.552954912185669s
Received healthy response to inference request with status code 200 in 1.5530619621276855s
Received healthy response to inference request with status code 200 in 1.5689237117767334s
Received healthy response to inference request with status code 200 in 1.5574150085449219s
Received healthy response to inference request with status code 200 in 1.6158475875854492s
Received healthy response to inference request with status code 200 in 1.5934820175170898s
Received healthy response to inference request with status code 200 in 1.5392374992370605s
Received healthy response to inference request with status code 200 in 1.5569489002227783s
Received healthy response to inference request with status code 200 in 1.5803463459014893s
Received healthy response to inference request with status code 200 in 1.5515265464782715s
Received healthy response to inference request with status code 200 in 1.5526163578033447s
Received healthy response to inference request with status code 200 in 1.5422475337982178s
Received healthy response to inference request with status code 200 in 1.5583581924438477s
Received healthy response to inference request with status code 200 in 1.6828551292419434s
Received healthy response to inference request with status code 200 in 1.5842673778533936s
Received healthy response to inference request with status code 200 in 1.5648677349090576s
Received healthy response to inference request with status code 200 in 1.682920217514038s
Received healthy response to inference request with status code 200 in 1.5494513511657715s
Received healthy response to inference request with status code 200 in 1.5671052932739258s
Received healthy response to inference request with status code 200 in 1.690244197845459s
Received healthy response to inference request with status code 200 in 1.5804784297943115s
Received healthy response to inference request with status code 200 in 1.5488977432250977s
Received healthy response to inference request with status code 200 in 1.557523250579834s
Received healthy response to inference request with status code 200 in 1.54805588722229s
Received healthy response to inference request with status code 200 in 1.5564424991607666s
Received healthy response to inference request with status code 200 in 1.5558738708496094s
Received healthy response to inference request with status code 200 in 1.5723021030426025s
Received healthy response to inference request with status code 200 in 1.5640461444854736s
Received healthy response to inference request with status code 200 in 1.5761704444885254s
Received healthy response to inference request with status code 200 in 1.5594098567962646s
Received healthy response to inference request with status code 200 in 1.54581618309021s
Received healthy response to inference request with status code 200 in 1.553633451461792s
Received healthy response to inference request with status code 200 in 1.551551103591919s
Received healthy response to inference request with status code 200 in 1.5520515441894531s
Received healthy response to inference request with status code 200 in 1.535111427307129s
Received healthy response to inference request with status code 200 in 1.0816879272460938s
Received healthy response to inference request with status code 200 in 1.5761473178863525s
Received healthy response to inference request with status code 200 in 1.5686771869659424s
Received healthy response to inference request with status code 200 in 1.544193983078003s
Received healthy response to inference request with status code 200 in 1.5548982620239258s
Received healthy response to inference request with status code 200 in 1.3949260711669922s
100 requests
0 failed requests
5th percentile: 1.51739159822464
10th percentile: 1.5347782135009767
20th percentile: 1.5453366756439209
30th percentile: 1.5519014120101928
40th percentile: 1.5571897983551026
50th percentile: 1.5607374906539917
60th percentile: 1.5663135528564451
70th percentile: 1.5764236450195312
80th percentile: 1.5922206401824952
90th percentile: 1.6828616380691528
95th percentile: 1.7455724716186523
99th percentile: 2.027363510131838
mean time: 1.5838536262512206
Pipeline stage StressChecker completed in 166.67s
Running pipeline stage SafetyScorer
Pipeline stage SafetyScorer completed in 37.97s
Running pipeline stage MEvalScorer
Running M-Eval for topic stay_in_character
Pipeline stage MEvalScorer completed in 380.86s
anhnv125-elephant_v9 status is now inactive due to auto deactivation removed underperforming models
anhnv125-elephant_v9 status is now deployed due to admin request
anhnv125-elephant_v9 status is now inactive due to auto deactivation removed underperforming models

Usage Metrics

Latency Metrics