chaoticneutrals-poppy-po_3816

developer_uid: robert_irvine

submission_id: chaoticneutrals-poppy-po_3816_v5

model_name: chaoticneutrals-poppy-po_3816_v5

model_group: ChaoticNeutrals/Poppy_Po

status: torndown

timestamp: 2024-06-10T04:39:45+00:00

num_battles: 22664

num_wins: 11812

celo_rating: 1196.72

family_friendly_score: 0.0

submission_type: basic

model_repo: ChaoticNeutrals/Poppy_Porpoise-0.72-L3-8B

model_architecture: LlamaForCausalLM

reward_repo: rirv938/gpt2_medium_pairwise_60m_step_468750

model_num_parameters: 8030261248.0

best_of: 8

max_input_tokens: 512

max_output_tokens: 64

display_name: chaoticneutrals-poppy-po_3816_v5

is_internal_developer: True

language_model: ChaoticNeutrals/Poppy_Porpoise-0.72-L3-8B

model_size: 8B

ranking_group: single

us_pacific_date: 2024-06-09

win_ratio: 0.5211789622308507

generation_params: {'temperature': 0.95, 'top_p': 1.0, 'min_p': 0.0, 'top_k': 40, 'presence_penalty': 0.0, 'frequency_penalty': 0.0, 'stopping_words': ['\n'], 'max_input_tokens': 512, 'best_of': 8, 'max_output_tokens': 64}

formatter: {'memory_template': "{bot_name}'s Persona: {memory}\n####\n", 'prompt_template': '{prompt}\n<START>\n', 'bot_template': '{bot_name}: {message}\n', 'user_template': '{user_name}: {message}\n', 'response_template': '{bot_name}:', 'truncate_by_message': False}

reward_formatter: {'bot_template': 'Bot: {message}\n', 'memory_template': 'Memory: {memory}\n', 'prompt_template': '{prompt}\n', 'response_template': 'Bot:', 'truncate_by_message': False, 'user_template': 'User: {message}\n'}

model_eval_status: success

Resubmit model

Running pipeline stage MKMLizer
Starting job with name chaoticneutrals-poppy-po-3816-v5-mkmlizer
Waiting for job on chaoticneutrals-poppy-po-3816-v5-mkmlizer to finish
chaoticneutrals-poppy-po-3816-v5-mkmlizer: ╔═════════════════════════════════════════════════════════════════════╗
chaoticneutrals-poppy-po-3816-v5-mkmlizer: ║     _____            __           __                                ║
chaoticneutrals-poppy-po-3816-v5-mkmlizer: ║    / _/ /_ ___    __/ /  ___ ___ / /                                ║
chaoticneutrals-poppy-po-3816-v5-mkmlizer: ║   / _/ / // / |/|/ / _ \/ -_) -_) /                                 ║
chaoticneutrals-poppy-po-3816-v5-mkmlizer: ║  /_//_/\_, /|__,__/_//_/\__/\__/_/                                  ║
chaoticneutrals-poppy-po-3816-v5-mkmlizer: ║       /___/                                                         ║
chaoticneutrals-poppy-po-3816-v5-mkmlizer: ║                                                                     ║
chaoticneutrals-poppy-po-3816-v5-mkmlizer: ║  Version: 0.8.14                                                    ║
chaoticneutrals-poppy-po-3816-v5-mkmlizer: ║  Copyright 2023 MK ONE TECHNOLOGIES Inc.                            ║
chaoticneutrals-poppy-po-3816-v5-mkmlizer: ║  https://mk1.ai                                                     ║
chaoticneutrals-poppy-po-3816-v5-mkmlizer: ║                                                                     ║
chaoticneutrals-poppy-po-3816-v5-mkmlizer: ║  The license key for the current software has been verified as      ║
chaoticneutrals-poppy-po-3816-v5-mkmlizer: ║  belonging to:                                                      ║
chaoticneutrals-poppy-po-3816-v5-mkmlizer: ║                                                                     ║
chaoticneutrals-poppy-po-3816-v5-mkmlizer: ║  Chai Research Corp.                                                ║
chaoticneutrals-poppy-po-3816-v5-mkmlizer: ║  Account ID: 7997a29f-0ceb-4cc7-9adf-840c57b4ae6f                   ║
chaoticneutrals-poppy-po-3816-v5-mkmlizer: ║  Expiration: 2024-07-15 23:59:59                                    ║
chaoticneutrals-poppy-po-3816-v5-mkmlizer: ║                                                                     ║
chaoticneutrals-poppy-po-3816-v5-mkmlizer: ╚═════════════════════════════════════════════════════════════════════╝
chaoticneutrals-poppy-po-3816-v5-mkmlizer: /opt/conda/lib/python3.10/site-packages/huggingface_hub/utils/_deprecation.py:131: FutureWarning: 'list_files_info' (from 'huggingface_hub.hf_api') is deprecated and will be removed from version '0.23'. Use `list_repo_tree` and `get_paths_info` instead.
chaoticneutrals-poppy-po-3816-v5-mkmlizer:   warnings.warn(warning_message, FutureWarning)
chaoticneutrals-poppy-po-3816-v5-mkmlizer: Downloaded to shared memory in 137.792s
chaoticneutrals-poppy-po-3816-v5-mkmlizer: quantizing model to /dev/shm/model_cache
chaoticneutrals-poppy-po-3816-v5-mkmlizer: Saving flywheel model at /dev/shm/model_cache
chaoticneutrals-poppy-po-3816-v5-mkmlizer: 
Loading 0:   0%|          | 0/291 [00:00<?, ?it/s]
Loading 0:   2%|▏         | 7/291 [00:00<00:04, 67.60it/s]
Loading 0:   7%|▋         | 20/291 [00:00<00:02, 93.14it/s]
Loading 0:  10%|█         | 30/291 [00:00<00:03, 86.20it/s]
Loading 0:  14%|█▍        | 42/291 [00:00<00:04, 58.93it/s]
Loading 0:  17%|█▋        | 50/291 [00:00<00:03, 63.73it/s]
Loading 0:  22%|██▏       | 63/291 [00:00<00:02, 77.05it/s]
Loading 0:  25%|██▌       | 73/291 [00:00<00:02, 81.17it/s]
Loading 0:  28%|██▊       | 82/291 [00:01<00:03, 59.39it/s]
Loading 0:  32%|███▏      | 92/291 [00:01<00:03, 66.31it/s]
Loading 0:  36%|███▌      | 105/291 [00:01<00:02, 80.66it/s]
Loading 0:  40%|████      | 117/291 [00:01<00:02, 85.45it/s]
Loading 0:  42%|████▏     | 123/291 [00:12<00:01, 85.45it/s]
Loading 0:  43%|████▎     | 124/291 [00:12<00:58,  2.85it/s]
Loading 0:  45%|████▌     | 132/291 [00:12<00:41,  3.80it/s]
Loading 0:  49%|████▉     | 144/291 [00:13<00:26,  5.63it/s]
Loading 0:  53%|█████▎    | 153/291 [00:13<00:18,  7.59it/s]
Loading 0:  57%|█████▋    | 165/291 [00:13<00:11, 11.18it/s]
Loading 0:  60%|██████    | 175/291 [00:13<00:07, 15.03it/s]
Loading 0:  63%|██████▎   | 184/291 [00:13<00:06, 17.64it/s]
Loading 0:  67%|██████▋   | 194/291 [00:13<00:04, 23.31it/s]
Loading 0:  70%|███████   | 204/291 [00:13<00:02, 30.01it/s]
Loading 0:  75%|███████▍  | 218/291 [00:14<00:01, 41.73it/s]
Loading 0:  78%|███████▊  | 228/291 [00:14<00:01, 38.81it/s]
Loading 0:  83%|████████▎ | 241/291 [00:14<00:01, 49.63it/s]
Loading 0:  87%|████████▋ | 254/291 [00:14<00:00, 62.04it/s]
Loading 0:  91%|█████████ | 264/291 [00:14<00:00, 57.58it/s]
Loading 0:  94%|█████████▍| 274/291 [00:14<00:00, 64.46it/s]
Loading 0:  98%|█████████▊| 284/291 [00:15<00:00, 71.38it/s]
                                                            
Special tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained.
chaoticneutrals-poppy-po-3816-v5-mkmlizer: quantized model in 26.611s
chaoticneutrals-poppy-po-3816-v5-mkmlizer: Processed model ChaoticNeutrals/Poppy_Porpoise-0.72-L3-8B in 167.338s
chaoticneutrals-poppy-po-3816-v5-mkmlizer: creating bucket guanaco-mkml-models
chaoticneutrals-poppy-po-3816-v5-mkmlizer: Bucket 's3://guanaco-mkml-models/' created
chaoticneutrals-poppy-po-3816-v5-mkmlizer: uploading /dev/shm/model_cache to s3://guanaco-mkml-models/chaoticneutrals-poppy-po-3816-v5
chaoticneutrals-poppy-po-3816-v5-mkmlizer: DEBUG retryable error: RequestError: send request failed
chaoticneutrals-poppy-po-3816-v5-mkmlizer: caused by: dial tcp 216.153.53.63:443: i/o timeout
chaoticneutrals-poppy-po-3816-v5-mkmlizer: cp /dev/shm/model_cache/special_tokens_map.json s3://guanaco-mkml-models/chaoticneutrals-poppy-po-3816-v5/special_tokens_map.json
chaoticneutrals-poppy-po-3816-v5-mkmlizer: cp /dev/shm/model_cache/tokenizer_config.json s3://guanaco-mkml-models/chaoticneutrals-poppy-po-3816-v5/tokenizer_config.json
chaoticneutrals-poppy-po-3816-v5-mkmlizer: cp /dev/shm/model_cache/config.json s3://guanaco-mkml-models/chaoticneutrals-poppy-po-3816-v5/config.json
chaoticneutrals-poppy-po-3816-v5-mkmlizer: cp /dev/shm/model_cache/tokenizer.json s3://guanaco-mkml-models/chaoticneutrals-poppy-po-3816-v5/tokenizer.json
chaoticneutrals-poppy-po-3816-v5-mkmlizer: loading reward model from rirv938/gpt2_medium_pairwise_60m_step_468750
chaoticneutrals-poppy-po-3816-v5-mkmlizer: /opt/conda/lib/python3.10/site-packages/transformers/models/auto/configuration_auto.py:913: FutureWarning: The `use_auth_token` argument is deprecated and will be removed in v5 of Transformers. Please use `token` instead.
chaoticneutrals-poppy-po-3816-v5-mkmlizer:   warnings.warn(
chaoticneutrals-poppy-po-3816-v5-mkmlizer: /opt/conda/lib/python3.10/site-packages/transformers/models/auto/tokenization_auto.py:757: FutureWarning: The `use_auth_token` argument is deprecated and will be removed in v5 of Transformers. Please use `token` instead.
chaoticneutrals-poppy-po-3816-v5-mkmlizer:   warnings.warn(
chaoticneutrals-poppy-po-3816-v5-mkmlizer: /opt/conda/lib/python3.10/site-packages/transformers/models/auto/auto_factory.py:468: FutureWarning: The `use_auth_token` argument is deprecated and will be removed in v5 of Transformers. Please use `token` instead.
chaoticneutrals-poppy-po-3816-v5-mkmlizer:   warnings.warn(
chaoticneutrals-poppy-po-3816-v5-mkmlizer: Saving model to /tmp/reward_cache/reward.tensors
chaoticneutrals-poppy-po-3816-v5-mkmlizer: Saving duration: 0.468s
chaoticneutrals-poppy-po-3816-v5-mkmlizer: Processed model rirv938/gpt2_medium_pairwise_60m_step_468750 in 12.597s
chaoticneutrals-poppy-po-3816-v5-mkmlizer: creating bucket guanaco-reward-models
chaoticneutrals-poppy-po-3816-v5-mkmlizer: WARNING: Retrying failed request: /?location ([Errno 110] Connection timed out)
chaoticneutrals-poppy-po-3816-v5-mkmlizer: WARNING: Waiting 3 sec...
chaoticneutrals-poppy-po-3816-v5-mkmlizer: cp /tmp/reward_cache/reward.tensors s3://guanaco-reward-models/chaoticneutrals-poppy-po-3816-v5_reward/reward.tensors
Job chaoticneutrals-poppy-po-3816-v5-mkmlizer completed after 369.1s with status: succeeded
Stopping job with name chaoticneutrals-poppy-po-3816-v5-mkmlizer
Pipeline stage MKMLizer completed in 371.66s
Running pipeline stage MKMLKubeTemplater
Pipeline stage MKMLKubeTemplater completed in 0.09s
Running pipeline stage ISVCDeployer
Creating inference service chaoticneutrals-poppy-po-3816-v5
Waiting for inference service chaoticneutrals-poppy-po-3816-v5 to be ready
Inference service chaoticneutrals-poppy-po-3816-v5 ready after 311.58355689048767s
Pipeline stage ISVCDeployer completed in 318.54s
Running pipeline stage StressChecker
Received healthy response to inference request in 1.9798502922058105s
%s, retrying in %s seconds...
Received healthy response to inference request in 1.2226605415344238s
Received healthy response to inference request in 1.1819806098937988s
Received healthy response to inference request in 1.171074628829956s
Received healthy response to inference request in 1.173403024673462s
Received healthy response to inference request in 1.184279203414917s
5 requests
0 failed requests
5th percentile: 1.1715403079986573
10th percentile: 1.1720059871673585
20th percentile: 1.1729373455047607
30th percentile: 1.1751185417175294
40th percentile: 1.178549575805664
50th percentile: 1.1819806098937988
60th percentile: 1.1829000473022462
70th percentile: 1.1838194847106933
80th percentile: 1.1919554710388183
90th percentile: 1.207308006286621
95th percentile: 1.2149842739105225
99th percentile: 1.2211252880096435
mean time: 1.1866796016693115
Pipeline stage StressChecker completed in 28.61s
Running pipeline stage DaemonicModelEvalScorer
Pipeline stage DaemonicModelEvalScorer completed in 0.03s
Running pipeline stage DaemonicSafetyScorer
Running M-Eval for topic stay_in_character
Pipeline stage DaemonicSafetyScorer completed in 0.04s
M-Eval Dataset for topic stay_in_character is loaded
chaoticneutrals-poppy-po_3816_v5 status is now deployed due to DeploymentManager action
chaoticneutrals-poppy-po_3816_v5 status is now inactive due to auto deactivation removed underperforming models
admin requested tearing down of chaoticneutrals-poppy-po_3816_v5
Running pipeline stage ISVCDeleter
Checking if service chaoticneutrals-poppy-po-3816-v5 is running
Skipping teardown as no inference service was found
Pipeline stage ISVCDeleter completed in 2.45s
Running pipeline stage MKMLModelDeleter
Cleaning model data from S3
Cleaning model data from model cache
Deleting key chaoticneutrals-poppy-po-3816-v5/config.json from bucket guanaco-mkml-models
Deleting key chaoticneutrals-poppy-po-3816-v5/flywheel_model.0.safetensors from bucket guanaco-mkml-models
Deleting key chaoticneutrals-poppy-po-3816-v5/special_tokens_map.json from bucket guanaco-mkml-models
Deleting key chaoticneutrals-poppy-po-3816-v5/tokenizer.json from bucket guanaco-mkml-models
Deleting key chaoticneutrals-poppy-po-3816-v5/tokenizer_config.json from bucket guanaco-mkml-models
Cleaning model data from model cache
Deleting key chaoticneutrals-poppy-po-3816-v5_reward/config.json from bucket guanaco-reward-models
Deleting key chaoticneutrals-poppy-po-3816-v5_reward/merges.txt from bucket guanaco-reward-models
Deleting key chaoticneutrals-poppy-po-3816-v5_reward/reward.tensors from bucket guanaco-reward-models
Deleting key chaoticneutrals-poppy-po-3816-v5_reward/special_tokens_map.json from bucket guanaco-reward-models
Deleting key chaoticneutrals-poppy-po-3816-v5_reward/tokenizer.json from bucket guanaco-reward-models
Deleting key chaoticneutrals-poppy-po-3816-v5_reward/tokenizer_config.json from bucket guanaco-reward-models
Deleting key chaoticneutrals-poppy-po-3816-v5_reward/vocab.json from bucket guanaco-reward-models
Pipeline stage MKMLModelDeleter completed in 3.50s
chaoticneutrals-poppy-po_3816_v5 status is now torndown due to DeploymentManager action