developer_uid: robert_irvine
submission_id: chaoticneutrals-poppy-p_3816_v11
model_name: chaoticneutrals-poppy-p_3816_v11
model_group: ChaoticNeutrals/Poppy_Po
status: torndown
timestamp: 2024-06-10T17:53:39+00:00
num_battles: 37770
num_wins: 18716
celo_rating: 1192.45
family_friendly_score: 0.0
submission_type: basic
model_repo: ChaoticNeutrals/Poppy_Porpoise-0.72-L3-8B
model_architecture: LlamaForCausalLM
reward_repo: rirv938/reward_gpt2_medium_preference_24m_e2
model_num_parameters: 8030261248.0
best_of: 8
max_input_tokens: 512
max_output_tokens: 64
display_name: chaoticneutrals-poppy-p_3816_v11
is_internal_developer: True
language_model: ChaoticNeutrals/Poppy_Porpoise-0.72-L3-8B
model_size: 8B
ranking_group: single
us_pacific_date: 2024-06-10
win_ratio: 0.49552554937781307
generation_params: {'temperature': 0.95, 'top_p': 1.0, 'min_p': 0.0, 'top_k': 40, 'presence_penalty': 0.0, 'frequency_penalty': 0.0, 'stopping_words': ['\n'], 'max_input_tokens': 512, 'best_of': 8, 'max_output_tokens': 64}
formatter: {'memory_template': "{bot_name}'s Persona: {memory}\n####\n", 'prompt_template': '{prompt}\n<START>\n', 'bot_template': '{bot_name}: {message}\n', 'user_template': '{user_name}: {message}\n', 'response_template': '{bot_name}:', 'truncate_by_message': False}
reward_formatter: {'bot_template': 'Bot: {message}\n', 'memory_template': 'Memory: {memory}\n', 'prompt_template': '{prompt}\n', 'response_template': 'Bot:', 'truncate_by_message': False, 'user_template': 'User: {message}\n'}
model_eval_status: pending
Resubmit model
Running pipeline stage MKMLizer
Starting job with name chaoticneutrals-poppy-p-3816-v11-mkmlizer
Waiting for job on chaoticneutrals-poppy-p-3816-v11-mkmlizer to finish
chaoticneutrals-poppy-p-3816-v11-mkmlizer: ╔═════════════════════════════════════════════════════════════════════╗
chaoticneutrals-poppy-p-3816-v11-mkmlizer: ║     _____            __           __                                ║
chaoticneutrals-poppy-p-3816-v11-mkmlizer: ║    / _/ /_ ___    __/ /  ___ ___ / /                                ║
chaoticneutrals-poppy-p-3816-v11-mkmlizer: ║   / _/ / // / |/|/ / _ \/ -_) -_) /                                 ║
chaoticneutrals-poppy-p-3816-v11-mkmlizer: ║  /_//_/\_, /|__,__/_//_/\__/\__/_/                                  ║
chaoticneutrals-poppy-p-3816-v11-mkmlizer: ║       /___/                                                         ║
chaoticneutrals-poppy-p-3816-v11-mkmlizer: ║                                                                     ║
chaoticneutrals-poppy-p-3816-v11-mkmlizer: ║  Version: 0.8.14                                                    ║
chaoticneutrals-poppy-p-3816-v11-mkmlizer: ║  Copyright 2023 MK ONE TECHNOLOGIES Inc.                            ║
chaoticneutrals-poppy-p-3816-v11-mkmlizer: ║  https://mk1.ai                                                     ║
chaoticneutrals-poppy-p-3816-v11-mkmlizer: ║                                                                     ║
chaoticneutrals-poppy-p-3816-v11-mkmlizer: ║  The license key for the current software has been verified as      ║
chaoticneutrals-poppy-p-3816-v11-mkmlizer: ║  belonging to:                                                      ║
chaoticneutrals-poppy-p-3816-v11-mkmlizer: ║                                                                     ║
chaoticneutrals-poppy-p-3816-v11-mkmlizer: ║  Chai Research Corp.                                                ║
chaoticneutrals-poppy-p-3816-v11-mkmlizer: ║  Account ID: 7997a29f-0ceb-4cc7-9adf-840c57b4ae6f                   ║
chaoticneutrals-poppy-p-3816-v11-mkmlizer: ║  Expiration: 2024-07-15 23:59:59                                    ║
chaoticneutrals-poppy-p-3816-v11-mkmlizer: ║                                                                     ║
chaoticneutrals-poppy-p-3816-v11-mkmlizer: ╚═════════════════════════════════════════════════════════════════════╝
chaoticneutrals-poppy-p-3816-v11-mkmlizer: /opt/conda/lib/python3.10/site-packages/huggingface_hub/utils/_deprecation.py:131: FutureWarning: 'list_files_info' (from 'huggingface_hub.hf_api') is deprecated and will be removed from version '0.23'. Use `list_repo_tree` and `get_paths_info` instead.
chaoticneutrals-poppy-p-3816-v11-mkmlizer:   warnings.warn(warning_message, FutureWarning)
chaoticneutrals-poppy-p-3816-v11-mkmlizer: Downloaded to shared memory in 16.628s
chaoticneutrals-poppy-p-3816-v11-mkmlizer: quantizing model to /dev/shm/model_cache
chaoticneutrals-poppy-p-3816-v11-mkmlizer: Saving flywheel model at /dev/shm/model_cache
chaoticneutrals-poppy-p-3816-v11-mkmlizer: 
Loading 0:   0%|          | 0/291 [00:00<?, ?it/s]
Loading 0:   7%|▋         | 20/291 [00:00<00:01, 186.59it/s]
Loading 0:  14%|█▍        | 41/291 [00:00<00:01, 198.84it/s]
Loading 0:  21%|██        | 61/291 [00:00<00:02, 108.68it/s]
Loading 0:  27%|██▋       | 80/291 [00:00<00:02, 94.30it/s] 
Loading 0:  34%|███▍      | 99/291 [00:00<00:01, 113.66it/s]
Loading 0:  42%|████▏     | 123/291 [00:05<00:12, 13.49it/s]
Loading 0:  46%|████▌     | 133/291 [00:09<00:23,  6.73it/s]
Loading 0:  49%|████▉     | 144/291 [00:09<00:17,  8.43it/s]
Loading 0:  57%|█████▋    | 165/291 [00:10<00:09, 13.30it/s]
Loading 0:  63%|██████▎   | 182/291 [00:10<00:06, 17.79it/s]
Loading 0:  70%|██████▉   | 203/291 [00:10<00:03, 26.11it/s]
Loading 0:  76%|███████▌  | 220/291 [00:10<00:02, 32.23it/s]
Loading 0:  83%|████████▎ | 241/291 [00:10<00:01, 44.83it/s]
Loading 0:  90%|█████████ | 263/291 [00:10<00:00, 55.86it/s]
Loading 0:  96%|█████████▌| 279/291 [00:11<00:00, 66.90it/s]
                                                            
Special tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained.
chaoticneutrals-poppy-p-3816-v11-mkmlizer: quantized model in 17.106s
chaoticneutrals-poppy-p-3816-v11-mkmlizer: Processed model ChaoticNeutrals/Poppy_Porpoise-0.72-L3-8B in 34.720s
chaoticneutrals-poppy-p-3816-v11-mkmlizer: creating bucket guanaco-mkml-models
chaoticneutrals-poppy-p-3816-v11-mkmlizer: Bucket 's3://guanaco-mkml-models/' created
chaoticneutrals-poppy-p-3816-v11-mkmlizer: uploading /dev/shm/model_cache to s3://guanaco-mkml-models/chaoticneutrals-poppy-p-3816-v11
chaoticneutrals-poppy-p-3816-v11-mkmlizer: cp /dev/shm/model_cache/config.json s3://guanaco-mkml-models/chaoticneutrals-poppy-p-3816-v11/config.json
chaoticneutrals-poppy-p-3816-v11-mkmlizer: cp /dev/shm/model_cache/special_tokens_map.json s3://guanaco-mkml-models/chaoticneutrals-poppy-p-3816-v11/special_tokens_map.json
chaoticneutrals-poppy-p-3816-v11-mkmlizer: cp /dev/shm/model_cache/tokenizer_config.json s3://guanaco-mkml-models/chaoticneutrals-poppy-p-3816-v11/tokenizer_config.json
chaoticneutrals-poppy-p-3816-v11-mkmlizer: cp /dev/shm/model_cache/tokenizer.json s3://guanaco-mkml-models/chaoticneutrals-poppy-p-3816-v11/tokenizer.json
chaoticneutrals-poppy-p-3816-v11-mkmlizer: cp /dev/shm/model_cache/flywheel_model.0.safetensors s3://guanaco-mkml-models/chaoticneutrals-poppy-p-3816-v11/flywheel_model.0.safetensors
chaoticneutrals-poppy-p-3816-v11-mkmlizer: loading reward model from rirv938/reward_gpt2_medium_preference_24m_e2
chaoticneutrals-poppy-p-3816-v11-mkmlizer: /opt/conda/lib/python3.10/site-packages/transformers/models/auto/configuration_auto.py:913: FutureWarning: The `use_auth_token` argument is deprecated and will be removed in v5 of Transformers. Please use `token` instead.
chaoticneutrals-poppy-p-3816-v11-mkmlizer:   warnings.warn(
chaoticneutrals-poppy-p-3816-v11-mkmlizer: /opt/conda/lib/python3.10/site-packages/transformers/models/auto/tokenization_auto.py:757: FutureWarning: The `use_auth_token` argument is deprecated and will be removed in v5 of Transformers. Please use `token` instead.
chaoticneutrals-poppy-p-3816-v11-mkmlizer:   warnings.warn(
chaoticneutrals-poppy-p-3816-v11-mkmlizer: /opt/conda/lib/python3.10/site-packages/transformers/models/auto/auto_factory.py:468: FutureWarning: The `use_auth_token` argument is deprecated and will be removed in v5 of Transformers. Please use `token` instead.
chaoticneutrals-poppy-p-3816-v11-mkmlizer:   warnings.warn(
chaoticneutrals-poppy-p-3816-v11-mkmlizer: cp /tmp/reward_cache/reward.tensors s3://guanaco-reward-models/chaoticneutrals-poppy-p-3816-v11_reward/reward.tensors
Job chaoticneutrals-poppy-p-3816-v11-mkmlizer completed after 57.44s with status: succeeded
Stopping job with name chaoticneutrals-poppy-p-3816-v11-mkmlizer
Pipeline stage MKMLizer completed in 58.88s
Running pipeline stage MKMLKubeTemplater
Pipeline stage MKMLKubeTemplater completed in 0.30s
Running pipeline stage ISVCDeployer
Creating inference service chaoticneutrals-poppy-p-3816-v11
Waiting for inference service chaoticneutrals-poppy-p-3816-v11 to be ready
Inference service chaoticneutrals-poppy-p-3816-v11 ready after 40.58027911186218s
Pipeline stage ISVCDeployer completed in 47.31s
Running pipeline stage StressChecker
Received healthy response to inference request in 2.060555934906006s
Received healthy response to inference request in 1.2635698318481445s
Received healthy response to inference request in 1.17335844039917s
Received healthy response to inference request in 1.2146403789520264s
Received healthy response to inference request in 1.2142226696014404s
5 requests
0 failed requests
5th percentile: 1.1815312862396241
10th percentile: 1.189704132080078
20th percentile: 1.2060498237609862
30th percentile: 1.2143062114715577
40th percentile: 1.214473295211792
50th percentile: 1.2146403789520264
60th percentile: 1.2342121601104736
70th percentile: 1.253783941268921
80th percentile: 1.4229670524597169
90th percentile: 1.7417614936828614
95th percentile: 1.9011587142944335
99th percentile: 2.0286764907836914
mean time: 1.3852694511413575
Pipeline stage StressChecker completed in 9.11s
Running pipeline stage DaemonicModelEvalScorer
Pipeline stage DaemonicModelEvalScorer completed in 0.11s
Running M-Eval for topic stay_in_character
Running pipeline stage DaemonicSafetyScorer
M-Eval Dataset for topic stay_in_character is loaded
Pipeline stage DaemonicSafetyScorer completed in 0.24s
%s, retrying in %s seconds...
chaoticneutrals-poppy-p_3816_v11 status is now deployed due to DeploymentManager action
%s, retrying in %s seconds...
chaoticneutrals-poppy-p_3816_v11 status is now inactive due to auto deactivation removed underperforming models
admin requested tearing down of chaoticneutrals-poppy-p_3816_v11
Running pipeline stage ISVCDeleter
Checking if service chaoticneutrals-poppy-p-3816-v11 is running
Skipping teardown as no inference service was found
Pipeline stage ISVCDeleter completed in 2.52s
Running pipeline stage MKMLModelDeleter
Cleaning model data from S3
Cleaning model data from model cache
Deleting key chaoticneutrals-poppy-p-3816-v11/config.json from bucket guanaco-mkml-models
Deleting key chaoticneutrals-poppy-p-3816-v11/flywheel_model.0.safetensors from bucket guanaco-mkml-models
Deleting key chaoticneutrals-poppy-p-3816-v11/special_tokens_map.json from bucket guanaco-mkml-models
Deleting key chaoticneutrals-poppy-p-3816-v11/tokenizer.json from bucket guanaco-mkml-models
Deleting key chaoticneutrals-poppy-p-3816-v11/tokenizer_config.json from bucket guanaco-mkml-models
Cleaning model data from model cache
Deleting key chaoticneutrals-poppy-p-3816-v11_reward/config.json from bucket guanaco-reward-models
Deleting key chaoticneutrals-poppy-p-3816-v11_reward/merges.txt from bucket guanaco-reward-models
Deleting key chaoticneutrals-poppy-p-3816-v11_reward/reward.tensors from bucket guanaco-reward-models
Deleting key chaoticneutrals-poppy-p-3816-v11_reward/special_tokens_map.json from bucket guanaco-reward-models
Deleting key chaoticneutrals-poppy-p-3816-v11_reward/tokenizer.json from bucket guanaco-reward-models
Deleting key chaoticneutrals-poppy-p-3816-v11_reward/tokenizer_config.json from bucket guanaco-reward-models
Deleting key chaoticneutrals-poppy-p-3816-v11_reward/vocab.json from bucket guanaco-reward-models
Pipeline stage MKMLModelDeleter completed in 3.17s
chaoticneutrals-poppy-p_3816_v11 status is now torndown due to DeploymentManager action