submission_id: chaoticneutrals-poppy-p_3816_v11
developer_uid: robert_irvine
status: inactive
model_repo: ChaoticNeutrals/Poppy_Porpoise-0.72-L3-8B
reward_repo: rirv938/reward_gpt2_medium_preference_24m_e2
generation_params: {'temperature': 0.95, 'top_p': 1.0, 'min_p': 0.0, 'top_k': 40, 'presence_penalty': 0.0, 'frequency_penalty': 0.0, 'stopping_words': ['\n'], 'max_input_tokens': 512, 'best_of': 8, 'max_output_tokens': 64}
formatter: {'memory_template': "{bot_name}'s Persona: {memory}\n####\n", 'prompt_template': '{prompt}\n<START>\n', 'bot_template': '{bot_name}: {message}\n', 'user_template': '{user_name}: {message}\n', 'response_template': '{bot_name}:', 'truncate_by_message': False}
reward_formatter: {'memory_template': 'Memory: {memory}\n', 'prompt_template': '{prompt}\n', 'bot_template': 'Bot: {message}\n', 'user_template': 'User: {message}\n', 'response_template': 'Bot:', 'truncate_by_message': False}
timestamp: 2024-06-10T17:53:39+00:00
model_name: chaoticneutrals-poppy-p_3816_v11
model_eval_status: pending
model_group: ChaoticNeutrals/Poppy_Po
num_battles: 37770
num_wins: 18716
celo_rating: 1192.45
propriety_score: 0.7175774697625144
propriety_total_count: 11327.0
submission_type: basic
model_architecture: LlamaForCausalLM
model_num_parameters: 8030261248.0
best_of: 8
max_input_tokens: 512
max_output_tokens: 64
display_name: chaoticneutrals-poppy-p_3816_v11
ineligible_reason: None
language_model: ChaoticNeutrals/Poppy_Porpoise-0.72-L3-8B
model_size: 8B
reward_model: rirv938/reward_gpt2_medium_preference_24m_e2
us_pacific_date: 2024-06-10
win_ratio: 0.49552554937781307
Resubmit model
Running pipeline stage MKMLizer
Starting job with name chaoticneutrals-poppy-p-3816-v11-mkmlizer
Waiting for job on chaoticneutrals-poppy-p-3816-v11-mkmlizer to finish
chaoticneutrals-poppy-p-3816-v11-mkmlizer: ╔═════════════════════════════════════════════════════════════════════╗
chaoticneutrals-poppy-p-3816-v11-mkmlizer: ║ _____ __ __ ║
chaoticneutrals-poppy-p-3816-v11-mkmlizer: ║ / _/ /_ ___ __/ / ___ ___ / / ║
chaoticneutrals-poppy-p-3816-v11-mkmlizer: ║ / _/ / // / |/|/ / _ \/ -_) -_) / ║
chaoticneutrals-poppy-p-3816-v11-mkmlizer: ║ /_//_/\_, /|__,__/_//_/\__/\__/_/ ║
chaoticneutrals-poppy-p-3816-v11-mkmlizer: ║ /___/ ║
chaoticneutrals-poppy-p-3816-v11-mkmlizer: ║ ║
chaoticneutrals-poppy-p-3816-v11-mkmlizer: ║ Version: 0.8.14 ║
chaoticneutrals-poppy-p-3816-v11-mkmlizer: ║ Copyright 2023 MK ONE TECHNOLOGIES Inc. ║
chaoticneutrals-poppy-p-3816-v11-mkmlizer: ║ https://mk1.ai ║
chaoticneutrals-poppy-p-3816-v11-mkmlizer: ║ ║
chaoticneutrals-poppy-p-3816-v11-mkmlizer: ║ The license key for the current software has been verified as ║
chaoticneutrals-poppy-p-3816-v11-mkmlizer: ║ belonging to: ║
chaoticneutrals-poppy-p-3816-v11-mkmlizer: ║ ║
chaoticneutrals-poppy-p-3816-v11-mkmlizer: ║ Chai Research Corp. ║
chaoticneutrals-poppy-p-3816-v11-mkmlizer: ║ Account ID: 7997a29f-0ceb-4cc7-9adf-840c57b4ae6f ║
chaoticneutrals-poppy-p-3816-v11-mkmlizer: ║ Expiration: 2024-07-15 23:59:59 ║
chaoticneutrals-poppy-p-3816-v11-mkmlizer: ║ ║
chaoticneutrals-poppy-p-3816-v11-mkmlizer: ╚═════════════════════════════════════════════════════════════════════╝
chaoticneutrals-poppy-p-3816-v11-mkmlizer: /opt/conda/lib/python3.10/site-packages/huggingface_hub/utils/_deprecation.py:131: FutureWarning: 'list_files_info' (from 'huggingface_hub.hf_api') is deprecated and will be removed from version '0.23'. Use `list_repo_tree` and `get_paths_info` instead.
chaoticneutrals-poppy-p-3816-v11-mkmlizer: warnings.warn(warning_message, FutureWarning)
chaoticneutrals-poppy-p-3816-v11-mkmlizer: Downloaded to shared memory in 16.628s
chaoticneutrals-poppy-p-3816-v11-mkmlizer: quantizing model to /dev/shm/model_cache
chaoticneutrals-poppy-p-3816-v11-mkmlizer: Saving flywheel model at /dev/shm/model_cache
chaoticneutrals-poppy-p-3816-v11-mkmlizer: Loading 0: 0%| | 0/291 [00:00<?, ?it/s] Loading 0: 7%|▋ | 20/291 [00:00<00:01, 186.59it/s] Loading 0: 14%|█▍ | 41/291 [00:00<00:01, 198.84it/s] Loading 0: 21%|██ | 61/291 [00:00<00:02, 108.68it/s] Loading 0: 27%|██▋ | 80/291 [00:00<00:02, 94.30it/s] Loading 0: 34%|███▍ | 99/291 [00:00<00:01, 113.66it/s] Loading 0: 42%|████▏ | 123/291 [00:05<00:12, 13.49it/s] Loading 0: 46%|████▌ | 133/291 [00:09<00:23, 6.73it/s] Loading 0: 49%|████▉ | 144/291 [00:09<00:17, 8.43it/s] Loading 0: 57%|█████▋ | 165/291 [00:10<00:09, 13.30it/s] Loading 0: 63%|██████▎ | 182/291 [00:10<00:06, 17.79it/s] Loading 0: 70%|██████▉ | 203/291 [00:10<00:03, 26.11it/s] Loading 0: 76%|███████▌ | 220/291 [00:10<00:02, 32.23it/s] Loading 0: 83%|████████▎ | 241/291 [00:10<00:01, 44.83it/s] Loading 0: 90%|█████████ | 263/291 [00:10<00:00, 55.86it/s] Loading 0: 96%|█████████▌| 279/291 [00:11<00:00, 66.90it/s] Special tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained.
chaoticneutrals-poppy-p-3816-v11-mkmlizer: quantized model in 17.106s
chaoticneutrals-poppy-p-3816-v11-mkmlizer: Processed model ChaoticNeutrals/Poppy_Porpoise-0.72-L3-8B in 34.720s
chaoticneutrals-poppy-p-3816-v11-mkmlizer: creating bucket guanaco-mkml-models
chaoticneutrals-poppy-p-3816-v11-mkmlizer: Bucket 's3://guanaco-mkml-models/' created
chaoticneutrals-poppy-p-3816-v11-mkmlizer: uploading /dev/shm/model_cache to s3://guanaco-mkml-models/chaoticneutrals-poppy-p-3816-v11
chaoticneutrals-poppy-p-3816-v11-mkmlizer: cp /dev/shm/model_cache/config.json s3://guanaco-mkml-models/chaoticneutrals-poppy-p-3816-v11/config.json
chaoticneutrals-poppy-p-3816-v11-mkmlizer: cp /dev/shm/model_cache/special_tokens_map.json s3://guanaco-mkml-models/chaoticneutrals-poppy-p-3816-v11/special_tokens_map.json
chaoticneutrals-poppy-p-3816-v11-mkmlizer: cp /dev/shm/model_cache/tokenizer_config.json s3://guanaco-mkml-models/chaoticneutrals-poppy-p-3816-v11/tokenizer_config.json
chaoticneutrals-poppy-p-3816-v11-mkmlizer: cp /dev/shm/model_cache/tokenizer.json s3://guanaco-mkml-models/chaoticneutrals-poppy-p-3816-v11/tokenizer.json
chaoticneutrals-poppy-p-3816-v11-mkmlizer: cp /dev/shm/model_cache/flywheel_model.0.safetensors s3://guanaco-mkml-models/chaoticneutrals-poppy-p-3816-v11/flywheel_model.0.safetensors
chaoticneutrals-poppy-p-3816-v11-mkmlizer: loading reward model from rirv938/reward_gpt2_medium_preference_24m_e2
chaoticneutrals-poppy-p-3816-v11-mkmlizer: /opt/conda/lib/python3.10/site-packages/transformers/models/auto/configuration_auto.py:913: FutureWarning: The `use_auth_token` argument is deprecated and will be removed in v5 of Transformers. Please use `token` instead.
chaoticneutrals-poppy-p-3816-v11-mkmlizer: warnings.warn(
chaoticneutrals-poppy-p-3816-v11-mkmlizer: /opt/conda/lib/python3.10/site-packages/transformers/models/auto/tokenization_auto.py:757: FutureWarning: The `use_auth_token` argument is deprecated and will be removed in v5 of Transformers. Please use `token` instead.
chaoticneutrals-poppy-p-3816-v11-mkmlizer: warnings.warn(
chaoticneutrals-poppy-p-3816-v11-mkmlizer: /opt/conda/lib/python3.10/site-packages/transformers/models/auto/auto_factory.py:468: FutureWarning: The `use_auth_token` argument is deprecated and will be removed in v5 of Transformers. Please use `token` instead.
chaoticneutrals-poppy-p-3816-v11-mkmlizer: warnings.warn(
chaoticneutrals-poppy-p-3816-v11-mkmlizer: cp /tmp/reward_cache/reward.tensors s3://guanaco-reward-models/chaoticneutrals-poppy-p-3816-v11_reward/reward.tensors
Job chaoticneutrals-poppy-p-3816-v11-mkmlizer completed after 57.44s with status: succeeded
Stopping job with name chaoticneutrals-poppy-p-3816-v11-mkmlizer
Pipeline stage MKMLizer completed in 58.88s
Running pipeline stage MKMLKubeTemplater
Pipeline stage MKMLKubeTemplater completed in 0.30s
Running pipeline stage ISVCDeployer
Creating inference service chaoticneutrals-poppy-p-3816-v11
Waiting for inference service chaoticneutrals-poppy-p-3816-v11 to be ready
Inference service chaoticneutrals-poppy-p-3816-v11 ready after 40.58027911186218s
Pipeline stage ISVCDeployer completed in 47.31s
Running pipeline stage StressChecker
Received healthy response to inference request in 2.060555934906006s
Received healthy response to inference request in 1.2635698318481445s
Received healthy response to inference request in 1.17335844039917s
Received healthy response to inference request in 1.2146403789520264s
Received healthy response to inference request in 1.2142226696014404s
5 requests
0 failed requests
5th percentile: 1.1815312862396241
10th percentile: 1.189704132080078
20th percentile: 1.2060498237609862
30th percentile: 1.2143062114715577
40th percentile: 1.214473295211792
50th percentile: 1.2146403789520264
60th percentile: 1.2342121601104736
70th percentile: 1.253783941268921
80th percentile: 1.4229670524597169
90th percentile: 1.7417614936828614
95th percentile: 1.9011587142944335
99th percentile: 2.0286764907836914
mean time: 1.3852694511413575
Pipeline stage StressChecker completed in 9.11s
Running pipeline stage DaemonicModelEvalScorer
Pipeline stage DaemonicModelEvalScorer completed in 0.11s
Running M-Eval for topic stay_in_character
Running pipeline stage DaemonicSafetyScorer
M-Eval Dataset for topic stay_in_character is loaded
Pipeline stage DaemonicSafetyScorer completed in 0.24s
%s, retrying in %s seconds...
chaoticneutrals-poppy-p_3816_v11 status is now deployed due to DeploymentManager action
%s, retrying in %s seconds...
chaoticneutrals-poppy-p_3816_v11 status is now inactive due to auto deactivation removed underperforming models

Usage Metrics

Latency Metrics