developer_uid: Locutusque
submission_id: locutusque-apollo-0-4-ll_7459_v2
model_name: locutusque-apollo-0-4-ll_7459_v2
model_group: Locutusque/Apollo-0.4-Ll
status: torndown
timestamp: 2024-08-08T18:16:36+00:00
num_battles: 17394
num_wins: 7683
celo_rating: 1178.09
family_friendly_score: 0.0
submission_type: basic
model_repo: Locutusque/Apollo-0.4-Llama-3.1-8B
model_architecture: LlamaForCausalLM
reward_repo: ChaiML/gpt2_xl_pairwise_89m_step_347634
model_num_parameters: 8030261248.0
best_of: 4
max_input_tokens: 1024
max_output_tokens: 64
display_name: locutusque-apollo-0-4-ll_7459_v2
is_internal_developer: False
language_model: Locutusque/Apollo-0.4-Llama-3.1-8B
model_size: 8B
ranking_group: single
us_pacific_date: 2024-08-08
win_ratio: 0.44170403587443946
generation_params: {'temperature': 0.94, 'top_p': 0.95, 'min_p': 0.05, 'top_k': 40, 'presence_penalty': 0.03, 'frequency_penalty': 0.0, 'stopping_words': ['<|im_end|>', '<|im_start|>'], 'max_input_tokens': 1024, 'best_of': 4, 'max_output_tokens': 64, 'reward_max_token_input': 512}
formatter: {'memory_template': '<|im_start|>system\n{memory}<|im_end|>\n', 'prompt_template': '<|im_start|>user\n{prompt}<|im_end|>\n', 'bot_template': '<|im_start|>assistant\n{bot_name}: {message}<|im_end|>\n', 'user_template': '<|im_start|>user\n{user_name}: {message}<|im_end|>\n', 'response_template': '<|im_start|>assistant\n{bot_name}:', 'truncate_by_message': False}
reward_formatter: {'bot_template': '{bot_name}: {message}\n', 'memory_template': "{bot_name}'s Persona: {memory}\n####\n", 'prompt_template': '{prompt}\n<START>\n', 'response_template': '{bot_name}:', 'truncate_by_message': False, 'user_template': '{user_name}: {message}\n'}
Resubmit model
Running pipeline stage MKMLizer
Starting job with name locutusque-apollo-0-4-ll-7459-v2-mkmlizer
Waiting for job on locutusque-apollo-0-4-ll-7459-v2-mkmlizer to finish
locutusque-apollo-0-4-ll-7459-v2-mkmlizer: ╔═════════════════════════════════════════════════════════════════════╗
locutusque-apollo-0-4-ll-7459-v2-mkmlizer: ║     _____            __           __                                ║
locutusque-apollo-0-4-ll-7459-v2-mkmlizer: ║    / _/ /_ ___    __/ /  ___ ___ / /                                ║
locutusque-apollo-0-4-ll-7459-v2-mkmlizer: ║   / _/ / // / |/|/ / _ \/ -_) -_) /                                 ║
locutusque-apollo-0-4-ll-7459-v2-mkmlizer: ║  /_//_/\_, /|__,__/_//_/\__/\__/_/                                  ║
locutusque-apollo-0-4-ll-7459-v2-mkmlizer: ║       /___/                                                         ║
locutusque-apollo-0-4-ll-7459-v2-mkmlizer: ║                                                                     ║
locutusque-apollo-0-4-ll-7459-v2-mkmlizer: ║  Version: 0.9.9                                                     ║
locutusque-apollo-0-4-ll-7459-v2-mkmlizer: ║  Copyright 2023 MK ONE TECHNOLOGIES Inc.                            ║
locutusque-apollo-0-4-ll-7459-v2-mkmlizer: ║  https://mk1.ai                                                     ║
locutusque-apollo-0-4-ll-7459-v2-mkmlizer: ║                                                                     ║
locutusque-apollo-0-4-ll-7459-v2-mkmlizer: ║  The license key for the current software has been verified as      ║
locutusque-apollo-0-4-ll-7459-v2-mkmlizer: ║  belonging to:                                                      ║
locutusque-apollo-0-4-ll-7459-v2-mkmlizer: ║                                                                     ║
locutusque-apollo-0-4-ll-7459-v2-mkmlizer: ║  Chai Research Corp.                                                ║
locutusque-apollo-0-4-ll-7459-v2-mkmlizer: ║  Account ID: 7997a29f-0ceb-4cc7-9adf-840c57b4ae6f                   ║
locutusque-apollo-0-4-ll-7459-v2-mkmlizer: ║  Expiration: 2024-10-15 23:59:59                                    ║
locutusque-apollo-0-4-ll-7459-v2-mkmlizer: ║                                                                     ║
locutusque-apollo-0-4-ll-7459-v2-mkmlizer: ╚═════════════════════════════════════════════════════════════════════╝
locutusque-apollo-0-4-ll-7459-v2-mkmlizer: Downloaded to shared memory in 30.450s
locutusque-apollo-0-4-ll-7459-v2-mkmlizer: quantizing model to /dev/shm/model_cache, profile:s0, folder:/tmp/tmp7yycfh4g, device:0
locutusque-apollo-0-4-ll-7459-v2-mkmlizer: Saving flywheel model at /dev/shm/model_cache
locutusque-apollo-0-4-ll-7459-v2-mkmlizer: quantized model in 26.894s
locutusque-apollo-0-4-ll-7459-v2-mkmlizer: Processed model Locutusque/Apollo-0.4-Llama-3.1-8B in 57.344s
locutusque-apollo-0-4-ll-7459-v2-mkmlizer: creating bucket guanaco-mkml-models
locutusque-apollo-0-4-ll-7459-v2-mkmlizer: Bucket 's3://guanaco-mkml-models/' created
locutusque-apollo-0-4-ll-7459-v2-mkmlizer: uploading /dev/shm/model_cache to s3://guanaco-mkml-models/locutusque-apollo-0-4-ll-7459-v2
locutusque-apollo-0-4-ll-7459-v2-mkmlizer: cp /dev/shm/model_cache/config.json s3://guanaco-mkml-models/locutusque-apollo-0-4-ll-7459-v2/config.json
locutusque-apollo-0-4-ll-7459-v2-mkmlizer: cp /dev/shm/model_cache/special_tokens_map.json s3://guanaco-mkml-models/locutusque-apollo-0-4-ll-7459-v2/special_tokens_map.json
locutusque-apollo-0-4-ll-7459-v2-mkmlizer: cp /dev/shm/model_cache/tokenizer_config.json s3://guanaco-mkml-models/locutusque-apollo-0-4-ll-7459-v2/tokenizer_config.json
locutusque-apollo-0-4-ll-7459-v2-mkmlizer: cp /dev/shm/model_cache/tokenizer.json s3://guanaco-mkml-models/locutusque-apollo-0-4-ll-7459-v2/tokenizer.json
locutusque-apollo-0-4-ll-7459-v2-mkmlizer: cp /dev/shm/model_cache/flywheel_model.0.safetensors s3://guanaco-mkml-models/locutusque-apollo-0-4-ll-7459-v2/flywheel_model.0.safetensors
locutusque-apollo-0-4-ll-7459-v2-mkmlizer: /opt/conda/lib/python3.10/site-packages/transformers/models/auto/tokenization_auto.py:785: FutureWarning: The `use_auth_token` argument is deprecated and will be removed in v5 of Transformers. Please use `token` instead.
locutusque-apollo-0-4-ll-7459-v2-mkmlizer:   warnings.warn(
locutusque-apollo-0-4-ll-7459-v2-mkmlizer: /opt/conda/lib/python3.10/site-packages/transformers/models/auto/auto_factory.py:469: FutureWarning: The `use_auth_token` argument is deprecated and will be removed in v5 of Transformers. Please use `token` instead.
locutusque-apollo-0-4-ll-7459-v2-mkmlizer:   warnings.warn(
locutusque-apollo-0-4-ll-7459-v2-mkmlizer: 
Downloading shards:   0%|          | 0/2 [00:00<?, ?it/s]
Downloading shards:  50%|█████     | 1/2 [00:06<00:06,  6.33s/it]
Downloading shards: 100%|██████████| 2/2 [00:09<00:00,  4.68s/it]
Downloading shards: 100%|██████████| 2/2 [00:09<00:00,  4.93s/it]
locutusque-apollo-0-4-ll-7459-v2-mkmlizer: 
Loading checkpoint shards:   0%|          | 0/2 [00:00<?, ?it/s]
Loading checkpoint shards:  50%|█████     | 1/2 [00:00<00:00,  2.27it/s]
Loading checkpoint shards: 100%|██████████| 2/2 [00:00<00:00,  3.76it/s]
Loading checkpoint shards: 100%|██████████| 2/2 [00:00<00:00,  3.42it/s]
locutusque-apollo-0-4-ll-7459-v2-mkmlizer: Saving model to /tmp/reward_cache/reward.tensors
locutusque-apollo-0-4-ll-7459-v2-mkmlizer: Saving duration: 1.304s
locutusque-apollo-0-4-ll-7459-v2-mkmlizer: Processed model ChaiML/gpt2_xl_pairwise_89m_step_347634 in 14.795s
locutusque-apollo-0-4-ll-7459-v2-mkmlizer: creating bucket guanaco-reward-models
locutusque-apollo-0-4-ll-7459-v2-mkmlizer: Bucket 's3://guanaco-reward-models/' created
locutusque-apollo-0-4-ll-7459-v2-mkmlizer: uploading /tmp/reward_cache to s3://guanaco-reward-models/locutusque-apollo-0-4-ll-7459-v2_reward
locutusque-apollo-0-4-ll-7459-v2-mkmlizer: cp /tmp/reward_cache/config.json s3://guanaco-reward-models/locutusque-apollo-0-4-ll-7459-v2_reward/config.json
locutusque-apollo-0-4-ll-7459-v2-mkmlizer: cp /tmp/reward_cache/tokenizer_config.json s3://guanaco-reward-models/locutusque-apollo-0-4-ll-7459-v2_reward/tokenizer_config.json
locutusque-apollo-0-4-ll-7459-v2-mkmlizer: cp /tmp/reward_cache/special_tokens_map.json s3://guanaco-reward-models/locutusque-apollo-0-4-ll-7459-v2_reward/special_tokens_map.json
locutusque-apollo-0-4-ll-7459-v2-mkmlizer: cp /tmp/reward_cache/merges.txt s3://guanaco-reward-models/locutusque-apollo-0-4-ll-7459-v2_reward/merges.txt
locutusque-apollo-0-4-ll-7459-v2-mkmlizer: cp /tmp/reward_cache/vocab.json s3://guanaco-reward-models/locutusque-apollo-0-4-ll-7459-v2_reward/vocab.json
locutusque-apollo-0-4-ll-7459-v2-mkmlizer: cp /tmp/reward_cache/tokenizer.json s3://guanaco-reward-models/locutusque-apollo-0-4-ll-7459-v2_reward/tokenizer.json
locutusque-apollo-0-4-ll-7459-v2-mkmlizer: cp /tmp/reward_cache/reward.tensors s3://guanaco-reward-models/locutusque-apollo-0-4-ll-7459-v2_reward/reward.tensors
Job locutusque-apollo-0-4-ll-7459-v2-mkmlizer completed after 104.83s with status: succeeded
Stopping job with name locutusque-apollo-0-4-ll-7459-v2-mkmlizer
Pipeline stage MKMLizer completed in 105.90s
Running pipeline stage MKMLKubeTemplater
Pipeline stage MKMLKubeTemplater completed in 0.10s
Running pipeline stage ISVCDeployer
Creating inference service locutusque-apollo-0-4-ll-7459-v2
Waiting for inference service locutusque-apollo-0-4-ll-7459-v2 to be ready
Inference service locutusque-apollo-0-4-ll-7459-v2 ready after 191.1983494758606s
Pipeline stage ISVCDeployer completed in 193.01s
Running pipeline stage StressChecker
Received healthy response to inference request in 2.087592840194702s
Received healthy response to inference request in 1.2657854557037354s
Received healthy response to inference request in 1.2683007717132568s
Received healthy response to inference request in 1.272892951965332s
Received healthy response to inference request in 1.283339500427246s
5 requests
0 failed requests
5th percentile: 1.2662885189056396
10th percentile: 1.266791582107544
20th percentile: 1.2677977085113525
30th percentile: 1.269219207763672
40th percentile: 1.2710560798645019
50th percentile: 1.272892951965332
60th percentile: 1.2770715713500977
70th percentile: 1.2812501907348632
80th percentile: 1.4441901683807374
90th percentile: 1.76589150428772
95th percentile: 1.9267421722412108
99th percentile: 2.055422706604004
mean time: 1.4355823040008544
Pipeline stage StressChecker completed in 7.92s
locutusque-apollo-0-4-ll_7459_v2 status is now deployed due to DeploymentManager action
locutusque-apollo-0-4-ll_7459_v2 status is now inactive due to auto deactivation removed underperforming models
locutusque-apollo-0-4-ll_7459_v2 status is now deployed due to admin request
locutusque-apollo-0-4-ll_7459_v2 status is now inactive due to auto deactivation removed underperforming models
locutusque-apollo-0-4-ll_7459_v2 status is now torndown due to DeploymentManager action