locutusque-apollo-0-3-ll_5757

developer_uid: Locutusque

submission_id: locutusque-apollo-0-3-ll_5757_v1

model_name: locutusque-apollo-0-3-ll_5757_v1

model_group: Locutusque/Apollo-0.3-Ll

status: torndown

timestamp: 2024-08-04T23:58:11+00:00

num_battles: 15403

num_wins: 6726

celo_rating: 1173.18

family_friendly_score: 0.0

submission_type: basic

model_repo: Locutusque/Apollo-0.3-Llama-3.1-8B

model_architecture: LlamaForCausalLM

reward_repo: ChaiML/gpt2_xl_pairwise_89m_step_347634

model_num_parameters: 8030261248.0

best_of: 4

max_input_tokens: 512

max_output_tokens: 64

display_name: locutusque-apollo-0-3-ll_5757_v1

is_internal_developer: False

language_model: Locutusque/Apollo-0.3-Llama-3.1-8B

model_size: 8B

ranking_group: single

us_pacific_date: 2024-08-04

win_ratio: 0.4366681815230799

generation_params: {'temperature': 0.94, 'top_p': 0.95, 'min_p': 0.05, 'top_k': 40, 'presence_penalty': 0.05, 'frequency_penalty': 0.0, 'stopping_words': ['<|im_end|>', '<|im_start|>'], 'max_input_tokens': 512, 'best_of': 4, 'max_output_tokens': 64, 'reward_max_token_input': 512}

formatter: {'memory_template': '<|im_start|>system\n{memory}<|im_end|>\n', 'prompt_template': '<|im_start|>user\n{prompt}<|im_end|>\n', 'bot_template': '<|im_start|>assistant\n{bot_name}: {message}<|im_end|>\n', 'user_template': '<|im_start|>user\n{user_name}: {message}<|im_end|>\n', 'response_template': '<|im_start|>assistant\n{bot_name}:', 'truncate_by_message': False}

reward_formatter: {'bot_template': '{bot_name}: {message}\n', 'memory_template': "{bot_name}'s Persona: {memory}\n####\n", 'prompt_template': '{prompt}\n<START>\n', 'response_template': '{bot_name}:', 'truncate_by_message': False, 'user_template': '{user_name}: {message}\n'}

Resubmit model

Running pipeline stage MKMLizer
Starting job with name locutusque-apollo-0-3-ll-5757-v1-mkmlizer
Waiting for job on locutusque-apollo-0-3-ll-5757-v1-mkmlizer to finish
locutusque-apollo-0-3-ll-5757-v1-mkmlizer: ╔═════════════════════════════════════════════════════════════════════╗
locutusque-apollo-0-3-ll-5757-v1-mkmlizer: ║     _____            __           __                                ║
locutusque-apollo-0-3-ll-5757-v1-mkmlizer: ║    / _/ /_ ___    __/ /  ___ ___ / /                                ║
locutusque-apollo-0-3-ll-5757-v1-mkmlizer: ║   / _/ / // / |/|/ / _ \/ -_) -_) /                                 ║
locutusque-apollo-0-3-ll-5757-v1-mkmlizer: ║  /_//_/\_, /|__,__/_//_/\__/\__/_/                                  ║
locutusque-apollo-0-3-ll-5757-v1-mkmlizer: ║       /___/                                                         ║
locutusque-apollo-0-3-ll-5757-v1-mkmlizer: ║                                                                     ║
locutusque-apollo-0-3-ll-5757-v1-mkmlizer: ║  Version: 0.9.9                                                     ║
locutusque-apollo-0-3-ll-5757-v1-mkmlizer: ║  Copyright 2023 MK ONE TECHNOLOGIES Inc.                            ║
locutusque-apollo-0-3-ll-5757-v1-mkmlizer: ║  https://mk1.ai                                                     ║
locutusque-apollo-0-3-ll-5757-v1-mkmlizer: ║                                                                     ║
locutusque-apollo-0-3-ll-5757-v1-mkmlizer: ║  The license key for the current software has been verified as      ║
locutusque-apollo-0-3-ll-5757-v1-mkmlizer: ║  belonging to:                                                      ║
locutusque-apollo-0-3-ll-5757-v1-mkmlizer: ║                                                                     ║
locutusque-apollo-0-3-ll-5757-v1-mkmlizer: ║  Chai Research Corp.                                                ║
locutusque-apollo-0-3-ll-5757-v1-mkmlizer: ║  Account ID: 7997a29f-0ceb-4cc7-9adf-840c57b4ae6f                   ║
locutusque-apollo-0-3-ll-5757-v1-mkmlizer: ║  Expiration: 2024-10-15 23:59:59                                    ║
locutusque-apollo-0-3-ll-5757-v1-mkmlizer: ║                                                                     ║
locutusque-apollo-0-3-ll-5757-v1-mkmlizer: ╚═════════════════════════════════════════════════════════════════════╝
locutusque-apollo-0-3-ll-5757-v1-mkmlizer: Downloaded to shared memory in 36.880s
locutusque-apollo-0-3-ll-5757-v1-mkmlizer: quantizing model to /dev/shm/model_cache, profile:s0, folder:/tmp/tmp4ui765nn, device:0
locutusque-apollo-0-3-ll-5757-v1-mkmlizer: Saving flywheel model at /dev/shm/model_cache
locutusque-apollo-0-3-ll-5757-v1-mkmlizer: quantized model in 26.496s
locutusque-apollo-0-3-ll-5757-v1-mkmlizer: Processed model Locutusque/Apollo-0.3-Llama-3.1-8B in 63.376s
locutusque-apollo-0-3-ll-5757-v1-mkmlizer: creating bucket guanaco-mkml-models
locutusque-apollo-0-3-ll-5757-v1-mkmlizer: Bucket 's3://guanaco-mkml-models/' created
locutusque-apollo-0-3-ll-5757-v1-mkmlizer: uploading /dev/shm/model_cache to s3://guanaco-mkml-models/locutusque-apollo-0-3-ll-5757-v1
locutusque-apollo-0-3-ll-5757-v1-mkmlizer: cp /dev/shm/model_cache/config.json s3://guanaco-mkml-models/locutusque-apollo-0-3-ll-5757-v1/config.json
locutusque-apollo-0-3-ll-5757-v1-mkmlizer: cp /dev/shm/model_cache/special_tokens_map.json s3://guanaco-mkml-models/locutusque-apollo-0-3-ll-5757-v1/special_tokens_map.json
locutusque-apollo-0-3-ll-5757-v1-mkmlizer: cp /dev/shm/model_cache/tokenizer_config.json s3://guanaco-mkml-models/locutusque-apollo-0-3-ll-5757-v1/tokenizer_config.json
locutusque-apollo-0-3-ll-5757-v1-mkmlizer: cp /dev/shm/model_cache/tokenizer.json s3://guanaco-mkml-models/locutusque-apollo-0-3-ll-5757-v1/tokenizer.json
locutusque-apollo-0-3-ll-5757-v1-mkmlizer: cp /dev/shm/model_cache/flywheel_model.0.safetensors s3://guanaco-mkml-models/locutusque-apollo-0-3-ll-5757-v1/flywheel_model.0.safetensors
locutusque-apollo-0-3-ll-5757-v1-mkmlizer: loading reward model from ChaiML/gpt2_xl_pairwise_89m_step_347634
locutusque-apollo-0-3-ll-5757-v1-mkmlizer: 
Loading 0:   0%|          | 0/291 [00:00<?, ?it/s]
Loading 0:   2%|▏         | 5/291 [00:00<00:07, 36.48it/s]
Loading 0:   4%|▍         | 13/291 [00:00<00:04, 58.58it/s]
Loading 0:   7%|▋         | 20/291 [00:00<00:04, 54.22it/s]
Loading 0:   9%|▉         | 26/291 [00:00<00:04, 53.59it/s]
Loading 0:  11%|█         | 32/291 [00:00<00:05, 44.77it/s]
Loading 0:  14%|█▍        | 41/291 [00:00<00:04, 56.52it/s]
Loading 0:  16%|█▋        | 48/291 [00:01<00:06, 38.28it/s]
Loading 0:  19%|█▉        | 55/291 [00:01<00:05, 42.06it/s]
Loading 0:  21%|██        | 61/291 [00:01<00:05, 44.65it/s]
Loading 0:  23%|██▎       | 67/291 [00:01<00:04, 47.08it/s]
Loading 0:  25%|██▌       | 73/291 [00:01<00:04, 43.71it/s]
Loading 0:  27%|██▋       | 80/291 [00:01<00:04, 48.94it/s]
Loading 0:  30%|██▉       | 86/291 [00:01<00:04, 46.08it/s]
Loading 0:  31%|███▏      | 91/291 [00:01<00:04, 44.91it/s]
Loading 0:  34%|███▎      | 98/291 [00:02<00:03, 49.47it/s]
Loading 0:  36%|███▌      | 104/291 [00:02<00:04, 44.86it/s]
Loading 0:  37%|███▋      | 109/291 [00:02<00:05, 33.95it/s]
Loading 0:  39%|███▉      | 113/291 [00:02<00:05, 34.17it/s]
Loading 0:  42%|████▏     | 121/291 [00:02<00:03, 42.87it/s]
Loading 0:  44%|████▎     | 127/291 [00:02<00:03, 42.20it/s]
Loading 0:  45%|████▌     | 132/291 [00:02<00:03, 43.42it/s]
Loading 0:  48%|████▊     | 139/291 [00:03<00:03, 49.11it/s]
Loading 0:  50%|████▉     | 145/291 [00:03<00:03, 46.78it/s]
Loading 0:  52%|█████▏    | 150/291 [00:03<00:03, 44.89it/s]
Loading 0:  54%|█████▍    | 157/291 [00:03<00:02, 50.21it/s]
Loading 0:  56%|█████▌    | 163/291 [00:03<00:02, 45.72it/s]
Loading 0:  58%|█████▊    | 168/291 [00:03<00:03, 35.44it/s]
Loading 0:  60%|██████    | 175/291 [00:03<00:02, 41.99it/s]
Loading 0:  62%|██████▏   | 181/291 [00:04<00:02, 40.61it/s]
Loading 0:  64%|██████▍   | 186/291 [00:04<00:02, 40.95it/s]
Loading 0:  66%|██████▋   | 193/291 [00:04<00:02, 46.12it/s]
Loading 0:  68%|██████▊   | 199/291 [00:04<00:02, 43.69it/s]
Loading 0:  70%|███████   | 204/291 [00:04<00:01, 43.97it/s]
Loading 0:  73%|███████▎  | 211/291 [00:04<00:01, 49.53it/s]
Loading 0:  75%|███████▍  | 217/291 [00:04<00:01, 45.61it/s]
Loading 0:  76%|███████▋  | 222/291 [00:04<00:01, 45.23it/s]
Loading 0:  78%|███████▊  | 227/291 [00:05<00:01, 37.30it/s]
Loading 0:  80%|███████▉  | 232/291 [00:05<00:01, 38.93it/s]
Loading 0:  82%|████████▏ | 238/291 [00:05<00:01, 42.72it/s]
Loading 0:  84%|████████▍ | 244/291 [00:05<00:01, 41.64it/s]
Loading 0:  86%|████████▌ | 249/291 [00:05<00:00, 42.48it/s]
Loading 0:  88%|████████▊ | 256/291 [00:05<00:00, 47.94it/s]
Loading 0:  90%|█████████ | 262/291 [00:05<00:00, 44.89it/s]
Loading 0:  92%|█████████▏| 267/291 [00:06<00:00, 44.96it/s]
Loading 0:  94%|█████████▍| 274/291 [00:06<00:00, 49.53it/s]
Loading 0:  96%|█████████▌| 280/291 [00:06<00:00, 44.32it/s]
Loading 0:  98%|█████████▊| 286/291 [00:11<00:01,  3.56it/s]
Loading 0: 100%|█████████▉| 290/291 [00:11<00:00,  4.46it/s]
                                                            
/opt/conda/lib/python3.10/site-packages/transformers/models/auto/configuration_auto.py:957: FutureWarning: The `use_auth_token` argument is deprecated and will be removed in v5 of Transformers. Please use `token` instead.
locutusque-apollo-0-3-ll-5757-v1-mkmlizer:   warnings.warn(
locutusque-apollo-0-3-ll-5757-v1-mkmlizer: /opt/conda/lib/python3.10/site-packages/transformers/models/auto/tokenization_auto.py:785: FutureWarning: The `use_auth_token` argument is deprecated and will be removed in v5 of Transformers. Please use `token` instead.
locutusque-apollo-0-3-ll-5757-v1-mkmlizer:   warnings.warn(
locutusque-apollo-0-3-ll-5757-v1-mkmlizer: /opt/conda/lib/python3.10/site-packages/transformers/models/auto/auto_factory.py:469: FutureWarning: The `use_auth_token` argument is deprecated and will be removed in v5 of Transformers. Please use `token` instead.
locutusque-apollo-0-3-ll-5757-v1-mkmlizer:   warnings.warn(
locutusque-apollo-0-3-ll-5757-v1-mkmlizer: 
Downloading shards:   0%|          | 0/2 [00:00<?, ?it/s]
Downloading shards:  50%|█████     | 1/2 [00:05<00:05,  5.62s/it]
Downloading shards: 100%|██████████| 2/2 [00:09<00:00,  4.30s/it]
Downloading shards: 100%|██████████| 2/2 [00:09<00:00,  4.50s/it]
locutusque-apollo-0-3-ll-5757-v1-mkmlizer: 
Loading checkpoint shards:   0%|          | 0/2 [00:00<?, ?it/s]
Loading checkpoint shards:  50%|█████     | 1/2 [00:00<00:00,  2.49it/s]
Loading checkpoint shards: 100%|██████████| 2/2 [00:00<00:00,  3.77it/s]
Loading checkpoint shards: 100%|██████████| 2/2 [00:00<00:00,  3.50it/s]
locutusque-apollo-0-3-ll-5757-v1-mkmlizer: Saving model to /tmp/reward_cache/reward.tensors
locutusque-apollo-0-3-ll-5757-v1-mkmlizer: Saving duration: 1.319s
locutusque-apollo-0-3-ll-5757-v1-mkmlizer: Processed model ChaiML/gpt2_xl_pairwise_89m_step_347634 in 14.149s
locutusque-apollo-0-3-ll-5757-v1-mkmlizer: creating bucket guanaco-reward-models
locutusque-apollo-0-3-ll-5757-v1-mkmlizer: Bucket 's3://guanaco-reward-models/' created
locutusque-apollo-0-3-ll-5757-v1-mkmlizer: uploading /tmp/reward_cache to s3://guanaco-reward-models/locutusque-apollo-0-3-ll-5757-v1_reward
locutusque-apollo-0-3-ll-5757-v1-mkmlizer: cp /tmp/reward_cache/config.json s3://guanaco-reward-models/locutusque-apollo-0-3-ll-5757-v1_reward/config.json
locutusque-apollo-0-3-ll-5757-v1-mkmlizer: cp /tmp/reward_cache/special_tokens_map.json s3://guanaco-reward-models/locutusque-apollo-0-3-ll-5757-v1_reward/special_tokens_map.json
locutusque-apollo-0-3-ll-5757-v1-mkmlizer: cp /tmp/reward_cache/tokenizer_config.json s3://guanaco-reward-models/locutusque-apollo-0-3-ll-5757-v1_reward/tokenizer_config.json
locutusque-apollo-0-3-ll-5757-v1-mkmlizer: cp /tmp/reward_cache/merges.txt s3://guanaco-reward-models/locutusque-apollo-0-3-ll-5757-v1_reward/merges.txt
locutusque-apollo-0-3-ll-5757-v1-mkmlizer: cp /tmp/reward_cache/vocab.json s3://guanaco-reward-models/locutusque-apollo-0-3-ll-5757-v1_reward/vocab.json
locutusque-apollo-0-3-ll-5757-v1-mkmlizer: cp /tmp/reward_cache/tokenizer.json s3://guanaco-reward-models/locutusque-apollo-0-3-ll-5757-v1_reward/tokenizer.json
locutusque-apollo-0-3-ll-5757-v1-mkmlizer: cp /tmp/reward_cache/reward.tensors s3://guanaco-reward-models/locutusque-apollo-0-3-ll-5757-v1_reward/reward.tensors
Job locutusque-apollo-0-3-ll-5757-v1-mkmlizer completed after 104.67s with status: succeeded
Stopping job with name locutusque-apollo-0-3-ll-5757-v1-mkmlizer
Pipeline stage MKMLizer completed in 105.70s
Running pipeline stage MKMLKubeTemplater
Pipeline stage MKMLKubeTemplater completed in 0.09s
Running pipeline stage ISVCDeployer
Creating inference service locutusque-apollo-0-3-ll-5757-v1
Waiting for inference service locutusque-apollo-0-3-ll-5757-v1 to be ready
Inference service locutusque-apollo-0-3-ll-5757-v1 ready after 171.01473593711853s
Pipeline stage ISVCDeployer completed in 173.21s
Running pipeline stage StressChecker
Received healthy response to inference request in 2.0382821559906006s
Received healthy response to inference request in 1.1766688823699951s
Received healthy response to inference request in 1.2097573280334473s
Received healthy response to inference request in 1.2064530849456787s
Received healthy response to inference request in 1.2181882858276367s
5 requests
0 failed requests
5th percentile: 1.1826257228851318
10th percentile: 1.1885825634002685
20th percentile: 1.200496244430542
30th percentile: 1.2071139335632324
40th percentile: 1.20843563079834
50th percentile: 1.2097573280334473
60th percentile: 1.213129711151123
70th percentile: 1.2165020942687987
80th percentile: 1.3822070598602296
90th percentile: 1.7102446079254152
95th percentile: 1.8742633819580077
99th percentile: 2.005478401184082
mean time: 1.3698699474334717
Pipeline stage StressChecker completed in 7.51s
locutusque-apollo-0-3-ll_5757_v1 status is now deployed due to DeploymentManager action
locutusque-apollo-0-3-ll_5757_v1 status is now inactive due to auto deactivation removed underperforming models
locutusque-apollo-0-3-ll_5757_v1 status is now torndown due to DeploymentManager action