developer_uid: chai_backend_admin
submission_id: chaiml-sao10k-l3-rp-v3-3_v40
model_name: chaiml-sao10k-l3-rp-v3-3_v40
model_group: ChaiML/sao10k-l3-rp-v3-3
status: torndown
timestamp: 2024-07-17T22:45:42+00:00
num_battles: 21449
num_wins: 11961
celo_rating: 1224.83
family_friendly_score: 0.0
submission_type: basic
model_repo: ChaiML/sao10k-l3-rp-v3-3
model_architecture: LlamaForCausalLM
reward_repo: ChaiML/gpt2_medium_pairwise_60m_step_937500
model_num_parameters: 8030261248.0
best_of: 8
max_input_tokens: 1024
max_output_tokens: 64
display_name: chaiml-sao10k-l3-rp-v3-3_v40
is_internal_developer: True
language_model: ChaiML/sao10k-l3-rp-v3-3
model_size: 8B
ranking_group: single
us_pacific_date: 2024-07-17
win_ratio: 0.5576483752156278
generation_params: {'temperature': 0.95, 'top_p': 0.95, 'min_p': 0.05, 'top_k': 80, 'presence_penalty': 0.0, 'frequency_penalty': 0.0, 'stopping_words': ['\n', '<|end_header_id|>,', '<|eot_id|>,', '\n\n{user_name}'], 'max_input_tokens': 1024, 'best_of': 8, 'max_output_tokens': 64}
formatter: {'memory_template': "<|begin_of_text|><|start_header_id|>system<|end_header_id|>\n\n{bot_name}'s Persona: {memory}\n\n", 'prompt_template': '{prompt}<|eot_id|>', 'bot_template': '<|start_header_id|>assistant<|end_header_id|>\n\n{bot_name}: {message}<|eot_id|>', 'user_template': '<|start_header_id|>user<|end_header_id|>\n\n{user_name}: {message}<|eot_id|>', 'response_template': '<|start_header_id|>assistant<|end_header_id|>\n\n{bot_name}:', 'truncate_by_message': False}
reward_formatter: {'bot_template': '{bot_name}: {message}\n', 'memory_template': "{bot_name}'s Persona: {memory}\n####\n", 'prompt_template': '{prompt}\n<START>\n', 'response_template': '{bot_name}:', 'truncate_by_message': False, 'user_template': '{user_name}: {message}\n'}
Resubmit model
Running pipeline stage MKMLizer
Starting job with name chaiml-sao10k-l3-rp-v3-3-v40-mkmlizer
Waiting for job on chaiml-sao10k-l3-rp-v3-3-v40-mkmlizer to finish
chaiml-sao10k-l3-rp-v3-3-v40-mkmlizer: ╔═════════════════════════════════════════════════════════════════════╗
chaiml-sao10k-l3-rp-v3-3-v40-mkmlizer: ║     _____            __           __                                ║
chaiml-sao10k-l3-rp-v3-3-v40-mkmlizer: ║    / _/ /_ ___    __/ /  ___ ___ / /                                ║
chaiml-sao10k-l3-rp-v3-3-v40-mkmlizer: ║   / _/ / // / |/|/ / _ \/ -_) -_) /                                 ║
chaiml-sao10k-l3-rp-v3-3-v40-mkmlizer: ║  /_//_/\_, /|__,__/_//_/\__/\__/_/                                  ║
chaiml-sao10k-l3-rp-v3-3-v40-mkmlizer: ║       /___/                                                         ║
chaiml-sao10k-l3-rp-v3-3-v40-mkmlizer: ║                                                                     ║
chaiml-sao10k-l3-rp-v3-3-v40-mkmlizer: ║  Version: 0.9.5.post3                                               ║
chaiml-sao10k-l3-rp-v3-3-v40-mkmlizer: ║  Copyright 2023 MK ONE TECHNOLOGIES Inc.                            ║
chaiml-sao10k-l3-rp-v3-3-v40-mkmlizer: ║  https://mk1.ai                                                     ║
chaiml-sao10k-l3-rp-v3-3-v40-mkmlizer: ║                                                                     ║
chaiml-sao10k-l3-rp-v3-3-v40-mkmlizer: ║  The license key for the current software has been verified as      ║
chaiml-sao10k-l3-rp-v3-3-v40-mkmlizer: ║  belonging to:                                                      ║
chaiml-sao10k-l3-rp-v3-3-v40-mkmlizer: ║                                                                     ║
chaiml-sao10k-l3-rp-v3-3-v40-mkmlizer: ║  Chai Research Corp.                                                ║
chaiml-sao10k-l3-rp-v3-3-v40-mkmlizer: ║  Account ID: 7997a29f-0ceb-4cc7-9adf-840c57b4ae6f                   ║
chaiml-sao10k-l3-rp-v3-3-v40-mkmlizer: ║  Expiration: 2024-10-15 23:59:59                                    ║
chaiml-sao10k-l3-rp-v3-3-v40-mkmlizer: ║                                                                     ║
chaiml-sao10k-l3-rp-v3-3-v40-mkmlizer: ╚═════════════════════════════════════════════════════════════════════╝
chaiml-sao10k-l3-rp-v3-3-v40-mkmlizer: Downloaded to shared memory in 24.860s
chaiml-sao10k-l3-rp-v3-3-v40-mkmlizer: quantizing model to /dev/shm/model_cache, profile:s0, folder:/tmp/tmps5w6_2dj, device:0
chaiml-sao10k-l3-rp-v3-3-v40-mkmlizer: Saving flywheel model at /dev/shm/model_cache
chaiml-sao10k-l3-rp-v3-3-v40-mkmlizer: 
Loading 0:   0%|          | 0/291 [00:00<?, ?it/s]
Loading 0:   1%|          | 2/291 [00:04<11:24,  2.37s/it]
Loading 0:   4%|▍         | 13/291 [00:04<01:16,  3.63it/s]
Loading 0:   8%|▊         | 23/291 [00:04<00:35,  7.50it/s]
Loading 0:  11%|█▏        | 33/291 [00:05<00:20, 12.51it/s]
Loading 0:  17%|█▋        | 50/291 [00:05<00:10, 23.62it/s]
Loading 0:  21%|██▏       | 62/291 [00:05<00:09, 25.08it/s]
Loading 0:  27%|██▋       | 78/291 [00:05<00:05, 37.00it/s]
Loading 0:  33%|███▎      | 96/291 [00:05<00:03, 52.61it/s]
Loading 0:  39%|███▉      | 114/291 [00:05<00:02, 69.51it/s]
Loading 0:  45%|████▌     | 132/291 [00:06<00:01, 86.72it/s]
Loading 0:  51%|█████     | 149/291 [00:06<00:01, 102.38it/s]
Loading 0:  57%|█████▋    | 166/291 [00:06<00:01, 77.13it/s] 
Loading 0:  63%|██████▎   | 184/291 [00:06<00:01, 88.74it/s]
Loading 0:  69%|██████▉   | 202/291 [00:06<00:00, 103.13it/s]
Loading 0:  74%|███████▍  | 216/291 [00:06<00:00, 110.50it/s]
Loading 0:  79%|███████▉  | 231/291 [00:06<00:00, 117.75it/s]
Loading 0:  86%|████████▌ | 249/291 [00:07<00:00, 130.09it/s]
Loading 0:  91%|█████████▏| 266/291 [00:07<00:00, 87.07it/s] 
Loading 0:  98%|█████████▊| 284/291 [00:07<00:00, 102.21it/s]
                                                             
Special tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained.
chaiml-sao10k-l3-rp-v3-3-v40-mkmlizer: quantized model in 28.266s
chaiml-sao10k-l3-rp-v3-3-v40-mkmlizer: Processed model ChaiML/sao10k-l3-rp-v3-3 in 53.127s
chaiml-sao10k-l3-rp-v3-3-v40-mkmlizer: creating bucket guanaco-mkml-models
chaiml-sao10k-l3-rp-v3-3-v40-mkmlizer: Bucket 's3://guanaco-mkml-models/' created
chaiml-sao10k-l3-rp-v3-3-v40-mkmlizer: uploading /dev/shm/model_cache to s3://guanaco-mkml-models/chaiml-sao10k-l3-rp-v3-3-v40
chaiml-sao10k-l3-rp-v3-3-v40-mkmlizer: cp /dev/shm/model_cache/special_tokens_map.json s3://guanaco-mkml-models/chaiml-sao10k-l3-rp-v3-3-v40/special_tokens_map.json
chaiml-sao10k-l3-rp-v3-3-v40-mkmlizer: cp /dev/shm/model_cache/config.json s3://guanaco-mkml-models/chaiml-sao10k-l3-rp-v3-3-v40/config.json
chaiml-sao10k-l3-rp-v3-3-v40-mkmlizer: cp /dev/shm/model_cache/tokenizer_config.json s3://guanaco-mkml-models/chaiml-sao10k-l3-rp-v3-3-v40/tokenizer_config.json
chaiml-sao10k-l3-rp-v3-3-v40-mkmlizer: cp /dev/shm/model_cache/tokenizer.json s3://guanaco-mkml-models/chaiml-sao10k-l3-rp-v3-3-v40/tokenizer.json
chaiml-sao10k-l3-rp-v3-3-v40-mkmlizer: cp /dev/shm/model_cache/flywheel_model.0.safetensors s3://guanaco-mkml-models/chaiml-sao10k-l3-rp-v3-3-v40/flywheel_model.0.safetensors
chaiml-sao10k-l3-rp-v3-3-v40-mkmlizer: loading reward model from ChaiML/gpt2_medium_pairwise_60m_step_937500
chaiml-sao10k-l3-rp-v3-3-v40-mkmlizer: /opt/conda/lib/python3.10/site-packages/transformers/models/auto/configuration_auto.py:950: FutureWarning: The `use_auth_token` argument is deprecated and will be removed in v5 of Transformers. Please use `token` instead.
chaiml-sao10k-l3-rp-v3-3-v40-mkmlizer:   warnings.warn(
chaiml-sao10k-l3-rp-v3-3-v40-mkmlizer: /opt/conda/lib/python3.10/site-packages/transformers/models/auto/tokenization_auto.py:778: FutureWarning: The `use_auth_token` argument is deprecated and will be removed in v5 of Transformers. Please use `token` instead.
chaiml-sao10k-l3-rp-v3-3-v40-mkmlizer:   warnings.warn(
chaiml-sao10k-l3-rp-v3-3-v40-mkmlizer: /opt/conda/lib/python3.10/site-packages/transformers/models/auto/auto_factory.py:469: FutureWarning: The `use_auth_token` argument is deprecated and will be removed in v5 of Transformers. Please use `token` instead.
chaiml-sao10k-l3-rp-v3-3-v40-mkmlizer:   warnings.warn(
chaiml-sao10k-l3-rp-v3-3-v40-mkmlizer: Saving model to /tmp/reward_cache/reward.tensors
chaiml-sao10k-l3-rp-v3-3-v40-mkmlizer: Saving duration: 0.485s
chaiml-sao10k-l3-rp-v3-3-v40-mkmlizer: Processed model ChaiML/gpt2_medium_pairwise_60m_step_937500 in 5.582s
Job chaiml-sao10k-l3-rp-v3-3-v40-mkmlizer completed after 174.39s with status: succeeded
Stopping job with name chaiml-sao10k-l3-rp-v3-3-v40-mkmlizer
Pipeline stage MKMLizer completed in 176.02s
Running pipeline stage MKMLKubeTemplater
Pipeline stage MKMLKubeTemplater completed in 0.26s
Running pipeline stage ISVCDeployer
Creating inference service chaiml-sao10k-l3-rp-v3-3-v40
Waiting for inference service chaiml-sao10k-l3-rp-v3-3-v40 to be ready
Inference service chaiml-sao10k-l3-rp-v3-3-v40 ready after 122.52549433708191s
Pipeline stage ISVCDeployer completed in 124.23s
Running pipeline stage StressChecker
HTTP Request: %s %s "%s %d %s"
Received healthy response to inference request in 2.2951090335845947s
HTTP Request: %s %s "%s %d %s"
Received healthy response to inference request in 1.512831449508667s
HTTP Request: %s %s "%s %d %s"
Received healthy response to inference request in 1.4336261749267578s
HTTP Request: %s %s "%s %d %s"
Received healthy response to inference request in 1.454160451889038s
HTTP Request: %s %s "%s %d %s"
Received healthy response to inference request in 1.5257666110992432s
5 requests
0 failed requests
5th percentile: 1.437733030319214
10th percentile: 1.4418398857116699
20th percentile: 1.450053596496582
30th percentile: 1.4658946514129638
40th percentile: 1.4893630504608155
50th percentile: 1.512831449508667
60th percentile: 1.5180055141448974
70th percentile: 1.523179578781128
80th percentile: 1.6796350955963135
90th percentile: 1.9873720645904542
95th percentile: 2.141240549087524
99th percentile: 2.2643353366851806
mean time: 1.6442987442016601
Pipeline stage StressChecker completed in 10.33s
chaiml-sao10k-l3-rp-v3-3_v40 status is now deployed due to DeploymentManager action
chaiml-sao10k-l3-rp-v3-3_v40 status is now inactive due to auto deactivation removed underperforming models
admin requested tearing down of chaiml-sao10k-l3-rp-v3-3_v40
Running pipeline stage ISVCDeleter
Checking if service chaiml-sao10k-l3-rp-v3-3-v40 is running
Tearing down inference service chaiml-sao10k-l3-rp-v3-3-v40
Service chaiml-sao10k-l3-rp-v3-3-v40 has been torndown
Pipeline stage ISVCDeleter completed in 4.83s
Running pipeline stage MKMLModelDeleter
Cleaning model data from S3
Cleaning model data from model cache
Deleting key chaiml-sao10k-l3-rp-v3-3-v40/config.json from bucket guanaco-mkml-models
Deleting key chaiml-sao10k-l3-rp-v3-3-v40/flywheel_model.0.safetensors from bucket guanaco-mkml-models
Deleting key chaiml-sao10k-l3-rp-v3-3-v40/special_tokens_map.json from bucket guanaco-mkml-models
Deleting key chaiml-sao10k-l3-rp-v3-3-v40/tokenizer.json from bucket guanaco-mkml-models
Deleting key chaiml-sao10k-l3-rp-v3-3-v40/tokenizer_config.json from bucket guanaco-mkml-models
Cleaning model data from model cache
Deleting key chaiml-sao10k-l3-rp-v3-3-v40_reward/config.json from bucket guanaco-reward-models
Deleting key chaiml-sao10k-l3-rp-v3-3-v40_reward/merges.txt from bucket guanaco-reward-models
Deleting key chaiml-sao10k-l3-rp-v3-3-v40_reward/reward.tensors from bucket guanaco-reward-models
Deleting key chaiml-sao10k-l3-rp-v3-3-v40_reward/special_tokens_map.json from bucket guanaco-reward-models
Deleting key chaiml-sao10k-l3-rp-v3-3-v40_reward/tokenizer.json from bucket guanaco-reward-models
Deleting key chaiml-sao10k-l3-rp-v3-3-v40_reward/tokenizer_config.json from bucket guanaco-reward-models
Deleting key chaiml-sao10k-l3-rp-v3-3-v40_reward/vocab.json from bucket guanaco-reward-models
Pipeline stage MKMLModelDeleter completed in 5.31s
chaiml-sao10k-l3-rp-v3-3_v40 status is now torndown due to DeploymentManager action