submission_id: chaiml-sao10k-l3-rp-v3-3_v40
developer_uid: chai_backend_admin
alignment_samples: 0
best_of: 8
celo_rating: 1224.85
display_name: chaiml-sao10k-l3-rp-v3-3_v40
formatter: {'memory_template': "<|begin_of_text|><|start_header_id|>system<|end_header_id|>\n\n{bot_name}'s Persona: {memory}\n\n", 'prompt_template': '{prompt}<|eot_id|>', 'bot_template': '<|start_header_id|>assistant<|end_header_id|>\n\n{bot_name}: {message}<|eot_id|>', 'user_template': '<|start_header_id|>user<|end_header_id|>\n\n{user_name}: {message}<|eot_id|>', 'response_template': '<|start_header_id|>assistant<|end_header_id|>\n\n{bot_name}:', 'truncate_by_message': False}
generation_params: {'temperature': 0.95, 'top_p': 0.95, 'min_p': 0.05, 'top_k': 80, 'presence_penalty': 0.0, 'frequency_penalty': 0.0, 'stopping_words': ['\n', '<|end_header_id|>,', '<|eot_id|>,', '\n\n{user_name}'], 'max_input_tokens': 1024, 'best_of': 8, 'max_output_tokens': 64}
is_internal_developer: True
language_model: ChaiML/sao10k-l3-rp-v3-3
max_input_tokens: 1024
max_output_tokens: 64
model_architecture: LlamaForCausalLM
model_group: ChaiML/sao10k-l3-rp-v3-3
model_name: chaiml-sao10k-l3-rp-v3-3_v40
model_num_parameters: 8030261248.0
model_repo: ChaiML/sao10k-l3-rp-v3-3
model_size: 8B
num_battles: 21449
num_wins: 11961
propriety_score: 0.7257240204429302
propriety_total_count: 1761.0
ranking_group: single
reward_formatter: {'bot_template': '{bot_name}: {message}\n', 'memory_template': "{bot_name}'s Persona: {memory}\n####\n", 'prompt_template': '{prompt}\n<START>\n', 'response_template': '{bot_name}:', 'truncate_by_message': False, 'user_template': '{user_name}: {message}\n'}
reward_repo: ChaiML/gpt2_medium_pairwise_60m_step_937500
status: torndown
submission_type: basic
timestamp: 2024-07-17T22:45:42+00:00
us_pacific_date: 2024-07-17
win_ratio: 0.5576483752156278
Resubmit model
Running pipeline stage MKMLizer
Starting job with name chaiml-sao10k-l3-rp-v3-3-v40-mkmlizer
Waiting for job on chaiml-sao10k-l3-rp-v3-3-v40-mkmlizer to finish
chaiml-sao10k-l3-rp-v3-3-v40-mkmlizer: ╔═════════════════════════════════════════════════════════════════════╗
chaiml-sao10k-l3-rp-v3-3-v40-mkmlizer: ║ _____ __ __ ║
chaiml-sao10k-l3-rp-v3-3-v40-mkmlizer: ║ / _/ /_ ___ __/ / ___ ___ / / ║
chaiml-sao10k-l3-rp-v3-3-v40-mkmlizer: ║ / _/ / // / |/|/ / _ \/ -_) -_) / ║
chaiml-sao10k-l3-rp-v3-3-v40-mkmlizer: ║ /_//_/\_, /|__,__/_//_/\__/\__/_/ ║
chaiml-sao10k-l3-rp-v3-3-v40-mkmlizer: ║ /___/ ║
chaiml-sao10k-l3-rp-v3-3-v40-mkmlizer: ║ ║
chaiml-sao10k-l3-rp-v3-3-v40-mkmlizer: ║ Version: 0.9.5.post3 ║
chaiml-sao10k-l3-rp-v3-3-v40-mkmlizer: ║ Copyright 2023 MK ONE TECHNOLOGIES Inc. ║
chaiml-sao10k-l3-rp-v3-3-v40-mkmlizer: ║ https://mk1.ai ║
chaiml-sao10k-l3-rp-v3-3-v40-mkmlizer: ║ ║
chaiml-sao10k-l3-rp-v3-3-v40-mkmlizer: ║ The license key for the current software has been verified as ║
chaiml-sao10k-l3-rp-v3-3-v40-mkmlizer: ║ belonging to: ║
chaiml-sao10k-l3-rp-v3-3-v40-mkmlizer: ║ ║
chaiml-sao10k-l3-rp-v3-3-v40-mkmlizer: ║ Chai Research Corp. ║
chaiml-sao10k-l3-rp-v3-3-v40-mkmlizer: ║ Account ID: 7997a29f-0ceb-4cc7-9adf-840c57b4ae6f ║
chaiml-sao10k-l3-rp-v3-3-v40-mkmlizer: ║ Expiration: 2024-10-15 23:59:59 ║
chaiml-sao10k-l3-rp-v3-3-v40-mkmlizer: ║ ║
chaiml-sao10k-l3-rp-v3-3-v40-mkmlizer: ╚═════════════════════════════════════════════════════════════════════╝
chaiml-sao10k-l3-rp-v3-3-v40-mkmlizer: Downloaded to shared memory in 24.860s
chaiml-sao10k-l3-rp-v3-3-v40-mkmlizer: quantizing model to /dev/shm/model_cache, profile:s0, folder:/tmp/tmps5w6_2dj, device:0
chaiml-sao10k-l3-rp-v3-3-v40-mkmlizer: Saving flywheel model at /dev/shm/model_cache
chaiml-sao10k-l3-rp-v3-3-v40-mkmlizer: Loading 0: 0%| | 0/291 [00:00<?, ?it/s] Loading 0: 1%| | 2/291 [00:04<11:24, 2.37s/it] Loading 0: 4%|▍ | 13/291 [00:04<01:16, 3.63it/s] Loading 0: 8%|▊ | 23/291 [00:04<00:35, 7.50it/s] Loading 0: 11%|█▏ | 33/291 [00:05<00:20, 12.51it/s] Loading 0: 17%|█▋ | 50/291 [00:05<00:10, 23.62it/s] Loading 0: 21%|██▏ | 62/291 [00:05<00:09, 25.08it/s] Loading 0: 27%|██▋ | 78/291 [00:05<00:05, 37.00it/s] Loading 0: 33%|███▎ | 96/291 [00:05<00:03, 52.61it/s] Loading 0: 39%|███▉ | 114/291 [00:05<00:02, 69.51it/s] Loading 0: 45%|████▌ | 132/291 [00:06<00:01, 86.72it/s] Loading 0: 51%|█████ | 149/291 [00:06<00:01, 102.38it/s] Loading 0: 57%|█████▋ | 166/291 [00:06<00:01, 77.13it/s] Loading 0: 63%|██████▎ | 184/291 [00:06<00:01, 88.74it/s] Loading 0: 69%|██████▉ | 202/291 [00:06<00:00, 103.13it/s] Loading 0: 74%|███████▍ | 216/291 [00:06<00:00, 110.50it/s] Loading 0: 79%|███████▉ | 231/291 [00:06<00:00, 117.75it/s] Loading 0: 86%|████████▌ | 249/291 [00:07<00:00, 130.09it/s] Loading 0: 91%|█████████▏| 266/291 [00:07<00:00, 87.07it/s] Loading 0: 98%|█████████▊| 284/291 [00:07<00:00, 102.21it/s] Special tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained.
chaiml-sao10k-l3-rp-v3-3-v40-mkmlizer: quantized model in 28.266s
chaiml-sao10k-l3-rp-v3-3-v40-mkmlizer: Processed model ChaiML/sao10k-l3-rp-v3-3 in 53.127s
chaiml-sao10k-l3-rp-v3-3-v40-mkmlizer: creating bucket guanaco-mkml-models
chaiml-sao10k-l3-rp-v3-3-v40-mkmlizer: Bucket 's3://guanaco-mkml-models/' created
chaiml-sao10k-l3-rp-v3-3-v40-mkmlizer: uploading /dev/shm/model_cache to s3://guanaco-mkml-models/chaiml-sao10k-l3-rp-v3-3-v40
chaiml-sao10k-l3-rp-v3-3-v40-mkmlizer: cp /dev/shm/model_cache/special_tokens_map.json s3://guanaco-mkml-models/chaiml-sao10k-l3-rp-v3-3-v40/special_tokens_map.json
chaiml-sao10k-l3-rp-v3-3-v40-mkmlizer: cp /dev/shm/model_cache/config.json s3://guanaco-mkml-models/chaiml-sao10k-l3-rp-v3-3-v40/config.json
chaiml-sao10k-l3-rp-v3-3-v40-mkmlizer: cp /dev/shm/model_cache/tokenizer_config.json s3://guanaco-mkml-models/chaiml-sao10k-l3-rp-v3-3-v40/tokenizer_config.json
chaiml-sao10k-l3-rp-v3-3-v40-mkmlizer: cp /dev/shm/model_cache/tokenizer.json s3://guanaco-mkml-models/chaiml-sao10k-l3-rp-v3-3-v40/tokenizer.json
chaiml-sao10k-l3-rp-v3-3-v40-mkmlizer: cp /dev/shm/model_cache/flywheel_model.0.safetensors s3://guanaco-mkml-models/chaiml-sao10k-l3-rp-v3-3-v40/flywheel_model.0.safetensors
chaiml-sao10k-l3-rp-v3-3-v40-mkmlizer: loading reward model from ChaiML/gpt2_medium_pairwise_60m_step_937500
chaiml-sao10k-l3-rp-v3-3-v40-mkmlizer: /opt/conda/lib/python3.10/site-packages/transformers/models/auto/configuration_auto.py:950: FutureWarning: The `use_auth_token` argument is deprecated and will be removed in v5 of Transformers. Please use `token` instead.
chaiml-sao10k-l3-rp-v3-3-v40-mkmlizer: warnings.warn(
chaiml-sao10k-l3-rp-v3-3-v40-mkmlizer: /opt/conda/lib/python3.10/site-packages/transformers/models/auto/tokenization_auto.py:778: FutureWarning: The `use_auth_token` argument is deprecated and will be removed in v5 of Transformers. Please use `token` instead.
chaiml-sao10k-l3-rp-v3-3-v40-mkmlizer: warnings.warn(
chaiml-sao10k-l3-rp-v3-3-v40-mkmlizer: /opt/conda/lib/python3.10/site-packages/transformers/models/auto/auto_factory.py:469: FutureWarning: The `use_auth_token` argument is deprecated and will be removed in v5 of Transformers. Please use `token` instead.
chaiml-sao10k-l3-rp-v3-3-v40-mkmlizer: warnings.warn(
chaiml-sao10k-l3-rp-v3-3-v40-mkmlizer: Saving model to /tmp/reward_cache/reward.tensors
chaiml-sao10k-l3-rp-v3-3-v40-mkmlizer: Saving duration: 0.485s
chaiml-sao10k-l3-rp-v3-3-v40-mkmlizer: Processed model ChaiML/gpt2_medium_pairwise_60m_step_937500 in 5.582s
Job chaiml-sao10k-l3-rp-v3-3-v40-mkmlizer completed after 174.39s with status: succeeded
Stopping job with name chaiml-sao10k-l3-rp-v3-3-v40-mkmlizer
Pipeline stage MKMLizer completed in 176.02s
Running pipeline stage MKMLKubeTemplater
Pipeline stage MKMLKubeTemplater completed in 0.26s
Running pipeline stage ISVCDeployer
Creating inference service chaiml-sao10k-l3-rp-v3-3-v40
Waiting for inference service chaiml-sao10k-l3-rp-v3-3-v40 to be ready
Inference service chaiml-sao10k-l3-rp-v3-3-v40 ready after 122.52549433708191s
Pipeline stage ISVCDeployer completed in 124.23s
Running pipeline stage StressChecker
HTTP Request: %s %s "%s %d %s"
Received healthy response to inference request in 2.2951090335845947s
HTTP Request: %s %s "%s %d %s"
Received healthy response to inference request in 1.512831449508667s
HTTP Request: %s %s "%s %d %s"
Received healthy response to inference request in 1.4336261749267578s
HTTP Request: %s %s "%s %d %s"
Received healthy response to inference request in 1.454160451889038s
HTTP Request: %s %s "%s %d %s"
Received healthy response to inference request in 1.5257666110992432s
5 requests
0 failed requests
5th percentile: 1.437733030319214
10th percentile: 1.4418398857116699
20th percentile: 1.450053596496582
30th percentile: 1.4658946514129638
40th percentile: 1.4893630504608155
50th percentile: 1.512831449508667
60th percentile: 1.5180055141448974
70th percentile: 1.523179578781128
80th percentile: 1.6796350955963135
90th percentile: 1.9873720645904542
95th percentile: 2.141240549087524
99th percentile: 2.2643353366851806
mean time: 1.6442987442016601
Pipeline stage StressChecker completed in 10.33s
chaiml-sao10k-l3-rp-v3-3_v40 status is now deployed due to DeploymentManager action
chaiml-sao10k-l3-rp-v3-3_v40 status is now inactive due to auto deactivation removed underperforming models
admin requested tearing down of chaiml-sao10k-l3-rp-v3-3_v40
Running pipeline stage ISVCDeleter
Checking if service chaiml-sao10k-l3-rp-v3-3-v40 is running
Tearing down inference service chaiml-sao10k-l3-rp-v3-3-v40
Service chaiml-sao10k-l3-rp-v3-3-v40 has been torndown
Pipeline stage ISVCDeleter completed in 4.83s
Running pipeline stage MKMLModelDeleter
Cleaning model data from S3
Cleaning model data from model cache
Deleting key chaiml-sao10k-l3-rp-v3-3-v40/config.json from bucket guanaco-mkml-models
Deleting key chaiml-sao10k-l3-rp-v3-3-v40/flywheel_model.0.safetensors from bucket guanaco-mkml-models
Deleting key chaiml-sao10k-l3-rp-v3-3-v40/special_tokens_map.json from bucket guanaco-mkml-models
Deleting key chaiml-sao10k-l3-rp-v3-3-v40/tokenizer.json from bucket guanaco-mkml-models
Deleting key chaiml-sao10k-l3-rp-v3-3-v40/tokenizer_config.json from bucket guanaco-mkml-models
Cleaning model data from model cache
Deleting key chaiml-sao10k-l3-rp-v3-3-v40_reward/config.json from bucket guanaco-reward-models
Deleting key chaiml-sao10k-l3-rp-v3-3-v40_reward/merges.txt from bucket guanaco-reward-models
Deleting key chaiml-sao10k-l3-rp-v3-3-v40_reward/reward.tensors from bucket guanaco-reward-models
Deleting key chaiml-sao10k-l3-rp-v3-3-v40_reward/special_tokens_map.json from bucket guanaco-reward-models
Deleting key chaiml-sao10k-l3-rp-v3-3-v40_reward/tokenizer.json from bucket guanaco-reward-models
Deleting key chaiml-sao10k-l3-rp-v3-3-v40_reward/tokenizer_config.json from bucket guanaco-reward-models
Deleting key chaiml-sao10k-l3-rp-v3-3-v40_reward/vocab.json from bucket guanaco-reward-models
Pipeline stage MKMLModelDeleter completed in 5.31s
chaiml-sao10k-l3-rp-v3-3_v40 status is now torndown due to DeploymentManager action

Usage Metrics

Latency Metrics