submission_id: juvi21-kumiho-v1-rp-uwu-8b_v1
developer_uid: juvi21
alignment_samples: 0
best_of: 4
celo_rating: 1157.51
display_name: juvi21-kumiho-v1-rp-uwu-8b_v1
formatter: {'memory_template': "{bot_name}'s Persona: {memory}\n####\n", 'prompt_template': '{prompt}\n<START>\n', 'bot_template': '{bot_name}: {message}\n', 'user_template': '{user_name}: {message}\n', 'response_template': '{bot_name}:', 'truncate_by_message': False}
generation_params: {'temperature': 1.4, 'top_p': 1.0, 'min_p': 0.1, 'top_k': 40, 'presence_penalty': 0.0, 'frequency_penalty': 0.0, 'stopping_words': ['\n'], 'max_input_tokens': 512, 'best_of': 4, 'max_output_tokens': 64, 'reward_max_token_input': 1024}
is_internal_developer: False
language_model: juvi21/Kumiho-v1-rp-UwU-8B
max_input_tokens: 512
max_output_tokens: 64
model_architecture: LlamaForCausalLM
model_group: juvi21/Kumiho-v1-rp-UwU-
model_name: juvi21-kumiho-v1-rp-uwu-8b_v1
model_num_parameters: 8030261248.0
model_repo: juvi21/Kumiho-v1-rp-UwU-8B
model_size: 8B
num_battles: 18354
num_wins: 7446
propriety_score: 0.7307692307692307
propriety_total_count: 1612.0
ranking_group: single
reward_formatter: {'bot_template': '{bot_name}: {message}\n', 'memory_template': "{bot_name}'s Persona: {memory}\n####\n", 'prompt_template': '{prompt}\n<START>\n', 'response_template': '{bot_name}:', 'truncate_by_message': False, 'user_template': '{user_name}: {message}\n'}
reward_repo: ChaiML/gpt2_xl_pairwise_89m_step_347634
status: torndown
submission_type: basic
timestamp: 2024-08-03T12:25:28+00:00
us_pacific_date: 2024-08-03
win_ratio: 0.40568813337692056
Download Preference Data
Resubmit model
Running pipeline stage MKMLizer
Starting job with name juvi21-kumiho-v1-rp-uwu-8b-v1-mkmlizer
Waiting for job on juvi21-kumiho-v1-rp-uwu-8b-v1-mkmlizer to finish
juvi21-kumiho-v1-rp-uwu-8b-v1-mkmlizer: ╔═════════════════════════════════════════════════════════════════════╗
juvi21-kumiho-v1-rp-uwu-8b-v1-mkmlizer: ║ _____ __ __ ║
juvi21-kumiho-v1-rp-uwu-8b-v1-mkmlizer: ║ / _/ /_ ___ __/ / ___ ___ / / ║
juvi21-kumiho-v1-rp-uwu-8b-v1-mkmlizer: ║ / _/ / // / |/|/ / _ \/ -_) -_) / ║
juvi21-kumiho-v1-rp-uwu-8b-v1-mkmlizer: ║ /_//_/\_, /|__,__/_//_/\__/\__/_/ ║
juvi21-kumiho-v1-rp-uwu-8b-v1-mkmlizer: ║ /___/ ║
juvi21-kumiho-v1-rp-uwu-8b-v1-mkmlizer: ║ ║
juvi21-kumiho-v1-rp-uwu-8b-v1-mkmlizer: ║ Version: 0.9.9 ║
juvi21-kumiho-v1-rp-uwu-8b-v1-mkmlizer: ║ Copyright 2023 MK ONE TECHNOLOGIES Inc. ║
juvi21-kumiho-v1-rp-uwu-8b-v1-mkmlizer: ║ https://mk1.ai ║
juvi21-kumiho-v1-rp-uwu-8b-v1-mkmlizer: ║ ║
juvi21-kumiho-v1-rp-uwu-8b-v1-mkmlizer: ║ The license key for the current software has been verified as ║
juvi21-kumiho-v1-rp-uwu-8b-v1-mkmlizer: ║ belonging to: ║
juvi21-kumiho-v1-rp-uwu-8b-v1-mkmlizer: ║ ║
juvi21-kumiho-v1-rp-uwu-8b-v1-mkmlizer: ║ Chai Research Corp. ║
juvi21-kumiho-v1-rp-uwu-8b-v1-mkmlizer: ║ Account ID: 7997a29f-0ceb-4cc7-9adf-840c57b4ae6f ║
juvi21-kumiho-v1-rp-uwu-8b-v1-mkmlizer: ║ Expiration: 2024-10-15 23:59:59 ║
juvi21-kumiho-v1-rp-uwu-8b-v1-mkmlizer: ║ ║
juvi21-kumiho-v1-rp-uwu-8b-v1-mkmlizer: ╚═════════════════════════════════════════════════════════════════════╝
juvi21-kumiho-v1-rp-uwu-8b-v1-mkmlizer: Downloaded to shared memory in 30.371s
juvi21-kumiho-v1-rp-uwu-8b-v1-mkmlizer: quantizing model to /dev/shm/model_cache, profile:s0, folder:/tmp/tmp1mho0bur, device:0
juvi21-kumiho-v1-rp-uwu-8b-v1-mkmlizer: Saving flywheel model at /dev/shm/model_cache
juvi21-kumiho-v1-rp-uwu-8b-v1-mkmlizer: quantized model in 26.130s
juvi21-kumiho-v1-rp-uwu-8b-v1-mkmlizer: Processed model juvi21/Kumiho-v1-rp-UwU-8B in 56.502s
juvi21-kumiho-v1-rp-uwu-8b-v1-mkmlizer: creating bucket guanaco-mkml-models
juvi21-kumiho-v1-rp-uwu-8b-v1-mkmlizer: Bucket 's3://guanaco-mkml-models/' created
juvi21-kumiho-v1-rp-uwu-8b-v1-mkmlizer: uploading /dev/shm/model_cache to s3://guanaco-mkml-models/juvi21-kumiho-v1-rp-uwu-8b-v1
juvi21-kumiho-v1-rp-uwu-8b-v1-mkmlizer: cp /dev/shm/model_cache/config.json s3://guanaco-mkml-models/juvi21-kumiho-v1-rp-uwu-8b-v1/config.json
juvi21-kumiho-v1-rp-uwu-8b-v1-mkmlizer: cp /dev/shm/model_cache/special_tokens_map.json s3://guanaco-mkml-models/juvi21-kumiho-v1-rp-uwu-8b-v1/special_tokens_map.json
juvi21-kumiho-v1-rp-uwu-8b-v1-mkmlizer: cp /dev/shm/model_cache/tokenizer_config.json s3://guanaco-mkml-models/juvi21-kumiho-v1-rp-uwu-8b-v1/tokenizer_config.json
juvi21-kumiho-v1-rp-uwu-8b-v1-mkmlizer: cp /dev/shm/model_cache/tokenizer.json s3://guanaco-mkml-models/juvi21-kumiho-v1-rp-uwu-8b-v1/tokenizer.json
juvi21-kumiho-v1-rp-uwu-8b-v1-mkmlizer: cp /dev/shm/model_cache/flywheel_model.0.safetensors s3://guanaco-mkml-models/juvi21-kumiho-v1-rp-uwu-8b-v1/flywheel_model.0.safetensors
juvi21-kumiho-v1-rp-uwu-8b-v1-mkmlizer: /opt/conda/lib/python3.10/site-packages/transformers/models/auto/tokenization_auto.py:785: FutureWarning: The `use_auth_token` argument is deprecated and will be removed in v5 of Transformers. Please use `token` instead.
juvi21-kumiho-v1-rp-uwu-8b-v1-mkmlizer: warnings.warn(
juvi21-kumiho-v1-rp-uwu-8b-v1-mkmlizer: /opt/conda/lib/python3.10/site-packages/transformers/models/auto/auto_factory.py:469: FutureWarning: The `use_auth_token` argument is deprecated and will be removed in v5 of Transformers. Please use `token` instead.
juvi21-kumiho-v1-rp-uwu-8b-v1-mkmlizer: warnings.warn(
juvi21-kumiho-v1-rp-uwu-8b-v1-mkmlizer: Downloading shards: 0%| | 0/2 [00:00<?, ?it/s] Downloading shards: 50%|█████ | 1/2 [00:05<00:05, 5.43s/it] Downloading shards: 100%|██████████| 2/2 [00:08<00:00, 4.12s/it] Downloading shards: 100%|██████████| 2/2 [00:08<00:00, 4.31s/it]
juvi21-kumiho-v1-rp-uwu-8b-v1-mkmlizer: Loading checkpoint shards: 0%| | 0/2 [00:00<?, ?it/s] Loading checkpoint shards: 50%|█████ | 1/2 [00:00<00:00, 2.41it/s] Loading checkpoint shards: 100%|██████████| 2/2 [00:00<00:00, 3.95it/s] Loading checkpoint shards: 100%|██████████| 2/2 [00:00<00:00, 3.60it/s]
juvi21-kumiho-v1-rp-uwu-8b-v1-mkmlizer: Saving model to /tmp/reward_cache/reward.tensors
juvi21-kumiho-v1-rp-uwu-8b-v1-mkmlizer: Saving duration: 1.383s
juvi21-kumiho-v1-rp-uwu-8b-v1-mkmlizer: Processed model ChaiML/gpt2_xl_pairwise_89m_step_347634 in 13.780s
juvi21-kumiho-v1-rp-uwu-8b-v1-mkmlizer: creating bucket guanaco-reward-models
juvi21-kumiho-v1-rp-uwu-8b-v1-mkmlizer: Bucket 's3://guanaco-reward-models/' created
juvi21-kumiho-v1-rp-uwu-8b-v1-mkmlizer: uploading /tmp/reward_cache to s3://guanaco-reward-models/juvi21-kumiho-v1-rp-uwu-8b-v1_reward
juvi21-kumiho-v1-rp-uwu-8b-v1-mkmlizer: cp /tmp/reward_cache/config.json s3://guanaco-reward-models/juvi21-kumiho-v1-rp-uwu-8b-v1_reward/config.json
juvi21-kumiho-v1-rp-uwu-8b-v1-mkmlizer: cp /tmp/reward_cache/special_tokens_map.json s3://guanaco-reward-models/juvi21-kumiho-v1-rp-uwu-8b-v1_reward/special_tokens_map.json
juvi21-kumiho-v1-rp-uwu-8b-v1-mkmlizer: cp /tmp/reward_cache/tokenizer_config.json s3://guanaco-reward-models/juvi21-kumiho-v1-rp-uwu-8b-v1_reward/tokenizer_config.json
juvi21-kumiho-v1-rp-uwu-8b-v1-mkmlizer: cp /tmp/reward_cache/merges.txt s3://guanaco-reward-models/juvi21-kumiho-v1-rp-uwu-8b-v1_reward/merges.txt
juvi21-kumiho-v1-rp-uwu-8b-v1-mkmlizer: cp /tmp/reward_cache/vocab.json s3://guanaco-reward-models/juvi21-kumiho-v1-rp-uwu-8b-v1_reward/vocab.json
juvi21-kumiho-v1-rp-uwu-8b-v1-mkmlizer: cp /tmp/reward_cache/tokenizer.json s3://guanaco-reward-models/juvi21-kumiho-v1-rp-uwu-8b-v1_reward/tokenizer.json
juvi21-kumiho-v1-rp-uwu-8b-v1-mkmlizer: cp /tmp/reward_cache/reward.tensors s3://guanaco-reward-models/juvi21-kumiho-v1-rp-uwu-8b-v1_reward/reward.tensors
Job juvi21-kumiho-v1-rp-uwu-8b-v1-mkmlizer completed after 104.81s with status: succeeded
Stopping job with name juvi21-kumiho-v1-rp-uwu-8b-v1-mkmlizer
Pipeline stage MKMLizer completed in 105.92s
Running pipeline stage MKMLKubeTemplater
Pipeline stage MKMLKubeTemplater completed in 0.09s
Running pipeline stage ISVCDeployer
Creating inference service juvi21-kumiho-v1-rp-uwu-8b-v1
Waiting for inference service juvi21-kumiho-v1-rp-uwu-8b-v1 to be ready
Inference service juvi21-kumiho-v1-rp-uwu-8b-v1 ready after 160.97401094436646s
Pipeline stage ISVCDeployer completed in 163.26s
Running pipeline stage StressChecker
Received healthy response to inference request in 2.218636989593506s
Received healthy response to inference request in 1.283423900604248s
Received healthy response to inference request in 1.2647368907928467s
Received healthy response to inference request in 1.3188481330871582s
Received healthy response to inference request in 1.2785077095031738s
5 requests
0 failed requests
5th percentile: 1.267491054534912
10th percentile: 1.2702452182769775
20th percentile: 1.2757535457611084
30th percentile: 1.2794909477233887
40th percentile: 1.2814574241638184
50th percentile: 1.283423900604248
60th percentile: 1.297593593597412
70th percentile: 1.3117632865905762
80th percentile: 1.4988059043884279
90th percentile: 1.858721446990967
95th percentile: 2.038679218292236
99th percentile: 2.182645435333252
mean time: 1.4728307247161865
Pipeline stage StressChecker completed in 8.04s
juvi21-kumiho-v1-rp-uwu-8b_v1 status is now deployed due to DeploymentManager action
juvi21-kumiho-v1-rp-uwu-8b_v1 status is now inactive due to auto deactivation removed underperforming models
juvi21-kumiho-v1-rp-uwu-8b_v1 status is now torndown due to DeploymentManager action

Usage Metrics

Latency Metrics