submission_id: jellywibble-lora-120k-pr_7199_v2
developer_uid: Jellywibble
status: inactive
model_repo: Jellywibble/lora_120k_pref_data_ep3_stacked_elo_only
reward_repo: ChaiML/gpt2_xl_pairwise_89m_step_347634
generation_params: {'temperature': 0.95, 'top_p': 1.0, 'min_p': 0.08, 'top_k': 40, 'presence_penalty': 0.0, 'frequency_penalty': 0.0, 'stopping_words': ['\n', '<|eot_id|>'], 'max_input_tokens': 512, 'best_of': 16, 'max_output_tokens': 64}
formatter: {'memory_template': "<|begin_of_text|><|start_header_id|>system<|end_header_id|>\n\n{bot_name}'s Persona: {memory}\n\n", 'prompt_template': '{prompt}<|eot_id|>', 'bot_template': '<|start_header_id|>assistant<|end_header_id|>\n\n{bot_name}: {message}<|eot_id|>', 'user_template': '<|start_header_id|>user<|end_header_id|>\n\n{user_name}: {message}<|eot_id|>', 'response_template': '<|start_header_id|>assistant<|end_header_id|>\n\n{bot_name}:', 'truncate_by_message': False}
reward_formatter: {'memory_template': "{bot_name}'s Persona: {memory}\n####\n", 'prompt_template': '{prompt}\n<START>\n', 'bot_template': '{bot_name}: {message}\n', 'user_template': '{user_name}: {message}\n', 'response_template': '{bot_name}:', 'truncate_by_message': False}
timestamp: 2024-07-08T07:01:42+00:00
model_name: nitral-ai-hathor-l3-8b-v-01_v1
model_group: Jellywibble/lora_120k_pr
num_battles: 33030
num_wins: 20001
celo_rating: 1275.48
propriety_score: 0.7133746678476528
propriety_total_count: 5645.0
submission_type: basic
model_architecture: LlamaForCausalLM
model_num_parameters: 8030261248.0
best_of: 16
max_input_tokens: 512
max_output_tokens: 64
display_name: nitral-ai-hathor-l3-8b-v-01_v1
ineligible_reason: None
language_model: Jellywibble/lora_120k_pref_data_ep3_stacked_elo_only
model_size: 8B
reward_model: ChaiML/gpt2_xl_pairwise_89m_step_347634
us_pacific_date: 2024-07-08
win_ratio: 0.6055404178019982
preference_data_url: None
Resubmit model
Running pipeline stage MKMLizer
Starting job with name jellywibble-lora-120k-pr-7199-v2-mkmlizer
Waiting for job on jellywibble-lora-120k-pr-7199-v2-mkmlizer to finish
jellywibble-lora-120k-pr-7199-v2-mkmlizer: ╔═════════════════════════════════════════════════════════════════════╗
jellywibble-lora-120k-pr-7199-v2-mkmlizer: ║ _____ __ __ ║
jellywibble-lora-120k-pr-7199-v2-mkmlizer: ║ / _/ /_ ___ __/ / ___ ___ / / ║
jellywibble-lora-120k-pr-7199-v2-mkmlizer: ║ / _/ / // / |/|/ / _ \/ -_) -_) / ║
jellywibble-lora-120k-pr-7199-v2-mkmlizer: ║ /_//_/\_, /|__,__/_//_/\__/\__/_/ ║
jellywibble-lora-120k-pr-7199-v2-mkmlizer: ║ /___/ ║
jellywibble-lora-120k-pr-7199-v2-mkmlizer: ║ ║
jellywibble-lora-120k-pr-7199-v2-mkmlizer: ║ Version: 0.8.14 ║
jellywibble-lora-120k-pr-7199-v2-mkmlizer: ║ Copyright 2023 MK ONE TECHNOLOGIES Inc. ║
jellywibble-lora-120k-pr-7199-v2-mkmlizer: ║ https://mk1.ai ║
jellywibble-lora-120k-pr-7199-v2-mkmlizer: ║ ║
jellywibble-lora-120k-pr-7199-v2-mkmlizer: ║ The license key for the current software has been verified as ║
jellywibble-lora-120k-pr-7199-v2-mkmlizer: ║ belonging to: ║
jellywibble-lora-120k-pr-7199-v2-mkmlizer: ║ ║
jellywibble-lora-120k-pr-7199-v2-mkmlizer: ║ Chai Research Corp. ║
jellywibble-lora-120k-pr-7199-v2-mkmlizer: ║ Account ID: 7997a29f-0ceb-4cc7-9adf-840c57b4ae6f ║
jellywibble-lora-120k-pr-7199-v2-mkmlizer: ║ Expiration: 2024-07-15 23:59:59 ║
jellywibble-lora-120k-pr-7199-v2-mkmlizer: ║ ║
jellywibble-lora-120k-pr-7199-v2-mkmlizer: ╚═════════════════════════════════════════════════════════════════════╝
jellywibble-lora-120k-pr-7199-v2-mkmlizer: Downloaded to shared memory in 31.128s
jellywibble-lora-120k-pr-7199-v2-mkmlizer: quantizing model to /dev/shm/model_cache
jellywibble-lora-120k-pr-7199-v2-mkmlizer: Saving flywheel model at /dev/shm/model_cache
jellywibble-lora-120k-pr-7199-v2-mkmlizer: Loading 0: 0%| | 0/291 [00:00<?, ?it/s] Loading 0: 2%|▏ | 5/291 [00:00<00:07, 38.41it/s] Loading 0: 5%|▍ | 14/291 [00:00<00:04, 55.51it/s] Loading 0: 8%|▊ | 23/291 [00:00<00:04, 60.54it/s] Loading 0: 11%|█▏ | 33/291 [00:00<00:05, 46.40it/s] Loading 0: 13%|█▎ | 39/291 [00:00<00:06, 40.69it/s] Loading 0: 15%|█▌ | 44/291 [00:00<00:05, 42.03it/s] Loading 0: 17%|█▋ | 50/291 [00:01<00:05, 44.92it/s] Loading 0: 20%|██ | 59/291 [00:01<00:04, 52.17it/s] Loading 0: 23%|██▎ | 67/291 [00:01<00:03, 57.99it/s] Loading 0: 25%|██▌ | 74/291 [00:01<00:03, 56.17it/s] Loading 0: 27%|██▋ | 80/291 [00:01<00:05, 37.87it/s] Loading 0: 29%|██▉ | 85/291 [00:01<00:05, 38.67it/s] Loading 0: 31%|███▏ | 91/291 [00:01<00:04, 42.93it/s] Loading 0: 34%|███▍ | 100/291 [00:02<00:03, 50.34it/s] Loading 0: 37%|███▋ | 108/291 [00:02<00:03, 56.42it/s] Loading 0: 40%|███▉ | 115/291 [00:02<00:02, 58.86it/s] Loading 0: 43%|████▎ | 125/291 [00:02<00:02, 62.29it/s] Loading 0: 45%|████▌ | 132/291 [00:02<00:02, 59.48it/s] Loading 0: 48%|████▊ | 139/291 [00:03<00:04, 34.93it/s] Loading 0: 51%|█████ | 147/291 [00:03<00:03, 39.91it/s] Loading 0: 53%|█████▎ | 153/291 [00:03<00:03, 42.66it/s] Loading 0: 55%|█████▍ | 159/291 [00:03<00:02, 45.32it/s] Loading 0: 57%|█████▋ | 167/291 [00:03<00:02, 49.58it/s] Loading 0: 60%|██████ | 176/291 [00:03<00:02, 54.56it/s] Loading 0: 64%|██████▍ | 186/291 [00:03<00:02, 51.57it/s] Loading 0: 66%|██████▌ | 192/291 [00:04<00:02, 44.49it/s] Loading 0: 68%|██████▊ | 197/291 [00:04<00:02, 44.39it/s] Loading 0: 69%|██████▉ | 202/291 [00:04<00:01, 45.16it/s] Loading 0: 72%|███████▏ | 210/291 [00:04<00:01, 49.75it/s] Loading 0: 75%|███████▍ | 217/291 [00:04<00:01, 54.38it/s] Loading 0: 77%|███████▋ | 223/291 [00:04<00:01, 54.11it/s] Loading 0: 79%|███████▉ | 230/291 [00:04<00:01, 57.30it/s] Loading 0: 81%|████████ | 236/291 [00:04<00:01, 45.42it/s] Loading 0: 83%|████████▎ | 241/291 [00:05<00:01, 41.57it/s] Loading 0: 85%|████████▍ | 247/291 [00:05<00:01, 42.86it/s] Loading 0: 88%|████████▊ | 255/291 [00:05<00:00, 47.79it/s] Loading 0: 89%|████████▉ | 260/291 [00:05<00:00, 46.67it/s] Loading 0: 91%|█████████▏| 266/291 [00:05<00:00, 46.81it/s] Loading 0: 95%|█████████▍| 275/291 [00:05<00:00, 53.23it/s] Loading 0: 98%|█████████▊| 286/291 [00:11<00:01, 4.54it/s] Special tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained.
jellywibble-lora-120k-pr-7199-v2-mkmlizer: quantized model in 24.057s
jellywibble-lora-120k-pr-7199-v2-mkmlizer: Processed model Jellywibble/lora_120k_pref_data_ep3_stacked_elo_only in 55.186s
jellywibble-lora-120k-pr-7199-v2-mkmlizer: creating bucket guanaco-mkml-models
jellywibble-lora-120k-pr-7199-v2-mkmlizer: Bucket 's3://guanaco-mkml-models/' created
jellywibble-lora-120k-pr-7199-v2-mkmlizer: uploading /dev/shm/model_cache to s3://guanaco-mkml-models/jellywibble-lora-120k-pr-7199-v2
jellywibble-lora-120k-pr-7199-v2-mkmlizer: cp /dev/shm/model_cache/config.json s3://guanaco-mkml-models/jellywibble-lora-120k-pr-7199-v2/config.json
jellywibble-lora-120k-pr-7199-v2-mkmlizer: cp /dev/shm/model_cache/tokenizer_config.json s3://guanaco-mkml-models/jellywibble-lora-120k-pr-7199-v2/tokenizer_config.json
jellywibble-lora-120k-pr-7199-v2-mkmlizer: cp /dev/shm/model_cache/special_tokens_map.json s3://guanaco-mkml-models/jellywibble-lora-120k-pr-7199-v2/special_tokens_map.json
jellywibble-lora-120k-pr-7199-v2-mkmlizer: cp /dev/shm/model_cache/tokenizer.json s3://guanaco-mkml-models/jellywibble-lora-120k-pr-7199-v2/tokenizer.json
jellywibble-lora-120k-pr-7199-v2-mkmlizer: cp /dev/shm/model_cache/flywheel_model.0.safetensors s3://guanaco-mkml-models/jellywibble-lora-120k-pr-7199-v2/flywheel_model.0.safetensors
jellywibble-lora-120k-pr-7199-v2-mkmlizer: /opt/conda/lib/python3.10/site-packages/transformers/models/auto/configuration_auto.py:919: FutureWarning: The `use_auth_token` argument is deprecated and will be removed in v5 of Transformers. Please use `token` instead.
jellywibble-lora-120k-pr-7199-v2-mkmlizer: warnings.warn(
jellywibble-lora-120k-pr-7199-v2-mkmlizer: loading reward model from ChaiML/gpt2_xl_pairwise_89m_step_347634
jellywibble-lora-120k-pr-7199-v2-mkmlizer: /opt/conda/lib/python3.10/site-packages/huggingface_hub/file_download.py:1132: FutureWarning: `resume_download` is deprecated and will be removed in version 1.0.0. Downloads always resume when possible. If you want to force a new download, use `force_download=True`.
jellywibble-lora-120k-pr-7199-v2-mkmlizer: warnings.warn(
jellywibble-lora-120k-pr-7199-v2-mkmlizer: /opt/conda/lib/python3.10/site-packages/transformers/models/auto/tokenization_auto.py:769: FutureWarning: The `use_auth_token` argument is deprecated and will be removed in v5 of Transformers. Please use `token` instead.
jellywibble-lora-120k-pr-7199-v2-mkmlizer: warnings.warn(
jellywibble-lora-120k-pr-7199-v2-mkmlizer: /opt/conda/lib/python3.10/site-packages/transformers/models/auto/auto_factory.py:468: FutureWarning: The `use_auth_token` argument is deprecated and will be removed in v5 of Transformers. Please use `token` instead.
jellywibble-lora-120k-pr-7199-v2-mkmlizer: warnings.warn(
jellywibble-lora-120k-pr-7199-v2-mkmlizer: Downloading shards: 0%| | 0/2 [00:00<?, ?it/s] Downloading shards: 50%|█████ | 1/2 [00:04<00:04, 4.50s/it] Downloading shards: 100%|██████████| 2/2 [00:05<00:00, 2.70s/it] Downloading shards: 100%|██████████| 2/2 [00:05<00:00, 2.97s/it]
Job jellywibble-lora-120k-pr-7199-v2-mkmlizer completed after 119.72s with status: succeeded
Stopping job with name jellywibble-lora-120k-pr-7199-v2-mkmlizer
Pipeline stage MKMLizer completed in 120.62s
Running pipeline stage MKMLKubeTemplater
Pipeline stage MKMLKubeTemplater completed in 0.10s
Running pipeline stage ISVCDeployer
Creating inference service jellywibble-lora-120k-pr-7199-v2
Waiting for inference service jellywibble-lora-120k-pr-7199-v2 to be ready
Inference service jellywibble-lora-120k-pr-7199-v2 ready after 60.67513728141785s
Pipeline stage ISVCDeployer completed in 67.70s
Running pipeline stage StressChecker
Received healthy response to inference request in 2.416836977005005s
Received healthy response to inference request in 1.6287627220153809s
Received healthy response to inference request in 1.6180944442749023s
Received healthy response to inference request in 1.6001698970794678s
Received healthy response to inference request in 1.654573917388916s
5 requests
0 failed requests
5th percentile: 1.6037548065185547
10th percentile: 1.6073397159576417
20th percentile: 1.6145095348358154
30th percentile: 1.620228099822998
40th percentile: 1.6244954109191894
50th percentile: 1.6287627220153809
60th percentile: 1.639087200164795
70th percentile: 1.649411678314209
80th percentile: 1.807026529312134
90th percentile: 2.1119317531585695
95th percentile: 2.2643843650817868
99th percentile: 2.3863464546203614
mean time: 1.7836875915527344
Pipeline stage StressChecker completed in 9.58s
jellywibble-lora-120k-pr_7199_v2 status is now deployed due to DeploymentManager action
jellywibble-lora-120k-pr_7199_v2 status is now inactive due to auto deactivation removed underperforming models

Usage Metrics

Latency Metrics