submission_id: jellywibble-lora-120k-pr_2827_v1
developer_uid: Jellywibble
status: inactive
model_repo: Jellywibble/lora_120k_pref_data_ep2
reward_repo: ChaiML/reward_gpt2_medium_preference_24m_e2
generation_params: {'temperature': 0.95, 'top_p': 1.0, 'min_p': 0.08, 'top_k': 40, 'presence_penalty': 0.0, 'frequency_penalty': 0.0, 'stopping_words': ['\n', '<|eot_id|>'], 'max_input_tokens': 512, 'best_of': 16, 'max_output_tokens': 64}
formatter: {'memory_template': "<|begin_of_text|><|start_header_id|>system<|end_header_id|>\n\n{bot_name}'s Persona: {memory}\n\n", 'prompt_template': '{prompt}<|eot_id|>', 'bot_template': '<|start_header_id|>assistant<|end_header_id|>\n\n{bot_name}: {message}<|eot_id|>', 'user_template': '<|start_header_id|>user<|end_header_id|>\n\n{user_name}: {message}<|eot_id|>', 'response_template': '<|start_header_id|>assistant<|end_header_id|>\n\n{bot_name}:', 'truncate_by_message': False}
reward_formatter: {'memory_template': "{bot_name}'s Persona: {memory}\n####\n", 'prompt_template': '{prompt}\n<START>\n', 'bot_template': '{bot_name}: {message}\n', 'user_template': '{user_name}: {message}\n', 'response_template': '{bot_name}:', 'truncate_by_message': False}
timestamp: 2024-07-04T03:53:48+00:00
model_name: nitral-ai-hathor-l3-8b-v-01_v1
model_group: Jellywibble/lora_120k_pr
num_battles: 12653
num_wins: 7097
celo_rating: 1233.52
propriety_score: 0.7036540395574925
propriety_total_count: 5966.0
submission_type: basic
model_architecture: LlamaForCausalLM
model_num_parameters: 8030261248.0
best_of: 16
max_input_tokens: 512
max_output_tokens: 64
display_name: nitral-ai-hathor-l3-8b-v-01_v1
ineligible_reason: None
language_model: Jellywibble/lora_120k_pref_data_ep2
model_size: 8B
reward_model: ChaiML/reward_gpt2_medium_preference_24m_e2
us_pacific_date: 2024-07-03
win_ratio: 0.5608946494902395
Resubmit model
Running pipeline stage MKMLizer
Starting job with name jellywibble-lora-120k-pr-2827-v1-mkmlizer
Waiting for job on jellywibble-lora-120k-pr-2827-v1-mkmlizer to finish
jellywibble-lora-120k-pr-2827-v1-mkmlizer: ╔═════════════════════════════════════════════════════════════════════╗
jellywibble-lora-120k-pr-2827-v1-mkmlizer: ║ _____ __ __ ║
jellywibble-lora-120k-pr-2827-v1-mkmlizer: ║ / _/ /_ ___ __/ / ___ ___ / / ║
jellywibble-lora-120k-pr-2827-v1-mkmlizer: ║ / _/ / // / |/|/ / _ \/ -_) -_) / ║
jellywibble-lora-120k-pr-2827-v1-mkmlizer: ║ /_//_/\_, /|__,__/_//_/\__/\__/_/ ║
jellywibble-lora-120k-pr-2827-v1-mkmlizer: ║ /___/ ║
jellywibble-lora-120k-pr-2827-v1-mkmlizer: ║ ║
jellywibble-lora-120k-pr-2827-v1-mkmlizer: ║ Version: 0.8.14 ║
jellywibble-lora-120k-pr-2827-v1-mkmlizer: ║ Copyright 2023 MK ONE TECHNOLOGIES Inc. ║
jellywibble-lora-120k-pr-2827-v1-mkmlizer: ║ https://mk1.ai ║
jellywibble-lora-120k-pr-2827-v1-mkmlizer: ║ ║
jellywibble-lora-120k-pr-2827-v1-mkmlizer: ║ The license key for the current software has been verified as ║
jellywibble-lora-120k-pr-2827-v1-mkmlizer: ║ belonging to: ║
jellywibble-lora-120k-pr-2827-v1-mkmlizer: ║ ║
jellywibble-lora-120k-pr-2827-v1-mkmlizer: ║ Chai Research Corp. ║
jellywibble-lora-120k-pr-2827-v1-mkmlizer: ║ Account ID: 7997a29f-0ceb-4cc7-9adf-840c57b4ae6f ║
jellywibble-lora-120k-pr-2827-v1-mkmlizer: ║ Expiration: 2024-07-15 23:59:59 ║
jellywibble-lora-120k-pr-2827-v1-mkmlizer: ║ ║
jellywibble-lora-120k-pr-2827-v1-mkmlizer: ╚═════════════════════════════════════════════════════════════════════╝
jellywibble-lora-120k-pr-2827-v1-mkmlizer: Downloaded to shared memory in 120.662s
jellywibble-lora-120k-pr-2827-v1-mkmlizer: quantizing model to /dev/shm/model_cache
jellywibble-lora-120k-pr-2827-v1-mkmlizer: Saving flywheel model at /dev/shm/model_cache
jellywibble-lora-120k-pr-2827-v1-mkmlizer: Loading 0: 0%| | 0/291 [00:00<?, ?it/s] Loading 0: 3%|▎ | 8/291 [00:00<00:03, 74.66it/s] Loading 0: 5%|▌ | 16/291 [00:00<00:03, 74.92it/s] Loading 0: 8%|▊ | 24/291 [00:00<00:03, 73.12it/s] Loading 0: 11%|█▏ | 33/291 [00:00<00:06, 41.37it/s] Loading 0: 13%|█▎ | 39/291 [00:00<00:05, 42.66it/s] Loading 0: 16%|█▋ | 48/291 [00:00<00:04, 51.33it/s] Loading 0: 20%|█▉ | 57/291 [00:01<00:04, 57.93it/s] Loading 0: 23%|██▎ | 66/291 [00:01<00:03, 62.97it/s] Loading 0: 25%|██▌ | 74/291 [00:01<00:03, 65.29it/s] Loading 0: 28%|██▊ | 81/291 [00:01<00:05, 38.97it/s] Loading 0: 31%|███ | 90/291 [00:01<00:04, 46.38it/s] Loading 0: 34%|███▍ | 99/291 [00:01<00:03, 52.92it/s] Loading 0: 37%|███▋ | 108/291 [00:01<00:03, 58.62it/s] Loading 0: 40%|███▉ | 116/291 [00:02<00:02, 61.06it/s] Loading 0: 43%|████▎ | 126/291 [00:02<00:02, 67.07it/s] Loading 0: 46%|████▌ | 134/291 [00:02<00:03, 41.13it/s] Loading 0: 48%|████▊ | 140/291 [00:02<00:03, 42.30it/s] Loading 0: 51%|█████ | 149/291 [00:02<00:02, 49.77it/s] Loading 0: 54%|█████▍ | 158/291 [00:02<00:02, 55.89it/s] Loading 0: 57%|█████▋ | 167/291 [00:03<00:02, 60.95it/s] Loading 0: 60%|██████ | 176/291 [00:03<00:01, 65.58it/s] Loading 0: 64%|██████▍ | 186/291 [00:03<00:02, 45.52it/s] Loading 0: 66%|██████▌ | 192/291 [00:03<00:02, 46.41it/s] Loading 0: 69%|██████▉ | 201/291 [00:03<00:01, 53.55it/s] Loading 0: 72%|███████▏ | 210/291 [00:03<00:01, 59.89it/s] Loading 0: 75%|███████▌ | 219/291 [00:04<00:01, 65.11it/s] Loading 0: 78%|███████▊ | 227/291 [00:04<00:00, 66.72it/s] Loading 0: 81%|████████ | 235/291 [00:04<00:01, 42.71it/s] Loading 0: 83%|████████▎ | 242/291 [00:04<00:01, 47.18it/s] Loading 0: 85%|████████▌ | 248/291 [00:04<00:00, 49.40it/s] Loading 0: 88%|████████▊ | 257/291 [00:04<00:00, 56.53it/s] Loading 0: 91%|█████████▏| 266/291 [00:04<00:00, 60.81it/s] Loading 0: 94%|█████████▍| 274/291 [00:05<00:00, 65.14it/s] Loading 0: 98%|█████████▊| 284/291 [00:05<00:00, 73.14it/s] Special tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained.
jellywibble-lora-120k-pr-2827-v1-mkmlizer: quantized model in 26.153s
jellywibble-lora-120k-pr-2827-v1-mkmlizer: Processed model Jellywibble/lora_120k_pref_data_ep2 in 146.815s
jellywibble-lora-120k-pr-2827-v1-mkmlizer: creating bucket guanaco-mkml-models
jellywibble-lora-120k-pr-2827-v1-mkmlizer: Bucket 's3://guanaco-mkml-models/' created
jellywibble-lora-120k-pr-2827-v1-mkmlizer: uploading /dev/shm/model_cache to s3://guanaco-mkml-models/jellywibble-lora-120k-pr-2827-v1
jellywibble-lora-120k-pr-2827-v1-mkmlizer: cp /dev/shm/model_cache/special_tokens_map.json s3://guanaco-mkml-models/jellywibble-lora-120k-pr-2827-v1/special_tokens_map.json
jellywibble-lora-120k-pr-2827-v1-mkmlizer: cp /dev/shm/model_cache/tokenizer_config.json s3://guanaco-mkml-models/jellywibble-lora-120k-pr-2827-v1/tokenizer_config.json
jellywibble-lora-120k-pr-2827-v1-mkmlizer: cp /dev/shm/model_cache/config.json s3://guanaco-mkml-models/jellywibble-lora-120k-pr-2827-v1/config.json
jellywibble-lora-120k-pr-2827-v1-mkmlizer: cp /dev/shm/model_cache/tokenizer.json s3://guanaco-mkml-models/jellywibble-lora-120k-pr-2827-v1/tokenizer.json
jellywibble-lora-120k-pr-2827-v1-mkmlizer: cp /dev/shm/model_cache/flywheel_model.0.safetensors s3://guanaco-mkml-models/jellywibble-lora-120k-pr-2827-v1/flywheel_model.0.safetensors
jellywibble-lora-120k-pr-2827-v1-mkmlizer: loading reward model from ChaiML/reward_gpt2_medium_preference_24m_e2
jellywibble-lora-120k-pr-2827-v1-mkmlizer: /opt/conda/lib/python3.10/site-packages/transformers/models/auto/configuration_auto.py:919: FutureWarning: The `use_auth_token` argument is deprecated and will be removed in v5 of Transformers. Please use `token` instead.
jellywibble-lora-120k-pr-2827-v1-mkmlizer: warnings.warn(
jellywibble-lora-120k-pr-2827-v1-mkmlizer: /opt/conda/lib/python3.10/site-packages/huggingface_hub/file_download.py:1132: FutureWarning: `resume_download` is deprecated and will be removed in version 1.0.0. Downloads always resume when possible. If you want to force a new download, use `force_download=True`.
jellywibble-lora-120k-pr-2827-v1-mkmlizer: warnings.warn(
jellywibble-lora-120k-pr-2827-v1-mkmlizer: /opt/conda/lib/python3.10/site-packages/transformers/models/auto/tokenization_auto.py:769: FutureWarning: The `use_auth_token` argument is deprecated and will be removed in v5 of Transformers. Please use `token` instead.
jellywibble-lora-120k-pr-2827-v1-mkmlizer: warnings.warn(
jellywibble-lora-120k-pr-2827-v1-mkmlizer: /opt/conda/lib/python3.10/site-packages/transformers/models/auto/auto_factory.py:468: FutureWarning: The `use_auth_token` argument is deprecated and will be removed in v5 of Transformers. Please use `token` instead.
jellywibble-lora-120k-pr-2827-v1-mkmlizer: warnings.warn(
Failed to get response for submission trace2333-joint-filtered_3791_v1: ('http://trace2333-joint-filtered-3791-v1-predictor-default.tenant-chaiml-guanaco.knative.ord1.coreweave.cloud/v1/models/GPT-J-6B-lit-v2:predict', 'upstream connect error or disconnect/reset before headers. reset reason: connection failure')
jellywibble-lora-120k-pr-2827-v1-mkmlizer: /opt/conda/lib/python3.10/site-packages/torch/_utils.py:831: UserWarning: TypedStorage is deprecated. It will be removed in the future and UntypedStorage will be the only storage class. This should only matter to you if you are using storages directly. To access UntypedStorage directly, use tensor.untyped_storage() instead of tensor.storage()
jellywibble-lora-120k-pr-2827-v1-mkmlizer: return self.fget.__get__(instance, owner)()
jellywibble-lora-120k-pr-2827-v1-mkmlizer: Saving model to /tmp/reward_cache/reward.tensors
jellywibble-lora-120k-pr-2827-v1-mkmlizer: Saving duration: 0.435s
jellywibble-lora-120k-pr-2827-v1-mkmlizer: Processed model ChaiML/reward_gpt2_medium_preference_24m_e2 in 12.554s
jellywibble-lora-120k-pr-2827-v1-mkmlizer: creating bucket guanaco-reward-models
jellywibble-lora-120k-pr-2827-v1-mkmlizer: Bucket 's3://guanaco-reward-models/' created
jellywibble-lora-120k-pr-2827-v1-mkmlizer: uploading /tmp/reward_cache to s3://guanaco-reward-models/jellywibble-lora-120k-pr-2827-v1_reward
jellywibble-lora-120k-pr-2827-v1-mkmlizer: cp /tmp/reward_cache/special_tokens_map.json s3://guanaco-reward-models/jellywibble-lora-120k-pr-2827-v1_reward/special_tokens_map.json
jellywibble-lora-120k-pr-2827-v1-mkmlizer: cp /tmp/reward_cache/tokenizer_config.json s3://guanaco-reward-models/jellywibble-lora-120k-pr-2827-v1_reward/tokenizer_config.json
jellywibble-lora-120k-pr-2827-v1-mkmlizer: cp /tmp/reward_cache/config.json s3://guanaco-reward-models/jellywibble-lora-120k-pr-2827-v1_reward/config.json
jellywibble-lora-120k-pr-2827-v1-mkmlizer: cp /tmp/reward_cache/merges.txt s3://guanaco-reward-models/jellywibble-lora-120k-pr-2827-v1_reward/merges.txt
jellywibble-lora-120k-pr-2827-v1-mkmlizer: cp /tmp/reward_cache/tokenizer.json s3://guanaco-reward-models/jellywibble-lora-120k-pr-2827-v1_reward/tokenizer.json
jellywibble-lora-120k-pr-2827-v1-mkmlizer: cp /tmp/reward_cache/vocab.json s3://guanaco-reward-models/jellywibble-lora-120k-pr-2827-v1_reward/vocab.json
jellywibble-lora-120k-pr-2827-v1-mkmlizer: cp /tmp/reward_cache/reward.tensors s3://guanaco-reward-models/jellywibble-lora-120k-pr-2827-v1_reward/reward.tensors
Job jellywibble-lora-120k-pr-2827-v1-mkmlizer completed after 196.1s with status: succeeded
Stopping job with name jellywibble-lora-120k-pr-2827-v1-mkmlizer
Pipeline stage MKMLizer completed in 196.96s
Running pipeline stage MKMLKubeTemplater
Pipeline stage MKMLKubeTemplater completed in 0.09s
Running pipeline stage ISVCDeployer
Creating inference service jellywibble-lora-120k-pr-2827-v1
Waiting for inference service jellywibble-lora-120k-pr-2827-v1 to be ready
Inference service jellywibble-lora-120k-pr-2827-v1 ready after 40.21506214141846s
Pipeline stage ISVCDeployer completed in 46.90s
Running pipeline stage StressChecker
Received healthy response to inference request in 2.174496650695801s
Received healthy response to inference request in 1.347921371459961s
Received healthy response to inference request in 1.3108124732971191s
Received healthy response to inference request in 1.2771615982055664s
Received healthy response to inference request in 1.3581912517547607s
5 requests
0 failed requests
5th percentile: 1.283891773223877
10th percentile: 1.2906219482421875
20th percentile: 1.3040822982788085
30th percentile: 1.3182342529296875
40th percentile: 1.3330778121948241
50th percentile: 1.347921371459961
60th percentile: 1.3520293235778809
70th percentile: 1.3561372756958008
80th percentile: 1.5214523315429689
90th percentile: 1.847974491119385
95th percentile: 2.0112355709075924
99th percentile: 2.141844434738159
mean time: 1.4937166690826416
Pipeline stage StressChecker completed in 8.11s
jellywibble-lora-120k-pr_2827_v1 status is now deployed due to DeploymentManager action
jellywibble-lora-120k-pr_2827_v1 status is now inactive due to auto deactivation removed underperforming models

Usage Metrics

Latency Metrics