submission_id: hf-100-llama-3-spellboun_6094_v1
developer_uid: hf-100
status: inactive
model_repo: hf-100/Llama-3-Spellbound-Instruct-8B-0.3
reward_repo: ChaiML/reward_gpt2_medium_preference_24m_e2
generation_params: {'temperature': 1.0, 'top_p': 1.0, 'min_p': 0.0, 'top_k': 40, 'presence_penalty': 0.0, 'frequency_penalty': 0.0, 'stopping_words': ['\n'], 'max_input_tokens': 512, 'best_of': 4, 'max_output_tokens': 64}
formatter: {'memory_template': "{bot_name}'s Persona: {memory}\n####\n", 'prompt_template': '{prompt}\n<START>\n', 'bot_template': '{bot_name}: {message}\n', 'user_template': '{user_name}: {message}\n', 'response_template': '{bot_name}:', 'truncate_by_message': False}
reward_formatter: {'memory_template': "{bot_name}'s Persona: {memory}\n####\n", 'prompt_template': '{prompt}\n<START>\n', 'bot_template': '{bot_name}: {message}\n', 'user_template': '{user_name}: {message}\n', 'response_template': '{bot_name}:', 'truncate_by_message': False}
timestamp: 2024-07-09T01:13:28+00:00
model_name: hf-100-llama-3-spellboun_6094_v1
model_group: hf-100/Llama-3-Spellboun
num_battles: 31263
num_wins: 13432
celo_rating: 1146.59
alignment_score: None
alignment_samples: 0
propriety_score: 0.7070087376835843
propriety_total_count: 5379.0
submission_type: basic
model_architecture: LlamaForCausalLM
model_num_parameters: 8030261248.0
best_of: 4
max_input_tokens: 512
max_output_tokens: 64
display_name: hf-100-llama-3-spellboun_6094_v1
ineligible_reason: None
language_model: hf-100/Llama-3-Spellbound-Instruct-8B-0.3
model_size: 8B
reward_model: ChaiML/reward_gpt2_medium_preference_24m_e2
us_pacific_date: 2024-07-08
win_ratio: 0.4296452675686914
preference_data_url: None
Resubmit model
Running pipeline stage MKMLizer
Starting job with name hf-100-llama-3-spellboun-6094-v1-mkmlizer
Waiting for job on hf-100-llama-3-spellboun-6094-v1-mkmlizer to finish
hf-100-llama-3-spellboun-6094-v1-mkmlizer: ╔═════════════════════════════════════════════════════════════════════╗
hf-100-llama-3-spellboun-6094-v1-mkmlizer: ║ _____ __ __ ║
hf-100-llama-3-spellboun-6094-v1-mkmlizer: ║ / _/ /_ ___ __/ / ___ ___ / / ║
hf-100-llama-3-spellboun-6094-v1-mkmlizer: ║ / _/ / // / |/|/ / _ \/ -_) -_) / ║
hf-100-llama-3-spellboun-6094-v1-mkmlizer: ║ /_//_/\_, /|__,__/_//_/\__/\__/_/ ║
hf-100-llama-3-spellboun-6094-v1-mkmlizer: ║ /___/ ║
hf-100-llama-3-spellboun-6094-v1-mkmlizer: ║ ║
hf-100-llama-3-spellboun-6094-v1-mkmlizer: ║ Version: 0.8.14 ║
hf-100-llama-3-spellboun-6094-v1-mkmlizer: ║ Copyright 2023 MK ONE TECHNOLOGIES Inc. ║
hf-100-llama-3-spellboun-6094-v1-mkmlizer: ║ https://mk1.ai ║
hf-100-llama-3-spellboun-6094-v1-mkmlizer: ║ ║
hf-100-llama-3-spellboun-6094-v1-mkmlizer: ║ The license key for the current software has been verified as ║
hf-100-llama-3-spellboun-6094-v1-mkmlizer: ║ belonging to: ║
hf-100-llama-3-spellboun-6094-v1-mkmlizer: ║ ║
hf-100-llama-3-spellboun-6094-v1-mkmlizer: ║ Chai Research Corp. ║
hf-100-llama-3-spellboun-6094-v1-mkmlizer: ║ Account ID: 7997a29f-0ceb-4cc7-9adf-840c57b4ae6f ║
hf-100-llama-3-spellboun-6094-v1-mkmlizer: ║ Expiration: 2024-07-15 23:59:59 ║
hf-100-llama-3-spellboun-6094-v1-mkmlizer: ║ ║
hf-100-llama-3-spellboun-6094-v1-mkmlizer: ╚═════════════════════════════════════════════════════════════════════╝
hf-100-llama-3-spellboun-6094-v1-mkmlizer: Downloaded to shared memory in 40.580s
hf-100-llama-3-spellboun-6094-v1-mkmlizer: quantizing model to /dev/shm/model_cache
hf-100-llama-3-spellboun-6094-v1-mkmlizer: Saving flywheel model at /dev/shm/model_cache
hf-100-llama-3-spellboun-6094-v1-mkmlizer: Loading 0: 0%| | 0/291 [00:00<?, ?it/s] Loading 0: 4%|▍ | 12/291 [00:00<00:02, 118.10it/s] Loading 0: 8%|▊ | 24/291 [00:00<00:04, 59.17it/s] Loading 0: 11%|█ | 32/291 [00:00<00:04, 64.35it/s] Loading 0: 16%|█▋ | 48/291 [00:00<00:02, 89.92it/s] Loading 0: 21%|██ | 61/291 [00:00<00:03, 74.38it/s] Loading 0: 25%|██▍ | 72/291 [00:00<00:02, 80.40it/s] Loading 0: 29%|██▉ | 85/291 [00:01<00:02, 89.92it/s] Loading 0: 34%|███▍ | 99/291 [00:01<00:01, 101.48it/s] Loading 0: 38%|███▊ | 111/291 [00:01<00:02, 77.45it/s] Loading 0: 42%|████▏ | 122/291 [00:01<00:02, 82.39it/s] Loading 0: 47%|████▋ | 137/291 [00:01<00:01, 96.37it/s] Loading 0: 51%|█████ | 148/291 [00:01<00:01, 74.98it/s] Loading 0: 56%|█████▌ | 162/291 [00:01<00:01, 87.86it/s] Loading 0: 60%|██████ | 175/291 [00:02<00:01, 95.44it/s] Loading 0: 64%|██████▍ | 186/291 [00:02<00:01, 74.61it/s] Loading 0: 69%|██████▉ | 201/291 [00:02<00:01, 89.64it/s] Loading 0: 73%|███████▎ | 212/291 [00:02<00:00, 91.74it/s] Loading 0: 77%|███████▋ | 223/291 [00:02<00:00, 77.28it/s] Loading 0: 80%|████████ | 233/291 [00:02<00:00, 81.47it/s] Loading 0: 85%|████████▍ | 247/291 [00:02<00:00, 91.92it/s] Loading 0: 90%|████████▉ | 261/291 [00:03<00:00, 102.73it/s] Loading 0: 94%|█████████▍| 273/291 [00:03<00:00, 79.14it/s] Loading 0: 98%|█████████▊| 284/291 [00:03<00:00, 84.12it/s] Special tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained.
hf-100-llama-3-spellboun-6094-v1-mkmlizer: quantized model in 24.229s
hf-100-llama-3-spellboun-6094-v1-mkmlizer: Processed model hf-100/Llama-3-Spellbound-Instruct-8B-0.3 in 64.810s
hf-100-llama-3-spellboun-6094-v1-mkmlizer: creating bucket guanaco-mkml-models
hf-100-llama-3-spellboun-6094-v1-mkmlizer: Bucket 's3://guanaco-mkml-models/' created
hf-100-llama-3-spellboun-6094-v1-mkmlizer: uploading /dev/shm/model_cache to s3://guanaco-mkml-models/hf-100-llama-3-spellboun-6094-v1
hf-100-llama-3-spellboun-6094-v1-mkmlizer: cp /dev/shm/model_cache/special_tokens_map.json s3://guanaco-mkml-models/hf-100-llama-3-spellboun-6094-v1/special_tokens_map.json
hf-100-llama-3-spellboun-6094-v1-mkmlizer: cp /dev/shm/model_cache/config.json s3://guanaco-mkml-models/hf-100-llama-3-spellboun-6094-v1/config.json
hf-100-llama-3-spellboun-6094-v1-mkmlizer: cp /dev/shm/model_cache/tokenizer.json s3://guanaco-mkml-models/hf-100-llama-3-spellboun-6094-v1/tokenizer.json
hf-100-llama-3-spellboun-6094-v1-mkmlizer: cp /dev/shm/model_cache/tokenizer_config.json s3://guanaco-mkml-models/hf-100-llama-3-spellboun-6094-v1/tokenizer_config.json
hf-100-llama-3-spellboun-6094-v1-mkmlizer: cp /dev/shm/model_cache/flywheel_model.0.safetensors s3://guanaco-mkml-models/hf-100-llama-3-spellboun-6094-v1/flywheel_model.0.safetensors
hf-100-llama-3-spellboun-6094-v1-mkmlizer: /opt/conda/lib/python3.10/site-packages/transformers/models/auto/tokenization_auto.py:769: FutureWarning: The `use_auth_token` argument is deprecated and will be removed in v5 of Transformers. Please use `token` instead.
hf-100-llama-3-spellboun-6094-v1-mkmlizer: warnings.warn(
hf-100-llama-3-spellboun-6094-v1-mkmlizer: /opt/conda/lib/python3.10/site-packages/transformers/models/auto/auto_factory.py:468: FutureWarning: The `use_auth_token` argument is deprecated and will be removed in v5 of Transformers. Please use `token` instead.
hf-100-llama-3-spellboun-6094-v1-mkmlizer: warnings.warn(
hf-100-llama-3-spellboun-6094-v1-mkmlizer: /opt/conda/lib/python3.10/site-packages/torch/_utils.py:831: UserWarning: TypedStorage is deprecated. It will be removed in the future and UntypedStorage will be the only storage class. This should only matter to you if you are using storages directly. To access UntypedStorage directly, use tensor.untyped_storage() instead of tensor.storage()
hf-100-llama-3-spellboun-6094-v1-mkmlizer: return self.fget.__get__(instance, owner)()
hf-100-llama-3-spellboun-6094-v1-mkmlizer: Saving model to /tmp/reward_cache/reward.tensors
hf-100-llama-3-spellboun-6094-v1-mkmlizer: Saving duration: 0.411s
hf-100-llama-3-spellboun-6094-v1-mkmlizer: Processed model ChaiML/reward_gpt2_medium_preference_24m_e2 in 4.749s
hf-100-llama-3-spellboun-6094-v1-mkmlizer: creating bucket guanaco-reward-models
hf-100-llama-3-spellboun-6094-v1-mkmlizer: Bucket 's3://guanaco-reward-models/' created
hf-100-llama-3-spellboun-6094-v1-mkmlizer: uploading /tmp/reward_cache to s3://guanaco-reward-models/hf-100-llama-3-spellboun-6094-v1_reward
hf-100-llama-3-spellboun-6094-v1-mkmlizer: cp /tmp/reward_cache/config.json s3://guanaco-reward-models/hf-100-llama-3-spellboun-6094-v1_reward/config.json
hf-100-llama-3-spellboun-6094-v1-mkmlizer: cp /tmp/reward_cache/tokenizer_config.json s3://guanaco-reward-models/hf-100-llama-3-spellboun-6094-v1_reward/tokenizer_config.json
hf-100-llama-3-spellboun-6094-v1-mkmlizer: cp /tmp/reward_cache/special_tokens_map.json s3://guanaco-reward-models/hf-100-llama-3-spellboun-6094-v1_reward/special_tokens_map.json
hf-100-llama-3-spellboun-6094-v1-mkmlizer: cp /tmp/reward_cache/merges.txt s3://guanaco-reward-models/hf-100-llama-3-spellboun-6094-v1_reward/merges.txt
hf-100-llama-3-spellboun-6094-v1-mkmlizer: cp /tmp/reward_cache/vocab.json s3://guanaco-reward-models/hf-100-llama-3-spellboun-6094-v1_reward/vocab.json
hf-100-llama-3-spellboun-6094-v1-mkmlizer: cp /tmp/reward_cache/tokenizer.json s3://guanaco-reward-models/hf-100-llama-3-spellboun-6094-v1_reward/tokenizer.json
hf-100-llama-3-spellboun-6094-v1-mkmlizer: cp /tmp/reward_cache/reward.tensors s3://guanaco-reward-models/hf-100-llama-3-spellboun-6094-v1_reward/reward.tensors
Job hf-100-llama-3-spellboun-6094-v1-mkmlizer completed after 96.19s with status: succeeded
Stopping job with name hf-100-llama-3-spellboun-6094-v1-mkmlizer
Pipeline stage MKMLizer completed in 97.60s
Running pipeline stage MKMLKubeTemplater
Pipeline stage MKMLKubeTemplater completed in 0.12s
Running pipeline stage ISVCDeployer
Creating inference service hf-100-llama-3-spellboun-6094-v1
Waiting for inference service hf-100-llama-3-spellboun-6094-v1 to be ready
Inference service hf-100-llama-3-spellboun-6094-v1 ready after 50.23305535316467s
Pipeline stage ISVCDeployer completed in 57.54s
Running pipeline stage StressChecker
Received healthy response to inference request in 1.5009691715240479s
Received healthy response to inference request in 1.100257158279419s
Received healthy response to inference request in 0.6626298427581787s
Received healthy response to inference request in 0.6457521915435791s
Received healthy response to inference request in 1.0971322059631348s
5 requests
0 failed requests
5th percentile: 0.649127721786499
10th percentile: 0.652503252029419
20th percentile: 0.6592543125152588
30th percentile: 0.7495303153991699
40th percentile: 0.9233312606811523
50th percentile: 1.0971322059631348
60th percentile: 1.0983821868896484
70th percentile: 1.099632167816162
80th percentile: 1.1803995609283449
90th percentile: 1.3406843662261962
95th percentile: 1.420826768875122
99th percentile: 1.4849406909942626
mean time: 1.001348114013672
Pipeline stage StressChecker completed in 5.76s
hf-100-llama-3-spellboun_6094_v1 status is now deployed due to DeploymentManager action
hf-100-llama-3-spellboun_6094_v1 status is now inactive due to auto deactivation removed underperforming models

Usage Metrics

Latency Metrics