submission_id: alkahestry-llmama3-monol_3361_v8
developer_uid: alkacchi
status: inactive
model_repo: alkahestry/Llmama3-Monologue-8B
reward_repo: Jellywibble/CHAI_alignment_reward_model
generation_params: {'temperature': 0.72, 'top_p': 0.73, 'min_p': 0.1, 'top_k': 1000, 'presence_penalty': 0.82, 'frequency_penalty': 0.2, 'stopping_words': ['\n', '</s>', '<|im_end|>', '\n{{User}}', '<|eot_id|>', '<|end_of_text|>'], 'max_input_tokens': 512, 'best_of': 16, 'max_output_tokens': 64}
formatter: {'memory_template': "<|begin_of_text|><|start_header_id|>system\n{bot_name}'s persona: {memory}<|end_header_id|>", 'prompt_template': '{prompt}<|eot_id|>', 'bot_template': '<|start_header_id|>assistant<|end_header_id|>\n\n{bot_name}: {message}<|eot_id|>', 'user_template': '<|start_header_id|>user<|end_header_id|>\n\n{user_name}: {message}<|eot_id|>', 'response_template': '<|start_header_id|>assistant<|end_header_id|>\n\n{bot_name}:', 'truncate_by_message': False}
reward_formatter: {'memory_template': "{bot_name}'s Persona: {memory}\n####\n", 'prompt_template': '{prompt}\n<START>\n', 'bot_template': '{bot_name}: {message}\n', 'user_template': '{user_name}: {message}\n', 'response_template': '{bot_name}:', 'truncate_by_message': False}
timestamp: 2024-06-30T01:16:42+00:00
model_name: alkacchi-monologue-8B
model_group: alkahestry/Llmama3-Monol
num_battles: 17276
num_wins: 7195
celo_rating: 1096.67
propriety_score: 0.7540238358520703
propriety_total_count: 8139.0
submission_type: basic
model_architecture: LlamaForCausalLM
model_num_parameters: 8030261248.0
best_of: 16
max_input_tokens: 512
max_output_tokens: 64
display_name: alkacchi-monologue-8B
ineligible_reason: None
language_model: alkahestry/Llmama3-Monologue-8B
model_size: 8B
reward_model: Jellywibble/CHAI_alignment_reward_model
us_pacific_date: 2024-06-29
win_ratio: 0.41647372076869643
Resubmit model
Running pipeline stage MKMLizer
Starting job with name alkahestry-llmama3-monol-3361-v8-mkmlizer
Waiting for job on alkahestry-llmama3-monol-3361-v8-mkmlizer to finish
alkahestry-llmama3-monol-3361-v8-mkmlizer: ╔═════════════════════════════════════════════════════════════════════╗
alkahestry-llmama3-monol-3361-v8-mkmlizer: ║ _____ __ __ ║
alkahestry-llmama3-monol-3361-v8-mkmlizer: ║ / _/ /_ ___ __/ / ___ ___ / / ║
alkahestry-llmama3-monol-3361-v8-mkmlizer: ║ / _/ / // / |/|/ / _ \/ -_) -_) / ║
alkahestry-llmama3-monol-3361-v8-mkmlizer: ║ /_//_/\_, /|__,__/_//_/\__/\__/_/ ║
alkahestry-llmama3-monol-3361-v8-mkmlizer: ║ /___/ ║
alkahestry-llmama3-monol-3361-v8-mkmlizer: ║ ║
alkahestry-llmama3-monol-3361-v8-mkmlizer: ║ Version: 0.8.14 ║
alkahestry-llmama3-monol-3361-v8-mkmlizer: ║ Copyright 2023 MK ONE TECHNOLOGIES Inc. ║
alkahestry-llmama3-monol-3361-v8-mkmlizer: ║ https://mk1.ai ║
alkahestry-llmama3-monol-3361-v8-mkmlizer: ║ ║
alkahestry-llmama3-monol-3361-v8-mkmlizer: ║ The license key for the current software has been verified as ║
alkahestry-llmama3-monol-3361-v8-mkmlizer: ║ belonging to: ║
alkahestry-llmama3-monol-3361-v8-mkmlizer: ║ ║
alkahestry-llmama3-monol-3361-v8-mkmlizer: ║ Chai Research Corp. ║
alkahestry-llmama3-monol-3361-v8-mkmlizer: ║ Account ID: 7997a29f-0ceb-4cc7-9adf-840c57b4ae6f ║
alkahestry-llmama3-monol-3361-v8-mkmlizer: ║ Expiration: 2024-07-15 23:59:59 ║
alkahestry-llmama3-monol-3361-v8-mkmlizer: ║ ║
alkahestry-llmama3-monol-3361-v8-mkmlizer: ╚═════════════════════════════════════════════════════════════════════╝
alkahestry-llmama3-monol-3361-v8-mkmlizer: /opt/conda/lib/python3.10/site-packages/huggingface_hub/utils/_deprecation.py:131: FutureWarning: 'list_files_info' (from 'huggingface_hub.hf_api') is deprecated and will be removed from version '0.23'. Use `list_repo_tree` and `get_paths_info` instead.
alkahestry-llmama3-monol-3361-v8-mkmlizer: warnings.warn(warning_message, FutureWarning)
alkahestry-llmama3-monol-3361-v8-mkmlizer: Downloaded to shared memory in 52.340s
alkahestry-llmama3-monol-3361-v8-mkmlizer: quantizing model to /dev/shm/model_cache
alkahestry-llmama3-monol-3361-v8-mkmlizer: Saving flywheel model at /dev/shm/model_cache
alkahestry-llmama3-monol-3361-v8-mkmlizer: Loading 0: 0%| | 0/291 [00:00<?, ?it/s] Loading 0: 4%|▍ | 12/291 [00:00<00:02, 105.29it/s] Loading 0: 8%|▊ | 23/291 [00:00<00:03, 87.25it/s] Loading 0: 11%|█ | 32/291 [00:00<00:02, 87.37it/s] Loading 0: 14%|█▍ | 41/291 [00:00<00:02, 85.46it/s] Loading 0: 17%|█▋ | 50/291 [00:00<00:02, 83.85it/s] Loading 0: 21%|██ | 60/291 [00:00<00:02, 88.75it/s] Loading 0: 24%|██▍ | 71/291 [00:00<00:02, 93.51it/s] Loading 0: 28%|██▊ | 81/291 [00:00<00:02, 94.11it/s] Loading 0: 31%|███▏ | 91/291 [00:01<00:03, 54.88it/s] Loading 0: 35%|███▌ | 102/291 [00:01<00:02, 64.90it/s] Loading 0: 38%|███▊ | 112/291 [00:01<00:02, 68.97it/s] Loading 0: 42%|████▏ | 122/291 [00:01<00:02, 72.18it/s] Loading 0: 46%|████▌ | 134/291 [00:01<00:01, 82.46it/s] Loading 0: 49%|████▉ | 144/291 [00:01<00:01, 86.14it/s] Loading 0: 54%|█████▎ | 156/291 [00:01<00:01, 89.82it/s] Loading 0: 57%|█████▋ | 166/291 [00:02<00:01, 87.61it/s] Loading 0: 62%|██████▏ | 179/291 [00:02<00:01, 94.58it/s] Loading 0: 65%|██████▍ | 189/291 [00:02<00:01, 57.31it/s] Loading 0: 69%|██████▉ | 201/291 [00:02<00:01, 66.13it/s] Loading 0: 73%|███████▎ | 211/291 [00:02<00:01, 70.33it/s] Loading 0: 76%|███████▌ | 221/291 [00:02<00:00, 74.81it/s] Loading 0: 80%|████████ | 234/291 [00:02<00:00, 87.05it/s] Loading 0: 85%|████████▍ | 246/291 [00:03<00:00, 92.40it/s] Loading 0: 88%|████████▊ | 257/291 [00:03<00:00, 88.91it/s] Loading 0: 92%|█████████▏| 269/291 [00:03<00:00, 96.63it/s] Loading 0: 97%|█████████▋| 281/291 [00:03<00:00, 97.30it/s] Special tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained.
alkahestry-llmama3-monol-3361-v8-mkmlizer: quantized model in 25.234s
alkahestry-llmama3-monol-3361-v8-mkmlizer: Processed model alkahestry/Llmama3-Monologue-8B in 80.076s
alkahestry-llmama3-monol-3361-v8-mkmlizer: creating bucket guanaco-mkml-models
alkahestry-llmama3-monol-3361-v8-mkmlizer: Bucket 's3://guanaco-mkml-models/' created
alkahestry-llmama3-monol-3361-v8-mkmlizer: uploading /dev/shm/model_cache to s3://guanaco-mkml-models/alkahestry-llmama3-monol-3361-v8
alkahestry-llmama3-monol-3361-v8-mkmlizer: cp /dev/shm/model_cache/config.json s3://guanaco-mkml-models/alkahestry-llmama3-monol-3361-v8/config.json
alkahestry-llmama3-monol-3361-v8-mkmlizer: cp /dev/shm/model_cache/special_tokens_map.json s3://guanaco-mkml-models/alkahestry-llmama3-monol-3361-v8/special_tokens_map.json
alkahestry-llmama3-monol-3361-v8-mkmlizer: cp /dev/shm/model_cache/tokenizer_config.json s3://guanaco-mkml-models/alkahestry-llmama3-monol-3361-v8/tokenizer_config.json
alkahestry-llmama3-monol-3361-v8-mkmlizer: cp /dev/shm/model_cache/tokenizer.json s3://guanaco-mkml-models/alkahestry-llmama3-monol-3361-v8/tokenizer.json
alkahestry-llmama3-monol-3361-v8-mkmlizer: cp /dev/shm/model_cache/flywheel_model.0.safetensors s3://guanaco-mkml-models/alkahestry-llmama3-monol-3361-v8/flywheel_model.0.safetensors
alkahestry-llmama3-monol-3361-v8-mkmlizer: /opt/conda/lib/python3.10/site-packages/transformers/models/auto/auto_factory.py:468: FutureWarning: The `use_auth_token` argument is deprecated and will be removed in v5 of Transformers. Please use `token` instead.
alkahestry-llmama3-monol-3361-v8-mkmlizer: warnings.warn(
alkahestry-llmama3-monol-3361-v8-mkmlizer: Saving model to /tmp/reward_cache/reward.tensors
alkahestry-llmama3-monol-3361-v8-mkmlizer: Saving duration: 0.168s
alkahestry-llmama3-monol-3361-v8-mkmlizer: Processed model Jellywibble/CHAI_alignment_reward_model in 11.945s
alkahestry-llmama3-monol-3361-v8-mkmlizer: creating bucket guanaco-reward-models
alkahestry-llmama3-monol-3361-v8-mkmlizer: Bucket 's3://guanaco-reward-models/' created
alkahestry-llmama3-monol-3361-v8-mkmlizer: uploading /tmp/reward_cache to s3://guanaco-reward-models/alkahestry-llmama3-monol-3361-v8_reward
alkahestry-llmama3-monol-3361-v8-mkmlizer: cp /tmp/reward_cache/special_tokens_map.json s3://guanaco-reward-models/alkahestry-llmama3-monol-3361-v8_reward/special_tokens_map.json
alkahestry-llmama3-monol-3361-v8-mkmlizer: cp /tmp/reward_cache/config.json s3://guanaco-reward-models/alkahestry-llmama3-monol-3361-v8_reward/config.json
alkahestry-llmama3-monol-3361-v8-mkmlizer: cp /tmp/reward_cache/tokenizer_config.json s3://guanaco-reward-models/alkahestry-llmama3-monol-3361-v8_reward/tokenizer_config.json
alkahestry-llmama3-monol-3361-v8-mkmlizer: cp /tmp/reward_cache/merges.txt s3://guanaco-reward-models/alkahestry-llmama3-monol-3361-v8_reward/merges.txt
alkahestry-llmama3-monol-3361-v8-mkmlizer: cp /tmp/reward_cache/vocab.json s3://guanaco-reward-models/alkahestry-llmama3-monol-3361-v8_reward/vocab.json
alkahestry-llmama3-monol-3361-v8-mkmlizer: cp /tmp/reward_cache/tokenizer.json s3://guanaco-reward-models/alkahestry-llmama3-monol-3361-v8_reward/tokenizer.json
alkahestry-llmama3-monol-3361-v8-mkmlizer: cp /tmp/reward_cache/reward.tensors s3://guanaco-reward-models/alkahestry-llmama3-monol-3361-v8_reward/reward.tensors
Job alkahestry-llmama3-monol-3361-v8-mkmlizer completed after 114.33s with status: succeeded
Stopping job with name alkahestry-llmama3-monol-3361-v8-mkmlizer
Pipeline stage MKMLizer completed in 115.65s
Running pipeline stage MKMLKubeTemplater
Pipeline stage MKMLKubeTemplater completed in 0.11s
Running pipeline stage ISVCDeployer
Creating inference service alkahestry-llmama3-monol-3361-v8
Waiting for inference service alkahestry-llmama3-monol-3361-v8 to be ready
Connection pool is full, discarding connection: %s
Inference service alkahestry-llmama3-monol-3361-v8 ready after 80.4055585861206s
Pipeline stage ISVCDeployer completed in 87.27s
Running pipeline stage StressChecker
Received healthy response to inference request in 2.1277265548706055s
Received healthy response to inference request in 1.3164124488830566s
Received healthy response to inference request in 1.2934658527374268s
Received healthy response to inference request in 1.257279872894287s
Received healthy response to inference request in 1.3174848556518555s
5 requests
0 failed requests
5th percentile: 1.264517068862915
10th percentile: 1.271754264831543
20th percentile: 1.2862286567687988
30th percentile: 1.2980551719665527
40th percentile: 1.3072338104248047
50th percentile: 1.3164124488830566
60th percentile: 1.3168414115905762
70th percentile: 1.3172703742980958
80th percentile: 1.4795331954956057
90th percentile: 1.8036298751831055
95th percentile: 1.9656782150268552
99th percentile: 2.0953168869018555
mean time: 1.4624739170074463
Pipeline stage StressChecker completed in 8.04s
alkahestry-llmama3-monol_3361_v8 status is now deployed due to DeploymentManager action
alkahestry-llmama3-monol_3361_v8 status is now inactive due to auto deactivation removed underperforming models

Usage Metrics

Latency Metrics