submission_id: openlynn-llama-3-soliloq_8807_v2
developer_uid: Azazelle
status: inactive
model_repo: openlynn/Llama-3-Soliloquy-8B-v2
reward_repo: ChaiML/reward_gpt2_medium_preference_24m_e2
generation_params: {'temperature': 0.95, 'top_p': 1.0, 'min_p': 0.08, 'top_k': 50, 'presence_penalty': 0.0, 'frequency_penalty': 0.0, 'stopping_words': ['\n', '<|eot_id|>'], 'max_input_tokens': 512, 'best_of': 16, 'max_output_tokens': 64}
formatter: {'memory_template': "<|begin_of_text|><|start_header_id|>system<|end_header_id|>\n\n{bot_name}'s Persona: {memory}\n\n", 'prompt_template': '{prompt}<|eot_id|>', 'bot_template': '<|start_header_id|>assistant<|end_header_id|>\n\n{bot_name}: {message}<|eot_id|>', 'user_template': '<|start_header_id|>user<|end_header_id|>\n\n{user_name}: {message}<|eot_id|>', 'response_template': '<|start_header_id|>assistant<|end_header_id|>\n\n{bot_name}:', 'truncate_by_message': False}
reward_formatter: {'memory_template': "{bot_name}'s Persona: {memory}\n####\n", 'prompt_template': '{prompt}\n<START>\n', 'bot_template': '{bot_name}: {message}\n', 'user_template': '{user_name}: {message}\n', 'response_template': '{bot_name}:', 'truncate_by_message': False}
timestamp: 2024-07-06T04:53:14+00:00
model_name: openlynn-Soliloquy-8B-v2
model_group: openlynn/Llama-3-Soliloq
num_battles: 31633
num_wins: 15426
celo_rating: 1175.38
propriety_score: 0.7129159616469262
propriety_total_count: 5319.0
submission_type: basic
model_architecture: LlamaForCausalLM
model_num_parameters: 8030261248.0
best_of: 16
max_input_tokens: 512
max_output_tokens: 64
display_name: openlynn-Soliloquy-8B-v2
ineligible_reason: None
language_model: openlynn/Llama-3-Soliloquy-8B-v2
model_size: 8B
reward_model: ChaiML/reward_gpt2_medium_preference_24m_e2
us_pacific_date: 2024-07-05
win_ratio: 0.48765529668384283
Resubmit model
Running pipeline stage MKMLizer
Starting job with name openlynn-llama-3-soliloq-8807-v2-mkmlizer
Waiting for job on openlynn-llama-3-soliloq-8807-v2-mkmlizer to finish
openlynn-llama-3-soliloq-8807-v2-mkmlizer: ╔═════════════════════════════════════════════════════════════════════╗
openlynn-llama-3-soliloq-8807-v2-mkmlizer: ║ _____ __ __ ║
openlynn-llama-3-soliloq-8807-v2-mkmlizer: ║ / _/ /_ ___ __/ / ___ ___ / / ║
openlynn-llama-3-soliloq-8807-v2-mkmlizer: ║ / _/ / // / |/|/ / _ \/ -_) -_) / ║
openlynn-llama-3-soliloq-8807-v2-mkmlizer: ║ /_//_/\_, /|__,__/_//_/\__/\__/_/ ║
openlynn-llama-3-soliloq-8807-v2-mkmlizer: ║ /___/ ║
openlynn-llama-3-soliloq-8807-v2-mkmlizer: ║ ║
openlynn-llama-3-soliloq-8807-v2-mkmlizer: ║ Version: 0.8.14 ║
openlynn-llama-3-soliloq-8807-v2-mkmlizer: ║ Copyright 2023 MK ONE TECHNOLOGIES Inc. ║
openlynn-llama-3-soliloq-8807-v2-mkmlizer: ║ https://mk1.ai ║
openlynn-llama-3-soliloq-8807-v2-mkmlizer: ║ ║
openlynn-llama-3-soliloq-8807-v2-mkmlizer: ║ The license key for the current software has been verified as ║
openlynn-llama-3-soliloq-8807-v2-mkmlizer: ║ belonging to: ║
openlynn-llama-3-soliloq-8807-v2-mkmlizer: ║ ║
openlynn-llama-3-soliloq-8807-v2-mkmlizer: ║ Chai Research Corp. ║
openlynn-llama-3-soliloq-8807-v2-mkmlizer: ║ Account ID: 7997a29f-0ceb-4cc7-9adf-840c57b4ae6f ║
openlynn-llama-3-soliloq-8807-v2-mkmlizer: ║ Expiration: 2024-07-15 23:59:59 ║
openlynn-llama-3-soliloq-8807-v2-mkmlizer: ║ ║
openlynn-llama-3-soliloq-8807-v2-mkmlizer: ╚═════════════════════════════════════════════════════════════════════╝
openlynn-llama-3-soliloq-8807-v2-mkmlizer: Downloaded to shared memory in 53.212s
openlynn-llama-3-soliloq-8807-v2-mkmlizer: quantizing model to /dev/shm/model_cache
openlynn-llama-3-soliloq-8807-v2-mkmlizer: Saving flywheel model at /dev/shm/model_cache
openlynn-llama-3-soliloq-8807-v2-mkmlizer: Loading 0: 0%| | 0/291 [00:00<?, ?it/s] Loading 0: 4%|▍ | 13/291 [00:00<00:02, 123.50it/s] Loading 0: 10%|█ | 30/291 [00:00<00:01, 139.12it/s] Loading 0: 15%|█▌ | 44/291 [00:00<00:01, 134.93it/s] Loading 0: 20%|█▉ | 58/291 [00:00<00:01, 134.35it/s] Loading 0: 26%|██▌ | 75/291 [00:00<00:01, 140.26it/s] Loading 0: 31%|███ | 89/291 [00:00<00:02, 75.90it/s] Loading 0: 35%|███▌ | 103/291 [00:01<00:02, 87.12it/s] Loading 0: 41%|████ | 120/291 [00:01<00:01, 102.08it/s] Loading 0: 46%|████▌ | 133/291 [00:01<00:01, 107.94it/s] Loading 0: 51%|█████ | 148/291 [00:01<00:01, 115.11it/s] Loading 0: 57%|█████▋ | 165/291 [00:01<00:01, 124.97it/s] Loading 0: 62%|██████▏ | 180/291 [00:01<00:00, 127.43it/s] Loading 0: 67%|██████▋ | 194/291 [00:01<00:01, 77.86it/s] Loading 0: 73%|███████▎ | 211/291 [00:02<00:00, 92.34it/s] Loading 0: 78%|███████▊ | 228/291 [00:02<00:00, 105.18it/s] Loading 0: 83%|████████▎ | 241/291 [00:02<00:00, 109.94it/s] Loading 0: 88%|████████▊ | 256/291 [00:02<00:00, 116.32it/s] Loading 0: 93%|█████████▎| 270/291 [00:02<00:00, 122.11it/s] Loading 0: 98%|█████████▊| 284/291 [00:02<00:00, 121.40it/s] Special tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained.
openlynn-llama-3-soliloq-8807-v2-mkmlizer: quantized model in 23.795s
openlynn-llama-3-soliloq-8807-v2-mkmlizer: Processed model openlynn/Llama-3-Soliloquy-8B-v2 in 77.008s
openlynn-llama-3-soliloq-8807-v2-mkmlizer: creating bucket guanaco-mkml-models
openlynn-llama-3-soliloq-8807-v2-mkmlizer: Bucket 's3://guanaco-mkml-models/' created
openlynn-llama-3-soliloq-8807-v2-mkmlizer: uploading /dev/shm/model_cache to s3://guanaco-mkml-models/openlynn-llama-3-soliloq-8807-v2
openlynn-llama-3-soliloq-8807-v2-mkmlizer: cp /dev/shm/model_cache/tokenizer_config.json s3://guanaco-mkml-models/openlynn-llama-3-soliloq-8807-v2/tokenizer_config.json
openlynn-llama-3-soliloq-8807-v2-mkmlizer: cp /dev/shm/model_cache/special_tokens_map.json s3://guanaco-mkml-models/openlynn-llama-3-soliloq-8807-v2/special_tokens_map.json
openlynn-llama-3-soliloq-8807-v2-mkmlizer: cp /dev/shm/model_cache/config.json s3://guanaco-mkml-models/openlynn-llama-3-soliloq-8807-v2/config.json
openlynn-llama-3-soliloq-8807-v2-mkmlizer: cp /dev/shm/model_cache/tokenizer.json s3://guanaco-mkml-models/openlynn-llama-3-soliloq-8807-v2/tokenizer.json
openlynn-llama-3-soliloq-8807-v2-mkmlizer: cp /dev/shm/model_cache/flywheel_model.0.safetensors s3://guanaco-mkml-models/openlynn-llama-3-soliloq-8807-v2/flywheel_model.0.safetensors
openlynn-llama-3-soliloq-8807-v2-mkmlizer: loading reward model from ChaiML/reward_gpt2_medium_preference_24m_e2
openlynn-llama-3-soliloq-8807-v2-mkmlizer: /opt/conda/lib/python3.10/site-packages/transformers/models/auto/configuration_auto.py:919: FutureWarning: The `use_auth_token` argument is deprecated and will be removed in v5 of Transformers. Please use `token` instead.
openlynn-llama-3-soliloq-8807-v2-mkmlizer: warnings.warn(
openlynn-llama-3-soliloq-8807-v2-mkmlizer: /opt/conda/lib/python3.10/site-packages/huggingface_hub/file_download.py:1132: FutureWarning: `resume_download` is deprecated and will be removed in version 1.0.0. Downloads always resume when possible. If you want to force a new download, use `force_download=True`.
openlynn-llama-3-soliloq-8807-v2-mkmlizer: warnings.warn(
openlynn-llama-3-soliloq-8807-v2-mkmlizer: /opt/conda/lib/python3.10/site-packages/transformers/models/auto/tokenization_auto.py:769: FutureWarning: The `use_auth_token` argument is deprecated and will be removed in v5 of Transformers. Please use `token` instead.
openlynn-llama-3-soliloq-8807-v2-mkmlizer: warnings.warn(
openlynn-llama-3-soliloq-8807-v2-mkmlizer: /opt/conda/lib/python3.10/site-packages/transformers/models/auto/auto_factory.py:468: FutureWarning: The `use_auth_token` argument is deprecated and will be removed in v5 of Transformers. Please use `token` instead.
openlynn-llama-3-soliloq-8807-v2-mkmlizer: warnings.warn(
openlynn-llama-3-soliloq-8807-v2-mkmlizer: /opt/conda/lib/python3.10/site-packages/torch/_utils.py:831: UserWarning: TypedStorage is deprecated. It will be removed in the future and UntypedStorage will be the only storage class. This should only matter to you if you are using storages directly. To access UntypedStorage directly, use tensor.untyped_storage() instead of tensor.storage()
openlynn-llama-3-soliloq-8807-v2-mkmlizer: return self.fget.__get__(instance, owner)()
openlynn-llama-3-soliloq-8807-v2-mkmlizer: Saving model to /tmp/reward_cache/reward.tensors
openlynn-llama-3-soliloq-8807-v2-mkmlizer: Saving duration: 0.412s
openlynn-llama-3-soliloq-8807-v2-mkmlizer: Processed model ChaiML/reward_gpt2_medium_preference_24m_e2 in 12.789s
openlynn-llama-3-soliloq-8807-v2-mkmlizer: creating bucket guanaco-reward-models
openlynn-llama-3-soliloq-8807-v2-mkmlizer: Bucket 's3://guanaco-reward-models/' created
openlynn-llama-3-soliloq-8807-v2-mkmlizer: uploading /tmp/reward_cache to s3://guanaco-reward-models/openlynn-llama-3-soliloq-8807-v2_reward
openlynn-llama-3-soliloq-8807-v2-mkmlizer: cp /tmp/reward_cache/special_tokens_map.json s3://guanaco-reward-models/openlynn-llama-3-soliloq-8807-v2_reward/special_tokens_map.json
openlynn-llama-3-soliloq-8807-v2-mkmlizer: cp /tmp/reward_cache/tokenizer_config.json s3://guanaco-reward-models/openlynn-llama-3-soliloq-8807-v2_reward/tokenizer_config.json
openlynn-llama-3-soliloq-8807-v2-mkmlizer: cp /tmp/reward_cache/merges.txt s3://guanaco-reward-models/openlynn-llama-3-soliloq-8807-v2_reward/merges.txt
openlynn-llama-3-soliloq-8807-v2-mkmlizer: cp /tmp/reward_cache/config.json s3://guanaco-reward-models/openlynn-llama-3-soliloq-8807-v2_reward/config.json
openlynn-llama-3-soliloq-8807-v2-mkmlizer: cp /tmp/reward_cache/vocab.json s3://guanaco-reward-models/openlynn-llama-3-soliloq-8807-v2_reward/vocab.json
openlynn-llama-3-soliloq-8807-v2-mkmlizer: cp /tmp/reward_cache/tokenizer.json s3://guanaco-reward-models/openlynn-llama-3-soliloq-8807-v2_reward/tokenizer.json
Job openlynn-llama-3-soliloq-8807-v2-mkmlizer completed after 114.46s with status: succeeded
Stopping job with name openlynn-llama-3-soliloq-8807-v2-mkmlizer
Pipeline stage MKMLizer completed in 115.30s
Running pipeline stage MKMLKubeTemplater
Pipeline stage MKMLKubeTemplater completed in 0.10s
Running pipeline stage ISVCDeployer
Creating inference service openlynn-llama-3-soliloq-8807-v2
Waiting for inference service openlynn-llama-3-soliloq-8807-v2 to be ready
Inference service openlynn-llama-3-soliloq-8807-v2 ready after 40.178550481796265s
Pipeline stage ISVCDeployer completed in 46.74s
Running pipeline stage StressChecker
Received healthy response to inference request in 2.0758113861083984s
Received healthy response to inference request in 1.577347993850708s
Received healthy response to inference request in 1.3579456806182861s
Received healthy response to inference request in 1.3176655769348145s
Received healthy response to inference request in 1.334073543548584s
5 requests
0 failed requests
5th percentile: 1.3209471702575684
10th percentile: 1.3242287635803223
20th percentile: 1.33079195022583
30th percentile: 1.3388479709625245
40th percentile: 1.3483968257904053
50th percentile: 1.3579456806182861
60th percentile: 1.4457066059112549
70th percentile: 1.5334675312042236
80th percentile: 1.6770406723022462
90th percentile: 1.8764260292053223
95th percentile: 1.9761187076568603
99th percentile: 2.055872850418091
mean time: 1.5325688362121581
Pipeline stage StressChecker completed in 8.43s
openlynn-llama-3-soliloq_8807_v2 status is now deployed due to DeploymentManager action
openlynn-llama-3-soliloq_8807_v2 status is now inactive due to auto deactivation removed underperforming models

Usage Metrics

Latency Metrics