submission_id: nousresearch-meta-llama_4941_v99
developer_uid: chai_backend_admin
alignment_samples: 0
best_of: 16
celo_rating: 1184.71
display_name: nousresearch-meta-llama_4941_v99
formatter: {'memory_template': "{bot_name}'s Persona: {memory}\n####\n", 'prompt_template': '{prompt}\n<START>\n', 'bot_template': '{bot_name}: {message}\n', 'user_template': '{user_name}: {message}\n', 'response_template': '{bot_name}:', 'truncate_by_message': False}
generation_params: {'temperature': 1.0, 'top_p': 1.0, 'min_p': 0.0, 'top_k': 40, 'presence_penalty': 0.0, 'frequency_penalty': 0.0, 'stopping_words': ['\n'], 'max_input_tokens': 512, 'best_of': 16, 'max_output_tokens': 64, 'reward_max_token_input': 256}
is_internal_developer: True
language_model: NousResearch/Meta-Llama-3-8B-Instruct
max_input_tokens: 512
max_output_tokens: 64
model_architecture: LlamaForCausalLM
model_group: NousResearch/Meta-Llama-
model_name: nousresearch-meta-llama_4941_v99
model_num_parameters: 8030261248.0
model_repo: NousResearch/Meta-Llama-3-8B-Instruct
model_size: 8B
num_battles: 10693
num_wins: 4977
propriety_score: 0.6799163179916318
propriety_total_count: 956.0
ranking_group: single
reward_formatter: {'bot_template': '{bot_name}: {message}\n', 'memory_template': "{bot_name}'s Persona: {memory}\n####\n", 'prompt_template': '{prompt}\n<START>\n', 'response_template': '{bot_name}:', 'truncate_by_message': False, 'user_template': '{user_name}: {message}\n'}
reward_repo: ChaiML/gpt2_xl_pairwise_89m_step_347634
status: torndown
submission_type: basic
timestamp: 2024-08-05T03:47:26+00:00
us_pacific_date: 2024-08-04
win_ratio: 0.465444683437763
Download Preference Data
Resubmit model
Running pipeline stage MKMLizer
Starting job with name nousresearch-meta-llama-4941-v99-mkmlizer
Waiting for job on nousresearch-meta-llama-4941-v99-mkmlizer to finish
nousresearch-meta-llama-4941-v99-mkmlizer: ╔═════════════════════════════════════════════════════════════════════╗
nousresearch-meta-llama-4941-v99-mkmlizer: ║ _____ __ __ ║
nousresearch-meta-llama-4941-v99-mkmlizer: ║ / _/ /_ ___ __/ / ___ ___ / / ║
nousresearch-meta-llama-4941-v99-mkmlizer: ║ / _/ / // / |/|/ / _ \/ -_) -_) / ║
nousresearch-meta-llama-4941-v99-mkmlizer: ║ /_//_/\_, /|__,__/_//_/\__/\__/_/ ║
nousresearch-meta-llama-4941-v99-mkmlizer: ║ /___/ ║
nousresearch-meta-llama-4941-v99-mkmlizer: ║ ║
nousresearch-meta-llama-4941-v99-mkmlizer: ║ Version: 0.9.9 ║
nousresearch-meta-llama-4941-v99-mkmlizer: ║ Copyright 2023 MK ONE TECHNOLOGIES Inc. ║
nousresearch-meta-llama-4941-v99-mkmlizer: ║ https://mk1.ai ║
nousresearch-meta-llama-4941-v99-mkmlizer: ║ ║
nousresearch-meta-llama-4941-v99-mkmlizer: ║ The license key for the current software has been verified as ║
nousresearch-meta-llama-4941-v99-mkmlizer: ║ belonging to: ║
nousresearch-meta-llama-4941-v99-mkmlizer: ║ ║
nousresearch-meta-llama-4941-v99-mkmlizer: ║ Chai Research Corp. ║
nousresearch-meta-llama-4941-v99-mkmlizer: ║ Account ID: 7997a29f-0ceb-4cc7-9adf-840c57b4ae6f ║
nousresearch-meta-llama-4941-v99-mkmlizer: ║ Expiration: 2024-10-15 23:59:59 ║
nousresearch-meta-llama-4941-v99-mkmlizer: ║ ║
nousresearch-meta-llama-4941-v99-mkmlizer: ╚═════════════════════════════════════════════════════════════════════╝
nousresearch-meta-llama-4941-v99-mkmlizer: Downloaded to shared memory in 22.300s
nousresearch-meta-llama-4941-v99-mkmlizer: quantizing model to /dev/shm/model_cache, profile:s0, folder:/tmp/tmpxperxjnk, device:0
nousresearch-meta-llama-4941-v99-mkmlizer: Saving flywheel model at /dev/shm/model_cache
nousresearch-meta-llama-4941-v99-mkmlizer: quantized model in 27.330s
nousresearch-meta-llama-4941-v99-mkmlizer: Processed model NousResearch/Meta-Llama-3-8B-Instruct in 49.630s
nousresearch-meta-llama-4941-v99-mkmlizer: creating bucket guanaco-mkml-models
nousresearch-meta-llama-4941-v99-mkmlizer: Bucket 's3://guanaco-mkml-models/' created
nousresearch-meta-llama-4941-v99-mkmlizer: uploading /dev/shm/model_cache to s3://guanaco-mkml-models/nousresearch-meta-llama-4941-v99
nousresearch-meta-llama-4941-v99-mkmlizer: cp /dev/shm/model_cache/config.json s3://guanaco-mkml-models/nousresearch-meta-llama-4941-v99/config.json
nousresearch-meta-llama-4941-v99-mkmlizer: cp /dev/shm/model_cache/special_tokens_map.json s3://guanaco-mkml-models/nousresearch-meta-llama-4941-v99/special_tokens_map.json
nousresearch-meta-llama-4941-v99-mkmlizer: cp /dev/shm/model_cache/tokenizer_config.json s3://guanaco-mkml-models/nousresearch-meta-llama-4941-v99/tokenizer_config.json
nousresearch-meta-llama-4941-v99-mkmlizer: cp /dev/shm/model_cache/tokenizer.json s3://guanaco-mkml-models/nousresearch-meta-llama-4941-v99/tokenizer.json
nousresearch-meta-llama-4941-v99-mkmlizer: cp /dev/shm/model_cache/flywheel_model.0.safetensors s3://guanaco-mkml-models/nousresearch-meta-llama-4941-v99/flywheel_model.0.safetensors
nousresearch-meta-llama-4941-v99-mkmlizer: /opt/conda/lib/python3.10/site-packages/transformers/models/auto/tokenization_auto.py:785: FutureWarning: The `use_auth_token` argument is deprecated and will be removed in v5 of Transformers. Please use `token` instead.
nousresearch-meta-llama-4941-v99-mkmlizer: warnings.warn(
nousresearch-meta-llama-4941-v99-mkmlizer: /opt/conda/lib/python3.10/site-packages/transformers/models/auto/auto_factory.py:469: FutureWarning: The `use_auth_token` argument is deprecated and will be removed in v5 of Transformers. Please use `token` instead.
nousresearch-meta-llama-4941-v99-mkmlizer: warnings.warn(
nousresearch-meta-llama-4941-v99-mkmlizer: Loading checkpoint shards: 0%| | 0/2 [00:00<?, ?it/s] Loading checkpoint shards: 50%|█████ | 1/2 [00:00<00:00, 2.20it/s] Loading checkpoint shards: 100%|██████████| 2/2 [00:00<00:00, 3.63it/s] Loading checkpoint shards: 100%|██████████| 2/2 [00:00<00:00, 3.31it/s]
nousresearch-meta-llama-4941-v99-mkmlizer: Saving model to /tmp/reward_cache/reward.tensors
nousresearch-meta-llama-4941-v99-mkmlizer: Saving duration: 1.423s
nousresearch-meta-llama-4941-v99-mkmlizer: Processed model ChaiML/gpt2_xl_pairwise_89m_step_347634 in 13.128s
nousresearch-meta-llama-4941-v99-mkmlizer: creating bucket guanaco-reward-models
nousresearch-meta-llama-4941-v99-mkmlizer: Bucket 's3://guanaco-reward-models/' created
nousresearch-meta-llama-4941-v99-mkmlizer: uploading /tmp/reward_cache to s3://guanaco-reward-models/nousresearch-meta-llama-4941-v99_reward
nousresearch-meta-llama-4941-v99-mkmlizer: cp /tmp/reward_cache/config.json s3://guanaco-reward-models/nousresearch-meta-llama-4941-v99_reward/config.json
nousresearch-meta-llama-4941-v99-mkmlizer: cp /tmp/reward_cache/special_tokens_map.json s3://guanaco-reward-models/nousresearch-meta-llama-4941-v99_reward/special_tokens_map.json
nousresearch-meta-llama-4941-v99-mkmlizer: cp /tmp/reward_cache/tokenizer_config.json s3://guanaco-reward-models/nousresearch-meta-llama-4941-v99_reward/tokenizer_config.json
nousresearch-meta-llama-4941-v99-mkmlizer: cp /tmp/reward_cache/merges.txt s3://guanaco-reward-models/nousresearch-meta-llama-4941-v99_reward/merges.txt
nousresearch-meta-llama-4941-v99-mkmlizer: cp /tmp/reward_cache/vocab.json s3://guanaco-reward-models/nousresearch-meta-llama-4941-v99_reward/vocab.json
nousresearch-meta-llama-4941-v99-mkmlizer: cp /tmp/reward_cache/tokenizer.json s3://guanaco-reward-models/nousresearch-meta-llama-4941-v99_reward/tokenizer.json
nousresearch-meta-llama-4941-v99-mkmlizer: cp /tmp/reward_cache/reward.tensors s3://guanaco-reward-models/nousresearch-meta-llama-4941-v99_reward/reward.tensors
Job nousresearch-meta-llama-4941-v99-mkmlizer completed after 103.44s with status: succeeded
Stopping job with name nousresearch-meta-llama-4941-v99-mkmlizer
Pipeline stage MKMLizer completed in 105.19s
Running pipeline stage MKMLKubeTemplater
Pipeline stage MKMLKubeTemplater completed in 0.46s
Running pipeline stage ISVCDeployer
Creating inference service nousresearch-meta-llama-4941-v99
Waiting for inference service nousresearch-meta-llama-4941-v99 to be ready
Inference service nousresearch-meta-llama-4941-v99 ready after 172.67363214492798s
Pipeline stage ISVCDeployer completed in 175.47s
Running pipeline stage StressChecker
HTTP Request: %s %s "%s %d %s"
Received healthy response to inference request in 10.206346988677979s
HTTP Request: %s %s "%s %d %s"
Received healthy response to inference request in 2.1800949573516846s
HTTP Request: %s %s "%s %d %s"
Received healthy response to inference request in 1.8254601955413818s
HTTP Request: %s %s "%s %d %s"
Received healthy response to inference request in 1.8737611770629883s
HTTP Request: %s %s "%s %d %s"
Received healthy response to inference request in 1.840907096862793s
5 requests
0 failed requests
5th percentile: 1.828549575805664
10th percentile: 1.8316389560699462
20th percentile: 1.8378177165985108
30th percentile: 1.847477912902832
40th percentile: 1.8606195449829102
50th percentile: 1.8737611770629883
60th percentile: 1.9962946891784668
70th percentile: 2.1188282012939452
80th percentile: 3.785345363616945
90th percentile: 6.995846176147461
95th percentile: 8.601096582412719
99th percentile: 9.885296907424927
mean time: 3.585314083099365
Pipeline stage StressChecker completed in 21.69s
nousresearch-meta-llama_4941_v99 status is now deployed due to DeploymentManager action
nousresearch-meta-llama_4941_v99 status is now inactive due to auto deactivation removed underperforming models
nousresearch-meta-llama_4941_v99 status is now torndown due to DeploymentManager action

Usage Metrics

Latency Metrics