submission_id: nousresearch-meta-llama_4941_v60
developer_uid: chai_backend_admin
status: inactive
model_repo: NousResearch/Meta-Llama-3-8B-Instruct
reward_repo: ChaiML/gpt2_medium_pairwise_60m_step_937500
generation_params: {'temperature': 1.0, 'top_p': 1.0, 'min_p': 0.0, 'top_k': 40, 'presence_penalty': 0.0, 'frequency_penalty': 0.0, 'stopping_words': ['\n'], 'max_input_tokens': 512, 'best_of': 16, 'max_output_tokens': 64}
formatter: {'memory_template': "{bot_name}'s Persona: {memory}\n####\n", 'prompt_template': '{prompt}\n<START>\n', 'bot_template': '{bot_name}: {message}\n', 'user_template': '{user_name}: {message}\n', 'response_template': '{bot_name}:', 'truncate_by_message': False}
reward_formatter: {'memory_template': "{bot_name}'s Persona: {memory}\n####\n", 'prompt_template': '{prompt}\n<START>\n', 'bot_template': '{bot_name}: {message}\n', 'user_template': '{user_name}: {message}\n', 'response_template': '{bot_name}:', 'truncate_by_message': False}
timestamp: 2024-06-27T19:29:54+00:00
model_name: nousresearch-meta-llama_4941_v60
model_group: NousResearch/Meta-Llama-
num_battles: 17542
num_wins: 8837
celo_rating: 1189.21
propriety_score: 0.735873540167512
propriety_total_count: 8477.0
submission_type: basic
model_architecture: LlamaForCausalLM
model_num_parameters: 8030261248.0
best_of: 16
max_input_tokens: 512
max_output_tokens: 64
display_name: nousresearch-meta-llama_4941_v60
ineligible_reason: None
language_model: NousResearch/Meta-Llama-3-8B-Instruct
model_size: 8B
reward_model: ChaiML/gpt2_medium_pairwise_60m_step_937500
us_pacific_date: 2024-06-27
win_ratio: 0.5037623988142743
Resubmit model
Running pipeline stage MKMLizer
Starting job with name nousresearch-meta-llama-4941-v60-mkmlizer
Waiting for job on nousresearch-meta-llama-4941-v60-mkmlizer to finish
nousresearch-meta-llama-4941-v60-mkmlizer: ╔═════════════════════════════════════════════════════════════════════╗
nousresearch-meta-llama-4941-v60-mkmlizer: ║ _____ __ __ ║
nousresearch-meta-llama-4941-v60-mkmlizer: ║ / _/ /_ ___ __/ / ___ ___ / / ║
nousresearch-meta-llama-4941-v60-mkmlizer: ║ / _/ / // / |/|/ / _ \/ -_) -_) / ║
nousresearch-meta-llama-4941-v60-mkmlizer: ║ /_//_/\_, /|__,__/_//_/\__/\__/_/ ║
nousresearch-meta-llama-4941-v60-mkmlizer: ║ /___/ ║
nousresearch-meta-llama-4941-v60-mkmlizer: ║ ║
nousresearch-meta-llama-4941-v60-mkmlizer: ║ Version: 0.8.14 ║
nousresearch-meta-llama-4941-v60-mkmlizer: ║ Copyright 2023 MK ONE TECHNOLOGIES Inc. ║
nousresearch-meta-llama-4941-v60-mkmlizer: ║ https://mk1.ai ║
nousresearch-meta-llama-4941-v60-mkmlizer: ║ ║
nousresearch-meta-llama-4941-v60-mkmlizer: ║ The license key for the current software has been verified as ║
nousresearch-meta-llama-4941-v60-mkmlizer: ║ belonging to: ║
nousresearch-meta-llama-4941-v60-mkmlizer: ║ ║
nousresearch-meta-llama-4941-v60-mkmlizer: ║ Chai Research Corp. ║
nousresearch-meta-llama-4941-v60-mkmlizer: ║ Account ID: 7997a29f-0ceb-4cc7-9adf-840c57b4ae6f ║
nousresearch-meta-llama-4941-v60-mkmlizer: ║ Expiration: 2024-07-15 23:59:59 ║
nousresearch-meta-llama-4941-v60-mkmlizer: ║ ║
nousresearch-meta-llama-4941-v60-mkmlizer: ╚═════════════════════════════════════════════════════════════════════╝
nousresearch-meta-llama-4941-v60-mkmlizer: Downloaded to shared memory in 26.798s
nousresearch-meta-llama-4941-v60-mkmlizer: quantizing model to /dev/shm/model_cache
nousresearch-meta-llama-4941-v60-mkmlizer: Saving flywheel model at /dev/shm/model_cache
nousresearch-meta-llama-4941-v60-mkmlizer: Loading 0: 0%| | 0/291 [00:00<?, ?it/s] Loading 0: 4%|▍ | 13/291 [00:00<00:02, 114.44it/s] Loading 0: 9%|▉ | 26/291 [00:00<00:02, 122.53it/s] Loading 0: 14%|█▎ | 40/291 [00:00<00:02, 122.92it/s] Loading 0: 19%|█▊ | 54/291 [00:00<00:01, 128.91it/s] Loading 0: 23%|██▎ | 67/291 [00:00<00:01, 124.81it/s] Loading 0: 27%|██▋ | 80/291 [00:00<00:01, 125.63it/s] Loading 0: 32%|███▏ | 93/291 [00:01<00:02, 69.52it/s] Loading 0: 36%|███▌ | 104/291 [00:01<00:02, 75.76it/s] Loading 0: 41%|████ | 120/291 [00:01<00:01, 92.50it/s] Loading 0: 45%|████▌ | 132/291 [00:01<00:01, 95.28it/s] Loading 0: 51%|█████ | 147/291 [00:01<00:01, 106.33it/s] Loading 0: 55%|█████▍ | 159/291 [00:01<00:01, 105.61it/s] Loading 0: 59%|█████▉ | 173/291 [00:01<00:01, 113.70it/s] Loading 0: 64%|██████▍ | 187/291 [00:02<00:01, 72.56it/s] Loading 0: 68%|██████▊ | 197/291 [00:02<00:01, 74.20it/s] Loading 0: 73%|███████▎ | 211/291 [00:02<00:00, 84.70it/s] Loading 0: 78%|███████▊ | 228/291 [00:02<00:00, 99.80it/s] Loading 0: 82%|████████▏ | 240/291 [00:02<00:00, 103.32it/s] Loading 0: 88%|████████▊ | 255/291 [00:02<00:00, 114.33it/s] Loading 0: 92%|█████████▏| 268/291 [00:02<00:00, 114.53it/s] Loading 0: 97%|█████████▋| 281/291 [00:02<00:00, 117.04it/s] Special tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained.
nousresearch-meta-llama-4941-v60-mkmlizer: quantized model in 25.650s
nousresearch-meta-llama-4941-v60-mkmlizer: Processed model NousResearch/Meta-Llama-3-8B-Instruct in 52.448s
nousresearch-meta-llama-4941-v60-mkmlizer: creating bucket guanaco-mkml-models
nousresearch-meta-llama-4941-v60-mkmlizer: Bucket 's3://guanaco-mkml-models/' created
nousresearch-meta-llama-4941-v60-mkmlizer: uploading /dev/shm/model_cache to s3://guanaco-mkml-models/nousresearch-meta-llama-4941-v60
nousresearch-meta-llama-4941-v60-mkmlizer: cp /dev/shm/model_cache/config.json s3://guanaco-mkml-models/nousresearch-meta-llama-4941-v60/config.json
nousresearch-meta-llama-4941-v60-mkmlizer: cp /dev/shm/model_cache/special_tokens_map.json s3://guanaco-mkml-models/nousresearch-meta-llama-4941-v60/special_tokens_map.json
nousresearch-meta-llama-4941-v60-mkmlizer: cp /dev/shm/model_cache/tokenizer_config.json s3://guanaco-mkml-models/nousresearch-meta-llama-4941-v60/tokenizer_config.json
nousresearch-meta-llama-4941-v60-mkmlizer: cp /dev/shm/model_cache/tokenizer.json s3://guanaco-mkml-models/nousresearch-meta-llama-4941-v60/tokenizer.json
nousresearch-meta-llama-4941-v60-mkmlizer: cp /dev/shm/model_cache/flywheel_model.0.safetensors s3://guanaco-mkml-models/nousresearch-meta-llama-4941-v60/flywheel_model.0.safetensors
nousresearch-meta-llama-4941-v60-mkmlizer: /opt/conda/lib/python3.10/site-packages/transformers/models/auto/auto_factory.py:468: FutureWarning: The `use_auth_token` argument is deprecated and will be removed in v5 of Transformers. Please use `token` instead.
nousresearch-meta-llama-4941-v60-mkmlizer: warnings.warn(
nousresearch-meta-llama-4941-v60-mkmlizer: Saving model to /tmp/reward_cache/reward.tensors
nousresearch-meta-llama-4941-v60-mkmlizer: Saving duration: 0.463s
nousresearch-meta-llama-4941-v60-mkmlizer: Processed model ChaiML/gpt2_medium_pairwise_60m_step_937500 in 7.981s
nousresearch-meta-llama-4941-v60-mkmlizer: creating bucket guanaco-reward-models
nousresearch-meta-llama-4941-v60-mkmlizer: Bucket 's3://guanaco-reward-models/' created
nousresearch-meta-llama-4941-v60-mkmlizer: uploading /tmp/reward_cache to s3://guanaco-reward-models/nousresearch-meta-llama-4941-v60_reward
nousresearch-meta-llama-4941-v60-mkmlizer: cp /tmp/reward_cache/config.json s3://guanaco-reward-models/nousresearch-meta-llama-4941-v60_reward/config.json
nousresearch-meta-llama-4941-v60-mkmlizer: cp /tmp/reward_cache/special_tokens_map.json s3://guanaco-reward-models/nousresearch-meta-llama-4941-v60_reward/special_tokens_map.json
nousresearch-meta-llama-4941-v60-mkmlizer: cp /tmp/reward_cache/tokenizer_config.json s3://guanaco-reward-models/nousresearch-meta-llama-4941-v60_reward/tokenizer_config.json
nousresearch-meta-llama-4941-v60-mkmlizer: cp /tmp/reward_cache/merges.txt s3://guanaco-reward-models/nousresearch-meta-llama-4941-v60_reward/merges.txt
nousresearch-meta-llama-4941-v60-mkmlizer: cp /tmp/reward_cache/vocab.json s3://guanaco-reward-models/nousresearch-meta-llama-4941-v60_reward/vocab.json
nousresearch-meta-llama-4941-v60-mkmlizer: cp /tmp/reward_cache/tokenizer.json s3://guanaco-reward-models/nousresearch-meta-llama-4941-v60_reward/tokenizer.json
nousresearch-meta-llama-4941-v60-mkmlizer: cp /tmp/reward_cache/reward.tensors s3://guanaco-reward-models/nousresearch-meta-llama-4941-v60_reward/reward.tensors
Job nousresearch-meta-llama-4941-v60-mkmlizer completed after 231.32s with status: succeeded
Stopping job with name nousresearch-meta-llama-4941-v60-mkmlizer
Pipeline stage MKMLizer completed in 233.59s
Running pipeline stage MKMLKubeTemplater
Pipeline stage MKMLKubeTemplater completed in 0.48s
Running pipeline stage ISVCDeployer
Creating inference service nousresearch-meta-llama-4941-v60
Waiting for inference service nousresearch-meta-llama-4941-v60 to be ready
Inference service nousresearch-meta-llama-4941-v60 ready after 41.41575598716736s
Pipeline stage ISVCDeployer completed in 43.19s
Running pipeline stage StressChecker
HTTP Request: %s %s "%s %d %s"
Received healthy response to inference request in 2.5990710258483887s
HTTP Request: %s %s "%s %d %s"
Received healthy response to inference request in 2.7198100090026855s
HTTP Request: %s %s "%s %d %s"
Received healthy response to inference request in 2.6279349327087402s
HTTP Request: %s %s "%s %d %s"
Received healthy response to inference request in 2.67278790473938s
HTTP Request: %s %s "%s %d %s"
Received healthy response to inference request in 1.6870617866516113s
5 requests
0 failed requests
5th percentile: 1.8694636344909668
10th percentile: 2.051865482330322
20th percentile: 2.416669178009033
30th percentile: 2.604843807220459
40th percentile: 2.6163893699645997
50th percentile: 2.6279349327087402
60th percentile: 2.6458761215209963
70th percentile: 2.663817310333252
80th percentile: 2.682192325592041
90th percentile: 2.7010011672973633
95th percentile: 2.7104055881500244
99th percentile: 2.7179291248321533
mean time: 2.461333131790161
Pipeline stage StressChecker completed in 17.21s
nousresearch-meta-llama_4941_v60 status is now deployed due to DeploymentManager action
nousresearch-meta-llama_4941_v60 status is now inactive due to auto deactivation removed underperforming models

Usage Metrics

Latency Metrics