submission_id: neversleep-noromaid-v0_8068_v142
developer_uid: chai_backend_admin
alignment_samples: 340779
alignment_score: 0.7120439747573355
best_of: 4
celo_rating: 1181.98
display_name: neversleep-noromaid-v0_8068_v142
formatter: {'memory_template': '<s>[INST] This is an entertaining conversation. You are {bot_name} who has the persona: {memory}.\nPlay the role of {bot_name}. Engage in a chat with {user_name} while staying in character. You should create a fun dialogue which entertains {user_name}.\n', 'prompt_template': '{prompt}\n', 'bot_template': '{bot_name}: {message}</s>', 'user_template': '[INST] {user_name}: {message} [/INST]', 'response_template': '{bot_name}:', 'truncate_by_message': False}
generation_params: {'temperature': 1.0, 'top_p': 1.0, 'min_p': 0.0, 'top_k': 50, 'presence_penalty': 0.0, 'frequency_penalty': 0.0, 'stopping_words': ['\n', '</s>', '<|user|>', '###'], 'max_input_tokens': 512, 'best_of': 4, 'max_output_tokens': 64, 'reward_max_token_input': 256}
is_internal_developer: True
language_model: NeverSleep/Noromaid-v0.1-mixtral-8x7b-Instruct-v3
max_input_tokens: 512
max_output_tokens: 64
model_architecture: MixtralForCausalLM
model_group: NeverSleep/Noromaid-v0.1
model_name: neversleep-noromaid-v0_8068_v142
model_num_parameters: 46702792704.0
model_repo: NeverSleep/Noromaid-v0.1-mixtral-8x7b-Instruct-v3
model_size: 47B
num_battles: 348079
num_wins: 153905
propriety_score: 0.726011114082791
propriety_total_count: 33651.0
ranking_group: single
reward_formatter: {'bot_template': 'Bot: {message}\n', 'memory_template': '', 'prompt_template': '', 'response_template': 'Bot:', 'truncate_by_message': False, 'user_template': 'User: {message}\n'}
reward_repo: ChaiML/gpt2_xl_pairwise_89m_step_347634
status: torndown
submission_type: basic
timestamp: 2024-07-31T01:41:07+00:00
us_pacific_date: 2024-07-30
win_ratio: 0.4421553727745713
Download Preference Data
Resubmit model
Running pipeline stage MKMLizer
Starting job with name neversleep-noromaid-v0-8068-v142-mkmlizer
Waiting for job on neversleep-noromaid-v0-8068-v142-mkmlizer to finish
neversleep-noromaid-v0-8068-v142-mkmlizer: ╔═════════════════════════════════════════════════════════════════════╗
neversleep-noromaid-v0-8068-v142-mkmlizer: ║ _____ __ __ ║
neversleep-noromaid-v0-8068-v142-mkmlizer: ║ / _/ /_ ___ __/ / ___ ___ / / ║
neversleep-noromaid-v0-8068-v142-mkmlizer: ║ / _/ / // / |/|/ / _ \/ -_) -_) / ║
neversleep-noromaid-v0-8068-v142-mkmlizer: ║ /_//_/\_, /|__,__/_//_/\__/\__/_/ ║
neversleep-noromaid-v0-8068-v142-mkmlizer: ║ /___/ ║
neversleep-noromaid-v0-8068-v142-mkmlizer: ║ ║
neversleep-noromaid-v0-8068-v142-mkmlizer: ║ Version: 0.9.7 ║
neversleep-noromaid-v0-8068-v142-mkmlizer: ║ Copyright 2023 MK ONE TECHNOLOGIES Inc. ║
neversleep-noromaid-v0-8068-v142-mkmlizer: ║ https://mk1.ai ║
neversleep-noromaid-v0-8068-v142-mkmlizer: ║ ║
neversleep-noromaid-v0-8068-v142-mkmlizer: ║ The license key for the current software has been verified as ║
neversleep-noromaid-v0-8068-v142-mkmlizer: ║ belonging to: ║
neversleep-noromaid-v0-8068-v142-mkmlizer: ║ ║
neversleep-noromaid-v0-8068-v142-mkmlizer: ║ Chai Research Corp. ║
neversleep-noromaid-v0-8068-v142-mkmlizer: ║ Account ID: 7997a29f-0ceb-4cc7-9adf-840c57b4ae6f ║
neversleep-noromaid-v0-8068-v142-mkmlizer: ║ Expiration: 2024-10-15 23:59:59 ║
neversleep-noromaid-v0-8068-v142-mkmlizer: ║ ║
neversleep-noromaid-v0-8068-v142-mkmlizer: ╚═════════════════════════════════════════════════════════════════════╝
HTTP Request: %s %s "%s %d %s"
HTTP Request: %s %s "%s %d %s"
HTTP Request: %s %s "%s %d %s"
neversleep-noromaid-v0-8068-v142-mkmlizer: Downloaded to shared memory in 174.491s
neversleep-noromaid-v0-8068-v142-mkmlizer: quantizing model to /dev/shm/model_cache, profile:s0, folder:/tmp/tmpbojge2_f, device:0
neversleep-noromaid-v0-8068-v142-mkmlizer: Saving flywheel model at /dev/shm/model_cache
neversleep-noromaid-v0-8068-v142-mkmlizer: quantized model in 88.938s
neversleep-noromaid-v0-8068-v142-mkmlizer: Processed model NeverSleep/Noromaid-v0.1-mixtral-8x7b-Instruct-v3 in 263.429s
neversleep-noromaid-v0-8068-v142-mkmlizer: creating bucket guanaco-mkml-models
neversleep-noromaid-v0-8068-v142-mkmlizer: Bucket 's3://guanaco-mkml-models/' created
neversleep-noromaid-v0-8068-v142-mkmlizer: uploading /dev/shm/model_cache to s3://guanaco-mkml-models/neversleep-noromaid-v0-8068-v142
neversleep-noromaid-v0-8068-v142-mkmlizer: cp /dev/shm/model_cache/config.json s3://guanaco-mkml-models/neversleep-noromaid-v0-8068-v142/config.json
neversleep-noromaid-v0-8068-v142-mkmlizer: cp /dev/shm/model_cache/special_tokens_map.json s3://guanaco-mkml-models/neversleep-noromaid-v0-8068-v142/special_tokens_map.json
neversleep-noromaid-v0-8068-v142-mkmlizer: cp /dev/shm/model_cache/tokenizer_config.json s3://guanaco-mkml-models/neversleep-noromaid-v0-8068-v142/tokenizer_config.json
neversleep-noromaid-v0-8068-v142-mkmlizer: cp /dev/shm/model_cache/tokenizer.model s3://guanaco-mkml-models/neversleep-noromaid-v0-8068-v142/tokenizer.model
neversleep-noromaid-v0-8068-v142-mkmlizer: cp /dev/shm/model_cache/tokenizer.json s3://guanaco-mkml-models/neversleep-noromaid-v0-8068-v142/tokenizer.json
neversleep-noromaid-v0-8068-v142-mkmlizer: cp /dev/shm/model_cache/flywheel_model.3.safetensors s3://guanaco-mkml-models/neversleep-noromaid-v0-8068-v142/flywheel_model.3.safetensors
neversleep-noromaid-v0-8068-v142-mkmlizer: cp /dev/shm/model_cache/flywheel_model.2.safetensors s3://guanaco-mkml-models/neversleep-noromaid-v0-8068-v142/flywheel_model.2.safetensors
neversleep-noromaid-v0-8068-v142-mkmlizer: cp /dev/shm/model_cache/flywheel_model.0.safetensors s3://guanaco-mkml-models/neversleep-noromaid-v0-8068-v142/flywheel_model.0.safetensors
neversleep-noromaid-v0-8068-v142-mkmlizer: cp /dev/shm/model_cache/flywheel_model.1.safetensors s3://guanaco-mkml-models/neversleep-noromaid-v0-8068-v142/flywheel_model.1.safetensors
neversleep-noromaid-v0-8068-v142-mkmlizer: /opt/conda/lib/python3.10/site-packages/transformers/models/auto/tokenization_auto.py:785: FutureWarning: The `use_auth_token` argument is deprecated and will be removed in v5 of Transformers. Please use `token` instead.
neversleep-noromaid-v0-8068-v142-mkmlizer: warnings.warn(
neversleep-noromaid-v0-8068-v142-mkmlizer: /opt/conda/lib/python3.10/site-packages/transformers/models/auto/auto_factory.py:469: FutureWarning: The `use_auth_token` argument is deprecated and will be removed in v5 of Transformers. Please use `token` instead.
neversleep-noromaid-v0-8068-v142-mkmlizer: warnings.warn(
neversleep-noromaid-v0-8068-v142-mkmlizer: Downloading shards: 0%| | 0/2 [00:00<?, ?it/s] Downloading shards: 50%|█████ | 1/2 [00:05<00:05, 5.49s/it] Downloading shards: 100%|██████████| 2/2 [00:08<00:00, 4.23s/it] Downloading shards: 100%|██████████| 2/2 [00:08<00:00, 4.42s/it]
neversleep-noromaid-v0-8068-v142-mkmlizer: Loading checkpoint shards: 0%| | 0/2 [00:00<?, ?it/s] Loading checkpoint shards: 50%|█████ | 1/2 [00:00<00:00, 2.41it/s] Loading checkpoint shards: 100%|██████████| 2/2 [00:00<00:00, 3.99it/s] Loading checkpoint shards: 100%|██████████| 2/2 [00:00<00:00, 3.63it/s]
neversleep-noromaid-v0-8068-v142-mkmlizer: Saving model to /tmp/reward_cache/reward.tensors
neversleep-noromaid-v0-8068-v142-mkmlizer: Saving duration: 1.339s
neversleep-noromaid-v0-8068-v142-mkmlizer: Processed model ChaiML/gpt2_xl_pairwise_89m_step_347634 in 14.027s
neversleep-noromaid-v0-8068-v142-mkmlizer: creating bucket guanaco-reward-models
neversleep-noromaid-v0-8068-v142-mkmlizer: Bucket 's3://guanaco-reward-models/' created
neversleep-noromaid-v0-8068-v142-mkmlizer: uploading /tmp/reward_cache to s3://guanaco-reward-models/neversleep-noromaid-v0-8068-v142_reward
neversleep-noromaid-v0-8068-v142-mkmlizer: cp /tmp/reward_cache/config.json s3://guanaco-reward-models/neversleep-noromaid-v0-8068-v142_reward/config.json
neversleep-noromaid-v0-8068-v142-mkmlizer: cp /tmp/reward_cache/tokenizer_config.json s3://guanaco-reward-models/neversleep-noromaid-v0-8068-v142_reward/tokenizer_config.json
neversleep-noromaid-v0-8068-v142-mkmlizer: cp /tmp/reward_cache/special_tokens_map.json s3://guanaco-reward-models/neversleep-noromaid-v0-8068-v142_reward/special_tokens_map.json
neversleep-noromaid-v0-8068-v142-mkmlizer: cp /tmp/reward_cache/merges.txt s3://guanaco-reward-models/neversleep-noromaid-v0-8068-v142_reward/merges.txt
neversleep-noromaid-v0-8068-v142-mkmlizer: cp /tmp/reward_cache/vocab.json s3://guanaco-reward-models/neversleep-noromaid-v0-8068-v142_reward/vocab.json
neversleep-noromaid-v0-8068-v142-mkmlizer: cp /tmp/reward_cache/tokenizer.json s3://guanaco-reward-models/neversleep-noromaid-v0-8068-v142_reward/tokenizer.json
neversleep-noromaid-v0-8068-v142-mkmlizer: cp /tmp/reward_cache/reward.tensors s3://guanaco-reward-models/neversleep-noromaid-v0-8068-v142_reward/reward.tensors
Job neversleep-noromaid-v0-8068-v142-mkmlizer completed after 336.19s with status: succeeded
Stopping job with name neversleep-noromaid-v0-8068-v142-mkmlizer
Pipeline stage MKMLizer completed in 338.89s
Running pipeline stage MKMLKubeTemplater
Pipeline stage MKMLKubeTemplater completed in 0.43s
Running pipeline stage ISVCDeployer
Creating inference service neversleep-noromaid-v0-8068-v142
Waiting for inference service neversleep-noromaid-v0-8068-v142 to be ready
HTTP Request: %s %s "%s %d %s"
Inference service neversleep-noromaid-v0-8068-v142 ready after 122.40167021751404s
Pipeline stage ISVCDeployer completed in 123.87s
Running pipeline stage StressChecker
HTTP Request: %s %s "%s %d %s"
Received healthy response to inference request in 3.826902151107788s
HTTP Request: %s %s "%s %d %s"
Received healthy response to inference request in 2.8084871768951416s
HTTP Request: %s %s "%s %d %s"
Received healthy response to inference request in 2.8024678230285645s
HTTP Request: %s %s "%s %d %s"
Received healthy response to inference request in 2.623591184616089s
HTTP Request: %s %s "%s %d %s"
Received healthy response to inference request in 2.6518802642822266s
5 requests
0 failed requests
5th percentile: 2.6292490005493163
10th percentile: 2.6349068164825438
20th percentile: 2.646222448348999
30th percentile: 2.681997776031494
40th percentile: 2.7422327995300293
50th percentile: 2.8024678230285645
60th percentile: 2.8048755645751955
70th percentile: 2.807283306121826
80th percentile: 3.012170171737671
90th percentile: 3.4195361614227298
95th percentile: 3.6232191562652587
99th percentile: 3.7861655521392823
mean time: 2.942665719985962
Pipeline stage StressChecker completed in 18.31s
neversleep-noromaid-v0_8068_v142 status is now deployed due to DeploymentManager action
neversleep-noromaid-v0_8068_v142 status is now inactive due to admin request
neversleep-noromaid-v0_8068_v142 status is now torndown due to DeploymentManager action

Usage Metrics

Latency Metrics