submission_id: neversleep-noromaid-v0-_8068_v87
developer_uid: end_to_end_test
status: inactive
model_repo: NeverSleep/Noromaid-v0.1-mixtral-8x7b-Instruct-v3
reward_repo: ChaiML/reward_models_100_170000000_cp_498032
generation_params: {'temperature': 1.0, 'top_p': 0.99, 'min_p': 0.1, 'top_k': 40, 'presence_penalty': 0.0, 'frequency_penalty': 0.0, 'stopping_words': ['\n'], 'max_input_tokens': 512, 'best_of': 4, 'max_output_tokens': 64}
formatter: {'memory_template': 'character: {bot_name} {memory}\n', 'prompt_template': '{prompt}', 'bot_template': '{bot_name}: {message}', 'user_template': '{user_name}: {message}', 'response_template': '{bot_name}:', 'truncate_by_message': False}
reward_formatter: {'memory_template': 'character: {bot_name} {memory}\n', 'prompt_template': '{prompt}', 'bot_template': '{bot_name}: {message}', 'user_template': '{user_name}: {message}', 'response_template': '{bot_name}:', 'truncate_by_message': False}
timestamp: 2024-07-01T19:56:33+00:00
model_name: neversleep-noromaid-v0-_8068_v87
model_group: NeverSleep/Noromaid-v0.1
num_battles: 12582
num_wins: 5792
celo_rating: 1149.92
propriety_score: 0.7302586793737236
propriety_total_count: 5876.0
submission_type: basic
model_architecture: MixtralForCausalLM
model_num_parameters: 46702792704.0
best_of: 4
max_input_tokens: 512
max_output_tokens: 64
display_name: neversleep-noromaid-v0-_8068_v87
ineligible_reason: None
language_model: NeverSleep/Noromaid-v0.1-mixtral-8x7b-Instruct-v3
model_size: 47B
reward_model: ChaiML/reward_models_100_170000000_cp_498032
us_pacific_date: 2024-07-01
win_ratio: 0.4603401684946749
Resubmit model
Running pipeline stage MKMLizer
Starting job with name neversleep-noromaid-v0-8068-v87-mkmlizer
Waiting for job on neversleep-noromaid-v0-8068-v87-mkmlizer to finish
neversleep-noromaid-v0-8068-v87-mkmlizer: ╔═════════════════════════════════════════════════════════════════════╗
neversleep-noromaid-v0-8068-v87-mkmlizer: ║ _____ __ __ ║
neversleep-noromaid-v0-8068-v87-mkmlizer: ║ / _/ /_ ___ __/ / ___ ___ / / ║
neversleep-noromaid-v0-8068-v87-mkmlizer: ║ / _/ / // / |/|/ / _ \/ -_) -_) / ║
neversleep-noromaid-v0-8068-v87-mkmlizer: ║ /_//_/\_, /|__,__/_//_/\__/\__/_/ ║
neversleep-noromaid-v0-8068-v87-mkmlizer: ║ /___/ ║
neversleep-noromaid-v0-8068-v87-mkmlizer: ║ ║
neversleep-noromaid-v0-8068-v87-mkmlizer: ║ Version: 0.8.14 ║
neversleep-noromaid-v0-8068-v87-mkmlizer: ║ Copyright 2023 MK ONE TECHNOLOGIES Inc. ║
neversleep-noromaid-v0-8068-v87-mkmlizer: ║ https://mk1.ai ║
neversleep-noromaid-v0-8068-v87-mkmlizer: ║ ║
neversleep-noromaid-v0-8068-v87-mkmlizer: ║ The license key for the current software has been verified as ║
neversleep-noromaid-v0-8068-v87-mkmlizer: ║ belonging to: ║
neversleep-noromaid-v0-8068-v87-mkmlizer: ║ ║
neversleep-noromaid-v0-8068-v87-mkmlizer: ║ Chai Research Corp. ║
neversleep-noromaid-v0-8068-v87-mkmlizer: ║ Account ID: 7997a29f-0ceb-4cc7-9adf-840c57b4ae6f ║
neversleep-noromaid-v0-8068-v87-mkmlizer: ║ Expiration: 2024-07-15 23:59:59 ║
neversleep-noromaid-v0-8068-v87-mkmlizer: ║ ║
neversleep-noromaid-v0-8068-v87-mkmlizer: ╚═════════════════════════════════════════════════════════════════════╝
neversleep-noromaid-v0-8068-v87-mkmlizer: Downloaded to shared memory in 193.531s
neversleep-noromaid-v0-8068-v87-mkmlizer: quantizing model to /dev/shm/model_cache
neversleep-noromaid-v0-8068-v87-mkmlizer: Saving flywheel model at /dev/shm/model_cache
neversleep-noromaid-v0-8068-v87-mkmlizer: quantized model in 114.041s
neversleep-noromaid-v0-8068-v87-mkmlizer: Processed model NeverSleep/Noromaid-v0.1-mixtral-8x7b-Instruct-v3 in 307.573s
neversleep-noromaid-v0-8068-v87-mkmlizer: creating bucket guanaco-mkml-models
neversleep-noromaid-v0-8068-v87-mkmlizer: Bucket 's3://guanaco-mkml-models/' created
neversleep-noromaid-v0-8068-v87-mkmlizer: uploading /dev/shm/model_cache to s3://guanaco-mkml-models/neversleep-noromaid-v0-8068-v87
neversleep-noromaid-v0-8068-v87-mkmlizer: cp /dev/shm/model_cache/special_tokens_map.json s3://guanaco-mkml-models/neversleep-noromaid-v0-8068-v87/special_tokens_map.json
neversleep-noromaid-v0-8068-v87-mkmlizer: cp /dev/shm/model_cache/tokenizer.json s3://guanaco-mkml-models/neversleep-noromaid-v0-8068-v87/tokenizer.json
neversleep-noromaid-v0-8068-v87-mkmlizer: cp /dev/shm/model_cache/tokenizer.model s3://guanaco-mkml-models/neversleep-noromaid-v0-8068-v87/tokenizer.model
neversleep-noromaid-v0-8068-v87-mkmlizer: cp /dev/shm/model_cache/config.json s3://guanaco-mkml-models/neversleep-noromaid-v0-8068-v87/config.json
neversleep-noromaid-v0-8068-v87-mkmlizer: cp /dev/shm/model_cache/tokenizer_config.json s3://guanaco-mkml-models/neversleep-noromaid-v0-8068-v87/tokenizer_config.json
neversleep-noromaid-v0-8068-v87-mkmlizer: cp /dev/shm/model_cache/flywheel_model.3.safetensors s3://guanaco-mkml-models/neversleep-noromaid-v0-8068-v87/flywheel_model.3.safetensors
neversleep-noromaid-v0-8068-v87-mkmlizer: cp /dev/shm/model_cache/flywheel_model.0.safetensors s3://guanaco-mkml-models/neversleep-noromaid-v0-8068-v87/flywheel_model.0.safetensors
neversleep-noromaid-v0-8068-v87-mkmlizer: cp /dev/shm/model_cache/flywheel_model.2.safetensors s3://guanaco-mkml-models/neversleep-noromaid-v0-8068-v87/flywheel_model.2.safetensors
neversleep-noromaid-v0-8068-v87-mkmlizer: cp /dev/shm/model_cache/flywheel_model.1.safetensors s3://guanaco-mkml-models/neversleep-noromaid-v0-8068-v87/flywheel_model.1.safetensors
neversleep-noromaid-v0-8068-v87-mkmlizer: loading reward model from ChaiML/reward_models_100_170000000_cp_498032
neversleep-noromaid-v0-8068-v87-mkmlizer: Loading 0: 0%| | 0/995 [00:00<?, ?it/s] Loading 0: 5%|▌ | 52/995 [00:05<01:42, 9.21it/s] Loading 0: 11%|█ | 107/995 [00:10<01:29, 9.91it/s] Loading 0: 16%|█▋ | 162/995 [00:16<01:23, 9.93it/s] Loading 0: 21%|██ | 210/995 [00:20<01:14, 10.47it/s] Loading 0: 27%|██▋ | 265/995 [00:21<00:49, 14.67it/s] Loading 0: 28%|██▊ | 277/995 [00:42<00:48, 14.67it/s] Loading 0: 28%|██▊ | 278/995 [00:42<02:54, 4.11it/s] Loading 0: 32%|███▏ | 320/995 [00:45<02:01, 5.58it/s] Loading 0: 37%|███▋ | 368/995 [00:46<01:17, 8.12it/s] Loading 0: 43%|████▎ | 423/995 [00:47<00:48, 11.81it/s] Loading 0: 48%|████▊ | 478/995 [00:48<00:31, 16.23it/s] Loading 0: 53%|█████▎ | 526/995 [00:49<00:22, 20.40it/s] Loading 0: 57%|█████▋ | 564/995 [01:03<00:21, 20.40it/s] Loading 0: 57%|█████▋ | 565/995 [01:11<01:18, 5.51it/s] Loading 0: 58%|█████▊ | 581/995 [01:12<01:09, 5.94it/s] Loading 0: 64%|██████▍ | 636/995 [01:13<00:39, 9.03it/s] Loading 0: 69%|██████▉ | 691/995 [01:15<00:24, 12.64it/s] Loading 0: 74%|███████▍ | 739/995 [01:16<00:15, 16.02it/s] Loading 0: 80%|███████▉ | 794/995 [01:17<00:09, 20.37it/s] Loading 0: 85%|████████▍ | 845/995 [01:34<00:07, 20.37it/s] Loading 0: 85%|████████▌ | 846/995 [01:34<00:20, 7.30it/s] Loading 0: 85%|████████▌ | 849/995 [01:36<00:21, 6.90it/s] Loading 0: 90%|█████████ | 897/995 [01:37<00:09, 9.90it/s] Loading 0: 96%|█████████▌| 952/995 [01:38<00:03, 14.24it/s] Loading 0: 100%|██████████| 995/995 [01:39<00:00, 16.96it/s] /opt/conda/lib/python3.10/site-packages/transformers/models/auto/configuration_auto.py:919: FutureWarning: The `use_auth_token` argument is deprecated and will be removed in v5 of Transformers. Please use `token` instead.
neversleep-noromaid-v0-8068-v87-mkmlizer: warnings.warn(
neversleep-noromaid-v0-8068-v87-mkmlizer: /opt/conda/lib/python3.10/site-packages/huggingface_hub/file_download.py:1132: FutureWarning: `resume_download` is deprecated and will be removed in version 1.0.0. Downloads always resume when possible. If you want to force a new download, use `force_download=True`.
neversleep-noromaid-v0-8068-v87-mkmlizer: warnings.warn(
neversleep-noromaid-v0-8068-v87-mkmlizer: /opt/conda/lib/python3.10/site-packages/transformers/models/auto/tokenization_auto.py:769: FutureWarning: The `use_auth_token` argument is deprecated and will be removed in v5 of Transformers. Please use `token` instead.
neversleep-noromaid-v0-8068-v87-mkmlizer: warnings.warn(
neversleep-noromaid-v0-8068-v87-mkmlizer: /opt/conda/lib/python3.10/site-packages/transformers/models/auto/auto_factory.py:468: FutureWarning: The `use_auth_token` argument is deprecated and will be removed in v5 of Transformers. Please use `token` instead.
neversleep-noromaid-v0-8068-v87-mkmlizer: warnings.warn(
neversleep-noromaid-v0-8068-v87-mkmlizer: /opt/conda/lib/python3.10/site-packages/torch/_utils.py:831: UserWarning: TypedStorage is deprecated. It will be removed in the future and UntypedStorage will be the only storage class. This should only matter to you if you are using storages directly. To access UntypedStorage directly, use tensor.untyped_storage() instead of tensor.storage()
neversleep-noromaid-v0-8068-v87-mkmlizer: return self.fget.__get__(instance, owner)()
neversleep-noromaid-v0-8068-v87-mkmlizer: Saving model to /tmp/reward_cache/reward.tensors
neversleep-noromaid-v0-8068-v87-mkmlizer: Saving duration: 0.164s
neversleep-noromaid-v0-8068-v87-mkmlizer: Processed model ChaiML/reward_models_100_170000000_cp_498032 in 4.220s
neversleep-noromaid-v0-8068-v87-mkmlizer: creating bucket guanaco-reward-models
neversleep-noromaid-v0-8068-v87-mkmlizer: Bucket 's3://guanaco-reward-models/' created
neversleep-noromaid-v0-8068-v87-mkmlizer: uploading /tmp/reward_cache to s3://guanaco-reward-models/neversleep-noromaid-v0-8068-v87_reward
neversleep-noromaid-v0-8068-v87-mkmlizer: cp /tmp/reward_cache/config.json s3://guanaco-reward-models/neversleep-noromaid-v0-8068-v87_reward/config.json
neversleep-noromaid-v0-8068-v87-mkmlizer: cp /tmp/reward_cache/special_tokens_map.json s3://guanaco-reward-models/neversleep-noromaid-v0-8068-v87_reward/special_tokens_map.json
neversleep-noromaid-v0-8068-v87-mkmlizer: cp /tmp/reward_cache/tokenizer_config.json s3://guanaco-reward-models/neversleep-noromaid-v0-8068-v87_reward/tokenizer_config.json
neversleep-noromaid-v0-8068-v87-mkmlizer: cp /tmp/reward_cache/merges.txt s3://guanaco-reward-models/neversleep-noromaid-v0-8068-v87_reward/merges.txt
neversleep-noromaid-v0-8068-v87-mkmlizer: cp /tmp/reward_cache/vocab.json s3://guanaco-reward-models/neversleep-noromaid-v0-8068-v87_reward/vocab.json
neversleep-noromaid-v0-8068-v87-mkmlizer: cp /tmp/reward_cache/tokenizer.json s3://guanaco-reward-models/neversleep-noromaid-v0-8068-v87_reward/tokenizer.json
neversleep-noromaid-v0-8068-v87-mkmlizer: cp /tmp/reward_cache/reward.tensors s3://guanaco-reward-models/neversleep-noromaid-v0-8068-v87_reward/reward.tensors
Job neversleep-noromaid-v0-8068-v87-mkmlizer completed after 358.21s with status: succeeded
Stopping job with name neversleep-noromaid-v0-8068-v87-mkmlizer
Pipeline stage MKMLizer completed in 359.89s
Running pipeline stage MKMLKubeTemplater
Pipeline stage MKMLKubeTemplater completed in 0.46s
Running pipeline stage ISVCDeployer
Creating inference service neversleep-noromaid-v0-8068-v87
Waiting for inference service neversleep-noromaid-v0-8068-v87 to be ready
Inference service neversleep-noromaid-v0-8068-v87 ready after 60.87680649757385s
Pipeline stage ISVCDeployer completed in 67.43s
Running pipeline stage StressChecker
Received healthy response to inference request in 2.7792842388153076s
Received healthy response to inference request in 2.228625774383545s
Received healthy response to inference request in 1.930753231048584s
Received healthy response to inference request in 1.6370632648468018s
Received healthy response to inference request in 2.2959084510803223s
5 requests
0 failed requests
5th percentile: 1.6958012580871582
10th percentile: 1.7545392513275146
20th percentile: 1.8720152378082275
30th percentile: 1.990327739715576
40th percentile: 2.1094767570495607
50th percentile: 2.228625774383545
60th percentile: 2.2555388450622558
70th percentile: 2.2824519157409666
80th percentile: 2.3925836086273193
90th percentile: 2.5859339237213135
95th percentile: 2.6826090812683105
99th percentile: 2.759949207305908
mean time: 2.174326992034912
Pipeline stage StressChecker completed in 14.79s
neversleep-noromaid-v0-_8068_v87 status is now deployed due to DeploymentManager action
neversleep-noromaid-v0-_8068_v87 status is now inactive due to auto deactivation removed underperforming models

Usage Metrics

Latency Metrics