submission_id: neversleep-noromaid-v0_8068_v133
developer_uid: chai_backend_admin
status: inactive
model_repo: NeverSleep/Noromaid-v0.1-mixtral-8x7b-Instruct-v3
reward_repo: ChaiML/gpt2_xl_pairwise_89m_step_347634
generation_params: {'temperature': 1.0, 'top_p': 1.0, 'min_p': 0.0, 'top_k': 50, 'presence_penalty': 0.0, 'frequency_penalty': 0.0, 'stopping_words': ['\n', '</s>', '<|user|>', '###'], 'max_input_tokens': 512, 'best_of': 4, 'max_output_tokens': 64}
formatter: {'memory_template': '<s>[INST] This is an entertaining conversation. You are {bot_name} who has the persona: {memory}.\nPlay the role of {bot_name}. Engage in a chat with {user_name} while staying in character. You should create a fun dialogue which entertains {user_name}.\n', 'prompt_template': '{prompt}\n', 'bot_template': '{bot_name}: {message}</s>', 'user_template': '[INST] {user_name}: {message} [/INST]', 'response_template': '{bot_name}:', 'truncate_by_message': False}
reward_formatter: {'memory_template': 'Memory: {memory}\n', 'prompt_template': '{prompt}\n', 'bot_template': 'Bot: {message}\n', 'user_template': 'User: {message}\n', 'response_template': 'Bot:', 'truncate_by_message': False}
timestamp: 2024-07-11T21:10:20+00:00
model_name: neversleep-noromaid-v0_8068_v133
model_group: NeverSleep/Noromaid-v0.1
num_battles: 5209302
num_wins: 2476720
celo_rating: 1184.13
alignment_score: None
alignment_samples: 0
propriety_score: 0.7318193711918802
propriety_total_count: 561793.0
submission_type: basic
model_architecture: MixtralForCausalLM
model_num_parameters: 46702792704.0
best_of: 4
max_input_tokens: 512
max_output_tokens: 64
display_name: neversleep-noromaid-v0_8068_v133
ineligible_reason: None
language_model: NeverSleep/Noromaid-v0.1-mixtral-8x7b-Instruct-v3
model_size: 47B
reward_model: ChaiML/gpt2_xl_pairwise_89m_step_347634
us_pacific_date: 2024-07-11
win_ratio: 0.4754418154293992
preference_data_url: None
Resubmit model
Running pipeline stage MKMLizer
Starting job with name neversleep-noromaid-v0-8068-v133-mkmlizer
Waiting for job on neversleep-noromaid-v0-8068-v133-mkmlizer to finish
neversleep-noromaid-v0-8068-v133-mkmlizer: ╔═════════════════════════════════════════════════════════════════════╗
neversleep-noromaid-v0-8068-v133-mkmlizer: ║ _____ __ __ ║
neversleep-noromaid-v0-8068-v133-mkmlizer: ║ / _/ /_ ___ __/ / ___ ___ / / ║
neversleep-noromaid-v0-8068-v133-mkmlizer: ║ / _/ / // / |/|/ / _ \/ -_) -_) / ║
neversleep-noromaid-v0-8068-v133-mkmlizer: ║ /_//_/\_, /|__,__/_//_/\__/\__/_/ ║
neversleep-noromaid-v0-8068-v133-mkmlizer: ║ /___/ ║
neversleep-noromaid-v0-8068-v133-mkmlizer: ║ ║
neversleep-noromaid-v0-8068-v133-mkmlizer: ║ Version: 0.9.5.post1 ║
neversleep-noromaid-v0-8068-v133-mkmlizer: ║ Copyright 2023 MK ONE TECHNOLOGIES Inc. ║
neversleep-noromaid-v0-8068-v133-mkmlizer: ║ https://mk1.ai ║
neversleep-noromaid-v0-8068-v133-mkmlizer: ║ ║
neversleep-noromaid-v0-8068-v133-mkmlizer: ║ The license key for the current software has been verified as ║
neversleep-noromaid-v0-8068-v133-mkmlizer: ║ belonging to: ║
neversleep-noromaid-v0-8068-v133-mkmlizer: ║ ║
neversleep-noromaid-v0-8068-v133-mkmlizer: ║ Chai Research Corp. ║
neversleep-noromaid-v0-8068-v133-mkmlizer: ║ Account ID: 7997a29f-0ceb-4cc7-9adf-840c57b4ae6f ║
neversleep-noromaid-v0-8068-v133-mkmlizer: ║ Expiration: 2024-10-15 23:59:59 ║
neversleep-noromaid-v0-8068-v133-mkmlizer: ║ ║
neversleep-noromaid-v0-8068-v133-mkmlizer: ╚═════════════════════════════════════════════════════════════════════╝
neversleep-noromaid-v0-8068-v133-mkmlizer: Downloaded to shared memory in 138.905s
neversleep-noromaid-v0-8068-v133-mkmlizer: quantizing model to /dev/shm/model_cache
neversleep-noromaid-v0-8068-v133-mkmlizer: Saving flywheel model at /dev/shm/model_cache
neversleep-noromaid-v0-8068-v133-mkmlizer: quantized model in 102.583s
neversleep-noromaid-v0-8068-v133-mkmlizer: Processed model NeverSleep/Noromaid-v0.1-mixtral-8x7b-Instruct-v3 in 241.488s
neversleep-noromaid-v0-8068-v133-mkmlizer: creating bucket guanaco-mkml-models
neversleep-noromaid-v0-8068-v133-mkmlizer: Bucket 's3://guanaco-mkml-models/' created
neversleep-noromaid-v0-8068-v133-mkmlizer: uploading /dev/shm/model_cache to s3://guanaco-mkml-models/neversleep-noromaid-v0-8068-v133
neversleep-noromaid-v0-8068-v133-mkmlizer: cp /dev/shm/model_cache/tokenizer_config.json s3://guanaco-mkml-models/neversleep-noromaid-v0-8068-v133/tokenizer_config.json
neversleep-noromaid-v0-8068-v133-mkmlizer: cp /dev/shm/model_cache/tokenizer.model s3://guanaco-mkml-models/neversleep-noromaid-v0-8068-v133/tokenizer.model
neversleep-noromaid-v0-8068-v133-mkmlizer: cp /dev/shm/model_cache/tokenizer.json s3://guanaco-mkml-models/neversleep-noromaid-v0-8068-v133/tokenizer.json
neversleep-noromaid-v0-8068-v133-mkmlizer: cp /dev/shm/model_cache/flywheel_model.5.safetensors s3://guanaco-mkml-models/neversleep-noromaid-v0-8068-v133/flywheel_model.5.safetensors
neversleep-noromaid-v0-8068-v133-mkmlizer: cp /dev/shm/model_cache/flywheel_model.1.safetensors s3://guanaco-mkml-models/neversleep-noromaid-v0-8068-v133/flywheel_model.1.safetensors
neversleep-noromaid-v0-8068-v133-mkmlizer: cp /dev/shm/model_cache/flywheel_model.2.safetensors s3://guanaco-mkml-models/neversleep-noromaid-v0-8068-v133/flywheel_model.2.safetensors
neversleep-noromaid-v0-8068-v133-mkmlizer: cp /dev/shm/model_cache/flywheel_model.3.safetensors s3://guanaco-mkml-models/neversleep-noromaid-v0-8068-v133/flywheel_model.3.safetensors
neversleep-noromaid-v0-8068-v133-mkmlizer: cp /dev/shm/model_cache/flywheel_model.4.safetensors s3://guanaco-mkml-models/neversleep-noromaid-v0-8068-v133/flywheel_model.4.safetensors
neversleep-noromaid-v0-8068-v133-mkmlizer: cp /dev/shm/model_cache/flywheel_model.0.safetensors s3://guanaco-mkml-models/neversleep-noromaid-v0-8068-v133/flywheel_model.0.safetensors
neversleep-noromaid-v0-8068-v133-mkmlizer: loading reward model from ChaiML/gpt2_xl_pairwise_89m_step_347634
neversleep-noromaid-v0-8068-v133-mkmlizer: Loading 0: 0%| | 0/995 [00:00<?, ?it/s] Loading 0: 5%|▌ | 52/995 [00:01<00:25, 36.70it/s] Loading 0: 11%|█ | 107/995 [00:02<00:22, 39.11it/s] Loading 0: 16%|█▋ | 162/995 [00:04<00:20, 40.36it/s] Loading 0: 18%|█▊ | 182/995 [00:17<02:03, 6.59it/s] Loading 0: 21%|██ | 210/995 [00:19<01:37, 8.08it/s] Loading 0: 27%|██▋ | 265/995 [00:20<00:58, 12.56it/s] Loading 0: 32%|███▏ | 320/995 [00:21<00:39, 17.26it/s] Loading 0: 36%|███▌ | 355/995 [00:34<00:37, 17.26it/s] Loading 0: 36%|███▌ | 356/995 [00:34<01:26, 7.37it/s] Loading 0: 37%|███▋ | 368/995 [00:35<01:24, 7.45it/s] Loading 0: 43%|████▎ | 423/995 [00:37<00:49, 11.49it/s] Loading 0: 48%|████▊ | 478/995 [00:38<00:32, 15.91it/s] Loading 0: 53%|█████▎ | 523/995 [00:50<01:01, 7.73it/s] Loading 0: 53%|█████▎ | 526/995 [00:52<01:05, 7.20it/s] Loading 0: 58%|█████▊ | 581/995 [00:53<00:36, 11.21it/s] Loading 0: 64%|██████▍ | 635/995 [00:53<00:20, 17.68it/s] Loading 0: 66%|██████▌ | 654/995 [00:54<00:19, 17.30it/s] Loading 0: 69%|██████▉ | 691/995 [00:56<00:15, 19.59it/s] Loading 0: 71%|███████ | 702/995 [01:08<00:52, 5.63it/s] Loading 0: 74%|███████▍ | 739/995 [01:10<00:32, 7.86it/s] Loading 0: 80%|███████▉ | 794/995 [01:11<00:16, 12.20it/s] Loading 0: 85%|████████▌ | 849/995 [01:12<00:08, 16.93it/s] Loading 0: 87%|████████▋ | 863/995 [01:25<00:07, 16.93it/s] Loading 0: 87%|████████▋ | 864/995 [01:25<00:20, 6.24it/s] Loading 0: 90%|█████████ | 897/995 [01:26<00:12, 7.98it/s] Loading 0: 96%|█████████▌| 952/995 [01:27<00:03, 12.24it/s] Loading 0: 100%|██████████| 995/995 [01:29<00:00, 14.58it/s] /opt/conda/lib/python3.10/site-packages/transformers/models/auto/configuration_auto.py:950: FutureWarning: The `use_auth_token` argument is deprecated and will be removed in v5 of Transformers. Please use `token` instead.
neversleep-noromaid-v0-8068-v133-mkmlizer: warnings.warn(
neversleep-noromaid-v0-8068-v133-mkmlizer: /opt/conda/lib/python3.10/site-packages/transformers/models/auto/tokenization_auto.py:778: FutureWarning: The `use_auth_token` argument is deprecated and will be removed in v5 of Transformers. Please use `token` instead.
neversleep-noromaid-v0-8068-v133-mkmlizer: warnings.warn(
neversleep-noromaid-v0-8068-v133-mkmlizer: /opt/conda/lib/python3.10/site-packages/transformers/models/auto/auto_factory.py:469: FutureWarning: The `use_auth_token` argument is deprecated and will be removed in v5 of Transformers. Please use `token` instead.
neversleep-noromaid-v0-8068-v133-mkmlizer: warnings.warn(
neversleep-noromaid-v0-8068-v133-mkmlizer: Downloading shards: 0%| | 0/2 [00:00<?, ?it/s] Downloading shards: 50%|█████ | 1/2 [00:07<00:07, 7.05s/it] Downloading shards: 100%|██████████| 2/2 [00:09<00:00, 4.14s/it] Downloading shards: 100%|██████████| 2/2 [00:09<00:00, 4.58s/it]
neversleep-noromaid-v0-8068-v133-mkmlizer: Loading checkpoint shards: 0%| | 0/2 [00:00<?, ?it/s] Loading checkpoint shards: 50%|█████ | 1/2 [00:00<00:00, 1.57it/s] Loading checkpoint shards: 100%|██████████| 2/2 [00:00<00:00, 2.59it/s] Loading checkpoint shards: 100%|██████████| 2/2 [00:00<00:00, 2.36it/s]
neversleep-noromaid-v0-8068-v133-mkmlizer: Saving model to /tmp/reward_cache/reward.tensors
neversleep-noromaid-v0-8068-v133-mkmlizer: Saving duration: 2.231s
neversleep-noromaid-v0-8068-v133-mkmlizer: Processed model ChaiML/gpt2_xl_pairwise_89m_step_347634 in 14.306s
neversleep-noromaid-v0-8068-v133-mkmlizer: creating bucket guanaco-reward-models
neversleep-noromaid-v0-8068-v133-mkmlizer: Bucket 's3://guanaco-reward-models/' created
neversleep-noromaid-v0-8068-v133-mkmlizer: uploading /tmp/reward_cache to s3://guanaco-reward-models/neversleep-noromaid-v0-8068-v133_reward
neversleep-noromaid-v0-8068-v133-mkmlizer: cp /tmp/reward_cache/config.json s3://guanaco-reward-models/neversleep-noromaid-v0-8068-v133_reward/config.json
neversleep-noromaid-v0-8068-v133-mkmlizer: cp /tmp/reward_cache/tokenizer_config.json s3://guanaco-reward-models/neversleep-noromaid-v0-8068-v133_reward/tokenizer_config.json
neversleep-noromaid-v0-8068-v133-mkmlizer: cp /tmp/reward_cache/special_tokens_map.json s3://guanaco-reward-models/neversleep-noromaid-v0-8068-v133_reward/special_tokens_map.json
neversleep-noromaid-v0-8068-v133-mkmlizer: cp /tmp/reward_cache/merges.txt s3://guanaco-reward-models/neversleep-noromaid-v0-8068-v133_reward/merges.txt
neversleep-noromaid-v0-8068-v133-mkmlizer: cp /tmp/reward_cache/vocab.json s3://guanaco-reward-models/neversleep-noromaid-v0-8068-v133_reward/vocab.json
neversleep-noromaid-v0-8068-v133-mkmlizer: cp /tmp/reward_cache/tokenizer.json s3://guanaco-reward-models/neversleep-noromaid-v0-8068-v133_reward/tokenizer.json
Job neversleep-noromaid-v0-8068-v133-mkmlizer completed after 309.76s with status: succeeded
Stopping job with name neversleep-noromaid-v0-8068-v133-mkmlizer
Pipeline stage MKMLizer completed in 313.77s
Running pipeline stage MKMLKubeTemplater
Pipeline stage MKMLKubeTemplater completed in 0.27s
Running pipeline stage ISVCDeployer
Creating inference service neversleep-noromaid-v0-8068-v133
Waiting for inference service neversleep-noromaid-v0-8068-v133 to be ready
Inference service neversleep-noromaid-v0-8068-v133 ready after 91.78639960289001s
Pipeline stage ISVCDeployer completed in 93.29s
Running pipeline stage StressChecker
HTTP Request: %s %s "%s %d %s"
Received healthy response to inference request in 3.5309934616088867s
HTTP Request: %s %s "%s %d %s"
Received healthy response to inference request in 2.449643611907959s
HTTP Request: %s %s "%s %d %s"
Received healthy response to inference request in 2.4448983669281006s
HTTP Request: %s %s "%s %d %s"
Received healthy response to inference request in 2.4586355686187744s
HTTP Request: %s %s "%s %d %s"
Received healthy response to inference request in 2.2994747161865234s
5 requests
0 failed requests
5th percentile: 2.3285594463348387
10th percentile: 2.3576441764831544
20th percentile: 2.4158136367797853
30th percentile: 2.445847415924072
40th percentile: 2.447745513916016
50th percentile: 2.449643611907959
60th percentile: 2.453240394592285
70th percentile: 2.4568371772766113
80th percentile: 2.673107147216797
90th percentile: 3.102050304412842
95th percentile: 3.3165218830108643
99th percentile: 3.488099145889282
mean time: 2.6367291450500487
Pipeline stage StressChecker completed in 15.61s
neversleep-noromaid-v0_8068_v133 status is now deployed due to DeploymentManager action

Usage Metrics

Latency Metrics