submission_id: neversleep-noromaid-v0_8068_v102
developer_uid: end_to_end_test
status: inactive
model_repo: NeverSleep/Noromaid-v0.1-mixtral-8x7b-Instruct-v3
reward_repo: ChaiML/gpt2_xl_pairwise_89m_step_347634
generation_params: {'temperature': 1.0, 'top_p': 0.99, 'min_p': 0.1, 'top_k': 40, 'presence_penalty': 0.0, 'frequency_penalty': 0.0, 'stopping_words': ['\n'], 'max_input_tokens': 512, 'best_of': 4, 'max_output_tokens': 64}
formatter: {'memory_template': 'character: {bot_name} {memory}\n', 'prompt_template': '{prompt}', 'bot_template': '{bot_name}: {message}', 'user_template': '{user_name}: {message}', 'response_template': '{bot_name}:', 'truncate_by_message': False}
reward_formatter: {'memory_template': 'character: {bot_name} {memory}\n', 'prompt_template': '{prompt}', 'bot_template': '{bot_name}: {message}', 'user_template': '{user_name}: {message}', 'response_template': '{bot_name}:', 'truncate_by_message': False}
timestamp: 2024-07-02T01:15:45+00:00
model_name: neversleep-noromaid-v0_8068_v102
model_group: NeverSleep/Noromaid-v0.1
num_battles: 11494
num_wins: 5790
celo_rating: 1177.75
propriety_score: 0.7119794533113191
propriety_total_count: 5451.0
submission_type: basic
model_architecture: MixtralForCausalLM
model_num_parameters: 46702792704.0
best_of: 4
max_input_tokens: 512
max_output_tokens: 64
display_name: neversleep-noromaid-v0_8068_v102
ineligible_reason: None
language_model: NeverSleep/Noromaid-v0.1-mixtral-8x7b-Instruct-v3
model_size: 47B
reward_model: ChaiML/gpt2_xl_pairwise_89m_step_347634
us_pacific_date: 2024-07-01
win_ratio: 0.5037410823038106
Resubmit model
Running pipeline stage MKMLizer
Starting job with name neversleep-noromaid-v0-8068-v102-mkmlizer
Waiting for job on neversleep-noromaid-v0-8068-v102-mkmlizer to finish
neversleep-noromaid-v0-8068-v102-mkmlizer: ╔═════════════════════════════════════════════════════════════════════╗
neversleep-noromaid-v0-8068-v102-mkmlizer: ║ _____ __ __ ║
neversleep-noromaid-v0-8068-v102-mkmlizer: ║ / _/ /_ ___ __/ / ___ ___ / / ║
neversleep-noromaid-v0-8068-v102-mkmlizer: ║ / _/ / // / |/|/ / _ \/ -_) -_) / ║
neversleep-noromaid-v0-8068-v102-mkmlizer: ║ /_//_/\_, /|__,__/_//_/\__/\__/_/ ║
neversleep-noromaid-v0-8068-v102-mkmlizer: ║ /___/ ║
neversleep-noromaid-v0-8068-v102-mkmlizer: ║ ║
neversleep-noromaid-v0-8068-v102-mkmlizer: ║ Version: 0.8.14 ║
neversleep-noromaid-v0-8068-v102-mkmlizer: ║ Copyright 2023 MK ONE TECHNOLOGIES Inc. ║
neversleep-noromaid-v0-8068-v102-mkmlizer: ║ https://mk1.ai ║
neversleep-noromaid-v0-8068-v102-mkmlizer: ║ ║
neversleep-noromaid-v0-8068-v102-mkmlizer: ║ The license key for the current software has been verified as ║
neversleep-noromaid-v0-8068-v102-mkmlizer: ║ belonging to: ║
neversleep-noromaid-v0-8068-v102-mkmlizer: ║ ║
neversleep-noromaid-v0-8068-v102-mkmlizer: ║ Chai Research Corp. ║
neversleep-noromaid-v0-8068-v102-mkmlizer: ║ Account ID: 7997a29f-0ceb-4cc7-9adf-840c57b4ae6f ║
neversleep-noromaid-v0-8068-v102-mkmlizer: ║ Expiration: 2024-07-15 23:59:59 ║
neversleep-noromaid-v0-8068-v102-mkmlizer: ║ ║
neversleep-noromaid-v0-8068-v102-mkmlizer: ╚═════════════════════════════════════════════════════════════════════╝
neversleep-noromaid-v0-8068-v102-mkmlizer: Downloaded to shared memory in 123.973s
neversleep-noromaid-v0-8068-v102-mkmlizer: quantizing model to /dev/shm/model_cache
neversleep-noromaid-v0-8068-v102-mkmlizer: Saving flywheel model at /dev/shm/model_cache
neversleep-noromaid-v0-8068-v102-mkmlizer: quantized model in 98.017s
neversleep-noromaid-v0-8068-v102-mkmlizer: Processed model NeverSleep/Noromaid-v0.1-mixtral-8x7b-Instruct-v3 in 221.990s
neversleep-noromaid-v0-8068-v102-mkmlizer: creating bucket guanaco-mkml-models
neversleep-noromaid-v0-8068-v102-mkmlizer: Bucket 's3://guanaco-mkml-models/' created
neversleep-noromaid-v0-8068-v102-mkmlizer: uploading /dev/shm/model_cache to s3://guanaco-mkml-models/neversleep-noromaid-v0-8068-v102
neversleep-noromaid-v0-8068-v102-mkmlizer: cp /dev/shm/model_cache/config.json s3://guanaco-mkml-models/neversleep-noromaid-v0-8068-v102/config.json
neversleep-noromaid-v0-8068-v102-mkmlizer: cp /dev/shm/model_cache/tokenizer_config.json s3://guanaco-mkml-models/neversleep-noromaid-v0-8068-v102/tokenizer_config.json
neversleep-noromaid-v0-8068-v102-mkmlizer: cp /dev/shm/model_cache/special_tokens_map.json s3://guanaco-mkml-models/neversleep-noromaid-v0-8068-v102/special_tokens_map.json
neversleep-noromaid-v0-8068-v102-mkmlizer: cp /dev/shm/model_cache/tokenizer.json s3://guanaco-mkml-models/neversleep-noromaid-v0-8068-v102/tokenizer.json
neversleep-noromaid-v0-8068-v102-mkmlizer: cp /dev/shm/model_cache/tokenizer.model s3://guanaco-mkml-models/neversleep-noromaid-v0-8068-v102/tokenizer.model
neversleep-noromaid-v0-8068-v102-mkmlizer: cp /dev/shm/model_cache/flywheel_model.3.safetensors s3://guanaco-mkml-models/neversleep-noromaid-v0-8068-v102/flywheel_model.3.safetensors
neversleep-noromaid-v0-8068-v102-mkmlizer: cp /dev/shm/model_cache/flywheel_model.0.safetensors s3://guanaco-mkml-models/neversleep-noromaid-v0-8068-v102/flywheel_model.0.safetensors
neversleep-noromaid-v0-8068-v102-mkmlizer: cp /dev/shm/model_cache/flywheel_model.2.safetensors s3://guanaco-mkml-models/neversleep-noromaid-v0-8068-v102/flywheel_model.2.safetensors
neversleep-noromaid-v0-8068-v102-mkmlizer: cp /dev/shm/model_cache/flywheel_model.1.safetensors s3://guanaco-mkml-models/neversleep-noromaid-v0-8068-v102/flywheel_model.1.safetensors
neversleep-noromaid-v0-8068-v102-mkmlizer: loading reward model from ChaiML/gpt2_xl_pairwise_89m_step_347634
neversleep-noromaid-v0-8068-v102-mkmlizer: Loading 0: 0%| | 0/995 [00:00<?, ?it/s] Loading 0: 5%|▌ | 52/995 [00:04<01:26, 10.88it/s] Loading 0: 11%|█ | 107/995 [00:09<01:23, 10.68it/s] Loading 0: 16%|█▋ | 162/995 [00:10<00:48, 17.09it/s] Loading 0: 21%|██ | 210/995 [00:12<00:35, 21.90it/s] Loading 0: 27%|██▋ | 265/995 [00:13<00:25, 28.39it/s] Loading 0: 28%|██▊ | 277/995 [00:33<00:25, 28.39it/s] Loading 0: 28%|██▊ | 278/995 [00:33<02:27, 4.87it/s] Loading 0: 32%|███▏ | 320/995 [00:34<01:37, 6.89it/s] Loading 0: 37%|███▋ | 368/995 [00:35<01:03, 9.95it/s] Loading 0: 43%|████▎ | 423/995 [00:36<00:40, 14.29it/s] Loading 0: 48%|████▊ | 478/995 [00:37<00:27, 18.65it/s] Loading 0: 53%|█████▎ | 526/995 [00:38<00:21, 22.11it/s] Loading 0: 57%|█████▋ | 564/995 [00:56<00:19, 22.11it/s] Loading 0: 57%|█████▋ | 565/995 [00:56<01:04, 6.65it/s] Loading 0: 58%|█████▊ | 581/995 [00:57<00:59, 7.02it/s] Loading 0: 64%|██████▍ | 636/995 [00:59<00:34, 10.53it/s] Loading 0: 69%|██████▉ | 691/995 [01:00<00:20, 14.75it/s] Loading 0: 74%|███████▍ | 739/995 [01:01<00:13, 18.48it/s] Loading 0: 80%|███████▉ | 794/995 [01:02<00:08, 22.82it/s] Loading 0: 85%|████████▍ | 845/995 [01:19<00:06, 22.82it/s] Loading 0: 85%|████████▌ | 846/995 [01:19<00:19, 7.62it/s] Loading 0: 85%|████████▌ | 849/995 [01:20<00:20, 7.16it/s] Loading 0: 90%|█████████ | 897/995 [01:22<00:09, 10.22it/s] Loading 0: 96%|█████████▌| 952/995 [01:23<00:02, 14.60it/s] Loading 0: 100%|██████████| 995/995 [01:24<00:00, 17.66it/s] /opt/conda/lib/python3.10/site-packages/transformers/models/auto/configuration_auto.py:919: FutureWarning: The `use_auth_token` argument is deprecated and will be removed in v5 of Transformers. Please use `token` instead.
neversleep-noromaid-v0-8068-v102-mkmlizer: warnings.warn(
neversleep-noromaid-v0-8068-v102-mkmlizer: /opt/conda/lib/python3.10/site-packages/huggingface_hub/file_download.py:1132: FutureWarning: `resume_download` is deprecated and will be removed in version 1.0.0. Downloads always resume when possible. If you want to force a new download, use `force_download=True`.
neversleep-noromaid-v0-8068-v102-mkmlizer: warnings.warn(
neversleep-noromaid-v0-8068-v102-mkmlizer: /opt/conda/lib/python3.10/site-packages/transformers/models/auto/tokenization_auto.py:769: FutureWarning: The `use_auth_token` argument is deprecated and will be removed in v5 of Transformers. Please use `token` instead.
neversleep-noromaid-v0-8068-v102-mkmlizer: warnings.warn(
neversleep-noromaid-v0-8068-v102-mkmlizer: /opt/conda/lib/python3.10/site-packages/transformers/models/auto/auto_factory.py:468: FutureWarning: The `use_auth_token` argument is deprecated and will be removed in v5 of Transformers. Please use `token` instead.
neversleep-noromaid-v0-8068-v102-mkmlizer: warnings.warn(
neversleep-noromaid-v0-8068-v102-mkmlizer: Downloading shards: 0%| | 0/2 [00:00<?, ?it/s] Downloading shards: 50%|█████ | 1/2 [00:06<00:06, 6.08s/it] Downloading shards: 100%|██████████| 2/2 [00:08<00:00, 3.82s/it] Downloading shards: 100%|██████████| 2/2 [00:08<00:00, 4.16s/it]
neversleep-noromaid-v0-8068-v102-mkmlizer: Loading checkpoint shards: 0%| | 0/2 [00:00<?, ?it/s] Loading checkpoint shards: 50%|█████ | 1/2 [00:00<00:00, 1.90it/s] Loading checkpoint shards: 100%|██████████| 2/2 [00:00<00:00, 3.20it/s] Loading checkpoint shards: 100%|██████████| 2/2 [00:00<00:00, 2.90it/s]
neversleep-noromaid-v0-8068-v102-mkmlizer: Saving model to /tmp/reward_cache/reward.tensors
neversleep-noromaid-v0-8068-v102-mkmlizer: Saving duration: 2.766s
neversleep-noromaid-v0-8068-v102-mkmlizer: Processed model ChaiML/gpt2_xl_pairwise_89m_step_347634 in 13.125s
neversleep-noromaid-v0-8068-v102-mkmlizer: creating bucket guanaco-reward-models
neversleep-noromaid-v0-8068-v102-mkmlizer: Bucket 's3://guanaco-reward-models/' created
neversleep-noromaid-v0-8068-v102-mkmlizer: uploading /tmp/reward_cache to s3://guanaco-reward-models/neversleep-noromaid-v0-8068-v102_reward
neversleep-noromaid-v0-8068-v102-mkmlizer: cp /tmp/reward_cache/config.json s3://guanaco-reward-models/neversleep-noromaid-v0-8068-v102_reward/config.json
neversleep-noromaid-v0-8068-v102-mkmlizer: cp /tmp/reward_cache/merges.txt s3://guanaco-reward-models/neversleep-noromaid-v0-8068-v102_reward/merges.txt
neversleep-noromaid-v0-8068-v102-mkmlizer: cp /tmp/reward_cache/vocab.json s3://guanaco-reward-models/neversleep-noromaid-v0-8068-v102_reward/vocab.json
neversleep-noromaid-v0-8068-v102-mkmlizer: cp /tmp/reward_cache/tokenizer_config.json s3://guanaco-reward-models/neversleep-noromaid-v0-8068-v102_reward/tokenizer_config.json
neversleep-noromaid-v0-8068-v102-mkmlizer: cp /tmp/reward_cache/special_tokens_map.json s3://guanaco-reward-models/neversleep-noromaid-v0-8068-v102_reward/special_tokens_map.json
neversleep-noromaid-v0-8068-v102-mkmlizer: cp /tmp/reward_cache/tokenizer.json s3://guanaco-reward-models/neversleep-noromaid-v0-8068-v102_reward/tokenizer.json
neversleep-noromaid-v0-8068-v102-mkmlizer: cp /tmp/reward_cache/reward.tensors s3://guanaco-reward-models/neversleep-noromaid-v0-8068-v102_reward/reward.tensors
Job neversleep-noromaid-v0-8068-v102-mkmlizer completed after 477.69s with status: succeeded
Stopping job with name neversleep-noromaid-v0-8068-v102-mkmlizer
Pipeline stage MKMLizer completed in 479.37s
Running pipeline stage MKMLKubeTemplater
Pipeline stage MKMLKubeTemplater completed in 0.45s
Running pipeline stage ISVCDeployer
Creating inference service neversleep-noromaid-v0-8068-v102
Waiting for inference service neversleep-noromaid-v0-8068-v102 to be ready
Inference service neversleep-noromaid-v0-8068-v102 ready after 131.90288996696472s
Pipeline stage ISVCDeployer completed in 138.57s
Running pipeline stage StressChecker
Received healthy response to inference request in 2.444279670715332s
Received healthy response to inference request in 2.339982032775879s
Received healthy response to inference request in 1.6633210182189941s
Received healthy response to inference request in 2.3021411895751953s
Received healthy response to inference request in 2.453624963760376s
5 requests
0 failed requests
5th percentile: 1.7910850524902344
10th percentile: 1.9188490867614747
20th percentile: 2.1743771553039553
30th percentile: 2.309709358215332
40th percentile: 2.3248456954956054
50th percentile: 2.339982032775879
60th percentile: 2.3817010879516602
70th percentile: 2.4234201431274416
80th percentile: 2.446148729324341
90th percentile: 2.449886846542358
95th percentile: 2.451755905151367
99th percentile: 2.453251152038574
mean time: 2.240669775009155
Pipeline stage StressChecker completed in 14.69s
neversleep-noromaid-v0_8068_v102 status is now deployed due to DeploymentManager action
neversleep-noromaid-v0_8068_v102 status is now inactive due to auto deactivation removed underperforming models

Usage Metrics

Latency Metrics