submission_id: neversleep-noromaid-v0-_8068_v21
developer_uid: chai_backend_admin
status: inactive
model_repo: NeverSleep/Noromaid-v0.1-mixtral-8x7b-Instruct-v3
reward_repo: rirv938/reward_gpt2_medium_preference_24m_e2
generation_params: {'temperature': 0.8, 'top_p': 0.9, 'top_k': 70, 'presence_penalty': 0.9, 'frequency_penalty': 0.9, 'stopping_words': ['\n', '</s>', '<|user|>', '###'], 'max_input_tokens': 512, 'best_of': 8, 'max_output_tokens': 128}
formatter: {'memory_template': '<s>[INST] This is an entertaining conversation. You are {bot_name} who has the persona: {memory}.\nEngage in a chat with {user_name} while staying in character. Try to flirt with {user_name}. Engage in *roleplay* actions. Describe the scene dramatically. \n', 'prompt_template': '{prompt}\n', 'bot_template': '{bot_name}: {message}</s>', 'user_template': '[INST] {user_name}: {message} [/INST]', 'response_template': '{bot_name}:'}
reward_formatter: {'memory_template': 'Memory: {memory}\n', 'prompt_template': '{prompt}\n', 'bot_template': 'Bot: {message}\n', 'user_template': 'User: {message}\n', 'response_template': 'Bot:'}
timestamp: 2024-03-31T02:15:38+00:00
model_name: Noromaid-v0-1-128
model_eval_status: success
safety_score: 0.97
entertaining: 7.36
stay_in_character: 8.52
user_preference: 7.8
double_thumbs_up: 3445
thumbs_up: 4475
thumbs_down: 1805
num_battles: 313220
num_wins: 186821
win_ratio: 0.5964529723517017
celo_rating: 1224.82
Resubmit model
Running pipeline stage MKMLizer
Starting job with name neversleep-noromaid-v0-8068-v21-mkmlizer
Waiting for job on neversleep-noromaid-v0-8068-v21-mkmlizer to finish
neversleep-noromaid-v0-8068-v21-mkmlizer: ╔═════════════════════════════════════════════════════════════════════╗
neversleep-noromaid-v0-8068-v21-mkmlizer: ║ _____ __ __ ║
neversleep-noromaid-v0-8068-v21-mkmlizer: ║ / _/ /_ ___ __/ / ___ ___ / / ║
neversleep-noromaid-v0-8068-v21-mkmlizer: ║ / _/ / // / |/|/ / _ \/ -_) -_) / ║
neversleep-noromaid-v0-8068-v21-mkmlizer: ║ /_//_/\_, /|__,__/_//_/\__/\__/_/ ║
neversleep-noromaid-v0-8068-v21-mkmlizer: ║ /___/ ║
neversleep-noromaid-v0-8068-v21-mkmlizer: ║ ║
neversleep-noromaid-v0-8068-v21-mkmlizer: ║ Version: 0.8.6 ║
neversleep-noromaid-v0-8068-v21-mkmlizer: ║ Copyright 2023 MK ONE TECHNOLOGIES Inc. ║
neversleep-noromaid-v0-8068-v21-mkmlizer: ║ ║
neversleep-noromaid-v0-8068-v21-mkmlizer: ║ The license key for the current software has been verified as ║
neversleep-noromaid-v0-8068-v21-mkmlizer: ║ belonging to: ║
neversleep-noromaid-v0-8068-v21-mkmlizer: ║ ║
neversleep-noromaid-v0-8068-v21-mkmlizer: ║ Chai Research Corp. ║
neversleep-noromaid-v0-8068-v21-mkmlizer: ║ Account ID: 7997a29f-0ceb-4cc7-9adf-840c57b4ae6f ║
neversleep-noromaid-v0-8068-v21-mkmlizer: ║ Expiration: 2024-07-15 23:59:59 ║
neversleep-noromaid-v0-8068-v21-mkmlizer: ║ ║
neversleep-noromaid-v0-8068-v21-mkmlizer: ╚═════════════════════════════════════════════════════════════════════╝
neversleep-noromaid-v0-8068-v21-mkmlizer: /opt/conda/lib/python3.10/site-packages/huggingface_hub/utils/_deprecation.py:131: FutureWarning: 'list_files_info' (from 'huggingface_hub.hf_api') is deprecated and will be removed from version '0.23'. Use `list_repo_tree` and `get_paths_info` instead.
neversleep-noromaid-v0-8068-v21-mkmlizer: warnings.warn(warning_message, FutureWarning)
Failed to get response for submission anhnv125-llama-op-v17-1_v27: ('http://anhnv125-llama-op-v17-1-v27-predictor-default.tenant-chaiml-guanaco.knative.ord1.coreweave.cloud/v1/models/GPT-J-6B-lit-v2:predict', 'read tcp 127.0.0.1:40450->127.0.0.1:8080: read: connection reset by peer\n')
Exception raised while processing tagging_function
Traceback (most recent call last): File "/code/guanaco/guanaco_services/src/guanaco_model_service/chat_api.py", line 274, in resolve_chat_api conversation_tag = self.tagging_function(conversation_state) File "/home/zongyi/gitlab/zztools/zztools/llm/guanaco/submit_routing_model.py", line 176, in last_user_message_length TypeError: 'ConversationState' object is not subscriptable
neversleep-noromaid-v0-8068-v21-mkmlizer: Downloaded to shared memory in 160.938s
neversleep-noromaid-v0-8068-v21-mkmlizer: quantizing model to /dev/shm/model_cache
neversleep-noromaid-v0-8068-v21-mkmlizer: Saving flywheel model at /dev/shm/model_cache
neversleep-noromaid-v0-8068-v21-mkmlizer: quantized model in 60.960s
neversleep-noromaid-v0-8068-v21-mkmlizer: Processed model NeverSleep/Noromaid-v0.1-mixtral-8x7b-Instruct-v3 in 229.311s
neversleep-noromaid-v0-8068-v21-mkmlizer: creating bucket guanaco-mkml-models
neversleep-noromaid-v0-8068-v21-mkmlizer: Bucket 's3://guanaco-mkml-models/' created
neversleep-noromaid-v0-8068-v21-mkmlizer: uploading /dev/shm/model_cache to s3://guanaco-mkml-models/neversleep-noromaid-v0-8068-v21
neversleep-noromaid-v0-8068-v21-mkmlizer: cp /dev/shm/model_cache/config.json s3://guanaco-mkml-models/neversleep-noromaid-v0-8068-v21/config.json
neversleep-noromaid-v0-8068-v21-mkmlizer: cp /dev/shm/model_cache/tokenizer_config.json s3://guanaco-mkml-models/neversleep-noromaid-v0-8068-v21/tokenizer_config.json
neversleep-noromaid-v0-8068-v21-mkmlizer: cp /dev/shm/model_cache/tokenizer.model s3://guanaco-mkml-models/neversleep-noromaid-v0-8068-v21/tokenizer.model
neversleep-noromaid-v0-8068-v21-mkmlizer: cp /dev/shm/model_cache/special_tokens_map.json s3://guanaco-mkml-models/neversleep-noromaid-v0-8068-v21/special_tokens_map.json
neversleep-noromaid-v0-8068-v21-mkmlizer: cp /dev/shm/model_cache/tokenizer.json s3://guanaco-mkml-models/neversleep-noromaid-v0-8068-v21/tokenizer.json
neversleep-noromaid-v0-8068-v21-mkmlizer: cp /dev/shm/model_cache/flywheel_model.1.safetensors s3://guanaco-mkml-models/neversleep-noromaid-v0-8068-v21/flywheel_model.1.safetensors
neversleep-noromaid-v0-8068-v21-mkmlizer: cp /dev/shm/model_cache/flywheel_model.3.safetensors s3://guanaco-mkml-models/neversleep-noromaid-v0-8068-v21/flywheel_model.3.safetensors
neversleep-noromaid-v0-8068-v21-mkmlizer: cp /dev/shm/model_cache/flywheel_model.0.safetensors s3://guanaco-mkml-models/neversleep-noromaid-v0-8068-v21/flywheel_model.0.safetensors
neversleep-noromaid-v0-8068-v21-mkmlizer: cp /dev/shm/model_cache/flywheel_model.2.safetensors s3://guanaco-mkml-models/neversleep-noromaid-v0-8068-v21/flywheel_model.2.safetensors
neversleep-noromaid-v0-8068-v21-mkmlizer: loading reward model from rirv938/reward_gpt2_medium_preference_24m_e2
neversleep-noromaid-v0-8068-v21-mkmlizer: Loading 0: 0%| | 0/995 [00:00<?, ?it/s] Loading 0: 5%|▌ | 52/995 [00:01<00:21, 43.92it/s] Loading 0: 11%|█ | 107/995 [00:02<00:19, 45.53it/s] Loading 0: 16%|█▋ | 162/995 [00:03<00:17, 47.82it/s] Loading 0: 21%|██ | 210/995 [00:04<00:17, 45.53it/s] Loading 0: 27%|██▋ | 265/995 [00:05<00:15, 46.56it/s] Loading 0: 28%|██▊ | 278/995 [00:15<01:14, 9.56it/s] Loading 0: 32%|███▏ | 320/995 [00:17<00:54, 12.32it/s] Loading 0: 37%|███▋ | 368/995 [00:18<00:38, 16.17it/s] Loading 0: 43%|████▎ | 423/995 [00:19<00:26, 21.37it/s] Loading 0: 48%|████▊ | 478/995 [00:20<00:19, 26.63it/s] Loading 0: 53%|█████▎ | 526/995 [00:21<00:15, 30.18it/s] Loading 0: 57%|█████▋ | 564/995 [00:32<00:14, 30.18it/s] Loading 0: 57%|█████▋ | 565/995 [00:32<00:40, 10.64it/s] Loading 0: 58%|█████▊ | 581/995 [00:33<00:38, 10.75it/s] Loading 0: 64%|██████▍ | 636/995 [00:34<00:23, 15.51it/s] Loading 0: 69%|██████▉ | 691/995 [00:35<00:14, 20.54it/s] Loading 0: 74%|███████▍ | 739/995 [00:36<00:10, 24.84it/s] Loading 0: 80%|███████▉ | 794/995 [00:37<00:06, 30.59it/s] Loading 0: 85%|████████▍ | 845/995 [00:48<00:04, 30.59it/s] Loading 0: 85%|████████▌ | 846/995 [00:48<00:12, 11.85it/s] Loading 0: 85%|████████▌ | 849/995 [00:49<00:13, 10.83it/s] Loading 0: 90%|█████████ | 897/995 [00:50<00:06, 15.11it/s] Loading 0: 96%|█████████▌| 952/995 [00:51<00:02, 21.15it/s] Loading 0: 100%|██████████| 995/995 [00:52<00:00, 23.91it/s] /opt/conda/lib/python3.10/site-packages/transformers/models/auto/configuration_auto.py:1096: FutureWarning: The `use_auth_token` argument is deprecated and will be removed in v5 of Transformers. Please use `token` instead.
neversleep-noromaid-v0-8068-v21-mkmlizer: warnings.warn(
neversleep-noromaid-v0-8068-v21-mkmlizer: /opt/conda/lib/python3.10/site-packages/transformers/models/auto/tokenization_auto.py:720: FutureWarning: The `use_auth_token` argument is deprecated and will be removed in v5 of Transformers. Please use `token` instead.
neversleep-noromaid-v0-8068-v21-mkmlizer: warnings.warn(
neversleep-noromaid-v0-8068-v21-mkmlizer: /opt/conda/lib/python3.10/site-packages/transformers/models/auto/auto_factory.py:466: FutureWarning: The `use_auth_token` argument is deprecated and will be removed in v5 of Transformers. Please use `token` instead.
neversleep-noromaid-v0-8068-v21-mkmlizer: warnings.warn(
neversleep-noromaid-v0-8068-v21-mkmlizer: /opt/conda/lib/python3.10/site-packages/torch/_utils.py:831: UserWarning: TypedStorage is deprecated. It will be removed in the future and UntypedStorage will be the only storage class. This should only matter to you if you are using storages directly. To access UntypedStorage directly, use tensor.untyped_storage() instead of tensor.storage()
neversleep-noromaid-v0-8068-v21-mkmlizer: return self.fget.__get__(instance, owner)()
neversleep-noromaid-v0-8068-v21-mkmlizer: Saving model to /tmp/reward_cache/reward.tensors
neversleep-noromaid-v0-8068-v21-mkmlizer: Saving duration: 0.288s
neversleep-noromaid-v0-8068-v21-mkmlizer: Processed model rirv938/reward_gpt2_medium_preference_24m_e2 in 3.964s
neversleep-noromaid-v0-8068-v21-mkmlizer: creating bucket guanaco-reward-models
neversleep-noromaid-v0-8068-v21-mkmlizer: Bucket 's3://guanaco-reward-models/' created
neversleep-noromaid-v0-8068-v21-mkmlizer: uploading /tmp/reward_cache to s3://guanaco-reward-models/neversleep-noromaid-v0-8068-v21_reward
neversleep-noromaid-v0-8068-v21-mkmlizer: cp /tmp/reward_cache/special_tokens_map.json s3://guanaco-reward-models/neversleep-noromaid-v0-8068-v21_reward/special_tokens_map.json
neversleep-noromaid-v0-8068-v21-mkmlizer: cp /tmp/reward_cache/config.json s3://guanaco-reward-models/neversleep-noromaid-v0-8068-v21_reward/config.json
neversleep-noromaid-v0-8068-v21-mkmlizer: cp /tmp/reward_cache/vocab.json s3://guanaco-reward-models/neversleep-noromaid-v0-8068-v21_reward/vocab.json
neversleep-noromaid-v0-8068-v21-mkmlizer: cp /tmp/reward_cache/merges.txt s3://guanaco-reward-models/neversleep-noromaid-v0-8068-v21_reward/merges.txt
neversleep-noromaid-v0-8068-v21-mkmlizer: cp /tmp/reward_cache/tokenizer_config.json s3://guanaco-reward-models/neversleep-noromaid-v0-8068-v21_reward/tokenizer_config.json
neversleep-noromaid-v0-8068-v21-mkmlizer: cp /tmp/reward_cache/tokenizer.json s3://guanaco-reward-models/neversleep-noromaid-v0-8068-v21_reward/tokenizer.json
neversleep-noromaid-v0-8068-v21-mkmlizer: cp /tmp/reward_cache/reward.tensors s3://guanaco-reward-models/neversleep-noromaid-v0-8068-v21_reward/reward.tensors
Exception raised while processing tagging_function
Traceback (most recent call last): File "/code/guanaco/guanaco_services/src/guanaco_model_service/chat_api.py", line 274, in resolve_chat_api conversation_tag = self.tagging_function(conversation_state) File "/home/zongyi/gitlab/zztools/zztools/llm/guanaco/submit_routing_model.py", line 176, in last_user_message_length TypeError: 'ConversationMessage' object is not subscriptable
Job neversleep-noromaid-v0-8068-v21-mkmlizer completed after 363.41s with status: succeeded
Stopping job with name neversleep-noromaid-v0-8068-v21-mkmlizer
Pipeline stage MKMLizer completed in 367.92s
Running pipeline stage MKMLKubeTemplater
Pipeline stage MKMLKubeTemplater completed in 0.13s
Running pipeline stage ISVCDeployer
Creating inference service neversleep-noromaid-v0-8068-v21
Waiting for inference service neversleep-noromaid-v0-8068-v21 to be ready
Inference service neversleep-noromaid-v0-8068-v21 ready after 50.27430987358093s
Pipeline stage ISVCDeployer completed in 59.32s
Running pipeline stage StressChecker
Received healthy response to inference request in 4.339120864868164s
Received healthy response to inference request in 3.7814245223999023s
Received healthy response to inference request in 2.9079976081848145s
Received healthy response to inference request in 3.1809592247009277s
Received healthy response to inference request in 4.412293910980225s
5 requests
0 failed requests
5th percentile: 2.9625899314880373
10th percentile: 3.0171822547912597
20th percentile: 3.126366901397705
30th percentile: 3.301052284240723
40th percentile: 3.5412384033203126
50th percentile: 3.7814245223999023
60th percentile: 4.004503059387207
70th percentile: 4.227581596374511
80th percentile: 4.353755474090576
90th percentile: 4.3830246925354
95th percentile: 4.3976593017578125
99th percentile: 4.4093669891357425
mean time: 3.724359226226807
Pipeline stage StressChecker completed in 19.31s
Running pipeline stage DaemonicModelEvalScorer
Pipeline stage DaemonicModelEvalScorer completed in 0.04s
Running pipeline stage DaemonicSafetyScorer
Running M-Eval for topic stay_in_character
Pipeline stage DaemonicSafetyScorer completed in 0.04s
M-Eval Dataset for topic stay_in_character is loaded
neversleep-noromaid-v0-_8068_v21 status is now deployed due to DeploymentManager action
neversleep-noromaid-v0-_8068_v21 status is now inactive due to auto deactivation removed underperforming models

Usage Metrics

Latency Metrics