submission_id: mistralai-mixtral-8x7b-_3473_v28
developer_uid: chai_backend_admin
status: torndown
model_repo: mistralai/Mixtral-8x7B-Instruct-v0.1
reward_repo: rirv938/reward_gpt2_medium_preference_24m_e2
generation_params: {'temperature': 0.9, 'top_p': 1.0, 'min_p': 0.0, 'top_k': 50, 'presence_penalty': 0.5, 'frequency_penalty': 0.5, 'stopping_words': ['\n', '</s>', '<|user|>', '###'], 'max_input_tokens': 512, 'best_of': 4, 'max_output_tokens': 64}
formatter: {'memory_template': '<s>[INST] This is an entertaining conversation. You are {bot_name} who has the persona: {memory}.\nEngage in a chat with {user_name} while staying in character. Try to flirt with {user_name}. Engage in *roleplay* actions. Describe the scene dramatically. \n', 'prompt_template': '{prompt}\n', 'bot_template': '{bot_name}: {message}</s>', 'user_template': '[INST] {user_name}: {message} [/INST]', 'response_template': '{bot_name}:', 'truncate_by_message': False}
reward_formatter: {'memory_template': 'Memory: {memory}\n', 'prompt_template': '{prompt}\n', 'bot_template': 'Bot: {message}\n', 'user_template': 'User: {message}\n', 'response_template': 'Bot:', 'truncate_by_message': False}
timestamp: 2024-04-27T00:27:39+00:00
model_name: mixtral_v3
model_eval_status: success
model_group: mistralai/Mixtral-8x7B-I
num_battles: 7240
num_wins: 3628
celo_rating: 1153.4
propriety_score: 0.0
propriety_total_count: 0.0
submission_type: basic
model_architecture: MixtralForCausalLM
model_num_parameters: 46702792704.0
best_of: 4
max_input_tokens: 512
max_output_tokens: 64
display_name: mixtral_v3
ineligible_reason: propriety_total_count < 800
language_model: mistralai/Mixtral-8x7B-Instruct-v0.1
model_size: 47B
reward_model: rirv938/reward_gpt2_medium_preference_24m_e2
us_pacific_date: 2024-04-26
win_ratio: 0.5011049723756906
preference_data_url: None
Resubmit model
Running pipeline stage MKMLizer
Starting job with name mistralai-mixtral-8x7b-3473-v28-mkmlizer
Waiting for job on mistralai-mixtral-8x7b-3473-v28-mkmlizer to finish
mistralai-mixtral-8x7b-3473-v28-mkmlizer: ╔═════════════════════════════════════════════════════════════════════╗
mistralai-mixtral-8x7b-3473-v28-mkmlizer: ║ _____ __ __ ║
mistralai-mixtral-8x7b-3473-v28-mkmlizer: ║ / _/ /_ ___ __/ / ___ ___ / / ║
mistralai-mixtral-8x7b-3473-v28-mkmlizer: ║ / _/ / // / |/|/ / _ \/ -_) -_) / ║
mistralai-mixtral-8x7b-3473-v28-mkmlizer: ║ /_//_/\_, /|__,__/_//_/\__/\__/_/ ║
mistralai-mixtral-8x7b-3473-v28-mkmlizer: ║ /___/ ║
mistralai-mixtral-8x7b-3473-v28-mkmlizer: ║ ║
mistralai-mixtral-8x7b-3473-v28-mkmlizer: ║ Version: 0.8.10 ║
mistralai-mixtral-8x7b-3473-v28-mkmlizer: ║ Copyright 2023 MK ONE TECHNOLOGIES Inc. ║
mistralai-mixtral-8x7b-3473-v28-mkmlizer: ║ ║
mistralai-mixtral-8x7b-3473-v28-mkmlizer: ║ The license key for the current software has been verified as ║
mistralai-mixtral-8x7b-3473-v28-mkmlizer: ║ belonging to: ║
mistralai-mixtral-8x7b-3473-v28-mkmlizer: ║ ║
mistralai-mixtral-8x7b-3473-v28-mkmlizer: ║ Chai Research Corp. ║
mistralai-mixtral-8x7b-3473-v28-mkmlizer: ║ Account ID: 7997a29f-0ceb-4cc7-9adf-840c57b4ae6f ║
mistralai-mixtral-8x7b-3473-v28-mkmlizer: ║ Expiration: 2024-07-15 23:59:59 ║
mistralai-mixtral-8x7b-3473-v28-mkmlizer: ║ ║
mistralai-mixtral-8x7b-3473-v28-mkmlizer: ╚═════════════════════════════════════════════════════════════════════╝
mistralai-mixtral-8x7b-3473-v28-mkmlizer: /opt/conda/lib/python3.10/site-packages/huggingface_hub/utils/_deprecation.py:131: FutureWarning: 'list_files_info' (from 'huggingface_hub.hf_api') is deprecated and will be removed from version '0.23'. Use `list_repo_tree` and `get_paths_info` instead.
mistralai-mixtral-8x7b-3473-v28-mkmlizer: warnings.warn(warning_message, FutureWarning)
mistralai-mixtral-8x7b-3473-v28-mkmlizer: Downloaded to shared memory in 78.353s
mistralai-mixtral-8x7b-3473-v28-mkmlizer: quantizing model to /dev/shm/model_cache
mistralai-mixtral-8x7b-3473-v28-mkmlizer: Saving flywheel model at /dev/shm/model_cache
mistralai-mixtral-8x7b-3473-v28-mkmlizer: quantized model in 58.701s
mistralai-mixtral-8x7b-3473-v28-mkmlizer: Processed model mistralai/Mixtral-8x7B-Instruct-v0.1 in 144.189s
mistralai-mixtral-8x7b-3473-v28-mkmlizer: creating bucket guanaco-mkml-models
mistralai-mixtral-8x7b-3473-v28-mkmlizer: Bucket 's3://guanaco-mkml-models/' created
mistralai-mixtral-8x7b-3473-v28-mkmlizer: uploading /dev/shm/model_cache to s3://guanaco-mkml-models/mistralai-mixtral-8x7b-3473-v28
mistralai-mixtral-8x7b-3473-v28-mkmlizer: cp /dev/shm/model_cache/tokenizer_config.json s3://guanaco-mkml-models/mistralai-mixtral-8x7b-3473-v28/tokenizer_config.json
mistralai-mixtral-8x7b-3473-v28-mkmlizer: cp /dev/shm/model_cache/special_tokens_map.json s3://guanaco-mkml-models/mistralai-mixtral-8x7b-3473-v28/special_tokens_map.json
mistralai-mixtral-8x7b-3473-v28-mkmlizer: cp /dev/shm/model_cache/config.json s3://guanaco-mkml-models/mistralai-mixtral-8x7b-3473-v28/config.json
mistralai-mixtral-8x7b-3473-v28-mkmlizer: cp /dev/shm/model_cache/tokenizer.model s3://guanaco-mkml-models/mistralai-mixtral-8x7b-3473-v28/tokenizer.model
mistralai-mixtral-8x7b-3473-v28-mkmlizer: cp /dev/shm/model_cache/tokenizer.json s3://guanaco-mkml-models/mistralai-mixtral-8x7b-3473-v28/tokenizer.json
mistralai-mixtral-8x7b-3473-v28-mkmlizer: cp /dev/shm/model_cache/flywheel_model.3.safetensors s3://guanaco-mkml-models/mistralai-mixtral-8x7b-3473-v28/flywheel_model.3.safetensors
mistralai-mixtral-8x7b-3473-v28-mkmlizer: cp /dev/shm/model_cache/flywheel_model.0.safetensors s3://guanaco-mkml-models/mistralai-mixtral-8x7b-3473-v28/flywheel_model.0.safetensors
mistralai-mixtral-8x7b-3473-v28-mkmlizer: cp /dev/shm/model_cache/flywheel_model.2.safetensors s3://guanaco-mkml-models/mistralai-mixtral-8x7b-3473-v28/flywheel_model.2.safetensors
mistralai-mixtral-8x7b-3473-v28-mkmlizer: cp /dev/shm/model_cache/flywheel_model.1.safetensors s3://guanaco-mkml-models/mistralai-mixtral-8x7b-3473-v28/flywheel_model.1.safetensors
mistralai-mixtral-8x7b-3473-v28-mkmlizer: loading reward model from rirv938/reward_gpt2_medium_preference_24m_e2
mistralai-mixtral-8x7b-3473-v28-mkmlizer: Loading 0: 0%| | 0/995 [00:00<?, ?it/s] Loading 0: 5%|▌ | 52/995 [00:01<00:20, 45.28it/s] Loading 0: 11%|█ | 107/995 [00:02<00:18, 46.86it/s] Loading 0: 16%|█▋ | 162/995 [00:03<00:17, 47.73it/s] Loading 0: 21%|██ | 210/995 [00:04<00:16, 46.91it/s] Loading 0: 27%|██▋ | 265/995 [00:05<00:15, 48.15it/s] Loading 0: 28%|██▊ | 282/995 [00:15<01:12, 9.78it/s] Loading 0: 32%|███▏ | 320/995 [00:16<00:53, 12.53it/s] Loading 0: 37%|███▋ | 368/995 [00:17<00:37, 16.94it/s] Loading 0: 43%|████▎ | 423/995 [00:18<00:25, 22.59it/s] Loading 0: 48%|████▊ | 478/995 [00:20<00:18, 27.90it/s] Loading 0: 53%|█████▎ | 526/995 [00:21<00:14, 31.44it/s] Loading 0: 53%|█████▎ | 526/995 [00:31<00:14, 31.44it/s] Loading 0: 56%|█████▋ | 561/995 [00:31<00:40, 10.64it/s] Loading 0: 59%|█████▉ | 588/995 [00:32<00:33, 12.17it/s] Loading 0: 64%|██████▍ | 641/995 [00:33<00:20, 16.99it/s] Loading 0: 70%|██████▉ | 693/995 [00:34<00:13, 22.01it/s] Loading 0: 74%|███████▍ | 740/995 [00:35<00:09, 26.20it/s] Loading 0: 80%|███████▉ | 794/995 [00:36<00:06, 31.09it/s] Loading 0: 85%|████████▍ | 842/995 [00:46<00:13, 11.45it/s] Loading 0: 88%|████████▊ | 872/995 [00:47<00:09, 13.15it/s] Loading 0: 92%|█████████▏| 919/995 [00:48<00:04, 17.06it/s] Loading 0: 96%|█████████▌| 952/995 [00:51<00:02, 16.76it/s] /opt/conda/lib/python3.10/site-packages/transformers/models/auto/configuration_auto.py:913: FutureWarning: The `use_auth_token` argument is deprecated and will be removed in v5 of Transformers. Please use `token` instead.
mistralai-mixtral-8x7b-3473-v28-mkmlizer: warnings.warn(
mistralai-mixtral-8x7b-3473-v28-mkmlizer: /opt/conda/lib/python3.10/site-packages/transformers/models/auto/tokenization_auto.py:757: FutureWarning: The `use_auth_token` argument is deprecated and will be removed in v5 of Transformers. Please use `token` instead.
mistralai-mixtral-8x7b-3473-v28-mkmlizer: warnings.warn(
mistralai-mixtral-8x7b-3473-v28-mkmlizer: /opt/conda/lib/python3.10/site-packages/transformers/models/auto/auto_factory.py:468: FutureWarning: The `use_auth_token` argument is deprecated and will be removed in v5 of Transformers. Please use `token` instead.
mistralai-mixtral-8x7b-3473-v28-mkmlizer: warnings.warn(
mistralai-mixtral-8x7b-3473-v28-mkmlizer: /opt/conda/lib/python3.10/site-packages/torch/_utils.py:831: UserWarning: TypedStorage is deprecated. It will be removed in the future and UntypedStorage will be the only storage class. This should only matter to you if you are using storages directly. To access UntypedStorage directly, use tensor.untyped_storage() instead of tensor.storage()
mistralai-mixtral-8x7b-3473-v28-mkmlizer: return self.fget.__get__(instance, owner)()
mistralai-mixtral-8x7b-3473-v28-mkmlizer: Saving model to /tmp/reward_cache/reward.tensors
mistralai-mixtral-8x7b-3473-v28-mkmlizer: Saving duration: 0.263s
mistralai-mixtral-8x7b-3473-v28-mkmlizer: Processed model rirv938/reward_gpt2_medium_preference_24m_e2 in 4.368s
mistralai-mixtral-8x7b-3473-v28-mkmlizer: creating bucket guanaco-reward-models
mistralai-mixtral-8x7b-3473-v28-mkmlizer: Bucket 's3://guanaco-reward-models/' created
mistralai-mixtral-8x7b-3473-v28-mkmlizer: uploading /tmp/reward_cache to s3://guanaco-reward-models/mistralai-mixtral-8x7b-3473-v28_reward
Job mistralai-mixtral-8x7b-3473-v28-mkmlizer completed after 187.72s with status: succeeded
Stopping job with name mistralai-mixtral-8x7b-3473-v28-mkmlizer
Pipeline stage MKMLizer completed in 189.28s
Running pipeline stage MKMLKubeTemplater
Pipeline stage MKMLKubeTemplater completed in 0.44s
Running pipeline stage ISVCDeployer
Creating inference service mistralai-mixtral-8x7b-3473-v28
Waiting for inference service mistralai-mixtral-8x7b-3473-v28 to be ready
Inference service mistralai-mixtral-8x7b-3473-v28 ready after 50.911850929260254s
Pipeline stage ISVCDeployer completed in 57.40s
Running pipeline stage StressChecker
HTTP Request: %s %s "%s %d %s"
Received healthy response to inference request in 3.266991138458252s
HTTP Request: %s %s "%s %d %s"
Received healthy response to inference request in 1.790414810180664s
HTTP Request: %s %s "%s %d %s"
Received healthy response to inference request in 1.6843929290771484s
HTTP Request: %s %s "%s %d %s"
Received healthy response to inference request in 2.0096089839935303s
HTTP Request: %s %s "%s %d %s"
Received healthy response to inference request in 2.1062698364257812s
5 requests
0 failed requests
5th percentile: 1.7055973052978515
10th percentile: 1.7268016815185547
20th percentile: 1.769210433959961
30th percentile: 1.8342536449432374
40th percentile: 1.9219313144683838
50th percentile: 2.0096089839935303
60th percentile: 2.0482733249664307
70th percentile: 2.086937665939331
80th percentile: 2.3384140968322757
90th percentile: 2.802702617645264
95th percentile: 3.0348468780517575
99th percentile: 3.220562286376953
mean time: 2.171535539627075
Pipeline stage StressChecker completed in 14.98s
Running pipeline stage DaemonicModelEvalScorer
Pipeline stage DaemonicModelEvalScorer completed in 0.16s
Running pipeline stage DaemonicSafetyScorer
Pipeline stage DaemonicSafetyScorer completed in 0.17s
Running M-Eval for topic stay_in_character
%s, retrying in %s seconds...
mistralai-mixtral-8x7b-_3473_v28 status is now deployed due to DeploymentManager action
M-Eval Dataset for topic stay_in_character is loaded
%s, retrying in %s seconds...
mistralai-mixtral-8x7b-_3473_v28 status is now inactive due to auto deactivation removed underperforming models
admin requested tearing down of mistralai-mixtral-8x7b-_3473_v28
Running pipeline stage ISVCDeleter
Checking if service mistralai-mixtral-8x7b-3473-v28 is running
Tearing down inference service mistralai-mixtral-8x7b-3473-v28
Toredown service mistralai-mixtral-8x7b-3473-v28
Pipeline stage ISVCDeleter completed in 8.51s
Running pipeline stage MKMLModelDeleter
Cleaning model data from S3
Cleaning model data from model cache
Deleting key mistralai-mixtral-8x7b-3473-v28/config.json from bucket guanaco-mkml-models
Deleting key mistralai-mixtral-8x7b-3473-v28/flywheel_model.0.safetensors from bucket guanaco-mkml-models
Deleting key mistralai-mixtral-8x7b-3473-v28/flywheel_model.1.safetensors from bucket guanaco-mkml-models
Deleting key mistralai-mixtral-8x7b-3473-v28/flywheel_model.2.safetensors from bucket guanaco-mkml-models
Deleting key mistralai-mixtral-8x7b-3473-v28/flywheel_model.3.safetensors from bucket guanaco-mkml-models
Deleting key mistralai-mixtral-8x7b-3473-v28/special_tokens_map.json from bucket guanaco-mkml-models
Deleting key mistralai-mixtral-8x7b-3473-v28/tokenizer.json from bucket guanaco-mkml-models
Deleting key mistralai-mixtral-8x7b-3473-v28/tokenizer.model from bucket guanaco-mkml-models
Deleting key mistralai-mixtral-8x7b-3473-v28/tokenizer_config.json from bucket guanaco-mkml-models
Cleaning model data from model cache
Deleting key mistralai-mixtral-8x7b-3473-v28_reward/config.json from bucket guanaco-reward-models
Deleting key mistralai-mixtral-8x7b-3473-v28_reward/merges.txt from bucket guanaco-reward-models
Deleting key mistralai-mixtral-8x7b-3473-v28_reward/reward.tensors from bucket guanaco-reward-models
Deleting key mistralai-mixtral-8x7b-3473-v28_reward/special_tokens_map.json from bucket guanaco-reward-models
Deleting key mistralai-mixtral-8x7b-3473-v28_reward/tokenizer.json from bucket guanaco-reward-models
Deleting key mistralai-mixtral-8x7b-3473-v28_reward/tokenizer_config.json from bucket guanaco-reward-models
Deleting key mistralai-mixtral-8x7b-3473-v28_reward/vocab.json from bucket guanaco-reward-models
Pipeline stage MKMLModelDeleter completed in 5.41s
mistralai-mixtral-8x7b-_3473_v28 status is now torndown due to DeploymentManager action

Usage Metrics

Latency Metrics