submission_id: mlabonne-neuralmarcoro14-7b_v46
developer_uid: end_to_end_test
status: inactive
model_repo: mlabonne/NeuralMarcoro14-7B
reward_repo: ChaiML/reward_models_100_170000000_cp_498032
generation_params: {'temperature': 1.0, 'top_p': 0.99, 'min_p': 0.0, 'top_k': 40, 'presence_penalty': 0.0, 'frequency_penalty': 0.0, 'stopping_words': ['\n'], 'max_input_tokens': 512, 'best_of': 4, 'max_output_tokens': 64}
formatter: {'memory_template': 'character: {bot_name} {memory}\n', 'prompt_template': '{prompt}', 'bot_template': '{bot_name}: {message}', 'user_template': '{user_name}: {message}', 'response_template': '{bot_name}:', 'truncate_by_message': False}
reward_formatter: {'memory_template': 'character: {bot_name} {memory}\n', 'prompt_template': '{prompt}', 'bot_template': '{bot_name}: {message}', 'user_template': '{user_name}: {message}', 'response_template': '{bot_name}:', 'truncate_by_message': False}
timestamp: 2024-05-09T14:05:24+00:00
model_name: mlabonne-neuralmarcoro14-7b_v46
model_eval_status: pending
double_thumbs_up: 278
thumbs_up: 511
thumbs_down: 363
num_battles: 32988
num_wins: 13497
celo_rating: 1107.94
entertaining: None
stay_in_character: None
user_preference: None
safety_score: None
submission_type: basic
model_architecture: MistralForCausalLM
model_num_parameters: 7241732096.0
best_of: 4
max_input_tokens: 512
max_output_tokens: 64
display_name: mlabonne-neuralmarcoro14-7b_v46
double_thumbs_up_ratio: 0.24131944444444445
feedback_count: 1152
ineligible_reason: None
language_model: mlabonne/NeuralMarcoro14-7B
model_score: None
model_size: 7B
reward_model: ChaiML/reward_models_100_170000000_cp_498032
single_thumbs_up_ratio: 0.4435763888888889
thumbs_down_ratio: 0.3151041666666667
thumbs_up_ratio: 0.6848958333333334
us_pacific_date: 2024-05-09
win_ratio: 0.40914878137504546
Resubmit model
Running pipeline stage MKMLizer
Starting job with name mlabonne-neuralmarcoro14-7b-v46-mkmlizer
Waiting for job on mlabonne-neuralmarcoro14-7b-v46-mkmlizer to finish
mlabonne-neuralmarcoro14-7b-v46-mkmlizer: ╔═════════════════════════════════════════════════════════════════════╗
mlabonne-neuralmarcoro14-7b-v46-mkmlizer: ║ _____ __ __ ║
mlabonne-neuralmarcoro14-7b-v46-mkmlizer: ║ / _/ /_ ___ __/ / ___ ___ / / ║
mlabonne-neuralmarcoro14-7b-v46-mkmlizer: ║ / _/ / // / |/|/ / _ \/ -_) -_) / ║
mlabonne-neuralmarcoro14-7b-v46-mkmlizer: ║ /_//_/\_, /|__,__/_//_/\__/\__/_/ ║
mlabonne-neuralmarcoro14-7b-v46-mkmlizer: ║ /___/ ║
mlabonne-neuralmarcoro14-7b-v46-mkmlizer: ║ ║
mlabonne-neuralmarcoro14-7b-v46-mkmlizer: ║ Version: 0.8.10 ║
mlabonne-neuralmarcoro14-7b-v46-mkmlizer: ║ Copyright 2023 MK ONE TECHNOLOGIES Inc. ║
mlabonne-neuralmarcoro14-7b-v46-mkmlizer: ║ ║
mlabonne-neuralmarcoro14-7b-v46-mkmlizer: ║ The license key for the current software has been verified as ║
mlabonne-neuralmarcoro14-7b-v46-mkmlizer: ║ belonging to: ║
mlabonne-neuralmarcoro14-7b-v46-mkmlizer: ║ ║
mlabonne-neuralmarcoro14-7b-v46-mkmlizer: ║ Chai Research Corp. ║
mlabonne-neuralmarcoro14-7b-v46-mkmlizer: ║ Account ID: 7997a29f-0ceb-4cc7-9adf-840c57b4ae6f ║
mlabonne-neuralmarcoro14-7b-v46-mkmlizer: ║ Expiration: 2024-07-15 23:59:59 ║
mlabonne-neuralmarcoro14-7b-v46-mkmlizer: ║ ║
mlabonne-neuralmarcoro14-7b-v46-mkmlizer: ╚═════════════════════════════════════════════════════════════════════╝
mlabonne-neuralmarcoro14-7b-v46-mkmlizer: /opt/conda/lib/python3.10/site-packages/huggingface_hub/utils/_deprecation.py:131: FutureWarning: 'list_files_info' (from 'huggingface_hub.hf_api') is deprecated and will be removed from version '0.23'. Use `list_repo_tree` and `get_paths_info` instead.
mlabonne-neuralmarcoro14-7b-v46-mkmlizer: warnings.warn(warning_message, FutureWarning)
mlabonne-neuralmarcoro14-7b-v46-mkmlizer: Downloaded to shared memory in 21.746s
mlabonne-neuralmarcoro14-7b-v46-mkmlizer: quantizing model to /dev/shm/model_cache
mlabonne-neuralmarcoro14-7b-v46-mkmlizer: Saving flywheel model at /dev/shm/model_cache
mlabonne-neuralmarcoro14-7b-v46-mkmlizer: quantized model in 10.028s
mlabonne-neuralmarcoro14-7b-v46-mkmlizer: Processed model mlabonne/NeuralMarcoro14-7B in 32.781s
mlabonne-neuralmarcoro14-7b-v46-mkmlizer: creating bucket guanaco-mkml-models
mlabonne-neuralmarcoro14-7b-v46-mkmlizer: Bucket 's3://guanaco-mkml-models/' created
mlabonne-neuralmarcoro14-7b-v46-mkmlizer: uploading /dev/shm/model_cache to s3://guanaco-mkml-models/mlabonne-neuralmarcoro14-7b-v46
mlabonne-neuralmarcoro14-7b-v46-mkmlizer: cp /dev/shm/model_cache/config.json s3://guanaco-mkml-models/mlabonne-neuralmarcoro14-7b-v46/config.json
mlabonne-neuralmarcoro14-7b-v46-mkmlizer: cp /dev/shm/model_cache/tokenizer_config.json s3://guanaco-mkml-models/mlabonne-neuralmarcoro14-7b-v46/tokenizer_config.json
mlabonne-neuralmarcoro14-7b-v46-mkmlizer: cp /dev/shm/model_cache/special_tokens_map.json s3://guanaco-mkml-models/mlabonne-neuralmarcoro14-7b-v46/special_tokens_map.json
mlabonne-neuralmarcoro14-7b-v46-mkmlizer: cp /dev/shm/model_cache/tokenizer.model s3://guanaco-mkml-models/mlabonne-neuralmarcoro14-7b-v46/tokenizer.model
mlabonne-neuralmarcoro14-7b-v46-mkmlizer: cp /dev/shm/model_cache/tokenizer.json s3://guanaco-mkml-models/mlabonne-neuralmarcoro14-7b-v46/tokenizer.json
mlabonne-neuralmarcoro14-7b-v46-mkmlizer: cp /dev/shm/model_cache/flywheel_model.0.safetensors s3://guanaco-mkml-models/mlabonne-neuralmarcoro14-7b-v46/flywheel_model.0.safetensors
mlabonne-neuralmarcoro14-7b-v46-mkmlizer: loading reward model from ChaiML/reward_models_100_170000000_cp_498032
mlabonne-neuralmarcoro14-7b-v46-mkmlizer: Loading 0: 0%| | 0/291 [00:00<?, ?it/s] Loading 0: 42%|████▏ | 122/291 [00:01<00:01, 121.82it/s] Loading 0: 70%|███████ | 204/291 [00:02<00:01, 63.96it/s] /opt/conda/lib/python3.10/site-packages/transformers/models/auto/configuration_auto.py:913: FutureWarning: The `use_auth_token` argument is deprecated and will be removed in v5 of Transformers. Please use `token` instead.
mlabonne-neuralmarcoro14-7b-v46-mkmlizer: warnings.warn(
mlabonne-neuralmarcoro14-7b-v46-mkmlizer: /opt/conda/lib/python3.10/site-packages/transformers/models/auto/tokenization_auto.py:757: FutureWarning: The `use_auth_token` argument is deprecated and will be removed in v5 of Transformers. Please use `token` instead.
mlabonne-neuralmarcoro14-7b-v46-mkmlizer: warnings.warn(
mlabonne-neuralmarcoro14-7b-v46-mkmlizer: /opt/conda/lib/python3.10/site-packages/torch/_utils.py:831: UserWarning: TypedStorage is deprecated. It will be removed in the future and UntypedStorage will be the only storage class. This should only matter to you if you are using storages directly. To access UntypedStorage directly, use tensor.untyped_storage() instead of tensor.storage()
mlabonne-neuralmarcoro14-7b-v46-mkmlizer: return self.fget.__get__(instance, owner)()
mlabonne-neuralmarcoro14-7b-v46-mkmlizer: Saving model to /tmp/reward_cache/reward.tensors
mlabonne-neuralmarcoro14-7b-v46-mkmlizer: Saving duration: 0.099s
mlabonne-neuralmarcoro14-7b-v46-mkmlizer: Processed model ChaiML/reward_models_100_170000000_cp_498032 in 3.704s
mlabonne-neuralmarcoro14-7b-v46-mkmlizer: creating bucket guanaco-reward-models
mlabonne-neuralmarcoro14-7b-v46-mkmlizer: Bucket 's3://guanaco-reward-models/' created
mlabonne-neuralmarcoro14-7b-v46-mkmlizer: uploading /tmp/reward_cache to s3://guanaco-reward-models/mlabonne-neuralmarcoro14-7b-v46_reward
mlabonne-neuralmarcoro14-7b-v46-mkmlizer: cp /tmp/reward_cache/config.json s3://guanaco-reward-models/mlabonne-neuralmarcoro14-7b-v46_reward/config.json
mlabonne-neuralmarcoro14-7b-v46-mkmlizer: cp /tmp/reward_cache/special_tokens_map.json s3://guanaco-reward-models/mlabonne-neuralmarcoro14-7b-v46_reward/special_tokens_map.json
mlabonne-neuralmarcoro14-7b-v46-mkmlizer: cp /tmp/reward_cache/tokenizer_config.json s3://guanaco-reward-models/mlabonne-neuralmarcoro14-7b-v46_reward/tokenizer_config.json
mlabonne-neuralmarcoro14-7b-v46-mkmlizer: cp /tmp/reward_cache/merges.txt s3://guanaco-reward-models/mlabonne-neuralmarcoro14-7b-v46_reward/merges.txt
mlabonne-neuralmarcoro14-7b-v46-mkmlizer: cp /tmp/reward_cache/vocab.json s3://guanaco-reward-models/mlabonne-neuralmarcoro14-7b-v46_reward/vocab.json
mlabonne-neuralmarcoro14-7b-v46-mkmlizer: cp /tmp/reward_cache/tokenizer.json s3://guanaco-reward-models/mlabonne-neuralmarcoro14-7b-v46_reward/tokenizer.json
mlabonne-neuralmarcoro14-7b-v46-mkmlizer: cp /tmp/reward_cache/reward.tensors s3://guanaco-reward-models/mlabonne-neuralmarcoro14-7b-v46_reward/reward.tensors
Job mlabonne-neuralmarcoro14-7b-v46-mkmlizer completed after 60.26s with status: succeeded
Stopping job with name mlabonne-neuralmarcoro14-7b-v46-mkmlizer
Pipeline stage MKMLizer completed in 62.18s
Running pipeline stage MKMLKubeTemplater
Pipeline stage MKMLKubeTemplater completed in 0.41s
Running pipeline stage ISVCDeployer
Creating inference service mlabonne-neuralmarcoro14-7b-v46
Waiting for inference service mlabonne-neuralmarcoro14-7b-v46 to be ready
Inference service mlabonne-neuralmarcoro14-7b-v46 ready after 40.72921395301819s
Pipeline stage ISVCDeployer completed in 50.47s
Running pipeline stage StressChecker
Received healthy response to inference request in 2.4951331615448s
Received healthy response to inference request in 1.279303789138794s
Received healthy response to inference request in 1.2218658924102783s
Received healthy response to inference request in 1.2063689231872559s
Received healthy response to inference request in 1.2117421627044678s
5 requests
0 failed requests
5th percentile: 1.2074435710906983
10th percentile: 1.2085182189941406
20th percentile: 1.2106675148010253
30th percentile: 1.21376690864563
40th percentile: 1.2178164005279541
50th percentile: 1.2218658924102783
60th percentile: 1.2448410511016845
70th percentile: 1.2678162097930907
80th percentile: 1.5224696636199953
90th percentile: 2.0088014125823976
95th percentile: 2.2519672870635983
99th percentile: 2.4464999866485595
mean time: 1.4828827857971192
Pipeline stage StressChecker completed in 11.42s
Running pipeline stage DaemonicModelEvalScorer
Pipeline stage DaemonicModelEvalScorer completed in 0.15s
Running pipeline stage DaemonicSafetyScorer
Running M-Eval for topic stay_in_character
Pipeline stage DaemonicSafetyScorer completed in 0.15s
%s, retrying in %s seconds...
M-Eval Dataset for topic stay_in_character is loaded
mlabonne-neuralmarcoro14-7b_v46 status is now deployed due to DeploymentManager action
%s, retrying in %s seconds...
Scoring model output for bot %s
mlabonne-neuralmarcoro14-7b_v46 status is now inactive due to auto deactivation removed underperforming models

Usage Metrics

Latency Metrics