submission_id: mistralai-mistral-nemo-i_9330_v9
developer_uid: chai_backend_admin
alignment_samples: 0
best_of: 4
celo_rating: 1196.56
display_name: mistralai-mistral-nemo-i_9330_v9
formatter: {'memory_template': '<s>[INST] This is an entertaining conversation. You are {bot_name} who has the persona: {memory}.\nEngage in a chat with {user_name} while staying in character. Try to flirt with {user_name}. Engage in *roleplay* actions. Describe the scene dramatically. \n', 'prompt_template': '{prompt}\n', 'bot_template': '{bot_name}: {message}</s>', 'user_template': '[INST] {user_name}: {message} [/INST]', 'response_template': '{bot_name}:', 'truncate_by_message': False}
generation_params: {'temperature': 0.9, 'top_p': 1.0, 'min_p': 0.0, 'top_k': 50, 'presence_penalty': 0.5, 'frequency_penalty': 0.5, 'stopping_words': ['\n', '</s>', '<|user|>', '###'], 'max_input_tokens': 512, 'best_of': 4, 'max_output_tokens': 64, 'reward_max_tokens': 256}
is_internal_developer: True
language_model: mistralai/Mistral-Nemo-Instruct-2407
max_input_tokens: 512
max_output_tokens: 64
model_architecture: MistralForCausalLM
model_group: mistralai/Mistral-Nemo-I
model_name: mistralai-mistral-nemo-i_9330_v9
model_num_parameters: 12772070400.0
model_repo: mistralai/Mistral-Nemo-Instruct-2407
model_size: 13B
num_battles: 14701
num_wins: 7277
propriety_score: 0.7252747252747253
propriety_total_count: 1547.0
ranking_group: single
reward_formatter: {'bot_template': '{bot_name}: {message}\n', 'memory_template': "''", 'prompt_template': "''", 'response_template': '{bot_name}:', 'truncate_by_message': False, 'user_template': '{user_name}: {message}\n'}
reward_repo: ChaiML/gpt2_xl_pairwise_89m_step_347634
status: torndown
submission_type: basic
timestamp: 2024-07-23T01:36:49+00:00
us_pacific_date: 2024-07-22
win_ratio: 0.4950003401129175
Download Preference Data
Resubmit model
Running pipeline stage MKMLizer
Starting job with name mistralai-mistral-nemo-i-9330-v9-mkmlizer
Waiting for job on mistralai-mistral-nemo-i-9330-v9-mkmlizer to finish
mistralai-mistral-nemo-i-9330-v9-mkmlizer: ╔═════════════════════════════════════════════════════════════════════╗
mistralai-mistral-nemo-i-9330-v9-mkmlizer: ║ _____ __ __ ║
mistralai-mistral-nemo-i-9330-v9-mkmlizer: ║ / _/ /_ ___ __/ / ___ ___ / / ║
mistralai-mistral-nemo-i-9330-v9-mkmlizer: ║ / _/ / // / |/|/ / _ \/ -_) -_) / ║
mistralai-mistral-nemo-i-9330-v9-mkmlizer: ║ /_//_/\_, /|__,__/_//_/\__/\__/_/ ║
mistralai-mistral-nemo-i-9330-v9-mkmlizer: ║ /___/ ║
mistralai-mistral-nemo-i-9330-v9-mkmlizer: ║ ║
mistralai-mistral-nemo-i-9330-v9-mkmlizer: ║ Version: 0.9.6 ║
mistralai-mistral-nemo-i-9330-v9-mkmlizer: ║ Copyright 2023 MK ONE TECHNOLOGIES Inc. ║
mistralai-mistral-nemo-i-9330-v9-mkmlizer: ║ https://mk1.ai ║
mistralai-mistral-nemo-i-9330-v9-mkmlizer: ║ ║
mistralai-mistral-nemo-i-9330-v9-mkmlizer: ║ The license key for the current software has been verified as ║
mistralai-mistral-nemo-i-9330-v9-mkmlizer: ║ belonging to: ║
mistralai-mistral-nemo-i-9330-v9-mkmlizer: ║ ║
mistralai-mistral-nemo-i-9330-v9-mkmlizer: ║ Chai Research Corp. ║
mistralai-mistral-nemo-i-9330-v9-mkmlizer: ║ Account ID: 7997a29f-0ceb-4cc7-9adf-840c57b4ae6f ║
mistralai-mistral-nemo-i-9330-v9-mkmlizer: ║ Expiration: 2024-10-15 23:59:59 ║
mistralai-mistral-nemo-i-9330-v9-mkmlizer: ║ ║
mistralai-mistral-nemo-i-9330-v9-mkmlizer: ╚═════════════════════════════════════════════════════════════════════╝
mistralai-mistral-nemo-i-9330-v9-mkmlizer: quantized model in 35.318s
mistralai-mistral-nemo-i-9330-v9-mkmlizer: Processed model mistralai/Mistral-Nemo-Instruct-2407 in 83.857s
mistralai-mistral-nemo-i-9330-v9-mkmlizer: creating bucket guanaco-mkml-models
mistralai-mistral-nemo-i-9330-v9-mkmlizer: Bucket 's3://guanaco-mkml-models/' created
mistralai-mistral-nemo-i-9330-v9-mkmlizer: uploading /dev/shm/model_cache to s3://guanaco-mkml-models/mistralai-mistral-nemo-i-9330-v9
mistralai-mistral-nemo-i-9330-v9-mkmlizer: cp /dev/shm/model_cache/config.json s3://guanaco-mkml-models/mistralai-mistral-nemo-i-9330-v9/config.json
mistralai-mistral-nemo-i-9330-v9-mkmlizer: cp /dev/shm/model_cache/special_tokens_map.json s3://guanaco-mkml-models/mistralai-mistral-nemo-i-9330-v9/special_tokens_map.json
mistralai-mistral-nemo-i-9330-v9-mkmlizer: cp /dev/shm/model_cache/tokenizer_config.json s3://guanaco-mkml-models/mistralai-mistral-nemo-i-9330-v9/tokenizer_config.json
mistralai-mistral-nemo-i-9330-v9-mkmlizer: cp /dev/shm/model_cache/tokenizer.json s3://guanaco-mkml-models/mistralai-mistral-nemo-i-9330-v9/tokenizer.json
mistralai-mistral-nemo-i-9330-v9-mkmlizer: cp /dev/shm/model_cache/flywheel_model.0.safetensors s3://guanaco-mkml-models/mistralai-mistral-nemo-i-9330-v9/flywheel_model.0.safetensors
mistralai-mistral-nemo-i-9330-v9-mkmlizer: loading reward model from ChaiML/gpt2_xl_pairwise_89m_step_347634
mistralai-mistral-nemo-i-9330-v9-mkmlizer: Loading 0: 0%| | 0/363 [00:00<?, ?it/s] Loading 0: 1%|▏ | 5/363 [00:00<00:09, 35.84it/s] Loading 0: 4%|▎ | 13/363 [00:00<00:06, 56.51it/s] Loading 0: 5%|▌ | 19/363 [00:00<00:06, 50.53it/s] Loading 0: 7%|▋ | 25/363 [00:00<00:06, 51.38it/s] Loading 0: 9%|▊ | 31/363 [00:00<00:06, 53.33it/s] Loading 0: 10%|█ | 37/363 [00:00<00:06, 49.85it/s] Loading 0: 12%|█▏ | 43/363 [00:00<00:06, 50.79it/s] Loading 0: 13%|█▎ | 49/363 [00:00<00:05, 52.56it/s] Loading 0: 15%|█▌ | 55/363 [00:01<00:06, 49.18it/s] Loading 0: 17%|█▋ | 61/363 [00:01<00:07, 38.12it/s] Loading 0: 18%|█▊ | 66/363 [00:01<00:07, 38.86it/s] Loading 0: 20%|█▉ | 72/363 [00:01<00:06, 42.59it/s] Loading 0: 21%|██▏ | 78/363 [00:01<00:06, 42.28it/s] Loading 0: 23%|██▎ | 83/363 [00:01<00:06, 42.65it/s] Loading 0: 25%|██▍ | 90/363 [00:01<00:05, 47.91it/s] Loading 0: 26%|██▋ | 96/363 [00:02<00:05, 45.00it/s] Loading 0: 28%|██▊ | 101/363 [00:02<00:05, 44.32it/s] Loading 0: 30%|██▉ | 108/363 [00:02<00:05, 50.28it/s] Loading 0: 31%|███▏ | 114/363 [00:02<00:05, 45.47it/s] Loading 0: 33%|███▎ | 119/363 [00:02<00:05, 43.75it/s] Loading 0: 34%|███▍ | 125/363 [00:02<00:05, 45.19it/s] Loading 0: 36%|███▌ | 130/363 [00:02<00:05, 45.64it/s] Loading 0: 37%|███▋ | 135/363 [00:02<00:04, 46.05it/s] Loading 0: 39%|███▉ | 141/363 [00:03<00:04, 44.46it/s] Loading 0: 40%|████ | 146/363 [00:03<00:06, 33.03it/s] Loading 0: 41%|████▏ | 150/363 [00:03<00:06, 33.17it/s] Loading 0: 43%|████▎ | 156/363 [00:03<00:05, 38.77it/s] Loading 0: 44%|████▍ | 161/363 [00:03<00:04, 40.57it/s] Loading 0: 46%|████▌ | 166/363 [00:03<00:04, 42.61it/s] Loading 0: 47%|████▋ | 172/363 [00:03<00:04, 42.36it/s] Loading 0: 49%|████▉ | 177/363 [00:04<00:04, 42.00it/s] Loading 0: 51%|█████ | 184/363 [00:04<00:03, 46.77it/s] Loading 0: 52%|█████▏ | 190/363 [00:04<00:03, 44.97it/s] Loading 0: 54%|█████▎ | 195/363 [00:04<00:03, 43.59it/s] Loading 0: 55%|█████▌ | 201/363 [00:04<00:03, 47.32it/s] Loading 0: 57%|█████▋ | 206/363 [00:04<00:03, 47.42it/s] Loading 0: 58%|█████▊ | 211/363 [00:04<00:03, 46.79it/s] Loading 0: 60%|█████▉ | 217/363 [00:04<00:03, 43.43it/s] Loading 0: 61%|██████▏ | 223/363 [00:05<00:04, 34.09it/s] Loading 0: 63%|██████▎ | 228/363 [00:05<00:03, 35.56it/s] Loading 0: 64%|██████▍ | 232/363 [00:05<00:03, 35.42it/s] Loading 0: 65%|██████▌ | 237/363 [00:05<00:03, 37.81it/s] Loading 0: 66%|██████▋ | 241/363 [00:05<00:03, 37.81it/s] Loading 0: 68%|██████▊ | 247/363 [00:05<00:02, 41.73it/s] Loading 0: 70%|██████▉ | 253/363 [00:05<00:02, 41.09it/s] Loading 0: 71%|███████ | 258/363 [00:05<00:02, 41.39it/s] Loading 0: 73%|███████▎ | 265/363 [00:06<00:02, 46.64it/s] Loading 0: 75%|███████▍ | 271/363 [00:06<00:02, 44.08it/s] Loading 0: 76%|███████▌ | 276/363 [00:06<00:02, 43.05it/s] Loading 0: 78%|███████▊ | 283/363 [00:06<00:01, 47.39it/s] Loading 0: 80%|███████▉ | 289/363 [00:06<00:01, 44.20it/s] Loading 0: 81%|████████ | 294/363 [00:06<00:01, 42.04it/s] Loading 0: 82%|████████▏ | 299/363 [00:06<00:01, 43.78it/s] Loading 0: 84%|████████▎ | 304/363 [00:13<00:23, 2.53it/s] Loading 0: 85%|████████▍ | 308/363 [00:13<00:16, 3.27it/s] Loading 0: 86%|████████▌ | 312/363 [00:13<00:11, 4.27it/s] Loading 0: 88%|████████▊ | 320/363 [00:13<00:06, 7.12it/s] Loading 0: 90%|████████▉ | 326/363 [00:14<00:03, 9.63it/s] Loading 0: 91%|█████████ | 331/363 [00:14<00:02, 12.23it/s] Loading 0: 93%|█████████▎| 338/363 [00:14<00:01, 17.15it/s] Loading 0: 95%|█████████▍| 344/363 [00:14<00:00, 20.97it/s] Loading 0: 96%|█████████▌| 349/363 [00:14<00:00, 24.29it/s] Loading 0: 98%|█████████▊| 356/363 [00:14<00:00, 31.06it/s] Loading 0: 100%|█████████▉| 362/363 [00:14<00:00, 33.19it/s] /opt/conda/lib/python3.10/site-packages/transformers/models/auto/configuration_auto.py:950: FutureWarning: The `use_auth_token` argument is deprecated and will be removed in v5 of Transformers. Please use `token` instead.
mistralai-mistral-nemo-i-9330-v9-mkmlizer: warnings.warn(
mistralai-mistral-nemo-i-9330-v9-mkmlizer: /opt/conda/lib/python3.10/site-packages/transformers/models/auto/tokenization_auto.py:778: FutureWarning: The `use_auth_token` argument is deprecated and will be removed in v5 of Transformers. Please use `token` instead.
mistralai-mistral-nemo-i-9330-v9-mkmlizer: warnings.warn(
mistralai-mistral-nemo-i-9330-v9-mkmlizer: /opt/conda/lib/python3.10/site-packages/transformers/models/auto/auto_factory.py:469: FutureWarning: The `use_auth_token` argument is deprecated and will be removed in v5 of Transformers. Please use `token` instead.
mistralai-mistral-nemo-i-9330-v9-mkmlizer: warnings.warn(
mistralai-mistral-nemo-i-9330-v9-mkmlizer: Downloading shards: 0%| | 0/2 [00:00<?, ?it/s] Downloading shards: 50%|█████ | 1/2 [00:05<00:05, 5.76s/it] Downloading shards: 100%|██████████| 2/2 [00:08<00:00, 4.23s/it] Downloading shards: 100%|██████████| 2/2 [00:08<00:00, 4.46s/it]
mistralai-mistral-nemo-i-9330-v9-mkmlizer: Loading checkpoint shards: 0%| | 0/2 [00:00<?, ?it/s] Loading checkpoint shards: 50%|█████ | 1/2 [00:00<00:00, 2.42it/s] Loading checkpoint shards: 100%|██████████| 2/2 [00:00<00:00, 4.00it/s] Loading checkpoint shards: 100%|██████████| 2/2 [00:00<00:00, 3.64it/s]
mistralai-mistral-nemo-i-9330-v9-mkmlizer: Saving model to /tmp/reward_cache/reward.tensors
mistralai-mistral-nemo-i-9330-v9-mkmlizer: Saving duration: 1.351s
mistralai-mistral-nemo-i-9330-v9-mkmlizer: Processed model ChaiML/gpt2_xl_pairwise_89m_step_347634 in 13.908s
mistralai-mistral-nemo-i-9330-v9-mkmlizer: creating bucket guanaco-reward-models
mistralai-mistral-nemo-i-9330-v9-mkmlizer: Bucket 's3://guanaco-reward-models/' created
mistralai-mistral-nemo-i-9330-v9-mkmlizer: uploading /tmp/reward_cache to s3://guanaco-reward-models/mistralai-mistral-nemo-i-9330-v9_reward
mistralai-mistral-nemo-i-9330-v9-mkmlizer: cp /tmp/reward_cache/reward.tensors s3://guanaco-reward-models/mistralai-mistral-nemo-i-9330-v9_reward/reward.tensors
Job mistralai-mistral-nemo-i-9330-v9-mkmlizer completed after 226.83s with status: succeeded
Stopping job with name mistralai-mistral-nemo-i-9330-v9-mkmlizer
Pipeline stage MKMLizer completed in 228.38s
Running pipeline stage MKMLKubeTemplater
Pipeline stage MKMLKubeTemplater completed in 0.27s
Running pipeline stage ISVCDeployer
Creating inference service mistralai-mistral-nemo-i-9330-v9
Waiting for inference service mistralai-mistral-nemo-i-9330-v9 to be ready
Inference service mistralai-mistral-nemo-i-9330-v9 ready after 142.3313000202179s
Pipeline stage ISVCDeployer completed in 143.81s
Running pipeline stage StressChecker
HTTP Request: %s %s "%s %d %s"
Received healthy response to inference request in 2.0450973510742188s
HTTP Request: %s %s "%s %d %s"
Received healthy response to inference request in 1.3745582103729248s
HTTP Request: %s %s "%s %d %s"
Received healthy response to inference request in 1.1171274185180664s
HTTP Request: %s %s "%s %d %s"
Received healthy response to inference request in 1.574282169342041s
HTTP Request: %s %s "%s %d %s"
Received healthy response to inference request in 1.6390020847320557s
5 requests
0 failed requests
5th percentile: 1.168613576889038
10th percentile: 1.2200997352600098
20th percentile: 1.3230720520019532
30th percentile: 1.414503002166748
40th percentile: 1.4943925857543945
50th percentile: 1.574282169342041
60th percentile: 1.6001701354980469
70th percentile: 1.6260581016540527
80th percentile: 1.7202211380004884
90th percentile: 1.8826592445373536
95th percentile: 1.963878297805786
99th percentile: 2.028853540420532
mean time: 1.5500134468078612
Pipeline stage StressChecker completed in 9.87s
mistralai-mistral-nemo-i_9330_v9 status is now deployed due to DeploymentManager action
mistralai-mistral-nemo-i_9330_v9 status is now inactive due to auto deactivation removed underperforming models
admin requested tearing down of mistralai-mistral-nemo-i_9330_v9
Running pipeline stage ISVCDeleter
Checking if service mistralai-mistral-nemo-i-9330-v9 is running
Tearing down inference service mistralai-mistral-nemo-i-9330-v9
Service mistralai-mistral-nemo-i-9330-v9 has been torndown
Pipeline stage ISVCDeleter completed in 4.78s
Running pipeline stage MKMLModelDeleter
Cleaning model data from S3
Cleaning model data from model cache
Deleting key mistralai-mistral-nemo-i-9330-v9/config.json from bucket guanaco-mkml-models
Deleting key mistralai-mistral-nemo-i-9330-v9/flywheel_model.0.safetensors from bucket guanaco-mkml-models
Deleting key mistralai-mistral-nemo-i-9330-v9/special_tokens_map.json from bucket guanaco-mkml-models
Deleting key mistralai-mistral-nemo-i-9330-v9/tokenizer.json from bucket guanaco-mkml-models
Deleting key mistralai-mistral-nemo-i-9330-v9/tokenizer_config.json from bucket guanaco-mkml-models
Cleaning model data from model cache
Deleting key mistralai-mistral-nemo-i-9330-v9_reward/config.json from bucket guanaco-reward-models
Deleting key mistralai-mistral-nemo-i-9330-v9_reward/merges.txt from bucket guanaco-reward-models
Deleting key mistralai-mistral-nemo-i-9330-v9_reward/reward.tensors from bucket guanaco-reward-models
Deleting key mistralai-mistral-nemo-i-9330-v9_reward/special_tokens_map.json from bucket guanaco-reward-models
Deleting key mistralai-mistral-nemo-i-9330-v9_reward/tokenizer.json from bucket guanaco-reward-models
Deleting key mistralai-mistral-nemo-i-9330-v9_reward/tokenizer_config.json from bucket guanaco-reward-models
Deleting key mistralai-mistral-nemo-i-9330-v9_reward/vocab.json from bucket guanaco-reward-models
Pipeline stage MKMLModelDeleter completed in 5.98s
mistralai-mistral-nemo-i_9330_v9 status is now torndown due to DeploymentManager action

Usage Metrics

Latency Metrics