submission_id: mistralai-mistral-nemo-_9330_v67
developer_uid: chai_backend_admin
alignment_samples: 11384
alignment_score: -0.030220474155263667
best_of: 4
celo_rating: 1210.15
display_name: mistralai-mistral-nemo-_9330_v67
formatter: {'memory_template': "{bot_name}'s Persona: {memory}\n####\n", 'prompt_template': '{prompt}\n<START>\n', 'bot_template': '{bot_name}: {message}\n', 'user_template': '{user_name}: {message}\n', 'response_template': '{bot_name}:', 'truncate_by_message': False}
generation_params: {'temperature': 1.0, 'top_p': 1.0, 'min_p': 0.05, 'top_k': 80, 'presence_penalty': 0.0, 'frequency_penalty': 0.0, 'stopping_words': ['\n', '</s>', '###', 'Bot:', 'User:', 'You:', '<|im_end|>'], 'max_input_tokens': 512, 'best_of': 4, 'max_output_tokens': 64}
is_internal_developer: True
language_model: mistralai/Mistral-Nemo-Instruct-2407
max_input_tokens: 512
max_output_tokens: 64
model_architecture: MistralForCausalLM
model_group: mistralai/Mistral-Nemo-I
model_name: mistralai-mistral-nemo-_9330_v67
model_num_parameters: 12772070400.0
model_repo: mistralai/Mistral-Nemo-Instruct-2407
model_size: 13B
num_battles: 11384
num_wins: 5290
propriety_score: 0.74375
propriety_total_count: 960.0
ranking_group: single
status: inactive
submission_type: basic
timestamp: 2024-08-29T03:05:57+00:00
us_pacific_date: 2024-08-28
win_ratio: 0.4646872803935348
Download Preference Data
Resubmit model
Running pipeline stage MKMLizer
Starting job with name mistralai-mistral-nemo-9330-v67-mkmlizer
Waiting for job on mistralai-mistral-nemo-9330-v67-mkmlizer to finish
Stopping job with name mistralai-mistral-nemo-9330-v67-mkmlizer
%s, retrying in %s seconds...
Starting job with name mistralai-mistral-nemo-9330-v67-mkmlizer
Waiting for job on mistralai-mistral-nemo-9330-v67-mkmlizer to finish
mistralai-mistral-nemo-9330-v67-mkmlizer: ╔═════════════════════════════════════════════════════════════════════╗
mistralai-mistral-nemo-9330-v67-mkmlizer: ║ _____ __ __ ║
mistralai-mistral-nemo-9330-v67-mkmlizer: ║ / _/ /_ ___ __/ / ___ ___ / / ║
mistralai-mistral-nemo-9330-v67-mkmlizer: ║ / _/ / // / |/|/ / _ \/ -_) -_) / ║
mistralai-mistral-nemo-9330-v67-mkmlizer: ║ /_//_/\_, /|__,__/_//_/\__/\__/_/ ║
mistralai-mistral-nemo-9330-v67-mkmlizer: ║ /___/ ║
mistralai-mistral-nemo-9330-v67-mkmlizer: ║ ║
mistralai-mistral-nemo-9330-v67-mkmlizer: ║ Version: 0.10.1 ║
mistralai-mistral-nemo-9330-v67-mkmlizer: ║ Copyright 2023 MK ONE TECHNOLOGIES Inc. ║
mistralai-mistral-nemo-9330-v67-mkmlizer: ║ https://mk1.ai ║
mistralai-mistral-nemo-9330-v67-mkmlizer: ║ ║
mistralai-mistral-nemo-9330-v67-mkmlizer: ║ The license key for the current software has been verified as ║
mistralai-mistral-nemo-9330-v67-mkmlizer: ║ belonging to: ║
mistralai-mistral-nemo-9330-v67-mkmlizer: ║ ║
mistralai-mistral-nemo-9330-v67-mkmlizer: ║ Chai Research Corp. ║
mistralai-mistral-nemo-9330-v67-mkmlizer: ║ Account ID: 7997a29f-0ceb-4cc7-9adf-840c57b4ae6f ║
mistralai-mistral-nemo-9330-v67-mkmlizer: ║ Expiration: 2024-10-15 23:59:59 ║
mistralai-mistral-nemo-9330-v67-mkmlizer: ║ ║
mistralai-mistral-nemo-9330-v67-mkmlizer: ╚═════════════════════════════════════════════════════════════════════╝
mistralai-mistral-nemo-9330-v67-mkmlizer: Downloaded to shared memory in 47.156s
mistralai-mistral-nemo-9330-v67-mkmlizer: quantizing model to /dev/shm/model_cache, profile:s0, folder:/tmp/tmpcf0sb5u8, device:0
mistralai-mistral-nemo-9330-v67-mkmlizer: Saving flywheel model at /dev/shm/model_cache
mistralai-mistral-nemo-9330-v67-mkmlizer: quantized model in 35.584s
mistralai-mistral-nemo-9330-v67-mkmlizer: Processed model mistralai/Mistral-Nemo-Instruct-2407 in 82.741s
mistralai-mistral-nemo-9330-v67-mkmlizer: creating bucket guanaco-mkml-models
mistralai-mistral-nemo-9330-v67-mkmlizer: Bucket 's3://guanaco-mkml-models/' created
mistralai-mistral-nemo-9330-v67-mkmlizer: uploading /dev/shm/model_cache to s3://guanaco-mkml-models/mistralai-mistral-nemo-9330-v67
mistralai-mistral-nemo-9330-v67-mkmlizer: cp /dev/shm/model_cache/config.json s3://guanaco-mkml-models/mistralai-mistral-nemo-9330-v67/config.json
mistralai-mistral-nemo-9330-v67-mkmlizer: cp /dev/shm/model_cache/special_tokens_map.json s3://guanaco-mkml-models/mistralai-mistral-nemo-9330-v67/special_tokens_map.json
mistralai-mistral-nemo-9330-v67-mkmlizer: cp /dev/shm/model_cache/tokenizer_config.json s3://guanaco-mkml-models/mistralai-mistral-nemo-9330-v67/tokenizer_config.json
mistralai-mistral-nemo-9330-v67-mkmlizer: cp /dev/shm/model_cache/tokenizer.json s3://guanaco-mkml-models/mistralai-mistral-nemo-9330-v67/tokenizer.json
mistralai-mistral-nemo-9330-v67-mkmlizer: cp /dev/shm/model_cache/flywheel_model.0.safetensors s3://guanaco-mkml-models/mistralai-mistral-nemo-9330-v67/flywheel_model.0.safetensors
mistralai-mistral-nemo-9330-v67-mkmlizer: Loading 0: 0%| | 0/363 [00:00<?, ?it/s] Loading 0: 1%|▏ | 5/363 [00:00<00:10, 35.16it/s] Loading 0: 4%|▎ | 13/363 [00:00<00:06, 55.64it/s] Loading 0: 5%|▌ | 19/363 [00:00<00:07, 44.05it/s] Loading 0: 7%|▋ | 24/363 [00:00<00:07, 43.71it/s] Loading 0: 9%|▊ | 31/363 [00:00<00:06, 50.72it/s] Loading 0: 10%|█ | 37/363 [00:00<00:06, 47.85it/s] Loading 0: 12%|█▏ | 42/363 [00:00<00:07, 45.17it/s] Loading 0: 13%|█▎ | 49/363 [00:01<00:06, 50.61it/s] Loading 0: 15%|█▌ | 55/363 [00:01<00:06, 48.34it/s] Loading 0: 17%|█▋ | 61/363 [00:01<00:08, 36.31it/s] Loading 0: 18%|█▊ | 66/363 [00:01<00:07, 37.53it/s] Loading 0: 20%|█▉ | 72/363 [00:01<00:07, 41.52it/s] Loading 0: 21%|██▏ | 78/363 [00:01<00:06, 40.78it/s] Loading 0: 23%|██▎ | 83/363 [00:01<00:06, 41.45it/s] Loading 0: 25%|██▍ | 90/363 [00:02<00:05, 46.61it/s] Loading 0: 26%|██▌ | 95/363 [00:02<00:05, 46.81it/s] Loading 0: 28%|██▊ | 100/363 [00:02<00:06, 38.78it/s] Loading 0: 29%|██▉ | 107/363 [00:02<00:05, 45.99it/s] Loading 0: 31%|███ | 113/363 [00:02<00:06, 40.94it/s] Loading 0: 33%|███▎ | 118/363 [00:02<00:05, 40.95it/s] Loading 0: 34%|███▍ | 125/363 [00:02<00:05, 46.97it/s] Loading 0: 36%|███▌ | 131/363 [00:02<00:04, 48.04it/s] Loading 0: 38%|███▊ | 137/363 [00:03<00:05, 41.50it/s] Loading 0: 39%|███▉ | 142/363 [00:03<00:06, 33.71it/s] Loading 0: 40%|████ | 147/363 [00:03<00:06, 34.99it/s] Loading 0: 42%|████▏ | 151/363 [00:03<00:05, 35.94it/s] Loading 0: 43%|████▎ | 157/363 [00:03<00:05, 40.03it/s] Loading 0: 45%|████▍ | 163/363 [00:03<00:05, 39.27it/s] Loading 0: 46%|████▋ | 168/363 [00:04<00:05, 38.62it/s] Loading 0: 48%|████▊ | 174/363 [00:04<00:04, 42.69it/s] Loading 0: 49%|████▉ | 179/363 [00:04<00:04, 43.74it/s] Loading 0: 51%|█████ | 184/363 [00:04<00:04, 44.06it/s] Loading 0: 52%|█████▏ | 190/363 [00:04<00:04, 42.82it/s] Loading 0: 54%|█████▎ | 195/363 [00:04<00:03, 42.34it/s] Loading 0: 55%|█████▌ | 201/363 [00:04<00:03, 46.54it/s] Loading 0: 57%|█████▋ | 206/363 [00:04<00:03, 45.97it/s] Loading 0: 58%|█████▊ | 211/363 [00:04<00:03, 45.90it/s] Loading 0: 60%|█████▉ | 216/363 [00:05<00:03, 46.32it/s] Loading 0: 61%|██████ | 222/363 [00:05<00:03, 43.54it/s] Loading 0: 63%|██████▎ | 227/363 [00:05<00:04, 32.03it/s] Loading 0: 64%|██████▎ | 231/363 [00:05<00:04, 31.59it/s] Loading 0: 65%|██████▌ | 237/363 [00:05<00:03, 37.02it/s] Loading 0: 67%|██████▋ | 242/363 [00:05<00:03, 39.49it/s] Loading 0: 68%|██████▊ | 247/363 [00:05<00:02, 40.88it/s] Loading 0: 69%|██████▉ | 252/363 [00:06<00:02, 42.86it/s] Loading 0: 71%|███████ | 257/363 [00:06<00:02, 36.75it/s] Loading 0: 73%|███████▎ | 264/363 [00:06<00:02, 43.51it/s] Loading 0: 74%|███████▍ | 269/363 [00:06<00:02, 44.28it/s] Loading 0: 75%|███████▌ | 274/363 [00:06<00:02, 44.17it/s] Loading 0: 77%|███████▋ | 280/363 [00:06<00:01, 43.21it/s] Loading 0: 79%|███████▊ | 285/363 [00:06<00:01, 41.47it/s] Loading 0: 80%|████████ | 291/363 [00:06<00:01, 45.51it/s] Loading 0: 82%|████████▏ | 296/363 [00:07<00:01, 44.51it/s] Loading 0: 83%|████████▎ | 301/363 [00:07<00:01, 45.31it/s] Loading 0: 84%|████████▍ | 306/363 [00:13<00:22, 2.49it/s] Loading 0: 85%|████████▌ | 310/363 [00:13<00:16, 3.25it/s] Loading 0: 87%|████████▋ | 314/363 [00:14<00:11, 4.27it/s] Loading 0: 88%|████████▊ | 320/363 [00:14<00:06, 6.37it/s] Loading 0: 90%|████████▉ | 326/363 [00:14<00:04, 8.97it/s] Loading 0: 91%|█████████ | 331/363 [00:14<00:02, 11.65it/s] Loading 0: 93%|█████████▎| 338/363 [00:14<00:01, 16.69it/s] Loading 0: 95%|█████████▍| 344/363 [00:14<00:00, 20.64it/s] Loading 0: 96%|█████████▌| 349/363 [00:14<00:00, 24.14it/s] Loading 0: 98%|█████████▊| 356/363 [00:14<00:00, 31.06it/s] Loading 0: 100%|█████████▉| 362/363 [00:14<00:00, 32.87it/s]
Job mistralai-mistral-nemo-9330-v67-mkmlizer completed after 104.93s with status: succeeded
Stopping job with name mistralai-mistral-nemo-9330-v67-mkmlizer
Pipeline stage MKMLizer completed in 106.38s
Running pipeline stage MKMLKubeTemplater
Pipeline stage MKMLKubeTemplater completed in 0.09s
Running pipeline stage ISVCDeployer
Creating inference service mistralai-mistral-nemo-9330-v67
Waiting for inference service mistralai-mistral-nemo-9330-v67 to be ready
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Inference service mistralai-mistral-nemo-9330-v67 ready after 181.36739349365234s
Pipeline stage ISVCDeployer completed in 182.06s
Running pipeline stage StressChecker
Received healthy response to inference request in 2.1110475063323975s
Received healthy response to inference request in 1.2239434719085693s
Received healthy response to inference request in 4.821318626403809s
Received healthy response to inference request in 1.3624813556671143s
Received healthy response to inference request in 1.77945876121521s
5 requests
0 failed requests
5th percentile: 1.2516510486602783
10th percentile: 1.2793586254119873
20th percentile: 1.3347737789154053
30th percentile: 1.4458768367767334
40th percentile: 1.6126677989959717
50th percentile: 1.77945876121521
60th percentile: 1.9120942592620849
70th percentile: 2.04472975730896
80th percentile: 2.6531017303466804
90th percentile: 3.7372101783752445
95th percentile: 4.279264402389526
99th percentile: 4.712907781600952
mean time: 2.25964994430542
Pipeline stage StressChecker completed in 12.31s
mistralai-mistral-nemo-_9330_v67 status is now deployed due to DeploymentManager action
mistralai-mistral-nemo-_9330_v67 status is now inactive due to auto deactivation removed underperforming models

Usage Metrics

Latency Metrics