submission_id: mistralai-mistral-nemo-_9330_v69
developer_uid: chai_backend_admin
alignment_samples: 11362
alignment_score: -0.40245224060299795
best_of: 4
celo_rating: 1212.36
display_name: mistralai-mistral-nemo-_9330_v69
formatter: {'memory_template': "{bot_name}'s Persona: {memory}\n####\n", 'prompt_template': '{prompt}\n<START>\n', 'bot_template': '{bot_name}: {message}\n', 'user_template': '{user_name}: {message}\n', 'response_template': '{bot_name}:', 'truncate_by_message': False}
generation_params: {'temperature': 1.0, 'top_p': 1.0, 'min_p': 0.05, 'top_k': 80, 'presence_penalty': 0.0, 'frequency_penalty': 0.0, 'stopping_words': ['\n', '</s>', '###', 'Bot:', 'User:', 'You:', '<|im_end|>'], 'max_input_tokens': 512, 'best_of': 4, 'max_output_tokens': 64}
is_internal_developer: True
language_model: mistralai/Mistral-Nemo-Instruct-2407
max_input_tokens: 512
max_output_tokens: 64
model_architecture: MistralForCausalLM
model_group: mistralai/Mistral-Nemo-I
model_name: mistralai-mistral-nemo-_9330_v69
model_num_parameters: 12772070400.0
model_repo: mistralai/Mistral-Nemo-Instruct-2407
model_size: 13B
num_battles: 11362
num_wins: 5321
propriety_score: 0.7181347150259068
propriety_total_count: 965.0
ranking_group: single
status: inactive
submission_type: basic
timestamp: 2024-08-29T03:11:42+00:00
us_pacific_date: 2024-08-28
win_ratio: 0.4683154374229889
Download Preference Data
Resubmit model
Running pipeline stage MKMLizer
Starting job with name mistralai-mistral-nemo-9330-v69-mkmlizer
Waiting for job on mistralai-mistral-nemo-9330-v69-mkmlizer to finish
Stopping job with name mistralai-mistral-nemo-9330-v69-mkmlizer
%s, retrying in %s seconds...
Starting job with name mistralai-mistral-nemo-9330-v69-mkmlizer
Waiting for job on mistralai-mistral-nemo-9330-v69-mkmlizer to finish
Connection pool is full, discarding connection: %s. Connection pool size: %s
mistralai-mistral-nemo-9330-v69-mkmlizer: ╔═════════════════════════════════════════════════════════════════════╗
mistralai-mistral-nemo-9330-v69-mkmlizer: ║     _____            __           __                                ║
mistralai-mistral-nemo-9330-v69-mkmlizer: ║    / _/ /_ ___    __/ /  ___ ___ / /                                ║
mistralai-mistral-nemo-9330-v69-mkmlizer: ║   / _/ / // / |/|/ / _ \/ -_) -_) /                                 ║
mistralai-mistral-nemo-9330-v69-mkmlizer: ║  /_//_/\_, /|__,__/_//_/\__/\__/_/                                  ║
mistralai-mistral-nemo-9330-v69-mkmlizer: ║       /___/                                                         ║
mistralai-mistral-nemo-9330-v69-mkmlizer: ║                                                                     ║
mistralai-mistral-nemo-9330-v69-mkmlizer: ║  Version: 0.10.1                                                    ║
mistralai-mistral-nemo-9330-v69-mkmlizer: ║  Copyright 2023 MK ONE TECHNOLOGIES Inc.                            ║
mistralai-mistral-nemo-9330-v69-mkmlizer: ║  https://mk1.ai                                                     ║
mistralai-mistral-nemo-9330-v69-mkmlizer: ║                                                                     ║
mistralai-mistral-nemo-9330-v69-mkmlizer: ║  The license key for the current software has been verified as      ║
mistralai-mistral-nemo-9330-v69-mkmlizer: ║  belonging to:                                                      ║
mistralai-mistral-nemo-9330-v69-mkmlizer: ║                                                                     ║
mistralai-mistral-nemo-9330-v69-mkmlizer: ║  Chai Research Corp.                                                ║
mistralai-mistral-nemo-9330-v69-mkmlizer: ║  Account ID: 7997a29f-0ceb-4cc7-9adf-840c57b4ae6f                   ║
mistralai-mistral-nemo-9330-v69-mkmlizer: ║  Expiration: 2024-10-15 23:59:59                                    ║
mistralai-mistral-nemo-9330-v69-mkmlizer: ║                                                                     ║
mistralai-mistral-nemo-9330-v69-mkmlizer: ╚═════════════════════════════════════════════════════════════════════╝
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
mistralai-mistral-nemo-9330-v69-mkmlizer: Downloaded to shared memory in 47.054s
mistralai-mistral-nemo-9330-v69-mkmlizer: quantizing model to /dev/shm/model_cache, profile:s0, folder:/tmp/tmpeo7jy5vh, device:0
mistralai-mistral-nemo-9330-v69-mkmlizer: Saving flywheel model at /dev/shm/model_cache
mistralai-mistral-nemo-9330-v69-mkmlizer: quantized model in 35.557s
mistralai-mistral-nemo-9330-v69-mkmlizer: Processed model mistralai/Mistral-Nemo-Instruct-2407 in 82.611s
mistralai-mistral-nemo-9330-v69-mkmlizer: creating bucket guanaco-mkml-models
mistralai-mistral-nemo-9330-v69-mkmlizer: Bucket 's3://guanaco-mkml-models/' created
mistralai-mistral-nemo-9330-v69-mkmlizer: uploading /dev/shm/model_cache to s3://guanaco-mkml-models/mistralai-mistral-nemo-9330-v69
mistralai-mistral-nemo-9330-v69-mkmlizer: cp /dev/shm/model_cache/config.json s3://guanaco-mkml-models/mistralai-mistral-nemo-9330-v69/config.json
mistralai-mistral-nemo-9330-v69-mkmlizer: cp /dev/shm/model_cache/special_tokens_map.json s3://guanaco-mkml-models/mistralai-mistral-nemo-9330-v69/special_tokens_map.json
mistralai-mistral-nemo-9330-v69-mkmlizer: cp /dev/shm/model_cache/tokenizer_config.json s3://guanaco-mkml-models/mistralai-mistral-nemo-9330-v69/tokenizer_config.json
mistralai-mistral-nemo-9330-v69-mkmlizer: cp /dev/shm/model_cache/tokenizer.json s3://guanaco-mkml-models/mistralai-mistral-nemo-9330-v69/tokenizer.json
mistralai-mistral-nemo-9330-v69-mkmlizer: cp /dev/shm/model_cache/flywheel_model.0.safetensors s3://guanaco-mkml-models/mistralai-mistral-nemo-9330-v69/flywheel_model.0.safetensors
mistralai-mistral-nemo-9330-v69-mkmlizer: 
Loading 0:   0%|          | 0/363 [00:00<?, ?it/s]
Loading 0:   1%|▏         | 5/363 [00:00<00:10, 32.63it/s]
Loading 0:   4%|▎         | 13/363 [00:00<00:06, 52.87it/s]
Loading 0:   5%|▌         | 19/363 [00:00<00:07, 47.76it/s]
Loading 0:   7%|▋         | 24/363 [00:00<00:07, 45.93it/s]
Loading 0:   9%|▊         | 31/363 [00:00<00:06, 51.68it/s]
Loading 0:  10%|█         | 37/363 [00:00<00:06, 47.47it/s]
Loading 0:  12%|█▏        | 42/363 [00:00<00:07, 44.71it/s]
Loading 0:  13%|█▎        | 49/363 [00:01<00:06, 48.86it/s]
Loading 0:  15%|█▌        | 55/363 [00:01<00:06, 46.04it/s]
Loading 0:  17%|█▋        | 61/363 [00:01<00:08, 35.27it/s]
Loading 0:  18%|█▊        | 65/363 [00:01<00:08, 34.96it/s]
Loading 0:  20%|█▉        | 71/363 [00:01<00:07, 39.36it/s]
Loading 0:  21%|██        | 76/363 [00:01<00:07, 40.58it/s]
Loading 0:  22%|██▏       | 81/363 [00:01<00:06, 42.81it/s]
Loading 0:  24%|██▎       | 86/363 [00:01<00:06, 43.43it/s]
Loading 0:  25%|██▌       | 91/363 [00:02<00:07, 35.96it/s]
Loading 0:  27%|██▋       | 99/363 [00:02<00:05, 44.37it/s]
Loading 0:  29%|██▊       | 104/363 [00:02<00:05, 45.34it/s]
Loading 0:  30%|███       | 109/363 [00:02<00:05, 46.19it/s]
Loading 0:  31%|███▏      | 114/363 [00:02<00:06, 40.51it/s]
Loading 0:  33%|███▎      | 119/363 [00:02<00:05, 40.83it/s]
Loading 0:  35%|███▍      | 126/363 [00:02<00:05, 45.98it/s]
Loading 0:  36%|███▋      | 132/363 [00:03<00:05, 43.33it/s]
Loading 0:  38%|███▊      | 137/363 [00:03<00:05, 41.83it/s]
Loading 0:  39%|███▉      | 142/363 [00:03<00:06, 32.01it/s]
Loading 0:  40%|████      | 146/363 [00:03<00:06, 32.97it/s]
Loading 0:  41%|████▏     | 150/363 [00:03<00:06, 32.93it/s]
Loading 0:  43%|████▎     | 156/363 [00:03<00:05, 38.72it/s]
Loading 0:  44%|████▍     | 161/363 [00:03<00:05, 40.13it/s]
Loading 0:  46%|████▌     | 166/363 [00:04<00:04, 41.10it/s]
Loading 0:  47%|████▋     | 171/363 [00:04<00:04, 42.68it/s]
Loading 0:  48%|████▊     | 176/363 [00:04<00:05, 36.13it/s]
Loading 0:  50%|█████     | 183/363 [00:04<00:04, 43.39it/s]
Loading 0:  52%|█████▏    | 188/363 [00:04<00:04, 43.36it/s]
Loading 0:  53%|█████▎    | 193/363 [00:04<00:03, 42.94it/s]
Loading 0:  55%|█████▍    | 199/363 [00:04<00:03, 41.20it/s]
Loading 0:  56%|█████▌    | 204/363 [00:04<00:03, 40.35it/s]
Loading 0:  58%|█████▊    | 211/363 [00:05<00:03, 45.49it/s]
Loading 0:  60%|█████▉    | 217/363 [00:05<00:03, 43.49it/s]
Loading 0:  61%|██████▏   | 223/363 [00:05<00:04, 34.45it/s]
Loading 0:  63%|██████▎   | 227/363 [00:05<00:03, 35.38it/s]
Loading 0:  64%|██████▎   | 231/363 [00:05<00:03, 34.59it/s]
Loading 0:  65%|██████▌   | 237/363 [00:05<00:03, 39.54it/s]
Loading 0:  67%|██████▋   | 242/363 [00:05<00:02, 41.02it/s]
Loading 0:  68%|██████▊   | 247/363 [00:06<00:02, 42.08it/s]
Loading 0:  70%|██████▉   | 253/363 [00:06<00:02, 40.18it/s]
Loading 0:  71%|███████   | 258/363 [00:06<00:02, 39.71it/s]
Loading 0:  73%|███████▎  | 264/363 [00:06<00:02, 43.53it/s]
Loading 0:  74%|███████▍  | 269/363 [00:06<00:02, 43.90it/s]
Loading 0:  75%|███████▌  | 274/363 [00:06<00:01, 44.54it/s]
Loading 0:  77%|███████▋  | 280/363 [00:06<00:01, 42.02it/s]
Loading 0:  79%|███████▊  | 285/363 [00:06<00:01, 41.37it/s]
Loading 0:  80%|████████  | 291/363 [00:07<00:01, 44.83it/s]
Loading 0:  82%|████████▏ | 296/363 [00:07<00:01, 44.48it/s]
Loading 0:  83%|████████▎ | 302/363 [00:07<00:01, 47.86it/s]
Loading 0:  85%|████████▍ | 307/363 [00:14<00:21,  2.55it/s]
Loading 0:  86%|████████▌ | 312/363 [00:14<00:14,  3.48it/s]
Loading 0:  88%|████████▊ | 320/363 [00:14<00:07,  5.56it/s]
Loading 0:  90%|████████▉ | 326/363 [00:14<00:04,  7.49it/s]
Loading 0:  91%|█████████ | 331/363 [00:14<00:03,  9.55it/s]
Loading 0:  93%|█████████▎| 338/363 [00:14<00:01, 13.51it/s]
Loading 0:  95%|█████████▍| 344/363 [00:14<00:01, 16.85it/s]
Loading 0:  96%|█████████▌| 349/363 [00:14<00:00, 19.91it/s]
Loading 0:  98%|█████████▊| 356/363 [00:15<00:00, 26.14it/s]
Loading 0: 100%|█████████▉| 362/363 [00:15<00:00, 29.39it/s]
                                                            

Job mistralai-mistral-nemo-9330-v69-mkmlizer completed after 105.79s with status: succeeded
Stopping job with name mistralai-mistral-nemo-9330-v69-mkmlizer
Pipeline stage MKMLizer completed in 107.06s
Running pipeline stage MKMLKubeTemplater
Pipeline stage MKMLKubeTemplater completed in 0.08s
Running pipeline stage ISVCDeployer
Creating inference service mistralai-mistral-nemo-9330-v69
Waiting for inference service mistralai-mistral-nemo-9330-v69 to be ready
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Inference service mistralai-mistral-nemo-9330-v69 ready after 180.8274371623993s
Pipeline stage ISVCDeployer completed in 181.44s
Running pipeline stage StressChecker
Received healthy response to inference request in 2.3171820640563965s
Received healthy response to inference request in 2.1629750728607178s
Received healthy response to inference request in 1.9343085289001465s
Received healthy response to inference request in 1.6892335414886475s
Received healthy response to inference request in 1.6187148094177246s
5 requests
0 failed requests
5th percentile: 1.6328185558319093
10th percentile: 1.6469223022460937
20th percentile: 1.6751297950744628
30th percentile: 1.7382485389709472
40th percentile: 1.836278533935547
50th percentile: 1.9343085289001465
60th percentile: 2.025775146484375
70th percentile: 2.1172417640686034
80th percentile: 2.1938164710998533
90th percentile: 2.255499267578125
95th percentile: 2.286340665817261
99th percentile: 2.311013784408569
mean time: 1.9444828033447266
Pipeline stage StressChecker completed in 10.62s
mistralai-mistral-nemo-_9330_v69 status is now deployed due to DeploymentManager action
mistralai-mistral-nemo-_9330_v69 status is now inactive due to auto deactivation removed underperforming models

Usage Metrics

Latency Metrics