mistralai-mistral-nemo-_9330

developer_uid: albert_chai
submission_id: mistralai-mistral-nemo-_9330_v59
model_name: nemo-dpo-bo8x1024_T12
model_group: mistralai/Mistral-Nemo-I
status: torndown
timestamp: 2024-08-24T05:54:53+00:00
num_battles: 10160
num_wins: 5128
celo_rating: 1238.91
family_friendly_score: 0.0
submission_type: basic
model_repo: mistralai/Mistral-Nemo-Instruct-2407
model_architecture: MistralForCausalLM
model_num_parameters: 12772070400.0
best_of: 8
max_input_tokens: 1024
max_output_tokens: 64
display_name: nemo-dpo-bo8x1024_T12
is_internal_developer: True
language_model: mistralai/Mistral-Nemo-Instruct-2407
model_size: 13B
ranking_group: single
us_pacific_date: 2024-08-23
win_ratio: 0.5047244094488189
generation_params: {'temperature': 1.2, 'top_p': 1.0, 'min_p': 0.0, 'top_k': 40, 'presence_penalty': 0.0, 'frequency_penalty': 0.0, 'stopping_words': ['\n'], 'max_input_tokens': 1024, 'best_of': 8, 'max_output_tokens': 64}
formatter: {'memory_template': '', 'prompt_template': '', 'bot_template': '{bot_name}: {message}\n', 'user_template': '{user_name}: {message}\n', 'response_template': '{bot_name}:', 'truncate_by_message': False}
Resubmit model
Running pipeline stage MKMLizer
Starting job with name mistralai-mistral-nemo-9330-v59-mkmlizer
Waiting for job on mistralai-mistral-nemo-9330-v59-mkmlizer to finish
mistralai-mistral-nemo-9330-v59-mkmlizer: ╔═════════════════════════════════════════════════════════════════════╗
mistralai-mistral-nemo-9330-v59-mkmlizer: ║     _____            __           __                                ║
mistralai-mistral-nemo-9330-v59-mkmlizer: ║    / _/ /_ ___    __/ /  ___ ___ / /                                ║
mistralai-mistral-nemo-9330-v59-mkmlizer: ║   / _/ / // / |/|/ / _ \/ -_) -_) /                                 ║
mistralai-mistral-nemo-9330-v59-mkmlizer: ║  /_//_/\_, /|__,__/_//_/\__/\__/_/                                  ║
mistralai-mistral-nemo-9330-v59-mkmlizer: ║       /___/                                                         ║
mistralai-mistral-nemo-9330-v59-mkmlizer: ║                                                                     ║
mistralai-mistral-nemo-9330-v59-mkmlizer: ║  Version: 0.10.1                                                    ║
mistralai-mistral-nemo-9330-v59-mkmlizer: ║  Copyright 2023 MK ONE TECHNOLOGIES Inc.                            ║
mistralai-mistral-nemo-9330-v59-mkmlizer: ║  https://mk1.ai                                                     ║
mistralai-mistral-nemo-9330-v59-mkmlizer: ║                                                                     ║
mistralai-mistral-nemo-9330-v59-mkmlizer: ║  The license key for the current software has been verified as      ║
mistralai-mistral-nemo-9330-v59-mkmlizer: ║  belonging to:                                                      ║
mistralai-mistral-nemo-9330-v59-mkmlizer: ║                                                                     ║
mistralai-mistral-nemo-9330-v59-mkmlizer: ║  Chai Research Corp.                                                ║
mistralai-mistral-nemo-9330-v59-mkmlizer: ║  Account ID: 7997a29f-0ceb-4cc7-9adf-840c57b4ae6f                   ║
mistralai-mistral-nemo-9330-v59-mkmlizer: ║  Expiration: 2024-10-15 23:59:59                                    ║
mistralai-mistral-nemo-9330-v59-mkmlizer: ║                                                                     ║
mistralai-mistral-nemo-9330-v59-mkmlizer: ╚═════════════════════════════════════════════════════════════════════╝
mistralai-mistral-nemo-9330-v59-mkmlizer: Downloaded to shared memory in 48.787s
mistralai-mistral-nemo-9330-v59-mkmlizer: quantizing model to /dev/shm/model_cache, profile:s0, folder:/tmp/tmpghf7dsvw, device:0
mistralai-mistral-nemo-9330-v59-mkmlizer: Saving flywheel model at /dev/shm/model_cache
mistralai-mistral-nemo-9330-v59-mkmlizer: quantized model in 37.561s
mistralai-mistral-nemo-9330-v59-mkmlizer: Processed model mistralai/Mistral-Nemo-Instruct-2407 in 86.348s
mistralai-mistral-nemo-9330-v59-mkmlizer: creating bucket guanaco-mkml-models
mistralai-mistral-nemo-9330-v59-mkmlizer: Bucket 's3://guanaco-mkml-models/' created
mistralai-mistral-nemo-9330-v59-mkmlizer: uploading /dev/shm/model_cache to s3://guanaco-mkml-models/mistralai-mistral-nemo-9330-v59
mistralai-mistral-nemo-9330-v59-mkmlizer: cp /dev/shm/model_cache/config.json s3://guanaco-mkml-models/mistralai-mistral-nemo-9330-v59/config.json
mistralai-mistral-nemo-9330-v59-mkmlizer: cp /dev/shm/model_cache/special_tokens_map.json s3://guanaco-mkml-models/mistralai-mistral-nemo-9330-v59/special_tokens_map.json
mistralai-mistral-nemo-9330-v59-mkmlizer: cp /dev/shm/model_cache/tokenizer_config.json s3://guanaco-mkml-models/mistralai-mistral-nemo-9330-v59/tokenizer_config.json
mistralai-mistral-nemo-9330-v59-mkmlizer: cp /dev/shm/model_cache/tokenizer.json s3://guanaco-mkml-models/mistralai-mistral-nemo-9330-v59/tokenizer.json
mistralai-mistral-nemo-9330-v59-mkmlizer: cp /dev/shm/model_cache/flywheel_model.0.safetensors s3://guanaco-mkml-models/mistralai-mistral-nemo-9330-v59/flywheel_model.0.safetensors
mistralai-mistral-nemo-9330-v59-mkmlizer: 
Loading 0:   0%|          | 0/363 [00:00<?, ?it/s]
Loading 0:   1%|▏         | 5/363 [00:00<00:11, 31.00it/s]
Loading 0:   3%|▎         | 12/363 [00:00<00:07, 47.44it/s]
Loading 0:   5%|▍         | 18/363 [00:00<00:07, 48.59it/s]
Loading 0:   7%|▋         | 24/363 [00:00<00:08, 39.52it/s]
Loading 0:   9%|▊         | 31/363 [00:00<00:07, 45.17it/s]
Loading 0:  10%|█         | 37/363 [00:00<00:07, 41.86it/s]
Loading 0:  12%|█▏        | 42/363 [00:01<00:07, 40.59it/s]
Loading 0:  13%|█▎        | 48/363 [00:01<00:06, 45.30it/s]
Loading 0:  15%|█▍        | 53/363 [00:01<00:06, 44.73it/s]
Loading 0:  16%|█▌        | 58/363 [00:01<00:06, 45.17it/s]
Loading 0:  17%|█▋        | 63/363 [00:01<00:09, 30.11it/s]
Loading 0:  19%|█▉        | 69/363 [00:01<00:09, 32.19it/s]
Loading 0:  20%|██        | 73/363 [00:01<00:09, 31.45it/s]
Loading 0:  22%|██▏       | 80/363 [00:02<00:07, 38.49it/s]
Loading 0:  23%|██▎       | 85/363 [00:02<00:07, 38.40it/s]
Loading 0:  25%|██▍       | 90/363 [00:02<00:07, 38.11it/s]
Loading 0:  26%|██▌       | 95/363 [00:02<00:06, 40.54it/s]
Loading 0:  28%|██▊       | 100/363 [00:02<00:07, 34.34it/s]
Loading 0:  29%|██▉       | 106/363 [00:02<00:06, 38.82it/s]
Loading 0:  31%|███       | 112/363 [00:02<00:05, 42.88it/s]
Loading 0:  32%|███▏      | 117/363 [00:02<00:05, 41.16it/s]
Loading 0:  34%|███▎      | 122/363 [00:03<00:05, 42.68it/s]
Loading 0:  35%|███▍      | 127/363 [00:03<00:06, 35.12it/s]
Loading 0:  37%|███▋      | 134/363 [00:03<00:05, 41.62it/s]
Loading 0:  38%|███▊      | 139/363 [00:03<00:05, 41.87it/s]
Loading 0:  40%|███▉      | 144/363 [00:03<00:08, 25.82it/s]
Loading 0:  41%|████      | 149/363 [00:04<00:07, 28.03it/s]
Loading 0:  43%|████▎     | 156/363 [00:04<00:06, 34.44it/s]
Loading 0:  44%|████▍     | 161/363 [00:04<00:05, 35.71it/s]
Loading 0:  46%|████▌     | 166/363 [00:04<00:05, 37.21it/s]
Loading 0:  47%|████▋     | 171/363 [00:04<00:04, 39.69it/s]
Loading 0:  48%|████▊     | 176/363 [00:04<00:05, 32.90it/s]
Loading 0:  50%|█████     | 183/363 [00:04<00:04, 39.51it/s]
Loading 0:  52%|█████▏    | 188/363 [00:04<00:04, 39.60it/s]
Loading 0:  53%|█████▎    | 193/363 [00:05<00:04, 39.86it/s]
Loading 0:  55%|█████▍    | 198/363 [00:05<00:03, 41.78it/s]
Loading 0:  56%|█████▌    | 203/363 [00:05<00:04, 34.17it/s]
Loading 0:  57%|█████▋    | 208/363 [00:05<00:04, 37.62it/s]
Loading 0:  59%|█████▊    | 213/363 [00:05<00:04, 36.03it/s]
Loading 0:  60%|██████    | 218/363 [00:05<00:03, 37.39it/s]
Loading 0:  61%|██████▏   | 223/363 [00:06<00:04, 28.67it/s]
Loading 0:  63%|██████▎   | 227/363 [00:06<00:04, 29.50it/s]
Loading 0:  64%|██████▎   | 231/363 [00:06<00:04, 28.74it/s]
Loading 0:  65%|██████▌   | 237/363 [00:06<00:03, 34.18it/s]
Loading 0:  66%|██████▋   | 241/363 [00:06<00:03, 34.19it/s]
Loading 0:  68%|██████▊   | 246/363 [00:06<00:03, 37.46it/s]
Loading 0:  69%|██████▉   | 250/363 [00:06<00:03, 36.64it/s]
Loading 0:  70%|███████   | 255/363 [00:06<00:02, 38.11it/s]
Loading 0:  71%|███████▏  | 259/363 [00:07<00:02, 37.09it/s]
Loading 0:  73%|███████▎  | 264/363 [00:07<00:02, 39.60it/s]
Loading 0:  74%|███████▍  | 269/363 [00:07<00:02, 39.50it/s]
Loading 0:  75%|███████▌  | 273/363 [00:07<00:02, 39.24it/s]
Loading 0:  76%|███████▋  | 277/363 [00:07<00:02, 35.90it/s]
Loading 0:  78%|███████▊  | 282/363 [00:07<00:02, 37.74it/s]
Loading 0:  79%|███████▉  | 286/363 [00:07<00:02, 36.79it/s]
Loading 0:  80%|████████  | 291/363 [00:07<00:01, 39.13it/s]
Loading 0:  81%|████████▏ | 295/363 [00:07<00:01, 38.14it/s]
Loading 0:  82%|████████▏ | 299/363 [00:08<00:01, 36.63it/s]
Loading 0:  84%|████████▎ | 304/363 [00:14<00:28,  2.09it/s]
Loading 0:  85%|████████▍ | 307/363 [00:15<00:21,  2.63it/s]
Loading 0:  86%|████████▌ | 312/363 [00:15<00:13,  3.89it/s]
Loading 0:  88%|████████▊ | 320/363 [00:15<00:06,  6.66it/s]
Loading 0:  90%|████████▉ | 325/363 [00:15<00:04,  8.82it/s]
Loading 0:  91%|█████████ | 330/363 [00:15<00:03, 10.74it/s]
Loading 0:  93%|█████████▎| 336/363 [00:15<00:01, 14.74it/s]
Loading 0:  94%|█████████▍| 341/363 [00:15<00:01, 17.81it/s]
Loading 0:  95%|█████████▌| 346/363 [00:15<00:00, 21.78it/s]
Loading 0:  97%|█████████▋| 351/363 [00:16<00:00, 25.55it/s]
Loading 0:  98%|█████████▊| 356/363 [00:16<00:00, 28.60it/s]
Loading 0:  99%|█████████▉| 361/363 [00:16<00:00, 31.87it/s]
                                                            
Job mistralai-mistral-nemo-9330-v59-mkmlizer completed after 128.93s with status: succeeded
Stopping job with name mistralai-mistral-nemo-9330-v59-mkmlizer
Pipeline stage MKMLizer completed in 129.96s
Running pipeline stage MKMLKubeTemplater
Pipeline stage MKMLKubeTemplater completed in 0.10s
Running pipeline stage ISVCDeployer
Creating inference service mistralai-mistral-nemo-9330-v59
Waiting for inference service mistralai-mistral-nemo-9330-v59 to be ready
Inference service mistralai-mistral-nemo-9330-v59 ready after 232.4188244342804s
Pipeline stage ISVCDeployer completed in 233.24s
Running pipeline stage StressChecker
Received healthy response to inference request in 1.4586868286132812s
Received healthy response to inference request in 1.751861333847046s
Received healthy response to inference request in 1.6973342895507812s
Received healthy response to inference request in 1.7602057456970215s
Received healthy response to inference request in 1.971355676651001s
5 requests
0 failed requests
5th percentile: 1.5064163208007812
10th percentile: 1.5541458129882812
20th percentile: 1.6496047973632812
30th percentile: 1.708239698410034
40th percentile: 1.73005051612854
50th percentile: 1.751861333847046
60th percentile: 1.755199098587036
70th percentile: 1.7585368633270264
80th percentile: 1.8024357318878175
90th percentile: 1.8868957042694092
95th percentile: 1.929125690460205
99th percentile: 1.9629096794128418
mean time: 1.7278887748718261
Pipeline stage StressChecker completed in 9.34s
mistralai-mistral-nemo-_9330_v59 status is now deployed due to DeploymentManager action
mistralai-mistral-nemo-_9330_v59 status is now inactive due to auto deactivation removed underperforming models
mistralai-mistral-nemo-_9330_v59 status is now torndown due to DeploymentManager action