mistralai-mistral-nemo_9330

developer_uid: chai_backend_admin

submission_id: mistralai-mistral-nemo_9330_v160

model_name: mistralai-mistral-nemo_9330_v160

model_group: mistralai/Mistral-Nemo-I

status: torndown

timestamp: 2024-10-11T22:05:20+00:00

num_battles: 8159

num_wins: 3971

celo_rating: 1241.95

family_friendly_score: 0.5546985473711274

family_friendly_standard_error: 0.005602841297404391

submission_type: basic

model_repo: mistralai/Mistral-Nemo-Instruct-2407

model_architecture: MistralForCausalLM

model_num_parameters: 12772070400.0

best_of: 8

max_input_tokens: 1024

max_output_tokens: 64

display_name: mistralai-mistral-nemo_9330_v160

is_internal_developer: True

language_model: mistralai/Mistral-Nemo-Instruct-2407

model_size: 13B

ranking_group: single

us_pacific_date: 2024-10-11

win_ratio: 0.4867018016913838

generation_params: {'temperature': 1.0, 'top_p': 1.0, 'min_p': 0.0, 'top_k': 40, 'presence_penalty': 0.0, 'frequency_penalty': 0.0, 'stopping_words': ['\n', '<|eot_id|>', '<|end_of_text|>', 'You:'], 'max_input_tokens': 1024, 'best_of': 8, 'max_output_tokens': 64}

formatter: {'memory_template': '', 'prompt_template': '', 'bot_template': '{bot_name}: {message}\n', 'user_template': '{user_name}: {message}\n', 'response_template': '### Response:\n{bot_name}:', 'truncate_by_message': False}

Resubmit model

Shutdown handler not registered because Python interpreter is not running in the main thread
run pipeline %s
run pipeline stage %s
Running pipeline stage MKMLizer
Starting job with name mistralai-mistral-nemo-9330-v160-mkmlizer
Waiting for job on mistralai-mistral-nemo-9330-v160-mkmlizer to finish
mistralai-mistral-nemo-9330-v160-mkmlizer: ╔═════════════════════════════════════════════════════════════════════╗
mistralai-mistral-nemo-9330-v160-mkmlizer: ║     _____            __           __                                ║
mistralai-mistral-nemo-9330-v160-mkmlizer: ║    / _/ /_ ___    __/ /  ___ ___ / /                                ║
mistralai-mistral-nemo-9330-v160-mkmlizer: ║   / _/ / // / |/|/ / _ \/ -_) -_) /                                 ║
mistralai-mistral-nemo-9330-v160-mkmlizer: ║  /_//_/\_, /|__,__/_//_/\__/\__/_/                                  ║
mistralai-mistral-nemo-9330-v160-mkmlizer: ║       /___/                                                         ║
mistralai-mistral-nemo-9330-v160-mkmlizer: ║                                                                     ║
mistralai-mistral-nemo-9330-v160-mkmlizer: ║  Version: 0.11.12                                                   ║
mistralai-mistral-nemo-9330-v160-mkmlizer: ║  Copyright 2023 MK ONE TECHNOLOGIES Inc.                            ║
mistralai-mistral-nemo-9330-v160-mkmlizer: ║  https://mk1.ai                                                     ║
mistralai-mistral-nemo-9330-v160-mkmlizer: ║                                                                     ║
mistralai-mistral-nemo-9330-v160-mkmlizer: ║  The license key for the current software has been verified as      ║
mistralai-mistral-nemo-9330-v160-mkmlizer: ║  belonging to:                                                      ║
mistralai-mistral-nemo-9330-v160-mkmlizer: ║                                                                     ║
mistralai-mistral-nemo-9330-v160-mkmlizer: ║  Chai Research Corp.                                                ║
mistralai-mistral-nemo-9330-v160-mkmlizer: ║  Account ID: 7997a29f-0ceb-4cc7-9adf-840c57b4ae6f                   ║
mistralai-mistral-nemo-9330-v160-mkmlizer: ║  Expiration: 2024-10-15 23:59:59                                    ║
mistralai-mistral-nemo-9330-v160-mkmlizer: ║                                                                     ║
mistralai-mistral-nemo-9330-v160-mkmlizer: ╚═════════════════════════════════════════════════════════════════════╝
mistralai-mistral-nemo-9330-v160-mkmlizer: Downloaded to shared memory in 47.482s
mistralai-mistral-nemo-9330-v160-mkmlizer: quantizing model to /dev/shm/model_cache, profile:s0, folder:/tmp/tmp5aoosino, device:0
mistralai-mistral-nemo-9330-v160-mkmlizer: Saving flywheel model at /dev/shm/model_cache
mistralai-mistral-nemo-9330-v160-mkmlizer: quantized model in 35.403s
mistralai-mistral-nemo-9330-v160-mkmlizer: Processed model mistralai/Mistral-Nemo-Instruct-2407 in 82.886s
mistralai-mistral-nemo-9330-v160-mkmlizer: creating bucket guanaco-mkml-models
mistralai-mistral-nemo-9330-v160-mkmlizer: Bucket 's3://guanaco-mkml-models/' created
mistralai-mistral-nemo-9330-v160-mkmlizer: uploading /dev/shm/model_cache to s3://guanaco-mkml-models/mistralai-mistral-nemo-9330-v160
mistralai-mistral-nemo-9330-v160-mkmlizer: cp /dev/shm/model_cache/config.json s3://guanaco-mkml-models/mistralai-mistral-nemo-9330-v160/config.json
mistralai-mistral-nemo-9330-v160-mkmlizer: cp /dev/shm/model_cache/special_tokens_map.json s3://guanaco-mkml-models/mistralai-mistral-nemo-9330-v160/special_tokens_map.json
mistralai-mistral-nemo-9330-v160-mkmlizer: cp /dev/shm/model_cache/tokenizer_config.json s3://guanaco-mkml-models/mistralai-mistral-nemo-9330-v160/tokenizer_config.json
mistralai-mistral-nemo-9330-v160-mkmlizer: cp /dev/shm/model_cache/tokenizer.json s3://guanaco-mkml-models/mistralai-mistral-nemo-9330-v160/tokenizer.json
mistralai-mistral-nemo-9330-v160-mkmlizer: cp /dev/shm/model_cache/flywheel_model.0.safetensors s3://guanaco-mkml-models/mistralai-mistral-nemo-9330-v160/flywheel_model.0.safetensors
mistralai-mistral-nemo-9330-v160-mkmlizer: 
Loading 0:   0%|          | 0/363 [00:00<?, ?it/s]
Loading 0:   1%|▏         | 5/363 [00:00<00:11, 32.34it/s]
Loading 0:   4%|▎         | 13/363 [00:00<00:06, 52.19it/s]
Loading 0:   5%|▌         | 19/363 [00:00<00:07, 47.05it/s]
Loading 0:   7%|▋         | 24/363 [00:00<00:07, 45.26it/s]
Loading 0:   9%|▊         | 31/363 [00:00<00:06, 50.81it/s]
Loading 0:  10%|█         | 37/363 [00:00<00:06, 47.34it/s]
Loading 0:  12%|█▏        | 42/363 [00:00<00:07, 45.81it/s]
Loading 0:  13%|█▎        | 49/363 [00:01<00:06, 50.76it/s]
Loading 0:  15%|█▌        | 55/363 [00:01<00:06, 47.50it/s]
Loading 0:  17%|█▋        | 61/363 [00:01<00:08, 35.09it/s]
Loading 0:  18%|█▊        | 66/363 [00:01<00:08, 36.57it/s]
Loading 0:  20%|█▉        | 72/363 [00:01<00:07, 40.90it/s]
Loading 0:  21%|██▏       | 78/363 [00:01<00:06, 40.75it/s]
Loading 0:  23%|██▎       | 83/363 [00:01<00:06, 41.43it/s]
Loading 0:  25%|██▍       | 90/363 [00:02<00:05, 46.19it/s]
Loading 0:  26%|██▌       | 95/363 [00:02<00:05, 47.09it/s]
Loading 0:  28%|██▊       | 100/363 [00:02<00:06, 39.80it/s]
Loading 0:  29%|██▉       | 107/363 [00:02<00:05, 46.96it/s]
Loading 0:  31%|███       | 113/363 [00:02<00:05, 42.26it/s]
Loading 0:  33%|███▎      | 118/363 [00:02<00:05, 42.09it/s]
Loading 0:  35%|███▍      | 126/363 [00:02<00:04, 48.74it/s]
Loading 0:  36%|███▋      | 132/363 [00:03<00:05, 44.91it/s]
Loading 0:  38%|███▊      | 137/363 [00:03<00:05, 43.87it/s]
Loading 0:  39%|███▉      | 142/363 [00:03<00:06, 34.03it/s]
Loading 0:  40%|████      | 146/363 [00:03<00:06, 34.78it/s]
Loading 0:  41%|████▏     | 150/363 [00:03<00:06, 34.38it/s]
Loading 0:  43%|████▎     | 156/363 [00:03<00:05, 39.97it/s]
Loading 0:  44%|████▍     | 161/363 [00:03<00:04, 41.45it/s]
Loading 0:  46%|████▌     | 166/363 [00:03<00:04, 42.94it/s]
Loading 0:  47%|████▋     | 172/363 [00:04<00:04, 41.87it/s]
Loading 0:  49%|████▉     | 177/363 [00:04<00:04, 41.27it/s]
Loading 0:  50%|█████     | 183/363 [00:04<00:03, 45.78it/s]
Loading 0:  52%|█████▏    | 188/363 [00:04<00:03, 45.98it/s]
Loading 0:  53%|█████▎    | 193/363 [00:04<00:03, 46.13it/s]
Loading 0:  55%|█████▍    | 199/363 [00:04<00:03, 43.84it/s]
Loading 0:  56%|█████▌    | 204/363 [00:04<00:03, 43.17it/s]
Loading 0:  58%|█████▊    | 210/363 [00:04<00:03, 47.29it/s]
Loading 0:  59%|█████▉    | 215/363 [00:04<00:03, 46.24it/s]
Loading 0:  61%|██████    | 220/363 [00:05<00:03, 46.69it/s]
Loading 0:  62%|██████▏   | 225/363 [00:05<00:04, 29.38it/s]
Loading 0:  63%|██████▎   | 230/363 [00:05<00:04, 31.95it/s]
Loading 0:  65%|██████▌   | 237/363 [00:05<00:03, 39.62it/s]
Loading 0:  67%|██████▋   | 242/363 [00:05<00:03, 40.15it/s]
Loading 0:  68%|██████▊   | 247/363 [00:05<00:02, 41.41it/s]
Loading 0:  69%|██████▉   | 252/363 [00:05<00:02, 43.39it/s]
Loading 0:  71%|███████   | 257/363 [00:06<00:02, 37.51it/s]
Loading 0:  73%|███████▎  | 264/363 [00:06<00:02, 45.19it/s]
Loading 0:  74%|███████▍  | 269/363 [00:06<00:02, 45.00it/s]
Loading 0:  75%|███████▌  | 274/363 [00:06<00:01, 45.21it/s]
Loading 0:  77%|███████▋  | 280/363 [00:06<00:01, 43.67it/s]
Loading 0:  79%|███████▊  | 285/363 [00:06<00:01, 43.05it/s]
Loading 0:  80%|████████  | 291/363 [00:06<00:01, 46.72it/s]
Loading 0:  82%|████████▏ | 296/363 [00:06<00:01, 43.62it/s]
Loading 0:  83%|████████▎ | 302/363 [00:07<00:01, 47.72it/s]
Loading 0:  85%|████████▍ | 307/363 [00:13<00:21,  2.56it/s]
Loading 0:  86%|████████▌ | 312/363 [00:13<00:14,  3.49it/s]
Loading 0:  88%|████████▊ | 320/363 [00:14<00:07,  5.58it/s]
Loading 0:  90%|████████▉ | 326/363 [00:14<00:04,  7.50it/s]
Loading 0:  91%|█████████ | 331/363 [00:14<00:03,  9.53it/s]
Loading 0:  93%|█████████▎| 338/363 [00:14<00:01, 13.46it/s]
Loading 0:  95%|█████████▍| 344/363 [00:14<00:01, 16.85it/s]
Loading 0:  96%|█████████▌| 349/363 [00:14<00:00, 20.03it/s]
Loading 0:  98%|█████████▊| 356/363 [00:14<00:00, 26.20it/s]
Loading 0: 100%|█████████▉| 362/363 [00:14<00:00, 29.45it/s]
                                                            
Job mistralai-mistral-nemo-9330-v160-mkmlizer completed after 103.91s with status: succeeded
Stopping job with name mistralai-mistral-nemo-9330-v160-mkmlizer
Pipeline stage MKMLizer completed in 104.41s
run pipeline stage %s
Running pipeline stage MKMLTemplater
Pipeline stage MKMLTemplater completed in 0.18s
run pipeline stage %s
Running pipeline stage MKMLDeployer
Creating inference service mistralai-mistral-nemo-9330-v160
Waiting for inference service mistralai-mistral-nemo-9330-v160 to be ready
Inference service mistralai-mistral-nemo-9330-v160 ready after 140.51224970817566s
Pipeline stage MKMLDeployer completed in 141.36s
run pipeline stage %s
Running pipeline stage StressChecker
Received healthy response to inference request in 2.113466739654541s
Received healthy response to inference request in 1.576326608657837s
Received healthy response to inference request in 1.4697692394256592s
Received healthy response to inference request in 1.5112330913543701s
Received healthy response to inference request in 1.555737018585205s
5 requests
0 failed requests
5th percentile: 1.4780620098114015
10th percentile: 1.4863547801971435
20th percentile: 1.5029403209686278
30th percentile: 1.520133876800537
40th percentile: 1.537935447692871
50th percentile: 1.555737018585205
60th percentile: 1.5639728546142577
70th percentile: 1.5722086906433106
80th percentile: 1.683754634857178
90th percentile: 1.8986106872558595
95th percentile: 2.0060387134552
99th percentile: 2.091981134414673
mean time: 1.6453065395355224
Pipeline stage StressChecker completed in 9.58s
Shutdown handler de-registered
mistralai-mistral-nemo_9330_v160 status is now deployed due to DeploymentManager action
mistralai-mistral-nemo_9330_v160 status is now inactive due to auto deactivation removed underperforming models
mistralai-mistral-nemo_9330_v160 status is now torndown due to DeploymentManager action