mistralai-mistral-nem_93303

submission_id: mistralai-mistral-nem_93303_v262
developer_uid: azuruce
status: failed
model_repo: mistralai/Mistral-Nemo-Instruct-2407
generation_params: {'temperature': 1.0, 'top_p': 0.99, 'min_p': 0.0, 'top_k': 80, 'presence_penalty': 0.0, 'frequency_penalty': 0.0, 'stopping_words': ['\n,', '</s>,', '####,', 'Bot:,', 'User:,', 'You:,', '<|im_end|>,', '<|eot_id|>,', 'Me:'], 'max_input_tokens': 1024, 'best_of': 8, 'max_output_tokens': 64}
formatter: {'memory_template': '<|im_start|>system\n{memory}<|im_end|>\n', 'prompt_template': '<|im_start|>user\n{prompt}<|im_end|>\n', 'bot_template': '{bot_name}: {message}\n', 'user_template': '{user_name}: {message}\n', 'response_template': '{bot_name}:', 'truncate_by_message': False}
timestamp: 2025-01-10T18:54:49+00:00
model_name: mistralai-mistral-nem_93303_v262
Resubmit model
Shutdown handler not registered because Python interpreter is not running in the main thread
run pipeline %s
run pipeline stage %s
Running pipeline stage MKMLizer
Starting job with name mistralai-mistral-nem-93303-v262-mkmlizer
Waiting for job on mistralai-mistral-nem-93303-v262-mkmlizer to finish
mistralai-mistral-nem-93303-v262-mkmlizer: ╔═════════════════════════════════════════════════════════════════════╗
mistralai-mistral-nem-93303-v262-mkmlizer: ║     _____            __           __                                ║
mistralai-mistral-nem-93303-v262-mkmlizer: ║    / _/ /_ ___    __/ /  ___ ___ / /                                ║
mistralai-mistral-nem-93303-v262-mkmlizer: ║   / _/ / // / |/|/ / _ \/ -_) -_) /                                 ║
mistralai-mistral-nem-93303-v262-mkmlizer: ║  /_//_/\_, /|__,__/_//_/\__/\__/_/                                  ║
mistralai-mistral-nem-93303-v262-mkmlizer: ║       /___/                                                         ║
mistralai-mistral-nem-93303-v262-mkmlizer: ║                                                                     ║
mistralai-mistral-nem-93303-v262-mkmlizer: ║  Version: 0.11.12                                                   ║
mistralai-mistral-nem-93303-v262-mkmlizer: ║  Copyright 2023 MK ONE TECHNOLOGIES Inc.                            ║
mistralai-mistral-nem-93303-v262-mkmlizer: ║  https://mk1.ai                                                     ║
mistralai-mistral-nem-93303-v262-mkmlizer: ║                                                                     ║
mistralai-mistral-nem-93303-v262-mkmlizer: ║  The license key for the current software has been verified as      ║
mistralai-mistral-nem-93303-v262-mkmlizer: ║  belonging to:                                                      ║
mistralai-mistral-nem-93303-v262-mkmlizer: ║                                                                     ║
mistralai-mistral-nem-93303-v262-mkmlizer: ║  Chai Research Corp.                                                ║
mistralai-mistral-nem-93303-v262-mkmlizer: ║  Account ID: 7997a29f-0ceb-4cc7-9adf-840c57b4ae6f                   ║
mistralai-mistral-nem-93303-v262-mkmlizer: ║  Expiration: 2025-04-15 23:59:59                                    ║
mistralai-mistral-nem-93303-v262-mkmlizer: ║                                                                     ║
mistralai-mistral-nem-93303-v262-mkmlizer: ╚═════════════════════════════════════════════════════════════════════╝
mistralai-mistral-nem-93303-v262-mkmlizer: Downloaded to shared memory in 54.529s
mistralai-mistral-nem-93303-v262-mkmlizer: quantizing model to /dev/shm/model_cache, profile:s0, folder:/tmp/tmphw8ye9l1, device:0
mistralai-mistral-nem-93303-v262-mkmlizer: Saving flywheel model at /dev/shm/model_cache
mistralai-mistral-nem-93303-v262-mkmlizer: quantized model in 36.079s
mistralai-mistral-nem-93303-v262-mkmlizer: Processed model mistralai/Mistral-Nemo-Instruct-2407 in 90.609s
mistralai-mistral-nem-93303-v262-mkmlizer: creating bucket guanaco-mkml-models
mistralai-mistral-nem-93303-v262-mkmlizer: Bucket 's3://guanaco-mkml-models/' created
mistralai-mistral-nem-93303-v262-mkmlizer: uploading /dev/shm/model_cache to s3://guanaco-mkml-models/mistralai-mistral-nem-93303-v262
mistralai-mistral-nem-93303-v262-mkmlizer: cp /dev/shm/model_cache/special_tokens_map.json s3://guanaco-mkml-models/mistralai-mistral-nem-93303-v262/special_tokens_map.json
mistralai-mistral-nem-93303-v262-mkmlizer: cp /dev/shm/model_cache/config.json s3://guanaco-mkml-models/mistralai-mistral-nem-93303-v262/config.json
mistralai-mistral-nem-93303-v262-mkmlizer: cp /dev/shm/model_cache/tokenizer_config.json s3://guanaco-mkml-models/mistralai-mistral-nem-93303-v262/tokenizer_config.json
mistralai-mistral-nem-93303-v262-mkmlizer: cp /dev/shm/model_cache/tokenizer.json s3://guanaco-mkml-models/mistralai-mistral-nem-93303-v262/tokenizer.json
mistralai-mistral-nem-93303-v262-mkmlizer: cp /dev/shm/model_cache/flywheel_model.0.safetensors s3://guanaco-mkml-models/mistralai-mistral-nem-93303-v262/flywheel_model.0.safetensors
mistralai-mistral-nem-93303-v262-mkmlizer: 
Loading 0:   0%|          | 0/363 [00:00<?, ?it/s]
Loading 0:   1%|▏         | 5/363 [00:00<00:11, 30.56it/s]
Loading 0:   4%|▎         | 13/363 [00:00<00:06, 51.27it/s]
Loading 0:   5%|▌         | 19/363 [00:00<00:07, 47.63it/s]
Loading 0:   7%|▋         | 24/363 [00:00<00:07, 45.41it/s]
Loading 0:   9%|▊         | 31/363 [00:00<00:06, 50.97it/s]
Loading 0:  10%|█         | 37/363 [00:00<00:06, 48.07it/s]
Loading 0:  12%|█▏        | 42/363 [00:00<00:06, 46.23it/s]
Loading 0:  13%|█▎        | 49/363 [00:01<00:06, 51.31it/s]
Loading 0:  15%|█▌        | 55/363 [00:01<00:06, 49.15it/s]
Loading 0:  17%|█▋        | 61/363 [00:01<00:08, 36.45it/s]
Loading 0:  18%|█▊        | 66/363 [00:01<00:08, 36.56it/s]
Loading 0:  20%|█▉        | 72/363 [00:01<00:07, 40.01it/s]
Loading 0:  21%|██        | 77/363 [00:01<00:06, 41.42it/s]
Loading 0:  23%|██▎       | 82/363 [00:01<00:07, 35.70it/s]
Loading 0:  25%|██▍       | 90/363 [00:02<00:06, 43.95it/s]
Loading 0:  26%|██▋       | 96/363 [00:02<00:06, 43.57it/s]
Loading 0:  28%|██▊       | 101/363 [00:02<00:06, 43.17it/s]
Loading 0:  30%|██▉       | 108/363 [00:02<00:05, 49.63it/s]
Loading 0:  31%|███▏      | 114/363 [00:02<00:05, 46.23it/s]
Loading 0:  33%|███▎      | 119/363 [00:02<00:05, 45.42it/s]
Loading 0:  35%|███▍      | 126/363 [00:02<00:04, 49.01it/s]
Loading 0:  36%|███▋      | 132/363 [00:02<00:04, 46.41it/s]
Loading 0:  38%|███▊      | 137/363 [00:03<00:05, 44.79it/s]
Loading 0:  39%|███▉      | 142/363 [00:03<00:06, 33.87it/s]
Loading 0:  40%|████      | 146/363 [00:03<00:06, 33.59it/s]
Loading 0:  41%|████▏     | 150/363 [00:03<00:06, 33.35it/s]
Loading 0:  43%|████▎     | 157/363 [00:03<00:05, 39.88it/s]
Loading 0:  45%|████▍     | 163/363 [00:03<00:05, 39.10it/s]
Loading 0:  46%|████▋     | 168/363 [00:03<00:04, 39.77it/s]
Loading 0:  48%|████▊     | 175/363 [00:04<00:04, 44.98it/s]
Loading 0:  50%|████▉     | 181/363 [00:04<00:04, 44.12it/s]
Loading 0:  51%|█████     | 186/363 [00:04<00:04, 43.71it/s]
Loading 0:  53%|█████▎    | 193/363 [00:04<00:03, 48.41it/s]
Loading 0:  55%|█████▍    | 199/363 [00:04<00:03, 47.23it/s]
Loading 0:  56%|█████▌    | 204/363 [00:04<00:03, 46.24it/s]
Loading 0:  58%|█████▊    | 210/363 [00:04<00:03, 49.32it/s]
Loading 0:  60%|█████▉    | 216/363 [00:04<00:02, 50.78it/s]
Loading 0:  61%|██████    | 222/363 [00:05<00:02, 48.24it/s]
Loading 0:  63%|██████▎   | 227/363 [00:05<00:03, 34.96it/s]
Loading 0:  64%|██████▍   | 232/363 [00:05<00:03, 36.15it/s]
Loading 0:  65%|██████▌   | 237/363 [00:05<00:03, 39.12it/s]
Loading 0:  67%|██████▋   | 242/363 [00:05<00:02, 41.09it/s]
Loading 0:  68%|██████▊   | 247/363 [00:05<00:02, 41.90it/s]
Loading 0:  70%|██████▉   | 253/363 [00:05<00:02, 42.47it/s]
Loading 0:  71%|███████   | 258/363 [00:06<00:02, 41.71it/s]
Loading 0:  73%|███████▎  | 265/363 [00:06<00:02, 47.09it/s]
Loading 0:  75%|███████▍  | 271/363 [00:06<00:01, 46.37it/s]
Loading 0:  76%|███████▌  | 276/363 [00:06<00:01, 45.57it/s]
Loading 0:  78%|███████▊  | 282/363 [00:06<00:01, 49.29it/s]
Loading 0:  79%|███████▉  | 288/363 [00:06<00:01, 50.79it/s]
Loading 0:  81%|████████  | 294/363 [00:06<00:01, 45.09it/s]
Loading 0:  83%|████████▎ | 301/363 [00:06<00:01, 51.16it/s]
Loading 0:  85%|████████▍ | 307/363 [00:13<00:19,  2.84it/s]
Loading 0:  86%|████████▌ | 312/363 [00:13<00:13,  3.73it/s]
Loading 0:  88%|████████▊ | 320/363 [00:14<00:07,  5.76it/s]
Loading 0:  90%|████████▉ | 326/363 [00:14<00:04,  7.63it/s]
Loading 0:  91%|█████████ | 331/363 [00:14<00:03,  9.61it/s]
Loading 0:  93%|█████████▎| 338/363 [00:14<00:01, 13.44it/s]
Loading 0:  95%|█████████▍| 344/363 [00:14<00:01, 16.76it/s]
Loading 0:  96%|█████████▌| 349/363 [00:14<00:00, 19.57it/s]
Loading 0:  98%|█████████▊| 356/363 [00:14<00:00, 25.28it/s]
Loading 0: 100%|█████████▉| 362/363 [00:14<00:00, 28.38it/s]
                                                            
Job mistralai-mistral-nem-93303-v262-mkmlizer completed after 115.54s with status: succeeded
Stopping job with name mistralai-mistral-nem-93303-v262-mkmlizer
Pipeline stage MKMLizer completed in 116.04s
run pipeline stage %s
Running pipeline stage MKMLTemplater
Pipeline stage MKMLTemplater completed in 0.22s
run pipeline stage %s
Running pipeline stage MKMLDeployer
Creating inference service mistralai-mistral-nem-93303-v262
Waiting for inference service mistralai-mistral-nem-93303-v262 to be ready
Failed to get response for submission mistralai-mistral-nem_93303_v260: HTTPConnectionPool(host='mistralai-mistral-nem-93303-v260-predictor.tenant-chaiml-guanaco.k.chaiverse.com', port=80): Read timed out. (read timeout=12.0)
Failed to get response for submission mistralai-mistral-nem_93303_v260: HTTPConnectionPool(host='mistralai-mistral-nem-93303-v260-predictor.tenant-chaiml-guanaco.k.chaiverse.com', port=80): Read timed out. (read timeout=12.0)
Failed to get response for submission mistralai-mistral-nem_93303_v261: HTTPConnectionPool(host='mistralai-mistral-nem-93303-v261-predictor.tenant-chaiml-guanaco.k.chaiverse.com', port=80): Read timed out. (read timeout=12.0)
Failed to get response for submission mistralai-mistral-nem_93303_v261: HTTPConnectionPool(host='mistralai-mistral-nem-93303-v261-predictor.tenant-chaiml-guanaco.k.chaiverse.com', port=80): Read timed out. (read timeout=12.0)
Failed to get response for submission mistralai-mistral-nem_93303_v261: HTTPConnectionPool(host='mistralai-mistral-nem-93303-v261-predictor.tenant-chaiml-guanaco.k.chaiverse.com', port=80): Read timed out. (read timeout=12.0)
Failed to get response for submission mistralai-mistral-sma_53415_v101: HTTPConnectionPool(host='mistralai-mistral-sma-53415-v101-predictor.tenant-chaiml-guanaco.k.chaiverse.com', port=80): Read timed out. (read timeout=12.0)
Failed to get response for submission mistralai-mistral-nem_93303_v261: HTTPConnectionPool(host='mistralai-mistral-nem-93303-v261-predictor.tenant-chaiml-guanaco.k.chaiverse.com', port=80): Read timed out. (read timeout=12.0)
Failed to get response for submission mistralai-mistral-sma_53415_v101: HTTPConnectionPool(host='mistralai-mistral-sma-53415-v101-predictor.tenant-chaiml-guanaco.k.chaiverse.com', port=80): Read timed out. (read timeout=12.0)
Connection pool is full, discarding connection: %s. Connection pool size: %s
Inference service mistralai-mistral-nem-93303-v262 ready after 373.07527136802673s
Pipeline stage MKMLDeployer completed in 373.63s
run pipeline stage %s
Running pipeline stage StressChecker
Received healthy response to inference request in 12.396153926849365s
Received healthy response to inference request in 7.077518939971924s
Received healthy response to inference request in 2.1485888957977295s
Received healthy response to inference request in 7.365248680114746s
Received healthy response to inference request in 6.7137322425842285s
5 requests
0 failed requests
5th percentile: 3.0616175651550295
10th percentile: 3.9746462345123295
20th percentile: 5.8007035732269285
30th percentile: 6.786489582061767
40th percentile: 6.9320042610168455
50th percentile: 7.077518939971924
60th percentile: 7.192610836029052
70th percentile: 7.307702732086182
80th percentile: 8.37142972946167
90th percentile: 10.383791828155518
95th percentile: 11.38997287750244
99th percentile: 12.19491771697998
mean time: 7.140248537063599
%s, retrying in %s seconds...
Received healthy response to inference request in 2.478317975997925s
Received healthy response to inference request in 6.69285249710083s
Received healthy response to inference request in 2.4129831790924072s
Received healthy response to inference request in 6.961701393127441s
Received healthy response to inference request in 1.9206182956695557s
5 requests
0 failed requests
5th percentile: 2.019091272354126
10th percentile: 2.117564249038696
20th percentile: 2.314510202407837
30th percentile: 2.426050138473511
40th percentile: 2.452184057235718
50th percentile: 2.478317975997925
60th percentile: 4.164131784439086
70th percentile: 5.849945592880248
80th percentile: 6.7466222763061525
90th percentile: 6.8541618347167965
95th percentile: 6.907931613922119
99th percentile: 6.950947437286377
mean time: 4.093294668197632
%s, retrying in %s seconds...
Received healthy response to inference request in 1.6821792125701904s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Received healthy response to inference request in 6.906485557556152s
Received healthy response to inference request in 1.9312877655029297s
Received healthy response to inference request in 6.837583303451538s
Received healthy response to inference request in 2.059429883956909s
5 requests
0 failed requests
5th percentile: 1.7320009231567384
10th percentile: 1.781822633743286
20th percentile: 1.8814660549163817
30th percentile: 1.9569161891937257
40th percentile: 2.008173036575317
50th percentile: 2.059429883956909
60th percentile: 3.9706912517547606
70th percentile: 5.881952619552612
80th percentile: 6.851363754272461
90th percentile: 6.8789246559143065
95th percentile: 6.89270510673523
99th percentile: 6.903729467391968
mean time: 3.883393144607544
clean up pipeline due to error=DeploymentChecksError('Unacceptable 70th percentile latency 5.881952619552612s')
Shutdown handler de-registered
mistralai-mistral-nem_93303_v262 status is now failed due to DeploymentManager action