mistralai-mistral-sma_53415

submission_id: mistralai-mistral-sma_53415_v102
developer_uid: azuruce
status: failed
model_repo: mistralai/Mistral-Small-Instruct-2409
generation_params: {'temperature': 1.0, 'top_p': 0.99, 'min_p': 0.0, 'top_k': 80, 'presence_penalty': 0.0, 'frequency_penalty': 0.0, 'stopping_words': ['\n,', '</s>,', '####,', 'Bot:,', 'User:,', 'You:,', '<|im_end|>,', '<|eot_id|>,', 'Me:'], 'max_input_tokens': 1024, 'best_of': 8, 'max_output_tokens': 64}
formatter: {'memory_template': '<|im_start|>system\n{memory}<|im_end|>\n', 'prompt_template': '<|im_start|>user\n{prompt}<|im_end|>\n', 'bot_template': '{bot_name}: {message}\n', 'user_template': '{user_name}: {message}\n', 'response_template': '{bot_name}:', 'truncate_by_message': False}
timestamp: 2025-01-10T18:57:41+00:00
model_name: mistralai-mistral-sma_53415_v102
Resubmit model
Shutdown handler not registered because Python interpreter is not running in the main thread
run pipeline %s
run pipeline stage %s
Running pipeline stage MKMLizer
Starting job with name mistralai-mistral-sma-53415-v102-mkmlizer
Waiting for job on mistralai-mistral-sma-53415-v102-mkmlizer to finish
mistralai-mistral-sma-53415-v102-mkmlizer: ╔═════════════════════════════════════════════════════════════════════╗
mistralai-mistral-sma-53415-v102-mkmlizer: ║     _____            __           __                                ║
mistralai-mistral-sma-53415-v102-mkmlizer: ║    / _/ /_ ___    __/ /  ___ ___ / /                                ║
mistralai-mistral-sma-53415-v102-mkmlizer: ║   / _/ / // / |/|/ / _ \/ -_) -_) /                                 ║
mistralai-mistral-sma-53415-v102-mkmlizer: ║  /_//_/\_, /|__,__/_//_/\__/\__/_/                                  ║
mistralai-mistral-sma-53415-v102-mkmlizer: ║       /___/                                                         ║
mistralai-mistral-sma-53415-v102-mkmlizer: ║                                                                     ║
mistralai-mistral-sma-53415-v102-mkmlizer: ║  Version: 0.11.12                                                   ║
mistralai-mistral-sma-53415-v102-mkmlizer: ║  Copyright 2023 MK ONE TECHNOLOGIES Inc.                            ║
mistralai-mistral-sma-53415-v102-mkmlizer: ║  https://mk1.ai                                                     ║
mistralai-mistral-sma-53415-v102-mkmlizer: ║                                                                     ║
mistralai-mistral-sma-53415-v102-mkmlizer: ║  The license key for the current software has been verified as      ║
mistralai-mistral-sma-53415-v102-mkmlizer: ║  belonging to:                                                      ║
mistralai-mistral-sma-53415-v102-mkmlizer: ║                                                                     ║
mistralai-mistral-sma-53415-v102-mkmlizer: ║  Chai Research Corp.                                                ║
mistralai-mistral-sma-53415-v102-mkmlizer: ║  Account ID: 7997a29f-0ceb-4cc7-9adf-840c57b4ae6f                   ║
mistralai-mistral-sma-53415-v102-mkmlizer: ║  Expiration: 2025-04-15 23:59:59                                    ║
mistralai-mistral-sma-53415-v102-mkmlizer: ║                                                                     ║
mistralai-mistral-sma-53415-v102-mkmlizer: ╚═════════════════════════════════════════════════════════════════════╝
Failed to get response for submission function_lijit_2024-11-26: ('http://zonemercy-lexical-viral-7348-v17-predictor.tenant-chaiml-guanaco.k2.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'read tcp 127.0.0.1:38768->127.0.0.1:8080: read: connection reset by peer\n')
mistralai-mistral-sma-53415-v102-mkmlizer: Downloaded to shared memory in 89.356s
mistralai-mistral-sma-53415-v102-mkmlizer: quantizing model to /dev/shm/model_cache, profile:s0, folder:/tmp/tmp6xnrd7p_, device:0
mistralai-mistral-sma-53415-v102-mkmlizer: Saving flywheel model at /dev/shm/model_cache
mistralai-mistral-sma-53415-v102-mkmlizer: quantized model in 44.036s
mistralai-mistral-sma-53415-v102-mkmlizer: Processed model mistralai/Mistral-Small-Instruct-2409 in 133.393s
mistralai-mistral-sma-53415-v102-mkmlizer: creating bucket guanaco-mkml-models
mistralai-mistral-sma-53415-v102-mkmlizer: Bucket 's3://guanaco-mkml-models/' created
mistralai-mistral-sma-53415-v102-mkmlizer: uploading /dev/shm/model_cache to s3://guanaco-mkml-models/mistralai-mistral-sma-53415-v102
mistralai-mistral-sma-53415-v102-mkmlizer: cp /dev/shm/model_cache/config.json s3://guanaco-mkml-models/mistralai-mistral-sma-53415-v102/config.json
mistralai-mistral-sma-53415-v102-mkmlizer: cp /dev/shm/model_cache/special_tokens_map.json s3://guanaco-mkml-models/mistralai-mistral-sma-53415-v102/special_tokens_map.json
mistralai-mistral-sma-53415-v102-mkmlizer: cp /dev/shm/model_cache/flywheel_model.1.safetensors s3://guanaco-mkml-models/mistralai-mistral-sma-53415-v102/flywheel_model.1.safetensors
mistralai-mistral-sma-53415-v102-mkmlizer: cp /dev/shm/model_cache/flywheel_model.0.safetensors s3://guanaco-mkml-models/mistralai-mistral-sma-53415-v102/flywheel_model.0.safetensors
mistralai-mistral-sma-53415-v102-mkmlizer: 
Loading 0:   0%|          | 0/507 [00:00<?, ?it/s]
Loading 0:   1%|          | 5/507 [00:00<00:19, 25.62it/s]
Loading 0:   2%|▏         | 12/507 [00:00<00:12, 41.25it/s]
Loading 0:   3%|▎         | 17/507 [00:00<00:12, 39.22it/s]
Loading 0:   4%|▍         | 22/507 [00:00<00:13, 36.36it/s]
Loading 0:   5%|▌         | 26/507 [00:00<00:13, 36.41it/s]
Loading 0:   6%|▌         | 30/507 [00:00<00:13, 36.66it/s]
Loading 0:   7%|▋         | 34/507 [00:00<00:13, 35.44it/s]
Loading 0:   8%|▊         | 39/507 [00:01<00:12, 37.11it/s]
Loading 0:   8%|▊         | 43/507 [00:01<00:12, 35.94it/s]
Loading 0:   9%|▉         | 47/507 [00:01<00:12, 35.39it/s]
Loading 0:  10%|█         | 51/507 [00:01<00:13, 35.05it/s]
Loading 0:  11%|█         | 55/507 [00:01<00:17, 25.44it/s]
Loading 0:  12%|█▏        | 59/507 [00:01<00:16, 27.70it/s]
Loading 0:  12%|█▏        | 63/507 [00:01<00:14, 29.79it/s]
Loading 0:  13%|█▎        | 67/507 [00:02<00:14, 30.35it/s]
Loading 0:  14%|█▍        | 72/507 [00:02<00:12, 34.92it/s]
Loading 0:  15%|█▌        | 78/507 [00:02<00:10, 39.89it/s]
Loading 0:  16%|█▋        | 83/507 [00:02<00:11, 38.55it/s]
Loading 0:  17%|█▋        | 88/507 [00:02<00:11, 37.73it/s]
Loading 0:  18%|█▊        | 92/507 [00:02<00:11, 37.68it/s]
Loading 0:  19%|█▉        | 96/507 [00:02<00:10, 38.11it/s]
Loading 0:  20%|█▉        | 100/507 [00:02<00:11, 36.33it/s]
Loading 0:  21%|██        | 105/507 [00:02<00:10, 37.28it/s]
Loading 0:  21%|██▏       | 109/507 [00:03<00:11, 35.83it/s]
Loading 0:  22%|██▏       | 113/507 [00:03<00:14, 27.70it/s]
Loading 0:  23%|██▎       | 117/507 [00:03<00:14, 27.09it/s]
Loading 0:  24%|██▍       | 122/507 [00:03<00:14, 27.33it/s]
Loading 0:  25%|██▌       | 129/507 [00:03<00:10, 34.46it/s]
Loading 0:  26%|██▌       | 133/507 [00:03<00:10, 34.15it/s]
Loading 0:  27%|██▋       | 138/507 [00:04<00:10, 35.72it/s]
Loading 0:  28%|██▊       | 142/507 [00:04<00:10, 35.19it/s]
Loading 0:  29%|██▉       | 147/507 [00:04<00:09, 37.68it/s]
Loading 0:  30%|██▉       | 151/507 [00:04<00:09, 35.69it/s]
Loading 0:  31%|███       | 156/507 [00:04<00:09, 36.37it/s]
Loading 0:  32%|███▏      | 160/507 [00:04<00:10, 33.92it/s]
Loading 0:  32%|███▏      | 164/507 [00:04<00:10, 33.26it/s]
Loading 0:  33%|███▎      | 169/507 [00:05<00:12, 27.74it/s]
Loading 0:  34%|███▍      | 172/507 [00:05<00:12, 27.01it/s]
Loading 0:  35%|███▍      | 176/507 [00:05<00:12, 26.68it/s]
Loading 0:  36%|███▌      | 183/507 [00:05<00:09, 33.68it/s]
Loading 0:  37%|███▋      | 187/507 [00:05<00:09, 33.04it/s]
Loading 0:  38%|███▊      | 192/507 [00:05<00:08, 36.05it/s]
Loading 0:  39%|███▊      | 196/507 [00:05<00:08, 35.12it/s]
Loading 0:  40%|███▉      | 201/507 [00:05<00:08, 37.35it/s]
Loading 0:  40%|████      | 205/507 [00:06<00:08, 35.49it/s]
Loading 0:  41%|████▏     | 210/507 [00:06<00:07, 37.40it/s]
Loading 0:  42%|████▏     | 214/507 [00:06<00:08, 34.87it/s]
Loading 0:  43%|████▎     | 218/507 [00:06<00:08, 33.57it/s]
Loading 0:  44%|████▍     | 222/507 [00:06<00:08, 34.01it/s]
Loading 0:  45%|████▍     | 226/507 [00:06<00:10, 26.56it/s]
Loading 0:  45%|████▌     | 230/507 [00:06<00:10, 26.28it/s]
Loading 0:  47%|████▋     | 237/507 [00:07<00:07, 33.77it/s]
Loading 0:  48%|████▊     | 241/507 [00:07<00:07, 33.78it/s]
Loading 0:  49%|████▊     | 246/507 [00:07<00:07, 36.54it/s]
Loading 0:  49%|████▉     | 250/507 [00:07<00:07, 34.27it/s]
Loading 0:  50%|█████     | 255/507 [00:07<00:06, 36.56it/s]
Loading 0:  51%|█████     | 259/507 [00:07<00:06, 35.68it/s]
Loading 0:  52%|█████▏    | 264/507 [00:07<00:06, 37.03it/s]
Loading 0:  53%|█████▎    | 268/507 [00:07<00:06, 35.24it/s]
Loading 0:  54%|█████▍    | 273/507 [00:08<00:06, 37.14it/s]
Loading 0:  55%|█████▍    | 277/507 [00:08<00:06, 36.00it/s]
Loading 0:  56%|█████▌    | 282/507 [00:08<00:05, 39.58it/s]
Loading 0:  57%|█████▋    | 287/507 [00:08<00:08, 24.97it/s]
Loading 0:  58%|█████▊    | 293/507 [00:08<00:07, 28.15it/s]
Loading 0:  59%|█████▉    | 299/507 [00:23<00:07, 28.15it/s]
Loading 0:  59%|█████▉    | 300/507 [00:23<02:44,  1.26it/s]
Loading 0:  60%|█████▉    | 302/507 [00:23<02:24,  1.42it/s]
Loading 0:  61%|██████    | 307/507 [00:23<01:38,  2.04it/s]
Loading 0:  61%|██████▏   | 311/507 [00:23<01:12,  2.69it/s]
Loading 0:  62%|██████▏   | 316/507 [00:24<00:49,  3.86it/s]
Loading 0:  63%|██████▎   | 320/507 [00:24<00:37,  5.00it/s]
Loading 0:  64%|██████▍   | 327/507 [00:24<00:22,  7.88it/s]
Loading 0:  65%|██████▌   | 331/507 [00:24<00:18,  9.70it/s]
Loading 0:  66%|██████▌   | 335/507 [00:24<00:14, 11.92it/s]
Loading 0:  67%|██████▋   | 340/507 [00:24<00:11, 13.94it/s]
Loading 0:  68%|██████▊   | 344/507 [00:24<00:09, 16.33it/s]
Loading 0:  69%|██████▊   | 348/507 [00:25<00:08, 17.79it/s]
Loading 0:  69%|██████▉   | 352/507 [00:25<00:07, 20.95it/s]
Loading 0:  70%|███████   | 356/507 [00:25<00:06, 21.80it/s]
Loading 0:  71%|███████   | 361/507 [00:25<00:05, 26.75it/s]
Loading 0:  72%|███████▏  | 365/507 [00:25<00:05, 26.51it/s]
Loading 0:  73%|███████▎  | 372/507 [00:25<00:03, 33.94it/s]
Loading 0:  74%|███████▍  | 376/507 [00:25<00:03, 33.17it/s]
Loading 0:  75%|███████▌  | 381/507 [00:26<00:03, 34.74it/s]
Loading 0:  76%|███████▌  | 385/507 [00:26<00:03, 32.59it/s]
Loading 0:  77%|███████▋  | 389/507 [00:26<00:03, 33.19it/s]
Loading 0:  78%|███████▊  | 393/507 [00:26<00:03, 33.81it/s]
Loading 0:  78%|███████▊  | 397/507 [00:26<00:04, 23.91it/s]
Loading 0:  79%|███████▉  | 401/507 [00:26<00:04, 23.42it/s]
Loading 0:  80%|████████  | 406/507 [00:26<00:03, 28.46it/s]
Loading 0:  81%|████████  | 410/507 [00:27<00:03, 27.89it/s]
Loading 0:  82%|████████▏ | 417/507 [00:27<00:02, 35.63it/s]
Loading 0:  83%|████████▎ | 421/507 [00:27<00:02, 34.46it/s]
Loading 0:  84%|████████▍ | 426/507 [00:27<00:02, 35.98it/s]
Loading 0:  85%|████████▍ | 430/507 [00:27<00:02, 35.83it/s]
Loading 0:  86%|████████▌ | 435/507 [00:27<00:01, 37.97it/s]
Loading 0:  87%|████████▋ | 439/507 [00:27<00:01, 35.60it/s]
Loading 0:  88%|████████▊ | 444/507 [00:27<00:01, 36.90it/s]
Loading 0:  88%|████████▊ | 448/507 [00:28<00:01, 35.76it/s]
Loading 0:  90%|████████▉ | 454/507 [00:28<00:01, 37.51it/s]
Loading 0:  90%|█████████ | 458/507 [00:30<00:08,  6.07it/s]
Loading 0:  91%|█████████ | 461/507 [00:30<00:06,  7.28it/s]
Loading 0:  92%|█████████▏| 465/507 [00:30<00:04,  9.30it/s]
Loading 0:  93%|█████████▎| 472/507 [00:30<00:02, 14.41it/s]
Loading 0:  94%|█████████▍| 476/507 [00:31<00:01, 16.74it/s]
Loading 0:  95%|█████████▍| 481/507 [00:31<00:01, 20.93it/s]
Loading 0:  96%|█████████▌| 485/507 [00:31<00:00, 23.09it/s]
Loading 0:  97%|█████████▋| 490/507 [00:31<00:00, 27.46it/s]
Loading 0:  97%|█████████▋| 494/507 [00:31<00:00, 28.52it/s]
Loading 0:  98%|█████████▊| 499/507 [00:31<00:00, 31.50it/s]
Loading 0:  99%|█████████▉| 503/507 [00:31<00:00, 32.13it/s]
                                                            
Job mistralai-mistral-sma-53415-v102-mkmlizer completed after 166.06s with status: succeeded
Stopping job with name mistralai-mistral-sma-53415-v102-mkmlizer
Pipeline stage MKMLizer completed in 166.93s
run pipeline stage %s
Running pipeline stage MKMLTemplater
Pipeline stage MKMLTemplater completed in 0.15s
run pipeline stage %s
Running pipeline stage MKMLDeployer
Creating inference service mistralai-mistral-sma-53415-v102
Waiting for inference service mistralai-mistral-sma-53415-v102 to be ready
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Inference service mistralai-mistral-sma-53415-v102 ready after 361.57422280311584s
Pipeline stage MKMLDeployer completed in 362.08s
run pipeline stage %s
Running pipeline stage StressChecker
Received healthy response to inference request in 14.23509693145752s
Received healthy response to inference request in 3.666872501373291s
Received healthy response to inference request in 8.078462839126587s
Received healthy response to inference request in 8.14612078666687s
Received healthy response to inference request in 2.768407106399536s
5 requests
0 failed requests
5th percentile: 2.948100185394287
10th percentile: 3.1277932643890383
20th percentile: 3.48717942237854
30th percentile: 4.54919056892395
40th percentile: 6.313826704025269
50th percentile: 8.078462839126587
60th percentile: 8.105526018142701
70th percentile: 8.132589197158813
80th percentile: 9.363916015625001
90th percentile: 11.79950647354126
95th percentile: 13.017301702499388
99th percentile: 13.991537885665894
mean time: 7.378992033004761
%s, retrying in %s seconds...
Received healthy response to inference request in 7.802833318710327s
Received healthy response to inference request in 8.088423728942871s
Received healthy response to inference request in 2.8298895359039307s
Received healthy response to inference request in 14.908605098724365s
Received healthy response to inference request in 9.414496898651123s
5 requests
0 failed requests
5th percentile: 3.82447829246521
10th percentile: 4.819067049026489
20th percentile: 6.808244562149048
30th percentile: 7.859951400756836
40th percentile: 7.974187564849854
50th percentile: 8.088423728942871
60th percentile: 8.618852996826172
70th percentile: 9.149282264709472
80th percentile: 10.513318538665771
90th percentile: 12.710961818695068
95th percentile: 13.809783458709717
99th percentile: 14.688840770721436
mean time: 8.608849716186523
%s, retrying in %s seconds...
Received healthy response to inference request in 3.2220938205718994s
Received healthy response to inference request in 8.364717245101929s
Received healthy response to inference request in 8.787675857543945s
Received healthy response to inference request in 3.103149890899658s
Received healthy response to inference request in 2.694340944290161s
5 requests
0 failed requests
5th percentile: 2.7761027336120607
10th percentile: 2.85786452293396
20th percentile: 3.0213881015777586
30th percentile: 3.1269386768341065
40th percentile: 3.1745162487030028
50th percentile: 3.2220938205718994
60th percentile: 5.279143190383911
70th percentile: 7.336192560195922
80th percentile: 8.449308967590332
90th percentile: 8.61849241256714
95th percentile: 8.703084135055542
99th percentile: 8.770757513046265
mean time: 5.234395551681518
clean up pipeline due to error=DeploymentChecksError('Unacceptable 70th percentile latency 7.336192560195922s')
Shutdown handler de-registered
mistralai-mistral-sma_53415_v102 status is now failed due to DeploymentManager action