mistralai-mistral-nem_93303

submission_id: mistralai-mistral-nem_93303_v317
developer_uid: RandomForest1024
status: failed
model_repo: mistralai/Mistral-Nemo-Instruct-2407
generation_params: {'temperature': 1.0, 'top_p': 1.0, 'min_p': 0.0, 'top_k': 40, 'presence_penalty': 0.0, 'frequency_penalty': 0.0, 'stopping_words': ['\n'], 'max_input_tokens': 1024, 'best_of': 8, 'max_output_tokens': 64}
formatter: {'memory_template': "{bot_name}'s Persona: {memory}\n####\n", 'prompt_template': "'<Context>Previous conversation shows your personality as described above.</Context> {prompt}\n<START>\n'", 'bot_template': "'{bot_name}: {message}\n [Maintain teenage perspective while being sophisticated. Use occasional gaming or sports references. Keep political topics neutral.]'", 'user_template': "'{user_name}: {message}\n [Focus on building genuine connections while maintaining appropriate boundaries.]'", 'response_template': "'{bot_name}: [Think as a Gen Z teenager with unique life experiences, then respond]', 'truncate_by_message': False}", 'truncate_by_message': False}
timestamp: 2025-02-10T18:37:06+00:00
model_name: mistralai-mistral-nem_93303_v317
Resubmit model
Shutdown handler not registered because Python interpreter is not running in the main thread
run pipeline %s
run pipeline stage %s
Running pipeline stage MKMLizer
Starting job with name mistralai-mistral-nem-93303-v317-mkmlizer
Waiting for job on mistralai-mistral-nem-93303-v317-mkmlizer to finish
mistralai-mistral-nem-93303-v317-mkmlizer: ╔═════════════════════════════════════════════════════════════════════╗
mistralai-mistral-nem-93303-v317-mkmlizer: ║     _____            __           __                                ║
mistralai-mistral-nem-93303-v317-mkmlizer: ║    / _/ /_ ___    __/ /  ___ ___ / /                                ║
mistralai-mistral-nem-93303-v317-mkmlizer: ║   / _/ / // / |/|/ / _ \/ -_) -_) /                                 ║
mistralai-mistral-nem-93303-v317-mkmlizer: ║  /_//_/\_, /|__,__/_//_/\__/\__/_/                                  ║
mistralai-mistral-nem-93303-v317-mkmlizer: ║       /___/                                                         ║
mistralai-mistral-nem-93303-v317-mkmlizer: ║                                                                     ║
mistralai-mistral-nem-93303-v317-mkmlizer: ║  Version: 0.11.12                                                   ║
mistralai-mistral-nem-93303-v317-mkmlizer: ║  Copyright 2023 MK ONE TECHNOLOGIES Inc.                            ║
mistralai-mistral-nem-93303-v317-mkmlizer: ║  https://mk1.ai                                                     ║
mistralai-mistral-nem-93303-v317-mkmlizer: ║                                                                     ║
mistralai-mistral-nem-93303-v317-mkmlizer: ║  The license key for the current software has been verified as      ║
mistralai-mistral-nem-93303-v317-mkmlizer: ║  belonging to:                                                      ║
mistralai-mistral-nem-93303-v317-mkmlizer: ║                                                                     ║
mistralai-mistral-nem-93303-v317-mkmlizer: ║  Chai Research Corp.                                                ║
mistralai-mistral-nem-93303-v317-mkmlizer: ║  Account ID: 7997a29f-0ceb-4cc7-9adf-840c57b4ae6f                   ║
mistralai-mistral-nem-93303-v317-mkmlizer: ║  Expiration: 2025-04-15 23:59:59                                    ║
mistralai-mistral-nem-93303-v317-mkmlizer: ║                                                                     ║
mistralai-mistral-nem-93303-v317-mkmlizer: ╚═════════════════════════════════════════════════════════════════════╝
Failed to get response for submission randomforest1024-v25_v6: HTTPConnectionPool(host='randomforest1024-v25-v6-predictor.tenant-chaiml-guanaco.k.chaiverse.com', port=80): Read timed out. (read timeout=12.0)
mistralai-mistral-nem-93303-v317-mkmlizer: Downloaded to shared memory in 56.129s
mistralai-mistral-nem-93303-v317-mkmlizer: quantizing model to /dev/shm/model_cache, profile:s0, folder:/tmp/tmpmvv6mr6j, device:0
mistralai-mistral-nem-93303-v317-mkmlizer: Saving flywheel model at /dev/shm/model_cache
Failed to get response for submission randomforest1024-v25_v6: HTTPConnectionPool(host='randomforest1024-v25-v6-predictor.tenant-chaiml-guanaco.k.chaiverse.com', port=80): Read timed out. (read timeout=12.0)
mistralai-mistral-nem-93303-v317-mkmlizer: quantized model in 41.787s
mistralai-mistral-nem-93303-v317-mkmlizer: Processed model mistralai/Mistral-Nemo-Instruct-2407 in 97.917s
mistralai-mistral-nem-93303-v317-mkmlizer: creating bucket guanaco-mkml-models
mistralai-mistral-nem-93303-v317-mkmlizer: Bucket 's3://guanaco-mkml-models/' created
mistralai-mistral-nem-93303-v317-mkmlizer: uploading /dev/shm/model_cache to s3://guanaco-mkml-models/mistralai-mistral-nem-93303-v317
mistralai-mistral-nem-93303-v317-mkmlizer: cp /dev/shm/model_cache/config.json s3://guanaco-mkml-models/mistralai-mistral-nem-93303-v317/config.json
mistralai-mistral-nem-93303-v317-mkmlizer: cp /dev/shm/model_cache/special_tokens_map.json s3://guanaco-mkml-models/mistralai-mistral-nem-93303-v317/special_tokens_map.json
mistralai-mistral-nem-93303-v317-mkmlizer: cp /dev/shm/model_cache/tokenizer_config.json s3://guanaco-mkml-models/mistralai-mistral-nem-93303-v317/tokenizer_config.json
mistralai-mistral-nem-93303-v317-mkmlizer: cp /dev/shm/model_cache/tokenizer.json s3://guanaco-mkml-models/mistralai-mistral-nem-93303-v317/tokenizer.json
mistralai-mistral-nem-93303-v317-mkmlizer: cp /dev/shm/model_cache/flywheel_model.0.safetensors s3://guanaco-mkml-models/mistralai-mistral-nem-93303-v317/flywheel_model.0.safetensors
mistralai-mistral-nem-93303-v317-mkmlizer: 
Loading 0:   0%|          | 0/363 [00:00<?, ?it/s]
Loading 0:   1%|▏         | 5/363 [00:00<00:15, 22.44it/s]
Loading 0:   3%|▎         | 10/363 [00:00<00:10, 32.66it/s]
Loading 0:   4%|▍         | 14/363 [00:00<00:11, 31.51it/s]
Loading 0:   6%|▌         | 21/363 [00:00<00:07, 42.82it/s]
Loading 0:   7%|▋         | 26/363 [00:00<00:08, 42.03it/s]
Loading 0:   9%|▊         | 31/363 [00:00<00:07, 41.59it/s]
Loading 0:  10%|▉         | 36/363 [00:00<00:07, 42.37it/s]
Loading 0:  11%|█▏        | 41/363 [00:01<00:10, 29.68it/s]
Loading 0:  13%|█▎        | 48/363 [00:01<00:08, 37.87it/s]
Loading 0:  15%|█▍        | 53/363 [00:01<00:07, 38.95it/s]
Loading 0:  16%|█▌        | 58/363 [00:01<00:07, 40.67it/s]
Loading 0:  17%|█▋        | 63/363 [00:01<00:13, 22.72it/s]
Loading 0:  18%|█▊        | 67/363 [00:02<00:11, 25.35it/s]
Loading 0:  20%|█▉        | 71/363 [00:02<00:10, 27.97it/s]
Loading 0:  21%|██        | 75/363 [00:02<00:09, 28.88it/s]
Loading 0:  22%|██▏       | 80/363 [00:02<00:08, 32.65it/s]
Loading 0:  23%|██▎       | 84/363 [00:02<00:08, 32.68it/s]
Loading 0:  25%|██▍       | 89/363 [00:02<00:07, 35.18it/s]
Loading 0:  26%|██▌       | 93/363 [00:02<00:08, 31.66it/s]
Loading 0:  27%|██▋       | 97/363 [00:02<00:08, 31.95it/s]
Loading 0:  28%|██▊       | 101/363 [00:03<00:08, 31.01it/s]
Loading 0:  29%|██▉       | 106/363 [00:03<00:07, 34.34it/s]
Loading 0:  31%|███       | 112/363 [00:03<00:06, 38.88it/s]
Loading 0:  32%|███▏      | 117/363 [00:03<00:06, 37.37it/s]
Loading 0:  33%|███▎      | 121/363 [00:03<00:06, 36.58it/s]
Loading 0:  34%|███▍      | 125/363 [00:03<00:07, 32.35it/s]
Loading 0:  36%|███▌      | 129/363 [00:03<00:07, 30.49it/s]
Loading 0:  37%|███▋      | 134/363 [00:04<00:06, 33.27it/s]
Loading 0:  38%|███▊      | 138/363 [00:04<00:06, 33.04it/s]
Loading 0:  39%|███▉      | 142/363 [00:04<00:11, 19.99it/s]
Loading 0:  40%|███▉      | 145/363 [00:04<00:11, 18.47it/s]
Loading 0:  41%|████      | 149/363 [00:04<00:10, 19.68it/s]
Loading 0:  42%|████▏     | 154/363 [00:05<00:08, 24.77it/s]
Loading 0:  44%|████▎     | 158/363 [00:05<00:08, 24.61it/s]
Loading 0:  45%|████▍     | 163/363 [00:05<00:07, 27.20it/s]
Loading 0:  46%|████▌     | 167/363 [00:05<00:08, 23.63it/s]
Loading 0:  47%|████▋     | 172/363 [00:05<00:06, 28.42it/s]
Loading 0:  48%|████▊     | 176/363 [00:05<00:06, 27.41it/s]
Loading 0:  50%|████▉     | 181/363 [00:05<00:05, 30.81it/s]
Loading 0:  51%|█████     | 185/363 [00:06<00:06, 28.59it/s]
Loading 0:  52%|█████▏    | 190/363 [00:06<00:05, 30.68it/s]
Loading 0:  53%|█████▎    | 194/363 [00:06<00:06, 27.72it/s]
Loading 0:  55%|█████▌    | 201/363 [00:06<00:04, 34.93it/s]
Loading 0:  56%|█████▋    | 205/363 [00:06<00:04, 34.88it/s]
Loading 0:  58%|█████▊    | 210/363 [00:06<00:04, 36.40it/s]
Loading 0:  59%|█████▉    | 214/363 [00:06<00:04, 36.33it/s]
Loading 0:  60%|██████    | 218/363 [00:07<00:04, 31.19it/s]
Loading 0:  61%|██████▏   | 223/363 [00:07<00:05, 26.12it/s]
Loading 0:  62%|██████▏   | 226/363 [00:07<00:05, 25.65it/s]
Loading 0:  63%|██████▎   | 230/363 [00:07<00:05, 25.09it/s]
Loading 0:  65%|██████▍   | 235/363 [00:07<00:04, 29.74it/s]
Loading 0:  66%|██████▌   | 239/363 [00:07<00:04, 29.38it/s]
Loading 0:  68%|██████▊   | 246/363 [00:08<00:03, 35.82it/s]
Loading 0:  69%|██████▉   | 250/363 [00:08<00:03, 34.31it/s]
Loading 0:  70%|██████▉   | 254/363 [00:08<00:03, 32.71it/s]
Loading 0:  71%|███████   | 258/363 [00:08<00:03, 30.42it/s]
Loading 0:  72%|███████▏  | 262/363 [00:08<00:03, 31.91it/s]
Loading 0:  73%|███████▎  | 266/363 [00:08<00:03, 29.66it/s]
Loading 0:  75%|███████▍  | 271/363 [00:08<00:02, 34.30it/s]
Loading 0:  76%|███████▌  | 275/363 [00:08<00:02, 31.51it/s]
Loading 0:  77%|███████▋  | 280/363 [00:09<00:02, 35.03it/s]
Loading 0:  78%|███████▊  | 284/363 [00:09<00:02, 31.73it/s]
Loading 0:  80%|███████▉  | 289/363 [00:09<00:02, 35.78it/s]
Loading 0:  81%|████████  | 293/363 [00:09<00:02, 32.88it/s]
Loading 0:  82%|████████▏ | 299/363 [00:09<00:01, 37.88it/s]
Loading 0:  84%|████████▎ | 304/363 [00:16<00:26,  2.22it/s]
Loading 0:  85%|████████▍ | 307/363 [00:16<00:20,  2.75it/s]
Loading 0:  86%|████████▌ | 312/363 [00:16<00:12,  3.96it/s]
Loading 0:  88%|████████▊ | 320/363 [00:17<00:06,  6.66it/s]
Loading 0:  90%|████████▉ | 325/363 [00:17<00:04,  8.76it/s]
Loading 0:  91%|█████████ | 330/363 [00:17<00:03, 10.63it/s]
Loading 0:  92%|█████████▏| 335/363 [00:17<00:02, 13.72it/s]
Loading 0:  94%|█████████▎| 340/363 [00:17<00:01, 16.63it/s]
Loading 0:  95%|█████████▌| 346/363 [00:17<00:00, 21.73it/s]
Loading 0:  97%|█████████▋| 351/363 [00:17<00:00, 25.51it/s]
Loading 0:  98%|█████████▊| 356/363 [00:17<00:00, 29.55it/s]
Loading 0: 100%|█████████▉| 362/363 [00:18<00:00, 32.56it/s]
                                                            
Job mistralai-mistral-nem-93303-v317-mkmlizer completed after 150.16s with status: succeeded
Stopping job with name mistralai-mistral-nem-93303-v317-mkmlizer
Pipeline stage MKMLizer completed in 150.73s
run pipeline stage %s
Running pipeline stage MKMLTemplater
Pipeline stage MKMLTemplater completed in 0.18s
run pipeline stage %s
Running pipeline stage MKMLDeployer
Creating inference service mistralai-mistral-nem-93303-v317
Waiting for inference service mistralai-mistral-nem-93303-v317 to be ready
Failed to get response for submission randomforest1024-v25_v6: HTTPConnectionPool(host='randomforest1024-v25-v6-predictor.tenant-chaiml-guanaco.k.chaiverse.com', port=80): Read timed out. (read timeout=12.0)
Inference service mistralai-mistral-nem-93303-v317 ready after 200.7161259651184s
Pipeline stage MKMLDeployer completed in 201.30s
run pipeline stage %s
Running pipeline stage StressChecker
{"detail":"Single '}' encountered in format string"}
Received unhealthy response to inference request!
{"detail":"Single '}' encountered in format string"}
Received unhealthy response to inference request!
{"detail":"Single '}' encountered in format string"}
Received unhealthy response to inference request!
{"detail":"Single '}' encountered in format string"}
Received unhealthy response to inference request!
{"detail":"Single '}' encountered in format string"}
Received unhealthy response to inference request!
5 requests
5 failed requests
5th percentile: 0.1663487434387207
10th percentile: 0.1674569606781006
20th percentile: 0.16967339515686036
30th percentile: 0.17131953239440917
40th percentile: 0.17239537239074706
50th percentile: 0.17347121238708496
60th percentile: 0.18195500373840331
70th percentile: 0.19043879508972167
80th percentile: 0.19694790840148926
90th percentile: 0.20148234367370604
95th percentile: 0.20374956130981445
99th percentile: 0.20556333541870117
mean time: 0.18203816413879395
%s, retrying in %s seconds...
{"detail":"Single '}' encountered in format string"}
Received unhealthy response to inference request!
{"detail":"Single '}' encountered in format string"}
Received unhealthy response to inference request!
{"detail":"Single '}' encountered in format string"}
Received unhealthy response to inference request!
{"detail":"Single '}' encountered in format string"}
Received unhealthy response to inference request!
{"detail":"Single '}' encountered in format string"}
Received unhealthy response to inference request!
5 requests
5 failed requests
5th percentile: 0.17427501678466797
10th percentile: 0.17635173797607423
20th percentile: 0.1805051803588867
30th percentile: 0.1830744743347168
40th percentile: 0.18405961990356445
50th percentile: 0.1850447654724121
60th percentile: 0.1857680320739746
70th percentile: 0.1864912986755371
80th percentile: 0.24451265335083014
90th percentile: 0.35983209609985356
95th percentile: 0.4174918174743652
99th percentile: 0.4636195945739746
mean time: 0.24036588668823242
%s, retrying in %s seconds...
{"detail":"Single '}' encountered in format string"}
Received unhealthy response to inference request!
{"detail":"Single '}' encountered in format string"}
Received unhealthy response to inference request!
read tcp 127.0.0.1:50644->127.0.0.1:8080: read: connection reset by peer
Received unhealthy response to inference request!
{"detail":"Single '}' encountered in format string"}
Received unhealthy response to inference request!
{"detail":"Single '}' encountered in format string"}
Received unhealthy response to inference request!
5 requests
5 failed requests
5th percentile: 0.19444336891174316
10th percentile: 0.19663529396057128
20th percentile: 0.20101914405822754
30th percentile: 0.21049423217773439
40th percentile: 0.2250605583190918
50th percentile: 0.23962688446044922
60th percentile: 0.2655949592590332
70th percentile: 0.2915630340576172
80th percentile: 0.3361382007598877
90th percentile: 0.39932045936584476
95th percentile: 0.43091158866882323
99th percentile: 0.45618449211120604
mean time: 0.2804278373718262
clean up pipeline due to error=DeploymentChecksError('Unacceptable number of predict errors: 100.0%')
Shutdown handler de-registered
mistralai-mistral-nem_93303_v317 status is now failed due to DeploymentManager action