submission_id: mistralai-mistral-nem_93303_v319
developer_uid: RandomForest1024
status: failed
model_repo: mistralai/Mistral-Nemo-Instruct-2407
generation_params: {'temperature': 1.0, 'top_p': 1.0, 'min_p': 0.0, 'top_k': 40, 'presence_penalty': 0.0, 'frequency_penalty': 0.0, 'stopping_words': ['\n'], 'max_input_tokens': 1024, 'best_of': 8, 'max_output_tokens': 64}
formatter: {'memory_template': "{bot_name}'s Persona: {memory}\n####\n", 'prompt_template': "'<Context>Previous conversation shows your personality as described above.</Context> {prompt}\n<START>\n'", 'bot_template': "'{bot_name}: {message}\n [Maintain teenage perspective while being sophisticated. Use occasional gaming or sports references. Keep political topics neutral.]'", 'user_template': "'{user_name}: {message}\n [Focus on building genuine connections while maintaining appropriate boundaries.]'", 'response_template': "'{bot_name}: [Think as a Gen Z teenager with unique life experiences, then respond]', 'truncate_by_message': False}", 'truncate_by_message': False}
timestamp: 2025-02-10T18:38:23+00:00
model_name: mistralai-mistral-nem_93303_v319
Resubmit model
Shutdown handler not registered because Python interpreter is not running in the main thread
run pipeline %s
run pipeline stage %s
Running pipeline stage MKMLizer
Starting job with name mistralai-mistral-nem-93303-v319-mkmlizer
Waiting for job on mistralai-mistral-nem-93303-v319-mkmlizer to finish
Failed to get response for submission randomforest1024-v25_v6: HTTPConnectionPool(host='randomforest1024-v25-v6-predictor.tenant-chaiml-guanaco.k.chaiverse.com', port=80): Read timed out. (read timeout=12.0)
mistralai-mistral-nem-93303-v319-mkmlizer: ╔═════════════════════════════════════════════════════════════════════╗
mistralai-mistral-nem-93303-v319-mkmlizer: ║     _____            __           __                                ║
mistralai-mistral-nem-93303-v319-mkmlizer: ║    / _/ /_ ___    __/ /  ___ ___ / /                                ║
mistralai-mistral-nem-93303-v319-mkmlizer: ║   / _/ / // / |/|/ / _ \/ -_) -_) /                                 ║
mistralai-mistral-nem-93303-v319-mkmlizer: ║  /_//_/\_, /|__,__/_//_/\__/\__/_/                                  ║
mistralai-mistral-nem-93303-v319-mkmlizer: ║       /___/                                                         ║
mistralai-mistral-nem-93303-v319-mkmlizer: ║                                                                     ║
mistralai-mistral-nem-93303-v319-mkmlizer: ║  Version: 0.11.12                                                   ║
mistralai-mistral-nem-93303-v319-mkmlizer: ║  Copyright 2023 MK ONE TECHNOLOGIES Inc.                            ║
mistralai-mistral-nem-93303-v319-mkmlizer: ║  https://mk1.ai                                                     ║
mistralai-mistral-nem-93303-v319-mkmlizer: ║                                                                     ║
mistralai-mistral-nem-93303-v319-mkmlizer: ║  The license key for the current software has been verified as      ║
mistralai-mistral-nem-93303-v319-mkmlizer: ║  belonging to:                                                      ║
mistralai-mistral-nem-93303-v319-mkmlizer: ║                                                                     ║
mistralai-mistral-nem-93303-v319-mkmlizer: ║  Chai Research Corp.                                                ║
mistralai-mistral-nem-93303-v319-mkmlizer: ║  Account ID: 7997a29f-0ceb-4cc7-9adf-840c57b4ae6f                   ║
mistralai-mistral-nem-93303-v319-mkmlizer: ║  Expiration: 2025-04-15 23:59:59                                    ║
mistralai-mistral-nem-93303-v319-mkmlizer: ║                                                                     ║
mistralai-mistral-nem-93303-v319-mkmlizer: ╚═════════════════════════════════════════════════════════════════════╝
Failed to get response for submission randomforest1024-v25_v6: HTTPConnectionPool(host='randomforest1024-v25-v6-predictor.tenant-chaiml-guanaco.k.chaiverse.com', port=80): Read timed out. (read timeout=12.0)
mistralai-mistral-nem-93303-v319-mkmlizer: Downloaded to shared memory in 54.925s
mistralai-mistral-nem-93303-v319-mkmlizer: quantizing model to /dev/shm/model_cache, profile:s0, folder:/tmp/tmpej6noa9r, device:0
mistralai-mistral-nem-93303-v319-mkmlizer: Saving flywheel model at /dev/shm/model_cache
mistralai-mistral-nem-93303-v319-mkmlizer: quantized model in 39.035s
mistralai-mistral-nem-93303-v319-mkmlizer: Processed model mistralai/Mistral-Nemo-Instruct-2407 in 93.960s
mistralai-mistral-nem-93303-v319-mkmlizer: creating bucket guanaco-mkml-models
mistralai-mistral-nem-93303-v319-mkmlizer: Bucket 's3://guanaco-mkml-models/' created
mistralai-mistral-nem-93303-v319-mkmlizer: uploading /dev/shm/model_cache to s3://guanaco-mkml-models/mistralai-mistral-nem-93303-v319
mistralai-mistral-nem-93303-v319-mkmlizer: cp /dev/shm/model_cache/special_tokens_map.json s3://guanaco-mkml-models/mistralai-mistral-nem-93303-v319/special_tokens_map.json
mistralai-mistral-nem-93303-v319-mkmlizer: cp /dev/shm/model_cache/config.json s3://guanaco-mkml-models/mistralai-mistral-nem-93303-v319/config.json
mistralai-mistral-nem-93303-v319-mkmlizer: cp /dev/shm/model_cache/tokenizer_config.json s3://guanaco-mkml-models/mistralai-mistral-nem-93303-v319/tokenizer_config.json
mistralai-mistral-nem-93303-v319-mkmlizer: cp /dev/shm/model_cache/tokenizer.json s3://guanaco-mkml-models/mistralai-mistral-nem-93303-v319/tokenizer.json
mistralai-mistral-nem-93303-v319-mkmlizer: cp /dev/shm/model_cache/flywheel_model.0.safetensors s3://guanaco-mkml-models/mistralai-mistral-nem-93303-v319/flywheel_model.0.safetensors
mistralai-mistral-nem-93303-v319-mkmlizer: 
Loading 0:   0%|          | 0/363 [00:00<?, ?it/s]
Loading 0:   1%|▏         | 5/363 [00:00<00:12, 27.80it/s]
Loading 0:   3%|▎         | 10/363 [00:00<00:09, 37.50it/s]
Loading 0:   4%|▍         | 15/363 [00:00<00:09, 35.39it/s]
Loading 0:   6%|▌         | 21/363 [00:00<00:08, 40.96it/s]
Loading 0:   7%|▋         | 26/363 [00:00<00:08, 40.31it/s]
Loading 0:   9%|▊         | 31/363 [00:00<00:08, 40.08it/s]
Loading 0:  10%|▉         | 36/363 [00:00<00:07, 41.21it/s]
Loading 0:  11%|█▏        | 41/363 [00:01<00:09, 32.82it/s]
Loading 0:  13%|█▎        | 48/363 [00:01<00:07, 39.48it/s]
Loading 0:  15%|█▍        | 53/363 [00:01<00:07, 38.90it/s]
Loading 0:  16%|█▌        | 58/363 [00:01<00:07, 39.67it/s]
Loading 0:  17%|█▋        | 63/363 [00:01<00:12, 24.35it/s]
Loading 0:  18%|█▊        | 67/363 [00:02<00:11, 26.47it/s]
Loading 0:  20%|█▉        | 71/363 [00:02<00:10, 28.43it/s]
Loading 0:  21%|██        | 75/363 [00:02<00:09, 29.02it/s]
Loading 0:  22%|██▏       | 80/363 [00:02<00:08, 32.11it/s]
Loading 0:  23%|██▎       | 84/363 [00:02<00:08, 31.53it/s]
Loading 0:  25%|██▍       | 89/363 [00:02<00:08, 34.05it/s]
Loading 0:  26%|██▌       | 93/363 [00:02<00:08, 32.97it/s]
Loading 0:  27%|██▋       | 98/363 [00:02<00:07, 34.76it/s]
Loading 0:  28%|██▊       | 102/363 [00:03<00:07, 33.67it/s]
Loading 0:  29%|██▉       | 106/363 [00:03<00:07, 33.84it/s]
Loading 0:  31%|███       | 112/363 [00:03<00:06, 37.42it/s]
Loading 0:  32%|███▏      | 116/363 [00:03<00:07, 33.29it/s]
Loading 0:  33%|███▎      | 120/363 [00:03<00:07, 32.67it/s]
Loading 0:  34%|███▍      | 125/363 [00:03<00:06, 35.28it/s]
Loading 0:  36%|███▌      | 129/363 [00:03<00:06, 34.27it/s]
Loading 0:  37%|███▋      | 133/363 [00:03<00:06, 35.62it/s]
Loading 0:  38%|███▊      | 137/363 [00:04<00:07, 32.24it/s]
Loading 0:  39%|███▉      | 141/363 [00:04<00:06, 34.02it/s]
Loading 0:  40%|███▉      | 145/363 [00:04<00:10, 20.74it/s]
Loading 0:  41%|████      | 149/363 [00:04<00:09, 23.05it/s]
Loading 0:  43%|████▎     | 156/363 [00:04<00:06, 31.73it/s]
Loading 0:  44%|████▍     | 160/363 [00:04<00:06, 33.03it/s]
Loading 0:  45%|████▌     | 165/363 [00:04<00:05, 36.59it/s]
Loading 0:  47%|████▋     | 170/363 [00:05<00:04, 39.09it/s]
Loading 0:  48%|████▊     | 175/363 [00:05<00:04, 40.67it/s]
Loading 0:  50%|████▉     | 181/363 [00:05<00:04, 39.10it/s]
Loading 0:  51%|█████     | 186/363 [00:05<00:04, 38.43it/s]
Loading 0:  53%|█████▎    | 192/363 [00:05<00:03, 42.91it/s]
Loading 0:  54%|█████▍    | 197/363 [00:05<00:03, 41.74it/s]
Loading 0:  56%|█████▌    | 202/363 [00:05<00:03, 42.50it/s]
Loading 0:  57%|█████▋    | 207/363 [00:05<00:03, 44.28it/s]
Loading 0:  58%|█████▊    | 212/363 [00:06<00:04, 36.65it/s]
Loading 0:  60%|██████    | 218/363 [00:06<00:03, 41.88it/s]
Loading 0:  61%|██████▏   | 223/363 [00:06<00:04, 28.02it/s]
Loading 0:  63%|██████▎   | 227/363 [00:06<00:04, 29.49it/s]
Loading 0:  64%|██████▎   | 231/363 [00:06<00:04, 28.58it/s]
Loading 0:  65%|██████▍   | 235/363 [00:06<00:04, 30.42it/s]
Loading 0:  66%|██████▌   | 239/363 [00:07<00:04, 27.93it/s]
Loading 0:  68%|██████▊   | 246/363 [00:07<00:03, 35.43it/s]
Loading 0:  69%|██████▉   | 250/363 [00:07<00:03, 35.19it/s]
Loading 0:  70%|███████   | 255/363 [00:07<00:02, 36.93it/s]
Loading 0:  71%|███████▏  | 259/363 [00:07<00:02, 35.62it/s]
Loading 0:  73%|███████▎  | 264/363 [00:07<00:02, 37.39it/s]
Loading 0:  74%|███████▍  | 268/363 [00:07<00:02, 35.15it/s]
Loading 0:  75%|███████▍  | 272/363 [00:07<00:02, 35.87it/s]
Loading 0:  76%|███████▌  | 276/363 [00:08<00:02, 31.83it/s]
Loading 0:  77%|███████▋  | 280/363 [00:08<00:02, 33.76it/s]
Loading 0:  78%|███████▊  | 284/363 [00:08<00:02, 31.45it/s]
Loading 0:  80%|███████▉  | 289/363 [00:08<00:02, 35.77it/s]
Loading 0:  81%|████████  | 293/363 [00:08<00:02, 32.62it/s]
Loading 0:  82%|████████▏ | 298/363 [00:08<00:01, 36.07it/s]
Loading 0:  83%|████████▎ | 303/363 [00:08<00:01, 36.46it/s]
Loading 0:  85%|████████▍ | 307/363 [00:15<00:27,  2.05it/s]
Loading 0:  86%|████████▌ | 312/363 [00:15<00:17,  2.95it/s]
Loading 0:  88%|████████▊ | 319/363 [00:16<00:09,  4.73it/s]
Loading 0:  89%|████████▉ | 323/363 [00:16<00:06,  5.99it/s]
Loading 0:  90%|█████████ | 328/363 [00:16<00:04,  8.12it/s]
Loading 0:  92%|█████████▏| 333/363 [00:16<00:02, 10.66it/s]
Loading 0:  93%|█████████▎| 338/363 [00:16<00:01, 13.74it/s]
Loading 0:  94%|█████████▍| 343/363 [00:16<00:01, 17.51it/s]
Loading 0:  96%|█████████▌| 348/363 [00:16<00:00, 19.43it/s]
Loading 0:  98%|█████████▊| 356/363 [00:16<00:00, 27.65it/s]
Loading 0: 100%|█████████▉| 362/363 [00:17<00:00, 30.50it/s]
                                                            

Job mistralai-mistral-nem-93303-v319-mkmlizer completed after 124.49s with status: succeeded
Stopping job with name mistralai-mistral-nem-93303-v319-mkmlizer
Pipeline stage MKMLizer completed in 124.94s
run pipeline stage %s
Running pipeline stage MKMLTemplater
Pipeline stage MKMLTemplater completed in 0.61s
run pipeline stage %s
Running pipeline stage MKMLDeployer
Creating inference service mistralai-mistral-nem-93303-v319
Waiting for inference service mistralai-mistral-nem-93303-v319 to be ready
Inference service mistralai-mistral-nem-93303-v319 ready after 200.82187390327454s
Pipeline stage MKMLDeployer completed in 201.32s
run pipeline stage %s
Running pipeline stage StressChecker
{"detail":"Single '}' encountered in format string"}
Received unhealthy response to inference request!
{"detail":"Single '}' encountered in format string"}
Received unhealthy response to inference request!
{"detail":"Single '}' encountered in format string"}
Received unhealthy response to inference request!
{"detail":"Single '}' encountered in format string"}
Received unhealthy response to inference request!
{"detail":"Single '}' encountered in format string"}
Received unhealthy response to inference request!
5 requests
5 failed requests
5th percentile: 0.16329917907714844
10th percentile: 0.16514105796813966
20th percentile: 0.16882481575012206
30th percentile: 0.17145857810974122
40th percentile: 0.17304234504699706
50th percentile: 0.17462611198425293
60th percentile: 0.1805473804473877
70th percentile: 0.18646864891052245
80th percentile: 0.18996725082397461
90th percentile: 0.19104318618774413
95th percentile: 0.1915811538696289
99th percentile: 0.1920115280151367
mean time: 0.1776597023010254
%s, retrying in %s seconds...
{"detail":"Single '}' encountered in format string"}
Received unhealthy response to inference request!
{"detail":"Single '}' encountered in format string"}
Received unhealthy response to inference request!
{"detail":"Single '}' encountered in format string"}
Received unhealthy response to inference request!
{"detail":"Single '}' encountered in format string"}
Received unhealthy response to inference request!
{"detail":"Single '}' encountered in format string"}
Received unhealthy response to inference request!
5 requests
5 failed requests
5th percentile: 0.16509571075439453
10th percentile: 0.1652524948120117
20th percentile: 0.1655660629272461
30th percentile: 0.16608004570007323
40th percentile: 0.16679444313049316
50th percentile: 0.16750884056091309
60th percentile: 0.180228853225708
70th percentile: 0.19294886589050292
80th percentile: 0.24366860389709477
90th percentile: 0.3323880672454834
95th percentile: 0.3767477989196777
99th percentile: 0.41223558425903317
mean time: 0.22371740341186525
%s, retrying in %s seconds...
{"detail":"Single '}' encountered in format string"}
Received unhealthy response to inference request!
{"detail":"Single '}' encountered in format string"}
Received unhealthy response to inference request!
{"detail":"Single '}' encountered in format string"}
Received unhealthy response to inference request!
{"detail":"Single '}' encountered in format string"}
Received unhealthy response to inference request!
{"detail":"Single '}' encountered in format string"}
Received unhealthy response to inference request!
5 requests
5 failed requests
5th percentile: 0.18593249320983887
10th percentile: 0.18593831062316896
20th percentile: 0.1859499454498291
30th percentile: 0.18611516952514648
40th percentile: 0.1864339828491211
50th percentile: 0.1867527961730957
60th percentile: 0.19086189270019532
70th percentile: 0.19497098922729492
80th percentile: 0.20804100036621095
90th percentile: 0.23007192611694335
95th percentile: 0.24108738899230955
99th percentile: 0.24989975929260252
mean time: 0.20155272483825684
clean up pipeline due to error=DeploymentChecksError('Unacceptable number of predict errors: 100.0%')
Shutdown handler de-registered
mistralai-mistral-nem_93303_v319 status is now failed due to DeploymentManager action
ChatRequest
Generation Params
Prompt Formatter
Chat History
ChatMessage 1