submission_id: mistralai-mistral-nem_93303_v319
developer_uid: RandomForest1024
status: failed
model_repo: mistralai/Mistral-Nemo-Instruct-2407
generation_params: {'temperature': 1.0, 'top_p': 1.0, 'min_p': 0.0, 'top_k': 40, 'presence_penalty': 0.0, 'frequency_penalty': 0.0, 'stopping_words': ['\n'], 'max_input_tokens': 1024, 'best_of': 8, 'max_output_tokens': 64}
formatter: {'memory_template': "{bot_name}'s Persona: {memory}\n####\n", 'prompt_template': "'<Context>Previous conversation shows your personality as described above.</Context> {prompt}\n<START>\n'", 'bot_template': "'{bot_name}: {message}\n [Maintain teenage perspective while being sophisticated. Use occasional gaming or sports references. Keep political topics neutral.]'", 'user_template': "'{user_name}: {message}\n [Focus on building genuine connections while maintaining appropriate boundaries.]'", 'response_template': "'{bot_name}: [Think as a Gen Z teenager with unique life experiences, then respond]', 'truncate_by_message': False}", 'truncate_by_message': False}
timestamp: 2025-02-10T18:38:23+00:00
model_name: mistralai-mistral-nem_93303_v319
Resubmit model
Shutdown handler not registered because Python interpreter is not running in the main thread
run pipeline %s
run pipeline stage %s
Running pipeline stage MKMLizer
Starting job with name mistralai-mistral-nem-93303-v319-mkmlizer
Waiting for job on mistralai-mistral-nem-93303-v319-mkmlizer to finish
Failed to get response for submission randomforest1024-v25_v6: HTTPConnectionPool(host='randomforest1024-v25-v6-predictor.tenant-chaiml-guanaco.k.chaiverse.com', port=80): Read timed out. (read timeout=12.0)
mistralai-mistral-nem-93303-v319-mkmlizer: ╔═════════════════════════════════════════════════════════════════════╗
mistralai-mistral-nem-93303-v319-mkmlizer: ║ _____ __ __ ║
mistralai-mistral-nem-93303-v319-mkmlizer: ║ / _/ /_ ___ __/ / ___ ___ / / ║
mistralai-mistral-nem-93303-v319-mkmlizer: ║ / _/ / // / |/|/ / _ \/ -_) -_) / ║
mistralai-mistral-nem-93303-v319-mkmlizer: ║ /_//_/\_, /|__,__/_//_/\__/\__/_/ ║
mistralai-mistral-nem-93303-v319-mkmlizer: ║ /___/ ║
mistralai-mistral-nem-93303-v319-mkmlizer: ║ ║
mistralai-mistral-nem-93303-v319-mkmlizer: ║ Version: 0.11.12 ║
mistralai-mistral-nem-93303-v319-mkmlizer: ║ Copyright 2023 MK ONE TECHNOLOGIES Inc. ║
mistralai-mistral-nem-93303-v319-mkmlizer: ║ https://mk1.ai ║
mistralai-mistral-nem-93303-v319-mkmlizer: ║ ║
mistralai-mistral-nem-93303-v319-mkmlizer: ║ The license key for the current software has been verified as ║
mistralai-mistral-nem-93303-v319-mkmlizer: ║ belonging to: ║
mistralai-mistral-nem-93303-v319-mkmlizer: ║ ║
mistralai-mistral-nem-93303-v319-mkmlizer: ║ Chai Research Corp. ║
mistralai-mistral-nem-93303-v319-mkmlizer: ║ Account ID: 7997a29f-0ceb-4cc7-9adf-840c57b4ae6f ║
mistralai-mistral-nem-93303-v319-mkmlizer: ║ Expiration: 2025-04-15 23:59:59 ║
mistralai-mistral-nem-93303-v319-mkmlizer: ║ ║
mistralai-mistral-nem-93303-v319-mkmlizer: ╚═════════════════════════════════════════════════════════════════════╝
Failed to get response for submission randomforest1024-v25_v6: HTTPConnectionPool(host='randomforest1024-v25-v6-predictor.tenant-chaiml-guanaco.k.chaiverse.com', port=80): Read timed out. (read timeout=12.0)
mistralai-mistral-nem-93303-v319-mkmlizer: Downloaded to shared memory in 54.925s
mistralai-mistral-nem-93303-v319-mkmlizer: quantizing model to /dev/shm/model_cache, profile:s0, folder:/tmp/tmpej6noa9r, device:0
mistralai-mistral-nem-93303-v319-mkmlizer: Saving flywheel model at /dev/shm/model_cache
mistralai-mistral-nem-93303-v319-mkmlizer: quantized model in 39.035s
mistralai-mistral-nem-93303-v319-mkmlizer: Processed model mistralai/Mistral-Nemo-Instruct-2407 in 93.960s
mistralai-mistral-nem-93303-v319-mkmlizer: creating bucket guanaco-mkml-models
mistralai-mistral-nem-93303-v319-mkmlizer: Bucket 's3://guanaco-mkml-models/' created
mistralai-mistral-nem-93303-v319-mkmlizer: uploading /dev/shm/model_cache to s3://guanaco-mkml-models/mistralai-mistral-nem-93303-v319
mistralai-mistral-nem-93303-v319-mkmlizer: cp /dev/shm/model_cache/special_tokens_map.json s3://guanaco-mkml-models/mistralai-mistral-nem-93303-v319/special_tokens_map.json
mistralai-mistral-nem-93303-v319-mkmlizer: cp /dev/shm/model_cache/config.json s3://guanaco-mkml-models/mistralai-mistral-nem-93303-v319/config.json
mistralai-mistral-nem-93303-v319-mkmlizer: cp /dev/shm/model_cache/tokenizer_config.json s3://guanaco-mkml-models/mistralai-mistral-nem-93303-v319/tokenizer_config.json
mistralai-mistral-nem-93303-v319-mkmlizer: cp /dev/shm/model_cache/tokenizer.json s3://guanaco-mkml-models/mistralai-mistral-nem-93303-v319/tokenizer.json
mistralai-mistral-nem-93303-v319-mkmlizer: cp /dev/shm/model_cache/flywheel_model.0.safetensors s3://guanaco-mkml-models/mistralai-mistral-nem-93303-v319/flywheel_model.0.safetensors
mistralai-mistral-nem-93303-v319-mkmlizer: Loading 0: 0%| | 0/363 [00:00<?, ?it/s] Loading 0: 1%|▏ | 5/363 [00:00<00:12, 27.80it/s] Loading 0: 3%|▎ | 10/363 [00:00<00:09, 37.50it/s] Loading 0: 4%|▍ | 15/363 [00:00<00:09, 35.39it/s] Loading 0: 6%|▌ | 21/363 [00:00<00:08, 40.96it/s] Loading 0: 7%|▋ | 26/363 [00:00<00:08, 40.31it/s] Loading 0: 9%|▊ | 31/363 [00:00<00:08, 40.08it/s] Loading 0: 10%|▉ | 36/363 [00:00<00:07, 41.21it/s] Loading 0: 11%|█▏ | 41/363 [00:01<00:09, 32.82it/s] Loading 0: 13%|█▎ | 48/363 [00:01<00:07, 39.48it/s] Loading 0: 15%|█▍ | 53/363 [00:01<00:07, 38.90it/s] Loading 0: 16%|█▌ | 58/363 [00:01<00:07, 39.67it/s] Loading 0: 17%|█▋ | 63/363 [00:01<00:12, 24.35it/s] Loading 0: 18%|█▊ | 67/363 [00:02<00:11, 26.47it/s] Loading 0: 20%|█▉ | 71/363 [00:02<00:10, 28.43it/s] Loading 0: 21%|██ | 75/363 [00:02<00:09, 29.02it/s] Loading 0: 22%|██▏ | 80/363 [00:02<00:08, 32.11it/s] Loading 0: 23%|██▎ | 84/363 [00:02<00:08, 31.53it/s] Loading 0: 25%|██▍ | 89/363 [00:02<00:08, 34.05it/s] Loading 0: 26%|██▌ | 93/363 [00:02<00:08, 32.97it/s] Loading 0: 27%|██▋ | 98/363 [00:02<00:07, 34.76it/s] Loading 0: 28%|██▊ | 102/363 [00:03<00:07, 33.67it/s] Loading 0: 29%|██▉ | 106/363 [00:03<00:07, 33.84it/s] Loading 0: 31%|███ | 112/363 [00:03<00:06, 37.42it/s] Loading 0: 32%|███▏ | 116/363 [00:03<00:07, 33.29it/s] Loading 0: 33%|███▎ | 120/363 [00:03<00:07, 32.67it/s] Loading 0: 34%|███▍ | 125/363 [00:03<00:06, 35.28it/s] Loading 0: 36%|███▌ | 129/363 [00:03<00:06, 34.27it/s] Loading 0: 37%|███▋ | 133/363 [00:03<00:06, 35.62it/s] Loading 0: 38%|███▊ | 137/363 [00:04<00:07, 32.24it/s] Loading 0: 39%|███▉ | 141/363 [00:04<00:06, 34.02it/s] Loading 0: 40%|███▉ | 145/363 [00:04<00:10, 20.74it/s] Loading 0: 41%|████ | 149/363 [00:04<00:09, 23.05it/s] Loading 0: 43%|████▎ | 156/363 [00:04<00:06, 31.73it/s] Loading 0: 44%|████▍ | 160/363 [00:04<00:06, 33.03it/s] Loading 0: 45%|████▌ | 165/363 [00:04<00:05, 36.59it/s] Loading 0: 47%|████▋ | 170/363 [00:05<00:04, 39.09it/s] Loading 0: 48%|████▊ | 175/363 [00:05<00:04, 40.67it/s] Loading 0: 50%|████▉ | 181/363 [00:05<00:04, 39.10it/s] Loading 0: 51%|█████ | 186/363 [00:05<00:04, 38.43it/s] Loading 0: 53%|█████▎ | 192/363 [00:05<00:03, 42.91it/s] Loading 0: 54%|█████▍ | 197/363 [00:05<00:03, 41.74it/s] Loading 0: 56%|█████▌ | 202/363 [00:05<00:03, 42.50it/s] Loading 0: 57%|█████▋ | 207/363 [00:05<00:03, 44.28it/s] Loading 0: 58%|█████▊ | 212/363 [00:06<00:04, 36.65it/s] Loading 0: 60%|██████ | 218/363 [00:06<00:03, 41.88it/s] Loading 0: 61%|██████▏ | 223/363 [00:06<00:04, 28.02it/s] Loading 0: 63%|██████▎ | 227/363 [00:06<00:04, 29.49it/s] Loading 0: 64%|██████▎ | 231/363 [00:06<00:04, 28.58it/s] Loading 0: 65%|██████▍ | 235/363 [00:06<00:04, 30.42it/s] Loading 0: 66%|██████▌ | 239/363 [00:07<00:04, 27.93it/s] Loading 0: 68%|██████▊ | 246/363 [00:07<00:03, 35.43it/s] Loading 0: 69%|██████▉ | 250/363 [00:07<00:03, 35.19it/s] Loading 0: 70%|███████ | 255/363 [00:07<00:02, 36.93it/s] Loading 0: 71%|███████▏ | 259/363 [00:07<00:02, 35.62it/s] Loading 0: 73%|███████▎ | 264/363 [00:07<00:02, 37.39it/s] Loading 0: 74%|███████▍ | 268/363 [00:07<00:02, 35.15it/s] Loading 0: 75%|███████▍ | 272/363 [00:07<00:02, 35.87it/s] Loading 0: 76%|███████▌ | 276/363 [00:08<00:02, 31.83it/s] Loading 0: 77%|███████▋ | 280/363 [00:08<00:02, 33.76it/s] Loading 0: 78%|███████▊ | 284/363 [00:08<00:02, 31.45it/s] Loading 0: 80%|███████▉ | 289/363 [00:08<00:02, 35.77it/s] Loading 0: 81%|████████ | 293/363 [00:08<00:02, 32.62it/s] Loading 0: 82%|████████▏ | 298/363 [00:08<00:01, 36.07it/s] Loading 0: 83%|████████▎ | 303/363 [00:08<00:01, 36.46it/s] Loading 0: 85%|████████▍ | 307/363 [00:15<00:27, 2.05it/s] Loading 0: 86%|████████▌ | 312/363 [00:15<00:17, 2.95it/s] Loading 0: 88%|████████▊ | 319/363 [00:16<00:09, 4.73it/s] Loading 0: 89%|████████▉ | 323/363 [00:16<00:06, 5.99it/s] Loading 0: 90%|█████████ | 328/363 [00:16<00:04, 8.12it/s] Loading 0: 92%|█████████▏| 333/363 [00:16<00:02, 10.66it/s] Loading 0: 93%|█████████▎| 338/363 [00:16<00:01, 13.74it/s] Loading 0: 94%|█████████▍| 343/363 [00:16<00:01, 17.51it/s] Loading 0: 96%|█████████▌| 348/363 [00:16<00:00, 19.43it/s] Loading 0: 98%|█████████▊| 356/363 [00:16<00:00, 27.65it/s] Loading 0: 100%|█████████▉| 362/363 [00:17<00:00, 30.50it/s]
Job mistralai-mistral-nem-93303-v319-mkmlizer completed after 124.49s with status: succeeded
Stopping job with name mistralai-mistral-nem-93303-v319-mkmlizer
Pipeline stage MKMLizer completed in 124.94s
run pipeline stage %s
Running pipeline stage MKMLTemplater
Pipeline stage MKMLTemplater completed in 0.61s
run pipeline stage %s
Running pipeline stage MKMLDeployer
Creating inference service mistralai-mistral-nem-93303-v319
Waiting for inference service mistralai-mistral-nem-93303-v319 to be ready
Inference service mistralai-mistral-nem-93303-v319 ready after 200.82187390327454s
Pipeline stage MKMLDeployer completed in 201.32s
run pipeline stage %s
Running pipeline stage StressChecker
{"detail":"Single '}' encountered in format string"}
Received unhealthy response to inference request!
{"detail":"Single '}' encountered in format string"}
Received unhealthy response to inference request!
{"detail":"Single '}' encountered in format string"}
Received unhealthy response to inference request!
{"detail":"Single '}' encountered in format string"}
Received unhealthy response to inference request!
{"detail":"Single '}' encountered in format string"}
Received unhealthy response to inference request!
5 requests
5 failed requests
5th percentile: 0.16329917907714844
10th percentile: 0.16514105796813966
20th percentile: 0.16882481575012206
30th percentile: 0.17145857810974122
40th percentile: 0.17304234504699706
50th percentile: 0.17462611198425293
60th percentile: 0.1805473804473877
70th percentile: 0.18646864891052245
80th percentile: 0.18996725082397461
90th percentile: 0.19104318618774413
95th percentile: 0.1915811538696289
99th percentile: 0.1920115280151367
mean time: 0.1776597023010254
%s, retrying in %s seconds...
{"detail":"Single '}' encountered in format string"}
Received unhealthy response to inference request!
{"detail":"Single '}' encountered in format string"}
Received unhealthy response to inference request!
{"detail":"Single '}' encountered in format string"}
Received unhealthy response to inference request!
{"detail":"Single '}' encountered in format string"}
Received unhealthy response to inference request!
{"detail":"Single '}' encountered in format string"}
Received unhealthy response to inference request!
5 requests
5 failed requests
5th percentile: 0.16509571075439453
10th percentile: 0.1652524948120117
20th percentile: 0.1655660629272461
30th percentile: 0.16608004570007323
40th percentile: 0.16679444313049316
50th percentile: 0.16750884056091309
60th percentile: 0.180228853225708
70th percentile: 0.19294886589050292
80th percentile: 0.24366860389709477
90th percentile: 0.3323880672454834
95th percentile: 0.3767477989196777
99th percentile: 0.41223558425903317
mean time: 0.22371740341186525
%s, retrying in %s seconds...
{"detail":"Single '}' encountered in format string"}
Received unhealthy response to inference request!
{"detail":"Single '}' encountered in format string"}
Received unhealthy response to inference request!
{"detail":"Single '}' encountered in format string"}
Received unhealthy response to inference request!
{"detail":"Single '}' encountered in format string"}
Received unhealthy response to inference request!
{"detail":"Single '}' encountered in format string"}
Received unhealthy response to inference request!
5 requests
5 failed requests
5th percentile: 0.18593249320983887
10th percentile: 0.18593831062316896
20th percentile: 0.1859499454498291
30th percentile: 0.18611516952514648
40th percentile: 0.1864339828491211
50th percentile: 0.1867527961730957
60th percentile: 0.19086189270019532
70th percentile: 0.19497098922729492
80th percentile: 0.20804100036621095
90th percentile: 0.23007192611694335
95th percentile: 0.24108738899230955
99th percentile: 0.24989975929260252
mean time: 0.20155272483825684
clean up pipeline due to error=DeploymentChecksError('Unacceptable number of predict errors: 100.0%')
Shutdown handler de-registered
mistralai-mistral-nem_93303_v319 status is now failed due to DeploymentManager action
ChatRequest
Generation Params
Prompt Formatter
Chat History
ChatMessage 1