sicariussicariistuff-imp_9734

submission_id: sicariussicariistuff-imp_9734_v1
developer_uid: Meliodia
formatter: {'memory_template': "{bot_name}'s Persona: {memory}\n####\n", 'prompt_template': '{prompt}\n<START>\n', 'bot_template': '{bot_name}: {message}\n', 'user_template': '{user_name}: {message}\n', 'response_template': '{bot_name}:', 'truncate_by_message': False}
generation_params: {'temperature': 1.0, 'top_p': 1.0, 'min_p': 0.0, 'top_k': 40, 'presence_penalty': 0.0, 'frequency_penalty': 0.0, 'stopping_words': ['\n'], 'max_input_tokens': 1024, 'best_of': 8, 'max_output_tokens': 64}
model_name: sicariussicariistuff-imp_9734_v1
model_repo: SicariusSicariiStuff/Impish_LLAMA_3B
status: torndown
timestamp: 2024-10-02T23:49:27+00:00
Resubmit model
Shutdown handler not registered because Python interpreter is not running in the main thread
run pipeline %s
run pipeline stage %s
Running pipeline stage MKMLizer
Starting job with name sicariussicariistuff-imp-9734-v1-mkmlizer
Waiting for job on sicariussicariistuff-imp-9734-v1-mkmlizer to finish
sicariussicariistuff-imp-9734-v1-mkmlizer: ╔═════════════════════════════════════════════════════════════════════╗
sicariussicariistuff-imp-9734-v1-mkmlizer: ║     _____            __           __                                ║
sicariussicariistuff-imp-9734-v1-mkmlizer: ║    / _/ /_ ___    __/ /  ___ ___ / /                                ║
sicariussicariistuff-imp-9734-v1-mkmlizer: ║   / _/ / // / |/|/ / _ \/ -_) -_) /                                 ║
sicariussicariistuff-imp-9734-v1-mkmlizer: ║  /_//_/\_, /|__,__/_//_/\__/\__/_/                                  ║
sicariussicariistuff-imp-9734-v1-mkmlizer: ║       /___/                                                         ║
sicariussicariistuff-imp-9734-v1-mkmlizer: ║                                                                     ║
sicariussicariistuff-imp-9734-v1-mkmlizer: ║  Version: 0.11.12                                                   ║
sicariussicariistuff-imp-9734-v1-mkmlizer: ║  Copyright 2023 MK ONE TECHNOLOGIES Inc.                            ║
sicariussicariistuff-imp-9734-v1-mkmlizer: ║  https://mk1.ai                                                     ║
sicariussicariistuff-imp-9734-v1-mkmlizer: ║                                                                     ║
sicariussicariistuff-imp-9734-v1-mkmlizer: ║  The license key for the current software has been verified as      ║
sicariussicariistuff-imp-9734-v1-mkmlizer: ║  belonging to:                                                      ║
sicariussicariistuff-imp-9734-v1-mkmlizer: ║                                                                     ║
sicariussicariistuff-imp-9734-v1-mkmlizer: ║  Chai Research Corp.                                                ║
sicariussicariistuff-imp-9734-v1-mkmlizer: ║  Account ID: 7997a29f-0ceb-4cc7-9adf-840c57b4ae6f                   ║
sicariussicariistuff-imp-9734-v1-mkmlizer: ║  Expiration: 2024-10-15 23:59:59                                    ║
sicariussicariistuff-imp-9734-v1-mkmlizer: ║                                                                     ║
sicariussicariistuff-imp-9734-v1-mkmlizer: ╚═════════════════════════════════════════════════════════════════════╝
sicariussicariistuff-imp-9734-v1-mkmlizer: Downloaded to shared memory in 18.894s
sicariussicariistuff-imp-9734-v1-mkmlizer: quantizing model to /dev/shm/model_cache, profile:s0, folder:/tmp/tmpryepc2ly, device:0
sicariussicariistuff-imp-9734-v1-mkmlizer: Saving flywheel model at /dev/shm/model_cache
sicariussicariistuff-imp-9734-v1-mkmlizer: quantized model in 11.995s
sicariussicariistuff-imp-9734-v1-mkmlizer: Processed model SicariusSicariiStuff/Impish_LLAMA_3B in 30.890s
sicariussicariistuff-imp-9734-v1-mkmlizer: creating bucket guanaco-mkml-models
sicariussicariistuff-imp-9734-v1-mkmlizer: Bucket 's3://guanaco-mkml-models/' created
sicariussicariistuff-imp-9734-v1-mkmlizer: uploading /dev/shm/model_cache to s3://guanaco-mkml-models/sicariussicariistuff-imp-9734-v1
sicariussicariistuff-imp-9734-v1-mkmlizer: cp /dev/shm/model_cache/config.json s3://guanaco-mkml-models/sicariussicariistuff-imp-9734-v1/config.json
sicariussicariistuff-imp-9734-v1-mkmlizer: cp /dev/shm/model_cache/special_tokens_map.json s3://guanaco-mkml-models/sicariussicariistuff-imp-9734-v1/special_tokens_map.json
sicariussicariistuff-imp-9734-v1-mkmlizer: cp /dev/shm/model_cache/tokenizer_config.json s3://guanaco-mkml-models/sicariussicariistuff-imp-9734-v1/tokenizer_config.json
sicariussicariistuff-imp-9734-v1-mkmlizer: cp /dev/shm/model_cache/tokenizer.json s3://guanaco-mkml-models/sicariussicariistuff-imp-9734-v1/tokenizer.json
sicariussicariistuff-imp-9734-v1-mkmlizer: cp /dev/shm/model_cache/flywheel_model.0.safetensors s3://guanaco-mkml-models/sicariussicariistuff-imp-9734-v1/flywheel_model.0.safetensors
sicariussicariistuff-imp-9734-v1-mkmlizer: 
Loading 0:   0%|          | 0/254 [00:00<?, ?it/s]
Loading 0:   2%|▏         | 5/254 [00:00<00:05, 45.06it/s]
Loading 0:   6%|▌         | 14/254 [00:00<00:04, 57.37it/s]
Loading 0:   9%|▉         | 23/254 [00:00<00:04, 57.61it/s]
Loading 0:  13%|█▎        | 32/254 [00:00<00:03, 60.14it/s]
Loading 0:  16%|█▌        | 41/254 [00:00<00:03, 61.46it/s]
Loading 0:  20%|█▉        | 50/254 [00:00<00:03, 62.13it/s]
Loading 0:  23%|██▎       | 59/254 [00:00<00:03, 62.24it/s]
Loading 0:  27%|██▋       | 68/254 [00:01<00:03, 60.70it/s]
Loading 0:  30%|███       | 77/254 [00:01<00:02, 60.66it/s]
Loading 0:  34%|███▍      | 86/254 [00:01<00:02, 61.29it/s]
Loading 0:  37%|███▋      | 95/254 [00:01<00:02, 61.92it/s]
Loading 0:  41%|████      | 104/254 [00:01<00:02, 61.63it/s]
Loading 0:  44%|████▍     | 113/254 [00:01<00:02, 61.21it/s]
Loading 0:  47%|████▋     | 120/254 [00:02<00:02, 57.96it/s]
Loading 0:  50%|█████     | 128/254 [00:02<00:02, 56.52it/s]
Loading 0:  54%|█████▍    | 137/254 [00:02<00:02, 57.06it/s]
Loading 0:  57%|█████▋    | 146/254 [00:02<00:01, 58.80it/s]
Loading 0:  61%|██████    | 155/254 [00:02<00:01, 59.58it/s]
Loading 0:  65%|██████▍   | 164/254 [00:02<00:01, 58.50it/s]
Loading 0:  68%|██████▊   | 173/254 [00:02<00:01, 58.91it/s]
Loading 0:  72%|███████▏  | 182/254 [00:03<00:01, 57.23it/s]
Loading 0:  74%|███████▍  | 188/254 [00:03<00:01, 43.95it/s]
Loading 0:  78%|███████▊  | 199/254 [00:03<00:01, 51.15it/s]
Loading 0:  82%|████████▏ | 208/254 [00:03<00:00, 53.23it/s]
Loading 0:  85%|████████▌ | 217/254 [00:03<00:00, 56.15it/s]
Loading 0:  89%|████████▉ | 226/254 [00:03<00:00, 57.64it/s]
Loading 0:  93%|█████████▎| 235/254 [00:04<00:00, 59.36it/s]
Loading 0:  96%|█████████▌| 244/254 [00:04<00:00, 60.95it/s]
Loading 0: 100%|█████████▉| 253/254 [00:04<00:00, 61.21it/s]
                                                            
Job sicariussicariistuff-imp-9734-v1-mkmlizer completed after 51.64s with status: succeeded
Stopping job with name sicariussicariistuff-imp-9734-v1-mkmlizer
Pipeline stage MKMLizer completed in 52.44s
run pipeline stage %s
Running pipeline stage MKMLTemplater
Pipeline stage MKMLTemplater completed in 0.09s
run pipeline stage %s
Running pipeline stage MKMLDeployer
Creating inference service sicariussicariistuff-imp-9734-v1
Waiting for inference service sicariussicariistuff-imp-9734-v1 to be ready
Inference service sicariussicariistuff-imp-9734-v1 ready after 150.342200756073s
Pipeline stage MKMLDeployer completed in 150.72s
run pipeline stage %s
Running pipeline stage StressChecker
HTTPSConnectionPool(host='guanaco-submitter.chai-research.com', port=443): Read timed out. (read timeout=20)
Received unhealthy response to inference request!
HTTPSConnectionPool(host='guanaco-submitter.chai-research.com', port=443): Read timed out. (read timeout=20)
Received unhealthy response to inference request!
HTTPSConnectionPool(host='guanaco-submitter.chai-research.com', port=443): Read timed out. (read timeout=20)
Received unhealthy response to inference request!
HTTPSConnectionPool(host='guanaco-submitter.chai-research.com', port=443): Read timed out. (read timeout=20)
Received unhealthy response to inference request!
HTTPSConnectionPool(host='guanaco-submitter.chai-research.com', port=443): Read timed out. (read timeout=20)
Received unhealthy response to inference request!
5 requests
5 failed requests
5th percentile: 20.07552056312561
10th percentile: 20.075961446762086
20th percentile: 20.076843214035033
30th percentile: 20.079977798461915
40th percentile: 20.085365200042723
50th percentile: 20.090752601623535
60th percentile: 20.091354084014892
70th percentile: 20.09195556640625
80th percentile: 20.092877197265626
90th percentile: 20.094118976593016
95th percentile: 20.094739866256713
99th percentile: 20.09523657798767
mean time: 20.086146688461305
%s, retrying in %s seconds...
HTTPSConnectionPool(host='guanaco-submitter.chai-research.com', port=443): Read timed out. (read timeout=20)
Received unhealthy response to inference request!
HTTPSConnectionPool(host='guanaco-submitter.chai-research.com', port=443): Read timed out. (read timeout=20)
Received unhealthy response to inference request!
HTTPSConnectionPool(host='guanaco-submitter.chai-research.com', port=443): Read timed out. (read timeout=20)
Received unhealthy response to inference request!
HTTPSConnectionPool(host='guanaco-submitter.chai-research.com', port=443): Read timed out. (read timeout=20)
Received unhealthy response to inference request!
HTTPSConnectionPool(host='guanaco-submitter.chai-research.com', port=443): Read timed out. (read timeout=20)
Received unhealthy response to inference request!
5 requests
5 failed requests
5th percentile: 20.068527698516846
10th percentile: 20.068679809570312
20th percentile: 20.068984031677246
30th percentile: 20.070390129089354
40th percentile: 20.07289810180664
50th percentile: 20.075406074523926
60th percentile: 20.076632022857666
70th percentile: 20.077857971191406
80th percentile: 20.078888130187988
90th percentile: 20.079722499847414
95th percentile: 20.080139684677125
99th percentile: 20.080473432540895
mean time: 20.074389123916625
%s, retrying in %s seconds...
HTTPSConnectionPool(host='guanaco-submitter.chai-research.com', port=443): Read timed out. (read timeout=20)
Received unhealthy response to inference request!
HTTPSConnectionPool(host='guanaco-submitter.chai-research.com', port=443): Read timed out. (read timeout=20)
Received unhealthy response to inference request!
HTTPSConnectionPool(host='guanaco-submitter.chai-research.com', port=443): Read timed out. (read timeout=20)
Received unhealthy response to inference request!
HTTPSConnectionPool(host='guanaco-submitter.chai-research.com', port=443): Read timed out. (read timeout=20)
Received unhealthy response to inference request!
HTTPSConnectionPool(host='guanaco-submitter.chai-research.com', port=443): Read timed out. (read timeout=20)
Received unhealthy response to inference request!
5 requests
5 failed requests
5th percentile: 20.079092741012573
10th percentile: 20.081079244613647
20th percentile: 20.085052251815796
30th percentile: 20.088191270828247
40th percentile: 20.090496301651
50th percentile: 20.092801332473755
60th percentile: 20.095589685440064
70th percentile: 20.098378038406373
80th percentile: 25.573967266082768
90th percentile: 36.522357368469244
95th percentile: 41.99655241966247
99th percentile: 46.37590846061706
mean time: 25.565493202209474
clean up pipeline due to error=DeploymentChecksError('Unacceptable number of predict errors: 100.0%')
Shutdown handler de-registered
sicariussicariistuff-imp_9734_v1 status is now failed due to DeploymentManager action
sicariussicariistuff-imp_9734_v1 status is now torndown due to DeploymentManager action