arushimgupta-output

submission_id: arushimgupta-output_v5
developer_uid: immaculate_possum_03470
formatter: {'memory_template': "{bot_name}'s Persona: {memory}\n####\n", 'prompt_template': '{prompt}\n<START>\n', 'bot_template': '{bot_name}: {message}\n', 'user_template': '{user_name}: {message}\n', 'response_template': '{bot_name}:', 'truncate_by_message': False}
generation_params: {'temperature': 0.95, 'top_p': 1.0, 'min_p': 0.075, 'top_k': 5, 'presence_penalty': 0.0, 'frequency_penalty': 0.0, 'stopping_words': ['\n', '<|eot_id|>'], 'max_input_tokens': 1024, 'best_of': 8, 'max_output_tokens': 64}
model_name: nemo_base_1
model_repo: arushimgupta/output
status: torndown
timestamp: 2024-09-27T22:59:22+00:00
Resubmit model
Shutdown handler not registered because Python interpreter is not running in the main thread
run pipeline %s
run pipeline stage %s
Running pipeline stage MKMLizer
Starting job with name arushimgupta-output-v5-mkmlizer
Waiting for job on arushimgupta-output-v5-mkmlizer to finish
arushimgupta-output-v5-mkmlizer: ╔═════════════════════════════════════════════════════════════════════╗
arushimgupta-output-v5-mkmlizer: ║     _____            __           __                                ║
arushimgupta-output-v5-mkmlizer: ║    / _/ /_ ___    __/ /  ___ ___ / /                                ║
arushimgupta-output-v5-mkmlizer: ║   / _/ / // / |/|/ / _ \/ -_) -_) /                                 ║
arushimgupta-output-v5-mkmlizer: ║  /_//_/\_, /|__,__/_//_/\__/\__/_/                                  ║
arushimgupta-output-v5-mkmlizer: ║       /___/                                                         ║
arushimgupta-output-v5-mkmlizer: ║                                                                     ║
arushimgupta-output-v5-mkmlizer: ║  Version: 0.11.12                                                   ║
arushimgupta-output-v5-mkmlizer: ║  Copyright 2023 MK ONE TECHNOLOGIES Inc.                            ║
arushimgupta-output-v5-mkmlizer: ║  https://mk1.ai                                                     ║
arushimgupta-output-v5-mkmlizer: ║                                                                     ║
arushimgupta-output-v5-mkmlizer: ║  The license key for the current software has been verified as      ║
arushimgupta-output-v5-mkmlizer: ║  belonging to:                                                      ║
arushimgupta-output-v5-mkmlizer: ║                                                                     ║
arushimgupta-output-v5-mkmlizer: ║  Chai Research Corp.                                                ║
arushimgupta-output-v5-mkmlizer: ║  Account ID: 7997a29f-0ceb-4cc7-9adf-840c57b4ae6f                   ║
arushimgupta-output-v5-mkmlizer: ║  Expiration: 2024-10-15 23:59:59                                    ║
arushimgupta-output-v5-mkmlizer: ║                                                                     ║
arushimgupta-output-v5-mkmlizer: ╚═════════════════════════════════════════════════════════════════════╝
arushimgupta-output-v5-mkmlizer: Downloaded to shared memory in 30.729s
arushimgupta-output-v5-mkmlizer: quantizing model to /dev/shm/model_cache, profile:s0, folder:/tmp/tmp_xjck2pj, device:0
arushimgupta-output-v5-mkmlizer: Saving flywheel model at /dev/shm/model_cache
arushimgupta-output-v5-mkmlizer: quantized model in 43.306s
arushimgupta-output-v5-mkmlizer: Processed model arushimgupta/output in 74.036s
arushimgupta-output-v5-mkmlizer: creating bucket guanaco-mkml-models
arushimgupta-output-v5-mkmlizer: cp /dev/shm/model_cache/special_tokens_map.json s3://guanaco-mkml-models/arushimgupta-output-v5/special_tokens_map.json
arushimgupta-output-v5-mkmlizer: cp /dev/shm/model_cache/config.json s3://guanaco-mkml-models/arushimgupta-output-v5/config.json
arushimgupta-output-v5-mkmlizer: cp /dev/shm/model_cache/tokenizer_config.json s3://guanaco-mkml-models/arushimgupta-output-v5/tokenizer_config.json
arushimgupta-output-v5-mkmlizer: cp /dev/shm/model_cache/tokenizer.json s3://guanaco-mkml-models/arushimgupta-output-v5/tokenizer.json
arushimgupta-output-v5-mkmlizer: cp /dev/shm/model_cache/flywheel_model.0.safetensors s3://guanaco-mkml-models/arushimgupta-output-v5/flywheel_model.0.safetensors
arushimgupta-output-v5-mkmlizer: 
Loading 0:   0%|          | 0/363 [00:00<?, ?it/s]
Loading 0:   1%|▏         | 5/363 [00:00<00:13, 27.22it/s]
Loading 0:   3%|▎         | 12/363 [00:00<00:07, 45.60it/s]
Loading 0:   5%|▍         | 18/363 [00:00<00:07, 47.49it/s]
Loading 0:   7%|▋         | 24/363 [00:00<00:08, 39.71it/s]
Loading 0:   9%|▊         | 31/363 [00:00<00:07, 45.55it/s]
Loading 0:  10%|█         | 37/363 [00:00<00:07, 42.59it/s]
Loading 0:  12%|█▏        | 42/363 [00:00<00:07, 41.72it/s]
Loading 0:  13%|█▎        | 49/363 [00:01<00:06, 46.41it/s]
Loading 0:  15%|█▍        | 54/363 [00:01<00:06, 46.63it/s]
Loading 0:  17%|█▋        | 60/363 [00:01<00:07, 42.96it/s]
Loading 0:  18%|█▊        | 65/363 [00:01<00:10, 29.42it/s]
Loading 0:  20%|█▉        | 71/363 [00:01<00:08, 34.92it/s]
Loading 0:  21%|██        | 76/363 [00:01<00:07, 37.23it/s]
Loading 0:  22%|██▏       | 81/363 [00:02<00:07, 39.37it/s]
Loading 0:  24%|██▎       | 86/363 [00:02<00:06, 41.28it/s]
Loading 0:  25%|██▌       | 91/363 [00:02<00:07, 35.63it/s]
Loading 0:  27%|██▋       | 98/363 [00:02<00:06, 42.66it/s]
Loading 0:  28%|██▊       | 103/363 [00:02<00:06, 42.64it/s]
Loading 0:  30%|██▉       | 108/363 [00:02<00:05, 44.47it/s]
Loading 0:  31%|███       | 113/363 [00:02<00:06, 36.58it/s]
Loading 0:  33%|███▎      | 118/363 [00:02<00:06, 35.14it/s]
Loading 0:  34%|███▍      | 125/363 [00:03<00:05, 40.87it/s]
Loading 0:  36%|███▌      | 130/363 [00:03<00:05, 40.39it/s]
Loading 0:  37%|███▋      | 135/363 [00:03<00:05, 40.83it/s]
Loading 0:  39%|███▊      | 140/363 [00:03<00:05, 42.80it/s]
Loading 0:  40%|███▉      | 145/363 [00:03<00:08, 26.49it/s]
Loading 0:  41%|████      | 149/363 [00:03<00:07, 26.87it/s]
Loading 0:  43%|████▎     | 156/363 [00:04<00:05, 34.71it/s]
Loading 0:  44%|████▍     | 161/363 [00:04<00:05, 35.69it/s]
Loading 0:  46%|████▌     | 166/363 [00:04<00:05, 37.10it/s]
Loading 0:  47%|████▋     | 171/363 [00:04<00:04, 38.85it/s]
Loading 0:  48%|████▊     | 176/363 [00:04<00:05, 32.82it/s]
Loading 0:  50%|█████     | 183/363 [00:04<00:04, 39.19it/s]
Loading 0:  52%|█████▏    | 188/363 [00:04<00:04, 38.82it/s]
Loading 0:  53%|█████▎    | 193/363 [00:05<00:04, 38.84it/s]
Loading 0:  55%|█████▍    | 198/363 [00:05<00:04, 40.31it/s]
Loading 0:  56%|█████▌    | 203/363 [00:05<00:04, 33.13it/s]
Loading 0:  58%|█████▊    | 210/363 [00:05<00:03, 40.00it/s]
Loading 0:  59%|█████▉    | 215/363 [00:05<00:03, 39.76it/s]
Loading 0:  61%|██████    | 220/363 [00:05<00:03, 40.82it/s]
Loading 0:  62%|██████▏   | 225/363 [00:06<00:05, 24.73it/s]
Loading 0:  63%|██████▎   | 230/363 [00:06<00:04, 26.72it/s]
Loading 0:  65%|██████▌   | 237/363 [00:06<00:03, 33.82it/s]
Loading 0:  67%|██████▋   | 242/363 [00:06<00:03, 35.07it/s]
Loading 0:  68%|██████▊   | 247/363 [00:06<00:03, 36.03it/s]
Loading 0:  69%|██████▉   | 252/363 [00:06<00:02, 38.13it/s]
Loading 0:  71%|███████   | 257/363 [00:06<00:03, 31.91it/s]
Loading 0:  73%|███████▎  | 264/363 [00:07<00:02, 38.73it/s]
Loading 0:  74%|███████▍  | 269/363 [00:07<00:02, 38.20it/s]
Loading 0:  75%|███████▌  | 274/363 [00:07<00:02, 38.04it/s]
Loading 0:  77%|███████▋  | 279/363 [00:07<00:02, 39.72it/s]
Loading 0:  78%|███████▊  | 284/363 [00:07<00:02, 33.41it/s]
Loading 0:  80%|████████  | 291/363 [00:07<00:01, 39.78it/s]
Loading 0:  82%|████████▏ | 296/363 [00:07<00:01, 39.48it/s]
Loading 0:  83%|████████▎ | 301/363 [00:08<00:01, 40.92it/s]
Loading 0:  84%|████████▍ | 306/363 [00:15<00:24,  2.31it/s]
Loading 0:  85%|████████▌ | 310/363 [00:15<00:17,  2.99it/s]
Loading 0:  86%|████████▌ | 313/363 [00:15<00:13,  3.67it/s]
Loading 0:  88%|████████▊ | 319/363 [00:15<00:07,  5.64it/s]
Loading 0:  89%|████████▉ | 323/363 [00:15<00:05,  7.24it/s]
Loading 0:  90%|█████████ | 328/363 [00:15<00:03,  9.95it/s]
Loading 0:  92%|█████████▏| 333/363 [00:15<00:02, 13.00it/s]
Loading 0:  93%|█████████▎| 338/363 [00:16<00:01, 16.47it/s]
Loading 0:  94%|█████████▍| 343/363 [00:16<00:00, 20.45it/s]
Loading 0:  96%|█████████▌| 348/363 [00:16<00:00, 21.32it/s]
Loading 0:  98%|█████████▊| 355/363 [00:16<00:00, 28.36it/s]
Loading 0:  99%|█████████▉| 360/363 [00:16<00:00, 30.57it/s]
                                                            
Job arushimgupta-output-v5-mkmlizer completed after 102.33s with status: succeeded
Stopping job with name arushimgupta-output-v5-mkmlizer
Pipeline stage MKMLizer completed in 103.11s
run pipeline stage %s
Running pipeline stage MKMLTemplater
Pipeline stage MKMLTemplater completed in 0.09s
run pipeline stage %s
Running pipeline stage MKMLDeployer
Creating inference service arushimgupta-output-v5
Waiting for inference service arushimgupta-output-v5 to be ready
Inference service arushimgupta-output-v5 ready after 240.6365852355957s
Pipeline stage MKMLDeployer completed in 241.01s
run pipeline stage %s
Running pipeline stage StressChecker
Received healthy response to inference request in 8.147582530975342s
HTTPSConnectionPool(host='guanaco-submitter.chai-research.com', port=443): Read timed out. (read timeout=20)
Received unhealthy response to inference request!
Received healthy response to inference request in 2.425957679748535s
Received healthy response to inference request in 14.130488157272339s
HTTPSConnectionPool(host='guanaco-submitter.chai-research.com', port=443): Read timed out. (read timeout=20)
Received unhealthy response to inference request!
5 requests
2 failed requests
5th percentile: 3.5702826499938967
10th percentile: 4.714607620239258
20th percentile: 7.003257560729981
30th percentile: 9.344163656234741
40th percentile: 11.73732590675354
50th percentile: 14.130488157272339
60th percentile: 16.506548357009887
70th percentile: 18.882608556747435
80th percentile: 20.07148051261902
90th percentile: 20.073164224624634
95th percentile: 20.07400608062744
99th percentile: 20.074679565429687
mean time: 12.969902992248535
%s, retrying in %s seconds...
HTTPSConnectionPool(host='guanaco-submitter.chai-research.com', port=443): Read timed out. (read timeout=20)
Received unhealthy response to inference request!
Received healthy response to inference request in 2.2874646186828613s
Received healthy response to inference request in 3.6930313110351562s
Received healthy response to inference request in 2.0263173580169678s
Received healthy response to inference request in 1.8085055351257324s
5 requests
1 failed requests
5th percentile: 1.8520678997039794
10th percentile: 1.8956302642822265
20th percentile: 1.9827549934387207
30th percentile: 2.0785468101501463
40th percentile: 2.183005714416504
50th percentile: 2.2874646186828613
60th percentile: 2.849691295623779
70th percentile: 3.411917972564697
80th percentile: 6.974874925613406
90th percentile: 13.538562154769899
95th percentile: 16.82040576934814
99th percentile: 19.44588066101074
mean time: 5.983513641357422
%s, retrying in %s seconds...
HTTPSConnectionPool(host='guanaco-submitter.chai-research.com', port=443): Read timed out. (read timeout=20)
Received unhealthy response to inference request!
Received healthy response to inference request in 2.5767550468444824s
Received healthy response to inference request in 12.169520378112793s
Received healthy response to inference request in 1.9272246360778809s
Received healthy response to inference request in 2.3669967651367188s
5 requests
1 failed requests
5th percentile: 2.0151790618896483
10th percentile: 2.103133487701416
20th percentile: 2.2790423393249513
30th percentile: 2.4089484214782715
40th percentile: 2.492851734161377
50th percentile: 2.5767550468444824
60th percentile: 6.413861179351805
70th percentile: 10.25096731185913
80th percentile: 13.753730583190919
90th percentile: 16.92215099334717
95th percentile: 18.506361198425292
99th percentile: 19.773729362487792
mean time: 7.826213645935058
clean up pipeline due to error=DeploymentChecksError('Unacceptable number of predict errors: 20.0%')
Shutdown handler de-registered
arushimgupta-output_v5 status is now failed due to DeploymentManager action
arushimgupta-output_v5 status is now torndown due to DeploymentManager action