arushimgupta-output

submission_id: arushimgupta-output_v3
developer_uid: immaculate_possum_03470
formatter: {'memory_template': "{bot_name}'s Persona: {memory}\n####\n", 'prompt_template': '{prompt}\n<START>\n', 'bot_template': '{bot_name}: {message}\n', 'user_template': '{user_name}: {message}\n', 'response_template': '{bot_name}:', 'truncate_by_message': False}
generation_params: {'temperature': 0.95, 'top_p': 1.0, 'min_p': 0.075, 'top_k': 60, 'presence_penalty': 0.0, 'frequency_penalty': 0.0, 'stopping_words': ['\n', '<|eot_id|>'], 'max_input_tokens': 1024, 'best_of': 8, 'max_output_tokens': 64}
model_name: nemo_base_1
model_repo: arushimgupta/output
status: torndown
timestamp: 2024-09-27T22:44:17+00:00
Resubmit model
Shutdown handler not registered because Python interpreter is not running in the main thread
run pipeline %s
run pipeline stage %s
Running pipeline stage MKMLizer
Starting job with name arushimgupta-output-v3-mkmlizer
Waiting for job on arushimgupta-output-v3-mkmlizer to finish
arushimgupta-output-v3-mkmlizer: ╔═════════════════════════════════════════════════════════════════════╗
arushimgupta-output-v3-mkmlizer: ║     _____            __           __                                ║
arushimgupta-output-v3-mkmlizer: ║    / _/ /_ ___    __/ /  ___ ___ / /                                ║
arushimgupta-output-v3-mkmlizer: ║   / _/ / // / |/|/ / _ \/ -_) -_) /                                 ║
arushimgupta-output-v3-mkmlizer: ║  /_//_/\_, /|__,__/_//_/\__/\__/_/                                  ║
arushimgupta-output-v3-mkmlizer: ║       /___/                                                         ║
arushimgupta-output-v3-mkmlizer: ║                                                                     ║
arushimgupta-output-v3-mkmlizer: ║  Version: 0.11.12                                                   ║
arushimgupta-output-v3-mkmlizer: ║  Copyright 2023 MK ONE TECHNOLOGIES Inc.                            ║
arushimgupta-output-v3-mkmlizer: ║  https://mk1.ai                                                     ║
arushimgupta-output-v3-mkmlizer: ║                                                                     ║
arushimgupta-output-v3-mkmlizer: ║  The license key for the current software has been verified as      ║
arushimgupta-output-v3-mkmlizer: ║  belonging to:                                                      ║
arushimgupta-output-v3-mkmlizer: ║                                                                     ║
arushimgupta-output-v3-mkmlizer: ║  Chai Research Corp.                                                ║
arushimgupta-output-v3-mkmlizer: ║  Account ID: 7997a29f-0ceb-4cc7-9adf-840c57b4ae6f                   ║
arushimgupta-output-v3-mkmlizer: ║  Expiration: 2024-10-15 23:59:59                                    ║
arushimgupta-output-v3-mkmlizer: ║                                                                     ║
arushimgupta-output-v3-mkmlizer: ╚═════════════════════════════════════════════════════════════════════╝
arushimgupta-output-v3-mkmlizer: Downloaded to shared memory in 29.106s
arushimgupta-output-v3-mkmlizer: quantizing model to /dev/shm/model_cache, profile:s0, folder:/tmp/tmpy3vs9y3f, device:0
arushimgupta-output-v3-mkmlizer: Saving flywheel model at /dev/shm/model_cache
arushimgupta-output-v3-mkmlizer: quantized model in 42.350s
arushimgupta-output-v3-mkmlizer: Processed model arushimgupta/output in 71.456s
arushimgupta-output-v3-mkmlizer: creating bucket guanaco-mkml-models
arushimgupta-output-v3-mkmlizer: Bucket 's3://guanaco-mkml-models/' created
arushimgupta-output-v3-mkmlizer: uploading /dev/shm/model_cache to s3://guanaco-mkml-models/arushimgupta-output-v3
arushimgupta-output-v3-mkmlizer: cp /dev/shm/model_cache/config.json s3://guanaco-mkml-models/arushimgupta-output-v3/config.json
arushimgupta-output-v3-mkmlizer: cp /dev/shm/model_cache/special_tokens_map.json s3://guanaco-mkml-models/arushimgupta-output-v3/special_tokens_map.json
arushimgupta-output-v3-mkmlizer: cp /dev/shm/model_cache/tokenizer_config.json s3://guanaco-mkml-models/arushimgupta-output-v3/tokenizer_config.json
arushimgupta-output-v3-mkmlizer: cp /dev/shm/model_cache/tokenizer.json s3://guanaco-mkml-models/arushimgupta-output-v3/tokenizer.json
arushimgupta-output-v3-mkmlizer: cp /dev/shm/model_cache/flywheel_model.0.safetensors s3://guanaco-mkml-models/arushimgupta-output-v3/flywheel_model.0.safetensors
arushimgupta-output-v3-mkmlizer: 
Loading 0:   0%|          | 0/363 [00:00<?, ?it/s]
Loading 0:   1%|▏         | 5/363 [00:00<00:12, 29.40it/s]
Loading 0:   4%|▎         | 13/363 [00:00<00:07, 49.85it/s]
Loading 0:   5%|▌         | 19/363 [00:00<00:07, 45.59it/s]
Loading 0:   7%|▋         | 24/363 [00:00<00:07, 44.46it/s]
Loading 0:   8%|▊         | 30/363 [00:00<00:06, 48.43it/s]
Loading 0:  10%|▉         | 35/363 [00:00<00:06, 47.38it/s]
Loading 0:  11%|█         | 40/363 [00:00<00:06, 46.27it/s]
Loading 0:  12%|█▏        | 45/363 [00:00<00:06, 46.69it/s]
Loading 0:  14%|█▍        | 50/363 [00:01<00:08, 38.62it/s]
Loading 0:  16%|█▌        | 58/363 [00:01<00:06, 48.36it/s]
Loading 0:  18%|█▊        | 64/363 [00:01<00:09, 30.49it/s]
Loading 0:  20%|█▉        | 71/363 [00:01<00:07, 37.50it/s]
Loading 0:  21%|██        | 76/363 [00:01<00:07, 39.38it/s]
Loading 0:  22%|██▏       | 81/363 [00:01<00:06, 41.16it/s]
Loading 0:  24%|██▍       | 87/363 [00:02<00:06, 40.07it/s]
Loading 0:  25%|██▌       | 92/363 [00:02<00:06, 40.56it/s]
Loading 0:  27%|██▋       | 98/363 [00:02<00:05, 44.54it/s]
Loading 0:  28%|██▊       | 103/363 [00:02<00:05, 44.91it/s]
Loading 0:  30%|███       | 109/363 [00:02<00:05, 48.84it/s]
Loading 0:  32%|███▏      | 115/363 [00:02<00:05, 43.68it/s]
Loading 0:  33%|███▎      | 120/363 [00:02<00:05, 41.74it/s]
Loading 0:  35%|███▍      | 126/363 [00:02<00:05, 43.65it/s]
Loading 0:  36%|███▌      | 131/363 [00:03<00:05, 43.77it/s]
Loading 0:  37%|███▋      | 136/363 [00:03<00:06, 35.21it/s]
Loading 0:  39%|███▉      | 142/363 [00:03<00:07, 30.61it/s]
Loading 0:  40%|████      | 146/363 [00:03<00:06, 31.85it/s]
Loading 0:  41%|████▏     | 150/363 [00:03<00:06, 31.32it/s]
Loading 0:  43%|████▎     | 156/363 [00:03<00:05, 37.58it/s]
Loading 0:  44%|████▍     | 161/363 [00:04<00:05, 38.01it/s]
Loading 0:  46%|████▌     | 166/363 [00:04<00:05, 39.07it/s]
Loading 0:  47%|████▋     | 171/363 [00:04<00:04, 41.58it/s]
Loading 0:  48%|████▊     | 176/363 [00:04<00:05, 34.96it/s]
Loading 0:  50%|█████     | 183/363 [00:04<00:04, 42.02it/s]
Loading 0:  52%|█████▏    | 188/363 [00:04<00:04, 41.42it/s]
Loading 0:  53%|█████▎    | 193/363 [00:04<00:04, 42.07it/s]
Loading 0:  55%|█████▍    | 198/363 [00:04<00:03, 44.05it/s]
Loading 0:  56%|█████▌    | 203/363 [00:05<00:04, 36.70it/s]
Loading 0:  58%|█████▊    | 210/363 [00:05<00:03, 44.23it/s]
Loading 0:  59%|█████▉    | 215/363 [00:05<00:03, 42.77it/s]
Loading 0:  61%|██████    | 220/363 [00:05<00:03, 43.42it/s]
Loading 0:  62%|██████▏   | 225/363 [00:05<00:05, 25.48it/s]
Loading 0:  63%|██████▎   | 230/363 [00:05<00:04, 27.71it/s]
Loading 0:  65%|██████▌   | 237/363 [00:06<00:03, 35.31it/s]
Loading 0:  67%|██████▋   | 242/363 [00:06<00:03, 36.78it/s]
Loading 0:  68%|██████▊   | 247/363 [00:06<00:03, 37.92it/s]
Loading 0:  69%|██████▉   | 252/363 [00:06<00:02, 39.58it/s]
Loading 0:  71%|███████   | 257/363 [00:06<00:03, 34.11it/s]
Loading 0:  73%|███████▎  | 264/363 [00:06<00:02, 41.31it/s]
Loading 0:  74%|███████▍  | 269/363 [00:06<00:02, 40.86it/s]
Loading 0:  75%|███████▌  | 274/363 [00:06<00:02, 40.52it/s]
Loading 0:  77%|███████▋  | 279/363 [00:07<00:01, 42.42it/s]
Loading 0:  78%|███████▊  | 284/363 [00:07<00:02, 35.99it/s]
Loading 0:  80%|████████  | 291/363 [00:07<00:01, 43.12it/s]
Loading 0:  82%|████████▏ | 296/363 [00:07<00:01, 43.02it/s]
Loading 0:  83%|████████▎ | 301/363 [00:07<00:01, 44.66it/s]
Loading 0:  84%|████████▍ | 306/363 [00:14<00:24,  2.28it/s]
Loading 0:  85%|████████▌ | 310/363 [00:15<00:17,  2.97it/s]
Loading 0:  87%|████████▋ | 314/363 [00:15<00:12,  3.90it/s]
Loading 0:  88%|████████▊ | 320/363 [00:15<00:07,  5.83it/s]
Loading 0:  90%|████████▉ | 326/363 [00:15<00:04,  8.17it/s]
Loading 0:  91%|█████████ | 330/363 [00:15<00:03,  9.93it/s]
Loading 0:  93%|█████████▎| 338/363 [00:15<00:01, 15.21it/s]
Loading 0:  94%|█████████▍| 343/363 [00:15<00:01, 18.64it/s]
Loading 0:  96%|█████████▌| 348/363 [00:16<00:00, 19.81it/s]
Loading 0:  98%|█████████▊| 355/363 [00:16<00:00, 26.37it/s]
Loading 0:  99%|█████████▉| 360/363 [00:16<00:00, 29.14it/s]
                                                            
Job arushimgupta-output-v3-mkmlizer completed after 92.28s with status: succeeded
Stopping job with name arushimgupta-output-v3-mkmlizer
Pipeline stage MKMLizer completed in 93.07s
run pipeline stage %s
Running pipeline stage MKMLTemplater
Pipeline stage MKMLTemplater completed in 0.08s
run pipeline stage %s
Running pipeline stage MKMLDeployer
Creating inference service arushimgupta-output-v3
Waiting for inference service arushimgupta-output-v3 to be ready
Inference service arushimgupta-output-v3 ready after 230.5014123916626s
Pipeline stage MKMLDeployer completed in 230.84s
run pipeline stage %s
Running pipeline stage StressChecker
Received healthy response to inference request in 12.855850219726562s
Received healthy response to inference request in 2.1004037857055664s
Received healthy response to inference request in 2.191563129425049s
HTTPSConnectionPool(host='guanaco-submitter.chai-research.com', port=443): Read timed out. (read timeout=20)
Received unhealthy response to inference request!
Received healthy response to inference request in 2.162919044494629s
5 requests
1 failed requests
5th percentile: 2.112906837463379
10th percentile: 2.1254098892211912
20th percentile: 2.1504159927368165
30th percentile: 2.168647861480713
40th percentile: 2.1801054954528807
50th percentile: 2.191563129425049
60th percentile: 6.457277965545654
70th percentile: 10.722992801666258
80th percentile: 14.297270536422731
90th percentile: 17.180111169815063
95th percentile: 18.62153148651123
99th percentile: 19.774667739868164
mean time: 7.8747375965118405
%s, retrying in %s seconds...
Received healthy response to inference request in 2.0876784324645996s
HTTPSConnectionPool(host='guanaco-submitter.chai-research.com', port=443): Read timed out. (read timeout=20)
Received unhealthy response to inference request!
Received healthy response to inference request in 2.745032787322998s
Received healthy response to inference request in 2.180508613586426s
Received healthy response to inference request in 2.158282995223999s
5 requests
1 failed requests
5th percentile: 2.1017993450164796
10th percentile: 2.1159202575683596
20th percentile: 2.144162082672119
30th percentile: 2.162728118896484
40th percentile: 2.171618366241455
50th percentile: 2.180508613586426
60th percentile: 2.4063182830810548
70th percentile: 2.6321279525756833
80th percentile: 6.211415910720828
90th percentile: 13.144182157516482
95th percentile: 16.610565280914305
99th percentile: 19.383671779632568
mean time: 5.849690246582031
%s, retrying in %s seconds...
Received healthy response to inference request in 1.8729920387268066s
Received healthy response to inference request in 12.89944314956665s
Received healthy response to inference request in 2.20816707611084s
Received healthy response to inference request in 4.124302387237549s
Received healthy response to inference request in 2.1227433681488037s
5 requests
0 failed requests
5th percentile: 1.922942304611206
10th percentile: 1.9728925704956055
20th percentile: 2.0727931022644044
30th percentile: 2.139828109741211
40th percentile: 2.1739975929260256
50th percentile: 2.20816707611084
60th percentile: 2.9746212005615233
70th percentile: 3.741075325012207
80th percentile: 5.87933053970337
90th percentile: 9.38938684463501
95th percentile: 11.144414997100828
99th percentile: 12.548437519073486
mean time: 4.6455296039581295
clean up pipeline due to error=DeploymentChecksError('Unacceptable 70th percentile latency 3.741075325012207s')
Shutdown handler de-registered
arushimgupta-output_v3 status is now failed due to DeploymentManager action
arushimgupta-output_v3 status is now torndown due to DeploymentManager action