arushimgupta-output

submission_id: arushimgupta-output_v4
developer_uid: immaculate_possum_03470
formatter: {'memory_template': "{bot_name}'s Persona: {memory}\n####\n", 'prompt_template': '{prompt}\n<START>\n', 'bot_template': '{bot_name}: {message}\n', 'user_template': '{user_name}: {message}\n', 'response_template': '{bot_name}:', 'truncate_by_message': False}
generation_params: {'temperature': 0.95, 'top_p': 1.0, 'min_p': 0.075, 'top_k': 20, 'presence_penalty': 0.0, 'frequency_penalty': 0.0, 'stopping_words': ['\n', '<|eot_id|>'], 'max_input_tokens': 1024, 'best_of': 8, 'max_output_tokens': 64}
model_name: nemo_base_1
model_repo: arushimgupta/output
status: torndown
timestamp: 2024-09-27T22:44:25+00:00
Resubmit model
Shutdown handler not registered because Python interpreter is not running in the main thread
run pipeline %s
run pipeline stage %s
Running pipeline stage MKMLizer
Starting job with name arushimgupta-output-v4-mkmlizer
Waiting for job on arushimgupta-output-v4-mkmlizer to finish
arushimgupta-output-v4-mkmlizer: ╔═════════════════════════════════════════════════════════════════════╗
arushimgupta-output-v4-mkmlizer: ║     _____            __           __                                ║
arushimgupta-output-v4-mkmlizer: ║    / _/ /_ ___    __/ /  ___ ___ / /                                ║
arushimgupta-output-v4-mkmlizer: ║   / _/ / // / |/|/ / _ \/ -_) -_) /                                 ║
arushimgupta-output-v4-mkmlizer: ║  /_//_/\_, /|__,__/_//_/\__/\__/_/                                  ║
arushimgupta-output-v4-mkmlizer: ║       /___/                                                         ║
arushimgupta-output-v4-mkmlizer: ║                                                                     ║
arushimgupta-output-v4-mkmlizer: ║  Version: 0.11.12                                                   ║
arushimgupta-output-v4-mkmlizer: ║  Copyright 2023 MK ONE TECHNOLOGIES Inc.                            ║
arushimgupta-output-v4-mkmlizer: ║  https://mk1.ai                                                     ║
arushimgupta-output-v4-mkmlizer: ║                                                                     ║
arushimgupta-output-v4-mkmlizer: ║  The license key for the current software has been verified as      ║
arushimgupta-output-v4-mkmlizer: ║  belonging to:                                                      ║
arushimgupta-output-v4-mkmlizer: ║                                                                     ║
arushimgupta-output-v4-mkmlizer: ║  Chai Research Corp.                                                ║
arushimgupta-output-v4-mkmlizer: ║  Account ID: 7997a29f-0ceb-4cc7-9adf-840c57b4ae6f                   ║
arushimgupta-output-v4-mkmlizer: ║  Expiration: 2024-10-15 23:59:59                                    ║
arushimgupta-output-v4-mkmlizer: ║                                                                     ║
arushimgupta-output-v4-mkmlizer: ╚═════════════════════════════════════════════════════════════════════╝
arushimgupta-output-v4-mkmlizer: Downloaded to shared memory in 28.078s
arushimgupta-output-v4-mkmlizer: quantizing model to /dev/shm/model_cache, profile:s0, folder:/tmp/tmp5ql2_9vp, device:0
arushimgupta-output-v4-mkmlizer: Saving flywheel model at /dev/shm/model_cache
arushimgupta-output-v4-mkmlizer: quantized model in 35.711s
arushimgupta-output-v4-mkmlizer: Processed model arushimgupta/output in 63.789s
arushimgupta-output-v4-mkmlizer: creating bucket guanaco-mkml-models
arushimgupta-output-v4-mkmlizer: Bucket 's3://guanaco-mkml-models/' created
arushimgupta-output-v4-mkmlizer: uploading /dev/shm/model_cache to s3://guanaco-mkml-models/arushimgupta-output-v4
arushimgupta-output-v4-mkmlizer: cp /dev/shm/model_cache/config.json s3://guanaco-mkml-models/arushimgupta-output-v4/config.json
arushimgupta-output-v4-mkmlizer: cp /dev/shm/model_cache/special_tokens_map.json s3://guanaco-mkml-models/arushimgupta-output-v4/special_tokens_map.json
arushimgupta-output-v4-mkmlizer: cp /dev/shm/model_cache/tokenizer_config.json s3://guanaco-mkml-models/arushimgupta-output-v4/tokenizer_config.json
arushimgupta-output-v4-mkmlizer: cp /dev/shm/model_cache/tokenizer.json s3://guanaco-mkml-models/arushimgupta-output-v4/tokenizer.json
arushimgupta-output-v4-mkmlizer: cp /dev/shm/model_cache/flywheel_model.0.safetensors s3://guanaco-mkml-models/arushimgupta-output-v4/flywheel_model.0.safetensors
arushimgupta-output-v4-mkmlizer: 
Loading 0:   0%|          | 0/363 [00:00<?, ?it/s]
Loading 0:   1%|▏         | 5/363 [00:00<00:12, 28.29it/s]
Loading 0:   4%|▎         | 13/363 [00:00<00:07, 49.85it/s]
Loading 0:   5%|▌         | 19/363 [00:00<00:07, 48.15it/s]
Loading 0:   7%|▋         | 25/363 [00:00<00:06, 49.63it/s]
Loading 0:   9%|▊         | 31/363 [00:00<00:06, 52.31it/s]
Loading 0:  10%|█         | 37/363 [00:00<00:06, 49.93it/s]
Loading 0:  12%|█▏        | 43/363 [00:00<00:06, 50.45it/s]
Loading 0:  13%|█▎        | 49/363 [00:00<00:05, 52.65it/s]
Loading 0:  15%|█▌        | 55/363 [00:01<00:06, 49.64it/s]
Loading 0:  17%|█▋        | 61/363 [00:01<00:08, 35.84it/s]
Loading 0:  18%|█▊        | 66/363 [00:01<00:08, 36.02it/s]
Loading 0:  20%|█▉        | 72/363 [00:01<00:07, 40.60it/s]
Loading 0:  21%|██▏       | 78/363 [00:01<00:06, 42.06it/s]
Loading 0:  23%|██▎       | 83/363 [00:01<00:06, 42.76it/s]
Loading 0:  25%|██▍       | 90/363 [00:01<00:05, 47.84it/s]
Loading 0:  26%|██▋       | 96/363 [00:02<00:05, 46.39it/s]
Loading 0:  28%|██▊       | 101/363 [00:02<00:05, 45.34it/s]
Loading 0:  30%|██▉       | 108/363 [00:02<00:04, 51.43it/s]
Loading 0:  31%|███▏      | 114/363 [00:02<00:05, 45.08it/s]
Loading 0:  33%|███▎      | 119/363 [00:02<00:05, 41.53it/s]
Loading 0:  35%|███▍      | 126/363 [00:02<00:05, 46.28it/s]
Loading 0:  36%|███▋      | 132/363 [00:02<00:05, 44.58it/s]
Loading 0:  38%|███▊      | 137/363 [00:03<00:05, 43.90it/s]
Loading 0:  39%|███▉      | 142/363 [00:03<00:06, 33.77it/s]
Loading 0:  40%|████      | 146/363 [00:03<00:06, 33.98it/s]
Loading 0:  41%|████▏     | 150/363 [00:03<00:06, 32.87it/s]
Loading 0:  43%|████▎     | 157/363 [00:03<00:05, 40.24it/s]
Loading 0:  45%|████▍     | 163/363 [00:03<00:04, 40.83it/s]
Loading 0:  46%|████▋     | 168/363 [00:03<00:04, 40.28it/s]
Loading 0:  48%|████▊     | 174/363 [00:04<00:04, 44.26it/s]
Loading 0:  49%|████▉     | 179/363 [00:04<00:04, 43.41it/s]
Loading 0:  51%|█████     | 184/363 [00:04<00:04, 44.59it/s]
Loading 0:  52%|█████▏    | 190/363 [00:04<00:03, 43.87it/s]
Loading 0:  54%|█████▎    | 195/363 [00:04<00:03, 42.91it/s]
Loading 0:  56%|█████▌    | 202/363 [00:04<00:03, 47.39it/s]
Loading 0:  57%|█████▋    | 208/363 [00:04<00:03, 44.66it/s]
Loading 0:  59%|█████▊    | 213/363 [00:04<00:03, 42.48it/s]
Loading 0:  60%|██████    | 218/363 [00:05<00:03, 43.49it/s]
Loading 0:  61%|██████▏   | 223/363 [00:05<00:04, 32.88it/s]
Loading 0:  63%|██████▎   | 227/363 [00:05<00:04, 33.24it/s]
Loading 0:  64%|██████▎   | 231/363 [00:05<00:04, 31.87it/s]
Loading 0:  66%|██████▌   | 238/363 [00:05<00:03, 39.48it/s]
Loading 0:  67%|██████▋   | 244/363 [00:05<00:02, 40.11it/s]
Loading 0:  69%|██████▊   | 249/363 [00:05<00:02, 40.04it/s]
Loading 0:  71%|███████   | 256/363 [00:06<00:02, 45.22it/s]
Loading 0:  72%|███████▏  | 262/363 [00:06<00:02, 43.40it/s]
Loading 0:  74%|███████▎  | 267/363 [00:06<00:02, 39.96it/s]
Loading 0:  75%|███████▌  | 273/363 [00:06<00:02, 43.06it/s]
Loading 0:  77%|███████▋  | 278/363 [00:06<00:01, 42.91it/s]
Loading 0:  78%|███████▊  | 283/363 [00:06<00:01, 40.38it/s]
Loading 0:  80%|███████▉  | 289/363 [00:06<00:01, 41.37it/s]
Loading 0:  81%|████████  | 294/363 [00:06<00:01, 40.97it/s]
Loading 0:  83%|████████▎ | 301/363 [00:07<00:01, 47.43it/s]
Loading 0:  84%|████████▍ | 306/363 [00:13<00:21,  2.66it/s]
Loading 0:  85%|████████▌ | 310/363 [00:13<00:15,  3.39it/s]
Loading 0:  87%|████████▋ | 315/363 [00:14<00:10,  4.68it/s]
Loading 0:  88%|████████▊ | 320/363 [00:14<00:06,  6.39it/s]
Loading 0:  90%|████████▉ | 326/363 [00:14<00:04,  8.91it/s]
Loading 0:  91%|█████████ | 331/363 [00:14<00:02, 11.55it/s]
Loading 0:  93%|█████████▎| 338/363 [00:14<00:01, 16.52it/s]
Loading 0:  95%|█████████▍| 344/363 [00:14<00:00, 19.83it/s]
Loading 0:  96%|█████████▌| 349/363 [00:14<00:00, 22.54it/s]
Loading 0:  98%|█████████▊| 356/363 [00:14<00:00, 28.87it/s]
Loading 0: 100%|█████████▉| 362/363 [00:15<00:00, 32.08it/s]
                                                            
Job arushimgupta-output-v4-mkmlizer completed after 214.01s with status: succeeded
Stopping job with name arushimgupta-output-v4-mkmlizer
Pipeline stage MKMLizer completed in 214.91s
run pipeline stage %s
Running pipeline stage MKMLTemplater
Pipeline stage MKMLTemplater completed in 0.09s
run pipeline stage %s
Running pipeline stage MKMLDeployer
Creating inference service arushimgupta-output-v4
Waiting for inference service arushimgupta-output-v4 to be ready
Inference service arushimgupta-output-v4 ready after 230.52767896652222s
Pipeline stage MKMLDeployer completed in 230.87s
run pipeline stage %s
Running pipeline stage StressChecker
Received healthy response to inference request in 3.3905792236328125s
Received healthy response to inference request in 2.3421008586883545s
HTTPSConnectionPool(host='guanaco-submitter.chai-research.com', port=443): Read timed out. (read timeout=20)
Received unhealthy response to inference request!
Received healthy response to inference request in 2.763214111328125s
Received healthy response to inference request in 2.2464165687561035s
5 requests
1 failed requests
5th percentile: 2.265553426742554
10th percentile: 2.284690284729004
20th percentile: 2.322964000701904
30th percentile: 2.4263235092163087
40th percentile: 2.5947688102722166
50th percentile: 2.763214111328125
60th percentile: 3.01416015625
70th percentile: 3.265106201171875
80th percentile: 6.725981569290164
90th percentile: 13.396786260604859
95th percentile: 16.732188606262206
99th percentile: 19.400510482788086
mean time: 6.16198034286499
%s, retrying in %s seconds...
Received healthy response to inference request in 4.278409957885742s
Received healthy response to inference request in 1.6667373180389404s
HTTPSConnectionPool(host='guanaco-submitter.chai-research.com', port=443): Read timed out. (read timeout=20)
Received unhealthy response to inference request!
Received healthy response to inference request in 12.119145393371582s
Received healthy response to inference request in 3.797473430633545s
5 requests
1 failed requests
5th percentile: 2.0928845405578613
10th percentile: 2.5190317630767822
20th percentile: 3.371326208114624
30th percentile: 3.8936607360839846
40th percentile: 4.086035346984863
50th percentile: 4.278409957885742
60th percentile: 7.414704132080077
70th percentile: 10.550998306274412
80th percentile: 13.711038303375245
90th percentile: 16.89482412338257
95th percentile: 18.486717033386228
99th percentile: 19.76023136138916
mean time: 8.38807520866394
%s, retrying in %s seconds...
Received healthy response to inference request in 2.5425944328308105s
Received healthy response to inference request in 1.6654884815216064s
Received healthy response to inference request in 8.040712833404541s
HTTPSConnectionPool(host='guanaco-submitter.chai-research.com', port=443): Read timed out. (read timeout=20)
Received unhealthy response to inference request!
Received healthy response to inference request in 3.65682053565979s
5 requests
1 failed requests
5th percentile: 1.8409096717834472
10th percentile: 2.016330862045288
20th percentile: 2.3671732425689695
30th percentile: 2.7654396533966064
40th percentile: 3.2111300945281984
50th percentile: 3.65682053565979
60th percentile: 5.41037745475769
70th percentile: 7.16393437385559
80th percentile: 10.448055553436282
90th percentile: 15.262740993499758
95th percentile: 17.670083713531493
99th percentile: 19.595957889556885
mean time: 7.196608543395996
clean up pipeline due to error=DeploymentChecksError('Unacceptable number of predict errors: 20.0%')
Shutdown handler de-registered
arushimgupta-output_v4 status is now failed due to DeploymentManager action
arushimgupta-output_v4 status is now torndown due to DeploymentManager action