rica40325-feedback-11

submission_id: rica40325-feedback-11_v1
developer_uid: rica40325
formatter: {'memory_template': "{bot_name}'s Persona: {memory}\n####\n", 'prompt_template': '{prompt}\n<START>\n', 'bot_template': '{bot_name}: {message}\n', 'user_template': '{user_name}: {message}\n', 'response_template': '{bot_name}:', 'truncate_by_message': False}
generation_params: {'temperature': 1.0, 'top_p': 1.0, 'min_p': 0.0, 'top_k': 40, 'presence_penalty': 0.0, 'frequency_penalty': 0.0, 'stopping_words': ['\n'], 'max_input_tokens': 512, 'best_of': 16, 'max_output_tokens': 64}
model_name: rica40325-feedback-11_v1
model_repo: rica40325/feedback-11
status: torndown
timestamp: 2024-09-12T10:02:39+00:00
Resubmit model
Shutdown handler not registered because Python interpreter is not running in the main thread
run pipeline %s
run pipeline stage %s
Running pipeline stage MKMLizer
Starting job with name rica40325-feedback-11-v1-mkmlizer
Waiting for job on rica40325-feedback-11-v1-mkmlizer to finish
rica40325-feedback-11-v1-mkmlizer: ╔═════════════════════════════════════════════════════════════════════╗
rica40325-feedback-11-v1-mkmlizer: ║     _____            __           __                                ║
rica40325-feedback-11-v1-mkmlizer: ║    / _/ /_ ___    __/ /  ___ ___ / /                                ║
rica40325-feedback-11-v1-mkmlizer: ║   / _/ / // / |/|/ / _ \/ -_) -_) /                                 ║
rica40325-feedback-11-v1-mkmlizer: ║  /_//_/\_, /|__,__/_//_/\__/\__/_/                                  ║
rica40325-feedback-11-v1-mkmlizer: ║       /___/                                                         ║
rica40325-feedback-11-v1-mkmlizer: ║                                                                     ║
rica40325-feedback-11-v1-mkmlizer: ║  Version: 0.10.1                                                    ║
rica40325-feedback-11-v1-mkmlizer: ║  Copyright 2023 MK ONE TECHNOLOGIES Inc.                            ║
rica40325-feedback-11-v1-mkmlizer: ║  https://mk1.ai                                                     ║
rica40325-feedback-11-v1-mkmlizer: ║                                                                     ║
rica40325-feedback-11-v1-mkmlizer: ║  The license key for the current software has been verified as      ║
rica40325-feedback-11-v1-mkmlizer: ║  belonging to:                                                      ║
rica40325-feedback-11-v1-mkmlizer: ║                                                                     ║
rica40325-feedback-11-v1-mkmlizer: ║  Chai Research Corp.                                                ║
rica40325-feedback-11-v1-mkmlizer: ║  Account ID: 7997a29f-0ceb-4cc7-9adf-840c57b4ae6f                   ║
rica40325-feedback-11-v1-mkmlizer: ║  Expiration: 2024-10-15 23:59:59                                    ║
rica40325-feedback-11-v1-mkmlizer: ║                                                                     ║
rica40325-feedback-11-v1-mkmlizer: ╚═════════════════════════════════════════════════════════════════════╝
rica40325-feedback-11-v1-mkmlizer: Downloaded to shared memory in 60.456s
rica40325-feedback-11-v1-mkmlizer: quantizing model to /dev/shm/model_cache, profile:s0, folder:/tmp/tmp67240rcj, device:0
rica40325-feedback-11-v1-mkmlizer: Saving flywheel model at /dev/shm/model_cache
rica40325-feedback-11-v1-mkmlizer: quantized model in 29.324s
rica40325-feedback-11-v1-mkmlizer: Processed model rica40325/feedback-11 in 89.780s
rica40325-feedback-11-v1-mkmlizer: creating bucket guanaco-mkml-models
rica40325-feedback-11-v1-mkmlizer: Bucket 's3://guanaco-mkml-models/' created
rica40325-feedback-11-v1-mkmlizer: uploading /dev/shm/model_cache to s3://guanaco-mkml-models/rica40325-feedback-11-v1
rica40325-feedback-11-v1-mkmlizer: cp /dev/shm/model_cache/config.json s3://guanaco-mkml-models/rica40325-feedback-11-v1/config.json
rica40325-feedback-11-v1-mkmlizer: cp /dev/shm/model_cache/special_tokens_map.json s3://guanaco-mkml-models/rica40325-feedback-11-v1/special_tokens_map.json
rica40325-feedback-11-v1-mkmlizer: cp /dev/shm/model_cache/tokenizer_config.json s3://guanaco-mkml-models/rica40325-feedback-11-v1/tokenizer_config.json
rica40325-feedback-11-v1-mkmlizer: cp /dev/shm/model_cache/tokenizer.json s3://guanaco-mkml-models/rica40325-feedback-11-v1/tokenizer.json
rica40325-feedback-11-v1-mkmlizer: cp /dev/shm/model_cache/flywheel_model.0.safetensors s3://guanaco-mkml-models/rica40325-feedback-11-v1/flywheel_model.0.safetensors
rica40325-feedback-11-v1-mkmlizer: 
Loading 0:   0%|          | 0/291 [00:00<?, ?it/s]
Loading 0:   2%|▏         | 5/291 [00:00<00:11, 25.17it/s]
Loading 0:   4%|▍         | 12/291 [00:00<00:08, 33.93it/s]
Loading 0:   5%|▌         | 16/291 [00:00<00:08, 31.27it/s]
Loading 0:   7%|▋         | 21/291 [00:00<00:07, 34.49it/s]
Loading 0:   9%|▊         | 25/291 [00:00<00:08, 32.18it/s]
Loading 0:  10%|▉         | 29/291 [00:00<00:07, 33.94it/s]
Loading 0:  11%|█▏        | 33/291 [00:01<00:11, 23.08it/s]
Loading 0:  12%|█▏        | 36/291 [00:01<00:12, 21.06it/s]
Loading 0:  14%|█▍        | 41/291 [00:01<00:10, 23.14it/s]
Loading 0:  16%|█▋        | 48/291 [00:01<00:08, 30.27it/s]
Loading 0:  18%|█▊        | 52/291 [00:01<00:08, 29.45it/s]
Loading 0:  20%|█▉        | 57/291 [00:01<00:07, 31.52it/s]
Loading 0:  21%|██        | 61/291 [00:02<00:07, 30.92it/s]
Loading 0:  23%|██▎       | 66/291 [00:02<00:06, 33.54it/s]
Loading 0:  24%|██▍       | 70/291 [00:02<00:06, 32.41it/s]
Loading 0:  25%|██▌       | 74/291 [00:02<00:06, 33.16it/s]
Loading 0:  27%|██▋       | 78/291 [00:02<00:06, 33.43it/s]
Loading 0:  28%|██▊       | 82/291 [00:02<00:08, 23.23it/s]
Loading 0:  29%|██▉       | 85/291 [00:02<00:08, 24.35it/s]
Loading 0:  31%|███       | 90/291 [00:03<00:07, 27.79it/s]
Loading 0:  32%|███▏      | 94/291 [00:03<00:07, 28.14it/s]
Loading 0:  34%|███▍      | 99/291 [00:03<00:06, 31.97it/s]
Loading 0:  35%|███▌      | 103/291 [00:03<00:06, 30.35it/s]
Loading 0:  37%|███▋      | 108/291 [00:03<00:05, 33.23it/s]
Loading 0:  38%|███▊      | 112/291 [00:03<00:05, 31.08it/s]
Loading 0:  40%|███▉      | 116/291 [00:03<00:05, 31.23it/s]
Loading 0:  42%|████▏     | 122/291 [00:04<00:04, 35.67it/s]
Loading 0:  44%|████▎     | 127/291 [00:04<00:04, 32.98it/s]
Loading 0:  46%|████▌     | 133/291 [00:04<00:05, 29.66it/s]
Loading 0:  47%|████▋     | 137/291 [00:04<00:05, 29.93it/s]
Loading 0:  48%|████▊     | 141/291 [00:04<00:05, 27.92it/s]
Loading 0:  51%|█████     | 147/291 [00:04<00:04, 33.07it/s]
Loading 0:  52%|█████▏    | 151/291 [00:05<00:04, 31.41it/s]
Loading 0:  54%|█████▎    | 156/291 [00:05<00:04, 32.96it/s]
Loading 0:  55%|█████▍    | 160/291 [00:05<00:04, 30.99it/s]
Loading 0:  57%|█████▋    | 165/291 [00:05<00:03, 32.85it/s]
Loading 0:  58%|█████▊    | 169/291 [00:05<00:03, 31.41it/s]
Loading 0:  60%|█████▉    | 174/291 [00:05<00:03, 33.90it/s]
Loading 0:  61%|██████    | 178/291 [00:05<00:03, 31.83it/s]
Loading 0:  63%|██████▎   | 183/291 [00:05<00:03, 35.91it/s]
Loading 0:  64%|██████▍   | 187/291 [00:06<00:03, 26.14it/s]
Loading 0:  66%|██████▌   | 191/291 [00:06<00:03, 27.13it/s]
Loading 0:  67%|██████▋   | 195/291 [00:06<00:03, 26.07it/s]
Loading 0:  69%|██████▉   | 201/291 [00:06<00:02, 31.48it/s]
Loading 0:  70%|███████   | 205/291 [00:06<00:02, 30.81it/s]
Loading 0:  72%|███████▏  | 210/291 [00:06<00:02, 33.29it/s]
Loading 0:  74%|███████▎  | 214/291 [00:07<00:02, 31.51it/s]
Loading 0:  75%|███████▌  | 219/291 [00:07<00:02, 34.18it/s]
Loading 0:  77%|███████▋  | 223/291 [00:07<00:02, 31.85it/s]
Loading 0:  78%|███████▊  | 227/291 [00:07<00:02, 31.64it/s]
Loading 0:  79%|███████▉  | 231/291 [00:07<00:01, 31.54it/s]
Loading 0:  81%|████████  | 235/291 [00:07<00:02, 24.37it/s]
Loading 0:  82%|████████▏ | 239/291 [00:08<00:02, 24.43it/s]
Loading 0:  85%|████████▍ | 246/291 [00:08<00:01, 32.17it/s]
Loading 0:  86%|████████▌ | 250/291 [00:08<00:01, 30.59it/s]
Loading 0:  88%|████████▊ | 255/291 [00:08<00:01, 33.14it/s]
Loading 0:  89%|████████▉ | 259/291 [00:08<00:00, 32.39it/s]
Loading 0:  91%|█████████ | 264/291 [00:08<00:00, 35.28it/s]
Loading 0:  92%|█████████▏| 268/291 [00:08<00:00, 33.97it/s]
Loading 0:  94%|█████████▍| 273/291 [00:08<00:00, 36.50it/s]
Loading 0:  95%|█████████▌| 277/291 [00:09<00:00, 34.18it/s]
Loading 0:  97%|█████████▋| 281/291 [00:09<00:00, 32.97it/s]
Loading 0:  98%|█████████▊| 285/291 [00:09<00:00, 34.61it/s]
Loading 0:  99%|█████████▉| 289/291 [00:14<00:00,  2.36it/s]
                                                            
Job rica40325-feedback-11-v1-mkmlizer completed after 113.95s with status: succeeded
Stopping job with name rica40325-feedback-11-v1-mkmlizer
Pipeline stage MKMLizer completed in 114.93s
run pipeline stage %s
Running pipeline stage MKMLTemplater
Pipeline stage MKMLTemplater completed in 0.10s
run pipeline stage %s
Running pipeline stage MKMLDeployer
Creating inference service rica40325-feedback-11-v1
Waiting for inference service rica40325-feedback-11-v1 to be ready
Failed to get response for submission chaiml-llama-8b-pairwis_8189_v19: HTTPConnectionPool(host='zonemercy-virgo-edit-v1-1e5-v12-predictor.tenant-chaiml-guanaco.k2.chaiverse.com', port=80): Max retries exceeded with url: /v1/models/GPT-J-6B-lit-v2:predict (Caused by ConnectTimeoutError(<urllib3.connection.HTTPConnection object at 0x7fe70857b890>, 'Connection to zonemercy-virgo-edit-v1-1e5-v12-predictor.tenant-chaiml-guanaco.k2.chaiverse.com timed out. (connect timeout=None)'))
Inference service rica40325-feedback-11-v1 ready after 170.7097773551941s
Pipeline stage MKMLDeployer completed in 171.49s
run pipeline stage %s
Running pipeline stage StressChecker
Received healthy response to inference request in 8.11637830734253s
Received healthy response to inference request in 4.163012742996216s
Received healthy response to inference request in 1.475658655166626s
Failed to get response for submission zonemercy-virgo-edit-v1-1e5_v12: HTTPConnectionPool(host='zonemercy-virgo-edit-v1-1e5-v12-predictor.tenant-chaiml-guanaco.k2.chaiverse.com', port=80): Max retries exceeded with url: /v1/models/GPT-J-6B-lit-v2:predict (Caused by ConnectTimeoutError(<urllib3.connection.HTTPConnection object at 0x7fe697a4d6d0>, 'Connection to zonemercy-virgo-edit-v1-1e5-v12-predictor.tenant-chaiml-guanaco.k2.chaiverse.com timed out. (connect timeout=None)'))
Received healthy response to inference request in 4.301113843917847s
Received healthy response to inference request in 8.979495525360107s
5 requests
0 failed requests
5th percentile: 2.013129472732544
10th percentile: 2.550600290298462
20th percentile: 3.625541925430298
30th percentile: 4.190632963180542
40th percentile: 4.2458734035491945
50th percentile: 4.301113843917847
60th percentile: 5.827219629287719
70th percentile: 7.353325414657592
80th percentile: 8.289001750946046
90th percentile: 8.634248638153077
95th percentile: 8.806872081756591
99th percentile: 8.944970836639405
mean time: 5.407131814956665
%s, retrying in %s seconds...
Received healthy response to inference request in 1.9218523502349854s
Received healthy response to inference request in 1.718106746673584s
Received healthy response to inference request in 6.704678058624268s
Received healthy response to inference request in 1.906466007232666s
Received healthy response to inference request in 7.896427154541016s
5 requests
0 failed requests
5th percentile: 1.7557785987854004
10th percentile: 1.7934504508972169
20th percentile: 1.8687941551208496
30th percentile: 1.9095432758331299
40th percentile: 1.9156978130340576
50th percentile: 1.9218523502349854
60th percentile: 3.834982633590698
70th percentile: 5.74811291694641
80th percentile: 6.9430278778076175
90th percentile: 7.419727516174317
95th percentile: 7.658077335357666
99th percentile: 7.848757190704346
mean time: 4.029506063461303
%s, retrying in %s seconds...
Received healthy response to inference request in 8.225492715835571s
Received healthy response to inference request in 7.048065185546875s
Received healthy response to inference request in 6.5204432010650635s
Received healthy response to inference request in 9.307820320129395s
Received healthy response to inference request in 3.549027681350708s
5 requests
0 failed requests
5th percentile: 4.143310785293579
10th percentile: 4.73759388923645
20th percentile: 5.926160097122192
30th percentile: 6.625967597961425
40th percentile: 6.83701639175415
50th percentile: 7.048065185546875
60th percentile: 7.519036197662354
70th percentile: 7.990007209777832
80th percentile: 8.441958236694337
90th percentile: 8.874889278411866
95th percentile: 9.09135479927063
99th percentile: 9.264527215957642
mean time: 6.9301698207855225
clean up pipeline due to error=%s
Shutdown handler de-registered
rica40325-feedback-11_v1 status is now failed due to DeploymentManager action
rica40325-feedback-11_v1 status is now torndown due to DeploymentManager action