submission_id: juvi21-uwu-8b_v2
developer_uid: juvi21
formatter: {'memory_template': "<|begin_of_text|><|start_header_id|>system<|end_header_id|>\n\n{bot_name}'s Persona: {memory}\n\n", 'prompt_template': '{prompt}<|eot_id|>', 'bot_template': '<|start_header_id|>assistant<|end_header_id|>\n\n{bot_name}: {message}<|eot_id|>', 'user_template': '<|start_header_id|>user<|end_header_id|>\n\n{user_name}: {message}<|eot_id|>', 'response_template': '<|start_header_id|>assistant<|end_header_id|>\n\n{bot_name}:', 'truncate_by_message': False}
generation_params: {'temperature': 1.21, 'top_p': 1.0, 'min_p': 0.1, 'top_k': 40, 'presence_penalty': 0.0, 'frequency_penalty': 0.0, 'stopping_words': ['\n', '<|end_header_id|>,', '<|eot_id|>,', '\n\n{user_name}', 'You:', '\n\n'], 'max_input_tokens': 512, 'best_of': 16, 'max_output_tokens': 64, 'reward_max_token_input': 256}
model_name: uwu-8b
model_repo: juvi21/UwU-8B
reward_formatter: {'bot_template': '{bot_name}: {message}\n', 'memory_template': "{bot_name}'s Persona: {memory}\n####\n", 'prompt_template': '{prompt}\n<START>\n', 'response_template': '{bot_name}:', 'truncate_by_message': False, 'user_template': '{user_name}: {message}\n'}
reward_repo: Jellywibble/gpt2_xl_pairwise_89m_step_347634
status: torndown
timestamp: 2024-08-03T14:48:33+00:00
Resubmit model
Running pipeline stage MKMLizer
Starting job with name juvi21-uwu-8b-v2-mkmlizer
Waiting for job on juvi21-uwu-8b-v2-mkmlizer to finish
juvi21-uwu-8b-v2-mkmlizer: ╔═════════════════════════════════════════════════════════════════════╗
juvi21-uwu-8b-v2-mkmlizer: ║     _____            __           __                                ║
juvi21-uwu-8b-v2-mkmlizer: ║    / _/ /_ ___    __/ /  ___ ___ / /                                ║
juvi21-uwu-8b-v2-mkmlizer: ║   / _/ / // / |/|/ / _ \/ -_) -_) /                                 ║
juvi21-uwu-8b-v2-mkmlizer: ║  /_//_/\_, /|__,__/_//_/\__/\__/_/                                  ║
juvi21-uwu-8b-v2-mkmlizer: ║       /___/                                                         ║
juvi21-uwu-8b-v2-mkmlizer: ║                                                                     ║
juvi21-uwu-8b-v2-mkmlizer: ║  Version: 0.9.9                                                     ║
juvi21-uwu-8b-v2-mkmlizer: ║  Copyright 2023 MK ONE TECHNOLOGIES Inc.                            ║
juvi21-uwu-8b-v2-mkmlizer: ║  https://mk1.ai                                                     ║
juvi21-uwu-8b-v2-mkmlizer: ║                                                                     ║
juvi21-uwu-8b-v2-mkmlizer: ║  The license key for the current software has been verified as      ║
juvi21-uwu-8b-v2-mkmlizer: ║  belonging to:                                                      ║
juvi21-uwu-8b-v2-mkmlizer: ║                                                                     ║
juvi21-uwu-8b-v2-mkmlizer: ║  Chai Research Corp.                                                ║
juvi21-uwu-8b-v2-mkmlizer: ║  Account ID: 7997a29f-0ceb-4cc7-9adf-840c57b4ae6f                   ║
juvi21-uwu-8b-v2-mkmlizer: ║  Expiration: 2024-10-15 23:59:59                                    ║
juvi21-uwu-8b-v2-mkmlizer: ║                                                                     ║
juvi21-uwu-8b-v2-mkmlizer: ╚═════════════════════════════════════════════════════════════════════╝
juvi21-uwu-8b-v2-mkmlizer: Downloaded to shared memory in 22.086s
juvi21-uwu-8b-v2-mkmlizer: quantizing model to /dev/shm/model_cache, profile:s0, folder:/tmp/tmp36f0r3x1, device:0
juvi21-uwu-8b-v2-mkmlizer: Saving flywheel model at /dev/shm/model_cache
juvi21-uwu-8b-v2-mkmlizer: quantized model in 26.476s
juvi21-uwu-8b-v2-mkmlizer: Processed model juvi21/UwU-8B in 48.563s
juvi21-uwu-8b-v2-mkmlizer: creating bucket guanaco-mkml-models
juvi21-uwu-8b-v2-mkmlizer: Bucket 's3://guanaco-mkml-models/' created
juvi21-uwu-8b-v2-mkmlizer: uploading /dev/shm/model_cache to s3://guanaco-mkml-models/juvi21-uwu-8b-v2
juvi21-uwu-8b-v2-mkmlizer: cp /dev/shm/model_cache/special_tokens_map.json s3://guanaco-mkml-models/juvi21-uwu-8b-v2/special_tokens_map.json
juvi21-uwu-8b-v2-mkmlizer: cp /dev/shm/model_cache/config.json s3://guanaco-mkml-models/juvi21-uwu-8b-v2/config.json
juvi21-uwu-8b-v2-mkmlizer: cp /dev/shm/model_cache/tokenizer_config.json s3://guanaco-mkml-models/juvi21-uwu-8b-v2/tokenizer_config.json
juvi21-uwu-8b-v2-mkmlizer: cp /dev/shm/model_cache/tokenizer.json s3://guanaco-mkml-models/juvi21-uwu-8b-v2/tokenizer.json
juvi21-uwu-8b-v2-mkmlizer: 
Loading 0:   0%|          | 0/291 [00:00<?, ?it/s]
Loading 0:   1%|          | 2/291 [00:04<11:29,  2.38s/it]
Loading 0:   2%|▏         | 6/291 [00:04<03:03,  1.55it/s]
Loading 0:   4%|▍         | 13/291 [00:05<01:05,  4.24it/s]
Loading 0:   7%|▋         | 19/291 [00:05<00:37,  7.21it/s]
Loading 0:   8%|▊         | 24/291 [00:05<00:27,  9.77it/s]
Loading 0:  11%|█         | 32/291 [00:05<00:16, 15.89it/s]
Loading 0:  13%|█▎        | 38/291 [00:05<00:12, 19.47it/s]
Loading 0:  15%|█▍        | 43/291 [00:05<00:10, 23.14it/s]
Loading 0:  17%|█▋        | 50/291 [00:05<00:08, 30.05it/s]
Loading 0:  19%|█▉        | 56/291 [00:05<00:07, 32.33it/s]
Loading 0:  21%|██        | 61/291 [00:06<00:08, 25.93it/s]
Loading 0:  23%|██▎       | 68/291 [00:06<00:06, 32.53it/s]
Loading 0:  25%|██▌       | 74/291 [00:06<00:06, 34.01it/s]
Loading 0:  27%|██▋       | 79/291 [00:06<00:05, 35.64it/s]
Loading 0:  30%|██▉       | 86/291 [00:06<00:04, 41.39it/s]
Loading 0:  32%|███▏      | 92/291 [00:06<00:04, 40.66it/s]
Loading 0:  33%|███▎      | 97/291 [00:06<00:04, 41.12it/s]
Loading 0:  36%|███▌      | 104/291 [00:07<00:04, 46.70it/s]
Loading 0:  38%|███▊      | 110/291 [00:07<00:04, 43.28it/s]
Loading 0:  40%|███▉      | 115/291 [00:07<00:04, 43.56it/s]
Loading 0:  42%|████▏     | 122/291 [00:07<00:03, 48.32it/s]
Loading 0:  44%|████▍     | 128/291 [00:07<00:03, 45.41it/s]
Loading 0:  46%|████▌     | 133/291 [00:07<00:03, 43.98it/s]
Loading 0:  48%|████▊     | 140/291 [00:07<00:03, 48.38it/s]
Loading 0:  50%|████▉     | 145/291 [00:07<00:03, 47.81it/s]
Loading 0:  52%|█████▏    | 150/291 [00:08<00:03, 40.42it/s]
Loading 0:  54%|█████▍    | 157/291 [00:08<00:02, 46.67it/s]
Loading 0:  56%|█████▌    | 162/291 [00:08<00:02, 46.12it/s]
Loading 0:  57%|█████▋    | 167/291 [00:08<00:03, 31.94it/s]
Loading 0:  59%|█████▉    | 173/291 [00:08<00:03, 33.56it/s]
Loading 0:  61%|██████    | 177/291 [00:08<00:03, 34.00it/s]
Loading 0:  64%|██████▎   | 185/291 [00:09<00:02, 43.04it/s]
Loading 0:  66%|██████▌   | 191/291 [00:09<00:02, 42.43it/s]
Loading 0:  67%|██████▋   | 196/291 [00:09<00:02, 43.11it/s]
Loading 0:  70%|██████▉   | 203/291 [00:09<00:01, 49.09it/s]
Loading 0:  72%|███████▏  | 209/291 [00:09<00:01, 44.99it/s]
Loading 0:  74%|███████▎  | 214/291 [00:09<00:01, 42.69it/s]
Loading 0:  76%|███████▌  | 221/291 [00:09<00:01, 48.00it/s]
Loading 0:  78%|███████▊  | 227/291 [00:09<00:01, 44.57it/s]
Loading 0:  80%|███████▉  | 232/291 [00:10<00:01, 43.03it/s]
Loading 0:  82%|████████▏ | 239/291 [00:10<00:01, 47.78it/s]
Loading 0:  84%|████████▍ | 245/291 [00:10<00:01, 44.76it/s]
Loading 0:  86%|████████▌ | 250/291 [00:10<00:00, 43.93it/s]
Loading 0:  88%|████████▊ | 257/291 [00:10<00:00, 48.62it/s]
Loading 0:  90%|█████████ | 263/291 [00:10<00:00, 45.06it/s]
Loading 0:  92%|█████████▏| 268/291 [00:11<00:00, 31.33it/s]
Loading 0:  94%|█████████▍| 274/291 [00:11<00:00, 36.68it/s]
Loading 0:  96%|█████████▌| 279/291 [00:11<00:00, 38.81it/s]
Loading 0:  98%|█████████▊| 284/291 [00:11<00:00, 40.49it/s]
Loading 0:  99%|█████████▉| 289/291 [00:11<00:00, 42.25it/s]
                                                            
ERROR "cp /dev/shm/model_cache/flywheel_model.0.safetensors s3://guanaco-mkml-models/juvi21-uwu-8b-v2/flywheel_model.0.safetensors": MultipartUpload: upload multipart failed upload id: 2~b2mOVnfAkB48r6MHvvYkNZ4C3jMd0ho caused by: QuotaExceeded:  status code: 403, request id: tx00000893e08446a2bc99e-0066ae4382-a42d6dcf0-default, host id:
Job juvi21-uwu-8b-v2-mkmlizer completed after 73.47s with status: failed
Stopping job with name juvi21-uwu-8b-v2-mkmlizer
%s, retrying in %s seconds...
Starting job with name juvi21-uwu-8b-v2-mkmlizer
Waiting for job on juvi21-uwu-8b-v2-mkmlizer to finish
juvi21-uwu-8b-v2-mkmlizer: ╔═════════════════════════════════════════════════════════════════════╗
juvi21-uwu-8b-v2-mkmlizer: ║     _____            __           __                                ║
juvi21-uwu-8b-v2-mkmlizer: ║    / _/ /_ ___    __/ /  ___ ___ / /                                ║
juvi21-uwu-8b-v2-mkmlizer: ║   / _/ / // / |/|/ / _ \/ -_) -_) /                                 ║
juvi21-uwu-8b-v2-mkmlizer: ║  /_//_/\_, /|__,__/_//_/\__/\__/_/                                  ║
juvi21-uwu-8b-v2-mkmlizer: ║       /___/                                                         ║
juvi21-uwu-8b-v2-mkmlizer: ║                                                                     ║
juvi21-uwu-8b-v2-mkmlizer: ║  Version: 0.9.9                                                     ║
juvi21-uwu-8b-v2-mkmlizer: ║  Copyright 2023 MK ONE TECHNOLOGIES Inc.                            ║
juvi21-uwu-8b-v2-mkmlizer: ║  https://mk1.ai                                                     ║
juvi21-uwu-8b-v2-mkmlizer: ║                                                                     ║
juvi21-uwu-8b-v2-mkmlizer: ║  The license key for the current software has been verified as      ║
juvi21-uwu-8b-v2-mkmlizer: ║  belonging to:                                                      ║
juvi21-uwu-8b-v2-mkmlizer: ║                                                                     ║
juvi21-uwu-8b-v2-mkmlizer: ║  Chai Research Corp.                                                ║
juvi21-uwu-8b-v2-mkmlizer: ║  Account ID: 7997a29f-0ceb-4cc7-9adf-840c57b4ae6f                   ║
juvi21-uwu-8b-v2-mkmlizer: ║  Expiration: 2024-10-15 23:59:59                                    ║
juvi21-uwu-8b-v2-mkmlizer: ║                                                                     ║
juvi21-uwu-8b-v2-mkmlizer: ╚═════════════════════════════════════════════════════════════════════╝
juvi21-uwu-8b-v2-mkmlizer: Downloaded to shared memory in 21.212s
juvi21-uwu-8b-v2-mkmlizer: quantizing model to /dev/shm/model_cache, profile:s0, folder:/tmp/tmpttuml2wa, device:0
juvi21-uwu-8b-v2-mkmlizer: Saving flywheel model at /dev/shm/model_cache
juvi21-uwu-8b-v2-mkmlizer: quantized model in 26.014s
juvi21-uwu-8b-v2-mkmlizer: Processed model juvi21/UwU-8B in 47.226s
juvi21-uwu-8b-v2-mkmlizer: creating bucket guanaco-mkml-models
juvi21-uwu-8b-v2-mkmlizer: Bucket 's3://guanaco-mkml-models/' created
juvi21-uwu-8b-v2-mkmlizer: uploading /dev/shm/model_cache to s3://guanaco-mkml-models/juvi21-uwu-8b-v2
juvi21-uwu-8b-v2-mkmlizer: cp /dev/shm/model_cache/config.json s3://guanaco-mkml-models/juvi21-uwu-8b-v2/config.json
juvi21-uwu-8b-v2-mkmlizer: cp /dev/shm/model_cache/special_tokens_map.json s3://guanaco-mkml-models/juvi21-uwu-8b-v2/special_tokens_map.json
juvi21-uwu-8b-v2-mkmlizer: cp /dev/shm/model_cache/tokenizer_config.json s3://guanaco-mkml-models/juvi21-uwu-8b-v2/tokenizer_config.json
juvi21-uwu-8b-v2-mkmlizer: cp /dev/shm/model_cache/tokenizer.json s3://guanaco-mkml-models/juvi21-uwu-8b-v2/tokenizer.json
juvi21-uwu-8b-v2-mkmlizer: 
Loading 0:   0%|          | 0/291 [00:00<?, ?it/s]
Loading 0:   1%|          | 2/291 [00:04<11:25,  2.37s/it]
Loading 0:   2%|▏         | 6/291 [00:04<03:02,  1.56it/s]
Loading 0:   4%|▍         | 13/291 [00:04<01:04,  4.28it/s]
Loading 0:   7%|▋         | 20/291 [00:05<00:35,  7.60it/s]
Loading 0:   9%|▊         | 25/291 [00:05<00:25, 10.46it/s]
Loading 0:  11%|█         | 32/291 [00:05<00:16, 15.76it/s]
Loading 0:  13%|█▎        | 38/291 [00:05<00:12, 19.71it/s]
Loading 0:  15%|█▍        | 43/291 [00:05<00:10, 23.47it/s]
Loading 0:  17%|█▋        | 50/291 [00:05<00:07, 30.51it/s]
Loading 0:  19%|█▉        | 56/291 [00:05<00:07, 32.84it/s]
Loading 0:  21%|██        | 61/291 [00:06<00:08, 26.62it/s]
Loading 0:  23%|██▎       | 67/291 [00:06<00:06, 32.16it/s]
Loading 0:  25%|██▍       | 72/291 [00:06<00:06, 34.33it/s]
Loading 0:  26%|██▋       | 77/291 [00:06<00:05, 37.41it/s]
Loading 0:  29%|██▊       | 83/291 [00:06<00:05, 38.40it/s]
Loading 0:  30%|███       | 88/291 [00:06<00:05, 39.76it/s]
Loading 0:  33%|███▎      | 95/291 [00:06<00:04, 46.50it/s]
Loading 0:  35%|███▍      | 101/291 [00:06<00:04, 44.65it/s]
Loading 0:  36%|███▋      | 106/291 [00:07<00:04, 44.60it/s]
Loading 0:  39%|███▉      | 113/291 [00:07<00:03, 50.36it/s]
Loading 0:  41%|████      | 119/291 [00:07<00:03, 46.84it/s]
Loading 0:  43%|████▎     | 124/291 [00:07<00:03, 46.31it/s]
Loading 0:  45%|████▌     | 131/291 [00:07<00:03, 51.12it/s]
Loading 0:  47%|████▋     | 137/291 [00:07<00:03, 46.90it/s]
Loading 0:  49%|████▉     | 142/291 [00:07<00:03, 45.10it/s]
Loading 0:  51%|█████     | 149/291 [00:07<00:02, 49.84it/s]
Loading 0:  53%|█████▎    | 155/291 [00:08<00:02, 45.68it/s]
Loading 0:  55%|█████▍    | 160/291 [00:08<00:02, 45.59it/s]
Loading 0:  57%|█████▋    | 166/291 [00:08<00:03, 34.17it/s]
Loading 0:  58%|█████▊    | 170/291 [00:08<00:03, 34.53it/s]
Loading 0:  60%|██████    | 176/291 [00:08<00:02, 39.13it/s]
Loading 0:  63%|██████▎   | 182/291 [00:08<00:02, 39.30it/s]
Loading 0:  64%|██████▍   | 187/291 [00:08<00:02, 40.49it/s]
Loading 0:  67%|██████▋   | 194/291 [00:09<00:02, 46.13it/s]
Loading 0:  69%|██████▊   | 200/291 [00:09<00:02, 42.94it/s]
Loading 0:  70%|███████   | 205/291 [00:09<00:01, 43.34it/s]
Loading 0:  73%|███████▎  | 212/291 [00:09<00:01, 49.11it/s]
Loading 0:  75%|███████▍  | 218/291 [00:09<00:01, 45.93it/s]
Loading 0:  77%|███████▋  | 223/291 [00:09<00:01, 45.99it/s]
Loading 0:  79%|███████▉  | 230/291 [00:09<00:01, 50.82it/s]
Loading 0:  81%|████████  | 236/291 [00:10<00:01, 46.07it/s]
Loading 0:  83%|████████▎ | 241/291 [00:10<00:01, 45.04it/s]
Loading 0:  85%|████████▍ | 247/291 [00:10<00:00, 47.93it/s]
Loading 0:  87%|████████▋ | 252/291 [00:10<00:00, 47.58it/s]
Loading 0:  89%|████████▊ | 258/291 [00:10<00:00, 43.06it/s]
Loading 0:  91%|█████████▏| 266/291 [00:10<00:00, 35.84it/s]
Loading 0:  93%|█████████▎| 272/291 [00:10<00:00, 36.53it/s]
Loading 0:  95%|█████████▍| 276/291 [00:11<00:00, 36.63it/s]
Loading 0:  98%|█████████▊| 284/291 [00:11<00:00, 44.94it/s]
Loading 0: 100%|█████████▉| 290/291 [00:11<00:00, 42.74it/s]
                                                            
ERROR "cp /dev/shm/model_cache/flywheel_model.0.safetensors s3://guanaco-mkml-models/juvi21-uwu-8b-v2/flywheel_model.0.safetensors": MultipartUpload: upload multipart failed upload id: 2~4Yvd_jbETahrrJ4j6iYoUINMt3Phlrj caused by: QuotaExceeded:  status code: 403, request id: tx000001034e6dcc16e2528-0066ae43c6-a41e57da0-default, host id:
Job juvi21-uwu-8b-v2-mkmlizer completed after 63.21s with status: failed
Stopping job with name juvi21-uwu-8b-v2-mkmlizer
%s, retrying in %s seconds...
Starting job with name juvi21-uwu-8b-v2-mkmlizer
Waiting for job on juvi21-uwu-8b-v2-mkmlizer to finish
juvi21-uwu-8b-v2-mkmlizer: ╔═════════════════════════════════════════════════════════════════════╗
juvi21-uwu-8b-v2-mkmlizer: ║     _____            __           __                                ║
juvi21-uwu-8b-v2-mkmlizer: ║    / _/ /_ ___    __/ /  ___ ___ / /                                ║
juvi21-uwu-8b-v2-mkmlizer: ║   / _/ / // / |/|/ / _ \/ -_) -_) /                                 ║
juvi21-uwu-8b-v2-mkmlizer: ║  /_//_/\_, /|__,__/_//_/\__/\__/_/                                  ║
juvi21-uwu-8b-v2-mkmlizer: ║       /___/                                                         ║
juvi21-uwu-8b-v2-mkmlizer: ║                                                                     ║
juvi21-uwu-8b-v2-mkmlizer: ║  Version: 0.9.9                                                     ║
juvi21-uwu-8b-v2-mkmlizer: ║  Copyright 2023 MK ONE TECHNOLOGIES Inc.                            ║
juvi21-uwu-8b-v2-mkmlizer: ║  https://mk1.ai                                                     ║
juvi21-uwu-8b-v2-mkmlizer: ║                                                                     ║
juvi21-uwu-8b-v2-mkmlizer: ║  The license key for the current software has been verified as      ║
juvi21-uwu-8b-v2-mkmlizer: ║  belonging to:                                                      ║
juvi21-uwu-8b-v2-mkmlizer: ║                                                                     ║
juvi21-uwu-8b-v2-mkmlizer: ║  Chai Research Corp.                                                ║
juvi21-uwu-8b-v2-mkmlizer: ║  Account ID: 7997a29f-0ceb-4cc7-9adf-840c57b4ae6f                   ║
juvi21-uwu-8b-v2-mkmlizer: ║  Expiration: 2024-10-15 23:59:59                                    ║
juvi21-uwu-8b-v2-mkmlizer: ║                                                                     ║
juvi21-uwu-8b-v2-mkmlizer: ╚═════════════════════════════════════════════════════════════════════╝
juvi21-uwu-8b-v2-mkmlizer: Downloaded to shared memory in 21.852s
juvi21-uwu-8b-v2-mkmlizer: quantizing model to /dev/shm/model_cache, profile:s0, folder:/tmp/tmpkggk9d_r, device:0
juvi21-uwu-8b-v2-mkmlizer: Saving flywheel model at /dev/shm/model_cache
juvi21-uwu-8b-v2-mkmlizer: quantized model in 26.011s
juvi21-uwu-8b-v2-mkmlizer: Processed model juvi21/UwU-8B in 47.863s
juvi21-uwu-8b-v2-mkmlizer: creating bucket guanaco-mkml-models
juvi21-uwu-8b-v2-mkmlizer: Bucket 's3://guanaco-mkml-models/' created
juvi21-uwu-8b-v2-mkmlizer: uploading /dev/shm/model_cache to s3://guanaco-mkml-models/juvi21-uwu-8b-v2
juvi21-uwu-8b-v2-mkmlizer: cp /dev/shm/model_cache/config.json s3://guanaco-mkml-models/juvi21-uwu-8b-v2/config.json
juvi21-uwu-8b-v2-mkmlizer: cp /dev/shm/model_cache/special_tokens_map.json s3://guanaco-mkml-models/juvi21-uwu-8b-v2/special_tokens_map.json
juvi21-uwu-8b-v2-mkmlizer: cp /dev/shm/model_cache/tokenizer_config.json s3://guanaco-mkml-models/juvi21-uwu-8b-v2/tokenizer_config.json
juvi21-uwu-8b-v2-mkmlizer: cp /dev/shm/model_cache/tokenizer.json s3://guanaco-mkml-models/juvi21-uwu-8b-v2/tokenizer.json
juvi21-uwu-8b-v2-mkmlizer: 
Loading 0:   0%|          | 0/291 [00:00<?, ?it/s]
Loading 0:   1%|          | 2/291 [00:04<11:27,  2.38s/it]
Loading 0:   2%|▏         | 6/291 [00:04<03:03,  1.56it/s]
Loading 0:   4%|▍         | 13/291 [00:05<01:05,  4.25it/s]
Loading 0:   6%|▌         | 18/291 [00:05<00:40,  6.67it/s]
Loading 0:   8%|▊         | 23/291 [00:05<00:27,  9.69it/s]
Loading 0:  10%|▉         | 29/291 [00:05<00:19, 13.55it/s]
Loading 0:  12%|█▏        | 34/291 [00:05<00:14, 17.16it/s]
Loading 0:  14%|█▍        | 41/291 [00:05<00:10, 23.97it/s]
Loading 0:  16%|█▌        | 47/291 [00:05<00:08, 27.47it/s]
Loading 0:  18%|█▊        | 52/291 [00:05<00:07, 30.71it/s]
Loading 0:  20%|██        | 59/291 [00:05<00:06, 37.47it/s]
Loading 0:  22%|██▏       | 65/291 [00:06<00:08, 26.50it/s]
Loading 0:  24%|██▍       | 70/291 [00:06<00:07, 29.33it/s]
Loading 0:  26%|██▋       | 77/291 [00:06<00:05, 36.04it/s]
Loading 0:  29%|██▊       | 83/291 [00:06<00:05, 37.25it/s]
Loading 0:  30%|███       | 88/291 [00:06<00:05, 38.64it/s]
Loading 0:  33%|███▎      | 95/291 [00:06<00:04, 44.78it/s]
Loading 0:  35%|███▍      | 101/291 [00:07<00:04, 43.90it/s]
Loading 0:  36%|███▋      | 106/291 [00:07<00:04, 44.25it/s]
Loading 0:  39%|███▉      | 113/291 [00:07<00:03, 49.10it/s]
Loading 0:  41%|████      | 119/291 [00:07<00:03, 46.18it/s]
Loading 0:  43%|████▎     | 124/291 [00:07<00:03, 45.63it/s]
Loading 0:  45%|████▌     | 131/291 [00:07<00:03, 50.58it/s]
Loading 0:  47%|████▋     | 137/291 [00:07<00:03, 45.05it/s]
Loading 0:  49%|████▉     | 142/291 [00:07<00:03, 45.07it/s]
Loading 0:  51%|█████     | 149/291 [00:08<00:02, 49.96it/s]
Loading 0:  53%|█████▎    | 155/291 [00:08<00:02, 45.36it/s]
Loading 0:  55%|█████▍    | 160/291 [00:08<00:02, 45.43it/s]
Loading 0:  57%|█████▋    | 166/291 [00:08<00:03, 34.09it/s]
Loading 0:  59%|█████▉    | 171/291 [00:08<00:03, 37.00it/s]
Loading 0:  61%|██████    | 177/291 [00:08<00:03, 36.50it/s]
Loading 0:  64%|██████▎   | 185/291 [00:09<00:02, 44.96it/s]
Loading 0:  66%|██████▌   | 191/291 [00:09<00:02, 43.54it/s]
Loading 0:  67%|██████▋   | 196/291 [00:09<00:02, 43.08it/s]
Loading 0:  70%|██████▉   | 203/291 [00:09<00:01, 48.74it/s]
Loading 0:  72%|███████▏  | 209/291 [00:09<00:01, 46.04it/s]
Loading 0:  74%|███████▎  | 214/291 [00:09<00:01, 44.87it/s]
Loading 0:  76%|███████▌  | 221/291 [00:09<00:01, 50.02it/s]
Loading 0:  78%|███████▊  | 227/291 [00:09<00:01, 46.44it/s]
Loading 0:  80%|███████▉  | 232/291 [00:10<00:01, 46.20it/s]
Loading 0:  82%|████████▏ | 239/291 [00:10<00:01, 50.67it/s]
Loading 0:  84%|████████▍ | 245/291 [00:10<00:00, 46.12it/s]
Loading 0:  86%|████████▌ | 250/291 [00:10<00:00, 45.58it/s]
Loading 0:  88%|████████▊ | 257/291 [00:10<00:00, 50.19it/s]
Loading 0:  90%|█████████ | 263/291 [00:10<00:00, 47.33it/s]
Loading 0:  92%|█████████▏| 268/291 [00:10<00:00, 32.05it/s]
Loading 0:  95%|█████████▍| 275/291 [00:11<00:00, 38.96it/s]
Loading 0:  96%|█████████▌| 280/291 [00:11<00:00, 41.27it/s]
Loading 0:  98%|█████████▊| 285/291 [00:11<00:00, 38.03it/s]
                                                            
ERROR "cp /dev/shm/model_cache/flywheel_model.0.safetensors s3://guanaco-mkml-models/juvi21-uwu-8b-v2/flywheel_model.0.safetensors": MultipartUpload: upload multipart failed upload id: 2~5GDSmBByzqp1d_zUNwrokU0TQ6kG-MF caused by: QuotaExceeded:  status code: 403, request id: tx00000ca5e37ae35bd825c-0066ae4405-a41e57da0-default, host id:
Job juvi21-uwu-8b-v2-mkmlizer completed after 63.95s with status: failed
Stopping job with name juvi21-uwu-8b-v2-mkmlizer
MKMLizerError('')
juvi21-uwu-8b_v2 status is now failed due to DeploymentManager action
admin requested tearing down of juvi21-uwu-8b_v2
Running pipeline stage ISVCDeleter
Skipping teardown as no inference service was successfully deployed
Running pipeline stage ISVCDeleter
Skipping teardown as no inference service was successfully deployed
Pipeline stage ISVCDeleter completed in 0.22s
Running pipeline stage MKMLModelDeleter
Skipping deletion as no model was successfully uploaded
Pipeline stage MKMLModelDeleter completed in 0.17s
juvi21-uwu-8b_v2 status is now torndown due to DeploymentManager action