submission_id: juvi21-uwu-8b_v1
developer_uid: juvi21
formatter: {'memory_template': "<|begin_of_text|><|start_header_id|>system<|end_header_id|>\n\n{bot_name}'s Persona: {memory}\n\n", 'prompt_template': '{prompt}<|eot_id|>', 'bot_template': '<|start_header_id|>assistant<|end_header_id|>\n\n{bot_name}: {message}<|eot_id|>', 'user_template': '<|start_header_id|>user<|end_header_id|>\n\n{user_name}: {message}<|eot_id|>', 'response_template': '<|start_header_id|>assistant<|end_header_id|>\n\n{bot_name}:', 'truncate_by_message': False}
generation_params: {'temperature': 1.21, 'top_p': 1.0, 'min_p': 0.1, 'top_k': 40, 'presence_penalty': 0.0, 'frequency_penalty': 0.0, 'stopping_words': ['\n', '<|end_header_id|>', '<|eot_id|>', '\n\n{user_name}', 'You:', '\n\n'], 'max_input_tokens': 512, 'best_of': 16, 'max_output_tokens': 64, 'reward_max_token_input': 256}
model_name: juvi21-uwu-8b_v1
model_repo: juvi21/UwU-8B
reward_formatter: {'bot_template': '<|im_start|>assistant\n{bot_name}: {message}<|im_end|>\n', 'memory_template': '<|im_start|>system\n{memory}<|im_end|>\n', 'prompt_template': '<|im_start|>user\n{prompt}<|im_end|>\n', 'response_template': '<|im_start|>assistant\n{bot_name}:', 'truncate_by_message': False, 'user_template': '<|im_start|>user\n{user_name}: {message}<|im_end|>\n'}
reward_repo: ChaiML/gpt2_xl_pairwise_89m_step_347634
status: torndown
timestamp: 2024-08-03T14:36:51+00:00
Resubmit model
Running pipeline stage MKMLizer
Starting job with name juvi21-uwu-8b-v1-mkmlizer
Waiting for job on juvi21-uwu-8b-v1-mkmlizer to finish
Stopping job with name juvi21-uwu-8b-v1-mkmlizer
%s, retrying in %s seconds...
Starting job with name juvi21-uwu-8b-v1-mkmlizer
Waiting for job on juvi21-uwu-8b-v1-mkmlizer to finish
juvi21-uwu-8b-v1-mkmlizer: ╔═════════════════════════════════════════════════════════════════════╗
juvi21-uwu-8b-v1-mkmlizer: ║ _____ __ __ ║
juvi21-uwu-8b-v1-mkmlizer: ║ / _/ /_ ___ __/ / ___ ___ / / ║
juvi21-uwu-8b-v1-mkmlizer: ║ / _/ / // / |/|/ / _ \/ -_) -_) / ║
juvi21-uwu-8b-v1-mkmlizer: ║ /_//_/\_, /|__,__/_//_/\__/\__/_/ ║
juvi21-uwu-8b-v1-mkmlizer: ║ /___/ ║
juvi21-uwu-8b-v1-mkmlizer: ║ ║
juvi21-uwu-8b-v1-mkmlizer: ║ Version: 0.9.9 ║
juvi21-uwu-8b-v1-mkmlizer: ║ Copyright 2023 MK ONE TECHNOLOGIES Inc. ║
juvi21-uwu-8b-v1-mkmlizer: ║ https://mk1.ai ║
juvi21-uwu-8b-v1-mkmlizer: ║ ║
juvi21-uwu-8b-v1-mkmlizer: ║ The license key for the current software has been verified as ║
juvi21-uwu-8b-v1-mkmlizer: ║ belonging to: ║
juvi21-uwu-8b-v1-mkmlizer: ║ ║
juvi21-uwu-8b-v1-mkmlizer: ║ Chai Research Corp. ║
juvi21-uwu-8b-v1-mkmlizer: ║ Account ID: 7997a29f-0ceb-4cc7-9adf-840c57b4ae6f ║
juvi21-uwu-8b-v1-mkmlizer: ║ Expiration: 2024-10-15 23:59:59 ║
juvi21-uwu-8b-v1-mkmlizer: ║ ║
juvi21-uwu-8b-v1-mkmlizer: ╚═════════════════════════════════════════════════════════════════════╝
juvi21-uwu-8b-v1-mkmlizer: Downloaded to shared memory in 30.917s
juvi21-uwu-8b-v1-mkmlizer: quantizing model to /dev/shm/model_cache, profile:s0, folder:/tmp/tmprmd7m1jg, device:0
juvi21-uwu-8b-v1-mkmlizer: Saving flywheel model at /dev/shm/model_cache
juvi21-uwu-8b-v1-mkmlizer: quantized model in 26.051s
juvi21-uwu-8b-v1-mkmlizer: Processed model juvi21/UwU-8B in 56.969s
juvi21-uwu-8b-v1-mkmlizer: creating bucket guanaco-mkml-models
juvi21-uwu-8b-v1-mkmlizer: Bucket 's3://guanaco-mkml-models/' created
juvi21-uwu-8b-v1-mkmlizer: uploading /dev/shm/model_cache to s3://guanaco-mkml-models/juvi21-uwu-8b-v1
juvi21-uwu-8b-v1-mkmlizer: cp /dev/shm/model_cache/config.json s3://guanaco-mkml-models/juvi21-uwu-8b-v1/config.json
juvi21-uwu-8b-v1-mkmlizer: cp /dev/shm/model_cache/special_tokens_map.json s3://guanaco-mkml-models/juvi21-uwu-8b-v1/special_tokens_map.json
juvi21-uwu-8b-v1-mkmlizer: cp /dev/shm/model_cache/tokenizer_config.json s3://guanaco-mkml-models/juvi21-uwu-8b-v1/tokenizer_config.json
juvi21-uwu-8b-v1-mkmlizer: cp /dev/shm/model_cache/tokenizer.json s3://guanaco-mkml-models/juvi21-uwu-8b-v1/tokenizer.json
juvi21-uwu-8b-v1-mkmlizer: Loading 0: 0%| | 0/291 [00:00<?, ?it/s] Loading 0: 1%| | 2/291 [00:04<11:29, 2.39s/it] Loading 0: 2%|▏ | 6/291 [00:04<03:03, 1.56it/s] Loading 0: 4%|▍ | 13/291 [00:05<01:05, 4.26it/s] Loading 0: 6%|▌ | 18/291 [00:05<00:40, 6.69it/s] Loading 0: 8%|▊ | 23/291 [00:05<00:27, 9.74it/s] Loading 0: 10%|▉ | 29/291 [00:05<00:18, 13.83it/s] Loading 0: 12%|█▏ | 34/291 [00:05<00:14, 17.70it/s] Loading 0: 14%|█▍ | 42/291 [00:05<00:10, 23.88it/s] Loading 0: 17%|█▋ | 50/291 [00:05<00:07, 32.08it/s] Loading 0: 19%|█▉ | 56/291 [00:05<00:06, 34.19it/s] Loading 0: 21%|██▏ | 62/291 [00:06<00:08, 28.09it/s] Loading 0: 23%|██▎ | 68/291 [00:06<00:06, 33.03it/s] Loading 0: 25%|██▌ | 74/291 [00:06<00:06, 35.27it/s] Loading 0: 27%|██▋ | 79/291 [00:06<00:05, 37.40it/s] Loading 0: 30%|██▉ | 86/291 [00:06<00:04, 42.75it/s] Loading 0: 32%|███▏ | 92/291 [00:06<00:04, 41.80it/s] Loading 0: 33%|███▎ | 97/291 [00:06<00:04, 42.40it/s] Loading 0: 36%|███▌ | 104/291 [00:07<00:03, 48.40it/s] Loading 0: 38%|███▊ | 110/291 [00:07<00:03, 46.32it/s] Loading 0: 40%|███▉ | 115/291 [00:07<00:03, 45.08it/s] Loading 0: 42%|████▏ | 122/291 [00:07<00:03, 49.95it/s] Loading 0: 44%|████▍ | 128/291 [00:07<00:03, 46.70it/s] Loading 0: 46%|████▌ | 133/291 [00:07<00:03, 44.21it/s] Loading 0: 48%|████▊ | 140/291 [00:07<00:03, 49.19it/s] Loading 0: 50%|█████ | 146/291 [00:07<00:03, 46.63it/s] Loading 0: 52%|█████▏ | 151/291 [00:08<00:03, 46.23it/s] Loading 0: 54%|█████▍ | 158/291 [00:08<00:02, 51.13it/s] Loading 0: 56%|█████▋ | 164/291 [00:08<00:02, 47.41it/s] Loading 0: 58%|█████▊ | 169/291 [00:08<00:03, 32.29it/s] Loading 0: 60%|██████ | 176/291 [00:08<00:02, 39.12it/s] Loading 0: 63%|██████▎ | 182/291 [00:08<00:02, 39.92it/s] Loading 0: 64%|██████▍ | 187/291 [00:08<00:02, 41.26it/s] Loading 0: 67%|██████▋ | 194/291 [00:09<00:02, 46.94it/s] Loading 0: 69%|██████▊ | 200/291 [00:09<00:02, 44.95it/s] Loading 0: 70%|███████ | 205/291 [00:09<00:01, 45.43it/s] Loading 0: 73%|███████▎ | 212/291 [00:09<00:01, 49.79it/s] Loading 0: 75%|███████▍ | 218/291 [00:09<00:01, 47.70it/s] Loading 0: 77%|███████▋ | 223/291 [00:09<00:01, 47.89it/s] Loading 0: 79%|███████▉ | 230/291 [00:09<00:01, 53.37it/s] Loading 0: 81%|████████ | 236/291 [00:09<00:01, 49.00it/s] Loading 0: 83%|████████▎ | 242/291 [00:10<00:01, 48.87it/s] Loading 0: 85%|████████▌ | 248/291 [00:10<00:00, 50.30it/s] Loading 0: 87%|████████▋ | 254/291 [00:10<00:00, 45.39it/s] Loading 0: 89%|████████▉ | 259/291 [00:10<00:00, 44.11it/s] Loading 0: 91%|█████████ | 265/291 [00:10<00:00, 47.72it/s] Loading 0: 93%|█████████▎| 270/291 [00:10<00:00, 32.27it/s] Loading 0: 95%|█████████▍| 276/291 [00:11<00:00, 31.95it/s] Loading 0: 97%|█████████▋| 283/291 [00:11<00:00, 39.14it/s] Loading 0: 99%|█████████▉| 288/291 [00:11<00:00, 41.17it/s] ERROR "cp /dev/shm/model_cache/flywheel_model.0.safetensors s3://guanaco-mkml-models/juvi21-uwu-8b-v1/flywheel_model.0.safetensors": MultipartUpload: upload multipart failed upload id: 2~ainYfuLhjkFGAhNknPuaX3SDK_ya9jS caused by: QuotaExceeded: status code: 403, request id: tx000000c6c158866436159-0066ae40ca-a41e00e2b-default, host id:
Job juvi21-uwu-8b-v1-mkmlizer completed after 74.75s with status: failed
Stopping job with name juvi21-uwu-8b-v1-mkmlizer
%s, retrying in %s seconds...
Starting job with name juvi21-uwu-8b-v1-mkmlizer
Waiting for job on juvi21-uwu-8b-v1-mkmlizer to finish
juvi21-uwu-8b-v1-mkmlizer: ╔═════════════════════════════════════════════════════════════════════╗
juvi21-uwu-8b-v1-mkmlizer: ║ _____ __ __ ║
juvi21-uwu-8b-v1-mkmlizer: ║ / _/ /_ ___ __/ / ___ ___ / / ║
juvi21-uwu-8b-v1-mkmlizer: ║ / _/ / // / |/|/ / _ \/ -_) -_) / ║
juvi21-uwu-8b-v1-mkmlizer: ║ /_//_/\_, /|__,__/_//_/\__/\__/_/ ║
juvi21-uwu-8b-v1-mkmlizer: ║ /___/ ║
juvi21-uwu-8b-v1-mkmlizer: ║ ║
juvi21-uwu-8b-v1-mkmlizer: ║ Version: 0.9.9 ║
juvi21-uwu-8b-v1-mkmlizer: ║ Copyright 2023 MK ONE TECHNOLOGIES Inc. ║
juvi21-uwu-8b-v1-mkmlizer: ║ https://mk1.ai ║
juvi21-uwu-8b-v1-mkmlizer: ║ ║
juvi21-uwu-8b-v1-mkmlizer: ║ The license key for the current software has been verified as ║
juvi21-uwu-8b-v1-mkmlizer: ║ belonging to: ║
juvi21-uwu-8b-v1-mkmlizer: ║ ║
juvi21-uwu-8b-v1-mkmlizer: ║ Chai Research Corp. ║
juvi21-uwu-8b-v1-mkmlizer: ║ Account ID: 7997a29f-0ceb-4cc7-9adf-840c57b4ae6f ║
juvi21-uwu-8b-v1-mkmlizer: ║ Expiration: 2024-10-15 23:59:59 ║
juvi21-uwu-8b-v1-mkmlizer: ║ ║
juvi21-uwu-8b-v1-mkmlizer: ╚═════════════════════════════════════════════════════════════════════╝
juvi21-uwu-8b-v1-mkmlizer: Downloaded to shared memory in 23.025s
juvi21-uwu-8b-v1-mkmlizer: quantizing model to /dev/shm/model_cache, profile:s0, folder:/tmp/tmp9jh8epva, device:0
juvi21-uwu-8b-v1-mkmlizer: Saving flywheel model at /dev/shm/model_cache
juvi21-uwu-8b-v1-mkmlizer: quantized model in 25.611s
juvi21-uwu-8b-v1-mkmlizer: Processed model juvi21/UwU-8B in 48.636s
juvi21-uwu-8b-v1-mkmlizer: creating bucket guanaco-mkml-models
juvi21-uwu-8b-v1-mkmlizer: Bucket 's3://guanaco-mkml-models/' created
juvi21-uwu-8b-v1-mkmlizer: uploading /dev/shm/model_cache to s3://guanaco-mkml-models/juvi21-uwu-8b-v1
juvi21-uwu-8b-v1-mkmlizer: cp /dev/shm/model_cache/config.json s3://guanaco-mkml-models/juvi21-uwu-8b-v1/config.json
juvi21-uwu-8b-v1-mkmlizer: cp /dev/shm/model_cache/special_tokens_map.json s3://guanaco-mkml-models/juvi21-uwu-8b-v1/special_tokens_map.json
juvi21-uwu-8b-v1-mkmlizer: cp /dev/shm/model_cache/tokenizer_config.json s3://guanaco-mkml-models/juvi21-uwu-8b-v1/tokenizer_config.json
juvi21-uwu-8b-v1-mkmlizer: cp /dev/shm/model_cache/tokenizer.json s3://guanaco-mkml-models/juvi21-uwu-8b-v1/tokenizer.json
juvi21-uwu-8b-v1-mkmlizer: Loading 0: 0%| | 0/291 [00:00<?, ?it/s] Loading 0: 1%| | 2/291 [00:04<11:23, 2.37s/it] Loading 0: 2%|▏ | 6/291 [00:04<03:01, 1.57it/s] Loading 0: 4%|▍ | 13/291 [00:04<01:05, 4.27it/s] Loading 0: 7%|▋ | 19/291 [00:05<00:37, 7.25it/s] Loading 0: 8%|▊ | 24/291 [00:05<00:27, 9.86it/s] Loading 0: 11%|█ | 32/291 [00:05<00:16, 16.07it/s] Loading 0: 13%|█▎ | 38/291 [00:05<00:12, 20.20it/s] Loading 0: 15%|█▌ | 44/291 [00:05<00:09, 25.19it/s] Loading 0: 18%|█▊ | 51/291 [00:05<00:08, 29.22it/s] Loading 0: 20%|██ | 59/291 [00:05<00:06, 37.70it/s] Loading 0: 22%|██▏ | 65/291 [00:06<00:07, 28.98it/s] Loading 0: 24%|██▍ | 70/291 [00:06<00:06, 32.25it/s] Loading 0: 26%|██▋ | 77/291 [00:06<00:05, 39.28it/s] Loading 0: 29%|██▊ | 83/291 [00:06<00:05, 40.18it/s] Loading 0: 31%|███ | 89/291 [00:06<00:04, 43.21it/s] Loading 0: 33%|███▎ | 96/291 [00:06<00:04, 43.14it/s] Loading 0: 36%|███▌ | 104/291 [00:06<00:03, 50.58it/s] Loading 0: 38%|███▊ | 110/291 [00:07<00:03, 47.45it/s] Loading 0: 40%|███▉ | 116/291 [00:07<00:03, 49.74it/s] Loading 0: 42%|████▏ | 122/291 [00:07<00:03, 51.97it/s] Loading 0: 44%|████▍ | 128/291 [00:07<00:03, 48.76it/s] Loading 0: 46%|████▌ | 134/291 [00:07<00:03, 50.25it/s] Loading 0: 48%|████▊ | 141/291 [00:07<00:03, 46.62it/s] Loading 0: 51%|█████ | 149/291 [00:07<00:02, 53.49it/s] Loading 0: 53%|█████▎ | 155/291 [00:07<00:02, 49.74it/s] Loading 0: 55%|█████▌ | 161/291 [00:08<00:02, 50.03it/s] Loading 0: 57%|█████▋ | 167/291 [00:08<00:03, 37.37it/s] Loading 0: 59%|█████▉ | 173/291 [00:08<00:03, 38.77it/s] Loading 0: 61%|██████ | 178/291 [00:08<00:02, 40.71it/s] Loading 0: 64%|██████▎ | 185/291 [00:08<00:02, 46.21it/s] Loading 0: 66%|██████▌ | 191/291 [00:08<00:02, 45.32it/s] Loading 0: 67%|██████▋ | 196/291 [00:08<00:02, 45.98it/s] Loading 0: 70%|██████▉ | 203/291 [00:09<00:01, 51.12it/s] Loading 0: 72%|███████▏ | 209/291 [00:09<00:01, 47.64it/s] Loading 0: 74%|███████▎ | 214/291 [00:09<00:01, 47.53it/s] Loading 0: 76%|███████▌ | 221/291 [00:09<00:01, 53.23it/s] Loading 0: 78%|███████▊ | 227/291 [00:09<00:01, 49.69it/s] Loading 0: 80%|████████ | 233/291 [00:09<00:01, 51.51it/s] Loading 0: 82%|████████▏ | 240/291 [00:09<00:01, 47.60it/s] Loading 0: 85%|████████▌ | 248/291 [00:09<00:00, 53.99it/s] Loading 0: 87%|████████▋ | 254/291 [00:10<00:00, 48.61it/s] Loading 0: 89%|████████▉ | 260/291 [00:10<00:00, 49.91it/s] Loading 0: 91%|█████████▏| 266/291 [00:10<00:00, 36.22it/s] Loading 0: 93%|█████████▎| 272/291 [00:10<00:00, 37.51it/s] Loading 0: 95%|█████████▌| 277/291 [00:10<00:00, 39.93it/s] Loading 0: 98%|█████████▊| 284/291 [00:10<00:00, 46.17it/s] Loading 0: 100%|█████████▉| 290/291 [00:10<00:00, 43.58it/s] ERROR "cp /dev/shm/model_cache/flywheel_model.0.safetensors s3://guanaco-mkml-models/juvi21-uwu-8b-v1/flywheel_model.0.safetensors": MultipartUpload: upload multipart failed upload id: 2~TjgSAEasg4VVJAAxISdRTKun9uExg7j caused by: QuotaExceeded: status code: 403, request id: tx0000075d8c269fc2631c3-0066ae410a-a41e00e2b-default, host id:
Job juvi21-uwu-8b-v1-mkmlizer completed after 62.9s with status: failed
Stopping job with name juvi21-uwu-8b-v1-mkmlizer
MKMLizerError('')
juvi21-uwu-8b_v1 status is now failed due to DeploymentManager action
admin requested tearing down of juvi21-uwu-8b_v1
Running pipeline stage ISVCDeleter
Skipping teardown as no inference service was successfully deployed
Pipeline stage ISVCDeleter completed in 0.17s
Running pipeline stage MKMLModelDeleter
Skipping deletion as no model was successfully uploaded
Pipeline stage MKMLModelDeleter completed in 0.14s
juvi21-uwu-8b_v1 status is now torndown due to DeploymentManager action