submission_id: oliveiratime-axolotl_v6
developer_uid: Oliveiratime
formatter: {'memory_template': "{bot_name}'s Persona: {memory}\n####\n", 'prompt_template': '{prompt}\n<START>\n', 'bot_template': '{bot_name}: {message}\n', 'user_template': '{user_name}: {message}\n', 'response_template': '{bot_name}:', 'truncate_by_message': False}
generation_params: {'temperature': 1.15, 'top_p': 1.0, 'min_p': 0.06, 'top_k': 250, 'presence_penalty': 0.0, 'frequency_penalty': 0.1, 'stopping_words': ['\n'], 'max_input_tokens': 512, 'best_of': 16, 'max_output_tokens': 64}
model_name: oliveiratime-axolotl_v2
model_repo: Oliveiratime/axolotl
status: torndown
timestamp: 2024-08-24T09:30:48+00:00
Resubmit model
Running pipeline stage MKMLizer
Starting job with name oliveiratime-axolotl-v6-mkmlizer
Waiting for job on oliveiratime-axolotl-v6-mkmlizer to finish
oliveiratime-axolotl-v6-mkmlizer: ╔═════════════════════════════════════════════════════════════════════╗
oliveiratime-axolotl-v6-mkmlizer: ║     _____            __           __                                ║
oliveiratime-axolotl-v6-mkmlizer: ║    / _/ /_ ___    __/ /  ___ ___ / /                                ║
oliveiratime-axolotl-v6-mkmlizer: ║   / _/ / // / |/|/ / _ \/ -_) -_) /                                 ║
oliveiratime-axolotl-v6-mkmlizer: ║  /_//_/\_, /|__,__/_//_/\__/\__/_/                                  ║
oliveiratime-axolotl-v6-mkmlizer: ║       /___/                                                         ║
oliveiratime-axolotl-v6-mkmlizer: ║                                                                     ║
oliveiratime-axolotl-v6-mkmlizer: ║  Version: 0.10.1                                                    ║
oliveiratime-axolotl-v6-mkmlizer: ║  Copyright 2023 MK ONE TECHNOLOGIES Inc.                            ║
oliveiratime-axolotl-v6-mkmlizer: ║  https://mk1.ai                                                     ║
oliveiratime-axolotl-v6-mkmlizer: ║                                                                     ║
oliveiratime-axolotl-v6-mkmlizer: ║  The license key for the current software has been verified as      ║
oliveiratime-axolotl-v6-mkmlizer: ║  belonging to:                                                      ║
oliveiratime-axolotl-v6-mkmlizer: ║                                                                     ║
oliveiratime-axolotl-v6-mkmlizer: ║  Chai Research Corp.                                                ║
oliveiratime-axolotl-v6-mkmlizer: ║  Account ID: 7997a29f-0ceb-4cc7-9adf-840c57b4ae6f                   ║
oliveiratime-axolotl-v6-mkmlizer: ║  Expiration: 2024-10-15 23:59:59                                    ║
oliveiratime-axolotl-v6-mkmlizer: ║                                                                     ║
oliveiratime-axolotl-v6-mkmlizer: ╚═════════════════════════════════════════════════════════════════════╝
oliveiratime-axolotl-v6-mkmlizer: Downloaded to shared memory in 9.418s
oliveiratime-axolotl-v6-mkmlizer: quantizing model to /dev/shm/model_cache, profile:s0, folder:/tmp/tmpd4yib3jk, device:0
oliveiratime-axolotl-v6-mkmlizer: Saving flywheel model at /dev/shm/model_cache
oliveiratime-axolotl-v6-mkmlizer: Traceback (most recent call last):
oliveiratime-axolotl-v6-mkmlizer:   File "/code/uploading/mkmlize.py", line 151, in <module>
oliveiratime-axolotl-v6-mkmlizer:     cli()
oliveiratime-axolotl-v6-mkmlizer:   File "/opt/conda/lib/python3.10/site-packages/click/core.py", line 1157, in __call__
oliveiratime-axolotl-v6-mkmlizer:     return self.main(*args, **kwargs)
oliveiratime-axolotl-v6-mkmlizer:   File "/opt/conda/lib/python3.10/site-packages/click/core.py", line 1078, in main
oliveiratime-axolotl-v6-mkmlizer:     rv = self.invoke(ctx)
oliveiratime-axolotl-v6-mkmlizer:   File "/opt/conda/lib/python3.10/site-packages/click/core.py", line 1688, in invoke
oliveiratime-axolotl-v6-mkmlizer:     return _process_result(sub_ctx.command.invoke(sub_ctx))
oliveiratime-axolotl-v6-mkmlizer:   File "/opt/conda/lib/python3.10/site-packages/click/core.py", line 1434, in invoke
oliveiratime-axolotl-v6-mkmlizer:     return ctx.invoke(self.callback, **ctx.params)
oliveiratime-axolotl-v6-mkmlizer:   File "/opt/conda/lib/python3.10/site-packages/click/core.py", line 783, in invoke
oliveiratime-axolotl-v6-mkmlizer:     return __callback(*args, **kwargs)
oliveiratime-axolotl-v6-mkmlizer:   File "/code/uploading/mkmlize.py", line 42, in quantize
oliveiratime-axolotl-v6-mkmlizer:     quantize_model(temp_folder, output_path, profile, device)
oliveiratime-axolotl-v6-mkmlizer:   File "/code/uploading/mkmlize.py", line 135, in quantize_model
oliveiratime-axolotl-v6-mkmlizer:     flywheel.instrument(
oliveiratime-axolotl-v6-mkmlizer:   File "/opt/conda/lib/python3.10/site-packages/mk1/flywheel/instrument.py", line 90, in instrument
oliveiratime-axolotl-v6-mkmlizer:     compiler.save_pretrained(input_model_path, output_model_path, storage_format)
oliveiratime-axolotl-v6-mkmlizer:   File "/opt/conda/lib/python3.10/site-packages/mk1/flywheel/functional/compiler.py", line 23, in save_pretrained
oliveiratime-axolotl-v6-mkmlizer:     self.save_st_pretrained(input_model_path, output_model_path)
oliveiratime-axolotl-v6-mkmlizer:   File "/opt/conda/lib/python3.10/site-packages/mk1/flywheel/functional/compiler.py", line 31, in save_st_pretrained
oliveiratime-axolotl-v6-mkmlizer:     model_iterator = self.factory.model_iterator()
oliveiratime-axolotl-v6-mkmlizer:   File "/opt/conda/lib/python3.10/site-packages/mk1/flywheel/functional/loader.py", line 255, in model_iterator
oliveiratime-axolotl-v6-mkmlizer:     raise RuntimeError(f"Unable to construct an interator for the model")
oliveiratime-axolotl-v6-mkmlizer: RuntimeError: Unable to construct an interator for the model
Job oliveiratime-axolotl-v6-mkmlizer completed after 22.38s with status: failed
Stopping job with name oliveiratime-axolotl-v6-mkmlizer
%s, retrying in %s seconds...
Starting job with name oliveiratime-axolotl-v6-mkmlizer
Waiting for job on oliveiratime-axolotl-v6-mkmlizer to finish
Failed to get response for submission mistralai-mistral-nemo-_9330_v59: ('http://mistralai-mistral-nemo-9330-v59-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'activator request timeout')
oliveiratime-axolotl-v6-mkmlizer: ╔═════════════════════════════════════════════════════════════════════╗
oliveiratime-axolotl-v6-mkmlizer: ║     _____            __           __                                ║
oliveiratime-axolotl-v6-mkmlizer: ║    / _/ /_ ___    __/ /  ___ ___ / /                                ║
oliveiratime-axolotl-v6-mkmlizer: ║   / _/ / // / |/|/ / _ \/ -_) -_) /                                 ║
oliveiratime-axolotl-v6-mkmlizer: ║  /_//_/\_, /|__,__/_//_/\__/\__/_/                                  ║
oliveiratime-axolotl-v6-mkmlizer: ║       /___/                                                         ║
oliveiratime-axolotl-v6-mkmlizer: ║                                                                     ║
oliveiratime-axolotl-v6-mkmlizer: ║  Version: 0.10.1                                                    ║
oliveiratime-axolotl-v6-mkmlizer: ║  Copyright 2023 MK ONE TECHNOLOGIES Inc.                            ║
oliveiratime-axolotl-v6-mkmlizer: ║  https://mk1.ai                                                     ║
oliveiratime-axolotl-v6-mkmlizer: ║                                                                     ║
oliveiratime-axolotl-v6-mkmlizer: ║  The license key for the current software has been verified as      ║
oliveiratime-axolotl-v6-mkmlizer: ║  belonging to:                                                      ║
oliveiratime-axolotl-v6-mkmlizer: ║                                                                     ║
oliveiratime-axolotl-v6-mkmlizer: ║  Chai Research Corp.                                                ║
oliveiratime-axolotl-v6-mkmlizer: ║  Account ID: 7997a29f-0ceb-4cc7-9adf-840c57b4ae6f                   ║
oliveiratime-axolotl-v6-mkmlizer: ║  Expiration: 2024-10-15 23:59:59                                    ║
oliveiratime-axolotl-v6-mkmlizer: ║                                                                     ║
oliveiratime-axolotl-v6-mkmlizer: ╚═════════════════════════════════════════════════════════════════════╝
oliveiratime-axolotl-v6-mkmlizer: Downloaded to shared memory in 7.339s
oliveiratime-axolotl-v6-mkmlizer: quantizing model to /dev/shm/model_cache, profile:s0, folder:/tmp/tmpz6dn98ru, device:0
oliveiratime-axolotl-v6-mkmlizer: Saving flywheel model at /dev/shm/model_cache
oliveiratime-axolotl-v6-mkmlizer: Traceback (most recent call last):
oliveiratime-axolotl-v6-mkmlizer:   File "/code/uploading/mkmlize.py", line 151, in <module>
oliveiratime-axolotl-v6-mkmlizer:     cli()
oliveiratime-axolotl-v6-mkmlizer:   File "/opt/conda/lib/python3.10/site-packages/click/core.py", line 1157, in __call__
oliveiratime-axolotl-v6-mkmlizer:     return self.main(*args, **kwargs)
oliveiratime-axolotl-v6-mkmlizer:   File "/opt/conda/lib/python3.10/site-packages/click/core.py", line 1078, in main
oliveiratime-axolotl-v6-mkmlizer:     rv = self.invoke(ctx)
oliveiratime-axolotl-v6-mkmlizer:   File "/opt/conda/lib/python3.10/site-packages/click/core.py", line 1688, in invoke
oliveiratime-axolotl-v6-mkmlizer:     return _process_result(sub_ctx.command.invoke(sub_ctx))
oliveiratime-axolotl-v6-mkmlizer:   File "/opt/conda/lib/python3.10/site-packages/click/core.py", line 1434, in invoke
oliveiratime-axolotl-v6-mkmlizer:     return ctx.invoke(self.callback, **ctx.params)
oliveiratime-axolotl-v6-mkmlizer:   File "/opt/conda/lib/python3.10/site-packages/click/core.py", line 783, in invoke
oliveiratime-axolotl-v6-mkmlizer:     return __callback(*args, **kwargs)
oliveiratime-axolotl-v6-mkmlizer:   File "/code/uploading/mkmlize.py", line 42, in quantize
oliveiratime-axolotl-v6-mkmlizer:     quantize_model(temp_folder, output_path, profile, device)
oliveiratime-axolotl-v6-mkmlizer:   File "/code/uploading/mkmlize.py", line 135, in quantize_model
oliveiratime-axolotl-v6-mkmlizer:     flywheel.instrument(
oliveiratime-axolotl-v6-mkmlizer:   File "/opt/conda/lib/python3.10/site-packages/mk1/flywheel/instrument.py", line 90, in instrument
oliveiratime-axolotl-v6-mkmlizer:     compiler.save_pretrained(input_model_path, output_model_path, storage_format)
oliveiratime-axolotl-v6-mkmlizer:   File "/opt/conda/lib/python3.10/site-packages/mk1/flywheel/functional/compiler.py", line 23, in save_pretrained
oliveiratime-axolotl-v6-mkmlizer:     self.save_st_pretrained(input_model_path, output_model_path)
oliveiratime-axolotl-v6-mkmlizer:   File "/opt/conda/lib/python3.10/site-packages/mk1/flywheel/functional/compiler.py", line 31, in save_st_pretrained
oliveiratime-axolotl-v6-mkmlizer:     model_iterator = self.factory.model_iterator()
oliveiratime-axolotl-v6-mkmlizer:   File "/opt/conda/lib/python3.10/site-packages/mk1/flywheel/functional/loader.py", line 255, in model_iterator
oliveiratime-axolotl-v6-mkmlizer:     raise RuntimeError(f"Unable to construct an interator for the model")
oliveiratime-axolotl-v6-mkmlizer: RuntimeError: Unable to construct an interator for the model
Job oliveiratime-axolotl-v6-mkmlizer completed after 22.75s with status: failed
Stopping job with name oliveiratime-axolotl-v6-mkmlizer
%s, retrying in %s seconds...
Starting job with name oliveiratime-axolotl-v6-mkmlizer
Waiting for job on oliveiratime-axolotl-v6-mkmlizer to finish
oliveiratime-axolotl-v6-mkmlizer: ╔═════════════════════════════════════════════════════════════════════╗
oliveiratime-axolotl-v6-mkmlizer: ║     _____            __           __                                ║
oliveiratime-axolotl-v6-mkmlizer: ║    / _/ /_ ___    __/ /  ___ ___ / /                                ║
oliveiratime-axolotl-v6-mkmlizer: ║   / _/ / // / |/|/ / _ \/ -_) -_) /                                 ║
oliveiratime-axolotl-v6-mkmlizer: ║  /_//_/\_, /|__,__/_//_/\__/\__/_/                                  ║
oliveiratime-axolotl-v6-mkmlizer: ║       /___/                                                         ║
oliveiratime-axolotl-v6-mkmlizer: ║                                                                     ║
oliveiratime-axolotl-v6-mkmlizer: ║  Version: 0.10.1                                                    ║
oliveiratime-axolotl-v6-mkmlizer: ║  Copyright 2023 MK ONE TECHNOLOGIES Inc.                            ║
oliveiratime-axolotl-v6-mkmlizer: ║  https://mk1.ai                                                     ║
oliveiratime-axolotl-v6-mkmlizer: ║                                                                     ║
oliveiratime-axolotl-v6-mkmlizer: ║  The license key for the current software has been verified as      ║
oliveiratime-axolotl-v6-mkmlizer: ║  belonging to:                                                      ║
oliveiratime-axolotl-v6-mkmlizer: ║                                                                     ║
oliveiratime-axolotl-v6-mkmlizer: ║  Chai Research Corp.                                                ║
oliveiratime-axolotl-v6-mkmlizer: ║  Account ID: 7997a29f-0ceb-4cc7-9adf-840c57b4ae6f                   ║
oliveiratime-axolotl-v6-mkmlizer: ║  Expiration: 2024-10-15 23:59:59                                    ║
oliveiratime-axolotl-v6-mkmlizer: ║                                                                     ║
oliveiratime-axolotl-v6-mkmlizer: ╚═════════════════════════════════════════════════════════════════════╝
oliveiratime-axolotl-v6-mkmlizer: Downloaded to shared memory in 7.500s
oliveiratime-axolotl-v6-mkmlizer: quantizing model to /dev/shm/model_cache, profile:s0, folder:/tmp/tmpwh1yyirt, device:0
oliveiratime-axolotl-v6-mkmlizer: Saving flywheel model at /dev/shm/model_cache
oliveiratime-axolotl-v6-mkmlizer: Traceback (most recent call last):
oliveiratime-axolotl-v6-mkmlizer:   File "/code/uploading/mkmlize.py", line 151, in <module>
oliveiratime-axolotl-v6-mkmlizer:     cli()
oliveiratime-axolotl-v6-mkmlizer:   File "/opt/conda/lib/python3.10/site-packages/click/core.py", line 1157, in __call__
oliveiratime-axolotl-v6-mkmlizer:     return self.main(*args, **kwargs)
oliveiratime-axolotl-v6-mkmlizer:   File "/opt/conda/lib/python3.10/site-packages/click/core.py", line 1078, in main
oliveiratime-axolotl-v6-mkmlizer:     rv = self.invoke(ctx)
oliveiratime-axolotl-v6-mkmlizer:   File "/opt/conda/lib/python3.10/site-packages/click/core.py", line 1688, in invoke
oliveiratime-axolotl-v6-mkmlizer:     return _process_result(sub_ctx.command.invoke(sub_ctx))
oliveiratime-axolotl-v6-mkmlizer:   File "/opt/conda/lib/python3.10/site-packages/click/core.py", line 1434, in invoke
oliveiratime-axolotl-v6-mkmlizer:     return ctx.invoke(self.callback, **ctx.params)
oliveiratime-axolotl-v6-mkmlizer:   File "/opt/conda/lib/python3.10/site-packages/click/core.py", line 783, in invoke
oliveiratime-axolotl-v6-mkmlizer:     return __callback(*args, **kwargs)
oliveiratime-axolotl-v6-mkmlizer:   File "/code/uploading/mkmlize.py", line 42, in quantize
oliveiratime-axolotl-v6-mkmlizer:     quantize_model(temp_folder, output_path, profile, device)
oliveiratime-axolotl-v6-mkmlizer:   File "/code/uploading/mkmlize.py", line 135, in quantize_model
oliveiratime-axolotl-v6-mkmlizer:     flywheel.instrument(
oliveiratime-axolotl-v6-mkmlizer:   File "/opt/conda/lib/python3.10/site-packages/mk1/flywheel/instrument.py", line 90, in instrument
oliveiratime-axolotl-v6-mkmlizer:     compiler.save_pretrained(input_model_path, output_model_path, storage_format)
oliveiratime-axolotl-v6-mkmlizer:   File "/opt/conda/lib/python3.10/site-packages/mk1/flywheel/functional/compiler.py", line 23, in save_pretrained
oliveiratime-axolotl-v6-mkmlizer:     self.save_st_pretrained(input_model_path, output_model_path)
oliveiratime-axolotl-v6-mkmlizer:   File "/opt/conda/lib/python3.10/site-packages/mk1/flywheel/functional/compiler.py", line 31, in save_st_pretrained
oliveiratime-axolotl-v6-mkmlizer:     model_iterator = self.factory.model_iterator()
oliveiratime-axolotl-v6-mkmlizer:   File "/opt/conda/lib/python3.10/site-packages/mk1/flywheel/functional/loader.py", line 255, in model_iterator
oliveiratime-axolotl-v6-mkmlizer:     raise RuntimeError(f"Unable to construct an interator for the model")
oliveiratime-axolotl-v6-mkmlizer: RuntimeError: Unable to construct an interator for the model
Job oliveiratime-axolotl-v6-mkmlizer completed after 23.63s with status: failed
Stopping job with name oliveiratime-axolotl-v6-mkmlizer
MKMLizerError('')
oliveiratime-axolotl_v6 status is now failed due to DeploymentManager action
admin requested tearing down of oliveiratime-axolotl_v6
Running pipeline stage ISVCDeleter
Running pipeline stage ISVCDeleter
Skipping teardown as no inference service was successfully deployed
Pipeline stage ISVCDeleter completed in 0.22s
Running pipeline stage MKMLModelDeleter
Skipping deletion as no model was successfully uploaded
Pipeline stage MKMLModelDeleter completed in 0.24s
oliveiratime-axolotl_v6 status is now torndown due to DeploymentManager action