submission_id: sao10k-12b-spiced-v1_v2
developer_uid: sao10k
formatter: {'memory_template': '<|im_start|>system\n{memory}<|im_end|>\n', 'prompt_template': '<|im_start|>user\n{prompt}<|im_end|>\n', 'bot_template': '<|im_start|>assistant\n{bot_name}: {message}<|im_end|>\n', 'user_template': '<|im_start|>user\n{user_name}: {message}<|im_end|>\n', 'response_template': '<|im_start|>assistant\n{bot_name}:', 'truncate_by_message': False}
generation_params: {'temperature': 0.75, 'top_p': 1.0, 'min_p': 0.1, 'top_k': 40, 'presence_penalty': 0.0, 'frequency_penalty': 0.0, 'stopping_words': ['\n', '\n\n', '\nYou:', '[/INST]', '<|im_end|>', '</s>'], 'max_input_tokens': 512, 'best_of': 8, 'max_output_tokens': 64}
model_name: spice-1-fixed-chatml
model_repo: Sao10K/12B-Spiced-v1
status: torndown
timestamp: 2024-10-17T00:01:33+00:00
Resubmit model
Shutdown handler not registered because Python interpreter is not running in the main thread
run pipeline %s
run pipeline stage %s
Running pipeline stage MKMLizer
Starting job with name sao10k-12b-spiced-v1-v2-mkmlizer
Waiting for job on sao10k-12b-spiced-v1-v2-mkmlizer to finish
sao10k-12b-spiced-v1-v2-mkmlizer: ╔═════════════════════════════════════════════════════════════════════╗
sao10k-12b-spiced-v1-v2-mkmlizer: ║ _____ __ __ ║
sao10k-12b-spiced-v1-v2-mkmlizer: ║ / _/ /_ ___ __/ / ___ ___ / / ║
sao10k-12b-spiced-v1-v2-mkmlizer: ║ / _/ / // / |/|/ / _ \/ -_) -_) / ║
sao10k-12b-spiced-v1-v2-mkmlizer: ║ /_//_/\_, /|__,__/_//_/\__/\__/_/ ║
sao10k-12b-spiced-v1-v2-mkmlizer: ║ /___/ ║
sao10k-12b-spiced-v1-v2-mkmlizer: ║ ║
sao10k-12b-spiced-v1-v2-mkmlizer: ║ Version: 0.11.12 ║
sao10k-12b-spiced-v1-v2-mkmlizer: ║ Copyright 2023 MK ONE TECHNOLOGIES Inc. ║
sao10k-12b-spiced-v1-v2-mkmlizer: ║ https://mk1.ai ║
sao10k-12b-spiced-v1-v2-mkmlizer: ║ ║
sao10k-12b-spiced-v1-v2-mkmlizer: ║ The license key for the current software has been verified as ║
sao10k-12b-spiced-v1-v2-mkmlizer: ║ belonging to: ║
sao10k-12b-spiced-v1-v2-mkmlizer: ║ ║
sao10k-12b-spiced-v1-v2-mkmlizer: ║ Chai Research Corp. ║
sao10k-12b-spiced-v1-v2-mkmlizer: ║ Account ID: 7997a29f-0ceb-4cc7-9adf-840c57b4ae6f ║
sao10k-12b-spiced-v1-v2-mkmlizer: ║ Expiration: 2025-01-15 23:59:59 ║
sao10k-12b-spiced-v1-v2-mkmlizer: ║ ║
sao10k-12b-spiced-v1-v2-mkmlizer: ╚═════════════════════════════════════════════════════════════════════╝
sao10k-12b-spiced-v1-v2-mkmlizer: Downloaded to shared memory in 24.641s
sao10k-12b-spiced-v1-v2-mkmlizer: quantizing model to /dev/shm/model_cache, profile:s0, folder:/tmp/tmpp6cwm8dy, device:0
sao10k-12b-spiced-v1-v2-mkmlizer: Saving flywheel model at /dev/shm/model_cache
sao10k-12b-spiced-v1-v2-mkmlizer: Traceback (most recent call last):
sao10k-12b-spiced-v1-v2-mkmlizer: File "/code/uploading/mkmlize.py", line 151, in <module>
sao10k-12b-spiced-v1-v2-mkmlizer: cli()
sao10k-12b-spiced-v1-v2-mkmlizer: File "/opt/conda/lib/python3.10/site-packages/click/core.py", line 1157, in __call__
sao10k-12b-spiced-v1-v2-mkmlizer: return self.main(*args, **kwargs)
sao10k-12b-spiced-v1-v2-mkmlizer: File "/opt/conda/lib/python3.10/site-packages/click/core.py", line 1078, in main
sao10k-12b-spiced-v1-v2-mkmlizer: rv = self.invoke(ctx)
sao10k-12b-spiced-v1-v2-mkmlizer: File "/opt/conda/lib/python3.10/site-packages/click/core.py", line 1688, in invoke
sao10k-12b-spiced-v1-v2-mkmlizer: return _process_result(sub_ctx.command.invoke(sub_ctx))
sao10k-12b-spiced-v1-v2-mkmlizer: File "/opt/conda/lib/python3.10/site-packages/click/core.py", line 1434, in invoke
sao10k-12b-spiced-v1-v2-mkmlizer: return ctx.invoke(self.callback, **ctx.params)
sao10k-12b-spiced-v1-v2-mkmlizer: File "/opt/conda/lib/python3.10/site-packages/click/core.py", line 783, in invoke
sao10k-12b-spiced-v1-v2-mkmlizer: return __callback(*args, **kwargs)
sao10k-12b-spiced-v1-v2-mkmlizer: File "/code/uploading/mkmlize.py", line 42, in quantize
sao10k-12b-spiced-v1-v2-mkmlizer: quantize_model(temp_folder, output_path, profile, device)
sao10k-12b-spiced-v1-v2-mkmlizer: File "/code/uploading/mkmlize.py", line 135, in quantize_model
sao10k-12b-spiced-v1-v2-mkmlizer: flywheel.instrument(
sao10k-12b-spiced-v1-v2-mkmlizer: File "/opt/conda/lib/python3.10/site-packages/mk1/flywheel/instrument.py", line 93, in instrument
sao10k-12b-spiced-v1-v2-mkmlizer: compiler.save_pretrained(input_model_path, output_model_path, storage_format)
sao10k-12b-spiced-v1-v2-mkmlizer: File "/opt/conda/lib/python3.10/site-packages/mk1/flywheel/functional/compiler.py", line 23, in save_pretrained
sao10k-12b-spiced-v1-v2-mkmlizer: self.save_st_pretrained(input_model_path, output_model_path)
sao10k-12b-spiced-v1-v2-mkmlizer: File "/opt/conda/lib/python3.10/site-packages/mk1/flywheel/functional/compiler.py", line 31, in save_st_pretrained
sao10k-12b-spiced-v1-v2-mkmlizer: model_iterator = self.factory.model_iterator()
sao10k-12b-spiced-v1-v2-mkmlizer: File "/opt/conda/lib/python3.10/site-packages/mk1/flywheel/functional/loader.py", line 255, in model_iterator
sao10k-12b-spiced-v1-v2-mkmlizer: raise RuntimeError(f"Unable to construct an interator for the model")
sao10k-12b-spiced-v1-v2-mkmlizer: RuntimeError: Unable to construct an interator for the model
Job sao10k-12b-spiced-v1-v2-mkmlizer completed after 44.31s with status: failed
Stopping job with name sao10k-12b-spiced-v1-v2-mkmlizer
%s, retrying in %s seconds...
Starting job with name sao10k-12b-spiced-v1-v2-mkmlizer
Waiting for job on sao10k-12b-spiced-v1-v2-mkmlizer to finish
sao10k-12b-spiced-v1-v2-mkmlizer: Downloaded to shared memory in 24.641s
sao10k-12b-spiced-v1-v2-mkmlizer: quantizing model to /dev/shm/model_cache, profile:s0, folder:/tmp/tmpp6cwm8dy, device:0
sao10k-12b-spiced-v1-v2-mkmlizer: Saving flywheel model at /dev/shm/model_cache
sao10k-12b-spiced-v1-v2-mkmlizer: Traceback (most recent call last):
sao10k-12b-spiced-v1-v2-mkmlizer: File "/code/uploading/mkmlize.py", line 151, in <module>
sao10k-12b-spiced-v1-v2-mkmlizer: cli()
sao10k-12b-spiced-v1-v2-mkmlizer: File "/opt/conda/lib/python3.10/site-packages/click/core.py", line 1157, in __call__
sao10k-12b-spiced-v1-v2-mkmlizer: return self.main(*args, **kwargs)
sao10k-12b-spiced-v1-v2-mkmlizer: File "/opt/conda/lib/python3.10/site-packages/click/core.py", line 1078, in main
sao10k-12b-spiced-v1-v2-mkmlizer: rv = self.invoke(ctx)
sao10k-12b-spiced-v1-v2-mkmlizer: File "/opt/conda/lib/python3.10/site-packages/click/core.py", line 1688, in invoke
sao10k-12b-spiced-v1-v2-mkmlizer: return _process_result(sub_ctx.command.invoke(sub_ctx))
sao10k-12b-spiced-v1-v2-mkmlizer: File "/opt/conda/lib/python3.10/site-packages/click/core.py", line 1434, in invoke
sao10k-12b-spiced-v1-v2-mkmlizer: return ctx.invoke(self.callback, **ctx.params)
sao10k-12b-spiced-v1-v2-mkmlizer: File "/opt/conda/lib/python3.10/site-packages/click/core.py", line 783, in invoke
sao10k-12b-spiced-v1-v2-mkmlizer: return __callback(*args, **kwargs)
sao10k-12b-spiced-v1-v2-mkmlizer: File "/code/uploading/mkmlize.py", line 42, in quantize
sao10k-12b-spiced-v1-v2-mkmlizer: quantize_model(temp_folder, output_path, profile, device)
sao10k-12b-spiced-v1-v2-mkmlizer: File "/code/uploading/mkmlize.py", line 135, in quantize_model
sao10k-12b-spiced-v1-v2-mkmlizer: flywheel.instrument(
sao10k-12b-spiced-v1-v2-mkmlizer: File "/opt/conda/lib/python3.10/site-packages/mk1/flywheel/instrument.py", line 93, in instrument
sao10k-12b-spiced-v1-v2-mkmlizer: compiler.save_pretrained(input_model_path, output_model_path, storage_format)
sao10k-12b-spiced-v1-v2-mkmlizer: File "/opt/conda/lib/python3.10/site-packages/mk1/flywheel/functional/compiler.py", line 23, in save_pretrained
sao10k-12b-spiced-v1-v2-mkmlizer: self.save_st_pretrained(input_model_path, output_model_path)
sao10k-12b-spiced-v1-v2-mkmlizer: File "/opt/conda/lib/python3.10/site-packages/mk1/flywheel/functional/compiler.py", line 31, in save_st_pretrained
sao10k-12b-spiced-v1-v2-mkmlizer: model_iterator = self.factory.model_iterator()
sao10k-12b-spiced-v1-v2-mkmlizer: File "/opt/conda/lib/python3.10/site-packages/mk1/flywheel/functional/loader.py", line 255, in model_iterator
sao10k-12b-spiced-v1-v2-mkmlizer: raise RuntimeError(f"Unable to construct an interator for the model")
sao10k-12b-spiced-v1-v2-mkmlizer: RuntimeError: Unable to construct an interator for the model
Job sao10k-12b-spiced-v1-v2-mkmlizer completed after 53.42s with status: failed
Stopping job with name sao10k-12b-spiced-v1-v2-mkmlizer
%s, retrying in %s seconds...
Starting job with name sao10k-12b-spiced-v1-v2-mkmlizer
Waiting for job on sao10k-12b-spiced-v1-v2-mkmlizer to finish
Job sao10k-12b-spiced-v1-v2-mkmlizer completed after 51.88s with status: failed
Stopping job with name sao10k-12b-spiced-v1-v2-mkmlizer
clean up pipeline due to error=MKMLizerError('')
Shutdown handler de-registered
sao10k-12b-spiced-v1_v2 status is now failed due to DeploymentManager action
sao10k-12b-spiced-v1_v2 status is now torndown due to DeploymentManager action