submission_id: sao10k-12b-spiced-v1_v1
developer_uid: sao10k
status: torndown
model_repo: Sao10K/12B-Spiced-v1
generation_params: {'temperature': 0.75, 'top_p': 1.0, 'min_p': 0.1, 'top_k': 40, 'presence_penalty': 0.0, 'frequency_penalty': 0.0, 'stopping_words': ['\n', '\n\n', '\nYou:', '[/INST]', '<|im_end|>', '</s>'], 'max_input_tokens': 512, 'best_of': 8, 'max_output_tokens': 64}
formatter: {'memory_template': '<|im_start|>system\n{memory}<|im_end|>\n', 'prompt_template': '<|im_start|>user\n{prompt}<|im_end|>\n', 'bot_template': '<|im_start|>assistant\n{bot_name}: {message}<|im_end|>\n', 'user_template': '<|im_start|>user\n{user_name}: {message}<|im_end|>\n', 'response_template': '<|im_start|>assistant\n{bot_name}:', 'truncate_by_message': False}
timestamp: 2024-10-16T22:54:18+00:00
model_name: spice-1-fixed-chatml
Resubmit model
Shutdown handler not registered because Python interpreter is not running in the main thread
run pipeline %s
run pipeline stage %s
Running pipeline stage MKMLizer
Starting job with name sao10k-12b-spiced-v1-v1-mkmlizer
Waiting for job on sao10k-12b-spiced-v1-v1-mkmlizer to finish
sao10k-12b-spiced-v1-v1-mkmlizer: ╔═════════════════════════════════════════════════════════════════════╗
sao10k-12b-spiced-v1-v1-mkmlizer: ║ _____ __ __ ║
sao10k-12b-spiced-v1-v1-mkmlizer: ║ / _/ /_ ___ __/ / ___ ___ / / ║
sao10k-12b-spiced-v1-v1-mkmlizer: ║ / _/ / // / |/|/ / _ \/ -_) -_) / ║
sao10k-12b-spiced-v1-v1-mkmlizer: ║ /_//_/\_, /|__,__/_//_/\__/\__/_/ ║
sao10k-12b-spiced-v1-v1-mkmlizer: ║ /___/ ║
sao10k-12b-spiced-v1-v1-mkmlizer: ║ ║
sao10k-12b-spiced-v1-v1-mkmlizer: ║ Version: 0.11.12 ║
sao10k-12b-spiced-v1-v1-mkmlizer: ║ Copyright 2023 MK ONE TECHNOLOGIES Inc. ║
sao10k-12b-spiced-v1-v1-mkmlizer: ║ https://mk1.ai ║
sao10k-12b-spiced-v1-v1-mkmlizer: ║ ║
sao10k-12b-spiced-v1-v1-mkmlizer: ║ The license key for the current software has been verified as ║
sao10k-12b-spiced-v1-v1-mkmlizer: ║ belonging to: ║
sao10k-12b-spiced-v1-v1-mkmlizer: ║ ║
sao10k-12b-spiced-v1-v1-mkmlizer: ║ Chai Research Corp. ║
sao10k-12b-spiced-v1-v1-mkmlizer: ║ Account ID: 7997a29f-0ceb-4cc7-9adf-840c57b4ae6f ║
sao10k-12b-spiced-v1-v1-mkmlizer: ║ Expiration: 2025-01-15 23:59:59 ║
sao10k-12b-spiced-v1-v1-mkmlizer: ║ ║
sao10k-12b-spiced-v1-v1-mkmlizer: ╚═════════════════════════════════════════════════════════════════════╝
Connection pool is full, discarding connection: %s. Connection pool size: %s
sao10k-12b-spiced-v1-v1-mkmlizer: ╔═════════════════════════════════════════════════════════════════════╗
sao10k-12b-spiced-v1-v1-mkmlizer: ║ _____ __ __ ║
sao10k-12b-spiced-v1-v1-mkmlizer: ║ / _/ /_ ___ __/ / ___ ___ / / ║
sao10k-12b-spiced-v1-v1-mkmlizer: ║ / _/ / // / |/|/ / _ \/ -_) -_) / ║
sao10k-12b-spiced-v1-v1-mkmlizer: ║ /_//_/\_, /|__,__/_//_/\__/\__/_/ ║
sao10k-12b-spiced-v1-v1-mkmlizer: ║ /___/ ║
sao10k-12b-spiced-v1-v1-mkmlizer: ║ ║
sao10k-12b-spiced-v1-v1-mkmlizer: ║ Version: 0.11.12 ║
sao10k-12b-spiced-v1-v1-mkmlizer: ║ Copyright 2023 MK ONE TECHNOLOGIES Inc. ║
sao10k-12b-spiced-v1-v1-mkmlizer: ║ https://mk1.ai ║
sao10k-12b-spiced-v1-v1-mkmlizer: ║ ║
sao10k-12b-spiced-v1-v1-mkmlizer: ║ The license key for the current software has been verified as ║
sao10k-12b-spiced-v1-v1-mkmlizer: ║ belonging to: ║
sao10k-12b-spiced-v1-v1-mkmlizer: ║ ║
sao10k-12b-spiced-v1-v1-mkmlizer: ║ Chai Research Corp. ║
sao10k-12b-spiced-v1-v1-mkmlizer: ║ Account ID: 7997a29f-0ceb-4cc7-9adf-840c57b4ae6f ║
sao10k-12b-spiced-v1-v1-mkmlizer: ║ Expiration: 2025-01-15 23:59:59 ║
sao10k-12b-spiced-v1-v1-mkmlizer: ║ ║
sao10k-12b-spiced-v1-v1-mkmlizer: ╚═════════════════════════════════════════════════════════════════════╝
Job sao10k-12b-spiced-v1-v1-mkmlizer completed after 67.51s with status: failed
Stopping job with name sao10k-12b-spiced-v1-v1-mkmlizer
%s, retrying in %s seconds...
Starting job with name sao10k-12b-spiced-v1-v1-mkmlizer
Waiting for job on sao10k-12b-spiced-v1-v1-mkmlizer to finish
Stopping job with name sao10k-12b-spiced-v1-v1-mkmlizer
%s, retrying in %s seconds...
Stopping job with name sao10k-12b-spiced-v1-v1-mkmlizer
%s, retrying in %s seconds...
Stopping job with name sao10k-12b-spiced-v1-v1-mkmlizer
%s, retrying in %s seconds...
Starting job with name sao10k-12b-spiced-v1-v1-mkmlizer
Stopping job with name sao10k-12b-spiced-v1-v1-mkmlizer
%s, retrying in %s seconds...
Stopping job with name sao10k-12b-spiced-v1-v1-mkmlizer
clean up pipeline due to error=MKMLizerError('(500)\nReason: Internal Server Error\nHTTP response headers: HTTPHeaderDict({\'Audit-Id\': \'183c4f18-bc63-41bf-81f7-468c31a356f4, db75bc31-a8e9-41fd-a253-4b45e5e9b7f7\', \'Cache-Control\': \'no-cache, private, no-cache, private\', \'Content-Length\': \'531\', \'Content-Type\': \'application/json\', \'Date\': \'Wed, 16 Oct 2024 22:56:54 GMT\', \'X-Kubernetes-Pf-Flowschema-Uid\': \'514c121f-0f8a-452c-aa56-437270a02244\', \'X-Kubernetes-Pf-Prioritylevel-Uid\': \'48ad322a-4034-4c03-9ea4-7745e1e2c31a\'})\nHTTP response body: {"kind":"Status","apiVersion":"v1","metadata":{},"status":"Failure","message":"Internal error occurred: failed calling webhook \\"validate.kyverno.svc-fail\\": failed to call webhook: Post \\"https://kyverno-svc.kyverno.svc:443/validate/fail?timeout=10s\\": context deadline exceeded","reason":"InternalError","details":{"causes":[{"message":"failed calling webhook \\"validate.kyverno.svc-fail\\": failed to call webhook: Post \\"https://kyverno-svc.kyverno.svc:443/validate/fail?timeout=10s\\": context deadline exceeded"}]},"code":500}\n\n')
Shutdown handler de-registered
sao10k-12b-spiced-v1_v1 status is now failed due to DeploymentManager action
sao10k-12b-spiced-v1_v1 status is now torndown due to DeploymentManager action