mlabonne-neuralmarcoro14-7b

submission_id: mlabonne-neuralmarcoro14-7b_v53
developer_uid: end_to_end_test
formatter: {'memory_template': "{bot_name}'s Persona: {memory}\n####\n", 'prompt_template': '{prompt}\n<START>\n', 'bot_template': '{bot_name}: {message}\n', 'user_template': '{user_name}: {message}\n', 'response_template': '{bot_name}:', 'truncate_by_message': False}
generation_params: {'temperature': 1.0, 'top_p': 0.99, 'min_p': 0.1, 'top_k': 40, 'presence_penalty': 0.0, 'frequency_penalty': 0.0, 'stopping_words': ['\n'], 'max_input_tokens': 512, 'best_of': 4, 'max_output_tokens': 64}
model_name: mlabonne-neuralmarcoro14-7b_v53
model_repo: mlabonne/NeuralMarcoro14-7B
status: torndown
timestamp: 2024-10-21T15:16:38+00:00
Resubmit model
Shutdown handler not registered because Python interpreter is not running in the main thread
run pipeline %s
run pipeline stage %s
Running pipeline stage MKMLizer
Starting job with name mlabonne-neuralmarcoro14-7b-v53-mkmlizer
Waiting for job on mlabonne-neuralmarcoro14-7b-v53-mkmlizer to finish
mlabonne-neuralmarcoro14-7b-v53-mkmlizer: ╔═════════════════════════════════════════════════════════════════════╗
mlabonne-neuralmarcoro14-7b-v53-mkmlizer: ║     _____            __           __                                ║
mlabonne-neuralmarcoro14-7b-v53-mkmlizer: ║    / _/ /_ ___    __/ /  ___ ___ / /                                ║
mlabonne-neuralmarcoro14-7b-v53-mkmlizer: ║   / _/ / // / |/|/ / _ \/ -_) -_) /                                 ║
mlabonne-neuralmarcoro14-7b-v53-mkmlizer: ║  /_//_/\_, /|__,__/_//_/\__/\__/_/                                  ║
mlabonne-neuralmarcoro14-7b-v53-mkmlizer: ║       /___/                                                         ║
mlabonne-neuralmarcoro14-7b-v53-mkmlizer: ║                                                                     ║
mlabonne-neuralmarcoro14-7b-v53-mkmlizer: ║  Version: 0.11.12                                                   ║
mlabonne-neuralmarcoro14-7b-v53-mkmlizer: ║  Copyright 2023 MK ONE TECHNOLOGIES Inc.                            ║
mlabonne-neuralmarcoro14-7b-v53-mkmlizer: ║  https://mk1.ai                                                     ║
mlabonne-neuralmarcoro14-7b-v53-mkmlizer: ║                                                                     ║
mlabonne-neuralmarcoro14-7b-v53-mkmlizer: ║  The license key for the current software has been verified as      ║
mlabonne-neuralmarcoro14-7b-v53-mkmlizer: ║  belonging to:                                                      ║
mlabonne-neuralmarcoro14-7b-v53-mkmlizer: ║                                                                     ║
mlabonne-neuralmarcoro14-7b-v53-mkmlizer: ║  Chai Research Corp.                                                ║
mlabonne-neuralmarcoro14-7b-v53-mkmlizer: ║  Account ID: 7997a29f-0ceb-4cc7-9adf-840c57b4ae6f                   ║
mlabonne-neuralmarcoro14-7b-v53-mkmlizer: ║  Expiration: 2025-01-15 23:59:59                                    ║
mlabonne-neuralmarcoro14-7b-v53-mkmlizer: ║                                                                     ║
mlabonne-neuralmarcoro14-7b-v53-mkmlizer: ╚═════════════════════════════════════════════════════════════════════╝
mlabonne-neuralmarcoro14-7b-v53-mkmlizer: Downloaded to shared memory in 28.094s
mlabonne-neuralmarcoro14-7b-v53-mkmlizer: quantizing model to /dev/shm/model_cache, profile:s0, folder:/tmp/tmprmwged8m, device:0
mlabonne-neuralmarcoro14-7b-v53-mkmlizer: Saving flywheel model at /dev/shm/model_cache
mlabonne-neuralmarcoro14-7b-v53-mkmlizer: quantized model in 16.935s
mlabonne-neuralmarcoro14-7b-v53-mkmlizer: Processed model mlabonne/NeuralMarcoro14-7B in 45.029s
mlabonne-neuralmarcoro14-7b-v53-mkmlizer: creating bucket guanaco-mkml-models
mlabonne-neuralmarcoro14-7b-v53-mkmlizer: Bucket 's3://guanaco-mkml-models/' created
mlabonne-neuralmarcoro14-7b-v53-mkmlizer: uploading /dev/shm/model_cache to s3://guanaco-mkml-models/mlabonne-neuralmarcoro14-7b-v53
mlabonne-neuralmarcoro14-7b-v53-mkmlizer: cp /dev/shm/model_cache/flywheel_model.0.safetensors s3://guanaco-mkml-models/mlabonne-neuralmarcoro14-7b-v53/flywheel_model.0.safetensors
mlabonne-neuralmarcoro14-7b-v53-mkmlizer: 
Loading 0:   0%|          | 0/291 [00:00<?, ?it/s]
Loading 0:   2%|▏         | 5/291 [00:00<00:07, 35.78it/s]
Loading 0:   4%|▍         | 13/291 [00:00<00:04, 56.52it/s]
Loading 0:   7%|▋         | 19/291 [00:00<00:06, 42.28it/s]
Loading 0:   8%|▊         | 24/291 [00:00<00:06, 42.20it/s]
Loading 0:  10%|▉         | 29/291 [00:00<00:06, 37.65it/s]
Loading 0:  13%|█▎        | 38/291 [00:00<00:05, 43.94it/s]
Loading 0:  16%|█▌        | 47/291 [00:01<00:05, 48.24it/s]
Loading 0:  19%|█▉        | 56/291 [00:01<00:04, 51.39it/s]
Loading 0:  22%|██▏       | 65/291 [00:01<00:04, 52.21it/s]
Loading 0:  25%|██▌       | 74/291 [00:01<00:04, 52.28it/s]
Loading 0:  28%|██▊       | 82/291 [00:01<00:03, 57.01it/s]
Loading 0:  30%|███       | 88/291 [00:01<00:03, 51.93it/s]
Loading 0:  32%|███▏      | 94/291 [00:01<00:03, 52.55it/s]
Loading 0:  34%|███▍      | 100/291 [00:02<00:04, 40.81it/s]
Loading 0:  36%|███▌      | 105/291 [00:02<00:04, 40.45it/s]
Loading 0:  38%|███▊      | 112/291 [00:02<00:03, 45.90it/s]
Loading 0:  41%|████      | 118/291 [00:02<00:03, 44.92it/s]
Loading 0:  42%|████▏     | 123/291 [00:02<00:03, 45.31it/s]
Loading 0:  45%|████▍     | 130/291 [00:02<00:03, 51.26it/s]
Loading 0:  47%|████▋     | 136/291 [00:02<00:03, 47.05it/s]
Loading 0:  48%|████▊     | 141/291 [00:03<00:03, 42.66it/s]
Loading 0:  51%|█████     | 148/291 [00:03<00:02, 48.06it/s]
Loading 0:  53%|█████▎    | 154/291 [00:03<00:02, 47.09it/s]
Loading 0:  55%|█████▍    | 159/291 [00:03<00:02, 46.52it/s]
Loading 0:  57%|█████▋    | 166/291 [00:03<00:02, 50.86it/s]
Loading 0:  59%|█████▉    | 172/291 [00:03<00:02, 48.82it/s]
Loading 0:  61%|██████    | 177/291 [00:03<00:02, 48.40it/s]
Loading 0:  63%|██████▎   | 184/291 [00:03<00:02, 52.27it/s]
Loading 0:  65%|██████▌   | 190/291 [00:03<00:02, 49.13it/s]
Loading 0:  67%|██████▋   | 195/291 [00:04<00:02, 44.98it/s]
Loading 0:  70%|██████▉   | 203/291 [00:04<00:01, 47.29it/s]
Loading 0:  71%|███████▏  | 208/291 [00:05<00:08, 10.34it/s]
Loading 0:  73%|███████▎  | 213/291 [00:06<00:06, 12.98it/s]
Loading 0:  76%|███████▌  | 221/291 [00:06<00:03, 18.88it/s]
Loading 0:  78%|███████▊  | 227/291 [00:06<00:02, 22.68it/s]
Loading 0:  80%|███████▉  | 232/291 [00:06<00:02, 25.58it/s]
Loading 0:  82%|████████▏ | 239/291 [00:06<00:01, 32.35it/s]
Loading 0:  84%|████████▍ | 245/291 [00:06<00:01, 35.73it/s]
Loading 0:  86%|████████▋ | 251/291 [00:06<00:00, 40.41it/s]
Loading 0:  89%|████████▊ | 258/291 [00:06<00:00, 41.27it/s]
Loading 0:  91%|█████████▏| 266/291 [00:06<00:00, 48.32it/s]
Loading 0:  93%|█████████▎| 272/291 [00:07<00:00, 46.17it/s]
Loading 0:  96%|█████████▌| 278/291 [00:07<00:00, 48.75it/s]
Loading 0:  98%|█████████▊| 284/291 [00:07<00:00, 51.50it/s]
Loading 0: 100%|█████████▉| 290/291 [00:07<00:00, 49.11it/s]
                                                            
Job mlabonne-neuralmarcoro14-7b-v53-mkmlizer completed after 68.62s with status: succeeded
Stopping job with name mlabonne-neuralmarcoro14-7b-v53-mkmlizer
Pipeline stage MKMLizer completed in 70.02s
run pipeline stage %s
Running pipeline stage MKMLTemplater
Pipeline stage MKMLTemplater completed in 0.39s
run pipeline stage %s
Running pipeline stage MKMLDeployer
%s, retrying in %s seconds...
%s, retrying in %s seconds...
clean up pipeline due to error=DeploymentError("No matches found for {'api_version': 'serving.kserve.io/v1beta1', 'kind': 'InferenceService'}")
Shutdown handler de-registered
mlabonne-neuralmarcoro14-7b_v53 status is now failed due to DeploymentManager action
mlabonne-neuralmarcoro14-7b_v53 status is now torndown due to DeploymentManager action