Shutdown handler not registered because Python interpreter is not running in the main thread
run pipeline %s
run pipeline stage %s
Running pipeline stage MKMLizer
Starting job with name rirv938-llama-8b-scaleup-400k-v3-mkmlizer
Waiting for job on rirv938-llama-8b-scaleup-400k-v3-mkmlizer to finish
Shutdown handler not registered because Python interpreter is not running in the main thread
run pipeline %s
run pipeline stage %s
Running pipeline stage MKMLizer
Starting job with name rirv938-llama-8b-scaleup-200k-v1-mkmlizer
Waiting for job on rirv938-llama-8b-scaleup-200k-v1-mkmlizer to finish
rirv938-mistral-24b-sca-46352-v1-mkmlizer: ╔═════════════════════════════════════════════════════════════════════╗
rirv938-mistral-24b-sca-46352-v1-mkmlizer: ║ ║
rirv938-mistral-24b-sca-46352-v1-mkmlizer: ║ ██████ ██████ █████ ████ ████ ║
rirv938-mistral-24b-sca-46352-v1-mkmlizer: ║ ░░██████ ██████ ░░███ ███░ ░░███ ║
rirv938-mistral-24b-sca-46352-v1-mkmlizer: ║ ░███░█████░███ ░███ ███ ░███ ║
rirv938-mistral-24b-sca-46352-v1-mkmlizer: ║ ░███░░███ ░███ ░███████ ░███ ║
rirv938-mistral-24b-sca-46352-v1-mkmlizer: ║ ░███ ░░░ ░███ ░███░░███ ░███ ║
rirv938-mistral-24b-sca-46352-v1-mkmlizer: ║ ░███ ░███ ░███ ░░███ ░███ ║
rirv938-mistral-24b-sca-46352-v1-mkmlizer: ║ █████ █████ █████ ░░████ █████ ║
rirv938-mistral-24b-sca-46352-v1-mkmlizer: ║ ░░░░░ ░░░░░ ░░░░░ ░░░░ ░░░░░ ║
rirv938-mistral-24b-scal-2131-v4-mkmlizer: ╔═════════════════════════════════════════════════════════════════════╗
rirv938-mistral-24b-sca-46352-v1-mkmlizer: ║ ║
rirv938-mistral-24b-scal-2131-v4-mkmlizer: ║ ║
rirv938-mistral-24b-sca-46352-v1-mkmlizer: ║ Version: 0.25.10 ║
rirv938-mistral-24b-scal-2131-v4-mkmlizer: ║ ██████ ██████ █████ ████ ████ ║
rirv938-mistral-24b-sca-46352-v1-mkmlizer: ║ Features: FLYWHEEL, CUDA ║
rirv938-mistral-24b-scal-2131-v4-mkmlizer: ║ ░░██████ ██████ ░░███ ███░ ░░███ ║
rirv938-mistral-24b-sca-46352-v1-mkmlizer: ║ Copyright 2023-2025 MK ONE TECHNOLOGIES Inc. ║
rirv938-mistral-24b-scal-2131-v4-mkmlizer: ║ ░███░█████░███ ░███ ███ ░███ ║
rirv938-mistral-24b-sca-46352-v1-mkmlizer: ║ https://mk1.ai ║
rirv938-mistral-24b-scal-2131-v4-mkmlizer: ║ ░███░░███ ░███ ░███████ ░███ ║
rirv938-mistral-24b-sca-46352-v1-mkmlizer: ║ ║
rirv938-llama-8b-scaleup-400k-v3-mkmlizer: ╔═════════════════════════════════════════════════════════════════════╗
rirv938-mistral-24b-scal-2131-v4-mkmlizer: ║ ░███ ░░░ ░███ ░███░░███ ░███ ║
rirv938-mistral-24b-sca-46352-v1-mkmlizer: ║ The license key for the current software has been verified as ║
rirv938-llama-8b-scaleup-400k-v3-mkmlizer: ║ ║
rirv938-mistral-24b-scal-2131-v4-mkmlizer: ║ ░███ ░███ ░███ ░░███ ░███ ║
rirv938-mistral-24b-sca-46352-v1-mkmlizer: ║ belonging to: ║
rirv938-llama-8b-scaleup-400k-v3-mkmlizer: ║ ██████ ██████ █████ ████ ████ ║
rirv938-mistral-24b-scal-2131-v4-mkmlizer: ║ █████ █████ █████ ░░████ █████ ║
rirv938-mistral-24b-sca-46352-v1-mkmlizer: ║ ║
rirv938-llama-8b-scaleup-400k-v3-mkmlizer: ║ ░░██████ ██████ ░░███ ███░ ░░███ ║
rirv938-mistral-24b-scal-2131-v4-mkmlizer: ║ ░░░░░ ░░░░░ ░░░░░ ░░░░ ░░░░░ ║
rirv938-mistral-24b-sca-46352-v1-mkmlizer: ║ Chai Research Corp. ║
rirv938-llama-8b-scaleup-400k-v3-mkmlizer: ║ ░███░█████░███ ░███ ███ ░███ ║
rirv938-mistral-24b-scal-2131-v4-mkmlizer: ║ ║
rirv938-mistral-24b-sca-46352-v1-mkmlizer: ║ Account ID: 7997a29f-0ceb-4cc7-9adf-840c57b4ae6f ║
rirv938-llama-8b-scaleup-400k-v3-mkmlizer: ║ ░███░░███ ░███ ░███████ ░███ ║
rirv938-mistral-24b-scal-2131-v4-mkmlizer: ║ Version: 0.25.10 ║
rirv938-mistral-24b-sca-46352-v1-mkmlizer: ║ Expiration: 2025-04-15 23:59:59 ║
rirv938-llama-8b-scaleup-400k-v3-mkmlizer: ║ ░███ ░░░ ░███ ░███░░███ ░███ ║
rirv938-mistral-24b-scal-2131-v4-mkmlizer: ║ Features: FLYWHEEL, CUDA ║
rirv938-mistral-24b-sca-46352-v1-mkmlizer: ║ ║
rirv938-llama-8b-scaleup-400k-v3-mkmlizer: ║ ░███ ░███ ░███ ░░███ ░███ ║
rirv938-mistral-24b-scal-2131-v4-mkmlizer: ║ Copyright 2023-2025 MK ONE TECHNOLOGIES Inc. ║
rirv938-mistral-24b-sca-46352-v1-mkmlizer: ╚═════════════════════════════════════════════════════════════════════╝
rirv938-llama-8b-scaleup-400k-v3-mkmlizer: ║ █████ █████ █████ ░░████ █████ ║
rirv938-mistral-24b-scal-2131-v4-mkmlizer: ║ https://mk1.ai ║
rirv938-llama-8b-scaleup-400k-v3-mkmlizer: ║ ░░░░░ ░░░░░ ░░░░░ ░░░░ ░░░░░ ║
rirv938-mistral-24b-scal-2131-v4-mkmlizer: ║ ║
rirv938-llama-8b-scaleup-400k-v3-mkmlizer: ║ ║
rirv938-mistral-24b-scal-2131-v4-mkmlizer: ║ The license key for the current software has been verified as ║
rirv938-llama-8b-scaleup-400k-v3-mkmlizer: ║ Version: 0.25.10 ║
rirv938-mistral-24b-scal-2131-v4-mkmlizer: ║ belonging to: ║
rirv938-llama-8b-scaleup-400k-v3-mkmlizer: ║ Features: FLYWHEEL, CUDA ║
rirv938-mistral-24b-scal-2131-v4-mkmlizer: ║ ║
rirv938-llama-8b-scaleup-400k-v3-mkmlizer: ║ Copyright 2023-2025 MK ONE TECHNOLOGIES Inc. ║
rirv938-mistral-24b-scal-2131-v4-mkmlizer: ║ Chai Research Corp. ║
rirv938-llama-8b-scaleup-400k-v3-mkmlizer: ║ https://mk1.ai ║
rirv938-mistral-24b-scal-2131-v4-mkmlizer: ║ Account ID: 7997a29f-0ceb-4cc7-9adf-840c57b4ae6f ║
rirv938-llama-8b-scaleup-400k-v3-mkmlizer: ║ ║
rirv938-mistral-24b-scal-2131-v4-mkmlizer: ║ Expiration: 2025-04-15 23:59:59 ║
rirv938-llama-8b-scaleup-400k-v3-mkmlizer: ║ The license key for the current software has been verified as ║
rirv938-mistral-24b-scal-2131-v4-mkmlizer: ║ ║
rirv938-llama-8b-scaleup-400k-v3-mkmlizer: ║ belonging to: ║
rirv938-mistral-24b-scal-2131-v4-mkmlizer: ╚═════════════════════════════════════════════════════════════════════╝
rirv938-llama-8b-scaleup-400k-v3-mkmlizer: ║ ║
rirv938-llama-8b-scaleup-400k-v3-mkmlizer: ║ Chai Research Corp. ║
rirv938-llama-8b-scaleup-400k-v3-mkmlizer: ║ Account ID: 7997a29f-0ceb-4cc7-9adf-840c57b4ae6f ║
rirv938-llama-8b-scaleup-400k-v3-mkmlizer: ║ Expiration: 2025-04-15 23:59:59 ║
rirv938-llama-8b-scaleup-400k-v3-mkmlizer: ║ ║
rirv938-llama-8b-scaleup-400k-v3-mkmlizer: ╚═════════════════════════════════════════════════════════════════════╝
rirv938-llama-8b-scaleup-200k-v1-mkmlizer: ╔═════════════════════════════════════════════════════════════════════╗
rirv938-llama-8b-scaleup-200k-v1-mkmlizer: ║ ║
rirv938-llama-8b-scaleup-200k-v1-mkmlizer: ║ ██████ ██████ █████ ████ ████ ║
rirv938-llama-8b-scaleup-200k-v1-mkmlizer: ║ ░░██████ ██████ ░░███ ███░ ░░███ ║
rirv938-llama-8b-scaleup-200k-v1-mkmlizer: ║ ░███░█████░███ ░███ ███ ░███ ║
rirv938-llama-8b-scaleup-200k-v1-mkmlizer: ║ ░███░░███ ░███ ░███████ ░███ ║
rirv938-llama-8b-scaleup-200k-v1-mkmlizer: ║ ░███ ░░░ ░███ ░███░░███ ░███ ║
rirv938-llama-8b-scaleup-200k-v1-mkmlizer: ║ ░███ ░███ ░███ ░░███ ░███ ║
rirv938-llama-8b-scaleup-200k-v1-mkmlizer: ║ █████ █████ █████ ░░████ █████ ║
rirv938-llama-8b-scaleup-200k-v1-mkmlizer: ║ ░░░░░ ░░░░░ ░░░░░ ░░░░ ░░░░░ ║
rirv938-llama-8b-scaleup-200k-v1-mkmlizer: ║ ║
rirv938-llama-8b-scaleup-200k-v1-mkmlizer: ║ Version: 0.25.10 ║
rirv938-llama-8b-scaleup-200k-v1-mkmlizer: ║ Features: FLYWHEEL, CUDA ║
rirv938-llama-8b-scaleup-200k-v1-mkmlizer: ║ Copyright 2023-2025 MK ONE TECHNOLOGIES Inc. ║
rirv938-llama-8b-scaleup-200k-v1-mkmlizer: ║ https://mk1.ai ║
rirv938-llama-8b-scaleup-200k-v1-mkmlizer: ║ ║
rirv938-llama-8b-scaleup-200k-v1-mkmlizer: ║ The license key for the current software has been verified as ║
rirv938-llama-8b-scaleup-200k-v1-mkmlizer: ║ belonging to: ║
rirv938-llama-8b-scaleup-200k-v1-mkmlizer: ║ ║
rirv938-llama-8b-scaleup-200k-v1-mkmlizer: ║ Chai Research Corp. ║
rirv938-llama-8b-scaleup-200k-v1-mkmlizer: ║ Account ID: 7997a29f-0ceb-4cc7-9adf-840c57b4ae6f ║
rirv938-llama-8b-scaleup-200k-v1-mkmlizer: ║ Expiration: 2025-04-15 23:59:59 ║
rirv938-llama-8b-scaleup-200k-v1-mkmlizer: ║ ║
rirv938-llama-8b-scaleup-200k-v1-mkmlizer: ╚═════════════════════════════════════════════════════════════════════╝
rirv938-llama-8b-scaleup-400k-v3-mkmlizer: Downloaded to shared memory in 26.872s
rirv938-llama-8b-scaleup-400k-v3-mkmlizer: quantizing model to /dev/shm/model_cache, profile:t0, folder:/tmp/tmpoobp3nn1, device:0
rirv938-llama-8b-scaleup-400k-v3-mkmlizer: Saving flywheel model at /dev/shm/model_cache
rirv938-llama-8b-scaleup-200k-v1-mkmlizer: Downloaded to shared memory in 33.113s
rirv938-llama-8b-scaleup-200k-v1-mkmlizer: quantizing model to /dev/shm/model_cache, profile:t0, folder:/tmp/tmpcjulah14, device:0
rirv938-llama-8b-scaleup-200k-v1-mkmlizer: Saving flywheel model at /dev/shm/model_cache
rirv938-llama-8b-scaleup-400k-v3-mkmlizer: quantized model in 20.063s
rirv938-llama-8b-scaleup-400k-v3-mkmlizer: Processed model rirv938/llama_8b_scaleup_400k in 46.936s
rirv938-llama-8b-scaleup-400k-v3-mkmlizer: creating bucket guanaco-mkml-models
rirv938-llama-8b-scaleup-400k-v3-mkmlizer: Bucket 's3://guanaco-mkml-models/' created
rirv938-llama-8b-scaleup-400k-v3-mkmlizer: uploading /dev/shm/model_cache to s3://guanaco-mkml-models/rirv938-llama-8b-scaleup-400k-v3
rirv938-llama-8b-scaleup-400k-v3-mkmlizer: cp /dev/shm/model_cache/config.json s3://guanaco-mkml-models/rirv938-llama-8b-scaleup-400k-v3/config.json
rirv938-llama-8b-scaleup-400k-v3-mkmlizer: cp /dev/shm/model_cache/special_tokens_map.json s3://guanaco-mkml-models/rirv938-llama-8b-scaleup-400k-v3/special_tokens_map.json
rirv938-llama-8b-scaleup-400k-v3-mkmlizer: cp /dev/shm/model_cache/tokenizer_config.json s3://guanaco-mkml-models/rirv938-llama-8b-scaleup-400k-v3/tokenizer_config.json
rirv938-llama-8b-scaleup-400k-v3-mkmlizer: cp /dev/shm/model_cache/tokenizer.json s3://guanaco-mkml-models/rirv938-llama-8b-scaleup-400k-v3/tokenizer.json
rirv938-llama-8b-scaleup-200k-v1-mkmlizer: quantized model in 19.942s
rirv938-llama-8b-scaleup-200k-v1-mkmlizer: Processed model rirv938/llama_8b_scaleup_200k in 53.056s
rirv938-llama-8b-scaleup-200k-v1-mkmlizer: creating bucket guanaco-mkml-models
rirv938-llama-8b-scaleup-200k-v1-mkmlizer: Bucket 's3://guanaco-mkml-models/' created
rirv938-llama-8b-scaleup-200k-v1-mkmlizer: uploading /dev/shm/model_cache to s3://guanaco-mkml-models/rirv938-llama-8b-scaleup-200k-v1
rirv938-llama-8b-scaleup-200k-v1-mkmlizer: cp /dev/shm/model_cache/config.json s3://guanaco-mkml-models/rirv938-llama-8b-scaleup-200k-v1/config.json
rirv938-llama-8b-scaleup-200k-v1-mkmlizer: cp /dev/shm/model_cache/special_tokens_map.json s3://guanaco-mkml-models/rirv938-llama-8b-scaleup-200k-v1/special_tokens_map.json
rirv938-llama-8b-scaleup-200k-v1-mkmlizer: cp /dev/shm/model_cache/tokenizer_config.json s3://guanaco-mkml-models/rirv938-llama-8b-scaleup-200k-v1/tokenizer_config.json
rirv938-llama-8b-scaleup-200k-v1-mkmlizer: cp /dev/shm/model_cache/tokenizer.json s3://guanaco-mkml-models/rirv938-llama-8b-scaleup-200k-v1/tokenizer.json
rirv938-llama-8b-scaleup-200k-v1-mkmlizer: cp /dev/shm/model_cache/flywheel_model.0.safetensors s3://guanaco-mkml-models/rirv938-llama-8b-scaleup-200k-v1/flywheel_model.0.safetensors
rirv938-mistral-24b-sca-46352-v1-mkmlizer: Downloaded to shared memory in 83.036s
rirv938-mistral-24b-sca-46352-v1-mkmlizer: quantizing model to /dev/shm/model_cache, profile:t0, folder:/tmp/tmpcgwc94xl, device:0
rirv938-mistral-24b-sca-46352-v1-mkmlizer: Saving flywheel model at /dev/shm/model_cache
rirv938-mistral-24b-scal-2131-v4-mkmlizer: Downloaded to shared memory in 86.872s
rirv938-mistral-24b-scal-2131-v4-mkmlizer: quantizing model to /dev/shm/model_cache, profile:t0, folder:/tmp/tmp5c6g4zd_, device:0
rirv938-mistral-24b-scal-2131-v4-mkmlizer: Saving flywheel model at /dev/shm/model_cache
Job rirv938-llama-8b-scaleup-200k-v1-mkmlizer completed after 102.78s with status: succeeded
Stopping job with name rirv938-llama-8b-scaleup-200k-v1-mkmlizer
Pipeline stage MKMLizer completed in 105.31s
run pipeline stage %s
Running pipeline stage MKMLTemplater
Pipeline stage MKMLTemplater completed in 0.83s
run pipeline stage %s
Running pipeline stage MKMLDeployer
Creating inference service rirv938-llama-8b-scaleup-200k-v1
Waiting for inference service rirv938-llama-8b-scaleup-200k-v1 to be ready
Job rirv938-llama-8b-scaleup-400k-v3-mkmlizer completed after 112.55s with status: succeeded
Stopping job with name rirv938-llama-8b-scaleup-400k-v3-mkmlizer
Pipeline stage MKMLizer completed in 115.39s
run pipeline stage %s
Running pipeline stage MKMLTemplater
Pipeline stage MKMLTemplater completed in 1.04s
run pipeline stage %s
Running pipeline stage MKMLDeployer
Creating inference service rirv938-llama-8b-scaleup-400k-v3
Waiting for inference service rirv938-llama-8b-scaleup-400k-v3 to be ready
rirv938-mistral-24b-sca-46352-v1-mkmlizer: quantized model in 50.512s
rirv938-mistral-24b-sca-46352-v1-mkmlizer: Processed model rirv938/mistral_24b_scaleup_200k in 133.554s
rirv938-mistral-24b-sca-46352-v1-mkmlizer: creating bucket guanaco-mkml-models
rirv938-mistral-24b-sca-46352-v1-mkmlizer: Bucket 's3://guanaco-mkml-models/' created
rirv938-mistral-24b-sca-46352-v1-mkmlizer: uploading /dev/shm/model_cache to s3://guanaco-mkml-models/rirv938-mistral-24b-sca-46352-v1
rirv938-mistral-24b-sca-46352-v1-mkmlizer: cp /dev/shm/model_cache/config.json s3://guanaco-mkml-models/rirv938-mistral-24b-sca-46352-v1/config.json
rirv938-mistral-24b-sca-46352-v1-mkmlizer: cp /dev/shm/model_cache/special_tokens_map.json s3://guanaco-mkml-models/rirv938-mistral-24b-sca-46352-v1/special_tokens_map.json
rirv938-mistral-24b-scal-2131-v4-mkmlizer: quantized model in 50.314s
rirv938-mistral-24b-sca-46352-v1-mkmlizer: cp /dev/shm/model_cache/tokenizer_config.json s3://guanaco-mkml-models/rirv938-mistral-24b-sca-46352-v1/tokenizer_config.json
rirv938-mistral-24b-scal-2131-v4-mkmlizer: Processed model rirv938/mistral_24b_scaleup_400k in 137.187s
rirv938-mistral-24b-sca-46352-v1-mkmlizer: cp /dev/shm/model_cache/flywheel_model.2.safetensors s3://guanaco-mkml-models/rirv938-mistral-24b-sca-46352-v1/flywheel_model.2.safetensors
rirv938-mistral-24b-scal-2131-v4-mkmlizer: cp /dev/shm/model_cache/tokenizer_config.json s3://guanaco-mkml-models/rirv938-mistral-24b-scal-2131-v4/tokenizer_config.json
rirv938-mistral-24b-sca-46352-v1-mkmlizer: cp /dev/shm/model_cache/flywheel_model.0.safetensors s3://guanaco-mkml-models/rirv938-mistral-24b-sca-46352-v1/flywheel_model.0.safetensors
rirv938-mistral-24b-scal-2131-v4-mkmlizer: cp /dev/shm/model_cache/tokenizer.json s3://guanaco-mkml-models/rirv938-mistral-24b-scal-2131-v4/tokenizer.json
rirv938-mistral-24b-sca-46352-v1-mkmlizer: cp /dev/shm/model_cache/flywheel_model.1.safetensors s3://guanaco-mkml-models/rirv938-mistral-24b-sca-46352-v1/flywheel_model.1.safetensors
rirv938-mistral-24b-scal-2131-v4-mkmlizer: cp /dev/shm/model_cache/flywheel_model.2.safetensors s3://guanaco-mkml-models/rirv938-mistral-24b-scal-2131-v4/flywheel_model.2.safetensors
rirv938-mistral-24b-sca-46352-v1-mkmlizer:
Loading 0: 0%| | 0/363 [00:00<?, ?it/s]
Loading 0: 1%| | 4/363 [00:00<00:09, 37.63it/s]
Loading 0: 2%|▏ | 8/363 [00:00<00:12, 29.15it/s]
Loading 0: 3%|▎ | 12/363 [00:00<00:11, 30.53it/s]
Loading 0: 4%|▍ | 16/363 [00:00<00:12, 28.27it/s]
Loading 0: 6%|▌ | 21/363 [00:00<00:10, 32.20it/s]
Loading 0: 7%|▋ | 25/363 [00:00<00:11, 29.37it/s]
Loading 0: 8%|▊ | 30/363 [00:00<00:09, 34.21it/s]
Loading 0: 9%|▉ | 34/363 [00:01<00:14, 22.18it/s]
Loading 0: 10%|█ | 37/363 [00:01<00:15, 20.40it/s]
Loading 0: 11%|█▏ | 41/363 [00:01<00:16, 19.64it/s]
Loading 0: 13%|█▎ | 46/363 [00:01<00:12, 24.95it/s]
Loading 0: 14%|█▍ | 50/363 [00:02<00:14, 22.08it/s]
Loading 0: 16%|█▌ | 57/363 [00:02<00:11, 27.51it/s]
Loading 0: 17%|█▋ | 61/363 [00:02<00:11, 26.02it/s]
Loading 0: 18%|█▊ | 65/363 [00:02<00:11, 25.68it/s]
Loading 0: 19%|█▉ | 70/363 [00:02<00:13, 22.39it/s]
Loading 0: 20%|██ | 73/363 [00:03<00:15, 18.98it/s]
Loading 0: 21%|██▏ | 78/363 [00:03<00:11, 24.05it/s]
Loading 0: 22%|██▏ | 81/363 [00:03<00:13, 21.12it/s]
Loading 0: 24%|██▎ | 86/363 [00:03<00:11, 23.76it/s]
Loading 0: 25%|██▍ | 89/363 [00:03<00:11, 23.59it/s]
Loading 0: 25%|██▌ | 92/363 [00:03<00:13, 19.94it/s]
Loading 0: 27%|██▋ | 99/363 [00:04<00:10, 26.32it/s]
Loading 0: 28%|██▊ | 102/363 [00:04<00:11, 23.64it/s]
Loading 0: 29%|██▉ | 107/363 [00:04<00:11, 22.36it/s]
Loading 0: 31%|███ | 111/363 [00:04<00:09, 25.29it/s]
Loading 0: 31%|███▏ | 114/363 [00:04<00:10, 23.30it/s]
Loading 0: 33%|███▎ | 120/363 [00:04<00:08, 29.05it/s]
Loading 0: 34%|███▍ | 124/363 [00:05<00:08, 27.74it/s]
Loading 0: 36%|███▌ | 129/363 [00:05<00:07, 29.77it/s]
Loading 0: 37%|███▋ | 133/363 [00:05<00:08, 28.74it/s]
Loading 0: 38%|███▊ | 138/363 [00:05<00:07, 30.76it/s]
Loading 0: 39%|███▉ | 142/363 [00:05<00:07, 28.95it/s]
Loading 0: 39%|███▉ | 142/363 [00:20<00:07, 28.95it/s]
Loading 0: 39%|███▉ | 143/363 [00:20<04:52, 1.33s/it]
Loading 0: 40%|███▉ | 145/363 [00:20<03:53, 1.07s/it]
Loading 0: 41%|████ | 149/363 [00:20<02:28, 1.44it/s]
Loading 0: 42%|████▏ | 152/363 [00:20<01:50, 1.90it/s]
Loading 0: 43%|████▎ | 155/363 [00:20<01:20, 2.59it/s]
Loading 0: 44%|████▎ | 158/363 [00:21<01:00, 3.39it/s]
Loading 0: 45%|████▌ | 165/363 [00:21<00:31, 6.22it/s]
Loading 0: 46%|████▋ | 168/363 [00:21<00:26, 7.30it/s]
Loading 0: 48%|████▊ | 174/363 [00:21<00:17, 10.81it/s]
Loading 0: 49%|████▉ | 177/363 [00:21<00:15, 11.91it/s]
Loading 0: 50%|█████ | 182/363 [00:21<00:11, 15.66it/s]
Loading 0: 52%|█████▏ | 187/363 [00:22<00:10, 17.04it/s]
Loading 0: 52%|█████▏ | 190/363 [00:22<00:10, 17.25it/s]
Loading 0: 53%|█████▎ | 194/363 [00:22<00:09, 17.44it/s]
Loading 0: 55%|█████▌ | 201/363 [00:22<00:06, 23.88it/s]
Loading 0: 56%|█████▌ | 204/363 [00:22<00:07, 22.13it/s]
Loading 0: 58%|█████▊ | 210/363 [00:22<00:05, 27.07it/s]
Loading 0: 59%|█████▉ | 214/363 [00:23<00:05, 26.60it/s]
Loading 0: 60%|██████ | 218/363 [00:23<00:05, 27.30it/s]
Loading 0: 61%|██████ | 221/363 [00:23<00:05, 25.90it/s]
Loading 0: 62%|██████▏ | 224/363 [00:23<00:06, 20.36it/s]
Loading 0: 63%|██████▎ | 228/363 [00:23<00:05, 23.96it/s]
Loading 0: 64%|██████▎ | 231/363 [00:23<00:06, 21.36it/s]
Loading 0: 65%|██████▌ | 237/363 [00:24<00:04, 27.08it/s]
Loading 0: 66%|██████▌ | 240/363 [00:24<00:05, 24.42it/s]
Loading 0: 68%|██████▊ | 246/363 [00:24<00:04, 28.79it/s]
Loading 0: 69%|██████▉ | 250/363 [00:24<00:04, 27.43it/s]
Loading 0: 70%|███████ | 255/363 [00:24<00:03, 29.70it/s]
Loading 0: 71%|███████▏ | 259/363 [00:24<00:03, 27.08it/s]
Loading 0: 73%|███████▎ | 264/363 [00:25<00:03, 30.70it/s]
Loading 0: 74%|███████▍ | 268/363 [00:25<00:04, 22.51it/s]
Loading 0: 75%|███████▍ | 271/363 [00:25<00:04, 20.81it/s]
Loading 0: 75%|███████▌ | 274/363 [00:25<00:03, 22.37it/s]
Loading 0: 76%|███████▋ | 277/363 [00:25<00:03, 22.48it/s]
Loading 0: 78%|███████▊ | 282/363 [00:25<00:03, 25.33it/s]
Loading 0: 79%|███████▊ | 285/363 [00:26<00:03, 22.87it/s]
Loading 0: 80%|████████ | 291/363 [00:26<00:02, 28.08it/s]
Loading 0: 81%|████████ | 294/363 [00:26<00:02, 24.69it/s]
Loading 0: 82%|████████▏ | 299/363 [00:26<00:02, 27.39it/s]
Loading 0: 84%|████████▎ | 304/363 [00:26<00:02, 25.38it/s]
Loading 0: 85%|████████▍ | 307/363 [00:26<00:02, 23.03it/s]
Loading 0: 85%|████████▍ | 308/363 [00:41<00:02, 23.03it/s]
Loading 0: 85%|████████▌ | 309/363 [00:41<01:13, 1.35s/it]
Loading 0: 86%|████████▌ | 311/363 [00:42<00:57, 1.11s/it]
Loading 0: 87%|████████▋ | 316/363 [00:42<00:30, 1.52it/s]
Loading 0: 88%|████████▊ | 319/363 [00:42<00:22, 1.99it/s]
Loading 0: 89%|████████▊ | 322/363 [00:42<00:15, 2.65it/s]
Loading 0: 90%|████████▉ | 325/363 [00:42<00:10, 3.55it/s]
Loading 0: 90%|█████████ | 328/363 [00:42<00:07, 4.69it/s]
Loading 0: 91%|█████████ | 331/363 [00:43<00:05, 6.07it/s]
Loading 0: 92%|█████████▏| 335/363 [00:43<00:03, 8.27it/s]
Loading 0: 93%|█████████▎| 338/363 [00:43<00:02, 9.87it/s]
Loading 0: 94%|█████████▍| 341/363 [00:43<00:02, 10.24it/s]
Loading 0: 95%|█████████▌| 345/363 [00:43<00:01, 13.52it/s]
Loading 0: 96%|█████████▌| 348/363 [00:43<00:01, 13.63it/s]
Loading 0: 98%|█████████▊| 354/363 [00:44<00:00, 18.56it/s]
Loading 0: 98%|█████████▊| 357/363 [00:44<00:00, 17.88it/s]
Loading 0: 99%|█████████▉| 361/363 [00:44<00:00, 21.60it/s]
rirv938-mistral-24b-scal-2131-v4-mkmlizer: cp /dev/shm/model_cache/flywheel_model.1.safetensors s3://guanaco-mkml-models/rirv938-mistral-24b-scal-2131-v4/flywheel_model.1.safetensors
Job rirv938-mistral-24b-sca-46352-v1-mkmlizer completed after 175.04s with status: succeeded
rirv938-mistral-24b-scal-2131-v4-mkmlizer: cp /dev/shm/model_cache/flywheel_model.0.safetensors s3://guanaco-mkml-models/rirv938-mistral-24b-scal-2131-v4/flywheel_model.0.safetensors
Stopping job with name rirv938-mistral-24b-sca-46352-v1-mkmlizer
rirv938-mistral-24b-scal-2131-v4-mkmlizer:
Loading 0: 0%| | 0/363 [00:00<?, ?it/s]
Loading 0: 1%| | 4/363 [00:00<00:10, 34.90it/s]
Loading 0: 2%|▏ | 8/363 [00:00<00:10, 32.33it/s]
Loading 0: 3%|▎ | 12/363 [00:00<00:11, 31.31it/s]
Loading 0: 4%|▍ | 16/363 [00:00<00:12, 27.34it/s]
Loading 0: 6%|▌ | 21/363 [00:00<00:11, 30.80it/s]
Loading 0: 7%|▋ | 25/363 [00:00<00:11, 28.20it/s]
Loading 0: 8%|▊ | 30/363 [00:00<00:09, 33.39it/s]
Loading 0: 9%|▉ | 34/363 [00:01<00:13, 24.01it/s]
Loading 0: 10%|█ | 37/363 [00:01<00:14, 21.81it/s]
Loading 0: 11%|█ | 40/363 [00:01<00:13, 23.36it/s]
Loading 0: 12%|█▏ | 43/363 [00:01<00:13, 23.43it/s]
Loading 0: 13%|█▎ | 48/363 [00:01<00:11, 27.08it/s]
Loading 0: 14%|█▍ | 51/363 [00:01<00:13, 23.81it/s]
Loading 0: 16%|█▌ | 57/363 [00:02<00:10, 29.45it/s]
Loading 0: 17%|█▋ | 61/363 [00:02<00:10, 27.60it/s]
Loading 0: 18%|█▊ | 65/363 [00:02<00:10, 27.75it/s]
Loading 0: 19%|█▉ | 70/363 [00:02<00:11, 25.18it/s]
Loading 0: 20%|██ | 73/363 [00:02<00:13, 21.30it/s]
Loading 0: 22%|██▏ | 79/363 [00:02<00:10, 26.44it/s]
Loading 0: 23%|██▎ | 82/363 [00:03<00:10, 26.02it/s]
Loading 0: 24%|██▎ | 86/363 [00:03<00:10, 27.42it/s]
Loading 0: 25%|██▍ | 89/363 [00:03<00:09, 27.44it/s]
Loading 0: 25%|██▌ | 92/363 [00:03<00:12, 22.47it/s]
Loading 0: 27%|██▋ | 99/363 [00:03<00:08, 29.64it/s]
Loading 0: 28%|██▊ | 103/363 [00:03<00:09, 28.29it/s]
Loading 0: 29%|██▉ | 107/363 [00:04<00:10, 24.20it/s]
Loading 0: 31%|███ | 111/363 [00:04<00:09, 27.08it/s]
Loading 0: 31%|███▏ | 114/363 [00:04<00:10, 24.18it/s]
Loading 0: 33%|███▎ | 120/363 [00:04<00:08, 29.36it/s]
Loading 0: 34%|███▍ | 124/363 [00:04<00:08, 27.23it/s]
Loading 0: 36%|███▌ | 129/363 [00:04<00:07, 29.57it/s]
Loading 0: 37%|███▋ | 133/363 [00:04<00:08, 27.86it/s]
Loading 0: 38%|███▊ | 138/363 [00:05<00:07, 30.11it/s]
Loading 0: 39%|███▉ | 142/363 [00:05<00:07, 28.40it/s]
Loading 0: 40%|███▉ | 144/363 [00:20<00:07, 28.40it/s]
Loading 0: 40%|███▉ | 145/363 [00:20<04:11, 1.15s/it]
Loading 0: 41%|████ | 149/363 [00:20<02:54, 1.22it/s]
Loading 0: 42%|████▏ | 152/363 [00:20<02:14, 1.56it/s]
Loading 0: 43%|████▎ | 155/363 [00:20<01:40, 2.06it/s]
Loading 0: 44%|████▎ | 158/363 [00:20<01:16, 2.68it/s]
Loading 0: 45%|████▌ | 165/363 [00:21<00:40, 4.85it/s]
Loading 0: 46%|████▋ | 168/363 [00:21<00:33, 5.82it/s]
Loading 0: 48%|████▊ | 174/363 [00:21<00:21, 8.78it/s]
Loading 0: 49%|████▉ | 178/363 [00:21<00:17, 10.63it/s]
Loading 0: 50%|█████ | 182/363 [00:21<00:13, 13.01it/s]
Loading 0: 52%|█████▏ | 187/363 [00:21<00:11, 14.79it/s]
Loading 0: 52%|█████▏ | 190/363 [00:22<00:11, 15.35it/s]
Loading 0: 53%|█████▎ | 194/363 [00:22<00:10, 15.97it/s]
Loading 0: 55%|█████▌ | 201/363 [00:22<00:07, 22.33it/s]
Loading 0: 56%|█████▌ | 204/363 [00:22<00:07, 21.81it/s]
Loading 0: 58%|█████▊ | 210/363 [00:22<00:05, 27.19it/s]
Loading 0: 59%|█████▉ | 214/363 [00:22<00:05, 25.79it/s]
Loading 0: 60%|██████ | 218/363 [00:23<00:05, 25.92it/s]
Loading 0: 61%|██████ | 221/363 [00:23<00:05, 24.93it/s]
Loading 0: 62%|██████▏ | 224/363 [00:23<00:07, 19.43it/s]
Loading 0: 63%|██████▎ | 228/363 [00:23<00:05, 22.74it/s]
Loading 0: 64%|██████▎ | 231/363 [00:23<00:06, 21.00it/s]
Loading 0: 65%|██████▌ | 237/363 [00:23<00:04, 26.63it/s]
Loading 0: 66%|██████▌ | 240/363 [00:24<00:05, 23.77it/s]
Loading 0: 68%|██████▊ | 246/363 [00:24<00:04, 28.58it/s]
Loading 0: 69%|██████▉ | 250/363 [00:24<00:04, 26.84it/s]
Loading 0: 70%|███████ | 255/363 [00:24<00:03, 29.05it/s]
Loading 0: 71%|███████▏ | 259/363 [00:24<00:03, 27.33it/s]
Loading 0: 73%|███████▎ | 264/363 [00:24<00:03, 31.83it/s]
Loading 0: 74%|███████▍ | 268/363 [00:25<00:04, 23.42it/s]
Loading 0: 75%|███████▍ | 271/363 [00:25<00:04, 21.66it/s]
Loading 0: 75%|███████▌ | 274/363 [00:25<00:03, 23.17it/s]
Loading 0: 76%|███████▋ | 277/363 [00:25<00:03, 22.75it/s]
Loading 0: 77%|███████▋ | 280/363 [00:25<00:03, 23.50it/s]
Loading 0: 78%|███████▊ | 283/363 [00:25<00:03, 23.85it/s]
Loading 0: 79%|███████▉ | 286/363 [00:25<00:03, 23.43it/s]
Loading 0: 80%|████████ | 291/363 [00:26<00:02, 27.05it/s]
Loading 0: 81%|████████ | 294/363 [00:26<00:02, 23.16it/s]
Loading 0: 82%|████████▏ | 299/363 [00:26<00:02, 25.29it/s]
Loading 0: 84%|████████▎ | 304/363 [00:26<00:02, 22.08it/s]
Loading 0: 85%|████████▍ | 307/363 [00:26<00:02, 19.92it/s]
Loading 0: 85%|████████▍ | 308/363 [00:42<00:02, 19.92it/s]
Loading 0: 85%|████████▌ | 309/363 [00:42<01:16, 1.41s/it]
Loading 0: 86%|████████▌ | 311/363 [00:42<00:59, 1.14s/it]
Loading 0: 87%|████████▋ | 316/363 [00:42<00:31, 1.49it/s]
Loading 0: 88%|████████▊ | 319/363 [00:42<00:22, 1.95it/s]
Loading 0: 89%|████████▊ | 322/363 [00:42<00:15, 2.61it/s]
Loading 0: 90%|█████████ | 327/363 [00:42<00:08, 4.14it/s]
Loading 0: 91%|█████████ | 330/363 [00:43<00:06, 5.13it/s]
Loading 0: 92%|█████████▏| 334/363 [00:43<00:04, 7.00it/s]
Loading 0: 93%|█████████▎| 337/363 [00:43<00:03, 8.20it/s]
Loading 0: 94%|█████████▍| 341/363 [00:43<00:02, 9.71it/s]
Loading 0: 95%|█████████▌| 345/363 [00:43<00:01, 12.81it/s]
Loading 0: 96%|█████████▌| 348/363 [00:43<00:01, 13.96it/s]
Loading 0: 97%|█████████▋| 352/363 [00:44<00:00, 16.72it/s]
Loading 0: 98%|█████████▊| 355/363 [00:44<00:00, 18.76it/s]
Loading 0: 99%|█████████▊| 358/363 [00:44<00:00, 20.52it/s]
Pipeline stage MKMLizer completed in 177.38s
Job rirv938-mistral-24b-scal-2131-v4-mkmlizer completed after 173.51s with status: succeeded
run pipeline stage %s
Stopping job with name rirv938-mistral-24b-scal-2131-v4-mkmlizer
Running pipeline stage MKMLTemplater
Pipeline stage MKMLizer completed in 175.52s
Pipeline stage MKMLTemplater completed in 0.86s
run pipeline stage %s
run pipeline stage %s
Running pipeline stage MKMLTemplater
Running pipeline stage MKMLDeployer
Pipeline stage MKMLTemplater completed in 0.66s
Creating inference service rirv938-mistral-24b-sca-46352-v1
run pipeline stage %s
Waiting for inference service rirv938-mistral-24b-sca-46352-v1 to be ready
Running pipeline stage MKMLDeployer
Creating inference service rirv938-mistral-24b-scal-2131-v4
Waiting for inference service rirv938-mistral-24b-scal-2131-v4 to be ready
Inference service rirv938-llama-8b-scaleup-200k-v1 ready after 91.1041796207428s
Pipeline stage MKMLDeployer completed in 94.13s
run pipeline stage %s
Running pipeline stage StressChecker
Inference service rirv938-llama-8b-scaleup-400k-v3 ready after 91.02529239654541s
Pipeline stage MKMLDeployer completed in 94.11s
Received healthy response to inference request in 6.875030279159546s
run pipeline stage %s
Running pipeline stage StressChecker
Received healthy response to inference request in 4.999146938323975s
Received healthy response to inference request in 5.838494062423706s
Received healthy response to inference request in 2.5466840267181396s
Received healthy response to inference request in 3.137603282928467s
Received healthy response to inference request in 4.695525646209717s
Received healthy response to inference request in 4.727989196777344s
Received healthy response to inference request in 3.140216588973999s
5 requests
0 failed requests
5th percentile: 2.6653905391693113
10th percentile: 2.7840970516204835
20th percentile: 3.0215100765228273
30th percentile: 3.4512784004211428
40th percentile: 4.073402023315429
Received healthy response to inference request in 4.965777635574341s
50th percentile: 4.695525646209717
60th percentile: 4.81697416305542
70th percentile: 4.938422679901123
80th percentile: 5.374323606491089
90th percentile: 6.124676942825317
95th percentile: 6.499853610992432
Received healthy response to inference request in 2.7075490951538086s
99th percentile: 6.799994945526123
5 requests
mean time: 4.451320695877075
0 failed requests
Pipeline stage StressChecker completed in 34.67s
5th percentile: 2.79355993270874
run pipeline stage %s
10th percentile: 2.8795707702636717
Running pipeline stage OfflineFamilyFriendlyTriggerPipeline
20th percentile: 3.0515924453735352
run_pipeline:run_in_cloud %s
30th percentile: 3.4556804656982423
starting trigger_guanaco_pipeline args=%s
40th percentile: 4.091834831237793
50th percentile: 4.727989196777344
triggered trigger_guanaco_pipeline args=%s
60th percentile: 4.823104572296143
Pipeline stage OfflineFamilyFriendlyTriggerPipeline completed in 3.37s
70th percentile: 4.918219947814942
run pipeline stage %s
80th percentile: 5.140320920944214
Running pipeline stage TriggerMKMLProfilingPipeline
90th percentile: 5.48940749168396
run_pipeline:run_in_cloud %s
95th percentile: 5.663950777053833
starting trigger_guanaco_pipeline args=%s
99th percentile: 5.8035854053497316
mean time: 4.275482654571533
triggered trigger_guanaco_pipeline args=%s
Pipeline stage StressChecker completed in 34.95s
Pipeline stage TriggerMKMLProfilingPipeline completed in 3.04s
run pipeline stage %s
Shutdown handler de-registered
Running pipeline stage OfflineFamilyFriendlyTriggerPipeline
rirv938-llama-8b-scaleup-200k_v1 status is now deployed due to DeploymentManager action
run_pipeline:run_in_cloud %s
starting trigger_guanaco_pipeline args=%s
triggered trigger_guanaco_pipeline args=%s
Pipeline stage OfflineFamilyFriendlyTriggerPipeline completed in 2.55s
run pipeline stage %s
Running pipeline stage TriggerMKMLProfilingPipeline
run_pipeline:run_in_cloud %s
starting trigger_guanaco_pipeline args=%s
triggered trigger_guanaco_pipeline args=%s
Pipeline stage TriggerMKMLProfilingPipeline completed in 2.22s
Shutdown handler de-registered
rirv938-llama-8b-scaleup-400k_v3 status is now deployed due to DeploymentManager action
Shutdown handler registered
run pipeline %s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyScorer
Evaluating %s Family Friendly Score with %s threads
%s, retrying in %s seconds...
%s, retrying in %s seconds...
%s, retrying in %s seconds...
%s, retrying in %s seconds...
%s, retrying in %s seconds...
Shutdown handler registered
run pipeline %s
run pipeline stage %s
Running pipeline stage MKMLProfilerDeleter
Skipping teardown as no inference service was successfully deployed
Pipeline stage MKMLProfilerDeleter completed in 0.23s
run pipeline stage %s
Running pipeline stage MKMLProfilerTemplater
Pipeline stage MKMLProfilerTemplater completed in 0.17s
run pipeline stage %s
Running pipeline stage MKMLProfilerDeployer
Creating inference service rirv938-llama-8b-scaleup-400k-v3-profiler
Waiting for inference service rirv938-llama-8b-scaleup-400k-v3-profiler to be ready
Inference service rirv938-llama-8b-scaleup-400k-v3-profiler ready after 90.36773729324341s
Pipeline stage MKMLProfilerDeployer completed in 90.94s
run pipeline stage %s
Running pipeline stage MKMLProfilerRunner
kubectl cp /code/guanaco/guanaco_inference_services/src/inference_scripts tenant-chaiml-guanaco/rirv938-llama-8b-sca25f30d6aa831ff31898bb040a3a5ea03-deplollfp8:/code/chaiverse_profiler_1741725877 --namespace tenant-chaiml-guanaco
kubectl exec -it rirv938-llama-8b-sca25f30d6aa831ff31898bb040a3a5ea03-deplollfp8 --namespace tenant-chaiml-guanaco -- sh -c 'cd /code/chaiverse_profiler_1741725877 && python profiles.py profile --best_of_n 1 --auto_batch 5 --batches 1,5,10,15,20,25,30,35,40,45,50,55,60,65,70,75,80,85,90,95,100,105,110,115,120,125,130,135,140,145,150,155,160,165,170,175,180,185,190,195 --samples 200 --input_tokens 256 --output_tokens 1 --summary /code/chaiverse_profiler_1741725877/summary.json'
Received signal 15, running shutdown handler
run pipeline stage %s
Running pipeline stage MKMLProfilerDeleter
Checking if service rirv938-llama-8b-scaleup-400k-v3-profiler is running
Tearing down inference service rirv938-llama-8b-scaleup-400k-v3-profiler
Service rirv938-llama-8b-scaleup-400k-v3-profiler has been torndown
Pipeline stage MKMLProfilerDeleter completed in 1.40s
Shutdown handler de-registered
Shutdown handler registered
run pipeline %s
run pipeline stage %s
Running pipeline stage MKMLProfilerDeleter
Checking if service rirv938-llama-8b-scaleup-400k-v3-profiler is running
Skipping teardown as no inference service was found
Pipeline stage MKMLProfilerDeleter completed in 1.36s
run pipeline stage %s
Running pipeline stage MKMLProfilerTemplater
Pipeline stage MKMLProfilerTemplater completed in 0.17s
run pipeline stage %s
Running pipeline stage MKMLProfilerDeployer
Creating inference service rirv938-llama-8b-scaleup-400k-v3-profiler
Waiting for inference service rirv938-llama-8b-scaleup-400k-v3-profiler to be ready
Inference service rirv938-llama-8b-scaleup-400k-v3-profiler ready after 90.37240695953369s
Pipeline stage MKMLProfilerDeployer completed in 90.85s
run pipeline stage %s
Running pipeline stage MKMLProfilerRunner
kubectl cp /code/guanaco/guanaco_inference_services/src/inference_scripts tenant-chaiml-guanaco/rirv938-llama-8b-sca25f30d6aa831ff31898bb040a3a5ea03-deplo8txwq:/code/chaiverse_profiler_1741729509 --namespace tenant-chaiml-guanaco
kubectl exec -it rirv938-llama-8b-sca25f30d6aa831ff31898bb040a3a5ea03-deplo8txwq --namespace tenant-chaiml-guanaco -- sh -c 'cd /code/chaiverse_profiler_1741729509 && python profiles.py profile --best_of_n 1 --auto_batch 5 --batches 1,5,10,15,20,25,30,35,40,45,50,55,60,65,70,75,80,85,90,95,100,105,110,115,120,125,130,135,140,145,150,155,160,165,170,175,180,185,190,195 --samples 200 --input_tokens 256 --output_tokens 1 --summary /code/chaiverse_profiler_1741729509/summary.json'
Received signal 15, running shutdown handler
run pipeline stage %s
Running pipeline stage MKMLProfilerDeleter
Checking if service rirv938-llama-8b-scaleup-400k-v3-profiler is running
Tearing down inference service rirv938-llama-8b-scaleup-400k-v3-profiler
Service rirv938-llama-8b-scaleup-400k-v3-profiler has been torndown
Pipeline stage MKMLProfilerDeleter completed in 1.52s
Shutdown handler de-registered
rirv938-llama-8b-scaleup-400k_v3 status is now inactive due to auto deactivation removed underperforming models
Shutdown handler registered
run pipeline %s
run pipeline stage %s
Running pipeline stage MKMLProfilerDeleter
Checking if service rirv938-llama-8b-scaleup-400k-v3-profiler is running
Skipping teardown as no inference service was found
Pipeline stage MKMLProfilerDeleter completed in 1.23s
run pipeline stage %s
Running pipeline stage MKMLProfilerTemplater
Pipeline stage MKMLProfilerTemplater completed in 0.16s
run pipeline stage %s
Running pipeline stage MKMLProfilerDeployer
Creating inference service rirv938-llama-8b-scaleup-400k-v3-profiler
Waiting for inference service rirv938-llama-8b-scaleup-400k-v3-profiler to be ready
Inference service rirv938-llama-8b-scaleup-400k-v3-profiler ready after 30.15902805328369s
Pipeline stage MKMLProfilerDeployer completed in 30.61s
run pipeline stage %s
Running pipeline stage MKMLProfilerRunner
kubectl cp /code/guanaco/guanaco_inference_services/src/inference_scripts tenant-chaiml-guanaco/rirv938-llama-8b-sca25f30d6aa831ff31898bb040a3a5ea03-deplo5mh2w:/code/chaiverse_profiler_1741731399 --namespace tenant-chaiml-guanaco
kubectl exec -it rirv938-llama-8b-sca25f30d6aa831ff31898bb040a3a5ea03-deplo5mh2w --namespace tenant-chaiml-guanaco -- sh -c 'cd /code/chaiverse_profiler_1741731399 && python profiles.py profile --best_of_n 1 --auto_batch 5 --batches 1,5,10,15,20,25,30,35,40,45,50,55,60,65,70,75,80,85,90,95,100,105,110,115,120,125,130,135,140,145,150,155,160,165,170,175,180,185,190,195 --samples 200 --input_tokens 256 --output_tokens 1 --summary /code/chaiverse_profiler_1741731399/summary.json'
Received signal 15, running shutdown handler
run pipeline stage %s
Running pipeline stage MKMLProfilerDeleter
Checking if service rirv938-llama-8b-scaleup-400k-v3-profiler is running
Tearing down inference service rirv938-llama-8b-scaleup-400k-v3-profiler
Service rirv938-llama-8b-scaleup-400k-v3-profiler has been torndown
Pipeline stage MKMLProfilerDeleter completed in 1.45s
Shutdown handler de-registered
rirv938-llama-8b-scaleup-400k_v3 status is now torndown due to DeploymentManager action