Shutdown handler not registered because Python interpreter is not running in the main thread
run pipeline %s
run pipeline stage %s
Running pipeline stage MKMLizer
Starting job with name rica40325-fliter65kv1-v2-mkmlizer
Waiting for job on rica40325-fliter65kv1-v2-mkmlizer to finish
Failed to get response for submission chaiml-cyndonia24b-cpos_11400_v1: ('http://chaiml-cyndonia24b-cpos-11400-v1-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', '')
rica40325-fliter65kv1-v2-mkmlizer: ╔═════════════════════════════════════════════════════════════════════╗
rica40325-fliter65kv1-v2-mkmlizer: ║ ║
rica40325-fliter65kv1-v2-mkmlizer: ║ ██████ ██████ █████ ████ ████ ║
rica40325-fliter65kv1-v2-mkmlizer: ║ ░░██████ ██████ ░░███ ███░ ░░███ ║
rica40325-fliter65kv1-v2-mkmlizer: ║ ░███░█████░███ ░███ ███ ░███ ║
rica40325-fliter65kv1-v2-mkmlizer: ║ ░███░░███ ░███ ░███████ ░███ ║
rica40325-fliter65kv1-v2-mkmlizer: ║ ░███ ░░░ ░███ ░███░░███ ░███ ║
rica40325-fliter65kv1-v2-mkmlizer: ║ ░███ ░███ ░███ ░░███ ░███ ║
rica40325-fliter65kv1-v2-mkmlizer: ║ █████ █████ █████ ░░████ █████ ║
rica40325-fliter65kv1-v2-mkmlizer: ║ ░░░░░ ░░░░░ ░░░░░ ░░░░ ░░░░░ ║
rica40325-fliter65kv1-v2-mkmlizer: ║ ║
rica40325-fliter65kv1-v2-mkmlizer: ║ Version: 0.27.1+vampire_v3 ║
rica40325-fliter65kv1-v2-mkmlizer: ║ Features: FLYWHEEL, CUDA ║
rica40325-fliter65kv1-v2-mkmlizer: ║ Copyright 2023-2025 MK ONE TECHNOLOGIES Inc. ║
rica40325-fliter65kv1-v2-mkmlizer: ║ https://mk1.ai ║
rica40325-fliter65kv1-v2-mkmlizer: ║ ║
rica40325-fliter65kv1-v2-mkmlizer: ║ The license key for the current software has been verified as ║
rica40325-fliter65kv1-v2-mkmlizer: ║ belonging to: ║
rica40325-fliter65kv1-v2-mkmlizer: ║ ║
rica40325-fliter65kv1-v2-mkmlizer: ║ Chai Research Corp. ║
rica40325-fliter65kv1-v2-mkmlizer: ║ Account ID: 7997a29f-0ceb-4cc7-9adf-840c57b4ae6f ║
rica40325-fliter65kv1-v2-mkmlizer: ║ Expiration: 2028-03-31 23:59:59 ║
rica40325-fliter65kv1-v2-mkmlizer: ║ ║
rica40325-fliter65kv1-v2-mkmlizer: ╚═════════════════════════════════════════════════════════════════════╝
Failed to get response for submission chaiml-cyndonia24b-cpos_11400_v1: ('http://chaiml-cyndonia24b-cpos-11400-v1-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', '')
rica40325-fliter65kv1-v2-mkmlizer: Downloaded to shared memory in 32.792s
rica40325-fliter65kv1-v2-mkmlizer: quantizing model to /dev/shm/model_cache, profile:s0, folder:/tmp/tmprt10z6qq, device:0
rica40325-fliter65kv1-v2-mkmlizer: Saving flywheel model at /dev/shm/model_cache
Failed to get response for submission chaiml-cyndonia24b-cpos_11400_v1: ('http://chaiml-cyndonia24b-cpos-11400-v1-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', '')
rica40325-fliter65kv1-v2-mkmlizer: quantized model in 33.145s
rica40325-fliter65kv1-v2-mkmlizer: Processed model rica40325/fliter65kv1 in 65.938s
rica40325-fliter65kv1-v2-mkmlizer: creating bucket guanaco-mkml-models
rica40325-fliter65kv1-v2-mkmlizer: Bucket 's3://guanaco-mkml-models/' created
rica40325-fliter65kv1-v2-mkmlizer: uploading /dev/shm/model_cache to s3://guanaco-mkml-models/rica40325-fliter65kv1-v2
rica40325-fliter65kv1-v2-mkmlizer: cp /dev/shm/model_cache/tokenizer.json s3://guanaco-mkml-models/rica40325-fliter65kv1-v2/tokenizer.json
rica40325-fliter65kv1-v2-mkmlizer: cp /dev/shm/model_cache/flywheel_model.0.safetensors s3://guanaco-mkml-models/rica40325-fliter65kv1-v2/flywheel_model.0.safetensors
rica40325-fliter65kv1-v2-mkmlizer:
Loading 0: 0%| | 0/363 [00:00<?, ?it/s]
Loading 0: 1%|▏ | 5/363 [00:00<00:12, 27.76it/s]
Loading 0: 3%|▎ | 12/363 [00:00<00:08, 42.54it/s]
Loading 0: 5%|▍ | 17/363 [00:00<00:08, 39.77it/s]
Loading 0: 6%|▌ | 22/363 [00:00<00:08, 38.49it/s]
Loading 0: 7%|▋ | 26/363 [00:00<00:08, 37.61it/s]
Loading 0: 8%|▊ | 30/363 [00:00<00:08, 37.45it/s]
Loading 0: 9%|▉ | 34/363 [00:00<00:09, 35.61it/s]
Loading 0: 11%|█ | 39/363 [00:01<00:08, 37.85it/s]
Loading 0: 12%|█▏ | 43/363 [00:01<00:08, 35.79it/s]
Loading 0: 13%|█▎ | 48/363 [00:01<00:08, 37.91it/s]
Loading 0: 14%|█▍ | 52/363 [00:01<00:08, 35.60it/s]
Loading 0: 15%|█▌ | 56/363 [00:01<00:08, 35.27it/s]
Loading 0: 17%|█▋ | 61/363 [00:01<00:12, 24.30it/s]
Loading 0: 18%|█▊ | 64/363 [00:02<00:12, 23.25it/s]
Loading 0: 20%|█▉ | 71/363 [00:02<00:09, 31.24it/s]
Loading 0: 21%|██ | 75/363 [00:02<00:09, 30.81it/s]
Loading 0: 22%|██▏ | 80/363 [00:02<00:08, 33.53it/s]
Loading 0: 23%|██▎ | 84/363 [00:02<00:08, 32.77it/s]
Loading 0: 24%|██▍ | 88/363 [00:02<00:08, 34.05it/s]
Loading 0: 25%|██▌ | 92/363 [00:02<00:08, 31.53it/s]
Loading 0: 27%|██▋ | 98/363 [00:02<00:07, 36.22it/s]
Loading 0: 28%|██▊ | 102/363 [00:03<00:07, 34.65it/s]
Loading 0: 29%|██▉ | 106/363 [00:03<00:07, 34.00it/s]
Loading 0: 31%|███ | 112/363 [00:03<00:06, 37.64it/s]
Loading 0: 32%|███▏ | 116/363 [00:03<00:06, 35.36it/s]
Loading 0: 33%|███▎ | 120/363 [00:03<00:07, 33.37it/s]
Loading 0: 34%|███▍ | 124/363 [00:03<00:06, 34.92it/s]
Loading 0: 35%|███▌ | 128/363 [00:03<00:07, 31.69it/s]
Loading 0: 36%|███▋ | 132/363 [00:03<00:06, 33.61it/s]
Loading 0: 37%|███▋ | 136/363 [00:04<00:07, 30.76it/s]
Loading 0: 39%|███▉ | 141/363 [00:04<00:06, 35.26it/s]
Loading 0: 40%|███▉ | 145/363 [00:04<00:10, 21.61it/s]
Loading 0: 41%|████ | 149/363 [00:04<00:09, 22.58it/s]
Loading 0: 42%|████▏ | 154/363 [00:04<00:07, 27.30it/s]
Loading 0: 44%|████▎ | 158/363 [00:04<00:07, 26.94it/s]
Loading 0: 45%|████▌ | 165/363 [00:05<00:05, 34.19it/s]
Loading 0: 47%|████▋ | 169/363 [00:05<00:05, 33.22it/s]
Loading 0: 48%|████▊ | 174/363 [00:05<00:05, 34.84it/s]
Loading 0: 49%|████▉ | 178/363 [00:05<00:05, 33.81it/s]
Loading 0: 50%|█████ | 183/363 [00:05<00:05, 35.87it/s]
Loading 0: 52%|█████▏ | 187/363 [00:05<00:05, 34.25it/s]
Loading 0: 53%|█████▎ | 192/363 [00:05<00:04, 35.92it/s]
Loading 0: 54%|█████▍ | 196/363 [00:05<00:04, 34.53it/s]
Loading 0: 55%|█████▌ | 201/363 [00:06<00:04, 36.12it/s]
Loading 0: 56%|█████▋ | 205/363 [00:06<00:04, 34.35it/s]
Loading 0: 58%|█████▊ | 210/363 [00:06<00:04, 35.76it/s]
Loading 0: 59%|█████▉ | 214/363 [00:06<00:04, 34.22it/s]
Loading 0: 60%|██████ | 218/363 [00:06<00:04, 33.13it/s]
Loading 0: 61%|██████▏ | 223/363 [00:06<00:05, 23.98it/s]
Loading 0: 62%|██████▏ | 226/363 [00:07<00:05, 23.32it/s]
Loading 0: 63%|██████▎ | 230/363 [00:07<00:05, 23.66it/s]
Loading 0: 65%|██████▍ | 235/363 [00:07<00:04, 28.37it/s]
Loading 0: 66%|██████▌ | 239/363 [00:07<00:04, 27.78it/s]
Loading 0: 67%|██████▋ | 244/363 [00:07<00:03, 32.42it/s]
Loading 0: 68%|██████▊ | 248/363 [00:07<00:03, 30.57it/s]
Loading 0: 70%|██████▉ | 253/363 [00:07<00:03, 35.06it/s]
Loading 0: 71%|███████ | 257/363 [00:08<00:03, 31.51it/s]
Loading 0: 72%|███████▏ | 262/363 [00:08<00:02, 35.55it/s]
Loading 0: 73%|███████▎ | 266/363 [00:08<00:03, 32.17it/s]
Loading 0: 75%|███████▍ | 271/363 [00:08<00:02, 36.38it/s]
Loading 0: 76%|███████▌ | 275/363 [00:08<00:02, 32.83it/s]
Loading 0: 77%|███████▋ | 280/363 [00:08<00:02, 36.67it/s]
Loading 0: 78%|███████▊ | 284/363 [00:08<00:02, 33.28it/s]
Loading 0: 80%|███████▉ | 289/363 [00:08<00:01, 37.27it/s]
Loading 0: 81%|████████ | 293/363 [00:09<00:02, 33.43it/s]
Loading 0: 82%|████████▏ | 298/363 [00:09<00:01, 37.22it/s]
Loading 0: 83%|████████▎ | 303/363 [00:09<00:01, 38.16it/s]
Loading 0: 85%|████████▍ | 307/363 [00:09<00:03, 18.07it/s]
Loading 0: 86%|████████▌ | 312/363 [00:09<00:02, 20.59it/s]
Loading 0: 88%|████████▊ | 319/363 [00:10<00:01, 27.70it/s]
Loading 0: 89%|████████▉ | 323/363 [00:10<00:01, 28.03it/s]
Loading 0: 90%|█████████ | 328/363 [00:10<00:01, 31.03it/s]
Loading 0: 91%|█████████▏| 332/363 [00:10<00:01, 30.86it/s]
Loading 0: 93%|█████████▎| 337/363 [00:10<00:00, 33.74it/s]
Loading 0: 94%|█████████▍| 341/363 [00:10<00:00, 32.52it/s]
Loading 0: 95%|█████████▌| 346/363 [00:10<00:00, 35.05it/s]
Loading 0: 96%|█████████▋| 350/363 [00:10<00:00, 33.80it/s]
Loading 0: 98%|█████████▊| 355/363 [00:11<00:00, 36.18it/s]
Loading 0: 99%|█████████▉| 359/363 [00:11<00:00, 34.32it/s]
Failed to get response for submission chaiml-cyndonia24b-cpos_11400_v1: ('http://chaiml-cyndonia24b-cpos-11400-v1-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', '')
Job rica40325-fliter65kv1-v2-mkmlizer completed after 105.94s with status: succeeded
Stopping job with name rica40325-fliter65kv1-v2-mkmlizer
Pipeline stage MKMLizer completed in 106.48s
run pipeline stage %s
Running pipeline stage MKMLTemplater
Pipeline stage MKMLTemplater completed in 0.17s
run pipeline stage %s
Running pipeline stage MKMLDeployer
Creating inference service rica40325-fliter65kv1-v2
Waiting for inference service rica40325-fliter65kv1-v2 to be ready
Failed to get response for submission chaiml-cyndonia24b-cpos_11400_v1: ('http://chaiml-cyndonia24b-cpos-11400-v1-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', '')
Failed to get response for submission chaiml-cyndonia24b-cpos_11400_v1: ('http://chaiml-cyndonia24b-cpos-11400-v1-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', '')
Failed to get response for submission chaiml-cyndonia24b-cpos_11400_v1: ('http://chaiml-cyndonia24b-cpos-11400-v1-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', '')
Failed to get response for submission chaiml-cyndonia24b-cpos_11400_v1: ('http://chaiml-cyndonia24b-cpos-11400-v1-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', '')
Failed to get response for submission chaiml-cyndonia24b-cpos_11400_v1: ('http://chaiml-cyndonia24b-cpos-11400-v1-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', '')
Failed to get response for submission chaiml-cyndonia24b-cpos_11400_v1: ('http://chaiml-cyndonia24b-cpos-11400-v1-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', '')
Tearing down inference service rica40325-fliter65kv1-v2
%s, retrying in %s seconds...
Creating inference service rica40325-fliter65kv1-v2
Waiting for inference service rica40325-fliter65kv1-v2 to be ready
Failed to get response for submission chaiml-cyndonia24b-cpos_11400_v1: ('http://chaiml-cyndonia24b-cpos-11400-v1-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', '')
Failed to get response for submission chaiml-cyndonia24b-cpos_11400_v1: ('http://chaiml-cyndonia24b-cpos-11400-v1-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', '')
Failed to get response for submission chaiml-cyndonia24b-cpos_11400_v1: ('http://chaiml-cyndonia24b-cpos-11400-v1-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', '')
Tearing down inference service rica40325-fliter65kv1-v2
%s, retrying in %s seconds...
Creating inference service rica40325-fliter65kv1-v2
Waiting for inference service rica40325-fliter65kv1-v2 to be ready
Failed to get response for submission chaiml-cyndonia24b-cpos_11400_v1: ('http://chaiml-cyndonia24b-cpos-11400-v1-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', '')
Failed to get response for submission chaiml-cyndonia24b-cpos_11400_v1: ('http://chaiml-cyndonia24b-cpos-11400-v1-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', '')
Failed to get response for submission chaiml-cyndonia24b-cpos_11400_v1: ('http://chaiml-cyndonia24b-cpos-11400-v1-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', '')
Failed to get response for submission chaiml-cyndonia24b-cpos_11400_v1: ('http://chaiml-cyndonia24b-cpos-11400-v1-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', '')
Failed to get response for submission chaiml-cyndonia24b-cpos_11400_v1: ('http://chaiml-cyndonia24b-cpos-11400-v1-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', '')
Failed to get response for submission chaiml-cyndonia24b-cpos_11400_v1: ('http://chaiml-cyndonia24b-cpos-11400-v1-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', '')
Failed to get response for submission chaiml-cyndonia24b-cpos_11400_v1: ('http://chaiml-cyndonia24b-cpos-11400-v1-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', '')
Failed to get response for submission chaiml-cyndonia24b-cpos_11400_v1: ('http://chaiml-cyndonia24b-cpos-11400-v1-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', '')
Failed to get response for submission chaiml-cyndonia24b-cpos_11400_v1: ('http://chaiml-cyndonia24b-cpos-11400-v1-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', '')
Tearing down inference service rica40325-fliter65kv1-v2
clean up pipeline due to error=DeploymentError('Timeout to start the InferenceService rica40325-fliter65kv1-v2. The InferenceService is as following: {\'apiVersion\': \'serving.kserve.io/v1beta1\', \'kind\': \'InferenceService\', \'metadata\': {\'annotations\': {\'autoscaling.knative.dev/class\': \'hpa.autoscaling.knative.dev\', \'autoscaling.knative.dev/container-concurrency-target-percentage\': \'70\', \'autoscaling.knative.dev/initial-scale\': \'1\', \'autoscaling.knative.dev/max-scale-down-rate\': \'1.1\', \'autoscaling.knative.dev/max-scale-up-rate\': \'2\', \'autoscaling.knative.dev/metric\': \'mean_pod_latency_ms_v2\', \'autoscaling.knative.dev/panic-threshold-percentage\': \'650\', \'autoscaling.knative.dev/panic-window-percentage\': \'35\', \'autoscaling.knative.dev/scale-down-delay\': \'30s\', \'autoscaling.knative.dev/scale-to-zero-grace-period\': \'10m\', \'autoscaling.knative.dev/stable-window\': \'180s\', \'autoscaling.knative.dev/target\': \'4000\', \'autoscaling.knative.dev/target-burst-capacity\': \'-1\', \'autoscaling.knative.dev/tick-interval\': \'15s\', \'features.knative.dev/http-full-duplex\': \'Enabled\', \'networking.knative.dev/ingress-class\': \'istio.ingress.networking.knative.dev\'}, \'creationTimestamp\': \'2025-06-10T11:40:58Z\', \'finalizers\': [\'inferenceservice.finalizers\'], \'generation\': 1, \'labels\': {\'knative.coreweave.cloud/ingress\': \'istio.ingress.networking.knative.dev\', \'prometheus.k.chaiverse.com\': \'true\', \'qos.coreweave.cloud/latency\': \'low\'}, \'managedFields\': [{\'apiVersion\': \'serving.kserve.io/v1beta1\', \'fieldsType\': \'FieldsV1\', \'fieldsV1\': {\'f:metadata\': {\'f:annotations\': {\'.\': {}, \'f:autoscaling.knative.dev/class\': {}, \'f:autoscaling.knative.dev/container-concurrency-target-percentage\': {}, \'f:autoscaling.knative.dev/initial-scale\': {}, \'f:autoscaling.knative.dev/max-scale-down-rate\': {}, \'f:autoscaling.knative.dev/max-scale-up-rate\': {}, \'f:autoscaling.knative.dev/metric\': {}, \'f:autoscaling.knative.dev/panic-threshold-percentage\': {}, \'f:autoscaling.knative.dev/panic-window-percentage\': {}, \'f:autoscaling.knative.dev/scale-down-delay\': {}, \'f:autoscaling.knative.dev/scale-to-zero-grace-period\': {}, \'f:autoscaling.knative.dev/stable-window\': {}, \'f:autoscaling.knative.dev/target\': {}, \'f:autoscaling.knative.dev/target-burst-capacity\': {}, \'f:autoscaling.knative.dev/tick-interval\': {}, \'f:features.knative.dev/http-full-duplex\': {}, \'f:networking.knative.dev/ingress-class\': {}}, \'f:labels\': {\'.\': {}, \'f:knative.coreweave.cloud/ingress\': {}, \'f:prometheus.k.chaiverse.com\': {}, \'f:qos.coreweave.cloud/latency\': {}}}, \'f:spec\': {\'.\': {}, \'f:predictor\': {\'.\': {}, \'f:affinity\': {\'.\': {}, \'f:nodeAffinity\': {\'.\': {}, \'f:tion\': {}, \'f:requiredDuringSchedulingIgnoredDuringExecution\': {}}}, \'f:containerConcurrency\': {}, \'f:containers\': {}, \'f:imagePullSecrets\': {}, \'f:maxReplicas\': {}, \'f:minReplicas\': {}, \'f:timeout\': {}, \'f:volumes\': {}}}}, \'manager\': \'OpenAPI-Generator\', \'operation\': \'Update\', \'time\': \'2025-06-10T11:40:58Z\'}, {\'apiVersion\': \'serving.kserve.io/v1beta1\', \'fieldsType\': \'FieldsV1\', \'fieldsV1\': {\'f:metadata\': {\'f:finalizers\': {\'.\': {}, \'v:"inferenceservice.finalizers"\': {}}}}, \'manager\': \'manager\', \'operation\': \'Update\', \'time\': \'2025-06-10T11:40:58Z\'}, {\'apiVersion\': \'serving.kserve.io/v1beta1\', \'fieldsType\': \'FieldsV1\', \'fieldsV1\': {\'f:status\': {\'.\': {}, \'f:components\': {\'.\': {}, \'f:predictor\': {\'.\': {}, \'f:latestCreatedRevision\': {}}}, \'f:conditions\': {}, \'f:modelStatus\': {\'.\': {}, \'f:states\': {\'.\': {}, \'f:activeModelState\': {}, \'f:targetModelState\': {}}, \'f:transitionStatus\': {}}, \'f:observedGeneration\': {}}}, \'manager\': \'manager\', \'operation\': \'Update\', \'subresource\': \'status\', \'time\': \'2025-06-10T11:51:00Z\'}], \'name\': \'rica40325-fliter65kv1-v2\', \'namespace\': \'tenant-chaiml-guanaco\', \'resourceVersion\': \'416061891\', \'uid\': \'5d0ae3db-8610-4264-8e7a-842f16197731\'}, \'spec\': {\'predictor\': {\'affinity\': {\'nodeAffinity\': {\'tion\': [{\'preference\': {\'matchExpressions\': [{\'key\': \'gpu.nvidia.com/class\', \'operator\': \'In\', \'values\': [\'RTX_A5000\']}]}, \'weight\': 5}], \'requiredDuringSchedulingIgnoredDuringExecution\': {\'nodeSelectorTerms\': [{\'matchExpressions\': [{\'key\': \'gpu.nvidia.com/class\', \'operator\': \'In\', \'values\': [\'RTX_A5000\']}]}]}}}, \'containerConcurrency\': 0, \'containers\': [{\'env\': [{\'name\': \'MAX_TOKEN_INPUT\', \'value\': \'1024\'}, {\'name\': \'BEST_OF\', \'value\': \'8\'}, {\'name\': \'TEMPERATURE\', \'value\': \'1.0\'}, {\'name\': \'PRESENCE_PENALTY\', \'value\': \'0.0\'}, {\'name\': \'FREQUENCY_PENALTY\', \'value\': \'0.0\'}, {\'name\': \'TOP_P\', \'value\': \'1.0\'}, {\'name\': \'MIN_P\', \'value\': \'0.0\'}, {\'name\': \'TOP_K\', \'value\': \'40\'}, {\'name\': \'STOPPING_WORDS\', \'value\': \'["\\\\\\\\n"]\'}, {\'name\': \'MAX_TOKENS\', \'value\': \'64\'}, {\'name\': \'MAX_BATCH_SIZE\', \'value\': \'128\'}, {\'name\': \'MAX_CACHED_RESPONSES\', \'value\': \'-1\'}, {\'name\': \'URL_ROUTE\', \'value\': \'GPT-J-6B-lit-v2\'}, {\'name\': \'OBJ_ACCESS_KEY_ID\', \'value\': \'LETMTTRMLFFAMTBK\'}, {\'name\': \'OBJ_SECRET_ACCESS_KEY\', \'value\': \'VwwZaqefOOoaouNxUk03oUmK9pVEfruJhjBHPGdgycK\'}, {\'name\': \'OBJ_ENDPOINT\', \'value\': \'https://accel-object.ord1.coreweave.com\'}, {\'name\': \'TENSORIZER_URI\', \'value\': \'s3://guanaco-mkml-models/rica40325-fliter65kv1-v2\'}, {\'name\': \'RESERVE_MEMORY\', \'value\': \'2048\'}, {\'name\': \'DOWNLOAD_TO_LOCAL\', \'value\': \'/dev/shm/model_cache\'}, {\'name\': \'NUM_GPUS\', \'value\': \'1\'}, {\'name\': \'MK1_MKML_LICENSE_KEY\', \'valueFrom\': {\'secretKeyRef\': {\'key\': \'key\', \'name\': \'mkml-license-key\'}}}], \'image\': \'gcr.io/chai-959f8/chai-guanaco/mkml:v1.18.35\', \'imagePullPolicy\': \'IfNotPresent\', \'name\': \'kserve-container\', \'readinessProbe\': {\'exec\': {\'command\': [\'cat\', \'/tmp/ready\']}, \'failureThreshold\': 1, \'initialDelaySeconds\': 10, \'periodSeconds\': 10, \'successThreshold\': 1, \'timeoutSeconds\': 5}, \'resources\': {\'limits\': {\'cpu\': \'2\', \'memory\': \'14Gi\', \'nvidia.com/gpu\': \'1\'}, \'requests\': {\'cpu\': \'2\', \'memory\': \'14Gi\', \'nvidia.com/gpu\': \'1\'}}, \'volumeMounts\': [{\'mountPath\': \'/dev/shm\', \'name\': \'shared-memory-cache\'}]}], \'imagePullSecrets\': [{\'name\': \'docker-creds\'}], \'maxReplicas\': 500, \'minReplicas\': 0, \'timeout\': 60, \'volumes\': [{\'emptyDir\': {\'medium\': \'Memory\'}, \'name\': \'shared-memory-cache\'}]}}, \'status\': {\'components\': {\'predictor\': {\'latestCreatedRevision\': \'rica40325-fliter65kv1-v2-predictor-00001\'}}, \'conditions\': [{\'lastTransitionTime\': \'2025-06-10T11:41:00Z\', \'reason\': \'PredictorConfigurationReady not ready\', \'severity\': \'Info\', \'status\': \'False\', \'type\': \'LatestDeploymentReady\'}, {\'lastTransitionTime\': \'2025-06-10T11:51:00Z\', \'message\': \'Revision "rica40325-fliter65kv1-v2-predictor-00001" failed with message: 0/4225 nodes are available: 1 node(s) had taint {node.coreweave.cloud/reserved: 007b3cc3da717eac69ab0559a137e7f3b606c461}, that the pod didn\\\'t tolerate, 1 node(s) had taint {node.coreweave.cloud/reserved: 0916d1e497c1c48dbbfe549139a7a5898c3cc36c}, that the pod didn\\\'t tolerate, 1 node(s) had taint {node.coreweave.cloud/reserved: 1a595a58a4eeff882120a1e1b0d1010e09698d99}, that the pod didn\\\'t tolerate, 1 node(s) had taint {node.coreweave.cloud/reserved: 1ab2bafe389d143597c5335bc6baa73e7629eb1c}, that the pod didn\\\'t tolerate, 1 node(s) had taint {node.coreweave.cloud/reserved: 326155b28370d67b46839f7a77fb42d24e633355}, that the pod didn\\\'t tolerate, 1 node(s) had taint {node.coreweave.cloud/reserved: 3a45a883af800a19f23cadb8f85db86d74b5a84f}, that the pod didn\\\'t tolerate, 1 node(s) had taint {node.coreweave.cloud/reserved: 3b3525672b1d07bb0e82ed6aa93fb7b63151b984}, that the pod didn\\\'t tolerate, 1 node(s) had taint {node.coreweave.cloud/reserved: 4ad14ec6a7b860f9abd125bc6f682c4f86c03bab}, that the pod didn\\\'t tolerate, 1 node(s) had taint {node.coreweave.cloud/reserved: 8475d6203fe60d2b8cb5af41ca53d6d5a4777694}, that the pod didn\\\'t tolerate, 1 node(s) had taint {node.coreweave.cloud/reserved: 85bb0ba151c747754c81f7c9f79197511768624a}, that the pod didn\\\'t tolerate, 1 node(s) had taint {node.coreweave.cloud/reserved: 960b232b8c784c940161614146fd165b5bd0be0a}, that the pod didn\\\'t tolerate, 1 node(s) had taint {node.coreweave.cloud/reserved: 9dedfac275b2b916c916a8f5b21f7e7efa369b15}, that the pod didn\\\'t tolerate, 1 node(s) had taint {node.coreweave.cloud/reserved: 9f636c92f7d8da46b07c40d851caf4968df86237}, that the pod didn\\\'t tolerate, 1 node(s) had taint {node.coreweave.cloud/reserved: a23e6272a875746a522968abe77c4ff953358e92}, that the pod didn\\\'t tolerate, 1 node(s) had taint {node.coreweave.cloud/reserved: a5120afadb4b45cbd040add46dc20e5015a987bf}, that the pod didn\\\'t tolerate, 1 node(s) had taint {node.coreweave.cloud/reserved: ab6668f5db9960265cd5619120217d08181b955e}, that the pod didn\\\'t tolerate, 1 node(s) had taint {node.coreweave.cloud/reserved: afdef22bd9179253164b37cabb64f3e40675acb2}, that the pod didn\\\'t tolerate, 1 node(s) had taint {node.coreweave.cloud/reserved: c155caf3468ba5b86882781ef1b4b1d508c91f3d}, that the pod didn\\\'t tolerate, 1 node(s) had taint {node.coreweave.cloud/reserved: c34219f02015e9725e0e83dae49cbb2bc89dbac3}, that the pod didn\\\'t tolerate, 1 node(s) had taint {node.coreweave.cloud/reserved: d9a4805717baccf21a30a19f47cb010767a4f67b}, that the pod didn\\\'t tolerate, 1 node(s) had taint {node.coreweave.cloud/reserved: e239a1e1f1cd3fefe8276e3d58646858101fa194}, that the pod didn\\\'t tolerate, 1 node(s) had taint {node.coreweave.cloud/reserved: e603c298c138ac32e157f8d732075ff63d7af158}, that the pod didn\\\'t tolerate, 1 node(s) had taint {node.coreweave.cloud/reserved: fb5beb3113ffb96ddf1b1bcb694dbc0afa65bfc6}, that the pod didn\\\'t tolerate, 1 node(s) had taint {node.coreweave.cloud/reserved: ffb2607e06ed03796809e5e2c78fd2c1c2d8ec65}, that the pod didn\\\'t tolerate, 1 node(s) had taint {node.kubernetes.io/unreachable: }, that the pod didn\\\'t tolerate, 1 node(s) had taint {test.foo.com/thing: }, that the pod didn\\\'t tolerate, 12 node(s) had taint {node.coreweave.cloud/reserved: 7bb0910539250d61afdb118ca00f35954c4c65a3}, that the pod didn\\\'t tolerate, 13 node(s) had taint {node.coreweave.cloud/reserved: 0e588f0c71cf3df28088e9c13ae1fd11a80165e7}, that the pod didn\\\'t tolerate, 1399 node(s) were unschedulable, 14 node(s) had taint {node.coreweave.cloud/reserved: b30d62b659c7adc22a11354c4debfa194e7fb193}, that the pod didn\\\'t tolerate, 152 node(s) had taint {node.coreweave.cloud/reserved: mimir}, that the pod didn\\\'t tolerate, 1744 node(s) didn\\\'t match Pod\\\'s node affinity, 2 node(s) had taint {node.coreweave.cloud/reservation-policy: local}, that the pod didn\\\'t tolerate, 2 node(s) had taint {node.coreweave.cloud/reserved: 047edcf4a982e8f4954be13f0346f48956e44b61}, that the pod didn\\\'t tolerate, 2 node(s) had taint {node.coreweave.cloud/reserved: 068123c732583ca97229d877d4556e1e1f4ca50d}, that the pod didn\\\'t tolerate, 2 node(s) had taint {node.coreweave.cloud/reserved: 07b2baa5117bf3dd29052fdc67f601965171b005}, that the pod didn\\\'t tolerate, 2 node(s) had taint {node.coreweave.cloud/reserved: 08c1468d7e24e3b0938e976db9cc5cd234ce0b06}, that the pod didn\\\'t tolerate, 2 node(s) had taint {node.coreweave.cloud/reserved: 2f70572f9f29c093e947a8e5963e95291e1dcb9b}, that the pod didn\\\'t tolerate, 2 node(s) had taint {node.coreweave.cloud/reserved: 6c3c6b209d62ed4178811e7d043d6bbcc1a9e43a}, that the pod didn\\\'t tolerate, 2 node(s) had taint {node.coreweave.cloud/reserved: b528168f007b9294330e209e0ff29d083c6363c1}, that the pod didn\\\'t tolerate, 2 node(s) had taint {node.coreweave.cloud/reserved: ddfd41dda0413d090ac08ed7cab356be13b4c10c}, that the pod didn\\\'t tolerate, 2 node(s) had taint {node.coreweave.cloud/reserved_for_prometheus: true}, that the pod didn\\\'t tolerate, 24 node(s) had taint {node.coreweave.cloud/reserved: 9d310b2299204b884162349bd9e1c6ba8269dbc5}, that the pod didn\\\'t tolerate, 3 node(s) had taint {node.coreweave.cloud/reserved: bb01192a8ff7186ad7285ee0b54492896962197f}, that the pod didn\\\'t tolerate, 3 node(s) had taint {node.coreweave.cloud/reserved: f7a44a72a965b42d74feced14310079a26b3230d}, that the pod didn\\\'t tolerate, 50 node(s) had taint {node.coreweave.cloud/reserved: 6c7fa72bb0e687df2f2a055b49e2e7687c0dc25e}, that the pod didn\\\'t tolerate, 655 node(s) had taint {node.coreweave.cloud/hypervisor: true}, that the pod didn\\\'t tolerate, 7 node(s) had taint {node.coreweave.cloud/reserved: 04688a0d6a3e07a42ec8266db6b2253d1faf71fc}, that the pod didn\\\'t tolerate, 9 node(s) had taint {node.coreweave.cloud/reserved: d9d52a7cb8a5be4cf618a4f0417eac9e6df4dd57}, that the pod didn\\\'t tolerate, 94 Insufficient nvidia.com/gpu..\', \'reason\': \'RevisionFailed\', \'severity\': \'Info\', \'status\': \'False\', \'type\': \'PredictorConfigurationReady\'}, {\'lastTransitionTime\': \'2025-06-10T11:41:00Z\', \'message\': \'Configuration "rica40325-fliter65kv1-v2-predictor" does not have any ready Revision.\', \'reason\': \'RevisionMissing\', \'status\': \'False\', \'type\': \'PredictorReady\'}, {\'lastTransitionTime\': \'2025-06-10T11:41:00Z\', \'message\': \'Configuration "rica40325-fliter65kv1-v2-predictor" does not have any ready Revision.\', \'reason\': \'RevisionMissing\', \'severity\': \'Info\', \'status\': \'False\', \'type\': \'PredictorRouteReady\'}, {\'lastTransitionTime\': \'2025-06-10T11:41:00Z\', \'message\': \'Configuration "rica40325-fliter65kv1-v2-predictor" does not have any ready Revision.\', \'reason\': \'RevisionMissing\', \'status\': \'False\', \'type\': \'Ready\'}, {\'lastTransitionTime\': \'2025-06-10T11:41:00Z\', \'reason\': \'PredictorRouteReady not ready\', \'severity\': \'Info\', \'status\': \'False\', \'type\': \'RoutesReady\'}], \'modelStatus\': {\'states\': {\'activeModelState\': \'\', \'targetModelState\': \'Pending\'}, \'transitionStatus\': \'InProgress\'}, \'observedGeneration\': 1}}')
Shutdown handler de-registered
rica40325-fliter65kv1_v2 status is now failed due to DeploymentManager action