Fortifying AI Security in Kubernetes with Confidential Containers (CoCo)

Fortifying AI Security in Kubernetes with Confidential Containers (CoCo) - Suraj Deshmukh, Microsoft & Pradipta Banerjee, Red Hat

AI models have become valuable intellectual property that can provide organizations with a competitive edge. Users are searching for ways to secure their AI models without implicitly trusting third-party platform providers. While encryption is available to keep models secure when they’re stored & transferred, they’re still decrypted & loaded into memory during inferencing, potentially exposing them to unintentional or intentional exfiltration. This is where "confidential computing" comes in. This technology encrypts memory to protect data in use. Confidential Containers (CoCo) is a CNCF sandbox project that aims to bring confidential computing to k8s. The k8s AI/ML ecosystem is mature & offers many AI/ML training & inferencing options. The focus is on using CoCo with Kserve project to show how CoCo strengthens AI model protection. Apart from inferencing, we will explore broader application of CoCo, emphasizing its role in providing general memory protection for foundational platforms.