Securing Kubernetes Workloads
Securing Kubernetes Workloads
Thank you for being here. If you’ve found your way over by some miracle but are not yet subscribed, let me help you with that.
Kubernetes and the Supply Chain
Running your workloads on Kubernetes(k8s) is becoming a de facto standard for many companies building software. The usual steps involve building software, running tests, building a container, hopefully doing some security scanning in the process, pushing to the registry, and deploying.
In general, there are two different risks with k8s images deployed:
Vulnerabilities in Operating System packages and the software packages
Tampered or deliberately deployed images.
Many of the workloads running on your cluster didn’t pass through your security tooling in your CI/CD pipelines. Tools directly installed using helm with publicly available container images.
These usually have a high degree of vulnerabilities that are hidden away from you.
Also, software that is not deployed all the time can amass vulnerabilities over time. While code doesn’t change, people keep finding vulnerabilities in existing software.
There are four stages of kubernetes supply chain security:
CI/CD: scan your images and dependencies
Container registry scanning
Workload scanning while running and deployed in the cluster
Admission control only allows images based on an allow list and valid signatures.
CI/CD image scanning
Tools like Trivy can detect vulnerabilities in the container image. Next to the operating system issues, it can also detect problems with installed libraries of your software and secrets might have accidentally been added.
This will catch issues at the time of the build and prevent potentially shipping new vulnerabilities.
Container registry scanning
Registries like the ones offered by Google and AWS have the ability to scan existing images. This can make sense, but it usually is a dump yard that many people never look into.
If set up properly with alerting this can be useful. However, in my experience these are not on the radar of many teams.
Workload Scanning
Scanning and alerting on your currently running images helps you understand your current problems. It also catches container images that did not pass through your regular CI/CD pipelines or your registry.
Tools like Trivy operator and Clair can support this. They are regularly scanning your running containers and will emit alerts for you. This adds extra load to your system but allows for the most accurate representation of vulnerabilities in your clusters.
Admission control
By controlling what can run on your cluster, you enhance an extra level of security.
Having an allowed list of images will reduce attack vectors by quite a bit. If, on top if this, you verify the signatures of the images you will get even more peace of mind. Docker Content Trust and Cosign by sigstore are currently the best supported.
What do you think? Let me know at andy@occamslabs.com.
Links
CNAPPGoat: The Multicloud Open-Source Tool for Deploying Vulnerable-by-Design Cloud Resources
A multi-level approach to get a better understanding of cloud security issues. It is also useful for testing your security tooling to spot various misconfigurations.
kubefuzz - admission controller fuzzing
Kubefuzz is a generative and mutative fuzzer for kubernetes admission controller chains. It can be used to uncover unexpected behavior in complex admission controller setups.
6 Best Practices to Ensure Kubernetes Security Meets Compliance Regulations
Security must be precise enough to meet compliance requirements without impeding DevOps and developer productivity. A fairly high level overview of some useful practices.
Krane - Kubernetes RBAC Analysis made Easy
Krane is a simple Kubernetes RBAC static analysis tool. It identifies potential security risks in K8s RBAC design and makes suggestions on how to mitigate them. Krane dashboard presents current RBAC security posture and lets you navigate through its definition
.
Thank you
Thank you for reading along. If you have feedback or questions, message me: andy@occamslabs.com. I would appreciate it if you shared it with people that can benefit from the newsletter.