Secure All Software

Share this post

User's avatar
Secure All Software
Secure All Software [#14] - Signing Containers
Copy link
Facebook
Email
Notes
More
User's avatar
Discover more from Secure All Software
A Newsletter for Developers, DevOps and Security Engineers
Already have an account? Sign in

Secure All Software [#14] - Signing Containers

Container image tags are overly trusted and rarely validated. Cryptographic signing is available. Here are a few ways to do it.

Andreas Tiefenthaler's avatar
Andreas Tiefenthaler
Jun 23, 2023

Share this post

User's avatar
Secure All Software
Secure All Software [#14] - Signing Containers
Copy link
Facebook
Email
Notes
More
Share

Happy Friday, everyone 🎉. A long and eventful week is wrapping up. I dealt with some interesting issues, one related to Kubernetes and how to ensure the authenticity of the images running within.

Why Docker image tags are bad

When we build containers, we primarily control what's going on. Once pushed to the registry, it is considered stable and safe.

Docker tags are not as stable as we like to think they are.

Dependency integrity is a - more or less - solved problem for operating systems and library dependency managers. These usually use cryptographic hashes to verify what you wanted to download and that what you downloaded is the same.

In the container universe, we would rely on tags for years to decide what is installed in our clusters and systems. Ignoring that these tags can be recreated and something else could be installed without us knowing. Even when building our own images, integrity is not always guaranteed by just using a tag.

How do you improve this situation and mitigate the risk?

There are two ways to improve on this. First, we can rely on the sha256 of the image instead of the tag. The tag is for humans to read, but machines are okay with reading and comparing long hash strings. The sha256 of an image changes as soon as some of the content inside changes.

Whether for your in-house built images, or external ones, I recommend using the hash and not the tag. If you build on top of public images, rely on the SHA and not the tag to be in control. It makes updating things more inconvenient but guarantees you the authenticity of the image you build on top of.

On top of relying on the hash, signatures have been established to use cryptographic public/private-key methods to guarantee the authenticity of an image. After building an image and before sending it to the remote registry, the author would create a signature with the private key. The signature and the image are then pushed to the registry. It guarantees that what is downloaded is the same as what was uploaded.

At the moment, there are two different methods for signing images.

Out of the box, Docker has rolled out Docker Content Trust (DCT). It is s a built-in mechanism in Docker and allows for the verification of images. To use this in Kubernetes, you need to deploy a custom admission controller or use connaisseur.

The second option is cosign by sigstore. The basic concept is the same. However,, you can rely on cloud-native key management systems (KMS) like Hashicorp Vault, AWS KMS, and others.

Cosign offers a policy controller that can be deployed in Kubernetes. Connaisseur also supports cosign. Cosign can also be used to sign any other build and deployment artifacts.

Unfortunately, most publishers don't use either DCT or cosign to sign their images.

The way, for now, is to mirror the images you require, inspect them, and sign the ones you trust.

Enabling container signing for your images is easy and straightforward.

In the future, this will become more widespread and a standard.

Loading...

What do you think? Let me know at andy@occamslabs.com.

Share

Links

🔐 15 Kubectl Plugins for Security Engineers

Kubernetes has a good plugin ecosystem. Here are some useful tools day to day work with kubernetes. Especially interesting is the Cilium Plugin, which replaces the current container networking of k8s with a eBPF-based Networking layer for better observability and security.

☠️ Destroyed by Breach

Adrian Sanabria is curating a list of companies that went out of business due to “cyber security attacks”. A very interesting read. This could be eye-opening for CTOs and company owners to understand the impact of a breach.

👥 K9 Security - AWS IAM is a mess

Whenever dealing with AWS IAM, it’s a tricky walk. The system has great power and allows for super fine-grained access permissions. On the other hand, it is a complex beast to tame. My friend Stephen Kuenzli is building a tool to help to handle this complexity.

Thank you

Thank you for reading along. If you have feedback or questions, message me: andy@occamslabs.com. I would appreciate it if you shared it with people that can benefit from the newsletter.

Share Secure All Software

Share this post

User's avatar
Secure All Software
Secure All Software [#14] - Signing Containers
Copy link
Facebook
Email
Notes
More
Share

Discussion about this post

User's avatar
Burn your API keys!
Looking at API keys as a threat to your companies existence opens up the idea of using application identities and create temporary credentials based on…
Mar 29, 2024 • 
Andreas Tiefenthaler
3

Share this post

User's avatar
Secure All Software
Burn your API keys!
Copy link
Facebook
Email
Notes
More
Open Source Security
Innovation and risk lie in the vast landscape of open-source software (OSS).
Apr 5, 2024 • 
Andreas Tiefenthaler
2

Share this post

User's avatar
Secure All Software
Open Source Security
Copy link
Facebook
Email
Notes
More
You are doing it wrong! Kubernetes Image Tags
9 out of 10 deployments are doing it wrong, can you guess why?
Sep 15, 2023 • 
Andreas Tiefenthaler
1

Share this post

User's avatar
Secure All Software
You are doing it wrong! Kubernetes Image Tags
Copy link
Facebook
Email
Notes
More

Ready for more?

© 2025 Substack Inc
Privacy ∙ Terms ∙ Collection notice
Start writingGet the app
Substack is the home for great culture

Share

Copy link
Facebook
Email
Notes
More

Create your profile

User's avatar

Only paid subscribers can comment on this post

Already a paid subscriber? Sign in

Check your email

For your security, we need to re-authenticate you.

Click the link we sent to , or click here to sign in.