Entry through the gift shop
How to not manage access to your systems.
Thank you for being here. Please subscribe if you’ve found your way over but are not yet subscribed. This newsletter is living on your feedback. Please don’t hesitate to share your feedback with me.
Links
A collection of useful tools, articles, and other security-related topics. If you have anything that you would like to have featured, please let me know.
Allows you to extract secrets stored inside CI/CD environments by deploying malicious pipelines. It currently supports Azure DevOps, GitHub, and GitLab—a fun tool to have around. We need to find better ways to store secrets for pipelines these days.
A great resource to stay up to date on many security topics. The field of IT Security has many facets and directions, and the list has something for everyone.
Running a Web Application Firewall? This is a tool that helps you test your effectiveness.
Cloud Native Computing Foundation Announces Cilium Graduation
Cilium - an advanced Kubernetes Container Network Interface - officially graduated the CNF. The graduation highlights its evolution from a simple CNI to a complete networking, observability, and security solution that prepares platforms and organizations for the next steps on their cloud-native journey.
Over 40K admin accounts use “admin” as a password
While we work hard to use technology to protect everything, our biggest weaknesses are people. Let’s not forget about them and teach them how to do better. Then we can build more technology and to prevent the use of admin as a password again.
Basti (from Bastion Host) is a CLI tool for securely accessing your DB instances and other AWS resources in private networks at almost no cost.
Nobody needs permanent access to production systems.
Everyone knows the famous movie scenes where the main protagonist needs to get into a nightclub. The Bouncer won’t let them in through the front door. So they easily sneak in through the side entrance conveniently left open by the staff. This pretty much reflects the state-of-the-art access control for many companies.
Very commonly, you will find that too many people have permanent access to systems that they should not. Worse, they have full administrative rights. Often there are many reasons and arguments thrown around why this is that way:
It is required to keep the current system running.
There are many issues and problems, so requesting access is holding up progress.
All customer bug reports need to be debugged in production.
I can not get my work done otherwise and will be much slower to progress.
This is how we have been doing it (my personal favorite).
Often people with the most access have been the longest within the company—they were an essential part of bringing the product to the company where it is now. Access was collected over the tenure and never returned. The access rights are almost a badge of honor, carried by the single person keeping everything alive.
Unfortunately, this is very risky and can lead to massive damage for the company.
What are some of the risks associated with this?
Accidental modification of environments and the resulting downtime can be disastrous. An engineer working on some infrastructure or trying something new? Setting the access config wrongly and applying the changes to the wrong environment can lead to massive damage.
Directly accessing the database and manipulating some data because of a bug to unblock a user? There is No audit logging, validations, or safety net to keep you from causing inconsistent data or worse. Data manipulation directly on live systems is like open heart surgeries—sometimes essential but always highly risky.
Lost credentials - especially when not detected for long - is one if not the most dangerous scenario. Any attacker can access infrastructure and data. They can install code, extract data and do much more harm. This can lead to the exodus of the whole company.
Similarly, employees who are - for various reasons - threatened or let go without doing proper cleanup can lead to similar outcomes.
No matter how you look at it. The risks outweigh the “benefits” by far.
Here are some better alternatives:
There is a process and a technology-based way to handle this issue. Both follow the same approach. An engineer requests certain access rights to a system. The grantor of the access reviews this request. This is often Teamlead or the CTO, depending on the kind of access. The request can now be approved or rejected if not necessary or too wide. Once approved, the rights will be assigned to the requester. After the work has been completed, the rights will be removed again.
The process implementation is fairly lightweight and works well with small teams and infrequent access. The approver/assigner usually has only the right to give access to someone else, not for their own benefit. A simple ticketing system like Jira or even a Slack thread an be sufficient. For more complex scenarios and bigger teams automation technology is better.
It is okay to choose what works best for you in the beginning. Many companies start the process first and later add the technology. With this, the team is already in the habit, and it won’t cause too much disruption.
There are some good just-in-time access providers out there. Most are fairly expensive and clunky. Building a solution that suits your needs best and grows with you using your current cloud service provider if you have an in-house DevOps and engineering team is not too difficult.
Let’s reap the Benefits of Limiting Access
Overall, there is less risk to the system and the company. Without permanent access, you can reduce the attack surface significantly.
The limitations in access will result in better logging and monitoring. Access to these systems is a lot less risky and easier to manage. Because access is more restricted, the need for proper administrative tooling is stronger. Investments into building those will pay off as the company grows and onboarding new team members.
Overall there is a better audit trail and compliance setup. This will make customers, auditors and investors happy and in return foster trust and can positively impact sales and revenue.
You will have better emergency and disaster recovery. You will start to think about how to access systems during those and how to manage them.
Of course, there are downsides to limiting access.
When there are no proper systems in place, you will have a harder time recovering from bugs and downtimes. Getting access and following the processes will be slower and more painful. It is a lot harder, especially in the beginning. It is very important to start as early as possible. Culture change and building the necessary tools and infrastructure will take time.
Another downside is for smaller, early-stage companies with small teams and early product maturity. These companies are still trying to find their business case, product-market fit, and customers. Sometimes being able to “just help a customer” might be the difference between keeping or losing them. At this stage, it might hurt you more than it helps. When is the right time to start implementing Temporary Access Control? As early as possible, as simple as possible, and adapt as you grow.
Conclusion
Permanent access to live systems poses significant risks, including accidental modifications, unauthorized access, and potential abuse by rogue employees. Alternatives such as role-based access control, just-in-time access, and emergency privilege escalation offer better security. Limited access provides benefits such as reduced risk, improved administrative backend and tooling, better audit logs, and emergency escalation when needed. However, limited access may slow recovery in an emergency and can be challenging for early-stage companies. In conclusion, permanent access to live systems should be avoided due to the associated risks and the availability of better practices.
Working Together?
Thank you for reading along. If you have feedback or questions, message me any time andy@occamslabs.com. If you want to work together, here are a few different ways I can help you:
Security audit of your systems
Improving the security of your current systems
Designing secure systems from the ground up