DevOps: easy answers, hard questions


A few years ago I have moved from a mostly routine security role to a more creative one, where I am doing everything, from architecture definition to implementation and operation of a various security systems like internal PKI.

Our team is trying to move fast, but at the same time safely operate and maintain already deployed stack, for which we have adopted DevOps principles. We are using Terraform for provisioning, Ansible and homegrown scripts for deployment. Pipelines are running on Gitlab runners. Infrastructure and services are defined as a code, the changes are done via git flow.

After all those years I can say, that we have already seen a myriad of problems that comes from the “Build it, maintain it, operate it” territory.

Below is a list of the most obvious problems I have encountered. There is lots of talks and easy answers on how to solve them, but in the reality it is a painful process, especially if you want to solve it once and for everybody in your organization.
Not all of those questions and answers are related to DevOps, but lots of them are easy solved if you are doing things manually.

Easy answers you will find on every meetup or conference:

After some digging and trying to be production ready, problems with those answers arise. Nobody will tell you the right solution. Everybody will reinvent the wheel with their own internal one.

Hard questions:

While solving all your engineering problems, do not forget that business requirements does not change with every new technology.