Are you sick of filling out security questionnaires to meet some compliance or audit objectives? Most of the software engineers we talk to that have to fill these pesky forms out hate the process. Many of them say to us that they really want something that could automate responses to security questionnaires.

I had to fill out my first security questionnaire back in 2002. The questionnaire came in the form of a large Excel spreadsheet and I remember looking at it and thinking at the time, “Man, that’s a LOT of questions!!” I wondered how they would know if I was telling the truth or not? I could just put anything down and how would they know if I was being honest? Would they come onsite and audit us?

Fast forward to 2022. We still use spreadsheets for our security questionnaires. They pretty much ask the same questions they did 20 years ago, and they are still being used by the same people. It is a hallmark of #infosec that you get told to fill out the questionnaire and it’s just something we all do.

Here’s the thing: Security questionnaires aren’t that good at determining risk

Security questionnaires are lengthy and take a long time to fill out. The industry doesn’t trust them very much but continues to use them for lack of anything better. Only 14% of organizations surveyed said they are highly confident that security questionnaires represent true security profiles.
Finding the right people to talk to and getting responses from them is a real challenge. This is especially true if you are trying to ascertain compliance around software development processes and you need to talk to software engineers. Engineers are busy delivering software and often actively hostile to the audit or compliance processes. Software engineers will sometimes not respond to queries which leaves the party responsible for collecting the data in a lurch.
Questionnaires are usually not very accurate. The person filling it out usually doesn’t have the answers and often makes up answers or guesses, neither of which help. When asked, most risk professionals admit they have very little confidence in security questionnaires.
Security questionnaires are a single point-in-time snapshot. Even if somehow, the original data captured in the questionnaire was accurate, they don’t get updated which means that you can’t really trust them for very long.

Developers are usually not aligned with security or compliance objectives

As mentioned above, security questionnaires aren’t great at determining risk in normal circumstances, but it’s even harder when you are trying to audit your software development processes. Gathering data from software engineers can be really tough. They are often resistant, and sometimes downright hostile to the idea that their development processes need to meet some compliance requirements.

Software engineers’ metric of success is not usually security or compliance-related. Instead, it’s how fast they can deliver new features. So, this explains in part why developers are often so against security teams asking them questions about their security protocols.

How do you audit the CI/CD process?

Auditing the CI/CD processes to identify whether there are security or compliance gaps can be difficult. Often, even gaining the right access can be a problem. Engineering teams are often the ones with administrative access to the source code management (SCM) and CI/CD platforms, so getting them to provide access can be challenging.

Wouldn’t it be better if we could somehow automate the responses to these security questionnaires?