Skip to content

Confidence Predicts Accuracy

Confidence Predicts Accuracy and Other Lies About Cloud Security

Thinking Fast and Persuing Posture Perfect

The industry holds onto the belief that you can have a perfectly configured least-privileged account. But why do organizations persue posture perfect? Intuition is thinking that you know without knowing why you do. But confidence is no objective accuracy. This reaction is fast thinking. The false confidence can be dangerous and not obvious.

Misleading metrics

When we get a dopamine hit from closing a ticket, the bias towards this action will continue. With the numbers on the dashboard, we'll prove the path we're on. We're not incentivised to look for metrics that are not readily available.

The information that the metrics are showing is heavily dependant on the context. You could be leaving out the impact, the severity, the impact on business. When you leave the context out, you will get back to your pattern of just keep closing the Jira tickets.

Be careful when analysing threat reports. What is the sample size, what is the demographic? Is it including all the context or is it just trying to sell you stuff?

What you see is all there is

Fast thinking will not look for information it doesn't have. It takes limited data and forms it into a coherent story.

Thinking slow

Challenging an intuitive believe is not only an intellectual excercise, but also an emotional excercise. We need to deliberately engage this mode of thought.

We will try to avoid complex work. This is where checklists and frameworks come in, they force you to do step-by-step analysis. It's a structured process that directs your way of thinking and avoids you falling back to lazy mode. A pilot's preflight checklist prevents the cognitive ease of everything is probably fine, we need the checklist to go through everyhing to double check.

The NIST Cybersecurity framework provides an outside view that was collected from experiences from the past. It's technology agnostic and helps you move from intuition mode to critical thinking mode. The detect and respond functions are a direct response to the fast thinking intuitive approach.

The more things change, the more we stay the same

Why are we pursuing posture perfect? Why is protect the singular focus? Is is the easiest thing to do? This section of the posture has the metrics that are the easiest to gather.

When a business moves to the cloud, what changes? What extra skills or functions do people need?

A misconfigured S3 bucket is the new unpatched server.

The onprem infrastrcuture team needs to expand their skills to configuration status of cloud resources.

Closing thoughts

Vendors are catering to fast thinking. They sell tools that show you a list of misconfigured resources. The future cannot rest on the shoulders of the handfull of security engineers, we need to expand the mission.

So what do I do now?

  • Recognize: Recognize the deminishing returns of single-pronged strategies
  • Use: Force slow thinking. Use systems like NIST framework
  • Don't create: Make cloud security everyone's responsibility

The power of now: small tangible goals feel good. Complex problems where the gain is far in the future feel like smaller returns with less gratification.