Threat Modeling: A Structured Way to Ask What Could Go Wrong

Threat modeling is a fascinating concept: you review an application, look for opportunities for something to go wrong, and then document your findings. There are many frameworks, tools, and techniques for doing that. The common task is to examine what can go wrong from several perspectives.

The model I like most is STRIDE, which stands for:

• S: Spoofing
• T: Tampering
• R: Repudiation
• I: Information Disclosure
• D: Denial of Service
• E: Elevation of Privilege

I first took a few classes on Threat Modeling offered by Robert Hurlbut when I was supporting a government contract. What piqued my interest the most was the structured, orderly nature of the review process. Our internal processes were already formalized, with an extensive documentation process that took about twelve to eighteen months with a small, dedicated security compliance team to complete.

One of the most significant parts of the ATO (Authority To Operate) compliance process was the pentest.

The pentest (penetration test) brought in an outside team of specialized white hat security engineers for about two weeks of work in a separate environment. We would provide them with API keys, tokens, accounts, passwords, and documentation about how the application worked. If things went smoothly, they would come back with either a clean bill of health or a list of problems they discovered. Having gone through penetration testing training privately (e.g., the Certified Ethical Hacker course), I was confident about what they would do. (Full disclosure: I wasn’t on any of the pentesting teams that did the actual testing on these contracts.)

One thing that surprised me was that Threat Modeling wasn’t a part of that process. I can understand why a threat model wouldn’t necessarily be provided to the penetration testing team: a realistic test may intentionally limit what testers know. Still, penetration testing and threat modeling answer different questions, and that distinction made the omission interesting to me.

Going to the Doctor

The omission reminded me of going to a new doctor and asking whether or not you’re healthy without giving them a medical history. For example, it would be helpful to tell a doctor about a family history of heart disease, even if they know perfectly well how to use a stethoscope. It’s like walking into a medical office and asking the staff, “Am I healthy?” without telling them where it hurts.

A Threat Model is to a security team what a medical history is to a doctor.

A Threat Model is a map that directs the security team toward areas to inspect and vulnerabilities to research.

For example, suppose that we know a particular application reads its configuration from a particular file. That file contains a password.

If I’m brand new to the application and I don’t know anything about how it works, would I know to look for that particular file and test it? A Threat Model says, “We read from this sensitive file for an important configuration secret; please verify that we’re properly protecting the tooling around that file.”

I understand why a Threat Model can seem scary: it points out as many potential weaknesses as possible and documents how they might be exploited. If a Threat Model fell into the wrong hands, it could give a threat actor a useful map of potential weaknesses.

Honesty in Security

I have a different perspective. I believe a Threat Model is essential to this kind of security work. A Threat Model tells me where to look for potential vulnerabilities and what exploits may be used against them. I do not advocate distributing Threat Model documentation for critical systems; there is no reason to provide adversaries with unnecessary assistance. I also do not believe security teams should avoid creating them.

I’ll frame my reasoning like this:

Do you want the application to seem secure or be secure?

I do not support security theater, and I do not accept “security through obscurity” as a substitute for examining actual risk.

The Myth of the TV Hacker

If I had to guess, I would say that many people see computer security as a black box where technomancers cast a few spells (not surprisingly, all including the material component of money), wave a magic wand, and make an application secure. TV shows portray hackers in hoodies sitting around in dark rooms with monitors displaying random source code samples scrolling endlessly. They can “decrypt” passwords in a matter of minutes. Firewalls can be bypassed at will. A boss screaming, “I don’t care! Get it done!!” is all that’s needed to make things happen at speed. Also, every hack makes the screen display a laughing skull. The skull is the key piece. It has to be a laughing skull against a black background.

It’s possible that a hack in the real world could display a laughing skull against a black background. Why, though, would one want that? Weak passwords can sometimes be cracked quickly under the right conditions, which differs from the television trope of a hacker “decrypting” a password on command.

What’s Actually Required

For the most part, the real world isn’t like that. There is no magic. Hoodies are not required. Having several monitors helps, but so does having the lights turned on.

What matters is having a structured way of looking at an application and going through it piece by piece, aspect by aspect. That’s where Threat Modeling is important.

If I want my doctor to do a good job helping me maintain my health, I want them to know where it hurts, what health problems my close relatives have encountered, and where my health has been suboptimal. I want them to know that my parents both had some genetic condition that, most likely, will affect me, too.

Similarly, I want my security team to know where the problems are so that they can test them. I want my application development team to know where they are so that they can mitigate those conditions before they become vulnerabilities that are exploited.

Taking STRIDEs

This will not replace a course on using STRIDE. I can point you in the direction of some excellent teachers if you’re inclined to learn more. Here, I am using STRIDE as an example of how Threat Modeling works.

With STRIDE, the researcher takes a structured, methodical approach to reviewing the application. For example, let’s look at the “S” in STRIDE, spoofing: an attacker steals an employee’s username and password, then uses them to sign in to the company’s system. The system believes it is interacting with the employee when it is actually interacting with the attacker.

When looking at an application, someone could ask, “What would happen if someone were to spoof their identity?”

Another kind of spoofing is done over a network: an attacker tricks an application into thinking that it’s interacting with a legitimate source when, in fact, it is not.

Yet another kind of spoofing is when an attacker tricks email recipients into believing that an email came from a certain source when, in fact, it did not.

The common question behind all of these is, “How do we know someone (or something) is what they claim to be?”

If I were using STRIDE while Threat Modeling, I would go component by component and ask that same question every time some piece of code interacted with an external resource: how do we know that resource is what it claims to be?

That is the work: review each component methodically and ask how the application answers those questions.

The Threat Model examines different threats and records how they apply to the application. After Threat Modeling, the next step is to test those threats. When a component is vulnerable to a particular form of attack, put together a strategy to deal with those risks:

• Accept: acknowledge that a risk exists and decide not to do anything about it
• Mitigate: take steps to make sure the risk’s probability or impact is minimized
• Transfer: put the responsibility on a third party
• Avoid: avoid the risk by eliminating the component generating the risk

Finding frightening possibilities is the Threat Model’s job. What matters next is whether we understand those possibilities well enough to make an intentional decision about what to do next.