Naming without Knowing: An Introduction
This is the first in a series about how we describe vulnerabilities and why it matters. Naming a vulnerability is not the same as knowing it. Vulnerability records succeed at creating a shared name, but the data associated with that name are too coarse to reflect patterns of exploitation or defenders’ priorities. This series is about closing that gap.
I recently gave a presentation at VulnOpticon where I tried a little experiment. I first presented a bog-standard definition of a vulnerability:
A vulnerability is a weakness in a system that can be exploited to violate confidentiality, integrity or availability.
Then, using live polling, I gave the audience a brief and somewhat ambiguous scenario involving a code example from an application that clearly allowed SQL injection. I asked whether they would label it a vulnerability.
Then I changed the scenario. What if the code existed only in a developer’s branch and was never merged or deployed? I presented several variations and the result, which surprised me, was that the audience did not agree on a single one in spite of more detail. The weakness never changed (SQL injection is fairly straightforward) but I changed where the flaw sat in relation to a system and its environment.
Even though I had provided a very typical definition of a vulnerability, there was still wiggle room. Or what I think is more likely is that when I asked. “Is this a vulnerability?” people probably answered one or more specific questions, things like: :
Is there a weakness in the code?
Is this something the security team should be tracking?
Can someone actually reach and trigger the flaw?
Are there any set of possible circumstances that could lead to exploitation?
Could we plausibly rationalize the outcome as a security violation?
Is the issue important?
Should this receive a CVE record?
And then they would make some determination if the answers justified the label of “vulnerability.” Someone may have identified the weakness and imagined a series of (perhaps highly unlikely) events for exploitability and answered “yes”. Someone may have thought about how nobody could reach it right now and answered “no”. Maybe they work for a vendor in product security and they applied the CNA rules from the CVE program in their answer. So while their answers appeared to conflict, they probably were not disagreeing about the same thing. Which changes the question we need to ask:
What must be true or exist before we call something a vulnerability?
We should resist answering that question too quickly. It is a deceptively complex question and will be the subject of many of my future articles. Part of the ambiguity arises because we treat “vulnerability” as a descriptive label. But calling something a vulnerability initiates a chain of work and has practical consequences that vary depending on their role, what deliverables and decisions they are responsible for. Product security teams, CNAs, vulnerability-intelligence providers and scanner vendors all work on producing data for the organizations that must determine whether the systems they operate are affected, how urgently they need to respond, and whether to patch, mitigate, or accept the risk.
All of these participants appear to be discussing the same vulnerability, but they obviously have different tasks and goals. The different decisions they need to make create different demands for information. A description and CPE string may fill out a CVE record, but it’s not sufficient to create a scanner signature or help discover affected systems. CWE and CVSS may tell us which way the wind is blowing, but they do not identify appropriate security controls or detection techniques.
On the vulnerability record
Folks often treat the CVE record as the center of the vulnerability-management ecosystem. It creates a common identifier and one of the places to initiate the vulnerability management lifecycle. The ambiguity around what a vulnerability is becomes obvious going through these records. Poorly described vulnerabilities are (I’ll say it) common, and disagreements and disputes naturally follow. All of the missing information is filled by scanner vendors, intelligence providers, security teams, and individual practitioners. Think about the debt each record is carrying downstream. Every organization is either forced to pay a third party for help or they must have people reconstructing the information they need from various scattered sources.
This is where we are and it’s become normal. We accept that vulnerabilities are vague, we accept that understanding a vulnerability requires dedicated effort. We’ve normalized and accepted that all of the gaps and debt in our vulnerability records are filled by third-parties and consumers and that work is repeated over and over. This is the tension in vulnerability management: we have built a global information ecosystem around vulnerabilities without an understanding of exactly what they are.
The next article will examine the definitions the field already uses (like the standard definition above) and the ambiguity they generate in practice.
