A Slightly Messy KEV is still a good KEV
All clean data sets are clean in the same way, but each messy data set is messy in its own way1. Unfortunately, in the world of security data, all data sets are messy. For a data scientist this is normal. The largest part of the job is examining, cleaning, and organizing data for analysis. But there is a certain mindset in security that sees the flaws in data collection and reporting, and immediately assumes the whole of the information contained therein is useless or worse, harmful. To use a somewhat macabre idiom, they are throwing out the data baby with the bathwater.2
This is a short sighted and counter productive view. Even though all security data is messy to some degree, what matters is how it can, in aggregate, inform the risks faced by organizations. There are always going to be quirks, erroneous values, and simple mistakes, but if we can both identify and defuse them, we find that even the messiest of data sources can provide a nudge in the right direction to assess, and therefore mitigate, risk.
CISA KEV, the cleanest of messy security data
I want to look at one data source that is in many ways about as clean as it gets, the Cybersecurity Infrastructure Security Agency’s (CISA) Known Exploited Vulnerabilities (KEV) catalog. While I highly doubt anyone who finds themselves reading this blog is unfamiliar with “the KEV”, it never hurts to give a little reminder background. The KEV was established as a public resource in November of 2021 to enumerate vulnerabilities which the federal government was aware had active exploitation activity. The public list grew out of a “Binding Operational Directive”, an executive decree that gave CISA the authority to require civilian3 federal agencies to fix specific vulnerabilities within a certain timeline. CISA issued a number of decrees about these vulnerabilities before the public KEV existed, but eventually decided, correctly, that such a list would be a public good, even if they couldn’t force anyone but federal agencies to fix something.
There are three criteria for a vulnerability to be added to the KEV:
The vulnerability must have an assigned CVE.
There must be a known fix for the vulnerability that can be recommended to agencies.
There has to be evidence of exploitation attempts.
A fourth, implicit requirement is that the vulnerability must impact the federal government in some way. While the above seems straightforward, there are ambiguities (what exactly is the evidence for exploitation attempts? And what constitutes a fix?), but we aren’t going to explore them here. It is worthwhile to denote two things the KEV is explicitly not designed to be, even though some treat it like it is.
It is not a comprehensive list of exploitation and was never claimed to be. Indeed many vulnerabilities are exploited which never show up on the KEV (~18.7k by Empirical’s count).
The KEV does not indicate when a vulnerability was first exploited. Older vulns which may have long running exploitation that are finally rising to the level of impacting the federal mission may get added to the KEV. This is certainly true for older vulns.
At this point a lot of digital ink has been spilled on analysis of KEV data and its use in vulnerability management systems - largely, I suspect, because the data is so clean and accessible. Hit up the website and a clean, fresh csv or json, with all the CVEs ever added to the KEV, is ready for download. CVE-IDs give an easy way to match other sources of vulnerability information and you can start doing research or prioritizing the same vulns for remediation the federal government is telling itself to remediate.
But exactly how fresh and clean is that data? Is it so fresh and so clean, clean or might it be posted a little dirty? For example, a few weeks ago we noted that a Microsoft CVE (CVE-2026-69836), briefly appeared on the KEV (August 21, 2026 12:28PM CDT) and then disappeared (August 21, 2026 1:36PM CDT) roughly an hour later. These events caught our attention because we are in the business of producing exploitation probabilities. Our models generally indicate that addition to the KEV is likely to result in future exploitation, and thus raises the scores we report. When a CVE’s score goes up and drops down again, we tend to notice.
Now, Empirical Security is far from the first to notice these quiet changes. Others have written about these changes. For example, Kevin McCammon noted some of the thrashing back in November of 2023, and offers suggestions for changes to the schema as well as praise for the KEV as an excellent public service. While others have written about these data changes more… acerbically, we agree with Kevin that the KEV is on the whole good, it just has some kinks to work out.
BUT, this left us with the obvious question: does the KEV have other quirks like this? The answer is, of course. Like all security data, there are a small number of mistakes and erroneous additions to the KEV, and we think it’s worth investigating them, so with a little bit of care and feeding we can make the KEV even more fit for purpose. So what kind of changes4 can show up?
KEV Quirks and Features
There are three specific things we are going to be looking for, all of which from our experience tend to make data analysis and vulnerability prioritization a little more difficult.
Entire CVE entries that disappear and reappear without mention
Duplicate CVE entries
Changes in properties from one export to the next.
The first question is how often do CVEs disappear from the KEV Catalog, and in asking this we found that we also get to ask “how often CVEs disappear and come back?”. Figure 1 shows the 25 CVEs that have been added and removed at least once from the KEV Catalog.
Figure 1 — Most CVEs appear and disappear on their own, When we have a group we show them all together, the rest have their own timescale below.
There have been 8 vulnerabilities that have been added to the KEV 3 times (removed twice), and 1 that has been removed 3 times. While most (75% of) additions and removals (and readditions) are resolved in about 30 days, however, 5 lasted longer than that, with two CVEs (CVE-2022-31460 and CVE-2022-28958) taking more than 400 days to come to their final resting state (both removed).
This type of finding might be a cause for consternation, but it’s really the sign of a well-cared-for dataset that is the most accurate reflection of reality known to its stewards. If a vuln is added erroneously it should be removed! If later evidence indicates it should be re-added then by all means. Moreover, we only have evidence of 0.06% of CVEs on the KEV experiencing this type of thrashing, nearly 80x less than the quantity of rejected CVEs currently on the MITRE’s CVE list.5
Once a behavior like this has wormed its way into the consciousness of a data scientist like me, it becomes imperative to look for other issues. One that can trip up even the best of us is duplicate keys occurring in a dataset when they should be unique. This has occurred several times in the existence of the KEV, but mostly in the ancient past (2 years ago).
| CVE | Duplication Window |
|---|---|
| CVE-2016-3976 | 2021-11-03 → 2021-11-17 |
| CVE-2017-11882 | 2021-11-03 → 2021-11-17 |
| CVE-2021-30860 | 2021-11-03 → 2021-11-17 |
| CVE-2021-38647 | 2021-11-03 → 2021-11-17 |
| CVE-2014-6324 | 2022-03-31 |
| CVE-2017-17562 | 2022-03-26 → 2022-03-27 |
| CVE-2014-1776 | 2023-03-30 |
| CVE-2019-1069 | 2024-09-18 |
Beyond entries appearing and disappearing or appearing more than once, they also sometimes mutate. 43 CVE-IDs had (at least) one of their fields changed in its history on the KEV.
Figure 2 — Most of these changes are minor, or come with new information about the CVE, for example, addition of CWEs, more elaborate descriptions, corrected vulnerability names, consistent product and vendor names. Interesting changes in Due Date are almost always asking federal agencies to fix things faster.
What to do with imperfect signals
At this point you might be doubting the utility of one of the more venerated sources of public vulnerability intelligence information. I’d start by reiterating what I said at the beginning: these types of changes are the nature of the world of security data, and CISA KEV is just about the gold standard when it comes to these kinds of things. The CVE list marks a little less than 1 in 20 CVEs as REJECTED, a number that may increase in the future. Approximately 17% of CVEs that have ever had an entry in the once redoubtable NVD have had some meaningful change made to their entries.
This implies that an important part of vulnerability management is not pivoting on any single binary data point, like “is it on the KEV”, but rather to see the whole picture. As you attempt to keep a system from being compromised (or at least compromised too hard), KEV is just one of many signals that interact with the other factors in your own risk management approach, alongside the software running on your network, what CVEs attackers are targeting around the world and on your endpoints, and what controls you have in place.
There are, furthermore, two additional perspectives that can help us take the good from the KEV without being bogged down by its limitations. The first is that it is a mistake to fixate on past assertions that have been shown to be incorrect, simply because at any point in time, the KEV was always just an assertion. It was an assertion when it was incorrect in the past, and it remains an assertion today. In all likelihood, there are things on the KEV today that will certainly be amended in the future as more information comes to light. The point is that at any moment, intelligence is only ever for the present. We can look at past intelligence and scoff at its misses, but we also have to remember that at some point in the future, we or someone else will look back at the present and do the same thing.
That is not to say that we should be sanguine about procedural or structural limitations in the KEV - mistakes must be fixed, processes should be refined, and oversights must be eradicated. Duplicate entries really have no excuse. But the facts of the fixing, corrections, and eradications are not a reflection of the weakness of the source, they are one reflection of its strength over time. If intelligence only needs to function for the present, then a constantly correcting intelligence source is a good thing, not a bad one.
Footnotes
1With apologies to Tolstoy.
2Though often it seems that this is taken to such an extreme that we might modify the saying to “reject the idea of bathing babies all together”, but I digress.
3 “Require” here is a little optimistic. As far as I can tell there is no real penalty for not fixing a KEV by the denoted deadline. However, it is part of an agency’s Federal Information Security Modernization Act (FISMA) metrics report card. Specifically, an agency’s mean time to remediate KEVs (FY2026 Question number CIO-FY26-0121).
4But how do we track changes? Well this turns out to be a little bit of a challenge as the KEV has existed as a public service since November of 2021 there is no complete history of all its updates. CISA officially supported a github repository starting a little less than a year ago. Two github repos have tracked it since the beginning but at a daily scale (hrbrmstr/cisa-known-exploited-vulns, lucagrippa/cisa-kev-archive), perhaps missing intra-day changes. Three others also have daily changes, but not for the full history of the KEV (lodestonesecurity/cisa_kevc_history, myseq/cisa_kev_utils, and momokii/kev-mirror) . Empirical has kept hourly snapshots since we became a company, which is, at great tragedy to the world, not the full history of the KEV. So we cobbled together snapshots from our own data, the github repositories, and any snapshots collected from archive.org. Not a complete picture, but just about as good as we are going to get.
5As of September 10, 2026 4.7% of CVEs in the CVE list were listed as “REJECTED”.
