Known Exploited vs Recently Exploited (series 2 of 5)
This is the second post in a series on Re-Exploitation activity (link to first post).
The previous post in this series found that if a vulnerability has had any exploitation activity, then the probability of observing exploitation activity again in the next month is about 65% (52% over the next week and 33% tomorrow). This post extends that asking, “how does the probability change if we know exactly when the most recent exploitation activity was?”
Recent exploitation is a powerful predictor of future exploitation, but conversely not seeing recent exploitation significantly drops the probability of re-exploitation as time passes.
Re-exploitation events follow a rather predictable pattern and they are neither independent nor random (in the big picture).
Reporting something as exploited, while helpful, is only part of the story. We can improve prioritization efforts through systemic detection and reporting of exploitation activity.
Another assumption within cybersecurity is that once a vulnerability is exploited in the wild, a switch has been flipped and that particular vulnerability is exploited everywhere and consistently. This assumption means that all we need to know is that exploitation has occurred at some point. But, and I hope you say this with me: past exploitation is not a guarantee of future exploitation.
There isn’t one pattern for exploitation activity, some vulnerabilities are exploited consistently and relatively widely, but the majority of activity is bursty and sparse and sometimes activity is only observed for a brief period and then stops completely. If we have a systematic and consistent method for observing exploitation activity we can attempt to capture and explain this pattern. By tracking and factoring in recent exploitation activity, we can build a rather strong model for future exploitation activity. We can certainly do better than any approach based on just knowing “exploited at some point”.
The above chart is saying quite a lot. It’s constructed by tracking daily exploitation activity (through various sources such as IDS/IPS, malware detections and EDR data), and for each day and each vulnerability with activity at some point in the past, we observe how long since the last observed exploitation activity and if any activity was observed over the next 30 days. We ended up with tens of millions of CVE+day combinations and the proportion of observations with exploitation in the “next” 30 days are represented as the faint points behind the line. The dark green line represents a best fit logistic regression model. The dashed purple line represents the naïve prediction where we only know that exploitation has occurred, but we don’t know exactly how recently.
To evaluate this approach, we can calculate a Brier score, it’s a method to measure the accuracy of probability-based predictions. Brier scores range from 0 to 1, with 0 being absolutely perfect predictions and 1 being completely and consistently wrong predictions. As a point of reference, short-term weather forecasts (1-2 days out) typically have a Brier score in the 0.06 to 0.2 range. Forecasts of US presidential elections typically range from 0.1 to 0.2 and EPSS falls in a nice range of 0.017 to 0.02 (yay team!).
The Brier score of the naïve model (which assumes a flat 65% probability of re-exploitation in the next 30 days regardless of recent activity), is 0.387. Which is not that impressive, Clearly there is a pattern in re-exploitation activity that a simple model misses. It under-estimates recently exploited vulnerabilities and greatly over-estimates the probability of re-exploitation when exploitation activity hasn’t been observed in more than 3 weeks. When exploitation activity is tracked systematically and consistently we can factor that into our estimations. The Brier score drops significantly, down to a rather impressive 00004.
What you should take away
Again, I don’t want anyone to read this and think that they should not be prioritizing known exploited vulnerabilities. Prioritizing flaws that have had exploitation activity observed is a pretty good strategy in the grand scheme of things. But we must understand that exploitation activity is not a one-way door, and it is not binary. Vulnerability exploitation activity has a temporal quality and knowledge of that greatly improves prioritization. Lists of known exploited vulnerability lists are better than nothing, but they could be greatly improved with systemic detection capability so that recency could be factored into prioritization models.
