Research & Articles

Sharing what the data shows us.

Jay Jacobs Jay Jacobs

Probability and Prediction

Luckily EPSS v4 is scoring well over 250,000 vulnerabilities and each one of those is a probabilistic statement. We can measure the accuracy of the EPSS predictions by looking at all of those statements as a whole. One method of doing that is a calibration plot, which plots the prediction against reality. The horizontal on this plot bins up the predicted probabilities and the vertical looks up the proportion of each bin that were actually exploited in the wild.

Read More
Michael Roytman Michael Roytman

AI In Cybersecurity: Looking Beyond The SOC

Artificial intelligence (AI) has become more than a buzzword in cybersecurity. Every new vendor, startup, and product marketing pitch touts advanced algorithms, machine learning, and real-time threat detection. While there’s no denying that AI can streamline operations in the Security Operations Center (SOC), an overemphasis on SOC-centric use cases creates unintended blind spots. The bulk of AI-driven innovation today aims to automate incident detection, speed up alert triage, and drive machine-scale resolutions for immediate threats. Although these capabilities are critical, focusing too heavily on the detect-and-respond phase overlooks the foundational predict-and-prevent side of cybersecurity. We’ll explore why the SOC is such a magnet for AI, what we risk by confining new technologies to that arena, and how the industry can shift toward a more proactive and holistic security strategy.

Read More
Michael Roytman Michael Roytman

Cybersecurity is Ready for Local Models

To gain a real advantage in the age of generally available machine learning, defenders will have to move to the best possible models—and that means local models, using all the data at hand.

Read More
Michael Roytman Michael Roytman

The Future of AI and ML In Cybersecurity

The future of AI/ML is the ability for a new way to interact with knowledge—one that does not require the skills that are already causing a workforce shortage in security. Vendors have access to troves of security data that the average customer or enterprise using their technology does not.

Read More
Jay Jacobs Jay Jacobs

Exploit Prediction Scoring System

Despite the large investments in information security technologies and research over the past decades, the information security industry is still immature when it comes to vulnerability management. In particular, the prioritization of remediation efforts within vulnerability management programs predominantly relies on a mixture of subjective expert opinion and severity scores. Compounding the need for prioritization is the increase in the number of vulnerabilities the average enterprise has to remediate. This article describes the first open, data-driven framework for assessing vulnerability threat, that is, the probability that a vulnerability will be exploited in the wild within the first 12 months after public disclosure. This scoring system has been designed to be simple enough to be implemented by practitioners without specialized tools or software yet provides accurate estimates (ROC AUC =0.838)=0.838) of exploitation. Moreover, the implementation is flexible enough that it can be updated as more, and better, data becomes available. We call this system the Exploit Prediction Scoring System (EPSS).

Read More
Jay Jacobs Jay Jacobs

Improving vulnerability remediation through better exploit prediction

We construct a series of vulnerability remediation strategies and compare how each perform in regard to trading off coverage and efficiency. We expand and improve upon the small body of literature that uses predictions of ‘published exploits’, by instead using ‘exploits in the wild’ as our outcome variable.

Read More
Michael Roytman Michael Roytman

The Complexity of Prioritising Patching

As American journalist and essayist HL Mencken once wrote: “For every complex problem there is a solution that is concise, clear, simple, and wrong.” Anyone working in or around vulnerability remediation knows the apparently ‘simple’ task of applying a patch is anything but. The vulnerability lifecycle is filled with pitfalls and deceptively complex tasks.Anyone working in or around vulnerability remediation knows that the apparently ‘simple’ task of applying a patch is anything but. The vulnerability lifecycle is filled with pitfalls.The time and effort needed to remediate any single vulnerability across an entire enterprise are often underestimated. This creates an obvious and urgent demand for prioritisation, which requires we understand more about the world of vulnerabilities. Michael Roytman of Kenna Security and Jay Jacobs at the Cyentia Institute explore what the open vulnerability landscape looks like and investigate multiple factors contributing to the remediation efforts.

Read More
Michael Roytman Michael Roytman

For Good Measure: Remember the Recall

We exist in a dualstage testing regime. We are subject to a low prevalence (rare event) environment. To act rationally in this scenario, the first test must remove as many false negatives as it can.

Read More
Jay Jacobs Jay Jacobs

Exploring with a Purpose

We have the better, if harder, problem of the meta-analysis (“research about research”) of many observations, always remembering that the purpose of security metrics is decision support.

Read More
Michael Roytman Michael Roytman

Measuring vs. Modelling

Using CVSS to steer remediation is nuts, ineffective, deeply diseconomic, and knee jerk; given the availability of data it is also passé, which we will now demonstrate.

Read More