Do not use CVSS alone
Keep severity as impact context, but do not treat a CVSS threshold as a complete remediation strategy.
MSc Cyber Security & Digital Forensics · 2026
Predicting real-world vulnerability exploitation.
A strict temporal evaluation of CVSS, EPSS and machine-learning models, using confirmed exploitation and the patching constraints security teams actually face.
71,224Held-out CVEs
208Confirmed exploited cases
2015–2022Training period
2023–2024Test period
Executive summary
Severity is not likelihood.
Security teams cannot patch everything at once. CVSS is widely used to order remediation work, but it measures technical severity rather than the probability that attackers will exploit a vulnerability.
I compared CVSS base scores with historically aligned EPSS scores and two models trained on structured NVD data: logistic regression and XGBoost. CISA's Known Exploited Vulnerabilities catalogue provided a high-confidence definition of confirmed malicious exploitation.
EPSS was the strongest method when remediation capacity was tight. At a 5% patch budget it captured 34.1% of confirmed exploits, compared with 4.3% for CVSS.
Research design
Test on the future, not a random slice of the past.
01Train2015–2022
NVD features and KEV-derived outcomes used to fit the internal models.
02Test2023–2024
71,224 previously unseen CVEs evaluated with no random split leakage.
03Measure30 / 365 days
Short-term and longer-term exploitation windows tested separately.
The primary metric was precision-recall AUC because only 0.29% of the test set was labelled as exploited. ROC-AUC was reported, but treated cautiously because true negatives dominate this dataset.
365-day evaluation
The main results
| Method | PR-AUC | ROC-AUC | Exploits found at 5% | Exploits found at 20% |
|---|---|---|---|---|
| CVSS base score | 0.010 | 0.762 | 4.3% | 51.0% |
| EPSS v3 | 0.136 | 0.767 | 34.1% | 62.0% |
| Logistic regression | 0.012 | 0.810 | 25.0% | 67.8% |
| XGBoost | 0.013 | 0.817 | 32.7% | 64.9% |
Held-out 2023–2024 test set. PR-AUC confidence intervals: CVSS 0.008–0.013; EPSS 0.090–0.196; logistic regression 0.010–0.017; XGBoost 0.010–0.017.
Finding 01
EPSS gave a better patching queue.
EPSS achieved a PR-AUC of 0.136, compared with 0.010 for CVSS. The non-overlapping confidence intervals support a meaningful difference on the metric best suited to the highly imbalanced data.
The operational simulation made the result clearer: at 5% capacity, EPSS found 34.1% of confirmed exploits while CVSS found 4.3%. For a team with a long vulnerability backlog, that difference directly changes what gets fixed first.
Finding 02
Complexity did not beat better data.
Logistic regression and XGBoost performed almost identically on PR-AUC: 0.012 and 0.013. Their confidence intervals overlapped and there was no statistically meaningful advantage from the more complex model.
That supports using the simpler, more interpretable model when both are trained on the same structured NVD features. The much larger advantage of EPSS suggests that richer exploitation signals mattered more than model complexity alone.
Calibration
A ranking score is not always a probability.
EPSS produced the lowest Brier score at 0.003 and a modestly positive Brier skill score of +0.058. Logistic regression and XGBoost had strongly negative skill scores because class balancing improved ranking but distorted their probability outputs.
Operationally, the internal model scores should be treated as relative rankings rather than literal probabilities of exploitation.
Recommendations for security teams
Use a layered prioritisation decision.
Use EPSS under tight capacity
When only around 10% or less can be addressed, exploitation likelihood provides the strongest return.
Add context and ownership
Combine exploitation evidence with asset exposure, business criticality and clear exception governance.
Limitations
What the study cannot prove
CISA KEV is a high-confidence but incomplete source of exploitation ground truth. A CVE missing from KEV is not proof that exploitation never occurred, and catalogue inclusion is affected by visibility and policy.
EPSS score history, NVD enrichment changes and disclosure timing also constrain the analysis. The results support a stronger prioritisation signal, not a universal threshold that every organisation should copy without testing against its own assets and capacity.