As the world praises Mythos, and the Chinese rush to create their own variant, a report painting an entirely different picture comes from Cobalt.
The cybersecurity company just published the Cobalt State of Pentesting Report 2026, based on two comparative surveys, one in 2025 and one in 2026. Polling around 450 cybersecurity professionals, Cobalt wanted to see how confident the cybersecurity community is in automated AI testing for vulnerabilities and it turns out – not that much.
Last year, just below a third (29%) relied entirely on AI automation for testing. This year, the figure dropped to 9%. Cobalt suggests that the key reason for such a steep drop in confidence is the fact that 78% saw fully automated scanning tools missing critical vulnerabilities. Another key reason is the complexity of the AI attack surface the scanners are testing.
Context-dependent vulnerabilities
Roughly one in three findings from an AI pentest are rated “high-risk” – which is 2.7 times the average of conventional software, it was said. Also, at the time of analysis, less than two-fifths (38%) of LLM vulnerabilities were fixed, while 62% remained open. Mean time to resolve (MTTR) for AI/LLM security issues rose from 19 days to 36 days.
“LLM vulnerabilities are deeply context-dependent and invisible to tools that lack an architectural understanding of the application,” said Andrew Obadiaru, CISO of Cobalt. “To close the validation gap, automation should be deployed exactly where it excels, but elite human expertise remains foundational to uncovering and remediating the most complex business logic risks.”
It took the cybersecurity community less than a year to almost completely abandon fully automated AI testing and replace it with a hybrid model – something around 47% said they now prefer. This model has surged 22% year-over-year, while the percentage of organizations using automation for low-risk environments also increased to 47%.
“While the industry is rightfully excited about the potential of Mythos-class tools, unguided algorithms are inherently prone to returning even more false positives and costly false negatives than the automated scanners we have today,” continued Obadiaru.
Via Infosecurity Magazine
The best antivirus for all budgets
Follow TechRadar on Google News and add us as a preferred source to get our expert news, reviews, and opinion in your feeds.