AI & Pen Testing

As I continue to build Nido, one question I keep getting from clients, friends, and even my own mother is some variation of what are your thoughts on AI and pen testing? Well, since I have "Founder" and "Chief" in my job titles, I suppose I am contractually obligated to have a public take.

Background on AI

Before I dive into that though, quick context (see what I did there?) on my overall stance around AI: it's a tool. It lowers the barrier to entry to fields of knowledge someone may have little-to-no experience in. My washer and dishwasher both broke in the same week (yay homeownership), and I spent probably 2 to 3 hours watching random YouTube videos trying to fix them. Claude helped me figure it out in maybe 30 minutes? AI also helped me figure out the name Nido for my company. I didn't explicitly ask Claude "what should I name my company?", but it helped by providing tips and exercises for workshopping possible names.

AI is also a tool that can extend the capabilities of already really smart and talented people in their areas of expertise. When I'm testing a web application, if I find an exposed JavaScript source map, I use sourcemapper to reconstruct it and have Claude help with parsing. More times than I ever would've imagined, this process has helped me uncover things like cloud access keys, administrative credentials, and deprecated-but-not-actually endpoints my usual tooling would miss. Sagi Tzadik, a security researcher at Wiz, was able to uncover a remote code execution vulnerability in GitHub Enterprise leading to CVE-2026-3854. This was thanks in part to IDA MCP making it easier to automate reverse engineering GitHub Enterprise. James Kettle, the godfather of web application security research, gave a talk at DEFCON 34 on using AI to do novel security research. Please go read his research, it's incredibly fascinating. The main thing I want to point out though is that he was able to use AI to come up with more research hypotheses and experiments than he would've been able to without it. There's no denying that using AI in pen testing and security can be extremely effective.

Major emphasis on can though. Just like the rusty hammer sitting in my garage, it can be a tool for the job, but that doesn't necessarily mean it's the right tool. Spend just a few minutes on social media and you're bound to see someone advertising an AI agent framework or skill for something that already existed pre-AI. An agent that can autonomously test for and exploit SQL injection with just a single prompt? No thanks, I'd rather just use sqlmap. A skill for identifying Cross-Site Scripting in a web application? The PortSwigger XSS cheat sheet already has context appropriate payloads for a variety of tags and events.

AI & Pen Testing

When it comes to AI and pen testing, my overall stance still applies, AI is a tool. I do believe it is and will be another tool in the arsenal of the best pen testers. Companies like Nido will find creative ways to leverage AI's strengths to extend our testing capabilities. But I'm skeptical about AI being this pen test be-all-end-all utopia that LinkedIn keyboard warriors thought leaders make it out to be. It's a bold stance to take given all the noise around AI, but there are a handful of signals that cut through that noise which I think are meaningful.

Open Source

First is open source AI-powered pen testing tools. Guardian has almost 2,000 stars on GitHub at the time of writing. It touts itself as an AI-powered penetration testing automation framework. But when you look at its own README, it mainly leverages all the tried-and-true CLI tools.

It's own "Quick Web Application Pen Test" section just runs nuclei and httpx.

Don't get me wrong, having a single interface for all of your tools is really nice to have, that's what makes frameworks like Metasploit extremely popular. For anyone who is first learning about hacking or hasn't experimented with many of those CLI tools, this framework does lower the barrier to entry for them. To a seasoned penetration tester though, a tool like this doesn't really move the needle that much. And I can't help but wonder how many companies that are championing AI-powered penetration testing are just using tools like Guardian. I'm not saying it's a bad tool to use, but a single tool isn't going to drive me to change Nido's domain to nido.burp, nido.nmap, or nido.ai.

Cost

Another signal is the elephant in the room every AI-enabled company is grappling with: cost (see, Uber). Tools like Guardian, and likely internally developed tools, require you to either A: bring your AI provider key or B: have your own locally hosted AI model. If you go with option A, you're forced to deal with every new frontier model release being accompanied by a cost jump:

And I suspect that trend won't change anytime soon as AI companies are going to have to eventually start trying to climb out of the red. When AI companies increase their rates or reduce usage limits, that puts AI-powered pen test companies stuck in between a rock and hard place. Do they increase the price of their services? Or cut back on the AI-powered tools they invested so heavily into? As for option B, that's likely the endgame most companies are striving for, including Nido. The cost to purchase AI-capable workstations and servers needed to power AI tools aren't exactly cheap though, and would require some serious upfront purchasing power that'll make most finance departments sweat.

Humans

The third signal that stands out is... well, humans. Early in the AI era, I saw a post online that suggested if you want to know what jobs are safe from AI, look to see what roles OpenAI and Anthropic are hiring for. When you apply similar logic to other companies in the penetration testing space, there are still people powering the pen test.

One recently funded AI-powered penetration testing startup, ~22 people with pen testing backgrounds on staff.

A mid-size penetration testing company with AI proudly displayed on their website, advertising hiring for penetration testers.

Another mid-size penetration testing company, boldly adding humans to their autonomous testing.

I'm by no means suggesting these companies should layoff their pen test teams. Human expertise, creativity, and problem solving skills is still the #1 asset powering a great pen test. But there is a misalignment between the public messaging and the actions of these companies that does make me wonder about the viability of AI-powered testing.

Risk

Quick sidebar, I actually almost had this blog written and published, then OpenAI went and hacked Hugging Face. Anthropic admitted they did too. And so did Meta. I could probably add another 1,000 words to this blog discussing these incidents, but I'll save that for another time. These incidents are a signal that can't be ignored though. These are AI providers, with a seemingly infinite budget of cash, and an army of incredibly smart employees on staff. And yet they failed to control their own tools. I am curious as to what AI-powered pen testing companies are going to do to make sure their agents don't do something similar.

Nido & AI

Again, I'm not anti-AI. There are useful cases for it, and here at Nido, we will use it in places where it's the right tool for the job. Just a few low-hanging fruit examples that come to mind:

• Parsing large Nmap files to come up with testing plans
• Researching information about unfamiliar network services
• Problem solving particularly tricky WAF filters
• Building a Burp Suite extension or hacking together Python scripts
• Reverse engineering published CVEs to create a PoC

However, we won't use AI just for the sake of AI. A basic Python script can recreate the same outputs as Guardian at the cost of 0 tokens.

Ultimately though, at the end of the day, whether or not a company uses AI for pen testing isn't what matters the most. What matters is whether or not a company can delivery a high quality, thorough test, that provides actual meaning and value to their client. The tool choice can help with that, but it pales in comparison to having a comprehensive testing process paired with deep fundamental knowledge about security. And its those two pillars that ultimately make Nido a great pen testing partner.

Thank You For Coming To My Ted Talk

Anyways, I think that's enough rambling about AI for now, back to hacking. If you're someone whose organization is on the fence about AI-powered pen testing, I'd be more than happy to chat about it. My inbox is open at [email protected].

Until then, remember to drink water.

- Joe, Founder & Chief Hacking Officer