A few weeks ago, I watched a video of a developer generating a fully working API integration in under a minute using an AI coding assistant. It was one of those moments where you stop and think: this is really amazing.
After further reflection, though, I started to wonder about basic security questions regarding this AI-generated code. Was there input validation? Were secrets hardcoded? Was the authentication logic strong enough? Yes, the code functioned, but was it secure enough?
That moment stuck with me because it captures the reality we’re now living in: AI not only changes how we write code, but it also changes how we introduce risk. If we’re not careful, we’re not only accelerating development; we’re also accelerating vulnerabilities and business risk.
What Has Changed: The Seismic Shift That Few Talk About
Not very long ago, developers wrote every line of code themselves. They understood it, owned it, and (ideally) secured it. Today, with AI coding assistants, that’s no longer the case.
This shift isn’t theoretical. As outlined in the “Governing Human‑AI Software Creation” white paper, AI-assisted development is fundamentally breaking long-held assumptions about code ownership, authorship, and accountability, forcing security teams to validate what is produced, not just what is written.
With tools like Claude, Gemini, Cursor, and Copilot, developers are increasingly delegating code creation to AI. AI writes, suggests, refactors, and even fixes code in real time. The productivity gains are undeniable: faster development, fewer repetitive tasks, and less cognitive load.
But there’s a tradeoff that’s easy to miss.
AI doesn’t understand your organization’s security policies. It doesn’t inherently follow your approved libraries, your compliance requirements, or your architectural standards. It generates code based on patterns that may include good, outdated, or insecure code.
Recent research shows that almost half of AI-generated code contains security vulnerabilities, underscoring that speed doesn’t guarantee safety. At the same time, while AI adoption has reached over 80% of developers, trust in its accuracy remains low.
Given these dynamics, the developer role is quietly shifting. Developers are no longer just writing code. They are reviewing it, questioning it, and ultimately taking responsibility for something they didn’t fully write. And that changes everything.
What Hasn’t Changed: Secure Coding Fundamentals Still Apply
It’s tempting to think AI has rewritten the rules of secure coding. The fundamentals haven’t changed at all. Input still needs to be validated. Authentication still needs to be enforced. Sensitive data still needs to be protected. Dependencies still need to be trusted. Access still needs to follow least privilege.
In fact, emerging research reinforces this: even as development accelerates, security must evolve from periodic checks to continuous validation embedded directly into developer workflows.
Because when AI is generating code at scale, any weakness in those fundamentals gets amplified quickly.
I’ve seen teams assume that because AI is “smart,” it will naturally produce secure code. But that assumption doesn’t hold up. AI can assist with secure coding, even accelerate it, but it cannot replace the discipline behind it.
Some AI coding tools are touting their ability to scan code for vulnerabilities. Is Anthropic’s Claude Code Security, as an example, a replacement for application security testing (AST)? In short–No! Claude Code Security is not a replacement. It’s an augmentation layer, not a substitute. In fact, both Anthropic and industry experts explicitly position it that way.
The reality is simple: application security testing has broad testing capabilities and governance. Claude operates at the code layer, while AST covers the entire application lifecycle. IDE-based AI helps a developer fix the issue they’re working on, while AST helps the organization address issues it’s not actively reviewing.
Why Is AI Tool Sprawl a Security Risk?
AI tool sprawl is another aspect of AI adoption. When teams adopt multiple disconnected AI tools (Copilot, Claude, Gemini, Cursor, niche scanners, etc.) without coordination or controls, it exacerbates operational, security, and governance risks.
Managing multiple tools, along with the complexity of operational control, can create security gaps. Each tool generates code differently and interprets “secure coding” differently, resulting in inconsistent coding practices. As an example, one tool might use approved encryption libraries, while another operates with outdated or insecure ones.
AI should never be trusted to police itself. There should be strict organizational policies on AI tool adoption, with a clear validation process to test and remediate AI output.
What Security Risks Does AI-Generated Code Actually Introduce?
What we’re seeing now isn’t just the same security challenges at a faster pace. It’s a new category of risk altogether.
AI introduces vulnerabilities in ways that traditional development never did. It can unintentionally reproduce insecure patterns and propagate them with speed. It can be influenced by poorly designed prompts. It can generate logic that looks correct but fails under real-world conditions. And perhaps most concerning, it can give developers a false sense of confidence. Up to 62% of AI-generated code contains design or security flaws, reinforcing that AI output cannot be assumed secure by default.
When something is generated instantly and appears polished, it’s easy to trust it. That’s where risk creeps in.
Because AI doesn’t validate its own output. It doesn’t test its assumptions or verify whether what it produced meets your security requirements. It just generates code.
Gartner® recently reinforced what many of us are seeing firsthand: AI-enabled applications can introduce entirely new classes of vulnerabilities, from prompt injection to access control abuse. These systems still need to be tested like any other software. AI can absolutely strengthen security, but only if we take responsibility for securing AI itself.
Application security testing is not replaced by AI; it becomes more critical because of it.
How Should Developers Write Secure Prompts for AI Coding Tools?
Here’s what I’ve observed across teams adopting AI coding tools. Development teams often focus heavily on enablement and very little on guardrails. Developers are given access to powerful tools, but without clear guidance on how to use them securely. Prompts are ad hoc, policies are loosely enforced, and governance is treated as an afterthought.
And that’s where things start to break down. Prompts are not just instructions. They are security controls at creation time.
The organizations that are succeeding with AI are doing something different. They’re treating prompting itself as a security control. Instead of asking AI to simply “write a login function,” they define expectations up front. They instruct the AI to follow secure coding standards, to use approved frameworks, and to include validation and error handling.
Those prompts are more than instructions; they become part of the guardrails necessary for coding securely.
From there, organizations can take it a step further by controlling which AI tools are approved, ensuring outputs are auditable, and embedding security expectations directly into the development workflow.
But even with all of that in place, one thing remains constant: developers are still accountable. AI can suggest, generate, and even accelerate, but it cannot own the outcome. That responsibility doesn’t shift.
The Insight Most Teams Miss
There’s one realization that tends to change how people think about AI and security: AI is probabilistic while security must be deterministic. AI generates responses based on likelihood and patterns. That means the same prompt can produce different results at different times; correctness isn’t guaranteed, and security cannot be assumed.
And that’s the core gap. You cannot rely on a probabilistic system to enforce something as critical as security.
Why Validation Is the Most Critical Step in AI-Driven Development
Once you accept that reality, the next step becomes obvious. You need a layer of validation that operates independently of AI. A system that doesn’t guess, doesn’t approximate, but proves that code is secure. This is where deterministic validation comes in.
Instead of trusting AI suggestions, organizations are introducing systems that test and verify them. Code is scanned. Policies are enforced. Fixes are validated to ensure they actually resolve vulnerabilities without introducing new ones. The difference is subtle, but powerful.
It’s the shift from: “This looks like a good fix” to “This is a proven, secure fix.” That distinction is what separates experimentation from maturity in AI-driven development.
From Code Generation to Code Validation: The New Standard
If you step back, what’s emerging is a new development model altogether. AI-powered application security becomes less about enforcing checkpoints and more about providing continuous, contextual validation that keeps pace with AI-generated code.
The development and validation process is straightforward. AI generates or modifies code. SAST/SCA tools scan and analyze it. AI-assisted triage and remediation help accelerate those processes. But before anything is trusted or merged, the code must pass through a validation layer that ensures it meets an organization’s security standards and policies. Then and only then should the code be trusted and merged.
It’s no longer just about writing code faster. It’s about ensuring that speed doesn’t come at the expense of safety. And perhaps the most important rule in this new model is simple: AI-generated code should never be trusted without validation.
The Bigger Shift: Secure Coding as a System
What this comes down to isn’t just tools or processes. It’s really adopting a new mindset. Secure coding used to be viewed as a developer skill. Something individuals were responsible for learning and applying. Now, it’s a system—a combination of:
- AI tools,
- Guardrails and policies,
- Automated testing and validation,
- Continuous learning and improvement,
- Developer in the loop.
The organizations that embrace this shift aren’t trying to replace developers with AI. They’re using AI to amplify what developers can do, while putting the right controls in place to keep everything secure.
What Does a Secure AI-Coding Workflow Look Like?
If you’re already using AI coding tools or planning to, you don’t need to overhaul everything at once. Start small and keep it simple.
Take a prompt your development team uses today and rewrite it with security in mind. Add requirements around validation, frameworks, and compliance. See how the output changes. Then ask yourself:
- Where are we relying on AI without validation?
- Where are our policies unclear or unenforced?
- Where are developers over-trusting AI-generated code?
These are small questions, but they can be the first steps to meaningful change.
10-Point Checklist: Reviewing AI-Generated Code Before Merging
As we navigate how teams are adopting AI coding tools, one thing has become very clear to me: what code we merge matters more than what code we generate. AI can produce functional code in seconds, but it doesn’t change basic accountability. That responsibility still sits with developers. And if we’re not deliberate about how we review AI-generated code, we risk introducing vulnerabilities just as quickly as we write features. This checklist reflects the practical steps I believe every team should take to validate AI-generated code before trusting it:
- Validate Input Handling: confirm all inputs are properly validated, sanitized, and protected against injection attacks.
- Verify Authentication and Authorization: ensure access controls are correctly implemented and enforce least privilege.
- Check for Hardcoded Secrets: scan for embedded credentials, API keys, or sensitive data that should be externally managed.
- Review Dependency Usage: confirm that only approved and up-to-date libraries are used—no insecure or unvetted components.
- Assess Business Logic Accuracy: validate that the logic behaves correctly under real-world conditions, not just that it compiles or passes basic tests.
- Evaluate Error Handling and Logging: ensure errors are handled securely without exposing sensitive system details.
- Scan for Known Vulnerabilities: run SAST, SCA, and other security tests to detect issues AI may have introduced or overlooked.
- Confirm Compliance with Internal Policies: verify alignment with organizational coding standards, frameworks, and regulatory requirements.
- Require Deterministic Validation: ensure fixes are tested and proven to resolve vulnerabilities, not just suggested by AI.
- Maintain Developer Accountability: require human review and sign-off so developers understand and take responsibility for the code before merging.
The Bottom Line
AI is quickly becoming the default way software gets built, and that shift is irreversible. What remains in our control is how we manage and contain the risk it introduces. AI doesn’t define your outcomes—your systems, controls, and decisions do. AI is simply a tool within that system, and without structure around it, even the most advanced models generate more noise than value.
Security needs to evolve, from something that happens after the fact, to something that is built into every step of the development process.
Because the future isn’t just about AI-assisted development, it’s about ensuring that AI is controlled, validated, and aligned with the standards you trust.
For additional insight, read our white paper: “Governing Human‑AI Software Creation.”
Ready to take control of AI-driven development risk? Talk to our experts.
Start a Conversation with Us
We’re here to help you find the right solutions and support you in achieving your business goals.


