Who Owns Risk When AI Writes the Code?

After years in application security, I’ve consistently viewed risk as something that requires a clear, accountable owner.

If you wrote the code and understood how it worked, then you also owned the consequences when it failed.

I haven’t seen any policy that explicitly states this, nor any framework that formally enforces it. Yet every mature application security program operates as though it were the rule.

That assumption is now undeniably strained.

AI has not dramatically changed how software is delivered, but it has changed something more foundational. AI has weakened the link between writing code, understanding it and being accountable for what it does once it’s in production.

Large parts of modern applications are no longer written line by line in the traditional sense. They are suggested, generated, completed, or refactored by systems that do not understand the business context in which they are operating. Developers review the output, approve it, and move on—often at a pace that would have been difficult to imagine only a few years ago.

The software works. The pipeline passes.

Yet the question of who actually owns the risk has become far less clear.

The Assumption Driving Our Behavior

Most AppSec programs are built on long-standing, unspoken assumptions:

1. Someone authored the code.
2. Someone understood how it worked.
3. Someone can explain its behavior when things go wrong.

Security reviews, approvals, and even governance models are anchored to this idea. They assume human authorship implies human comprehension and that this comprehension implies a level of accountability.

AI-assisted development breaks that chain.

A Scenario We All Recognize

Here is a typical scenario and not an extreme edge case.

A developer is making a late-sprint update to an API. The domain is familiar, but the edge cases are tricky. Instead of writing the logic from scratch, the developer prompts an AI assistant for help.

“Add authorization checks so only managers can update team records.”

The response looks reasonable. The code is clean. The tests pass. A quick review reveals no immediate concerns.

The change is thus merged into the code stream.

Weeks later, a customer reports unexpected access. Users with indirect role inheritance can update records they should not be able to touch. There is no exploit signature and no obvious vulnerability. The issue sits in the interaction between role resolution, caching, and a fallback condition that appeared sensible but was wrong for this system.

No one intended this behavior, nor did they spot this in the review. Ultimately, no one can honestly say they fully understood the logic behind it.

At that point, the question of ownership becomes very uncomfortable.

Nothing Actually Went Wrong Here

From a delivery perspective, the process worked.

The developer reviewed the change, the internal standards were followed, the automated checks passed, and the release was shipped on time.

The failure wasn’t in the process, but in assuming that a review equates to true understanding.

AI‑generated code frequently appears correct in exactly the ways seasoned engineers expect: It compiles cleanly, reflects known patterns, and reads smoothly. The trouble only becomes visible once it hits real data, real traffic, and real‑world complexity.

So, this failure cannot be considered negligence. It is, in effect, structural.

Why Authorship No Longer Defines Accountability

When humans write code, intent and implementation tend to stay close. When AI generates code, it can separate.

The developer’s role shifts from author to approver. Approval becomes a stand-in for responsibility, even when understanding is partial. Over time, risk accumulates quietly. Logic is trusted because it looks familiar. Reviews become visual pattern checks rather than deep reasoning. Behavior is assumed rather than proven.

Security programs that still anchor accountability to authorship struggle in this model because authorship no longer carries the same meaning it once did.

The real question now isn’t who wrote the code, but what it actually does when the system is under real pressure.

Where Threat Modelling Quietly Breaks Down

One of the first things to suffer in AI-assisted development is informal threat modelling.

When humans wrote most of the code, much of the reasoning lived in their heads. Developers understood which execution paths mattered, where assumptions existed and which edge cases were dangerous. That knowledge was rarely documented, but it was real.

When code is generated, that implicit reasoning rarely occurs. Or it happens briefly and optimistically during review. Everyone assumes someone else has already thought it through.

This is where the role of threat modeling must fundamentally shift.

Not as a one-time exercise, and not as a documentation artifact, but as a way of making reasoning explicit again. A way of recording which assumptions the system depends on, where trust boundaries sit, and how authorization and state are expected to behave when things get complicated.

In an AI-assisted world, threat modeling becomes less about predicting future attacks and more about codifying understanding in systems where that understanding is no longer guaranteed.

What Security Must Do to Adapt

In an AI-assisted development model, security cannot rely on intent, confidence or code quality as sufficient signals of safety. They still matter, but they’re no longer decisive.

Security has to become far more explicit about what it trusts and why.

That means shifting the focus away from how code was produced and toward what can be demonstrated about its behavior. Not in theory, but under realistic conditions. Across execution paths that were never discussed in design. Authorization issues often surface only when identity, caching, and state interact under real‑world data and usage patterns—not the idealized ones used in basic testing.

This also changes how success is measured.

Many security metrics still measure activity rather than assurance. We count how many findings were raised, how many scans were run, and how many reviews were completed—yet none of these numbers tell us whether the system is actually operating safely.

As authorship becomes shared and understanding becomes more fragmented, our metrics must shift to reflect behavior: whether authorization decisions remain consistent over time, whether access paths drift as systems evolve, and whether controls silently erode as new features are added.

These are harder things to measure, but they provide something far more valuable than volume–based metrics. They provide confidence.

In practical terms, this means security teams have to stop treating reviews as a moment in time and start treating verification as an ongoing obligation. Confidence can no longer come from the fact that someone approved a change. It has to be supported by evidence that the system continues to behave as expected as it evolves.

This is not about banning AI tools or second-guessing developers. It is about recognizing that when authorship is shared, verification must be shared as well, and that it must be deliberate.

Where the Governance Challenge Emerges

The consequences of this shift do not stay within engineering teams.

When incidents occur, the questions that follow are rarely technical. Executives ask who approved the change. Auditors ask about what controls were in place. Boards ask how risk was assessed before release and how that assessment was validated.

These are tough questions to answer when all you can show is that the process ran and the pipeline went green.

This is also where audit quietly changes its purpose.

In many organizations, audit still focuses on whether the right steps were followed. Whether approvals happened. Whether checklists were completed. That approach works when authorship implies understanding. It breaks down when it does not.

In an AI-assisted development model, an audit has to focus on evidence. Evidence that risk was examined rather than assumed. Evidence that behavior was questioned rather than trusted. Evidence that someone asked what happens if this logic is wrong.

Without that shift, organizations find themselves defending decisions that were never consciously made. They were simply inherited from a development model that no longer fully applies.

At this point, this isn’t an engineering failure. It’s a governance gap.

Something Worth Reflecting On

AI does not remove responsibility. It essentially redistributes it.

If no single person fully authored the code, and no single reviewer fully understood its behavior, then accountability shifts toward evidence and verification rather than intent and trust.

The organizations that adapt will not be the ones that slow development down. They will be the ones who acknowledge that authorship no longer means what it used to and adjust their security posture accordingly.

Security is already moving away from trusting how code was written and toward proving how it behaves.

Whether that shift is recognized deliberately or forced by experience is largely a leadership decision.

Read whitepaper