Quick answer: AI code security is the set of controls that keep code produced by assistants and coding agents from introducing vulnerabilities, malicious dependencies or leaked secrets. The evidence so far is consistent: AI-written code is roughly as flawed as the average human code it was trained on, developers trust it more than they should, and the fix is not banning it but treating it as untrusted input to the same pipeline gates, reviews and dependency policies you apply to everything else.
AI-generated code security went from a conference curiosity to a board question in about eighteen months. A large share of new enterprise code is now written or shaped by assistants, and the shift to autonomous coding agents that open pull requests, run tools and install packages has changed the threat model again. "Vibe coding", accepting generated code without reading it closely, is now happening in regulated environments on production systems.
The AppSec response has split between prohibition, which engineering ignores, and uncritical adoption, which treats the assistant as a senior reviewer. This article is for AppSec and engineering leaders who need a third position: a governance model that assumes AI-assisted development is the default, identifies where its risks differ from human-written code, and routes them into controls the organization already runs.
Is AI-generated code actually less secure?
The research direction is clear even where the precise numbers vary by study, model and language.
- Vulnerability rates are comparable to human code, not better. Veracode's 2025 GenAI Code Security Report, which tested leading models against tasks with known secure and insecure implementations, found that roughly 45% of generated samples contained a security flaw, with weaknesses such as cross-site scripting and log injection being chosen insecurely most of the time. Newer, larger models wrote more functional code but not meaningfully more secure code.
- Developers over-trust it. A Stanford study of developers using an AI assistant found that participants with assistant access wrote less secure code on several tasks and were more confident that their code was secure than those without. That confidence gap, not the raw defect rate, is the risk that compounds.
- Models reproduce the training distribution. Assistants learn the outdated patterns in public repositories: string-concatenated SQL, weak hashing, permissive CORS, deprecated crypto APIs, hard-coded example secrets.
A model is a very fast junior developer with an excellent memory and no judgment about your threat model. Treat its output accordingly.
What new risks do coding assistants and agents introduce?
Beyond ordinary defect rates, three risk classes are specific to AI-assisted development and are often missing from existing controls.
- Package hallucination and slopsquatting — What happens: The model recommends a package that does not exist; attackers register that name on npm or PyPI with malicious content, and the next developer who accepts the same suggestion installs it. Research has shown hallucinated package names recur predictably across prompts, making them easy to squat · Why existing controls miss it: SCA checks known vulnerabilities in real packages; it does not flag that a brand-new, low-download package was pulled in because a model invented it
- Prompt injection into coding agents — What happens: An agent that reads issues, documentation, web pages or repository files can be instructed by hidden text to exfiltrate secrets, modify CI configuration or introduce a backdoor. Supply chain attacks via poisoned README files and tool descriptions have been demonstrated · Why existing controls miss it: Code review looks at the diff, not at what instructed the agent; CI runners give agents broad credentials
- Data leakage and secret exposure — What happens: Developers paste proprietary code, credentials or customer data into prompts; assistants autocomplete secrets from context; agents commit .env files or embed keys · Why existing controls miss it: DLP tools rarely cover IDE plugins and agent traffic; secrets scanners catch some, after the fact
A fourth, quieter risk is volume: if assistants lift engineering throughput substantially, AppSec fix capacity does not grow with it, which makes vulnerability prioritization and automation the prerequisite for AI-scale development.
How should an enterprise govern AI coding assistants?
A governance model that practitioners are converging on has five layers. None of them is new technology; the work is in applying them deliberately to AI-generated code.
1. Approved tooling and data boundaries. Sanction specific assistants and agents under enterprise agreements with no training on your code, data residency you can defend, and audit logging. Block unsanctioned tools at the network and IDE level where feasible. Define what may never be in a prompt (production credentials, customer data) and enforce it with DLP on the approved tools' traffic rather than relying on policy alone.
2. Dependency policy that assumes hallucination. Extend your supply chain controls: a private registry with allow-listing or a cooling-off period for new packages, automatic flagging of any dependency with very low download counts or a very recent first publish date, and verification of provenance before install. Agents should install from the proxy registry only, never directly from public registries. This closes slopsquatting almost entirely.
3. Pipeline gates that do not care who wrote the code. SAST, secrets detection, SCA with reachability and IaC scanning run on every pull request regardless of author. The key adjustment is to tune for the failure modes assistants favor: injection sinks, insecure deserialization, weak crypto, hard-coded secrets, overly broad IAM policies. If your gates are advisory today, AI-scale development is the reason to make them blocking for the top tier.
4. Review rules proportionate to risk, with provenance. Require that AI-generated contributions be identifiable, through commit trailers, pull request labels or tooling metadata, not to shame anyone but so review effort can be targeted and defects can be analyzed by origin. Mandate human review for AI-generated changes to authentication, authorization, cryptography, payment flows and infrastructure. Allow AI-assisted review (a second model critiquing the first) as an additional layer, never as the only one.
5. Agent-specific controls. Treat coding agents as non-human identities with least-privilege, short-lived credentials; run them in isolated environments with no production access; restrict their tool and network permissions; log every action; and treat any content they read (issues, docs, web pages, tool descriptions) as untrusted input that may contain instructions. OWASP's work on LLM application risks is the right reference for the prompt injection class.
How to run a Claude Code security review (and the same for Copilot and Cursor)
AI-assisted review is now built into the assistants themselves, which makes it tempting to treat it as the security gate. Treat it as one more reviewer instead, and set it up the same way across tools.
Where the review runs. Claude Code ships a /security-review command that a developer can run against pending changes locally, and Anthropic publishes an open-source GitHub Action that runs the same kind of review on pull requests and posts findings as comments. GitHub Copilot offers code review on pull requests and Copilot Autofix suggestions on CodeQL code scanning alerts. Cursor's Bugbot reviews pull requests in the same way. Pick one per repository and run it on every PR, not on request.
What to trust it with. Model review is good at spotting logic flaws, missing authorization checks and risky patterns a rule set does not cover. It is not deterministic, so it cannot replace blocking SAST, SCA and secrets gates. Before relying on it, run it against code you know is vulnerable (a deliberately insecure app such as OWASP Juice Shop, or your own seeded test cases) and record what it catches and what it invents.
Lock down the reviewer itself. A review agent reads untrusted input: the diff, comments and files an attacker may have written. Anthropic's documentation for its action warns that it is not hardened against prompt injection and should be used on trusted pull requests. Run review jobs with read-only repository tokens and no deployment secrets, and use each tool's permission settings (Claude Code's allow and deny rules for commands and files, organization-managed settings, and the equivalent controls for Copilot's coding agent and Cursor's auto-run mode) so the assistant cannot execute arbitrary commands unattended.
What do NIST SP 800-218A and other frameworks say?
Regulators and standards bodies are catching up quickly, and 2027 is when several of them bite.
- NIST SP 800-218A is a community profile of the Secure Software Development Framework for generative AI and dual-use foundation models. It augments the SSDF's four practice groups (Prepare the Organization, Protect the Software, Produce Well-Secured Software, Respond to Vulnerabilities) with AI-specific tasks, and is the natural framework for documenting how AI-assisted development is governed in a US federal supply chain context.
- The EU AI Act applies obligations by risk tier; most coding assistants are general-purpose AI systems whose providers carry transparency and documentation duties, while deploying organizations in regulated sectors must still meet existing software security obligations under NIS2, DORA and the Cyber Resilience Act for the code they ship.
- CISA's Secure by Design principles apply directly: take ownership of customer security outcomes, embrace radical transparency, and lead from the top. AI-generated code does not change who is accountable for a product's security.
- Financial regulators (the UK FCA and PRA, APRA under CPS 234, the SEC) have not issued AI-coding-specific rules, but their third-party and operational resilience expectations already cover the tools and the code. Expect examiners to ask how AI-generated code is identified and reviewed.
In practice: a bank rolls out coding agents to 2,000 engineers
Scenario. A bank has run an approved assistant for a year and now wants to allow autonomous coding agents to pick up tickets, write code and open pull requests. Security has 90 days to produce the governance model.
Weeks 1-3: boundaries. Two agent products are approved under contracts with no training on bank code and full audit logging. Agents run only in isolated cloud workspaces with scoped, short-lived credentials: read access to the target repository, write access to a branch, no production or data access. Network egress is restricted to the private package registry and the source control system.
Weeks 4-6: supply chain. The private registry adds a 14-day cooling-off rule and blocks packages below a download threshold without a champion override. In the first month this quarantines eleven packages suggested by agents, three of which do not exist on the public registry at all.
Weeks 7-10: pipeline and review. Agent-authored pull requests carry a mandatory label and commit trailer. Pipeline gates (SAST, secrets, SCA with reachability) become blocking for critical and high findings on all PRs. A rule in the review system requires two human approvals for agent PRs touching the authentication, payments or IAM code paths, and a second-model review runs on every agent PR as an additional, non-authoritative signal.
Weeks 11-13: measurement. The ASPM layer tags findings by author type. After one quarter, agent PRs show a defect rate per thousand lines comparable to human PRs, a higher rate of hard-coded test credentials, and a much lower rate of merged dependency violations, because the registry policy makes those impossible. The data goes to the risk committee as evidence rather than anecdote.
Engineering was not slowed down. The risk was made visible and routed through existing controls, with three targeted additions: registry policy, provenance labeling and agent identity management.
Key takeaways
- AI-generated code is about as flawed as human code and trusted more; the governance target is the confidence gap, not the tool.
- Slopsquatting and prompt injection into agents are new risk classes; registry policy and agent identity controls close most of the exposure.
- Pipeline gates should not care who wrote the code, but should be tuned for the failure modes assistants favor and made blocking for the top tier.
- Label AI-generated changes and require human review for authentication, cryptography, payments and infrastructure.
- NIST SP 800-218A, the EU AI Act and existing sector regulation already frame the accountability; examiners will ask how AI code is identified and reviewed.
Join your peers at AppSec World 2027
AppSec World is the global application security forum: one meeting place for AppSec, product security and engineering leaders, with content by practitioners for practitioners and no product pitches. Join us in 2027 in New York, May 5, San Francisco, May 20, Melbourne, June 3, Sydney, June 10 and London, October 27.
Not ready to register? Sign up for updates and we will let you know when agendas and speakers are announced.