llm-security

by semgrep

Security guidelines for LLM applications based on OWASP Top 10 for LLM 2025. Use when building LLM apps, reviewing AI security, implementing RAG systems, or… npx skills add https://github.com/semgrep/skills --skill llm-security Download ZIPGitHub232

LLM Security Guidelines (OWASP Top 10 for LLM 2025)

Security rules for building secure LLM applications, based on the OWASP Top 10 for LLM Applications 2025.

What Are You Building?

Use this to quickly identify which rules matter most for the user's task:

Building... Priority Rules Chatbot / conversational AI Prompt Injection (LLM01), System Prompt Leakage (LLM07), Output Handling (LLM05), Unbounded Consumption (LLM10) RAG system Vector/Embedding Weaknesses (LLM08), Prompt Injection (LLM01), Sensitive Disclosure (LLM02), Misinformation (LLM09) AI agent with tools Excessive Agency (LLM06), Prompt Injection (LLM01), Output Handling (LLM05), Sensitive Disclosure (LLM02) Fine-tuning / training Data Poisoning (LLM04), Supply Chain (LLM03), Sensitive Disclosure (LLM02) LLM-powered API Unbounded Consumption (LLM10), Prompt Injection (LLM01), Output Handling (LLM05), Sensitive Disclosure (LLM02) Content generation Misinformation (LLM09), Output Handling (LLM05), Prompt Injection (LLM01)

Categories

Critical Impact

• LLM01: Prompt Injection (rules/prompt-injection.md) - Prevent direct and indirect prompt manipulation

• LLM02: Sensitive Information Disclosure (rules/sensitive-disclosure.md) - Protect PII, credentials, and proprietary data

• LLM03: Supply Chain (rules/supply-chain.md) - Secure model sources, training data, and dependencies

• LLM04: Data and Model Poisoning (rules/data-poisoning.md) - Prevent training data manipulation and backdoors

• LLM05: Improper Output Handling (rules/output-handling.md) - Sanitize LLM outputs before downstream use

High Impact

• LLM06: Excessive Agency (rules/excessive-agency.md) - Limit LLM permissions, functionality, and autonomy

• LLM07: System Prompt Leakage (rules/system-prompt-leakage.md) - Protect system prompts from disclosure

• LLM08: Vector and Embedding Weaknesses (rules/vector-embedding.md) - Secure RAG systems and embeddings

• LLM09: Misinformation (rules/misinformation.md) - Mitigate hallucinations and false outputs

• LLM10: Unbounded Consumption (rules/unbounded-consumption.md) - Prevent DoS, cost attacks, and model theft

See rules/_sections.md for the full index with OWASP/MITRE references.

Quick Reference

Vulnerability Key Prevention Prompt Injection Input validation, output filtering, privilege separation Sensitive Disclosure Data sanitization, access controls, encryption Supply Chain Verify models, SBOM, trusted sources only Data Poisoning Data validation, anomaly detection, sandboxing Output Handling Treat LLM as untrusted, encode outputs, parameterize queries Excessive Agency Least privilege, human-in-the-loop, minimize extensions System Prompt Leakage No secrets in prompts, external guardrails Vector/Embedding Access controls, data validation, monitoring Misinformation RAG, fine-tuning, human oversight, cross-verification Unbounded Consumption Rate limiting, input validation, resource monitoring

Key Principles

• Never trust LLM output - Validate and sanitize all outputs before use

• Least privilege - Grant minimum necessary permissions to LLM systems

• Defense in depth - Layer multiple security controls

• Human oversight - Require approval for high-impact actions

• Monitor and log - Track all LLM interactions for anomaly detection

References

• OWASP Top 10 for LLM Applications 2025

• MITRE ATLAS - Adversarial Threat Landscape for AI Systems

• NIST AI Risk Management Framework