Course Outline
Module 1: AI Systems for Security Engineers
- LLMs versus normal applications
- AI inference pipelines and prompt flow
- RAG architecture, embeddings, and vector databases
- Agentic workflows and tool calling
- AI gateways, copilots, MCP, and agent protocols
- Where WAF visibility exists, and where it disappears
Module 2: OWASP GenAI Top 10
- Prompt injection, insecure output handling, training data poisoning, model denial of service
- Supply chain vulnerabilities, sensitive information disclosure, excessive agency
- Vector and embedding weaknesses, misinformation, unbounded consumption
- Differences from the classic OWASP list
- Mapping categories to WAF, gateway, and application controls
- Where each control helps and where it fails
Module 3: Prompt Injection Detection
- Direct and indirect prompt injection
- Hidden instructions and document-based attacks
- HTML and Markdown injection
- Jailbreak patterns, context override, and role confusion
- Detection: keyword heuristics, semantic classification, prompt linting, boundary enforcement
- Allow and deny policies, and AI-aware regex patterns
- Lab 02: attack a chatbot, bypass naïve filtering, build layered detection
Module 4: AI-Aware WAF Rules
- Protecting LLM endpoints and inference APIs
- Token-aware rate limiting and prompt size inspection
- AI-specific signatures and conversation anomaly detection
- Multi-turn abuse patterns, model enumeration, inference scraping
- Denial of wallet protection
- Examples: protecting chat completions, streaming APIs, and recursive agent calls
- Lab 03: build AI-aware WAF rules
Module 5: Securing RAG Pipelines
- Vector database threats and embedding poisoning
- Malicious PDFs and documents, and semantic poisoning
- Retrieval manipulation and hidden instructions
- Cross-document contamination and data exfiltration via retrieval
- Defenses: ingestion sanitization, trust scoring, metadata isolation, provenance, retrieval policies
- Lab 04: upload a poisoned PDF and take over the assistant
Module 6: Agentic AI Security
- Excessive agency, tool abuse, API chaining, autonomous loops
- Permission escalation, memory poisoning, indirect tool execution
- Agent impersonation, credential leakage, multi-agent attacks
- Task-scoped authority and distinct user, runtime, task, and session identities
- Least privilege, approval gates, runtime policy engines, sandboxing, tool whitelisting
- Lab 05: secure an autonomous agent
Module 7: API Security for AI
- API gateways, GraphQL AI risks, MCP and API abuse
- JWT protection, AI plugin security, agent authentication
- Delegated authorization and short-lived task credentials
- Separate task stopping, revocation, and downstream verification
- Secret management, signed prompts, and an API inventory for AI
- Tying controls back to the OWASP API Security Top 10
- Lab 06: detect denial-of-wallet attacks
Module 8: Detection Engineering and SOC Integration
- AI telemetry, prompt logging, token analytics
- Anomaly detection and semantic SIEM pipelines
- AI attack indicators and threat hunting for LLM abuse
- Runtime observability and correlating principal, task, action, policy decision, and effect
- Detecting jailbreak campaigns, automated agent abuse, model scraping
- Agent incident drills: stop, contain, recover, learn
- Lab 07: monitor AI abuse patterns in logs
Module 9: Cloud WAFs and AI Security
- AWS WAF for AI APIs, Azure WAF, Cloudflare AI Gateway
- API gateways, Envoy AI filtering, Kong AI Gateway, NGINX patterns
- Traditional WAF versus AI gateway versus application-layer guardrail
- Proxy-based versus semantic filtering
- Shared responsibility for inference, harnesses, and enterprise connectors
- Placement of action, resource, and egress enforcement
Module 10: Building a Layered AI Defense
- No single layer secures AI, a WAF least of all
- The layers: WAF, API gateway, AI gateway, guardrails, runtime monitoring
- Identity and authorization, sandboxing, human approval
- Observability and incident response
- Lab 08: build a layered AI defense architecture
Module 11: AI for Defense, from Findings to Verified Fixes
- The defender's window and continuous defense
- Scoped access, security context, and security invariants
- AI-assisted triage and evidence-based prioritization
- Duplicate findings and incomplete validation environments
- Local vulnerability reproduction, and security regression tests
- Focused patch generation with independent human review
- Defensive assistant permission checks and preserving security tests
- Lab 09: investigate five reports against a synthetic multi-tenant support-ticket API
Module 12: AI-Enabled Cyber Defense, the Defensive Loop
- Prevent, detect, respond, learn, improve, prevent
- Detect and respond: recognize a compromised agent and contain it at runtime
- Discover and validate: find real vulnerabilities at scale and separate them from noise
- Remediate and prove: generate a patch, verify it, and prove the vulnerability is gone
- The capability ladder, and which controls need privileged access
- Lab 10: close the defender's window
Capstone: Defend a Simulated Enterprise Assistant
- Attackers attempt prompt injection, tool abuse, credential theft
- Attackers attempt retrieval poisoning, excessive API consumption, agent escalation
- Teams build WAF rules, AI gateway policies, runtime detection
- Teams build guardrails and an incident response runbook
Requirements
- Working knowledge of application security, API security, or SOC operations.
- Familiarity with web attack classes such as injection and access control failures.
- Basic understanding of how LLM applications are built.
- Comfort with command line tools and HTTP traffic inspection.
Audience
- Security engineers and AppSec practitioners.
- SOC analysts and detection engineers.
- API and cloud security engineers.
- Platform, infrastructure, and WAF engineers.
- Security architects and team leads responsible for AI systems in production.
Testimonials (3)
AI-ə daxilindəki müxtəlif risk məruz qalmalarını qeydiyyata almaq və müəyyən etmək
Gary Cook - Cybersecurity and Information Technology Risk Division
Course - Introduction to AI Trust, Risk, and Security Management (AI TRiSM)
Machine Translated
I really enjoyed learning about AI attacks and the tools out there to begin practicing and actively using for security testing. I took a lot of knowledge away which I didn't have at the beginning and the course met what I hoped it would be. My favorite part shown from the training was Comet Browser and was amazed at what it could do. Definitely something will be looking into more. Overall it was a great course and enjoyed learning all OWASP GenAI Top 10.
Patrick Collins - Optum
Course - OWASP GenAI Security
The profesional knolage and the way how he presented it before us