Get in Touch
 Duration 21 hours

Course Outline

Module 1: How AI Apps Break

  • LLM, RAG, and agent architectures from the developer's side
  • The request and response lifecycle of an AI feature
  • Prompt flow: system, developer, user, and tool messages
  • Where untrusted data enters and re-enters the model
  • The trust boundaries a developer owns versus inherits
  • Why AI attacks are semantic, not syntactic
  • Mapping the OWASP LLM Top 10 to code you write

Module 2: Prompt Injection for Builders

  • Direct versus indirect prompt injection
  • Hidden instructions in documents, web pages, and tool output
  • Jailbreaks and role confusion
  • Why instruction and data separation matters
  • Defensive prompt design: delimiters, structure, minimal authority
  • Why prevention is partial, and designing for containment
  • Lab 01: attack your own chatbot, bypass a naïve filter, restructure the prompt

Module 3: Treating Model Output as Untrusted

  • Model output as untrusted input to the rest of the application
  • Insecure output handling: XSS, SSRF, and downstream command or SQL injection
  • Never eval, exec, or render raw model output
  • Structured outputs and schema validation
  • Output encoding and allowlists
  • Safe rendering in web and UI contexts
  • Lab 02: find and fix an insecure-output-handling bug

Module 4: RAG Security

  • Vector database and retrieval threats
  • Ingestion sanitization
  • Document provenance and trust scoring
  • Retrieval scoping and metadata isolation
  • Hidden instructions in retrieved content
  • Data exfiltration through retrieval
  • Lab 03: poison a RAG pipeline, then defend it

Module 5: Agent and Tool Safety

  • Excessive agency and tool abuse
  • Least privilege for agents
  • Tool allowlists and argument validation
  • Approval gates and human-in-the-loop
  • Sandboxing tool execution
  • Scoped, short-lived credentials and limiting autonomous loops
  • Lab 04: lock down an over-permissioned agent

Module 6: Secrets, Identity, and Cost

  • API key and secret management: never in prompts, code, or logs
  • Per-user authentication and authorization for AI features
  • Propagating user identity to tools and retrieval
  • Denial of wallet: unbounded token and cost consumption
  • Rate limits, token budgets, and timeouts
  • Logging without leaking secrets or PII
  • Lab 05: get secrets out of the prompt path and add cost limits

Module 7: Guardrail Libraries

  • What guardrail frameworks do and do not do
  • Input guardrails: injection, PII, and topic classifiers
  • Output guardrails: validation, filtering, grounding checks
  • When a guardrail is right versus your own deterministic check
  • Layering guardrails with the controls from earlier modules
  • Performance, false positives, and failure modes
  • Lab 06: add an input and output guardrail layer and measure it

Module 8: Red-Teaming Your Own App

  • Building an abuse and test suite for AI features
  • Automated prompt injection and jailbreak tests
  • Regression testing guardrails and policies
  • Running AI security checks in continuous integration
  • Model and dependency supply chain: provenance and pinning
  • A pre-ship security checklist for AI features
  • Lab 07: write automated red-team tests and wire them into CI

Module 9: Scoring AI Security, the SAIS-100 Framework

  • The AI Security Hexagon: six questions instead of is it secure
  • The six scored categories: data, prompt, agent, supply chain, detection, governance
  • The 100-point rubric and its weightings
  • Verdict bands and the single-category override rule
  • Scoring before and after hardening as a metric
  • Scoring exercise against the capstone application

Capstone: Harden an AI Application

  • The starter app: an injectable prompt, insecure output handling, unscoped RAG
  • The starter app: an over-permissioned agent, secrets in the prompt path, no cost limits
  • Restructuring prompts for containment and validating model output
  • Sanitizing and scoping retrieval, then applying least privilege and approval gates
  • Moving secrets out and adding cost and rate limits
  • Adding guardrails and automated red-team tests
  • Deliverable: a hardened app plus an OWASP LLM Top 10 self-assessment

Requirements

  • Comfort building and consuming REST APIs.
  • A scripting language, with labs delivered in Python.
  • Basic application authentication, git, and command line familiarity.
  • No machine learning background required; this is an application security course for people who build with LLMs, not one that trains them.

Audience

  • Software and backend engineers building LLM features.
  • Full-stack and API developers.
  • AI and ML application engineers.
  • Platform engineers shipping copilots and agents.
  • Tech leads and senior engineers who own AI features.

Testimonials (3)

Related Categories