AI Red Teaming Course
Learn how AI systems get attacked and how to test them: prompt injection, jailbreaks, data leaks, and defenses.
26 lessons · about 7 hours of reading · Free
Start lesson 1Course syllabus26 Lessons
What Is AI Red Teaming?
Learn how an AI red team tests an AI system for security, safety, privacy, and other failures, then helps its owners fix and retest them.
Ethics, Authorization, and Rules of Engagement
Learn how to confirm permission, set safe boundaries, handle test data, and prepare rules of engagement before testing an AI system.
AI and Machine Learning Basics for Red Teamers
Learn the basic AI and machine learning terms a red team needs, from data and training to prompts, model behavior, and the deployed application.
Security Principles and AI Threat Modeling
Learn to map what an AI system must protect, where risks can enter, and how to turn those risks into safe, testable questions.
Build a Safe AI Red Team Test Lab
Set up an offline HelpDesk Bot practice target with a stub model, a fake canary, request logs, and repeatable tests.
Jailbreaks and Safety Filter Bypasses
Learn how jailbreaks challenge AI safety controls and how to measure those controls with authorized, harmless test cases.
Prompt Injection: Direct and Indirect
Learn how instructions can reach an AI through a user message or retrieved content, and test both paths with a synthetic canary.
Code Injection and Tool Abuse
Learn why model output must be treated as untrusted, and test tool permissions and confirmation with a harmless local simulator.
System Prompt and Data Leakage
Test where hidden context and synthetic user data can escape through model responses, retrieval, and logs.
Countermeasures for Prompt Attacks
Reduce prompt attack impact with layered controls, then retest the local HelpDesk Bot against direct, indirect, and tool paths.
Data Poisoning and Backdoors
Learn how corrupted training examples can alter model behavior and test a trigger-based backdoor in a tiny offline classifier.
Adversarial Examples
Test how small input changes can shift a classifier's prediction using a two-feature local model and a bounded synthetic example.
Model Extraction and Weight Theft
Understand how exposed prediction interfaces and weak artifact access can put model value at risk, and review practical safeguards.
Model Inversion and Privacy Leaks
Learn how model behavior can reveal training-set membership or sensitive patterns, and run a synthetic membership test locally.
Model Manipulation and the AI Supply Chain
Trace the components behind an AI model and use a practical intake checklist to spot tampering, untrusted code, and unclear provenance.
AI Infrastructure, Deserialization, and Remote Code Execution
Review how model files are loaded and how exposed services, notebooks, secrets, containers, and networks can put an AI deployment at risk.
API Authentication and Authorization
Test who can call an AI endpoint, which data and tools each identity can reach, and how quotas limit accidental or abusive usage.
Black, Grey, and White Box Testing
Choose a testing approach based on what the red team can see, then map that access to the risks and evidence the engagement needs.
Manual, Automated, and Continuous AI Testing
Combine human-led exploration with repeatable security checks, then run the right tests during development, release, and operation.
AI Red Teaming Tools, Frameworks, and Datasets
Choose tools by the question you need to answer, and evaluate datasets for fit, safety, provenance, and coverage.
Write Your Own AI Security Test Harness
Build a small offline runner that replays synthetic security cases, applies explicit graders, records safe results, and compares versions.
Report AI Security Findings and Track Fixes
Turn test results into clear, redacted findings that engineers can fix and teams can retest.
Defense in Depth for AI Systems
Plan controls across the model, application, data, tools, and operations, then monitor whether they hold.
Red Teaming LLM Applications and Agents
Plan an authorized test across model, data, tools, memory, and agent connections, then record what the local lab can and cannot establish.
Disclosure, Law, Careers, and What's Next
Learn how to report a security issue responsibly, respect legal and program boundaries, and build a practical path into AI red teaming.
Capstone: Red Team a Practice App
Run an authorized end-to-end security review of the fictional HelpDesk Bot, from scope and threat model to fixes, retest, and final report.
Related courses
Get AI Red Teaming by email
One lesson a week, in order. Free, and you can unsubscribe from any email.
Frequently asked questions
Who is the AI Red Teaming course for?
Developers, security engineers, ML engineers and product owners who need to find weaknesses in AI systems before attackers do, plus students and career-changers.
What do I need before starting the AI Red Teaming course?
No security background. Basic familiarity with using language models helps.
What will I learn in the AI Red Teaming course?
You will red team a practice app using a repeatable method: scope, threat model, test, record, report, fix and retest.
How long does the AI Red Teaming course take?
It has 26 lessons and about 7 hours of reading, before you spend time on the practice tasks. Go at your own pace; the lessons are written to be done in order.
Is the AI Red Teaming course free?
Yes. Every lesson is free to read, and you do not need to sign in.