Red-Teaming

Red-teaming is the practice of systematically attempting to elicit harmful, incorrect, or policy-violating behavior from an AI system by simulating adversarial users. Red teams use creative prompting, jailbreak attempts, edge-case scenarios, and multi-step manipulation strategies to find gaps in safety training before public deployment. Findings from red-teaming exercises directly inform additional safety fine-tuning, guardrail rules, and policy updates. Both human red teams and automated LLM-based red-teamers are used at scale.