On this pageHow the Terms Are DefinedDifferences Side By SideOne Organization Under Both ExercisesHow a Red Team Exercise RunsFindings From CISA Red Team AssessmentsRecords From Each ExerciseMeasuring and Using the ResultsReadiness for Each ExerciseRed Teaming By Organization SizeWhere Purple Teaming FitsRegulator-Led Testing FrameworksStandards That Require Each ExerciseAI Red TeamingPreparing for a Red Team ExerciseThe Testing Side With ThreatDefenceFrequently Asked Questions Red teaming is an authorized simulation of a real adversary pursuing an objective across the whole organization, so the owner learns whether its people, processes and technology detect and stop the attack. The terms are often used loosely, and many engagements sit somewhere between them. This guide explains how the two differ in goal, scope, duration and output, where purple teaming fits, which regulator-led frameworks exist, and how to tell which exercise an organization is ready for. How the Terms Are DefinedBoth terms have settled definitions in public standards. The NIST glossary carries the NIST SP 800-115 definition of penetration testing: security testing in which assessors imitate real attacks to get past the security features of an application, system or network. The same glossary describes a red team as a group authorized and organized to emulate a potential adversary’s attack capabilities against an enterprise’s security posture. NIST SP 800-53 connects them. Its penetration testing control, CA-8, lists red team exercises as an enhancement that extends the objectives of a penetration test to the organization’s capability to mount an effective defense. The control adds that penetration testing may be primarily laboratory-based, while a red team exercise reflects real-world conditions and may include social engineering. Other roles appear throughout this guide. Blue team: The blue team is the organization’s defenders, typically the SOC, the incident responders and the administrators who support them. Control group: The control group is the few people inside the organization who know about the exercise, approve its risks and can stop it, and some documents call it the white team. Our guide to penetration testing covers test types, knowledge levels and phases in detail. Differences Side By SideThe clearest difference is the question each exercise answers. One asks which weaknesses in the agreed systems can be exploited, and the other asks whether a capable attacker could reach a specific objective before the organization noticed. Penetration test Red team exercise Goal Find and confirm as many exploitable weaknesses as possible. Reach an agreed objective the way a real adversary would. What is measured The security of the systems in scope. Prevention, detection and response across people, process and technology. Scope A defined list, such as an application, an address range or a cloud tenant. The organization, with a short list of exclusions. Social engineering and physical access may be included. Approach Broad coverage, with noise accepted. One workable path at a time, kept quiet. Duration Typically days to a few weeks. Typically weeks to months. Who knows The system owners and usually the security team. The control group. The defenders are typically unaware. Threat intelligence Optional. Usually shapes the scenarios. Output Findings with severity, proof and fix steps. An attack narrative with a timeline, the points where defenses held or missed, and recommendations for detection and response. Follow-up A retest of each fix. A replay with the defenders, then changes to detections, playbooks and training. The trade is coverage for realism. A penetration tester who finds ten weaknesses reports all ten. A red team that finds ten may use the quietest and report the path, so its report is evidence about detection and response and an incomplete inventory of vulnerabilities. Automated scanning sits a level below both, and our comparison of penetration testing vs vulnerability scanning covers that boundary. One Organization Under Both ExercisesThe same organization gets different answers from each exercise, and a worked case shows how little the two reports overlap. Take a mid-sized insurer with a policyholder portal, a VPN gateway, a Microsoft 365 tenant and an internal system that approves claims payments. The penetration test is scoped to the portal and the gateway, runs for two weeks, and the security team knows the dates. The tester reports every weakness found on both systems. Password reset on the portal: The reset form confirms whether an email address belongs to a policyholder, which lets an attacker build a list of valid accounts. Sign-in on the portal: Failed sign-ins are never slowed or locked, so that list can be tried against common passwords. VPN gateway: The gateway runs a software version with a published vulnerability, and the tester confirms the flaw is exploitable from the internet. The red team exercise takes a business objective: stage a claims payment to an account the team controls, without releasing it. Only the control group knows, and the exercise runs for ten weeks. Week two. A phishing page captures one claims handler’s password and an approved sign-in prompt, and the team connects through the VPN gateway as that user. Another recipient reports the email, and the SOC closes the report as spam. Week four. A shared mailbox holds a spreadsheet with a service account password, and the team uses it on an internal server. The sign-in raises a low-severity alert that stays unworked. Week seven. From that server the team reaches the claims payment system and stages a payment for approval. Week ten. The control group ends the exercise and the defenders are told. The red team report says little about the portal and leaves out the gateway’s software flaw, since the team signed in with a valid account. It shows that a reported phishing email was closed without a check on who else received it, that a service account signing in from a new host was rated low, and that the payment system accepted a staged payment from an account with no claims role. Each report describes weaknesses the other exercise had no reason to look for. How a Red Team Exercise RunsA red team exercise follows the course of a real intrusion, with the control group managing risk at each step. Set the objective. The control group and the red team agree what the adversary is after in business terms, such as access to a named system or a payment staged but never sent. Gather threat intelligence. The team studies which adversaries target the sector, how they operate, and what the organization exposes. Agree the rules of engagement. Exclusions, actions that need approval, emergency contacts and the way the exercise is halted are written down and signed. Gain initial access. The team may use phishing, an exposed service or valid credentials. Where time is short, the control group may grant a foothold so the exercise starts from an assumed breach. Move toward the objective. Operators escalate privileges and move between systems while staying quiet. Demonstrate the objective. The team proves it reached the target with the least intrusive evidence and stops short of real harm. Close out. The defenders are told, the attacker’s timeline is compared with what the SOC recorded, and many exercises end with a replay. Engagements differ in how much of that sequence they cover. Full-scope exercise: The team starts with no access and no inside knowledge and works through every step, including initial access. Assumed breach exercise: The team starts from a workstation or account that the control group provides, so the available time goes into what an attacker does after getting in. Adversary emulation: The team copies the documented techniques of one threat group that targets the sector, commonly planned against MITRE ATT&CK. Physical and social engineering scenarios: Entry to an office, calls to the help desk or impersonation of staff are added where the rules of engagement allow them. The control group can also assist the red team in the open while an exercise is running. Australia’s CORIE framework calls this a concession: help that the control group authorizes and the execution report records, such as a foothold where one could not be obtained in a reasonable time, or access to a non-production copy of a system where the same action in production is judged too risky. Where the defenders have noticed nothing, the same framework lets the red team seek approval for increasingly noisy actions until something is detected. Findings From CISA Red Team AssessmentsCISA publishes the results of red team assessments it carries out for federal agencies and critical infrastructure organizations, and the reports show what an exercise finds that a list of vulnerabilities leaves out. CISA describes its SILENTSHIELD assessments as a “no-notice, long-term simulation of nation-state cyber operations”. In an assessment of a federal civilian agency published in July 2024, the work ran for about eight months, the red team remained undetected by network defenders throughout the first phase, and three of those months were spent in technical collaboration with the agency’s defenders. Access came through an unpatched web server in one part of the network and through phishing in another. A separate CISA red team assessment of a critical infrastructure organization, published in November 2024, ran for about three months. Defenders detected the initial compromise and some of the red team’s movement, and CISA concluded that the organization relied too heavily on host-based endpoint detection and lacked sufficient network-layer protections. The lessons CISA drew from these and from an earlier assessment sit above the level of a single weakness. Detection built on known indicators: In the federal agency assessment, CISA found that a “known-bad” detection approach hampered detection of alternate techniques, and it recommended detections based on behavior over indicators tied to one tool. Logs that expire or go unread: The same agency kept some logs for only 60 to 90 days, and some data that recorded the intrusion was captured but never analyzed. Events that draw no response: In a 2022 assessment of a large critical infrastructure organization, the red team ran 13 measurable events designed to provoke a response, and CISA’s results table records no reaction to ten of them. Risk decisions above the security team: In the November 2024 assessment, CISA found that leadership had deprioritized the treatment of a vulnerability that the organization’s own cybersecurity team had identified. Most of these are properties of monitoring, staffing and decision-making, which is the layer a red team exercise is designed to examine. Records From Each ExerciseEach exercise leaves records that outlast the report, and a red team exercise is only as useful as the comparison between the attacker’s log and the defender’s. Record Contents Produced by Rules of engagement Scope, exclusions, approvals and stop conditions. Signed before either exercise and held by the system owner or control group. Findings report Each weakness with its severity, affected asset, proof such as a request and response, and fix steps. A retest result is added when the fix is checked. The penetration tester. Targeted threat intelligence report The adversaries and scenarios the red team will copy. The threat intelligence provider, before testing starts. Red team operator log Each action with its time, source host, account and technique. The red team, commonly labeled with MITRE ATT&CK technique IDs. Defender telemetry The alerts raised, cases opened and response times for the same period. The SIEM, the endpoint console, identity provider sign-in logs, firewall and proxy logs, and SOC case records. Detection and response matrix For each technique, whether it was prevented, detected, alerted on, responded to or missed. Built at close-out by lining up the operator log against defender telemetry. Remediation plan An owner and a date for each gap. The organization, after the debrief. Log retention decides whether the comparison is possible. Where logs expire before close-out, the early weeks of the attack path cannot be matched to anything the defenders saw. Measuring and Using the ResultsA red team exercise is measured in times and decisions, where a penetration test is measured in findings opened and closed. Objective reached: The report states whether the team reached the objective and how much assistance from the control group it needed. Time to first detection: This is the interval between the team’s first action and the first alert or staff report that a defender treated as suspicious. Time to containment: This is the interval between that detection and the point where the team lost its access. Detection coverage: The detection and response matrix gives the share of the team’s techniques that were prevented, alerted on or missed. Escalation quality: The comparison shows whether alerts and staff reports were triaged correctly, linked to each other and passed to the right people. CORIE states the aim of its replay exercise as reducing the mean time to detect and respond to real adversaries, and it asks replay reports to show improvement in time and coverage. The results are then worked through in order. Clean up. The red team removes its accounts, tools and persistence, and reports anything it could not remove so the organization can finish the job. Debrief the defenders. The attack narrative is walked through with the SOC against its own alerts and cases. Assign each gap. Missed detections go to whoever writes detection rules, process gaps go to the SOC manager, and weaknesses in systems go to their owners, each with a date. Check the changes. Techniques that were missed are run again once the fixes are in place. Report upward. Leadership receives the objective, the outcome, the times and the plan in business terms. Readiness for Each ExerciseThe right exercise depends on what the organization can already do. Penetration testing comes once scanning and patching are routine, and a red team exercise comes last, after a SOC is working alerts and purple teaming has confirmed what the monitoring sees. A red team exercise measures detection and response, so both need to exist first, and the guide to offensive security sets out the full order of disciplines by maturity. Specific situations point to a specific exercise. A new application or a major change: A penetration test of that system fits. A contractual or compliance requirement: A penetration test meets it in most cases, and the standards section below lists the frameworks that ask for more. A new SOC, SIEM or detection tool: A purple team exercise confirms what the new monitoring sees. A board asking whether a determined attacker could reach a critical system: A red team exercise takes that system as its objective. Red Teaming By Organization SizeHow much adversary simulation an organization needs depends on its size, its monitoring and who runs that monitoring. Smaller organization without a SOC: Penetration testing and patching give the most value, and a red team exercise would mostly confirm the absence of monitoring. Mid-sized organization with a managed SOC: A short assumed breach exercise tests what the provider detects and how quickly it escalates. The control group decides in advance who at the provider is told, and CORIE lists holding back the escalation of a detection, for example to an external incident response provider, among the concessions a control group may authorize. Large enterprise: An internal red team may run exercises through the year, with an external team brought in periodically for an independent view. Supervised financial institution: The regulator’s framework sets the scope, the providers and the timing. MSP or MSSP: A provider’s remote management tools and administrative accounts reach every client, so the provider’s own environment is a sound objective for an exercise. When a client commissions one, the provider’s SOC is part of the blue team, and the service agreement should say how exercise activity is confirmed with the client’s control group. The exercise also asks different things of the people involved. Security lead: The security lead usually sits in the control group, owns the rules of engagement and carries the remediation plan afterward. IT manager: The IT manager is commonly in the control group as the person who can judge the operational risk of a planned action and arrange the clean-up. SOC manager: The SOC manager is normally kept outside the control group and receives the operator log at closure to compare with the SOC’s cases. Where Purple Teaming FitsPurple teaming is a collaborative exercise in which red team operators run attacker techniques one at a time, in the open, while the defenders look for each one in their logs and alerts and fix what was missed. A red team exercise measures detection under realistic conditions, and purple teaming is how that detection gets built, so it typically comes first and takes days or weeks where a red team exercise takes months. Our guide to purple teaming covers how an exercise runs, the working record it keeps, and how to plan one with MITRE ATT&CK. Regulator-Led Testing FrameworksFinancial regulators in several jurisdictions run intelligence-led red team programs for the institutions they supervise. Framework Run by How it works CBEST Bank of England, Prudential Regulation Authority and Financial Conduct Authority, United Kingdom Intelligence-led testing of the important business services of firms and financial market infrastructures, on live production systems, by providers the Bank of England accredits. The 2024 implementation guide gives an average project length of 9 to 12 months. CORIE Council of Financial Regulators, Australia Cyber Operational Resilience Intelligence-led Exercises, published in December 2020 and revised in July 2022 after a pilot. Institutions take part by invitation. The red team works in the production network, and the red team exercise is expected to last between four and nine months depending on the institution’s tier. TIBER-EU European Central Bank and national central banks A European framework for threat intelligence-based ethical red teaming, published in May 2018 and updated in February 2025 to align with the threat-led penetration testing standards under the EU’s Digital Operational Resilience Act. The frameworks share a structure that organizations outside financial services can borrow. Threat intelligence first: Scenarios are built from intelligence about adversaries relevant to the institution. Production systems: CBEST and CORIE both test live production environments under a documented risk process. Confined knowledge: CBEST and CORIE keep knowledge of the test within the control group, so the defenders respond as they would to a real intrusion. A learning outcome: CORIE and TIBER-EU both state that the exercise carries no pass or fail grade. Closure with the defenders: CBEST assesses the firm’s detection and response capability, CORIE ends with a purple exercise of 10 to 20 days that it calls a replay adversary attack simulation, and TIBER-EU has required purple teaming since its 2025 update. The CORIE framework is published in full and serves as a practical template for roles, phases and deliverables. Standards That Require Each ExerciseMany security standards require penetration testing, and a smaller group of frameworks and laws asks for a red team exercise. Standard or law Reference What it asks for NIST SP 800-53 Rev. 5 CA-8 and CA-8(2) Penetration testing as the base control, and as an enhancement, red team exercises that simulate attempts by adversaries to compromise organizational systems under applicable rules of engagement. FedRAMP CA-8 and CA-8(2) Penetration testing of cloud services used by US federal agencies, with the red team enhancement applied to the Moderate and High baselines. PCI DSS v4 Requirement 11.4 Internal and external penetration testing of the cardholder data environment. EU Digital Operational Resilience Act Article 26 Threat-led penetration testing at least every 3 years for the financial entities that authorities identify, covering several or all critical or important functions on live production systems. The Digital Operational Resilience Act, Regulation (EU) 2022/2554, has applied since 17 January 2025. It defines threat-led penetration testing as a controlled, intelligence-led red team test of the entity’s critical live production systems. Internal testers are permitted under conditions, and an entity that uses them contracts external testers for every third test. For an organization outside those regimes, a red team exercise is a choice made on its own merits, and contracts and audits typically ask for a penetration test report. Our guide to continuous penetration testing covers what standards expect on testing frequency. AI Red Teaming“AI red teaming” usually means testing AI models and the applications built on them, which is a different subject from the red teaming in this guide. Microsoft notes that the term historically described systematic adversarial attacks for testing security vulnerabilities, and that with large language models it has extended to many kinds of probing, testing and attacking of AI systems, including for harmful output. CISA describes AI red teaming as third-party safety and security evaluation of AI systems, a subset of AI testing, evaluation, verification and validation. AI also appears on both sides of conventional testing, since attackers and authorized testers use the same capability. Our guide to AI penetration testing explains how AI agents carry out tests. Preparing for a Red Team ExercisePreparation decides how much an organization learns from a red team exercise. Objective: The target is tied to a system or process whose loss would matter, and the control group agrees in advance what counts as reaching it. Control group: Membership is small and senior, with the authority to pause the exercise at any hour. Emergency contacts: CORIE expects 24-hour access to the control group and the red team during an exercise, since a real attack may also arrive outside business hours. Log retention: Sign-in, endpoint and network logs are kept for at least the planned length of the exercise. Deconfliction: The control group can check quickly whether suspicious activity belongs to the exercise, and a real intrusion found during the exercise is handled as a real incident. Closure: Time and budget are set aside for the replay and the remediation plan. The Testing Side With ThreatDefenceThreatDefence covers the penetration testing side of this comparison. Its agentic penetration testing runs continuously against an organization’s internet-facing systems, from inside an AI SOC platform that automates monitoring, compliance work and cyber risk management, with a 24x7 SOC behind it. Testing by an agent: The testing agent works through exposed systems in a scope the customer has confirmed and authorized, and it reports a weakness once it holds proof that the weakness can be exploited. Runs through the year: A changed host, a fresh advisory for an exposed product or a reported fix starts a run, and a rolling schedule covers the rest. Agreed limits: Request rates are capped, higher-impact tests wait for approval, and denial of service, data modification and data removal stay excluded. This is testing, and it is separate from a red team exercise. An unannounced adversary simulation, social engineering and physical access sit outside it. An organization that wants a red team engagement arranges one with an independent, human-led team, and the two can run alongside each other. Frequently Asked QuestionsIs red teaming the same as penetration testing?No. A penetration test looks for every exploitable weakness in an agreed set of systems and reports them all. A red team exercise pursues one objective across the organization, quietly, to measure whether the defenders detect and stop it. NIST SP 800-53 treats red team exercises as an extension of penetration testing that examines the organization’s capability to defend itself.Is a red team exercise better than a penetration test?No. They answer different questions, and each is a poor substitute for the other. A red team stops looking for weaknesses once it has a path that works, so its report leaves many vulnerabilities unlisted. A penetration test lists them and says little about detection. Most organizations need regular penetration testing first and a red team exercise once monitoring and response are in place.How long does a red team exercise take?Typically weeks to months. Regulator-led programs sit at the long end: the Bank of England’s CBEST guide gives an average project length of 9 to 12 months including threat intelligence and closure, and Australia’s CORIE framework expects the red team exercise itself to last between four and nine months, depending on the institution’s tier. A scoped exercise that starts from an assumed breach may take a few weeks.Does the blue team know about a red team exercise?No, in most exercises. Knowledge is limited to a control group so that the defenders respond as they would to a real intrusion. CORIE states that blue teams are expected to have no prior knowledge, and CBEST makes the control group responsible for preserving secrecy. The defenders are told at closure, when the timelines are compared.Does a red team exercise include social engineering?Yes, in many exercises. NIST SP 800-53 notes that red team exercises can include technology-based attacks and social engineering-based attacks, and phishing is a common way to gain initial access. The rules of engagement state which methods are permitted, who may be targeted and what is excluded. An exercise can also skip that step and begin from a foothold that the control group grants.What is the difference between a red team and a blue team?The red team plays the adversary and the blue team defends. In an exercise the red team is a group authorized to emulate an attacker, and the blue team is the organization’s own SOC, incident responders and administrators doing their normal work. A control group sits between them, manages risk and decides when the defenders are told.When is an organization ready for a red team exercise?An organization is ready when it has working monitoring and response, has fixed the findings from recent penetration tests, and has run at least some purple team work to confirm that its logging sees common techniques. Without those, the exercise tends to end quickly and confirm gaps that cheaper testing would have found.What is an assumed breach exercise?An assumed breach exercise is a red team engagement that starts from access the control group hands over, such as a standard workstation or user account, in place of the team earning its own way in. It suits a shorter engagement and an organization that already accepts phishing or an exposed service will succeed eventually. The time goes into testing privilege escalation, movement between systems and the defenders’ response.Do any standards require a red team exercise?Yes, a few do. NIST SP 800-53 lists red team exercises as enhancement CA-8(2) of its penetration testing control, and FedRAMP applies that enhancement to its Moderate and High baselines. The EU’s Digital Operational Resilience Act requires threat-led penetration testing at least every 3 years for financial entities that authorities identify. Widely used standards such as PCI DSS set requirements for penetration testing.Is AI red teaming the same as red teaming?No. AI red teaming usually means testing AI models and applications for security flaws and harmful output. CISA describes it as third-party safety and security evaluation of AI systems. Red teaming in the sense used here simulates an adversary attacking an organization. The shared name reflects a shared adversarial mindset, and the skills, targets and outputs differ.