Why Red Teaming? Methodologies and Comparative Analysis Within the Security Assessment Spectrum

Why Red Teaming?
Methodologies and Comparative Analysis Within the Security Assessment Spectrum

Author: Technology Consulting Group　Edoardo Tuni
Supervisor: Technology Consulting Group　Kazuyuki Nakamura

Red Teaming and its Neighbors: Vulnerability Assessment and Penetration Testing

First and foremost, what is Red Teaming? According to NIST:

(Red Team Exercise is) An exercise, reflecting real-world conditions, that is conducted as a simulated adversarial attempt to compromise organizational missions and/or business processes to provide a comprehensive assessment of the security capability of the information system and organization. [1]

In a cybersecurity context, a Red Team is a group of ethical hackers who simulates real-word attacks to identify and address security weaknesses in an organization. Now let's compare Red Teaming with Vulnerability Assessment and Penetration Testing.
NIST defines Vulnerability Assessment as follows:

A systematic examination of an information system or product to determine the adequacy of security measures, identify security deficiencies, provide data from which to predict the effectiveness of proposed security measures, and confirm the adequacy of such measures after implementation. [2]

In simple terms, a VA (Vulnerability Assessment) is a systematic approach that uses automated tools to detect weaknesses and vulnerabilities of the defined target. While this process might involve sending specially crafted network requests or performing local checks on the target, there is no real exploitation of the vulnerabilities discovered. In a Red Teaming engagement, detecting vulnerabilities is only the first step to a full compromise of the system. Most commonly, a VA will take advantage of scanning tools like Nessus (Tenable's proprietary vulnerability scanner) or OpenVAS (an open source scanner), and the main goal will be to reduce the company's attack surface by applying patches and software updates, based on the scanner's findings.

Picture 1: Tenable Plugin no.186364 for Apache Tomcat's missing updates. As shown in the highlighted text, vulnerability scans typically only check the versions of installed software and cross-reference them with online vulnerability databases, without performing active tests to verify the exploitability of the identified vulnerabilities. [3]

Let's explore Penetration Testing next. NIST defines Penetration Testing as:

Security testing in which evaluators mimic real-world attacks in an attempt to identify ways to circumvent the security features of an application, system, or network. Penetration testing often involves issuing real attacks on real systems and data, using the same tools and techniques used by actual attackers. Most penetration tests involve looking for combinations of vulnerabilities on a single system or multiple systems that can be used to gain more access than could be achieved through a single vulnerability. [4]

By looking at the definitions alone it is less clear if and how Penetration Testing and Red Teaming are any different. They are often confused because both use real-world attacks to compromise the system and provide a security assessment. The key difference is that a Red Team engagement is a simulated adversarial attempt, as it mimics APTs (Advanced Persistent Threat) groups and TTPs (Tactics, techniques and Procedures), including tactics like phishing and lateral movement.

Picture 2: Spear-phishing email used in the attack campaign by APT28. This kind of phishing attack might be emulated by Red Teams to test organizational defenses against social engineering. [5]

A Penetration Test will aim at providing a full picture of all the security flaws discovered, but it may target only a single web application. On the other hand, Red Team exercises are typically more targeted, attacking sensitive data or systems with the goal of gaining access or compromising the system through social engineering, lateral movement, and data exfiltration, mimicking typical TTPs. 
Furthermore, Red Team exercises will also focus on the overall response of the organization, testing not only the technical defenses but also their incident response methods and detection capabilities. By doing so, they provide a good chance to train security defenders and validate security designs and practices. [6]
A Penetration Test, usually, will also act within the limits of the scope described in the Rules of Engagement (RoE), which define specific IP port ranges, ports, and testing schedules.

Despite the crucial differences between Red Team Assessments and Penetration Tests, the overall methodology remains similar in structure, as shown by the key phases of a Penetration Test and a Red Team assessment outlined in the diagram below. However, in Red Teaming, techniques like phishing (Information Gathering) or vulnerability exploitation (Exploitation) are not just used to uncover weaknesses but to test detection and response capabilities.

In summary, while a Penetration Test certainly has the objective advantage over Vulnerability Assessments of measuring the organizational impact caused by the actual exploitation of vulnerabilities, it falls short of several benefits of Red Teaming, such as:

• Ignoring attack vectors typical of real adversaries like social engineering and physical intrusions, helpful in assessing the security awareness of the employees
• Assessing the capacity of the company defense mechanism to detect and react promptly to a stealthy, offensive simulation
• Exposing and training defenders to real-world attack scenarios

The relationship between Vulnerability Assessment, Penetration Testing and Red Teaming is illustrated in the diagram below. [7] Vulnerability Assessments provide the widest breadth because of their extensive coverage, scanning multiple systems and networks to discover weaknesses. However, they stop at a superficial level because they do not perform thorough tests. In contrast, Penetration Testing and Red Teaming dive deeper by actively exploiting identified weaknesses, with Red Teaming offering the most comprehensive approach through real-world adversarial simulations.

Red Teaming: Methodology, Roles and Responsibilities

In practice, Red Teams emulate real-world attackers, such as APTs, using frameworks like MITRE ATT&CK. 
MITRE ATT&CK® is a globally-accessible knowledge base of adversary tactics and techniques based on real-world observations. The ATT&CK knowledge base is used as a foundation for the development of specific threat models and methodologies in the private sector, in government, and in the cybersecurity product and service community. [8]

For example, to simulate APT28, a Red team might start with spear-phishing (Initial Access), escalate privileges through Token Impersonation/Theft (Credential Access), and move laterally through SMB protocol, as highlighted by MITRE ATT&CK Navigator. [9]

Picture 3: The MITRE ATT&CK Framework organizes adversarial behavior into tactics, such as Initial Access or Privilege Escalation, each further divided into techniques and sub-techniques that guide Red Team simulations.

Next, let's explore the internal structure of a Red Team. A Red Team will typically be composed of the following roles.

Table 1. Roles and Responsibilities
Source: Based on " Red Team Development and Operations: A practical guide" by Joe Vest and James Tubberville.
Independently published (First edition published on January 20, 2020)

The following diagram shows the relationship and communication paths among the different groups during the engagement. [10]

Summary

In this article, we have examined and compared Red Teaming to other common security assessment methods, highlighting the differences between each of them. We have also explored the typical roles and responsibilities within a Red Team engagement. In light of this, we are now able to answer the questions raised at the beginning: what are the pecularieties of Red Teaming, and why is it necessary?

Red Teaming stands out because of its adversarial approach. Unlike other assessment methods, it not only identifies vulnerabilities but also evaluates the organization's ability to detect and respond to real-world attack scenarios. By testing beyond technical boundaries through realistic threat simulations such as people and processes, Red Teaming ensures a comprehensive understanding of security posture. Therefore, Red Teaming is particularly recommended when organizations seek a realistic and comprehensive security assessment that include their detection and response capabilities, and it becomes crucial when they aim to identify weaknesses and enhance their security posture based on the findings.

【プロフィール】

文責:トゥーニ　エドアルド（Tuni Edoardo）
2023年にデジタルアーツコンサルティング株式会社（DAC）
※2024/4よりアイディルートコンサルティング株式会社（IDR-C）へ社名変更

ネットワーク監視およびインフラ運用の経験を経て、現在は脆弱性管理に特化した業務に従事しています。ペネトレーションテストやレッド・チームの活動を目指し、日々取り組んでいます。

監修: 中村　和之（なかむら かずゆき）
2023年にデジタルアーツコンサルティング株式会社（DAC）
※2024/4よりアイディルートコンサルティング株式会社（IDR-C）へ社名変更

ヒューレットパッカード、パロアルトネットワークスを経て、ITインフラ・サイバーセキュリティの分野においてコンサルタントとして活動。最近では、多岐に渡るセキュリティ製品を顧客要件に応じて適切に組み合わせる知見作りにフォーカスした活動に注力しています。

【Notes and Bibliography】

[1] National Institute of Standards and Technology (NIST), "Red Team Exercise," Computer Security Resource Center (CSRC), accessed December 25, 2024.
https://csrc.nist.gov/glossary/term/red_team_exercise

[2] National Institute of Standards and Technology (NIST), "vulnerability analysis," Computer Security Resource Center (CSRC), accessed December 25, 2024.
https://csrc.nist.gov/glossary/term/vulnerability_analysis

[3] "Apache Tomcat 8.5.0 < 8.5.96", Tenable, Accessed December 25, 2024
https://www.tenable.com/plugins/nessus/186364

[4] National Institute of Standards and Technology (NIST), "penetration testing," Computer Security Resource Center (CSRC), accessed December 25, 2024.
https://csrc.nist.gov/glossary/term/penetration_testing

[5] Bryan Lee, Mike Harbison and Robert Falcone.
 "Sofacy Attacks Multiple Government Entities." Palo Alto Networks Unit 42, February 28, 2018,
https://unit42.paloaltonetworks.com/unit42-sofacy-attacks-multiple-government-entities/

[6] Mike Chapple, and David Seidl, CompTIA PenTest+ Study Guide: Exam PT0-001 (Sybex Study Guide), 1^st ed. Sybex, 2018

[7] James Tubberville, and Joe Vest, Red Team Development and Operations: A practical guide, 2020 edition

[8] MITRE. MITRE ATT&CK® Framework, accessed December 25, 2024
https://attack.mitre.org/

[9] MITRE. MITRE ATT&CK Navigator, accessed December 25, 2024,
https://mitre-attack.github.io/attack-navigator/

[10] James Tubberville, and Joe Vest, Red Team Development and Operations: A practical guide, 2020 edition