Building PCI DSS pentest evidence with release-paced testing

The situation

The company takes card payments and is assessed against PCI DSS. Its pentest requirement (requirement 11.4) expects testing at least once every 12 months and after significant changes, and expects exploitable findings to be corrected and the testing repeated to verify the corrections. The team ships often, so a single annual test describes a system that no longer exists by the time the report arrives. Evidence gets gathered in a scramble just before the assessment window, and engineers are pulled off the roadmap to do it.

Scope and rules of engagement

The in-scope set is the payments web application, the API behind it and the cloud environment that hosts them. Scope, and what counts as a “significant change”, are agreed with the company and shared with its assessor up front, because the assessor, not the tester, decides what satisfies the requirement. Testing runs against staging with test accounts and test card data. Production cardholder data and denial-of-service testing are out of scope.

What we tested

Testing follows the release calendar instead of a single yearly window. Each cycle has two parts: a targeted test of what changed (new endpoints, changed authorization rules, changed cloud configuration) and a check of the baseline that must stay true. A tester works by hand, chaining access-control and business-logic flaws the way an attacker would, and every finding goes through an internal QA review before the team sees it.

The finding

To show the shape of the evidence, take one example. In a release that adds a refund endpoint, the tester finds that the API accepts a payment reference belonging to a different merchant and starts a refund against it. The server confirms the caller is authenticated but not that the payment is theirs. That is broken object-level authorization (OWASP API1:2023, CWE-639) with a financial effect. The finding is scored with CVSS v3.1 and ranked against the company’s own risk ranking.

The proof

The record carries the exact request and response, the test accounts used and numbered reproduction steps, so a developer can reproduce it and an assessor can see that it was demonstrated rather than assumed.

Remediation guidance

Check that the authenticated merchant owns the payment before any refund logic runs, in a shared authorization layer rather than per endpoint, and add an automated test that submits another merchant’s reference and expects a denial.

Retest to verified fix

After the fix reaches staging, the tester repeats the original request and its variants. In this scenario the refusal is confirmed, the retest result and its evidence are attached to the original finding, and the finding is closed. That sequence (found, proven, fixed, retested, closed) is the trail an assessor can follow for each issue.

How it shows up in the platform

• Findings: each issue with severity, CVSS, CWE, proof and reproduction steps.
• Remediations: an owner, a status and an SLA for each fix, so open items are visible between tests.
• Retests: the verification result and evidence for every closed finding.
• Reports: the written report for the assessor, with scope, method, risk-ranked findings, proof and retest records in one place.

What to take away

• Evidence that accrues with each release is easier to assemble than evidence reconstructed at the end of the year.
• The assessor decides what satisfies a requirement. Good testing gives them records they can follow and check.
• A fix counts when a retest confirms it.