Should You Run a Penetration Test or Fix the Obvious First?

A quote arrives for a two-week penetration test of your website and its admin. Before you sign it, answer one question honestly: if a tester logged in tomorrow morning, how many of the findings would be things somebody on your team already knows about? If the answer is more than two or three, you are about to pay expert day rates for a list you could have written yourself.

That is the argument in one paragraph. A penetration test is a measurement instrument, not a remediation service. It earns its price when it tells you something nobody on your side could have found, and it wastes most of that price when its first two days go on confirming that a plugin nobody updated in two years is still installed and still reachable from the internet.

So there is a bar. Below it, testing is theatre with an invoice attached. Above it, testing is one of the few ways to learn something about your own system that no tool will ever tell you. This piece sets the bar, says plainly what a test covers and what it never covers, and argues that the remediation budget should exist before the testing budget does. If you have not yet looked at what an automated scanner finds on your site today, start there and come back with the output.

What a penetration test actually buys

You are not buying a tool run. Tools are cheap, several good ones cost nothing, and any of them can be scheduled by someone on your side. You are buying a person’s judgement: the ability to look at three findings that are individually dull and notice that together they let an unauthenticated visitor read another customer’s order. Scanners match signatures. Testers reason about your application, its roles, its assumptions, and the distance between what the documentation claims and what the code does.

The categories in the OWASP Top 10 are a reasonable map of what goes wrong, but a map is not a survey of your own terrain. Broken access control is the entry everybody can name and the one that hides in the dullest places: an invoice endpoint that trusts a numeric identifier, an export route that forgot the tenant check, an admin action protected only by the absence of a link in the menu.

The part only a human finds

The findings that justify the fee usually look like this.

• Authorisation between two real accounts. Two logged-in users, one changed identifier, and a record that should have been invisible. No scanner holds your account model, so no scanner tests this.
• A workflow that runs out of order. Skip the second step of a checkout, or replay the third, and see what the server accepted anyway.
• A chain of small things. An open redirect, a token carried in a URL and a permissive referrer policy are three low findings until somebody makes them one high one.
• Trust placed in the browser. Prices, roles, discounts or feature flags decided on the client and believed by the server without a second check.
• Reset and recovery paths. Password reset, invitation acceptance and email change are where authentication is most often quietly undone, and where testers look early.

Penetration testing readiness: the bar to clear first

Here is the bar. It is not a maturity model and it is not a certification. It is the set of things a competent tester finds in the first afternoon, which means every one of them is something you would be paying premium rates to be told.

1. Every component sits on a version you can name. Platform, plugins, themes, packages, runtime, database. Not “roughly current”. A version number, written down, that somebody checked this month.
2. Your dependencies have been inventoried. The code you shipped but did not write is the larger half of most applications, and a published advisory sitting in it is not a penetration test finding. It is a patching failure with a public identifier attached.
3. The admin surface is enumerated and current. Every account that can change something, every integration key, every legacy login nobody has thought about since the migration. Leavers removed. Multi-factor authentication on anything that can publish or take money.
4. Transport and headers meet a sane baseline. The security headers that actually stop something take an afternoon to get right. A report telling you strict transport security is missing is an expensive way to learn it.
5. A backup has been restored, not merely configured. A backup nobody has ever restored is a belief rather than a control, and the day you find out is never a good day to find out.
6. You can answer who did what. Application logs, administrative action logs, and access logs that survive the incident which made you want to read them.
7. Somebody owns the fixes and has hours in their week. A named person, with time reserved, before the report lands. Reports do not remediate themselves and they do not wait politely for a quiet quarter.

The bar is not perfection

Nobody clears all seven cleanly, and waiting until you do is its own kind of failure. The bar is about known problems: things with a public identifier, a documented fix, or an obvious owner. Unknown problems are exactly what you are hiring a tester to surface. If you have three unglamorous items outstanding and a plan with dates against them, book the test. If you have thirty and no plan, the test is not the intervention you need this quarter.

Where the money goes: scan, review, test

Six activities get filed under “security work” and they are not substitutes for one another. Confusing them is how a team ends up holding an expensive report and an unpatched server at the same time.

The first three belong to ongoing maintenance rather than to a project. They are cheap, repeatable and boring, which is precisely why they get skipped and precisely why a tester finds their leftovers. Paying a specialist to run a scanner you could have scheduled is the most common way a security budget evaporates without improving anything.

What a test does not cover

A penetration test is time-boxed, scope-boxed and dated. All three matter, and all three are routinely misread.

Time-boxed means the tester stops when the days run out, not when the application runs out of flaws. A clean report says “nothing found, in this scope, in this window, by this person”. It does not say the application is secure, and a decent testing firm writes that sentence on the first page themselves.

Scope-boxed means the boundary was agreed in advance, in writing, and it is almost always narrower than the business assumes. The marketing site is in scope; the third-party booking widget embedded halfway down it is not. Production is in scope; the forgotten staging environment holding a copy of the real customer database is not, because nobody mentioned that it existed.

Scope is the document that matters

Read the scope like a contract, because it is one. Ask what is excluded and why. Denial-of-service testing is almost always out. Social engineering and phishing are separate engagements with separate consent. Physical access is a different discipline entirely. And detection is usually not tested at all: testers are commonly allowlisted so their traffic does not trip the controls you would most want exercised, which is a fair trade for coverage but leaves did anyone notice completely unanswered. That question belongs to the work you do before the day you are compromised, and it is a separate exercise with a separate budget.

Budget the remediation before the test

Here is the position, stated flatly: if you can only fund one of the two, fund the fixing. A test with no remediation budget produces a document that makes your position worse, because you now hold written evidence of problems you chose not to address. Insurers, auditors and enterprise customers all read that differently from “we did not know”.

So plan the money in the other order. Decide what you are willing to spend on fixing, reserve at least as much again for remediation as the engagement itself costs, and only then scope the test to what that reserve can absorb. A narrow test you can act on completely beats a wide test that produces a backlog nobody has funded. The wide one feels thorough in the kick-off meeting and looks negligent in the incident review.

The practical shape of this is a findings pipeline rather than a document. Every finding carries a severity, a priority that reflects your context rather than a tool’s default, an owner, a decision and a verification step. Tooling exists for exactly that translation, and Prooflin takes that approach by resolving findings, severities, priorities and recommendations into reviewable reports. A maintained spreadsheet works too. What never works is a long attachment sitting unopened in a shared drive.

Our parent company has written about how to read an audit you were sold, and the same reading applies to a test report. The useful question is never how many findings there are. It is how many of them change what somebody does on Monday.

Reading the report you paid for

Findings are not tasks. A finding describes a condition; a task describes a change to your system, in a specific file or setting, made by a named person, confirmed by something you can re-run. The translation between the two is where most of the value of a test is either captured or thrown away, and it happens in the two weeks after the engagement ends, when everyone has moved on to something else.

It is worth reading the anatomy of a finding before the report arrives rather than after. In short: a finding you can act on carries the evidence, the affected component, the condition that makes it exploitable, and a fix expressed as a change rather than an aspiration. “Improve input validation” is not a fix. “Reject this parameter unless it matches this pattern, in this handler” is.

Severity is what a finding is worth in general. Priority is what it is worth on your system, this quarter, with your people. A report that only gives you the first has done half the job.

The retest clause

Negotiate the retest before the first test, not after it. A fixed-scope retest of the findings you remediated, inside an agreed window, at an agreed price, is the difference between a report and an outcome. Without it, fixes go in unverified and the next engagement rediscovers half of them, usually because the patch closed one code path and left a second one open.

Settle who does the fixing before the report exists, too. If the team that built the application is remediating, the work lands inside ordinary application development and can be scheduled like anything else. If a maintenance provider is remediating, check what a maintenance retainer actually buys, because “security updates” in a contract sometimes means core updates only, and a finding in a custom theme is nobody’s job by default.

When to book the test anyway

Everything above argues for sequencing. There are cases where we would book the test before the bar is clear, and it is only fair to name them.

A contractual deadline is the most common. When a customer, an insurer or a certification body requires a test by a date, the test happens on that date and the readiness argument turns into a scoping argument instead: test the narrow thing the requirement actually names, keep the rest of the money for fixing, and do not let the exercise expand to fill the calendar.

A system nobody understands is the second. When an application was inherited, is undocumented, and no current employee can say what its admin surface even contains, a tester’s reconnaissance is genuinely the cheapest way to find out what exists. In that case you are buying a map rather than an assessment, and the brief should say so in plain words.

The third is a new build with a real threat model, where the obvious has not had time to accumulate. Testing a system while changing it is still cheap is the best money in this article. That is a different transaction from testing a long-neglected site that has never been patched, which is buying an expensive description of a problem you could have described for free.

Where we would start

Assume the week above has happened and the easy findings are gone. Now scope the test around what is left: the parts of the system where money moves, where personal data sits, and where one account could plausibly reach another account’s records. Those three places are where a person’s judgement is worth paying for. The rest of your attack surface is a maintenance problem wearing a security costume.

Then give the tester what they need to be useful. Credentials for every role, not just an anonymous view. A staging environment that genuinely resembles production, or written permission to work against production inside agreed hours. A named contact who can answer a question the same day. Testers spend a surprising share of a short engagement waiting for access, and every hour spent waiting is an hour not spent on your authorisation model.

And if the answer to “what would a tester find first” is honestly “I do not know”, that is itself the finding. It tells you the standing controls are not standing, and no engagement will fix that for you. Fix it, then test, then retest. If you want a second read on a scope document, or on a report somebody has already handed you, tell us what you are running and what the test is meant to prove.