Conversion Rate Optimisation When You Do Not Have the Traffic to Test
Most sites cannot run a valid A/B test and never will. That is not a reason to stop optimising: it is a reason to trade statistical proof for evidence you can gather, defend and reverse.
On This Page
Your site sees a few hundred sessions a week and a handful of enquiries a month. Someone has proposed an A/B test on the hero. Before anybody builds it, work through what that test would need: enough conversions in each variant that a realistic difference is distinguishable from ordinary weekly noise. At your volume, the run would outlast your pricing, your campaign mix and probably your job title. The test cannot finish. It can only be stopped.
Low traffic CRO is a real discipline, and it is not experimentation with a smaller sample. It is a different method with a different standard of proof, and the first step is admitting that statistical certainty is not on the menu. Once that is said out loud, the work gets easier, because you stop pretending a stopped test is a result and start collecting evidence you can actually defend in a room.
What follows is what we do on sites that will never have testing volume. None of it produces proof. All of it produces reasons, written down, plus the ability to put a change back if it looks wrong. Reversibility is the substitute for certainty, and on most sites it is a better trade than people expect.
Why the test you were sold will never finish
What a test actually needs
A/B tests do not run on visits. They run on conversions, split between variants, and they need enough of them that a genuine difference stands clear of the week-to-week wobble every site has. Two conditions decide how many you need. The size of the effect you are hoping for, and how noisy your baseline is. Small effects on noisy baselines need enormous samples, and almost every change worth making in the real world is a small effect.
Which means the honest question is not "can we test this" but "what is the smallest difference this test could detect, and would we act on anything smaller?" On a low-volume site the answer is usually that the test can only detect a change so large that you would have seen it without measuring.
The run-time trap
Suppose you accept a long run anyway. Over that period the traffic mix changes, a campaign starts, a competitor changes price, a season turns. The variant that wins may simply be the variant that was live when the good enquiries arrived. Worse, somebody will look at the dashboard on a Tuesday, see a lead, and stop. Stopping a test because it currently looks good is the most reliable way to generate a confident answer that is wrong.
A test you cannot finish is not evidence. It is a delay with a dashboard attached.
What replaces the test
Three things stand in for statistical proof, and together they are enough to run a serious conversion programme on a small site. Evidence that a specific problem exists. An argument, written before the change, about why the fix should help. A defined way to notice if it did harm, and a decision made in advance about what would make you revert.
That last part is what separates this from guessing. Guessing ships a change and moves on. This ships a change, records what was expected, watches a small number of things for a fixed window, and either keeps it, reverts it, or admits nothing visible happened. The third outcome is common and perfectly acceptable, provided somebody writes it down.
Heuristic review: the cheapest evidence in the building
A heuristic review is a structured pass over the conversion path by somebody who did not build it. Not a critique of taste. A hunt for defects, each one written as a specific failure with a reason attached. It is the fastest way to find the things costing you enquiries, and on most sites it finds more in an afternoon than a quarter of testing would. Done properly it overlaps with ordinary product design work, because the failures are usually structural rather than cosmetic.
The questions that find most of it
- Can a stranger tell what this is in five seconds? Read the page as someone who arrived from a search result and has never heard of you. Most heroes describe an ambition rather than a service.
- What is the next action, and is it obvious? If there are four equally weighted buttons, there is no next action. There is a menu.
- What does the page ask for, and has it earned that? A demo request at the top of a first visit is asking a stranger for a meeting. Match the ask to what the visitor has been given so far.
- Which questions does the page refuse to answer? Price bands, coverage, timescales, who this is not for. Silence on those does not delay the objection; it moves it off your site.
- What happens when something goes wrong? Submit the form with a bad value, on a phone, on a slow connection. Broken validation is invisible in analytics and fatal in practice.
Session evidence: watching instead of counting
When the numbers are too thin to be statistical, they are still thick enough to be anecdotal, and anecdote is underrated. Twenty session recordings on the pricing page will show you the same hesitation three times, and three times is enough to justify a fix that costs an hour. Form field analytics tell you which question makes people stop typing. On-site search tells you the vocabulary your visitors use instead of yours.
The richest source is usually the one nobody exports: the questions people ask before they buy. Sales calls, support tickets, and the chat log. A lightweight widget like Flidu, which answers from the site’s own content and carries contact and conversion actions in the same place, leaves behind a record of exactly what visitors could not find on the page they were standing on. That log is a content brief with the guesswork removed.
Two cautions. Recordings show you the people who stayed, never the ones who left in the first second, so the worst failures are the least visible. And qualitative work is easy to run as theatre: watching sessions, nodding, changing nothing. We have written about that failure mode in research that could have changed the plan, and it applies here without modification.
Session recording captures behaviour on pages that often contain personal data. Mask input fields by default, exclude payment and account screens, honour consent choices rather than recording first and asking later, and set a retention period you can justify. This is a data protection decision with a legal basis behind it, not a settings tab, and it belongs in the privacy notice before the first session is captured.
Ship the obvious fixes without testing them
Here is the position we will defend: you do not A/B test a defect. If the mobile form submits to a dead endpoint, if the primary button is invisible against its background, if the page takes eight seconds to show anything on a phone, there is no version of the experiment that ends with "leave it broken". Fixing it is not a hypothesis. It is maintenance, and it should ship the day it is found.
The order matters, because the cheap fixes are also the ones with the largest effect on a small site.
- Fix what is broken. Failed submissions, dead links, forms that reject valid input, pages that error on a real device rather than in a simulator.
- Make the offer legible. What it is, who it is for, what happens next. One sentence each, above the fold, in the visitor’s vocabulary rather than the internal one.
- Cut fields nobody acts on. Every field you never read is a cost you charge the visitor for no return. Keep the ones that change how you respond.
- Repair the failure states. Inline errors that say what to do, a confirmation that confirms something specific, and a message when delivery has genuinely failed.
- Fix the accessibility basics. Contrast, labels, visible focus, target sizes, keyboard order. These are conversion fixes for everyone, not a compliance chore for a minority.
- Then speed, on the pages that matter. The landing pages people actually arrive on, measured on a mid-range phone rather than a laptop on office broadband.
Two of those repay reading in more depth. What a page should ask for, and in what order, is the subject of what a landing page should ask for; the accessibility items are set out in accessibility as a default. And the form itself is worth treating as an instrument rather than a formality, which our parent company argues in a form is a triage instrument.
The honest limit of this argument: "obvious" does a lot of work in that sentence. The test for obviousness is whether you can name the defect without using the word "better". If the case for a change is that the new headline feels stronger, that is a preference, and preferences are exactly what you cannot settle without data. Ship the defects. Queue the preferences.
Judging a change when you cannot test it
For changes that are not defects, run a sequential judgement instead of a split test. Write the expectation first: what should move, in which direction, and roughly by when. Change one thing. Log the date in the same place your analytics live, so the annotation is visible next to the chart. Watch a small set of measures for a fixed window, including at least one guardrail you do not want to damage, such as enquiry quality or reply rate.
Then judge it in bands rather than points. A number that sits inside its usual range is unchanged, whatever the percentage on the dashboard says. Only a move that leaves the range for several consecutive periods is worth calling a result, and even then you are describing a correlation you chose to believe. Say so in the write-up. Teams that are honest about the strength of their evidence get trusted with bigger decisions later.
| Method | What it can tell you | What it cannot | Reach for it when |
|---|---|---|---|
| A/B test | Which of two versions performs better, with a stated confidence | Anything, on low volume, in useful time | One page carries enough conversions that a week is a real sample |
| Heuristic review | Where the path breaks and why | How much any single fix is worth | Always, and first |
| Session evidence | What confused a real person on a real page | How common that confusion is | A specific step is losing people and nobody knows why |
| Interviews and task tests | The vocabulary, the objections, the missing information | Anything about scale or preference share | The offer itself is unclear, not the layout |
| Sequential before and after | Whether a shipped change coincided with visible harm | Causation, and small effects of any kind | The change is reversible and the expectation was written first |
How not to fool yourself
Without a control group, the burden of scepticism moves onto you. These are the traps we see most often, roughly in order of how much damage they do.
- Acting after an unusually bad week. Numbers drift back towards their average on their own. Change something after a trough and the recovery will be credited to the change.
- Ignoring where the traffic came from. A redesign that coincides with a new campaign is not a redesign result. Mix explains more variation than layout on most small sites.
- Changing twelve things and claiming one worked. If a release touched the menu, the copy and the form, you have learned that the release helped. Nothing more granular is available.
- Listening to the loudest voice. One articulate customer on a call is a hypothesis, not a mandate. Wait until the same objection arrives from someone who has no relationship with you.
- Comparing across a tracking change. New consent settings, a new tag, a new definition of a conversion. Any of them can move a chart without a single visitor behaving differently.
- Trusting the channel labels. The source that gets credited is frequently the last one, not the persuasive one.
That last trap deserves its own attention, because it quietly corrupts every conversion decision downstream of it. We have unpacked it in why your best performing channel is probably being miscredited, and the short version is that a page optimised for a channel that never really sent those buyers is optimised for a ghost.
When testing becomes worth it again
The threshold is not a number somebody can hand you, because it depends on your conversion rate, your seasonality and how big a difference you would act on. The practical signal is this: one page produces enough conversions that a single week feels like a sample rather than an anecdote, and the change under discussion is one nobody can settle by argument. Until both are true, testing is an expensive way to postpone a decision.
Where the advice reverses: if you run a busy store with thousands of orders a month, most of this is beneath you and you should be testing properly, with a control group, a stopping rule set in advance, and someone who knows what a false positive costs. The methods here are what you use before that, not instead of it.
Run one heuristic review with a person who did not build the site. Watch fifteen recordings of the main conversion path. Fix every defect on the list, in the order above, without testing any of them. Write down what you expect to change and check it once a month. Then, and only then, argue about the headline.
If you have a page you suspect is losing people and no volume to prove it, that is a normal situation rather than a hopeless one. Send us the page and what you already know about who lands on it.
Common questions.
How much traffic do you need before A/B testing makes sense?
There is no universal figure, because the requirement depends on your conversion rate and the size of difference you would act on. A practical signal is whether a single page produces enough conversions that one week feels like a sample rather than an anecdote. Below that, tests take so long to run that seasonality, campaign changes and pricing moves contaminate the result before it arrives.
Is it acceptable to change a page without testing it?
Yes, when the change fixes a defect rather than expressing a preference. A form that fails on mobile, a button nobody can see, or an error message that gives no instruction are faults, and no experiment should end with leaving them in place. Changes you can only justify as feeling better are the ones that need evidence, and those are worth queuing until you have some.
What is a heuristic review in conversion optimisation?
A heuristic review is a structured pass over a conversion path by someone who did not build it, looking for specific failures rather than offering opinions. Each finding names what breaks, for whom, and why it matters. It is the cheapest evidence available on a small site, and it usually surfaces more actionable problems in an afternoon than a quarter of low-volume testing would.
Are session recordings reliable evidence?
They are reliable about individuals and unreliable about proportions. A recording shows exactly what one person did, which is enough to justify a small fix when the same hesitation appears repeatedly. It cannot tell you how common the problem is, and it never shows the visitors who left immediately. Treat recordings as a source of hypotheses and defect reports, not as a measurement instrument.
How do I judge a change without a control group?
Write the expected effect before you ship, change one thing, annotate the date where your analytics live, and watch a small set of measures plus one guardrail for a fixed window. Judge movement against the normal range rather than against a single previous figure. Decide in advance what result would make you revert, and record the outcome even when nothing visible happened.
Does site speed count as conversion work on a small site?
Yes, and it is often the highest-value item on the list because it needs no persuasion to justify. Slow first paint on the pages people actually arrive at costs you visitors who never appear in any funnel report, since they leave before anything is recorded. Measure on a mid-range phone rather than an office connection, and fix the landing pages before the pages nobody enters through.
Facing this in your
own business?
Tell us where you’re headed — we’ll map the shortest honest route.