A/B Testing a Landing Page With Little Traffic
By CodexierPublished 5 min read
A/B testing sounds like the responsible way to improve a landing page: show two versions, let the numbers decide. On a site with a few hundred visitors and a handful of enquiries a week, the numbers usually cannot decide anything. This guide explains why, and which methods give a small business honest answers in weeks instead of months.
Why small samples mislead
Imagine version A gets six enquiries in two weeks and version B gets nine. That looks like a big win for B, but with numbers that small, the same page could easily produce six one fortnight and nine the next. The test has not detected a difference; it has detected noise. To separate a modest real improvement from noise you typically need hundreds of conversions per version, which many small sites collect in a year, not a month.
The second trap is peeking. If you check results daily and stop when one version leads, you will often stop on a random high point. Testing tools warn about this, but the temptation is strong when traffic is slow.
Big changes instead of small tweaks
The smaller your traffic, the bigger the difference must be to show up at all. Button colours and single words will never produce a visible signal on a small site. Test ideas that change the visitor's decision instead.
| Too small to detect | Big enough to matter |
|---|---|
| Button colour or shape | A different offer: free quote vs fixed price from |
| One word in the headline | A headline built on a different customer problem |
| Image swap | Form replaced with booking directly in a calendar |
| Moving a testimonial | Adding clear prices where there were none |
Before-and-after with care
A before-and-after comparison changes the whole page and compares periods. It is weaker evidence than a true A/B test, because other things change too: season, ad spend, a competitor's campaign, school holidays. Done carefully it is still useful.
- Compare equal periods and avoid holidays, sales and campaign starts
- Keep ad budget, targeting and keywords unchanged during the comparison
- Compare conversion rate, not only the number of enquiries
- Note every other change in a simple log
- Treat the result as a direction, not proof, and keep watching
Qualitative testing with real users
When numbers are too thin, watch people. Five users who fit your customer profile, each trying to complete a task on the page while thinking aloud, will reveal most of the obvious problems: the unclear offer, the form field nobody understands, the price they cannot find.
- Five-user tests: in person or on a video call, one task each, you stay silent
- Session recordings, only with consent and masked form fields
- A one-question survey on the thank-you page: 'What almost stopped you?'
- Listen to sales calls: the questions customers ask are the gaps in your page
Our guide to user testing on a budget shows how to run these sessions without a lab.
When not to test at all
Some changes need no test. If the page is slow on mobile, if the form breaks on some phones, if the phone number is not clickable or the offer is missing, fix it. Testing a known problem wastes the few visitors you have. The same goes for legal fixes such as a missing privacy notice or company details: those are requirements, not experiments.
And be honest about scale. If you have a few enquiries a month, the best investment is usually more qualified traffic, not optimisation. A page cannot convert visitors who never arrive.
What we do on small sites instead
When we build or rework a landing page, we start from what users struggle with, not from a testing tool: a short review of the current page, a few user sessions and a clear hypothesis for the new version. Then we measure the new version against a clean baseline. Prices are on our pricing page.
If you are unsure whether your traffic is enough for testing, book a short call and bring your monthly visitor and enquiry numbers. We will tell you honestly which method fits.
Frequently asked questions
How much traffic do I need for a real A/B test?
It depends on the conversions, not the visitors, and on how big a difference you want to detect. As a rule of thumb, if each version will not reach well over a hundred conversions within a few weeks, a classic A/B test will struggle to give a reliable answer.
Are the 'winner' notices in testing tools reliable on small sites?
Only if you let the test run to its planned length and sample size. Early winners on low traffic are often random. Decide the duration before you start and do not stop early.
Is before-and-after comparison worthless then?
No, it is useful if you control what you can: equal periods, unchanged ads and a log of other changes. Treat it as a signal to keep watching rather than a final verdict.
What should I test first on a small site?
The offer and the headline, because they change the decision. Small visual changes are rarely worth testing until you have much more traffic.
Not sure what to change first?
Bring your page and your numbers to a short call. We will tell you whether testing makes sense or whether a direct fix is the faster route.
Book a free 15-minute call