A/B Testing for Ecommerce: Using Data to Optimise Product Pages

Your product pages attract visitors, but too many leave without buying.

The next change could be clearer delivery information, better product images or an easier way to select a size.

A/B testing helps you establish whether that change improves performance.

The challenge is choosing a useful experiment and collecting evidence you can trust.

The situation

A product detail page has several jobs. It needs to explain the product, answer practical questions and help customers decide whether to buy.

When one of those jobs is handled poorly, shoppers can hesitate even when the product itself meets their needs.

According to Baymard Institute, 62% of the mobile ecommerce sites assessed in its 2026 benchmark had a product-page experience rated “mediocre” or worse. Its research identifies recurring problems with size selection, delivery costs and returns information.

These findings point to potential improvements. They do not predict the conversion increase an individual shop will achieve.

For your own shop, the starting point should be a specific unanswered question.

Are customers struggling to choose the right size? Are delivery charges appearing too late? Does the product gallery leave them uncertain about dimensions?

Combine analytics with customer service enquiries and usability observations. Analytics can reveal where customers leave; direct feedback can help explain why.

Before changing the page, record its eligible visitor numbers, purchase conversion rate and revenue per visitor. Separate mobile and desktop performance where useful, and note promotions or stock shortages that could affect the experiment.

This gives the test a defined problem and a measurable starting point.

What to try first

Begin with the obstacle that has the clearest evidence behind it.

A useful hypothesis connects an observed problem, a proposed change and an expected outcome:

Making delivery information easier to find near the purchase controls will reduce uncertainty and increase the proportion of eligible visitors who complete an order.

This is more useful than “make the page more engaging”. It explains what the experiment is intended to resolve.

Avoid combining unrelated changes simply because they are convenient to release together. If you change the gallery, price presentation, reviews and purchase button at once, the experiment can evaluate the combined design. It cannot establish which individual change caused the result.

A coordinated redesign can still be a valid test. The important distinction is whether you want to evaluate an entire experience or learn about one specific customer problem.

For an initial experiment, a focused change is usually easier to build, check and interpret.

What to build

Build the smallest variation that meaningfully addresses the hypothesis.

Keep the existing page as the control, then create a variation with the proposed improvement. Randomly allocate eligible visitors and use consistent assignment so that returning visitors do not unnecessarily switch experiences.

The following diagram shows a proposed experiment structure, rather than a completed client implementation.

flowchart TD
    A[Eligible product-page visitor] --> B[Random assignment]
    B --> C[Control: existing page]
    B --> D[Variation: proposed improvement]
    C --> E[Record assignment and customer outcomes]
    D --> E
    E --> F[Check allocation and tracking quality]
    F --> G[Evaluate the primary metric and guardrails]
    G --> H[Implement, reject or record as inconclusive]

Choose the implementation method to suit the change. A small copy adjustment may only need an editor. A different product-page layout may require a template test.

Intelligems, for example, offers content edits, redirects, injected styles and JavaScript, theme changes and template changes as separate ways to build a content experiment.

Match the method to your requirements and check the relevant platform documentation before building. Consider how the variation interacts with product options, stock availability and existing integrations.

Choose the primary metric before launch

For a product-page experiment, purchase conversion rate is often a useful primary metric. Define it precisely, including which visitors qualify and how long they have to complete a purchase.

Revenue per visitor may be more appropriate when an experiment could affect basket value as well as purchasing behaviour.

According to Intelligems’ revenue documentation, revenue per visitor is net revenue divided by unique visitors, where net revenue includes product and shipping revenue net of discounts, and refunded or cancelled orders are excluded.

Check these definitions before comparing results across platforms. Two dashboards can disagree because they calculate revenue differently.

Use add-to-basket activity and checkout starts to help explain the customer journey. An increase in these intermediate actions does not, by itself, establish a commercial improvement.

Also define guardrails: measures that should not deteriorate beyond an acceptable level. Depending on the change, these might include page performance, purchase errors, average order value or returns.

Prioritise product-page questions worth testing

Customer question	Possible experiment	What to examine
When will it arrive?	Make delivery information easier to find	Purchases and delivery-related enquiries
Will it fit?	Improve access to size guidance	Purchases and subsequent returns
How big is it?	Add an image showing the product in context	Purchases and size-related enquiries
Which options are available?	Make selection and stock status clearer	Selection errors and purchases
How do I buy it?	Improve purchase-control visibility on mobile	Purchases, accidental actions and usability

These are hypotheses to prioritise using your own evidence.

According to Baymard Institute, shoppers use contextual product images to judge an item’s size. Showing a product beside familiar objects can answer questions that an isolated photograph leaves unresolved.

That makes scale imagery a sensible candidate where customers struggle with dimensions. A furniture shop might investigate whether room-context photographs help customers assess proportions. A beauty retailer might examine whether images communicate the size of a container clearly enough.

The useful experiment depends on the uncertainty customers actually face.

A published example: sticky purchase controls

GrowthRock, a conversion-optimisation agency, has published two tests it ran for an online supplement retailer. The store’s platform, location and traffic were not disclosed.

On desktop, the control kept the purchase area in its normal position on the product page. The variation fixed the purchase area to the side of the screen while the customer scrolled. The test ran for 14 days with roughly 2,000 orders per variation. The agency reports 7.9% more completed orders for the sticky version at 99% statistical significance, and 8.6% more add-to-basket actions.

A separate mobile test compared three versions: no sticky button, a sticky button that scrolled the customer back to the purchase controls, and a sticky button that opened the purchase area as a drawer from the bottom of the screen. The scrolling button showed no statistically significant difference. The drawer produced a reported 5.2% increase in orders at 98% significance. That test also ran for 14 days, with more than 3,000 conversion events per variation.

Two points carry over to your own shop.

First, the behaviour of a control mattered as much as its visibility. The same idea, a button that stays on screen, produced no measurable effect in one form and a measurable effect in another. A hypothesis about purchase controls should specify what the control does, not only where it appears.

Second, the agency itself says the result applies to that store and should be tested rather than copied. The figures are one retailer’s outcome, reported by the team that ran the work. Treat them as a reason to put purchase-control behaviour on your list of hypotheses, not as the uplift to expect.

What can break

A convincing dashboard can still contain unreliable evidence. Build checks into the experiment before launch.

The recorded traffic split is unexpectedly uneven

A planned 50/50 allocation will rarely produce identical visitor counts. However, a sufficiently large discrepancy can indicate an assignment, eligibility or tracking problem.

According to Microsoft Research, an unexpected imbalance between experiment groups can signal data-quality issues. Microsoft’s experimentation platform requires every analysis to pass a sample ratio mismatch check before it reveals the results.

Investigate an unexpected imbalance before declaring a winner. Otherwise, the apparent improvement may reflect differences in which visitors were recorded.

Check whether both variations use the same eligibility rules and whether any redirect or loading failure prevents visitors from being counted consistently.

Tracking behaves differently between variations

A redesigned purchase control might trigger an event twice. A redirect could interrupt measurement. A purchase might be recorded without the visitor’s experiment assignment.

Before launch, check the complete journey through both experiences. Confirm that product options, basket updates and completed orders are recorded consistently.

Write down the expected event behaviour so that the team can distinguish normal activity from a tracking fault.

If measurement breaks during the experiment, document when it happened and which data were affected. Decide whether the remaining evidence is usable before interpreting performance.

A mobile improvement blocks another action

A sticky purchase button may improve visibility while covering size options, support controls or other content.

Test the variation on representative devices and screen sizes. Check scrolling, option selection, basket feedback and keyboard behaviour.

According to Baymard Institute’s mobile usability research, small screens, touch interaction and limited page overview create distinct usability challenges.

A desktop preview alone cannot establish whether the mobile experience works. Check the full purchase journey, including what happens when a customer selects an unavailable option or changes the quantity.

The test is stopped when the result looks favourable

Stopping rules depend on the statistical method.

According to Optimizely’s fixed-horizon testing guidance, results are evaluated after the required sample size and any minimum duration have been met.

For this approach, agree the sample requirement and analysis point in advance. Repeatedly checking an ordinary significance test and stopping when it becomes favourable can increase false positives.

According to research on sequential inference, appropriately designed sequential methods allow repeated evaluation while maintaining statistical validity.

Follow the rules of the method your platform implements. Do not assume that every dashboard supports the same approach to monitoring and stopping.

How to judge the result

There is no universal conversion uplift to expect from a successful product-page testing programme.

An experiment might improve performance, reduce it or leave too much uncertainty to support a decision. Your reporting should accommodate all three outcomes.

Record the test dates, eligible visitors, converting visitors, primary metric, estimated effect and uncertainty. Include relevant guardrails and any incidents that affected interpretation.

Distinguish relative uplift from the percentage-point change. Both describe the result, but they answer different questions and should be labelled clearly.

Statistical evidence also needs commercial context. A small improvement may be valuable on a busy shop, while an expensive implementation may be difficult to justify on a lower-volume page.

Consider the ongoing cost of maintaining the change. A variation that adds complexity to every future product-page update needs a stronger business case than a straightforward content adjustment.

If the result is inconclusive, describe what remains uncertain. A broad interval can leave room for both meaningful benefit and harm. An interval concentrated around negligible effects may support a different decision.

Avoid extending a test indefinitely because the preferred variation has not won. Review whether the experiment can answer the question within a useful timeframe.

Keep a record of unsuccessful tests too. Knowing that a proposed change did not produce convincing evidence can prevent the same idea being rebuilt later.

When not to run this test

A/B testing is less useful when the available traffic cannot support the decision you need to make.

According to research on A/B test sample-size estimation, sample planning is essential for detecting meaningful effects while controlling statistical errors.

There is no dependable universal cut-off based on monthly orders alone. Sample requirements depend on baseline performance, the effect you want to detect, statistical power and the chosen analysis method. Smaller effects generally require more evidence.

For a shop with limited eligible traffic, customer interviews, usability testing and support-ticket analysis may provide more useful direction.

Also consider postponing the experiment when:

• Purchase tracking is unreliable.
• Important products are repeatedly out of stock.
• Another overlapping experiment makes interpretation difficult.
• The team cannot implement or maintain a successful variation.

Clear functional defects deserve particular attention. A broken size selector or failed purchase button should normally be repaired directly.

Testing is most useful when there is genuine uncertainty between workable alternatives.

If you are doing this yourself

Use this checklist before launch:

1. Define one decision. State what the experiment will help you decide and why it matters.
2. Document the evidence. Connect the hypothesis to observed customer behaviour or feedback.
3. Agree the measurement plan. Define the primary metric, guardrails, eligibility and attribution window.
4. Plan the analysis. Establish sample requirements and stopping rules appropriate to your method.
5. Check both experiences. Verify product selection, basket behaviour, purchases and mobile usability.
6. Monitor data quality. Investigate tracking failures and unexpected allocation imbalances promptly.
7. Record the decision. Keep the result, uncertainty, limitations and implementation outcome together.

Start with a product-page problem that customers demonstrably encounter. Build a focused variation, measure the complete purchase journey and decide using evidence that survives basic quality checks. That gives the next development decision a stronger foundation than a promising headline percentage.

Related reading

• Keep experiment results visible alongside the rest of the shop’s numbers with automated reporting.
• For the funnel beyond the product page, see improving ecommerce conversion with data automation.
• Page speed affects both variations and your guardrails, so fixing a slow store can come before testing.
• Store experiment records in reliable reporting pipelines so past results stay comparable.

Need help setting up product-page experiments for your store? Get in touch or explore our ecommerce optimisation services.