Free Shipping Mention Test

Let’s say you work at an eCommerce startup. You run an A/B test on the company’s checkout product page with the hypothesis that surfacing free shipping will increase conversions.

The experiment group specifies whether the product qualifies for free shipping or not. In the control group, there is no specification of free shipping.

Let’s say we run the experiment for two weeks and get these results.

Control:

• 3056 page visits
• 1413 conversions
• 46% conversion

Experiment:

• 2947 samples
• 1533 conversions
• 52% conversion

How would you evaluate the results and if the test was successful or not?