A/B testing

What can you A/B test

The A/B testing feature allows you to test different content/style of specific elements on your site in order to improve conversion rates. Currently, you can A/B test:

• Text of elements (eg. BUY NOW vs GET STARTED)

• Style of elements (eg. change colors, make them bigger, hide them, etc.).

How can you A/B test

userTrack allows you to quickly create tests and make changes for each variant in a test. To add a new change you can use the visual editor, where you click an element on your site in order to get its selector and then input the new value for that element. You can immediately preview all the changes in a specific variant, and can also quickly switch between variants to compare them.

The structure of a test

• TEST 1

  • Default

  • Variant 1

    • Change X

    • Change Y

  • Variant 2

    • Change Z

• TEST 2

  • Default

  • Variant 1

    • Change W

You can have multiple tests running at the same time, but statistically it is better to only test one thing at a time in order to make sure that the result of one test is not influenced by another.

Each test has multiple variants, one of them always being the default variant (current version of the site, with nothing changed). For example, you can have a test named "hero text", in which you want to find the best copy for the hero text on your landing page. In this test you could have 3 variants:

• default variant - hero text is the description of your product

• benefits variant - hero text is changed to showcase benefits of your product instead of its description

• colored variant - same text as default, but text color is different

Each of those variant has a list of changes, for the example above, those changes would be:

• default

  • no changes

• benefits

  • Change: Replace h1 text on homepage with the given text

• colored

  • Change: Change style of h1 text on homepage with the given style

Each test can have unlimited variants and each variant can consist of unlimited changes.

It is usually recommended to keep the number of variants low (2 or 3) in order to have a big enough sample size (number of users) that see each variant.

Results of an A/B test

After an A/B test is created and published, your users will see one of the variants created. When a user sees a specific variant, it will be tagged with the variant that he saw. This allows you to use segments in order to view stats for users who saw a specific variant. One way to compare results of an A/B test is to create a segment for each variant (i.e. each tag created for those variants).

Results UI:

Since version 8.5.0, results for A/B tests are automatically calculated and shown on the A/B testing page. Those results are based on your existing tests, variants, goals and currently selected segments.

For example, for the test "pricing-h3" with two variants (Default and Variant 1), with the goal "Pricing" and segments "All" and "Google" enabled: