Member engagement is a key measure of an association’s marketing success. One way to track this engagement is through conversions on your website, like email newsletter signups, call-to-action click-throughs, downloads, and article shares. Using your website’s analytics platform, you can monitor each of these conversions to understand whether your site’s content is engaging members.

Reviewing your analytics might reveal that conversion rates aren’t as high as you’d like and need some improvement. A/B testing is a great way to analyze them more closely and figure out how to optimize them so you can maximize member engagement. Here are some A/B testing basics to get you started.

What Is A/B Testing?

A/B testing involves testing changes to your website to see which versions resonate more with your audiences. It compares two variations of content shown to similar audiences at the same time to tell you which one is more effective. By understanding what content works better with your members, you can finetune your content to increase site engagement.

What Is Multivariate Testing?

Multivariate testing is like A/B testing, except you test multiple variables at once. Each variation of an element you want to test must be included in the test to find the combination that generates the best results with your members. For example, testing variations of both the color and copy of an event registration call-to-action.

What’s an Appropriate Test?

There are lots of different ways to conduct A/B tests, but your goal should always be to optimize member conversions.

Your considerations should include:

• Does the test have the potential to lift conversions? (click-throughs, email signups, shares, etc.)
• What type of resources will be needed to implement the test? Is your development team required to make code changes to create the test? Is your creative team required to design the new version of the test?
• What are the elements that you want to test, and how many variations will this require?

These are all important questions to ask when deciding what tests to run. If a particular test will take a lot of resources but the impact is low, then it may not be worthwhile. Prioritize your tests based on the ones that will have the highest impact on your website based on your association’s digital goals.

When considering tests for your association’s website, it’s critical to take a strategic approach and map out a clear plan.

How Do You Know It’s Working?

The most effective way to determine whether your A/B test is successful is to use the hypothesis testing method called statistical significance. Statistical significance is a calculation to determine whether your result is due to random chance or an actual factor of interest in your test. For a result to be statistically significant means that you were able to run a test on a sample of users that accurately represents your entire population of users.

It allows you to be certain that the results of the test will lead to improvements in conversion rates. You want to aim for at least a 95 percent significance level for a variation to win. This will give you confidence that the changes in your association’s website based on A/B and multivariate testing are not due to chance and will actually improve member engagement.

The average A/B test needs to run long enough to reach statistically significant results. The number of variations a test has and the level of traffic to the page tested also affect how long it needs to run. If the page has low traffic (fewer than 1,000 visits per week), it will take a long time to reach statistical significance. Also, the more variations a test has, the longer it needs to run.

Testing Takeaways

When considering tests for your association’s website, it’s critical to take a strategic approach and map out a clear plan. Use data in your current analytics tools to identify areas of opportunity on your website and create a roadmap of tests you would like to run, ordered by highest impact to lowest impact.

Keep in mind seasonality changes on your website and try to pick times to test when your traffic isn’t affected by major campaigns or events. Finally, continuously conduct tests on your website to ensure you’re consistently meeting your members’ needs and generating the highest engagement possible from them.