Our August meetup was an excellent session on A/B testing strategy with Melanie Bowles from InfoTrust. To go along with the theme of A/B testing we also stepped up our door prizes and offered the crowd multiple variants of door prizes: including AirPods and wine from campaign tracking service Claravine.
Melanie did talk a bit about how one might evaluate different testing tools, but she was smartly tool-agnostic in her presentation. While there are many great discussions to be had about the different tools and the math behind them, without a good strategy on items like how to generate testing ideas and prioritize running those tests you could have the best tool in the world and your testing practice could stall out and go nowhere.
A key part of the testing process is consistency and replicability. This requires a good strategy thought about ahead of time! Anyone who has run multiple A/B tests will know that actually making decisions from your test outcomes can be hard. Simply running a test without deciding before-hand what your success conditions might be is very tempting, but it’s rarely the case (especially with a mature product) that the results will speak for themselves in a vacuum — and then what do you do?
Melanie recommended using templates to make sure your process is consistent and sustainable, and was kind enough to provide her slides including some example templates!
Please join us again next month back at Rev1!