Conversion rate optimisation (CRO) is a journey filled with potential pitfalls, from hypothesis to test execution. In this webinar (hosted by Convert.com and moderated by Deborah O’Malley of GuessTheTest), Dave Gowans, founder and CEO of Browser to Buyer, explains how to avoid these pitfalls.
Dave drew on his 14 years experience in CRO, sharing actionable insights on transitioning seamlessly from hypothesis to test execution. His focus was on the challenge of taking a “supposed winning hypothesis” and making sure that it turns into a valid and successful test, which he describes as “walking over a slippery floor carrying a priceless piece of pottery”.
To ensure your winning hypotheses deliver great results, Dave recommends you:
In this clip, Dave lays out how to challenge the design and concept at every step to identify flaws and areas for improvement:
TIP: Challenge yourself at every step: What does the hypothesis look like on the site? How can we improve the design? Perfect the copy. Iterate: Write-Rewrite-Get feedback to ensure the best execution before moving forward.
Usability mistakes can derail test execution. Many tests fail because they overlook the actual user experience on real devices and real-world variables, such as different device screen sizes and user behaviours, are crucial. Here Dave shares an example of a site where key elements were pushed below the fold on smaller screens, underscoring the need to design for real-world conditions.
Tip: Ensure key elements are visible across different screen sizes to avoid missing crucial information and view your designs on real devices.
Thorough QA testing on actual devices—not just browser emulators—is vital. This practice helps catch issues that might not be evident in design files or on emulators. In the clip below, Dave highlights the importance of considering various screen sizes and device types, ensuring the test performs well in real-world conditions. Real devices (particularly mobile) show up issues which aren’t visible in browsers and emulators, such as strange scrolling behaviours (maps are particularly bad at this) or issues when the user’s keyboard is open.
Isolating changes to understand their specific impacts, rather than making multiple changes at once, is essential to avoid inconclusive results. It’s important to remember that everything you add to a page (for example) means removing or moving something else, which could have a negative impact. It’s all too easy to start ‘improving’ the design of a page or element being tested, which can have just as much effect as the change you actually intended to make. It’s easily possible to ‘miss’ a winner, by making another negative change at the same time.
Advice: Focus on one change at a time to accurately measure its effect.
Getting from a strong test hypothesis to a winning test is surprisingly difficult. There are opportunities to fail at every step and any mistakes can stop a good hypothesis from winning. By being critical of your test at every stage, testing and retesting it before getting too far through the process, and taking an approach of trying to find the problems as early as possible, you can significantly increase the effectiveness of your testing.
If you’d like to learn more about how to create a strong hypothesis and take it through to a successful test, below is a summary of the whole webinar or you can view the replay of the full (1.5 hour) session here.
Focusing on going from hypothesis to a Live Test, In a Way That Matters to Your Audience, the panel discussions spanned localizing test parameters, e-commerce best practices, and AI’s role in testing particularly covering:
The importance of Localization: localized testing for different markets, including checkout experiences and payment methods, is crucial for accurate results and better user engagement.
Iterative Design and Thorough QA: Dave highlighted the need for refining execution through iterative design processes and thorough QA testing on actual devices to ensure real-world applicability and avoid usability mistakes.
Rigorous Testing Protocols: The “ALARM” protocol for mitigating risks and ensuring robust test designs plus methods for using AI for predicting test outcomes and improving prioritization.
Critical Approach to Hypotheses: The importance of a critical approach to hypotheses, questioning and validating each step to design successful tests and avoid inconclusive results.
Balancing Templates and Customization: The balance between using templates and customization in e-commerce, stressing the importance of testing over blindly copying competitors and leveraging customer research for informed decisions.