If you like this presentation – show it...
Growing Your Business Through Experimentation Hiten Shah email@example.com
What do you do? Product, Design, Engineering, Marketing, Growth, Sales
How do you make all the things better?! Figuring this out is your job.
Only 1 out of 5 tests win. 4 out of 5 tests cost you money.
So, why experiment if so many tests fail?
Go Where Customers Are! Long, slow, SaaS ramp of death from 2006…
Go Where Customers Are! Now…
Go Where Customers Are! Growth can be sped up!
Increase Hello Bar’s installation rate
Improve the KISSmetrics features page
Increase signups from Crazy Egg’s homepage
Grow Crazy Egg’s revenue
Results from 3 months of focused growth effort
Results: KISSmetrics signups over time
Simple 4-step experimentation process Data Figure out what to improve Hypothesis Make an educated guess Experiment Test your guess Act Decide what’s next
Data Figure out what to improve
Data doesn’t make decisions for you.
Data informs your decision making.
Data helps you identify the most painful problems people have. (not the ones that bother you the most)
Find problems worth solving by starting with quantitative data.
When in doubt, look at your top pages. Usually the homepage.
A simple framework for choosing what to improve
Find pages with high traffic and high bounce rate HTTP://GOOGLE.COM/ANALYTICS
Find pages with high traffic and low conversion rate HTTP://UNBOUNCE.COM
Find sources with high traffic and low conversion rate HTTP://HUBSPOT.COM
Find opportunities based on user paths HTTP://KISS.LY/KMPATH
How To Figure Out What To Improve High Traffic Increase Conversions Do a Happy Dance By A/B Testing But Watch it Closely Low Conversions High Conversions Get More Traffic Get More Traffic So You Can Start A/B Testing So You Can Do a Happy Dance Low Traffic
Where do you have the most visitors, users, customers, emails, or notifications? More volume helps you test faster.
Common areas with large volume and high impact potential ๏ Website Traffic / App Installs ๏ Welcome Emails ๏ Web / Mobile Onboarding ๏ Email Digests ๏ Triggered Notifications ๏ Cart / Funnel Abandonment ๏ Dormant / Churned users
When you’ve already got usage, analyze it. What percentage of your customers or users have adopted each feature? Intercom on Product Management
Find out why by getting qualitative data.
A few qualitative tools HTTP://USERTESTING.COM HTTP://SURVEYMONKEY.COM
A few more qualitative tools HTTP://QUALAROO.COM HTTP://OLARK.COM
Use exit surveys to learn why people bounce HTTP://QUALAROO.COM
Use post-conversion surveys to understand why HTTP://QUALAROO.COM
Discover if your web pages are effective at their goal HTTP://QUALAROO.COM
What frustrates people? EMAIL
Find people who don’t do things and email them EMAIL + SURVEYMONKEY
User Research gives you a headstart
Hypothesis Make an educated guess
Ideas Designs Solutions } Assumptions (A guess about the best way to solve a problem)
Everything can start with a testable hypothesis.
Before you create anything, make an educated guess.
Use simple equations to assess potential impact of each idea visits x conversion rate x order value x li4 = ? ESTIMATED MODELED users x order value x resurrec7on rate = ? ESTIMATED MODELED DAN MCKINLEY HTTP://KISS.LY/DANETSY
Prioritize your tests Assess the potential impact, difficulty and priority of each test idea you have.
Experiment Test your guess
Test your hypothesis by starting an experiment.
Not button colors…
Don’t copy “best practices” blindly
Test specific button copy
Run sensitivity tests on your web pages Sensitivity tests help you learn what your visitors care about and what they don’t.
Remove graphical elements and see what happens
Does social proof and logos matter for your visitors?
Remove social proof and learn what happens
Act Decide what’s next
You’ve probably seen an A/B testing report before
Document every A/B test and create a playbook 1. Your starting hypothesis. 2. Dates for when you ran the test. 3. Screenshots of control and all variations used during the test. 4. Expected change in conversion. 5. Probability of change. 6. Screenshot of the raw data used in your analysis. 7. A screenshot to the A/B Test Report in KISSmetrics. 8. The decision you've made from the test. 9. What you learned from the test.
What learnings do we want to carry over from the past? Helps you avoid unknowingly decreasing your conversions.
Create experiment checklists based on learnings ✓ Keep the footer links ✓ Autofocus on the first form field ✓ Do not include social proof ✓ Link to the features page ✓ Use “Log in” copy on the button
☀ If you’re not testing, you’re not learning.
⌾ Always have tests running, no matter how small.
∞ If you don't have a process, you won't continuously improve.
Work on things that matter. Make those things better. Be happier ☺
Thank Y0u =) Got questions? Need help? Contact me any time! Hiten Shah firstname.lastname@example.org