Понравилась презентация – покажи это...
Слайд 0
Growing Your Business Through
Experimentation
Hiten Shah
hnshah@gmail.com
Слайд 1
What do you do?
Product, Design, Engineering, Marketing, Growth, Sales
Слайд 2
Слайд 3
How do you make
all the things
better?!
Figuring this out is your job.
Слайд 4
A/b testing
Слайд 5
Only 1 out of 5 tests win.
4 out of 5 tests cost
you money.
Слайд 6
So, why experiment if so many tests fail?
Слайд 7
Go Where Customers Are!
Long, slow, SaaS ramp of death from 2006…
Слайд 8
Go Where Customers Are!
Now…
Слайд 9
Go Where Customers Are!
Growth can be sped up!
Слайд 10
Increase Hello Bar’s installation rate
Слайд 11
Improve the KISSmetrics features page
Слайд 12
Increase signups from Crazy Egg’s homepage
Слайд 13
Grow Crazy Egg’s revenue
Слайд 14
Results from 3 months of focused growth effort
Слайд 15
Results: KISSmetrics signups over time
Слайд 16
Simple 4-step experimentation process
Data
Figure out what to improve
Hypothesis
Make an educated guess
Experiment
Test your guess
Act
Decide what’s next
Слайд 17
Data
Figure out what to improve
Слайд 18
Data doesn’t make decisions for you.
Слайд 19
Data informs your decision making.
Слайд 20
Data helps you identify the most
painful problems people have.
(not the ones that bother you the most)
Слайд 21
Find problems worth solving by
starting with quantitative data.
Слайд 22
When in doubt, look at your top pages. Usually the homepage.
Слайд 23
A simple framework for choosing what to improve
Слайд 24
Find pages with high traffic and high bounce rate
HTTP://GOOGLE.COM/ANALYTICS
Слайд 25
Find pages with high traffic and low conversion rate
HTTP://UNBOUNCE.COM
Слайд 26
Find sources with high traffic and low conversion rate
HTTP://HUBSPOT.COM
Слайд 27
Find opportunities based on user paths
HTTP://KISS.LY/KMPATH
Слайд 28
How To Figure Out What To Improve
High Traffic
Increase Conversions
Do a Happy Dance
By A/B Testing
But Watch it Closely
Low Conversions
High Conversions
Get More Traffic
Get More Traffic
So You Can Start A/B Testing
So You Can Do a Happy Dance
Low Traffic
Слайд 29
Where do you have the most
visitors, users, customers,
emails, or notifications?
More volume helps you test faster.
Слайд 30
Common areas with large volume and high impact potential
๏ Website Traffic / App Installs
๏ Welcome Emails
๏ Web / Mobile Onboarding
๏ Email Digests
๏ Triggered Notifications
๏ Cart / Funnel Abandonment
๏ Dormant / Churned users
Слайд 31
When you’ve already got usage, analyze it.
What percentage of your
customers or users have
adopted each feature?
Intercom on Product Management
Слайд 32
Find out why by getting
qualitative data.
Слайд 33
A few qualitative tools
HTTP://USERTESTING.COM
HTTP://SURVEYMONKEY.COM
Слайд 34
A few more qualitative tools
HTTP://QUALAROO.COM
HTTP://OLARK.COM
Слайд 35
Use exit surveys to learn why people bounce
HTTP://QUALAROO.COM
Слайд 36
Use post-conversion surveys to understand why
HTTP://QUALAROO.COM
Слайд 37
Discover if your web pages are effective at their goal
HTTP://QUALAROO.COM
Слайд 38
What frustrates people?
EMAIL
Слайд 39
Find people who don’t do things and email them
EMAIL + SURVEYMONKEY
Слайд 40
User Research gives you a headstart
Слайд 41
Hypothesis
Make an educated guess
Слайд 42
Ideas
Designs
Solutions
}
Assumptions
(A guess about the best way to solve a problem)
Слайд 43
Everything can start with a
testable hypothesis.
Слайд 44
Before you create anything, make an educated guess.
Слайд 45
Use simple equations to assess potential impact of each idea
visits x conversion rate x order value x li4 = ?
ESTIMATED
MODELED
users x order value x resurrec7on rate = ?
ESTIMATED
MODELED
DAN MCKINLEY HTTP://KISS.LY/DANETSY
Слайд 46
Prioritize your tests
Assess the potential impact,
difficulty and priority of
each test idea you have.
Слайд 47
Experiment
Test your guess
Слайд 48
Test your hypothesis by starting
an experiment.
Слайд 49
Not button colors…
Слайд 50
Don’t copy “best practices” blindly
Слайд 51
Test specific button copy
Слайд 52
Run sensitivity tests on your web pages
Sensitivity tests help you
learn what your visitors care
about and what they don’t.
Слайд 53
Remove graphical elements and see what happens
Слайд 54
Does social proof and logos matter for your visitors?
Слайд 55
Remove social proof and learn what happens
Слайд 56
Act
Decide what’s next
Слайд 57
You’ve probably seen an A/B testing report before
Слайд 58
Document every A/B test and create a playbook
1. Your starting hypothesis.
2. Dates for when you ran the test.
3. Screenshots of control and all variations used during the test.
4. Expected change in conversion.
5. Probability of change.
6. Screenshot of the raw data used in your analysis.
7. A screenshot to the A/B Test Report in KISSmetrics.
8. The decision you've made from the test.
9. What you learned from the test.
Слайд 59
What learnings do we want to
carry over from the past?
Helps you avoid unknowingly
decreasing your conversions.
Слайд 60
Create experiment checklists based on learnings
✓ Keep the footer links
✓ Autofocus on the first form field
✓ Do not include social proof
✓ Link to the features page
✓ Use “Log in” copy on the button
Слайд 61
☀
If you’re not testing,
you’re not learning.
Слайд 62
⌾
Always have tests running,
no matter how small.
Слайд 63
∞
If you don't have a process,
you won't continuously improve.
Слайд 64
Work on things that matter.
Make those things better.
Be happier ☺
Слайд 65
Thank Y0u =)
Got questions? Need help?
Contact me any time!
Hiten Shah
hnshah@gmail.com
Слайд 66
