Growing Your Business Through Experimentation

If you like this presentation – show it...

Slide 0

Growing Your Business Through Experimentation Hiten Shah hnshah@gmail.com

Slide 1

What do you do?  Product, Design, Engineering, Marketing, Growth, Sales

Slide 2

Slide 3

How do you make all the things better?! Figuring this out is your job.

Slide 4

A/b testing

Slide 5

Only  1  out  of  5  tests  win. 4  out  of  5  tests  cost   you  money.

Slide 6

So, why experiment if so many tests fail?

Slide 7

Go Where Customers Are! Long, slow, SaaS ramp of death from 2006…

Slide 8

Go Where Customers Are! Now…

Slide 9

Go Where Customers Are! Growth can be sped up!

Slide 10

Increase Hello Bar’s installation rate

Slide 11

Improve the KISSmetrics features page

Slide 12

Increase signups from Crazy Egg’s homepage

Slide 13

Grow Crazy Egg’s revenue

Slide 14

Results from 3 months of focused growth effort

Slide 15

Results: KISSmetrics signups over time

Slide 16

Simple 4-step experimentation process Data Figure out what to improve Hypothesis Make an educated guess Experiment Test your guess Act Decide what’s next

Slide 17

Data Figure out what to improve

Slide 18

Data doesn’t make decisions for you.

Slide 19

Data informs your decision making.

Slide 20

Data helps you identify the most painful problems people have. (not the ones that bother you the most)

Slide 21

Find problems worth solving by starting with quantitative data.

Slide 22

When in doubt, look at your top pages. Usually the homepage.

Slide 23

A simple framework for choosing what to improve

Slide 24

Find pages with high traffic and high bounce rate HTTP://GOOGLE.COM/ANALYTICS

Slide 25

Find pages with high traffic and low conversion rate HTTP://UNBOUNCE.COM

Slide 26

Find sources with high traffic and low conversion rate HTTP://HUBSPOT.COM

Slide 27

Find opportunities based on user paths HTTP://KISS.LY/KMPATH

Slide 28

How To Figure Out What To Improve High Traffic Increase Conversions Do a Happy Dance By A/B Testing But Watch it Closely Low Conversions High Conversions Get More Traffic Get More Traffic So You Can Start A/B Testing So You Can Do a Happy Dance Low Traffic

Slide 29

Where do you have the most visitors, users, customers, emails, or notifications? More volume helps you test faster.

Slide 30

Common areas with large volume and high impact potential ๏ Website Traffic / App Installs ๏ Welcome Emails ๏ Web / Mobile Onboarding ๏ Email Digests ๏ Triggered Notifications ๏ Cart / Funnel Abandonment ๏ Dormant / Churned users

Slide 31

When you’ve already got usage, analyze it. What percentage of your customers or users have adopted each feature? Intercom on Product Management

Slide 32

Find out why by getting qualitative data.

Slide 33


Slide 34

A few more qualitative tools HTTP://QUALAROO.COM HTTP://OLARK.COM

Slide 35

Use exit surveys to learn why people bounce HTTP://QUALAROO.COM

Slide 36

Use post-conversion surveys to understand why HTTP://QUALAROO.COM

Slide 37

Discover if your web pages are effective at their goal HTTP://QUALAROO.COM

Slide 38

What frustrates people? EMAIL

Slide 39

Find people who don’t do things and email them EMAIL + SURVEYMONKEY

Slide 40

User Research gives you a headstart

Slide 41

Hypothesis Make an educated guess

Slide 42

Ideas Designs Solutions } Assumptions (A guess about the best way to solve a problem)

Slide 43

Everything can start with a testable hypothesis.

Slide 44

Before you create anything, make an educated guess.

Slide 45

Use simple equations to assess potential impact of each idea visits  x  conversion  rate  x  order  value  x  li4  =  ? ESTIMATED MODELED users  x  order  value  x  resurrec7on  rate  =  ? ESTIMATED MODELED DAN MCKINLEY HTTP://KISS.LY/DANETSY

Slide 46

Prioritize your tests Assess the potential impact, difficulty and priority of each test idea you have.

Slide 47

Experiment Test your guess

Slide 48

Test your hypothesis by starting an experiment.

Slide 49

Not button colors…

Slide 50

Don’t copy “best practices” blindly

Slide 51

Test specific button copy

Slide 52

Run sensitivity tests on your web pages Sensitivity tests help you learn what your visitors care about and what they don’t.

Slide 53

Remove graphical elements and see what happens

Slide 54

Does social proof and logos matter for your visitors?

Slide 55

Remove social proof and learn what happens

Slide 56

Act Decide what’s next

Slide 57

You’ve probably seen an A/B testing report before

Slide 58

Document every A/B test and create a playbook 1. Your starting hypothesis. 2. Dates for when you ran the test. 3. Screenshots of control and all variations used during the test. 4. Expected change in conversion. 5. Probability of change. 6. Screenshot of the raw data used in your analysis. 7. A screenshot to the A/B Test Report in KISSmetrics. 8. The decision you've made from the test. 9. What you learned from the test.

Slide 59

What learnings do we want to carry over from the past? Helps you avoid unknowingly decreasing your conversions.

Slide 60

Create experiment checklists based on learnings ✓ Keep the footer links ✓ Autofocus on the first form field ✓ Do not include social proof ✓ Link to the features page ✓ Use “Log in” copy on the button

Slide 61

☀ If you’re not testing, you’re not learning.

Slide 62

⌾ Always have tests running, no matter how small.

Slide 63

∞ If you don't have a process, you won't continuously improve.

Slide 64

Work on things that matter. Make those things better. Be happier ☺

Slide 65

Thank Y0u =) Got questions? Need help? Contact me any time! Hiten Shah hnshah@gmail.com

Slide 66