Metrics, metrics everywhere (but where the heck do you start?)

If you like this presentation – show it...

Slide 0

Metrics, metrics everywhere (but where the heck do you start?) Tammy Everts & Cliff Crocker Velocity Santa Clara 2015

Slide 1

@tameverts @cliffcrocker #velocityconf

Slide 2

Slide 3

Who cares about performance today? How do I measure performance? How fast am I? How fast should I be? How do I get there?

Slide 4

Slide 5

The myth of a single metric

Slide 6

Slide 7

Slide 8

Who cares about performance?

Slide 9

“47% of consumers expect a web page to load in 2 seconds or less.” Akamai, 2009

Slide 10

Slide 11

1s = $27M DNS 144ms Start render 1.59s Hero image render 2.01s

Slide 12

How do I measure performance?

Slide 13

Slide 14

Slide 15

Slide 16

Android device fragmentation OpenSignal, August 2014

Slide 17

RUM versus plus synthetic

Slide 18

RUM 101

Slide 19

Technology for collecting performance metrics directly from the end user’s browser Involves instrumenting your site via JavaScript Measurements are fired across the network to a collection point through a small request object (beacon)

Slide 20

What makes RUM great Always on Every user, every browser, everywhere Able to capture human behavior/events Only getting better

Slide 21

Questions your RUM data can answer

Slide 22

What are my users’ environments?

Slide 23

How do visitors move through my site?

Slide 24

How are my third-party scripts performing in real time?

Slide 25

What impact does performance have on my business?

Slide 26

Synthetic Monitoring 101

Slide 27

Uses automated agents (bots) to measure your website from different physical locations A set “path” or URL is defined Tests are run either ad hoc or scheduled, and data is collected

Slide 28

What makes synthetic monitoring great Rich data collected (waterfall, video, filmstrip, HTTP headers) Consistent “clean room” baseline Nothing to install Doesn’t require users / ability to measure pre-production and competition

Slide 29

Questions your synthetic data can answer

Slide 30

How do I compare to the competition?

Slide 31

How does the design of my pages affect performance?

Slide 32

How does the newest version of my site compare to previous versions?

Slide 33

How well am I sticking to my performance budget?

Slide 34

What if my third parties fail? Original: 3.5s SPOF: 22.7s

Slide 35

Slide 36

Common things we hear about RUM & synthetic Why are these numbers so different? I don’t trust your data. Your numbers are wrong. How are you calculating page load time? I can’t share two sets of numbers with the business?

Slide 37

“But it loads so much faster for me!?!” 2015 Macbook Pro Warm browser cache FIOS X86 – Windows 7 VM Completely cold cache/DNS Throttled bandwidth

Slide 38

boomerang.js Episodes

Slide 39

W3C Performance Working Group

Slide 40

Slide 41

Slide 42

How fast am I?

Slide 43

Navigation Timing API

Slide 44

Browser support for Navigation Timing !

Slide 45

Role-specific use cases Network operations Front-end developer Marketing and site operations Designer C-level

Slide 46

Use case: Measure network performance

Slide 47

I need visibility into… issues with authoritative DNS servers impact of denial of service attacks on end users efficiency of connection re-use tier 1 connectivity issues (load balancer, web server)

Slide 48

Slide 49

Measuring DNS and TCP function getPerf() { var timing = window.performance.timing; return { dns: timing.domainLookupEnd - timing.domainLookupStart}; connect: timing.connectEnd - timing.connectStart}; }

Slide 50

What’s with all those zeros! Connection reuse DNS caching Prefetching

Slide 51

Focus on higher percentiles 85th percentile Median (50th)

Slide 52

Use case: Measure front-end browser events

Slide 53

How do I… understand the impact of back-end versus front-end on page speed? investigate how DOM complexity affects performance? measure a “fully loaded” page?

Slide 54

Slide 55

Isolate front-end vs. back-end

Slide 56

Isolate front-end vs. back-end function getPerf() { var timing = window.performance.timing; return { ttfb: timing.responseStart - timing.connectEnd}; basePage: timing.responseEnd - timing.responseStart}; frontEnd: timing.loadEventStart - timing.responseEnd}; }

Slide 57

Slide 58

Front-end Back-end

Slide 59

Investigate DOM events function getPerf() { var timing = window.performance.timing; return { DLoading: timing.domLoading – timing.navigationStart}; DInt: timing.domInteractive – timing.navigationStart}; DContLoaded: timing.domContentLoadedEventEnd – timing.navigationStart; DContLoadTime: timing.domContentLoadedEventEnd – timing.domContentLoadedEventStart}; DComplete: timing.domComplete - timing.navigationStart}; PLoad: timing.loadEventStart - timing.navigationStart}; }

Slide 60

2618 DOM nodes 86 DOM nodes Visualizing DOM complexity

Slide 61

Use case: Measure object-level performance

Slide 62

I need to understand… how third-party content affects my performance how long a group of assets takes to download how assets served by a CDN are performing

Slide 63

Slide 64

Resource Timing interface

Slide 65

Browser support for Resource Timing

Slide 66

Cross-Origin Resource Sharing (CORS) Start/End time only unless Timing-Allow-Origin HTTP Response Header defined Timing-Allow-Origin = "Timing-Allow-Origin" ":" origin-list-or-null | "*"

Slide 67

Resource Timing var rUrl = ‘http://www.akamai.com/images/img/cliff-crocker-dualtone-150x150.png’; var me = performance.getEntriesByName(rUrl)[0]; var timings = { loadTime: me.duration, dns: me.domainLookupEnd - me.domainLookupStart, tcp: me.connectEnd - me.connectStart, waiting: me.responseStart - me.requestStart, fetch: me.responseEnd - me.responseStart } Measuring a single resource

Slide 68

Other uses for Resource Timing Slowest resources Time to first image (download) Response time by domain Time a group of assets Response time by initiator type (element type) Cache-hit ratio for resources For examples see: https://github.com/lognormal/beyond-page-metrics

Slide 69

Using Resource Groups PLT impact not due resource groups PLT impact correlates with improvement from image domains

Slide 70

Use case: Measure the user experience

Slide 71

I just need to understand… when users perceive the page to be ready how long until my page begins to render when content above the fold is visible

Slide 72

Slide 73

The fallacy of “First Paint” in the wild Support for First Paint is not standardized between browsers Metric can be misleading (i.e. painting a white screen)

Slide 74

First Paint is not equal to Start Render! Chrome – “First Paint” True Start Render

Slide 75

Start Render and filmstrips

Slide 76

User Timing Interface Allows developers to measure performance of their applications through high-precision timestamps Consists of “marks” and “measures” PerformanceMark: Timestamp PerformanceMeasure: Duration between two given marks

Slide 77

Measure duration between two marks performance.mark(“startTask”); //Some stuff you want to measure happens here performance.mark(“stopTask”); //Measure the duration between the two marks performance.measure(“taskDuration”,“startTask”,“stopTask”);

Slide 78

How long does it take to display the main product image on my site?

Slide 79

Record when an image loads <img src=“image1.gif” onload=“performance.mark(‘image1’)”> For more interesting examples, see: Measure hero image delay http://www.stevesouders.com/blog/2015/05/12/hero-image-custom-metrics/ Measure aggregate times to get an “above fold time” http://blog.patrickmeenan.com/2013/07/measuring-performance-of-user-experience.html

Slide 80

How do I measure performance when the onload event no longer matters? Use case: Measure performance of SPAs

Slide 81

onload event visible resources

Slide 82

Measuring SPAs Accept the fact that onload no longer matters Tie into routing events of SPA framework Measure downloads during soft refreshes (support in boomerang.js for Angular and other SPA frameworks) See: http://www.soasta.com/blog/angularjs-real-user-monitoring-single-page-applications/

Slide 83

How fast should I be?

Slide 84

Use case: Measure business impact

Slide 85

I need to understand… how performance affects business KPIs how our site compares to our competitors

Slide 86

Slide 87

Slide 88

Slide 89

2% increase in conversions for every 1 second of improvement

Slide 90

Cut load times in half Increased sales by 13%

Slide 91

Slide 92

So what? You must look at your own data.

Slide 93

Slide 94

Slide 95

Not all pages are created equal Top of funnel impact (“browse” pages) For a typical ecommerce site, conversion rate drops by up to 50% when “browse” pages increase from 1 to 6 seconds

Slide 96

Not all pages are created equal Bottom of funnel impact (“checkout” pages) However, there is much less impact to conversion when “checkout” pages degrade

Slide 97

Conversion Impact Score

Slide 98

How do I get there?

Slide 99

Slide 100

Create a performance budget See: Setting a Performance Budget http://timkadlec.com/2013/01/setting-a-performance-budget/ Performance Budget Metrics http://timkadlec.com/2014/11/performance-budget-metrics/

Slide 101

Set meaningful, page-specific SLAs

Slide 102

“Response time measured using resource timing from Chrome browsers in the United States should not exceed a median (50th percentile) of 100ms or a 95th percentile of 500ms for a population of more than 500 users in a 24-hour period.”

Slide 103

“Vendor will make an effort to ensure average response times for content is within reasonable limits.”

Slide 104

Slide 105

Chapter 8: Changing Culture at Your Organization

Slide 106

performancebacon.com performancebeacon.com

Slide 107


Slide 108

Meet us at booth #801