Metrics, metrics everywhere (but where the heck do you start?)

Понравилась презентация – покажи это...

Слайд 0

Metrics, metrics everywhere (but where the heck do you start?) Tammy Everts & Cliff Crocker Velocity Santa Clara 2015

Слайд 1

@tameverts @cliffcrocker #velocityconf

Слайд 2

Слайд 3

Who cares about performance today? How do I measure performance? How fast am I? How fast should I be? How do I get there?

Слайд 4

Слайд 5

The myth of a single metric

Слайд 6

Слайд 7

Слайд 8

Who cares about performance?

Слайд 9

“47% of consumers expect a web page to load in 2 seconds or less.” Akamai, 2009

Слайд 10

Слайд 11

1s = $27M DNS 144ms Start render 1.59s Hero image render 2.01s

Слайд 12

How do I measure performance?

Слайд 13

Слайд 14

Слайд 15

Слайд 16

Android device fragmentation OpenSignal, August 2014

Слайд 17

RUM versus plus synthetic

Слайд 18

RUM 101

Слайд 19

Technology for collecting performance metrics directly from the end user’s browser Involves instrumenting your site via JavaScript Measurements are fired across the network to a collection point through a small request object (beacon)

Слайд 20

What makes RUM great Always on Every user, every browser, everywhere Able to capture human behavior/events Only getting better

Слайд 21

Questions your RUM data can answer

Слайд 22

What are my users’ environments?

Слайд 23

How do visitors move through my site?

Слайд 24

How are my third-party scripts performing in real time?

Слайд 25

What impact does performance have on my business?

Слайд 26

Synthetic Monitoring 101

Слайд 27

Uses automated agents (bots) to measure your website from different physical locations A set “path” or URL is defined Tests are run either ad hoc or scheduled, and data is collected

Слайд 28

What makes synthetic monitoring great Rich data collected (waterfall, video, filmstrip, HTTP headers) Consistent “clean room” baseline Nothing to install Doesn’t require users / ability to measure pre-production and competition

Слайд 29

Questions your synthetic data can answer

Слайд 30

How do I compare to the competition?

Слайд 31

How does the design of my pages affect performance?

Слайд 32

How does the newest version of my site compare to previous versions?

Слайд 33

How well am I sticking to my performance budget?

Слайд 34

What if my third parties fail? Original: 3.5s SPOF: 22.7s

Слайд 35

Слайд 36

Common things we hear about RUM & synthetic Why are these numbers so different? I don’t trust your data. Your numbers are wrong. How are you calculating page load time? I can’t share two sets of numbers with the business?

Слайд 37

“But it loads so much faster for me!?!” 2015 Macbook Pro Warm browser cache FIOS X86 – Windows 7 VM Completely cold cache/DNS Throttled bandwidth

Слайд 38

boomerang.js Episodes

Слайд 39

W3C Performance Working Group

Слайд 40

Слайд 41

Слайд 42

How fast am I?

Слайд 43

Navigation Timing API

Слайд 44

Browser support for Navigation Timing !

Слайд 45

Role-specific use cases Network operations Front-end developer Marketing and site operations Designer C-level

Слайд 46

Use case: Measure network performance

Слайд 47

I need visibility into… issues with authoritative DNS servers impact of denial of service attacks on end users efficiency of connection re-use tier 1 connectivity issues (load balancer, web server)

Слайд 48

Слайд 49

Measuring DNS and TCP function getPerf() { var timing = window.performance.timing; return { dns: timing.domainLookupEnd - timing.domainLookupStart}; connect: timing.connectEnd - timing.connectStart}; }

Слайд 50

What’s with all those zeros! Connection reuse DNS caching Prefetching

Слайд 51

Focus on higher percentiles 85th percentile Median (50th)

Слайд 52

Use case: Measure front-end browser events

Слайд 53

How do I… understand the impact of back-end versus front-end on page speed? investigate how DOM complexity affects performance? measure a “fully loaded” page?

Слайд 54

Слайд 55

Isolate front-end vs. back-end

Слайд 56

Isolate front-end vs. back-end function getPerf() { var timing = window.performance.timing; return { ttfb: timing.responseStart - timing.connectEnd}; basePage: timing.responseEnd - timing.responseStart}; frontEnd: timing.loadEventStart - timing.responseEnd}; }

Слайд 57

Слайд 58

Front-end Back-end

Слайд 59

Investigate DOM events function getPerf() { var timing = window.performance.timing; return { DLoading: timing.domLoading – timing.navigationStart}; DInt: timing.domInteractive – timing.navigationStart}; DContLoaded: timing.domContentLoadedEventEnd – timing.navigationStart; DContLoadTime: timing.domContentLoadedEventEnd – timing.domContentLoadedEventStart}; DComplete: timing.domComplete - timing.navigationStart}; PLoad: timing.loadEventStart - timing.navigationStart}; }

Слайд 60

2618 DOM nodes 86 DOM nodes Visualizing DOM complexity

Слайд 61

Use case: Measure object-level performance

Слайд 62

I need to understand… how third-party content affects my performance how long a group of assets takes to download how assets served by a CDN are performing

Слайд 63

Слайд 64

Resource Timing interface

Слайд 65

Browser support for Resource Timing

Слайд 66

Cross-Origin Resource Sharing (CORS) Start/End time only unless Timing-Allow-Origin HTTP Response Header defined Timing-Allow-Origin = "Timing-Allow-Origin" ":" origin-list-or-null | "*"

Слайд 67

Resource Timing var rUrl = ‘http://www.akamai.com/images/img/cliff-crocker-dualtone-150x150.png’; var me = performance.getEntriesByName(rUrl)[0]; var timings = { loadTime: me.duration, dns: me.domainLookupEnd - me.domainLookupStart, tcp: me.connectEnd - me.connectStart, waiting: me.responseStart - me.requestStart, fetch: me.responseEnd - me.responseStart } Measuring a single resource

Слайд 68

Other uses for Resource Timing Slowest resources Time to first image (download) Response time by domain Time a group of assets Response time by initiator type (element type) Cache-hit ratio for resources For examples see: https://github.com/lognormal/beyond-page-metrics

Слайд 69

Using Resource Groups PLT impact not due resource groups PLT impact correlates with improvement from image domains

Слайд 70

Use case: Measure the user experience

Слайд 71

I just need to understand… when users perceive the page to be ready how long until my page begins to render when content above the fold is visible

Слайд 72

Слайд 73

The fallacy of “First Paint” in the wild Support for First Paint is not standardized between browsers Metric can be misleading (i.e. painting a white screen)

Слайд 74

First Paint is not equal to Start Render! Chrome – “First Paint” True Start Render

Слайд 75

Start Render and filmstrips

Слайд 76

User Timing Interface Allows developers to measure performance of their applications through high-precision timestamps Consists of “marks” and “measures” PerformanceMark: Timestamp PerformanceMeasure: Duration between two given marks

Слайд 77

Measure duration between two marks performance.mark(“startTask”); //Some stuff you want to measure happens here performance.mark(“stopTask”); //Measure the duration between the two marks performance.measure(“taskDuration”,“startTask”,“stopTask”);

Слайд 78

How long does it take to display the main product image on my site?

Слайд 79

Record when an image loads <img src=“image1.gif” onload=“performance.mark(‘image1’)”> For more interesting examples, see: Measure hero image delay http://www.stevesouders.com/blog/2015/05/12/hero-image-custom-metrics/ Measure aggregate times to get an “above fold time” http://blog.patrickmeenan.com/2013/07/measuring-performance-of-user-experience.html

Слайд 80

How do I measure performance when the onload event no longer matters? Use case: Measure performance of SPAs

Слайд 81

onload event visible resources

Слайд 82

Measuring SPAs Accept the fact that onload no longer matters Tie into routing events of SPA framework Measure downloads during soft refreshes (support in boomerang.js for Angular and other SPA frameworks) See: http://www.soasta.com/blog/angularjs-real-user-monitoring-single-page-applications/

Слайд 83

How fast should I be?

Слайд 84

Use case: Measure business impact

Слайд 85

I need to understand… how performance affects business KPIs how our site compares to our competitors

Слайд 86

Слайд 87

Слайд 88

Слайд 89

2% increase in conversions for every 1 second of improvement

Слайд 90

Cut load times in half Increased sales by 13%

Слайд 91

Слайд 92

So what? You must look at your own data.

Слайд 93

Слайд 94

Слайд 95

Not all pages are created equal Top of funnel impact (“browse” pages) For a typical ecommerce site, conversion rate drops by up to 50% when “browse” pages increase from 1 to 6 seconds

Слайд 96

Not all pages are created equal Bottom of funnel impact (“checkout” pages) However, there is much less impact to conversion when “checkout” pages degrade

Слайд 97

Conversion Impact Score

Слайд 98

How do I get there?

Слайд 99

Слайд 100

Create a performance budget See: Setting a Performance Budget http://timkadlec.com/2013/01/setting-a-performance-budget/ Performance Budget Metrics http://timkadlec.com/2014/11/performance-budget-metrics/

Слайд 101

Set meaningful, page-specific SLAs

Слайд 102

“Response time measured using resource timing from Chrome browsers in the United States should not exceed a median (50th percentile) of 100ms or a 95th percentile of 500ms for a population of more than 500 users in a 24-hour period.”

Слайд 103

“Vendor will make an effort to ensure average response times for content is within reasonable limits.”

Слайд 104

Слайд 105

Chapter 8: Changing Culture at Your Organization

Слайд 106

performancebacon.com performancebeacon.com

Слайд 107


Слайд 108

Meet us at booth #801