Java Performance: Biggest Mistakes

Понравилась презентация – покажи это...

Слайд 0

More on http://blog.dynatrace.com Dynatrace Free Trial: http://bit.ly/dttrial Andreas Grabner - @grabnerandi Java Performance: Biggest Mistakes

Слайд 1

And this is where I am from

Слайд 2

Слайд 3

Слайд 4

Can you tell my age by looking at my first computer?

Слайд 5

That’s why I ended up talking about performance

Слайд 6

Nobody wants this …

Слайд 7

Слайд 8

Nor this …

Слайд 9

Слайд 10

As it leads to this …

Слайд 11

The “War Room” Facebook – December 2012

Слайд 12

And potentially to this …

Слайд 13

Слайд 14

And this …

Слайд 15

Слайд 16

And that’s why Business doesn’t like it either …

Слайд 17

~80% of problems caused by ~20% patterns YES we know this 80% Dev Time in Bug Fixing $60B Defect Costs BUT

Слайд 18

6 Situations on WHY this happened, HOW to avoid it

Слайд 19

Слайд 20

“Blindly” (Re)use Existing Components

Слайд 21

Requirement: We need a report

Слайд 22

Using Hibernate results in 4k+ SQL Statements to display 3 items! Hibernate Executes 4k+ Statements Individual Execution VERY FAST But Total SUM takes 6s

Слайд 23

Слайд 24

Not every Architect makes good decisions

Слайд 25

Symptoms HTML takes between 60 and 120s to render High GC Time Assumptions Bad GC Tuning Probably bad Database Performance as rendering was simple Project: Online Room Reservation System

Слайд 26

Developers built own monitoring void roomreservationReport(int roomid) { long startTime = System.currentTimeMillis(); Object data = loadDataForRoom(roomid); long dataLoadTime = System.currentTimeMillis() - startTime; generateReport(data, roomid); } Result: Avg. Data Load Time: 45s! DB Tool says: Avg. SQL Query: <1ms!

Слайд 27

#1: Loading too much data 24889! Calls to the Database API! High CPU and High Memory Usage to keep all data in Memory

Слайд 28

#2: On individual connections 12444! individual connections Classical N+1 Query Problem Individual SQL really <1ms

Слайд 29

#3: Putting all data in temp Hashtable Lots of time spent in Hashtable.get Called from their Entity Objects

Слайд 30

Custom Measuring Was impacted by Garbage Collection Just measured overall time but not # SQL Executions Learn SQL and don’t use Hashtables as Workaround Lesson Learned void roomreservationReport(int roomid) { long startTime = System.currentTimeMillis(); Object data = loadDataForRoom(roomid); long dataLoadTime = System.currentTimeMillis() - startTime; generateReport(data, roomid); }

Слайд 31

Слайд 32

Implementation Flaws

Слайд 33

Business Impact requires Action!

Слайд 34

Solution: Cache to the RESCUE!!

Слайд 35

Implementation and Rollout Implemented InMemory Cache Worked well in Load Testing

Слайд 36

Result: Out of Memory Crashes!! Still crashes Problem fixed! Fixed Version Deployed

Слайд 37

Слайд 38

Disconnected Teams

Слайд 39

“Teamwork” between Dev and Ops SEV1 Problem in Production Need access to log files Where are they? Can’t get them Need to increase log level Can’t do! Can’t change config files in prod!

Слайд 40

Solution: Implement a Custom “On Demand” Remote Logger

Слайд 41

Implementation and Rollout Implemented Custom Logger Worked well in Load Testing

Слайд 42

What happened? ~ 1Mio Lock Exceptions in 30 mins

Слайд 43

Root Cause: A special WebSphere Setting!

Слайд 44

Слайд 45

“Deployment” Gone Bad!

Слайд 46

Test Environment Production Environment 8x slower 3x more SQL

Слайд 47

Test Environment Production Environment Hibernate, Classloading, XML – The Key Hotspots Hibernate, Classloading, XML – The Key Hotspots I/O for Web Requests doesn’t even show up! That’s Normal: Having I/O for Web Request as main contributor

Слайд 48

Top Contributor Class.getInterfaces Called from Hibernates FieldInterceptionHelper These calls all originate form thousands of calls to find item by code

Слайд 49

Top Methods related to XML Processing Classloading is triggered through CustomMonnkey and the Xalan Parser Classloading is triggered through CustomMonkey and the Xalan Parser

Слайд 50

Слайд 51

#Push without a Plan

Слайд 52

Mobile Landing Page of Super Bowl Ad 434 Resources in total on that page: 230 JPEGs, 75 PNGs, 50 GIFs, … Total size of ~ 20MB

Слайд 53

Fifa.com during Worldcup http://apmblog.compuware.com/2014/05/21/is-the-fifa-world-cup-website-ready-for-the-tournament/

Слайд 54

Waiting for Bug Reports

Слайд 55

Look behind the scenes

Слайд 56

Слайд 57

Performance Clinic: Bring Your Own App When: Thursday, December 18th Lunch Session: NOON – 3PM After Work Session: 4PM – 7PM Where: Dynatrace Office 404 Wyman Street, Waltham Register: http://bit.ly/onlineperfclinic

Слайд 58

Quick Demo

Слайд 59

Get it here: http://bit.ly/dttrial YouTube Tutorials: http://bit.ly/dttutorials Live Q&A Sessions: http://bit.ly/onlineperfclinic Contact me: agrabner@dynatrace.com - @grabnerandi Special Offer: SaaS VIP Program: http://bit.ly/dtaasvip 30 Days Dynatrace Free Trial

Слайд 60

Andreas Grabner Dynatrace Developer Advocate @grabnerandi http://blog.dynatrace.com