Понравилась презентация – покажи это...
DevOps and PerformanceWhy, How & Best Practices @grabnerandi http://apmblog.compuware.com
What you may have heard about Austrians
And just very recently @ Euro Song Contest
How we would like the world to see us ?
What we are also proud of ?
What you should check out …
Tonight we are here for …
Others Call It The stuff we did when we were a Start Up and we All were Devs, Testers and Ops
Or – The Evolution of Software Engineering
YOU ARE NOT ALONE: Popularity on Google
Who is doing it? How many successful deployments can they do? 300 Deployments / Year 50-60 Deployments / Day 10+ Deployments / Day Every 11.6 seconds
More on Amazons Story 75% fewer outages since 2006 90% fewer outage minutes ~0.001% of deployments cause a problem Instantaneous automatic rollback Deploying every 11.6s
Testing is Important – and gives Confidence
But are we ready for “The Real” world?
Measure Performance during the game Ball Possession: 40 : 60 Fouls: 0 : 0 Score: 0 : 0 Minute 1 - 5
Measure Performance during the game Minute 6 - 35 Ball Possession: 80 : 20 Fouls: 2 : 12 Score: 0 : 0
Deep Dive Analysis
Options “To Fix” the situation
Not always a happy ending ? Minute 90 Ball Possession: 80 : 20 Fouls: 4 : 25 Score: 3 : 0
FRUSTRATED FANS!! 25
How does that relate to Software?
From Deploy to … Deploy Timeline
The “War Room” – back then 'Houston, we have a problem‘NASA Mission Control Center, Apollo 13, 1970
The “War Room” – NOW Facebook – December 2012
3 Situations on WHY this happens, HOW to avoid it
Image taken from https://www.scriptrock.com/blog/devops-whats-hype-about/
“Teamwork” between Dev and Ops SEV1 Problem in Production Need access to log files Where are they? Can’t get them Need to increase log level Can’t do! Can’t change config files in prod!
“Solution”: Implement a Custom “On Demand” Remote Logger
Implementation and Rollout Implemented Custom Logger Worked well in Load Testing
What happened? ~ 1Mio Lock Exceptions in 30 mins
Root Cause: A special WebSphere Setting!
Metrics: # Log Messages, # Exceptions Share: Same Server Settings DevOps: Agree on Data for Troubleshooting
12 000 000 $
#No “Agile” Deployment
Ad on air Availability dropped to 0% Load Spike resulted in Unavailability
Alternative: “GoDaddy goes DevOps” Response time improved 4x 1h before SuperBowl KickOff 1h after Game ended
Behind the Scenes
Metrics: Availability Page Size, # Objects # Hosts, # Connections DevOps: “Feature” Switches
#Push without a Plan
Mobile Landing Page of Super Bowl Ad 434 Resources in total on that page: 230 JPEGs, 75 PNGs, 50 GIFs, … Total size of ~ 20MB
Critical Pages not Optimized! Browse, Search and Product Info performs well … because they don’t follow best practices: 87 Requests, 28 Redirects, … Critical Pages such as Shopping Cart are very slow …
Metrics: Load Time, # Resources (Images, …), # HTTP 3xx, 4xx, 5xx Dev: Build for Mobile Test: Test on Mobile Ops: Monitor Mobile
#1: Culture Change
#2: Define Measures # of Requests / User # of Log Messages # of Exceptions # Objects Allocated # Objects In Cache Cache Hit Ratio # of Images # of SQLs # SQLs per Request Availability # HTTP 3xx, 4xx Page Size
54 #3: Share Results
#4: Automate Commit Stage Compile Execute Unit Test Code Analysis Build installers Automated Acceptance Testing Automated Capacity Testing Manual testing Key showcases Exploratory testing Unit & Integration Tests Functional Tests Performance Tests Production Monitoring Functional Tests
If we do all that CANCELLED
Which gives you more time for the real important things in life …
Want MORE of these and more details? http://apmblog.compuware.com
Recommended Book https://itrevolution.wufoo.com/forms/phoenix-project-ebook-offer/
FREE Products & More Info dynaTrace Enterprise Full End-to-End Visibility in your Java, .NET, PHP Apps Sign up for a 15 Days Free Trial on http://compuwareapm.com dynaTrace AJAX Edition Browser Diagnostics for IE + FF Download @ http://ajax.dynatrace.com Our Blog: http://apmblog.compuware.com