Big Trends in Big Data & Analytics

If you like this presentation – show it...

Slide 0

Big Trends in Big Data & Analytics Timo Elliott VP, Global Innovation Evangelist AKA “What I personally find interesting”

Slide 1

Congratulations! YOU WON

Slide 2

88% How Do Executives Make Decisions? Aspect Consulting, 1997 12% Hard Facts Gut Feel 90% 10% Hard Facts Gut Feel Economist Intelligence Unit, 2014 Why the worst-practice shaded 3D donut charts? JUST TO ANNOY DATA VIZ EXPERTS! ?

Slide 3

Biggest Barriers to Business Intelligence 2015 2003 Sources: InformationWeek Survey 2015, BusinessWeek Survey, 2003

Slide 4

Plus Ca Change… Petabytes Data Scientists + IoT Big Data

Slide 5

Business Intelligence Success… Sources: InformationWeek Survey 2015, BusinessWeek Survey, 2003 ?

Slide 6

Use Analytics Today Need Analytics by 2020 Gartner, 2014 The Opportunity Inability to see, understand, and optimize new opportunities Inaccessible data and technology Insights remain hidden Complexity, cost, confusion Silos of approaches and analytic technologies 75% 10% Slow decision making lacking future view Rear view mirror BI mentality

Slide 7

cloud data mobile MORE! competition speed social connected There’s Been An Explosion of New Technology Means new opportunities…

Slide 8

Big Data Discovery = Big Data Data Discovery Data Science Gartner Strategic Planning Assumption: By 2017, Big Data Discovery Will Evolve Into a Distinct Market Category

Slide 9

Big Data Discovery Volume, velocity, or variety of data Potential business impact Difficult to implement Potentially expensive Lack of skills available Ease of use Agility and flexibility Time-to-results Installed user base Complexity of analysis Potential impact Range of tools Smart algorithms Difficult to implement Slow and complex Narrow focus of analysis Limited depth of information exploration Low complexity of analysis BIG DATA DATA SCIENCE DATA DISCOVERY

Slide 10

Big Data Discovery Simpler to use than data science Accessible to a wider range of users Broad range of data manipulation features Able to handle new types of data sources With adequate performance for big data BIG DATA DISCOVERY

Slide 11

Potential impact per user Potential user base The Rise of the Citizen Data Scientist? Business analyst Data scientist Citizen data scientist

Slide 12

New Products & Services

Slide 13

The Opportunity

Slide 14

SAP’s Opportunity Big Data Discovery SAP HANA (+ Hadoop etc.) SAP Predictive Analytics 2.0 SAP Lumira

Slide 15

The Landscape is Converging

Slide 16

May Imply Differently Sliced Products? Big Data Discovery Basic Big Data Discovery Team ETL BI Q&R OLAP Predictive Big Data Discovery Advanced Example only — not a product plan!

Slide 17

Boardroom Redefined Source: In-Memory Data Management: An Inflection Point for Enterprise Applications. Hasso Plattner Alexander Zeier

Slide 18

“Intricate calculations of sales by territories will appear as if by magic in the digital age ahead”

Slide 19

Decision Cockpits

Slide 20

Slide 21

Wal-Mart’s Data Cafe (“Collaborative Analytics Facilities for Enterprise”) Data from 245M customers/week, 11,000 stores under 71 banners in 27 countries and e-commerce websites in 11 countries with $482.2 Bn sales and 2.2M employees. 250 Bn rows of data 94% of queries run < 2s >1,000 concurrent users even under heavy loads. Data load throughput >20 million records/hour Suja Chandrasekaran CTO of Walmart Technology “In-memory cannot economically, or even practically, scale to the volumes of today’s data warehouses — Neil Raden, 2012”

Slide 22

Mercy Health Mercy Named One of Nation’s Most Wired for 11th Year 40K employees, >8M patients/year, 9 years of data, structured & unstructured

Slide 23

Hadoop Rising (?) 1Q 2014 1Q 2015 1Q 2013

Slide 24

The End of the Hadoop Honeymoon? "Despite considerable hype and reported successes for early adopters, 54% of survey respondents report no plans to invest at this time, while only 18%have plans to invest in Hadoop over the next two years. Furthermore, the early adopters don't appear to be championing for substantial Hadoop adoption over the next 24 months; in fact, there are fewer who plan to begin in the next two years than already have.” Nick Heudecker, research director at Gartner.

Slide 25

SAP, Open Source & Hadoop SAP Contributes to over 100 Open Source Projects

Slide 26

Bringing Enterprise Data to Hadoop and Hadoop Data to The Enterprise SPATIAL PROCESSING ANALYTICS, TEXT, GRAPH, PREDICTIVE ENGINES CONSUME COMPUTE STORAGE SOURCE INGEST Transformations & Cleansing Smart Data Integration Smart Data Quality Stream Processing Smart Data Streaming STREAM PROCESSING Mobile applications and BI Smart Data Access Virtual Tables User Defined Functions But there is more work to do…

Slide 27

The New Multi-Polar World of Big Data Architectures Data Warehouse Hybrid Transaction/Analytical Processing Hadoop, MongoDB, Spark, etc Where does data arrive? When does it need to move? Where does modeling happen? What can users do themselves? What governance is required? Big Data Architectures got complicated What we want — consistent, seamless solution

Slide 28

Apache Atlas

Slide 29

Data Wrangling Eats Into ETL “We had a short period of time to complete a massive data migration project which required us to extract, organize and clean 30 million records being moved from a legacy environment into an SAP system” Matt Heinz, Head of BI at Del Monte Foods, Inc. “Self-service data integration will do for traditional IT-centric data integration what data discovery platforms have done for traditional IT-centric BI… shifting much of the activity from IT to the business user” Rita Sallam, Gartner Analyst

Slide 30

Data Preparation is a Highly Iterative and Time-consuming Process Commonly accepted that ~80% of the work on data analytics is in preparation

Slide 31

Self-service Data Preparation Tools Reduce the Time and Complexity of Preparing the Data Source: Gartner Gartner predicts by 2018 most business users will have access to self-service tools to prepare data for analytics

Slide 32

SAP Agile Data Preparation: Cleanse

Slide 33

SAP Agile Data Preparation: De-Duplicate

Slide 34

SAP Agile Data Preparation: Merge

Slide 35

SAP Agile Data Preparation: Admin

Slide 36

SAP Agile Data Preparation: Operationalize Export Action History and Import as a flowgraph in HANA EIM

Slide 37

Data Visualization is Cool… (but) Not using pie charts Ease of use, training, data quality, incentives, organization, process, etc. etc. Importance for BI Success of:

Slide 38

We Still Need Reporting and Dashboards! Source: InformationWeek BI Survey 2015 Question: “To what extent are the following technologies used to share analytic and BI insights within your organization?” and response: “Used Extensively”

Slide 39

We Need To Support The Analytics Lifecycle

Slide 40

Descriptive: What happened? Diagnostic: Why did it happen? Predictive: What will happen? Prescriptive: How can we make it happen? Taking Analytics To The Next Level Hindsight Insight Foresight

Slide 41

Transport For London

Slide 42

Centerpoint Energy

Slide 43

DATA SCIENCE QUIZ. These numbers were found in two tax declarations. One is entirely made up. Which one? EUR 127,- 2.863,- 10.983,- 694,- 29.309,- 32,- 843,- 119.846,- 18.744,- 1.946,- 275,- EUR 937,- 82.654,- 18.465,- 725,- 98.832,- 7.363,- 4.538,- 38,- 8.327,- 482,- 2.945,- Benford's Law, also called the First-Digit Law

Slide 44

Benford’s Law Distribution of the first digit of real-world sets of numbers that uniformly span several orders of magnitude

Slide 45

1999 to 2009 “Greece shows the largest deviation from Benford’s law with respect to all measures. [And] the suspicion of manipulating data has officially been confirmed by the European Commission.” Fact and Fiction in EU-GovernmentalEconomic Data, 2011

Slide 46

Repeat purchases A B Big Data looks Beyond Sales of two new products six weeks after market introduction

Slide 47

Kaeser Compressors Enabling Predictive Maintenance A global leader in air compressors ?€500 million, 4,800 employees, 50 countries, partners in additional 60 countries

Slide 48

Modeling Example E.g. Total energy consumption Aggregation of 10 sec values Calculation of typical consumption patterns Pattern associated with each compressor and day Repeat for temperature, pressure, vibration, etc.

Slide 49

Predictive Examples Model combines sensor readings and ERP data (location, type of usage, last service, etc.) Status alerts: “Oil change / oil analyze / no action” Predict machine failure 24 hours in advance

Slide 50

High-Level Technical View Predictive Model (in-memory) Long-term disk storage User Interfaces CRM ERP Event Stream Processing all sampled Customer Field Svs Sales R&D DW

Slide 51

Benefits Customers Less downtime Decreased time to resolution Optimal longevity and performance Kaeser More efficient use of spare parts, etc New sales opportunities Better product development “We are seeing improved uptime of equipment, decreased time to resolution, reduced operational risks and accelerated innovation cycles. Most importantly, we have been able to align our products and services more closely with our customers’ needs.” Kaeser CIO Falko Lameter Next Steps: New Business Models

Slide 52

SAP HANA Cloud Platform - the Internet of Things enabled in-memory platform-as-a-service Machine Cloud (SAP) HANA Cloud IoT Services End Customer (On site) Business owner (SAP Customer) HANA Cloud Integration Business Suite Systems (ERP, CRM , etc.) SAP Connector Device HANA Cloud Platform Machine Integration Process Integration IoT Applications (SAP, Partner and Custom apps)

Slide 53

SIEMENS Cloud for Industry The SIEMENS ‘Cloud for Industry’ connects the worlds of machines and business via: the HCP for IoT open APIs easy connectivity. It is the successor of the SIEMENS Plant Data Services. It is planned to be an open platform: Open to non-Siemens assets and non-SAP back-ends Endorsing the OPC UA Standards Creating a separate, yet adjacent & complementary partner developer network Partner Connectivity Customer Connectivity SAP Connectivity SIEMENS Connectivity Partner Applications Customer Applications SAP Applications SIEMENS Applications Machine connectivity to SIEMENS customers plants Business Process Integration (SIEMENS or SIEMENS customers) Cloud for Industry

Slide 54

Tweeting Sharks!

Slide 55


Slide 56

Time to Reach For The Clouds?

Slide 57

Finance & Analytics: It’s Deja Vu All Over Again Cloud Enterprise Performance Management Governance, Risk, and Compliance Discover Inform Anticipate Plan Business Intelligence Predictive Analytics Real-time Business

Slide 58

Is This Your Finance Team? "With 90% certainty, here’s where we closed last month…"

Slide 59

Finance wants to be a business partner. And that requires better, more forward-looking data.

Slide 60

Rise of the Ops People* How are you feeling about the quarter? Good. Pipeline coverage is at 2.5x. The rep-level forecast implies a result of 105% of plan. The stage and category forecasts are between 99% and 104%. So, Good. BEFORE NOW * Source: Dave Kellogg, Ex CMO of BusinessObjects, now CEO of Cloud EPM Vendor Host Analytics!

Slide 61

Ops People Everywhere = Financial & Analytics Ops People: Modeling Planning Budgeting Reporting Analysis “What’s the impact on the results if we hire 10 more sales people in the UK?”

Slide 62

Planning For The Rest of Us

Slide 63

It’s Not You, It’s Your Data… “We found, on average, that 45% of the data business people use resides outside of the enterprise BI environments. An astonishingly miniscule 2% of business decision-makers reported using solely enterprise BI applications. This is undoubtedly connected to 76% of business respondents indicating they continue to resort to spreadsheets and other homegrown BI applications to analyze BI data. ” Source: Forrester

Slide 64

Suits vs. Hoodies

Slide 65

Advanced Governance Central IT no longer has a veto — you need the “consent of the governed” This means you have to behave more like a politician… Vote for my policies!

Slide 66

Build and Nurture a Community Regular face-to-face meetings Bring people together across silos: IT, Analysts, Business Leaders, Execs Presentations of successes best practices Invite external speakers Virtual communities Leverage internal social tools for people to share information Community-driven BI content Community self-policing Act as BICC eyes and ears to discover projects, opportunities Social mechanisms to ensure the “right behaviors” Ensure support at all levels Not just executives — middle and users

Slide 67

Conclusion: There’s a LOT Going On in Analytics The future of the boardroom (finally) SAP HANA & Hadoop Multi-polar big data architectures Self-service data preparation Supporting the analytics lifecycle Prescriptive and predictive analytics Internet of things for business Big data discovery Finance and analytics converge (again) Analytics culture & governance

Slide 68

“Judge a man by his questions rather than his answers.” Voltaire “Status Quo is, you know, latin for “the mess we’re in” Ronald Reagan “Any intelligent fool can make things bigger and more complex. It takes a touch of genius and a lot of courage to move in the opposite direction.” E.F. Schumacher

Slide 69

Thank You! Timo Elliott VP, Global innovation Evangelist Timo.Elliott@sap.com @timoelliott timoelliott.com/docs/UKISUG_top_analytic_trends.zip