Понравилась презентация – покажи это...
Big Trends in Big Data & Analytics Timo Elliott VP, Global Innovation Evangelist AKA “What I personally find interesting”
Congratulations! YOU WON
88% How Do Executives Make Decisions? Aspect Consulting, 1997 12% Hard Facts Gut Feel 90% 10% Hard Facts Gut Feel Economist Intelligence Unit, 2014 Why the worst-practice shaded 3D donut charts? JUST TO ANNOY DATA VIZ EXPERTS! ?
Biggest Barriers to Business Intelligence 2015 2003 Sources: InformationWeek Survey 2015, BusinessWeek Survey, 2003
Plus Ca Change… Petabytes DataScientists + IoT Big Data
Business Intelligence Success… Sources: InformationWeek Survey 2015, BusinessWeek Survey, 2003 ?
Use AnalyticsToday NeedAnalytics by 2020 Gartner, 2014 The Opportunity Inability to see, understand, and optimize new opportunities Inaccessible dataand technology Insights remain hidden Complexity, cost, confusion Silos of approaches and analytic technologies 75% 10% Slow decision making lacking future view Rear view mirrorBI mentality
cloud data mobile MORE! competition speed social connected There’s Been An Explosion of New Technology Means new opportunities…
Big Data Discovery = Big Data Data Discovery Data Science Gartner Strategic Planning Assumption: By 2017, Big Data Discovery Will Evolve Into a Distinct Market Category
Big Data Discovery Volume, velocity, or variety of data Potential business impact Difficult to implement Potentially expensive Lack of skills available Ease of use Agility and flexibility Time-to-results Installed user base Complexity of analysis Potential impact Range of tools Smart algorithms Difficult to implement Slow and complex Narrow focus of analysis Limited depth of information exploration Low complexity of analysis BIG DATA DATA SCIENCE DATA DISCOVERY
Big Data Discovery Simpler to use than data science Accessible to a wider range of users Broad range of data manipulation features Able to handle new types of data sources With adequate performance for big data BIG DATA DISCOVERY
Potential impact per user Potential user base The Rise of the Citizen Data Scientist? Business analyst Data scientist Citizen data scientist
New Products & Services
SAP’s Opportunity Big Data Discovery SAP HANA (+ Hadoop etc.) SAP Predictive Analytics 2.0 SAP Lumira
The Landscape is Converging
May Imply Differently Sliced Products? Big Data Discovery Basic Big Data Discovery Team ETL BI Q&R OLAP Predictive Big Data Discovery Advanced Example only — not a product plan!
Boardroom Redefined Source: In-Memory Data Management: An Inflection Point for Enterprise Applications. Hasso Plattner Alexander Zeier
“Intricate calculations of sales by territories will appear as if by magic in the digital age ahead”
Wal-Mart’s Data Cafe (“Collaborative Analytics Facilities for Enterprise”) Data from 245M customers/week, 11,000 stores under 71 banners in 27 countries and e-commerce websites in 11 countries with $482.2 Bn sales and 2.2M employees. 250 Bn rows of data 94% of queries run < 2s >1,000 concurrent users even under heavy loads. Data load throughput >20 million records/hour Suja ChandrasekaranCTO of Walmart Technology “In-memory cannot economically, or even practically, scale to the volumes of today’s data warehouses — Neil Raden, 2012”
Mercy Health Mercy Named One of Nation’s Most Wired for 11th Year 40K employees, >8M patients/year, 9 years of data, structured & unstructured
Hadoop Rising (?) 1Q 2014 1Q 2015 1Q 2013
The End of the Hadoop Honeymoon? "Despite considerable hype and reported successes for early adopters, 54% of survey respondents report no plans to invest at this time, while only 18%have plans to invest in Hadoop over the next two years. Furthermore, the early adopters don't appear to be championing for substantial Hadoop adoption over the next 24 months; in fact, there are fewer who plan to begin in the next two years than already have.” Nick Heudecker, research director at Gartner.
SAP, Open Source & Hadoop SAP Contributes to over 100 Open Source Projects
Bringing Enterprise Data to Hadoop and Hadoop Data to The Enterprise SPATIAL PROCESSING ANALYTICS, TEXT, GRAPH, PREDICTIVE ENGINES CONSUME COMPUTE STORAGE SOURCE INGEST Transformations & Cleansing Smart Data IntegrationSmart Data Quality StreamProcessing Smart Data Streaming STREAM PROCESSING Mobile applications and BI Smart Data Access Virtual Tables User Defined Functions But there is more work to do…
The New Multi-Polar World of Big Data Architectures Data Warehouse Hybrid Transaction/Analytical Processing Hadoop, MongoDB, Spark, etc Where does data arrive? When does it need to move? Where does modeling happen? What can users do themselves? What governance is required? Big Data Architectures got complicated What we want — consistent, seamless solution
Data Wrangling Eats Into ETL “We had a short period of time to complete a massive data migration project which required us to extract, organize and clean 30 million records being moved from a legacy environment into an SAP system” Matt Heinz, Head of BI at Del Monte Foods, Inc. “Self-service data integration will do for traditional IT-centric data integration what data discovery platforms have done for traditional IT-centric BI… shifting much of the activity from IT to the business user” Rita Sallam, Gartner Analyst
Data Preparation is a Highly Iterative and Time-consuming ProcessCommonly accepted that ~80% of the work on data analytics is in preparation
Self-service Data Preparation Tools Reduce the Time and Complexity of Preparing the Data Source: Gartner Gartner predicts by 2018 most business users will have access to self-service tools to prepare data for analytics
SAP Agile Data Preparation: Cleanse
SAP Agile Data Preparation: De-Duplicate
SAP Agile Data Preparation: Merge
SAP Agile Data Preparation: Admin
SAP Agile Data Preparation: Operationalize Export Action History and Import as a flowgraph in HANA EIM
Data Visualization is Cool… (but) Not using pie charts Ease of use, training, data quality, incentives, organization, process, etc. etc. Importance for BI Success of:
We Still Need Reporting and Dashboards! Source: InformationWeek BI Survey 2015 Question: “To what extent are the following technologies used to share analytic and BI insights within your organization?” and response: “Used Extensively”
We Need To Support The Analytics Lifecycle
Descriptive: What happened? Diagnostic: Why did it happen? Predictive: What will happen? Prescriptive: How can we make it happen? Taking Analytics To The Next Level Hindsight Insight Foresight
Transport For London
DATA SCIENCE QUIZ. These numbers were found in two tax declarations. One is entirely made up. Which one? EUR 127,- 2.863,- 10.983,- 694,- 29.309,- 32,- 843,- 119.846,- 18.744,- 1.946,- 275,- EUR 937,- 82.654,- 18.465,- 725,- 98.832,- 7.363,- 4.538,- 38,- 8.327,- 482,- 2.945,- Benford's Law, also called the First-Digit Law
Benford’s Law Distribution of the first digit of real-world sets of numbers that uniformly span several orders of magnitude
1999 to 2009 “Greece shows the largest deviation from Benford’s law with respect to all measures. [And] the suspicion of manipulating data has officially been confirmed by the European Commission.” Fact and Fiction in EU-GovernmentalEconomic Data, 2011
Repeat purchases A B Big Data looks Beyond Sales of two new products six weeks after market introduction
Kaeser Compressors Enabling Predictive Maintenance A global leader in air compressors ?€500 million, 4,800 employees, 50 countries, partners in additional 60 countries
Modeling Example E.g. Total energy consumption Aggregation of 10 sec values Calculation of typical consumption patterns Pattern associated with each compressor and day Repeat for temperature, pressure, vibration, etc.
Predictive Examples Model combines sensor readings and ERP data (location, type of usage, last service, etc.) Status alerts: “Oil change / oil analyze / no action” Predict machine failure 24 hours in advance
High-Level Technical View Predictive Model(in-memory) Long-term disk storage User Interfaces CRMERP Event Stream Processing all sampled Customer Field Svs Sales R&D DW
Benefits Customers Less downtime Decreased time to resolution Optimal longevity and performance Kaeser More efficient use of spare parts, etc New sales opportunities Better product development “We are seeing improved uptime of equipment, decreased time to resolution, reduced operational risks and accelerated innovation cycles. Most importantly, we have been able to align our products and services more closely with our customers’ needs.” Kaeser CIO Falko Lameter Next Steps: New Business Models
SAP HANA Cloud Platform - the Internet of Things enabled in-memory platform-as-a-service Machine Cloud (SAP) HANA CloudIoT Services End Customer(On site) Business owner(SAP Customer) HANA Cloud Integration Business Suite Systems(ERP, CRM , etc.) SAP Connector Device HANA Cloud Platform Machine Integration Process Integration IoT Applications (SAP, Partner and Custom apps)
SIEMENS Cloud for Industry The SIEMENS ‘Cloud for Industry’ connects the worlds of machines and business via: the HCP for IoT open APIs easy connectivity. It is the successor of the SIEMENS Plant Data Services. It is planned to be an open platform: Open to non-Siemens assets and non-SAP back-ends Endorsing the OPC UA Standards Creating a separate, yet adjacent & complementary partner developer network PartnerConnectivity CustomerConnectivity SAPConnectivity SIEMENSConnectivity PartnerApplications CustomerApplications SAPApplications SIEMENSApplications Machine connectivity to SIEMENS customers plants Business Process Integration (SIEMENS or SIEMENS customers) Cloud for Industry
Time to Reach For The Clouds?
Finance & Analytics: It’s Deja Vu All Over Again Cloud EnterprisePerformanceManagement Governance,Risk, andCompliance Discover Inform Anticipate Plan BusinessIntelligence PredictiveAnalytics Real-timeBusiness
Is This Your Finance Team? "With 90% certainty, here’s where we closed last month…"
Finance wants to be a business partner. And that requires better, more forward-looking data.
Rise of the Ops People* How are you feeling about the quarter? Good. Pipeline coverage is at 2.5x. The rep-level forecast implies a result of 105% of plan. The stage and category forecasts are between 99% and 104%. So, Good. BEFORE NOW * Source: Dave Kellogg, Ex CMO of BusinessObjects, now CEO of Cloud EPM Vendor Host Analytics!
Ops People Everywhere = Financial & Analytics Ops People: Modeling Planning Budgeting Reporting Analysis “What’s the impact on the results if we hire 10 more sales people in the UK?”
Planning For The Rest of Us
It’s Not You, It’s Your Data… “We found, on average, that 45% of the data business people use resides outside of the enterprise BI environments. An astonishingly miniscule 2% of business decision-makers reported using solely enterprise BI applications. This is undoubtedly connected to 76% of business respondents indicating they continue to resort to spreadsheets and other homegrown BI applications to analyze BI data. ” Source: Forrester
Suits vs. Hoodies
Advanced Governance Central IT no longer has a veto — you need the “consent of the governed” This means you have to behave more like a politician… Vote for my policies!
Build and Nurture a Community Regular face-to-face meetings Bring people together across silos: IT, Analysts, Business Leaders, Execs Presentations of successes best practices Invite external speakers Virtual communities Leverage internal social tools for people to share information Community-driven BI content Community self-policing Act as BICC eyes and ears to discover projects, opportunities Social mechanisms to ensure the “right behaviors” Ensure support at all levels Not just executives — middle and users
Conclusion: There’s a LOT Going On in Analytics The future of the boardroom (finally) SAP HANA & Hadoop Multi-polar big data architectures Self-service data preparation Supporting the analytics lifecycle Prescriptive and predictive analytics Internet of things for business Big data discovery Finance and analytics converge (again) Analytics culture & governance
“Judge a man by his questions rather than his answers.” Voltaire “Status Quo is, you know, latin for “the mess we’re in” Ronald Reagan “Any intelligent fool can make things bigger and more complex. It takes a touch of genius and a lot of courage to move in the opposite direction.” E.F. Schumacher
Thank You! Timo Elliott VP, Global innovation Evangelist Timo.Elliott@sap.com @timoelliott timoelliott.com/docs/UKISUG_top_analytic_trends.zip