If you like this presentation – show it...
4 Layers Everyone Must Know
There is still so much confusion surrounding Big Data. I thought it might help to clarify the 4 key layers of a big data system - i.e. the different stages the data itself has to pass through on its journey from raw statistic or snippet of unstructured data (for example, social media post) to actionable insight. © 2014 Bernard Marr, Advanced Performance Institute, BWMC Ltd. All rights reserved.
The whole point of a big data strategy is to develop a system which moves data along this path – raw data to actionable insights. Here, I will attempt to define the basic layers you will need to have in place, if you are getting to grips with how big data could help your business. © 2014 Bernard Marr, Advanced Performance Institute, BWMC Ltd. All rights reserved.
Although people have come up with different names for these layers, as we’re charting a brave new world where little is set in stone, I think this is the simplest and most accurate breakdown: © 2014 Bernard Marr, Advanced Performance Institute, BWMC Ltd. All rights reserved.
Data sources layer This is where the data arrives at your organization. It includes everything from your sales records, customer database, feedback, social media channels, marketing list, email archives and any data gleaned from monitoring or measuring aspects of your operations. One of the first steps in setting up a data strategy is assessing what you have here, and measuring it against what you need to answer the critical questions you want help with. You might have everything you need already, or you might need to establish new sources. © 2014 Bernard Marr, Advanced Performance Institute, BWMC Ltd. All rights reserved.
Data storage layer This is where your Big Data lives, once it is gathered from your sources. As the volume of data generated and stored by companies has started to explode, sophisticated but accessible systems and tools have been developed – such as Apache Hadoop DFS (distributed file system), or Google File System, to help with this task. As well as a system for storing data that your computer system will understand (the file system) you will need a system for organizing and categorizing it in a way that people will understand – the database. Hadoop has its own, known as HBase, but others including Amazon’s DynamoDB, MongoDB and Cassandra (used by Facebook), all based on the NoSQL architecture, are popular too. © 2014 Bernard Marr, Advanced Performance Institute, BWMC Ltd. All rights reserved.
Data processing/ analysis layer When you want to use the data you have stored to find out something useful, you will need to process and analyze it. A common method is by using a MapReduce tool. Essentially, this is used to select the elements of the data that you want to analyze, and putting it into a format from which insights can be gleaned. If you are a large organization which has invested in its own data analytics team, they will form a part of this layer, too. They will employ tools such as Apache PIG or HIVE to query the data, and might use automated pattern recognition tools to determine trends, as well as drawing their conclusions from manual analysis. © 2014 Bernard Marr, Advanced Performance Institute, BWMC Ltd. All rights reserved.
Data output layer This is how the insights gleaned through the analysis is passed on to the people who can take action to benefit from them. Clear and concise communication (particularly if your decision-makers don’t have a background in statistics) is essential, and this output can take the form of reports, charts, figures and key recommendations. Ultimately, your Big Data system’s main task is to show, at this stage of the process, how measurable improvement in at least one KPI that can be achieved by taking action based on the analysis you have carried out. © 2014 Bernard Marr, Advanced Performance Institute, BWMC Ltd. All rights reserved.
If you set up a system which works through all those stages to arrive at this destination, then congratulations! You’re in Big Data. And hopefully, ready to start reaping the benefits! © 2014 Bernard Marr, Advanced Performance Institute, BWMC Ltd. All rights reserved.
For more on the topic, check out my LinkedIn Influencer blog: © 2014 Bernard Marr, Advanced Performance Institute, BWMC Ltd. All rights reserved. BLOG
About This overview was put together by Bernard Marr, Founder and CEO of the Advanced Performance Institute. Bernard Mar is a bestselling business author, keynote speaker, strategic performance consultant, and analytics, KPI & Big Data guru. He helps companies with their big data strategies and trains teams to better leverage data in their companies. Read Bernard’s blogs: LinkedIn Influencer Blog Connect with Bernard on: LinkedIn Twitter Advanced Performance Institute © 2014 Bernard Marr, Advanced Performance Institute, BWMC Ltd. All rights reserved.
Other slideshares You can find my other slideshares here: Slideshares