'

Spring XD

Понравилась презентация – покажи это...





Слайд 0

Spring XD Glenn Renfro grenfro @pivotal.io @CPPWFS


Слайд 1

420 Million Wearables 90% of enterprise data is unstructured 60-100 sensors in each car 22 Billion sensors by 2020 86% suspect data inaccuracy 30% revenue loss due to bad data quality 500 million tweets each day 2.3 Trillion GBs of each day Data Data Points: McKinsey, Twitter, Gartner, IBM


Слайд 2

Batch and Streaming often handled by multiple platforms Fragmented Big Data Ecosystem Not all data Hadoop bound


Слайд 3

“One stop shop for developing and deploying Big Data Applications” SPRING XD EXTREME DATA


Слайд 4

Batch and Streaming often handled by multiple platforms Fragmented Big Data Ecosystem Not all data Hadoop bound Portable on-prem, YARN, EC2, PCF, Mesos, Docker etc. Easy to Use, Extend and Integrate with other Technologies Built on proven Spring EAI and Batch projects (Volume, Velocity, Veracity, and Variety) Unified Stream and Batch Operations Hadoop Batch Workflow Orchestration Predictive Analytics and Model Scoring Spring XD to Rescue


Слайд 5


Слайд 6

Spring XD - 10,000 Foot View


Слайд 7

Streams


Слайд 8

Create a stream with http as a source and hdfs as a sink. The hdfs —rollover is set to a small value so that we can read the file on hdfs.


Слайд 9

Spring XD - Distributed Runtime Container State


Слайд 10


Слайд 11


Слайд 12

Spring XD - Analytics Counters and Gauges Simple & Field Value Counter (how many tweets for #java) Aggregate Counter (how many tweets for #java in the week/day/hr) Gauge & Rich Gauge (how many requests / minute?) Abstract API implemented in Redis in-memory Predictive Model Evaluation JPMML Is this transaction fraudulent? What group does this user belong to? Interoperable with R, Rattle, KNIME, RapidMiner, MADLib


Слайд 13

Jobs


Слайд 14

FILES Spring XD GemFire XD GemFire XD SPEED LAYER BATCH LAYER SERVING LAYER PCF - BOSH Service PCF - Apps MOBILE SENSORS SOCIAL


Слайд 15

Unified runtime for both Real-time and Batch use cases Scalable, Distributed and Fault Tolerant Runtime Increased Productivity through out-of-the-box components Closed Loop Analytics through online (stream) and offline (batch) data Swiss-army knife of data movement and data pipelines Repeatable ‘turnkey’ solution for next generation data-centric use cases


Слайд 16

Agility: Easy to Setup and Run Writing HTTP Data to HDFS …that simple! or or or


Слайд 17

Spring XD on YARN Spring XD Running on YARN!


Слайд 18

Even easier with PCF


Слайд 19

Natural Fit: Reactive Streaming Pipelines


Слайд 20

Deployment Manifest – Module Count http | doWork | hdfs http http doWork doWork doWork doWork hdfs hdfs hdfs stream deploy –name s1 --properties module.http.count=2, module.doWork.count=4, module.hdfs.count=3


Слайд 21

Deployment Manifest – Module Placement http | doWork | hdfs http http doWork doWork doWork doWork hdfs hdfs hdfs stream deploy –name s1 --properties module.http.count=2, module.doWork.count=4, module.hdfs.count=3, module.http.criteria = groups.contains(‘WEB’)


Слайд 22

Deployment Manifest – Data Partitioning http | doWork | hdfs http http doWork doWork doWork doWork hdfs hdfs hdfs stream deploy –name s1 --properties ... module.http.producer .partitionKeyExpression = payload.customerId doWork modules will always process the same set of customer IDs


Слайд 23

Learn More Project: http://projects.spring.io/spring-xd/ GitHub: https://github.com/spring-projects/spring-xd/ Wiki: https://github.com/spring-projects/spring-xd/wiki Samples: https://github.com/spring-projects/spring-xd-samples


Слайд 24


×

HTML:





Ссылка: