'

Dato Keynote

Понравилась презентация – покажи это...





Слайд 0


Слайд 1

The ML pipeline circa 2013 Data ML Algorithm My curve is better than your curve Write a paper


Слайд 2


Слайд 3

Movie Distribution Coupons Search Retail Disruptive companies differentiated by Networking INTELLIGENT APPLICATIONS Music Advertising Campaigning using Real Estate Machine Learning Human Resources Legal Advice Dating Wearables Taxis CRM


Слайд 4

Dato’s mission is to accelerate the creation of intelligent applications by making sophisticated machine learning as easy as “Hello world!”


Слайд 5

Since last year… •  Released 3 products GraphLab Create Dato Distributed Dato Predictive Services •  More than 10,000 downloads


Слайд 6

Since last year… Our customers…


Слайд 7

Demo: Intelligent application (Gift for Julia)


Слайд 8

Challenge today: Path from inspiration to production Inspiration Sophisticated ML Deploy Service Production Scale Prototyping People Data scientist Sophisticated ML is impractical •  Hard to match algo to app •  Algos trapped in paper Systems Production Elastic, scalable Scaling is costly •  Rewrite algo from scratch •  Expensive infrastructure Slow & expensive process Deployment: more costly infrastructure & time •  Build custom services & API •  Model quality deteriorates


Слайд 9

Sophisticated ML is impractical


Слайд 10

Inspiration for Intelligent Application Top down solution would be easiest but not possible: Application is innovative → no black box solution available ML development today And again Data Fine approach if it’s 2013 & I’m obsessed with Try again “my curve is better than your curve” Tune parameter (i.e., yet another solution for same old problem) Choose model or not primarily focused on accelerating creation of intelligent applications Create features Extract text Read data Forced to go bottoms up


Слайд 11

Inspiration for Intelligent Application If in 5 years all applications intelligent, ML needs: And again Try again Start from relevant, high-level, sophisticated ML building blocks Tune parameter Choose model ML done differently, Create features Extract text Let’s see how… Read data Quickly write code: combine, blend, understand, adapt, improve, optimize Forced to go bottoms up Data Don’t waste time on boring stuff, like parameter search or worry about specialized ML knowledge, like SGD


Слайд 12

Demo: Building an intelligent application with GraphLab Create (Restaurant recommender)


Слайд 13

High-level ML toolkits get started with 4 lines of code, then modify, blend, add yours… Recommender Image search Sentiment analysis Data matching import graphlab as gl Object Product Click Fraud detection data = gl.SFrame.read_csv('my_data.csv') detector sentiment prediction model = gl.recommender.create(data, user_id='user', Anomaly Document detection clustering Search ranking item_id='movie’, Forecasting target='rating') recommendations = model.recommend(k=5) Auto tagging Churn predictor User segmentation Data completion Summarization …


Слайд 14

Sophisticated machine learning made easy Create Intelligence Accelerants High-level ML toolkits AutoML tune params, model selection,… è so you can focus on creative parts Reusable features transferrable feature engineering è accuracy with less data & less effort


Слайд 15

Very hard! Usually: Simple models & lots of feature engineering Krishna’s talk tomorrow @9:10am: auto feature engineering Next: Transfer learning can provide complex models with less work & less data Feature engineering Understand & scale complex models Representation challenge Modeling challenge Need for lots of labeled data Data challenge Makes ML hard


Слайд 16

Example: Deep learning in computer vision (or the deep devil is in the deep details)


Слайд 17

Image features •  Features = local detectors o  o  Combined to make prediction (in reality, features are more low-level) Nose Eye Eye Mouth Face!


Слайд 18

Many hand create features exist… Computer$vision$features$ SIFT$ Spin$image$ HoG$ RIFT$ Textons$ GLOH$ Slide$Credit:$Honglak$Lee$


Слайд 19

Standard image classification approach Input Extract features Use simple classifier e.g., logistic regression, SVMs Computer$vision$features$ SIFT$ Spin$image$ HoG$ RIFT$ Textons$ GLOH$ Slide$Credit:$Honglak$Lee$ Car?


Слайд 20

Many hand create features exist… Computer$vision$features$ SIFT$ Spin$image$ HoG$ RIFT$ Textons$ GLOH$ Slide$Credit:$Honglak$Lee$ … but very painful to design


Слайд 21

Deep neural networks implicitly learn features Deep Learning = Learning Hierarchical Representations Y LeCun MA Ranzato It's deep if it has more than one stage of non-linear feature transformation Low-Level Mid-Level High-Level Trainable Feature Feature Feature Classifier Color & edge detectors Geometric detectors Car-specific detectors Feature visualization of convolutional net trained on ImageNet from [Zeiler & Fergus 2013] Each layer learns features, at different levels of abstraction


Слайд 22

Deep learning has yielded exciting accuracy, e.g., Krizhevsky et al. won 2012 ImageNet competition impressively Huge gain


Слайд 23

Challenges of deep learning


Слайд 24

Deep learning workflow Training set 80% Learn deep neural net model Lots of labeled data 20% Validation set Validate


Слайд 25

Many tricks needed to work well… Different types of layers, connections,… needed for high accuracy Krizhevsky et al. ‘12


Слайд 26

GraphLab Create adds deep features Deep learning + Transfer learning


Слайд 27

Change image classification approach? Input Extract features Use simple classifier e.g., logistic regression, SVMs Computer$vision$features$ Can we learn features from data, even when we don’t have data or time? SIFT$ Spin$image$ HoG$ RIFT$ Textons$ GLOH$ Slide$Credit:$Honglak$Lee$ Car?


Слайд 28

Transfer learning: Use data from one domain to help learn on another Old idea, explored for deep learning by Donahue et al. ’14 Lots of data: vs. Some data: Learn neural net Neural net as feature extractor + Simple classifier Great accuracy on cat v. dog Great accuracy on 101 categories


Слайд 29

What’s learned in a neural net vs. Neural net trained for Task 1: cat vs. dog More generic Can be used as feature extractor Very specific to Task 1 Should be ignored for other tasks


Слайд 30

Transfer learning in more detail… For Task 2, predicting 101 categories, learn only end part Use simple classifier e.g., logistic regression, SVMs Class? Neural net trained for Task 1: cat vs. dog More generic Can be used as feature extractor Keep weights fixed! Very specific to Task 1 Should be ignored for other tasks


Слайд 31

Transfer learning with deep features Training set Some labeled data Extract features with neural net trained on different task 80% 20% Validation set Learn simple model Validate Deep learning tutorial tomorrow, 4pm! Deploy in production


Слайд 32

Demo: The power of deep features, a.k.a., transfer learning (Shoes, please)


Слайд 33

How general are deep features? Talk by founder, Jason Gates, tomorrow 9:40am


Слайд 34

GraphLab Create includes easy to use, deep learning on multi-GPUs graphlab.deeplearning.create(data,target=label') Deep learning in 1 line of code Average Pooling Layer Convolution Layer Sigmoid Layer Dropout Layer SoftMax Layer Flatten Layer SoftPlus Layer Full Connection Layer Sum Pooling Layer Max Pooling Layer You can also open the box and add your own layers Rectified Linear Layer Tanh Layer Deep learning tutorial tomorrow, 4pm!


Слайд 35

Digit recognition benchmark Test Error 0.85% 0.80% 0.75% 0.70% H2O.ai: 10 machines/80 cores 0.65% 0.60% 0 10 GraphLab Create 5 4 min on 4 GPUs Hours 15


Слайд 36

Tables, graphs, text, images High-level ML toolkits (4 lines of code gets you started) deep learning, recommender, product reviews, data matching, sentiment, image search, churn, click prediction, customer segmentation, fraud detection,… GraphLab Create for intelligent applications Auto Feature Engineering (automate, achieve high accuracy) . deep & reusable features . data transformation pipelines . kernels & hashing, encodings Scalable viz for TBs of data Including Matplotlib at scale AutoML (automate to focus on creativity) . parameter search . model selection . algorithm selection . distributed


Слайд 37

Anthony Goldbloom Founder & CEO Debora Donato Sr. Director of Personalization & Principal Data Scientist


Слайд 38

Native Advertising – The opportunity of making ads valuable For  the   publishers   For  the  users  


Слайд 39

Bad advertising does not work for anybody


Слайд 40

The data: •  400k raw html pages containing: o  text, images, links, and well, everything web pages have The task: •  predict which pages are organic and which are sponsored advertising When: •  starts August 1! The Prize •  Fame!!! •  Knowledge!!! •  $10,000


Слайд 41

A lot of effort in Kaggle competitions involves running many experiments… …can get slow L


Слайд 42

Sophisticated machine learning made scalable Data Structures to Create Intelligence SFrame ❤️ all ML tools SGraph


Слайд 43

Data frames user movie rating When you choose a data frame, have your application in mind SFrame is optimized for ML ML has specific data access patterns, we make them fast, really fast (Columnar transformations, creating new features, iterations,…)


Слайд 44

SFrame: Scalable data frame optimized for ML Never run out of memory Sharded, compressed, out-of-core, columnar Arbitrary lambda transformations, joins,… from Python user movie rating … Same code Large data on one machine? Limited RAM è Must use disk (out-of-core computation) Talk tomorrow with details: Yucheng @11am


Слайд 45

Opportunity for Out-of-Core ML Opportunity for big data on 1 machine Capacity Throughput 10 TB 1 TB 0.1 TB 0.1 GB/s 0.5 GB/s 1 GB/s For sequential reads only! Random access very slow Out-of-core ML opportunity is huge Fast, but significantly limits data size Usual design → Lots of random access → Slow GraphChi early example SFrame data frame for ML Design to maximize sequential access for ML algo patterns


Слайд 46

Demo: 10TBs of data on one machine!


Слайд 47

SFrame ❤️ all ML


Слайд 48

scikit-learn is awesome, but... Out of RAM Numpy in memory only Runtime (s) 4000 3000 scikit-learn + Numpy 2000 1000 0 0 Airline Delay Dataset, SGDLinearClassifier 50 100 150 200 250 Millions of Rows 300 350 400


Слайд 49

Demo: 10TBs of data on one machine redux


Слайд 50

Numpy Automatically Backed by Sframes → Scale many Python packages (scikit-learn, scipy,…) import graphlab.numpy Scalable numpy activation successful Runtime (s) Out of RAM Graphlab Create 4000 + Caveats apply scikit-learn scikit-learn 3000 - Scales most memory-bound sklearn algorithms + + Numpy Numpy 2000 - Sequential access highly preferred for performance 1000 0 0 Airline Delay Dataset, SGDLinearClassifier 50 100 150 200 250 Millions of Rows 300 350 400


Слайд 51

ML is not just about tables


Слайд 52

ML pipelines combine multiple data types Hyperlinks Raw Wikipedia <<//>> </> XML PageRank Title PR Text Table Title Body Top 20 Pages Term-Doc Graph Topic Model (LDA) Word Topics Word Topic


Слайд 53

SGraph Graph processing & analytics Backed by SFrame Out-of-core & scalable Neighborhoods, paths, graph algos, community detection, label propagation, ML on graphs, viz, …


Слайд 54

Performance of SGraph Connected components in Twitter graph 1 machine GraphLabSGraph Create 70 sec GraphX 251 sec Giraph 16 machines 200 sec Spark 2,128 sec 0 Source(s): Gonzalez et. al. (OSDI 2014) Twitter: 41 million Nodes, 1.4 billion Edges 55   750 1500 2250


Слайд 55

Pagerank on Common Crawl Graph Minutes per iteration 3.5 billion Nodes and 128 billion Edges 10 8 6 4 2 0 1 machine 16 CPUs, 1 SSD


Слайд 56

We ❤️ open source


Слайд 57

Tables, graphs, text, images SFrame & SGraph Open-source ❤️ BSD license (August) Optimized out-of-core computation for ML High Performance Optimized for ML 1 machine can handle: . Columnar transformation . Create features . Iterators . Filter, join, group-by, aggregate . User-defined functions . Easily extended through SDK TBs of data 100s Billions of edges


Слайд 58

Sophisticated machine learning made distributed Create Intelligence on Huge Data Distributed machine learning Your big data infrastructure (cloud, hadoop, spark,..)


Слайд 59

Pagerank on Common Crawl Graph Minutes per iteration 3.5 billion Nodes and 128 billion Edges 10 8 6 4 2 0 45 secs/iteration 3B edges/sec 1 machine 16 machines 16 CPUs 256 CPUs


Слайд 60

Criteo Terabyte Click Prediction 4000 4.4 Billion Rows 13 Features 3000 Runtime ½ TB of data 3630s 3500 2500 2000 1500 225s 1000 500 0 0 4 8 #Machines 12 16


Слайд 61

Single machine ML code Same code, distributed ML gl.deploy.ec2_cluster.load(’s3://…') gl.deploy.spark_cluster.load(’hdfs://…') c = gl.deploy.hadoop_cluster.load(’hdfs://…') gl.set_distributed_execution_environment(c) import graphlab as gl data = gl.SFrame.read_csv(’s3://…') model = gl.classifier.create(data, target=’click’)


Слайд 62

Dato machine learning platform GraphLab Create ML Toolkits Canvas Reusable Features AutoML Inspiration Sophisticated ML Machine Learning In Production Scale GraphLab Create SFrame SGraph Create Engine Dato Distributed Distributed ML Job Mgmt Distributed Engine Optimized for ML performance, for any data size, on any infrastructure


Слайд 63

Machine Learning in Production Easily serve live predictions Deployment


Слайд 64

Deploying ML models Exciting new deep learning model. Dato Predictive Services HowRESTis this long API! Igoing to take?! will be done today. It’s accurate! Data Scientists Deployment Engineers


Слайд 65

Machine Learning in Production Easily serve live predictions Deployment Measuring quality of deployed models Choosing between deployed models Management Evaluation Tracking model operations Monitoring Talk tomorrow with details: Alice & Rajat @1:45pm


Слайд 66

Dato machine learning platform GraphLab Create ML Toolkits Canvas Reusable Features AutoML Inspiration Sophisticated ML Deployment Evaluation Scale GraphLab Create SFrame SGraph Create Engine Management Dato Distributed Distributed ML Job Mgmt Distributed Engine Monitoring Optimized for ML performance, for any data size, on any infrastructure


Слайд 67

Dato machine learning platform Create of intelligent applications faster & cheaper GraphLab Create ML Toolkits Dato Predictive Services Canvas Reusable Features AutoML REST Client Robust, Elastic Direct Model Mgmt Sophisticated ML Inspiration Deploy Service Production Scale GraphLab Create SFrame SGraph Create Engine Dato Distributed Distributed ML Job Mgmt Distributed Engine Optimized for ML performance, for any data size, on any infrastructure


Слайд 68

My curve is better than your curve INTELLIGENT APPLICATIONS Phase transition of machine learning are disrupting markets Accelerate this process > pip install graphlab-create @guestrin jobs@dato.com


Слайд 69


×

HTML:





Ссылка: