'

Spotify’s Music Recommendations Lambda Architecture

Понравилась презентация – покажи это...





Слайд 0

Spotify’s Music Recommendations Lambda Architecture Esh Kumar @eshvk Emily Samuels @emilymsa


Слайд 1

Overview Why Lambda? Use Case: Discover Recommendations Batch Architecture Real-time Architecture Challenges Future Work


Слайд 2

Why Lambda? 1 new user every 3 seconds. Contextual, time based recs more & more important


Слайд 3

Discover Recs


Слайд 4

The Discover Page Algorithmically generated fresh recs for users.


Слайд 5

The Discover Batch Pipeline


Слайд 6

Machine Learning Deep Dive


Слайд 7

Word2Vec Words with similar contexts have similar meaning


Слайд 8

Word2Vec King – Man + Woman = Queen


Слайд 9

Annoy Approximate Nearest Neighbors Oh Yeah! https://github.com/spotify/annoy


Слайд 10

Batch Architecture Strengths Recs based on complete user history Weakness User vector generation time increasing with no. users. Not reflective of current mood.


Слайд 11

Intro to Storm


Слайд 12

Storm Distributed real-time computation system


Слайд 13

Storm @ Spotify


Слайд 14

Real-time Architecture


Слайд 15

Workers die -> Cascading JVM Process death Memcache flakiness Cassandra JVM problems due to write/overwrite pattern Challenges


Слайд 16

Future/Ongoing Work Simplify the topology Keep listens for 24 hours Ongoing work on other real time personalization features.


Слайд 17

Questions Esh Kumar eshvk@spotify.com Emily Samuels esamuels@spotify.com


×

HTML:





Ссылка: