Google changed the YouTube algorithm in 2016 to a new one made by their Artificial intelligence company Deep Mind. It uses deep learning neural networks to optimize for the user’s expected watch time. The algorithm has two steps, both of which use multi-layer neural networks. Recommender Systems are among the most common forms of Machine Learning that users will encounter, whether they’re aware of it or not. It powers curated timelines on Facebook and Twitter, and “suggested videos” on YouTube.
How does the algorithm work?
The recommendation algorithm of YouTube, that plays an important part in feeding relevant content to its users was the brainchild project of The Google Brain team, which is currently driven by their deep learning systems. Basically, the algorithm consists of two neural networks. The first neural network works on candidate generation which means the network utilises the users’ watch history and applies the concept of collaborative filtering technique to suggest similar videos based on watch history. The best algorithms usually are obtained by A/B testing, which uses two variables– for example, two versions of the same web-page, to determine which one fetches more views. This way, the user experience is improved.
YouTube represents one of the largest scale and most sophisticated industrial recommendation systems in existence. In this paper, we describe the system at a high level and focus on the dramatic performance improvements brought by deep learning. The paper is split according to the classic two-stage information retrieval dichotomy: first, we detail a deep candidate generation model and then describe a separate deep ranking model. We also provide practical lessons and insights derived from designing, iterating and maintaining a massive recommendation system with enormous user-facing impact.
