There are many people out there that can feel the adrenaline slowly upping its quantities in their bodies, and the stress starting as we near the jolly time of Christmas. We are not all bound to spend fortunes on presents to celebrate the birth of Christ but, if you are and unsure what you should give your loved ones, then you have an excellent opportunity to try out implementing a recommender system.
If you have a friend whom you know likes books, or just looking for a book yourself, a way to approach it is by downloading one of the public datasets around with book ratings. For example, there is one collected with the book-crossings website. Using that and implementing a content-based recommender you could find similar books to the ones that your friend likes. The cool thing about content-based recommenders is that they don’t need ratings from users, so if you have descriptions of other books you can compare them with the descriptions of the books your friend liked.
So, finding the perfect present for a book-loving friend could only be few steps away:
- Get a list of what the friend likes (probably that can be found on goodreads.com)
- A data set with book descriptions
- (optional) My book to understand how to implement a content-based recommender (book)
You could also go more general and take Amazon’s product dataset, or use one with movies in which case I also describe other types of recommenders you can use, collaborative filtering, or learning to rank. I had much fun with the last one.
If you do implement a recommender, please write a comment and happy seasonal festivity, if you celebrate this one.
I am just back from Como, Italy where I attended RecSys2017. It was great to meet all the people in person whose work I have been reading and whose presentations I have YouTubed. And more importantly, it was great to talk about my work and get it verified by people working on similar problems. I am in the legal content business now, and I have to admit that I didn’t find anybody who was having the same type of content as us, but it has some of the same types of challenges as for example news sites battle with. A large part of my reason going to a conference like the RecSys, is also to polish off the reasons why I do what I do. To meet so many geeks and discuss all the nerdy details is a great way to charge my motivation for doing it.
RecSys is an academic conference, where aspiring researchers come to show off their research, improving accuracy in predicting ratings on a narrow list of datasets. But it also has an industry track, where the industry comes to show off how they do recommender systems in practice. I think that both parts are great, only I would wish that research is a little bit more geared towards running them online, and the industry being a bit more technical. But it is a huge strength that both industry and academia are present. And I hope that will continue.
At lunch, a group of us would meet under the trees of the beautiful Villa Erba, where we had lots of interesting talks about how we could make the research part more relevant for business’ – most of us being professionals. And we all agreed (stop me if I am wrong :)) it is a shame that all research is evaluated offline using metrics that most businesses (and researchers) actually agree that it doesn’t say anything about the quality of the recommender system. The result is that researchers are left with the task of optimizing machine learning algorithms using datasets, which are meant to be used for recommender systems, rather than actually doing recommender systems.
The metrics used are also a point to discuss because using precision and recall at K is not a good measurement of a recommendation, but it is the best that they/we have come up with. But if that is what we should measure the quality of research, and use it as a benchmark, we should also agree on the how to split the data into training and test set, how we should sort the ratings split by the users in the test set. I have seen several good talks at the conference about creating a framework for doing this, but it seems that it gets forgotten again as soon as the question round is finished.
Next year’s conference has a challenge sponsored by Spotify, which is great, and I look forward to playing with the data, I wish that they could also make a live service available for researchers to try out their algorithms live, and there by also focusing research on many of practical issues that are faced by a recommender systems engineer.
No matter what I hope to be there again at RecSys 2018.
For those of you who read my introduction of my book, but haven’t bought it (yet :)), there is a small excerpt of the first chapter over on Dzone.
It is part of the description of what evidence is for a recommender system.
Front page of Practical Recommender Systems
For a computer scientist like me, the world of IT is such an exciting place! Since I started at university, I have seen the creation of companies like Amazon and Google, and later Netflix. They were for sure lucky to be in the right place at the right time. But it was ingenuity that has kept them in the market. What they did is a long story, but what I find interesting is that they have taken large quantities of content and made it accessible to the masses.
One of the advantages of being an internet business is the fact that you are not limited by physical walls like traditional shops and your list of products can be close to never ending. If a physical store was truly so vast, customers would struggle to find anything and simply get lost. They would probably go to the shop next door, which has fewer products and buy things that are not exactly what they wanted, but are easily accessible.
Offering lots of content does not ensure success, not even if you have precisely what your users want. Often 20% of your content will produce 80% of your business, if you can match the rest of the 80% of the content with your users, you will have more happy users and more business. The problem of activating the last 80% of the content is called the long tail problem.
A way to enhance the accessibility to the content for the users is to add a recommender system to you site. This can attempt to predict what your customers want and serve it to them.
Implementation of Recommender systems is an intriguing task. The actual algorithms like collaborative or content-based filtering are just a small part of it. If you do not feed the algorithm with the right data, it will not produce anything worth looking at. Using user ratings will often not produce the results that users want. Looking at context is also often something worth thinking about. And when it is all implemented and running, how do you know that it is working, how do you measure improvements?
I never found a book answering these questions; I found lots of good books explaining how to implement the algorithms mentioned above, but never a book that described everything around as well. So I started working on one. It just came out in an early release at Manning
Go and have a look, the first chapter is free!