How do Recommender Systems work?
In order to be effective, a Recommender System (RS) uses a combination of data and Machine Learning. Data represent the building blocks from which models derive and are therefore fundamental in the development of a recommendation engine: the more data there is, the more efficient and effective it will be in making relevant suggestions. Typically, the process followed by a recommendation engine has four stages: –
Collection
The first is that of data collection. It works both on implicit data, that is to say on the information collected by activities such as the search history, clicks, events that have affected the shopping cart, search log, history of orders and so on. The explicit data, that is on the information collected on customer’s input, such as reviews and ratings, preferences, likes, and product comments. Recommender Systems also use customer attribute data such as age, gender, geographic location, psychographic, interest and value demographics. The goal is to identify similar customers. Attributes relating to the products or their characteristics and the related tags are used instead to identify the similarities between different products.
Archive
The second phase is that of data archiving. It is an important phase that must take into account not only the current volume of data but also its quantitative and qualitative growth. For this reason, it is good to adopt sufficiently robust and scalable solutions to familiarise the dynamism of the scenario.
Analysis
The third phase of the process is represented by the analysis of the collected data. In order to be used, the data must therefore be deepened and examined, according to different approaches and methodologies. You can work on real time analysis, in which the data are processed in real-time, while they are created; with batch analysis, which provides periodic data analysis; with near-real-time analysis, in which the data, if not immediately needed, is processed in a few minutes instead of seconds.
Filtering
The fourth and final stage of the process is that of data filtering. It is at this moment that matrices, mathematical rules or formulas are applied to the data, based on the type of recommendation filter (collaborative, content-based or hybrid). It is from this last step that the actual recommendations derive.
Latest developments
Everything described so far belongs to the more traditional approach of Recommender Systems. However, one cannot ignore the latest trends in this market, which leverage the most recent developments in the field of Artificial Intelligence, Machine Learning, and Natural Language Processing. From these developments conversational recommender systems come to life in which users and the system can communicate dynamically through natural language interactions.
It is certainly a new and fundamental lever, since it offers unprecedented opportunities to obtain exact preferences of users but, at the same time, it has not yet reached full maturity. Because it entails quite a few challenges, both in relation to the clarification of questions in relation to the construction of conversational models and the evaluation and interpretation phases.
We are therefore in a very interesting and, as often it happens, a multidisciplinary research field because it simultaneously involves information retrieval techniques, Natural Language Recognition interpretation techniques and Human-Computer Interaction (HCI) methodologies that allow a new level of multi-shift dialogue with users, in terms of accuracy, prediction, and reliability.