News recommendation systems (Neural Network)

1. Introduction

This paper discusses a recommendation system for news platforms. News recommendation system aims to give a better news list to the logged in user. Recommendation means promoting something for a person to be consumed or used. In the case of the news recommendation system, several news articles are recommended to specific users to be read. Better recommendations are needed in order to give better interest matches to each user, so individuals do not need to scroll the whole page filled with undesired content.

News platforms need to implement the recommendation system in return for the high supply of the incoming articles. News lists can be troublesome to dig into. In general, news recommendation systems consist of two categories which are content based recommendation system, and personalized recommendation system. Content recommendation system gives recommendations based on the opened article. On the other hand, personalized recommendation gives recommendation based on the user’s history that represents their interests. However, the combination of both approaches is also possible to give a more robust recommendation.

Content based recommendations often use the article’s texts to gain information using feature extraction. The extracted features can be used for classification to give recommendations. Personalized recommendation extracts user’s history for classification. Recommendations are given based on the interest of the user.

This paper is organized in the following order: Sect. 2 Problem Background discusses the problem of the news recommendation system in detail., Sect. 3 Approaches in the news recommendation system in several methods, and Sect. 4 The conclusion of this paper.        

2. Problem Background

Usage and implementation of recommendation systems in big tech companies has successfully provided more personalized and optimized content for the users. Youtube, Spotify, and Amazon are some of many companies that use recommendation systems in their platform to obtain more user conversion and user engagement. On the other hand, the use of recommendation systems for “news portals” hasn’t been as popular as other platforms with a lot of different challenges.

Some of the challenges that often found in the implementation of recommendation system in news are:

• User Profiling; Most news readers do not have an identifiable identity (anonymous) and only read parts of the existing news.

• Fast Growing Number of Items. Big number of new news popping up in a day.

• Accelerated Decay of Item’s Value. The topic of interest to news readers is news that has just emerged, this causes each news to have appeal and a short shelf life.

• User Preferences Shift. User interest often dynamically shift between topics. This process occurs relatively quick, and more frequent compared to other domains. Despite some user interests shifting over time, other long-term interests might also remain stable.

• Broad News Topic. The topic of news is very broad which makes it difficult to recommend articles according to the interests of readers

• Possibility of Cold Case Problem. Articles that are said to be good may experience a “cold-case” because the newly published article has no readers at all

 3. Approaches in news recommendation problem

 A. CNN and RNN

Park et al. (2017) made an experiment using real world news on NAVER Corp. news portal where the dataset was collected over a month long period (April 2017). The experiment was conducted by making their own model which combines RNN and CNN methodology. Session based RNN was to capture the dynamic changes in reader’s interest and to find out user history. Meanwhile CNN is used to capture long-term user preferences and personalize recommendation results.

The first step is session-based RNN retrieves newly published articles relevant to the user’s interest by entering the user’s current session into the network. Then the model was continued by history-based RNN which retrieves recently published articles that are relevant to the user’s short-term interest by entering the user’s history into the network (if available). Lastly, reorder the articles generated by the RNN by combining the similarities between long-term user preferences and candidate article categories. If the news category is not available, we use the category predicted by the CNN classification.

Fig 3. 1 Session-based and History-based Model Evaluation Results

The result and evaluation was divided into two sections, one focuses on recommendation results and methods, and the other one is to evaluate the personalization result. For the recommendation result evaluation, Session based with BPR loss metric for evaluation was the best in all measures. Despite that, history based RNN also results in relevant recommendations on long term interest even if the user reads different topics of articles recently. For the personalization result evaluation, the curve between user’s preferences and recommended articles results from session based RNN model was proportional but it is important to find the optimal metrics to achieve user satisfaction.

Fig 3.2 GNewsRec Architecture with GNN and LSTM (Hu et al., 2020)

B. Deep Attention Neural Network (DAN)

Zhu et al. (2019) combined CNN, RNN, and attention mechanism which is referred as Deep Attention Neural Network (DAN) for news recommendation. CNN is used to combine user interest features and RNN to capture sequential features of each click to generate news recommendations.

The dataset was retrieved from Addressa, which is an event-based news data set that includes anonymous users with the news articles they clicked on. In this research, the dataset are divided into two types:

1. Addressa-1week: Dataset for 1 week (from 1 January to 7 January 2017)
2. Addressa-10week: Dataset for 10 weeks (from January 1 to March 31, 2017)

The features used from the dataset are the sessionStart, sessionStop, userId, newsId, timestamp, title and profile of news as our selection for generating our datasets. Specifically we first string the sequence of news articles into sessions according to the timestamp for a particular user and divide the longer news sequences into shorter sequences. The proposed model consistently outperforms all baselines on both datasets, which is at least 3.19% on F1, 2.64% on AUC higher than other models such as LibFM and DMF.

C. GNN and LSTM

Hu et al. (2020) issued an article about a news recommendation system using a hybrid approach consisting of GNN and LSTM called GNewsRec. This model tries to capture both the long term and short term user’s interest. This paper aims to make a better personalized news recommendation not only based on the clicked news, but also user’s history data that shows their interest. The main problems that were aimed by Hu et al. (2020) is to capture the latent information of the history data, to capture less clicked news that actually can be used as a bridge article to another recommendation, and also to capture not only the long term interest but also the short term interests as well.

GNN is a Graph Neural Network that is commonly used for complex graph pattern detection. Graph was created using a heterogeneous graph consisting of users click, like, history view and others. After the graph is created GNN is used to learn the pattern of the created graph for a specific user. On the other hand LSTM collects the user’s clicks to detect their short term interests. The results of the LSTM are passed to the attention mechanism layer to help detect the similar events that have already happened before and recurred in the current situation. Both of the recommendation results are combined to give the click probability rate.

GNewsRec shows a promising result. It shows the accuracy of 81.16 with 1 week data and 78.62 for the 10 weeks. Compared to the models that were implemented using GNN, short-term interest, or topic, the GNewsRec with full implementation outperformed the rest.

D. Fuzzy Logic

Manoharan et al. (2020) used a fuzzy logic approach to predict the interests and categories of various users by analyzing the users’ implicit social media profiles. Dataset used to predict various categories of user interest are the number of Frequent Clicks (CF) and Specific Search Query (SSC) from implicit profiles of 25 users in 15 days of monitoring. CF data for certain categories was obtained using Directory.Mozilla.Org (DMOZ), and SSC data was determined using clustering technique. Meanwhile, dataset used as news recommendations to users is obtained from Facebook, Twitter, and News feed. The news taken is divided into five categories, namely business, sports, technology, entertainment, and politics.

Fig 3.3 Architecture of the news recommendation framework using MFIS (Manoharan et al., 2020)

Mamdani fuzzy inference system (MFIS) was utilized to predict user interest in five categories. One user can have several categories of interest. The following news articles are recommended according to the user’s interest category:

1. Two popular news articles were taken from Facebook along with title, description, category, date, time, and number of shares and likes.
2. Two popular (most liked) hashtag tweets with related news articles pulled from Twitter along with date, time and category.
3. Latest News feed article along with title, description, published date, and time.

The input membership function for CF and SSQ of a category will be classified into “Low”, “Medium”, and “High” with different degrees of fuzzy sets. The output membership function will be classified into “Not interested”, “Interested”, and “Highly-interested”, also with different degrees. The MFIS base rule is created using “if-else” fuzzy rules to represent the relationship between input and output. The experimental results show that the proposed approach (MFIS) achieves an overall 84.238% in the prediction of user interest categories.

E. Chameleon – Meta Architecture with CNN and LSTM

Moreira et al. (2018) published an article regarding the implementation of a news recommendation system using Chameleon – Meta Architecture based on user session. The main reason behind it was the difficulties of determining user profiles. Moreira et al. (2018) stated that the majority of news readers are anonymous and read only a few stories from the entire repository; this results in extreme levels of sparsity in the user-item matrix, as users usually have tracked very little information about their past behavior and interest. The proposed Chameleon – Meta Architecture has been modified with two complementary modules namely Article Content Representation (ACR) and NextArticle Recommendation (NAR). In a single user session, CNN algorithm will be used for the ACR module which focuses on extracting textual feature representation from the current article read by the user. These extracted features will later be transferred to the second module called NAR which utilizes the LSTM approach to provide the next article for the users. Results were collected over a 24-hour period on October 16, 2017 and evaluated using HR@5 and MRR@5 metrics. From the evaluation carried out, it shows that the proposed Chameleon architecture gets a higher conversion value for users and news readers as shown in Fig 3.5.

Fig 3.4 Chameleon – Meta Architecture with CNN and LSTM (Moreira et al., 2018)

Fig 3.5 Results and Evaluation of Proposed Chameleon Architecture (Moreira et al., 2018)

4. Conclusion

In this paper, some research has been conducted to address the news recommendation problem. Some models have been proposed to solve the news recommendation system over the past years. Recommendation system has been a very important aspect of the news platform. Its ability to give targeted content for users has been really helpful to reduce users’ searching time. Although not all models have the same environment, generally all models show the significance of the recommendation system to the news platform.

From the review, the recommendation system can be divided into two main categories which are the content based recommendation and personalized recommendation. Both of the approaches gave promising results. However, those different approaches should be used based on the needs and the platform’s concept. Moreover, there is no model better than the other, instead all of the models have their own unique concept and architecture that can be good for many different cases. On one hand, approaches without personalized concepts can be more simple and focused. On the other hand, extended models with the personalized feature can be more targeted and robust against the content based only models. In return, it requires more resources for storage and operations that can be a huge drawback for small news platforms.

 References

de Souza Pereira Moreira, G., Ferreira, F., & da Cunha, A. M. (2018, October). News session-based recommendations using deep neural networks. In Proceedings of the 3rd Workshop on Deep Learning for Recommender Systems (pp. 15-23).

Hu, L., Li, C., Shi, C., Yang, C., & Shao, C. (2020). Graph neural news recommendation with long-term and short-term interest modeling. Information Processing & Management, 57(2), 102142.

Manoharan, S., & Senthilkumar, R. (2020). An Intelligent Fuzzy Rule-Based Personalized News Recommendation Using Social Media Mining. Computational Intelligence and Neuroscience, 2020, 1–10. doi:10.1155/2020/3791541

Park, K., Lee, J., & Choi, J. (2017). Deep Neural Networks for News Recommendations. Proceedings of the 2017 ACM on Conference on Information and Knowledge Management  – CIKM  ’17. doi:10.1145/3132847.3133154

Zhu, Q., Zhou, X., Song, Z., Tan, J., & Guo, L. (2019). DAN: Deep Attention Neural Network for News Recommendation. Proceedings of the AAAI Conference on Artificial Intelligence, 33, 5973–5980. doi:10.1609/aaai.v33i01.33015973