AMFB: : Attention based multimodal Factorized Bilinear Pooling for multimodal Fake News Detection

Fake news is the information or stories that are intentionally created to deceive or mislead the readers. In recent times, Fake news detection has attracted the attention of researchers and practitioners due to its many-fold benefits, including bringing in preventive measures to tackle the dissemination of misinformation that could otherwise disturb the social fabrics. Social media in recent times are heavily loaded with multimedia news and information. People prefer online news reading and find it more informative and convenient if they have access to multimedia content in the forms of text, images, audio, and videos. In early studies, researchers have proposed several fake news detection mechanisms that mostly utilize the textual features and not proper to learn multimodal (textual + visual) shared representation.

To overcome these limitations, in this paper, we propose a multimodal fake news detection framework with appropriate multimodal feature fusion that leverages information from text and image and tries to maximize the correlation between them to get the efficient multimodal shared representation. We empirically show that text, when combined with the image, can improve the performance of the model. The model detects the post once it is introduced into the network in an early stage. At the early stage of a news post’s introduction into the network, the model takes the text and image of the post as input and decides whether this is fake or genuine. Since this model only analyzes news contents, It does not require any prior information regarding the user and network details. This framework has four different sub-modules viz. Attention Based Stacked Bidirectional Long Short Term Memory (ABS-BiLSTM) for textual feature representation, Attention Based Multilevel Convolutional Neural Network–Recurrent Neural Network (ABM-CNN–RNN) for visual feature extraction, multimodal Factorized Bilinear Pooling (MFB) for feature fusion and finally Multi-Layer Perceptron (MLP) for the classification. We perform experiments on two publicly available datasets, viz. Twitter and Weibo. Evaluation results show the efficacy of our proposed approach that performs significantly better compared to the state-of-the-art models. It shows to outperform the current state-of-the-art by approximately 10 points for the Twitter dataset. In contrast, the Weibo dataset achieves an overall better performance with balanced F1-scores between fake and real classes. Furthermore, the complexity of our proposed model is significantly lower than the state-of-the-art.