gGe8PWZi

1\documentclass[11pt]{article}
2
3\usepackage{graphicx}
4\usepackage[utf8]{inputenc}
5\usepackage[T1]{fontenc}
6
7\usepackage{float}
8\usepackage{mathpazo} 
9
10
11\usepackage[a4paper, margin=1in]{geometry}
12
13\begin{document}
14\begin{titlepage}
15	\newcommand{\HRule}{\rule{\linewidth}{0.5mm}} 
16	\center
17	\textsc{\LARGE University of Dhaka}\\[1.5cm]
18	\includegraphics[width=0.2\textwidth]{logo.png}\\[1cm]
19	\textsc{\LARGE Department of Computer Science \& Engineering}\\[1.5cm]
20	\HRule\\[0.4cm]
21	
22	{\huge\bfseries Project Title: A Multimodal Approach for Fake News Detection}\\[0.4cm] 
23	\HRule\\[1.5cm]
24	\begin{minipage}{0.4\textwidth}
25		\begin{flushleft}
26			\large
27			\textit{Author}\\
28			{Amar Debnath (Roll: 23)}\\
29			{Email:amar.csedu@gmail.com}
30			{Signature:                            }\\
31			\bigskip
32			\bigskip
33			{Redoan Rahman (Roll: 41)}\\
34			{Email:redoanrahman744@ gmail.com}\\
35			{Signature:                            }\\
36		\end{flushleft}
37	\end{minipage}
38	\begin{minipage}{0.4\textwidth}
39		\begin{flushright}
40    		\begin{flushleft}
41    			\large
42    			\textit{Supervisor}\\
43    			{Md. Mofijul Islam}\\
44    			{Lecturer}\\
45    			{Department of Computer Science and Engineering}\\
46    			{University of Dhaka}\\
47    			{Email: akash@cse.du.ac.bd}\\
48    			{Signature:                            }\\
49		        \bigskip
50			    \bigskip
51    		\end{flushleft}
52		\end{flushright}
53	\end{minipage}
54	\vfill\vfill\vfill
55	{\large\today}
56	\vfill
57	
58\end{titlepage}
59
60\begin{samepage}
61
62\tableofcontents
63\newpage
64\section{Abstract}
65Automatic fake news detection is a challenging problem in deception detection, and it has tremendous real-world political and social impacts. In most available researches on this field, the researchers concentrate on using a statement for fact-checking. A large portion of data, which is the evidence or justification of that is left ignored due to the fact that the evidences are not in machine readable form. In this paper we provide a data-set containing claims and their justifying evidences along with other data and a learning system that can use provided evidence to justify a claim.
66\section{Introduction}
67In this modern age of technology and Intelligent systems, humans have developed many techniques to cope with the problems of their surroundings. As many problems are being solved, more new problems are rising. Social media for news consumption is a double-edged sword. On the one hand, its low cost, easy access, and rapid dissemination of information lead people to seek out and consume news from social media. On the other hand, it enables the wide spread of “fake news", i.e., low quality news with intentionally false information. The extensive spread of fake news has the potential for extremely negative impacts on individuals and society. The extensive spread of fake news can have a serious negative impact on individuals and society. 
68\begin{itemize}
69    \item First, fake news can break the authenticity balance of the news ecosystem.
70    \item Second, fake news intentionally persuades consumers to accept biased or false beliefs. Fake news is usually manipulated by propagandists to convey political messages or influence.
71    \item Third, fake news changes the way people interpret and respond to real news. For example, some fake news was just created to trigger people’s distrust and make them confused, impeding their abilities to differentiate what is true from what is not.
72    \item To help mitigate the negative effects caused by fake news both to benefit the public and the news ecosystem It’s critical that we develop methods to automatically detect fake news on social media. 
73    \item During United States Election of 2016, there were 156 fake news that were identified later, among which 115 were pro-trump and 41 of them were pro-clinton\cite{fake}.Therefore, fake news detection on social media has recently become an emerging research that is attracting tremendous attention.
74\end{itemize}
75\subsection{Problem Definition}
76Several projects and researches are being conducted and it has grown to be a trending topic to detect fake news and reviews as well as trying to increase the accuracy of the existing models. Because of the possible mischief fake news causes to social media, It’s quite an urgent situation to deal with this issue in this modern age. Automatic fake news detection is a challenging problem in deception detection, and it has tremendous real-world political and social impacts. However, statistical approaches to combating fake news has been dramatically limited by the lack of labeled benchmark data sets.
77\bigskip
78\subsection{Motivation}
79This era is driven by the power of Artificial Intelligence which is making the lives of humans more convenient. But it’s still a long way to go before our network can identify all the malicious or misleading news. If we can look around our society we can see that this false news is not only confusing people but also doing harm as well. For example: last year, a man carried an AR-15 rifle and walked in a Washington DC Pizzeria, because he recently read online that “this pizzeria was harboring young children as sex slaves as part of a child abuse ring led by Hillary Clinton”\cite{pizzaria}. The man was later arrested by police, and he was charged for firing an assault rifle in the restaurant. Besides some report also shows that Russia has created fake accounts and social bots to spread false stories\cite{TIME}. So we want to work on improving the fake news detection system. The main contributions of our work are given below:
80\begin{itemize}
81    \item In an age where information is power, misinformation can be very harmful. Simple, small sized fake news can cause uproar, panic and many other problems on a state, national, even on an international scale. So improving the detection technique of fake news even marginally can contribute greatly to benefit of mankind.
82    \item  The researches being conducted on this field are mostly focusing on the news title. But we want to concentrate on the actual news or the justification of the news.
83    \item We aim to create a multimodal data-set containing different types of data. For example: evidence justifying a claim, speaker profile, source authenticity, sentiment etc.  
84\end{itemize}
85\subsection{Objective}
86For designing an intelligent and accurate fake news detection system we must first find a moderate labeled data set to work on. We must experiment on that data set to evaluate our training model. After managing a labeled data set we will then develop and apply various machine learning algorithms to find a tolerable accuracy for our fake news detection system.
87
88\section{Related Works}
89Since it’s an emerging topic, There has been a lot of works related to fake review or fake news detection and judgment. Below we describe some of the papers that where many techniques and approaches are discussed.
90\subsection{Fake Review}
91Advertisers, marketers, and other stakeholders have motivation to produce fake positive user reviews for products they wish to promote or fake negative user reviews for products which they wish to disparage.In a fake user review, an actor will create a user account based on some marketing persona and post a user review purporting to be a real person with the traits of the persona.This is a misuse of the user review system, which universally only invite reviews from typical users and not paid fake personalities.
92\subsubsection{Behavioral Analysis of Review Fraud: Linking Malicious Crowd sourcing to Amazon and Beyond\cite{behaivour}}
93\par User reviews are a cornerstone of how we make decisions. From deciding what movies to view, products to purchase, restaurants to patronize, and even doctors to visit, user review aggregators like Amazon, Netflix, and Yelp shape our experiences. And yet, these reviews are vulnerable to manipulation. This manipulation threatens to degrade trust in these online platforms and in their products and services
94\bigskip
95\paragraph{Proposed Solution}
96In this paper the authors focus on tasks posted to a single crowd sourcing site – Rapid Workers – that target Amazon. Typically, tasks on these sites pay workers from $0.10 to $1.50 per task, where a single target (e.g., a product on Amazon) may be subject to dozens of fake reviews launched from these crowd sourcing sites. The data set they made contains the following information: product ID, review ID, reviewer ID, review title, review content, rating, time-stamp and “verified purchase” flag. In total, they identified 5,200 unique reviewers and 350,000 unique reviews. They consider a reviewer to be a fraudulent reviewer if they have reviewed two or more products that have been targeted by a crowd sourcing effort and non-fraudulent otherwise. Then they made assumption that fraud reviewers tend to stay with their review style and make a lot of similar comment and word combinations. Keeping this assumption, they then find the Jaccard similarities and made proper calculations to justify their assumption.
97\bigskip
98\paragraph{Future Works}
99This behavioral approach for the reviewers open a new path for the novel fake review detection architecture. This observation can be applied not only to online stores like amazon, but also various app stores and blogs as well. Besides, observing their linguistic evolution more carefully, we can find more insights into the behaviors of the fraud reviewers.
100\subsubsection{Classification of Fake Product Ratings Using a Time line Based Approach\cite{timeline}}
101\par Detection of fake review and reviewers is currently a challenging problem in cyberspace. It is challenging primarily due to the dynamic nature of the methodology used to fake the review. There are several aspects to be considered when analyzing reviews to classify them effective into genuine and fake. Sentiment analysis, opinion mining and intend mining are fields of research that try to accomplish the goal through Natural Language Processing of the text content of the review.
102\paragraph{Proposed Solution}
103The three primary factors used in the proposed strategy to classify fake and genuine review
104ratings presented in this paper include: Time span between the first and last review, Average review rating over the time span, Number of review ratings. The Amazon Product Data set was used for the research analysis presented in this subsection. The data set was obtained from SNAP - Stanford Network Analysis Platform, a general purpose network analysis and graph mining library\cite{graph}.They then applied their custom algorithm which involves - Sorting of the time stamps in ascending order, Determination of the inter review period in Unix time by calculating the difference between the time periods of two consecutive ratings, Conversion of the time period of each rating into seconds, minutes and then days, thereby attaining the time interval between each rating in days, Calculation of the average of all the ratings for each time-period, Plotting of a graph with the information deduced, with average ratings for each of the time intervals and Calculation of the average of all ratings for a product. Using their algorithm they calculated the estimated average rating of the products. 
105\paragraph{Future Works}
106With a growing trend in online shopping there is a need currently to provide an authentic environment for analyzing product reviews and ratings. This research paper expounds a methodology to identify fake ratings among genuine ones over time. This can be used to implicitly identify fake reviewers in cyberspace. A simple classification tool to identify the product specific point on the time line has also been used in the research work presented in this paper. The proposed fake rating filter therefore helps to tag possible fake reviewers, besides indicating the optimal period for the rating assessment for each product.
107\bigskip{}
108\subsection{Fake News}
109\subsubsection{“Liar, Liar Pants on Fire”: A New Benchmark Data set for Fake News Detection\cite{LIAR}}
110\par Automatic fake news detection is a challenging problem in deception detection, and it has tremendous real-world political and social impacts. However, statistical approaches to combating fake news has been dramatically limited by the lack of labeled benchmark data sets. There are a lot of public data set of political or other news but most of them are not properly labeled or have much low content for the machine learning algorithms to trains. 
111\paragraph{Proposed Solution}
112In this paper A new data set is proposed “LIAR” a new, publicly
113available dataset for fake news detection. The authors collected a decade-long, 12.8K manually labeled short statements in various contexts from POLITIFACT.COM, which provides detailed analysis report and links to source documents for each case. This dataset can be used for fact-checking research as well. Notably, this new data set is an order of magnitude larger than previously largest public fake news data sets of similar type. Empirically, they investigate automatic fake news detection based on surface-level linguistic patterns. They have designed a novel, hybrid convolutional neural network to integrate meta data with text. They showed that this hybrid approach can improve a text-only deep learning model. 
114\paragraph{Future Works}
115LIAR’s authentic, real-world short statements from various contexts with diverse speakers also make the research on developing broad-coverage fake news detector possible. When combining meta-data with text, significant improvements can be achieved for fine-grained fake news detection. Given the detailed analysis report and links to source documents in this data set, it is also possible to explore the task of automatic fact-checking over knowledge base in the future. The corpus can also be used for stance classification, argument mining, topic modeling, rumor detection, and political NLP research. Besides, The accuracy involving the features of the data set is not too high to apply in real life situations, so they data set can be modified or incorporated in a way such that the accuracy of the fake news detection can be improved.
116\subsubsection{Fake News Detection Through Multi-Perspective Speaker Profiles\cite{notunLIAR}}
117\par The data set LIAR\cite{LIAR} given by professor William Yang Wang is an astounding empirical labeled dataset which helps for the fake news detection research. But the features given in the data set is not enough to make a promising accuracy for detection. This paper proposes a novel method to incorporate speaker profiles into an attention based LSTM model for fake news detection. By adding more features, the performance metric of the learning model can be improved. Despite having a perfect labeled characteristic, the LIAR data set is still insufficient to help establish a fake news detection system which can be applied to real world. 
118
119\paragraph{Proposed Solution}
120The solution of this paper involves incorporating speaker profiles into an LSTM model for fake news detection. Speaker profiles contribute to the model in two ways. One is to include them in the attention model. The other includes them as additional input data. By adding speaker profiles such as party affiliation, speaker title, location and credit history, the proposed model outperforms the state-of-the-art method by 14.5\% in accuracy using a benchmark fake news detection data set. This proves that speaker profiles provide valuable information to validate the credibility of news articles. evaluation is performed using the LIAR data set by Wang\cite{notunLIAR}. The maximum accuracy gained by LIAR data set is almost 27\%, and the maximum accuracy of the proposed LSTM model in this paper is around 41.5\%.
121\bigskip
122\paragraph{Future Works}
123Augmenting speaker profiles have proven to improve the learning models, this not only adds extra accuracy but opens more possibilities that we can corporate other important characteristics of speakers and other attributes as well. Besides this hybrid LSTM model can be used in other applications like calculating the ratings of products and whether a person's speech can be trusted or not.  
124
125\subsubsection{FEVER: a large-scale dataset for Fact Extraction and VERification\cite{FEVER}}
126\par In most of the predecessor works on claim verification such as “Liar, Liar Pants on Fire”: A New Benchmark Data set for Fake News Detection"\cite{LIAR} and "Fake News Detection Through Multi-Perspective Speaker Profiles"\cite{notunLIAR}, he evidences or justifications were not used because the justifications provided by the journalists are not in machine readable forms. Even in other data-sets such as Emergent\cite{emergent}, the classification of the claim was done with respect to article headline instead of the complete article. In this paper a new data-set is presented for claim verification, FEVER: Fact Extraction and VERification. It consists of 185,445 claims manually verified against the introductory sections of Wikipedia pages and classified as SUPPORTED, REFUTED or NOTENOUGHINFO. For the first two classes, systems and annotators need to also return the combination of sentences forming the necessary evidence supporting or refuting the claim. 
127
128\paragraph{Proposed Solution}
129\par The solution proposed in this paper is textual claims against textual clarification. The first part of the solution is fact extraction and verification of data-set. The data-set was constructed in two stages. First stage is Claim Generation where the information was extracted from Wikipedia and claims were generated from it by one group of annotators. Second stage is Claim Labeling which involved the claims being labeled SUPPORTED, REFUTED or NOTENOUGHINFO by a different group of annotators.
130
131\par The next part of the solution is the development of the system which can retrieve the necessary documents, select sentences from the documents and can recognize textual entailment of the documents. In the proposed solution, the document retrieval
132component from the DrQA system \cite{DrQA} is used which returns the k nearest documents for a query using cosine similarity between binned unigram and bigram Term Frequency-Inverse Document Frequency (TF-IDF) vectors. Sentence selection was done by ranking sentences based on TF-IDF similarity to the claim. For recognizing textual entailment, a multi-layer perceptron (MLP)\cite{MLP} with a single hidden layer which uses term frequencies and TF-IDF cosine similarity between the claim and evidence as features. Evaluating the state-of-the-art in RTE, we used a decomposable attention (DA)\cite{DA} model between the claim and the evidence passage. 
133
134\par The proposed solution yields an accuracy of 31.87\% when requiring correct evidence to be retrieved for claims SUPPORTED or REFUTED, and 50.91\% if the correctness of the evidence is ignored, both indicating the difficulty but also the feasibility of the task.
135
136\paragraph{Future Works}
137\par One use case for the FEVER dataset is claim extraction: generating short concise textual facts from longer encyclopedic texts. For sources like Wikipedia or news articles, the sentences can contain multiple individual claims, making them not only difficult to parse, but also hard to evaluate against evidence. During the construction on the FEVER dataset, we allowed for an extension of the task where simple claims can be extracted from multiple complex sentences.
138\par Also, the pipeline solution proposed is just one approach to the problem. There are other approaches which could yield similar or better results such as natural logic inference or question generation followed by a question answering model.
139
140\subsubsection{Fishing for Clickbaits in Social Images and Texts with Linguistically-Infused Neural Network Models\cite{clickbait}} 
141\par Clickbait posts (aka eye-catching headlines) are designed to lure readers into clicking associated links by exploiting curiosity with vague, exaggerated, sensational and misleading content. The intent behind clickbait messages varies from attention redirection to monetizing and traffic attraction. Earlier works on detecting clickbaits\cite{clickbase} uses handcrafted features, but in this paper representations from images and text were used to train neural network models. In addition, linguistic resources were used to automatically extract biased language markers to further enrich models with linguistic cues of uncertainty.
142
143\paragraph{Proposed Solution}
144\par In the data-sets used for the proposed solutions, the content of the post (text, media, and timestamp) and the primary content of the linked web page is included for each post. Scores for these posts were calculated as the mean judgment of at least five annotators who judge the post on a 4-point scale as not click baiting (0.0), slightly click baiting (0.3) considerably click baiting (0.66), or heavily click baiting (1.0). There were 3 data-sets, two labelled data-sets containing 2K and 20K data and one unlabelled data-set with 80K data.\\ The proposed solution was linguistically-infused neural network models that learn strength of clickbait content from the text present not only in the tweets themselves but also the linked articles, as well as images present in the tweets. It was hypothesized that while we should be able to get a certain level of performance using the tweet text or images alone, including the linked article content or a joint representation of the tweet and linked article would lead to a significant boost in performance. The models that were used were state-of-the-art Long Short-Term Memory (LSTM) and Convolutional Neural Network (CNN). The linguistic cues found in the content of the tweet and the title, keywords, description, and paragraphs of the associated article were used to boost performance of the models. The best LSTM models with images provided results with a mean squared error of 0.0444.
145
146\paragraph{Future Works}
147Applying an approach to capture relationships the clickbait posts and their associated images in order to extract more precise features from the tweets, capable of demonstrating how revealing an image is of its article’s content would yield better results. In addition, incorporating joint representations of the relationships between different components of clickbait posts in the vector representations.
148\newpage
149\section{Proposed System}
150    \begin{figure}[H]
151    \centerline{\includegraphics[width=0.8\textwidth]{diagram.PNG}}
152    \caption{Fake News Detection Model}
153    \end{figure}
154 \par There is a decent data set consisting of a decade-long, 12.8K manually labeled short statements in various contexts from POLITIFACT.COM, which provides detailed analysis report and links to source documents for each case\cite{LIAR}. This data set can be used for fact-checking research as well. Notably, this new data set is an order of magnitude larger than previously largest public fake news data sets of similar type. But novel approaches in this data set results in quite a low accuracy which cannot be applied to real world situations. Despite using powerful learning algorithms, promising results still haven’t been found to fight with the fake news growth. There is also another data-set FEVER\cite{FEVER} which contains 185K labeled data-sets with claims and evidence to support or refute the claims. This data-set is by far the largest data-set available for claim verification. The claims were generated from Wikipedia and verified by different groups of annotators. In the data-set FEVER\cite{FEVER}, the evidence or justification were used for the claim verification process but in LIAR data-set the evidences were ignored.
155 \par We propose that our multimodal data-set will include evidence that supports or refutes a claim. We will need to design a document retrieval component to retrieve necessary documents for justifying a claim. Then we will need to design a model that can justify or refute claim using the retrieved document. To design the model, we will have to select features that are best-suited for the purpose from the data-set. We will need to design truth scale which we will use to determine truth level of a statement. There are many different features that can be used in order to determine a claim's truth level such as speaker history, speaker's affiliation, source authenticity, clickbaitness, text similarity, sentiment, image related to claims etc. After completing a data-set with a suitable set of feature, we will move on to finding the best combination of the available algorithms such as Support Vector Machine (SVM), Bi-directional Long Short Term Memory model( Bi-LSTM), Convolutional Neural Network (CNN), Recurrent Neural Network (RNN) etc. with the available set of features to be used. We plan to use different attention models if necessary in order to improve our achieved accuracy. We will continue to evaluate and re-evaluate our program and change our set of features or the model until we achieve a noticeable improvement in accuracy.
156 \par After designing the learning system, we want to provide an interface to use it. For this we plan to create a website or a web application so that it is available to the general people.
157 
158 \section{Experiment Analysis}
159 
160 \section{Future Works}
161 We are now building a multimodal data-set for experimentation which is yet not completed. For completing the data-set, we plan to include new data that can be used for different feature selection algorithm such as text similarity, textual classification, candidate sentence selection, tone detection, sentiment analysis, image analysis etc. This data can include combination of different verification resources of a claim, images related to a claim etc.
162 \par After the creation of the data-set, we will have to choose a proper combination of model that will yield us with the best accuracy possible. There are many different kinds of state of the art models available now such as Bi-LSTM, Recurrent Neural Network, Multi-Layer Perceptron etc. There are also different attention models that we can use in order to improve  our results. We will have employ proper feature selection algorithms in order to use our design model efficiently. After that we will need to use different evaluation algorithms to determine achieved accuracy. Provided that we can achieve satisfactory accuracy, we will move on to designing a interface for user purpose.
163 
164 \section{Summary}
165 In this report we proposed a system using multimodal data-set that uses a learning system which can use textual classification to determine justification of a claim that is retrieved using a document retrieval component. The data-set will contain different types of data necessary for helping the designed model to determine the claim's truth level. We discussed different features and feature selection method that can be used as well as different candidate models. We discussed the data-set and the experiments conducted on the data-set that has been acquired until now. We also discussed the necessary works that need to be done in order to succeed in our quest. We hope that we will be able create a system that can determine a claim or statement's truthness more accurately than currently available ways.
166 
167\end{samepage}
168\bigskip
169\begin{thebibliography}{9}
170\bibitem{fake}
171Journal of Economic Perspectives: \textit{Volume 31 Number 2—Spring 2017—Page 212}
172\bibitem{pizzaria} 
173NYTIMES: In Washington Pizzeria Attack, Fake News Brought Real Guns
174\textit{https://www.nytimes.com/2016/12/05/business/media/comet-ping-pong-pizza-shooting-fake-news-consequences.html}.
175\bibitem{TIME} 
176TIME: Inside Russia’s Social Media War on America
177\textit{http://time.com/4783932/inside-russia-social-media-waramerica/}.
178\bibitem{behaivour}
179Kaghazgaran, Parisa, James Caverlee, and Majid Alfifi. "Behavioral Analysis of Review Fraud: Linking Malicious Crowdsourcing to Amazon and Beyond." \textit{ICWSM. 2017.}
180\bibitem{timeline}
181Thomas, Neha, and Susan Elias. "Classification of Fake Product Ratings Using a Timeline Based Approach." \textit{International Journal of Business Administration and Management Research 3.2 (2017): 12-15.}
182\bibitem{graph} 
183Jure Leskovec and Andrej Krevl, “SNAP Datasets: Stanford Large Network Dataset Collection”, June 2014 
184\bibitem{LIAR} 
185Wang, William Yang. "" Liar, Liar Pants on Fire": A New Benchmark Dataset for Fake News Detection." \textit{arXiv preprint arXiv:1705.00648 (2017)}.
186\bibitem{notunLIAR}
187Long Y, Lu Q, Xiang R, Li M, Huang CR. "Fake News Detection Through Multi-Perspective Speaker Profiles". \textit{In Proceedings of the Eighth International Joint Conference on Natural Language Processing (Volume 2: Short Papers) 2017 (Vol. 2, pp. 252-256).} Accessed: February 2018
188\bibitem{FEVER}
189Thorne, James, et al. "FEVER: a large-scale dataset for Fact Extraction and VERification." \textit{arXiv preprint arXiv:1803.05355 (2018).}
190\bibitem{emergent}
191Ferreira, William, and Andreas Vlachos. "Emergent: a novel data-set for stance classification." \textit{Proceedings of the 2016 conference of the North American chapter of the association for computational linguistics: Human language technologies. 2016.}
192\bibitem{DrQA}
193Chen, Danqi, et al. "Reading wikipedia to answer open-domain questions." \textit{arXiv preprint arXiv:1704.00051 (2017).}
194\bibitem{MLP}
195Riedel, Benjamin, et al. "A simple but tough-to-beat baseline for the Fake News Challenge stance detection task." \textit{arXiv preprint arXiv:1707.03264 (2017).}
196\bibitem{DA}
197Parikh, Ankur P., et al. "A decomposable attention model for natural language inference." \textit{arXiv preprint arXiv:1606.01933 (2016).}\
198\bibitem{clickbait}
199Glenski, Maria, et al. "Fishing for Clickbaits in Social Images and Texts with Linguistically-Infused Neural Network Models." \textit{arXiv preprint arXiv:1710.06390 (2017).}
200\bibitem{clickbase}
201Potthast, Martin, et al. "Clickbait detection." European Conference on Information Retrieval. Springer, Cham, 2016.
202\end{thebibliography}
203\end{document}