Review of Sentiment Classification Methods and Opinion Mining: The Future Roadmap.

Review of Sentiment Classification Methods and Opinion Mining: The Future Roadmap.

1. Aishwarya Bhole,ME Student, 2) Prof. V. D. Thombre,HOD,computer

   SKNSIT Lonavala Engineering, SKNSIT Lonavala .

   Abstract: Since last decade, web documents especially social networking documents like twitter messages, Face book messages received more attention in order to analyze the individuals sentiments and opinions regarding to any specific subject such as politics, sports, social work etc. This approach greatly helps for automatic extraction as well as analysis of end user opinion and sentiment analysis using the web documents like reviews of customers on specific products, comments on any specific news etc. Use of such web documents for sentiment analysis is increasing day by day due to its easy availability as well as machine readable format already for further process. Recently social media is growing very fast on Web, and hence in this media organizations, political parties, individuals increasingly making use of public opinions for the accurate decision making. But finding opinion sites on Web and then monitoring opinion sites for mining the required information for sentiment analysis is still formidable task. This research area is commonly known as Opinion Mining and Sentiment Analysis. In this review paper our aim is to present the details study over Opinion Mining and Sentiment Analysis, its challenges, different methods etc.

   Keywords: Opinion mining, sentiment analysis, web documents, face book, twitter, natural language processing (LNP), text mining.

   1. INTRODUCTION

      The sentiment is a natural language processing and information extraction task that writers expressed feelings positive or negative comment too in order to obtain objective questions and analysis there, documents a & request large numbers is by checking for example: \I today so happy, good morning everyone ", a general positive text. Generally speaking, Sentiment analysis with respect to the subject a speaker or an author's point of view or a document to determine the overall functionality. Sentiment analysis is also known as opinion mining. In fact, to identify the emotion analysis task views expressed in a text is positive or negative weather. Natural language processing (NLP) computer science, Artificial intelligence, and computers and human (natural) concerned with interactions between languages is an area of Linguistics. According to a recent statistics by the Social Media tracking company Technocrat, four out of every five users of Internet use social media in some form. This includes friendship networks, blogging and micro- blogging sites, content and video sharing sites etc. It is worth observing that the World Wide Web has now completely transformed into a more participative and co-creative Web. It

      allows a large number of users to contribute in a variety of forms. The fact is that even those who are virtually novice to the technicalities of the Web publishing are creating content on the Web. In fact the value of a Website is now determined largely by its user base, which in turn decides the amount of data available on it. It may perhaps be true to say that Data is the new Intel inside [1].

      One such interesting form of user contributions on the Web is reviews. Many sites on the Web allow users to write their experiences or opinion about a product or service in form of a review. For example, a user looking for a hotel in a particular tourist city may prefer to go through the reviews of available hotels in the city before making a decision to book in one of them. Or a user willing to buy a particular model of digital camera may first look at reviews posted by many other users about that camera before making a buying decision. This not only helps in allowing the user to get more and relevant information about different products and services on a mouse click, but also helps in arriving at a more informed decision. Sometimes users prefer to write their experiences about a product or service as form of a blog post rather than an explicit review. However, in both case the data is basically textual. Popular sites like carwale.com, imdb.com are now full of user reviews, in this case reviews of cars and movies respectively. [3]

      Though these reviews and posts are beyond doubt very useful and valuable, but at the same time it is also quite difficult for a new user (or a prospective customer) to read all the reviews/ posts in a short span of time. Fortunately we have a solution to this information overload problem which can present a comprehensive summary result out of a large number of reviews. The new Information Retrieval formulations, popularly called sentiment classifiers, now not only allow to automatically label a review as positive or negative, but to extract and highlight positive and negative aspects of a product or services. Sentiment analysis is now an important part of Information Retrieval based formulations in a variety of domains.

      It is traditionally used for automatic extraction of opinions types about a product and for highlighting positive or negative aspects of a product. It is widely believed that Sentiment analysis is needed and useful. It is also widely accepted that extracting sentiment from text is a hard semantic problem even for human beings. So in general, Sentiment Analysis will be useful for extracting sentiments available on Blogging sites, Social Network, Discussion Forum in order to benefit both company and customer/user. In this paper we are presenting the detailed discussion over

      the Opinion Mining and Sentiment Analysis with their different methods of doing so. Below in II section we are first discussing concept of opinion mining in detail and presenting its different aspects. In section III we are taking the review of different techniques presented for sentiment analysis and classification. In section IV we are discussing the different performance measures used to predict the accuracy of sentiment classification methods.

   2. REVIEW OF OPINION MINING

      The concept of opinion mining is nothing but web data mining. Following figure 1 is showing the synonyms of opinion mining. Below is the definition for the same: If a set of text documents (T) are given, that have opinions on an object, opinion mining intends to identify attributes of the object on which opinion have been given, in each of the document t T and to find orientation of the comments i.e. whether the comments are positive or negative.

      Figure 1: Synonyms of Opinion Mining

      2.1 Scientific Fundamentals

      2.1.1 Model of Opinion Mining: As people are free to give their opinions on anything, e.g., they buy a product and then they express their views on products features in various forums. The term object is used for the entity on which comments have been given.

      Definition (object): An object A is an entity. It is related to a pair, A: (C, R), where C is the components and sub- components of A, and R is the attributes of A. Each component can have its own sub-components and attributes. Features can refer to either components or attributes. It is also commonly used for objects. Let us consider a document t, which contains opinions on an object A. Generally, t is composed of sentences t = (s1, s2, s3 Sn).

      Definition (opinion passage on a feature): An object removed from a document t, a special feature on the opinion- opinion on t f is a group of sentences. Many features of a product on a sentence, e.g., "the camera's picture quality is good, but the battery life is short" opinions can be expressed.

      Defiition (opinion holder): opinion on the person the holder of his/her opinion.

      Definition (semantic classification of an opinion orientation/spirit): a meaning of an opinion on the orientation feature f States that opinion is positive, negative or neutral level i.e. the classification of an object regardless of whether the sentence at the facility include a sentence in a positive feedback or negative feedback can be on it. Put things together, an object and a set of opinions on the characteristics of the object to be a model, which is called feature-based opinion mining model can be defined.

      1. Model of Feature-Based Opinion Mining: An object A can be represented with a set of features, F = {f1, f2 fn}, which includes the object itself. Each feature Wi-Fi f words or phrases are synonyms that can be expressed with a finite set is a set that same choice. A set of W = {W1, W2, Wn} n facilities. an opinion holder comments each feature fi Wk, and then that can be positive, negative or neutral opinions on the word Fi by choosing this feature to describe any document, opinion mining information useful (opinion feelings) from a given document to remove.

      2. Mining Output: Given an evaluative document t having opinions on an object A, the result is a set of quadruples. Every Quadruple is represented by (H, A, f, S), where H being the opinion holder, A being the object, f being feature of the object A and S being semantic orientation of the opinion on feature f in document t.

      3. Opinion Summary: There are several ways to utilize the results of opinion mining. One way is to represent a summary of opinions on features of the objects. This is explained with an example in following section [4, 23]

      4. Mining comparative and superlative (regular) opinion sentences:

         A person, who purchased a certain product, expresses opinions and directs on the characteristics of the object as "Feature1" this product is great. "This product is so "Feature2. This product features a 3 are direct such bogus. Opinion or is regular opinion. In this case, opinion mining, commodities, their features and opinion orientation to be identified expressed such opinion. Is the best such as adjectives/adverbs Feature4 third using the form "is" the worst Feature4.

         Some people express their views on products in a comparative way e.g. Feature1 of product A is better than Feature 1 of product B. Thus e.g. Word and adjective/adverbs than "better the second-last sentence. Forms are made using comparisons are related to but different from direct feedback are also what it things than be text. to be identified which featured are being compared and the objects in their opinion is given preference by the holders [4, 10, 11].

      5. Sentiment Classification:

      Sentiment classification has been widely studied in the natural language processing community. It is defined as follows: If a set of evaluative documents T are given, it determines whether each document t T expresses a positive

      or negative opinion (or sentiment) on an object. Emotion classification basically satisfies the above assumption each evaluation document object O defines the meaning of the opinions expressed on the orientation study document-level sentiment classification. In addition, sentence-level classification, i.e., every sentence a subjective or objective sentences and to express a positive or negative opinion be classified as.

   3. REVIEW OF SENTIMENT CLASSIFICATION AND ANALYSIS.

      More research is user feedback data, which mainly end user review judges exists on the analysis of these studies, sentiment analysis is often held in one of three levels: document level, sentence level or attribute level. With regard to the analysis, machine learning and spirit semantic orientation was of two types of techniques including literature survey. In addition to that Nature language processing (NLP) techniques is especially in the area of document used in a sense. Current-day sentiment detection is thus a discipline at the crossroads of NLP and Information retrieval, and as such it shares a number of characteristics with other tasks such as information extraction and text- mining, computational linguistics, psychology and predicative analysis.

      1. Machine Learning

         The machine learning approach applicable to sentiment analysis mostly belongs to supervised classification in general and text classification techniques in particular. Thus, it is called "the" supervised learning ". A machine learning- based classification, two sets of documents are required: training and test set a training set to learn the difference signs documents an automatic classifier is used by, and a test set to validate the performance of the automatic classifier Is used for machine learning techniques to classify a number of review adopted. Naive Bayes (NB), maximum entropy (me), machine learning techniques like and support vector machine (SVM) has achieved great success in text classification. Other noted machine learning ways the subsequent natural language processing and Qiang Ye, 2009). The basic idea combined words and categories of prospects, given test document categories using prospects. Such model is the notion of independence is part of the word naïve simplicity of perception Naive Bayes Classifier of the computation makes far more efficient. Support vector machines (SVM), a discriminative classifier is considered the best text classification method (Rui Xia, 2011; Ziqiong, 2011; Songho tan, 2008 and Rudy Prabowo, 2009). . The support vector machine is a statistical classification method proposed by Vapnik. Computational learning theory based on minimum principle of structural risk, SVM training data points to the surface a decision to separate the two sections and support vector training set is effective elements in selected decisions based on many of the variants. SVM spirit in which multiple class classification (Kaiquan Xu, 2011) is used for has been developed.

         The idea behind the centroid classification algorithm is extremely simple and straightforward (Songho tan, 2008). Initially the prototype vector or centroid vector for each training class is calculated, then the similarity between a testing document to all centroid is computed, finally based on these similarities, document is assigned to the class corresponding to the most similar centroid.

         K-nearest neighbors (KNN) classifier that the basis of a specific example of a clear, declarative representation does not build, but document the same training documents depends on the category labels associated with a given document d, the system finds the closest neighbors k between training documents. Score of resemblance to test documents each document (the document closest neighbor Songho Tan2008) used as weight classes. Winnow is a well-known online mistaken-driven method. It works by updating its weights in a sequence of trials. On each trial, it first makes a prediction for one document and then receives feedback; if a mistake is made, it updates its weight vector using the document. During the training phase, with a collection of training data, this process is repeated several times by iterating on the data (Songho tan, 2008). Besides these classifiers other classifiers like ID3 and C5 are also investigated. Besides using these above said machine learning methods individually for sentiment classification, various comparative studies have been done to find the best choice of machine learning method for sentiment classification. Songbo Tan (2008) presents an empirical study of sentiment categorization on Chinese documents. He has four feature selection methods (MI, IG, Chi and DF) and a Chinese emotion corpus five teaching methods (K-nearest neighbor, centroid classifier, SVM classifier, Naive Bayes and winnow). results from the IG concluded that the emotional conditions performs best for selection and SVM Naïve Bayes and show the best the other two classifier outperforms n-gram model destination reviews, Ye et al. (2009).

         Rudy Prabowo (2009) described an extension by combining rule-based classification, Supervised learning and learning in a new combined machine for each sample set, out 10-fold cross-validation is performed for each fold, the associated samples divided into a training and test set if one passes a document containing the classifier The throat is classified by the classifier, document, or some other classifier for classifying documents exists for each test specimen fails, a hybrid classification, i.e., given a training set. Rules-based classifier (RBC) a narrator, a set of rules and a set of test samples representing used to generate and test a training set rules derived from the sample used to characterize. If the test sample was unclassified, half of the people on the linked Statistics Canada And classifier (SBC), passed the test sample if SBC cannot classify; Classifier based on General Inquirer joined SBC (which determine the consequents of people by simple rules 3672 used GIBC), support vector machine passed. (SVM) is a training set if the same failed to classify three classifiers for classifying the test sample.

         An ensemble technique is one which combines the outputs of several base classification models to form an integrated output. Hai Xia (2011) used this approach efficiently in a

         variety of feature sets and classification algorithms to synthesize a more precise classification process by integrating emotion classification for a comparative study of the effectiveness of education and technology. in his work the spirit of two types of feature sets are designed for classification, Namely the part and text-to-speech feature sets-based feature-based classification algorithms then, three text sets, namely Bayes, maximum entropy and support vector machine classifiers feature sets, naïve base-as for each of the classification score to predict. three types of ensemble methods, namely fixed combinations, and weighted meta- classifier combination, are rated for three ensemble strategies IE feature set ensemble classification algorithms of ensemble, Facility, and both ensemble sets and classification algorithms.

         In most of the comparative studies it is found that SVM outperforms other machine learning methods in sentiment classification. Ziqiong Zhang (2011) showed a contradiction in the performance of SVM. They focused their interest on written Cantonese, a written variety of Chinese. They learn a method which proved that the chosen method and machine learning model lexical elements in a piece of Cantonese review distribution may be able to draw their own conclusions from completely first proposed the use of knowledge-free supervision machine. Despite its unrealistic independence assumption, Naïve Bayes Classifier achieves better performance compared to the SVM amazing.

         Sentiment classification is done by constructing a text classifier by extracting association rules that associate the terms of a document and its categories, by modeling the text documents as a collection of transactions where each transaction represents a text document, And transaction to documents and items in a document is assigned categories are then selected conditions, system documents to them in words and associations between labels assigned to each category shows up as a separate text collection are considered. The Association rule mining and applied it to all categories of rules separately. generate a classifier (Weitong Huang, 2008) are then combined to form training set classification quality tests number of classified documents, cover text, rules, Used to evaluate and attributive use probability. Yulan emotion classifier is learning from the unlabeled documents to create a novel framework for the (2010). Annotated text and the UN process a sense begins with a collection of lexicon.

         Yulan He (2010) attempted to create a novel framework for sentiment classifier learning from unlabeled documents. Annotated text and the UN process a sense begins with a collection of lexicon. An initial sense dictionary classifier which words marked with their respective polarity involves a list of already trained by incorporating information. Directly model the label features predictions unlabeled instances using generalized expectation criteria use to constrain the trained classifier in generalized expectation. Annotated text then use is applied to the United Nations and with high confidence relies on documents automatically labeled domain features to get scholars are fed into the extractor facilities. Such features self-contained learned later that final results of the repeat examination set to receive another classifier are

         applied to train. A few recent studies in this field explained the use of neural networks in sentiment classification. Zhu Jian (2010) proposed an individual model based on Artificial neural networks to divide the movie review corpus into positive , negative and fuzzy tone which is based on the advanced recursive least squares back propagation training algorithm. Long-Sheng Chen (2011) proposed a neural network based approach, which combines the advantages of the machine learning techniques and the information retrieval techniques.

      2. Orientation of Semantic

         Semantic orientation approach sentiment analysis "because it does not require training in" unsupervised learning "to mine data. Instead, it measures how far a word towards a positive and negative sense. Unsupervised classification of research into lexical uses is available in resources. Kamps et al (2004) lexical relations in spirit are focusing on the use of classification. Andrea Esuli and Fabrizio Sebastiani (2005) the proposed method of teaching a semi supervision Initial seed WorldNet began to detail set using their basic assumption identical terms tend to have similar glosses with orientation they expanded by statistical techniques to seed the word orientation is determined through the gloss assortment.

         When feedback to determine the orientation of sufficient information, where relevant an opinion cannot provide Chunxu Wu lie (2009) an approach that other reviews useful relevant information for me to discuss the same subject to resort, means equality measures USE Industry opinion and proposed to judge the orientation they are independent of the orientation context, attempted to deal with this problem by the context-dependent feedback estimate different opinions of the linguistic context-dependent orientation consider using rules, relevant information. This device complies with the other reviews that obscure-dependent feedback context to judge on the same product feature remove. Review each part of speech sense phrases (POS) extracting rules by an unsupervised learning pattern algorithm Ting-Chun Peng and Chia-Chun Shih (2010) was investigated for each unknown by the sense of the phrase, they call it a query term as top-N to get a relevant degree respectively used for search engines. Next, a lexicon, unknown spirit gathered a sense of feeling feelings known phrases predictive sense words are computed based on nearby inside snippets. They consider only opinionated sentences containing at least one detected sentiment phrase for opinion extraction. Using the POS pattern opinion extraction is done. Gang Li & Fei Liu (2010) developed an approach based on the k-means clustering algorithm. The technique of TF-IDF (term frequency inverse document frequency) weighting is applied on the raw data. After that voting mechanism is handled to extract more stable clustering result. The result is obtained based on multiple implementations of the clustering process. Finally, the term score is used to further enhance the clustering result. Documents are clustered into positive group and negative group.

         Chaovalit and Zhou (2005) compared the Semantic Orientation approach with the N-gram model machine learning approach by applying to movie reviews. They confirmed from the results that the machine learning

         	Predicted positives		Predicted negatives
         Actual positive instances	True Instances(TP)	Positive	False instances(FN)	negative
         Actual negative instances	False instances(FP)	positive	True instances(TN)	negative

         approach is more accurate but requires a significant amount of time to train the model. In comparison, the semantic orientation approach is slightly less accurate but is more efficient to use in real-time applications. Performance of semantic orientation is relies on the performance of the underlying POS tagger.

      3. Role of Negation

         Negation is a very common linguistic construction that affects the polarity and therefore to be taken into account in the analysis, sentiment needs to have not only given general prohibition. Negation words (no, neither, nor), but also by other lexical units. The research has shown that there are many other words that expressed an opinion, such as polarity inverted connectives or shifters, modals valence. "I get the new mobile functionality, less practical for an example valence shifter," maybe it's a great phone, but I fail to see why ", reflects the influence of connectives. An example sentences using modal," in principle, the phone should also work under water. As can be seen from these examples, negation is a difficult yet important aspect of sentiment analysis.

         Kennedy and Inkpen (2005) evaluate a negation model which is fairly identical to the one proposed by Polanyi and Zaenen (2004) in document-level polarity classification. A polar expression is thought to be negated if the negation word immediately precedes it. Wilson et al. (2005) carry out more advanced negation modeling on expression-level polarity classification. The work uses supervised machine learning where negation modeling is mostly encoded as features using polar expressions. Jin-Cheon Na (2005) reported a study in automatically classifying documents as expressing positive or negative. He investigated the use of simple linguistic processing to address the problems of negation phrase.

         In sentiment analysis, the most prominent work examining the impact of different scope models for negation is Jia et al. (2009). They define scope.

         Focused on polar expressions Static delimiters are unambiguous words, such as because or unless marking the beginning of another clause. Dynamic delimiters are, however, rules, using contextual information such as their pertaining part-of-speech tag. These delimiters suitably account for various complex sentence types so that only the clause containing the negation is considered. The heuristic rules focus on cases in which polar expressions in specific syntactic configurations are directly preceded by negation words which results in the polar expression becoming a

         delimiter itself.

   4. Sentiment Classification Performance Metrics

      Generally, the performance of sentiment classification is evaluated by using four indexes: Accuracy with Precision plus Recall and F1-score. This is the normal option to compute these indexes which are depend on the confusion matrix shown in Table 1.

      TABLE 1 CONFUSION MATRIX

      The equations show the indexes:

      Precision = (1)

      Recall = (2)

      Accuracy = (3)

      F-measures = (4)

      The accuracy of prediction predicts all against all instances of true examples. Instances of a 100% accuracy to predict exactly the same as the actual examples are meant to predict all positive instances against true. Precision predicts positive instances. Remember that part of the true positive prediction turned positive in all instances is against real F1 recall and precision of a harmonic.

   5. Conclusion and Future Work

      As this is our first review paper, we have discussed the review of sentiment analysis and opinion mining in detail. In this first we have discussed the details of opinion mining. We observed that opinion mining is greatly helps individuals to take right decision after analysis of particular thing is done. This is possible only through the opinion mining through web documents. We have discussed the different issues faced by opinion mining as well. Later we have presented different methods those are presented by various researchers for sentiment classification. Finally the measures of sentiment classification methods presented. For the future work, we suggest to present new improved method for sentiment classification and analysis using some social networking messages dataset like twitter under real time environment.

   6. ACKNOWLEDGEMENT

      I would like to thank my guide prof. Vikas Thombre, HOD of computer department, SKNSIT ,Lonavala , provide a help- ful guidance for writing this paper.

   7. REFERENCES

1. Z. M. Ma.2005.Databases Modeling of Engineering Information, Northeastern University, China

2. M. Mehdi Owrang O.2007.Discovering Quality Knowledge from Relational Databases, American University, USA.

3. [3] Bing Liu.2011.Sentiment Analysis Tutorial – Given at AAAI-2011, San Francisco, USA.

4. Bing Liu.2007.Opinion Mining, Department of Computer Science University of Illinois at Chicago.

5. Andreas Auinger, Martin Fischer.2008.Mining consumers opinions on the web.

6. Qingyu Zhang and Richard S. Segall .2008.Web Mining: a Survey of Current Research, Techniques, and Software.

7. Lita van Wel and Lamber Royakkers.2004. Ethical issues in web data mining, Department of Philosophy and Ethics of Technology.

8. Liu, B., Hu, M. and Cheng, J. 2005. Opinion Observer. Analyzing and Comparing Opinions on the Web. Proceedings of International World Wide Web Conference (WWW05).

9. Ganesan, K. A., and H. D. Kim. 2008. Opinion Mining – A Short Tutorial (Talk), University of Illinois at Urbana Champaign.

10. Ganapathibhotla, G. and Liu, B. 2008.Identifying Preferred Entities in Comparative Sentences. To appear in Proceedings of the 22nd International Conference on Computational Linguistics (COLING08).

11. Jindal, N. and Liu, B. Mining.2006.Comparative Sentences and Relations. Proceedings of National Conference on Artificial Intelligence (AAAI06).

12. Ana Azevedo AND M F Santos.2008.Kdd, Semma and Crisp- Dm. A Parallel Overview

13. H D U K Kim, K. Ganesan, P Sondhi, C Zhai. 2011. Comprehensive Review of Opinion Summarization.

14. Alexandra Balahur and Andrés Montoy.2010.OpAL. Applying Opinion Mining Techniques for the Disambiguation of Sentiment Ambiguous Adjectives in SemEval-2 Task 18. Anna Stavrianou and Jean-Hugues Chauchat.2008. Opinion Mining Issues and Agreement Identification in Forum Texts.

15. Andrea Esuli. 2008. Automatic Generation of Lexical Resources for Opinion Mining. Models, Algorithms and Applications.

16. Krisztian Balog, Maarten de Rijke. 2006.Decomposing Bloggers' Moods – Towards a Time Series Analysis of Moods in the Blogosphere

17. [18] Marina Sokolova, Guy Lapalme.2008.Verbs Speak Loud. Verb Categories in Learning Polarity IS Strength of Opinions.

18. Animesh Kar, Deba Prasad Mandal.2011.Finding Opinion Strength Using Fuzzy Logic on Web Reviews.

19. [20] B. Ohana, B. Tierney.2011.Opinion Mining with IS Sent Word Net.

20. Zhongwu Zhai, Bing Liu.2011.Identifying Evaluative Sentences in Online Discussions.

21. Hyun Duk Kim, ChengXiang Zhai.2009.Generating comparative summaries of contradictory opinions in text.

22. MurthyGanapathibhotla BingLiu.2008.MiningOpinions in comparative sentences.

23. Long-Sheng Chen, Cheng-Hsiang Liu, Hui-Ju Chiu, A neural network based approach for sentiment classification in the blogosphere, Journal of Informatics 5 (2011) 33 322.

24. Martin, J. (2005). Blogging for dollars. Fortune Small Business, 15(10), 8892.

25. Melville, Wojciech Gryc, Sentiment Analysis of Blogs by Combining Lexical Knowledge with Text Classification, KDD09, June 28July 1, 2009, Paris, France. Copyright 2009 ACM 978-1-60558-495-9/09/06.

26. Namrata Godbole, Manjunath Srinivasaiah, Steven Skiena, a Large scale Sentiment Analysis for News and the Blogs, ICWSM-2007 Boulder, Colorado, USA.

27. Nan Li , Desheng Dash Wu , Using text mining and sentiment analysis for online forums hotspot detection and forecast, Decision Support Systems 48 (2010) 354368.

28. Polanyi and A. Zaenen, Contextual lexical valence shifters, in Proceedings of the AAAI Spring Symposium on Exploring Attitude and Affect in Text, AAAI technical report SS-04-07, 2004.

29. Popescu, A. M., Etzioni, O.: Extracting Product Features and Opinions from Reviews, In Proc. Conf. Human Language Technology and Empirical Methods in Natural Language Processing, Vancouver, British Columbia, 2005, 339346.

30. Qiang Ye, Ziqiong Zhang, Rob Law, Sentiment classification of online reviews to travel destinations by supervised machine learning approaches, Expert Systems with Applications 36 (2009) 65276535.

31. Qingliang Miao, Qiudan Li, Ruwei Dai, AMAZING: A sentiment mining and retrieval system, Expert Systems with Applications 36 (2009) 71927198.

32. Rudy Prabowo, Mike Thelwall, Sentiment analysis: A combined approach., Journal of Informetrics 3 (2009) 143 157.

33. Rui Xia, Chengqing Zong, Shoushan Li, Ensemble of feature sets and classification algorithms for sentiment classification is Information Sciences 181 (2011) 11381152.

34. Songbo Tan, Jin Zhang, An empirical study of sentiment analysis for Chinese documents, Expert Systems with Applications 34 (2008) 26222629.

35. Teng-Kai Fan, Chia-Hui Chang , Blogger-Centric

36. Contextual Advertising Expert Systems with Applications 2011) 17771788.

37. Ting-Chun Peng and Chia-Chun Shih, An Unsupervised Snippet-based Sentiment Classification Method for Chinese

38. Unknown Phrases without using Reference Word Pairs,

39. 2010 IEEE/WIC/ACM International Conference on Web

40. Intelligence and intelligent Agent Technology JOURNAL.