logo

SCIENTIA SINICA Informationis, Volume 49, Issue 8: 1019-1030(2019) https://doi.org/10.1360/N112018-00280

Aspect-based sentiment analysis based on dynamic attention GRU

More info
  • ReceivedOct 18, 2018
  • AcceptedMay 5, 2019
  • PublishedAug 7, 2019

Abstract

Aspect-level sentiment analysis is a fine-grained task that aims to identify the sentiment polarity (i.e., negative, neutral, or positive) of a specific target opinion in its context. Since the sentiment polarity of a target depends on the target itself and the semantics of the context, the target and the sentence should be treated equally and modeled interactively. For aspect-level sentiment analysis, we propose (1) a method to encode the aspect and sentence simultaneously, and (2) a neural network based on a dynamic attention gated recurrent unit. The simultaneous encoding manner can generate the target representation, which contains more contextual clues. The dynamic attention mechanism can achieve the attention values of contextual words and further generate the target representation dynamically. Experimental results achieved on a SemEval 2014 dataset (Laptop and Restaurant) show that our approach achieves a significant improvement in the accuracy rates over the standard attention-based models.


Funded by

国家自然科学基金(61672126)


References

[1] Pang B, Lee L. Opinion mining and sentiment analysis. Foundations and trends in information retrieval, 2008, 2(1-2):1-135. Google Scholar

[2] Liu B. Sentiment analysis and opinion mining. Synthesis Lectures on Human Language Technologies, 2012, 5(1):1-167. Google Scholar

[3] Pontiki M, Galanis D, Pavlopoulos J, et al. Semeval-2014 task 4: aspect based sentiment analysis. In: Proceedings of the 8th International Workshop on Semantic Evaluation, Dublin, 2014. 27--35. Google Scholar

[4] Wagner J, Arora P, Cortes S, et al. Dcu: aspect-based polarity classification for semeval task 4. In: Proceedings of the 8th International Workshop on Semantic Evaluation, Dublin, 2014. 223--229. Google Scholar

[5] Kiritchenko S, Zhu X D, Cherry C, et al. NRC-Canada-2014: detecting aspects and sentiment in customer reviews. In: Proceedings of the 8th International Workshop on Semantic Evaluation, Dublin, 2014. 437--442. Google Scholar

[6] Vo D, Zhang Y. Target-dependent twitter sentiment classification with rich automatic features. In: Proceedings of the IJCAI, Buenos Aires, 2015. 1347-1353. Google Scholar

[7] Tang D Y, Qin B, Feng X C, et al. Effective lstms for target-dependent sentiment classification. In: Proceedings of COLING, Osaka, 2016. 3298--3307. Google Scholar

[8] Wang Y Q, Huang M L, Zhao L, et al. Attention-based lstm for aspect-level sentiment classification. In: Proceedings of the Conference on Empirical Methods in Natural Language Processing, 2016. 606--615. Google Scholar

[9] Tang D Y, Qin B, Liu T. Aspect level sentiment classification with deep memory network. In: Proceedings of the Conference on Empirical Methods in Natural Language Processing, Austin, 2016. 214--224. Google Scholar

[10] Chen P, Sun Z Q, Bing L D, et al. Recurrent Attention Network on Memory for Aspect Sentiment Analysis. In: Proceedings of the Conference on Empirical Methods in Natural Language Processing, Copenhagen, 2017. 452--461. Google Scholar

[11] Socher R, Pennington J, Huang H, et al. Semi-supervised recursive autoencoders for predicting sentiment distributions. In: Proceedings of Conference on Empirical Methods in Natural Language Processing, Edinburgh, 2011. 151--161. Google Scholar

[12] Dong L, Wei F R, Tan C Q, et al. Adaptive recursive neural network for target-dependent twitter sentiment classification. In: Proceedings of the 52th Annual Meeting of the Association for Computational Linguistics, Baltimore, 2014. 49--54. Google Scholar

[13] Qian Q, Tian B, Huang M L, et al. Learning tag embeddings and tag-specific composition functions in recursive neural network. In: Proceedings of the 53th Annual Meeting of the Association for Computational Linguistics, Beijing, 2015. 1365--1374. Google Scholar

[14] Mikolov T, Karafi'at M, Burget L, et al. Recurrent neural network based language model. In: Proceedings of the Interspeech, Makuhari, 2010. 1045--1048. Google Scholar

[15] Tai K S, Socher R, Manning C D. Improved semantic representations from tree-structured long short-term memory networks. In: Proceedings of Annual Meeting of the Association for Computational Linguistics, Beijing, 2015. 1556--1566. Google Scholar

[16] Jiang L, Yu M, Zhou M, et al. Target-dependent twitter sentiment classification. In: Proceedings of Annual Meeting of the Association for Computational Linguistics: Human Language Technologies, Portland, 2011. 151--160. Google Scholar

[17] Ma D H, Li S J, Zhang X D, et al. Interactive attention networks for aspect-level sentiment classification. In: Proceedings of the IJCAI, Melbourne, 2017. 4068--4074. Google Scholar

[18] Huang B X, Ou Y L, Carley K M. Aspect level sentiment classification with attention-over-attention neural networks. 2018,. arXiv Google Scholar

[19] Zheng S L, Xia R. Left-center-right separated neural network for aspect-based sentiment analysis with rotatory attention. 2018,. arXiv Google Scholar

[20] Cho K, Merrienboer B V, Gulcehre C, et al. Learning phrase representations using RNN encoder-decoder for statistical machine translation. In: Proceedings of Conference on Empirical Methods in Natural Language Processing, Doha, 2014. 1724--1734. Google Scholar

[21] Xiong C M, Zhong V, Socher R. Dynamic coattention networks for question answering. 2017,. arXiv Google Scholar

[22] Pennington J, Socher R, Manning C D. Glove: global vectors for word representation. In: Proceedings of Conference on Empirical Methods in Natural Language Processing, Doha, 2014. 1532--1543. Google Scholar

[23] Tieleman T, Hinton G. Lecture 6.5-rmsprop: divide the gradient by a running average of its recent magnitude. COURSERA: Neural Networks for Machine Learning, 2012, 4: 26--30. Google Scholar

  • Figure 1

    (Color online) The architecture of DAGRU model

  • Figure 2

    (Color online) DAGRU unit

  • Table 1   Examples of sentimental classification
    Sentence Target entity Sentiment polarity
    (a) Great food but the service was dreadful Food Positive
    Service Negative
    (b) Except Patrick, all other actors don't play well Patrick Positive
  • Table 2   The statistics of the datasets
    Sentiment polarity Laptop Restaurant
    Train Test Train Test
    Positive 994 341 2164 728
    Neutral 464 169 637 196
    Negative 870 128 807 196
    Total 2328 638 3608 1120
  • Table 3   Hyperparameter configuration
    Parameter Value
    BiGRU hidden units 64
    DAGRU hidden units 128
    Dropout 0.5
    Recurrent dropout 0.5
    DAGRU layers $K$ 5
    Batch size 64
    Learning rate 0.001
  • Table 4   The impacts of DAGRU layers and encoding$^{\rm~a)}$
    Layers Laptop (%) Restaurant (%)
    Joint encoding Individual encoding Joint encoding Individual encoding
    0 74.92 56.27 80.09 67.50
    1 75.70 72.57 80.80 80.18
    2 76.02 72.88 81.07 80.18
    3 75.86 72.88 81.16 80.80
    4 $\boldsymbol{76.49}$ 73.04 81.25 $\boldsymbol{81.61}$
    5 76.33 $\boldsymbol{73.51}$ $\boldsymbol{81.96}$ 81.07

    a) The blod number represents the highest result.

  • Table 5   Performance comparison of the models
    Model Laptop (%) Restaurant (%)
    Majority 53.45 65.00
    Simple-SVM 66.97 73.22
    Feature-enhauced SVM (Kiritchenko et al.) [5] 72.10 80.89
    TD-LSTM (Tang et al.) [7] 68.13 75.63
    AE-LSTM (Wang et al.) [8] 68.90 76.60
    ATAE-LSTM (Wang et al.) [8] 68.70 77.20
    MemNet (Tang et al.) [9] 70.33 79.98
    IAN (Ma et al.) [17] 72.10 78.60
    RAM (Chen et al.) [10] 74.49 80.23
    AOA-LSTM (Huang et al.) [18] 74.50 81.20
    LCR-Rot (Zheng and Xia) [19] 75.24 81.34
    DAGRU (proposed model) 76.33 81.96

Copyright 2019 Science China Press Co., Ltd. 《中国科学》杂志社有限责任公司 版权所有

京ICP备18024590号-1