logo

SCIENTIA SINICA Informationis, Volume 48, Issue 11: 1521-1532(2018) https://doi.org/10.1360/N112018-00208

Knowledge-representation-enhanced question-answering system

More info
  • ReceivedAug 10, 2018
  • AcceptedOct 22, 2018
  • PublishedNov 15, 2018

Abstract

A knowledge-based, question-answering system can automatically answer natural language questions using triples in a knowledge graph. Simple questions are the most common type of questions which can be answered by a single triple. However, answering simple questions is still challenging when faced with a large-scale knowledge graph. Currently, most end-to-end models learn the distributional representations of entities, relations, and questions, and compute semantic relevance based on these representations. These models ignore the structural information of knowledge graphs, which is important for entity linking and relation recognition in questions. In this paper, we propose a unified representation learning method for text and knowledge to learn the representations of questions, entities, and relations. The structural information of a knowledge graph constrains the word representation and the word composition model. Experimental results over a knowledge-based question-answering system show that the question representation and the knowledge representation learned based on the unified representation can achieve a better performance.


Funded by

国家自然科学基金(61433015,61572477,61772505)

中国科协青年人才托举工程(YESS20160177)


References

[1] Bollacker K, Evans C, Paritosh P, et al. Freebase: a collaboratively created graph database for structuring human knowledge. In: Proceedings of ACM SIGMOD International Conference on Management of Data, 2000. 1247--1250. Google Scholar

[2] Suchanek F M, Kasneci G, Weikum G. Yago: a core of semantic knowledge. In: Proceedings of International Conference on World Wide Web, 2007. 697--706. Google Scholar

[3] Miller G A. WordNet: a lexical database for English. Commun ACM, 1995, 38: 39--41. Google Scholar

[4] Zettlemoyer L S, Collins M. Learning context-dependent mappings from sentences to logical form. In: Proceedings of the Meeting of the Association for Computational Linguistics, 2009. 976--984. Google Scholar

[5] Bos J, Clark S, Steedman M, et al. Scaling semantic parsers with on-the-fly ontology matching. In: Proceedings of the Conference on Empirical Methods in Natural Language Processing, 2013. 1545--1556. Google Scholar

[6] Golub D, He X D. Character-level question answering with attention. In: Proceedings of the Conference on Empirical Methods in Natural Language Processing, 2016. 1598--1607. Google Scholar

[7] Bordes A, Usunier N, Chopra S, et al. Large-scale simple question answering with memory networks. 2015,. arXiv Google Scholar

[8] Yin W P, Yu M, Xiang B, et al. Simple question answering by attentive convolutional neural network. 2016,. arXiv Google Scholar

[9] Zhou B T, Sun C J, Lin L, et al. LSTM based question answering for large scale knowledge base. Acta Sci Natl Univ Pekinensis, 2018, 54: 286--292. Google Scholar

[10] Hao Y C, Zhang Y Z, Liu K, et al. An end-to-end model for question answering over knowledge base with cross-attention combining global knowledge. In: Proceedings of Meeting of the Association for Computational Linguistics, 2017. 221--231. Google Scholar

[11] Bordes A, Usunier N, Garcia-Duran A, et al. Translating embeddings for modeling multi-relational data. In: Proceedings of Advances in Neural Information Processing Systems, 2013. 2787--2795. Google Scholar

[12] Collobert R, Weston J. A unified architecture for natural language processing: deep neural networks with multitask learning. In: Proceedings of the 25th International Conference on Machine Learning, 2008. 160--167. Google Scholar

[13] Mikolov T, Sutskever I, Chen K, et al. Distributed representations of words and phrases and their compositionality. In: Proceedings of Advances in Neural Information Processing Systems, 2013. 3111--3119. Google Scholar

[14] Pennington J, Socher R, Manning C D. Glove: global vectors for word representation. In: Proceedings of Conference on Empirical Methods in Natural Language Processing, 2014. 1532--1543. Google Scholar

[15] Zhang X, Zhao J B, Lecun Y. Character-level convolutional networks for text classification. In: Proceedings of International Conference on Neural Information Processing Systems, 2015. 649--657. Google Scholar

[16] Ling W, Lu'ıs T, Marujo L, et al. Finding function in form: compositional character models for open vocabulary word representation. 2015,. arXiv Google Scholar

[17] Mitchell J, Lapata M. Composition in distributional models of semantics.. Cognitive Sci, 2010, 34: 1388-1429 CrossRef PubMed Google Scholar

[18] Paulus R, Socher R, Manning C D. Global belief recursive neural networks. In: Proceedings of Advances in Neural Information Processing Systems, 2014. 2888--2896. Google Scholar

[19] Melamud O, Goldberger J, Dagan I. Context2vec: learning generic context embedding with bidirectional LSTM. In: Proceedings of SIGNLL Conference on Computational Natural Language Learning, 2016. 51--61. Google Scholar

[20] Vaswani A, Shazeer N, Parmar N, et al. Attention is all you need. 2017,. arXiv Google Scholar

[21] Bordes A, Glorot X, Weston J. A semantic matching energy function for learning with multi-relational data. Mach Learn, 2014, 94: 233-259 CrossRef Google Scholar

[22] Bordes A, Weston J, Collobert R, et al. Learning structured embeddings of knowledge bases. In: Proceedings of AAAI Conference on Artificial Intelligence, 2011. Google Scholar

[23] Socher R, Chen D, Manning C D, et al. Reasoning with neural tensor networks for knowledge base completion. In: Proceedings of International Conference on Intelligent Control and Information Processing, 2013. 464--469. Google Scholar

[24] Sutskever I. Modelling relational data using bayesian clustered tensor factorization. In: Proceedings of Advances in Neural Information Processing Systems, 2009. 1821--1828. Google Scholar

[25] Yang B, Yih W T, He X. Learning multi-relational semantics using neural-embedding models. 2014,. arXiv Google Scholar

[26] Trouillon T, Welbl J, Riedel S, et al. Complex embeddings for simple link prediction. In: Proceedings of International Conference on Machine Learning, 2016. 2071--2080. Google Scholar

[27] Wang Z, Zhang J W, Feng J L, et al. Knowledge graph embedding by translating on hyperplanes. In: Proceedings of the 28th AAAI Conference on Artificial Intelligence, 2014. 1112--1119. Google Scholar

[28] Fan M, Zhou Q, Chang E, et al. Transition-based knowledge graph embedding with relational mapping properties. In: Proceedings of Pacific Asia Conference on Language, Information and Computation, 2014. 328--337. Google Scholar

[29] Lin Y K, Liu Z Y, Sun M S, et al. Learning entity and relation embeddings for knowledge graph completion. In: Proceedings of Conference on Artificial Intelligence, 2015. 2181--2187. Google Scholar

[30] Lin Y K, Liu Z Y, Luan H B, et al. Modeling relation paths for representation learning of knowledge bases. 2015,. arXiv Google Scholar

[31] Wang Q, Wang B, Guo L. Knowledge base completion using embeddings and rules. In: Proceedings of International Conference on Artificial Intelligence, 2015. 1859--1865. Google Scholar

[32] Rocktäschel T, Singh S, Riedel S. Injecting logical background knowledge into embeddings for relation extraction. In: Proceedings of Conference of the North American Chapter of the Association for Computational Linguistics, 2015. 1119--1129. Google Scholar

[33] Guo S, Wang Q, Wang L H, et al. Jointly embedding knowledge graphs and logical rules. In: Proceedings of Conference on Empirical Methods on Natural Language Processing, 2016. 192--202. Google Scholar

[34] Milajevs D, Kartsaklis D, Sadrzadeh M, et al. Evaluating neural word representations in tensor-based compositional settings. 2014,. arXiv Google Scholar

[35] Milajevs D, Kartsaklis D, Sadrzadeh M, et al. Evaluating neural word representations in tensor-based compositional settings. 2014,. arXiv Google Scholar

[36] Zettlemoyer L S, Collins M. Learning to map sentences to logical form: structured classification with probabilistic categorial grammars. 2012,. arXiv Google Scholar

[37] Lukovnikov D, Fischer A, Lehmann J. Neural network-based question answering over knowledge graphs on word and character level. In: Proceedings of International Conference on World Wide Web, 2017. 1211--1220. Google Scholar

[38] Dai Z H, Li L, Xu W. CFO: conditional focused neural question answering with large-scale knowledge bases. 2016,. arXiv Google Scholar

Copyright 2019 Science China Press Co., Ltd. 《中国科学》杂志社有限责任公司 版权所有

京ICP备18024590号-1