ICTACT Journal on Soft Computing

Open Access Open Access  Restricted Access Subscription Access
Open Access Open Access Open Access  Restricted Access Restricted Access Subscription Access

Deepreply - An Automatic Email Reply System with Unsupervised Cloze Translation and Deep Learning


Affiliations
1 Department of Computer Science and Engineering, Rajalakshmi Engineering College, India
2 Department of Information Technology, Karpagam College of Engineering, India
     

   Subscribe/Renew Journal


Electronic mail (E-mail) has been the primary mode of communication for official purposes and it continues to be the same in all work environments even today. With the growing number of emails and most of them requiring only trivial replies, more tools are needed to generate replies to emails by reusing past replies. Although there are expert systems that can assist us in replying to incoming emails, they produce a generic reply to all. So an intelligent system that can generate replies for an incoming email in a very precise manner and generating the text reply in the user’s style is the identified requirement. This work is divided into two portions. First, translating an incoming email into cloze representation and extract the entities from it for generating a context, question and answer triplets. This is used for synthesising the training data for Extractive Question Answering later. The mentioned triplets are generated from a corpus of random emails belonging to different contexts and then the answers are extracted by recognising the named entities and random phrases of nouns from these paragraphs. The second ploy is to find the similarity between an incoming email that requires a reply and an old email that contains the reply to it. As a solution to these challenges, we propose a new deep neural network-based approach that relies on coarse-grained sentence modelling using CNN and a LSTM model. Our experimental results show that the approach outperforms the state-of-the-art approaches that are existing on a cleaner corpus.

Keywords

Deep Learning, E-mail, Unsupervised, Questioning.
Subscription Login to verify subscription
User
Notifications
Font Size

  • B. Agarwal, H. Ramampiaro, H. Langseth and M. Ruocco, “A Deep Network Model for Paraphrase Detection in Short Text Messages”, Information Processing and Management, Vol. 54, No. 6, pp. 922-937, 2018.

  • K. Amin, “Answering with Cases: A CBR Approach to Deep Learning”, Proceedings of International Conference on Case-Based Reasoning, pp. 1-12, 2018.

  • W. Xu, C.C. Burch, W.B. Dolan and Y. Ji, “Extracting Lexically Divergent Paraphrases from Twitter”, Proceedings of International Conference on Transactions of the Association for Computational Linguistics, Vol. 2, pp. 435-448, 2014.

  • W. Xu, C.C. Burch and W.B. Dolan, “SemEval-2015 Task 1: Paraphrase and Semantic Similarity in Twitter (PIT)”, Proceedings of 9th International Workshop on Semantic Evaluation, pp. 1-7, 2015.

  • K. Dey, S. Ritvik and K. Saroj, “A Paraphrase and Semantic Similarity Detection System for User Generated Short-Text Content on Microblogs”, Proceedings of International Conference on Computational Linguistics: Technical Papers, pp. 1-7, 2016.

  • N. Madnani, T. Joel and C. Martin, “Re-Examining Machine Translation Metrics for Paraphrase Identification”, Proceedings of Conference on North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pp. 1-8, 2012.

  • D. Das and N.A. Smith, “Paraphrase Identification as Probabilistic Quasi-Synchronous Recognition”, Proceedings of the Joint Conference of 47th Annual Meeting of Computational Linguistics, 2009.

  • M. Sahi and V. Gupta, “A Novel Technique for Detecting Plagiarism in Documents Exploiting Information Sources”, Cognitive Computation, Vol. 9, No. 6, pp. 852-867, 2017.

  • K. Vani and G. Deepa, “Unmasking Text Plagiarism using Syntactic-Semantic based Natural Language Processing Techniques: Comparisons, Analysis and Challenges”, Information Processing and Management, Vol. 54, No. 3, pp. 408-432, 2018. [10] Y. Jiang, “Wikipedia-Based Information Content and Semantic Similarity Computation”, Information Processing and Management, Vol. 53, No. 1, 2017.

  • Franco-Salvador, Paolo Rosso, and Manuel Montes Y. Gomez. “A Systematic Study of Knowledge Graph Analysis for Cross-Language Plagiarism Detection”, Information Processing and Management, Vol. 52, No. 4, pp. 550-570, 2016.

  • S. Arora, Y. Liang and T, Ma. “A Simple but Tough-to-Beat Baseline for Sentence Embeddings”, Proceedings of 5th International Conference on Learning Representations, pp. 1-12, 2016.

  • P. Bojanowski, E. Grave and A. Joulin, “Enriching Word Vectors with Subword Information”, Transactions of the Association for Computational Linguistics, Vol. 5, pp. 135-146, 2017.

  • M. Pagliardini, P. Gupta and M. Jaggi, “Unsupervised Learning of Sentence Embeddings using Compositional N-Gram Features”, Proceedings of North American Conference on Chapter of the Association for Computational Linguistics: Human Language Technologies, pp. 528-540, 2017.

  • Y. Kim, “Convolutional Neural Networks for Sentence Classification”, Proceedings of Conference on Empirical Methods in Natural Language Processing, pp. 1746-1751, 2014.

  • R. Kiros, Y. Zhu, R.S. Zemel and S. Fidler, “Skip-Thought Vectors”, Proceedings of International Conference on Advances in Neural Information Processing Systems, pp. 3294-3302, 2015.

  • Y. Kim, “Character-Aware Neural Language Models”, Proceedings of 13th AAAI Conference on Artificial Intelligence, pp. 1111-1119, 2016.

  • X. Wang, J. Weijie and L. Zhiyong, “Combination of Convolutional and Recurrent Neural Network for Sentiment Analysis of Short Texts”, Proceedings of 26th International Conference on Computational Linguistics: Technical Papers, pp. 1-9, 2016.

  • W. Guo and M. Diab, “Modeling Sentences in the Latent Space”, Proceedings of 50th Annual Meeting of the Association for Computational Linguistics: Long Papers, pp. 864-872, 2012.

  • G. Zarrella, J.C. Henderson, E.M. Merkhofer and L. Strickhart, “MITRE: Seven Systems for Semantic Similarity in Tweets”, Proceedings of 9th International Workshop on Semantic Evaluation, pp. 12-17, 2015.

  • J. Zhao and M. Lan, “ECNU: Leveraging Word Embeddings to Boost Performance for Paraphrase in Twitter”, Proceedings of 9th International Workshop on Semantic Evaluation, pp. 34-39, 2015.

  • N.P.A. Vo, S. Magnolini and O. Popescu, “Paraphrase Identification and Semantic Similarity in Twitter with Simple Features”, Proceedings of International Conference on Association for Computational Linguistics, pp. 10-19, 2015.


Abstract Views: 274

PDF Views: 0




  • Deepreply - An Automatic Email Reply System with Unsupervised Cloze Translation and Deep Learning

Abstract Views: 274  |  PDF Views: 0

Authors

P. V. Rajaraman
Department of Computer Science and Engineering, Rajalakshmi Engineering College, India
M. Prakash
Department of Information Technology, Karpagam College of Engineering, India

Abstract


Electronic mail (E-mail) has been the primary mode of communication for official purposes and it continues to be the same in all work environments even today. With the growing number of emails and most of them requiring only trivial replies, more tools are needed to generate replies to emails by reusing past replies. Although there are expert systems that can assist us in replying to incoming emails, they produce a generic reply to all. So an intelligent system that can generate replies for an incoming email in a very precise manner and generating the text reply in the user’s style is the identified requirement. This work is divided into two portions. First, translating an incoming email into cloze representation and extract the entities from it for generating a context, question and answer triplets. This is used for synthesising the training data for Extractive Question Answering later. The mentioned triplets are generated from a corpus of random emails belonging to different contexts and then the answers are extracted by recognising the named entities and random phrases of nouns from these paragraphs. The second ploy is to find the similarity between an incoming email that requires a reply and an old email that contains the reply to it. As a solution to these challenges, we propose a new deep neural network-based approach that relies on coarse-grained sentence modelling using CNN and a LSTM model. Our experimental results show that the approach outperforms the state-of-the-art approaches that are existing on a cleaner corpus.

Keywords


Deep Learning, E-mail, Unsupervised, Questioning.

References