A deep learning approach for text generation

One of the most challenging language modeling problems is text generation. The importance of language modeling comes from its involvement in many language processing tasks such as conversational system, speech to text, and text summarization. The language models are typically trained to learn the occurrence of next word in a sequence based on previous words in the text. However, when it comes to testing, it is highly expected that the entire sequence will be generated from the scratch which is computationally not suitable to many applications. In this paper, one of the popular deep learning architectures called bidirectional recurrent neural network (BRNN) to develop a text generation approach. The recurrent neural network used in the developed approach uses long short-term memory (LSTM). By using LMST recurrent neural network, the proposed approach is well suited to making predictions based on time series data. Two representations are considered in this paper; word to vector (word2vec) and one-hot representations. The word2vec model is used with two datasets called Twilight and Alice in Wonderland stories while the one-hot model is used only with Alice in Wonderland story only. The experimental results show that the word2vec model outperforms the one-hot model when using to large data sets.