TensorFlow: simple recurrent neural network

I've built some neural networks with TensorFlow, like basic MLPs and convolutional neural networks. Now I want to move on to recurrent neural networks. However, I'm not experienced in natural language processing. Therefore the TensorFlow NLP tutorials for RNNs are not easy to read for me (and not really interesting, too).

Basically I want to start off with something simple, not a LSTM.

How would one build a simple recurrent neural network, like an Elman network, in TensorFlow?

I were only able to find GRU- or LSTM RNN examples for TensorFlow, mostly for NLP. Does anyone know of some simple recurrent neural network tutorials or examples for TensorFlow?

This figure shows a basic Elman network, which is often simply called SRN (simple recurrent network):

---

One option is to use the built-in RNNCell located in tensorflow/python/ops/rnn_cell.py.

If you don't want to do that you can make your own RNN. The RNN will train using back-propagation through time. Try unrolling the network a fixed number of steps, eg consider input sequences of length ten. Then you can write a loop in python to do all of the matrix multiplications for each step of the network. Each time you can take the output from the previous step and concatenate it with the input to that step. It will not be too many lines of code to get this working.

---

One other option is to make use of the scan function of Tensorflow for implementing the recurrence. My result (which is a Simple Recurrent Neural Network / Elman network) can be found here: https://www.data-blogger.com/2017/05/17/elman-rnn-implementation-in-tensorflow/

链接地址: http://www.djcxy.com/p/32076.html

上一篇: 预训练嵌入类型错误

下一篇: TensorFlow：简单循环神经网络