LSTM output

This post has two pop quizzes relating to the output of LSTM.

Q1) What is the output of the following model prediction?

1
2
3
4
5


model = Sequential()
model.add(LSTM(32, input_shape=(10,5)))
model.compile(optimizer='adam', loss='mse')
data = np.random.randn(1,10,5)
model.predict(data).shape

It is (1,32)

Q2) What is the output of the following model prediction?

1
2
3
4
5


model = Sequential()
model.add(LSTM(32, input_shape=(10,5), return_sequences=True))
model.compile(optimizer='adam', loss='mse')
data = np.random.randn(1,10,5)
model.predict(data).shape

It is (1,10,32) The reason for this is that we have specified an argument return_sequence as True is that LSTM cell will output a value for each of the time steps mentioned. Since there are 10 time steps, there are 10 values from each of the LSTM cell resulting in 320 values taking in to consideration 32 cells.

This is a very important thing to keep in mind, while using LSTM

Contents