Train a Simple RNN to track a simple shift

In this article, I will explain the way you can code a simple RNN that tracks a simple shift in the pattern Create Training and Validation Data 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 import numpy as np import re from sklearn.model_selection import train_test_split import tensorflow as tf input_seed = 1234 time_steps = 24 n_samples = 100000 X = np.random.randint(1,30,n_samples*time_steps).reshape(n_samples, time_steps) Y = np.

Train a Simple RNN to track cumulative sums

In this article, I will explain the way you can code a simple RNN that tracks cumulative sums of a sequence Create Training and Validation Data 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 import numpy as np import re from sklearn.model_selection import train_test_split import tensorflow as tf input_seed = 1234 time_steps = 24 n_samples = 100000 X = np.random.randint(1,30,n_samples*time_steps).reshape(n_samples, time_steps) Y = np.

Train an XOR using Simple RNN

I have been struggling to implement parity of sequence since many weeks. Finally after going through Geron’s book, I am now able to successfully implement an algo that learns the parity of a sequence. This does not use the classification tweak that many apply to solve the parity problem Create Training and Validation Data 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 import numpy as np import re from sklearn.

Example of why word embeddings matter

The word “Bond” can depend on the context that you come across. If it is a community news letter, the bond could be used in the context of “Walk to Bond”, “Run to Bond”, “Meet to Bond”. Here the word “Bond”, means bonding. In the context of Financial news, “Bond” could in all probability mean a financial instrument. If you have a word2vec model, “Bond” is collapsed in to one dimension and hence loses all the context.

The Transformer

The following are the learnings from the podcast: Transfer learning entails reusing existing models. Use the model that comes from training on different tasks Value delivery through custom feature engineering is not required. Most of the recent successes are in the field of computer vision If you do not have a lot of training data, then you can use a model that is already trained on a large image dataset(ImageNet).