The neural network trained below was created for the MNIST digit recognition challenge on Kaggle. It produced a public test set accuracy of 0.99700, earning me the 94th spot out of approximately 2100 entries. This is in the top 5% of scores and well above a line in the rankings at an accuracy of 0.99514 which states “Something Fishy Going On If You’re Above This”. I hope you’ll agree that my model isn’t doing anything fishy, it just squeezes the data generation techniques available in Keras for everything they are worth.
Importing and Splitting the Data into Training and Validation Sets
Before diving into the modelling steps, it is necessary to create a new training set and a validation set. This will allow for a fair assessment of how well the model is doing on the more general problem of digit recognition without the effects of overfitting on the training data. It also provides an opportunity to tune various parameters of the neural network such as the batch size, number of epochs for training, data generation strategy, and model architecture. Importing the provided data and then splitting it into the new training and validation sets is done using pandas and scikit-learn.
Training the Convolutional Neural Network
The model below leverages Keras’ built in ImageDataGenerator method with shifts in both dimensions, rotations, shear mapping, and zooming in and out. These randomly distributed transformations are useful to extend the 37,800 training samples we were provided with into a much more diverse training set. By showing the model variations of the original training images, we are also hoping that the neural network will become more robust and less overfit to patterns specific to the data we have. To this end, the model also makes extensive use of dropout, another technique designed to prevent overfitting. The hope is that all of these techniques will allow the neural network to successfully generalize to other digits it hasn’t see before.