Exercise 1: TensorFlow Playground#
For this exercise, go to playground.tensorflow.org by clicking on the image below. The purpose of this exercise is to get comfortable with neural networks (and how they fail) in an interactive, manual way that builds intutiion before we start working with them in code.
Press the “play” button to start training the network. The colored background (the model) should progressively become a better fit to the points (the training data).
Next, try each of the following:
Increase the noise in the data and fit it again.
Select the “circle” dataset and attempt to fit it again. Can you fit it if you add hidden layers? What’s the smallest number of hidden layers (and nodes) you need to fit the data?
Do the same for the “xor” dataset.
Do the same for the “spiral” dataset.
What if you use different features instead? Repeat 2‒4 using features instead of hidden layers. What’s the smallest set of features you can use for each, and why do they work?
What about different activation functions?
We’ll discuss regularization later, but play with it and see if you can figure out what it does. Try making a neural network with too many layers and nodes for a simple problem (such as the linearly separable blobs) and turn on L1 regularization with a mid-range regularization rate. What happens if you let it run for a while?
Other things to explore: the 2 regression problems,
the learning rate,
and the batch size,
Have fun!