OverviewTeaching: 5 min
Exercises: 15 minQuestions
What other machine learning libraries can I use?
How do classifiers built with different libraries compare?Objectives
Train a neural network model in TensorFlow.
Compare your TensorFlow neural network with your other classifiers.
In this section we will examine the fully connected neural network (NN) in TensorFlow, rather than PyTorch.
TensorFlow is an end-to-end open-source platform for machine learning. It is used for building and deploying machine learning models. See the documentation.
TensorFlow does have GPU support, therefore can be used train complicated neural network model that require a lot of GPU power.
TensorFlow is also interoperable with other packages discussed so far, datatypes from NumPy and pandas are often used in TensorFlow.
Here we will import all the required TensorFlow libraries for the rest of the tutorial.
from tensorflow import keras # tensorflow wrapper from tensorflow.random import set_seed # import set_seed function for TensorFlow set_seed(seed_value) # set TensorFlow random seed
From the previous pages, we already have our data in NumPy arrays, like
X, that can be used in TensorFlow.
To use a neural network with TensorFlow, we modularize its construction using a function. We will later pass this function into a Keras wrapper.
def build_model( n_hidden=1, n_neurons=5, learning_rate=1e-3 ): # function to build a neural network model # Build model = keras.models.Sequential() # initialise the model for layer in range(n_hidden): # loop over hidden layers model.add( keras.layers.Dense(n_neurons, activation="relu") ) # add layer to your model model.add(keras.layers.Dense(2, activation="softmax")) # add output layer # Compile optimizer = keras.optimizers.SGD( learning_rate=learning_rate ) # define the optimizer model.compile( loss="sparse_categorical_crossentropy", optimizer=optimizer, metrics=["accuracy"], ) # compile your model return model
For now, ignore all the complicated hyperparameters, but note that the loss used is
sparse_categorical_crossentropy; the neural network uses gradient descent to optimize its parameters (in this case these parameters are neuron weights).
We need to keep some events for validation. Validation sets are used to select and tune the final neural network model.
X_valid_scaled, X_train_nn_scaled = ( X_train_scaled[:100], X_train_scaled[100:], ) # first 100 events for validation
Assign the first 100 events in y_train for validation (
y_valid) and the rest for training your neural network (
y_valid, y_train_nn = y_train[:100], y_train[100:] # first 100 events for validation
With these parameters, the network can be trained as follows:
tf_clf = keras.wrappers.scikit_learn.KerasClassifier( build_model ) # call the build_model function defined earlier tf_clf.fit( X_train_nn_scaled, y_train_nn, validation_data=(X_valid_scaled, y_valid) ) # fit your neural network
Get the predicted y values for the TensorFlow neural network,
y_pred_tf. Once you have
y_pred_tf, see how well your TensorFlow neural network classifier does using accurarcy_score.
y_pred_tf = tf_clf.predict(X_test_scaled) # make predictions on the test data # See how well the classifier does print(accuracy_score(y_test, y_pred_tf))
The TensorFlow neural network should also have a similar accuracy score to the PyTorch neural network.
As shown in the lesson on Overfitting check, make the same overfitting check for your TensorFlow neural network and decide whether any overfitting is present.
compare_train_test(tf_clf, X_train_nn_scaled, y_train_nn, X_test_scaled, y_test, 'TensorFlow Neural Network output')
Print the same classification report as the Model Comparison lesson for your TensorFlow neural network.
# TensorFlow Neural Network Report print (classification_report(y_test, y_pred_tf, target_names=["background", "signal"]))
As in the Model Comparison lesson, get the decisions of the TensorFlow neural network classifier,
decisions_tf = tf_clf.predict_proba(X_test_scaled)[:,1] # get the decisions of the TensorFlow neural network
Get the FPRs, TPRs and thresholds for the TensorFlow neural network classifier (
thresholds_tf), as was done in the Model Comparison lesson.
fpr_tf, tpr_tf, thresholds_tf = roc_curve(y_test, decisions_tf) # get FPRs, TPRs and thresholds for TensorFlow neural network
Modify your ROC curve plot from the Model Comparison lesson to include another line for the TensorFlow neural network
plt.plot(fpr_rf, tpr_rf, label='Random Forest') # plot random forest ROC plt.plot(fpr_nn, tpr_nn, linestyle='dashed', label='PyTorch Neural Network') # plot PyTorch neural network ROC plt.plot(fpr_tf, tpr_tf, linestyle='dashdot', label='TensorFlow Neural Network') # plot TensorFlow neural network ROC plt.plot([0, 1], [0, 1], linestyle='dotted', color='grey', label='Luck') # plot diagonal line to indicate luck plt.xlabel('False Positive Rate') # x-axis label plt.ylabel('True Positive Rate') # y-axis label plt.grid() # add a grid to the plot plt.legend() # add a legend
Use the function you defined in the Model Comparison lesson to get the AMS score for your TensorFlow neural network classifier,
ams_tf = AMS(tpr_tf, fpr_tf, 0.001) # get AMS for TensorFlow neural network
Modify your threshold plot from the Model Comparison lesson to include another line for the TensorFlow neural network
plt.plot(thresholds_rf, ams_rf, label='Random Forest') # plot random forest AMS plt.plot(thresholds_nn, ams_nn, linestyle='dashed', label='PyTorch Neural Network') # plot PyTorch neural network AMS plt.plot(thresholds_tf, ams_tf, linestyle='dashdot', label='TensorFlow Neural Network') # plot TensorFlow neural network AMS plt.xlabel('Threshold') # x-axis label plt.ylabel('AMS') # y-axis label plt.title('AMS with $b_r=0.001$') # add plot title plt.legend() # add a legend
All things considered, how does your TensorFlow neural network compare to your PyTorch neural network and scikit-learn random forest?
TensorFlow is another good option for machine learning in Python.