Understanding_and_Creating_Binary_Classification_NNs

Nothing But NumPy: Understanding & Creating Neural Networks with Computational Graphs from Scratch

This repository is part of the blog post "Nothing but NumPy: Understanding & Creating Binary Classification Neural Networks with Computational Graphs from Scratch"

Forks rather than Clones, Stars rather than Views 🙏

Layers

The Layers directory contains the classes for:

1. Linear Layer
2. Activation Layer:
   • Sigmoid Layer

util

util directory contains utility fuctions. It has the following files with the following functions:

1. paramInitializer.py

   • initialize_parameters(n_in, n_out, ini_type='plain')

2. utilities.py

   • predict(X, Y, Zs, As, thresh=0.5)
   • plot_learning_curve(costs, learning_rate, total_epochs, save=False)
   • predict_dec(Zs, As, X, thresh=0.5)
   • plot_decision_boundary(model, X, Y, feat_crosses=None, axis_lines=False,save=False)
   • plot_decision_boundary_shaded(model, X, Y, feat_crosses=None, axis_lines=False,save=False)
   • plot_decision_boundary_distances(model, X, Y, feat_crosses=None, axis_lines=False, save=False)

3. cost_functions.py

   • compute_bce_cost(Y, P_hat)
   • compute_stable_bce_cost(Y, Z)
   • compute_keras_like_bce_cost(Y, P_hat, from_logits=False)
   • compute_mse_cost(Y, Y_hat)

---

The following examples have been implemented:

1. A 1-layer neural network on AND data that uses MSE Cost

   This notebook shows how easy it is to break backpropagation when Mean Squared Error(MSE) Cost function is used in a binary classification setting. The following neural network architecture has been implemented:

   

2. A 2-layer neural network on XOR data that uses Keras-like BCE Cost

   This notebook trains a 2-layer neural net on XOR data using the Keras-like Binary Cross-Entropy(BCE) Cost function. The following neural network architecture has been implemented:

   

3. A 1-layer neural network on Iris flower petals data

   This notebook trains a 1-layer neural net to classify "iris-versicolor flower vs. other flowers" using only petal length & width as features and the "stable" BCE Cost function. The following neural network architecture has been implemented:

   

4. A 2-layer neural network on all Iris flowers data

   This notebook trains a 2-layer neural net to classify "iris-versicolor flower vs. other flowers" using all of the 4 flower features and the "stable" BCE Cost function. The following neural network architecture has been implemented:

   

5. A 3-layer neural network on Iris flower sepals data

   This notebook trains a 3-layer neural net to classify "iris-versicolor flower vs. other flowers" using only sepal length & width as features and the "stable" BCE Cost function. The following neural network architecture has been implemented:

   

---

Find me on twitter for any questions or suggestions.