Understand_Model_Behavior_in_Training_8.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "provenance": [],
      "gpuType": "T4",
      "authorship_tag": "ABX9TyM34afawcTgh2Ypghzx/gJ4",
      "include_colab_link": true
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    },
    "language_info": {
      "name": "python"
    },
    "accelerator": "GPU"
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/Ash100/Biopython/blob/main/Understand_Model_Behavior_in_Training_8.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "##Understanding Model Behavior in Training\n",
        "I am **Dr. Ashfaq Ahmad**, and this notebook is created for teaching and research purposes. Refering to the people working in the field of Biology, I have tried my level best to keep it as simple as possible. For Detailed instruction and understandings, please watch a video tutorial on **https://www.youtube.com/@Bioinformaticsinsights**\n",
        "\n",
        "This notebook is based on the book **Deep Learning with Python** by Jason Brownlee."
      ],
      "metadata": {
        "id": "UtSyA5hRV9wO"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "##Aims and Objectives\n",
        "You can learn a lot about neural networks and deep learning models by observing their performance over time during training. In this lesson you will discover how you can review and visualize the performance of deep learning models over time during training in Python with\n",
        "Keras. After completing this lesson you will know:\n",
        "\n",
        "How to inspect the history metrics collected during training.\n",
        "\n",
        "How to plot accuracy metrics on training and validation datasets during training.\n",
        "\n",
        "How to plot model loss metrics on training and validation datasets during training.\n",
        "Let’s get started."
      ],
      "metadata": {
        "id": "dzaBPJFKWK32"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "Keras provides the capability to register callbacks when training a deep learning model. One of the default callbacks that is registered when training all deep learning models is the History callback. It records training metrics for each epoch. This includes the loss and the accuracy (for classification problems) as well as the loss and accuracy for the validation dataset, if one is set.\n",
        "The history object is returned from calls to the *fit()* function used to train the model."
      ],
      "metadata": {
        "id": "ZW1t32uCWuPu"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "We can use the data collected in the history object to create plots. The plots can provide an indication of useful things about the training of the model, such as:\n",
        "It’s speed of convergence over epochs (slope).\n",
        "\n",
        "Whether the model may have already converged (plateau of the line).\n",
        "\n",
        "Whether the model may be over-learning the training data (inflection for validation line)."
      ],
      "metadata": {
        "id": "68bnMdOeXQGN"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "##Visualize Model History in Keras\n",
        "We can create plots from the collected history data. In the example below we create a small network to model the Pima Indians onset of diabetes binary classification problem. The example collects the history, returned from training the model and creates\n",
        "two charts:\n",
        "1. A plot of accuracy on the training and validation datasets over training epochs.\n",
        "2. A plot of loss on the training and validation datasets over training epochs."
      ],
      "metadata": {
        "id": "PowpaduhXetR"
      }
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "ZDrDx9_mV9Ao"
      },
      "outputs": [],
      "source": [
        "#Install Keras\n",
        "!pip install keras"
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "from keras.models import Sequential\n",
        "from keras.layers import Dense\n",
        "import matplotlib.pyplot as plt\n",
        "import numpy\n",
        "\n",
        "# fix random seed for reproducibility\n",
        "seed = 7\n",
        "numpy.random.seed(seed)\n",
        "\n",
        "# load pima indians dataset\n",
        "dataset = numpy.loadtxt(\"/content/sample_data/diabetes_1.csv\", delimiter=\",\")\n",
        "# split into input (X) and output (Y) variables\n",
        "X = dataset[:,0:8]\n",
        "Y = dataset[:,8]\n",
        "\n",
        "# create model\n",
        "model = Sequential()\n",
        "model.add(Dense(12, input_dim=8, kernel_initializer='uniform', activation='relu'))\n",
        "model.add(Dense(8, kernel_initializer='uniform', activation='relu'))\n",
        "model.add(Dense(1, kernel_initializer='uniform', activation='sigmoid'))\n",
        "\n",
        "# Compile model\n",
        "model.compile(loss='binary_crossentropy', optimizer='adam', metrics=['accuracy'])\n",
        "\n",
        "# Fit the model\n",
        "history = model.fit(X, Y, validation_split=0.33, epochs=150, batch_size=10, verbose=0)\n",
        "\n",
        "# list all data in history\n",
        "print(history.history.keys())\n",
        "\n",
        "# summarize history for accuracy\n",
        "plt.plot(history.history['accuracy'])\n",
        "plt.plot(history.history['val_accuracy'])\n",
        "plt.title('Model accuracy')\n",
        "plt.ylabel('Accuracy')\n",
        "plt.xlabel('Epoch')\n",
        "plt.legend(['Train', 'Test'], loc='upper left')\n",
        "plt.show()\n",
        "\n",
        "# summarize history for loss\n",
        "plt.plot(history.history['loss'])\n",
        "plt.plot(history.history['val_loss'])\n",
        "plt.title('Model loss')\n",
        "plt.ylabel('Loss')\n",
        "plt.xlabel('Epoch')\n",
        "plt.legend(['Train', 'Test'], loc='upper left')\n",
        "plt.show()"
      ],
      "metadata": {
        "id": "75h-F9zyYNwL"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "The plots are provided Above.\n",
        "**1. Figure 1** The history for the validation dataset is labeled test by convention as it is indeed a test dataset for the model. From the plot of accuracy we can see that the model could probably be trained a little more as the trend for accuracy on both datasets is still rising for the last few epochs. We can also see that the model has not yet over-learned the training dataset, showing comparable skill on both datasets.\n",
        "**2. Figure 2**. From the plot of loss, we can see that the model has comparable performance on both train\n",
        "and validation datasets (labeled test). If these parallel plots start to depart consistently, it might be a sign to stop training at an earlier epoch."
      ],
      "metadata": {
        "id": "5I3VjQAXZZlB"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "**I suggest to read further, there are sufficient literature available**"
      ],
      "metadata": {
        "id": "i0u1lvZxaYsG"
      }
    },
    {
      "cell_type": "code",
      "source": [],
      "metadata": {
        "id": "RYmQ5ehLaWEs"
      },
      "execution_count": null,
      "outputs": []
    }
  ]
}