Keras_Models_for_General_Machine_Learning_2.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "provenance": [],
      "authorship_tag": "ABX9TyPbQ9LUmOIkM1RYTVV2whVV",
      "include_colab_link": true
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    },
    "language_info": {
      "name": "python"
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/Ash100/Biopython/blob/main/Keras_Models_for_General_Machine_Learning_2.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "##Using Your Keras Models with Scikit Learn for Optimization\n",
        "I am **Dr. Ashfaq Ahmad**, and this notebook is created for teaching and research purposes. Refering to the people working in the field of Biology, I have tried my level best to keep it as simple as possible. For Detailed instruction and understandings, please watch a video tutorial on **https://www.youtube.com/@Bioinformaticsinsights**\n",
        "\n",
        "This notebook is based on the book **Deep Learning with Python** by Jason Brownlee."
      ],
      "metadata": {
        "id": "CCj-Ooyra39I"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "#Overview of this Notebook\n",
        "Keras is a popular library for deep learning in Python. The scikit-learn library in Python is built upon the SciPy stack for numerical computation. Here we will learn,\n",
        "Evaluation of models using resampling methods like k-fold cross validation.\n",
        "Evaluation of model hyperparameters.\n"
      ],
      "metadata": {
        "id": "If0q_xvllQuA"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "In this sections we will work through examples of using the *KerasClassifier wrapper* for a classification neural network created in Keras and used in the scikit-learn library."
      ],
      "metadata": {
        "id": "fGl0qiegl1J9"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "#1. Evaluation of Model with Cross-Validation\n",
        "The KerasClassifier and KerasRegressor classes in Keras take an argument *build_fn* which is the name of the function to call to create your model. First we will make a model with a function *create model()* that create a simple multilayer neural network for theproblem.\n",
        "Next, We will pass this function name to the KerasClassifier class by the *build_fn* argument.\n",
        "We also pass in additional arguments of epoch=150 and batch size=10 in *fit()* function which is called internally by the\n",
        "KerasClassifier class. In this example we use the scikit-learn StratifiedKFold to perform 10-fold stratified cross validation. This technique will provide a robust\n",
        "estimate of the performance of a machine learning model on unseen data."
      ],
      "metadata": {
        "id": "eq2NTvQRmQ8T"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "#Some required installation"
      ],
      "metadata": {
        "id": "yCqZtWgwkw5e"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "!pip install --upgrade keras"
      ],
      "metadata": {
        "id": "kOsGwsG2cOTY"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "!pip install --upgrade scikit_learn"
      ],
      "metadata": {
        "id": "3QDx_0QvdR0o"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "From the following, install the specific one, as per your need"
      ],
      "metadata": {
        "id": "S_6oW4wXk2yu"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "!pip install scikeras[tensorflow-cpu]"
      ],
      "metadata": {
        "id": "v9ScZjx8en-g"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "!pip install scikeras[tensorflow]      # gpu compute platform\n"
      ],
      "metadata": {
        "id": "UnkbTbHkeyZr"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "# MLP for Pima Indians Dataset with 10-fold cross validation via sklearn\n",
        "from keras.models import Sequential\n",
        "from keras.layers import Dense\n",
        "from tensorflow.keras.wrappers.scikit_learn import KerasClassifier\n",
        "from scikeras.wrappers import KerasClassifier\n",
        "from sklearn.model_selection import StratifiedKFold\n",
        "from sklearn.model_selection import cross_val_score\n",
        "import numpy"
      ],
      "metadata": {
        "id": "_X-uApXzcHlb"
      },
      "execution_count": 1,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "# Function to create model, required for KerasClassifier\n",
        "def create_model():\n",
        "    # create model\n",
        "    model = Sequential()\n",
        "    model.add(Dense(12, input_dim=8, activation='relu'))\n",
        "    model.add(Dense(8, activation='relu'))\n",
        "    model.add(Dense(1, activation='sigmoid'))\n",
        "    # Compile model\n",
        "    model.compile(loss='binary_crossentropy', optimizer='adam', metrics=['accuracy'])\n",
        "    return model"
      ],
      "metadata": {
        "id": "mYzLOH6Ufc8i"
      },
      "execution_count": 3,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "# fix random seed for reproducibility\n",
        "seed = 7\n",
        "numpy.random.seed(seed)\n",
        "# load pima indians dataset\n",
        "dataset = numpy.loadtxt(\"/content/sample_data/diabetes_1.csv\", delimiter=\",\")\n",
        "# split into input (X) and output (Y) variables\n",
        "X = dataset[:,0:8]\n",
        "Y = dataset[:,8]\n",
        "# create model\n",
        "model = KerasClassifier(build_fn=create_model, epochs=150, batch_size=10, verbose=0)\n",
        "# evaluate using 10-fold cross validation\n",
        "kfold = StratifiedKFold(n_splits=10, shuffle=True, random_state=seed)\n",
        "results = cross_val_score(model, X, Y, cv=kfold)\n",
        "print(results.mean())"
      ],
      "metadata": {
        "id": "5n--fu_Qf8Wb"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "#2. Grid Search Deep Learning Model Parameters\n",
        "We already know that we can provide arguments to the *fit()* function. We can use these arguments to **further customize the construction of the model**.\n",
        "In this example we use a grid search to evaluate different configurations for our neural network model and adjust the combination that provides the best estimated performance.\n",
        "\n",
        "The *create_model()* function is defined to take two arguments, we will tweak some settings to see evaluation and optimization schemes for our model."
      ],
      "metadata": {
        "id": "5mwtI3m-nrqx"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# Function to create model, required for KerasClassifier\n",
        "def create_model():\n",
        "    # create model\n",
        "    model = Sequential()\n",
        "    model.add(Dense(12, input_dim=8, activation='relu'))\n",
        "    model.add(Dense(8, activation='relu'))\n",
        "    model.add(Dense(1, activation='sigmoid'))\n",
        "    # Compile model\n",
        "    model.compile(loss='binary_crossentropy', optimizer='adam', metrics=['accuracy'])\n",
        "    return model"
      ],
      "metadata": {
        "id": "8QfAYACGi6ES"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "# fix random seed for reproducibility\n",
        "import numpy\n",
        "seed = 7\n",
        "numpy.random.seed(seed)\n",
        "\n",
        "# Load the necessary modules\n",
        "from tensorflow.keras.models import Sequential\n",
        "from tensorflow.keras.layers import Dense\n",
        "from tensorflow.keras.wrappers.scikit_learn import KerasClassifier\n",
        "from sklearn.model_selection import GridSearchCV\n",
        "\n",
        "# Function to create model, required for KerasClassifier\n",
        "def create_model(optimizer='adam', init='glorot_uniform'):\n",
        "    # create model\n",
        "    model = Sequential()\n",
        "    model.add(Dense(12, input_dim=8, kernel_initializer=init, activation='relu'))\n",
        "    model.add(Dense(8, kernel_initializer=init, activation='relu'))\n",
        "    model.add(Dense(1, kernel_initializer=init, activation='sigmoid'))\n",
        "    # Compile model\n",
        "    model.compile(loss='binary_crossentropy', optimizer=optimizer, metrics=['accuracy'])\n",
        "    return model\n",
        "\n",
        "# load pima indians dataset\n",
        "dataset = numpy.loadtxt(\"/content/sample_data/diabetes_1.csv\", delimiter=\",\")\n",
        "# split into input (X) and output (Y) variables\n",
        "X = dataset[:,0:8]\n",
        "Y = dataset[:,8]\n",
        "\n",
        "# create model\n",
        "model = KerasClassifier(build_fn=create_model, verbose=0)\n",
        "\n",
        "# grid search epochs, batch size and optimizer\n",
        "optimizers = ['rmsprop', 'adam']\n",
        "init = ['glorot_uniform', 'normal', 'uniform']\n",
        "epochs = [50, 100, 150]\n",
        "batches = [5, 10, 20]\n",
        "param_grid = dict(optimizer=optimizers, epochs=epochs, batch_size=batches, init=init)\n",
        "grid = GridSearchCV(estimator=model, param_grid=param_grid)\n",
        "grid_result = grid.fit(X, Y)\n",
        "\n",
        "# summarize results\n",
        "print(\"Best: %f using %s\" % (grid_result.best_score_, grid_result.best_params_))\n",
        "means = grid_result.cv_results_['mean_test_score']\n",
        "stds = grid_result.cv_results_['std_test_score']\n",
        "params = grid_result.cv_results_['params']\n",
        "for mean, stdev, param in zip(means, stds, params):\n",
        "    print(\"%f (%f) with: %r\" % (mean, stdev, param))"
      ],
      "metadata": {
        "id": "RQrTnSobjQsd"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "#What we get?\n",
        "Now, we came to know that a batch size with 5, and 150 epochs. Our model performs better."
      ],
      "metadata": {
        "id": "RVCH6udW0WwY"
      }
    }
  ]
}