Module 01 Lab 03 Augmentation.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "provenance": [],
      "include_colab_link": true
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    },
    "language_info": {
      "name": "python"
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/Nitin-Sagar-B/FMML_PROJECTS_AND_LABS/blob/main/Module%2001%20Lab%2003%20Augmentation.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "w3yfry25JgZK"
      },
      "source": [
        "# Data augmentation\n",
        "\n",
        "FMML Module 1, Lab 3<br>\n",
        "\n",
        " In this lab, we will see how augmentation of data samples help in improving the machine learning performance"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "xZU8_elooqP0"
      },
      "source": [
        "import numpy as np\n",
        "import matplotlib.pyplot as plt\n",
        "from keras.datasets import mnist\n",
        "# set randomseed\n",
        "rng = np.random.default_rng(seed=42)\n",
        "from sklearn.utils.extmath import cartesian\n",
        "from skimage.transform import rotate, AffineTransform, warp\n",
        "import math"
      ],
      "execution_count": 1,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "T5pHYogSMHiE"
      },
      "source": [
        "Augmentation is useful when we have less training data available. Augmentation allows us to 'create' a larger dataset programatically.\n",
        "\n",
        "For this lab we will use a subset of MNIST that is very small, to better understand the effect of augmentation."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "gJvmWJ58ovx5",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "266f65a5-2651-46f2-ef3f-1c2363f7586b"
      },
      "source": [
        "#loading the dataset\n",
        "(train_X, train_y), (test_X, test_y) = mnist.load_data()\n",
        "train_X = train_X/255\n",
        "test_X = test_X/255\n",
        "\n",
        "train_X = train_X[::1200,:,:].copy() # subsample. Otherwise it will take too long!\n",
        "train_y = train_y[::1200].copy() # do the same to the labels"
      ],
      "execution_count": 2,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Downloading data from https://storage.googleapis.com/tensorflow/tf-keras-datasets/mnist.npz\n",
            "11490434/11490434 [==============================] - 0s 0us/step\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "8XamH6z1Rt7S"
      },
      "source": [
        "Let us borrow a few functions from the previous labs:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Zk2W5_3BRLMS"
      },
      "source": [
        "def NN1(traindata, trainlabel, query):\n",
        "  diff  = traindata - query  # find the difference between features. Numpy automatically takes care of the size here\n",
        "  sq = diff*diff # square the differences\n",
        "  dist = sq.sum(1) # add up the squares\n",
        "  label = trainlabel[np.argmin(dist)] # our predicted label is the label of the training data which has the least distance from the query\n",
        "  return label\n",
        "\n",
        "def NN(traindata, trainlabel, testdata):\n",
        "  # we will run nearest neighbour for each sample in the test data\n",
        "  # and collect the predicted classes in an array using list comprehension\n",
        "  traindata = traindata.reshape(-1, 28*28)\n",
        "  testdata = testdata.reshape(-1,28*28)\n",
        "  predlabel = np.array([NN1(traindata, trainlabel, i) for i in testdata])\n",
        "  return predlabel\n",
        "\n",
        "def Accuracy(gtlabel, predlabel):\n",
        "  assert len(gtlabel)==len(predlabel), \"Length of the groundtruth labels and predicted labels should be the same\"\n",
        "  correct = (gtlabel==predlabel).sum() # count the number of times the groundtruth label is equal to the predicted label.\n",
        "  return correct/len(gtlabel)"
      ],
      "execution_count": 3,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "eGiA3LmDSJZo"
      },
      "source": [
        "In this lab, we will use the image pixels themselves as features, instead of extracting features. Each image has 28*28 pixels, so we will flatten them to 784 pixels to use as features. Note that this is very compute intensive and will take a long time.<br>\n",
        "\n",
        "Let us check the baseline accuracy on the test set without any augmentations. We hope that adding augmentations will help us to get better results."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "4tQvnoasRNEV",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "a43a8e4d-8f2f-490d-9e83-1ee44c990606"
      },
      "source": [
        "testpred = NN(train_X, train_y, test_X)\n",
        "print('Baseline accuracy without augmentation is ', Accuracy(test_y, testpred))"
      ],
      "execution_count": 4,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Baseline accuracy without augmentation is  0.6472\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ZfkcMfhIZQ7U"
      },
      "source": [
        "Let us try to improve this accuracy using augmentations. When we create augmentations, we have to make sure that the changes reflect what will naturally occur in the dataset. For example, we should not add colour to our samples as an augmentation because they do not naturally occur. We should not also flip the images in MNIST, because flipped images have different meanings for digits.\n",
        "\n",
        "### Augmentation 1: rotation\n",
        "\n",
        "Let us try rotating the image a little. We will use skimage library for this."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "z5WolJ9fZE7L",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 860
        },
        "outputId": "851cc8bc-8f66-4fab-863c-19874d3ec3bc"
      },
      "source": [
        "plt.imshow(train_X[2], cmap='gray')\n",
        "plt.show()\n",
        "plt.imshow(rotate(train_X[2],25), cmap='gray')"
      ],
      "execution_count": 5,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "<Figure size 640x480 with 1 Axes>"
            ],
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAaAAAAGdCAYAAABU0qcqAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/bCgiHAAAACXBIWXMAAA9hAAAPYQGoP6dpAAAb+UlEQVR4nO3df2xV9f3H8dctP66g7cVa++NKwYI/2EBqROkaleFoKHUhVsnir2XI/DGwuCn+WOpUdJJ0w2wal06XzFDdRMVlgJqlC1RbMldwIAx/bA3tqi3pDyaGe0uxhdDP9w++3nmlBc7l3r572+cj+SS955z3PW8OJ/fVc+/p5/qcc04AAAyyFOsGAAAjEwEEADBBAAEATBBAAAATBBAAwAQBBAAwQQABAEwQQAAAE6OtG/i6vr4+tbW1KTU1VT6fz7odAIBHzjl1dXUpGAwqJWXg65whF0BtbW3Kzc21bgMAcJpaW1s1ceLEAdcPubfgUlNTrVsAAMTByV7PExZAlZWVOv/883XGGWeooKBA77333inV8bYbAAwPJ3s9T0gAvfbaa1qxYoVWrlyp999/X/n5+SouLta+ffsSsTsAQDJyCTB79mxXVlYWeXz06FEXDAZdRUXFSWtDoZCTxGAwGIwkH6FQ6ISv93G/Ajp8+LB27NihoqKiyLKUlBQVFRWpvr7+uO17e3sVDoejBgBg+It7AH322Wc6evSosrKyopZnZWWpo6PjuO0rKioUCAQigzvgAGBkML8Lrry8XKFQKDJaW1utWwIADIK4/x1QRkaGRo0apc7OzqjlnZ2dys7OPm57v98vv98f7zYAAENc3K+Axo4dq1mzZqmmpiayrK+vTzU1NSosLIz37gAASSohMyGsWLFCixcv1uWXX67Zs2frmWeeUXd3t5YsWZKI3QEAklBCAujGG2/Uf//7Xz322GPq6OjQpZdequrq6uNuTAAAjFw+55yzbuKrwuGwAoGAdRsAgNMUCoWUlpY24Hrzu+AAACMTAQQAMEEAAQBMEEAAABMEEADABAEEADBBAAEATBBAAAATBBAAwAQBBAAwQQABAEwQQAAAEwQQAMAEAQQAMEEAAQBMEEAAABMEEADABAEEADBBAAEATBBAAAATBBAAwAQBBAAwQQABAEwQQAAAEwQQAMAEAQQAMEEAAQBMEEAAABMEEADABAEEADBBAAEATBBAAAATBBAAwAQBBAAwQQABAEwQQAAAEwQQAMAEAQQAMEEAAQBMEEAAABMEEADABAEEADBBAAEATBBAAAATBBAAwAQBBAAwQQABAEwQQAAAEwQQAMAEAQQAMEEAAQBMEEAAABOjrRtA8vL7/Z5rfvjDHyagk+Olp6fHVPfkk0/GuZP+/epXv/JcU1VV5bnmo48+8lwDDBaugAAAJgggAICJuAfQ448/Lp/PFzWmTZsW790AAJJcQj4Dmj59ujZv3vy/nYzmoyYAQLSEJMPo0aOVnZ2diKcGAAwTCfkMaM+ePQoGg5oyZYpuvfVWtbS0DLhtb2+vwuFw1AAADH9xD6CCggJVVVWpurpazz33nJqbm3X11Verq6ur3+0rKioUCAQiIzc3N94tAQCGoLgHUElJib73ve9p5syZKi4u1l/+8hcdOHBA69at63f78vJyhUKhyGhtbY13SwCAISjhdwdMmDBBF110kRobG/td7/f7Y/qDRgBAckv43wEdPHhQTU1NysnJSfSuAABJJO4B9MADD6iurk6ffPKJ/v73v+v666/XqFGjdPPNN8d7VwCAJBb3t+D27t2rm2++Wfv379e5556rq666Slu3btW5554b710BAJKYzznnrJv4qnA4rEAgYN3GiHLZZZfFVPfiiy96rvnmN78Z074gtbW1ea6J9Z2HnTt3eq7p7u6OaV8YvkKhkNLS0gZcz1xwAAATBBAAwAQBBAAwQQABAEwQQAAAEwQQAMAEAQQAMEEAAQBMEEAAABMEEADABAEEADBBAAEATCT8C+kw9JWVlcVUx8SigysYDHqu2bJlS0z7evrppz3X3H///THtCyMXV0AAABMEEADABAEEADBBAAEATBBAAAATBBAAwAQBBAAwQQABAEwQQAAAEwQQAMAEAQQAMEEAAQBMEEAAABPMhg1dfvnl1i2cUGNjo+eaP/3pTwnopH/Tp0/3XLNw4cIEdBI/paWlnmteeOEFzzUff/yx5xoMH1wBAQBMEEAAABMEEADABAEEADBBAAEATBBAAAATBBAAwAQBBAAwQQABAEwQQAAAEwQQAMAEAQQAMMFkpBjyGhoaPNf87Gc/S0An/TvzzDM916xatcpzzXXXXee55vzzz/dcE2tdbW2t55r169d7rnnjjTc81/zjH//wXCNJ+/bti6kOp4YrIACACQIIAGCCAAIAmCCAAAAmCCAAgAkCCABgggACAJgggAAAJgggAIAJAggAYIIAAgCYIIAAACZ8zjln3cRXhcNhBQIB6zZGlCVLlsRU9+yzz3quGT9+vOeaWCaEXLhwoecaSdq+fXtMdV5lZ2d7rlm9erXnmu9///ueayRpiL0snLZY/19jOfcOHTrkuebhhx/2XNPU1OS5ZrCFQiGlpaUNuJ4rIACACQIIAGDCcwBt2bJFCxcuVDAYlM/n04YNG6LWO+f02GOPKScnR+PGjVNRUZH27NkTr34BAMOE5wDq7u5Wfn6+Kisr+12/evVqPfvss3r++ee1bds2nXnmmSouLlZPT89pNwsAGD48fyNqSUmJSkpK+l3nnNMzzzyjRx55JPLtjS+99JKysrK0YcMG3XTTTafXLQBg2IjrZ0DNzc3q6OhQUVFRZFkgEFBBQYHq6+v7rent7VU4HI4aAIDhL64B1NHRIUnKysqKWp6VlRVZ93UVFRUKBAKRkZubG8+WAABDlPldcOXl5QqFQpHR2tpq3RIAYBDENYC+/OO6zs7OqOWdnZ0D/uGd3+9XWlpa1AAADH9xDaC8vDxlZ2erpqYmsiwcDmvbtm0qLCyM564AAEnO811wBw8eVGNjY+Rxc3Ozdu3apfT0dE2aNEn33nuvVq1apQsvvFB5eXl69NFHFQwGVVpaGs++AQBJznMAbd++Xddcc03k8YoVKyRJixcvVlVVlR566CF1d3frrrvu0oEDB3TVVVepurpaZ5xxRvy6BgAkPSYjRcz+8Ic/eK655ZZbEtDJ8X7/+9/HVPejH/3Ic82ll17queaNN97wXHPeeed5rvH5fJ5rpOE3GWmsqqurPde0t7d7rrnjjjs81yQDJiMFAAxJBBAAwAQBBAAwQQABAEwQQAAAEwQQAMAEAQQAMEEAAQBMEEAAABMEEADABAEEADBBAAEATBBAAAATzIaNmKWnp3uu2bx5s+ea/Px8zzV9fX2ea6Rj33fl1ejRnr/VROPHj/dcE4uhPhv2Rx995Lnmn//8ZwI66d+6des812zatMlzTU9Pj+eaZMBs2ACAIYkAAgCYIIAAACYIIACACQIIAGCCAAIAmCCAAAAmCCAAgAkCCABgggACAJgggAAAJgggAIAJ77MoAv/v888/91xTWVk5KDVjxozxXCPphBMnJqMPPvggprpVq1Z5rmlra/Ncs3fvXs81n376qecaDE1cAQEATBBAAAATBBAAwAQBBAAwQQABAEwQQAAAEwQQAMAEAQQAMEEAAQBMEEAAABMEEADABAEEADDhc8456ya+KhwOKxAIWLeBIeSTTz7xXJObmxv/RpLQqFGjrFvACBYKhU44wS9XQAAAEwQQAMAEAQQAMEEAAQBMEEAAABMEEADABAEEADBBAAEATBBAAAATBBAAwAQBBAAwQQABAEyMtm4AOJmUFO+/J/l8vgR0AiCeuAICAJgggAAAJjwH0JYtW7Rw4UIFg0H5fD5t2LAhav1tt90mn88XNRYsWBCvfgEAw4TnAOru7lZ+fr4qKysH3GbBggVqb2+PjFdeeeW0mgQADD+eb0IoKSlRSUnJCbfx+/3Kzs6OuSkAwPCXkM+AamtrlZmZqYsvvljLli3T/v37B9y2t7dX4XA4agAAhr+4B9CCBQv00ksvqaamRr/85S9VV1enkpISHT16tN/tKyoqFAgEIiM3NzfeLQEAhiCfc87FXOzzaf369SotLR1wm//85z+aOnWqNm/erHnz5h23vre3V729vZHH4XCYEEKUlpYWzzUTJ05MQCfJJ5a/oQLiJRQKKS0tbcD1CT87p0yZooyMDDU2Nva73u/3Ky0tLWoAAIa/hAfQ3r17tX//fuXk5CR6VwCAJOL5LriDBw9GXc00Nzdr165dSk9PV3p6up544gktWrRI2dnZampq0kMPPaQLLrhAxcXFcW0cAJDcPAfQ9u3bdc0110Qer1ixQpK0ePFiPffcc9q9e7defPFFHThwQMFgUPPnz9eTTz4pv98fv64BAEnPcwDNnTtXJ7pv4a9//etpNYTkMXq097lsV65c6bkmMzPTc81p3FszrHzwwQcx1T344IOea6qrq2PaF0YubpEBAJgggAAAJgggAIAJAggAYIIAAgCYIIAAACYIIACACQIIAGCCAAIAmCCAAAAmCCAAgAkCCABgggACAJg4ra/kToRwOKxAIGDdBk5BLP9Pn3/+eQI6sXXHHXd4rrn77rs918yYMcNzTaxfg1JfX++55qtf03KqDh8+7LkGycP8K7kBAOgPAQQAMEEAAQBMEEAAABMEEADABAEEADBBAAEATBBAAAATBBAAwAQBBAAwQQABAEwQQAAAE0xGipgxGekxZ599tueacDjsuaaystJzzbJlyzzXSFIsLwsvvfSS55olS5Z4rkHyYDJSAMCQRAABAEwQQAAAEwQQAMAEAQQAMEEAAQBMEEAAABMEEADABAEEADBBAAEATBBAAAATBBAAwMRo6waAZLdq1SrPNT/+8Y891zzyyCOea2KdjDQWhw4dGrR9YXjgCggAYIIAAgCYIIAAACYIIACACQIIAGCCAAIAmCCAAAAmCCAAgAkCCABgggACAJgggAAAJgggAIAJn3POWTfxVeFwWIFAwLoNnIIxY8Z4rnnqqac819x9992ea0aNGuW5JlZtbW2ea37wgx94rmlqavJc88knn3iukaRYXhZCoZDnmvT0dM81SB6hUEhpaWkDrucKCABgggACAJjwFEAVFRW64oorlJqaqszMTJWWlqqhoSFqm56eHpWVlemcc87RWWedpUWLFqmzszOuTQMAkp+nAKqrq1NZWZm2bt2qTZs26ciRI5o/f766u7sj29x3331688039frrr6uurk5tbW264YYb4t44ACC5efpG1Orq6qjHVVVVyszM1I4dOzRnzhyFQiG98MILWrt2rb7zne9IktasWaNvfOMb2rp1q771rW/Fr3MAQFI7rc+Avrzr5cs7WXbs2KEjR46oqKgoss20adM0adIk1dfX9/scvb29CofDUQMAMPzFHEB9fX269957deWVV2rGjBmSpI6ODo0dO1YTJkyI2jYrK0sdHR39Pk9FRYUCgUBk5ObmxtoSACCJxBxAZWVl+vDDD/Xqq6+eVgPl5eUKhUKR0draelrPBwBIDp4+A/rS8uXL9dZbb2nLli2aOHFiZHl2drYOHz6sAwcORF0FdXZ2Kjs7u9/n8vv98vv9sbQBAEhinq6AnHNavny51q9fr7ffflt5eXlR62fNmqUxY8aopqYmsqyhoUEtLS0qLCyMT8cAgGHB0xVQWVmZ1q5dq40bNyo1NTXyuU4gENC4ceMUCAR0++23a8WKFUpPT1daWpruueceFRYWcgccACCKpwB67rnnJElz586NWr5mzRrddtttkqSnn35aKSkpWrRokXp7e1VcXKzf/va3cWkWADB8MBkphrxYJtTkbspjfD5fTHVMRop4YDJSAMCQRAABAEwQQAAAEwQQAMAEAQQAMEEAAQBMEEAAABMEEADABAEEADBBAAEATBBAAAATBBAAwAQBBAAwEdM3ogKD6f777/dcs27dugR0knxinQ373Xff9Vxz7bXXxrQvjFxcAQEATBBAAAATBBAAwAQBBAAwQQABAEwQQAAAEwQQAMAEAQQAMEEAAQBMEEAAABMEEADABAEEADDBZKQY8t555x3PNaWlpTHt68knn/RcM336dM81KSmD87ufcy6mutbWVs81XV1dMe0LIxdXQAAAEwQQAMAEAQQAMEEAAQBMEEAAABMEEADABAEEADBBAAEATBBAAAATBBAAwAQBBAAwQQABAEz4XKyzFSZIOBxWIBCwbgM4ZUuWLPFcU15e7rlm6tSpnmuWL1/uuUaSXnvtNc81n3/+eUz7wvAVCoWUlpY24HqugAAAJgggAIAJAggAYIIAAgCYIIAAACYIIACACQIIAGCCAAIAmCCAAAAmCCAAgAkCCABgggACAJhgMlIAQEIwGSkAYEgigAAAJjwFUEVFha644gqlpqYqMzNTpaWlamhoiNpm7ty58vl8UWPp0qVxbRoAkPw8BVBdXZ3Kysq0detWbdq0SUeOHNH8+fPV3d0dtd2dd96p9vb2yFi9enVcmwYAJL/RXjaurq6OelxVVaXMzEzt2LFDc+bMiSwfP368srOz49MhAGBYOq3PgEKhkCQpPT09avnLL7+sjIwMzZgxQ+Xl5Tp06NCAz9Hb26twOBw1AAAjgIvR0aNH3Xe/+1135ZVXRi3/3e9+56qrq93u3bvdH//4R3feeee566+/fsDnWblypZPEYDAYjGE2QqHQCXMk5gBaunSpmzx5smttbT3hdjU1NU6Sa2xs7Hd9T0+PC4VCkdHa2mp+0BgMBoNx+uNkAeTpM6AvLV++XG+99Za2bNmiiRMnnnDbgoICSVJjY6OmTp163Hq/3y+/3x9LGwCAJOYpgJxzuueee7R+/XrV1tYqLy/vpDW7du2SJOXk5MTUIABgePIUQGVlZVq7dq02btyo1NRUdXR0SJICgYDGjRunpqYmrV27Vtdee63OOecc7d69W/fdd5/mzJmjmTNnJuQfAABIUl4+99EA7/OtWbPGOedcS0uLmzNnjktPT3d+v99dcMEF7sEHHzzp+4BfFQqFzN+3ZDAYDMbpj5O99jMZKQAgIZiMFAAwJBFAAAATBBAAwAQBBAAwQQABAEwQQAAAEwQQAMAEAQQAMEEAAQBMEEAAABMEEADABAEEADBBAAEATBBAAAATBBAAwAQBBAAwQQABAEwQQAAAEwQQAMAEAQQAMEEAAQBMEEAAABMEEADABAEEADBBAAEATAy5AHLOWbcAAIiDk72eD7kA6urqsm4BABAHJ3s997khdsnR19entrY2paamyufzRa0Lh8PKzc1Va2ur0tLSjDq0x3E4huNwDMfhGI7DMUPhODjn1NXVpWAwqJSUga9zRg9iT6ckJSVFEydOPOE2aWlpI/oE+xLH4RiOwzEch2M4DsdYH4dAIHDSbYbcW3AAgJGBAAIAmEiqAPL7/Vq5cqX8fr91K6Y4DsdwHI7hOBzDcTgmmY7DkLsJAQAwMiTVFRAAYPgggAAAJgggAIAJAggAYCJpAqiyslLnn3++zjjjDBUUFOi9996zbmnQPf744/L5fFFj2rRp1m0l3JYtW7Rw4UIFg0H5fD5t2LAhar1zTo899phycnI0btw4FRUVac+ePTbNJtDJjsNtt9123PmxYMECm2YTpKKiQldccYVSU1OVmZmp0tJSNTQ0RG3T09OjsrIynXPOOTrrrLO0aNEidXZ2GnWcGKdyHObOnXvc+bB06VKjjvuXFAH02muvacWKFVq5cqXef/995efnq7i4WPv27bNubdBNnz5d7e3tkfG3v/3NuqWE6+7uVn5+viorK/tdv3r1aj377LN6/vnntW3bNp155pkqLi5WT0/PIHeaWCc7DpK0YMGCqPPjlVdeGcQOE6+urk5lZWXaunWrNm3apCNHjmj+/Pnq7u6ObHPffffpzTff1Ouvv666ujq1tbXphhtuMOw6/k7lOEjSnXfeGXU+rF692qjjAbgkMHv2bFdWVhZ5fPToURcMBl1FRYVhV4Nv5cqVLj8/37oNU5Lc+vXrI4/7+vpcdna2e+qppyLLDhw44Px+v3vllVcMOhwcXz8Ozjm3ePFid91115n0Y2Xfvn1Okqurq3POHfu/HzNmjHv99dcj2/zrX/9yklx9fb1Vmwn39ePgnHPf/va33U9+8hO7pk7BkL8COnz4sHbs2KGioqLIspSUFBUVFam+vt6wMxt79uxRMBjUlClTdOutt6qlpcW6JVPNzc3q6OiIOj8CgYAKCgpG5PlRW1urzMxMXXzxxVq2bJn2799v3VJChUIhSVJ6erokaceOHTpy5EjU+TBt2jRNmjRpWJ8PXz8OX3r55ZeVkZGhGTNmqLy8XIcOHbJob0BDbjLSr/vss8909OhRZWVlRS3PysrSv//9b6OubBQUFKiqqkoXX3yx2tvb9cQTT+jqq6/Whx9+qNTUVOv2THR0dEhSv+fHl+tGigULFuiGG25QXl6empqa9PDDD6ukpET19fUaNWqUdXtx19fXp3vvvVdXXnmlZsyYIenY+TB27FhNmDAhatvhfD70dxwk6ZZbbtHkyZMVDAa1e/du/fSnP1VDQ4P+/Oc/G3YbbcgHEP6npKQk8vPMmTNVUFCgyZMna926dbr99tsNO8NQcNNNN0V+vuSSSzRz5kxNnTpVtbW1mjdvnmFniVFWVqYPP/xwRHwOeiIDHYe77ror8vMll1yinJwczZs3T01NTZo6depgt9mvIf8WXEZGhkaNGnXcXSydnZ3Kzs426mpomDBhgi666CI1NjZat2Lmy3OA8+N4U6ZMUUZGxrA8P5YvX6633npL77zzTtTXt2RnZ+vw4cM6cOBA1PbD9XwY6Dj0p6CgQJKG1Pkw5ANo7NixmjVrlmpqaiLL+vr6VFNTo8LCQsPO7B08eFBNTU3KycmxbsVMXl6esrOzo86PcDisbdu2jfjzY+/evdq/f/+wOj+cc1q+fLnWr1+vt99+W3l5eVHrZ82apTFjxkSdDw0NDWppaRlW58PJjkN/du3aJUlD63ywvgviVLz66qvO7/e7qqoq9/HHH7u77rrLTZgwwXV0dFi3Nqjuv/9+V1tb65qbm927777rioqKXEZGhtu3b591awnV1dXldu7c6Xbu3OkkuV//+tdu586d7tNPP3XOOfeLX/zCTZgwwW3cuNHt3r3bXXfddS4vL8998cUXxp3H14mOQ1dXl3vggQdcfX29a25udps3b3aXXXaZu/DCC11PT49163GzbNkyFwgEXG1trWtvb4+MQ4cORbZZunSpmzRpknv77bfd9u3bXWFhoSssLDTsOv5OdhwaGxvdz3/+c7d9+3bX3NzsNm7c6KZMmeLmzJlj3Hm0pAgg55z7zW9+4yZNmuTGjh3rZs+e7bZu3Wrd0qC78cYbXU5Ojhs7dqw777zz3I033ugaGxut20q4d955x0k6bixevNg5d+xW7EcffdRlZWU5v9/v5s2b5xoaGmybToATHYdDhw65+fPnu3PPPdeNGTPGTZ482d15553D7pe0/v79ktyaNWsi23zxxRfu7rvvdmeffbYbP368u/766117e7td0wlwsuPQ0tLi5syZ49LT053f73cXXHCBe/DBB10oFLJt/Gv4OgYAgIkh/xkQAGB4IoAAACYIIACACQIIAGCCAAIAmCCAAAAmCCAAgAkCCABgggACAJgggAAAJgggAIAJAggAYOL/AC0/Gz+zK6mJAAAAAElFTkSuQmCC\n"
          },
          "metadata": {}
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<matplotlib.image.AxesImage at 0x7f0738d6cc10>"
            ]
          },
          "metadata": {},
          "execution_count": 5
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "<Figure size 640x480 with 1 Axes>"
            ],
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAaAAAAGdCAYAAABU0qcqAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/bCgiHAAAACXBIWXMAAA9hAAAPYQGoP6dpAAAd0UlEQVR4nO3de3BU9f3/8dcGyAIx2TSE3ORiAAXKJa1UIkURJQNE6wgyHW/tYMfiYIMVEWmpVdR2Ji2dsY4ORZ12oE7FC22Baju0GCS0NeBwG6rVSDJRQEhQLLshQIjJ5/cHP/fryvWz7OadhOdj5jNjds8r++ZwzIuzuzkbcM45AQDQzlKsBwAAXJgoIACACQoIAGCCAgIAmKCAAAAmKCAAgAkKCABgggICAJjobj3Al7W1tWnfvn1KT09XIBCwHgcA4Mk5p8bGRhUUFCgl5fTnOR2ugPbt26f+/ftbjwEAOE979uxRv379Tnt/h3sKLj093XoEAEACnO3nedIKaMmSJbrkkkvUs2dPFRcX66233jqnHE+7AUDXcLaf50kpoJdfflnz5s3TokWLtG3bNhUVFWnKlCk6cOBAMh4OANAZuSQYO3asKysri37d2trqCgoKXHl5+Vmz4XDYSWKxWCxWJ1/hcPiMP+8TfgZ0/Phxbd26VSUlJdHbUlJSVFJSoqqqqpO2b25uViQSiVkAgK4v4QX0ySefqLW1Vbm5uTG35+bmqr6+/qTty8vLFQqFoot3wAHAhcH8XXALFy5UOByOrj179liPBABoBwn/PaDs7Gx169ZNDQ0NMbc3NDQoLy/vpO2DwaCCwWCixwAAdHAJPwNKTU3VmDFjVFFREb2tra1NFRUVGjduXKIfDgDQSSXlSgjz5s3TzJkz9Y1vfENjx47Vk08+qaamJn3ve99LxsMBADqhpBTQLbfcoo8//liPPPKI6uvr9bWvfU1r16496Y0JAIALV8A556yH+KJIJKJQKGQ9BgDgPIXDYWVkZJz2fvN3wQEALkwUEADABAUEADBBAQEATFBAAAATFBAAwAQFBAAwQQEBAExQQAAAExQQAMAEBQQAMEEBAQBMUEAAABMUEADABAUEADBBAQEATFBAAAATFBAAwAQFBAAwQQEBAExQQAAAExQQAMAEBQQAMEEBAQBMUEAAABMUEADABAUEADBBAQEATFBAAAATFBAAwAQFBAAwQQEBAExQQAAAExQQAMAEBQQAMEEBAQBMUEAAABMUEADABAUEADBBAQEATFBAAAATFBAAwAQFBAAwQQEBAExQQAAAExQQAMAEBQQAMEEBAQBMUEAAABMUEADABAUEADBBAQEATHS3HgA4m6KiIu9Ma2trXI/16aefemeOHj3qnTl8+LB3pqWlxTuD85OS4v9v9La2tiRM0jVxBgQAMEEBAQBMJLyAHn30UQUCgZg1bNiwRD8MAKCTS8prQCNGjNDrr7/+fw/SnZeaAACxktIM3bt3V15eXjK+NQCgi0jKa0C7du1SQUGBBg0apDvuuEO7d+8+7bbNzc2KRCIxCwDQ9SW8gIqLi7V8+XKtXbtWS5cuVV1dna6++mo1Njaecvvy8nKFQqHo6t+/f6JHAgB0QAHnnEvmAxw6dEgDBw7UE088obvuuuuk+5ubm9Xc3Bz9OhKJUEKIwe8BncDvAbU/fg/o/ITDYWVkZJz2/qS/OyAzM1OXXXaZampqTnl/MBhUMBhM9hgAgA4m6b8HdPjwYdXW1io/Pz/ZDwUA6EQSXkDz589XZWWlPvjgA7355puaPn26unXrpttuuy3RDwUA6MQS/hTc3r17ddttt+ngwYPq27evrrrqKm3atEl9+/ZN9EMBADqxpL8JwVckElEoFLIeA+dg+vTp3pmHHnrIO5Oamuqd+d///uedkaSmpibvzDvvvOOd2bdvn3fmt7/9rXcmnj+PJKWnp3tn4vl7iucNHNddd513Jt6fKX/5y1+8M+FwOK7H6orO9iYErgUHADBBAQEATFBAAAATFBAAwAQFBAAwQQEBAExQQAAAExQQAMAEBQQAMEEBAQBMUEAAABMUEADARNI/kA7xCwQC3plRo0Z5Zy6//HLvjKS4PmJjzJgx3pkOdr3ck5SWlnpn6urqvDMDBw70zsRzsU9J+uyzz7wzWVlZ3pkhQ4Z4Z774Ccrnav/+/d4ZSXr55ZfjyuHccAYEADBBAQEATFBAAAATFBAAwAQFBAAwQQEBAExQQAAAExQQAMAEBQQAMEEBAQBMUEAAABMUEADABAUEADDB1bA7sMzMTO/Mfffd55255pprvDOSNGjQIO9MPFe2bm1t9c4cPnzYOyNJoVDIO3P8+HHvTDxXti4rK/POHDt2zDsjSWlpaXHlfMVzZeu///3v3plXX33VOyPFd+zh3HEGBAAwQQEBAExQQAAAExQQAMAEBQQAMEEBAQBMUEAAABMUEADABAUEADBBAQEATFBAAAATFBAAwAQXI20n8VxY9Mc//rF35tvf/rZ35qKLLvLOSPFdSHLTpk3emXfffdc78/HHH3tnJGn48OHemSFDhnhnhg4d6p3p1auXd6a9LioqxbfPd+7c6Z355z//6Z3Ztm2bd0biYqTJxhkQAMAEBQQAMEEBAQBMUEAAABMUEADABAUEADBBAQEATFBAAAATFBAAwAQFBAAwQQEBAExQQAAAE1yMtJ2EQiHvTF5enncmnguLtrW1eWckac+ePd6Z5557zjuzbt0670w4HPbOSFJhYaF35oYbbvDOLFiwwDvTu3dv74xzzjsjSS0tLd6Z9evXe2eefvpp78ybb77pnUHHxBkQAMAEBQQAMOFdQBs3btSNN96ogoICBQIBrV69OuZ+55weeeQR5efnq1evXiopKdGuXbsSNS8AoIvwLqCmpiYVFRVpyZIlp7x/8eLFeuqpp/TMM89o8+bNSktL05QpU3Ts2LHzHhYA0HV4vwmhtLRUpaWlp7zPOacnn3xSP/3pT3XTTTdJkp5//nnl5uZq9erVuvXWW89vWgBAl5HQ14Dq6upUX1+vkpKS6G2hUEjFxcWqqqo6Zaa5uVmRSCRmAQC6voQWUH19vSQpNzc35vbc3NzofV9WXl6uUCgUXf3790/kSACADsr8XXALFy5UOByOrnh+twQA0PkktIA+/8XJhoaGmNsbGhpO+0uVwWBQGRkZMQsA0PUltIAKCwuVl5enioqK6G2RSESbN2/WuHHjEvlQAIBOzvtdcIcPH1ZNTU3067q6Ou3YsUNZWVkaMGCA5s6dq5///Oe69NJLVVhYqIcfflgFBQWaNm1aIucGAHRy3gW0ZcsWXXvttdGv582bJ0maOXOmli9frgULFqipqUl33323Dh06pKuuukpr165Vz549Ezc1AKDT8y6giRMnnvECh4FAQI8//rgef/zx8xqsq/nwww+9Mykp/s+QNjU1eWfS0tK8M1J8F6zMycnxznTv7n/N3NbWVu+MpLheg3zwwQe9M19+p2iyvPPOO3HlXnjhBe/MypUrvTO86ejCZv4uOADAhYkCAgCYoIAAACYoIACACQoIAGCCAgIAmKCAAAAmKCAAgAkKCABgggICAJiggAAAJiggAIAJCggAYML/MsNoN3/729+8MxMmTPDOxPtRGUOGDPHOzJ8/3zuTnp7unXnllVe8M5J0++23e2cyMzPjeixfn376qXfmT3/6U1yPFc/VsLmyNXxxBgQAMEEBAQBMUEAAABMUEADABAUEADBBAQEATFBAAAATFBAAwAQFBAAwQQEBAExQQAAAExQQAMBEwDnnrIf4okgkolAoZD1GhxAIBLwz3//+970z3/nOd7wzkjRixAjvTDwX7jxw4IB3Jt5jKCXF/99kqamp3pl4/m4bGxu9M9nZ2d4ZSWppaYkrB3xROBxWRkbGae/nDAgAYIICAgCYoIAAACYoIACACQoIAGCCAgIAmKCAAAAmKCAAgAkKCABgggICAJiggAAAJiggAIAJLkbaxfTu3ds789WvfjWux7ruuuu8M9/97ne9M8OHD/fOxHNR0Y4unguYzp8/P67H2rJli3emvr7eO/P+++97ZzrYjyycARcjBQB0SBQQAMAEBQQAMEEBAQBMUEAAABMUEADABAUEADBBAQEATFBAAAATFBAAwAQFBAAwQQEBAExwMVLELRgMemeuv/5678zs2bO9M9/85je9M5KUlpbmnYnnIqHt5ejRo3Hl4rmw6LZt27wzTz75pHempqbGOxPPnwfnj4uRAgA6JAoIAGDCu4A2btyoG2+8UQUFBQoEAlq9enXM/XfeeacCgUDMmjp1aqLmBQB0Ed4F1NTUpKKiIi1ZsuS020ydOlX79++PrhdffPG8hgQAdD3dfQOlpaUqLS094zbBYFB5eXlxDwUA6PqS8hrQhg0blJOTo6FDh+qee+7RwYMHT7ttc3OzIpFIzAIAdH0JL6CpU6fq+eefV0VFhX75y1+qsrJSpaWlam1tPeX25eXlCoVC0dW/f/9EjwQA6IC8n4I7m1tvvTX636NGjdLo0aM1ePBgbdiwQZMmTTpp+4ULF2revHnRryORCCUEABeApL8Ne9CgQcrOzj7tL48Fg0FlZGTELABA15f0Atq7d68OHjyo/Pz8ZD8UAKAT8X4K7vDhwzFnM3V1ddqxY4eysrKUlZWlxx57TDNmzFBeXp5qa2u1YMECDRkyRFOmTEno4ACAzs27gLZs2aJrr702+vXnr9/MnDlTS5cu1c6dO/X73/9ehw4dUkFBgSZPnqyf/exncV03DADQdXExUrSr7t393/dy5ZVXemf++Mc/emckKTc3N66cr3j+t9u7d693Jj093Tsjxff3lJLi/4z+Bx984J156qmnvDPPPvusdwbnj4uRAgA6JAoIAGCCAgIAmKCAAAAmKCAAgAkKCABgggICAJiggAAAJiggAIAJCggAYIICAgCYoIAAACYoIACAiYR/JDdwJp999pl3prq62jvz4YcfemckqW/fvt6ZQCDgnXnjjTe8MytWrPDOHDlyxDsjKeYjV87VrFmzvDPDhw/3ztx+++3emcrKSu+MJL333ntx5XBuOAMCAJiggAAAJiggAIAJCggAYIICAgCYoIAAACYoIACACQoIAGCCAgIAmKCAAAAmKCAAgAkKCABggouRol3Fc+HOESNGeGfS09O9M1J888Vj7Nix3pkFCxZ4Z95//33vjCT99a9/9c7k5+d7Z771rW95Z+LZd7Nnz/bOSNLcuXPjyuHccAYEADBBAQEATFBAAAATFBAAwAQFBAAwQQEBAExQQAAAExQQAMAEBQQAMEEBAQBMUEAAABMUEADABBcjRYe3fft270yfPn2SMEnipKWleWeWLl3qnVm1apV3RpL+8Y9/eGc++uijuB7L15EjR7wz27Zti+ux4rk4rXMurse6EHEGBAAwQQEBAExQQAAAExQQAMAEBQQAMEEBAQBMUEAAABMUEADABAUEADBBAQEATFBAAAATFBAAwAQXI0W7iudCjc3Nzd6Z5557zjsjSQ899FBcufbw9a9/3TszZMiQuB7rhz/8oXcmNzc3rsfydeDAAe9MPBcwlbiwaLJxBgQAMEEBAQBMeBVQeXm5rrjiCqWnpysnJ0fTpk1TdXV1zDbHjh1TWVmZ+vTpo4suukgzZsxQQ0NDQocGAHR+XgVUWVmpsrIybdq0SevWrVNLS4smT56spqam6Db333+/Xn31Va1cuVKVlZXat2+fbr755oQPDgDo3LzehLB27dqYr5cvX66cnBxt3bpVEyZMUDgc1u9+9zutWLFC1113nSRp2bJlGj58uDZt2qQrr7wycZMDADq183oNKBwOS5KysrIkSVu3blVLS4tKSkqi2wwbNkwDBgxQVVXVKb9Hc3OzIpFIzAIAdH1xF1BbW5vmzp2r8ePHa+TIkZKk+vp6paamKjMzM2bb3Nxc1dfXn/L7lJeXKxQKRVf//v3jHQkA0InEXUBlZWV6++239dJLL53XAAsXLlQ4HI6uPXv2nNf3AwB0DnH9IuqcOXP02muvaePGjerXr1/09ry8PB0/flyHDh2KOQtqaGhQXl7eKb9XMBhUMBiMZwwAQCfmdQbknNOcOXO0atUqrV+/XoWFhTH3jxkzRj169FBFRUX0turqau3evVvjxo1LzMQAgC7B6wyorKxMK1as0Jo1a5Senh59XScUCqlXr14KhUK66667NG/ePGVlZSkjI0P33nuvxo0bxzvgAAAxvApo6dKlkqSJEyfG3L5s2TLdeeedkqRf//rXSklJ0YwZM9Tc3KwpU6boN7/5TUKGBQB0HV4FdC4X5uvZs6eWLFmiJUuWxD0U8EXHjx/3znz+jyVfo0aN8s6MHz/eO/Pld4qei27durXL43R0GRkZ3plAIJCESXC+uBYcAMAEBQQAMEEBAQBMUEAAABMUEADABAUEADBBAQEATFBAAAATFBAAwAQFBAAwQQEBAExQQAAAExQQAMBEXJ+ICrSntrY278znn1Xl64EHHvDOlJSUeGdmzpzpnYnnM7Wampq8M5LUu3dv70xra6t3JiXF/9/AL730knfmzTff9M5I8V1F+1w+NQAncAYEADBBAQEATFBAAAATFBAAwAQFBAAwQQEBAExQQAAAExQQAMAEBQQAMEEBAQBMUEAAABMUEADABBcjRZcUzwVMJam2ttY788EHH3hn/vOf/3hnxowZ45257LLLvDOSdPjwYe9Mv379vDO7d+/2zixdutQ789FHH3lnkHycAQEATFBAAAATFBAAwAQFBAAwQQEBAExQQAAAExQQAMAEBQQAMEEBAQBMUEAAABMUEADABAUEADARcM456yG+KBKJKBQKWY8BJFUwGPTO9OzZ0zsTDoe9M5KUlpbmnYnnArBHjx71zqDzCIfDysjIOO39nAEBAExQQAAAExQQAMAEBQQAMEEBAQBMUEAAABMUEADABAUEADBBAQEATFBAAAATFBAAwAQFBAAw0d16AOBC1Nzc3C6ZeDU1NbXbY+HCxRkQAMAEBQQAMOFVQOXl5briiiuUnp6unJwcTZs2TdXV1THbTJw4UYFAIGbNnj07oUMDADo/rwKqrKxUWVmZNm3apHXr1qmlpUWTJ08+6fniWbNmaf/+/dG1ePHihA4NAOj8vN6EsHbt2pivly9frpycHG3dulUTJkyI3t67d2/l5eUlZkIAQJd0Xq8Bff5xv1lZWTG3v/DCC8rOztbIkSO1cOFCHTly5LTfo7m5WZFIJGYBAC4ALk6tra3uhhtucOPHj4+5/dlnn3Vr1651O3fudH/4wx/cxRdf7KZPn37a77No0SInicVisVhdbIXD4TP2SNwFNHv2bDdw4EC3Z8+eM25XUVHhJLmamppT3n/s2DEXDoeja8+ePeY7jcVisVjnv85WQHH9IuqcOXP02muvaePGjerXr98Zty0uLpYk1dTUaPDgwSfdHwwGFQwG4xkDANCJeRWQc0733nuvVq1apQ0bNqiwsPCsmR07dkiS8vPz4xoQANA1eRVQWVmZVqxYoTVr1ig9PV319fWSpFAopF69eqm2tlYrVqzQ9ddfrz59+mjnzp26//77NWHCBI0ePTopfwAAQCfl87qPTvM837Jly5xzzu3evdtNmDDBZWVluWAw6IYMGeIefPDBsz4P+EXhcNj8eUsWi8Vinf8628/+wP8vlg4jEokoFApZjwEAOE/hcFgZGRmnvZ9rwQEATFBAAAATFBAAwAQFBAAwQQEBAExQQAAAExQQAMAEBQQAMEEBAQBMUEAAABMUEADABAUEADBBAQEATFBAAAATFBAAwAQFBAAwQQEBAExQQAAAExQQAMAEBQQAMEEBAQBMUEAAABMUEADABAUEADBBAQEATHS4AnLOWY8AAEiAs/0873AF1NjYaD0CACABzvbzPOA62ClHW1ub9u3bp/T0dAUCgZj7IpGI+vfvrz179igjI8NoQnvshxPYDyewH05gP5zQEfaDc06NjY0qKChQSsrpz3O6t+NM5yQlJUX9+vU74zYZGRkX9AH2OfbDCeyHE9gPJ7AfTrDeD6FQ6KzbdLin4AAAFwYKCABgolMVUDAY1KJFixQMBq1HMcV+OIH9cAL74QT2wwmdaT90uDchAAAuDJ3qDAgA0HVQQAAAExQQAMAEBQQAMNFpCmjJkiW65JJL1LNnTxUXF+utt96yHqndPfroowoEAjFr2LBh1mMl3caNG3XjjTeqoKBAgUBAq1evjrnfOadHHnlE+fn56tWrl0pKSrRr1y6bYZPobPvhzjvvPOn4mDp1qs2wSVJeXq4rrrhC6enpysnJ0bRp01RdXR2zzbFjx1RWVqY+ffrooosu0owZM9TQ0GA0cXKcy36YOHHiScfD7NmzjSY+tU5RQC+//LLmzZunRYsWadu2bSoqKtKUKVN04MAB69Ha3YgRI7R///7o+te//mU9UtI1NTWpqKhIS5YsOeX9ixcv1lNPPaVnnnlGmzdvVlpamqZMmaJjx46186TJdbb9IElTp06NOT5efPHFdpww+SorK1VWVqZNmzZp3bp1amlp0eTJk9XU1BTd5v7779err76qlStXqrKyUvv27dPNN99sOHXinct+kKRZs2bFHA+LFy82mvg0XCcwduxYV1ZWFv26tbXVFRQUuPLycsOp2t+iRYtcUVGR9RimJLlVq1ZFv25ra3N5eXnuV7/6VfS2Q4cOuWAw6F588UWDCdvHl/eDc87NnDnT3XTTTSbzWDlw4ICT5CorK51zJ/7ue/To4VauXBnd5t1333WSXFVVldWYSffl/eCcc9dcc42777777IY6Bx3+DOj48ePaunWrSkpKorelpKSopKREVVVVhpPZ2LVrlwoKCjRo0CDdcccd2r17t/VIpurq6lRfXx9zfIRCIRUXF1+Qx8eGDRuUk5OjoUOH6p577tHBgwetR0qqcDgsScrKypIkbd26VS0tLTHHw7BhwzRgwIAufTx8eT987oUXXlB2drZGjhyphQsX6siRIxbjnVaHuxjpl33yySdqbW1Vbm5uzO25ubl67733jKayUVxcrOXLl2vo0KHav3+/HnvsMV199dV6++23lZ6ebj2eifr6ekk65fHx+X0XiqlTp+rmm29WYWGhamtr9ZOf/ESlpaWqqqpSt27drMdLuLa2Ns2dO1fjx4/XyJEjJZ04HlJTU5WZmRmzbVc+Hk61HyTp9ttv18CBA1VQUKCdO3fqRz/6kaqrq/XnP//ZcNpYHb6A8H9KS0uj/z169GgVFxdr4MCBeuWVV3TXXXcZToaO4NZbb43+96hRozR69GgNHjxYGzZs0KRJkwwnS46ysjK9/fbbF8TroGdyuv1w9913R/971KhRys/P16RJk1RbW6vBgwe395in1OGfgsvOzla3bt1OehdLQ0OD8vLyjKbqGDIzM3XZZZeppqbGehQznx8DHB8nGzRokLKzs7vk8TFnzhy99tpreuONN2I+viUvL0/Hjx/XoUOHYrbvqsfD6fbDqRQXF0tShzoeOnwBpaamasyYMaqoqIje1tbWpoqKCo0bN85wMnuHDx9WbW2t8vPzrUcxU1hYqLy8vJjjIxKJaPPmzRf88bF3714dPHiwSx0fzjnNmTNHq1at0vr161VYWBhz/5gxY9SjR4+Y46G6ulq7d+/uUsfD2fbDqezYsUOSOtbxYP0uiHPx0ksvuWAw6JYvX+7++9//urvvvttlZma6+vp669Ha1QMPPOA2bNjg6urq3L///W9XUlLisrOz3YEDB6xHS6rGxka3fft2t337difJPfHEE2779u3uww8/dM4594tf/MJlZma6NWvWuJ07d7qbbrrJFRYWuqNHjxpPnlhn2g+NjY1u/vz5rqqqytXV1bnXX3/dXX755e7SSy91x44dsx49Ye655x4XCoXchg0b3P79+6PryJEj0W1mz57tBgwY4NavX++2bNnixo0b58aNG2c4deKdbT/U1NS4xx9/3G3ZssXV1dW5NWvWuEGDBrkJEyYYTx6rUxSQc849/fTTbsCAAS41NdWNHTvWbdq0yXqkdnfLLbe4/Px8l5qa6i6++GJ3yy23uJqaGuuxku6NN95wkk5aM2fOdM6deCv2ww8/7HJzc10wGHSTJk1y1dXVtkMnwZn2w5EjR9zkyZNd3759XY8ePdzAgQPdrFmzutw/0k7155fkli1bFt3m6NGj7gc/+IH7yle+4nr37u2mT5/u9u/fbzd0EpxtP+zevdtNmDDBZWVluWAw6IYMGeIefPBBFw6HbQf/Ej6OAQBgosO/BgQA6JooIACACQoIAGCCAgIAmKCAAAAmKCAAgAkKCABgggICAJiggAAAJiggAIAJCggAYIICAgCY+H/DSIQ17Yr0ZQAAAABJRU5ErkJggg==\n"
          },
          "metadata": {}
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "KE33Yxgggu0c"
      },
      "source": [
        "After rotating, the the class of the image is still the same. Let us make a function to rotate multiple images by random angles. We want a slightly different image every time we run this function. So, we generate a random number between 0 and 1 and change it so that it lies between -constraint/2 and +constraint/2"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "vyM7pUV7Reze"
      },
      "source": [
        "def augRotate(sample, angleconstraint):\n",
        "  if angleconstraint==0:\n",
        "    return sample\n",
        "  if len(sample.shape)==2:\n",
        "    sample = np.expand_dims(sample, 0)  # make sure the sample is 3 dimensional\n",
        "  angle = rng.random(len(sample)) # generate random numbers for angles\n",
        "  angle = (angle-0.5)*angleconstraint # make the random angle constrained\n",
        "  nsample = sample.copy() # preallocate the augmented array to make it faster\n",
        "  for ii in range(len(sample)):\n",
        "    nsample[ii] = rotate(sample[ii], angle[ii])\n",
        "  return np.squeeze(nsample) # take care if the input had only one sample."
      ],
      "execution_count": 6,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "kDk-N5VNjar9"
      },
      "source": [
        "This function returns a slightly different image each time we call it. So we can increase the number of images in the sample by any multiple."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Vw3O9zGFgI8K",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 639
        },
        "outputId": "7970711e-2f4c-425b-ff49-7b926ccae9ac"
      },
      "source": [
        "sample = train_X[20]\n",
        "angleconstraint = 70\n",
        "# show the original image\n",
        "plt.imshow(sample, cmap='gray')\n",
        "plt.show()\n",
        "plt.subplot(1,3,1)\n",
        "plt.imshow(augRotate(sample, angleconstraint), cmap='gray') # show an augmented image\n",
        "plt.subplot(1,3,2)\n",
        "plt.imshow(augRotate(sample, angleconstraint), cmap='gray') # show another augmented image from the same sample\n",
        "plt.subplot(1,3,3)\n",
        "plt.imshow(augRotate(sample, angleconstraint), cmap='gray') # one more image from the same sample"
      ],
      "execution_count": 7,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "<Figure size 640x480 with 1 Axes>"
            ],
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAaAAAAGdCAYAAABU0qcqAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/bCgiHAAAACXBIWXMAAA9hAAAPYQGoP6dpAAAblElEQVR4nO3df2yV9fn/8dcp0kPV9nSltqeVHxZQWURYxqQ2aEXpKNUQUebAOYOL08CKGzB16TJAN5duLNkMruL+MHRu4g82gcm2LlBsyWaLoUKI2dbQrht10IIsnAPFFkLf3z/4ej4eKT/u03N6nbbPR/JOeu77vnpf3t6cV+9z7vM+PuecEwAAAyzFugEAwPBEAAEATBBAAAATBBAAwAQBBAAwQQABAEwQQAAAEwQQAMDEFdYNfFZvb68OHTqk9PR0+Xw+63YAAB4553TixAnl5+crJeXC1zlJF0CHDh3S2LFjrdsAAPRTe3u7xowZc8H1SfcSXHp6unULAIA4uNTzecICqKqqStddd51GjRqlwsJCvffee5dVx8tuADA0XOr5PCEB9MYbb2jlypVas2aN3n//fU2bNk2lpaU6cuRIInYHABiMXALMmDHDlZeXRx6fPXvW5efnu8rKykvWhkIhJ4nBYDAYg3yEQqGLPt/H/Qro9OnTampqUklJSWRZSkqKSkpK1NDQcN72PT09CofDUQMAMPTFPYA++ugjnT17Vrm5uVHLc3Nz1dHRcd72lZWVCgQCkcEdcAAwPJjfBVdRUaFQKBQZ7e3t1i0BAAZA3D8HlJ2drREjRqizszNqeWdnp4LB4Hnb+/1++f3+eLcBAEhycb8CSk1N1fTp01VbWxtZ1tvbq9raWhUVFcV7dwCAQSohMyGsXLlSixcv1pe+9CXNmDFDzz//vLq6uvSNb3wjEbsDAAxCCQmghQsX6ujRo1q9erU6Ojr0hS98QTU1NefdmAAAGL58zjln3cSnhcNhBQIB6zYAAP0UCoWUkZFxwfXmd8EBAIYnAggAYIIAAgCYIIAAACYIIACACQIIAGCCAAIAmCCAAAAmCCAAgAkCCABgggACAJgggAAAJgggAIAJAggAYIIAAgCYIIAAACYIIACACQIIAGCCAAIAmCCAAAAmCCAAgAkCCABgggACAJgggAAAJgggAIAJAggAYIIAAgCYIIAAACYIIACACQIIAGCCAAIAmCCAAAAmCCAAgAkCCABgggACAJgggAAAJgggAIAJAggAYIIAAgCYuMK6AWCwKy8v91yzbt06zzUpKd7/Xuzt7fVcI0kbNmzwXNPU1OS55je/+Y3nmpMnT3quQXLiCggAYIIAAgCYIIAAACYIIACACQIIAGCCAAIAmCCAAAAmCCAAgAkCCABgggACAJgggAAAJgggAIAJn3POWTfxaeFwWIFAwLoNDFPf/va3Pdc899xznmuuvPJKzzU+n89zTZL98z7PAw884Lmmvr7ec83//vc/zzXov1AopIyMjAuu5woIAGCCAAIAmIh7AD3zzDPy+XxRY/LkyfHeDQBgkEvIF9LddNNN2rFjx//t5Aq+9w4AEC0hyXDFFVcoGAwm4lcDAIaIhLwHdODAAeXn52vChAl66KGHdPDgwQtu29PTo3A4HDUAAENf3AOosLBQ1dXVqqmp0fr169XW1qbbb79dJ06c6HP7yspKBQKByBg7dmy8WwIAJKG4B1BZWZkeeOABTZ06VaWlpfrTn/6k48eP68033+xz+4qKCoVCochob2+Pd0sAgCSU8LsDMjMzdcMNN6ilpaXP9X6/X36/P9FtAACSTMI/B3Ty5Em1trYqLy8v0bsCAAwicQ+gJ598UvX19fr3v/+td999V/fdd59GjBihBx98MN67AgAMYnF/Ce7DDz/Ugw8+qGPHjumaa67RbbfdpsbGRl1zzTXx3hUAYBBjMlIkvZQU7xfqDz/8cEz7eumllzzXjBw5MqZ9eRXLZKSxfqyht7c3pjqv0tPTPdds2bLFc81Xv/pVzzXoPyYjBQAkJQIIAGCCAAIAmCCAAAAmCCAAgAkCCABgggACAJgggAAAJgggAIAJAggAYIIAAgCYIIAAACYS/oV0QH89+uijnmvWr1+fgE76duTIEc81L7/8sueaWCZlXbdunecaSers7Iypzqu2tjbPNXfddZfnmltvvdVzjSQ1NjbGVIfLwxUQAMAEAQQAMEEAAQBMEEAAABMEEADABAEEADBBAAEATBBAAAATBBAAwAQBBAAwQQABAEwQQAAAEwQQAMAEs2Ej6c2bN8+6hYuKZeboVatWJaCT4SEzM9NzTXFxcUz7YjbsxOIKCABgggACAJgggAAAJgggAIAJAggAYIIAAgCYIIAAACYIIACACQIIAGCCAAIAmCCAAAAmCCAAgAkmIwUQFw888IDnmuzsbM81R44c8Vzz+9//3nMNEo8rIACACQIIAGCCAAIAmCCAAAAmCCAAgAkCCABgggACAJgggAAAJgggAIAJAggAYIIAAgCYIIAAACaYjBRJ7w9/+IPnmrvvvjsBnfQtGAx6riksLPRcs3v3bs81sRo1apTnmiVLlniuSUtL81xz9OhRzzWtra2ea5B4XAEBAEwQQAAAE54DaNeuXZo3b57y8/Pl8/m0ZcuWqPXOOa1evVp5eXlKS0tTSUmJDhw4EK9+AQBDhOcA6urq0rRp01RVVdXn+rVr12rdunV66aWXtHv3bl111VUqLS1Vd3d3v5sFAAwdnm9CKCsrU1lZWZ/rnHN6/vnn9YMf/ED33nuvJOmVV15Rbm6utmzZokWLFvWvWwDAkBHX94Da2trU0dGhkpKSyLJAIKDCwkI1NDT0WdPT06NwOBw1AABDX1wDqKOjQ5KUm5sbtTw3Nzey7rMqKysVCAQiY+zYsfFsCQCQpMzvgquoqFAoFIqM9vZ265YAAAMgrgH0yQfyOjs7o5Z3dnZe8MN6fr9fGRkZUQMAMPTFNYAKCgoUDAZVW1sbWRYOh7V7924VFRXFc1cAgEHO811wJ0+eVEtLS+RxW1ub9u3bp6ysLI0bN07Lly/Xc889p+uvv14FBQVatWqV8vPzNX/+/Hj2DQAY5DwH0J49e3TnnXdGHq9cuVKStHjxYlVXV+vpp59WV1eXHn/8cR0/fly33XabampqYppbCgAwdPmcc866iU8Lh8MKBALWbSCJpKR4f6V47dq1Me1r+fLlMdV59fHHH3uu+fQffpdrz549nmskRX2U4nLV1NR4rvnvf//ruWbhwoWeaxobGz3XoP9CodBF39c3vwsOADA8EUAAABMEEADABAEEADBBAAEATBBAAAATBBAAwAQBBAAwQQABAEwQQAAAEwQQAMAEAQQAMEEAAQBMMBs2hqTx48fHVBfLjM7XX399TPvyatOmTZ5r3nzzzZj29eKLL3quieWp5Ctf+YrnmnfffddzDWwwGzYAICkRQAAAEwQQAMAEAQQAMEEAAQBMEEAAABMEEADABAEEADBBAAEATBBAAAATBBAAwAQBBAAwwWSkwKdMmjTJc80f//hHzzUTJ070XOPz+TzXDOQ/78rKSs81q1atSkAnSBZMRgoASEoEEADABAEEADBBAAEATBBAAAATBBAAwAQBBAAwQQABAEwQQAAAEwQQAMAEAQQAMEEAAQBMMBkp0E9Tp071XLN3717PNSkp3v9e7O3t9VwjSV//+tc917z22msx7QtDF5ORAgCSEgEEADBBAAEATBBAAAATBBAAwAQBBAAwQQABAEwQQAAAEwQQAMAEAQQAMEEAAQBMEEAAABNXWDcADHYTJ070XBPLHMCxTCwa61zDSTZHMYYoroAAACYIIACACc8BtGvXLs2bN0/5+fny+XzasmVL1PpHHnlEPp8vasydOzde/QIAhgjPAdTV1aVp06apqqrqgtvMnTtXhw8fjgy+qAoA8Fmeb0IoKytTWVnZRbfx+/0KBoMxNwUAGPoS8h5QXV2dcnJydOONN2rp0qU6duzYBbft6elROByOGgCAoS/uATR37ly98sorqq2t1U9/+lPV19errKxMZ8+e7XP7yspKBQKByBg7dmy8WwIAJKG4fw5o0aJFkZ9vvvlmTZ06VRMnTlRdXZ1mz5593vYVFRVauXJl5HE4HCaEAGAYSPht2BMmTFB2drZaWlr6XO/3+5WRkRE1AABDX8ID6MMPP9SxY8eUl5eX6F0BAAYRzy/BnTx5Mupqpq2tTfv27VNWVpaysrL07LPPasGCBQoGg2ptbdXTTz+tSZMmqbS0NK6NAwAGN88BtGfPHt15552Rx5+8f7N48WKtX79e+/fv169//WsdP35c+fn5mjNnjn70ox/J7/fHr2sAwKDnOYBmzZp10YkK//KXv/SrISAeRo0aFVPdL3/5S88199xzT0z78urUqVOea9LS0mLaV6x1gBfMBQcAMEEAAQBMEEAAABMEEADABAEEADBBAAEATBBAAAATBBAAwAQBBAAwQQABAEwQQAAAEwQQAMAEAQQAMBH3r+QG4i0QCHiueeutt2La1x133BFTnVexzBq/bds2zzUvvPCC5xpJWrduneeaf/3rX55r6uvrPddg6OAKCABgggACAJgggAAAJgggAIAJAggAYIIAAgCYIIAAACYIIACACQIIAGCCAAIAmCCAAAAmCCAAgAkmI8WAys3N9VxTXV3tuWagJhWVpN/97neea775zW96rnn44Yc918QqLS3Nc01JSYnnGiYjHd64AgIAmCCAAAAmCCAAgAkCCABgggACAJgggAAAJgggAIAJAggAYIIAAgCYIIAAACYIIACACQIIAGCCyUgRs0Ag4LkmlolFv/zlL3uuidVbb73luWbp0qWea06ePOm5BhhquAICAJgggAAAJgggAIAJAggAYIIAAgCYIIAAACYIIACACQIIAGCCAAIAmCCAAAAmCCAAgAkCCABggslIEbNYJuEcqIlF//znP8dU9/DDD3uu6enpiWlfwHDHFRAAwAQBBAAw4SmAKisrdcsttyg9PV05OTmaP3++mpubo7bp7u5WeXm5Ro8erauvvloLFixQZ2dnXJsGAAx+ngKovr5e5eXlamxs1Pbt23XmzBnNmTNHXV1dkW1WrFiht99+W5s2bVJ9fb0OHTqk+++/P+6NAwAGN083IdTU1EQ9rq6uVk5OjpqamlRcXKxQKKSXX35ZGzdu1F133SVJ2rBhgz7/+c+rsbFRt956a/w6BwAMav16DygUCkmSsrKyJElNTU06c+aMSkpKIttMnjxZ48aNU0NDQ5+/o6enR+FwOGoAAIa+mAOot7dXy5cv18yZMzVlyhRJUkdHh1JTU5WZmRm1bW5urjo6Ovr8PZWVlQoEApExduzYWFsCAAwiMQdQeXm5PvjgA73++uv9aqCiokKhUCgy2tvb+/X7AACDQ0wfRF22bJm2bdumXbt2acyYMZHlwWBQp0+f1vHjx6Ougjo7OxUMBvv8XX6/X36/P5Y2AACDmKcrIOecli1bps2bN2vnzp0qKCiIWj99+nSNHDlStbW1kWXNzc06ePCgioqK4tMxAGBI8HQFVF5ero0bN2rr1q1KT0+PvK8TCASUlpamQCCgRx99VCtXrlRWVpYyMjL0xBNPqKioiDvgAABRPAXQ+vXrJUmzZs2KWr5hwwY98sgjkqRf/OIXSklJ0YIFC9TT06PS0lK9+OKLcWkWADB0eAog59wltxk1apSqqqpUVVUVc1NAf8V6O/9AfWh60aJFnms+/fGGRNuxY4fnmh//+McJ6ARDGXPBAQBMEEAAABMEEADABAEEADBBAAEATBBAAAATBBAAwAQBBAAwQQABAEwQQAAAEwQQAMAEAQQAMEEAAQBMxPSNqIAkHT161HPN6dOnPdekpqZ6rlm4cKHnmv7UDQSfz+e55nJmsO9LdXW155ru7u6Y9oXhiysgAIAJAggAYIIAAgCYIIAAACYIIACACQIIAGCCAAIAmCCAAAAmCCAAgAkCCABgggACAJgggAAAJpiMFDF7+eWXPddMmjTJc81TTz3luWYo2rVrl+ea119/PaZ97dmzJ6Y6wAuugAAAJgggAIAJAggAYIIAAgCYIIAAACYIIACACQIIAGCCAAIAmCCAAAAmCCAAgAkCCABgggACAJjwOeecdROfFg6HFQgErNsAAPRTKBRSRkbGBddzBQQAMEEAAQBMEEAAABMEEADABAEEADBBAAEATBBAAAATBBAAwAQBBAAwQQABAEwQQAAAEwQQAMAEAQQAMEEAAQBMEEAAABOeAqiyslK33HKL0tPTlZOTo/nz56u5uTlqm1mzZsnn80WNJUuWxLVpAMDg5ymA6uvrVV5ersbGRm3fvl1nzpzRnDlz1NXVFbXdY489psOHD0fG2rVr49o0AGDwu8LLxjU1NVGPq6urlZOTo6amJhUXF0eWX3nllQoGg/HpEAAwJPXrPaBQKCRJysrKilr+6quvKjs7W1OmTFFFRYVOnTp1wd/R09OjcDgcNQAAw4CL0dmzZ90999zjZs6cGbX8V7/6laupqXH79+93v/3tb921117r7rvvvgv+njVr1jhJDAaDwRhiIxQKXTRHYg6gJUuWuPHjx7v29vaLbldbW+skuZaWlj7Xd3d3u1AoFBnt7e3mB43BYDAY/R+XCiBP7wF9YtmyZdq2bZt27dqlMWPGXHTbwsJCSVJLS4smTpx43nq/3y+/3x9LGwCAQcxTADnn9MQTT2jz5s2qq6tTQUHBJWv27dsnScrLy4upQQDA0OQpgMrLy7Vx40Zt3bpV6enp6ujokCQFAgGlpaWptbVVGzdu1N13363Ro0dr//79WrFihYqLizV16tSE/AcAAAYpL+/76AKv823YsME559zBgwddcXGxy8rKcn6/302aNMk99dRTl3wd8NNCoZD565YMBoPB6P+41HO/7/8HS9IIh8MKBALWbQAA+ikUCikjI+OC65kLDgBgggACAJgggAAAJgggAIAJAggAYIIAAgCYIIAAACYIIACACQIIAGCCAAIAmCCAAAAmCCAAgAkCCABgggACAJgggAAAJgggAIAJAggAYIIAAgCYIIAAACYIIACACQIIAGCCAAIAmCCAAAAmCCAAgAkCCABgIukCyDln3QIAIA4u9XyedAF04sQJ6xYAAHFwqedzn0uyS47e3l4dOnRI6enp8vl8UevC4bDGjh2r9vZ2ZWRkGHVoj+NwDsfhHI7DORyHc5LhODjndOLECeXn5ysl5cLXOVcMYE+XJSUlRWPGjLnoNhkZGcP6BPsEx+EcjsM5HIdzOA7nWB+HQCBwyW2S7iU4AMDwQAABAEwMqgDy+/1as2aN/H6/dSumOA7ncBzO4Ticw3E4ZzAdh6S7CQEAMDwMqisgAMDQQQABAEwQQAAAEwQQAMDEoAmgqqoqXXfddRo1apQKCwv13nvvWbc04J555hn5fL6oMXnyZOu2Em7Xrl2aN2+e8vPz5fP5tGXLlqj1zjmtXr1aeXl5SktLU0lJiQ4cOGDTbAJd6jg88sgj550fc+fOtWk2QSorK3XLLbcoPT1dOTk5mj9/vpqbm6O26e7uVnl5uUaPHq2rr75aCxYsUGdnp1HHiXE5x2HWrFnnnQ9Lliwx6rhvgyKA3njjDa1cuVJr1qzR+++/r2nTpqm0tFRHjhyxbm3A3XTTTTp8+HBk/PWvf7VuKeG6uro0bdo0VVVV9bl+7dq1WrdunV566SXt3r1bV111lUpLS9Xd3T3AnSbWpY6DJM2dOzfq/HjttdcGsMPEq6+vV3l5uRobG7V9+3adOXNGc+bMUVdXV2SbFStW6O2339amTZtUX1+vQ4cO6f777zfsOv4u5zhI0mOPPRZ1Pqxdu9ao4wtwg8CMGTNceXl55PHZs2ddfn6+q6ysNOxq4K1Zs8ZNmzbNug1TktzmzZsjj3t7e10wGHQ/+9nPIsuOHz/u/H6/e+211ww6HBifPQ7OObd48WJ37733mvRj5ciRI06Sq6+vd86d+38/cuRIt2nTpsg2//jHP5wk19DQYNVmwn32ODjn3B133OG+853v2DV1GZL+Cuj06dNqampSSUlJZFlKSopKSkrU0NBg2JmNAwcOKD8/XxMmTNBDDz2kgwcPWrdkqq2tTR0dHVHnRyAQUGFh4bA8P+rq6pSTk6Mbb7xRS5cu1bFjx6xbSqhQKCRJysrKkiQ1NTXpzJkzUefD5MmTNW7cuCF9Pnz2OHzi1VdfVXZ2tqZMmaKKigqdOnXKor0LSrrJSD/ro48+0tmzZ5Wbmxu1PDc3V//85z+NurJRWFio6upq3XjjjTp8+LCeffZZ3X777frggw+Unp5u3Z6Jjo4OSerz/Phk3XAxd+5c3X///SooKFBra6u+//3vq6ysTA0NDRoxYoR1e3HX29ur5cuXa+bMmZoyZYqkc+dDamqqMjMzo7YdyudDX8dBkr72ta9p/Pjxys/P1/79+/W9731Pzc3Neuuttwy7jZb0AYT/U1ZWFvl56tSpKiws1Pjx4/Xmm2/q0UcfNewMyWDRokWRn2+++WZNnTpVEydOVF1dnWbPnm3YWWKUl5frgw8+GBbvg17MhY7D448/Hvn55ptvVl5enmbPnq3W1lZNnDhxoNvsU9K/BJedna0RI0acdxdLZ2engsGgUVfJITMzUzfccINaWlqsWzHzyTnA+XG+CRMmKDs7e0ieH8uWLdO2bdv0zjvvRH19SzAY1OnTp3X8+PGo7Yfq+XCh49CXwsJCSUqq8yHpAyg1NVXTp09XbW1tZFlvb69qa2tVVFRk2Jm9kydPqrW1VXl5edatmCkoKFAwGIw6P8LhsHbv3j3sz48PP/xQx44dG1Lnh3NOy5Yt0+bNm7Vz504VFBRErZ8+fbpGjhwZdT40Nzfr4MGDQ+p8uNRx6Mu+ffskKbnOB+u7IC7H66+/7vx+v6uurnZ///vf3eOPP+4yMzNdR0eHdWsD6rvf/a6rq6tzbW1t7m9/+5srKSlx2dnZ7siRI9atJdSJEyfc3r173d69e50k9/Of/9zt3bvX/ec//3HOOfeTn/zEZWZmuq1bt7r9+/e7e++91xUUFLiPP/7YuPP4uthxOHHihHvyySddQ0ODa2trczt27HBf/OIX3fXXX++6u7utW4+bpUuXukAg4Orq6tzhw4cj49SpU5FtlixZ4saNG+d27tzp9uzZ44qKilxRUZFh1/F3qePQ0tLifvjDH7o9e/a4trY2t3XrVjdhwgRXXFxs3Hm0QRFAzjn3wgsvuHHjxrnU1FQ3Y8YM19jYaN3SgFu4cKHLy8tzqamp7tprr3ULFy50LS0t1m0l3DvvvOMknTcWL17snDt3K/aqVatcbm6u8/v9bvbs2a65udm26QS42HE4deqUmzNnjrvmmmvcyJEj3fjx491jjz025P5I6+u/X5LbsGFDZJuPP/7Yfetb33Kf+9zn3JVXXunuu+8+d/jwYbumE+BSx+HgwYOuuLjYZWVlOb/f7yZNmuSeeuopFwqFbBv/DL6OAQBgIunfAwIADE0EEADABAEEADBBAAEATBBAAAATBBAAwAQBBAAwQQABAEwQQAAAEwQQAMAEAQQAMEEAAQBM/D9E7dVnW1sBsQAAAABJRU5ErkJggg==\n"
          },
          "metadata": {}
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<matplotlib.image.AxesImage at 0x7f0738e11990>"
            ]
          },
          "metadata": {},
          "execution_count": 7
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "<Figure size 640x480 with 3 Axes>"
            ],
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAh8AAADACAYAAACkqgECAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/bCgiHAAAACXBIWXMAAA9hAAAPYQGoP6dpAAAbyUlEQVR4nO3de3BU9f3/8XeC2SVA2HBNSCEYEAtIuZgSjLcRm0KpMFAYKVNntNWWCokdZKZ0cIrOOJ2m1altoahUC2hbLtJyURhgnHArTIByFQQiVJRUSLiUbELIjeR8/3DMz/h+748NLGf3bJ6PmfOHL87uOQvvrG8O7/M5CY7jOAIAAOCSxGifAAAAaFtoPgAAgKtoPgAAgKtoPgAAgKtoPgAAgKtoPgAAgKtoPgAAgKtoPgAAgKtoPgAAgKtoPgAAgKtuu1VvvHDhQnn55ZelrKxMhg0bJgsWLJCcnJzrvq6pqUnOnj0rKSkpkpCQcKtOD3HOcRypqqqSjIwMSUxsXY9N7SKaqF14Vatq17kFVqxY4fh8Pmfx4sXOhx9+6PzkJz9xUlNTnfLy8uu+trS01BERNraIbKWlpdQumyc3apfNq1s4tXtLmo+cnBwnPz+/+b8bGxudjIwMp7Cw8LqvraioiPpvHFv8bBUVFdQumyc3apfNq1s4tRvxmY/6+nrZv3+/5OXlNWeJiYmSl5cnxcXFav+6ujqprKxs3qqqqiJ9SmjDWnMJmdpFLKF24VXh1G7Em4+LFy9KY2OjpKWltcjT0tKkrKxM7V9YWCiBQKB569OnT6RPCQgLtQuvonbhNVG/22Xu3LkSDAabt9LS0mifEhAWahdeRe0i2iJ+t0v37t2lXbt2Ul5e3iIvLy+X9PR0tb/f7xe/3x/p0wBajdqFV1G78JqIX/nw+XySnZ0tRUVFzVlTU5MUFRVJbm5upA8HRAy1C6+iduE5rRqnDtOKFSscv9/vLF261Dl27Jgzffp0JzU11SkrK7vua4PBYNQnddniZwsGg9Qumyc3apfNq1s4tXtLmg/HcZwFCxY4mZmZjs/nc3Jycpzdu3eH9Tp+CNgiubX2C5zaZYuVjdpl8+oWTu0mOI7jSAyprKyUQCAQ7dNAnAgGg9K5c2dXjkXtIpKoXXhVOLV7y5ZXh/uSkpJU1tjYaO7b1NR0q08HAOJK+/btVVZfX68yvl+vL+q32gIAgLaF5gMAALiK5gMAALiK5gMAALiK5gMAALiKu11iTM+ePVU2evRolVm3MV24cEFlBw4cMI9j7VtTUxPOKQJhs55uGWN39wPy7W9/W2VZWVkqq66uVtnBgwdVdv78efM4Fy9evIGzi09c+QAAAK6i+QAAAK6i+QAAAK6i+QAAAK5i4DTCrAE765HWc+fONV8/fvx4lV25ckVl1sBoenq6yhYtWmQep6KiQmWvvfaayi5duqSyUEu2o2277Tb9ddKvXz+VPfPMMyqbOnWqyqzaExE5fvy4ytavX68yaxDw6NGjKrt27Zp5HHjbjBkzVGbVnohI7969VWYNh2ZmZqps48aNKktMtP9e/6c//Ullu3fvVtnly5fN18cTrnwAAABX0XwAAABX0XwAAABX0XwAAABXMXAaYe3atVPZY489prIxY8aYr7dWf/T7/Srr0aOHympra1X23e9+1zyONRyYnJyssjfffFNlp06dMt8TbYNVjyIiw4cPV1lBQYHKJk+erLL27durzKpxEZG+ffuqbMiQISrbvHmzyv74xz+q7MyZM+ZxGhoazBzR5fP5VPb444+r7JVXXlGZVWci9hB9WlqayqxBUOu71KpHEZHnnnsurPNcs2aN+fp4wpUPAADgKpoPAADgKpoPAADgKpoPAADgKgZOI8x6NPODDz6oslAr4FmOHDmisiVLlqhs5cqVKsvOzjbfc/HixSp79NFHVbZv3z6VMXDadqSkpKjsnnvuMfedOXOmyh555BGVWUPZllADn9ZwYK9evVQ2btw4lVmrmf797383j7N///7rnSKiwBoEffjhh1VmfcdaA/0iIlVVVSqbP3++yqyB5atXr6rs3//+t3kc6/vYWl21Y8eOKquurjbf06u48gEAAFxF8wEAAFxF8wEAAFxF8wEAAFzFwOlNsAaf8vLyVHbnnXeq7Ny5c+Z7rlu3TmV79+5V2T/+8Q+V1dTUqKy4uNg8Tl1dnco6d+6ssgsXLqjMWtGPx5LHJ2tA7he/+IW57+jRo1V25coVle3cuVNlJ0+eVFlJSYl5nBMnTqisQ4cOKps2bZrKJk6cqLLS0lLzOMePH1eZNVwId1kr3I4YMSKs1xYVFZm5NVy8aNEilVkrnHbq1EllVo2L2MOl1srAbaHOuPIBAABcRfMBAABcRfMBAABcRfMBAABcRfMBAABcxd0uYQi1HPTgwYNVNnXqVJVZd4eEWjramrA+evTo9U4xpLvvvtvML168qDJrKe2vf/3rKtu+ffsNnw+85a677lJZ//79w359MBhU2dtvv62yDRs2qCzUUtjWXV1NTU0qGzlypMqsJdcHDhxoHic9PV1lH3/8sbkvIi8hIcHMrTtbevbsqTJrOfJQS+YXFhaqzKpdi9/vV1lSUpK5r3VHofX/F2u/cM/HK7jyAQAAXEXzAQAAXEXzAQAAXEXzAQAAXMXAaRgaGxvNPNSgWjisIVQRkWHDhqns2LFjKrMG7BITdS/Zvn178zjdu3dX2enTp1W2a9cu8/VoG6w6aw3rZ8daotpahr017rjjDpVZw7LWcF+oQb5QP/dwR79+/cz8gQceUFlqaqrKrCXKrZsERG5umNMabA01cFpRUaGyPXv2qMx6/EW84coHAABwFc0HAABwFc0HAABwVaubjx07dsiECRMkIyNDEhISZO3atS1+3XEcef7556VXr16SnJwseXl55uOyAbdRu/AqahfxptUDp9XV1TJs2DB58sknZfLkyerXX3rpJZk/f7689dZbkpWVJfPmzZOxY8fKsWPHQg4/elVZWZnKLl26pLKMjAyVdenSxXzPH/zgByo7fvy4yqyVFquqqlQWapVI6/iLFy8O6zheRe223oEDB1R25swZc1+rzgOBgMoyMzNVZq2ua9WziD2sPWHCBJVZq/vW19erbOfOneZxzp07Z+bR0BZr98KFC2aenJysMuvPtUOHDiqzBvpFRAoKClT2/vvvq6y8vFxlPp9PZaG+d63/Z2zdulVltbW15uvjSaubj3HjxplLFIt8/hv+hz/8QX75y1/KxIkTReTzpZTT0tJk7dq1Mm3atJs7W+AmULvwKmoX8SaiMx+nT5+WsrIyycvLa84CgYCMGjVKiouLzdfU1dVJZWVliw1wG7ULr6J24UURbT6+uKSUlpbWIk9LSzMvN4l8/kCfQCDQvPXp0yeSpwSEhdqFV1G78KKo3+0yd+5cCQaDzVtpaWm0TwkIC7ULr6J2EW0RXeH0i0dQl5eXS69evZrz8vJyGT58uPkav99vPpLYC3bs2KGyTZs2qewb3/iGyqxhKBGR0aNHq2zbtm0qsx4BbT0ueubMmeZxrL8RnThxQmVtYfBJpO3Vbrj27t2rsqVLl5r7ZmVlqcwaQv3d736nMmtQe+PGjeZxrJUvrSFM67Hk7777rsoOHTpkHscaYoxF8Vq7of4paP369Sr76lUfEZHs7Oyw9hP5vBn7qp///Ocq+/Wvf62ys2fPquz+++83j3Pq1Ckzb4sieuUjKytL0tPTpaioqDmrrKyUPXv2SG5ubiQPBUQUtQuvonbhRa2+8nHlypUW3dvp06fl0KFD0rVrV8nMzJRZs2bJr371KxkwYEDzLV8ZGRkyadKkSJ430GrULryK2kW8aXXzsW/fvhb/NDB79mwREXniiSdk6dKlMmfOHKmurpbp06dLRUWF3H///bJp0ybP3muO+EHtwquoXcSbVjcfDz30UMgFVEREEhIS5MUXX5QXX3zxpk4MiDRqF15F7SLeRP1uFwAA0LZE9G6Xtsaa0H/11VdV9sknn6jsscceM9/Tmk7v2LGjyqZPn64yaznigQMHmsf517/+pTJrGff/39+2EP+sP/8NGzaY+1rzBT169FBZp06dVPboo4+qrKGhwTzOk08+qbL+/fur7K233lLZm2++qTLrbgXErlWrVqns8uXLKrNWdn3ggQfM9+zevbvKrGX858yZE9Z+1mMFRETeeOMNlVl3GbYFXPkAAACuovkAAACuovkAAACuovkAAACuSnBibKKwsrIy5LCOV6Wmpqps1KhR5r7jx49XmTXIZy1bnZioe8lQQ3s//vGPVbZy5UqV1dXVma/3imAwaC6zfSvEY+22Rt++fVW2evVqlQ0dOlRl1lLm5eXl5nGsJbKtJbet4cAzZ86oLMa+AptRu+FLSEhQ2eDBg1X2ne98x3y9NYhqfUdbtWcdO1RN/exnP1PZsmXLVPa///3PfL1XhFO7XPkAAACuovkAAACuovkAAACuovkAAACuYuA0xnTp0kVl3/zmN1U2f/58lVmrmYb64124cKHKFixYoLKPPvrIfL1XMLTnHmvgeeTIkSorLi5WmTW0F2pYurCwUGXvvPOOyj788EPz9V5B7d4cq6batWtn7tutWzeV5eTkqGzevHkqs2o81PeutTrwb3/7W5UdPnxYZVVVVeZ7xiIGTgEAQMyh+QAAAK6i+QAAAK6i+QAAAK7SzwJGVFmPhv70009VduTIEZUNGDBAZY2NjeZx7r//fpVZjxa3Hl996tQp8z3RNliPEBcR6dmzp8ruvvtulYU7415ZWWnmBw4cUJnXh0sReVadXbt2zdzXWk334MGDKtu1a5fKrBoPNSw9duxYlfXr109l1g0F7733nsqs72yv4MoHAABwFc0HAABwFc0HAABwFc0HAABwFQOnMcZa4fT73/++ykaPHq2yzz77TGWbN282j/OjH/1IZYMGDVKZz+dT2aJFi1RWVlZmHgfelpKSojJrxV0RkdmzZ6ssNzdXZU1NTSqzBgGvXLliHifGFmVGHLBW57Vqd8KECSo7d+6cyt59913zOEOHDg3rOE899ZTKrAHs5cuXm8fxAq58AAAAV9F8AAAAV9F8AAAAV9F8AAAAV9F8AAAAV3G3S4y56667VDZ58mSVWXfFrF69WmXFxcXmcYYPH66y7OxslVnLsO/cuVNl3O3iLdZ0v7U8+rhx41Q2Z84c8z379++vskuXLqnMWrL/6tWrKrPu8gqVHz58WGXWYwkAi9/vV1leXp7KMjIyVLZnzx6VhbrbpaKiQmX33nuvyqzv4lGjRqls27Zt5nGsO3BiDVc+AACAq2g+AACAq2g+AACAq2g+AACAqxg4jZLk5GQzt4bp+vXrp7KTJ0+qbMOGDSr7z3/+Yx7n0KFDKrOGnAYOHKiyIUOGqKyoqMg8DqLLGiwVEbn99ttVNnXqVJVZyzz36dPHfM9Tp06p7I033lDZ1q1bVTZz5kyVhfoZsR4DMHjwYJUxcIqvCvXzYA36W4+wuO02/b/MgwcPqizUoL9V09brR4wYobJu3bqpzHpUgVdw5QMAALiK5gMAALiK5gMAALiK5gMAALiKgVMXWENK1qp2IiIPP/ywyqwhKWugacuWLSqrq6szj2MNOVn7NjQ0qMxa0S8hIcE8juM4Zg539OrVy8ytweYZM2aozBpys2pHROTPf/6zylasWKEya4C6Y8eOKgtVU9aQnc/nM/cFvixUnUycOFFlPXr0UNknn3yish07dqjMWrFXROTIkSMqu3Dhgspqa2tVVlNTozJrxVSv4MoHAABwFc0HAABwFc0HAABwVauaj8LCQhk5cqSkpKRIz549ZdKkSVJSUtJin9raWsnPz5du3bpJp06dZMqUKVJeXh7RkwZai9qFV1G7iEetGjjdvn275Ofny8iRI+XatWvy3HPPyZgxY+TYsWPNA2PPPvusbNiwQVatWiWBQEAKCgpk8uTJsmvXrlvyAbygffv2Kps2bZq574ABA1T23//+V2Vvv/22yqwBPWuVPhF7NcukpCSVWatEWkOksT5Y2lZrNysry8ytuvja176mso8++khlixcvNt/znXfeUZk1xGz9PFgDp6FYg3ehBvziQVut3VshMzPTzIcPH64yqyZ3796tsk2bNqnsjjvuMI+Tn5+vstzcXJVZPyOXL19WWffu3c3jfPbZZ2YeS1rVfHz1N3np0qXSs2dP2b9/vzz44IMSDAblL3/5iyxbtqz5ro0lS5bIoEGDZPfu3XLPPfdE7syBVqB24VXULuLRTc18BINBERHp2rWriIjs379fGhoaJC8vr3mfgQMHSmZmZsi17uvq6qSysrLFBtxq1C68itpFPLjh5qOpqUlmzZol9913X/ODxsrKysTn80lqamqLfdPS0qSsrMx8n8LCQgkEAs1bqIdWAZFC7cKrqF3EixtuPvLz8+Xo0aPmIkKtMXfuXAkGg81baWnpTb0fcD3ULryK2kW8uKEVTgsKCmT9+vWyY8cO6d27d3Oenp4u9fX1UlFR0aILLy8vl/T0dPO9/H6/+P3+GzkNz7AeXx5qIKldu3Yqq6qqUllOTo7K7rzzTpU98sgj5nGsAavGxkaVnT17VmVfXPb1orZWu9bquiL2So/hDg1bjx8XEXn88cdVZtW5Nezav39/lZ05c8Y8ztq1a1X2wQcfmPvGk7ZWuzfLGsDv0KGDue8XV5G+zPoutgZBf/rTn6os1Pe7tZJqcnKyyvbu3auyffv2qay6uto8jhe06sqH4zhSUFAga9askS1btqhJ+uzsbElKSpKioqLmrKSkRM6cOWNO9AJuoXbhVdQu4lGrrnzk5+fLsmXLZN26dZKSktL874mBQECSk5MlEAjIU089JbNnz5auXbtK586d5ZlnnpHc3FwmrhFV1C68itpFPGpV8/Haa6+JiMhDDz3UIl+yZIn88Ic/FBGR3//+95KYmChTpkyRuro6GTt2rLz66qsROVngRlG78CpqF/GoVc1HOP8m3L59e1m4cKEsXLjwhk8KiDRqF15F7SIe8WwXAADgqhu62wWtY91xcO7cubBf37dvX5U9/fTTYe0XivW3KWsp5r/97W8qa825I7pC3THy1WeDiIgMHTpUZdYdVFbWGk1NTSpLTNR/D1q3bp35+s2bN6vs/PnzN3VOiD/Wd9yX7xL6MusxEtZS7NZdMVb2xQJw4bCWQrf+ycy6y6uhoSHs48QarnwAAABX0XwAAABX0XwAAABX0XwAAABXMXDqghMnTqgs1NMmx48frzJr+d1wh0s3bNhg5v/85z9VtmrVKpVdvXo1rOMgNn388cdm/te//lVl1oDeoEGDVBZqmK579+4q69atm8qsYdetW7eqbOPGjeZxrOeQhLs0PNo2a4lyEZEtW7aozFqgLRAIqMxahv3YsWPmcaxboV9//XVz33jHlQ8AAOAqmg8AAOAqmg8AAOAqmg8AAOAqBk5dUFtbq7LVq1eb+waDQZXde++9Kjt8+LDKrCFSVn6EZefOnWFlt0JKSorKrFWA6+vrzdfX1NRE/JzQNnzxROCvsr47rX1HjBihsh07dqhs+fLlN3B2bQtXPgAAgKtoPgAAgKtoPgAAgKtoPgAAgKsSnBhbGrCystJcRa6tSEpKUpmXH5scbcFgUDp37uzKsdp67SKyqN3o8vl8Kgs1BI2WwqldrnwAAABX0XwAAABX0XwAAABX0XwAAABX0XwAAABXsbx6jOHOFgCIPu5subW48gEAAFxF8wEAAFxF8wEAAFxF8wEAAFxF8wEAAFxF8wEAAFxF8wEAAFxF8wEAAFwVc82H4zjRPgXEETfridpFJFG78Kpw6inmmo+qqqponwLiiJv1RO0ikqhdeFU49ZTgxFjL29TUJGfPnpWUlBSpqqqSPn36SGlpqXTu3Dnap3bTKisr+TwucRxHqqqqJCMjQxIT3emxqV3viOXPQ+1GViz/Wd+IWP48randmHu2S2JiovTu3VtERBISEkREpHPnzjH3m3wz+DzuCAQCrh6P2vWeWP081G7k8XncEW7txtw/uwAAgPhG8wEAAFwV082H3++XF154Qfx+f7RPJSL4PG1HvP3e8Hnajnj7veHzxKaYGzgFAADxLaavfAAAgPhD8wEAAFxF8wEAAFxF8wEAAFwVs83HwoUL5fbbb5f27dvLqFGjZO/evdE+pbDt2LFDJkyYIBkZGZKQkCBr165t8euO48jzzz8vvXr1kuTkZMnLy5OTJ09G52Svo7CwUEaOHCkpKSnSs2dPmTRpkpSUlLTYp7a2VvLz86Vbt27SqVMnmTJlipSXl0fpjGODV+uX2qV2qd3YEO/1G5PNx8qVK2X27NnywgsvyIEDB2TYsGEyduxYOX/+fLRPLSzV1dUybNgwWbhwofnrL730ksyfP19ef/112bNnj3Ts2FHGjh0rtbW1Lp/p9W3fvl3y8/Nl9+7d8v7770tDQ4OMGTNGqqurm/d59tln5b333pNVq1bJ9u3b5ezZszJ58uQonnV0ebl+qV1ql9qNDXFfv04MysnJcfLz85v/u7Gx0cnIyHAKCwujeFY3RkScNWvWNP93U1OTk56e7rz88svNWUVFheP3+53ly5dH4Qxb5/z5846IONu3b3cc5/NzT0pKclatWtW8z/Hjxx0RcYqLi6N1mlEVL/VL7bY91G7sirf6jbkrH/X19bJ//37Jy8trzhITEyUvL0+Ki4ujeGaRcfr0aSkrK2vx+QKBgIwaNcoTny8YDIqISNeuXUVEZP/+/dLQ0NDi8wwcOFAyMzM98XkiLZ7rl9qNb9RubIu3+o255uPixYvS2NgoaWlpLfK0tDQpKyuL0llFzhefwYufr6mpSWbNmiX33XefDBkyREQ+/zw+n09SU1Nb7OuFz3MrxHP9UrvxjdqNXfFYvzH3VFvErvz8fDl69Kjs3Lkz2qcCtAq1Cy+Lx/qNuSsf3bt3l3bt2qmJ3fLycklPT4/SWUXOF5/Ba5+voKBA1q9fL1u3bm1+9LbI55+nvr5eKioqWuwf65/nVonn+qV24xu1G5vitX5jrvnw+XySnZ0tRUVFzVlTU5MUFRVJbm5uFM8sMrKysiQ9Pb3F56usrJQ9e/bE5OdzHEcKCgpkzZo1smXLFsnKymrx69nZ2ZKUlNTi85SUlMiZM2di8vPcavFcv9RufKN2Y0vc12+UB15NK1ascPx+v7N06VLn2LFjzvTp053U1FSnrKws2qcWlqqqKufgwYPOwYMHHRFxXnnlFefgwYPOp59+6jiO4/zmN79xUlNTnXXr1jkffPCBM3HiRCcrK8upqamJ8plrM2bMcAKBgLNt2zbn3LlzzdvVq1eb93n66aedzMxMZ8uWLc6+ffuc3NxcJzc3N4pnHV1erl9ql9qldmNDvNdvTDYfjuM4CxYscDIzMx2fz+fk5OQ4u3fvjvYphW3r1q2OiKjtiSeecBzn89u+5s2b56SlpTl+v9/51re+5ZSUlET3pEOwPoeIOEuWLGnep6amxpk5c6bTpUsXp0OHDs73vvc959y5c9E76Rjg1fqldqldajc2xHv9JjiO49zaaysAAAD/T8zNfAAAgPhG8wEAAFxF8wEAAFxF8wEAAFxF8wEAAFxF8wEAAFxF8wEAAFxF8wEAAFxF8wEAAFxF8wEAAFxF8wEAAFxF8wEAAFz1fz8F/pMD4ty4AAAAAElFTkSuQmCC\n"
          },
          "metadata": {}
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ytv3NxF-kgxN"
      },
      "source": [
        "Let us augment the whole dataset and see if this improves the test accuracy"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "iNzNAoDBkRzj",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "02d1a238-789d-4aaf-ede7-61b58fefd172"
      },
      "source": [
        "# hyperparameters\n",
        "angleconstraint = 60\n",
        "naugmentations = 5\n",
        "\n",
        "# augment\n",
        "augdata = train_X # we include the original images also in the augmented dataset\n",
        "auglabel = train_y\n",
        "for ii in range(naugmentations):\n",
        "  augdata = np.concatenate((augdata, augRotate(train_X, angleconstraint))) # concatenate the augmented data to the set\n",
        "  auglabel = np.concatenate((auglabel, train_y))  # the labels don't change when we augment\n",
        "\n",
        "# check the test accuracy\n",
        "testpred = NN(augdata, auglabel, test_X)\n",
        "print('Accuracy after rotation augmentation is ', Accuracy(test_y, testpred))"
      ],
      "execution_count": 8,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Accuracy after rotation augmentation is  0.6616\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "E88Nt9s1p5R6"
      },
      "source": [
        "The angle constraint is a hyperparameter which we have to tune using a validation set. (Here we are not doing that for time constraints). Let us try a grid search to find the best angle constraint."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "aiaFRLREmGp6",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "bada13ad-505d-49bf-ecf2-f8bf5d965737"
      },
      "source": [
        "angleconstraints = [0,10,20,30,40,50,60,70,80,90] # the values we want to test\n",
        "accuracies = np.zeros(len(angleconstraints), dtype=np.float) # we will save the values here\n",
        "\n",
        "for ii in range(len(angleconstraints)):\n",
        "  # create the augmented dataset\n",
        "  augdata = train_X # we include the original images also in the augmented dataset\n",
        "  auglabel = train_y\n",
        "  for jj in range(naugmentations):\n",
        "    augdata = np.concatenate((augdata, augRotate(train_X, angleconstraints[ii]))) # concatenate the augmented data to the set\n",
        "    auglabel = np.concatenate((auglabel, train_y))  # the labels don't change when we augment\n",
        "\n",
        "  # check the test accuracy\n",
        "  testpred = NN(augdata, auglabel, test_X)\n",
        "  accuracies[ii] = Accuracy(test_y, testpred)\n",
        "  print('Accuracy after rotation augmentation constrained by ',angleconstraints[ii], ' is ', accuracies[ii], flush=True)"
      ],
      "execution_count": 9,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "<ipython-input-9-2f14d3721156>:2: DeprecationWarning: `np.float` is a deprecated alias for the builtin `float`. To silence this warning, use `float` by itself. Doing this will not modify any behavior and is safe. If you specifically wanted the numpy scalar type, use `np.float64` here.\n",
            "Deprecated in NumPy 1.20; for more details and guidance: https://numpy.org/devdocs/release/1.20.0-notes.html#deprecations\n",
            "  accuracies = np.zeros(len(angleconstraints), dtype=np.float) # we will save the values here\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Accuracy after rotation augmentation constrained by  0  is  0.6472\n",
            "Accuracy after rotation augmentation constrained by  10  is  0.6727\n",
            "Accuracy after rotation augmentation constrained by  20  is  0.673\n",
            "Accuracy after rotation augmentation constrained by  30  is  0.6737\n",
            "Accuracy after rotation augmentation constrained by  40  is  0.6731\n",
            "Accuracy after rotation augmentation constrained by  50  is  0.6701\n",
            "Accuracy after rotation augmentation constrained by  60  is  0.6675\n",
            "Accuracy after rotation augmentation constrained by  70  is  0.661\n",
            "Accuracy after rotation augmentation constrained by  80  is  0.6462\n",
            "Accuracy after rotation augmentation constrained by  90  is  0.6469\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "2oVDRYP2rxob"
      },
      "source": [
        "Let us see the best value for angle constraint: (Ideally this should be done on validation set, not test set)"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "LqthJa_pmMHz",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 485
        },
        "outputId": "08ef09cf-d2cf-4d1e-d555-f24f91437696"
      },
      "source": [
        "fig = plt.figure()\n",
        "ax = fig.add_axes([0.1, 0.1, 0.8, 0.8]) # main axes\n",
        "# plot the variation of accuracy\n",
        "ax.plot(angleconstraints, accuracies)\n",
        "ax.set_xlabel('angle')\n",
        "ax.set_ylabel('accuracy')\n",
        "# plot the maximum accuracy\n",
        "maxind = np.argmax(accuracies)\n",
        "plt.scatter(angleconstraints[maxind], accuracies[maxind], c='red')"
      ],
      "execution_count": 10,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<matplotlib.collections.PathCollection at 0x7f0738b87280>"
            ]
          },
          "metadata": {},
          "execution_count": 10
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "<Figure size 640x480 with 1 Axes>"
            ],
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAlgAAAHDCAYAAAD4L2TBAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/bCgiHAAAACXBIWXMAAA9hAAAPYQGoP6dpAABeOElEQVR4nO3deVzUdf4H8NfcMNwyHAMil6ImHoVJpGKtGFnblrmua7aatbYpJcJuKR26mWHH6lrZ5mqrududPytT1FUs79v1SuVG8OCWWxiY+f7+AEYnQBEGvnO8no/HPIgvn5l5j1+LV9/P5/v+SARBEEBEREREZiMVuwAiIiIiW8OARURERGRmDFhEREREZsaARURERGRmDFhEREREZsaARURERGRmDFhEREREZsaARURERGRmDFhEREREZsaARURERGRmFhGwPvzwQwQFBcHBwQGRkZE4fPjwTceXl5cjLi4OWq0WKpUKYWFhSElJMf48KCgIEomk1SMuLs445r777mv18+eee67bPiMRERHZD7nYBXz11VdITEzEypUrERkZieXLlyM2NhZpaWnw9vZuNV6n02HcuHHw9vbG+vXr4e/vjwsXLsDd3d045siRI9Dr9cbvz5w5g3HjxmHSpEkmrzVz5kwsWrTI+L1arTb/ByQiIiK7I3rAWrZsGWbOnIkZM2YAAFauXInNmzdjzZo1mD9/fqvxa9asQVlZGfbv3w+FQgGg6YrVjby8vEy+f+uttxAaGooxY8aYHFer1fD19e1U3QaDAZcvX4aLiwskEkmnXoOIiIisiyAIqKqqgp+fH6TSm0wECiKqr68XZDKZ8O2335ocnzZtmvCb3/ymzeeMHz9emDp1qjBz5kzB29tbGDRokPDmm28KjY2N7b6Hp6en8Oabb5ocHzNmjKDRaARPT09h0KBBwvz584Wampp2a62rqxMqKiqMj7NnzwoA+OCDDz744IMPO3zk5+ffNOOIegWrpKQEer0ePj4+Jsd9fHxw/vz5Np+TnZ2NnTt3YurUqUhJSUFmZiZmz56NhoYGLFy4sNX47777DuXl5XjqqadMjj/xxBMIDAyEn58fTp06hXnz5iEtLQ0bNmxo832XLFmC119/vdXx/Px8uLq6dvATExERkTWrrKxEQEAAXFxcbjpOIgiC0EM1tXL58mX4+/tj//79iIqKMh5/6aWXsGvXLhw6dKjVc8LCwlBXV4ecnBzIZDIATdOM7777Lq5cudJqfGxsLJRKJX744Yeb1rJz506MHTsWmZmZCA0NbfXz+vp61NfXG79v+QOuqKhgwCIiIrITlZWVcHNzu+Xvf1GvYGk0GshkMhQWFpocLywsbHdtlFarhUKhMIYrABg4cCAKCgqg0+mgVCqNxy9cuIAdO3a0e1XqRpGRkQDQbsBSqVRQqVQd+lxERERk30Rt06BUKhEREYHU1FTjMYPBgNTUVJMrWjcaOXIkMjMzYTAYjMfS09Oh1WpNwhUArF27Ft7e3nj44YdvWcuJEycANAU4IiIioq4QvQ9WYmIiVq9ejXXr1uHcuXOYNWsWampqjHcVTps2DUlJScbxs2bNQllZGeLj45Geno7NmzcjOTnZpMcV0BTU1q5di+nTp0MuN71Ql5WVhTfeeAPHjh1Dbm4uNm7ciGnTpiE6OhpDhgzp/g9NRERENk30Ng2TJ09GcXExFixYgIKCAgwbNgxbt241LnzPy8szuQ0yICAA27ZtQ0JCAoYMGQJ/f3/Ex8dj3rx5Jq+7Y8cO5OXl4emnn271nkqlEjt27MDy5ctRU1ODgIAATJw4Ea+++mr3flgiIiKyC6IucrdmHV3kRkRERLajo7//RZ8iJCIiIrI1DFhEREREZsaARURERGRmDFhEREREZib6XYREJBK9HtizB7hyBdBqgdGjgRsa+BIRUecxYBHZow0bgPh44OLF68d69wbeew94/HHx6iIishGcIiSyNxs2AL/9rWm4AoBLl5qOd2BrKSIiujkGLCJ7otc3Xblqq/1dy7G5c5vGERFRp3GKkMiOVO7cjQzBGWlDY5GuCUSaVyCKnTwQUFGI4LJLCL56uenrf3dBG3s/pFKJ2CUTEVkldnLvJHZyJ0tW16BHZlE10gqqkF5YhbTCKqQXVOFyRV2HX0MllyLI0wnBGicEezkh2LP5q8YJnk5KSCQMX0Rkfzr6+59XsIisWIPegNySGmOASiusQnphNS6U1sDQzv86aSuLEVZyAf2LLyCs5AK8q6/iops3cnr5I8fDH9m9/JCvCUB9owFpzeHsl1xUcmPYuvERpHGCq4Oimz81EZHlY8AisgIGg4CLV681B6gq45WprOJqNOjbTlIeagX6+7qgv48Lwpq/9tOo4TawX9OC9rYuXkskQO/eaMzMwqUqHbJLapBbUoOc5kd2cQ0uV1xDVX0jTl2swKmLFa1eQuOsNAlcIRonBGucEeiphoOCbSCIyD4wYBFZEEEQUFRVf31qr/lremE1rjW0vfDcSSkzBqgwHxf09236qnFuZxrvvfea7haUSExDVsvY5cshVyoQ6KlAoKcT0N/06XUNeuSV1SK7uAa5pTXIKW4OYKU1KK6qR0m1DiXVOhzJvWryPIkE8HNzbCN8OaG3hyPkMt5zQ0S2g2uwOolrsKirrtbomsNTyxqpaqQVVqHiWkOb45VyKfp6ORsDVH9fZ4T5uMDPzfH2F6O31QcrIABYvrxLfbCq6hqQW1KLHGPwqm668lVSg6q6xnafJ5dK0KeXunX48nKCj4sDF9sTkcXo6O9/BqxOYsCijqqpb0RGUfUNa6SarkwVVdW3OV4qAYI1TteDVPMUX2AvtXmv8vRgJ3dBEFBWozNONd74yC2tQV2Dod3nOiiaFtuHeDkZF92HeDVNO3qoFVxsT0Q9igGrmzFg0S/VN+qRXVxjMrWXVliF/LJr7T6nt4ejyRqpMB8XhHg52dVaJYNBQEFlXevgVVKDvLJaNLa3Wh+Aq4McwV7OCNE0hy8vJ9zVxx29PdQ9+AmIyJ4wYHUzBiz7pTcIuFDaEqSqjUEqp6QG+nbCgJeL6oY1Uk1Te/18XOCs4jLIm2nQG3Dx6rXmqcZa45RjbkktLpW3HVyVMimSHhqAp+4N4tUtIjI7BqxuxoBlG3SNBpRf06G8tgFXa3S4WtuAimtNX6/W6lBe0/y15ftrDSiv1bV7556rg/yGNVJNX8N8XNDLSdnDn8z2XdPpcaGs6UpXdknTmq+zVyrx8+VKAEDMQB+8+9sh8OCfPRGZEQNWN2PAsiyCIKCyrhHltTeEI2MwajAeb/radLy8tgHV9e0vvL4ZB4XUGJ5unOLzcVXxqomIBEHAvw9cwJubz0GnN0Dr5oD3fn8nRgT3Ers0IrIRDFjdjAGr+9Q16I1XjG4MQy2h6cbAdLVWh4raBpRfa2h3eu5WJBLAzVEBD7US7urrX90dlfBQK+Du1PT1xp/7uvLONkt25lIFXvjif8gpqYFUAiTEhGH2/X0h4zkjoi5iwOpmDFgdV1HbgMziapNwdD0wtQSp68fb6/fUEY4KWVMoUivh4dT8tTkUtYSo68ebfubioOAvXhtUXd+IBd+dwYb/XQIA3BvqieWTh8Hb1UHkyojImjFgdTMGrI6pqG3AqHd23rQHUltkUgk81Iobriw1ByWnX1xdag5MLQHKnu6+o475v2MX8dr3Z1Cr08PTSYmlvxuK+/p7i10WEVkp7kVIFuH0pQpU1TVCJW9as9QyxWYMR81frx9Xwt1JAReVnGuZyCwmRvTGsD7ueP7z/+HclUo8tfYI/hQdgr/E9oeC3eOJqJswYFG3yixq2ig4OswLq6cNF7kaslehXs74dva9SE45h38fuIB/7s7GoZwyfDDlTgT0Ys8sIjI//u8bdavM4moAQF9vZ5ErIXvnoJBh0aPhWPlkBFwd5DiRX46H3tuDzaeuiF0aEdkgBizqVplFzQHLiwGLLMOD4b5IiR+NiEAPVNU3Iu7z43j529Oo68LNFUREv8SARd0qs6gGABDKK1hkQXp7qPHls/dg9n2hkEiAzw/l4dEV+5BRWCV2aURkIxiwqNtU1DagpLppQ+NQLyeRqyEypZBJ8dKDA/Dvp0dA46xCWmEVHlmxF18fyQdvriairmLAom6TWdx0NcDX1QEuDgqRqyFq2+h+XtgSPxqj+2lQ12DAS/93CvFfnkBVXYPYpRGRFWPAom5jXH/F6UGycF4uKqybMQIvPdgfMqkEG09exq8/2ItTF8vFLo2IrBQDFnUbBiyyJlKpBLPv64uv/xQFf3dHXCitxcSP9uNfe3M4ZUhEt40Bi7pNVjEXuJP1iQj0QMqc0XhwkC8a9ALe2HQWf1x3FGU1OrFLIyIrwoBF3YYtGshauakV+OjJu/DGo4OglEuRer4ID723B4eyS8UujYisBAMWdYu6Bj3yr9YC4BQhWSeJRII/RAXhu9kjEeLlhILKOkxZfRDv7ciA3sApQyK6OQYs6hbZxTUQBMDNUQGNs1Lscog67Q4/V2x6YRR+G9EbBgH4+450TP34IAor68QujYgsGAMWdYsbt8jhps1k7dRKOf42aSj+Pnko1EoZDmaXYfx7e/Dj+SKxSyMiC8WARd2iZf0VG4ySLZlwZ29semEUBvm5oqxGhxmfHMGbm89C12gQuzQisjAMWNQtstiigWxUiJczNsy+F0/dGwQAWL0nB5NW7kdeaa24hRGRRWHAom7BHlhky1RyGf76m0FY9YcIuDkqcPJiBR5+fw9+OHlZ7NKIyEIwYJHZNeoNyClp6oHV18tF5GqIus8Dg3yxJX40hgd6oKq+ES988T8kbTiFazq92KURkcgYsMjs8q9eg05vgEouhb+Ho9jlEHUrP3dHfPnsPXjhV30hkQBfHM7Hox/uRXphldilEZGIGLDI7FrWX4V4OUMm5R2EZPvkMin+/EB/fPpMJLxcVEgvrMZvVuzFF4fzuM0OkZ1iwCKzu7FFA5E9GdlXgy3xoxEd5oW6BgOSNpzGC1/8D5V1DWKXRkQ9jAGLzI5b5JA90zir8MlTdyNp/ADIpRJsOnUFv35/L07ml4tdGhH1IAYsMjveQUj2TiqV4E9jQvH1c1Ho7eGIvLJaTPxoP1bvzoaB2+wQ2QUGLDIrQRDYA4uo2V19PLB5zmg8NNgXjQYBb6acwzPrjqC0ul7s0oiomzFgkVkVVdWjqr4RUgkQpFGLXQ6R6NwcFfjwibvw5oRwqORS/JhWjIfe34MDWaVil0ZE3YgBi8yqZXqwTy81VHKZyNUQWQaJRIKpkYH4/vmR6OvtjMLKejzx8UEs256ORj232SGyRQxYZFZcf0XUvgG+rtj4/Ej8bnhvCALwfmoGnvj4EK5UXBO7NCIyMwYsMivjJs8MWERtUivleOe3Q/He74fBSSnD4ZwyPPTeHqSeKxS7NCIyIwYsMiu2aCDqmEeH+WPznNEI93fF1doGPLPuKBb9cBb1jdxmh8gWMGCRWWWxyShRhwVpnPB/s+7F0yODAQBr9uXgtx8dQG7zXp5EZL0sImB9+OGHCAoKgoODAyIjI3H48OGbji8vL0dcXBy0Wi1UKhXCwsKQkpJi/HlQUBAkEkmrR1xcnHFMXV0d4uLi4OnpCWdnZ0ycOBGFhbxE3xWVdQ0oqmq6/ZxThEQdo5LLsOCRO/DxtOFwVytw+lIFfv3BXnx/4pLYpRFRF4gesL766iskJiZi4cKFOH78OIYOHYrY2FgUFRW1OV6n02HcuHHIzc3F+vXrkZaWhtWrV8Pf39845siRI7hy5YrxsX37dgDApEmTjGMSEhLwww8/4JtvvsGuXbtw+fJlPP744937YW1cy/Sgj6sKrg4Kkashsi4xd/hgS/xojAjqher6RsR/eQIvrT+JWl2j2KURUSdIBJF3Io2MjMTdd9+NFStWAAAMBgMCAgLwwgsvYP78+a3Gr1y5Eu+++y7Onz8PhaJjv8Tnzp2LTZs2ISMjAxKJBBUVFfDy8sLnn3+O3/72twCA8+fPY+DAgThw4ADuueeeW75mZWUl3NzcUFFRAVdX19v4xLbr66P5eGn9KYzs64nP/njrP0Miaq1Rb8D7OzPxwc4MCAIQ5KlG8oTBuLevRuzSiAgd//0v6hUsnU6HY8eOISYmxnhMKpUiJiYGBw4caPM5GzduRFRUFOLi4uDj44Pw8HAkJydDr297YahOp8Onn36Kp59+GhKJBABw7NgxNDQ0mLzvgAED0KdPn3bft76+HpWVlSYPMpXFBe5EXSaXSZE4Lgyf/TESvq4OyC2txRMfH8JL60+ivFYndnlE1EGiBqySkhLo9Xr4+PiYHPfx8UFBQUGbz8nOzsb69euh1+uRkpKC1157DUuXLsXixYvbHP/dd9+hvLwcTz31lPFYQUEBlEol3N3dO/y+S5YsgZubm/EREBDQ8Q9qJ9gDi8h87g3VYHtiNP5wTyAA4OujFxGzbBd+OHkZIk88EFEHiL4G63YZDAZ4e3tj1apViIiIwOTJk/HKK69g5cqVbY7/17/+hfHjx8PPz69L75uUlISKigrjIz8/v0uvZ4sym+8gDOUVLCKzcHFQ4I3HwrH+uSj09XZGSbUOL3zxPzyz7igulbM5KZElEzVgaTQayGSyVnfvFRYWwtfXt83naLVahIWFQSa7vg3LwIEDUVBQAJ3O9PL5hQsXsGPHDvzxj380Oe7r6wudTofy8vIOv69KpYKrq6vJg66ra9Ajv6wWAK9gEZnb8KBe2DxnFObG9INCJsHO80V4YNkufLIvB3oDr2YRWSJRA5ZSqURERARSU1ONxwwGA1JTUxEVFdXmc0aOHInMzEwYDNf370pPT4dWq4VSqTQZu3btWnh7e+Phhx82OR4REQGFQmHyvmlpacjLy2v3fenmckpqYBAAFwc5vFxUYpdDZHNUchnmxoQhZc5oRAR6oEanx19/OIvfrtyPtIIqscsjol8QfYowMTERq1evxrp163Du3DnMmjULNTU1mDFjBgBg2rRpSEpKMo6fNWsWysrKEB8fj/T0dGzevBnJyckmPa6ApqC2du1aTJ8+HXK53ORnbm5ueOaZZ5CYmIgff/wRx44dw4wZMxAVFdWhOwiptRvXX7XcTEBE5tfPxwXf/CkKbzw6CM4qOf6XV45ff7AHy/6bhroGdoEnshTyWw/pXpMnT0ZxcTEWLFiAgoICDBs2DFu3bjUufM/Ly4NUej0HBgQEYNu2bUhISMCQIUPg7++P+Ph4zJs3z+R1d+zYgby8PDz99NNtvu/f//53SKVSTJw4EfX19YiNjcU//vGP7vugNs7YwZ3rr4i6nVQqwR+ighBzhw8WfP8ztp8txPs7M7Hp9BW89fgQjAjuJXaJRHZP9D5Y1op9sEw9//lxbDp1BUnjB+BPY0LFLofIbgiCgK1nCrBg488obt5J4YnIPpg/fgAb/hJ1A6vog0W2gy0aiMQhkUgwfrAWOxLGYMqIpvYxnx/KQ8zSXdh6pu22M0TU/RiwqMv0BgHZzZvTMmARicNNrcCSx4fgy2fvQbDGCUVV9Xju02P403+OorCyTuzyiOwOAxZ12cWrtdA1GqCUS9HbQy12OUR27Z4QT2yJH424+0Mhl0qw7edCxCzdhU8PXoCBLR2IegwDFnVZy/RgiMYJMinvICQSm4NChhdjB+CHF0ZhaIA7quob8ep3ZzB51QHjv69E1L0YsKjLWv6DHcrpQSKLMlDrig2z7sWCX98BtVKGI7lX8dB7e/B+agZ0jYZbvwARdRoDFnVZJjd5JrJYMqkET48Kxn8TonF/fy/o9AYs256OX3+wB8cuXBW7PCKbxYBFXdayByEXuBNZrt4eaqx56m689/th8HRSIr2wGr9duR8Lvz+D6vpGscsjsjkMWNQlgiCwRQORlZBIJHh0mD92JI7BbyN6QxCAdQcuYNyyXUg9V3jrFyCiDmPAoi4prq5HVV0jpBIgWOMkdjlE1AEeTkr8bdJQfPpMJPr0UuNKRR2eWXcUcZ8fNzYrJaKuYcCiLmm5ehXQSw0HhUzkaojodozqp8G2udH4U3QIZFIJNp+6gphlu/D1kXxwkw+irmHAoi7J4gJ3IqvmqJQh6aGB+D5uJAb5uaLiWgNe+r9TmPrxIeQ2NxAmotvHgEVdwvVXRLYh3N8N38eNxMsPDYCDQor9WaWIXb4b//gpEw16tnQgul0MWNQlLXcQsgcWkfWTy6R4NjoU/507BqP6alDfaMA7W9PwmxX7cDK/XOzyiKwKAxZ1Ca9gEdmePp5q/OeZEVg6aSjc1Qqcu1KJCf/Yhzc2nUWtji0diDqCAYs6rbKuAYWVTXcchXINFpFNkUgkmBjRGzsSx+DRYX4wCMC/9ubggb/vxq70YrHLI7J4DFjUaS0L3L1cVHBzVIhcDRF1B42zCu/9/k6snXE3/N0dcfHqNUxfcxhzv/wfSqvZ0oGoPQxY1GncIofIftzf3xv/TYjG0yODIZUA3524jJhlu7Dh+EW2dCBqAwMWdVpWcdMt3Fx/RWQfnFRyLHjkDmyYPRIDfF1wtbYBiV+fxLQ1h5FfVit2eUQWhQGLOo0L3Ins07AAd/zwwii8GNsfSrkUezJK8MDfd+PjPdloZEsHIgAMWNQFWdzkmchuKWRSxN3fF1vjR+OekF641qDH4s3nMOEf+/Hz5QqxyyMSHQMWdUp9ox4XSjlFSGTvQryc8cXMe/D2xMFwdZDj9KUK/GbFPizZcg7XdHqxyyMSDQMWdUpuSS0MAuCiksPbRSV2OUQkIolEgsl398GOP4/Bw4O10BsE/HNXNh58bzf2ZZaIXR6RKBiwqFNa1l+FejtDIpGIXA0RWQJvFwd8OPUurJ42HL6uDrhQWoupHx/Ci9+cRHmtTuzyiHoUAxZ1Che4E1F7xt3hg+2J0ZgWFQiJBPjm2EXELNuFjScvs6UD2Q0GLOoU4x6E7IFFRG1wcVBg0aPhWP9cFPp5O6OkWoc5X/wPH+/JEbs0oh7BgEWdwitYRNQREYG9sGnOKDw3JhQAsHR7GvJK2TOLbB8DFt02vUFANls0EFEHqeQyzHuwP+4N9URdgwGvfn+GU4Vk8xiw6LZdLr+G+kYDlDIpAjwcxS6HiKyARCLB4sfCoZRLsTu9GBtPXha7JKJuxYBFt61lejBY4wS5jH+FiKhjQryc8fz9fQEAb2w6i4raBpErIuo+/O1It43rr4ios/40JgR9mxe9v7X1nNjlEHUbBiy6bTf2wCIiuh0quQzJEwYDAL44nI8juWUiV0TUPRiw6LZlcoE7EXXBiOBe+P3dAQCAlzechq6RG0ST7WHAotsiCML1KUL2wCKiTpo/fgA0zkpkFFVj1e4sscshMjsGLLotJdU6VFxrgEQChHg5iV0OEVkpd7USr/36DgDA+zszkVtSI3JFRObFgEW3peXqVYCHGg4KmcjVEJE1+81QP4zup4Gu0YBXvjvN3lhkUxiw6LZc3yKHV6+IqGtaemOp5FLsyyzFdycuiV0SkdkwYNFtyWKLBiIyo0BPJ8wZ2w8A8Mamc7haoxO5IiLzYMCi25LFOwiJyMxmjg5BmI8zymp0WLKFvbHINjBg0W1hk1EiMjelXIoljzf1xvr66EUczC4VuSKirmPAog6rrm/ElYo6AEBfLxeRqyEiWxIR2AtPRPYBALz87WnUN+pFroioaxiwqMNa1l9pnFVwUytEroaIbM28BwdA46xCdnENPvqJvbHIujFgUYddnx7kHYREZH5ujgosfKSpN9Y/fswyrvkkskYMWNRh3CKHiLrbr4docV9/L+j0BrzyLXtjkfViwKIO4xY5RNTdJBIJ3ng0HA4KKQ5ml2H9sYtil0TUKQxY1GHXe2BxgTsRdZ+AXmokxIQBAN5MOYfS6nqRKyK6fQxY1CG6RgMulNUCAEK5BouIutnTo4IxwNcF5bUNeDOFvbHI+jBgUYfkltZAbxDgrJLD19VB7HKIyMYpZE29sSQSYMPxS9ifWSJ2SUS3hQGLOqRlejDUywkSiUTkaojIHtzZxwN/uCcQAPDKd2dQ18DeWGQ9GLCoQ1oWuIfyDkIi6kF/ie0PbxcVckpq8I8fM8Uuh6jDGLCoQ9iigYjE4OqgwOu/GQQA+GhXFjIKq0SuiKhjGLCoQ9iigYjE8mC4L8YO8EaDXsDL356GwcDeWGT5GLDolgwGwdhRmVewiKinSSQSLHosHGqlDEdyr+Lro/lil0R0SwxYdEuXyq+hrsEApUyKPr3UYpdDRHbI390RieOaemMlp5xDcRV7Y5FlEz1gffjhhwgKCoKDgwMiIyNx+PDhm44vLy9HXFwctFotVCoVwsLCkJKSYjLm0qVLePLJJ+Hp6QlHR0cMHjwYR48eNf78qaeegkQiMXk8+OCD3fL5bEHL+qsgjRpymeh/ZYjITj11bxAG+bmisq4RizefFbscopsS9bflV199hcTERCxcuBDHjx/H0KFDERsbi6KiojbH63Q6jBs3Drm5uVi/fj3S0tKwevVq+Pv7G8dcvXoVI0eOhEKhwJYtW3D27FksXboUHh4eJq/14IMP4sqVK8bHF1980a2f1Zpd7+DO6UEiEo+8uTeWVAJ8f+IydqcXi10SUbvkYr75smXLMHPmTMyYMQMAsHLlSmzevBlr1qzB/PnzW41fs2YNysrKsH//figUCgBAUFCQyZi3334bAQEBWLt2rfFYcHBwq9dSqVTw9fU146exXcYWDVzgTkQiG9LbHdPvDcLafbl49bsz2DY3Go5KmdhlEbUi2hUsnU6HY8eOISYm5noxUiliYmJw4MCBNp+zceNGREVFIS4uDj4+PggPD0dycjL0er3JmOHDh2PSpEnw9vbGnXfeidWrV7d6rZ9++gne3t7o378/Zs2ahdLS0pvWW19fj8rKSpOHvcjkFSwisiB/fqA/tG4OyCurxQc7M8Quh6hNogWskpIS6PV6+Pj4mBz38fFBQUFBm8/Jzs7G+vXrodfrkZKSgtdeew1Lly7F4sWLTcZ89NFH6NevH7Zt24ZZs2Zhzpw5WLdunXHMgw8+iH//+99ITU3F22+/jV27dmH8+PEmQe2XlixZAjc3N+MjICCgi38C1qPlDkJewSIiS+Cskht7Y63anY3zBfbzP7xkPSSCIIjSUOTy5cvw9/fH/v37ERUVZTz+0ksvYdeuXTh06FCr54SFhaGurg45OTmQyZouCS9btgzvvvsurly5AgBQKpUYPnw49u/fb3zenDlzcOTIkXavjGVnZyM0NBQ7duzA2LFj2xxTX1+P+vrrd61UVlYiICAAFRUVcHV1vf0/ACtRWl2PiMU7IJEAZ19/kJfiichiPPvvo/jv2ULc1ccd65+7F1Ipt/Gi7ldZWQk3N7db/v4X7QqWRqOBTCZDYWGhyfHCwsJ210ZptVqEhYUZwxUADBw4EAUFBdDpdMYxd9xxh8nzBg4ciLy8vHZrCQkJgUajQWZm+9swqFQquLq6mjzsQcv0oL+7I8MVEVmU1x8dBCelDMfzyvHFkfb/G08kBtECllKpREREBFJTU43HDAYDUlNTTa5o3WjkyJHIzMyEwWAwHktPT4dWq4VSqTSOSUtLM3leeno6AgMD263l4sWLKC0thVar7cpHskncIoeILJXWzRF/ie0PAHhry3kUVdaJXBHRdaK2aUhMTMTq1auxbt06nDt3DrNmzUJNTY3xrsJp06YhKSnJOH7WrFkoKytDfHw80tPTsXnzZiQnJyMuLs44JiEhAQcPHkRycjIyMzPx+eefY9WqVcYx1dXVePHFF3Hw4EHk5uYiNTUVjz76KPr27YvY2Nie/QOwAtwih4gs2bSoIAzp7YaqukYs2sTeWGQ5RG3TMHnyZBQXF2PBggUoKCjAsGHDsHXrVuPC97y8PEil1zNgQEAAtm3bhoSEBAwZMgT+/v6Ij4/HvHnzjGPuvvtufPvtt0hKSsKiRYsQHByM5cuXY+rUqQAAmUyGU6dOYd26dSgvL4efnx8eeOABvPHGG1CpVD37B2AFeAchEVkymVSC5AmD8ZsVe7Hp1BVMjCjC/f29xS6LSLxF7tauo4vcrN29S1JxuaIO65+LwvCgXmKXQ0TUpsWbzuLjvTnwd3fE9sRoqJWiXj8gG2bxi9zJ8tXUN+JyRdOaBrZoICJLljAuDP7ujrhUfg3v7WBvLBIfAxa1q6X/laeTEh5OSpGrISJqn5NKjkWPNvXG+nhvDs5eZm8sEhcDFrXLuEUO118RkRUYO9AHDw32hd4gIOnb09AbuAKGxMOARe3KYosGIrIyCx8ZBBeVHCfzy/HZoQtil0N2jAGL2sUWDURkbXxcHfDSg029sd7ZmoaCCvbGInEwYFG72KKBiKzRE5GBGBbgjur6Rrz+w89il0N2igGL2tSgN+BCaS0ABiwisi4yqQRLHh8MmVSCLWcKsONs4a2fRGRmDFjUpgulNWg0CHBSyqB1cxC7HCKi2zJQ64o/jg4GACz4/gxq6htFrojsDQMWtenGOwglEu5QT0TWJ35sP/T2cMTlijr8fXu62OWQnWHAojZxgTsRWTu1Uo43HgsHAKzZl4MzlypErojsCQMWtYk9sIjIFtzf3xu/HqKFQQCSNrA3FvUcBixqU2ZzDyxukUNE1m7BI3fAxUGO05cqsG5/rtjlkJ1gwKJWDAYBWUU1AHgHIRFZP28XB8wfPwAAsPS/abhcfk3kisgeMGBRK1cq63CtQQ+5VIJAT7XY5RARddmUu/sgItADNTo9Fm5kbyzqfgxY1ErL+qsgjRMUMv4VISLrJ5VKkDxhMORSCbafLcS2nwvELolsHH97Uiu8g5CIbFF/Xxf8aUwIAGDh9z+jqq5B5IrIljFgUSvcIoeIbNULv+qHQE81CirrsPS/7I1F3YcBi1rJYsAiIhvloJBhcXNvrHUHcnEyv1zcgshmMWBRKy0tGhiwiMgWje7nhceG+UFo7o3VqDeIXRLZIAYsMlFWo0NZjQ4AEOLlJHI1RETd49Vf3wE3RwXOXqnE2n25YpdDNogBi0y0rL/yd3eEWikXuRoiou6hcVbh5YeaemMt256Oi1drRa6IbA0DFpngAncisheTIgIwIqgXrjXoseD7nyEI3EaHzIcBi0xkcYscIrITUqkEyY+HQyGTYOf5Imw5w95YZD4MWGSCV7CIyJ709XbBrDGhAIC/bvwZleyNRWbCgEUmGLCIyN7Mvr8vgjVOKKqqx7tb08Quh2wEAxYZ1eoacal5E1QGLCKyFw4KGd5s7o316aELOJ53VeSKyBYwYJFRdnENAKCXkxK9nJQiV0NE1HPu7avBxLt6QxCAlzecRgN7Y1EXMWCREfcgJCJ79srDA+GhVuB8QRX+tTdH7HLIynUqYP3444/mroMsQEvACuX0IBHZoV5OSrzy8B0AgOU70pFfxt5Y1HmdClgPPvggQkNDsXjxYuTn55u7JhIJF7gTkb2beJc/okI8UddgwCvfnWFvLOq0TgWsS5cu4fnnn8f69esREhKC2NhYfP3119DpdOauj3oQ9yAkInsnkUjw5oRwKGVS7E4vxg+nrohdElmpTgUsjUaDhIQEnDhxAocOHUJYWBhmz54NPz8/zJkzBydPnjR3ndTNGvQG5JY0LXJnwCIiexbi5Yy4+/sCABb9cBYVteyNRbevy4vc77rrLiQlJeH5559HdXU11qxZg4iICIwePRo///yzOWqkHpBXVotGgwBHhQxaVwexyyEiEtVz94Ug1MsJJdX1eGvrebHLISvU6YDV0NCA9evX46GHHkJgYCC2bduGFStWoLCwEJmZmQgMDMSkSZPMWSt1o+sL3J0glUpEroaISFwquQzJEwYDAL44nIejuWUiV0TWplMB64UXXoBWq8Wf/vQnhIWF4X//+x8OHDiAP/7xj3ByckJQUBD+9re/4fx5pn5rwRYNRESmIkM88bvhvQEASRtOQ9fI3ljUcZ0KWGfPnsUHH3yAy5cvY/ny5QgPD281RqPRsJ2DFcniHYRERK28/NBAeDopkVFUjdV7ssUuh6xIpwJWamoqpkyZApVK1e4YuVyOMWPGdLow6lm8g5CIqDV3tRKv/nogAOD91AzjzUBEt9KpgLVkyRKsWbOm1fE1a9bg7bff7nJR1LMEQeAVLCKidjw2zB+j+mpQ32jAq+yNRR3UqYD1z3/+EwMGDGh1fNCgQVi5cmWXi6KedaWiDjU6PeRSCQI9ncQuh4jIokgkEix+LBwquRR7M0vw/YnLYpdEVqBTAaugoABarbbVcS8vL1y5wqZs1qZlgXugpxoKGbenJCL6pSCNE+aM7QcAeGPTWZTXsrE23VynfpsGBARg3759rY7v27cPfn5+XS6Keha3yCEiurWZo0MQ5uOM0hodlqTwLnm6uU4FrJkzZ2Lu3LlYu3YtLly4gAsXLmDNmjVISEjAzJkzzV0jdTMucCciujWlXGrsjfXV0Xwcyi4VuSKyZPLOPOnFF19EaWkpZs+ebdx/0MHBAfPmzUNSUpJZC6Tu17LAPZQ9sIiIbmp4UC9MGdEHXxzOw8vfnsaW+Ggo5VxaQa116m+FRCLB22+/jeLiYhw8eBAnT55EWVkZFixYYO76qAdk8QoWEVGHzX9wADydlMgqrsGu9GKxyyEL1aXY7ezsjLvvvhvh4eE37YlFlqu8VoeS6qarkLyCRUR0a25qBcYP9gUA7MlgwKK2dWqKEACOHj2Kr7/+Gnl5ecZpwhYbNmzocmHUM1oWuPu5OcBJ1em/DkREdiW6nxc+PZiH3byCRe3o1BWsL7/8Evfeey/OnTuHb7/9Fg0NDfj555+xc+dOuLm5mbtG6kbXN3nm1Ssioo6KCvWEXCpBbmkt8kprxS6HLFCnAlZycjL+/ve/44cffoBSqcR7772H8+fP43e/+x369Olj7hqpG7FFAxHR7XNxUOCuPh4AgN2cJqQ2dCpgZWVl4eGHHwYAKJVK1NTUQCKRICEhAatWrTJrgdS92KKBiKhzRvfTAOA6LGpbpwKWh4cHqqqqAAD+/v44c+YMAKC8vBy1tbxUak2MV7C4wJ2I6LZEh3kBAPZnlqJBbxC5GrI0nQpY0dHR2L59OwBg0qRJiI+Px8yZMzFlyhSMHTvWrAVS97mm0+NS+TUAvIJFRHS7wv3d4K5WoKq+ESfzy8UuhyxMp24bW7FiBerq6gAAr7zyChQKBfbv34+JEyfi1VdfNWuB1H2yiqshCICHWgFPZ7bZICK6HTKpBKP6arDp1BXszijB8KBeYpdEFuS2r2A1NjZi06ZNkMlkTS8glWL+/PnYuHEjli5dCg8Pj9t6vQ8//BBBQUFwcHBAZGQkDh8+fNPx5eXliIuLg1arhUqlQlhYGFJSUkzGXLp0CU8++SQ8PT3h6OiIwYMH4+jRo8afC4KABQsWQKvVwtHRETExMcjIyLitum1BS4NR9r8iIuqc6H5N04Rs10C/dNsBSy6X47nnnjNeweqKr776ComJiVi4cCGOHz+OoUOHIjY2FkVFRW2O1+l0GDduHHJzc7F+/XqkpaVh9erV8Pf3N465evUqRo4cCYVCgS1btuDs2bOtgt8777yD999/HytXrsShQ4fg5OSE2NhYs3wma5LFOwiJiLpkdFjTQvdTF8tRXqu7xWiyJ52aIhwxYgROnDiBwMDALr35smXLMHPmTMyYMQMAsHLlSmzevBlr1qzB/PnzW41fs2YNysrKsH//figUCgBAUFCQyZi3334bAQEBWLt2rfFYcHCw8Z8FQcDy5cvx6quv4tFHHwUA/Pvf/4aPjw++++47/P73v+/SZ7ImvIOQiKhrtG6O6OftjIyiauzLLMXDQ7Ril0QWolOL3GfPno3ExESsWLECBw4cwKlTp0weHaHT6XDs2DHExMRcL0YqRUxMDA4cONDmczZu3IioqCjExcXBx8cH4eHhSE5Ohl6vNxkzfPhwTJo0Cd7e3rjzzjuxevVq489zcnJQUFBg8r5ubm6IjIxs930BoL6+HpWVlSYPa8cmo0REXddyNyHbNdCNOnUFq+Uqz5w5c4zHJBIJBEGARCIxCTztKSkpgV6vh4+Pj8lxHx8fnD9/vs3nZGdnY+fOnZg6dSpSUlKQmZmJ2bNno6GhAQsXLjSO+eijj5CYmIiXX34ZR44cwZw5c6BUKjF9+nQUFBQY3+eX79vys7YsWbIEr7/++i0/l7Vo1BuQU1IDgC0aiIi6YnQ/Df61Nwe704uNvweJOhWwcnJyzF1HhxgMBnh7e2PVqlWQyWSIiIjApUuX8O677xoDlsFgwPDhw5GcnAwAuPPOO3HmzBmsXLkS06dP7/R7JyUlITEx0fh9ZWUlAgICuvaBRJRXVosGvQBHhQz+7o5il0NEZLUigz2hlEtxuaIOWcU1XHZBADoZsLq69goANBoNZDIZCgsLTY4XFhbC19e3zedotVooFArjHYwAMHDgQBQUFECn00GpVEKr1eKOO+4wed7AgQPxf//3fwBgfO3CwkJotdfnygsLCzFs2LB261WpVFCpbKeVQcv0YIiXE6RS/t8WEVFnOSplGBHUC3szS7A7vZgBiwB0MmD9+9//vunPp02bdsvXUCqViIiIQGpqKh577DEATVefUlNT8fzzz7f5nJEjR+Lzzz+HwWCAVNq0fCw9PR1arRZKpdI4Ji0tzeR56enpxlAYHBwMX19fpKamGgNVZWUlDh06hFmzZt2yblvBBe5EROYTHabB3swS7MkoxtOjgm/9BLJ5nQpY8fHxJt83NDSgtrYWSqUSarW6QwELABITEzF9+nQMHz4cI0aMwPLly1FTU2O8q3DatGnw9/fHkiVLAACzZs3CihUrEB8fjxdeeAEZGRlITk42WQuWkJCAe++9F8nJyfjd736Hw4cPY9WqVcY9EiUSCebOnYvFixejX79+CA4OxmuvvQY/Pz9j0LMH3CKHiMh8RvfzAnAeB7PLUN+oh0ouu+VzyLZ1KmBdvXq11bGMjAzMmjULL774YodfZ/LkySguLsaCBQtQUFCAYcOGYevWrcYF6Hl5ecYrVQAQEBCAbdu2ISEhAUOGDIG/vz/i4+Mxb94845i7774b3377LZKSkrBo0SIEBwdj+fLlmDp1qnHMSy+9hJqaGjz77LMoLy/HqFGjsHXrVjg4OHTmj8MqsQcWEZH5DPB1gZeLCsVV9TiWexX39tWIXRKJTCIIgmCuFzt69CiefPLJdu8CtCWVlZVwc3NDRUUFXF1dxS7ntgiCgMF//S+q6xuxPSEa/XxcxC6JiMjqJX59AhuOX8KfxoQgafxAscuhbtLR3/+d6oPVHrlcjsuXL5vzJakbFFbWo7q+ETKpBIGeTmKXQ0RkE1q2zdmTXiJyJWQJOjVFuHHjRpPvBUHAlStXsGLFCowcOdIshVH3aVl/FdhLDaXcrBmbiMhujerXNC149koliqvq4eViO3ee0+3rVMD65WJwiUQCLy8v/OpXv8LSpUvNURd1o8yiKgDs4E5EZE4aZxUG+bni58uV2JtZjAl39ha7JBJRpwKWwWAwdx3Ug9iigYioe0SHeeHny5XYnV7CgGXnOD9kh9iigYioe4xunibck1ECg8Fs95CRFepUwJo4cSLefvvtVsffeecdTJo0qctFUffKLGreg5BXsIiIzCoi0ANqpQwl1fU4V1Apdjkkok4FrN27d+Ohhx5qdXz8+PHYvXt3l4ui7lNR24CS6noAXINFRGRuKrkM94R4Ami6ikX2q1MBq7q62rg1zY0UCgUqK5nYLVlmcdMCd62bA5xVnVqCR0RENxFtnCYsFrkSElOnAtbgwYPx1VdftTr+5ZdfttpomSxLJju4ExF1q9FhTf2wjuRcRa2uUeRqSCyduoTx2muv4fHHH0dWVhZ+9atfAQBSU1PxxRdf4JtvvjFrgWReWcVN669CucCdiKhbhGic4O/uiEvl13Aopwz39/cWuyQSQaeuYD3yyCP47rvvkJmZidmzZ+PPf/4zLl68iB07dtjVhsnWqOUKFtdfERF1D4lEguiwpmnC3emcJrRXnV6E8/DDD+Phhx82Zy3UA9iigYio+0X388IXh/O50N2OdeoK1pEjR3Do0KFWxw8dOoSjR492uSjqHnUNeuRfrQXANVhERN3p3lANpJKm/6m9XH5N7HJIBJ0KWHFxccjPz291/NKlS4iLi+tyUdQ9sotrIAiAm6MCGufWd4ESEZF5uKkVGBbgDoB3E9qrTgWss2fP4q677mp1/M4778TZs2e7XBR1jxu3yJFIJCJXQ0Rk20b3a7qbcHc6pwntUacClkqlQmFhYavjV65cgVzO3kqWiuuviIh6TnRzu4a9mSXQc9scu9OpgPXAAw8gKSkJFRUVxmPl5eV4+eWXMW7cOLMVR+aVxR5YREQ9ZmhvN7g4yFFxrQGnLpaLXQ71sE4FrL/97W/Iz89HYGAg7r//ftx///0IDg5GQUEBli5dau4ayUzYZJSIqOfIZVKM6nt982eyL50KWP7+/jh16hTeeecd3HHHHYiIiMB7772H06dPIyAgwNw1khk06g3IKeEmz0REPen6OiwudLc3nV4w5eTkhFGjRqFPnz7Q6XQAgC1btgAAfvOb35inOjKbi1evQac3QCWXwt/dUexyiIjswujmfQn/l1+OyroGuDooRK6IekqnAlZ2djYmTJiA06dPQyKRQBAEk7vS9Hq92Qok82iZHgzxcoZUyjsIiYh6QkAvNUI0TsguqcH+zFI8GO4rdknUQzo1RRgfH4/g4GAUFRVBrVbjzJkz2LVrF4YPH46ffvrJzCWSOdzYooGIiHpOy1Us9sOyL50KWAcOHMCiRYug0WgglUohk8kwatQoLFmyBHPmzDF3jWQGbNFARCSOlnYNuzOKIQhs12AvOhWw9Ho9XFxcAAAajQaXL18GAAQGBiItLc181ZHZ8A5CIiJx3BPiCYVMgvyya7hQWit2OdRDOhWwwsPDcfLkSQBAZGQk3nnnHezbtw+LFi1CSEiIWQukrhMEgT2wiIhE4qSSIyLQA0DTVSyyD50KWK+++ioMBgMAYNGiRcjJycHo0aORkpKC999/36wFUtcVVdWjqr4RUgkQpFGLXQ4Rkd3htjn2p1N3EcbGxhr/uW/fvjh//jzKysrg4eHBPe4sUMv0YKCnE1RymcjVEBHZnzFhXnh3WxoOZJWgQW+AQtap6xtkRcx2hnv16sVwZaFaAlYoF7gTEYniDq0rPJ2UqNHpcfzCVbHLoR7ACG0HstiigYhIVFKpBKP6cdsce8KAZQd4ByERkfiM67C40N0uMGDZgetThE4iV0JEZL+im69gnb5UgbIancjVUHdjwLJxlXUNKKqqBwCE8goWEZFovF0dMMDXBYIA7M3kNKGtY8CycS1Xr3xcVdxklIhIZC1d3fekc5rQ1jFg2TiuvyIishwt+xJy2xzbx4Bl47K4ByERkcW4O6gXVHIpCivrkdH832eyTQxYNo5XsIiILIeDQobIEE8AwG5OE9o0Biwbl9ncA4sL3ImILEO0cZqQC91tGQOWDatr0CO/rGnndl7BIiKyDC0L3Q9ll6KuQS9yNdRdGLBsWG5pDQwC4Oogh5ezSuxyiIgIQD9vZ/i6OqC+0YAjuWVil0PdhAHLht24/or7RBIRWQaJRHL9bkKuw7JZDFg2jAvciYgs0+iWflhch2WzGLBs2PUtchiwiIgsyai+GkgkwPmCKhRW1oldDnUDBiwbxitYRESWqZeTEoP93QDwKpatYsCyUXqDgOySGgAMWERElii6X9M0Iddh2SYGLBt18WotdI0GKOVS9PZQi10OERH9QstC972ZJTAYuG2OrWHAslEt04MhGifIpLyDkIjI0twV6AEnpQxlNTqcvVIpdjlkZgxYNorrr4iILJtCJkVUaNNVrF2cJrQ5DFg2igGLiMjyjQlrClh7MhiwbA0Dlo3KKmbAIiKydKObF7ofu3AVNfWNIldD5sSAZYMEQeAVLCIiKxCkcUKfXmo06AUczC4VuxwyIwYsG1RcXY/KukZIJUCQp5PY5RAR0U1w2xzbxIBlg1quXgX0UsNBIRO5GiIiuplobptjkxiwbFBWy/Qgt8ghIrJ4UaGekEklyC6pQX5ZrdjlkJlYRMD68MMPERQUBAcHB0RGRuLw4cM3HV9eXo64uDhotVqoVCqEhYUhJSXF+PO//vWvkEgkJo8BAwaYvMZ9993Xasxzzz3XLZ+vp3H9FRGR9XB1UOCuPu4AeBXLlsjFLuCrr75CYmIiVq5cicjISCxfvhyxsbFIS0uDt7d3q/E6nQ7jxo2Dt7c31q9fD39/f1y4cAHu7u4m4wYNGoQdO3YYv5fLW3/UmTNnYtGiRcbv1Wrb6Hie2XwHYSgDFhGRVRjdzwtHcq9id3oxnojsI3Y5ZAaiB6xly5Zh5syZmDFjBgBg5cqV2Lx5M9asWYP58+e3Gr9mzRqUlZVh//79UCgUAICgoKBW4+RyOXx9fW/63mq1+pZjrBGvYBERWZfoMC8s256OfVklaNQbIJdZxAQTdYGoZ1Cn0+HYsWOIiYkxHpNKpYiJicGBAwfafM7GjRsRFRWFuLg4+Pj4IDw8HMnJydDr9SbjMjIy4Ofnh5CQEEydOhV5eXmtXuuzzz6DRqNBeHg4kpKSUFvb/tx3fX09KisrTR6WqLKuAYWV9QAYsIiIrMVgfze4qxWoqmvEyYvlYpdDZiBqwCopKYFer4ePj4/JcR8fHxQUFLT5nOzsbKxfvx56vR4pKSl47bXXsHTpUixevNg4JjIyEp988gm2bt2Kjz76CDk5ORg9ejSqqqqMY5544gl8+umn+PHHH5GUlIT//Oc/ePLJJ9utdcmSJXBzczM+AgICuvjpu0fLAndvFxVcHRQiV0NERB0hk0owsm9Luwauw7IFok8R3i6DwQBvb2+sWrUKMpkMERERuHTpEt59910sXLgQADB+/Hjj+CFDhiAyMhKBgYH4+uuv8cwzzwAAnn32WeOYwYMHQ6vVYuzYscjKykJoaGir901KSkJiYqLx+8rKSosMWVnFNQB49YqIyNpE99Ng86kr2J1RjIRxYWKXQ10kasDSaDSQyWQoLCw0OV5YWNju2iitVguFQgGZ7Hp/p4EDB6KgoAA6nQ5KpbLVc9zd3REWFobMzMx2a4mMjAQAZGZmthmwVCoVVCpVhz6XmLj+iojIOrVsm3MyvxwVtQ1wU3MWwpqJOkWoVCoRERGB1NRU4zGDwYDU1FRERUW1+ZyRI0ciMzMTBoPBeCw9PR1arbbNcAUA1dXVyMrKglarbbeWEydOAMBNx1gDBiwiIuvk5+6Ivt7OMAjAvixOE1o70W9TSExMxOrVq7Fu3TqcO3cOs2bNQk1NjfGuwmnTpiEpKck4ftasWSgrK0N8fDzS09OxefNmJCcnIy4uzjjmL3/5C3bt2oXc3Fzs378fEyZMgEwmw5QpUwAAWVlZeOONN3Ds2DHk5uZi48aNmDZtGqKjozFkyJCe/QMws5ZNnkPZZJSIyOq0bJuzJ4Pb5lg70ddgTZ48GcXFxViwYAEKCgowbNgwbN261bjwPS8vD1Lp9RwYEBCAbdu2ISEhAUOGDIG/vz/i4+Mxb94845iLFy9iypQpKC0thZeXF0aNGoWDBw/Cy6vp8qtSqcSOHTuwfPly1NTUICAgABMnTsSrr77asx/ezOob9bhQyjVYRETWKjrMC2v35WJ3egkEQYBEIhG7JOokiSAIgthFWKPKykq4ubmhoqICrq6uYpcDAEgrqELs8t1wUclx6q8P8F9MIiIrU6trxLDXt0OnNyD1z2M4G2GBOvr7X/QpQjKflvVXod7ODFdERFZIrZTj7mAPAMCedE4TWjMGLBvCBe5ERNav5W7C3dyX0KoxYNmQlj0IGbCIiKxXdHPAOpBVivpG/S1Gk6ViwLIhxitYnLMnIrJaA3xdoHFW4VqDHscuXBW7HOokBiwbYTAIyOYVLCIiqyeVShBtbNfAaUJrxYBlIy6VX0N9owFKuRQBvdRil0NERF0wOqxlX0IudLdWDFg2omV6METjBJmUdxASEVmzUX2b1mH9fLkSJdX1IldDncGAZSOMLRq4/oqIyOp5uahwh7apx9JeThNaJQYsG3FjDywiIrJ+0WEt7Ro4TWiNGLBsBFs0EBHZlhsXunPTFevDgGUDBEFgiwYiIhsTEeQBR4UMxVX1OF9QJXY5dJsYsGxASbUOFdcaIJEAIV5OYpdDRERmoJLLcE9ILwC8m9AaMWDZgJarVwEeajgoZCJXQ0RE5tKyDov9sKwPA5YN4PorIiLb1LIv4eHcMlzTcdsca8KAZQOyuMkzEZFNCvVygp+bA3SNBhzKKRW7HLoNDFg2IKuYC9yJiGyRRCK53q4hndOE1oQBywawBxYRke1qmSbcw35YVoUBy8pV1zfiSkUdAF7BIiKyRSP7ekIqATKKqnG5/JrY5VAHMWBZuZb1VxpnFdzUCpGrISIic3NXKzGktzsAbptjTRiwrJyxwag3+18REdkqbptjfRiwrBxbNBAR2b6WbXP2ZpZAb+C2OdaAAcvKcYscIiLbNyzAHS4qOcprG3DmUoXY5VAHMGBZues9sFxEroSIiLqLXCbFvX09AXDbHGvBgGXFdI0GXCirBcApQiIiW8dtc6wLA5YVu1BaA71BgLNKDh9XldjlEBFRN4pu7od1PO8qquoaRK6GboUBy4rd2GBUIpGIXA0REXWngF5qBGuc0GgQcCCL2+ZYOgYsK8YF7kRE9mV0892EbNdg+RiwrBhbNBAR2ZfoflyHZS0YsKyYcYrQi01GiYjswT2hnpBLJbhQWosLpTVil0M3wYBlpQwGAVm8gkVEZFecVXJEBHoAAHbzKpZFY8CyUpfKr6GuwQClTIo+vdRil0NERD3EuG0O+2FZNAYsK9Wy/ipIo4ZcxtNIRGQvWtZhHcgqRYPeIHI11B7+ZrZS1zu4c3qQiMieDPJzRS8nJarrG/G/vHKxy6F2MGBZKeP6K7ZoICKyK1KpBKP6NrVr2MN2DRaLActK3dhklIiI7IuxHxbXYVksBiwrlckpQiIiuzW6eR3WqUsVuFqjE7kaagsDlhUqra7H1doGSCRAKKcIiYjsjq+bA/r7uEAQgH1ZbNdgiRiwrFDL1aveHo5wUMhEroaIiMTAaULLxoBlhVpaNPDqFRGR/Wrph7UnowSCIIhcDf0SA5YV4ibPREQ0IrgXVHIprlTUGX8vkOVgwLJCXOBOREQOChlGBPcCwG1zLBEDlhVik1EiIgKud3XnOizLw4BlZWrqG3G5og4AAxYRkb1rWYd1KKcUdQ16kauhGzFgWZns4hoAgMZZCXe1UuRqiIhITGE+zvBxVaGuwYCjuVfFLoduwIBlZTKLqwDwDkIiIgIkEomx6Si3zbEsDFhWhgvciYjoRi39sHZxHZZFYcCyMgxYRER0o9H9vCCRAOcLqlBUWSd2OdSMAcvKMGAREdGNejkpEe7nBqCp6ShZBgYsK9KgN+BCaS0ArsEiIqLrosOapgm5DstyMGBZkQulNWg0CHBSyqB1cxC7HCIishDXF7qXwGDgtjmWgAHLirRMD4Z6O0MikYhcDRERWYq7+njASSlDaY0OZ69Uil0OgQHLqnAPQiIiaotSLkVUqCcAYDenCS2CRQSsDz/8EEFBQXBwcEBkZCQOHz580/Hl5eWIi4uDVquFSqVCWFgYUlJSjD//61//ColEYvIYMGCAyWvU1dUhLi4Onp6ecHZ2xsSJE1FYWNgtn89cbryCRUREdKOWru570rnQ3RKIHrC++uorJCYmYuHChTh+/DiGDh2K2NhYFBUVtTlep9Nh3LhxyM3Nxfr165GWlobVq1fD39/fZNygQYNw5coV42Pv3r0mP09ISMAPP/yAb775Brt27cLly5fx+OOPd9vnNIes5i7uvIOQiIh+qWUd1tELZaipbxS5GpKLXcCyZcswc+ZMzJgxAwCwcuVKbN68GWvWrMH8+fNbjV+zZg3Kysqwf/9+KBQKAEBQUFCrcXK5HL6+vm2+Z0VFBf71r3/h888/x69+9SsAwNq1azFw4EAcPHgQ99xzj5k+nfkYDAKyitmigYiI2hbkqUZvD0dcvHoNh3JK8asBPmKXZNdEvYKl0+lw7NgxxMTEGI9JpVLExMTgwIEDbT5n48aNiIqKQlxcHHx8fBAeHo7k5GTo9aabXGZkZMDPzw8hISGYOnUq8vLyjD87duwYGhoaTN53wIAB6NOnT7vvW19fj8rKSpNHT7pSWYdanR4KmQSBvdQ9+t5ERGT5JBKJcZpwN6cJRSdqwCopKYFer4ePj2nK9vHxQUFBQZvPyc7Oxvr166HX65GSkoLXXnsNS5cuxeLFi41jIiMj8cknn2Dr1q346KOPkJOTg9GjR6Oqqmkfv4KCAiiVSri7u3f4fZcsWQI3NzfjIyAgoAuf/Pa1rL8K8nSCXCb6zC4REVmg6OZtc7jQXXxW95vaYDDA29sbq1atQkREBCZPnoxXXnkFK1euNI4ZP348Jk2ahCFDhiA2NhYpKSkoLy/H119/3en3TUpKQkVFhfGRn59vjo/TYezgTkREtxIVqoFMKkF2cQ0uXq0Vuxy7JmrA0mg0kMlkre7eKywsbHf9lFarRVhYGGQymfHYwIEDUVBQAJ1O1+Zz3N3dERYWhszMTACAr68vdDodysvLO/y+KpUKrq6uJo+exIBFRES34uaowLAAdwDcNkdsogYspVKJiIgIpKamGo8ZDAakpqYiKiqqzeeMHDkSmZmZMBgMxmPp6enQarVQKpVtPqe6uhpZWVnQarUAgIiICCgUCpP3TUtLQ15eXrvvK7aslhYN7IFFREQ3EW3s6s5pQjGJPkWYmJiI1atXY926dTh37hxmzZqFmpoa412F06ZNQ1JSknH8rFmzUFZWhvj4eKSnp2Pz5s1ITk5GXFycccxf/vIX7Nq1C7m5udi/fz8mTJgAmUyGKVOmAADc3NzwzDPPIDExET/++COOHTuGGTNmICoqyiLvIASATN5BSEREHTC6eV/CvRklaNQbbjGauovobRomT56M4uJiLFiwAAUFBRg2bBi2bt1qXPiel5cHqfR6DgwICMC2bduQkJCAIUOGwN/fH/Hx8Zg3b55xzMWLFzFlyhSUlpbCy8sLo0aNwsGDB+Hl5WUc8/e//x1SqRQTJ05EfX09YmNj8Y9//KPnPvhtKKvRoaymafozxMtJ5GqIiMiSDe3tDlcHOSrrGnHqUgXu6uMhdkl2SSIIAneF7ITKykq4ubmhoqKi29djHc4pw+/+eQD+7o7YN/9X3fpeRERk/WZ/dgwppwswN6Yf5saEiV2OTeno73/Rpwjp1thglIiIbsf1dVhc6C4WBiwrwDsIiYjodoxubjh6Ir8cFdcaRK7GPjFgWQEGLCIiuh3+7o4I9XKC3iDgQBavYomBAcsKMGAREdHtatn8eRe3zREFA5aFq9U14lL5NQBAX/bAIiKiDhpj3JewGLyfrecxYFm47OIaAICnkxIeTm03UiUiIvqlyJBeUMqkuFR+DTklNWKXY3cYsCxcJju4ExFRJ6iVcgwPauqBxbsJex4DloUzBiyuvyIiotvUsg5rdzq3zelpDFgWjgvciYios6Kbt805kF0KXSO3zelJDFgWjnsQEhFRZw30dYXGWYlanR7HLlwVuxy7woBlwRr0BlwobVqYyIBFRES3SyqVGKcJ92RwmrAnMWBZsLyyWjToBaiVMvi5OYhdDhERWaHR/ZqmCbnQvWcxYFmwG+8glEgkIldDRETWaFRzwDpzuQKl1fUiV2M/GLAsGBe4ExFRV3m7OGCg1hWCAOzN5FWsnsKAZcGyGLCIiMgMopuvYu3mtjk9hgHLgrXcQcgmo0RE1BXRYdcXunPbnJ7BgGWhBEG44QqWk8jVEBGRNYsI9ICDQoqiqnqkFVaJXY5dYMCyUFcq6lCj00MulSDQkwGLiIg6z0Ehwz0hngCAPZwm7BEMWBaqZYF7oKcaChlPExERdY1x2xz2w+oR/M1toXgHIRERmdOY5m1zDuWUoa5BL3I1to8By0JlcYscIiIyo1AvZ2jdHKBrNOBQTpnY5dg8BiwLxStYRERkThKJBNEt2+akc5qwuzFgWSjjFSwvF5ErISIiWzG6eZqQ67C6HwOWBSqv1aGkWgcACGWLBiIiMpNRfTWQSID0wmoUVNSJXY5NY8CyQC3Tg/7ujlAr5SJXQ0REtsJdrcSQ3u4AeBWruzFgWSDjJs9cf0VERGY2pnnbnD0Z7IfVnRiwLJAxYHlxepCIiMxrdPO2OXsziqE3cNuc7sKAZYEy2aKBiIi6ybAAd7io5Lha24CfL1eIXY7NYsCyQMYWDdzkmYiIzEwhkyIqtGnbnN1s19BtGLAsjCAImHCnP2IH+aCfD1s0EBGR+UWHtWybw3VY3YW3qFkYiUSCPz/QX+wyiIjIhrU0HD1+4Sqq6xvhrGIcMDf+iRIREdmZPp5qBHqqcaG0FgeySjHuDh+xS+oSvUFAflkt0gqrkFFYhbTCatwZ4I6nRwWLVhMDFhERkR2K7ueF/5RewJ6MYqsJWIIgoKCyDmkFVUgvrEJaQTXSC6uQUVSFugaDydjqugYGLCIiIupZo/tp8J+DFyx2oXtZje56kCqsQnpB09equsY2x6vkUvTzcUaYjwv6+7hgaIB7zxb8CwxYREREdigq1BNyqQS5pbXIK61FH0+1KHVU1zci/YYA1XJlqqS6vs3xMqkEIRonhPk2BakwHxf093VBn15qyKSSHq6+fQxYREREdsjFQYG7+njgcG4ZdmcU40nPwG59v7oGPbKKq02m9tIKqnCp/Fq7z+nTS90coJyNQSpY4wSVXNattZoDAxYREZGdig7T4HBuGfZkFOPJe8wTsBr1BuSW1hoDVMsUX25JDdprHO/jqjJO7bVcmerr7QwnK7670XorJyIioi4Z3c8Lf/tvOvZnlqJBb4BC1vH2mAaDgEvl136xRqoaWUXV0OkNbT7HXa1oFaTCfJzhrlaa6yNZDAYsIiIiOxXu7wYPtQJXaxtwMr8cw4N6tRojCAKKq+uRXlBtstg8o7AKNTp9m6+rVsrQz8cF/X2uT+3193GBl4sKEonlrJPqTgxYREREdkomlWBkXw02nbqC3enF6OftgvSiG6b2mr9erW1o8/lKmRQhXk7o7+tivDLV39cF/u6OkFrQgnMxMGARERHZsegwL2w6dQUf/pSF93dmtjlGKgGCPJ0QdsPUXn9fZwR6Ot3WtKI9YcAiIiKyY/eFecFBITU26vR3d7x+Rar57r1QL2c4KCz/zj1LwoBFRERkx7xdHbAlPhplNTqE+TjDxUEhdkk2gQGLiIjIzgVrnBCscRK7DJvCiVMiIiIiM2PAIiIiIjIzBiwiIiIiM2PAIiIiIjIzBiwiIiIiM2PAIiIiIjIzBiwiIiIiM2PAIiIiIjIzBiwiIiIiM7OIgPXhhx8iKCgIDg4OiIyMxOHDh286vry8HHFxcdBqtVCpVAgLC0NKSkqbY9966y1IJBLMnTvX5Ph9990HiURi8njuuefM9ZGIiIjIjom+Vc5XX32FxMRErFy5EpGRkVi+fDliY2ORlpYGb2/vVuN1Oh3GjRsHb29vrF+/Hv7+/rhw4QLc3d1bjT1y5Aj++c9/YsiQIW2+98yZM7Fo0SLj92q12myfi4iIiOyX6Fewli1bhpkzZ2LGjBm44447sHLlSqjVaqxZs6bN8WvWrEFZWRm+++47jBw5EkFBQRgzZgyGDh1qMq66uhpTp07F6tWr4eHh0eZrqdVq+Pr6Gh+urq5m/3xERERkf0QNWDqdDseOHUNMTIzxmFQqRUxMDA4cONDmczZu3IioqCjExcXBx8cH4eHhSE5Ohl6vNxkXFxeHhx9+2OS1f+mzzz6DRqNBeHg4kpKSUFtb2+7Y+vp6VFZWmjyIiIiI2iLqFGFJSQn0ej18fHxMjvv4+OD8+fNtPic7Oxs7d+7E1KlTkZKSgszMTMyePRsNDQ1YuHAhAODLL7/E8ePHceTIkXbf+4knnkBgYCD8/Pxw6tQpzJs3D2lpadiwYUOb45csWYLXX3+9k5+UiIiI7Inoa7Bul8FggLe3N1atWgWZTIaIiAhcunQJ7777LhYuXIj8/HzEx8dj+/btcHBwaPd1nn32WeM/Dx48GFqtFmPHjkVWVhZCQ0NbjU9KSkJiYqLx+4qKCvTp04dXsoiIiOxIy+99QRBuOk7UgKXRaCCTyVBYWGhyvLCwEL6+vm0+R6vVQqFQQCaTGY8NHDgQBQUFxinHoqIi3HXXXcaf6/V67N69GytWrEB9fb3Jc1tERkYCADIzM9sMWCqVCiqVyvh9yx9wQEDAbXxiIiIisgVVVVVwc3Nr9+eiBiylUomIiAikpqbiscceA9B0hSo1NRXPP/98m88ZOXIkPv/8cxgMBkilTUvI0tPTodVqoVQqMXbsWJw+fdrkOTNmzMCAAQMwb968NsMVAJw4cQJAU4DrCD8/P+Tn58PFxQUSiaRDz+moyspKBAQEID8/nwvvLRDPj+XiubFcPDeWjeen4wRBQFVVFfz8/G46TvQpwsTEREyfPh3Dhw/HiBEjsHz5ctTU1GDGjBkAgGnTpsHf3x9LliwBAMyaNQsrVqxAfHw8XnjhBWRkZCA5ORlz5swBALi4uCA8PNzkPZycnODp6Wk8npWVhc8//xwPPfQQPD09cerUKSQkJCA6Orrdlg6/JJVK0bt3b3P9MbTJ1dWVf9EtGM+P5eK5sVw8N5aN56djbnblqoXoAWvy5MkoLi7GggULUFBQgGHDhmHr1q3Ghe95eXnGK1VA05Tctm3bkJCQgCFDhsDf3x/x8fGYN29eh99TqVRix44dxjAXEBCAiRMn4tVXXzX75yMiIiL7IxFutUqLelxlZSXc3NxQUVHB/5OwQDw/lovnxnLx3Fg2nh/zE73RKLWmUqmwcOFCk0X1ZDl4fiwXz43l4rmxbDw/5scrWERERERmxitYRERERGbGgEVERERkZgxYRERERGbGgEVERERkZgxYFujDDz9EUFAQHBwcEBkZicOHD4tdkt1ZsmQJ7r77bri4uMDb2xuPPfYY0tLSTMbU1dUhLi4Onp6ecHZ2xsSJE1tt+0Td76233oJEIsHcuXONx3huxHXp0iU8+eST8PT0hKOjIwYPHoyjR48afy4IAhYsWACtVgtHR0fExMQgIyNDxIrtg16vx2uvvYbg4GA4OjoiNDQUb7zxhsmeejw35sOAZWG++uorJCYmYuHChTh+/DiGDh2K2NhYFBUViV2aXdm1axfi4uJw8OBBbN++HQ0NDXjggQdQU1NjHJOQkIAffvgB33zzDXbt2oXLly/j8ccfF7Fq+3PkyBH885//bLUDA8+NeK5evYqRI0dCoVBgy5YtOHv2LJYuXQoPDw/jmHfeeQfvv/8+Vq5ciUOHDsHJyQmxsbGoq6sTsXLb9/bbb+Ojjz7CihUrcO7cObz99tt455138MEHHxjH8NyYkUAWZcSIEUJcXJzxe71eL/j5+QlLliwRsSoqKioSAAi7du0SBEEQysvLBYVCIXzzzTfGMefOnRMACAcOHBCrTLtSVVUl9OvXT9i+fbswZswYIT4+XhAEnhuxzZs3Txg1alS7PzcYDIKvr6/w7rvvGo+Vl5cLKpVK+OKLL3qiRLv18MMPC08//bTJsccff1yYOnWqIAg8N+bGK1gWRKfT4dixY4iJiTEek0qliImJwYEDB0SsjCoqKgAAvXr1AgAcO3YMDQ0NJudqwIAB6NOnD89VD4mLi8PDDz9scg4Anhuxbdy4EcOHD8ekSZPg7e2NO++8E6tXrzb+PCcnBwUFBSbnx83NDZGRkTw/3ezee+9Famoq0tPTAQAnT57E3r17MX78eAA8N+Ym+l6EdF1JSQn0er1xH8YWPj4+OH/+vEhVkcFgwNy5czFy5EjjhuEFBQVQKpVwd3c3Gevj44OCggIRqrQvX375JY4fP44jR460+hnPjbiys7Px0UcfITExES+//DKOHDmCOXPmQKlUYvr06cZz0NZ/53h+utf8+fNRWVmJAQMGQCaTQa/X480338TUqVMBgOfGzBiwiG4hLi4OZ86cwd69e8UuhQDk5+cjPj4e27dvh4ODg9jl0C8YDAYMHz4cycnJAIA777wTZ86cwcqVKzF9+nSRq7NvX3/9NT777DN8/vnnGDRoEE6cOIG5c+fCz8+P56YbcIrQgmg0GshkslZ3OxUWFsLX11ekquzb888/j02bNuHHH39E7969jcd9fX2h0+lQXl5uMp7nqvsdO3YMRUVFuOuuuyCXyyGXy7Fr1y68//77kMvl8PHx4bkRkVarxR133GFybODAgcjLywMA4zngf+d63osvvoj58+fj97//PQYPHow//OEPSEhIwJIlSwDw3JgbA5YFUSqViIiIQGpqqvGYwWBAamoqoqKiRKzM/giCgOeffx7ffvstdu7cieDgYJOfR0REQKFQmJyrtLQ05OXl8Vx1s7Fjx+L06dM4ceKE8TF8+HBMnTrV+M88N+IZOXJkq5Ym6enpCAwMBAAEBwfD19fX5PxUVlbi0KFDPD/drLa2FlKp6a99mUwGg8EAgOfG7MReZU+mvvzyS0GlUgmffPKJcPbsWeHZZ58V3N3dhYKCArFLsyuzZs0S3NzchJ9++km4cuWK8VFbW2sc89xzzwl9+vQRdu7cKRw9elSIiooSoqKiRKzaft14F6Eg8NyI6fDhw4JcLhfefPNNISMjQ/jss88EtVotfPrpp8Yxb731luDu7i58//33wqlTp4RHH31UCA4OFq5duyZi5bZv+vTpgr+/v7Bp0yYhJydH2LBhg6DRaISXXnrJOIbnxnwYsCzQBx98IPTp00dQKpXCiBEjhIMHD4pdkt0B0OZj7dq1xjHXrl0TZs+eLXh4eAhqtVqYMGGCcOXKFfGKtmO/DFg8N+L64YcfhPDwcEGlUgkDBgwQVq1aZfJzg8EgvPbaa4KPj4+gUqmEsWPHCmlpaSJVaz8qKyuF+Ph4oU+fPoKDg4MQEhIivPLKK0J9fb1xDM+N+UgE4YYWrkRERETUZVyDRURERGRmDFhEREREZsaARURERGRmDFhEREREZsaARURERGRmDFhEREREZsaARURERGRmDFhEREREZsaARURkZn/9618xbNgwscsgIhExYBERERGZGQMWERERkZkxYBGRXdm6dStGjRoFd3d3eHp64te//jWysrIAALm5uZBIJNiwYQPuv/9+qNVqDB06FAcOHDB5jdWrVyMgIABqtRoTJkzAsmXL4O7uftP3/fjjjzFw4EA4ODhgwIAB+Mc//tFdH5GILAADFhHZlZqaGiQmJuLo0aNITU2FVCrFhAkTYDAYjGNeeeUV/OUvf8GJEycQFhaGKVOmoLGxEQCwb98+PPfcc4iPj8eJEycwbtw4vPnmmzd9z88++wwLFizAm2++iXPnziE5ORmvvfYa1q1b162flYjEIxEEQRC7CCIisZSUlMDLywunT5+Gs7MzgoOD8fHHH+OZZ54BAJw9exaDBg3CuXPnMGDAAPz+979HdXU1Nm3aZHyNJ598Eps2bUJ5eTmApkXu3333HU6cOAEA6Nu3L9544w1MmTLF+JzFixcjJSUF+/fv77HPSkQ9h1ewiMiuZGRkYMqUKQgJCYGrqyuCgoIAAHl5ecYxQ4YMMf6zVqsFABQVFQEA0tLSMGLECJPX/OX3N6qpqUFWVhaeeeYZODs7Gx+LFy82Tk0Ske2Ri10AEVFPeuSRRxAYGIjVq1fDz88PBoMB4eHh0Ol0xjEKhcL4zxKJBABMphBvR3V1NYCmdVuRkZEmP5PJZJ16TSKyfAxYRGQ3SktLkZaWhtWrV2P06NEAgL17997Wa/Tv3x9HjhwxOfbL72/k4+MDPz8/ZGdnY+rUqbdfNBFZJQYsIrIbHh4e8PT0xKpVq6DVapGXl4f58+ff1mu88MILiI6OxrJly/DII49g586d2LJli/FKV1tef/11zJkzB25ubnjwwQdRX1+Po0eP4urVq0hMTOzqxyIiC8Q1WERkN6RSKb788kscO3YM4eHhSEhIwLvvvntbrzFy5EisXLkSy5Ytw9ChQ7F161YkJCTAwcGh3ef88Y9/xMcff4y1a9di8ODBGDNmDD755BMEBwd39SMRkYXiXYRERF00c+ZMnD9/Hnv27BG7FCKyEJwiJCK6TX/7298wbtw4ODk5YcuWLVi3bh0bhxKRCV7BIiK6Tb/73e/w008/oaqqCiEhIXjhhRfw3HPPiV0WEVkQBiwiIiIiM+MidyIiIiIzY8AiIiIiMjMGLCIiIiIzY8AiIiIiMjMGLCIiIiIzY8AiIiIiMjMGLCIiIiIzY8AiIiIiMrP/B9GEDIVpzb9VAAAAAElFTkSuQmCC\n"
          },
          "metadata": {}
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "eJ8YuVfCuGTj"
      },
      "source": [
        "Let us try one more augmentation: shear. Here is what this looks like:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "pMiw46NLwssK"
      },
      "source": [
        "def shear(sample, amount):\n",
        "  tform = AffineTransform(shear = amount) # create the shear transform\n",
        "  img = warp(sample, tform) # apply the shear\n",
        "  # this makes the digit off-center. Since all the images in the test set are centralized, we will do the same here\n",
        "  col = img.sum(0).nonzero()[0]\n",
        "  row = img.sum(1).nonzero()[0]\n",
        "  if len(col)>0 and len(row)>0:\n",
        "    xshift = int(sample.shape[0]/2 - (row[0]+row[-1])/2)\n",
        "    yshift = int(sample.shape[1]/2 - (col[0]+col[-1])/2)\n",
        "    img = np.roll(img, (xshift, yshift),(0,1))\n",
        "  return img"
      ],
      "execution_count": 11,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "4_u_EYpmnABK",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 860
        },
        "outputId": "59d287c0-28f1-4f64-a601-de09808e6d51"
      },
      "source": [
        "sample = train_X[2]\n",
        "plt.imshow(sample, cmap='gray')\n",
        "plt.show()\n",
        "\n",
        "# apply shear\n",
        "plt.imshow(shear(sample, 0.4), cmap='gray')"
      ],
      "execution_count": 12,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "<Figure size 640x480 with 1 Axes>"
            ],
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAaAAAAGdCAYAAABU0qcqAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/bCgiHAAAACXBIWXMAAA9hAAAPYQGoP6dpAAAb+UlEQVR4nO3df2xV9f3H8dctP66g7cVa++NKwYI/2EBqROkaleFoKHUhVsnir2XI/DGwuCn+WOpUdJJ0w2wal06XzFDdRMVlgJqlC1RbMldwIAx/bA3tqi3pDyaGe0uxhdDP9w++3nmlBc7l3r572+cj+SS955z3PW8OJ/fVc+/p5/qcc04AAAyyFOsGAAAjEwEEADBBAAEATBBAAAATBBAAwAQBBAAwQQABAEwQQAAAE6OtG/i6vr4+tbW1KTU1VT6fz7odAIBHzjl1dXUpGAwqJWXg65whF0BtbW3Kzc21bgMAcJpaW1s1ceLEAdcPubfgUlNTrVsAAMTByV7PExZAlZWVOv/883XGGWeooKBA77333inV8bYbAAwPJ3s9T0gAvfbaa1qxYoVWrlyp999/X/n5+SouLta+ffsSsTsAQDJyCTB79mxXVlYWeXz06FEXDAZdRUXFSWtDoZCTxGAwGIwkH6FQ6ISv93G/Ajp8+LB27NihoqKiyLKUlBQVFRWpvr7+uO17e3sVDoejBgBg+It7AH322Wc6evSosrKyopZnZWWpo6PjuO0rKioUCAQigzvgAGBkML8Lrry8XKFQKDJaW1utWwIADIK4/x1QRkaGRo0apc7OzqjlnZ2dys7OPm57v98vv98f7zYAAENc3K+Axo4dq1mzZqmmpiayrK+vTzU1NSosLIz37gAASSohMyGsWLFCixcv1uWXX67Zs2frmWeeUXd3t5YsWZKI3QEAklBCAujGG2/Uf//7Xz322GPq6OjQpZdequrq6uNuTAAAjFw+55yzbuKrwuGwAoGAdRsAgNMUCoWUlpY24Hrzu+AAACMTAQQAMEEAAQBMEEAAABMEEADABAEEADBBAAEATBBAAAATBBAAwAQBBAAwQQABAEwQQAAAEwQQAMAEAQQAMEEAAQBMEEAAABMEEADABAEEADBBAAEATBBAAAATBBAAwAQBBAAwQQABAEwQQAAAEwQQAMAEAQQAMEEAAQBMEEAAABMEEADABAEEADBBAAEATBBAAAATBBAAwAQBBAAwQQABAEwQQAAAEwQQAMAEAQQAMEEAAQBMEEAAABMEEADABAEEADBBAAEATBBAAAATBBAAwAQBBAAwQQABAEwQQAAAEwQQAMAEAQQAMEEAAQBMEEAAABOjrRtA8vL7/Z5rfvjDHyagk+Olp6fHVPfkk0/GuZP+/epXv/JcU1VV5bnmo48+8lwDDBaugAAAJgggAICJuAfQ448/Lp/PFzWmTZsW790AAJJcQj4Dmj59ujZv3vy/nYzmoyYAQLSEJMPo0aOVnZ2diKcGAAwTCfkMaM+ePQoGg5oyZYpuvfVWtbS0DLhtb2+vwuFw1AAADH9xD6CCggJVVVWpurpazz33nJqbm3X11Verq6ur3+0rKioUCAQiIzc3N94tAQCGoLgHUElJib73ve9p5syZKi4u1l/+8hcdOHBA69at63f78vJyhUKhyGhtbY13SwCAISjhdwdMmDBBF110kRobG/td7/f7Y/qDRgBAckv43wEdPHhQTU1NysnJSfSuAABJJO4B9MADD6iurk6ffPKJ/v73v+v666/XqFGjdPPNN8d7VwCAJBb3t+D27t2rm2++Wfv379e5556rq666Slu3btW5554b710BAJKYzznnrJv4qnA4rEAgYN3GiHLZZZfFVPfiiy96rvnmN78Z074gtbW1ea6J9Z2HnTt3eq7p7u6OaV8YvkKhkNLS0gZcz1xwAAATBBAAwAQBBAAwQQABAEwQQAAAEwQQAMAEAQQAMEEAAQBMEEAAABMEEADABAEEADBBAAEATCT8C+kw9JWVlcVUx8SigysYDHqu2bJlS0z7evrppz3X3H///THtCyMXV0AAABMEEADABAEEADBBAAEATBBAAAATBBAAwAQBBAAwQQABAEwQQAAAEwQQAMAEAQQAMEEAAQBMEEAAABPMhg1dfvnl1i2cUGNjo+eaP/3pTwnopH/Tp0/3XLNw4cIEdBI/paWlnmteeOEFzzUff/yx5xoMH1wBAQBMEEAAABMEEADABAEEADBBAAEATBBAAAATBBAAwAQBBAAwQQABAEwQQAAAEwQQAMAEAQQAMMFkpBjyGhoaPNf87Gc/S0An/TvzzDM916xatcpzzXXXXee55vzzz/dcE2tdbW2t55r169d7rnnjjTc81/zjH//wXCNJ+/bti6kOp4YrIACACQIIAGCCAAIAmCCAAAAmCCAAgAkCCABgggACAJgggAAAJgggAIAJAggAYIIAAgCYIIAAACZ8zjln3cRXhcNhBQIB6zZGlCVLlsRU9+yzz3quGT9+vOeaWCaEXLhwoecaSdq+fXtMdV5lZ2d7rlm9erXnmu9///ueayRpiL0snLZY/19jOfcOHTrkuebhhx/2XNPU1OS5ZrCFQiGlpaUNuJ4rIACACQIIAGDCcwBt2bJFCxcuVDAYlM/n04YNG6LWO+f02GOPKScnR+PGjVNRUZH27NkTr34BAMOE5wDq7u5Wfn6+Kisr+12/evVqPfvss3r++ee1bds2nXnmmSouLlZPT89pNwsAGD48fyNqSUmJSkpK+l3nnNMzzzyjRx55JPLtjS+99JKysrK0YcMG3XTTTafXLQBg2IjrZ0DNzc3q6OhQUVFRZFkgEFBBQYHq6+v7rent7VU4HI4aAIDhL64B1NHRIUnKysqKWp6VlRVZ93UVFRUKBAKRkZubG8+WAABDlPldcOXl5QqFQpHR2tpq3RIAYBDENYC+/OO6zs7OqOWdnZ0D/uGd3+9XWlpa1AAADH9xDaC8vDxlZ2erpqYmsiwcDmvbtm0qLCyM564AAEnO811wBw8eVGNjY+Rxc3Ozdu3apfT0dE2aNEn33nuvVq1apQsvvFB5eXl69NFHFQwGVVpaGs++AQBJznMAbd++Xddcc03k8YoVKyRJixcvVlVVlR566CF1d3frrrvu0oEDB3TVVVepurpaZ5xxRvy6BgAkPSYjRcz+8Ic/eK655ZZbEtDJ8X7/+9/HVPejH/3Ic82ll17queaNN97wXHPeeed5rvH5fJ5rpOE3GWmsqqurPde0t7d7rrnjjjs81yQDJiMFAAxJBBAAwAQBBAAwQQABAEwQQAAAEwQQAMAEAQQAMEEAAQBMEEAAABMEEADABAEEADBBAAEATBBAAAATzIaNmKWnp3uu2bx5s+ea/Px8zzV9fX2ea6Rj33fl1ejRnr/VROPHj/dcE4uhPhv2Rx995Lnmn//8ZwI66d+6des812zatMlzTU9Pj+eaZMBs2ACAIYkAAgCYIIAAACYIIACACQIIAGCCAAIAmCCAAAAmCCAAgAkCCABgggACAJgggAAAJgggAIAJ77MoAv/v888/91xTWVk5KDVjxozxXCPphBMnJqMPPvggprpVq1Z5rmlra/Ncs3fvXs81n376qecaDE1cAQEATBBAAAATBBAAwAQBBAAwQQABAEwQQAAAEwQQAMAEAQQAMEEAAQBMEEAAABMEEADABAEEADDhc8456ya+KhwOKxAIWLeBIeSTTz7xXJObmxv/RpLQqFGjrFvACBYKhU44wS9XQAAAEwQQAMAEAQQAMEEAAQBMEEAAABMEEADABAEEADBBAAEATBBAAAATBBAAwAQBBAAwQQABAEyMtm4AOJmUFO+/J/l8vgR0AiCeuAICAJgggAAAJjwH0JYtW7Rw4UIFg0H5fD5t2LAhav1tt90mn88XNRYsWBCvfgEAw4TnAOru7lZ+fr4qKysH3GbBggVqb2+PjFdeeeW0mgQADD+eb0IoKSlRSUnJCbfx+/3Kzs6OuSkAwPCXkM+AamtrlZmZqYsvvljLli3T/v37B9y2t7dX4XA4agAAhr+4B9CCBQv00ksvqaamRr/85S9VV1enkpISHT16tN/tKyoqFAgEIiM3NzfeLQEAhiCfc87FXOzzaf369SotLR1wm//85z+aOnWqNm/erHnz5h23vre3V729vZHH4XCYEEKUlpYWzzUTJ05MQCfJJ5a/oQLiJRQKKS0tbcD1CT87p0yZooyMDDU2Nva73u/3Ky0tLWoAAIa/hAfQ3r17tX//fuXk5CR6VwCAJOL5LriDBw9GXc00Nzdr165dSk9PV3p6up544gktWrRI2dnZampq0kMPPaQLLrhAxcXFcW0cAJDcPAfQ9u3bdc0110Qer1ixQpK0ePFiPffcc9q9e7defPFFHThwQMFgUPPnz9eTTz4pv98fv64BAEnPcwDNnTtXJ7pv4a9//etpNYTkMXq097lsV65c6bkmMzPTc81p3FszrHzwwQcx1T344IOea6qrq2PaF0YubpEBAJgggAAAJgggAIAJAggAYIIAAgCYIIAAACYIIACACQIIAGCCAAIAmCCAAAAmCCAAgAkCCABgggACAJg4ra/kToRwOKxAIGDdBk5BLP9Pn3/+eQI6sXXHHXd4rrn77rs918yYMcNzTaxfg1JfX++55qtf03KqDh8+7LkGycP8K7kBAOgPAQQAMEEAAQBMEEAAABMEEADABAEEADBBAAEATBBAAAATBBAAwAQBBAAwQQABAEwQQAAAE0xGipgxGekxZ599tueacDjsuaaystJzzbJlyzzXSFIsLwsvvfSS55olS5Z4rkHyYDJSAMCQRAABAEwQQAAAEwQQAMAEAQQAMEEAAQBMEEAAABMEEADABAEEADBBAAEATBBAAAATBBAAwMRo6waAZLdq1SrPNT/+8Y891zzyyCOea2KdjDQWhw4dGrR9YXjgCggAYIIAAgCYIIAAACYIIACACQIIAGCCAAIAmCCAAAAmCCAAgAkCCABgggACAJgggAAAJgggAIAJn3POWTfxVeFwWIFAwLoNnIIxY8Z4rnnqqac819x9992ea0aNGuW5JlZtbW2ea37wgx94rmlqavJc88knn3iukaRYXhZCoZDnmvT0dM81SB6hUEhpaWkDrucKCABgggACAJjwFEAVFRW64oorlJqaqszMTJWWlqqhoSFqm56eHpWVlemcc87RWWedpUWLFqmzszOuTQMAkp+nAKqrq1NZWZm2bt2qTZs26ciRI5o/f766u7sj29x3331688039frrr6uurk5tbW264YYb4t44ACC5efpG1Orq6qjHVVVVyszM1I4dOzRnzhyFQiG98MILWrt2rb7zne9IktasWaNvfOMb2rp1q771rW/Fr3MAQFI7rc+Avrzr5cs7WXbs2KEjR46oqKgoss20adM0adIk1dfX9/scvb29CofDUQMAMPzFHEB9fX269957deWVV2rGjBmSpI6ODo0dO1YTJkyI2jYrK0sdHR39Pk9FRYUCgUBk5ObmxtoSACCJxBxAZWVl+vDDD/Xqq6+eVgPl5eUKhUKR0draelrPBwBIDp4+A/rS8uXL9dZbb2nLli2aOHFiZHl2drYOHz6sAwcORF0FdXZ2Kjs7u9/n8vv98vv9sbQBAEhinq6AnHNavny51q9fr7ffflt5eXlR62fNmqUxY8aopqYmsqyhoUEtLS0qLCyMT8cAgGHB0xVQWVmZ1q5dq40bNyo1NTXyuU4gENC4ceMUCAR0++23a8WKFUpPT1daWpruueceFRYWcgccACCKpwB67rnnJElz586NWr5mzRrddtttkqSnn35aKSkpWrRokXp7e1VcXKzf/va3cWkWADB8MBkphrxYJtTkbspjfD5fTHVMRop4YDJSAMCQRAABAEwQQAAAEwQQAMAEAQQAMEEAAQBMEEAAABMEEADABAEEADBBAAEATBBAAAATBBAAwAQBBAAwEdM3ogKD6f777/dcs27dugR0knxinQ373Xff9Vxz7bXXxrQvjFxcAQEATBBAAAATBBAAwAQBBAAwQQABAEwQQAAAEwQQAMAEAQQAMEEAAQBMEEAAABMEEADABAEEADDBZKQY8t555x3PNaWlpTHt68knn/RcM336dM81KSmD87ufcy6mutbWVs81XV1dMe0LIxdXQAAAEwQQAMAEAQQAMEEAAQBMEEAAABMEEADABAEEADBBAAEATBBAAAATBBAAwAQBBAAwQQABAEz4XKyzFSZIOBxWIBCwbgM4ZUuWLPFcU15e7rlm6tSpnmuWL1/uuUaSXnvtNc81n3/+eUz7wvAVCoWUlpY24HqugAAAJgggAIAJAggAYIIAAgCYIIAAACYIIACACQIIAGCCAAIAmCCAAAAmCCAAgAkCCABgggACAJhgMlIAQEIwGSkAYEgigAAAJjwFUEVFha644gqlpqYqMzNTpaWlamhoiNpm7ty58vl8UWPp0qVxbRoAkPw8BVBdXZ3Kysq0detWbdq0SUeOHNH8+fPV3d0dtd2dd96p9vb2yFi9enVcmwYAJL/RXjaurq6OelxVVaXMzEzt2LFDc+bMiSwfP368srOz49MhAGBYOq3PgEKhkCQpPT09avnLL7+sjIwMzZgxQ+Xl5Tp06NCAz9Hb26twOBw1AAAjgIvR0aNH3Xe/+1135ZVXRi3/3e9+56qrq93u3bvdH//4R3feeee566+/fsDnWblypZPEYDAYjGE2QqHQCXMk5gBaunSpmzx5smttbT3hdjU1NU6Sa2xs7Hd9T0+PC4VCkdHa2mp+0BgMBoNx+uNkAeTpM6AvLV++XG+99Za2bNmiiRMnnnDbgoICSVJjY6OmTp163Hq/3y+/3x9LGwCAJOYpgJxzuueee7R+/XrV1tYqLy/vpDW7du2SJOXk5MTUIABgePIUQGVlZVq7dq02btyo1NRUdXR0SJICgYDGjRunpqYmrV27Vtdee63OOecc7d69W/fdd5/mzJmjmTNnJuQfAABIUl4+99EA7/OtWbPGOedcS0uLmzNnjktPT3d+v99dcMEF7sEHHzzp+4BfFQqFzN+3ZDAYDMbpj5O99jMZKQAgIZiMFAAwJBFAAAATBBAAwAQBBAAwQQABAEwQQAAAEwQQAMAEAQQAMEEAAQBMEEAAABMEEADABAEEADBBAAEATBBAAAATBBAAwAQBBAAwQQABAEwQQAAAEwQQAMAEAQQAMEEAAQBMEEAAABMEEADABAEEADBBAAEATAy5AHLOWbcAAIiDk72eD7kA6urqsm4BABAHJ3s997khdsnR19entrY2paamyufzRa0Lh8PKzc1Va2ur0tLSjDq0x3E4huNwDMfhGI7DMUPhODjn1NXVpWAwqJSUga9zRg9iT6ckJSVFEydOPOE2aWlpI/oE+xLH4RiOwzEch2M4DsdYH4dAIHDSbYbcW3AAgJGBAAIAmEiqAPL7/Vq5cqX8fr91K6Y4DsdwHI7hOBzDcTgmmY7DkLsJAQAwMiTVFRAAYPgggAAAJgggAIAJAggAYCJpAqiyslLnn3++zjjjDBUUFOi9996zbmnQPf744/L5fFFj2rRp1m0l3JYtW7Rw4UIFg0H5fD5t2LAhar1zTo899phycnI0btw4FRUVac+ePTbNJtDJjsNtt9123PmxYMECm2YTpKKiQldccYVSU1OVmZmp0tJSNTQ0RG3T09OjsrIynXPOOTrrrLO0aNEidXZ2GnWcGKdyHObOnXvc+bB06VKjjvuXFAH02muvacWKFVq5cqXef/995efnq7i4WPv27bNubdBNnz5d7e3tkfG3v/3NuqWE6+7uVn5+viorK/tdv3r1aj377LN6/vnntW3bNp155pkqLi5WT0/PIHeaWCc7DpK0YMGCqPPjlVdeGcQOE6+urk5lZWXaunWrNm3apCNHjmj+/Pnq7u6ObHPffffpzTff1Ouvv666ujq1tbXphhtuMOw6/k7lOEjSnXfeGXU+rF692qjjAbgkMHv2bFdWVhZ5fPToURcMBl1FRYVhV4Nv5cqVLj8/37oNU5Lc+vXrI4/7+vpcdna2e+qppyLLDhw44Px+v3vllVcMOhwcXz8Ozjm3ePFid91115n0Y2Xfvn1Okqurq3POHfu/HzNmjHv99dcj2/zrX/9yklx9fb1Vmwn39ePgnHPf/va33U9+8hO7pk7BkL8COnz4sHbs2KGioqLIspSUFBUVFam+vt6wMxt79uxRMBjUlClTdOutt6qlpcW6JVPNzc3q6OiIOj8CgYAKCgpG5PlRW1urzMxMXXzxxVq2bJn2799v3VJChUIhSVJ6erokaceOHTpy5EjU+TBt2jRNmjRpWJ8PXz8OX3r55ZeVkZGhGTNmqLy8XIcOHbJob0BDbjLSr/vss8909OhRZWVlRS3PysrSv//9b6OubBQUFKiqqkoXX3yx2tvb9cQTT+jqq6/Whx9+qNTUVOv2THR0dEhSv+fHl+tGigULFuiGG25QXl6empqa9PDDD6ukpET19fUaNWqUdXtx19fXp3vvvVdXXnmlZsyYIenY+TB27FhNmDAhatvhfD70dxwk6ZZbbtHkyZMVDAa1e/du/fSnP1VDQ4P+/Oc/G3YbbcgHEP6npKQk8vPMmTNVUFCgyZMna926dbr99tsNO8NQcNNNN0V+vuSSSzRz5kxNnTpVtbW1mjdvnmFniVFWVqYPP/xwRHwOeiIDHYe77ror8vMll1yinJwczZs3T01NTZo6depgt9mvIf8WXEZGhkaNGnXcXSydnZ3Kzs426mpomDBhgi666CI1NjZat2Lmy3OA8+N4U6ZMUUZGxrA8P5YvX6633npL77zzTtTXt2RnZ+vw4cM6cOBA1PbD9XwY6Dj0p6CgQJKG1Pkw5ANo7NixmjVrlmpqaiLL+vr6VFNTo8LCQsPO7B08eFBNTU3KycmxbsVMXl6esrOzo86PcDisbdu2jfjzY+/evdq/f/+wOj+cc1q+fLnWr1+vt99+W3l5eVHrZ82apTFjxkSdDw0NDWppaRlW58PJjkN/du3aJUlD63ywvgviVLz66qvO7/e7qqoq9/HHH7u77rrLTZgwwXV0dFi3Nqjuv/9+V1tb65qbm927777rioqKXEZGhtu3b591awnV1dXldu7c6Xbu3OkkuV//+tdu586d7tNPP3XOOfeLX/zCTZgwwW3cuNHt3r3bXXfddS4vL8998cUXxp3H14mOQ1dXl3vggQdcfX29a25udps3b3aXXXaZu/DCC11PT49163GzbNkyFwgEXG1trWtvb4+MQ4cORbZZunSpmzRpknv77bfd9u3bXWFhoSssLDTsOv5OdhwaGxvdz3/+c7d9+3bX3NzsNm7c6KZMmeLmzJlj3Hm0pAgg55z7zW9+4yZNmuTGjh3rZs+e7bZu3Wrd0qC78cYbXU5Ojhs7dqw777zz3I033ugaGxut20q4d955x0k6bixevNg5d+xW7EcffdRlZWU5v9/v5s2b5xoaGmybToATHYdDhw65+fPnu3PPPdeNGTPGTZ482d15553D7pe0/v79ktyaNWsi23zxxRfu7rvvdmeffbYbP368u/766117e7td0wlwsuPQ0tLi5syZ49LT053f73cXXHCBe/DBB10oFLJt/Gv4OgYAgIkh/xkQAGB4IoAAACYIIACACQIIAGCCAAIAmCCAAAAmCCAAgAkCCABgggACAJgggAAAJgggAIAJAggAYOL/AC0/Gz+zK6mJAAAAAElFTkSuQmCC\n"
          },
          "metadata": {}
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<matplotlib.image.AxesImage at 0x7f0738c52260>"
            ]
          },
          "metadata": {},
          "execution_count": 12
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "<Figure size 640x480 with 1 Axes>"
            ],
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAaAAAAGdCAYAAABU0qcqAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/bCgiHAAAACXBIWXMAAA9hAAAPYQGoP6dpAAAeG0lEQVR4nO3de2zV9f3H8dcplwNoe7AUepGCBS8sAt1kUBuQoTSUqkSUGbxtsBmMWNgAnaaLiswt/cmWzbgxXbIFNBNUFoHIlEwLLV4KjiohxK2hWKQKLcLGOVBswfbz+4PYeeTm53ja9znl+Ui+CT3n++r37ZevffHtOf004JxzAgCgi6VYDwAAOD9RQAAAExQQAMAEBQQAMEEBAQBMUEAAABMUEADABAUEADDR03qAr2pvb9e+ffuUmpqqQCBgPQ4AwJNzTkeOHFFOTo5SUs58n5NwBbRv3z7l5uZajwEA+IYaGho0ePDgMz6fcN+CS01NtR4BABAH5/p63mkFtGzZMl1yySXq06ePCgoK9O67736tHN92A4Du4VxfzzulgF588UUtWrRIixcv1nvvvaf8/HwVFxfrwIEDnXE4AEAycp1g3LhxrrS0tOPjtrY2l5OT48rLy8+ZDYfDThIbGxsbW5Jv4XD4rF/v434HdPz4cdXU1KioqKjjsZSUFBUVFam6uvqU/VtbWxWJRKI2AED3F/cCOnjwoNra2pSZmRn1eGZmphobG0/Zv7y8XKFQqGPjHXAAcH4wfxdcWVmZwuFwx9bQ0GA9EgCgC8T954AyMjLUo0cPNTU1RT3e1NSkrKysU/YPBoMKBoPxHgMAkODifgfUu3dvjRkzRhUVFR2Ptbe3q6KiQoWFhfE+HAAgSXXKSgiLFi3SrFmz9N3vflfjxo3Tk08+qebmZv3oRz/qjMMBAJJQpxTQzJkz9emnn+rRRx9VY2Ojvv3tb2vDhg2nvDEBAHD+CjjnnPUQXxaJRBQKhazHAAB8Q+FwWGlpaWd83vxdcACA8xMFBAAwQQEBAExQQAAAExQQAMAEBQQAMEEBAQBMUEAAABMUEADABAUEADBBAQEATFBAAAATFBAAwAQFBAAwQQEBAExQQAAAExQQAMAEBQQAMEEBAQBMUEAAABMUEADABAUEADBBAQEATFBAAAATFBAAwAQFBAAw0dN6AMRXMBj0zlx99dUxHSsQCHhnQqGQd+aGG27wzrS1tXlnJOkf//iHd6a6uto709jY6J0BuhvugAAAJiggAIAJCggAYIICAgCYoIAAACYoIACACQoIAGCCAgIAmKCAAAAmKCAAgAkKCABgggICAJhgMdIEdtFFF3lnioqKvDM33nijd0aSMjMzvTO9evXyzowaNco7M3DgQO+MJE2ePNk7884773hn/vrXv3pn3n33Xe/MkSNHvDOS5JyLKQf44A4IAGCCAgIAmKCAAAAmKCAAgAkKCABgggICAJiggAAAJiggAIAJCggAYIICAgCYoIAAACYoIACACRYjTWD9+vXzzlx11VXemWnTpnlnJKl///4x5bpCrItpXnrppd6ZSy65xDtTUFDgnVm1apV3Zv369d4ZSXrvvfdiygE+uAMCAJiggAAAJuJeQI899pgCgUDUNmLEiHgfBgCQ5DrlNaArr7xSb7zxxv8O0pOXmgAA0TqlGXr27KmsrKzO+NQAgG6iU14D2rVrl3JycjRs2DDdeeed2rt37xn3bW1tVSQSidoAAN1f3AuooKBAK1as0IYNG/T000+rvr5e11xzzRl/N315eblCoVDHlpubG++RAAAJKO4FVFJSoltvvVWjR49WcXGxXn31VR0+fFgvvfTSafcvKytTOBzu2BoaGuI9EgAgAXX6uwP69++vyy+/XHV1dad9PhgMKhgMdvYYAIAE0+k/B3T06FHt3r1b2dnZnX0oAEASiXsBPfDAA6qqqtKePXv0zjvv6Oabb1aPHj10++23x/tQAIAkFvdvwX388ce6/fbbdejQIQ0cOFATJkzQli1bNHDgwHgfCgCQxAIu1lUbO0kkElEoFLIeI2n9+c9/9s7Eenfat29f78zRo0e9M7G8MaWtrc07I0lpaWnemYsvvtg706NHD+9MIBDwzrz66qveGUkqLS31znz00UfemQT78oM4C4fDZ/1/irXgAAAmKCAAgAkKCABgggICAJiggAAAJiggAIAJCggAYIICAgCYoIAAACYoIACACQoIAGCCAgIAmOj0X0iHrrVt2zbvzI033hjTsWJZjLS5udk7s3nzZu/Mli1bvDPSyV+g6GvSpEnemfHjx3tnYplt4sSJ3hlJ+tWvfuWdee6557wztbW13pn//Oc/3plYrjsp9kVt8fVwBwQAMEEBAQBMUEAAABMUEADABAUEADBBAQEATFBAAAATFBAAwAQFBAAwQQEBAExQQAAAExQQAMAEBQQAMBFwzjnrIb4sEokoFApZj5G0BgwY4J15+eWXYzpWLCs6nzhxwjuzfv1678ysWbO8M5J07Ngx70xGRoZ3Zt68ed6ZmTNnemdGjBjhnZGk9vZ278ynn37qnamsrPTOvPnmm96Zmpoa74wkHTx40Dvz+eefe2f27NnjnUkG4XBYaWlpZ3yeOyAAgAkKCABgggICAJiggAAAJiggAIAJCggAYIICAgCYoIAAACYoIACACQoIAGCCAgIAmKCAAAAmWIwUmjBhQky5xx9/3DtTWFjonWlpafHOLFmyxDsjSc8++6x3JhKJeGeGDRvmnZkzZ4535gc/+IF3RpIyMzO9Mwn2pSRKrIt9fvjhh96Z//73v96ZW2+91TuTDFiMFACQkCggAIAJCggAYIICAgCYoIAAACYoIACACQoIAGCCAgIAmKCAAAAmKCAAgAkKCABgggICAJjoaT0A7L311lsx5R544AHvzBNPPOGdufbaa70zP/nJT7wzktTe3u6d+dvf/uadueGGG7wzP/7xj70z6enp3plE19bW5p25+OKLYzpWz57+XyLffvvtmI51PuIOCABgggICAJjwLqDNmzdr2rRpysnJUSAQ0Nq1a6Oed87p0UcfVXZ2tvr27auioiLt2rUrXvMCALoJ7wJqbm5Wfn6+li1bdtrnly5dqqeeekrPPPOMtm7dqgsuuEDFxcUx/VIxAED35f0KW0lJiUpKSk77nHNOTz75pB5++GHddNNNkqTnnntOmZmZWrt2rW677bZvNi0AoNuI62tA9fX1amxsVFFRUcdjoVBIBQUFqq6uPm2mtbVVkUgkagMAdH9xLaDGxkZJp/4++czMzI7nvqq8vFyhUKhjy83NjedIAIAEZf4uuLKyMoXD4Y6toaHBeiQAQBeIawFlZWVJkpqamqIeb2pq6njuq4LBoNLS0qI2AED3F9cCysvLU1ZWlioqKjoei0Qi2rp1qwoLC+N5KABAkvN+F9zRo0dVV1fX8XF9fb22b9+u9PR0DRkyRAsWLNAvf/lLXXbZZcrLy9MjjzyinJwcTZ8+PZ5zAwCSnHcBbdu2LWptrkWLFkmSZs2apRUrVujBBx9Uc3Oz7rnnHh0+fFgTJkzQhg0b1KdPn/hNDQBIegHnnLMe4ssikYhCoZD1GOgkGRkZ3pl169Z5Z8aNG+edkU7+LJuvcDjsnenVq5d3JjU11TsTCAS8M7HmuupLSW1trXdmx44dMR0rloVP16xZ451ZvXq1dyYZhMPhs76ub/4uOADA+YkCAgCYoIAAACYoIACACQoIAGCCAgIAmKCAAAAmKCAAgAkKCABgggICAJiggAAAJiggAIAJCggAYILVsNGlYlllefz48d6ZxYsXe2ckRf2qka8r1hWnE/U4kvTpp596Z9auXeudqays9M7s27fPO/PRRx95Z6TYVjo/ceKEd+bo0aPemWTAatgAgIREAQEATFBAAAATFBAAwAQFBAAwQQEBAExQQAAAExQQAMAEBQQAMEEBAQBMUEAAABMUEADABIuRIuGlpPj/O+k73/lOTMd69dVXvTMDBw6M6ViJ7MMPP/TOFBcXe2diWfT0888/984cP37cOxPrsfA/LEYKAEhIFBAAwAQFBAAwQQEBAExQQAAAExQQAMAEBQQAMEEBAQBMUEAAABMUEADABAUEADBBAQEATPS0HgA4l/b2du9MTU1NTMd6++23vTPXX3+9dyYYDHpnutLw4cO9M5mZmd6Z3bt3e2fQfXAHBAAwQQEBAExQQAAAExQQAMAEBQQAMEEBAQBMUEAAABMUEADABAUEADBBAQEATFBAAAATFBAAwASLkSLhBQIB70xeXl5Mx2poaPDOOOdiOlZ386c//ck7s2zZMu/M3//+d+9MLH+v6HzcAQEATFBAAAAT3gW0efNmTZs2TTk5OQoEAlq7dm3U87Nnz1YgEIjapk6dGq95AQDdhHcBNTc3Kz8//6zfu506dar279/fsa1ateobDQkA6H6834RQUlKikpKSs+4TDAaVlZUV81AAgO6vU14Dqqys1KBBg3TFFVdo7ty5OnTo0Bn3bW1tVSQSidoAAN1f3Ato6tSpeu6551RRUaEnnnhCVVVVKikpUVtb22n3Ly8vVygU6thyc3PjPRIAIAHF/eeAbrvtto4/jxo1SqNHj9bw4cNVWVmpyZMnn7J/WVmZFi1a1PFxJBKhhADgPNDpb8MeNmyYMjIyVFdXd9rng8Gg0tLSojYAQPfX6QX08ccf69ChQ8rOzu7sQwEAkoj3t+COHj0adTdTX1+v7du3Kz09Xenp6VqyZIlmzJihrKws7d69Ww8++KAuvfRSFRcXx3VwAEBy8y6gbdu26dprr+34+IvXb2bNmqWnn35aO3bs0LPPPqvDhw8rJydHU6ZM0eOPP65gMBi/qQEASS/gEmwlxUgkolAoZD0Gkly/fv1iym3atMk7M2bMGO9MSkrXrIIV6w+BT5gwwTuTk5PjnYnlxy6ef/5578yX3+jk4/PPP48ph5PC4fBZX9dnLTgAgAkKCABgggICAJiggAAAJiggAIAJCggAYIICAgCYoIAAACYoIACACQoIAGCCAgIAmKCAAAAmKCAAgIm4/0puIBHEusj7Bx984J0ZO3ZsTMfqCnv37o0pF8vq0dOmTfPOlJSUeGfuuusu78zx48e9M5L0hz/8wTuzZ8+emI51PuIOCABgggICAJiggAAAJiggAIAJCggAYIICAgCYoIAAACYoIACACQoIAGCCAgIAmKCAAAAmKCAAgImAi3XVxk4SiUQUCoWsx0CS69GjR0y5iRMnemcqKipiOlZXqK2tjSk3e/Zs70xdXZ13Zv78+d6ZxYsXe2cOHDjgnZGkJ5980jtTXl4e07G6o3A4rLS0tDM+zx0QAMAEBQQAMEEBAQBMUEAAABMUEADABAUEADBBAQEATFBAAAATFBAAwAQFBAAwQQEBAExQQAAAEyxGCnxJ3759vTMvvfSSd6aoqMg7EwwGvTOx2rRpk3fm/vvv98588skn3pmqqirvzIgRI7wzkvTmm296Z2bNmuWd2bNnj3cmGbAYKQAgIVFAAAATFBAAwAQFBAAwQQEBAExQQAAAExQQAMAEBQQAMEEBAQBMUEAAABMUEADABAUEADDBYqTAlwQCAe9Mfn6+d+bBBx/0zsyYMcM706tXL++MJLW0tHhnYlkkdN26dd6ZwsJC78wPf/hD74wktba2emd+97vfeWfKysq8M8mAxUgBAAmJAgIAmPAqoPLyco0dO1apqakaNGiQpk+frtra2qh9WlpaVFpaqgEDBujCCy/UjBkz1NTUFNehAQDJz6uAqqqqVFpaqi1btuj111/XiRMnNGXKFDU3N3fss3DhQr3yyitavXq1qqqqtG/fPt1yyy1xHxwAkNx6+uy8YcOGqI9XrFihQYMGqaamRhMnTlQ4HNZf/vIXrVy5Utddd50kafny5frWt76lLVu26Oqrr47f5ACApPaNXgMKh8OSpPT0dElSTU2NTpw4EfXrhkeMGKEhQ4aourr6tJ+jtbVVkUgkagMAdH8xF1B7e7sWLFig8ePHa+TIkZKkxsZG9e7dW/3794/aNzMzU42Njaf9POXl5QqFQh1bbm5urCMBAJJIzAVUWlqqnTt36oUXXvhGA5SVlSkcDndsDQ0N3+jzAQCSg9drQF+YN2+e1q9fr82bN2vw4MEdj2dlZen48eM6fPhw1F1QU1OTsrKyTvu5gsGggsFgLGMAAJKY1x2Qc07z5s3TmjVrtHHjRuXl5UU9P2bMGPXq1UsVFRUdj9XW1mrv3r0x/fQyAKD78roDKi0t1cqVK7Vu3TqlpqZ2vK4TCoXUt29fhUIh3X333Vq0aJHS09OVlpam+fPnq7CwkHfAAQCieBXQ008/LUmaNGlS1OPLly/X7NmzJZ1cByklJUUzZsxQa2uriouL9cc//jEuwwIAug8WIwUM9OnTxzsTy2KfY8eO9c50R7EsMitJR44c8c7885//9M5MnjzZO5MMWIwUAJCQKCAAgAkKCABgggICAJiggAAAJiggAIAJCggAYIICAgCYoIAAACYoIACACQoIAGCCAgIAmKCAAAAmYvqNqAC+mZaWFu/M3LlzvTNLlizxzkjSdddd552J5Tcbp6Qk9r+BP/nkE+/Mb37zm06YpHtK7L99AEC3RQEBAExQQAAAExQQAMAEBQQAMEEBAQBMUEAAABMUEADABAUEADBBAQEATFBAAAATFBAAwASLkQJJYseOHd6Z+fPnx3SsadOmeWfuu+8+78zw4cO9Mz17+n/Zam9v985I0oUXXuidyc/P98689tpr3pnugDsgAIAJCggAYIICAgCYoIAAACYoIACACQoIAGCCAgIAmKCAAAAmKCAAgAkKCABgggICAJiggAAAJgLOOWc9xJdFIhGFQiHrMYDzWo8ePbwzwWDQO/P973/fO7Nw4ULvzJVXXumdkaQnnnjCO/Pss896Z+rq6rwzySAcDistLe2Mz3MHBAAwQQEBAExQQAAAExQQAMAEBQQAMEEBAQBMUEAAABMUEADABAUEADBBAQEATFBAAAATFBAAwASLkQIw07t3b+/M2Ra3PJMLLrjAOyNJLS0t3plwONwlx0kGLEYKAEhIFBAAwIRXAZWXl2vs2LFKTU3VoEGDNH36dNXW1kbtM2nSJAUCgajt3nvvjevQAIDk51VAVVVVKi0t1ZYtW/T666/rxIkTmjJlipqbm6P2mzNnjvbv39+xLV26NK5DAwCSX0+fnTds2BD18YoVKzRo0CDV1NRo4sSJHY/369dPWVlZ8ZkQANAtfaPXgL54t0d6enrU488//7wyMjI0cuRIlZWV6dixY2f8HK2trYpEIlEbAKD787oD+rL29nYtWLBA48eP18iRIzsev+OOOzR06FDl5ORox44deuihh1RbW6uXX375tJ+nvLxcS5YsiXUMAECSivnngObOnavXXntNb731lgYPHnzG/TZu3KjJkyerrq5Ow4cPP+X51tZWtba2dnwciUSUm5sby0gAkgw/BxT7cZLBuX4OKKY7oHnz5mn9+vXavHnzWctHkgoKCiTpjAUUDAYVDAZjGQMAkMS8Csg5p/nz52vNmjWqrKxUXl7eOTPbt2+XJGVnZ8c0IACge/IqoNLSUq1cuVLr1q1TamqqGhsbJUmhUEh9+/bV7t27tXLlSl1//fUaMGCAduzYoYULF2rixIkaPXp0p/wHAACSk9drQIFA4LSPL1++XLNnz1ZDQ4Puuusu7dy5U83NzcrNzdXNN9+shx9++Gt/35a14IDzB68BxX6cZBDX14DO1VW5ubmqqqry+ZQAgPNUzG/DBoBv6vjx496ZgwcPdkkGnY/FSAEAJiggAIAJCggAYIICAgCYoIAAACYoIACACQoIAGCCAgIAmKCAAAAmKCAAgAkKCABgggICAJiggAAAJiggAIAJCggAYIICAgCYoIAAACYoIACACQoIAGCCAgIAmKCAAAAmKCAAgAkKCABgggICAJhIuAJyzlmPAACIg3N9PU+4Ajpy5Ij1CACAODjX1/OAS7Bbjvb2du3bt0+pqakKBAJRz0UiEeXm5qqhoUFpaWlGE9rjPJzEeTiJ83AS5+GkRDgPzjkdOXJEOTk5Skk5831Ozy6c6WtJSUnR4MGDz7pPWlraeX2BfYHzcBLn4STOw0mch5Osz0MoFDrnPgn3LTgAwPmBAgIAmEiqAgoGg1q8eLGCwaD1KKY4DydxHk7iPJzEeTgpmc5Dwr0JAQBwfkiqOyAAQPdBAQEATFBAAAATFBAAwETSFNCyZct0ySWXqE+fPiooKNC7775rPVKXe+yxxxQIBKK2ESNGWI/V6TZv3qxp06YpJydHgUBAa9eujXreOadHH31U2dnZ6tu3r4qKirRr1y6bYTvRuc7D7NmzT7k+pk6dajNsJykvL9fYsWOVmpqqQYMGafr06aqtrY3ap6WlRaWlpRowYIAuvPBCzZgxQ01NTUYTd46vcx4mTZp0yvVw7733Gk18eklRQC+++KIWLVqkxYsX67333lN+fr6Ki4t14MAB69G63JVXXqn9+/d3bG+99Zb1SJ2uublZ+fn5WrZs2WmfX7p0qZ566ik988wz2rp1qy644AIVFxerpaWliyftXOc6D5I0derUqOtj1apVXThh56uqqlJpaam2bNmi119/XSdOnNCUKVPU3Nzcsc/ChQv1yiuvaPXq1aqqqtK+fft0yy23GE4df1/nPEjSnDlzoq6HpUuXGk18Bi4JjBs3zpWWlnZ83NbW5nJyclx5ebnhVF1v8eLFLj8/33oMU5LcmjVrOj5ub293WVlZ7te//nXHY4cPH3bBYNCtWrXKYMKu8dXz4Jxzs2bNcjfddJPJPFYOHDjgJLmqqirn3Mm/+169ernVq1d37POvf/3LSXLV1dVWY3a6r54H55z73ve+537605/aDfU1JPwd0PHjx1VTU6OioqKOx1JSUlRUVKTq6mrDyWzs2rVLOTk5GjZsmO68807t3bvXeiRT9fX1amxsjLo+QqGQCgoKzsvro7KyUoMGDdIVV1yhuXPn6tChQ9YjdapwOCxJSk9PlyTV1NToxIkTUdfDiBEjNGTIkG59PXz1PHzh+eefV0ZGhkaOHKmysjIdO3bMYrwzSrjFSL/q4MGDamtrU2ZmZtTjmZmZ+ve//200lY2CggKtWLFCV1xxhfbv368lS5bommuu0c6dO5Wammo9nonGxkZJOu318cVz54upU6fqlltuUV5ennbv3q2f//znKikpUXV1tXr06GE9Xty1t7drwYIFGj9+vEaOHCnp5PXQu3dv9e/fP2rf7nw9nO48SNIdd9yhoUOHKicnRzt27NBDDz2k2tpavfzyy4bTRkv4AsL/lJSUdPx59OjRKigo0NChQ/XSSy/p7rvvNpwMieC2227r+POoUaM0evRoDR8+XJWVlZo8ebLhZJ2jtLRUO3fuPC9eBz2bM52He+65p+PPo0aNUnZ2tiZPnqzdu3dr+PDhXT3maSX8t+AyMjLUo0ePU97F0tTUpKysLKOpEkP//v11+eWXq66uznoUM19cA1wfpxo2bJgyMjK65fUxb948rV+/Xps2bYr69S1ZWVk6fvy4Dh8+HLV/d70eznQeTqegoECSEup6SPgC6t27t8aMGaOKioqOx9rb21VRUaHCwkLDyewdPXpUu3fvVnZ2tvUoZvLy8pSVlRV1fUQiEW3duvW8vz4+/vhjHTp0qFtdH845zZs3T2vWrNHGjRuVl5cX9fyYMWPUq1evqOuhtrZWe/fu7VbXw7nOw+ls375dkhLrerB+F8TX8cILL7hgMOhWrFjhPvjgA3fPPfe4/v37u8bGRuvRutT999/vKisrXX19vXv77bddUVGRy8jIcAcOHLAerVMdOXLEvf/+++799993ktxvf/tb9/7777uPPvrIOefc//3f/7n+/fu7devWuR07dribbrrJ5eXluc8++8x48vg623k4cuSIe+CBB1x1dbWrr693b7zxhrvqqqvcZZdd5lpaWqxHj5u5c+e6UCjkKisr3f79+zu2Y8eOdexz7733uiFDhriNGze6bdu2ucLCQldYWGg4dfyd6zzU1dW5X/ziF27btm2uvr7erVu3zg0bNsxNnDjRePJoSVFAzjn3+9//3g0ZMsT17t3bjRs3zm3ZssV6pC43c+ZMl52d7Xr37u0uvvhiN3PmTFdXV2c9VqfbtGmTk3TKNmvWLOfcybdiP/LIIy4zM9MFg0E3efJkV1tbazt0JzjbeTh27JibMmWKGzhwoOvVq5cbOnSomzNnTrf7R9rp/vslueXLl3fs89lnn7n77rvPXXTRRa5fv37u5ptvdvv377cbuhOc6zzs3bvXTZw40aWnp7tgMOguvfRS97Of/cyFw2Hbwb+CX8cAADCR8K8BAQC6JwoIAGCCAgIAmKCAAAAmKCAAgAkKCABgggICAJiggAAAJiggAIAJCggAYIICAgCYoIAAACb+H2nB5KWJYkRWAAAAAElFTkSuQmCC\n"
          },
          "metadata": {}
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "lGnWMoyM2pK4"
      },
      "source": [
        "Create an augmentation function which applies a random shear according to the constraint we provide:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "-qLDJyGytwP5"
      },
      "source": [
        "def augShear(sample, shearconstraint):\n",
        "  if shearconstraint==0:\n",
        "    return sample\n",
        "  if len(sample.shape)==2:\n",
        "    sample = np.expand_dims(sample, 0)  # make sure the sample is 3 dimensional\n",
        "  amt = rng.random(len(sample)) # generate random numbers for shear\n",
        "  amt = (amt-0.5)*shearconstraint # make the random shear constrained\n",
        "  nsample = sample.copy() # preallocate the augmented array to make it faster\n",
        "  for ii in range(len(sample)):\n",
        "    nsample[ii] = shear(sample[ii], amt[ii])\n",
        "  return np.squeeze(nsample) # take care if the input had only one sample."
      ],
      "execution_count": 13,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "s6lQcWW93suJ"
      },
      "source": [
        "Let us do a grid search to find the best shear constraint."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "l_wrqPkrzBb_",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "9214bc9d-38db-42f4-d8ec-81d9084e7350"
      },
      "source": [
        "shearconstraints = [0, 0.2,0.4,0.6,0.8,1.0,1.2,1.4,1.6,1.8,2.0] # the values we want to test\n",
        "accuracies = np.zeros(len(shearconstraints), dtype=np.float) # we will save the values here\n",
        "\n",
        "for ii in range(len(shearconstraints)):\n",
        "  # create the augmented dataset\n",
        "  augdata = train_X # we include the original images also in the augmented dataset\n",
        "  auglabel = train_y\n",
        "  for jj in range(naugmentations):\n",
        "    augdata = np.concatenate((augdata, augShear(train_X, shearconstraints[ii]))) # concatenate the augmented data to the set\n",
        "    auglabel = np.concatenate((auglabel, train_y))  # the labels don't change when we augment\n",
        "\n",
        "  # check the test accuracy\n",
        "  testpred = NN(augdata, auglabel, test_X)\n",
        "  accuracies[ii] = Accuracy(test_y, testpred)\n",
        "  print('Accuracy after shear augmentation constrained by ',shearconstraints[ii], ' is ', accuracies[ii], flush=True)"
      ],
      "execution_count": 14,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "<ipython-input-14-8a89fef4983d>:2: DeprecationWarning: `np.float` is a deprecated alias for the builtin `float`. To silence this warning, use `float` by itself. Doing this will not modify any behavior and is safe. If you specifically wanted the numpy scalar type, use `np.float64` here.\n",
            "Deprecated in NumPy 1.20; for more details and guidance: https://numpy.org/devdocs/release/1.20.0-notes.html#deprecations\n",
            "  accuracies = np.zeros(len(shearconstraints), dtype=np.float) # we will save the values here\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Accuracy after shear augmentation constrained by  0  is  0.6472\n",
            "Accuracy after shear augmentation constrained by  0.2  is  0.6147\n",
            "Accuracy after shear augmentation constrained by  0.4  is  0.632\n",
            "Accuracy after shear augmentation constrained by  0.6  is  0.6351\n",
            "Accuracy after shear augmentation constrained by  0.8  is  0.6636\n",
            "Accuracy after shear augmentation constrained by  1.0  is  0.6652\n",
            "Accuracy after shear augmentation constrained by  1.2  is  0.6495\n",
            "Accuracy after shear augmentation constrained by  1.4  is  0.6181\n",
            "Accuracy after shear augmentation constrained by  1.6  is  0.6323\n",
            "Accuracy after shear augmentation constrained by  1.8  is  0.6432\n",
            "Accuracy after shear augmentation constrained by  2.0  is  0.6277\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "EKaH-YR-zVnA",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 481
        },
        "outputId": "2253e744-5df0-40c4-bbfd-8a788455faf7"
      },
      "source": [
        "fig = plt.figure()\n",
        "ax = fig.add_axes([0.1, 0.1, 0.8, 0.8]) # main axes\n",
        "# plot the variation of accuracy\n",
        "ax.plot(shearconstraints, accuracies)\n",
        "ax.set_xlabel('angle')\n",
        "ax.set_ylabel('accuracy')\n",
        "# plot the maximum accuracy\n",
        "maxind = np.argmax(accuracies)\n",
        "plt.scatter(shearconstraints[maxind], accuracies[maxind], c='red')"
      ],
      "execution_count": 15,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<matplotlib.collections.PathCollection at 0x7f0738b16080>"
            ]
          },
          "metadata": {},
          "execution_count": 15
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "<Figure size 640x480 with 1 Axes>"
            ],
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAlAAAAG/CAYAAACXGOmPAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/bCgiHAAAACXBIWXMAAA9hAAAPYQGoP6dpAABs0ElEQVR4nO3deXhTZfo38O9J0iRd05buC12AsnVjtxREFMRlGNEZBcYNRpkRywzCzLzK+FPUUVBHnU1HhJHFmVFAFHUEQUUWZS/72lK6l6606d6mTc77RxaoLdC0SU+W7+e6cimnT5L7kJLcec793I8giqIIIiIiIuo2mdQBEBERETkbJlBEREREVmICRURERGQlJlBEREREVmICRURERGQlJlBEREREVmICRURERGQlJlBEREREVlJIHYAjMhgMuHTpEnx9fSEIgtThEBERUR8QRRH19fWIiIiATHb9OSYmUF24dOkSoqOjpQ6DiIiIJFBUVISoqKjrjmEC1QVfX18Axr9APz8/iaMhIiKivlBXV4fo6GhLHnA9TKC6YL5s5+fnxwSKiIjIzXSnfIdF5ERERERWYgJFREREZCUmUERERERWYgJFREREZCUmUERERERWYgJFREREZCUmUERERERWYgJFREREZCUmUERERERWYidyIiIzvR74/nugtBQIDwcmTgTkcqmjIiIHxASKiAgAPv0UWLgQKC6+ciwqCvjb34D77pMuLiJySLyER0T06afAz3/eMXkCgJIS4/FPP5UmLiJyWEygiMi96fXGmSdR7Pwz87GnnjKOIyIy4SU8InJv33+PxvIqHItJwaHo4ciMHA51eyv+8cXr8G5rMSZRRUXG2qhbbpE6WiJyEEygiMjtXG5oxeH8GhzOr8bhI3U489QG6GUdi8U3JU3Bo0e/vHKgtLSPoyQiR8YEiohcmiiKKK5pxqG8ahzOr8ah/GrkVjZeNUIOyIDI2gqMKT4DmSji08RbsW7kT/Dw0S2QwXQZLzxckviJyDExgSIil2IwiMgqrzfOLuXX4HBeNcrqWjqNSwj1wZjYQIyN8cfo2Xcj8sJpQBTRoPTEN4PGIbdfFPbEjcAt+ceMq/EmTpTgbIjIUTGBIiKn1tqux+mSWhzKM16Sy8yvRl1Le4cxCpmApCgNxsQGYkxsIEbHBCDAW3llwPIXjKvtBAE+umbcf/IbrB4zA2tH/dSYQP31r+wHRUQdCKLY1dIT91ZXVweNRoPa2lr4+flJHQ4RXaW+pQ1HC7U4nGe8HHeiSIvWdkOHMV5KOUb2DzAmTHEBGBEdAE/lDRKgq/pA5fuHY/Kv3oMoyLAzWYe4X9xrxzMiIkdhzec/Z6CIyKFV1rcaa5fyqpFZUI2zl+pg+NHXvn7eSoyONSZMY+MCMSzcDwq5lV1a7rsPuOce4PvvEVtaismXPPBdpR7rfBLwgs3OhohcBRMoInIYoiii4HKTqX7JWMOUV9XYaVx0oKcxWYoNxOjYQAwI9oYgCL0PQC63tCqYk12J71YfwqYjxfjd7QnwVXv0/vGJyGUwgSIiyegNIs6X1eFwXrWlrUBFfWuHMYIADA71NV2OMyZNYRq13WObOCgIA4K9cbGyEZ8cKcac9Di7PycROQ8mUETUZ1ra9DhZXGu5JHe0oAb1rR0Lvj3kApKj/E2X4wIwqn8gNF59P/sjCALmjI/Fc5+fwbr9BXgkLRYymQ1muYjIJTCBIiK7qW1uw9GCGhwyrY47UVQLnb5jwbePSoGRMQEYa6phSon2h9rDMVa83TcyCq9vy0JeVSP2XKjELYNDpA6JiBwEEygisrn8qkYsXH8MJ0tqO20xF+Sjwti4AEtLgSFhvtYXfPcRb5UC94+Oxuq9eVi7L58JFBFZMIEiIpvbkFmEE8W1AIDYfl6WZGlMXCBi+3nZpuC7jzySFoM1+/KwK6sSuZUNiA/2kTokInIATKCIyOYO51UDAJbdm4RfjOsvcTS9ExvkjVsHh2DH+Qp8sL8AL/x0uNQhEZEDcMx5cyJyWuZCcQAYP6CfxNHYxpz0WADAx5lFqG9pkzYYInIITKCIyKZOFhsLxYN8VIjp5yV1ODYxYaCxpUGjTo9PjhRLHQ4ROQAmUERkU4fzjZfvxsYFOFWt0/WYWxoAwLr9BTD8uBU6EbkdJlBEZFPmBGp0TKDEkdjWfSOj4KtSIK+qEbsvVEodDhFJjAkUEdmM3iDiSEENAGBsnGslUN4qBR4YEw0AWLs3X9pgiEhyTKCIyGayyupR39IOb6UcQ8J8pQ7H5h5Ji4EgALuzK3GxskHqcIhIQkygiMhmMguMl+9GxgQ4bHPM3ojpZ2xpAAD/3l8gcTREJCXXe4cjIskcMvV/GhPrWpfvrsaWBkQEMIEiIhsRRdFSQO7KCdSEgUEYGOKDRp0em9jSgMhtMYEiIpsormlGeV0rPOQCUqP9pQ7HbgRBwKPmlgb78tnSgMhNMYEiIpswzz4lRmrgqZRLHI193TciEr5qBfIvN7GlAZGbYgJFRDZxON/YvsCVL9+ZeasUeGA0WxoQuTMmUERkE+5Q/3Q1tjQgcm9MoIio16obdcipMCYRo2MCJI6mb8T088ZtQ4wtDT7Yly9tMETU55hAEVGvZZpmnwaF+CDAWylxNH1nzvg4AMCmI8VsaUDkZphAEVGvZZq2bxntJpfvzNIH9mNLAyI3xQSKiHrN3EBzbJx7XL4zY0sDIvfFBIqIeqVZp8fpkloAwOgY95qBAn7U0iCbLQ2I3AUTKCLqlWNFNWg3iAjXqBEV4Cl1OH3OW6XATFNLgzUsJidyG5InUO+88w5iY2OhVqsxbtw4HDp06LrjtVotMjIyEB4eDpVKhYSEBGzdurXDmJKSEjz00EPo168fPD09kZSUhMzMTHueBpHbysy/Uv8kCILE0UjjkbRYCAKwhy0NiNyGpAnUhg0bsHjxYixduhRHjx5FSkoKpk2bhoqKii7H63Q6TJ06Ffn5+di0aROysrKwatUqREZGWsbU1NQgPT0dHh4e+Oqrr3D27Fm8+eabCAhwr9oMor5ypf+T+/4b69/Piy0NiNyMQsonf+uttzBv3jzMnTsXALBixQps2bIFq1evxjPPPNNp/OrVq1FdXY19+/bBw8MDABAbG9thzGuvvYbo6GisWbPGciwuLs5+J0Hkxtr1BhwtcJ8O5NczZ3wcvj1XgU1HivG7aYPhp/aQOiQisiPJZqB0Oh2OHDmCKVOmXAlGJsOUKVOwf//+Lu/zxRdfIC0tDRkZGQgNDUViYiKWLVsGvV7fYczo0aNx//33IyQkBCNGjMCqVauuG0trayvq6uo63Ijoxs6V1qNRp4evWoGEUF+pw5FUh5YGmWxpQOTqJEugqqqqoNfrERoa2uF4aGgoysrKurxPbm4uNm3aBL1ej61bt+K5557Dm2++iZdffrnDmHfffReDBg3C9u3bMX/+fPz2t7/FunXrrhnL8uXLodFoLLfo6GjbnCSRizNfvhsdEwC5zD3rn8wEQcAcc0uD/WxpQOTqJC8it4bBYEBISAhWrlyJUaNGYebMmXj22WexYsWKDmNGjhyJZcuWYcSIEfjVr36FefPmdRjzY0uWLEFtba3lVlRU1BenQ+T0LAmUm1++M7tvpLGlQcHlJuzK7rqWk4hcg2QJVFBQEORyOcrLyzscLy8vR1hYWJf3CQ8PR0JCAuRyueXY0KFDUVZWBp1OZxkzbNiwDvcbOnQoCgsLrxmLSqWCn59fhxsRXZ8oijhsWoE3No4JFAB4Ka+0NFi7r0DiaIjIniRLoJRKJUaNGoUdO3ZYjhkMBuzYsQNpaWld3ic9PR05OTkwGAyWY9nZ2QgPD4dSqbSMycrK6nC/7OxsxMTE2OEsiNxX/uUmVDW0QimXISlSI3U4DuPqlgbmDZaJyPVIeglv8eLFWLVqFdatW4dz585h/vz5aGxstKzKe+SRR7BkyRLL+Pnz56O6uhoLFy5EdnY2tmzZgmXLliEjI8MyZtGiRThw4ACWLVuGnJwcfPjhh1i5cmWHMUTUe+bLdynRGqg95DcY7T6MLQ2MtZ0f7M+XNhgishtJ2xjMnDkTlZWVeP7551FWVobU1FRs27bNUlheWFgImexKjhcdHY3t27dj0aJFSE5ORmRkJBYuXIinn37aMmbMmDHYvHkzlixZgpdeeglxcXH461//igcffLDPz4/IlR3OY/3TtcxNj8W358qx6Ugxfs+WBkQuSRBFkUtFfqSurg4ajQa1tbWshyK6hslv7EJeVSPWzBmDyaYmkmQkiiJu/8seXKhowHM/GYbHJrAXHZEzsObz36lW4RGRY6iob0FeVSMEARgZ474dyK9FEAQ8ampp8AFbGhC5JCZQRGS1I6bVd4NDfaHx5OWprrClAZFrYwJFRFY7ZNn/jvVP1+KlVGDWGGNLgzV786UNhohsjgkUEVkt0zQDNYb9n67L3NLg+wtVyKmolzocIrIhJlBEZJWG1nacuVQLABgTy/qn64kOvNLSYB0baxK5FCZQRGSVY4U1MIhAVIAnwjWeUofj8OamxwIAPjlajLqWNmmDISKbYQJFRFYx939i/VP3jB/QD4NCfNCk0+PjzGKpwyEiG2ECRURWMe9/xwSqewRBwBzTLNS6ffnQs6UBkUtgAkVE3aZrN+BYkTmBYv1Td907IhJ+agUKq5uwK4stDYhcARMoIuq2M5dq0dJmQICXBwaG+EgdjtPwUiow09TSYO2+fGmDISKbYAJFRN1mbl8wKiYQgiBIHI1zYUsDItfCBIqIus3cQHNsHC/fWSs60AtThrKlAZGrYAJFRN0iiiIyTQnUaBaQ98hc0/54nxwtRm0zWxoQOTMmUETULRcrG1DT1Aa1hwyJERqpw3FKaQP6ISHU3NKgSOpwiKgXmEARUbeY2xekRvtDqeBbR08IgoBHTbNQH+wvYEsDIifGd0Ei6hZzA82xvHzXK2xpQOQamEARUbccLmD9ky14KRWYNbY/ALY0IHJmTKCI6IbKaltQVN0MmQCMjOEKvN56+KYYyEwtDS6Us6UBkTNiAkVEN3TYtPpuWIQffFQKiaNxfh1aGuzPlzYYIuoRJlBEdEPmBGp0DC/f2cocUzH5p0dL2NKAyAkxgSKiGzKvwBsbxwTKVtjSgMi5MYEiouuqbW7D+bI6AMBobiBsM4IgYM74OABsaUDkjJhAEdF1HS2sgSgCsf28EOKrljoclzJjRAQ0nh4orG7CzvNsaUDkTJhAEdF1mfs/sX2B7XkpFZg1JhoAWxoQORsmUER0XZnm+icmUHbxkKmlwQ85bGlA5EyYQBHRNbW263G8WAuA9U/2wpYGRM6JCRQRXdOp4lro2g0I8lEiLshb6nBc1pz0WADAJ0fY0oDIWTCBIqJrOnRV/ydBECSOxnWlxffD4FBfNLexpQGRs2ACRUTXZK5/GsP+T3YlCIJlFmrd/ny2NCByAkygiKhLBoOITNMM1BjWP9ndjNRIaDw9UFTdzJYGRE6ACRQRdSm7oh51Le3wUsoxLNxP6nBcnqdSzpYGRE6ECRQRdcnc/2lk/wAo5Hyr6AtsaUDkPPiuSERdMu9/N4b9n/pMdKAXpg4ztjTgLBSRY2MCRUSdiKKIw6x/koR5f7xPj5agtoktDYgcFRMoIuqkRNuM0toWKGQCUvv7Sx2OW7kpPvBKS4MjbGlA5KiYQBFRJ+bZp+GRGngpFRJH417Y0oDIOTCBIqJODlv2v+PlOylc3dLgO7Y0IHJITKCIqBPzCrzRLCCXhKdSjlljzS0N8iSOhoi6wgSKiDqoadThQkUDAGB0DGegpPKwqaXB3pzLbGlA5ICYQBFRB0cKjJfvBgR7o5+PSuJo3FdUAFsaEDkyJlBE1IG5gHws97+THFsaEDkuJlBE1IE5gRodwwRKajfFB2JImLGlwcZMtjQgciRMoIjIoqVNj1MltQA4A+UIBEHAnPGxANjSgMjRMIEiIovjRVq06UWE+qkQFeApdTgE4B5TS4PiGrY0IHIkTKCIyMLcvmBMbCAEQZA4GgLY0oDIUTGBIiKLwwXcQNgRXd3SIJstDYgcAhMoIgIA6A0ijjKBckhRAV64fVgYALY0IHIUTKCICABwrrQODa3t8FUpMDjMV+pw6EfM++NtZksDIofABIqIAFxpXzAyJgByGeufHM24OLY0IHIkTKCICACQad5AmO0LHBJbGhA5FiZQRARRFHHI0kCT+985qntSI+HvZWxpsONcudThELk1JlBEhMLqJlTWt0IplyEl2l/qcOgaPJVyzBrTHwCLyYmkxgSKiHDI1P8pKUoDtYdc4mjoeh5OM7Y02HfxMrLK2NKASCpMoIjIUv/E9gWOL9Lf09LSYN3+fGmDIXJjTKCIyLICb0ws65+cgbmlwadHi9nSgEgiTKCI3FxVQytyqxoBAKNjOAPlDMwtDVraDNiQWSh1OERuiQkUkZvLNM0+DQ71hcbLQ+JoqDsEQcBc0yzUun0FbGlAJAEmUERu7rC5/imOl++cibmlQYm2Gd+ypQFRn3OIBOqdd95BbGws1Go1xo0bh0OHDl13vFarRUZGBsLDw6FSqZCQkICtW7dafv7CCy9AEIQOtyFDhtj7NIic0pX6J16+cyZqjystDdaxpQFRn5M8gdqwYQMWL16MpUuX4ujRo0hJScG0adNQUVHR5XidToepU6ciPz8fmzZtQlZWFlatWoXIyMgO44YPH47S0lLL7YcffuiL0yFyKo2t7ThzqQ4AEyhnxJYGRNKRPIF66623MG/ePMydOxfDhg3DihUr4OXlhdWrV3c5fvXq1aiursZnn32G9PR0xMbGYtKkSUhJSekwTqFQICwszHILCgrqi9MhcirHCrXQG0RE+nsiwt9T6nDISpH+npg23NjSgI01ifqWpAmUTqfDkSNHMGXKFMsxmUyGKVOmYP/+/V3e54svvkBaWhoyMjIQGhqKxMRELFu2DHq9vsO4CxcuICIiAvHx8XjwwQdRWHjtlSqtra2oq6vrcCNyB2xf4PzM++NtPlYMbZNO2mCI3IikCVRVVRX0ej1CQ0M7HA8NDUVZWVmX98nNzcWmTZug1+uxdetWPPfcc3jzzTfx8ssvW8aMGzcOa9euxbZt2/Duu+8iLy8PEydORH1911Pcy5cvh0ajsdyio6Ntd5JEDsycQI3m5TunNTYuEEPD/YwtDQ4XSR0OkduQ/BKetQwGA0JCQrBy5UqMGjUKM2fOxLPPPosVK1ZYxtx55524//77kZycjGnTpmHr1q3QarXYuHFjl4+5ZMkS1NbWWm5FRXwTItfXpjfgWKEWgPFDmJyTIAiYMz4GAPDBfrY0IOorkiZQQUFBkMvlKC/vuAS3vLwcYWFhXd4nPDwcCQkJkMuv7Nc1dOhQlJWVQafrevra398fCQkJyMnJ6fLnKpUKfn5+HW5Eru7MpTo0t+mh8fTAwGAfqcOhXmBLA6K+J2kCpVQqMWrUKOzYscNyzGAwYMeOHUhLS+vyPunp6cjJyYHBYLAcy87ORnh4OJRKZZf3aWhowMWLFxEeHm7bEyByYplX1T/JZILE0VBvqD3kmD3W2NKAl/GI+obkl/AWL16MVatWYd26dTh37hzmz5+PxsZGzJ07FwDwyCOPYMmSJZbx8+fPR3V1NRYuXIjs7Gxs2bIFy5YtQ0ZGhmXM73//e+zevRv5+fnYt28f7r33XsjlcsyePbvPz4/IUR3KY/2TK7kz0Thrf6SgBqLIy3hE9qaQOoCZM2eisrISzz//PMrKypCamopt27ZZCssLCwshk13J86Kjo7F9+3YsWrQIycnJiIyMxMKFC/H0009bxhQXF2P27Nm4fPkygoODMWHCBBw4cADBwcF9fn5EjkgURWQWmDqQM4FyCUPC/KBUyFDb3Ib8y02IC/KWOiQilyaI/KrSSV1dHTQaDWpra1kPRS7pYmUDbntzN1QKGU69MA1KheST0WQD9/1zL44WavGXmSm4d0SU1OEQOR1rPv/5rknkhg6bLt+lRvszeXIhqdHGfl7HTasrich++M5J5IYsGwjz8p1LSe3vDwA4XqSVNA4id8AEisgNWTqQs/+TSxkR7Q8AOFtah5Y2/fUHE1GvMIEicjPldS0orG6CTABGmmYsyDVEBXiin7cSbXoRZ0u5JRWRPTGBInIz5tmnoeF+8FV7SBwN2ZIgCEg1zUKxDorIvphAEbmZTNY/ubQUcwLFOigiu2ICReRmzA00mUC5plQmUER9ggkUkRupa2nD+TJjbczo2ACJoyF7MM9AFVY3obqx6/1Biaj3mEARuZGjBTUwiED/QC+E+qmlDofsQOPpgfhgYxfyE5yFIrIbJlBEboT1T+7BfBnvGBMoIrthAkXkRg6Z+z/x8p1LG8E6KCK7YwLVxzLzq/HOzhycvcQeLdS3Wtv1lks6bKDp2sxbupwo0oLbnRLZh0LqANzNmr352HKqFIIADIvgRsXUd06X1KK13YB+3krEB3lLHQ7Z0ZBwX6gUMtQ2tyGvqhHxwT5Sh0TkcjgD1ceSozQAgJNFtRJHQu7GvP/d6NgACIIgcTRkTx5yGRIjje81vIxHZB9MoPpYcpQ/AOBksVbSOMj9HGb/J7fCflBE9sUEqo8lRWkgCMCl2hZU1rdKHQ65CYNBRGYBV+C5E3YkJ7IvJlB9zEelwABTPQJnoaivXKhoQG1zGzw95Ky9cxPmlXjnSuvQ0qaXNhgiF8QESgLmOqgTxayDor5h3kB4ZIw/POT8Z+8OogI80c9biTa9iLOlXPVLZGt8J5WAuTaBM1DUV8wJ1OgYXr5zF4IgXKmDKtRKGguRK2ICJYErheS17NFCfcLcgXws+z+5FRaSE9kPEygJDA33hYdcQHWjDsU1zVKHQy6uRNuMEm0z5LIrMxLkHlL7+wNgAkVkD0ygJKBSyDEkzFjIe5J1UGRnmabLd4kRfvBWsXeuOzHPdhdWN+FyA1f9EtkSEyiJWBpqsg6K7OyQqf/TaLYvcDsaTw8MCDZ2nT/B9xoim2ICJZEU0zdDvqmRvZnrn9j/yT2Z98VjITmRbTGBkkhytHEG6nRJHQwGFpKTfWibdMgqrwdg3MKF3E+q6b3mGOugiGyKCZREBgb7wNNDjobWduRWNUgdDrmoI6bu4/HB3gjyUUkcDUnBPAN1okjLL2tENsQESiIKuQyJkcZC8hPcWJjs5JCpgHwM+z+5rSHhvlApZKhraUfe5UapwyFyGUygJMSNhcneLPVP7P/ktjzkMiRGmnY/4GU8IpthAiUhbulC9tTSprck52NY/+TW2FCTyPaYQEnIvBLvbGkddO0GaYMhl3OiSIs2vYgQXxX6B3pJHQ5JiAkUke0xgZJQTD8vaDw9oGs3INu0UorIVjILrrQvEARB4mhISuYE6lxpHVra9NIGQ+QimEBJSBCEqy7jaaUNhlyOuYEmL99RVIAngnyUaNOLOHOpTupwiFwCEyiJWTqScyUe2ZDeIOKoaQaKHchJEARexiOysR4lUDt37rR1HG4rmR3JyQ7Ol9WhvrUdPioFhob7SR0OOQAmUES21aME6o477sCAAQPw8ssvo6ioyNYxuRVzIfmFigY06dqlDYZchrl9wciYAMhlrH8iIMWSQNVIGwiRi+hRAlVSUoIFCxZg06ZNiI+Px7Rp07Bx40bodDpbx+fywjRqhPiqoDewNoFsx9xAcyzrn8jEPNtdVN2Myw2t0gZD5AJ6lEAFBQVh0aJFOH78OA4ePIiEhAQ8+eSTiIiIwG9/+1ucOHHC1nG6NMtlPE6tkw2IoohMUwLF+icy03h6YECwNwCWDBDZQq+LyEeOHIklS5ZgwYIFaGhowOrVqzFq1ChMnDgRZ86csUWMLi/FXEjOhppkA0XVzSiva4WH/ErhMBFwZV+844VaaQMhcgE9TqDa2tqwadMm3HXXXYiJicH27dvx9ttvo7y8HDk5OYiJicH9999vy1hdVrLpQ45bupAtHDbNPiVFaqD2kEscDTmS1P7+AIBjnO0m6jVFT+70m9/8Bh999BFEUcTDDz+M119/HYmJiZafe3t744033kBERITNAnVlyaZ9qvIvN6G2qQ0aLw+JIyJnZk6gxvDyHf3ICNOXtRNFWhgMImRcYOC2vj5Thk1HivHyjESE+KmlDscp9SiBOnv2LP7xj3/gvvvug0ql6nJMUFAQ2x10U4C3Ev0DvVBY3YSTJVpMHBQsdUjkxJhA0bUMDvOFSiFDXUs78i43YkCwj9QhkQTK61qweOMJNLS2I1yjxov3JN74TtRJjy7h7dixA7Nnz75m8gQACoUCkyZN6nFg7iaZdVBkA5cbWnGxshEAMCqGK/CoIw+5DEmmGW/WQbmvl748i4ZWY9ucj48Uo7a5TeKInFOPEqjly5dj9erVnY6vXr0ar732Wq+DckcpXIlHNmDe/y4h1AcB3kqJoyFHxIaa7m1XVgW2nCyFXCYgXKNGk06PjzPZz7EnepRAvffeexgyZEin48OHD8eKFSt6HZQ74gwU2cLhPLYvoOszF5IzgXI/LW16PP+5cXX83PGx+M2tgwAAa/flQ28QpQzNKfUogSorK0N4eHin48HBwSgtLe11UO4oMVIDmQCU1bWgoq5F6nDISR02zUCNZQJF12Ce7T5XWoeWNr20wVCfevu7HBRWNyFco8ZTUxNw74hI+Ht5oLimGd+eK5c6PKfTowQqOjoae/fu7XR87969XHnXQ94qBQaGGAs6T3AWinqgSdeOMyXG353R7EBO1xAV4IkgHyXaDSLOXOJ7jbvIqajHe3suAgCWTh8OH5UCnko5Zo/tDwBYszdPyvCcUo8SqHnz5uGpp57CmjVrUFBQgIKCAqxevRqLFi3CvHnzbB2j2zB3JGc/KOqJ44VatBtERGjUiArwkjocclCCIFxVB8UEyh2IoohnN59Gm17EbUNCMG14qOVnD98UA7lMwIHcaibUVupRG4M//OEPuHz5Mp588knL/ndqtRpPP/00lixZYtMA3UlKtD82HSnmDBT1yCFu30LdlBrtj2/PVbAOyk18erQEB/OqofaQ4YWfDocgXOn/FeHviTsSw7DlZCnW7s3Hn+9PkTBS59KjGShBEPDaa6+hsrISBw4cwIkTJ1BdXY3nn3/e1vG5lStbumghiizoI+tk5hvrn8bEMYGi67Ns6VJUI3EkZG/aJh1e2XoOALDwtgREB3aenf5leiwA4PMTl7jRtBV6tReej48PxowZg8TExOv2hKLuGRLmB6VcBm1TG4qqm6UOh5xIu96Ao4WmBIr1T3QDydEaCIJx30R+YLq217adR3WjDgmhPnh8YlyXY0b2D0BKlAa6dgM+PFjYxxE6rx5dwgOAzMxMbNy4EYWFhZbLeGaffvpprwNzR0qFDEPDfXGiuBYnirXo3491LNQ9Z0vr0KTTw0+tQEKIr9ThkIPzU3tgQLAPcioacLxIi9uGht74TuR0jhRU46NDxh5Pr9ybBA9513MmgiBgbnocntpwHP8+UIBfTxoApaJX8ytuoUd/Q+vXr8f48eNx7tw5bN68GW1tbThz5gy+++47aDQaW8foVlhITj1x6Kr+T9zfjLqDDTVdW5vegGc3nwYAzBwdfcOtne5KCkeIrwoV9a3YeortiLqjRwnUsmXL8Je//AX/+9//oFQq8be//Q3nz5/HAw88gP79+9s6RrdibqjJQnKyhqX+iQXk1E1MoFzb6h/ycL6sHgFeHnjmzs6Nr39MqZDhoZtiABhbGrAO98Z6lEBdvHgRd999NwBAqVSisbERgiBg0aJFWLlypU0DdDcppje10yW17AxL3SKK4lUbCLP+ibrn6gTKwPcal1Jc04S/fnsBAPDHu4Z2e1unX4zrD6VchhPFtTjKvRJvqEcJVEBAAOrr6wEAkZGROH3aOE2o1WrR1NRku+jc0IBgH3gp5WjS6XGxskHqcMgJ5FU14nKjDkqFDElRvIRO3TM4zBcqhQz1Le3IrWqUOhyyoRe+OIvmNj3GxgXi56Oiun2/IB8V7kk1NsNmY80b61ECdfPNN+Obb74BANx///1YuHAh5s2bh9mzZ+O2226zaYDuRi4TkGjaLZ0bC1N3mC/fpUb5Q6WQSxwNOQsPuQxJpvcaXsZzHV+fKcO358qhkAl4ZUZih55P3TE33bhS76vTZSit5Wrw6+lRAvX2229j1qxZAIBnn30WixcvRnl5OX72s5/h/ffft2mA7iiFGwuTFcwNNMfE8fIdWcd8GY9f1lxDY2s7XvjCuFnwr26Ox6BQ61fkDovww7i4QOgNIj7YX2DrEF2K1W0M2tvb8eWXX2LatGkAAJlMhmeeecbmgbkzrsQja2SyAzn1UGp/fwCcgXIVf9txAZdqWxAd6Inf3Dqox48zNz0OB/Oq8dGhQvz21kHwVHJmuytWz0ApFAo88cQTaGlpsVkQ77zzDmJjY6FWqzFu3DgcOnTouuO1Wi0yMjIQHh4OlUqFhIQEbN26tcuxr776KgRBwFNPPWWzeO3tym7p9dC1G6QNhhxaRX0L8i83QRCAUTGcgSLrmGegzpXWoaVNL20w1CvnSuvw/g/GuqWXfprYq6Rn6rBQRAV4QtvUhs+Ol9gqRJfTo0t4Y8eOxfHjx20SwIYNG7B48WIsXboUR48eRUpKCqZNm4aKiooux+t0OkydOhX5+fnYtGkTsrKysGrVKkRGRnYae/jwYbz33ntITk62Sax9JTrQEwFeHtDpDThfVid1OOTAzPVPQ8L84Kf2kDgacjaR/p4I8lGh3SByI1knZjCI+OPmU9AbRNyVFIbJQ0J69XhymYA542MBsKXB9fQogXryySexePFivP3229i/fz9OnjzZ4WaNt956C/PmzcPcuXMxbNgwrFixAl5eXli9enWX41evXo3q6mp89tlnSE9PR2xsLCZNmoSUlI4bIDY0NODBBx/EqlWrEBDgXN/MBUFAkmkWirUJdD3mBppj2b6AekAQBMss1DEuW3da6w8X4VihFt5KOZ7/yXCbPOb9o6PhpZQju7wBe3Mu2+QxXU2PEqhZs2YhLy8Pv/3tb5Geno7U1FSMGDHC8t/u0ul0OHLkCKZMmXIlIJkMU6ZMwf79+7u8zxdffIG0tDRkZGQgNDQUiYmJWLZsGfT6jtPPGRkZuPvuuzs89rW0trairq6uw01qKWyoSd2QWcD6J+qdEayDcmpVDa149SvjZsG/u30wwjRqmzyuxtPD0gKBLQ261qO98PLybPOXWVVVBb1ej9DQjvswhYaG4vz5813eJzc3F9999x0efPBBbN26FTk5OXjyySfR1taGpUuXAjBuNXP06FEcPny4W3EsX74cL774Yu9OxsZYSE43Ut/ShrOXjMk+O5BTT7EjuXNbtuUc6lraMTzCD4+kxdj0sR8dH4sP9hfgu6wK5Fc1IjbI26aP7+x6lEDFxNj2RbKGwWBASEgIVq5cCblcjlGjRqGkpAR//vOfsXTpUhQVFWHhwoX45ptvoFZ3LxNfsmQJFi9ebPlzXV0doqOj7XUK3WKegcqpaEBjazu8VT3e95lc1LFCLQyisWbOVt86yf0kRWkgCEBxTTOqGloR5KOSOiTqpn0Xq/DpsRIIgnGzYMU1NgvuqQHBPpg8OBg7syqxdl8+XvipbS4PuooefSp/8MEH1/35I4880q3HCQoKglwuR3l5eYfj5eXlCAsL6/I+4eHh8PDwgFx+ZYXB0KFDUVZWZrkkWFFRgZEjR1p+rtfrsWfPHrz99ttobW3tcF8AUKlUUKkc600jxE+NMD81yupacLqkFuPi+0kdEjmYK9u3cPaJes5P7YEBwT7IqWjA8UItpgwLvfGdSHKt7Xr832fGXUAeGhdjmUm0tbnpcdiZVYlNR4rxu9sT4MvFKhY9SqAWLlzY4c9tbW1oamqCUqmEl5dXtxMopVKJUaNGYceOHZgxYwYA4wzTjh07sGDBgi7vk56ejg8//BAGgwEymTHbzs7ORnh4OJRKJW677TacOnWqw33mzp2LIUOG4Omnn+6UPDmy5CgNys624GQxEyjqjAkU2UpqtD9yKhpwopgJlLNYuTsXuZWNCPZV4Q93DLbb80wcFISBIcYEe2NmMR6bEGe353I2PZrvq6mp6XBraGhAVlYWJkyYgI8++siqx1q8eDFWrVqFdevW4dy5c5g/fz4aGxsxd+5cAMbZrCVLlljGz58/H9XV1Vi4cCGys7OxZcsWLFu2DBkZGQAAX19fJCYmdrh5e3ujX79+SExM7MnpSsa8sfAJ1kHRj+jaDZZVU9xAmHqLdVDOJb+qEf/YmQMAeO4nw+zawkQQrrQ0WLcvn5vcX8VmhTWDBg3Cq6++ioceeuiaBeBdmTlzJiorK/H888+jrKwMqamp2LZtm6WwvLCw0DLTBADR0dHYvn07Fi1ahOTkZERGRmLhwoV4+umnbXUqDiOZW7rQNZy+VIvWdgMCvIyXX4h64+oEymAQIZNZt38a9R1RFPHc56ehazdg4qAgTE8Ot/tz3jcyEq9vO4/C6iZ8d74CUzlLCcCGCRRg7FJ+6dIlq++3YMGCa16y27VrV6djaWlpOHDgQLcfv6vHcAbJkf4AgMLqJtQ06hDgrZQ2IHIYh/OutC+wdrNQoh8bEuYLtYcM9S3tyK1qxMAQJuWO6suTpfj+QhWUChleusf6zYJ7wkupwOxx/fHe7lys2ZvHBMqkRwnUF1980eHPoiiitLQUb7/9NtLT020SGAEaLw/E9vNC/uUmnCypxaSEYKlDIgdx2NSBfCzrn8gGFHIZkiI1OJxfg+NFWiZQDqqupQ0vfXkWAJBxy0DE9WFbgUfSYvGv7/Ow7+JlnC+rw5Awvz57bkfVowTKXPBtJggCgoODceutt+LNN9+0RVxkkhzlb0ygirRMoAiAcduGKw00Wf9EtpEa7W9KoGosDRTJsby5PQuV9a2ID/LGE7fE9+lzR/p7YtrwUGw9VYY1P+TjtZ871xZp9tCjInKDwdDhptfrUVZWhg8//BDh4fa/HutOktmRnH7kYmUDtE1tUHvIkBipkTocchGp0cZknIXkjulksRYfHCgAAPxpRiJUir5fUT433bgC77PjJahu1PX58zsa23bdIpszr8RjR3IyO2RqXzAiOgAeNm6cR+4r1bSly/nSerS06a8/mPqU3rRZsCgCM1IjkD4wSJI4RscEIClSg9Z2Az46VChJDI6kR+++P/vZz/Daa691Ov7666/j/vvv73VQdMXwCD/IBKCivhVltS1Sh0MOINNU/zQmjvVPZDsRGjWCfVVoN4g4XcIZb0fywf58nC6pg59agWfvHiZZHIIgYG56LADg3/sL0KY3SBaLI+hRArVnzx7cddddnY7feeed2LNnT6+Doiu8lAokhPoCYD8oMjqUZ26gyfonsh1BEJBi2oOTl/EcR1ltC978OhsA8PSdQxDsK+2uGXcnhyPIR4WyuhZ8dbpM0lik1qMEqqGhAUpl5yX1Hh4eqKur63VQ1NGVflBaaQMhyV3SNqNE2wy5TMCI/kygyLZGmC7jMYFyHH/68iwaWtsxor8/Zo/pL3U4UCnkeOgmYxyrf8iTOBpp9SiBSkpKwoYNGzodX79+PYYNk2560VVdqYPitLq7M2/fMizcDz7cYJpsjB3JHcvOrApsOVUKuUzAKzOSHKbB6YPjYqCUy3C8SItjhTVShyOZHr0DP/fcc7jvvvtw8eJF3HrrrQCAHTt24KOPPsLHH39s0wAJlmn1k8W1EEWRjRPdmKX+if2fyA6SozQQBKC4phlVDa0I8nGsTdbdSUubHs9/btwseO74WAyLcJy+S8G+KkxPicAnR4uxZm++286G92gGavr06fjss8+Qk5ODJ598Er/73e9QXFyMb7/9tlOPKOq9wWG+UCpkqG1uQ8HlJqnDIQld2UDYPd+wyL581R4YaNoa6Lhpr0WSxtvf5aCouhnhGjWempogdTidmIvJt54qddsFTj1eA3333Xdj7969aGxsRFVVFb777jtMmjTJlrGRiYdchmHhxm8fLCR3X7VNbcgqrwdg3MKFyB54GU96ORX1eG/PRQDA0unDHfJyfWKkBmNjA9FuEPHvA/lShyOJHiVQhw8fxsGDBzsdP3jwIDIzM3sdFHWWwo2F3d6RwmqIIhAX5C35ShxyXaksJJeUKIp4dvNptOlFTBkagmnDHXffOfMs1IcHC92yd1iPEqiMjAwUFRV1Ol5SUoKMjIxeB0WdJVvqoLSSxkHSOWypf+LlO7If8wzUiSItDAZR2mDc0CdHS3AwrxqeHnK88NPhDl3zOnVYKCL9PVHT1IbPj5dIHU6f61ECdfbsWYwcObLT8REjRuDs2bO9Doo6S4k2zkCdLqlDu5s3L3NXh/PM+9/x8h3Zz+BQX3h6yFHf2o7cqgapw3ErNY06LNt6DgCwcMogRAV4SRzR9SnkMjw6PgYAsGZvPkTRvRLuHiVQKpUK5eXlnY6XlpZCoXC8a7WuID7IBz4qBZrb9Mip5Juau2lp01su345lAkV2pJDLkGTaY/EYC8n71GvbzqO6UYeEUB88NiFO6nC6Zebo/vD0kON8WT32516WOpw+1aME6vbbb8eSJUtQW3ulHker1eKPf/wjpk6darPg6AqZTEBipLGQ/GQR66DczamSWuj0BgT5qBDTz7G/lZLzM894sw6q72TmV2P9YWNpzCv3JjnNPpcaLw/8bFQkAGD1D/nSBtPHevQKvfHGGygqKkJMTAwmT56MyZMnIy4uDmVlZXjzzTdtHSOZmPtBcSWe+zFv3zI2LsChayLINaRGG+vs+F7TN9r0Bjy72djzaeboaKfr8zZnvHG2bMf5chRcbpQ4mr7TowQqMjISJ0+exOuvv45hw4Zh1KhR+Nvf/oZTp04hOjra1jGSSTITKLeVaer/NDrGud5YyTmZV+KdL613y9VVfW31D3nIKq9HoLcSz9w5ROpwrDYwxAc3JwRDFIF1+wqkDqfP9HiO0NvbGxMmTMD06dNx8803w9/fH1999RW++OILW8ZHVzHvicc3NfeiN4jILDCuwBsbxwSK7C9Co0awrwrtBhGnS1gyYE/FNU3467cXAAB/vGsoArw77zPrDH5pamnwcWYRGlrbpQ2mj/So4js3Nxf33nsvTp06BUEQOm0votfzw90eogI8EeitRHWjDudK69y2fb67yS6vR31LO7yVcgwJ85U6HHIDgiAgNdof35wtx/EiLVd+2okoinjhizNobtNjXFwgfjYyUuqQeuzmQcGID/ZGbmUjNmUWYU66cxTB90aPZqAWLlyIuLg4VFRUwMvLC6dPn8bu3bsxevRo7Nq1y8YhkpkgCJZZKDbUdB/m7VtGxgRA4SSFpeT8zP2gjrGQ3G6+PluOb89VwEMu4JV7E526vlEmEzB3fCwAYO2+fLfoIdajd+P9+/fjpZdeQlBQEGQyGeRyOSZMmIDly5fjt7/9ra1jpKuwDsr9HOYGwiSBEeYtXdjKwC4aW9vxwhdnAAC/ujkeA0Ocf3b5vpFR8FUrkH+5CTuzKqQOx+56lEDp9Xr4+hpf7KCgIFy6dAkAEBMTg6ysLNtFR51wSxf3IoqipYEmEyjqS0lRGggCUKJtRmV9q9ThuJy/fpuN0toWRAd6YsHkQVKHYxPeKgVmj+0PwNhY09X1KIFKTEzEiRMnAADjxo3D66+/jr179+Kll15CfHy8TQOkjswzUBcrG9ymUM+dFdc0o6yuBR5ywXJJhagv+Ko9MCjEBwD7Qdna2Ut1WG1KMF76aSI8lXJpA7KhR9JiIBOAH3KqkG3a/NxV9SiB+r//+z8YDMbtRF566SXk5eVh4sSJ2Lp1K/7+97/bNEDqKNhXhQiNGqIInOIslMsz1z8lRmpc6k2WnIM5aT9eVCNtIC7EYBDx7GenoDeIuCspDJOHhEgdkk1FBXjh9mFhAFx/FqpHCdS0adNw3333AQAGDhyI8+fPo6qqChUVFbj11lttGiB1xo2F3Qfrn0hKKZYESitpHK5k/eEiHCvUwkelwPM/GS51OHYx19TS4NOjxahp1EkbjB3ZbElPYGCgU68gcCbJ0ayDcgdnL9VhT3YlAGB0DFtWUN8zz0CdLKp1i1VV9lZZ34pXvzJuFvy72xMQplFLHJF9jI0LxLBwP7S2G/DR4UKpw7Ebrol2QtzSxXWJooi9OVV4+P2DuOvv36NE2wyNpwcbaJIkBof6wtNDjvrWduRWcRPz3lq29RzqWtqRGOmHh2+KkTocuxEEAb80bYb87/0FaNMbJI7IPphAOaFE007pxTXNuNzA1TGuoF1vwP9OXML0t3/Ag/86iO8vVEEmANNTIrDx12nw93LO7sTk3BRyGZJM7zfH2M6gV/blVGHzsRIIAvDKjCSX7+k2PSUcQT5KlNa2YPuZMqnDsQvXfgVdlMbTA/FB3gCAk9xmwak16/T4YH8+Jr+5C7/56BhOl9RB7SHDnPGx2P2HyfjH7BEYzO7jJCHzvnisg+q51nY9/u8z42bBD98UY6ktc2UqhRy/GGecZVv9Q57E0dhHj7ZyIeklR2mQW9WIk0W1mDzYtVZxuIPqRh3W7cvHB/vzUdPUBgAI9Fbi0bRYPJwWg0An3Q+LXE8qC8l77b3ducitakSwrwq/nzZY6nD6zEM39ce7u3JwtFCLE0Val0scmUA5qeQof3x2/BJX4jmZwstN+NcPudiYWYSWNmNdQP9AL8ybGIefj4pmqwJyOOYE6nxZPZp1ev6OWim/qhFv78wBADz3k2HwU3tIHFHfCfFV4yfJEdh8rARr9ubhr7NGSB2STTGBclIpppV4J4prO23mTI7nVHEt3ttzEVtPlcK8mCkpUoNfT4rHHcPDXL4egpxXuEaNEF8VKupbcfpSLVtqWEEURTz3+Wno2g2YOCgI05PDpQ6pz/0yPQ6bj5Vgy6lS/PGuoQjxc52Vh0ygnNSwcA3kMgFVDa0orW1BhL+n1CHRj4iiiD0XqrByz0XszblsOT4pIRi/nhSPtPh+THzJ4QmCsQv+12fLcbxQywTKCv87WYrvL1RBqZDhT/c492bBPZUUpcHomABkFtTgPwcKsPh217mEyQTKSXkq5UgI9cW50jqcLNYygXIgbXoDtpwsxXt7cnGutA4AIJcJ+GlKBOZNjMewCD+JIySyTmp/UwLFOqhuq21uw5++PAsAWDB5IGJNC3/c0dz0OGQW1OC/Bwvx5OSBUHu4xmVgJlBOLCVKg3OldThRXIs7Et1vatjRNLa2Y8PhIrz/Qx5KtM0AAC+lHLPG9McvJ8QiKsBL4giJeibV1HuOCVT3vfl1FirrWxEf5I1fT3LvPWKnDQ9FhEaNS7Ut+OLEJTwwOlrqkGyCCZQTS4n2x/rDRSwkl1hlfSvW7cvHvw8UoLbZuKIuyEeJOeNj8dBNMezhRE4vKUoDQQBKtM2orG9FsK9K6pAc2okiLf59oAAA8PKMRKgUrjHj0lMKuQyPjI/Fq1+dx5q9+bh/VJRLXM5kAuXEkqOubOliMIiQyZz/F9KZ5FU1YtX3udh0pBi6duOKurggb8ybGI/7Rka6zDQ1ka/aA4NCfJBd3oDjRVpMHRYqdUgOq11vwB83n4IoAveOiMT4gUFSh+QQZo2Jxl+/zca50joczKvGTfH9pA6p15hAObGEUF+oFDLUt7Qj/3Ij4oN9pA7JLRwrrMHKPbnYdqYMomlFXUq0P+ZPisfUYWGQM5ElF5Qa7W9KoGqYQF3Hvw8U4MylOvipFfjjXUOlDsdh+Hspcd/IKHx4sBBr9uYxgSJpechlGB7hh6OFWpwsrmUCZUcGg4hd2RVYsTsXh/KqLcdvHRKCX98cj7Fx3EybXFtqdAA2ZhazDuo6ympb8ObX2QCAp+8cwkudPzJ3fCw+PFiIr8+Wo6i6CdGBzl0XygTKySVH+Ru7vBZrMWNEpNThuBxduwFfnLiElXsuIrvcuJmqh1zAPamR+NXN8UgI5TYr5B7MDTVPFrFk4Fpe+vIMGlrbMaK/P2aP6S91OA5nUKgvJg4KwvcXqrBuXz7+7yfDpA6pV5hAOTlzQ82TxdwTz5bqW9qw/pBxRV1ZXQsAwEelwC/G9cfc9FiEa9g2gtxLQqgPPD3kqG9tx8XKBgzil4cOdp6vwNZTZZDLBLwyI4kJ5jX8Mj0O31+owobMIiyamgBvlfOmIc4bOQEwzkABwJlLtWjTG+DBjta9UlHXgjX78vGfAwWob2kHAAT7qvDL9Dj8Ylx/aDzdZxsGoqsp5DIkRWlwKK8ax4q0TKCu0qzT4/kvjJsF/zI9lr3ermNSQjDig7yRW9WIT44W45G0WKlD6jF+2jq5uH7e8FUp0NJmQHZ5vdThOK2cigY8vekkJry2E+/uuoj6lnYMCPbGaz9Lwg9PT8b8WwYweSK3N4IbC3fp7Z0XUFTdjHCNGk9NSZA6HIcmkwl4dHwsAGDt3nwYzHtbOSHOQDk5mUxAUpQG+y5exsniWgyP0EgdklM5UlCNFbtz8c3Zcsux0TEB+PWkAbhtSAin4YmuYq6DOl6olTQOR3KhvB4r9+QCAF746XCnviTVV342KgpvbM9CblUjdmdXYvKQEKlD6hG+0i4gOcrflEBpMXssCxdvxGAQseN8Bd7bfRGZBTWW41OHheLXN8djNPf6IupSiimByiqvR7NOD0+le/c6E0URz352Gm16EVOGhuB2tnfoFh+VAg+Micb7P+Rh9d48JlAknRRTQ80TRSwkv57Wdj0+O1aClXtycbGyEQCglMtw74hIzLs5HgND2AaC6HrCNWqE+KpQUd+K05dq3X5j4e1nynEorxqeHnK88NPhbGVihTnjY7Fmbx6+v1CFnIp6DAxxvpo6JlAuIPmqb4UtbXp2wP6RNr0B//o+D2v25qGivhUA4KtW4KGbYjB3fCxC/NQSR0jkHARBQGq0aWPhQq1bJ1CiKOKfu3IAgHtd9kB0oBemDA3F12fLsWZvPl65N0nqkKzGInIXEKFRI8hHCb1BxJlLdVKH43D+seMCXtt2HhX1rQjzU+PZu4Zi3zO34uk7hjB5IrJSan9/ACwk35tjrDtVe8jwy/Q4qcNxSnNNf2+fHi2BtkkncTTWYwLlAgRBsLQz4MbCHbXpDfjwUBEA4A/TBmPP/5uMeTfHw1fNFXVEPZHKlXgAgHd2GmefZo3pj34+7DjeEzfFB2JImC+a2/RYf7hI6nCsxgTKRVy9sTBd8d35ClQ1tCLIR4lf3RwPpYK/8kS9kRzlD0EASrTNqKhvkTocSRwrrMH+3MtQyATMuzle6nCcliAI+OUE4yzUB/vy0a43SByRdfhp4iJSTDNQJzgD1cFG07ean42KYpNRIhvwUSmQYCr4ddd2Bv/cdREAMGNEJCL9uStBb/w0JQL9vJW4VNuCr69qJ+MM+IniIswzULmVjahraZM4GsdQVtuCnVkVAIAHRkdLHA2R63Dny3hZZfX45mw5BAF4YtIAqcNxemoPOX4xzth+Z83ePImjsQ4TKBfRz0dl+SZ0mpfxAACbjhTBIAJjYwMxIJgtCohsxZ0LyVfsNs4+3TE8jK1PbOShm2KgkAk4nF+DU070+cUEyoWYNxY+4US/gPZiMIjYmFkMAJg5hrNPRLZknoE6WVwLvRNvxWGtouomfHHiEgDgyVsGShyN6wj1U+Pu5HAAzjULxQTKhXAl3hUHci+jsLoJvioF7koKlzocIpcyKMQHnh5yNLS2I7eyQepw+sx7ey5CbxAxcVAQkqK4bZYtmVsa/O/kJadZnMAEyoVwJd4V5iWxP02NcPvtJohsTSGXWRKIY25yGa+ivsUyq83ZJ9tLjfbHyP7+aNOL+O+BQqnD6RYmUC4kKVJjWV5c1dAqdTiS0TbpsO1MGQBjjxYisr0RblZI/v4PedC1GzCyvz9uinffDuz2ZJ6F+u/BArS26yWO5sYcIoF65513EBsbC7VajXHjxuHQoUPXHa/VapGRkYHw8HCoVCokJCRg69atlp+/++67SE5Ohp+fH/z8/JCWloavvvrK3qchOV+1B+KDvAG492W8z46VQNduwNBwPyRG+kkdDpFLsqzEc4NWBrVNbZZZkSdvGcg97+zkjsQwhPmpUdWgw/9OlEodzg1JnkBt2LABixcvxtKlS3H06FGkpKRg2rRpqKio6HK8TqfD1KlTkZ+fj02bNiErKwurVq1CZGSkZUxUVBReffVVHDlyBJmZmbj11ltxzz334MyZM311WpKx9INy042FRVG0XL6bNSaab3REdmJeiZdVXo9mnePPFvTGB/vz0dDajsGhvrh1SIjU4bgsD7kMD6fFADAWk4uiYy9QkDyBeuuttzBv3jzMnTsXw4YNw4oVK+Dl5YXVq1d3OX716tWorq7GZ599hvT0dMTGxmLSpElISUmxjJk+fTruuusuDBo0CAkJCXjllVfg4+ODAwcO9NVpSeZKHZRW2kAkcqqkFufL6qFUyDAjNfLGdyCiHgnXeCLUTwW9QcSpEtf9wtas02PNvnwAwJOTB0Am45cye/rF2P5QKWQ4c6kOh/NrpA7nuiRNoHQ6HY4cOYIpU6ZYjslkMkyZMgX79+/v8j5ffPEF0tLSkJGRgdDQUCQmJmLZsmXQ67v+BqTX67F+/Xo0NjYiLS3NLufhSJKvWl7s6Nm7PZhnn+5MDIPGi/vdEdnTlYaajv1B1xvrDxeiulGH/oFeuJsreu0uwFuJ+0Yav/w6eksDSROoqqoq6PV6hIaGdjgeGhqKsrKyLu+Tm5uLTZs2Qa/XY+vWrXjuuefw5ptv4uWXX+4w7tSpU/Dx8YFKpcITTzyBzZs3Y9iwYV0+ZmtrK+rq6jrcnNWwcD8oZAIuN+pQom2WOpw+1aRrx/+OG3u0sPcTkf2lRgcAcN1Ccl27ASv35AIAfnVzPBTcDqpPzBlvLCbffqYMxTVNEkdzbU7322AwGBASEoKVK1di1KhRmDlzJp599lmsWLGiw7jBgwfj+PHjOHjwIObPn49HH30UZ8+e7fIxly9fDo1GY7lFRzvvh6/aQ47BYcZ9qtytncHWU2Wob21H/0Av3BTXT+pwiFyeqxeSf3a8BKW1LQj2VeHno6KkDsdtDA7zRfrAfjCIwAf7C6QO55okTaCCgoIgl8tRXt5xA8Hy8nKEhYV1eZ/w8HAkJCRALr/S22fo0KEoKyuDTqezHFMqlRg4cCBGjRqF5cuXIyUlBX/729+6fMwlS5agtrbWcisqKrLB2Ukn2U03Ft5w2LhKZuaYaNYpEPWB5CgNZAJwqbYFFXXO0fywu/QGEStMmwY/PiEOag/2k+tLvzS1NFh/qBBNunaJo+mapAmUUqnEqFGjsGPHDssxg8GAHTt2XLNeKT09HTk5OTAYDJZj2dnZCA8Ph1KpvOZzGQwGtLZ23RtJpVJZWh6Yb84s1bSly0k3Wol3sbIBh/NrIBPAb4pEfcRbpcCgEOOMt6tdxtt+pgy5VY3wUyvw4E0xUofjdiYPDkFsPy/UtbTjk6MlUofTJckv4S1evBirVq3CunXrcO7cOcyfPx+NjY2YO3cuAOCRRx7BkiVLLOPnz5+P6upqLFy4ENnZ2diyZQuWLVuGjIwMy5glS5Zgz549yM/Px6lTp7BkyRLs2rULDz74YJ+fnxTMM1CnS2phcJN9qjaaiscnDw5BqJ9a4miI3EeqCzbUFEUR/9yVAwCYMz4WPiqFxBG5H5lMwKPjYwEAa/fmOeRnmeS/FTNnzkRlZSWef/55lJWVITU1Fdu2bbMUlhcWFkImu5LnRUdHY/v27Vi0aBGSk5MRGRmJhQsX4umnn7aMqaiowCOPPILS0lJoNBokJydj+/btmDp1ap+fnxQGhfhA7SFDfWs7cqsaXX7H8Da9AZ8c5cbBRFJI7e+PDZlFLpVA7blQhdMldfD0kGOO6VIS9b2fj4rCm19n42JlI77PqcKkhGCpQ+pA8gQKABYsWIAFCxZ0+bNdu3Z1OpaWlnbdnk7vv/++rUJzSgq5DIkRGmQW1OBksdblE6gd5ypQ1aBDsK8Kk9nkjqhPpV7VOkVvECF3gfrDf+40zj7NHtsfgd7XLg0h+/JVe+D+0VFYszcfq3/Ic7gESvJLeGQflkJyF/pWeC3m4vGfjYyCB5cZE/WphFBfeCnlaGhtx8XKBqnD6bUjBdU4mFcND7mAeTdz9klqc8bHQhCA3dmVyKlwrN8vftq4qBRTIfkJF29lUFrbjN3ZlQB4+Y5ICnKZgKRI4/uNK7Qz+OdO48q7+0ZEIVzjKXE0FNPPG7cNMZb0rDN1hHcUTKBclHkG6mxpHXTthusPdmKbMothEIFxcYGIM22kTER9y7wv3jEnn/E+V1qHHecrIAjAryfFSx0OmfwyPRYAsOlIMWqb2qQN5ipMoFxUbD8v+KkV0LUbkF1eL3U4dmEwiNiQaVx9x9knIumMcJGVeO+a+j7dlRiO+GDXrh11JmkD+mFwqC+a2/TYkFkodTgWTKBclCAILt9Qc9/FyyiuaYavWoE7E7lHFZFUzFu6ZJXVOWzTwxspuNyIL08at4Kaf8sAiaOhqwmCgLmmWah1+wrQrneMqypMoFxYcpRrN9Q0zz7dkxoBTyW7BBNJJUyjRpifGgYROF3inHuJvrcnFwYRmJQQjERTTRc5jhkjIhHg5YESbTO+PVd+4zv0ASZQLsyVZ6BqGnXYftq44fSsMf0ljoaIzAtXjhfVSByJ9crrWrAp09hLLmPyQImjoa6oPeT4xTjje/3qvfnSBmPCBMqFmd/QLlQ0oFmnlzga2/rseAl0egOGR/jx2yKRAzBfxnPGOqj3f8iDTm/A6JgAjI0LlDocuoaHb4qFQibgUF41TpdIf2WFCZQLC/NTI9hXBb1BxJlL0v+y2Yooilh/iMXjRI7EsqWLk7Uy0Dbp8J8DBQCAJyez9smRhWnUuDPJWO+6xgFmoZhAuTBBEJAS5Xr9oE4U1yKrvB4qhQz3pERKHQ4RwVhzKROAS7UtqKhrkTqcblu3rwBNOj2GhPli8mDuZODozMXk/ztxCVUNrZLGwgTKxZnroE66UB3UBtPGwXclhUPj5SFxNEQEAN4qBRJCfQE4Tz+oxtZ2rNmXBwB4cvJACILzb0Pj6kb2D8C8iXFYPWcM+km8zQ4TKBdnWYnnIjNQja3t+OJ4CQDggdG8fEfkSFKdrB/UR4cKoW1qQ2w/L9ydxFYozuLZu4dhwqAgyRNeJlAuzjwDlVfViNpmx+ng2lNbTpWiUadHbD8v3BTPYk8iR+JMdVCt7Xr863vj7NOvJw1wiU2QqW8xgXJxgd5KRAca93M65QKzUBtNl+8eGBMt+bcPIurIvKXLyWIt9AZR2mBuYPPREpTVtSDUT4X7RrKWkqzHBMoNuEo/qJyKemQW1EAuE/DzkVFSh0NEPzIoxBfeSjkadXrkVDRIHc416Q0i3tuTCwCYNzEeKgUb8ZL1mEC5gRRLHZRW2kB6yVw8PnlwCEL81BJHQ0Q/JpcJSDKv/HXgOqivTpcir6oR/l4emD2WjXipZ5hAuYErK/Gc9xKert2AT48ai8dnsfcTkcNKMdVBOepKPFEU8c5O46bBc8bHwlulkDgiclZMoNxAYqQGggCU1ragot55+rNcbce5clxu1CHEV4VbBgdLHQ4RXcMIB1+Jtyu7EudK6+CllGPO+FipwyEnxgTKDfioFBgY7APAeTcWNm8c/PNRUVDI+WtL5KjMW7pkldWhSdcucTSd/XNnDgDgF2P7w99L2j5C5Nz4SeQmnLmh5iVtM3ZnVwJg7yciRxemUSPMTw2D6Hgrfw/nV+Nwfg2UchkenxgvdTjk5JhAuQnzxsLOuKXLx5nFEEXgpvhAxAZ5Sx0OEd2AozbUNM8+/WxUJMI0XIhCvcMEyk1cPQMlio7dn+VqBoOIjabLd7PGcLUMkTMw94NypATqzKVa7MyqhEwAfn0zNw2m3mMC5SaGhvvCQy6gpqkNxTXNUofTbXsvVqFE2wxftQJ3JIZJHQ4RdYMjzkC9u8u48u7u5AjOZJNNMIFyEyqFHEPC/AA4V0PN9abeT/eOiITag83uiJxBUqQGMtPK3/I66Vf+5lU1YuupUgDA/EmcfSLbYALlRpxtY+HqRh2+OVMOAJjJ3k9ETsNbpUBCqC8A4JgD7Iv33u6LMIjArUNCMCzCT+pwyEUwgXIj5gZ3jtwh+Gqbj5VApzcgMdIPwyM0UodDRFYwX8aTesa7rLYFnxwtBgA8eQtnn8h2mEC5kRRTIfmpklqH3+hTFEVsOFwIAJjJ4nEip2Opg5J4BmrV97lo04sYGxeI0bGBksZCroUJlBsZGOIDL6UcTTo9LlY67kafgLH4NLu8AWoPGX6aEiF1OERkJfNKvJPFWsm+sNU06vDRIeMXMc4+ka0xgXIjcpmAxAjH3+gTuLJx8F2J4dB4ekgcDRFZa1CIL7yVcjTq9MipkOYL29p9+WjS6TE8wg+TErgFFNkWEyg34wyF5I2t7fjfiUsAWDxO5KzkMgFJpveb40U1ff78Da3tWLsvHwDw5C0DIQhCn8dAro0JlJtJNtUlOPKWLltOlqJRp0dckDfGxrFmgchZmffFk6If1EcHC1Hb3Ib4IG/2kCO7YALlZlJM3wjPldZD126QOJqurTcVjz8wOprfGomcmLmQvK9bGbS267Hq+1wAwBOTBkAu4/sI2R4TKDfTP9AL/l4e0OkNOF9WJ3U4nVwor8fRQi3kMgE/GxUpdThE1AsjTIXk2eX1aGxt77Pn/eRICSrqWxGuUWPGCL6PkH0wgXIzgiAgKdJxNxY2F4/fOiQEIb7c7JPImYX6qRGuUcMgGtun9IV2vQHv7TFu2/L4xHgoFfyYI/vgb5YbMveDOulgK/F07QZ8eqwEADCLxeNELiG1jxv4bjlVioLLTQjw8sDssXwfIfthAuWGHHUl3rfnylHdqEOon4pLjolcREofbiwsiqJl0+C56XHwUirs/pzkvphAuSHzG9qFino06fquLuFGzBsH/3xUFBRy/moSuYLUPkygvjtfgfNl9fBWyvFoWqzdn4/cGz+l3FConxqhfioYROB0iWMUkhfXNOH7C5UAjKvviMg1JEVqIBOA0toWlNe12O15RFHEP02zTw/dFAONFxvwkn0xgXJTyeY6KAfpB7XpSDFEERg/oB9i+nlLHQ4R2Yi3SoGEUF8A9m1ncCivGkcKaqBUyPDYhDi7PQ+RGRMoN2XuB+UIK/H0BhEfZxp3S2fncSLXY25nYM/LeO+YZp/uHxWFED+u4CX7YwLlphxpBuqHnCqUaJuh8fTAtOHsGEzkaq7UQdlnS5fTJbXYk10JmQD8+mZuGkx9gwmUmzKvxCu43ARtk07SWDaaisfvHREJtYdc0liIyPbMW7qcKq6F3iDa/PHNK++mp0Sgfz8vmz8+UVeYQLkpfy8lYkxvNFK2M7jc0Iqvz5YBYPE4kasaGOIDb6UcjTo9LlTU2/SxL1Y2YOvpUgDA/Fs4+0R9hwmUG3OEy3ibj5WgTS8iOUqDYRF+ksVBRPYjlwmW95vjNi4kf2/3RYgiMGVoCIaE8T2E+g4TKDcmdSG5KIqWrVtYPE7k2lJNheQnbPiF7ZK2GZ8eNe5e8OTkgTZ7XKLuYALlxqSegTpaqMWFigaoPWSYnhIhSQxE1DfMW0jZspXBqu9z0W4QcVN8IEb2D7DZ4xJ1BxMoN5YY6QeZAJTXtdq1wd21mIvH706KgJ+aTe+IXJm5lUF2eT0aW3u/A8LlhlasP2R8D3nyFs4+Ud9jAuXGvJQKDAoxNrjrq40+zRpa2/G/k5cA8PIdkTsI9VMjXKOGQQROlfS+bGDtvnw0t+mRFKnBxEFBNoiQyDpMoNycVBsLf3niEpp0esQHeWNMLKfeidyBrfbFq29pw7p9+QCAJ28ZAEEQehcYUQ8wgXJzyaY3NFsWdnbHhswrxeN88yNyD5YEqpd1UP89WIi6lnYMCPZm812SDBMoN2deiXeqpBaiaPsGd13JKqvHsUItFDIB942M6pPnJCLp2WIGqqVNj/d/yAMAPDFpAGQyfgEjaTCBcnNDwvyglMugbWpDYXVTnzynuXXBbUNDEOyr6pPnJCLpJUVpIJcJKKtrQVltzxaubDpSjMr6VkRo1LgnNdLGERJ1HxMoN6dUyDA03FhIbs+NPs1a2/XYfMy4cfCsMf3t/nxE5Di8lAokhJrfb6zfF69db8B7e4zbtvzq5ngoFfwII+nwt4+u6gdl/0Lyb86Wo6apDWF+atycEGz35yMix2K+jHesB1/YvjxZiqLqZvTzVmImv4CRxJhAEVJMb2h90VDTfPnu/tFRkLN2gcjtjDAvXLEygTIYRPxzVw4A4JcT4uCp5MbjJC0mUGQpJD9dUod2vcFuz1NU3YQfcqoAcONgIndl/sJ2qrgWekP3F67sOF+B7PIG+KgUeOimGDtFR9R9TKAI8cHGndKb2/TIqWyw2/N8fKQYogikD+yH6EAvuz0PETmugSHG95tGnR4XKuq7dR9RFPHOTuPs00M3xUDjyZ0LSHpMoAhymYDESFNDzSL71EHpDSI2WXo/sXaByF3JZYKl7rK7/aD2517G8SItVAoZHpsQZ7/giKzgEAnUO++8g9jYWKjVaowbNw6HDh267nitVouMjAyEh4dDpVIhISEBW7dutfx8+fLlGDNmDHx9fRESEoIZM2YgKyvL3qfh1FLs3FDz+wuVuFTbAn8vD9w+LNQuz0FEziHVtC9ed1f+vrvLuPLugdHRbH1CDkPyBGrDhg1YvHgxli5diqNHjyIlJQXTpk1DRUVFl+N1Oh2mTp2K/Px8bNq0CVlZWVi1ahUiI6/0A9m9ezcyMjJw4MABfPPNN2hra8Ptt9+OxsbGvjotp2PvLV3MxeMzUiOh9mDxJ5E7s6ah5sliLb6/UAW5TMCvbo63b2BEVlBIHcBbb72FefPmYe7cuQCAFStWYMuWLVi9ejWeeeaZTuNXr16N6upq7Nu3Dx4exuvgsbGxHcZs27atw5/Xrl2LkJAQHDlyBDfffLN9TsTJpZim1M+X1aG1XQ+VwnZJTlVDK749Vw6AGwcT0ZWVeNnl9WhsbYe36tofRf/caZx9uiclgrWT5FAknYHS6XQ4cuQIpkyZYjkmk8kwZcoU7N+/v8v7fPHFF0hLS0NGRgZCQ0ORmJiIZcuWQa/XX/N5amuNsyqBgYG2PQEXEhXgiQAvD7TpRZwr7V5hZ3dtPlqCNr2IlCgNhob72fSxicj5hPipEaFRwyBef9Y7p6Ie28+WAQCeuGVAX4VH1C2SJlBVVVXQ6/UIDe1YExMaGoqysrIu75Obm4tNmzZBr9dj69ateO655/Dmm2/i5Zdf7nK8wWDAU089hfT0dCQmJnY5prW1FXV1dR1u7kYQhKsaampt9riiKGL94UIALB4noiu6Uwf17q5ciCJw+7BQSwdzIkcheQ2UtQwGA0JCQrBy5UqMGjUKM2fOxLPPPosVK1Z0OT4jIwOnT5/G+vXrr/mYy5cvh0ajsdyio93zMpO5H9QJG67EO1pYg4uVjfD0kGN6SrjNHpeInNuVOqiut3Qp0Tbj8+MlAIAnJw/sq7CIuk3SBCooKAhyuRzl5eUdjpeXlyMsLKzL+4SHhyMhIQFy+ZUanaFDh6KsrAw6na7D2AULFuDLL7/Ezp07ERUVdc04lixZgtraWsutqKioF2flvOwxA7X+kPHv8u7kcPiq2buFiIxSowMAXPsL26o9uWg3iEgf2M+SbBE5EkkTKKVSiVGjRmHHjh2WYwaDATt27EBaWlqX90lPT0dOTg4Mhisds7OzsxEeHg6lUgnAeNlowYIF2Lx5M7777jvExV2/b4hKpYKfn1+HmztKjjbOQOVUNqChtb3Xj1ff0oYvT5YCAGaxeJyIrpIY6Qe5TEBZXQvKals6/KyqoRUfHTJe+n/yFs4+kWOS/BLe4sWLsWrVKqxbtw7nzp3D/Pnz0djYaFmV98gjj2DJkiWW8fPnz0d1dTUWLlyI7OxsbNmyBcuWLUNGRoZlTEZGBv7zn//gww8/hK+vL8rKylBWVobm5uY+Pz9nEuKrRrhGDVEETpf0/jLelydL0dymx4Bgb4yKCbBBhETkKryUCktd048v463Zm4fWdgNSojQYP6CfFOER3ZDkbQxmzpyJyspKPP/88ygrK0Nqaiq2bdtmKSwvLCyETHYlz4uOjsb27duxaNEiJCcnIzIyEgsXLsTTTz9tGfPuu+8CAG655ZYOz7VmzRrMmTPH7ufkzJKjNCitbcHJYi1uiu/dG9f6w+bO49EQBG4cTEQdpUb741xpHY4VaXFHorFGsq6lDR/sKwBgrH3iewc5KskTKMBYq7RgwYIuf7Zr165Ox9LS0nDgwIFrPp4odn+DSuooOcof28+U40QvG2qeL6vDiSItFDIB9428dv0ZEbmvEdH++OhQYYctXf5zoAD1re0YFOKDqUO5awE5LodIoMhxpNiokNzceXzqsFAE+XDrBSLqzNzK4FRJLfQGEW16A1b/kAcAmH/LAMhknH0ix8UEijpIMrUyKKpuRnWjDoHeSqsfo7Vdj83HjMuPH2DxOBFdw4BgH/ioFGhobUd2eT0O51ejqkGHSH9PTE+JkDo8ouuSvIicHIvG0wNxQd4Aej4L9fWZcmib2hCuUePmQcE2jI6IXIlcJlj24czMr8Z7u3MBAL+eFA8POT+eyLHxN5Q66e3GwubLd/ePjoacU/BEdB3mHk9//y4HJdpmBPko8cBozlyT42MCRZ30pqFmUXUTfsipgiAA949i8TgRXZ85gaqsbwUA/HJCHNQettvMnMhemEBRJ5YtXYprrV7R+HGmcfZpwsAg7pxORDd0dZdxX5UCD90UI10wRFZgAkWdDI/QQC4TUFnfirK6lhvfwURvELExsxgAOAVPRN0S4qdGhEYNAHhkfAz8uOUTOQkmUNSJp1KOQSE+AKzbWHhPdiXK6lrg7+WB24ezfwsRdc//u2MIfpIcjl9NHCB1KETdxgSKutSTflDm4vH7RkRBpWANAxF1z4wRkXj7FyOh8eLsEzkPJlDUJfPGwie6mUBV1rfi23PlAIxbtxAREbkyJlDUpSszULUwGG5cSP7p0WK0G0SkRvtjcJivnaMjIiKSFhMo6tLgMF8oFTLUt7Qj/3LjdceKoogNptV3szj7REREboAJFHXJQy7DsHA/ADduqJlZUIPcykZ4KeX4CbdfICIiN8AEiq7pSj8o7XXHmYvHf5IcDh8Vt1ckIiLXxwSKrinF1ODuejNQdS1t2HKyFAAwc0z/vgiLiIhIckyg6JrMW7qcuVSLdr2hyzH/O3EJzW16DAzxwcj+/n0XHBERkYSYQNE1xQd5w1elQEubAdnlDV2O2Xj4SvG4IHDjYCIicg9MoOiaZDIBiZHGOqiuGmqevVSHE8W18JALuHdEZB9HR0REJB0mUHRdVxpqdq6D2mhqXTB1WCj6+aj6NC4iIiIpMYGi67rWli4tbXpsPlYCgMXjRETkfphA0XUlm1oZZJXVo6VNbzm+/UwZapvbEKFRY8LAIKnCIyIikgQTKLquSH9P9PNWot0g4mxpneW4uffT/aOjIZexeJyIiNwLEyi6LkEQLLNQJ4u0AIDCy03Yd/EyBAG4f3SUhNERERFJgwkU3VDyVRsLA1eKxycMDEJUgJdUYREREUmGCRTdUEr0lS1d2vUGfHzE3PuJxeNEROSemEDRDZlnoHKrGrHlVCnK61oR6K3ElGEh0gZGREQkESZQdENBPipE+ntCFIHlW88DAO4dEQmVQi5xZERERNJgAkXdYi4kL6trAQDMHBMtZThERESSYgJF3WK+jAcAI/v7IyHUV7pgiIiIJMYEirolxTQDBXD2iYiISCF1AOQckqP94e/lAYVMwN3JEVKHQ0REJCkmUNQtPioFvvzNBMhlAnxU/LUhIiL3xk9C6jY2zSQiIjJiDRQRERGRlZhAEREREVmJCRQRERGRlZhAEREREVmJCRQRERGRlZhAEREREVmJCRQRERGRlZhAEREREVmJCRQRERGRlZhAEREREVmJCRQRERGRlZhAEREREVmJCRQRERGRlZhAEREREVlJIXUAjkgURQBAXV2dxJEQERFRXzF/7pvzgOthAtWF+vp6AEB0dLTEkRAREVFfq6+vh0ajue4YQexOmuVmDAYDLl26BF9fXwiCYNPHrqurQ3R0NIqKiuDn52fTx3Y07nKuPE/XwvN0Pe5yrjzP3hNFEfX19YiIiIBMdv0qJ85AdUEmkyEqKsquz+Hn5+fSv+BXc5dz5Xm6Fp6n63GXc+V59s6NZp7MWEROREREZCUmUERERERWYgLVx1QqFZYuXQqVSiV1KHbnLufK83QtPE/X4y7nyvPsWywiJyIiIrISZ6CIiIiIrMQEioiIiMhKTKCIiIiIrMQEioiIiMhKTKBs4J133kFsbCzUajXGjRuHQ4cOXXf8xx9/jCFDhkCtViMpKQlbt27t8HNRFPH8888jPDwcnp6emDJlCi5cuGDPU+gWa85z1apVmDhxIgICAhAQEIApU6Z0Gj9nzhwIgtDhdscdd9j7NG7ImvNcu3Ztp3NQq9Udxjjq6wlYd6633HJLp3MVBAF33323ZYyjvaZ79uzB9OnTERERAUEQ8Nlnn93wPrt27cLIkSOhUqkwcOBArF27ttMYa//N9wVrz/XTTz/F1KlTERwcDD8/P6SlpWH79u0dxrzwwgudXs8hQ4bY8SxuzNrz3LVrV5e/t2VlZR3GOdprau15dvVvTxAEDB8+3DLGEV/P5cuXY8yYMfD19UVISAhmzJiBrKysG97PET5HmUD10oYNG7B48WIsXboUR48eRUpKCqZNm4aKiooux+/btw+zZ8/GY489hmPHjmHGjBmYMWMGTp8+bRnz+uuv4+9//ztWrFiBgwcPwtvbG9OmTUNLS0tfnVYn1p7nrl27MHv2bOzcuRP79+9HdHQ0br/9dpSUlHQYd8cdd6C0tNRy++ijj/ridK7J2vMEjN1wrz6HgoKCDj93xNcTsP5cP/300w7nefr0acjlctx///0dxjnSa9rY2IiUlBS888473Rqfl5eHu+++G5MnT8bx48fx1FNP4fHHH++QWPTkd6QvWHuue/bswdSpU7F161YcOXIEkydPxvTp03Hs2LEO44YPH97h9fzhhx/sEX63WXueZllZWR3OIyQkxPIzR3xNrT3Pv/3tbx3Or6ioCIGBgZ3+fTra67l7925kZGTgwIED+Oabb9DW1obbb78djY2N17yPw3yOitQrY8eOFTMyMix/1uv1YkREhLh8+fIuxz/wwAPi3Xff3eHYuHHjxF//+teiKIqiwWAQw8LCxD//+c+Wn2u1WlGlUokfffSRHc6ge6w9zx9rb28XfX19xXXr1lmOPfroo+I999xj61B7xdrzXLNmjajRaK75eI76eopi71/Tv/zlL6Kvr6/Y0NBgOeaIr6kZAHHz5s3XHfP//t//E4cPH97h2MyZM8Vp06ZZ/tzbv7e+0J1z7cqwYcPEF1980fLnpUuXiikpKbYLzMa6c547d+4UAYg1NTXXHOPor2lPXs/NmzeLgiCI+fn5lmOO/nqKoihWVFSIAMTdu3dfc4yjfI5yBqoXdDodjhw5gilTpliOyWQyTJkyBfv37+/yPvv37+8wHgCmTZtmGZ+Xl4eysrIOYzQaDcaNG3fNx7S3npznjzU1NaGtrQ2BgYEdju/atQshISEYPHgw5s+fj8uXL9s0dmv09DwbGhoQExOD6Oho3HPPPThz5ozlZ474egK2eU3ff/99zJo1C97e3h2OO9Jraq0b/fu0xd+bozIYDKivr+/0b/TChQuIiIhAfHw8HnzwQRQWFkoUYe+kpqYiPDwcU6dOxd69ey3HXfU1ff/99zFlyhTExMR0OO7or2dtbS0AdPo9vJqjfI4ygeqFqqoq6PV6hIaGdjgeGhra6fq6WVlZ2XXHm/9rzWPaW0/O88eefvppREREdPiFvuOOO/DBBx9gx44deO2117B7927ceeed0Ov1No2/u3pynoMHD8bq1avx+eef4z//+Q8MBgPGjx+P4uJiAI75egK9f00PHTqE06dP4/HHH+9w3NFeU2td699nXV0dmpubbfJvwVG98cYbaGhowAMPPGA5Nm7cOKxduxbbtm3Du+++i7y8PEycOBH19fUSRmqd8PBwrFixAp988gk++eQTREdH45ZbbsHRo0cB2Ob9zdFcunQJX331Vad/n47+ehoMBjz11FNIT09HYmLiNcc5yueowmaPRHQNr776KtavX49du3Z1KLCeNWuW5f+TkpKQnJyMAQMGYNeuXbjtttukCNVqaWlpSEtLs/x5/PjxGDp0KN577z386U9/kjAy+3r//feRlJSEsWPHdjjuCq+pO/rwww/x4osv4vPPP+9QG3TnnXda/j85ORnjxo1DTEwMNm7ciMcee0yKUK02ePBgDB482PLn8ePH4+LFi/jLX/6Cf//73xJGZj/r1q2Dv78/ZsyY0eG4o7+eGRkZOH36tOR1Wd3FGaheCAoKglwuR3l5eYfj5eXlCAsL6/I+YWFh1x1v/q81j2lvPTlPszfeeAOvvvoqvv76ayQnJ193bHx8PIKCgpCTk9PrmHuiN+dp5uHhgREjRljOwRFfT6B359rY2Ij169d36w1X6tfUWtf69+nn5wdPT0+b/I44mvXr1+Pxxx/Hxo0bO10W+TF/f38kJCQ4zet5LWPHjrWcg6u9pqIoYvXq1Xj44YehVCqvO9aRXs8FCxbgyy+/xM6dOxEVFXXdsY7yOcoEqheUSiVGjRqFHTt2WI4ZDAbs2LGjw6zE1dLS0jqMB4BvvvnGMj4uLg5hYWEdxtTV1eHgwYPXfEx768l5AsZVEH/605+wbds2jB49+obPU1xcjMuXLyM8PNwmcVurp+d5Nb1ej1OnTlnOwRFfT6B35/rxxx+jtbUVDz300A2fR+rX1Fo3+vdpi98RR/LRRx9h7ty5+Oijjzq0o7iWhoYGXLx40Wlez2s5fvy45Rxc7TXdvXs3cnJyuvUFxxFeT1EUsWDBAmzevBnfffcd4uLibngfh/kctVk5uptav369qFKpxLVr14pnz54Vf/WrX4n+/v5iWVmZKIqi+PDDD4vPPPOMZfzevXtFhUIhvvHGG+K5c+fEpUuXih4eHuKpU6csY1599VXR399f/Pzzz8WTJ0+K99xzjxgXFyc2Nzf3+fmZWXuer776qqhUKsVNmzaJpaWlllt9fb0oiqJYX18v/v73vxf3798v5uXlid9++604cuRIcdCgQWJLS4sk5yiK1p/niy++KG7fvl28ePGieOTIEXHWrFmiWq0Wz5w5YxnjiK+nKFp/rmYTJkwQZ86c2em4I76m9fX14rFjx8Rjx46JAMS33npLPHbsmFhQUCCKoig+88wz4sMPP2wZn5ubK3p5eYl/+MMfxHPnzonvvPOOKJfLxW3btlnG3OjvTSrWnut///tfUaFQiO+8806Hf6NardYy5ne/+524a9cuMS8vT9y7d684ZcoUMSgoSKyoqOjz8zOz9jz/8pe/iJ999pl44cIF8dSpU+LChQtFmUwmfvvtt5YxjviaWnueZg899JA4bty4Lh/TEV/P+fPnixqNRty1a1eH38OmpibLGEf9HGUCZQP/+Mc/xP79+4tKpVIcO3aseODAAcvPJk2aJD766KMdxm/cuFFMSEgQlUqlOHz4cHHLli0dfm4wGMTnnntODA0NFVUqlXjbbbeJWVlZfXEq12XNecbExIgAOt2WLl0qiqIoNjU1ibfffrsYHBwsenh4iDExMeK8efMk/xASRevO86mnnrKMDQ0NFe+66y7x6NGjHR7PUV9PUbT+d/f8+fMiAPHrr7/u9FiO+Jqal7D/+GY+r0cffVScNGlSp/ukpqaKSqVSjI+PF9esWdPpca/39yYVa8910qRJ1x0visYWDuHh4aJSqRQjIyPFmTNnijk5OX17Yj9i7Xm+9tpr4oABA0S1Wi0GBgaKt9xyi/jdd991elxHe0178rur1WpFT09PceXKlV0+piO+nl2dI4AO/+4c9XNUMJ0AEREREXUTa6CIiIiIrMQEioiIiMhKTKCIiIiIrMQEioiIiMhKTKCIiIiIrMQEioiIiMhKTKCIiIiIrMQEiojISi+88AJSU1OlDoOIJMQEioiIiMhKTKCIiIiIrMQEiohcyrZt2zBhwgT4+/ujX79++MlPfoKLFy8CAPLz8yEIAj799FNMnjwZXl5eSElJwf79+zs8xqpVqxAdHQ0vLy/ce++9eOutt+Dv73/d5/3Xv/6FoUOHQq1WY8iQIfjnP/9pr1MkIgfABIqIXEpjYyMWL16MzMxM7NixAzKZDPfeey8MBoNlzLPPPovf//73OH78OBISEjB79my0t7cDAPbu3YsnnngCCxcuxPHjxzF16lS88sor133O//73v3j++efxyiuv4Ny5c1i2bBmee+45rFu3zq7nSkTS4WbCROTSqqqqEBwcjFOnTsHHxwdxcXH417/+hcceewwAcPbsWQwfPhznzp3DkCFDMGvWLDQ0NODLL7+0PMZDDz2EL7/8ElqtFoCxiPyzzz7D8ePHAQADBw7En/70J8yePdtyn5dffhlbt27Fvn37+uxciajvcAaKiFzKhQsXMHv2bMTHx8PPzw+xsbEAgMLCQsuY5ORky/+Hh4cDACoqKgAAWVlZGDt2bIfH/PGfr9bY2IiLFy/iscceg4+Pj+X28ssvWy4dEpHrUUgdABGRLU2fPh0xMTFYtWoVIiIiYDAYkJiYCJ1OZxnj4eFh+X9BEACgwyU+azQ0NAAw1k2NGzeuw8/kcnmPHpOIHB8TKCJyGZcvX0ZWVhZWrVqFiRMnAgB++OEHqx5j8ODBOHz4cIdjP/7z1UJDQxEREYHc3Fw8+OCD1gdNRE6JCRQRuYyAgAD069cPK1euRHh4OAoLC/HMM89Y9Ri/+c1vcPPNN+Ott97C9OnT8d133+Grr76yzFR15cUXX8Rvf/tbaDQa3HHHHWhtbUVmZiZqamqwePHi3p4WETkg1kARkcuQyWRYv349jhw5gsTERCxatAh//vOfrXqM9PR0rFixAm+99RZSUlKwbds2LFq0CGq1+pr3efzxx/Gvf/0La9asQVJSEiZNmoS1a9ciLi6ut6dERA6Kq/CIiG5g3rx5OH/+PL7//nupQyEiB8FLeEREP/LGG29g6tSp8Pb2xldffYV169axMSYRdcAZKCKiH3nggQewa9cu1NfXIz4+Hr/5zW/wxBNPSB0WETkQJlBEREREVmIROREREZGVmEARERERWYkJFBEREZGVmEARERERWYkJFBEREZGVmEARERERWYkJFBEREZGVmEARERERWYkJFBEREZGV/j8nOl6WsUCOPwAAAABJRU5ErkJggg==\n"
          },
          "metadata": {}
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ccfdbRcQ7Zgg"
      },
      "source": [
        "We can do multiple augmentations at the same time. Here is a function to do both shear and rotation to the sample. In this case, we will have two hyperparameters."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "sh8S_Pxa0XCv"
      },
      "source": [
        "def augRotateShear(sample, angleconstraint, shearconstraint):\n",
        "  if len(sample.shape)==2:\n",
        "    sample = np.expand_dims(sample, 0)  # make sure the sample is 3 dimensional\n",
        "  amt = rng.random(len(sample)) # generate random numbers for shear\n",
        "  amt = (amt-0.5)*shearconstraint # make the random shear constrained\n",
        "  angle = rng.random(len(sample)) # generate random numbers for angles\n",
        "  angle = (angle-0.5)*angleconstraint # make the random angle constrained\n",
        "  nsample = sample.copy() # preallocate the augmented array to make it faster\n",
        "  for ii in range(len(sample)):\n",
        "    nsample[ii] = rotate(shear(sample[ii], amt[ii]), angle[ii]) # first apply shear, then rotate\n",
        "  return np.squeeze(nsample) # take care if the input had only one sample."
      ],
      "execution_count": 16,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "OGKyjjNx-NQ4"
      },
      "source": [
        "Since we have two hyperparameters, we have to do the grid search on a 2 dimensional matrix. We can use our previous experience to inform where to search for the best hyperparameters."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "TJC45WRg0pOP",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "c5242db3-8765-4db1-8b98-6d22610f783b"
      },
      "source": [
        "shearconstraints = [0, 0.2,0.4,0.6,0.8,1.0,1.2,1.4,1.6] # the values we want to test\n",
        "angleconstraints = [0,10,20,30,40,50,60] # the values we want to test\n",
        "hyp = cartesian((shearconstraints, angleconstraints)) # cartesian product of both\n",
        "\n",
        "accuracies = np.zeros(len(hyp), dtype=np.float) # we will save the values here\n",
        "\n",
        "for ii in range(len(hyp)):\n",
        "  # create the augmented dataset\n",
        "  augdata = train_X # we include the original images also in the augmented dataset\n",
        "  auglabel = train_y\n",
        "  for jj in range(naugmentations):\n",
        "    augdata = np.concatenate((augdata, augRotateShear(train_X, hyp[ii][0], hyp[ii][1]))) # concatenate the augmented data to the set\n",
        "    auglabel = np.concatenate((auglabel, train_y))  # the labels don't change when we augment\n",
        "\n",
        "  # check the test accuracy\n",
        "  testpred = NN(augdata, auglabel, test_X)\n",
        "  accuracies[ii] = Accuracy(test_y, testpred)\n",
        "  print('Accuracy after augmentation shear:',hyp[ii][0], 'angle:',hyp[ii][1], ' is ', accuracies[ii], flush=True)"
      ],
      "execution_count": 17,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "<ipython-input-17-09de780976d0>:5: DeprecationWarning: `np.float` is a deprecated alias for the builtin `float`. To silence this warning, use `float` by itself. Doing this will not modify any behavior and is safe. If you specifically wanted the numpy scalar type, use `np.float64` here.\n",
            "Deprecated in NumPy 1.20; for more details and guidance: https://numpy.org/devdocs/release/1.20.0-notes.html#deprecations\n",
            "  accuracies = np.zeros(len(hyp), dtype=np.float) # we will save the values here\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Accuracy after augmentation shear: 0.0 angle: 0.0  is  0.6332\n",
            "Accuracy after augmentation shear: 0.0 angle: 10.0  is  0.6151\n",
            "Accuracy after augmentation shear: 0.0 angle: 20.0  is  0.5997\n",
            "Accuracy after augmentation shear: 0.0 angle: 30.0  is  0.5915\n",
            "Accuracy after augmentation shear: 0.0 angle: 40.0  is  0.6198\n",
            "Accuracy after augmentation shear: 0.0 angle: 50.0  is  0.6031\n",
            "Accuracy after augmentation shear: 0.0 angle: 60.0  is  0.5916\n",
            "Accuracy after augmentation shear: 0.2 angle: 0.0  is  0.634\n",
            "Accuracy after augmentation shear: 0.2 angle: 10.0  is  0.5991\n",
            "Accuracy after augmentation shear: 0.2 angle: 20.0  is  0.6044\n",
            "Accuracy after augmentation shear: 0.2 angle: 30.0  is  0.6358\n",
            "Accuracy after augmentation shear: 0.2 angle: 40.0  is  0.5827\n",
            "Accuracy after augmentation shear: 0.2 angle: 50.0  is  0.579\n",
            "Accuracy after augmentation shear: 0.2 angle: 60.0  is  0.5877\n",
            "Accuracy after augmentation shear: 0.4 angle: 0.0  is  0.6344\n",
            "Accuracy after augmentation shear: 0.4 angle: 10.0  is  0.6151\n",
            "Accuracy after augmentation shear: 0.4 angle: 20.0  is  0.5988\n",
            "Accuracy after augmentation shear: 0.4 angle: 30.0  is  0.6107\n",
            "Accuracy after augmentation shear: 0.4 angle: 40.0  is  0.6177\n",
            "Accuracy after augmentation shear: 0.4 angle: 50.0  is  0.5855\n",
            "Accuracy after augmentation shear: 0.4 angle: 60.0  is  0.5893\n",
            "Accuracy after augmentation shear: 0.6 angle: 0.0  is  0.6339\n",
            "Accuracy after augmentation shear: 0.6 angle: 10.0  is  0.6191\n",
            "Accuracy after augmentation shear: 0.6 angle: 20.0  is  0.595\n",
            "Accuracy after augmentation shear: 0.6 angle: 30.0  is  0.5963\n",
            "Accuracy after augmentation shear: 0.6 angle: 40.0  is  0.6203\n",
            "Accuracy after augmentation shear: 0.6 angle: 50.0  is  0.5859\n",
            "Accuracy after augmentation shear: 0.6 angle: 60.0  is  0.6094\n",
            "Accuracy after augmentation shear: 0.8 angle: 0.0  is  0.6354\n",
            "Accuracy after augmentation shear: 0.8 angle: 10.0  is  0.6208\n",
            "Accuracy after augmentation shear: 0.8 angle: 20.0  is  0.5928\n",
            "Accuracy after augmentation shear: 0.8 angle: 30.0  is  0.5875\n",
            "Accuracy after augmentation shear: 0.8 angle: 40.0  is  0.6242\n",
            "Accuracy after augmentation shear: 0.8 angle: 50.0  is  0.5796\n",
            "Accuracy after augmentation shear: 0.8 angle: 60.0  is  0.5947\n",
            "Accuracy after augmentation shear: 1.0 angle: 0.0  is  0.6334\n",
            "Accuracy after augmentation shear: 1.0 angle: 10.0  is  0.6178\n",
            "Accuracy after augmentation shear: 1.0 angle: 20.0  is  0.6074\n",
            "Accuracy after augmentation shear: 1.0 angle: 30.0  is  0.5934\n",
            "Accuracy after augmentation shear: 1.0 angle: 40.0  is  0.6199\n",
            "Accuracy after augmentation shear: 1.0 angle: 50.0  is  0.5865\n",
            "Accuracy after augmentation shear: 1.0 angle: 60.0  is  0.5952\n",
            "Accuracy after augmentation shear: 1.2 angle: 0.0  is  0.6328\n",
            "Accuracy after augmentation shear: 1.2 angle: 10.0  is  0.5727\n",
            "Accuracy after augmentation shear: 1.2 angle: 20.0  is  0.617\n",
            "Accuracy after augmentation shear: 1.2 angle: 30.0  is  0.5852\n",
            "Accuracy after augmentation shear: 1.2 angle: 40.0  is  0.5813\n",
            "Accuracy after augmentation shear: 1.2 angle: 50.0  is  0.587\n",
            "Accuracy after augmentation shear: 1.2 angle: 60.0  is  0.6195\n",
            "Accuracy after augmentation shear: 1.4 angle: 0.0  is  0.6344\n",
            "Accuracy after augmentation shear: 1.4 angle: 10.0  is  0.6069\n",
            "Accuracy after augmentation shear: 1.4 angle: 20.0  is  0.601\n",
            "Accuracy after augmentation shear: 1.4 angle: 30.0  is  0.5848\n",
            "Accuracy after augmentation shear: 1.4 angle: 40.0  is  0.6005\n",
            "Accuracy after augmentation shear: 1.4 angle: 50.0  is  0.6045\n",
            "Accuracy after augmentation shear: 1.4 angle: 60.0  is  0.6129\n",
            "Accuracy after augmentation shear: 1.6 angle: 0.0  is  0.6369\n",
            "Accuracy after augmentation shear: 1.6 angle: 10.0  is  0.6097\n",
            "Accuracy after augmentation shear: 1.6 angle: 20.0  is  0.6078\n",
            "Accuracy after augmentation shear: 1.6 angle: 30.0  is  0.6048\n",
            "Accuracy after augmentation shear: 1.6 angle: 40.0  is  0.585\n",
            "Accuracy after augmentation shear: 1.6 angle: 50.0  is  0.5915\n",
            "Accuracy after augmentation shear: 1.6 angle: 60.0  is  0.5941\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "PT6CnvSDEX7a"
      },
      "source": [
        "Let us plot it two dimensionally to see which is the best value for the hyperparameters:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "jD2i7msI_cLd",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 481
        },
        "outputId": "0b88a4bf-b9a5-4eb0-ee87-f18084d78a17"
      },
      "source": [
        "fig = plt.figure()\n",
        "ax = fig.add_axes([0.1, 0.1, 0.8, 0.8]) # main axes\n",
        "im = ax.imshow(accuracies.reshape((len(shearconstraints), len(angleconstraints))), cmap='inferno')\n",
        "ax.set_xlabel('angle')\n",
        "ax.set_ylabel('shear')\n",
        "ax.set_xticks(np.arange(len(angleconstraints)));\n",
        "ax.set_xticklabels(angleconstraints);\n",
        "ax.set_yticks(np.arange(len(shearconstraints)));\n",
        "ax.set_yticklabels(shearconstraints);\n",
        "plt.colorbar(im)"
      ],
      "execution_count": 18,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<matplotlib.colorbar.Colorbar at 0x7f07389ed8a0>"
            ]
          },
          "metadata": {},
          "execution_count": 18
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "<Figure size 640x480 with 2 Axes>"
            ],
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAccAAAG/CAYAAAAtjRweAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/bCgiHAAAACXBIWXMAAA9hAAAPYQGoP6dpAAA/VUlEQVR4nO3df1hUZd4/8PcMMoOooGYMP0SxNX+QiSyuNNoPXUfJfNis3Y01d2HZcr8a7JKz7aNsCbkqtFuy1j4kiSH2WKlxqVkq5qLoY+IaKJuWoYYKGQOS8WtSRuec7x/IqRlQZ5iBMzO8X133dTln7nPPZ+jCj/fn3Oc+ClEURRAREZFEKXcARERErobJkYiIyAqTIxERkRUmRyIiIitMjkRERFaYHImIiKwwORIREVlhciQiIrLSR+4AepogCPj6668xYMAAKBQKucMhIuqUKIpobm5GcHAwlMruncdcvXoVJpPJKWOpVCr4+Pg4ZSw59brk+PXXXyM0NFTuMIiIbFJdXY2hQ4d22/hXr17FiBGBMBganTJeYGAgzp075/YJstclxwEDBgAAPv/PbzFggErmaG7v+vIiuUOwyQeHpsgdgs2OXe4rdwg2+9sz78gdgk32vf+w3CHYbO/Xg+UOwSYmwYSNl9dLf2d12+eYTDAYGlF54R/w83Psd6Op6QruGr4IJpOJydHdtJdSBwxQwc8NkuM1lZfcIdikr5fr/yzbqZRquUOwmZ+Pe5T+fb285Q7BZu70/x9Aj13+8fPr63By9CS9LjkSEVFHongdonjd4TE8BZMjERFBFM0QRbPDY3gK3spBRERkhTNHIiKCIF6H4GBZ1NHzXQmTIxER8ZqjFZZViYiIrHDmSERENxbkODpz9JwFOUyOREQEUbgOUXAwOTp4vithWZWIiMgKZ45ERASI19uao2N4CCZHIiLialUrLKsSERFZ4cyRiIgA4TogXHN8DA/B5EhERDfKqo49BYhlVReQnZ2NsLAw+Pj4IDo6GkePHpU7JCIi8hBumRw3b94MvV6P9PR0HDt2DBEREYiJiUFdXZ3coRERuSfhunOah3DL5JiVlYX58+cjMTER4eHhyMnJga+vL/Ly8uQOjYjIPTE5WnC75GgymVBWVgadTicdUyqV0Ol0KCkp6dC/tbUVTU1NFo2IiOhW3C451tfXw2w2Q6PRWBzXaDQwGAwd+mdmZsLf319qoaGhPRUqEZEbMX+/EUBXGzxnb1W3S472Sk1NRWNjo9Sqq6vlDomIyOUohOtOaZ7C7W7lGDJkCLy8vFBbW2txvLa2FoGBgR36q9VqqNXqngqPiIg8gNvNHFUqFaKiolBUVCQdEwQBRUVF0Gq1MkZGROTGuCDHgtvNHAFAr9cjISEBEydOxKRJk7B69WoYjUYkJibKHRoRkXsSrgOCg/MlJkd5xcXF4dKlS0hLS4PBYMCECRNQWFjYYZEOERFRV7hlcgSA5ORkJCcnyx0GEZFHUIjXoRAdmzkqPGj7OLdNjkRE5ESCAAgO3oohCM6JxQW43YIcIiKi7saZIxER3bhPUeHwGJ6CyZGIiNpKqg6vVuUOOURERB6LM0ciIrpxn6NjZVXe50hERB5FIZihcLCsqmBZlYiIyHNx5khERIDohAU5ImeORETkQRSCcKO06kizfxOA7OxshIWFwcfHB9HR0Th69Ogt+zc0NCApKQlBQUFQq9UYNWoUdu3aJb2/Zs0ajB8/Hn5+fvDz84NWq8Xu3bvtjoszRyIiksXmzZuh1+uRk5OD6OhorF69GjExMaioqEBAQECH/iaTCTNmzEBAQAAKCgoQEhKCCxcuYODAgVKfoUOH4qWXXsLdd98NURSxYcMGPProozh+/Djuuecem2NjciQiohv3OTq6WtW+smpWVhbmz58vPVEpJycHO3fuRF5eHpYsWdKhf15eHi5fvozDhw/D29sbABAWFmbRJzY21uL1ypUrsWbNGhw5csSu5MiyKhEROaGkapZWqzY1NVm01tbWDp9nMplQVlYGnU4nHVMqldDpdCgpKek0xh07dkCr1SIpKQkajQbjxo1DRkYGzObOk7LZbMamTZtgNBrtft4vkyMRETlVaGgo/P39pZaZmdmhT319Pcxmc4dHDWo0GhgMhk7HraysREFBAcxmM3bt2oWlS5di1apVWLFihUW/EydOoH///lCr1ViwYAG2bduG8PBwu74Dy6pEROTUsmp1dTX8/Pykw2q12rFx24cXBAQEBGDt2rXw8vJCVFQULl68iJdffhnp6elSv9GjR6O8vByNjY0oKChAQkICDhw4YFeC7L3JsU9/wNs5/8O6U0HxQ3KHYJMwv0a5Q7DZvE8j5Q7BZpMGPiJ3CDapVZyTOwSbaZWD5Q7BJtdEsUc/r60s6ujG423JsX2l6K0MGTIEXl5eqK2ttTheW1uLwMDATs8JCgqCt7c3vLy8pGNjx46FwWCAyWSCSqUCAKhUKowcORIAEBUVhU8++QSvvvoq3njjDZu/C8uqRETU41QqFaKiolBUVCQdEwQBRUVFN70+OGXKFJw9exbCD24ZOX36NIKCgqTE2BlBEDq97nkrTI5ERHSjrOqEZge9Xo/c3Fxs2LABp06dwsKFC2E0GqXVq/Hx8UhNTZX6L1y4EJcvX0ZKSgpOnz6NnTt3IiMjA0lJSVKf1NRUHDx4EOfPn8eJEyeQmpqK4uJizJs3z67Yem9ZlYiIJApB7NJN/NZj2CMuLg6XLl1CWloaDAYDJkyYgMLCQmmRTlVVFZTK7+dwoaGh2LNnDxYtWoTx48cjJCQEKSkpWLx4sdSnrq4O8fHxqKmpgb+/P8aPH489e/ZgxowZdsXG5EhERLJJTk5GcnJyp+8VFxd3OKbVanHkyJGbjvfmm286JS4mRyIiulEWdcIYHoLJkYiIbmw87oQxPAQX5BAREVnhzJGIiKAQBShEB+9zFB2deroOJkciIuI1RyssqxIREVnhzJGIiABBcMLeqiyrEhGRJ2FytMCyKhERkRXOHImICApBgMLBiZ+j28+5EiZHIiK6UVZ1whgegmVVIiIiKy6RHLOzsxEWFgYfHx9ER0fj6NGjN+2bm5uLBx54AIMGDcKgQYOg0+lu2Z+IiGwgCM5pHkL25Lh582bo9Xqkp6fj2LFjiIiIQExMDOrq6jrtX1xcjLlz52L//v0oKSlBaGgoZs6ciYsXL/Zw5EREHoTJ0YLsyTErKwvz589HYmIiwsPDkZOTA19fX+Tl5XXa/+2338YzzzyDCRMmYMyYMVi3bp309GgiIiJnkHVBjslkQllZmcWTnpVKJXQ6HUpKSmwa47vvvsO1a9cwePDgTt9vbW1Fa2ur9LqpqcmxoImIPJFoBux8WHHHMThzdIr6+nqYzWbpqc/tNBoNDAaDTWMsXrwYwcHB0Ol0nb6fmZkJf39/qYWGhjocNxGRp2m7lcPx5ilkL6s64qWXXsKmTZuwbds2+Pj4dNonNTUVjY2NUquuru7hKImIyN3IWlYdMmQIvLy8UFtba3G8trYWgYGBtzz3lVdewUsvvYR//etfGD9+/E37qdVqqNVqp8RLROSxeJ+jBVlnjiqVClFRURaLadoX12i12pue9/e//x3Lly9HYWEhJk6c2BOhEhF5Nq5WtSD7Djl6vR4JCQmYOHEiJk2ahNWrV8NoNCIxMREAEB8fj5CQEGRmZgIA/va3vyEtLQ3vvPMOwsLCpGuT/fv3R//+/WX7HkRE5DlkT45xcXG4dOkS0tLSYDAYMGHCBBQWFkqLdKqqqqBUfj/BXbNmDUwmE37xi19YjJOeno4XX3yxJ0MnIvIcguj4zM/R1a4uRPbkCADJyclITk7u9L3i4mKL1+fPn+/+gIiIehtBdMI1R89Jjm69WpWIiKg7uMTMkYiIZOaUhx17zsyRyZGIiJgcrbCsSkREZIUzRyIi4oIcK0yORETUtmm46GBZVfSc5MiyKhERkRXOHImIqG3W52hZ1YNmjkyORETEa45WWFYlIiKywpkjERFx5miFyZGIiNoWqzqYHB0935WwrEpERGSFM0ciImJZ1UqvTY4KLzUUXj5yh3Fbv5h6QO4QbLJl/1S5Q7DZXb94R+4QbHb4D5Vyh2CTn732M7lDsFk/lYM3uveQa47ekG8vAU5Ijs4IxDWwrEpERGSl184ciYjoBzhztMDkSEREgHijOTqGh2BZlYiIyApnjkREBFFQQHTwYceedJ8jkyMREfGaoxWWVYmIiKxw5khERG0POnawrOpJC3KYHImIiNccrbCsSkREZIUzRyIiaiupOlpW5cyRiIg8iqhwTrNTdnY2wsLC4OPjg+joaBw9evSW/RsaGpCUlISgoCCo1WqMGjUKu3btkt7PzMzET37yEwwYMAABAQGYM2cOKioq7I6LyZGIiGSxefNm6PV6pKen49ixY4iIiEBMTAzq6uo67W8ymTBjxgycP38eBQUFqKioQG5uLkJCQqQ+Bw4cQFJSEo4cOYK9e/fi2rVrmDlzJoxGo12xsaxKRESyLMjJysrC/PnzkZiYCADIycnBzp07kZeXhyVLlnTon5eXh8uXL+Pw4cPw9vYGAISFhVn0KSwstHidn5+PgIAAlJWV4cEHH7Q5Ns4ciYgIEJTOaQCamposWmtra4ePM5lMKCsrg06nk44plUrodDqUlJR0GuKOHTug1WqRlJQEjUaDcePGISMjA2az+aZfq7GxEQAwePBgu34cTI5ERORUoaGh8Pf3l1pmZmaHPvX19TCbzdBoNBbHNRoNDAZDp+NWVlaioKAAZrMZu3btwtKlS7Fq1SqsWLGi0/6CIODZZ5/FlClTMG7cOLu+g0skR3svyLbbtGkTFAoF5syZ070BEhF5uvbVqo42ANXV1WhsbJRaamqqc0IUBAQEBGDt2rWIiopCXFwcnn/+eeTk5HTaPykpCSdPnsSmTZvs/izZk6O9F2TbnT9/Hs899xweeOCBHoqUiMhziaLCKQ0A/Pz8LJpare7weUOGDIGXlxdqa2stjtfW1iIwMLDTGIOCgjBq1Ch4eXlJx8aOHQuDwQCTyWTRNzk5GR9++CH279+PoUOH2v3zkD05/vCCbHh4OHJycuDr64u8vLybnmM2mzFv3jwsW7YMd911Vw9GS0REzqBSqRAVFYWioiLpmCAIKCoqglar7fScKVOm4OzZsxCE71f+nD59GkFBQVCpVAAAURSRnJyMbdu2Yd++fRgxYkSX4pM1OXblgiwA/PWvf0VAQACeeuqp235Ga2trh4vDRERkxYkLcmyl1+uRm5uLDRs24NSpU1i4cCGMRqO0ejU+Pt6iJLtw4UJcvnwZKSkpOH36NHbu3ImMjAwkJSVJfZKSkrBx40a88847GDBgAAwGAwwGA65cuWJXbLLeynGrC7JffPFFp+ccOnQIb775JsrLy236jMzMTCxbtszRUImIPJoowAm3cti383hcXBwuXbqEtLQ0GAwGTJgwAYWFhVJOqKqqglL5fcINDQ3Fnj17sGjRIowfPx4hISFISUnB4sWLpT5r1qwBAEydOtXis9avX4/f/va3NsfmVvc5Njc34ze/+Q1yc3MxZMgQm85JTU2FXq+XXjc1NSE0NLS7QiQiIjskJycjOTm50/eKi4s7HNNqtThy5MhNxxNF5zwaRNbkaO8F2S+//BLnz59HbGysdKy99tynTx9UVFTgRz/6kcU5arW604vBRET0A055ZJWD57sQWa852ntBdsyYMThx4gTKy8ul9rOf/QzTpk1DeXk5Z4RERF3kzNWqnkD2sqper0dCQgImTpyISZMmYfXq1R0uyIaEhCAzMxM+Pj4dbuQcOHAgANh9gycREdHNyJ4c7b0gS0RE3aALq007juGcUFyB7MkRsP+C7A/l5+c7PyAiol7GORuPe05ZlVMyIiIiKy4xcyQiInk5Y0ENF+QQEZFn4TVHCyyrEhERWeHMkYiIuCDHCpMjERHxmqMVllWJiIiscOZIRERckGOFyZGIiHjN0QrLqkRERFY4cyQiIi7IscLkSEREgOiEa47Oec6wS2BZlYiIyApnjkRExAU5VpgciYgIouj4NUPRg8qqvTY5iuZWiGa5o7g93+BLcodgk5orKrlDsFnBiQi5Q7CZfvJ/5A7BJkqF+8wY/hJ9Uu4QbNJy/Tre+5fcUfRevTY5EhHRDzihrAqWVYmIyJOIohKi6NgaTdGD6qpcrUpERGSFM0ciImoribKsKmFyJCIi7pBjhWVVIiIiK5w5EhERNwGwwuRIRERcrWqFZVUiIiIrnDkSERHLqlaYHImIiKtVrbCsSkREZIUzRyIi4szRCpMjERG1JUdHrzl6UHJkWZWIiMgKZ45ERMT7HK24xMwxOzsbYWFh8PHxQXR0NI4ePXrL/g0NDUhKSkJQUBDUajVGjRqFXbt29VC0RESep/1WDkebp5B95rh582bo9Xrk5OQgOjoaq1evRkxMDCoqKhAQENChv8lkwowZMxAQEICCggKEhITgwoULGDhwYM8HT0REHkn25JiVlYX58+cjMTERAJCTk4OdO3ciLy8PS5Ys6dA/Ly8Ply9fxuHDh+Ht7Q0ACAsL68mQiYg8DlerWpK1rGoymVBWVgadTicdUyqV0Ol0KCkp6fScHTt2QKvVIikpCRqNBuPGjUNGRgbMZnOn/VtbW9HU1GTRiIjIUntydLR5ClmTY319PcxmMzQajcVxjUYDg8HQ6TmVlZUoKCiA2WzGrl27sHTpUqxatQorVqzotH9mZib8/f2lFhoa6vTvQUREnsUlFuTYQxAEBAQEYO3atYiKikJcXByef/555OTkdNo/NTUVjY2NUquuru7hiImIXJ8oOGNRjtzfwnlkveY4ZMgQeHl5oba21uJ4bW0tAgMDOz0nKCgI3t7e8PLyko6NHTsWBoMBJpMJKpXKor9arYZarXZ+8EREHoTXHC3JOnNUqVSIiopCUVGRdEwQBBQVFUGr1XZ6zpQpU3D27FkIwvf/RDl9+jSCgoI6JEYiIqKukL2sqtfrkZubiw0bNuDUqVNYuHAhjEajtHo1Pj4eqampUv+FCxfi8uXLSElJwenTp7Fz505kZGQgKSlJrq9AROT22jcBcLR5Ctlv5YiLi8OlS5eQlpYGg8GACRMmoLCwUFqkU1VVBaXy+x94aGgo9uzZg0WLFmH8+PEICQlBSkoKFi9eLNdXICJye4KogOBgWdTR812J7MkRAJKTk5GcnNzpe8XFxR2OabVaHDlypJujIiKi3spz5sBERNR1ztg6rgvbxzl7+9CDBw8iNjYWwcHBUCgU2L59u90xAUyOREQEeTYBaN8+ND09HceOHUNERARiYmJQV1fXaf/27UPPnz+PgoICVFRUIDc3FyEhIVIfo9GIiIgIZGdnO/TzcImyKhER9T7dsX3orFmzMGvWLIdj48yRiIh6fObYE9uHOoIzRyIicuomANZ7WHe2Gcuttg/94osvOh2/srIS+/btw7x587Br1y6cPXsWzzzzDK5du4b09HSHYrfGmSMRETlVaGioxZ7WmZmZThnX3u1DHcGZIxERQRCVEBy8ib/9/Orqavj5+UnHO9vCsye2D3UEZ45ERNRWVnX0do4bZVU/Pz+L1llydPXtQ5kciYhIFt2xfWhLSwvKy8tRXl4OADh37hzKy8tRVVVlV2wsqxIRkSxP5eiO7UNLS0sxbdo06bVerwcAJCQkID8/3+bYmByJiEi2R1Y5e/vQqVOnQhRFu+OwxrIqERGRFc4ciYiIT+WwwuRIRESylVVdFcuqREREVnrtzLGPjwZ9fHzkDuO2vPpdlTsEmzRfd59/Z33aIHcEtvMa5vjCgp7wueJzuUOw2bufjZc7BJu0CiYA/9djn8eZo6VemxyJiOh7vOZoyX3+uU9ERNRD7E6O165dw/Tp03HmzJnuiIeIiGQgis54bJXc38J57C6rent749NPP+2OWIiISCa85mipS2XVX//613jzzTedHQsREZFL6NKCnOvXryMvLw//+te/EBUVhX79+lm8n5WV5ZTgiIioZ4hOWJDjSTPHLiXHkydP4sc//jGAtseF/JBC4Tk/HCKi3oJlVUtdSo779+93dhxEREQug/c5EhERZ45WupwcS0tLsWXLFlRVVcFkMlm8t3XrVocDIyKinsNNACx1abXqpk2bMHnyZJw6dQrbtm3DtWvX8Nlnn2Hfvn3w9/d3doxEREQ9qkvJMSMjA//4xz/wwQcfQKVS4dVXX8UXX3yBJ554AsOGDXN2jERE1M0c3wDA8bKsK+lScvzyyy8xe/ZsAIBKpYLRaIRCocCiRYuwdu1apwZIRETdr72s6mjzFF1KjoMGDUJzczMAICQkBCdPngQANDQ04LvvvnNedERERDLo0oKcBx98EHv37sW9996LX/7yl0hJScG+ffuwd+9eTJ8+3dkxEhFRNxOhgAgHV6s6eL4r6VJy/J//+R9cvdr2nMHnn38e3t7eOHz4MH7+85/jhRdecGqARETU/Xgrh6UulVUHDx6M4ODgtgGUSixZsgQ7duzAqlWrMGjQILvHy87ORlhYGHx8fBAdHY2jR4/esv/q1asxevRo9O3bF6GhoVi0aJGUrImIiBzV5ec5fvnll3jhhRcwd+5c1NXVAQB2796Nzz77zK5xNm/eDL1ej/T0dBw7dgwRERGIiYmRxrT2zjvvYMmSJUhPT8epU6fw5ptvYvPmzfjLX/7S1a9CRNTrcUGOpS4lxwMHDuDee+/Fv//9b2zduhUtLS0AgP/85z9IT0+3a6ysrCzMnz8fiYmJCA8PR05ODnx9fZGXl9dp/8OHD2PKlCl48sknERYWhpkzZ2Lu3Lm3nW0SEdHN8VYOS11KjkuWLMGKFSuwd+9eqFQq6fhPf/pTHDlyxOZxTCYTysrKoNPpvg9IqYROp0NJSUmn50yePBllZWVSMqysrMSuXbvwyCOPdNq/tbUVTU1NFo2IiOhWurQg58SJE3jnnXc6HA8ICEB9fb3N49TX18NsNkOj0Vgc12g0+OKLLzo958knn0R9fT3uv/9+iKKI69evY8GCBTctq2ZmZmLZsmU2x0RE1BsJcML2cR60WrVLM8eBAweipqamw/Hjx48jJCTE4aBupbi4GBkZGXj99ddx7NgxbN26FTt37sTy5cs77Z+amorGxkapVVdXd2t8RETuiGVVS12aOf7qV7/C4sWL8d5770GhUEAQBHz88cd47rnnEB8fb/M4Q4YMgZeXF2pray2O19bWIjAwsNNzli5dit/85jd4+umnAQD33nsvjEYjfv/73+P555+HUmmZ79VqNdRqtZ3fkIiIerMu7606ZswYhIaGoqWlBeHh4XjwwQcxefJku+5zVKlUiIqKQlFRkXRMEAQUFRVBq9V2es53333XIQF6eXkBAERR7MK3ISIiAQqnNE/RpZmjSqVCbm4uli5dipMnT6KlpQWRkZG4++677R5Lr9cjISEBEydOxKRJk7B69WoYjUYkJiYCAOLj4xESEoLMzEwAQGxsLLKyshAZGYno6GicPXsWS5cuRWxsrJQkiYjITs4oi/b2smq7YcOGOfwUjri4OFy6dAlpaWkwGAyYMGECCgsLpUU6VVVVFjPFF154AQqFAi+88AIuXryIO++8E7GxsVi5cqVDcRAREbXrUnI0m83Iz89HUVER6urqIAiCxfv79u2za7zk5GQkJyd3+l5xcbHF6z59+iA9Pd3u+ymJiOjm+LBjS11KjikpKcjPz8fs2bMxbtw4KBSe8wMhIuqNuLeqpS4lx02bNmHLli03vfGeiIjInXV5Qc7IkSOdHQsREclEuNEcHcNTdOlWjj/96U949dVXeesEEZGH4CYAlmyeOT7++OMWr/ft24fdu3fjnnvugbe3t8V7W7dudU50REREMrA5Ofr7+1u8fuyxx5weDBERyUMQHV9tKnhQMdHm5Lh+/Xrpz1euXIEgCOjXrx8A4Pz589i+fTvGjh2LmJgY50dJRETdSoQCooM73Dh6vivp0jXHRx99FP/7v/8LAGhoaMB9992HVatWYc6cOVizZo1TAyQiIuppXUqOx44dwwMPPAAAKCgogEajwYULF/DWW2/htddec2qARETU/do3AXC0eYou3crx3XffYcCAAQCAjz76CI8//jiUSiXuu+8+XLhwwakBEhFR92u75uj4GJ6iSzPHkSNHYvv27aiursaePXswc+ZMAEBdXR38/PycGiAREVFP61JyTEtLw3PPPYewsDBER0dLj5f66KOPEBkZ6dQAiYio+7UvyHG0eYoulVV/8Ytf4P7770dNTQ0iIiKk49OnT+ctHkREbogbj1vq8iOrAgMDERgYaHFs0qRJDgdEREQkN4ee5+jOrhvP47qXWu4wbku4NFDuEGwyd9RZuUOw2ZYzP5I7BJuZL3TpykePi8A4uUOwmZebTG56Ok5RbGuOjuEpem1yJCKi74lQQOAmABL3+GcpERFRD2JyJCIi2Z7KkZ2djbCwMPj4+CA6OhpHjx69Zf+GhgYkJSUhKCgIarUao0aNwq5duxwaszNMjkREJMsOOZs3b4Zer0d6ejqOHTuGiIgIxMTEoK6urtP+JpMJM2bMwPnz51FQUICKigrk5uYiJCSky2PeDJMjERHJIisrC/Pnz0diYiLCw8ORk5MDX19f5OXlddo/Ly8Ply9fxvbt2zFlyhSEhYXhoYcesril0N4xb4bJkYiIIDqpAUBTU5NFa21t7fB5JpMJZWVl0Ol00jGlUgmdToeSkpJOY9yxYwe0Wi2SkpKg0Wgwbtw4ZGRkwGw2d3nMm2FyJCIip5ZVQ0ND4e/vL7XMzMwOn1dfXw+z2QyNRmNxXKPRwGAwdBpjZWUlCgoKYDabsWvXLixduhSrVq3CihUrujzmzfBWDiIicqrq6mqLfbbVaufcUy4IAgICArB27Vp4eXkhKioKFy9exMsvv4z09HSnfEY7JkciIoJwozk6BgD4+fnd9iEUQ4YMgZeXF2pray2O19bWdth9rV1QUBC8vb3h5eUlHRs7diwMBgNMJlOXxrwZllWJiKjHb+VQqVSIiopCUVGRdEwQBBQVFUkPs7A2ZcoUnD17FoLwfRo/ffo0goKCoFKpujTmzTA5EhGRLPR6PXJzc7FhwwacOnUKCxcuhNFoRGJiIgAgPj4eqampUv+FCxfi8uXLSElJwenTp7Fz505kZGQgKSnJ5jFtxbIqERHJ8lSOuLg4XLp0CWlpaTAYDJgwYQIKCwulBTVVVVVQKr+fw4WGhmLPnj1YtGgRxo8fj5CQEKSkpGDx4sU2j2krJkciIrK4FcORMeyVnJyM5OTkTt8rLi7ucEyr1eLIkSNdHtNWLKsSERFZ4cyRiIj4sGMrTI5EROTUWzk8gaxl1YMHDyI2NhbBwcFQKBTYvn37bc8pLi7Gj3/8Y6jVaowcORL5+fndHicREfUusiZHo9GIiIgIZGdn29T/3LlzmD17NqZNm4by8nI8++yzePrpp7Fnz55ujpSIyLPJ9cgqVyVrWXXWrFmYNWuWzf1zcnIwYsQIrFq1CkDbzgiHDh3CP/7xD8TExHRXmEREHk+E42VRR1e7uhK3Wq1aUlJisds6AMTExNxyt/XW1tYOO8QTERHdilslR4PB0Olu601NTbhy5Uqn52RmZlrsDh8aGtoToRIRuRURTiirwnPKqm6VHLsiNTUVjY2NUquurpY7JCIilyOIzmmewq1u5QgMDOx0t3U/Pz/07du303PUarXTHpdCRES9g1vNHLVarcVu6wCwd+9eu3dbJyIiS6KTmqeQNTm2tLSgvLwc5eXlANpu1SgvL0dVVRWAtpJofHy81H/BggWorKzEf//3f+OLL77A66+/ji1btmDRokVyhE9E5DHad8hxtHkKWZNjaWkpIiMjERkZCaDtUSORkZFIS0sDANTU1EiJEgBGjBiBnTt3Yu/evYiIiMCqVauwbt063sZBREROJes1x6lTp0IUbz4R72z3m6lTp+L48ePdGBURUe/D7eMsudWCHCIi6h7O2OHGk3bIcasFOURERD2BM0ciImJZ1QqTIxERQRTbmqNjeAqWVYmIiKxw5khERBCggODg3qiOnu9KmByJiMgpe6N60t6qLKsSERFZ4cyRiIgAJyzI8aTNVZkciYiI1xytsKxKRERkhTNHIiLifY5WmByJiIg75FhhWZWIiMgKZ45ERMT7HK303uR43QhcuyZ3FLc14G91codgk0b9CblDsNknR4fKHYLNxq2YJncINhmtcJ9ViqnzN8odgk2aror428qe+zwRjt+J4UG5kWVVIiIia7135khERJK2sqqD9zl60NSRyZGIiHgrhxWWVYmIiKxw5khERLzP0QqTIxERsaxqhWVVIiIiK5w5EhERy6pWmByJiAiiE3bIYVmViIjIg3HmSERE3D7OCpMjERFx43ErLKsSERFZ4cyRiIh4n6MVJkciIuKtHFZYViUiIrIia3I8ePAgYmNjERwcDIVCge3bt9+y/9atWzFjxgzceeed8PPzg1arxZ49e3omWCIiD9a+IMfR5ilkTY5GoxERERHIzs62qf/BgwcxY8YM7Nq1C2VlZZg2bRpiY2Nx/Pjxbo6UiMiziU5qnkLWa46zZs3CrFmzbO6/evVqi9cZGRl4//338cEHHyAyMtLJ0RERUW/l1tccBUFAc3MzBg8efNM+ra2taGpqsmhERGRJrrJqdnY2wsLC4OPjg+joaBw9evSmffPz86FQKCyaj4+PRZ/a2lr89re/RXBwMHx9ffHwww/jzJkzdsfl1snxlVdeQUtLC5544omb9snMzIS/v7/UQkNDezBCIiL30H4rh6PNHps3b4Zer0d6ejqOHTuGiIgIxMTEoK6u7qbn+Pn5oaamRmoXLlz4wXcQMWfOHFRWVuL999/H8ePHMXz4cOh0OhiNRrtic9vk+M4772DZsmXYsmULAgICbtovNTUVjY2NUquuru7BKImI6GaysrIwf/58JCYmIjw8HDk5OfD19UVeXt5Nz1EoFAgMDJSaRqOR3jtz5gyOHDmCNWvW4Cc/+QlGjx6NNWvW4MqVK3j33Xftis0tk+OmTZvw9NNPY8uWLdDpdLfsq1ar4efnZ9GIiMiS4KQGoMOlrNbW1g6fZzKZUFZWZvF3uFKphE6nQ0lJyU3jbGlpwfDhwxEaGopHH30Un332mfRe++f8sNSqVCqhVqtx6NAhu34ebpcc3333XSQmJuLdd9/F7Nmz5Q6HiMgjCHDCNccbY4WGhlpczsrMzOzwefX19TCbzRYzPwDQaDQwGAydxjh69Gjk5eXh/fffx8aNGyEIAiZPnoyvvvoKADBmzBgMGzYMqamp+Pbbb2EymfC3v/0NX331FWpqauz6eci6WrWlpQVnz56VXp87dw7l5eUYPHiw9AUvXryIt956C0BbKTUhIQGvvvoqoqOjpR9g37594e/vL8t3ICIiS9XV1RZVOrVa7ZRxtVottFqt9Hry5MkYO3Ys3njjDSxfvhze3t7YunUrnnrqKQwePBheXl7Q6XSYNWsWRDsviMo6cywtLUVkZKR0G4Zer0dkZCTS0tIAADU1NaiqqpL6r127FtevX0dSUhKCgoKklpKSIkv8RESewpn3OVpfyuosOQ4ZMgReXl6ora21OF5bW4vAwECbYvb29kZkZKTFJCsqKgrl5eVoaGhATU0NCgsL8c033+Cuu+6y9UcBQOaZ49SpU2+ZzfPz8y1eFxcXd29ARES9lPiDsqgjY9hKpVIhKioKRUVFmDNnDoC22/OKioqQnJxs0xhmsxknTpzAI4880uG99mrimTNnUFpaiuXLl9seHLjxOBERyUSv1yMhIQETJ07EpEmTsHr1ahiNRiQmJgIA4uPjERISIl2z/Otf/4r77rsPI0eORENDA15++WVcuHABTz/9tDTme++9hzvvvBPDhg3DiRMnkJKSgjlz5mDmzJl2xcbkSEREbfcpOmEMe8TFxeHSpUtIS0uDwWDAhAkTUFhYKC3SqaqqglL5/dW/b7/9FvPnz4fBYMCgQYMQFRWFw4cPIzw8XOpTU1MDvV6P2tpaBAUFIT4+HkuXLrX7uyhEe69Surmmpib4+/uj+kw8/Aao5A7ntvwD18sdgk0a9cFyh2Cz2DW2b1kot6+Vna/aczWjFe6zuUbB4rflDsEmTVdFaFa23Z/dnbegtf+d+Lj//4O3wrGFM9fEVmxtfKPbY+4JbncrBxERUXdjWZWIiG7cp+hYIdGTHlnF5EhERE555JQH5UaWVYmIiKxx5khERBbbvzkyhqdgciQiIog3/nN0DE/BsioREZEVzhyJiIhlVStMjkREZPE8RkfG8BQsqxIREVnhzJGIiCCKTliQ40G7kTI5EhERy6pWem1yVPiGQNHPR+4wbqvkQffYJHvHnjvlDsFmjm6R1ZNeGD5E7hBsEtjvktwh2OybUyPkDsEmzdfMAD6VO4xeq9cmRyIi+h7LqpaYHImICCIcL4t6TmrkalUiIqIOOHMkIiIIouiER1Z5ztyRyZGIiLi3qhWWVYmIiKxw5khERLzP0QqTIxERQYATrjmyrEpEROS5OHMkIiKuVrXC5EhERFytaoVlVSIiIiucORIRERfkWGFyJCIiJkcrLKsSERFZ4cyRiIi4IMeKrDPHgwcPIjY2FsHBwVAoFNi+fbvN53788cfo06cPJkyY0G3xERH1FuKNsqojjcnRSYxGIyIiIpCdnW3XeQ0NDYiPj8f06dO7KTIiIurNZC2rzpo1C7NmzbL7vAULFuDJJ5+El5eXXbNNIiLqnKAQoFA4tjuq4EG7q7rdgpz169ejsrIS6enpNvVvbW1FU1OTRSMiIkuOllSdsdrVlbhVcjxz5gyWLFmCjRs3ok8f2ya9mZmZ8Pf3l1poaGg3R0lERO7ObZKj2WzGk08+iWXLlmHUqFE2n5eamorGxkapVVdXd2OURETuyTnzRs8pq7rNrRzNzc0oLS3F8ePHkZycDAAQBAGiKKJPnz746KOP8NOf/rTDeWq1Gmq1uqfDJSJyKwIAhcObAHgOt0mOfn5+OHHihMWx119/Hfv27UNBQQFGjBghU2RERORpZE2OLS0tOHv2rPT63LlzKC8vx+DBgzFs2DCkpqbi4sWLeOutt6BUKjFu3DiL8wMCAuDj49PhOBER2YerVS3JmhxLS0sxbdo06bVerwcAJCQkID8/HzU1NaiqqpIrPCKiXkOAAIWDyY3J0UmmTp0K8RYPx8zPz7/l+S+++CJefPFF5wZFRES9nttccyQiou7DmaMlJkciInLKrRiedCuH29znSERE1FM4cyQiIq5WtcLkSEREECE4nNxYViUiIvJgnDkSERFEmCE6OF8SYXZSNPJjciQiohslVV5zbMeyKhERkRUmRyIiunGXo6P/2f9Uj+zsbISFhcHHxwfR0dE4evToTfvm5+dDoVBYNB8fH4s+LS0tSE5OxtChQ9G3b1+Eh4cjJyfH7rhYViUiohvXHBUOj2GPzZs3Q6/XIycnB9HR0Vi9ejViYmJQUVGBgICATs/x8/NDRUWF9FqhsIxZr9dj37592LhxI8LCwvDRRx/hmWeeQXBwMH72s5/ZHBtnjkREJIusrCzMnz8fiYmJ0gzP19cXeXl5Nz1HoVAgMDBQahqNxuL9w4cPIyEhAVOnTkVYWBh+//vfIyIi4pYz0s4wORIRkRNKqt/fJ9nU1GTRWltbO3yeyWRCWVkZdDqddEypVEKn06GkpOSmcba0tGD48OEIDQ3Fo48+is8++8zi/cmTJ2PHjh24ePEiRFHE/v37cfr0acycOdOunweTIxERSXurOtoAIDQ0FP7+/lLLzMzs8Hn19fUwm80dZn4ajQYGg6HTGEePHo28vDy8//772LhxIwRBwOTJk/HVV19Jff75z38iPDwcQ4cOhUqlwsMPP4zs7Gw8+OCDdv08eu01R99+I+Dbz1fuMG7Lr99+uUOwyRil+9zflN6/n9wh2Gz5p4PlDsEmY/oHyh2CzYprNLfv5AJaBROAT+UOo0uqq6vh5+cnvVar1U4ZV6vVQqvVSq8nT56MsWPH4o033sDy5csBtCXHI0eOYMeOHRg+fDgOHjyIpKQkBAcHW8xSb6fXJkciIvqeADPg4IIc4caCHD8/P4vk2JkhQ4bAy8sLtbW1Fsdra2sRGGjbP7a8vb0RGRmJs2fPAgCuXLmCv/zlL9i2bRtmz54NABg/fjzKy8vxyiuv2JUcWVYlIiKnllVtoVKpEBUVhaKiIumYIAgoKiqymB3eitlsxokTJxAUFAQAuHbtGq5duwal0jK1eXl5QRDs26CAM0ciIpKFXq9HQkICJk6ciEmTJmH16tUwGo1ITEwEAMTHxyMkJES6ZvnXv/4V9913H0aOHImGhga8/PLLuHDhAp5++mkAbTPWhx56CH/+85/Rt29fDB8+HAcOHMBbb72FrKwsu2JjciQiIgiiE8qqon1rD+Li4nDp0iWkpaXBYDBgwoQJKCwslBbpVFVVWcwCv/32W8yfPx8GgwGDBg1CVFQUDh8+jPDwcKnPpk2bkJqainnz5uHy5csYPnw4Vq5ciQULFtgVm0IURfu3NHBjTU1N8Pf3x+WGtfDzc/0FORWz18kdgk2+u9JX7hBs1nCFC3KcbUx/n9t3chGD1e7xV16rYMI/vn4DjY2Nt71+54j2vxPv8I2CUuHYfEkQr+Ob78q6PeaewGuOREREVlhWJSKiGwtqHLsly5MedszkSEREEEUBgqN7q4qekxxZViUiIrLCmSMREd0oiTr6VA7PmTkyORIREUQ7b8PorjFcBcuqREREVjhzJCIitC3HYVm1HZMjERHdWGnK1artWFYlIiKywpkjERE5vAGAs8ZwFbLOHA8ePIjY2FgEBwdDoVBg+/bttz2ntbUVzz//PIYPHw61Wo2wsDDk5eV1f7BERB5MFEWIouBgc499a20h68zRaDQiIiICv/vd7/D444/bdM4TTzyB2tpavPnmmxg5ciRqamrsfk4XERHRrciaHGfNmoVZs2bZ3L+wsBAHDhxAZWUlBg9ue1pBWFhYN0VHRNR7OGOlqSetVnWrBTk7duzAxIkT8fe//x0hISEYNWoUnnvuOVy5cuWm57S2tqKpqcmiERGRJVE0O6V5CrdakFNZWYlDhw7Bx8cH27ZtQ319PZ555hl88803WL9+fafnZGZmYtmyZT0cKRERuTO3mjkKggCFQoG3334bkyZNwiOPPIKsrCxs2LDhprPH1NRUNDY2Sq26urqHoyYicn2OL8YRPOo+R7eaOQYFBSEkJAT+/v7SsbFjx0IURXz11Ve4++67O5yjVquhVqt7MkwiIrfDa46W3GrmOGXKFHz99ddoaWmRjp0+fRpKpRJDhw6VMTIiIvIksibHlpYWlJeXo7y8HABw7tw5lJeXo6qqCkBbSTQ+Pl7q/+STT+KOO+5AYmIiPv/8cxw8eBB//vOf8bvf/Q59+/aV4ysQEXkEllUtyZocS0tLERkZicjISACAXq9HZGQk0tLSAAA1NTVSogSA/v37Y+/evWhoaMDEiRMxb948xMbG4rXXXpMlfiIiTyFCcErzFLJec5w6deotd1TIz8/vcGzMmDHYu3dvN0ZFRES9nVstyCEiou7Rdo+iY9u/eVJZlcmRiIjQlhgdTW6es7eqW61WJSIi6gmcORIRkZMeduw5M0cmRyIiurHS1MHkyLIqERGR5+LMkYiIACfMHD1pQQ6TIxERAU645ggPuubIsioREZEVzhyJiIgLcqwwORIREXjN0RLLqkRERFY4cyQiIgCiEyZ+njNzZHIkIiI454ohk6Pbat/eqKnpisyR2Kbl2nW5Q7DJd9evyR2CzYxuFOt10SR3CDYxCY5eq+o5rW7y4IhWoe3/fc9uyeY5yc1RvS45Njc3AwDChqXIHAmR5zh8Ve4IPFdzczP8/f27bXyVSoXAwEAYDAanjBcYGAiVSuWUseSkED1pp1gbCIKAr7/+GgMGDIBC4bx/7TY1NSE0NBTV1dXw8/Nz2rjO5i5xAoy1O7hLnABjFUURzc3NCA4OhlLZvWsnr169CpPJOVUKlUoFHx8fp4wlp143c1QqlRg6dGi3je/n5+fyv8iA+8QJMNbu4C5xAr071u6cMf6Qj4+PRyQ0Z+KtHERERFaYHImIiKwwOTqJWq1Geno61Gq13KHckrvECTDW7uAucQKMleTV6xbkEBER3Q5njkRERFaYHImIiKwwORIREVlhciQiIrLC5OgE2dnZCAsLg4+PD6Kjo3H06FG5Q8LBgwcRGxuL4OBgKBQKbN++3eJ9URSRlpaGoKAg9O3bFzqdDmfOnOnxODMzM/GTn/wEAwYMQEBAAObMmYOKigqLPlevXkVSUhLuuOMO9O/fHz//+c9RW1vb47GuWbMG48ePl2701mq12L17t8vFae2ll16CQqHAs88+Kx1zlVhffPFFKBQKizZmzBiXi7PdxYsX8etf/xp33HEH+vbti3vvvRelpaXS+67ye0WOY3J00ObNm6HX65Geno5jx44hIiICMTExqKurkzUuo9GIiIgIZGdnd/r+3//+d7z22mvIycnBv//9b/Tr1w8xMTG4erVnN8k8cOAAkpKScOTIEezduxfXrl3DzJkzYTQapT6LFi3CBx98gPfeew8HDhzA119/jccff7xH4wSAoUOH4qWXXkJZWRlKS0vx05/+FI8++ig+++wzl4rzhz755BO88cYbGD9+vMVxV4r1nnvuQU1NjdQOHTrkknF+++23mDJlCry9vbF79258/vnnWLVqFQYNGiT1cZXfK3ICkRwyadIkMSkpSXptNpvF4OBgMTMzU8aoLAEQt23bJr0WBEEMDAwUX375ZelYQ0ODqFarxXfffVeGCL9XV1cnAhAPHDggxeXt7S2+9957Up9Tp06JAMSSkhK5wpQMGjRIXLdunUvG2dzcLN59993i3r17xYceekhMSUkRRdG1fqbp6eliREREp++5UpyiKIqLFy8W77///pu+78q/V2Q/zhwdYDKZUFZWBp1OJx1TKpXQ6XQoKSmRMbJbO3fuHAwGg0Xc/v7+iI6Olj3uxsZGAMDgwYMBAGVlZbh27ZpFrGPGjMGwYcNkjdVsNmPTpk0wGo3QarUuGWdSUhJmz55tERPgej/TM2fOIDg4GHfddRfmzZuHqqoql4xzx44dmDhxIn75y18iICAAkZGRyM3Nld535d8rsh+TowPq6+thNpuh0Wgsjms0Gqc9/qU7tMfmanELgoBnn30WU6ZMwbhx4wC0xapSqTBw4ECLvnLFeuLECfTv3x9qtRoLFizAtm3bEB4e7nJxbtq0CceOHUNmZmaH91wp1ujoaOTn56OwsBBr1qzBuXPn8MADD6C5udml4gSAyspKrFmzBnfffTf27NmDhQsX4o9//CM2bNgAwHV/r6hret1TOch1JSUl4eTJkxbXnFzN6NGjUV5ejsbGRhQUFCAhIQEHDhyQOywL1dXVSElJwd69e13+SQuzZs2S/jx+/HhER0dj+PDh2LJlC/r27StjZB0JgoCJEyciIyMDABAZGYmTJ08iJycHCQkJMkdHzsaZowOGDBkCLy+vDqvnamtrERgYKFNUt9cemyvFnZycjA8//BD79++3eKRYYGAgTCYTGhoaLPrLFatKpcLIkSMRFRWFzMxMRERE4NVXX3WpOMvKylBXV4cf//jH6NOnD/r06YMDBw7gtddeQ58+faDRaFwmVmsDBw7EqFGjcPbsWZf6mQJAUFAQwsPDLY6NHTtWKgO74u8VdR2TowNUKhWioqJQVFQkHRMEAUVFRdBqtTJGdmsjRoxAYGCgRdxNTU3497//3eNxi6KI5ORkbNu2Dfv27cOIESMs3o+KioK3t7dFrBUVFaiqqnKJn7EgCGhtbXWpOKdPn44TJ06gvLxcahMnTsS8efOkP7tKrNZaWlrw5ZdfIigoyKV+pgAwZcqUDrcZnT59GsOHDwfgWr9X5ARyrwhyd5s2bRLVarWYn58vfv755+Lvf/97ceDAgaLBYJA1rubmZvH48ePi8ePHRQBiVlaWePz4cfHChQuiKIriSy+9JA4cOFB8//33xU8//VR89NFHxREjRohXrlzp0TgXLlwo+vv7i8XFxWJNTY3UvvvuO6nPggULxGHDhon79u0TS0tLRa1WK2q12h6NUxRFccmSJeKBAwfEc+fOiZ9++qm4ZMkSUaFQiB999JFLxdmZH65WFUXXifVPf/qTWFxcLJ47d078+OOPRZ1OJw4ZMkSsq6tzqThFURSPHj0q9unTR1y5cqV45swZ8e233xZ9fX3FjRs3Sn1c5feKHMfk6AT//Oc/xWHDhokqlUqcNGmSeOTIEblDEvfv3y8C6NASEhJEUWxbdr506VJRo9GIarVanD59ulhRUdHjcXYWIwBx/fr1Up8rV66IzzzzjDho0CDR19dXfOyxx8Sampoej/V3v/udOHz4cFGlUol33nmnOH36dCkxulKcnbFOjq4Sa1xcnBgUFCSqVCoxJCREjIuLE8+ePetycbb74IMPxHHjxolqtVocM2aMuHbtWov3XeX3ihzHR1YRERFZ4TVHIiIiK0yOREREVpgciYiIrDA5EhERWWFyJCIissLkSEREZIXJkYiIyAqTI1EPePHFFzFhwgS5wyAiGzE5EhERWWFyJCIissLkSL1eYWEh7r//fgwcOBB33HEH/uu//gtffvklAOD8+fNQKBTYunUrpk2bBl9fX0RERHR4sntubi5CQ0Ph6+uLxx57DFlZWR0e0mtt3bp1GDt2LHx8fDBmzBi8/vrr3fUVichOTI7U6xmNRuj1epSWlqKoqAhKpRKPPfYYBEGQ+jz//PN47rnnUF5ejlGjRmHu3Lm4fv06AODjjz/GggULkJKSgvLycsyYMQMrV6685We+/fbbSEtLw8qVK3Hq1ClkZGRg6dKl0lPliUhe3HicyEp9fT3uvPNOnDhxAv3798eIESOwbt06PPXUUwCAzz//HPfccw9OnTqFMWPG4Fe/+hVaWlrw4YcfSmP8+te/xocffig9qPfFF1/E9u3bUV5eDgAYOXIkli9fjrlz50rnrFixArt27cLhw4d77LsSUec4c6Re78yZM5g7dy7uuusu+Pn5ISwsDACkJ7wDwPjx46U/BwUFAQDq6uoAtD2Ad9KkSRZjWr/+IaPRiC+//BJPPfUU+vfvL7UVK1ZI5VwiklcfuQMgkltsbCyGDx+O3NxcBAcHQxAEjBs3DiaTSerj7e0t/VmhUACARdnVHi0tLQDarlNGR0dbvOfl5dWlMYnIuZgcqVf75ptvUFFRgdzcXDzwwAMAgEOHDtk1xujRo/HJJ59YHLN+/UMajQbBwcGorKzEvHnz7A+aiLodkyP1aoMGDcIdd9yBtWvXIigoCFVVVViyZIldY/zhD3/Agw8+iKysLMTGxmLfvn3YvXu3NMPszLJly/DHP/4R/v7+ePjhh9Ha2orS0lJ8++230Ov1jn4tInIQrzlSr6ZUKrFp0yaUlZVh3LhxWLRoEV5++WW7xpgyZQpycnKQlZWFiIgIFBYWYtGiRfDx8bnpOU8//TTWrVuH9evX495778VDDz2E/Px8jBgxwtGvREROwNWqRN1g/vz5+OKLL/B///d/codCRF3AsiqRE7zyyiuYMWMG+vXrh927d2PDhg28qZ/IjXHmSOQETzzxBIqLi9Hc3Iy77roLf/jDH7BgwQK5wyKiLmJyJCIissIFOURERFaYHImIiKwwORIREVlhciQiIrLC5EhERGSFyZGIiMgKkyMREZEVJkciIiIrTI5ERERW/j/yy/8hMKoLuQAAAABJRU5ErkJggg==\n"
          },
          "metadata": {}
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "OHcZWJiFJDMh"
      },
      "source": [
        "It seems that rotation and shear don't mix! The best accuracy is when rotation is zero."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "PAasQo1C3x4A"
      },
      "source": [
        "## Questions\n",
        "Try these questions for better understanding. You may not be able to solve all of them.\n",
        "1. What is the best value for angle constraint and shear constraint you got? How much did the accuracy improve as compared to not using augmentations?\n",
        "2. Can you increase the accuracy by increasing the number of augmentations from each sample?\n",
        "3. Try implementing a few augmentations of your own and experimenting with them. A good reference is <a href=https://www.analyticsvidhya.com/blog/2019/12/image-augmentation-deep-learning-pytorch/>here. </a>\n",
        "4. Try combining various augmentations. What is the highest accuracy you can get? What is the smallest training dataset you can take and still get accuracy above 50%?\n",
        "\n",
        "Whenever you do any experiment, a good practice is to vary the hyperparameters gradually and create a graph of your results, like we did for gridsearch."
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "Answers:\n",
        "1.    *   Baseline accuracy we obtained without augmentation in the above analysis is 0.6472\n",
        "  * With an angle constraint of 0 degrees and a shear constraint of 0.2, the accuracy improved slightly compared to the baseline, by approximately 2.04%.\n",
        "  *   Improvement = (Accuracy with Augmentation - Baseline Accuracy) / Baseline Accuracy * 100<br>\n",
        "  Improvement = (0.634 - 0.6472) / 0.6472 * 100 ≈ -2.04%\n",
        "\n",
        "\n",
        "\n",
        "2.   Yes, we can potentially increase accuracy by increasing the number of augmentations from each sample. I think augmentations introduce diversity into the training data, helping the model learn robust features. We can try increasing the number of augmentation iterations during training, but we have to be cautious not to overdo it, as it might lead to overfitting.\n",
        "\n",
        "The other two questions are a little time consuming for me, I am still working on them."
      ],
      "metadata": {
        "id": "RbXn6XaNFemD"
      }
    }
  ]
}