HAR_LSTM.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Human Activity Recognition with LSTMS\n",
    "\n",
    "Hello everybody!\n",
    "This is our *notebook* for Human Activity Recognition with LSTMs with help of simple smartphone-sensor data. \n",
    ">**This is part of an Android-Application-Project which will be used for non-intrusive medical surveillance systems.** \n",
    "\n",
    "* **The goal shortterm** is to further adapt existing HAR-classifications to a more standard environment (smartphone which is carried in the pocket or in the handbag, etc.) and to implement and combine the whole in a user-friendly (android) application which aims to secure and help the elderly by providing a remote surveillance possibility for realtives. \n",
    "* **The goal longterm** is to build an open-source application which can be utalized and adapted to any kind of non-intrusive (no camera, microphone, etc. needed) medical surveillance system for all kinds of different people in need. \n",
    "\n",
    "### Ressources\n",
    "\n",
    "*Speaking for me,* I'm no expert in HAR with deep-learning.\n",
    "That's why I did some research over already existing projects and tutorials (there are quite some.)\n",
    "Check them out in our **spreasheet under the RESSOURCE-TAB: **\n",
    "https://docs.google.com/spreadsheets/d/1EDc84oX6Z9HOHBKIYNhE6iemN4Il1MR1ylGJyiCkh-M/edit#gid=251000036\n",
    "\n",
    "### What's next?\n",
    "\n",
    "As I said in our *slack-channel* there will be a small team responsible for different parts of this projects.<br > \n",
    "**The topics/groups are the following** *(If you have a better idea, feel free to tell me!)* <br>\n",
    "**--> ADD YOUR NAME BEHIND ONE (OR MORE) OF THE TOPICS AND START WORKING WITH THE OTHERS IN YOUR TEAM!!**\n",
    "\n",
    "* _Data-Extraction/Generation and Pre-Processing:_ Standardize, Normalize, Batching-fct., etc.\n",
    "<br> `In Charge: @Nicolas Remerscheid`\n",
    "* _The Model:_ Maybe Embedded Layer (Dim. Reduction), LSTMs, Linear-Layer, etc.\n",
    "<br> `In Charge: ...`\n",
    "* _The Training:_ Training loops, visualization\n",
    "<br> `In Charge: ...`\n",
    "* _Validation and Testing:_ finding best hyperparams (Validation), test-loops, check-points\n",
    "<br> `In Charge: ...`\n",
    "\n",
    "One everybody has chosen a section, feel free to edit this notebook and **add your section!**"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Imports"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import torch \n",
    "import os\n",
    "from torch.utils.data.sampler import SubsetRandomSampler\n",
    "from torchvision import datasets, models, transforms\n",
    "import matplotlib.pyplot as plt\n",
    "\n",
    "# For inline-visual.\n",
    "%matplotlib inline"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 1. Data-Extraction/Generation and Pre-Processing\n",
    "* For now use mostly the concepts, principles of: *(Added custom implementations at times and explanations)*\n",
    "> Guillaume Chevalier, LSTMs for Human Activity Recognition, 2016, https://github.com/guillaume-chevalier/LSTM-Human-Activity-Recognition\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "######## DEFINED PARAMS ########\n",
    "# DATA: \n",
    "#    - UCI-Dataset: 60999314 data-examples\n",
    "# Model: \n",
    "#    - Input-dimension/features: \n",
    "#    - Output-dimension: 6 (WALKING, STANDING, ...)\n",
    "#    - Seq.-Length: 128 (2.56 sec with 1/2 Hz sensor-output on data in UCL data-set) -> TO BE VERIFIED!\n",
    "#    - Overlap: 50% between data-windows \n",
    "#    - Batch-Size: 1500 (1500 Sequences to be processed in parallel -> vectorization!)\n",
    "#    -> More Params to be defined!\n",
    "\n",
    "# Note/Reminder: unrolled LSTM-Layer consists of the same strcuture -> one pair of weights per Layer/Cell"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Useful Constants\n",
    "\n",
    "# Those are separate normalised input features for the neural network \n",
    "# Equal exactly the folder names of the UCI-Data-set -> array later used for data loading!\n",
    "INPUT_SIGNAL_TYPES = [\n",
    "    \"body_acc_x_\",\n",
    "    \"body_acc_y_\",\n",
    "    \"body_acc_z_\",\n",
    "    \"body_gyro_x_\",\n",
    "    \"body_gyro_y_\",\n",
    "    \"body_gyro_z_\",\n",
    "    \"total_acc_x_\",\n",
    "    \"total_acc_y_\",\n",
    "    \"total_acc_z_\"\n",
    "]\n",
    "\n",
    "# Output classes to learn how to classify \n",
    "LABELS = [\n",
    "    \"WALKING\", \n",
    "    \"WALKING_UPSTAIRS\", \n",
    "    \"WALKING_DOWNSTAIRS\", \n",
    "    \"SITTING\", \n",
    "    \"STANDING\", \n",
    "    \"LAYING\"\n",
    "]\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Downloading the UCI-Dataset\n",
    "* More information on the data-set: UCI Machine Learning Repository\n",
    "* **for now:** 1:1 from Guillaume Chevalier"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "Dataset is now located at: data/UCI HAR Dataset/\n"
     ]
    }
   ],
   "source": [
    "# Note: Linux bash commands start with a \"!\" inside those \"ipython notebook\" cells\n",
    "\n",
    "DATA_PATH = \"data/\"\n",
    "\n",
    "# ******* Downloading-mechanism (with 'download_dataset.py')*******\n",
    "#!pwd && ls\n",
    "#os.chdir(DATA_PATH)\n",
    "#!pwd && ls\n",
    "\n",
    "#!python download_dataset.py\n",
    "\n",
    "#!pwd && ls\n",
    "#os.chdir(\"..\")\n",
    "#!pwd && ls\n",
    "\n",
    "# -> Also manually implementable!\n",
    "\n",
    "DATASET_PATH = DATA_PATH + \"UCI HAR Dataset/\"\n",
    "print(\"\\n\" + \"Dataset is now located at: \" + DATASET_PATH)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Utility functions for training\n",
    "* data-processing-functions\n",
    "* How Guillaume Chevalier did batching in his project, ..."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "# TODO: UNDERSTAND WORKING PRINCIPLE (of the for-loop)\n",
    "\n",
    "#def extract_batch_size(_train, step, batch_size):\n",
    "#    # Function to fetch a \"batch_size\" amount of data from \"(X|y)_train\" data.\n",
    "\n",
    "    # TODO: Why \"list\" is used? \n",
    "#    shape = list(_train.shape)\n",
    "    # Only first dimension has to be changed: as TIME-STEP x INPUT-VECTOR remains the same \n",
    "    # Only number of seauences varies -> has to be limited to batch_size \n",
    "#    shape[0] = batch_size\n",
    "#    batch_s = np.empty(shape)\n",
    "\n",
    "#    for i in range(batch_size):\n",
    "        # Loop index\n",
    "        # step := time-steps per sequence \n",
    "#        index = ((step-1)*batch_size + i) % len(_train)\n",
    "        # First index of _train := index of sequence \n",
    "#        batch_s[i] = _train[index]\n",
    "\n",
    "#    return batch_s\n",
    "\n",
    "\n",
    "def one_hot(y_, n_classes):\n",
    "    # Function to encode neural one-hot output labels from number indexes\n",
    "    # e.g. WITH 3 ENTRIES IN Y:\n",
    "    # one_hot(y_=[[5], [0], [3]], n_classes=6):\n",
    "    #     return [[0, 0, 0, 0, 0, 1], [1, 0, 0, 0, 0, 0], [0, 0, 0, 1, 0, 0]]\n",
    "\n",
    "    y_ = y_.reshape(len(y_))\n",
    "    return np.eye(n_classes)[np.array(y_, dtype=np.int32)]  # Returns FLOATS"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Preparing dataset:\n",
    "* **Loading-Mechanism:** *from Guillaume Chevalier*\n",
    "* Added seperation of the TEST-Dataset into 30% Validation and 70% Testing\n",
    "* Instead of implementing the batching thru a function (extract_batch_size) made us of DATA-LOADERS"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "# UCI dataset is stored with a classic folder structure \n",
    "TRAIN = \"train/\"\n",
    "TEST = \"test/\"\n",
    "\n",
    "\n",
    "# ***** FUNCTION DESCRIPTION *****\n",
    "# Input (Argument): X_signals_paths := array which contains different path-locations\n",
    "# Ouput: 3D-array containing the complete uci-data-set (sorted in the categories)\n",
    "# Functionality: Takes as input array of paths to different data-examples (sorted) and load them \n",
    "#                one by one into a single 3D array: TIME-STEP x INPUT-VECTOR(one row) x SERIES \n",
    "# SERIES are ordered after ROWS in the different FILES which PATHS are stored in INPUT_SIGNAL_TYPES\n",
    "\n",
    "# TODO: DIM-DESC of RETURN-ARR to ne VERIFIED!\n",
    "\n",
    "def load_X(X_signals_paths):\n",
    "    X_signals = []\n",
    "\n",
    "    for signal_type_path in X_signals_paths:\n",
    "        file = open(signal_type_path, 'r')\n",
    "        # Read dataset from disk, dealing with text files' syntax\n",
    "        X_signals.append(\n",
    "            [np.array(serie, dtype=np.float32) for serie in [\n",
    "                row.replace('  ', ' ').strip().split(' ') for row in file\n",
    "            ]]\n",
    "        )\n",
    "        file.close()\n",
    "\n",
    "    return np.transpose(np.array(X_signals), (1, 2, 0))\n",
    "\n",
    "# ***** FUNCTION DESCRIPTION *****\n",
    "# SAME AS FOR X but only stored in one .txt file in one location \n",
    "\n",
    "# TODO: \n",
    "#     1. Check how order is concerning different input-signal-types \n",
    "#     2. Check wether we use MANY_TO_ONE or MANY_TO_MANY Architecture\n",
    "\n",
    "def load_y(y_path):\n",
    "    file = open(y_path, 'r')\n",
    "    # Read dataset from disk, dealing with text file's syntax\n",
    "    y_ = np.array(\n",
    "        [elem for elem in [\n",
    "            row.replace('  ', ' ').strip().split(' ') for row in file\n",
    "        ]],\n",
    "        dtype=np.int32\n",
    "    )\n",
    "    file.close()\n",
    "\n",
    "    # Substract 1 to each output class for friendly 0-based indexing\n",
    "    return y_ - 1\n",
    "\n",
    "# 1. Load Train and Testing Data in: INPUTS\n",
    "\n",
    "# UCI-Data is seperated in the following folder structure: i.e.: body acc.-data for training\n",
    "# data/UCI HAR Dataset/train/Interial Signals/body_acc_x_/train.txt\n",
    "X_train_signals_paths = [\n",
    "    DATASET_PATH + TRAIN + \"Inertial Signals/\" + signal + \"train.txt\" for signal in INPUT_SIGNAL_TYPES\n",
    "]\n",
    "X_test_signals_paths = [\n",
    "    DATASET_PATH + TEST + \"Inertial Signals/\" + signal + \"test.txt\" for signal in INPUT_SIGNAL_TYPES\n",
    "]\n",
    "\n",
    "# Input path-arrays into the load-function \n",
    "x_train = load_X(X_train_signals_paths)\n",
    "x_test = load_X(X_test_signals_paths)\n",
    "\n",
    "# 2. Load Test and Training Data in: TARGET/LABELS \n",
    "\n",
    "y_train_path = DATASET_PATH + TRAIN + \"y_train.txt\"\n",
    "y_test_path = DATASET_PATH + TEST + \"y_test.txt\"\n",
    "\n",
    "y_train = load_y(y_train_path)\n",
    "y_test = load_y(y_test_path)\n",
    "\n",
    "\n",
    "# 3. One-Hot encode LABEL/TARGET data for later error-usage \n",
    "\n",
    "# Resulting dimension: #DATA_EXAMPLES x #CLASSES(=6)\n",
    "y_train = one_hot(y_train, 6)\n",
    "y_test = one_hot(y_test, 6)\n",
    "\n",
    "\n",
    "# 4. Seperate Testing data into validation data and testing data \n",
    "\n",
    "# assumption: testing data is already shuffled -> no need to further shuffle before splitting into testing and val.\n",
    "limit_ind = int(len(x_test) * 0.3)\n",
    "x_val, y_val = x_test[:limit_ind], y_test[:limit_ind]\n",
    "x_test, y_test = x_test[limit_ind:], y_test[limit_ind:]\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Create Data-Loaders for later use in Training, Vaidatiion and Testing Loops \n",
    "from torch.utils.data import TensorDataset, DataLoader\n",
    "\n",
    "# 5. create Tensor datasets\n",
    "\n",
    "train_data = TensorDataset(torch.from_numpy(x_train), torch.from_numpy(y_train))\n",
    "valid_data = TensorDataset(torch.from_numpy(x_val), torch.from_numpy(y_val))\n",
    "test_data = TensorDataset(torch.from_numpy(x_test), torch.from_numpy(y_test))\n",
    "\n",
    "# 6. Define BASIC TRAINING PARAMS (1500 from Guillaume Chevalier's Project)\n",
    "# TODO: Adjust params if necessary (@the person who does the training)\n",
    "batch_size = 1500\n",
    "\n",
    "# 7. Create Data-Loaders and do the SHUFFLING as well \n",
    "\n",
    "train_loader = DataLoader(train_data, shuffle=True, batch_size=batch_size)\n",
    "valid_loader = DataLoader(valid_data, shuffle=True, batch_size=batch_size)\n",
    "test_loader = DataLoader(test_data, shuffle=True, batch_size=batch_size)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Sample input size:  torch.Size([1500, 128, 9])\n",
      "Sample input: \n",
      " tensor([[[-0.0557,  0.0406, -0.1243,  ...,  0.9464, -0.2393, -0.1661],\n",
      "         [-0.0604,  0.0076, -0.1454,  ...,  0.9417, -0.2718, -0.1861],\n",
      "         [-0.0635,  0.0014, -0.1504,  ...,  0.9387, -0.2777, -0.1900],\n",
      "         ...,\n",
      "         [ 0.3361,  0.1754, -0.1093,  ...,  1.3202, -0.1211, -0.2267],\n",
      "         [ 0.3717,  0.0962, -0.1715,  ...,  1.3555, -0.2003, -0.2888],\n",
      "         [ 0.1735, -0.0609, -0.1232,  ...,  1.1570, -0.3573, -0.2404]],\n",
      "\n",
      "        [[ 0.0002,  0.0057, -0.0197,  ...,  1.0071, -0.2273, -0.0802],\n",
      "         [-0.0014,  0.0053, -0.0168,  ...,  1.0056, -0.2275, -0.0774],\n",
      "         [-0.0017,  0.0012, -0.0206,  ...,  1.0053, -0.2313, -0.0812],\n",
      "         ...,\n",
      "         [ 0.0340,  0.1306,  0.0128,  ...,  1.0402, -0.1039, -0.0269],\n",
      "         [ 0.0337,  0.1500,  0.0059,  ...,  1.0400, -0.0836, -0.0346],\n",
      "         [ 0.0257,  0.1688,  0.0051,  ...,  1.0323, -0.0639, -0.0362]],\n",
      "\n",
      "        [[ 0.2448,  0.1185, -0.2049,  ...,  1.2364, -0.1530, -0.3211],\n",
      "         [ 0.1955,  0.0518, -0.2523,  ...,  1.1867, -0.2194, -0.3687],\n",
      "         [ 0.1184, -0.1865, -0.2556,  ...,  1.1092, -0.4573, -0.3724],\n",
      "         ...,\n",
      "         [ 0.3225,  0.0583, -0.1963,  ...,  1.3039, -0.2138, -0.3221],\n",
      "         [ 0.3365,  0.0384, -0.2304,  ...,  1.3177, -0.2331, -0.3560],\n",
      "         [ 0.1789, -0.1305, -0.1540,  ...,  1.1599, -0.4012, -0.2794]],\n",
      "\n",
      "        ...,\n",
      "\n",
      "        [[-0.2121, -0.0287,  0.0145,  ...,  0.7659, -0.2662,  0.2044],\n",
      "         [-0.1946, -0.0427,  0.0139,  ...,  0.7844, -0.2800,  0.2030],\n",
      "         [-0.2059, -0.0401,  0.0317,  ...,  0.7742, -0.2772,  0.2200],\n",
      "         ...,\n",
      "         [ 0.2599, -0.1354,  0.1631,  ...,  1.2521, -0.3173,  0.3407],\n",
      "         [ 0.2529, -0.1148,  0.0750,  ...,  1.2442, -0.2961,  0.2542],\n",
      "         [ 0.1897, -0.1326,  0.0452,  ...,  1.1802, -0.3134,  0.2262]],\n",
      "\n",
      "        [[-0.0977,  0.0221,  0.1323,  ...,  0.9170, -0.1290,  0.2191],\n",
      "         [-0.0896,  0.0380,  0.1137,  ...,  0.9251, -0.1133,  0.2007],\n",
      "         [-0.0892,  0.0583,  0.0952,  ...,  0.9254, -0.0932,  0.1827],\n",
      "         ...,\n",
      "         [ 0.2622,  0.1978, -0.1223,  ...,  1.2728,  0.0803, -0.0198],\n",
      "         [ 0.2648,  0.0741, -0.2895,  ...,  1.2751, -0.0429, -0.1869],\n",
      "         [ 0.2191, -0.1734, -0.2939,  ...,  1.2291, -0.2899, -0.1912]],\n",
      "\n",
      "        [[ 0.0070, -0.0220, -0.0392,  ...,  1.0057,  0.0717,  0.1131],\n",
      "         [ 0.0135, -0.0156, -0.0491,  ...,  1.0129,  0.0758,  0.1019],\n",
      "         [ 0.0108, -0.0157, -0.0609,  ...,  1.0109,  0.0734,  0.0887],\n",
      "         ...,\n",
      "         [-0.0007,  0.0084,  0.0056,  ...,  1.0145,  0.0973,  0.0908],\n",
      "         [-0.0056,  0.0096,  0.0076,  ...,  1.0096,  0.0989,  0.0928],\n",
      "         [-0.0059,  0.0098,  0.0080,  ...,  1.0092,  0.0996,  0.0932]]])\n",
      "\n",
      "Sample label size (one-hot-encoded):  torch.Size([1500, 6])\n",
      "Sample label: \n",
      " tensor([[1., 0., 0., 0., 0., 0.],\n",
      "        [0., 0., 0., 0., 1., 0.],\n",
      "        [1., 0., 0., 0., 0., 0.],\n",
      "        ...,\n",
      "        [0., 1., 0., 0., 0., 0.],\n",
      "        [1., 0., 0., 0., 0., 0.],\n",
      "        [0., 0., 0., 1., 0., 0.]], dtype=torch.float64)\n"
     ]
    }
   ],
   "source": [
    "# obtain one batch of training data -> TO TEST if Data-Loader are functioning properly!\n",
    "dataiter = iter(train_loader)\n",
    "sample_x, sample_y = dataiter.next()\n",
    "\n",
    "print('Sample input size: ', sample_x.size()) # batch_size, seq_length\n",
    "print('Sample input: \\n', sample_x)\n",
    "print()\n",
    "print('Sample label size (one-hot-encoded): ', sample_y.size()) # batch_size\n",
    "print('Sample label: \\n', sample_y)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 2. The Model\n",
    "### Basic params \n",
    "* Centrally fix all **params** and the overall **network-structure**\n",
    "* Params **1:1 same as from Guillaume Chevalier** *-> to be adapted and tuned in the future* "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "############ START OF 2.,3.,4. TOPIC (Model, Training, Validation/Test.) ############\n",
    "# MY SUGGESTION: ALSO USE EXISTING PROJECTS (i.e.: the SAME as me) AS BASIS \n",
    "# WE CAN THEN FIRSTLY REPRODUCE THIS AND IMPLEMENT IT IN AN APPLICATION AND THE TUNE IT!\n",
    "\n",
    "# The training and testing can be done exactly THE SAME AS in the SENTIMENT_RNN_EXERCISE \n",
    "# Use the DATA-LOADERS to create the TRAIN-, VAL- and TEST-LOOPS!\n",
    "\n",
    "# Input Data\n",
    "training_data_count = len(X_train)  # 7352 training series (with 50% overlap between each serie)\n",
    "test_data_count = len(X_test)  # 2947 testing series\n",
    "n_steps = len(X_train[0])  # 128 timesteps per series\n",
    "n_input = len(X_train[0][0])  # 9 input parameters per timestep\n",
    "\n",
    "# LSTM Neural Network's internal structure\n",
    "\n",
    "n_hidden = 32 # Hidden layer num of features\n",
    "n_classes = 6 # Total classes (should go up, or should go down)\n",
    "\n",
    "# Training \n",
    "learning_rate = 0.0025\n",
    "lambda_loss_amount = 0.0015 # Depending on optimization-algorithm NOT NECESSARY\n",
    "training_iters = training_data_count * 300  # Loop 300 times on the dataset\n",
    "display_iter = 30000  # To show test set accuracy during training\n",
    "\n",
    "# Some debugging info\n",
    "\n",
    "print(\"Some useful info to get an insight on dataset's shape and normalisation:\")\n",
    "print(\"(X shape, y shape, every X's mean, every X's standard deviation)\")\n",
    "print(X_test.shape, y_test.shape, np.mean(X_test), np.std(X_test))\n",
    "print(\"The dataset is therefore properly normalised, as expected, but not yet one-hot encoded.\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# TODO: MODEL-CREATION"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 3. Training & Validation\n",
    "### To be edited"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# TODO: Implement Training-Loop and Validation-Loop (same as in SENTIMENT_RNN_EXERCISE)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 4. Testing\n",
    "### To be edited"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# TODO: Implement TEST-Loop and Visualizations(same as in SENTIMENT_RNN_EXERCISE)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 5. Deployement \n",
    "### To be imported into an android application\n",
    "* After converting the trained model to a KERAS-Model it has to be converted to a TensorFlow Mobile or TensorFlow Light Model and then imported into an Android-Application, I can then be used for inference theree. \n",
    "* A very good tutorial: https://heartbeat.fritz.ai/deploying-pytorch-and-keras-models-to-android-with-tensorflow-mobile-a16a1fb83f2"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# TODO: Save trained model "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.7"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}