mlp/mlp_quiz.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Quiz: Multi Layer Perceptron\n",
    "==================\n",
    "\n",
    "In this quiz you are asked to modify the MLP model used in the tutorial in order to train the network on the [Iris flower dataset](../iris/iris.ipynb). The dataset is already provided in the TFRecord format and you can find it [here](../iris). In the snippets below you will find an **hashtag** `#QUIZ` in the parts where you are asked to complete the code. Remember that the Tensorflow official documentation is your best friend when you cannot find a specific method."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Defining the model\n",
    "---------------------\n",
    "\n",
    "Here you have to define the MLP model for the new dataset. What you are asked to do is:\n",
    "\n",
    "- Define the model. The dataset has **4 input features** (sepal length, sepal width, petal length, petal width) meaning that your MLP must have 4 input units. Morevoer the dataset has **3 classes** (0=Setosa, 1=Versicolor, 2=Virginica) meaning that your perceptron must have 3 output units.\n",
    "\n",
    "- Design a new **accuracy metric**. In the XOR example there were only two possible classes, whereas here there are three. You should use the `argmax()` method in tensorflow to get the class with higher probability returned from the MLP. Then you must compare it with the actual target class that is stored in a one-hot vector."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "import tensorflow as tf"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "def my_model_fn(features, labels, mode):\n",
    "    #QUIZ: here you must define the model for the new dataset \n",
    "\n",
    "        \n",
    "    #PREDICT mode\n",
    "    if mode == tf.estimator.ModeKeys.PREDICT:\n",
    "        predictions = {\"classes\": tf.round(y),\n",
    "                       \"probabilities\": y}\n",
    "        return tf.estimator.EstimatorSpec(mode=mode, predictions=predictions)\n",
    "    #TRAIN mode\n",
    "    elif mode == tf.estimator.ModeKeys.TRAIN:\n",
    "        loss = tf.losses.mean_squared_error(labels=labels, predictions=y)\n",
    "        #optimizer = tf.train.GradientDescentOptimizer(learning_rate=0.001)\n",
    "        optimizer = tf.train.AdamOptimizer()\n",
    "        train_op = optimizer.minimize(loss=loss, global_step=tf.train.get_global_step())\n",
    "        #QUIZ: here you must define a new accuracy metric.\n",
    "        #accuracy = tf.metrics.accuracy(???)\n",
    "        \n",
    "        tf.summary.scalar('accuracy', accuracy[1]) #<-- accuracy[1] to grab the value\n",
    "        logging_hook = tf.train.LoggingTensorHook({\"accuracy\" : accuracy[1]}, every_n_iter=250)\n",
    "        return tf.estimator.EstimatorSpec(mode=mode, loss=loss, train_op=train_op, training_hooks =[logging_hook])\n",
    "    #EVAL mode\n",
    "    elif mode == tf.estimator.ModeKeys.EVAL:\n",
    "        loss = tf.losses.mean_squared_error(labels=labels, predictions=y)\n",
    "        #QUIZ: here you must define a new accuracy metric.      \n",
    "        #accuracy = tf.metrics.accuracy(???)\n",
    "        \n",
    "        eval_metric = {\"accuracy\": accuracy}\n",
    "        return tf.estimator.EstimatorSpec(mode=mode, loss=loss, eval_metric_ops=eval_metric)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "mlp = tf.estimator.Estimator(model_fn=my_model_fn, model_dir=\"./tf_mlp_model\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Train the model\n",
    "------------------\n",
    "\n",
    "In this section you have to define a dataset object and an iterator. The parse function is already implemented. The TFRecord files for the Iris dataset are ready to be used and included in this repository. Here you have to declare a Tensorflow dataset and an iterator. I suggest you to read the official documentation of Tensorflow for knowning more about those classes. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "def my_input_fn():  \n",
    "    def _parse_function(example_proto):\n",
    "        features = {\"feature\": tf.VarLenFeature(tf.float32),\n",
    "                    \"label\": tf.FixedLenFeature((), tf.int64, default_value=0)}\n",
    "        parsed_features = tf.parse_single_example(example_proto, features)\n",
    "        feature = tf.cast(parsed_features[\"feature\"], tf.float32)\n",
    "        feature = tf.sparse_tensor_to_dense(feature, default_value=0)\n",
    "        label_one_hot = tf.one_hot(parsed_features[\"label\"], depth=3)\n",
    "        return feature, label_one_hot\n",
    "\n",
    "    #QUIZ: load the TFRecord dataset using the method tf.data.TFRecordDataset()\n",
    "    \n",
    "    #QUIZ: parse the dataset using the map() method pointing to the _parse_function() above\n",
    "\n",
    "    #QUIZ: store the dataset in cache\n",
    "    \n",
    "    #QUIZ: shuffle the dataset using the shuffle() method (be carefull with the buffer_size parameter)\n",
    "\n",
    "    #QUIZ: use the method repeat() to decide how many time you would like to use the dataset\n",
    "\n",
    "    #QUIZ: define the batch size through the batch() method\n",
    "\n",
    "    #QUIZ: make a one-shot iterator\n",
    "    \n",
    "    #QUIZ: get new values using get_next()\n",
    "    \n",
    "    return batch_features, batch_labels"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "tf.logging.set_verbosity(tf.logging.INFO)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "mlp.train(input_fn=my_input_fn, steps=5000)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Test the model\n",
    "------------------\n",
    "\n",
    "Now you have to test the model on the test set. You have to declare a new input function, a dataset and an iterator. You can reuse the code above but you must be careful with the `batch()` and `repeat()` methods."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "def my_input_fn():\n",
    "        #QUIZ: load and parse the test set. This part is very similar to the function\n",
    "        #you already implemented above. \n",
    "    \n",
    "    \n",
    "        return batch_features, batch_labels"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Improve the performances\n",
    "-----------------------------\n",
    "\n",
    "Now you have to modify some critical parameters in your model in order to improve the performances. I suggest you  to play with the number of hidden units. You can try to increase the number of units and use **dropout** to avoid overfitting. Moreover you can use a different optimizer. Good optimizers are the RMSProp, Adam, and Adagrad. However, sometimes nothing is better than a well-tuned stocastich gradient descent with decaying learning rate. Good luck..."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 2",
   "language": "python",
   "name": "python2"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 2
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython2",
   "version": "2.7.6"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 0
}