Week_5_Assignment_2.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "name": "Week_5_Assignment_2.ipynb",
      "provenance": [],
      "authorship_tag": "ABX9TyOiueHadAbaTbftGwapneR3",
      "include_colab_link": true
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/ShrayankM/FoundationsOFDataScience/blob/master/Week_5_Assignment_2.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "-gLtFCi4EywW",
        "colab_type": "text"
      },
      "source": [
        "#Statement 1-Most frequent Words"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "AULOMmc7Bo8E",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "with open(\"poem.txt\",\"r\") as f:\n",
        "    lines = f.readlines()"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "_w6wqNq9EVWt",
        "colab_type": "code",
        "outputId": "d8961e16-0a0b-4e78-f59e-344a52616d5f",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 55
        }
      },
      "source": [
        "print(lines)"
      ],
      "execution_count": 4,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "['Unending Love by Rabindranath Tagore\\n', '\\n', 'I seem to have loved you in numberless forms, numberless times\\n', 'In life after life, in age after age, forever.\\n', 'My spellbound heart has made and remade the necklace of songs,\\n', 'That you take as a gift, wear round your neck in your many forms,\\n', 'In life after life, in age after age, forever.\\n', '\\n', 'Whenever I hear old chronicles of love, its age-old pain,\\n', 'Its ancient tale of being apart or together.\\n', 'As I stare on and on into the past, in the end you emerge,\\n', 'Clad in the light of a pole-star piercing the darkness of time:\\n', 'You become an image of what is remembered forever.\\n', '\\n', 'You and I have floated here on the stream that brings from the fount.\\n', 'At the heart of time, love of one for another.\\n', 'We have played along side millions of lovers, shared in the same\\n', 'Shy sweetness of meeting, the same distressful tears of farewell-\\n', 'Old love but in shapes that renew and renew forever.\\n', '\\n', 'Today it is heaped at your feet, it has found its end in you\\n', 'The love of all man’s days both past and forever:\\n', 'Universal joy, universal sorrow, universal life.\\n', 'The memories of all loves merging with this one love of ours –\\n', 'And the songs of every poet past and forever.']\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "xN4-6jQzB5vc",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "dict_words = dict()\n",
        "char_sequence = ['\\n','.',',',':','-']"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "eaZBgyABCQ7i",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "for i in range(len(lines)):\n",
        "    for c in char_sequence:\n",
        "        lines[i] = lines[i].replace(c,\"\")\n",
        "    lines[i] = lines[i].lower()"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "H-Hnpx3qEabt",
        "colab_type": "code",
        "outputId": "ab9d876a-34e4-4a8f-bf43-76de7521ec68",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 55
        }
      },
      "source": [
        "print(lines)"
      ],
      "execution_count": 7,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "['unending love by rabindranath tagore', '', 'i seem to have loved you in numberless forms numberless times', 'in life after life in age after age forever', 'my spellbound heart has made and remade the necklace of songs', 'that you take as a gift wear round your neck in your many forms', 'in life after life in age after age forever', '', 'whenever i hear old chronicles of love its ageold pain', 'its ancient tale of being apart or together', 'as i stare on and on into the past in the end you emerge', 'clad in the light of a polestar piercing the darkness of time', 'you become an image of what is remembered forever', '', 'you and i have floated here on the stream that brings from the fount', 'at the heart of time love of one for another', 'we have played along side millions of lovers shared in the same', 'shy sweetness of meeting the same distressful tears of farewell', 'old love but in shapes that renew and renew forever', '', 'today it is heaped at your feet it has found its end in you', 'the love of all man’s days both past and forever', 'universal joy universal sorrow universal life', 'the memories of all loves merging with this one love of ours –', 'and the songs of every poet past and forever']\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "T4vzVdaCB622",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "for line in lines:\n",
        "    words = line.split()\n",
        "    for word in words:\n",
        "        if word in dict_words:\n",
        "            dict_words[word] += 1\n",
        "        else:\n",
        "            dict_words[word] = 1"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "bA43f5E7Ecrp",
        "colab_type": "code",
        "outputId": "a6d01794-29f5-4fba-fc45-0746d0b4d72f",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 55
        }
      },
      "source": [
        "print(dict_words)"
      ],
      "execution_count": 9,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "{'unending': 1, 'love': 6, 'by': 1, 'rabindranath': 1, 'tagore': 1, 'i': 4, 'seem': 1, 'to': 1, 'have': 3, 'loved': 1, 'you': 6, 'in': 11, 'numberless': 2, 'forms': 2, 'times': 1, 'life': 5, 'after': 4, 'age': 4, 'forever': 6, 'my': 1, 'spellbound': 1, 'heart': 2, 'has': 2, 'made': 1, 'and': 7, 'remade': 1, 'the': 13, 'necklace': 1, 'of': 15, 'songs': 2, 'that': 3, 'take': 1, 'as': 2, 'a': 2, 'gift': 1, 'wear': 1, 'round': 1, 'your': 3, 'neck': 1, 'many': 1, 'whenever': 1, 'hear': 1, 'old': 2, 'chronicles': 1, 'its': 3, 'ageold': 1, 'pain': 1, 'ancient': 1, 'tale': 1, 'being': 1, 'apart': 1, 'or': 1, 'together': 1, 'stare': 1, 'on': 3, 'into': 1, 'past': 3, 'end': 2, 'emerge': 1, 'clad': 1, 'light': 1, 'polestar': 1, 'piercing': 1, 'darkness': 1, 'time': 2, 'become': 1, 'an': 1, 'image': 1, 'what': 1, 'is': 2, 'remembered': 1, 'floated': 1, 'here': 1, 'stream': 1, 'brings': 1, 'from': 1, 'fount': 1, 'at': 2, 'one': 2, 'for': 1, 'another': 1, 'we': 1, 'played': 1, 'along': 1, 'side': 1, 'millions': 1, 'lovers': 1, 'shared': 1, 'same': 2, 'shy': 1, 'sweetness': 1, 'meeting': 1, 'distressful': 1, 'tears': 1, 'farewell': 1, 'but': 1, 'shapes': 1, 'renew': 2, 'today': 1, 'it': 2, 'heaped': 1, 'feet': 1, 'found': 1, 'all': 2, 'man’s': 1, 'days': 1, 'both': 1, 'universal': 3, 'joy': 1, 'sorrow': 1, 'memories': 1, 'loves': 1, 'merging': 1, 'with': 1, 'this': 1, 'ours': 1, '–': 1, 'every': 1, 'poet': 1}\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "rLVWZxJlDis8",
        "colab_type": "code",
        "outputId": "3db50620-f04a-45d2-9823-1d776362e4aa",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 55
        }
      },
      "source": [
        "print(dict_words.items())"
      ],
      "execution_count": 10,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "dict_items([('unending', 1), ('love', 6), ('by', 1), ('rabindranath', 1), ('tagore', 1), ('i', 4), ('seem', 1), ('to', 1), ('have', 3), ('loved', 1), ('you', 6), ('in', 11), ('numberless', 2), ('forms', 2), ('times', 1), ('life', 5), ('after', 4), ('age', 4), ('forever', 6), ('my', 1), ('spellbound', 1), ('heart', 2), ('has', 2), ('made', 1), ('and', 7), ('remade', 1), ('the', 13), ('necklace', 1), ('of', 15), ('songs', 2), ('that', 3), ('take', 1), ('as', 2), ('a', 2), ('gift', 1), ('wear', 1), ('round', 1), ('your', 3), ('neck', 1), ('many', 1), ('whenever', 1), ('hear', 1), ('old', 2), ('chronicles', 1), ('its', 3), ('ageold', 1), ('pain', 1), ('ancient', 1), ('tale', 1), ('being', 1), ('apart', 1), ('or', 1), ('together', 1), ('stare', 1), ('on', 3), ('into', 1), ('past', 3), ('end', 2), ('emerge', 1), ('clad', 1), ('light', 1), ('polestar', 1), ('piercing', 1), ('darkness', 1), ('time', 2), ('become', 1), ('an', 1), ('image', 1), ('what', 1), ('is', 2), ('remembered', 1), ('floated', 1), ('here', 1), ('stream', 1), ('brings', 1), ('from', 1), ('fount', 1), ('at', 2), ('one', 2), ('for', 1), ('another', 1), ('we', 1), ('played', 1), ('along', 1), ('side', 1), ('millions', 1), ('lovers', 1), ('shared', 1), ('same', 2), ('shy', 1), ('sweetness', 1), ('meeting', 1), ('distressful', 1), ('tears', 1), ('farewell', 1), ('but', 1), ('shapes', 1), ('renew', 2), ('today', 1), ('it', 2), ('heaped', 1), ('feet', 1), ('found', 1), ('all', 2), ('man’s', 1), ('days', 1), ('both', 1), ('universal', 3), ('joy', 1), ('sorrow', 1), ('memories', 1), ('loves', 1), ('merging', 1), ('with', 1), ('this', 1), ('ours', 1), ('–', 1), ('every', 1), ('poet', 1)])\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "9JKJ-9OyCpG7",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "sorted_words = sorted(dict_words.items(),key = lambda item:item[1],reverse=True)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "T8Y4jihvEhGG",
        "colab_type": "code",
        "outputId": "e16fc392-1570-4daf-c753-ddd7675f9daa",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 55
        }
      },
      "source": [
        "print(sorted_words)"
      ],
      "execution_count": 12,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "[('of', 15), ('the', 13), ('in', 11), ('and', 7), ('love', 6), ('you', 6), ('forever', 6), ('life', 5), ('i', 4), ('after', 4), ('age', 4), ('have', 3), ('that', 3), ('your', 3), ('its', 3), ('on', 3), ('past', 3), ('universal', 3), ('numberless', 2), ('forms', 2), ('heart', 2), ('has', 2), ('songs', 2), ('as', 2), ('a', 2), ('old', 2), ('end', 2), ('time', 2), ('is', 2), ('at', 2), ('one', 2), ('same', 2), ('renew', 2), ('it', 2), ('all', 2), ('unending', 1), ('by', 1), ('rabindranath', 1), ('tagore', 1), ('seem', 1), ('to', 1), ('loved', 1), ('times', 1), ('my', 1), ('spellbound', 1), ('made', 1), ('remade', 1), ('necklace', 1), ('take', 1), ('gift', 1), ('wear', 1), ('round', 1), ('neck', 1), ('many', 1), ('whenever', 1), ('hear', 1), ('chronicles', 1), ('ageold', 1), ('pain', 1), ('ancient', 1), ('tale', 1), ('being', 1), ('apart', 1), ('or', 1), ('together', 1), ('stare', 1), ('into', 1), ('emerge', 1), ('clad', 1), ('light', 1), ('polestar', 1), ('piercing', 1), ('darkness', 1), ('become', 1), ('an', 1), ('image', 1), ('what', 1), ('remembered', 1), ('floated', 1), ('here', 1), ('stream', 1), ('brings', 1), ('from', 1), ('fount', 1), ('for', 1), ('another', 1), ('we', 1), ('played', 1), ('along', 1), ('side', 1), ('millions', 1), ('lovers', 1), ('shared', 1), ('shy', 1), ('sweetness', 1), ('meeting', 1), ('distressful', 1), ('tears', 1), ('farewell', 1), ('but', 1), ('shapes', 1), ('today', 1), ('heaped', 1), ('feet', 1), ('found', 1), ('man’s', 1), ('days', 1), ('both', 1), ('joy', 1), ('sorrow', 1), ('memories', 1), ('loves', 1), ('merging', 1), ('with', 1), ('this', 1), ('ours', 1), ('–', 1), ('every', 1), ('poet', 1)]\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "LVayZTtQDL_v",
        "colab_type": "code",
        "outputId": "e242c207-4bbe-4bf4-d5a7-9732ca2db1f8",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 55
        }
      },
      "source": [
        "freq_words = sorted_words[:10]\n",
        "print(freq_words)"
      ],
      "execution_count": 13,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "[('of', 15), ('the', 13), ('in', 11), ('and', 7), ('love', 6), ('you', 6), ('forever', 6), ('life', 5), ('i', 4), ('after', 4)]\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "MFC_YOthDf5s",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "with open(\"freq_words.txt\",\"w\") as f:\n",
        "    for word in freq_words:\n",
        "        f.write(word[0]+\"\\n\")"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "_O_Gc8QUE4zT",
        "colab_type": "text"
      },
      "source": [
        "#Statement2 - Most Frequent 2 Grams"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "DPw25EJwD8r2",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 55
        },
        "outputId": "c1bffd66-35dd-4bf6-c8c0-ca7e493e7976"
      },
      "source": [
        "lines_single = \" \".join(lines)\n",
        "print(lines_single)"
      ],
      "execution_count": 15,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "unending love by rabindranath tagore  i seem to have loved you in numberless forms numberless times in life after life in age after age forever my spellbound heart has made and remade the necklace of songs that you take as a gift wear round your neck in your many forms in life after life in age after age forever  whenever i hear old chronicles of love its ageold pain its ancient tale of being apart or together as i stare on and on into the past in the end you emerge clad in the light of a polestar piercing the darkness of time you become an image of what is remembered forever  you and i have floated here on the stream that brings from the fount at the heart of time love of one for another we have played along side millions of lovers shared in the same shy sweetness of meeting the same distressful tears of farewell old love but in shapes that renew and renew forever  today it is heaped at your feet it has found its end in you the love of all man’s days both past and forever universal joy universal sorrow universal life the memories of all loves merging with this one love of ours – and the songs of every poet past and forever\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "PbgJgB9VL6QM",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 55
        },
        "outputId": "fb127ebd-5c6e-43f0-848c-bdf7e79c7df2"
      },
      "source": [
        "words_list = lines_single.split()\n",
        "print(words_list)"
      ],
      "execution_count": 17,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "['unending', 'love', 'by', 'rabindranath', 'tagore', 'i', 'seem', 'to', 'have', 'loved', 'you', 'in', 'numberless', 'forms', 'numberless', 'times', 'in', 'life', 'after', 'life', 'in', 'age', 'after', 'age', 'forever', 'my', 'spellbound', 'heart', 'has', 'made', 'and', 'remade', 'the', 'necklace', 'of', 'songs', 'that', 'you', 'take', 'as', 'a', 'gift', 'wear', 'round', 'your', 'neck', 'in', 'your', 'many', 'forms', 'in', 'life', 'after', 'life', 'in', 'age', 'after', 'age', 'forever', 'whenever', 'i', 'hear', 'old', 'chronicles', 'of', 'love', 'its', 'ageold', 'pain', 'its', 'ancient', 'tale', 'of', 'being', 'apart', 'or', 'together', 'as', 'i', 'stare', 'on', 'and', 'on', 'into', 'the', 'past', 'in', 'the', 'end', 'you', 'emerge', 'clad', 'in', 'the', 'light', 'of', 'a', 'polestar', 'piercing', 'the', 'darkness', 'of', 'time', 'you', 'become', 'an', 'image', 'of', 'what', 'is', 'remembered', 'forever', 'you', 'and', 'i', 'have', 'floated', 'here', 'on', 'the', 'stream', 'that', 'brings', 'from', 'the', 'fount', 'at', 'the', 'heart', 'of', 'time', 'love', 'of', 'one', 'for', 'another', 'we', 'have', 'played', 'along', 'side', 'millions', 'of', 'lovers', 'shared', 'in', 'the', 'same', 'shy', 'sweetness', 'of', 'meeting', 'the', 'same', 'distressful', 'tears', 'of', 'farewell', 'old', 'love', 'but', 'in', 'shapes', 'that', 'renew', 'and', 'renew', 'forever', 'today', 'it', 'is', 'heaped', 'at', 'your', 'feet', 'it', 'has', 'found', 'its', 'end', 'in', 'you', 'the', 'love', 'of', 'all', 'man’s', 'days', 'both', 'past', 'and', 'forever', 'universal', 'joy', 'universal', 'sorrow', 'universal', 'life', 'the', 'memories', 'of', 'all', 'loves', 'merging', 'with', 'this', 'one', 'love', 'of', 'ours', '–', 'and', 'the', 'songs', 'of', 'every', 'poet', 'past', 'and', 'forever']\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "dcP9X1pOL-_U",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "two_gram_words = zip(words_list,words_list[1:])"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "yVJFC1Z8MJLC",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "two_gram_list = []\n",
        "for item in two_gram_words:\n",
        "    two_gram_list.append(item)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "YbSsqJ9fMObe",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 55
        },
        "outputId": "b097db07-828f-4839-e7b3-3ca84ac1d118"
      },
      "source": [
        "print(two_gram_list)"
      ],
      "execution_count": 30,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "[('unending', 'love'), ('love', 'by'), ('by', 'rabindranath'), ('rabindranath', 'tagore'), ('tagore', 'i'), ('i', 'seem'), ('seem', 'to'), ('to', 'have'), ('have', 'loved'), ('loved', 'you'), ('you', 'in'), ('in', 'numberless'), ('numberless', 'forms'), ('forms', 'numberless'), ('numberless', 'times'), ('times', 'in'), ('in', 'life'), ('life', 'after'), ('after', 'life'), ('life', 'in'), ('in', 'age'), ('age', 'after'), ('after', 'age'), ('age', 'forever'), ('forever', 'my'), ('my', 'spellbound'), ('spellbound', 'heart'), ('heart', 'has'), ('has', 'made'), ('made', 'and'), ('and', 'remade'), ('remade', 'the'), ('the', 'necklace'), ('necklace', 'of'), ('of', 'songs'), ('songs', 'that'), ('that', 'you'), ('you', 'take'), ('take', 'as'), ('as', 'a'), ('a', 'gift'), ('gift', 'wear'), ('wear', 'round'), ('round', 'your'), ('your', 'neck'), ('neck', 'in'), ('in', 'your'), ('your', 'many'), ('many', 'forms'), ('forms', 'in'), ('in', 'life'), ('life', 'after'), ('after', 'life'), ('life', 'in'), ('in', 'age'), ('age', 'after'), ('after', 'age'), ('age', 'forever'), ('forever', 'whenever'), ('whenever', 'i'), ('i', 'hear'), ('hear', 'old'), ('old', 'chronicles'), ('chronicles', 'of'), ('of', 'love'), ('love', 'its'), ('its', 'ageold'), ('ageold', 'pain'), ('pain', 'its'), ('its', 'ancient'), ('ancient', 'tale'), ('tale', 'of'), ('of', 'being'), ('being', 'apart'), ('apart', 'or'), ('or', 'together'), ('together', 'as'), ('as', 'i'), ('i', 'stare'), ('stare', 'on'), ('on', 'and'), ('and', 'on'), ('on', 'into'), ('into', 'the'), ('the', 'past'), ('past', 'in'), ('in', 'the'), ('the', 'end'), ('end', 'you'), ('you', 'emerge'), ('emerge', 'clad'), ('clad', 'in'), ('in', 'the'), ('the', 'light'), ('light', 'of'), ('of', 'a'), ('a', 'polestar'), ('polestar', 'piercing'), ('piercing', 'the'), ('the', 'darkness'), ('darkness', 'of'), ('of', 'time'), ('time', 'you'), ('you', 'become'), ('become', 'an'), ('an', 'image'), ('image', 'of'), ('of', 'what'), ('what', 'is'), ('is', 'remembered'), ('remembered', 'forever'), ('forever', 'you'), ('you', 'and'), ('and', 'i'), ('i', 'have'), ('have', 'floated'), ('floated', 'here'), ('here', 'on'), ('on', 'the'), ('the', 'stream'), ('stream', 'that'), ('that', 'brings'), ('brings', 'from'), ('from', 'the'), ('the', 'fount'), ('fount', 'at'), ('at', 'the'), ('the', 'heart'), ('heart', 'of'), ('of', 'time'), ('time', 'love'), ('love', 'of'), ('of', 'one'), ('one', 'for'), ('for', 'another'), ('another', 'we'), ('we', 'have'), ('have', 'played'), ('played', 'along'), ('along', 'side'), ('side', 'millions'), ('millions', 'of'), ('of', 'lovers'), ('lovers', 'shared'), ('shared', 'in'), ('in', 'the'), ('the', 'same'), ('same', 'shy'), ('shy', 'sweetness'), ('sweetness', 'of'), ('of', 'meeting'), ('meeting', 'the'), ('the', 'same'), ('same', 'distressful'), ('distressful', 'tears'), ('tears', 'of'), ('of', 'farewell'), ('farewell', 'old'), ('old', 'love'), ('love', 'but'), ('but', 'in'), ('in', 'shapes'), ('shapes', 'that'), ('that', 'renew'), ('renew', 'and'), ('and', 'renew'), ('renew', 'forever'), ('forever', 'today'), ('today', 'it'), ('it', 'is'), ('is', 'heaped'), ('heaped', 'at'), ('at', 'your'), ('your', 'feet'), ('feet', 'it'), ('it', 'has'), ('has', 'found'), ('found', 'its'), ('its', 'end'), ('end', 'in'), ('in', 'you'), ('you', 'the'), ('the', 'love'), ('love', 'of'), ('of', 'all'), ('all', 'man’s'), ('man’s', 'days'), ('days', 'both'), ('both', 'past'), ('past', 'and'), ('and', 'forever'), ('forever', 'universal'), ('universal', 'joy'), ('joy', 'universal'), ('universal', 'sorrow'), ('sorrow', 'universal'), ('universal', 'life'), ('life', 'the'), ('the', 'memories'), ('memories', 'of'), ('of', 'all'), ('all', 'loves'), ('loves', 'merging'), ('merging', 'with'), ('with', 'this'), ('this', 'one'), ('one', 'love'), ('love', 'of'), ('of', 'ours'), ('ours', '–'), ('–', 'and'), ('and', 'the'), ('the', 'songs'), ('songs', 'of'), ('of', 'every'), ('every', 'poet'), ('poet', 'past'), ('past', 'and'), ('and', 'forever')]\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "0_A8F9rlMqEY",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "two_gram_dict = dict()\n",
        "for item in two_gram_list:\n",
        "    if item in two_gram_dict:\n",
        "        two_gram_dict[item] += 1\n",
        "    else:\n",
        "        two_gram_dict[item] = 1"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "WczlironNCOE",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 55
        },
        "outputId": "a6250fb7-6886-4dba-d17b-b19b9efd7c81"
      },
      "source": [
        "print(two_gram_dict)"
      ],
      "execution_count": 32,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "{('unending', 'love'): 1, ('love', 'by'): 1, ('by', 'rabindranath'): 1, ('rabindranath', 'tagore'): 1, ('tagore', 'i'): 1, ('i', 'seem'): 1, ('seem', 'to'): 1, ('to', 'have'): 1, ('have', 'loved'): 1, ('loved', 'you'): 1, ('you', 'in'): 1, ('in', 'numberless'): 1, ('numberless', 'forms'): 1, ('forms', 'numberless'): 1, ('numberless', 'times'): 1, ('times', 'in'): 1, ('in', 'life'): 2, ('life', 'after'): 2, ('after', 'life'): 2, ('life', 'in'): 2, ('in', 'age'): 2, ('age', 'after'): 2, ('after', 'age'): 2, ('age', 'forever'): 2, ('forever', 'my'): 1, ('my', 'spellbound'): 1, ('spellbound', 'heart'): 1, ('heart', 'has'): 1, ('has', 'made'): 1, ('made', 'and'): 1, ('and', 'remade'): 1, ('remade', 'the'): 1, ('the', 'necklace'): 1, ('necklace', 'of'): 1, ('of', 'songs'): 1, ('songs', 'that'): 1, ('that', 'you'): 1, ('you', 'take'): 1, ('take', 'as'): 1, ('as', 'a'): 1, ('a', 'gift'): 1, ('gift', 'wear'): 1, ('wear', 'round'): 1, ('round', 'your'): 1, ('your', 'neck'): 1, ('neck', 'in'): 1, ('in', 'your'): 1, ('your', 'many'): 1, ('many', 'forms'): 1, ('forms', 'in'): 1, ('forever', 'whenever'): 1, ('whenever', 'i'): 1, ('i', 'hear'): 1, ('hear', 'old'): 1, ('old', 'chronicles'): 1, ('chronicles', 'of'): 1, ('of', 'love'): 1, ('love', 'its'): 1, ('its', 'ageold'): 1, ('ageold', 'pain'): 1, ('pain', 'its'): 1, ('its', 'ancient'): 1, ('ancient', 'tale'): 1, ('tale', 'of'): 1, ('of', 'being'): 1, ('being', 'apart'): 1, ('apart', 'or'): 1, ('or', 'together'): 1, ('together', 'as'): 1, ('as', 'i'): 1, ('i', 'stare'): 1, ('stare', 'on'): 1, ('on', 'and'): 1, ('and', 'on'): 1, ('on', 'into'): 1, ('into', 'the'): 1, ('the', 'past'): 1, ('past', 'in'): 1, ('in', 'the'): 3, ('the', 'end'): 1, ('end', 'you'): 1, ('you', 'emerge'): 1, ('emerge', 'clad'): 1, ('clad', 'in'): 1, ('the', 'light'): 1, ('light', 'of'): 1, ('of', 'a'): 1, ('a', 'polestar'): 1, ('polestar', 'piercing'): 1, ('piercing', 'the'): 1, ('the', 'darkness'): 1, ('darkness', 'of'): 1, ('of', 'time'): 2, ('time', 'you'): 1, ('you', 'become'): 1, ('become', 'an'): 1, ('an', 'image'): 1, ('image', 'of'): 1, ('of', 'what'): 1, ('what', 'is'): 1, ('is', 'remembered'): 1, ('remembered', 'forever'): 1, ('forever', 'you'): 1, ('you', 'and'): 1, ('and', 'i'): 1, ('i', 'have'): 1, ('have', 'floated'): 1, ('floated', 'here'): 1, ('here', 'on'): 1, ('on', 'the'): 1, ('the', 'stream'): 1, ('stream', 'that'): 1, ('that', 'brings'): 1, ('brings', 'from'): 1, ('from', 'the'): 1, ('the', 'fount'): 1, ('fount', 'at'): 1, ('at', 'the'): 1, ('the', 'heart'): 1, ('heart', 'of'): 1, ('time', 'love'): 1, ('love', 'of'): 3, ('of', 'one'): 1, ('one', 'for'): 1, ('for', 'another'): 1, ('another', 'we'): 1, ('we', 'have'): 1, ('have', 'played'): 1, ('played', 'along'): 1, ('along', 'side'): 1, ('side', 'millions'): 1, ('millions', 'of'): 1, ('of', 'lovers'): 1, ('lovers', 'shared'): 1, ('shared', 'in'): 1, ('the', 'same'): 2, ('same', 'shy'): 1, ('shy', 'sweetness'): 1, ('sweetness', 'of'): 1, ('of', 'meeting'): 1, ('meeting', 'the'): 1, ('same', 'distressful'): 1, ('distressful', 'tears'): 1, ('tears', 'of'): 1, ('of', 'farewell'): 1, ('farewell', 'old'): 1, ('old', 'love'): 1, ('love', 'but'): 1, ('but', 'in'): 1, ('in', 'shapes'): 1, ('shapes', 'that'): 1, ('that', 'renew'): 1, ('renew', 'and'): 1, ('and', 'renew'): 1, ('renew', 'forever'): 1, ('forever', 'today'): 1, ('today', 'it'): 1, ('it', 'is'): 1, ('is', 'heaped'): 1, ('heaped', 'at'): 1, ('at', 'your'): 1, ('your', 'feet'): 1, ('feet', 'it'): 1, ('it', 'has'): 1, ('has', 'found'): 1, ('found', 'its'): 1, ('its', 'end'): 1, ('end', 'in'): 1, ('in', 'you'): 1, ('you', 'the'): 1, ('the', 'love'): 1, ('of', 'all'): 2, ('all', 'man’s'): 1, ('man’s', 'days'): 1, ('days', 'both'): 1, ('both', 'past'): 1, ('past', 'and'): 2, ('and', 'forever'): 2, ('forever', 'universal'): 1, ('universal', 'joy'): 1, ('joy', 'universal'): 1, ('universal', 'sorrow'): 1, ('sorrow', 'universal'): 1, ('universal', 'life'): 1, ('life', 'the'): 1, ('the', 'memories'): 1, ('memories', 'of'): 1, ('all', 'loves'): 1, ('loves', 'merging'): 1, ('merging', 'with'): 1, ('with', 'this'): 1, ('this', 'one'): 1, ('one', 'love'): 1, ('of', 'ours'): 1, ('ours', '–'): 1, ('–', 'and'): 1, ('and', 'the'): 1, ('the', 'songs'): 1, ('songs', 'of'): 1, ('of', 'every'): 1, ('every', 'poet'): 1, ('poet', 'past'): 1}\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "6Nh0xKrnNEVi",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "sorted_two_gram = sorted(two_gram_dict.items() , key = lambda item:item[1],reverse=True)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "K-txbGq3N1cm",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 55
        },
        "outputId": "7c4f5d93-5d56-448d-884b-d2449919cb1c"
      },
      "source": [
        "print(sorted_two_gram)"
      ],
      "execution_count": 49,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "[(('in', 'the'), 3), (('love', 'of'), 3), (('in', 'life'), 2), (('life', 'after'), 2), (('after', 'life'), 2), (('life', 'in'), 2), (('in', 'age'), 2), (('age', 'after'), 2), (('after', 'age'), 2), (('age', 'forever'), 2), (('of', 'time'), 2), (('the', 'same'), 2), (('of', 'all'), 2), (('past', 'and'), 2), (('and', 'forever'), 2), (('unending', 'love'), 1), (('love', 'by'), 1), (('by', 'rabindranath'), 1), (('rabindranath', 'tagore'), 1), (('tagore', 'i'), 1), (('i', 'seem'), 1), (('seem', 'to'), 1), (('to', 'have'), 1), (('have', 'loved'), 1), (('loved', 'you'), 1), (('you', 'in'), 1), (('in', 'numberless'), 1), (('numberless', 'forms'), 1), (('forms', 'numberless'), 1), (('numberless', 'times'), 1), (('times', 'in'), 1), (('forever', 'my'), 1), (('my', 'spellbound'), 1), (('spellbound', 'heart'), 1), (('heart', 'has'), 1), (('has', 'made'), 1), (('made', 'and'), 1), (('and', 'remade'), 1), (('remade', 'the'), 1), (('the', 'necklace'), 1), (('necklace', 'of'), 1), (('of', 'songs'), 1), (('songs', 'that'), 1), (('that', 'you'), 1), (('you', 'take'), 1), (('take', 'as'), 1), (('as', 'a'), 1), (('a', 'gift'), 1), (('gift', 'wear'), 1), (('wear', 'round'), 1), (('round', 'your'), 1), (('your', 'neck'), 1), (('neck', 'in'), 1), (('in', 'your'), 1), (('your', 'many'), 1), (('many', 'forms'), 1), (('forms', 'in'), 1), (('forever', 'whenever'), 1), (('whenever', 'i'), 1), (('i', 'hear'), 1), (('hear', 'old'), 1), (('old', 'chronicles'), 1), (('chronicles', 'of'), 1), (('of', 'love'), 1), (('love', 'its'), 1), (('its', 'ageold'), 1), (('ageold', 'pain'), 1), (('pain', 'its'), 1), (('its', 'ancient'), 1), (('ancient', 'tale'), 1), (('tale', 'of'), 1), (('of', 'being'), 1), (('being', 'apart'), 1), (('apart', 'or'), 1), (('or', 'together'), 1), (('together', 'as'), 1), (('as', 'i'), 1), (('i', 'stare'), 1), (('stare', 'on'), 1), (('on', 'and'), 1), (('and', 'on'), 1), (('on', 'into'), 1), (('into', 'the'), 1), (('the', 'past'), 1), (('past', 'in'), 1), (('the', 'end'), 1), (('end', 'you'), 1), (('you', 'emerge'), 1), (('emerge', 'clad'), 1), (('clad', 'in'), 1), (('the', 'light'), 1), (('light', 'of'), 1), (('of', 'a'), 1), (('a', 'polestar'), 1), (('polestar', 'piercing'), 1), (('piercing', 'the'), 1), (('the', 'darkness'), 1), (('darkness', 'of'), 1), (('time', 'you'), 1), (('you', 'become'), 1), (('become', 'an'), 1), (('an', 'image'), 1), (('image', 'of'), 1), (('of', 'what'), 1), (('what', 'is'), 1), (('is', 'remembered'), 1), (('remembered', 'forever'), 1), (('forever', 'you'), 1), (('you', 'and'), 1), (('and', 'i'), 1), (('i', 'have'), 1), (('have', 'floated'), 1), (('floated', 'here'), 1), (('here', 'on'), 1), (('on', 'the'), 1), (('the', 'stream'), 1), (('stream', 'that'), 1), (('that', 'brings'), 1), (('brings', 'from'), 1), (('from', 'the'), 1), (('the', 'fount'), 1), (('fount', 'at'), 1), (('at', 'the'), 1), (('the', 'heart'), 1), (('heart', 'of'), 1), (('time', 'love'), 1), (('of', 'one'), 1), (('one', 'for'), 1), (('for', 'another'), 1), (('another', 'we'), 1), (('we', 'have'), 1), (('have', 'played'), 1), (('played', 'along'), 1), (('along', 'side'), 1), (('side', 'millions'), 1), (('millions', 'of'), 1), (('of', 'lovers'), 1), (('lovers', 'shared'), 1), (('shared', 'in'), 1), (('same', 'shy'), 1), (('shy', 'sweetness'), 1), (('sweetness', 'of'), 1), (('of', 'meeting'), 1), (('meeting', 'the'), 1), (('same', 'distressful'), 1), (('distressful', 'tears'), 1), (('tears', 'of'), 1), (('of', 'farewell'), 1), (('farewell', 'old'), 1), (('old', 'love'), 1), (('love', 'but'), 1), (('but', 'in'), 1), (('in', 'shapes'), 1), (('shapes', 'that'), 1), (('that', 'renew'), 1), (('renew', 'and'), 1), (('and', 'renew'), 1), (('renew', 'forever'), 1), (('forever', 'today'), 1), (('today', 'it'), 1), (('it', 'is'), 1), (('is', 'heaped'), 1), (('heaped', 'at'), 1), (('at', 'your'), 1), (('your', 'feet'), 1), (('feet', 'it'), 1), (('it', 'has'), 1), (('has', 'found'), 1), (('found', 'its'), 1), (('its', 'end'), 1), (('end', 'in'), 1), (('in', 'you'), 1), (('you', 'the'), 1), (('the', 'love'), 1), (('all', 'man’s'), 1), (('man’s', 'days'), 1), (('days', 'both'), 1), (('both', 'past'), 1), (('forever', 'universal'), 1), (('universal', 'joy'), 1), (('joy', 'universal'), 1), (('universal', 'sorrow'), 1), (('sorrow', 'universal'), 1), (('universal', 'life'), 1), (('life', 'the'), 1), (('the', 'memories'), 1), (('memories', 'of'), 1), (('all', 'loves'), 1), (('loves', 'merging'), 1), (('merging', 'with'), 1), (('with', 'this'), 1), (('this', 'one'), 1), (('one', 'love'), 1), (('of', 'ours'), 1), (('ours', '–'), 1), (('–', 'and'), 1), (('and', 'the'), 1), (('the', 'songs'), 1), (('songs', 'of'), 1), (('of', 'every'), 1), (('every', 'poet'), 1), (('poet', 'past'), 1)]\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "UAzuUq9xNJlT",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 108
        },
        "outputId": "824d2927-2290-4313-84fb-382c184709ea"
      },
      "source": [
        "sorted_two_gram[:5]"
      ],
      "execution_count": 41,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "[(('in', 'the'), 3),\n",
              " (('love', 'of'), 3),\n",
              " (('in', 'life'), 2),\n",
              " (('life', 'after'), 2),\n",
              " (('after', 'life'), 2)]"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 41
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "YNRDzZMZNh2T",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "with open(\"freq_two_gram.txt\",\"w\") as f:\n",
        "    for item in sorted_two_gram[:5]:\n",
        "        f.write(item[0][0] +\" ,\" + item[0][1]+\"\\n\")"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "ahIA7V_fOGbs",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        ""
      ],
      "execution_count": 0,
      "outputs": []
    }
  ]
}