task-4/Email Spam Detection with Machine Learning task-4.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "162d4932-e6d3-4bf6-9b43-02a036540aff",
   "metadata": {},
   "source": [
    "## Problem Statement:\n",
    "\n",
    "### EMAIL SPAM DETECTION WITH MACHINE LEARNING (Naive Bayes algorithm and NLP)\n",
    "n\r\n",
    "We’ve all been the recipient of spam emails before. Spam mail, or junk mail, is a type of email\r\n",
    "that is sent to a massive number of users at one time, frequently containing cryptic\r\n",
    "messages, scams, or most dangerously, phishing cont\n",
    ".\r\n",
    "\r\n",
    "\r\n",
    "\r\n",
    "In this Project, use Python to build an email spam detector. Then, use machine learning to\r\n",
    "train the spam detector to recognize and classify emails into spam and  lets gets started.et\r\n",
    "start\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "id": "37cee41c-dc53-450d-ba3c-707ca2ff491e",
   "metadata": {},
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "import seaborn as sns\n",
    "from sklearn.model_selection import train_test_split\n",
    "from sklearn.feature_extraction.text import CountVectorizer\n",
    "from sklearn.naive_bayes import MultinomialNB\n",
    "from sklearn.metrics import accuracy_score, confusion_matrix, classification_report\n",
    "\n",
    "import warnings\n",
    "warnings.filterwarnings('ignore')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "22e4f440-4d71-4cf0-a7e5-f9c4f86c5d94",
   "metadata": {},
   "source": [
    "## 1. Data collection"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "id": "e30eff59-1388-4dec-b15e-836363884381",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>v1</th>\n",
       "      <th>v2</th>\n",
       "      <th>Unnamed: 2</th>\n",
       "      <th>Unnamed: 3</th>\n",
       "      <th>Unnamed: 4</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>ham</td>\n",
       "      <td>Go until jurong point, crazy.. Available only ...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>ham</td>\n",
       "      <td>Ok lar... Joking wif u oni...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>spam</td>\n",
       "      <td>Free entry in 2 a wkly comp to win FA Cup fina...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>ham</td>\n",
       "      <td>U dun say so early hor... U c already then say...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>ham</td>\n",
       "      <td>Nah I don't think he goes to usf, he lives aro...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "     v1                                                 v2 Unnamed: 2  \\\n",
       "0   ham  Go until jurong point, crazy.. Available only ...        NaN   \n",
       "1   ham                      Ok lar... Joking wif u oni...        NaN   \n",
       "2  spam  Free entry in 2 a wkly comp to win FA Cup fina...        NaN   \n",
       "3   ham  U dun say so early hor... U c already then say...        NaN   \n",
       "4   ham  Nah I don't think he goes to usf, he lives aro...        NaN   \n",
       "\n",
       "  Unnamed: 3 Unnamed: 4  \n",
       "0        NaN        NaN  \n",
       "1        NaN        NaN  \n",
       "2        NaN        NaN  \n",
       "3        NaN        NaN  \n",
       "4        NaN        NaN  "
      ]
     },
     "execution_count": 30,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "file_path = 'spam.csv'\n",
    "encoding = 'utf-8'  # Adjust the encoding\n",
    "\n",
    "# Read the file line by line and handle decoding errors\n",
    "lines = []\n",
    "with open(file_path, 'r', encoding=encoding, errors='replace') as file:\n",
    "    for line in file:\n",
    "        lines.append(line)\n",
    "\n",
    "# Join the lines into a string and read with Pandas\n",
    "df = pd.read_csv(io.StringIO(''.join(lines)))\n",
    "\n",
    "# Now 'data' contains the CSV data\n",
    "df.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f22f1772-0fb6-49d3-a652-e4c13066db5c",
   "metadata": {},
   "source": [
    "## 2. Data preprocessing, cleaning"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "id": "5548fdfd-7b6e-4519-b2ec-37ad167104a0",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(5572, 5)"
      ]
     },
     "execution_count": 31,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "id": "b2bf8ce6-4c7c-46a6-b6d9-7a8dada67692",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "v1               0\n",
       "v2               0\n",
       "Unnamed: 2    5522\n",
       "Unnamed: 3    5560\n",
       "Unnamed: 4    5566\n",
       "dtype: int64"
      ]
     },
     "execution_count": 32,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# check for null values\n",
    "df.isnull().sum()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "id": "13ff1fce-09e8-4b11-83ad-99c017abbbe7",
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>v1</th>\n",
       "      <th>v2</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>ham</td>\n",
       "      <td>Go until jurong point, crazy.. Available only ...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>ham</td>\n",
       "      <td>Ok lar... Joking wif u oni...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>spam</td>\n",
       "      <td>Free entry in 2 a wkly comp to win FA Cup fina...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>ham</td>\n",
       "      <td>U dun say so early hor... U c already then say...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>ham</td>\n",
       "      <td>Nah I don't think he goes to usf, he lives aro...</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "     v1                                                 v2\n",
       "0   ham  Go until jurong point, crazy.. Available only ...\n",
       "1   ham                      Ok lar... Joking wif u oni...\n",
       "2  spam  Free entry in 2 a wkly comp to win FA Cup fina...\n",
       "3   ham  U dun say so early hor... U c already then say...\n",
       "4   ham  Nah I don't think he goes to usf, he lives aro..."
      ]
     },
     "execution_count": 33,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# drop cols that are independent or not relevant for our prediction\n",
    "unwanted_cols = ['Unnamed: 2', 'Unnamed: 3', 'Unnamed: 4']\n",
    "\n",
    "df = df.drop(columns = unwanted_cols)\n",
    "\n",
    "# look now how it was ?\n",
    "df.head() # valid right !..."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 65,
   "id": "4bdbd27d-7218-49e7-84a4-7efbe41eda57",
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "v1\n",
       "ham     4825\n",
       "spam     747\n",
       "Name: count, dtype: int64"
      ]
     },
     "execution_count": 65,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# its like counting values group by v1 column [aggregation]\n",
    "df['v1'].value_counts()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 47,
   "id": "bc525d5f-a9f9-4bf8-9444-70c0d7dd2185",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "v2\n",
       "Sorry, I'll call later                                                                                                                                                 30\n",
       "I cant pick the phone right now. Pls send a message                                                                                                                    12\n",
       "Ok...                                                                                                                                                                  10\n",
       "7 wonders in My WORLD 7th You 6th Ur style 5th Ur smile 4th Ur Personality 3rd Ur Nature 2nd Ur SMS and 1st \\Ur Lovely Friendship\\\"... good morning dear\"               4\n",
       "Say this slowly.? GOD,I LOVE YOU &amp; I NEED YOU,CLEAN MY HEART WITH YOUR BLOOD.Send this to Ten special people &amp; u c miracle tomorrow, do it,pls,pls do it...     4\n",
       "                                                                                                                                                                       ..\n",
       "I gotta collect da car at 6 lei.                                                                                                                                        1\n",
       "No. On the way home. So if not for the long dry spell the season would have been over                                                                                   1\n",
       "Urgent! Please call 09061743811 from landline. Your ABTA complimentary 4* Tenerife Holiday or �5000 cash await collection SAE T&Cs Box 326 CW25WX 150ppm                1\n",
       "Dear 0776xxxxxxx U've been invited to XCHAT. This is our final attempt to contact u! Txt CHAT to 86688 150p/MsgrcvdHG/Suite342/2Lands/Row/W1J6HL LDN 18yrs              1\n",
       "Rofl. Its true to its name                                                                                                                                              1\n",
       "Name: count, Length: 5169, dtype: int64"
      ]
     },
     "execution_count": 47,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df['v2'].value_counts() "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 46,
   "id": "f71a3c0a-1872-41e4-a99d-e49bac4a0055",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>v1</th>\n",
       "      <th>v2</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>count</th>\n",
       "      <td>5572</td>\n",
       "      <td>5572</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>unique</th>\n",
       "      <td>2</td>\n",
       "      <td>5169</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>top</th>\n",
       "      <td>ham</td>\n",
       "      <td>Sorry, I'll call later</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>freq</th>\n",
       "      <td>4825</td>\n",
       "      <td>30</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "          v1                      v2\n",
       "count   5572                    5572\n",
       "unique     2                    5169\n",
       "top      ham  Sorry, I'll call later\n",
       "freq    4825                      30"
      ]
     },
     "execution_count": 46,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# describe [talented python]\n",
    "df.describe()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "612b6716-b042-4fd4-ad3e-0b81ce7f7220",
   "metadata": {},
   "source": [
    "## 3. Data visualization"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 74,
   "id": "83cb2d19-dfad-4762-a18d-71cc7dd24608",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAdcAAAHWCAYAAAAl2MNkAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8pXeV/AAAACXBIWXMAAA9hAAAPYQGoP6dpAAAzrElEQVR4nO3de1yUdf7//+cAAioOKCnoioq6GXhcDymVuR6SDLMD9smyJFNLv2gpla67LqnZ0mqeylOuJlb6Se1Tuh6RRO0gmtHiqeRWG6abAm0Ko6YgML8/unH9nEAFfOuIPO6323W7Oe/rNe95vQF5cs11zYzN6XQ6BQAAjPFwdwMAANxsCFcAAAwjXAEAMIxwBQDAMMIVAADDCFcAAAwjXAEAMIxwBQDAMMIVAADDCFegHAoLCzV+/HiFhITIw8NDDz74oLtbuik99dRTatasmbvbqBaaNWump556yrq9Y8cO2Ww27dixw2093UwIV5Tbv//9bz377LNq3ry5fH19Zbfbdeedd2ru3Lk6d+6cu9uTJC1YsECJiYnG53377bc1Y8YMDRw4UMuXL9e4ceMuWVtcXKx33nlHXbt2Vb169VSnTh3deuutGjJkiHbv3m28t+roj3/8o9q0aVPmviNHjshms+n111+/zl1d2lNPPSWbzVbm5uvr6+72cA14ubsBVA0bN27UI488Ih8fHw0ZMkRt2rRRQUGBPvvsM7300ks6dOiQFi9e7O42tWDBAt1yyy0uf5GbkJKSot/97neaPXv2FWufe+45zZ8/Xw888IAGDx4sLy8vZWRkaPPmzWrevLm6detmtDdUDT4+PlqyZEmpcU9PTzd0I2VkZMjDg+Ora4VwxRVlZmZq0KBBatq0qVJSUtSwYUNrX2xsrL777jtt3LjRjR1eezk5OQoICLhiXXZ2thYsWKARI0aU+mNjzpw5+umnn65Rh7jReXl56YknnnB3GxYfHx93t3BT488WXNH06dN15swZLV261CVYS7Rs2VLPP/+8dbuwsFCvvPKKWrRoIR8fHzVr1kx//vOflZ+f73I/m82myZMnl5rvt+eCEhMTZbPZ9PnnnysuLk7169dX7dq19dBDD7mEVbNmzXTo0CHt3LnTesrtj3/842XXdvbsWb3wwgsKCQmRj4+PWrVqpddff10lHxZV8hTj9u3bdejQIWveS52XyszMlNPp1J133llqn81mU4MGDUqt65NPPtGzzz6rwMBA2e12DRkyRKdOnXK577p16xQVFaVGjRrJx8dHLVq00CuvvKKioiKXupKnS/fv368ePXqoVq1aatmypT744ANJ0s6dO9W1a1fVrFlTrVq10scff3zZr48kFRQUKD4+Xp06dZK/v79q166t7t27a/v27S51Fz8du3jxYuv736VLF+3du7fUvGvXrlWbNm3k6+urNm3a6KOPPrpiL5V18uRJvfjii2rbtq38/Pxkt9vVr18/7du3z6Wu5Lzj6tWrNWXKFP3ud79TnTp1NHDgQOXl5Sk/P19jx45VgwYN5Ofnp6FDh5b6ub4aJT8Tn332mZ577jnVr19fAQEBevbZZ1VQUKDc3FwNGTJEdevWVd26dTV+/Hj99oPNXn/9dd1xxx0KDAxUzZo11alTJ+v7f7Hf/j8ry7fffqvo6GgFBwfL19dXjRs31qBBg5SXl2dszTcrjlxxRevXr1fz5s11xx13lKt++PDhWr58uQYOHKgXXnhBe/bsUUJCgr755pur+gU6ZswY1a1bVy+//LKOHDmiOXPmaPTo0Vq1apWkX48Mx4wZIz8/P/3lL3+RJAUFBV1yPqfTqQEDBmj79u0aNmyYOnTooKSkJL300kv68ccfNXv2bNWvX1/vvvuuXn31VZ05c0YJCQmSpLCwsDLnbNq0qSRpzZo1euSRR1SrVq0rrmv06NEKCAjQ5MmTlZGRoYULF+qHH36wftFLv/7S9fPzU1xcnPz8/JSSkqL4+Hg5HA7NmDHDZb5Tp06pf//+GjRokB555BEtXLhQgwYN0ooVKzR27FiNHDlSjz/+uHUO+dixY6pTp84l+3M4HFqyZIkee+wxjRgxQqdPn9bSpUsVGRmpL774Qh06dHCpX7lypU6fPq1nn31WNptN06dP18MPP6zvv/9eNWrUkCRt3bpV0dHRCg8PV0JCgn7++WcNHTpUjRs3vuLXq0RRUZH++9//lhr/7R8mkvT9999r7dq1euSRRxQaGqrs7Gy99dZb6tGjh77++ms1atTIpT4hIUE1a9bUn/70J3333Xd68803VaNGDXl4eOjUqVOaPHmydu/ercTERIWGhio+Pr5cPZfVr7e3t+x2u8vYmDFjFBwcrClTpmj37t1avHixAgICtGvXLjVp0kR/+9vftGnTJs2YMUNt2rTRkCFDrPvOnTtXAwYM0ODBg1VQUKD3339fjzzyiDZs2KCoqKhy9Sn9+kdVZGSk8vPzrX5+/PFHbdiwQbm5ufL39y/3XNWSE7iMvLw8pyTnAw88UK769PR0pyTn8OHDXcZffPFFpyRnSkqKNSbJ+fLLL5eao2nTps6YmBjr9rJly5ySnH369HEWFxdb4+PGjXN6eno6c3NzrbHWrVs7e/ToUa5e165d65TknDZtmsv4wIEDnTabzfndd99ZYz169HC2bt26XPMOGTLEKclZt25d50MPPeR8/fXXnd98802pupJ1derUyVlQUGCNT58+3SnJuW7dOmvsl19+KXX/Z5991lmrVi3n+fPnXfqU5Fy5cqU1dvjwYackp4eHh3P37t3WeFJSklOSc9myZZddT2FhoTM/P99l7NSpU86goCDn008/bY1lZmY6JTkDAwOdJ0+etMbXrVvnlORcv369NdahQwdnw4YNXb53W7dudUpyNm3a9LL9XLzOy20zZsyw6s+fP+8sKipymSMzM9Pp4+PjnDp1qjW2fft2pyRnmzZtXL4njz32mNNmszn79evnMkdERES5+o2Jiblkn5GRkVZdyc9EZGSky896RESE02azOUeOHGmNFRYWOhs3blzq5/23PysFBQXONm3aOHv16uUy/tv/ZyVr3759u9PpdDr/9a9/OSU516xZc8X1oTSeFsZlORwOSbrskc3FNm3aJEmKi4tzGX/hhRck6arOzT7zzDPWkZwkde/eXUVFRfrhhx8qNd+mTZvk6emp5557rlSvTqdTmzdvrtS8y5Yt07x58xQaGqqPPvpIL774osLCwtS7d2/9+OOPpeqfeeYZ64hOkkaNGiUvLy/raylJNWvWtP59+vRp/fe//1X37t31yy+/6PDhwy7z+fn5adCgQdbtVq1aKSAgQGFhYeratas1XvLv77///rLr8fT0lLe3t6Rfr4Q+efKkCgsL1blzZ3311Vel6h999FHVrVvXut29e3eXxzlx4oTS09MVExPjcvRzzz33KDw8/LK9XKxZs2ZKTk4utb333nulan18fKyLd4qKivTzzz/Lz89PrVq1KnMNQ4YMcfmedO3aVU6nU08//bRLXdeuXXXs2DEVFhZesV9fX98y+33ttddK1Q4bNszlZ73k8YcNG2aNeXp6qnPnzqW+fxf/rJw6dUp5eXnq3r17meu8nJLvTVJSkn755ZcK3Rc8LYwrKHm66vTp0+Wq/+GHH+Th4aGWLVu6jAcHBysgIKDSQShJTZo0cbld8gu8rKcBy+OHH35Qo0aNSv3hUPKUb2V79fDwUGxsrGJjY/Xzzz/r888/16JFi7R582YNGjRIn376qUv973//e5fbfn5+atiwoY4cOWKNHTp0SJMmTVJKSor1B0+J357/aty4scsvZunXX5QhISGlxqTyff2WL1+umTNn6vDhw7pw4YI1HhoaWqr2St+nkq/rb9ct6ZJhV5batWurT58+pcYv/rqVKC4u1ty5c7VgwQJlZma6nKsODAy84hpKvlZlfQ2Li4uVl5dX5jwX8/T0LLPfslTk8X/7/duwYYOmTZum9PR0l/PBv/2ZuJLQ0FDFxcVp1qxZWrFihbp3764BAwboiSee4CnhcuDIFZdlt9vVqFEjHTx4sEL3q+h/5Iv99iKdEpd6yYLzNxd03EgCAwM1YMAAbdq0ST169NBnn31W4dDOzc1Vjx49tG/fPk2dOlXr169XcnKy/v73v0v6NTgudqmvU2W/fu+9956eeuoptWjRQkuXLtWWLVuUnJysXr16lXrsq3mca+lvf/ub4uLidPfdd+u9995TUlKSkpOT1bp16wqt4XqtrSKPf/Fjf/rppxowYIB8fX21YMECbdq0ScnJyXr88ccr1ePMmTO1f/9+/fnPf9a5c+f03HPPqXXr1vrPf/5T4bmqG45ccUX9+/fX4sWLlZqaqoiIiMvWNm3aVMXFxfr2229dLvrJzs5Wbm6udcGP9OsRTW5ursv9CwoKdOLEiUr3WpFQb9q0qT7++GOdPn3a5ei15GnWi3s1oXPnztq5c6dOnDjhMve3336rnj17WrfPnDmjEydO6L777pP06xWsP//8sz788EPdfffdVl1mZqbR/i7lgw8+UPPmzfXhhx+6fH1ffvnlSs1XsvZvv/221L6MjIzKNXkFH3zwgXr27KmlS5e6jOfm5uqWW265Jo/pDv/3f/8nX19fJSUlubzUZtmyZZWes23btmrbtq0mTZqkXbt26c4779SiRYs0bdo0Ey3ftDhyxRWNHz9etWvX1vDhw5WdnV1q/7///W/NnTtXkqxAmDNnjkvNrFmzJMnlasUWLVrok08+calbvHjxJY9cy6N27dqlAvtS7rvvPhUVFWnevHku47Nnz5bNZlO/fv0q/PhZWVn6+uuvS40XFBRo27ZtZT5lvnjxYpenWhcuXKjCwkLr8UuOVi4+8igoKNCCBQsq3F9llPX4e/bsUWpqaqXma9iwoTp06KDly5e7PKWdnJxc5tfOBE9Pz1JHbmvWrCnzHHhV5unpKZvN5vJ/6MiRI1q7dm2F53I4HKXOJbdt21YeHh5GX350s+LIFVfUokULrVy5Uo8++qjCwsJc3qFp165dWrNmjfV6ufbt2ysmJkaLFy+2ns784osvtHz5cj344IMuR2jDhw/XyJEjFR0drXvuuUf79u1TUlLSVR1JdOrUSQsXLtS0adPUsmVLNWjQQL169Sqz9v7771fPnj31l7/8RUeOHFH79u21detWrVu3TmPHjlWLFi0q/Pj/+c9/dPvtt6tXr17q3bu3goODlZOTo//93//Vvn37NHbs2FLrKygoUO/evfU///M/ysjI0IIFC3TXXXdpwIABkqQ77rhDdevWVUxMjJ577jnZbDa9++671+1p1v79++vDDz/UQw89pKioKGVmZmrRokUKDw/XmTNnKjVnQkKCoqKidNddd+npp5/WyZMn9eabb6p169aVnvNy+vfvr6lTp2ro0KG64447dODAAa1YsULNmzc3/liXUlhYWObFVpL00EMPqXbt2lf9GFFRUZo1a5buvfdePf7448rJydH8+fPVsmVL7d+/v0JzpaSkaPTo0XrkkUd06623qrCwUO+++648PT0VHR191b3e7AhXlMuAAQO0f/9+zZgxQ+vWrdPChQvl4+Ojdu3aaebMmRoxYoRVu2TJEjVv3lyJiYn66KOPFBwcrIkTJ5Z6GnHEiBHKzMy0zuN1795dycnJ6t27d6X7jI+P1w8//KDp06fr9OnT6tGjxyXD1cPDQ//85z8VHx+vVatWadmyZWrWrJlmzJhhXd1cUa1atdKcOXO0adMmLViwQNnZ2dabJPzjH/9wudqzxLx587RixQrFx8frwoULeuyxx/TGG29YT8EGBgZqw4YNeuGFFzRp0iTVrVtXTzzxhHr37q3IyMhK9VkRTz31lLKysvTWW28pKSlJ4eHheu+997RmzZpKv8n7vffeqzVr1mjSpEmaOHGiWrRooWXLlmndunXX5I3j//znP+vs2bNauXKlVq1apY4dO2rjxo3605/+ZPyxLiU/P19PPvlkmfsyMzONhGuvXr20dOlSvfbaaxo7dqxCQ0P197//XUeOHKlwuLZv316RkZFav369fvzxR9WqVUvt27fX5s2beQvPcrA5b+SrQYCbWGJiooYOHaq9e/eqc+fO7m4HgEGccwUAwDDCFQAAwwhXAAAM45wrAACGceQKAIBhhCsAAIbxOtdyKC4u1vHjx1WnTp2res9cAEDV5nQ6dfr0aTVq1Mj6pKWyEK7lcPz48VKfRgEAqL6OHTumxo0bX3I/4VoOJW/qfuzYMesj2AAA1Y/D4VBISMgVP+OacC2HkqeC7XY74QoAuOIpQi5oAgDAMMIVAADDCFcAAAwjXAEAMIxwBQDAMMIVAADDCFcAAAwjXAEAMIxwBQDAMMIVAADDCFcAAAwjXAEAMIxwBQDAMMIVAADDCFcAAAzj81xvAh2mTXZ3CwBgVPqkye5u4apw5AoAgGGEKwAAhhGuAAAYRrgCAGAY4QoAgGGEKwAAhhGuAAAYRrgCAGAY4QoAgGGEKwAAhhGuAAAYRrgCAGAY4QoAgGGEKwAAhhGuAAAYRrgCAGAY4QoAgGGEKwAAhhGuAAAYRrgCAGAY4QoAgGGEKwAAhhGuAAAYRrgCAGAY4QoAgGGEKwAAhhGuAAAYRrgCAGAY4QoAgGGEKwAAhhGuAAAYRrgCAGAY4QoAgGFuDdfJkyfLZrO5bLfddpu1//z584qNjVVgYKD8/PwUHR2t7OxslzmOHj2qqKgo1apVSw0aNNBLL72kwsJCl5odO3aoY8eO8vHxUcuWLZWYmHg9lgcAqKbcfuTaunVrnThxwto+++wza9+4ceO0fv16rVmzRjt37tTx48f18MMPW/uLiooUFRWlgoIC7dq1S8uXL1diYqLi4+OtmszMTEVFRalnz55KT0/X2LFjNXz4cCUlJV3XdQIAqg8vtzfg5aXg4OBS43l5eVq6dKlWrlypXr16SZKWLVumsLAw7d69W926ddPWrVv19ddf6+OPP1ZQUJA6dOigV155RRMmTNDkyZPl7e2tRYsWKTQ0VDNnzpQkhYWF6bPPPtPs2bMVGRl5XdcKAKge3H7k+u2336pRo0Zq3ry5Bg8erKNHj0qS0tLSdOHCBfXp08eqve2229SkSROlpqZKklJTU9W2bVsFBQVZNZGRkXI4HDp06JBVc/EcJTUlc5QlPz9fDofDZQMAoLzcGq5du3ZVYmKitmzZooULFyozM1Pdu3fX6dOnlZWVJW9vbwUEBLjcJygoSFlZWZKkrKwsl2At2V+y73I1DodD586dK7OvhIQE+fv7W1tISIiJ5QIAqgm3Pi3cr18/69/t2rVT165d1bRpU61evVo1a9Z0W18TJ05UXFycddvhcBCwAIByc/vTwhcLCAjQrbfequ+++07BwcEqKChQbm6uS012drZ1jjY4OLjU1cMlt69UY7fbLxngPj4+stvtLhsAAOV1Q4XrmTNn9O9//1sNGzZUp06dVKNGDW3bts3an5GRoaNHjyoiIkKSFBERoQMHDignJ8eqSU5Olt1uV3h4uFVz8RwlNSVzAABgmlvD9cUXX9TOnTt15MgR7dq1Sw899JA8PT312GOPyd/fX8OGDVNcXJy2b9+utLQ0DR06VBEREerWrZskqW/fvgoPD9eTTz6pffv2KSkpSZMmTVJsbKx8fHwkSSNHjtT333+v8ePH6/Dhw1qwYIFWr16tcePGuXPpAICbmFvPuf7nP//RY489pp9//ln169fXXXfdpd27d6t+/fqSpNmzZ8vDw0PR0dHKz89XZGSkFixYYN3f09NTGzZs0KhRoxQREaHatWsrJiZGU6dOtWpCQ0O1ceNGjRs3TnPnzlXjxo21ZMkSXoYDALhmbE6n0+nuJm50DodD/v7+ysvLuyHPv3aYNtndLQCAUemTJru7hTKVNw9uqHOuAADcDAhXAAAMI1wBADCMcAUAwDDCFQAAwwhXAAAMI1wBADCMcAUAwDDCFQAAwwhXAAAMI1wBADCMcAUAwDDCFQAAwwhXAAAMI1wBADCMcAUAwDDCFQAAwwhXAAAMI1wBADCMcAUAwDDCFQAAwwhXAAAMI1wBADCMcAUAwDDCFQAAwwhXAAAMI1wBADCMcAUAwDDCFQAAwwhXAAAMI1wBADCMcAUAwDDCFQAAwwhXAAAMI1wBADCMcAUAwDDCFQAAwwhXAAAMI1wBADCMcAUAwDDCFQAAwwhXAAAMI1wBADCMcAUAwDDCFQAAwwhXAAAMI1wBADCMcAUAwDDCFQAAwwhXAAAMI1wBADCMcAUAwDDCFQAAwwhXAAAMI1wBADDshgnX1157TTabTWPHjrXGzp8/r9jYWAUGBsrPz0/R0dHKzs52ud/Ro0cVFRWlWrVqqUGDBnrppZdUWFjoUrNjxw517NhRPj4+atmypRITE6/DigAA1dUNEa579+7VW2+9pXbt2rmMjxs3TuvXr9eaNWu0c+dOHT9+XA8//LC1v6ioSFFRUSooKNCuXbu0fPlyJSYmKj4+3qrJzMxUVFSUevbsqfT0dI0dO1bDhw9XUlLSdVsfAKB6cXu4njlzRoMHD9Y//vEP1a1b1xrPy8vT0qVLNWvWLPXq1UudOnXSsmXLtGvXLu3evVuStHXrVn399dd677331KFDB/Xr10+vvPKK5s+fr4KCAknSokWLFBoaqpkzZyosLEyjR4/WwIEDNXv2bLesFwBw83N7uMbGxioqKkp9+vRxGU9LS9OFCxdcxm+77TY1adJEqampkqTU1FS1bdtWQUFBVk1kZKQcDocOHTpk1fx27sjISGuOsuTn58vhcLhsAACUl5c7H/z999/XV199pb1795bal5WVJW9vbwUEBLiMBwUFKSsry6q5OFhL9pfsu1yNw+HQuXPnVLNmzVKPnZCQoClTplR6XQCA6s1tR67Hjh3T888/rxUrVsjX19ddbZRp4sSJysvLs7Zjx465uyUAQBXitnBNS0tTTk6OOnbsKC8vL3l5eWnnzp1644035OXlpaCgIBUUFCg3N9flftnZ2QoODpYkBQcHl7p6uOT2lWrsdnuZR62S5OPjI7vd7rIBAFBebgvX3r1768CBA0pPT7e2zp07a/Dgwda/a9SooW3btln3ycjI0NGjRxURESFJioiI0IEDB5STk2PVJCcny263Kzw83Kq5eI6SmpI5AAAwzW3nXOvUqaM2bdq4jNWuXVuBgYHW+LBhwxQXF6d69erJbrdrzJgxioiIULdu3SRJffv2VXh4uJ588klNnz5dWVlZmjRpkmJjY+Xj4yNJGjlypObNm6fx48fr6aefVkpKilavXq2NGzde3wUDAKoNt17QdCWzZ8+Wh4eHoqOjlZ+fr8jISC1YsMDa7+npqQ0bNmjUqFGKiIhQ7dq1FRMTo6lTp1o1oaGh2rhxo8aNG6e5c+eqcePGWrJkiSIjI92xJABANWBzOp1Odzdxo3M4HPL391deXt4Nef61w7TJ7m4BAIxKnzTZ3S2Uqbx54PbXuQIAcLMhXAEAMIxwBQDAMMIVAADDCFcAAAwjXAEAMIxwBQDAMMIVAADDCFcAAAwjXAEAMIxwBQDAMMIVAADDCFcAAAwjXAEAMIxwBQDAMMIVAADDCFcAAAwjXAEAMIxwBQDAMMIVAADDCFcAAAwjXAEAMIxwBQDAMMIVAADDCFcAAAwjXAEAMIxwBQDAMMIVAADDCFcAAAwjXAEAMIxwBQDAMMIVAADDCFcAAAwjXAEAMIxwBQDAMMIVAADDCFcAAAwjXAEAMIxwBQDAMMIVAADDCFcAAAwjXAEAMIxwBQDAMMIVAADDCFcAAAyrVLg2b95cP//8c6nx3NxcNW/e/KqbAgCgKqtUuB45ckRFRUWlxvPz8/Xjjz9edVMAAFRlXhUp/uc//2n9OykpSf7+/tbtoqIibdu2Tc2aNTPWHAAAVVGFwvXBBx+UJNlsNsXExLjsq1Gjhpo1a6aZM2caaw4AgKqoQuFaXFwsSQoNDdXevXt1yy23XJOmAACoyioUriUyMzNN9wEAwE2jUuEqSdu2bdO2bduUk5NjHdGWePvtt6+6MQAAqqpKheuUKVM0depUde7cWQ0bNpTNZjPdFwAAVValwnXRokVKTEzUk08+abofAACqvEq9zrWgoEB33HHHVT/4woUL1a5dO9ntdtntdkVERGjz5s3W/vPnzys2NlaBgYHy8/NTdHS0srOzXeY4evSooqKiVKtWLTVo0EAvvfSSCgsLXWp27Nihjh07ysfHRy1btlRiYuJV9w4AwKVUKlyHDx+ulStXXvWDN27cWK+99prS0tL05ZdfqlevXnrggQd06NAhSdK4ceO0fv16rVmzRjt37tTx48f18MMPW/cvKipSVFSUCgoKtGvXLi1fvlyJiYmKj4+3ajIzMxUVFaWePXsqPT1dY8eO1fDhw5WUlHTV/QMAUBab0+l0VvROzz//vN555x21a9dO7dq1U40aNVz2z5o1q9IN1atXTzNmzNDAgQNVv359rVy5UgMHDpQkHT58WGFhYUpNTVW3bt20efNm9e/fX8ePH1dQUJCkX5+ynjBhgn766Sd5e3trwoQJ2rhxow4ePGg9xqBBg5Sbm6stW7aUqyeHwyF/f3/l5eXJbrdXem3XSodpk93dAgAYlT5psrtbKFN586BSR6779+9Xhw4d5OHhoYMHD+pf//qXtaWnp1eq4aKiIr3//vs6e/asIiIilJaWpgsXLqhPnz5WzW233aYmTZooNTVVkpSamqq2bdtawSpJkZGRcjgc1tFvamqqyxwlNSVzlCU/P18Oh8NlAwCgvCp1QdP27duNNXDgwAFFRETo/Pnz8vPz00cffaTw8HClp6fL29tbAQEBLvVBQUHKysqSJGVlZbkEa8n+kn2Xq3E4HDp37pxq1qxZqqeEhARNmTLF1BIBANWM2z9yrlWrVkpPT9eePXs0atQoxcTE6Ouvv3ZrTxMnTlReXp61HTt2zK39AACqlkodufbs2fOyr21NSUkp91ze3t5q2bKlJKlTp07au3ev5s6dq0cffVQFBQXKzc11OXrNzs5WcHCwJCk4OFhffPGFy3wlVxNfXPPbK4yzs7Nlt9vLPGqVJB8fH/n4+JR7DQAAXKxSR64dOnRQ+/btrS08PFwFBQX66quv1LZt26tqqLi4WPn5+erUqZNq1Kihbdu2WfsyMjJ09OhRRURESJIiIiJ04MAB5eTkWDXJycmy2+0KDw+3ai6eo6SmZA4AAEyr1JHr7NmzyxyfPHmyzpw5U+55Jk6cqH79+qlJkyY6ffq0Vq5cqR07dlgfZzds2DDFxcWpXr16stvtGjNmjCIiItStWzdJUt++fRUeHq4nn3xS06dPV1ZWliZNmqTY2FjryHPkyJGaN2+exo8fr6efflopKSlavXq1Nm7cWJmlAwBwRZV+b+GyPPHEE7r99tv1+uuvl6s+JydHQ4YM0YkTJ+Tv76927dopKSlJ99xzj6RfQ9zDw0PR0dHKz89XZGSkFixYYN3f09NTGzZs0KhRoxQREaHatWsrJiZGU6dOtWpCQ0O1ceNGjRs3TnPnzlXjxo21ZMkSRUZGmlw6AACWSr3O9VLeffddTZgwQcePHzc15Q2B17kCwPVV1V/nWqkj14vfJUmSnE6nTpw4oS+//FJ//etfKzMlAAA3jUqFq7+/v8ttDw8PtWrVSlOnTlXfvn2NNAYAQFVVqXBdtmyZ6T4AALhpXNUFTWlpafrmm28kSa1bt9Yf/vAHI00BAFCVVSpcc3JyNGjQIO3YscN6g4fc3Fz17NlT77//vurXr2+yRwAAqpRKvYnEmDFjdPr0aR06dEgnT57UyZMndfDgQTkcDj333HOmewQAoEqp1JHrli1b9PHHHyssLMwaCw8P1/z587mgCQBQ7VXqyLW4uLjUZ7hKUo0aNVRcXHzVTQEAUJVVKlx79eql559/3uXNIn788UeNGzdOvXv3NtYcAABVUaXCdd68eXI4HGrWrJlatGihFi1aKDQ0VA6HQ2+++abpHgEAqFIqdc41JCREX331lT7++GMdPnxYkhQWFqY+ffoYbQ4AgKqoQkeuKSkpCg8Pl8PhkM1m0z333KMxY8ZozJgx6tKli1q3bq1PP/30WvUKAECVUKFwnTNnjkaMGFHmmxX7+/vr2Wef1axZs4w1BwBAVVShcN23b5/uvffeS+7v27ev0tLSrropAACqsgqFa3Z2dpkvwSnh5eWln3766aqbAgCgKqtQuP7ud7/TwYMHL7l///79atiw4VU3BQBAVVahcL3vvvv017/+VefPny+179y5c3r55ZfVv39/Y80BAFAVVeilOJMmTdKHH36oW2+9VaNHj1arVq0kSYcPH9b8+fNVVFSkv/zlL9ekUQAAqooKhWtQUJB27dqlUaNGaeLEiXI6nZIkm82myMhIzZ8/X0FBQdekUQAAqooKv4lE06ZNtWnTJp06dUrfffednE6nfv/736tu3brXoj8AAKqcSn9Yet26ddWlSxeTvQAAcFOo1HsLAwCASyNcAQAwjHAFAMAwwhUAAMMIVwAADCNcAQAwjHAFAMAwwhUAAMMIVwAADCNcAQAwjHAFAMAwwhUAAMMIVwAADCNcAQAwjHAFAMAwwhUAAMMIVwAADCNcAQAwjHAFAMAwwhUAAMMIVwAADCNcAQAwjHAFAMAwwhUAAMMIVwAADCNcAQAwjHAFAMAwwhUAAMMIVwAADCNcAQAwjHAFAMAwwhUAAMMIVwAADCNcAQAwzK3hmpCQoC5duqhOnTpq0KCBHnzwQWVkZLjUnD9/XrGxsQoMDJSfn5+io6OVnZ3tUnP06FFFRUWpVq1aatCggV566SUVFha61OzYsUMdO3aUj4+PWrZsqcTExGu9PABANeXWcN25c6diY2O1e/duJScn68KFC+rbt6/Onj1r1YwbN07r16/XmjVrtHPnTh0/flwPP/ywtb+oqEhRUVEqKCjQrl27tHz5ciUmJio+Pt6qyczMVFRUlHr27Kn09HSNHTtWw4cPV1JS0nVdLwCgerA5nU6nu5so8dNPP6lBgwbauXOn7r77buXl5al+/fpauXKlBg4cKEk6fPiwwsLClJqaqm7dumnz5s3q37+/jh8/rqCgIEnSokWLNGHCBP3000/y9vbWhAkTtHHjRh08eNB6rEGDBik3N1dbtmy5Yl8Oh0P+/v7Ky8uT3W6/Nou/Ch2mTXZ3CwBgVPqkye5uoUzlzYMb6pxrXl6eJKlevXqSpLS0NF24cEF9+vSxam677TY1adJEqampkqTU1FS1bdvWClZJioyMlMPh0KFDh6yai+coqSmZ47fy8/PlcDhcNgAAyuuGCdfi4mKNHTtWd955p9q0aSNJysrKkre3twICAlxqg4KClJWVZdVcHKwl+0v2Xa7G4XDo3LlzpXpJSEiQv7+/tYWEhBhZIwCgerhhwjU2NlYHDx7U+++/7+5WNHHiROXl5VnbsWPH3N0SAKAK8XJ3A5I0evRobdiwQZ988okaN25sjQcHB6ugoEC5ubkuR6/Z2dkKDg62ar744guX+UquJr645rdXGGdnZ8tut6tmzZql+vHx8ZGPj4+RtQEAqh+3Hrk6nU6NHj1aH330kVJSUhQaGuqyv1OnTqpRo4a2bdtmjWVkZOjo0aOKiIiQJEVEROjAgQPKycmxapKTk2W32xUeHm7VXDxHSU3JHAAAmOTWI9fY2FitXLlS69atU506daxzpP7+/qpZs6b8/f01bNgwxcXFqV69erLb7RozZowiIiLUrVs3SVLfvn0VHh6uJ598UtOnT1dWVpYmTZqk2NhY6+hz5MiRmjdvnsaPH6+nn35aKSkpWr16tTZu3Oi2tQMAbl5uPXJduHCh8vLy9Mc//lENGza0tlWrVlk1s2fPVv/+/RUdHa27775bwcHB+vDDD639np6e2rBhgzw9PRUREaEnnnhCQ4YM0dSpU62a0NBQbdy4UcnJyWrfvr1mzpypJUuWKDIy8rquFwBQPdxQr3O9UfE6VwC4vnidKwAAcEG4AgBgGOEKAIBhhCsAAIYRrgAAGEa4AgBgGOEKAIBhhCsAAIYRrgAAGEa4AgBgGOEKAIBhhCsAAIYRrgAAGEa4AgBgGOEKAIBhhCsAAIYRrgAAGEa4AgBgGOEKAIBhhCsAAIYRrgAAGEa4AgBgGOEKAIBhhCsAAIYRrgAAGEa4AgBgGOEKAIBhhCsAAIYRrgAAGEa4AgBgGOEKAIBhhCsAAIYRrgAAGEa4AgBgGOEKAIBhhCsAAIYRrgAAGEa4AgBgGOEKAIBhhCsAAIYRrgAAGEa4AgBgGOEKAIBhhCsAAIYRrgAAGEa4AgBgGOEKAIBhhCsAAIYRrgAAGEa4AgBgGOEKAIBhhCsAAIYRrgAAGEa4AgBgGOEKAIBhbg3XTz75RPfff78aNWokm82mtWvXuux3Op2Kj49Xw4YNVbNmTfXp00fffvutS83Jkyc1ePBg2e12BQQEaNiwYTpz5oxLzf79+9W9e3f5+voqJCRE06dPv9ZLAwBUY24N17Nnz6p9+/aaP39+mfunT5+uN954Q4sWLdKePXtUu3ZtRUZG6vz581bN4MGDdejQISUnJ2vDhg365JNP9Mwzz1j7HQ6H+vbtq6ZNmyotLU0zZszQ5MmTtXjx4mu+PgBA9eTlzgfv16+f+vXrV+Y+p9OpOXPmaNKkSXrggQckSe+8846CgoK0du1aDRo0SN988422bNmivXv3qnPnzpKkN998U/fdd59ef/11NWrUSCtWrFBBQYHefvtteXt7q3Xr1kpPT9esWbNcQhgAAFNu2HOumZmZysrKUp8+fawxf39/de3aVampqZKk1NRUBQQEWMEqSX369JGHh4f27Nlj1dx9993y9va2aiIjI5WRkaFTp06V+dj5+flyOBwuGwAA5XXDhmtWVpYkKSgoyGU8KCjI2peVlaUGDRq47Pfy8lK9evVcasqa4+LH+K2EhAT5+/tbW0hIyNUvCABQbdyw4epOEydOVF5enrUdO3bM3S0BAKqQGzZcg4ODJUnZ2dku49nZ2da+4OBg5eTkuOwvLCzUyZMnXWrKmuPix/gtHx8f2e12lw0AgPK6YcM1NDRUwcHB2rZtmzXmcDi0Z88eRURESJIiIiKUm5urtLQ0qyYlJUXFxcXq2rWrVfPJJ5/owoULVk1ycrJatWqlunXrXqfVAACqE7eG65kzZ5Senq709HRJv17ElJ6erqNHj8pms2ns2LGaNm2a/vnPf+rAgQMaMmSIGjVqpAcffFCSFBYWpnvvvVcjRozQF198oc8//1yjR4/WoEGD1KhRI0nS448/Lm9vbw0bNkyHDh3SqlWrNHfuXMXFxblp1QCAm51bX4rz5ZdfqmfPntbtksCLiYlRYmKixo8fr7Nnz+qZZ55Rbm6u7rrrLm3ZskW+vr7WfVasWKHRo0erd+/e8vDwUHR0tN544w1rv7+/v7Zu3arY2Fh16tRJt9xyi+Lj43kZDgDgmrE5nU6nu5u40TkcDvn7+ysvL++GPP/aYdpkd7cAAEalT5rs7hbKVN48uGHPuQIAUFURrgAAGEa4AgBgGOEKAIBhhCsAAIYRrgAAGEa4AgBgGOEKAIBhhCsAAIYRrgAAGEa4AgBgGOEKAIBhhCsAAIYRrgAAGEa4AgBgGOEKAIBhhCsAAIYRrgAAGEa4AgBgGOEKAIBhhCsAAIYRrgAAGEa4AgBgGOEKAIBhhCsAAIYRrgAAGEa4AgBgGOEKAIBhhCsAAIYRrgAAGEa4AgBgGOEKAIBhhCsAAIYRrgAAGEa4AgBgGOEKAIBhhCsAAIYRrgAAGEa4AgBgGOEKAIBhhCsAAIYRrgAAGEa4AgBgGOEKAIBhhCsAAIYRrgAAGEa4AgBgGOEKAIBhhCsAAIYRrgAAGEa4AgBgGOEKAIBhhCsAAIYRrgAAGEa4AgBgWLUK1/nz56tZs2by9fVV165d9cUXX7i7JQDATajahOuqVasUFxenl19+WV999ZXat2+vyMhI5eTkuLs1AMBNptqE66xZszRixAgNHTpU4eHhWrRokWrVqqW3337b3a0BAG4yXu5u4HooKChQWlqaJk6caI15eHioT58+Sk1NLVWfn5+v/Px863ZeXp4kyeFwXPtmK6HofP6ViwCgCrlRf9+W9OV0Oi9bVy3C9b///a+KiooUFBTkMh4UFKTDhw+Xqk9ISNCUKVNKjYeEhFyzHgEA/z//V19zdwuXdfr0afn7+19yf7UI14qaOHGi4uLirNvFxcU6efKkAgMDZbPZ3NgZ4D4Oh0MhISE6duyY7Ha7u9sB3MLpdOr06dNq1KjRZeuqRbjecsst8vT0VHZ2tst4dna2goODS9X7+PjIx8fHZSwgIOBatghUGXa7nXBFtXa5I9YS1eKCJm9vb3Xq1Enbtm2zxoqLi7Vt2zZFRES4sTMAwM2oWhy5SlJcXJxiYmLUuXNn3X777ZozZ47Onj2roUOHurs1AMBNptqE66OPPqqffvpJ8fHxysrKUocOHbRly5ZSFzkBKJuPj49efvnlUqdMAJRmc17pemIAAFAh1eKcKwAA1xPhCgCAYYQrAACGEa4AABhGuAIoFz6yESg/whXAFfGRjUDF8FIcAFfUtWtXdenSRfPmzZP06zuchYSEaMyYMfrTn/7k5u6AGw9HrgAuq+QjG/v06WONXe4jGwEQrgCu4HIf2ZiVleWmroAbG+EKAIBhhCuAy6roRzYCIFwBXAEf2QhUXLX5VBwAlcdHNgIVQ7gCuCI+shGoGF7nCgCAYZxzBQDAMMIVAADDCFcAAAwjXAEAMIxwBQDAMMIVAADDCFcAAAwjXAEAMIxwBXBFiYmJCggIuOp5bDab1q5de9XzADc6whWoJp566ik9+OCD7m4DqBYIVwAADCNcAWjWrFlq27atateurZCQEP2///f/dObMmVJ1a9eu1e9//3v5+voqMjJSx44dc9m/bt06dezYUb6+vmrevLmmTJmiwsLC67UM4IZBuAKQh4eH3njjDR06dEjLly9XSkqKxo8f71Lzyy+/6NVXX9U777yjzz//XLm5uRo0aJC1/9NPP9WQIUP0/PPP6+uvv9Zbb72lxMREvfrqq9d7OYD7OQFUCzExMc4HHnigXLVr1qxxBgYGWreXLVvmlOTcvXu3NfbNN984JTn37NnjdDqdzt69ezv/9re/uczz7rvvOhs2bGjdluT86KOPKr8IoIrg81wB6OOPP1ZCQoIOHz4sh8OhwsJCnT9/Xr/88otq1aolSfLy8lKXLl2s+9x2220KCAjQN998o9tvv1379u3T559/7nKkWlRUVGoeoDogXIFq7siRI+rfv79GjRqlV199VfXq1dNnn32mYcOGqaCgoNyheObMGU2ZMkUPP/xwqX2+vr6m2wZuaIQrUM2lpaWpuLhYM2fOlIfHr5dhrF69ulRdYWGhvvzyS91+++2SpIyMDOXm5iosLEyS1LFjR2VkZKhly5bXr3ngBkW4AtVIXl6e0tPTXcZuueUWXbhwQW+++abuv/9+ff7551q0aFGp+9aoUUNjxozRG2+8IS8vL40ePVrdunWzwjY+Pl79+/dXkyZNNHDgQHl4eGjfvn06ePCgpk2bdj2WB9wwuFoYqEZ27NihP/zhDy7bu+++q1mzZunvf/+72rRpoxUrVighIaHUfWvVqqUJEybo8ccf15133ik/Pz+tWrXK2h8ZGakNGzZo69at6tKli7p166bZs2eradOm13OJwA3B5nQ6ne5uAgCAmwlHrgAAGEa4AgBgGOEKAIBhhCsAAIYRrgAAGEa4AgBgGOEKAIBhhCsAAIYRrgAAGEa4AgBgGOEKAIBh/x917wPPxjkSBwAAAABJRU5ErkJggg==",
      "text/plain": [
       "<Figure size 500x500 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# create a label that 1 for spam and 0 for ham\n",
    "df['label'] = df['v1'].apply(lambda x: 1 if x == 'spam' else 0)\n",
    "\n",
    "# Visualization using count plot for 'spam' and 'ham' counts\n",
    "plt.figure(figsize=(5, 5))\n",
    "sns.countplot(df['label'], palette='viridis')  # Using a color palette for better visualization\n",
    "plt.xlabel('Label')\n",
    "plt.ylabel('Count')\n",
    "plt.title('Count of Spam and Ham Emails')\n",
    "plt.show()\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5e76091b-8030-4824-a65c-3db7aaac2a22",
   "metadata": {},
   "source": [
    "## 4. Text Preprocessing"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 77,
   "id": "c171d9da-9691-4048-a59b-ef1b7b30bde0",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Text preprocessing\n",
    "from sklearn.feature_extraction.text import CountVectorizer\n",
    "from nltk.corpus import stopwords\n",
    "import string\n",
    "import nltk"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 78,
   "id": "2564b9ce-2cd3-4cf4-bae4-159c3ecaff8c",
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[nltk_data] Error loading stopwords: <urlopen error [WinError 10060] A\n",
      "[nltk_data]     connection attempt failed because the connected party\n",
      "[nltk_data]     did not properly respond after a period of time, or\n",
      "[nltk_data]     established connection failed because connected host\n",
      "[nltk_data]     has failed to respond>\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "{'a',\n",
       " 'about',\n",
       " 'above',\n",
       " 'after',\n",
       " 'again',\n",
       " 'against',\n",
       " 'ain',\n",
       " 'all',\n",
       " 'am',\n",
       " 'an',\n",
       " 'and',\n",
       " 'any',\n",
       " 'are',\n",
       " 'aren',\n",
       " \"aren't\",\n",
       " 'as',\n",
       " 'at',\n",
       " 'be',\n",
       " 'because',\n",
       " 'been',\n",
       " 'before',\n",
       " 'being',\n",
       " 'below',\n",
       " 'between',\n",
       " 'both',\n",
       " 'but',\n",
       " 'by',\n",
       " 'can',\n",
       " 'couldn',\n",
       " \"couldn't\",\n",
       " 'd',\n",
       " 'did',\n",
       " 'didn',\n",
       " \"didn't\",\n",
       " 'do',\n",
       " 'does',\n",
       " 'doesn',\n",
       " \"doesn't\",\n",
       " 'doing',\n",
       " 'don',\n",
       " \"don't\",\n",
       " 'down',\n",
       " 'during',\n",
       " 'each',\n",
       " 'few',\n",
       " 'for',\n",
       " 'from',\n",
       " 'further',\n",
       " 'had',\n",
       " 'hadn',\n",
       " \"hadn't\",\n",
       " 'has',\n",
       " 'hasn',\n",
       " \"hasn't\",\n",
       " 'have',\n",
       " 'haven',\n",
       " \"haven't\",\n",
       " 'having',\n",
       " 'he',\n",
       " 'her',\n",
       " 'here',\n",
       " 'hers',\n",
       " 'herself',\n",
       " 'him',\n",
       " 'himself',\n",
       " 'his',\n",
       " 'how',\n",
       " 'i',\n",
       " 'if',\n",
       " 'in',\n",
       " 'into',\n",
       " 'is',\n",
       " 'isn',\n",
       " \"isn't\",\n",
       " 'it',\n",
       " \"it's\",\n",
       " 'its',\n",
       " 'itself',\n",
       " 'just',\n",
       " 'll',\n",
       " 'm',\n",
       " 'ma',\n",
       " 'me',\n",
       " 'mightn',\n",
       " \"mightn't\",\n",
       " 'more',\n",
       " 'most',\n",
       " 'mustn',\n",
       " \"mustn't\",\n",
       " 'my',\n",
       " 'myself',\n",
       " 'needn',\n",
       " \"needn't\",\n",
       " 'no',\n",
       " 'nor',\n",
       " 'not',\n",
       " 'now',\n",
       " 'o',\n",
       " 'of',\n",
       " 'off',\n",
       " 'on',\n",
       " 'once',\n",
       " 'only',\n",
       " 'or',\n",
       " 'other',\n",
       " 'our',\n",
       " 'ours',\n",
       " 'ourselves',\n",
       " 'out',\n",
       " 'over',\n",
       " 'own',\n",
       " 're',\n",
       " 's',\n",
       " 'same',\n",
       " 'shan',\n",
       " \"shan't\",\n",
       " 'she',\n",
       " \"she's\",\n",
       " 'should',\n",
       " \"should've\",\n",
       " 'shouldn',\n",
       " \"shouldn't\",\n",
       " 'so',\n",
       " 'some',\n",
       " 'such',\n",
       " 't',\n",
       " 'than',\n",
       " 'that',\n",
       " \"that'll\",\n",
       " 'the',\n",
       " 'their',\n",
       " 'theirs',\n",
       " 'them',\n",
       " 'themselves',\n",
       " 'then',\n",
       " 'there',\n",
       " 'these',\n",
       " 'they',\n",
       " 'this',\n",
       " 'those',\n",
       " 'through',\n",
       " 'to',\n",
       " 'too',\n",
       " 'under',\n",
       " 'until',\n",
       " 'up',\n",
       " 've',\n",
       " 'very',\n",
       " 'was',\n",
       " 'wasn',\n",
       " \"wasn't\",\n",
       " 'we',\n",
       " 'were',\n",
       " 'weren',\n",
       " \"weren't\",\n",
       " 'what',\n",
       " 'when',\n",
       " 'where',\n",
       " 'which',\n",
       " 'while',\n",
       " 'who',\n",
       " 'whom',\n",
       " 'why',\n",
       " 'will',\n",
       " 'with',\n",
       " 'won',\n",
       " \"won't\",\n",
       " 'wouldn',\n",
       " \"wouldn't\",\n",
       " 'y',\n",
       " 'you',\n",
       " \"you'd\",\n",
       " \"you'll\",\n",
       " \"you're\",\n",
       " \"you've\",\n",
       " 'your',\n",
       " 'yours',\n",
       " 'yourself',\n",
       " 'yourselves'}"
      ]
     },
     "execution_count": 78,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Download stopwords from NLTK\n",
    "nltk.download('stopwords')\n",
    "stop_words = set(stopwords.words('english'))\n",
    "\n",
    "stop_words"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 80,
   "id": "12428362-1303-4d7f-9273-c51eabe391aa",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>v1</th>\n",
       "      <th>v2</th>\n",
       "      <th>label</th>\n",
       "      <th>processed_text</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>ham</td>\n",
       "      <td>Go until jurong point, crazy.. Available only ...</td>\n",
       "      <td>0</td>\n",
       "      <td>go jurong point crazy available bugis n great ...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>ham</td>\n",
       "      <td>Ok lar... Joking wif u oni...</td>\n",
       "      <td>0</td>\n",
       "      <td>ok lar joking wif u oni</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>spam</td>\n",
       "      <td>Free entry in 2 a wkly comp to win FA Cup fina...</td>\n",
       "      <td>1</td>\n",
       "      <td>free entry 2 wkly comp win fa cup final tkts 2...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>ham</td>\n",
       "      <td>U dun say so early hor... U c already then say...</td>\n",
       "      <td>0</td>\n",
       "      <td>u dun say early hor u c already say</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>ham</td>\n",
       "      <td>Nah I don't think he goes to usf, he lives aro...</td>\n",
       "      <td>0</td>\n",
       "      <td>nah dont think goes usf lives around though</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "     v1                                                 v2  label  \\\n",
       "0   ham  Go until jurong point, crazy.. Available only ...      0   \n",
       "1   ham                      Ok lar... Joking wif u oni...      0   \n",
       "2  spam  Free entry in 2 a wkly comp to win FA Cup fina...      1   \n",
       "3   ham  U dun say so early hor... U c already then say...      0   \n",
       "4   ham  Nah I don't think he goes to usf, he lives aro...      0   \n",
       "\n",
       "                                      processed_text  \n",
       "0  go jurong point crazy available bugis n great ...  \n",
       "1                            ok lar joking wif u oni  \n",
       "2  free entry 2 wkly comp win fa cup final tkts 2...  \n",
       "3                u dun say early hor u c already say  \n",
       "4        nah dont think goes usf lives around though  "
      ]
     },
     "execution_count": 80,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "def preprocess_text(text):\n",
    "    # Convert to lowercase\n",
    "    text = text.lower()\n",
    "    # Remove punctuation\n",
    "    text = ''.join([char for char in text if char not in string.punctuation])\n",
    "    # Remove stopwords\n",
    "    text = ' '.join([word for word in text.split() if word not in stop_words])\n",
    "    return text\n",
    "\n",
    "df['processed_text'] = df['v2'].apply(preprocess_text)\n",
    "\n",
    "df.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0cbcc315-31a7-42ad-9db6-e00f2571dc7a",
   "metadata": {},
   "source": [
    "## 5. train, test split"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 82,
   "id": "0bf3f7c9-4f43-4ea0-bcc2-b25d43ee2afe",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Split the data\n",
    "X = df['processed_text'] # feature\n",
    "y = df['label'] # target variable\n",
    "\n",
    "X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6a1a6653-95af-44c3-b1ab-fed1874ad66b",
   "metadata": {},
   "source": [
    "### 5.1 Model training (Naive Bayes Classifier)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 86,
   "id": "16fa1143-ff1f-4d53-bc2c-fa057a989d23",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<style>#sk-container-id-2 {color: black;}#sk-container-id-2 pre{padding: 0;}#sk-container-id-2 div.sk-toggleable {background-color: white;}#sk-container-id-2 label.sk-toggleable__label {cursor: pointer;display: block;width: 100%;margin-bottom: 0;padding: 0.3em;box-sizing: border-box;text-align: center;}#sk-container-id-2 label.sk-toggleable__label-arrow:before {content: \"▸\";float: left;margin-right: 0.25em;color: #696969;}#sk-container-id-2 label.sk-toggleable__label-arrow:hover:before {color: black;}#sk-container-id-2 div.sk-estimator:hover label.sk-toggleable__label-arrow:before {color: black;}#sk-container-id-2 div.sk-toggleable__content {max-height: 0;max-width: 0;overflow: hidden;text-align: left;background-color: #f0f8ff;}#sk-container-id-2 div.sk-toggleable__content pre {margin: 0.2em;color: black;border-radius: 0.25em;background-color: #f0f8ff;}#sk-container-id-2 input.sk-toggleable__control:checked~div.sk-toggleable__content {max-height: 200px;max-width: 100%;overflow: auto;}#sk-container-id-2 input.sk-toggleable__control:checked~label.sk-toggleable__label-arrow:before {content: \"▾\";}#sk-container-id-2 div.sk-estimator input.sk-toggleable__control:checked~label.sk-toggleable__label {background-color: #d4ebff;}#sk-container-id-2 div.sk-label input.sk-toggleable__control:checked~label.sk-toggleable__label {background-color: #d4ebff;}#sk-container-id-2 input.sk-hidden--visually {border: 0;clip: rect(1px 1px 1px 1px);clip: rect(1px, 1px, 1px, 1px);height: 1px;margin: -1px;overflow: hidden;padding: 0;position: absolute;width: 1px;}#sk-container-id-2 div.sk-estimator {font-family: monospace;background-color: #f0f8ff;border: 1px dotted black;border-radius: 0.25em;box-sizing: border-box;margin-bottom: 0.5em;}#sk-container-id-2 div.sk-estimator:hover {background-color: #d4ebff;}#sk-container-id-2 div.sk-parallel-item::after {content: \"\";width: 100%;border-bottom: 1px solid gray;flex-grow: 1;}#sk-container-id-2 div.sk-label:hover label.sk-toggleable__label {background-color: #d4ebff;}#sk-container-id-2 div.sk-serial::before {content: \"\";position: absolute;border-left: 1px solid gray;box-sizing: border-box;top: 0;bottom: 0;left: 50%;z-index: 0;}#sk-container-id-2 div.sk-serial {display: flex;flex-direction: column;align-items: center;background-color: white;padding-right: 0.2em;padding-left: 0.2em;position: relative;}#sk-container-id-2 div.sk-item {position: relative;z-index: 1;}#sk-container-id-2 div.sk-parallel {display: flex;align-items: stretch;justify-content: center;background-color: white;position: relative;}#sk-container-id-2 div.sk-item::before, #sk-container-id-2 div.sk-parallel-item::before {content: \"\";position: absolute;border-left: 1px solid gray;box-sizing: border-box;top: 0;bottom: 0;left: 50%;z-index: -1;}#sk-container-id-2 div.sk-parallel-item {display: flex;flex-direction: column;z-index: 1;position: relative;background-color: white;}#sk-container-id-2 div.sk-parallel-item:first-child::after {align-self: flex-end;width: 50%;}#sk-container-id-2 div.sk-parallel-item:last-child::after {align-self: flex-start;width: 50%;}#sk-container-id-2 div.sk-parallel-item:only-child::after {width: 0;}#sk-container-id-2 div.sk-dashed-wrapped {border: 1px dashed gray;margin: 0 0.4em 0.5em 0.4em;box-sizing: border-box;padding-bottom: 0.4em;background-color: white;}#sk-container-id-2 div.sk-label label {font-family: monospace;font-weight: bold;display: inline-block;line-height: 1.2em;}#sk-container-id-2 div.sk-label-container {text-align: center;}#sk-container-id-2 div.sk-container {/* jupyter's `normalize.less` sets `[hidden] { display: none; }` but bootstrap.min.css set `[hidden] { display: none !important; }` so we also need the `!important` here to be able to override the default hidden behavior on the sphinx rendered scikit-learn.org. See: https://github.com/scikit-learn/scikit-learn/issues/21755 */display: inline-block !important;position: relative;}#sk-container-id-2 div.sk-text-repr-fallback {display: none;}</style><div id=\"sk-container-id-2\" class=\"sk-top-container\"><div class=\"sk-text-repr-fallback\"><pre>MultinomialNB()</pre><b>In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook. <br />On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.</b></div><div class=\"sk-container\" hidden><div class=\"sk-item\"><div class=\"sk-estimator sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-2\" type=\"checkbox\" checked><label for=\"sk-estimator-id-2\" class=\"sk-toggleable__label sk-toggleable__label-arrow\">MultinomialNB</label><div class=\"sk-toggleable__content\"><pre>MultinomialNB()</pre></div></div></div></div></div>"
      ],
      "text/plain": [
       "MultinomialNB()"
      ]
     },
     "execution_count": 86,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Feature extraction using Count Vectorization\n",
    "\n",
    "vectorizer = CountVectorizer()\n",
    "X_train_count = vectorizer.fit_transform(X_train)\n",
    "X_test_count = vectorizer.transform(X_test)\n",
    "\n",
    "# Train the Naive Bayes classifier\n",
    "nb_classifier = MultinomialNB()\n",
    "nb_classifier.fit(X_train_count, y_train)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3cf91571-0c65-4d11-8577-044ff2f7d4d4",
   "metadata": {},
   "source": [
    "### 5.2 Model Evaluation"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 84,
   "id": "449d1aaa-bc7a-4d63-b83b-d0e39bb195cf",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Accuracy: 0.9802690582959641\n",
      "Confusion Matrix:\n",
      " [[961   4]\n",
      " [ 18 132]]\n",
      "Classification Report:\n",
      "               precision    recall  f1-score   support\n",
      "\n",
      "           0       0.98      1.00      0.99       965\n",
      "           1       0.97      0.88      0.92       150\n",
      "\n",
      "    accuracy                           0.98      1115\n",
      "   macro avg       0.98      0.94      0.96      1115\n",
      "weighted avg       0.98      0.98      0.98      1115\n",
      "\n"
     ]
    }
   ],
   "source": [
    "# Predictions\n",
    "y_pred = nb_classifier.predict(X_test_count)\n",
    "\n",
    "# Model evaluation\n",
    "accuracy = accuracy_score(y_test, y_pred)\n",
    "conf_matrix = confusion_matrix(y_test, y_pred)\n",
    "classification_rep = classification_report(y_test, y_pred)\n",
    "\n",
    "print(\"Accuracy:\", accuracy)\n",
    "print(\"Confusion Matrix:\\n\", conf_matrix)\n",
    "print(\"Classification Report:\\n\", classification_rep)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "75c8e842-1c13-42c7-88c0-144ce635318c",
   "metadata": {},
   "source": [
    "### 5.3 Model testing"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 85,
   "id": "f779bcc1-867f-4171-9516-3a1e6a7c5e9f",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Predictions for new emails:\n",
      "Email: Win a free iPhone now!, Predicted label: spam\n",
      "Email: Meeting scheduled for tomorrow, Predicted label: ham\n"
     ]
    }
   ],
   "source": [
    "# Example prediction for new data\n",
    "new_emails = ['Win a free iPhone now!', 'Meeting scheduled for tomorrow']\n",
    "new_emails_processed = [preprocess_text(email) for email in new_emails]\n",
    "new_emails_count = vectorizer.transform(new_emails_processed)\n",
    "predictions = nb_classifier.predict(new_emails_count)\n",
    "\n",
    "print(\"Predictions for new emails:\")\n",
    "for i, email in enumerate(new_emails):\n",
    "    print(f\"Email: {email}, Predicted label: {'spam' if predictions[i] == 1 else 'ham'}\")\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "307547cf-93c0-46b2-beec-6f7da9883d4a",
   "metadata": {},
   "source": [
    "## 6. Conclusion\n",
    "\n",
    "By preprocessing the data, training the model, and evaluating its performance, we were able to classify emails into spam and non-spam categories with a certain level of accuracy.\n",
    "\n",
    "In conclusion, we successfully built an email spam detection system using the Naive Bayes algorithm."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.11.5"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}