{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "b95ae0d2",
   "metadata": {
    "_cell_guid": "b1076dfc-b9ad-4769-8c92-a6c4dae69d19",
    "_uuid": "8f2839f25d086af736a60e9eeb907d3b93b6e0e5",
    "papermill": {
     "duration": 0.026717,
     "end_time": "2022-08-28T13:50:03.826565",
     "exception": false,
     "start_time": "2022-08-28T13:50:03.799848",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "<div style=\"padding:25px;color:white;margin:0;font-size:250%;text-align:center;display:fill;border-radius:10px;background-color:#298f50;overflow:hidden;font-weight:500;font-family:magra\">Introduction</div>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8ff3e001",
   "metadata": {
    "papermill": {
     "duration": 0.021342,
     "end_time": "2022-08-28T13:50:03.869546",
     "exception": false,
     "start_time": "2022-08-28T13:50:03.848204",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "* In this work i will do a cluster analysis for the problem of customer segmentation in the store \n",
    "* Cluster analysis refers to an unsupervised machine learning task (when there are no labels in the dataset)\n",
    "* Cluster analysis is used where we need to break the general population into different groups, where the objects in each group are most similar to each other. In our example, we break the general population into separate groups in order to analyze these groups separately and, for example, make recommendations for purchases for them\n",
    "* As part of this work, all stages of machine learning for cluster analysis will be considered\n",
    "* The goal of this work is to obtain the necessary recommendations for business based on the results of cluster analysis\n",
    "\n",
    "## If the work is useful for you, please upvote\n",
    "## If you have any questions about the work, I will be happy to answer in the comments"
   ]
  },
  {
   "attachments": {
    "3998abe6-3b5c-438e-96bb-98f1655a94ad.jpg": {
     "image/jpeg": "[encoded data, 336768 characters]"
    }
   },
   "cell_type": "markdown",
   "id": "41472a72",
   "metadata": {
    "papermill": {
     "duration": 0.021129,
     "end_time": "2022-08-28T13:50:03.912088",
     "exception": false,
     "start_time": "2022-08-28T13:50:03.890959",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "![pre-strategii-segmentatsii-klientov-vse-marketologi-dolzhny-uchitsya.jpg](attachment:3998abe6-3b5c-438e-96bb-98f1655a94ad.jpg)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b9181d5d",
   "metadata": {
    "papermill": {
     "duration": 0.022276,
     "end_time": "2022-08-28T13:50:03.958617",
     "exception": false,
     "start_time": "2022-08-28T13:50:03.936341",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "<div style=\"padding:25px;color:white;margin:0;font-size:250%;text-align:center;display:fill;border-radius:10px;background-color:#298f50;overflow:hidden;font-weight:500;font-family:magra\">Table of content</div>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "72dd993b",
   "metadata": {
    "papermill": {
     "duration": 0.021173,
     "end_time": "2022-08-28T13:50:04.002367",
     "exception": false,
     "start_time": "2022-08-28T13:50:03.981194",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "1) **Choice of Metrics**  \n",
    "2) **Library import & Dataset**  \n",
    "3.1) **Feature generation**  \n",
    "3.2) **Data cleaning**              \n",
    "3.3) **Data scaling (Normalization | Standartization)**  \n",
    "4) **Dimensionality reduction**  \n",
    "5) **Clustering**  \n",
    "6) **Business Recommendations**  "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "25ec315a",
   "metadata": {
    "papermill": {
     "duration": 0.021734,
     "end_time": "2022-08-28T13:50:04.045639",
     "exception": false,
     "start_time": "2022-08-28T13:50:04.023905",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "<div style=\"padding:25px;color:white;margin:0;font-size:250%;text-align:center;display:fill;border-radius:10px;background-color:#298f50;overflow:hidden;font-weight:500;font-family:magra\">1) Choice of Metrics  </div>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "99948a3c",
   "metadata": {
    "papermill": {
     "duration": 0.021363,
     "end_time": "2022-08-28T13:50:04.088385",
     "exception": false,
     "start_time": "2022-08-28T13:50:04.067022",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "There are various metrics for clustering, for example:\n",
    "* Silhouette coefficient  (https://scikit-learn.org/stable/modules/clustering.html#silhouette-coefficient)\n",
    "* Calinski-Harabasz Index (https://scikit-learn.org/stable/modules/clustering.html#calinski-harabasz-index)\n",
    "* Davies-Bouldin Index (https://scikit-learn.org/stable/modules/clustering.html#davies-bouldin-index)\n",
    "\n",
    "In this work, in order to evaluate the quality of clustering, we will use the Silhouette coefficient metric as well as after dimensionality reduction, we will look at the data in three-dimensional space for a visual assessment of the quality of clustering. "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "900f1f40",
   "metadata": {
    "papermill": {
     "duration": 0.021263,
     "end_time": "2022-08-28T13:50:04.131480",
     "exception": false,
     "start_time": "2022-08-28T13:50:04.110217",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2f870b78",
   "metadata": {
    "papermill": {
     "duration": 0.021887,
     "end_time": "2022-08-28T13:50:04.174852",
     "exception": false,
     "start_time": "2022-08-28T13:50:04.152965",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "<div style=\"padding:25px;color:white;margin:0;font-size:250%;text-align:center;display:fill;border-radius:10px;background-color:#298f50;overflow:hidden;font-weight:500;font-family:magra\">2) Library import & Dataset </div>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "1fbdee6d",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-08-28T13:50:04.222093Z",
     "iopub.status.busy": "2022-08-28T13:50:04.220859Z",
     "iopub.status.idle": "2022-08-28T13:50:07.209795Z",
     "shell.execute_reply": "2022-08-28T13:50:07.208214Z"
    },
    "papermill": {
     "duration": 3.016161,
     "end_time": "2022-08-28T13:50:07.212927",
     "exception": false,
     "start_time": "2022-08-28T13:50:04.196766",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import pandas as pd \n",
    "import matplotlib.pyplot as plt\n",
    "import seaborn as sns\n",
    "import plotly.express as px\n",
    "import sklearn\n",
    "from sklearn.preprocessing import MinMaxScaler\n",
    "from sklearn.metrics import silhouette_score\n",
    "from sklearn.metrics import silhouette_samples\n",
    "from sklearn.decomposition import PCA\n",
    "from sklearn.cluster import KMeans, AgglomerativeClustering, SpectralClustering, DBSCAN\n",
    "from datetime import datetime\n",
    "import warnings\n",
    "import sys\n",
    "import os"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "db39198d",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-08-28T13:50:07.260113Z",
     "iopub.status.busy": "2022-08-28T13:50:07.259192Z",
     "iopub.status.idle": "2022-08-28T13:50:07.297684Z",
     "shell.execute_reply": "2022-08-28T13:50:07.296601Z"
    },
    "papermill": {
     "duration": 0.065015,
     "end_time": "2022-08-28T13:50:07.300452",
     "exception": false,
     "start_time": "2022-08-28T13:50:07.235437",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "df = pd.read_csv('../input/customer-personality-analysis/marketing_campaign.csv', sep='\\t')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "d4a47db8",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-08-28T13:50:07.347340Z",
     "iopub.status.busy": "2022-08-28T13:50:07.346437Z",
     "iopub.status.idle": "2022-08-28T13:50:07.387384Z",
     "shell.execute_reply": "2022-08-28T13:50:07.385894Z"
    },
    "papermill": {
     "duration": 0.068972,
     "end_time": "2022-08-28T13:50:07.391082",
     "exception": false,
     "start_time": "2022-08-28T13:50:07.322110",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>ID</th>\n",
       "      <th>Year_Birth</th>\n",
       "      <th>Education</th>\n",
       "      <th>Marital_Status</th>\n",
       "      <th>Income</th>\n",
       "      <th>Kidhome</th>\n",
       "      <th>Teenhome</th>\n",
       "      <th>Dt_Customer</th>\n",
       "      <th>Recency</th>\n",
       "      <th>MntWines</th>\n",
       "      <th>...</th>\n",
       "      <th>NumWebVisitsMonth</th>\n",
       "      <th>AcceptedCmp3</th>\n",
       "      <th>AcceptedCmp4</th>\n",
       "      <th>AcceptedCmp5</th>\n",
       "      <th>AcceptedCmp1</th>\n",
       "      <th>AcceptedCmp2</th>\n",
       "      <th>Complain</th>\n",
       "      <th>Z_CostContact</th>\n",
       "      <th>Z_Revenue</th>\n",
       "      <th>Response</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>5524</td>\n",
       "      <td>1957</td>\n",
       "      <td>Graduation</td>\n",
       "      <td>Single</td>\n",
       "      <td>58138.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>04-09-2012</td>\n",
       "      <td>58</td>\n",
       "      <td>635</td>\n",
       "      <td>...</td>\n",
       "      <td>7</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>11</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2174</td>\n",
       "      <td>1954</td>\n",
       "      <td>Graduation</td>\n",
       "      <td>Single</td>\n",
       "      <td>46344.0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>08-03-2014</td>\n",
       "      <td>38</td>\n",
       "      <td>11</td>\n",
       "      <td>...</td>\n",
       "      <td>5</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>11</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>4141</td>\n",
       "      <td>1965</td>\n",
       "      <td>Graduation</td>\n",
       "      <td>Together</td>\n",
       "      <td>71613.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>21-08-2013</td>\n",
       "      <td>26</td>\n",
       "      <td>426</td>\n",
       "      <td>...</td>\n",
       "      <td>4</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>11</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>6182</td>\n",
       "      <td>1984</td>\n",
       "      <td>Graduation</td>\n",
       "      <td>Together</td>\n",
       "      <td>26646.0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>10-02-2014</td>\n",
       "      <td>26</td>\n",
       "      <td>11</td>\n",
       "      <td>...</td>\n",
       "      <td>6</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>11</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>5324</td>\n",
       "      <td>1981</td>\n",
       "      <td>PhD</td>\n",
       "      <td>Married</td>\n",
       "      <td>58293.0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>19-01-2014</td>\n",
       "      <td>94</td>\n",
       "      <td>173</td>\n",
       "      <td>...</td>\n",
       "      <td>5</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>11</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>7446</td>\n",
       "      <td>1967</td>\n",
       "      <td>Master</td>\n",
       "      <td>Together</td>\n",
       "      <td>62513.0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>09-09-2013</td>\n",
       "      <td>16</td>\n",
       "      <td>520</td>\n",
       "      <td>...</td>\n",
       "      <td>6</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>11</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>965</td>\n",
       "      <td>1971</td>\n",
       "      <td>Graduation</td>\n",
       "      <td>Divorced</td>\n",
       "      <td>55635.0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>13-11-2012</td>\n",
       "      <td>34</td>\n",
       "      <td>235</td>\n",
       "      <td>...</td>\n",
       "      <td>6</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>11</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>6177</td>\n",
       "      <td>1985</td>\n",
       "      <td>PhD</td>\n",
       "      <td>Married</td>\n",
       "      <td>33454.0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>08-05-2013</td>\n",
       "      <td>32</td>\n",
       "      <td>76</td>\n",
       "      <td>...</td>\n",
       "      <td>8</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>11</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>4855</td>\n",
       "      <td>1974</td>\n",
       "      <td>PhD</td>\n",
       "      <td>Together</td>\n",
       "      <td>30351.0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>06-06-2013</td>\n",
       "      <td>19</td>\n",
       "      <td>14</td>\n",
       "      <td>...</td>\n",
       "      <td>9</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>11</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>5899</td>\n",
       "      <td>1950</td>\n",
       "      <td>PhD</td>\n",
       "      <td>Together</td>\n",
       "      <td>5648.0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>13-03-2014</td>\n",
       "      <td>68</td>\n",
       "      <td>28</td>\n",
       "      <td>...</td>\n",
       "      <td>20</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>11</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>10 rows × 29 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "     ID  Year_Birth   Education Marital_Status   Income  Kidhome  Teenhome  \\\n",
       "0  5524        1957  Graduation         Single  58138.0        0         0   \n",
       "1  2174        1954  Graduation         Single  46344.0        1         1   \n",
       "2  4141        1965  Graduation       Together  71613.0        0         0   \n",
       "3  6182        1984  Graduation       Together  26646.0        1         0   \n",
       "4  5324        1981         PhD        Married  58293.0        1         0   \n",
       "5  7446        1967      Master       Together  62513.0        0         1   \n",
       "6   965        1971  Graduation       Divorced  55635.0        0         1   \n",
       "7  6177        1985         PhD        Married  33454.0        1         0   \n",
       "8  4855        1974         PhD       Together  30351.0        1         0   \n",
       "9  5899        1950         PhD       Together   5648.0        1         1   \n",
       "\n",
       "  Dt_Customer  Recency  MntWines  ...  NumWebVisitsMonth  AcceptedCmp3  \\\n",
       "0  04-09-2012       58       635  ...                  7             0   \n",
       "1  08-03-2014       38        11  ...                  5             0   \n",
       "2  21-08-2013       26       426  ...                  4             0   \n",
       "3  10-02-2014       26        11  ...                  6             0   \n",
       "4  19-01-2014       94       173  ...                  5             0   \n",
       "5  09-09-2013       16       520  ...                  6             0   \n",
       "6  13-11-2012       34       235  ...                  6             0   \n",
       "7  08-05-2013       32        76  ...                  8             0   \n",
       "8  06-06-2013       19        14  ...                  9             0   \n",
       "9  13-03-2014       68        28  ...                 20             1   \n",
       "\n",
       "   AcceptedCmp4  AcceptedCmp5  AcceptedCmp1  AcceptedCmp2  Complain  \\\n",
       "0             0             0             0             0         0   \n",
       "1             0             0             0             0         0   \n",
       "2             0             0             0             0         0   \n",
       "3             0             0             0             0         0   \n",
       "4             0             0             0             0         0   \n",
       "5             0             0             0             0         0   \n",
       "6             0             0             0             0         0   \n",
       "7             0             0             0             0         0   \n",
       "8             0             0             0             0         0   \n",
       "9             0             0             0             0         0   \n",
       "\n",
       "   Z_CostContact  Z_Revenue  Response  \n",
       "0              3         11         1  \n",
       "1              3         11         0  \n",
       "2              3         11         0  \n",
       "3              3         11         0  \n",
       "4              3         11         0  \n",
       "5              3         11         0  \n",
       "6              3         11         0  \n",
       "7              3         11         0  \n",
       "8              3         11         1  \n",
       "9              3         11         0  \n",
       "\n",
       "[10 rows x 29 columns]"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.head(10)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c9694fd0",
   "metadata": {
    "papermill": {
     "duration": 0.021969,
     "end_time": "2022-08-28T13:50:07.434882",
     "exception": false,
     "start_time": "2022-08-28T13:50:07.412913",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "# Features:\n",
    "______________________________________________________________________________________________________________\n",
    "### People\n",
    "\n",
    "* ID: Customer's unique identifier  \n",
    "* Year_Birth: Customer's birth year  \n",
    "* Education: Customer's education level  \n",
    "* Marital_Status: Customer's marital status  \n",
    "* Income: Customer's yearly household income  \n",
    "* Kidhome: Number of children in customer's household  \n",
    "* Teenhome: Number of teenagers in customer's household  \n",
    "* Dt_Customer: Date of customer's enrollment with the company  \n",
    "* Recency: Number of days since customer's last purchase  \n",
    "* Complain: 1 if the customer complained in the last 2 years, 0 otherwise  \n",
    "\n",
    "### Products\n",
    "\n",
    "* MntWines: Amount spent on wine in last 2 years\n",
    "* MntFruits: Amount spent on fruits in last 2 years\n",
    "* MntMeatProducts: Amount spent on meat in last 2 years\n",
    "* MntFishProducts: Amount spent on fish in last 2 years\n",
    "* MntSweetProducts: Amount spent on sweets in last 2 years\n",
    "* MntGoldProds: Amount spent on gold in last 2 years\n",
    "\n",
    "### Promotion\n",
    "\n",
    "* NumDealsPurchases: Number of purchases made with a discount\n",
    "* AcceptedCmp1: 1 if customer accepted the offer in the 1st campaign, 0 otherwise\n",
    "* AcceptedCmp2: 1 if customer accepted the offer in the 2nd campaign, 0 otherwise\n",
    "* AcceptedCmp3: 1 if customer accepted the offer in the 3rd campaign, 0 otherwise\n",
    "* AcceptedCmp4: 1 if customer accepted the offer in the 4th campaign, 0 otherwise\n",
    "* AcceptedCmp5: 1 if customer accepted the offer in the 5th campaign, 0 otherwise\n",
    "* Response: 1 if customer accepted the offer in the last campaign, 0 otherwise\n",
    "\n",
    "### Place\n",
    "\n",
    "* NumWebPurchases: Number of purchases made through the company’s website\n",
    "* NumCatalogPurchases: Number of purchases made using a catalogue\n",
    "* NumStorePurchases: Number of purchases made directly in stores\n",
    "* NumWebVisitsMonth: Number of visits to company’s website in the last month\n",
    "______________________________________________________________________________________________________________\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "f0dbda9a",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-08-28T13:50:07.482194Z",
     "iopub.status.busy": "2022-08-28T13:50:07.480839Z",
     "iopub.status.idle": "2022-08-28T13:50:07.511325Z",
     "shell.execute_reply": "2022-08-28T13:50:07.509843Z"
    },
    "papermill": {
     "duration": 0.057434,
     "end_time": "2022-08-28T13:50:07.514712",
     "exception": false,
     "start_time": "2022-08-28T13:50:07.457278",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<class 'pandas.core.frame.DataFrame'>\n",
      "RangeIndex: 2240 entries, 0 to 2239\n",
      "Data columns (total 29 columns):\n",
      " #   Column               Non-Null Count  Dtype  \n",
      "---  ------               --------------  -----  \n",
      " 0   ID                   2240 non-null   int64  \n",
      " 1   Year_Birth           2240 non-null   int64  \n",
      " 2   Education            2240 non-null   object \n",
      " 3   Marital_Status       2240 non-null   object \n",
      " 4   Income               2216 non-null   float64\n",
      " 5   Kidhome              2240 non-null   int64  \n",
      " 6   Teenhome             2240 non-null   int64  \n",
      " 7   Dt_Customer          2240 non-null   object \n",
      " 8   Recency              2240 non-null   int64  \n",
      " 9   MntWines             2240 non-null   int64  \n",
      " 10  MntFruits            2240 non-null   int64  \n",
      " 11  MntMeatProducts      2240 non-null   int64  \n",
      " 12  MntFishProducts      2240 non-null   int64  \n",
      " 13  MntSweetProducts     2240 non-null   int64  \n",
      " 14  MntGoldProds         2240 non-null   int64  \n",
      " 15  NumDealsPurchases    2240 non-null   int64  \n",
      " 16  NumWebPurchases      2240 non-null   int64  \n",
      " 17  NumCatalogPurchases  2240 non-null   int64  \n",
      " 18  NumStorePurchases    2240 non-null   int64  \n",
      " 19  NumWebVisitsMonth    2240 non-null   int64  \n",
      " 20  AcceptedCmp3         2240 non-null   int64  \n",
      " 21  AcceptedCmp4         2240 non-null   int64  \n",
      " 22  AcceptedCmp5         2240 non-null   int64  \n",
      " 23  AcceptedCmp1         2240 non-null   int64  \n",
      " 24  AcceptedCmp2         2240 non-null   int64  \n",
      " 25  Complain             2240 non-null   int64  \n",
      " 26  Z_CostContact        2240 non-null   int64  \n",
      " 27  Z_Revenue            2240 non-null   int64  \n",
      " 28  Response             2240 non-null   int64  \n",
      "dtypes: float64(1), int64(25), object(3)\n",
      "memory usage: 507.6+ KB\n"
     ]
    }
   ],
   "source": [
    "df.info()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "f7586411",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-08-28T13:50:07.561859Z",
     "iopub.status.busy": "2022-08-28T13:50:07.561390Z",
     "iopub.status.idle": "2022-08-28T13:50:07.567743Z",
     "shell.execute_reply": "2022-08-28T13:50:07.566238Z"
    },
    "papermill": {
     "duration": 0.03321,
     "end_time": "2022-08-28T13:50:07.570611",
     "exception": false,
     "start_time": "2022-08-28T13:50:07.537401",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Dataset size: (2240, 29)\n"
     ]
    }
   ],
   "source": [
    "print(f'Dataset size: {df.shape}')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "f264ddf4",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-08-28T13:50:07.617293Z",
     "iopub.status.busy": "2022-08-28T13:50:07.616780Z",
     "iopub.status.idle": "2022-08-28T13:50:07.621925Z",
     "shell.execute_reply": "2022-08-28T13:50:07.620972Z"
    },
    "papermill": {
     "duration": 0.031732,
     "end_time": "2022-08-28T13:50:07.624479",
     "exception": false,
     "start_time": "2022-08-28T13:50:07.592747",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "warnings.filterwarnings('ignore')  # ignore notifications"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "d9a8233c",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-08-28T13:50:07.672093Z",
     "iopub.status.busy": "2022-08-28T13:50:07.671209Z",
     "iopub.status.idle": "2022-08-28T13:50:07.685753Z",
     "shell.execute_reply": "2022-08-28T13:50:07.684290Z"
    },
    "papermill": {
     "duration": 0.041657,
     "end_time": "2022-08-28T13:50:07.688614",
     "exception": false,
     "start_time": "2022-08-28T13:50:07.646957",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "def reduce_mem_usage(df: pd.DataFrame) -> pd.DataFrame:\n",
    "    \"\"\"Iterate through all the columns of a dataframe and modify the data type\n",
    "        to reduce memory usage\"\"\"\n",
    "    \n",
    "    start_mem = df.memory_usage().sum() / 1024**2\n",
    "    print('Memory usage of DataFrame is {:.2f} MB'.format(start_mem))\n",
    "    \n",
    "    for col in df.columns:\n",
    "        col_type = df[col].dtype\n",
    "        \n",
    "        if col_type != object:\n",
    "            c_min = df[col].min()\n",
    "            c_max = df[col].max()\n",
    "            if str(col_type)[:3] == 'int':\n",
    "                if c_min > np.iinfo(np.int8).min and c_max < np.iinfo(np.int8).max:\n",
    "                    df[col] = df[col].astype(np.int8)\n",
    "                elif c_min > np.iinfo(np.int16).min and c_max < np.iinfo(np.int16).max:\n",
    "                    df[col] = df[col].astype(np.int16)\n",
    "                elif c_min > np.iinfo(np.int32).min and c_max < np.iinfo(np.int32).max:\n",
    "                    df[col] = df[col].astype(np.int32)\n",
    "                elif c_min > np.iinfo(np.int64).min and c_max < np.iinfo(np.int64).max:\n",
    "                    df[col] = df[col].astype(np.int64)  \n",
    "            else:\n",
    "                if c_min > np.finfo(np.float32).min and c_max < np.finfo(np.float32).max:\n",
    "                    df[col] = df[col].astype(np.float32)\n",
    "                else:\n",
    "                    df[col] = df[col].astype(np.float64)\n",
    "        else:\n",
    "            df[col] = df[col].astype('category')\n",
    "\n",
    "    end_mem = df.memory_usage().sum() / 1024**2\n",
    "    print('Memory usage after optimization is: {:.2f} MB'.format(end_mem))\n",
    "    print('Decreased by {:.1f}%'.format(100 * (start_mem - end_mem) / start_mem))\n",
    "    \n",
    "    return df"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "dddb9771",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-08-28T13:50:07.736125Z",
     "iopub.status.busy": "2022-08-28T13:50:07.735629Z",
     "iopub.status.idle": "2022-08-28T13:50:07.767871Z",
     "shell.execute_reply": "2022-08-28T13:50:07.765951Z"
    },
    "papermill": {
     "duration": 0.059641,
     "end_time": "2022-08-28T13:50:07.770647",
     "exception": false,
     "start_time": "2022-08-28T13:50:07.711006",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Memory usage of DataFrame is 0.50 MB\n",
      "Memory usage after optimization is: 0.11 MB\n",
      "Decreased by 78.0%\n"
     ]
    }
   ],
   "source": [
    "df = reduce_mem_usage(df)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d8c840c5",
   "metadata": {
    "papermill": {
     "duration": 0.022941,
     "end_time": "2022-08-28T13:50:07.816120",
     "exception": false,
     "start_time": "2022-08-28T13:50:07.793179",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "<div style=\"padding:25px;color:white;margin:0;font-size:250%;text-align:center;display:fill;border-radius:10px;background-color:#298f50;overflow:hidden;font-weight:500;font-family:magra\">3.1) Feature generation  </div>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "66231ffd",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-08-28T13:50:07.866854Z",
     "iopub.status.busy": "2022-08-28T13:50:07.865910Z",
     "iopub.status.idle": "2022-08-28T13:50:07.875389Z",
     "shell.execute_reply": "2022-08-28T13:50:07.874066Z"
    },
    "papermill": {
     "duration": 0.038343,
     "end_time": "2022-08-28T13:50:07.878640",
     "exception": false,
     "start_time": "2022-08-28T13:50:07.840297",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "df['Age'] = datetime.now().year - df['Year_Birth']  # Client Age"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "6d70dd7b",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-08-28T13:50:07.927656Z",
     "iopub.status.busy": "2022-08-28T13:50:07.927187Z",
     "iopub.status.idle": "2022-08-28T13:50:07.955447Z",
     "shell.execute_reply": "2022-08-28T13:50:07.954213Z"
    },
    "papermill": {
     "duration": 0.056185,
     "end_time": "2022-08-28T13:50:07.958468",
     "exception": false,
     "start_time": "2022-08-28T13:50:07.902283",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "def get_different_year():\n",
    "    registration_year = pd.to_datetime(df['Dt_Customer'], format='%d-%m-%Y').apply(lambda x: x.year)\n",
    "    current_year = datetime.now().year\n",
    "    return current_year - registration_year\n",
    "\n",
    "df['Years_Since_Registration'] = get_different_year()  # Number of years since customer registration"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "id": "1add1b3f",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-08-28T13:50:08.005426Z",
     "iopub.status.busy": "2022-08-28T13:50:08.004545Z",
     "iopub.status.idle": "2022-08-28T13:50:08.016059Z",
     "shell.execute_reply": "2022-08-28T13:50:08.015005Z"
    },
    "papermill": {
     "duration": 0.037797,
     "end_time": "2022-08-28T13:50:08.018592",
     "exception": false,
     "start_time": "2022-08-28T13:50:07.980795",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "df[\"Education\"] = df[\"Education\"].replace({\"Basic\": 0, \"Graduation\": 1, \"2n Cycle\": 2, \"Master\": 2, \"PhD\": 3})\n",
    "# Encoded in numbers in order"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "id": "1e356ea5",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-08-28T13:50:08.066519Z",
     "iopub.status.busy": "2022-08-28T13:50:08.065713Z",
     "iopub.status.idle": "2022-08-28T13:50:08.081091Z",
     "shell.execute_reply": "2022-08-28T13:50:08.079966Z"
    },
    "papermill": {
     "duration": 0.043231,
     "end_time": "2022-08-28T13:50:08.083904",
     "exception": false,
     "start_time": "2022-08-28T13:50:08.040673",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "df['Marital_Status'] = df[\"Marital_Status\"].replace({\"Married\":\"Partner\", \"Together\":\"Partner\", \"Absurd\":\"Alone\",\n",
    "    \"Widow\":\"Alone\", \"YOLO\":\"Alone\", \"Divorced\":\"Alone\", \"Single\":\"Alone\"})\n",
    "\n",
    "step_2 = df['Marital_Status'].replace({'Alone': 1, 'Partner': 2})\n",
    "\n",
    "df[\"Family_Size\"] = step_2 + df['Kidhome'] + df['Teenhome']  # Total number of people in the family\n",
    "\n",
    "df = pd.concat([df, pd.get_dummies(df['Marital_Status'])], axis=1)  # Separate features for Alone and Partner"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "id": "ac685e10",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-08-28T13:50:08.131079Z",
     "iopub.status.busy": "2022-08-28T13:50:08.130324Z",
     "iopub.status.idle": "2022-08-28T13:50:08.139672Z",
     "shell.execute_reply": "2022-08-28T13:50:08.138374Z"
    },
    "papermill": {
     "duration": 0.036065,
     "end_time": "2022-08-28T13:50:08.142371",
     "exception": false,
     "start_time": "2022-08-28T13:50:08.106306",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "mnt_cols = ['MntWines', 'MntFruits', 'MntMeatProducts', 'MntFishProducts', 'MntSweetProducts', 'MntGoldProds']\n",
    "df['Sum_Mnt'] = df[mnt_cols].sum(axis=1)  # Total amount spent on products"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "id": "bf8e3514",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-08-28T13:50:08.190191Z",
     "iopub.status.busy": "2022-08-28T13:50:08.189415Z",
     "iopub.status.idle": "2022-08-28T13:50:08.198065Z",
     "shell.execute_reply": "2022-08-28T13:50:08.197120Z"
    },
    "papermill": {
     "duration": 0.035303,
     "end_time": "2022-08-28T13:50:08.200563",
     "exception": false,
     "start_time": "2022-08-28T13:50:08.165260",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "accepted_cmp_cols = ['AcceptedCmp1', 'AcceptedCmp2', 'AcceptedCmp3', 'AcceptedCmp4', 'AcceptedCmp5',\n",
    "                     'Response']\n",
    "df['Num_Accepted_Cmp'] = df[accepted_cmp_cols].sum(axis=1)  \n",
    "# Number of companies in which the client accepted the offer"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "id": "bdf8dac4",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-08-28T13:50:08.247275Z",
     "iopub.status.busy": "2022-08-28T13:50:08.246531Z",
     "iopub.status.idle": "2022-08-28T13:50:08.255458Z",
     "shell.execute_reply": "2022-08-28T13:50:08.254344Z"
    },
    "papermill": {
     "duration": 0.035527,
     "end_time": "2022-08-28T13:50:08.258183",
     "exception": false,
     "start_time": "2022-08-28T13:50:08.222656",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "total_purchases = ['NumWebPurchases', 'NumCatalogPurchases', 'NumStorePurchases']\n",
    "df['Num_Total_Purchases'] = df[total_purchases].sum(axis=1)  # Total number of purchases"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d8ef8896",
   "metadata": {
    "papermill": {
     "duration": 0.0219,
     "end_time": "2022-08-28T13:50:08.302246",
     "exception": false,
     "start_time": "2022-08-28T13:50:08.280346",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "# Let's look at EDA:\n",
    "______________________________________________________________________________________________________________"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "40e210cd",
   "metadata": {
    "papermill": {
     "duration": 0.021869,
     "end_time": "2022-08-28T13:50:08.346271",
     "exception": false,
     "start_time": "2022-08-28T13:50:08.324402",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "## First, let's look at the correlation matrix:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "id": "94fe9d5c",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-08-28T13:50:08.393693Z",
     "iopub.status.busy": "2022-08-28T13:50:08.392815Z",
     "iopub.status.idle": "2022-08-28T13:50:10.957802Z",
     "shell.execute_reply": "2022-08-28T13:50:10.956231Z"
    },
    "papermill": {
     "duration": 2.594148,
     "end_time": "2022-08-28T13:50:10.962906",
     "exception": false,
     "start_time": "2022-08-28T13:50:08.368758",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "image/png": "[encoded data, 218988 characters]\n",
      "text/plain": [
       "<Figure size 1800x1008 with 2 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "to_corr = ['Age', 'Income', 'Kidhome', 'Teenhome', 'Recency', 'Complain', \n",
    "          'MntWines', 'MntFruits', 'MntMeatProducts', 'MntFishProducts', 'MntSweetProducts', \n",
    "          'MntGoldProds', 'NumDealsPurchases', 'AcceptedCmp1', 'AcceptedCmp2', 'AcceptedCmp3', \n",
    "          'AcceptedCmp4', 'AcceptedCmp5', 'Response', 'NumWebPurchases', 'NumCatalogPurchases', \n",
    "          'NumStorePurchases', 'NumWebVisitsMonth', 'Years_Since_Registration', 'Family_Size',\n",
    "           'Sum_Mnt', 'Num_Accepted_Cmp', 'Num_Total_Purchases', 'Alone', 'Partner']\n",
    "\n",
    "cmap = sns.diverging_palette(220, 10, as_cmap=True)\n",
    "matrix = np.triu(df[to_corr].corr())\n",
    "plt.figure(figsize=(25, 14))\n",
    "plt.title('Correlation matrix', fontsize=18)\n",
    "sns.heatmap(df[to_corr].corr(), annot=True,\n",
    "            fmt='.1f', vmin=-0.4, center=0, cmap=cmap, mask=matrix)\n",
    "\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "90faf051",
   "metadata": {
    "papermill": {
     "duration": 0.026543,
     "end_time": "2022-08-28T13:50:11.019982",
     "exception": false,
     "start_time": "2022-08-28T13:50:10.993439",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "## Next, look at the distributions of quantitative variables:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "id": "2116d718",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-08-28T13:50:11.074918Z",
     "iopub.status.busy": "2022-08-28T13:50:11.074100Z",
     "iopub.status.idle": "2022-08-28T13:50:23.869035Z",
     "shell.execute_reply": "2022-08-28T13:50:23.867626Z"
    },
    "papermill": {
     "duration": 12.833249,
     "end_time": "2022-08-28T13:50:23.878991",
     "exception": false,
     "start_time": "2022-08-28T13:50:11.045742",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "image/png": "[encoded data, 705896 characters]\n",
      "text/plain": [
       "<Figure size 1159.38x1080 with 42 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "to_plot = ['Income', 'Recency', 'Age', 'Years_Since_Registration', 'Sum_Mnt',\n",
    "           'Num_Total_Purchases', 'Marital_Status']\n",
    "sns.pairplot(df[to_plot], hue='Marital_Status', palette='Set1')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "301760a7",
   "metadata": {
    "papermill": {
     "duration": 0.037934,
     "end_time": "2022-08-28T13:50:23.956788",
     "exception": false,
     "start_time": "2022-08-28T13:50:23.918854",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "<div style=\"padding:25px;color:white;margin:0;font-size:250%;text-align:center;display:fill;border-radius:10px;background-color:#298f50;overflow:hidden;font-weight:500;font-family:magra\">3.2) Data cleaning </div>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "becb2250",
   "metadata": {
    "papermill": {
     "duration": 0.0365,
     "end_time": "2022-08-28T13:50:24.030576",
     "exception": false,
     "start_time": "2022-08-28T13:50:23.994076",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "## Remove gaps in the data:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "id": "e85427d0",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-08-28T13:50:24.107029Z",
     "iopub.status.busy": "2022-08-28T13:50:24.106545Z",
     "iopub.status.idle": "2022-08-28T13:50:24.119539Z",
     "shell.execute_reply": "2022-08-28T13:50:24.118496Z"
    },
    "papermill": {
     "duration": 0.053881,
     "end_time": "2022-08-28T13:50:24.121797",
     "exception": false,
     "start_time": "2022-08-28T13:50:24.067916",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "ID                           0\n",
       "Year_Birth                   0\n",
       "Education                    0\n",
       "Marital_Status               0\n",
       "Income                      24\n",
       "Kidhome                      0\n",
       "Teenhome                     0\n",
       "Dt_Customer                  0\n",
       "Recency                      0\n",
       "MntWines                     0\n",
       "MntFruits                    0\n",
       "MntMeatProducts              0\n",
       "MntFishProducts              0\n",
       "MntSweetProducts             0\n",
       "MntGoldProds                 0\n",
       "NumDealsPurchases            0\n",
       "NumWebPurchases              0\n",
       "NumCatalogPurchases          0\n",
       "NumStorePurchases            0\n",
       "NumWebVisitsMonth            0\n",
       "AcceptedCmp3                 0\n",
       "AcceptedCmp4                 0\n",
       "AcceptedCmp5                 0\n",
       "AcceptedCmp1                 0\n",
       "AcceptedCmp2                 0\n",
       "Complain                     0\n",
       "Z_CostContact                0\n",
       "Z_Revenue                    0\n",
       "Response                     0\n",
       "Age                          0\n",
       "Years_Since_Registration     0\n",
       "Family_Size                  0\n",
       "Alone                        0\n",
       "Partner                      0\n",
       "Sum_Mnt                      0\n",
       "Num_Accepted_Cmp             0\n",
       "Num_Total_Purchases          0\n",
       "dtype: int64"
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.isna().sum()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "id": "5a689bb4",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-08-28T13:50:24.197184Z",
     "iopub.status.busy": "2022-08-28T13:50:24.195980Z",
     "iopub.status.idle": "2022-08-28T13:50:24.205508Z",
     "shell.execute_reply": "2022-08-28T13:50:24.204269Z"
    },
    "papermill": {
     "duration": 0.051032,
     "end_time": "2022-08-28T13:50:24.208553",
     "exception": false,
     "start_time": "2022-08-28T13:50:24.157521",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "df.dropna(inplace=True)  # Removed objects with gaps in income"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "cc804158",
   "metadata": {
    "papermill": {
     "duration": 0.036219,
     "end_time": "2022-08-28T13:50:24.280612",
     "exception": false,
     "start_time": "2022-08-28T13:50:24.244393",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "## Remove outliers in the data:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "id": "21982d02",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-08-28T13:50:24.357434Z",
     "iopub.status.busy": "2022-08-28T13:50:24.356548Z",
     "iopub.status.idle": "2022-08-28T13:50:24.370733Z",
     "shell.execute_reply": "2022-08-28T13:50:24.369666Z"
    },
    "papermill": {
     "duration": 0.05638,
     "end_time": "2022-08-28T13:50:24.373367",
     "exception": false,
     "start_time": "2022-08-28T13:50:24.316987",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "def remove_outliers(data: pd.DataFrame, column: str) -> pd.Series:\n",
    "    q3, q1 = np.nanpercentile(data[column], [75, 25])\n",
    "    iqr = q3 - q1\n",
    "    upper_bound = q3 + 1.5 * iqr\n",
    "    lower_bound = q1 - 1.5 * iqr\n",
    "    data = data[(data[column] > lower_bound) & (data[column] < upper_bound)]\n",
    "\n",
    "    return data\n",
    "\n",
    "df = remove_outliers(df, 'Age')\n",
    "df = remove_outliers(df, 'Income')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "bdddf64e",
   "metadata": {
    "papermill": {
     "duration": 0.035761,
     "end_time": "2022-08-28T13:50:24.445017",
     "exception": false,
     "start_time": "2022-08-28T13:50:24.409256",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "## Remove unnecessary features:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "id": "fe757220",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-08-28T13:50:24.518855Z",
     "iopub.status.busy": "2022-08-28T13:50:24.518398Z",
     "iopub.status.idle": "2022-08-28T13:50:24.526627Z",
     "shell.execute_reply": "2022-08-28T13:50:24.525500Z"
    },
    "papermill": {
     "duration": 0.048269,
     "end_time": "2022-08-28T13:50:24.529075",
     "exception": false,
     "start_time": "2022-08-28T13:50:24.480806",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "df.drop(['Year_Birth', 'ID', 'Z_CostContact', 'Z_Revenue', 'Dt_Customer', 'Marital_Status'], axis=1, inplace=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2a8eda0c",
   "metadata": {
    "papermill": {
     "duration": 0.03541,
     "end_time": "2022-08-28T13:50:24.600872",
     "exception": false,
     "start_time": "2022-08-28T13:50:24.565462",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "<div style=\"padding:25px;color:white;margin:0;font-size:250%;text-align:center;display:fill;border-radius:10px;background-color:#298f50;overflow:hidden;font-weight:500;font-family:magra\">3.3) Data scaling (Normalization | Standartization)  </div>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d9275da3",
   "metadata": {
    "papermill": {
     "duration": 0.035406,
     "end_time": "2022-08-28T13:50:24.672663",
     "exception": false,
     "start_time": "2022-08-28T13:50:24.637257",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "Data scaling is necessary to bring all features to the same scale. If this is not done, then the attention of the algorithm will be attracted to features that include large values (this is bad)  \n",
    "In this work, normalization will be used as data scaling (as a result of normalization, all features are in the range from 0 to 1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "id": "95d6b516",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-08-28T13:50:24.748441Z",
     "iopub.status.busy": "2022-08-28T13:50:24.747424Z",
     "iopub.status.idle": "2022-08-28T13:50:24.760373Z",
     "shell.execute_reply": "2022-08-28T13:50:24.759307Z"
    },
    "papermill": {
     "duration": 0.055109,
     "end_time": "2022-08-28T13:50:24.763522",
     "exception": false,
     "start_time": "2022-08-28T13:50:24.708413",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "def scaling_func(df: pd.DataFrame) -> None:\n",
    "    mms = MinMaxScaler()\n",
    "    return pd.DataFrame(data=mms.fit_transform(df), columns=df.columns)\n",
    "\n",
    "\n",
    "df_scaled = scaling_func(df)\n",
    "df_scaled.index = df.index  # for convenient work with dataframes"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "238ce006",
   "metadata": {
    "papermill": {
     "duration": 0.036003,
     "end_time": "2022-08-28T13:50:24.835891",
     "exception": false,
     "start_time": "2022-08-28T13:50:24.799888",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "<div style=\"padding:25px;color:white;margin:0;font-size:250%;text-align:center;display:fill;border-radius:10px;background-color:#298f50;overflow:hidden;font-weight:500;font-family:magra\">4) Dimensionality reduction  </div>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7af67158",
   "metadata": {
    "papermill": {
     "duration": 0.035997,
     "end_time": "2022-08-28T13:50:24.908448",
     "exception": false,
     "start_time": "2022-08-28T13:50:24.872451",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "The dimensionality reduction problem is used in the following situations:\n",
    "* There are a lot of features in the dataset, and we want to reduce their number, leaving as much information as possible\n",
    "* We have many features in the dataset, and we want to visualize the data (for example, in 3D space)\n",
    "* Solving the problem of multicollinearity\n",
    "\n",
    "There are many different options for dimensionality reduction, which are based on different methods, for example:\n",
    "* Principal component analysis (PCA)\n",
    "* Uniform Manifold Approximation and Projection (UMAP)\n",
    "* t-distributed Stochastic Neighbor Embedding (t-SNE)\n",
    "* Locally-Linear Embedding (LLE)\n",
    "* Multidimensional Scaling (MDS)\n",
    "\n",
    "The most commonly used dimensionality reduction algorithm is PCA, and we also use it in this work.  \n",
    "Essentially, in PCA we make a transition from one variable space to another, with the new space containing fewer variables (n_component), where the new variables are uncorrelated and are the weighted sum of the old variables.  \n",
    "As a result we get m variables: {PC1, PC2, PC3... PCm} , where PC1 will receive the most information(maximum sample variance), PC2 - less, and so on (A variable is considered informative if it has a high sample variance).  \n",
    "8 components will be used in this work."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "id": "6c8a817f",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-08-28T13:50:24.986241Z",
     "iopub.status.busy": "2022-08-28T13:50:24.985289Z",
     "iopub.status.idle": "2022-08-28T13:50:25.127909Z",
     "shell.execute_reply": "2022-08-28T13:50:25.126448Z"
    },
    "papermill": {
     "duration": 0.189403,
     "end_time": "2022-08-28T13:50:25.134659",
     "exception": false,
     "start_time": "2022-08-28T13:50:24.945256",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "def dim_reduction(data: pd.DataFrame) -> pd.DataFrame:\n",
    "    n_components = 8\n",
    "    pca = PCA(n_components=n_components, random_state=42)\n",
    "    data = pd.DataFrame(pca.fit_transform(data), columns =([f'PC{i}' for i in range(1, n_components + 1)]))\n",
    "    return data\n",
    "\n",
    "df_transform = dim_reduction(df_scaled)  # df: scaling + dimensionality reduction\n",
    "df_transform.index = df_scaled.index  # for convenient work with dataframes"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "811f7208",
   "metadata": {
    "papermill": {
     "duration": 0.036946,
     "end_time": "2022-08-28T13:50:25.291626",
     "exception": false,
     "start_time": "2022-08-28T13:50:25.254680",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "<div style=\"padding:25px;color:white;margin:0;font-size:250%;text-align:center;display:fill;border-radius:10px;background-color:#298f50;overflow:hidden;font-weight:500;font-family:magra\">5) Clustering</div>"
   ]
  },
  {
   "attachments": {
    "f9463dd0-b39a-4e48-9646-e7aa2ffd82c8.jpg": {
     "image/jpeg": "[encoded data, 325484 characters]"
    }
   },
   "cell_type": "markdown",
   "id": "e5c1260e",
   "metadata": {
    "papermill": {
     "duration": 0.036951,
     "end_time": "2022-08-28T13:50:25.365206",
     "exception": false,
     "start_time": "2022-08-28T13:50:25.328255",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "![Sxema-1-01.jpg](attachment:f9463dd0-b39a-4e48-9646-e7aa2ffd82c8.jpg)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "35bdc8bd",
   "metadata": {
    "papermill": {
     "duration": 0.03694,
     "end_time": "2022-08-28T13:50:25.439167",
     "exception": false,
     "start_time": "2022-08-28T13:50:25.402227",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "So, we have finished data preprocessing  \n",
    "In total, we have three dataframes:\n",
    "* df (with feature generation and data cleaning)  \n",
    "* df_scaled (scaled version of df)\n",
    "* df_transform (dimensionality reduction for df_scaled)\n",
    "\n",
    "## Important:\n",
    "We will apply clustering algorithms for df_transform, based on this we will get labels for all objects, then we will combine the label column for objects with df in order to get information about clusters (on graphs).\n",
    "\n",
    "In this work, the following clustering algorithms will be used: (https://scikit-learn.org/stable/modules/clustering.html)\n",
    "* K-means ++ \n",
    "* Aglomerative clustering\n",
    "* Spectral clustering\n",
    "\n",
    "\n",
    "It is noteworthy that in these algorithms it is necessary to pre-specify the number of clusters.     \n",
    "We assume that the minimum number of clusters is two, the maximum number of clusters is eight. Based on this, for different cases of the number of clusters, we will build models.    \n",
    "If the metric is maximum for models with a large number of clusters, then we will iteratively increase the maximum number of clusters and build new models.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "id": "7f355417",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-08-28T13:50:25.515926Z",
     "iopub.status.busy": "2022-08-28T13:50:25.515437Z",
     "iopub.status.idle": "2022-08-28T13:52:29.397837Z",
     "shell.execute_reply": "2022-08-28T13:52:29.396253Z"
    },
    "papermill": {
     "duration": 123.925892,
     "end_time": "2022-08-28T13:52:29.401999",
     "exception": false,
     "start_time": "2022-08-28T13:50:25.476107",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "image/png": "[encoded data, 28836 characters]\n",
      "text/plain": [
       "<Figure size 1800x360 with 2 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "****************************************************************************************************************\n"
     ]
    },
    {
     "data": {
      "image/png": "[encoded data, 37668 characters]\n",
      "text/plain": [
       "<Figure size 1800x360 with 2 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "****************************************************************************************************************\n"
     ]
    },
    {
     "data": {
      "image/png": "[encoded data, 38532 characters]\n",
      "text/plain": [
       "<Figure size 1800x360 with 2 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "****************************************************************************************************************\n"
     ]
    },
    {
     "data": {
      "image/png": "[encoded data, 39600 characters]\n",
      "text/plain": [
       "<Figure size 1800x360 with 2 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "****************************************************************************************************************\n"
     ]
    },
    {
     "data": {
      "image/png": "[encoded data, 39660 characters]\n",
      "text/plain": [
       "<Figure size 1800x360 with 2 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "****************************************************************************************************************\n"
     ]
    },
    {
     "data": {
      "image/png": "[encoded data, 39212 characters]\n",
      "text/plain": [
       "<Figure size 1800x360 with 2 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "****************************************************************************************************************\n"
     ]
    },
    {
     "data": {
      "image/png": "[encoded data, 40732 characters]\n",
      "text/plain": [
       "<Figure size 1800x360 with 2 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "****************************************************************************************************************\n"
     ]
    },
    {
     "data": {
      "image/png": "[encoded data, 52168 characters]\n",
      "text/plain": [
       "<Figure size 1800x576 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "****************************************************************************************************************\n"
     ]
    },
    {
     "data": {
      "image/png": "[encoded data, 50864 characters]\n",
      "text/plain": [
       "<Figure size 1800x576 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "****************************************************************************************************************\n"
     ]
    },
    {
     "data": {
      "image/png": "[encoded data, 40108 characters]\n",
      "text/plain": [
       "<Figure size 1800x360 with 2 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "****************************************************************************************************************\n"
     ]
    },
    {
     "data": {
      "image/png": "[encoded data, 40664 characters]\n",
      "text/plain": [
       "<Figure size 1800x360 with 2 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "****************************************************************************************************************\n"
     ]
    },
    {
     "data": {
      "image/png": "[encoded data, 41904 characters]\n",
      "text/plain": [
       "<Figure size 1800x360 with 2 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "****************************************************************************************************************\n"
     ]
    },
    {
     "data": {
      "image/png": "[encoded data, 42156 characters]\n",
      "text/plain": [
       "<Figure size 1800x360 with 2 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "****************************************************************************************************************\n"
     ]
    },
    {
     "data": {
      "image/png": "[encoded data, 43380 characters]\n",
      "text/plain": [
       "<Figure size 1800x360 with 2 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "****************************************************************************************************************\n"
     ]
    },
    {
     "data": {
      "image/png": "[encoded data, 43892 characters]\n",
      "text/plain": [
       "<Figure size 1800x360 with 2 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "****************************************************************************************************************\n"
     ]
    },
    {
     "data": {
      "image/png": "[encoded data, 44804 characters]\n",
      "text/plain": [
       "<Figure size 1800x360 with 2 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "****************************************************************************************************************\n"
     ]
    },
    {
     "data": {
      "image/png": "[encoded data, 56424 characters]\n",
      "text/plain": [
       "<Figure size 1800x576 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "****************************************************************************************************************\n"
     ]
    },
    {
     "data": {
      "image/png": "[encoded data, 38836 characters]\n",
      "text/plain": [
       "<Figure size 1800x360 with 2 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "****************************************************************************************************************\n"
     ]
    },
    {
     "data": {
      "image/png": "[encoded data, 40396 characters]\n",
      "text/plain": [
       "<Figure size 1800x360 with 2 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "****************************************************************************************************************\n"
     ]
    },
    {
     "data": {
      "image/png": "[encoded data, 40996 characters]\n",
      "text/plain": [
       "<Figure size 1800x360 with 2 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "****************************************************************************************************************\n"
     ]
    },
    {
     "data": {
      "image/png": "[encoded data, 40656 characters]\n",
      "text/plain": [
       "<Figure size 1800x360 with 2 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "****************************************************************************************************************\n"
     ]
    },
    {
     "data": {
      "image/png": "[encoded data, 41852 characters]\n",
      "text/plain": [
       "<Figure size 1800x360 with 2 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "****************************************************************************************************************\n"
     ]
    },
    {
     "data": {
      "image/png": "[encoded data, 43092 characters]\n",
      "text/plain": [
       "<Figure size 1800x360 with 2 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "****************************************************************************************************************\n"
     ]
    },
    {
     "data": {
      "image/png": "[encoded data, 43676 characters]\n",
      "text/plain": [
       "<Figure size 1800x360 with 2 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "****************************************************************************************************************\n"
     ]
    },
    {
     "data": {
      "image/png": "[encoded data, 57756 characters]\n",
      "text/plain": [
       "<Figure size 1800x576 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "****************************************************************************************************************\n"
     ]
    }
   ],
   "source": [
    "class Clustering:\n",
    "    algorithms = ['KMeans', 'AgglomerativeClustering', 'SpectralClustering']\n",
    "    min_n_clusters = 2\n",
    "    max_n_clusters = 8\n",
    "    kmeans_distortions = []  # Model distortions for different hyperparameters(n_clusters) \n",
    "    \n",
    "    @staticmethod\n",
    "    def sse_plot(): \n",
    "        \"\"\"Graphs for the SSE values (KMeans)\"\"\"\n",
    "        plt.figure(figsize=(25, 8))\n",
    "        sns.set_style(\"darkgrid\")\n",
    "        plt.title('Distortion values for different number of clusters (for Kmeans)', fontsize=14, fontweight='bold')\n",
    "        plt.xlabel('Clusters', fontsize=14, fontweight='bold')\n",
    "        plt.ylabel('SSE',fontsize=14, fontweight='bold')\n",
    "        plt.xticks(fontsize=14)\n",
    "        plt.yticks(fontsize=14)\n",
    "        plt.plot(list(range(Clustering.min_n_clusters, Clustering.max_n_clusters + 1)), Clustering.kmeans_distortions, marker='o')\n",
    "        plt.show()\n",
    "        \n",
    "    \n",
    "    @staticmethod\n",
    "    def silhouette_result(metrics_score:list, algorithm: str):\n",
    "        plt.figure(figsize=(25, 8))\n",
    "        sns.set_style(\"darkgrid\")\n",
    "        plt.title(f'Silhouette score for different number of clusters (algorithm: {algorithm})', fontsize=14, fontweight='bold')\n",
    "        plt.xlabel('Clusters', fontsize=14, fontweight='bold')\n",
    "        plt.ylabel('Silhouette', fontsize=14, fontweight='bold')\n",
    "        plt.xticks(fontsize=14)\n",
    "        plt.yticks(fontsize=14)\n",
    "        plt.plot(list(range(Clustering.min_n_clusters, Clustering.max_n_clusters + 1)), metrics_score, marker='o')\n",
    "        plt.show()\n",
    "        print('*' * 112)\n",
    "    \n",
    "          \n",
    "    def make_models(self, data_old: pd.DataFrame, data_modified: pd.DataFrame):  \n",
    "        \"\"\"Method for creating models\"\"\"\n",
    "        for algorithm in Clustering.algorithms:  \n",
    "            metrics = []  # metrics: silhouette score\n",
    "            for n_clusters in range(Clustering.min_n_clusters, Clustering.max_n_clusters + 1):  \n",
    "                if algorithm == 'AgglomerativeClustering':\n",
    "                    ac = AgglomerativeClustering(n_clusters=n_clusters,\n",
    "                                                linkage='complete')\n",
    "                    pred = ac.fit_predict(df_transform)\n",
    "                    labels = pd.DataFrame(pred, columns =['Labels'], index=df.index)\n",
    "                    metrics.append(silhouette_score(df_transform, labels))\n",
    "                    Clustering.object_plot(labels, pred, algorithm)\n",
    "                    \n",
    "                if algorithm == 'SpectralClustering':\n",
    "                    sc = SpectralClustering(n_clusters=n_clusters)\n",
    "                    pred = sc.fit_predict(df_transform)\n",
    "                    labels = pd.DataFrame(pred, columns =['Labels'], index=df.index)\n",
    "                    metrics.append(silhouette_score(df_transform, labels))\n",
    "                    Clustering.object_plot(labels, pred, algorithm)\n",
    "                                              \n",
    "                if algorithm == 'KMeans':  \n",
    "                    kmeans = KMeans(n_clusters=n_clusters, \n",
    "                                    init=\"k-means++\", \n",
    "                                    n_init=10,\n",
    "                                    max_iter=280, \n",
    "                                    random_state=42)\n",
    "                    pred = kmeans.fit_predict(df_transform) \n",
    "                    labels = pd.DataFrame(pred, columns=['Labels'], index=df.index)\n",
    "                    metrics.append(silhouette_score(df_transform, labels))\n",
    "                    Clustering.kmeans_distortions.append(kmeans.inertia_)  \n",
    "                    Clustering.object_plot(labels, pred, algorithm)\n",
    "                        \n",
    "            if algorithm == 'KMeans':\n",
    "                Clustering.sse_plot()\n",
    "                print('*' * 112)\n",
    "                    \n",
    "            Clustering.silhouette_result(metrics, algorithm)\n",
    "                                     \n",
    "                        \n",
    "    @staticmethod\n",
    "    def object_plot(labs: pd.DataFrame, predict:np.array, algorithm:str):  # здесь cтроим графики кол-ва объектов в кластере и силуэтного скора для модели \n",
    "        \"\"\"Graphs of the number of objects in a cluster and the silhouette coefficient\"\"\"\n",
    "        cluster_group = labs.groupby('Labels').size()\n",
    "        plt.figure(figsize=(25, 5))\n",
    "        plt.subplot(1, 2, 1)\n",
    "        sns.set_style(\"darkgrid\")\n",
    "        plt.title(f'Cluster size (algorithm: {algorithm})', fontsize=14, fontweight='bold')\n",
    "        sns.barplot(x=cluster_group.values, y=list(map(str, cluster_group.index)))\n",
    "        plt.xlabel('Number of objects', fontsize=14, fontweight='bold')\n",
    "        plt.ylabel('Сlusters', fontsize=14, fontweight='bold')\n",
    "        plt.xticks(fontsize=14)\n",
    "        plt.yticks(fontsize=14)\n",
    "                \n",
    "        plt.subplot(1, 2, 2)         \n",
    "        cluster_labels = np.unique(predict)\n",
    "        silhouette_vals = silhouette_samples(df_transform, predict, metric='euclidean')\n",
    "        y_ax_lower, y_ax_upper = 0, 0\n",
    "        yticks = []\n",
    "        for i, с in enumerate(cluster_labels):\n",
    "            c_silhouette_vals = silhouette_vals[predict == с]\n",
    "            c_silhouette_vals.sort()\n",
    "            y_ax_upper += len(c_silhouette_vals)\n",
    "            plt.barh(range(y_ax_lower, y_ax_upper),\n",
    "                   c_silhouette_vals,\n",
    "                   height = 1,\n",
    "                   edgecolor='none')\n",
    "\n",
    "            yticks.append((y_ax_lower + y_ax_upper) / 2)\n",
    "            y_ax_lower += len(c_silhouette_vals) \n",
    "        silhouette_avg = np.mean(silhouette_vals)\n",
    "        plt.axvline(silhouette_avg, color=\"red\", linestyle=\"--\")\n",
    "        sns.set_style(\"darkgrid\")\n",
    "        plt.title(f'Silhouette plot for different clusters (algorithm: {algorithm})', fontsize=14, fontweight='bold')   \n",
    "        val = float(''.join([i for i in str(silhouette_avg)][0:5]))\n",
    "        plt.xlabel(f'Silhouette = {val}', fontsize=14, fontweight='bold')\n",
    "        plt.ylabel('Clusters', fontsize=14, fontweight='bold')\n",
    "        plt.yticks(yticks, cluster_labels, fontsize=14)\n",
    "        plt.show()\n",
    "        print('*' * 112)\n",
    "                                                 \n",
    "\n",
    "clustering = Clustering()\n",
    "clustering.make_models(df, df_transform)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6234c083",
   "metadata": {
    "papermill": {
     "duration": 0.058793,
     "end_time": "2022-08-28T13:52:29.521183",
     "exception": false,
     "start_time": "2022-08-28T13:52:29.462390",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "# What can we say from these graphs:  \n",
    "* Good metric results for Agglomerative and Spectral Clustering algorithms  \n",
    "* Best Model: AgglomerativeClustering(n_clusters=3)."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a0db62cd",
   "metadata": {
    "papermill": {
     "duration": 0.056866,
     "end_time": "2022-08-28T13:52:29.635184",
     "exception": false,
     "start_time": "2022-08-28T13:52:29.578318",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "# Now let's build graphs of the main characteristics for features int | float for the best models (since we got object labels)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "id": "10d77a73",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-08-28T13:52:29.756771Z",
     "iopub.status.busy": "2022-08-28T13:52:29.756257Z",
     "iopub.status.idle": "2022-08-28T13:52:29.939735Z",
     "shell.execute_reply": "2022-08-28T13:52:29.938377Z"
    },
    "papermill": {
     "duration": 0.247759,
     "end_time": "2022-08-28T13:52:29.942709",
     "exception": false,
     "start_time": "2022-08-28T13:52:29.694950",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "best_model = AgglomerativeClustering(n_clusters=3)\n",
    "labels = pd.DataFrame(best_model.fit_predict(df_transform), columns=['Clusters'], index=df.index)\n",
    "df = pd.concat([df, labels], axis=1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "id": "c474828f",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-08-28T13:52:30.058532Z",
     "iopub.status.busy": "2022-08-28T13:52:30.057754Z",
     "iopub.status.idle": "2022-08-28T13:52:30.065701Z",
     "shell.execute_reply": "2022-08-28T13:52:30.064655Z"
    },
    "papermill": {
     "duration": 0.070059,
     "end_time": "2022-08-28T13:52:30.068513",
     "exception": false,
     "start_time": "2022-08-28T13:52:29.998454",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "object_cols = ['Education', 'Kidhome', 'Teenhome', 'Family_Size', 'Alone', 'Partner']\n",
    "num_cols = ['Income', 'Recency', 'MntWines',\n",
    "            'MntFruits', 'MntMeatProducts', 'MntFishProducts',\n",
    "            'MntSweetProducts', 'MntGoldProds', 'NumDealsPurchases',\n",
    "            'NumWebPurchases', 'NumCatalogPurchases','NumStorePurchases',\n",
    "            'NumWebVisitsMonth', 'Age', 'Years_Since_Registration',\n",
    "            'Sum_Mnt', 'Num_Accepted_Cmp', 'Num_Total_Purchases']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "id": "4e349f89",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-08-28T13:52:30.193718Z",
     "iopub.status.busy": "2022-08-28T13:52:30.193231Z",
     "iopub.status.idle": "2022-08-28T13:52:44.046400Z",
     "shell.execute_reply": "2022-08-28T13:52:44.044829Z"
    },
    "papermill": {
     "duration": 13.91993,
     "end_time": "2022-08-28T13:52:44.049878",
     "exception": false,
     "start_time": "2022-08-28T13:52:30.129948",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "image/png": "[encoded data, 196428 characters]\n",
      "text/plain": [
       "<Figure size 1800x2160 with 18 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "[encoded data, 410292 characters]\n",
      "text/plain": [
       "<Figure size 1800x2160 with 18 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "[encoded data, 61260 characters]\n",
      "text/plain": [
       "<Figure size 1800x720 with 6 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "def graph_builder():\n",
    "    \"\"\"Graphs of the main characteristics for features int | float for the best models.\n",
    "    The best model was chosen based on the results of the graphs\"\"\"\n",
    "    def boxplot():\n",
    "        n_cols = len(num_cols)\n",
    "        rows = n_cols // 3 \n",
    "        _, axes = plt.subplots(ncols=3, nrows=rows, figsize=(25, 5 * rows))\n",
    "        plt.subplots_adjust(wspace=0.3, hspace=0.4)\n",
    "        for num, col in enumerate(num_cols):\n",
    "            b = sns.boxplot(\n",
    "                 data=df,\n",
    "                 y=col,\n",
    "                 x=\"Clusters\",\n",
    "                 palette=\"Paired\",\n",
    "                 ax=axes.reshape(-1)[num])\n",
    "            b.set_xlabel('Clusters', fontsize=14)\n",
    "            b.set_ylabel(f'{df[col].name}', fontsize=14)\n",
    "            sns.set_style('darkgrid')\n",
    "    \n",
    "    def histplot():    \n",
    "        n_cols = len(num_cols)\n",
    "        rows = n_cols // 3\n",
    "        _, axes = plt.subplots(ncols=3, nrows=rows, figsize=(25, 5 * rows))\n",
    "        plt.subplots_adjust(wspace=0.3, hspace=0.4)\n",
    "        for num, col in enumerate(num_cols):\n",
    "            h = sns.histplot(\n",
    "                 data=df,\n",
    "                 x=col,\n",
    "                 hue=\"Clusters\",\n",
    "                 multiple=\"stack\",\n",
    "                 kde=True,\n",
    "                 palette=\"Paired\",\n",
    "                 ax=axes.reshape(-1)[num])\n",
    "                \n",
    "            h.set_xlabel(f'{df[col].name}', fontsize=14)\n",
    "            h.set_ylabel('Count', fontsize=14)\n",
    "            sns.set_style('darkgrid')\n",
    "            \n",
    "    def barplot():\n",
    "        n_cols = len(object_cols)\n",
    "        rows = n_cols // 3 + n_cols % 3\n",
    "        size_cluster = df.groupby(\"Clusters\").count().iloc[:, 0]\n",
    "        _, axes = plt.subplots(ncols=3, nrows=rows, figsize=(25, 5 * rows))\n",
    "        plt.subplots_adjust(wspace=0.3, hspace=0.4)    \n",
    "        for num, col in enumerate(object_cols):\n",
    "            data = (df.groupby([\"Clusters\"])[col].value_counts().unstack(fill_value=0).T)\n",
    "            data = pd.DataFrame(data.unstack())\n",
    "            data.columns = [\"Number of objects\"]\n",
    "            data = data.reset_index()\n",
    "            b= sns.barplot(\n",
    "                 data=data,\n",
    "                 x=data[col],\n",
    "                 hue=data.Clusters,\n",
    "                 y=data['Number of objects'],\n",
    "                 ax=axes.reshape(-1)[num])\n",
    "            b.set_xlabel(f'{data[col].name}', fontsize=14)\n",
    "            b.set_ylabel('Number of objects', fontsize=14)\n",
    "            sns.set_style('darkgrid')\n",
    "    \n",
    "    boxplot()\n",
    "    histplot()\n",
    "    barplot()\n",
    "    \n",
    "graph_builder()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "bae4e70e",
   "metadata": {
    "papermill": {
     "duration": 0.068264,
     "end_time": "2022-08-28T13:52:44.191676",
     "exception": false,
     "start_time": "2022-08-28T13:52:44.123412",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "<div style=\"padding:25px;color:white;margin:0;font-size:250%;text-align:center;display:fill;border-radius:10px;background-color:#298f50;overflow:hidden;font-weight:500;font-family:magra\">6) Business Recommendations</div>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3804fd18",
   "metadata": {
    "papermill": {
     "duration": 0.066646,
     "end_time": "2022-08-28T13:52:44.325998",
     "exception": false,
     "start_time": "2022-08-28T13:52:44.259352",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "# As a result of cluster analysis, we received three groups of buyers (clusters):\n",
    "# 0 cluster:\n",
    "* Middle income people (average income equals 50000)\n",
    "* Average age is 52 years\n",
    "* Have an education (Graduation, 2n Cycle, Master, PhD)\n",
    "* People without family, people with families with and without children\n",
    "* Quite often buy wines, but they also often buy meat\n",
    "* Most often make purchases on the web\n",
    "* The average number of purchases is 13\n",
    "\n",
    "# 1 cluster:\n",
    "* High income people (average income equals 70000)\n",
    "* Average age is 55 years\n",
    "* Have an education (Graduation, 2n Cycle, Master, PhD)\n",
    "* Have a family with children (Teenhome)\n",
    "* Quite often buy wines, but they also often buy meat\n",
    "* Most often make purchases in the stores themselves\n",
    "* Most often make purchases (compared to other clusters)\n",
    "\n",
    "# 2 cluster:\n",
    "* Average income equals 38000 \n",
    "* Average age is 49 years\n",
    "* Have an education (Graduation, 2n Cycle, Master, PhD)\n",
    "* People with families with and without children\n",
    "* A low number of purchases and, accordingly, spend little money on purchases "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a50e0f32",
   "metadata": {
    "papermill": {
     "duration": 0.066087,
     "end_time": "2022-08-28T13:52:44.458496",
     "exception": false,
     "start_time": "2022-08-28T13:52:44.392409",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "<div style=\"padding:25px;color:white;margin:0;font-size:250%;text-align:center;display:fill;border-radius:10px;background-color:#298f50;overflow:hidden;font-weight:500;font-family:magra\">END</div>"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.12"
  },
  "papermill": {
   "default_parameters": {},
   "duration": 172.940742,
   "end_time": "2022-08-28T13:52:46.152462",
   "environment_variables": {},
   "exception": null,
   "input_path": "__notebook__.ipynb",
   "output_path": "__notebook__.ipynb",
   "parameters": {},
   "start_time": "2022-08-28T13:49:53.211720",
   "version": "2.3.4"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}
