{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "vsX0L1sG1iZj"
   },
   "source": [
    "# Capstone Project\n",
    "## Neural translation model\n",
    "### Instructions\n",
    "\n",
    "In this notebook, you will create a neural network that translates from English to German. You will use concepts from throughout this course, including building more flexible model architectures, freezing layers, data processing pipeline and sequence modelling.\n",
    "\n",
    "This project is peer-assessed. Within this notebook you will find instructions in each section for how to complete the project. Pay close attention to the instructions as the peer review will be carried out according to a grading rubric that checks key parts of the project instructions. Feel free to add extra cells into the notebook as required.\n",
    "\n",
    "### How to submit\n",
    "\n",
    "When you have completed the Capstone project notebook, you will submit a pdf of the notebook for peer review. First ensure that the notebook has been fully executed from beginning to end, and all of the cell outputs are visible. This is important, as the grading rubric depends on the reviewer being able to view the outputs of your notebook. Save the notebook as a pdf (you could download the notebook with File -> Download .ipynb, open the notebook locally, and then File -> Download as -> PDF via LaTeX), and then submit this pdf for review.\n",
    "\n",
    "### Let's get started!\n",
    "\n",
    "We'll start by running some imports, and loading the dataset. For this project you are free to make further imports throughout the notebook as you wish. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "2VyTvxPN1iZn"
   },
   "outputs": [],
   "source": [
    "import tensorflow as tf\n",
    "import tensorflow_hub as hub\n",
    "import unicodedata\n",
    "import re\n",
    "from tensorflow.keras.preprocessing.text import Tokenizer\n",
    "import numpy as np\n",
    "from tensorflow.keras.preprocessing.sequence import pad_sequences\n",
    "from IPython.display import Image\n",
    "from sklearn.model_selection import train_test_split\n",
    "from tensorflow.keras.layers import Layer,Input,Masking,LSTM,Embedding,Dense\n",
    "from tensorflow.keras import Model\n",
    "import time\n",
    "from tqdm import tqdm_notebook as tqdm\n",
    "import warnings\n",
    "warnings.simplefilter(\"ignore\")\n",
    "from prettytable import PrettyTable\n",
    "import matplotlib.pyplot as plt\n",
    "%matplotlib inline"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "wuIi4lAR1iZt"
   },
   "source": [
    "For the capstone project, you will use a language dataset from http://www.manythings.org/anki/ to build a neural translation model. This dataset consists of over 200,000 pairs of sentences in English and German. In order to make the training quicker, we will restrict to our dataset to 20,000 pairs. Feel free to change this if you wish - the size of the dataset used is not part of the grading rubric.\n",
    "\n",
    "Your goal is to develop a neural translation model from English to German, making use of a pre-trained English word embedding module."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "pi9Dq6vv3FVO"
   },
   "source": [
    "#### Import the data\n",
    "\n",
    "The dataset is available for download as a zip file at the following link:\n",
    "\n",
    "https://drive.google.com/open?id=1KczOciG7sYY7SB9UlBeRP1T9659b121Q\n",
    "\n",
    "You should store the unzipped folder in Drive for use in this Colab notebook."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 35
    },
    "colab_type": "code",
    "id": "Nw99tEEQ3bKL",
    "outputId": "8032cf5c-0d46-4cbc-e357-0a6f2a259c85"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Drive already mounted at /content/gdrive; to attempt to forcibly remount, call drive.mount(\"/content/gdrive\", force_remount=True).\n"
     ]
    }
   ],
   "source": [
    "# Run this cell to connect to your Drive folder\n",
    "\n",
    "from google.colab import drive\n",
    "drive.mount('/content/gdrive')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "o8PetPpw1iZu"
   },
   "outputs": [],
   "source": [
    "# Run this cell to load the dataset\n",
    "\n",
    "NUM_EXAMPLES = 20000\n",
    "data_examples = []\n",
    "with open('/content/gdrive/My Drive/deu.txt', 'r', encoding='utf8') as f:\n",
    "    for line in f.readlines():\n",
    "        if len(data_examples) < NUM_EXAMPLES:\n",
    "            data_examples.append(line)\n",
    "        else:\n",
    "            break"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "JumLjJ631iZy"
   },
   "outputs": [],
   "source": [
    "# These functions preprocess English and German sentences\n",
    "\n",
    "def unicode_to_ascii(s):\n",
    "    return ''.join(c for c in unicodedata.normalize('NFD', s) if unicodedata.category(c) != 'Mn')\n",
    "\n",
    "def preprocess_sentence(sentence):\n",
    "    sentence = sentence.lower().strip()\n",
    "    sentence = re.sub(r\"ü\", 'ue', sentence)\n",
    "    sentence = re.sub(r\"ä\", 'ae', sentence)\n",
    "    sentence = re.sub(r\"ö\", 'oe', sentence)\n",
    "    sentence = re.sub(r'ß', 'ss', sentence)\n",
    "    \n",
    "    sentence = unicode_to_ascii(sentence)\n",
    "    sentence = re.sub(r\"([?.!,])\", r\" \\1 \", sentence)\n",
    "    sentence = re.sub(r\"[^a-z?.!,']+\", \" \", sentence)\n",
    "    sentence = re.sub(r'[\" \"]+', \" \", sentence)\n",
    "    \n",
    "    return sentence.strip()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "XFJap-TW1iZ2"
   },
   "source": [
    "#### The custom translation model\n",
    "The following is a schematic of the custom translation model architecture you will develop in this project."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 648
    },
    "colab_type": "code",
    "id": "gBF1K2JN4RFJ",
    "outputId": "966a5049-6036-4a25-bc16-7ed1d61e845f"
   },
   "outputs": [
    {
     "data": {
      "image/png": "[encoded data, 48576 characters]\n",
      "text/plain": [
       "<IPython.core.display.Image object>"
      ]
     },
     "execution_count": 6,
     "metadata": {
      "tags": []
     },
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Run this cell to download and view a schematic diagram for the neural translation model\n",
    "\n",
    "!wget -q -O neural_translation_model.png --no-check-certificate \"https://docs.google.com/uc?export=download&id=1XsS1VlXoaEo-RbYNilJ9jcscNZvsSPmd\"\n",
    "Image(\"neural_translation_model.png\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "0fP7P-yK4RS7"
   },
   "source": [
    "The custom model consists of an encoder RNN and a decoder RNN. The encoder takes words of an English sentence as input, and uses a pre-trained word embedding to embed the words into a 128-dimensional space. To indicate the end of the input sentence, a special end token (in the same 128-dimensional space) is passed in as an input. This token is a TensorFlow Variable that is learned in the training phase (unlike the pre-trained word embedding, which is frozen).\n",
    "\n",
    "The decoder RNN takes the internal state of the encoder network as its initial state. A start token is passed in as the first input, which is embedded using a learned German word embedding. The decoder RNN then makes a prediction for the next German word, which during inference is then passed in as the following input, and this process is repeated until the special `<end>` token is emitted from the decoder."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "z70nu6_01iZ3"
   },
   "source": [
    "## 1. Text preprocessing\n",
    "* Create separate lists of English and German sentences, and preprocess them using the `preprocess_sentence` function provided for you above.\n",
    "* Add a special `\"<start>\"` and `\"<end>\"` token to the beginning and end of every German sentence.\n",
    "* Use the Tokenizer class from the `tf.keras.preprocessing.text` module to tokenize the German sentences, ensuring that no character filters are applied. _Hint: use the Tokenizer's \"filter\" keyword argument._\n",
    "* Print out at least 5 randomly chosen examples of (preprocessed) English and German sentence pairs. For the German sentence, print out the text (with start and end tokens) as well as the tokenized sequence.\n",
    "* Pad the end of the tokenized German sequences with zeros, and batch the complete set of sequences into one numpy array."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "9G20C4bk1iZ4",
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "english_sent = [sentence.split('\\t')[0] for sentence in data_examples]\n",
    "german_sent = [sentence.split('\\t')[1] for sentence in data_examples]\n",
    "processed_english = []\n",
    "processed_german = []\n",
    "for sentence in english_sent:\n",
    "    processed_english.append(preprocess_sentence(sentence))\n",
    "for sentence in german_sent:\n",
    "    processed_german.append(preprocess_sentence(sentence))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "Sxwl-1rB1iZ8"
   },
   "outputs": [],
   "source": [
    "p_german_1 = [\"<start> \"+ sentence + \" <end>\" for sentence in processed_german]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "WnIdqZFk1iaA"
   },
   "outputs": [],
   "source": [
    "tokenizer = Tokenizer(num_words=None,filters = '',lower=False,char_level=False)\n",
    "\n",
    "tokenizer.fit_on_texts(p_german_1)\n",
    "tokenizer_seq = tokenizer.texts_to_sequences(p_german_1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 381
    },
    "colab_type": "code",
    "id": "5UlnBdIK1iaE",
    "outputId": "3668d45e-80d9-449f-f3e9-6f56d4cfdc85"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "German Sentences :\n",
      "<start> er ist noch in den anfangssemestern . <end>\n",
      "<start> du fehlst mir auch . <end>\n",
      "<start> halte mich auf dem laufenden . <end>\n",
      "<start> ist sie verheiratet ? <end>\n",
      "<start> tom ist betruegerisch . <end>\n",
      "\n",
      "Token sequences :\n",
      "[1, 14, 6, 72, 46, 53, 5563, 3, 2]\n",
      "[1, 13, 1104, 21, 112, 3, 2]\n",
      "[1, 288, 22, 29, 118, 1408, 3, 2]\n",
      "[1, 6, 8, 703, 7, 2]\n",
      "[1, 5, 6, 5206, 3, 2]\n",
      "\n",
      "English Sentences :\n",
      "he's an undergrad .\n",
      "i miss you , too .\n",
      "keep me posted .\n",
      "is she married ?\n",
      "tom is deceitful .\n"
     ]
    }
   ],
   "source": [
    "num_of_sentences = len(p_german_1)\n",
    "\n",
    "random_ind = np.random.choice(num_of_sentences,5)\n",
    "print('German Sentences :')\n",
    "for ind in random_ind:\n",
    "    print(p_german_1[ind])\n",
    "print()\n",
    "print('Token sequences :')\n",
    "for ind in random_ind:\n",
    "    print(tokenizer_seq[ind])\n",
    "print()\n",
    "print('English Sentences :')\n",
    "for ind in random_ind:\n",
    "    print(processed_english[ind])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "ZGTaqt1t1iaI"
   },
   "outputs": [],
   "source": [
    "padded_seq = pad_sequences(tokenizer_seq,maxlen = None,padding = \"post\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "foL7Ihs21iaP"
   },
   "source": [
    "## 2. Prepare the data"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "-9rCEE4z1iaQ"
   },
   "source": [
    "#### Load the embedding layer\n",
    "As part of the dataset preproceessing for this project, you will use a pre-trained English word embedding module from TensorFlow Hub. The URL for the module is https://tfhub.dev/google/tf2-preview/nnlm-en-dim128-with-normalization/1.\n",
    "\n",
    "This embedding takes a batch of text tokens in a 1-D tensor of strings as input. It then embeds the separate tokens into a 128-dimensional space. \n",
    "\n",
    "The code to load and test the embedding layer is provided for you below.\n",
    "\n",
    "**NB:** this model can also be used as a sentence embedding module. The module will process each token by removing punctuation and splitting on spaces. It then averages the word embeddings over a sentence to give a single embedding vector. However, we will use it only as a word embedding module, and will pass each word in the input sentence as a separate token."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "ywZgobCh1iaR"
   },
   "outputs": [],
   "source": [
    "# Load embedding module from Tensorflow Hub\n",
    "\n",
    "embedding_layer = hub.KerasLayer(\"https://tfhub.dev/google/tf2-preview/nnlm-en-dim128/1\", \n",
    "                                 output_shape=[128], input_shape=[], dtype=tf.string)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 35
    },
    "colab_type": "code",
    "id": "TiY8QEDp1iaV",
    "outputId": "ec5b02c9-833b-494d-c6d4-852456d40c24"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "TensorShape([7, 128])"
      ]
     },
     "execution_count": 13,
     "metadata": {
      "tags": []
     },
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Test the layer\n",
    "\n",
    "embedding_layer(tf.constant([\"these\", \"aren't\", \"the\", \"droids\", \"you're\", \"looking\", \"for\"])).shape"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "KjuzXc-z1iaY"
   },
   "source": [
    "You should now prepare the training and validation Datasets.\n",
    "\n",
    "* Create a random training and validation set split of the data, reserving e.g. 20% of the data for validation (NB: each English dataset example is a single sentence string, and each German dataset example is a sequence of padded integer tokens).\n",
    "* Load the training and validation sets into a tf.data.Dataset object, passing in a tuple of English and German data for both training and validation sets.\n",
    "* Create a function to map over the datasets that splits each English sentence at spaces. Apply this function to both Dataset objects using the map method. _Hint: look at the tf.strings.split function._\n",
    "* Create a function to map over the datasets that embeds each sequence of English words using the loaded embedding layer/model. Apply this function to both Dataset objects using the map method.\n",
    "* Create a function to filter out dataset examples where the English sentence is greater than or equal to than 13 (embedded) tokens in length. Apply this function to both Dataset objects using the filter method.\n",
    "* Create a function to map over the datasets that pads each English sequence of embeddings with some distinct padding value before the sequence, so that each sequence is length 13. Apply this function to both Dataset objects using the map method. _Hint: look at the tf.pad function. You can extract a Tensor shape using tf.shape; you might also find the tf.math.maximum function useful._\n",
    "* Batch both training and validation Datasets with a batch size of 16.\n",
    "* Print the `element_spec` property for the training and validation Datasets. \n",
    "* Using the Dataset `.take(1)` method, print the shape of the English data example from the training Dataset.\n",
    "* Using the Dataset `.take(1)` method, print the German data example Tensor from the validation Dataset."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "Q-BUJOl_1iaZ"
   },
   "outputs": [],
   "source": [
    "x_train,x_valid,y_train,y_valid = train_test_split(processed_english,padded_seq,test_size = 0.20)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "4w6rM8Bl1iad"
   },
   "outputs": [],
   "source": [
    "train_dataset = tf.data.Dataset.from_tensor_slices((x_train,y_train))\n",
    "valid_dataset = tf.data.Dataset.from_tensor_slices((x_valid,y_valid))\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "D7bn3mRs1iaj"
   },
   "outputs": [],
   "source": [
    "def spliter(english,german):\n",
    " \n",
    "  english = tf.strings.split(english,sep = \" \")\n",
    "\n",
    "  return english,german\n",
    "\n",
    "train_dataset = train_dataset.map(spliter)\n",
    "valid_dataset = valid_dataset.map(spliter)\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "q0Fdso381ian"
   },
   "outputs": [],
   "source": [
    "\n",
    "def embedder(english,german):\n",
    "\n",
    "  english = embedding_layer(english)\n",
    "  return english,german\n",
    "\n",
    "train_dataset = train_dataset.map(embedder)\n",
    "valid_dataset = valid_dataset.map(embedder)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "oWS26VBJ-SCZ"
   },
   "outputs": [],
   "source": [
    "def lengther(english,german):\n",
    "  length = tf.constant(13,dtype = tf.int32)\n",
    "\n",
    "  return tf.math.greater_equal(length,tf.cast(len(english),tf.int32))\n",
    "\n",
    "train_dataset = train_dataset.filter(lengther)\n",
    "valid_dataset = valid_dataset.filter(lengther)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "MdqSXEoX1iav"
   },
   "outputs": [],
   "source": [
    "def padder(english,german):\n",
    "\n",
    "  paddings = [[13-len(english),0],[0,0]]\n",
    "  english = tf.pad(english, paddings = paddings)\n",
    "\n",
    "  return english,german\n",
    "\n",
    "train_dataset = train_dataset.map(padder)\n",
    "valid_dataset = valid_dataset.map(padder) "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "QZCCfB8fWaLF"
   },
   "outputs": [],
   "source": [
    "train_dataset = train_dataset.batch(16,drop_remainder= True)\n",
    "valid_dataset = valid_dataset.batch(16,drop_remainder= True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 90
    },
    "colab_type": "code",
    "id": "ackk7HptX8cX",
    "outputId": "456094d4-0398-480d-8813-b0ec2da388ff"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Training Dataset: \n",
      "(TensorSpec(shape=(16, None, 128), dtype=tf.float32, name=None), TensorSpec(shape=(16, 14), dtype=tf.int32, name=None))\n",
      "Validation Dataset: \n",
      "(TensorSpec(shape=(16, None, 128), dtype=tf.float32, name=None), TensorSpec(shape=(16, 14), dtype=tf.int32, name=None))\n"
     ]
    }
   ],
   "source": [
    "print(\"Training Dataset: \")\n",
    "print(train_dataset.element_spec)\n",
    "print(\"Validation Dataset: \")\n",
    "print(valid_dataset.element_spec)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 1890
    },
    "colab_type": "code",
    "id": "jtozc65MECNw",
    "outputId": "07102487-1efe-43d7-d7fe-333b3ca5849c"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tf.Tensor(\n",
      "[[[ 0.          0.          0.         ...  0.          0.\n",
      "    0.        ]\n",
      "  [ 0.          0.          0.         ...  0.          0.\n",
      "    0.        ]\n",
      "  [ 0.          0.          0.         ...  0.          0.\n",
      "    0.        ]\n",
      "  ...\n",
      "  [-0.03925273  0.02352522  0.02837687 ... -0.09089245 -0.02715905\n",
      "    0.05939376]\n",
      "  [-0.02130977 -0.07366709  0.10384148 ... -0.22099297  0.07892267\n",
      "   -0.01285619]\n",
      "  [ 0.012986    0.08981702  0.16017003 ...  0.06796802  0.13528903\n",
      "   -0.022035  ]]\n",
      "\n",
      " [[ 0.          0.          0.         ...  0.          0.\n",
      "    0.        ]\n",
      "  [ 0.          0.          0.         ...  0.          0.\n",
      "    0.        ]\n",
      "  [ 0.          0.          0.         ...  0.          0.\n",
      "    0.        ]\n",
      "  ...\n",
      "  [ 0.12846488  0.07159402  0.09918732 ... -0.07272145  0.03883429\n",
      "    0.04847484]\n",
      "  [-0.03209486 -0.04160203 -0.12152159 ... -0.12753211 -0.02127644\n",
      "   -0.17632811]\n",
      "  [ 0.012986    0.08981702  0.16017003 ...  0.06796802  0.13528903\n",
      "   -0.022035  ]]\n",
      "\n",
      " [[ 0.          0.          0.         ...  0.          0.\n",
      "    0.        ]\n",
      "  [ 0.          0.          0.         ...  0.          0.\n",
      "    0.        ]\n",
      "  [ 0.          0.          0.         ...  0.          0.\n",
      "    0.        ]\n",
      "  ...\n",
      "  [ 0.09539654  0.04058193  0.04366608 ... -0.03468065  0.05563864\n",
      "    0.00659631]\n",
      "  [ 0.02027391  0.04513401 -0.15236828 ... -0.10530912  0.10933939\n",
      "   -0.0068233 ]\n",
      "  [ 0.012986    0.08981702  0.16017003 ...  0.06796802  0.13528903\n",
      "   -0.022035  ]]\n",
      "\n",
      " ...\n",
      "\n",
      " [[ 0.          0.          0.         ...  0.          0.\n",
      "    0.        ]\n",
      "  [ 0.          0.          0.         ...  0.          0.\n",
      "    0.        ]\n",
      "  [ 0.          0.          0.         ...  0.          0.\n",
      "    0.        ]\n",
      "  ...\n",
      "  [ 0.1278139  -0.03678622  0.06628644 ... -0.03481541 -0.00253005\n",
      "   -0.06914762]\n",
      "  [-0.02380057  0.12938826 -0.12002522 ... -0.0778875   0.01935136\n",
      "   -0.01938629]\n",
      "  [ 0.012986    0.08981702  0.16017003 ...  0.06796802  0.13528903\n",
      "   -0.022035  ]]\n",
      "\n",
      " [[ 0.          0.          0.         ...  0.          0.\n",
      "    0.        ]\n",
      "  [ 0.          0.          0.         ...  0.          0.\n",
      "    0.        ]\n",
      "  [ 0.          0.          0.         ...  0.          0.\n",
      "    0.        ]\n",
      "  ...\n",
      "  [ 0.2000517   0.0272701  -0.03822284 ...  0.1073221  -0.01488957\n",
      "   -0.01846376]\n",
      "  [ 0.03175933  0.06343     0.05124436 ... -0.16108854  0.23792052\n",
      "    0.04684178]\n",
      "  [ 0.012986    0.08981702  0.16017003 ...  0.06796802  0.13528903\n",
      "   -0.022035  ]]\n",
      "\n",
      " [[ 0.          0.          0.         ...  0.          0.\n",
      "    0.        ]\n",
      "  [ 0.          0.          0.         ...  0.          0.\n",
      "    0.        ]\n",
      "  [ 0.          0.          0.         ...  0.          0.\n",
      "    0.        ]\n",
      "  ...\n",
      "  [ 0.02199916  0.08012285 -0.10019045 ... -0.12652187 -0.05457912\n",
      "    0.14176568]\n",
      "  [ 0.14766233 -0.06297084  0.07939632 ... -0.02586731  0.17180535\n",
      "   -0.03599825]\n",
      "  [ 0.012986    0.08981702  0.16017003 ...  0.06796802  0.13528903\n",
      "   -0.022035  ]]], shape=(16, 13, 128), dtype=float32)\n",
      "tf.Tensor(\n",
      "[[   1    5    6  447    3    2    0    0    0    0    0    0    0    0]\n",
      " [   1    4  456  176  254  125    3    2    0    0    0    0    0    0]\n",
      " [   1 3458  126    3    2    0    0    0    0    0    0    0    0    0]\n",
      " [   1    4   18  318 2558    3    2    0    0    0    0    0    0    0]\n",
      " [   1    5  686   34  205    3    2    0    0    0    0    0    0    0]\n",
      " [   1   17  522   71  330    3    2    0    0    0    0    0    0    0]\n",
      " [   1 2190    8   22   12    3    2    0    0    0    0    0    0    0]\n",
      " [   1    5    6   50  781    3    2    0    0    0    0    0    0    0]\n",
      " [   1    6   33  366    7    2    0    0    0    0    0    0    0    0]\n",
      " [   1    4   24   49    3    2    0    0    0    0    0    0    0    0]\n",
      " [   1    4  110   19 1596    3    2    0    0    0    0    0    0    0]\n",
      " [   1   14  998   21   20    3    2    0    0    0    0    0    0    0]\n",
      " [   1   26  851   23 1269    3    2    0    0    0    0    0    0    0]\n",
      " [   1   17  381   80    3    2    0    0    0    0    0    0    0    0]\n",
      " [   1    4  940  115    3    2    0    0    0    0    0    0    0    0]\n",
      " [   1   11   97  249   41    3    2    0    0    0    0    0    0    0]], shape=(16, 14), dtype=int32)\n"
     ]
    }
   ],
   "source": [
    "for english,german in train_dataset.take(1):\n",
    "  print(english)\n",
    "  print(german)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 1890
    },
    "colab_type": "code",
    "id": "mYhlOwDTEWj5",
    "outputId": "9634d9a1-79d2-4210-af1d-6d12d14523f0"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tf.Tensor(\n",
      "[[[ 0.          0.          0.         ...  0.          0.\n",
      "    0.        ]\n",
      "  [ 0.          0.          0.         ...  0.          0.\n",
      "    0.        ]\n",
      "  [ 0.          0.          0.         ...  0.          0.\n",
      "    0.        ]\n",
      "  ...\n",
      "  [ 0.03970453 -0.04158472  0.10027828 ... -0.0063081  -0.0119952\n",
      "   -0.04243935]\n",
      "  [ 0.03519357  0.01258469 -0.04993629 ... -0.08874417  0.1515415\n",
      "   -0.00328062]\n",
      "  [ 0.012986    0.08981702  0.16017003 ...  0.06796802  0.13528903\n",
      "   -0.022035  ]]\n",
      "\n",
      " [[ 0.          0.          0.         ...  0.          0.\n",
      "    0.        ]\n",
      "  [ 0.          0.          0.         ...  0.          0.\n",
      "    0.        ]\n",
      "  [ 0.          0.          0.         ...  0.          0.\n",
      "    0.        ]\n",
      "  ...\n",
      "  [ 0.04712053  0.00124522  0.03287347 ... -0.07834134  0.06874365\n",
      "   -0.08857308]\n",
      "  [ 0.09144389 -0.10057256 -0.07086372 ... -0.09084993  0.01224649\n",
      "   -0.03558629]\n",
      "  [ 0.012986    0.08981702  0.16017003 ...  0.06796802  0.13528903\n",
      "   -0.022035  ]]\n",
      "\n",
      " [[ 0.          0.          0.         ...  0.          0.\n",
      "    0.        ]\n",
      "  [ 0.          0.          0.         ...  0.          0.\n",
      "    0.        ]\n",
      "  [ 0.          0.          0.         ...  0.          0.\n",
      "    0.        ]\n",
      "  ...\n",
      "  [-0.17962365  0.04181888 -0.00170533 ...  0.18615879  0.06329069\n",
      "    0.06792238]\n",
      "  [-0.02130977 -0.07366709  0.10384148 ... -0.22099297  0.07892267\n",
      "   -0.01285619]\n",
      "  [ 0.012986    0.08981702  0.16017003 ...  0.06796802  0.13528903\n",
      "   -0.022035  ]]\n",
      "\n",
      " ...\n",
      "\n",
      " [[ 0.          0.          0.         ...  0.          0.\n",
      "    0.        ]\n",
      "  [ 0.          0.          0.         ...  0.          0.\n",
      "    0.        ]\n",
      "  [ 0.          0.          0.         ...  0.          0.\n",
      "    0.        ]\n",
      "  ...\n",
      "  [ 0.22104432 -0.01606884  0.00432623 ...  0.13655335  0.01242723\n",
      "    0.00964247]\n",
      "  [ 0.10299458  0.16180418 -0.08432977 ...  0.04323117  0.08910137\n",
      "   -0.00400291]\n",
      "  [ 0.012986    0.08981702  0.16017003 ...  0.06796802  0.13528903\n",
      "   -0.022035  ]]\n",
      "\n",
      " [[ 0.          0.          0.         ...  0.          0.\n",
      "    0.        ]\n",
      "  [ 0.          0.          0.         ...  0.          0.\n",
      "    0.        ]\n",
      "  [ 0.          0.          0.         ...  0.          0.\n",
      "    0.        ]\n",
      "  ...\n",
      "  [ 0.16666627  0.01299293  0.04397342 ...  0.10751364 -0.00171146\n",
      "   -0.0662554 ]\n",
      "  [ 0.09130137 -0.02096943 -0.02457789 ... -0.12161301  0.150714\n",
      "   -0.07756138]\n",
      "  [ 0.012986    0.08981702  0.16017003 ...  0.06796802  0.13528903\n",
      "   -0.022035  ]]\n",
      "\n",
      " [[ 0.          0.          0.         ...  0.          0.\n",
      "    0.        ]\n",
      "  [ 0.          0.          0.         ...  0.          0.\n",
      "    0.        ]\n",
      "  [ 0.          0.          0.         ...  0.          0.\n",
      "    0.        ]\n",
      "  ...\n",
      "  [ 0.03970453 -0.04158472  0.10027828 ... -0.0063081  -0.0119952\n",
      "   -0.04243935]\n",
      "  [ 0.04465724  0.07450107 -0.04335912 ... -0.1517998   0.18933882\n",
      "   -0.08688068]\n",
      "  [ 0.012986    0.08981702  0.16017003 ...  0.06796802  0.13528903\n",
      "   -0.022035  ]]], shape=(16, 13, 128), dtype=float32)\n",
      "tf.Tensor(\n",
      "[[   1    5   24   12  741    3    2    0    0    0    0    0    0    0]\n",
      " [   1    4  180  147    3    2    0    0    0    0    0    0    0    0]\n",
      " [   1    4   15  447    3    2    0    0    0    0    0    0    0    0]\n",
      " [   1    4   15  854    3    2    0    0    0    0    0    0    0    0]\n",
      " [   1   33   23   25    3    2    0    0    0    0    0    0    0    0]\n",
      " [   1   10   16  223  967    3    2    0    0    0    0    0    0    0]\n",
      " [   1   10    6  189    3    2    0    0    0    0    0    0    0    0]\n",
      " [   1  236   13   22    7    2    0    0    0    0    0    0    0    0]\n",
      " [   1    4   18   40  107   88    3    2    0    0    0    0    0    0]\n",
      " [   1  288   53  780    9    2    0    0    0    0    0    0    0    0]\n",
      " [   1   83    6    5  205    7    2    0    0    0    0    0    0    0]\n",
      " [   1    4   15 1484    3    2    0    0    0    0    0    0    0    0]\n",
      " [   1   43    6  106  208    7    2    0    0    0    0    0    0    0]\n",
      " [   1    5    6  128    3    2    0    0    0    0    0    0    0    0]\n",
      " [   1   73   12  245    3    2    0    0    0    0    0    0    0    0]\n",
      " [   1    5   24   12  429    3    2    0    0    0    0    0    0    0]], shape=(16, 14), dtype=int32)\n"
     ]
    }
   ],
   "source": [
    "for english,german in valid_dataset.take(1):\n",
    "  print(english)\n",
    "  print(german)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "isIYhjq01iay"
   },
   "source": [
    "## 3. Create the custom layer\n",
    "You will now create a custom layer to add the learned end token embedding to the encoder model:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 423
    },
    "colab_type": "code",
    "id": "e22f1Xyh6xvE",
    "outputId": "794ca554-5c3c-42e8-f56b-3e9d7928b66b"
   },
   "outputs": [
    {
     "data": {
      "image/png": "[encoded data, 23580 characters]\n",
      "text/plain": [
       "<IPython.core.display.Image object>"
      ]
     },
     "execution_count": 24,
     "metadata": {
      "tags": []
     },
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Run this cell to download and view a schematic diagram for the encoder model\n",
    "\n",
    "!wget -q -O neural_translation_model.png --no-check-certificate \"https://docs.google.com/uc?export=download&id=1JrtNOzUJDaOWrK4C-xv-4wUuZaI12sQI\"\n",
    "Image(\"neural_translation_model.png\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "M6gLIHG81iaz"
   },
   "source": [
    "You should now build the custom layer.\n",
    "* Using layer subclassing, create a custom layer that takes a batch of English data examples from one of the Datasets, and adds a learned embedded ‘end’ token to the end of each sequence. \n",
    "* This layer should create a TensorFlow Variable (that will be learned during training) that is 128-dimensional (the size of the embedding space). _Hint: you may find it helpful in the call method to use the tf.tile function to replicate the end token embedding across every element in the batch._\n",
    "* Using the Dataset `.take(1)` method, extract a batch of English data examples from the training Dataset and print the shape. Test the custom layer by calling the layer on the English data batch Tensor and print the resulting Tensor shape (the layer should increase the sequence length by one)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "yg9hjZz11ia0"
   },
   "outputs": [],
   "source": [
    "class EndTokenLayer(Layer):\n",
    "  \n",
    "  def __init__(self, embedding_dim=128, **kwargs):\n",
    "    super(EndTokenLayer, self).__init__(**kwargs)\n",
    "    self.embedding_dim = embedding_dim\n",
    "  def build(self, input_shape):\n",
    "    self.end_token_emb = self.add_weight(shape=(input_shape[-1],),\n",
    "                                          initializer='random_uniform',\n",
    "                                         trainable= True)\n",
    "  def call(self, inputs):\n",
    "    end_token = tf.tile(tf.reshape(self.end_token_emb, shape=(1, 1, self.end_token_emb.shape[0])), [tf.shape(inputs)[0],1,1])\n",
    "    return tf.keras.layers.concatenate([inputs, end_token], axis=1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "vFJUMAbNqDqp"
   },
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": 64,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 54
    },
    "colab_type": "code",
    "id": "4PwI32O11ia3",
    "outputId": "598a9bbb-0ea7-4971-c385-5bcddde14d4a"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "english sentences shape\n",
      "(16, 13, 128)\n"
     ]
    }
   ],
   "source": [
    "endlayer = EndTokenLayer()\n",
    "for english,german in train_dataset.take(1):\n",
    "  temp_layer = endlayer(english)\n",
    "  print(\"english sentences shape\")\n",
    "  print(english.shape) "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 65,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 54
    },
    "colab_type": "code",
    "id": "XlhLpsO_lIF0",
    "outputId": "36785279-7e7e-4d54-b897-bc6cd7ad5d3d"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "end added shape of english sentences:\n",
      "(16, 14, 128)\n"
     ]
    }
   ],
   "source": [
    "print(\"end added shape of english sentences:\")\n",
    "print(temp_layer.shape)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "OAd3i4_y1ia-"
   },
   "source": [
    "## 4. Build the encoder network\n",
    "The encoder network follows the schematic diagram above. You should now build the RNN encoder model.\n",
    "* Using the functional API, build the encoder network according to the following spec:\n",
    "    * The model will take a batch of sequences of embedded English words as input, as given by the Dataset objects.\n",
    "    * The next layer in the encoder will be the custom layer you created previously, to add a learned end token embedding to the end of the English sequence.\n",
    "    * This is followed by a Masking layer, with the `mask_value` set to the distinct padding value you used when you padded the English sequences with the Dataset preprocessing above.\n",
    "    * The final layer is an LSTM layer with 512 units, which also returns the hidden and cell states.\n",
    "    * The encoder is a multi-output model. There should be two output Tensors of this model: the hidden state and cell states of the LSTM layer. The output of the LSTM layer is unused.\n",
    "* Using the Dataset `.take(1)` method, extract a batch of English data examples from the training Dataset and test the encoder model by calling it on the English data Tensor, and print the shape of the resulting Tensor outputs.\n",
    "* Print the model summary for the encoder network."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "6R2LqbfV1ia_"
   },
   "outputs": [],
   "source": [
    "def encoder(input_shape):\n",
    "  inputs = Input([13,input_shape])\n",
    "  h = EndTokenLayer()(inputs)\n",
    "  h = Masking([(lambda x: x*0)(x) for x in range(128)])(h)\n",
    "  lstm , hidden_state, cell_state = LSTM(512,return_sequences = True,return_state=True)(h)\n",
    "  model = Model(inputs=inputs, outputs=[hidden_state, cell_state])\n",
    "\n",
    "  return model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 308
    },
    "colab_type": "code",
    "id": "e5XW6NxL1ibC",
    "outputId": "2c9d3ef7-738d-492b-fad6-c206844a1122"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Model: \"model\"\n",
      "_________________________________________________________________\n",
      "Layer (type)                 Output Shape              Param #   \n",
      "=================================================================\n",
      "input_1 (InputLayer)         [(None, 13, 128)]         0         \n",
      "_________________________________________________________________\n",
      "end_token_layer_1 (EndTokenL (None, 14, 128)           128       \n",
      "_________________________________________________________________\n",
      "masking (Masking)            (None, 14, 128)           0         \n",
      "_________________________________________________________________\n",
      "lstm (LSTM)                  [(None, 14, 512), (None,  1312768   \n",
      "=================================================================\n",
      "Total params: 1,312,896\n",
      "Trainable params: 1,312,896\n",
      "Non-trainable params: 0\n",
      "_________________________________________________________________\n"
     ]
    }
   ],
   "source": [
    "encoder_model = encoder(128)\n",
    "encoder_model.summary()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "jEk9ikVh1ibL"
   },
   "outputs": [],
   "source": [
    "for english,german in train_dataset.take(1):\n",
    "  result_1,result_2 = encoder_model(english)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 54
    },
    "colab_type": "code",
    "id": "HsVi4ZNnlGa_",
    "outputId": "80782bcb-b1e4-4ca2-e9eb-e3f48a4ed043"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tf.Tensor([ 16 512], shape=(2,), dtype=int32)\n",
      "tf.Tensor([ 16 512], shape=(2,), dtype=int32)\n"
     ]
    }
   ],
   "source": [
    "print(tf.shape(result_1))\n",
    "print(tf.shape(result_2))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "KvkzpCeZ1ibR"
   },
   "source": [
    "## 5. Build the decoder network\n",
    "The decoder network follows the schematic diagram below. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 501
    },
    "colab_type": "code",
    "id": "yOjEb7cH7Y4S",
    "outputId": "5b44888c-9039-4cfc-da6e-b4813fd4d782"
   },
   "outputs": [
    {
     "data": {
      "image/png": "[encoded data, 29812 characters]\n",
      "text/plain": [
       "<IPython.core.display.Image object>"
      ]
     },
     "execution_count": 32,
     "metadata": {
      "tags": []
     },
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Run this cell to download and view a schematic diagram for the decoder model\n",
    "\n",
    "!wget -q -O neural_translation_model.png --no-check-certificate \"https://docs.google.com/uc?export=download&id=1DTeaXD8tA8RjkpVrB2mr9csSBOY4LQiW\"\n",
    "Image(\"neural_translation_model.png\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "VPBK8WGL1ibS"
   },
   "source": [
    "You should now build the RNN decoder model.\n",
    "* Using Model subclassing, build the decoder network according to the following spec:\n",
    "    * The initializer should create the following layers:\n",
    "        * An Embedding layer with vocabulary size set to the number of unique German tokens, embedding dimension 128, and set to mask zero values in the input.\n",
    "        * An LSTM layer with 512 units, that returns its hidden and cell states, and also returns sequences.\n",
    "        * A Dense layer with number of units equal to the number of unique German tokens, and no activation function.\n",
    "    * The call method should include the usual `inputs` argument, as well as the additional keyword arguments `hidden_state` and `cell_state`. The default value for these keyword arguments should be `None`.\n",
    "    * The call method should pass the inputs through the Embedding layer, and then through the LSTM layer. If the `hidden_state` and `cell_state` arguments are provided, these should be used for the initial state of the LSTM layer. _Hint: use the_ `initial_state` _keyword argument when calling the LSTM layer on its input._\n",
    "    * The call method should pass the LSTM output sequence through the Dense layer, and return the resulting Tensor, along with the hidden and cell states of the LSTM layer.\n",
    "* Using the Dataset `.take(1)` method, extract a batch of English and German data examples from the training Dataset. Test the decoder model by first calling the encoder model on the English data Tensor to get the hidden and cell states, and then call the decoder model on the German data Tensor and hidden and cell states, and print the shape of the resulting decoder Tensor outputs.\n",
    "* Print the model summary for the decoder network."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "5i5XkhtsGhsO"
   },
   "outputs": [],
   "source": [
    "unique_tokens = len(tokenizer.word_index) + 1\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "l50qhnXD1ibT"
   },
   "outputs": [],
   "source": [
    "class DecoderModel(Model):\n",
    "  def __init__(self,initial_state=True,**kwargs):\n",
    "    super(DecoderModel, self).__init__(**kwargs)\n",
    "    self.embedding = Embedding(input_dim = unique_tokens,output_dim = 128,mask_zero = True)\n",
    "    self.lstm = LSTM(512, return_sequences=True, return_state=True)\n",
    "    self.dense = Dense(unique_tokens)\n",
    "    self.initial_state = initial_state\n",
    "\n",
    "  def call(self,inputs,hidden_state = None,cell_state = None):\n",
    "    h = self.embedding(inputs)\n",
    "    if hidden_state != None and cell_state != None:\n",
    "      lstm,hidden_1,cell_1 = self.lstm(h,initial_state = [hidden_state,cell_state])\n",
    "    else:\n",
    "      lstm,hidden_1,cell_1 = self.lstm(h)\n",
    "    h = self.dense(lstm)\n",
    "    return h,hidden_1,cell_1\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "vG5CRIXV1ibi"
   },
   "outputs": [],
   "source": [
    "decoder_model = DecoderModel()\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 126
    },
    "colab_type": "code",
    "id": "SvqqT_ET1ibl",
    "outputId": "f1ec691f-d17d-478c-cc7b-05f846334624"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "decoder output shape:\n",
      "tf.Tensor([  16   14 5744], shape=(3,), dtype=int32)\n",
      "hidden state shape:\n",
      "tf.Tensor([ 16 512], shape=(2,), dtype=int32)\n",
      "cell state shape:\n",
      "tf.Tensor([ 16 512], shape=(2,), dtype=int32)\n"
     ]
    }
   ],
   "source": [
    "for english,german in train_dataset.take(1):\n",
    "  temp,hidden_1,cell_1 = decoder_model(german)\n",
    "print(\"decoder output shape:\")\n",
    "print(tf.shape(temp))\n",
    "print(\"hidden state shape:\")\n",
    "print(tf.shape(hidden_1))\n",
    "print(\"cell state shape:\")\n",
    "print(tf.shape(cell_1))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 272
    },
    "colab_type": "code",
    "id": "wF3B3d2y9LCn",
    "outputId": "599233b8-01ef-4173-9269-f68402b147ce"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Model: \"decoder_model\"\n",
      "_________________________________________________________________\n",
      "Layer (type)                 Output Shape              Param #   \n",
      "=================================================================\n",
      "embedding (Embedding)        multiple                  735232    \n",
      "_________________________________________________________________\n",
      "lstm_1 (LSTM)                multiple                  1312768   \n",
      "_________________________________________________________________\n",
      "dense (Dense)                multiple                  2946672   \n",
      "=================================================================\n",
      "Total params: 4,994,672\n",
      "Trainable params: 4,994,672\n",
      "Non-trainable params: 0\n",
      "_________________________________________________________________\n"
     ]
    }
   ],
   "source": [
    "decoder_model.summary()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "pST9XGJ81ibo"
   },
   "source": [
    "## 6. Make a custom training loop\n",
    "You should now write a custom training loop to train your custom neural translation model.\n",
    "* Define a function that takes a Tensor batch of German data (as extracted from the training Dataset), and returns a tuple containing German inputs and outputs for the decoder model (refer to schematic diagram above).\n",
    "* Define a function that computes the forward and backward pass for your translation model. This function should take an English input, German input and German output as arguments, and should do the following:\n",
    "    * Pass the English input into the encoder, to get the hidden and cell states of the encoder LSTM.\n",
    "    * These hidden and cell states are then passed into the decoder, along with the German inputs, which returns a sequence of outputs (the hidden and cell state outputs of the decoder LSTM are unused in this function).\n",
    "    * The loss should then be computed between the decoder outputs and the German output function argument.\n",
    "    * The function returns the loss and gradients with respect to the encoder and decoder’s trainable variables.\n",
    "    * Decorate the function with `@tf.function`\n",
    "* Define and run a custom training loop for a number of epochs (for you to choose) that does the following:\n",
    "    * Iterates through the training dataset, and creates decoder inputs and outputs from the German sequences.\n",
    "    * Updates the parameters of the translation model using the gradients of the function above and an optimizer object.\n",
    "    * Every epoch, compute the validation loss on a number of batches from the validation and save the epoch training and validation losses.\n",
    "* Plot the learning curves for loss vs epoch for both training and validation sets.\n",
    "\n",
    "_Hint: This model is computationally demanding to train. The quality of the model or length of training is not a factor in the grading rubric. However, to obtain a better model we recommend using the GPU accelerator hardware on Colab._"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "7hJHbWqs1ibr"
   },
   "outputs": [],
   "source": [
    "def german_io(german_data):\n",
    "  \n",
    "  input_data = german_data[:,0:tf.shape(german_data)[1]-1]\n",
    "  output_data = german_data[:,1:tf.shape(german_data)[1]]\n",
    "  return(input_data,output_data)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "CykC5OlL71VK"
   },
   "outputs": [],
   "source": [
    "loss_object = tf.keras.losses.SparseCategoricalCrossentropy(from_logits = True)\n",
    "\n",
    "optimizer = tf.keras.optimizers.Adam(learning_rate=0.001)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "Jvu4J4-u1ibu"
   },
   "outputs": [],
   "source": [
    "@tf.function\n",
    "def fb_passes(english_input,german_input,german_output):\n",
    "  with tf.GradientTape() as tape:\n",
    "    hidden_state ,cell_state = encoder_model(english_input)\n",
    "    dense_output, _, _ = decoder_model(german_input, hidden_state, cell_state)\n",
    "    loss = tf.math.reduce_mean(loss_object(german_output,dense_output))\n",
    "    trainable_variables = encoder_model.trainable_variables + decoder_model.trainable_variables\n",
    "    gradients = tape.gradient(loss, trainable_variables)\n",
    "  return loss, gradients"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 1252,
     "referenced_widgets": [
      "7cf2f09cbb2e49baa326b070a57135b6",
      "a61b88812be7458e9384797ce7322974",
      "8f56f460ce0743adb631f81b47279032",
      "5ac9165d021f4013aa0943b6b4d78218",
      "3d6a8e27074349418cd71c5290f326b5",
      "0bbdb676c6f44fea87b6db3c5d3f4450",
      "fc62d2b357614a41a04a474d45311b2e",
      "f5263bdc9a1143289e04950c75cb57f7",
      "aaa773cbcae14260bdf841d28d29301b",
      "a0366ad869bc4e1e8237816dc4b3521b",
      "e58007a175724a93826f54ad010eb3e0",
      "2593f807e9054c629b228302b026fe78",
      "e1f1bd459a3e4fc29763bac02058bcc4",
      "3948b71f1b314f68b2d11ca915553e30",
      "d0b9276d37814f3e9fbe1caaad9e30d7",
      "eb69ae54b2794c698af7b7399c566aee",
      "eb8c2ad4b54e4322af6eaf9acb2146f9",
      "c82d275680734e5289db19cb909ee021",
      "1a44ba624b804903934006950be10946",
      "77fa3907e9f94a0ebe7dcae9fc452048",
      "efeac8374c38426f906da594f014959d",
      "14d9aa924abf48d7845629ced2003e60",
      "ec1f39d1218246db9625b116bf0046b5",
      "f5909acd99cc4cc984d2b5c8433ef031",
      "803248d7f99146919fc755a33d488aba",
      "062179ac0d2b48bcbb69c74d05a900e3",
      "b76f983e43cd4e71b7c7c062ded7d3d5",
      "47194203538f443fbdc98751785303f2",
      "6bc41a9601fd4e23833e7095a84bcc6a",
      "b33c81981277493ba2f6d83c20e2e21e",
      "6dbd979f74144c668d628d75abd2d6dd",
      "e42fea6d80ee41fa87eb05bf6619a171",
      "43909ce2bd4f494ca7f48dfa4df64ac4",
      "6d8f80f9f3c4430b83cc986c07513c9a",
      "f36b7603103b41afa037b1736ce7987c",
      "0aae27c452e24ff0bf6e87b3393109c2",
      "ca9714a20e7b48939748a81acd7815d7",
      "e1df1eda8fd946388b80cf7a753bb8d2",
      "2d67fd09c8d0414eb38f44790b4098fe",
      "4aea9d5a2e0a4084807550d0180374ce",
      "a13c646a04d24355a523f240b81c5613",
      "91ecd50ff6394107bf31db544b61847a",
      "678138fdac934d97a51537537e6d60cf",
      "44869506689c4caf9f401df6f3567e1f",
      "d543e2ba6e1d4791a834d818d2dfa85d",
      "67692bbfc09e45c7a462e5008b9547c5",
      "a01c8bc77dc44043ad075bf5dfe9cf60",
      "1be220b3c4b84ce18ff68ce962e9a949",
      "d5dd318e95cb4ff6b03aaa25233a5497",
      "be8601fb370b4df8a8fe9c4ec895c1f4",
      "3d0e43a0a080403b94b64e3e83909e9c",
      "720237b0357f4d5b9635cb5a975686bf",
      "eb9b6424c9f045b19a65d349c8004828",
      "2bcb448b88dd411aa0cee1979281e041",
      "9f3cfc90bd844f75b29e8a5615e5fd40",
      "bf6a570468014399b38142fc9a22fd76",
      "f85725cd811c41049ca3b4a1a1d8d52b",
      "fae25072555940479f91fe5fe7027ef5",
      "ef88874f5bc346c393e6bbb6e18562f7",
      "2b720265187e48e1a35107e338a9a999",
      "21bc74bd84ad4daba065f1772ed662fd",
      "5396e1fd60504686b6c9bc64fb4a4d06",
      "c473432110a24a788d9abf8d15425deb",
      "771ed83b268a4f66b226abae17711c6f",
      "77ee6c65243d4aea94c0ca6f324e9600",
      "bb98fe55077e456b8816a15522976bf9",
      "5728de623f4c4440b9b49f92c2c95910",
      "235185543ac74a9984ed589ae4df2c47",
      "5b4f4314bc894bc9a810bdf1b622857b",
      "5f12262f399a402a80641ef275c3e6c2",
      "d9740797e3754ef385f6f8336cbe6fac",
      "c27f6a8a581741388b6b6a76097962b9",
      "e185261a681f4af0bdb2e3823e3891b6",
      "76d7f0a0ba2f444eacaa6ac45eadeb8b",
      "7365309c89e144b1b9235fb9c758aa82",
      "13a52a32bbb040309a1c777f87f29bb1",
      "72b499272b804968bd9732141dc021cb",
      "929e4d34db9c47b5abeb6e8e6527e40d",
      "bbdb4c77129e47e2bdbc9f595a2aa2db",
      "483e36286fe34ab2bbbe30683736c09d",
      "480c4b953af740848089472d2b12f54f",
      "506b8e8184934f31acfc219574d48aff",
      "3b45631e38144ad1859402a2944c936d",
      "ed916bc1bd1242d3a68568625c73417d",
      "ee3e8f88a4ef45979d7ca5e30ad9c125",
      "282b77cd3c4c4d8aa3fa66aad2cc579f",
      "2eeda4cb83aa44ea916d98091e5f2bae",
      "2eacf0a522cb42929f123e2b46b24aa3",
      "c00601a1aea542d6b7e2626f19b8f038",
      "0b06f6925dbf42d48e7e90c4f6c5d999",
      "2a26535ac3a746dfbcaa1e41d64a622d",
      "5893aac2ec524ccf8d307f8fefbe5600",
      "77e5649da409410abd7f1e8363ff16e0",
      "5d349e33912a4c69b47c5ae49853a34c",
      "dd2540db949d42748cdfe86b49e1f63e",
      "0c9dcfe6797b4089a47b1778bfb12dc0",
      "78a2ee48e8744d7dbd4095e7d0de5a0a",
      "9bb82a154d00476d861a90eaf105c994",
      "ef6fc0069caa408681ed24e43b075c31",
      "fd2bd8c066944b2bb3681179d1e773d0",
      "631b21b11c5a45879c4f5070dc0feaf1",
      "87daeb0bd0754c75b5661f7840233ad0",
      "8953c30291cf4f699814869d4b5a09c3",
      "ba293c1ad08b435db5558ddf5f82baaa",
      "b6129ba9f3b84f3b9fdbde3ea309cad2",
      "198803aa7c9a45dbaf88e265562cec93",
      "67773255ad0d4f1e9547aa99893c5783",
      "6f805de20301483480f3b17d495c7e3f",
      "e40a6afc8cd3425199bb0787f4b10f6d",
      "ed76467bae354c5197fc851cfee38704",
      "3ac927282a364bf9a1a3b180772ab840",
      "89077b33906c4e73a5af69151ff3a697",
      "a668e92a47644d52b97309a0b4c5b69b",
      "4b18e248eca649b9bec73bea37b5c11a",
      "30d271d2736a4d978d3e82bfe56eabc2",
      "1d37494429ea4c94b6a04d777e102288",
      "b41dc937f8c440c0ae891ce92ea1429e",
      "428fed747b004809964f74862290b10e",
      "830f327e49fe457f977d507460ceae7c",
      "2ba405c81c874e678f65c4dfe8522e91",
      "1d836faa041e49bfb60069abae3388dc",
      "b450702e1ec544ec9a4ea23952810a30",
      "edd5097c68f34aceb7b864e24ccccf4d",
      "c30365ecb3d84928907be05d41656ecd",
      "29b12965e6734707b300c42529f1c434",
      "26a0a00f29d749c583b9bb3daf0068f4",
      "5b4da417f7c14198857f103fe0ec21fb",
      "5a921281ee6b42bab84f09fe3947a94a",
      "8d32d1e1e9914826870413592b41b10b",
      "be66e21ac4a24a2d81b776f14a3806b0",
      "a09cfde07faa4705ae747aff01a1583d",
      "c8c5a85ed6ce4411b0aa5779c4b7e978",
      "539571ba1f43413593c939524bbd2d1b",
      "4cd46a0bcd284b1db0211389fe3ad7ec",
      "ffce7755cefa4c688d4eab7cd9876674",
      "5b76bc51b4494c0fb95df76ab2fb1d65",
      "68600582496043389e99e8b8a9aa88ab",
      "af7f03e4bb454dccbc41f71a3f928966",
      "3459b019efd740fca23d7ba661a7f29b",
      "645160414a094c2d80982b9acaeec514",
      "4c60c8f3fd1844efaccef38f5b81838e",
      "7f00757074234050b67266815b0d60ee",
      "0b9b818e1d404904895a9e55d75a9365",
      "9d8745f319584c6ea64bb9af0bde2790",
      "2644b241608a46ab9a032bfc8e31e263",
      "900a79e911f14c648599db9d99c15dd0",
      "29ea5efaf26f4416bd5a1d27284e42f7",
      "19174ad2f0334918ae64ee5d5c1350f8",
      "5de3d949221643deabc9d182baa5c46d",
      "8208c8bce4594578aa3931dae0187e41",
      "25c48651674747ad928ec3a6f4e68c8c",
      "a384224fbe1f4655b65e2176681ea327",
      "0694697d66344a34994953682f8bf350",
      "e36ae6801f864a11a5838cf9ed4ba353",
      "e6e7fc5c6cc34762855f85d90369a60f",
      "94153c16c4a44ed4a0dcccb0a750d361",
      "ae1ad1e6d4414093bbaee30ef5f7c64c",
      "2e7ec1c896904a79ae2924a0c072c9f4",
      "3283a19b26ab4019b0df651228ae6bd4",
      "842fe98ab1544003a3410feb0ea2489d",
      "0122fad2eef947b98affc30bb5d523f0",
      "d9605817015940aab431158b5b132ba6",
      "ce81663a9ca146aeb8b3e1975bf802ad",
      "7243954bcaa941f49d2e3f81b4ad11c6",
      "415b9b7ca7254a89b742425f8d8ea152",
      "8acb2018f7854a839fc18946c1f931f7",
      "ce002367634e4ea198e91c66393ea2e9",
      "d46e63ac23794b049ebee823eaf35c7c"
     ]
    },
    "colab_type": "code",
    "id": "AhMJRoJmJn-g",
    "outputId": "5eca09e7-07b3-45a6-8584-de4796b35af0"
   },
   "outputs": [
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "7cf2f09cbb2e49baa326b070a57135b6",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "HBox(children=(FloatProgress(value=0.0, max=10.0), HTML(value='')))"
      ]
     },
     "metadata": {
      "tags": []
     },
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "aaa773cbcae14260bdf841d28d29301b",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "HBox(children=(FloatProgress(value=0.0, max=1000.0), HTML(value='')))"
      ]
     },
     "metadata": {
      "tags": []
     },
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "eb8c2ad4b54e4322af6eaf9acb2146f9",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "HBox(children=(FloatProgress(value=0.0, max=250.0), HTML(value='')))"
      ]
     },
     "metadata": {
      "tags": []
     },
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "Epoch 01: Avg. training loss = 4.303783, Avg. validation loss = 3.971151 \n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "803248d7f99146919fc755a33d488aba",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "HBox(children=(FloatProgress(value=0.0, max=1000.0), HTML(value='')))"
      ]
     },
     "metadata": {
      "tags": []
     },
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "43909ce2bd4f494ca7f48dfa4df64ac4",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "HBox(children=(FloatProgress(value=0.0, max=250.0), HTML(value='')))"
      ]
     },
     "metadata": {
      "tags": []
     },
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "Epoch 02: Avg. training loss = 3.408585, Avg. validation loss = 3.229689 \n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "a13c646a04d24355a523f240b81c5613",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "HBox(children=(FloatProgress(value=0.0, max=1000.0), HTML(value='')))"
      ]
     },
     "metadata": {
      "tags": []
     },
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "d5dd318e95cb4ff6b03aaa25233a5497",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "HBox(children=(FloatProgress(value=0.0, max=250.0), HTML(value='')))"
      ]
     },
     "metadata": {
      "tags": []
     },
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "Epoch 03: Avg. training loss = 2.586687, Avg. validation loss = 2.563772 \n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "f85725cd811c41049ca3b4a1a1d8d52b",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "HBox(children=(FloatProgress(value=0.0, max=1000.0), HTML(value='')))"
      ]
     },
     "metadata": {
      "tags": []
     },
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "77ee6c65243d4aea94c0ca6f324e9600",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "HBox(children=(FloatProgress(value=0.0, max=250.0), HTML(value='')))"
      ]
     },
     "metadata": {
      "tags": []
     },
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "Epoch 04: Avg. training loss = 1.840311, Avg. validation loss = 1.982391 \n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "e185261a681f4af0bdb2e3823e3891b6",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "HBox(children=(FloatProgress(value=0.0, max=1000.0), HTML(value='')))"
      ]
     },
     "metadata": {
      "tags": []
     },
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "480c4b953af740848089472d2b12f54f",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "HBox(children=(FloatProgress(value=0.0, max=250.0), HTML(value='')))"
      ]
     },
     "metadata": {
      "tags": []
     },
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "Epoch 05: Avg. training loss = 1.217730, Avg. validation loss = 1.538847 \n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "c00601a1aea542d6b7e2626f19b8f038",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "HBox(children=(FloatProgress(value=0.0, max=1000.0), HTML(value='')))"
      ]
     },
     "metadata": {
      "tags": []
     },
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "78a2ee48e8744d7dbd4095e7d0de5a0a",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "HBox(children=(FloatProgress(value=0.0, max=250.0), HTML(value='')))"
      ]
     },
     "metadata": {
      "tags": []
     },
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "Epoch 06: Avg. training loss = 0.775655, Avg. validation loss = 1.278047 \n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "b6129ba9f3b84f3b9fdbde3ea309cad2",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "HBox(children=(FloatProgress(value=0.0, max=1000.0), HTML(value='')))"
      ]
     },
     "metadata": {
      "tags": []
     },
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "a668e92a47644d52b97309a0b4c5b69b",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "HBox(children=(FloatProgress(value=0.0, max=250.0), HTML(value='')))"
      ]
     },
     "metadata": {
      "tags": []
     },
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "Epoch 07: Avg. training loss = 0.510702, Avg. validation loss = 1.157365 \n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "1d836faa041e49bfb60069abae3388dc",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "HBox(children=(FloatProgress(value=0.0, max=1000.0), HTML(value='')))"
      ]
     },
     "metadata": {
      "tags": []
     },
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "8d32d1e1e9914826870413592b41b10b",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "HBox(children=(FloatProgress(value=0.0, max=250.0), HTML(value='')))"
      ]
     },
     "metadata": {
      "tags": []
     },
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "Epoch 08: Avg. training loss = 0.357175, Avg. validation loss = 1.111811 \n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "68600582496043389e99e8b8a9aa88ab",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "HBox(children=(FloatProgress(value=0.0, max=1000.0), HTML(value='')))"
      ]
     },
     "metadata": {
      "tags": []
     },
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "2644b241608a46ab9a032bfc8e31e263",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "HBox(children=(FloatProgress(value=0.0, max=250.0), HTML(value='')))"
      ]
     },
     "metadata": {
      "tags": []
     },
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "Epoch 09: Avg. training loss = 0.266191, Avg. validation loss = 1.098457 \n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "0694697d66344a34994953682f8bf350",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "HBox(children=(FloatProgress(value=0.0, max=1000.0), HTML(value='')))"
      ]
     },
     "metadata": {
      "tags": []
     },
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "0122fad2eef947b98affc30bb5d523f0",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "HBox(children=(FloatProgress(value=0.0, max=250.0), HTML(value='')))"
      ]
     },
     "metadata": {
      "tags": []
     },
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "Epoch 10: Avg. training loss = 0.204196, Avg. validation loss = 1.086921 \n",
      "\n"
     ]
    }
   ],
   "source": [
    "train_loss_results = []\n",
    "val_loss_results = []\n",
    "\n",
    "for epoch in tqdm(range(10)):\n",
    "  \n",
    "  epoch_loss = 0\n",
    "  batch_no = 0\n",
    "  with tqdm(total= 1000) as t:\n",
    "    for english,german in train_dataset:\n",
    "      german_input, german_output = german_io(german)\n",
    "      loss, gradients = fb_passes(english, german_input, german_output)\n",
    "      epoch_loss += loss\n",
    "      batch_no += 1\n",
    "      optimizer.apply_gradients(zip(gradients, encoder_model.trainable_variables + decoder_model.trainable_variables))\n",
    "      epoch_avg_loss = epoch_loss / batch_no\n",
    "      train_loss_results.append(epoch_avg_loss)\n",
    "      t.update(1)\n",
    "  epoch_val_loss = 0\n",
    "  val_batch_no = 0\n",
    "  with tqdm(total= 250) as t:\n",
    "    for val_english,val_german in valid_dataset:\n",
    "      german_input, german_output = german_io(val_german)\n",
    "      loss, _ = fb_passes(val_english, german_input, german_output)\n",
    "      epoch_val_loss += loss\n",
    "      val_batch_no += 1\n",
    "      epoch_avg_val_loss = epoch_val_loss / val_batch_no\n",
    "      val_loss_results.append(epoch_avg_val_loss)\n",
    "      t.update(1)\n",
    "  print(\"Epoch {:02d}: Avg. training loss = {:.6f}, Avg. validation loss = {:.6f} \".format(epoch + 1,epoch_avg_loss,epoch_avg_val_loss))\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 53,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 621
    },
    "colab_type": "code",
    "id": "mWIlf_y2YHfl",
    "outputId": "b8cca73c-e388-46b8-99fa-d8b8639ba6d5"
   },
   "outputs": [
    {
     "data": {
      "image/png": "[encoded data, 30496 characters]\n",
      "text/plain": [
       "<Figure size 720x720 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light",
      "tags": []
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "epoch_range = range(0,10)\n",
    "from scipy.ndimage.filters import gaussian_filter1d\n",
    "train_smoothed = gaussian_filter1d(train_loss_results, sigma=1500)\n",
    "valid_smoothed = gaussian_filter1d(val_loss_results, sigma=1500)\n",
    "\n",
    "# The graph is inconsistent due to smaller data used to make training faster\n",
    "fig = plt.figure(figsize = (10, 10))\n",
    "ax = fig.add_subplot(1,1,1)\n",
    "ax.plot(train_smoothed, label=\"Training\")\n",
    "ax.plot(valid_smoothed, label=\"Validation\")\n",
    "ax.legend(loc='best')\n",
    "ax.set_title(\"Loss vs Epoch\")\n",
    "ax.set_xticklabels(epoch_range)\n",
    "ax.set_xlabel(\"Epoch number\")\n",
    "ax.set_ylabel(\"Avg. loss\") \n",
    "\n",
    "fig.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "xM2gvBM11ib-"
   },
   "source": [
    "## 7. Use the model to translate\n",
    "Now it's time to put your model into practice! You should run your translation for five randomly sampled English sentences from the dataset. For each sentence, the process is as follows:\n",
    "* Preprocess and embed the English sentence according to the model requirements.\n",
    "* Pass the embedded sentence through the encoder to get the encoder hidden and cell states.\n",
    "* Starting with the special  `\"<start>\"` token, use this token and the final encoder hidden and cell states to get the one-step prediction from the decoder, as well as the decoder’s updated hidden and cell states.\n",
    "* Create a loop to get the next step prediction and updated hidden and cell states from the decoder, using the most recent hidden and cell states. Terminate the loop when the `\"<end>\"` token is emitted, or when the sentence has reached a maximum length.\n",
    "* Decode the output token sequence into German text and print the English text and the model's German translation."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 59,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "hCgiIBiniZTe"
   },
   "outputs": [],
   "source": [
    "random_ind = np.random.choice(20000,5)\n",
    "examples = []\n",
    "for ind in random_ind:\n",
    "  examples.append(data_examples[ind])\n",
    "english_sentences = [sentence.split('\\t')[0] for sentence in examples]\n",
    "processed_english = []\n",
    "for sentence in english_sentences:\n",
    "    processed_english.append(preprocess_sentence(sentence))\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 60,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "bGnQtE7L1icA"
   },
   "outputs": [],
   "source": [
    "\n",
    "\n",
    "start = tokenizer.word_index['<start>']\n",
    "end = tokenizer.word_index['<end>']\n",
    "examples_tokens = []\n",
    "for p_english in processed_english:\n",
    "  english = tf.strings.split(p_english,sep = \" \")\n",
    "  english = embedding_layer(english)\n",
    "  english = tf.pad(english, [[13-len(english), 0], [0, 0]], constant_values = 0)\n",
    "  english = tf.expand_dims(english, 0)\n",
    "  hidden_state, cell_state = encoder_model(english)\n",
    "  translated_tokens = []\n",
    "  tf_token = tf.Variable([[start]])\n",
    "  while True:\n",
    "    output_1,hidden_state, cell_state = decoder_model(tf_token, hidden_state, cell_state)\n",
    "    output_2 = tf.argmax(output_1, 2).numpy()[0,0]\n",
    "    tf_token = tf.Variable([[output_2]])\n",
    "    if output_2 == end:\n",
    "      break\n",
    "    else:\n",
    "      translated_tokens.append(output_2)\n",
    "  examples_tokens.append(translated_tokens)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 61,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "Unk60cEy1icI"
   },
   "outputs": [],
   "source": [
    "inv_german_index = {value:key for key,value in tokenizer.word_index.items()}\n",
    "german_sentences = []\n",
    "for example_token in examples_tokens:\n",
    "  output_words = []\n",
    "  for token in example_token:\n",
    "    output_words.append(inv_german_index[token])\n",
    "  output = \" \".join(output_words)\n",
    "  german_sentences.append(output)\n",
    " "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 62,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 181
    },
    "colab_type": "code",
    "id": "QZtLHihbbBcG",
    "outputId": "da09ba59-f280-41f2-cac6-6d80c0db50e6"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "+--------------------+----------------------------+\n",
      "| English sentences  |    German Translations     |\n",
      "+--------------------+----------------------------+\n",
      "|  Tom may be out.   | tom wird sich verspaeten . |\n",
      "| Are you blushing?  |  bist du ueber achtzehn ?  |\n",
      "|   Is it popular?   |   ist es etwas ernstes ?   |\n",
      "| I have a solution. |  ich habe eine meinung .   |\n",
      "|    I'm starved.    |     ich bin verletzt .     |\n",
      "+--------------------+----------------------------+\n"
     ]
    }
   ],
   "source": [
    "table = PrettyTable(['English sentences', 'German Translations'])\n",
    "for english,german in zip(english_sentences,german_sentences):\n",
    "  table.add_row([english,german])\n",
    "  \n",
    "print(table)"
   ]
  }
 ],
 "metadata": {
  "accelerator": "GPU",
  "colab": {
   "collapsed_sections": [],
   "name": "Translation Capstone Project.ipynb",
   "provenance": []
  },
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.6"
  },
  "widgets": {
   "application/vnd.jupyter.widget-state+json": {
    "0122fad2eef947b98affc30bb5d523f0": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "HBoxModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_ce81663a9ca146aeb8b3e1975bf802ad",
       "IPY_MODEL_7243954bcaa941f49d2e3f81b4ad11c6"
      ],
      "layout": "IPY_MODEL_d9605817015940aab431158b5b132ba6"
     }
    },
    "062179ac0d2b48bcbb69c74d05a900e3": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "0694697d66344a34994953682f8bf350": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "HBoxModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_e6e7fc5c6cc34762855f85d90369a60f",
       "IPY_MODEL_94153c16c4a44ed4a0dcccb0a750d361"
      ],
      "layout": "IPY_MODEL_e36ae6801f864a11a5838cf9ed4ba353"
     }
    },
    "0aae27c452e24ff0bf6e87b3393109c2": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_4aea9d5a2e0a4084807550d0180374ce",
      "placeholder": "​",
      "style": "IPY_MODEL_2d67fd09c8d0414eb38f44790b4098fe",
      "value": " 250/250 [17:06&lt;00:00,  4.11s/it]"
     }
    },
    "0b06f6925dbf42d48e7e90c4f6c5d999": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "0b9b818e1d404904895a9e55d75a9365": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "0bbdb676c6f44fea87b6db3c5d3f4450": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "0c9dcfe6797b4089a47b1778bfb12dc0": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "13a52a32bbb040309a1c777f87f29bb1": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_483e36286fe34ab2bbbe30683736c09d",
      "placeholder": "​",
      "style": "IPY_MODEL_bbdb4c77129e47e2bdbc9f595a2aa2db",
      "value": " 1000/1000 [34:27&lt;00:00,  2.07s/it]"
     }
    },
    "14d9aa924abf48d7845629ced2003e60": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "19174ad2f0334918ae64ee5d5c1350f8": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_a384224fbe1f4655b65e2176681ea327",
      "placeholder": "​",
      "style": "IPY_MODEL_25c48651674747ad928ec3a6f4e68c8c",
      "value": " 250/250 [23:18&lt;00:00,  5.59s/it]"
     }
    },
    "198803aa7c9a45dbaf88e265562cec93": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "1a44ba624b804903934006950be10946": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "FloatProgressModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "success",
      "description": "100%",
      "description_tooltip": null,
      "layout": "IPY_MODEL_14d9aa924abf48d7845629ced2003e60",
      "max": 250,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_efeac8374c38426f906da594f014959d",
      "value": 250
     }
    },
    "1be220b3c4b84ce18ff68ce962e9a949": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "1d37494429ea4c94b6a04d777e102288": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_2ba405c81c874e678f65c4dfe8522e91",
      "placeholder": "​",
      "style": "IPY_MODEL_830f327e49fe457f977d507460ceae7c",
      "value": " 250/250 [12:33&lt;00:00,  3.01s/it]"
     }
    },
    "1d836faa041e49bfb60069abae3388dc": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "HBoxModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_edd5097c68f34aceb7b864e24ccccf4d",
       "IPY_MODEL_c30365ecb3d84928907be05d41656ecd"
      ],
      "layout": "IPY_MODEL_b450702e1ec544ec9a4ea23952810a30"
     }
    },
    "21bc74bd84ad4daba065f1772ed662fd": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "ProgressStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": "initial"
     }
    },
    "235185543ac74a9984ed589ae4df2c47": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_c27f6a8a581741388b6b6a76097962b9",
      "placeholder": "​",
      "style": "IPY_MODEL_d9740797e3754ef385f6f8336cbe6fac",
      "value": " 250/250 [41:47&lt;00:00, 10.03s/it]"
     }
    },
    "2593f807e9054c629b228302b026fe78": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_eb69ae54b2794c698af7b7399c566aee",
      "placeholder": "​",
      "style": "IPY_MODEL_d0b9276d37814f3e9fbe1caaad9e30d7",
      "value": " 1000/1000 [29:17&lt;00:00,  1.76s/it]"
     }
    },
    "25c48651674747ad928ec3a6f4e68c8c": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "2644b241608a46ab9a032bfc8e31e263": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "HBoxModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_29ea5efaf26f4416bd5a1d27284e42f7",
       "IPY_MODEL_19174ad2f0334918ae64ee5d5c1350f8"
      ],
      "layout": "IPY_MODEL_900a79e911f14c648599db9d99c15dd0"
     }
    },
    "26a0a00f29d749c583b9bb3daf0068f4": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "282b77cd3c4c4d8aa3fa66aad2cc579f": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "29b12965e6734707b300c42529f1c434": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "ProgressStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": "initial"
     }
    },
    "29ea5efaf26f4416bd5a1d27284e42f7": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "FloatProgressModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "success",
      "description": "100%",
      "description_tooltip": null,
      "layout": "IPY_MODEL_8208c8bce4594578aa3931dae0187e41",
      "max": 250,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_5de3d949221643deabc9d182baa5c46d",
      "value": 250
     }
    },
    "2a26535ac3a746dfbcaa1e41d64a622d": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "FloatProgressModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "success",
      "description": "100%",
      "description_tooltip": null,
      "layout": "IPY_MODEL_5d349e33912a4c69b47c5ae49853a34c",
      "max": 1000,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_77e5649da409410abd7f1e8363ff16e0",
      "value": 1000
     }
    },
    "2b720265187e48e1a35107e338a9a999": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_771ed83b268a4f66b226abae17711c6f",
      "placeholder": "​",
      "style": "IPY_MODEL_c473432110a24a788d9abf8d15425deb",
      "value": " 1000/1000 [49:28&lt;00:00,  2.97s/it]"
     }
    },
    "2ba405c81c874e678f65c4dfe8522e91": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "2bcb448b88dd411aa0cee1979281e041": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "2d67fd09c8d0414eb38f44790b4098fe": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "2e7ec1c896904a79ae2924a0c072c9f4": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "2eacf0a522cb42929f123e2b46b24aa3": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "2eeda4cb83aa44ea916d98091e5f2bae": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "30d271d2736a4d978d3e82bfe56eabc2": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "FloatProgressModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "success",
      "description": "100%",
      "description_tooltip": null,
      "layout": "IPY_MODEL_428fed747b004809964f74862290b10e",
      "max": 250,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_b41dc937f8c440c0ae891ce92ea1429e",
      "value": 250
     }
    },
    "3283a19b26ab4019b0df651228ae6bd4": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "3459b019efd740fca23d7ba661a7f29b": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "FloatProgressModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "success",
      "description": "100%",
      "description_tooltip": null,
      "layout": "IPY_MODEL_7f00757074234050b67266815b0d60ee",
      "max": 1000,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_4c60c8f3fd1844efaccef38f5b81838e",
      "value": 1000
     }
    },
    "3948b71f1b314f68b2d11ca915553e30": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "3ac927282a364bf9a1a3b180772ab840": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "3b45631e38144ad1859402a2944c936d": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "FloatProgressModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "success",
      "description": "100%",
      "description_tooltip": null,
      "layout": "IPY_MODEL_282b77cd3c4c4d8aa3fa66aad2cc579f",
      "max": 250,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_ee3e8f88a4ef45979d7ca5e30ad9c125",
      "value": 250
     }
    },
    "3d0e43a0a080403b94b64e3e83909e9c": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "FloatProgressModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "success",
      "description": "100%",
      "description_tooltip": null,
      "layout": "IPY_MODEL_2bcb448b88dd411aa0cee1979281e041",
      "max": 250,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_eb9b6424c9f045b19a65d349c8004828",
      "value": 250
     }
    },
    "3d6a8e27074349418cd71c5290f326b5": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "ProgressStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": "initial"
     }
    },
    "415b9b7ca7254a89b742425f8d8ea152": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "ProgressStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": "initial"
     }
    },
    "428fed747b004809964f74862290b10e": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "43909ce2bd4f494ca7f48dfa4df64ac4": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "HBoxModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_f36b7603103b41afa037b1736ce7987c",
       "IPY_MODEL_0aae27c452e24ff0bf6e87b3393109c2"
      ],
      "layout": "IPY_MODEL_6d8f80f9f3c4430b83cc986c07513c9a"
     }
    },
    "44869506689c4caf9f401df6f3567e1f": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_1be220b3c4b84ce18ff68ce962e9a949",
      "placeholder": "​",
      "style": "IPY_MODEL_a01c8bc77dc44043ad075bf5dfe9cf60",
      "value": " 1000/1000 [09:45&lt;00:00,  1.71it/s]"
     }
    },
    "47194203538f443fbdc98751785303f2": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_e42fea6d80ee41fa87eb05bf6619a171",
      "placeholder": "​",
      "style": "IPY_MODEL_6dbd979f74144c668d628d75abd2d6dd",
      "value": " 1000/1000 [19:30&lt;00:00,  1.17s/it]"
     }
    },
    "480c4b953af740848089472d2b12f54f": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "HBoxModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_3b45631e38144ad1859402a2944c936d",
       "IPY_MODEL_ed916bc1bd1242d3a68568625c73417d"
      ],
      "layout": "IPY_MODEL_506b8e8184934f31acfc219574d48aff"
     }
    },
    "483e36286fe34ab2bbbe30683736c09d": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "4aea9d5a2e0a4084807550d0180374ce": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "4b18e248eca649b9bec73bea37b5c11a": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "4c60c8f3fd1844efaccef38f5b81838e": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "ProgressStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": "initial"
     }
    },
    "4cd46a0bcd284b1db0211389fe3ad7ec": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "506b8e8184934f31acfc219574d48aff": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "539571ba1f43413593c939524bbd2d1b": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "ProgressStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": "initial"
     }
    },
    "5396e1fd60504686b6c9bc64fb4a4d06": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "5728de623f4c4440b9b49f92c2c95910": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "FloatProgressModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "success",
      "description": "100%",
      "description_tooltip": null,
      "layout": "IPY_MODEL_5f12262f399a402a80641ef275c3e6c2",
      "max": 250,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_5b4f4314bc894bc9a810bdf1b622857b",
      "value": 250
     }
    },
    "5893aac2ec524ccf8d307f8fefbe5600": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_0c9dcfe6797b4089a47b1778bfb12dc0",
      "placeholder": "​",
      "style": "IPY_MODEL_dd2540db949d42748cdfe86b49e1f63e",
      "value": " 1000/1000 [24:41&lt;00:00,  1.48s/it]"
     }
    },
    "5a921281ee6b42bab84f09fe3947a94a": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "5ac9165d021f4013aa0943b6b4d78218": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_f5263bdc9a1143289e04950c75cb57f7",
      "placeholder": "​",
      "style": "IPY_MODEL_fc62d2b357614a41a04a474d45311b2e",
      "value": " 10/10 [1:37:32&lt;00:00, 585.29s/it]"
     }
    },
    "5b4da417f7c14198857f103fe0ec21fb": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "5b4f4314bc894bc9a810bdf1b622857b": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "ProgressStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": "initial"
     }
    },
    "5b76bc51b4494c0fb95df76ab2fb1d65": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "5d349e33912a4c69b47c5ae49853a34c": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "5de3d949221643deabc9d182baa5c46d": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "ProgressStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": "initial"
     }
    },
    "5f12262f399a402a80641ef275c3e6c2": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "631b21b11c5a45879c4f5070dc0feaf1": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "ProgressStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": "initial"
     }
    },
    "645160414a094c2d80982b9acaeec514": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_9d8745f319584c6ea64bb9af0bde2790",
      "placeholder": "​",
      "style": "IPY_MODEL_0b9b818e1d404904895a9e55d75a9365",
      "value": " 1000/1000 [02:24&lt;00:00,  6.92it/s]"
     }
    },
    "67692bbfc09e45c7a462e5008b9547c5": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "67773255ad0d4f1e9547aa99893c5783": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "FloatProgressModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "success",
      "description": "100%",
      "description_tooltip": null,
      "layout": "IPY_MODEL_ed76467bae354c5197fc851cfee38704",
      "max": 1000,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_e40a6afc8cd3425199bb0787f4b10f6d",
      "value": 1000
     }
    },
    "678138fdac934d97a51537537e6d60cf": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "FloatProgressModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "success",
      "description": "100%",
      "description_tooltip": null,
      "layout": "IPY_MODEL_67692bbfc09e45c7a462e5008b9547c5",
      "max": 1000,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_d543e2ba6e1d4791a834d818d2dfa85d",
      "value": 1000
     }
    },
    "68600582496043389e99e8b8a9aa88ab": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "HBoxModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_3459b019efd740fca23d7ba661a7f29b",
       "IPY_MODEL_645160414a094c2d80982b9acaeec514"
      ],
      "layout": "IPY_MODEL_af7f03e4bb454dccbc41f71a3f928966"
     }
    },
    "6bc41a9601fd4e23833e7095a84bcc6a": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "ProgressStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": "initial"
     }
    },
    "6d8f80f9f3c4430b83cc986c07513c9a": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "6dbd979f74144c668d628d75abd2d6dd": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "6f805de20301483480f3b17d495c7e3f": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_89077b33906c4e73a5af69151ff3a697",
      "placeholder": "​",
      "style": "IPY_MODEL_3ac927282a364bf9a1a3b180772ab840",
      "value": " 1000/1000 [14:57&lt;00:00,  1.11it/s]"
     }
    },
    "720237b0357f4d5b9635cb5a975686bf": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_bf6a570468014399b38142fc9a22fd76",
      "placeholder": "​",
      "style": "IPY_MODEL_9f3cfc90bd844f75b29e8a5615e5fd40",
      "value": " 250/250 [56:49&lt;00:00, 13.64s/it]"
     }
    },
    "7243954bcaa941f49d2e3f81b4ad11c6": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_d46e63ac23794b049ebee823eaf35c7c",
      "placeholder": "​",
      "style": "IPY_MODEL_ce002367634e4ea198e91c66393ea2e9",
      "value": " 250/250 [07:21&lt;00:00,  1.77s/it]"
     }
    },
    "72b499272b804968bd9732141dc021cb": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "ProgressStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": "initial"
     }
    },
    "7365309c89e144b1b9235fb9c758aa82": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "FloatProgressModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "success",
      "description": "100%",
      "description_tooltip": null,
      "layout": "IPY_MODEL_929e4d34db9c47b5abeb6e8e6527e40d",
      "max": 1000,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_72b499272b804968bd9732141dc021cb",
      "value": 1000
     }
    },
    "76d7f0a0ba2f444eacaa6ac45eadeb8b": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "771ed83b268a4f66b226abae17711c6f": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "77e5649da409410abd7f1e8363ff16e0": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "ProgressStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": "initial"
     }
    },
    "77ee6c65243d4aea94c0ca6f324e9600": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "HBoxModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_5728de623f4c4440b9b49f92c2c95910",
       "IPY_MODEL_235185543ac74a9984ed589ae4df2c47"
      ],
      "layout": "IPY_MODEL_bb98fe55077e456b8816a15522976bf9"
     }
    },
    "77fa3907e9f94a0ebe7dcae9fc452048": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_f5909acd99cc4cc984d2b5c8433ef031",
      "placeholder": "​",
      "style": "IPY_MODEL_ec1f39d1218246db9625b116bf0046b5",
      "value": " 250/250 [26:53&lt;00:00,  6.45s/it]"
     }
    },
    "78a2ee48e8744d7dbd4095e7d0de5a0a": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "HBoxModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_ef6fc0069caa408681ed24e43b075c31",
       "IPY_MODEL_fd2bd8c066944b2bb3681179d1e773d0"
      ],
      "layout": "IPY_MODEL_9bb82a154d00476d861a90eaf105c994"
     }
    },
    "7cf2f09cbb2e49baa326b070a57135b6": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "HBoxModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_8f56f460ce0743adb631f81b47279032",
       "IPY_MODEL_5ac9165d021f4013aa0943b6b4d78218"
      ],
      "layout": "IPY_MODEL_a61b88812be7458e9384797ce7322974"
     }
    },
    "7f00757074234050b67266815b0d60ee": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "803248d7f99146919fc755a33d488aba": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "HBoxModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_b76f983e43cd4e71b7c7c062ded7d3d5",
       "IPY_MODEL_47194203538f443fbdc98751785303f2"
      ],
      "layout": "IPY_MODEL_062179ac0d2b48bcbb69c74d05a900e3"
     }
    },
    "8208c8bce4594578aa3931dae0187e41": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "830f327e49fe457f977d507460ceae7c": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "842fe98ab1544003a3410feb0ea2489d": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "87daeb0bd0754c75b5661f7840233ad0": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "89077b33906c4e73a5af69151ff3a697": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "8953c30291cf4f699814869d4b5a09c3": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "8acb2018f7854a839fc18946c1f931f7": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "8d32d1e1e9914826870413592b41b10b": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "HBoxModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_a09cfde07faa4705ae747aff01a1583d",
       "IPY_MODEL_c8c5a85ed6ce4411b0aa5779c4b7e978"
      ],
      "layout": "IPY_MODEL_be66e21ac4a24a2d81b776f14a3806b0"
     }
    },
    "8f56f460ce0743adb631f81b47279032": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "FloatProgressModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "success",
      "description": "100%",
      "description_tooltip": null,
      "layout": "IPY_MODEL_0bbdb676c6f44fea87b6db3c5d3f4450",
      "max": 10,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_3d6a8e27074349418cd71c5290f326b5",
      "value": 10
     }
    },
    "900a79e911f14c648599db9d99c15dd0": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "91ecd50ff6394107bf31db544b61847a": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "929e4d34db9c47b5abeb6e8e6527e40d": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "94153c16c4a44ed4a0dcccb0a750d361": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_842fe98ab1544003a3410feb0ea2489d",
      "placeholder": "​",
      "style": "IPY_MODEL_3283a19b26ab4019b0df651228ae6bd4",
      "value": " 1000/1000 [15:56&lt;00:00,  1.05it/s]"
     }
    },
    "9bb82a154d00476d861a90eaf105c994": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "9d8745f319584c6ea64bb9af0bde2790": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "9f3cfc90bd844f75b29e8a5615e5fd40": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "a01c8bc77dc44043ad075bf5dfe9cf60": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "a0366ad869bc4e1e8237816dc4b3521b": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "a09cfde07faa4705ae747aff01a1583d": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "FloatProgressModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "success",
      "description": "100%",
      "description_tooltip": null,
      "layout": "IPY_MODEL_4cd46a0bcd284b1db0211389fe3ad7ec",
      "max": 250,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_539571ba1f43413593c939524bbd2d1b",
      "value": 250
     }
    },
    "a13c646a04d24355a523f240b81c5613": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "HBoxModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_678138fdac934d97a51537537e6d60cf",
       "IPY_MODEL_44869506689c4caf9f401df6f3567e1f"
      ],
      "layout": "IPY_MODEL_91ecd50ff6394107bf31db544b61847a"
     }
    },
    "a384224fbe1f4655b65e2176681ea327": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "a61b88812be7458e9384797ce7322974": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "a668e92a47644d52b97309a0b4c5b69b": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "HBoxModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_30d271d2736a4d978d3e82bfe56eabc2",
       "IPY_MODEL_1d37494429ea4c94b6a04d777e102288"
      ],
      "layout": "IPY_MODEL_4b18e248eca649b9bec73bea37b5c11a"
     }
    },
    "aaa773cbcae14260bdf841d28d29301b": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "HBoxModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_e58007a175724a93826f54ad010eb3e0",
       "IPY_MODEL_2593f807e9054c629b228302b026fe78"
      ],
      "layout": "IPY_MODEL_a0366ad869bc4e1e8237816dc4b3521b"
     }
    },
    "ae1ad1e6d4414093bbaee30ef5f7c64c": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "ProgressStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": "initial"
     }
    },
    "af7f03e4bb454dccbc41f71a3f928966": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "b33c81981277493ba2f6d83c20e2e21e": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "b41dc937f8c440c0ae891ce92ea1429e": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "ProgressStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": "initial"
     }
    },
    "b450702e1ec544ec9a4ea23952810a30": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "b6129ba9f3b84f3b9fdbde3ea309cad2": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "HBoxModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_67773255ad0d4f1e9547aa99893c5783",
       "IPY_MODEL_6f805de20301483480f3b17d495c7e3f"
      ],
      "layout": "IPY_MODEL_198803aa7c9a45dbaf88e265562cec93"
     }
    },
    "b76f983e43cd4e71b7c7c062ded7d3d5": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "FloatProgressModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "success",
      "description": "100%",
      "description_tooltip": null,
      "layout": "IPY_MODEL_b33c81981277493ba2f6d83c20e2e21e",
      "max": 1000,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_6bc41a9601fd4e23833e7095a84bcc6a",
      "value": 1000
     }
    },
    "ba293c1ad08b435db5558ddf5f82baaa": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "bb98fe55077e456b8816a15522976bf9": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "bbdb4c77129e47e2bdbc9f595a2aa2db": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "be66e21ac4a24a2d81b776f14a3806b0": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "be8601fb370b4df8a8fe9c4ec895c1f4": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "bf6a570468014399b38142fc9a22fd76": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "c00601a1aea542d6b7e2626f19b8f038": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "HBoxModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_2a26535ac3a746dfbcaa1e41d64a622d",
       "IPY_MODEL_5893aac2ec524ccf8d307f8fefbe5600"
      ],
      "layout": "IPY_MODEL_0b06f6925dbf42d48e7e90c4f6c5d999"
     }
    },
    "c27f6a8a581741388b6b6a76097962b9": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "c30365ecb3d84928907be05d41656ecd": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_5a921281ee6b42bab84f09fe3947a94a",
      "placeholder": "​",
      "style": "IPY_MODEL_5b4da417f7c14198857f103fe0ec21fb",
      "value": " 1000/1000 [05:13&lt;00:00,  3.19it/s]"
     }
    },
    "c473432110a24a788d9abf8d15425deb": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "c82d275680734e5289db19cb909ee021": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "c8c5a85ed6ce4411b0aa5779c4b7e978": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_5b76bc51b4494c0fb95df76ab2fb1d65",
      "placeholder": "​",
      "style": "IPY_MODEL_ffce7755cefa4c688d4eab7cd9876674",
      "value": " 250/250 [08:05&lt;00:00,  1.94s/it]"
     }
    },
    "ca9714a20e7b48939748a81acd7815d7": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "ProgressStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": "initial"
     }
    },
    "ce002367634e4ea198e91c66393ea2e9": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "ce81663a9ca146aeb8b3e1975bf802ad": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "FloatProgressModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "success",
      "description": "100%",
      "description_tooltip": null,
      "layout": "IPY_MODEL_8acb2018f7854a839fc18946c1f931f7",
      "max": 250,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_415b9b7ca7254a89b742425f8d8ea152",
      "value": 250
     }
    },
    "d0b9276d37814f3e9fbe1caaad9e30d7": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "d46e63ac23794b049ebee823eaf35c7c": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "d543e2ba6e1d4791a834d818d2dfa85d": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "ProgressStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": "initial"
     }
    },
    "d5dd318e95cb4ff6b03aaa25233a5497": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "HBoxModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_3d0e43a0a080403b94b64e3e83909e9c",
       "IPY_MODEL_720237b0357f4d5b9635cb5a975686bf"
      ],
      "layout": "IPY_MODEL_be8601fb370b4df8a8fe9c4ec895c1f4"
     }
    },
    "d9605817015940aab431158b5b132ba6": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "d9740797e3754ef385f6f8336cbe6fac": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "dd2540db949d42748cdfe86b49e1f63e": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "e185261a681f4af0bdb2e3823e3891b6": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "HBoxModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_7365309c89e144b1b9235fb9c758aa82",
       "IPY_MODEL_13a52a32bbb040309a1c777f87f29bb1"
      ],
      "layout": "IPY_MODEL_76d7f0a0ba2f444eacaa6ac45eadeb8b"
     }
    },
    "e1df1eda8fd946388b80cf7a753bb8d2": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "e1f1bd459a3e4fc29763bac02058bcc4": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "ProgressStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": "initial"
     }
    },
    "e36ae6801f864a11a5838cf9ed4ba353": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "e40a6afc8cd3425199bb0787f4b10f6d": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "ProgressStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": "initial"
     }
    },
    "e42fea6d80ee41fa87eb05bf6619a171": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "e58007a175724a93826f54ad010eb3e0": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "FloatProgressModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "success",
      "description": "100%",
      "description_tooltip": null,
      "layout": "IPY_MODEL_3948b71f1b314f68b2d11ca915553e30",
      "max": 1000,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_e1f1bd459a3e4fc29763bac02058bcc4",
      "value": 1000
     }
    },
    "e6e7fc5c6cc34762855f85d90369a60f": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "FloatProgressModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "success",
      "description": "100%",
      "description_tooltip": null,
      "layout": "IPY_MODEL_2e7ec1c896904a79ae2924a0c072c9f4",
      "max": 1000,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_ae1ad1e6d4414093bbaee30ef5f7c64c",
      "value": 1000
     }
    },
    "eb69ae54b2794c698af7b7399c566aee": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "eb8c2ad4b54e4322af6eaf9acb2146f9": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "HBoxModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_1a44ba624b804903934006950be10946",
       "IPY_MODEL_77fa3907e9f94a0ebe7dcae9fc452048"
      ],
      "layout": "IPY_MODEL_c82d275680734e5289db19cb909ee021"
     }
    },
    "eb9b6424c9f045b19a65d349c8004828": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "ProgressStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": "initial"
     }
    },
    "ec1f39d1218246db9625b116bf0046b5": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "ed76467bae354c5197fc851cfee38704": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "ed916bc1bd1242d3a68568625c73417d": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_2eacf0a522cb42929f123e2b46b24aa3",
      "placeholder": "​",
      "style": "IPY_MODEL_2eeda4cb83aa44ea916d98091e5f2bae",
      "value": " 250/250 [32:03&lt;00:00,  7.69s/it]"
     }
    },
    "edd5097c68f34aceb7b864e24ccccf4d": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "FloatProgressModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "success",
      "description": "100%",
      "description_tooltip": null,
      "layout": "IPY_MODEL_26a0a00f29d749c583b9bb3daf0068f4",
      "max": 1000,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_29b12965e6734707b300c42529f1c434",
      "value": 1000
     }
    },
    "ee3e8f88a4ef45979d7ca5e30ad9c125": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "ProgressStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": "initial"
     }
    },
    "ef6fc0069caa408681ed24e43b075c31": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "FloatProgressModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "success",
      "description": "100%",
      "description_tooltip": null,
      "layout": "IPY_MODEL_87daeb0bd0754c75b5661f7840233ad0",
      "max": 250,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_631b21b11c5a45879c4f5070dc0feaf1",
      "value": 250
     }
    },
    "ef88874f5bc346c393e6bbb6e18562f7": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "FloatProgressModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "success",
      "description": "100%",
      "description_tooltip": null,
      "layout": "IPY_MODEL_5396e1fd60504686b6c9bc64fb4a4d06",
      "max": 1000,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_21bc74bd84ad4daba065f1772ed662fd",
      "value": 1000
     }
    },
    "efeac8374c38426f906da594f014959d": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "ProgressStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": "initial"
     }
    },
    "f36b7603103b41afa037b1736ce7987c": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "FloatProgressModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "success",
      "description": "100%",
      "description_tooltip": null,
      "layout": "IPY_MODEL_e1df1eda8fd946388b80cf7a753bb8d2",
      "max": 250,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_ca9714a20e7b48939748a81acd7815d7",
      "value": 250
     }
    },
    "f5263bdc9a1143289e04950c75cb57f7": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "f5909acd99cc4cc984d2b5c8433ef031": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "f85725cd811c41049ca3b4a1a1d8d52b": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "HBoxModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_ef88874f5bc346c393e6bbb6e18562f7",
       "IPY_MODEL_2b720265187e48e1a35107e338a9a999"
      ],
      "layout": "IPY_MODEL_fae25072555940479f91fe5fe7027ef5"
     }
    },
    "fae25072555940479f91fe5fe7027ef5": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "fc62d2b357614a41a04a474d45311b2e": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "fd2bd8c066944b2bb3681179d1e773d0": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_ba293c1ad08b435db5558ddf5f82baaa",
      "placeholder": "​",
      "style": "IPY_MODEL_8953c30291cf4f699814869d4b5a09c3",
      "value": " 250/250 [22:18&lt;00:00,  5.35s/it]"
     }
    },
    "ffce7755cefa4c688d4eab7cd9876674": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    }
   }
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}
