From 9bf542d98820f54aa117fcc76b6971b318ccd4a6 Mon Sep 17 00:00:00 2001 From: ayasyrev Date: Wed, 12 Jan 2022 11:40:22 +0300 Subject: [PATCH 1/5] fix mixup cutmix args --- nbs/01b_training.ipynb | 11 +++++---- nbs/05a_mixup_cutmix.ipynb | 47 +++++++++++++++----------------------- 2 files changed, 26 insertions(+), 32 deletions(-) diff --git a/nbs/01b_training.ipynb b/nbs/01b_training.ipynb index 9f0ad17..c91463a 100644 --- a/nbs/01b_training.ipynb +++ b/nbs/01b_training.ipynb @@ -385,16 +385,19 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "To enable either mixup or cutmix, simply add the `--mixup` or `--cutmix` flag with probability of applying the augmentation. \n", + "To enable either mixup or cutmix, simply add the `--mixup` or `--cutmix` flag with alpha value. \n", + "Default probability of applying the augmentation is 1.0. If you need to change it, use `--mixup-prob` argument with new value.\n", "\n", "For example, to enable mixup, \n", "```python \n", "train.py ../imagenette2-320 --mixup 0.5\n", + "train.py ../imagenette2-320 --mixup 0.5 --mixup-prob 0.7\n", "```\n", "\n", "Or for Cutmix, \n", "```python \n", "train.py ../imagenette2-320 --cutmix 0.5\n", + "train.py ../imagenette2-320 --cutmix 0.5 --mixup-prob 0.7\n", "```" ] }, @@ -408,7 +411,7 @@ "python train.py ../imagenette2-320 --mixup 0.5 --cutmix 0.5 --mixup-switch-prob 0.3\n", "```\n", "\n", - "The above command will use either Mixup or Cutmix as data augmentation techniques and apply it to the batch with 50% probability. It will also switch between the two with 30% probability." + "The above command will use either Mixup or Cutmix as data augmentation techniques and apply it to the batch with 50% probability. It will also switch between the two with 30% probability (Mixup - 70%, 30% switch to Cutmix)." ] }, { @@ -427,7 +430,7 @@ ], "metadata": { "kernelspec": { - "display_name": "Python 3", + "display_name": "Python 3 (ipykernel)", "language": "python", "name": "python3" }, @@ -441,7 +444,7 @@ "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", - "version": "3.8.5" + "version": "3.9.6" }, "toc": { "base_numbering": 1, diff --git a/nbs/05a_mixup_cutmix.ipynb b/nbs/05a_mixup_cutmix.ipynb index 5dc970b..24d400a 100644 --- a/nbs/05a_mixup_cutmix.ipynb +++ b/nbs/05a_mixup_cutmix.ipynb @@ -70,64 +70,55 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "To train a network with only mixup enabled, simply pass in the `--mixup` argument with a probability of augmentation. \n", + "To train a network with only mixup enabled, simply pass in the `--mixup` argument with value of Mixup alpha. \n", + "Default probability of augmentation is 1.0, if you need to change it, use `--mixup-prob` argument with new value. \n", "\n", "```python\n", - "python train.py ../imagenette2-320 --mixup 0.7\n", + "python train.py ../imagenette2-320 --mixup 0.5\n", + "python train.py ../imagenette2-320 --mixup 0.5 --mixup-prob 0.7\n", "```" ] }, { "cell_type": "markdown", - "metadata": { - "heading_collapsed": true - }, + "metadata": {}, "source": [ "### Only CutMix" ] }, { "cell_type": "markdown", - "metadata": { - "hidden": true - }, + "metadata": {}, "source": [ - "To train a network only CutMix enabled, simply pass in the `--cutmix` argument with a probability of augmentation. \n", - "\n", + "To train a network only CutMix enabled, simply pass in the `--cutmix` argument with with value of Cutmix alpha. \n", + "Default probability of augmentation is 1.0, if you need to change it, use `--mixup-prob` argument with new value. \n", "```python\n", "python train.py ../imagenette2-320 --cutmix 0.2\n", + "python train.py ../imagenette2-320 --cutmix 0.2 --mixup-prob 0.7\n", "```" ] }, { "cell_type": "markdown", - "metadata": { - "heading_collapsed": true - }, + "metadata": {}, "source": [ "### Both Mixup and Cutmix" ] }, { "cell_type": "markdown", - "metadata": { - "hidden": true - }, + "metadata": {}, "source": [ "To train a nueral network with both enabled, \n", "\n", "```python\n", "python train.py ../imagenette2-320 --cutmix 0.4 --mixup 0.5\n", - "```" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "hidden": true - }, - "source": [ - "When both Mixup and CutMix are enabled, then for each batch, `timm` either " + "``` \n", + "Default probability of switching betwin mixup and cutmix is 0.5. \n", + "To change it use `--mixup-switch-prob` argument. It is probability to switch to cutmix.\n", + "```python\n", + "python train.py ../imagenette2-320 --cutmix 0.4 --mixup 0.5 --mixup-switch-prob 0.4\n", + "``` " ] }, { @@ -526,7 +517,7 @@ ], "metadata": { "kernelspec": { - "display_name": "Python 3", + "display_name": "Python 3 (ipykernel)", "language": "python", "name": "python3" }, @@ -540,7 +531,7 @@ "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", - "version": "3.8.5" + "version": "3.9.6" }, "toc": { "base_numbering": 1, From b4869af9e5a039588b107d07dae1fc9e0453ddea Mon Sep 17 00:00:00 2001 From: ayasyrev Date: Wed, 12 Jan 2022 11:45:15 +0300 Subject: [PATCH 2/5] mixup, cutmix docs update --- README.md | 23 +++++------------------ docs/dataset.html | 9 ++++----- docs/mixup_cutmix.html | 32 ++++++++++++++++---------------- docs/training.html | 23 ++++++++++++----------- 4 files changed, 37 insertions(+), 50 deletions(-) diff --git a/README.md b/README.md index 4459edf..d037223 100644 --- a/README.md +++ b/README.md @@ -1,16 +1,3 @@ - - -**Table of Contents** - -- [Pytorch Image Models (timm)](#pytorch-image-models-timm) - - [Install](#install) - - [How to use](#how-to-use) - - [Create a model](#create-a-model) - - [List Models with Pretrained Weights](#list-models-with-pretrained-weights) - - [Search for model architectures by Wildcard](#search-for-model-architectures-by-wildcard) - - - # Pytorch Image Models (timm) > `timm` is a deep-learning library created by Ross Wightman and is a collection of SOTA computer vision models, layers, utilities, optimizers, schedulers, data-loaders, augmentations and also training/validating scripts with ability to reproduce ImageNet training results. @@ -32,7 +19,7 @@ cd pytorch-image-models && pip install -e . ### Create a model -```python +``` import timm import torch @@ -45,7 +32,7 @@ It is that simple to create a model using `timm`. The `create_model` function is To create a pretrained model, simply pass in `pretrained=True`. -```python +``` pretrained_resnet_34 = timm.create_model('resnet34', pretrained=True) ``` @@ -54,7 +41,7 @@ pretrained_resnet_34 = timm.create_model('resnet34', pretrained=True) To create a model with a custom number of classes, simply pass in `num_classes=`. -```python +``` import timm import torch @@ -75,7 +62,7 @@ model(x).shape `timm.list_models()` returns a complete list of available models in `timm`. To have a look at a complete list of pretrained models, pass in `pretrained=True` in `list_models`. -```python +``` avail_pretrained_models = timm.list_models(pretrained=True) len(avail_pretrained_models), avail_pretrained_models[:5] ``` @@ -98,7 +85,7 @@ There are a total of **271** models with pretrained weights currently available It is also possible to search for model architectures using Wildcard as below: -```python +``` all_densenet_models = timm.list_models('*densenet*') all_densenet_models ``` diff --git a/docs/dataset.html b/docs/dataset.html index c7ed1a7..1ef0a27 100644 --- a/docs/dataset.html +++ b/docs/dataset.html @@ -66,15 +66,14 @@

Parser

The parser is set automatically using a create_parser factory method. The parser finds all images and targets in root where the root folder is structured like so:

- -
markdown
-root/dog/xxx.png
+
root/dog/xxx.png
 root/dog/xxy.png
 root/dog/xxz.png
 
 root/cat/123.png
 root/cat/nsdf3.png
-root/cat/asd932_.png
+root/cat/asd932_.png +
@@ -297,7 +296,7 @@

Usage

<ipython-input-14-9085b17eda0c> in <module> ----> 1 iterable_dataset[0] -~/opt/anaconda3/lib/python3.8/site-packages/torch/utils/data/dataset.py in __getitem__(self, index) +~/opt/anaconda3/lib/python3.8/site-packages/torch/utils/data/dataset.py in __getitem__(self, index) 30 31 def __getitem__(self, index) -> T_co:---> 32 raise NotImplementedError 33 34 def __add__(self, other: 'Dataset[T_co]') -> 'ConcatDataset[T_co]': diff --git a/docs/mixup_cutmix.html b/docs/mixup_cutmix.html index 7488b2a..8872005 100644 --- a/docs/mixup_cutmix.html +++ b/docs/mixup_cutmix.html @@ -46,9 +46,7 @@

Training Neura

The various training arguments that are of interest when applying Mixup/CutMix data augmentations are:

- -
markdown
---mixup MIXUP         mixup alpha, mixup enabled if > 0. (default: 0.)
+
--mixup MIXUP         mixup alpha, mixup enabled if > 0. (default: 0.)
 --cutmix CUTMIX       cutmix alpha, cutmix enabled if > 0. (default: 0.)
 --cutmix-minmax CUTMIX_MINMAX [CUTMIX_MINMAX ...]
                     cutmix min/max ratio, overrides alpha and enables
@@ -60,9 +58,10 @@ 

Training Neura Probability of switching to cutmix when both mixup and cutmix enabled --mixup-mode MIXUP_MODE - How to apply mixup/cutmix params. Per "batch", "pair", - or "elem" ---mixup-off-epoch N Turn off mixup after this epoch, disabled if 0. (default: 0.)

+ How to apply mixup/cutmix params. Per "batch", "pair", + or "elem" +--mixup-off-epoch N Turn off mixup after this epoch, disabled if 0. (default: 0.) +
@@ -75,8 +74,10 @@

Only Mixup
-

To train a network with only mixup enabled, simply pass in the --mixup argument with a probability of augmentation.

-
python train.py ../imagenette2-320 --mixup 0.7
+

To train a network with only mixup enabled, simply pass in the --mixup argument with value of Mixup alpha.
+Default probability of augmentation is 1.0, if you need to change it, use --mixup-prob argument with new value.

+
python train.py ../imagenette2-320 --mixup 0.5
+python train.py ../imagenette2-320 --mixup 0.5 --mixup-prob 0.7
 
@@ -90,8 +91,10 @@

Only CutMix

-

To train a network only CutMix enabled, simply pass in the --cutmix argument with a probability of augmentation.

+

To train a network only CutMix enabled, simply pass in the --cutmix argument with with value of Cutmix alpha.
+Default probability of augmentation is 1.0, if you need to change it, use --mixup-prob argument with new value.

python train.py ../imagenette2-320 --cutmix 0.2
+python train.py ../imagenette2-320 --cutmix 0.2 --mixup-prob 0.7
 
@@ -108,13 +111,10 @@

Both Mixup and CutmixTo train a nueral network with both enabled,

python train.py ../imagenette2-320 --cutmix 0.4 --mixup 0.5
 
- -

-
-
-
-
-

When both Mixup and CutMix are enabled, then for each batch, timm either

+

Default probability of switching betwin mixup and cutmix is 0.5.
+To change it use --mixup-switch-prob argument. It is probability to switch to cutmix.

+
python train.py ../imagenette2-320 --cutmix 0.4 --mixup 0.5 --mixup-switch-prob 0.4
+
diff --git a/docs/training.html b/docs/training.html index 3fc01d7..9b0813f 100644 --- a/docs/training.html +++ b/docs/training.html @@ -220,7 +220,7 @@

Augmix

python train.py ./imagenette2-320 --aug-splits 3 --jsd
 

timm also supports augmix with RandAugment and AutoAugment like so:

-
python train.py ./imagenette2-320 --aug-splits 3 --jsd --aa rand-m9-mstd0.5-inc1
+
python train.py ./imagenette2-320 --aug-splits 3 --jsd --aa rand-m9-mstd0.5-inc1
 
@@ -266,11 +266,10 @@

Auxiliary Batch Norm/ SplitB

From the paper,

+
Batch normalization serves as an essential component for many state-of-the-art computer vision models. Specifically, BN normalizes input features by the mean and variance computed within each mini-batch. **One intrinsic assumption of utilizing BN is that the input features should come from a single or similar distributions.** This normalization behavior could be problematic if the mini-batch contains data from different distributions, there- fore resulting in inaccurate statistics estimation.
 
-
markdown
-Batch normalization serves as an essential component for many state-of-the-art computer vision models. Specifically, BN normalizes input features by the mean and variance computed within each mini-batch. **One intrinsic assumption of utilizing BN is that the input features should come from a single or similar distributions.** This normalization behavior could be problematic if the mini-batch contains data from different distributions, there- fore resulting in inaccurate statistics estimation.
-
-To disentangle this mixture distribution into two simpler ones respectively for the clean and adversarial images, we hereby propose an auxiliary BN to guarantee its normalization statistics are exclusively preformed on the adversarial examples.
+To disentangle this mixture distribution into two simpler ones respectively for the clean and adversarial images, we hereby propose an auxiliary BN to guarantee its normalization statistics are exclusively preformed on the adversarial examples. +
@@ -278,7 +277,7 @@

Auxiliary Batch Norm/ SplitB

To enable split batch norm,

-
python train.py ./imagenette2-320 --aug-splits 3 --aa rand-m9-mstd0.5-inc1 --split-bn
+
python train.py ./imagenette2-320 --aug-splits 3 --aa rand-m9-mstd0.5-inc1 --split-bn
 
@@ -300,9 +299,8 @@

Synchronized Batch Norm

Synchronized batch norm is only used when training on multiple GPUs. From papers with code:

- -
markdown
-Synchronized Batch Normalization (SyncBN) is a type of batch normalization used for multi-GPU training. Standard batch normalization only normalizes the data within each device (GPU). SyncBN normalizes the input within the whole mini-batch.
+
Synchronized Batch Normalization (SyncBN) is a type of batch normalization used for multi-GPU training. Standard batch normalization only normalizes the data within each device (GPU). SyncBN normalizes the input within the whole mini-batch.
+
@@ -324,12 +322,15 @@

Mixup and Cutmix -

The above command will use either Mixup or Cutmix as data augmentation techniques and apply it to the batch with 50% probability. It will also switch between the two with 30% probability.

+

The above command will use either Mixup or Cutmix as data augmentation techniques and apply it to the batch with 50% probability. It will also switch between the two with 30% probability (Mixup - 70%, 30% switch to Cutmix).

From a5afa5fbf528518f476cf503312eed662ca0b913 Mon Sep 17 00:00:00 2001 From: ayasyrev Date: Wed, 12 Jan 2022 09:00:50 +0000 Subject: [PATCH 3/5] docs: update TOC --- README.md | 13 +++++++++++++ 1 file changed, 13 insertions(+) diff --git a/README.md b/README.md index d037223..c5988ee 100644 --- a/README.md +++ b/README.md @@ -1,3 +1,16 @@ + + +**Table of Contents** + +- [Pytorch Image Models (timm)](#pytorch-image-models-timm) + - [Install](#install) + - [How to use](#how-to-use) + - [Create a model](#create-a-model) + - [List Models with Pretrained Weights](#list-models-with-pretrained-weights) + - [Search for model architectures by Wildcard](#search-for-model-architectures-by-wildcard) + + + # Pytorch Image Models (timm) > `timm` is a deep-learning library created by Ross Wightman and is a collection of SOTA computer vision models, layers, utilities, optimizers, schedulers, data-loaders, augmentations and also training/validating scripts with ability to reproduce ImageNet training results. From d791736fd12c8a18da7cbe2672b33bba6d9783d3 Mon Sep 17 00:00:00 2001 From: ayasyrev Date: Tue, 25 Jan 2022 14:56:12 +0300 Subject: [PATCH 4/5] PolyLR, MultistepLR --- README.md | 13 - docs/PolyLR.html | 1003 +++++++++++++++++++++++++ docs/RandAugment.html | 2 +- docs/SGDR.html | 3 +- docs/_data/sidebars/home_sidebar.yml | 6 + docs/dataloader.html | 7 +- docs/images/MultistepLr.png | Bin 0 -> 6421 bytes docs/images/PolyLR.png | Bin 0 -> 18542 bytes docs/jsd_cross_entropy.html | 2 +- docs/multistepLR.html | 557 ++++++++++++++ docs/plateau.html | 2 +- docs/schedulers.html | 90 ++- docs/sidebar.json | 4 +- docs/stepLR.html | 4 +- docs/tanh.html | 2 +- nbs/.last_checked | 0 nbs/07_schedulers.ipynb | 100 ++- nbs/07f_multistepLR.ipynb | 662 +++++++++++++++++ nbs/07g_PolyLR.ipynb | 1022 ++++++++++++++++++++++++++ nbs/images/MultistepLr.png | Bin 0 -> 6421 bytes nbs/images/PolyLR.png | Bin 0 -> 18542 bytes 21 files changed, 3428 insertions(+), 51 deletions(-) create mode 100644 docs/PolyLR.html create mode 100644 docs/images/MultistepLr.png create mode 100644 docs/images/PolyLR.png create mode 100644 docs/multistepLR.html create mode 100644 nbs/.last_checked create mode 100644 nbs/07f_multistepLR.ipynb create mode 100644 nbs/07g_PolyLR.ipynb create mode 100644 nbs/images/MultistepLr.png create mode 100644 nbs/images/PolyLR.png diff --git a/README.md b/README.md index c5988ee..d037223 100644 --- a/README.md +++ b/README.md @@ -1,16 +1,3 @@ - - -**Table of Contents** - -- [Pytorch Image Models (timm)](#pytorch-image-models-timm) - - [Install](#install) - - [How to use](#how-to-use) - - [Create a model](#create-a-model) - - [List Models with Pretrained Weights](#list-models-with-pretrained-weights) - - [Search for model architectures by Wildcard](#search-for-model-architectures-by-wildcard) - - - # Pytorch Image Models (timm) > `timm` is a deep-learning library created by Ross Wightman and is a collection of SOTA computer vision models, layers, utilities, optimizers, schedulers, data-loaders, augmentations and also training/validating scripts with ability to reproduce ImageNet training results. diff --git a/docs/PolyLR.html b/docs/PolyLR.html new file mode 100644 index 0000000..080b15c --- /dev/null +++ b/docs/PolyLR.html @@ -0,0 +1,1003 @@ +--- + +title: PolyLRScheduler + + +keywords: fastai +sidebar: home_sidebar + + + +nb_path: "nbs/07g_PolyLR.ipynb" +--- + + +
+ + {% raw %} + +
+ +
+ {% endraw %} + +
+
+

In this tutorial we are going to be looking at the PolyLRScheduler in the timm library.

+ +
+
+
+
+
+

PolyLRScheduler is very similar to CosineLRScheduler and TanhLRScheduler.
+Difference is PolyLRScheduler use Polynomial function to anneal learning rate.
+It is cyclic, can do warmup, add noise and k-decay.

+ +
+
+
+ {% raw %} + +
+ +
+
+ +
+ + +
+

class PolyLRScheduler[source]

PolyLRScheduler(optimizer:Optimizer, t_initial:int, power:float=0.5, lr_min:float=0.0, cycle_mul:float=1.0, cycle_decay:float=1.0, cycle_limit:int=1, warmup_t=0, warmup_lr_init=0, warmup_prefix=False, t_in_epochs=True, noise_range_t=None, noise_pct=0.67, noise_std=1.0, noise_seed=42, k_decay=1.0, initialize=True) :: Scheduler

+
+

Polynomial LR Scheduler w/ warmup, noise, and k-decay

+

k-decay option based on k-decay: A New Method For Learning Rate Schedule - https://arxiv.org/abs/2004.05909

+ +
+ +
+ +
+
+ +
+ {% endraw %} + +
+
+

The schedule looks something like:

+ +
+
+
+ {% raw %} + +
+ +
+
+ +
+ + + +
+ +
+ +
+ +
+
+ +
+ {% endraw %} + +
+
+

Using PolyLRScheduler scheduler with timm training script

+
+
+
+
+
+

To train models using the PolyLRScheduler we simply update the training script args passed by passing in --sched poly parameter alongside the necessary hyperparams. In this section we will also look at how each of the hyperparams update the PolyLRScheduler scheduler.

+ +
+
+
+
+
+

The training command to use PolyLRScheduler scheduler looks something like:

+
python train.py ../imagenette2-320/ --sched poly
+
+ +
+
+
+
+
+

Availible parameters are:

+

--epochs - initial number of epoch to train. default 300.
+--lr - learning rate (default: 0.05)
+--min-lr - lower lr bound for cyclic schedulers that hit 0 (1e-5)
+--lr-k-decay - 'learning rate k-decay for cosine/poly (default: 1.0)

+

--decay-rate - polynomial power, (default: 0.1)

+

cycle parameters:
+--lr-cycle-limit - learning rate cycle limit, cycles enabled if > 1
+--lr-cycle-decay - amount to decay each learning rate cycle (default: 0.5)
+--lr-cycle-mul - learning rate cycle len multiplier (default: 1.0)

+

warmup parameters:
+--warmup-lr' - warmup learning rate (default: 0.0001)
+--warmup-epochs - epochs to warmup LR, if scheduler supports (default: 3)

+

noise parameters:
+--lr-noise - learning rate noise on/off epoch percentages
+--lr-noise-pct - learning rate noise limit percent (default: 0.67)
+--seed - random seed (default: 42) to seed noise generator.

+ +
+
+
+
+
+

Note! PolyLRScheduler is cyclyc sheduler, so real number of train epoch will differ from --epochs number! \ +If we lunch script with default settings, it will train for 310 epochs - 300 defaupt for --epochs and 10 default for --cooldown-epochs. \ +If we lunch script with parameters:--epochs 50 --lr-cycle-limit 2 \ +It will be 110 epochs - two cyles by 50 epochs plus 10 for cooldown.

+
+ +
+
+
+
+
+

Using in python script.

+
+
+
+
+
+

PolyLRScheduler accepts two required arguments - an optimizer and t_initial, and also some hyperparams which we will look into in detail below.

+ +
+
+
+
+
+

Basic usage like this:

+ +
+
+
+
+
+
from timm.scheduler.poly_lr import PolyLRScheduler
+scheduler = PolyLRScheduler(optimizer, t_initial=num_epoch)
+
+ +
+
+
+
+
+

Required arguments.

+
+
+
+
+
+

optimizer is object of torch.optim.Optimizer
+t_initial - initial number of epochs to train. It will be different from t_initial when using cycle arguments, see detailed explanation and examples below.

+ +
+
+
+
+
+

Default schedule:

+ +
+
+
+ {% raw %} + +
+
+ +
+
+
scheduler = PolyLRScheduler(optimizer, t_initial=50)
+plot_lr(scheduler)
+
+ +
+
+
+ +
+
+ +
+ + + +
+ +
+ +
+ +
+
+ +
+ {% endraw %} + +
+
+

PolyLR specific Args

+
+
+
+
+
+

power

+
+
+
+
+
+

"Power" of polynomial function, default is 0.5.\ +Note, when you start training script, power sets by --decay-rate parameter, that default is 0.1\ +When power=1 annealing is linear.
+Lets look at default and compare with 1. and 2.

+ +
+
+
+ {% raw %} + +
+
+ +
+
+
scheduler = PolyLRScheduler(optimizer, t_initial=t_initial)
+plot_lr(scheduler, label='power=0.5, default')
+
+scheduler = PolyLRScheduler(optimizer, t_initial=t_initial, power=1)
+plot_lr(scheduler, label='power=1')
+
+scheduler = PolyLRScheduler(optimizer, t_initial=t_initial, power=2)
+plot_lr(scheduler, label='power=2')
+
+scheduler = PolyLRScheduler(optimizer, t_initial=t_initial, power=0.1)
+plot_lr(scheduler, label='power=0.1, default from train script')
+
+plt.legend();
+
+ +
+
+
+ +
+
+ +
+ + + +
+ +
+ +
+ +
+
+ +
+ {% endraw %} + +
+
+

lr_min

+
+
+
+
+
+

lr_min is value of lower lr bound, default is 0.

+ +
+
+
+ {% raw %} + +
+
+ +
+
+
scheduler = PolyLRScheduler(optimizer, t_initial=t_initial)
+plot_lr(scheduler, label='default')
+
+scheduler = PolyLRScheduler(optimizer, t_initial=t_initial, lr_min=0.01)
+plot_lr(scheduler, label='lr_min=0.01')
+plt.legend();
+
+ +
+
+
+ +
+
+ +
+ + + +
+ +
+ +
+ +
+
+ +
+ {% endraw %} + +
+
+

k_decay

+
+
+
+
+
+

k_decay k_decay rate.

+ +
+
+
+ {% raw %} + +
+
+ +
+
+
scheduler = PolyLRScheduler(optimizer, t_initial=t_initial)
+plot_lr(scheduler, label='default')
+
+scheduler = PolyLRScheduler(optimizer, t_initial=t_initial, k_decay=2.)
+plot_lr(scheduler, label='k_decay=2.')
+
+scheduler = PolyLRScheduler(optimizer, t_initial=t_initial, k_decay=.5)
+plot_lr(scheduler, label='k_decay=0.5')
+plt.legend();
+
+ +
+
+
+ +
+
+ +
+ + + +
+ +
+ +
+ +
+
+ +
+ {% endraw %} + +
+
+

Cycle Args.

+
+
+
+
+
+

cycle_limit

+
+
+
+
+
+

The number of cycles.
+Note, what full namber of epochs will be different with t_initial.

+ +
+
+
+
+
+

after which to decay the learning rate where the new learning rate value equals lr * decay_rate.

+ +
+
+
+ {% raw %} + +
+
+ +
+
+
t_initial = 50
+print(f"{t_initial=}")
+scheduler = PolyLRScheduler(optimizer, t_initial=t_initial, cycle_limit=2)
+plot_lr(scheduler)
+total_epochs = scheduler.get_cycle_length()
+print(f"{total_epochs=}")
+
+ +
+
+
+ +
+
+ +
+ +
+
t_initial=50
+total_epochs=100
+
+
+
+ +
+ + + +
+ +
+ +
+ +
+
+ +
+ {% endraw %} + + {% raw %} + +
+
+ +
+
+
scheduler = PolyLRScheduler(optimizer, t_initial=t_initial, cycle_limit=3)
+plot_lr(scheduler)
+total_epochs = scheduler.get_cycle_length()
+print(f"{t_initial=}")
+print(f"{total_epochs=}")
+
+ +
+
+
+ +
+
+ +
+ +
+
t_initial=50
+total_epochs=150
+
+
+
+ +
+ + + +
+ +
+ +
+ +
+
+ +
+ {% endraw %} + +
+
+

cycle_decay

+
+
+
+
+
+

When cycle_decay > 0 and <1., at every cycle the starting learning rate is decayed by new learning rate which equals lr * cycle_decay. So if cycle_decay=0.5, then in that case, the new learning rate becomes half the initial lr. +Default is 1., its mean no decay.

+ +
+
+
+ {% raw %} + +
+
+ +
+
+
t_initial = 50
+scheduler = PolyLRScheduler(optimizer, t_initial=t_initial, cycle_limit=3)
+plot_lr(scheduler, label='default, 1')
+scheduler = PolyLRScheduler(optimizer, t_initial=t_initial, cycle_limit=3, cycle_decay=0.5)
+plot_lr(scheduler, label="cycle_decay=0.5")
+plt.legend();
+
+ +
+
+
+ +
+
+ +
+ + + +
+ +
+ +
+ +
+
+ +
+ {% endraw %} + +
+
+

cycle_mul

+
+
+
+
+
+

cycle_mul is cycle len multiplier. So, if cycle_mul=2, next cycle will be twice longer.

+ +
+
+
+ {% raw %} + +
+
+ +
+
+
t_initial = 50
+print(f"{t_initial=}")
+scheduler = PolyLRScheduler(optimizer, t_initial=t_initial, cycle_limit=2, cycle_mul=2)
+total_epochs_2cycles = scheduler.get_cycle_length()
+print(f"{total_epochs_2cycles=}")
+
+scheduler = PolyLRScheduler(optimizer, t_initial=t_initial, cycle_limit=3, cycle_mul=2)
+total_epochs_3cycles = scheduler.get_cycle_length()
+print(f"{total_epochs_3cycles=}")
+
+ +
+
+
+ +
+
+ +
+ +
+
t_initial=50
+total_epochs_2cycles=150
+total_epochs_3cycles=350
+
+
+
+ +
+
+ +
+ {% endraw %} + + {% raw %} + +
+
+ +
+
+
num_epoch = 50
+cycle_limit=3
+scheduler = PolyLRScheduler(optimizer, t_initial=t_initial, cycle_limit=cycle_limit)
+plot_lr(scheduler, label='default, 1')
+scheduler = PolyLRScheduler(optimizer, t_initial=t_initial, cycle_limit=cycle_limit, cycle_mul=1.5)
+plot_lr(scheduler, label="cycle_mul=1.5")
+scheduler = PolyLRScheduler(optimizer, t_initial=t_initial, cycle_limit=cycle_limit, cycle_mul=2)
+plot_lr(scheduler, label="cycle_mul=2")
+plt.legend();
+
+ +
+
+
+ +
+
+ +
+ + + +
+ +
+ +
+ +
+
+ +
+ {% endraw %} + +
+
+

Warmup Args.

+
+
+
+
+
+

warmup_t

+
+
+
+
+
+

Defines the number of warmup epochs.

+ +
+
+
+
+
+

warmup_lr_init

+
+
+
+
+
+

The initial learning rate during warmup. Default is 0.

+ +
+
+
+ {% raw %} + +
+
+ +
+
+
scheduler = PolyLRScheduler(optimizer, t_initial=t_initial)
+plot_lr(scheduler, label='default')
+
+scheduler = PolyLRScheduler(optimizer, t_initial=t_initial, warmup_t=2)
+plot_lr(scheduler, label='warmup, default warmup_lr_init')
+
+scheduler = PolyLRScheduler(optimizer, t_initial=t_initial, warmup_t=2, warmup_lr_init=0.05)
+plot_lr(scheduler, label='warmup, warmup_lr_init=0.05')
+plt.legend();
+
+ +
+
+
+ +
+
+ +
+ + + +
+ +
+ +
+ +
+
+ +
+ {% endraw %} + +
+
+

As we can see by setting up warmup_t and warmup_lr_init, the scheduler first starts with a value of warmup_lr_init, then during warmup_t number of epochs gradually progresses up to the LR value at epoch warmup_t + 1.

+ +
+
+
+
+
+

warmup_prefix

+
+
+
+
+
+

If warmup_prefix is True, after warmup annealing starts from initial LR value.

+ +
+
+
+ {% raw %} + +
+
+ +
+
+
scheduler = PolyLRScheduler(optimizer, t_initial=t_initial)
+plot_lr(scheduler, label='no warmup')
+
+scheduler = PolyLRScheduler(optimizer, t_initial=t_initial, warmup_t=10, warmup_prefix=True)
+plot_lr(scheduler, label='warmup_prefix=True')
+
+scheduler = PolyLRScheduler(optimizer, t_initial=t_initial, warmup_t=10)
+plot_lr(scheduler, label='warmup_prefix=False')
+plt.legend();
+
+ +
+
+
+ +
+
+ +
+ + + +
+ +
+ +
+ +
+
+ +
+ {% endraw %} + +
+
+

Noise Args.

+
+
+
+
+
+

noise_range_t

+
+
+
+
+
+

If it is number - its number of epoch when noise starts.
+If list or tuple (of two elements) - first and second element is epoch number range, when noise applied.

+ +
+
+
+
+
+

The upper and lower limit of noise.

+ +
+
+
+
+
+

noise_pct

+
+
+
+
+
+

Percentage of noise to add.

+ +
+
+
+ {% raw %} + +
+
+ +
+
+
scheduler = PolyLRScheduler(optimizer, t_initial=100, noise_range_t=60)
+plot_noisy_lr(scheduler, label='noise_pct=0.65, def')
+
+scheduler = PolyLRScheduler(optimizer, t_initial=100, noise_range_t=[10, 40], noise_pct=0.2)
+plot_noisy_lr(scheduler, label='noise_pct=0.2')
+plt.legend();
+
+ +
+
+
+ +
+
+ +
+ + + +
+ +
+ +
+ +
+
+ +
+ {% endraw %} + +
+
+

noise_std

+
+
+
+
+
+

Noise standard deviation. Now it is not implemented.

+ +
+
+
+
+
+

noise_seed

+
+
+
+
+
+

Seed to use to add random noise.

+ +
+
+
+
+
+

Miscellaneous.

+
+
+
+
+
+

t_in_epochs

+
+
+
+
+
+

If set to False, the learning rates returned for epoch t are None.

+ +
+
+
+ {% raw %} + +
+
+ +
+
+
scheduler = PolyLRScheduler(optimizer, t_initial=5, t_in_epochs=False)
+lr_per_epoch = calculate_lr(scheduler)
+
+lr_per_epoch[:5]
+
+ +
+
+
+ +
+
+ +
+ + + +
+
[None, None, None, None, None]
+
+ +
+ +
+
+ +
+ {% endraw %} + +
+
+

initialize

+
+
+
+
+
+

If True, then inside each param group of the optimizer a new field is set called initial_{field_name} where field_name refers to the field in param group that we are scheduling. Typically field_name='lr'.

+ +
+
+
+
+ + diff --git a/docs/RandAugment.html b/docs/RandAugment.html index bd4a3bb..31f891f 100644 --- a/docs/RandAugment.html +++ b/docs/RandAugment.html @@ -47,7 +47,7 @@

Training

To train your models using randaugment, simply pass the --aa argument to the training script with a value. Something like:

-
python train.py ../imagenette2-320 --aa rand-m9-mstd0.5
+
python train.py ../imagenette2-320 --aa rand-m9-mstd0.5
 
diff --git a/docs/SGDR.html b/docs/SGDR.html index e5263a6..44326c7 100644 --- a/docs/SGDR.html +++ b/docs/SGDR.html @@ -79,12 +79,13 @@
-

class CosineLRScheduler[source]

CosineLRScheduler(optimizer:Optimizer, t_initial:int, t_mul:float=1.0, lr_min:float=0.0, decay_rate:float=1.0, warmup_t=0, warmup_lr_init=0, warmup_prefix=False, cycle_limit=0, t_in_epochs=True, noise_range_t=None, noise_pct=0.67, noise_std=1.0, noise_seed=42, initialize=True) :: Scheduler

+

class CosineLRScheduler[source]

CosineLRScheduler(optimizer:Optimizer, t_initial:int, lr_min:float=0.0, cycle_mul:float=1.0, cycle_decay:float=1.0, cycle_limit:int=1, warmup_t=0, warmup_lr_init=0, warmup_prefix=False, t_in_epochs=True, noise_range_t=None, noise_pct=0.67, noise_std=1.0, noise_seed=42, k_decay=1.0, initialize=True) :: Scheduler

Cosine decay with restarts. This is described in the paper https://arxiv.org/abs/1608.03983.

Inspiration from https://github.com/allenai/allennlp/blob/master/allennlp/training/learning_rate_schedulers/cosine.py

+

k-decay option based on k-decay: A New Method For Learning Rate Schedule - https://arxiv.org/abs/2004.05909

diff --git a/docs/_data/sidebars/home_sidebar.yml b/docs/_data/sidebars/home_sidebar.yml index 3ae52a6..529151a 100644 --- a/docs/_data/sidebars/home_sidebar.yml +++ b/docs/_data/sidebars/home_sidebar.yml @@ -96,6 +96,12 @@ entries: - output: web,pdf title: Plateau url: /plateau + - output: web,pdf + title: Multistep + url: /multistepLR + - output: web,pdf + title: PolyLR + url: /PolyLR title: Schedulers - output: web subfolderitems: diff --git a/docs/dataloader.html b/docs/dataloader.html index 1b3109a..ce3da21 100644 --- a/docs/dataloader.html +++ b/docs/dataloader.html @@ -322,14 +322,13 @@

Prefetch loader

From the documentation available on PyTorch:

- -
markdown
-A CUDA stream is a linear sequence of execution that belongs to a specific device. You normally do not need to create one explicitly: by default, each device uses its own “default” stream.
+
A CUDA stream is a linear sequence of execution that belongs to a specific device. You normally do not need to create one explicitly: by default, each device uses its own “default” stream.
 
 Operations inside each stream are serialized in the order they are created, but operations from different streams can execute concurrently in any relative order, unless explicit synchronization functions (such as synchronize() or wait_stream()) are used.
 
 
-When the “current stream” is the default stream, PyTorch automatically performs necessary synchronization when data is moved around. However, when using non-default streams, it is the user’s responsibility to ensure proper synchronization.
+When the “current stream” is the default stream, PyTorch automatically performs necessary synchronization when data is moved around. However, when using non-default streams, it is the user’s responsibility to ensure proper synchronization. +

diff --git a/docs/images/MultistepLr.png b/docs/images/MultistepLr.png new file mode 100644 index 0000000000000000000000000000000000000000..6f7eb01e563a9e8064f326b602f12deb731716fd GIT binary patch literal 6421 zcmai3cU)81woU*+L13aHD$NN*M`=pX(2+5MA%YT+-h?3tA^`#@p#^a;il7GRbtp=c zA|O%%8AVVjfdCN%LPROjODGfCJ4ffvoBQtE-}@tE?S1xMXJxN%eQT}F-AkrMqQZxS zArOct<~$k)fj~)sytY#i_&ZXnyb642`k%A*Hz(lzZ(Q+ngP2_L_wge5dwIAV4s`P) zdJw#kDyLLVDjmM+@9#s@QdPbF*8&xSpSx<`l|>q$WS7r*8zKZEvKRb7At`AGAdmyM zFlYmdn`tvc;bj)%p~DNSzIQf47*zv6AbDh7VT-I`kf@%Qy9p+hu&% zH0r{s-!jn1KQxbdYxn&Y_pZ~U*#8jT^9kJTabMP>IId~1dG;>v5f9{JMZ0_V-6j>4 zgbobYLk;oAvku9YGxxgL4O+LRS`u8N^sP9}?MG`Y24& z8IPxpW}%R+nnJW>n-0y1_BOu`0|GvQ_$ExUb9%OJF7|Sf<#OF_WJSaxxBFt_K3J7y zi<%_2$nu$sr)S}?TT-&D+%{ESxA|qKXE_O`UG3Aygx+SyT4shi+`@bm^y0U!@pDVv z=3Wb39c|rss%mFF$!h6Q97J{1SN@z5)q`G&eVEk@mh{w^SNS zZCo$6n3}x9*y{)H@K|j@|3R|o%RrUp3_eTsY1NA#KX$dd-@)bCi?2po zm8{n6LAFK*-rCw(X36x#Ae@mWy2cvAe^|@Utv)JwML(soUn)OA7-C zh26$vvJ`yK7x>~#u4^B9@!(Qsg2!?{jm}o_ls?}3+=z=$P#Hd$dJ<*CwbIQ-RrqU6 zAo1Sl@P&KeBQc@SW(z_`m-a-vl8hMde!-x9z~=%fB=%0Eo@*g-Y9jO`s|`1j#Ypn= zy$$;vp<1e5@fgrxxThD_B~L}>Z0$Y&f$kXRAi`&sjcfM`# zJnD~vbi;gg;3a(P5erZBKGPe1S;ug~Fxgvge0Cm&oAM9yH{P0JVT9Nd+5m(ovJtD;!T z!9LSnINPVT$&|jmFjv?~%c5Q_jhK*cc<3?hl!LO2ZIufIyabP=@Gd~?;$}?fzfpE# zBrumD7&$0je&pJ*-VD2F9RLvf+O-4r!YZh;P%*w8&IGh{eC6Jxdz4+AlN?`6H{E8^ zNf3gKh@^U7l3U&mQ8k4VjuTrIzxlS4B^~+~@>Kc7tJVgi>EZ!aBl86baij5$fP0V4 zr>yL@wN^SDgeKs>`gFQ?zNO?V{d5x;6#yfPJbNyHVce%IMt-5w(X04l030mxEVzrt zNbIkXdO3xow+KmU6&3*I-h+6H{ywMLoMLkvK-uME60~~YNWuM{@4y8BaC45kF#r^5 z0ne-Q?Z13n<`g>p;Ei|i>vZpVw%SaxAmE3<*YrL(e)=1l3rCD3Qf-WZ^`qdyzcI)( zLyb>>(W0U`fh@FL0mH1zvJ?C9r>{x59+&|;X;rM&Luq{_kIT%4GfX`U14D9T9yKq~ zuLisvW>KCTaP9w+Uhn}8N!T^AQG^3#{p=Y{IGev-ZY`1M#J4i$9Wx=r#k0}_4Y^X} zuM`bgm{r6k{ib=Qud~zhO$DDu1t`=4ArVmxwTehIFVGqHZtn^Pf~&UMv{RMeRDC-0 z!bjVcyA&IaPT6hJgp)vn=!VR^9DKxUN&I1@3G;DoL2`2PfZko5!pES?{kc4sX+e(G z{3hL#gHoUs)tIBF?^4Ps2a0+@#e7wL{*1@r^Zd3a4!lE^OUZ)~EwQJNZe? zUJd>YrFnh%5nFVc;1=ox>w4GgY}?~Jm*V7rXX#n`e8usVwzyq=yDz9OCqrHw8Av6c zg~58BUenusxIlPNBt$pn)g6O8`+4A0Yflq8j}m7eK!hU3BaXsH@^jOvvKMM~1(hDD zi*9J0s1_ii%FZ**CY=t1AAp6Yu3)~zgIp8kqMZ(R*^}huInY%Wn#{w!A7eON?e(51 zNQed*=tGFRSJY+W9i_D=D@p(#7TX>t$B(iIAgx{S@pK~>Cx&v7a@ zdt<&o@5Mb*tUx8vW+iW@+KdCB2wXCIlhJUpLInV!*R0u_#1I2kTP-JxQD&zq2Zo&1 zI3U+Z-VO#(Jp`cII&T`lQ2+yY8EDKQK!>GrFsL7#LRk&1JvoM)x)rJS+M9-A0jQly z1LUy;E7G3|`fk12nKwcK?TmkM%aLh_X;Co2eq@i_v|&bkS-XNMqOLr^lAe5Gh+?L? zb$<;hN)UDoaK$?$gFKhtxV{W^mMJ^OtbO7w&OZ)!5uN|3Oyn?pO9Rd_Y9evQBM62~ z$glC4;qkbxsfqSt#hLpP|Ap>6Co2$t6c0reg0$%1d2}KXdi2oB#Ut01*U?Q%KC;%# zo^f|FqnZb$zQCkV)lVUj-KIllb8yFb%I1Li)n8nSbA!oP*g~&mi@eK4^!;;9*3T@1 z6^#!a==J-ZkgQfwQn{D=lq?~--WvBJAyh#kLOTK}+JagV%n1r6E^Cs2giuVJ5!i$| zKg;c<8uy$H&MsgGl%48l3rOFiKpv{|ZK!8t&YchKjR8-rW+X%g?qbQ5ryFEJrei2M z)IdIS*h#34ppJ^iAr0(it8>R4@?L=3Hy4*maVn&syyO5kc9sx@f^mLcaNX$jiW}!Q z)Y|km-|S3%(u z0@<{k=JsCvG_t}JKvl^k$Sa9}`%$a;E~R;>Z?m&wc2B@46a!Ywj#1dxzP1;i1#)lQ zGTR#bsYpOxk_Pb8YLMw$Z8@az=UiP)EL+m``&vZ_!=GZ`nxpWnlfXMM+206EQku)3 zWxLX9)K*Ls<3Twf245TKL-xmS-tqx476!2}J~w>A4KG9N6eoMud|OfcEGx0njCm5% zO?^?V;h5lmSToF;Ss3Ly7tcjeV~t6s%vbxSL zNU>?`i%fypLI(+8h7;Cy~Jtbgq5)|e@* z{3yH`?ow=7wEQynck^SCcjrZ30f!9W>6r7m*ISFZ&)hP9aojTH>|XBLb~0+h!hCln z354YiWZyAa9?Q80FmvEKV30iB7E^?XC%4nl=9#;kuRW*j(%82g6ctGiR=7-6C5Q8c zD5o>;80D$2Y8f~^JXJAZl~#Hd-DXU>O}Z*crontIn5NFh;YY|V0W zBE??%3K~3ZcVKYl&w6f(5*c>J;{6T~hw=c1%)VD~S|eF^2=yEo&^=!BxFNX7eIwma91<(KFpOKgw8lP*0(2d*pgK_MGzWV<7akkoM5D8hQyVPKO3B+QE zfW|KAN9r(GjOrg@-_oqYb-_TWew=jhk8svgyOsHZ$75_M?N*0gTxQY93Ik?`w(B>( zlRm=?bC$lko9D~vezy`wmphNU7a&HfF_PGxjBxjkY8ZVz@d{oSHc7j$V&6H3Q+oOr zH0ip89-z`gqzfe|v{+(!!t-kL1o;tcyZ#yGX=f?=`*=mdlt9!Qu!E4>r0;R01*VHM zoIQ zi3zg7h_vo+jA3k!(10EXxALlO$Luz((Pb=N5g;PJ;`He9|I45-Uj)1Zf^O=ji_pab z?J8{?dkkA4Vx*~pe{V{9-1+)aya;?^=g{X15t775Toq?!ADwh&yWsHEn)5Xpz`KNK z%N_ORsCQ}W|D9AJoU?RO%m=a*9mwNes-{g|KVY&?gS8W>@R=si+Xx&<{TQh3NU zxlp4DahiB;SYc1>v@89i!@5GxPRn+TtC+a%5H*?<$u4H-WLG;9J5!f!u6O<1;ZWOr zS-<5ismXxAawlSy81~$JHt97ua>Np7V_B#vBw%9-xKWzRx+X@d(5nkJ+br*XD#K;T zgF{705v39(pynVo8xRtpRp-EspR-V{_Ho9Q^=+KGMjJ{-5XIynh zH$szQDW74!5`E(LQOgH~RuBUh`Bt%e$hn!2hF#Joc`g;whYrGStj}6cJqo-PcF-sL z%Jwfz*7J+#Hoox)1a&`jTX<~-wf?t+?L7mSQ|J23iWGWsrJJ4u^ZnGWb*?Fcav+*g zZu^Y&ncDXJ-HjI8(tQIH#D5ioz2mvLaG#?-f-i;o6uu7gM@L9eRg9YC{Ow-au$5$# zmj(-#0I>Sy|5+2Jkg^DZVK2bcEwLu$yw~Ta$yLAY&oyFHJyCm~x2aW@Qp&`!R}v+| zJB7h+JVAcRbV%VJibKOt)38Hp0h|EriT3ICUc=VGof^Rlo{iY@G!J+5#lLqX?MsA^ z+DKkol;QQRXH{I{)MPu8qwU(R_6qzpKxX}$U_hXa0IsDN7rf2(d(>UJwksh)MTlk- z1ypNOKnU1G^=#BjO+mj+1)*3X0IYT|^u#RRiexLD^~`Z|tkG#5y%vUfnb!h9?8lfJ zEh#`R0Z0wnEYk0wKr{&Fxe3v}Kga53SIhbU_Pjh}E390$Z)b=l&Tg*Yl$2#vmzq`SFlVFa1GpEhEzc9B`=g~FP>Seb&h-ThjWX5# zustuGsPGXSy6wn}2vppxZR``u#OC8)Gzn&KOj#z7z|YqjG0ixdoV}vIhsWh!r#?|I zX>V&A+_Fg;rDWc4^_cqOO&>)=8a5{wwr?#r_g7`!Up)y;<(*PDaQIlee{V)28~18- z_x%XZ5K@7JH+H+UM@2mKR=WPbwIUtw{P$Y%yq>whL#`?3PK2a+!JyL9S!?TD3u)g@ zJI^r(=>OLQ%L%-|)(Q?n!j8cmC*Ryn8vRuvfoEc#ko_V-%dkn|M1RkVoGa^U^OvF- zx?(zPw;24{r>AYlDkIrApmc0k%O4vbvc3oU@(UoOfPw{p_?3~%B@T_im4*_KmGKAF ztD6?vek}?uX z>_!{p3Ks}^tPLLy(9^W5O#l?|wsDQ_*|4-&1f&V}&vP$72Rz^d8f&&vq@iI`tlY)$ zU)*261L``w=J-Ee7Jj?s14uA=G%b?drZm0}wpsG<MdXcyc83rPP7u4-&lkJJF2b zmz6~J+&l{6+k5V4?VZm}KNL(^=)#vL=EB3^f*$l2Az-UgqA*rBO|>JkPTO4%7@P zH|EP+^=d`48S=7uA5Q`1%nM`wx{%)l$6N9ysF0ULE0(SN_7T;!25wuQZb)>3T44PCrrORb-gYQsUn zU1WSz=Ud-xHwWpjD$?)BkA%`m&N;ck5e@)jmJ4bK=?pgR37)hfk4vrG8LfwId7z64 z*80N>b0n2pvVD*XMokNpm%`IR&Fl8&z!jepzk6y*OGN0q6+O;T_WKmJFl=FlvgY@E zQ+RoME%DJ3@jbKM5FD8v|zQ+h21`fZVKOot~mm6>afUL-&&^=peX7lVs}f> z+0E7sRRvvT4?8AnVXoFmECUIHQT^)hmP|NDgkEac)Vgfo=-|lEg_rP8A(RET_N{%| zvg`?_hV9mnfnCJbmNG`UuZ84C9_QxK&8ZFSoAZI;AC}az^>-9s?E~(wM5E`7d3xk8 zbKRjOPmd}nXUQ&xOT)sc>f&!|_|Z+PWL3WHAFP!;yt5KohF8TQ;>M&$Jd5c&5!>gt z7*b@4M*K!-3NLY8uy)OXfd#L(j#z!cg2Bx=MT4gipdn3&DrW>&hcgPZSB6vXN}o}G4y3%?UG(X zDY@c&PJ9`kS!pwkPxLNk4iFQ$j+2sX;T|D8;AT_IT(%yQCbA|xPlWmAg~zX$ss${i z!WLX#`>@|>Zh1Vn6VQ`{HOhPJlG!R5KLXs;7gyC^7)na+mjcqmzakGHhyM)H*=_!9 Y2|44ps*0z9e^Ed%=SKL*n**_r@Fdo*WT6hO;KJF6O9xN1_lQ6i_|A&7#LVD;OAQuB;f0xO4b|T8^6nE zO&1k=GZ*)-PNpz&UtJt*>|Jauji}vBot!Q0?YP*u+1|5ITe!G5I18||+y0*m*zBFm z*%!W^LxDk19i+6JVPMelUO!=BQqzfHU?i8nd=gjnNI%MO_rO;p3R`bYlaQkR4oMY* z%Rxk;1zlDIxV-sJBZq*VyP(9fXKqg&K^+kgjt`57FOK-;A7l+gP3rf)8;&DU3If@=^bWlu127I&-XC-hY@B?KGt{3o! zQ-}BoX5hDBc@X^^MFt^qDkx@CLRA$92?0StBMDjvcvHPhr|~#Go>QpQHx%#w*Dr>j zKYw2Gd<3puK?C&$3LdHCko|v+)YG>|fEKd*Gci#@S{hBGQh#sykOG;Yu>6z^f7>ATJ^?)~*JVYPm zE0P5K4Q#cTJLuCPYY!HjZy=Q&e*?!Uk|LQD!Ws@3e<&dYEu=e7ktU1xcxQb)b3%^h zJ4?ZedF$^F+3;1u#sTq89ky9liN-o`{!)wz%hxZkb#R3{OH<@cUiTcO<~~xR=Sm1+ z5O6yfN1YOE+-c%=xs9(sRQ&dVuib@)K$>8Ho&(`LOnh<(N0&T4fDt1?7;J#c7yr*2 zEj7#zTZ=;Hd`=u39@r@spj;rm=((S^nAR4DBuW2O#7)N6H_K-U+i0;+v6>aXve1v3 zz+LXVyYiQ_A z&@uX}3Hjp{o37C2LYkrnBIlC;4C@o!~&`o_m&tBnT)wgpUw6HT}Jqiq&zLgM4`%|?|TEmL=1;_MYiR`1K)3=#lS5Zk1S`2)i%5v!94?^eLa8VI)*T zvna#jS~E;zy78(Do%K>v?rg|NXXQ<93xif_%V?OHI#7e-3@5ZLo&|I}Fr2d>Kh!oV zZ#IUspsX4u;yt05U*zR?F8@ut94`E5YU10UDel=BO|vI;-|R-r&CB~DBLlzjcz40; zJC0SmC~IUy6NW7)ckk4jIWel8osL68Dwp31D|{&Q0^V4a#j9csNOHC{W#dRe5VlS# z)as+bc(7!Cc`z01uoLMH%h(DpUPQOL4pJGc^1`rrgym5P(|*c1Q~XV}FFTE&8Nf|} zZWaXZb7wHXR2R)WCR58Zy_GiWq7e0WdoF=t;*{Uwl;uzduCyJ}u^z^~mHm$yXB~Tl zsgz`6)N^^k?{C}3aT1+e82h?XoB4&yOBcU~BVO%#t3EROHt}du?%TWFEUh+=_fYQ} zGhhXH`;m~5r}Mwy`8{1}2)G^ouCtupn<#*2ONG40}j^)hvR3XpC;fKr5!CycfL&cX_XsGn&f%mYX}_V%Pqe_VV}JSdOpV2rzWA$zmOTPc_i7 zHK=#7K2+dyBw>Na*`MCNKK|X%!%nt&urxnavj1?yJeRM2Bw^y<{(^m={e|I{-djkv zarDobSk$d6ew{nRE_p&5;UP)A5~X&Fs>Jt`*D6xvFzEVoemL$U!7&ElE*5*q%_B`# z%_jax=Fa(%;d18566yE)>CGwacym*dQ;vbr{1Pfhg}9`o8I zn4`XNQw&r1*;vIh^|C8)8!ArMjI22%C%#QQpO;_G@4!*4<`!xo$E6HSFt8Ye9cXBo z@aW61Sf~Q26w1OH4*sNtgoG$%iv=$CUmdTeQ3`wg-5WKhV_}K9S@+MnS@RzMdc8wf ziF1#!)Jw2?V@~R5Z=)ol-4rQrtX}nSi)dSfqfNY9ZMtNAeJJ^Xef?)e=_r~p=rj`@Zo20dE8gonN`_)n zjkUNQghxgi?@i>$rhZ@?b`$unut(R{;vrhm5mdR)#svO6OwJ%0z|TRZsE zym8r3_w_QPHzJx+ZHVn9eYy=!~_PHZl;+zcxAOCF-HZm9`sXkwmmGy z&c9{o3=>(e*xX)BYxWquP9)aFzkkd%{lh|qI_)jKM6kP9@f$A~8QTo7QQP&BwO86t z>o)a0*zTZ5k_$ySkpuM%UwZaQvl&rY&1ZJ?(j(R)FYG;Ds*23uGuF+$okN9Rryh_x zLxSdumeS9mr^6daLQB6erspVgXMY!zEhUv4U7iKIz1?ib{yJZI`UJo9!W*)A`*<-9 z#M(liyYv6tPq28@h8mpcCeb z`C>5Z*X4c4*s7*7L;S}Nhg4?oD8SOFn*`Y}CpfyY7fsjf)62Rc@+j8F!uKP+No*sJ zMPyU0c9!jW&+N7mPdz*FwM%0q894(kQmg2A%iVo(G4HiEuNZR1Avi(E)(J*k-z9nz z89F1mc)1+T1?2VG3b$u(GO$4fhSD0WZk4A3vhHtMniIrCsn+%tLk6Glnq5Z0MeG`< zi&@@MwVI$E$~}I(F-pjZK&NR^8M~?npY&cWb&Uh{G}6ph%>(I2=)pT6_-&6L4S*QP z??+~Z9vCy~)4fA99C_d03r<$)@}Y-)$d%J|lz41T1Ki{Mq*w+=jjBZoo2hF4nrcK5h1dU7QV2tv9UN@Tfq&TuW4o!L4pb$M=toxeyU#CVcdrUGXcA zx@sV5dd-O1@Dcs;yfza}J`YI5E<=mI`O;t;%Yc@>j;CkK_)L zH|WXX=J2N0WX0;J;H4?}`}}ZH1fZeA)z&1}gSoRc?-R#Ik7K`>-SI31DXH*}w!^yM zW{1u0C`#ca(5@QT9{E|sOuxMn7qBvDmFAz28*ScP(T_D-QeA-pPu+R8ecQrr#N>R$ ze`2vBTzzOr9K{AS_H~h_TIY<%4>pXEqpmsItyRWskIyA46$iGDSMvt9_xI-u#<8ml z>gC$pq#rCg(b?O?5}4F}0f23~NCA0cZ!+(L)y!UKQATwOHxpBYN|F4k1t+IrCk&HD zg={1tYm?JwZ}{a_clL7adKwXt%ur0S8t2_|3hQIp)fQI@^(D<&nNh_0VF>5TztA=j zbLOn)>a-4bA|sp9N|cuZ?2$v{)xr|oI?5Qj%Q5CpW}K+^v$2KxiyWDm4S`|wbd#m> zZ`XcZ5sLuWA?SzV4Qi3zdch+PevGZ-A%CRWETN!T&s@3Ax7gSY+Xr9~7Xj3k#$)?u zrdVaBL@iPxn)Iv(TXdqvR0fz?XrIyM`Sx%lMXUKZWubh=(7b`a+Ow(unm zJTjKa;X-vhvqsEpsm4VwoKvUB&&u%)$q9LMSThb5Y|rGwXt1HqWMqMMGH6m233w^x9&!%b~%x0gd z@;gs@>Ri^7cjOzpmNgq}NLE|j2LPX|vrr=CwHsNgw=PyICBzneBSHAa1CEiV6@j92Qd=WFUgd609!gNs#*ae?$z`SDl|^+|6k z$Lot#B2zEdG_TXk{<$pM+>cOZ3Gr&a@PnThsQ7l@P|+>fN+h>VjCW=msy{GMjX8_j zhnR^(5#JKwvu)uDrb}RDHrD>qZRX@MAFK2B$&-lIY;j@ic)n-$eYoCU zG|wtJ-3eE+T)5bvMc@=A7dX-uNv6l~dtEHSga$`IL9Q$>PxhsvnY^5OKvx zs?d?jnxd%Cb10_B1lO94Hn|;<0-&(9+V)~+6as;8_};s)5Tes7WSab&DtJdn=d{rY z8$rlA6aWkF@%M+MW``eCZ=O*x8)kBH^6@rL{ajaiuuv}TP=JSnR0Ksm5d*iy_gM{T) zc=$3I#q2MoU;zIE&YXJVeLct7H@2~|{h?nrV~`6u%S5t0HQvJr>oXd{zHMT6AUU!A zk6KVdF^@^29nT@3S7rg8!S19r-fiAx?T3F)r0r%mr;964*=i+4(+Jmm!)$hT;gx7s zHT~TCS#DX1I~xBJ^($#1R4&4NEFh^d5wMG$n2VZuJNS~GyhJ5+Bwrf55ecC|kDm}E z1|gtk7;J!3c!Vr7y3O(0cc0eXtXyUB?P9gGF(%31Wp!Xz$NDRH87hUQ`z0YuvTii8 zN?!5Ed{qSg8yCD8wk5OPC+u!gnfFPVSvVDw{G+)G4#4lb9F$_kXS>ntl-zg8?dk>BAG;|4;x3 z2!HJ+GXtLoyr^ld#HKYEn)g_rttLa;4{}LP^>6{0wHcB~uwn)$mjf#JkU0TALxJJ_ zoW1&EoZ+!^TlQ(vt!ypX`e zJTR;TK0!nK+~rHv8E9H{&%C6@+||~F$)HUg0MrMk|EOB4HPuF@_VTsLldGwt7%pMd zTE({ItNsc<%G>S&QO0MMIny6V(?`3*nspyXu|1ik_w+trQ7X0*uy~;>-zd`wg=b~{ z8i(h8OI&Y(?Mq*_d0BAarBnPHDf085;MrtK;Q$<<7At2QOQ-MrDKR|McWdeZPlL9d z_mck(!00Oc1p48)UK${TI*J@_<`hBVGWGMjl)EMwZV``crqUN-xn-r7spZ1CM4ryF9N28H53bEJ2|te7xyO2YRTMuK8ClD7}=d^y}L zD5u;P&+v{uw_@bhH(74pNMs%(K6mdT$z;K#_Ds+*+;gC04XU@?ul zJ&tzE+c=zSH;LLk$mU$+_Atp>24xRoyF@k7H%+X>7wXmD)lyc5#cG{yXq4W5y%OJ6 z^?(IT8jUk?%yo2fH@>dw%kgd^_RQDn%;myqL^lb%Tg|N6011?;MTK%W?a>8iJ2ahq z=47PZ|h&>AW}bveZK0)1iz^|FCJ)!-Oe5I#JG;7r>- zXrO~~ZI*pQaN;01IuIN|2oAg8g5S-%cetro!NMIq*ID*Urktp+mB`k^I|u43&Z7>^ z{&N$x^92;m)h~OnB|_OyVKy)HvDFpt55OetALjlPo4p^AlFm8MVCK)2cJW~0M}y*Y zV?-ex!y5%2gToheLa&g=q!^6@-gF{29O=RMGn%SVQ+j;ZN(z`)c~D}|r||Petw@L3 zx?f0tPPs9G?+$+Y$L+_x{#dI3H9C-$fJbS3;|0n-D$?HDX(X~=w&g+1v{edpg#KA{ zKgWHY;l6LN0C9-}&$Y+pH?f*{r>yvM0S?SPhv8(b&UeeZDPqnb2+X_h1}K~Gf8i_O zcPWgTTa7tCw*GMVm%rm^Dz2*OXC|tYm6#BDc3mk*%;}rw)i?J>rX@|b->gs(|AQy0 z+cTV?|EQBxB;dWCioR%nFA8>fOYiKq{ea~ozDDu=vY}Xb!q{M|74t*65Pmm6@xx>X z0y!cU9F+{C!nzkMs8`3pr&mOS?!{Rxs21dMGbFnA%EiM=hK)TfGQ%HXAbNdHA>EO z2|wOacl^_>HosWxKzfbx^p%=(`UYjWfn{K!`vYR2YBq9}m~;wVfeU&R+lD z7%;ZOxWnmirQFC4PO!aRJs%fUW#ku>7N7bc&$O7Peu)s3-%5M7?{DY>LprexL*t#& zaQ?7@K6x)!or(Rzp&{`g{4Lc*j){N@YRGx&PNptcIowNg@P2E<8nJs||e(ND>vxAN@hY^!9jJq<7!@!3$<9fBNx1Kcr{PkT)d`)CLr-XD013*15 z`%N}LYvMW8rSE=R*!7K~R>HVahGE8mXc-4KMa=Lj>!Y`OkpE~t*)!e5YK?Uu9Wni7 z1PGwjxjbYk58?>*kdR7?n9o=c#5*oiG2512$vNGT+v@VyTyQv&ak(vL;=f%%_QzB$x{KxDEd*VzV!boA^8&K zBqH_0nv>3W^!RWRKE4EAO`s=2$ziS?)fa>SWb*9;ebhri&mXUDviyeo%K;lh4lk)A zHh9p3Ro(w6cQwp3KDS(lC!WaMZl?!PP)%sIOR5wZV#nEZBPV9jcjeS2T>bg(tlA3& zRaaeU_3rr! ziS40Fh&>g3UAEz?qR|NJ2dedS@he6$%HDvJpUG8^2J|Gw4*1f0(NZS4bA)zxztW^h zmV+x7ds}ZbUMujCb+33zgGyyNI`U5vbJ4M}S!0QO;r9w31AkT-{#tbgli)MrbY4mx zI4Bh4w(;8w+K(+43TTiJ1C!j&LeQUDeQ@3iCCCBvFI3W!cC&dCLS-7V1y#s}XJaJ39Ur?!573AlYOBn+9w0AdZ z1(2KjhHs(Bo^M^Y#KG(cbsl7FB{W}zGqfd(1y=}tnI1#qjWQOW!a0}U$u(J0abwt@=T6rUoJ*gV+J9MZuej^iYBO<-I&2gO2<*3z|e)~h!_r{GgaF(d*aQQkn6;|2E#lN6O-`fUzRsmJ^t}t}B z*PJ+GWt*W0K4FV5LroC0OnE~T!~Zv4*eb~gpnq+IeKL_dwEL=^OBB@_JX$7#)y27g zCG*h#g~47KALtFQnEj{29&tAOynUCFrrM;43``4eR3CBp%4WH1JMp1}(A=m)8(@)T z0i?4~IrZmQs^6qc5*=#bXTgb&RUWF>0o;u)ZX?>QTFBkr)2vc-#5iugc#512&;bSC zXcPW<=xX`qz_j?asGJC||J7rCR$pP$qUT{-&)AXD%IV(d{nDn5)n(8GF{`@0}z;-%FyQu>OgM!}n!tsH>@k~>4!xnt3f^V6V5X$l~qbsy)`QECGPM6BW@;3J2fq@qT5Mam#SHVHB@(i>C&=c>5fnWi(tE)QKr&v5z{bmx<@(!q09 zfd1ZO3CVBi`=9uOCUkJ2aaRwQogFO=B~!yJ;l7@me9FNq1hzHQV^JRrHYhh5cO2*t z_7DW@pF#@Fb|7Jer%$2?I;8!*9`^0YN~uVeVSH@i@2aaz8DUxXFoY6#%y~A8S=NkC z0Q)0Lril!C<$am@=DQej+|5VYC;k%fwG+6FI4+sOJ^2u?7ZJX~e8(NL!gBt-I%^V= zN}vXdP@PJhtGp9XFpYAiDU{-vMxN`Mx5<5IQnL}w?Ojs>hH@`)2|A59oo{{F_7n{C z@iS{b&Jf!=-^9p(hLBi8$7_kEWQ4sYx*>@NRfj4?`h~I{IMY<+9|I~=G68yiK#BG` zMhq4D3S6f(HfHP=&hu2*2{uX-`0Z3h(v!^WuUj}`ss5!H$1+M8 zf$FFjet+Y{xDbG>YQzJu3^_LlObx+Cj@r{GI<11{Z#JyWbSphbdm(32WuqgyyTW*N z;6m?8No;Kr*Hs*kna~PfoDy z+QdbEiyVOT^o^ekTkt7WLd@g9#0ClU181Fx$ET9J+mUnfYpVH)@l>yVf&QAX^qz2t z6DxHt;p5Yrd;{nEZO-0yb=9w#8P(s`8$A4`9piWh_P8u=$&-C3w8gmvhakvjJpoGr z`2+K%Gr1+LpHQJi)DXf_(JZ)w4(CDI3!rKQ!d8i7D2K;B((}sk;*+?Xhvyy=brbQ- zK6YA>U|o=vTQ9H=A0(}uSy!cjXK~g{konE+h6=^W1@y%v92zZ@_a7pkp26Y&8lFl+ zaFkN6`%ppZXrKlF-JyWs$Spc(xJ*;Q#Vt!eU6F}$Tk!6le3K_C39D(*PJ6Krw0;~m zhfF|bgD%~2kjv2%fH<-=^;wlgJ>cYFe&=!agO?vP0*^=)RJZayZvq`O(rK|FLQS~G z1__6nYWi43iu(2S3Cx7g;Xj%&F!(P97KEG#D=s=~?5xaAsFs1K* zx)bON+OTBd1y(o<;lJmwa0TvwQDEy}cw`uPVWk*V3gS!Zqqx4=Arz1;w0X@#{}2wH z4E%lm20?z9zFFp`Y+mStobTByo>-n@x)f_*;~2DXyfSL*gEE&XRqEXA)p*{J_IKa; zzY6pKw6 zp8RY|tne8~cepkw!=8ZIvwV2)1AV&CLH@!x@P!dCKOS{+b5ovY5Ix)~J0VGr2d=`v z?O(xLG(^s3e;UHlgs}d~>tBVqbe7^?A6dP;iyz+IB3d5s>dL-Wd#8bX>W6Zszw4qk z%VH`w_P;E=y$lQdExZ2KX1RvkIC7W>L3y(6r9gQ|MR{nr(WdWV z$+-*=_szf^pOo44i-4!$b6?~tG#z1bapnpP$r;XG|JtFVHS;b=i|t^UMY|ri77EqK@5#|YLBmJi?1n* zKc`r3m@kNTw#}-PW^4u6g8cYz%0s>>F;Du^T3XUtpQTf_P0G5zn0Q*$>gQu1?V&mLZ^*zQtZ@8IZ9;d9nYXE!F;p}Kk+jqN!Tf@((@sj>zB>GP=%?>uD zY;?EDa<6KY+FP)P)jrZ3g@ZgVi~DCF`lyt|G#IHDY5F34)KSa#t;hALb4b!56qgx4 z-J1<6E^B^MIzLc~TrNJ==!?ss&k-O=M_c_8Z(eY-gGhV1CKi{|;(Y}Gn>;qhcM=3U zAymAzI|euBO;fDuBbsV^JMsvGbLac+Bh#a$$S9cS=71l;+{@_>r^r}cu>fHlOL}(F z=!#jQS*TKk3y_@puVUzP6-AKb<&Cn)5;Ej!rF^O^qV@5$RCZBM(L)O=ifugUb9>G& zTu%_NvY_3agI_3roF!<)Pk5sXm~WWi;O-4T{OUPSyL#;YI%<{S)>f5V4jukJ-ESNZ z+`2bYqW?s&6UR2I+4_RPKDu<+@C7CYj0$a3Y^hL3Hor4(@w&>ck9cTt7XmQH_8H?r zQ>NYQEaj^bdDa!A2+k6AMYb$8b-mt)eI+6!!qDEpsv~p0Y888H;+YSjF|#tKXD6*G zCy1ktueGE-O^L`SZpijqJ?+JFd6yCrM`N|ETho_)TlogfGIgIBlQQ2KU);xYvhGqM z@F_s@&9D%3*K+aH&~B%%(ftbohR7RKEU(!Xn4c{jkJ;5x%{JZ6Z4>lImT@O*bh1qS ze6gpt*N=4`wzNMx?q&(S7X)(XPN_aBWaq|wb|exPkf3@Jc^Vc7Wl;%Qs$*M%>%FdIMCX=qVF3DzWL_|nE2;7Vk~V_Q zj|(`8)D}@an<^RIfh}rw{af;RX4rth_42KsuxVfNwaNeqAM@^G{aTHVc}rQNjl3$r z9$*Vvr|{X2O&77n>I04!A=dg&i6q~x(n%%R8_J|!K~7d{badF*zHN*Qfb`1a zEz#b5I~_-n;vLH^c*e}N#M4N}COS0fTT}5$bB2%nY@G?rM_2iEJ|uMN*g$dl+TwF+ z=%0D(M^UtDGO4fsIt~h~oPN*+W0hmX+T4m5N*BP`zoAIXDk`+wSm1x6kAh{fTxtu% zz1PLn#Rr;8MyEIH9M`7Mwr`9NKc1zxxMUO+wzyX!cmrdN#@k}iZf#~MU~bPM1Du>% zrALKs<7scaYrfXy`1dN!4>`OkKCTVDn8i2o4bK}{3V@a9N+j_AmR~1mCHh3Xp#EW((|&pP|Pv>1U;>=E9J>c z#~E)GS7wqM6f`OG6B8Qqwh~w}x1m*ZS}Tk9s*%;H9*y6aS{o!ddkL z%*}27{MLFJ;?ZQ)wvb5%VoU{tBAnIr zcze{o(#19}V^(!EfE526OPXJx;|WBlK6+if{~Yw%T2{7_ zGz9dhT=EQTGURA1wB09djC5#^)pjx9QTDcGl>Btrz{n#!s2x%^gV^~1;dUNBDj*eV z*mL7>{T*0LiQQWifmxQp-N>6JJ>P$FyQTK0dUZPE-OBTO@+Yr#0aDt_ zh?idzND=CjZC<5k_cln#x14{v_Q{3a6L@!;8W_|h_WJW7A}1d_2=iyF^nIh1Lx`u^ z$enp8H+M1nA52|%0b4wV4d%~wevu0;uf1%blldb<1W@SPx0XF$cWedz)Et<_i*b}r z$gMG|l9t>AT*Ym)9b|sB^T@TtsLw?^TN>BlFEegP$^#72loD~2EyeQqlmmi#1>H zE3Nx9G^{(rMBxG9rEPv&fe` z^a{X(tjHH!^He^=_xxwkJ#tJDpI01%v<@<;B%wcC^Un7uBztznv<(-fpsdYrg#ibz z(j`6_Ten;|Howk>=sOD9ZOp<8>58WI{_1YycTc;}7+n(bnk8Qe#E)t4yu!mgm_-NO zRymW4m~~x<+Bq%b0FB3==Txn*KQoanmGY(_P`Q44yUOGVXfE=Yo0_cBdl}%oUCfhN zBKG9i`Q`o^w-=H|%vT3Ai3W^bs<3LjfI%KGm2>A=@!6B`NT{n)Cr02%K-u$hEJrau zG0@l$2yAM#n8P7Jy3%Wz({0iDZ=@8vMFVLRMn1Mov3DQ!aA1p8^J!u9VgP0$Ce&`BBdO%-5-mkyc<@F7&Pd}Eq{+D zNfx#y>EQ+e=%hS!(D@;|ht*gBCe9|y9BwoO2TlxS^35%g!;!K?i1Dulc^%e=8|2Wk zz<6Z*Z^eRiYI#Up;HKyho)0LAGG?qM^-eORZrqA*865r&^l54^E^~RikM(^@_TaFCS0#v9$84u~XPbe7^tZKhgOl)T z*LIXz)LJt>6DGHprRNn=nN5m#|MJ#9nfbe0_b0)u#mHKE<1qNQ`SC=xWbX>d=?i4P zE7L*3!UPxJeXb~8K#J%CPz2^&2oo1N-(H47&u-lxFQfxj zZ)Tt>n`pJeU#}GDOpJH-N;4+(Q zLAm}Do$*y7G@+0nVuGfDA&>CL(ZUTf)Zz#W`k`3csJ?g}RXt6d1;}MNcTb+SW9p5+ z%Nr&4=Ha#>IFn!$(QGn~o5c6dfg%m#8G(Juc{%Z%($XY|Q3z`Z%6^-~(*$ewDM}G?_gHLoxh0wdp)D*cn5HIe|r4J%(gj^D-=1F*G z;>BaY_L0q^s4NI>FkJcZ%McuURM66LXtZSHAU*szTw{SD(+W8#`yWU7`ffj?mrfk( zj}Umz8cU3eM)IvJVXV{(B}%9@gC;KSeOEtK-5Ltg&NC@(F@ak6jsNn$x;lNyuNqPM z()Nf_Oc^O_hf}F>c!`NJ;bSO5ar*k7_iJh19GeYsSv!Gfvx=SP85(w=|_Swy)VWyG``xpOOe_n zrBzg*?8Q=t=PGrXmCoCTWRj&4n$}nY6A5MMH3g5iE_p!Lsl3B`p#L?ksOO8Zd4npG zen1844_W>Ctf!(Vol99%D2@9k-q2s6RhPz1ep>d{H3e!3Ml%siuO-f(KlMTtu-}0! z!FUxGU{B5BsbDYT^mmbP`;0ryDZu47| zYVy*5orbmv3PnlM#)1F!HtVXg>1tDEivn0o{PsW($~yQ%k~A?#GZ1F&&b1XZ&6+U5 zd>m8QvoU0;c~!8M`1f~YITPIJ;7RMrJvo>)*-7{goV{kaz)hw66G$LBfL88Vt%k@` zkiJC(Ki35u(P$Eso`Xc#e(fW2SWK=gC5guvlmx)To#jU~Zw0efsfQS#^9-Gi)uq6J z0uUlQIW#X5f%`}e?9&!2en|e|f9xhznDq8_B=1xx+B`dHI}SmovD^?YKotGW@;*LB z09qv#zmg?wQI#b~+}_wPG=2h}vd;up$l2cWI-{+3oGe;R#{d=j=a^9$wZt`;+kmXa z5~NCJ!AI`(q%=XeqA;#VsJhDi@f*is;m(eSr78 z0#cPHfW>Wdr#2#WV30nreEgn;{r35{Z<)Rnx_bP?L6CjG9MR^fz`|YW|6H$KZPks5 zf6R}2;a6l>KygB24}J?cQxZ=2pjZIN7uP5+k%i9r0bvK;k(?uyPH8 zSPan@9TAzZAx>Y(DNp9+0Hb8KHVR3O>zPXYe2@O0^Y<>lJ z)p4RVwmd9{x=-aTND#v#~(+Aun|oNB{J@-y~& zR+__gRD4W#sNM2YYF`9NDcUM=$e-D4D?%$47ovj%AH?sp^bN%xTrx%ixM`C-q8}U`R(Bd5DDyU1 zE(lPWs+|m&2I*_qzNVX1|9M-MU1Jv|SuR<}N#4<6wl{?W6r6FyYROdUR5JzdzX6B0 zE-VBvM%qtz$-H!avg+Z{oN=QUmW?fyj!RXlt`=Z~$95#zc^b8<+YfeLmZ?Mp7qQp6 zjDJZo&_RWEdm<1`rwW}UBR?elvmK>C#l9{w-8LH0PU=oLX!H)TB>aZ0PSEn6shR>* z)1ZDj(pk=`qsAy<;V0T|;5xBAuMQZ1?Fz}dzSp13YER=JY(F00MO}7mTZGvKSFQRW z1UYoZ(cf?;_Jx~VrY4-h0I$(97h!TXw&hm&+YL}s3C8-^6-yPId?IrU%B>}d3Uz-_ z`)NA2HYV8pXecAwo+L~Bd8W;{rDlvFOh5Cs8Xq^gdahpWHL#{DBp_4QV6og&>5l4A zcxZqi*(#$bfO8CY%?$G%P1o3!0ZOg>#eBWfVh7-0wC?W!wWU4}oy&guP10Nk5EC6l zz@5G)J=>}uJG2|4nXx+CMtQ^RS$NO)5+=w};renWovRKsKHenR8kE0#kk>O>eltHx z4`9%yP+ab835BECdy`|GyXK(Z-GKn_;G~l-GMa0;ib8wYUo=LI!id=cMM(SrY}B6* zCHOr9?`qPqod|)4uE@J{I330AfEq5)jRV!W;chkhp~$m=%1;-aIZQL!%g_N{7;Pvv zy?gh5HQ+Y1A?FXTh`Yl^WWOf;b?eN(!N(fapVcpI9G_fHWwE5tAdlOivu6g3&cXuR z|5!ZaAzN2Q^IRl-zT=;ydR?h{P>*?K)$;~PlI}QL4iUHx7kmhCw&l9gayJ8*yc zcUsJ6j-B2^K$|_gdEd<96ZhY-;mdw#7W4&WXqT{2X$D0Om}XQK7w^**0t=`PY~>Q` zBAL6MXNGO~%S=%Ubjq0|PVDg6^6DE`&htP-2Z+eGQ^4s4zg4FhT4B?pvpu{(6UJ>3 z#%)(a(uvK)_Y?tN8pi*_frAQGs0&7J6Y4R4(?d?(A)9Etg{l<4<|x#Cu;|_PZAWeE zN|Z4SF8t$6|2234Meml`2M;8$bQk+L*z0=K^fDV@0MlDMGSnW@4K`;fbJ$T$#*P)L zZHJuzTAzXs8M4N?18WG4pITUDz^};ZTi&2T=)icdyW9mNCDH_R^XEikDMtI5JPl^! zRg4ZXizn3dB}C?^dABOdKAOKbTwMO87rSqCIZ2oyzoH1oJTI{xkkLRtAvj;?6@OfQ~BvO+06<)8_N{H)$fM1@JSF zZj4jV?+2IuE3y3+atezESKM&&rc*)8p3yY%WlDL-QF#afI5pzo;cHSw2pguG$)wk@+2lE~e`che_vH5%UJ6J+7jlldK_ncw!24kKMlvV#X zLY}14ffay{EQnV|c?0)|hLNDm=Kg&YO+g&RLQr|QUF)_6paedtT2Rw|Jg=s?!?z@4 zZU@Wtx3G5gZDjIptap69Z~+|c8`AcNFeA?zXXrh7$Qq%uKdvUqA&i+6VOc>9z}45cMs3`dT!1P9%ndap+fSB(`MU6D zVkTC=U>V@HGLJntKTy#1LnG-g8+MRPdx9CXJF`r^(1n3hp8r2gcEgdq4C93iW2yPQ zak{^0$a_H6!v7ZH`4L0yvdmAGc()?I`pS?ArW@df>>>vdDB1%Kh`v5w0NfD#Xby^T z-SG?_!_5 zVBZ!q8_FN#ymbkLo&KD`Vc=xyj)-Y8j{H+HN4WvZuQD=)&O>;L4hXY(Jy2+6TLi)k z5uh4r)m-?U0fCMfc^zi&lr~+oEFpy(22e}uh7z*X`@PUUGk@GlGCW{e-~^akn9&@~ z3wx&dsf=km6(Fx@ewqsk9q%bD4h_$vlOXC}H_>S>onjUaLmC(R#9A)Y-HZ<{&jC!O zFM+N+s{yX$po}(v;BwcWqX;CWjGml)tp6##L(*k*+@$&fd3{885S$OF?AR>n^0Ca7Cp{Jxc)?e(^>_)0d%}O25mx;syxz$k6c^lj!2lLI4z?HZaEO>=wa|8-HcU**6KrrGQo1kUiv`lC$gO<3!rLT{9c%hgTnr?G*O4aj+1uOmtR z^U_A_etH4=sK&;7o8Oa^8eQst-b%{WcE3HaUAPTioBNBpoy)Ki*Hqzfj03bGqe*lM z_hh)UZclpCgT-2v95#O?8c^wC07z}Qq!sLWiFQ-aRSTSBO^gmm$jfl2066u?SSpt% ztZt^xOa1RZ^BX_A1I@xA4>SmM&%{ji^G;bA^%H<1;f|+>Lz#+t*JJ2J&sU>lAL$eg zT;>gQ`yls!ynCp-b^xU3NA8&*#?OJczzItP{D47<&08KQ>k!aNLU0@l+$96SG|k!; zg@X0ksw*CjIv@Ws1zqIMP}!TWMaw?&5a4OMxlYf*>{RLoYnp@Wo1l5;S!#O{Kg2gV zd8%*1zS$bSI6+x!(19&H-o;Dhl7*y3IU!B<@?2u9%npDWPCRuiN_`aZ&zleORnF~e zgxf$rb%^=q7IH>=pf-G6C~Ym}Tw?7mB0-+UinLTd-Suaq`VjBZeqt}ga^T9{ubkos zu|OcLN7~xb=}dWY_ZI;!HTnFkXdBef?F+P_sY?hRMXaftzJf<{8U%pr6*jj*FYV_J z6!u?PZC2fraHM|>&OR`)I7hQdMZ-G|o(ANWSz90)c<^i=2g3S`JD?^{k?ddwS`#_D z18rduha3PuBr{>llz$xj{tEO@=VOQy;BG=$6Ez%$?`Oy`4(PfU?yFAhA7X)BTNM^j zzWI#+M@y#Ei@%^&-3Ms<2F6Bihn@IiyJtT9{k8(CZZG6&KQ>?{+m8lJ|F2jVipL(k zZQu3t_UUI|rOT1t%>x9+a_;jkSgLbu2zg)%9PQZ0k>g9SpI+UMsF>UCLuo&|^hl6}x` zuwAExSo{-PCWvV5;idZEgN0NXMvC1PI(Yh7=<$^L@cHSQ^hP}rZJEJIp$ny)do!S; z?!$=adVm6RPvyAvm6I-U4-scrBr4DoV7H-9ji`z4lB8@m{$U4|T3-4t87G&_p(FlW z9Q28WgFp&F8pa=PyC6PMP>&r15~rhWHV@$oNpsrddlwy6zqNVJ2)K>RsW)|haqzYHII3o{VlU}&*Zoiecc&kOMHmLGm{=(~r>$gO_UpG4t1 z3_WN+^G@Sa_7t8i0PS;c2%!YZQ&j?;Uwkv7(X>T?!-q@-2F8 z4*>P@Q_nQ`&aw_9j&yha?^hXo{R!Sg@OjMfkvWQ;g)4IFsb+O>!9yULUls@{`Yx)9 zPBQ0#QeiO1^p$cs-}4dCE>1GcFuXG_-ie$8>qD%*xidbRErHDs>7}fu3NM|!Q`JYf zdY6&k+VapS;kfGmBI2X3;v*;@38@3`rNEc(rOYuvkH-UlLI*tt*mbVx^p7CkFNnA! z5z4UMiW2Ra4G(=o8iKVZH*Nu5j=^vU8kmS?-Fbdjg~I{AF8v><0rxXM-Z}8B(*)4X z9Y~LoU>h8{0*=H4I2c-^Uf*}2$d^f8Yo{AM)S|?IAF&drHy5qFU-aL+S))g;9}OfA z=gIho94URsG2VPL`UYdobA3M(8n&|bKLXp20isTivtw;IRAlxEQ@c zp-?Dp#uGhyg+ifFWI_qOLZMK2I36SPIW;i%1cVTgLWsYF5Zi?i+k_BL+qM@LLi{0w z*kk*2;y4crIB z0^Mzm$vus9;5jK}EKmm+WV_Fyz#=JSIbf0PyC?EAup`s=4}cL;${64b&|ep;oX2Ot z0x4xhpc>k}d-SswLX-ra0urQ@RnSVKcf&KE^+cf6E!*z&eD6c+=TQ!=?dq7^T*xDY z9!La|rIh=ml-Gf2KnI;|ayU}Tqf*NCw$B&Pe(+18-RmUVut`8iT?}+0LWl^U4e*Wa zy8xO2Gi^gW>b`{#1%XCry&jNKUYAmy#S5M_TAz%3wr+KSz)?K#SxYIGqV?~{%$=TZ z2jDxYu?h$gLfGBN*i(c)s|M!I>9T7Ogb-0c4PZ4;R7!c&w(A&BR4==n$VlKtz-s#* z3Y?WvrrW-I+B1&^P688b;i_+h5c%=&XDy{X415e6L=*UPz&bqpS!d=>&o@sR=P5$( zRR|%1fLXv0DdqXh`)n;7Jt}8b2+nTF-HBHEc5WZ+4KSfG8+qU_5#%rAI?-P32=OpR_?S&Bg(ZX`g z(Bju2w%2>`yl0*Qq(~`O+rH046Mj#B*7pGaN-0lDDbs-2XwTo1pLJ&L^nCNAah@Xd z_B1eedu(y(@xTr#<)@i;Wp2*i2bkk(+nkA%@?|Mygp@K0?b$AtQuYM?LTg439(LbS z%45JGAw&h+rsin9mp%GfqlIkh2q6M(|KBt2+hd>gE!^q(&INkgvTy1D=Q4$Oaz;w& z5r2Tzz;0kKFidZIoOF*3&|c*>0~>9BTBF6Q7Xf>KKY_w}+v>!BgC@W)w$EjNH9#`( zBjB&MeGb0Y>Ohk1dH)O)@`h&(yb0_CwgQs@e@}eYz*Mw(h%~e}xc+x~zJNf>EZhrh zLCZ?A`&z05f_g0RsaA1 literal 0 HcmV?d00001 diff --git a/docs/jsd_cross_entropy.html b/docs/jsd_cross_entropy.html index fc89fa1..a3a95b9 100644 --- a/docs/jsd_cross_entropy.html +++ b/docs/jsd_cross_entropy.html @@ -352,7 +352,7 @@
-
plt.plot([ .1*i-1 for i in range(len(losses))],[loss for loss in losses])
+
plt.plot([ .1*i-1 for i in range(len(losses))],[loss for loss in losses])
 plt.ylabel('JS Divergence')
 plt.xlabel('Change in output')
 plt.show()
diff --git a/docs/multistepLR.html b/docs/multistepLR.html
new file mode 100644
index 0000000..0035af3
--- /dev/null
+++ b/docs/multistepLR.html
@@ -0,0 +1,557 @@
+---
+
+title: MultiStepLRScheduler
+
+
+keywords: fastai
+sidebar: home_sidebar
+
+
+
+nb_path: "nbs/07f_multistepLR.ipynb"
+---
+
+
+
+ + {% raw %} + +
+ +
+ {% endraw %} + +
+
+

MultiStepLRScheduler decay learning rate at given epochs.

+ +
+
+
+
+
+

MultiStepLRScheduler is very similar to StepLRScheduler.\ +Difference is MultiStepLRScheduler decay LR at given epochs but StepLRScheduler decay LR after every decay_t epochs.
+It is possible to do warmup and add noise.

+ +
+
+
+ {% raw %} + +
+ +
+
+ +
+ + +
+

class MultiStepLRScheduler[source]

MultiStepLRScheduler(optimizer:Optimizer, decay_t:List[int], decay_rate:float=1.0, warmup_t=0, warmup_lr_init=0, t_in_epochs=True, noise_range_t=None, noise_pct=0.67, noise_std=1.0, noise_seed=42, initialize=True) :: Scheduler

+
+ +
+ +
+ +
+
+ +
+ {% endraw %} + +
+
+

The schedule looks something like:

+ +
+
+
+ {% raw %} + +
+ +
+
+ +
+ + + +
+ +
+ +
+ +
+
+ +
+ {% endraw %} + +
+
+

Using MultiStepLRScheduler scheduler with timm training script

+
+
+
+
+
+

To train models using the MultiStepLRScheduler we simply update the training script args passed by passing in --sched multistep parameter alongside the necessary hyperparams. In this section we will also look at how each of the hyperparams update the MultiStepLRScheduler scheduler.

+ +
+
+
+
+
+

The training command to use MultiStepLRScheduler scheduler looks something like:

+
python train.py ../imagenette2-320/ --sched multistep
+
+ +
+
+
+
+
+

Availible parameters are:

+

--epochs - initial number of epoch to train. default 300.
+--lr - learning rate (default: 0.05)

+

--decay-epochs - epoch interval to decay LR , (default: 100), decay_t argument in python script
+--decay-rate - LR decay rate (default: 0.1)

+

warmup parameters:
+--warmup-lr' - warmup learning rate (default: 0.0001)
+--warmup-epochs - epochs to warmup LR, if scheduler supports (default: 3)

+

noise parameters:
+--lr-noise - learning rate noise on/off epoch percentages
+--lr-noise-pct - learning rate noise limit percent (default: 0.67)
+--seed - random seed (default: 42) to seed noise generator.

+ +
+
+
+
+
+

Using in python script.

+
+
+
+
+
+

MultiStepLRScheduler accepts two required arguments - an optimizer and decay_t, and also some hyperparams which we will look into in detail below.

+ +
+
+
+
+
+

Basic usage like this:

+ +
+
+
+
+
+
from timm.scheduler.poly_lr import MultiStepLRScheduler
+scheduler = MultiStepLRScheduler(optimizer, decay_t=[step_t, step_t1])
+
+ +
+
+
+
+
+

Required arguments.

+
+
+
+
+
+

optimizer is object of torch.optim.Optimizer
+decay_t - list of epochs number to decay LR.

+ +
+
+
+
+
+

Default schedule:\ +Default value for decay_rate is 1., so no decay.\ +Default parameter for decay_rate at train script --decay-rate if is 0.1.

+ +
+
+
+ {% raw %} + +
+
+ +
+
+
scheduler = MultiStepLRScheduler(optimizer, decay_t=[10, 30])
+plot_lr(scheduler, label='default')
+
+scheduler = MultiStepLRScheduler(optimizer, decay_t=[10, 30], decay_rate=0.1)
+plot_lr(scheduler, label='decay_rate=0.1, default at train script')
+
+scheduler = MultiStepLRScheduler(optimizer, decay_t=[20], decay_rate=0.3)
+plot_lr(scheduler, label='decay_t=[20], decay_rate=0.3')
+
+plt.legend();
+
+ +
+
+
+ +
+
+ +
+ + + +
+ +
+ +
+ +
+
+ +
+ {% endraw %} + +
+
+

decay_rate

+
+
+
+
+
+

decay_rate - multiplier for decay learning rate. So, at every step number at the decay_t list, learning rate is decayed by decay_rate. If decay_t=0.5 than at epoch decay_t[0] new learning rate will be set to lr * decay_rate. And so on on next step at next element from decay_t.

+ +
+
+
+
+
+

By setting decay_t = 5 and decay_rate = 1., we are telling the schedule to reduce the learning rate by decay_rate where new lr lr * decay_rate every 5 epochs.

+ +
+
+
+ {% raw %} + +
+
+ +
+
+
scheduler = MultiStepLRScheduler(optimizer, decay_t=[20, 30], decay_rate=0.5)
+plot_lr(scheduler, label='decay_rate=0.5, decay_t=[20, 30]')
+
+scheduler = MultiStepLRScheduler(optimizer, decay_t=[15], decay_rate=0.3)
+plot_lr(scheduler, label='decay_rate=0.3, decay_t=[15]')
+
+scheduler = MultiStepLRScheduler(optimizer, decay_t=[10, 30], decay_rate=0.1)
+plot_lr(scheduler, label='decay_rate=0.1, decay_t=[10, 30]')
+plt.legend();
+
+ +
+
+
+ +
+
+ +
+ + + +
+ +
+ +
+ +
+
+ +
+ {% endraw %} + +
+
+

Warmup Args.

+
+
+
+
+
+

warmup_t

+
+
+
+
+
+

Defines the number of warmup epochs.

+ +
+
+
+
+
+

warmup_lr_init

+
+
+
+
+
+

The initial learning rate during warmup. Default is 0.

+ +
+
+
+ {% raw %} + +
+
+ +
+
+
scheduler = MultiStepLRScheduler(optimizer, decay_t=[20, 30], decay_rate=0.5)
+plot_lr(scheduler, label='no warmup')
+
+scheduler = MultiStepLRScheduler(optimizer, decay_t=[20, 30], decay_rate=0.5, warmup_t=2)
+plot_lr(scheduler, label=', warmup_t=2')
+
+scheduler = MultiStepLRScheduler(optimizer, decay_t=[20, 30], decay_rate=0.5, warmup_t=2, warmup_lr_init=0.05)
+plot_lr(scheduler, label='warmup_t=2, warmup_lr_init=0.05')
+
+plt.legend();
+
+ +
+
+
+ +
+
+ +
+ + + +
+ +
+ +
+ +
+
+ +
+ {% endraw %} + +
+
+

As we can see by setting up warmup_t and warmup_lr_init, the scheduler first starts with a value of warmup_lr_init, then during warmup_t number of epochs gradually progresses up to the LR value at epoch warmup_t + 1.

+ +
+
+
+
+
+

Noise Args.

+
+
+
+
+
+

noise_range_t

+
+
+
+
+
+

If it is number - its number of epoch when noise starts.
+If list or tuple (of two elements) - first and second element is epoch number range, when noise applied.

+ +
+
+
+
+
+

The upper and lower limit of noise.

+ +
+
+
+
+
+

noise_pct

+
+
+
+
+
+

Percentage of noise to add.

+ +
+
+
+ {% raw %} + +
+
+ +
+
+
scheduler = MultiStepLRScheduler(optimizer, decay_t=[15, 30], decay_rate=0.5, noise_range_t=35)
+plot_noisy_lr(scheduler, label='noise_pct=0.65, def')
+
+scheduler = MultiStepLRScheduler(optimizer, decay_t=[15, 30], decay_rate=0.5, noise_range_t=[10, 30], noise_pct=0.2)
+plot_noisy_lr(scheduler, label='noise_pct=0.2')
+plt.legend();
+
+ +
+
+
+ +
+
+ +
+ + + +
+ +
+ +
+ +
+
+ +
+ {% endraw %} + +
+
+

noise_std

+
+
+
+
+
+

Noise standard deviation. Now it is not implemented.

+ +
+
+
+
+
+

noise_seed

+
+
+
+
+
+

Seed to use to add random noise.

+ +
+
+
+
+
+

Miscellaneous.

+
+
+
+
+
+

t_in_epochs

+
+
+
+
+
+

If set to False, the learning rates returned for epoch t are None.

+ +
+
+
+ {% raw %} + +
+
+ +
+
+
scheduler = MultiStepLRScheduler(optimizer, decay_t=[0], t_in_epochs=False)
+lr_per_epoch = calculate_lr(scheduler, 5)
+
+lr_per_epoch[:5]
+
+ +
+
+
+ +
+
+ +
+ + + +
+
[None, None, None, None, None]
+
+ +
+ +
+
+ +
+ {% endraw %} + +
+
+

initialize

+
+
+
+
+
+

If True, then inside each param group of the optimizer a new field is set called initial_{field_name} where field_name refers to the field in param group that we are scheduling. Typically field_name='lr'.

+ +
+
+
+
+ + diff --git a/docs/plateau.html b/docs/plateau.html index a2314aa..156e541 100644 --- a/docs/plateau.html +++ b/docs/plateau.html @@ -65,7 +65,7 @@
-

class PlateauLRScheduler[source]

PlateauLRScheduler(optimizer, decay_rate=0.1, patience_t=10, verbose=True, threshold=0.0001, cooldown_t=0, warmup_t=0, warmup_lr_init=0, lr_min=0, mode='max', noise_range_t=None, noise_type='normal', noise_pct=0.67, noise_std=1.0, noise_seed=None, initialize=True) :: Scheduler

+

class PlateauLRScheduler[source]

PlateauLRScheduler(optimizer, decay_rate=0.1, patience_t=10, verbose=True, threshold=0.0001, cooldown_t=0, warmup_t=0, warmup_lr_init=0, lr_min=0, mode='max', noise_range_t=None, noise_type='normal', noise_pct=0.67, noise_std=1.0, noise_seed=None, initialize=True) :: Scheduler

Decay the LR by a factor every time the validation loss plateaus.

diff --git a/docs/schedulers.html b/docs/schedulers.html index 02e1c02..455ddd8 100644 --- a/docs/schedulers.html +++ b/docs/schedulers.html @@ -31,12 +31,14 @@
-

In timm, essentially we have a total of four different schedulers:

+

In timm, essentially we have a total of six different schedulers:

    -
  1. SGDR: Stochastic Gradient Descent with Warm Restarts
  2. -
  3. Stochastic Gradient Descent with Hyperbolic-Tangent Decay on Classification
  4. -
  5. StepLR
  6. +
  7. CosineLRScheduler - SGDR: Stochastic Gradient Descent with Warm Restarts
  8. +
  9. TanhLRScheduler - Stochastic Gradient Descent with Hyperbolic-Tangent Decay on Classification
  10. +
  11. StepLRScheduler
  12. PlateauLRScheduler
  13. +
  14. MultiStepLRScheduler
  15. +
  16. PolyLRScheduler - SGDR scheduler with Polynomial anneling function.

In this tutorial we are going to look at each one of them in detail and also look at how we can train our models using these schedulers using the timm training script or use them as standalone schedulers for custom PyTorch training scripts.

@@ -58,13 +60,13 @@

Available Schedulers
-

SGDR

+

CosineLRScheduler - SGDR

-

First, let's look at the SGDR scheduler also referred to as the cosine scheduler in timm.

+

First, let's look at the CosineLRScheduler - SGDR scheduler also referred to as the cosine scheduler in timm.

The SGDR scheduler, or the Stochastic Gradient Descent with Warm Restarts scheduler schedules the learning rate using a cosine schedule but with a tweak. It resets the learning rate to the initial value after some number of epochs.

@@ -86,13 +88,13 @@

SGDR

-

StepLR

+

StepLRScheduler

-

The StepLR is a basic step LR schedule with warmup, noise. +

The StepLRScheduler is a basic step LR schedule with warmup, noise. {% include note.html content='PyTorch’s implementation does not support warmup or noise. ' %}

@@ -121,13 +123,13 @@

StepLR

-

Stochastic Gradient Descent with Hyperbolic-Tangent Decay on Classification

+

TanhLRScheduler - SGDR with Hyperbolic-Tangent Decay.

-

This is also referred to as the tanh annealing. tanh stands for hyperbolic tangent decay. The annealing using this scheduler looks something like:

+

TanhLRScheduler is also referred to as the tanh annealing. tanh stands for hyperbolic tangent decay. The annealing using this scheduler looks something like:

@@ -154,7 +156,71 @@

PlateauLRScheduler
-

This scheduler is very similar to PyTorch's ReduceLROnPlateau scheduler. The basic idea is to track an eval metric and based on the evaluation metric's value, the lr is reduced using StepLR if the eval metric is stagnant for a certain number of epochs.

+

This scheduler is based on PyTorch's ReduceLROnPlateau scheduler with possible warmup and noise. The basic idea is to track an eval metric and based on the evaluation metric's value, the lr is reduced using StepLR if the eval metric is stagnant for a certain number of epochs.

+ +
+
+

+
+
+

MultiStepLRScheduler

+
+
+
+
+
+

The MultiStepLRScheduler is LR schedule with decay learning rate at given epochs, with warmup, noise.

+ +
+
+
+
+
+

The schedule for MultiStepLRScheduler annealing (with 'step' at 10, 30) looks something like:

+ +
+
+
+
+
+

{% include image.html alt="StepLR" width="500" max-width="500" file="/timmdocs/images/MultistepLr.png" %}

+ +
+
+
+
+
+

In the above MultiStepLRScheduler schedule, decay_t=[10, 30] and decay_rate=0.5 with an initial lr of 0.1.\ +At epoch number 10 and 30, the learning rate is updated to be lr * decay_rate.

+ +
+
+
+
+
+

PolyLRScheduler - SGDR with Polynomial Decay.

+
+
+
+
+
+

PolyLRScheduler is also referred to as the poly annealing.\ +poly stands for polynomial decay.\ +The annealing using this scheduler looks something like:

+ +
+
+
+
+
+

{% include image.html alt="Tanh" width="500" max-width="500" file="/timmdocs/images/PolyLR.png" %}

+ +
+
+
+
+
+

It is similar to the cosine and tanhin the sense that the learning rate is set to the initial lr after a certain number of epochs but the annealing is done using the poly function.

@@ -179,6 +245,8 @@

Using the vari
  • For PlatueLRScheduler we pass in --sched plateau.
  • For TanhLRScheduler, we pass in --sched tanh.
  • For StepLR, we pass in --sched step.
  • +
  • For MultiStepLRScheduler, we pass in --sched multistep.
  • +
  • For PolyLRScheduler, we pass in --sched poly.
  • diff --git a/docs/sidebar.json b/docs/sidebar.json index 300252d..98c6f99 100644 --- a/docs/sidebar.json +++ b/docs/sidebar.json @@ -36,7 +36,9 @@ "Cosine": "/SGDR", "StepLR": "/stepLR", "Tanh": "/tanh", - "Plateau": "/plateau" + "Plateau": "/plateau", + "Multistep": "/multistepLR", + "PolyLR": "/PolyLR" }, "Tutorials": { "What all goes on inside the create_model function?": "/tutorial_feature_extractor", diff --git a/docs/stepLR.html b/docs/stepLR.html index e987543..2de0518 100644 --- a/docs/stepLR.html +++ b/docs/stepLR.html @@ -79,7 +79,7 @@
    -

    class StepLRScheduler[source]

    StepLRScheduler(optimizer:Optimizer, decay_t:float, decay_rate:float=1.0, warmup_t=0, warmup_lr_init=0, t_in_epochs=True, noise_range_t=None, noise_pct=0.67, noise_std=1.0, noise_seed=42, initialize=True) :: Scheduler

    +

    class StepLRScheduler[source]

    StepLRScheduler(optimizer:Optimizer, decay_t:float, decay_rate:float=1.0, warmup_t=0, warmup_lr_init=0, t_in_epochs=True, noise_range_t=None, noise_pct=0.67, noise_std=1.0, noise_seed=42, initialize=True) :: Scheduler

    @@ -321,7 +321,7 @@

    decay_rate=0.5
    num_epoch = 50
    -scheduler = StepLRScheduler(optimizer, decay_t = 5, decay_rate=.5)
    +scheduler = StepLRScheduler(optimizer, decay_t = 5, decay_rate=.5)
     lr_per_epoch = get_lr_per_epoch(scheduler, num_epoch)
     
     plt.plot([i for i in range(num_epoch)], lr_per_epoch);
    diff --git a/docs/tanh.html b/docs/tanh.html
    index 177fc07..cc74646 100644
    --- a/docs/tanh.html
    +++ b/docs/tanh.html
    @@ -72,7 +72,7 @@
     
     
     
    -

    class TanhLRScheduler[source]

    TanhLRScheduler(optimizer:Optimizer, t_initial:int, lb:float=-6.0, ub:float=4.0, t_mul:float=1.0, lr_min:float=0.0, decay_rate:float=1.0, warmup_t=0, warmup_lr_init=0, warmup_prefix=False, cycle_limit=0, t_in_epochs=True, noise_range_t=None, noise_pct=0.67, noise_std=1.0, noise_seed=42, initialize=True) :: Scheduler

    +

    class TanhLRScheduler[source]

    TanhLRScheduler(optimizer:Optimizer, t_initial:int, lb:float=-7.0, ub:float=3.0, lr_min:float=0.0, cycle_mul:float=1.0, cycle_decay:float=1.0, cycle_limit:int=1, warmup_t=0, warmup_lr_init=0, warmup_prefix=False, t_in_epochs=True, noise_range_t=None, noise_pct=0.67, noise_std=1.0, noise_seed=42, initialize=True) :: Scheduler

    Hyberbolic-Tangent decay with restarts. This is described in the paper https://arxiv.org/abs/1806.01593

    diff --git a/nbs/.last_checked b/nbs/.last_checked new file mode 100644 index 0000000..e69de29 diff --git a/nbs/07_schedulers.ipynb b/nbs/07_schedulers.ipynb index 7925107..3862372 100644 --- a/nbs/07_schedulers.ipynb +++ b/nbs/07_schedulers.ipynb @@ -11,12 +11,14 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "In `timm`, essentially we have a total of four different schedulers: \n", + "In `timm`, essentially we have a total of six different schedulers: \n", "\n", - "1. [SGDR: Stochastic Gradient Descent with Warm Restarts](https://arxiv.org/abs/1608.03983)\n", - "2. [Stochastic Gradient Descent with Hyperbolic-Tangent Decay on Classification](https://arxiv.org/abs/1806.01593)\n", - "3. [StepLR](https://github.com/rwightman/pytorch-image-models/blob/master/timm/scheduler/step_lr.py#L13)\n", + "1. [CosineLRScheduler](https://github.com/rwightman/pytorch-image-models/blob/master/timm/scheduler/cosine_lr.py#L18) - [SGDR: Stochastic Gradient Descent with Warm Restarts](https://arxiv.org/abs/1608.03983)\n", + "2. [TanhLRScheduler](https://github.com/rwightman/pytorch-image-models/blob/master/timm/scheduler/tanh_lr.py#L18) - [Stochastic Gradient Descent with Hyperbolic-Tangent Decay on Classification](https://arxiv.org/abs/1806.01593)\n", + "3. [StepLRScheduler](https://github.com/rwightman/pytorch-image-models/blob/master/timm/scheduler/step_lr.py#L13)\n", "4. [PlateauLRScheduler](https://github.com/rwightman/pytorch-image-models/blob/master/timm/scheduler/plateau_lr.py#L12)\n", + "5. [MultiStepLRScheduler](https://github.com/rwightman/pytorch-image-models/blob/master/timm/scheduler/multistep_lr.py#L10)\n", + "6. [PolyLRScheduler](https://github.com/rwightman/pytorch-image-models/blob/master/timm/scheduler/poly_lr.py#L18) - SGDR scheduler with Polynomial anneling function.\n", "\n", "In this tutorial we are going to look at each one of them in detail and also look at how we can train our models using these schedulers using the `timm` training script or use them as standalone schedulers for custom PyTorch training scripts." ] @@ -39,14 +41,14 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "### SGDR" + "### CosineLRScheduler - SGDR" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ - "First, let's look at the `SGDR` scheduler also referred to as the `cosine` scheduler in `timm`. \n", + "First, let's look at the `CosineLRScheduler` - `SGDR` scheduler also referred to as the `cosine` scheduler in `timm`. \n", "\n", "The `SGDR` scheduler, or the `Stochastic Gradient Descent with Warm Restarts` scheduler schedules the learning rate using a cosine schedule but with a tweak. It resets the learning rate to the initial value after some number of epochs. " ] @@ -69,14 +71,14 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "### StepLR" + "### StepLRScheduler" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ - "The `StepLR` is a basic step LR schedule with warmup, noise. \n", + "The `StepLRScheduler` is a basic step LR schedule with warmup, noise. \n", "\n", "> NOTE: PyTorch's implementation does not support warmup or noise. " ] @@ -106,14 +108,14 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "### Stochastic Gradient Descent with Hyperbolic-Tangent Decay on Classification " + "### TanhLRScheduler - SGDR with Hyperbolic-Tangent Decay." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ - "This is also referred to as the `tanh` annealing. `tanh` stands for hyperbolic tangent decay. The annealing using this scheduler looks something like: " + "`TanhLRScheduler` is also referred to as the `tanh` annealing. `tanh` stands for hyperbolic tangent decay. The annealing using this scheduler looks something like: " ] }, { @@ -141,7 +143,73 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "This scheduler is very similar to PyTorch's [ReduceLROnPlateau](https://pytorch.org/docs/stable/_modules/torch/optim/lr_scheduler.html#ReduceLROnPlateau) scheduler. The basic idea is to track an eval metric and based on the evaluation metric's value, the `lr` is reduced using `StepLR` if the eval metric is stagnant for a certain number of epochs. " + "This scheduler is based on PyTorch's [ReduceLROnPlateau](https://pytorch.org/docs/stable/_modules/torch/optim/lr_scheduler.html#ReduceLROnPlateau) scheduler with possible warmup and noise. The basic idea is to track an eval metric and based on the evaluation metric's value, the `lr` is reduced using `StepLR` if the eval metric is stagnant for a certain number of epochs. " + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### MultiStepLRScheduler" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "The `MultiStepLRScheduler` is LR schedule with decay learning rate at given epochs, with warmup, noise. " + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "The schedule for `MultiStepLRScheduler` annealing (with 'step' at 10, 30) looks something like: " + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "\"StepLR\"" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "In the above `MultiStepLRScheduler` schedule, `decay_t=[10, 30]` and `decay_rate=0.5` with an initial `lr` of 0.1.\\\n", + "At epoch number 10 and 30, the learning rate is updated to be `lr * decay_rate`. " + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### PolyLRScheduler - SGDR with Polynomial Decay." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "`PolyLRScheduler` is also referred to as the `poly` annealing.\\\n", + "`poly` stands for polynomial decay.\\\n", + "The annealing using this scheduler looks something like: " + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "\"Tanh\"" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "It is similar to the `cosine` and `tanh`in the sense that the learning rate is set to the initial `lr` after a certain number of epochs but the annealing is done using the `poly` function. " ] }, { @@ -165,7 +233,9 @@ "- For `SGDR`, we pass in `--sched cosine`. \n", "- For `PlatueLRScheduler` we pass in `--sched plateau`. \n", "- For `TanhLRScheduler`, we pass in `--sched tanh`.\n", - "- For `StepLR`, we pass in `--sched step`." + "- For `StepLR`, we pass in `--sched step`.\n", + "- For `MultiStepLRScheduler`, we pass in `--sched multistep`.\n", + "- For `PolyLRScheduler`, we pass in `--sched poly`." ] }, { @@ -182,7 +252,7 @@ ], "metadata": { "kernelspec": { - "display_name": "Python 3", + "display_name": "Python 3 (ipykernel)", "language": "python", "name": "python3" }, @@ -196,7 +266,7 @@ "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", - "version": "3.8.5" + "version": "3.9.7" }, "toc": { "base_numbering": 1, @@ -209,7 +279,7 @@ "toc_cell": false, "toc_position": {}, "toc_section_display": true, - "toc_window_display": false + "toc_window_display": true } }, "nbformat": 4, diff --git a/nbs/07f_multistepLR.ipynb b/nbs/07f_multistepLR.ipynb new file mode 100644 index 0000000..a2ffcee --- /dev/null +++ b/nbs/07f_multistepLR.ipynb @@ -0,0 +1,662 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# MultiStepLRScheduler" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "`MultiStepLRScheduler` decay learning rate at given epochs." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "`MultiStepLRScheduler` is very similar to `StepLRScheduler`.\\\n", + "Difference is `MultiStepLRScheduler` decay LR at given epochs but `StepLRScheduler` decay LR after every `decay_t` epochs. \n", + "It is possible to do warmup and add noise." + ] + }, + { + "cell_type": "code", + "execution_count": 1, + "metadata": {}, + "outputs": [], + "source": [ + "#hide\n", + "%load_ext autoreload\n", + "%autoreload 2" + ] + }, + { + "cell_type": "code", + "execution_count": 2, + "metadata": {}, + "outputs": [], + "source": [ + "#hide \n", + "import torch\n", + "from matplotlib import pyplot as plt\n", + "from timm.scheduler.step_lr import StepLRScheduler" + ] + }, + { + "cell_type": "code", + "execution_count": 3, + "metadata": {}, + "outputs": [ + { + "data": { + "text/markdown": [ + "

    class MultiStepLRScheduler[source]

    \n", + "\n", + "> MultiStepLRScheduler(**`optimizer`**:`Optimizer`, **`decay_t`**:`List`\\[`int`\\], **`decay_rate`**:`float`=*`1.0`*, **`warmup_t`**=*`0`*, **`warmup_lr_init`**=*`0`*, **`t_in_epochs`**=*`True`*, **`noise_range_t`**=*`None`*, **`noise_pct`**=*`0.67`*, **`noise_std`**=*`1.0`*, **`noise_seed`**=*`42`*, **`initialize`**=*`True`*) :: `Scheduler`\n", + "\n", + " " + ], + "text/plain": [ + "" + ] + }, + "metadata": {}, + "output_type": "display_data" + } + ], + "source": [ + "from nbdev.showdoc import show_doc\n", + "from timm.scheduler.multistep_lr import MultiStepLRScheduler\n", + "show_doc(MultiStepLRScheduler)" + ] + }, + { + "cell_type": "code", + "execution_count": 4, + "metadata": {}, + "outputs": [], + "source": [ + "#hide\n", + "model = torch.nn.Linear(1, 1) # Simple dummy model to create dummy optimizer\n", + "lr = 0.1\n", + "optimizer = torch.optim.Adam(model.parameters(), lr=lr)\n", + "t_initial = 50" + ] + }, + { + "cell_type": "code", + "execution_count": 5, + "metadata": {}, + "outputs": [], + "source": [ + "#hide\n", + "def calculate_lr(scheduler: MultiStepLRScheduler, num_epoch):\n", + " return [scheduler.get_epoch_values(epoch) for epoch in range(num_epoch)]" + ] + }, + { + "cell_type": "code", + "execution_count": 6, + "metadata": {}, + "outputs": [], + "source": [ + "#hide\n", + "def calculate_noisy_lr(scheduler: MultiStepLRScheduler, num_epoch):\n", + " lr_list = []\n", + " for epoch in range(num_epoch):\n", + " lr_list.append(scheduler.optimizer.param_groups[0]['lr'])\n", + " scheduler.step(epoch)\n", + " return lr_list" + ] + }, + { + "cell_type": "code", + "execution_count": 7, + "metadata": {}, + "outputs": [], + "source": [ + "#hide\n", + "def plot_lr(scheduler, num_epochs=50, label=''):\n", + " plt.plot(calculate_lr(scheduler, num_epochs), label=label)" + ] + }, + { + "cell_type": "code", + "execution_count": 8, + "metadata": {}, + "outputs": [], + "source": [ + "#hide\n", + "def plot_noisy_lr(scheduler, num_epochs=50, label=''):\n", + " plt.plot(calculate_noisy_lr(scheduler, num_epochs),label=label)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "The schedule looks something like:" + ] + }, + { + "cell_type": "code", + "execution_count": 9, + "metadata": {}, + "outputs": [ + { + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXoAAAD4CAYAAADiry33AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/YYfK9AAAACXBIWXMAAAsTAAALEwEAmpwYAAAn1klEQVR4nO3deXxU5b3H8c8vCWTCkoQlVJIBAxc3lhAwIohVKLKIAvWKV5AqddeW2rrT3lulXmxrXypK4XVvubUKWilWi9KKtYpQ3CEgooALIspWhEiQRZaQ5/4xkzEkASYhZybn5Pt+vfKamXPOzPkdGL55eM45z2POOUREJLhSkl2AiIh4S0EvIhJwCnoRkYBT0IuIBJyCXkQk4NKSXUBVbdu2dfn5+ckuQ0TEV5YtW7bdOZdT07oGF/T5+fkUFxcnuwwREV8xs8+OtE5dNyIiAaegFxEJOAW9iEjANbg+ehGvHTx4kI0bN7Jv375klyJSa6FQiHA4TJMmTeJ+j4JeGp2NGzfSsmVL8vPzMbNklyMSN+ccJSUlbNy4kU6dOsX9vri6bsxsmJl9aGZrzWxiDevPMbPlZlZmZqOrrBtvZh9Hf8bHXZmIR/bt20ebNm0U8uI7ZkabNm1q/b/RYwa9maUC04Hzga7AWDPrWmWzz4HvA09WeW9r4G7gTKAPcLeZtapVhSIeUMiLX9XluxtP100fYK1zbl10J38CRgGrKzZwzq2Priuv8t6hwEvOuS+j618ChgGza13pMXyx6VPWvfDb+v7YBmVfWgveaT8GZ6me7SPcuhn/UdTBs88XkcSLJ+jzgA2VXm8k0kKPR03vzau6kZldB1wH0LFjxzg/+nClWz+nz4Y/1Om9fpBikXkDpnzUlpXu3zzZR8XUBMO6n0BmKP4TPVJ7Zsa4ceN44oknACgrK6N9+/aceeaZ/O1vfzvqe1u0aMHu3btZv349b7zxBpdddhkAxcXFzJo1i6lTp7J161auvvpqNmzYwMGDB8nPz2f+/PnV3lMXFTc1tm3bNrbsscce4/bbbycvL499+/Zx/fXXc/PNN8f9mStWrGDz5s0MHz68znXV1aRJk2jRogW33XbbUbf7z//8T2bNmsWOHTvYvXt3bHnlYweYMGEC11xzzVE/a9iwYWzZsoWysjK+/e1vM336dFJTU5k0aRL/93//R05O5AbXX/7yl/XyZ9IgTsY652YAMwCKiorqNBPKyb3Phd6l9VlWw/LZm/DoMJ67qht0GeTJLuYs/Zw7n3mPr74+qKD3WPPmzXn//ff5+uuvycjI4KWXXooFRbzWr1/Pk08+GQvtoqIiioqKALjrrrsYPHgwP/7xjwFYuXJlje+pT5deeinTpk2jpKSEU045hdGjR9OhQ3z/O1yxYgXFxcVJCfp4jRgxggkTJnDSSSdVW1dx7PF66qmnyMzMxDnH6NGj+fOf/8yYMWMAuPnmm4/5S6e24jkZuwmo/LcVji6Lx/G8VyoLZUYe93/l2S5aRsN9174yz/Yh3xg+fDjPP/88ALNnz2bs2LGxdZMmTeL++++Pve7evTvr168/7P0TJ07k1VdfpbCwkClTprBo0SIuvPBCALZs2UI4HI5tW1BQUON7Dh06xO23384ZZ5xBQUEBv/vd7wBYtGgR55xzDhdccAGnnHIKN9xwA+XlVXtma9amTRu6dOnCli1bqq1bsmQJ/fr1o1evXpx11ll8+OGHHDhwgLvuuos5c+ZQWFjInDlzDntPXWqcPXs2PXr0oHv37tx5552xz/r73/9O79696dmzJ4MGfdNgWr16NQMGDKBz585MnTq1xuPq27cv7du3j+vP4FgyMyP/nsvKyjhw4IDn54ziadEvBU4ys05EQnoMEG9z4EXgl5VOwA4BflrrKgXSo0G/z8ugj3wdGlPQ/+Kvq1i9uX7/TLvmZnL3iG7H3G7MmDHcc889XHjhhaxcuZKrrrqKV199Ne79/PrXv+b++++PdfUsWrQotu6HP/xhrJV53nnnceWVV5Kbm1vtPTNmzCArK4ulS5eyf/9++vfvz5AhQ4BIKK9evZoTTzyRYcOG8Ze//IXRo0dXq6Oqzz//nH379sV+uVR26qmn8uqrr5KWlsbLL7/Mz372M5555hnuueceiouLa2wVP/LII7Wq8ayzzuLOO+9k2bJltGrViiFDhvDss8/Sv39/rr32WhYvXkynTp348ssvY/v44IMPWLhwIbt27eKUU07hxhtvrNV16s888wyLFy/m5JNPZsqUKXH9T2bo0KEsWbKE888//7A/12nTpjFr1iyKiop44IEHaNXq+K9fOWaL3jlXBkwgEtprgKecc6vM7B4zGwlgZmeY2UbgEuB3ZrYq+t4vgf8m8stiKXBPxYlZqaWEtugPerYP+UZBQQHr169n9uzZ9d5lMXToUNatW8e1117LBx98QK9evdi2bVu17f7xj38wa9YsCgsLOfPMMykpKeHjjz8GoE+fPnTu3JnU1FTGjh3La6+9dtR9zpkzh4KCArp06cIPfvADQqFQtW127tzJJZdcQvfu3bn55ptZtWrVMY+ltjUuXbqUAQMGkJOTQ1paGuPGjWPx4sW89dZbnHPOObHrz1u3bh3bxwUXXEB6ejpt27alXbt2bN269Zh1VRgxYgTr169n5cqVDB48mPHj47uK/MUXX2TLli3s37+fV155BYAbb7yRTz75hBUrVtC+fXtuvfXWuOs4mrj66J1z84H5VZbdVen5UiLdMjW99w9AcM+SJkrTloCpRV/P4ml5e2nkyJHcdtttLFq0iJKSktjytLS0w7pK6nIXb+vWrbnsssu47LLLuPDCC1m8eDFt2rQ5bBvnHL/97W8ZOnToYcsXLVpUrTvhWN0LFf+DKC4uZsiQIYwcOZITTjjhsG1+/vOfM3DgQObOncv69esZMGDAMY+jPms8kvT09Njz1NRUysri/zdQ+c/0mmuu4Y477oj7vaFQiFGjRvHcc88xePBgvvWtb8XWXXvttbGuuOOlsW78IiUF0lt63KKvCHq16BPlqquu4u6776ZHjx6HLc/Pz2f58uUALF++nE8//bTae1u2bMmuXbtq/NxXXnmFvXv3ArBr1y4++eQTOnbsWO09Q4cO5X/+5384eDDyd/7RRx+xZ88eINIt8umnn1JeXs6cOXM4++yz4zqmoqIiLr/8ch5++OFq63bu3Bk76fzYY4/FdSy1rbFPnz7885//ZPv27Rw6dIjZs2dz7rnn0rdvXxYvXhz7s6zcdXM8Kp+LmDdvHqeddlrs9amnnlpt+927d8feU1ZWxvPPPx/brvJnzZ07l+7du9dLjQp6P0nP9LRFX3GlzVeNqEWfbOFwmJtuuqna8osvvpgvv/ySbt26MW3aNE4++eRq2xQUFJCamkrPnj2ZMmXKYeuWLVtGUVERBQUF9OvXj2uuuSZ2MrPye6655hq6du1K79696d69O9dff32sNXvGGWcwYcIETjvtNDp16sRFF1102L7D4TDhcJhbbrmlWm133nknjz76aLXwvuOOO/jpT39Kr169Dms1Dxw4kNWrV9d4Mra2NbZv355f//rXDBw4kJ49e3L66aczatQocnJymDFjBv/+7/9Oz549ufTSS4/111Ot9nA4zN69ewmHw0yaNAmAqVOn0q1bN3r27MnUqVNjv8C2b9+Oc9UvItyzZw8jR46koKCAwsJC2rVrxw033BDbR48ePSgoKGDhwoXV/l7rzDnXoH5OP/10J0cwva9zsy/z7OPLy8tdl5897341f41n+2gIVq9enewSGryFCxe6Cy64INllHFVDr/Gvf/2re/jhhz357Jq+w0CxO0KuNojr6CVO6Zmwb6dnH29mtAw1UdeNSD2or/71+qCg95NQJuyO/2qAumgZSmtUJ2OlZgMGDIjrRGky+aHGhkJ99H4SyvK0jx4qgl4tepEgUdD7icddNxA5IasWvUiwKOj9JJQZubyyhjP59aVlKI2v1KIXCRQFvZ+kZ0J5GRz82rNdtFSLXiRwFPR+kpBhEHQyNhHuvfdeunXrFruW+u233wbgoYceit3oVBdVB0OrkJqaSmFhId27d2fEiBGUlpbWeR9emTZtGl26dMHM2L59e2y5c46bbrqJLl26UFBQELuRLJEqDxjnRwp6P0nPijx6OgxCE3bvL+NQuXfdQ43dm2++yd/+9jeWL1/OypUrefnll2ODYB1v0B9JRkYGK1as4P3336d169ZMnz693vdxJIcOHYpru/79+/Pyyy9z4oknHrb8hRde4OOPP+bjjz9mxowZ3HjjjV6UGWgKej9JQIs+MzoMwu79atV7ZcuWLbRt2zY2vkrbtm3Jzc1l6tSpbN68mYEDBzJw4EAgMqBXv3796N27N5dccklswov8/PzYXZR9+vRh7dq1ce+/X79+bNpU82jhTzzxBH369KGwsJDrr7+eQ4cO8b//+7/cfvvtsW0ee+wxJkyYcMTtITI5yq233krPnj259957+e53vxt7/0svvXTYXbYVevXqRX5+frXlzz33HFdccQVmRt++fSktLa1xCOSj1XLzzTfTrVs3Bg0aFBvcbcWKFfTt25eCggIuuugiduzYAcDatWs577zz6NmzJ7179+aTTz4BIkMXjB49mlNPPZVx48bF7nqdOHEiXbt2paCgoN7Hka8vCno/iQ1V7N2VN41uvJsXJsKjF9TvzwsTj7rLIUOGsGHDBk4++WR+8IMf8M9//hOAm266idzcXBYuXMjChQvZvn07kydP5uWXX2b58uUUFRXx4IMPxj4nKyuL9957jwkTJvCTn/wkrsM9dOgQCxYsYOTIkdXWrVmzhjlz5vD666+zYsUKUlNT+eMf/8jFF1/M3LlzY9vNmTOHMWPGHHF7iNzmf+aZZ/Luu+/y85//nA8++CAWsI8++ihXXXVVXPUCbNq06bBhf8PhcLVfVMeqpaioiFWrVnHuuefyi1/8AoArrriC++67j5UrV9KjR4/Y8nHjxvHDH/6Qd999lzfeeCM2Bv0777zDQw89xOrVq1m3bh2vv/46JSUlzJ07l1WrVrFy5Ur+67/+K+7jSiQFvZ9o8pFAaNGiBcuWLWPGjBnk5ORw6aWXHjbAV4W33nqL1atX079/fwoLC5k5cyafffZZbH3FRCVjx47lzTffPOo+v/76awoLCznhhBPYunUrgwcPrrbNggULWLZsGWeccQaFhYUsWLCAdevWkZOTQ+fOnXnrrbcoKSnhgw8+oH///kfcHiLnBC6++GIgcsf15ZdfzhNPPEFpaSlvvvkm559/fl3/+Gp0tFpSUlJi49p873vf47XXXmPnzp2UlpZy7rnnAjB+/HgWL17Mrl272LRpU+x/HKFQiGbNmgGRIZHD4TApKSkUFhayfv16srKyCIVCXH311fzlL3+JbdvQ6M5YP9HkI/Xv/F8nZbepqamxOzt79OjBzJkz+f73v3/YNs45Bg8ezOzZs2v8jMpD8h5reN6KPvq9e/cydOhQpk+fXm0wNecc48eP51e/+lW1948ZM4annnqKU089lYsuuggzO+r2oVCI1NRvJrG/8sorGTFiBKFQiEsuuYS0tPijJy8vjw0bvpl6euPGjdWmXTxaLVXV51DGaWlpLFmyhAULFvD0008zbdq02NjyDYla9H6iyUcC4cMPP4xNnAGRvuKKE5CVh+vt27cvr7/+eqz/fc+ePXz00Uex91WM8jhnzhz69esX176bNWvG1KlTeeCBB6qNuT5o0CCefvppvvjiCyAyjG/F/yAuuuginnvuOWbPnh2b2/Ro21eVm5tLbm4ukydP5sorr4yr1gojR45k1qxZOOd46623yMrKqjal39FqKS8v5+mnnwbgySef5OyzzyYrK4tWrVrFZvR6/PHHOffcc2nZsiXhcJhnn30WgP379x/15Pju3bvZuXMnw4cPZ8qUKbz77ru1OrZEUYveTzT5SCDs3r2bH/3oR5SWlpKWlkaXLl2YMWMGANdddx3Dhg2L9dU/9thjjB07lv379wMwefLk2JDFO3bsoKCggPT09MNa/ZMnT+ahhx6Kvd64ceNh++/VqxcFBQXMnj2byy+/PLa8a9euTJ48mSFDhlBeXk6TJk2YPn06J554Iq1ateK0005j9erV9OnT55jb12TcuHFs27btsPHaK5s6dSq/+c1v+Ne//kVBQQHDhw/n97//PcOHD2f+/Pl06dKFZs2a8eijj1Z779Fqad68OUuWLGHy5Mm0a9cu9gty5syZ3HDDDezdu5fOnTvHPvfxxx/n+uuv56677qJJkyb8+c9/PuLf5a5duxg1ahT79u3DOXfYOZSGxJyHd1nWRVFRkSsuLk52GQ3XrzpCzzEw/DeefPwXu/bR594F/PeoblzeL9+TfSTbmjVrjhg2fpGfn09xcTFt27ZNdilxmzBhAr169eLqq69O6H5btGgRu1opKGr6DpvZMudcUU3bq0XvNxXDIHhEk4+IF04//XSaN2/OAw88kOxSGiUFvd94PMtUeloKTVNT1HXTwK1fvz7ZJdTKsmXLkrbvoLXm60InY/3G4xZ9ZPKR4A9s1tC6LEXiVZfvroLebxIwVHHQx7sJhUKUlJQo7MV3nHOUlJQQCoVq9T513fhNKBO2f+jpLoI+nWA4HGbjxo2xOzVF/CQUChEOh2v1HgW933jcRw/Bb9E3adKETp06JbsMkYRR143fJGjykSC36EUaGwW932jyERGpJQW932jyERGpJQW934SyI4+afERE4qSg95t0TT4iIrWjoPebiq6bfaWe7aLRTT4iEnAKer9JyJj0mnxEJEgU9H6TkHljFfQiQaKg95uEzjKlrhuRIIgr6M1smJl9aGZrzazazMdmlm5mc6Lr3zaz/OjyJmY208zeM7M1ZvbTeq6/8WnaAjDPL68EAj+wmUhjccygN7NUYDpwPtAVGGtmXatsdjWwwznXBZgC3BddfgmQ7pzrAZwOXF/xS0DqKCXF82EQ1EcvEizxtOj7AGudc+uccweAPwGjqmwzCpgZff40MMgiM/A6oLmZpQEZwAHA24FaGgOPhyrWdIIiwRJP0OcBGyq93hhdVuM2zrkyYCfQhkjo7wG2AJ8D9zvnvqy6AzO7zsyKzaxYIwrGweMWfahJKk1TU9R1IxIQXp+M7QMcAnKBTsCtZta56kbOuRnOuSLnXFFOTo7HJQWAxy160DAIIkEST9BvAjpUeh2OLqtxm2g3TRZQAlwG/N05d9A59wXwOlDj5LVSC5p8RERqIZ6gXwqcZGadzKwpMAaYV2WbecD46PPRwCsuMn3P58B3AMysOdAX+KA+Cm/UQlkJaNEHe/IRkcbkmEEf7XOfALwIrAGecs6tMrN7zGxkdLNHgDZmtha4Bai4BHM60MLMVhH5hfGoc25lfR9EoxNSi15E4hfXDFPOufnA/CrL7qr0fB+RSymrvm93TcvlOFWcjHUOzDzZRctQGtu37/Hks0UksXRnrB+FMsEdgoN7PduFJh8RCQ4FvR8laBgEBb1IMCjo/SiUFXn0eGAzTT4iEgwKej9K4MBmmnxExP8U9H4UG6rYuytvKoYq/uprXWIp4ncKej9K6FDFatGL+J2C3o8SMPnINyNYqkUv4ncKej9Si15EakFB70dNW4ClJGao4v1q0Yv4nYLej1JSIL2lp8MgaPIRkeBQ0PtVepa6bkQkLgp6v/J4THpNPiISHAp6v/J4linQMAgiQaGg96tQpqc3TIGCXiQoFPR+lZAWvSYfEQkCBb1fJWDe2MwMtehFgkBB71eVJx/xSMv0JhrrRiQAFPR+lZDJR9SiFwkCBb1fJWQYBPXRiwSBgt6vEjD5SMtQGnsOHNLkIyI+p6D3q4qgT8TkI+q+EfE1Bb1fxbpuEjD5iLpvRHxNQe9XCZhlSuPdiASDgt6vEnQyFjT5iIjfKej9KiGzTKlFLxIECnq/qph8JBFDFWvyERFfU9D7lVlk8pGEzBurFr2Inyno/UyTj4hIHBT0fpaIyUfSNPmIiN8p6P0sAUMVZ4bS+OprtehF/ExB72cJmXxE492I+J2C3s9C3vbRg0awFAkCBb2fpWd6OgQCVAS9WvQifhZX0JvZMDP70MzWmtnEGtanm9mc6Pq3zSy/0roCM3vTzFaZ2XtmFqrH+hu3UCbs3+X55CNq0Yv42zGD3sxSgenA+UBXYKyZda2y2dXADudcF2AKcF/0vWnAE8ANzrluwABAzcP6kh6dfOTAHs92oa4bEf+Lp0XfB1jrnFvnnDsA/AkYVWWbUcDM6POngUFmZsAQYKVz7l0A51yJc+5Q/ZQuiRkGQSdjRfwunqDPAzZUer0xuqzGbZxzZcBOoA1wMuDM7EUzW25mdxx/yRKTkIHNNPmIiN95fTI2DTgbGBd9vMjMBlXdyMyuM7NiMyvetm2bxyUFSIJmmQJNPiLiZ/EE/SagQ6XX4eiyGreJ9stnASVEWv+LnXPbnXN7gflA76o7cM7NcM4VOeeKcnJyan8UjVUCWvSafETE/+IJ+qXASWbWycyaAmOAeVW2mQeMjz4fDbzinHPAi0APM2sW/QVwLrC6fkoXTT4iIvFIO9YGzrkyM5tAJLRTgT8451aZ2T1AsXNuHvAI8LiZrQW+JPLLAOfcDjN7kMgvCwfMd84979GxND6afERE4nDMoAdwzs0n0u1SedldlZ7vAy45wnufIHKJpdS3BFx1k5mhFr2I3+nOWD9LyOQj6qMX8TsFvZ+ZeT4MgvroRfxPQe93Ho9J/03Qq0Uv4lcKer/zeJap9LTI5CNq0Yv4l4Le7zxu0UN08hEFvYhvKej9LgGzTGm8GxF/U9D7XUJmmdIIliJ+pqD3u4S06DX5iIifKej9TpOPiMgxKOj9TpOPiMgxKOj9TpOPiMgxKOj9LoGTj5QdKvdsHyLiHQW934WyI4+eDmwWGe9m935134j4kYLe7yq6bjTejYgcQVzDFEsDVtF189Um2LPdk120sa9ozVds2bKB5mUtPNnHUaU2xVUcp9SrpmkpsRFKJbjMeXhZXl0UFRW54uLiZJfhH7u/gPtPSnYVnip3xugDd7PcnZzsUgInxWDehLPpnpeV7FLkOJnZMudcUU3r1KL3uxbt4NInYNe/PNvFofJyVm7cyYEknIxNO/Q1p3/0EHcUHuSjjt0Svv8g27n3IA+89BFrtnyloA84BX0QnDbC049PBXp5uoejKC+HydPp22YvffvlJ6uKQNpfdogHXvqIzaX7kl2KeEwnY6VhS0mBrDwo3ZDsSgInPS2VnJbpbCrdm+xSxGMKemn4sjrAzo3JriKQcrMz1KJvBBT00vAp6D0Tzs5gc+nXyS5DPKagl4YvKwy7NsMhXcdf33KzQ2wq/ZqGdvWd1C8FvTR8WWFw5bBrS7IrCZzc7Az2l5Xz5Z4DyS5FPKSgl4YvKxx53KkTsvUtNzsDQP30Aaegl4Yvq0PkUf309S4vGvS68ibYFPTS8GXlRR7Voq93ubGgV4s+yBT00vA1bQ7N2qhF74FWzZqQ0SRVV94EnIJe/CErrKD3gJmRmx1S0Aecgl78QdfSeyZX19IHnoJe/CErHBkGQdd717u87Az10Qecgl78ISsMB3Z5OsFKY5WbncH23fvZd/BQsksRjyjoxR9i19Kr+6a+VVx5s2WnWvVBpaAXf9C19J7JzQ4BqJ8+wBT04g+xoNe19PUtnN0MgE0K+sCKK+jNbJiZfWhma81sYg3r081sTnT922aWX2V9RzPbbWa31VPd0tg0z4HUpgp6D3wrKx0zteiD7JhBb2apwHTgfKArMNbMulbZ7Gpgh3OuCzAFuK/K+geBF46/XGm0UlIgM09dNx5IT0slp0W6gj7A4mnR9wHWOufWOecOAH8CRlXZZhQwM/r8aWCQmRmAmX0X+BRYVS8VS+Olm6Y8k5udoa6bAIsn6POAyv9f3hhdVuM2zrkyYCfQxsxaAHcCvzjaDszsOjMrNrPibdu2xVu7NDa6acozeZppKtC8Phk7CZjinNt9tI2cczOcc0XOuaKcnByPSxLfygpHxqQ/dDDZlQSOJiAJtrQ4ttkEdKj0OhxdVtM2G80sDcgCSoAzgdFm9hsgGyg3s33OuWnHW7g0QtkdvpmAJLtjsqsJlLzsDA6UlVOy5wBtW6QnuxypZ/G06JcCJ5lZJzNrCowB5lXZZh4wPvp8NPCKi/i2cy7fOZcPPAT8UiEvdVZx01Sprrypb99MQKJ++iA6ZtBH+9wnAC8Ca4CnnHOrzOweMxsZ3ewRIn3ya4FbgGqXYIocN9005RkFfbDF03WDc24+ML/KsrsqPd8HXHKMz5hUh/pEvpGpCUi8UjHT1MYdCvog0p2x4h9Nm2kCEo9kxyYg0ZU3QaSgF3/RtfSe0AQkwaagF3/J6qCuG4/ktWrG5p0K+iBS0Iu/VNw0peu9612eWvSBpaAXf8kKw4HdsK802ZUETm5WBtt3H9AEJAGkoBd/0QQkntEEJMGloBd/0bX0nqkI+k26xDJwFPTiL2rReyZPN00FloJe/KV5DqSmQ+nnya4kcE7ICmGmmaaCSEEv/pKSAlmagMQLTdNSaNdSE5AEkYJe/Ec3TXkmNztD19IHkIJe/EcTkHgmVxOQBJKCXvxHE5B4Ji86pWB5uW5ICxIFvfhPVhhw8FXV+W/keOVmhWITkEhwKOjFf3QtvWfyWjUDdIll0CjoxX8U9J7JzQ4BCvqgUdCL/2RpAhKvVNw0pWvpg0VBL/7TJAOatVWL3gNZGU1o1lQTkASNgl78SdfSeyIyAUkGm0r3JrsUqUcKevGnrDCUquvGC7qWPngU9OJP2R01AYlH8rIzdDI2YBT04k9ZYTi4B77ekexKAicvO0TJHk1AEiQKevEnDVfsmVwNVxw4CnrxJwW9Z74JevXTB0VasgsQqZOKm6bW/BX27UxuLV5JSYNTzof0FgndbcW19M+/t4WtXynsEyncKoMzO7ep989V0Is/Nc+B5u3g3ScjP0F13iQ4++aE7vKErBBZGU2YveRzZi/RBC+JdGFBewW9SIwZ/GgZ7C1JdiXe+eMl8NkbCQ/6JqkpvHrnQEr3aHTQRMtomurJ5yroxb9CmZGfoMrvD+/PhfJDkOJNABxJZqgJmaEmCd2neEcnY0Uaqo79YP9O+GJNsisRn1PQizRUHftFHj9/M7l1iO8p6EUaquyO0DJXQS/HTUEv0lCZQce+8NmbGupBjouCXqQhO/Es2LVZY+/LcYkr6M1smJl9aGZrzWxiDevTzWxOdP3bZpYfXT7YzJaZ2XvRx+/Uc/0iwdaxb+TxM3XfSN0dM+jNLBWYDpwPdAXGmlnXKptdDexwznUBpgD3RZdvB0Y453oA44HH66twkUahXVdIz1Q/vRyXeFr0fYC1zrl1zrkDwJ+AUVW2GQXMjD5/GhhkZuace8c5tzm6fBWQYWbp9VG4SKOQkgodzoTP30p2JeJj8QR9HlC5g3BjdFmN2zjnyoCdQNX7eC8Gljvn9lfdgZldZ2bFZla8bdu2eGsXaRw69oVta2Dvl8muRHwqISdjzawbke6c62ta75yb4Zwrcs4V5eTkJKIkEf+ouJ5+w9vJrUN8K56g3wR0qPQ6HF1W4zZmlgZkASXR12FgLnCFc+6T4y1YpNHJOx1Sm6qfXuosnqBfCpxkZp3MrCkwBphXZZt5RE62AowGXnHOOTPLBp4HJjrnXq+nmkUalyYhyO2lK2+kzo4Z9NE+9wnAi8Aa4Cnn3Cozu8fMRkY3ewRoY2ZrgVuAikswJwBdgLvMbEX0p129H4VI0HXsB5vfgYOa9Ulqz1wDu+OuqKjIFRcXJ7sMkYblw7/D7Evh+/Mjo1qKVGFmy5xzRTWt052xIn7QoU/k8fM3kluH+JKCXsQPmrWO3Dyl6+mlDhT0In7RsS9sWBKZiESkFhT0In7RsR/s/wq2rkp2JeIzCnoRv4hNRKLuG6kdBb2IX2R3gMywbpySWlPQi/hJx76RoG9gl0VLw6agF/GTE/vBri1Q+lmyKxEfSUt2ASJSCxX99J+9AVkdjr6tn6WkJruCQFHQi/hJzmkQyoZnb4z8BNV5k+Dsm5NdRWAo6EX8JCUFLn4ENi9PdiXe+fgf8NpDcMY1kN4y2dUEgoJexG9OOi/yE1T/Ngh+/x1Y9hic9aNkVxMIOhkrIg1L+HTodA68OR3Kqk1IJ3WgoBeRhufsWyJXF707O9mVBIKCXkQans4DoH0hvP6wxvapBwp6EWl4zODbt8CX62D1c8muxvcU9CLSMJ06AtqcBK89qDuBj5OCXkQappQU6P9j+Nd78MmCZFfjawp6EWm4Ci6FzDx4dUqyK/E1Bb2INFxpTaHfBPjstcikK1InCnoRadh6XwEZreDVB5NdiW8p6EWkYUtvAWfeAB+9AFtXJ7saX9IQCCLS8PW5Dl6fCrNGRSZKD6ou58HQe+v9YxX0ItLwNWsNFzwQadUHWWauJx+roBcRfygcG/mRWlMfvYhIwCnoRUQCTkEvIhJwCnoRkYBT0IuIBJyCXkQk4BT0IiIBp6AXEQk4cw1sQH8z2wZ8dhwf0RbYXk/l+ImOu3HRcTcu8Rz3ic65nJpWNLigP15mVuycK0p2HYmm425cdNyNy/Eet7puREQCTkEvIhJwQQz6GckuIEl03I2LjrtxOa7jDlwfvYiIHC6ILXoREalEQS8iEnCBCXozG2ZmH5rZWjObmOx6vGJmfzCzL8zs/UrLWpvZS2b2cfSxVTJr9IKZdTCzhWa22sxWmdmPo8sDfexmFjKzJWb2bvS4fxFd3snM3o5+3+eYWdNk1+oFM0s1s3fM7G/R143luNeb2XtmtsLMiqPL6vxdD0TQm1kqMB04H+gKjDWzrsmtyjOPAcOqLJsILHDOnQQsiL4OmjLgVudcV6Av8MPo33HQj30/8B3nXE+gEBhmZn2B+4ApzrkuwA7g6uSV6KkfA2sqvW4sxw0w0DlXWOn6+Tp/1wMR9EAfYK1zbp1z7gDwJ2BUkmvyhHNuMfBllcWjgJnR5zOB7yaypkRwzm1xzi2PPt9F5B9/HgE/dhexO/qySfTHAd8Bno4uD9xxA5hZGLgA+H30tdEIjvso6vxdD0rQ5wEbKr3eGF3WWHzLObcl+vxfwLeSWYzXzCwf6AW8TSM49mj3xQrgC+Al4BOg1DlXFt0kqN/3h4A7gPLo6zY0juOGyC/zf5jZMjO7Lrqszt91TQ4eMM45Z2aBvWbWzFoAzwA/cc59FWnkRQT12J1zh4BCM8sG5gKnJrci75nZhcAXzrllZjYgyeUkw9nOuU1m1g54ycw+qLyytt/1oLToNwEdKr0OR5c1FlvNrD1A9PGLJNfjCTNrQiTk/+ic+0t0caM4dgDnXCmwEOgHZJtZRUMtiN/3/sBIM1tPpCv2O8DDBP+4AXDObYo+fkHkl3sfjuO7HpSgXwqcFD0j3xQYA8xLck2JNA8YH30+HnguibV4Ito/+wiwxjn3YKVVgT52M8uJtuQxswxgMJHzEwuB0dHNAnfczrmfOufCzrl8Iv+eX3HOjSPgxw1gZs3NrGXFc2AI8D7H8V0PzJ2xZjacSJ9eKvAH59y9ya3IG2Y2GxhAZNjSrcDdwLPAU0BHIkM8/4dzruoJW18zs7OBV4H3+KbP9mdE+ukDe+xmVkDkxFsqkYbZU865e8ysM5GWbmvgHeB7zrn9yavUO9Gum9uccxc2huOOHuPc6Ms04Enn3L1m1oY6ftcDE/QiIlKzoHTdiIjIESjoRUQCTkEvIhJwCnoRkYBT0IuIBJyCXkQk4BT0IiIB9/+gdCqftSQfxgAAAABJRU5ErkJggg==", + "text/plain": [ + "
    " + ] + }, + "metadata": { + "needs_background": "light" + }, + "output_type": "display_data" + } + ], + "source": [ + "#hide_input\n", + "scheduler = MultiStepLRScheduler(optimizer, decay_t=[15, 35], decay_rate=0.5)\n", + "plot_lr(scheduler, label='MultiStepLR at epoch 15, 35')\n", + "\n", + "scheduler = StepLRScheduler(optimizer, decay_t=10, decay_rate=0.5)\n", + "plot_lr(scheduler, label='StepLR every 10 epochs')\n", + "plt.legend();" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Using `MultiStepLRScheduler` scheduler with `timm` training script" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "To train models using the `MultiStepLRScheduler` we simply update the training script args passed by passing in `--sched multistep` parameter alongside the necessary hyperparams. In this section we will also look at how each of the hyperparams update the `MultiStepLRScheduler` scheduler. " + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "The training command to use `MultiStepLRScheduler` scheduler looks something like: \n", + "\n", + "```python \n", + "python train.py ../imagenette2-320/ --sched multistep\n", + "```" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Availible parameters are: \n", + "\n", + "--epochs - initial number of epoch to train. default 300. \n", + "--lr - learning rate (default: 0.05) \n", + "\n", + "--decay-epochs - epoch interval to decay LR , (default: 100), `decay_t` argument in python script \n", + "--decay-rate - LR decay rate (default: 0.1) \n", + "\n", + "\n", + "warmup parameters: \n", + "--warmup-lr' - warmup learning rate (default: 0.0001) \n", + "--warmup-epochs - epochs to warmup LR, if scheduler supports (default: 3) \n", + "\n", + "noise parameters: \n", + "--lr-noise - learning rate noise on/off epoch percentages \n", + "--lr-noise-pct - learning rate noise limit percent (default: 0.67) \n", + "--seed - random seed (default: 42) to seed noise generator. \n" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Using in python script." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "`MultiStepLRScheduler` accepts two required arguments - an `optimizer` and `decay_t`, and also some hyperparams which we will look into in detail below." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Basic usage like this:" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "\n", + "```python \n", + "from timm.scheduler.poly_lr import MultiStepLRScheduler\n", + "scheduler = MultiStepLRScheduler(optimizer, decay_t=[step_t, step_t1])\n", + "```\n" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Required arguments." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "`optimizer` is object of `torch.optim.Optimizer` \n", + "`decay_t` - list of epochs number to decay LR." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Default schedule:\\\n", + "Default value for `decay_rate` is 1., so no decay.\\\n", + "Default parameter for `decay_rate` at train script `--decay-rate` if is 0.1." + ] + }, + { + "cell_type": "code", + "execution_count": 10, + "metadata": { + "scrolled": true + }, + "outputs": [ + { + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXoAAAD4CAYAAADiry33AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/YYfK9AAAACXBIWXMAAAsTAAALEwEAmpwYAAAoeklEQVR4nO3de3hU1b3/8feXzEgCKshFTyUoCHi4gxIBtXKRnxVvYBVvpU/B00rR+mj1tBbbcwSp9Gih3k5blXq3KlL9KVSxVAV/8rReCBYQESsgHkAOhihIhEAu398fMxlDCDBJZjJ7Zn9ez5MnM3vvmVl7Mnxmsdbaa5m7IyIiuatFpgsgIiLppaAXEclxCnoRkRynoBcRyXEKehGRHBfJdAHq6tChg3fp0iXTxRARySrLli3b5u4d69sXuKDv0qULxcXFmS6GiEhWMbNPDrRPTTciIjlOQS8ikuMU9CIiOU5BLyKS4xT0IiI5LqmgN7PRZvahma01syn17B9mZu+aWaWZjauzb4KZfRT/mZCqgouISHIOGfRmlgf8DjgH6A1cYWa96xz2P8BE4Kk6j20HTAWGAIOBqWZ2VNOLLSIiyUpmHP1gYK27rwcwsznAWGB1zQHuviG+r7rOY88GXnH3z+P7XwFGA083ueR1bN26klkv/oJde6pS/dTNbmeLI9kY7ZLpYjS7iLelXdXwTBdDJGN6H3skUy/ok/LnTSboOwEba93fRKyGnoz6Htup7kFmNgmYBHDccccl+dT7Kin9kIV5H0OrRj08cPzr79FwsNi6CEdUDSBK28yWRSTHBOLKWHefDcwGKCoqatRKKH17X8LK3pektFwZseQ38Np0+I/PINIy06VpNgvWL+BnS37Gby7vSdc2XTNdHJGckkxn7Gagc637hfFtyWjKY8Mpkh/7XbE7s+VoZgWRAgB2Ve7KcElEck8yQb8U6GFmXc3sMOByYH6Sz78Q+JaZHRXvhP1WfJscSE3QV5ZnthzNrFU01ua2O2RfcCLN4ZBB7+6VwLXEAvoDYK67v29m081sDICZnWJmm4BLgAfM7P34Yz8Hfknsy2IpML2mY1YOIBqr2apGLyKpklQbvbsvABbU2XZLrdtLiTXL1PfYh4GHm1DGcAlpjb4m6HdXhusLTqQ56MrYoAlpjT7RdKOgF0k5BX3QJGr0ezJbjmaWaLqpUNONSKop6IMmEfThqtm2iqhGL5IuCvqgidYMrwxXG33LvJYYpqAXSQMFfdDEmzDCVqM3MwoiBRp1I5IGCvqgCWmNHmIdsqrRi6Segj5oQlqjh1iHrDpjRVJPQR800XCOuoFY0KtGL5J6CvqgiYRzHD3ERt4o6EVST0EfNHlRwEJ3ZSygzliRNFHQB41Z7OrYMNbo1RkrkhYK+iCK5Ie3Rq/OWJGUU9AHUbQglMMr1Rkrkh4K+iCK5IdyeKU6Y0XSQ0EfRNGCcA6vjMZq9NVed415EWkKBX0QRVqGszM2PrFZeQj7J0TSSUEfRJGC0HbGgmawFEk1BX0QRfNDWaPXcoIi6aGgD6KQDq/UKlMi6aGgD6KQXjClVaZE0kNBH0QhrdGrjV4kPRT0QRQNZ2eslhMUSQ8FfRBFWob2ylhQZ6xIqinogyhSELsy1j3TJWlW6owVSQ8FfRCFdPGRRBt9CDuiRdJJQR9EIV1OUE03IumhoA+ikC4QHmkR4bAWh6npRiTFFPRBlKjRhyvoITaxmcbRi6SWgj6IIi1jv8MY9JqTXiTlFPRBFNUC4SKSOgr6IIrUjLoJZ41enbEiqZVU0JvZaDP70MzWmtmUeva3NLNn4vvfNrMu8e1RM3vMzN4zsw/M7OYUlz83hblGrwXCRVLukEFvZnnA74BzgN7AFWbWu85h3we+cPfuwF3AHfHtlwAt3b0fMAj4Yc2XgBxEyGv0CnqR1EqmRj8YWOvu6919LzAHGFvnmLHAY/HbzwKjzMwAB1qbWQQoAPYCX6ak5LksxDX6gohG3YikWjJB3wnYWOv+pvi2eo9x90pgB9CeWOh/BWwB/geY5e6f130BM5tkZsVmVlxSUtLgk8g5kXBeGQvqjBVJh3R3xg4GqoBjga7Av5vZCXUPcvfZ7l7k7kUdO3ZMc5GyQCLowxd46owVSb1kgn4z0LnW/cL4tnqPiTfTtAFKge8Af3H3Cnf/DPgbUNTUQue8kF4ZC2qjF0mHZIJ+KdDDzLqa2WHA5cD8OsfMBybEb48DFrm7E2uuORPAzFoDQ4E1qSh4TgvpXDcQG3VTWV1JRVVFposikjMOGfTxNvdrgYXAB8Bcd3/fzKab2Zj4YQ8B7c1sLXAjUDME83fA4Wb2PrEvjEfcfWWqTyLn5EXBWoS2Rg+a2EwklSLJHOTuC4AFdbbdUut2ObGhlHUfV1bfdjkEs/ic9OEL+tqrTLVp2SbDpRHJDboyNqii+aEdXglafEQklRT0QRXJD+XwSjXdiKSegj6oIvmh7YwFrTIlkkoK+qCKFqgzVkRSQkEfVCGt0auNXiT1FPRBFdIafaLpRkEvkjIK+qAKeY1eE5uJpI6CPqgiLUM56qb2OHoRSQ0FfVBFC0I5jr5lXksMU9CLpJCCPqgi+aG8MtbMNIOlSIop6IMqpDV60HKCIqmmoA+qkNboQatMiaSagj6oovFJzdwzXZJmpznpRVJLQR9UIV4gXMsJiqSWgj6oQhz06owVSS0FfVCFeDlBdcaKpJaCPqhCvJxgQaRAs1eKpJCCPqhCXKNX041IainogyrENXp1xoqkloI+qMJco4/GhldWe3WmiyKSExT0QZUYdRO+mm3NDJblIRxxJJIOCvqgSgS9ZrAUkaZR0AdVNN5GH8LRJ1pOUCS1FPRBFeILprTKlEhqKeiDSjV6Bb1IiijogyrENXotJyiSWgr6oErU6MMX9OqMFUktBX1QtYiAtQh3jV6dsSIpEcl0AeQAzGJXxwYk6CsqKti0aRPl5ekvT1V1FXf3vps2ZW344IMP0v56ItkkPz+fwsJCotFo0o9R0AdZND8wnbGbNm3iiCOOoEuXLphZWl+rqroK+9w4pvUxdCjokNbXEskm7k5paSmbNm2ia9euST9OTTdBFqAafXl5Oe3bt097yAO0sNjHUlMgiOzLzGjfvn2D/2edVNCb2Wgz+9DM1prZlHr2tzSzZ+L73zazLrX29TezN83sfTN7z8zyG1TCMAtQjR5olpCveR0zw0O4jKLIoTTm3+Ehg97M8oDfAecAvYErzKx3ncO+D3zh7t2Bu4A74o+NAH8EJrt7H2AEUNHgUoZVgGr0za2FtVCNXiRFkqnRDwbWuvt6d98LzAHG1jlmLPBY/PazwCiLfe18C1jp7isA3L3U3atSU/QQiLQMVI2+ObXg0EE/bdo0Zs2adcD9JSUlDBkyhJNOOoklS5Y0uAyPPvoo1157LQAvvPACq1evbvBziARBMkHfCdhY6/6m+LZ6j3H3SmAH0B44EXAzW2hm75rZTfW9gJlNMrNiMysuKSlp6DnkrmhBKCc1g3iNnqbV6F977TX69evHP/7xD84444wmPZeCXrJZujtjI8A3gfHx3982s1F1D3L32e5e5O5FHTt2THORskgkP5TTFEOsHbK+Gv2MGTM48cQT+eY3v8mHH34IwLp16xg9ejSDBg3ijDPOYM2aNSxfvpybbrqJefPmMXDgQHbv3s3VV19NUVERffr0YerUqYnn7NKlC9u2bQOguLiYESNG7POaf//735k/fz4//elPGThwIOvWrUvfiYukQTLDKzcDnWvdL4xvq++YTfF2+TZAKbHa/xvuvg3AzBYAJwOvNbHc4RDND+SVsbf++X1Wf/plSp+z97FHMvWCPon79bXRL1u2jDlz5rB8+XIqKys5+eSTGTRoEJMmTeL++++nR48evP3221xzzTUsWrSI6dOnU1xczG9/+1sg9iXRrl07qqqqGDVqFCtXrqR///6HLNtpp53GmDFjOP/88xk3blxKz1ukOSQT9EuBHmbWlVigXw58p84x84EJwJvAOGCRu7uZLQRuMrNWwF5gOLHOWklGpCC0NfoW1oLK6sp9ti1ZsoRvf/vbtGoVmyJhzJgxlJeX8/e//51LLrkkcdyePfU3d82dO5fZs2dTWVnJli1bWL16dVJBL5LtDhn07l5pZtcCC4E84GF3f9/MpgPF7j4feAh4wszWAp8T+zLA3b8wszuJfVk4sMDdX0rTueSegNboa9e80yXZUTfV1dW0bduW5cuXH/S4jz/+mFmzZrF06VKOOuooJk6cmBiLHIlEqK6OvVZzXPkr0tySaqN39wXufqK7d3P3GfFtt8RDHncvd/dL3L27uw929/W1HvtHd+/j7n3dvd7OWDmAkNfo6wb9sGHDeOGFF9i9ezc7d+7kz3/+M61ataJr16786U9/AmJXDq5YsWK/5/vyyy9p3bo1bdq0YevWrbz88suJfV26dGHZsmUAPPfcc/WW54gjjmDnzp2pOj2RZqUrY4Ms0jKQNfrmUN/wypNPPpnLLruMAQMGcM4553DKKacA8OSTT/LQQw8xYMAA+vTpw7x58/Z7vgEDBnDSSSfRs2dPvvOd73D66acn9k2dOpXrr7+eoqIi8vLy6i3P5ZdfzsyZMznppJPUGStZx4J29WFRUZEXFxdnuhjBsPhX8P/ugKnbY5OcZdAHH3xAr169mu31tn61ldLyUnq3r3ttnojU9+/RzJa5e1F9x6tGH2QhXnykhbXA3XV1rEgKKOiDLMTLCWpiM5HUUdAHWchr9IAmNhNJAQV9kKlGrxq9SAoo6IMs0jL2O4Q1eiPW+dzU+W5EREEfbPG1U8MY9KrRi6SOgj7IovE2+hCOpVfQi6SOgj7IEjV6tdHXdai56IPi7rvvZteuXU1+Hnfnuuuuo3v37vTv359333233uN+8Ytf0LlzZw4//PBGvc6IESM41HUsS5YsoU+fPolZQRtq4sSJPPvss0Dj3p9HH32UTz/9tMH7Dub+++/n8ccfb/DjGqO4uJjrrrvuoMds2LCBp556KmWvqaAPMtXoA1+jd/fEPDn1SVXQv/zyy3z00Ud89NFHzJ49m6uvvrre4y644ALeeeedJr/ewTz55JPcfPPNLF++nIKCgiY9V3MGfVXVgdc8mjx5Mt/73vcaVI7GqKyspKioiHvvvfegx6U66JOZvVIyJaht9C9Pgf99L7XP+S/94JzbE3frC/oZM2bw2GOPcfTRR9O5c2cGDRoExOaj/9GPfkRJSQmtWrXiD3/4Az179mTr1q1MnjyZ9etjUy/dd999nHbaaVx44YVs3LiR8vJyrr/+eiZNmsTDDz/MypUrufvuuwH4wx/+wOrVq7nrrv0nW92wYQNnn302Q4YMYdmyZSxYsIDbb7+dpUuXsnv3bsaNG8ett97Kvffey6effsrIkSPp0KEDixcv5q9//StTp05lz549dOvWjUceeSSp2ve8efP43ve+h5kxdOhQtm/fzpYtW/jGN76xz3FDhw5t0Nu+e/durrzySlasWEHPnj33qaHXV9Y5c+Ywd+5cFi5cyMsvv8wDDzzA2LFj+eKLL6ioqOC2225j7NixbNiwgfPPP59Vq1YBMGvWLMrKypg2bVri+et7f2qbPn06f/7zn9m9ezennXYaDzzwAM899xzFxcWMHz+egoIC3nzzzcSXzbPPPrvfvl69enHZZZfxyiuvcNNNN7Fz505mz57N3r176d69O0888QStWrVi2rRpHH744fzkJz9hxIgRDBkyhMWLF7N9+3Yeeuih/Rau2bJlC5dddhlffvkllZWV3HfffZxxxhn85S9/4ec//zlVVVV06NCB1157jWnTprFu3TrWr1/Pcccdxw9/+ENmzZrFiy++mNi3du1atm3bxk033cRVV13FlClT+OCDDxg4cCATJkzghhtuaNDfdT/uHqifQYMGucR9vsF96pHuyx7PdEl89erVX99Z8DP3h89N7c+Cn+3zelXVVb6qZJV/9tVn7u5eXFzsffv29a+++sp37Njh3bp185kzZ7q7+5lnnun//Oc/3d39rbfe8pEjR7q7+6WXXup33XWXu7tXVlb69u3b3d29tLTU3d137drlffr08W3btvnOnTv9hBNO8L1797q7+6mnnuorV66s9734+OOP3cz8zTffTGyrec7KykofPny4r1ixwt3djz/+eC8pKXF395KSEj/jjDO8rKzM3d1vv/12v/XWW93d/cc//rEPGDBgv5//+q//cnf38847z5csWZJ4vTPPPNOXLl16wL9X69atD7ivtt/85jd+5ZVXurv7ihUrPC8vz5cuXXrQsk6YMMH/9Kc/ubt7RUWF79ixI3F+3bp18+rqav/444+9T58+ideZOXOmT506db/H135/6qp5T93dv/vd7/r8+fPd3X348OEHPPe6+44//ni/4447Eve3bduWuP2LX/zC7733Xnd3nzp1auLzNHz4cL/xxhvd3f2ll17yUaNG7fc6s2bN8ttuu83dY3/zL7/80j/77DMvLCz09evX71P+qVOn+sknn+y7du1yd/fFixf7eeedl9jXv39/37Vrl5eUlHhhYaFv3rx5n2Pqs8+/xzhiswnXm6uq0QdZNKA1+lo173SpGV7pxC6Yqm8ueoCysrIDzke/aNGiRLtrXl4ebdq0AWI1yeeffx6AjRs38tFHHzF06FDOPPNMXnzxRXr16kVFRQX9+vU7YPmOP/74fWrPycx1/9Zbb7F69erEhGp79+7l1FNPBaj3fw7N4Y033ki0F/fv3z9R5oOVtTZ35+c//zlvvPEGLVq0YPPmzWzdujUlZVu8eDG//vWv2bVrF59//jl9+vThggsuaPDzXHbZZYnbq1at4j/+4z/Yvn07ZWVlnH322fU+5qKLLgJg0KBBbNiwYb/9p5xyCv/2b/9GRUUFF154IQMHDuT1119n2LBhdO3aFYB27doljh8zZswBm7nGjh1LQUEBBQUFjBw5knfeeYe2bds2+DwPRkEfZCG+MtbMkpqTPtn56Gu8/vrrvPrqq7z55pu0atWKESNGJOag/8EPfsCvfvUrevbsyZVXXnnQ52ndunXi9sHmuq/N3TnrrLN4+umn99t3ww037Nd0AbFZM6dMmUKnTp3YuPHrpZs3bdpEp051l25OnYOVtbYnn3ySkpISli1bRjQapUuXLpSXl+8zxz80fJ7/8vJyrrnmGoqLi+ncuTPTpk1r9FoBtf9WEydO5IUXXmDAgAE8+uijvP766/U+pmXL2DUseXl5VFZW7rd/2LBhvPHGG7z00ktMnDiRG2+8kaOOOiqpMtRldSYsrHs/FdQZG2SJK2PDF/Sw75z09c1FD3DkkUcecD76UaNGcd999wGxjrgdO3awY8cOjjrqKFq1asWaNWt46623Eq83ZMgQNm7cyFNPPcUVV1yRdDkPNtd97Xnshw4dyt/+9jfWrl0LwFdffcU///lPIFajX758+X4/U6ZMAWI1wscffxx356233qJNmzb7tc8fzPPPP8/NN9+83/Zhw4YlOv1WrVrFypUrD1nW2nbs2MHRRx9NNBpl8eLFfPLJJwAcc8wxfPbZZ5SWlrJnzx5efPHFest1oHn+a0K9Q4cOlJWVJUbpHOwxh9oHsHPnTr7xjW9QUVHBk08+ecDjDuWTTz7hmGOO4aqrruIHP/gB7777LkOHDuWNN97g448/BuDzzz9P6rnmzZtHeXk5paWlvP7665xyyikpX/9AQR9keVGwvFAOr4R9g/5Ac9HDgeejv+eee1i8eDH9+vVj0KBBrF69mtGjR1NZWUmvXr2YMmXKfp2Xl156KaeffvpBa2d1HWyu+0mTJjF69GhGjhxJx44defTRR7niiivo378/p556KmvWrEnqNc4991xOOOEEunfvzlVXXcXvf//7xL6BAwcmbt90000UFhaya9cuCgsLE52f69at48gjj9zvea+++mrKysro1asXt9xyS6KDO9myjh8/nuLiYvr168fjjz9Oz549AYhGo9xyyy0MHjyYs846K7G9rtrvT21t27blqquuom/fvpx99tn7/L0nTpzI5MmT6x3eebB9AL/85S8ZMmQIp59++gHLlIzXX3898Xd/5plnuP766+nYsSOzZ8/moosuYsCAAfs0GR1M//79GTlyJEOHDuU///M/OfbYY+nfvz95eXkMGDAgJc16mo8+6H7VCU6eAKN/ldFiNPd89ABrt6/lsBaHcdyRxzXba55//vnccMMNjBo1qtleszl897vf5a677qJjx46ZLorUUnu0T0NoPvpcE8lXjb4ZbN++nRNPPJGCgoKcC3mAP/7xjwr5EFNnbNBFgrlAeHNoQYtmm9Ssbdu2+7VBl5aW1hv6r732Gu3bt2+Wckluq31dQTop6IMuGu4afWX1/iMemkv79u2THs0jEmRqugm6SAFU7sl0KTKiOZtuRHKZgj7oovmhXHgEFPQiqaKgD7pIfigvmIKvFwgXkaZR0AddtCD0NXqFvUjTKOiDLtIy1DV6qH+q4kzOR799+/Z9LlhKRpcuXejXr19irvfx48fzr//6r/Tt2zcxZwoceN75devWMXDgwAbPM695++uft3/06NGJC+wmT5580CmMc4GCPugiIa7Rxz+eNRObBUVjgh5ik3QVFcWuZxk/fjxr1qzhvffeY/fu3Tz44IPAgeed79atW1aPAPKAzds/d+5cVqxYwapVqygpKUlMoZGrNLwy6KLBa6O/4507WPN5cpfuJ6tnu578bPDP9tlWt0YflPnop0yZkqhhn3XWWcycObPB53vuuecmbg8ePJhNmzYByc87fzBBeZ+CPG9/zXQQlZWV7N27Ny0TiQWJgj7oIgWBC/rmUjvoly1bxpw5c1i+fDmVlZWcfPLJiQCbNGkS999/Pz169ODtt9/mmmuuYdGiRVx33XUMHz6c559/nqqqKsrKygB4+OGHadeuHbt37+aUU07h4osv5tJLL2XGjBnMnDmTaDTKI488wgMPPFBvuW6//XZWrVqVqGHv3Llzv4Upajz11FP07t37gOdYUVHBE088wT333APA5s2b6dy5c2J/YWEhmzdvTjrog/Q+AXz00Uc89thjiTmFZsyYQbt27aiqqmLUqFGsXLmS6667jjvvvJPFixfToUMHtm3bxm233carr75K69atueOOO7jzzju55ZZbDjnLZ0Pev7PPPpt33nmHc845h3HjxiX1/mYrBX3QRYN3ZWzdmne61NSyqr06cPPR13bEEUc0ulnlmmuuYdiwYQf8omiooL1PQZ63f+HChZSXlzN+/HgWLVrEWWedlbLnDhoFfdBFCqBqD1RXQ4twdakks25spuajr62xNfpbb72VkpKSfWrE6Zp3XvP21y8/P5+xY8cyb968nA76pJLDzEab2YdmttbMptSzv6WZPRPf/7aZdamz/zgzKzOzhk3RJl8vEB7C5pvaQR+k+ejrzhVeU6Ov7+dAIf/ggw+ycOFCnn76aVrU+gJPdt75zZs31zsPT5Dep7qCMm9/WVkZW7ZsAWJt9C+99FKTpizOBocMejPLA34HnAP0Bq4ws7qf3u8DX7h7d+Au4I46++8EXkYaLsSrTNWMuqmmOlDz0bdv357TTz+dvn378tOf/rRR5zZ58mS2bt3KqaeeysCBA5k+fTpw8Hnna9uyZQuRyP7/IQ/S+1RXUObt/+qrrxgzZgz9+/dn4MCBHH300UyePDnp88hKB1pMtuYHOBVYWOv+zcDNdY5ZCJwavx0BtvH1XPcXAjOBacBPDvV6Why8jqUPxxYI374po8WobzHidNtbuddXlazy0t2lhz44Rc477zx/9dVXU/68B1sEuyFqFv3+7//+b583b16Tn6+x0vU+SXIaujh4Mk03nYCNte5vim+r9xh3rwR2AO3N7HDgZ8CtB3sBM5tkZsVmVlxSUpJEkUIkqAuEN4Nk2uhTJd3z0Xfs2JFRo0bR2EV1aoZzHnPMMQBce+21iY7W5pTr8/bnqnR3xk4D7nL3soONU3X32cBsiK0wleYyZZcwN900Y9Cnez76pUuXNql8QblgSvP2Z6dkgn4z0LnW/cL4tvqO2WRmEaANUAoMAcaZ2a+BtkC1mZW7+2+bWvDQCNAC4e7erBeWmBlmlrG5bjQffXL0PjWvxvx7SCbolwI9zKwrsUC/HPhOnWPmAxOAN4FxwKJ4m1FizJmZTQPKFPINlKjRZ3YahPz8fEpLS2nfvn2zhr2mKhb5mrtTWlpKfn5+gx53yKB390ozu5ZYh2se8LC7v29m04k1/s8HHgKeMLO1wOfEvgwkFWqCPsM1+sLCQjZt2kRz96Fs3bWVL1p8wfb87c36uiJBlZ+fT2FhYYMek1QbvbsvABbU2XZLrdvlwCV1H1fn+GkNKpnERINRo49Go3Tt2rXZX3fKC1Po3rY7vznpN83+2iK5IlyXWmajSHDa6DOhIFLArsqmz2ooEmYK+qAL8ZWxEAv63SFdHF0kVRT0QRcJ7zh6gFbRVgp6kSZS0AddTY0+pIuPFEQK2FWhphuRplDQB13Ya/QR1ehFmkpBH3R5EbC8cNfo1Rkr0iQK+mwQDe8qU+qMFWk6BX02iOSHtkbfKtqKyupKKqoqMl0UkayloM8G0QKo3JPpUmREQbyPQs03Io2noM8GkfyMXxmbKTVBr+YbkcZT0GeDAC4Q3lxaRWKLXCvoRRpPQZ8NVKNX041IEyjos0EkxDX6aLxGH9LOaJFUSPcKU5IK0QLY82WmS5ERNTX629+5nbYt22a2MCJpVvQvRUwekPqFyhX02SCSH9pRN93admN44XB27t1JRbWGWEpuS9ciOwr6bBAtCO04+tbR1vx2lBYlE2kKtdFng0h+aK+MFZGmU9BngxB3xopI0ynos0E0vMMrRaTpFPTZIFIAVXuhuirTJRGRLKSgzwYhX05QRJpGQZ8NEouPhHOIpYg0jYI+G4R8OUERaRoFfTYI+XKCItI0CvpsEGkZ+60avYg0goI+G0RVoxeRxlPQZ4OI2uhFpPEU9NlANXoRaQIFfTaIaBy9iDSegj4b1NToNd+NiDSCgj4b1Iy60Xw3ItIISQW9mY02sw/NbK2ZTalnf0szeya+/20z6xLffpaZLTOz9+K/z0xx+cMhohq9iDTeIYPezPKA3wHnAL2BK8ysd53Dvg984e7dgbuAO+LbtwEXuHs/YALwRKoKHiqJuW5UoxeRhkumRj8YWOvu6919LzAHGFvnmLHAY/HbzwKjzMzc/R/u/ml8+/tAgZm1TEXBQ0U1ehFpgmSCvhOwsdb9TfFt9R7j7pXADqB9nWMuBt519/1m5jKzSWZWbGbFJSUlyZY9PPIi0CKiUTci0ijN0hlrZn2INef8sL797j7b3Yvcvahjx47NUaTsEylQ0ItIoyQT9JuBzrXuF8a31XuMmUWANkBp/H4h8DzwPXdf19QCh1Y0X1fGikijJBP0S4EeZtbVzA4DLgfm1zlmPrHOVoBxwCJ3dzNrC7wETHH3v6WozOGkBcJFpJEOGfTxNvdrgYXAB8Bcd3/fzKab2Zj4YQ8B7c1sLXAjUDME81qgO3CLmS2P/xyd8rMIg4hq9CLSOJFkDnL3BcCCOttuqXW7HLiknsfdBtzWxDIKxBcIV41eRBpOV8Zmi0iBavQi0igK+mwRzdeasSLSKAr6bBEp0JWxItIoCvpsEWmpK2NFpFEU9Nkiqhq9iDSOgj5bRPJVoxeRRlHQZ4uopkAQkcZR0GcLXTAlIo2koM8W0QKoroDqqkyXRESyjII+W2iBcBFpJAV9tqgJenXIikgDKeizhZYTFJFGUtBnCy0nKCKNlNTslRIANTX6OVd83YwjuaHlEXDJo3C4ZvCW9FDQZ4vOQ6HPRRpimWu8Gj5aCCvmwOnXZbo0kqMU9NniiGPgkkcyXQpJhweGw6rnFPSSNmqjF8m0fuNgy3Io1ZLKkh4KepFM63NR7Peq5zJbDslZCnqRTGvTCY47Dd57FtwzXRrJQQp6kSDodzFs+xC2vp/pkkgOUtCLBEHvC8Hy1HwjaaGgFwmC1h3ghBGxoFfzjaSYgl4kKPpeDNs/gc3LMl0SyTEKepGg6HU+5B0W65QVSSEFvUhQ5LeBHt+C95/XugOSUgp6kSDpezGU/S988vdMl0RyiIJeJEhOHA3R1rBKzTeSOgp6kSA5rBX0PBdWz4OqikyXRnKEgl4kaPpeDLu/gHWLM10SyREKepGg6TYq1jGri6ckRRT0IkETOQx6jYE1L2r9AUmJpOajN7PRwD1AHvCgu99eZ39L4HFgEFAKXObuG+L7bga+D1QB17n7wpSVXiRX9RsH/3gCfj9UK4qFSff/A2fPSPnTHjLozSwP+B1wFrAJWGpm8919da3Dvg984e7dzexy4A7gMjPrDVwO9AGOBV41sxPdXYOERQ6myxkwZDLs3JLpkkhzOvLYtDxtMjX6wcBad18PYGZzgLFA7aAfC0yL334W+K2ZWXz7HHffA3xsZmvjz/dmaoovkqNa5ME5d2S6FJIjkmmj7wRsrHV/U3xbvce4eyWwA2if5GMxs0lmVmxmxSUlJcmXXkREDikQnbHuPtvdi9y9qGPHjpkujohITkkm6DcDnWvdL4xvq/cYM4sAbYh1yibzWBERSaNkgn4p0MPMuprZYcQ6V+fXOWY+MCF+exywyN09vv1yM2tpZl2BHsA7qSm6iIgk45Cdse5eaWbXAguJDa982N3fN7PpQLG7zwceAp6Id7Z+TuzLgPhxc4l13FYCP9KIGxGR5mUesNVsioqKvLi4ONPFEBHJKma2zN2L6tsXiM5YERFJHwW9iEiOC1zTjZmVAJ804Sk6ANtSVJxsovMOF513uCRz3se7e73j0wMX9E1lZsUHaqfKZTrvcNF5h0tTz1tNNyIiOU5BLyKS43Ix6GdnugAZovMOF513uDTpvHOujV5ERPaVizV6ERGpRUEvIpLjcibozWy0mX1oZmvNbEqmy5MuZvawmX1mZqtqbWtnZq+Y2Ufx30dlsozpYGadzWyxma02s/fN7Pr49pw+dzPLN7N3zGxF/LxvjW/vamZvxz/vz8QnHMw5ZpZnZv8wsxfj98Ny3hvM7D0zW25mxfFtjf6s50TQ11ru8BygN3BFfBnDXPQoMLrOtinAa+7eA3gtfj/XVAL/7u69gaHAj+J/41w/9z3Ame4+ABgIjDazocSW67zL3bsDXxBbzjMXXQ98UOt+WM4bYKS7D6w1fr7Rn/WcCHpqLXfo7nuBmuUOc467v0FshtDaxgKPxW8/BlzYnGVqDu6+xd3fjd/eSewffydy/Nw9pix+Nxr/ceBMYst2Qg6eN4CZFQLnAQ/G7xshOO+DaPRnPVeCPqklC3PYMe5es4r0/wLHZLIw6WZmXYCTgLcJwbnHmy+WA58BrwDrgO3xZTshdz/vdwM3AdXx++0Jx3lD7Mv8r2a2zMwmxbc1+rOezOLgkkXc3c0sZ8fMmtnhwHPAj939y1glLyZXzz2+hsNAM2sLPA/0zGyJ0s/Mzgc+c/dlZjYiw8XJhG+6+2YzOxp4xczW1N7Z0M96rtTow75k4VYz+wZA/PdnGS5PWphZlFjIP+nu/ze+ORTnDuDu24HFwKlA2/iynZCbn/fTgTFmtoFYU+yZwD3k/nkD4O6b478/I/blPpgmfNZzJeiTWe4wl9VeynECMC+DZUmLePvsQ8AH7n5nrV05fe5m1jFek8fMCoCziPVPLCa2bCfk4Hm7+83uXujuXYj9e17k7uPJ8fMGMLPWZnZEzW3gW8AqmvBZz5krY83sXGJtejXLHc7IbInSw8yeBkYQm7Z0KzAVeAGYCxxHbIrnS929bodtVjOzbwJLgPf4us3258Ta6XP23M2sP7GOtzxiFbO57j7dzE4gVtNtB/wD+K6778lcSdMn3nTzE3c/PwznHT/H5+N3I8BT7j7DzNrTyM96zgS9iIjUL1eabkRE5AAU9CIiOU5BLyKS4xT0IiI5TkEvIpLjFPQiIjlOQS8ikuP+P5+cF7TL6BFSAAAAAElFTkSuQmCC", + "text/plain": [ + "
    " + ] + }, + "metadata": { + "needs_background": "light" + }, + "output_type": "display_data" + } + ], + "source": [ + "scheduler = MultiStepLRScheduler(optimizer, decay_t=[10, 30])\n", + "plot_lr(scheduler, label='default')\n", + "\n", + "scheduler = MultiStepLRScheduler(optimizer, decay_t=[10, 30], decay_rate=0.1)\n", + "plot_lr(scheduler, label='decay_rate=0.1, default at train script')\n", + "\n", + "scheduler = MultiStepLRScheduler(optimizer, decay_t=[20], decay_rate=0.3)\n", + "plot_lr(scheduler, label='decay_t=[20], decay_rate=0.3')\n", + "\n", + "plt.legend();" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### `decay_rate`" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "`decay_rate` - multiplier for decay learning rate. So, at every step number at the `decay_t` list, learning rate is decayed by `decay_rate`. If `decay_t=0.5` than at epoch `decay_t[0]` new learning rate will be set to `lr * decay_rate`. And so on on next step at next element from `decay_t`." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "By setting `decay_t` = 5 and `decay_rate` = 1., we are telling the schedule to reduce the learning rate by decay_rate where new lr `lr * decay_rate` every 5 epochs." + ] + }, + { + "cell_type": "code", + "execution_count": 11, + "metadata": { + "scrolled": true + }, + "outputs": [ + { + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXoAAAD4CAYAAADiry33AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/YYfK9AAAACXBIWXMAAAsTAAALEwEAmpwYAAAzEElEQVR4nO3de3xU1bn4/8+TmWQmQe4QCAQFAeUeVBQEFShSQKqgqIiccqnKV6ngl36xYtsDSksLv3hErbfjDbnYgkdFrKIcbhZvKAEJQsCCAhJIASOEiwkhyfr9MZMxCQmZJDOzdibP+/XKi8yePbPXHibPrHnW2s8SYwxKKaWiV4ztBiillAovDfRKKRXlNNArpVSU00CvlFJRTgO9UkpFObftBpTVrFkz07ZtW9vNUEqpWmXz5s3fG2Oal3ef4wJ927ZtSUtLs90MpZSqVURkf0X3aepGKaWinAZ6pZSKchrolVIqyjkuR6/qhrNnz5KZmUleXp7tpihVq3i9XpKTk4mNjQ36MRrolRWZmZnUr1+ftm3bIiK2m6NUrWCMITs7m8zMTNq1axf044JK3YjIUBH5WkT2iMiMcu6/TkS2iEiBiNxa5r7xIrLb/zM+6JapqJaXl0fTpk01yCtVBSJC06ZNq/xNuNJALyIu4BlgGNAFGCMiXcrs9h0wAfhbmcc2AWYBvYGrgFki0rhKLVRRS4O8UlVXnb+bYFI3VwF7jDHf+g+yFBgBZBTvYIzZ57+vqMxjhwCrjTE/+O9fDQwF/l7lllZixzdpLPnn7FA/rRUtTT16myQrx87xtGJHixvDfpx+zc7y75zSvZJYl9CkXpx+ACgVYsEE+tbAgRK3M/H10INR3mNbl91JRCYBkwAuvPDCIJ+6tP1ZGbwX8221HuskRgSXMdy/930iHe5ixLc2wf/LaMtp4sN6rB43JnHk5LlfP+t7Y4lza6BXKqSMMef9AW4FXipx+5fA0xXs+ypwa4nb04E/lLj9n8D08x3viiuuMHXZi9teNN1e7WbOFJyJ/MG/eMmYWQ2MOZEV9kNlZGSUun389BmTfuCY+TG/IOzHLs+sWbNMamqqlWNXxfz5883p06dr/DxFRUVmypQppn379qZ79+5m8+bN5e7Xv39/c8kll5iUlBSTkpJiDh8+XKXj9O/f32zatKnG7a2OL7/80rz33ntB7793717j9XpNSkqKMcaY7777zgwYMMB07tzZdOnSxTzxxBOBfbOzs831119vOnToYK6//nrzww8/nPe59+3bZy677DKTkpJiunTpYp577rnAfWlpaaZbt26mffv2ZsqUKaaoqMgYY8z06dNNixYtyn1flv37McYYIM1UEFeDGYw9CLQpcTvZvy0YNXlsnRQXEwdAXqGFaYdx9Xz/5p+O+KFjYny9+KKiur3imTGGoqKyGdCfPPHEE/z44481Ps7777/P7t272b17Ny+88AL33Xdfhfu+9tprbN26la1bt5KYmFjjY0fK1q1bWblyZZUe0759e7Zu3QqA2+3mv/7rv8jIyGDjxo0888wzZGT4MtZz585l0KBB7N69m0GDBjF37tzzPm9SUhKfffYZW7du5fPPP2fu3LkcOnQIgPvuu48XX3wx8P/xwQcfAJCamsq9995bxbMuXzCpm01ARxFphy9I3wHcGeTzrwL+XGIA9ufAw1VuZR3idXsBOFNwBuIifHBLgf7Rf+xg+8EccvML8ca6cMXUPHXTpVUDZt3Y9bz7zJkzh4ULF5KYmEibNm244oorAPjmm2/49a9/zdGjR0lISODFF1+kU6dOHD58mHvvvZdvv/WlCJ977jn69u3LyJEjOXDgAHl5eTzwwANMmjSJV155hW3btvHEE08A8OKLL5KRkcH8+fPPace+ffsYMmQIvXv3ZvPmzaxcuZK5c+eyadMmcnNzufXWW3n00Ud56qmnOHToEAMHDqRZs2asX7+e//3f/2XWrFmcOXOG9u3bs2DBAi644IJKX58VK1Ywbtw4RIQ+ffpw/PhxsrKySEqq2dhQbm4uEydOJD09nU6dOpGbmxu4r6K2btq0iQceeIDTp0/j8XhYu3Yt2dnZ/PKXv+T0ad978emnn6Zv376MGzeOW265hZEjRwIwduxYbr/9dkaMGFGqHfn5+cycOZPc3Fw+/vhjHn74YUaPHl2lc0lKSgq8HvXr16dz584cPHiQLl26sGLFCj788EMAxo8fz4ABA5g3b16FzxUX99Mf85kzZwIf5llZWZw4cYI+ffoAMG7cON5++22GDRtWpbZWptIevTGmALgfX9DeCbxujNkhIrNF5CYAEblSRDKB24D/FpEd/sf+APwR34fFJmC2f5uqgMflASz16GMTfP9a6NEXh/ZI9ec3b97M0qVLA72+TZs2Be6bNGkSf/3rX9m8eTOPPfYYkydPBmDq1Kn079+f9PR0tmzZQteuvg+SV155hc2bN5OWlsZTTz1FdnY2t99+O//4xz84e/YsAAsWLOBXv/pVhe3ZvXs3kydPZseOHVx00UXMmTOHtLQ0tm3bxj//+U+2bdvG1KlTadWqFevXr2f9+vV8//33/OlPf2LNmjVs2bKFXr168fjjjwMwbdo0evbsec5Pcc/z4MGDtGnz05ft5ORkDh4s/8v2xIkT6dmzJ3/84x+LU7AVeu6550hISGDnzp08+uijbN68GaDCtubn5zN69GiefPJJ0tPTWbNmDfHx8SQmJrJ69Wq2bNnCsmXLmDp1KgB33XUXr776KgA5OTl8+umnDB8+/Jx2xMXFMXv2bEaPHs3WrVsZPXo069evL/c16du373nPCXwfxl9++SW9e/uGJw8fPhz4EGjZsiWHDx+u9DkOHDhAjx49aNOmDQ899BCtWrXi4MGDJCcnB/Y53/9DTQR1wZQxZiWwssy2mSV+34QvLVPeY18BXqlBG+sUj9sX6M8UnIn8wYt79GcjG+hn3diVgsIiMrJO0KpRPM0u8IT9mB999BE333wzCQm+D7ebbroJgFOnTvHpp59y2223BfY9c8b3f7Fu3ToWLVoEgMvlomHDhgA89dRTLF++HPD9Me/evZs+ffrws5/9jHfffZfOnTtz9uxZunfvXmF7LrrookCvDuD111/nhRdeoKCggKysLDIyMujRo0epx2zcuJGMjAz69esH+HqxV199NUC53xyq47XXXqN169acPHmSUaNGsXjxYsaNG1fh/hs2bAgE5R49egTaXFFbv/76a5KSkrjyyisBaNCgAQCnT5/m/vvvZ+vWrbhcLv71r38B0L9/fyZPnszRo0d58803GTVqFG53cNd9Dhw4MJCWqYpTp04xatQonnjiiUD7ShKRoGaKtWnThm3btnHo0CFGjhzJrbfeWuljQkWvjHUYr8ufuim0GOjza54DrqriHH2h5Rx9UVERjRo1CjogfPjhh6xZs4bPPvuMhIQEBgwYELiY5e677+bPf/4znTp1YuLEied9nnr16gV+37t3L4899hibNm2icePGTJgwodwLZIwxDB48mL///dzZytOmTWP9+vXnbL/jjjuYMWMGrVu35sCBnybEZWZm0rr1ORPiAtvq16/PnXfeyRdffHHeQF+Ritr61Vdflbv//PnzadGiBenp6RQVFeH1egP3jRs3jiVLlrB06VIWLFgQdBvWr1/PtGnTztmekJDAp59+Wu5jzp49y6hRoxg7diy33HJLYHuLFi0Cqa6srKwqjV20atWKbt268dFHH9GvXz8yMzMD91X0/1BTWtTMYYpTN1YCvcXUTYwIMSIRC/TXXXcdb7/9Nrm5uZw8eZJ//OMfgK9H2a5dO/7nf/4H8AWo9PR0AAYNGsRzzz0HQGFhITk5OeTk5NC4cWMSEhLYtWsXGzduDByjd+/eHDhwgL/97W+MGTMm6LadOHGCevXq0bBhQw4fPsz7778fuK9+/fqcPHkSgD59+vDJJ5+wZ88ewNcLLu75zp8/PzCAWvJnxgzfhe033XQTixYtwhjDxo0badiw4Tn5+YKCAr7//nvAF/DeffddunXrBsDy5ct5+OFzh9uuu+46/vY333WT27dvZ9u2bedt66WXXkpWVlYgdXby5EkKCgrIyckhKSmJmJgYFi9eTGFhYeAYEyZMCIx9dOlS9trNn5R8reCnHn3Zn4qCvDGGu+66i86dO/Ob3/ym1H033XQTCxcuBGDhwoWBMYKKPggzMzMD4xXHjh3j448/5tJLLyUpKYkGDRqwceNGjDEsWrTonPGGUNBA7zBWc/Rx/kG8CKduirliJGKzbi6//HJGjx5NSkoKw4YNC6QOwJeuePnll0lJSaFr166sWLECgCeffJL169fTvXt3rrjiCjIyMhg6dCgFBQV07tyZGTNmlEq/ANx+++3069ePxo2DvyA8JSWFyy67jE6dOnHnnXcG0h3gGz8YOnQoAwcOpHnz5rz66quMGTOGHj16cPXVV7Nr166gjnHDDTdw8cUX06FDB+655x6effbZwH09e/YEfCmrIUOG0KNHD3r27Enr1q255557AN+AdXlpjPvuu49Tp07RuXNnZs6cGRjgrqitcXFxLFu2jClTppCSksLgwYPJy8tj8uTJLFy4kJSUFHbt2lXqG0+LFi3o3Llzpd+SBg4cSEZGBj179mTZsmVBvS4lffLJJyxevJh169YF8vnFs3hmzJjB6tWr6dixI2vWrAl8gH733XfEx597DcrOnTvp3bs3KSkp9O/fn+nTpwdSec8++yx33303HTp0oH379iEfiAUqn0cf6Z+6Po9++/fbTbdXu5l1+9dF/uB5J3zz6D9+MuyHKm8e8Nf/PmH2fX8q7MeOpOHDh5s1a9bYbkbIjR071hw5csTKsU+fPm0uvvhic/z48ZA+7969e03Xrl1r9BzTp0836enpIWpRxdd3hGMevYogqzl6i6kbAFcEUzfhdvz4cS655BLi4+MZNGiQ7eaE3JIlS2jevNzlScNqzZo1dO7cmSlTpgQGw0PF5XKRk5MT+EZTHampqecMmlfXgw8+yJIlS0p9m6kuHYx1GKupmxgXuOOtpm7OFlZ8sVBt0qhRo0C+vFh2dna5QX/t2rU0bdo0Uk2r1a6//nr27y+9NOqqVat46KGHSm1r165dYCZUsNq0aVNqgNq21NRUUlNTQ/JcGugdptQFUzbEJVjr0cfECIUF0dGjL0/Tpk2rNb1Pnd+QIUMYMmSI7WY4mqZuHMbqrBuA2HpWpldCZAdjlapLNNA7jNUcPfjm0ttK3YhQWESlV18qpapGA73DuGPcCGInRw9WUzeuGDAYtFOvVGhpoHcYEcHr9lrM0dtL3Tjl6liloo0GegfyuDz2evSx9axOrwQospC6eeSRR3jsscciftyqClWZYmMMU6dOpUOHDvTo0YMtW7aUu9/QoUMDF47de++9pa5QDUbbtm0DV9dG2ocffljhVa8V7d+wYUNuuOGGwLahQ4fSqFEjfvGLX5Tad8KECbRr1y5wIVXxIPuyZcvo0KHDOfvbpoHegTwuj8UcfYLV6ZVQt3v0xmH16F9//XXS09PZvn07R48eDZSGqA2qGugBrr322lI17B988EEWL15c7r6pqamBMgrFc+9Hjx7NSy+9VO02h4tOr3SgOpe6eX8G/PsrEozh4vxCPLExEFPDPkjL7jDs/ItBaD36yuvRF5c5KCgoID8/v9IqjdnZ2YwZM4aDBw9y9dVXlxpYX7JkCU899RT5+fn07t2bZ599FpfLxQcffMDvfvc7CgsLadasGWvXruWLL77ggQceIC8vj/j4eBYsWMCll17Kddddx1NPPRUIrNdccw3PPPMMKSkp57ymzz//PC6XiyVLlvDXv/6Va6+9ttLXpaxBgwYF6s7XZtqjdyCrPXqLqZtI1qTXevTB16MfMmQIiYmJ1K9fv9LSuo8++ijXXHMNO3bs4Oabb+a7774DfLVeli1bxieffBIoPfzaa69x9OhR7rnnHt58803S09MD3xg6derERx99xJdffsns2bP53e9+B5SuR/+vf/2LvLy8c4I8+FJG9957L9OmTWPr1q1ce+21vPbaa+W+JtUtF/z73/+eHj16MG3atEApa6fSHr0D2U3d+KdXGgNB1NgOCX/Pu6iwiG8jVJNe69EHb9WqVeTl5TF27FjWrVvH4MGDK9x3w4YNvPXWWwAMHz48UMxt7dq1bN68OVA8Ljc3l8TERDZu3Mh1111Hu3btAGjSpAngW1Rk/Pjx7N69GxEJfGDedttt/PGPfyQ1NZVXXnmFCRMmBH0eY8eOZezYsVU+//L85S9/oWXLluTn5zNp0iTmzZvHzJkzK3+gJRroHcjqYGxcApgiKDgDsd7K9w8hJ8y60Xr05fN6vYwYMYIVK1acN9BXxBjD+PHj+ctf/lJqe3F56LL+8z//k4EDB7J8+XL27dvHgAEDAF/t+MGDB7NixQpef/31wApWwXjttdfKLSnQoUMH3njjjeBPBgIpLo/Hw8SJEx0/kK+pGwfyuD32cvSxFhcI99ekj8TVsVqPvvJ69KdOnSIrKwvw5ejfe+89OnXqBPjWcH366afLfV2L69G///77HDt2LPDavfHGGxw5cgSAH374gf3799OnTx82bNjA3r17A9vB16Mv/uApTtUUu/vuu5k6dSpXXnnlecs/l61HP3bs2HJfk6oGeSDwuhhjePvttwN1+p1KA70DeV1eiz16O8sJFnPFRKaCpdajr7we/enTp7npppsC9egTExO59957Adi1a1e5hdhmzZrFhg0b6Nq1K2+99RYXXngh4Fsg5E9/+hM///nP6dGjB4MHDyYrK4vmzZvzwgsvcMstt5CSkhJYwPu3v/0tDz/8MJdddhkFBQWljnHFFVfQoEGDSr8l3XjjjSxfvpyePXvy0UcfBfW6lHXttddy2223sXbtWpKTk1m1ahXg+9Do3r073bt35/vvv+cPf/hDtZ4/YiqqX2zrp67XozfGmN/+87dm2JvD7Bz8qzd8NekPn1vvOpTKq6dtjDFfZ0VXTfporUc/fPhwc+bMGSvHPnjwoOnYsaMpLCwM6fOuX7/eDB8+3DHPcz5ajz4KWJ1eGUjd2Ls6Nhrm0Ud7Pfp3332XuLi4iB930aJF9O7dmzlz5hBT0ym4ZcTFxbF9+/ZSF0xV1bJly5g8eXKVvsFFgg7GOpDH5eFMkcVZN2A1dVNwnguGagutRx8e48aNO2dN1gULFvDkk0+W2tavXz+eeeaZKj1337592bdvX43aN3r06ED6yUk00DuQx2VxMDbO8ipTMUJ+lNak13r04TFx4sRK8/V1naZuHKh4eqWxUa63eIFwq8sJWjm0UlFLA70DFa8ylV+UH/mDF68be9bW4iO+efRWPuSUilIa6B0osG5sgYUplnH25tGDbzBWa9IrFVoa6B3I6nKClgN9oFSxRnqlQkYDvQMFFgi3EehdcSAuq4OxAIURTt1oPfry69H//ve/p02bNkFVxCzPgAEDSEtLq0lTq624YF2w9u3bR3x8fOCCMYBf/epXJCYmnnPl6w8//MDgwYPp2LEjgwcPDlwBXJH9+/dz+eWX07NnT7p27crzzz8fuG/z5s10796dDh06MHXq1EDa8sEHH6Rly5YheV9qoHegOJdvfrKVmTci/sJm9hYIh7pbk944rB79jTfeyBdffFHj49lQ1UAP0L59+1IzoyZMmMAHH3xwzn5z585l0KBB7N69m0GDBgWqglYkKSmJzz77jK1bt/L5558zd+5cDh06BMB9993Hiy++GPj/KD5eampq4ErkmtLplQ7kiAXCI9ijn/fFPHb94Lt0v8gYcvML8ca6AkG/Ojo16cRDVz103n20Hn3l9ejLlnSoTG5uLhMnTiQ9PZ1OnTqRm5sbuK+itm7atIkHHniA06dP4/F4WLt2LdnZ2fzyl7/k9Gnf+/Dpp5+mb9++jBs3jltuuYWRI0cCvlIEt99+OyNGjCjVjvz8fGbOnElubi4ff/wxDz/8cLXmt1933XXlzq1fsWJFoE79+PHjGTBgAPPmzavweUpeXHbmzJnAh3lWVhYnTpwIvM7jxo3j7bffZtiwYVVu6/looHegwGCsteUE7S0QXizc/fmS9egLCgq4/PLLA4F+0qRJPP/883Ts2JHPP/+cyZMns27dukA9+uXLl1NYWMipU6cAXz36Jk2akJuby5VXXsmoUaO4/fbbmTNnDqmpqcTGxrJgwQL++7//u8L27N69m4ULFwb+4OfMmUOTJk0oLCxk0KBBgXr0jz/+OOvXr6dZs2al6tHXq1ePefPm8fjjjzNz5sxKq1dWVI++bKCvqueee46EhAR27tzJtm3buPzyywEqbOuMGTMYPXo0y5Yt48orr+TEiRPEx8eTmJjI6tWr8Xq97N69mzFjxpCWlsZdd93F/PnzGTlyJDk5OXz66acsXLjwnHbExcUxe/Zs0tLSAsXX1q9fz7Rp087ZNyEhocorUR0+fDjwWrVs2ZLDhw9X+pgDBw4wfPhw9uzZQ2pqKq1atSItLY3k5OTAPudbF6Amggr0IjIUeBJwAS8ZY+aWud8DLAKuALKB0caYfSISC7wEXO4/1iJjTOk6peocgRy9zVWmIpi6KdnzPltYxM4I1KTXevThsWHDBqZOnQpAjx49Am2uqK1ff/01SUlJgaJyxStanT59mvvvvz+wSEnxVcb9+/dn8uTJHD16lDfffJNRo0bhdgfXXx04cGBYLlgTkUpX3gJo06YN27Zt49ChQ4wcObLaC55UR6WvkIi4gGeAwUAmsElE3jHGZJTY7S7gmDGmg4jcAcwDRgO3AR5jTHcRSQAyROTvxph9oT6RaGK9Rx/h1E1JxekaW7NutB59eFTU1q+++qrc/efPn0+LFi1IT0+nqKgIr/entRHGjRvHkiVLWLp0KQsWLAi6DaHs0bdo0SKQ6srKyiIxMTHox7Zq1Ypu3brx0Ucf0a9fPzIzMwP3hev/IZjB2KuAPcaYb40x+cBSYESZfUYAxd+f3gAGie8jzgD1RMQNxAP5wImQtDyKWc/RW0zdFNekD/esG61HX3k9+vNZvnw5Dz/88DnbS9aj3759O9u2bTtvWy+99FKysrICSzmePHmSgoICcnJySEpKIiYmhsWLF1NYWBg4xoQJEwJjH126dKmwjWXr0Rf36Mv+VDXIg+/1K04ZLVy4MDBG8MUXX5xTiwd8Abx4vOLYsWN8/PHHXHrppSQlJdGgQQM2btyIMYZFixadM94QCsEE+tbAgRK3M/3byt3HGFMA5ABN8QX900AW8B3wmDHmh7IHEJFJIpImImlHjx6t8klEG4/b4jx6sDrrBiJTk17r0Vdejx58deGTk5P58ccfSU5O5pFHHgF8A9bFaZaS7rvvPk6dOkXnzp2ZOXNmYNyjorbGxcWxbNkypkyZQkpKCoMHDyYvL4/JkyezcOFCUlJS2LVrV6lvPC1atKBz586VfksaOHAgGRkZ9OzZk2XLlgX1upQ1ZsyYQIopOTmZl19+GYAZM2awevVqOnbsyJo1awIfoN999x3x8fHnPM/OnTvp3bs3KSkp9O/fn+nTpwdSec8++yx33303HTp0oH379iEfiAUqr0cP3IovL198+5fA02X22Q4kl7j9DdAM6Ae8BsQCicDXwMXnO57Wozfm6I9HTbdXu5mlO5faacBb/8eYx7uG9RAV1aM3xphdUVSTPlrr0Y8dO9YcOXLEyrFPnz5tLr74YnP8+PGQPu/evXtN1641e99Pnz7dpKenh6hFxsyaNcukpqaesz0c9egPAm1K3E72byt3H3+apiG+Qdk7gQ+MMWeNMUeAT4BeVfokqoOs5+gtz7qJ1CpT4RTt9eiXLFlC8+bNI37cNWvW0LlzZ6ZMmRIYDA8Vl8tFTk5OqW80VZWamnrOoHl1PfjggyxZsqTUt5nqCma4ehPQUUTa4Qvod+AL4CW9A4wHPsP3DWCdMcaIyHfAz4DFIlIP6AM8UeNWRznrOXpHpG5qdwlLrUcfHtdffz379+8vtW3VqlU89FDpaybatWsXmAkVrDZt2pQaoLYtNTW13MXMq6PSQG+MKRCR+4FV+KZXvmKM2SEis/F9VXgHeBlfMN8D/IDvwwB8s3UWiMgOQIAFxphtIWl5FHPHuImRGDtFzcAX6AvyoKgQYlxhO4wxptxpaS4R8mt3nC+X1qMPjyFDhjBkyBDbzYgYU42JCkFNQDXGrARWltk2s8TvefimUpZ93KnytqvzExHf4iM2e/TgS994zx1wCwWv10t2djZNmzY9J9gXlypWSpVmjCE7O7vUdNNg6JWxDuV1ee1OrwRf+iZMgT45OZnMzEzKm2WVk3uWU2cKkJxzZy8oVdd5vd5SV9MGQwO9Q8W54pzRow+T2NhY2rVrV+59z334DfM+2MXO2UOJjwtf6kipukKrVzqU1+21WwIBrM28aRDv63+cyDtr5fhKRRsN9A5VvG6sFbF2Fwhv4I0F4ESuBnqlQkEDvUNZzdEX9+jP2urR+wO99uiVCgkN9A7lcXvsTq8EyLczl76B15+6yS2wcnyloo0GeoeyOr3SdupGe/RKhZQGeoeyO4/ev0KRrdSN5uiVCikN9A5lN9AX9+jtpG7qF6du8jR1o1QoaKB3KKvTKy2nbryxLjzuGE3dKBUiGugdyur0yhgXuOOtpW7Al6fXwVilQkMDvUNZnV4JvvSNxVLFDbxu7dErFSIa6B3K4/bl6KtTqS4kYutZy9ED1PfG6mCsUiGigd6hihcfsVuT3nLqRgdjlQoJDfQOZT/Q20/dnNQevVIhoYHeoewHerupG1+PXgO9UqGggd6hvG7/coLWplhaTt14fbNurI1RKBVFNNA7lPUFwm2nbuLd5BcWcaYgCtcUVCrCNNA7lCMWCLeZutEyCEqFjAZ6h/K4/T16WxUsY+tZ7tFrYTOlQkUDvUPZH4xN8OXoLeXIi0sV5+jVsUrVmAZ6h7If6OuBKQJLg8Hao1cqdDTQO5T1HH2s5XVjNUevVMhooHco6zl668sJ+lI3J/XqWKVqTAO9Q9lP3ditSR/o0WvqRqka00DvUHU9deONdRHnjtFSxUqFgAZ6h6rrqRvQUsVKhYoGeodyi5sYiXFA6sZ2GQQN9ErVlAZ6hxIRu+vGWk7dANTXUsVKhYQGegezuspUIHVjswyCW3v0SoWABnoH87g99nP0lssgaI5eqZoLKtCLyFAR+VpE9ojIjHLu94jIMv/9n4tI2xL39RCRz0Rkh4h8JSLeELY/qjmiR289R6+pG6VqqtJALyIu4BlgGNAFGCMiXcrsdhdwzBjTAZgPzPM/1g0sAe41xnQFBgDaRQuSx+WxV6bYFQfisl6qWHv0StVcMD36q4A9xphvjTH5wFJgRJl9RgAL/b+/AQwSEQF+DmwzxqQDGGOyjTGFoWl69PO4PPYWHhHxrxtrt1RxfkEReWf1LaNUTQQT6FsDB0rczvRvK3cfY0wBkAM0BS4BjIisEpEtIvLb8g4gIpNEJE1E0o4ePVrVc4haHrfFWTfgr0mvpYqVqu3CPRjrBq4Bxvr/vVlEBpXdyRjzgjGmlzGmV/PmzcPcpNrD6vRKgFj7C4QDmqdXqoaCCfQHgTYlbif7t5W7jz8v3xDIxtf732CM+d4Y8yOwEri8po2uK6wOxoK/Jr3dBcIBTmqPXqkaCSbQbwI6ikg7EYkD7gDeKbPPO8B4/++3AuuMb1XnVUB3EUnwfwD0BzJC0/ToZ3V6JUDcBdZn3QB60ZRSNeSubAdjTIGI3I8vaLuAV4wxO0RkNpBmjHkHeBlYLCJ7gB/wfRhgjDkmIo/j+7AwwEpjzHthOpeoY71HH5sAucesHb5hfHHqRnv0StVEpYEewBizEl/apeS2mSV+zwNuq+CxS/BNsVRVZHV6JfgGY0+UzdJFjpYqVio09MpYB7M6vRKsz7qpH1hlSlM3StWEBnoH87g95BflYywt0G171o03NoZYl2iPXqka0kDvYPZXmbJ7wZSIaKlipUJAA72DWV9lKq4eFORBkb0rUxtoqWKlakwDvYM5ZpUpyxdNaY9eqZrRQO9g1nv0sf5VpixfNKU5eqVqRgO9g8W54gDsTbF0RI9ec/RK1ZQGegcr7tHnF+bbaYATAn28W3P0StWQBnoHs56jj9UFwpWKBhroHcx6jj6wbqzdUsVnCoo4U6A16ZWqLg30DlY8j95+jt7uAuEAJzV9o1S1aaB3sOLUjbUyCE5I3RQvPqLpG6WqTQO9g9m/MvYC3782UzdaqlipGguqeqWyw37qprhHb3Meve8t+uf3dtL0gjhr7QinCzxuHrmpK/U8+ueowkPfWQ5mfXqlA1I3HRLr0+uixhzPzed4rqXXIYzyzhbx3Q8/cmNKK667RJfRVOGhgd7BrE+vjHGBO95q6qZhfCxv3NfX2vHD7dDxXPrOXUfmsVzbTVFRTHP0DhYbE4tLXPbXjbXYo492LRp4cccImcfspcdU9NNA73DWV5mKrWc1Rx/tXDFCq0bx2qNXYaWB3uG8bq/9VaYspm7qgtaN4jl4XAO9Ch8N9A4X54qzvG6spm7CLblxvKZuVFhpoHc4r8trOUevqZtwS26cwOETZ7TMgwobDfQO53F57Ab6WE3dhFty43gADh23+M1NRTUN9A7ncXss5+g1dRNuxYFe0zcqXDTQO5ymbqJfchPfhWk680aFiwZ6h3PG9Ert0YdTi/oeXDqXXoWRBnqH87gckLo5exqMsdeGKOd2xZDU0MtB7dGrMNFA73Aet+UefVw9MEVg88OmDvBNsdRAr8JDA73DWc/Rx9pfN7YuSG6coIFehY0GeoezPr3SAcsJ1gXJjeM5fDJP59KrsNBA73COmF4JOvMmzJIbJ2AMZOlcehUGGugdzuvykl+UT5EpstMATd1ExE9z6TV9o0IvqEAvIkNF5GsR2SMiM8q53yMiy/z3fy4ibcvcf6GInBKR6SFqd51hfzlBTd1EQutGvkB/8Lh+c1KhV2mgFxEX8AwwDOgCjBGRLmV2uws4ZozpAMwH5pW5/3Hg/Zo3t+4JBHpb6Zs4+6tM1QVJDb3+ufTao1ehF0yP/ipgjzHmW2NMPrAUGFFmnxHAQv/vbwCDREQARGQksBfYEZIW1zGBVaZsTbHU1E1EuF0xtGzg1UCvwiKYQN8aOFDidqZ/W7n7GGMKgBygqYhcADwEPHq+A4jIJBFJE5G0o0ePBtv2OsH6urGB1I2mFMJNyxWrcAn3YOwjwHxjzKnz7WSMecEY08sY06t5c10guaTi1I21Hn2c9ugjRefSq3AJZnHwg0CbEreT/dvK2ydTRNxAQyAb6A3cKiL/H9AIKBKRPGPM0zVteF3hdft69PZy9BroIyW5cTz/PpFHfkERcW6dEKdCJ5hAvwnoKCLt8AX0O4A7y+zzDjAe+Ay4FVhnjDHAtcU7iMgjwCkN8lVjvUfvigNxaeomApIbx/vm0ufkclHTerabo6JIpd0Gf879fmAVsBN43RizQ0Rmi8hN/t1expeT3wP8BjhnCqaqHuvTK0X8pYq1Rx9urf1z6bW4mQq1YHr0GGNWAivLbJtZ4vc84LZKnuORarSvzrM+vRI00EdIm8Zal16FhyYCHc769EqAWF1lKhJaNvQSI7rSlAo9DfQOZ316Jfhr0mvwCbdYVwxJDbVcsQo9DfQOZ30wFiDuAu3RR0hrrUuvwkADvcNZn14JmrqJIL1oSoWDBnqHc0aPvp6mbiIkuZFvLv3ZQkvVSlVU0kDvcO4YNy5x2V98RHv0EZHcOIEiA//O0br0KnQ00NcCHpeHvAKddVMXFNelP6DpGxVCGuhrAa/b8rqxmrqJmGSdS6/CQAN9LeCIdWML8qBI1zMNt5/m0mugV6Gjgb4WsB7oY3XxkUiJcxfXpddvUCp0NNDXAl63134JBND0TYRouWIVahroa4E4V5z96ZWgPfoIad04XgubqZDSQF8LeF0OGIwFDfQRUlyXvkDn0qsQCap6pbLL4/JwMv+kvQYU5+jfmQLeBvbaUUf8x4kz9HGdomDBM7hjtS9Wp7S9Fvr/NuRPq4G+FvC6vXaLmiX1hA6DIf8UFJ611446whtTSKwUkJ+fhzcm1nZzVCSFaWabBvpawOPy2M3R12sK//GGvePXMcezT3N76oekXtWD23q1qfwBSlVCvxfWAtanV6qISmoYj+hcehVCGuhrAY/LY3d6pYqon+bSa6BXoaGBvhbwuC2nblTEtW4Uz8Hjet2CCg0N9LWA1+XlbNFZCrUEQZ2RrAuQqBDSQF8LBBYI1zx9nZHcOIGsHJ1Lr0JDA30tULzKlNUpliqikhvHU1hk+PcJTdmpmtNAXws4YpUpFVFarliFks6jrwU0dVP3FC9A8oe3t9OkXpzl1qhI6du+Kf/3+ktC/rwa6GuBQI/e5ipTKqLaNEng5stak5WjPfq6RJCwPK8G+lqgOEevPfq6wxUjzB/d03YzVJTQHH0toKkbpVRNaKCvBTR1o5SqCQ30tYBOr1RK1YQG+lpAp1cqpWpCA30toDl6pVRNBBXoRWSoiHwtIntEZEY593tEZJn//s9FpK1/+2AR2SwiX/n//VmI218naI5eKVUTlQZ6EXEBzwDDgC7AGBHpUma3u4BjxpgOwHxgnn/798CNxpjuwHhgcagaXpfo9EqlVE0E06O/CthjjPnWGJMPLAVGlNlnBLDQ//sbwCAREWPMl8aYQ/7tO4B4EfGEouF1iebolVI1EUygbw0cKHE707+t3H2MMQVADtC0zD6jgC3GmHO6pSIySUTSRCTt6NGjwba9znDHuHGLW2fdKKWqJSKDsSLSFV865/+Ud78x5gVjTC9jTK/mzZtHokm1jsft0Ry9Uqpaggn0B4GSKxQn+7eVu4+IuIGGQLb/djKwHBhnjPmmpg2uq3TdWKVUdQUT6DcBHUWknYjEAXcA75TZ5x18g60AtwLrjDFGRBoB7wEzjDGfhKjNdZIGeqVUdVUa6P059/uBVcBO4HVjzA4RmS0iN/l3exloKiJ7gN8AxVMw7wc6ADNFZKv/JzHkZ1EHeFyaulFKVU9Q1SuNMSuBlWW2zSzxex5wWzmP+xPwpxq2UeGbYqk9eqVUdeiVsbWEx+XR6ZVKqWrRQF9LeF1enV6plKoWDfS1hE6vVEpVlwb6WkJn3SilqksDfS2hgV4pVV0a6GsJnV6plKouDfS1hE6vVEpVlwb6WkKnVyqlqksDfS3hdXkpKCqgsKjQdlOUUrWMBvpawuPW5QSVUtWjgb6W0HVjlVLVpYG+ltBAr5SqLg30tYQuEK6Uqq6gqlcq+4oXCJ+6fipel9dya1QoJcQm8Fj/x2gW38x2U1SU0kBfS1yWeBlD2w7VHn2UKaKIDZkbePebd5nQbYLt5qgoJcYY220opVevXiYtLc12M5SKmNHvjgZg2S+WWW6Jqs1EZLMxpld592mOXinLbmh3AxnZGew/sd92U1SU0kCvlGVD2g4B4P2971tuiYpWGuiVsqxlvZZcnng57+99H6elUlV00ECvlAPc0O4Gvs35ln8d+5ftpqgopIFeKQcY3HYwLnHxwb4PbDdFRSEN9Eo5QBNvE/ok9dH0jQoLDfRKOcTQdkM5eOogX33/le2mqCijgV4phxh04SBiY2J19o0KOQ30SjlE/bj6XNv6WlbtW6XrDqiQ0kCvlIMMu3gYR3OPsuXIFttNUVFEA71SDtI/uT/x7nhW7l1puykqimigV8pB4t3xDGwzkNX7V3O26Kzt5qgooYFeKYcZ1m4YOWdy+OzQZ7aboqKEBnqlHKZfq37Uj6vPB3v14ikVGhrolXKYWFcsgy8azNrv1ur6Ayokglp4RESGAk8CLuAlY8zcMvd7gEXAFUA2MNoYs89/38PAXUAhMNUYsypkrVcqSg1rN4y3dr/FzStuDiwjqaLfNa2vYfqV00P+vJUGehFxAc8Ag4FMYJOIvGOMySix213AMWNMBxG5A5gHjBaRLsAdQFegFbBGRC4xxugkYaXO48oWVzK281iO/HjEdlNUBCUmJIbleYPp0V8F7DHGfAsgIkuBEUDJQD8CeMT/+xvA0yIi/u1LjTFngL0issf/fDrKpNR5uGJczLhqhu1mqCgRTI6+NXCgxO1M/7Zy9zHGFAA5QNMgH4uITBKRNBFJO3r0aPCtV0opVSlHDMYaY14wxvQyxvRq3ry57eYopVRUCSbQHwTalLid7N9W7j4i4gYa4huUDeaxSimlwiiYQL8J6Cgi7UQkDt/g6jtl9nkHGO///VZgnfEV1X4HuENEPCLSDugIfBGapiullApGpYOxxpgCEbkfWIVveuUrxpgdIjIbSDPGvAO8DCz2D7b+gO/DAP9+r+MbuC0Afq0zbpRSKrLEaavZ9OrVy6SlpdluhlJK1SoistkY06u8+xwxGKuUUip8NNArpVSUc1zqRkSOAvtr8BTNgO9D1JzaRM+7btHzrluCOe+LjDHlzk93XKCvKRFJqyhPFc30vOsWPe+6pabnrakbpZSKchrolVIqykVjoH/BdgMs0fOuW/S865YanXfU5eiVUkqVFo09eqWUUiVooFdKqSgXNYFeRIaKyNciskdEonbFBhF5RUSOiMj2EtuaiMhqEdnt/7exzTaGg4i0EZH1IpIhIjtE5AH/9qg+dxHxisgXIpLuP+9H/dvbicjn/vf7Mn/BwagjIi4R+VJE3vXfrivnvU9EvhKRrSKS5t9W7fd6VAT6EssdDgO6AGP8yxhGo1eBoWW2zQDWGmM6Amv9t6NNAfD/jDFdgD7Ar/3/x9F+7meAnxljUoCewFAR6YNvuc75xpgOwDF8y3lGoweAnSVu15XzBhhojOlZYv58td/rURHoKbHcoTEmHyhe7jDqGGM24KsQWtIIYKH/94XAyEi2KRKMMVnGmC3+30/i++NvTZSfu/E55b8Z6/8xwM/wLdsJUXjeACKSDAwHXvLfFurAeZ9Htd/r0RLog1qyMIq1MMZk+X//N9DCZmPCTUTaApcBn1MHzt2fvtgKHAFWA98Ax/3LdkL0vt+fAH4LFPlvN6VunDf4Psz/V0Q2i8gk/7Zqv9eDWRxc1SLGGCMiUTtnVkQuAN4E/q8x5oSvk+cTrefuX8Ohp4g0ApYDney2KPxE5BfAEWPMZhEZYLk5NlxjjDkoIonAahHZVfLOqr7Xo6VHX9eXLDwsIkkA/n+PWG5PWIhILL4g/5ox5i3/5jpx7gDGmOPAeuBqoJF/2U6Izvd7P+AmEdmHLxX7M+BJov+8ATDGHPT/ewTfh/tV1OC9Hi2BPpjlDqNZyaUcxwMrLLYlLPz52ZeBncaYx0vcFdXnLiLN/T15RCQeGIxvfGI9vmU7IQrP2xjzsDEm2RjTFt/f8zpjzFii/LwBRKSeiNQv/h34ObCdGrzXo+bKWBG5AV9Or3i5wzl2WxQeIvJ3YAC+sqWHgVnA28DrwIX4SjzfbowpO2Bbq4nINcBHwFf8lLP9Hb48fdSeu4j0wDfw5sLXMXvdGDNbRC7G19NtAnwJ/Icx5oy9loaPP3Uz3Rjzi7pw3v5zXO6/6Qb+ZoyZIyJNqeZ7PWoCvVJKqfJFS+pGKaVUBTTQK6VUlNNAr5RSUU4DvVJKRTkN9EopFeU00CulVJTTQK+UUlHu/wdq5P3YaoUF7QAAAABJRU5ErkJggg==", + "text/plain": [ + "
    " + ] + }, + "metadata": { + "needs_background": "light" + }, + "output_type": "display_data" + } + ], + "source": [ + "scheduler = MultiStepLRScheduler(optimizer, decay_t=[20, 30], decay_rate=0.5)\n", + "plot_lr(scheduler, label='decay_rate=0.5, decay_t=[20, 30]')\n", + "\n", + "scheduler = MultiStepLRScheduler(optimizer, decay_t=[15], decay_rate=0.3)\n", + "plot_lr(scheduler, label='decay_rate=0.3, decay_t=[15]')\n", + "\n", + "scheduler = MultiStepLRScheduler(optimizer, decay_t=[10, 30], decay_rate=0.1)\n", + "plot_lr(scheduler, label='decay_rate=0.1, decay_t=[10, 30]')\n", + "plt.legend();" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Warmup Args." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### `warmup_t` " + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Defines the number of warmup epochs. " + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### `warmup_lr_init` " + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "The initial learning rate during warmup. Default is 0." + ] + }, + { + "cell_type": "code", + "execution_count": 12, + "metadata": {}, + "outputs": [ + { + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXoAAAD4CAYAAADiry33AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/YYfK9AAAACXBIWXMAAAsTAAALEwEAmpwYAAAtSUlEQVR4nO3de3RU9bn/8feTyW1ygZCQIAQwCJRrIIHIRQgVEQVFqCKKtQrnuGr1VOvRnwraFiz9+auKt56j1ntBW48oKmKLVZfAMUorBEQuQUu4WJJwDUnI/Tbf3x8zGZMQyEAm7Jk9z2stVjJ775l5dkg++eY733m2GGNQSillX2FWF6CUUqpzadArpZTNadArpZTNadArpZTNadArpZTNhVtdQGvdu3c3aWlpVpehlFJBZfPmzceMMclt7Qu4oE9LSyM3N9fqMpRSKqiIyHen2qdTN0opZXMa9EopZXMa9EopZXMa9EopZXMa9EopZXM+Bb2ITBORb0UkX0QWtrF/kohsEZEGEbm21b55IrLb82+evwpXSinlm3aDXkQcwLPAdGAocIOIDG112L+A+cAbre6bCCwGxgJjgMUi0q3jZSullPKVL+voxwD5xpi9ACLyJjALyGs6wBiz37PP1eq+lwOfGGOOe/Z/AkwD/qfDlbdyqKyGN7485TLSNjUcf5eSqr/7uxR1lrpGJ3P/9S8S5nBYXYpStuJL0KcCB5rdLsA9QvdFW/dNbX2QiNwK3ArQt29fHx+6pcMnavjvdflndJ8h/d7gQBSI9uS3nBGB+n3M2JfL8AG+fnsppXwREO+MNca8CLwIkJWVdVapO7JPAvt+d+WZPCfjli/kJxHnseDHn5zNUyo/emHVgzxT9gFHjh/A93GEUsoXvrwYWwj0aXa7t2ebLzpy305VVltGlUCvyASrS1FA15juABwvP2xxJUrZjy9BvwkYKCL9RCQSmAus9vHxPwIuE5FunhdhL/Nss1xheQEAvaKTLK5EASTE9QDgRNUxiytRyn7aDXpjTANwB+6A3gW8ZYzZKSJLRGQmgIhcKCIFwBzgBRHZ6bnvceC3uH9ZbAKWNL0wa7XC0j0ApMb2sLgSBZDU9TwATlQXW1yJUvbj0xy9MWYNsKbVtkXNPt+Ee1qmrfu+CrzagRo7RVHZPgB6xZ302rCyQEo39wxfZV2ZxZUoZT8h+87YwhMHiG900UVH9AGhR5J7nFBVf8LiSpSyn5AN+qLKg6Q2NEB0V6tLUUB0VAwxLhdVjZVWl6KU7YRu0FcfoZcGfUCJdUG1q9rqMpSynZAMemMMhbXHNegDTIwrjBpTa3UZStlOSAZ9SW0J1a56UhsaITrB6nKUh9M4qJF6q8tQynZCMuiLKooA3HP0UV0srkY1cRJBtTRYXYZSthOSQV9Y4X5zbi+iwBEQXSAUEE0klSf1xVNKdVRIBn3TiL5XRJzFlajmnOKkKiS/I5XqXCH5Y1VYUUgXHMRH6QuxgSTGEUNFmNDQoPP0SvlTyAZ9qnHoipsA44yIx4hwpKTI6lKUspWQDPqiiiJ6uVwa9AEmLtL9/+FuVayU8peQC3pjDEUVRaTW6xr6QBMX7b7K5LHSgxZXopS9hFzQH685Tk1jDb1qqzXoA0wXp7snfYn2pFfKr0Iu6L1r6KsrNOgDTLd4d4O5sqqjFleilL2EXNAXVnrW0Gv7g4CT1KUXABU1AXHJAqVsI/SCvlyDPlD1SHRfG6CiVnvSK+VPIRf0RRVFdI2II84YcCZYXY5qpkdSXwCqGsotrkQpewm5oC+sLCQ1KtF9Q0f0ASUyMopYl4uqxgqrS1HKVkIu6IsqikiN8DQy06APOLEuqNGe9Er5VUgFfdMa+l6OGPcGDfqAE+MKo1p70ivlVyEV9MU1xdQ21tJLIt0bNOgDjtM4qNae9Er5VUgFfVN74lQc7g3aiz7gRJsIqqXR6jKUspWQCnrvm6UajTvkwxwWV6Rac0oU1dqTXim/Cqmg915wpL5ep20ClFOiqQyp70qlOl9I/UgVVRTRLaobMbXa/iBQOcNjqdSe9Er5VcgFfa+4XlBTpkEfoGLC3T3pDx0vtLoUpWwjpIK+sKJQgz7AxWpPeqX8LmSC3mVc7jdLxaVq0AewuGj3u5aLS/UqU0r5S8gEfXF1MXWuumZBn2B1SaoNCTHJABwv14uPKOUvIRP03hU3sT2h9oSO6APU9z3piy2uRCn7CJmg966hj+gCGA36AJWUoD3plfI3n4JeRKaJyLciki8iC9vYHyUiKzz7vxSRNM/2CBFZLiLbRWSXiDzg5/p9VlTpDvqeDqd7gwZ9QEpJ7ANAZZ32pFfKX9oNehFxAM8C04GhwA0iMrTVYbcAJcaYAcBTwKOe7XOAKGNMOjAa+FnTL4FzraC8gMToRGIaPA2zNOgD0nmJqYgx2pNeKT/yZUQ/Bsg3xuw1xtQBbwKzWh0zC1ju+XwlMEVEBDBArIiEA06gDjjhl8rPUFFFEb1iPUsrQYM+QIWHRxDrMlQ3VFpdilK24UvQpwLNFzUXeLa1eYwxpgEoA5Jwh34lcBD4F/C4MeakyVcRuVVEckUk9+jRzrkwdFFlEanxqRr0QSDWBdWmxuoylLKNzn4xdgzQCPQC+gH/R0QuaH2QMeZFY0yWMSYrOTnZ70U0raH3vlkKNOgDmNNoT3ql/MmXoC8E+jS73duzrc1jPNM0XYFi4MfA34wx9caYI8AXQFZHiz5Tx6qPUe+qJzVWR/TBwGkc1GhPeqX8xpeg3wQMFJF+IhIJzAVWtzpmNTDP8/m1wFpjjME9XXMJgIjEAuOAb/xR+JloWlr5/YhetBd9AHNqT3ql/KrdoPfMud8BfATsAt4yxuwUkSUiMtNz2CtAkojkA/cATUswnwXiRGQn7l8YfzTGbPP3SbSnoKIA4Pt3xUZ1gbCQeQtB0HFKFFVh2pNeKX8J9+UgY8waYE2rbYuafV6Deyll6/tVtLX9XDtpRO/UaZtA5gyLoSKsxOoylLKNkBjWFlUUkRSdRHR4tDY0CwJORyxVYWHU1FZZXYpSthASQV9YUeietgGoLtWGZgEuJjwegKPak14pvwiJoPcurQQd0QeBuCj3/89hDXql/ML2Qe8yLooqi+gZ19O9QYM+4Hl70p/QnvRK+YPtg/5I1REaXA30juvt3qBBH/C6enrSl5QftrgSpezB9kHvbU8clwqNDVBXrkEf4Jp60p+oPmZxJUrZg+2D3nvBkbhe7guOgAZ9gOue4J5mq6jRJZZK+YPtg75pRN8ztqe2PwgSTT3pK7QnvVJ+Yf+gryyiu7P792voQYM+wKV064UYQ3W99qRXyh9sH/SFFYUtl1aCBn2ACw+PIM5lqGrUN0wp5Q/2D/ryQnfXSmgW9AmW1aN8E+sSqk211WUoZQu2DvpGVyOHKg+5LzgCOqIPIjFGtCe9Un5i66A/Wn2UBtPQbOqm1P1Rgz7gOU04NdJgdRlK2YKtg75paWWLqRsJg8g4C6tSvojWnvRK+Y2tg75Fe2LQXvRBJFp70ivlN7ZOvKYLjmifm+DjDHNSaevvTqXOHVv/KBVVFJHiTCHKEeXeoEEfNGK0J71SfmP7oPdO24AGfRCJiXBf0/dwcYHFlSgV/Gwd9C3eLAUa9EEkNtL9/3Sk5IDFlSgV/Gwb9A2uBg5XHv7+ylLgCfoEy2pSvuviTAKguOyQxZUoFfxsG/RHq1qtoQcd0QeRLjHdASit0J70SnWUbYO+acWNN+gbG6CuApwJ1hWlfJbU5TwAyiqPWlyJUsHPtkHftIbee2Up7UUfVJIS3FNu5bXak16pjrJ10AvCebHukaG2PwguPTw96Su1J71SHRZudQGdpbCikOSYZCIdke4N1aXujxr0QSE54TzCjKGqscLqUpQKevYd0VcWnbziBjTog0SYw0Gcy1DdWGl1KUoFPfsGfVtvlgIN+iDi7klfY3UZSgU9WwZ9g6uBQ5WH6BWrQR/MnEaoMXVWl6FU0LNl0B+uOkyjaaR3fO/vN2rQBx2nCada6q0uQ6mgZ8ugP6k9MWgv+iDkNJFUi7YqVqqjfAp6EZkmIt+KSL6ILGxjf5SIrPDs/1JE0prtGyEifxeRnSKyXUSi/Vh/m0664Ah8/65Ykc5+euUn0RKpPemV8oN2g15EHMCzwHRgKHCDiAxtddgtQIkxZgDwFPCo577hwJ+A24wxw4CLgU7/W/ykNfSg7Q+CkDMsRnvSK+UHvvwYjQHyjTF7jTF1wJvArFbHzAKWez5fCUwREQEuA7YZY74GMMYUG2M6/fpwhRWFpMSkEOGI+H6jNjQLOjGOOKrDwqiq0SWWSnWEL0GfCjTvFVvg2dbmMcaYBqAMSAJ+ABgR+UhEtojI/W09gYjcKiK5IpJ79GjHe5sUVhS2XEMPOqIPQrGR8QAcOrrf2kKUCnKd/YdxODARuNHz8WoRmdL6IGPMi8aYLGNMVnJycoeftKiiSIPeBmIjEwA4UlpobSFKBTlfgr4Q6NPsdm/PtjaP8czLdwWKcY/+PzPGHDPGVAFrgFEdLfp06l31HK463HLFDbh73WjQB5V4T0/649qTXqkO8SXoNwEDRaSfiEQCc4HVrY5ZDczzfH4tsNYYY4CPgHQRifH8AvghkOef0tt2uPIwLuPSEb0NJMS6/7orrdSe9Ep1RLtNzYwxDSJyB+7QdgCvGmN2isgSINcYsxp4BXhdRPKB47h/GWCMKRGRJ3H/sjDAGmPMXzvpXIBTrKFvqIP6Kn0xNsh0i+8BQFnVcYsrUSq4+dS90hizBve0S/Nti5p9XgPMOcV9/4R7ieU50bSGvkXQay/6oNQ9wf1/WFGjQa9UR9hulXJhRSFhEnbyGnrQoA8yPZLcLw1V1WtPeqU6wnb96IsqiugR04OIsOZr6EvdHzXog0pSlx44jKFSe9Ir1SG2HNGfvOJGR/TBqKknfU1jldWlKBXUbBf0J11wBDTog1iM9qRXqsNsFfT1jfUcqTpy6hG9M+Gc16Q6JsaEUa096ZXqEFsF/aHKQ7iMq+UFR0BH9EFMe9Ir1XG2CvrCSvfSyhYXHAF30IeFQ0SMBVWpjnASQbV0eh88pWzNVkHf5pulQHvRB7FooqkKM1aXoVRQs1XQF1YU4hAHPWJ6tNxRXarTNkHKGeakIkx/QSvVEbYK+qY19OFhrd4eoH1ugpYzPJbaMKG8stTqUpQKWrYL+pOmbUCDPojFRnQB4HBxgcWVKBW8bBX0BRUFGvQ2E+fpSX+0VINeqbNlm6Cva6zjaNVResf1PnmnBn3Qio9JBOD4Ce1Jr9TZsk2vm0OVhzAYHdHbTEJMCgDL8p5m5c4/WFxN54iSSH47ZwXJ3dr43lXKD2wT9FUNVQxIGEDfLn1b7miog4ZqDfoglTXscgbtfo4qaaRK7Nfzpl4MByOq+PjLP3HjtDYvqaxUh9km6AcnDua9We+dvMP7rtiEc1qP8o/eKWmsvPVrq8voNNt2/4MbN/yUwpJ8q0tRNmabOfpT0qBXAWzw+ZmEG8OxKr0Auuo8thnRn5L2uVEBLDIyiuQGOO7Sq2ipzhMCI/pS90cNehWgklwRlGC/1x9U4NCgV8pi3YjnmKPB6jKUjYVA0OvUjQpsiRHJHA8Po7T8mNWlKJvSoFfKYilx7oug5+3dZHElyq5CI+jDIiDCaXUlSrWpT/fBAOwt2mZxJcquQiPotRe9CmAD+2QCUFiy2+JKlF2FTtArFaAGp41yr6WvLrK6FGVTobGOXoNeBbDw8AhSGuC4q8TqUpRN6YheqQCQ6IqkhEqry1A2FRpB70ywugqlTsu9ll4vgq46R2gEvY7oVYBLjOxOSXgYJWVHrS5F2ZD9g14vDK6CQI+48wHYufdLiytRdmTvoK+vgcZaDXoV8Po2raU/qGvplf/5FPQiMk1EvhWRfBFZ2Mb+KBFZ4dn/pYiktdrfV0QqROReP9XtG31XrAoSP+g7GoCi0j0WV6LsqN2gFxEH8CwwHRgK3CAiQ1sddgtQYowZADwFPNpq/5PAhx0v9wxpL3oVJAb2HUGErqVXncSXEf0YIN8Ys9cYUwe8CcxqdcwsYLnn85XAFBH3W1FF5EfAPmCnXyo+EzqiV0EiPDyC5AYoadC19Mr/fAn6VOBAs9sFnm1tHmOMaQDKgCQRiQMWAL853ROIyK0ikisiuUeP+nHVgQa9CiJJjZEct+F1cZX1OvvF2IeAp4wxFac7yBjzojEmyxiTlZyc7L9n1170Koh0ky4cC9e19Mr/fGmBUAj0aXa7t2dbW8cUiEg40BUoBsYC14rIY0AC4BKRGmPMMx0t3Cc6oldBJCkqmVKKKS49RFLCeVaXo2zElxH9JmCgiPQTkUhgLrC61TGrgXmez68F1hq3bGNMmjEmDXga+H/nLORBg14FlRTPWvq8vRstrkTZTbtB75lzvwP4CNgFvGWM2SkiS0RkpuewV3DPyecD9wAnLcG0RE0ZOKK0F70KCt6+9Ae3W1yJshufulcaY9YAa1ptW9Ts8xpgTjuP8dBZ1NcxNaU6mldBY3Df0bAfDupaeuVn9n5nrPa5UUGkf5/hRLoMR3UtvfIzDXqlAkR4eAQpjVCifemVn2nQKxVAEhujOC7VVpehbEaDXqkA0i1M+9Ir/9OgVyqAJEamUOYI42iJztMr/7Fv0BujQa+CTo94T1/6PdqXXvmPfYO+oQYa6zToVVDp230IAPsO7bC4EmUn9g16fVesCkKDzs8C4FCZrqVX/qNBr1QAGdB7GFEuw7Hqg1aXomwkBII+wdIylDoTYQ4HyY1wvLHU6lKUjdg36KtL3R+dCVZWodQZS2yMokTX0is/sm/Q69SNClLdwrpwVPvSKz+ycdCXuj9q0Ksg0z2yByccYRw6dqD9g5XygY2D3jOij+pibR1KnaGmtfS79mlfeuUf9g768GiIiLa6EqXOSN+UprX0Oy2uRNmFvYNep21UEBrU172W/mDZXosrUXbh04VHgpIGvQpSF6QOIcplKK7XtfTKPzTolQowYQ4HKQ1wXPvSKz/RqRulAlCiK5qSMF1Lr/xDg16pAJQY1oWjDpfVZSibsHHQl2rQq6CVFJVCuSOMoqPfWV2KsgF7Br23F32C1ZUodVZS4vsB8M2+TRZXouzAni/G1leBq0FH9CponZ88BE78hfW73uJYmV5tKlT0TEoje9RMvz+uPYNe+9yoIJc+YCLh+Y/xnuzivaJdVpejzpEx++I06H2mQa+CXJ/zLuC/Rz5C0TF901Qo6ZmU1imPq0GvVICamDnD6hKUTdjzxVi96IhSSnnZPOh1RK+UUjp1o86p+vp6CgoKqKmpsboUpYJSdHQ0vXv3JiIiwuf72DToS90fo7UXfaApKCggPj6etLQ0RMTqcpQKKsYYiouLKSgooF+/fj7fz6epGxGZJiLfiki+iCxsY3+UiKzw7P9SRNI826eKyGYR2e75eInPlXVEdSmEOyE86pw8nfJdTU0NSUlJGvJKnQURISkp6Yz/Im436EXEATwLTAeGAjeIyNBWh90ClBhjBgBPAY96th8DrjLGpAPzgNfPqLqzpX1uApqGvFJn72x+fnwZ0Y8B8o0xe40xdcCbwKxWx8wClns+XwlMERExxnxljGl6W99OwCkinT/M1qBXSikvX4I+FWh+leICz7Y2jzHGNABlQFKrY2YDW4wxta2fQERuFZFcEck9evSor7WfWk0ZOBM6/jhKKWUD52R5pYgMwz2d87O29htjXjTGZBljspKTkzv+hDqiVwGsoaHB6hJUiPFl1U0h0KfZ7d6ebW0dUyAi4UBXoBhARHoD7wE3G2P2dLhiX9SUQfeB5+Sp1Nn7zQc7ySs64dfHHNqrC4uvGnbK/fv372f69OlMnDiRDRs2kJqayvvvv4/T6WTr1q3cdtttVFVV0b9/f1599VW6devmvW9jYyMDBgxg7969lJWVkZSUxLp165g0aRKTJk3ilVdeoaSkhLvuuouamhqcTid//OMfGTRoEMuWLePdd9+loqKCxsZG/u3f/o1Vq1ZRWVnJ7t27uffee6mrq+P1118nKiqKNWvWkJiYyMUXX8zjjz9OVlYWx44dIysri/3797Ns2TLee+89ysrKKCws5Cc/+QmLFy/269dS2YcvI/pNwEAR6ScikcBcYHWrY1bjfrEV4FpgrTHGiEgC8FdgoTHmCz/V3D4d0avT2L17Nz//+c/ZuXMnCQkJvPPOOwDcfPPNPProo2zbto309HR+85vftLifw+Fg0KBB5OXl8fnnnzNq1ChycnKora3lwIEDDBw4kMGDB5OTk8NXX33FkiVLePDBB73337JlCytXruR///d/AdixYwfvvvsumzZt4pe//CUxMTF89dVXjB8/ntdee63d89i4cSPvvPMO27Zt4+233yY3N9ePXyVlJ+2O6I0xDSJyB/AR4ABeNcbsFJElQK4xZjXwCvC6iOQDx3H/MgC4AxgALBKRRZ5tlxljjvj7RJoVrEEfJE438u5M/fr1IyMjA4DRo0ezf/9+ysrKKC0t5Yc//CEA8+bNY86cOSfdNzs7m88++4x9+/bxwAMP8NJLL/HDH/6QCy+8EICysjLmzZvH7t27ERHq6+u99506dSqJiYne25MnTyY+Pp74+Hi6du3KVVddBUB6ejrbtm1r9zymTp1KUpL7pbBrrrmGzz//nKysrLP7oihb82mO3hizxhjzA2NMf2PMw55tizwhjzGmxhgzxxgzwBgzxhiz17P9/xpjYo0xGc3+dV7IA9RVgmnUoFenFBX1/cIvh8NxRnPmkyZNIicnh40bN3LFFVdQWlrK+vXryc7OBuDXv/41kydPZseOHXzwwQct1jvHxsaeso6wsDDv7bCwMG9N4eHhuFzuSwq2XjvdepmdLltVp2K/Xjfa/kCdha5du9KtWzdycnIAeP31172j++bGjBnDhg0bCAsLIzo6moyMDF544QUmTZoEuEf0qanuRWnLli3rcF1paWls3rwZgJUrV7bY98knn3D8+HGqq6tZtWoVEyZM6PDzKXvSoFfKY/ny5dx3332MGDGCrVu3smjRopOOiYqKok+fPowbNw5wT+WUl5eTnp4OwP33388DDzxAZmamX1bX3HvvvfzhD38gMzOTY8eOtdg3ZswYZs+ezYgRI5g9e7ZO26hTEmOM1TW0kJWVZTr0otJ3G+CP0+Gm96D/uem4oHy3a9cuhgwZYnUZQW/ZsmXk5ubyzDPPWF2KskBbP0cistkY0+Zvex3RK6WUzdmve6VedESFgPnz5zN//nyry1BBwsYj+gRLy1BKqUBh46DXXvRKKQV2DfqIWHD4fvUVpZSyMxsGfam+EKuUUs3YMOi1/YFSSjWnQa9UENu6dStr1qzx+fgDBw4wefJkhg4dyrBhw/j973/fidWpQGHP5ZVx51ldhfLFhwvh0Hb/PuZ56TD9Ef8+5hloaGggPPzc/Vht3bqV3NxcrrjiCp+ODw8P54knnmDUqFGUl5czevRopk6dytChra8OquzEfiP66lId0asztmnTJq655hoAb3/6uro6ampquOCCCwB46aWXuPDCCxk5ciSzZ8+mqqoKcK9pv+222xg7diz3338/8+fP5/bbb2fcuHFccMEFrF+/nn//939nyJAhLda+x8XFeT9fuXKld1/T42VlZfGDH/yAv/zlL23WXFdXx6JFi1ixYgUZGRmsWLGi3fPs2bMno0aNAiA+Pp4hQ4ZQWNj68hLKbuw5otegDw4Wjrxby8zMZOvWrQDk5OQwfPhwNm3aRENDA2PHjgXcrYB/+tOfAvCrX/2KV155hTvvvBOAgoICNmzYgMPhYP78+ZSUlPD3v/+d1atXM3PmTL744gtefvllLrzwQrZu3eptk3wq+/fvZ+PGjezZs4fJkyeTn59PdHR0i2MiIyNZsmRJi1YI69at4+677z7p8WJiYtiwYcNJz/HVV195z0/Zl72C3uWC2hMa9OqMhYeH079/f3bt2sXGjRu55557+Oyzz2hsbPS2IN6xYwe/+tWvKC0tpaKigssvv9x7/zlz5uBwOLy3r7rqKkSE9PR0evTo4W16NmzYMPbv399u0F933XWEhYUxcOBALrjgAr755pt27wPuHvdNv7BOp6KigtmzZ/P000/TpYu+58Tu7BX0dRVgXBr06qxMmjSJDz/8kIiICC699FLmz59PY2MjS5cuBdxTKqtWrWLkyJEsW7aM9evXe+97ql7zzfvMN91u6mrZvH+8v3rN+zKir6+vZ/bs2dx4443e6Splb/aao9eGZsoHU6ZMaXNeOjs7m6effprx48eTnJxMcXEx3377LcOHDwegvLycnj17Ul9fz5///OcO19GjRw927dqFy+Xivffea7Hv7bffxuVysWfPHvbu3cugQYPafIz4+HjKy8u9t5tG9K3/NYW8MYZbbrmFIUOGcM8993T4HFRwsGfQOxMsLUMFLpfLRX5+fotL+jUZO3Yshw8f9l5EZMSIEaSnp3tH07/97W8ZO3YsEyZMYPDgwR2u5ZFHHmHGjBlcdNFF9OzZs8W+vn37MmbMGKZPn87zzz9/0vx8k8mTJ5OXl+fzi7FffPEFr7/+OmvXriUjI4OMjIwzWp6pgpO9+tHv/wKWXQE3vw8XXOzXupR/WN2PfseOHbz66qs8+eSTltXQnvnz5zNjxgyuvfZaq0tRASq0+9Hr1I1qx/DhwwM65JXqDPZ6MVaDXtlAW9ea/eijj1iwYEGLbf369Ttpbl+pttg06BMsLUMpf7v88stbLOdU6kzYbOqm1P0xStcFK6VUE5sFfRlExoHDXn+oKKVUR9gv6HV+XimlWtCgV0opm9OgVyoIrFq1iry8PJ+P/+STTxg9ejTp6emMHj2atWvXdmJ11tm/f7/3ncu+KCoq8un9CVdccQWlpaWUlpby3HPPnVFN+/btY+zYsQwYMIDrr7+eurq6No/73e9+x4ABAxg0aBAfffSRd3taWhrp6elkZGSQldXmsvgzZq/J7JpS6JJqdRXKR49ufJRvjn/j18ccnDiYBWMWtH9gBzU2NrZoYtbZVq1axYwZM3zuG9+9e3c++OADevXqxY4dO7j88svPSTvic92P/0zr6NWrFytXrmz3/k3vFt6/fz/PPfcc//Ef/+Hzcy9YsIC7776buXPnctttt/HKK69w++23tzgmLy+PN998k507d1JUVMSll17KP//5T+/31Lp16+jevbvPz9keG47oE6yuQgWwpUuX8l//9V8A3H333VxyySUArF27lhtvvBGA22+/naysLIYNG8bixYu9901LS2PBggWMGjWKt99+m7S0NB544AHvyGvLli1cfvnl9O/fn+effx6A9evXM2PGDO9j3HHHHd518mlpadx///2kp6czZswY8vPz26x5w4YNrF69mvvuu4+MjAz27NnT7nlmZmbSq1cvwN0xs7q6mtra2lMef+TIEUaPHg3A119/jYjwr3/9C4D+/ftTVVXFBx98wNixY8nMzOTSSy/l8OHDADz00EPcdNNNTJgwgZtuuomHHnqIefPmkZ2dzfnnn8+7777rPc9p06ZRX1/vPf9jx44BkJuby8UXX9zi8caPH8/AgQN56aWX2j1fcL//YObMmVxyySVMmTKlzWOa/wWwbNkyrrnmGqZNm8bAgQO5//77vcc11bZw4UL27NlDRkYG9913X7s1GGNYu3at96+GefPmsWrVqpOOe//995k7dy5RUVH069ePAQMGsHHjRp/O82xY/6vXn3TqJqici5F3a9nZ2TzxxBP84he/IDc3l9raWurr68nJyfH2uHn44YdJTEyksbGRKVOmsG3bNkaMGAFAUlISW7ZsAWDhwoX07duXrVu3cvfddzN//ny++OILampqGD58OLfddlu79XTt2pXt27fz2muv8Z//+Z9tXmTkoosuYubMmS3aIixdurTNxmqTJk3y/iJr8s477zBq1KgWXTRbS0lJoaamhhMnTpCTk0NWVhY5OTlMnDiRlJQUYmJimDhxIv/4xz8QEV5++WUee+wxnnjiCcA9Qv38889xOp089NBD7Nmzh3Xr1pGXl8f48eN55513eOyxx7j66qv561//yo9+9KPTfl22bdvGP/7xDyorK8nMzOTKK6/0/uI6nS1btrBt27Y2exm1ZevWrXz11VdERUUxaNAg7rzzTvr06ePd/8gjj7Bjxw5v6+fy8nJv2+rW3njjDVJSUkhISPD+NdG7d+82/5IqLCxk3Lhx3tvNjxMRLrvsMkSEn/3sZ9x6660+ncvp2CfoXS6o0V706vRGjx7N5s2bOXHiBFFRUYwaNYrc3FxycnK8AfnWW2/x4osv0tDQwMGDB8nLy/MG/fXXX9/i8WbOnAlAeno6FRUVxMfHEx8fT1RUFKWlpe3Wc8MNN3g/ttVe+FTuu+8+n0aYO3fuZMGCBXz88cftHnvRRRfxxRdf8Nlnn/Hggw/yt7/9DWOMN9gKCgq4/vrrOXjwIHV1dfTr189735kzZ+J0Or23p0+fTkREBOnp6TQ2NjJt2jTA/XXav39/u7XMmjULp9OJ0+lk8uTJbNy4sd1fDgBTp071OeTB3cm0a1d3ZgwdOpTvvvuuRdC3Fh8ff9p+/01/oXTE559/TmpqKkeOHGHq1KkMHjzYOwg5Wz5N3YjINBH5VkTyRWRhG/ujRGSFZ/+XIpLWbN8Dnu3fikjnvbWvrhwwGvTqtCIiIujXrx/Lli3joosuIjs7m3Xr1pGfn8+QIUPYt28fjz/+OJ9++inbtm3jyiuvbNEr/kz7zoeHh+NyubzbT9d33tee8+Ae0Td1n2z+7xe/+IX3mIKCAq6++mpee+01+vfv3+5jTpo0iZycHL777jtmzZrF119/zeeff+4N+jvvvJM77riD7du388ILL/j8dYmIiPCeW/N+/M2/Nv7qx9+6jvY0/z9zOBze2k6lvLy8za97RkYGeXl5JCUlUVpa6n2cgoICUlNPft0wNTWVAwcOeG83P67pY0pKCldffbVfpnTaDXoRcQDPAtOBocANItL6FaFbgBJjzADgKeBRz32HAnOBYcA04DnP4/mf9rlRPsrOzubxxx9n0qRJZGdn8/zzz5OZmYmIcOLECWJjY+natSuHDx/mww8/7NBznX/++eTl5VFbW0tpaSmffvppi/1NrYVXrFjB+PHjT/k4rfvO33fffW32nW/6q6S0tJQrr7ySRx55hAkTJrR4rJtvvrnN8MjOzuZPf/oTAwcOJCwsjMTERNasWcPEiRMBKCsr84bQ8uXLz+Kr0VJaWhqbN28G3NNLzb3//vvU1NRQXFzM+vXrufDCCzv8fGej9de9aUTf1r+hQ4ciIkyePNn7gu/y5cuZNWvWSY87c+ZM3nzzTWpra9m3bx+7d+9mzJgxVFZWep+vsrKSjz/++IxWFZ2KLyP6MUC+MWavMaYOeBNoXfksoOl/fiUwRdy/gmcBbxpjao0x+4B8z+P5X3Wp+6MGvWpHdnY2Bw8eZPz48fTo0YPo6GjvqHXkyJFkZmYyePBgfvzjH58UkmeqT58+XHfddQwfPpzrrruOzMzMFvtLSkoYMWIEv//973nqqadO+Thz585l6dKlZGZm+vRi7DPPPEN+fj5LlizxjjiPHDkCuOe/25rvTktLwxjjnSaYOHEiCQkJdOvWDXC/SDpnzhxGjx7tlxUhixcv5q677iIrK+ukFUwjRoxg8uTJjBs3jl//+tc+zc93hqSkJCZMmMDw4cN9mioDePTRR3nyyScZMGAAxcXF3HLLLQCsXr2aRYsWAe4XyK+77jqGDh3KtGnTePbZZ3E4HBw+fJiJEycycuRIxowZw5VXXumd9uoQY8xp/wHXAi83u30T8EyrY3YAvZvd3gN0B54BftJs+yvAtW08x61ALpDbt29fc1aO7jZmxc3GFG09u/urcyIvL8/qEgLG+eefb44ePXpOn7OsrMxce+215/Q5z9TixYvN0qVLrS4joLX1cwTkmlPkeEAsrzTGvGiMyTLGZCUnJ5/dg3QfANcth54j/VucUjbSpUsX3n77bavLUOeYL6tuCoHmL0P39mxr65gCEQkHugLFPt5XqZDU1uqThx9++KQgnjNnDr/85S/PUVXWe+ihh07atn37dm666aYW26Kiovjyyy/bfIwzPd7u2r2UoCe4/wlMwR3Sm4AfG2N2Njvm50C6MeY2EZkLXGOMuU5EhgFv4J6X7wV8Cgw0xjSe6vk6dClBFfB27drF4MGDz2iFiVLqe8YYvvnmmzO6lGC7I3pjTIOI3AF8BDiAV40xO0VkCe45odW4595fF5F84DjulTZ4jnsLyAMagJ+fLuSV/UVHR1NcXExSUpKGvVJnyBhDcXHxKS8Wfyr2uji4Cnj19fUUFBSctG5aKeWb6OhoevfuTURERIvtHRrRK+VPTW9YUkqdOwGx6kYppVTn0aBXSimb06BXSimbC7gXY0XkKPBdBx6iO9DxFnLBR887tOh5hxZfzvt8Y0yb7zgNuKDvKBHJPdUrz3am5x1a9LxDS0fPW6dulFLK5jTolVLK5uwY9C9aXYBF9LxDi553aOnQedtujl4ppVRLdhzRK6WUakaDXimlbM42Qd/eBcztQkReFZEjIrKj2bZEEflERHZ7PnazssbOICJ9RGSdiOSJyE4Rucuz3dbnLiLRIrJRRL72nPdvPNv7iciXnu/3FSISaXWtnUFEHCLylYj8xXM7VM57v4hsF5GtIpLr2XbW3+u2CHofL2BuF8twX2i9uYXAp8aYgbh7/tvxF10D8H+MMUOBccDPPf/Hdj/3WuASY8xIIAOYJiLjgEeBp4wxA4AS4BbrSuxUdwG7mt0OlfMGmGyMyWi2fv6sv9dtEfT4dgFzWzDGfIa7539zzS/Ovhz40bms6Vwwxhw0xmzxfF6O+4c/FZufu+dyoBWemxGefwa4BFjp2W678wYQkd7AlcDLnttCCJz3aZz197pdgj4VONDsdoFnW6joYYw56Pn8ENDDymI6m4ikAZnAl4TAuXumL7YCR4BPgD1AqTGmwXOIXb/fnwbuB1ye20mExnmD+5f5xyKyWURu9Ww76+917UdvM8YYIyK2XTMrInHAO8B/GmNONL9KlV3P3XNVtgwRSQDeAwZbW1HnE5EZwBFjzGYRudjicqww0RhTKCIpwCci8k3znWf6vW6XEX2oX4T8sIj0BPB8PGJxPZ1CRCJwh/yfjTHvejaHxLkDGGNKgXXAeCDBcz1nsOf3+wRgpojsxz0Vewnwe+x/3gAYYwo9H4/g/uU+hg58r9sl6DcBAz2vyEfivmbtaotrOpdWA/M8n88D3rewlk7hmZ99BdhljHmy2S5bn7uIJHtG8oiIE5iK+/WJdcC1nsNsd97GmAeMMb2NMWm4f57XGmNuxObnDSAisSIS3/Q5cBmwgw58r9vmnbEicgXuOb2mC5g/bG1FnUNE/ge4GHfb0sPAYmAV8BbQF3eL5+uMMa1fsA1qIjIRyAG28/2c7YO45+lte+4iMgL3C28O3AOzt4wxS0TkAtwj3UTgK+Anxpha6yrtPJ6pm3uNMTNC4bw95/ie52Y48IYx5mERSeIsv9dtE/RKKaXaZpepG6WUUqegQa+UUjanQa+UUjanQa+UUjanQa+UUjanQa+UUjanQa+UUjb3/wGanHeGQBXCjQAAAABJRU5ErkJggg==", + "text/plain": [ + "
    " + ] + }, + "metadata": { + "needs_background": "light" + }, + "output_type": "display_data" + } + ], + "source": [ + "scheduler = MultiStepLRScheduler(optimizer, decay_t=[20, 30], decay_rate=0.5)\n", + "plot_lr(scheduler, label='no warmup')\n", + "\n", + "scheduler = MultiStepLRScheduler(optimizer, decay_t=[20, 30], decay_rate=0.5, warmup_t=2)\n", + "plot_lr(scheduler, label=', warmup_t=2')\n", + "\n", + "scheduler = MultiStepLRScheduler(optimizer, decay_t=[20, 30], decay_rate=0.5, warmup_t=2, warmup_lr_init=0.05)\n", + "plot_lr(scheduler, label='warmup_t=2, warmup_lr_init=0.05')\n", + "\n", + "plt.legend();" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "As we can see by setting up `warmup_t` and `warmup_lr_init`, the scheduler first starts with a value of `warmup_lr_init`, then during `warmup_t` number of epochs gradually progresses up to the LR value at epoch `warmup_t + 1`." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Noise Args." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### `noise_range_t`" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "If it is number - its number of epoch when noise starts. \n", + "If list or tuple (of two elements) - first and second element is epoch number range, when noise applied." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "The upper and lower limit of noise. " + ] + }, + { + "cell_type": "code", + "execution_count": 13, + "metadata": { + "scrolled": true + }, + "outputs": [ + { + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXoAAAD4CAYAAADiry33AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/YYfK9AAAACXBIWXMAAAsTAAALEwEAmpwYAAA87klEQVR4nO2deXzU5b3v399kMtkTAmQjAYKAVDYXqEvx1H232tOjrbRW29pb9dSeVo8e6e0toqc91eJVW7WnR4+21N5We7VWrqValVr3BRWURSBggLAkELLvyTz3j2d+k8lkJrNkJkMm3/frxWtmfr/n9/yegeEz3/k+30WMMSiKoiipS1qyF6AoiqIkFhV6RVGUFEeFXlEUJcVRoVcURUlxVOgVRVFSHFeyFxDI5MmTTVVVVbKXoSiKMqZ47733DhljioOdO+KEvqqqinXr1iV7GYqiKGMKEdkV6py6bhRFUVIcFXpFUZQUR4VeURQlxTnifPSKMtbp7e2ltraWrq6uZC9FSUGysrKorKwkIyMj4mtU6BUlztTW1pKfn09VVRUikuzlKCmEMYaGhgZqa2uZMWNGxNdF5LoRkfNFZKuIVIvIsiDnPysi74tIn4hcFnDuahHZ7v1zdcQrU5QxSldXF5MmTVKRV+KOiDBp0qSofy2GFXoRSQceBC4A5gJLRWRuwLDdwNeA3wVcOxG4DTgJOBG4TUSKolqhooxBVOSVRBHLZysS182JQLUxZqf3Jo8DlwKbnQHGmBrvOU/AtecBLxhjDnvPvwCcD/w+6pWGoX7vJ+z8y/3xnnZM0C8uPiz7Ap0ZifsOzcl08fUlVWS60hN2D0VREkMkQl8B7PF7XYu10CMh2LUVgYNE5FvAtwCmTZsW4dSDaarbzYl7Ho3p2rFOmhhe/aSZ/+q/JCHzOy0LFlQUsmTW5ITcQ1GUxHFEbMYaYx4CHgJYvHhxTJ1Qjj7hNDihKZ7LGjvcu4BlU3tZdtlFCZl+874WLvz5q7R29SZkfiX5LF++nM9+9rOcffbZyV5K3Fi/fj379u3jwgsvjOq6/v5+Fi9eTEVFBc8++ywAn3zyCVdccQUNDQ0sWrSIxx57DLfbnYhlJ4RINmP3AlP9Xld6j0XCSK5VIqVsPhzYmLDp87OsPdDa1ZeweyjJ5Y477kiIyPf1Je8zs379etasWRP1dT/72c845phjBh279dZbufHGG6murqaoqIhHHnkkXsscFSKx6N8FZovIDKxIXwF8OcL5nwf+w28D9lzg+1GvUhme0vmw7Tno7YSM7LhPn5dpPyZt3Sr00XL7/9vE5n0tcZ1z7pQCbvvcvGHH1NTUcMEFF3DqqafyxhtvUFFRwTPPPMPWrVu57rrr6OjoYObMmTz66KMUFRXxta99jYsvvpjLLruMZcuWsXr1alwuF+eeey533303Bw8e5LrrrmP37t0A3HfffSxZsiTovVesWMGOHTvYuXMn06ZN4yc/+Qlf/epXaW9vB+CBBx7gM5/5DC+//DIrVqxg8uTJbNy4kUWLFvHb3/4WEWHNmjXcdNNN5ObmsmTJEnbu3Mmzzz5Le3s73/nOd9i4cSO9vb2sWLGCSy+9dMgaenp6WL58OZ2dnbz22mt8//vf50tf+lLYv9va2lr+/Oc/84Mf/IB77rkHsCGNa9eu5Xe/s7EmV199NStWrOD6668PO9+RQlihN8b0icgNWNFOBx41xmwSkTuAdcaY1SLyaeBpoAj4nIjcboyZZ4w5LCL/jv2yALjD2ZhV4kjZAjAeqN8CFSfEffpcR+jVoh9TbN++nd///vc8/PDDfPGLX+Spp57ipz/9Kffffz+nnXYay5cv5/bbb+e+++7zXdPQ0MDTTz/Nxx9/jIjQ1NQEwHe/+11uvPFGTj31VHbv3s15553Hli1bQt578+bNvPbaa2RnZ9PR0cELL7xAVlYW27dvZ+nSpb7ChR988AGbNm1iypQpLFmyhNdff53Fixdz7bXX8sorrzBjxgyWLl3qm/fHP/4xZ555Jo8++ihNTU2ceOKJnH322eTm5g66v9vt5o477mDdunU88MADAPztb3/jxhtvHLLWnJwc3njjDQC+973v8dOf/pTW1tZBfycTJkzA5bL/DyorK9m7d2w5JiLy0Rtj1gBrAo4t93v+LtYtE+zaR4HxuUs6WpTNt491GxMi9G5XGm5XGm09KvTREs7yTiQzZszguOOOA2DRokXs2LGDpqYmTjvtNMBappdffvmgawoLC8nKyuKaa67h4osv5uKLLwbgxRdfZPNmX6AdLS0ttLW1kZeXF/Tel1xyCdnZ9tdlb28vN9xwA+vXryc9PZ1t27b5xp144olUVlrpOO6446ipqSEvL4+jjjrKlxC0dOlSHnroIQD++te/snr1au6++27A5izs3r17iKslGGeccQbr168Pef7ZZ5+lpKSERYsW8fLLL4edbyxxRGzGKiNkQhW48xLrp890qUU/xsjMzPQ9T09P91nnw+FyuXjnnXd46aWXePLJJ3nggQdYu3YtHo+Ht956i6ysrIju7W9h33vvvZSWlrJhwwY8Hs+gOQLXGM6nb4zhqaeeYs6cORGtw59wFv3rr7/O6tWrWbNmDV1dXbS0tHDllVfy2GOP0dTURF9fHy6Xi9raWioqhgQPHtFoUbNUIC0NSufBgY8Sdou8LJf66Mc4hYWFFBUV8eqrrwLw2GOP+ax7h7a2Npqbm7nwwgu599572bBhAwDnnnsu998/kKcynGUcSHNzM+Xl5aSlpfHYY4/R398/7Pg5c+awc+dOampqAHjiiSd858477zzuv/9+jDfm94MPPgg5T35+/iAXjGPRB/5x3DY/+clPqK2tpaamhscff5wzzzzTt2dwxhln8OSTTwKwatWqoPsCRzIq9KlC6Xyo2zQQ9B5nct1q0acCq1at4pZbbmHhwoWsX7+e5cuXDzrf2trKxRdfzMKFCzn11FN9G5I///nPWbduHQsXLmTu3Ln88pe/jPie//zP/8yqVas49thj+fjjj4f40wPJzs7mF7/4Beeffz6LFi0iPz+fwsJCAH74wx/S29vLwoULmTdvHj/84Q9DznPGGWewefNmjjvuuEFfFrFw1113cc899zBr1iwaGhq45pprRjTfaCMmQcIQK4sXLzbaYSoG1j0Kz94I3/0QiqbHffov/tebCPDEtafEfe5UY8uWLRH5jJXQOP5/Ywzf/va3mT17dlC3y3gl2GdMRN4zxiwONl4t+lShbKF9rEuMnz4/U103yujx8MMPc9xxxzFv3jyam5u59tprk72kMY1uxqYKJccAYjdkPxX/DNm8LBdtB1XolQF+9atf8bOf/WzQsSVLlvDggw+OeO4bb7wxYgv++eef59Zbbx10bMaMGTz99NMjXkeqoEKfKrhzYdJMOPBhQqbPzXTRrha94sfXv/51vv71ryd7GZx33nmcd955yV7GEY26blKJ0vkJdd1oCQRFGZuo0KcSZfOhsQa64ptyD7YMQnefh56+wErUiqIc6ajQpxKlC+xj/ebhx8VAnrewmbpvFGXsoUKfSjilEBKQOJWrhc0UZcyiQp9KFFRAdlFC/PT5KvQpzfLly3nxxReTvYy4Em2Z4pqaGrKzs331gQC+8Y1vUFJSwvz58weNPXz4MOeccw6zZ8/mnHPOobGxcdi5d+3axQknnOALGfVPOHvvvfdYsGABs2bN4l/+5V98Wb+33HILZWVlvro+I0GFPpUQsRuyCbDoHdeNCn1qovXoLTNnzhxU3uFrX/sazz333JBxd955J2eddRbbt2/nrLPO4s477xx23vLyct58803Wr1/P22+/zZ133sm+ffsAuP7663n44YfZvn0727dv991v5cqVXHfddVGtPxQaXplqlC2Adb8CTz+kxa+/a56WKo6NvyyL/xdv2QK4ILSwhKpFn52dzfr167UefRR89rOf9dXc8eeZZ57xVbi8+uqrOf3007nrrrtCzuPfjaq7uxuPxwY17N+/n5aWFk4++WQArrrqKv70pz9xwQUXRL3W4VChTzVK50NfJxzeCZNnx21abT4ytghWi/7KK6/kqquu0nr0AfjXo4+Uuro6ysvLASgrK6Ouri7sNXv27OGiiy6iurqalStXMmXKFNatW+cr0wyJq3WvQp9q+G/IxlPo1XUTG8NY3okksBZ9TU0Nzc3NWo8+iqqbkSIiiEjYcVOnTuXDDz9k3759fP7zn+eyyy6L+1pCoUKfahR/CtJcdkN2/hfiNq26bsYWgXXeOzs7I7puPNejj4bS0lL2799PeXk5+/fvp6SkJOJrp0yZwvz583n11VdZsmQJtbW1vnOJqnWvm7GphisTJs+JexOSXLda9GMdrUc/fD36aLjkkktYtWoVMLg+/TvvvMNVV101ZHxtba3vy7axsZHXXnuNOXPmUF5eTkFBAW+99RbGGH7zm98kpNa9Cn0qUhb/yJu0NCHXna5CP8bRevTRsXTpUk455RS2bt1KZWUljzzyCADLli3jhRdeYPbs2bz44ossW7YMgN27d/tcVv5s2bKFk046iWOPPZbTTjuNm2++mQULbILjL37xC775zW8ya9YsZs6cGfeNWMD+FDqS/ixatMgoI+S1nxlzW4Ex7Q1xnfbEH79g/u3/bojrnKnI5s2bk72EMU9ra6sxxhiPx2Ouv/56c8899yT8np988omZN2/eiOa4+eabzYYN8fs/ctttt5mVK1cOOR7sMwasMyF0VS36VCRBGbJ5WpNeGSWSUY8+PT2d5ubmQQlT0bJy5UoWLlwYl/Xccsst/Pa3vw37CygSdDM2FXFq3tRthKNOG35sFKjQK/6kWj36qVOnsmfPnqiuSSQrV65k5cqVcZlLhT4VySuGvNK4b8hqg/DIMcZEFHI3ltF69MnBxND+VV03qUrZgsS4bjS8MixZWVk0NDTE9B9SUYbDGENDQ0PEYa4OatGnKqXzYeffoa8HXO7w4yMgLzNDLfoIqKyspLa2loMHDyZ7KUoKkpWVNSibNhJU6FOVsgXg6YVD2wY2Z0dIXqaGV0ZCRkaGL6tTUY4E1HWTqpR6xT2OJYsdH726JBRlbKFCn6pMmgXpmVDzWtymzMvMoN9j6OrVdoKKMpZQoU9V0l1w3FL44DF4+7/iMqUWNlOUsYn66FOZC++GtoPwl38Ddy4cf+WIpsvLtPXt27r7KM7PDDNaUZQjBbXoU5n0DLj8V3DUGbD6O7ApugSSQPIyMwCtYKkoYw0V+lTHlQlX/B+YehI89U3Y9teYp3JKFbd298ZrdYqijAIq9OMBdy58+QkbifOHr8Inr8Y0Tb7XR9/ePXyZWUVRjixU6McLWYVw5R+hqAp+fwXUrot6ilxfO0G16BVlLKFCP57InQRf/RNkT4Q/3xT15SndZaqzMdkrUJSEoUI/3igoh9lnQ1P0Vfoc101rqoVXVr8EK2dDc/ybMvvo79MvEyVpqNCPR3JLoPOwFZ8oyHSl4UoT2lNN6A986C0XsTVx91j3KNy/CDyabKaMPhEJvYicLyJbRaRaRJYFOZ8pIk94z78tIlXe4xkiskpEPhKRLSLy/TivX4mF3Mn2saMhqstEhNxUrGDZuMs+NtcOP24kNFTbv+/ejsTdQ1FCEFboRSQdeBC4AJgLLBWRuQHDrgEajTGzgHuBu7zHLwcyjTELgEXAtc6XgJJE8rwd69vro78005V6rpum3d7HBDadcNw2Pe2Ju4eihCASi/5EoNoYs9MY0wM8DgS2Kb8UWOV9/iRwltiuCwbIFREXkA30AC1xWbkSO7nF9rE9+jK6+Vmu1HPdOEKfSIveJ/RtibuHooQgEqGvAPxNnVrvsaBjjDF9QDMwCSv67cB+YDdwtzHm8AjXrIyUXK9F3xa90OemWjtBY6DZ+/FuVoteSU0SvRl7ItAPTAFmAP8qIkcFDhKRb4nIOhFZp80aRgHHRx+DRZ9yXaba6qGvyz4fDYteffRKEohE6PcCU/1eV3qPBR3jddMUAg3Al4HnjDG9xph64HVgceANjDEPGWMWG2MWFxcXR/8ulOjIKoR0d2xCn5ViPvom70ZsyVxo2Zu4qJhO7w9Zdd0oSSASoX8XmC0iM0TEDVwBrA4Ysxq42vv8MmCtsd0pdgNnAohILnAy8HE8Fq6MABHrp4/FR5+ZYj56xz8/fQn098S0QR0Wjwc6m+xzdd0oSSCs0Ht97jcAzwNbgD8YYzaJyB0icol32CPAJBGpBm4CnBDMB4E8EdmE/cL4lTHmw3i/CSUGcifHJPQpF17pWPTTT7GPiXDfdDdj4xJQoVeSQkT16I0xa4A1AceW+z3vwoZSBl7XFuy4cgSQW2L901GSl+mivaeffo8hPU0SsLBRpmk35EyGyXPs6+Y9UDnEuzgy/DNiVeiVJKCZseOV3GJoPxT1Zb4Klj0pYtU37Yai6VBY6X2dgMibQUKvPnpl9FGhH6/kFVt/dJSNvp3CZinjp2/cBROmQfYEyCxIjOumw1/oNepGGX1U6McrucV287E7uvy13FSqYOnxWFfNhGn2dWFlYoReXTdKklGhH6/4smOjc9/kpVIFy7Y6+2U3SOgT6LpJy1DXjZIUVOjHK47QR7khm59KrhsntHLCdPuYaKEvKFeLXkkKKvTjlRjr3TgWfUq4bnxC71j0U60od8fZ6u5stP7/rEIVeiUpqNCPV2KsYJnrTiHXTVONffQXerAZsvGk87Dd7M3IVdeNkhRU6McrOZPsY5Q++vxUs+hzSyAj2752Qizj7b7pbLTtG925WutGSQoq9OOV9AwrPlH66HNTzUfvWPMAE7wWfbxj6TsbIbvICr26bpQkoEI/nomh3k1GehqZrrTUKFUcKPR5ZSDp8Q+x9Al9ngq9khRU6MczI8iOHfM+eo/HWu7+Qp/ugoIpCRR69dEryUGFfjzjZMdGe1kqFDZr3W8bghdNH3w83iGWHo+6bpSko0I/nomxVHFeKrQTDAytdCicGl+h724B44Ec72Zsfw/098ZvfkWJABX68UxuCXQ1Q193dJe5U8B1E5gs5VBYCS37wNMfn/s4yVKORQ9q1Sujjgr9eMbXUjD6EMsx77pxhN4JqXQorARPny2PEA9U6JUjABX68UyMSVO2Jv1YF/pdkFc6EEPvUBjnEMtBQp9nn6vQK6OMCv14ZgSFzeJu0fe0w5b/B22j1By+addQ/zwMxNLHy08f1KLXyBtldImow5SSosRY7yY3MwE++g2/hz//KyAw9ST41EX2z6SZ8b2PQ9NuqAjSSaqgwj7GK8TSJ/QTwe11B6lFr4wyatGPZ0ZQwbKnz0NPnyd+azn8Cbiy4PRltkzACz+E+0+AB0+CtT+Krzh6+q2QB7Pos7zFx+Iu9BNsrRtQoVdGHRX68Yw7F1zZ0VewTEQZBCdL9fRlcN2r8L2P4Py7bE2eV1bCtufid6/W/XbDNZjQQ3xDLDsbwZ1vS044rpteFXpldFGhH8+IeJOmoi1VnAEQ3zIITbsHNkLBivDJ18HS39vX8cxWDRVD71A4Nb4WfXaRfa5RN0qSUKEf78SQNJWXmQ5Aazw3ZJv3hHClFFqLuDmOpYMbd9nHwBh6h3hmx3Z4SxSDCr2SNFToxzu5JTEIfZwt+p526GgYiHgJpLAivjXifRZ9qPtV2kSyruj66Qals9FmxYJfeKVG3Sijiwr9eCd3ctQhjU6Xqbj56J2Y9VAWdkFF/F03+eXgygx+3leXPg739HfduNzevrFq0Sujiwr9eCevBDoO2eJbkV6SGecuU74s1dGy6EPE0Ds45+It9ADuHOjR5iPK6KJCP97JLbYRKF1NEV/iCH3ckqaaw2yOFlRa91KUNXlCEliHPpB4dZoyJojQa016ZfRRoR/vxJA0FX/XzW5Id9uSBMEo9CYxxcOq7+8LHUPvkFcKaa6RW/TdLWD6A4Rea9Iro48K/XgnhqSpnIx0ROLputljrei0EB9HX7ZqHIS+dZ8V3+GEPi3duy8wQovePyvWQWvSK0lAhX68E4NFn5Ym5LrjWO8mMIY+EMeVEg+LPlR54iH3jEMsvX+dGwd13ShJQIV+vOOrYBllYbNMF23dcWqgESqG3iGe9WfCJUs5FFYmSOjVdaOMPir0453sIpC06EsVZ7lo745Dc47eLlv7fTjhdefYdcbNopehdegDcRqQ9I/gV0swoc/IsbV8FGUUUaEf76SlQ87kmOrdxMVH71jN4Szsgsr4+Ogbdw0fQ+8wYar15bfuj/1eHYft4xCLXl03yuiiQq9YP320SVOZLtq64uC6afKWIxjORw/xi6UPF1rpu18ckqY6m+yj+uiVJKNCr9js2Bgs+riUQHAiW8Ja9FFkx3Y22RIGwWjaDUVhNmJh4ItnRELfaIXd5R445vjojYl9XkWJEhV6xW7IJstH37QHJN26U4ajsMImdXVHsJH5f78G9y2Ezc8MPt7fZ38VRGXRjyDEMjBZCqzQGw/0dcU+r6JEiQq94q1gGX3UTWtcXDe7rYinh2l25ljY4dw3xsC+963V/IerYPV3BlwlLXvDx9A7uHNt/PuIhX5CwLzaN1YZfbSVoGKFvqfN1mBx50R0ieO6McYgIrHfu3kPFEYgvP4hlsVzQo9rP2jdNuf8O3Qehtfug11vwD/9N3S32jGRCD2MPMSy83AQi97799vTbl1mijIKqEWvxFwGwWOgq3eE7QQj3hyNsAzCoW32sXQenL0Crl5tv8D++xx45W57LmKhH2HSVGfj4KxY0Jr0SlKISOhF5HwR2Soi1SKyLMj5TBF5wnv+bRGp8ju3UETeFJFNIvKRiGTFcf1KPIghaWqgguUI3Dd9PTZ8MVRdeH/ypwASPsTSEXrH6p/xWbj+dZhzPnzydztHQZgYeocRW/TBfPTqulFGn7CuGxFJBx4EzgFqgXdFZLUxZrPfsGuARmPMLBG5ArgL+JKIuIDfAl81xmwQkUlAnNIplbjhuBCi2JDNzxqoYFmSH+N9W/bajclILGyX234htYQR3oPbbBPu/CkDx3Imwhcfg/W/s8lZ/lEwwzFhqi1M1tk01NcejmCVK8HPotfsWGX0iMSiPxGoNsbsNMb0AI8DlwaMuRRY5X3+JHCWWMftucCHxpgNAMaYBmNMHEI1lLgSg+sm1+0V+pGEWDobneFi6B0KKiKz6CfPGlogTQSO/wr8w02Rr28ksfQ9bbb8c0ihD2PR93bBgydD9YvR31tRAohE6CsA/9CDWu+xoGOMMX1AMzAJOBowIvK8iLwvIv8W7AYi8i0RWSci6w4ejC6eW4kDMVSwdEoVj0joI6074xBJ0tSh7TD56NjXNOh+I4ild7JicwJ99BG6blr3w8Et8Mmr0d9bUQJI9GasCzgV+Ir38R9F5KzAQcaYh4wxi40xi4uLixO8JGUIGdm2AXcMPvoRVbBs2oP1mQfaDSFwyiCESjbq6bBNTCYPE5UTDY7QO9m70RCszg3YWjcAvWGEvtP7RdH4SfT3VpQAIhH6vYD/b+tK77GgY7x++UKgAWv9v2KMOWSM6QDWACeMdNFKAsgrji7qJjNOFn1+eeQ+88IKK5ChumE1bLePk2fHviZ/8krAlW3r40RLKKGP1HXT4b2+sSb6eytKAJEI/bvAbBGZISJu4ApgdcCY1cDV3ueXAWuNMQZ4HlggIjneL4DTgM0oRx65xVFtxsbFdROuPHEg4RqQHHKEPk6uGxFbLiGeFn2kQu+z6Guiv7eiBBBW6L0+9xuwor0F+IMxZpOI3CEil3iHPQJMEpFq4CZgmffaRuAe7JfFeuB9Y8yf4/4ulJETZXZsfCz6XZGFVjqEa0ByaJstuTzxqNjXFEhRVWxiG0ro09Ltr4RwUTeOj7+reWAuRYmRiDJjjTFrsG4X/2PL/Z53AZeHuPa32BBL5Ugmtxh2vxXx8ExXGhnpEtpHv/c9yCwI7Ubp77P13mOy6ENsjh7aZjtHZcQxVaOoCmpet/sC0WQAdwYpUewQSalif3FvrAk+j6JEiGbGKpa8EuhoAE9k0a8iQm6oCpYeD/zuCnjmhtATtO634YeRhlYC5JfZAmihLPqD24YvjxALE6ZDT+uAhR0pnU02nj9Y3fuIhN7vfod1Q1YZGSr0iiW3GDBRCVrIUsX7P7D+/j1vh3YHRVqe2J80b5XLYD56Tz80VMdvI9ahqMo+NtVEd12wZCmHSIS+4zDkejOW1U+vjBAVesUSQ3asbT4SROi3/dX7xMD2vw49D9HH0DuEiqVv2g393fHbiHVwatdHK7YjFfrOw/bvJmeyCr0yYlToFYtjPUYRYpmfFcKi3/48VH7aWt9b1ww9D94YesL3bg0kVAOSeEfcOEwYidBPCH4uUos+Z2Lsm8GK4ocKvWLxZcdGUQYhmOumrR72fQBHnwdHnw87/gZ93UMvbtplv1wysqNbZ2GF3cQNTJo6tNU+xlvoM/O8VnWUIZaOUAcjknaCnYdt5UsVeiUOqNArlrwYShUHE/rtL9jH2efBnAttGGFNkDT+aGPoHQoqrYsm0Pd/aJsV5FDiOhJiEduwrptw4ZXe64uq7C+Yfq0FqMSOCr1iyZoAaRlRV7Ac4qPf/rx12ZQtsCWCM3Jg61+GXty0O7oYegdfXfoA9008a9wEUlQVXdJUqMqVDuFcN309NtInZyJMnGG7Yo2k05Uy7lGhVywi3qSpEVj0/b3WVTP7HDtfRhYcdQZsfW6wq8XjsVZqTBZ9iOzYQ9viH3HjUDTd7in0R5gc1tMOnt7QQp+RA70doa/3T7Zyon7UfaOMABV6ZYDcyVFlx+Zmuujo6aff4xXx3W/a+u2zzxsYNOcCa30f+GjgWFsd9PdEF0PvEKx3bHuDzQGIdwy9Q1GVtarDVc50CJUV6+DOs0IfKmfBud7ZjAUVemVEqNArA+QWR1eq2FsGob3Ha+luex7S3XDU6QODjj4PENj23MAxXwz99BjWOBnSMwe7MpyuUoly3UQbeePLig21GeutdxPKqve/Pr/c/p2q0CsjQIVeGSCvJCqL3r/LFGA3YqcvsZEq/nNWLh7sp/fF0Mdg0YtAwZTBrhuf0CfKdVNlHyP104e16MMUNvOvZZ+Wbr9oNDtWGQEq9MoAuZPtZmyoeu8B5GVmAN7CZo01NsRx9rlDBx59Pux7H1r229eO0MfiugEbe98SIPSurNjnC0dBhS29ELFFH4HrBkILfeAvAg2xVEaICr0yQFEV9HXBlsAq1MHJzUwHvELvZMMefd7QgXMusI/bn7ePTbutiPlb/tEQ2FLw0HaYNMtav4kg3WV/fcRN6KOw6GFA6CP8AlaUQFTolQGO/ypULIanr4f6LWGHD3LdbH8eJs6ESTOHDiyZayNsHPdNrDH0DoUV3qJo3s3MQ1sT5593mDA98qSpsELv7TI1nEWf7h7oRlVU5W1SruWKldhQoVcGcGXClx6zFufjX7EVGIfBcd10trfY3qbBrHmwfvWjL4CdL9t2f7HG0DsUVNgomNYDtol2467EC3007pOOw1akQ5VLDue66fBmxTplkTXyRhkhKvTKYAqmwBd/Yzcen77WxryHwHHd5Ox9w2arBvPPO8y5wLqFdr5sY9Jjibhx8G9AcngHYBK3EetQVAUdh6A7TEYr2C/I4erH+1w3IebqbByc4Ttxhn3U/rFKjETUeEQZZ0w/Bc6/E9bcDH+/C874ftBh+V6LvunDZ+mSLL77ipu+V98NOjbdZHKf5LD1mfs5vq+T3201vLQ/+NhwVPY0cTvwn6v/DgjXAyve6GXPe7HNFwmfbu/jOmD5qj+z1z18B6sb6nYyuS+TFb8Ovp7Jvfu5C3h07UZef3doY/Rb9+/Cg4t7H1vHzefOYXashdUUxYsKvRKcT3/TFif7+51Qfix86sIhQ/KzXFw4v5STdr7HB67j2NvmAbpCTrkuYxFLOl8HoLp3InWtoccOR5unAAB3+z4yTTcehI+6J9PdE9t8kbC1bxIArtY91LmnDDs2o7eZRnJDvr9Oj/0h3dPZQl2Qv6/MvmZq0yt5flMdCysnMPuMWTbHQYVeiREVeiU4InDRPVC/Gf74Lfgfa6F4sB88LU34xTnZ8J8HKbnohzy76B+Gn3PDPnjaFjhb/pXzbD2cWDAG/iOXaxZkQnsL1E7lqX85J7a5IqV9Lqy8ieWfyYFTwrzPBz0w+Sie/VKIcb2d8GO47jPlXHdqkDF3dzP96NlMWJ/B/uZOe0xDLJURoD56JTQZWfDFx8Dlhse/DBv/aDdS/cP8tnlDJofzzzvMPsc274aRxbyLeMsV13pr3CR4Ixasz9ydH5nYOiWGQ+HKAiT4ZqwxvhLHZQVZHGj2lngumqFCr8SMWvTK8EyYCpevskL/5NftsVxvtmvFItj0J2uZF5SHnytnIkw7BQ5sDN2UI1KcBiQN1VB16sjmigQRb3GzMCGW4SpXOnOFqknf0+YtiDaRssIsDrT4WfQbn7SVLV3umN+GMj5RoVfCM+Mf4JYdULcR9r4Htetg77qB7lGn3Rr5XGffDod3jnxNhRVQ85oVxdGw6MGKbUP18GN6O2zBtuGEHkLXpHeSpbKLKC/MYuPeloF7G4/NQQiWq6Aow6BCr0SGyw0VJ9g/J/4Pe6yzEeo2w5TjI59n6qftn5FSUGlFHkZX6Ktfsla7E+MeSLhkKYdQNek7B7JiSwuyONTWTU+fB7d/LL0KvRIl6qNXYie7CKqWDGR6jiaFfmGJoyX0E6ZDX+fwFT5HKvQdA3VuygttwlV9a5dfLH1NdGtWFFTolbGK04Aka4ItxjYaRFLFMrBOTShC+ej9atGXFlihP9DcBXlltjyzCr0SAyr0ytjEyY4tnhPajRJviiJIXIrYos+JwKK3jdMPtHRBWpq9v2bHKjGgQq+MTRyLPtGlD/xxCrENV9wsXj767CLK/C160Fh6JWZU6JWxSWYenPxtOHbp6N0zI9u6UOJi0Q/juskshHQXBdkusjLSAoR+l5YrVqJGo26Uscv5/zH69wxnVXc22oSojOzh5xkuvDLHfkmICOWF2dZ1AzZpyilXHG4PQFH8UIteUaKhqGr4zdhwWbEOw7lu/H4NlBZkDrboQf30StSo0CtKNBRNtxm5fT3Bz7cdjCzr151r8wAC5+kY/EUx2KKvso/aP1aJEhV6RYmGoirA2AzVQBp2QPWLkZVkyPDWpO8NsOo7Dw9yy5QWZFHX0oXHYyKL+lGUIKjQK0o0DFcbfu2/2y5d/3Bz+HlC9Y3taAyw6LPo7Tcc7uix1+SVqtArUaNCryjRECppau/7sOlpOOUGyC8NP08woe/vg+7mIRY9aIilMjJU6BUlGvLLbePuQLF9cQXkTILPfCeyeXx9Y/0ib3yhmYMteggU+giblCuKFxV6RYmGtDSbOOUv9DvWwid/ty6brILI5glm0XcOLZ9Q5hX6/f4bsi3DbAYrY5ZdDe00tifm31WFXlGiZcL0Aava47HWfOE0+PQ1kc/hE/qOgWNBkq0m52WSnibUNfvF0jvlipWU4hu/fpcf/OmjhMwdkdCLyPkislVEqkVkWZDzmSLyhPf82yJSFXB+moi0iUgEu1SKcoTj7yff9EfYvwHO/IHdiI0Un9D7uW78atE7pKcJJfmZ7NdY+pSnvrWbkvyshMwdVuhFJB14ELgAmAssFZG5AcOuARqNMbOAe4G7As7fA/xl5MtVlCOAoiroaoL2Q7D2R1AyDxZcHt0cEbpuYCDE0ndv0A3ZFKOzp5/Wrj5KCqIwFqIgEov+RKDaGLPTGNMDPA5cGjDmUmCV9/mTwFkitqSgiHwe+ATYFJcVK0qyceLZX7rdWtZn3wZp6dHNEUzo/SpX+lNemDXQJDyv1JZYUKFPKepb7Rd50ix6oALwdwjWeo8FHWOM6QOagUkikgfcCtw+3A1E5Fsisk5E1h08eDDStStKcnCs6vd/A9M+E1lj9EAyQlj0aS7IzB801Fr03ibhaWkw8Sg4tD36eypHLM6/b2kSLfqRsAK41xgTpHrTAMaYh4wxi40xi4uLixO8JEUZIU7SFMA5t8dWD9/ltmGagT767IlD5isvzKKtu4/WLm/rxJK5toWjcsSyZX8Ln/nJS+w53BF+MEeGRb8XmOr3utJ7LOgYEXEBhUADcBLwUxGpAb4H/E8RuWFkS1aUJJM9AfKnwDGfg6knxj6PO9c2E3cIKH/g4IRY+vz0pfOgeTd0Ncd+byWhPPHuHvY1d7FpX2T/Rom26CMpU/wuMFtEZmAF/QrgywFjVgNXA28ClwFrjTEG+AdngIisANqMMQ/EYd2Kkly++WL4mvPhyAioYBlQ/sBhoAFJN7NK8qF0vj1RtxmmnzKyNShxp99jWPPRfsAv0S0M9a1duF1pFGZnJGRNYS16r8/9BuB5YAvwB2PMJhG5Q0Qu8Q57BOuTrwZuAoaEYCpKSlFYMfKm6IE16cNY9L4N2VJv0Fu9xjccibzzyWHqW62F7kt0C0N9Szcl+ZlIgtpiRtR4xBizBlgTcGy53/MuYNj4MmPMihjWpyipS2BN+s7GoCWOnXo3PtdNQQVkFUKdCv2RyLMf7iM7I528LFfEFn1dSxcl+Ylx24BmxipK8vAXemOG1KJ3yMpIpygnYyBpSsS6b1ToR41n1u/lv/6+I+y4vn4Pz208wFnHlDBjUu7Av1kY6lu7fV/oiUCFXlGShX/f2N4O6O8O2SJwUNIU2A3Zus3aP3YYtte18rn7X4tL/ZhHX/uEu577mE8OBekK5sebOxtoaO/h4oVTKCvMUoteUcY9/hZ9iGQpB5s0FSD0Pa3QtDvBixy7vF59iI/2NvPBnsYRzePxGLbVteEx8MuXh7fqn92wn7xMF6fPKbZC39KFCfNlPJAVqxa9oqQe7pwBoQ9R/sChrDDAoi+ZZx/VfROSmgYbulpdP2waT1j2NnXS2dvP5LxM/vhBLfuaOoOO6+nz8JeN+zl3bilZGemUFWTR0+ehsaN32PkHYujVoleU1MPfdRPGoi8ryOZQWw/dff32QMkx9lGFPiQ1DfbvdqRCv/VAKwDLPzcXY+ChV3YGHfda9UFauvq4+NhyYKCXgC9aKgROhI766BUlFXHCK42JwKK31l69UwohM8+WLK7bOBorHZPsipNFv7XOCv0Zc4r5/PEVPP7ubg61dQ8Z9+yG/RRmZ3DqLJvdXxbYNCYEzi+1RBU0AxV6RUke7lzAQG9neIu+MBuAA0M2ZNWiD0Zfv8dXfqC6vi2sn3w4ttW1UjEhm/ysDK4/fSbdfR4efW1wmeiu3n7+urmO8+eV4XZZWS33/puFi7xxvrxLE1T+AFToFSV5OO0EezuCNh3xpyywdyzYEMvDO+wXhTKIfU1d9HkMx5QX0NLVx8EgFnikbKtrY3ap/beaWZzHhfPLeezNXTR3DvjeX956kLbuAbcNwOQ8N2kSgUXf2oU7PY0JOYnJigUVekVJHv7NRzobrfC73EGHDql3AzZD1njg4MeJXumYw/HPn31MCQDVdbG5b/r6Peyob2NO6UBF0X8+Yyat3X089maN79izH+5jUq6bU46a5DvmSk+jJD9r8K+wIBxs6aY4gVmxoEKvKMkjw1tCoac9ZLKUQ0GWi+yM9IAQS6fmjbpvAtnlFfqzjikFoPpgbEJf09BBT7+Ho/2Eft6UQs78VAmPvPYJHT19dPT08dKWes6fX4YrfbCkRhJLX9falVD/PKjQK0rycFw3Pe3eOjehi6SJiC8u20dRlf2yUKEfQk1DB9kZ6SysKCQv0xXzhuw270bsnLLBPQK+fcZMGjt6+f07e1j7cT2dvf187tgpQ64f1DQmBPUt3Qn1z4MKvaIkD3/XTRiLHqyffpB1mJZuwyw18mYIuxramT4ph7Q0YWZx7oiEXsT65v1ZNH0iJx81kYde2cEf399LSX4mn64KXpBuf/PwSVN1LWrRK0rq4t9OMETlSn+CugGcyBsthTCImoYOqibZv9+ZJXkjEvrpE3PIdg9tFfntM2ZR19LN2o/ruXBBOelpQ33s5YVZdPT009rdF3T+rt5+Wrr6EhpDDyr0ipI8fELfEZlF782O9Xj8RL1kHnQ0QFt9Ahc6tuj3GHY3dDB9st0DmVWSR31rNy1dw2eoBmPrgdZB/nl/Tp01mWMrCwH4nF+0jT++yqMh/PROaGUis2JBhV5Rkocj9N0ttltUOIu+IIs+j6HBv0hXqVMKQd03Dgdauujp9/gs+tklVqijteq7evupaegY4p93EBFuu2QeV58yneOnBt9fCRdL7yt/oBa9oqQojtA31wImbMeqoJmWpVrzJpBd3gqT0ycNWPQQvdDvPNhOv8cwO4RFD3DCtCJuv3Q+aUHcNjBQBiFU5E2iWwg6qNArSrJwwiub99jHCDZjISA7Nmei7V+rQu/DKWbmWPRTi7Jxp6exI0qh317vjbgZRujD4WyyhrXoNepGUVKUtHQr9s219nUY182AdRgQrlc6T9sK+rGroR23K833xehKT2PG5Ogjb7YeaMWVJsyYnBvzWjJd6UzOc3OgJXiIZV1LNxnpQlECs2JBhV5Rkos7d0Dow1j0k/IySU+ToZmWpXPh4Fboj36zMRWpaWhn+sScQe6UWSV5USdNbatr5ajiXF/tmlgpC+wl4Ed9axcl+VkJzYoFFXpFSS7uXGjdb58PkzAFkJ4mlORnDhWN0vnQ3wMN1Qla5NhiV0MH0ycNtsJnluSx53AHXb39Ec+ztS50xE00DMl/8KO+pTvhMfSgQq8oySUj19argbAWPQRpQAK6IeuHMYaahnaqvBuxDrNK8vAYwrYCdOjo6WPP4c74CH1gRrMf1qJXoVeU1MaJvJF0yCoMO7ysIIgbYNJsSMtQocf6vLt6PUwP8KvPKo4u8ma7twhaPIS+vDCbpo5eOnuG/pqoa0lsU3AHFXpFSSaO0GcXQQR+2rLCrKHJNy43FM9RoWegamWgRX9UcS4ikQv91hA1bmIhaLQUNk6/ubNXLXpFSXn8hT4CygqyaO/ppzUwy7Nkrgo9A1UrqwJ89FkZ6Uwtyol4Q3bbgVYyXWlMm5gTfnAYQrUUPOhtIZjoZClQoVeU5OJUsAwTWukQsj1d6TxoqR1oYDJOqWnoICNdfOLqz6ySvIhj6bfVtzGrJC9o/ZpoCdpLgNFpCu6gQq8oycRn0Uco9CHcAAO16TfHa2Vjkl0N7UwtyhlSFx6s0O88ZLNdw7HtQOuIEqX8KfNZ9IP/zQayYtWiV5TUxu11DURo0YesneJE3tSPb6GvOdThK30QyKziPHr6BnrJhqK5o5cDLV0cHQf/PECO20VBlmvIr7D6FrXoFWV84LhuIvTROzHXQzZk88vsr4JxXNzMGOOtQx88k3WWt+/r9jDum21xKH0QSHlh9lCLvtXJig3ePjKeuBJ+B0VRQuO4biK06LMy0pmY62bVmzW8sKVu0Lkf9VUw8/0/sG/D6/Fe5RFDYXZGSAu4r9/wO1qoqM6Ch4aOOdZjeMbdQvlfsuC10Fb0Ue09POPu5Ji1+fBKfGzh/+xsp3+XgYcGGpgsPdzJJe5e0v77p343PwPOvi0u9/RHhV5RkkmUPnqAaz97FG/tbBhy/GX5J0zbmnit7Iijs6effS0eJpZMxhVkk7S9s5cGA5V5hZA7VMjTgdb0NDKMm5LcgpD3aWhrpUncZBQWx23tPa2ZHGrrYXbuZN+xw4eb6MswkOv3ay4r9LpGggq9oiSTKKNuAK49bSbXnjYzyJkTgRvjsqwjkXU1h7nil29yz7xj+cIJlUPOv/heLTdXb+DlL5zOpBCFyH7x8Fu09/TzzFeWhLzP8ofepDvLw2nDjImWv7ywjZ+v3c7WL13gq51z672vMH1SDou/sjhu9wmF+ugVJZlEGUc/nlk0vYhpE3P44/t7g57f1dBOeppQUZQdcg4nxDJUD1djDFvjGHHjUF6YhTEDIZUAda1doxJxAyr0ipJcpn8GPv1NqEi8VTfWERH+8fgKXt9xaEjyEdgY+sqibDKChFY6zCrJo627zxfaGMihth4aO3qHbTYSC4H5D919/TR1jE5WLKjQK0pyyS6Ci/73QJilMixfOKECY+BPH+wbcm64iBuHcDVvttfFP+IGhobF1o9iDD2o0CuKMoaYPimXxdOL+OP7tYPcL8YYPjk0tGplIANtBVuDnndq3Bxdlhf0fKwEWvSOC6d4FEoUgwq9oihjjH88oYLt9W1s2tfiO9bU0UtrV19Yi744P5P8LFfImjfb6lopysmgOC++AlyQ5SLHne7LaPZZ9AluIeigQq8oypji4gVTcKen8dT7tb5joapWBiIitttUENeNx2PYvK+F2aX5ce/4JCKDGpA4dW9Go+kIRCj0InK+iGwVkWoRWRbkfKaIPOE9/7aIVHmPnyMi74nIR97HM+O8fkVRxhmFORmcPbeE1ev30dtvm7bs8jYED2fRg/XTV9cPNCAxxrD24zo+98BrbKht5uSjJiVk3baloN1Erm/txpUmTByFrFiIQOhFJB14ELgAmAssFZG5AcOuARqNMbOAe4G7vMcPAZ8zxiwArgYei9fCFUUZv3zh+Eoa2nt4dftBwFr0IjB1YujQSodZJXkcauumuaOXN3c0cNkv3+Qbv15Ha1cf937pWL571uyErLms0N+i76Y4P3NQX9tEEknC1IlAtTFmJ4CIPA5cCvhXT7oUWOF9/iTwgIiIMeYDvzGbgGwRyTTGBI9tUhRFiYDT5hQzMdfNU+/v5cxPlVJzqJ0phdlkutLDXutsyH7poTf5+EArZQVZ/Mc/LuDyxZXDhmaOlPLCLOpau+n3GNtCcJQibiAyoa8A9vi9rgVOCjXGGNMnIs3AJKxF7/BPwPsq8oqijJSM9DQuOXYKv3tnN82dvdQ0dFA1ObIQ1U+VFyBiG3/8r4uO4cqTp5OVEf4LYqSUFWbT7zEcauumvqWbaWH2E+LJqJRAEJF5WHfOuSHOfwv4FsC0adNGY0mKooxxvnBCBb9+o4Y1H+1nV0M7Fywoj+i6ignZPPudU6malEtu5uhVgSkvGAixrG/t4tMzRi8bOpLfKXuBqX6vK73Hgo4RERdQCDR4X1cCTwNXGWN2BLuBMeYhY8xiY8zi4uL4FRJSFCV1WVBRyMziXFa9UUNjR2/YiBt/5k0pHFWRh4FY+l2HO2js6B210EqITOjfBWaLyAwRcQNXAKsDxqzGbrYCXAasNcYYEZkA/BlYZoxJ3dqpiqKMOiLCF06o5OMDNskpkoibZOII/Ue1TcDohVZCBEJvjOkDbgCeB7YAfzDGbBKRO0TkEu+wR4BJIlIN3AQ4IZg3ALOA5SKy3vunJO7vQlGUccnnj6/ACXkPbAh+pDExx407PY0Ne5qB0WkK7hDRbxdjzBpgTcCx5X7Pu4DLg1z3I+BHI1yjoihKUComZHPKUZN4Y0cD0yYe2fWC0tKE0sJMNu7zCv0oFTQDrUevKMoY55bz5vDWzsNkuxMfOTNSyguy2XPYJk2NVkEzUKFXFGWMc/y0Io6fNjbq+Zd6/fSjmRULWutGURRl1Cj3Cv1oZsWCCr2iKMqoUeZ114ymfx5U6BVFUUYNx6IfzYgbUKFXFEUZNZxYerXoFUVRUhSnpWDJKGbFggq9oijKqFFakMmNZx/NpcdNGdX7aniloijKKCEifPfsxNS7Hw616BVFUVIcFXpFUZQUR4VeURQlxVGhVxRFSXFU6BVFUVIcFXpFUZQUR4VeURQlxVGhVxRFSXHEGJPsNQxCRA4Cu0YwxWTgUJyWM5bQ9z2+0Pc9vojkfU83xhQHO3HECf1IEZF1xpjFyV7HaKPve3yh73t8MdL3ra4bRVGUFEeFXlEUJcVJRaF/KNkLSBL6vscX+r7HFyN63ynno1cURVEGk4oWvaIoiuKHCr2iKEqKkzJCLyLni8hWEakWkWXJXk+iEJFHRaReRDb6HZsoIi+IyHbvY1Ey15gIRGSqiPxNRDaLyCYR+a73eEq/dxHJEpF3RGSD933f7j0+Q0Te9n7enxARd7LXmghEJF1EPhCRZ72vx8v7rhGRj0RkvYis8x6L+bOeEkIvIunAg8AFwFxgqYjMTe6qEsavgfMDji0DXjLGzAZe8r5ONfqAfzXGzAVOBr7t/TdO9ffeDZxpjDkWOA44X0ROBu4C7jXGzAIagWuSt8SE8l1gi9/r8fK+Ac4wxhznFz8f82c9JYQeOBGoNsbsNMb0AI8DlyZ5TQnBGPMKcDjg8KXAKu/zVcDnR3NNo4ExZr8x5n3v81bsf/4KUvy9G0ub92WG948BzgSe9B5PufcNICKVwEXAf3tfC+PgfQ9DzJ/1VBH6CmCP3+ta77HxQqkxZr/3+QGgNJmLSTQiUgUcD7zNOHjvXvfFeqAeeAHYATQZY/q8Q1L1834f8G+Ax/t6EuPjfYP9Mv+riLwnIt/yHov5s67NwVMMY4wRkZSNmRWRPOAp4HvGmBZr5FlS9b0bY/qB40RkAvA08KnkrijxiMjFQL0x5j0ROT3Jy0kGpxpj9opICfCCiHzsfzLaz3qqWPR7gal+ryu9x8YLdSJSDuB9rE/yehKCiGRgRf7/GGP+6D08Lt47gDGmCfgbcAowQUQcQy0VP+9LgEtEpAbrij0T+Bmp/74BMMbs9T7WY7/cT2QEn/VUEfp3gdneHXk3cAWwOslrGk1WA1d7n18NPJPEtSQEr3/2EWCLMeYev1Mp/d5FpNhrySMi2cA52P2JvwGXeYel3Ps2xnzfGFNpjKnC/n9ea4z5Cin+vgFEJFdE8p3nwLnARkbwWU+ZzFgRuRDr00sHHjXG/Di5K0oMIvJ74HRs2dI64DbgT8AfgGnYEs9fNMYEbtiOaUTkVOBV4CMGfLb/E+unT9n3LiILsRtv6VjD7A/GmDtE5CispTsR+AC40hjTnbyVJg6v6+ZmY8zF4+F9e9/j096XLuB3xpgfi8gkYvysp4zQK4qiKMFJFdeNoiiKEgIVekVRlBRHhV5RFCXFUaFXFEVJcVToFUVRUhwVekVRlBRHhV5RFCXF+f8HvVzrgKCERQAAAABJRU5ErkJggg==", + "text/plain": [ + "
    " + ] + }, + "metadata": { + "needs_background": "light" + }, + "output_type": "display_data" + } + ], + "source": [ + "#hide\n", + "scheduler = MultiStepLRScheduler(optimizer, decay_t=[15, 30], decay_rate=0.5, noise_range_t=40)\n", + "plot_noisy_lr(scheduler, label='noise_range_t=40')\n", + "\n", + "scheduler = MultiStepLRScheduler(optimizer, decay_t=[15, 30], decay_rate=0.5, noise_range_t=[10, 30])\n", + "plot_noisy_lr(scheduler, label='noise_range_t=[10, 30]')\n", + "plt.legend();" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### `noise_pct`" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + " Percentage of noise to add. " + ] + }, + { + "cell_type": "code", + "execution_count": 14, + "metadata": {}, + "outputs": [ + { + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXoAAAD4CAYAAADiry33AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/YYfK9AAAACXBIWXMAAAsTAAALEwEAmpwYAAA57ElEQVR4nO3deXyU5dX4/8+Z7PtkA7ISEBRBFjW4YoWqKG60FhTbKvporf3qy6X160P7tFR5Htvan193+7gv1bZqRQu1WKuirXVBVpFNWYSsQPZ9m8z1++OeyTpJZpJJJjM579eLV2buZeaaEE4uzn2uc4sxBqWUUqHLFugBKKWUGl4a6JVSKsRpoFdKqRCngV4ppUKcBnqllApx4YEeQE9paWkmLy8v0MNQSqmgsnnz5nJjTLqnfV4FehG5AHgICAOeNsb8psf+bwAPArOAZcaY11zb5wD/CyQC7cA9xphX+nuvvLw8Nm3a5M2wlFJKuYjIob72DZi6EZEw4DFgETAduFJEpvc4rAC4Bvhjj+2NwNXGmBnABcCDImL3euRKKaWGzJsZ/SnAPmPMAQAReRlYDOxyH2CMOeja5+x6ojHmqy6PS0TkKJAOVA914EoppbzjzcXYLKCwy/Mi1zafiMgpQCSw38O+G0Rkk4hsKisr8/WllVJK9WNELsaKSAbwIrDcGOPsud8Y8yTwJEB+fr72ZFBjXltbG0VFRTQ3Nwd6KGqUiY6OJjs7m4iICK/P8SbQFwM5XZ5nu7Z5RUQSgb8B/2WM+dTrkSk1hhUVFZGQkEBeXh4iEujhqFHCGENFRQVFRUVMmjTJ6/O8Sd1sBKaKyCQRiQSWAWu9eXHX8W8Av3dX4iilBtbc3ExqaqoGedWNiJCamurz//QGDPTGGAdwM/A2sBt41RizU0RWicilrjefKyJFwFLgCRHZ6Tr9cuAbwDUiss31Z45PI1RqjNIgrzwZzM+FVzl6Y8w6YF2PbSu7PN6IldLped5LwEs+j0oFRmsj7Hwd5nwPNMgoFTK0BYLqtOdNWHMTlG4L9EiUUn6kgV51qnVdY6/8OrDjUEFp5cqVvPvuuwF57+eff56SkhKvjzfGcMsttzBlyhRmzZrFli1bPB7X2trKDTfcwLHHHsu0adNYvXp1x/ulp6czZ84c5syZw9NPP+3zeG+++eZ+j2lpaeHcc89lzpw5vPJKvw0FBjTqet2oAKo7Yn2t7nMltVJ9WrVqVcDe+/nnn+eEE04gMzPTq+Pfeust9u7dy969e9mwYQM/+tGP2LBhQ6/j7rnnHsaNG8dXX32F0+mksrKyY98VV1zBo48+6rfP0NPWrVsB2LZt25BfSwO96lRXan2tOhjQYaju7v7rTnaV1Pr1NadnJvLLS2b0uf/gwYMsWrSIefPm8fHHH5OVlcWaNWuIiYlh27Zt3HjjjTQ2NnLMMcfw7LPPkpyczDXXXMPFF1/MkiVLWLFiBWvXriU8PJyFCxdy3333UVZWxo033khBQQEADz74IGeeeabH97/rrrvYv38/+/bto7y8nDvvvJMf/OAHANx777289NJL2Gw2Fi1aRH5+Pps2beJ73/seMTExfPLJJ8TExPT7+desWcPVV1+NiHDaaadRXV1NaWkpGRkZ3Y579tln2bNnDwA2m420tDSvv8c9Pffcc/z617/Gbrcze/ZsoqKiADx+X6ZOncr3v/99ysrKmDNnDqtXr+aYY44Z9Htr6kZ1qnfN6Kt0Rq9g79693HTTTezcuRO73d6Rtrj66qu599572b59OzNnzuTuu+/udl5FRQVvvPEGO3fuZPv27fz85z8H4NZbb+X2229n48aNrF69muuvv77f99++fTvr16/nk08+YdWqVZSUlPDWW2+xZs0aNmzYwOeff86dd97JkiVLyM/P5w9/+APbtm0jJiaG22+/vSOt0vXPb35j9WMsLi4mJ6dzeVB2djbFxd2XB1VXVwPwi1/8gpNOOomlS5dy5MiRjv2rV69m1qxZLFmyhMLCQvpTWlrKL3/5Sz766CP+/e9/s2tXRwcZj9+XcePG8fTTT3PWWWexbdu2IQV50Bm96so9o9fUzajS38x7OE2aNIk5c+YAcPLJJ3Pw4EFqamqorq7m7LPPBmD58uUsXbq023lJSUlER0dz3XXXcfHFF3PxxRcD8O6773YLcLW1tdTX1xMfH+/x/RcvXkxMTAwxMTEsWLCAzz77jA8//JBrr72W2NhYAFJSUjye+8ADDwzpswM4HA6Kioo444wzuP/++7n//vu54447ePHFF7nkkku48soriYqK4oknnmD58uWsX7++z9fasGED8+fPJz3d6iJ8xRVX8NVXViuwvr4v/qSBXlmM6ZKjLwRnO9jCAjsmFVDu1AJAWFgYTU1NXp0XHh7OZ599xnvvvcdrr73Go48+yvr163E6nXz66adER0d79To968V9qR+//fbbef/993ttX7ZsGStWrCArK6vbLLyoqIisrO4tvFJTU4mNjeWyyy4DYOnSpTzzzDMd+9yuv/567rzzTq/H1pOv35fB0NSNsjTXgKMJUqeCsw1qva9gUGNHUlISycnJfPjhhwC8+OKLHbN7t/r6empqarjwwgt54IEH+PzzzwFYuHAhjzzySMdxA11kXLNmDc3NzVRUVPDBBx8wd+5czjvvPJ577jkaGxsBOi6OJiQkUFdX13HuAw88wLZt23r9WbFiBQCXXnopv//97zHG8Omnn5KUlNQrPy8iXHLJJXzwwQcAvPfee0yfbnVoLy0t7Thu7dq1HH/88R3Pp02b1uuznHrqqfzzn/+koqKCtrY2/vznP3fs8/X7Mhg6o1cWd34+91So2Gulb+w5/Z+jxqQXXnih42Ls5MmTee6557rtr6urY/HixTQ3N2OM4f777wfg4Ycf5qabbmLWrFk4HA6+8Y1v8Pjjj/f5PrNmzWLBggWUl5fzi1/8gszMTDIzM9m2bRv5+flERkZy4YUX8qtf/YprrrmGG2+80euLsRdeeCHr1q1jypQpxMbGdvsMc+bM6Qi29957L1dddRW33XYb6enpHcc9/PDDHRebU1JSeP755wEoLy/HmN59GTMyMrjrrrs4/fTTsdvtHSmxwXxfBkM8DSqQ8vPzjd5hKgAOfAC/XwyXPgprb4bFj8GJ3w/0qMas3bt3d5sljjV33XUX8fHx3HHHHYEeik/efPNNDhw4wC233DKs7+Pp50NENhtj8j0drzN6ZXHn57PzQWxaeaPUILgvPI82GuiVxV1xk5QNiVlaeaNGxHPPPcdDDz3UbduZZ57JY489FqARhSYN9MpSfwQi4yEqAZLzdNGUGhHXXnst1157baCHEfK06kZZ6kohYYL12D5RUzdKhRAN9MpSdwTiXYE+eSLUH4Y27+qmlVKjmwZ6Zek6o0/Os75WFwRsOEop/9FAr6xVsfVHuqduQNM3SoUIDfQKWmqhrbHLjN4V6LXyRvkg1PrRNzY2ctFFFzFt2jRmzJjRsao2GGmgV5019O4cffx4CI/Wyhvlk1WrVnHuuecG5L19DfRd+9E/+eST/OhHP/J43B133MGePXvYunUrH330EW+99Za/hjyitLxSddbQu2f0Iq7Km4MBG5Lq4q0VcPgL/77mhJmw6Df9HtJXT/ovv/xyTPSjj42NZcGCBQBERkZy0kknUVRU5PW3eDTRQK86+9y4Az1Y6RtN3Yx5e/fu5U9/+hNPPfUUl19+OatXr+a3v/0tjzzyCGeffTYrV67k7rvv5sEHH+w4x92Pfs+ePYhIR193d9/1efPmUVBQwPnnn8/u3bv7fO/t27fz6aef0tDQwIknnshFF13E559/3tGPPjY2lsrKSlJSUnj00Ue57777yM+3OgAM1L2yr370PRubuVVXV/PXv/6VW2+9dRDfxcDTQK96z+jBqrwp+NS6UOtDe1g1DAaYeQ+nnj3p9+/fP2b60bs5HA6uvPJKbrnlFiZPnuy31x1JGuiVlaOPiLNWxbrZJ1oXaZuqINbzPyYV+nr2pHfPzvsTKv3o3W644QamTp3Kbbfd5vX7jzZ6MVZ1r6F308ob5cFY6kcP8POf/5yamppuqalgpDN61b2G3s29aKrqEGSeOOJDUqPXWOlHX1RUxD333MO0adM46aSTALj55psHvNftaKT96BU8fKIVzJc827mtuRZ+kwPn3g3zbgvY0MYq7UcfnP3oR4qv/eg1dTPWue8VG99jRh+dCDHJmrpRKgRo6masa6mDtobeqRtwtSvWQK+Gj/ajHxleBXoRuQB4CAgDnjbG/KbH/m8ADwKzgGXGmNe67FsO/Nz19H+MMS/4YdzKXzzV0LvZJ/p/oY7ymjHGp0qTYKT96H03mHT7gKkbEQkDHgMWAdOBK0Vkeo/DCoBrgD/2ODcF+CVwKnAK8EsRSfZ5lGr4eKqhd0ueCDWF4HSO7JgU0dHRVFRUDOoftQpdxhgqKiq8LlF182ZGfwqwzxhzAEBEXgYWAx0rH4wxB137ekaE84F3jDGVrv3vABcAf/JplF6oKivF9tjJ/n7ZoODExn0s5005e+CDe1hkPuQ3wOIX9nNQmrvtW2Ka+AWtnLfqFarC03niqpM5eaL+nh4J2dnZFBUVUVZWFuihqFEmOjqa7Oxsn87xJtBnAYVdnhdhzdC94encXqsSROQG4AaA3NxcL1+6u8joGHakLxrUucFuetX7LI/dTMSU7/p87hlHnFAMp82ewYlhcd325dUeD/tgyWQHv97Vwvaiag30IyQiIoJJkyYFehgqRIyKi7HGmCeBJ8EqrxzMa8Ql2Dn1pmf8Oq6g8ddbSdjxBnddfDzYfCykehs4GstPvzW3d6uDimh4BK4/wcavd0FNU5vfhqyUGjneRIViIKfL82zXNm8M5VzlrdzToaUGyvpuENUn96pYTxf9knIAIaymgISocA30SgUpbwL9RmCqiEwSkUhgGbDWy9d/G1goIsmui7ALXduUP+WeZn0t+MT3cz3V0LuFR0JiFlQdJDEmgppGDfRKBaMBA70xxgHcjBWgdwOvGmN2isgqEbkUQETmikgRsBR4QkR2us6tBP4b65fFRmCV+8Ks8iP7RCtYF3zq+7me+tx05WpXbI+N0Bm9UkHKqxy9MWYdsK7HtpVdHm/ESst4OvdZ4FlP+5SfiFiz+oINvp/rqc9NV8l5sP99khIjqNZAr1RQ0hYIoSL3dKgpgBof7oDTUget9f0HevtEqCshNcrojF6pIKWBPlR05Ol9SN/0vFesJ652xXnhFRrolQpSGuhDxfgTIDLex0Dfz6pYN1e74mw5Sk1jm67UVCoIaaAPFWHhkD3Xt0DfX58bN7s1o89wHqG13Ulzm7ZDUCrYaKAPJbmnwZEd0Fzj3fHezOjjx0NYFGmOwwBUN7UOcZBKqZGmgT6U5J4GGCja6N3xdYchPAaiEvs+xmaD5Ikkt5QAujpWqWCkgT6UZOWDhHmfvqk73Peq2K7sE0loshY066IppYKPBvpQEhUPGbO8D/QD1dC7JU8kpsHqTae19EoFHw30oSb3dCjaBA4vcukDrYp1S84jrLWWROo1daNUENJAH2pyTgVHExzePvCxdYf7r6F3c1Xe5EiZpm6UCkIa6EONtw3OvFkV6+ZaNDXRVqYzeqWCkAb6UJMwAZInDZynr/Oiht4t0bpXTG5krQZ6pYKQBvpQlHu6Fej7W8Vab9XFexXoY5IBYUJ4g16MVSoIaaAPRbmnQWM5VOzv+5g6V6D3JkdvC4OYZMaF6cVYpYKRBvpQlHu69bW/PH2dDzN6gNhUUm111DTqylilgo0G+lCUNhViUvrP09eVQng0RCd595pxaSRTpzN6pYKQBvpQ1HEjkn5m9O7FUgOtinWLTSXJWaM5eqWCkAb6UJV7GlTuh/oyz/u9raF3i00hrr2G2qY2nE5tVaxUMNFAH6rcefrCPtI37j433opNI9ZRg9MY6lsdQx+fUmrEaKAPVRmzrRz8oT7SNz4H+lRsxkEijbo6Vqkgo4E+VIVHwcQzYPNzsGtN930t9dBa51ugj0sDIFn0gqxSwUYDfSj71uMwfga8ejX887edC6jqvbhXbE+xqQCkUku1zuiVCioa6ENZwnhY/ibMugLevwdWXwdtTb7X0ENHoNcZvVLBJzzQA1DDLCIavv0EpE+D91ZB5dcwc4m1bxCBPkUDvVJBR2f0Y4EInPVjWPYHKPsS3v6ZtX0QOfoU6vS+sUoFGQ30Y8m0i+C6tyEpx7pPbLTd+3MjYjHh0aSH6YxeqWCjqZuxZsJM+OG/oLbE+1WxACJIbBrj2xvYpxdjlQoqXs3oReQCEflSRPaJyAoP+6NE5BXX/g0ikufaHiEiL4jIFyKyW0R+6ufxq8GITYEJJwzqvDTtYKlU0Bkw0ItIGPAYsAiYDlwpItN7HHYdUGWMmQI8ANzr2r4UiDLGzAROBn7o/iWgglBsKqmiNx9RKth4M6M/BdhnjDlgjGkFXgYW9zhmMfCC6/FrwDkiIoAB4kQkHIgBWoFav4xcjby4NJJMndbRKxVkvAn0WUBhl+dFrm0ejzHGOIAaIBUr6DcApUABcJ8xprLnG4jIDSKySUQ2lZX10YRLBV5sKonOGp3RKxVkhrvq5hSgHcgEJgE/EZHJPQ8yxjxpjMk3xuSnp6cP85DUoMWmEeNsoKGpKdAjUUr5wJtAXwzkdHme7drm8RhXmiYJqAC+C/zdGNNmjDkKfATkD3XQKkBiUwCIbKnC0e4M8GCUUt7yJtBvBKaKyCQRiQSWAWt7HLMWWO56vARYb4wxWOmabwKISBxwGrDHHwNXAeBeNCV11DZrq2KlgsWAgd6Vc78ZeBvYDbxqjNkpIqtE5FLXYc8AqSKyD/gx4C7BfAyIF5GdWL8wnjPGbPf3h1AjpKMNQi3Veu9YpYKGVwumjDHrgHU9tq3s8rgZq5Sy53n1nrarIBXb2QZBL8gqFTy0BYLyXtcZvQZ6pYKGBnrlvZhkDGLl6DXQKxU0NNAr74WFY6LtJGvqRqmgooFe+URiU0kVXR2rVDDRQK98InFppNm0sZlSwUQDvfJNbCppNp3RKxVMNNAr38Slao5eqSCjgV75JjaVJFNLTWNLoEeilPKSBnrlm9g0wnHQ1lQT6JEopbykgV75xrVoKqypIsADUUp5SwO98o2rsVlYU1WAB6KU8pYGeuUbV6vieGcNzW3tAR6MUsobGuiVb1yNzfTesUoFDw30yjeuHL2WWCoVPDTQK99ExuG0RZIiGuiVChYa6JVvRHDEpJKCro5VKlhooFe+i0klRXP0SgUNDfTKZxKfRorU6e0ElQoSGuiVz8JdgV5vPqJUcNBAr3wmcWl6MVapIKKBXvkuNpVEGqltbAr0SJRSXtBAr3znqqV31pcHeCBKKW9ooFe+cwV6GrWxmVLBQAO98p2rsVl4U2WAB6KU8oYGeuU714w+okU7WCoVDDTQK9+5An1UWxXGmAAPRik1EA30yncxVqtiu6mloVVbFSs12mmgV74LC6clIpEUqdXVsUoFAa8CvYhcICJfisg+EVnhYX+UiLzi2r9BRPK67JslIp+IyE4R+UJEov04fhUgjqgUXTSlVJAYMNCLSBjwGLAImA5cKSLTexx2HVBljJkCPADc6zo3HHgJuNEYMwOYD2hkCAHtMalWT3rtYKnUqOfNjP4UYJ8x5oAxphV4GVjc45jFwAuux68B54iIAAuB7caYzwGMMRXGGE3qhoLYVFJ1Rq9UUPAm0GcBhV2eF7m2eTzGGOMAaoBU4FjAiMjbIrJFRO709AYicoOIbBKRTWVlZb5+BhUAYfFpJId6oK8uhBcugU3PBXokSg1J+Ai8/jxgLtAIvCcim40x73U9yBjzJPAkQH5+vtbrBYGIxHRSCOGLsWVfwovfhtpi+PpfEB4Nc64M9KiUGhRvZvTFQE6X59mubR6PceXlk4AKrNn/v4wx5caYRmAdcNJQB60CLyI+jUhpp6m+2j8v2FAOb94OzTX+eb2hKNoEz54PTgdcvx7yzoI1N8HuNwM9MqUGxZtAvxGYKiKTRCQSWAas7XHMWmC56/ESYL2xVtK8DcwUkVjXL4CzgV3+GboKJIlLB8BR56fGZltegE3Pwp51/nm9wdq/Hl64FKKT4D/ehuyT4co/QeYceO1aOPBBYMen1CAMGOhdOfebsYL2buBVY8xOEVklIpe6DnsGSBWRfcCPgRWuc6uA+7F+WWwDthhj/ub3T6FGXkdjsz4C/can4Z+/9f71drxufT3w/tDGNRQ7Xoc/XA4pk+E//gEpk6ztUQnwvdcgdQr86bvWjF+pIOJVjt4Ysw4r7dJ128ouj5uBpX2c+xJWiaUKJa5AL546WDpa4L3/htZ6OPlaiE/v/7WO7oEjOyA8xpoxGwMi/h9zfz57Ctb9X8g93ZrBx9i7749NgavesFI6L30Hrl0H42eM7BiVGiRdGasGJ84K9GHNHjpYfvV3aK62ctzbXxn4tXa+Dgh84ydQfwSO7vbrUAdU+jmsuwOOvQCuer13kHdLmABXr4GIGOtCbcX+ER2mUoOlgV4NjruxWauHQL/tT5CQAZknwdYXrRl6X4yxUiZ582DWMmvbSOfBD/7b+nrxA1YQ709yHlz1F2hvtX45KBUENNCrwYmMxyGRRLdVd99eXwb73oFZl8PJy6FsDxRv7vt1Dn8BFXvhhO+APcfKg490oC/aCEk5kJjh3fHjpsGJ34evP4SW+uEdm1J+oIFeDY4ITRF24hy1tDu7zNi/+LOVspl9Jcy4zMq7b32x79fZsRps4XC867r+5PnWDNsxgvX5hRshe65v50xdCM42+PqfwzMmpfxIA70atNbIZFKklrrmLqtjP/8jZMyBccdDdCLM+BZ8sRpaG3u/gDFWfn7ygo6cP5MXQFsDFI9QZUttCdQW+R7oc06DyHjY+87wjEspP9JArwbNEW11sKx2NzY7vMNKxcz5budBJ34fWutgd8+lF1gpneoCOOGyzm1580BsI5e+Kdpofc05xbfzwiOt/33sfaf/axBKjQIa6NWgmdhUUqjt7Hfz+Z/AFgEnLOk8aOKZkDwJtnqosN2xGsIiYdpFndti7NZF3P0jVE9ftNEaw4SZvp87daH1v4GRrhJSykca6NWgSVyqNaNvaoN2B2x/FY49vzMNA1Y9/Infg4MfQuWBzu3OdqvaZupCaxVqV8cssGb7I9EOoXCjlWoKj/L93CnnWl/3afpGjW4a6NWghSekkySN1DY0Wq0DGo5aF2F7mv1dQGDbHzu3FXwC9Ye7p23cJs8H095Z9jhcHK1Qus33/LxbUhaMP0Hz9GrU00CvBi0ywVrx2lxbbl2EjUmxZug9JWXBlHOsQO903Y5gx2qIiLUWKfWUPdfaN9x5+iM7wNEMOYMM9ABTz7N+aTXX+m9cSvmZBno1aNFJVqC3Ve63mpHNXGpdpPTkxO9bLX/3v2+leXatgeMWQWRc72PDo6zc/nAHeveF2MHO6AGmnGeVk2qzMzWKDXc/ehXCIhLGAZC2+/fQ3sLvqudS8pcvPB4b5szlzrAkDvztUTanHOKaxgr+UH8yu/s4/ozG47iw/B1+++p71EaOG5bxLz30DnkR6fx/71cBVYN6DZuJ5ae2eHa++wp/+WqSX8cnCN89NZfjMxL9+rpq7NFArwbP1QbhzNaP2E82z+xLAjnc5+HHOeextPptKqrrqCeWRwvyaC30fPxek8eFQOPu93jLtmA4Rs+N7Z+zkWN464u+x+yNs8xMZld+zFvVpX02Y4s3DZxlNvOOnIFDvPtnV9XYisPp5NeXzRrS+JTSQK8GLy4NgHCcHHPuD9g8z0N+vqvDE+DxvzGfTTD7u3zy7Yv6PtYYuO9e7ppcxl3fOc+Pg3apPwr3HSV74S1sPmOIr7/1CKy5ic03ZPRdpvnadbDjNf5n0nZY+kLfjdO6+PbvPuJQhYeFZkr5SHP0avBikq2vYoNZVwx8/ISZVikjWL1t+iNiVd+42xb7mz/y827uMsu9//C8f+87sOM167iDH8EzC6HqYP+v6WhlcfgGmsoLhj4+NeZpoFeDFxZhpW8mL/C+IdhZP4aJ82Dy2QMfO3m+VbJ51MebktUdgapD/R9TtNHqsZMx27fX9iRhgvU6nsosWxvgzR9D2rGw7I9WT/v6I/DUOVD4We/jnU5rPcKj+VxTcjffavwzrQ7n0MeoxjQN9Gpolj4PF9/v/fHTF8O1f7N+SQxk8nzrq7cVLa2N8MG98NBseHJ+/yWPhRthwqyB2xJ7a8p5VuBu6nFR9/1fQU0BXPKQVU006Sy4/l3rrlXPX2yVmYL1v5av/gFPnAWv/wCiEmmMHk82ZRRXN/lnjGrM0kCvhmbSN6we7cMhKRtSpw7cDsEYK2A+Ohc++BVMPAOaKuGzJzwf3+6Aki3+Sdu4TV1oLfLqOtaSbfDp7+Dka6wxuaVNhevfg6yT4LX/gLf/C567EP641Lor12VPww//RUv6TLKknEMVDf4bpxqTNNCr0W3yfDj0Ud9ti0u2wnOLrIAZmwLXrLPuEnXsIvj4Uc9tFI7uhLZG3xuZ9Sc7H6Ltnembdgf89RaIS4dz7+59fFyqdbeqWVfAJ49CxT648D64aSPMWgo2G1FpE8mScgo00Ksh0qobNbodswA2PgVv/9RKd7Q2WCma1npr1v71h9Z1gksethZl2cKs8+avgCfPhk8fh/n/2f01Oy7E5vtvnLYwa/XvvnetPPuGx61bFC59vu8Km/Ao+PYTcOJV1uy+x+KxmLQ8RJo4cvQI4N8afTW2aKBXo1vePKu6Z+PTVmfMyDirD3xkHETGwpm3wFk/6d0YLXMOHHcRfPIYnPrD7sG2aBPEjQP7RP+OdepCK4W056/w/j1We4fp3+r/HBErb+9plz0XgMajB4HT/DpUNbZooFejW3QS/OQr63Ff7RX6Mn+FdXHz0/+FBT/t3F74mZWf72Nx06Adc4719fUbQMKsVMxQ3sOeA4CzeoAKIqUGoDl6NfqFR/oe5AEyZsHxl1gXRN3VMI2VULl/aI3M+hKfbvXSdzTDOb/oCNSDlmTN6CPqSzB6cxM1BBroVWg7ewW01FopHPDvQilPTvuRdeOVU24Y+mvFpeGwRTHOWUZZXcvQX0+NWRroVWibcIKVJ//0cWs2X7TRSqtknjg87zfrcljyTOdF4aEQoTUukywp41CltkJQg6eBXoW++SusKp2PH7Hy8+NneG6PPAqJPcdVS6+BXg2eBnoV+sYdDzO+DRuesCpuhittMwwiU/PI1lr6MeFQRQOHa5qH5bW9CvQicoGIfCki+0RkhYf9USLyimv/BhHJ67E/V0TqReQOP41bKd/MX2Etkmpr8O9CqWEWlpxLmtRSUj64fvkqeKxcs5Mf/H7TsLz2gIFeRMKAx4BFwHTgShGZ3uOw64AqY8wU4AHg3h777wfeGvpwlRqk9ONg5hLrcRDN6N2VO03lWmIZ6oqqGslO9lPvpR68mdGfAuwzxhwwxrQCLwOLexyzGHjB9fg14BwRq4BYRL4FfA3s9MuIlRqsC34Dlz0FqccEeiTeS7ICvakuDPBA1HAyxlBU1RTQQJ8FdP0pK3Jt83iMMcYB1ACpIhIP/CfgodlHJxG5QUQ2icimsrIyb8eulG/i0qyqmGCSlA1AQsth6lscAR6M8oUv7aXL6ltocTjJSYkdlrEM98XYu4AHjDH1/R1kjHnSGJNvjMlPT08f5iEpFUQSMzHYrBJLvSAbNA6U1TP77n/wr6+8m7gWVVmtqAM5oy8Gui7xy3Zt83iMiIQDSUAFcCrwWxE5CNwG/ExEbh7akJUaQ8IiaIub4OpiqSWWweL5jw/S1NbO54XVXh1f6FonkZ0cuBn9RmCqiEwSkUhgGbC2xzFrgeWux0uA9cZyljEmzxiTBzwI/MoY86h/hq7U2GBLziVbykNq0ZQxhnv+tosvD9cFeih+V9PUxmubiwC8/jsL+IzelXO/GXgb2A28aozZKSKrRORS12HPYOXk9wE/BnqVYCqlBic8OZdsWwUFIRToD9c289SHX/Pm9pJAD8Xv/rypkMbWdtITorz+X1hRVROpcZHERg5Pn0mvXtUYsw5Y12Pbyi6Pm4GlA7zGXYMYn1LKnsN4KiksD53Zb4nr9ojDfZvEHcU1zMhMRIbYqfSlTw/x+pYi/viD04iO6Lu9RbvT8MInBzklL4Xc1Fg+3Ottjr6R7GG6EAu6Mlap0S8ph3DaaawoCvRI/MadqiitHp6VoABfFNVw8SP/5t3dR4f8Wuu+KGVLQTWP/3N/v8e9t/sIhZVNXHtmHhNTYjlS20JTa/uArz+cpZWggV6p0c9VSx9WV0Rbu/cle6NZiSvAl9QM34x+e3E1AJsPDW1VcbvT8HlhNeE24Xcf7O+3+um5jw6SZY/hvOnjyU21ZugDpdycTkOxBnqlxjjX6tgMU0Zx1fCmOkaKO3VTWtOM0zk8vfb3lFqpri9cAX+w9h2tp6G1nTvOP47IMBu/XLvT4/0B9hyu5ZMDFVx1+kTCw2zkpVqN8wYqiz1a10Jru5OcYaq4AQ30So1+rkVTWSFUeePOzbc6nFQ09HHj9yHac7gWgO1FNUO6ccvWAut/BOfPmMBt507lgy/LeHvnkV7HPf/RQaIjbCyba/1inuia0Q/UebSoyl1aqTN6pcauyDicMalkSehU3pRUNxEZboWf0mFI3xhj2HO4joSocOqaHUNq87ytsBp7bAR5qbFcc0Ye0yYksOqvO2ls7VypXNnQyhtbi/n2idnYY627odljI0mMDudQZf8z+sKq4a2hBw30SgUFseeQYwuddsXFVU3MzrZu6F4yDJU3JTXN1DU7uGROJgDbi2sG/VpbC6qZk2NHRAgPs/E/3zqBkppmHn5vX8cxL28soMXh5Noz87qdOzE1buAZfeXw1tCDBnqlgoLYc5gYXhkSNyCpbW6jrsVBfl4K0Hlh1p/2lFppm0tnZxIZbuOLoupBvU59i4OvjtYxJ8fesS0/L4WlJ2fz9IcH2HukjrZ2Jy9+cogzp6Ry7PiEbudPTI0d8H9hRVVNpCdE9Vu2OVQa6JUKBkm5TDBlITGjd8/gZ2QmEh1hG5YZ/R7XitsZmYlMz0hke9HgZvTbC6sxBk7MTe62fcWiacRFhfOLNTt4e+dhSmuaufaMSb3On5gaS1FVU7/VUoXD2J7YTQO9UsEgKZso00xt1ZEhXVgcDdyVQ1n2GDLtMZR6eVelz76u5OXPCrw6ds/hOrKTY0iIjmBWdhI7imsGVd2z1dWrZk62vdv21Pgo7rzgOD49UMnP/7KD3JRYFkwb1+v8iSlxtDtNv7/MiqqahrXiBjTQKxUcXCWWKW1HKKtvCfBghsYd9LLsMWQmxXi9OvbJfx1g5dqdtDgGXoC0p7SWaRMSAZiZlURDazsHyn3/39DWgmomp8WRFBvRa9+yubnMzk6iurGN5WfkEWbrvfp2oMob9y8BndErpToWTWVJRUenw2BVVN1EZJiNtPgoMu3RXlfdHKxooNXhHDAN0+Kwgvq0CVa+fJZrNu5rPb0xhm2F1czJtXvcH2YTfrtkNhfNyuDy/GyPx0x019L38Xd2uLYZh9MMWx96Nw30SgUDey7gqqUP8guyJdXNZNijsdmEjKQYa8HQADfpaHeajgZhGw9W9nvsvqP1tDsN0zKsQH9MehwxEWE+5+mLqpoor2/hxC4XYns6bkICj333JBKie8/4AcYlRBEVbuNQH/+bKKoc/hp60ECvVHCIScZExIVEoC+uaiQzyQpsWfYYjIEjtf3n6Uuqm2h1XdDc+HX/gd69ItadugkPszEjM5EvfAz021z5+Z4XYn1hswm5KbF9zug72xPrjF4pJYLYc5gSWRn0i6ZKqpvJcs1gM+zRrm39p28OuqqNJqfHselQFe39XFjdc7iWqHAbeamdwXNmdhI7S2px+NAraGtBNVHhNo6bkDDwwf2YmBrXZ7viwqpGRCDT9X0YLhrolQoWSTnkhFUE9S0F29qdHKlrJtNuBXr314Gamx10pT4uz8+hrtnR7w1L9hyuY+r4eMLDOsPbrOwkmtra2V/m/fduW2EVs7KTiAgbWpicmBrLocoGj9VSRVVNjE+IJip8+GroQQO9UsEjKZvxzjIKKoO3sdnhmmaMgSzXDNadwhlo0dSB8gZiIsK4aGYG0H+efs/huo60jdvMLDsA271cONXqcLKjpLbbQqnBmpgaS3Obk6N1vaulCiuHv4YeNNArFTzsOcS119BQX0NDi2Pg40eh4o7SSiutEhMZRnJsxMCpm/IG8tLiyE6OISMpms/6CPTl9S2U1bV0VNy4TU6LIy4yjC+8bIWwu7SWVodzSPl5t47KGw/pm6KqpmGvuAEN9EoFjySr8iYziJubuRdLdc1JZyQNvGjqYEUjk9JiERHm5qWw8etKj6kQd0qn54zeZhNOyEryuvLG3bHSLzP6FHctffe0kaPdyeHaZp3RK6W6cC2ayg7iyhv3zN2dm3c/7m9G72h3UljZ2NHffe6kFI7WtVDoIYW129Xjxl1a2dWs7CR2ldZ6dfOWbYXVjE+MIiNp6BdJs5JjCLNJr7+z0ppm2p1m2FfFggZ6pYJHx6KpcgoGaH07WpXUNJEWH9mtgVemPXrAFgEOpyEvzQr0p7iaoXlK33x5uI60+CjS4qN67ZuVbafV4eSrIwPfe3drYWfHyqGKCLORaY/uVWJZOAJ96N000CsVLBImgC2cyRHB28WyqKqJLHv3wJZpj6G22UF9H9cdvnalPCa5Av3UcfEkxUR4rKffc7iO4z3M5sGa0QMD1tNXNrRyqKLRL/l5t7zUuF4N6Uaqhh5ARluDpPz8fLNp06ZAD0Op0enBmXzQNJkb6m8kPjq8267LnO9wo3mFMIb3vrIRYUJ8VPjAB3pQ3dRGmAgJXcbe4nBS3+IgKSaCcA/9YprbnDS0OkiOjcS9u67ZQbvTYO/Sg8YAVQ2tREWEERfZu1zRAFWNrUSGhREf1Xc5Y2u7k7pmB4nREUSEDX1GD9DQ0k5LezsprpuSADS2ttPU1k5KXCQd75IxB656fVDvISKbjTH5nvYN7m9LKRUYSbmcGFHPFcfndNs8vmkfN371LKWxUymJnTZsb1/T1Mbhmma+PT2rW7D2hjHwt88KOHZ8PHNd6ReAmrpm/r7jCOfkjOs12wfY/nUl+8vqWTYjF3dELCipYfOhapZOzSbGFdRrm9p4c1sJZ0xKZUp6fK/XEWDzriO0OJxcPCOjz3HuKqzmi+Ials3IgSHW0LsddI33iuNyiHLdWWvLvnKO1DbznRld+uS4Wl34mwZ6pYKJPYekqg/572+d0LnN0QJP/RBik8n5P2+SE5c6bG9fWtPED3+znvKEKfx44XE+nVvV0Mp//fsdVs6eztx5nb3b26qbWLltPRFTZ3LlKb0D3SPPbKA6rY0rL57Xsa2loIqVv/uY9CknschVW//v7aWs3LSFN8+bB1lJHsewJWwPT394gHMXnt/njT7+3zMbKE9r5apLz/Lp8/WnaOdhVu7fzJyTz+xosvbw45/AOPjORaf77X36ojl6pYJJUg7UlUB7W+e2938FR3bA4kdhGIM8WKWQZ01N57XNRf22IfCks7Sy+6x9fEIUNum7DcLBioaOC7FuJ2QmER1h63ZBds/hWsJswpRxvWfzbrOykmhrN32urHU6rY6VJ/bRsXKwPLUrLqxqHJGKG9BAr1RwseeAcUJtifX80Cfw0UNw0nI49vwRGcLl+dmU1DTz8f5yn85zL5bqWWUSHmZjfGK0x9WxrQ4nxVVNTErtHhAjw23MybF3WyG7u7SOSWlx/d6Sb6brgmxf95A9UF5PXbPDL/XzXeX2qKVvdYxcDT1ooFcquLhKLKkphJY6eOOHkDwRzv/ViA3hvOnjscdG8OqmIp/O81RD79ZXLX1BZSNOQ68ZPVhllrtKajuqdb48UttrRWxPWfYYUuIi+7yH7NYCa/tJfp7Rx0aGk54Q1TGjL6luwhhGZFUsaKBXKri4L9ZVF8LbP4PqAvj2ExDVd7rC36LCw1g8O5O3dx6mprFt4BNciqubiI6wkezhbk0ZSZ5vQOJuZuYp0M+dlILTwJZDVdQ1t1FY2cTxGYm9jutKRJjZzwrZzYeqSIgOZ3Ka/7+feamd7Yo7SytH0YxeRC4QkS9FZJ+IrPCwP0pEXnHt3yAiea7t54nIZhH5wvX1m34ev1JjS2KW9XXLC7Dl93DmrZB72ogPY2l+Dq0OJ2s/L/b6nJJqq4be0yKkLHsMJTXNvdoauNsTT0rtHehPyk0mzCZsPFjZsQjquPEDtxSelZ3E3qP1NLVatyRsbmvnja1FfOd/P+bljYWccUwqNg9lnkOVm9LZrrhoBBdLgRdVNyISBjwGnAcUARtFZK0xZleXw64DqowxU0RkGXAvcAVQDlxijCkRkROAt4Esf38IpcaMiGiIGwcFn8D4E2DBzwIyjBOykjg+I5FXNxVx1el5Xp1TXN3kMW0D1oy+1eGkoqG126rWr8sbSIqJIDkustc5cVHhzMhM5LOvKxmfaLUq8NT6oKeZWUm0Ow3/2HWYXaW1/HlTEZUNreSlxvJfFx7PFafkDPgagzExNZbVW5ppbmunsKqRcJswIXF4+9C7eVNeeQqwzxhzAEBEXgYWA10D/WLgLtfj14BHRUSMMVu7HLMTiBGRKGNMcN/dWKlAsudCczVc9iSE917qP1Iuz8/m7r/uYndp7YApE7Bm9DMyPR/X0Ze+uqlXoJ/kIW3jNjcvhZc+PUReahwJUeEe6/B7cpc33vryNsJswnnHj+f7p00ctpm8m7vypqCykaKqJjLs0d165g8nb94lCyjs8ryI3rPyjmOMMQ6gBuhZ5/UdYIunIC8iN4jIJhHZVFZW5u3YlRqbvvlzuPz3MH5GQIexeE4WEWHCn724KNvc1k55fWtH//meOgN998qbgwMG+mRaHE7WfVHKcRMSvOpNMz4xih+cNYnbzp3KR//5TR6/6mTmTU0b1iAP3dsVF1U1kW0fmQuxMEILpkRkBlY6Z6Gn/caYJ4EnwWqBMBJjUipoHbMg0CMAICUukvOmj+cv24pZsWgakeF9zxvdFTVZfeSku87o3Zrb2impae7oWulJvmuFbV2Lw6u0DVgXZP/rouleHetPXdsVF1Y2Mv+49BF7b29m9MVA16RVtmubx2NEJBxIAipcz7OBN4CrjTH7hzpgpdTosfTkHCobWlm/50i/x7ln6n3l6JNjI4gKt3UL9O5SxLy0vme+afFRTE63fhH07EE/2thjI0iIDuerI3UcrWsZkWZmbt4E+o3AVBGZJCKRwDJgbY9j1gLLXY+XAOuNMUZE7MDfgBXGmI/8NGal1Chx1tQ0xidGDZi+Ka62gnZfOXQRIcve/QYkX5d371rZF3fb4r66Vo4WIkJeahyfHKgAICdlZCpuwItA78q534xVMbMbeNUYs1NEVonIpa7DngFSRWQf8GPAXYJ5MzAFWCki21x/xvn9UyilAiI8zMZ3Tsrm/S+PcqS277tEFVc3IwIT+rmRR4Y9umP1LHSWVnqqoe/qktmZTJuQ4NUF4UDLTY3tuGHKaJvRY4xZZ4w51hhzjDHmHte2lcaYta7HzcaYpcaYKcaYU9wVOsaY/zHGxBlj5nT5c3T4Po5SaqQtOTkbp4HXt/RdU19c1cT4hGgi+qkyyUyK6bZo6mB5A6lxkSRG915g1dWZU9L4+23fIDZy9PdonNhlJexI1dCDroxVSg3R5PR45uYl8+fNhR7v4wquxVIDBLYMewxH61podVj99AcqrQxG7gvLEWHC+ISRqaEHDfRKKT/47qm5HChr4B+7PF+ULanpe7GUW5Y9GmPoSAF9Xd67a2Wwy3XV0mfZY4a9nLMrDfRKqSG7ZFYmk9LieOCdr3D2aF/sdBpKq5vJtPc/g81I6iyxbGhxcLSuJeRm9O5FUyOZnwcN9EopPwgPs3HrOVPZc7iOv+883G1feX0Lre1OsgeY0btn/KU1zZ0XYvupoQ9G4xOiiYkI65jZjxQN9Eopv7hkdiZTxsXzwDtfdbspSVE/7Ym7cs/4i6ubOFg+cA19MLLZhKeuzuf/zD9mZN93RN9NKRWywmzCbedOZe/Ret7cXtKxfaBVsW6xkeHYYyMorWkK2Rk9wLypaZq6UUoFrwtPyGDahAQeencvjnareqa/G470lJEUQ0l1M1+XNzAuIYq4qNFfMhkMNNArpfzGZhNuO/dYDpQ3sGabNasvrmoiITp8wHp4sCpvSqqbBmxmpnyjgV4p5VfnzxjPjMxEHl6/l7Z2J8XVzV61Dwb3jN5K3Wig9x8N9EopvxIRbj/3WA5VNPL6lqJ+bzjSU6Y9htpmB+X1rSFXQx9IGuiVUn53zvHjmJ2dxMPv7aOoqtHrGX3XWvtQvBAbKBrolVJ+JyLcft6xFFc3Udfs8GlG76apG//RQK+UGhZnH5vOyROTgYFLK90yunS3nDjCi4pCmQZ6pdSwEBHuPP84YiLCmO5lC+HxidHYBDKToomOCBvmEY4dWqSqlBo2p05OZcfd5xPmZQOviDAb4xKimZSuaRt/0kCvlBpW3gZ5tzvOP45xCVHDNJqxSQO9UmpUWXJydqCHEHI0R6+UUiFOA71SSoU4DfRKKRXiNNArpVSI00CvlFIhTgO9UkqFOA30SikV4jTQK6VUiBNjzMBHjSARKQMODeEl0oByPw0nmOjnHlv0c48t3nzuicaYdE87Rl2gHyoR2WSMyQ/0OEaafu6xRT/32DLUz62pG6WUCnEa6JVSKsSFYqB/MtADCBD93GOLfu6xZUifO+Ry9EoppboLxRm9UkqpLjTQK6VUiAuZQC8iF4jIlyKyT0RWBHo8w0lEnhWRoyKyo8u2FBF5R0T2ur4mB3KM/iYiOSLyvojsEpGdInKra3uof+5oEflMRD53fe67XdsnicgG18/7KyISGeixDgcRCRORrSLypuv5WPncB0XkCxHZJiKbXNsG/bMeEoFeRMKAx4BFwHTgShGZHthRDavngQt6bFsBvGeMmQq853oeShzAT4wx04HTgJtcf8eh/rlbgG8aY2YDc4ALROQ04F7gAWPMFKAKuC5wQxxWtwK7uzwfK58bYIExZk6X+vlB/6yHRKAHTgH2GWMOGGNagZeBxQEe07AxxvwLqOyxeTHwguvxC8C3RnJMw80YU2qM2eJ6XIf1jz+L0P/cxhhT73oa4fpjgG8Cr7m2h9znBhCRbOAi4GnXc2EMfO5+DPpnPVQCfRZQ2OV5kWvbWDLeGFPqenwYGB/IwQwnEckDTgQ2MAY+tyt9sQ04CrwD7AeqjTEO1yGh+vP+IHAn4HQ9T2VsfG6wfpn/Q0Q2i8gNrm2D/lnXm4OHIGOMEZGQrJsVkXhgNXCbMabWmuRZQvVzG2PagTkiYgfeAKYFdkTDT0QuBo4aYzaLyPwADycQ5hljikVkHPCOiOzputPXn/VQmdEXAzldnme7to0lR0QkA8D19WiAx+N3IhKBFeT/YIx53bU55D+3mzGmGngfOB2wi4h7ohaKP+9nApeKyEGsVOw3gYcI/c8NgDGm2PX1KNYv91MYws96qAT6jcBU1xX5SGAZsDbAYxppa4HlrsfLgTUBHIvfufKzzwC7jTH3d9kV6p873TWTR0RigPOwrk+8DyxxHRZyn9sY81NjTLYxJg/r3/N6Y8z3CPHPDSAicSKS4H4MLAR2MISf9ZBZGSsiF2Ll9MKAZ40x9wR2RMNHRP4EzMdqXXoE+CXwF+BVIBerzfPlxpieF2yDlojMAz4EvqAzZ/szrDx9KH/uWVgX3sKwJmavGmNWichkrJluCrAV+L4xpiVwIx0+rtTNHcaYi8fC53Z9xjdcT8OBPxpj7hGRVAb5sx4ygV4ppZRnoZK6UUop1QcN9EopFeI00CulVIjTQK+UUiFOA71SSoU4DfRKKRXiNNArpVSI+/8BFHOk/C5dj3gAAAAASUVORK5CYII=", + "text/plain": [ + "
    " + ] + }, + "metadata": { + "needs_background": "light" + }, + "output_type": "display_data" + } + ], + "source": [ + "scheduler = MultiStepLRScheduler(optimizer, decay_t=[15, 30], decay_rate=0.5, noise_range_t=35)\n", + "plot_noisy_lr(scheduler, label='noise_pct=0.65, def')\n", + "\n", + "scheduler = MultiStepLRScheduler(optimizer, decay_t=[15, 30], decay_rate=0.5, noise_range_t=[10, 30], noise_pct=0.2)\n", + "plot_noisy_lr(scheduler, label='noise_pct=0.2')\n", + "plt.legend();" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### `noise_std`" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + " Noise standard deviation. Now it is not implemented." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### `noise_seed`" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Seed to use to add random noise." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Miscellaneous." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### `t_in_epochs`\n" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "If set to False, the learning rates returned for epoch `t` are `None`." + ] + }, + { + "cell_type": "code", + "execution_count": 15, + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "[None, None, None, None, None]" + ] + }, + "execution_count": 15, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "scheduler = MultiStepLRScheduler(optimizer, decay_t=[0], t_in_epochs=False)\n", + "lr_per_epoch = calculate_lr(scheduler, 5)\n", + "\n", + "lr_per_epoch[:5]" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### `initialize`" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "If True, then inside each param group of the `optimizer` a new field is set called `initial_{field_name}` where `field_name` refers to the field in param group that we are scheduling. Typically `field_name='lr'`." + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3 (ipykernel)", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.9.6" + }, + "toc": { + "base_numbering": 1, + "nav_menu": {}, + "number_sections": true, + "sideBar": true, + "skip_h1_title": false, + "title_cell": "Table of Contents", + "title_sidebar": "Contents", + "toc_cell": false, + "toc_position": { + "height": "calc(100% - 180px)", + "left": "10px", + "top": "150px", + "width": "341.323px" + }, + "toc_section_display": true, + "toc_window_display": true + } + }, + "nbformat": 4, + "nbformat_minor": 4 +} diff --git a/nbs/07g_PolyLR.ipynb b/nbs/07g_PolyLR.ipynb new file mode 100644 index 0000000..4f512d7 --- /dev/null +++ b/nbs/07g_PolyLR.ipynb @@ -0,0 +1,1022 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# PolyLRScheduler" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "In this tutorial we are going to be looking at the `PolyLRScheduler` in the `timm` library." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "PolyLRScheduler is very similar to CosineLRScheduler and TanhLRScheduler. \n", + "Difference is PolyLRScheduler use Polynomial function to anneal learning rate. \n", + "It is cyclic, can do warmup, add noise and k-decay." + ] + }, + { + "cell_type": "code", + "execution_count": 1, + "metadata": {}, + "outputs": [ + { + "data": { + "text/markdown": [ + "

    class PolyLRScheduler[source]

    \n", + "\n", + "> PolyLRScheduler(**`optimizer`**:`Optimizer`, **`t_initial`**:`int`, **`power`**:`float`=*`0.5`*, **`lr_min`**:`float`=*`0.0`*, **`cycle_mul`**:`float`=*`1.0`*, **`cycle_decay`**:`float`=*`1.0`*, **`cycle_limit`**:`int`=*`1`*, **`warmup_t`**=*`0`*, **`warmup_lr_init`**=*`0`*, **`warmup_prefix`**=*`False`*, **`t_in_epochs`**=*`True`*, **`noise_range_t`**=*`None`*, **`noise_pct`**=*`0.67`*, **`noise_std`**=*`1.0`*, **`noise_seed`**=*`42`*, **`k_decay`**=*`1.0`*, **`initialize`**=*`True`*) :: `Scheduler`\n", + "\n", + "Polynomial LR Scheduler w/ warmup, noise, and k-decay\n", + "\n", + "k-decay option based on `k-decay: A New Method For Learning Rate Schedule` - https://arxiv.org/abs/2004.05909" + ], + "text/plain": [ + "" + ] + }, + "metadata": {}, + "output_type": "display_data" + } + ], + "source": [ + "from nbdev.showdoc import show_doc\n", + "from timm.scheduler.poly_lr import PolyLRScheduler\n", + "show_doc(PolyLRScheduler)" + ] + }, + { + "cell_type": "code", + "execution_count": 2, + "metadata": {}, + "outputs": [], + "source": [ + "#hide\n", + "%load_ext autoreload\n", + "%autoreload 2" + ] + }, + { + "cell_type": "code", + "execution_count": 3, + "metadata": {}, + "outputs": [], + "source": [ + "#hide \n", + "import torch\n", + "from matplotlib import pyplot as plt\n", + "from timm.scheduler.cosine_lr import CosineLRScheduler" + ] + }, + { + "cell_type": "code", + "execution_count": 4, + "metadata": {}, + "outputs": [], + "source": [ + "#hide\n", + "model = torch.nn.Linear(1, 1) # Simple dummy model to create dummy optimizer\n", + "lr = 0.1\n", + "optimizer = torch.optim.Adam(model.parameters(), lr=lr)\n", + "t_initial = 50" + ] + }, + { + "cell_type": "code", + "execution_count": 5, + "metadata": {}, + "outputs": [], + "source": [ + "#hide\n", + "def calculate_lr(scheduler: PolyLRScheduler):\n", + " num_epoch = scheduler.get_cycle_length()\n", + " return [scheduler.get_epoch_values(epoch) for epoch in range(num_epoch)]" + ] + }, + { + "cell_type": "code", + "execution_count": 6, + "metadata": {}, + "outputs": [], + "source": [ + "#hide\n", + "def calculate_noisy_lr(scheduler: PolyLRScheduler):\n", + " num_epoch = scheduler.get_cycle_length()\n", + " lr_list = []\n", + " for epoch in range(num_epoch):\n", + " lr_list.append(scheduler.optimizer.param_groups[0]['lr'])\n", + " scheduler.step(epoch)\n", + " return lr_list" + ] + }, + { + "cell_type": "code", + "execution_count": 7, + "metadata": {}, + "outputs": [], + "source": [ + "#hide\n", + "def plot_lr(scheduler, label=''):\n", + " plt.plot(calculate_lr(scheduler),label=label)" + ] + }, + { + "cell_type": "code", + "execution_count": 8, + "metadata": {}, + "outputs": [], + "source": [ + "#hide\n", + "def plot_noisy_lr(scheduler, label=''):\n", + " plt.plot(calculate_noisy_lr(scheduler),label=label)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "The schedule looks something like:" + ] + }, + { + "cell_type": "code", + "execution_count": 9, + "metadata": {}, + "outputs": [ + { + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXoAAAD4CAYAAADiry33AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/YYfK9AAAACXBIWXMAAAsTAAALEwEAmpwYAABH20lEQVR4nO2dd3gUVReH37ubRkggkISaQAKhBQglIfSugHQRkCJNBAui2FEs6GdvIIogioCIFAEFFEEFEemE3iEgJXRC78nu/f6YDQYIyYbs7szu3vd5eNiduTNzhiG/nDn33HOElBKFQqFQeC4mvQ1QKBQKhXNRQq9QKBQejhJ6hUKh8HCU0CsUCoWHo4ReoVAoPBwfvQ24lbCwMBkVFaW3GQqFQuFWrFu37pSUMjyrfYYT+qioKJKSkvQ2Q6FQKNwKIcSBO+1ToRuFQqHwcJTQKxQKhYejhF6hUCg8HMPF6BUKheeSlpZGSkoKV69e1dsUtyUgIICIiAh8fX3tPkYJvUKhcBkpKSkEBwcTFRWFEEJvc9wOKSWpqamkpKQQHR1t93F2hW6EEK2EELuEEMlCiKFZ7G8khFgvhEgXQnS+ZV8fIcQe258+dlumUCg8jqtXrxIaGqpE/i4RQhAaGprrN6IchV4IYQZGA/cBsUB3IUTsLcMOAn2BH245tjDwBlAbSATeEEIUypWFCoXCo1Ainzfu5t/PntBNIpAspdxnu8g0oAOwPWOAlHK/bZ/1lmNbAn9IKU/b9v8BtAKm5trSHLh6+jCrZ3xIzfJRBIeEQr5CEFYeQmPAZHb05RReyqkj+zi+dCIVShXHJzAE8odB0coQXByUgCkMij1CXxI4lOl7CpqHbg9ZHVvy1kFCiIHAQIBSpUrZeeqb2Zu8kwZHJ2E+dkt9fZ982g9i2aZQsS0Ur6Z+IBV3TfLCr6hzYCzsvGVHYBiUqA7lWkLFNlDwtv/mCoNgNpupWrUq6enpVKpUiUmTJhEYGJjl2IkTJ5KUlMQXX3xxx/P17duXtm3b0rnzf1Hr/fv3U6lSJSpUqMD169dJSEhg/PjxuZpAdSSGSK+UUo6TUiZIKRPCw7NcwZsjlRObc/DJQ/QtOot6V0cxrMgXpN7zGcT31Tz6fz6BcY1hZBz89S5cPOHYm1B4BTL9OgAt/SbR+NoIJlYYw/UW70OFVnDmAPz2AoyIha+bwfrJkKayS4xGvnz52LhxI1u3bsXPz4+xY8c65Tply5Zl48aNbNmyhZSUFGbMmOGU69iDPUJ/GIjM9D3Cts0e8nJsrokOD+bbR5sz6P4mzDlehAYLizE++FEs/RbC83ug/RcQXh7+/gBGVIafHoeTu51ljsITsaZzXZqZ+WwbGtVOZPimgtyzrCIrKr8Jg5Ng0Fpo/jpcvwxzn4SRVeCv9+DKWb0tV2RBw4YNSU5O5vTp03Ts2JG4uDjq1KnD5s2bbxp34cIFoqOjSUtLA+D8+fM3fc8Os9lMYmIihw87TfpyxJ7QzVqgnBAiGk2kuwE97Dz/QuDdTBOwLYCXc21lLjCZBD1rl6ZZxSIM+2kr//tlO79uPsKHneOIqdkLavaCU8mwegxs/AE2T4da/aHxUMgf6kzTFJ6ANR0LZoIDfPlfxyq0jSvOS7M20+Ob1XRPjOTl1pUo0PA5aPAs7FsCq76Ev9+HNeOg6SsQ3w/MKqsZ4M1529h+5LxDzxlbogBvtKts19j09HR+++03WrVqxRtvvEGNGjX4+eefWbx4Mb1792bjxo03xgYHB9OkSRN+/fVXOnbsyLRp0+jUqZNdoZirV6+yevVqPvvss7u9rTyTo0cvpUwHnkQT7R3ADCnlNiHEW0KI9gBCiFpCiBSgC/CVEGKb7djTwP/QflmsBd7KmJh1NsUL5mN8nwRGPlidfacu0XrUMsYs2Uu6xQphMdDmExiyRQvtrP0GRtWANV+D9db5ZIUiE9KCRfw3uV+7TCgLhjTi0UZlmL72EC1HLGXJrhPaPFDZptDzR3h0qTZPNP95GFMPDqzQ8QYUV65coXr16iQkJFCqVCn69+/PsmXL6NWrFwDNmjUjNTWV8+dv/iX0yCOPMGHCBAAmTJhAv379sr3O3r17qV69OkWLFqV48eLExcU554bswC7XQko5H5h/y7bXM31eixaWyerYb4Fv82DjXSOEoGONktSPCeP1OVv5YMFOFmw9ykddqlG+aLCWMdH2U0gcAAte1n4Qd8yDjl9CwSxvR+HtWC1YuDmLK8DXzMutK9GqSjFenLmZvhPW0iU+glfbxlIwn6+WANBnHuz6DRa+DBNaQ91B0Ow18A3Q6Ub0x17P29FkxOhzS/369dm/fz9LlizBYrFQpUqVbMdnxOhPnTpF/fr1mTt3Lu3bt79Lq/OGISZjnU14sD9jHopndI+apJy5QttRyxj9V7Lm3QMUqQS9foJ2n8HhdfBlXdj8o75GK4yJNR3LHX5sapQqxC9PNWBQ07LM3nCYliOW8tdO26S/EFCxNTy2HBIehpVfaMkBx7a60HjFnWjYsCFTpkwBYMmSJYSFhVGgQIHbxvXu3ZsePXrk6M1nJiwsjPfff5/33nvPYfbmFq8Q+gzaxBXn92cacW9sUT5auItOY1aw+/gFbacQWhjnsWXaa/bsRzQv35Kuq80KYyHk7R59Zvx9zLzQsiI/PVGPAvl86DdxLc//uIlzV2yTdv5B2lvkQ7O0Cdrx98LW2a4xXnFHhg8fzrp164iLi2Po0KFMmjQpy3E9e/bkzJkzdO/e/abtjz76KBEREURERFC3bt3bjuvYsSOXL1/mn3/+cYr9OSGklDmPciEJCQnSFY1Hft18lNfmbOXi1XSeubc8AxuVwWyy5ddb0uD317QJ2+hG0HmimqhVALB6RDeizq+l6Bt7cxx7Ld3CqEV7GPv3PsKD/PmgcxyNy2dKH75wHGb0gkOrof4QLVvHwxf37dixg0qVKultxl0zc+ZM5syZw+TJk3W1I6t/RyHEOillQlbjvcqjz0yGd9+sYhE+WLCTzmNXsPfkRW2n2Rfuex86joGDq+GbZnB6n74GK4yBLevGHjK8+9mP1yMowIc+367h5dmbuXjN9pYYXBT6/KJl4iwfCdN7QdoV59muyBODBw9m6NChvPbaa3qbkmu8VugBwoL8GfNQTT7rVp19Jy/RZtQ/TFj+L1ar7S2neg/oNx+unoPxLeHo5uxPqPB4hLRgFbnzuqtFhvDL4AY82qgM09YeotXIpazal6rt9PGDdiPhvg9h16/w/QPa/zeF4fj8889JTk6mfPnyepuSa7xa6EHLzOlQvSR/PNOIumVCeXPednp8s4pDpy9rAyIS4OGFmpc/sQ3sX66vwQpdETIdq50efWYyMnNmPlYXH5Og27hVvDVvO1fTLNqA2o/CA+O1MM7ENmrltsKheL3QZ1CkQADf9q3FBw9UZUvKOe777B9mJB1CSgnhFaD/7xBcTPO49i/T21yFTghr7j36zMSXLsz8pxvSu25pvl3+L21G/cPmlLPazqqdocd0SN0LE9vCxZOOMVrh9Sihz4QQggdrlWLBkEbElijAizM3M3DyOk5dvKbl1fedDyGlYEpXLXav8DpMdxG6uZVAPx/e6lCFyf0TuXTNQqcvV/DZn3tIs1gh5h7oORPOHoTvOsBll6wvVHg4SuizILJwINMG1GFY60r8veskLUcs5c/txyEoHPrM1Tz7KZ21nHuFV3E3Mfo70bBcOAuHNKJNXHFG/LmbzmNX8u+pSxBVH7pPhdRkmNxR1clR5Bkl9HfAZBIMaFSGeYMbUKRAAI98l8TLszdzyS9MW+WYrxBM7qSKonkZQqY7TOgBCgb68lm3GnzRowb7T12i9Wf/MGX1AWSZJtBtCpzYAT90Vdk4DubYsWN069aNsmXLEh8fT+vWrdm9O3c/y61bt+bs2bN3df3hw4fz8ccf37bdbDZTvXp1qlSpQrt27e76/LeihD4HKhQL5udB9Xi0sZYx0XrUP2w4F6h59mY/mPKAlg+t8Aq00I3ji5K1jSvBwiGNSIgqxLCfttJ/UhInizWCTl/DoTUweyBYLQ6/rjcipeT++++nSZMm7N27l3Xr1vHee+9x/Hjufo7nz59PSEiIQ23LXEK5cOHCjB492iHnVUJvB/4+Zl6+rxLTBtQh3SLpPHYlo9ankd5tGlw6pXlc1y7qbabCBZikBelAjz4zxQoGMKlfIm+0i2VZ8inu+2wpi811oeW7sGMu/P6qU67rbfz111/4+vry2GOP3dhWrVo1GjRowAsvvECVKlWoWrUq06dPB+Do0aM0atTohqedsbo1KiqKU6dO3WgyMmDAACpXrkyLFi24ckV7A9u7dy+tWrUiPj6ehg0bsnPnrR1r7kzdunUdVtpY1UvNBbXLhDL/6Ya8Pmcrn/6xm793F2Jsq68I/6UPzOwH3aaqErQejib0zitEZjIJ+tWPpl7ZMJ6etoGHJybRq04j3qj1GD6rvoSCkVD3Cadd36X8NhSObXHsOYtV1RY7ZsPWrVuJj4+/bfvs2bPZuHEjmzZt4tSpU9SqVYtGjRrxww8/0LJlS4YNG4bFYuHy5cu3Hbtnzx6mTp3K119/TdeuXZk1axYPPfQQAwcOZOzYsZQrV47Vq1fzxBNPsHjx4hxvw2KxsGjRIvr372//vWeDUqVcUjCfFlNtVrEIr/60lWZz/fmu+uvU2DQcFr0JLf6nt4kKJ2KSFixOCN3cSoViwcx5sj4fLdjFN8v+ZVV4K2ZFH6TA78O0dN+Y5k63wdtYtmwZ3bt3x2w2U7RoURo3bszatWupVasWDz/8MGlpaXTs2JHq1avfdmx0dPSN7fHx8ezfv5+LFy+yYsUKunTpcmPctWvXsrUho4Ty4cOHqVSpEvfee69D7k0J/V3SoXpJapYqxDPTN3L/6vJMLtaRhitGQbE4iOuS8wkUbokJC2kuqkfj72Pm1baxNCofznM/bqLh7q78VWgPhWY+jBj4FxQu4xI7nEYOnrezqFy5MjNnzrR7fKNGjVi6dCm//vorffv25dlnn6V37943jfH397/x2Ww2c+XKFaxWKyEhIbkqiZwRo798+TItW7Zk9OjRPPXUU3YffydUjD4PRBYOZNrAOgy5pxz9j3dio6ky1jmD4MhGvU1TOAlnxujvRKPy4Sx4uiG1ykfQPnUQl65bSJ/SHa5dcKkdnkKzZs24du0a48aNu7Ft8+bNhISEMH36dCwWCydPnmTp0qUkJiZy4MABihYtyoABA3jkkUdYv369XdcpUKAA0dHR/PijVvJcSsmmTZvsOjYwMJBRo0bxySefkJ6e9wq6SujziI/ZxJB7yvPDow0Z5vs8x9KDuDCpK9aLp/Q2TeEEzNICLgjd3EpokD9f905gYPumDEp7ClJ3c+r7/mCw6rPugBCCn376iT///JOyZctSuXJlXn75ZXr06EFcXBzVqlWjWbNmfPjhhxQrVowlS5ZQrVo1atSowfTp03n66aftvtaUKVMYP3481apVo3LlysyZM+fGvrfffvtGaeOIiNsbHdWoUYO4uDimTp2a93v21jLFzuDc5TTGTJ3JMwefZEe+mpR4Yg5FCgTqbZbCgRwYXpGzIZWpNmSWbjbsOHqepRNf59FrE/ij9LM06f0avmb38NncvUyxUVBlinWkYKAvLz3cjS1VXqL61TVMHfkif+9W9Uo8CTMWpA4efWYqFS9A72c/ZntwfRrv/4yXv5j0XxE+hSILlNA7GCEECZ1f4EKZNjxpncJnE77n/d92anVMFG6PWVoM0Rwkn78PsY9/T3pgUYacfpcHRy3gty1H9TZLYVCU0DsDIQjuOgYRUopvg75k6t+bePCrlRw+q5axuztmrEiTQZLVAgsT2GMSJc2n+djvGx6fso7X52z9r/SxQTFauNjduJt/PyX0ziKgIKauEwmxnGZ+2dnsPn6B1p/9wx/bVbkEd8aMBYwi9ACRiYjmr1Pv+nI+r7id71YeoNOXK9h/6pLelmVJQEAAqampSuzvEiklqampBATkbtGegf7HeiAlakDTVyi56C3+atGGvuuiGfBdEgMbleGFlhXcZgJNoSGlNJ7QA9QdDHv+oN2Rzyj8wM88Mf8MbT9fxgcPxNEmrrje1t1EREQEKSkpnDyp5q7uloCAgCyzdLLDYP9jPZD6Q2DPH4QvHcasAf/w9rILjFu6j6T9p/miR01KhOTT20KFnaRbJWashojR34TJpPU3HlOf+ptfZf7gWTw5bRODfljP6n9LM6xNJfx9jGGzr68v0dHRepvhdSiX0tmYzHD/WJCSgHmDeLt9LJ93r8GuYxdoM+oflZXjRqRbJD5G9OgBQiKhzcdwaBUlt45l+sC6PNIgmu9WHqDL2JUqK8fLUULvCgpFwX0fwIFlsPor2lUrwbzBDShaIIC+E9bw6R+7sVhVzNLopFmtmLEgjCj0AFW7QOVOsOQ9/E5t59W2sXzVK55/T12i7efLWLRDzQ95K0roXUX1HlC+FSx6C1L3UiY8iJ+eqM8DNSMYtWgPfSes4fSl63pbqcgGi0Xig9WYHj2AENDmE60pzpwnwJJGy8rF+GVwAyIK5aP/pCQ+WrhTORVeiBJ6VyEEtB2hNSuZOxisVvL5mfm4SzU+eKAqq/89TdtR/7Dx0Fm9LVXcgTRLOiYhjV2KOrCwJvZHN8HyzwAoHZqfWY/Xo1utSEb/tZc+364h9WL2VRQVnoUSeldSoAS0fAcOLIek8Tc2P1irFLMfr4fJJOgydgWTVx1Q6WcGJD0tDQCTUT36DGI7QGxH+PsDOKE1ugjwNfP+A3F8+EAca/efpu3ny9hw8Iy+dipchhJ6V1PjISjbHP54A84cuLG5SsmC/DK4AQ1iwnjt5628MHOz4Re+eBvWdE3oDRu6yUzrj8E/WAvhZGpB2LVWJLMer4ePWfDgV6v4YfVB5VR4AUroXY0Q0O4z7e/5z99UfTAk0I/xfWrxdPNyzFyXQuexK1S2hIFIswm9MHLoJoOgcLjvQzi8DtZ+c9OuKiULMu/JBtQtG8orP23hpVnKqfB07BJ6IUQrIcQuIUSyEGJoFvv9hRDTbftXCyGibNt9hRCThBBbhBA7hBAvO9h+9yQkEpoOgz2/w/afb9plMgmeubc84/skcCD1Mu2/WMbyZFXy2AhY3UnoAao8oL09LnoLzt3cezQk0I9v+9biqWYxzEhKoetXKzmiSnR4LDkKvRDCDIwG7gNige5CiNhbhvUHzkgpY4ARwAe27V0AfyllVSAeeDTjl4DXU/tRKF4dfnsJrpy9bXfzSkWZ92QDwoP96TV+Nd/8s0+9YutMRgMItxF6IaDtp1ro5rcXb9ttNgmebVGBcb3i2XfyEu2/WMbqfak6GKpwNvZ49IlAspRyn5TyOjAN6HDLmA7AJNvnmUBzIYQAJJBfCOED5AOuA+cdYrm7YzJDu5Fw6aTmcWVBVFh+Zj9Rn5aVi/H2rzsYMn2jesXWEUu6m0zGZqZQFDR5CXb+Ajt/zXJIi8rF+HlQfQrk86XnN6v5buV+5VR4GPYIfUngUKbvKbZtWY6RUqYD54BQNNG/BBwFDgIfSylP33oBIcRAIUSSECLJq2pglKgBtR+DpG/h0JoshwT5+/Blz5q80LICczcdofPYFaoKpk5YLLbJWHfx6DOo+yQUqQzzX7hj+8GYIkHMGVSfJhXCeX3ONobO2sK1dOVUeArOnoxNBCxACSAaeE4IcVtHYynlOCllgpQyITw83MkmGYymwyC4OPz63E3ZEZkRQjCoaQzf9E5g/6nLtP98GWv+ve33pcLJWG2hG7PZV2dLconZV3t7PH8Y/v7wjsOCA3wZ1yuBwc1imJ50iO7jVnHiwlXX2alwGvYI/WEgMtP3CNu2LMfYwjQFgVSgB7BASpkmpTwBLAeybHXltfgHabn1xzZrnn02NK9UlJ8H1adgPl96frOKaWsOushIBfwXunGbGH1mIhOh+kOw6ks4ueuOw0wmwXMtKvBlz5rsOHqBDl8sZ0vKORcaqnAG9gj9WqCcECJaCOEHdAPm3jJmLtDH9rkzsFhqQb6DQDMAIUR+oA6w0xGGexSV74foRrD4f3Ap+wybmCJa6YQ6ZUIZOnsLw+duI111r3IJVlvoxuSOQg9wz3Dwy69NzOYQg29dtTgzH6+LSQi6fLWCeZuOuMZGhVPIUehtMfcngYXADmCGlHKbEOItIUR727DxQKgQIhl4FshIwRwNBAkhtqH9wpggpdzs6Jtwe4TQFrhcvwR/Ds9xeMFAXyb0rUX/BtFMXLGfvhPWcu5ymvPt9HIyYvQmHzcL3WQQFA5NX4V9S2D7nByHVy5RkDlP1qdKiYIMnrqBT37fhVXVyXFL7IrRSynnSynLSynLSinfsW17XUo51/b5qpSyi5QyRkqZKKXcZ9t+0ba9spQyVkr5kfNuxc0JrwB1HocNkyElKcfhPmYTr7WN5cMH4lj9byr3j1nOvwbtKuQpZMTo3dajB0h4GIpWhYWvaI5FDoQF+TNlQG26xEfw+eJknpy6nivX1SStu6FWxhqJxi9BUDHt1dpqXzima61IJvevzZlL1+k4ejkr96o8aGdhtWQIvZt69KBlDLX5WJuYXTbSrkP8fcx82DmOV1pX5Letx+j61UqOn1eTtO6EEnoj4R+sxVEPr4MtM+w+rE6ZUH4eVP/G4qoZSYdyPkiRazImY83u7NEDlKoDVTrDilFw1r4JfSEEAxuV5eteCew7eZEOXyxn2xE1SesuKKE3GnEPQsl4LVZ/7aLdh2WUoq1bNpQXZ27mwwU7VTzVwUirB4RuMrj3TUBoxfVywT2xRfnxsXoIAV3GrlTNTNwEJfRGw2SCVu/DhaOwfGSuDi2Yz5dv+9aie2Ipvlyyl8FTN6iVtA5EZsTofTxA6AtGQP2nYdtsOLAiV4fGlijAnEH1KRsexIDvkpiw/F8nGalwFErojUhkotYWbvmom0oZ24Ov2cS791dhWOtKzN96lB5fr1JNJhxERtaNj4+fzpY4iPpPQ4GSsGCo3XNCGRQpEMD0R+twT6WivDlvO2/O26Y6VxkYJfRG5Z43tXo4dqRb3ooQggGNyvBlj5psO3Ke+79cwb6T9oeBFHfA6kEePYBfINz7ltaNatPUXB8e6OfDmIfi6d8gmgnL9/PY9+u4fD3dCYYq8ooSeqNSsCTUG6y9WtuRbpkV91UtztSBdbh4LZ1OY1aQtF+VTcgLFosWBnO7EgjZUeUBKJkAi9+G67nvfWA2CV5rG8vwdrEs2nGc7uNWcUq9QRoOJfRGpt5TkL8ILByW40rGO1GzVCF+eqIehQL96PnNahZsPepgI70HeSN040FCLwS0eBsuHIGVo+/6NH3rR/NVrwR2Hb/AA2NWqDUdBkMJvZHxD4Jmw+DQKtgx765PUzo0PzMfq0ul4gV4fMp6JqrJs7tCelroJoPSdaFSO1g2Ai7cfRbNvbFF+WFAHS5cTeeBMStUT1oDoYTe6FR/CMIrwZ9vQPr1uz5NaJA/UwfUoXnFogyft50PFuxUNcdzibQtmPKYydjM3PMmWK7BkvfydJqapQox6/F6BPn70P3rVSzeqdIvjYASeqNj9oEW/4PT+3KsbpkT+fzMfNUrnh61SzFmyV6e/3Ezaaogmv3YhN7saR49QGhZqPUIrJ8EJ/JWdzA6TFvTEVMkiAHfrVML+AyAEnp3IOYeiG4Mf38AV/O2GtFsErzTsQpD7inHrPUpDPwuSWVK2ElG6AZ36jCVGxq9CH5BsOjNPJ8qPNifaQPrUreMtoDvyyXJ6g1SR5TQuwNCaKURrpyGFZ874HSCIfeU5937q/L37pM89M1qzl6++7CQt+DxQp8/VMut3zUfDqzM8+mC/H34tm8tOlQvwYcLdvH2rzvUam2dUELvLpSsCZU7aZkRF4455JQ9apdidI+abD18nge/WsWxc6pQVbZ4utCDVkE1qJg2J+QAD9zPx8SIrtXpWy+K8cv+5fmZm1S4UAeU0LsTzV4Fy/Vs28HllvuqFmdiv1qknLlM57Er2K/S4u5IxmSsRwu9X35oMhQOrYZdvznklCaT4I12sTx3b3lmrz/MY5PXqdIcLkYJvTsRWhbi+8G6iXAq2WGnrRcTxtSBdbh83ULnsSvZeey8w87tUdzw6M362uFsavSC0BgtVm9xzPyNEILBzcvxdscqLN51gr4T1nDxmpobchVK6N2Nxi+CT4DWdtCBxEWEMOPROviYBA9+tUrlQGdFRvN2T/boQcv0av4GnNx5V6URsuOhOqUZ+WB11u4/Q8+vV3HmkpobcgVK6N2NoCJQ70nY/jMc2ejQU8cUCebHx+oSEuhLz29Ws2Jv9v1rvQ5viNFnUKmdVi57yfuQ5ti5mw7VS/LVQ/HsOHaBB8et5IRqYuJ0lNC7I3UHQb5CsOgth586snAgPz5al4hC+eg3YS1Ldp1w+DXcFm8SeiGg+etwPiXP6zey4p7Yora5oSs8OG4VR85ecfg1FP+hhN4dCSgIDZ6FvYtg/zKHn75IgQCmDaxLuaJavfEFWx2T5ePuiIzQjfCSH5syTbT1G/98AtcuOPz09cqGMbl/bU5dvEaXsSs5kKoSAZyFl/yP9UASB0Bwcc2rd8JClML5/ZjySB2qlCzIoB/WM3fTEYdfw+2wppOOWfN2vYXmb8DlU7BqjFNOH1+6EFMH1OHy9XS6frWSvaqctlNQQu+u+ObTJmYPrYbdC51yiYL5fJncvzbxpQsxZNoGftqQ4pTruAtCpmPBwzNubiUiHiq21RbqXXZOmesqJQsybWBdLFbJg1+tYs9xx789eDtK6N2ZGr2gcBktAyeXHYLsJcjfh4n9alGnTCjPztjk3XVLrBaswsuEHrT1G9cuaNUtnUSFYsFMG1gHIaDbuFUqxdfBKKF3Z8y+0OQVOL4Vdsxx2mUC/XwY36cWDWLCeHHmZqatOei0axkZr/ToAYpU0prWr/k6T2WMcyKmSDDTB9bB12yi+7hVbD+ixN5RKKF3d6p0gvCK8Ne7/+V5O4F8fma+7p1A4/LhDJ29hRlrvc+zF9JLPXrQwoSW67DsU6depkx4ENMfrUOAr5me36xix1El9o5ACb27YzJD01fg1G7Y8qNTLxXgq5U5blQ+nJdmb/a6MI6wWrB6o0cP2qrsGj21VMtzzp2rKR2an6kD6uDvY6bnN6tVGMcBKKH3BCq2g2JxWtMIW7s7ZxHga2Zcr3gaxITx0qzNzFznPRO0QqZ7r0cPWhljgKUfOf1SUWH5mTqwDr5mQY+vV7NbTdDmCSX0noDJpE2YndkPG6c4/XIBvloYp37ZMF6cuYk5Gw87/ZpGQHjrZGwGIZEQ3xc2fK81wnEy0WGaZ+9j0sRepV7ePUroPYVyLSCiFvz9EaRfc/rlMsS+VlRhnp2xid+2eH7TcZM3x+gzaPgcmHwdWkE1O8qEB/HDgNqApMfXq9SiqrtECb2nIIQWqz+fAuu/c8kl8/mZ+bZvLapHhjB46gb+3O7Z/UFNeHnoBiC4GNTqD5unO7SCanbEFAnm+0dqcy3dSo+vV3NYlUvINXYJvRCilRBilxAiWQgxNIv9/kKI6bb9q4UQUZn2xQkhVgohtgkhtgghAhxovyIzZZpCqbraknUHF6K6E/n9fZjQrxaxJQrwxA/rWZHsuYXQhNWC9HahB60LlU+A1trSRVQsVoDv+9fm/NU0en69ihMXVCG03JCj0AshzMBo4D4gFuguhIi9ZVh/4IyUMgYYAXxgO9YH+B54TEpZGWgCOHe20JvJ8OovHNWaPLuIAgG+TOqXSHRofh75Lon1HlriWAvdeEFBs5wIKqKV4Ng6E07uctllq5QsyMR+tTh+/hq9x69R7S9zgT0efSKQLKXcJ6W8DkwDOtwypgOQoSwzgeZCCAG0ADZLKTcBSClTpZSqtYwziW4EpRvYvHrXveIWyu/H5P6JhAf70/fbNR652MWE8uhvUO9p8A10qVcPEF+6MF/3TmDfyUv0mbBWNS+xE3uEviSQOWE6xbYtyzFSynTgHBAKlAekEGKhEGK9EOLFrC4ghBgohEgSQiSdPHkyt/eguJWmL8PF404pL5sdRQoEMOWR2uT396H3t2s8buLMJJXQ3yB/KNR+FLbOhhM7XHrpBuXC+KJHDbYePsfA75K4lq58x5xw9mSsD9AA6Gn7+34hRPNbB0kpx0kpE6SUCeHh4U42yQuIaqCVl102Aq5fdumlIwoFMrl/IharlYfGr/aophImaUF6Qy16e6n7JPgFaes3XEyLysX48IE4VuxNZci0jVisjq/g6knYI/SHgchM3yNs27IcY4vLFwRS0bz/pVLKU1LKy8B8oGZejVbYQZOX4dJJSBrv8kvHFAlmQr9EUi9ep/e3azh32TOmZcwqdHMzgYWhzuOwfQ4c3+byyz8QH8GrbSrx29ZjvPrzFqQTynV7CvYI/VqgnBAiWgjhB3QD5t4yZi7Qx/a5M7BYav/qC4GqQohA2y+AxsB2x5iuyJbSdbXGEcs/g+uuD6FUjwxhXK8E9p68yCPfreVqmvu/XiuPPgvqPA7+BVweq8/gkYZleKJJWaauOcSnf+zWxQZ3IEeht8Xcn0QT7R3ADCnlNiHEW0KI9rZh44FQIUQy8Cww1HbsGeBTtF8WG4H1UspfHX4XiqxpPNTm1bs2Vp9Bg3JhjLA1gn562ga3fr2WUiqPPisCC0Ptx3Tz6gFeaFmBBxMi+XxxMt+vOqCLDUbHrhi9lHK+lLK8lLKslPId27bXpZRzbZ+vSim7SCljpJSJUsp9mY79XkpZWUpZRUqZ5WSswklkePXLRuri1QO0jSvBa21jWbjtOMPnbnPb12uLVRN6r+gXm1syvPol7+tyeSEE79xfhWYVi/D6nK0s3KZaX96KWhnr6TR5WWsFt9b1sfoM+jeIZmCjMkxedYAvl+zVzY68kG6V+GBFmpRHfxsZXv2OuXBsqy4m+JhNfNGjBlUjQnhq6gbWHXBONyx3RQm9p1OqjrZiVqdYfQZDW1WkfbUSfLRwl1sWQUvP8OjVgqmsuRGr18erB61Bzrd9EiheMIBHJiWx/5RnpffmBSX03kCTobp79SaT4KMucSRGFeaFHzez5l/38rjSLVbMWFXo5k4EFtby6nfM082rBwgN8mdCv0QA+k1cy5lLavUsKKH3DkrV0WL1K0a5PK8+M/4+Zsb1jieiUD4GTk5inxuVnU2zSHywaI1eFFlT5wnwC4alrqlseSeiw/Lzde8EDp+9wsDJSR6R8ZVXlNB7Czpn4GQQEujHhH61MAnBw27kcVmsErOwqvTK7Mjw6rfPgeP6ZlEnRBXmky7VWLv/DC/N2uy2SQCOQgm9t1C6rlYHZ/lnunr1oLWK+7p3PEfOXuXxKeu4nm7V1R57SLNY8cGCUEKfPXUHaatldfbqAdpVK8HzLcozZ+MRRv/lmpLKRkUJvTfReChcOgHrJuptCfGlC/P+A1VZte80b8zdaniP68ZkrFkJfbYEFobEgbDtZzixU29rGNQ0ho7VS/Dx77uZ7wXNce6EEnpvIqo+RDWE5SNdWtnyTnSqGXFjVeP4Zf/qbU62WKxWfNRkrH3UfRL88hvCqxdC8P4DcdQsFcKzMzayJeWc3ibpghJ6b6PJUK2y5TrX1avPjudbVKBl5aK8O38HS3cbt3JpmkXz6FXoxg7yh9rq1c92ab36OxHga+arXgmE5vdn4OQkTl5wfqtNo6GE3tuIaqDVq18+0mVdqLLDZBJ82rU65YsGM3jqBsOWNrbYFkwJFbqxj7qDtXr1Sz/W2xIAwoP9Gdc7njOXr/OEm8wLORIl9N5I4xe1LlQbJuttCaC1IxzXKwGAgd+t45IBm0mkWazKo88N+UO13rJbZ7qst2xOVC5RkA87a5k4b87Tpy6PXiih90aiG2m9ZZeNgHRjvMaWCg1kdI+a7DlxgedmbDLc5KxWAsGiPPrcUO8pMPvD0o/0tuQG7auV4NHGZZiy+iA/rD6otzkuQwm9NyKE5tWfP2wYrx60apevtK7Egm3H+GrpvpwPcCHp6RbMQiqPPjcEhWte/ZYZkGqcGkcvtqxI4/LhDJ+7jY2HzuptjktQQu+tlGkKEYnwzwhIN86ipf4NomkTV5wPF+xkxd5TeptzA4tFa56iPPpcUu8pMPsZJlYPYDYJRj5YnfBgf574fh2n3WTRXl5QQu+tCAFNXoLzKbBxit7W3EAIwQcPxBEdlp+npm7g2Dn9J4wBLGnavIES+lwSXBQSHobN0+G0cd7SCuX3Y+xD8Zy6dJ2nprp3rwR7UELvzZRtDiXj4Z9PDeXVB/n78FWveK5ct/DElHWkWfTPkLBYNKE3mXx1tsQNqfeUtv7gn0/0tuQmqkYU5K32lVmWfIoRHt6dSgm9NyOEtlr23EHYPE1va24ipkgwH3auxvqDZ/loof652JZ0W+jGR3n0uaZAcYjvC5umwZn9eltzE90SS9E1IYLRS5INvY4jryih93bK3QslamgxVIuxmni3iStOrzqlGbd0H4t3HtfVlgyP3mxW1SvvigZDQJi0t0eD8Wb7KpQvEswz0zdy4rwxQoWORgm9t5Ph1Z89oMVRDcawNpWILV6AZ2ds4shZ/co2WC1aaEuYVejmrihQAmr20eaDzhorrTGfn5nRPWtw+bqFp6dt9Mh4vRJ6BZRvCcWr2bx6Yy1WCvA180WPGqSlW3lq6gbSdYrXW9IzYvQqdHPXNBgCCEN69TFFgvlfxyqs3JfKqEV79DbH4SihV9i8+pfgzL9azrPBKBMexLudqpJ04AyjFuuzylJmhG58lEd/1xSMgJq9YMP3cPaQ3tbcRuf4CDrVLMnni/e4XQe0nFBCr9Co0BqKVdVWMRrMqwfoUL0knWqW5IvFe0ja7/ofQmu6yqN3CA2e1f5eNkJfO+7AWx2qEFk4kGemb+TcFWPNWeUFJfQKjQyv/vQ+rT6JAXmzfWUiCgUyZPpGzl917Q+hRXn0jiEkEmr01FZknzNek/ggfx9GPlidY+ev8voc/XrfOhol9Ir/qNAGilbRvHqr8fpsBgf4MuLB6hw9d5U35ri2KFVG6MakhD7vNHgWpNWwXn2NUoUY0rwcczYe4ecNxvtldDcooVf8h8mk1cBJTYats/S2JkviSxfiqWbl+GnDYeZtOuKy62aUQDCr0E3eKVQaqveA9ZPgvOueYW54omkMtaIK8drPWzmsY7aXo1BCr7iZiu2gSCz8/aEhvXqAQU3LUi0yhNfmbOXEBdfkPd+YjFVC7xgaPmdor95s65NglZKXZm7G6uYpl0roFTdjMmmx+tQ9WocgA+JjNvFJl2pcuW7hldlbXFLSWFpV6MahFIqCat21TmcG9eojCwfySptKLEs+xZTVB/Q2J08ooVfcTqX2Nq/+A8N69TFFgnihZQX+3HGCWeudH0e9kXWj8ugdR8PnQFpg2Ui9LbkjPRJL0bBcGO/O32nY7mf2oIRecTtu4NUDPFw/msSowrw5d5vzV83aPHrVHNyBFI62efUTDevVCyH4sHMcPmbB8z9uctsQjhJ6RdZUag9FKhvaqzeZBB91iSPdKnnt561ODeFYLUronUKj5w3v1RcvmI/X28aydv8Ztw3hKKFXZI3JpNWrT91j2AwcgNKh+XmuRXkW7TzBr1uOOu06Unn0zqFQlJaBY2CvHrRVsw3LhfHBgl261ly6W+wSeiFEKyHELiFEshBiaBb7/YUQ0237Vwshom7ZX0oIcVEI8byD7Fa4gortDO/VA/StF0VcREGGz93G2cvOqav/n9Cr6pUO50as3pgZOKCFcN69vyoWF7w9OoMchV4IYQZGA/cBsUB3IUTsLcP6A2eklDHACOCDW/Z/CvyWd3MVLuWGV58MW4y5Wha0LJz3O8Vx5nIa7/y6wzkXUaEb55HZqzfgatkMIgsHuuTt0RnY49EnAslSyn1SyuvANKDDLWM6AJNsn2cCzYUQAkAI0RH4F3DtUkaFY6jYTlst+/cHhqyBk0FsiQI82qgMP65LYXmy43vNSiX0zqXh87a8euNVtsxM5rfHc5fdpxaOPUJfEshcai7Fti3LMVLKdOAcECqECAJeAt7M7gJCiIFCiCQhRNLJk57b5cUtMZmgyctweq8hK1tm5qnm5SgdGshrc7ZyPd3B5YxVjN65FCoNNXppefUGrGyZQcbb4+lL1/n4d/07n9mLsydjhwMjpJQXsxskpRwnpUyQUiaEh4c72SRFrqnYBorFaatlDdaFKjMBvmaGt6/MvpOXGL/sX8eeXMXonU/D57TiegbrLXsrsSUK0LtuFN+vPsCWlHN6m2MX9gj9YSAy0/cI27YsxwghfICCQCpQG/hQCLEfGAK8IoR4Mm8mK1yOEND0Fa1e/SZj9Za9laYVitAitiijFu1xaHaEyrpxASGRULO3VtnyjLHTGJ+5tzyh+f15bc5Wt8itt0fo1wLlhBDRQgg/oBsw95Yxc4E+ts+dgcVSo6GUMkpKGQWMBN6VUn7hGNMVLqV8K1tv2Q8h3TmZLY7itbaxSCRv/7rdcSfNyDpSQu9cGjwLwqxVUDUwBfP58krrimw8dJYf1xk31JRBjkJvi7k/CSwEdgAzpJTbhBBvCSHa24aNR4vJJwPPArelYCrcHCGg6TCt3+fGKXpbky2RhQN5smkM87cc4589DprzUR69ayhYEhL6wcYftN4IBub+GiWpFVWI93/b6bS0XkdhV4xeSjlfSlleSllWSvmObdvrUsq5ts9XpZRdpJQxUspEKeVtT0hKOVxK+bFjzVe4lJh7IKKW5m2luaZq5N0yoFEZSocG8r9ftjukz6xQQu86GjwDZl9tTsjACCF4q0MVzl1JY9QifVpc2otaGauwHyGg2atw/rBWS9zA+PuYGdqqIruPX2RGUkreT3gjdKMmY51OcDFIHACbp8NJY2e2VCpegK4JkUxetZ/9p4xb9EwJvSJ3RDeGqIaw9GO4fllva7KlVZVi1IoqxKd/7OLitTyuAZDKo3cp9YeAbyAseU9vS3Lk2Rbl8TWbeP+3nXqbckeU0CtyR0as/tIJWPu13tZkixCCV9vEcuridcYu2Zu3c6nQjWvJHwZ1HodtP8GxLXpbky1FggN4vHFZFmw7xpp/Xd+43h6U0CtyT+m6Wrx+2Ui4el5va7KlWmQIHaqX4Ot/9uUt3VJl3bieuk9CQEH46129LcmRRxqWoViBAN75dbsh0y2V0CvujqbD4MppWPWl3pbkyIutKgLwye+77/ocQi2Ycj35QqDeYNg1H1KS9LYmW/L5mXmxVQU2pZzjFwPWwVFCr7g7StaEim1hxRdw2ZivqxmUDMlHrzql+WlDCntPZrtI+46YpAULJi10pXAdtR+DwDBY9JbeluRIx+olqVA0mJF/7nZIppcjUUKvuHuavQrXLxq+EBXAY03KEuBrZuSfe+7uBDIdq1BhG5fjH6yVRvj3b9i3RG9rssVkEjxzb3n2nbzEzxuNVVtfCb3i7ilSCap1gzVfG7ppBEBYkD/96kfxy+Yj7DyW+3kFk7RgFSpsowsJD0OBCM2rN3gd+JaVi1KlZAE+W7SbNAN59UroFXmjyVBtotLgi1sABjQsQ5CfDyP+yH2sXkgLVpTQ64JvgNYX4fA62Pmr3tZkixCC51pU4NDpK/zoiPUbDkIJvSJvFIqC+L5aIarUvKUwOpuQQD8eaViGhduO57rqoLCmK49eT6r1gNBysPhtQ3c7A2hSPpz40oX4fPEerqYZw1Yl9Iq80+gFMPvBX+/obUmOPNwgipBAX0b+mTuv3oQFqYReP8w+0GwYnNyhrZg1MEIInru3PEfPXWVGkjEKnimhV+Sd4KLa4pats+DIRr2tyZbgAF/61oti0c4T7Dp2we7jTFYVo9edSh2geHUtr97gtZbqlg2lZqkQxi3dZ4gMHCX0CsdQ/2nIVwj+HK63JTnSp24U+XzNfLXU/lCT5tGrrBtdMZng3jfh3CFY+43e1mSLEILHGpcl5cwVQ/SXVUKvcAwBBbUQzr6/YO9iva3JlkL5/eiWGMncjUc4bOdqWZO0INViKf0p0wTKNoN/PoYrZ/W2JlvuqVSUmCJBjP17H1LnbCEl9ArHUesRKFhK8+qt+r+uZscjDcsAMP6fnFsOWqwSMyp0YxjuGQ5XzsDyz/S2JFtMJsHARmXYcfQ8S/c4vmF9rmzR9eoKz8LHX5swO7oJts3W25psKRmSj/bVSjB1zUHOXMq+aUSaxYoZqwrdGIXi1aBqF1g1xvDrNzpWL0mxAgGMWaJvvXol9ArHUrULFK2iLW5Jv6a3NdnyaOOyXEmz8N3K7PuTWqwSH1ToxlA0HaZ1/TJ4wTM/HxP9G0Szat9pNh46q5sdSugVjsVkhhb/g7MHtBWzBqZCsWCaVgjn+9UHsl3FmG6RyqM3GoWjofajsOF7OLZVb2uypXvtUgT7+zBxec5hQmehhF7heMo208oYL/3Q8AXPetUtzckL1/h92/E7jkm3WvHBoipXGo2Gz2lJAH+8rrcl2RLk70OnmiWZv+UYqRf1ectVQq9wDvf+D65d0PrLGpjG5YtQMiQf36+6c/gm3TYZK1UtemMRWBgavwh7F0Hyn3pbky0965TmusXKj+v0KYughF7hHIrGQo2HtPCNgUsjmE2CHrVLsXJfKsknsi5hnGax4iOsoLJujEetR7QyHL+/ZujSCOWLBpMYVZgfVh/UpTGJEnqF82g6DMy+8OcbeluSLV0TIvE1C6asztqrz0ivVN2lDIiPv5ZueWK7Vm/JwPSsU4qDpy+zdM9Jl19bCb3CeQQX05o875gH+5fpbc0dCQ/2p2XlYsxal8KV67d7hWkWiQ9WFboxKrEdIbK2VvDMwK0tW1UpRmh+P75fddDl11ZCr3Au9QZrtcR/G2roV+uH6pTm/NV05m26PS873WrFhBWhJmONiRDQ6j24dNLQc0L+Pma61opk8c7jdq/IdhRK6BXOxS9Qq09yfIuhX61rRxemXJEgflhzu7eVbpG2rBvl0RuWkvFaKeNVYww9J9QjsRQSmLHWtVUtldArnE+VByCyDiz6H1zNXR14VyGEoFPNCDYeOsvB1Ms37dOybqxK6I1O89e1ctm/v6a3JXcksnAgtaMLM2/TEZfWv1FCr3A+QsB978PlVEO/WreNKw7AvM03h28sN/LoldAbmgLFodFzsOtXQ/eXbV+tJPtOXWLbEdfNJyihV7iGEjWgRk/t1fpk7lv5uYLIwoHULBVyW5w+zaJl3Qgl9ManziAIKQ2/vQSWNL2tyZL7qhTDxySynA9yFkroFa6j+XDwzQ+/vWDYJs/tq5Vg57EL7D7+X1OSdFvWDWYl9IbHNwBavQ8nd8LqsXpbkyWF8vvRsFwYv2w+6rKceiX0CtcRFA7NXtVeq7fP0duaLGkdVxyT4CZvK91qxSwsCCX07kGF+6BcS1jyPpzXv+lHVrSrVoLDZ6+w/uAZl1zPLqEXQrQSQuwSQiQLIYZmsd9fCDHdtn+1ECLKtv1eIcQ6IcQW29/NHGy/wt1IeBiKVoWFw+D6Jb2tuY0iwQHULRt602RZhkevQjduQsackCUN/jDmxOy9sUXx9zG5LHyTo9ALIczAaOA+IBboLoSIvWVYf+CMlDIGGAF8YNt+CmgnpawK9AGMm1+ncA1mH2jzMZxPgaUf621NlrSLK8H+1MtsOaxlCKVbrSpG724ULqO1t9zyoyEX6wUH+NKsYhF+3XLUJT1l7fHoE4FkKeU+KeV1YBrQ4ZYxHYBJts8zgeZCCCGl3CClzPiVtQ3IJ4Twd4ThCjemVB2o1h1WfA4nduptzW20qlIMX/N/k2Xptnr0KnTjZjR4BkJKwS/PQnr2zWX0oH21Epy6eJ1V+5xf4dUeoS8JZM7uT7Fty3KMlDIdOAeE3jLmAWC9lNLY3SgUrqHF2+AfBL8MMVzbwZBAPxqVC2f+lmNIKW/Uo1crY90Mv0Bo/Qmc2gUrjNd2sGnFIuT3MzN/q/PnEVwyGSuEqIwWznn0DvsHCiGShBBJJ0+6vuCPQgfyh2mljA+uhI3f623NbTSpWITDZ6+wP/WyVr0SCyYfX73NUuSW8i20Wjh/f2S4FbMBvmbqlg1jmQv6ydoj9IeByEzfI2zbshwjhPABCgKptu8RwE9Abylllv/SUspxUsoEKWVCeHh47u5A4b7UeAhK19dWMl401i/4BjFhACxLPmWrXmnFpEI37sl9H2hVLn95xnBpvQ1iQjl4+jKHTl/OeXAesEfo1wLlhBDRQgg/oBsw95Yxc9EmWwE6A4ullFIIEQL8CgyVUi53kM0KT0EIaDtSy75Z+LLe1txEVGggJUPysSL5FGkWK75CTca6LcHF4J434N+/YfN0va25iQblNIdiebJzvfochd4Wc38SWAjsAGZIKbcJId4SQrS3DRsPhAohkoFngYwUzCeBGOB1IcRG258iDr8LhfsSXl5rCbflR9i9UG9rbiCEoH5MKCv2ppKWng6AyaxCN25L/MMQkQgLXoaLJ/S25gZlw4MoWsCfZXoLPYCUcr6UsryUsqyU8h3bttellHNtn69KKbtIKWOklIlSyn227W9LKfNLKatn+mOcf2WFMWj4HBSJhXlDDFX0rH5MGOeupLH1YCoAwkd59G6LyQQdvoDrF2H+83pbcwPNoQhjxd5Up66SVStjFfrj46f9EF48pi2kMgj1ymqv1SuTtcbhZhWjd2/CK0CTodqq7G0/623NDRrEhHH60nV2HHNekTMl9ApjUDIe6j2l1azfu1hvawCt81TFYsFcvKJlBKvQjQdQ72koXl3z6i+l6m0NoL05gnPj9EroFcahycsQVh7mPmWYEE79mDCtXyyorBtPwOwDHb+EK2e14noGoGiBAMoVCWJZsvN+8SihVxgH3wDo8CWcP6yVmTUADWLCtMqVoFbGegpFK0Pjl2DrLNgyU29rAM2hWPNvKtfSndNuUwm9wlhE1oKGz8OmqYaIoyZGF8bfZFu5q9IrPYcGz0BELa08wrkUva2hfkwYV9OsrD9w1innV0KvMB6NX4QSNbXyCDqXmc3v70O1kkHaFyX0noPZBzqNA2s6/PSY7mU4apcpjNkknBanV0KvMB5mX+2HMO0qzHlC9x/CTtWLaR+U0HsWhctAq/dg/z+w6ktdTSkQ4EutqEKcv+qcrlhK6BXGJKwctHxHy8BZ+YWuptxTwVafTwm951GzN1RoA4vehMPrdTXlh0fq8FaHKk45txJ6hXFJeBhiO8Cfw+Hgav3ssGorY1HVKz0PIbQ1HPmLwI99tWwcnTCZhPPO7bQzKxR5RQho/zmERMLMh+Gy8+t2Z8kNoVcevUcSWBi6TNCyveYONlzhM0eghF5hbAIKQucJcPE4/Py4PvF6JfSeT2QiNH8ddsyFNV/rbY3DUUKvMD4la2rx+t0LYNknrr++1ZbbrITes6k7WGsqvvAVfUOFTkAJvcI9SBwIcQ/C4ndg1wLXXlvF6L0Dkwk6faWFCmf0gvOuadztCpTQK9wDIaDdZ1A8DmY9Aid3u+7aKnTjPeQrBN1+0HokTOuppfh6AEroFe6Dbz54cIrWLWhad9dlSCih9y6KVIL7v4Ij6w3ZlepuUEKvcC9CIuHByXDmAEx/CNJd0GteCb33UamtVmRv0w+w9GO9rckzSugV7kfpeloFwv3/wJxBzs/EuTEZq2L0XkXjl6Bad/jrbdj4g97W5Anloijck7iucO4QLHoLCkbAPcOddy3l0XsnQkC7UXDhqJZfH1wMyjbT26q7Qnn0CvelwbMQ3w+WjYCVo513HSX03ouPH3T9DsIqwPRekJKkt0V3hRJ6hfsiBLT+GCq113KfnbXQRQm9dxNQEB6aCfnDYHInOLJRb4tyjRJ6hXtj9oEHxkP5+7T2cOu/c/w11IIpRYES0GceBBSAyR3h+Da9LcoVSugV7o+PH3SdBGWba20IHS32asGUAiCkFPSZCz75YFJ7OLpZb4vsRgm9wjPw8YduU7TJsrmDtbi9o/KfVehGkUHhMjax94eJbWD/cr0tsgsl9ArPwTcfdJ8GVR7QShv//qpjUi+V0CsyE1YOHl4IQUXh+06wc77eFuWIEnqFZ+HjB52+gVoDtIYlsx7WlrPnBRWjV9xKSKQm9kVitYV7q8YYegWtEnqF52EyQeuP4J43tQbj41vAmf13fz4Vo1dkRf5QbYK2wn2wYKhWRjvtit5WZYkSeoVnIgQ0GAI9Z2oLq8Y1gT1/3N25VOhGcSf8g6DrZGjyCmyaChPug9P/6m3VbSihV3g25e6BAX9BcAmY0hnmDYFrF3N3DiX0iuwwmaDJS9BtKqTugzH1IelbQ4VylNArPJ/QsjBgMdQbDOsmwtj6sO9v+49XQq+wh4qt4YkVEFlLq3o5pbNWfM8AKKFXeAe+AdDibehny5D4rr1Wbzx1b87HZkzGChWjV+RAwQh46Cdtxfb+5fBFLS0D7Op5Xc1SQq/wLkrXgydWaf1B9y2B0Ynwy7NwKvnOx1jTQZi0V3SFIidMJkgcAIOToPL92pqOz2vCspFw5Yw+JtkzSAjRSgixSwiRLIQYmsV+fyHEdNv+1UKIqEz7XrZt3yWEaOlA2xWKu8M3HzR8DgavhxoPwYbJ8EU8/PAg7P799hr3VosK2yhyT8EIrTXhgL+0NMw/34BPK8Ovz2v1clwYwxcyh4sJIczAbuBeIAVYC3SXUm7PNOYJIE5K+ZgQohtwv5TyQSFELDAVSARKAH8C5aWUljtdLyEhQSYluWeFOIWbcuE4JI2HtePh8inwC4byLSDmXiheDTZ8D+smwLCjeluqcGeObtby7bf8CNY0KBgJFdtAdCMoFqf9YhDirk8vhFgnpUzIcp8dQl8XGC6lbGn7/jKAlPK9TGMW2sasFEL4AMeAcGBo5rGZx93pekroFbqRfk0L5+yYB7t+00Q/A79geCVFN9MUHsSlVNj9G+z4BfYuBovtDTJfIajeE1q+c1enzU7o7XkfLQkcyvQ9Bah9pzFSynQhxDkg1LZ91S3HlszCwIHAQIBSpUrZYZJC4QR8/KF8S+2P1QKndsOxLXB0EwQX19s6haeQP1QLGdZ4CK5f1iphHtukefwFbpNHh2CIwKOUchwwDjSPXmdzFAptFWyRStqfuK56W6PwVPwCtXTMyFpOvYw9k7GHgchM3yNs27IcYwvdFARS7TxWoVAoFE7EHqFfC5QTQkQLIfyAbsDcW8bMBfrYPncGFkst+D8X6GbLyokGygFrHGO6QqFQKOwhx9CNLeb+JLAQMAPfSim3CSHeApKklHOB8cBkIUQycBrtlwG2cTOA7UA6MCi7jBuFQqFQOJ4cs25cjcq6USgUityTXdaNWuqnUCgUHo4SeoVCofBwlNArFAqFh6OEXqFQKDwcw03GCiFOAnkp4hwGnMpxlGfhjfcM3nnf6p69h9zed2kpZXhWOwwn9HlFCJF0p5lnT8Ub7xm8877VPXsPjrxvFbpRKBQKD0cJvUKhUHg4nij04/Q2QAe88Z7BO+9b3bP34LD79rgYvUKhUChuxhM9eoVCoVBkQgm9QqFQeDgeI/Q5NTD3BIQQkUKIv4QQ24UQ24QQT9u2FxZC/CGE2GP7u5DetjoDIYRZCLFBCPGL7Xu0rRl9sq05vZ/eNjoSIUSIEGKmEGKnEGKHEKKuNzxrIcQztv/fW4UQU4UQAZ74rIUQ3wohTgghtmbaluXzFRqjbPe/WQhRMzfX8gihtzUwHw3cB8QC3W2NyT2NdOA5KWUsUAcYZLvPocAiKWU5YJHtuyfyNLAj0/cPgBFSyhjgDNBfF6ucx2fAAillRaAa2r179LMWQpQEngISpJRV0Eqjd8Mzn/VEoNUt2+70fO9D6+dRDq3t6pjcXMgjhB5IBJKllPuklNeBaUAHnW1yOFLKo1LK9bbPF9B+8Eui3esk27BJQEddDHQiQogIoA3wje27AJoBM21DPOq+hRAFgUZovR6QUl6XUp7FC541Wp+MfLZudYHAUTzwWUspl6L178jMnZ5vB+A7qbEKCBFC2N3I2FOEPqsG5s7psmsQhBBRQA1gNVBUSnnUtusYUFQvu5zISOBFwGr7HgqclVKm27572jOPBk4CE2zhqm+EEPnx8GctpTwMfAwcRBP4c8A6PPtZZ+ZOzzdPGucpQu9VCCGCgFnAECnl+cz7bC0cPSpnVgjRFjghpVynty0uxAeoCYyRUtYALnFLmMZDn3UhNO81GigB5Of28IZX4Mjn6ylC7zVNyIUQvmgiP0VKOdu2+XjGa5zt7xN62eck6gPthRD70cJyzdDi1yG213vwvGeeAqRIKVfbvs9EE35Pf9b3AP9KKU9KKdOA2WjP35OfdWbu9HzzpHGeIvT2NDB3e2xx6fHADinlp5l2ZW7O3geY42rbnImU8mUpZYSUMgrt2S6WUvYE/kJrRg8edt9SymPAISFEBdum5mi9lz36WaOFbOoIIQJt/98z7ttjn/Ut3On5zgV627Jv6gDnMoV4ckZK6RF/gNbAbmAvMExve5x0jw3QXuU2Axttf1qjxasXAXuAP4HCetvqxH+DJsAvts9lgDVAMvAj4K+3fQ6+1+pAku15/wwU8oZnDbwJ7AS2ApMBf0981sBUtHmINLQ3uP53er6AQMss3AtsQctKsvtaqgSCQqFQeDieErpRKBQKxR1QQq9QKBQejhJ6hUKh8HCU0CsUCoWHo4ReoVAoPBwl9AqFQuHhKKFXKBQKD+f/fdbBLhqtUmwAAAAASUVORK5CYII=", + "text/plain": [ + "
    " + ] + }, + "metadata": { + "needs_background": "light" + }, + "output_type": "display_data" + } + ], + "source": [ + "#hide_input\n", + "\n", + "scheduler = PolyLRScheduler(optimizer, t_initial=t_initial, cycle_limit=2)\n", + "plot_lr(scheduler, label='PolyLR')\n", + "\n", + "scheduler = CosineLRScheduler(optimizer, t_initial=t_initial, cycle_limit=2)\n", + "plot_lr(scheduler, label='CosineLR')\n", + "plt.legend();" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Using `PolyLRScheduler` scheduler with `timm` training script" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "To train models using the `PolyLRScheduler` we simply update the training script args passed by passing in `--sched poly` parameter alongside the necessary hyperparams. In this section we will also look at how each of the hyperparams update the `PolyLRScheduler` scheduler. " + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "The training command to use `PolyLRScheduler` scheduler looks something like: \n", + "\n", + "```python \n", + "python train.py ../imagenette2-320/ --sched poly\n", + "```" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Availible parameters are: \n", + "\n", + "--epochs - initial number of epoch to train. default 300. \n", + "--lr - learning rate (default: 0.05) \n", + "--min-lr - lower lr bound for cyclic schedulers that hit 0 (1e-5) \n", + "--lr-k-decay - 'learning rate k-decay for cosine/poly (default: 1.0) \n", + " \n", + "--decay-rate - polynomial power, (default: 0.1) \n", + "\n", + "cycle parameters: \n", + "--lr-cycle-limit - learning rate cycle limit, cycles enabled if > 1 \n", + "--lr-cycle-decay - amount to decay each learning rate cycle (default: 0.5) \n", + "--lr-cycle-mul - learning rate cycle len multiplier (default: 1.0) \n", + "\n", + "warmup parameters: \n", + "--warmup-lr' - warmup learning rate (default: 0.0001) \n", + "--warmup-epochs - epochs to warmup LR, if scheduler supports (default: 3) \n", + "\n", + "noise parameters: \n", + "--lr-noise - learning rate noise on/off epoch percentages \n", + "--lr-noise-pct - learning rate noise limit percent (default: 0.67) \n", + "--seed - random seed (default: 42) to seed noise generator. \n" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "> Note! PolyLRScheduler is cyclyc sheduler, so real number of train epoch will differ from --epochs number! \\\n", + "If we lunch script with default settings, it will train for 310 epochs - 300 defaupt for --epochs and 10 default for --cooldown-epochs. \\\n", + "If we lunch script with parameters: \\\n", + "--epochs 50 --lr-cycle-limit 2 \\\n", + "It will be 110 epochs - two cyles by 50 epochs plus 10 for cooldown." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Using in python script." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "`PolyLRScheduler` accepts two required arguments - an `optimizer` and `t_initial`, and also some hyperparams which we will look into in detail below." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Basic usage like this:" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "\n", + "```python \n", + "from timm.scheduler.poly_lr import PolyLRScheduler\n", + "scheduler = PolyLRScheduler(optimizer, t_initial=num_epoch)\n", + "```\n" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Required arguments." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "`optimizer` is object of `torch.optim.Optimizer` \n", + "`t_initial` - initial number of epochs to train. It will be different from `t_initial` when using cycle arguments, see detailed explanation and examples below." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Default schedule:" + ] + }, + { + "cell_type": "code", + "execution_count": 10, + "metadata": {}, + "outputs": [ + { + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXoAAAD4CAYAAADiry33AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/YYfK9AAAACXBIWXMAAAsTAAALEwEAmpwYAAAieElEQVR4nO3dd3hVVb7G8e8vHUIogRAhhYRiAekh9CKMig0sKIIiioo46jg6zgxz79w7d5yrM1znDqJiV8QKiA11kKFIVyA06b0lQOgJLaSt+0eOXiYTJZCyT07ez/Pw5Oy9V3J++5nje9astffa5pxDREQCV5DXBYiISMVS0IuIBDgFvYhIgFPQi4gEOAW9iEiAC/G6gOIaNGjgkpKSvC5DRKRKWb58+SHnXExJx/wu6JOSkkhLS/O6DBGRKsXMdv3YMQ3diIgEOAW9iEiAU9CLiAQ4Bb2ISIBT0IuIBLhSBb2Z9TezTWa21cxGl3C8l5mtMLN8MxtU7NhwM9vi+ze8vAoXEZHSOWfQm1kwMB64BmgJDDGzlsWa7QbuBt4v9rvRwB+AzkAq8Aczq1f2skVEpLRK06NPBbY657Y753KBScDAsxs453Y6574DCov97tXATOfcEefcUWAm0L8c6v4Xp3ML+OPn6/gu/RhaellE5P+V5oapOGDPWdvpFPXQS6Ok340r3sjMRgIjARITE0v5p//Zmows3luymwmLdtKiYS0GdYznpvZxNKwdcUF/T0QkUPjFZKxz7lXnXIpzLiUmpsQ7eM8pNTmaZf/+M56+qTVRESH8efpGuvx5NsPfXMrnq/eSk1dQzlWLiFQNpenRZwAJZ23H+/aVRgbQp9jvzi3l7563OjVCGdo5kaGdE9l28AQfr0jn4xUZPPLBSmpHhHBD28bcmpJA2/g6mFlFlSEi4lfsXOPZZhYCbAb6URTcy4Chzrl1JbR9C/jCOTfVtx0NLAc6+JqsADo654782PulpKS48lzrpqDQ8c22w0xdvofpa/dzJr9QQzsiEnDMbLlzLqXEY6WZuDSza4FngWDgTefcU2b2JJDmnJtmZp2AT4B6QA6w3znXyve7I4B/8/2pp5xzE37qvco76M+WnZPHl9/tY+rydJbvOkqQQZ9LGnJrx3j6XRZLWIhfjGSJiJy3Mgd9ZarIoD/b9oMnmLq8aGhnf3YO9WqGcmP7OG7tmEDLxrUr/P1FRMqTgv4nFBQ6Fmw5yIfL05m5LpPcgkJaNa7NbSkJDGzXmLo1wyqtFhGRC6WgL6Vjp3L5bNVePly+h7UZ2YQFB3FVq1gGd0qge7MGBAVpAldE/JOC/gKs25vFh2npfLoqg2On8oirW4NBHeMZ1DGehOiaXpcnIvJPFPRlkJNXwKwNmUxetoeFWw/hHPRo3oDbOiVwVctYIkKDvS5RRERBX14yjp1malo6U9L2kHHsNHVrhnJjuzgGd0rgskaawBUR7yjoy1lhoWPRtkNMWrbnhwnctvF1GNwpkQHtGlMr3O8exSsiAU5BX4GOnMzlk5UZTF62m82ZJ6gZFsz1bRpxe2oi7RPq6g5cEakUCvpK4Jxj5Z5jTF66h8+/28up3AIuiY1icKcEbu4Qp8s0RaRCKegr2fGcPL74bh+Tlu5mdXoWYSFBXHv5RQxJTSQ1OVq9fBEpdwp6D63fm82kZbv5ZEUGx8/k0zQmkiGdErmlYzzRkerli0j5UND7gdO5BXy5Zh8fLN3N8l1HCQsO4urLL2JoaiJdmqqXLyJlo6D3M5v2H+eDpbv5eEU62TlFvfyhqYnc0iGeeurli8gFUND7qe97+e8v2cWK3cd+GMsf2rkJnZLqqZcvIqWmoK8CNu7P5oMlu/l4ZQbHc/Jp0bAWd3RO5KYO8dSpEep1eSLi5xT0Vcip3Hy+WL2P95bsYnV6FhGhQQxo25g7OjehbUJdr8sTET+loK+i1qRn8f7SXXy6ci+n8wpoHVeHOzoX3X1bM0x334rI/1PQV3HZOXl8tjKDd77dxebME0RFhHBLh3ju7NKE5g1reV2eiPgBBX2AcM6xbOdR3v12F9PX7iOvwNG1aX3u7NKEq1rFEhqsRyGKVFcK+gB08PgZpqTt4f0lu8k4dpqGUeEMSU1kaOdEYvXAc5FqR0EfwAoKHXM3HeDtb3Yxb/NBQoKMq1tdxLCuTeis5RZEqo2fCnrN6FVxwUFGv8ti6XdZLDsPneS9JbuYkpbOl2v2cXFsLYZ1TeLm9nFEaulkkWpLPfoAdDq3gM9X7+Xtb3eyNiObqPAQBqXEM6xLE5rGaPJWJBBp6Kaa+n7p5LcX7+TLNUWTt70ujmF41yb0uaQhwXrYuUjAUNALB4+fYdLS3by7ZBeZ2WdIiK7B8K5J3JqSoDtvRQKAgl5+kFdQyD/WZTJx8U6W7jxCjdBgbu4Qx93dkmgRG+V1eSJygRT0UqJ1e7OYuHgnn67aS25+Id2b1+fubsn0vVTDOiJVjYJeftKRk7l8sHQ37367i31ZOSRG1+Surk24rVMCtSM0rCNSFSjopVTyCwqZsS6TCYt2kLbrKJFhwQzqGM/d3ZNJbhDpdXki8hMU9HLe1qRnMWHRDj7/bi95BY4rLonh3h5N6d68vm7CEvFDCnq5YAeO5/Det7t5b8kuDp3I5ZLYKEb0SGJguzgiQoO9Lk9EfBT0UmY5eUU3Yb2xcAcb9x+nfmQYd3RO5M6uTWgYpbV1RLymoJdy45zjm+2HeXPhDmZvPEBIkDGgbRz39Uzmska1vS5PpNrSWjdSbsyMbs0a0K1ZA3YcOsmERTv4MC2dj1ak06N5A+7tmUzvFjEE6fJMEb+hHr2U2bFTuby/dDcTF+8kM/sMzRvW4r4eydzYXuP4IpVFQzdSKXLzC/lyzV5eX7CDdXuzqR8Zxl1dkxjWtQnRkWFelycS0BT0Uqm+H8d/fcEO5mw8QERoEIM6xnNvj6a6Hl+kgmiMXirV2eP4WzKP8/qCHUxZls57S3ZzVctYRvZqSscm0V6XKVJtlOoho2bW38w2mdlWMxtdwvFwM5vsO77EzJJ8+0PNbKKZrTGzDWb2u3KuX/xci9goxgxqw8LRV/BQn+Z8u/0It7z0DYNeWszM9ZkUFvrX/6MUCUTnDHozCwbGA9cALYEhZtayWLN7gaPOuebAWGCMb/+tQLhzrjXQEXjg+y8BqV4aRkXwxNWX8M3v+vJfN7RkX1YO97+dxpVj5zF52W7O5Bd4XaJIwCpNjz4V2Oqc2+6cywUmAQOLtRkITPS9ngr0s6L75B0QaWYhQA0gF8gul8qlSqoZFsLd3ZOZ9+s+PDekPRGhwfz2ozX0GPM1L87dSnZOntcligSc0gR9HLDnrO10374S2zjn8oEsoD5FoX8S2AfsBv7qnDtS/A3MbKSZpZlZ2sGDB8/7JKTqCQkOYkDbxnzxSA/eu68zl14Uxf98tYluf57Dn6dv4EB2jtcligSMip6MTQUKgMZAPWCBmc1yzm0/u5Fz7lXgVSi66qaCaxI/YmZ0b96A7s0bsDYji1fmb+e1+duZsHAnt3SMY2SvZrpSR6SMStOjzwASztqO9+0rsY1vmKYOcBgYCnzlnMtzzh0AFgElXv4jcnlcHZ4f0p6vn+jDrSnxfLQig77/O5efv7ecNelZXpcnUmWVJuiXAS3MLNnMwoDbgWnF2kwDhvteDwLmuKIL9HcDfQHMLBLoAmwsj8IlcDWpH8lTN7Vm0W/78mDvZizYfIgbXljIsDeW8O32w/jbvR8i/q5UN0yZ2bXAs0Aw8KZz7ikzexJIc85NM7MI4B2gPXAEuN05t93MagETKLpax4AJzrlnfuq9dMOUFJedk8e73+7izYU7OHQilw6Jdfl5n+b0u6yh1sYX8dGdsRIQcvIKmJK2h1fmbSfj2GkuvSiKB/s04/o2jfWMW6n2FPQSUPIKCpm2ai8vzdvG1gMnSG4QyYO9m3Fj+zjCQkp1D6BIwFHQS0AqLHTMWLefF77eyrq92cTVrcGo3k25NSVBq2ZKtaOgl4DmnGPupoM8P2cLK3YfIyYqnJE9m3JHl0Rqhmk5J6keFPRSLXy/auYLc7ayeNthoiPDuL9nU4Z1bUKtcAW+BDYFvVQ7aTuP8NycrczffJC6NUO5r0cyd3VLonZEqNeliVQIBb1UW6v2HOP52VuYvfEAtSNCuKd7MiN6JFOnhgJfAouCXqq9tRlZPDd7C/9Yn0lURAgjFPgSYBT0Ij7r92bz3OwtfLVuvwJfAoqCXqSYfwr88BDu6ZHMvQp8qcIU9CI/ongP//6eTbmnexJRmrSVKkZBL3IO6/Zm8eysLcxcn0ndmqGM7NWU4V2TiNRlmVJFKOhFSum79GOMnbmZrzcdJDoyjFG9mzKsSxI1wnSnrfg3Bb3IeVqx+yhjZ25mwZZDxESF80jf5gzulEB4iAJf/JOCXuQCLd1xhL/O2MTSnUeIq1uDR/u14OYOcYQEa/E08S8/FfT6tIr8hNTkaCY/0IW3R6RSv1YYv/noO64aO59pq/dSWOhfnSSRH6OgFzkHM6PXxTF89lB3XhnWkdDgIH7xwUque34hX288oCdeid9T0IuUkplxdauLmP5oT8bd3o6TZ/K5561l3PbKNyzbecTr8kR+lIJe5DwFBRkD28Ux6/He/OnGy9l5+BS3vvwN9761jA37sr0uT+RfaDJWpIxO5ebz1uKdvDx3G8fP5DOwbWN+ddUlJETX9Lo0qUZ01Y1IJcg6lcdL87YxYdEOnIM7uzTh4b7NiY4M87o0qQYU9CKVaF/WaZ6duYUPl+8hMiyEUX2aMaJ7sm66kgqloBfxwJbM44z5ahOzNmTSMCqcx668mFs7xusafKkQuo5exAMtYqN4fXgKH47qSkJ0TX738RquGbeAORszdUmmVCoFvUgF65QUzdRRXXn5zg7kFRQy4q00hr62hLUZWV6XJtWEgl6kEpgZ/S9vxD8e681/3dCSjfuzuf75hTw2eRUZx057XZ4EOI3Ri3ggOyePl+Zu442FOwC4r0cyP7+iObW0LLJcII3Ri/iZ2hGh/Lb/pXz9RB+ua92IF+duo88zc5m0dDcFWkNHypmCXsRDcXVrMHZwOz59qDtJ9Wsy+uM1XPfcAhZtPeR1aRJAFPQifqBdQl0+HNWV8UM7cOJMPne8voT7Ji5j+8ETXpcmAUBBL+InzIzr2jRi1uO9GX3NpXy7/QhXPzufp75cT3ZOntflSRWmoBfxMxGhwYzq3Yyvn+jDze3jeX3hDvr+dS6Tl+3WGvhyQRT0In4qJiqcMYPaMO2hHiTVj+S3H61h4PhFpGlJZDlPCnoRP9c6vg4fjurKuNvbcfD4GQa9/A2PTlpJZnaO16VJFaGgF6kCzIrWwJ/zRG8e6duc6Wv30/evc3ll3jZy8wu9Lk/8nIJepAqpGRbCr666hFmP9aZrswb8efpG+o+bz/zNB70uTfyYgl6kCkqsX5PXh6cw4e5OFBY67npzKQ+8k8aeI6e8Lk38kIJepAq74tKGzHisF7+++hLmbz7ElWPnMf7rrRrOkX9SqqA3s/5mtsnMtprZ6BKOh5vZZN/xJWaWdNaxNmb2jZmtM7M1ZhZRjvWLVHvhIcE8dEVzZv+qN30ubsgzMzZxzbj5LNbdteJzzqA3s2BgPHAN0BIYYmYtizW7FzjqnGsOjAXG+H43BHgXGOWcawX0AXTnh0gFaFy3Bi8P68iEuzuRV+AY+voSfvHBSg7o6pxqrzQ9+lRgq3Nuu3MuF5gEDCzWZiAw0fd6KtDPzAy4CvjOObcawDl32DlXUD6li0hJrri0If94rBe/6NeCr9bup9//zmPCoh1aLK0aK03QxwF7ztpO9+0rsY1zLh/IAuoDFwPOzGaY2Qoz+01Jb2BmI80szczSDh7U1QMiZRURGszjV17MjMd60S6xLn/8fD03vbhIDzuppip6MjYE6AHc4ft5k5n1K97IOfeqcy7FOZcSExNTwSWJVB/JDSJ5e0Qqzw9pz95jOQx4YSH//cV6Tp7J97o0qUSlCfoMIOGs7XjfvhLb+Mbl6wCHKer9z3fOHXLOnQL+DnQoa9EiUnpmxg1tGzP78d7cnprI6wt3cNXY+czekOl1aVJJShP0y4AWZpZsZmHA7cC0Ym2mAcN9rwcBc1zRo6tmAK3NrKbvC6A3sL58SheR81GnZihP39SaqaO6EhkezL0T03jw3eWarK0Gzhn0vjH3hykK7Q3AFOfcOjN70swG+Jq9AdQ3s63A48Bo3+8eBf5G0ZfFKmCFc+7Lcj8LESm1lKRovnikJ7+++hLmbDxAv7/NY9LS3fjbY0Wl/OiZsSLV2I5DJxn90Xcs2XGErk3r8+ebW5PUINLrsuQC6JmxIlKi5AaRfHB/F56+qTVrM7LoP24+r87fRn6B7qwNJAp6kWouKMgY2jmRmY/3pmeLGJ7++0ZuenExG/Zle12alBMFvYgAcFGdCF4d1pHxQzuwL+s0A15YyLhZW8hT777KU9CLyA++f27tzMd6c83ljRg7azMDX1jE+r3q3VdlCnoR+Rf1IsN4bkh7Xr6zIweOn1HvvopT0IvIj+p/+UXMfKwX17VR774qU9CLyE+qFxnGuNvb88qwot79wPELeXHuVi2SVoUo6EWkVK5udRH/eKwXV7aM5X++2sRtr3zDzkMnvS5LSkFBLyKlFh0ZxvihHRh3ezu2ZB7n2ucW8N6SXbqr1s8p6EXkvJgZA9vFMeOxXnRIrMe/f7KWe95apjVz/JiCXkQuSKM6NXh7RCpPDmzFt9sPc9Wz8/lq7X6vy5ISKOhF5IIFBRl3dU3i77/oSUK9mox6dzm/+/g7TuVqvXt/oqAXkTJrGlOLjx7sxoN9mjFp2R6uf24ha9L1NCt/oaAXkXIRFhLEb/tfynv3deZUbgE3v7SIl+dto1CXYXpOQS8i5apbswZ89cue/OyyWP4yfSN3vL6ETE3UekpBLyLlrm7NMF68owNjbmnNqj3HuGbcAuZuOuB1WdWWgl5EKoSZMbhTIp8/0p2YWuHcPWEZf5m+UevleEBBLyIVqnnDKD57uDtDOyfy8rxtDH7lG9KPnvK6rGpFQS8iFS4iNJinb2rN80PasznzBNeOW8CMdbrmvrIo6EWk0tzQtjFf/qIHTepH8sA7y/nTF+s1lFMJFPQiUqma1I9k6oNdGd61CW8s3MHQ177VVTkVTEEvIpUuPCSYPw68nHG3t2NtRjbXPbeQb7Yd9rqsgKWgFxHPDGwXx2cPd6d2jRDueP1bXp63TSthVgAFvYh46uLYKKY93INrWjfiL9M38sA7y8nOyfO6rICioBcRz9UKD+GFIe35z+tbMmfjAW58YRFbD5zwuqyAoaAXEb9gZozokcz793ch63QeN41fxOwNmV6XFRAU9CLiV1KTo5n2SA+aNKjJfW+nMf7rrRq3LyMFvYj4nbi6NfjwgW4MaNuYZ2Zs4uH3V2qN+zJQ0IuIX6oRFsyzg9vxb9deyvS1+7j5xcXsOaKlEy6Egl5E/JaZMbJXMybck8reY6cZOH4Ry3cd8bqsKkdBLyJ+r/fFMXz6UHfq1AhlyGtL+GxVhtclVSkKehGpEprG1OLjB7vRLqEuj05axbOzNmuStpQU9CJSZdSLDOPdeztzS4d4np21hV9OXkVOXoHXZfm9EK8LEBE5H2EhQfz11jY0jYnkmRmbSD96mleHdaR+rXCvS/Nb6tGLSJVjZjx0RXPGD+3A2owsbnpxMTsOnfS6LL+loBeRKuu6No2YNLILJ87kc8tLi1m955jXJfklBb2IVGntE+sxdVRXaoYFM+S1b/UQ8hKUKujNrL+ZbTKzrWY2uoTj4WY22Xd8iZklFTueaGYnzOyJcqpbROQHTWNq8fHPu5FUP5L7Jqbx8Yp0r0vyK+cMejMLBsYD1wAtgSFm1rJYs3uBo8655sBYYEyx438Dppe9XBGRkjWMimDyA11ITY7m8Smrtbb9WUrTo08FtjrntjvncoFJwMBibQYCE32vpwL9zMwAzOxGYAewrlwqFhH5EVERoUy4pxM3tG3MX6Zv5Mkv1lNYqLAvzeWVccCes7bTgc4/1sY5l29mWUB9M8sBfgtcCfzosI2ZjQRGAiQmJpa6eBGR4sJDghk3uB0xtcJ5c9EOsk/nM+aW1oQEV98pyYq+jv6/gLHOuRO+Dn6JnHOvAq8CpKSk6OtXRMokKMj4j+svo06NUMbO2kxOfgHPDm5HaDUN+9IEfQaQcNZ2vG9fSW3SzSwEqAMcpqjnP8jM/geoCxSaWY5z7oWyFi4i8lPMjEd/1oKaYcE89fcNnMkr4IWhHYgIDfa6tEpXmq+3ZUALM0s2szDgdmBasTbTgOG+14OAOa5IT+dcknMuCXgWeFohLyKV6f5eTfnTwFbM2nCA+yamVct17c8Z9M65fOBhYAawAZjinFtnZk+a2QBfszcoGpPfCjwO/MslmCIiXhnWNYlnBrVh8bZDDH9zKcer2cPHzd8uP0pJSXFpaWlelyEiAejz1Xt5bPIqWjWuzcQRqdStGeZ1SeXGzJY751JKOlY9ZyZEpFq6oW1jXr6zIxv2HeeuN5eSXU169gp6EalWftYylpfu7MD6vdnc/eZSTpwJ/DF7Bb2IVDv9LovlhaHtWZ2exYi3lgX8BK2CXkSqpf6XN2Ls4Hak7TzC/W+nBfQDTBT0IlJtDWjbmGcGtWXxtsOMenc5Z/IDM+wV9CJSrd3SMZ6nb2rN3E0Hefj9leQVFHpdUrlT0ItItTckNZEnB7Zi5vpMHp+yOuAWQtMzY0VEgLu6JnHyTAFjvtpIbFQ4v7+++GrsVZeCXkTEZ1TvpmRm5/D6wh1cVCeC+3o29bqkcqGgFxHxMTP+4/qWHDiew39/uYGYqHAGtovzuqwy0xi9iMhZgoOMv93WjtTkaJ74cDWLth7yuqQyU9CLiBQTERrMa8NSSG4QyQPvLGfd3iyvSyoTBb2ISAnq1Axl4ohUoiJCuHvCMvYcOeV1SRdMQS8i8iMa1anBxBGpnMkr4J63llXZ5Y0V9CIiP+Hi2CheHtaRHYdO8tjkqnmNvYJeROQcujVrwO+vu4xZGzJ5dvYWr8s5bwp6EZFSuLtbEoM6xvPc7C18tXaf1+WcFwW9iEgpmBn/fePltE2oy+NTVrNp/3GvSyo1Bb2ISClFhAbz6rCORIaHcP/baRw7let1SaWioBcROQ+xtSN4+c6O7M/K4ZEPVpJfBVa7VNCLiJynjk3q8acbW7FgyyHGfLXR63LOSUEvInIBBndK5K6uTXhtwQ5mrc/0upyfpKAXEblA/37dZbRsVJvffPQdB47neF3Oj1LQi4hcoPCQYJ4b0o6TZ/J54sPv/PZmKgW9iEgZNG8Yxe+vb8n8zQd5a/FOr8spkYJeRKSM7uycSL9LG/KX6RvZsC/b63L+hYJeRKSMzIwxg9pQu0Yov5y0ipy8Aq9L+icKehGRctCgVjh/vbUNmzKP85fp/nXJpYJeRKSc9LmkIXd3S+KtxTv5etMBr8v5gYJeRKQcjb7mUi6JjeLXH67m8IkzXpcDKOhFRMpVRGgw44a049ipPL+5a1ZBLyJSzi69qDYjeiQzJS2dFbuPel2Ogl5EpCL8ol8LYmuH85+fraXA4xupFPQiIhWgVngIv7+uJWszsnl/6W5Pa1HQi4hUkOvbNKJbs/o889VGTydmFfQiIhXEzPjjgFacyi3wdGJWQS8iUoFaxEZ5PjFbqqA3s/5mtsnMtprZ6BKOh5vZZN/xJWaW5Nt/pZktN7M1vp99y7l+ERG/9/3E7H986s3E7DmD3syCgfHANUBLYIiZtSzW7F7gqHOuOTAWGOPbfwi4wTnXGhgOvFNehYuIVBXfT8yu25vN+0t2Vfr7l6ZHnwpsdc5td87lApOAgcXaDAQm+l5PBfqZmTnnVjrn9vr2rwNqmFl4eRQuIlKV/DAxO2NTpU/Mlibo44A9Z22n+/aV2MY5lw9kAfWLtbkFWOGc+5czNLORZpZmZmkHDx4sbe0iIlXG9xOzJ3MLeHHutkp970qZjDWzVhQN5zxQ0nHn3KvOuRTnXEpMTExllCQiUulaxEZxQ5tGTFq6m6xTeZX2vqUJ+gwg4azteN++EtuYWQhQBzjs244HPgHucs5V7teYiIifGdmrGSdzC3i3EsfqSxP0y4AWZpZsZmHA7cC0Ym2mUTTZCjAImOOcc2ZWF/gSGO2cW1RONYuIVFktG9em18UxTFi0s9IeUHLOoPeNuT8MzAA2AFOcc+vM7EkzG+Br9gZQ38y2Ao8D31+C+TDQHPhPM1vl+9ew3M9CRKQKGdWrKYdOnOHTlcUHRyqGOedfTy1PSUlxaWlpXpchIlJhnHMMeGERJ8/kM+vx3gQFWZn/ppktd86llHRMd8aKiFQyM+OB3k3ZfugkMzdkVvj7KehFRDzQv9VFJEbX5OV526jokRUFvYiIB0KCg7i/ZzIrdx8jbVfFroGjoBcR8cigjglER4bxyryKvfJcQS8i4pEaYcHc1bUJszYcYEvm8Qp7HwW9iIiH7uqaRERoEK/O315h76GgFxHxUHRkGINTEvh0VQb7s3Iq5D0U9CIiHruvZ1MKCh0TFu2okL+voBcR8VhCdE2Gd0uiUZ2ICvn7IRXyV0VE5Lz84YZWFfa31aMXEQlwCnoRkQCnoBcRCXAKehGRAKegFxEJcAp6EZEAp6AXEQlwCnoRkQDnd48SNLODQFkej94AOFRO5VQlOu/qReddvZTmvJs452JKOuB3QV9WZpb2Y89NDGQ67+pF5129lPW8NXQjIhLgFPQiIgEuEIP+Va8L8IjOu3rReVcvZTrvgBujFxGRfxaIPXoRETmLgl5EJMAFTNCbWX8z22RmW81stNf1VBQze9PMDpjZ2rP2RZvZTDPb4vtZz8saK4KZJZjZ12a23szWmdmjvv0Bfe5mFmFmS81ste+8/+jbn2xmS3yf98lmFuZ1rRXBzILNbKWZfeHbri7nvdPM1pjZKjNL8+274M96QAS9mQUD44FrgJbAEDNr6W1VFeYtoH+xfaOB2c65FsBs33agyQd+5ZxrCXQBHvL9bxzo534G6Oucawu0A/qbWRdgDDDWOdccOArc612JFepRYMNZ29XlvAGucM61O+v6+Qv+rAdE0AOpwFbn3HbnXC4wCRjocU0Vwjk3HzhSbPdAYKLv9UTgxsqsqTI45/Y551b4Xh+n6D/+OAL83F2RE77NUN8/B/QFpvr2B9x5A5hZPHAd8Lpv26gG5/0TLvizHihBHwfsOWs73bevuoh1zu3zvd4PxHpZTEUzsySgPbCEanDuvuGLVcABYCawDTjmnMv3NQnUz/uzwG+AQt92farHeUPRl/k/zGy5mY307bvgz7oeDh5gnHPOzAL2mlkzqwV8BPzSOZdd1MkrEqjn7pwrANqZWV3gE+BSbyuqeGZ2PXDAObfczPp4XI4XejjnMsysITDTzDaeffB8P+uB0qPPABLO2o737asuMs2sEYDv5wGP66kQZhZKUci/55z72Le7Wpw7gHPuGPA10BWoa2bfd9QC8fPeHRhgZjspGortC4wj8M8bAOdchu/nAYq+3FMpw2c9UIJ+GdDCNyMfBtwOTPO4pso0DRjuez0c+MzDWiqEb3z2DWCDc+5vZx0K6HM3sxhfTx4zqwFcSdH8xNfAIF+zgDtv59zvnHPxzrkkiv57nuOcu4MAP28AM4s0s6jvXwNXAWspw2c9YO6MNbNrKRrTCwbedM495W1FFcPMPgD6ULRsaSbwB+BTYAqQSNESz7c554pP2FZpZtYDWACs4f/HbP+NonH6gD13M2tD0cRbMEUdsynOuSfNrClFPd1oYCVwp3PujHeVVhzf0M0Tzrnrq8N5+87xE99mCPC+c+4pM6vPBX7WAyboRUSkZIEydCMiIj9CQS8iEuAU9CIiAU5BLyIS4BT0IiIBTkEvIhLgFPQiIgHu/wANRsjox+cEOwAAAABJRU5ErkJggg==", + "text/plain": [ + "
    " + ] + }, + "metadata": { + "needs_background": "light" + }, + "output_type": "display_data" + } + ], + "source": [ + "#collapse_input open\n", + "scheduler = PolyLRScheduler(optimizer, t_initial=50)\n", + "plot_lr(scheduler)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## PolyLR specific Args" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### `power`" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "\"Power\" of polynomial function, default is 0.5.\\\n", + "Note, when you start training script, `power` sets by -`-decay-rate` parameter, that default is 0.1\\\n", + "When `power=1` annealing is linear. \n", + "Lets look at default and compare with 1. and 2." + ] + }, + { + "cell_type": "code", + "execution_count": 11, + "metadata": { + "scrolled": true + }, + "outputs": [ + { + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXoAAAD4CAYAAADiry33AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/YYfK9AAAACXBIWXMAAAsTAAALEwEAmpwYAABcrUlEQVR4nO3ddXxV5R/A8c+z3lgXSxgwcsToRkK6RCkFARGRMrB+ttiFiISEoCgiiGDQSneHdIwB627Wu8/vj3OZxIAB2+7d3fN+vfbajXPv+Z7F95z7nO/5PkJKiaIoimK6zAwdgKIoilK6VKJXFEUxcSrRK4qimDiV6BVFUUycSvSKoigmzsLQAdzM3d1dBgQEGDoMRVGUcuXw4cMJUkqPop4zukQfEBDAoUOHDB2GoihKuSKEuHK759TQjaIoiolTiV5RFMXEqUSvKIpi4lSiVxRFMXEq0SuKopi4YiV6IUQPIcQ5IUSIEOL1Ip7vIIQ4IoTIF0IMvOm5kUKIC/qvkSUVuKIoilI8d030QghzYDbQE6gHPC6EqHfTYmHAKOCXm17rCrwHtARaAO8JIVwePGxFURSluIpTR98CCJFShgIIIZYB/YHT1xaQUl7WP6e76bXdgY1SyiT98xuBHsDSB478JjEhp9n80ShqB/aheu16WDg5Yu6ofZld+25vjzBTo1WKolQsxUn0vkD4dfcj0I7Qi6Oo1/revJAQYiwwFqBKlSrFfOsb/btjOcH70jHbt5TY2y0kBGYODv/tAJwcMXdwxNxJvzNwctaeu3bf0QlzZydtWQcHtZNQFKVcMoorY6WU84H5AM2aNbuvmVC6j57CorpOfHtiIeNi0wiO92R+Zk9ycaS5mwUNnc2pZiMxy0inID0NXWoaBWlp5MRdpCAtFV1qGjI39/YrEKLwk4G5k1Phl5nTtfvO2nfn/54zd9Z2HMLK6n42SVEUpUQUJ9FHAv7X3ffTP1YckUDHm167rZivvWcjW73Ihfx4vrZezUz7ZL5Nn8NGtyd5K+FhElLBNtOcjrU96FHfi851PHGwsbzh9brsbApS09ClpVKQmkpBWtqN91PT9I9r9/MiIvTLpILu5lGr/5jZ2WHmrCV+C2dnzK7tBK59v/Z1/X0nJ/UJQlGUEiHuNpWgEMICOA90QUvcB4EnpJSnilh2EbBGSrlCf98VOAw00S9yBGh6bcy+KM2aNZMP0usmOz+bEetHEJEexlKL6lQ9vRbpXocTTT9geawPf5+KJT49BytzM9rVdKdnfS+61quMs939H3VLnQ5dRoaW9FNSKUhNoSAlRdsppKSgS03VHr/usYKUFArS0m6/gzAz0z49XEv8Li7/fXfRHrO44TEXbedgbn7f26EoSvklhDgspWxW5HPFmTNWCNELmA6YA99LKT8WQnwAHJJSrhJCNAf+AFyAbCBGShmkf+1o4E39W30spfzhTut60EQPEJURxZA1Q3CzceOX2qOxW/8GpEVC86fRdX6XI7EFrD8Zw4aTMUSmZGFhJmhdw42e9b3pFlQZd3vrB1p/cUmdDl16+q07AP1XfnLyf/eTr31PRubkFP2GQmg7B1fX/5K/izMWLvr7ri7azsHVFXMXVyxcXTCzsyuTbVUUpXQ9cKIvSyWR6AH2Ru1l3KZxdKnSha9aTUFs/Rj2zwMHb+g9Fer0RkrJichU1p+MYf2JaC4nZmImoEU1V3o18KZHkBeejjYlsFUlS5eVRYF+J5CfnKztBJKT9Y8l//dYUhIFydp98vOLfC9hY6PtAFzd9DsCV8zd3LBw0+8M3Fwxd3XFwlV73MzG+H4eiqJU0EQP8MPJH5h2eBqTm05mdP3REHEIVj0Pcaegbj/o9SU4eAEgpeRsTDrrT0Sz7mQMIXEZCAHNq7rSs4EXPet74+VUPpOclFIbWkpKIj8pSb9j0N9OStYeT06iIPG/77f71GBmZ4e5u3th4te+u2Lh5o6Fu5t+J6F9mTk5IYQo461VlIqpwiZ6KSWv7niVjVc2MufhObTxaQMFebBnBmz7HCxsoOv70GQk3HTi80JsOmtPRLP+RAznYtMBaFrVhV4NvOnVwAtvJ9sSidEYSSmRmZn6HUES+YlJ5Ccm6HcKieQnJhV+z09MpCApqehzDZaWhUnfwt0dc3c3LNw9sHB3x8LDHQs3N22n4eGBWaVKaqegKA+gwiZ6gMy8TIatG0ZcZhzLei/D31FfQJR4EVa/AJd3QpU20Pcb8KhV5HuExGWw/kQ0a09EczamYiX94pA6nTaMlJBAQWIi+QmJ2o7hutv5CQkUJCSSn5gIBQW3vIewsdHvADz+++553W0PDyw8PTF3dVXVSIpShAqd6AHC08N5fO3jeNh68HOvn6lkWUl7Qko4+jP88zbkZUKHV6Hti2Bx+wqc0PgM1p2IZu2JGM5EpwEq6d+LG3YKCdoOID/+2vf4/77Hx6NLS7v1DczNb0n+FpU9sfT01G57emJRuTLmzs7qE4JSoVT4RA//nZzt5N+JaR2nYSauOyrMiIP1/4NTv4NHXeg3A/xb3PU9i0r6zQNc6N3Am14NvI3yRG55osvO1j4RxMdpyT8uvnAnkB8XV/i9IDn5ltcKS8vCpK/tCCpjUbkyll6VsfDywrJyZSw8PNTFbIrJUIle76dTP/HloS+ZEDyB8Y3G37rA+b9hzUv6Uswx0OVdsHEs1ntfjM9g3fFo1hyP5lxsOkJAiwBX+jT0pmcD7zIr2ayIdLm5FMTHkxcXR35sHPlxseTHxZEXG0d+bCz5sbHkxcUhs7JufKEQmLu7YVnZCwuvylhW9sLS2wsLL2/te2UvLCt7Iiwti16xohgRlej1pJS8testVoeu5ptO39C5SudbF8pJhy0fXVeK+RXU6XVP67kQm86a49GsOR7FxfirmAloU8OdPg296VHf64EuzlLuj5QSXXo6eTEx2s4gNoa8mNj/vsfEkBcTgy49/cYXCoGFhweW3t5Y+Hhj6e2Dpbc3lj7e+u8+qrpIMQoq0V8nOz+bpzY8RWhqKEt6LSHQJbDoBa8vxazXH3p+UViKWVxSSs7FprPmXy3pX07MxMJM0L6mO30a+tAtqPItbRgUwyrIuEp+TDR50THkxUSTH63tAPKio8iPiiYvOvqWnkhmdnZY+vpg4eODpf7LytcXSz8/LH19tRPIakeglDKV6G8SczWGoWuGUsmyEr/0/gUna6eiFyzIg93fwPYvtFLMbh9A4xG3lGIWh5SSk5FprDkexZrj0USmZGFlYUan2h70a+RL5zqe2Fqp9gXGTkpJQVISeVHR5EVFFfmlS0294TXCxgZLX18sfX2w8vPXdgD+flj5+WHp74+5vb2BtkYxJSrRF+FY3DGe+vspWni1YHaX2ViY3aG/W0IIrHlRK8Ws2lYrxXSved/rllJyJCylMOnHp+dgZ2VO13qV6dfIh/Y1PbCyUCWE5VVBRgZ5kVHkRUbe8JUbEUFeRMQtw0PmTk5YVqmClb8/llX8te/+/lhVqYKFp6cqJ1WKRSX62/j9wu+8t+c9htcdzv9a/O/OC0sJRxfrSzGz9aWYL9yxFLM4CnSS/ZcSWf1vFOtOxJCalYeTrSU963vRL9iHltXcMDdTH/tNSUFqKrnhWtLPiwjXboeHkxseTl5U1A3XGQgbGy3xV62CVdWqWFWpilXVKlgFBKidgHIDlejv4PMDn/PzmZ+Z0noKj9V67O4vSI+FDf+DU3+AZz3oOwP8m5dILLn5OnaFxLPqWBT/nI4lM7cATwdr+jbyoX+wDw181Uk/Uyfz8siLjiY3LJy88DByw8LJvXKF3LAr5IWF33B+QNjaask/IACrgKpYVQ3AuloAVtWqYe50m+FIxWSpRH8H+bp8Jm2exP6Y/SzotoCmlZsW74XnNsDalyAtClo8o5ViWjuUWFxZuQVsOhPLqn+j2HYujrwCSYCbHf2CfXkk2IfqHmpct6KRBQXkx8Zqif/KFXIvXSb3sv4rIuKGTwLm7u5YV6uGVbVqWFWvhnX16ljXqIGFt7f6FGCiVKK/i7TcNIatHUZqTipL+yzF1/6W2Q6LlpMOmz+EA/PB0Ucrxazds8TjS83MY8OpaP48GsW+S4lICQ18negf7EO/Rj7qwiwFmZdHbniElvQvhZITGkpu6CVyQkNvODks7OwKk75VYA2sawRiXasmlj4+agdQzqlEXwyXUy/zxLon8KrkxeKei/9rk1Ac4Qdh9fMQdxrqPaIvxaxcKnHGpGaz5ngUfx6L5GRkGmYCWtdw45FgX3rU91LlmsoNpJQUJCeTe/EiORdDybl4kdyLIeSEXCQ/Lq5wOTM7O6xqBmJdsyY2NWtiXbMm1rVrY+HmZsDolXuhEn0x7Ynaw4RNE+jg14Hpnabf2CbhbvJzYc83sP1LsLSBrh9CkxFQimPqIXEZrDoWyZ/HoghLysTawoyH61VmQLAvHWqpyh3lzgrS0sgJuUhOyAVyLoSQc/48OefP39BSwtzDHZtatbGuUxubOnWwrl0b62rV1NXCRkgl+nuw5MwSPjvwGWMajOGFJi/c+xskhGhdMa/sgqrt9KWYt7koq4RIKTkansKfRyNZczyapKu5ONtZ0qehNwMa+9Kkios6iasUW35iIjnnz5N97hw5Z8+Rff4cuRdCkHl5AAgrK6xr18ambl1s6tXDJqge1rVqYWat2nwYkkr090BKyft732flhZV80u4T+tboe+9votNppZgb39FKMR96Fdo8eClmceQV6Nh5IZ4/jkax8XQM2Xk6qrrZ8UiwLwMa+xLgfg9DUoqiJ/PyyLl0iZxz58g+c5bs06fJPn36vw6j5uZYBwZiUz8I2wYNsGnQAJtatdSRfxlSif4e5RXkMW7TOI7GHWVBtwU0qdzk7i8qSnosrH8NTv+plWL2mwl+Rf4eSkVGTj4bTsbwx9EI9lzUTuI2ruLMo4196dPQB5dKqueOcv+klORFRpF9+pSW+E+eIvvECQr0J3+FlRXWdetgW78BtsGNsA0OxtLPT326LCUq0d+H1JxUhq8bTmpOKkt6L8Hfwf/+3+zcelj7sr4Ucyx0eadESzGLIzo1i7+ORfHHkUjOxaZjaS7oXMeTR5v40am2pxrPV0qElJK8iAiyT5wgS5/4s06dQmZmAmDu5oZtcDC2jRppyb9hQzUPcQlRif4+XUm7whNrn8Dd1p2fe/2Mg9UDJOfsNNjyIRz4Dhx99aWYPUou2GKSUnI6Oo0/jmgncRMycnCxs6RvIx8ebeJHIz91UZZSsmRBATkXLpB17BhZR4+RdewYuVeuaE9aWmIbFIRd82bYNm2KXZMmmDsWrzW4ciOV6B/AwZiDjP1nLC28i9ETpzjCD2hdMePPQNAArRTT3rNkgr1H+QU6dl5IYOWRCP45HUtuvo4aHpV4rKkfAxr7qtmylFKTn5ysJf7Dh8k8dJisU6cgLw+EwLpWLexatKBS61bYNW+OuUPZfvotr1Sif0DXeuIMqT2Et1q+9eBHvPm5sHs67PgSLG2h20fQ+MlSLcW8m9SsPNafiGblkQgOXk5GCGgX6M7Apn50q+elOmsqpUqXlUXWv8fJPHyIzEOHyDp6DJmdDWZm2NSvT6WWLanUuhW2TZqooZ7bUIm+BEw7NI0fTv3A6y1eZ1jdYSXzpvHntVLMsD0Q0F4rxXSrUTLv/QCuJF5l5ZFIVh6OIDIlC3trC/o09GZgUz+aVlWlmkrp0+XmknXsGJn79nN13z6yjh+H/HyEtTV2LVpg3749ldq3wyogQP096qlEXwIKdAVM3jaZ7RHbmdl5Jh38OpTMG+t0cPQn+OddyM+Gh17TumKaG74sTaeT7L+UxMojEaw7EU1mbgHV3CsxsKkfjzZRQztK2dFdvUrm4cNk7NrF1Z27yL10CQBLf38t6XdoT6XWrSt0Lb9K9CUkMy+TURtGcTntMj/2+JG6bnVL7s3TY/SlmH+BZ5A2QXkZlmLezdWcfNadiOa3wxEcuJSEmYB2NT30QzuVsbFUQztK2ckNDydj506u7tjJ1f37kVlZmNnZUalDBxy6dMG+40MVbmxfJfoSFJ8ZzxPrnkCn07Gk9xK8Kt3b9IJ3dXYtrH0F0qOh5bPQ+e0yL8W8myuJV1l5OIKVRyKJTMnC0caCRxr7MriZP/V9VXtcpWzpcnLIPHCA9E2bSd+ymYL4BLC0pFKLFjh0fRiHbt2wcHU1dJilTiX6EnY++Twj1o/A196Xn3r+dG8N0IojOw02fwAHF2ilmH2mQa3uJbuOEqDTSfZcTGT5oXA2nIohN19HPW9HBjfzo3+wr7ogSylzUqcj699/ydi8mfSNm7QyTnNzKrVtg1OfPth37oK5vWleHa4SfSnYE7mHCZsn0MqnFbM6z3rwssuihO3XumLGn4X6j0GPzwxWink3qZl5rPo3kuWHIjgRmYqVuRldgyoztLk/bWu4Y6ZmyVLKmJSSnPMXSFu7lrQ1a8iLikLY2ODQuROOffpg3769SbVoUIm+lKw8v5Ipe6cwuNZg3m71dumc/b+hFNNOX4o53KClmHdzOiqN5YfC+fNYJCmZefi52DKoqT+Dmvnh46xO4CplT+p0ZB07RtqaNaSt30BBcjLm7u44P/oozoMGYuX/AFe+GwmV6EvR9MPTWXhyIS83fZlR9UeV3oriz2tH92F7jaoU806y8wr453Qsyw+GsyskASGgQ00Phjb35+F6lbE0V20XlLIn8/LI2LWLlN9WkLFtG+h0VGrbFufBg3Ho3KncHuU/cKIXQvQAvgHMgQVSys9uet4a+AloCiQCQ6SUl4UQlsACoAlgAfwkpfz0Tusqb4leJ3W8tuM1/r78N1899BXdArqV4sp0cGQRbHwPCnK1Usw2zxtFKebdhCdl8tuhcJYfiiAmLRt3eysea+rH0OZVqKY6aioGkhcTQ8rKlaSsWEl+dDTm7u64DB6My5PDsXBxMXR49+SBEr0Qwhw4D3QFIoCDwONSytPXLTMBaCilHCeEGAoMkFIOEUI8AfSTUg4VQtgBp4GOUsrLt1tfeUv0ADkFOTzzzzOcSjjFd92+u/9ul8WVFg3rX4Uzq6FyfW2Ccr9iznVrYAU6yY7z8fxyIIwtZ+Mo0ElaVXfl8RZV6B7kpco0FYOQBQVk7NxJytJlZGzfjrCxwfmxx3B96ims/Io5taiBPWiibw1MkVJ2199/A+D6I3MhxN/6ZfYKISyAGMADGAo8AQwAnIC9QCspZdLt1lceEz1ASnYKT65/kqTsJBb3Wkx1p+qlv9Iza2DdK1oNfstx+lLM8jNpeGxaNisOR7DsYBjhSVk421nyWBM/Hm9RhUDP8rMdimnJCQkhceH3pK5eDVLi2KsXbmOexqZ2bUOHdkcPmugHAj2klGP0958EWkopJ123zEn9MhH6+xeBlkAqsBjoAtgBk6WU84tYx1hgLECVKlWaXrnW2a6ciUiPYPi64VibW7Ok9xLcbd1Lf6XZqbDpfTi0EJz8ofc0qFWKw0el4FqZ5tIDYfx9KoZ8naRFNVeGtVRH+Yrh5MXEkLToR1KWL0eXmYl9p054vvIy1jWM89yYIRN9bWACMApwAXYCPaWUobdbX3k9or/mVOIpntrwFAGOAfzQ44eSr7G/nbB9WlfMhHNGX4p5J/HpOaw4HMHSA2GEJWXioj/Kf6JlFap7qKN8pewVpKaS/MsvJC78Hl1WFs6DB+ExaZLRTZx+p0RfnLKHSOD62iM//WNFLqMfunFCOyn7BLBBSpknpYwDdgPGc11/KQhyC2LqQ1M5n3yel7e/TJ4ur2xWXKUVjNsJHd/Uxu5nNYejP4ORVVXdjYeDNeM71mDbKx35+emWtK7hxqI9l+n81Xae+G4f605Ek1egM3SYSgVi7uSE+/jx1Pjnb1yGDCFl+W9c7NadhPnfocvJMXR4xVKcI3oLtJOxXdAS+kHgCSnlqeuWmQg0uO5k7KNSysFCiP8BdaSUTwkhKulfO1RKefx26yvvR/TXXKuxHxA4gPfbvF+2Hfbiz+m7Yu6Fah2gz3SjL8W8k7j0bH47FMEv+8OITMnCw8Gaoc39GdqiCr6qLl8pYzmhocR9OZWMrVux8PHG8+WXcezVy+BdNEuivLIXMB2tvPJ7KeXHQogPgENSylVCCBu0sfjGQBJaMg8VQtgDPwD1AAH8IKX88k7rMpVEDzD72Gzm/juXcY3GMTF4Ytmu/JZSzP9Bm+fKRSnm7RToJNvPx7FkXxhbzsUhgM51PBneqiodanqoq2+VMnV1335iv/icnNNncOjaFa8P3jdoSaa6YMpApJS8u+dd/gz5k3davcPg2oPLPogbSjEbQL9vwLd8lGLeSURyJksPhPHrwXASMnKp6mbH8JZVGdjUT/XYUcqMLCggadEi4qZ/g4WzM96ffIJ9+3YGiUUlegPK1+XzwtYX2BW5i2kPTaNL1S6GCeRaKWZGrFaK2emtclWKeTu5+To2nIph8d7LHLycjLWFGX0b+fBkq6o08nc2dHhKBZF95gyRr75KbshFXIYPx/OVl8t8JiyV6A0sKz+LMf+M4WziWeZ1nUczLwOdj765FLPP11Czq2FiKQVnotP4ed8V/jgaSWZuAY38nRnZuiq9G3pjbaFKNJXSpcvOJm7aNJJ/WoxVjRr4fvkFNvXqldn6VaI3AinZKYzYMIKEzAQW9VxELZdahgvmhlLMgfpSTA/DxVPC0rPzWHk4gp/2XiE04SpulawY2sKfYS2rqqZqSqnL2L2b6DfeJD85Gd/PP8OxV68yWa9K9EYiOiOa4euHg4TFvRbjY+9juGDyc2DX17BjqjaE0+1jCH7CqLti3iudTrL7YgI/7b3C5jOxAHSr58WotgG0rOZq8CoJxXQVpKQQMek5Mo8cwfvDD3F+7NFSX6dK9EbkQvIFRm4YiZuNG4t7LsbZxtmwAcWf047uw/dBtYeg73RwLYP2DWUsPCmTn/df4deD4aRk5lHHy4FRbQLoH+yLrZUa1lFKni4ri4hJz3F1924qv/02rsOHler6VKI3ModjDzP2n7HUca3Dd92+w87SzrAB6XRw+AfYNEUrxez4OrSeVK5LMW8nO6+Av45F8sPuy5yNScfZzpKhzavwZOuqqiZfKXG63FwiJ79ExubNeLz8Eu7PPFNq61KJ3ghtCdvC5G2TaenVklldZmFlbgQlgWlRsO5VOLtGX4o5A3xLuROngUgp2X8piUW7L/PP6RiEEHQPqszottVoWtVFDesoJUbm5RH1+hukrV2L2/hxeDz/fKn8falEb6T+uPAH7+55l25Vu/FFhy8wNzOSIYTTq7SEfzUOWo6Hzm+Blen2jI9IzmTx3issPRBGWnY+Df2cGN22Gr0aeGNloSZHUR6cLCgg+r33SF2xEteRI/B8/fUST/Yq0RuxH0/9yNRDUxlUaxDvtHrHeI4ks1K0oZzDP4BTFX0p5sOGjqpUZebms/JIJD/sukRowlU8HawZ0boqT7Ssiqu6CEt5QFKnI/bTz0hevBiPyZNxf3Zsib6/SvRG7psj37DgxAKeafAMzzd53tDh3OjKXm0Kw4Tz0GCQVopZqQzaLxuQTifZfiGe73ddYueFBGwszXisiR+j21WjhuqgqTwAKSWRL04mY8sWqv31J9bVS67wQSV6Iyel5IN9H7Di/ApeafYKI4NGGjqkG+XnwM5psPMrrRSz+yfQ6HGTKsW8nXMx6Xy/6xJ/HIskN19HlzqePN2+Gq2ruxnPpy+lXMlPSOBi7z5Y16hB1Z8XI8xKZnhQJfpyoEBXwKs7XmXjlY181PYj+gf2N3RIt4o7o3XFDN8P1TtqwzkmWIpZlPj0HH7ed4XF+66QdDWXet6OPNOhGn0a+qhJzpV7lvLHn0S/8QaV33kb12ElU3apEn05kVuQy6TNk9gfs5+vHvqKh6sa4Zi4Tqe1UNj0PujyryvFtDB0ZGUiO6+AP49GsmDXJULiMvB2smF022oMbeGPg43plaMqpUNKSfiYZ8g6epTqq1dh6fvg89KqRF+OZOZlMnbjWE4lnmJW51m09W1r6JCKlhqpVeacWwteDbVSTJ/Gho6qzOh0km3n45i/I5R9oUk4WFvwRMsqPNW2Gl5OZdvMSimfciMiCe3XD7umTfGfP++BhwJVoi9n0nLTePrvp7mcepm5XefStLKRthWWEs5cK8WMh1YToNObJl2KWZTjESnM3xHKuhPRmAlB/2Bfnn2oOrUqOxg6NMXIJf20mNhPPsHn889w6v9gw7Uq0ZdDiVmJPPX3U8RlxrGw20KC3IMMHdLtZaXApvfg8CJwrqLNaBVooHbMBhSelMnCXZf49WA4WXkFdKnjybiONWge4Gro0BQjJQsKuDJsOLmXLlF93doHmodWJfpyKuZqDKM2jOJq3lV+6P4DgS6Bhg7pzi7v1k7WJl6AhkO06hwTL8UsSvLVXH7ae4VFey6RnJlH06oujHuoBl3qeKpZsJRb5ISEcGnAozh0fRjfadPu+31Uoi/HwtPCGblBK7f8sceP+Dv63+UVBpaXDbumaeWY1g7Q41Mt6VfAUsSs3AKWHwrnu52hRCRnEehpz/iHatAvWFXqKDeK//ZbEmbMxG/2LBy63N+nYZXoy7mQ5BCe+vspKllWYlGPRXhV8jJ0SHcXd0brihlxAKp30pdiVjN0VAaRX6Bj7Ylo5my7yNmYdHydbXn2oeoMbuaPjaWRtL1QDErm5nJp4CCsqlXD75vp9/UeKtGbgFOJpxjz9xhcbVxZ1GMRHnblYKKQm0sxO70BrSZWmFLMm0kp2XoujtlbL3L4SjLu9laMbleN4a2q4qhKMyu8vLg4LNzd7/sCKpXoTcSxuGOM3TgW70refN/9e9xs7//ETZlKjdTmqz23Tl+KORN8gg0dlcFIKTlwKYnZ2y6y43w8DjYWjGoTwFNtq6meOsp9U4nehByMOciETRPwd/Tn+27fG37ikuKSEk7/Betfq9ClmDc7GZnK7K0hrD8Zg52VOcNbVWVM+2p4OqhafOXeqERvYvZG7WXS5kkEugTyXbfvcLRyNHRIxZeVDBvfgyM/gnNVbey+ApZi3ux8bDrfbg1h1b9RWJqb8XiLKoztUF3NcasUm0r0JmhHxA5e2PoC9dzqMb/rfCpZlrMj41tKMT+FSuVkKKoUXUq4ypxtIfx+JBIhYFAzfyZ0rIGfi4FnIVOMnkr0Jmrzlc28vP1lGnk0Ys7Dcww/JeG9ysuGnVO1ScqtHSt0KebNIpIzmbPtIssPhQMwsKk/EzuphK/cnkr0JmzDpQ38b+f/aFa5GbO6zMLWohx+1I89rfW8jzgINTprwzkuAYaOyihEpWQxZ9tFfj0Yjk5KBjXzY0LHQPxdVcJXbqQSvYlbfXE1b+9+m+aVmzOzy8zymex1BXBwIWx+X7vd6U3thG0FLcW8WXSqlvCXHdAS/sCmfkzqHKiO8JVCKtFXAKsvruatXW/RwrsFMzuX02QPkBoBa1+B8+vBuxH0nVGhSzFvFpOazZxtISw9EI5EMqS5PxM7BeLtVE5/30qJUYm+glh1cRVv73qblt4tmdl5JjYW5bRET0o4/Sesew0yE6H1BOj4Jlipo9drolKymL01hOWHwhFC8ESLKkzoWANPx3L6O1cemEr0FchfIX/xzu53aOXdihmdZ5TfZA/6Usx34chPWilm3+naGL5SKDwpk1lbQlhxJAILM8GI1lUZ3zFQXXhVAd0p0RfrWlshRA8hxDkhRIgQ4vUinrcWQvyqf36/ECLguucaCiH2CiFOCSFOCCHKceYxfv0D+/NB2w/YF72PF7e+SE5BjqFDun+2LtpVtKPWgrklLB4Af4yDq4mGjsxo+Lva8fnAhmx5+SF6N/Rm4a5LdPhiK19vPE96dp6hw1OMxF2P6IUQ5sB5oCsQARwEHpdSnr5umQlAQynlOCHEUGCAlHKIEMICOAI8KaX8VwjhBqRIKQtutz51RF8y/rjwB+/teY/WPq35ptM35fvIHm4sxbRx0uruGw5WpZg3uRCbzrSN51l/MgYXO0vGd6zBiNYBqnlaBfCgR/QtgBApZaiUMhdYBtw8FUp/4Ef97RVAF6HNi9UNOC6l/BdASpl4pySvlJwBNQfwfpv3tatot0wiKz/L0CE9GEsb6Pw2PLtTm5D8j7Hw82OQfNnQkRmVmpUdmDO8KasmtaWBnzOfrDvLQ19uZcn+K+QV6AwdnmIgxUn0vkD4dfcj9I8VuYyUMh9IBdyAWoAUQvwthDgihHitqBUIIcYKIQ4JIQ7Fx8ff6zYotzGg5gA+bvdxYX+czLxMQ4f04CrXg9F/Q88vIXw/fNsa9syEgnxDR2ZUGvo589PoFiwb2wp/Fzve+uMk3b7ewdrj0RjbeTml9JX27AcWQDtgmP77ACHELY1NpJTzpZTNpJTNPDzKQfvdcqRvjb582u5TjsYdZfym8VzNu2rokB6cmTm0HAsT90O1DvDP27CgC0T/a+jIjE6r6m78Nq41C0c2w9JcMPGXIzwyezd7LiYYOjSlDBUn0UcC109r5Kd/rMhl9OPyTkAi2tH/DillgpQyE1gHNHnQoJV706t6Lz7v8DnH448zduNY0nPTDR1SyXDyg8eXwcAfIC0K5neCf96BXBP45FKChBB0qVuZ9S90YOqgRsSn5/DEd/sZ8f0BTkWlGjo8pQwUJ9EfBGoKIaoJIayAocCqm5ZZBYzU3x4IbJHa58O/gQZCCDv9DuAh4DRKmese0J2pD03ldOJpxv4zltQcE/kHFwLqPwqTDkDwE7BnBsxpDRe3Gjoyo2NuJhjY1I8tr3TkrV51OR6RQu8Zu3jp12NEpZTzczjKHRWrjl4I0QuYDpgD30spPxZCfAAcklKu0pdMLgYaA0nAUCllqP61w4E3AAmsk1IWOU5/jaq6KV3bwrfx0raXCHQOZF7XebjYuBg6pJJ1aafWFTPpIjR6XJug3M7V0FEZpdSsPOZsu8j3uy8hgKfbVWN8xxo4qNmuyiV1wZRyg12Ru3hx64v42fvxXbfvyse0hPciLwt2fAm7v9FKMXt8Bg0GqVLM24hIzmTq3+f481gUbpWsePHhmgxtUUVNYF7OqESv3OJgzEEmbp6Iu607C7otwMfex9AhlbyYk1pXzMjDUKOLvitmVUNHZbSOR6Tw8doz7L+URHWPSrzVqy6d63gi1A6yXFCJXinSv/H/Mn7jeCpZVWJBtwVUdTTBJKgrgAPfweYPAAmd3oKW41RXzNuQUrLpTByfrj9DaPxV2td0550+9ahV2cHQoSl3oRK9cltnEs/w7MZnMTcz57uu3xHoEmjokEpHSrg2Qfn5DeAdrLVW8G5o6KiMVl6BjsV7rzB903mu5hbwRIsqTO5aS/XQMWIq0St3dDHlIs/88wx5ujzmdp1LkFuQoUMqHVLCqd9h/f8gMwnaTIKHXlddMe8g+WouX286z5L9YVSyMueFh2vxZKuqWFmo8XtjoxK9cldhaWGM+WcM6bnpzOoyi6aVmxo6pNKTmQQb34GjP4NLNa0rZvWOho7KqJ2PTefDNafZeSGB6h6VmNI3iA61TOwkfjn3wN0rFdNXxbEKP/b4EXdbd57d+Cw7InYYOqTSY+cK/WfDyNVaJc5P/eGP8doOQClSrcoO/DS6BQtHNkOnk4z4/gDPLj5EeJK6OK08UEf0yg0SsxIZv2k8F5Iv8HG7j+lVvZehQypdeVmw/QvtQisbZ30p5kBVinkHOfkFLNh5iVlbQtBJybiHajC+Yw3VIdPA1NCNck/Sc9N5bstzHIk9wlst32JInSGGDqn0xZyEVc9B1BEI7Aq9v1KlmHcRlZLFJ+vOsOZ4NL7OtrzTpx7dgyqrckwDUUM3yj1xsHJg7sNz6eDXgY/2f8R3x78z/Y6HXvVhzCbtiP7KHvi2FeydrZVnKkXycbZl1hNNWPpMK+ytLRj382Ge/lEN5xgjdUSv3FaeLo93dr/D2tC1jKw3kpebvVwxjtZSwmDty3DhH/BprJViejUwdFRGLb9Ax6I9l/l643nydZLnOgfyTIfqWFuo4ZyyooZulPumkzo+O/AZS88upV+NfkxpMwVLswrQC0VKOLkSNryuL8V8Djq+Dpa2ho7MqEWnZvHhmtOsOxFDdY9KfNS/Pm0C3Q0dVoWgEr3yQKSUzDs+j9nHZtPOtx1fPfQVdpYVpPY8M0lrfXxMlWLei63n4njvr1OEJWXySLAPb/eph7u9taHDMmkq0Ssl4rfzv/HRvo+o71afWV1mmV7nyzsJ3a51xUy+BMHDoNtHqivmXWTnFfDt1hDmbL9IJWsL3upVl4FN/SrG8J8BqESvlJjNYZt5bftr+Nj7MK/rPNNshnY7eVmw/XPYPQNsXaDn51D/MVWKeRchcem8vvIEh64k0zbQjU8GNKCqWyVDh2VyVKJXStShmEM8v+V5bC1smdN1DrVcahk6pLIVcwJWPf9fKWafaeBcxdBRGTWdTvLLgTA+X3+W3AIdk7vWYky7alioVsglRpVXKiWqmVczFvVcBMCo9aM4FFPBdsxeDbRSzO6faqWYs1vB3m9VKeYdmJkJhreqysaXHqJjbQ8+W3+WfrN2czLSRGY6M3Iq0Sv3pZZLLRb3Woy7nTtjN45lw6UNhg6pbJmZQ+sJMHEfVG0Df78BCx7WjvaV2/JysmHek82YO7wp8Rk59J+9m6/+OUdOvtpJliaV6JX75mPvw+Kei2ng3oBXd7zKopOLTP/Cqps5V4Fhv8FjC7X6+/kdYdMUbTxfua0e9b3YOLkD/YN9mLklhH4zd3MiQh3dlxaV6JUH4mTtxPxu8+ke0J2vDn/FJ/s/oaCiDWEIofXHmXQQGg6BXV/DnDZapY5yW852VkwbHMzCkc1IycrlkW938+XfZ9XRfSlQiV55YNbm1nzR4QtGBY1i2bllTN42maz8CnhEa+cKj3wLI/7SLrj6qR/8OVF1xbyLLnUr88+LDzGgsS+zt16k78xdHI9IMXRYJkUleqVEmAkzXm72Mm+0eINt4dsY8/cYkrIraIKr3hHG74G2L8K/S2F2CzixQkv+SpGc7CyZOqgRPzzVnLSsfB79dg8zN18gv0Bn6NBMgiqvVErc5rDN/G/H//Cw9WD2w7Op7lTd0CEZTvRxbYLyqKNQs7vWFdPZ39BRGbXUzDze+eskq/6NonEVZ74eHEyAu6q7vxtVXqmUqS5VuvB99+/JzM9k+Lrh7I/eb+iQDMe7IYzZDN0/gcs7ta6Y++aqUsw7cLKzZMbjjZnxeGMuxmXQ85ud/LI/rOKd6C9BKtErpaKhR0N+6f0Lle0qM27jOP648IehQzIcM3NoPREm7IMqrWDD/2BhV4g9ZejIjFq/Rj78PbkDTau68OYfJ3j6x0PEpWcbOqxySSV6pdT42vvyU8+faOHdgnf3vMvXh79GJyvwmKtLVRi2Ah5dAMlXYF4H2PwB5KnkdTveTrb8NLoFU/rWY3dIAj2n72TbuThDh1XuqESvlCoHKwdmd5nN4FqD+f7k97yy/ZWKWZFzjRDQcJBWitlgMOz8SivFvLTT0JEZLTMzwai21VjzXDs8HKwZ9cNBPl57mtz8CnzQcI9UoldKnYWZBW+3eptXm73KpiubGL1hNHGZFfyozM4VBsyBJ/8EWQA/9oG/JkFWsqEjM1o1Kzvw58S2PNmqKt/tvMTAuXu4nHDV0GGVCyrRK2VCCMGIoBFM7zSdi6kXeXzN45xKUGPU1OgE4/dC2xfg2C8wq4U24Yk68VgkG0tzPnykPnOHN+VKYia9Z+zkz6ORhg7L6KlEr5SpzlU6s7jnYizMLBi5YWTF65FTFCs76PoBjN0Kjj6wYjT8MgRSwg0dmdHqUd+LdS+0p56PIy/+eoyXl/9LVq6qZLodleiVMlfbtTa/9P6FILcgXt3xKjOPzqzYJ2mv8W6klWJ2+1iVYhaDr7MtS59pxfNdavL70QgGfLubS2oop0gq0SsG4WbrxnfdvmNA4ADmH5/Py9teJjMv09BhGZ65BbSZBBP2gn9LfSlmN1WKeRsW5ma81LUWi55qQWxaNv1m7mLDyRhDh2V0ipXohRA9hBDnhBAhQojXi3jeWgjxq/75/UKIgJueryKEyBBCvFJCcSsmwMrcivfbvM9rzV9jS/gWRqwfQWSGGm8FwCUAhq+ER7/Tpi9UpZh39FAtD9Y8357qnvaM+/kwn6w7o9onXOeuiV4IYQ7MBnoC9YDHhRD1blrsaSBZShkIfA18ftPz04D1Dx6uYmqEEDxZ70lmd5lNVEYUQ9cMrdhX0l5PCGg4GCYdUqWYxeDrbMvyZ1vxZKuqzN8RyhML9hOXpnaMULwj+hZAiJQyVEqZCywD+t+0TH/gR/3tFUAXoZ8BWAjxCHAJUJ89ldtq59uOpX2W4mbjxtiNY/np1E/qkvdrCksx/1ClmHdhbaFV5Xw9pBHHI1LoPXMXh6+on1NxEr0vcP3p/wj9Y0UuI6XMB1IBNyGEPfA/4P07rUAIMVYIcUgIcSg+Pr64sSsmpqpjVZb0XkIn/058eehL3tj1Btn56oisUI3OWilmm+evK8X8XZViFmFAYz/+mtgOOytzHp+/j98OVewKptI+GTsF+FpKmXGnhaSU86WUzaSUzTw8PEo5JMWYVbKsxLSO05gUPIl1oesYsX4E0RnRhg7LeFjZQbcP9aWY3rDiKVg6FFIjDB2Z0ant5cBfE9vSvJoLr644zkdrTlOgq5g7xeIk+kjg+r6qfvrHilxGCGEBOAGJQEvgCyHEZeBF4E0hxKQHC1kxdWbCjGcbPcvMzjMJTw9nyJohatz+Zt6NYMwW6PYRXNoBs1vC/nmqFPMmznZWLHqqBaPaBLBg1yVGLzpIalaeocMqc8VJ9AeBmkKIakIIK2AosOqmZVYBI/W3BwJbpKa9lDJAShkATAc+kVLOKpnQFVP3kP9DLO29FBcbF8ZuHMvCEwvVuP31zC2gzXP6UswWsP41+L47xJ42dGRGxdLcjCn9gvj00QbsDklgwLe7CY2/4yCDyblrotePuU8C/gbOAMullKeEEB8IIfrpF1uINiYfArwE3FKCqSj3I8ApgKW9l9K1alemH5nOC1tfID033dBhGReXABj+OwyYD0mhMK89bP5QlWLe5PEWVVgypiUpmXn0n72b3SEJhg6pzKgZppRyQUrJz2d+ZtqhafjY+zCt4zRqu9Y2dFjG52oi/P0mHF8GboHQ9xsIaGfoqIxKeFImY348RGhCBl8ObMQjjW+uLSmf1AxTSrl3rd5+YfeFZOVnMXzdcFZfXG3osIxPJTd4dJ52hF+QB4t6w6rnISvF0JEZDX9XO5aPa03Tqi68+Osxvt0WYvJDgirRK+VKk8pNWN53OUHuQby5600+2PsBOQU5hg7L+AR20cbu2zwPR3/WJig/9acqxdRzsrXkx9Et6NfIhy82nOOdv06adEWOSvRKueNu68533b7jqaCn+O38bwxbO4wraVcMHZbxsaqklWI+swUcvOC3kbDsCUhVbSZAu7hq+pBgxj1Ug5/3hfHs4sMm2wFTJXqlXLI0s+SlZi8xs/NMoq9GM2TNENXy+HZ8gv8rxby4VSvFPPAd6FQvGDMzwes96/BB/yA2n43l8e/2kZhhep8QVaJXyrWO/h1Z0XcFNZxr8OqOV/lo30dqKKco10oxJ+4D/+aw7hWtFDPujKEjMwojWgcwZ1hTzkSnMXjeXmJNrEeOSvRKuedt782iHosYFTSKX8/9ypPrniQsLczQYRmn60sxE0NgbnvY8rEqxUSbzOSn0S2ISc1m8Ly9RCSbTttslegVk2BpZsnLzV5mZueZRGZEMnjNYNaErjF0WMZJCGg0ROuKWf8x2PEFzG0HV/YYOjKDa1ndjZ/HtCT5ai6D5+41mTlpVaJXTMq1oZzaLrV5Y+cbvLXrLa7mmcY/a4m7oRQzF37oCatfqPClmI2ruLB0bCuy83UMnreXC7Hl/wI9legVk+Nt783C7guZ0GgCa0LXMHj1YDUR+Z0UlmI+B0d+0k7Wnv6rQpdiBvk48evYVkhgyPx9nIpKNXRID0QlesUkWZhZMD54PN93/55cXS7D1w9n0clFam7a27GqpFXlPLMF7D1h+QhYNgzSogwdmcHUrOzA8mdbY2NhxuPz93EsPMXQId03legVk9a0clNW9F1BR7+OfHX4K8ZvGk9cZpyhwzJePo3hma3Q9UO4uEXreV+BSzGruVdi+bjWONtZ8eTC/ZyMLJ9H9irRKybPydqJaR2n8U6rdzgSe4RHVz3KxisbDR2W8TK3gLbPa8M5fk21UswfekDcWUNHZhB+LnYsHdsKB2sLRn5/gJC48tf5UiV6pUIQQjC49mCW912On70fL217ibd3vU1Gbvn7py0zrtXgyT/hkbmQcEGrzNn6CeRXvOsUfJ1t+XlMS4SAJxfuL3ellyrRKxVKNadqLO61mGcbPsvq0NUMXD2QI7FHDB2W8RICgh+HSQeh/qOw/XN9KeZeQ0dW5qp72LP46ZZczcln+IL9xKWXn2sPVKJXKhxLM0smNZ7Ejz1+RCB46u+nmHFkBnkFFW/moWKr5A6PzofhKyE/WxvKWf0iZJfPMev7VdfbkUWjWxCXnsOTCw6Qkplr6JCKpVz0o8/LyyMiIoLs7PKzB1XKB53UkZabRmZeJpZmljjbOGNpZmnosMqUjY0Nfn5+WFoWc7tzr2pDOPu+hUqe0OtLqNfv7q8zIbtDEnjqh4PU9XFkyZiW2FtbGDqkO/ajLxeJ/tKlSzg4OODm5oYQwkCRKaYsPTedqIwoCnQFuNu5427rjpkw/Q+8UkoSExNJT0+nWrVq9/biyCOw+nmIOQF1+mgJ39GndAI1Qv+cimH8kiO0CHDlx9EtsLIw7N9LuZ94JDs7WyV5pVQ5WDlQw7kGjtaOxGfGcyn1Etn5pv8JUgiBm5vb/X1a9m2ilWI+/D6EbNIutDq4oMKUYnYL8mLqoIbsDU3k7T9PGPXkJeUi0QMqySulzsLMAj8HP/wd/MnT5RGaGkp8ZrxR/wOXhAf63zK3hHYvaqWYPo1h7ctaK4UKUoo5oLEfz3UOZPmhCL7ffdnQ4dxWuUn0ilJWHK0dCXQOxMHKgbjMuApzdP9AXKvDiL/gkTmQcE5fivlphSjFnPxwLboHVebjtafZds44L8ZTid4EbNiwgdq1axMYGMhnn31W5DKLFi3Cw8OD4OBggoODWbBgwT2tY9GiRUyaNOmOy+Tk5PDwww8THBzMr7/+ek/vD7Bt2zb69OlTeHvPHsN1U7Qws8DP3g8/Bz9ydbmEpoQSlxmnWijciRAQ/ARMPAhBj8D2z7Q2yCZeimlmJpg2OJjaXo4898tRo7ygSiV6I5afn3/XZQoKCpg4cSLr16/n9OnTLF26lNOnTxe57JAhQzh27BjHjh1jzJgxJR0uR48eBeDYsWMMGTLkgd7L0IketCENJ2snAp0DC8fuQ1NDycwrXxfLlDl7D3hsAQxbAXlZWinmmskmXYpZydqCBSObYW1pxpgfDxpd2aXha4Lu0furT3E6Kq1E37OejyPv9Q267fOXL1+mR48eNG3alCNHjhAUFMRPP/2EnZ0dmzdv5pVXXiE/P5/mzZszZ84cjh8/zqeffsrvv//OX3/9xdChQ0lNTUWn01GvXj1CQ0O5ePEiEydOJD4+Hjs7O7777jvq1KnDqFGjsLGx4ejRo7Rt25Zp06bdMfYDBw4QGBhI9erVARg6dCh//fUX9erVe+Cfyw8//MCnn36Ks7MzjRo1wtraGoD4+HjGjRtHWJg2ucf06dOpWbMmw4cPJz4+nuDgYFauXMmSJUtYvXo1WVlZtGnThnnz5iGEoGPHjkydOpVmzZqRkJBAs2bNuHz58g0/77lz52Jubs7PP//MzJkzad++/QNvz/26NnbvZO1EVEYUl1Iv4WbrhoetB+Zm5gaLy+jV7KrNaHWtFPPceq0yp25fQ0dWKnydbZn3ZFMen7+fCUuO8OPoFliaG8extHFEUQ6cO3eOCRMmcObMGRwdHfn222/Jzs5m1KhR/Prrr5w4cYL8/HzmzJlD48aNOXbsGAA7d+6kfv36HDx4kP3799OyZUsAxo4dy8yZMzl8+DBTp05lwoQJheuKiIhgz549TJs2ja1btxYOt1z/1aZNGwAiIyPx9/cvfK2fnx+RkUVP/rxy5UoaNmzIwIEDCQ8Pv+P2RkdH895777F792527dp1w6eEF154gcmTJ3Pw4EFWrlzJmDFj8PT0ZMGCBbRv355jx45Ro0YNJk2axMGDBzl58iRZWVmsWVO8iUACAgIYN24ckydP5tixYwZN8tdzsHIg0DkQFxsXErMSuZh6kfTc8t+rvFRZVYLuH8OYzWDnDr8ON+mumE2ruvLJow3YczGRD1YX/cnaEMrdEf2djrxLk7+/P23btgVg+PDhzJgxg65du1KtWjVq1aoFwMiRI5k9ezYvvvgiNWrU4MyZMxw4cICXXnqJHTt2UFBQQPv27cnIyGDPnj0MGjSo8P1zcv47aTVo0CDMzbUjxU6dOhXuNB5E3759efzxx7G2tmbevHmMHDmSLVu23Hb5/fv307FjRzw8PABt2Of8+fMAbNq06YbEn5aWRkbGreOSW7du5YsvviAzM5OkpCSCgoLo27d8H82Zm5njY+9TeHQflhaGo7UjXnZeWJpXrAut7olvExi7FfbOgm2faaWYD78HTUeDmWkdbw5s6seF2HTm7QilSVVnBjT2M3RI5S/RG8rNJWh3K0nr0KED69evx9LSkocffphRo0ZRUFDAl19+iU6nw9nZ+bYJvFKlSoW3t27dyuTJk29Zxs7Ojj179uDr63vD0XlERAS+vr63LO/m5lZ4e8yYMbz22mt3jP9OdDod+/btw8bG5rbLZGdnM2HCBA4dOoS/vz9TpkwprNW2sLBAp6+1Lq9XO1eyrEQN5xokZiUSnxVPRm4Gle0q42LjokqBb8fcEtpNhrr9YM2LWinm8d+g7zfgWcfQ0ZWo13rU4fCVZN798xTNA1zxc7EzaDymtSstRWFhYezdq1UP/PLLL7Rr147atWtz+fJlQkJCAFi8eDEPPfQQAO3bt2f69Om0bt0aDw8PEhMTOXfuHPXr18fR0ZFq1arx22+/AdrVif/++2+R6712RH/z17UTlc2bN+fChQtcunSJ3Nxcli1bRr9+t16OHh0dXXh71apV1K1bt/B+nTq3/pO1bNmS7du3k5iYSF5eXmGsAN26dWPmzJmF94vaYV1L4O7u7mRkZLBixYrC5wICAjh8+DDADY9fz8HBgfR04x4WMRNmeNh5UMO5BrYWtkRfjeZS6iWy8rMMHZpxc6sBI1ZB/28h/qxJlmKamwm+HhKMTkpeXv4vBTrDXouhEn0x1a5dm9mzZ1O3bl2Sk5MZP348NjY2/PDDDwwaNIgGDRpgZmbGuHHjAC1RxsbG0qFDBwAaNmxIgwYNCo/2lixZwsKFC2nUqBFBQUH89ddf9xWXhYUFs2bNonv37tStW5fBgwcTFKQNb7377rusWrUKgBkzZhAUFESjRo2YMWMGixYtAiAhIaHIC4K8vb2ZMmUKrVu3pm3btjfsGGbMmMGhQ4do2LAh9erVY+7cube83tnZmWeeeYb69evTvXt3mjdvXvjcK6+8UnguIyEhocjt6tu3L3/88QfBwcHs3Lnzvn42ZcXa3JqqjlXxdfAtLMWMzoimQFdg6NCMlxDQeJg2QXm9/v+VYobtM3RkJcbf1Y73+gWx/1ISC3aGGjSWctHr5syZMzckmrJ2+fJl+vTpw8mTJw0WQ2lZs2YNoaGhPP/884YOxSTk6/KJy4wjOTsZczNzvOy8cLJ2MvrhHEP/j3Fho1aCmRoOzUbDw1PAxslw8ZQQKSXjfj7MlrNx/DWxHfV8HEttXeW+141Sevr06aOSfAmyMLPAx96H6s7VsTK3IjIjkktpajjnrmp2hQn7oNVEOLxIO1l7ZrWho3pgQgg+fbQhznZWTP71GNl5hvmUV6xEL4ToIYQ4J4QIEUK8XsTz1kKIX/XP7xdCBOgf7yqEOCyEOKH/3rmE4y8TAQEBJnk0r5QeWwtbqjlWw8feh9yC/4Zz8nV3vwiuwrK2hx6fwJhNN5ViRt/9tUbMtZIVXwxsyLnYdKb+fc4gMdw10QshzIHZQE+gHvC4EOLmq3GeBpKllIHA18Dn+scTgL5SygbASGBxSQWuKMZOCIGLjQs1nWviauNKUnYSIckhJGYlqlYKd+LbVCvFfHiKvitmCzi4sFx3xexU25MnW1Vlwa5L7Akp+rxUaSrOEX0LIERKGSqlzAWWAf1vWqY/8KP+9gqgixBCSCmPSimvXRlxCrAVQliXROCKUl6Ym5njbe9NDeca2FjYEHM1htCUUHWx1Z1cK8Ucvwd8gmHtS7CoF8Qb5oi4JLzZqy7V3Svx8m//kppZtrOZFSfR+wLXX0YZoX+syGWklPlAKuB20zKPAUeklLfUUAkhxgohDgkhDsXHxxc3dkUpV2wsbKjqWJUqjlWQSMLSwriSdoUcEyorLHGFpZizIe6MVoq57XPIN65eMsVha2XO10OCiUvPYdrGst1hlcnJWCFEENpwzrNFPS+lnC+lbCalbHbtSkxFMUVCiMJJTipXqkxmXiYhKSFEZUSp8fvbEQIaD9cmKK/bF7Z9AvPaQ9h+Q0d2zxr5OzOkuT+/HAjjSuLVMltvcRJ9JOB/3X0//WNFLiOEsACcgET9fT/gD2CElPLigwas3Gr06NF4enpSv359Q4eiFJOZMMPd1p2aLjVxsXEhOTuZC8kXiM+MV/X3t2PvCQO/hyd+0+at/b67dnVtOeuK+UKXmpibCb7653yZrbM4if4gUFMIUU0IYQUMBVbdtMwqtJOtAAOBLVJKKYRwBtYCr0spd5dQzBVGcdoUA4waNYoNGzaUcjRKabhWjhnoHIi9pT1xmXGEpISQnJ1s8jNb3bda3fSlmOPh0Pf6UsziNcwzBpUdbXi6XTVW/RvFyciy2UndtdeNlDJfCDEJ+BswB76XUp4SQnwAHJJSrgIWAouFECFAEtrOAGASEAi8K4R4V/9YNynl/U/Dsv51bTLikuTVAHoWPWHHNbdrVbx3716DtikGra/O9W1+lfLH2sIaf0d/MvMyicmMISojisSsRDztPHGwcjD6C67KnLU99PgUGgyEVc/Dr8O0YZ2eX4Kjt6Gju6tnH6rBkv1hfPH3OX4a3aLU11espmZSynXAupsee/e629nAoCJe9xHw0QPGaDTOnTvHwoULadu2LaNHj2batGnMmzePzZs3U6tWLUaMGMGcOXOYNGlSkW2K8/Pzb2hTPHfuXGrWrMn+/fuZMGFCYTfJa22Kzc3N79rUTDEtdpZ2VHOsRnpuOrGZsYSnh2NrYYunnSf2VvaGDs/4+DaFsdtgz0zY/jmEtoSu70OTkUbdFdPRxpKJHQP5eN0Z9oQk0CbQvVTXV/66V97lyLs03dyq+MMPPyw3bYqV8kMIgaO1Iw5WDqTkpBCfGc+VtCtUsqyEp50ndpaG7YRodMwtof1LWs+c1S9onTGPL9e6YnrUMnR0t/Vk66r8sPsSn284y58T25bqp7byl+gN6OZfhLOzM4mJiUUuW1ZtihXTde2CKydrJ5Kzk4nPiudS6iUcrBzwsPPA1sLW0CEaF7caMHI1HFsCf78Fc9tCh1eh7YtgYWXo6G5hY2nO5K61eHXFcdafjKFXg9IbcjLezzZG6OZWxdemwDNkm2LF9JkJM9xs3ajpXBNPO0+u5l0lNCWUsLQw1UPnZteXYtbpA1s/hnkdIPyAoSMr0qNN/KhV2Z6pf58jr6D0rvxVif4e3NyqePLkyQZvUwzw+OOP07p1a86dO4efnx8LFy588I1VjI65mTkedh7UcqmFh52HSvh3Yu8Jg36AJ5ZDTjos7AZrX4Hskp1v+kGZmwle616H0ISrLD905+k9H4RqU1xMptyqWCmfCnQFJGYnFvbOcbBywMPWA1vLex/SMYb/sVKTkw5bPoL988DBG3p/BXV6GTqqQlJKBs3dS1hSJttf7YSt1f1NOK/aFCuKCTI3M8fTzpNaLrXwtPMkMy+T0NRQLqdeJiM3Q9XhX2PtAD0/17pi2rrAssfh1ychPcbQkQHauZjXe9YhLj2H73dfKpV1qERfTKpVsWKsrg3p1HSpSeVKlckpyOFK2hUupV4iLSdNJfxr/JrBs9uhy7tw/m+Y1QIO/WAUXTGbBbjSu4E3OaXUr15V3SiKiTA3M8fd1h1XG1dSclJIzEokPD0cK3Mr3G3dcbJ2wkxU8GM7c0to/zLU7a+VYa55EU7oJyh3r2nQ0GY90bjUSiwr+G9dUUyPmTDD1caVQOdA/Bz8MBNmRGVEFfbSUc3TAPdArRSz30yIPQlz2sD2LwzaFbM06+hVolcUEyWEwMnaiepO1anqWBUbCxviMuM4n3yeqIwo1R5ZCGgyAiYehDq9jb4U80GoRK8oJk4Igb2VPVUdq1LDuQZO1k6k5KQQkhJCWFqYduKWCjyO71AZBi2Cx5dBTppWirnuVa1ax0SoRF/OhYeH06lTJ+rVq0dQUBDffPONoUNSjJiNhQ2+9r7UdKmJh50HmfmZXEm7QnxmPEvOLKnYs17V7gkT90OLsXDgO60r5tl1d39dOaASvRErTptiCwsLvvrqK06fPs2+ffuYPXs2p0+fLoPolPLM0syysDTT194XM2HGZwc+o8tvXfhw74dcSL5g6BANw9oBen0BT28EGyetFHP5CKMpxbxf5a7q5vMDn3M26WyJvmcd1zr8r8X/7riMsbYp9vb2xttb65Hh4OBA3bp1iYyMpF69m+dvV5RbmQkznG2ccbd1Z1nvZSw9u5Q/Q/5k+fnlNPFswmO1HqNr1a4Vr6+Of3MYux32fAPbv4SL26DbB9B4hFF3xbyd8hexAZ07d44JEyZw5swZHB0dmTZtGqNGjeLXX3/lxIkT5OfnM2fOHBo3blxkm+L9+/ff0KZ45syZHD58mKlTpzJhwoTC9VxrUzxt2jS2bt1KcHDwLV9t2rS5Jb7Lly9z9OjRwnUoyr0Icg/io3YfsWnQJl5q+hKJ2Ym8testuizvwsf7PuZcUvmdmPu+WFhpTdHG79HmrFj9AvzYBxLK36edcndEf7cj79JkzG2KMzIyeOyxx5g+fTqOjo4luNVKReNi48JT9Z9iVNAoDsUeYsX5Ffx+4XeWnVtGfbf6DKg5gB7VeuBoVUH+zq6VYh5dDBvfgTnXumK+YJRdMYtS7hK9IRlrm+K8vDwee+wxhg0bxqOPPnqfW6coNxJC0NyrOc29mpOak8rqi6tZeWElH+77kM8PfE5H/470D+xPG582WJiZeCoxM4OmI6FWD1j/Gmz9CE79Dn1naMM8Rk4N3dwDY2xTLKXk6aefpm7durz00kul/SNQKignayeG1xvO7/1+Z1nvZQysNZADMQeYuHkiD//2MF8c/IKzSWdNv92CQ2UY/KNWipmdCgu7wrrXjL4UUyX6e2CMbYp3797N4sWL2bJlS+H4/bp1plESphgfIQRB7kG80fINtgzawjedviHYM5ilZ5cyaPUg+v3Zj2+PfUtoaqihQy1dtXtqE5S3eAYOzIfZreDcBkNHdVuqTXExqTbFiil70P+x5OxkNl7ZyIbLGzgUcwiJpLZLbXpU60GPgB74OfiVYLRGJvyANkF5/Bmo9wj0/EI78i9jd2pTbOIDa4qilAUXGxcG1x7M4NqDicuM45/L/7D+8nq+OfIN3xz5hrqudelSpQtdqnShhnONUu3rUub8W8CzO2D3N7DjCwjdCl0/1NorGMl2qiN6RVFK7X8sIj2CjVc2sjlsM//Ga+ehAhwD6FylMw9XeZgg9yDT6qiZcEErw7yyGwLaQ5/pWtVOGbjTEb1K9IqilMn/WFxmHFvCtrA5bDMHYw5SIAtws3GjvV97Ovh1oLV3a+yt7Es1hjKh02mlmP+8A/nZ8NCr0Kb0SzFVolcU5Y7K+n8sNSeVHRE72B6xnT2Re0jPS8fCzIKmnk1p79eedr7tqO5UvXwP8aTHaKWYp/8Cz3paS2S/IvNwiVCJXlGUOzLk/1ieLo9jccfYGbmTnRE7CUnRypU9bT1p5dOKVt7al4edh0Hie2Bn18HalyE9Glo+C53f1nrqlDCV6BVFuSNj+h+Lyohib9Re9kbvZX/0flJyUgAIdA6kpXdLmlZuShPPJrjZuhk20HuRnQabP4CDC8DRF/pMg1rdS3QVanJwE7dhwwZq165NYGAgn332WZHL7NixgyZNmmBhYcGKFSvueR2LFi1i0qRJd1wmJyeHhx9+mODgYH799dd7Xse2bdvo06dP4e1rF4WV9HqK6/Lly/zyyy/39dqiehGVljFjxty1Y+miRYuIiooqo4gejI+9D4/VeoypD01l+5DtLO+znMlNJ+Nu687K8yt5adtLdFzekb5/9GXKnimsvriayIxI475Yy8YRek+Fp//RjuZ/GQy/PQUZcWWyelVeacTy8/OxsLjzr6igoICJEyeyceNG/Pz8aN68Of369bule2WVKlVYtGgRU6dOLbV4jx49ClCs3jx3s23bNuzt7YtMmHdaT0FBQWGfoAd1LdE/8cQTtzx3t9/N7XZSJa2goIAFCxbcdblFixZRv359fHx8yiCqkmMmzKjrVpe6bnUZXX80eQV5nE46zeHYwxyOPcw/l/9h5YWVALjZuNHQo6H25d6QIPcgKllWussaylhhKeZ02PElXNwC3T6CxsNLtRSz3CX6mE8+IedMybYptq5bB68337zt87drUWxnZ8fmzZsN2qb4wIEDBAYGUr16dQCGDh3KX3/9dUuiDwgIAMDsHlqs/vDDD3z66ac4OzvTqFEjrK2tAYiPj2fcuHGEhYUBMH36dGrWrMnw4cOJj48nODiYlStXsmTJElavXk1WVhZt2rRh3rx5CCHo2LEjU6dOpVmzZiQkJBS2krj+5z137lzMzc35+eefmTlzJu3btwcgLi7ulvV06dKFIUOGsHHjRl577TWklHzyySdIKenduzeff/45APb29owfP55169bh7e3NJ598wmuvvUZYWBjTp0+nX79+N2z/66+/zpkzZwgODmbkyJG4uLjw+++/k5GRQUFBAWvXrqV///4kJyeTl5fHRx99RP/+/QvXlZGRwbZt25gyZQru7u6cPHmSpk2b8vPPP99yknHGjBnMnTsXCwsL6tWrx7Jly8jIyOC5557j0KFDCCF47733eOyxx7C3t+fZZ59l06ZNzJ49m7fffrvw52lvb88zzzzDP//8g5eXF8uWLWP79u0cOnSIYcOGYWtry969e7G1LZ9thy3NLWnk0YhGHo0YXX80BboCLqRc4GjcUU7En+B4wnG2hm8FtJ1EDeca1HWtSx3XOoVfDlYlPz5+Tyys4KHXtIurVr8AqybB8V+1CcrdapTOKkvlXU3QuXPnWLhwIW3btmX06NF8++23TJo0iVGjRrF582Zq1arFiBEjmDNnDpMmTSqyTXF+fv4NbYrnzp1LzZo12b9/PxMmTGDLli3Af22Kzc3N79rULDIyEn9//8LH/fz82L9//wNvb3R0NO+99x6HDx/GycmJTp060bhxYwBeeOEFJk+eTLt27QgLC6N79+6cOXOGBQsWMHXqVNasWQPApEmTePfddwF48sknWbNmDX379r3rugMCAhg3bhz29va88sorNzzn6el5y3oA3NzcOHLkCFFRUbRq1YrDhw/j4uJCt27d+PPPP3nkkUe4evUqnTt35ssvv2TAgAG8/fbbbNy4kdOnTzNy5MhbEv1nn312w3oWLVrEkSNHOH78OK6uruTn5/PHH3/g6OhIQkICrVq1ol+/frck8aNHj3Lq1Cl8fHxo27Ytu3fvpl27dres69KlS1hbW5OSkgLAhx9+iJOTEydOnAAgOTkZgKtXr9KyZUu++uqrW352V69epVmzZnz99dd88MEHvP/++8yaNYtZs2YV7gxMibmZeWECf7zO44BW0XMi4QTH449zIuEEe6L2sOriqsLX+Nn7UdetLoHOgdRwrkENpxpUdayKpbll2QbvUQtGrYUjP8LG97QJyju/DW2eK/FVlbtEf6cj79J0c4viGTNm0LVrV6NpU1zS9u/fT8eOHfHw0CodhgwZwvnz5wHYtGnTDWPCaWlpZGRk3PIeW7du5YsvviAzM5OkpCSCgoKKlejvx5AhQwA4ePDgDXEPGzaMHTt28Mgjj2BlZUWPHj0AaNCgAdbW1lhaWtKgQYMbPlHcSdeuXXF1dQW0hnJvvvkmO3bswMzMjMjISGJjY/Hy8rrhNS1atMDPT2sBEBwczOXLl29J9A0bNmTYsGE88sgjPPLII4D2c162bFnhMi4uLgCYm5vz2GOPFRmfmZlZ4c9i+PDhFbKbqZO1E+1829HO97+fcUJWAmcSz3A26Sxnks5wLukcm65sKpwr10JYUMWxCjWcaxDgGIC/gz/+Dv5UcayCh61H6ZV5mplBs6e03jnrS685WrESvRCiB/ANYA4skFJ+dtPz1sBPQFMgERgipbysf+4N4GmgAHheSvl3iUVfhm7+Rd/tF19WbYp9fX0JDw8vfDwiIgJfX9972LJ7p9Pp2LdvHzY2NrddJjs7mwkTJnDo0CH8/f2ZMmUK2dnZgDb9oU6nK1yuJFz/M7sdS0vLwt+bmZlZ4VCUmZlZsaZtvHk9S5YsIT4+nsOHD2NpaUlAQECR23NtPaAl6aLWtXbtWnbs2MHq1av5+OOPC4/ii2JjY1Ps8xDlug69BLnbutPerz3t/doXPpadn83ltMuEpIQQmhLKxZSLnE8+z5awLRTIgsLlbC1s8XPww9feF+9K3nhV8sLLzgtve2+87LzwsPN48DbNDl4w+CftYqtScNfohBDmwGygKxABHBRCrJJSXn+a/2kgWUoZKIQYCnwODBFC1AOGAkGAD7BJCFFLyut+iuXEtRbFrVu35pdffqFdu3bUrl27sE1xYGDgLW2KR4wYwYgRIwrbFMfGxlK/fn2EEIVtigcNGoSUkuPHj9OoUaNb1nu3I/rmzZtz4cIFLl26hK+vL8uWLbvnSpE6depw9uyN5z1atmzJCy+8QGJiIo6Ojvz222+F8XXr1o2ZM2fy6quvAtpJ0eDg4Btefy3hubu7k5GRwYoVKxg4cCCgDc0cPnyYFi1a3LYCyMHBgbS0tHvaDtCOnp9//nkSEhJwcXFh6dKlPPfc/X0UdnBwID399kdYqampeHp6YmlpydatW7ly5cp9rUen0xVO8t6uXbvC8fmuXbsye/Zspk+fDmhDN9eO6u/0XitWrGDo0KGFf6fF2ZaKyMbCpnDY53p5ujxiMmIISw8jPD2csPQwwtLCiEiP4HDMYdLzbvw5CgTO1s642brhZuOGq60rbjZuuNm64WjliIOVQ+H3a192FnbYWNjc2v6hlKYpLM5uqAUQIqUMBRBCLAP6A9cn+v7AFP3tFcAsoR1K9AeWSSlzgEtCiBD9++0tmfDLzrUWxaNHj6ZevXqMHz8eGxubwjbF107G3qlNcUxMzA1tisePH89HH31EXl4eQ4cOLTLR342FhQWzZs2ie/fuFBQUMHr0aIKCggB49913adasGf369ePgwYMMGDCA5ORkVq9ezXvvvcepU6dISEgosizN29ubKVOm0Lp1a5ydnW9I5DNmzGDixIk0bNiQ/Px8OnTowNy5c294vbOzM8888wz169fHy8uL5s3/m5zhlVdeYfDgwcyfP5/evXsXuV19+/Zl4MCB/PXXXzecjL0bb29vPvvsMzp16lR4MvbaCdJ71bBhQ8zNzWnUqBGjRo26JckOGzaMvn370qBBA5o1a0adOnVu8053VlBQwPDhw0lNTUVKyfPPP4+zszNvv/02EydOpH79+pibm/Pee+/ddSimUqVKHDhwgI8++ghPT8/C8tNRo0Yxbty4cn8ytixYmlni7+iPv6N/kc9n5GYQczWGmMwYYq7GEJsZS2JWIknZSSRmJXIy4SSJWYlk5mfedV1WZlbYWNhgY2GDrYUtHf068krzV+76unt11wumhBADgR5SyjH6+08CLaWUk65b5qR+mQj9/YtAS7Tkv09K+bP+8YXAeinlipvWMRYYC1ClSpWmNx8ZGfpiDlNuUbxmzRpCQ0N5/vnnDR2KUgKuVfvcK0P/j5mi7Pxs0nPTSctNK/x+7XZWfhbZ+dlkF2Rr3/VfDTwa8GS9J+9rfUbfplhKOR+YD9qVsQYOp0K5doGSoigl69qRujG0bijOgFAkcP1nGD/9Y0UuI4SwAJzQTsoW57VGLyAgwCSP5hXTcz9H84rpK06iPwjUFEJUE0JYoZ1cXXXTMquAkfrbA4EtUhsTWgUMFUJYCyGqATWBA/cTqFFf3qwo5Zj63zJ9dx26kVLmCyEmAX+jlVd+L6U8JYT4ADgkpVwFLAQW60+2JqHtDNAvtxztxG0+MPF+Km5sbGxITEzEzc1NlYspSgmSUpKYmHjHUlml/CsX3Svz8vKIiIgosZprRVH+Y2Njg5+fH5aWZXxlqFKijP5k7N1YWlpSrVo1Q4ehKIpSLqk2xYqiKCZOJXpFURQTpxK9oiiKiTO6k7FCiHjg/pqGaNyBhBIKpzxR212xqO2uWIqz3VWllEVenWV0if5BCSEO3e7MsylT212xqO2uWB50u9XQjaIoiolTiV5RFMXEmWKin2/oAAxEbXfFora7Ynmg7Ta5MXpFURTlRqZ4RK8oiqJcRyV6RVEUE2cyiV4I0UMIcU4IESKEeN3Q8ZQWIcT3Qog4/axe1x5zFUJsFEJc0H+/88Si5ZAQwl8IsVUIcVoIcUoI8YL+cZPediGEjRDigBDiX/12v69/vJoQYr/+7/1XfQtxkyOEMBdCHBVCrNHfryjbfVkIcUIIcUwIcUj/2H3/rZtEor9uAvOeQD3gcf3E5KZoEdDjpsdeBzZLKWsCm/X3TU0+8LKUsh7QCpio/x2b+rbnAJ2llI2AYKCHEKIV8DnwtZQyEEgGnjZciKXqBeDMdfcrynYDdJJSBl9XP3/ff+smkei5bgJzKWUucG0Cc5MjpdyB1vP/ev2BH/W3fwQeKcuYyoKUMlpKeUR/Ox3tn98XE992qbk2bZSl/ksCnYFrcy+b3HYDCCH8gN7AAv19QQXY7ju47791U0n0vkD4dfcj9I9VFJWllNH62zFAZUMGU9qEEAFAY2A/FWDb9cMXx4A4YCNwEUiRUubrFzHVv/fpwGuATn/fjYqx3aDtzP8RQhwWQozVP3bff+vloh+9UnxSSimEMNmaWSGEPbASeFFKmXb9jGOmuu36WdmChRDOwB9AHcNGVPqEEH2AOCnlYSFERwOHYwjtpJSRQghPYKMQ4uz1T97r37qpHNGbxCTkDyBWCOENoP8eZ+B4SoUQwhItyS+RUv6uf7hCbDuAlDIF2Aq0BpyFENcO1Ezx770t0E8IcRltKLYz8A2mv90ASCkj9d/j0HbuLXiAv3VTSfTFmcDclF0/OftI4C8DxlIq9OOzC4EzUspp1z1l0tsuhPDQH8kjhLAFuqKdn9gKDNQvZnLbLaV8Q0rpJ6UMQPt/3iKlHIaJbzeAEKKSEMLh2m2gG3CSB/hbN5krY4UQvdDG9K5NYP6xYSMqHUKIpUBHtLalscB7wJ/AcqAKWovnwVLKm0/YlmtCiHbATuAE/43Zvok2Tm+y2y6EaIh24s0c7cBsuZTyAyFEdbQjXVfgKDBcSpljuEhLj37o5hUpZZ+KsN36bfxDf9cC+EVK+bEQwo37/Fs3mUSvKIqiFM1Uhm4URVGU21CJXlEUxcSpRK8oimLiVKJXFEUxcSrRK4qimDiV6BVFUUycSvSKoigm7v/eHjmOjbb8BgAAAABJRU5ErkJggg==", + "text/plain": [ + "
    " + ] + }, + "metadata": { + "needs_background": "light" + }, + "output_type": "display_data" + } + ], + "source": [ + "#collapse_output open\n", + "scheduler = PolyLRScheduler(optimizer, t_initial=t_initial)\n", + "plot_lr(scheduler, label='power=0.5, default')\n", + "\n", + "scheduler = PolyLRScheduler(optimizer, t_initial=t_initial, power=1)\n", + "plot_lr(scheduler, label='power=1')\n", + "\n", + "scheduler = PolyLRScheduler(optimizer, t_initial=t_initial, power=2)\n", + "plot_lr(scheduler, label='power=2')\n", + "\n", + "scheduler = PolyLRScheduler(optimizer, t_initial=t_initial, power=0.1)\n", + "plot_lr(scheduler, label='power=0.1, default from train script')\n", + "\n", + "plt.legend();" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### `lr_min`" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "`lr_min` is value of lower lr bound, default is 0." + ] + }, + { + "cell_type": "code", + "execution_count": 12, + "metadata": { + "scrolled": true + }, + "outputs": [ + { + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXoAAAD4CAYAAADiry33AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/YYfK9AAAACXBIWXMAAAsTAAALEwEAmpwYAAA0xUlEQVR4nO3deVhV1f7H8fdiVsQBxAlUQHAAwQlRnIdKLYc053JO07LxVrd+dRu8zXUrS8vMsdJMrdQGNed5AsUBxRFUQHEGHECG9ftjH80Q4aAcDhy+r+fxkbPP2vt8972nD9u1115Laa0RQghhu+ysXYAQQgjLkqAXQggbJ0EvhBA2ToJeCCFsnAS9EELYOAdrF5BT5cqVtY+Pj7XLEEKIEiUyMvKc1tozt/eKXdD7+PgQERFh7TKEEKJEUUodv9N70nUjhBA2ToJeCCFsnAS9EELYuGLXRy+EKN4yMjKIj48nLS3N2qWUSi4uLnh7e+Po6Gj2PhL0QogCiY+Px83NDR8fH5RS1i6nVNFac/78eeLj4/H19TV7P7O6bpRSXZVSB5VSR5RSr+Tyfjul1E6lVKZSqm+O94YppQ6b/gwzuzIhRLGUlpaGh4eHhLwVKKXw8PAo8L+m8g16pZQ9MBnoBgQCg5RSgTmanQCGA3Nz7OsOvAm0AMKAN5VSlQpUoRCi2JGQt567+d/enCv6MOCI1vqY1vo6MA/odWsDrXWc1noPkJ1j3y7ACq31Ba31RWAF0LXAVZrh2uUUtn41hiMRK9DZWZb4CCGEKJHMCXov4OQtr+NN28xh1r5KqTFKqQilVMTZs2fNPPQ/xe7bQpOkX/D/vS/n/hvAnmnjuBCzAbJz/u4RQtiSt956i08++eSO7589e5YWLVrQpEkTNmzYUODjz5o1i/HjxwOwaNEi9u/ff9e1WkuxGF6ptZ6qtQ7VWod6eub6BG++Alt2Ie35Q2xq9AHHHAOod3I+7vO6c/7duhz94VnS47aBLLIiRKmzatUqgoOD2bVrF23btr2nY9ly0CcANW957W3aZo572bfAKlR0p3XvcbT4v+Ukjt7LH/5vsV/XxvvwDzjPeoCL79Un6eeX0Qm7JPSFKMHeffdd6tatS5s2bTh48CAAR48epWvXrjRr1oy2bdsSExNDVFQUL7/8MosXL6Zx48Zcu3aNcePGERoaSlBQEG+++ebNY/r4+HDu3DkAIiIi6NChwz8+c/PmzSxZsoSXXnqJxo0bc/To0SI733tlzvDKHUCAUsoXI6QHAoPNPP5y4L1bbsA+ALxa4Crvgq93DXwfe56s7OfYfiCWYxvn45WwjNZ7pqH2fkNyGW8cgvvg2rQ/VG0IcnNJiAJ7+7do9iemFOoxA2uU580eQXd8PzIyknnz5hEVFUVmZiZNmzalWbNmjBkzhilTphAQEMC2bdt48sknWb16NRMmTCAiIoJJkyYBxi8Jd3d3srKy6Ny5M3v27CEkJCTfulq1akXPnj3p3r07ffv2zbd9cZJv0GutM5VS4zFC2x6YobWOVkpNACK01kuUUs2BX4FKQA+l1Nta6yCt9QWl1H8xflkATNBaX7DQueTK3k4RHuRHeNArpKT9iyURMSRtX0jDi6totW0SbP+Cy25+uDTpj0PwI+BZtyjLE0IU0IYNG+jduzdly5YFoGfPnqSlpbF582b69et3s116enqu+8+fP5+pU6eSmZnJqVOn2L9/v1lBX5KZ9cCU1vpP4M8c29645ecdGN0yue07A5hxDzUWmvIujjzSJhjaBHPs7Et8vXUvV6N+pd2lDbRY/yGs/4A0j0BcGveDhn2gko+1SxaiWMvryrsoZWdnU7FiRaKiovJsFxsbyyeffMKOHTuoVKkSw4cPvzkm3cHBgWzT4A1be+q3WNyMtQY/z3I83SOcF1/7kPTHFvO633zeyRpK9NkMWPU2TGxE5jedYMtXkHLK2uUKIUzatWvHokWLuHbtGqmpqfz222+ULVsWX19fFixYABhPkO7evfu2fVNSUnB1daVChQokJSWxdOnSm+/5+PgQGRkJwM8//5zrZ7u5uZGammqBs7KsUhv0N9jbKTrUq8J7Qx9g/Kv/Y1/XhYysOJ33MwZxKPE8LH8V/WkD9MyHIGImXC3SnichRA5NmzZlwIABNGrUiG7dutG8eXMA5syZw/Tp02nUqBFBQUEsXrz4tn0bNWpEkyZNqF+/PoMHD6Z169Y333vzzTd59tlnCQ0Nxd7ePtfPHjhwIB9//DFNmjQpUTdjlS5mo09CQ0N1cVh4JDoxmQUR8UTt2k6HjPX0cdxKLZ2ItnNA1ekMwX2h3oPgXM7apQpRpA4cOECDBg2sXUapltv/B0qpSK11aG7tZVKzOwiqUYGgnhVI61aflQe68Nr2E1w4FkkPu830PbaNyoeXox3KoOp1g+B+4N8ZHJytXbYQQtxGgj4fLo72dA+pQfeQGiRcasTCiAd4eMdxql/dTT+20j1mFWWjfwGXChDYC4L7Q+3WYFfqe8WEEMWEBH0BeFUsw7P3BfB0J382HW3EvB0deSs6gTC9m+HOO2i9ewGOO78DtxoQ/IgR+tWCZYy+EMKqJOjvgp2dom2AJ20DPLlwpSG/7mrIezvacjLpHA86RTGKCBps+Rq1+UuoXA9C+hvdO5VqW7t0IUQpJEF/j9xdnRjVxpeRrX3YdfISP20PoO+e1jhdv8TIirvpn7mFaqv/C6v/C7XCjcAP6g1l3a1duhCilJCgLyRKKZrWqkTTWpV4vXsDft9zinnbvfk0vg2+Duf4V/W9dEpeS9k/XoCl/4aA+40r/brdwNHF2uULIWyYBL0FuLk4MiisFoPCarE/MYV5O07w6s5qpKZ34H73MzztsZOGCSuwO/gnOJc3buI2Ggi1WslNXCFEoZNUsbDAGuWZ0Ksh21+7j0/6NeaCW316Hu5G4MVPmeT9CWe87kdH/wqzHoKJIbDybTgTY+2yhSjWypUr/OdXlixZwgcffHDPx4mNjaVFixb4+/szYMAArl+/nmu7999/H39/f+rVq8fy5ctvbh85ciRVqlShYcOG91zLDfLAlBUcPJ3Kj9tP8MvOeFLSMgmsbM+LtY/S9uoqHOPWgs6C6o2h0SBo+AiUu7s5+oWwhOLwwFS5cuW4fPnyP7ZlZmbi4GD9Tor+/fvTp08fBg4cyNixY2nUqBHjxo37R5v9+/czaNAgtm/fTmJiIvfddx+HDh3C3t6e9evXU65cOYYOHcq+ffty/YyCPjAlQW9F165n8cfeU8zddpydJy7h5GBH//pOjK4URa34xahTu8HOAfzvh0YDpD9fFAv/CJmlr8DpvYX7AdWCoVveV9Y3gn7t2rX85z//oVKlSsTExHDo0KHb2sbFxdG1a1datmzJ5s2bad68OSNGjODNN9/kzJkzzJkzh7CwMGbNmnVzOuPhw4dTvnx5IiIiOH36NB999JFZUxNrrfH09OT06dM4ODiwZcsW3nrrrX9csYNxNQ/w6qvGrO1dunThrbfeIjw8/GbN3bt3L7Sgt/6vv1KsjJM9fZt507eZNzGnU/hx2wl+2ZXAD2n1CagSylPh1+mWvQ7n/Qvh0FJwrgANe0OjwVAzTMbnCwHs3LmTffv24evre8c2R44cYcGCBcyYMYPmzZszd+5cNm7cyJIlS3jvvfdYtGjRbfucOnWKjRs3EhMTQ8+ePenbty+pqal3XKVq7ty5VKlShYoVK978l4W3tzcJCbevtZSQkEDLli1vvr5Tu8IiQV9M1K9Wnrd7NeTf3erz++5TzNl2nOfWXOcVx9b0CunDmJqJ1ElcAnvmQ+QscK9jdO00GgAVa1m7fFFa5XPlXRTCwsLyDHkAX19fgoODAQgKCqJz584opQgODiYuLi7XfR5++GHs7OwIDAwkKSkJMGavzGsq5BsrVBU3EvTFTFknB/o3r0n/5jXZG5/M3O3HWbQrkZ8iFcFeQxnW8Xl6OG7HOXoBrHnH+OPTFho/CoE9wcnV2qcgRJFydc3/O+/s/Pc8VHZ2djdf29nZkZmZme8+N7q487uib9CgAZcuXbp5vyA+Ph4vL6/b2np5eXHy5Mmbr+/UrrBI0Bdjwd4VeN87hFcfbMDiXQl8v/U4Ly45xtsu1Xik6YcMb2OHT8JvEDUXFo2FP1+EoIeN0K8VLl07QhSy/K7oATp27MjChQsZOHAgs2fPplevXre16dmzJ4MHD+aFF14gMTGRw4cPExYWZqGqZXhliVDexZEh4T4sf64d858Ip2O9KszZdpwO044x6GA7/ujwJ5nD/jSeuI1eBDO7wReNYd1HcOmEtcsXolT58MMP+fTTT/H39+f8+fOMGjUKMIZvvvGGsTBfUFAQ/fv3JzAwkK5duzJ58uSbc+APGjSI8PBwDh48iLe3N9OnT7/nmmTUTQl1NjWd+REnmbvtBAmXrlHFzZlBYbUY3NSDqvErIWoOxK4DFPi2gyZDoEF3cCxj7dJFCVcchleWdjK8spTJytasPXiG77YcZ92hszjYKboEVWNIeG1aVLqM2j3PCP1Lx41RO8GPQOPHwKupdO2IuyJBb30yvLKUsbdTdG5Qlc4NqhJ37gpzth1nfkQ8f+w9Rd2q5RgSPpA+Y5/H9dRW2DUHon6EiBng2QCaPAYhA+SBLGETzp8/T+fOnW/bvmrVKjw8PKxQUfEhV/Q26Nr1LH7bnch3W+PYl5CCm7MDfUO9GdKyNn5u2RD9C+z6AeJ3GA9k1etmdO3U6Qz28rtf5O3AgQPUr18fJf8itAqtNTExMdJ1Iwxaa3advMR3m+P4Y+8pMrI07ep6Miy8Nh3qVcH+3EHY9T3sngdXz4FbdWNsftMh4O5n7fJFMRUbG4ubmxseHh4S9kVMa8358+dJTU297dkBCXrB2dR05m0/wQ/bjpOUkk5N9zIMC/ehX2hNKjhqOLzcuMo//BfobOMGbtNhUL+7TLsg/iEjI4P4+HjS0tKsXUqp5OLigre3N46Ojv/YLkEvbsrIyuav6CRmb45je9wFyjja06epF8Nb+RBQ1Q1SEo1x+Tu/M27gulQ0plBuOhSqBlm7fCHEHUjQi1xFJyYze3Mci6ISuZ6ZTWt/D4a38qVT/SrYoyFugxH4B5ZA1nXwagbNhkNQH3Au/GlihRB3T4Je5OnClev8uP0EP2w9zqnkNGq5l2VoeG36N69JeRdHuHrB6MffORvOxoBTOQjua3Tt1GgiwzSFKAYk6IVZMrOyWR6dxMxNsUQcv4iraXbN4a198a3sClrDye1G4O/7BTKvQbUQaDYMgvuDS3lrn4IQpZYEvSiwvfHJzNwUy297EsnI0nSs58moNn609jeNtLh2CfYugMjZkLQXHF2Nh7GajTAexhJCFCkJenHXzqSmMWfrCeZsO865y9epV9WNkW186NXYCxdHe+MqP2EnRM4wrvIzrkL1RkbgB/cFZzdrn4IQpYIEvbhnaRnGQ1jTN8YSczoVD1cnHm1Ri8fCa1PFzTT8Mi3ZmC8/YiaciTb68kP6Q+goqFZ4618KIW4nQS8KjdaaLcfOM2NjLKtizuBgp+jZyIvH2/rSoHr5G40gPsKYaiH6F8hMA+8waD4KAh+WcflCWIAEvbCI2HNXmLkplgUR8VzLyKKNf2VGtfWlfYAndnamkThXL8Bu0/w6549AmUrGfPmhI8GjjnVPQAgbIkEvLOrS1evM3X6C2ZvjSEpJx79KOR5v48vDTUz9+GBc5cdtgB3TIeZ3yM6EOp2g+eMQ0EXm2BHiHknQiyJxPTObP/YmMm1DLNGJKXi4OjE03Ich4bVxd3X6u2HqaeNBrIiZkJoI5b0hdDg0GQpuVa1WvxAlmQS9KFI3+vGnbYhldcwZXBzt6NvMm1Ft/Izx+DdkZcKhpbBjGhxbC3aOENgLwsZAzTB5EEuIApCgF1ZzOCmVaRti+XVXAhnZ2TwQWJUx7fxoVtv9nw3PHTa6daLmQHoKVAs2Ar9hX3Aqa53ihShB7jnolVJdgYmAPTBNa/1Bjvedge+AZsB5YIDWOk4p5QhMA5piLHLyndb6/bw+S4LeNp1JTeO7zcf5futxkq9lEFq7Ek+0r0Pn+lX+vnELkH4Z9s6H7d/Cmf3GpGpNHjP68t1973h8IUq7ewp6pZQ9cAi4H4gHdgCDtNb7b2nzJBCitR6rlBoI9NZaD1BKDQZ6aq0HKqXKAvuBDlrruDt9ngS9bbt6PZP5O07y7YZYEi5do46nK2Pa+fFwEy+cHez/bqg1HN8M26fCgd+MqZPrdoUWT4BfB+nWESKHew36cOAtrXUX0+tXAW69MldKLTe12aKUcgBOA57AQGAw0BuoAGwBWmqtL9zp8yToS4fMrGz+3Heab9YdJToxBU83Z0a09uGxlrWNidRulZJoDM+MmGkskFK5HoSNNhZJkVk0hQDuPej7Al211o+bXg8BWmitx9/SZp+pTbzp9VGgBZAMfA90BsoCz2utp+byGWOAMQC1atVqdvz48QKfpCiZtNZsPnqeKeuOsuHwOco5O/Boy1qMau1LlfI5HqzKSIPoX2HbFDgVZSx23uQxI/SlW0eUctZcHDwMyAJqAJWADUqplVrrY7c2MoX/VDCu6C1ckyhGlFK09q9Ma//K7EtI5pv1x/h2/TFmbozjkWZejGlX5++ROo4u0HiQsRBK/A7Y9g1s/wa2fgX1HoSW48CnjXTrCJGDnRltEoCat7z2Nm3LtY2p66YCxk3ZwcAyrXWG1voMsAnI9TeOEA29KvDloCasebED/UK9+XlnAp3+t5Yn50SyNz7574ZKGcMv+06H5/ZC23/BiS0wuztMaQM7vzeu/oUQgHldNw4YN2M7YwT6DmCw1jr6ljZPAcG33Izto7Xur5T6N1Bfaz1CKeVq2neg1nrPnT5P+ujFDWdT05m5KZbvtxwnNT2TtgGVeaqjPy183W9flDrjmjFt8tYpxoRqZSsbc+s0fxzKVbHOCQhRhApjeOWDwOcYwytnaK3fVUpNACK01kuUUi4YffFNgAsYYX5MKVUOmAkEAgqYqbX+OK/PkqAXOaWkZfDD1uPM2BjLucvXaVqrIk928Kdzgyq3B77WELsetn5tPIxl7wTB/aDlkzKDprBp8sCUsAlpGVnMjzjJN+uOkXDpGvWruTGuQx26h9TA3i6XfvlzR2Db18Zi5xlXwbcdhI8H//vBzpxeSyFKDgl6YVMysrJZEpXI1+uOcuTMZXwruzKufR0ebuKFk0MuAX71grH84bapxtw6letC+FMQMlCmTBY2Q4Je2KTsbM3y6NNMWnOE6MQUvCqWYWx7P/qF1vx71sxbZWVA9CLY8iWc2m3044eNNvrxXSsXef1CFCYJemHTtNasPXiWL1cfZueJS3i6OTOmrR+PtqxFWadcRhBrDXEbYcskOLQMHFyMIZvhT0Nl/6I/ASEKgQS9KBVuzJo5afURNh89j7urE6Pb+jEkvDblnO/wyMjZQ7B1MkT9CFnXof5D0PpZY/imECWIBL0odSLiLvDF6iOsP3SWimUdebyNL0Nb+dw+vcINl88Y8+ps/xbSLkHNltD6GajbTW7cihJBgl6UWlEnL/HlqsOsijlDeRcHRrT2ZWQbXyqUuUPgX78Cu34wunUunQCPACPwQwaAg3PRFi9EAUjQi1JvX0IyX6w6zF/7k3BzcWBkfoGflQn7F8GmiXB6D7hVN8biNxsOLuWLsnQhzCJBL4TJ/sQUvlh1mGXRp80LfK3h2BrY+JnxIJZLBWOUToux8sStKFYk6IXI4R+B7+zAiDa+jMor8AESImHj58b8+A7OxsyZrZ6BSrWLrG4h7kSCXog7yHmFP7qtHyNa++B2p5u2YCx7uGki7J5nLIgS0h/aPA+e9YqucCFykKAXIh/Ricl8vvIwK/YnUbGsI2Pa+TEs3AfXOw3LBEhOMG7aRsyEzDRo0APavgA1mhRd4UKYSNALYaY98Zf4bMUh1hw8i7urE2Pb+zGkpQ9lnHJ50vaGK+eMxVC2TYX0ZKjTGdq9BLXDi65wUepJ0AtRQDtPXOSzFYfYcPgcnm7OPN3JnwHNa/5zXduc0lJgxzTYMtlY8tCnrRH4vu1kMRRhcRL0Qtyl7bEX+GT5QbbHXcCrYhme7RxAn6ZeONjn8RDV9SsQOdvox798GrzDjMAPuF8CX1iMBL0Q90BrzYbD5/jkr4PsiU/Gr7Irz91fl+7B1bHLbXrkGzLSIOoHY6RO8kmo3gjavwL1ukngi0InQS9EIdBa89f+JD796xAHk1JpUL08L3epR4d6nrcvgHKrzOuwZx5s+B9cjINqIdDhFWOdWwl8UUgk6IUoRNnZmt/2JPK/vw5x4sJVmvtU4uWu9Wnu4573jlkZsGc+rP8YLsZCtWDjCr/+QxL44p5J0AthAdczs/kp4iRfrDrM2dR0Otevwotd6tGgej5TJGRlGuvbrv8ILhyDqsHGFb4EvrgHEvRCWNDV65nM2hzHlLVHSU3PpFejGvzrgXrUdC+b945ZmbBvIaz7CC4cNfrwO74GAQ9I4IsCk6AXoggkX83g63VHmbkpFq3hsZa1Gd/JH3dXp7x3zMqEvfNh3YdGH75XKHT8P6jTSQJfmE2CXogidCr5Gp+vOMyCyJO4OjkwtkMdRrb2zfuhKzD68KPmGn34ySeNOfE7vQ6+bYumcFGiSdALYQWHk1L5cNlBVh5IooqbM8/fX5d+zbzzHoMPxiidXd/D+k+Mxcz9OkCnN8C7WZHULUomCXohrGhH3AU+WBpD5PGLBFQpx6sP1qdjvSp5D8kEYxx+xHRjWObV88ZwzI6vQbWGRVO4KFEk6IWwMq01y6NP88HSGOLOXyXcz4PXHmpAQ68K+e+cngpbp8DmLyE9BRo+YvThe9SxfOGixJCgF6KYuJ6Zzdxtx5m46jAXr2bQu4kXL3aph1fFMvnvfPWCEfbbphgLmTcdCu3/DW7VLF+4KPYk6IUoZlLSMvh67VGmb4wF4PE2vjzZ0Z9yeU2LfENqknHDNnIm2DtBy3HGAihlKlq2aFGsSdALUUwlXLrGJ8sP8uuuBCqXc+bFB+rSL7Qm9nnNoXPDhWOw+l1jLH6ZStDmBQgbA44uli9cFDsS9EIUc1EnL/HO7/uJOH6R+tXc+E/3QFr7VzZv51O7YdUEOLISynsbQzJDBoBdPqN7hE2RoBeiBNBa8+fe07y/9ADxF69xX4Mq/N+DDfDzLGfeAWLXw4o3IHGXMa3CAxOMh65EqSBBL0QJkpaRxazNcUxafYT0zCyGt/Lh6c4BlM9rHdsbsrMh+hfjCv/ScSPo759gTKAmbJoEvRAl0NnUdD5ZfpD5kSfxcHXipS716NesZt5z4N+QmW6sdrXuI0hLhkaDjC6dCl6WL1xYhQS9ECXY3vhk3v4tmojjFwn2qsCbPQIJzW9K5BuuXYQNn8K2b0DZQetnofUz4ORq2aJFkZOgF6KE01qzZHci7/8Zw+mUNHo1rsH/PdiAquXNHGFzMQ5WvgXRv4Jbdej8BoQMlBu2NkSCXggbcfV6Jl+vPco364/haKd4pnMAI1r74uRgZmCf2AbLX4WESGNa5C7vgU8byxYtioQEvRA25sT5q0z4fT8rDyTh5+nKWz2CaFfX07yds7Nh38/GFX5KPAT2gvv/C5VqW7RmYVkS9ELYqDUxZ3j7t2jizl+lS1BVXn8oMP8FT27IuGZMqbDxM9DZxtO1bZ6T/vsSSoJeCBuWnpnFtA2xTFp9BI3m6U4BjG7rZ353TnI8rHjTeMLWrYYxHDO4ryx6UsLkFfRmfROUUl2VUgeVUkeUUq/k8r6zUuon0/vblFI+t7wXopTaopSKVkrtVUrJ89lCFCJnB3ue6ujPqn+1p0PdKny8/CDdJq5n85Fz5h2ggjf0nQ4jlkE5T/jlcZjRFRKjLFq3KDr5Br1Syh6YDHQDAoFBSqnAHM1GARe11v7AZ8CHpn0dgB+AsVrrIKADkFFo1QshbqpRsQxThjRj5vDmZGRpBk/bxjM/7uJMSpp5B6gdDqPXQM8v4fwRmNoB/viXMURTlGjmXNGHAUe01se01teBeUCvHG16AbNNPy8EOitjVYUHgD1a690AWuvzWuuswildCJGbjvWr8Nfz7XimcwDL9p2m8//WMXNTLFnZZnTT2tkb0x8/HWlMkBYxA75sBju/N27iihLJnKD3Ak7e8jretC3XNlrrTCAZ8ADqAloptVwptVMp9XJuH6CUGqOUilBKRZw9e7ag5yCEyMHF0Z4X7q/L8ufb0bhWRd7+bT+9v9rEvoRk8w5QpiI8+BGMWQceAbBkPMx4QLpzSihLPy3hALQBHjX93Vsp1TlnI631VK11qNY61NPTzCFiQoh8+VZ25buRYXw5qAmJl9LoOWkj7/y+nyvpmeYdoHoIjFwGD08xHrqa2gH+eNGYVkGUGOYEfQJQ85bX3qZtubYx9ctXAM5jXP2v11qf01pfBf4Emt5r0UII8yml6NGoBqteaM/AsFpM2xjLA5+tZ9WBJHMPAI0HwfgIU3fOdJjUHPYuhGI2ak/kzpyg3wEEKKV8lVJOwEBgSY42S4Bhpp/7Aqu1MW5zORCslCpr+gXQHthfOKULIQqiQllH3usdzMKx4bg62zNqdgTjfog0/2btje6cx1cZ0yj8PAp+6APnj1q0bnHv8g16U5/7eIzQPgDM11pHK6UmKKV6mppNBzyUUkeAF4BXTPteBD7F+GURBezUWv9R6GchhDBbqI87vz/dlpe61GN1zBk6f7qOedtPYPYzNV5NYfRq6PYRnNwBX4XDuo+NGTNFsSQPTAlRisWeu8IrP+9hW+wFwv08eL9PMD6VC/BkbMopWPYK7F8EletCjy+MYZqiyN3zA1NCCNvkW9mVH0e35L3ewexLSKbrxPVMXX+UzCwzh1KWrw79Z8OjCyEjDWZ2Ncbep6VYtnBRIBL0QpRydnaKwS1qseKF9rQN8OS9P2Po/dVmDpwqQFgH3A9PboEW42DHdPiqJRxcZrmiRYFI0AshAKhWwYWpQ5oxeXBTTiVfo+ekjUxceZgMc6/unctBtw/g8ZXgUgF+HAALRsDlM5YtXORLgl4IcZNSiodCqrPi+fZ0a1idz1YeotekTexPLMDVvXeo8aBVx9cg5neYHAZ75stQTCuSoBdC3KaSqxNfDGrClMeacSY1veBX9w5O0P5leGIDePjDL6Nh3qOQaubYfVGoJOiFEHfUtWE1VjzfjodC7vLqvkp9GLncmPr4yEr4qgXsWSBX90VMgl4IkadKrk5MHNiEb4YYV/e9Jm/kq7VHzJskDYyJ0lo/C2M3gHsdYxrknx6Tq/siJEEvhDBLl6Bq/PV8O+4PrMpHyw7S/5stxJ27Yv4BPOvBqL+Mq/vDK4yr+32/WK5gcZMEvRDCbO6uTkwe3JSJAxtzOCmVB7/YwJxtx81/qvbWq/tKvrBwBPwyBq5dsmjdpZ0EvRCiQJRS9GrsxfLn29G0ViVe+3UfI2btMH/OHPj76r7Dq8bkaF+3htj1liu6lJOgF0LcleoVyvDdyDAm9Api67HzPPD5epbtO23+AewdocMrMGoFODjD7B6w/DXjCVtRqCTohRB3zc5OMTTchz+faUvNSmUZ+0Mkr/6yh6vXzZzvHsC7mdGV0/xx2DLJmPP+9F6L1VwaSdALIe6Zn2c5fh7XinEd6jBvx0m6f7GRvfEFWJzEyRUe+p8xZ861C/BtZ9g2VYZhFhIJeiFEoXBysOPfXesz5/EWXL2eRZ+vNzFl3VGyzR2GCcacOeM2g197WPoSzBsMVy9YruhSQoJeCFGoWtWpzLLn2nJfg6p8sDSGR6dtI6kgN2pdK8Pg+dDlfWMY5tetIXaD5QouBSTohRCFrmJZJ756tCkfPhJM1MlLdJu4gbUHCzC5mVIQ/qQxQZpTWeNG7ep3IKsAff/iJgl6IYRFKKUY0LwWvz3dGs9yzgyfuYMPlsaYP18OQI3GxgRpjQfD+o9h1kOQkmixmm2VBL0QwqL8q7ixeHxrBreoxZR1RxnwzRbiL141/wDO5eDhr6DPNGM0zpS2cGytxeq1RRL0QgiLc3G0573ewXw5qAmHki7z4MQNLI8uwJh7gJB+MGYNlPWA7x421qnNLsC/DkoxCXohRJHp0agGfzzThtoerjzxfST//X1/wbpyPOsZC5MH94U178Dc/jIqxwwS9EKIIlXbw5WF48IZFl6b6RtjGfzt1oKNynEuB32+hYc+hdh18E07iI+wXME2QIJeCFHknB3sebtXQyYObMy+hBQe+mIjW46eN/8ASkHzUcZc9yiY2Q12fmexeks6CXohhNX0auzF4vGtKV/GgUenbWXKuqPmz4QJ4NUUnlgHtVvDkqfhz5cgK8NyBZdQEvRCCKuqW9WNJePb0C24Oh8sjeGJ7yNJSStAWJd1N6ZOCB8P26fC973hyjnLFVwCSdALIayunLMDkwY14Y3ugayOOcPDkzZx5Mxl8w9g7wBd3oXeU+HkdpjaEU7tsVzBJYwEvRCiWFBKMbKNL3NHtyT5Wga9J29i1YECLjfYaACMXAY6C6Y/YMx1LyTohRDFS5ivO0uebkPtymV5/LsIJq85UvB++zFroXoj+HkUrHm/1M+CKUEvhCh2vCqWYcETrejZqAYfLz/I+Lm7CjbHfbkqMOw3aPworPsAfh0LmemWK7iYc7B2AUIIkZsyTvZ8PqAxQTXK88HSGI6evcy3Q0Op6V7WvAM4OEGvycbatGvegeR4GPC9cfO2lJEreiFEsaWUYky7OswcEUbipWv0mryJyOMFeBJWKWj/kjFPTvx2o9/+wjHLFVxMSdALIYq99nU9WfRUayqUcWTQt9tYHJVQsAOE9IOhi+HqOZh2nzEypxSRoBdClAh+nuX4ZVwrGtesyLPzovh85aGC3aSt3QpGrQSXCjCrOxz4zXLFFjMS9EKIEqOSqxM/jGrBI029+XzlYZ77KYq0jCzzD1DZ3wj76iEwfyjs/N5yxRYjEvRCiBLFycGOT/qF8FKXeiyOSuTRads4f7kAI2pcPYxuHL8OsGQ8bP7SYrUWFxL0QogSRynFUx39mTy4KfsSkun91WZiz10x/wBOrjDoJwjqDX+9Divfsumx9hL0QogS66GQ6swb05LL6Zk88vVmdp+8ZP7ODk7wyHRoNhw2fga/PwfZBegGKkEk6IUQJVqTWpVYODacsk72DPp2a8EWIbezh+6fQ5sXIHIWLBxpkw9WmRX0SqmuSqmDSqkjSqlXcnnfWSn1k+n9bUopnxzv11JKXVZKvVhIdQshxE1+nuX45clW+Hi48vjsCH7ZGW/+zkrBfW/CA+/A/kUw71GbC/t8g14pZQ9MBroBgcAgpVRgjmajgItaa3/gM+DDHO9/Ciy993KFECJ3Vdxc+OmJloT5uvPC/N0Fn9u+1dPG1f2RFfDTEJsKe3Ou6MOAI1rrY1rr68A8oFeONr2A2aafFwKdlVIKQCn1MBALRBdKxUIIcQduLo7MHNGcHo1q8MHSGCb8vp/s7AKEfegI6P4ZHF5uDL+0kbA3J+i9gJO3vI43bcu1jdY6E0gGPJRS5YB/A2/n9QFKqTFKqQilVMTZs2fNrV0IIW7j7GDPxAGNGdnal5mb4nhp4R4yC7IAeehIYz3aQ8tgwXDIvG6xWouKpW/GvgV8prXOcwUBrfVUrXWo1jrU09PTwiUJIWydnZ3iP90b8Px9dfl5ZzzP/hRFRkHCvvkoePATOPinTYS9ObNXJgA1b3ntbdqWW5t4pZQDUAE4D7QA+iqlPgIqAtlKqTSt9aR7LVwIIfKilOLZ+wIo62TPu38eID0ji0mDm+LiaG/eAcJGG3//+SIsHAH9ZoG9o8XqtSRzruh3AAFKKV+llBMwEFiSo80SYJjp577Aam1oq7X20Vr7AJ8D70nICyGK0uh2fvy3VxArD5zh8dkRBZvXPmw0dPsIYn6Hnx8vsePs8w16U5/7eGA5cACYr7WOVkpNUEr1NDWbjtEnfwR4AbhtCKYQQljLkHAfPu4bwuaj5xg2YzupBVl8vMUT8MC7xtDLpf8ukU/QqgINPyoCoaGhOiIiwtplCCFs0G+7E3n+pyiCapRn9sgwKpZ1Mn/nFW/AponQ6T/Qrvg9EqSUitRah+b2njwZK4QoNXo0qsGUx5px4FQqQ2dsJ6UgV/ad34KQgbD6v7DrB4vVaAkS9EKIUuW+wKp8/VhT9iemMHzGdi6nm9lnb2cHvSZBnU6w5Bk4tNyyhRYiCXohRKnTuUFVJg1uwu74ZEbO2mH+DVp7R+j/HVQLhvnDIL5kdDNL0AshSqWuDavz2YDGRMRdYPR3EeYvYOLsBo8uALdqMKcfnDts2UILgQS9EKLU6tmoBh/3bcTmo+cZ+0Mk6Zlmhn25KjDkF2P2yx/6wJXzli30HknQCyFKtUeaefNe72DWHjzL+Lm7zH+C1t0PBv8EqUnGA1VZBRifX8Qk6IUQpd6gsFpM6BXEiv1JvDB/t/kToXk1MyZBi10HqydYtsh7YM4UCEIIYfOGhvtwJT2LD5fFUNXNmde755yN/Q6aPAoJkcYY+xpNjOUJixkJeiGEMBnb3o+klDSmbYylWgUXHm/rZ96OXT+A03th0VNQuR5UNfOXRBGRrhshhDBRSvGf7oE8GFyNd/44wOKonPM33oGDkzHs0skVfnoMrl2yaJ0FJUEvhBC3sLdTfNq/MWG+7ry4YDebjpwzb8fy1Y2wv3Qcfn0CsgswLbKFSdALIUQOLo72fDskFN/KrjzxfSTRicnm7Vg7HLq8byxasv5jyxZZABL0QgiRiwplHZk9Mgw3FweGz9zByQtXzdsxbLQxJ87a9+HYOssWaSYJeiGEuIPqFcowe2QY6RlZjJi1w7zpjZUyhly6+8Hi8ZCeavlC8yFBL4QQeahb1Y0pQ5oRe+4Kz/9k5hh7p7Lw8FeQfBJWvGn5IvMhQS+EEPloVacyrz/UgJUHkvh8lZlz29RqCeFPQcR0q3fhSNALIYQZhrfyoW8zb75YdZhl+06Zt1PH18C9jtW7cCTohRDCDEop3nm4IY1qVuSF+bs5eNqM4P5HF84bli/yDiTohRDCTC6O9kwd0gxXZwdGfxfBpavX89/pZhfODDi21uI15kaCXgghCqBqeRemPNaM08lpPP3jLjLNme2y0+vg4Q+Ln7ZKF44EvRBCFFCz2pX478NBbDh8jg+XxeS/g2MZ6DXZal04EvRCCHEXBjSvxdDw2ny7IZaV+5Py3+HWLpzY9ZYv8BYS9EIIcZdee6gBgdXL8/LPeziTmpb/Dp1ehwo1jbH12sw57wuBBL0QQtwlZwd7vhjUmCvpmby4YE/+D1M5loF2L0HiTmM+nCIiQS+EEPfAv4obr3cPZP2hs8zaHJf/Do0HQyUfWPNukc1wKUEvhBD36LEWtehcvwofLI3hwKmUvBvbO0L7V4yFSmJ+K5L6JOiFEOIeKaX4sG8I5cs48ty8KNIysvLeIaQ/eATAmvchO5+2hUCCXgghCkHlcs580i+Eg0mpfLA0nyGXdvbQ4RU4ewCif7V4bRL0QghRSDrUq8LwVj7M2hzHmoNn8m4c1AeqBBrz1mdlWrQuCXohhChEr3SrT72qbry0YDfnL6ffuaGdHXR4Fc4fgb3zLVqTBL0QQhQiF0d7Jg5qzKWrGfk/NdugB1QLgXUfQpYZi5rcJQl6IYQoZPWrlWdkG1/mR8Sz88TFOzdUypjK+GIcRM2xWD0S9EIIYQHPdA6ganln3li8j6y8HqSq2wW8QmHdx5CZR1fPPZCgF0IICyjn7MDrDwWyLyGFudtP3LmhUtDpNUiJh53fWaQWCXohhLCQ7iHVaVXHg4+XxeR9Y9avI9RuDUn7LFKHBL0QQliIUoq3ewZx9XpW3jdmlYLHfoYeEy1ShwS9EEJYUEBVN/NuzDqWsVgNZgW9UqqrUuqgUuqIUuqVXN53Vkr9ZHp/m1LKx7T9fqVUpFJqr+nvToVcvxBCFHs3bsz+Z1E+N2YtJN+gV0rZA5OBbkAgMEgpFZij2SjgotbaH/gM+NC0/RzQQ2sdDAwDvi+swoUQoqS4cWM2OjGFuduOF/nnm3NFHwYc0Vof01pfB+YBvXK06QXMNv28EOislFJa611a60TT9migjFLKuTAKF0KIkuTmjdnlB/O+MWsB5gS9F3Dyltfxpm25ttFaZwLJgEeONo8AO7XWt52hUmqMUipCKRVx9uxZc2sXQogS48aN2SvXs/hq7dEi/ewiuRmrlArC6M55Irf3tdZTtdahWutQT0/PoihJCCGKXEBVN3qEVGfe9hMkX7XclAc5mRP0CUDNW157m7bl2kYp5QBUAM6bXnsDvwJDtdZF+2tMCCGKmTHt6nDlehY/FGFfvTlBvwMIUEr5KqWcgIHAkhxtlmDcbAXoC6zWWmulVEXgD+AVrfWmQqpZCCFKrMAa5WlX15OZm+LyX6CkkOQb9KY+9/HAcuAAMF9rHa2UmqCU6mlqNh3wUEodAV4AbgzBHA/4A28opaJMf6oU+lkIIUQJMradH+cup7NoV87OEctQWhf9mM68hIaG6oiICGuXIYQQFqO1puekTVxJz2TlC+2xs1P3fEylVKTWOjS39+TJWCGEKGJKKZ5o78exc1dYcSDJ4p8nQS+EEFbQNagatdzLMmXdUSzdsyJBL4QQVuBgb8fotr7sOnGJiON5zIFTCCTohRDCSvo2q4m7qxPfrLPsyHMJeiGEsJIyTvYMDa/NygNnOJyUarHPkaAXQggrGhrug4ujHVPXH7PYZ0jQCyGEFbm7OjEgtCaLohI4nZxmkc+QoBdCCCt7vK0fWdmamZtiLXJ8CXohhLCymu5lGdbKh+oVXCxyfAeLHFUIIUSBvNkjyGLHlit6IYSwcRL0Qghh4yTohRDCxknQCyGEjZOgF0IIGydBL4QQNk6CXgghbJwEvRBC2Lhit5SgUuoscC/Lo1cGzhVSOSWJnHfpIudduphz3rW11p65vVHsgv5eKaUi7rRuoi2T8y5d5LxLl3s9b+m6EUIIGydBL4QQNs4Wg36qtQuwEjnv0kXOu3S5p/O2uT56IYQQ/2SLV/RCCCFuIUEvhBA2zmaCXinVVSl1UCl1RCn1irXrsRSl1Ayl1Bml1L5btrkrpVYopQ6b/q5kzRotQSlVUym1Rim1XykVrZR61rTdps9dKeWilNqulNptOu+3Tdt9lVLbTN/3n5RSTtau1RKUUvZKqV1Kqd9Nr0vLeccppfYqpaKUUhGmbXf9XbeJoFdK2QOTgW5AIDBIKRVo3aosZhbQNce2V4BVWusAYJXpta3JBP6ltQ4EWgJPmf4/tvVzTwc6aa0bAY2BrkqplsCHwGdaa3/gIjDKeiVa1LPAgVtel5bzBuiotW58y/j5u/6u20TQA2HAEa31Ma31dWAe0MvKNVmE1no9cCHH5l7AbNPPs4GHi7KmoqC1PqW13mn6ORXjP34vbPzcteGy6aWj6Y8GOgELTdtt7rwBlFLewEPANNNrRSk47zzc9XfdVoLeCzh5y+t407bSoqrW+pTp59NAVWsWY2lKKR+gCbCNUnDupu6LKOAMsAI4ClzSWmeamtjq9/1z4GUg2/Tag9Jx3mD8Mv9LKRWplBpj2nbX33VZHNzGaK21Uspmx8wqpcoBPwPPaa1TjIs8g62eu9Y6C2islKoI/ArUt25FlqeU6g6c0VpHKqU6WLkca2ijtU5QSlUBViilYm59s6DfdVu5ok8Aat7y2tu0rbRIUkpVBzD9fcbK9ViEUsoRI+TnaK1/MW0uFecOoLW+BKwBwoGKSqkbF2q2+H1vDfRUSsVhdMV2AiZi++cNgNY6wfT3GYxf7mHcw3fdVoJ+BxBguiPvBAwElli5pqK0BBhm+nkYsNiKtViEqX92OnBAa/3pLW/Z9LkrpTxNV/IopcoA92Pcn1gD9DU1s7nz1lq/qrX21lr7YPz3vFpr/Sg2ft4ASilXpZTbjZ+BB4B93MN33WaejFVKPYjRp2cPzNBav2vdiixDKfUj0AFj2tIk4E1gETAfqIUxxXN/rXXOG7YlmlKqDbAB2Mvffbb/h9FPb7PnrpQKwbjxZo9xYTZfaz1BKeWHcaXrDuwCHtNap1uvUssxdd28qLXuXhrO23SOv5peOgBztdbvKqU8uMvvus0EvRBCiNzZSteNEEKIO5CgF0IIGydBL4QQNk6CXgghbJwEvRBC2DgJeiGEsHES9EIIYeP+H+dXtBIgkZglAAAAAElFTkSuQmCC", + "text/plain": [ + "
    " + ] + }, + "metadata": { + "needs_background": "light" + }, + "output_type": "display_data" + } + ], + "source": [ + "scheduler = PolyLRScheduler(optimizer, t_initial=t_initial)\n", + "plot_lr(scheduler, label='default')\n", + "\n", + "scheduler = PolyLRScheduler(optimizer, t_initial=t_initial, lr_min=0.01)\n", + "plot_lr(scheduler, label='lr_min=0.01')\n", + "plt.legend();" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### `k_decay`" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "`k_decay` k_decay rate." + ] + }, + { + "cell_type": "code", + "execution_count": 13, + "metadata": {}, + "outputs": [ + { + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXoAAAD4CAYAAADiry33AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/YYfK9AAAACXBIWXMAAAsTAAALEwEAmpwYAABL3ElEQVR4nO3dd1gUV9/G8e+hFxEQO6CgoEizYO819hprjIoaTaLRFFNM8kSNr3limqYZY4klib333qNGQUUULFhQwYYognTY8/4xxMcYjCgLuyznc117sWV25je43jucOXOOkFKiKIqimC4zQxegKIqiFCwV9IqiKCZOBb2iKIqJU0GvKIpi4lTQK4qimDgLQxfwuNKlS0sPDw9Dl6EoilKkHDt27I6Uskxurxld0Ht4eBAaGmroMhRFUYoUIcSVJ72mmm4URVFMnAp6RVEUE6eCXlEUxcQZXRu9oijGJTMzk5iYGNLS0gxdigLY2Njg5uaGpaVlnt+jgl5RlH8VExODg4MDHh4eCCEMXU6xJqUkPj6emJgYPD098/y+PDXdCCE6CCHOCSEuCCHG5/J6cyHEcSFElhCi92OvDRFCROXchuS5MkVRjEJaWhouLi4q5I2AEAIXF5dn/uvqqUEvhDAHZgAdAV9ggBDC97HFrgLBwOLH3lsKmAg0AOoDE4UQzs9UoaIoBqdC3ng8z79FXppu6gMXpJSXcjayFOgORP61gJQyOuc13WPvbQ/skFLezXl9B9ABWPLMlT5FdMxxvt32Ca86++Pj7IQwMwchQJhpNzMLMLME85zbX/ctrMHCFixt/vfT0hYs7cCqhPbTTJ2zVhSl6MpL0LsC1x55HIN2hJ4Xub3X9fGFhBAjgZEAlSpVyuOq/+589Fl2WVyldvRJaoQnPdc6cifAyl4LfSt7sHYAG0ftZuv0v/s2TmBXCuxc/n6zsNZjLYqiAEyaNIkSJUrw7rvv5vp6XFwcXbp0ISMjg++//55mzZo90/oXLFhAaGgoP/74I2vXrqVatWr4+j7ekFF0GMXJWCnlbGA2QN26dZ9rJpQXmr6EX8JGFljfZ7vDRxy5FA9SEuhakm6B5ejsV4YyduaQnQHZmaDL1H5mpUNWGmSmaresVMhMg8xkyMi5pT+AjKSc+0mQdh/u3ILUBO1+VuqTC7MqAfaloUQ5KFFW+2lfVrvvUB5KVgSHitqXgvrLQVH0YteuXQQEBDB37tx8r2vt2rV06dLF5IM+FnB/5LFbznN5EQu0fOy9e/P43mfWu1ovPj38Kd/1cKCcdRAbTl5nbVgsk7ZcZvLWyzT1LkPP2hV5wbcC9tZ6/I7LStdCP/UupMQ/drsLyXHw4BbciYLoPyD13j/XYW6VE/yuWvg7uoOTOzhV/t99K3v91awoRcxnn33GwoULKVu2LO7u7gQFBXHx4kVGjx5NXFwcdnZ2zJkzh7S0NN5//31SU1MJDQ3l8OHDvPPOO4SEhJCamkrv3r359NNPgf8NuVK6dGlCQ0N599132bt378NtHjp0iPXr17Nv3z6mTJnCqlWrqFq1qoF+A88vL2kXAngLITzRgrs/8FIe178N+O8jJ2BfAD585irzqINHB74M+ZLVUauZ1HgSrzSrwivNqnDh9gPWhcWy5kQsby87ia3laV7wK0eP2q408yqNhXk+j6QtrMGhnHbLi6wMSL4NSbcg6TokXofEWEi8od2PPQaR67W/Oh5l5wLOnlDK858/S5TTzkkoSgH6dEMEkdcT9bpO34olmdjV71+XOXbsGEuXLiUsLIysrCzq1KlDUFAQI0eO5Oeff8bb25sjR44watQodu/ezeTJkx82vYD2JVGqVCmys7Np06YN4eHhBAYGPrW2xo0b061bN7p06ULv3r2furyxemrQSymzhBBvoIW2OTBPShkhhJgMhEop1wsh6gFrAGegqxDiUymln5TyrhDi/9C+LAAm/3VitiCUsCpBu8rt2Bq9lffrvY+dpR0AXmVLMO6F6rzTrhrHrtxjzYlYNobfYF3YdUqXsKJrzYr0qu2Gv2vJwuldYGEFjm7ajaDcl9Fla38FJFyD+9cg4SokXIF70XDtCJxeBfKRc99WDuBSFVy8oLT3Iz+9wcqu4PdJUQrQgQMH6NmzJ3Z22me5W7dupKWlcejQIfr06fNwufT09Fzfv3z5cmbPnk1WVhY3btwgMjIyT0FvKvLUfiGl3Axsfuy5CY/cD0FrlsntvfOAefmo8Zn08u7F+ovr2XFlB929uv/tNSEEdT1KUdejFBO6+rL3XBxrT8Sy6M+rzD8YjVfZEvSs7UqP2q64OtkWVsm5MzPXmnBKViTXc99ZGdoXwN3LcPcSxF+A+Ci4dlT7EuCvUx0CnCtDmRpQpjqUrQFlfLT7lgbeR6XIedqRd2HS6XQ4OTkRFhb2r8tdvnyZr7/+mpCQEJydnQkODn7YD93CwgKdTjtgMuUrf43iZKw+1Slbh8olK7Pmwpp/BP2jrC3Mae9XnvZ+5bmfksmmUzdYcyKGr7ad46tt52hYpRS96rjR0b88DjZ5v9S40FhY5RzB59JemJmqhf+dKIg7B3FntJ8Xdv6vOUiYQamqUN4fyvlD+QDtZ8mKqglIMTrNmzcnODiYDz/8kKysLDZs2MCrr76Kp6cnK1asoE+fPkgpCQ8Pp2bNmn97b2JiIvb29jg6OnLr1i22bNlCy5YtAa2N/tixY3Ts2JFVq1blum0HBweSkvTZk6/wmVzQCyHo4dWD745/x5XEK1QuWfmp73G0s+SlBpV4qUElrt1NYc0JrT3//ZXhTFh3mvZ+5elVx40mVV3y355fGCxtoZyfdntUdqb2BXD7jHa7dRpij0PEmv8tY+sMFWpChVpQsZb209lDhb9iUHXq1KFfv37UrFmTsmXLUq9ePQAWLVrE66+/zpQpU8jMzKR///7/CPqaNWtSu3ZtfHx8cHd3p0mTJg9fmzhxIsOHD+eTTz55GP6P69+/PyNGjOD7779n5cqVRfJkrJDyuXozFpi6devK/E48EpcSR7uV7Qj2C+atoLeeax1SSk5cS2D18Rg2nLzB/dRMyjpY06O2K73quOJTvmS+ajQqaYlwK0IL/pvhcOMk3Ir839G/jaMW/q5B4FoX3OpqPYSUYuHMmTPUqFHD0GUoj8jt30QIcUxKWTe35U0y6AHe2PUGkfGRbO+9HQuz/P3hkp6VzZ6zt1l1PJY9Z2+TpZP4VSzJi3Xc6F6rIi4lTPCiqKx0uB0J18PgRhhcP6F9GeiytNdLuoFbTvC7N9CO/tXFYSZJBb3xedagN7mmm7/09O7Jvph9HIw9SAv3Fvlal7WFOR38K9DBvwJ3kzNYHxbLquOxTN4YyX83n6Fl9bL0DnKltU85rCyKQNNOXlhYQ8Xa2u0vmalwIxxiQ7UuoDGhELlOe808Z/lKDcC9oRb+9i6GqV1RlL8x2aBv7tacUjalWHNhTb6D/lGl7K0IbuJJcBNPzt9KYtWxGNaciGXnmVs421nSvZYrvYPc8KtYSF01C5OlrRbklR7pBfQgTuvuee1PuHoEDv8EB7/TXitdDTyaQuUm2k/V3KMoBmGyTTcA34R+w++Rv7Ojzw5K25bWyzpzk62THIiKY9XxWLZF3CQjS0f1cg70DnKje+2KlHWwKbBtG53MNK2Z59qfcOUQXDmsDR8BWt/+yk3AoxlUaaENA6EYPdV0Y3xUG/0jLiVcovu67owLGkewf7Be1vk091Mz2Rh+nZXHYjhxNQFzM0HLamXoU9fNtJp28io7SzvBe+WgNvzDlcOQfl97rayfFvhVWkLlxtqAcYrRUUFvfFTQP+blzS+TlJHE2u5rC70p5cLtB6w8FsPq4zHcTkqnlL0V3WtVpE+QO74VTajXzrPQZWu9ei7vg0t7teDPTteGkXarB1XbgFcbrVunGuTNKKigNz4q6B+zOmo1Ew9N5LeOv1GrbC29rfdZZGXrOBB1hxXHrrEj8haZ2RJ/15L0CXKne62KONlZGaQuo5CZqrXxX9oLF3drXwIAdqW1wPdqC1VbayOAKgahgt74qKB/THJmMu1WtKNCiQos7LCQElYl9Lbu53EvOYN1YbGsOBZDxPVErCzMaO9Xnr513WhStTRmZiZ2AvdZPYjTAv/CTri4SxsBFKH14a/eAap10K7gNbUT3UbMGII+OjqaLl26cPr06Wd6X8uWLfn666+pWzfX/Cs0KSkp9OnTh4sXL2Jubk7Xrl2ZOnXqc69Pda98jL2lPV+1+IrRu0bzzt53mNF2BpZmhhvSwPmRXjsR1++zIlTrtbPh5HVcnWzpHeRG7yA33EsV04HISpSBmv20m06n9eGP2gHnt8LuKdrN0R2qtddC37O56r+vFAnvvvsurVq1IiMjgzZt2rBlyxY6duxYKNs2+aAHaOLahImNJjLh0AQmHZrElCZTjKLro19FR/y6OTK+ow87Im+xPPQa3++O4vvdUTSpWpp+9dx5wa8c1hbmhi7VMMzMwLWOdmv5ASTdhKjtcG4rhC2GkLnaqJ3e7aBGF/B+QZ3QLWhbxsPNU/pdZ/kA6Jj3o9tLly7x4osvMnv27IdDIfwlNTWVoUOHcvLkSXx8fEhN/d+kQNu3b2fixImkp6dTtWpV5s+fT4kSJQgJCeHNN98kOTkZa2trdu3aRXx8PIMGDSI5ORmAH3/8kcaNGzN48GB69epFjx49ABg4cCB9+/ale/cnj6sFYGdnR6tWrQCwsrKiTp06xMTE5Hmf86tYBD1oF1DdSL7BzJMzqViiIqNrjTZ0SQ/ZWJrTtWZFutasSGxCKitDY1geeo0xS07gZGdJj1qu9KvnTo0KxfQE7l8cykOdwdotMw0u74ezG+HsJohYrV20VaUl1OgK1TupC7ZM0Llz5+jfvz8LFiz4x5g2ADNnzsTOzo4zZ84QHh5OnTp1ALhz5w5Tpkxh586d2Nvb88UXXzBt2jTGjx9Pv379WLZsGfXq1SMxMRFbW1vKli3Ljh07sLGxISoqigEDBhAaGsrw4cOZPn06PXr04P79+xw6dIiFCxdy7tw5+vXrl2vNe/fuxcnJ6eHjhIQENmzYwJtvvlkgv6PcFJugB3i95uvcSL7Bzyd/poJ9BXp59zJ0Sf/g6mTLm229GdPai4MX77As5BqLj1xlwaFoaro50q9eJbrVqkgJfc6QVRRZ2kC1F7Rbl+naCd0zG+DMRojaBsJc67rp1xN8umjz+Sr59wxH3voWFxdH9+7dWb169ROn9du/fz9jx44FIDAw8OGY83/++SeRkZEPBzTLyMigUaNGnDt3jgoVKjz8y6BkSe1gKjk5mTfeeIOwsDDMzc05f/48AC1atGDUqFHExcWxatUqXnzxRSwsLKhevfpTh0sGyMrKYsCAAYwdO5YqVark6/fxLIpVWgghmNBoArdTbjP58GTK2pWlqWtTQ5eVKzMzQTPvMjTzLsO95AzWnIhlWcg1PlpziimbIukSWIH+9StR293JKJqhDMrMXOuHX7kxtP+v1nMncq02Kuf6MbDxbajSKif0O2uTuitFjqOjI5UqVeKPP/545vlbpZS0a9eOJUuW/O35U6dyb4aaPn065cqV4+TJk+h0Omxs/nfR4+DBg/n9999ZunQp8+fPB8jzEf3IkSPx9vbmrbfeeqb6801KaVS3oKAgWdCS0pPki+telPV/ry8j7kQU+Pb0RafTyWNX7sr3V5yUNT7ZIit/sFG+MG2f/OXAJXkvOd3Q5RkfnU7K2ONSbv9Eyun+Uk4sKeXk0lIueUnKiLVSZqQausIiITIy0tAlyMuXL0s/Pz/54MED2aRJE7lo0aJcl/vmm2/k8OHDpZRSnjp1Spqbm8uQkBB5+/Zt6e7uLqOioqSUUj548ECeO3dOpqenS09PT3n06FEppZSJiYkyMzNTvvXWW/Lrr7+WUko5b948qUWl5ubNm7JSpUqyfv36z7QPH3/8sezVq5fMzs5+5v1/XG7/Jmgz/uWaqwYP9sdvhRH0Ukp5K/mWbLuirWy8uLE8cetEoWxTnxJTM+SiP6/Ibj8ckJU/2Ci9P94s31p6Qh65FC91Op2hyzM+Op2U10Kk3DJeyi+9tND/r7uUa0dLeWmflHr4z2eqjCnopZTy3r17sm7dunLdunX/WC4lJUX269dP+vj4yJ49e8r69evLkJAQKaWUu3btknXr1pUBAQEyICDg4fuPHj0qGzRoIAMDA2WDBg1kUlKSPH/+vAwICJCBgYHy/fffl/b29n/bTvv27eXMmTPzXP+1a9ckIH18fGTNmjVlzZo15Zw5c6SUUq5bt05+8sknz/T7eNagN/l+9P8mJimG13a+xs3km3zV/CtaVWpVKNvVt8jriSwNucqa47EkpWdRtYw9A+pX4sU6bjjbF+OLsZ4kO0u7MvfUCq1dP+MBlHSFwH5QayCU9jJ0hUbFGPrRG5OUlBQCAgI4fvw4jo6OBqlBXTD1jO6m3WX0ztFE3o1kQsMJvFjtxULbtr6lZGSxKfwGS45e5fjVBKzMzejgX56BDSpR37OUasvPTUYKnN8CJ5fBhR3ahOvuDaDWS1qbvo1h/iMbExX0/7Nz506GDx/O22+/Xfjt7I9QQf8cUjJTeGfvOxy8fpA3ar3ByMCRRT4Uz91MYsnRq6w6HkNSWhZeZUvkHOW7Fu8hF/5N0k0IXwYnFsGdc2Bhq3XVrDNYG2a5iH8mnpcxBv22bdv44IMP/vacp6cna9asecI7TIsK+ueUqctk4sGJbLi0gX7V+/Fh/Q8xNyv6FyqlZmSzMfw6i49e5cTVBKwtzOgcUIGBDStRp5Jzkf9CKxBSanPphi2C0ysh7b42kXpQsHakX8zG3THGoC/uVNDng07q+PbYt8yPmE+bSm34b9P/YmdpOkMRRF5PZPHRK6w9cZ0H6Vn4lHdgYMPK9KztqvrlP0lmqjaL1rEFcPUwmFlqV+EGBYNH82IxwqYKeuOjgl4Pfo/8na9Cv8KjpAfTW02nimPhXdhQGJLTs1h/8jq//3mFiOuJ2FuZ0722Ky83qFx8h0/Oi9tn4fivcHIxpN7TjvLrvaId5Ztw33wV9MZHBb2eHLlxhPf3v096djpTmkyhbeW2hi5J76SUnIy5z+9/XmHDyeukZ+moXcmJQQ0r0ymgAjaWRb/pqkBkpsGZ9dpYO9eOgKUdBPaFeiOgvL+hq9M7FfTGRwW9Ht1Mvsm4veMIvxPOUP+hjK09Fgsz02ziuJ+SycrjMSz68wqX7iTjbGdJ33ruDKxfmUouptN8pXc3TsLROVpXzaw0qNQI6o+EGt3A3DQ+Kyrojc+zBr3pNzDmQ3n78szvMJ9+1fsx//R8Xt3xKvGp8YYuq0A42lkyvKknu8a1YNErDWjg6cLcA5dp8fUehs4/yq4zt8jWGddBgVGoUBO6/wjvnIEXpkDSDVg5FL6vBQe/h9QEQ1doEqKjo/H3f/a/llq2bImxHDgeO3aMgIAAvLy8GDt2LLkdZO/duxdHR0dq1apFrVq1mDx5sl62rYL+KazMrfhPw/8wpckUTsadpO/Gvhy9cdTQZRUYIQRNvErz86Ag/vigFWNae3P6eiLDF4bS8us9zNp3kXvJGYYu0/jYlYLGY2DMcei/GJwqw45PYLofbPkA7l4ydIWKgb3++uvMmTOHqKgooqKi2Lp1a67LNWvWjLCwMMLCwpgwYYJetm0af1sWgu5e3aleqjrv7XuPV7a/QrBfMG/UfgMrc9Ptk17B0ZZ32lVjTGsvtkXc5NfDV/h8y1mm7ThPt5oVGdzIgwA3dUHR35iZawOn+XTWmnUO/wQhv8CRWdpzTd4C93pPXY2x+uLoF5y9e1av6/Qp5cMH9T94+oI5iuJ49Ddu3CAxMZGGDRsC2sBoa9euVROPGCOfUj4s67KMr0O/Zn7EfA7fOMzUZlOp6lTV0KUVKEtzM7oEVqRLYEXO3kzk18NXWHNcmw6xdiUnhjTyoFNABaws1B+If1OhJvSaBW0naSduQ3/Rxs+v3EQLfO92xfYirOdVVMejj42Nxc3N7eFzbm5uxMbG5rr84cOHqVmzJhUrVuTrr7/Gz88v/7+4Jw2CY6hbYQ1qll+7r+yWzZY0k0G/BcnfI38vdgOJJaRkyF8OXJItv9ojK3+wUQb93w75zfZz8tZ9NSLkE6UlSXlohpTf+GqDqv3UWMqTy6TMyjR0Zf/KWAY1K1u2rKxevbqMiHjyiLPdu3eXu3btevi4du3aMiQkRG7YsEG6uLg8HFCsRo0actiwYTI8PFw2btz4H+tJSEiQL7/8svT395c1a9aUtra2D1/z9fWVt2/fljNnzpTjxo3LU/0hISGyTZs2Dx/v379fdu7c+R/L3b9/XyYlJUkppdy0aZP08vLKdX3POqiZOqJ/Tq0qtSKgTAATDk5g6tGpHIg9wOTG2hj3xYGjrSXDmnoS3NiD/VFxLDwUzQ+7o/hpzwU6BlQguHFldeXt46xLQKNRWt/70yvh4HewegTs+j9o+hbUflnNf/svivJ49K6urn+bOjAmJgZXV9d/LPvXxCcAnTp1YtSoUdy5c4fSpfN5NfaTvgEMdSsqR/R/0el0csmZJTLotyDZaFEjuer8qmJ3dP+Xy3EP5KfrI6T/hK2y8gcbZZfvD8hVx67JtMwsQ5dmnLKzpTyzScrZrbUj/K99pPzzZykzUgxd2d8YyxF9UR+Pvl69evLw4cNSp9PJDh06yE2bNv1jmRs3bjzMjyNHjkh3d/dc80SNR28g0fej5ZAtQ6T/An/5yrZX5LXEa4YuyWAepGXKXw9Hyzbf7H3YrDNt+zl5K1E16+RKp5Pywm4pf+mgBf5X3lIe/EHK9AeGrkxKaVxBL2XRHI9eSq35xs/PT1apUkWOHj36YYDPnDnz4bp++OEH6evr+7CWgwcP5rouNR69AemkjhXnVjDt2DQkkjfrvMkAnwGYieJ5klJKyR8X7jD/YDS7z97G0lzQJbAiQ5t4EOjmZOjyjFP0H7DvC23ic7vS0ORNqD8CLG0NVpK6YOrviuJ49HlKICFEByHEOSHEBSHE+FxetxZCLMt5/YgQwiPneUshxEIhxCkhxBkhxIfPvktFh5kwo59PP9Z2X0udcnWYenQqQ7YM4VJC8exDLYQ27+284HrsebclAxtUZkfkLbr9eJDeMw+x+dQNsrJ1hi7TuHg0hSEbYNg2KB+g9cX/rpZ29W2Wun7B0Hbu3EmNGjUYM2aMwUL+eTz1iF4IYQ6cB9oBMUAIMEBKGfnIMqOAQCnla0KI/kBPKWU/IcRLQDcpZX8hhB0QCbSUUkY/aXtF+Yj+UVJKNlzawBdHvyAlK4Vgv2BGBo7E1sJwR2bGICktkxWhMcw/dJlrd1NxdbIluLEHfeu542hraejyjE/0Qdj9f9rImY6VoOUHENi/UIdXMMYjejUevZ7HuhFCNAImSSnb5zz+EEBK+fkjy2zLWeawEMICuAmUAfoDLwE9AUfgMNBQSnn3SdszlaD/S3xqPNOOTWP9xfW4lnDlowYf0dytuaHLMrhsnWTnmVvM++MyRy7fxc7KnD5Bbgxt4olHaXtDl2dcpISLu2D3FLh+Qhs1s/XH4NerUPrhnzlzBh8fH9WDykhIKTl79qzem25cgWuPPI7JeS7XZaSUWcB9wAVYCSQDN4CrwNe5hbwQYqQQIlQIERoXF5eHkooOF1sXPmv6GfPaz8Pa3JrRu0bz1p63uJl809ClGZS5maC9X3mWvdqIjWOa0sG/PIuPXqXVN3sZ+WsoIdF3MbbzRwYjBHi1hRF7oN8irQvmymEwpzVcPlDgm7exsSE+Pl79exgBKSXx8fF/6+6ZF3k5ou8NdJBSvpLzeBDQQEr5xiPLnM5ZJibn8UWgAVAdGAUEA87AAaCjlPKJjdamdkT/qMzsTBZGLmTWyVkIIXi95uu8XONlLM1VkwXA7cQ0fj18hd+PXCEhJZOabo4Mb1aFjv7lsTQvnie0c6XLhpNLYc9nkBgL3i9oV9+W08MVlLnIzMwkJiaGtLS0Alm/8mxsbGxwc3PD0vLvuWHIppsfgT+llL/lLDcP2CqlXP6k7Zly0P8l9kEsU49MZW/MXjxKevB+vfdp5tbM0GUZjZSMLFYdj2XeH5e5fCeZio42DG3iSf/67jjYqC/FhzJTtTF0DkyD9ESoNRBafQSO/7wQRzF9+Q16C7STsW2AWLSTsS9JKSMeWWY0EPDIydheUsq+QogPAB8p5VAhhH3Oe/tLKcOftL3iEPR/2R+zn69CviI6MZrmbs15v977VC5Z2dBlGQ2dTrL77G3mHLjEkct3cbC2YECDSgQ39qCiU/E+qf03KXfhwDdwdDYIc61LZpM3wUrNI1Cc5HviESFEJ+BbwByYJ6X8TAgxGa2D/nohhA3wG1AbuIsW5peEECWA+YAvIID5Usqv/m1bxSnoQWvOWXRmET+H/0x6djqDfAfxauCr2FuqE5KPCo9JYM6By2w+dQMBdK1ZkVeaeeJXseh0cStw96Jhx0SIXAslXaHtpxDQWw2cVkyoGaaKgDupd/ju+HesvbCW0ralGVN7DN2rdsfcTE3n96hrd1OYfzCaZSFXSc7IpqlXaV5tUYWmXqVVr5C/RB+ErePhZji41YeOU8E1yNBVKQVMBX0RciruFF+GfElYXBjVnKsxru44GldsbOiyjM791EwWH7nK/IOXuZ2Ujm+FkrzaogqdAypgoU7caidswxbDrsmQfBtqDtCO8B3KGboypYCooC9ipJRsv7Kd6cemE/sglqauTRkXNA4vZy9Dl2Z00rOyWXfiOrP2X+RiXDJuzra80tSTvvXcsbNSg7OSlqi13x+eoQ2j0PoTqDdcmyBFMSkq6IuojOwMlpxdwqyTs0jOSuZF7xcZVWsUpW3zOWSpCdLpJLvO3mbWvouEXrmHs50lwY09GdyoMs72pjsLWJ7duQCb34VLe6B8IHSeVqRnulL+SQV9EXcv7R4/n/yZ5eeWY2luyWDfwQz1H6pO2D5BaPRdft53iZ1nbmFrac6A+pV4pZmn6qkjpXaidutHkHQd6gyGNpPA3sXQlSl6oILeRFxJvMIPJ35gW/Q2StmUYmTgSPpW66suuHqC87eS+HnfRdaFXUcAPWq78lqLKniVdTB0aYaVnqSNkHn4J7ApCS9M0frgq5PZRZoKehNz+s5pph2bRsjNENxKuDG2zljae7QvtsMhP03MvRTmHrjM0pCrpGfpaO9bnlGtqqqhkm9FwqZ3tAHTPFtA12+hVBVDV6U8JxX0JkhKyR+xfzD9+HSi7kXhU8qHMbXH0My1mepm+AR3kzNYcPAyCw5Fk5iWRTPv0oxu5UUDz1LF93em08Gx+bBzEmRnQsvx0OiNQh0dU9EPFfQmLFuXzebLm5kRNoPYB7HUKVuHsXXGElRO9Zt+kqS0TBYducrcA5e58yCdoMrOjG5VlVbVyxbfwE+8Dpvfg7MbtZO13X6AirUMXZXyDFTQFwOZ2ZmsjlrNrPBZxKXG0cS1CWNrj8XX5dkmUS5O0jKzWR56jVn7LhGbkIpvhZKMae1Fe7/ymJkV08CPXK/1zkmOg8ZjoeWHYPlsIyUqhqGCvhhJzUpl6dml/HL6F+6n36dd5XaMqjlK9cH/F5nZOtaeiOWnvRe5fCcZ77IleKO1V/G9+Co1QZvZ6vivUMYHeswE1zqGrkp5ChX0xVBSRhILIxby+5nfSclMoaNnR16v+Toejh6GLs1oZeskG8OvM2PPBc7feoCHix2jWnnRs7Zr8RwmOWonrB8DD25Bs3eg+ftgoa5JMFYq6IuxhLQE5kfMZ8nZJaRnp9OlShdeq/ka7g7uhi7NaOl0ku2Rt/hhdxQR1xNxc7ZldCsvXqzjhpVFMQv81ARt3JyTS6BcAPScqc1lqxgdFfQKd1LvMO/0PJafW062LpvuXt0ZETgC1xJq7PInkVKy59xtvtsZxcmY+7g62TKqVVX6BLkXv8A/uxk2vAmpd7WeOU3fUcMoGBkV9MpDt1NuMyd8DquiViGlpId3D0YEjKBiiYqGLs1oSSnZez6O73ZGEXYtgYqONrzeyou+dd2wtihGYZdyFzaNg4jVUKkx9JoFTpUMXZWSQwW98g83k2/yy6lfVOA/AyklB6Lu8N2uKI5duUcFRxtGt/Kib91idIQvJYQvg03vgjCDLtO0Me8Vg1NBrzzR44GvmnSeTkrJwQvxTN95nmNX7uHqZMsbrb3oHeRWfE7a3r0Mq0dATIg2BHLHL7XhFBSDUUGvPNXjgd+1aldeCXiFSiXVn+ZPIqVkf9Qdpu84T9i1BNycbRnT2otedYpJ4Gdnwf4vYf9XWhNOr7lqREwDUkGv5Nmt5FvMj5jPyvMrydRl0tmzMyMCR+Dp6Gno0ozWX2343+44z8mY+1QqZcdbbb3pXssV8+Jw4dXVP2HVCG1EzDYTofEYNUCaAaigV57ZndQ7LDi9gOXnl5OWlUYHjw6MCByBt7O3oUszWlJqk5l/s/08kTcS8SpbgrfbVqOjfzG40jY1Ada/AWc2QPVO0OMnsHU2dFXFigp65bnFp8bza+SvLD27lJSsFNpUasOIwBH4ufgZujSjpdNJtkXcZNqO80TdfkCNCiUZ164abWqY+Fg6UsKRn2H7J1CyAvRZoOaqLUQq6JV8S0hLYNHZRSw6s4ikjCSauDbh1cBXqV22tqFLM1rZOsn6k7F8uzOKK/Ep1HJ34v321WnsZeIzhMUcgxXBkHRDG+u+wauqKacQqKBX9OZBxgOWnlvKrxG/ci/9HvXK12NEwAgaVmho2ker+ZCZrWPlsRi+3xXFjftpNPUqzXvtq1PT3cnQpRWclLuwdhSc3wK+3aH7DLAu5hO+FDAV9IrepWSmsCpqFQtOL+B26m38Xfx5JfAVWrm3UhOgPEFaZja//3mFn/Ze5G5yBu39yvHuC9XxLmeiASglHPoedn4Kpb2h/2JwqWroqkyWCnqlwGRkZ7D+4np+OfULMQ9iqOpYleEBw+no2RELMzV5RW6S0jL55Y/LzD1wmZSMLHrVcePtdtVwNdU5bS/t05pydNnw4lyo9oKhKzJJKuiVApely2Jb9DbmnprLhYQLuJZwZajfUHp498Da3NrQ5Rmlu8kZzNx7gYWHrwAwpFFlRrX0wtneBEeIvHcFlg2Em6eh9X+g2TjVbq9nKuiVQqOTOvZd28fcU3MJvxOOi40Lg3wH0a96P0pYlTB0eUbpekIq03ecZ+XxGEpYWzCqpRdDm3hgY2li4+hkpGjDHp9eCTW6aePcW6vPhL6ooFcKnZSSkJshzD01l8M3DuNg6UB/n/4MrDEQF1sXQ5dnlM7dTOLLrWfZdfY25Uva8HY7b16s42Zak59ICYd/hB0TtElN+i+GUupiPH1QQa8YVMSdCH45/Qs7r+zEytyKnl49CfYPVuPpPMGRS/FM3XqWE1cT8C5bgo861aBl9TKm1avp4m5YMRTMLLSwr9TA0BUVeSroFaNw6f4l5p+ez8ZLG5FS0tGzI0P9h1LNuZqhSzM6UmoXXU3dcpbo+BQaV3Xho0418Hd1NHRp+nPnAizuA/djtStp1SiY+aKCXjEqN5Nv8lvkb6w4v4LUrFSauzVnuP9w6pRT85I+LiNLx+IjV/huVxT3UjLpWduVd9tXN50eOsnxsOxluHoIWv0Hmr+rTtI+JxX0ilG6n36fpWeXsujMIu6l36NWmVoM8x9GC/cWqi/+YxLTMvlpz0XmHbwMwPCmnoxqWRUHG0sDV6YHWenaSdrwZdqQx12/AwvVU+tZqaBXjFpqViprotawMGIh15OvU9WxKsH+wXT27IyluQkEmR7F3Evh623nWBt2ndIlrBj3QnX61nUv+qNkSqkNd7znM6jcBPr9DnalDF1VkaKCXikSMnWZbIvexvzT8zl/7zzl7MoxyHcQvav1xt7S3tDlGZWwawlM2RhJ6JV7+JR34JMuvjQxhTF0Tq2Eta+DsycMWg2OboauqMhQQa8UKVJKDl4/yLzT8wi5GYKDlQP9q/fnpRovUdrWBMJMT6SUbDp1g6lbzhJzL5W2NcryUacaVClTxPumRx+EJf21sXFeXg1lfQxdUZGggl4pssLjwpl/ej67ru7C0sySHl49GOI3RM189Yi0zGzmHbzMT3sukp6VTXBjD8a08aZkUW6/v3kKfn8RsjPgpRVq5qo8yHfQCyE6AN8B5sBcKeXUx163Bn4FgoB4oJ+UMjrntUBgFlAS0AH1pJRpT9qWCnolN9H3o1kQsYD1F9eTpcuibeW2DPMfhn9pf0OXZjTiktL5ets5lh+7hou9Fe+396F3kFvRnfTk7mX4rSc8uAV9fwXvdoauyKjlK+iFEObAeaAdEAOEAAOklJGPLDMKCJRSviaE6A/0lFL2E0JYAMeBQVLKk0IIFyBBSpn9pO2poFf+zZ3UOyw6s4hlZ5eRlJlEvfL1GOo3lKauTU3rgqJ8CI9JYNL6CI5fTSDQzZGJXf0IqlxEZ3t6cFs7sr8dqQ2ZENjX0BUZrfwGfSNgkpSyfc7jDwGklJ8/ssy2nGUO54T7TaAM0BF4SUr5cl6LVUGv5EVyZjIrz6/kt8jfuJVyC29nb4b6DaWDZwcszYpwk4WeSClZF3adz7ec4VZiOj1ru/JhRx/KlrQxdGnPLi0Rlr4E0Qegw1Ro+LqhKzJK/xb0eems7Apce+RxTM5zuS4jpcwC7gMuQDVACiG2CSGOCyHef0KBI4UQoUKI0Li4uDyUpBR39pb2DPEbwpZeW/is6WdIKfnoj4/otLoTv0X+RkpmiqFLNCghBD1qu7J7XEtGt6rKpvAbtP5mH3MPXCIzW2fo8p6NTUkYuBJqdIWt4+GP6YauqMgp6KtSLICmwMCcnz2FEG0eX0hKOVtKWVdKWbdMmTIFXJJiSizNLelWtRuru61mRpsZuJVw48uQL2m7si3fH/+eO6l3DF2iQdlbW/Beex+2vd2cuh7OTNl0hs7fH+DwxXhDl/ZsLG2g9wLw7w07J8G+Lw1dUZGSl6CPBdwfeeyW81yuy+Q03TiinZSNAfZLKe9IKVOAzYC6zl3ROyEEzd2aM7/DfBZ3WkzDCg2Ze2ou7Ve259PDnxJ9P9rQJRqUZ2l75gfXY87guqRkZDNgzp+MWXKCm/ef2C/C+JhbQK/ZENhfu7Bq9xTtQivlqfLSRm+BdjK2DVqgh6C1u0c8ssxoIOCRk7G9pJR9hRDOwC60o/kMYCswXUq56UnbU230ir5cSbzCrxG/svbCWjJ1mbSu1Jqh/kOpWaamoUszqLTMbH7ae5Gf913E0kzwdrtqBDf2KDrDIeuyYcObcOI3aPIWtJ2kxsdBP90rOwHfonWvnCel/EwIMRkIlVKuF0LYAL8BtYG7QH8p5aWc974MfAhIYLOUMtd2+r+ooFf0LT41niVnl7Dk7BISMxKpU7YOQ/2H0tytebEeU+dqfAoT159mz7k4alQoyWc9/alTqYj0ztHpYPM4CJ0HDUdD+8+KfdirC6YUBW1C8zUX1vBrxK9cT76Op6MnQ/2G0rlKZ6zMTXD6vjz4azjkTzdEcjMxjQH1K/FBex8c7YpAzyUpYcsHcHQW1B8JHb8s1mGvgl5RHpGly2J79HYWRCzgzN0zlLEtw0s1XqJv9b6UtCpp6PIM4kF6Ft/uOM/8Q9E42Vryceca9KztavzXJkgJ2/+jzVrVeCy0m1xsw14FvaLkQkrJnzf+ZP7p+Ry+cRg7Czv6VOvDy74vU96+vKHLM4jI64l8vPYUJ64m0LiqC//tGYBHaSMfUE5K2PwuhMyFVh9Di39tHTZZKugV5SnOxJ9hQcQCtkVvQyDoVKUTwX7BeDt7G7q0QqfTSZaEXGXq5rNkZOsY28abkc2rYGnMJ2t1Olg3Ck4ugfafQ6NRhq6o0KmgV5Q8in0Qy2+Rv7E6ajWpWak0c23GUP+h1C1X1/ibMfTsVmIak9ZHsOX0TXzKO/B5rwBqG/PJ2uwsWDkUzqyHrt9D0BBDV1SoVNAryjNKSEtg6bmlLD6zmHvp9wgsHUiwfzCt3VtjbmZu6PIK1faIm0xYF8GtpDSGNPLg3fbVKWFtYeiycpeVAUsHwIVd8OLcYjUPrQp6RXlOaVlprLuwjgURC4h5EEPlkpUZ4jeEblW7YW1efKa7S0rL5Ktt5/jtzytUdLTl814BNK9mpFexZ6TAoj5w9bA2U5VPJ0NXVChU0CtKPmXrstl5dSfzTs8jMj6SUjalGFhjIP2q98PR2tHQ5RWaY1fu8v7KcC7GJdO3rhsfd/bF0dYIu2KmJ8Gv3bVx7QetAY+mhq6owKmgVxQ9kVIScjOEeafncfD6Qews7OhdrTeDfAcVm546aZnZfLcritn7L1G6hBX/7RlAmxrlDF3WP6XchXnttfHsh++AMtUNXVGBUkGvKAXg3N1zzI+Yz9bLWx/21BnqNxQvZy9Dl1YowmMSeH9lOGdvJtGjVkUmdvXD2d7ILjy7dwXmtgULG3hlBziY7pexCnpFKUCP99Rp4daCYf7DqFPO9Mfvy8jS8dPeC/y4+wJOdpZ83iuQdr5GdnR//QTM7wSlvSF4M1gX8Tl1n0AFvaIUgoS0BJacXcLis4tJSE+gdtnaDPMfVizG1DlzI5Fxy08SeSORF+u4MaGrkbXdn9+mTTju1Rb6L9FGwjQxKugVpRClZqWyJmoNCyMWcj35Ol5OXgz1H0pHz44mPftVRpaOH3dHMWPvRco6WPNV75o09S5t6LL+J3QebHwbgoKhy7cmN1SCCnpFMYBMXSbborcx7/Q8ou5FUd6+PEN8h9DLuxd2lnaGLq/AhF1LYNzyMC7GJTOoYWU+7OSDnZWRHEHvnKTNUNVmAjQbZ+hq9EoFvaIYkJSSA7EH+OXULxy/fRwnayde8nmJAT4DcLJxMnR5BSItM5uvtp1j3sHLVCplx7S+tYxjgnKdDlaPgNMrofc88H/R0BXpjQp6RTESJ26fYN6peeyN2YuthS0ver/IEL8hJts1889L8YxbfpKbiWmMae3FG628DD/BSVY6LOyq9bEfvh3KBxi2Hj1RQa8oRibqXhTzT89n8+XNCARdqnZhmP8wPB09DV2a3iWmZTJh7WnWhl2nTiUnvu1Xm0ouBm66SroJs1uCuRWM3At2pQxbjx6ooFcUIxX7IJaFEQtZHbWajOwM2lZuy/CA4fi5+Bm6NL1bFxbLf9aeRqeTTOrmR+8gN8MOFHctBBZ0gspN4OVVUMTHMFJBryhGLj41nkVnFrHk7BIeZD6gUYVGjAgcYXKjZsbcS+Gd5Sc5evkunQLK83nPQMPOZnVsIWwYC03e1CYtKcJU0CtKEZGUkcSyc8v4LfI37qbdpWaZmowIGEFzt+YmE/jZOsms/ReZtv085Ura8P2A2oY9Ubvxba3rZRE/OauCXlGKmLSsNNZeWMv80/O5nnydas7VGBEwgnaV25nMMMlh1xIYs+Q41xPSGPdCNV5rXhUzMwN8mWVl5JycDS/SJ2dV0CtKEZWpy2TL5S3MPTWXy/cvU7lkZYb7D6dLlS5Ymhf9i68S0zL5cNUpNp26QTPv0kzrW4syDgYY/jnpFsxuUaRPzqqgV5QiLluXze5ru5kTPoczd89Q3r48w/yH0dOrJzYWNoYuL1+klCw5eo1PN0RQ0taSb/vVoomXAa6ojQmF+R2hSkt4aXmRu3JWBb2imAgpJX/E/sHs8NmExYVR2rY0wX7B9KnWp8hfbXv2ZiJvLD7BxbgHjGnlxZttq2Fe2E05R2bDlvegwxfQ8LXC3XY+qaBXFBMjpST0Viizwmdx5MYRnKydeLnGywyoMYCSViUNXd5zS8nIYsK6CFYei6GZd2m+61+bUoU59LGUsGQAXNwFr+yCCoGFt+18UkGvKCYs7HYYc07NYX/MfkpYlmBgjYEM8h1UpGe+WhZylU/WRVDa3ooZA+sU7qTkyfEwszHYlNTa663sC2/b+aCCXlGKgTPxZ5gdPpudV3diZ2HHAJ8BDPYbTCmbondiEeB07H1eX3SMm/fT+KSLL4MaVi68LqaX9sKvPaDOYOj2feFsM59U0CtKMXL+3nnmhM9hW/Q2bCxs6FutL8H+wZS2NaIhg/Pofkom7ywPY9fZ23SrWZHPewVgb11II2H+NdJln4Xg16NwtpkPKugVpRi6lHCJOafmsPnyZizNLOlTrQ/D/IdRxq6MoUt7JjqdZOa+i3yz/RxeZUswZ3BdKrsUQnNKdqY252z8BXjtIDi5F/w280EFvaIUY1cSrzAnfA4bL23EwsyiyAb+H1F3GL34OAAzXqpTOJOa3L0EPzeH8v4wZKNRz0ylgl5RFK4lXmP2qdlsuLgBc2FOn+pa4Je1K2vo0vLsSnwyI389RtTtJD7qVIPhTT0Lvt3+5DJYMxJafggtxxfstvJBBb2iKA89Hvh9q/dleMDwItOGn5yexTvLw9gWcYtedVz5b88AbCwLeFiIVSMgYjWM3Kcd3RshFfSKovzDtaRrzA7XAt/SzJJ+1fsx1H8oLrYuhi7tqXQ6yQ+7LzB953lqujsx6+UgyjsW4BXCKXfhx3paO/0ru4xySGMV9IqiPNHVxKvMCp/FxksbsTa3ZoDPAIL9gnG2MYKp/55iW8RN3lkWhr21Bb8MqUeAWwFeO3B6FawcBi98Bo3fKLjtPCcV9IqiPNXl+5eZFT6LzZc2Y2thy8AaAwn2Dzb6K23P3Uxi2IIQ7iZn8P2A2rTzLVcwG5ISlvSHS/tg1GEoZVyzgamgVxQlzy4mXGTmyZlsi96Gg5UDwX7BvFzjZaMeS+d2UhojFoYSHnuf/3T2ZVgTj4I5SXs/FmY0ALcgGLTWqAY++7egz9MsvUKIDkKIc0KIC0KIf5x2FkJYCyGW5bx+RAjh8djrlYQQD4QQ7z7XHiiKUmiqOlXl6xZfs7LrSoLKBfHDiR/osKoDCyMWkpaVZujyclXWwYalIxvxgm85/m9jJBPXR5CVrdP/hhxdod0k7crZk0v0v/4C8tSgF0KYAzOAjoAvMEAI4fvYYsOBe1JKL2A68MVjr08DtuS/XEVRCkv1UtX5ofUPLOq0CJ9SPnwd+jWdV3dm2dllZOoyDV3eP9hamTNzYBAjm1fh18NXGPFrKA/Ss/S/oaBhUKkRbP0QHtzW//oLQF6O6OsDF6SUl6SUGcBSoPtjy3QHFubcXwm0ETl/NwkhegCXgQi9VKwoSqEKLBPI7BdmM6/9PFwdXJlyZArd13Zn06VN6GQBHDXng5mZ4KNONfispz/7o+7Q5+fD3ErU818hZmbQ9XvITIEtH+h33QUkL0HvClx75HFMznO5LiOlzALuAy5CiBLAB8Cn/7YBIcRIIUSoECI0Li4ur7UrilKI6pWvx8IOC5nRZga2FraMPzCevhv6sj9mP8Z2rm9gg8rMC67H1fhkev10iEtxD/S7gTLVoMX7Wt/6c8bfWJGnNvp8mARMl1L+629ZSjlbSllXSlm3TJmidVm2ohQnQgiauzVnRdcVfNHsC1KyUhi9azTBW4M5fuu4ocv7mxbVyrB0ZCPSMrPp8/NhTsXc1+8GGr8JZf1g4zuQrucvEj3LS9DHAo+O5uOW81yuywghLABHIB5oAHwphIgG3gI+EkIYXwdURVGeiZkwo1OVTqzrsY5PGn7C1aSrDNk6hDG7x3Ax4aKhy3sowM2RFa81wsbSnP6zD3Powh39rdzCCrp+C0nX4fCP+ltvAchL0IcA3kIITyGEFdAfWP/YMuuBITn3ewO7paaZlNJDSukBfAv8V0pp3L8RRVHyzNLMkr7V+7K512bG1h5L6M1Qeq3vxaRDk7iVfMvQ5QFQpUwJVo9qjJuzHcHzQ9h86ob+Vu5eH3x7wMHvIFGP69WzpwZ9Tpv7G8A24AywXEoZIYSYLITolrPYL2ht8heAdwDjHflHURS9s7WwZUTgCDb32sxLPi+x7uI6uqzpwnfHvyMpI8nQ5VGupA3LX21EoJsjoxcf5/c/r+hv5W0ngS4L9kzR3zr1TF0wpSiK3l1LusYPJ35gy+UtOFk78VrN1+hbvS+WZpYGrSs1I5s3Fh9n19nbfNDBh9dbVtXPird9DIdnwGt/GGzQs3xfMKUoivIs3B3c+bL5lyzrsoxqztWYenQqvdb1YvfV3QbtoWNrZc7Pg4LoXqsiX2w9yw+7ovSz4ubvgq0TbP+PNlSCkVFBryhKgfF18WXuC3P5sfWPCCF4c8+bDN8+nIh4w11WY2luxrS+tehVx5Vvdpxn+o7z+f/ysXWGFh/ApT1wYad+CtUj1XSjKEqhyNRlsur8Kn4K+4l76ffoWqUrY+uMpbx9eYPUk62TfLg6nOWhMYxuVZV3X6iev/FxsjLgpwZgbqVNPVjIs1GpphtFUQzO0syS/j792dRrE8P9h7Mtehvd1nbj55M/G2QMHXMzwdRegQyoX4kZey4ydcvZ/B3ZW1hB208h7iyc+E1/heqBCnpFUQqVg5UDbwW9xboe62jq2pQZYTPotrYbW6O3Fnr7vZmZ4LMe/gxqWJlZ+y/xfxvP5K+GGl21cXD2fAbphu9t9BcV9IqiGISbgxvTWk5jXvt5lLQqyXv73iN4azCR8ZGFWoeZmWBydz+GNvFg3sHL+Qt7IbSJSZLj4I9v9VpnfqigVxTFoOqVr8eyLsuY2Ggi0YnR9N/Yn0mHJnEv7V6h1SCEYEIX34dh/11+euO4BYF/b+1q2cTr+isyH1TQK4picOZm5vSu1puNPTcyyHcQay+spcuaLiw7u4xsXXah1CCE4JPOvvQJcuPbnVH88sfl519Zm08gO1PrW28EVNArimI0HKwceK/ee6zsuhKfUj5MOTKFAZsGEHY7rFC2b2Ym+LxXAB39y/N/GyNZHnrt6W/KjbMH+PeCYwsgNUGPFT4fFfSKohgdL2cv5r4wl6+af0V8WjyDtgzik4OfEJ8aX+DbtjA349v+tWjmXZrxq8LZ8rxj4zQeCxkPIHSefgt8DiroFUUxSkIIOnh2YEOPDQz1H8rGixvpurYrK86vKPAJT6wtzJk1KIjalZwZu/QE+88/xzwZFQKhahv4cyZkGnYKRhX0iqIYNTtLO94JeodV3VZR3bk6kw9PZsiWIZy/d75gt2tlwbzgeniVdeDV345x7MrdZ19Jkzch+TaEL9V/gc9ABb2iKEVCFacqzGs/j/9r8n9EJ0bTb0M/ph+bTmpWaoFt09HWkl+H1ae8ow3DF4Zy+U7ys63AszlUqAWHfoBCOqmcGxX0iqIUGUIIenj1YH2P9XSt2pV5p+fRc11P9sfsL7BtlnGwZuHQ+pgJwbAFISSkZDxLwdD0LYi/AOc2F1iNT6OCXlGUIsfZxpnJTSYzv/18rM2tGb1rNOMPjC+wvveVXOyYPSiI2HupvPb7MTKynuEcQY1uWi+cP7412MiWKugVRSmy6pavy8quK3m95utsi95Gj3U92Hq5YIZSqOtRii96B/Dnpbv8Z+2pvG/DzBwaj4HYULhySO915akEg2xVURRFTyzNLRlVaxTLuiyjon1F3tv/Hm/ueZPbKbf1vq2etd0Y09qL5aExzN5/Ke9vrDUQ7EprUw4agAp6RVFMQjXnavzW6TfGBY3j0PVD9Fjbg9VRq/V+dP9222p0DqzA1K1n2RZxM29vsrSFBq9B1Da4Vbhj+YAKekVRTIiFmQXB/sGs6raKaqWqMfHQREbtGkVcynP0g38CMzPBN31qEujmxFtLwzgdez9vb6w3HCzt4dD3eqslr1TQK4piciqXrMy89vMYX388oTdD6bm+J1ujt+pt/TaW5swZHEQpeyuGLwzhzoP0p7/JrhQEDYFTK+B+jN5qyQsV9IqimCQzYcbAGgNZ3nU5lRwq8d6+93h///vcT8/jEfhTlHWwYc7gutxLyeTtZWHodHloImo4CqQOQufrpYa8UkGvKIpJ83T05NeOv/JGrTfYEb2DXut6cTD2oF7W7VuxJBO7+nIg6g4z9118+huc3MGzBZxeWahdLVXQK4pi8izMLHi15qss6ryIktYleW3na3x+5HPSs/PQ5PIUL9WvRJfACkzbcZ6Q6DwMkxDQG+5FQ+zxfG87r1TQK4pSbPi6+LK0y1JervEyi88uZuCmgVy+n49x59Gu1v28VwDuzraMWXyCu8lPuXK2Rlcwt9ba6guJCnpFUYoVa3NrPqj/AT+2/pFbKbfot7Efa6LW5KsbpoONJT++VIe7yRmMW/6U9nobR/BuBxGrC238GxX0iqIUSy3cW7Cy60r8S/sz4dAExh8Yz4OMB8+9Pn9XR/7TpQZ7zsUx58BTLqYK6A0PbkH0H8+9vWehgl5RlGKrnH055rSbwxu13mBr9Fb6buxLRHzEc69vUMPKdAooz5fbznHsyr+Mu1OtA1iVKLTmGxX0iqIUa+Zm5rxa81Xmt59Ppi6TwZsHsyZqzXOtSwjB1BcDqehkw5jFx7mfkpn7gpa24NMFzqyHrPyfEH4aFfSKoihAnXJ1WN5lObXL1WbCoQlMPjyZjOxnGJI4R0kbS34cUIdbSelM3XrmyQsG9Ia0+3BhZz6qzhsV9IqiKDmcbZz5ue3PDPMfxorzKxi6dSg3k/M4ns0jaro7MbypJ0uOXuPo5Sd0uazSEuxc4NTK/BWdByroFUVRHmFhZsHbQW8zreU0LiRcoN/GfoTcDHnm9bzV1htXJ1s+XB1OelYuvWvMLcG3B5zbAunPfxI4L1TQK4qi5KJd5XYs7ryYklYlGbF9BIvOLHqmLph2VhZM6enPxbhkft77hF44Ab0hK7XAZ59SQa8oivIEVZ2qsqTzEpq7NWfq0an898h/ydJl5fn9raqXpUtgBWbsucDFuFyO2t0bQkm3Am++UUGvKIryL0pYleDbVt8S7BfM0nNLeWvPW6RkpuT5/RO6+mJjacbHa3KZlcrMDPx7wcVdkJKH4ROekwp6RVGUpzATZoyrO47/NPgPB2IPELw1OM8zWJV1sGF8xxr8eekuK47lMjxxQG/QZUHkWv0W/Yg8Bb0QooMQ4pwQ4oIQYnwur1sLIZblvH5ECOGR83w7IcQxIcSpnJ+t9Vy/oihKoenn048fWv9AdGI0AzcP5Py983l6X/967tSt7Mx/N58h/vGx68sHQulqcGpVAVSseWrQCyHMgRlAR8AXGCCE8H1sseHAPSmlFzAd+CLn+TtAVyllADAE+E1fhSuKohhCc7fmLOywEJ1Ox+AtgzkU+/QJv83MtIHPktOzmLLpsb71QoB/b7hyEO7HFkjNeTmirw9ckFJeklJmAEuB7o8t0x1YmHN/JdBGCCGklCeklNdzno8AbIUQ1vooXFEUxVBquNRgUedFuJZwZfSu0ey4suOp7/Eu58BrLaqy5kQsBy/c+fuLAb0BqQ10VgDyEvSuwLVHHsfkPJfrMlLKLOA+4PLYMi8Cx6WU/7jeVwgxUggRKoQIjYvT39yOiqIoBaW8fXkWdFiAf2l/3tv3HpsvPb2L5OhWXrg62TJtx/m/n5h1qQoVa8PNUwVSa6GcjBVC+KE157ya2+tSytlSyrpSyrplypQpjJIURVHyzcHKgVntZlGrbC0+/OND1l1Y96/L21ia82qLKhy7co8/Lz3Wy2bIBug1u0DqzEvQxwLujzx2y3ku12WEEBaAIxCf89gNWAMMllLmYa4tRVGUosPO0o6f2vxEvfL1+OTgJ6w6/+8nVfvWdaeMgzU/7on6+wvWDgVWY16CPgTwFkJ4CiGsgP7A+seWWY92shWgN7BbSimFEE7AJmC8lFI/kzQqiqIYGTtLO35s/SONXRsz6fAklp5d+sRlbSzNGdHMk4MX4jl+9V+GMtajpwZ9Tpv7G8A24AywXEoZIYSYLITolrPYL4CLEOIC8A7wVxfMNwAvYIIQIiznVlbve6EoimJgNhY2fN/qe1q6teSzI5/xW+STOxkObFAZJztLZuy+UCi1ifxMn1UQ6tatK0NDQw1dhqIoynPJzM7kgwMfsOPKDiY0mkCfan1yXe6HXVF8s+M8m8Y2xa+iY763K4Q4JqWsm9tr6spYRVEUPbI0t+SL5l/Q1LUpn/35GQdiDuS63ODGHjhYW/DTnoI/damCXlEURc8szSz5usXXVHOuxrh944iMj/zHMo62lgxuXJnNp29w4XZSgdajgl5RFKUA2Fva82ObH3G0dmT0rtHceHDjH8sMa+KJjYU5P+0t2KN6FfSKoigFpKxdWX5q8xNpWWmM2jWKxIzEv73uUsKalxpUYl3Yda7G531EzGelgl5RFKUAeTt7822rb4lOjObtPW+Tmf33CcNHNq+CuRD8vL/gjupV0CuKohSwBhUaMLnxZI7ePMrEQxP/NvxBuZI29KnrxsrQGG7eTyuQ7augVxRFKQRdq3ZldK3RbLi0gYURC//22mstqpItJbP3P2HKwXxSQa8oilJIXg18lTaV2vDDiR+4mPC/phr3Una82rwK1cqVKJDtqgumFEVRClF8ajw91/WkYomK/N7pdyzMLPSyXnXBlKIoipFwsXXh44YfExEfwbzT8wplmyroFUVRCll7j/Z08OjAzJMzOXf3XIFvTwW9oiiKAXzc4GMcrRz5+I+P/9HlUt9U0CuKohiAk40TExpN4Ny9c8wKn1Wg21JBryiKYiCtK7Wma5WuzD01l4g7EQW2HRX0iqIoBvRB/Q9wsXHh4z8+JiM7o0C2oYJeURTFgBytHZnUeBIX719kRtiMAtmGCnpFURQDa+bWjIE1BlLRvmKBrF8/PfUVRVGUfBlff/zTF3pO6oheURTFxKmgVxRFMXEq6BVFUUycCnpFURQTp4JeURTFxKmgVxRFMXEq6BVFUUycCnpFURQTZ3QzTAkh4oAr+VhFaeCOnsopStR+Fy9qv4uXvOx3ZSllmdxeMLqgzy8hROiTptMyZWq/ixe138VLfvdbNd0oiqKYOBX0iqIoJs4Ug362oQswELXfxYva7+IlX/ttcm30iqIoyt+Z4hG9oiiK8ggV9IqiKCbOZIJeCNFBCHFOCHFBCFFwI/gbmBBinhDithDi9CPPlRJC7BBCROX8dDZkjQVBCOEuhNgjhIgUQkQIId7Med6k910IYSOEOCqEOJmz35/mPO8phDiS83lfJoSwMnStBUEIYS6EOCGE2JjzuLjsd7QQ4pQQIkwIEZrz3HN/1k0i6IUQ5sAMoCPgCwwQQvgatqoCswDo8Nhz44FdUkpvYFfOY1OTBYyTUvoCDYHROf/Gpr7v6UBrKWVNoBbQQQjREPgCmC6l9ALuAcMNV2KBehM488jj4rLfAK2klLUe6T//3J91kwh6oD5wQUp5SUqZASwFuhu4pgIhpdwP3H3s6e7Awpz7C4EehVlTYZBS3pBSHs+5n4T2n98VE993qXmQ89Ay5yaB1sDKnOdNbr8BhBBuQGdgbs5jQTHY73/x3J91Uwl6V+DaI49jcp4rLspJKW/k3L8JlDNkMQVNCOEB1AaOUAz2Paf5Igy4DewALgIJUsqsnEVM9fP+LfA+oMt57ELx2G/Qvsy3CyGOCSFG5jz33J91NTm4iZFSSiGEyfaZFUKUAFYBb0kpE7WDPI2p7ruUMhuoJYRwAtYAPoatqOAJIboAt6WUx4QQLQ1cjiE0lVLGCiHKAjuEEGcfffFZP+umckQfC7g/8tgt57ni4pYQogJAzs/bBq6nQAghLNFCfpGUcnXO08Vi3wGklAnAHqAR4CSE+OtAzRQ/702AbkKIaLSm2NbAd5j+fgMgpYzN+Xkb7cu9Pvn4rJtK0IcA3jln5K2A/sB6A9dUmNYDQ3LuDwHWGbCWApHTPvsLcEZKOe2Rl0x634UQZXKO5BFC2ALt0M5P7AF65yxmcvstpfxQSukmpfRA+/+8W0o5EBPfbwAhhL0QwuGv+8ALwGny8Vk3mStjhRCd0Nr0zIF5UsrPDFtRwRBCLAFaog1beguYCKwFlgOV0IZ47iulfPyEbZEmhGgKHABO8b8224/Q2ulNdt+FEIFoJ97M0Q7MlkspJwshqqAd6ZYCTgAvSynTDVdpwclpunlXStmlOOx3zj6uyXloASyWUn4mhHDhOT/rJhP0iqIoSu5MpelGURRFeQIV9IqiKCZOBb2iKIqJU0GvKIpi4lTQK4qimDgV9IqiKCZOBb2iKIqJ+3+MDFdRcuEauwAAAABJRU5ErkJggg==", + "text/plain": [ + "
    " + ] + }, + "metadata": { + "needs_background": "light" + }, + "output_type": "display_data" + } + ], + "source": [ + "scheduler = PolyLRScheduler(optimizer, t_initial=t_initial)\n", + "plot_lr(scheduler, label='default')\n", + "\n", + "scheduler = PolyLRScheduler(optimizer, t_initial=t_initial, k_decay=2.)\n", + "plot_lr(scheduler, label='k_decay=2.')\n", + "\n", + "scheduler = PolyLRScheduler(optimizer, t_initial=t_initial, k_decay=.5)\n", + "plot_lr(scheduler, label='k_decay=0.5')\n", + "plt.legend();" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Cycle Args." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### `cycle_limit`" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "The number of cycles. \n", + "Note, what full namber of epochs will be different with t_initial." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "\n", + " after which to decay the learning rate where the new learning rate value equals `lr * decay_rate`. " + ] + }, + { + "cell_type": "code", + "execution_count": 14, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "t_initial=50\n", + "total_epochs=100\n" + ] + }, + { + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXoAAAD4CAYAAADiry33AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/YYfK9AAAACXBIWXMAAAsTAAALEwEAmpwYAAAttklEQVR4nO3deXxTZfb48c9J0oXuhZZCgVKgbGWHsjMyigsoUhdAQAUdRFwYnXGZYb6OjqOjP3VmXEEFREXZVFRgFEVFHRVZy77vSwHZ973t8/sjKVNLgbRNepOb8369eJHc3CQn3JvDyXPPfa4YY1BKKWVfDqsDUEop5V+a6JVSyuY00SullM1poldKKZvTRK+UUjbnsjqA4pKSkkx6errVYSilVFDJycnZZ4xJLumxgEv06enpLFy40OowlFIqqIjI1gs9pkM3Sillc5rolVLK5jTRK6WUzWmiV0opm9NEr5RSNudVoheR7iKyVkQ2iMjwEh6/TEQWiUieiPQu9tggEVnv+TPIV4ErpZTyziUTvYg4gZFADyAT6C8imcVW2wbcAUws9tzKwN+A9kA74G8iklj+sJVSSnnLm4q+HbDBGLPJGHMGmAxkF13BGLPFGLMMKCj23GuAr40xB4wxB4Gvge4+iPs8p87m8+T0lew/dtofL68UADsPneSDBdvIyy++qysVuLxJ9DWA7UXu53qWecOr54rI3SKyUEQW7t2718uX/rVluYeZOH8b1776I/M3HyjTayh1KR/n5PLnj5dz61vz2H3klNXhKOWVgDgYa4wZbYzJMsZkJSeXeAbvJbWrU5mp93UmKtxF/zFzGfndBgoK9KIqyrfOeCr5ZbmHufaVH/lxfdkKE6UqkjeJfgdQq8j9mp5l3ijPc0stMzWO6cM606NpNf45cy2Dxy3g4PEz/no7FYLyCgzhTgfTh3WmcnQ4A9+ez0tfryNfiwoVwLxJ9AuA+iJSR0TCgX7AdC9ffyZwtYgkeg7CXu1Z5jexkWG81r8VT2c3YfaG/fR87ScWbzvoz7dUISS/wOB0CPVTYpk2rDM3tarJK7PWM+jt+ezT40MqQF0y0Rtj8oBhuBP0auBDY8xKEXlKRHoBiEhbEckF+gCjRGSl57kHgKdx/2exAHjKs8yvRITbO6Yz5d6OiEDfUXN4+6fN6PVxVXnl5RtcDgEgKtzFv/o054Wbm7NgywGue/VHFmzR40Mq8EigJb+srCzjy9krD584y8MfLeWb1bu5rnl1nr+5OTERATdppwoST05fyaeLd7D0b1f/avmqnUe4b0IO2w+eZHj3Rtz1mzqIiEVRqlAkIjnGmKySHguIg7H+FB8VxpiBbRjeoxFfLN9FrxE/sfaXo1aHpYJUXkHBuYq+qMzUOKb/vgtXNU7hmRmruXf8Io6cOmtBhEqdz/aJHtxDOfd0rcfEIR04cjKPG0bOZtoSvx0TVjZWOEZfkrjIMN64rTV/va4xX6/eTfaI2VpUqIAQEom+UIe6Vfj8gS40rRHHg5OX8OT0lZzJ0xNflPeKjtGXRES46zd1mTSkA8dOa1GhAkNIJXqAlLhIJg7pwOAudXj35y30HzNXT3xRXssvMDidlx57b1enMp//vgvNasRrUaEsF3KJHiDM6eDxnpmMGNCK1buOcN2rPzFv036rw1JBIK/A4HJ497WpGhfJhCHtuctTVAwYM5c9WlQoC4Rkoi/Us3kqU+/vTFykiwFvzWOstmCqS7jYGH1JwpwO/tozk9f6t2LlziNc99pP2oKpKlxIJ3qABimxTB3WmSsaVeXpz1bxhw+WcPJMvtVhqQB1oa6bS7m+hbuoiA530n/0XN6bs0WLClVhQj7Rg7tbYtRtbXj4qgZMX7qTm9/4me0HTlgdlgpApa3oi2pYLZZpw7rQtUEyT0xbySMfLePUWS0qlP9povdwOITfd6vP23e0JffgCa4f8RM/rd9ndVgqwLjH6Mt+IlR8pTDGDMziwW71+XhRLn3enMPOQyd9GKFS59NEX8zlDasyfVgXqsZGMPDteYz5YZP+xFbnlKeiL+RwCH+8qgFvDcxi877jXP+aNgMo/9JEX4L0pGg+va8z1zSpxjMzVuu4vTrH3Ufvm6/NlZkpTL2/M/FRYdz61jwdt1d+o4n+AqIjXLx+a2sevaYh05fupM+on/UntvJJRV9URtUYpt7f+dy4/V8+Wc7pPC0qlG9por8IEeH+yzN4a2AWW/adoNcIbY0LdXkFBbi8OGGqNOIi3eP2wy7PYPKC7QwYM489R7XfXvmOJnovdGucwtT7OxEbGcaAMXOZPH+b1SEpi/i6oi/kcAiPXNOQEQNasWrnEbJHzGbFjsM+fx8VmjTReymjaixT7+tMh7pVGP7Jcp6cvlIvEB2Cytt1cyk9m6e6r6MA9H7zZ/6zdKff3kuFDk30pRAfFcY7d7Q9N0/OHe8s4PAJnYo2lPiroi+qSWo804Z1oWlqPL+ftJh/f7VWr3+sykUTfSm5PPPkvNC7OfM27+fG12ezae8xq8NSFaQ0c92UR3JsBBOGtKdvVk1e+3YD901YxIkzeX5/X2VPmujLqG9WLSYO6cChk2e5YeRsPbkqRFRERV8owuXk+Zub89frGvPVql/05CpVZproy6FtemWm3d+Z6vGVGPTOfN6fu9XqkJSflXWum7IqnN9+7KC2bN1/guyRs1my/VCFvb+yB0305VSrchQf39eJrg2SeXzqCj1Ia3P5+RVX0Rd1eaOqfHJfJyJcDm4ZNYfPlulBWuU9TfQ+EBPhYszArHMHaQePW8hRvV6oLeUVGJ/30XurQUos0+7vTLMa8QybuJhXZ63XM2mVVzTR+4jTITzeM5Nnb2zGTxv20fuNOeQe1Bkw7aYix+hLUiXGfZD2plY1ePHrdTz84VI9k1ZdkiZ6HxvQPo1xd7Zj5+GT3DDyZ5bqeKqtVFTXzcVEuJz8u28LHr6qAZ8s3sHtb83n4PEzlsakApsmej/oUj+JT+7tRGSYg1tGz+HLFbusDkn5iNUVfSER97Tar/ZvxZLcQ9z4+mw27ztudVgqQGmi95P6KbFMvb8zjarFce+ERbz1o053bAcV3XVzKb1apDJpSAeOnMrjptdn61xMqkSa6P0oKSaCyXd3oHuTavzj89U8MU07coJdoFT0RbWpncin93UiMSqcW8fMY7pOm6CK0UTvZ5FhTkYOaM3Qy+ry/tytDH0/R89wDGL+nuumrGpXiebjezvRslYCD0xazBvfb9RfkOocTfQVwOEQ/nJtY57ObsJ3a/fQb/Rc9h49bXVYqpQKCgzGgNPig7EXkhgdzvt3teP6Fqk8/+UaHp+2Qn9BKkATfYW6vWM6o2/PYv3uY9z4+mw26hw5QSXPM7GYVX303ohwOXnllpbc07Ue4+du01+QCtBEX+GuzExh8t0dOHU2n5vf+JmcrQetDkl5Kd+T6ANtjL44h0MY3qPRuV+Q/cfMY/8x/QUZyjTRW6BFrQQ+vrcTCZXcFzKZufIXq0NSXsgrcA+DBOIYfUlu75jOm7e1Yc2uI9z8xs9s3a/tl6FKE71FCg+eNaoex73jcxivE6IFvGCp6Iu6ukm1c7Os3vT6zyzLPWR1SMoCmugtVCUmgklD2vPbhlX569QVvPj1Ou2UCGDnxuiDKNGDu/3y43s7ERnmpN/oufywbq/VIakKponeYlHhLkbd3oY+bWry6qz1/N+n2ikRqP5X0Qff16Zecgyf3NeJtMpR/O7dBUxdvMPqkFQFCr491obCnA5e6N2cYZdnMGn+Nu6bsIhTZ3WiqkATrBV9oZS4SD68pyNZ6Yn84YMljP1ps9UhqQqiiT5AiAiPXNOQJ6/P5KtVuxn09nyO6FTHASU/P/jG6IuLiwxj3O/a0aNpNZ7+bBUvfLlGhwtDgCb6AHNH5zq80q8lOVsP0m/UXPYcPWV1SMrjXNdNAPfReyPC5WTEgNYMaJ/G699v5C+fLNfhQpvzKtGLSHcRWSsiG0RkeAmPR4jIB57H54lIumd5mIiME5HlIrJaRP7i4/htKbtlDcbe0ZbN+47T9805bD+g89oHgmDsurkQp0N45oamPHBFBpMXbGfYxMU6r72NXTLRi4gTGAn0ADKB/iKSWWy1wcBBY0wG8BLwvGd5HyDCGNMMaAMMLfxPQF1c1wbJTBjSnoMnztL7zZ9Zv/uo1SGFvGAfoy9ORHjo6oY80TOTL1f+wuB3F3L8tJ5Fa0feVPTtgA3GmE3GmDPAZCC72DrZwDjP7SlANxERwADRIuICKgFngCM+iTwEtE5L5MOhHTEG+oyaoxeFtlgwd91czO+61OHffVowZ9N+bn1rHodO6EVM7MabPbYGsL3I/VzPshLXMcbkAYeBKriT/nFgF7AN+Jcx5rwJs0XkbhFZKCIL9+7VHt+iGlaLZco9nYiLDOPWMXOZs3G/1SGFLLtV9EXd3KYmb97WhlW7jnDLqLnsOaLHhuzE36VJOyAfSAXqAA+LSN3iKxljRhtjsowxWcnJyX4OKfikVYnio3s6kppQiTvemc+3a3ZbHVJIyvccjLXDGH1JrspM4d072rL94An6jtJrHtuJN4l+B1CryP2anmUlruMZpokH9gMDgC+NMWeNMXuA2UBWeYMORSlxkXwwtCMNUmK5+70c/qMXl6hwefn2regLdcpIYvxd7Tlw/Ax93pyjM6zahDeJfgFQX0TqiEg40A+YXmyd6cAgz+3ewLfG3Zy7DbgCQESigQ7AGl8EHooqR4czcUh7Wqcl8uDkxUzJybU6pJBip66bi2mdlsjkuztyNr+AW0bNZe0v2ggQ7C6Z6D1j7sOAmcBq4ENjzEoReUpEenlWGwtUEZENwENAYQvmSCBGRFbi/g/jHWPMMl9/iFAS6znhpXNGEo98tJT3dTK0ChMM89H7SmZqHJPv7ojTAbeMnsPy3MNWh6TKwasxemPMDGNMA2NMPWPMM55lTxhjpntunzLG9DHGZBhj2hljNnmWH/Msb2KMyTTG/NN/HyV0VAp3MmZgFlc2rsrjU1cw5odNVocUEuzadXMhGVVj+GhoJ2IiXAwYM5ecrXrh8WAVGnusDUWGOXnjtjZc16w6z8xYzcjvNlgdku3ZuevmQtKqRPHh0I4kxUZw+9j5zNukXV/BSBN9EAtzOnilX0tuaJnKP2eu5SWd5tiv7N51cyGpCZX44O4OVI+PZNA785m9YZ/VIalS0kQf5FxOB//u25LebWryyqz1vDBzrSZ7PwnFir5Q1bhIJt/dkdqVo7nz3QV8v3aP1SGpUtBEbwNOh/DCzc3p3y6NN77fyHM6I6FfhErXzYUkx0Yw6e4OZCTHcPd7OXy3RpN9sNBEbxMOzyRVt3VIY9R/N/H/vtBk72v55yr60P3aFLb4NqgWw9D3c/TkvSARunusDTkcwtPZTRnUsTajf9jEM5+v1mTvQ4VDNyGc5wFIiApnwuAONKwWy9D3c/hmlSb7QBfiu6z9iAhP9mrCHZ3SeeunzTw7Q5O9r2hF/z/xUWGMv6s9mdXjuHeCVvaBTvdYGxIR/nZ9JoM61mbMj5t1zN5H8kJ8jL64+EphvDe4PY2qxXHP+4v4Tg/QBixN9DZVWNkXjtlrN0755XuuwhSKXTcXEl8pjPGD/zdm/991OvtsINJEb2MiwlO9mjKgvbsb56Vv1lsdUlA7V9GHwBQIpREf5U727m6chdpnH4A00ducwyH8I7spfbNq8uqs9XoGbTnkh3Af/aUkRIUz/q72pFeJ5q5xC5m/WadLCCSa6EOAwyH8v5uac2OrGvxz5lqdG6eMdIz+4ipHu5N9akIkd74zn0XbDlodkvLQRB8inA7hn72bc11z99w478/ZYnVIQUe7bi4tOTaCiUM6kBwbwaC357Nih856GQh0jw0hLqeDl29p6Z71ctpKnc++lM710WtBf1EpcZFMGNKBuMgwBr49Xy9sHwA00YeYMKeDEQNa0yUjiT9NWcqM5busDilo5BcU4HII7uveq4upkVCJCXe1x+UQbn1rHlv3H7c6pJCmiT4ERYY5GT2wDa3TEnlg0mKds8RLeQVGx+dLIT0pmgl3tedsfgEDxsxj1+GTVocUsjTRh6iocBdv39mWRtVjuWd8jnZJeCE/32jHTSnVT4nl/cHtOXLyLLe9NY/9x05bHVJI0kQfwuIiwxh3ZztqJFZi8LsL9MDZJWhFXzZNa8Tz1qAscg+e5I53FnD01FmrQwo5muhDXJWYCMYPbk9cJfeBs417j1kdUsDKLzC4nPqVKYv2davw5m1tWL3rCIPHLeTU2XyrQwoputcqUhMq8f7gdggwcOx8fjl8yuqQApJW9OVzeaOq/LtvCxZsOcDvJy0mzzOlhPI/TfQKgLrJMYz7XTsOnzzLwLfncejEGatDCjiFXTeq7LJb1uDJ65vw9ard/N+ny3X+pQqiiV6d07RGPKMHtmHLvhP87t0FnDiTZ3VIAUUret8Y1CmdB7rV58OFuTz35RqrwwkJmujVr3Sql8Sr/VuyZPshfj9Rf14XlV+gXTe+8scr63Nre/fMqmN/2mx1OLaniV6dp3vT6jyV3ZRZa/bw2Kcr9Oe1h1b0viMiPJXdlO5NqvH0Z6v4z9KdVodka5roVYlu61Cb31+RwQcLt+v0xh7uPnr9yviK0yG83K8l7dIr8/CHS/l5o05v7C+616oLeuiqBuemN544b5vV4VhOK3rfiwxzMmZgFulJUQx9L4c1vxyxOiRb0kSvLkhEePbGZvy2YTKPT1sR8lMl5BcU4NKLjvhcfFQY797ZjugIF3e+s0CnSvADTfTqolxOByMHtKZx9Vjun7gopM+e1Yref1ITKvH2HW05eiqPO/XsWZ/TRK8uKTrCxduD2pIYFc6d7y4g9+AJq0OyhHbd+Fdmahyv39qaDXuOcd+ERZzVji+f0USvvFI1LpJ372zL6bP5/O7dBRwJwYpLK3r/u6xBMs/e1Iwf1+/j8ana8eUrmuiV1+qnxPLmbW3YtPc494dgxeWu6PUr4299s2ox7PIMJi/Yzii97KVP6F6rSqVTRhLP3uiuuP42fWVIVVxa0Vech65qQM/m1XnuizV8oRfHKTeX1QGo4NO3bS027z/OG99vpE6VaIZcVtfqkCqEznVTcRwO4V99WrDj0En+8MESUhMq0aJWgtVhBS2t6FWZPHp1Q65tVo1nv1jNrNW7rQ6nQuTla0VfkQp77JNjIxjy3kKdVbUcNNGrMnE4hH/3aUmT1DgemLSYtb/Y/wLQ7vnoNdFXpKSYCMYOasvx03kMeW8hJ8/oPPZloYlelVmlcHfFFR3hYvC4Bba/TFx+gcGpB2MrXMNqsbzavxUrdh7m4Y+WUFAQOseFfEX3WlUu1eMrMXpgFnuPnube8Ys4k2ffTpw87aO3TLfGKfylRyNmLP+FV7/VuZdKy6tELyLdRWStiGwQkeElPB4hIh94Hp8nIulFHmsuInNEZKWILBeRSB/GrwJAy1oJvNC7OfO3HOCpz1ZaHY7f5GvXjaWG/KYuN7WuwcvfrGfmyl+sDieoXDLRi4gTGAn0ADKB/iKSWWy1wcBBY0wG8BLwvOe5LmA8cI8xpgnwWyD0zrQJAdktazC0a13Gz91m2wnQ8rTrxlKFcy+1qBnPQx8sYd1u+x8X8hVvKvp2wAZjzCZjzBlgMpBdbJ1sYJzn9hSgm4gIcDWwzBizFMAYs98Yo0dTbOpP1zSia4Nk/jZ9BQu2HLA6HJ/Tit56kWFORt2eRVSEiyHvLdRLXnrJm0RfA9he5H6uZ1mJ6xhj8oDDQBWgAWBEZKaILBKRP5X0BiJyt4gsFJGFe/fuLe1nUAHC6RBe7d+KmolR3Dt+ke3a4XSMPjBUi49k1O1t2HXoFA9MXkK+Hpy9JH8fjHUBXYBbPX/fKCLdiq9kjBltjMkyxmQlJyf7OSTlT/GVwhh9extOnMnj3gk5nM6zzw+4/HztugkUrdMS+Xt2E35Yt5eXvl5ndTgBz5u9dgdQq8j9mp5lJa7jGZePB/bjrv5/MMbsM8acAGYArcsbtAps9VNi+VefFizedoinP1tldTg+k6d99AGlf7s0+rWtxYjvNujB2UvwJtEvAOqLSB0RCQf6AdOLrTMdGOS53Rv41rgnQZkJNBORKM9/AF0B+3zz1QVd26z6uYOzHy7cfuknBAEdow88T/ZqQoua8Tz84VI27j1mdTgB65KJ3jPmPgx30l4NfGiMWSkiT4lIL89qY4EqIrIBeAgY7nnuQeBF3P9ZLAEWGWM+9/mnUAHp0asb0jmjCo9PXcHKncF/wRLtugk8kWFO3ritDREuB/eOz+HEmTyrQwpIXg04GmNmGGMaGGPqGWOe8Sx7whgz3XP7lDGmjzEmwxjTzhizqchzxxtjmhhjmhpjSjwYq+zJ5XTwar9WJEaFc9+ERRw+GbydtQUFhgKDVvQBKDWhEq/0a8X6Pcd47FOdw74kemRJ+VWVmAhGDGjFjoMnefSjpUH7Jcz3xK0VfWDqUj+JP17ZgE8X72DifHuex1EemuiV32WlV2Z4j0Z8tWo3Y34MzgtJFLbwaddN4Bp2eQZdGyTz9+mrWJZ7yOpwAorutapCDO5Shx5Nq/HCl2vJ2XrQ6nBKLa9AK/pA53AIL9/SkqSYcIZNXBySl7u8EE30qkKICM/d3JzqCZE8MGlx0J3RmJ9fWNFrog9kidHhvDagFTsOnWT4x8uCdqjQ1zTRqwoTXymMEf1bs+foKR6dElxfwrwC96yc2kcf+NrUrswjVzdkxvJfGG/TeZdKSxO9qlAtaiUwvEdjvl61m3dmb7E6HK/9b4xeE30wGHpZXbo2SObpz1bZorW3vDTRqwr3u87pXNk4hee+WMOKHcHxJdQx+uDicAgv9m1BYlQYD0xaHPL99ZroVYUTEV7o3ZzE6DAenBwcX0Ltugk+VWIieLFvSzbtO87Tn622OhxL6V6rLFE5OjyovoT5WtEHpc4ZSQy9rB6T5m/jyxW7rA7HMprolWWC6UuYp2P0QeuhqxrQvGY8f/54ObsOn7Q6HEtooleWKvolDOT567WiD17hLgev9GvF2fwCHvpgaUheXFwTvbJUuMvBy7e05HRePo9OCdwpEgrbKx2a6INSnaRo/nZ9JnM27eft2ZutDqfCaaJXlqubHMNj12Xy4/p9vD93q9XhlEgr+uDXN6sWVzZO4YWZa0PuerOa6FVAuK19Gl0bJPPsjNUBOa+4jtEHP/fZ2c2IjXDxh8lLOJNXYHVIFUYTvQoIIsI/ezcnMszJQx8sIS8/sL6E/6vo9SsTzJJiInju5uas2nWEl78JnUsQ6l6rAkbVuEieuaEZS3MPM+qHwJrlMk/nurGNqzJT6JtVkzf/u5El2w9ZHU6F0ESvAsp1zatzXbPqvPzNOtb+EjjjqOcqep3rxhb+2jOTlLhIHvloKafO2ucC9heiiV4FnKeymxAXGcYjHy3lbIAM4RR23WhFbw9xkWE8d3NzNuw5xsvfrLc6HL/TRK8CTpWYCJ6+oSnLdxxm1H83Wh0OoF03dtS1QTK3ZNVi9A8bWbwt+K6RUBqa6FVAurZZda5rXp1XZq0PiFY47bqxp8d6NiYlLpJHpyzjdJ59h3A00auA9VSvJsREuBj+8TLLz2bUrht7iosM49mbmrFhzzFe/y4wfj36g+61KmBViYng8Z6ZLNp2iPHzrD2RSit6+7q8YVWyW6by+vcbWB8Avx79QRO9Cmg3tqrBb+on8fwXa9h5yLoJqfILrzClid6WnuiZSUyEiz8HwK9Hf9BErwKaiPDsjc0oMPD41BWWzYWjffT2Fki/Hv1BE70KeLUqR/Hw1Q2YtWYPX6z4xZIYtI/e/or+egzkmVTLQhO9Cgp3dEons3ocT/1nFcdOV/wVqXSM3v5EhH/c0JS8AsPTn6+yOhyf0kSvgoLL6eCZG5uy++gpXvq64uco0a6b0FC7SjT3X57B58t28cO6vVaH4zO616qg0Sotkf7t0nj35y2s2nmkQt9bK/rQMbRrXeokRfPEtBW2mR5BE70KKn+6piEJlcL469TlFdodoV03oSPC5eTp7KZs2X+CNwPkzOzy0kSvgkpCVDh/ubYxi7YdYsqi3Ap7X63oQ0uX+klc3yKV17/fyLb9J6wOp9w00augc3PrGrROS+CFL9dy9NTZCnnP/Hyd6ybUPHZtY1wO4ZkZwX9gVhO9CjoiwpO9mrD/+Gle+3ZDhbynVvShp1p8JPdfnsHMlbuZvWGf1eGUiyZ6FZSa10ygT5uavDN7M5sq4NKD+QUGp0MQ0UQfSgZ3qUNa5Sj+/p+VAXfVs9LQRK+C1qPXNCLS5eTpz/z/0zrPk+hVaIkMc/LYdY1Zt/sY4wP0wvXe0ESvglZybAQPdKvPd2v38t3aPX59r/yCAh2fD1FXZ6bQJSOJF79ex8HjZ6wOp0w00augNqhTOulVonhuxppzJzX5g1b0oUtEeLxnJsdO5zHiu4o5JuRrmuhVUAt3OfhT90as3X2UKTnb/fY++QVGK/oQ1rBaLH3a1OK9OVuCst3Sq0QvIt1FZK2IbBCR4SU8HiEiH3genyci6cUeTxORYyLyiI/iVuqcHk2r0SotgX9/tY4TZ/wzD467ote6KJQ9dHUDnA7hhZlrrA6l1C6554qIExgJ9AAygf4ikllstcHAQWNMBvAS8Hyxx18Evih/uEqdT0R47NrG7Dl6mrE/bvbLe+Tna0Uf6lLiIhnym7p8tmwXS7YfsjqcUvGmRGkHbDDGbDLGnAEmA9nF1skGxnluTwG6iacPTURuADYDK30SsVIlyEqvzDVNUnjzvxvZe/S0z19fx+gVwNCu9UiKCefZGastuzZCWXiT6GsARQc/cz3LSlzHGJMHHAaqiEgM8Gfg7xd7AxG5W0QWisjCvXvtM2Ocqlh/7t6IU3kFjPTDAbP8ggKdi14RE+HiwSsbMH/zAb5fGzy5yt+Djk8CLxljLnpGizFmtDEmyxiTlZyc7OeQlF3VTY6hb1ZNJs7bxg4fX3ZQK3pVqF/bWqRVjuJfX60Nmqrem0S/A6hV5H5Nz7IS1xERFxAP7AfaAy+IyBbgD8D/iciw8oWs1IUNu6I+AK/NWu/T19WuG1UozOngwW71WbnzCF9adMWz0vIm0S8A6otIHREJB/oB04utMx0Y5LndG/jWuP3GGJNujEkHXgaeNcaM8E3oSp2vRkIlBrRP46OcXLbsO+6z19WuG1XUDa1qUC85mhe/XufX8zd85ZJ7rmfMfRgwE1gNfGiMWSkiT4lIL89qY3GPyW8AHgLOa8FUqqLcd3k9wpzCy9/47kpUWtGropwO4aGrGrJ+zzGmLy0+wBF4XN6sZIyZAcwotuyJIrdPAX0u8RpPliE+pUqtamwkgzqlM/qHTdx3eQYNUmLL/Zo6Rq+K69G0Go2rx/HyN+vp2TyVMGfg/uIL3MiUKod7LqtHdLiLV300Vq9z3ajiHA7h4asasHX/CaYuDuyqXhO9sqXE6HBu61Cbz5fv8sk0xnn5WtGr83VrXJXM6nG88f3GgB6r10SvbGtwlzqEOx288X35r/uZX2C0j16dR0S4//IMNu07zhcrdlkdzgVpole2lRwbQf92aXy6eAe5B8s3EZV23agL6d60GvWSoxn53caA7avXPVfZ2tCudRGB0T9sKtfraNeNuhCnQ7jvtxms3nWEb9f497oIZaWJXtla9fhK3Ny6JpMXbGfP0VNlfh3tulEX06tlKjUTKzHiuw0BWdVrole2d0/XeuTlFzD2p7LPbKldN+piwpwO7ulaj8XbDjFn036rwzmPJnple+lJ0fRoVp2J87Zx/HTZ5qvXil5dSu82NUmKCffbVNnloYlehYTBXepw9FQeU3Jyy/R8HaNXlxIZ5uTW9rWZtWaPT1p6fUkTvQoJrdMSaZWWwDuzN5ep39ndR69fF3Vxt3WoTbjTwTuzt1gdyq/onqtCxuAuddiy/wSzVu8u9XO1olfeSI6NILtlKlNycjl04ozV4ZyjiV6FjO5NqlEjoVKZDsrmFRicesKU8sLg39Th5Nl8Js7fZnUo52iiVyHD5XRwR6d05m0+wIodh0v1XO26Ud5qVC2OLhlJjPt5C2fyCqwOB9BEr0LMLe1qER3u5O3Zpavq87XrRpXC4C512H3kdMBMi6CJXoWUuMgwbmhVg8+X7eLwibNeP0/H6FVpdG2QTFrlKCbOC4zhG030KuQMaJ/G6bwCPlnsfaulznWjSsPhEPq1q8W8zQfYGACtlrrnqpDTJDWeFjXjmTR/m9enq7uHbvwcmLKV3m1q4nIIkwPgoKzuuiokDWifxrrdx8jZevCS6xpjtKJXpVY1NpKrm6QwJSeXU2fzLY1F91wVkno2TyUmwuVVC1zh+VU6Rq9Kq3+7NA6eOMvMlb9YGocmehWSoiNcZLdM9eqgbF6Bu0VOu25UaXWulxQQB2U10auQ5e1B2cIpE7SiV6UVKAdlNdGrkNUkNZ7mNeP5cOHFE32eJ9FrRa/KonebmjgdwkeX2M/8SRO9Cmk3tqrB6l1HWL/76AXXyc/Xil6VXdXYSH5TP4n/LN1JgUUXENdEr0Ladc2r4xCYtmTnBdc5V9Frf6Uqo+yWqew4dJKcbZfu8vIH3XNVSKsaG0nnjCSmLd1xwZ56HaNX5XVVZjUiwxxMW7LDkvfXRK9CXq8WqWw/cJLF2w+V+Lh23ajyiolwcWXjFD5ftouz+RU/0ZkmehXyrmlajXCXg2mLS662tKJXvpDdsgYHT5zlx/V7K/y9NdGrkBcXGcaVjavy2bJd5JVQbWnXjfKFrg2Sia8UdtHjQf6iiV4poFeLGuw/fobZG/ef99j/Knr9uqiyC3c5uLZZdb5auZsTZ8p2kfqy0j1XKeC3DZOJjXSVOHyTl68VvfKN7JapnDybz9erSn85y/LQRK8UEBnm5OrManyzevd5wzc6Rq98pV16ZarGRvDVSk30SlniysZVOXIq77wZLc913eg1Y1U5ORzCFY2q8sO6vRV6mUFN9Ep5dKmfRJhT+HbNnl8t14pe+dIVjapy9HQeC7ccqLD31ESvlEdsZBjt61RhVrFEr103ypc6ZyQR7nKct5/5kyZ6pYq4olFVNuw5xtb9x88t064b5UvRES461q1y3i9Hf9I9V6kiujWuCvCrL6FW9MrXujWuyuZ9x9lUQVMXa6JXqojaVaKplxz9q0Sf7zkYq2P0ylcub3h+QeFPXiV6EekuImtFZIOIDC/h8QgR+cDz+DwRSfcsv0pEckRkuefvK3wcv1I+161xCnM37efYafdJLdpHr3ytVuUoGqbEMmt1gCR6EXECI4EeQCbQX0Qyi602GDhojMkAXgKe9yzfB1xvjGkGDALe91XgSvnLFY2qcjbf8JNnTpJzY/TaXql86IrGVVmw5QCHT178Upa+4E1F3w7YYIzZZIw5A0wGsoutkw2M89yeAnQTETHGLDbGFE7ssBKoJCIRvghcKX9pUzuRuEjXuWorT9srlR90a1SVvAJTIZOceZPoawDbi9zP9SwrcR1jTB5wGKhSbJ2bgUXGmNPF30BE7haRhSKycO/eip/ZTamiwpwOLmuQzH/X7cUYc66id2rXjfKhVmmJJESF8d+1gZHoy01EmuAezhla0uPGmNHGmCxjTFZycnJFhKTURbWvU5k9R0+Te/CkVvTKL5wOIat2YoVcdcqbRL8DqFXkfk3PshLXEREXEA/s99yvCXwKDDTGbCxvwEpVhNa1EwHI2XrwXNeNHoxVvta6diKb9h7nwPEzfn0fbxL9AqC+iNQRkXCgHzC92DrTcR9sBegNfGuMMSKSAHwODDfGzPZRzEr5XcOUWKLDneRsPagVvfKbNmnugmLRVv9W9ZdM9J4x92HATGA18KExZqWIPCUivTyrjQWqiMgG4CGgsAVzGJABPCEiSzx/qvr8UyjlYy6ng1ZpiZ6KXtsrlX+0qJWAyyF+H75xebOSMWYGMKPYsieK3D4F9Cnhef8A/lHOGJWyROvaiYz4dj2HT7jb33QKBOVrkWFOmtSIJ2eLxRW9UqEqq3YiBQYWen5W6zTFyh+yaieyNPeQX6ct1kSv1AW0TEtAhHPz0+sYvfKHNrUTOZ1XwKpdR/z2HprolbqAuMgwGqbEnpsKQcfolT+08XR4+XN+ek30Sl1E4ZcQwCma6JXvpcRFUjOxEov8eEBWE71SF1GY6B3ivgycUv7Qpra7w8sY45fX10Sv1EVk1a4MaMeN8q+s2onsPuI+E9sfdO9V6iJqVa5EUkyEjs8rvyo8E9tfwzea6JW6CBGhTe0ETfTKrxpVizt3JrY/eHXClFKh7J6u9ehYt/hkrEr5jtMh9G1bi1qJUX55ffHX4H9ZZWVlmYULF1odhlJKBRURyTHGZJX0mA7dKKWUzWmiV0opm9NEr5RSNqeJXimlbE4TvVJK2ZwmeqWUsjlN9EopZXOa6JVSyuYC7oQpEdkLbC3HSyQB+3wUTrAIxc8Mofm59TOHjtJ+7trGmOSSHgi4RF9eIrLwQmeH2VUofmYIzc+tnzl0+PJz69CNUkrZnCZ6pZSyOTsm+tFWB2CBUPzMEJqfWz9z6PDZ57bdGL1SSqlfs2NFr5RSqghN9EopZXO2SfQi0l1E1orIBhEZbnU8/iAitUTkOxFZJSIrReRBz/LKIvK1iKz3/J1odaz+ICJOEVksIp957tcRkXmebf6BiIRbHaMviUiCiEwRkTUislpEOobCthaRP3r27xUiMklEIu24rUXkbRHZIyIriiwrcfuK26uez79MRFqX5r1skehFxAmMBHoAmUB/Ecm0Niq/yAMeNsZkAh2A+z2fczgwyxhTH5jluW9HDwKri9x/HnjJGJMBHAQGWxKV/7wCfGmMaQS0wP3Zbb2tRaQG8ACQZYxpCjiBfthzW78LdC+27ELbtwdQ3/PnbuCN0ryRLRI90A7YYIzZZIw5A0wGsi2OyeeMMbuMMYs8t4/i/uLXwP1Zx3lWGwfcYEmAfiQiNYHrgLc89wW4ApjiWcVWn1tE4oHLgLEAxpgzxphDhMC2xn0t60oi4gKigF3YcFsbY34ADhRbfKHtmw28Z9zmAgkiUt3b97JLoq8BbC9yP9ezzLZEJB1oBcwDUowxuzwP/QKkWBWXH70M/Ako8NyvAhwyxuR57tttm9cB9gLveIar3hKRaGy+rY0xO4B/AdtwJ/jDQA723tZFXWj7livH2SXRhxQRiQE+Bv5gjDlS9DHj7pe1Vc+siPQE9hhjcqyOpQK5gNbAG8aYVsBxig3T2HRbJ+KuXusAqUA05w9vhARfbl+7JPodQK0i92t6ltmOiIThTvITjDGfeBbvLvwZ5/l7j1Xx+UlnoJeIbME9LHcF7vHrBM/Pe7DfNs8Fco0x8zz3p+BO/Hbf1lcCm40xe40xZ4FPcG9/O2/roi60fcuV4+yS6BcA9T1H5sNxH7yZbnFMPucZlx4LrDbGvFjkoenAIM/tQcC0io7Nn4wxfzHG1DTGpOPett8aY24FvgN6e1az1ec2xvwCbBeRhp5F3YBV2Hxb4x6y6SAiUZ79vfBz23ZbF3Oh7TsdGOjpvukAHC4yxHNpxhhb/AGuBdYBG4HHrI7HT5+xC+6fcsuAJZ4/1+Ier54FrAe+ASpbHasf/w1+C3zmuV0XmA9sAD4CIqyOz8eftSWw0LO9pwKJobCtgb8Da4AVwPtAhB23NTAJ93GIs7h/wQ2+0PYFBHdn4UZgOe6uJK/fS6dAUEopm7PL0I1SSqkL0ESvlFI2p4leKaVsThO9UkrZnCZ6pZSyOU30Sillc5rolVLK5v4/9+xUVUUXbx8AAAAASUVORK5CYII=", + "text/plain": [ + "
    " + ] + }, + "metadata": { + "needs_background": "light" + }, + "output_type": "display_data" + } + ], + "source": [ + "t_initial = 50\n", + "print(f\"{t_initial=}\")\n", + "scheduler = PolyLRScheduler(optimizer, t_initial=t_initial, cycle_limit=2)\n", + "plot_lr(scheduler)\n", + "total_epochs = scheduler.get_cycle_length()\n", + "print(f\"{total_epochs=}\")" + ] + }, + { + "cell_type": "code", + "execution_count": 15, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "t_initial=50\n", + "total_epochs=150\n" + ] + }, + { + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXoAAAD4CAYAAADiry33AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/YYfK9AAAACXBIWXMAAAsTAAALEwEAmpwYAAA4SElEQVR4nO3deXyU1dXA8d+ZyUL2PYEkQBISCCHsiyBgFVxwQVyginWpr9bWt1qtWuvSamut1ldbtUoXW7VWrRtuuBXXuhcNyL6GsG8JW1gChMzc94+ZhBgSMklm5nlm5nw/Hz4mM5PMcZ7MeZ45995zxRiDUkqp8OWwOgCllFKBpYleKaXCnCZ6pZQKc5rolVIqzGmiV0qpMBdldQAtZWZmmoKCAqvDUEqpkDJ37tztxpis1u6zXaIvKCigoqLC6jCUUiqkiMi6tu7T0o1SSoU5TfRKKRXmNNErpVSY00SvlFJhThO9UkqFOZ8SvYhMEpEVIlIpIre0cv8JIjJPRBpEZGqL+y4TkVXef5f5K3CllFK+aTfRi4gTmAGcDpQB00WkrMXD1gPfB/7V4mfTgTuB44BRwJ0iktb1sJVSSvnKlyv6UUClMabKGFMPPA9Maf4AY8xaY8xCwN3iZ08D3jPG7DTG7ALeAyb5Ie6j7D/UwK9mLaH2wOFA/HrVBS/P3cjW2oNWh6FamL1kK5XVe60OQwWBL4k+D9jQ7PuN3tt84dPPishVIlIhIhU1NTU+/upvW751L8/OWcfVz8ylvqHl+UZZpbbuMDe+tICLH59DbZ2ehO3kZy8t4MLH5rBhZ53VoagAs8VgrDHmMWPMCGPMiKysVlfwtmt47zTuO38QX6zewa2vLEI3VLGHw27PSbeyeh8/fKZCT8I20uA2bN93iMv/8bWehMOcL4l+E9Cz2ff53tt80ZWf7bDzhuXz05P78vK8jTz8wapAPY3qALfbc8I9vk8G/63ayS0vL9STsE243IaRBWms27FfT8JhzpdE/zVQIiKFIhIDXAjM8vH3zwZOFZE07yDsqd7bAuYnE4uZOjyfh95fxcy5GwP5VMoH3jzP2YNzueGUvrzyzSYefF9PwnZgDIwoSOf+qYP1JBzm2m1qZoxpEJFr8CRoJ/CEMWaJiNwFVBhjZonISOBVIA2YLCK/NsYMMMbsFJHf4DlZANxljNkZoP8XAESEe84dyJbaA9zy8kJyU7pxfHFmIJ9SHYPLmzgcIlw7oZj1O+v44wer6JkWx7QRPdv5aRVILmNwCJwzNI8NO+v4/XsryU+P54ZT+lodmvIzn7pXGmPeBt5ucdsdzb7+Gk9ZprWffQJ4ogsxdlhMlIM/XzycaX/+kh8+M5eXrz6evjlJwQxBeTWWbhwOQUS497yBbK09yK2vLKJHShzjSvQkbBWX2+AUAeCaCcVs2OU5CeenxfFdPQmHFVsMxgZCcrdonrh8JHHRTi5/8muq9+j0Piu4m67oPd9HOx386eJh9MlK5Opn5rJiq07vs0JjiUa8iV5E+O25Axlfksltryzis1XbrQxP+VnYJnqAvNQ4nvj+SHbV1fM/T33N/kMNVocUcVzeK3pnY6bHcxJ+8vKRxMU4ufzJr9imJ+Gga+24RDsd/Ol7wyjO1pNwuAnrRA9QnpfCjIuGsXTzHn7y3Dc0uHRmQTC5m9Xom8v1noR3HzjMFXoSDrrGsZPmiR4gqVs0T3x/JPGxehIOJ2Gf6AFOKs3mrinlfLC8mrvfWmZ1OBGlcdZNy0QPR07Cy7bs5brnv2m6ylSB1zi5ppXD0nQSrvWehA/Uu4IbnPK7iEj0ABeP7s2V4wr5xxdr+dec9VaHEzGOlAhav/+k0mzunFzG+8uq+b/Zy4MYWWRrOi6tZXpgQG4Kj140jCWb93DjS/ObBtVVaIqYRA9w6xn9OalfFne8vpgvVutgUzA0JpTWrugbXTqmgItH9+KvH1fp2ocgaat009xJpdncdnp/3l60lYd0AWJIi6hE73QIf5w+lMLMBP732Xms3b7f6pDCnjlG6aa5OycPYGxxBre9soi56wK61EIBxjtUJe0clyvHFzJteD5//GAVbyzYHITIVCBEVKIHz2DT45eNRIAr/1nBnoPa4yOQfLlyBM+MjxkXDSM3tRtX/XMuG3dpo61Aajouxz4siAh3n1vOyII0bnppAQs27A58cMrvIi7RA/TKiOfPFw9n7fb9XPsvnYkTSK5mC6bakxofw98vG0m9y82VT1XoTJwAam16ZVtio5z85eLhZCXF8oN/VmjL6RAUkYkeYHRRBnefU87HK2u4520dBAwU02LBVHuKsxOZcdEwVlXv4/oXdBAwUFoumGpPRmIsj182kv2HGvjBPyt0Jk6IidhED3DhqF78z9hCnvh8Dc9/pTNxAqG92R2tOaFvFr88sz/vLd3G/e+uCFRoEc3Xklpz/bon8cfpQ1m8uZabZi7QBmghJKITPcBtZ5Tynb5Z/OK1xfy3aofV4YSdpqZmHUgoAJcdX8BFx/Xiz/9ZzSvzdCaOvzV+UOrICRhgYv8cbplUylsLt2gr8BAS8Yk+yungkYuGUpCZwNXPzGXdDp2J40++zrppSUT49dkDGFOUwS0vL2Luul0BiC5yNZbEOnhYALjqhCLOH+ZpBf7mQp2JEwoiPtGDp/fK45eNwABX/XOuDgL6UXsLpo6lsfdKj9Ru/PDpuToI6EcdGYxtSUS457xyRvT2zMRZunmPv8NTfqaJ3qt3RgKPTh/Gquq9/Ezrj37j6uCgX0tpCTH87dIRHKhv4Opn53KoQQcB/cHdiRp9c7FRTv588XBS42K46ukKdu2v92d4ys800TczriSTW70rAf/ycZXV4YQFYzo+GNtS35wkfv/dwXyzfje/mrXEX6FFNHcXT8AAWUmx/OWS4VTvOcS12jDQ1jTRt3Dl+EImD87l/2Yv5+OVNVaHE/Ia3/udvXJsNKm8Bz8+qQ/PfbWBZ+es80Nkka3puHQh0QMM6ZnK3eeU81nldu6frTOk7EoTfQsiwn3nD6RfThLX/mueDs520ZErx67/rhtO6ceJ/bL41awl2iahi1puCNMV3x3Zk0tG9+avn1RpmwSb0kTfiviYKP526QgcDuGHT8+lrl4HZzvL3YVBv5acDuHhC4aSmxrHj56Zp73Su6AjK5Z98cuzyhhZkMbNMxeybIsOztqNJvo29EyP55HpQ1m5bS8/m7lQB2c7yeWHGn1zKfHRPHbJCPYfauDqZ3RwtrPcfj4uMVEOZnxvGMlxUVz1dAW763Rw1k400R/D+JIsbvYuDnnsEx2c7YzGhTldGfRrqV/3JB6YNph563fz6zeW+u33RpKmDWH8mAGyk7rxl4uHs63WMzirG8nYhyb6dvzwhCLOHNSD+/69nC9X68rZjvJn6aa5Mwb24OoT+/CvOet5WXvYd5gv+wR0xtBeadw1ZQCfrtquK2dtRBN9OzyDs4MozEzg2ue+oVrrwh3SmV43vrrxlL6MKcrg9tcWsXyr1oU7oqvz6I/lwlG9mDY8n0c+XMV/VlT7/ferjtNE74PE2Cj+fPFw9h9q4BqdL9wh/px101KU08Efpw8luVs0Vz8zj726t4DP3AG6om9015Ry+uUkcf0L89m0+0BAnkP5ThO9j/rmJHHveQP5as1OnS/cAYG8cgTPop1HLxrG+p113KyD5j5rajYXoEQfF+PpYe9yGf732Xk6aG4xTfQdcM7QPM/epp9UMXvJVqvDCQn+WjB1LKMK07llUinvLN7K45+tCdjzhJPG82Egj0tBZgL3TxvMgg27+e1bywL2PKp9mug76JdnlTE4P4WbXlqgi6l8EMjSTXNXji/ktAE5/O6d5VSs1cVU7TkyGBvY55lU3p0fjC/kn1+u4/X5mwL7ZKpNmug7KDbKyYzvDcMhwo+emcfBw/qR9Fj8PV+7LSLC/dMGk58Wx4//NY/t+w4F9PlCXWf3CeiMmyeVMrIgjVtfWcSqbXsD/nzqaJroOyE/LZ6HLhjCsi17uOP1xVaHY2uBmsbXmuRu0fzpe8PZXXeY657XedzH4o9mc76Kdjp49KJhxMc4ufrZedoG3AKa6DvppNJsrjmpmBcrNvLaN/qRtC1HFuYEPqEAlOUm85sp5XxeuYMZH1UG5TlDUePYSTBOwAA5yd344/ShrK7Zxx2vawfSYNNE3wXXn1zCqIJ0bn91EWu2a72+NYFaMHUs00bkc86QXB56fyVfrdF6fWuO9LoJ3nMe3yeTayeU8PK8jbo9ZJBpou+CKKeDhy4cQnSUg2uf0ylkrXH5sUuir0SEu88dSO+MBK57/hvdFKMVJsDTXtvykwnFjCpM5xevLaaqZl9QnzuSaaLvotzUOO6fOpjFm/Zw79vLrQ7HdtwBnq/dlsTYKB6ZPpQd++q56SXdMaylQM+jb0uU08HDFw4hNsrBNf/6RiczBIkmej84pSyHy8cW8I8v1vLe0m1Wh2MrVpRuGpXnpXDrGaV8sLyaJz9fG/Tnt7NgDpK31CMljgemDWbplj387h29OAoGTfR+csvppZTnJfOzmQvYrEu+mzQNxlqQUAC+f3wBp5TlcO87y1i0sdaSGOzINB0Xa55/Yv8crhhXyD++WKuLD4NAE72fxEY5eWT6MA43uPmJ9sNpEqyFOW0REe6fOoisxFiueU774TRyWfhJq9HNk/oxMC+Fm2cu1H44AaaJ3o8KMxO457yBVKzbxUPva4tW8NToHeLffvQdlRofw8PTh7Jx1wFuf3Wx1uuxrkbfnOfiaCgut+Enz33DYb04ChifEr2ITBKRFSJSKSK3tHJ/rIi84L1/jogUeG+PFpGnRGSRiCwTkVv9HL/tTBmSx7Th+cz4TyVzqrR/vSfRW5dMGo0sSOenJ5cwa8FmXpmn6x5MEFfGHktBZgK/Pbecuet28eiHuu4hUNpN9CLiBGYApwNlwHQRKWvxsCuAXcaYYuBB4D7v7dOAWGPMQGA48MPGk0A4+9XZA+idHs8NLy6g9kBklwpcbuuTSaOrT/RM7btz1hI27KyzOhxLNTWbs8FJeMqQPM4blscjH65i7rpdVocTlny5oh8FVBpjqowx9cDzwJQWj5kCPOX9eiYwUTyf1Q2QICJRQBxQD4T9DhEJsVE8eMEQtu45yJ0R3iKhsXRjB06H8IfvDkaAn74wP6JbJDRNe7VJ8fbXZw8gNzWOn74wn33aIsHvfDnMecCGZt9v9N7W6mOMMQ1ALZCBJ+nvB7YA64EHjDFHLVUUkatEpEJEKmpqajr8P2FHQ3ul8ZMJJbw2f3NEd+1zu40trhob5afF85tzyqlYt4u/fLza6nAsY9X6hrYkdYvmwQuGsHFXHXe9oS0S/C3Q5/NRgAvIBQqBG0WkqOWDjDGPGWNGGGNGZGVlBTik4PnxSX0Y2iuVX7y2OGJnFbiMsU3pptGUIblMHpzLg++tZOHG3VaHY4lAbvHYWSML0vnfEz39o/69eIvV4YQVXxL9JqBns+/zvbe1+hhvmSYF2AFcBPzbGHPYGFMNfA6M6GrQoSLK6eChC4bgdhtufHF+0+KhSOJ222MwtjkR4e4p5WQnxXL98/Opq4+8UkGwm8356rqTSxiUn8Itryxim+7P7De+JPqvgRIRKRSRGOBCYFaLx8wCLvN+PRX40HiG9dcDEwBEJAEYDUTUUrjeGQncefYA/lu1k799WmV1OEHnNtbO1W5LSnw0D3x3MGt27I/I3Y/cFq9vaEu008GDFwzh4GEXN720ICIvjgKh3UTvrblfA8wGlgEvGmOWiMhdInK292GPAxkiUgncADROwZwBJIrIEjwnjCeNMQv9/T9hd9OG5zNpQHceeHcFSzZH1upMl02mV7bm+D6ZXDW+iGfnrOf9CGtd4bKoqZkv+mQl8oszy/h01Xae+nKt1eGEBZ9q9MaYt40xfY0xfYwxv/XedocxZpb364PGmGnGmGJjzChjTJX39n3e2wcYY8qMMfcH7n/FvkSEe84bSFp8DNc/Pz+iGjl5SjdWR9G2G07tS/8eyfz85YXU7I2cXansNhjb0veO68XE0mzufWc5K3VXqi6zyeSq8JeeEMP90wazqnofD7630upwgsZtjC2vGhvFRjl5+MIh7D3UwC9eWxQxq2bdFjY184WIcN/UQSTFRnHjiwt01WwXaaIPou/0zWL6qF489mlVxCwMcbntm0wa9c1J4oZT+jJ7yTZmLdhsdThB0bRgysYn4czEWO4+p5xFm2r5awRPhfUHTfRBdvuZ/clNieNnLy2IiBKO2xjbLMo5lh+ML2JIz1TunLWE6r3hP9vDbcGGMJ1x+sAenDWoBw9/sIrlW8N+rWXAhMBbMLwkxkbxf1MHUbV9P79/d4XV4QSc29hrwVRbnA7hgWmDqat3RUTjM7cxiMXN5nx115RyUuKitYTTBZroLTC2OJOLR/fi75+toWJteO9p6nLbb8FUW4qzE7np1L68t3Qbr88P7xKOy2Yrlo8lPSGGu88ZyJLNe/jzf7SE0xma6C1y6+n9yUuN42czF3KgPnxLOHbpXumrK8YVMayXt4QTxgt23Mb+YyfNTSrvztmDc/njB6tYullLOB2lid4iCd4Szprt+7l/dviWcNxuey2zb4/TIdw/bTAHD7u47dXwnYUTKmMnzf367AGkxsdw00sLqG/QEk5HhNihDi/H98nk0jG9efKLNXy1JjxLOC5vLTiU9MlK5Gen9eP9ZdW8+k14NqQLpdJNo7SEGO45t5ylW/Yw4yPtXd8Rmugt9vNJpfRMi+fmmQvCsueKsfk8+rZcPraQEb3T+NWsJWHZcyXUSmqNTh3QnXOH5jHjo0oWb4qsVeZdoYneYo0lnLU76sJy+0GXOzQTfWMJ51CDmzvCcE8BdwgNkrd05+Qy0hJi+PnLC3VvZh9poreB0UUZTB/Vi79/WhV2VykuExpT+FpTmJnA9Sd7FlL9e/FWq8PxK7s2m/NFanwMvz57AEs27+HJz9daHU5I0ERvE7ecXkpGYiy3vrIorK5SjDE4QzOfAHDl+EL690jmzlmL2XswfLaFdNlo56/OOL28Oyf3z+YP762M+G0hfaGJ3iZS4qL51eQBLNpUyz++WGt1OH4TqqWbRtFOB787byDVew+F1ewoO+4T0BEiwl1TynEI/OK18F/g1lWa6G3kjIGeq5Tfvxs+VykutwnZ0k2jwT1T+f7xBTz933Vh06PI7s3mfJGbGsfPTuvHxytrIqZHUWdporeRcLxKMSa05tG35cZT+9EjuRu3vbIoLOZwh0KzOV9cMqaAwT1TueuNpeyuq7c6HNvSRG8zualx3OS9SnljYejvm+kKgytH8PQo+s055azYtjcsdgoLxQVTrXE6hN+dN5DaA4e55+3I2ynMV2FwqMPPpWMKGJyfwl1vLAn5qxRP6cbqKPxjYv8czvR2Ulyzfb/V4XRJqDSb80X/Hsn84IQiXqzYyBert1sdji1porchp0O497xB7Ko7zL1vh/YWu6G6YKotd04uIzbKwW2vhHZ7BFeID8a2dN3EEnpnxHP7q4sjov13R2mit6my3GR+ML6IFyo28OXqHVaH02muMLpyBMhO6sZtZ/Tny6odzJy70epwOs1Tugmf49It2slvzxnImu37tT1CKzTR29h1E0vomR7HHa8vDtk+3C536C6YassFI3oysiCNe99ZTm1daM6tD7Vmc74YV5LJuUPz+OvHVVTV7LM6HFvRRG9jcTFO7jxrAKuq9/GPEF0B6CndWB2FfzkcntlRtQcO80CIbh4Tis3mfHHrGaXERjm4c9aSkC6t+VuYvQXDz8llOUwszeah91eytTb0mmuFWy24Uf8eyVw6pjfPzFnHoo2h17bCHeIL2dqSndSNG07ty6ertjN7SXi1regKTfQh4M7JAzjsNtz91lKrQ+kwV5jVgpv76Sl9yUiI5ZevL8btDq2rx1DtXumLS0b3prR7Ene9sTQsO8J2hib6ENArI57/PbEPby7cwueVoTV9LFwWTLUmuVs0t59ZyvwNu3mxYoPV4XSIyxC2J+Aop4PfnFPO5tqDPPKhDsyCJvqQ8aPv9KFXejx3vL44pFZmeko3VkcROOcMyWNUQTr3/Xs5u/aHzpqHUG82156RBemcPyyfv39aRWW1Dsxqog8R3aKd/OrsMlbX7OfJz9dYHY7PQmlz8M4QEe46ZwB7DjaE1MBsuI6dNHfL6aV0i3by6zd0YFYTfQiZUOoZmH3kw0pq9h6yOhyfmDCbR9+a0u7JXDK6N899tZ5lW0Jj4+pwPwEDZCXFcsMpnoHZD5ZVWx2OpTTRh5jbz+zPoQYXD4RIy1xXGA/6NXf9ySUkx0Vz1xtLQ+LqMZzHTpq7eHRvirMTufutpRxqiNwVs5roQ0xRViKXjSngxbkbQmI3KncYD/o1lxofww2n9OXLqh3MXrLN6nDa5QqTpmbtiXY6+MWZ/Vm7o46nwmifh46KgEMdfq6dWEJafExIXD165mtbHUVwXDSqF31zErnn7WW2v3qMhBp9oxP7ZXNSvywe+aCS7ftCo+TpbxHyFgwvKXHR3HhqX75au5O3F9l7UUiklG7AM63vl2eVsX5nHU98ttbqcI4p3JrNtecXZ5Vx4LCL34fQgLk/aaIPUReO7EVp9yTueXuZrbv1hfqWdR01viSLk/tn8+iHq6jea9+VzJF0Agbok5XIpWMKeP7rDSzZbP+Sp79pog9RTodwx+QyNu0+wN9tvBGG2xBRV44At59ZRr3LbesB83DZYaojrguhkqe/aaIPYcf3yeTUshz+8nGVbWuP4b5gqjWFmQlcNqaAmXM3snyrPadbhmOzufakxEfz05NLmLNmJx8uj6zplhF2qMPPzZNKOXDYxSMfrLI6lFaFW99zX10zoZjE2Cjue8eeG8dE0mBscxeO6kVhZgK/e2c5DSHa+rszNNGHuOLsRC4c2ZNn56y35fZ24dw861hS42P48UnFfLSihi9s2J8onJvNHUu008HPJ/VjVfW+kN44pqM00YeB604uISbKYcuasMsd/itj23LZ8QXkpcZx7zvLbdfdMlIWTLXmtAHdGdYrlT+8tzJiulv6lOhFZJKIrBCRShG5pZX7Y0XkBe/9c0SkoNl9g0TkSxFZIiKLRKSbH+NXeHpwX3VCEW8t2sI363dZHc63RMqCqdZ0i3Zy46l9WbSpljcWbrY6nG+JxLGTRiLCbWf0p3rvIR7/NHT6RnVFu4leRJzADOB0oAyYLiJlLR52BbDLGFMMPAjc5/3ZKOAZ4EfGmAHAiUBo7r1mcz8YX0RmYiz3vr3cNjMKGq9iIzWhgKe7Zf8eydw/e4WtFlFF6thJoxEF6Zw2IIe/fLzathMZ/MmXK/pRQKUxpsoYUw88D0xp8ZgpwFPer2cCE8WzUeipwEJjzAIAY8wOY4x9/trDSEJsFNefXMJXa3fy3lJ7LMF3eU84kVoiAM+nmdvOKGXjrgM8/eU6q8NpEmnrG1pz86RSDja4efh9e05k8CdfEn0e0HxXhY3e21p9jDGmAagFMoC+gBGR2SIyT0Rubu0JROQqEakQkYqampqO/j8orwtG9qQoK4EH3l2BywY1Ybc30UfylSN4FlGNL8lkxkeV7D1ojw+0rgjoKtqePlmJTB/Vk+e+Ws/6HXVWhxNQgR6MjQLGAd/z/vdcEZnY8kHGmMeMMSOMMSOysrICHFL4inY6uOGUvqzcto83bVATdntnr0X6lSPAz07rx666wzxpk03eI3nspLlrJ5TgdAgP23R6sr/4kug3AT2bfZ/vva3Vx3jr8inADjxX/58YY7YbY+qAt4FhXQ1ate2M8h7075HMg++t5LDF84SbSjc6t4tB+amcWpbD3z6pYned9TtRuSN4MLa5nORuXDqmN69+s5HK6r1WhxMwvrwFvwZKRKRQRGKAC4FZLR4zC7jM+/VU4EPjGRGcDQwUkXjvCeA7QOjtcB1CHA7hxlP6snZHHa/Ms3aecFPpRq/oAbjh1L7sq2/gsU+sb1nhirCmZsfyo+/0IS7ayYNhXKtvN9F7a+7X4Enay4AXjTFLROQuETnb+7DHgQwRqQRuAG7x/uwu4A94ThbzgXnGmLf8/n+hvmVi/2wG90zljx9UWjrT48isG00o4NmJavKgXJ78fK3lMz10MPaIjMRY/mdcIW8t3MLSzfZsWdFVPn2oNsa8bYzpa4zpY4z5rfe2O4wxs7xfHzTGTDPGFBtjRhljqpr97DPGmAHGmHJjTKuDscq/RISbTu3Lpt0HeP6rDe3/QIA0jgfrleMR159cQr3LzZ8+Wm1pHG6jJ+DmrhxfRHK3KP7wnv0WHfqDVk/D1LjiTI4rTOfRjyo5UG/NVX3jzB8d9DuiKCuR84fl8cycdWypPWBZHK4I2hDGFylx0Vx1QhHvL6u23aJDf9BDHaZEhBtP7UfN3kP888u1lsRwpEZvydPb1rUTSjDG8MiHlZbFEOkLplrz/bGFpCfE8Pt3V1odit9pog9jowrTGV+SyV8/qbKkp4dbF0y1qmd6PBeM7MlLFRvYtNuaq/pIbTZ3LImxUVz9nT58VrmdirU7rQ7HrzTRh7nrJpawc389/5qzPujP7dLB2DZdfWIxAI99bE2tPpKbzR3L90b3Ij0hhkc/su7TViBoog9zIwrSGVOUwWOfVAV9y8GmBVNaIjhKXmoc5w/L57mvN1C9J/hbDuqCqdbFx0RxxbhC/rOihoUbd1sdjt9ooo8A104opnrvIV4Kcv9tty6YOqarT+xDg8vN34K8FaQ2mzu2S8f0JrlbFI9aOIbib/oWjABj+mQwvHcaf/nPauobgrda1qULpo6pd0YCU4bk8cx/17Nzf/BWy+rYybEldYvm8rGFvLt0m223guwoTfQRQES4ZkIxm3Yf4LVvWnavCBxdMNW+H5/Uh4MNLh7/LHhX9S5tNteuy8cWkBDjDJurek30EeLEvlkMzEthxn8qg7ZXpi6Yal9xdhJnlPfgqS/WUVsXnM6W2myufanxMVwypoC3Fm1hdc0+q8PpMk30EaLxqn7djjreXLglKM/p0lqwT358UjH7DjXwVJDWO+jYiW+uHF9IbJTD8lXM/qCHOoKc0j+HfjlJzPioMih7mGpTM9+U5SZzcv9snvh8TVDWO+jYiW8yE2O5aFRvXpu/iQ07Q7tfvSb6COJwCD86sYhV1fv4eGXgN3g5cuWoCaU9V5/Yh911h3mpIvAzo3TsxHdXji9EwDb7CHSWJvoIc9agXLondwtKq1xdMOW74b3TGdYrlcc/WxPw3cF07MR3ualxTB6cywtfr6f2gD12B+sMTfQRJtrp4PKxBXxZtYPFm2oD+ly6lWDHXHVCEet31vHukq0BfR4dO+mYK8cXsr/exXNfBX91ub9ooo9A04/rRWJsVMAX6jRdOeoVvU9OKetO74x4/vpJFcYE7qpeT8AdMyA3hbHFGTz5+ZqgrkPxJ030ESi5WzQXjuzJmwu3BLSpll45dozTIVwxrpD5G3Yzd13gWuXqgqmO+8H4IrbtOcQbC6zfi7kzNNFHqMvHFQLw5GdrAvYcbu1H32FTh+eTGh8d0DEUHTvpuO/0zaJfThJ/+zSwn7YCRRN9hMpLjePMgT14/usN7DkYmEGmxtKNJhTfxcdEccno3ry3bBtrtu8PyHNos7mOExGuHF/I8q17+axyu9XhdJgm+gj2g/FF7DvUwPMBGmRy6cKcTrl0TAHRDkfA2iLogqnOOXtILtlJsbbY3L2j9FBHsIH5KYwuSufJz9cGpC2CLpjqnKykWM4dmsdLFRvZFYBmZ7pgqnNio5xcdnwBn67azoqte60Op0M00Ue4y8cWsqX2IO8vq/b779aFOZ13+bgCDjW4mRmA1tJ6XDrvolG9iI1y8PR/11odSodooo9wE0uzyU3pxjP/Xef339046KcLczqutHsyowrSeWbOOr+3q9AFU52XlhDD5MG5vDpvE3sDNLYVCJroI1yU08FFx/Xis8rtfu/Sp4OxXXPxmN6s21HHJ6v8265Cp712zSWje7O/3sWrQWz53VWa6BUXjOxFtFP8flV/ZGGOX39txJg0oDuZibE8/WWAjouegDtlcM9UBuWn8PSX60JmqqW+BRVZSbGcXt6DmXM3+rV7YlPpRhNKp8REOZg+qicfrqj2a/dEbTbXdZeM7s2q6n38t2qn1aH4RBO9Ajz7ZO492MDr8/238k+X2nfdRcf1wiHCs3P8NwVWF0x13eTBuaTERQdkbCsQNNErAIb3TqO0e5JfP45qiaDreqTEcXL/bF6s2MDBwy6//M6msRM9AXdat2gn3x2Rz+wlW9m256DV4bRLE70CPCv/Lh1TwNIte5i33j99Vhqn5mvppmsuHVPAzv31vL3IPzuDHTkB++XXRayLR/emwW1CoqulJnrVZMqQXJJio/w2+NeYUDTPd83xfTIoykrgaT+VCXTsxD96ZyTwnb5ZPPfVeg4HaR/mztJEr5okxEZx/vB83l601S8rMt06j94vRISLj+vNN+t3s3Tzni7/Ph078Z9LRvdm255DfBCABYf+pIlefcsFI3tS73Izyw/tWF06u8Nvzh2aR4zT4ZeVsk1NzfSKvstO7JdFdlJsQFYw+5MmevUt/XskMyA32T8JxTvop/mk69ISYji5LJvX5m/q8uYX2mzOf6KcDs4dlsdHK6qp2XvI6nDapIdaHWXq8HwWbapl+daulQncWgv2q6nD89m5v57/rOhamUBnQ/nXtOH5uNyG1+fbd6WsJnp1lClD8oh2CjMrunZVr71u/OuEkiwyE2N5qYuftrSpmX8VZycxpGcqM+dutO1KWU306ijpCTFMLM3htfmbujSb4MisG00o/hDldHDesDw+Wl7N9n2dLxPoCdj/pg7PZ/nWvSzxw2B5IGiiV62aOjyf7fvq+XhF5xtq6VJ7/5s6PJ8Gt+nSCmZtNud/kwflEhPln8HyQNBEr1r1nX5ZZCbGdOkPVxdM+V/fnCQG56d06bhoszn/S4mP5tQyz6fgQw3+WcHsTz4dahGZJCIrRKRSRG5p5f5YEXnBe/8cESlocX8vEdknIjf5KW4VYNFOB+cMyeOD5dvY2ck59bpgKjCmDs9n2ZY9LN5U26mfb/qkpQfGr6YOz2d33WE+tOGc+nYTvYg4gRnA6UAZMF1Eylo87ApglzGmGHgQuK/F/X8A3ul6uCqYpo7I57Cr87MJdMFUYEwenNulOfVNTc30uPjV+JIscpLtOafelyv6UUClMabKGFMPPA9MafGYKcBT3q9nAhPFOwInIucAa4AlfolYBU1p92QG5nW+TNC0k5FeOfpVanwMp5Tl8Hon59Tr9MrAcDqE84bl85+VNVTvtVejM18SfR6wodn3G723tfoYY0wDUAtkiEgi8HPg18d6AhG5SkQqRKSipsa/u+morjl/WB5LNu/p1GbILi3dBMz5w/PYVXe4U3Pq3Tp2EjDnD/PMqZ/lx3bf/hDo4ZhfAQ8aY465R50x5jFjzAhjzIisrKwAh6Q64sxBuTgE3lzY8T9ct9vgEJ1eGQjjS7JIi4/mzYUd72jp0sHYgCnOTmRAbnKnjksg+XKoNwE9m32f772t1ceISBSQAuwAjgP+T0TWAtcDt4nINV0LWQVTVlIsx/fJ5I0Fmzu8GMRtjJYHAiTa6eD0gT14b+m2Du8KpgumAmvy4Fzmb9jt113BusqXRP81UCIihSISA1wIzGrxmFnAZd6vpwIfGo/xxpgCY0wB8BBwjzHmUf+EroJl8uAerN1Rx+JNHVsM4jJGB/wCaPKgXA4cdvHh8o6Vb5rGTvTYBMSZA3sA8EYnPgUHSruJ3ltzvwaYDSwDXjTGLBGRu0TkbO/DHsdTk68EbgCOmoKpQtdpA7oT7ZQO/+G63UbrwAE0qjCd7KRY3uhgp1EdOwmsnunxDOuVyhsL7FO+8alKZ4x52xjT1xjTxxjzW+9tdxhjZnm/PmiMmWaMKTbGjDLGVLXyO35ljHnAv+GrYEiNj+GEkizeWril6WO/L9xGdzEKJKdDOHNQDz5aUcPeg4d9/jltNhd4kwfnsmzLHiqrOz6JIRB0OEb55KzBPdi0+wDfbNjt88+43Fq6CbSzBuVS3+Dm/WXbfP4ZbU0ReGcO7IEIthmU1USvfDKhNIdop/Dukq0+/4zbGE0mATa0Zyo5ybHMXux7om9cMKWzoQInO7kbw3ulMXuJ78clkDTRK5+kxEVzfJ9M/r1kq8+zb3TWTeA5HMKpZd35eGUNB+p967GiV/TBMam8O8u27GH9Dutn32iiVz6bVN6ddTvqWO7j4imXW6fwBcOk8u4cOOzik1W+LTbUFcvBcdqA7gDM7sCn4EDRRK98dkpZDiLw78W+/eG63Ua3qwuCUYXppMZHM9vH43KkdBPIqFTP9HgG5Cbzb030KpRkJsYysne6z1coWroJjming5P75/D+sm0+9b4xWroJmtMGdGfuul1U77G2940metUhp5V3Z/nWvazdvr/dx7o00QfNaQO6s+dgA3PW7Gj3sbpPQPBMKveUb95dau2grCZ61SGn9M8B8Gk1pqd0o8kkGMYVZxIb5eADH3qh64Kp4CnJTqRXenyHVy/7myZ61SG9MuLpk5XARz50TdQFU8ETF+NkbHEmHy6vbndWlDHabC5YRIQJpdl8sXo7Bw9bt/OUJnrVYRNKs5lTtZN9h47dTEt73QTXSaXZrN9Zx+qaY5fVXPpJK6hOKs3m4GE3X65uv6wWKJroVYdNKM2h3uXms1Xbj/k4T5tiTSjBMqE0G4CP2ikTuIzRq/kgOq4wnfgYJx8st65Or4leddiIgjSSukW1m1DcRpuaBVNeahyl3ZPaTSjG6EBsMHWL9pTVPlpe0+FW3/6iiV51WLTTwQklWXy04tj1YJdb9yUNtpNKs6lYu4s9x2hypqWb4JtQms2m3QdYue2YezAFjCZ61SknlWZTvfcQSza33aO+cdBPBc+E0mwa3IZPV7ZdVnO5jc64CbKT+nnKalbNvtFErzrlhJJMAD6rPEZC0aZmQTe0ZyqJsVHHPC5Gj0vQdU/pRr+cJD4/xnEJJE30qlOyk7vRNyfxmH+4Lh2MDboop4PRRRnHPi66kM0SY4sz+WrtTkumWWqiV502tjiTr9a0/YdrdB69JcYVZ7B+Z12bXRO12Zw1xpVkUN/gZu66XUF/bk30qtPGl2RyqMHNvDb+cHXQzxrjvGW1z1e3flXvKd0EMyIFcFxhBlEOOWZZLVD0cKtOG9XOH66WCKzRJyuRnOTYto+LltQskRAbxbBeaZbU6TXRq05LjI1iaK/UNv9wjSZ6S4gIY4sz+aJye6t7/HpaU+hxscLY4kwWbapld119UJ9XE73qkrHFmSzcVEtt3dHztrV0Y51xxZnsqjvM0i1HT3/VLR6tM64kA2PgiyC3Q9BEr7pkbHEmxsB/W2mP6zK6YMoqY4s9dfrW+qt4SjfBjkgBDMpPJSHGyRdtjJ8EiiZ61SWD8lOIiXK0OpNAF0xZJye5G73S46lYt/Oo+9zabM4y0U4Hw3qnUbE2uDNvNNGrLomNcjI4P4Wv1x6dUFxu7XVjpREFnoTSsk2F9iCy1oje6azYtpfaA223qfA3TfSqy0YUpLN4Uy0H6r89n96z1F4TilVGFqSzY389a1rsBqazbqw1siANY2De+uBd1WuiV102siCNwy7Dgo27v3W7Meh8bQuNLEgDOKpM4NaxE0sN6ZWK0yFUtPIpOFD0bai6bHivdICj/nC11421+mQlkhYffVRZzbPFo0VBKeJjoijPTebrINbp9XCrLkuJj6ZvTuJRf7huLd1YSkQY3jv9qIFyXchmvREF6SzYsJv6BndQnk8TvfKLEQXpzFu3C1ezBTo66Ge9kQVpVG3fz/Z9h5pu0wVT1htZkMahBjeLN9cG5fk00Su/GFmQxt5DDSzfemSBjpZurDei4OiymlsXsllueG/Pcfl6TXDq9JrolV8M6+UZ+Fuw4cgVituNbnBhsfK8ZKKdwvxmx0UXTFkvKymWXunxR01gCBRN9MoveqXHk9wt6lsfRbV0Y73YKCf9uiexeNO3j4uWbqw3MC+FxZva3qHNnzTRK78QEcrzUr6VULTXjT2U56aweHNt08IpTfT2MCAvmfU761rtE+VvmuiV35TnpbB8y14OuzwzCdwGnXVjA+V5KeyuO8zGXQcAz3HRE7D1BualALAkCAOymuiV35TnpVDvcrNy216gsUuixUEpylskFJdbe93YQXmu57gs2qSJXoWQ8txkAJZ4647a68YeSrsn4XRIU0Jxa7M5W0hLiCEvNY7FmwNfp9dEr/ymICOBxNiobyUULd1Yr1u0k5LsRBZ5T8A6SG4f5XnJ3xrXChSfEr2ITBKRFSJSKSK3tHJ/rIi84L1/jogUeG8/RUTmisgi738n+Dl+ZSMOhzAgN7lp5o3O17aPgXkpLNnkGZB1ubXXjV0MzEthzfb97D0Y2AHZdhO9iDiBGcDpQBkwXUTKWjzsCmCXMaYYeBC4z3v7dmCyMWYgcBnwtL8CV/ZUnpfCsi17aHC5vUvtrY5Igee47Nhfz5bag7h1Hr1tDGgaPwls+caXK/pRQKUxpsoYUw88D0xp8ZgpwFPer2cCE0VEjDHfGGM2e29fAsSJSKw/Alf2NDAvhYOH3ayu2a9dEm2kcUB28aZa3UrQRhoHZANdvvEl0ecBG5p9v9F7W6uPMcY0ALVARovHnA/MM8YcanE7InKViFSISEVNTY2vsSsbGtA4ILu51lO60VqwLZT1SEbEc+WoTc3sIysplpzkWJba4Iq+y0RkAJ5yzg9bu98Y85gxZoQxZkRWVlYwQlIB0jsjAadDWLN9vyYUG4mLcZKXGsea7fu9pRs9LnZRnJ1IVYvNYfzNl0S/CejZ7Pt8722tPkZEooAUYIf3+3zgVeBSY8zqrgas7C0mykHPtDiqavZjtHRjK4WZCVRt36cLpmymMDOBqpp9R2356E++JPqvgRIRKRSRGOBCYFaLx8zCM9gKMBX40BhjRCQVeAu4xRjzuZ9iVjZXlJVIZfU+AC3d2EifrETW1OzXrQRtpigzkT0HG9ixvz5gz9FuovfW3K8BZgPLgBeNMUtE5C4ROdv7sMeBDBGpBG4AGqdgXgMUA3eIyHzvv2y//18oWynKTGjap1QvHO2jKCuB/fUuavYe0uNiI0VZCQBU1QSufBPly4OMMW8Db7e47Y5mXx8EprXyc3cDd3cxRhViirISqff2u9HSjX0UZSYCUO9ya+nGRvpkeY5LVc0+RhWmB+Q5dGWs8rvCzISmrzWh2EfjlSPoCdhOclPjiIlyBHRAVhO98rs+zROK5hPb6J7cjbhoJ6DHxU6cDqEgIz6gpRtN9MrvspJiSYz1VAV10M8+HA6hwPtpSwfJ7aUoM5Gq7fsC9vs10Su/E5GmMoEmentpPC7abM5eirISWL+jrmkvB3/TRK8CorFOrzV6e+mjx8WWirISaXAbNuysC8jv10SvAqJxhocO+tlLkXeGhyZ6e2m8MApUnV4TvQqII6UbiwNR33KkdGNxIOpbGicwrAnQzBtN9CogGhOKDvrZS6EOxtpSanwM6QkxARuQ9WnBlFIdVdo9mWsnFHNiP10IbSdJ3aK5/Yz+jC3OtDoU1cKUIbnkp8UH5HdLIBvpdMaIESNMRUWF1WEopVRIEZG5xpgRrd2npRullApzmuiVUirMaaJXSqkwp4leKaXCnCZ6pZQKc5rolVIqzGmiV0qpMKeJXimlwpztFkyJSA2wrgu/IhPY7qdwAsHu8YHG6C8ao39ojL7pbYzJau0O2yX6rhKRirZWh9mB3eMDjdFfNEb/0Bi7Tks3SikV5jTRK6VUmAvHRP+Y1QG0w+7xgcboLxqjf2iMXRR2NXqllFLfFo5X9EoppZrRRK+UUmEubBK9iEwSkRUiUikit1gdD4CI9BSRj0RkqYgsEZHrvLeni8h7IrLK+980i+N0isg3IvKm9/tCEZnjfS1fEJEYK+PzxpQqIjNFZLmILBORMXZ6HUXkp95jvFhEnhORbnZ4HUXkCRGpFpHFzW5r9XUTjz96410oIsMsiu9+73FeKCKvikhqs/tu9ca3QkROC3R8bcXY7L4bRcSISKb3+6C/hr4Ii0QvIk5gBnA6UAZMF5Eya6MCoAG40RhTBowGfuyN6xbgA2NMCfCB93srXQcsa/b9fcCDxphiYBdwhSVRfdvDwL+NMaXAYDzx2uJ1FJE84CfACGNMOeAELsQer+M/gEktbmvrdTsdKPH+uwr4s0XxvQeUG2MGASuBWwG8750LgQHen/mT971vRYyISE/gVGB9s5uteA3bZ4wJ+X/AGGB2s+9vBW61Oq5W4nwdOAVYAfTw3tYDWGFhTPl43uwTgDcBwbPCL6q119aiGFOANXgnDzS73RavI5AHbADS8ezD/CZwml1eR6AAWNze6wb8FZje2uOCGV+L+84FnvV+/a33NTAbGGPFa+i9bSaei461QKaVr2F7/8Liip4jb7RGG7232YaIFABDgTlAjjFmi/eurUCOVXEBDwE3A27v9xnAbmNMg/d7O7yWhUAN8KS3xPR3EUnAJq+jMWYT8ACeK7stQC0wF/u9jo3aet3s+D76H+Ad79e2iU9EpgCbjDELWtxlmxibC5dEb2sikgi8DFxvjNnT/D7jOe1bMsdVRM4Cqo0xc614/g6IAoYBfzbGDAX206JMY/HrmAZMwXNCygUSaOWjvh1Z+bq1R0Rux1P+fNbqWJoTkXjgNuAOq2PxVbgk+k1Az2bf53tvs5yIRONJ8s8aY17x3rxNRHp47+8BVFsU3ljgbBFZCzyPp3zzMJAqIlHex9jhtdwIbDTGzPF+PxNP4rfL63gysMYYU2OMOQy8gue1tdvr2Kit18027yMR+T5wFvA978kI7BNfHzwn9QXe904+ME9EumOfGL8lXBL910CJd5ZDDJ4Bm1kWx4SICPA4sMwY84dmd80CLvN+fRme2n3QGWNuNcbkG2MK8LxmHxpjvgd8BEy1Or5GxpitwAYR6ee9aSKwFJu8jnhKNqNFJN57zBvjs9Xr2Exbr9ss4FLvzJHRQG2zEk/QiMgkPOXEs40xdc3umgVcKCKxIlKIZ8Dzq2DHZ4xZZIzJNsYUeN87G4Fh3r9TW7yGR7F6kMCPgyVn4BmhXw3cbnU83pjG4flYvBCY7/13Bp46+AfAKuB9IN0GsZ4IvOn9ugjPG6gSeAmItUF8Q4AK72v5GpBmp9cR+DWwHFgMPA3E2uF1BJ7DM25wGE9CuqKt1w3PQPwM73toEZ5ZRFbEV4mnzt34nvlLs8ff7o1vBXC6Va9hi/vXcmQwNuivoS//tAWCUkqFuXAp3SillGqDJnqllApzmuiVUirMaaJXSqkwp4leKaXCnCZ6pZQKc5rolVIqzP0/wMN3fgCsu/MAAAAASUVORK5CYII=", + "text/plain": [ + "
    " + ] + }, + "metadata": { + "needs_background": "light" + }, + "output_type": "display_data" + } + ], + "source": [ + "scheduler = PolyLRScheduler(optimizer, t_initial=t_initial, cycle_limit=3)\n", + "plot_lr(scheduler)\n", + "total_epochs = scheduler.get_cycle_length()\n", + "print(f\"{t_initial=}\")\n", + "print(f\"{total_epochs=}\")\n" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### `cycle_decay`" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "When `cycle_decay` > 0 and <1., at every cycle the starting learning rate is decayed by new learning rate which equals `lr * cycle_decay`. So if `cycle_decay=0.5`, then in that case, the new learning rate becomes half the initial `lr`. \n", + "Default is 1., its mean no decay." + ] + }, + { + "cell_type": "code", + "execution_count": 16, + "metadata": {}, + "outputs": [ + { + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXoAAAD4CAYAAADiry33AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/YYfK9AAAACXBIWXMAAAsTAAALEwEAmpwYAABMiElEQVR4nO2dd1hU19aH3z0zVEFEwEaRagEVKWo0iT2WxGhM7CmaZorJl5vebnq5yU1umulVY2LsMSbRVI0msUTsYsWOFVFBRQRm9vfHGRARZICZOTPDfp9nHodz9pyzOEd+Z81aa68tpJQoFAqFwnMx6G2AQqFQKByLEnqFQqHwcJTQKxQKhYejhF6hUCg8HCX0CoVC4eGY9DagIqGhoTI6OlpvMxQKhcKtWLVq1VEpZVhl+1xO6KOjo8nIyNDbDIVCoXArhBB7qtqnQjcKhULh4SihVygUCg9HCb1CoVB4OEroFQqFwsNRQq9QKBQejk1CL4QYIITYKoTIEkI8Vsn+7kKI1UKIEiHEsAr7xgohtltfY+1luEKhUChso1qhF0IYgfeAgUAiMFoIkVhh2F5gHDC1wmcbA88AXYDOwDNCiOC6m61QKBQKW7HFo+8MZEkpd0opi4BpwJDyA6SUu6WU6wFLhc/2B36VUh6TUh4HfgUG2MHuCzhz+iTL37uNvGM5jji8og7MWpXNwbwzepuhqMBPGw+x7fBJvc1QOAFbhD4c2Ffu52zrNluw6bNCiPFCiAwhREZOTu2Ees/GZaQe+ZbD719J/oncWh1DYX/yCop5aOY6Rny0jAMnlNi7Eo/MWsfwD5eReSBPb1MUDsYlkrFSyo+llOlSyvSwsEpn8FZLmy792HT5u8QU7+DAe4M4lX/czlYqakORWfuSt+/YGcZ8spzD+YU6W6QopdgsyTtTzA2frmDrIeXZezK2CP1+ILLczxHWbbZQl8/WmI59R7Ox25vEF21h77tXU3BKeSp6Y7GuYDYsLYKck2cZ/clyck6e1dkqBYBZSgYkNcPbZOD6T5eTdUSJvadii9CvBBKEEDFCCG9gFDDPxuP/DPQTQgRbk7D9rNscRkr/sazr/Bqtz25k58TBFBaccuTpFNVgtmhCn94ymC9u7szBE4Vc/+lyck8psdcbi0USE9aAqbdfAgjGfLKCXUdP622WwgFUK/RSyhLgHjSB3gzMkFJmCiGeF0IMBhBCdBJCZAPDgY+EEJnWzx4DXkB7WKwEnrducyhpV93GmrT/kFi4jm3vDKbwjPrPqxelQm8wCDrHNOazcensyS3ghs/+4URBkc7W1W/MUmIUgriwAL65vQtmi2TMJ8vZm1ugt2kKO2NTjF5KOV9K2UpKGSelfMm67Wkp5Tzr+5VSyggpZQMpZYiUMqncZz+XUsZbX1845te4kPTBd7Eq+Tk6FK5i6ztDKTqrYsN6UBq6MQoBQLe4UD65KZ0dOae48bN/yDtTrKd59RYpJVJqD2CAhKaBfHVbF84Umxn9yXKyjyux9yRcIhnrKDpdex8rkp4i+cwKMt++luIiFS5wNqUevdEqKADdW4Xx0Q1pbDmUz9jP/+FkoRJ7Z1N2X8S5+9K2eUO+urULJwuLGfPJClUS60F4tNADdBn+EMtbP0pKwd9seGc4JcUqXOBMSj16QzmhB+jVpgnvjUll4/48bv5iJafPluhhXr3FXPpNq4ICtAsP4stbu3DsdBFjPlnBEVUl5RF4vNADXDL6CZYnPEDqqcWsefd6LGaz3ibVG6zVled5jqX0S2rGxNEprNl3gtsmZ1BYrO6Ls7BY70vFBzBAx8hGTL6lE4fzCxnz6QqOnVbOkbtTL4Qe4JLrn2FZyzvplPcLKz8cj7RUnMSrcATnQjeV7x/Yvjn/G57M8l253PvNGorN6r44A7O8MHRTnrSWjflsbCf2Hitg3BcqvObu1BuhB7hk7H9Y3nQ0XXJmseLzh/Q2p15QFrqpQlAArkkJ5/nBSfy66TCPzFqPxfpwUDiOsiR5JR59KV3jQnh/TCqZB/K5/Uv1jcudqVdCLwwGutzxPv80upJLsj9j+dfP622Sx1NZMrYybuwazUP9WvHtmv08930mUiqxdySlD9OLPYAB+iY25X/Dk1mx6xj3TF2tvnG5KfVK6EET+7R7prA6oDuXbP8fK+e8rbdJHk1VydjKmNArnvHdY5m8bA9v/LrN0abVa2x9AMO5b1y/bT7CwzPXqW9cbohJbwP0wGgykXTPdNa/NYjUdc+w2q8hqQNv1tssj6RiHf3FEELw+MA25J8pZuLCLBr6enF791hHm1gvMdfgAQzaN678whJe+3krgb5ePD8kCWHDPVW4BvVS6AF8fP2Jv+dbtr89gHbLH2SDfxDte1yrt1keR1nVjY2CIoTgpaHtOVlYwkvzN9PQz8TITlEOtLB+YrlINVRV3N0zjrwzxXy8ZCdBfl481L+1g6xT2Jt6F7opj39AEM3v/p59pijiFt7Jln9+1dskj8NsYyy4PEaD4M2RHenRKozH52xg/oaDjjKv3lJVHf3FKP3GNapTJO8uyuLjJTscZJ3C3tRroQcICg4l6PbvyTWE0GL+TezYsFxvkzwKW6o7KsPbZODDG9JIjQrmvmlrWLxNLShjT2xNxlak9BvXVR2a8/L8LUz7Z68jzFPYmXov9AChzSIxjZvLGfwImj2SA7u36m2Sx1BdHf3F8PM28tm4TiQ0CeTOKatYu++EfY2rx9QkGVsRo0Hw5gjtG9cT327gl8xD9jZPYWeU0Ftp3rI1Z0bOwJtizJOv4XiOChfYA7MNdfQXI8jPi0m3dCI00JtbJq1UbXTthLmW37RK8TYZ+OCGVNpHNOLeb9aQsdvhTWkVdUAJfTmi26azf+AXhFlyOPLRELVwiR2w1MFzLKVJoC+Tb+4MwE2fr+DISdV/pa7UNnRTHn9vE5+PTadFIz9unZzBdrX+rMuihL4Cbbv0Z/OlbxFfvI3t76kmaHWlNsnYyogNC+DzcZ04erKIm79YySnVBK1O1NWjLyUkwIcvb+mMt8nA2M//4VCeegi7IkroKyGl3w1kJD1J8pkVrH5vrOqLUwdqm4ytjI6RjXj/hlS2HDrJnVNWUVSi7kttsdcDGCCysT9fjOtEfmEJYz9Xawy4Ikroq6DLiIdZHnErnU/MV31x6kBN6+iro1frJrxybXv+yjrKI7PULM3aYrHzfWkXHsRHN6ax8+gp1RfHBVFCfxG63PI6/wRfxSXZn7Fixmt6m+OW1DUZWxnD0yN5uH9r5q49wCs/bbHbcesTtamjr45L40P534iO/LPrGPdPX1v2rUGhP0roL4IwGEi9exLr/LqQnvkS6xZO09skt8MeydjKuLtnHDd1bcnHS3YyZdluux67PmDP0E15Bie34N9XtWXBxkP8Z/5mux5bUXuU0FeDycub+LtnsMsUS8Li/yNr3d96m+RWVLZknT0QQvDM1Un0adOEZ+ZlsnDLYbse39OxZ+6kIrdeFsPYri359K9d6iHsIiiht4EGgY0IunUO+SKQht/ewOFsNfXbVs41z7L/sY0GwTujU2jbvCH3TF1D5gFVDmsrjnoAg/YQfrrcQ3jRliN2P4eiZiiht5GwFtGcGT4Vf3mG059fy8k8NUHEFhwVuimlgY+Jz8d1opGfF7dMWqkWtLaRsjp6B92X8x/Cq9VDWGeU0NeAmKQu7Or9PlHmvez6YISqsbeB6passwdNG/ry+c2dOH3WzC2TMlSNvQ3Yq47+YpQ+hIPUQ1h3lNDXkPY9rmV1+6foULiSVR/epmrsq6HUc3R07/I2zRry/vWpbDt8kglfr6ZErYR0URyVjK2Iegi7Bkroa0HnYQ+wrPlNdMn9jhVfP6u3OS5NXZpn1ZTurcJ48Zp2LN6WwzPz1HKEF8ORydiKqIew/iihryVdbnuLVQE96Zz1Dmt/naq3OS6L2aq1jgzdlGd05yju7BHH1yv2Mmnpbqec0x0x12LhkbpQ/iH84o+q7NLZKKGvJQajkaS7v2aHVzwJf93PrswVepvkkpxL+jnvnI/0b80ViU154YdNLFF97CvFrMN9Gd05ilsvi2HS0t18o/rYOxUl9HXA1z+AoJtnUiD88Jl1A8eO7NfbJJfDGUm/ihgMgrdGdqRV00AmTF3NjpxTTju3u+DM0E15Hh/Yhh6twnhq7kaW78x16rnrM0ro60iT8BiOD55EY8txDn0ygqKzqntfeZyV9KtIAx8Tn45Nx9to4PbJGeQVqEZb5XFkHf3FMBkNTByTQssQf+76ahX7jhU49fz1FSX0dqBVak82dnqZxOKNrP3wVlWJUw5H19FfjIhgfz68MY19xwu45xuVBCxPqUfv6Gqoymjo68WnYzthkXDr5JWcLFQPYUejhN5OpA8az7Lwm+l8/AdWTP+P3ua4DM6oo78YnaIb89I17flz+1GVBCyHM6uhKiMmtAHvX5/KjpzT/GuaaoDmaJTQ25Eut/yPNf6X0mnLa2xYPEdvc1wCR8/AtIURnSK5zZoEnLpCJQFBv9BNeS6ND+XZqxP5fcsRXvtZrdPsSGwSeiHEACHEViFElhDisUr2+wghplv3rxBCRFu3ewkhJgshNgghNgshHrez/S6FwWik1V1T2WNsSctFE9iXtUFvk3THLKVuXmN5Hr+yLT1ahfH0dxtZqdY3LQvdOLPqpjJu7BrN9V2i+HDxDr5bq4oZHEW1t1kIYQTeAwYCicBoIURihWG3AsellPHAm8Cr1u3DAR8pZXsgDbij9CHgqTQIbITf2BmYMVIy9XpOnzyht0m6Yrbo6zWWUtp7JbKxP3d/vZrD+fU7aW7vBWHqwrODk+gc05hHZ69n04F8vc3xSGx5nncGsqSUO6WURcA0YEiFMUOAydb3s4A+QsvySKCBEMIE+AFFgMffyeYtW5Pd+z2izHvZ8lH9XorQIqXuXmMpQX5efHRjGqfPlnDXV/V7KUK9cyfl8TIaeHdMCkF+XtzxVQYnClQPKXtjy59gOLCv3M/Z1m2VjpFSlgB5QAia6J8GDgJ7gdellBd8bxZCjBdCZAghMnJyPGOCS/vuQ1gZdy9pp/5gxdTn9DZHN8wW6RJiUkqrpoG8NiyZ1XtP8PwPmXqboxuukDspT5NAXz64IY1DeYX8n0rO2h1H+1qdATPQAogBHhRCxFYcJKX8WEqZLqVMDwsLc7BJzqPLDc+xOqA7nba/zca/5ultji6YLdJlxKSUqzo0547usXy1fC8zMvZV/wEPxBWSsRVJjQrm2cFJLNmWw5u/btPbHI/CFqHfD0SW+znCuq3SMdYwTRCQC4wBfpJSFkspjwB/A+l1NdpdEAYDrcZ/yT5jBOG/TeDQ3u16m+R0LC6SjK3Iw/1bc2l8CP+eu5H12Sf0NsfpnEvGuta9GdM5ipHpkby7KIufMw/pbY7HYIvQrwQShBAxQghvYBRQ0T2dB4y1vh8GLJRa68C9QG8AIUQD4BKgXq3mHNAwGOPorzHJYk5OHkXhmdN6m+RUXC10U4rJaGDi6FTCAny4c8oqck+d1dskp6JXC4TqEELw3JAkkiOCeHDGOrKOqPYV9qBaobfG3O8BfgY2AzOklJlCiOeFEIOtwz4DQoQQWcADQGkJ5ntAgBAiE+2B8YWUcr29fwlXJzIhmR2XvUGCOYv1H9WvHvZaMta1xKSUxg28+fCGNI6eLuLeb9bUq5mzzu5eWRN8vYx8cEMaPiYDd0zJUDNn7YBNMXop5XwpZSspZZyU8iXrtqellPOs7wullMOllPFSys5Syp3W7aes25OklIlSytcc96u4Nh2vGMPyiFvpfGI+K799R29znIarevSltI8I4qVr2rF0Ry5v/lZ/4sKuUkdfFS0a+TFxTAq7cwt4fM4GtbZAHXHR2+yZdBr3Xzb4pNBh/Yvs3Fg/2hqbLa4XHqjI8PRIRnWK5L1FO/hja/1YyNoVk7EV6RYXyoP9WvHD+oN8pWY01wkl9E7EaDLR/OYpnBIN8Jo9jlP5x/U2yeG4Uh39xXh2cBJtmgXywIx19WJtU7173djKnd3j6Nk6jBe+38TG/WqB8driBn+CnkVos0gO9/uAFpaDbPnU8+P1rh66KcXXy8h716dyttjM/9WDeL1FSoTQp3tlTTAYBG+M6EhIgDcTpq4mX8Xra4USeh1I6nYl/8TcSXr+b6yc85be5jgUswsnYysSFxbAy9e2Z+Xu47z+i2fH693lAQxa0nzi6BSyj5/hsdnrVby+Fiih14kuN77Eet80kje8zI4Ny/U2x2FY3EhQAIZ0DGd0Z63J1qItnhuvd6cHMEB6dGMe7t+a+RsOMWX5Hr3NcTuU0OuEwWgk/OYvyROBeM+52WPj9WaLa06YuhjPXJ1I2+YNuX/GWg6c8Mx4vbs9gAHGXx5L7zZNePGHzWzIVvH6mqCEXkdCmkZwtP/7Wrz+E89cmcoipdOXEawrvl5G3huTQnGJhXu/WUOxB8br3aEaqiIGg+B/w5MJDfDm7qmryDuj4vW2ooReZxK7DuSf2LtJP/m7R9bXu6NHDxAbFsAr13Vg1Z7jHtl3RXsA621FzQlu4M3EMakcPFHIE9+q+npbUULvAnS54QU2+nSk3fqX2bd9nd7m2BWLdL1+KrZydXILRnWK5IPFO1i+M1dvc+yKuz6AAdJaBvNAv1b8uP4gs1erxUpsQQm9C2AwGmly0ySKhBeF02+luMhz+q5YpMTonnoCwFODEokOacAD09eSV+A5oQJXWfmrttzRPY4uMY155ruN7MmtX/2jaoMSehehSXgMO7u+QkLJdjImPay3OXbDnT1HgAY+Jt4a2ZEjJ8/yxFzPCRVYLO6XOymP0SB4c2RHjAbBfdPWemQexZ4ooXchUvvfyD/Bg+iy/0syl87X2xy7YHZzQQFIjmzE/Vd4VqjA3R/AoPXDefna9qzdd4KJC7P0NselUULvYrS79X32G5oT8su95B1z/9W2XLUffU25s4dnhQrMblgNVRmDOrRgWFoE7y7cToZa9L1KlNC7GP4BQRQO/ogQeZysz92/RYIneI7geaECi4fcF9D6FEUE+3PftLWqRUIVKKF3QRJSupMRexdpp/4gY977eptTJ8zS9fup2Mp5oYLf3Xu1MLN0vzr6qgjwMfHWqI4cyi/k6bkb9TbHJVFC76J0vv45Mr3bk7jmBfbvdN9FrLUZmHpbYT8GdWjBdakRvLsoi5VuHCrQkrF6W2E/UqOCua9PAnPXHmDuGs/Io9gTJfQuitFkIuTGSZiFgfxvbsdcUqK3SbXCU0I35XluSBLhwX48OGMdp8+q++Iq3N0zjvSWwTz13UYO5RXqbY5LoYTehWkWGc/Wjv+mbXEmK6e/pLc5tcIdWyBUR4CPideHJbPveAGvLHDPJZA9JRlbHpPRwOvDkykxSx5VXS7PQwm9i5M++C7W+HcjZdtE9mxdq7c5NcYTPUeALrEh3NwthinL9/B31lG9zakxnpSMLU90aAMeG9iGxdtymL5yn97muAxK6F0cYTAQedNHnBE+nJ15OyXFRXqbVCPcrR1uTXhkQGtiQxvwyKz1breAtbvPjL0YN17Skq6xIbz442ayjxfobY5LoITeDQhtFkVWp2dpVbKNlVOf1ducGuGO7XBtxdfLyOsjkjmYd4YXf9istzk1whMmslWFwSD477AOSCl5ZNZ6LBYVwlFC7yakDbyV1QHdSdv5Ibsy3WdhcU/2HEGr9hjfPY7pGfvcaqEST5nIVhWRjf158qpElu7I5asVaqESJfRugjAYiL7pQ06JBpR8O8FtQjgWCx7rOZZy/xUJtGoawBPfbnCbEI47LSVYW0Z3jqR7qzBeWbCFfcfqdwhHCb0b0bhJOLs6PaM1Ppv+st7m2ISWjNXbCsfiYzLy6nUdOJRfyKs/uUcVjsUCBg+/L0IIXh7aDqDe96738FvteaQOvIW1/l1J3v6eW0yk8vTQTSkpUcHc3C2Gr5bv5Z9drj+Rqr7cl4hgfx7p35o/tx9ljoc0pKsNSujdDGEw0HzMe5Rg5Pj0u12+F467t8OtCQ/1b0VEsB+PzV5PYbFZb3MuiicnYytyY9do0loG88KPmzh6ynPWeqgJSujdkKYRcWxq9xDtzq51+eUH64vnCODvbeLloe3ZefQ0Exe6di8cT0/GlsdoELx6XXsKzpp5dp7rfwt2BEro3ZRO195Ppnd72mx4lZwDu/U2p0rqk+cI0L1VGMPSIvhw8U4yD+TpbU6V1IdkbHnimwRyb+94flh/kF83HdbbHKejhN5NMRiNNBzxAd6ymH1f36u3OVXiqTMwL8a/r2pLsL83j8/ZgNlFa7jNFukxXUVt5Y4ecbRpFshTczdyyk17FNUWJfRuTGR8e9bE3Ebq6SWsWzRTb3MqpT6Fbkpp5O/NU4Pasj47j69dtIZbC93obYVz8TYZePna9hw+Wcibv27T2xynUs9uteeROupp9hrCCV3yJIUFp/Q25wLqQx19ZQxObsGl8SG89tNWjpx0vU6KntqDqDpSo4IZ1SmKSUt3s+lAvt7mOA0l9G6Oj68/+b1fIVweZs3Up/Q25wLM9dBzBK2G+4Uh7ThbYuGlH12vPYKU9fMBDPDogNY08vPi33M31Jv2CDb9CQohBgghtgohsoQQj1Wy30cIMd26f4UQIrrcvg5CiGVCiEwhxAYhhK8d7VcA7S4bTEbDK0jbN9nlOlzWt6RfeWLDArizZxzfrT3gch0u62NIrZRG/t48cWVbVu89wfSM+tHhslqhF0IYgfeAgUAiMFoIkVhh2K3AcSllPPAm8Kr1sybgK+BOKWUS0BNwjznibkb0mDcpFL6cmv1/LlNbX+oteWr3Slu4u2ccLUP8eWruRs6WuE5tfX1+AANcmxpOl5jGvLJgS72orbfFo+8MZEkpd0opi4BpwJAKY4YAk63vZwF9hJbS7wesl1KuA5BS5kopXed/uwcR2iySzYn/IqloHat++FhvcwDNawTqtaD4ehl5YUg7dh49zUeLd+ptThkWi+e2j7YFIQQvXtOO02dL+M9892hbURdsEfpwoPz3m2zrtkrHSClLgDwgBGgFSCHEz0KI1UKIRyo7gRBivBAiQwiRkZOTU9PfQWEl/doH2GZqRcvVr3Aq/7je5pSVFtZnQQGttv6qDs15b1EW+0+c0dscwBq6qccPYICEpoHc3j2W2auzWbVH/78XR+LoNJkJuAy43vrvUCFEn4qDpJQfSynTpZTpYWFhDjbJczGaTMgBrxLGcTZMe1pvc7CUevT1XOgBnryyLULAy/NdIzFrtqgHMMA9veJp2tCH57/P9OjErC1Cvx+ILPdzhHVbpWOscfkgIBfN+18ipTwqpSwA5gOpdTVaUTWt03uzMqg/afunkp21UVdbSj36+u45ArRo5MddPeL5cf1Blu/M1ducellHXxkNfEw8NrAN67LzmLU6W29zHIYtt3olkCCEiBFCeAOjgHkVxswDxlrfDwMWSq0n6M9AeyGEv/UB0APYZB/TFVURPfK/lGAkZ87DutpRmhNWnqPG+O6xhDfy47nvN+k+Y7a+J2PLc03HcFKjGvHfn7a6zXoCNaVaobfG3O9BE+3NwAwpZaYQ4nkhxGDrsM+AECFEFvAA8Jj1s8eBN9AeFmuB1VLKH+3+WyjOI6xFNOtibyelYCkblnyrmx3nkrG6meBS+HkbeeLKtmw+mM83/+zV1Zb6nowtjxCCZ65O4uips7y7MEtvcxyCTV/epJTzpZStpJRxUsqXrNuellLOs74vlFIOl1LGSyk7Syl3lvvsV1LKJCllOyllpclYhf1JHfkk2aIZgX88RXGRPuVjZaEbJShlXNm+GV1iGvO/X7aSV6Cf96iSseeTHNmI4WkRfP73LnYdPa23OXZHRek8FB9ff3K6PkW0ZR+rZr+uiw2lyVjlOZ6j1HvMO1PMm7/p12+lvrZAuBgPD2iNj8nIiz94XnRZCb0H07HvGDb4pNJ263vkHXN+2apKxlZOYouGjOocxVfL97AzR5/+RBapQjcVaRLoy7294/l9yxGXm8lcV5TQezDCYMD/qpcIlAVsmvms08+v6uir5v6+rfA2GXjt5626nF8lYytnbLdowhv58Z8Fmz2q3FIJvYcT16Ebqxr1I/XAdA7tde6qRxY1M7ZKwgJ9GN89lgUbD7F6r3Mn60gpsUj1AK4MXy8jD/Vvxcb9+Xy//oDe5tgNJfT1gMjrXgJg3+x/O/W8Khl7cW6/PJbQAB/+M38zUjrPeyx1VNUDuHKGJIeT2Lwhr/281aX6E9UFJfT1gGZRCaxpPoK0Ez+zY8Nyp5231KNXelI5DXxM3H9FAit3H3fq8nbnHsBOO6VbYTAInriyLdnHzzBlmWsuHFNT1K2uJ7Qd8RwnhT+nf3zSaec0WydMKY++akamRxIb1oBXf9pCidk5XUdVNVT1XJYQyuUJoUxcmKVrGay9UEJfTwhqHMbmhDvoUJjhtElUquqmekxGA48OaMOOnNPMyHDOFHx1X2zjsYFtyC8s5v3F7j+JSgl9PSLluoc5IJrgt/gFp/SsV56jbfRLbEp6y2De/G0bZ4ocHxM2q2ZzNpHUIoihKeF88fduDua5RtfR2qKEvh7h4+vP/uT7iDfvYO1vUx1+PuU52oYQgkcGtCHn5FmnLCZetiCMui/Vcn/fVlgskvcX7dDblDqhhL6ekXLVePaJFgQtfw2L2bHeo2pTbDudYxpzWXwoH/yxg4KiEoeeS1VD2U5kY39GdIpk2sq9LrOWQG1QQl/PMHl5czjlPmItu1n7y5cOPZcK3dSM+69oRe7pIr50cKWHWd2XGjGhVzwC4dYNz5TQ10NSrryNPYYIGq/8H+YSx3mPZVU3KkRgE2ktg+nZOoyPFu/g1FnH3ReLui81IryRH6M6RzIzYx/7jhXobU6tUEJfDzGaTBxNu59oyz7W/PyFw85zrgWCw07hcdzftxXHC4qZvHS3w85xLhnrsFN4HHf3jMdgEExc6NzZ5fZC3ep6SsqAm9ltiKLJqjcd5tVbpMSHItLmXA5ThkL2Koecx5NIjmxE37ZN+HjJTvIdtAiGSsbWnGZBvlzfJYrZq/ez2w3bGCuhr6cYjEaOdX6QKMt+1sz/xCHnMFskDTmNz+kDsGMhfNobvhkNhzY45Hyewr/6tiLvTDGf/7XLIcdXydjacVfPOLyMgnfc0KtXQl+P6XjFjewwxtB0zTsO8erNUuIjrMcd8Cr0+jfs/hs+vAxmjoMcfTo3ujrtwoPon9SUz/7cRd4Z+3v1qo6+djQJ9OXGS1oyd81+t1ucRAl9PcZgNJLX6V9EygOs/WWK3Y9vsUi8sQqVfwj0eBj+tQ66Pwzbf4X3L4E5d8CxnRc/UD3k3t4JnDxb4pC6ehW6qT3ju8dhMhr4eIl71dUroa/nJPe9gX2iBQ1XvWv32bJmi8Qbq0dv8tb+9QuG3v+G+9ZB1wmwaS5MTId5/wcn9tn1/O5Mu/AgurcK4/O/dlNYbN/5Dsqjrz1hgT6MSI9g9qr9HMkv1Nscm1FCX88xmkwcbDeeBHMWG//63q7HtshyHr3J9/ydDUKh34ua4He6DdZ9AxNTYcGjcNqzVvepLXf2iOXoqbPMXm3fHjhm5dHXifGXx1FisfDZ347JoTgCJfQKkq+6gxyCEX+/Ydfjmi2cE3qjd+WDApvBlf+Fe1dD8ij452N4uyMsfg2K3CsOam+6xoaQHNmIj5fsLBNne2BRXUXrRFSIP1d1aMHXy/c6JIfiCJTQK/Dx9WdH3E20O7uW7WuW2O24WjK21KP3ufjgRpEweCLcvRxie8CiF+GdFMj4HMyObQngqgghuKtHLHtyC1iw8aDdjqvq6OvOHd1jOeWgHIojULdaAUDS4H+Rjz8nf3/Nbse0lI/RG6sR+lLCWsOor+GWXyA4Bn64X0vabv4enLgKk6vQL7EZsWEN+OCPHXZbhUqFbuqOI3MojkAJvQKAwKDGZLYYTseTf7Jv+zq7HNNcvurGVEXopiqiusAtP8Gob0AYYPoN8Fk/2LPMLra5CwaD4I7usWQeyOfP7fbJXahmc/ahNIcya5Vz1hGoC0roFWUkDH6YYkwcXGAfr94sJT419ejLIwS0uRLuWqqFdfL2wRcDYMZNcHy3XWx0B65JCadpQx8+XGyfkj6Lah9tF8rnUJy1OlhtUUKvKCO0WSRrQ6+iY+4Ccg/X3UuxWCTeopYefXmMJki9SUvY9npSq8F/txP8+gwU5tfZTlfHx2Tk1stiWLojl4378+p8PNW90j6U5lD2HivgFyeu+VsblNArzqNZv/vxFiVsX/B+nY9llrWI0V8Mb3/o8QjcuwraXQd/v6WVZK6aBBbXj5PWhZGdovDzMvLlst11PlZp1Y2K0dedKxKbERHs59AmdPZACb3iPFq27sgGn1Ridk+jpLioTsc6b2ZsdVU3NaFhCxj6Idy+EBrHwff3wUfdYedi+53DxQjy8+La1HC+W3uA46frdl9U1Y39MBoEN17SkhW7jrHlkOt+u1S3WnEB5vTbaUouGxZ+U7fjOEroSwlP0xK2wydpIZwvB8M3Yzw2fn9T12jOlliYkVG3GcSqBYJ9GZEeiY/J4PAFY+qCEnrFBbTvNYKDhOG96tM6HccssW/opjKEgKShcM9K6PM07PwD3usCf7wCxe679FtltG4WyCWxjZmyfE+dJlCp7pX2JbiBN0M6tuDb1ftddgKVEnrFBRhNJvbEjiapaD27Nq2s9XEsFm3ClBQGLaHqSLx84fIHNcFvfSX88R+t/n7rT449r5MZ2zWa7ONnWLjlSK2PUZaMVR693bipazRnis0uW2qphF5RKW2unECh9OLI7+/W+hjm0l43jvLmKyMoHIZ/ATd9p7Vd+GYkTB0Fx9ynL8nFuCKxKc2DfOuUlLUoj97utAsPIq1lMFOW7S67vq6ETUIvhBgghNgqhMgSQjxWyX4fIcR06/4VQojoCvujhBCnhBAP2cluhYNpFNqM9cFX0P7oAvKO126iTln3yrqUVtaW2J5w599wxfOwa4nHhHNMRgPXd4niz+1HyTpyqlbHUN0rHcNNXVuyO7eAJdtz9DblAqoVeiGEEXgPGAgkAqOFEIkVht0KHJdSxgNvAq9W2P8GsKDu5iqcSeNeE/AXZ9m84MNafd5ikfg426Mvj8kbLr1PC+e0uepcOCfrd33ssROjOkfhbTTw1fLaJf9UCwTHMLBdc8ICfVyy1NIWj74zkCWl3CmlLAKmAUMqjBkCTLa+nwX0EUL7XySEuAbYBWTaxWKF04hPvowtpraEb/8ai7nmdepmKfEWOnn05SkfzjGY4KtrtQVPTufqa1ctCQ3w4aoOzZm1KptTZ2ve8E21QHAM3iYDYzpH8ce2HJdbV9YWoQ8HytdzZVu3VTpGSlkC5AEhQogA4FHguYudQAgxXgiRIYTIyMlxva899ZlTHcYSKQ+wadmPNf5saR290Mujr0hpOKf7w7BxFrzXCdZNd8tmaTdc0pJTZ0v4ft2BGn+2dLa+aoFgf8Z0icIgBNNWutYiOo5Oxj4LvCmlvGgwUUr5sZQyXUqZHhYW5mCTFDWhXd8bycefwn8mVz+4AmVrxjqihr62ePlqK1zd8Sc0joVvx8NX17ld7X1qVCMSmgQwvRaCUlZHr0ox7E7Thr70ah3G7NXZLtX/xpZbvR+ILPdzhHVbpWOEECYgCMgFugD/FULsBv4FPCGEuKduJiucia9/AJtDB9A+bzF5x2r2bctsQWtq5kpCX0rTRLjlZxj4GuxbAe93haXvuk3veyEEIztFsnbfCbYdPlmjz6pkrGMZkR5Jzsmz/LHVdaITtgj9SiBBCBEjhPAGRgHzKoyZB4y1vh8GLJQal0spo6WU0cBbwMtSytrX6yl0IeTyW/ERxWz59fMafc4iJT4U6ZeMrQ6DEbqMhwkrIKY7/PIkfNoHDrtHOmloSjheRsGMGnr1ZtW90qH0atOE0AAfptdxBrM9qVborTH3e4Cfgc3ADCllphDieSHEYOuwz9Bi8lnAA8AFJZgK9yU++TJ2GGMJ2Ta9Rp8zW1wkGVsdQREwehoM+wLy98NHPWDJ6y7v3YcE+NC3bVPmrNlPUYntYQKL6l7pULyMBq5LC2fhliMcOekaC4jbFKWTUs6XUraSUsZJKV+ybntaSjnP+r5QSjlcShkvpewspdxZyTGelVK+bl/zFc4iJ3448eYd7Ny4wubPmC3WfvSu6tGXRwhody3cvUIrxVz4AnzeD3K26W3ZRRmRHsmx00U1mimrPHrHMyI9ErNF8t2amifLHYFKxyhsonXfmymWRo78ZXtS1iKt/ehdMUZfFQ1CYMRkGPY5HNsJH12uxe5dtA3y5QmhhAZ4M3dNxbRZ1ZTV0SuP3mHEhQWQHNmIb2twXxyJEnqFTQSHNSezQWdiDy3AXGJbSKNsZqzRxUM3ldHuOs27j+2lxe4nXeWSbRRMRgNXJ7dg4ZYjNjfUUnX0zmFoxxZsOpjP1kM1S5Y7AiX0CpsxtxtOE46xefl8m8ZbSnvduJNHX57ApjD6G7jmAy1B++HlWt29izE0JZwis4UFGw7aNF7V0TuHQcktMBoEc9fq79UroVfYTFLPkZySfhRk2NanvqwfvTt69KUIAR3HwF1/Q7N2Wt397NugsO5L+tmL9uFBxIY1sDlMcC4Z60irFKEBPnRPCOW7Nft1b3SmbrXCZnz9A9gU3IvE44soLKi+oZbZYu1H764efXkaRcHYH7Q1azfOgQ8vg722J6YdiRCCoR3DWbHrGNnHC6odr5KxzuOalHAO5BWyYtcxXe1QQq+oEX6powgQZ9j85+xqx2qhmyLPEHrQeur3eERb1Qrgi4Hwx6suUYY5pKPWlWS+DeEbtfCI87gisSl+XkZ+WK9v9Y0SekWNaNt1IMdpiHnjd9WONVskXu5SXlkTIjvDnX9pCds/XoYvh8DJQ7qaFBXiT7vwhvy4oXo7LFIihPZNQOFY/L1N9G7ThJ8zD9VpVbC6ooReUSNMXt5sb9yDtvl/Vxu+kZYSjFg8x6Mvj28QXPcJXPMh7F+lJWp3/amrSVe2b866fSeqDd+YLVKFbZzIle2bc/RUEf/oGL5RQq+oMX7J19JAFLL5r7kXHWcwn9XeuHMytjo6jobbF2rC/+Vg+OtNsOjTzOqq9s0B+Gnjxb16s5Sqht6J9GoThq+XwaawmqNQQq+oMW26XsUJAjBv/Pai44S5SHvjiR59eZomwvhFkDgEfnsWpo2GM8edbkbLkAYktWhYraBYlEfvVPy9TfRq3YSfdAzfKKFX1Bgvbx+2NepOm7y/OVtYdZjAaLEKvSd79KX4BGq9cga+pq1g9VF3OLDG6WZc2b45q/ee4MCJqpdMNFtUItbZDGzfnJyTZ8nYrU/4Rgm9olb4JF9LgDjDlqU/VDlGWOqJR1+KEFo3zFt+0sI3nw+ADbOcasLAds0A+CWz6vCNRUqUzjuXPm2a4G0y8NNF7osjUUKvqBWtL7mSAulDYWbVK08ZLdYp+Z5WdVMdEekw/g9okQqzb4XfX3Ba3D42LIDYsAb8fpEmZ2aLVB69k2ngY6JbXAgLtxxB6rCimRJ6Ra3w9WvA1oBOtMz9C1mFiJWFbly9TbEjCAjT1qhNvQn+fB1m3Ahnq59kZg/6tm3Kip3HqlxP1iyV0OtBn7ZN2ZNbwI4c568nq4ReUWtK4vrRjKPszPyn0v1Gi7XqxuTrRKtcCJM3XP0ODHgVts6Hz/rB8T0OP23vNk0oMlv4a3vlKxxZLBKDSsY6nd5tmgCwcMthp59bCb2i1sR0GwrAkYzKq28MZaGbeujRlyIEXHInXD8L8rPhk16wZ6lDT5nWMpiGviZ+21x5+EaFbvQhvJEfbZoFVnlfHIkSekWtCW0WxTZTKxrvX1TpfmN9S8ZejPg+cNtC8GuszaTdWH0LidriZTTQs3UTFm05UmkzLbNUHr1e9G3blFV7jnOioMip51VCr6gTueG9SSjextFDF66Pea68Ugk9AKHxcNuvEJ4Os27RFjRxEH3aNiH3dBFrs09csE9KVV6pF73bNsFskSze5tyFw5XQK+pEWOrVGIRk14rvL9hnkNbQTX1MxlaFXzDc+K02ueqXJ+Gnxx1SkdOjVRhCwOKtFwqKCt3oR3JEIxo38K70vjgSJfSKOhHbrisnCICdiy/YZ1IefeV4+cKwSdDlLlj+Psy6GUrs+1W+kb837cODWLrj6AX7zKqOXjeMBkHXuBD+3nHUqWWWSugVdcJgNLIzIJWovJUXlFmW1dErj/5CDAYY+Ar0exE2zYVpY6C46tmstaFbXChr9p7gdIUyS4vy6HXl0rhQDuefdWqZpRJ6RZ0pjupOU3LZl7X+vO0mqTz6aul2r1aCmfUbfDUMztpvfdFL40MosUj+qTDt3qzKK3XlsvhQAP7OuvDblqNQQq+oM+GpAwE4uObn87abVNWNbaSNhes+hb3LtIqcAvv0Q+kU3Rhvk4G/t58vKBY1YUpXokL8iQj2U0KvcC/CYxM5RBjee5ect91YloxVQl8t7YfByClwaANMGgSnc+t8SF8vI2lRwfy94/xjqWSs/lwWH8qynbmUmJ3TGkMJvaLOCIOBvY06EXd6DeaSc/FgFbqpIW2ugjHT4dgOmGIfz/7S+BA2H8wn99TZsm1miRa6+fFBmHETrPwMjmZpdZcKp9AtPpSThSVsPJDvlPMpoVfYBUNcTxpymp0bzs36NMliLBi0tVYVthHXG0Z+DTlb4atroTCvToe71BoPXrbznFevtUAAVk+BLfPhxwfg3TR4IxHm3AFrp0Jedp3Oq7g43eJCAOfF6ZXQK+xCVGo/AHI3nyuz9JLFlAgvvUxyXxL6wogpcGgjfHVdnRK07cOD8Pc2krH73EIoZovERxSD+Sz0fBTuXQ2D3oKoLpD1K8y9C95MgndS4Pv7tFm8p5xb9+3phAb4kNAkwGn96ZWrpbALTcJjOEQoXgcyyraZrEKviitrQesBMPwLmDEWvh4ON8wBb/8aH8ZkNNAhIojVe8sJvZQ0lNZSTp8gCInTXuk3a5O3cjZr8yJ2LYGNc2DVJG1skySI7QFxfaBlt1rZozhHalQwP286hJTS4Qu1K49eYTf2B7Qj/FRm2c8mijEblMzXmrZXW6txlmt97c2Vtx2ujtSoYDYdyKew2AxooZsAYRV634bnDzYYoGkSdL0bxkyDR3ZpPXr6PA0NQrV4/tfXwavR8OU1sHQiHN6k4vu1ILVlI04UFLPzqOPr6ZVHr7AbxS3SabbtD3IO7CasRbQK3diDdtfC6aOw4GGY/xAMelPriFkDUqOCKbFI1mfn0TmmMWZZTuh9Ai/+YaMJItK01+UPQlEB7F0KWQthx+/wy7+Bf0NgCy2/EN8bYnuBf+Pa/b71iNSoYABW7zlOXFiAQ8+lhF5hNxq16gbbYN/6PwhrMQ4vWaQ8envQZTycPAB/vQlB4dD94Rp9vGNUIwDW7D1O55jGWCwSf4N1rd/qhL4i3v4Q31d7gZa03bFQWyd3yw+w9itAQHiqFuKJ7wMRncBgrNl56gFxYQEE+ppYs+8Ew9MjHXouJfQKuxHTritF35so2v0PMA4vijErj94+9HkG8g/Cwhc17znleps/GhrgQ8sQ/7I4vVlKAqil0FckKEJbRSv1JrCYYf9qzdPP+l1bWWvJf7XWzAlXQKv+mvj7NarbOT0Eg0HQMbIRq/ccr35wHbFJ6IUQA4C3ASPwqZTylQr7fYAvgTQgFxgppdwthLgCeAXwBoqAh6WUC+1ov8KF8PH1Z4tXAkG5awBr1Y1B1dDbBSFg8EQ4dQjm3QuNIiGmu80fT40K5q8srZGW2UI5j77hxT9YEwxGiOykvXo+BmeOw45FsO1n2P4LrJ8OBhNEdYVWA7RXaLz9zu+GpEYFM3Hhdk6dLSHAx3F+d7XJWCGEEXgPGAgkAqOFEIkVht0KHJdSxgNvAq9atx8FrpZStgfGAlPsZbjCNTkR0pHYou0UnS3Ei2IsBuXR2w2Tt1Z2GRIPM8fBiQvXAKiK1KhG5Jw8S/bxM1roRjpA6CviF6zlGK79CB7Oglt+gW7/BwW5Wovmd9PgnVT46QmtwqeWyWZ3JrVlMBYJ6/adcOh5bKm66QxkSSl3SimLgGnAkApjhgCTre9nAX2EEEJKuUZKecC6PRPws3r/Cg/FO7oLPqKY3ZnL8ZYlKkZvb3wbwqivwVwM02+wueNlSmnib+9xzLK80NcxdGMrBqNWp9/3Gbh7Gdy3Hq58HRrHwMpPYPLV8Ho8zJ0AW3+C4kLn2KUzHSMbATg8fGOL0IcD5V2HbOu2SsdIKUuAPCCkwpjrgNVSyrMVtiOEGC+EyBBCZOTkqIkZ7kzzxG4AHM/6R4vRK4/e/oQmwNCP4OBa+OEBm0ob2zQLxNtkYOP+PM2jtxSAwUu/PkTBLaHz7XDDbK2Ec8QUSOgHm7+Hb0bCa3Ew82bI/BbOntLHRicQ5OdFbGgDNuyv2wzo6nBKMlYIkYQWzulX2X4p5cfAxwDp6emqINeNaRaZwCnpB0c2W0M3yqN3CG2uhB6PweJXtAqXzrdfdLjJaCA+LIBth09hlhI/WaB5867QrtgnABIHa6+SIi2Ms3kebPkRMudovZLi+0DbwdpEMr9gvS22K62bBbLlkP3aU1eGLUK/Hyhf+xNh3VbZmGwhhAkIQkvKIoSIAL4FbpJS7qizxQqXRhgM7PdqSYP8LLwpoUQJvePo8ajm1f/0GDRPhsjOFx3eqmkA/+w6hsEg8LOcvnCylCtg8tZaQCT01eYM7F2mefmbv4et87VkbmxPaD9cawLnrNCTA0loGsjPmYcoLDbj6+WYMlRbQjcrgQQhRIwQwhsYBcyrMGYeWrIVYBiwUEophRCNgB+Bx6SUf9vJZoWLkxcYT3jRLnyE8ugdisEA134MDVtoM2eraYDWqlkgB/IKyTtTjG+pR+/KGIwQfRkMfBXuz4TbF0LXCZCzDb69A16L17pvbprn1jH91k0DsUjIOuK4EFW1Qm+Nud8D/AxsBmZIKTOFEM8LIQZbh30GhAghsoAHgMes2+8B4oGnhRBrra8mdv8tFC6FJawtwZwkjBNYjEroHYpvEFz3GeTthx8fuujQVk00YT9ZWIKf+bRjK27sjRAQngZXPA//Wq9V8KTeBHuWwowb4fUE+PYubaUuN6veadVUmxW7/Yjjwjc2xeillPOB+RW2PV3ufSEwvJLPvQi8WEcbKS4uJjs7m8JC931q1yf82g5kc0wnAEqMDdi8efNFx/v6+hIREYGXl0rc1orIzlrd+qKXtElJ7YdVOqx1s3MevK+lAHyaOctC+yKEVsET1QX6/wd2LdY6bG7+HtZNhYCm0GEkpNwAYa31trZaokMb4GUUbD3kOI/eLWbGZmdnExgYSHR0tMO7vCnqTnHxWbxyNgFQ4B2Cf2hUlWOllOTm5pKdnU1MTIyzTPQ8Ln9Qm5g0/2GI6QEBYRcMCW/kh5+XkTPFZnwtbubRV4XRpCVq4/vAVW9oE7PWTYPl78PSdyA8XZtFnHSty87I9TIaiA0NYPthx3n0btG9srCwkJCQECXyboLJ5E1J2X+ti98zIQQhISHq21pdMRhhyHtQdEprflbZEIMoCxP4WE67foy+pnj5apU7o6fCA5uh30tQXAA/3A//aw2zbtX68ljMelt6Aa2aBbK1vgs9oETejRBCUGydFydtuG/q3tqJJm20EM6mubD5h0qHJDTVxN3X7AbJ2LoQ0AS63QN3LYXbF2lhnKxfYcpQeLsj/Pk/l1pMpVWTALKPn+H0WcfkF9xG6BXuhblsnVgl4k6l233aAiE/Paa1FK5A66aBeFOsrefryUJfirB20rzqf/DgNhj2BTSOht+fhzfaal7+nmW699NvZc2fbHdQ5Y0S+lrw7LPP8vrrr190TE5ODl26dCElJYU///yzxueYNGkS99xzDwBz585l06ZN1X5my5YtdO3aFR8fn2rtczgmX+1fof6LORWjCa58DfL2wV9vXLC7VbNAAihddCTIycbpjJev1ntn7PcwYSV0ug22/wpfDIAPLtUWVank4egMWlm/aW1z0MQp9VfoIH7//Xfat2/PmjVruPzyy+t0LFuFvnHjxrzzzjs89NDFy+ycgbFsmTnl0Tud6Euh/Qj4+23IPX+OYqumAbYvOuLJhLWCga/Ag5u1rqBGk7ZI+ptJsPAlOHXEqeZENfbHx2Rgm4Pi9G5RdVOe577PZNOBfLseM7FFQ565OumiY1566SUmT55MkyZNiIyMJC0tDYAdO3YwYcIEcnJy8Pf355NPPqGwsJBHHnmEM2fOkJGRwbJly3jggQdYuXIlZ86cYdiwYTz33HMAREdHk5GRQWhoKBkZGTz00EP88ccfZeddunQp8+bNY/Hixbz44ovMnj2buLi4Sm1s0qQJTZo04ccff7TPhakD3n4NKMz3xejTQG9T6if9XtBaCCx8AYZPKtvcrKEvPVr6wiHqt9CX4t1Aq8dPuVFbsnHpRFjymvaQTB4JXe9xSomm0SBo0yyQ/MJihxzf7YReD1atWsW0adNYu3YtJSUlpKamlgn9+PHj+fDDD0lISGDFihXcfffdLFy4kOeff56MjAzeffddQHtQNG7cGLPZTJ8+fVi/fj0dOnSo9tzdunVj8ODBDBo0iGHDKq+PdkWMRhPGFm31NqP+EthMW/d1yWtw2f1aiwS0xPcL/SO1XrNK6M8hBLTsqr2OZsHy92DtVFj9JST0hx6PQES6Q0349u5LMRgc8w3Y7YS+Os/bEfz5558MHToUf38tHDF4sDYh+NSpUyxdupThw8/NFTt79oLmnADMmDGDjz/+mJKSEg4ePMimTZtsEnqFotZ0vQf++QR+fwFumHVu+1lreMAT6ugdQWi81men15Na3H7Fh/BpH4i/Ano+rq2f6wAcJfLghkLvSlgsFho1asTatWsvOm7Xrl28/vrrrFy5kuDgYMaNG1dWN24ymbBYLACqllxhX/waad78b89orQJaai2kzwm98ugvSoNQ6Pmo9s3on4+1sM6nvTUPv+ejWksGN0ElY22ge/fuzJ07lzNnznDy5Em+//57ABo2bEhMTAwzZ84EtFme69atu+Dz+fn5NGjQgKCgIA4fPsyCBQvK9kVHR7Nq1SoAZs+eXen5AwMDOXnyXJLm3XffLQsJKRQXpfN48A/VYs6lnLXmuJRHbxs+gdrM439tgN5PQfY/8ElvmDoSDldfJOEKKKG3gdTUVEaOHElycjIDBw6kU6dOZfu+/vprPvvsM5KTk0lKSuK777674PPJycmkpKTQpk0bxowZw6WXXlq275lnnuG+++4jPT0do7HyFqWjRo3itddeIyUlhR07drBlyxZCQiqu6wKHDh0iIiKCN954gxdffJGIiAjy8+2buFa4Gd7+Whnhtp/g6HZtW5nQK4++RvgEQveHtNWxev9ba6H84aXw/b+cXqVTU4TUeaJARdLT02VGRsZ52zZv3kzbtiqxV8qgQYOYM2cO3t6e0xlS3WMHcuoIvNlOmx066A347VlY+i48leMaC4+4KwXHYPGrsPJTMPnB5ffDJRO0en0dEEKsklJWmjFWHr0b8sMPP3iUyCscTEAT6DBCqyIpOKbF6H0bKpGvK/6NtV75d6+AmO7abNsPL4VdNZ8g6WiU0CsU9YGuE6DkDKz5ShN6FbaxH6HxWiO1G+aApQQmD9IWOS84prdlZSihVyjqA03aai17N85SQu8o4vvAXcu0Sqf10+DddG31KxdACb1CUV9ody0cXAcH16uKG0fh7Q99n4U7lkCjKG31qx/uh+IzupqlhF6hqC8kXqP9m5+thN7RNE3Sljvs9n+Q8Tl83EvXUkwl9ApFfSEoHKK6au9V6MbxmLy1nkM3zIGCXPikl5Yj0QEl9ApFfSLpWu1fJfTOI74P3PU3RHaB7ybAL/92+ipXSuidxO7du2nXrl2djzNu3DhmzZpV/UAnsGvXLrp06UJ8fDwjR46kqKjogjG7d+/Gz8+Pjh070rFjR+68804dLFWUkThEWyPAv7HeltQvAprADbO1yWtLJ8KsW6Dkwr8XR+F+vW4WPAaHNtj3mM3aa72pFTXi0Ucf5f7772fUqFHceeedfPbZZ9x1110XjIuLi6u2H5DCSQQ2hXE/QkiC3pbUP4xe2kpXwdGaV19cACO+BC8/h59aefQ14Msvv6RDhw4kJyczdOhQYmJiKC7W+kfn5+eX/ZyVlUXfvn1JTk4mNTWVHTvOX/zBbDbz8MMP06lTJzp06MBHH31U5TmllNxzzz20bt2avn37cuTIuanWq1atokePHqSlpdG/f38OHjwIUOn5T506RZ8+fUhNTaV9+/ZlrRqefvpp3nrrrbJjPvnkk7z99ttUh5SShQsXlrVOHjt2LHPnzrXpOip0pmU3CAjT24r6S7d7YdBb2upWM24Cs2PWiT0PKaVLvdLS0mRFNm3adME2Z7Nx40aZkJAgc3JypJRS5ubmynHjxslvv/1WSinlRx99JB944AEppZSdO3eWc+bMkVJKeebMGXn69Gm5a9cumZSUVDb2hRdekFJKWVhYKNPS0uTOnTsrPe/s2bNl3759ZUlJidy/f78MCgqSM2fOlEVFRbJr167yyJEjUkopp02bJm+++eYqz19cXCzz8vKklFLm5OTIuLg4abFY5K5du2RKSoqUUkqz2SxjY2Pl0aNHZX5+vkxOTq70lZmZWXaMUvbu3Vv2+5Vn165d0t/fX3bs2FF2795dLlmypNLf0xXusULhVFZ+LuUzDaWcO0FKi6XOhwMyZBW66n6hG51YuHAhw4cPJzQ0FNCW7bvtttv473//yzXXXMMXX3zBJ598wsmTJ9m/fz9Dhw4FwNf3wr4Xv/zyC+vXry+Ltefl5bF9+3ZiYmIuGLtkyRJGjx6N0WikRYsW9O7dG4CtW7eyceNGrrjiCkD7ltC8efMqz19cXMwTTzzBkiVLMBgM7N+/n8OHDxMdHU1ISAhr1qzh8OHDpKSklDVMu1i45ejRozZdt+bNm7N3715CQkJYtWoV11xzDZmZmTRsqMr7FPWc9Jshf7+2OEzjWLj8AYedSgl9Hbj00kvZvXs3f/zxB2azmXbt2p3XTrgqpJRMnDiR/v371/rcUkqSkpJYtmzZedurOv/XX39NTk4Oq1atwsvLi+jo6LL+97fddhuTJk3i0KFD3HLLLWXHqWqt26lTp9K2bVtOnDhBSUkJJpOJ7OxswsPDLxjr4+ODj48PAGlpacTFxbFt2zbS0x27Wo9C4Rb0elLrKrroJa06x7oSmL1RMXob6d27NzNnziQ3NxeAY8e0PhY33XQTY8aM4eabbwa03vERERFl8eqzZ89SUHD+yvL9+/fngw8+KIvvb9u2jdOnT1d63u7duzN9+nTMZjMHDx5k0aJFALRu3ZqcnJwyoS8uLiYzM7PK8+fl5dGkSRO8vLxYtGgRe/bsKTvH0KFD+emnn1i5cmXZwycwMJC1a9dW+kpMTEQIQa9evcq+lUyePJkhQ4ZcYH9OTg5ms1ZKtnPnTrZv305sbGwNrrxC4cEIoa1m5R8C394JJZWvUFdnqorp6PVy1Ri9lFJOmjRJJiUlyQ4dOsixY8dKKaU8ePCg9PX1lcePHy8bt23bNtmrVy/Zvn17mZqaKnfs2HFejN5sNsvHH39ctmvXTiYlJcmePXvKEydOVHpOi8UiJ0yYIFu1aiX79u0rBw4cKGfOnCmllHLNmjXy8ssvlx06dJCJiYny448/rvL8OTk58pJLLpHt2rWT48aNk23atJG7du0qO88dd9whH3300Rpdjx07dshOnTrJuLg4OWzYMFlYWCillPK7776TTz31lJRSylmzZsnExESZnJwsU1JS5Lx58yo9lqvcY4VCF7b+rMXrf3m61ofgIjF61Y++jsyaNYvvvvuOKVOm6G1KrbFYLKSmpjJz5kwSEvQpu3Ple6xQOIUFj0JoglZrXwsu1o9exejrwL333suCBQuYP3++3qbUmk2bNjFo0CCGDh2qm8grFAq03vYOQgl9HZg4caLdjrVhwwZuvPHG87b5+PiwYsUKu52jMhITE9m5c6dDz6FQKPTFbYReSonw4BVx2rdvX29nj7pa+FCh8DRsqroRQgwQQmwVQmQJIR6rZL+PEGK6df8KIUR0uX2PW7dvFULUqp7Q19eX3NxcJQgeiJSS3NzcSucbKBQK+1CtRy+EMALvAVcA2cBKIcQ8KWX55sq3AsellPFCiFHAq8BIIUQiMApIAloAvwkhWkkpa9S6LSIiguzsbHJycmryMYWb4OvrS0REhN5mKBQeiy2hm85AlpRyJ4AQYhowBCgv9EOAZ63vZwHvCi3OMgSYJqU8C+wSQmRZj3f+LJ9q8PLyqnTWqEKhUCiqx5bQTTiwr9zP2dZtlY6RUpYAeUCIjZ9FCDFeCJEhhMhQXrtCoVDYF5eYGSul/FhKmS6lTA8LU131FAqFwp7YIvT7gchyP0dYt1U6RghhAoKAXBs/q1AoFAoHUu3MWKtwbwP6oIn0SmCMlDKz3JgJQHsp5Z3WZOy1UsoRQogkYCpaXL4F8DuQcLFkrBAiB9hT1X4bCAVsa62oD65uHygb7YWy0T4oG22jpZSy0pBItclYKWWJEOIe4GfACHwupcwUQjyP1lthHvAZMMWabD2GVmmDddwMtMRtCTChuoqbqgy1FSFERlXTgF0BV7cPlI32QtloH5SNdcemCVNSyvnA/Arbni73vhAYXsVnXwJeqoONCoVCoagDLpGMVSgUCoXj8ESh/1hvA6rB1e0DZaO9UDbaB2VjHXG5NsUKhUKhsC+e6NErFAqFohxK6BUKhcLD8Rihr67Dph4IISKFEIuEEJuEEJlCiPus2xsLIX4VQmy3/huss51GIcQaIcQP1p9jrF1Is6xdSb31tM9qUyMhxCwhxBYhxGYhRFdXuo5CiPut93ijEOIbIYSvK1xHIcTnQogjQoiN5bZVet2ExjtWe9cLIVJ1su81631eL4T4VgjRqNy+OnfDtYeN5fY9KISQQohQ689Ov4a24BFCX67D5kAgERht7ZypNyXAg1LKROASYILVrseA36WUCWiTyPR+MN0HbC7386vAm1LKeOA4WndSvXkb+ElK2QZIRrPXJa6jECIc+D8gXUrZDm2+SWkXV72v4yRgQIVtVV23gUCC9TUe+EAn+34F2kkpO6BN1nwcoEI33AHA+9a/fT1sRAgRCfQD9pbbrMc1rJ6qFpN1pxfQFfi53M+PA4/rbVcldn6H1u55K9Dcuq05sFVHmyLQ/th7Az8AAm2Gn6mya6uTjUHALqzFA+W2u8R15FzzvsZoc1N+APq7ynUEooGN1V034CNgdGXjnGlfhX1Dga+t78/7u0abxNlVj2to3TYLzenYDYTqeQ2re3mER4+NXTL1RGiLsaQAK4CmUsqD1l2HgKZ62QW8BTwCWKw/hwAnpNaFFFzjWsYAOcAX1hDTp0KIBrjIdZRS7gdeR/PsDqJ1b12F613HUqq6bq74d3QLsMD63mXsE0IMAfZLKddV2OUyNpbHU4TepRFCBACzgX9JKfPL75PaY1+XGlchxCDgiJRylR7nrwEmIBX4QEqZApymQphG5+sYjLb2QgxaT6cGVPJV3xXR87pVhxDiSbTw59d621IeIYQ/8ATwdHVjXQVPEXqX7ZIphPBCE/mvpZRzrJsPCyGaW/c3B47oZN6lwGAhxG5gGlr45m2gkbWZHbjGtcwGsqWUpSulz0ITfle5jn2BXVLKHCllMTAH7dq62nUsparr5jJ/R0KIccAg4Hrrwwhcx744tIf6OuvfTgSwWgjRDNex8Tw8RehXAgnWKgdvtITNPJ1tQggh0Bq+bZZSvlFu1zxgrPX9WLTYvdORUj4upYyQUkajXbOFUsrrgUXAML3tK0VKeQjYJ4Robd3UB61RnktcR7SQzSVCCH/rPS+1z6WuYzmqum7zgJuslSOXAHnlQjxOQwgxAC2cOFhKWVBu1zxglNDWqI5BS3j+42z7pJQbpJRNpJTR1r+dbCDV+v/UJa7hBeidJLBjsuRKtAz9DuBJve2x2nQZ2tfi9cBa6+tKtDj478B24DegsQvY2hP4wfo+Fu0PKAuYCfi4gH0dgQzrtZwLBLvSdQSeA7YAG4EpgI8rXEfgG7S8QTGaIN1a1XVDS8S/Z/0b2oBWRaSHfVloce7Sv5kPy41/0mrfVmCgXtewwv7dnEvGOv0a2vJSLRAUCoXCw/GU0I1CoVAoqkAJvUKhUHg4SugVCoXCw1FCr1AoFB6OEnqFQqHwcJTQKxQKhYejhF6hUCg8nP8HWRuAWWH/XdUAAAAASUVORK5CYII=", + "text/plain": [ + "
    " + ] + }, + "metadata": { + "needs_background": "light" + }, + "output_type": "display_data" + } + ], + "source": [ + "t_initial = 50\n", + "scheduler = PolyLRScheduler(optimizer, t_initial=t_initial, cycle_limit=3)\n", + "plot_lr(scheduler, label='default, 1')\n", + "scheduler = PolyLRScheduler(optimizer, t_initial=t_initial, cycle_limit=3, cycle_decay=0.5)\n", + "plot_lr(scheduler, label=\"cycle_decay=0.5\")\n", + "plt.legend();" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "#### `cycle_mul`" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "`cycle_mul` is cycle len multiplier. So, if `cycle_mul=2`, next cycle will be twice longer." + ] + }, + { + "cell_type": "code", + "execution_count": 17, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "t_initial=50\n", + "total_epochs_2cycles=150\n", + "total_epochs_3cycles=350\n" + ] + } + ], + "source": [ + "t_initial = 50\n", + "print(f\"{t_initial=}\")\n", + "scheduler = PolyLRScheduler(optimizer, t_initial=t_initial, cycle_limit=2, cycle_mul=2)\n", + "total_epochs_2cycles = scheduler.get_cycle_length()\n", + "print(f\"{total_epochs_2cycles=}\")\n", + "\n", + "scheduler = PolyLRScheduler(optimizer, t_initial=t_initial, cycle_limit=3, cycle_mul=2)\n", + "total_epochs_3cycles = scheduler.get_cycle_length()\n", + "print(f\"{total_epochs_3cycles=}\")" + ] + }, + { + "cell_type": "code", + "execution_count": 18, + "metadata": {}, + "outputs": [ + { + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXoAAAD4CAYAAADiry33AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/YYfK9AAAACXBIWXMAAAsTAAALEwEAmpwYAABhxUlEQVR4nO2dd3wU5faHn3dLei+ENEihJvTeQVFAVLCA/dou4rVfvRaseP3pLcq1F1SwiwULYAdREEVKgNBb6Ak1IY0E0vb9/TG7yZLsJrvJ7s5mM8/nE3czM/vOYcx+58x5z3uOkFKioaGhoeG76NQ2QENDQ0PDvWhCr6GhoeHjaEKvoaGh4eNoQq+hoaHh42hCr6GhoeHjGNQ2oD4xMTEyJSVFbTM0NDQ0WhXr1q3Ll1LG2trndUKfkpJCVlaW2mZoaGhotCqEEAfs7dNCNxoaGho+jib0GhoaGj6OJvQaGhoaPo7Xxeg1NDQ8R1VVFbm5uZw5c0ZtUzQcJCAggKSkJIxGo8Of0YReQ6MNk5ubS2hoKCkpKQgh1DZHowmklBQUFJCbm0tqaqrDn3ModCOEmCCE2CmEyBFCzLCxf5QQYr0QoloIMaXevhuEELvNPzc4bJmGhobbOXPmDNHR0ZrItxKEEERHRzv9BNak0Ash9MBrwAVABnC1ECKj3mEHgRuBefU+GwXMBAYDg4CZQohIpyzU0NBwK5rIty6a8//LEY9+EJAjpdwrpawEPgUmWx8gpdwvpdwEmOp9djywREp5UkpZCCwBJjhtpQNsyVnNP+aMZ++hre4Y3uX8sfF73vn2n2qb4TDfbz7CkeLTrh3UVANZ78DpIpcOe6ryFF/t/oryqnKXjquh0VpxROgTgUNWv+eatzmCQ58VQkwXQmQJIbJOnDjh4NBnU1hyjMXGw3z06zPN+ryn+WTNs7xQ8AULVs9W2xSHuOuTDTz93XbXDnpsK3x7L3xyFVS57iay5MASZq6cyaQFk1h6cKnLxtVwP08++SSzZs1q9JgTJ04wePBg+vbty4oVK5w+x3vvvcedd94JwIIFC9i2bVuTn9mxYwdDhw7F39+/Sfu8Ea9Ir5RSviWlHCClHBAba3MFb5OM7DeJbhV6VlZuwlRT42ILXY9JKg8/z2x/jd0FO1W2pnGklNSYJIu3HiX/VIXrBjZVK68H/4QvboaaapcMWy2VcQw6A3//9e/c9ctdHDl1xCVja6jP0qVL6dmzJxs2bGDkyJEtGstRoY+KiuLll1/m/vvvb9H51MIRoc8Dkq1+TzJvc4SWfNZpRkSOJs8o+OKX19x1CpdhQhJaYyKkpoZ7Fk+juKJYbZPsYjI3IauqkXyxLtd1A1u6m3WZADu/h+/urdvWomGVMd6b8B739b+P1UdWM3nhZN7f+j7VJtfcTDRcxzPPPEOXLl0YMWIEO3fWOT179uxhwoQJ9O/fn5EjR7Jjxw6ys7N58MEHWbhwIX369OH06dPcdtttDBgwgMzMTGbOnFn7+ZSUFPLz8wHIyspizJgxZ5135cqVLFq0iAceeIA+ffqwZ88euza2a9eOgQMHOpXS6E04kl65FugshEhFEemrgGscHP8n4F9WE7DjgIedttJBbpjwOJ/N/5nFez/hCu5212lcRoCEW44F8mxCIQ8tu4/Xzn8LvU6vtlkNsG43+emag0wfmYZO54oJPPO4A/4KcT1gxSwIaQ/nPtqyUc32GnQGbupxE+NSxvHv1f9mVtYsvtnzDU8MfYJesb1aarzP8c9vtrLtcIlLx8xICGPmxZl2969bt45PP/2U7Oxsqqur6devH/379wdg+vTpzJ49m86dO7N69Wpuv/12fvnlF5566imysrJ49dVXAeVGERUVRU1NDWPHjmXTpk306tX0/99hw4YxadIkLrroIqZMmdLk8a2ZJj16KWU1cCeKaG8HPpdSbhVCPCWEmAQghBgohMgFpgJvCiG2mj97Evg/lJvFWuAp8za3EBEaw6CaeNb7l7Ivb4e7TuMiJDoJX1dO5+GCIv44uoaXN7ystlE2sch8p3Yh7C8oZ9XeAhcNbB5ZCDj3Mej7F/jtWVg7p2XDcvZTQWJIIq+c+wovjHmBwopCrvv+Op5e9TQlla4VNQ3nWbFiBZdeeilBQUGEhYUxadIkAE6dOsXKlSuZOnUqffr04dZbb+XIEdvht88//5x+/frRt29ftm7d6lAopq3h0IIpKeX3wPf1tj1h9X4tSljG1mffAd5pgY1OcWm/u1i65TE+XPoUT1w/r+kPqIRFijL7j+DMmqFM8c/inS3v0D26OxNS3JKY1GwsejyxR3ve//MAH685yLBOMa4Y2fwqFLG/6EUoy4fv7oegGMi8pJmjKuPqRJ0fI4TgvI7nMTRhKK9ueJV5O+ax9OBSHhz4IBNSJmgphtCo5+1pTCYTERERZGdnN3rcvn37mDVrFmvXriUyMpIbb7yxNsfcYDBgMilzYW195a9XTMa6ktH9J9O1Quf1k7ISiQCuHtSBWVVT+XuxpI/JyBO/P87Ok941OWsyK72/Uc/l/ZJcNylb69Gbf9cbYMo7kDwIvroF9jmfUaEMK83DNhTvYGMwDw16iE8u/IS4oDge/O1B/vbz3zhUcqjBsRruZ9SoUSxYsIDTp09TWlrKN998A0BYWBipqanMnz8fUP6fbty4scHnS0pKCA4OJjw8nGPHjvHDDz/U7ktJSWHdunUAfPnllzbPHxoaSmlpae3vr776am1IyJfwOaEHGBGhTMp++esbaptiF0XoBV3iQunSMYk3dH/h+dx9hAoD9/x6D0VnitQ2sQFCwDWDk6mqkXzpiklZaVl2YSXIfkFw9acQlQafXgNHNzs/LPaF3kJGdAYfT/yYhwc9zMYTG7l00aXM2TyHKlOV0+fTaD79+vXjyiuvpHfv3lxwwQUMHDiwdt/HH3/M3Llz6d27N5mZmSxcuLDB53v37k3fvn3p1q0b11xzDcOHD6/dN3PmTO655x4GDBiAXm977uuqq67iueeeo2/fvuzZs4cdO3YQHR3d4LijR4+SlJTE888/z9NPP01SUhIlJa0o9Cel9Kqf/v37y5ZSWHJCDp2bIf86e0iLx3IX094cKse9nSmllPLLdYdkykOLZPGrY+TG/3WSfT/oK6f9NE1W11SrbKVCeUW17PjQt/L1X3OklFJOfWOlHP3sL7KmxtSygff/IeXMMClzfmm4r+iQlP/rLuVznaUsPODUsB9u/VD2eK+HLDpT5NDxx8qOyXt/vVf2eK+HnLJoityav9Wp87Vmtm3bprYJXsWFF14oKyoq1DajSWz9fwOypB1d9UmPPiI0hoFePikrpaz1Nyf2jCc0wI/Xg26jV2k+jwZ1YdWRVby+8XVVbbRQF/NWfr96cLJrJmWtJ2PrE54E130F1Wfg4yucWj1bfzK2KdoFteP5Mc/z4pgXKThdwDXfXcPzWc9zutrFK4E1vJ5vv/0WPz8/tc1wOT4p9ACX9r2DKiH4cOlTaptiGyttCzDquaxfEnNzQjjd5yYu3/ITlySO5q1Nb/Fb7m/q2Wimvh5f0COe8EAjH6852NKRza92QiztusGVH0FBDnz+F6iudGxUS4zeyQnWsR3HsuCSBVzS6RLe3fouly+6nDVH1jg1hoaGN+KzQj9mwKV0qdCxqsI7J2WtPXqAawZ3UBYkhV0PAeE8mrefrpFdeXjFw+SdctsaM4ewTMZaYt7KjSmRJVuPUVTumPjapDGP3kLqKJj8Kuz7Db65x6EFVbVPIM348w7zC+PJYU8yd9xcAP66+K88ufJJLRVTo1Xjs0IPMCR0CIf8BD/8+aHapjSgvlx1iQuld1I4H28sQY55mIB9K3i+w8WYpIl/LPsHlTUtENQWUut3W+nx1P7JVNaYWJh92BUjN35Y76tgzCOwcR4sf9bh0VuSMjkofhBfTfqKm3rcxIKcBUxeMJmfD/zc7PE0NNTEp4X+2vMewd8k+WH7+2qb0hChLJiyZkr/JHYcLWVrwhSI7UaH5S/w9NAn2VqwlWfXOi5wrqbO8a4TzoyEMDLiw5i/rgVpiZasG+HAn+HoB6H3NbDsX5D9SePDNpJe6QwBhgDu638f8y6cR0xgDPcuu5d7f72XE+XNK7ynoaEWPi30CbEd6VsVxjr9cYpPuW1BbrOwFYGY1DsRP4OOLzYchfH/gsJ9jD28k5syb+KznZ/x7d5vPW8o1Dre9WVz6oAktuSVsP1IM8MajoRuLAgBF7+khHIW3aWEcuxgalAtu2VkRGcw78J53NPvHn7L/Y3JCyfz9e6vzyoNoaHhzfi00AOMTZnCKb2Ojxb/W21TzsKSR29NeJCRcRlxLMjOoyJlDHQeD789x91drqFfu3489edT5BTmeNzW2hh9PT2e3CcRo160oNCZg6EbCwY/uOJDiE6HT6+D47Yzqpo7GdsYRp2RaT2n8eWkL+kS2YUnVj7BbUtv42jZUZedQ0PDXfi80E859y7aVZv4I7911CWfOiCZovIqlm4/DuOfgapyDMv/zazRswgyBHHvsnspqyrzqE325Dgq2I/zusexYEMeVTXN8KKd8egtBEbAtfPBGAAfT4VTx23Y65rQjS1SwlN4Z/w7zBg0g/XH1nPZwss0715F9u/fT48ePbz+vI8++ijJycmEhIQ0OmZgYCB9+vShT58+/O1vf3OFqUAbEHqDwcggkc4Wv0o27V6ltjm1SGwvYhjRKYb2YQHMzzoEMZ1h0HRY/wGxJUd5bvRzHCw9yMyVMz0qLI15yFP6J1FQVskvOxoKrgMjm1+dFOSIDsrq2bIT8Nl1UG27HIPOkdh/M9AJHdd2v5YvL/6SLlGKd3/70ts1717DLhdffDFr1jSdqpuenk52djbZ2dnMnu26pkQOFTVr7Vw++D6+zbqLz/94jl6dbde88DT2ZFqvE1zWL5HZy/dwrOQMcaMfhI2fwo8PM/CGb7ir7128tP4lBscPZmqXqR611VZl4tFdYokN9Wd+Vi7jM9s7OXAzPHoLif3g0jdg/o1K2uUlb9SO46rJ2KZIDkvmnfHv8MmOT3hp/UtctvAyHhj4AJd0uqR1Fkn7YUazSk40SvuecMF/Gj3kgw8+YNasWQghSEtLIzs7m127dmE0GikpKaF3797s2rWLAwcO8Le//Y0TJ06g1+uZP3/+WaUNampqmDFjBsuWLaOiooI77riDW2+91eY5ly1bxsyZM4mIiGDz5s1cccUV9OzZk5deeonTp0+zYMEC0tPTufHGG88qYxwSEsKpU6ecvgxDhgxx+jOuxOc9eoABmWPoXqFnbfVO78mpF/b92Cn9kzBJ+HpDHgRGwjmPwP4VsPN7bu5xM8MShvHfNf9lV+Euj5gqbeVXmjHodVzWN5Ffdx7nRKmThc5qhb6Zf4aZl8KYh2HjJ/DHS3XD2ps9dgO2vPs7lt7BsbJj7j+5D7B161aefvppfvnlFzZu3MjcuXMZM2YM3333HQCffvopl112GUajkWuvvZY77riDjRs3snLlSuLj488aa+7cuYSHh7N27VrWrl3L22+/zb59++yee+PGjcyePZvt27fz4YcfsmvXLtasWcO0adN45ZVXHP437Ny5szbcUv+nqKjIqeuxb98++vbty+jRo5vVJtEebcKjBxgSPoJ3zyxn4W9vc+k5rot9NRcppV0hSosNoX/HSOZnHeLWUWmI/jfB6jfh5yfRdR7PMyOeYcqiKTyw/AE+ufATgoxB7reVxm9Mb/62l4XZeUwbmebMyObXFijy6IfgxA74+UmI6QLdJta2aXS3R29Nfe/+0oWXtj7vvgnP2x388ssvTJ06lZgYpex1VFQU06ZN49lnn+WSSy7h3Xff5e2336a0tJS8vDwuvfRSAAICAhqMtXjxYjZt2sQXX3wBQHFxMbt37yY1NdXmuQcOHFh7s0hPT2fcuHEA9OzZk19//dXhf0PXrl2bLKfsCPHx8Rw8eJDo6GjWrVvHJZdcwtatWwkLC2vx2G3Cowf4y7hHCDSZWLz7Y7VNARSJE9K+AEztn8SeE2VsOFSklO8970nI3wUbPiQmMIZ/j/w3+4r3eSS/vhGHHoDOcaH0To5gflauc3MH9csUNwchYPLrkNBHKW18bKtbJ2Mbw553f7y8OfMXbZfhw4ezf/9+li1bRk1NjcOTnlJKXnnlldoY9759+2rF2xb+/v6173U6Xe3vOp2O6mpz32GrmvYmk4nKyoYLF13l0fv7+9dWzuzfvz/p6ens2uWap/Y2I/SxkQn0q4pivfEkhcXesOBFNipDF/aKJ8Coq0td7HYhJA+BZf+GyjKGJgzlrz3/ype7v+SHfT80MpILLDXrsa4Rz3Rq/yR2HitlS54zOfUu8OhBKW181TzwC4F5V0GFkpWklidt8e5nDJrB2qNruWzRZfy0/ydVbPF2zj33XObPn09BgVIg7+RJZb3L9ddfzzXXXMNNN90EKHXjk5KSWLBgAQAVFRWUl5efNdb48eN54403qKpSSk3v2rWLsrKWZahZ17RftGhR7djWWDx6Wz8REREOn+vEiRPUmEPLe/fuZffu3aSlOfOEbJ82I/QA56ReTrlOx7yl6q0ytSBpXN5CA4yMy2jP95uPUFltUjzXcf8Hp47Bn0rz89v73E6f2D78889/urVxRp2HbJ+LesVj1AsWZDtRl6clk7H1CUuAq+dB2XHkVmXC3dMevTUW7/7ziz8nOSSZ+5ffz4wVM7SaOfXIzMzk0UcfZfTo0fTu3Zv77rsPgGuvvZbCwkKuvvrq2mM//PBDXn75ZXr16sWwYcM4evTsLKdp06aRkZFBv3796NGjB7feemutZ95cbrnlFpYvX07v3r35888/CQ4ObtY4Dz74IElJSZSXl5OUlMSTTz4JKDePJ55QmvX99ttv9OrViz59+jBlyhRmz55NVFRUi+yvxV79YrV+XFGP3h5VVZVyzJwMed1s953DUa5+s6+89M2ejR6zdPtR2fGhb+XirUfrNn56rZTPJEhZelxKKWVeaZ4cOm+ovPKbK2VldaVbbM0tLJcdH/pWfrqm8brw095fKwc+vURWO1qnfts3Sj36w9kusNLM5i/kqy8kyx7v9ZDS1MJ6+S6isqZSvr7hddn7/d5y7Odj5arDq9Q2qRZvrUc/f/58ed1116lthtei1aNvBIPBSD+ZzBb/M+rXqZdNe5wjO8cSFex3tpc89kmoOg3L/wtAQkgCTw17iq0FW3lx/YvuMdXBdMVL+iRyvLSCP/c4WqfeRaEba3pcjuw4DCGlMoHtBRh1Rm7rcxsfTfyIQEMg0xZP479r/suZ6rbdx9Qed911FzNmzODxxx9X2xSfoU0JPcCEHjdRLQSfLX9OVTtkI+mVFox6HRf2jOfnbcc4VWF+BI3pBP1vhHXvQsEeAM7reB5Xdr2SD7Z9wMrDK11vq4N6PLZ7O0L8DSx0NHzjytCNFaYOw5T4/E+PwP7fXTp2S+gR04PPL/6cq7tdzUfbP+LKb69kW8E2tc3yOl555RVycnLo0qVLi8favHlzgwnSwYMHu8DK1kWbE/qxA6eQXCnJOpWlsiWOZadc0jeBimoTP22xikeOmQF6f1j6z9pN9w+4n7TwNB7//XG39ZttbDIWlDr1E3q058ctRzlT5ch6BTd49ObhhNArfWc/vwGKXdDf1kUEGgJ5ZPAjvHn+m5yqPMW1313LW5veotrUsliyhm169uzZYIJ09erVapvlcdqc0Ov0evobu7PT38T6Ha5bkOAsTU3GWujXIZKkyMCzwzch7WD43bBtIRxaCygldf898t+crDjJU6uecmmJBGeyIC/pk0hpRbVjJRHc5NHXFoy7ap5SHuGz66DKu8IkwxKG8dXkrzi/4/m8suEVbvjxBg6WtLRjl4aGbdqc0ANcOuhOABasflk1G5rOY1EQQjC5TwJ/5ORzvNRKrIbeCcGx8Etdq8SM6Azu7HMnSw4sYdGeRS6z1V71SlsMTY8mNtSfBRscCN84U4/eCaSUSugmtgtcOhsOb4Dv7nOoO5UnCfcP59nRz/LsqGfZV7yPqd9MZWHOQq1AmobLaZNC3y9jNF0rdKyr3KFaSQTZRB69NZf0ScQk4btNR+o2+ofAyH8oddn3Lq/dfGPmjfSP68+/Vv+LQ6WuSblsasGUNXqd4OJeCSzbeYLi8oY5x3ZGboF1tka1KgHd/SIY9SBkfwxr57j0PK7igtQL+GrSV2REZ/DYH4/x0IqHKK0sVdssDR+iTQo9wIDg/hz0g1+yvlLNBkflrXNcKBnxYSyo37av/00Qlgi/PF3rrep1ev414l/ohI5Hf3/UJbFfi4fZVIzewiV9E6isMfHDliONH+jO0I31mGMehs7j4McZcOBPl57LVbQPbs+ccXO4q+9dLN6/mKnfTCX7eLbaZmn4CG1W6K8c8yAGKflx8zuqnF+J0TsucJf0TWDjoSL25Vut9DMGwKgHIHcN7F5SuzkhJIFHBj/ChuMbeGdLy/99zgYSeiaGkxYT7MTiKRdPxtZPXdXp4LK3lfLGn18PJU3cgFRCr9Mzvdd03r9AaX154483MnvjbGpMXlKIzwtpDfXoy8vLufDCC+nWrRuZmZnMmDHDzdY1pM0KfWpiN3pUBLBeHKS6uqkQg+uRSKcUdFLvRISgYepi3+sgMgV++T8w1TX/uCjtIiakTOCN7DfYmr+1ZbY62bFJmVdIZPW+kxwpPt3YwJYPtMi++pikqaGtgRHK5GzlKfjiZqjx3iyX3rG9+eLiL5iQOoHXsl/j5p9u5sgp77w5aTjG/fffz44dO9iwYQN//PEHP/zg3rIl9Wkz1SttMThmDG+e+omvfn2DK86/2+Pnd0bf2ocHMCQ1moXZh7lnbOc6IdMbldDE17fCjm8gY7J5bMFjQx5jw/ENzFgxg88v/pxAQ2Cz7GxO7bHJfRJ44eddLMo+zK2j0+0MbLkxuT50Y5N23eGiF+Hr6cok9vlP2T7OCwjxC+E/I//D8IThPLP6GS7/5nKeHPok41LsF+lqKf9d8192nHTtQsJuUd14aNBDjR7j6/Xog4KCOOeccwDw8/OjX79+5OZ6NuW3zXr0AFePvZ8Ak2TF/q89fm5Hs26smdQngX35ZWyr34y751SlRO+v/wKrx/xw/3CeHvE0+0v28/L65mcYOTMZayElJpjeSeF8t7kxT9R9MXq73aV6X6nMbfzxEuz43qXndQcXp1/M/IvmkxKWwj+W/4OZK2dSXlXe9AdbCW2tHn1RURHffPMNY8eOdfwiuYA27dFHR7SnZ1UwGw3HqayswM/Pv+kPuRDhZPB7fGZ7Hluwhe82HSEzIbxuh06vNCeZfyNs/kIRMzND4odwZdcr+Xj7x5zX8Tz6x/V32k5HqlfaYmLPeP79ww4OFpTTIdpGzXx3TcbKho3Xz2LCf+DweljwN7j1NyX05cUkhyXz/gXv80b2G8zZPIeNxzfyvzH/Iz3CzpNSM2nK83YHbakefXV1NVdffTV33323y6pSOkqb9ugBBsaeS6Fex9fL3/DoeZ1Jr7QQFezHsPRovt98pGGudffJStu2Zf+GmrPnHO7rfx8JIQk8/sfjzfIGHaleaYuJPZUv0fd2s2/ctDKWJia6jQEw9X3l9J/f4HWLqWxh1Bm5u9/dvHn+mxRWFHL1d1ezMGeh2ma5BV+tRz99+nQ6d+7M3//+d4f+Pa6kzQv9Fefcg79J8vt+z35pmiedinjuLyhvGL7R6eCcx6Bwn5IzbkWQMYj/G/5/HCo9xEvrX8JZLHO8zjreyVFB9E4K53t74Rs3plc2eWmjUpWes0eylZo4rYShCUP54uIv6BnTk8f+eIzHfn+sVYdy2ko9+scee4zi4mJefPHFFtnTXNq80EdHtKdXVTAbdUr4xlM4UtTMFuMz26PXibMXT1noMh4SB8Bv/2vg1Q9sP5Bru1/LvB3zWHt0rXO2tsDzntgznk25xRwssCVG7vHoTdLkWOpqtwth2F2QNRc2zXepDe4kNiiWt85/i9t638aiPYu45rtryCnMUdusZtEW6tHn5ubyzDPPsG3bNvr160efPn2YM8fDi/fs1S+2/gEmADuBHGCGjf3+wGfm/auBFPN2I/A+sBnYDjzc1LncWY/eHq9/+aDs8V4P+dnilzx2zslv9mx2Xfxr314lRz/7izTZqre+8yelxvu6DxrsKqsskxO/nCjHfzFellWWOXy+zblFsuND38oftxxx2taDBWWy40PfyjeW5TTcue59xdbCg06P2xhP//m0HP7JcMcOrq6Ucu54KZ+Ol/L4Tpfa4Qn+PPynHP3paDngwwHy691fO/15rR5968Tl9eiFEHrgNeACIAO4WgiRUe+wvwKFUspOwAvAf83bpwL+UsqeQH/gViFESrPuSG7kinPvxd/D2TdSOB+jt3BhLzvhG4DO50N8H1jxvwa54pYQzuFTh3l+3fNOn9fZyVhoInzj7qJmjqA3wpR3lLj9Fze3ini9NUPih/DFpC/oFduLx/94nEd/f7RVh3JAq0fvDhwJ3QwCcqSUe6WUlcCnwOR6x0xG8dwBvgDGCiXRWwLBQggDEAhUAl7XS02t8E1zaTR8I4SyWrZwH2z5osHufnH9uC7jOj7b+Rlrjqxx6Hwt7eFtP3zjvslYu+mVtghLgEvegGOb4eeZLrfF3cQExtSGcr7Z8w1Xf3d1qw3lgFaP3h048m1IBKyrY+Wat9k8RkpZDRQD0SiiXwYcAQ4Cs6SUJ+ufQAgxXQiRJYTIOnFCncbdA2PGUGjQsWC5Z7oSKfObzRO4qGA/hqbZyb4B6DoR4nrAb7POyqu3cHffu+kQ2oEn/3zSoS5HzlSvtIXd7Bs3plc6TZfxMPg2WD0bdnp21aIr0Ov03N7ndt4a9xbFFcVc8/01fL/XsXUCzbperQRfrEffnP9f7p6MHQTUAAlAKvAPIUSDBFIp5VtSygFSygGxsbFuNsk2lvDN7/sXeOycLZE3u9k3oGTgjLofCnbDtgUNdgcYApg5dCaHSg/xxsam00qbs2DKmuSoIHrZDN94oHqlM5z/TyVFdcHtUHK46eO9kCHxQ5h/8Xy6R3XnoRUP8Z81/6Gqxn6Jj4CAAAoKCnxa7H0JKSUFBQU21xE0hiMLpvKAZKvfk8zbbB2Taw7ThAMFwDXAj1LKKuC4EOIPYACw1ykrPUB0RHt6VgaTbThGdXUVBoPRredztqhZfcZnxvHYgs18v7ne4ikL3SdDTFfFq8+4VBF/KwbFD+Kyzpfx/tb3mZAyge7R3e3b6mStG1tM7BnPf37YwaGT5SRHmRdPuakevc1aN45g8Icp78Kbo+Cr6XD9QmUxWisjNiiWOePn8MK6F/hw24dsK9jGrNGzaBfUrsGxSUlJ5ObmotaTtIbzBAQEkJSU5NRnHBH6tUBnIUQqiqBfhSLg1iwCbgD+BKYAv0gppRDiIHAu8KEQIhgYArzolIUepF/0cLLKlvDdH+8zefQ0t5+vJX5sdIg/g1KjWLLtGA+M79bwAItX/9UtsONbyJjU4JD7+t/H8kPLmblyJvMunIdBZ/vPwRV+9/jM9vznhx0s2XaMm0eYVyq6KXQDLbiJxnSGic/Bwjvg9+eV+Y5WiFFn5MGBD9IzpiczV87kim+uYNboWQxoP+Ds44xGuytHNXyHJl0pc8z9TuAnlBTJz6WUW4UQTwkhLOoxF4gWQuQA9wGWOpyvASFCiK0oN4x3pZSbXP2PcBWXjrobg5T8nuP+GvUmh5sJ2mdcRnt2HTvF/nw7i0IyL1P6pv72rM3uSuH+4Tw8+GG2n9zOR9s+snseV3j0qTHBdIkLYcm2Yzb2eknoxkKfa6HHFPj133CwdcdzL0i9gHkT5xHqF8q0xdP4YOsHWpimDeLQM7OU8nspZRcpZbqU8hnztieklIvM789IKadKKTtJKQdJKfeat58yb8+UUmZIKZ9z3z+l5SS1S6FbpR9bpGd6d7bUkT0/Iw7AjngCeoPSheroZshZavOQcR3HMSZ5DK9lv2a3I1VLs26s7V2z/ySFZZX1BnZDrZuWjCkEXPQ8hCfBl9PgdJHLbFODTpGd+OTCTxiTPIbnsp7jgd8eaPUpmBrO0eZXxtanV3Bvco2C37PdW9lQChCyZQKXHBVE9/gwFm87av+gnldAaAL88aLN3UIIHh38KHqdnqf+tN1UvKWTsRbGZbSnxiStGoe7cTK2pcYGhCv59aWH4fv7XWOYioT4hfDCmBe4t/+9LDmwhKu/u5p9xfYrO2r4FprQ1+PCQUr96sUb31PXEAcZlxFH1oFC8k/Zyf83+MHQ22H/CshbZ/OQ9sHtubffvaw6sspmU/HmVq+sT8/EcNqHBdTdmNSqXukoSQNg9EOweT5s+bLl46mMEIKbe9zMW+e/ReGZQq797lpW5K5Q2ywND6AJfT16dR5CeqVgS4VrGzDUR9KymLeFcZlxSAlLt9sJ3wD0uwH8w5Ua7HaY2nUqfWL78L+s/1FcUXy2rZYYfQtt1ekE52fEsXzXCU5X1lhl3XhZjN6aEfcp9YO+va/VplzWZ3D8YD696FMSQxO5Y+kdvLPlHS1u7+NoQm+DHsbO7PaXbN9r2wN2BbL2Py0jIz6MxIhAFm9tROgDwmDgX2HbIijYY/MQndDx2JDHKK4sbtCkxOTCCMu4zDjOVJn4PScft4VuWhqjt0ZvgEvfhJpKWHinzUnt1khCSALvT3ifcSnjeGHdC8xYMcOhxXMarRNN6G1wfs/rAVi0yn016l3l0QshGJcZx4qcfMoqGqnUN/hvoPeDlfY753SN6so13a5h/q75bD6x2cpWi0ffcnsHp0YTGmBg8daj7i1T7EpiOsG4/4M9S2Gth6sOupEgYxDPjXqOu/vezQ/7fuCGH2/gaFkj8z0arRZN6G0wss9FJFRJNpVucOt5XBVeGJfRnspqEyt2N7LoJTQO+lwN2fPg1HG7h93R5w5iAmP4v1X/R42lfEJtjL7ltvoZdJzbrR1LdxzH5MaesU7VunGEAX+FTufB4schf7drx1YRIQS39LqFl899mQMlB7jq26vIPp6ttlkaLkYTehvo9Hp6kMQ2/wqO5ttOOWwpLc+ir2NgSiQRQcbGwzcAQ+9SQhCrZ9s9JMQvhAcGPsD2k9uZv2t+ra3gmicQUG5MJ8sqyT1ZbhnYJeNacNlkrDVCwKRXlSqXX01vUO+/tTMmeQwfT/yYIGMQN/10E1/tdv9aEg3PoQm9HUakX0K1EHz1m+NNgp1BulDqDfo6L7m6xmT/wJhO0P1iJfxQUWr3sAkpExgcP5iX179M/un8Fhc1q8/orrH46XXsPmqu0+Ni79sl6ZW2CIuHi15Q+s2ucL7Ms7eTHpHOJxd+wsC4gcxcOZN/r/431aaWNe7Q8A40obfDhcNvIqzGxMb8lW4Zv7kdpuxxXvc4ik9XseFQUeMHDv87nCmG9R/YPcSSW3+65jTPZz3vsgVTFkL8DQxOi2LvCcvNphV49BYyL4WeU5XVxke3uOccKhLuH87r573OXzL+wrwd87hj6R2UVtp3CjRaB5rQ28HPz5+M6nC2GQqprnb9Y3pLi5rVZ0TnGPQ6wa877MffAUjqD8lDYPWbNksYW0gNT+WmzJv4Zu837CreCLg2wnJO13actKyQ9fbJ2Ppc8CwERsLC2xs0d/EFDDoDDw58kCeHPsmaI2u4/ofryTtVv46hRmtCE/pG6BU5mCK9jsWrPlHblCYJCzAyoGMkv+50oArhkNug6ADs+rHRw27pdQsJwQl8vu9loMal4ZBzurVDtIb0SlsERcGF/4MjG2Gl883WWwuXd7mcN85/g2Nlx7jmu2vYdMJry1RpNIEm9I1w0bBbEVLy+27Xtxh0tUcPinhuP1LC0eIm8qG7XQThydBE+migIZAHBj7AkdP7MEaucam1qTHBRAaZq2W6waPXuftPO2MyZFwCy/4Dx927uE5NhsQP4aMLPyLIEMTNP93MT/t/UtskjWagCX0jpCZ2o3Olnu1VthcZtQTXrDU9m3O6KvXGf93ZRPhGb4CB05SyCE3Emcd2GEun0N74xSyhrNq1sdr0GKUu/emqRiaQm4lbPXoLE2eBX4hS0riRMFhrJy08jY8v/JjuUd25f/n9zNk8R1tJ28rQhL4JuhnTyfGX7Drg2sdWZcGUS4ekS1wICeEBTcfpAfpdD4ZAWN24Vy+E4NKOtyP0p/l637suslQhNSYYgNX7Cl06rsdEKCRWqV2flwWrXvfMOVUiKiCKOePncEHqBby0/iUe/+PxRjtXaXgXmtA3wahuUwD4frVrV0RKWl69sj5CCM7p1o4/cvKpqG7CwwyKUhZQbZoPZfmNHpoQmE5V0SB+zvuaPUWue7pJilTaof26y7XdjZrdYao59Lgcul4IvzwN+a23Ibcj+Ov9+e/I/3Jb79tYuGcht/58KyWVNlpZangdmtA3wdiBU4mqNrG5aK1rBxbuCS+c07UdZZU1ZO13wEse/DeoqYB1jXvqEqg8cT4B+kCeXfusyzxmo3mp7S87TrjUC3dpUbOmsNSuN/jDojvB5PowlDchhOD2PrfzrxH/YsPxDdzwg1Y2oTWgCX0TGAxGutdEsd1QwpkK1zVrcJccDOsUjZ9e51j4JrYrpJ8La+dCdaXdw0xSImtCuDztZlYeXslvub+5xlizuB8qOsOeE3a6ZDVnWE8KPUBoexj/Lzj4J2z40HPnVZGL0y9m9nmzOVJ2hOu+v47dhb5TFsIX0YTeAfrGjqBUr+OHlfYXGTUHd4hRkJ+yGKnJCVkLg2+D0iOwbaHdQyzO9vjky0gJS+G5rOdcFJ+tS69c5qi9Dg7r8lo3TdHnWug4ApY83mgtIV9icPxg3p/wPiZp4oYfbyDraJbaJmnYQRN6B7h4xN/QS8nKvd+4bEx3Thee07Ude06UcbDAgSeQTudBdKdG699YrDXqjDww8AEOlBzg812ft9xQaQIEXeJCrLpOuWBYdy+YsoUQSnmEynL46VHPn18lukZ15aOJHxETGMP0JdNZvH+x2iZp2EATegdIiO1Il0oDO0yu6yWr5NG75/Kf083BNEsAnU5JtczLUhYA2aC2w5QORiaOZHD7wby58c2WL42XEoTgnK7tWLv/JKcaK7PszLDuqnXTFLFdYOR9sPlz2POL58+vEgkhCXx4wYdkRmdy//L7+Xj7x2qbpFEPTegdpJt/Z/b74dI0S3dJUWpMMMlRgazY3Xg2TS29r1JSLbPesbm7LsAiEEJw74B7Kawo5N0tLU23VG53o7rEUlUjWb23oIXjKZikybMxemtG3AdR6fDdP6DqtDo2qEC4fzhvj3ubc5LP4T9r/sPz6563KkOtoTaa0DvI0E6TAPhpjWtyyU0C1yfSWzGiUyyr9hZQ1Vg1SwuBkUqa4Kb5cKZhulz96pWZ0ZlMTJ3IB9s+aFnGhdmj798xEn+Dztx1quV4fDLWGmOAEsI5uRdW/E8dG1QiwBDA82Oe58quV/Lulnd59PdHqTJpufbegCb0DjJ24BWE1ZjYWuiaCSd3lECwZmTnGE5VVLMpt8ixDwy4GarKYNNnDXbZql55V9+7MEkTr2e3ZKGQchUCjHoGpUbxu6NPIA4Mq0roxkLaaOh1Ffz+IpzYqZ4dKqDX6Xl08KPc3fduvt37Lff9ep/WotAL0ITeQfz8/OlaHcJO/UlMNa5Z7u5OKRqaFo0QOB6+SewH8b0h690GfVFtNR5JCk3i6m5Xs3DPQnYV7mqekdJUW4t+RKcYdh8/1XSdHkeGVStGb834Z8A/BL75u8/n1tfH0rXqscGPsTx3ObcvvZ2yKtelz2o4jyb0TtAttDf5Bh0rN/3Q4rHc7dFHBvvRMzHccS9ZCMWrP74VDq05a5e003hkeq/pBBuDeXHdi80z0hy6AaXMMsAfLgjfuLUevaMEx8D5T8HBlcrkbBvkym5X8p+R/2HDsQ389ae/UnjGtaUuNBxHE3onOK/3tQAs3zq/xWOZELjXp1e85A2Hiig942CctMcU8A+DrLlnbbbXeCTcP5xbet7CirwVrD6yuhkW1nXZ6t4+jOhgP5fE6VWN0VvT5zpIHABLnrA599EWmJg2kRfPeZGcohxu+vEmjpU10e5Swy1oQu8E/TJGE18l2VG21SXjuVuMRnSOocYkWbX3pGMf8A+BXlfC1gVQVpcBY8lLtxUOuab7NbQPbs/LG152voyBlUev0wmGd4rh95z8FpdDcHs9ekfR6WDis8oCqt+eVdsa1RidPJo3znuDo+VHueHHGzhU4p4+zBr20YTeSbrIWHb5nab8TMtijhLc7dDTv2MkAUadc+GQATcr9W+sJmUbayXor/dneq/pbDqxiRV5K5phZd2oIzrHcKK0gl3HTjVjnDq8xqMHSOwPfa9Tav+faOZchg8wsP1A5o6bS1lVGdf/eL1WMsHDaELvJD1jh1Ku0/HTny2raSIFbm+O4W/QMyg1mhW7nagOGZcBCf1gw0e1Cl+7YMqOl3xJp0tICkni1Q2vOpc7beXRgxJqApyz19awaqyMbYyxM8EYDD8+1GCiuy2RGZPJexPeQ4eOm366iW0F29Q2qc2gCb2TTBwyDSEla/Y33oavKTz1dR/ZKYY9J8qcy2bpe50yKXt4A2CddWP7cKPOyG19bmP7ye0sPbjU8fNYZd0AJEQEkhYb3OIJWa8J3VgIiYVzHlFWy+74Tm1rVCU9Ip33L3ifYEMw0xZPY2u+a8KgGo2jCb2TJLdPI7VKsLfqQIvGUUI37hejoenRAKze58Sq0x6XgyEAspWl7CYHvNALUy8kNTyV1za8Ro3D3ZbqJmNr7U2LJmt/IdWOLPSyO6r0fFGzphg4DdplwE8Pt6kVs7ZICk3i3QnvEuYXxi2Lb9F60XoAL/s2tA7SRQI5xiqKTzk4yWkDd6dXWugeH0ZogIFVzpQXCIyA7pNg83xFlCwx+kbM1ev03N7ndvYU7+FHR592pGwQ+B+SFk1pRTXbjjQ/S8Ur0ivrozfABc9C0UFY+ara1qhOQkgC7014j4iACKYvmU728Wy1TfJpNKFvBj3ihlOpEyxeNa/ZY3hK6PU6weDUKMczbyz0vQ7OFMOO7xrNurFmXMdxdInswuvZr1NtcqRAWUOPfnBaFIBzN6YGo3qh0AOkjoTuF8PvL0CplmbYPrg9745/l5jAGG5dcivrjq1T2ySfxSGhF0JMEELsFELkCCFm2NjvL4T4zLx/tRAixWpfLyHEn0KIrUKIzUKIABfarwrjBl+PkJLsQ07Eo+shhfCYGA1OjWZffhnHSpyI06eMhIgOsOFDq8nYxj+iEzru7HMnB0sP8s0eB0o615uMBWgXGkBabLDzN6azhm34pOA1nPdPJatp2b/UtsQriAuO493x7xIXHMdtP9/G2qMu7uSmATgg9EIIPfAacAGQAVwthMiod9hfgUIpZSfgBeC/5s8agI+Av0kpM4ExQKuvcpTULoXUKsGeqv3N+nxtCQUPTRgOSVPi9E55yTqd0kxj73ICy/IAx55AxiSPoUd0D2ZvnN10Qat6k7HW9q7dd5IaU/OmrL3WoweIToeBt8D6D+CYlnUCEBsUyzvj3yEhOIHbf76dPw//qbZJPocjHv0gIEdKuVdKWQl8Ckyud8xk4H3z+y+AsUJ5zh8HbJJSbgSQUhZIKV1TKEZl0kQ8OcYqSsuKnP6sJQXRU2KUkRBGqL/BeS+5zzUApOYp3accuS8JIbitz20cLjvMd3ubyjBpGLoBqzj94ebF6b1a6AFGPwj+ocqKWQ0AYgJjeGfCOySHJXPXL3dpnr2LcUToEwHrpWy55m02j5FSVgPFQDTQBZBCiJ+EEOuFEA/aOoEQYroQIksIkXXiRMtyqD1Fz7jhVOgEi1c7H6c3mSxC7xn0OsGg1CjnMm9ACd2kjiI1bxEgHbZ3ZOJIukV1Y87mOY1n4NgI3QAMSW1hnF6NVoLOEBQFox6AnCVtqkFJU0QFRDFn3BySQpK4Y+kd2gStC3H3t8EAjACuNb9eKoQYW/8gKeVbUsoBUsoBsbGxbjbJNZw/8C8IKVl/4GenP2syP9S4q8OULYakRbP3RBnHnYnTA/S6ktDTefQTux3OTRdCML3XdA6UHGDxgcZay9n26NuFWeL0zRN6iRfH6C0Mmg4RHWHx4+BwOqrvExUQxZzxc2gX1I7bfr5Ny7N3EY4oTR6QbPV7knmbzWPMcflwoADF+/9NSpkvpSwHvgf6tdRobyC5fRopVYI9Vfuc/qypsZoCbqI2m2Wfk+Gb7hdTrfNnkn6lU1MKYzuMJT08nbc2vWV/tawdjx6UCeQ1zYzTq9phylEM/nDek3BsC2z8RG1rvIqYwBjmjJtDuH8405dMZ+fJtlXT3x04IvRrgc5CiFQhhB9wFbCo3jGLgBvM76cAv0ilMtVPQE8hRJD5BjAa8JkZqPRmxulrapRJSk969Bnxlji9k15yQBi5saO5SL8KUeP4PLpO6Lil1y3kFOXw68FfbR9kZzIWYEhaVLPj9F4fo7eQeSkkDYRfnm7zi6jq0z64PXPHzyXQEMj0JdPZU7RHbZNaNU0qjTnmfieKaG8HPpdSbhVCPCWEmGQ+bC4QLYTIAe4DZpg/Wwg8j3KzyAbWSyl9Zg1499jBVOgEv6770qnP1U7GelCLDHod/VMiWbff+Zrge9tPIEaU4HfwN6c+Nz5lPB1CO/DmpjftVKS0HboBxaMHyDrgfJql15VAsIcQildfegTWzm3y8LZGYkgic8fPVZyGxbdwoKRlq9HbMg65lFLK76WUXaSU6VLKZ8zbnpBSLjK/PyOlnCql7CSlHCSl3Gv12Y+klJlSyh5SSpuTsa2VMX2mApC937l8eunhrBsL/TtEsut4KcWnnctwPRQzgmIZhN82525oBp2BaT2nsf3kdn7P+73hARK7d7v24QEkRgSSdaB5zSpahUcPkDIC0s6B35+HilK1rfE6OoZ1ZM64OVSbqpm2eFrLehS3Ybw4NcH76dKxF/FVkn2nnXuslOa4sydDNwD9UyKREjYcdE48a4Qf39UMxi/nB6h0rjzzRWkX0S6oHe9tfc/GXvsePUC/jpGsb4bQe0UrQWcY+ziUFyiljDUakB6Rzlvj3uJU5SluXXIrRWeK1Dap1aEJfQtJMYWzx3DKqT6y1eaFRJ7Wot5JEeh1gnVOiqdJShbWjEBUlcFO59ooGvVG/tL9L6w5uoatBfUyKBqZjAUY0DGSI8VnyCtyLn7tlbVuGiOxP3S7CFa+AuXNXxHsy3SL6sYr575C3qk8bl96O+VV5Wqb1KrQhL6FpId0p9CgY/0Ox5tumGozSTwrRsH+BrrHhzot9ABrZFdMYYmwyfn+p1O6TCHEGML7W96vt6dxj75/x0gA529MtIKsm/qc86gSuvnjJbUt8VoGtB/Ac6OeY1vBNu5ddi9VTiQHtHU0oW8hgzpfAMDKbQsc/oxaMXqAAR2jyD5U5FQZYClBoqOq+2WwZymUOVcvPsQvhCldprD4wGLyTlll5kpTox59t/ahBPnpnQ7ftJrJWGviMqDnVFj9JpRqcWh7nNPhHGYOncnKwyt59PdHnWt004bRhL6FDO99EcEmE7uKNjr8mbqsG8+LUb+OkZRX1rDjqOMTf5bqlabMKWCqhu0OFCyrx7Xdr0Ug+HCbVWeuJkI3Br2OPskRzcq8aXUePcCYGWCqghX/U9sSr+bSzpdyX//7+GH/D/xnzX9a3GO4LaAJfQvx8/MnrcqfAzju5VoWTHl6MhaaFw6xfI9kXCZEpcG2hU6ft31weyamTeSr3V9RXFFsGZmmwlf9O0ay/UgpZRWOlD222NsKPXpQCp71vQ6y3lXq1mvY5aYeN3FT5k18suMTZm+arbY5Xo8m9C4g1diRA0bJ4ROO5fnK2tii58UoMSKQ+PAAp9IWLVMKQugg4xLY91uzJg1vyLyB09Wn+XynOc7fhEcPyhNIjUmy8VCRw+dpNQumbDHqAeX19xdVNaM1cG//e5mcPpnXs1/n691fq22OV6MJvQvITBiBFIJf133m0PG1Hr1KXqezaYt1jUeAjMkga5rV+7RLZBeGJwxn3o555om0pj36fsnNeAJpbemV1oQnQd9rYcOHUHJYbWu8GiEEM4fNZHjCcJ768ylWHl6ptkleiyb0LmDsgKvQScmWI479odVNIKkjRgM6RpJXdJojxY6lLZ4VAo3vrRTjakb4BpRYff7pfJYcWNJoCQQL4UFGusSFsM6J3P9Wl15ZnxH3KoXO/nhZbUu8HqPOyKzRs0iLSOO+Zfexq3CX2iZ5JZrQu4C46EQ6VAkOVDkWV5XmaoU6tTz6DoqXvOFgkVOf0wmhuPUZk2HvMjjtfJrm8MThdAzryLwd8xwK3VjsXX+g0OFJt1YdugGITIHeV8G6d+HUcbWt8XpC/EJ4bexrBBuCuf3n2zlerl2z+mhC7yI6EMUBQ4VDC6eqTZYyxeqIUbf4UPz0OjbmFjl0vCXvv1aTMy5RskOcXDwFSrGzq7pexcYTG9lqKsORp5reyRGUnKnmQIFji2S8upWgo4z8B9RUKouoNJqkfXB7XjvvNUorS7lj6R2UVTm3gtvX0YTeRXQM6UqJ3tGFU+bQjUoevb9BT7f4UDbnFjd9MNBgeVdiPwhPhq0LmnX+yZ0mE2gIZF7VcYeuQc/EcAA25Tlqbyv36EHJwOlxuVLsrKz5jdLbEt2iujFr9Cx2F+7m/uX3O9igvm2gCb2L6J0yBoC1Dni5dbVu1BOjnonhbM4rtlqla5/a8vkWURYCul+shG8qTjl97lC/UCalT+JHUxGO5O50bR+Kn0HHZgefQKSU3t1hylFG3g9V5bDqdbUtaTWMTBrJI4Mf4fe83/lflrYewYIPfBu8gxG9L8bPJMk52fTCKctkrJpi1DspgtIz1ewvaPoR15J1o7O+L3WdCDUVzW6Fd023a6hE8qVf016XUa8jIz6MjQ4/gfiARw/Qrht0vwjWzmnWDbWtckXXK7iu+3V8tP0jLe3SjCb0LiI4KJSOVTpyTceaPLa2w5SaHn2SEg7Z7EA4pIFHD9BhKARENCtOD5AWkcYQEcJnhiqHHrF7JYWzNa/YoY5TrXbBlC2G3QNnipR0Sw2H+ceAfzA0fihPrXqKDcc3qG2O6mhC70KSRAz7jZVUVzdebKnGLGxqilHndiEEGHVsPOSI0NsQV70BuoyHXT82u+fpNfoojukkyw4ta/LYXkkRlFXWsC+/ac9W4lh2TqsgeaByU/3zdajRYs6OYtAZeG70cyQEJ/D3X//OkVNH1DZJVTShdyEpYRmU63Ss2rKk8QNVLGpmwaDXkZkQzua8oiaPtdsfpOsFcPokHFrTLBtGEUKcSfDF7i+aPLaX+QnEoRuTr4RuLAy7G4oPghOF8zQg3D+cV859hYqaCu759R5OV7fddo2a0LuQfuljAVi3+6dGj1OzqJk1PRPD2ZJX0mQlS7vZiuljQWeEnd836/x6AZfW+LEyb2WTHld6bAhBfnoHQ00+FLoB6DIBojvDypfrrV7TaIq0iDSeHfUsO07u4Ik/nmizBdA0oXchw3pNJMAk2Ve0tdHj1CxqZk3v5HBOV9Ww50TjE7ISaXtxV0AYpI5sdpweKbm0xh+Ar3ManzTT6wQ9EsLZ5EDmjWKvD/1p63Qw7E44slGpM6ThFKOSRvH3/n/nx/0/MndL2+zN60PfBvXx8/MnpcpArjzR6HGWRVVqe509EyMAmhRPU2MLWLtOhILdkL/beQOkiQRhZFjCML7O+ZqaJmL9PZPC2Xq4hKqmaunLVlqmuDF6XQXBsYpXr+E0N2XexAUpF/DKhldYfWS12uZ4HE3oXUySrh37jTWcqbC/ilNiETR1xSgtJpgQfwObmkhblI0JZ1el8UrzvHqlqNllnS/jaNnRJotS9UoKp6LaxO5jjU/ImvDBZhTGABh0K+T8DMe3q21Nq0MIwZPDniQlLIUHf3uQY2VNZ8f5EprQu5i0iB5U6AQrN9qPW9cumFLZo9fpBD0Sw5pccSpppKRAeBK0y1AEyFnMtW7OST6HqIAovtz9ZaOH90qKAGhyAtnnYvQWBtwMen9Y87balrRKgoxBvHDOC5ypPsM/lv+jTbUi1ITexfTvfD4A2fuW2T3G4nHqvODy90wMZ/uRJiZkZb3FUvXpNBYO/gmVztYXUTx6o97IpPRJLD+0nPzT9hu4dIwKItTfwJa8kiZG9bGsGwvB0UpZhI2fwhnHFo9pnE1aeBr/HP5PNp7YyPPrnlfbHI+hvtL4GAO6n4u/SXKo1H65VEvPWLVq3VjTPT6MymoT+/Lti7Qix43Ymj5WKcC1/3fnTm5VvfKyzpdRLatZmGO//LFOJ+gWH8r2I40LPU3Z25oZdAtUlUH2J2pb0mqZkDKhduXsj/t+VNscj6AJvYvx8/OnQ5WOIyb7E7I1Jkt6pfqXv3t8GADbGhFPk0k2fk/qMBSMQZCz1LmTW9WjTw1PpV+7fizIWdBoClz3+DB2HC1ttEaPz4ZuQCkolzgA1r4NJh+ci/AQ9w24j77t+jJz5UwOlDjWGa41o77S+CDxIpJDhkr7JYu9JI8elPx0o16w/Yj9ZuFN9oEyBkDKiGbE6c8eeXKnyewv2c+W/C12P9E9PoxTFdXkFtpf/OKzoRsLg6ZDQQ40Eh7UaByjzsizo57FqDfy4G8P+ny8XhN6N5AUlEqJXse2fVk290svWBlrwc+gIz02pNFwiBJhacLWTufByT1wcp/jJ6/XeOT8jufjp/Pjm73f2P2IQ08g0uQVN1G3kXkJBMVok7ItpH1we54a9hTbCrbx0vqX1DbHrWhC7wa6xA8CIGvnYpv7LQumvGVRT0Z8WONCTxOhG1Di9AB7nAzfWN3sQv1CGZM8hh/3/UiVybaH1TUuFCFo2l4vuIm6DYM/9L9RSWkt9P2wgzs5t8O5XNX1Kt7f9j4rch3pJdE68Q6l8TGG9rwQgD0nNtncb7IUNfMSMeoeH8bx0goKTlXY3O9Qw6bodIjoADlOlC220Urw4vSLKawoZGWe7Zz6QD89qdHBjU/IOvIE0toZcJMyv5HVNld6upJ/DPgHnSM789gfjzWa9dWa0YTeDSTEdiSuysThylyb+2Vd3V8PWmUfSzhkx1HbcXqHJjeFUMI3+5ZDdaWDZ5YNmoMPTxxOhH9Ek+Gb7UfbsEcPyvqFrhdA9jzw8fiyuwkwBPDcqOcoryrn4RUP19ai8iU0oXcTCTVBHNbZFs7aWjdeIvTd4kMB++EQu9Ur65N2DlSegsMO1v+28YUy6oxMSJnAskPLKK20ff26x4dy6ORpSs/YFjifKlPcGP2uh7ITSqlojRaRHpHOQ4MeYtWRVXy07SO1zXE5mtC7iURjPIcNUFTa8FGwbjJW72mzbBIT4k9sqL/dCU6He22njFBe9ztYeMtG6AaU8E1FTQU/H7CdxePIE4i3zH+4lfSxEBoP6z9Q2xKf4PLOlzMmaQwvrX+JvUV71TbHpbSBb4M6dIjojkkI1m5tODkpvSi90kLXuFByjtuuIWO3emV9gqIgrocTC6dsJ272jOlJYkgiPx2wXe65S5zyBGKv5k2bCN2A0vylz7VKWmvJYbWtafUIIZg5bCZBxiAe/f1Rn2ourgm9m+jeYQgAO3IbVsrzhp6x9enULoSc46dsLkRqtHplfVJGwMHVjsXp7Xj0QgjGdRzH6sOrKa5ouNQ/MSKQQKOe3cdbMKfgK/S9TgmBZX+stiU+QUxgDI8NeYwtBVuYu9l3JrodUhohxAQhxE4hRI4QYoaN/f5CiM/M+1cLIVLq7e8ghDglhLjfRXZ7Pf27n4NeSnJLGpbvrS2B4EVeZ+e4EMorazhc3HAhkjKl4KCtKSOg+jQcXu/AwfaXYo1PGU+1rOaXgw2zeHQ6UXtjsj1qG/HoAaJSIXUUrP9QWynrIsanjOeClAuYvXE2O07uUNscl9Ck0Ash9MBrwAVABnC1ECKj3mF/BQqllJ2AF4D/1tv/PNDM7hStk9DgCOKr4VhVw3Ko0ssmYwE6tzOHQ2yKpwN59BY6DgcE7HcgJ9mqBEJ9MqIzSAxJZPEB22sROrcLsR+6cXhSwUfoez0UHYADTtYa0rDLI4MfISIggkd+f4TKGkezyLwXRzz6QUCOlHKvlLIS+BSYXO+YycD75vdfAGOFWcWEEJcA+4DG2y75IHE1QRzTN6xLL83VK/VeFLrp3C4EgBwb4imbql5pjTNxejuhG6gL36w6vMpm+KZTXAhHS85QYiPzpk159ADdLwK/UNj0mdqW+AwRARH8c9g/2V24m9kbZ6ttTotxRGkSgUNWv+eat9k8RkpZDRQD0UKIEOAh4J+NnUAIMV0IkSWEyDpxovHuTK2JOGMcRwxQVn52LLm2k5IXefSRwX7EhPjZDIeYpJPCWRunt70Aq47Gq+iMSxlnN3xjeQLZY8Nen2sl2BTGQMiYDNsWQVXbbYDtakYljWJy+mTe3fIuO0/uVNucFuHub8OTwAtSykZbAkkp35JSDpBSDoiNjXWzSZ4jKawzNUKQtf1soZJe0jO2Pp3ahdic4GzE8baNJU6f10ScvomBM6Mz7YZvLE8gNkNNvthKsCl6TYWKEi2n3sXcP+B+wvzD+Oef/2yy1aU344jS5AHJVr8nmbfZPEYIYQDCgQJgMPCsEGI/8HfgESHEnS0zufXQLWkwANsOrjp7hxemV4LiJe8+fqpBmeAmq1fWp+Mw5RNNhm8aH1kIwfkdz2fV4VWUVJ6d458cFYSfQWf7CcQXWwk2RcpIJad+0+dqW+JTRARE8ODAB9mcv5lPd36qtjnNxhGhXwt0FkKkCiH8gKuARfWOWQTcYH4/BfhFKoyUUqZIKVOAF4F/SSlfdY3p3s/AzLEIKTlUdPbMvTdOxoLi0ZeeqeZ46dkhF4eqV1pjidM3NTkoG5ZAqM+5Hc6lWlY3qH2j1wnSYoLZfczWE0gbSq+0oNNDzymwewmUn1TbGp9iYupEhicO5+X1L3O07Kja5jSLJoXeHHO/E/gJ2A58LqXcKoR4SggxyXzYXJSYfA5wH9AgBbMtEhEaQ/tqOFZ55KztliX6SkKT91AbDqk3IetQ9cr6dBgMuVnQ2OOuAzGhXjG9iPSPZFnusob2xoXaDN20uclYC72uBFMVbP1abUt8CiEEjw1+DInk6VVPN9oYx1txKEgspfxeStlFSpkupXzGvO0JKeUi8/szUsqpUspOUspBUsoG64ellE9KKWe51nzvJ64mgGO6s9v0WWJ9Dq029SCdLJk39eL0TsfoAZIGKXVvjm9v5KCmg0J6nZ6RSSNZkbuiwUrFTrEh5Bae5nRlw5tJm/PoQXmKiu2uhW/cQFJoEnf0uYPlucv59dCvapvjNN41G+iDxOijOGqQZ3WbqvMIvEuMYkP9CfbTs7/g7JRQ6WzWDUCyUpOf3DX2j3HwDjI6aTQllSVkH88+a3tKTBAAB0/W2Wu5tt7QeN3jCKGEbw6tguL602gaLeWa7teQHp7Os2uf5Uz1GbXNcYo2+G3wLHGBSVToBDsPZNdu88ZaN6DY0zE6mAMFZz+BOFy90prIFAiOhUONCL2DVSaHJQzDoDOwPHf5WdtTY4IB2G9lb23lSu+6tJ4jw7zEZce36trhgxh1Rh4e/DB5p/J4b+t7apvjFJrQu5nk6G4AbN1Xl3lTt2DKu2L0oHjJBxp49M0IMwmhhG8aE3oHJmMBQvxCGBg3sIHQd4xShN76xmTyojaNqhDTWQnfbKufL6HhCgbHD2Zcx3HM2TyHw6daTyE5TejdTFdzCONgft3C4LrGI953+TtGB3OosJzqmroURafTKy0kD1L6yJbZ6dojTQ4/KoxOHs2+4n0cLDlYuy08yEhkkPGsUFPtRHdbFXqAjElwcCWcOq62JT7J/QPuRyCYldV6phy9T2l8jB6dBmGQkiOn6np7mrw0dAOQEh1EVY3kcFFdDNLU3NoxtXH6tXYOcPwWMiZ5DADLDi07a3vH6GD251uFmmqbd3nftfUY3ScpN1EtfOMW4kPiuaXXLSw5sIQ/D/+ptjkOoQm9mwnwD6JdNRTUWHu13lem2ELH6IZxb5pbIyyhLwi9faF3Ip0nMSSRThGdWJF3drG0lOizQ02aRw/EZUJUmha+cSM3ZN5AUkgSs7JmtYoVs96nND5IbE0A+aJhZog3ipFlgvNAvQnOZnnIxkBo1x0OZ9s5wLmg0NCEoaw/tv6sjIeUmGAOF5/mTFVNra3Qxj16IZRJ2f0rtMVTbsJf7889/e5hV+Euvtv3ndrmNIkm9B4gRhfBMYOpNsWyVuh13jcZ2y7UnwCj7uy4tzPVK+uT0AeOZFuK2p+Nkwn6wxKGUWmqZP2xuho6KdHBSAm5heXmIb33JupRuk8CUzXsbFPVwT3KuJRxZEZn8sqGV6ioaaqAn7poQu8BYgOTKNfpyMndDIBJKoLvjV6nEIKUenFvp6tXWhPfB8oLoDi34b5G6tHbon9cf4w6IysP15VD6Bit5NLvyy832+q98x8eJaEvhCXBzu/VtsRn0Qkd9/W/j6NlR5m3fZ7a5jSKJvQeIDmqCwBb9yoplt7udXaMDjo7N705K2MtJPRVXg9vsLHTudBNoCGQfu368eeRugmwlOiGoSbw3mvrMYSALuNgz68OlIvWaC6D4gcxInEEb29+22bfBG9BE3oP0M2cfbLv+BagLo9ep/POy58SE8yhk6epMfePbVFlj7hM0BmU8E19mnEHGZIwhF2Fu8g/rUxuRwb7ER5orL0xaZOxVnQeD1VlcOAPtS3xaf7e7++cqjzFnM1z1DbFLt6pND5Gj85D0UvJsTIlxdISrvbGrBtQvOTKGhNHzP1jm7VgyoIxUFnAY3NC1vkM/WEJwwDOSmuzzrzx1sqgqpA6CgwBsOsntS3xabpGdeXi9Iv5ePvHXlvd0juVxscICggmplpysroAsBYj75uMBUiOVOLeuYWWbkXNqF5pjb0J2WbUVuga2ZVQv1CyjmXVbkuKCqq1VfPorfALUsR+10+2J8M1XMbtfW5HSsnczXPVNsUmmtB7iCiTkUKU8II3T8YCJEYGAnVCb2pJjB4UoS8vgOJD9XY4VgLBGr1OT/92/ck6aiX0EYHkFZ7GZJJaemV9Oo+Dwn2Qv1ttS3yaxJBEJneazJe7v/RKr14Teg8RSQgn9ZYyu97tdcaHBwCQZ/GSW5J1A9C+t/J6rF5/eGmiOUuxBrQfwMHSgxwvV5b4J0YGUlljIv9UhddPdHuczucrr3sa9t3VcC239LoFKSXvbHlHbVMaoAm9h4gyRpOvF5SVlyolBQDhpTH6AKOedqH+dbnptNCjj+2qvDYQ+uYtuR0QNwCAdcfWAZBkeQIpOq3F6OsTmQKRqbB3mdqW+Dy1Xv2uLzlWdkxtc87CO5XGB4kNSkQKwfZ967CUQNB74YIpC4mRgeQV1U3Gtkg4A8IgvIONJiTNK5fWNaorwcbg2vBNYkTdnIJsWY6Qb5I2RunfW1OltiU+z7Se0zBJk9d59ZrQe4iEyM4A5ORtQJq8f1FPUmSQVYxetjwQEpcBx7edva2ZCfoGnYG+7frWTsha5hTyrIReC91YkX4OVJZC3jq1LfF5kkKTuDDtQr7O+ZqiM0Vqm1OLJvQeIj2xFwCHC3djKQDsraEbgMSIQI4U1+XSt/ie1K475O+C6kqrjc0ugMyAuAHsLd5LwekCQvwNRAQZyS0sr+sw5cXX1uOkjASEFr7xEDdm3sjp6tN8uvNTtU2pRfs2eIhuKf0BKCg/Yp6E9HahD6CqRlJwqqK5ofSzaZeh1F4pyKnb5mQJBGv6tlNW3G7OV8pKJIQHcrT4jObR2yIoSlmhvKf19TptjXSK7MTIxJF8suMTr6mB471K42OEBkcQUWOiqKqwVoz0XroyFiAuTMm8OVqiiGeLG5m36668WodvWlBboXt0d/RCz6YTmwBoHx6g2KpNxtombYxSLrrilNqWtAlu6nETJ8+cZNEe7ygV7b1K44NE1egolqes+pp672RsfLgS9z5SfKZltW4sxHRR/r1nTcg2P3QTaAikS2QXNuVbCb2VR69Rj47DQdY00gRGw5UMiBtAZnQmH2z9oLbQnppoQu9BwmUAxboKTOZGBS32kt1Ie3Mu/bGSMy2rXmnB4A/RnVzm0QP0iu3Flvwt1JhqiA8LoKCskjNVyloFzaOvR/IgJUx2cFXTx2q0GCEEN2beyP6S/Sw/tLzpD7gZTeg9SJgIoVBvql2O7s1iFB3sh1Evaj16l4S823Wvl0vfsoF7xvSkrKqMfcX7iDPfmPJLlZioFqOvR0AYxPVQeslqeITzOp5Hu6B2fLbrM7VN0YTek0QYoyjW66ioUdIWvXkyVqcTtAu1hENco/PEdoOig1BlrqEjnS+BYE2vWCWTaXP+5trVvMc1obdPx2GQm6Xl03sIg87AlM5TWJm3kkOl9ct/eBbvVRofJCYoAYCC00otDG9eMAVKKYSjxWegJdUrrYnpDEg4uVf5vYWhm45hHQn1C2XjiY21Qn+sVLmJaOmVNugwFKrK4chGtS1pM1zW+TJ0Qsf8XfNVtUP7NniQduEpABRWKVUsBd4t9HHhARwpPq3E6F3hIEd3Ul5rUyxb9qygEzp6xfRic/7m2iyhYyWnm/hUG6bDUOX1gBa+8RRxwXGMSR7Dgt0LqKypbPoDbkITeg+SHKusji2VSg0Zb56MBYgLDeBEaUXLa91YiE5XXi1C74J0nozoDPYW7cXfKAny01NQpnyZvHn+QzVC4yAqDQ7+2fSxGi7jiq5XUFhRyOIDi1WzQRN6D9KhfTcASnXmO7sX59EDxIT6UVZZQ3lljWti3v6hEBoP+a7x6AG6RHahWlazt3gvMSH+FJZrMfpGSRoEeeubPk7DZQyJH0KH0A58vvNz1WzwbqXxMRLbpWKQkiKduZWgl3udMSH+AJworXCNRw9K+KbAXBvdBR59F3M/3p0ndxIT4sfJMrPQe/m1VY2EvnDqKJQcUduSNoNO6Li8y+VsOL6BAyUH1LFBlbO2UQwGIxE1kkKDctl1OoPKFjVObKgi9PmnKlwnnFFpULhfeS9NLRb6DqEd8Nf7s7NwJ7Gh/py0hG40j942jTZr13AXF6VdhE7o+GbPN6qcXxN6DxNmqrvk3u7Rx5o9enBReiUo9dHLTpiX4rc8dGPQGegU0YldhbvMoRstRt8o7XsqKa2a0HuUdkHtGBI/hG/3fqvKSllN6D1MqMmv9r239oy1EGMt9K7SzcgU5bXogEtCN6DUp991chfRwX4UndY8+kbxCzI3a9eE3tNclHYReafy2HDc89feIaEXQkwQQuwUQuQIIWbY2O8vhPjMvH+1ECLFvP18IcQ6IcRm8+u5Lra/1REigmrfe7tHHx1idVNy1aCRHZXXwv24wqMHZUK2sKKQwKAyhFa9smkS+ipCrzUM9yhjO4wl0BCoSvimSaEXitv5GnABkAFcLYTIqHfYX4FCKWUn4AXgv+bt+cDFUsqewA3Ah64yvLUSqg+rfa/Xe7dHb9TriAwyAi68KUWmKq+F+13m0XeJVCZkK0Re7TYtdNMICX2gPB+Kc9W2pE0RZAzi3A7nsvTgUqpN1U1/wIU44tEPAnKklHullJXAp8DkesdMBt43v/8CGCuEEFLKDVLKw+btW4FAIYQ/bZhwvyir37w/cmYJ37hMNwMjwT/MLPTNr0dvjUXoS0wH8PbG615BQj/lVQvfeJzzO5xPUUVRbXc0T+HItywRsC7UkGveZvMYKWU1UAxE1zvmcmC9lLJBJX4hxHQhRJYQIuvEiROO2t4qiQiKq33fGpbp18XpXSScQkBERyi0iHLLxw33D6d9cHtOVOy3Ok+Lh/Vd4jKVktFHstW2pM0xPHE4gYZAluxf4tHzekRphBCZKOGcW23tl1K+JaUcIKUcEBsb6wmTVCMyuF3te2+P0QNEBiuhG5eaGp4EJYddFroBSA1L5Wj5IRDmVoKt4GlJNYwB5pLR9Zu1a7ibAEMAo5JG8fPBn6kxlyv3BI58G/KAZKvfk8zbbB4jhDAA4UCB+fck4GvgeinlnpYa3NqJCo2vfS+8vKgZQHigMiHr0ltSWAKU5OIqjx4gJTyFg6X7EcL7S0B7BbaatWt4hPM6nsfJMydZf9xzK5QdEfq1QGchRKoQwg+4CqjfH2sRymQrwBTgFymlFEJEAN8BM6SUf7jI5lZNbETdPdPQCoQ+IsgdHn0inC7ElQOnhKVQXl1OSJBSR0iL0TdBuwxlnkRrLehxRiWOwqgzerQhSZNCb4653wn8BGwHPpdSbhVCPCWEmGQ+bC4QLYTIAe4DLCmYdwKdgCeEENnmn3a0YdrHptT94uV59AARgYrQn65y4SKPsKS69y6ap0gNV7J5AoLylWE1oW+cdubEuRM71bWjDRJkDKJ/XH9+z/vdY+d0aA2+lPJ74Pt6256wen8GmGrjc08DT7fQRp+ifVTdPLbw8qJmAOFmoS857cJmFWEJVr+4KEZvFnp9QL4rh/VdrJu1J/VX15Y2yIjEEczKmsWRU0eID4lv+gMtxPuVxscwGIy171vDZKwldONSoQ+3Stpy0TWIC4rDX++PNChZW5pH3wQRHUFntOoNoOFJRiSOAOD3w57x6jWhVxG9lxc1g7rJ2JIzLhT6UNd79EIIEkISqBaKR98aUldVRW9QCsxpQq8KaeFptA9uzx95npm61L4NKqLTeb/XafHoi13p0RsDlEVT4NIQS2JIImfQYvQOE91JE3qVEEIwInEEq46sosoDPXw1oVeR1uB1WoS+qsbFdVGCzOvpXHgNEkPqQkImrYxL00SnK/17PZjPrVHH8IThlFWVsTl/s9vP5f1K48PoWkXWjV/TBzUHi9C70PNOCqnL5jlTpYlXk8R0hppKKDqotiVtkv5xyiS4J8ohaEKvIqIVhG4CjG76EwmOUV5dOCGdGFrn0ZdVaELfJFHmHr4n96prRxslMiCSThGdWHdsndvPpQm9CkTWKDnpohWEbty2wjQoxnIGlw1pHbopr/R8c4dWh2XxXvGhxo/TcBsD4gaw4fgGqkzujdN7v9L4IP8b/DJX6fsTHhLV9MFewMtX9+Wr24e5dtBgc+jGheVa2we3r30fZPT+jCbVCU1QFu0VaUKvFv3i+nG6+jQ5he6dFNe+DSowMHMsAzPHqm2Gw0zqndD0Qc5i8ejLT7psyAj/iNr3CZGBLhvXZ9EbICxRi9GrSM+YngBszt9M9+jubjuP5tFrqIMlRl9e4LIhrbOYtPRKB4lI1kI3KpIYkkhUQBQbT2x063k0oddQB0vWjQuF3hqteqWDRHTQQjcqIoSgV0wvNp3Y5NbzaEKvoQ61Qp/v0mHD/JSFWJpH7yDhyVB6GDywaEfDNhnRGRwoOUB5VbnbzqEJvYY6RKUpr90vdumwMYFKSEgTegeJSFZaOpbUbzGh4Sm6RHVBIskpct+ErDYZq6EOQVHw6FEwBLh02JjAGPYW7/V48+VWS/eLIWXk2aWjNTxK18iuAOws3Emv2F5uOYcm9BrqYXR9Zkx0oBISKjjjnti/zxEYqfxoqEZCSALBxmB2ndzltnNooRsNnyIzOhMAo87YxJEaGt6BTujoEtmFXYXuE3rNo9fwKf6S8ReSQpM4N/lctU3R0HCYlLAUVuStcNv4mkev4VPohI6xHcZq6ZUarYoOYR3IP53vtswbTeg1NDQ0VCY5VKk7dKjUPWsaNKHX0NDQUJkOoR0AOFjqnnIUmtBraGhoqIzFoz9Yogm9hoaGhk8S4hdCVECU20I3WtaNhoaGhhcwMXUiCSFuqBSLJvQaGhoaXsFDgx5y29ha6EZDQ0PDx9GEXkNDQ8PH0YReQ0NDw8fRhF5DQ0PDx9GEXkNDQ8PH0YReQ0NDw8fRhF5DQ0PDx9GEXkNDQ8PHEVJKtW04CyHECeBAC4aIAVzbcdp9tCZboXXZ25psBc1ed9KabIXm29tRShlra4fXCX1LEUJkSSkHqG2HI7QmW6F12duabAXNXnfSmmwF99irhW40NDQ0fBxN6DU0NDR8HF8U+rfUNsAJWpOt0LrsbU22gmavO2lNtoIb7PW5GL2GhoaGxtn4okevoaGhoWGFJvQaGhoaPo7PCL0QYoIQYqcQIkcIMUNte2whhNgvhNgshMgWQmSZt0UJIZYIIXabXyNVsu0dIcRxIcQWq202bRMKL5uv9SYhRD8vsfdJIUSe+fpmCyEmWu172GzvTiHEeA/bmiyE+FUIsU0IsVUIcY95u1de30bs9dbrGyCEWCOE2Gi295/m7alCiNVmuz4TQviZt/ubf88x70/xAlvfE0Lss7q2fczbXfO3IKVs9T+AHtgDpAF+wEYgQ227bNi5H4ipt+1ZYIb5/QzgvyrZNgroB2xpyjZgIvADIIAhwGovsfdJ4H4bx2aY/yb8gVTz34reg7bGA/3M70OBXWabvPL6NmKvt15fAYSY3xuB1ebr9jlwlXn7bOA28/vbgdnm91cBn3mBre8BU2wc75K/BV/x6AcBOVLKvVLKSuBTYLLKNjnKZOB98/v3gUvUMEJK+Rtwst5me7ZNBj6QCquACCFEvEcMNWPHXntMBj6VUlZIKfcBOSh/Mx5BSnlESrne/L4U2A4k4qXXtxF77aH29ZVSylPmX43mHwmcC3xh3l7/+lqu+xfAWCGEUNlWe7jkb8FXhD4RsG6fnkvjf5hqIYHFQoh1Qojp5m1xUsoj5vdHgTh1TLOJPdu8+XrfaX7EfccqDOY19prDBH1RPDmvv7717AUvvb5CCL0QIhs4DixBeaooklJW27Cp1l7z/mIgWi1bpZSWa/uM+dq+IITwr2+rmWZdW18R+tbCCCllP+AC4A4hxCjrnVJ5VvPKfFdvts2KN4B0oA9wBPifqtbUQwgRAnwJ/F1KWWK9zxuvrw17vfb6SilrpJR9gCSUp4lu6lpkn/q2CiF6AA+j2DwQiAJc2incV4Q+D0i2+j3JvM2rkFLmmV+PA1+j/EEeszyKmV+Pq2dhA+zZ5pXXW0p5zPwlMgFvUxc+UN1eIYQRRTQ/llJ+Zd7stdfXlr3efH0tSCmLgF+BoShhDoMNm2rtNe8PBwo8a+lZtk4wh8uklLICeBcXX1tfEfq1QGfzLLsfygTLIpVtOgshRLAQItTyHhgHbEGx8wbzYTcAC9Wx0Cb2bFsEXG/OCBgCFFuFIFSjXuzyUpTrC4q9V5mzLVKBzsAaD9olgLnAdinl81a7vPL62rPXi69vrBAiwvw+EDgfZV7hV2CK+bD619dy3acAv5ifqNSydYfVDV+gzCVYX9uW/y14arbZ3T8os9O7UGJzj6ptjw370lAyEzYCWy02osQGlwK7gZ+BKJXs+wTlcbwKJQ74V3u2oWQAvGa+1puBAV5i74dmezaZvyDxVsc/arZ3J3CBh20dgRKW2QRkm38meuv1bcReb72+vYANZru2AE+Yt6eh3HBygPmAv3l7gPn3HPP+NC+w9Rfztd0CfERdZo5L/ha0EggaGhoaPo6vhG40NDQ0NOygCb2GhoaGj6MJvYaGhoaPowm9hoaGho+jCb2GhoaGj6MJvYaGhoaPowm9hoaGho/z/1ENwpCHFfpdAAAAAElFTkSuQmCC", + "text/plain": [ + "
    " + ] + }, + "metadata": { + "needs_background": "light" + }, + "output_type": "display_data" + } + ], + "source": [ + "num_epoch = 50\n", + "cycle_limit=3\n", + "scheduler = PolyLRScheduler(optimizer, t_initial=t_initial, cycle_limit=cycle_limit)\n", + "plot_lr(scheduler, label='default, 1')\n", + "scheduler = PolyLRScheduler(optimizer, t_initial=t_initial, cycle_limit=cycle_limit, cycle_mul=1.5)\n", + "plot_lr(scheduler, label=\"cycle_mul=1.5\")\n", + "scheduler = PolyLRScheduler(optimizer, t_initial=t_initial, cycle_limit=cycle_limit, cycle_mul=2)\n", + "plot_lr(scheduler, label=\"cycle_mul=2\")\n", + "plt.legend();" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Warmup Args." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### `warmup_t` " + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Defines the number of warmup epochs. " + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### `warmup_lr_init` " + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "The initial learning rate during warmup. Default is 0." + ] + }, + { + "cell_type": "code", + "execution_count": 19, + "metadata": {}, + "outputs": [ + { + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXoAAAD4CAYAAADiry33AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/YYfK9AAAACXBIWXMAAAsTAAALEwEAmpwYAAA6kklEQVR4nO3deVhV1frA8e8CDiA4K5oDivOAIiiKE+aQOWSaOc/kQFlmZWXaLTVv3uqXpXWtDGdNc8yhcsgccRbUHBs0LcHZREVlOLB+f5wDFxDlqMCBfd7P8/icc/b4boSXxdprv0tprRFCCGFcTvYOQAghRM6SRC+EEAYniV4IIQxOEr0QQhicJHohhDA4F3sHkFHJkiW1j4+PvcMQQoh8JTIy8orW2iuzdXku0fv4+BAREWHvMIQQIl9RSv11r3XSdSOEEAYniV4IIQxOEr0QQhicJHohhDA4SfRCCGFwNiV6pVR7pdRvSqmTSqkxmaxvoZQ6oJQyK6W6Z1g3SCn1h/XfoOwKXAghhG2yTPRKKWfgC6ADUBvoo5SqnWGzv4EQYFGGfYsD44EgoBEwXilV7NHDFkIIYStbWvSNgJNa6z+11gnAYqBL2g201me01oeB5Az7tgM2aq3/0VpfAzYC7bMh7rvcijfz7x+Os+/0PyQlS+llIYRIYcsDU+WAs2k+R2Fpodsis33LZdxIKRUKhAJUqFDBxkOnd/z8DRbs+YtZO05TsqArbWuXpp3vYzStUhJXF7kVIYRwXHniyVitdRgQBhAYGPhQzfGGPsU58G5btv52ifVHL7Dm0Dm+3XeWQu4utKlZivZ1HuPx6qUo4OqcrbELIUReZ0uijwa803wub11mi2igZYZ9t9q47wMr6OZCJ7+ydPIrS1xiEjtPXmH90QtsPHGRVYfO4W5yomX1UnSo+xita5aikLspp0IRQog8w5ZEvx+oppSqhCVx9wb62nj8DcB/0tyAfRIY+8BRPgR3kzNtapWmTa3SmJOS2Xv6H9YfvcCGYxdYf+wCrs5OdKywnwpFL9Mt+GUqlq2eG2EJIUSuU7bMGauU6ghMBZyB2VrrSUqpiUCE1nqNUqohsBIoBsQBF7TWvtZ9BwNvWw81SWs9537nCgwM1DlZ1Cw5WXPw7DXWHbnArvO9OOsKzlpTO94V/8KBPNtsJFUr1Mmx8wshRE5QSkVqrQMzXZfXJgfP6USfIlkn02hePZomF8Bde/GL/otzJoWT1tRMMOHv6c+zTUdSo1JAjscihBCP6n6JPk/cjLWHy7cvE6+gaZFK9O62hOSkJLYeWMnGowv4RZ1ikTmCb7cNoMZGF/w96tGl8YvUqWrrYCMhhMg7HDbRn73xNwDeBUoB4OTsTOuG3Wnd0PJg746DP7Du0Cx+USdZnHSAxTuHUmOzE/UK1KVL4xH4VWtst9iFEOJBOG6iv/Y7AN6eZTNd3zygE80DOgGw65d1rD04k1/U7yxN/oWlu4ZRfYsT/pL0hRD5gOMm+pg/cdaaMoXuen7rLk3rdaBpvQ4A7DnyEz9GzuAX9etdSb9rk5ele0cIkec4bKKPuvE3j5mTMHmUeKD9Gtd9ksZ1nwQySfrW7p0AD8uN3FqVG+RE6EII8UAcNtGfvXUOb3MiuBd96GOkTfq7D6/nxwMz+EX9zuKkAyzZPoiaP7sQ4BlAt+avUb2iXzZFLoQQD8ZxE/2dyzyZaIYCRbPleE382tPEz1KvLeVG7kH1B4vMESze0pfaCSbqF2pEz8dfl4ezhBC5yiET/Y2EG1xPuoO32QzuRbL9+Glv5G6LXM36X2ZxwOlP5ifsYtFPO/GNdyOwWFN6tnqDsl4Vs/38QgiRlkMm+rM3LQU1vRPNj9R1Y4vHG3Th8QZdSE5K4ud9S9l4fAEHnf9i1p2tzP9xC37xHgR5taJ3mzcoVsQrR2MRQjgmx070SRpcPXPlnE7OzjzZpA9PNulDclISP+ycy5bfl3DQJZrIm2uZ890P+CUUpmnZDvRoPZJCnkVzJS4hhPE5ZKKPuhkFQHmXgqBUrp/fydmZzi2G0LnFEBIS4lm17Wu2n/mOQ6bL7P1nGTOXLMHfXIIWPl15tuWLuLq65XqMQgjjcMhEf/bmWYrjgmcOd9vYwtXVjZ5tR9KTkdyOu8XyzZ+xI+pHIk1XCT8/m+kLZuKvy9Kmeh+eajYIJ2eppy+EeDAOWdRs8IbBJFw4wjcJhWDY5hw918OKuXmFb3/+hL2XNnHY7TaJSlEmUVPfqTKd6r9Ac/+O9g5RCJGHSPXKDNoub0vgjWt8YKoAA77L0XNlh6hLZ1i65WP2X9/FcddEkpWicgI0cKtLt2aj8K2S6f+tEMKBSPXKNBKSErh46yLeiQoKF7V3ODYpX8qHUb2+AOC30wdZtuNTIviFZfoIy8NDqL3JRKMiTejd+i0ZrimEuIvDJfqo2Cg0Gu/4uBwfWpkTalQK4J1KCwDL07hrIqdzwOkkc+LCWfjDduoletL0sfb0eeINPD0K2TlaIURe4HiJ3jrixvv2jRx5WCo3pTyNm5yUxLrdC/jpxDcccDnP/mvfMWfxcgKSvGhTtTddWgyTm7hCODCHS/QpY+jLJ8RnW/kDe3Nyduap5iE81TwkdeROeNQP7DVdYdvfX/DlnGkEOlWmc+CLqWUahBCOw8neAeS2szfPUsDZnRLJyfmy6yYrHu6eDOz4NjNCd7H2mZ8Y6tEaryRX1jr9SejBN3k2zI9J34Tw59lj9g5VCJFLHLJF712gFIrfDdOivxevYmV5pcdnAJz4M5JlOz5hvzrC4qRIlm/qRd34AjQr3Za+bUfLk7hCGJhDJvrK7sUtH/J5H/2DqFW5AeMqLwJgy/4V/PjLTCJc/ubg9e+Zt2Q19ZNK0bZ6P55u/pz05wthMA6V6JN1MtE3o3m8VHnLAgN23diiVcNutGrYjYSEeJZt/oytf69ij+ky2858xpd/TKWhSw16Nn0dv+pN7R2qECIbOFSiv3T7EgnJCXg7e1gWGLzrJiuurm70az+afozmwpWzLNr0IXvid7Ja/c6aXaHU3mqiSbEW9G87lhJFH7N3uEKIh+RQiT51xE3KZTtoiz4zj5X0Tn0o68DxbazYO5X9Tn8w8/ZmFq38mYDE4jxRpTfPtnxBunaEyGccMtF7JytAgVth+waUR9Wv/Tj1az9OclISK7d9zaaTi4k0XWVn1HTC5nxFQ+fq9Gr2hnTtCJFPONTwyrM3z+KiXCiTmADuhcHJoS7/gTk5O9Ot9Yt8GbqdtV03MdSjNcWTTaxx+oP+u0LpE1af/y4fRczNK/YOVQhxHw6V6c7ePEuZgmVwibsh3TYPqETRx3ilx2csDj3IvIbT6KSrcdkpnrBbG+mw7HFGzGjF2h3zSU5KsneoQogMHK7rxruQN1y65vA3Yh9FSteO2ZzIii1fsOn0MsuonVMf8+WJjwlyr0e/lmOp7O1r71CFEDhgi967kDfExUiLPhu4uJjo1fZVwkJ3sqbTjwxwbYKrVixN/oVum3rx3NdBzF/7HxIS4u0dqhAOzWFa9Nfjr3Mz4aYl0d+JAa8a9g7JUMp6VWR0nzAAtkWuZvXBL9lniiLi8rfMXbCQIFWFHk3fpH7NYDtHKoTjcZhEnzq0slB5S4teum5yzOMNuvB4gy7cun2Tb376gO0XN/Cj25/8uGc4vttdCS75BAPavS1lF4TIJTZ13Sil2iulflNKnVRKjclkvZtSaol1/V6llI91uUkpNU8pdUQpdUIpNTab47dZ6tDKQt4Qd126bnKBp0chnn/mPyx8PpJvmobxtPUG7lc319FhSTNGzWzHjkNr7R2mEIaXZYteKeUMfAG0BaKA/UqpNVrr42k2GwJc01pXVUr1Bj4CegE9ADetdV2llAdwXCn1rdb6THZfSFZSW/TuXmCOkxZ9LvOr3hS/6isxmxNZ8vMUfv5rBVvcotn4y1vU3Pc2zYq2IKTDOIoWKmnvUIUwHFta9I2Ak1rrP7XWCcBioEuGbboA86zvlwNtlFIK0ICnUsoFKAAkADeyJfIHdPbmWUoWKImH2Xpj0IEKmuUlLi4m+rUfzZzn97Kk1SK64csNJzOz7myh47LHeWXGE2yNWGnvMIUwFFsSfTngbJrPUdZlmW6jtTYD14ESWJL+LeA88DcwWWv9T8YTKKVClVIRSqmIy5cvP/BF2CJ1xM2dGMsC6bqxu+oV/ZgwaDHrnvuFd8sOo3ZiYcJNF3j52Di6h9Vj6tKR8jCWENkgp4dXNgKSgLJAJeB1pVTljBtprcO01oFa60AvL68cCeR/QyuvWxZI102e4eTsTM+2I5n5/G5WtF1OD1WXWKek1Fb+qzOeIPzAGnuHKUS+ZUuijwa803wub12W6TbWbpoiwFWgL7Bea52otb4E7AQCHzXoBxVnjuPS7Uv/G3ED4F4st8MQNqhUribjBi5ibUor31yYbaYLvHjkX/QM82faije4eSvG3mEKka/Ykuj3A9WUUpWUUq5AbyBj82oNMMj6vjuwWWutsXTXtAZQSnkCjYFfsyPwBxEda/m9lL7rRvro87LUVn7obla0WUI3fIlxMvN17AY6LmnGm7M6EnFsq73DFCJfyDLRW/vcRwAbgBPAUq31MaXURKVUZ+tms4ASSqmTwCggZQjmF0BBpdQxLL8w5mitD2f3RWQl/dDKGMtC6brJNyp7+zJh0GLWhhxkTOkBVEn0ZKPz3wzeP4J+Xzdg9vcT5elbIe7DpgemtNZrgbUZlo1L8z4Oy1DKjPvFZrY8t6VP9BssC6VFn++kjNjpx2iOntzLwvAP2O3yB1P+Wcai+UtoavJlYKtxVK1Qx96hCpGnOMSTsWdvnsXT5Ekxt2KWrhvXguBssndY4hHUqRrEB1VXcTvuFgvWT2LLpXWs5AQ/bO5Ng4TCdKoewtPBQ2SSFCFwkKJmKSNulFLWgmbSmjcKD3dPnn/mPywOPchXfh8QnPgYR0zXeeev/9J1tj+fLnlRhmgKh+cQiT7qZpSl2wYsLXoZQ29IzQM68dmwn/n+mY30NwVhBubEhfPUssd5Y2YHDhzfZu8QhbALwyf6pOQkomOjLUMrQQqaOQCvYmV5q+9Mvh98iHfLDqN6YkF+djlLyL6XGBjWkEUbPpEJUoRDMXyiv3znMonJiZQvmJLopaCZo0gZojnn+b1803wWHZIrcdrlNh9cmEun2fX4aNFQrsZcsHeYQuQ4wyf6c7HnAChbsKxlwZ0YadE7oDpVg/ho8Pf80DOcIQVa4aqd+CZxL52+a8Oome1kTL4wNMMn+pSHpcoVtJbnkZuxDq1IweK82vNzvhtykPcrvoyvuTBbXKIZvH8Eg75uyNKNn0u3jjAcwyf6lBZ9Gc8ykJQICbHSdSNwcnamS8tQZobu5pvguXRIrsQp023+fW4Gz8z2Z+rSl6XUgjAM4yf6W+coWaAk7i7uEGetkCxdNyIN3yqBlm6dHtsIcWtOMjDrzlaeWtKMsXOe4fe/cv1hbiGyleETffTN6P/1z6cWNCtqr3BEHla0UEle7/0VawYf4l9lnqNiYgF+cDpFn819eTGsBdsiV9s7RCEeivETfWw05Tyt/fNS0EzYwMnZmd5PjmLB8xGEBXxMU7MX+0z/MOLoO/T7ugEL1/+f9OOLfMXQiT4pOYkLty5QrlDKjdhrllfpuhE2auLXnv8O28KKdt/xjK5JlEscH15cwNOz/fl0yUvSjy/yBUMn+st3LmPW5jRdN9ZJR6TrRjygimWr8++QZfzQeydDCrTCCZgTt51Oi5sxbl4Pzl74094hCnFPhk70qUMrM3bdSItePKRCnkV5tefnrB58iLGPhVAuyY2V/Eq3dU/z6sy2HPg13N4hCnEXQyf6ux6WSr0ZK3304tE4OTvTt93rLAo9wGe1xuOfWJRtLud5bs9whoY14afd39o7RCFSGTrRR8VGAVCmYBnLgjsx4OwGpgL2C0oYTutG3QkL3cmC5rN4wlyeI6YbvP77f+j7dX0W//Sp3LgVdmfoRH8u9hxeBbxwc3azLIi7Lt02IsfUqRrEJ0PXs+KpH+lKLaJc4pl0fg7Pzgrgq5VjZRYsYTeGT/SppQ/AWv6gqL3CEQ6ifCkfJg5ayve9whng2oQ7Tsl8eeMHOs1vwMffPi8jdUSuM3Sij45N87AUSEEzkauKFCzO6D5hfD8wkhGFO+GR7MT8hF08vbgZExf04/K1c/YOUTgIwyZ6c7KZi7cuZtKilxuxIne5urrxfNcP+G6IZXLz0kmuLEs+zDMr2/L2nK6cPf+HvUMUBmfYRH/5doYx9CCzSwm7cnJ2pl/70SwJPcj7Pq9QxezB904n6bb+Gd6Y2YHf/zpk7xCFQRk20aeMuEmX6OVmrMgjujw+lPmh+/m89gT8Egqz0eUsfTf359UZT3D05F57hycMxrCJPmUMfWrXTXKyzC4l8pxWDbsx8/ndTK8/mcDEYmwzXWDAjiG8FNZS5rgV2cbQiV6hLHXoAeJvAFr66EWe1MSvPdNDw5nV+CuaJ3qx2/UKg/e9xAthwew78rO9wxP5nGETfXRsNF4eXrg6u1oWpDwVK103Ig+rXzOY/w7bwoLgubQ0lyHCdI3QyFd5Pqw5e478ZO/wRD5l2ER/7lbGMfRS0EzkH75VApk6dCMLWy6glbkskaYYno8cRWhYM3YfXm/v8EQ+Y9hEn27CEZCCZiJfqlEpgClDf2JR6294wlyOA6brvHDgDULDmkmXjrCZIRO9OdnMxdsXKeuZdsRNjOVV+uhFPlS9oj+fDN3A4jaLecJcngOm64RGvsoLYcFy01ZkyZCJ/uLtiyTpJMoXKv+/hamzSxW1R0hCZIuqFerwydD1LGr9DS3NZdhvusaQfS/xUlhLDv++y97hiTzKkIn+rvLE8L8+eum6EQZQvaI/U4duZH6LuQQnlmKX6xVCdoUyckZrfjt90N7hiTzGpkSvlGqvlPpNKXVSKTUmk/VuSqkl1vV7lVI+adb5KaV2K6WOKaWOKKXcszH+TN014QhYum6UM7gWzOnTC5FrfKsE8vmwzcxrGkaTxBJsN12i39YBvDmro5RWEKmyTPRKKWfgC6ADUBvoo5SqnWGzIcA1rXVVYArwkXVfF+Ab4AWttS/QEkjMtujvIWUM/WOej/1v4Z0YS/+8Ujl9eiFynV/1pnwxbBthDaYSkFiYDc5/02P9M7wzpxsXr0bbOzxhZ7a06BsBJ7XWf2qtE4DFQJcM23QB5lnfLwfaKKUU8CRwWGv9C4DW+qrWOsdnYYiOjaaURylMzqb/LYyLkW4bYXiN6j7BjNBdfOb7HjUSPFjt9DvdVz/J+wsGSHlkB2ZLoi8HnE3zOcq6LNNttNZm4DpQAqgOaKXUBqXUAaXU6MxOoJQKVUpFKKUiLl++/KDXcJfo2Oj0Y+hByh8Ih9KqYTfmPb+fSZVepZzZlSXJh+j6bTM+W/aKTIDigHL6ZqwL0BzoZ33tqpRqk3EjrXWY1jpQax3o5eX1yCe9a8IRkFr0wiF1bjGExaEHGfvYQDy1EzNvb6bbvEDm/ThJpjh0ILYk+mjAO83n8tZlmW5j7ZcvAlzF0vrfrrW+orW+DawF6j9q0PeTmJxoGUOfdsQNyOxSwqH1bfcmK547wPBCHYhXmslXFtN7VgO+3z7b3qGJXGBLot8PVFNKVVJKuQK9gTUZtlkDDLK+7w5s1lprYANQVynlYf0F8DhwPHtCz9zFWxdJ1smZt+jlYSnhwFxcTLz47P+xsu9u+roEcsElkbdPT2Hw143Zf2yTvcMTOSjLRG/tcx+BJWmfAJZqrY8ppSYqpTpbN5sFlFBKnQRGAWOs+14DPsXyy+IQcEBr/WO2X0UamY6h11puxgph5elRiLH95rCi60Y6JVfhiGssoftf4bWZT8qQTINysWUjrfVaLN0uaZeNS/M+Duhxj32/wTLEMlekjKFPl+gTb0OyWbpuhEjDq1hZPnhuFQP/jOS/P49is+s59q57hvYu9Xit25cU8ixq7xBFNjHck7HRsdE4Kae7x9CDtOiFyEStyg34MnQbU33fo1KiO8v0EZ79tjlfrRwrN2wNwnCJ/lzsOUp7lMbklGEMPUgfvRD30aphNxY+H8nYxwbirhVf3viBXrMasGH3InuHJh6R4RJ9dGz03SNupKCZEDbr2+5NVgyKYIBrEy64JPLmb//hhbBgqaGTjxku0d814QhIQTMhHpCrqxuj+4Sx5Om1tDWXZ6/rNQZu7c+4eT24HvuPvcMTD8hQiT4xKZFLty9lPoYepEUvxAMq61WRT4au5+sGU6mV4MlKfqX74hbMXDNe+u/zEUMl+gu3LpCsk9NPOAJpum6kj16Ih9Go7hPMfX4f/yrzHCYNn137jgEzG7Hrl3X2Dk3YwFCJPvqWZWhluglHQG7GCpFNej85iuUD9tLLyZ9TrnG8dPBN3pjVQSpk5nGGSvSZPiwFlj56tyLg5GyHqIQwFg93T94ZsID5rb8lKKEYG1yi6LXqSf67fJR05+RRhkr00bHROCtnSnuUTr/iTgwUkNa8ENmpekU/poeGM6nSqxRNdibs1kb6zgpkx6G1We8scpWhEn3KGHoXpwwP/EpBMyFyTOcWQ1g6aD/9TI34yyWBlw+N5q3ZTxNz84q9QxNWhkv0d3XbgBQ0EyKHubq6MabvLOa1+ZaGCUVY63yGHktaMu/HSfYOTWCwRB8VG5V5oo+7LmPohcgF1Sv6ERa6k3fLDsOkYfKVxTz3dRDHTkXYOzSHZphEn5CUwOXblylfsPzdK6XrRohc1bPtSJb1201XanHE9RZDtg/iw0VDMJtzfMpokQnDJPoLty6g0ffuupEWvRC5ytOjEBMHLeWrRp9RKdGNhYn76D07kPADGaezEDnNMIn+tvk2VYtWpULhCulXmBPAfEf66IWwk4a+bVg4ZD9DCrTivIuZkYffZuycZ2Sy8lxkmERfs3hNVnZZSUCpgPQrUurcSNeNEHbj5OzMqz0/Z0HbpQQmFOYHp1P0WtSc1VvD7B2aQzBMor8nSfRC5BmVvX2ZEbqLN736EO+keffM54ya2Y5r1y/bOzRDc6BEL103QuQVAzu+zaJnfqJFohcbTefovawVK7dMt3dYhuUAiT7G8iqJXog8pXSJckwbtoV/lRlMEprxf03jtZlPSus+B0iiF0LYVe8nX+PbbhtpmViKn03n6SWt+2znAIleum6EyOu8ipXl82GbebfsMDQw/q9pvDnrKW7dvmnv0AxBEr0QIs/o2XYk33TdQPOEEqx3+ZveC5uyNWKlvcPK9xwj0TuZwFTA3pEIIWxQukQ5vgzdxmvFu3PdOZlRR99l4oJ+8lTtI3CMRO9eBJSydyRCiAcw+OnxzH5iCXXiC7As+TD9Zzfi8O+77B1WvuQ4iV4Ike9UrVCHuUP38Jx7MKdNCbywYxhfr3rb3mHlO5LohRB5mpOzM6N6fcm0Rp9T2uzMtOvfM2JGS6l3/wAk0Qsh8oWGvm34pv9O2iWWZ5vrVfotbsW2yNX2DitfcIxEL5UrhTAET49CTB66jrdK9eemczKvH/4X/7domMxVmwXHSPTSohfCUPp3eIuZbRZSPcGVBYl7GDqzGReunLV3WHmW8RO9TCMohCFVr+jP/CF76YYvB9xiGbSqA1v2r7B3WHmSsRN9YhwkxUuiF8KgXFxMTBi0mHcrvEic0rx5dDyfL3vV3mHlOTYleqVUe6XUb0qpk0qpMZmsd1NKLbGu36uU8smwvoJSKlYp9UY2xW0beSpWCIfQrfWLzHhiIZUTXZhxexMvz2glE5ukkWWiV0o5A18AHYDaQB+lVO0Mmw0BrmmtqwJTgI8yrP8UWPfo4T4gqUUvhMOoXtGf+YN20c5cnq2uVxi4sAWHftth77DyBFta9I2Ak1rrP7XWCcBioEuGbboA86zvlwNtlLI8iqqUegY4DRzLlogfhLTohXAo7m4eTB6yjpeLPsN5UxIv7XyexT9NsXdYdmdLoi8HpL2dHWVdluk2WmszcB0ooZQqCLwFvHe/EyilQpVSEUqpiMuXs7EWtSR6IRxSaJd/MzVwCkWSFB+em8UHC59z6CGYOX0zdgIwRWsde7+NtNZhWutArXWgl5dX9p1datEL4bAa132S2V3X4ZdQgEXmCF6e1cZhyx7bkuijAe80n8tbl2W6jVLKBSgCXAWCgP9TSp0BXgXeVkqNeLSQH4C06IVwaI+V9Gb24F10MFdgu9tVnvsmmD/P5n4vsr3Zkuj3A9WUUpWUUq5Ab2BNhm3WAIOs77sDm7VFsNbaR2vtA0wF/qO1npY9odtAEr0QDs/FxcT/DfmR5wu240+TmdANPdm8b7m9w8pVWSZ6a5/7CGADcAJYqrU+ppSaqJTqbN1sFpY++ZPAKOCuIZh2EXcdnN2kFr0QghHdJjOx2mjMCsYcG8/s7yfaO6Rco7TW9o4hncDAQB0REZE9B/v+Ffh1Lbz5R/YcTwiR7/12+iBv/TyI06Zk+rk2YXTfGfYOKVsopSK11oGZrTP2k7FS50YIkUGNSgHM6vETfvHuLEjcw+jZnQw/IkcSvRDC4ZQo+hgzQrbTIqE465z/4oVZLQw9IkcSvRDCIbm7efDfwZvpklyd3W43GPrN44atgCmJXgjhsJycnXn/uRU85/44v7omMGxlR46dyqZ7hHmIJHohhMMb1Wsabzw2kEsuyby6JYQDx7fZO6RsZdxEr7UkeiGEzfq1H834am9xy0nzxu4X2X14vb1DyjbGTfTmOEhKkEQvhLBZx+YDeb/OeyQqeGv/62yNWGnvkLKFcRO9PBUrhHgIrRt154P6H+MEvHP4HTbsXmTvkB6ZJHohhMiguX9HPm78Oe7JMOHEJFZvm2nvkB6JAyT6onYNQwiRPzX0bcOnLWZQOFkx6c8pLN34ub1DemjGTfR3YiyvBYraMwohRD7mV70pn7dZQEmz4uOoMFZvDbN3SA/FuIleum6EENmgRqUApj75DcWT4MM/P2PjniX2DumBGTjRx1heJdELIR5R9Yr+fBQ8nQIaJh6fyK5fcn8K7Edh4ERvbdG7FbZvHEIIQ/Cv0Zz3AycD8E7EG/lq4nFjJ3oXdzC52zsSIYRBNK3XgXG1x3FHwVvhL/D7X4fsHZJNjJ3opdtGCJHN2jbuxZjKr/CPM4z6aQBnz+f9+S4k0QshxAPq0jKU18o+xzmT5pXvu3Ht+mV7h3RfkuiFEOIh9G33Oi8U68pJ12TeWNI5T09eIoleCCEeUmiXf9NV+bLPLZZx83vaO5x7kkQvhBCPYHz/RTSJL8xqp9/5etXb9g4nUwZO9DGS6IUQOc7J2ZmP+35P9XgnZl5bw0+7v7V3SHcxZqJPrUVf1N6RCCEcQJGCxfmg3TwKJWs+PP5+nht2acxEn3gbks3SohdC5JrqFf0ZU/sdbjopxm4YxPXYf+wdUipjJnqpcyOEsIMnm/RhaNGn+d0tmdGL8s5IHEn0QgiRjZ7v+gGdk6uyy+06k5e8YO9wAEn0QgiR7d4bsJQ68SaWxe/OE3PPGjzRF7VrGEIIx+TiYuKdNmGYtObjPaO5HXfLrvEYPNFLi14IYR++VQIZULQTf7hp3v+2v11jcbHr2XOKARN9YmIiUVFRxMXF2TsUIezG3d2d8uXLYzKZ7B2KTYY/+xEHw3aw1vUPmmyfzdMtBtslDoMm+hjLq7txatFHRUVRqFAhfHx8UErZOxwhcp3WmqtXrxIVFUWlSpXsHY7NJnRdxIA1HZn2+6c09etIiaKP5XoMNnXdKKXaK6V+U0qdVEqNyWS9m1JqiXX9XqWUj3V5W6VUpFLqiPW1dTbHn7k7MeBSAFzccuV0uSEuLo4SJUpIkhcOSylFiRIl8t1ftWW9KvJCxeGcd4Hxy3rbJYYsE71Syhn4AugA1Ab6KKVqZ9hsCHBNa10VmAJ8ZF1+BXhaa10XGAQsyK7A78ugdW4kyQtHl19/Bno8MYInzd5sc73KnB/+nevnt6VF3wg4qbX+U2udACwGumTYpgswz/p+OdBGKaW01ge11uesy48BBZRSOd/MNmiiF0LkX+P7fItPAsy5tJiTfx/N1XPbkujLAWfTfI6yLst0G621GbgOlMiwTTfggNY6PuMJlFKhSqkIpVTE5cvZUMA/7joUKProxxH3NWHCBCZPnnzP9ZcvXyYoKIiAgADCw8Mf+Phz585lxIgRAKxatYrjx48/dKxC2Fshz6K8HjCRWCfFp+tfzNVz58rwSqWUL5bunOczW6+1DtNaB2qtA728vB79hNKizxM2bdpE3bp1OXjwIMHBwY90LEn0wghaBnblcXNZdrr+w46DP+TaeW0ZdRMNeKf5XN66LLNtopRSLkAR4CqAUqo8sBIYqLU+9cgR2yLuOpSsliunsof3vj/G8XM3svWYtcsWZvzTvlluN2nSJObNm0epUqXw9vamQYMGnDp1ipdeeonLly/j4eHBjBkziIuLY/To0dy5c4eIiAh2797NqFGj2L9/P3fu3KF79+689957APj4+BAREUHJkiWJiIjgjTfeYOvWrann3LVrF2vWrGHbtm28//77rFixgipVqmTr9QuRW0Z2+JxdP3Vj5r6JNA/olCvntCXR7weqKaUqYUnovYG+GbZZg+Vm626gO7BZa62VUkWBH4ExWuud2RZ1VqRFnyMiIyNZvHgxhw4dwmw2U79+fRo0aEBoaCjTp0+nWrVq7N27lxdffJHNmzczceJEIiIimDZtGmD5JVG8eHGSkpJo06YNhw8fxs/PL8vzNm3alM6dO9OpUye6d++e05cpRI6qVK4mbanJavffWb1tJl0eH5rj58wy0WutzUqpEcAGwBmYrbU+ppSaCERordcAs4AFSqmTwD9YfhkAjACqAuOUUuOsy57UWl/K7gtJE7DhE70tLe+cEB4eTteuXfHw8ACgc+fOxMXFsWvXLnr06JG6XXz8XbdhAFi6dClhYWGYzWbOnz/P8ePHbUr0QhjNa12/YPuKNiz8bRpPN38OJ2fnHD2fTQ9Maa3XAmszLBuX5n0c0COT/d4H3n/EGB9Mwi3QSYZO9HlJcnIyRYsW5dChQ/fd7vTp00yePJn9+/dTrFgxQkJCUsdDu7i4kJycDJDvxkgL8TBKFH2Mju5BLDTvZ8H6Dxn01L9y9HzGq3VjwPIHeUWLFi1YtWoVd+7c4ebNm3z//fd4eHhQqVIlli1bBlieXvzll1/u2vfGjRt4enpSpEgRLl68yLp161LX+fj4EBkZCcCKFSsyPXehQoW4efNmDlyVEPYxstt/KZuoWXZuMQkJmf8VnF0k0Qub1a9fn169elGvXj06dOhAw4YNAVi4cCGzZs2iXr16+Pr6snr16rv2rVevHgEBAdSsWZO+ffvSrFmz1HXjx4/nlVdeITAwEOd7/Anbu3dvPv74YwICAjh1Knfu6QuRkzzcPelSvCN/ucJXq0fn6LmU1jpHT/CgAgMDdURExMMf4K9dMKcDDFgJVXKn4kJuOHHiBLVq1bJ3GELYnZF+FszmRLrNqc8tlczKPjsp5Fn0oY+llIrUWgdmtk5a9EIIYScuLiZ6VQjhosmJz757OcfOY+BEX9SuYQghhC36tnudOvEm1ice4MKVs1nv8BAk0QshhJ0NqvM6152d+GzNSzlyfAMneuPUohdCGFv7pv0Iji+Ou4tHjhzfeBOPxF0Hkyc4548ZaIQQAuDL0G05dmwDtuhj5EasEEKkYcBEb+zyB0II8aAk0QuH07JlS7J6ViM8PBxfX1/8/f25c+fOA58jJCSE5cuXAzB16lRu3779ULHmRVu3bqVTJ9urLkZERDBy5Mgst2vatCkAZ86cYdGiRQ8dn7ibMfvoC+b+5Lu5at0YuHAke4/5WF3o8GH2HjMTSUlJ93z6NS9ZuHAhY8eOpX///o98rKlTp9K/f//UYnDZzWw24+Ji/x/le8URGBhIYGCmz/Gks2vXLuB/ib5v34xFcsXDMl6L/k6MtOhzwMcff8znn38OwGuvvUbr1panjjdv3ky/fv0AGD58OIGBgfj6+jJ+/PjUfX18fHjrrbeoX78+y5Ytw8fHh7Fjx+Lv709gYCAHDhygXbt2VKlShenTpwN3txpHjBjB3LlzU483evRo6tatS6NGjTh58uR9Y79z5w69e/emVq1adO3aNV0L/aeffqJJkybUr1+fHj16EBsby8yZM1m6dCnvvvsu/fr1IzY2ljZt2lC/fn3q1q2bWuLhzJkz1KlTJ/VYkydPZsKECenO/fnnn3Pu3DlatWpFq1at0q3bv38/zz77LACrV6+mQIECJCQkEBcXR+XKlQGYMWMGDRs2pF69enTr1i31L4OQkBBeeOEFgoKCGD16NCEhIQwfPpzGjRtTuXJltm7dyuDBg6lVqxYhISGp5yxYsGDq++XLl6euSzleYGAg1atX54cfbJsUY8KECQwYMIBmzZoxYMCATLdJ+385YcIEBg8eTMuWLalcuXLq91Ta2MaMGUN4eDj+/v5MmTLFpjjE/dm/GZDdHKHrJhda3hkFBwfzySefMHLkSCIiIoiPjycxMZHw8HBatGgB3L/efIkSJThw4ABg+UGuUKEChw4d4rXXXiMkJISdO3cSFxdHnTp1eOGFF7KMp0iRIhw5coT58+fz6quv3jcxffXVV3h4eHDixAkOHz5M/fr1Abhy5Qrvv/8+P//8M56ennz00Ud8+umnjBs3jh07dqTWvzebzaxcuZLChQtz5coVGjduTOfOnW36uo0cOZJPP/2ULVu2ULJkyXTrAgICUqt+hoeHU6dOHfbv34/ZbCYoKAiAZ599lmHDhgHwzjvvMGvWLF5+2fIEZVRUFLt27cLZ2ZmQkBCuXbvG7t27WbNmDZ07d2bnzp3MnDmThg0bcujQIfz9/e8b65kzZ9i3bx+nTp2iVatWnDx5End39yyv8fjx4+zYsYMCBQrY9DX59ddf2bJlCzdv3qRGjRoMHz4ck+l/o+Q+/PBDJk+ebPMvG5E1YyX65GSIv2H8RG8HDRo0IDIykhs3buDm5kb9+vWJiIggPDw8tVV2v3rzvXr1Sne8lERZt25dYmNjKVSoEIUKFcLNzY2YmJgs4+nTp0/q62uvvXbfbbdv357aR+zn55ca0549ezh+/HhqgbWEhASaNGly1/5aa95++222b9+Ok5MT0dHRXLx4McsYs+Li4kKVKlU4ceIE+/btY9SoUWzfvp2kpKTUqRePHj3KO++8Q0xMDLGxsbRr1y51/x49eqTrBnv66adRSlG3bl1Kly5N3bp1AfD19eXMmTNZJvqePXvi5OREtWrVqFy5Mr/++muW+4Dl/9LWJA/w1FNP4ebmhpubG6VKleLixYuUL1/e5v3FgzNWok+IBZ0siT4HmEwmKlWqxNy5c2natCl+fn5s2bKFkydPUqtWrfvWmwfw9PRMdzw3NzcAnJycUt+nfE7p602pUQ9316lXSmX6/kForWnbti3ffvvtfbdbuHAhly9fJjIyEpPJhI+PD3FxcVnGaIsWLVqwbt06TCYTTzzxBCEhISQlJfHxxx8Dli6VVatWUa9ePebOnZtuisUH/ZpC+q/V/b6mmX2+l4xxZCVtbM7OzqmxiZxjrD56KWiWo4KDg5k8eTItWrQgODiY6dOnExAQgFLqvvXmH0bFihU5fvw48fHxxMTEsGnTpnTrlyxZkvqa0gpfuXIlY8eOvetYLVq0SB3FcfToUQ4fPgxA48aN2blzZ2of/61bt/j999/v2v/69euUKlUKk8nEli1b+OuvvwAoXbo0ly5d4urVq8THx9+zq+F+tfSDg4OZOnUqTZo0wcvLi6tXr/Lbb7+l9v3fvHmTMmXKkJiYyMKFC+//RbNB6dKlOXHiBMnJyaxcuTLdumXLlpGcnMypU6f4888/qVGjxiOf72HI3APZz1gt+pREX6CoXcMwquDgYCZNmkSTJk3w9PTE3d09tYshbb15b2/vdPXmH4a3tzc9e/akTp06VKpUiYCAgHTrr127hp+fH25ubqkt8lOnTlG48N2lL4YPH85zzz1HrVq1qFWrFg0aNADAy8uLuXPn0qdPn9TpD99//32qV6+ebv9+/frx9NNPU7duXQIDA6lZsyZg+Stn3LhxNGrUiHLlyqUuzyg0NJT27dtTtmxZtmzZkm5dUFAQFy9eTL3P4efnx4ULF1Jb0//+978JCgrCy8uLoKCgR06AH374IZ06dcLLy4vAwEBiY2NT11WoUIFGjRpx48YNpk+fblP/fE7w8/PD2dmZevXqERISkmXXnMiaserRn9kJczvCwNVQuWW2xmVvRqrB/ah8fHyIiIi46+Zm//79mTJlCl5eXnaKLP8KCQnJN5Ovy89C5u5Xj96YLXrpunFI33zzjb1DECJPkkQv8p0zZ87YOwTDSXlGIa0NGzbw1ltvpVtWqVKlu/r2H3Z7kXsMmuiL2jUMIYygXbt26YZzZvf2IvcYbNRNjOXVTWrRCyFECoMl+uvgWhCcjfWHihBCPArjJXrpnxdCiHQk0QshhMFJohcij7Klbn5aQ4cO5fjx4/fdZvr06cyfPx+wjLQ5d+6czcfXWjNy5EiqVq2Kn59fapG6jCIjI6lbty5Vq1Zl5MiRpDyrM2HCBMqVK4e/vz/+/v6sXbvW5nOLR2Oszuy4GChczt5R5LiP9n3Er//8mq3HrFm8Jm81eivrDR9RfqlHn5HWGq01Tk72bxvd62s4c+bMLPdNWxl07ty51KlTh7Jly9p03nXr1vHHH3/wxx9/sHfvXoYPH87evXvv2m748OHMmDGDoKAgOnbsyPr16+nQoQNgKXH9xhtv2HQ+kX3s/12bneKuy9DKHJKf69E/9dRTqfVtAgICmDhxIgDjxo1jxowZ9603X6NGDQYOHEidOnUIDw+nZs2ahISEUL16dfr168fPP/9Ms2bNqFatGvv27QMsLdfJkyennr9OnTqcOXOGM2fOULNmTfr160etWrXo3r27zTNPFSxYkNdff5169eqxe/fuTLdJ+xdAwYIF+de//kW9evVo3LhxarXNlNiWL19OREQE/fr1s3kWrdWrVzNw4ECUUjRu3JiYmBjOnz+fbpvz589z48YNGjdujFKKgQMHsmrVKpuuUeQcg7XoHaPrJjda3hnl53r0wcHBhIeHU7FiRVxcXNi5cydgqQGfUtPlXvXm//jjD+bNm0fjxo05c+YMJ0+eZNmyZcyePZuGDRuyaNEiduzYwZo1a/jPf/6TZVL77bffmDVrFs2aNWPw4MF8+eWXNrVwb926RVBQEJ988kmW26Zs37hxYyZNmsTo0aOZMWMG77zzTur67t27M23aNCZPnpw6+9Nrr712Vy0egN69ezNmzBiio6Px9vZOXV6+fHmio6MpU6ZM6rLo6Oh0JYdTtkkxbdo05s+fT2BgIJ988gnFihWz6XrEozFOiz45GeKkFn1OyViPvkmTJqn16FMKmy1dupT69esTEBDAsWPH0vUX368efVBQEIUKFcLLy+uh6tHfq4WbIjg4mO3bt7Nz506eeuopYmNjuX37NqdPn6ZGjRqp9eb9/Px44okn0tWbr1ixIo0bN049VqVKlahbty5OTk74+vrSpk2b1Brwtjyxm7bgW//+/dmxY0eW+4ClnG+3bt1s2hbA1dU19S+iBg0a2BTblClTOHTo0F3/xowZY/N572f48OGcOnWKQ4cOUaZMGV5//fVsOa7Imk0teqVUe+AzwBmYqbX+MMN6N2A+0AC4CvTSWp+xrhsLDAGSgJFa6w3ZFn1aCTcBLYk+h+TnevQNGzYkIiKCypUr07ZtW65cucKMGTNSq1jeq978/eLOGHvamu/3i/1ha767u7s/0L0Nk8mUemxba75n1aIvV64cZ8+eTV0eFRVFuXLp74mVK1eOqKioTLcpXbp06vJhw4Y90ATj4tFk2aJXSjkDXwAdgNpAH6VU7QybDQGuaa2rAlOAj6z71gZ6A75Ae+BL6/Gyn9S5yXH5tR69q6sr3t7eLFu2jCZNmqS7Drh3vfmH5ePjk9pNdeDAAU6fPp267u+//079C2TRokU0b978kc71KDLWfc+qRd+5c2fmz5+P1po9e/ZQpEiRdN02AGXKlKFw4cLs2bMHrTXz58+nS5cuAOn681euXJluvl2Rs2xp0TcCTmqt/wRQSi0GugBpx3F1ASZY3y8HpilLc6ILsFhrHQ+cVkqdtB7v/n9rP4w7MZZXSfQ5Jr/Wo0+JfdOmTRQoUIDg4GCioqJSY79XvfmH1a1bN+bPn4+vry9BQUHp6tvXqFGDL774gsGDB1O7dm2GDx/+SOd6FCkTghcoUIDdu3dnOR1gx44dWbt2LVWrVsXDw4M5c+akrvP390+d//bLL78kJCSEO3fu0KFDh9QRN6NHj+bQoUMopfDx8eHrr7/OsWsTGaQMG7vXP6A7lu6alM8DgGkZtjkKlE/z+RRQEpgG9E+zfBbQPZNzhAIRQESFChX0Q7n8h9ZLBmp97tDD7Z/HHT9+3N4h5BkVK1bUly9fvmt5v3799KVLl+wQkW1Onz6tfX197R1Gvic/C5kDIvQ98nieGHWjtQ4DwsAy8chDHaRkVeg5LzvDEvmM1KMXInO2JPpowDvN5/LWZZltE6WUcgGKYLkpa8u+QjyQ/FqP3sfHh6NHj961vGvXrun68QE++uije5b8fdDthbAl0e8HqimlKmFJ0r2Bvhm2WQMMwtL33h3YrLXWSqk1wCKl1KdAWaAasC+7gnc0WmubR2mI/ONBJ+Zw5Ik8dB6b+jS/yDLRa63NSqkRwAYswytna62PKaUmYukTWoOl732B9WbrP1h+GWDdbimWG7dm4CWtdVIOXYuhubu7c/XqVUqUKCHJXjgkrTVXr16126Tl+ZmxJgc3sMTERKKiou4aTy6EI3F3d6d8+fKYTCZ7h5LnOM7k4AaW8sCSEEI8KOOUQBBCCJEpSfRCCGFwkuiFEMLg8tzNWKXUZeBRio2UBK5kUzj5iVy3Y5Hrdiy2XHdFrbVXZivyXKJ/VEqpiHvdeTYyuW7HItftWB71uqXrRgghDE4SvRBCGJwRE32YvQOwE7luxyLX7Vge6boN10cvhBAiPSO26IUQQqQhiV4IIQzOMIleKdVeKfWbUuqkUip7pq3Pg5RSs5VSl5RSR9MsK66U2qiU+sP6WsyeMeYEpZS3UmqLUuq4UuqYUuoV63JDX7tSyl0ptU8p9Yv1ut+zLq+klNpr/X5fopRytXesOUEp5ayUOqiU+sH62VGu+4xS6ohS6pBSKsK67KG/1w2R6G2cwNwo5mKZaD2tMcAmrXU1YJP1s9GYgde11rWBxsBL1v9jo197PNBaa10P8AfaK6UaAx8BU7TWVYFrwBD7hZijXgFOpPnsKNcN0Epr7Z9m/PxDf68bItGTZgJzrXUCkDKBueForbdjqfmfVhcgZR7FecAzuRlTbtBan9daH7C+v4nlh78cBr9263SgsdaPJus/DbQGlluXG+66AZRS5YGngJnWzwoHuO77eOjvdaMk+nLA2TSfo6zLHEVprfV56/sLQGl7BpPTlFI+QACwFwe4dmv3xSHgErAROAXEaK3N1k2M+v0+FRgNJFs/l8Axrhssv8x/UkpFKqVCrcse+ntd6tEbjHUKR8OOmVVKFQRWAK9qrW+knW3LqNdunZXNXylVFFgJ1LRvRDlPKdUJuKS1jlRKtbRzOPbQXGsdrZQqBWxUSv2aduWDfq8bpUXv6JOQX1RKlQGwvl6yczw5QillwpLkF2qtv7MudohrB9BaxwBbgCZAUaVUSkPNiN/vzYDOSqkzWLpiWwOfYfzrBkBrHW19vYTll3sjHuF73SiJPnUCc+td+N5YJix3FCmTs2N9XW3HWHKEtX92FnBCa/1pmlWGvnallJe1JY9SqgDQFsv9iS1Ad+tmhrturfVYrXV5rbUPlp/nzVrrfhj8ugGUUp5KqUIp74EngaM8wve6YZ6MVUp1xNKnlzKB+ST7RpQzlFLfAi2xlC29CIwHVgFLgQpYSjz31FpnvGGbrymlmgPhwBH+12f7NpZ+esNeu1LKD8uNN2csDbOlWuuJSqnKWFq6xYGDQH+tdbz9Is051q6bN7TWnRzhuq3XuNL60QVYpLWepJQqwUN+rxsm0QshhMicUbpuhBBC3IMkeiGEMDhJ9EIIYXCS6IUQwuAk0QshhMFJohdCCIOTRC+EEAb3/yIGbcf3NgowAAAAAElFTkSuQmCC", + "text/plain": [ + "
    " + ] + }, + "metadata": { + "needs_background": "light" + }, + "output_type": "display_data" + } + ], + "source": [ + "scheduler = PolyLRScheduler(optimizer, t_initial=t_initial)\n", + "plot_lr(scheduler, label='default')\n", + "\n", + "scheduler = PolyLRScheduler(optimizer, t_initial=t_initial, warmup_t=2)\n", + "plot_lr(scheduler, label='warmup, default warmup_lr_init')\n", + "\n", + "scheduler = PolyLRScheduler(optimizer, t_initial=t_initial, warmup_t=2, warmup_lr_init=0.05)\n", + "plot_lr(scheduler, label='warmup, warmup_lr_init=0.05')\n", + "plt.legend();" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "As we can see by setting up `warmup_t` and `warmup_lr_init`, the scheduler first starts with a value of `warmup_lr_init`, then during `warmup_t` number of epochs gradually progresses up to the LR value at epoch `warmup_t + 1`." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### `warmup_prefix` " + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "If `warmup_prefix` is `True`, after warmup annealing starts from initial LR value. " + ] + }, + { + "cell_type": "code", + "execution_count": 20, + "metadata": {}, + "outputs": [ + { + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXoAAAD4CAYAAADiry33AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/YYfK9AAAACXBIWXMAAAsTAAALEwEAmpwYAABCAUlEQVR4nO3dd1gUV9vH8e/ZBcGCBewVW6wgKmLsCrHGFnuLYkxMMUVTTa8+T4qJeU3X2E3svUWNWLBExYZib1HsFVQEYfe8f8zGR40FdZdZlvtzXV6yu7M7vyHkZjxz5j5Ka40QQgjPZTE7gBBCCNeSQi+EEB5OCr0QQng4KfRCCOHhpNALIYSH8zI7wK3y58+vAwMDzY4hhBCZyqZNm85qrQvc7jW3K/SBgYHExMSYHUMIITIVpdTfd3pNhm6EEMLDSaEXQggPJ4VeCCE8nBR6IYTwcFLohRDCw6Wr0CulWiil9iil9iulBt/m9YZKqc1KqTSlVKdbXuujlNrn+NPHWcGFEEKkzz0LvVLKCvwAtAQqA92VUpVv2ewIEAn8fst7/YEPgdpAGPChUirfw8cWQgiRXuk5ow8D9mutD2qtrwGTgXY3bqC1Pqy1jgXst7y3ObBUa31ea30BWAq0cELuf7mSksan83ey4dB5bPYs1HpZa9g8Hvb/CWnXzE4jhHBD6blhqhhw9IbH8Rhn6Olxu/cWu3UjpVR/oD9AyZIl0/nRN9t5IpEJf/3NqNWHyJ8rG00rF6J5lcLULZufbF4efCni2GaY+5LxtU8eqNACKrWBshGQLYe52YQQbsEt7ozVWo8ARgCEhoY+0Ol4rUB/Nr/flBV7TvPHjpPM3XqcSRuO4ufrRUTFgrSoWphGjxQkezarU7Obbvc8UFboOBL2R8GeBRA7BbyyQ/nHoGIbo/j75jE7qRDCJOkp9MeAEjc8Lu54Lj2OAY1vee+KdL73vuXy8aJ1cFFaBxclOdXGmv1n+WPHSZbuOsXsrcfx9bbQ+JGCtAwqTHjFgvj5ersqSsbZvQAC60PVjsYf2//B32tg1zzYPd/42+INZRpD5bZQoRXkzG92aiFEBlL3WkpQKeUF7AUiMAr3RqCH1jruNtuOBeZrrac7HvsDm4Aajk02AzW11ufvtL/Q0FDt7F43aTY76w+d548dJ1kcd5LTl1LIZrVQr1wALYOK0LRSIfLlzObUfWaIM3vhh1rQ8iuo3f/fr9vtcCwGds6BXXPh4hFQFihVDyq3g4qtIXeRjM8thHA6pdQmrXXobV9Lz5qxSqlWwLeAFRittR6ilPoEiNFaz1VK1QJmAfmAZOCk1rqK471PAe84PmqI1nrM3fblikJ/I7tds+XoBRZtP8miHSc5dvEqVouibtkAWlYtQrMqhcify8dl+3eq6G9g2ccwKA7yFL/7tlrDyVjjDH/nXDi7B1BQorZR9Cu3vfdnCCHc1kMX+ozk6kJ/I601O44lsmjHCRbtOMmhs1ewKAgr7U+roCK0qFKYgrl9MyTLAxkZDtoO/Vfc/3tP7zbO8nfOhVPbjeeK1XQU/XaQL9CZSYUQLiaFPh201uw5dYmF20+yaPsJ9p2+jFJQq5Q/LYMK06JqYYrkyZ7hue4o8Th8UwnC34eGrz/cZ5078L/hneNbjOeKhPyv6AeUfei4QgjXkkL/APafvsSC2JMs2nGC3ScvAVCzVD5aVi1Mq6AiFM1rctHf+CsseA1eWA8FKzrvcy/8bRT9nXOM8X2AwkGOov8E5C/nvH0JIZxGCv1DOnDmMou2n2DB9pPsOpEIQI2SeWkVVMS8oj++vXFx9aVNoJRr9nHxqHGWHzcb4jcYzxUKgipS9IVwN1LonejQ2Sss3H6CBbEn2HlL0X88uEjGDO9cvQhflYU6A6DpJ67fH0BCvDGev3M2HF1vPFcoCKq0hypPyPCOECaTQu8ityv6oaXy8XiwcaZfyFUXcmOnwsxnoN9SKBHmmn3czT9FP27W/870CwdD1Q5QuT34l874TEJkcVLoM8DBM5dZuP0E82ONMX2ljLt1WwcXoWXVIhTwc+KUzam94chf8OpusJjc3iEh3hjPj5sF8RuN54pWhyodjLP9vA/W0kIIcX+k0Gewfy7kzo89zr7Tl7EoqFM2gNbBRWlRpfDD3ZyVehW+LAvBXaDNt07L7BQXjxjj+XGz4Phm47niYf8705ebs4RwGSn0Jtp76hLztx1nXuwJDp29gpdFUb98floHF6VZlULkvt82DHv+gEldodcMKPeYa0I7w/lDRsGPmwkntwPKuCO36hNG0Zc2DEI4lRR6N6C1Ju54IvNjTzBv23GOXbxKNquFxhUK0DakKBEVC6Wv4dqcAcb4+BsHwCuTtG04s9co+DtmwNm9RhO2Mo2M3jwVW0P2vGYnFCLTk0LvZrTWbDl6kfnbTjA/9jinL6WQI5uVxyoVom21ojR4JD8+Xrcp+nYbDC1vNCjrNDrDcz80reFU3P+K/oXDYM0G5ZoawzsVWkK2nGanFCJTkkLvxmx2zYZD55kXe5xF209wISmV3L5etKxahHYhRaldJgCrxTFP/u+1MKYldBpjFMbMTGujl/6OGUbhv3QCvHMY3TWDOhn99DPLv1iEcANS6DOJVJud1fvPMm/rcRbHneTKNRsF/Xx4PLgIbasVJWTnl6iNvxrDNr65zY7rPHY7HFkH26cZM3iungffvMbduEGdjLF9i4etIyCEk0mhz4SSU20s23WauduOsXz3Ga7ZbKzN/irJecph7zGVcgVzmR3RNWypcGC5UfR3L4DUK+BXxBjPD+oMRaq57k5gITIxKfSZXMLVVP6InsW6/W9z4FpFdl+tRWDusjxRpSYdqpehcB437rD5MK5dgb1/wPbpsG8p2FMhoLwxtTSoE/iXMTuhEG5DCr0H+Gl2D35M2I6v1ZdkWzIAWit0qj95raVoVLwhLz76BEVz+5uc1EWSzhvDOtunw9+rjeeKhRpFv0oHyFXA3HxCmEwKvQfoNKYaOZQXY/ts5NilY+y9uJcN8XGsPbqDI1f2YLeeR9u9KGANoU2ZNjxb63Fy+mSSBVTuV0K8cRE3dprRS19ZoVwEBHWBiq1k5o7IkqTQZ3Lx8X/RctkzvJ7/Ufo8PvJfr9vtdmbt+osJ22dxIGk1WC+DLQelfOvQvUp7ugc1xGJ2qwRXObUTtk81in5iPHjnhEptjDP9Mo3lIq7IMqTQZ3ITFj7Ll2fWsjD8Z0qUqHfXbZNSUxizaQmz9s3nZNpGlCUVS1oAQXmb8GyNzjQoXTmDUmewf2buxE4xOmwmJ0CuQsYF3OCuRk99uYgrPJgU+kwucmwoiTqVmX233df7zlxO5Pv1s1h6ZCGJahdKaXxsgdQt1IxXHu1M2YDCLkpssrQU2LvYKPp7FxsXcQtUgmpdjeGdPMXMTiiE00mhz8TOn99Pk7nteSZPVV58YvIDf07cqaN8v2Ea688sJdUaj9YW8qlg2pRuw/O12+Dn40bLJDpT0nnjhqxtUxwtlRWUbgjVuhtDPD4eOk1VZDlS6DOxWX++wQfH/mDqo59RqUI7p3zm4n1bGLVlOrsurwBrItiyU8q3Lk9W7UjnqvU8dzz/3AGjl/+2SXDxb2M8v3JbqNYNAhvIeL7I1KTQZ2Ivja/L3rRE/oiMRTm5AF9LS2PM5iVM3zOHE9fH8wtQM6ApL4V1o3pRD11ARGujn/+2SUZb5ZQEyF3MGMsP6QH5y5udUIj7JoU+k0q6fJoG08PpkqM0b3WZ59J9nbx0geF/zWRZ/EKSLHvRWpFLV6B5yda8XKcDATn8XLp/06RehT2LjKK/fxloGxSvZQztVO0A2fOZnVCIdJFCn0ktXf0fXj0widHBA6lVvV+G7Xdj/H6+3zCFrReWYvc6h7b7UNy7Nj2rdKRntcaeO7Rz6ZQxVXPr73B6J1h9jHn5IT2hbLgM7Qi3JoU+k3r7tyasvnaG5b1i8PLO+DYHdrud32NX8lvcTI5eW4eypGBJC6C6f1Neqd3Ds4d2TmwzzvJjpxpN1vyKGEM71XvJ0I5wS1LoM6HU1CQaTQwj3Kcwn/X40+w4nEu6xPB1M1l8ZB5XLHvQWuGnK9IysC0vP/oEebN76N2oadeMfjtbfzP67WibsTxiSA9jaMc3j9kJhQCk0GdK6zb9TP8dPzD8kd40qfOG2XFusjF+P99tmMzWC0vRXufB5ksp33o8Va0r7SvV9uyhndgpRtE/sxu8shutlKv3crRS9tDjFpmCFPpM6LMprZibdIRV3aLxddMLgmk2GxO2RjFp5wyOp21AWVLxSitKnYIteK1ud8+9IeufRVO2TjSarKUkQr5ACOkFId0hT3GzE4osSAp9JmO3pdF0XAhB3nn49sk1ZsdJl+OJ5xm2bhrLjy0gxXoIra3kVyF0rtCRZ0Jbks3Ly+yIrnEtCXbPhy0T4NAqQBkXbmv0NlbLklWyRAaRQp/JbI+bRo+YT/hPyba0aTLE7Dj37c/92/hp0yT2XlkB1isoW16C8zzGwNo9CS1ezux4rnPhMGz5zZi1kxgPOQKMaZrVn4SCFc1OJzycFPpM5v9mdGLMpd2sbD+fPHkDzY7zwK6kpPDD+jnMOTiLBOIA8NMVaV26PS/XecJz2y7YbcYqWVvGw+6FRq+d4mHGWX6VJ6TtgnCJhy70SqkWwP8BVuBXrfXnt7zuA4wHagLngK5a68NKKW/gV6AG4AWM11r/9277kkIP7UYHU8Dqw699NpodxWm2HD/Et3/9zpYLSxwXcHPySM7GvFSrF43LVDU7nutcOQvbJsPm8XB2D2TzM1bHqtkHilY3O53wIA9V6JVSVmAv0BSIBzYC3bXWO2/Y5gUgWGv9nFKqG/CE1rqrUqoH0FZr3U0plQPYCTTWWh++0/6yeqE/dHgFbVe+xNuFGtKjxQ9mx3G6NJuNUZsWM2nXNM7qzShlJ7utHM1LtuO1ep09d5qm1nB0A2waC3GzIO0qFA42Cn5QZ5mmKR7a3Qp9euaDhQH7tdYHtdbXgMnArd212gHjHF9PByKUUgrQQE6llBeQHbgGJD7AMWQZUduNb2N4tadNTuIaXlYrz4a1YkWfMcx4fBF18vUmRScw+9jXNJjUhPaT32Dpvq1mx3Q+paBkbXjiJ3htN7QaCmhY8BoMrQCzB0B8jPELQQgnS88ZfSeghdb6acfjJ4HaWusXb9hmh2ObeMfjA0BtIAGYAEQAOYBBWusRt9lHf6A/QMmSJWv+/fffTji0zKnnmOrY0Ezuu9XsKBnGbrczfksUE3ZO4VTaRpTFhq+tLC1Ktvf8s/zjW4yz/O3TIfUKFKoKNSONFbLkLF/ch4c9o38YYYANKAqUBl5TSpW5dSOt9QitdajWOrRAgay7yPPpUzuItaQRERBsdpQMZbFYiKz5GMueHMncdoup79+XVH3pprP8qAOxZsd0PqWgWA1oO9w4y289DJQFFr4OX1eEOQPg2CazUwoPkJ7JzceAEjc8Lu547nbbxDuGafJgXJTtAfyhtU4FTiul1gChwMGHDe6Jlm81/rETXuVJk5OYp4x/IX5q8yp2+8DrZ/n7k5fyyuo/yLGqPI8HdmBQ3Y6eN2PHNzeEPgU1+8LxzY6z/BmwZSIUCTFeC+okC5+LB5KeoRsvjIuxERgFfSPQQ2sdd8M2A4CgGy7GdtBad1FKvQVU1Fr3VUrldLy3m9b6jqdnWfli7LPjwjhuu8rcyG1O7z2fmR04d5Iv1kxg/ZkF2L3OgS0Xlf0ieLV2H2qX9OAGY8kJRlO1mNFGN02f3EZjtdCnoJCHrv0rHpgzple2Ar7FmF45Wms9RCn1CRCjtZ6rlPLFGIuvDpzHKOYHlVK5gDFAZUABY7TWX91tX1m10CcmHKXRrJY86fcIr3acaXYct/TPjJ3fdk3hvN4KaPJQhc6PdOWFsNaee/et1nB0vVHw42aDLQVK1oFaT0OltnL3rQDkhqlMYcGKDxj89ywm1HybkKo9zI7j9mJPHuarNRPYlrAYbU1ApfkTlr8Vb9XrTfn8RcyO5zpXzhlN1WJGw4VDkLOAcSNWzb6Qt8S93y88lhT6TOC1iQ3YfO0Cy/psxWL10DNTF0hKTeH7dXOYeWCq0T7ZbqWIV236VetJl6r1PbeTpt0OB6Ng4yijjTLAIy2gVj8oEy6dNLMgKfRuLiU5gYaT6vF49uJ80O0Ps+NkWssPbuf/NoznwNUVYEnG21aCFsU78maDbp47RRPg4hGIGWPcfZt0FvzLQtgzRs98maKZZUihd3Or1g9jwO7R/Fz5WerVevHebxB3deZyIp+v/o2oY7NI8zoBthxUyvUYb9TtSy1PbqqWlgI758CGERC/EbxzQrWuUOsZuXibBUihd3MfTmrKkuQTrOqxHm8fDz7zzGD/3Ig1dsdvnNWbAU0+VY3elXvRr2ZTzx3WAaNf/sZfjRuxbCkQ2ABqP2u0Tpa1bz2SFHo3Zku7RviEGtT2DuDLXivNjuOxYk8e5vPVY9meuBisl7GmFSaiWAfeadiLgBx+ZsdznSvnjC6aG0dBwlHIUxLCnjZaJ+fwNzudcCIp9G5s07ZxRG4dylelO9Oi4Qdmx/F4CclJDF09lYVHpnHNegRsvjySM4I36/Tz7Dn5tjTYuwjW/wKHo41lEIO7GGf5haqYnU44gRR6N/bVtHZMunKAVR2XksvPg6cFuhm73c7UHasZuW0Cp2wbAI2/CuGpqr3pXT3cs4d1Tu4wxvFjp0BaMpRuBI++AOWbyWydTEwKvZvSdjstx1ajjFdOfuz9l9lxsqwdJ4/wn9Wj2Z74B1iv4J1WnFaluvBWg26e12rhRknnYfM42DASEo+Bfxmo/byx7q2PBw9neSgp9G5qz74FdFo7mA+LNqVT02/MjpPlXbx6hc+jf2fx0emkeR0HWy5C8rTivYb9qFCgqNnxXMeWCrvmwl8/GbN1fHIbN2GF9Yd8pcxOJ9JJCr2b+ml2T366uI2o1tPJn1/WFHUXdrudMZuXMnbHBC6qbWi7FyWy1ee12k/zWLlqZsdzraMbYf1PRqsFtNFioe5LUPy29UO4ESn0bqrzmGpkV16Mj5RWtO4q+tBOvvxrJIeSV6IsqfjpqvSp/CTPhLbw7HH8hHhjHD9mLKQkQInaUGcAVGwt0zPdlBR6N3Ts2AZa/NmP1/M/Sp/HR5odR9zD4fOn+XTVaDacnwvWS3ilFaFlya683bCHZ4/jp1w2euv89SNcOAx5S8GjzxvTM2WRc7cihd4NTVj4LF+eWcvC8J8pUaKe2XFEOl1KucqX0VOY//dk0ryOgc2PMP92fNT4aUrkDTA7nuvYbbBnIaz9Ho7+ZbRWCO1nTM/0K2x2OoEUerfUd2woF3Uqs/puMzuKeAB2u51Rm5YyZscYLlni0PZslM8ewbv1+xPqyW0WwBjHXzscds0DqzcEdTHG8QvKdSYzSaF3M+fP76fJ3PY8k6cqLz4x2ew44iH9sXcz36wfwXHbOkBTyFqbgaH9aVOpltnRXOv8QVj3o7EKVtpVYx5+vYFQqq6xTKLIUFLo3cysZW/yQfwiptT+lMoV25sdRzhJ7MnDfLpqBLuuLEFZUvCzV6Zf0FP0reHhfXWunDP66mwYYXTPLF7LKPgVWskNWBlICr2beWl8XfakJbI4MlaWDPRA8Qnn+WjFSNafmwPWS2SzlaRT2d68Vq+j566CBXAtybhwu/Y7uPg3BJSHei8byx96+ZidzuPdrdBLlclgSUlnWWdLJDxXaSnyHqp4Hn9+bfcWq3v8SYvCL5Kmr/L74c+oNa4pby4ewaWUq2ZHdI1sOYw++C9tho6jwNsX5r4E/1fNKP4pl81OmGVJpclgazePJMWiiCjf3uwowsXy+Obgq+bPsrHPEnqWfg8vlYNFJ7+j3m9NeWHeMM5cTjQ7omtYvSCoEzwbDU/OgoBysOQ9GFYFlv/XaL0gMpQM3WSwt39rwuprZ1jeKwYvb1+z44gMZLfbGRnzB6PjRpFk2Qu2HFTP24bPwp+lZN4CZsdzraMbYfU3xhRN75wQ2te4ASu3B7eWyGAyRu8mUlOTaDQxjHCfwnzW40+z4wgTTYmN5vstI7jIVrQ9G5VztuCTJs9TsUBxs6O51qmdsHoY7Jhh3GEb0gPqD4J8gWYny/RkjN5NxMSO55JFER7YzOwowmRdgxsQ3WcCQ+uOp5BXDXYmzaPT/Da0n/wGsScPmx3PdQpVho4j4aVNENITtv4Ow2vArOfh7D6z03ksOaPPQEOmtGJ20hFWdV1FdlndR9wg+tBOhqz5gfi01YCiVLZGfNTwJc9e4xYg4ZhxoXbTGGPN26odoMHrssbtA5ChGzeg7XYeGxtMkHcevn1yjdlxhJuKid/Px9E/cCh5OShNMa96fNjgZeqW8vC7Ti+fhnU/GPPxr102mqc1eguKBJudLNOQoRs3ELd7BqetivBijcyOItxYaPFyzOs+jMkt51LetznHUtfRf3lXmk18njV/7zI7nuvkKghNP4aB240Cf2gV/NIAJveEE9Im5GHJGX0GGT6zM6MTd7Gy/Xzy5A00O47IJHadjue95d+zJ2kJqDSKWuvyfv2XaVDaw4c2rl6Av342FkNJSTDusm30FhQNMTuZ25KhGzfQbnQwBaw+/Npno9lRRCa0+0w87y3/gd1XFoNKo4i1Lh83HOj5QzpXLxoLmv/1AyQnwCMtock7MqRzGzJ0Y7JDh1dw0KoJLxRmdhSRSVUsUJzpXf7LrDYLqJSjNSfSNtB/eVdaTBzAxvj9Zsdznex5ofFbxpBOk/fgyFpjSGdKL2OqpkgXKfQZIGr7OADCqz1tchKR2ZXPX4RpXf7D9NbGGH586lr6/tmRx39/hS3HD5kdz3V880CjN+CVWGg0GA6uhJ/qwrS+cGav2encngzdZICeY2qQhp0pfbeaHUV4mB0nj/DO8m85mBIFGsr6RvB5xCAqFfTwG6+SzsO6741x/LSrENQZGg8G/zJmJzPNQw/dKKVaKKX2KKX2K6UG3+Z1H6XUFMfr65VSgTe8FqyUWqeUilNKbVdKZan7/s+cjiPWkkpEQJDZUYQHqlq4JHO7f8OEZrMo5dOQAyl/0nlBW7pOe48jF8+YHc91cvhDxAcwMNZopbBzDnxfC+YPgsTjZqdzO/cs9EopK/AD0BKoDHRXSt16yb8fcEFrXQ4YBnzheK8XMBF4TmtdBWgMpDotfSawfMsIAMKrPGlyEuHJqhctzYIewxnRZAqFvUKJuzKXx2e1ou+s/3DqcoLZ8VwnZ35o9hm8vBVqRsLm8TC8utFE7co5s9O5jfSc0YcB+7XWB7XW14DJQLtbtmkHjHN8PR2IUEopoBkQq7XeBqC1Pqe1tjkneuYQdfIvStmgbOnHzI4isoC6pSry55Mj+LreOPJZKhGTOInHprZgwPxvPbc9MkDuIvD41/BiDFR5wljb9v+qGd0yUy6Znc506Sn0xYCjNzyOdzx322201mlAAhAAPAJopdRipdRmpdSbt9uBUqq/UipGKRVz5ozn/HPzUuIx1usrhOcuJ73nRYZqXr46q/qM5/0aP5FTFWfVuVHUm9iCD5eN41pamtnxXMe/NDzxM7ywDso2hpWfw/+FGFM0066Znc40rq4+XkB9oKfj7yeUUhG3bqS1HqG1DtVahxYo4DntWqO3jCBNKcIrdjY7isiiugTV56++03ip8pd4q5zMjB9K7XGt+f6vedjtdrPjuU7BStB1IjwdZXy96E34oRZsnw6efNx3kJ5CfwwoccPj4o7nbruNY1w+D3AO4+x/ldb6rNY6CVgI1HjY0JnFsqNRBNg0wZW6mB1FZHH9a7VkfZ95dCv1NnZS+GXPO9QZ15np2z2871LxmtBnHvScAdn8YEY/GNEI9i8zO1mGSk+h3wiUV0qVVkplA7oBc2/ZZi7Qx/F1JyBKG/M2FwNBSqkcjl8AjYAscZdDSnICq1Mv0CRHMSxWD14nVGQaXlYr7zbuwdpefxBR8DmS9HE+3vwcTcY/xV9H9pgdz3WUgvKPwbOroMNISL4IEzvA+PZwcrvZ6TLEPQu9Y8z9RYyivQuYqrWOU0p9opRq69hsFBCglNoPvAoMdrz3AvANxi+LrcBmrfUCpx+FG1q/bSxJFkVEmdZmRxHiJjl9fPi25QCiuiwmJFcXzti28nRUVzpOGezZUzItFgjuYlywbf5fOLEVfm4Aswd4/JRMuWHKRT6a1Iw/ko+zqsc6svn4mR1HiDuKO3WUN/78kiOpK1F2X+oX6MaXzZ7Hzye72dFc6+oFiP7auFCrrFD3Raj3CmTS/1+l100Gs6VdY3nycRp6B0iRF26vSqESLOz5HV/XG0duS1lWnx9D/Ykt+e/KKZ59wTZ7PmMO/osboWIrWPWVMQc/ZgzYPWsWuBR6F9gWN5nzFkV4yXCzowiRbs3LV2dN5CReqvwlFnz5/fBnPDq2E/N2bTA7mmvlC4ROo40ZOgHlYP5A+KWh0RPfQ0ihd4GovTPx1pr61fubHUWI+2bM0JlPi8IvcpUTvL3+aVpMHMCu0/FmR3Ot4jWh7yLoPBaSE2FcG2Phk/MHzU720KTQO5m221l26QC1VS5y+RUxO44QDySblxdfNX+WxZ2MtsjxaWvovKA9/WZ/QUJyktnxXEcp487aFzdA+PtwYDn8UBuWvG8U/0xKCr2T7Tu4mHgrRBSta3YUIR5a0dz+TOvyH0Y0mYy/pQobEibS8LdWfLN6pmeP33tnh4avw8ubjc6Ya4fDdzVgy2+Z8oYrKfROtmzHRJTWNA55xuwoQjhN3VIVWdVnHAOrDMWCN2MOfEj98T1ZdSjO7Giu5VcY2v8Izyw3xvLnvACjm8GxzWYnuy9S6J1s+YU4qpGN/AUqmR1FCKfrF9qcNb3n0zCgH5f0fl5Y2ZMu09717A6ZAMVqwFNLoP1PcOFvGBkOc1+GK2fNTpYuUuid6PjxGHZZbETkzzJdHkQWlMPbhx9aD2R66zkU86rLrqS5NJ36OJ8sn+jZwzkWC4T0gJdijB74W38zhnPWj3D76ZhS6J0oatuvAIQHR5obRIgMUKFAURb3+pH3a/yEN3mZduQL6o7ryoqDO8yO5lq+eaD5EHhuDRQJgUVvwMgmcGyT2cnuSAq9E0WdiqGc3ULJkvXNjiJEhukSVJ91vWfTtOALXNGHeXFVL3pM/4iLV6+YHc21ClaE3nOg0xi4dApGRsD8V407bt2MFHonuXD+AJtIJjxvRbOjCJHhsnl58U3L55neZg5FrY+y/coMGv3+ON//Nc/saK6lFFTtYNxdW/s52DTGWNJw2xRwo/YyUuidZOWWEdiVIqJSd7OjCGGaCgWKsuTJn3mt6jAU3vyy5x0aj4tkx8kjZkdzLd/c0PJz6L8C8paCWf2NG67O7jM7GSCF3mmWHYumiE1T6ZG2995YCA8XWfMx1vSaT+28vThrj6X7oo68uugn0mzufdHyoRWpBv2WQutv4WQs/FQPVn5l+upWUuidICnpLOtsiYTnKi1LBgrhkNPHh1/bvcWvEZPIRWmWnv6RuuM6EnUg1uxormWxQGhfGOBolrb8M2Oxk6MbzYtk2p49yNrNI0mxKJqUa2N2FCHczqMlK7Cmz2TaF3uNq5zg5ejePDnjU89erBzAr5DRN6f7FEhOgFFNYeEbpixWLoXeCaL+Xkoeu6ZmUG+zowjhliwWC58+FsnsdnMoYq3N1stTaTCxLZNjPadD5B1VaAED1kNYf9gw0uids3dxhkaQQv+QUlOTWHHtNI18CuHl7Wt2HCHcWtmAwix98heeq/Af7KTw2eYX6ThlMBeSLpsdzbV8/KDVl8b4vU9u+L0LzHo+w6ZiSqF/SJtiJ3DJoggPbGZ2FCEyjQGPtmFp13kEZgtnb/ICGk9qy2/bVpgdy/VK1IJnV0LDNyB2CvzwKOxZ5PLdSqF/SMv2z8HXrqkrTcyEuC+FcuVhfo9vGVTlazQ2/rvlZTpMfsvzz+69fCD8PXgmCnLmh0ndYOazkHTeZbuUQv8QtN1O1JUj1PXKQ/Yc/mbHESJTeiq0Gcu6zqOsT1P2pSyk8aQ2WePsvmiI0RWz0WDYMR1+fBR2L3DJrqTQP4S43TM4bVVEFGtkdhQhMrUCuXIzp/vXvFZ1GKD575aX6Tbtfa6kpJgdzbW8skGTt42Cn7Mg7P3DJbuRQv8QonZPxao1jWrIkoFCOENkzcdY0nUOJb0bEZc0mwa/tWfpvq1mx3K9IsHQfzm0+NwlHy+F/iEsu7iHUJWdPHkDzY4ihMcolCsPC3t+x1PlPiGViwxaHckL84Z5/l21Vm/IltMlHy2F/gEdOryCg1ZNk4K1zI4ihEcaVO8JZradST5LENHnR1N/fFe2HD9kdqxMSQr9A4raPg6A8Gr9TE4ihOcqn78IK58cR5siA7nMYXr/0ZUvV001O1amI4X+AUWd3UZlu5UiRWuaHUUIj2axWPhPs378GvEbPhRkwqFPaf37QM/vd+9EUugfwOlTO4i1pBLuH2R2FCGyjEdLVmBlzxlUyt6Wv1OX0eT3J/hz/zazY2UKUugfwIqtIwGIqPqkyUmEyFpy+vgwtcsQXqj0BWnqEgOjI3n9j589e61aJ5BC/wCWnVxHSRuULf2Y2VGEyJKeD2vFtDbTyM0jLD71AxETn+bkJfdbws9dSKG/T4kJR9mgk4jIXU56zwthoooFirOq9+/UydebM/ZNNJ/akcX7tpgdyy1JpbpP0VtGkqYU4RU6mR1FiCzPy2plRNs3GBzyLXaVzGur+/FR1HizY7mddBV6pVQLpdQepdR+pdTg27zuo5Sa4nh9vVIq8JbXSyqlLiulXndSbtNExS8nwKYJrtzV7ChCCIdeIU2Y1mYqOQlkxtGvaDNpkOcvbHIf7lnolVJW4AegJVAZ6K6UqnzLZv2AC1rrcsAw4ItbXv8GcH0vThdLSU5gdeoFmuQohsXqZXYcIcQNKhYozspek6mUoy2Hr/1Jk986s+nYAbNjuYX0nNGHAfu11ge11teAyUC7W7ZpB4xzfD0diFBKKQClVHvgEBDnlMQmWr9tNEkWRXiZx82OIoS4DV/vbEztPITeZT4kRZ0icnEPRscsMTuW6dJT6IsBR294HO947rbbaK3TgAQgQCmVC3gL+PhuO1BK9VdKxSilYs6cOZPe7Bku6uAicto1tav1NTuKEOIu3mjQiZ/Dx+Olc/PNjjcYuOiHLD0F09UXYz8Chmmt77qSgNZ6hNY6VGsdWqBAARdHejC2tGssTz5OA29/svn4mR1HCHEP9UpVYlGXqeRTwSw7/TNtJg/y/LbHd5CeQn8MKHHD4+KO5267jVLKC8gDnANqA18qpQ4DA4F3lFIvPlxkc2yLm8x5iyKiZITZUYQQ6VTYLx/Le42lao4nOJIaRfjv3Tlw7qTZsTJcegr9RqC8Uqq0Uiob0A2Ye8s2c4E+jq87AVHa0EBrHai1DgS+Bf6jtf7eOdEz1rK9M/DWmvrVpfe8EJmJl9XKpM6f0LnkW1zhEE/M7sofezebHStD3bPQO8bcXwQWA7uAqVrrOKXUJ0qpto7NRmGMye8HXgX+NQUzM9N2O1GXDlJb5SKXXxGz4wghHsAHTXrxSdiPoGy8vuYZvl072+xIGSZdcwS11guBhbc898ENXycDne/xGR89QD63sPfAIuKt8FSROmZHEUI8hA5V6lCpwGR6zX+OX/d+QHziSYa2eM7sWC4nd8amQ1Tc7yitaSLDNkJkepUKFmdJt0nkJYjFp36gx/QPPH5GjhT6dFh+IY5qZCN/gUpmRxFCOEFADj/+7DmGkt5N2H5lFi1+e8GjZ+RIob+HY8c2sMtiIzx/dbOjCCGcyNc7G/O6fUto7u6csK/hsd+f9NgOmFLo72F57GgAwoMizQ0ihHA6i8XCmCfeoW3RQVxSe2g5tTs7Th4xO5bTSaG/h2WnYihnU5Qq1cDsKEIIFxnS9CleqvxfUi1n6bmgNxvj95sdyamk0N/FhfMH2Ewy4flkbF4IT/dsWCs+CfsOu0qi35K+RB/aaXYkp5FCfxcrtvyCXSnCK0lLYiGygg5V6vBV/Z/RpPHC8qdZum+r2ZGcQgr9XUQdW01hm6byI+3NjiKEyCAtHqnBd41HorDwavSzzNu1wexID00K/R0kJZ1lnS2R8FyBsmSgEFlM4zJVGfHYKJT25Z11A5gSG212pIciFewO1m4eSYpFEV6u7b03FkJ4nEdLVmBcy7FYtB+fbhrI2E1/mh3pgUmhv4Oov5eS266pGdTb7ChCCJNUL1qayW3G4233Z2jsm0yOXWV2pAcihf42UlOTWHHtNI19CuLl7Wt2HCGEiSoVLM6ktmOw2vMyJOY1FuyJMTvSfZNCfxubYidwyaIID2xudhQhhBuoWKA4o1uMRGlfBq95iVWHMtfKqFLob2PZ/jn42DV1Q54xO4oQwk3ULFaW78N/BuDF5c9lqoXHpdDfQtvtRF05Ql1rHrLn8Dc7jhDCjTQsXYXP632HJpmn/niG3WfizY6ULlLob7FzzyxOWxURxaXlgRDi3x6vEMq7oV9js1yk+9x+HLl4xuxI9ySF/hbLdk3GqjWNqj9rdhQhhJvqFtyQV6r+l1TLKTrM7MeFpMtmR7orKfS3iLq4h5r4kjdfabOjCCHc2DO1mtOn3HskWw7TcfpAt168RAr9DQ4fXskBqya8UC2zowghMoE3GnSiTr4nOaPX02/O52bHuSMp9DeI2j4OgPBqT5ucRAiRWfzS5jUKW+oRkziJL1dNNTvObUmhv8Gys1upZLdSpGhNs6MIITIJi8XC9M7f4GMrzfgDnzNv10azI/2LFHqHM6fjiLWkEu5f1ewoQohMJo9vDia2+RGLzsm7a191u2mXUugdlm8ZAUBElV4mJxFCZEYVCxRnSJ1vsFuu0GveCyQkJ5kd6Top9A5RJ/+ihA3KlWlmdhQhRCbVplItepV9ixTrITpPf81tZuJIoQcuJR5jvb5CRO5y0nteCPFQBjfsSg2/bpywrebNJSPMjgNIoQcgessI0pQi/JGOZkcRQniAUe0Gk8teiT9OjHCLtWel0ANRR5cTYNMEV+5idhQhhAfwsloZ2XIoSnsxaPlbJKWmmJonyxf6lOQEolPP0zh7Uaxe2cyOI4TwEFULl6R72UGkWA/z3LyvTM3iZere3cD6bWNJsigiyrY2O4pbSE1NJT4+nuTkZLOjiEzG19eX4sWL4+3tbXYUt/FOo+4sP7KSzYnTmL69CZ2C6pmSI8sX+qiDC8hh19Su1tfsKG4hPj4ePz8/AgMDUUqZHUdkElprzp07R3x8PKVLS5+oG41r9x9aTGvPpxvep0nZOQTk8MvwDOkaulFKtVBK7VFK7VdKDb7N6z5KqSmO19crpQIdzzdVSm1SSm13/B3u5PwPxZZ2jeXJx2ng7U82n4z/5ruj5ORkAgICpMiL+6KUIiAgQP4leBtFc/vzWvUPsVnP0mf2+6ZkuGehV0pZgR+AlkBloLtSqvItm/UDLmitywHDgC8cz58F2mitg4A+wARnBXeG2J1TOW9RRJRwq98/ppMiLx6E/NzcWZ8aEVTO0Ya/U5cxfO2cDN9/es7ow4D9WuuDWutrwGSg3S3btAPGOb6eDkQopZTWeovW+rjj+Tggu1LKxxnBnWHZnul4aU396rJkoBDCtUa1ew9vW3FG7v6cfWdPZOi+01PoiwFHb3gc73jutttordOABCDglm06Apu11v+aZ6SU6q+UilFKxZw5kzGrtWi7nahLB6itcuKX+9bDEUII5/Lzyc4XDf6LVld5efF/MnTfGTK9UilVBWM457bLNmmtR2itQ7XWoQUKFMiISOw7uJijVggvUidD9icyj7S0NLMjCA/VtHwIj2RvztHUlSw/uD3D9pueWTfHgBI3PC7ueO5228QrpbyAPMA5AKVUcWAW0Ftr7TbLpkfF/YbSmnBZMvCOPp4Xx87jiU79zMpFc/Nhmyp3fP3w4cO0bNmS+vXrs3btWooVK8acOXPInj07W7du5bnnniMpKYmyZcsyevRo8uXLd/29NpuNcuXKcfDgQRISEggICGD58uU0bNiQhg0bMmrUKC5cuMArr7xCcnIy2bNnZ8yYMVSoUIGxY8cyc+ZMLl++jM1mo2/fvsyePZsrV66wb98+Xn/9da5du8aECRPw8fFh4cKF+Pv707hxY4YOHUpoaChnz54lNDSUw4cPM3bsWGbNmkVCQgLHjh2jV69efPjhh079XorMaWizV2k7ZxkfRQ+lSZlx936DE6TnjH4jUF4pVVoplQ3oBsy9ZZu5GBdbAToBUVprrZTKCywABmut1zgps1NEnd9BsM5G/gKVzI4ibrFv3z4GDBhAXFwcefPmZcaMGQD07t2bL774gtjYWIKCgvj4449vep/VaqVChQrs3LmT1atXU6NGDaKjo0lJSeHo0aOUL1+eihUrEh0dzZYtW/jkk0945513rr9/8+bNTJ8+nZUrVwKwY8cOZs6cycaNG3n33XfJkSMHW7ZsoU6dOowfP/6ex7FhwwZmzJhBbGws06ZNIyYmxonfJZFZlfEvRK28HTjPZqbERmfIPu95Rq+1TlNKvQgsBqzAaK11nFLqEyBGaz0XGAVMUErtB85j/DIAeBEoB3yglPrA8VwzrfVpZx/I/Th+PIZdFhuvBsgCI3dztzNvVypdujQhISEA1KxZk8OHD5OQkMDFixdp1KgRAH369KFz587/em+DBg1YtWoVhw4d4u2332bkyJE0atSIWrWM5SETEhLo06cP+/btQylFamrq9fc2bdoUf3//64+bNGmCn58ffn5+5MmThzZt2gAQFBREbGzsPY+jadOmBAQYl6o6dOjA6tWrCQ0NfbBvivAoQ5u/ROPJ8/lm0zA6V62HxcXNFNP16VrrhVrrR7TWZbXWQxzPfeAo8mitk7XWnbXW5bTWYVrrg47nP9Na59Rah9zwx9QiD7B82ygAwoMizQ0ibsvH538Ts6xW632NmTds2JDo6Gg2bNhAq1atuHjxIitWrKBBgwYAvP/++zRp0oQdO3Ywb968m+Z958yZ8445LBbL9ccWi+V6Ji8vr+utaG+dQ37rdEOZfij+EZDDj6ZFepJk2cePGxa4fH9ZstfNslMbKWdTlCrVwOwoIp3y5MlDvnz5iI42/qk7YcKE62f3NwoLC2Pt2rVYLBZ8fX0JCQnhl19+oWHDhoBxRl+smDHLauzYsQ+dKzAwkE2bNgEwffr0m15bunQp58+f5+rVq8yePZt69cy5/V24p88eexpLWgFGxX3PNRdPAMhyhf7ihUNsIpkmeSuaHUXcp3HjxvHGG28QHBzM1q1b+eCDD/61jY+PDyVKlODRRx8FjKGcS5cuERQUBMCbb77J22+/TfXq1Z0yu+b111/np59+onr16pw9e/am18LCwujYsSPBwcF07NhRhm3ETXJ4+9Cl7NOkeR1nyMrfXLovpbV26Q7uV2hoqHblRavZy97i/fiFTK79MVUqdnDZfjKrXbt2UamSXKB+WGPHjiUmJobvv//e7CgZSn5+7k+azUbtcW1I1VdY0+sP/HyyP/BnKaU2aa1vezaR5c7oo46tprBNU/mR9mZHEUJkcV5WK88FvYL2Os/gpb+4bD9ZqtAnJZ1lrS2B8FyBsmSgcKnIyMgsdzYvHswztZqTy16JVacnc/LSBZfsI0tVu3VbfiXFoggv19bsKEIIcd3g2q+B9QqvLxnuks/PUoU+6u8l5LZragb1NjuKEEJc165ybUp4NSand857b/wAsszCI6mpSaxIOU1jn4J4efuaHUcIIW6ysOd3LvvsLHNGv3n7RBItivBSzcyOIoQQGSrLFPpl++bga9fUrd7f7ChCCJGhskSh13Y7UVf+po41D9lz+N/7DUJksO7duxMcHMywYcP44IMP+PPPPx/4s4YMGUJISAghISFYrdbrXw8f7poLfcL9ZYkx+p17ZnHKqnipuLQ8uC+LBsNJJ/fMLhwELT937mfewmazYbVaXbqP+5WWloaX1+3/dzt58iQbN25k//79TtnXu+++y7vvvgtArly52Lp1602va63RWru8kZZwH1niv/SyXVOwak0j6T3v9r766qvrZ56DBg0iPNxYzzcqKoqePXvy/PPPExoaSpUqVW7q7x4YGMhbb71FjRo1mDZtGoGBgbz99tuEhIQQGhrK5s2bad68OWXLluXnn38GYMWKFbRu3fr6Z7z44ovX+98EBgby5ptvEhQURFhY2F2LcGRkJM899xyhoaE88sgjzJ8/HzDujm3bti3h4eFERERw5coVnnrqKcLCwqhevTpz5hhrhzZr1oxjx44REhJCdHQ0kZGRTJ8+nYSEBCpUqMCePXsA46x/5MiRD/R9PXz4MBUqVKB3795UrVqVo0ePkitXruuvT58+ncjISADOnDlDx44dqVWrFrVq1WLNGrfqMC4eQJY4o19+cTc1Lb7kzVfa7CiZi4vPvG+nQYMGfP3117z88svExMSQkpJCamoq0dHRNGzYkM6dO+Pv74/NZiMiIoLY2FiCg4MBCAgIYPPmzQAMHjyYkiVLsnXrVgYNGkRkZCRr1qwhOTmZqlWr8txzz90zS548edi+fTvjx49n4MCB1wv47Rw+fJgNGzZw4MABmjRpcv0Xw+bNm4mNjcXf35933nmH8PBwRo8ezcWLFwkLC+Oxxx5j7ty5tG7d+vqZ96hRo67v//vvvycyMpJXXnmFCxcu8MwzxvrGXbt2vf4L4EavvvoqvXvffvrwvn37GDdu3PU+QHfyyiuvMGjQIOrXr8+RI0do3rw5u3btuuf3S7gvjy/0f/8dzX6rZnDBWmZHEelQs2ZNNm3aRGJiIj4+PtSoUYOYmBiio6MZPnw4U6dOZcSIEaSlpXHixAl27tx5vdB37dr1ps9q29a4MS4oKIjLly9f7y3v4+PDxYsX75mle/fu1/8eNGjQXbft0qULFouF8uXLU6ZMGXbv3g3c3ON+yZIlzJ07l6FDhwJGW+MjR46QPfud+5s0bdqUadOmMWDAALZt23b9+SlTptwz/61KlSp1zyIP8Oeff7Jz587rjxMTE7l8+fJN/wIQmYvHF/qo7WMBCK/2tLlBRLp4e3tTunRpxo4dS926dQkODmb58uXs37+f7NmzM3ToUDZu3Ei+fPmIjIxMVz/5G3vJ//P4nzHzf3rJw937yd+rl/ydes/fmElrzYwZM6hQocJN2x4+fPiOn2u329m1axc5cuTgwoULFC9eHHiwM/pbvz83Zr7x2O12O3/99Re+vnK/iafw+DH6ZWe2UMlupUhRWU0qs2jQoAFDhw6lYcOGNGjQgJ9//pnq1auTmJhIzpw5yZMnD6dOnWLRokUPtZ9SpUqxc+dOUlJSuHjxIsuWLbvp9X/OmqdMmUKdOndfRH7atGnY7XYOHDjAwYMH/1XMAZo3b853333HPx1jt2zZcs+Mw4YNo1KlSvz+++/07dv3+opYU6ZMYevWrf/6c6cifzuFChVi165d2O12Zs2adf35Zs2a8d13/7t559aLuSLz8egz+jOn44hV1xiQr7rZUcR9aNCgAUOGDKFOnTrkzJkTX19fGjRoQLVq1ahevToVK1akRIkSD72QR4kSJejSpQtVq1aldOnSVK9+88/JhQsXCA4OxsfHh0mTJt31s0qWLElYWBiJiYn8/PPPtz0bfv/99xk4cCDBwcHY7XZKly5913H/PXv28Ouvv7Jhwwb8/Pxo2LAhn3322b/Wyn1Qn3/+Oa1bt6ZAgQKEhoZy+fJlAIYPH86AAQMIDg4mLS2Nhg0bXr+ALTInj+5HP3XJQD49sYyZ9b6ifLkWTvlMTyf9xA2BgYHExMSQP3/+e24bGRlJ69at6dSpUwYkc2/y82OeLNuPPurEOkrYoFwZaXsghMi6PHbo5lLiMdbrK/TKXV56z4v7drsLpEOGDGHatGk3Pde5c2enrD0rhCt5bKFfvWUkaUoR/khHs6MID3HjHadCZCYee6q77GgUATZNcOUuZkcRQghTeWShv5ZyiejU8zTOXhSrVzaz4wghhKk8stD/tXU0SRZFRNnW995YCCE8nEcW+qiDC8hh19Su1tfsKEIIYTqPK/S2tGssTz5OA29/svn4mR1HiHRxZj96gI8++ohixYpd70U/ePDgu277T/8d4Zk8btZN7M6pnLcoIkqEmx0l0/tiwxfsPr/bqZ9Z0b8ib4W95dTPvFVW70f/j0GDBvH666879TNF5uRxZ/RRe2fgpTX1qz9jdhTxAKQfvWv60QOMHDmSWrVqUa1aNTp27EhSUtK/thk+fDiVK1cmODiYbt26Adwxt8g8POqMXtvtLEvcT21rTvxyFzM7Tqbn6jPv25F+9M7rRz9s2DAmTpwIwBdffEGHDh2uv/+9995j1KhRvPTSSze9//PPP+fQoUM3tXIeMmTIbXPf2g1TuC+PKvT7Dy7hqBUii9y906BwX9KP/mYP04/+1qGblStX8t5773Hx4kUuX75M8+bN//We4OBgevbsSfv27Wnfvv1dc0tPm8wjXYVeKdUC+D/ACvyqtf78ltd9gPFATeAc0FVrfdjx2ttAP8AGvKy1Xuy09LdYFjcRpTVNQmTYJrOSfvQ3c2Y/+sjISGbPnk21atUYO3YsK1as+Nc2CxYsYNWqVcybN48hQ4awffv2O+YWmcc9x+iVUlbgB6AlUBnorpSqfMtm/YALWutywDDgC8d7KwPdgCpAC+BHx+e5RNT5HQTrbBQoWMVVuxAZQPrR/48z+9FfunSJIkWKkJqaym+//fav1+12O0ePHqVJkyZ88cUXJCQkXD/zv9/cwr2k54w+DNivtT4IoJSaDLQDdt6wTTvgI8fX04HvlXFK0w6YrLVOAQ4ppfY7Pm+dc+L/z/HjMeyy2BjkX8PZHy0ymPSjNzi7H/2nn35K7dq1KVCgALVr1+bSpUs3vW6z2ejVqxcJCQlorXn55ZfJmzfvfecWbkhrfdc/QCeM4Zp/Hj8JfH/LNjuA4jc8PgDkB74Het3w/Cig02320R+IAWJKliypH8T+/Uv0S+Pq6sOHVz3Q+4Vh586dZkdwC6VKldJnzpxJ17Z9+vTR06ZNc3GizEF+fswDxOg71HG3uBirtR4BjABj4ZEH+YyyZZsyvGxTp+YSQghPkJ5CfwwoccPj4o7nbrdNvFLKC8iDcVE2Pe8Vwu1IP3rhSdJT6DcC5ZVSpTGKdDegxy3bzAX6YIy9dwKitNZaKTUX+F0p9Q1QFCgPbHBWeOEaWut7zjLJiqQf/d1pN1uWVPzPPQu91jpNKfUisBhjeuVorXWcUuoTjDGhuRhj7xMcF1vPY/wywLHdVIwLt2nAAK21zUXHIpzA19eXc+fOERAQIMVepJvWmnPnzt32IrQwn0cvDi7uX2pqKvHx8f+aUy7Evfj6+lK8eHG8vb3NjpIl3W1xcLe4GCvcxz83LAkhPIfHNTUTQghxMyn0Qgjh4aTQCyGEh3O7i7FKqTPA3w/xEfmBs06Kk5nIcWctctxZS3qOu5TWusDtXnC7Qv+wlFIxd7ry7MnkuLMWOe6s5WGPW4ZuhBDCw0mhF0IID+eJhX6E2QFMIsedtchxZy0PddweN0YvhBDiZp54Ri+EEOIGUuiFEMLDeUyhV0q1UErtUUrtV0oNNjuPqyilRiulTiuldtzwnL9SaqlSap/j73xmZnQFpVQJpdRypdROpVScUuoVx/MefexKKV+l1Aal1DbHcX/seL60Umq94+d9ilIqm9lZXUEpZVVKbVFKzXc8zirHfVgptV0ptVUpFeN47oF/1j2i0KdzAXNPMRZjofUbDQaWaa3LA8scjz1NGvCa1roy8CgwwPHf2NOPPQUI11pXA0KAFkqpR4EvgGFa63LABaCfeRFd6hVg1w2Ps8pxAzTRWofcMH/+gX/WPaLQc8MC5lrra8A/C5h7HK31Koye/zdqB4xzfD0OaJ+RmTKC1vqE1nqz4+tLGP/zF8PDj92xHOhlx0Nvxx8NhAPTHc973HEDKKWKA48DvzoeK7LAcd/FA/+se0qhLwYcveFxvOO5rKKQ1vqE4+uTQCEzw7iaUioQqA6sJwscu2P4YitwGlgKHAAuaq3THJt46s/7t8CbgN3xOICscdxg/DJfopTapJTq73jugX/WpR+9h3Es4eixc2aVUrmAGcBArXXijatgeeqxO1ZlC1FK5QVmARXNTeR6SqnWwGmt9SalVGOT45ihvtb6mFKqILBUKbX7xhfv92fdU87os/oi5KeUUkUAHH+fNjmPSyilvDGK/G9a65mOp7PEsQNorS8Cy4E6QF6l1D8nap74814PaKuUOowxFBsO/B+ef9wAaK2POf4+jfHLPYyH+Fn3lEJ/fQFzx1X4bhgLlmcV/yzOjuPvOSZmcQnH+OwoYJfW+psbXvLoY1dKFXCcyaOUyg40xbg+sRzo5NjM445ba/221rq41joQ4//nKK11Tzz8uAGUUjmVUn7/fA00A3bwED/rHnNnrFKqFcaY3j8LmA8xN5FrKKUmAY0x2paeAj4EZgNTgZIYLZ67aK1vvWCbqSml6gPRwHb+N2b7DsY4vcceu1IqGOPCmxXjxGyq1voTpVQZjDNdf2AL0EtrnWJeUtdxDN28rrVunRWO23GMsxwPvYDftdZDlFIBPODPuscUeiGEELfnKUM3Qggh7kAKvRBCeDgp9EII4eGk0AshhIeTQi+EEB5OCr0QQng4KfRCCOHh/h9hQ8FKrY/pOgAAAABJRU5ErkJggg==", + "text/plain": [ + "
    " + ] + }, + "metadata": { + "needs_background": "light" + }, + "output_type": "display_data" + } + ], + "source": [ + "scheduler = PolyLRScheduler(optimizer, t_initial=t_initial)\n", + "plot_lr(scheduler, label='no warmup')\n", + "\n", + "scheduler = PolyLRScheduler(optimizer, t_initial=t_initial, warmup_t=10, warmup_prefix=True)\n", + "plot_lr(scheduler, label='warmup_prefix=True')\n", + "\n", + "scheduler = PolyLRScheduler(optimizer, t_initial=t_initial, warmup_t=10)\n", + "plot_lr(scheduler, label='warmup_prefix=False')\n", + "plt.legend();" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Noise Args." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### `noise_range_t`" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "If it is number - its number of epoch when noise starts. \n", + "If list or tuple (of two elements) - first and second element is epoch number range, when noise applied." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "The upper and lower limit of noise. " + ] + }, + { + "cell_type": "code", + "execution_count": 21, + "metadata": { + "scrolled": true + }, + "outputs": [ + { + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXoAAAD4CAYAAADiry33AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/YYfK9AAAACXBIWXMAAAsTAAALEwEAmpwYAABeQ0lEQVR4nO2dd5hU5dn/P8/U3dneKbuwlAUEQZSqoIgVjSW+wVhiLG8SSzQxmuSNyS9RY5pGX0uMxpjYEn0xiRWVxI5YUAEFpEhfYCkLbG/Tdp7fH+ec2ZnZKWd2Z/vzuS6unTlzyjPM7vfc833u576FlBKFQqFQDF4sfT0AhUKhUPQsSugVCoVikKOEXqFQKAY5SugVCoVikKOEXqFQKAY5tr4eQCSFhYWyvLy8r4ehUCgUA4o1a9YckVIWRXut3wl9eXk5q1ev7uthKBQKxYBCCLE71mvKulEoFIpBjhJ6hUKhGOQooVcoFIpBTr/z6BWKgY7P56Oqqgq3293XQ1EMQtLS0igtLcVut5s+Rgm9QpFiqqqqyMrKory8HCFEXw9HMYiQUlJTU0NVVRVjxowxfZyybhSKFON2uykoKFAir0g5QggKCgqS/raohF6h6AGUyCt6iq78bimhHwp4mmDdP/p6FAqFoo9QQj8U2PQyvHg11O7q65EoFIo+QAn9UKCtTvvpbujbcSj6LbfeeitvvfVWXw8jpaxdu5Zly5YldUx9fT2LFy9m0qRJHHXUUaxcuRKA2tpaTj/9dCoqKjj99NOpq6vriSH3GErohwLuRu2nt7lvx6Hot9xxxx2cdtppKT+v3+9P+TnN0hWhv/HGG1m0aBFffvkl69at46ijjgLgzjvv5NRTT2Xbtm2ceuqp3HnnnT0x5B5DpVcOBTxN4T8VvcYvX9nIpv2NKT3n5BHZ3HbulLj7VFZWctZZZzF//nw++ugjRo4cycsvv8yWLVu49tpraW1tZdy4cTz++OPk5eVx5ZVXcs4557B48WJuueUWli5dis1m44wzzuCee+7h8OHDXHvttezZsweA+++/n3nz5kW99u23386OHTvYuXMno0aN4ne/+x3f/OY3aWlpAeCPf/wjJ5xwAsuXL+f222+nsLCQDRs2MGPGDJ5++mmEECxbtoybb76ZjIwM5s2bx86dO3n11VdpaWnhe9/7Hhs2bMDn83H77bdz/vnndxqD1+vl1ltvpa2tjQ8++ICf/vSnXHTRRXH/zxoaGlixYgVPPvkkAA6HA4fDAcDLL7/M8uXLAbjiiis4+eSTueuuu+Kerz+hhH4o4NGFxqMi+qHEtm3bWLJkCX/5y1/4+te/zvPPP8/vf/97HnzwQRYsWMCtt97KL3/5S+6///7gMTU1Nbz44ot8+eWXCCGor68HtEj3pptuYv78+ezZs4czzzyTzZs3x7z2pk2b+OCDD0hPT6e1tZU333yTtLQ0tm3bxiWXXBIsXPj555+zceNGRowYwbx58/jwww+ZOXMm11xzDStWrGDMmDFccsklwfP+5je/4ZRTTuHxxx+nvr6e2bNnc9ppp5GRkRF2fYfDwR133MHq1av54x//CMC7777LTTfd1GmsLpeLjz76iF27dlFUVMRVV13FunXrmDFjBg888AAZGRlUV1czfPhwAIYNG0Z1dXWXPpO+wpTQCyEWAQ8AVuCvUso7I14/CbgfmAZcLKV8LuL1bGAT8JKU8oYUjFuRDIY371URfW+TKPLuScaMGcP06dMBmDFjBjt27KC+vp4FCxYAWmR64YUXhh2Tk5NDWloa3/rWtzjnnHM455xzAHjrrbfYtGlTcL/Gxkaam5vJzMyMeu3zzjuP9PR0QFspfMMNN7B27VqsVitbt24N7jd79mxKS0sBmD59OpWVlWRmZjJ27NjggqBLLrmERx99FIA33niDpUuXcs899wDamoU9e/YELZZ4LFy4kLVr18Z83e/389lnn/Hggw8yZ84cbrzxRu68805+9atfhe0nhBhw6bMJhV4IYQUeAk4HqoBVQoilUspNIbvtAa4EfhTjNL8CVnRvqIouk6x14/fAX0+FM34DYxf03LgUPYrT6Qw+tlqtweg8HjabjU8//ZS3336b5557jj/+8Y+88847BAIBPv74Y9LS0kxdOzTCvu+++ygpKWHdunUEAoGwc0SOMZGnL6Xk+eefZ+LEiabGEUqiiL60tJTS0lLmzJkDwOLFi4NefElJCQcOHGD48OEcOHCA4uLipK/fl5iZjJ0NbJdS7pRSeoFngTBTTEpZKaVcDwQiDxZCzABKgDdSMF5FV0jWummthYNfQPWGnhuTotfJyckhLy+P999/H4C///3vwejeoLm5mYaGBs4++2zuu+8+1q1bB8AZZ5zBgw8+GNwvXmQcSUNDA8OHD8disfD3v/+d9vb2uPtPnDiRnTt3UllZCcA//tGxBuTMM8/kwQcfREoJaNZPLLKysmhq6ghujIg+8t9HH30EaJZMWVkZW7ZsAeDtt99m8uTJgPYN5amnngLgqaeeijov0J8xI/Qjgb0hz6v0bQkRQliA/yV2pG/sd7UQYrUQYvXhw4fNnFqRDEYkbzbrxq8vr2739cx4FH3GU089xY9//GOmTZvG2rVrufXWW8Neb2pq4pxzzmHatGnMnz+fe++9F4A//OEPrF69mmnTpjF58mQeeeQR09f87ne/y1NPPcUxxxzDl19+2clPjyQ9PZ2HH36YRYsWMWPGDLKyssjJyQHgF7/4BT6fj2nTpjFlyhR+8YtfxDzPwoUL2bRpE9OnTw+7WcTjwQcf5Bvf+Ebw/+dnP/sZALfccgtvvvkmFRUVvPXWW9xyyy0m333/QBh3xpg7CLEYWCSl/Lb+/JvAnGheuxDiSeBVw6MXQtwAuKSUvxdCXAnMTOTRz5w5U6oOUynm7gpoOQTHXgbnP5R4/0Ob4eG5sPDnsODHPT++QcbmzZtNecaK2Bj+v5SS66+/noqKiqi2y1Al2u+YEGKNlHJmtP3NRPT7gLKQ56X6NjMcD9wghKgE7gEuF0IMrATUwUCy1o0R0QdSENG3HIGaHd0/j2JI8Ze//IXp06czZcoUGhoauOaaa/p6SAMaM1k3q4AKIcQYNIG/GLjUzMmllN8wHodE9APrO89Ax+/tEG6z1o3PsG683b/+O7+G3R/CDau6fy5Fv+KJJ57ggQceCNs2b948HnrIxLfGBNx0002mI/jXX3+dn/zkJ2HbxowZw4svvtjtcQwWEgq9lNKvWzCvo6VXPi6l3CiEuANYLaVcKoSYBbwI5AHnCiF+KaXsu7wyRQehmTams25S6NG3HoHmQ90/j6LfcdVVV3HVVVf19TA488wzOfPMM/t6GP0aU3n0UsplwLKIbbeGPF6FZunEO8eTwJNJj1DRPTwh9W2Stm5SsHzd2wrelu6fR6FQdBlV62awY0TxjizzC6b8Jq2bncuhemP8fXytmtfv95i7tkKhSDlK6Ac7RkGz7BFJRPS6KCeybl65ET64L/4+RjSvyi8oFH2GEvrBjidU6E1G9L427WcioW+t69g35rlatZ+q/IJC0WcooR/sGOKePdK8hWLsEy+9MhDQbiKJ7B2vLvQqou/XqHr0WsXP9PT0YH0ggP/+7/+muLiYo48+Omzfrtanb2xspLS0lBtu6FhOtGbNGqZOncr48eP5/ve/H1z1++Mf/5hhw4YF6/p0ByX0g51Q6wbMCa7fRETvaQBk4huHT7duVC38fo2qR68xbty4sPIOV155Jf/5z3867dfV+vS/+MUvOOmkk8K2XXfddfzlL39h27ZtbNu2LXi9u+++m2uvvTap8cdClSke7BhZN9nD9eeNkFEQ/xgzHr1REVNF9PH59y1a3aBUMmwqnBVbWGLVok9PT2ft2rWqHn0SnHTSScGaO6F0pT79mjVrqK6uZtGiRcEyzQcOHKCxsZG5c+cCcPnll/PSSy9x1llnJT3WeCihH+x4msDqBFeh9txMZG347vGsm7Z67Wc8oW/3dZxDefS9SrRa9JdddhmXX365qkcfgVG9MhmSrU8fCAT44Q9/yNNPPx1mke3bty9YphmgtLSUffvMFh4wjxL6wY67EZxZ4NTrhpuyboyIPo6IGxG9P84+ofnzQzWijxN59ySRtegrKytpaGhQ9eiTqLppFjP16R9++GHOPvvsMFHvTZTQD3Y8jZCWreXRg7nMm2AefRx/1V2v7xPHozcybkB59L1MZJ33trYE2VE6Q7kefTIkW59+5cqVvP/++zz88MM0Nzfj9XrJzMzkxhtvpKqqKrhfVVUVI0eaKg6cFGoydrDjaQJnthbVgzkLxcyCKTMevVcJfX9C1aOPX48+GWLVp//000+5/PLLO+3/zDPPsGfPHiorK7nnnnu4/PLLufPOOxk+fDjZ2dl8/PHHSCn529/+1iO17pXQDxQOrIeqNckf1yXrxkT1SsOjj2fd+JR1099Q9eiT45JLLuH4449ny5YtlJaW8thjjwGx69Pv2bMnaFmZ5eGHH+bb3/4248ePZ9y4cSmfiAW0r0L96d+MGTOkIgpPfEXKO4qk3LkiueMemivlkkulbK2T8rZsKT98MPExz1yk7fvQ8bH3eesObZ+7xsTep/IjbZ/bsqV89ebkxj2A2bRpU18PYcDT1NQkpZQyEAjI6667Tt577709fs1du3bJKVOmdOscP/rRj+S6detSNCIpb7vtNnn33Xd32h7tdwytyGRUXVUR/UChrV7zw5dcAvtjf13thGHdOPSI3oyFYsq6qdf3VRG9IvX0RT16q9VKQ0ND2IKpZLn77ruZNm1aSsbz4x//mKeffjrhNyAzqMnYgYKnEcacBLWV8PTX4Kr/QNGExMcZ1o3VBnZXcpOx8aybpDx6oTz6Qchgq0dfVlbG3r17E+/YS9x9993cfffdKTmXEvqBgrcZCifAOffD42fC0u/Bt16Pf4xRpiAtW3vuyEwyojeTR+8BKSFaepmRdZNRZL7OziBBSpkw5W6go+rR9w0yQfvXaCjrZqDgadIi84JxMO5UaNyf+BhfCyA16wa0CVlTEX0SK2Pj7Wfk0WeWDKmIPi0tjZqami79QSoU8ZBSUlNTYzrN1UBF9AMBv0ezSAyf3ZlpLk3SqHNjpFY6s8x55WZWxhoePWhjszminEeP6DOLoaGq8+uDlNLSUqqqqjh8+HBfD0UxCElLS0t64ZUS+oGAEYU7Qy0YE12bjBLFQesmy6R1k2xEH8On94YI/eEvE193kGC324OrOhWK/oCybgYCnojI3JGpiWu8jBcIuUHk6D8zO84VDzPVK9vqO84bq4Klr0Wrs5OWo7JuFIo+RAn9QCAo2IYFYzJVsqvWTaJaN742bRI2syj+ft5WcLjAkaFZTcqzVij6BFNCL4RYJITYIoTYLoS4JcrrJwkhPhNC+IUQi0O2TxdCrBRCbBRCrBdCJF8nVNEhzsGIXs+rTST0nawbE1k3UnZk3SAhEGW5umHbZOj1PWLeEFrBnqFdVwZCzpsEoWUUFApFl0go9EIIK/AQcBYwGbhECDE5Yrc9wJXA/0VsbwUul1JOARYB9wshcrs55qFHZEQfXPyUwKcPWj5JZN20+zRRNoqgRRNxI7UyUxf6WNaNt0WL6I1xJ2vfHPoSflcKh7cm3lehUMTETEQ/G9gupdwppfQCzwJhVXeklJVSyvVAIGL7VinlNv3xfuAQUJSSkQ8lYgl9IuGMtG4cWVpUHa8qpRF1G/ZQNJ/eiOgNoY9VwdLXqi3SCt6Yksylb6wC2Q6Nqa/PrVAMJcwI/UggdLlYlb4tKYQQswEHsCPKa1cLIVYLIVarlLQoRE7GOk0Kp6cJECFpmSYqWAaFXt83EOWmYKRWBq2bWHn0rZrNlExBtVCM8yZqUq5QKOLSK5OxQojhwN+Bq6SUgcjXpZSPSilnSilnFhWpgL8TnSJ6w6M3Yd04s8Gif8xBwU1C6KNZN8GIXv+s4mXdhEX0yQq9fu14+fwKhSIhZoR+H1AW8rxU32YKIUQ28Brw/6SUHyc3PAWgCaSwaKIJyVk3hmCbPc4XKfRRRDbo0Zfo+yTIuumqRx+M6BOkkSoUiriYEfpVQIUQYowQwgFcDCw1c3J9/xeBv0kpn+v6MIc4RvkDo3aK2Qg5tM4NdEzKxjsumYg+I0F6ZWjWDSTv0RvnVdaNQtEtEgq9lNIP3AC8DmwG/iml3CiEuEMIcR6AEGKWEKIKuBD4sxBio37414GTgCuFEGv1f9N74o0MaoxSwwZm8+gN6ybyuLjWjW7DGMfF8ujtrg4LKWHWTVc9em/4T4VC0SVMlUCQUi4DlkVsuzXk8So0SyfyuKeBp7s5RoUnwoKxu9BK/ybw6N2NHVE3hFg38YS+LXzfqFk39ZCWC1a9vk28hVXd8uiVdaNQpAK1MnYg4GnqEEvQLBxHZuII2dMUYd0YWTfxrBtP+L6x8ujTcsDmDD8mlEBAu2k4MszPKUQSjOjjN4xWKBTxUUI/EDA8+lAcGV2wbkxMihqVK+OmVzZAem78iN6oXGl3aU1PbGnd8OhVRK9QdAcl9AMBT3NnoXeaKGcQM+vGjEcfbzK2XovoTQl9ese1zVTcDEVZNwpFSlBCPxCIGtEnEE6/R1uxGmrd2ByaOMddMGVE9PpxsVbGpuXGt26MsRkTtk4TVlMkKutGoUgJSugHApFZNxDdo9/2Fjz335qnHVmi2CBRBctOEX20PPqGiIg+yj6h1g2Yr4UfilowpVCkBCX0/Z1AQIvAzVg329+EDc/D+n905LpH+yZgamWsbvNEiqzRhzY9FyxWENbotW6MqpNhEX2yHr2ybhSKVKCEvr9jiLkzM3x7tMnYtjrt53t3QWut9jgt4puAM0FknWhlrKcBkFpED5p9E8268enWTehq3q5G9Mq6USi6hRL6/k5knRuDaB59W52W3VK/Gz5+WD8uitAniuiFtUOgI0XW+KaQlqv9tDqiC3EwotfP0y2PXkX0CkV3UD1j+zvBiD6aBRMloh81VxPZjS/EPq4lToVQv1u7WVj0X41I68aoc2NE9FZHdOsm6NFndFxXLZhSKPoEFdH3dyIbgxs4MzV7JBBSDLStDtLz4ZSfd2zrZN0kEFy/G+xpsVMnjYg+PVf7aXNG710bzLoxInqTbQxDUQumFIqUoIS+vxNZi97AmOT0hdg3bXWQngdjF0D5ifpxEVk3iVbUGhG91a4972Td1Gs/TUf0ER59Mn1jlXWjUKQEZd30d+J59KBFzs4sLbI3hB7gnPtg44vgyg8/zpmdeDI2VOgjV8ZG9ejjRfQhWTdIfbyZnfePhrJuFIqUoCL6/o4h9I7IrJuI+jHeJq3XqyH0hRWw4H86ShsbGNZNoFP/F42gR29E9BEiG+nR2xzRrRtfqzapa1hAZvvchqKybhSKlKCEvr8TK6KPbCdopFYaQh+LRJUk/W7Nd49p3TRoTVCM8VidsfPoHRnJ19APxbi2WjClUHQLJfT9jb+eBsvv7HjuiZV1E9FO0KzQJ6pg6fdo9WkscTz6tJwOAbc5Y6yMbenw58FcLfxIlEevUKQE5dH3J9p9sG9Nhy0C2mSsLb0jwjZwRFSiNBZIRXrykQQrWMYQXF+blqljsWjWS2Q0bdS5MbDateJpkRhtBIPj7UpEr6wbhSIVqIi+P9GwV/PZ6/d0bItW0AxCInpdOJON6GMJvd+j3Vgg+kSrUYveIJZ1Y7QRDF63CzXp1WSsQpESlND3J+p2az/r93SkIcYS+sh2gkkLfZQoHDo8etCi9cgcdnd9Rw49aJOxUVfGtkRE9CaankSirBuFIiUooe9P1FVqP/1uaD6kPY4Z0UdksQSzYXLjX8NYeBXNbjGubUvTHltsMayb0IjeEaPWTWsKPXq1YEqh6A5K6PsT9btDHuv2TSKh94RE9I5MLcKOh7FSNl5Eb9eFPpp100nonTHy6Fs77KXQ8XYl60ZF9ApFtzAl9EKIRUKILUKI7UKIW6K8fpIQ4jMhhF8IsTjitSuEENv0f1ekauCDkrrd2gQodIh+tBLFENKeL0ToE9k2kDii94VE9NGsG29Lhw0Deh59jOqV9iiTsUl59GoyVqFIBQmzboQQVuAh4HSgClglhFgqpdwUstse4ErgRxHH5gO3ATMBCazRj61LzfA7CLS309LcEHptMjJzEJYB9KWlfjeMPA6qViWO6CG8VHFbXbh3HgtTHn2o0EdE0762jogf4levDPXoLRZtclZ59ApFr2MmvXI2sF1KuRNACPEscD4QFHopZaX+WuRyyzOBN6WUtfrrbwKLgCXdHnkE9TUHyX94cti2w+SxJ/MYfCPnUDTlZMonz8Zq68cZpXW7YdLZULvTpNBnhufRm4noLVYtIo8W0QfaNU8+6NHbwz36dh/I9o6sHIhf6yY06waSbz4Sa8GU3wNLLoHTboPhx5g/n0IxRDGjeiOBvSHPq4A5Js8f7diRkTsJIa4GrgYYNWqUyVOHk56RzccVN3dsaPdjO7KJ0sa1DNuyHLbcReMLLnalTaF1+GxyJy1g7DEn4kxzxTxnr+JphtYjkFcOuaPMC32oR188ydy10rKjR/RGd6kwjz5EZH16P1kjK8d47PdoWULGIiopO2fdGONNRQmEpgOw420Ye7ISeoXCBP0ivJVSPgo8CjBz5swkyht2kJ6Rxdxv3Bb1tQO7t7Bv3bu07/6IkrrPOGbXQ7DrITzL7GxyTKSheBYZFScy9rhTyMw2ERX3BIaw547WhL56k97g2xtb6ENLDpuN6EHz6d0NnbcbXnvQurGFi2zwRhAa0TsBqX0bsNpC9pPhHj1E74oVi0Cgo6BapHVj1NZprjZ3LoViiGNG6PcBZSHPS/VtZtgHnBxx7HKTx6aM4aMnMnz0ROBaAOoOH6Dy87fx7PiA/Jo1zKp6Ctu+J/C/a2GbfRw1BTNxjpvPmONOI7dwWO8M0ph8NSL6Lf/psFcccTx6d4MWQScj9LEi+siIPdK6Cb4e6tEbpRI8HUIf2S/WIJma9MZ1haWz0BtWkRJ6hcIUZoR+FVAhhBiDJtwXA5eaPP/rwG+FEIYCnQH8NOlRppi8ouHknXEZcBkALU31bP78XVq2vk/WoU859uBzOKuXwEewyzKaQ/kzsI2Zx+hjT6dwxOieGZSxWCp3tPav3QO1O7Rt8aybxv2aHRLwaU1HzODMjt5lyojYw1bGhkb0usCGRvTGTcHv6Vwjv1NEnwnNB82N0RB3R6Z2Uwq0a/MLoeNQQq9QmCKh0Esp/UKIG9BE2wo8LqXcKIS4A1gtpVwqhJgFvAjkAecKIX4ppZwipawVQvwK7WYBcIcxMdufyMjKZepJF8BJFwDgcbeyed37NGx+D9fBTzj68DIyjrwAq2CvGMGB3OOwjJlP6bGnM6xsfGoGUb9bE8aMQk3oAao3aD8TefRmV8UaOLO0Cd9IgkJvrIy1dUTnAP5oEX2UTlSR/WKD182EGpMRvXGDsbs0oW/3RRH6Q+bOpVAMcUx59FLKZcCyiG23hjxehWbLRDv2ceDxboyx13GmuThqzpkw50wA/D4vW79YSe2md0nb/zGT6t4lu+5V+OwW9osS9uUcR/uYhYycfxllBRkJzh6Dut2awAuhWTcA1Rv1ASXw6JMV+oSTsaERfX3H676IyVpjHwgX+mBEH/F/kUzf2GBEn9Hx3LiuMU4V0SsUpugXk7H9HZvdwYTjFsBxCwBo9/vZsXk1h794C+e+lYyv/4C8z//NmR8HaM6ZyJyx+cwdU8DcsQWU5acjIpt/RKN+N+TpkXyuPiWSSOiNyc02/UtSUpOx0Tz6iIjeYg/vMBUtog9aN2Yi+iQ8+k5C7+v8WludXoTNiUKhiI0S+i5gtdkYN3Uu46bOBSCwfz08eiI/nWXlH605LN9ymBc+0+arR+SkMXdsAXPHFXD82ALK8qOkc0qp1bkZPU977sgAV6GWeQOdG4MbODI1ITYsjGQi+nZPZ5EMWjehC6ZCJ2MjPHwIiehDcumD/WIjI/oM7bVAQFtAFQ/jusaK2tBvDMY4QXvvuaG5AgqFIhIl9CnAUjgOgJMLGzj5pBlIKdl2qJmPd9bw8c4a3tt6mBc+14R/ZG66Jvxj8zl+XAGleS6tlry3uSOiB82+2f+Z9jieRw8dqZmmI3q9Vo27ETKLOrZ3Sq+MWBlrRPSJrJtgv9go6ZVI7TyRGTmRBIVe3y80+yf024MSeoUiIUroU4EjA7JHQo2WJSOEYEJJFhNKsrj8+PIw4V+5o4Z3vqzm+c+qACjLT+fCYYf4PlDnGE5QqsOEPkYzbWN7g74mzUwJBAgvbBYm9BHWTCfrJuJGAB1F1ELFNxjRRxN69MVUiYTesG70c4RZNyHfHpRPr1AkRAl9qsgfCzXbo74UKfyBgGTroSZW7tCEf+/OFQBc+txBPO8uZ+64Ar7ly2ccaHnkkYJpYIhl/V7NTglNe4xHsLBZxKKpYPpkjIjeyKPvtGCKcPGNlUdvWDneZqA4/hjjWjdK6BWKZFBCnyoKxsOml0ztarEIJg3LZtKwbK6aN4bA++/D23DxGfN5b7eHpWv3Y/FLfm2HJtK599VNnDCukDlj88lOC2kpaCykathr3raB2KWKIxdERXr0kR4+dHj8UbNuYkX0rSQkWtZNcBxK6BWKZFBCnyoKxmtZIK21ifu2RmCp3w3p+VyxcBpXAP72ALs/boE3n8BjyWDJp3t44sNKLAKmluZywrgC5o0rZJY1HSdAQxXkjTF/wVilijt59LFq3URZGeuPjPxF528YkQ3N4xEv68a44TiylNArFCZQQp8qCvSFUzU7khb6sNRKwGa1MK5iMrwJhQWFrLv6DD7fU89HO2r4aPsR/rJiJ39avoNp1r0stQO+VposmaS3B7BZTZRljhXRR0bskR2mokX0Ua0bvRZ9ZFppMs1Hggum4qRX5pSqRVMKhQmU0KeKoNBvh7JZyR1btxuGTQ3fZiyacmbhtFn1TJ0Cbj59As0eP6sqa9m0wQFfaLt9sC/A/9zxJnPGFjBvfAHzxxcyvjgzeg5/zIg+cmVsZHplmybsoamR0fLofa2dM26gmxF9RHql1QlZJSqiVyhMoIQ+VeSN1rpDxZiQjUvjPph4Vvg2hwsyijqi4BAynTYWTixm4cjpQaGfMq6cc7JH8OH2I7y1WRO/4iwn88cXMm98IfMrCinJ1iNxZ5yI3pbWEYlbHVr9eSPv3e8JT62EkKJmEQumok0gG+Lv64pHH5FeaUuDzBLYszLxuRQpZXdNC15/gIqSGGm/in6HEvpUYbVrlSeTFXq/VxPYaE29p1yg2ROxCEm7HDVyBL87XftWsLe2lY92HOH9bUdYHpLDX1GcybzxhZxYUcgp9gxEZBMQnzt8AZVF//UI+MDi1NIvbRG+ezTrxhcjfbIr1k20iL7do6V1ZhZDU3V4LXxFj/ODf6zl8z31nDKpmOsXjmPG6CStSkWvo4Q+lRSMD+bSm8YQvWi58mffHf9YW5qWfikDYVk3ZfkuLsofxUWzRhEISDYfbOSDbUf4YPsRlny6hyc/quRjp4ONa7ex0baN+RWFTBuZg83vjrHq1avdAHzuzhF9tDx6T3MMoe+KdaP/v4TNFXj0iH6YJvruBvNrCBTdpqbZy6h8F5/vqeNrf1rJlSeUc/t5U/p6WIo4KKFPJQXjofJ9c0v8DYKrSLtQDE0ILfPE0xAzvdJiEUwZkcOUETlcs2Acbl87a3bXYXsuB2d7M/e9tZV739xKdpqNRzP3cXS7lbraVq1UQ9CW0UXW3xY+EQshEX2I0Lsbok9I29IBkaTQR1kw5fdoN6HMEu158yEl9L1Is8fP2VOH8bOzj+KSv3zCZ3tS3gJakWKU0KeSgrGa/9x0AHI6dUyMTneE3jgujtBHkma3Mm98IRQUMt/pYM3XTufD7Ud4f9thWje1sN8PZ/z+XcYUZnBz7hHOBVrcbjJc6NZOpNBHKYHgboD8KOmeFn3xlymhj2PdGHMJmfqiq+ZqKJpg5u0rUkCz20+m047LYWN4dhq7jiTRHlLRJyihTyVG5k3tjiSEXrduYnWRSoQzE5pIbsEUaCmW7gbyMxyce8wIzj1mBPL/svHU5XPb9Mms2HqYT3c1cq4FFt3zNsNHj+fetjrynHbSAxKLxZiwtWn2UegiJncDpOVEv64jo2vWTZhH79U9eiOiV5k3vYXH3463PUBWmiYdTrsFj7+9j0elSIRJf0FhitAUS7MEhb4bET0kL/RRShULn5u0NBdXzRvDE1fN5tbztcbbl84aQYvHz6HaetbsdzPrN2/xg2c/54XPqjjc5NHsG0OIpUyx0MdYMGV1hkf0Q4zqRjcrd9T0+nVbPJqoZzi0JjBOmwWPP9Dr41Akh4roU0nWCM2HTmZC1hNnMtYMRsTblYi+U3qlJ8yasTu0x9fNL+O6ogn4H3Jy2FrISXlFrNh6mJfW7gdgQ7qFLTsP0r6rlmOHO7AHfHGEPjM568YezaPXJ4fT8zTraAgK/Z/f28k/V+9lwy/P7NXrNru1IneZeikOp82qhH4AoIQ+lVgsUDAuyYje8Oh7WeijNR/xt4ULdGh6JWALeBhenMd9F04nEJBs3N/Iim2H8b9vZ8u+Gn7255WMTWvkHWDVwXbKGtwMy4nw9I1mKYlo92rrEqLV0mn3aDcqITT7Zgiujt1f30azx097QGK19F5qabNHF3qnbt3YLHh8yrrp7yihTzUF4zo6Q5mhu9aNM1OLamNVuIxFWo4m7O2+kHo1nvi15kMmYy0WwdTSHKaW5sDnmSweVUzexOPYtH4VbIOnPqvn1dVvM2lYFidPLObkiUXMGJ2H3eGK3t0qknavdv3gGCLTK/UbQGbxkIzoDzZqq5hbvX6yQgvdJYnXH+D+t7ZyyexR0ZviRNBJ6O3KuhkIKKFPNQXj4cvXwgU0HkGh72JEnztKK5Gc7IKh0DIIGQXaY19E+mQwvVKvSe9v65xHr+/nEH7Omjqcs3KHwTb4yQWzObp1Esu3HOKv7+/kkfd2kOW08bdMH2MsDXgb3RRnRzmXQbtPE3njW0Vk9UojrTOzpKPxyhDikC70bd72bgn9Yx/s4uHlOyjOcnLlvMSF8Zo92g0305iMtVnxByR+s3WWFH2CKaEXQiwCHgCswF+llHdGvO4E/gbMAGqAi6SUlUIIO/BX4Dj9Wn+TUv4uhePvfxSM15p11O2GwvGJ9zesm2QjcoMFP4F5NyZ/XLCwWUOH0Ee2FjSE3lis5Pd0XhkL2jFG1o1e475s+AiuLR3HtQvG0eT28eH2GpZvOcT+jVYK/fWc+Nu3OXpktlbKYVIxx5TmhlsQ7V7t+kJogt9pwVSI0O/9NPn3P4AJBCSHmrT/7xZv122TAw1tPPjONgAa3f4Ee2s065OxodYNgFcJfb8modALIazAQ8DpQBWwSgixVEq5KWS3bwF1UsrxQoiLgbuAi4ALAaeUcqoQwgVsEkIskVJWpvqN9BtKZ2nphh/eB+c/lHh/T7MWzZtdYBWJzdm15tjRCptFroy1RNSx8cWK6B0d+xjNTEK8/qw0O4uOHsaio4chneNp37CeHy+cyLtfHuKhd7fz4Dvbyc9wsGBCEadMKuakCUXkGNaNMY7IDlOhQt9aY/4b1CCgpsWLPyABzbrpKr9+bTPtAYnDaqGhzZf4ADomY4PplbrQe3wBXI4uD0XRw5iJ6GcD26WUOwGEEM8C5wOhQn8+cLv++Dngj0IrmyiBDCGEDUgHvIAJg3YAU1gB834AH9wLk86FiYvi7++NUS6gpwlG9CH1bvzu6BF9u18TUtkeO6IPCn29fv7oWTfC4cLmb+X6heO5fuF46lu9rNh2hHc2V7N8yyFe/HwfVovgiZz9TJeCQ4eaGWe1I2JaN8WAhJYjkD086f+GgUh1Y0dz9NYuRvQfbT/Ca+sP8IPTKvjnqr00mhV63brJCHr0Wpql8un7N2aEfiSwN+R5FTAn1j5SSr8QogEoQBP984EDgAu4SUpZG3kBIcTVwNUAo0aNSvIt9ENOvgW2vg6vfB/KPo5fn95M/9SeILKCpZSa0Ie1CQyJ6INtBGNE9P6IiN44fySOTO06gXawWMl1OTjvmBGcd8wI2gOStXvreHvzIQJrvBz2BDjt3vdYkw6VO6txbz/CrPJ8HJHWDWgTskNE6A82dE/o/e0Bblu6kbL8dK5dMI7/bDhIo9t8RC8EuOwdefSAWjTVz+lpU2020A6MAMYAPxRCjI3cSUr5qJRyppRyZlFRUeTLAw+bEy54RLMUlv04/r7e5q5PxHaHtAjrJthdKrR6ZYhHH63piIHV0VG90t2gRdvRbggQt7CZ1SKYMTqf/1k0iZPH5TCqOJdfffVosNipPFTPN/76CTN+9Qa0e9h82ENdixeyhmkHD6EUy+qmEKH3JG/dbKluYtuhZr53SgVpdivZaXbz1o2nnUyHLbgy2mlTEf1AwIzQ7wPKQp6X6tui7qPbNDlok7KXAv+RUvqklIeAD4GZ3R30gGD4NFhwC2x4DvZ8Ens/b0vfCH1kRB8U8mjVK33R2wgahFk3cVbFgvkKlu0+7HYn35w7moLsTM6fWsRfLp/JeUcXAvDKxlpm/PpNvvtyFQBH9myKd7ZBRXU3I3pD1EvztM86O91OY5vZyVhf0LaBcI9e0X8xI/SrgAohxBghhAO4GFgasc9S4Ar98WLgHSmlBPYApwAIITKAucCXqRj4gGDq17SftXFWynqbu74qtjtETsZGi+itRmqjr+P1yD6woFk8frNCb9SkTyT0IZOxVjs26ef0ySX85lyteNk35lVww8LxVPmyaZZpFH5wG3vumMzKP1/Pl5++Sbu/65OU/Z3qRg92qxZRd2UytkmfUDUazWen28xbNx5/MLUStDx6UNZNfyehR6977jcAr6OlVz4updwohLgDWC2lXAo8BvxdCLEdqEW7GYCWrfOEEGIjIIAnpJTre+KN9EvSdW++LU4ZV0+z1rCkt7E5tOjco3vq/igRe5h1EyeitzrDrZu4Qq+nkSZaHWvk0UN4k3L9hjOyMJebZ0/k5jMmcmDPh2z86F+4dr3OzP1LsB94mpplOezIm49jyrlMOuFc0lx9cDPtIQ42uhldkMH2Q81diuiNideg0Cdr3YRF9Mq6GQiYyqOXUi4DlkVsuzXksRstlTLyuOZo24cMzmwt1bK10/xzB301GQvhZRDqdms/jUJhEL4y1qfbBdG8d5szIqLPjX1N470maifY7u3Y12rvsIbaO3/zGD5qPMNH/RT4KY31Naz78AXEl8s4qvYdsj54jdb3b+TzzFn4K85mwomLySkoiX/tfk51o5vyAhc7Djd3KY/eyJnPTtf+/HPS7TR7/ARCq5LGoNntixB6FdEPBNTK2J7EYtFq0LTFE/rmrpco7i6hhc12rdBqy5SFJFSFplcGI/oY1k2oR587OvY1k7Ju9Po9oQumDAvJGn3tQHZuATO/8h34ynfwetx88cm/aV2/lDFHllO89gP8n/+cDWnTaC4/k/L5X2dYmYlFbf2M6kY3M8vzSLdbuzQZ26TbNIZgZ6fbkRKaPH5y0uOvRWj2+CnO6rjZB60b5dH3a5TQ9zTp+bGtGyn7Lo8ewiP6XStg5HEd2TgQkV4ZL+smGevGmIw1Y93o1w9dMBVtLiHWpZxpTD3pAjjpAmQgwNa171Oz5gVGHHibo7fcBVvuYputgiNlZzBi7tcZPXF6wnP2NW5fO3WtPkqy0nA5bLR2oaBYY5ufDIc1uJI1W/fcG9t8CYW+xdMe7tEr62ZAoIS+p0nPi23d+N1av9e+mIyFjoje0wT71sD8H4S/Hs2jj2rd6Hn0iWrRQ0ephyQnY4M3hijWjRmExcKE4xbAcQsA2LN1LftW/ouCvW9w/K6HYNdD7LaUsX/4aRTNvpBxU49HdHW1cg9yqFF7/yU5abgcXY/os0ME3RD3hjZfWHpdrGOVdTPwUELf07jyoXF/9NeMWvR9kV4JWkTfeAB2r9RWvY45Kfz1MOvGENhYEb1Xu3G1e3sg6yakxEISEX08Rk2YzqgJ04HfUF21g8oP/knmzmXMrnoS674n2P9SCXuKTyF35mImHLcQi9XareulCiOHviRbF/ouefS+YAkDICj6iTJvpJRa1k1UoVcRfX9GCX1Pk54Xu2xxd0sUdxcjot/1niamZRELni1WbTI5bGVsNI/eod0oDIsqRXn0wRuNNYp1E8Oj7wolpeMouVibzK09tI/t7/8L5/bXOO7gP3G8toRDr+Wzs+hUsmcsZuLM07Da+u7PxlgVO6w7Qt/mD2bcQEf2TaIyCG5fgIAkIr1St26UR9+vUULf06Tnx7ZuuluiuLs4czTbZtcKTeSjibjFnnhlrE2PvI3VqfGE3ubUJn2TjuiT9+i7Qn7xSGZ/7QfAD2ioO8L69/+F9ctXOPbQSzj/8y+O/CeXHYWnkHnsYibNObPXRd+oczMsO40Mp42Wrlg3Hl/YhKqRfZNo0VSTJ3wSF5R1M1BQQt/TuPLA19K5BDCEdJfqw4je2wwHv4CFP4u+jyGycSN6/X21HNbPG0fohTDXTjAsjz5+emVPkZNXyMzzroPzrqO5sY4N7z+P2Pwy0w6/SvqbL1DzZg7bCxaScdxijppzVq+IfnWjG6fNQna6jXS7VevZmySNbX7GFXWMNcekdRNsIxgi9DaLwCKUddPfUULf0xgt/trqOuqyGBgRvbOP0iuDhcckjFkQfR+rTV8Zq0f00SwTw2IxI/SgLZpKmHXj7TXrxgyZ2XnM+Mq34SvfprW5gTUrXkBsepGpR/6N682XOPKmHukfdyGTZp/RY6J/sNHDsJw0hBBkOG1dsm6a3L4w6ybDYcMiEls3LRG16AGEEKpv7ABACX1PE7o6NlLoPf3AowewZ2ipldEwcth9bZq4RstEMaJrM9YNaO/XzIKpHpyM7Q6uzBxmnH0VnH0Vrc0NfPb+C7DxRS3Sf+MFDr+Rx46i08iddRETZpyS0onc6kY3Jbrtku6wJl0CQUpJo9sftGtAawuZZWJ1bFNEdykDp131je3vKKHvaYyIPppP39fWjRHRjz4hdtMOI4c9sp9sKMlYN6A3CI9j3QTatcndUKE3Fkz1onVjBldmDseddRWcdRUtTfWsXvEvrBtf1Dz9Zf/i4LIiKoedQeHxlzJu6gndTtmsbnQzrTQXgIwuTMa2ettpD8hO7Qdz0u0Ju0xFs25AbxCuIvp+jRL6nsaoRR9tdWxQ6PvKutGvG5lWGYphm/jboq+KhY7JWNNCn8CjN2yaeNZNPxH6UDKycoOrcpsaalm//FkcX77IjAPPYn/xGfa+NIKq0rMZMf+bXVqcJaWkutHNsGztvbscmnVjpnSBQWRBM4PsdFti68YbS+iVddPfUULf08QrbObVuzv1VUQ//BgYezJM+WrsfUInQmNG9CFZN1ZH9MycUOwuaD0S+3XjemGtBCOsm1726JMlKyefWed/F87/Lg011Wx59xlc215mzp7HsCz5K9ut4zhSfg7lJ19uugxDY5sfty9Aid5U3eXQLCG3vx2Xw9yfsjHhmhVhv5gpbBaM6COtG5tFZd30c5TQ9zSJrBuLre+iU1c+XP5y/H0sdq3Zeaw2ghBu3aTlaJk18XBkQP3u2K8HI/pQ68YPgUC/juhjkVNQwuzFNwM3c2T/brYv/zv5O19m7o4HCGz/AxudR9NccQGTTr2cnPzYjXcONnYslgJw6ZF1i8e80Bt1brLTO1s3Ow7HnyBv8sSI6O0WlUffz1FC39M4MjShimbdePQ6N4mEsS8xInopY4trqHWTyLYB3bqJMxkbjOjt4T8DPs2jt9i0xVwDkMIRoym89OfAz9m3cyN73vsbI/e+wpSNd+Dd8Fs+z5iDnHohk0/+Omnp4d/0qiOFXl+spE3ImrvxGbny2VEi+kR59C0ePzaLCObOGyjrpv+jhL6nEUKvYBnNuumj7lLJYPjjAX/0HHroiLxbjkBOaeJzOjLip1dGWjehna5CG4MPcEaOncLIsXchA79j+xcfceSjpxlf/W8KP/mQxk9+xvq8hWTMuoyj5pyJxWoNRvTDdKHPcBpCb9426bBuOnv0ZqybzDQbIiIwUdZN/0cJfW8Qa3VsX/WLTQbDNvG7Y4/VEGLZbjKiT5B108m6CamiGW3h2QBHWCyMP2Y+44+ZT7vfzxcfLsX92bMcXfsWrjde4+AbRewaeQ57M04HoFifjE3X7ZpkUiwja9Eb5KTbafO14/UHcNiiZwY1efxkRLGInDYLzV1YoavoPZTQ9waufGir77y9L0sUm8Vi00Te54aMGP5xqPA6s6PvE4rDpZdV8HbYPqHEsm7adetmkAl9KFabjakL/gsW/BetzQ2sfmcJjk3PMbvqSY4XT3B6WgVpn++Ho79GhqMLEX1EdykDw7NvcvsoyIz+/9vi8XeaxAXdukmhR+/2tfOb1zZz8+kTyMuI8vuhSJr+V4d1MBKr+Yi3pe9KFJslLL0yQdYNmPfoIbZ9E9O60SN669D443dl5jDzvGuZdstb1F2zjo/H38SoLGDZj+B/J1Lx3vUstHxOa5v5MghNbj8Oq6WTz24Ifzz7JrJypYHTnlrr5ot9Dfz94928u+VQys451FERfW8Qqya9pxlyE1UA72OCtW7ciT16MG/dgL46Nr/z69GybkD/FuBJnL45CCkcMZrCy24HeRscXA9rl5C57h884ViG+7Un4OAlMP0yKJ4U9zyNbh/Z6Z199o56N7EtmGa3n1xX55tsqhdMGWmce2oTrJ5WmEZF9L2BMRkrZfj2gWLdGI1HYmbdhGxPRuhj+fTxrBu/J7rdM1QQQlv/cNad1Fyzju94b+ZQzjT4+E/w8Bz4y6mw+gmtAUwUGtt8nSZiIbSCZYKIPpZ1k0KhN9I4ldCnDlNCL4RYJITYIoTYLoS4JcrrTiHEP/TXPxFClIe8Nk0IsVIIsVEI8YUQYuiFY658zVuOrO8yILJuHCGToCmK6O0J2glGWzBlbG8fmhF9NFwuF28GZvL60f8LN38JZ/xG+x179Qdwz0R44Rqo/CAswGhy+zulVoJ56yYrmnVjS22tm2BEX6OEPlUktG6EEFbgIeB0oApYJYRYKqXcFLLbt4A6KeV4IcTFwF3ARUIIG/A08E0p5TohRAEQP4drMBK6OjY0gh8IEb3VrnWY8rUlXhkLkJab+JwJI/oY1k3Qox+8k7HJkK7n0bd4/ZBZBCfcAMdfD/s/g8/+Dhueh/XPQv5YOO5yOOZSvbtU54jeTKniZnc8jz6F1o1ePE1F9KnDTEQ/G9gupdwppfQCzwLnR+xzPvCU/vg54FShmYBnAOullOsApJQ1Usqhl3AbbXVsoF2LvvqqRLFZrHZtnPFWxnbZuonxh5zQulFCD2C1CNLsFtpCs26EgJEz4Nz74Ydb4KuPQNZweOt2uPcobq69g7mBNdrvXwjBdoIxFk0FApIWbzsZUSN6K/6AxN+eGrE36vEcavKEvzdFlzEj9COBvSHPq/RtUfeRUvqBBqAAmABIIcTrQojPhBD/E+0CQoirhRCrhRCrDx8+nOx76P9EK2xm2Dj9PaK32LUuVBA7orfYAH1yr0ezbgZ/emWyZDhswWJjnXC4YPolcNUyuGE1HH89U/ybuGH/T+GBY2D5XcF+xk6bBYfVEtO6Ma4RPb1SkxFvioUeYG+diupTQU9PxtqA+cA39J8XCCFOjdxJSvmolHKmlHJmUVHsWh8DltDmIwZ9XYveLFZ7SGngGEIvRIcYp2QyNrJ65dBMrzRDutlSxYUVcMavOLn9Tzw/7tdQMA6W/xbuOxqWXIrY/hY5aZaY1k1zjDo3ENJOMEW59KGLr5RPnxrMCP0+IDQHsFTfFnUf3ZfPAWrQov8VUsojUspWYBkQo8PFIMbw6EOtm74uUWyW0Dr18SZBjSjblNC7tJ8Js26MiF4Xl/ahm14ZiwyHjVaPOXvD6w/Q6BPsH7FIK2b3/c9h3veh6lN4ZjEvB77H7H1/00pZRGBMkEa1bowG4Sny6ZvdfoqytN+nVPr0e2tb2XygMWXnG0iYEfpVQIUQYowQwgFcDCyN2GcpcIX+eDHwjpRSAq8DU4UQLv0GsADYxFAjWkTf1yWKzWIJEfpYefQQEtGbWBlrN5teGWMydiinV0aQ7rDGtm4i6FS5Mn8snHY73LQJFj9Ora2Yrx55FO49Cp7/Duz5JJixE4zo41g3qVo01ezxU5aXTqbTllKhv33pRi58ZCX76ttSds6BQsKsGymlXwhxA5poW4HHpZQbhRB3AKullEuBx4C/CyG2A7VoNwOklHVCiHvRbhYSWCalfK2H3kv/xZ6m1WAPE3pd5Pr9ytgQUY0XSVsdmldvdyU+p82h7e9L0roxqleqiD5IhtNqesLS8L47+ew2Bxz9NX7/aRnZTTv4Y8XnsG4JfPFPKJkKs79Dq2uhdmyMyVhIXUTf5PaR63JQlu9KqdDvONxMs8fPT1/4gqeumtVp0dhgxtTKWCnlMjTbJXTbrSGP3cCFMY59Gi3FcmgTWdisr9sImsUa8isSazIWNLEwU4veIF5hs04RfUTWjfLog6TbbdS2mItQDf89ss6NQU66nY21w+HsS+DUW+GLf8Gnf4FXvs9sezY/s51Irmc0kauZU+3RN3n8lOa7SLdb2Z6gRr5ZfO0BquraKM1LZ8XWw/xrdRVfn9XPV6WnELUytreILFVsZLL09wVTodZNrPRK0HLbzfjzBvHaCcZbMKU8+jC0iN6sdRM7cwa0GvXBrBtnJsy8Cq77EK76NwcLj+e/rf9m3JL5sOQS2Lk8aOs47Sm2btzawqxRBVpEHwjIhMd8sO0IG/ZFXw0MsL++DX9A8r1TxjN7TD6/enUTBxqGjoWjhL63cEUUNgtG9P1c6EOjZzMRvVnsrjjplTGsG1+bns+v0isNXA4rLSatm2DlyvToEX12up1Gtw8ZWqpDCBh9Am8dfRfzPQ/gOf4HsPdT+Nv58KcTYM2TpKPdmFM2GatXySzLd+H1BzjUFL9om5SSa59ewwUPf8jfVlaGj1+nUs/eGVOYyd2Lp+ELBLj15Y0pGe9AQAl9b9HJuhlA6ZUG8SLptFzILDF/3kTWjcXWYQMZYzD+z5R1E8TlsJn26BtjtBE0yEm342uXuKNYMM1uPwcpwHrarXDTRjj/Ya3L1ys3Mv25efzQ9k9k04GuvxGd9oCk1dtOptPO6HxtvieRT1/b4qXZ4ycn3c6tL2/k5n+u6/R/srtG+10rL3AxuiCD75w4lrc2V3M4wU1ksKCEvreItG4GTERvUujPfwi+cq/58zoy4q+MDRVz47Gx9kBZN0Ey9KybaFFsJImtm9j1bpq9fpw2C3arRftmd+w34Jr34crXaBs+m+utL3PC0oXw4rVw8Isuv5/QBuSjdKE3RDoWxo3gtxdM5ebTJ/DS2n3c8Wp4tF55pJV0uzWYtnn21OFICW9vru7yWAcSSuh7C1d+eAVLb7MmWFZT8+F9h9n0yrzRkBO5YDoOjsyOCF1KaDrY8Vq7L/wGExR6fV5DpVcGSXfYkNKcbdLY5kMIyIzRSDxYwTLKoqlmd5SmI0JA+XyOnPMEJ3vvZdeYi2DTUnhkPjx1Hmx7s3PF1gQ06XVuspw2RuSmYxFa/ns8DKEvL8zg+6dWcPpRJXyyM7ws+O6aFkYXuIKZNpOGZVGWn84bm5TQK1JJep7mL3v0BRtGY/D+jtn0ymRxuLRvNVLCazdrKzQNa6tTRB9h3aiIPojRN7bFRCu/Rr0omcUSPTMqWNgsWkQfo+kIaFk3e2QJq4/6Cdy8UcvNP7IVnlkMf5oHa5do3cRMEJqv77BZGJGbntC6qarTJlXL8rRvAFNH5rDzSEtw3QDA7tpWygs6/t6EEJwxeRgfbD8yJNogKqHvLSJXxw6EEsUQkV4ZJ6JPFsOj/+QRWP24liPfoJdUaveFC70QmmdvRPTKow9iVLA0Uwah0e2LmVoJHdZNtIi+xeOPuioWQhdMBbSAZv5NcON6+OqfAAkvXQt/mA4rH+6w32LQHGEvjcp3sTtRRF/TSmGmk3S9teLRI7WkgE37taCqPSDZU9PK6ILwNR6nTy7B6w+wYusgrK8VgRL63iKysNlAaAwOEemVqYzoM6HlMLz+MyjSuyI16V+j273h1g1o4h6M6FXWjYEhvqaEvs0fcyIWOiZpo3n0TTFKFENICYTQSVybA6ZfCtd9BN94TluF+/pP4b4p8O5voaUm6rmaImrqjMp3mbJuRuV3BCFTRmqrszfoQn+w0Y23PcDogvBv0DNH55HnsvPmELBvlND3FpFlELzN/X9VLIQsWnKaXwxlBkeGZmUNmwoX6hWum3WfPtK6AU341WRsJ4wo1kwZhCa3L+ZELIRaN53P1RyjMTgkKIEgBFScDle+Ct96C8rnw3t3wf1Hw39+Fqye2THGiIi+wMWRZm9ce2VvXWtw4hagOCuNkmxnMK9+95GOjJtQbFYLpx5Vwtubq/GlqPJmf0UJfW9hWDfN+tdEb8sA8ej1P+54OfRdYcSxmshf8izklWvbghG9r7PQh5ZLVtZNkAx9YjVWimVojfhGtz+BdWNDCC1dMZLmONaNzSKwCBMTwmWz4OJn4LufwFHnabbdA8fAKz+Aut3adYysG6c2TkPAY0X1vvYA++vbwoQe4OgROUGhN3LoRxd2/ns7Y3IJjW4/n+6K0tN5EKGEvrfIG601gPj4Ia1j00CbjI23KrYrTPoKXPsBZI/QbiJpuRERfTzrRkX0Bi5H7MnY1ZW1HH3762zcrwlek9sXtY2ggc1qoTjL2WnFqJSSw00eCjOjW2ZCiOT6xhZPgv/6M3z/M5j+DVj7DDx4HLx0PZa6HUBH8bSOFMvoQr+/vo2AhNJIoR+Zw47DzbR6/eyuacFhszA8u/PvzYkVRaTZLbyx8WCn1wYTSuh7C5sTFt2p5Rh/+mc9ou/nJYqhw6NPdUQfSdawjhTLhNaNiugNDKFvi9Kz9fM99bh9Ae57cxugZdPE8+gBhueks7/eHbatoc1Hq7edEbmxb/ZOexf6xuaVa52wvr8WZn0HNjzH1z/+GvfYHyGjqRKAYTna793hJnfUUxgZOZ0i+pE5BCRsPtBIZU0Lo/JdUbON0h1WTqwo4j8bD+JOYd/b/oYS+t5k8vlQcQa88xttInJARPS6MKQ6oo8kswSa41g3VkdHaWcV0Qdx6dZNS5Sa9JX6QqO3Nlezdm89TZ7ojcFDGZmbzv6IiN5IXxyZG/v/3WnrRt/YnJFw1p1w43o+KV7MV6wfIx6aBS9eR55H8/BrW6I3RNlbq42ts9DrE7L7Gtld09rJnw/lyhPKqW708Md3tndt/AMAJfS9iRBw9t0gA+BvGyCTsb0Z0SfIupGBjscKAFxOI72ys3VTWdNCRXEmuS47v3ltE1IStTF4KMNz0thf3xa20nZ/vSH0scUyKesmFlklPF90PRc6/gRzroONL2B/eBb/m/ZX2nUPP5I9ta3YrYKSCFtmWHYahZkO1lc1UFnT0injJpR54wv52nGlPPLejkHbmEQJfW+TVw4L9Na5AyGiN6ybno6iM0s0j17KGNZNSCSq0iuDuOLk0VceaWXKiGyuPmksqyq1bC9j9WssRuSm4/YFqG/tiKANoR+RMKLvvvXR7PbjTS+CRb+FG9fBzG9xLiv43qaL4d8/geZDYfvvrW2lNM+FNcKWEUIwZUQO7209jNsXiBvRA/z8K0eRk27nlufX026iWuZAQwl9X3D8DTD7apiwqK9HkhhrLwl91jBN4NvqOpdAgJ5boTvAsVktOGyWTkLv9rWzv6GN8sIMrji+nIIM7f8vXtYNdIh5aBem/Q1unDYL+Rmxv0lpHn33UxTDVuBmDYOzf8/1BY/xvus0rTb+A8fA23dAWz2gpVaW5UcX8aNHZnOkWStaNipORA+Ql+HgtvOmsK6qgSc/quz2++hvKKHvC2wOzcIZNrWvR5KYoHXTCx49aBFb1Ig+SpEzBaAVNou0bqrqWpESygsyyHDauHbBOAByXImEXvucDzR0TH7uq29jZG563I5MKbFu0BZMZUbcjGR2KXc7rocbVsHEs+H9/9VW2n74B6pr6ijLi/67OXVkR9nsRBE9wLnThrNwYhH/+8aWQTcxq4ReEZ9gemUvRPSg2TdRJ2N7aIXuIMDlsHWK6CuP6LnjusBdcUI59110DLPL8zsdH8rwHE0094dG9PVtcTNuIHXWTbRFXfkZdmpbPFAwDhY/BtesgBHHwZu/4IXAjZzqfQcCnW8yU0ZoQm+zCEYmGD9ods9500fQ6m0PTkAPFpTQK+Jj6eWIvqk69mRs8HH8qHSo4YoS0RsZN2P0RUIOm4ULji3FZo3/J1+Q4cBhs4Rl3uyra4vrz0M3s25CMLpLhZKf4aSuJaQhyvBj4JsvsOvsJdTKLE7ZfCs8epLW9SqE0rx0ctLtlOalJ3zfBokWaA1UlNAr4mNMgvbGZCzoEX0U6yZ0UngINXU2gyb0ERF9TQs56XZyXcnZXBaL0DNvNOvG42/nUJPHRERvTb1Hr5OfYcfbHuhUBmGL61jO8/6avQv/AG0NWter/7sIDm8FtAh90ZRhzK8oNH39MpPNTgYapoReCLFICLFFCLFdCHFLlNedQoh/6K9/IoQoj3h9lBCiWQjxoxSNW9Fb9JZ148zS2gs2Vce3bqwq4yYSl8NGq6ezdVMeZcm/GUbkpAetm+oGbTIzodDbu2/dBLtLdbJutM+8LiKXfk9tKxIL2bMv1fz7034JlR/Cn47XMnTa6rhr8TR+/VXzc2FFmU7S7JahJ/RCCCvwEHAWMBm4RAgxOWK3bwF1UsrxwH3AXRGv3wv8u/vDVfQ6vbUyVoiOFMt41o1aFdsJl95lKpTKmhZTE5DRGJ6bxgFd6I3sm1JTHn33IvrmiMqVBvkZ2u9CTUt42789ta3kpNu1Ymz2NJj/A/j+53DsN+HTR+EPx+klsM3fgIQQjMp3DT2hB2YD26WUO6WUXuBZ4PyIfc4H9BKEPAecKvQpeiHEV4FdwNDpxDuYsFi1dNCJZ/f8tYxFU/GybtREbCdczvC+sR5/O/vr2+IuEorHyNx0Dja68esFw8BERJ+CrBujUUhkCmgwom8NL7a2t7ZzMTMyi7SyCtesgOLJ8OpN8OgC2POJ6XGYKY080DAj9COBvSHPq/RtUfeRUvqBBqBACJEJ/AT4ZbwLCCGuFkKsFkKsPnx48DcBGFAIAWf+Bkpn9vy1MkugcR8gYy+YUqmVnXDZwyP6vbVaoa8xhV2M6HPSCUg41OQJCr1RcyYWTlsXat1EENpdKpR8fZ6hpjlS6Fspy49xAxo2VSuNfOGTWrOfx8+Al77bUT02DqPyMzRbKMk2iP2Znp6MvR24T0oZt62MlPJRKeVMKeXMoqKiHh6Sot+SNUwXeuJYNyqij8TlDJ+MNZppdzWiNzJs9te3sa++jcJMJ2n6CtxYaB59N60bdwzrJlP77EMj+kBAUlXXFmwfGBUhYMoFcP2nWter9f+EP87Q7ZzYYx2Vn06rt52aKOWaBypmhH4fUBbyvFTfFnUfIYQNyAFqgDnA74UQlcAPgJ8JIW7o3pAVg5bMEs22gTjWjYroI8nQ8+iNCHRXsNFGV4Vez6VvcOuLpRLfXJ02K/6ADKt/nyxNMSL6DIcVh9USJryHmjx42wOdyhNHH1ym1sf2uo9g2DTNznn8DDi4IeruowoGX+aNGaFfBVQIIcYIIRzAxcDSiH2WAlfojxcD70iNE6WU5VLKcuB+4LdSyj+mZuiKQYexaApiZ92oiL4TuS477QHJjsOawO+uaSU7zUZeglWwsRie0xHRm1ksBR1dprwJhL7R7WNVZfQmH8F+sRERvRCC/AwHdSFCv69eE+HSGKtio1I0Aa54BS74M9Tu1Lz7t+8AX3gJ5MGYS59Q6HXP/QbgdWAz8E8p5UYhxB1CiPP03R5D8+S3AzcDnVIwFYqEGLn0ENu6UR59J7567EhcDiv3vanlj1fWtFBemBG3ZEE8stLsZKXZdKF3JyX08XLp27ztXP7Yp1z4yEq2Vjd1er2jjWDnG1RehiOs85WxcjVW+YOYCAHHXAw3rIZpF2nlFB6ZB7s/Cu5SqttBe2I0OxmImPLopZTLpJQTpJTjpJS/0bfdKqVcqj92SykvlFKOl1LOllLujHKO26WU96R2+IpBRbyIPrhgSuXRR1KY6eTbJ47ltS8OsL6qXk+t7F5l1JG56Wza30ibL37DEYNgg/AYPn17QPK9JZ+zrqoeIeCVdfs77dPs0bJuIq0b0FbsRhP6eKWT4+LKh68+DN98UVu38cRZsOx/wNtCmt1KSbZzyFk3CkXvEBbRx7JulNBH4zsnjiHPZee3yzazr66tyzn0BsNz0lhfpbUgNOfRx24QLqXktqUbeGtzNb88bwonjCvglXX7O2W1NLv9CNFRejmUzhF9KwUZjmBz9C4z7hT47kqYc63W+e1PJ0DlB4Mul14JvaL/kJ4PFiONMpZ1o4Q+Gllpdq5fOJ6Pd9YSkF3PuDEYkZse9NvNRM1OW+yI/qW1+3j64z1cs2Aslx9fznnHjKCyppUN+8KbfDR5/GQ6bFFb/hVkOMImY6vq2pLz5+PhyICz7oIrl2nPnzyH73qfpLqmPjXn7wcooVf0HyyWjqheTcYmzWVzRwcnUrta/sAg1K5JVNAM4nv0qyrryHPZ+cmZkwA4c8ow7FbB0nXhyXvNbn9U2wYgz+Wgye3Hp9989tW1Bb30lFE+T8vMmfnfLKz9B4+4f4x33/rUXqOPUEKv6F8EhV6VQEiWNLuVW86aRGGmg4qS7rWpNMQ9UcMRA6c9tnWzr66NspDm3LkuBydVFPHq+gMEQro5Nbn9nUoUGwRz6Vu8Wg59fQoj+lAcGXDOvXw45xHyRRP2x06FlQ9rnc8GMEroFf0LY0JWFTXrEudPH8mq/3dawk5SiTDq0idqOGIQz7qpqmvtJMrnHjOCAw1u1uypC26LVrnSwFgdW9vq5UizB68/0DNCr+M86kzO9NxJzfD58PpPtaqYLUd67Ho9jRJ6Rf8ipnVjRPRK6BPR1bTKUIxGHWYybiD2ZKyUMtihKpTTJpfgtFnCsm+idZcyML5V1DZ7qTKalfeg0I/Kd1FHNv+eci+c9XvY+S78aZ5WHXMAooRe0b8IRvSR1o3KuulNSrLTEMKcPw8h1k2ER1/T4sXtC3Ty0zOdNk47qoRlXxwIrqZtdvs6LZYyCAp9qzeYWplyjz6EoiwnTpuFPXVtMOca+M47mq3z1Lnwwf2mrJx99W3BOYW+Rgm9on+hIvp+gcNm4XunVPC140pN7R/LuunId+8cfZ83fQRHmr288+UhIIF1Ywh9i5equtaY50wVRrni3caiqWFT4erlcNS58NZt8OylwQbl0XD72jn93vd4ZPmOHhtjMiihV/QvxpwE5SdCXnn4dovy6Hubm0+fwJyxBab2jWXdGKJcGqXK5KmTihmek8bfVu4G4k/G5urlHDShbyM/w0FGjJtCquiUS5+WrVXDXHQXbHsD/npqsJtVJHtqW2n1tvPGpuoeHaNZlNAr+hcF47Tyss6s8O3KuunXdAh9eES/L05Eb7NauGzuaD7YfoQtB5uidpcysFst5KTbqW3xsq+us+ffE5TpdenDFnYJAXOv1WrmtNXDX0/l0OqXOi3+qtQLy32xr4EjzeENU/oCJfSKgYGybvo1wRIIvs7WTU66PWr9GoCLZ5XhsFn40/LtQOcSxaHk66tjo2Xx9ASj8l20eNs5HE2oR58AVy/Hkz2awleuZNuLvw3z7Y3m7ADvb+v7HhtK6BUDA7Uytl9jRPRuX2frJp4oF2Q6OXfaCJbq2TexrBvQhL6m2ZvaVbFxOG50HgArd9RE3yG3jA9OfJplgdlMWP97rfxxu1aYrbKmlVyXnYIMB8u3KKFXKMxhRPL2nv8DVySPzSKwiCjWTZTUykiuPKEcY91UpjN2/n+ey8G2Q814/J2zeHqCaSNzKMhw8K4+WRyN3Y2S7/m+x39yL4E1T8CSi8DTxG69sNxJE4pYsfUw7YG+XXClhF4xMCieDGffA+NP6+uRKKIghND7xnZE9FJKPfqOL8pTS3M4dlQuEL1ypUFBhiPod/dGRG+xCE2otx2JKdR761qRWLi7/WI49w+w41148hzqDx9gTGEGJ08soq7Vx4Z9DT0+3ngooVcMDCwWmP0dcPR8JKfoGpHtBOtafbR6202J8rfmjwFgWHbsvP28kFIMPblYKpSTJxZR2+JlfVV91Nf31mqTzZU1rXiP+SZcsgR5+EseaPsZUzIbmT++ECHoc/tGCb1CoUgJWoPwDqEPZtyYEOVzpo3g7R8uYOKwrJj7FIQKfS9k3QCcVFGERcC7MYS6qq4Vq0XQHpDaBOyEM9l3zjMUizou3fAdCtx7mTYyh/e2xrZ/egMl9AqFIiVEWjfBHHqT0fe4oviF2IyIPtcVO4sn1eRlOJhelst7WzoLtZSSvbWtzNAnbbdVNwOw2TGVi72/wC798ORXOH9UG2v31lMf0ty8t1FCr1AoUoLTFm7d7KtPbakCI6LvDX8+lIUTi1lX1cDhpvA0y7pWHy3edhZMKEIIgu0Rd9e0sEmW03bpixDwc9mX1zOKA7y/re+KoimhVygUKSHSo6+qayPLaSMnPTXRtxHRl3a1fWAXWTipGIAVW8PtG6N5eEVxJqPyXWw/pEX0u460kOuykz1qGlzxCnb8/MPxG7Z9ua5Xxx2KEnqFQpESolk3qZw0NSL63pqINZg8PJvCTCfvRtg3e3VrqizfRUVxFtsOGRF9a0eHr5LJiMuXkm7x842tP4CmvimJYErohRCLhBBbhBDbhRC3RHndKYT4h/76J0KIcn376UKINUKIL/Sfp6R4/AqFop8QORmb6oVNRVlO8lx2jinLTdk5zWCxCE6eWMT7244EK21CR8ZNWb6LipJMdh1pwdce0Juzh3zrGHY0fxt7N9ntdfDMYnA3Rl6ix0ko9EIIK/AQcBYwGbhECDE5YrdvAXVSyvHAfcBd+vYjwLlSyqnAFcDfUzVwhULRvwj16KWUKW/3l2a3svrnp3PutOEpO6dZTp5YREObj/Uh+fB761rJc9nJdNqoKM7E1y7ZWt3E/vo2yiN69qaVz+Y6743I6o3wz2+Cv3cnZs1E9LOB7VLKnVJKL/AscH7EPucDT+mPnwNOFUIIKeXnUkqjs8BGIF0IodawKxSDEKfNSqtXKwHQ2OanyeNP+cSp1SJS0lglWeaM0ap4rtpVG9y2t7aVsnztRjahREsLfffLQwQklBeG3+AqSrJYHpjOzuN/BzuXwys39mp7QjNCPxLYG/K8St8WdR8ppR9oACLrm34N+ExK2alCkBDiaiHEaiHE6sOH+74uhEKhSJ6jhmez43ALSz7dQ1V9z9eM702KspyMLcxgVWWH0FfVtVGmf2MZV5SJEPCmXpY4MqKvKNZSR1dmL4IFP4F1/wer/orb1443SvvFVNMrk7FCiClods410V6XUj4qpZwppZxZVFTUG0NSKBQp5vqF41gwoYifv7SB//tkD9CzXaB6m1nl+ayqrCMQkLQHpFawTa+zn+6wUpqXzroqzdqJFPrhOWlkOKxaZs6CW2DCIvjPLdz55yf50b96PhvHjNDvA8pCnpfq26LuI4SwATlAjf68FHgRuFxK2T/arSgUipRjs1p46BvHMaEki2eCQj84InqAWWPyaWjzse1QM9WNbnztMhjRA1QUa/ZNdpot2CjFQAjB+BI9M8digQv+TCBnFN89fAdfbtvaqZ59qjEj9KuACiHEGCGEA7gYWBqxz1K0yVaAxcA7UkophMgFXgNukVIOzK66CoXCNJlOG49fOZNh2WlkOjsL3kBmdnk+AJ/uqgnm0BsePUBFiWbPjCnMiDqPUFGcGVw9S3oua+Y+SAZt3OG/j70h9et7goRCr3vuNwCvA5uBf0opNwoh7hBCnKfv9hhQIITYDtwMGCmYNwDjgVuFEGv1f8UpfxcKhaLfMDwnnX9eczyPXDajTyZOe4qy/HRKsp18WlnHXr2OT1nINxYjoh8dYdt0vJ7JoSYPDa0+AN6pK+R2/xXMtWymfsXDPTp2U00XpZTLgGUR224NeewGLoxy3K+BX3dzjAqFYoAxqsDFqILB48+DZr/MKs9n1a5axhZmIET44i1jwrU8xvs2Iv7th5uYMTqfT3bW0D7sPFYc+YS5G/4XTr4Q8sf0yNjVyliFQqEwyewx+RxsdLNyZw0lWWk4bdbgaxOHZTFvfEGwZEIkRsS/rbqZVq+f9VUNnFBRxDPFP8QnLfDyDRDomQwcJfQKhUJhktljDJ++lrL88InmNLuVZ749l2NH5UU9dmRuOml2C9sONfPZ7nr8AcmcMfmMKq/g1/5vwu4PYPVjPTJuJfQKhUJhkgnFWcEibWVJpo5aLILxxZlsrW7ik101WATMLM9nelkeS3wn0Vi6ANYt6ZGoXgm9QqFQmMRiEczU68+HZtyYpaI4i+2HmvlkZy1Hj8wh02lj+qhcQLBs/C/hqn9r6ZcpRgm9QqFQJMEs3b7pitCPL87kQIObz/fWMUc/z4icNIqynHxSLcDWMxVilNArFApFEiycWEya3cK00pykjzUyc3ztMlg/RwjB9LJc1u6tT+Uww1BCr1AoFEkwcVgWm+9YFCxklgwV+jFCdHwzAJhelsuuIy091m5QCb1CoVAkSVcXgpXlpeOwWThqWHZY561j9Rr7PRXVK6FXKBSKXsJmtXDJrDIumzs6bPvU0hyEgHV7G2Ic2c3r9shZFQqFQhGVX55/dKdtWWl2KoozWbu3rkeuqYReoVAo+gHnHTOCNl974h27gBJ6hUKh6AfccEpFj51befQKhUIxyFFCr1AoFIMcJfQKhUIxyFFCr1AoFIMcJfQKhUIxyFFCr1AoFIMcJfQKhUIxyFFCr1AoFIMcIaXs6zGEIYQ4DOzuxikKgSMpGs5AYSi+Zxia73sovmcYmu872fc8WkpZFO2Ffif03UUIsVpKObOvx9GbDMX3DEPzfQ/F9wxD832n8j0r60ahUCgGOUroFQqFYpAzGIX+0b4eQB8wFN8zDM33PRTfMwzN952y9zzoPHqFQqFQhDMYI3qFQqFQhKCEXqFQKAY5g0bohRCLhBBbhBDbhRC39PV4egohRJkQ4l0hxCYhxEYhxI369nwhxJtCiG36z7y+HmuqEUJYhRCfCyFe1Z+PEUJ8on/m/xBCOPp6jKlGCJErhHhOCPGlEGKzEOL4wf5ZCyFu0n+3Nwghlggh0gbjZy2EeFwIcUgIsSFkW9TPVmj8QX//64UQxyVzrUEh9EIIK/AQcBYwGbhECDG5b0fVY/iBH0opJwNzgev193oL8LaUsgJ4W38+2LgR2Bzy/C7gPinleKAO+FafjKpneQD4j5RyEnAM2vsftJ+1EGIk8H1gppTyaMAKXMzg/KyfBBZFbIv12Z4FVOj/rgb+lMyFBoXQA7OB7VLKnVJKL/AscH4fj6lHkFIekFJ+pj9uQvvDH4n2fp/Sd3sK+GqfDLCHEEKUAl8B/qo/F8ApwHP6LoPxPecAJwGPAUgpvVLKegb5Z43W4jRdCGEDXMABBuFnLaVcAdRGbI712Z4P/E1qfAzkCiGGm73WYBH6kcDekOdV+rZBjRCiHDgW+AQokVIe0F86CJT01bh6iPuB/wEC+vMCoF5K6defD8bPfAxwGHhCt6z+KoTIYBB/1lLKfcA9wB40gW8A1jD4P2uDWJ9ttzRusAj9kEMIkQk8D/xAStkY+prUcmYHTd6sEOIc4JCUck1fj6WXsQHHAX+SUh4LtBBh0wzCzzoPLXodA4wAMuhsbwwJUvnZDhah3weUhTwv1bcNSoQQdjSRf0ZK+YK+udr4Kqf/PNRX4+sB5gHnCSEq0Wy5U9C861z96z0Mzs+8CqiSUn6iP38OTfgH82d9GrBLSnlYSukDXkD7/Af7Z20Q67PtlsYNFqFfBVToM/MOtMmbpX08ph5B96YfAzZLKe8NeWkpcIX++Arg5d4eW08hpfyplLJUSlmO9tm+I6X8BvAusFjfbVC9ZwAp5UFgrxBior7pVGATg/izRrNs5gohXPrvuvGeB/VnHUKsz3YpcLmefTMXaAixeBIjpRwU/4Czga3ADuD/9fV4evB9zkf7OrceWKv/OxvNs34b2Aa8BeT39Vh76P2fDLyqPx4LfApsB/4FOPt6fD3wfqcDq/XP+yUgb7B/1sAvgS+BDcDfAedg/KyBJWjzED60b2/fivXZAgIts3AH8AVaVpLpa6kSCAqFQjHIGSzWjUKhUChioIReoVAoBjlK6BUKhWKQo4ReoVAoBjlK6BUKhWKQo4ReoVAoBjlK6BUKhWKQ8/8BDQk+hGMJ2UoAAAAASUVORK5CYII=", + "text/plain": [ + "
    " + ] + }, + "metadata": { + "needs_background": "light" + }, + "output_type": "display_data" + } + ], + "source": [ + "#hide\n", + "scheduler = PolyLRScheduler(optimizer, t_initial=100, noise_range_t=60)\n", + "plot_noisy_lr(scheduler, label='noise_range_t=60')\n", + "\n", + "scheduler = PolyLRScheduler(optimizer, t_initial=100, noise_range_t=[10, 40])\n", + "plot_noisy_lr(scheduler, label='noise_range_t=[10, 40]')\n", + "plt.legend();" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### `noise_pct`" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + " Percentage of noise to add. " + ] + }, + { + "cell_type": "code", + "execution_count": 22, + "metadata": {}, + "outputs": [ + { + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXoAAAD4CAYAAADiry33AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/YYfK9AAAACXBIWXMAAAsTAAALEwEAmpwYAABV9klEQVR4nO2dd3ib1dmH76Mt720ndvbeITvsEAgJKxTCLIVSdkmhUErTllKgZaQfBQqkLWWPlhkgYY+EESA7JIHsnTjDew/Jks73x6tXlmXJkm3ZsZVzX1cuSe88suCnR7/znOcRUkoUCoVCEbsYjvYAFAqFQtGxKKFXKBSKGEcJvUKhUMQ4SugVCoUixlFCr1AoFDGO6WgPIJCMjAzZt2/foz0MhUKh6FasXbu2WEqZGWxflxP6vn37smbNmqM9DIVCoehWCCH2hdqnrBuFQqGIcZTQKxQKRYyjhF6hUChinC7n0SsUCmhoaCA/P5/6+vqjPRRFF8Nms5GXl4fZbI74HCX0CkUXJD8/n8TERPr27YsQ4mgPR9FFkFJSUlJCfn4+/fr1i/g8Zd0oFF2Q+vp60tPTlcgrmiCEID09vdW/9JTQKxRdFCXyimC05b8LJfSdhccN614Gt+toj0ShUBxjKKHvLPYvh8VzYe+yoz0ShUJxjKGEvrOor9Qe68qO7jgUig7i7rvv5vPPPz8q937hhRc4dOhQxMdLKbnlllsYOHAgo0ePZt26dUGPczqdXH/99QwePJihQ4eycOFC3/0yMzMZO3YsY8eO5Zlnnmn1eOfOndviMQ6Hg9NPP52xY8fy+uuvt+r6gaism87CWa091lcc3XEoFB3Efffdd9Tu/cILLzBy5Eh69uwZ0fEfffQRO3bsYMeOHaxcuZKbbrqJlStXNjvu/vvvJysri+3bt+PxeCgtLfXtu+SSS3jyySej9h4C+f777wFYv359u6+lhL6zcFRpj0roFa3k3vc2sflQZVSvObxnEn8+d0TI/Xv37mXWrFmceOKJfPfdd+Tm5rJo0SLsdjvr16/nxhtvpLa2lgEDBvDcc8+RmprKz3/+c8455xzmzJnDvHnzWLx4MSaTiRkzZvDwww9TVFTEjTfeyP79+wF47LHHOOGEE4Le/5577mHXrl3s3LmT4uJi7rzzTq677joA5s+fzyuvvILBYGDWrFlMmDCBNWvW8NOf/hS73c7y5cux2+0tvv9FixZx5ZVXIoRgypQplJeXc/jwYXr06NHkuOeee46tW7cCYDAYyMjIiPhvHMjzzz/Pgw8+SEpKCmPGjMFqtQIE/bsMGjSIK664gqKiIsaOHcvChQsZMGBAm++trJvOwlmjPdaXH9VhKBSRsmPHDm6++WY2bdpESkqKz7a48sormT9/Phs3bmTUqFHce++9Tc4rKSnhnXfeYdOmTWzcuJG77roLgFtvvZXbbruN1atXs3DhQq699toW779x40aWLl3K8uXLue+++zh06BAfffQRixYtYuXKlWzYsIE777yTOXPmMGHCBP773/+yfv167HY7t912m89W8f/30EMPAXDw4EF69erlu1deXh4HDx5scv/y8nIA/vSnPzFu3DguuugiCgoKfPsXLlzI6NGjmTNnDgcOHGjxvRw+fJg///nPfPvtt3zzzTds3rzZty/Y3yUrK4tnnnmGk046ifXr17dL5EFF9J2Hsm4UbaSlyLsj6devH2PHjgVg/Pjx7N27l4qKCsrLyznllFMAuOqqq7jooouanJecnIzNZuOaa67hnHPO4ZxzzgHg888/byJwlZWVVFdXk5CQEPT+s2fPxm63Y7fbmTZtGqtWrWLZsmVcffXVxMXFAZCWlhb03EcffbRd7x3A5XKRn5/P8ccfzyOPPMIjjzzCHXfcwcsvv8y5557LZZddhtVq5amnnuKqq65i6dKlIa+1cuVKTj31VDIztSrCl1xyCdu3bwdC/12iiRL6zsLRTqFfeB3kjocpN0ZvTApFC+jWAoDRaKSuri6i80wmE6tWrWLJkiW89dZbPPnkkyxduhSPx8OKFSuw2WwRXScwX7w1+eO33XYbX3zxRbPtl156KfPmzSM3N7dJFJ6fn09ubm6TY9PT04mLi+OCCy4A4KKLLuLZZ5/17dO59tprufPOOyMeWyCt/bu0BWXddBaRRPRSwld/g4LNzfdt/0SlZiqOOsnJyaSmprJsmfbf4ssvv+yL7nWqq6upqKjgrLPO4tFHH2XDhg0AzJgxgyeeeMJ3XLhJxkWLFlFfX09JSQlffvklEydO5IwzzuD555+ntrYWwDc5mpiYSFVVle/cRx99lPXr1zf7N2/ePADOO+88XnrpJaSUrFixguTk5Gb+vBCCc889ly+//BKAJUuWMHz4cECzYnQWL17MsGHDfK+HDh3a7L1MnjyZr776ipKSEhoaGnjzzTd9+1r7d2kLKqLvLCIR+voK+OJ+aKiF7Hsat7sc4KhQto+iS/Diiy/6JmP79+/P888/32R/VVUVs2fPpr6+HikljzzyCACPP/44N998M6NHj8blcnHyySfz73//O+R9Ro8ezbRp0yguLuZPf/oTPXv2pGfPnqxfv54JEyZgsVg466yzeOCBB/j5z3/OjTfeGPFk7FlnncWHH37IwIEDiYuLa/Iexo4d6xPb+fPn87Of/Yxf//rXZGZm+o57/PHHfZPNaWlpvPDCCwAUFxcjpWx2vx49enDPPfcwdepUUlJSfJZYW/4ubUEEG9TRZMKECTImO0z992LY8QlkDIa5q4MfU7QdFkyEsVfA+Qsat1fkw6MjIHsU3PRN+8bhrIVD66Dvie27jqJD2bJlS5Mo8VjjnnvuISEhgTvuuONoD6VVvP/+++zevZtbbrmlQ+8T7L8PIcRaKeWEYMeriL6ziCSirz7ifSwI2F7oPbe8/eP49h/w1UPw6x8gpXf7r6dQKHzoE89dDSX0nUVEQu8V9EChrykKf26kbH5XezywSgm94qjz/PPP849//KPJthNOOIEFCxaEOEPRFpTQdxZ61o2rHhrqwRxkhr3KG9Hrwq6jfwE4KrXiaAZj28ZQtB2KtMUfHFgFo+a07ToKRZS4+uqrufrqq4/2MGIelXXTWTj98mIdIVY56pF8dSF4PI3bawobn7cnqt+yWHvMGAz5q9p+HYVC0a1QQt9ZOGsgPkt7HkqsdaGXbqgr9dvuF+G3pyjalsWQNxGGnQtHftAmZhUKRcyjhL4z8Hi0iD7ZuyAjnNAHPm8S0Ze3bQxle+HwBk3k8yaBxwWHvm/btRQKRbciIqEXQswUQmwTQuwUQswLsv9kIcQ6IYRLCDEnYN9VQogd3n9XRWvg3YoGb52bJF3oy4MfV1UAdu+S7mo/ca8pAoN3OqUuxLnh2PKe9jjsPC2qB2XfKBTHCGGFXghhBBYAs4DhwGVCiOEBh+0Hfg78L+DcNODPwGRgEvBnIURq+4fdzdALmiXnaY8tRfQ5o7zP/YS+ughSvY2A2xrRb3lPu3ZaP4hPh7QBcCBEPr9C0QZirR59bW0tZ599NkOHDmXEiBG+VbXdkUiybiYBO6WUuwGEEK8BswHfOn0p5V7vPk/AuWcCn0kpS737PwNmAq+2e+QB7Nm8GvHWL3yvPRgoShtH/JjzGTJ5JmaLtYWzOxg94yapBevG5dR8+R6jYc9Xza2bXlOgZEfbIvrKw3BgJUy7q3Fbr0mw83Ot7ILqTaqIArFYj/6OO+5g2rRpOJ1Opk+fzkcffcSsWbOiPfwOJxKhzwX8a3Dmo0XokRDs3NzAg4QQ1wPXA/Tu3bbcbrPFTqG9r++1yV3HqKIPiFvyNuVLEtiRchLmkbMZesJ52OzxbbpHm3F6a3C05NHrPnzaADDZG1+7XVBbChmDYBtti+i3fag9Dj+vcVveRNjwqubdp/Vr/TUVncdH87TJ82iSMwpmPdTiIaFq0m/btu2YqEcfFxfHtGnTALBYLIwbN478/PyI/8RdiS6RRy+l/A/wH9BKILTlGnkDR5L32/eabKurqeL7bxfh/vFdhpR/RdI3H1Gz7NesTZoKw2Yz9KSfEJ+Y0u7xh0W3buIywGgJLvR6BJ+YAwmZjdZNbTEgIaUXGK1ti+jL9mrnZg5p3NbL+12dv1oJvSIkO3bs4NVXX+Xpp5/m4osvZuHChfztb3/jiSee4JRTTuHuu+/m3nvv5bHHHvOdo9ej37p1K0IIX113ve76iSeeyP79+znzzDPZsmVLyHtv3LiRFStWUFNTw3HHHcfZZ5/Nhg0bfPXo4+LiKC0tJS0tjSeffJKHH36YCRO0CgDhqleGqkcfWNhMp7y8nPfee49bb721DX/Fo08kQn8Q6OX3Os+7LRIOAqcGnPtlhOe2G3t8IsfNuAJmXIHTUc/G5R9Qv/EdBpZ+RdqqL6hf+Vu+j5+Ee+i5DD75YpJS0sNftC3o1o01AWzJwYW+yiv0CdnaP/+cetBSM+0pbYvoHZVgS2q6LWsYWBK1hVOjL279NRWdR5jIuyMJrEm/a9euY6YevY7L5eKyyy7jlltuoX///lG7bmcSidCvBgYJIfqhCfelwOURXv8T4AG/CdgZwO9bPcooYLHaGH3qhXDqhbhdLjat+pTq7xfSr2gpWeu+xbn2j2yIG49z0DkMOvkSUjJyondzfbGUJTG00FcHCH3pbu21buEkZIEtpW0LpuorwRog9AYj5I7TvHuFIgSBNen16LwlYqUevc7111/PoEGD+PWvfx3x/bsaYbNupJQuYC6aaG8B3pBSbhJC3CeEOA9ACDFRCJEPXAQ8JYTY5D23FPgL2pfFauA+fWL2aGI0mRhx/FlMvvlZMv60k63nvM26nIvJrN/LxI13k/DEMH54cBor3/w7JQVR8OT0frGW+AiEPkv756t7410sFZ+pRfRtsW6CRfSgTcgWbGq0lhSKMBxL9egB7rrrLioqKppYU92RiDx6KeWHwIcB2+72e74azZYJdu5zwHPtGGOHYjAaGTphOkyYjvR42LHxW4pXvUGvw58xatN9uH/8Cz/axlAz4GwGnHwpGTltmCzWhbQl66a6AOLSwWjWbJraEnA3NNa9ic/UInq9wmVrCBbRA2SP1Fbhlu5uTOtUKMJwrNSjz8/P5/7772fo0KGMGzcOgLlz54btddsV6RKTsV0FYTAwaOxJDBp7EtLjYdemVRSueI3cw58ycvP9eDY9wCbrSKoHnMOAky4jo2efyC7ss268Ql8R5FdCdSEkeO2ihCxAQk2xZt2YbGBN1CJ6vShZa3BUQmJ28+1x3jmJ2qP+I0vRBenbty8//vij77V/bfgVK1Y0O15vvgGwalXzxXgZGRm8/vrrEd9/9OjRvPTSS822z5s3r1lO+4UXXsiFF14Y8bWFECErZOq/NPLy8oI2EemOKKEPgTAYGDBqCgNGTUF6POzZupYjy1+jx8FPGLHlQTybH2KzdSRV/c9mwMmXtyz6jiowx2m+eMjJ2CNegUfz6EET+eoiLcIXwntueevfTH0lWJObb4/zTmTVKaFXKGIZJfQRIAwG+g2fSL/hE4G/s2/LWg599yo9Dn7M8K0P4dkynzW2SWw55d/MHJVLZmLA4ixnjebPQwvWTSGkD9Se60JfXaiJfUKm99wUTbQ9HjC0okxRKI9eL7fQWRH9xjc1i+r4uZ1zP0WXR9Wj7xyU0LeBPsPG02fYeOBh9m1ZQ/2S+Uwo/pSbFn/Hn99LZXK/dM4e3YNZI3NIT7Bq1o3Fm0JmS25ek15KzXvX7RVd2KsLtIheL51gTwGk1j/WHmElCY9bu38wj76zI/rVT2tfXkroI0JK2apMk+6IqkffetpiJ6nqle2kz7AJDDlDK73w1k/7MPe0QRRU1XPXuz8y6YElXPHMSg4WFOEy+0X00LQmfX05uJ2Nkbxezri6QIvo4zO856Zoj63JvNHvEyyiN1nBHA+17Sh9HClSQtG29pVZPoaw2WyUlJTEjEesiA5SSkpKSiJOUdVREX00SNLqa/QxlXP7GSdx2+mD2Hqkig82Hub9jYfYV1nIIeHmyedW8csMt1Y/or6i0ZP3XywFYInTIvCqAm1CVj/OnqI9tsanr/cKfbCIHrSovjMi+ppi77hF+7pkHSPk5eWRn59PUVFR+IMVxxQ2m428vKBJjiFRQh8N9GJllVr1PCEEw3okMaxHEr+ZMZi6BSYONySzq6iaf+0sZrIFHnh7BSMnx3P6sCziqgOEHjRxL9qipT/qEX60I3rQLKDO8OiLt3mfSO1LLi74ikaFhtlspl8/VZpCER2U0EeDuHSthk1l8zKpQgjiZB0D8gawbM40tq/1wPtwpPAI/3n1e2xmA/NyN/JzwGHPxDeNG5+lLWaCRs++O0f0xdv9xlSuhF6h6ESU0EcDISCxR1ChB7xZNwkIIRjSRysb9Njs/vw0fgrvbzxM2QZtLdqp/9rC8SMcnDumByfHZ2HY/512fodG9GlQfiD4vmhSvKPxufLpFYpORQl9tEjKbUHoA7JuAIOjgsmj0pncPx2PPQn3KisnDunHx5uPsHBdPg/ZHVzqPd0Tl6nNmvsi+lbUu/FF9EHy6KHzIvqibSCM3n64SugVis5ECX20SOoJB9c23y6lN72xqdD7i7WhuhASc/i/i8fylwY3X20vwrF0CZRo+89+bisnj5XMHt2TYQYzojXWTSQRfV1563PzW0vxDsgZqfWtbWs7RIVC0SZUemW0SOqpRfSB6XANtSA9jRG9yda8Jn11gW8i1mY2cuaIHM47YSwAHmGiR3YOzyzbw1lPfEO5jOPHXfvYX1Ib2bj0+7Tk0SPb3qIwEpw1ULG/sQa+iugVik5FCX20SMoFt6N5Bote0ExfGesrZRAg9IG1aLzCb0jI5LlfTGHVH6bzl/NHUmdIYG/+IU7+vy+44J/f8uJ3eympdoQel6NS+2Ixh8i71VfHdqT4luzUHpXQKxRHBSX00cKbS09lQE8WvUSxNbFxWzChTwgUeu8EbLyWcZOeYOVnU/rQM6cHp/ez8ruZQ6l1uvnz4k1MemAJP39+Fe9+f5Aah6vpdUJVrtTRV9h2ZIqlPhGbNVyrya+EXqHoVJRHHy38c+l7jG7c7l+5UseW3OhTuxya8CUENDrRM210wdexp2CrLeGmUwdw06kD2HqkkkXrD7Ho+4P8+vX12M1GzhyRzfnH5XLiwAxMjqrQ/jx0ThmEom0gDJA+QPtiUUKvUHQqSuijRaiIPtC6gaYR/RFvGdjUvk3P80byPsH3nZsCJbt8L4fmJDF0ZhK/nTGE1XtLeXf9IT7YeIh31x8iI8HC/+IOkmeOwx6qbkqnRPTbtfdnsnqbpyih786U1TixW4zYzGp1c3dBWTfRIiFLSx8MTLH09YsNYd1selvz0AfPaHqeyQL9ToE+U5tuD9E31mAQTO6fzoMXjGL1Xafz1M/GM7FvGlXlpawv9HD6I1/x5NId5JcFTOJ2RERfW9r4vkGzbjIGe8evIvruzk/++S2nPfwlX2wtPNpDUUSIiuijhcEYfNGUU28jGGDd1FdoKY0/vg0DT29Mu/TnqsXNt+l9Y1tIh7SatMydM0fk4H7SwCFjDunCysOfbufhT7czqV8aF47LZdaoHiRZkzVbJVoRvZTw3EztC+nqjwGpTcYOnK7tt6dC4eaWrqDo4hyqqMfl9nD1C6uZPbYn95w7gtR4y9EelqIFlNBHk6Seoa0baxChP7ASqg7BiPsiv4c9RUvXdFYF/3IIwOisole/43jjJ1M5UFrLu98f5J3vD/K7hT9w96JNnD48m0ctyZhqS6Lz8+7Qusa6NmufhwHTtGwkFdHHBG6PxOny8MtTB2A2GljwxU4SbSb+er5qRdmVUUIfTZJ6Ntan0dEtjECP3u2ADf/T8uqHzIz8Hv5lECIQev+mI73S4vjV9EHMPW0g6w+U8873B3lvwyEOuGzsXreVFWIzF4zLZUTPCK4bih/fBoMZcsfB5/eCwfufmE/oU7SxS6mlmiq6FfUNbgCS7WZuOGUAH/5wmJJq51EelSIcyqOPJnoZBP9FU76smwCPHuCHhTBoRlP/PhytKYPg8WjpnQHplUIIjuudyn2zR7LyD6eTlplDL1s9Ly3fy9mPf8PMx77mmWW7KapqIT8/1P1+fBsGnQGzF4CrDj75g7YvY5B3/KngaWj8paPoVtR5hd5u0SZizUYDDW5VM7+ro4Q+miT1hIaapiLsqPKuhvX78aRH5Q01MPKC1t1DPzeSlazOKkC2mF5pMRlITc9haJKLVX84nftmj8BqNvLXD7Yw5cElXPPCaj784TAOlzv8/fZ/p1lRIy/UhP2k32hfdPGZjZO+epaPsm+6JXVO7b8DPePGbDLQ4PYczSEpIkBZN9HEl2J5qDHy9u8Xq6NH9OZ4GHRm6+6hnxusXszK/0DZXpj5gPY6XIlinbg0OLKR1HgLV07ty5VT+7J/8wpe3ZfM298fZMnWQlLizJw3pidzxucxKjc5eKrmjwu1JuhDZmmvT7xN25bSu/EYf6FP6RXpu1Z0EXTrxu4VeotRKKHvBiihjyb+i6ayh2vP/StX6uhR+ZCZWjep1hCqJn3VEfjsbi0tUxf6cAXNfNcMaD5y6Ht6v3Emv7voRe44czbf7Cxm4dp8Xl99gJeW72NwdgIXje/F+cf5NUJ3N8CmdzWR17/YTFa45rOmXryK6KNCtcOFy+0hJa5zs13qAoRes26U0Hd1lNBHk2CLphzVzT341D6aZz/uytbfI1RN+q8f1jxxV522z57SuojeVQcNdWC2w6Hvte3bP8E44nxOGZzJKYMzqahr4IONh3lz7QHu/3AL8z/eyqlDsrhoQh7TTRsx1ZXCyDlNr61/MfleK6GPBve9t4k9xTW8eePxnXrfWq91E+f16E1GAzXOCGw9xVFFCX00ScwBRNNcemd1c+smIQvm7W9bWWBrorYwyz+iL9sLa1+A5N5alciKA5rA+iL6MFk0emGz2lJIzm3MHNr5eZN8/WS7mcsn9+byyb3ZWVjFm2vzeXvdQT7fUsCT9qeYbkwkP2kSg1q8lxL6aLCvpJbC1k6WRwE9ordZ/Kwbl4rouzpqMjaaGM1acTL/iD6YdQNtr/2uV7/0j+i/nK8tepo1X3tdvl97bE1ED42rYws2AQJqCqHgh6CnDMxK5PezhrF83mk8e9UETjJs5BPnGM54fCXnL/iW/63cT1V9Q/MTldBHhbJap88v70zqncq66Y4ooY82el16HUd108VS0cCeohUKO/Q95K+Fja/BpOug9xRtvy70Dm/2T1iP3i+ilxIKNsPQs7VtOz5r8VST0cD0Qakku8uYcdJU7jp7GLVOF3945wcm3v85t7+xnpW7S5B6yqnZrmUhKaFvF2W1DTiiEEm7PZIPfziMK0KxVh5990RZN9EmqWeTomNa1k0r8uQjIX0Q7PgE/nOq9tqSACferkXLloT2RfQV+doXxIDTtOvsXAIn39F47J6voceYpnZQjVbzJC49j2vH9+eaE/uxIb+C11cf4L0Nh3h73UH6Z8Rz0YReXDg+lyy1OrZdSCkpq3FiMrZ/wdnCdfnc+dZGXr5mEicNygx7vMqj754ooY82SbmwZ1nja2dVc4++vVzyCpTsgNLd2pdKzkiIT9f2pfT2i+grtZWpZnvL1/OP6HV/PnuktvDpm8e0dQG2ZM2zf+VCOOMvcMItjedXHdEevaWWhRCM7ZXC2F4p/OmcYXz4wxHeWH2A+R9v5e+fbuOrBBuWoiOkeSRGg1od21qqHC5cHolbSmSoqqQRIKXkuW/2AFBZ5wpztEZgHr3FJHCqiL7LE5HQCyFmAv8AjMAzUsqHAvZbgZeA8WidTi+RUu4VQpiBZ4Bx3nu9JKV8MIrj73ok9dQiYkeVFl13hHVjskD2CO1fICm9oXyf9lxvOhJOCPwjej3SzhqmNfJe9nfY/aW2gveD32j79C8SHV3oEwNq6gNxFhNzxucxZ3weu4qqeWP1AQpW2XDuz+fch5Zy8YQ8Lp7Yi7zUVqaZHsOU12hzH1KC0+3BampbueDvdpWw9YhWdK/GGZnQB+bRK+umexDWoxdCGIEFwCxgOHCZEGJ4wGHXAGVSyoHAo4B3VpCLAKuUchTal8ANQoi+URp716THGO3xx7e1piLSHXwytqMIjOjD+fOg5bub46G2TIvoU/po5+VNAqs3kv/6YS27x5LYvEJn1WHtMYjQ+zMgM4HfnzWMsYP7MTzVw5CcRJ74Yicn/e0LrnpuFR//eESJRgSU1jbWlqlvaPvf69lv9hDvtWBqAzuThaCuwY3RIDB7bSOz0YBLWTddnkgmYycBO6WUu6WUTuA1YHbAMbOBF73P3wKmC+33pATihRAmwA44gcqojLyr0v9U6DkOlj3cGB13ttDXV2hZOeHaCPoTl6ZF9AWbNNsGtLIN/U+BLe/Bt/+A0ZdqE76BFTqrC7Ssn/jwHi+AIS6NJFnNi7+YxLI7p/Gr0wax7UgVN76yluMfWsr/fbKVA6URNj8/BimraRT6iEpTBGFXUTVLtxZy5fF9ASLOha9zerCbjT67yGw0KOumGxCJ0OcCB/xe53u3BT1GSukCKoB0NNGvAQ4D+4GHpZTNCp8LIa4XQqwRQqwpKipq9ZvoUggBp/5ei6pX/UfbFm3rpiX0cgMVB7wRfYSVKO2pWqResqNxVS9otfLryrR5hhl/bZ5VBJp1E5+l1eSP6F4pvi/BvNQ4bj9jMN/8bhrPXjWBMXnJ/OvLXZz8f19w5XOr+PjHw8GjfHeQ1M1jhDK/iN7Rxoj+hW/3YjEa+MUJ/TAaBLURWjd1DS7fRCw0lkCQUkX1XZmOTq+cBLiBnkA/4DdCiP6BB0kp/yOlnCClnJCZGVlU2KUZdAbkjocV/9Red3ZED9oXTWsj+vw1Wq17f+9/8JmaXTPzQUjI1CabawrB5VeatuoIJGY3v2Yo7KnQUAsN9b5NJqOB6cOyeeaqiXzzu9O4dfogdhRUceMr6zjhoaX8/dNtjd2xCjbDA7mw/tXI7xlDlLYzoq+obeCttfnMHtuTzEQrcRYjNY5II3q3z58HLaKXUkvTVHRdIhH6g4B/9ak877agx3htmmS0SdnLgY+llA1SykLgW2BCewfd5dGjepdXyKKdddMSKX20x/L9kXv0oGXeNHhLB+vWDWi++7x9MPZy7bVe5kH35QGqj2jdtSJFXzQVogJnzxQ7vz59MN9camNT5h+ZnCN40uvl/+KF1ez87m2tnv/iudpE8TFGWTs9+lV7S6lrcHPxRO1/63iLiZpWePRNhN6kSYhKsezaRCL0q4FBQoh+QggLcCkQ2ONuMXCV9/kcYKnUfsvtB04DEELEA1OArdEYeJdn4OmQ6/1Oa029+fbin0vf2ogetMVMaQE/uvwtGf8KnTpVBdqK4NaMEcLm0hsPLCe+ag9PTCpj2Z3TuPnUgfxwsIL96z7joMimxN4Hz2tXaBH+MURZbaNt1ZaIXhf1dG/7vzir0VfDJhx1DR5f+QMAkzc9Vvn0XZuwQu/13OcCnwBbgDeklJuEEPcJIc7zHvYskC6E2AncDszzbl8AJAghNqF9YTwvpdwY7TfRJRECzrhXszpS+3XufVN6axkyrY3oQUurbMlr91Xo9P6oc7ugpihsxk3Te0VYBqHCOzW0+wvyUuO448whfHfnKZxk3ckW+3jOKfk1RQ4jpU/PZvPyj5CeY0Ns/Cdj2xLR66mU8VYtuzreYoo8vdLpxm5ulA2LL6I/Nv723ZWI8uillB8CHwZsu9vveT1aKmXgedXBth8z9D0Rbj8K0WZKb28Dbhn5rwk9og+Wm+9PYERfU6jdpyOEvtwr9Lu+9LUeNBf9CK5qTp99If1yZrJoaSIXbfkVwz+5lL2f9aZgyBWMmHU9CUmpkY+nm1FW68RqMuBwedoU0euLnvRJ1XirkdpIPfoGNxkJjaWRzUYl9N0BVesmFtEjeojcutEjen9/Phi2pKa59AGrYiO7VysiemGEynwo2alt2/ut9tjnBAZkJnD9JT/B9tstrB59Hy5hZvKWBxB/H8rKJ69m35a1kY+pG1FW00DPFG21c5sieq+ox3m99tZE9HUN7iZZNz6hdymPviujhD4W8e/oFKl1k+SdTNUXfLV4bM9G68a3KjbKHr2UWt0dvVvVri+0x33fanMISY2Tv/b4RCZecCsD/riG7ectYnPKKYwteo8+r5/GpgdO5vtPXsTVEDsNrEtrneQk2YC2efS1DS4sJgMmr0jHWU2Re/ROt6/8AeBbOKU8+q6NEvpYxF/orRHm0fc9Ca7+GHpPDX+sfy59tS70rci60WvqtyT0taVaCmbfEyG1L+z+QquNv+876HNC0FOEwcDgcacy8bY3qJ37A8v730Ka8xDHLb+FkvuHsuKFP1BaGJgw1r2QUlJe66RHsib0bYnoax1u34pYgHiLsc1ZNxZl3XQLlNDHIm2J6IWAPlPD18UBbULWZ90UAEJbMBUpQjRZNBWUCm8Zh+Re2mrjPcvgyAYtJbPviWFvkZrZg6lX/oWsu7ay/oR/UWjtzZS9C0hYMJrVj17Mju+/jny8XYhqh4sGtyTHK/SONtSkr3W6ibM0Ts/FWVoX0ccFs26U0HdplNDHInouPUTu0beGpJ5aJO92afn08ZlauYTWYE8N3uBcR5+ITc6D/tO0KqDfPq5tCxHRB8NoMjH2jMsZ9fsv2XfpF3yfeR7Dy79i0KJz2frXKaz54GkanJ3fqamtlHkLmvki+jbUpK91upqIdbzVSI3TFXZ1q5RS5dF3U5TQxyJ6Lj1EHtG3hqSe2gra6gLtX2v8eZ1wNekr8rXHlN7Q72RAwKa3tdcpvUKf1wJ9ho5j8tzn8dy+hRVD7iTBXc6E1XdQ9sBQlj//u25h6+iLpXKStcnYtpRAqA2IyuMsJqQMbwPpjU5sluYevYrouzZK6GMRPZceOiii98ulrzrSuowbnbBCf0CrqGlP1VI/ex6nbe8T3rYJR2JyGlMu+yM979rEhpOf4oitH1P3/Zv4BWNY9dhl7PphRbvv0VHolSvT4i1YjAbq2zIZ63Q1sW7irZpwh8u8qQtoIwjKo+8uKKGPVVJ6axOeHVF+wZdL7xX61uTQ69hTtV8DoRY5le/XInd9zmDANO2xb+S2TTgMRiNjTruU0fOWsu+SpazPOJuRZUsYsPBMNj1wMus/+x8ed+f3ZW0JfbFUWrxFy6WPUkQPhM2lD2wjCMqj7y4ooY9Vskd6xb4DOjjpQl9+QFsw1Rah7z9N8/c3vhZ8f8UBbSJWZ9RFWurnwDNaf68I6DNsPJN/9SINt25ixYBbSXceZOy3N3HoryNY+dqD1FSVd8h9W4te/iAtzoLVbGxjRO8mzuoX0VsijOgD2ghCo9A7VR59l0YJfaxyyp1w/Rcdc217KpjscHiD5tW3ps6NzuhLtMYmn/4p+KRs+QFtIlYnaxjc8HXb5gNaQXJaJlN+dh8Zf9zC2kmPUGNMZvLWh3D/fRgr/v1LCvJ3hb9IB1JW48QgINFmakdE7/ItlgJ8oh+uVHFgG0FQHn13QQl9rGKyNi5MijZCaFH9oXXa69bk0OsYDHD2w1qzky/ub7rPWaNtb+OkazQwmS2MP+sahty1kq3nvM2OhElMPPw/0p6eyJq/X8CO9cvCX6QDKK11khpnwWAQ2Mxt9OgdbuKsTfPogbCligPbCIKybroLSugVbSOpp9acHNpm3YBmxUy8FlY/o/060NEzbpJ7Bz+vkxk6YTrj71hEwdUrWZtzMUMrv2PQu+ew6YETWb/ktU718ctrnaR6q05aTcZW59FLKaltCOHRh4noa51BrBtV1KxboIRe0TaS/JqMtVXoAab9Uauz8+FvG7fpOfRHMaIPRs++Q5hy07/x3LaJFQNvI915mLHLbuDA/aNZtfBR6utqOnwMpTVOUuPMANjMBl/KY6Q4XB7cHhk866ZNk7F6CQTl0XdllNAr2oY+IQutWxUbiD0FTr4DDqyEwi3aNv9VsV2QpJR0plxxD+l/2Mya8X/DJSxM+uEeauYPY/kL86goKeiwe5fXNpAa1xjR17cyotd99rZE9PVBJmN96ZVtWLil6DyU0Cvahi70celgsrR8bDhGXIC2IOpd7XX5ATCY2vdLoRMwW6xMOPcG+v9xLT+e/jIHbYOZuvdfmB8fxYp/Xsfhfduifs/SGidpXuumLRG9rxZ90Dz6MBF9kDx65dF3D5TQK9qGbt20ZSI2kMRs6HM8bF6kva7I175IIm02fpQRBgMjTzyP0fM+Z89Fn7Ip5VTGFywk87kprHlkDns2rYzKfaSUlNU6SYnThb7tEb1/VG4zGRECasMUNlN59N0XJfSKtqFH9G1JrQzG8NlQtAWKtnlz6LvGRGxr6TdiMhNve4OSa1exJudihld8Tb83Z7Bh/gy2rPykXdeucbppcEvS4jWPXm8+0tprQGMUD2AwCOLMxvARfdA8ej29Unn0XRkl9Iq2Ec2IHmDYefjsm/IDXW4itrXk9BrIlJueouGWH1je50Z6121h2EcXs/X+qaxf8lqb2h7qq2JT2xHR61G73dy0CJ1Wkz6MR+/9IrCaGmVDCIHZKFRE38VRQq9oG3Hp2iRs5pDoXC+pB/SeAj8uhKpDXXYitrUkp2cz9er52H67mZXDfk9yQxFjl93A3r8ex5r3nmpVQ5TSAKFvS0RfGySiB0iwmiLKurGbjYiA1dZmo0EJfRdHCb2ibRgMMHc1TLkpetccPhuKt2mrbbt5RB+IPT6RyZfMI+MPm1h93IMI3ExYeycFD4xk5Zt/x1FfG/YaeuXK1Pi2R/T6ZKx/eqX22hh+ZWxAG0EdTeiVddOVUUKvaDv2FDCao3e9Yec1PvcvfxBDmC1WJs7+Jb3/uIHvj19ArTGJyZvuo+KhEaz431+oq6kKeW6ZX+VKaIzow9WR9ydYeiV4+8aGiehrnU1r0fvek9GgWgl2cZTQK7oOybnQa7L3efecjI0Ug9HIcTOuYOAfVvHDaS9QZMllyvaHqfu/4Sx/8Y9UVZQ2O6fU23REXzBlNRuRsnX9Wn2TsYERvTV8RF8fMqIXKo++i6OEXtG1GHeVlskTY9ZNKITBwKiTf8KIP3zDlllvkG8bwtQ9T+J5dCTLn7uTirJi37HltVpBsyRbY9YN0Cqfvs4r5oGCHW8xUR0uvbKFiF559F0bJfSKrsVxP4XfbNOKsh1jDJt8JqPnfc728xazO24MU/c/heGxkax45nYqSgoordFy6A0GbTJUryLZGp++xunGbBRYTE3/19c8+sgmYwPRsm6UR9+VaWWjT4WiE+iIGvrdiMHjToFxp7Br43dUfPIgU/Kfpfrx/zHIdBab7ef7jvNF9K0oVRwqKo+3mqgJu2DKQ7K9+ZyM8ui7PiqiVyi6KANGH8+4377Hnos/Z1viZK50vc3/am6AJX+B2lJfRO9oRaniGoeLeGvz+E6P6Fua2K13urGbm0uGxRRd62brkUpmPPqVb92Aov0ooVcoujj9hk9k/B2LOPLTpYjBZ8Cyv8Njoxmx9R8kUx22qbc/tSEmVOOtJlwe2WJkHtq6ia7QbzpYyfaCarYXhM5AUrQOZd0oFN2EnoPHweCXoGAzfDWf/pv/zTKrnerVN0D67WBLDnuNWoerWcYNNKZb1jrcWE3BawyFzqMXNESxlaBeaqGwyhG1ax7rqIheoehuZA+Hi19k47kf8K1nJD3XPwaPjYavHwZHdYun1jpDRPRe8W+pb2y9092kjaCO2WigoQ0lHULexyv0BZX1UbvmsY4SeoWimyKzRnJTw22sOvNd6D0Vlv4F/jEGvnsSGuqCnlPrdPtaB/qjtxYMlXkTrDOVjiXK1o0+hiIV0UeNiIReCDFTCLFNCLFTCDEvyH6rEOJ17/6VQoi+fvtGCyGWCyE2CSF+EELYojh+heKYxeqdGC1OGAqXvwbXLoGckfDpH+Hx42DNc+BuaHJOrdPVrPwB+EX0ITJvGtwSt0eG9uiVddOlCSv0QggjsACYBQwHLhNCDA847BqgTEo5EHgUmO891wS8AtwopRwBnAo0oFAo2o3NFJB1kzcBrlwEV70PKb3h/dvgyQmw8Q3wWiu1zuBRuc+jDxHR6+Ib1LqJctaNXqZBWTfRI5KIfhKwU0q5W0rpBF4DZgccMxt40fv8LWC60ErczQA2Sik3AEgpS6SUnddJWaGIYfSIvlnWTb+T4BefwOVvgCUR3r4OnjoJtn1MrcMVVOj1lMtQEX2wNoI6ZqOIah69LvQqoo8ekQh9LnDA73W+d1vQY6SULqACSAcGA1II8YkQYp0Q4s5gNxBCXC+EWCOEWFNUVNTa96BQHJP4IvpgK2OFgMFnwg1fw4XPgrMGXr2EZzx/YpBzc7PDw0b0QdoI6pgNUfbo1WRs1OnoyVgTcCLwU+/jT4QQ0wMPklL+R0o5QUo5ITMzs4OHpFDEBr4SCC3VujEYYNQcmLsa16y/00cUcMWm6+DVy6Bwq+8wX0QfIusmWBtBHbMpuiUQ9C+VqnpXq8swK4ITidAfBPwrTOV5twU9xuvLJwMlaNH/11LKYillLfAhMK69g1YoFK0sgWA0UzPqKk5xPMKaAXNh7zfwr6mw+FdQebhJHn0wfB59qHr0UaxeWdfQ+GVTWKnsm2gQidCvBgYJIfoJISzApcDigGMWA1d5n88BlkptLfUnwCghRJz3C+AUoPnvRoVC0WoMBoHFaKA+whIItQ0u6rCxY8gNcMt6mHQDrH8VHj+O+G8fIp66kBF9fQvWjSXKtW7qnG6M3sJtBVXKvokGYYXe67nPRRPtLcAbUspNQoj7hBB6p4hngXQhxE7gdmCe99wy4BG0L4v1wDop5QdRfxcKxTGK1WSIuKiZ3lgkzmKE+HSY9ZDWJWzoWRiWPcxX1tsZeuANcDcX+9qWPPoOyKPPTbED0Y3o9xTXcKTi2PziiKgEgpTyQzTbxX/b3X7P64GLQpz7ClqKpUKhiDJWszHiiL6xu5Tf//Zp/WDOczDlZvY/czMz9/0f/GsxzPgrDDrDV0lUt26CZeyYjQY8Etwe6YvE20N9g5s+6XHsL62N6oTsr15dh8st+fCWk3ylno8V1MpYhaIb06qI3mvLBFsZS954fm1/gGfz/goeF/zvInj5fCjYBITLo9dEM1pRfa3TTY9kG2ajiGqK5YHSOrYeqeKzLQVRu2Z3QQm9QtGNsZlb4dGH6C6lE2c1sdIyFX65AmY+BIfWw79PhPduherCkOdajJqMREvo6xrcxFlMZCXaKIySR1/f4KaiTlur+cTSHa3qsxsLKKFXKLoxVpMx4ohe99mD1aPXt9c63WCywJSb4JbvtQnb71/h/GXncr3xPeyiuX9v9gl9dMSzzlt4LSvJGjWPXreAJvdL48eDlXyxrTAq1+0uKKFXKLoxNrMh4sYjeupksAlV0Pz3Jlk3cWnahO0vV5CfdBx/ML9K3LMnwraPwC8iNkcxom9we3B5JHFmI1mJ1ogjemeY9E59EvbGUwaQl2rn8SU7j6moXgm9QtGNaV1E7/XoQ0X0FlPwPPqMQbw++GGucc9DGEzw6qXwyoVQvAPQSiBAeLGNhDq/UgtZiTYKIojonS4Pkx/4nLve/SGkeBd4vf68VDu/PHUg6w+U883O4qDHxiJK6BWKbkxrPPoaZ+jMGdBKFbeUR7/GNB5u+g7OfADyV8M/p8CndxEntZLI0YjofaUWLEayk6xU1DWEXR1bUFlPWW0Dr6zYzwMfbgkq9gXeiD4rycaF43PpkWzj6WV72j3e7oISeoWiG9OaiL7O6cYgGlfUBhJvMbVYvdJuNoLRDFNvhl+thTGXwndPMH3JWZxn+DYqq2P9a+pkJWoVzcPVpdczc0blJvP0sj08vmRns2MKKuuxm40k2UxYTUbOHJHD6j2lUc3/78oooVcoujGti+i1WvRCBM8hj7MaQ1avbNaZKiELZi+Aa5fisGfzuGUBeYsv0toctoNav18dmUlWgLA+fZF3/wM/GcWc8Xk8+vl2Fq1vWqXlSGU92UlW33sf3yeVugY3Ww5Xtmu83QUl9ApFN6a1EX0o2wa0iN7h8uAKEuXWh2gMTt541s14i983XIO1dKuWjvnpXWFbGoYco1++frY3og+XeaNH/NlJVh66YBTZSVa+3Na0Cm5hpYPspMaeRxP6pgKwZm9Zm8bZ3VBCr1B0Y1rr0bck9L7CZkE88VCNwQEsZjOvuqezYfYSOO4K+O4JWDAJNi9qkp0TCf6rd7O8EX241bGFVQ4MAtITrJiMBgZkJrCnuKbJMVpE3yj0PZLt5KbYWbtPCb1CoejiWM3GiEv51oVoI6jTUvOROmeIiJ7G9Mo6cwqc9zhc8xnY0+CNK+F/F0PZvojGB03LIafFWTAZwq+OLax0kJ5g9ZVf6JsRz96SRqGXUlJQWU9OctMupuP7pLJmX+kxkWaphF6h6MbYTAYcLk9EYlXjiCyirwmSYlnX4Ala/gAa0yt9E5u9JsH1X2rZOXu/hQWT4ZvHmvWvDYb/6l2DQZCZaA2bYllYVU9WotX3ul96POW1DZTVOAGoqGvA4fI0OQY0+6ag0kF+WfBG6rGEEnqFohtjNRuRkojKBNc2uIkLkUMPjQ3Ca4OkWNa3YN3oEX2TMRhNWnbO3FUwcDp8/mf4zzQ4uLbFMQa2LIxk0VRhlaOp0GfEA7DHG9XrXxTBInrgmLBvlNArFN0YX/ORCFIbax0u4kJE5aBl3UCIiN7pxm4OLhcW7xhcwUogJOfBpf+FS16B2mJ4ejp89LuQk7W+rBvvOLOSbBGlV+qpmKBZNwB7vT79Ea/H7+/RAwzNSSLBamLNvtIWrx8LKKFXKLoxvnaCEfj0tU63T8yD0VJEXxcq64YISyAMOxduXgUTr4WVT8E/p8LOz4PeB5pG9C1Nxro9kpJqB5l+EX3vtDgMolHo9cVSOQFCbzQIjuudckxk3iihVyi6Ma1pJ1jrdLWcXum1daqDTcY2uIO2EQS/Egjh7CNbEpz9MPziEzDbtDIK79wItY0RdZ3TjfBb1JWdZKOstiFkPZ+SagceiS9DB7RfGLmpdvaU1AKNWTuZAR49aPbNtoIqKuvDzx90Z5TQKxTdGD2iDyaEdU43Px6s8L2udbp9UXsw0uItAL5JTB2ny4PT5Ql5bqvLFPeeDDcsg5N/Cz+8qU3WbnnPN2a72ehb2KR776HsGz0jJ3CitV9GAnuKNXvoSGU9qXHmoJPJE/qkISV8v788srF3U5TQKxTdGD3yrQ8S0b+6aj/nPvkNe4trcHskDpcn5IQqQIrdjEFASYDQl3pfpydYgp7ns25aUwLBbIPT7oLrvoDEbHj9Cnjz51BX0uRXR5ZvdWxwode/ADITm9oy/dLj2Ftc602tdDTz53XG9k7BIGDt3tj26ZXQKxTdmJYi+v2ltUgJb6/Lb6xc2UJEbzAI0uItFFc3Ffriak1M0+ObWx8AJl96ZRvy0XuM1sR+2l2w5X1u3fozzjCs8e1OidO+XCpqg1srekZOYETfNyOeaoeL4monBQGLpfxJsJoY1iOJVUroFQpFV6Ulj173pheuO+jz3VuajAVNzEtrmkbPutBnJrYc0UeS4hkUoxlO+S3c8BVlpnQedD4Eb18PdWUk2bQvplAeul4eIdB/92XelNRoi6VCCD3A9KFZrNxTyq6itpVt6A4ooVcoujG+rJsgEf2RynosRgMHy+tYskXrqNTSZCxo9kxJQESvvw4V0Uet8Uj2CO7JeoJX7ZfDjwvhn8eTXvAdAFX1wYutFVY5SLY399/7pWtCv7OwmuJqB9lJwccOcOXxfbEYDfz7y13tG38XRgm9QtGNsZpbiOgr6jljRDaJNhMvLd8L0GIJBNDqxQR69CXeCD+UR280CIwGEZWSv9UuwaLUK7UyCtZEUhdezL2m56mrqQp6fOCqWJ28VDsmg2D1nlI8ErKTQ0f0GQlWLp3Yi3e+P8ih8thcJauEXqHoxthMwSN6j0dSWOWgT1oc543pyfYCzZYIG9HHW3xWjU5xtROLyUBCC6tqzUYRlZ6xvpo6uePghq+QU37JVabPuGD1T7Vm5QEUVjmapFbqmIwGeqfFsXKP5r1nJ4YWeoDrTu4PwNPLdrf7PXRFlNArFN2YUBF9cY0Dl0eSk2xjzvg83/ZwEX1GgoWqeleTyd3iageZCdaQdexBs2+i1UrQN0azHTHzQW4Qd2Ny18Az02HZI+BpHFthpTa2YPTNiOegN0IPLH8QSF5qHLPH5vLaqgOUVEenIXlXQgm9QtGN8UX0AStjCyr0Gu02xvZKYUCm5lmHi+jTvD58qZ99U1LtDGnb6FiMhqi1Egz027fYxzG/77Mw9BxYci+8NBsqDyGlpKjKQVaIida+Xp8eCBr1B3LTqf2pd7l54bu97XoPXREl9ApFN8YX0QdE0/71XYQQXDShFwApceYWr6cLuv+EbEmNg/T4loXebDQEr3XTSrSIvqnQJ9pMHGmIg4tegNn/hIPr4F/HU7NxMU5386qUOv28X25GgyAjxESyPwOzEpk+NJs31hxo9/voaiihVyi6MVZfRB9c6PW0wmtO7MebN06lR7K9xetl6ELvF9EXVznJCGGP6JhN0ZmMrXW6mi3qSrSZqKpvACHguJ/CDV9DSm8S3rmSe03Pkx0X/Fp65k1WohWDIbTt5M/I3CQKKh0x10tWCb1C0Y0xGgRmo2g2GVtYWY9BNAq32WhgYt+0sNfTUyh1n1pKqUX04YTeaGh7Hr0Xj0dS3+BpVjwt0WZuml6ZMRCu+YyDQ3/BVabPmPbtFVDSPDWyb4b2DRBqsVQwIm1I3t1QQq9QdHNsQfrGHqmoJzNRa63XGgKtm8p6Fw1u6fvCCEU0PHr9yyp4RB+QR2+ysmrIb7jG+RvsNQfhqZO13Hs/eibbsZgMLebQB6LbQOG6WnU3lNArFN0ca5C+sUfCrAYNRYLVhMVkoNibO69H9uEmY81GQ7vTKxv7xTYV+iSbOejK2MJKB0s846m75ivIHgFv/QI+/C24tDEbDIJrT+zH7LG5EY/BV1snTJ/a7oYSeoWim2MNEtG3VN+lJYQQpMc3ro7V696E8+hNxvZ79HrTkcCsmySbiWqHC4+n6RdJYZUDu9lIfGYf+PkHMHUurPoPPDcTyvcDcOfMoZw1qkfEY9Ctm2MyohdCzBRCbBNC7BRCzAuy3yqEeN27f6UQom/A/t5CiGohxB1RGrdCofBiCxbRVzRvhh0pWhmEgIg+TNZKNPLo6/0ag/uTaDMjJdQENETRF0sJIbR6OWfeDxe/DCU74alTYNfSVo8hI8GCEMeg0AshjMACYBYwHLhMCDE84LBrgDIp5UDgUWB+wP5HgI/aP1yFQhFIYERf53RTWe9qU0QPemEzb0Rfo0f0He/R14awbhK9hc0CffrCyiDlD4afpzUmT8zRGpssewQiaJyuYzIaSI+3UhSmT213I5KIfhKwU0q5W0rpBF4DZgccMxt40fv8LWC68C6jE0KcD+wBNkVlxAqFogk2s6HJStZQPVIjJT2hsVSxHtGnhc2jb38JhLoWInpoXsGyqMoRtGsU6QPg2s9hxE+0BVZv/Cxkj9pgaO0Lj7GIHsgF/FcQ5Hu3BT1GSukCKoB0IUQC8Dvg3pZuIIS4XgixRgixpqioKNKxKxQKmkf0R0L0SI2UjAQrJTUOpJQUVztIjTOHzd4xRyGi1ydjg2XdQJCIPqApeBMs8XDhszDjftj6ATw7A0r3RDSOrCSrr859rNDRk7H3AI9KKVv8OpVS/kdKOUFKOSEzM7ODh6RQxBaBHr1ehz4nOfK0Qn/S4y3UN3iodbq95Q/CX8dsan8efWBjcJ1GoW+M6GudLqodruARvY4QcPxcuGIhVB6Ep6fB7i/DjiMr0eqrcx8rRCL0B4Fefq/zvNuCHiOEMAHJQAkwGfibEGIv8GvgD0KIue0bskKh8KdZRN9O60a3aUqqnZrQh7FtIDoevS+90ty08Jpu3fhH9I0tBCP4MhtwGly3FBKy4eULYPWzLR6elWijuNqB29P+kg5dhUiEfjUwSAjRTwhhAS4FFgccsxi4yvt8DrBUapwkpewrpewLPAY8IKV8MjpDVygUEDyij7cYfQLZWvRUyuIaB8U1DjIiEFOzUbS71k2tN6K3WZrKUpJd7zLVKPR6KeVQdW6akT5Aq3E/4DT44Hb4aB64gzczyU6y4pGNdfhjgbBC7/Xc5wKfAFuAN6SUm4QQ9wkhzvMe9iyaJ78TuB1oloKpUCg6BpvZSHW9C+nNLimorG+x0UY49MVRpdVOiqscZEQQ0UfDo6/3Zd00jeiTfBF9o3WjR/Th8vubYEuCy1+HKb+Elf+C1y4DR/OGJnqj8ViybyLy6KWUH0opB0spB0gp7/duu1tKudj7vF5KeZGUcqCUcpKUsln1finlPVLKh6M7fIVCcVzvFEpqnKzbXw54c+jbaNsAPk/+cGU9lfWuyDz6CPPoP9tcwF/e3xx0n55eGZh1YzUZMBtFU+vGmxUUcUSvYzDCzAfhnEdh5xJ4/iyoPNzkEH11bCzVu1ErYxWKbs45o3sSbzHy+mptNWhBpaN9Qu+N4HcUaNFuJFGzxRS+BEJBZT23v7GeZ7/Z45sw9qeuwY3FZMAYUGlSCEGizUxlXdOIXojwaZ8hmfALLbov3a01NClozP7WvzyCjbG7ooReoejmxFtNnDumJ+9tOExlfUO7rRub2UiC1cS2I5rQh6tzA3oefeiIXkrJXe/+SI1Di8pX7C5pdkyd09UsmtcJLGxWXO0gLc7S6qJtTRh0Blz9EUiPVjZh7zdA4wRvLK2OVUKvUMQAl0zsRV2Dmxe/3YvLI8luraURQFq8hW2+iD680JsMBlwe2awejc4HPxzms80F/PbMoSTZTCzfFUTogzQd0fHVpPdSVOVonT8fih6jtcVViT20jJzNi7CajKTEmWMql14JvUIRA4ztlcKQ7ESe+UZbFNTWOjc66QkWyms1YQ1X5wY06wagwdM8qi+tcfLnRZsYnZfMdSf1Y1K/dJYHiehr9cbgQUi0mptF9BmJbbRtAknOg198DD3GwBtXwepnYi6XXgm9QhEDCCG4ZGIvKrw+dltz6HX8xT3S9EogqE//ry93UlHXwN/mjMZkNHD8gHT2ldT6Gnfr1De4my2W0kmym5rl0YdqCt4m4tLgykUw+Ez44DdcyzvKulEoFF2PnxyXi8XrWbc3otftGqvJQHyYhuKgZd0ANATJvNlZWM3QHokMzUkCYOqAdIBm9k2LEb3N7LNu9NIMUbFu/LHEwSWvwKiLubjieS4sfbpVBdG6MkroFYoYITXewsyROZiNot3Rrj4Bm5HgLQMcBp/QB5mQLa5u2nN2SHYiqXHmZkJf10JE7z8ZW+N0U9/giWxVbGsxmuEnT7Eu6wJ+5n4H+cFvIIgd1d1QQq9QxBB3nzucF6+e1L5sFBqtm0gybgDfL4lg9W4CJ04NBsGU/ums2F3iW+QFWgmEFiN6hwu3R7ZtsVRrMBhYP+pP/Mt1LmLNs/D+rd1e7JXQKxQxREaCleMHZrT7Ov4RfSSYTcE9er25eGD0PXVAOgfL6zhQ2ujTt5R1k+QtbFbtcPnKH3RIRO8lK9nGfNelFI+/Fda9BIvngscd/sQuihJ6hULRDF9EH+GCJN26cQVE9BV1Dd7m4gFC39/r0+8u9m2rdbZs3YBWBqHDI3r0loKCzUN+Baf+Htb/F979ZbcVeyX0CoWiGXpEH0n5A2gU+kDrJlSVyYFZCWQkWPnOz6evd7qxB1Su1PGvYNkZEX12kt+iqVPnwbS7YONr8N4t3dLGCf5XVSgUxzR6GQBd8MJh8U3GNrVuiqr16LvpLwMhBFP6p7F8l+bTCyG8k7HBY88kP6EvqnJgaE/5gwhobBLuXTR1ym/B0wBfzQeTDc56WKt33wJr9pYytldKu+dLosHRH4FCoehypCdYeeHqicwZnxfR8aGybnwRfZBfBicPyqSwysHmw5U4XR5cHtmscqWOv3VTXO0gLd7arCZONLFbjCRaTU0XTZ36ezj+Flj9DHx6V4upl/tKapjz7+W8vS6wdcfRQQm9QqEIyqlDsiKuae9bMBWQR6/3ng1ms0wfloVBwCebCnzdpWwt1LqBxog+krIM7SUzsKWgEHDGfTDpBlj+JCz7e8hz9UnmdfvLOnqYEaGEXqFQtBuzKbhHX1ztwGwUJNubf2GkJ1iZ0CeNTzcdaewuFXIytrFBeFG1s0P9eZ2gZRCEgJkPwehLYOlfYO0LQc/Vu3ytP1DesYOMECX0CoWi3ZgNITx6bw59qEVXM0Zks/VIla+AWkvVK8E7GRvt8gchyEq0BS+DYDDA7AUw8Azk+7eR/93rzQ7RSxzvKKym1hm8k1VnooReoVC0m8Y8+uYRfUtpkDOG5wCwaL3mZYdKr7SZjViMBm9E3zwvvyPITrJSUFkfvCKn0QwXv0h1+hgyP7mZPes+a7JbF3q3R7LpUGWHjzUcSugVCkW7CTUZqwl9aD+9d3ocQ3MS+eTHI0DoiB60qP5QeT1Ol6dDc+h1BmYl4HB52FNSE/wASzyfjn2cfJlBz4+vhZJdvl1HKup973tDF7BvlNArFIp24yuB4GqedRMu+p4xIoeaMB49QJLdzJ7iaqBjc+h1xvZKBVoW6oMOO79o+K2WWv+/i6G2FNAi+mE9kuiZbGNDfkWHjzUcSugVCkW7MQfJo/d4JCUBBc2CMWN4tu95qKwb0CL6PUVadN1ZEX28xdii0BdW1bNP5vBK3/uhfD+8fgW4HByp1Pr2js5LYWN+6PM7CyX0CoWi3TTWo2+M6MvrGnB5ZNjoe0TPJHJT7EDLEX2izeSL/DsjojcaBKPyklvMnNHXCXzXMBhm/xP2fYvnwzspqnKQk2xjTK8U9pXUUlbj7PDxtoQSeoVC0W709Ep/oS+ujqwmjRCCGSO0qD7UZCxoXaZ0OiOPHmBMrxQ2H67E4Qpe40bPyjlYXgejL4ITb8Ow7gUuNiwlO8nGmLxkADYePLr2jRJ6hULRboKVQChuRfGx607qzx0zBpPTQmcsPcXSaBCkxnWO0I/NS6HBLdlyuCrofj2izy+r00oun/YnKnNP5l7TCwxu2MrIvGSEOPoTskroFQpFuwmWdVPUiuJjPVPszD1tUItNTvRFU+nxFgwdWP7An7G9UwBYH2SFq5SSwioHVpOBWqdb67FrMLJmwv9RIFM5bvlckhpK6Z8Rf9R9eiX0CoWi3RgNAoMIEPoW6ty0BT2i7wx/XicnyUZWojVo5kxlvQuny8Norz2j98DNr7dxfcNvMDmr4O3rGJubxPoDFU2arHQ2SugVCkVUMBsNTUogFFc7sRgNJNmjUyQ3yVtGoTMybnSEEIzplRLUetG/yI7rraVh5pdpQn+kop6dog9y5nzY8xWXu96huNrB4Yr6ZtfoLJTQKxSKqGAxGmhwNUatevGxSHrORsLRiOgBxvZKYXdxDRW1DU226wXPxvZKASC/rBbQ6txkJVoxjL8SRlzAuF0LGCe2H1WfXgm9QqGICiajaJZ1kxFFUdbbCXZmRA+NQr4hwGfXI/rB2QnEWYw+66agsp7sZJtWAO3cx5DJeTxueZL9Bw915rCboIReoVBEBbPR0Ezoo1l8TJ+M7eyIflSIzJnG7lk2clPsHPSzbnzZQ7ZkDHOeJ0eUMXHLQ5057CYooVcoFFEh0KPXK1dGC73UcWcLfZLNzIDMhGYLp4qqHFhMBpJsJnJT7X4RvYNs/zTRvPG8ab+EcRWfwtYPO3HkjUQk9EKImUKIbUKInUKIeUH2W4UQr3v3rxRC9PVuP0MIsVYI8YP38bQoj1+hUHQRLCaDL4/e45GU1DjJSIxevvvwHkn89fyRTUomdBZj8lLYkF/eJHOmqMpBVqJWgjkv1U5+WR3VDhfVDhc5yU3XA3yXexW7DH3h/dugrvObkYQVeiGEEVgAzAKGA5cJIYYHHHYNUCalHAg8Csz3bi8GzpVSjgKuAl6O1sAVCkXXwmwUvg5TZbVO3B4ZVevGYBBcMaVPi/VwOooxvZIprnY2yZwp9CvYlpsSR0VdA7uLtKJrgb12c1KT+I3zBmRNEXz8h84buJdIIvpJwE4p5W4ppRN4DZgdcMxs4EXv87eA6UIIIaX8Xkqpz0BsAuxCiM793aVQKDoFf49ebyEYzcnYo8nQnCQAth1pXCFb5NcAJTdVq9Wzbp8WrWcHrPDNTbGz3tWH2sm3wIb/wfZPO2PYPiIR+lzggN/rfO+2oMdIKV1ABZAecMyFwDopZbOWLUKI64UQa4QQa4qKiiIdu0Kh6ELEW0zsLanB5fb46tx0RieozmBIdiIAW/2EvrCqnqwkPaLXhH6NV+gDSznkpsYBsGPIjZAxGD66E1xBuld1EJ0yGSuEGIFm59wQbL+U8j9SyglSygmZmZmdMSSFQhFlfn5CX3YV1fDct3t8GSmxEtEnx5npkWxj2xGtW5TT5aGstoHMBE3Q8wIi+kCPXv8iyK/ywKz5ULYHli+grMZJfUPwgmnRJBKhPwj08nud590W9BghhAlIBkq8r/OAd4ArpZS7UCgUMcmskTmcMTybRz7bzjpvbZjOznnvSIbkJPoi+pIa7YtMj+gzE6xYjAYOVdSTaDMRZ2m6Gli3dg6W1cGA02DI2civH+bnjy/m0c+2d/jYIxH61cAgIUQ/IYQFuBRYHHDMYrTJVoA5wFIppRRCpAAfAPOklN9GacwKhaILIoTgL7NHYjYYeGn5Pl/qYawwNCeJXUXVNLg9FFY2taYMBkHPFC2KD1aBM9luJtFq8qVgcub94HFxVe3zzRZidQRhhd7ruc8FPgG2AG9IKTcJIe4TQpznPexZIF0IsRO4HdBTMOcCA4G7hRDrvf+yov4uFApFlyAn2cbvZg0FNBGMVvmDrsDQnEQa3JI9xTV+i6Uaf7HoUXugbeO/X19URVo/9g35BRcYvyG+YG3HDhyI6OtWSvkh8GHAtrv9ntcDFwU576/AX9s5RoVC0Y24fFJvPth42NeMJFYYkqNNyG45XEmNQ/PVs/zSKHUfPjDjRkfPtdf5NO1yzpOvcX3Dy1TU3eBbENYRxM7vKoVC0SUwGAQvXTPpaA8j6gzITMBkEGw7UoXVpOXyp8f7C72WWROYQ9+4387K3aW+1z8UuTnoOo97zS+yfePnJE+e1WFjj62vXIVC0SUwGw2+ZiSxgsVkoH9mPNuOVFFYVU9qnBmL368WPfMmVJes3FQ7VQ4XFXVaFcythytZlXYuBTKF5FWPdujYY+uTUCgUig5kaE4SW49UecsfNBX0PulaRK979YHoEf/BsjrqG9zsLq5h2ohePOM5j+ySlbBveYeNWwm9QqFQRMiQnEQOltexu7imWXG18X1SefrKCZwyOHi+iS/FsryOnYXVuD2SkbnJrEg9l0pDCnw1P+h50UAJvUKhUETIUO+E7M7CarIChF4IwRnDszGG6GerT9YeLKtly+FK3/V6ZafzX9P5sPsLOLCqQ8athF6hUCgiRM+8gdaXS85IsGA1GThYXseWw1XYzAb6pMczMDOBBVUnI+PS4euHoz1kQGXdKBQKRcTkpthJtJqocrhaLfRCCK1BSXkd5bUNDMlJwmgQDMhKoFrayD/1MXoNGtMh41YRvUKhUESIEMIX1belAYq+aGrL4UqGea8zIDMBgI22iZDaJ3qD9UMJvUKhULSCdgl9ip0tR6ooq21gWA+t9LEu9Lu8tew7AiX0CoVC0Qp0ge6RHDyNsiVyU+w4vc1Z9Ildu8VIboqdnYUdJ/TKo1coFIpWMGd8HhkJFvplxLf6XP8c+6HeLwyAgVkJHSr0KqJXKBSKVmAzG5k5skebztVTLHNT7E1q2wzMSmB3cTUejwx1artQQq9QKBSdhB7RD/VL0wTNp69v8DSWMY4ySugVCoWik8hJspFoMzGuT2qT7QOzOnZCVnn0CoVC0UmYjAaW3H4KKXGWJtt1od9ZWM2pQ6LfskMJvUKhUHQiWUGqW6bFW0iNM6uIXqFQKGKZ84/LJS81rkOurYReoVAougB/PndEh11bTcYqFApFjKOEXqFQKGIcJfQKhUIR4yihVygUihhHCb1CoVDEOEroFQqFIsZRQq9QKBQxjhJ6hUKhiHGElB1TFrOtCCGKgH3tuEQGUByl4XQXjsX3DMfm+1bv+dihte+7j5QyM9iOLif07UUIsUZKOeFoj6MzORbfMxyb71u952OHaL5vZd0oFApFjKOEXqFQKGKcWBT6/xztARwFjsX3DMfm+1bv+dghau875jx6hUKhUDQlFiN6hUKhUPihhF6hUChinJgReiHETCHENiHETiHEvKM9no5ACNFLCPGFEGKzEGKTEOJW7/Y0IcRnQogd3sfUcNfqjgghjEKI74UQ73tf9xNCrPR+5q8LISzhrtGdEEKkCCHeEkJsFUJsEUJMPRY+ayHEbd7/vn8UQrwqhLDF4mcthHhOCFEohPjRb1vQz1doPO59/xuFEONac6+YEHohhBFYAMwChgOXCSGGH91RdQgu4DdSyuHAFOBm7/ucByyRUg4ClnhfxyK3Alv8Xs8HHpVSDgTKgGuOyqg6jn8AH0sphwJj0N57TH/WQohc4BZggpRyJGAELiU2P+sXgJkB20J9vrOAQd5/1wP/as2NYkLogUnATinlbimlE3gNmH2UxxR1pJSHpZTrvM+r0P7Hz0V7ry96D3sROP+oDLADEULkAWcDz3hfC+A04C3vITH1voUQycDJwLMAUkqnlLKcY+CzRmtxahdCmIA44DAx+FlLKb8GSgM2h/p8ZwMvSY0VQIoQokek94oVoc8FDvi9zvdui1mEEH2B44CVQLaU8rB31xEg+2iNqwN5DLgT8HhfpwPlUkqX93Wsfeb9gCLgea9d9YwQIp4Y/6yllAeBh4H9aAJfAawltj9rf0J9vu3SuFgR+mMKIUQCsBD4tZSy0n+f1PJlYypnVghxDlAopVx7tMfSiZiAccC/pJTHATUE2DQx+lmnokWv/YCeQDzN7Y1jgmh+vrEi9AeBXn6v87zbYg4hhBlN5P8rpXzbu7lA/xnnfSw8WuPrIE4AzhNC7EWz5U5D869TvD/vIfY+83wgX0q50vv6LTThj/XP+nRgj5SySErZALyN9vnH8mftT6jPt10aFytCvxoY5J2Zt6BN3iw+ymOKOl5f+llgi5TyEb9di4GrvM+vAhZ19tg6Einl76WUeVLKvmif7VIp5U+BL4A53sNi6n1LKY8AB4QQQ7ybpgObifHPGs2ymSKEiPP+966/75j9rAMI9fkuBq70Zt9MASr8LJ7wSClj4h9wFrAd2AX88WiPp4Pe44loP+U2Auu9/85C86uXADuAz4G0oz3WDvwbnAq8733eH1gF7ATeBKxHe3xRfq9jgTXez/tdIPVY+KyBe4GtwI/Ay4A1Fj9r4FW0eYgGtF9w14T6fAGBllm4C/gBLSsp4nupEggKhUIR48SKdaNQKBSKECihVygUihhHCb1CoVDEOEroFQqFIsZRQq9QKBQxjhJ6hUKhiHGU0CsUCkWM8/93lA+cwZvwbQAAAABJRU5ErkJggg==", + "text/plain": [ + "
    " + ] + }, + "metadata": { + "needs_background": "light" + }, + "output_type": "display_data" + } + ], + "source": [ + "scheduler = PolyLRScheduler(optimizer, t_initial=100, noise_range_t=60)\n", + "plot_noisy_lr(scheduler, label='noise_pct=0.65, def')\n", + "\n", + "scheduler = PolyLRScheduler(optimizer, t_initial=100, noise_range_t=[10, 40], noise_pct=0.2)\n", + "plot_noisy_lr(scheduler, label='noise_pct=0.2')\n", + "plt.legend();" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### `noise_std`" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + " Noise standard deviation. Now it is not implemented." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### `noise_seed`" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Seed to use to add random noise." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Miscellaneous." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### `t_in_epochs`\n" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "If set to False, the learning rates returned for epoch `t` are `None`." + ] + }, + { + "cell_type": "code", + "execution_count": 23, + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "[None, None, None, None, None]" + ] + }, + "execution_count": 23, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "scheduler = PolyLRScheduler(optimizer, t_initial=5, t_in_epochs=False)\n", + "lr_per_epoch = calculate_lr(scheduler)\n", + "\n", + "lr_per_epoch[:5]" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### `initialize`" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "If True, then inside each param group of the `optimizer` a new field is set called `initial_{field_name}` where `field_name` refers to the field in param group that we are scheduling. Typically `field_name='lr'`." + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3 (ipykernel)", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.9.6" + }, + "toc": { + "base_numbering": 1, + "nav_menu": {}, + "number_sections": true, + "sideBar": true, + "skip_h1_title": false, + "title_cell": "Table of Contents", + "title_sidebar": "Contents", + "toc_cell": false, + "toc_position": { + "height": "calc(100% - 180px)", + "left": "10px", + "top": "150px", + "width": "341.323px" + }, + "toc_section_display": true, + "toc_window_display": true + } + }, + "nbformat": 4, + "nbformat_minor": 4 +} diff --git a/nbs/images/MultistepLr.png b/nbs/images/MultistepLr.png new file mode 100644 index 0000000000000000000000000000000000000000..6f7eb01e563a9e8064f326b602f12deb731716fd GIT binary patch literal 6421 zcmai3cU)81woU*+L13aHD$NN*M`=pX(2+5MA%YT+-h?3tA^`#@p#^a;il7GRbtp=c zA|O%%8AVVjfdCN%LPROjODGfCJ4ffvoBQtE-}@tE?S1xMXJxN%eQT}F-AkrMqQZxS zArOct<~$k)fj~)sytY#i_&ZXnyb642`k%A*Hz(lzZ(Q+ngP2_L_wge5dwIAV4s`P) zdJw#kDyLLVDjmM+@9#s@QdPbF*8&xSpSx<`l|>q$WS7r*8zKZEvKRb7At`AGAdmyM zFlYmdn`tvc;bj)%p~DNSzIQf47*zv6AbDh7VT-I`kf@%Qy9p+hu&% zH0r{s-!jn1KQxbdYxn&Y_pZ~U*#8jT^9kJTabMP>IId~1dG;>v5f9{JMZ0_V-6j>4 zgbobYLk;oAvku9YGxxgL4O+LRS`u8N^sP9}?MG`Y24& z8IPxpW}%R+nnJW>n-0y1_BOu`0|GvQ_$ExUb9%OJF7|Sf<#OF_WJSaxxBFt_K3J7y zi<%_2$nu$sr)S}?TT-&D+%{ESxA|qKXE_O`UG3Aygx+SyT4shi+`@bm^y0U!@pDVv z=3Wb39c|rss%mFF$!h6Q97J{1SN@z5)q`G&eVEk@mh{w^SNS zZCo$6n3}x9*y{)H@K|j@|3R|o%RrUp3_eTsY1NA#KX$dd-@)bCi?2po zm8{n6LAFK*-rCw(X36x#Ae@mWy2cvAe^|@Utv)JwML(soUn)OA7-C zh26$vvJ`yK7x>~#u4^B9@!(Qsg2!?{jm}o_ls?}3+=z=$P#Hd$dJ<*CwbIQ-RrqU6 zAo1Sl@P&KeBQc@SW(z_`m-a-vl8hMde!-x9z~=%fB=%0Eo@*g-Y9jO`s|`1j#Ypn= zy$$;vp<1e5@fgrxxThD_B~L}>Z0$Y&f$kXRAi`&sjcfM`# zJnD~vbi;gg;3a(P5erZBKGPe1S;ug~Fxgvge0Cm&oAM9yH{P0JVT9Nd+5m(ovJtD;!T z!9LSnINPVT$&|jmFjv?~%c5Q_jhK*cc<3?hl!LO2ZIufIyabP=@Gd~?;$}?fzfpE# zBrumD7&$0je&pJ*-VD2F9RLvf+O-4r!YZh;P%*w8&IGh{eC6Jxdz4+AlN?`6H{E8^ zNf3gKh@^U7l3U&mQ8k4VjuTrIzxlS4B^~+~@>Kc7tJVgi>EZ!aBl86baij5$fP0V4 zr>yL@wN^SDgeKs>`gFQ?zNO?V{d5x;6#yfPJbNyHVce%IMt-5w(X04l030mxEVzrt zNbIkXdO3xow+KmU6&3*I-h+6H{ywMLoMLkvK-uME60~~YNWuM{@4y8BaC45kF#r^5 z0ne-Q?Z13n<`g>p;Ei|i>vZpVw%SaxAmE3<*YrL(e)=1l3rCD3Qf-WZ^`qdyzcI)( zLyb>>(W0U`fh@FL0mH1zvJ?C9r>{x59+&|;X;rM&Luq{_kIT%4GfX`U14D9T9yKq~ zuLisvW>KCTaP9w+Uhn}8N!T^AQG^3#{p=Y{IGev-ZY`1M#J4i$9Wx=r#k0}_4Y^X} zuM`bgm{r6k{ib=Qud~zhO$DDu1t`=4ArVmxwTehIFVGqHZtn^Pf~&UMv{RMeRDC-0 z!bjVcyA&IaPT6hJgp)vn=!VR^9DKxUN&I1@3G;DoL2`2PfZko5!pES?{kc4sX+e(G z{3hL#gHoUs)tIBF?^4Ps2a0+@#e7wL{*1@r^Zd3a4!lE^OUZ)~EwQJNZe? zUJd>YrFnh%5nFVc;1=ox>w4GgY}?~Jm*V7rXX#n`e8usVwzyq=yDz9OCqrHw8Av6c zg~58BUenusxIlPNBt$pn)g6O8`+4A0Yflq8j}m7eK!hU3BaXsH@^jOvvKMM~1(hDD zi*9J0s1_ii%FZ**CY=t1AAp6Yu3)~zgIp8kqMZ(R*^}huInY%Wn#{w!A7eON?e(51 zNQed*=tGFRSJY+W9i_D=D@p(#7TX>t$B(iIAgx{S@pK~>Cx&v7a@ zdt<&o@5Mb*tUx8vW+iW@+KdCB2wXCIlhJUpLInV!*R0u_#1I2kTP-JxQD&zq2Zo&1 zI3U+Z-VO#(Jp`cII&T`lQ2+yY8EDKQK!>GrFsL7#LRk&1JvoM)x)rJS+M9-A0jQly z1LUy;E7G3|`fk12nKwcK?TmkM%aLh_X;Co2eq@i_v|&bkS-XNMqOLr^lAe5Gh+?L? zb$<;hN)UDoaK$?$gFKhtxV{W^mMJ^OtbO7w&OZ)!5uN|3Oyn?pO9Rd_Y9evQBM62~ z$glC4;qkbxsfqSt#hLpP|Ap>6Co2$t6c0reg0$%1d2}KXdi2oB#Ut01*U?Q%KC;%# zo^f|FqnZb$zQCkV)lVUj-KIllb8yFb%I1Li)n8nSbA!oP*g~&mi@eK4^!;;9*3T@1 z6^#!a==J-ZkgQfwQn{D=lq?~--WvBJAyh#kLOTK}+JagV%n1r6E^Cs2giuVJ5!i$| zKg;c<8uy$H&MsgGl%48l3rOFiKpv{|ZK!8t&YchKjR8-rW+X%g?qbQ5ryFEJrei2M z)IdIS*h#34ppJ^iAr0(it8>R4@?L=3Hy4*maVn&syyO5kc9sx@f^mLcaNX$jiW}!Q z)Y|km-|S3%(u z0@<{k=JsCvG_t}JKvl^k$Sa9}`%$a;E~R;>Z?m&wc2B@46a!Ywj#1dxzP1;i1#)lQ zGTR#bsYpOxk_Pb8YLMw$Z8@az=UiP)EL+m``&vZ_!=GZ`nxpWnlfXMM+206EQku)3 zWxLX9)K*Ls<3Twf245TKL-xmS-tqx476!2}J~w>A4KG9N6eoMud|OfcEGx0njCm5% zO?^?V;h5lmSToF;Ss3Ly7tcjeV~t6s%vbxSL zNU>?`i%fypLI(+8h7;Cy~Jtbgq5)|e@* z{3yH`?ow=7wEQynck^SCcjrZ30f!9W>6r7m*ISFZ&)hP9aojTH>|XBLb~0+h!hCln z354YiWZyAa9?Q80FmvEKV30iB7E^?XC%4nl=9#;kuRW*j(%82g6ctGiR=7-6C5Q8c zD5o>;80D$2Y8f~^JXJAZl~#Hd-DXU>O}Z*crontIn5NFh;YY|V0W zBE??%3K~3ZcVKYl&w6f(5*c>J;{6T~hw=c1%)VD~S|eF^2=yEo&^=!BxFNX7eIwma91<(KFpOKgw8lP*0(2d*pgK_MGzWV<7akkoM5D8hQyVPKO3B+QE zfW|KAN9r(GjOrg@-_oqYb-_TWew=jhk8svgyOsHZ$75_M?N*0gTxQY93Ik?`w(B>( zlRm=?bC$lko9D~vezy`wmphNU7a&HfF_PGxjBxjkY8ZVz@d{oSHc7j$V&6H3Q+oOr zH0ip89-z`gqzfe|v{+(!!t-kL1o;tcyZ#yGX=f?=`*=mdlt9!Qu!E4>r0;R01*VHM zoIQ zi3zg7h_vo+jA3k!(10EXxALlO$Luz((Pb=N5g;PJ;`He9|I45-Uj)1Zf^O=ji_pab z?J8{?dkkA4Vx*~pe{V{9-1+)aya;?^=g{X15t775Toq?!ADwh&yWsHEn)5Xpz`KNK z%N_ORsCQ}W|D9AJoU?RO%m=a*9mwNes-{g|KVY&?gS8W>@R=si+Xx&<{TQh3NU zxlp4DahiB;SYc1>v@89i!@5GxPRn+TtC+a%5H*?<$u4H-WLG;9J5!f!u6O<1;ZWOr zS-<5ismXxAawlSy81~$JHt97ua>Np7V_B#vBw%9-xKWzRx+X@d(5nkJ+br*XD#K;T zgF{705v39(pynVo8xRtpRp-EspR-V{_Ho9Q^=+KGMjJ{-5XIynh zH$szQDW74!5`E(LQOgH~RuBUh`Bt%e$hn!2hF#Joc`g;whYrGStj}6cJqo-PcF-sL z%Jwfz*7J+#Hoox)1a&`jTX<~-wf?t+?L7mSQ|J23iWGWsrJJ4u^ZnGWb*?Fcav+*g zZu^Y&ncDXJ-HjI8(tQIH#D5ioz2mvLaG#?-f-i;o6uu7gM@L9eRg9YC{Ow-au$5$# zmj(-#0I>Sy|5+2Jkg^DZVK2bcEwLu$yw~Ta$yLAY&oyFHJyCm~x2aW@Qp&`!R}v+| zJB7h+JVAcRbV%VJibKOt)38Hp0h|EriT3ICUc=VGof^Rlo{iY@G!J+5#lLqX?MsA^ z+DKkol;QQRXH{I{)MPu8qwU(R_6qzpKxX}$U_hXa0IsDN7rf2(d(>UJwksh)MTlk- z1ypNOKnU1G^=#BjO+mj+1)*3X0IYT|^u#RRiexLD^~`Z|tkG#5y%vUfnb!h9?8lfJ zEh#`R0Z0wnEYk0wKr{&Fxe3v}Kga53SIhbU_Pjh}E390$Z)b=l&Tg*Yl$2#vmzq`SFlVFa1GpEhEzc9B`=g~FP>Seb&h-ThjWX5# zustuGsPGXSy6wn}2vppxZR``u#OC8)Gzn&KOj#z7z|YqjG0ixdoV}vIhsWh!r#?|I zX>V&A+_Fg;rDWc4^_cqOO&>)=8a5{wwr?#r_g7`!Up)y;<(*PDaQIlee{V)28~18- z_x%XZ5K@7JH+H+UM@2mKR=WPbwIUtw{P$Y%yq>whL#`?3PK2a+!JyL9S!?TD3u)g@ zJI^r(=>OLQ%L%-|)(Q?n!j8cmC*Ryn8vRuvfoEc#ko_V-%dkn|M1RkVoGa^U^OvF- zx?(zPw;24{r>AYlDkIrApmc0k%O4vbvc3oU@(UoOfPw{p_?3~%B@T_im4*_KmGKAF ztD6?vek}?uX z>_!{p3Ks}^tPLLy(9^W5O#l?|wsDQ_*|4-&1f&V}&vP$72Rz^d8f&&vq@iI`tlY)$ zU)*261L``w=J-Ee7Jj?s14uA=G%b?drZm0}wpsG<MdXcyc83rPP7u4-&lkJJF2b zmz6~J+&l{6+k5V4?VZm}KNL(^=)#vL=EB3^f*$l2Az-UgqA*rBO|>JkPTO4%7@P zH|EP+^=d`48S=7uA5Q`1%nM`wx{%)l$6N9ysF0ULE0(SN_7T;!25wuQZb)>3T44PCrrORb-gYQsUn zU1WSz=Ud-xHwWpjD$?)BkA%`m&N;ck5e@)jmJ4bK=?pgR37)hfk4vrG8LfwId7z64 z*80N>b0n2pvVD*XMokNpm%`IR&Fl8&z!jepzk6y*OGN0q6+O;T_WKmJFl=FlvgY@E zQ+RoME%DJ3@jbKM5FD8v|zQ+h21`fZVKOot~mm6>afUL-&&^=peX7lVs}f> z+0E7sRRvvT4?8AnVXoFmECUIHQT^)hmP|NDgkEac)Vgfo=-|lEg_rP8A(RET_N{%| zvg`?_hV9mnfnCJbmNG`UuZ84C9_QxK&8ZFSoAZI;AC}az^>-9s?E~(wM5E`7d3xk8 zbKRjOPmd}nXUQ&xOT)sc>f&!|_|Z+PWL3WHAFP!;yt5KohF8TQ;>M&$Jd5c&5!>gt z7*b@4M*K!-3NLY8uy)OXfd#L(j#z!cg2Bx=MT4gipdn3&DrW>&hcgPZSB6vXN}o}G4y3%?UG(X zDY@c&PJ9`kS!pwkPxLNk4iFQ$j+2sX;T|D8;AT_IT(%yQCbA|xPlWmAg~zX$ss${i z!WLX#`>@|>Zh1Vn6VQ`{HOhPJlG!R5KLXs;7gyC^7)na+mjcqmzakGHhyM)H*=_!9 Y2|44ps*0z9e^Ed%=SKL*n**_r@Fdo*WT6hO;KJF6O9xN1_lQ6i_|A&7#LVD;OAQuB;f0xO4b|T8^6nE zO&1k=GZ*)-PNpz&UtJt*>|Jauji}vBot!Q0?YP*u+1|5ITe!G5I18||+y0*m*zBFm z*%!W^LxDk19i+6JVPMelUO!=BQqzfHU?i8nd=gjnNI%MO_rO;p3R`bYlaQkR4oMY* z%Rxk;1zlDIxV-sJBZq*VyP(9fXKqg&K^+kgjt`57FOK-;A7l+gP3rf)8;&DU3If@=^bWlu127I&-XC-hY@B?KGt{3o! zQ-}BoX5hDBc@X^^MFt^qDkx@CLRA$92?0StBMDjvcvHPhr|~#Go>QpQHx%#w*Dr>j zKYw2Gd<3puK?C&$3LdHCko|v+)YG>|fEKd*Gci#@S{hBGQh#sykOG;Yu>6z^f7>ATJ^?)~*JVYPm zE0P5K4Q#cTJLuCPYY!HjZy=Q&e*?!Uk|LQD!Ws@3e<&dYEu=e7ktU1xcxQb)b3%^h zJ4?ZedF$^F+3;1u#sTq89ky9liN-o`{!)wz%hxZkb#R3{OH<@cUiTcO<~~xR=Sm1+ z5O6yfN1YOE+-c%=xs9(sRQ&dVuib@)K$>8Ho&(`LOnh<(N0&T4fDt1?7;J#c7yr*2 zEj7#zTZ=;Hd`=u39@r@spj;rm=((S^nAR4DBuW2O#7)N6H_K-U+i0;+v6>aXve1v3 zz+LXVyYiQ_A z&@uX}3Hjp{o37C2LYkrnBIlC;4C@o!~&`o_m&tBnT)wgpUw6HT}Jqiq&zLgM4`%|?|TEmL=1;_MYiR`1K)3=#lS5Zk1S`2)i%5v!94?^eLa8VI)*T zvna#jS~E;zy78(Do%K>v?rg|NXXQ<93xif_%V?OHI#7e-3@5ZLo&|I}Fr2d>Kh!oV zZ#IUspsX4u;yt05U*zR?F8@ut94`E5YU10UDel=BO|vI;-|R-r&CB~DBLlzjcz40; zJC0SmC~IUy6NW7)ckk4jIWel8osL68Dwp31D|{&Q0^V4a#j9csNOHC{W#dRe5VlS# z)as+bc(7!Cc`z01uoLMH%h(DpUPQOL4pJGc^1`rrgym5P(|*c1Q~XV}FFTE&8Nf|} zZWaXZb7wHXR2R)WCR58Zy_GiWq7e0WdoF=t;*{Uwl;uzduCyJ}u^z^~mHm$yXB~Tl zsgz`6)N^^k?{C}3aT1+e82h?XoB4&yOBcU~BVO%#t3EROHt}du?%TWFEUh+=_fYQ} zGhhXH`;m~5r}Mwy`8{1}2)G^ouCtupn<#*2ONG40}j^)hvR3XpC;fKr5!CycfL&cX_XsGn&f%mYX}_V%Pqe_VV}JSdOpV2rzWA$zmOTPc_i7 zHK=#7K2+dyBw>Na*`MCNKK|X%!%nt&urxnavj1?yJeRM2Bw^y<{(^m={e|I{-djkv zarDobSk$d6ew{nRE_p&5;UP)A5~X&Fs>Jt`*D6xvFzEVoemL$U!7&ElE*5*q%_B`# z%_jax=Fa(%;d18566yE)>CGwacym*dQ;vbr{1Pfhg}9`o8I zn4`XNQw&r1*;vIh^|C8)8!ArMjI22%C%#QQpO;_G@4!*4<`!xo$E6HSFt8Ye9cXBo z@aW61Sf~Q26w1OH4*sNtgoG$%iv=$CUmdTeQ3`wg-5WKhV_}K9S@+MnS@RzMdc8wf ziF1#!)Jw2?V@~R5Z=)ol-4rQrtX}nSi)dSfqfNY9ZMtNAeJJ^Xef?)e=_r~p=rj`@Zo20dE8gonN`_)n zjkUNQghxgi?@i>$rhZ@?b`$unut(R{;vrhm5mdR)#svO6OwJ%0z|TRZsE zym8r3_w_QPHzJx+ZHVn9eYy=!~_PHZl;+zcxAOCF-HZm9`sXkwmmGy z&c9{o3=>(e*xX)BYxWquP9)aFzkkd%{lh|qI_)jKM6kP9@f$A~8QTo7QQP&BwO86t z>o)a0*zTZ5k_$ySkpuM%UwZaQvl&rY&1ZJ?(j(R)FYG;Ds*23uGuF+$okN9Rryh_x zLxSdumeS9mr^6daLQB6erspVgXMY!zEhUv4U7iKIz1?ib{yJZI`UJo9!W*)A`*<-9 z#M(liyYv6tPq28@h8mpcCeb z`C>5Z*X4c4*s7*7L;S}Nhg4?oD8SOFn*`Y}CpfyY7fsjf)62Rc@+j8F!uKP+No*sJ zMPyU0c9!jW&+N7mPdz*FwM%0q894(kQmg2A%iVo(G4HiEuNZR1Avi(E)(J*k-z9nz z89F1mc)1+T1?2VG3b$u(GO$4fhSD0WZk4A3vhHtMniIrCsn+%tLk6Glnq5Z0MeG`< zi&@@MwVI$E$~}I(F-pjZK&NR^8M~?npY&cWb&Uh{G}6ph%>(I2=)pT6_-&6L4S*QP z??+~Z9vCy~)4fA99C_d03r<$)@}Y-)$d%J|lz41T1Ki{Mq*w+=jjBZoo2hF4nrcK5h1dU7QV2tv9UN@Tfq&TuW4o!L4pb$M=toxeyU#CVcdrUGXcA zx@sV5dd-O1@Dcs;yfza}J`YI5E<=mI`O;t;%Yc@>j;CkK_)L zH|WXX=J2N0WX0;J;H4?}`}}ZH1fZeA)z&1}gSoRc?-R#Ik7K`>-SI31DXH*}w!^yM zW{1u0C`#ca(5@QT9{E|sOuxMn7qBvDmFAz28*ScP(T_D-QeA-pPu+R8ecQrr#N>R$ ze`2vBTzzOr9K{AS_H~h_TIY<%4>pXEqpmsItyRWskIyA46$iGDSMvt9_xI-u#<8ml z>gC$pq#rCg(b?O?5}4F}0f23~NCA0cZ!+(L)y!UKQATwOHxpBYN|F4k1t+IrCk&HD zg={1tYm?JwZ}{a_clL7adKwXt%ur0S8t2_|3hQIp)fQI@^(D<&nNh_0VF>5TztA=j zbLOn)>a-4bA|sp9N|cuZ?2$v{)xr|oI?5Qj%Q5CpW}K+^v$2KxiyWDm4S`|wbd#m> zZ`XcZ5sLuWA?SzV4Qi3zdch+PevGZ-A%CRWETN!T&s@3Ax7gSY+Xr9~7Xj3k#$)?u zrdVaBL@iPxn)Iv(TXdqvR0fz?XrIyM`Sx%lMXUKZWubh=(7b`a+Ow(unm zJTjKa;X-vhvqsEpsm4VwoKvUB&&u%)$q9LMSThb5Y|rGwXt1HqWMqMMGH6m233w^x9&!%b~%x0gd z@;gs@>Ri^7cjOzpmNgq}NLE|j2LPX|vrr=CwHsNgw=PyICBzneBSHAa1CEiV6@j92Qd=WFUgd609!gNs#*ae?$z`SDl|^+|6k z$Lot#B2zEdG_TXk{<$pM+>cOZ3Gr&a@PnThsQ7l@P|+>fN+h>VjCW=msy{GMjX8_j zhnR^(5#JKwvu)uDrb}RDHrD>qZRX@MAFK2B$&-lIY;j@ic)n-$eYoCU zG|wtJ-3eE+T)5bvMc@=A7dX-uNv6l~dtEHSga$`IL9Q$>PxhsvnY^5OKvx zs?d?jnxd%Cb10_B1lO94Hn|;<0-&(9+V)~+6as;8_};s)5Tes7WSab&DtJdn=d{rY z8$rlA6aWkF@%M+MW``eCZ=O*x8)kBH^6@rL{ajaiuuv}TP=JSnR0Ksm5d*iy_gM{T) zc=$3I#q2MoU;zIE&YXJVeLct7H@2~|{h?nrV~`6u%S5t0HQvJr>oXd{zHMT6AUU!A zk6KVdF^@^29nT@3S7rg8!S19r-fiAx?T3F)r0r%mr;964*=i+4(+Jmm!)$hT;gx7s zHT~TCS#DX1I~xBJ^($#1R4&4NEFh^d5wMG$n2VZuJNS~GyhJ5+Bwrf55ecC|kDm}E z1|gtk7;J!3c!Vr7y3O(0cc0eXtXyUB?P9gGF(%31Wp!Xz$NDRH87hUQ`z0YuvTii8 zN?!5Ed{qSg8yCD8wk5OPC+u!gnfFPVSvVDw{G+)G4#4lb9F$_kXS>ntl-zg8?dk>BAG;|4;x3 z2!HJ+GXtLoyr^ld#HKYEn)g_rttLa;4{}LP^>6{0wHcB~uwn)$mjf#JkU0TALxJJ_ zoW1&EoZ+!^TlQ(vt!ypX`e zJTR;TK0!nK+~rHv8E9H{&%C6@+||~F$)HUg0MrMk|EOB4HPuF@_VTsLldGwt7%pMd zTE({ItNsc<%G>S&QO0MMIny6V(?`3*nspyXu|1ik_w+trQ7X0*uy~;>-zd`wg=b~{ z8i(h8OI&Y(?Mq*_d0BAarBnPHDf085;MrtK;Q$<<7At2QOQ-MrDKR|McWdeZPlL9d z_mck(!00Oc1p48)UK${TI*J@_<`hBVGWGMjl)EMwZV``crqUN-xn-r7spZ1CM4ryF9N28H53bEJ2|te7xyO2YRTMuK8ClD7}=d^y}L zD5u;P&+v{uw_@bhH(74pNMs%(K6mdT$z;K#_Ds+*+;gC04XU@?ul zJ&tzE+c=zSH;LLk$mU$+_Atp>24xRoyF@k7H%+X>7wXmD)lyc5#cG{yXq4W5y%OJ6 z^?(IT8jUk?%yo2fH@>dw%kgd^_RQDn%;myqL^lb%Tg|N6011?;MTK%W?a>8iJ2ahq z=47PZ|h&>AW}bveZK0)1iz^|FCJ)!-Oe5I#JG;7r>- zXrO~~ZI*pQaN;01IuIN|2oAg8g5S-%cetro!NMIq*ID*Urktp+mB`k^I|u43&Z7>^ z{&N$x^92;m)h~OnB|_OyVKy)HvDFpt55OetALjlPo4p^AlFm8MVCK)2cJW~0M}y*Y zV?-ex!y5%2gToheLa&g=q!^6@-gF{29O=RMGn%SVQ+j;ZN(z`)c~D}|r||Petw@L3 zx?f0tPPs9G?+$+Y$L+_x{#dI3H9C-$fJbS3;|0n-D$?HDX(X~=w&g+1v{edpg#KA{ zKgWHY;l6LN0C9-}&$Y+pH?f*{r>yvM0S?SPhv8(b&UeeZDPqnb2+X_h1}K~Gf8i_O zcPWgTTa7tCw*GMVm%rm^Dz2*OXC|tYm6#BDc3mk*%;}rw)i?J>rX@|b->gs(|AQy0 z+cTV?|EQBxB;dWCioR%nFA8>fOYiKq{ea~ozDDu=vY}Xb!q{M|74t*65Pmm6@xx>X z0y!cU9F+{C!nzkMs8`3pr&mOS?!{Rxs21dMGbFnA%EiM=hK)TfGQ%HXAbNdHA>EO z2|wOacl^_>HosWxKzfbx^p%=(`UYjWfn{K!`vYR2YBq9}m~;wVfeU&R+lD z7%;ZOxWnmirQFC4PO!aRJs%fUW#ku>7N7bc&$O7Peu)s3-%5M7?{DY>LprexL*t#& zaQ?7@K6x)!or(Rzp&{`g{4Lc*j){N@YRGx&PNptcIowNg@P2E<8nJs||e(ND>vxAN@hY^!9jJq<7!@!3$<9fBNx1Kcr{PkT)d`)CLr-XD013*15 z`%N}LYvMW8rSE=R*!7K~R>HVahGE8mXc-4KMa=Lj>!Y`OkpE~t*)!e5YK?Uu9Wni7 z1PGwjxjbYk58?>*kdR7?n9o=c#5*oiG2512$vNGT+v@VyTyQv&ak(vL;=f%%_QzB$x{KxDEd*VzV!boA^8&K zBqH_0nv>3W^!RWRKE4EAO`s=2$ziS?)fa>SWb*9;ebhri&mXUDviyeo%K;lh4lk)A zHh9p3Ro(w6cQwp3KDS(lC!WaMZl?!PP)%sIOR5wZV#nEZBPV9jcjeS2T>bg(tlA3& zRaaeU_3rr! ziS40Fh&>g3UAEz?qR|NJ2dedS@he6$%HDvJpUG8^2J|Gw4*1f0(NZS4bA)zxztW^h zmV+x7ds}ZbUMujCb+33zgGyyNI`U5vbJ4M}S!0QO;r9w31AkT-{#tbgli)MrbY4mx zI4Bh4w(;8w+K(+43TTiJ1C!j&LeQUDeQ@3iCCCBvFI3W!cC&dCLS-7V1y#s}XJaJ39Ur?!573AlYOBn+9w0AdZ z1(2KjhHs(Bo^M^Y#KG(cbsl7FB{W}zGqfd(1y=}tnI1#qjWQOW!a0}U$u(J0abwt@=T6rUoJ*gV+J9MZuej^iYBO<-I&2gO2<*3z|e)~h!_r{GgaF(d*aQQkn6;|2E#lN6O-`fUzRsmJ^t}t}B z*PJ+GWt*W0K4FV5LroC0OnE~T!~Zv4*eb~gpnq+IeKL_dwEL=^OBB@_JX$7#)y27g zCG*h#g~47KALtFQnEj{29&tAOynUCFrrM;43``4eR3CBp%4WH1JMp1}(A=m)8(@)T z0i?4~IrZmQs^6qc5*=#bXTgb&RUWF>0o;u)ZX?>QTFBkr)2vc-#5iugc#512&;bSC zXcPW<=xX`qz_j?asGJC||J7rCR$pP$qUT{-&)AXD%IV(d{nDn5)n(8GF{`@0}z;-%FyQu>OgM!}n!tsH>@k~>4!xnt3f^V6V5X$l~qbsy)`QECGPM6BW@;3J2fq@qT5Mam#SHVHB@(i>C&=c>5fnWi(tE)QKr&v5z{bmx<@(!q09 zfd1ZO3CVBi`=9uOCUkJ2aaRwQogFO=B~!yJ;l7@me9FNq1hzHQV^JRrHYhh5cO2*t z_7DW@pF#@Fb|7Jer%$2?I;8!*9`^0YN~uVeVSH@i@2aaz8DUxXFoY6#%y~A8S=NkC z0Q)0Lril!C<$am@=DQej+|5VYC;k%fwG+6FI4+sOJ^2u?7ZJX~e8(NL!gBt-I%^V= zN}vXdP@PJhtGp9XFpYAiDU{-vMxN`Mx5<5IQnL}w?Ojs>hH@`)2|A59oo{{F_7n{C z@iS{b&Jf!=-^9p(hLBi8$7_kEWQ4sYx*>@NRfj4?`h~I{IMY<+9|I~=G68yiK#BG` zMhq4D3S6f(HfHP=&hu2*2{uX-`0Z3h(v!^WuUj}`ss5!H$1+M8 zf$FFjet+Y{xDbG>YQzJu3^_LlObx+Cj@r{GI<11{Z#JyWbSphbdm(32WuqgyyTW*N z;6m?8No;Kr*Hs*kna~PfoDy z+QdbEiyVOT^o^ekTkt7WLd@g9#0ClU181Fx$ET9J+mUnfYpVH)@l>yVf&QAX^qz2t z6DxHt;p5Yrd;{nEZO-0yb=9w#8P(s`8$A4`9piWh_P8u=$&-C3w8gmvhakvjJpoGr z`2+K%Gr1+LpHQJi)DXf_(JZ)w4(CDI3!rKQ!d8i7D2K;B((}sk;*+?Xhvyy=brbQ- zK6YA>U|o=vTQ9H=A0(}uSy!cjXK~g{konE+h6=^W1@y%v92zZ@_a7pkp26Y&8lFl+ zaFkN6`%ppZXrKlF-JyWs$Spc(xJ*;Q#Vt!eU6F}$Tk!6le3K_C39D(*PJ6Krw0;~m zhfF|bgD%~2kjv2%fH<-=^;wlgJ>cYFe&=!agO?vP0*^=)RJZayZvq`O(rK|FLQS~G z1__6nYWi43iu(2S3Cx7g;Xj%&F!(P97KEG#D=s=~?5xaAsFs1K* zx)bON+OTBd1y(o<;lJmwa0TvwQDEy}cw`uPVWk*V3gS!Zqqx4=Arz1;w0X@#{}2wH z4E%lm20?z9zFFp`Y+mStobTByo>-n@x)f_*;~2DXyfSL*gEE&XRqEXA)p*{J_IKa; zzY6pKw6 zp8RY|tne8~cepkw!=8ZIvwV2)1AV&CLH@!x@P!dCKOS{+b5ovY5Ix)~J0VGr2d=`v z?O(xLG(^s3e;UHlgs}d~>tBVqbe7^?A6dP;iyz+IB3d5s>dL-Wd#8bX>W6Zszw4qk z%VH`w_P;E=y$lQdExZ2KX1RvkIC7W>L3y(6r9gQ|MR{nr(WdWV z$+-*=_szf^pOo44i-4!$b6?~tG#z1bapnpP$r;XG|JtFVHS;b=i|t^UMY|ri77EqK@5#|YLBmJi?1n* zKc`r3m@kNTw#}-PW^4u6g8cYz%0s>>F;Du^T3XUtpQTf_P0G5zn0Q*$>gQu1?V&mLZ^*zQtZ@8IZ9;d9nYXE!F;p}Kk+jqN!Tf@((@sj>zB>GP=%?>uD zY;?EDa<6KY+FP)P)jrZ3g@ZgVi~DCF`lyt|G#IHDY5F34)KSa#t;hALb4b!56qgx4 z-J1<6E^B^MIzLc~TrNJ==!?ss&k-O=M_c_8Z(eY-gGhV1CKi{|;(Y}Gn>;qhcM=3U zAymAzI|euBO;fDuBbsV^JMsvGbLac+Bh#a$$S9cS=71l;+{@_>r^r}cu>fHlOL}(F z=!#jQS*TKk3y_@puVUzP6-AKb<&Cn)5;Ej!rF^O^qV@5$RCZBM(L)O=ifugUb9>G& zTu%_NvY_3agI_3roF!<)Pk5sXm~WWi;O-4T{OUPSyL#;YI%<{S)>f5V4jukJ-ESNZ z+`2bYqW?s&6UR2I+4_RPKDu<+@C7CYj0$a3Y^hL3Hor4(@w&>ck9cTt7XmQH_8H?r zQ>NYQEaj^bdDa!A2+k6AMYb$8b-mt)eI+6!!qDEpsv~p0Y888H;+YSjF|#tKXD6*G zCy1ktueGE-O^L`SZpijqJ?+JFd6yCrM`N|ETho_)TlogfGIgIBlQQ2KU);xYvhGqM z@F_s@&9D%3*K+aH&~B%%(ftbohR7RKEU(!Xn4c{jkJ;5x%{JZ6Z4>lImT@O*bh1qS ze6gpt*N=4`wzNMx?q&(S7X)(XPN_aBWaq|wb|exPkf3@Jc^Vc7Wl;%Qs$*M%>%FdIMCX=qVF3DzWL_|nE2;7Vk~V_Q zj|(`8)D}@an<^RIfh}rw{af;RX4rth_42KsuxVfNwaNeqAM@^G{aTHVc}rQNjl3$r z9$*Vvr|{X2O&77n>I04!A=dg&i6q~x(n%%R8_J|!K~7d{badF*zHN*Qfb`1a zEz#b5I~_-n;vLH^c*e}N#M4N}COS0fTT}5$bB2%nY@G?rM_2iEJ|uMN*g$dl+TwF+ z=%0D(M^UtDGO4fsIt~h~oPN*+W0hmX+T4m5N*BP`zoAIXDk`+wSm1x6kAh{fTxtu% zz1PLn#Rr;8MyEIH9M`7Mwr`9NKc1zxxMUO+wzyX!cmrdN#@k}iZf#~MU~bPM1Du>% zrALKs<7scaYrfXy`1dN!4>`OkKCTVDn8i2o4bK}{3V@a9N+j_AmR~1mCHh3Xp#EW((|&pP|Pv>1U;>=E9J>c z#~E)GS7wqM6f`OG6B8Qqwh~w}x1m*ZS}Tk9s*%;H9*y6aS{o!ddkL z%*}27{MLFJ;?ZQ)wvb5%VoU{tBAnIr zcze{o(#19}V^(!EfE526OPXJx;|WBlK6+if{~Yw%T2{7_ zGz9dhT=EQTGURA1wB09djC5#^)pjx9QTDcGl>Btrz{n#!s2x%^gV^~1;dUNBDj*eV z*mL7>{T*0LiQQWifmxQp-N>6JJ>P$FyQTK0dUZPE-OBTO@+Yr#0aDt_ zh?idzND=CjZC<5k_cln#x14{v_Q{3a6L@!;8W_|h_WJW7A}1d_2=iyF^nIh1Lx`u^ z$enp8H+M1nA52|%0b4wV4d%~wevu0;uf1%blldb<1W@SPx0XF$cWedz)Et<_i*b}r z$gMG|l9t>AT*Ym)9b|sB^T@TtsLw?^TN>BlFEegP$^#72loD~2EyeQqlmmi#1>H zE3Nx9G^{(rMBxG9rEPv&fe` z^a{X(tjHH!^He^=_xxwkJ#tJDpI01%v<@<;B%wcC^Un7uBztznv<(-fpsdYrg#ibz z(j`6_Ten;|Howk>=sOD9ZOp<8>58WI{_1YycTc;}7+n(bnk8Qe#E)t4yu!mgm_-NO zRymW4m~~x<+Bq%b0FB3==Txn*KQoanmGY(_P`Q44yUOGVXfE=Yo0_cBdl}%oUCfhN zBKG9i`Q`o^w-=H|%vT3Ai3W^bs<3LjfI%KGm2>A=@!6B`NT{n)Cr02%K-u$hEJrau zG0@l$2yAM#n8P7Jy3%Wz({0iDZ=@8vMFVLRMn1Mov3DQ!aA1p8^J!u9VgP0$Ce&`BBdO%-5-mkyc<@F7&Pd}Eq{+D zNfx#y>EQ+e=%hS!(D@;|ht*gBCe9|y9BwoO2TlxS^35%g!;!K?i1Dulc^%e=8|2Wk zz<6Z*Z^eRiYI#Up;HKyho)0LAGG?qM^-eORZrqA*865r&^l54^E^~RikM(^@_TaFCS0#v9$84u~XPbe7^tZKhgOl)T z*LIXz)LJt>6DGHprRNn=nN5m#|MJ#9nfbe0_b0)u#mHKE<1qNQ`SC=xWbX>d=?i4P zE7L*3!UPxJeXb~8K#J%CPz2^&2oo1N-(H47&u-lxFQfxj zZ)Tt>n`pJeU#}GDOpJH-N;4+(Q zLAm}Do$*y7G@+0nVuGfDA&>CL(ZUTf)Zz#W`k`3csJ?g}RXt6d1;}MNcTb+SW9p5+ z%Nr&4=Ha#>IFn!$(QGn~o5c6dfg%m#8G(Juc{%Z%($XY|Q3z`Z%6^-~(*$ewDM}G?_gHLoxh0wdp)D*cn5HIe|r4J%(gj^D-=1F*G z;>BaY_L0q^s4NI>FkJcZ%McuURM66LXtZSHAU*szTw{SD(+W8#`yWU7`ffj?mrfk( zj}Umz8cU3eM)IvJVXV{(B}%9@gC;KSeOEtK-5Ltg&NC@(F@ak6jsNn$x;lNyuNqPM z()Nf_Oc^O_hf}F>c!`NJ;bSO5ar*k7_iJh19GeYsSv!Gfvx=SP85(w=|_Swy)VWyG``xpOOe_n zrBzg*?8Q=t=PGrXmCoCTWRj&4n$}nY6A5MMH3g5iE_p!Lsl3B`p#L?ksOO8Zd4npG zen1844_W>Ctf!(Vol99%D2@9k-q2s6RhPz1ep>d{H3e!3Ml%siuO-f(KlMTtu-}0! z!FUxGU{B5BsbDYT^mmbP`;0ryDZu47| zYVy*5orbmv3PnlM#)1F!HtVXg>1tDEivn0o{PsW($~yQ%k~A?#GZ1F&&b1XZ&6+U5 zd>m8QvoU0;c~!8M`1f~YITPIJ;7RMrJvo>)*-7{goV{kaz)hw66G$LBfL88Vt%k@` zkiJC(Ki35u(P$Eso`Xc#e(fW2SWK=gC5guvlmx)To#jU~Zw0efsfQS#^9-Gi)uq6J z0uUlQIW#X5f%`}e?9&!2en|e|f9xhznDq8_B=1xx+B`dHI}SmovD^?YKotGW@;*LB z09qv#zmg?wQI#b~+}_wPG=2h}vd;up$l2cWI-{+3oGe;R#{d=j=a^9$wZt`;+kmXa z5~NCJ!AI`(q%=XeqA;#VsJhDi@f*is;m(eSr78 z0#cPHfW>Wdr#2#WV30nreEgn;{r35{Z<)Rnx_bP?L6CjG9MR^fz`|YW|6H$KZPks5 zf6R}2;a6l>KygB24}J?cQxZ=2pjZIN7uP5+k%i9r0bvK;k(?uyPH8 zSPan@9TAzZAx>Y(DNp9+0Hb8KHVR3O>zPXYe2@O0^Y<>lJ z)p4RVwmd9{x=-aTND#v#~(+Aun|oNB{J@-y~& zR+__gRD4W#sNM2YYF`9NDcUM=$e-D4D?%$47ovj%AH?sp^bN%xTrx%ixM`C-q8}U`R(Bd5DDyU1 zE(lPWs+|m&2I*_qzNVX1|9M-MU1Jv|SuR<}N#4<6wl{?W6r6FyYROdUR5JzdzX6B0 zE-VBvM%qtz$-H!avg+Z{oN=QUmW?fyj!RXlt`=Z~$95#zc^b8<+YfeLmZ?Mp7qQp6 zjDJZo&_RWEdm<1`rwW}UBR?elvmK>C#l9{w-8LH0PU=oLX!H)TB>aZ0PSEn6shR>* z)1ZDj(pk=`qsAy<;V0T|;5xBAuMQZ1?Fz}dzSp13YER=JY(F00MO}7mTZGvKSFQRW z1UYoZ(cf?;_Jx~VrY4-h0I$(97h!TXw&hm&+YL}s3C8-^6-yPId?IrU%B>}d3Uz-_ z`)NA2HYV8pXecAwo+L~Bd8W;{rDlvFOh5Cs8Xq^gdahpWHL#{DBp_4QV6og&>5l4A zcxZqi*(#$bfO8CY%?$G%P1o3!0ZOg>#eBWfVh7-0wC?W!wWU4}oy&guP10Nk5EC6l zz@5G)J=>}uJG2|4nXx+CMtQ^RS$NO)5+=w};renWovRKsKHenR8kE0#kk>O>eltHx z4`9%yP+ab835BECdy`|GyXK(Z-GKn_;G~l-GMa0;ib8wYUo=LI!id=cMM(SrY}B6* zCHOr9?`qPqod|)4uE@J{I330AfEq5)jRV!W;chkhp~$m=%1;-aIZQL!%g_N{7;Pvv zy?gh5HQ+Y1A?FXTh`Yl^WWOf;b?eN(!N(fapVcpI9G_fHWwE5tAdlOivu6g3&cXuR z|5!ZaAzN2Q^IRl-zT=;ydR?h{P>*?K)$;~PlI}QL4iUHx7kmhCw&l9gayJ8*yc zcUsJ6j-B2^K$|_gdEd<96ZhY-;mdw#7W4&WXqT{2X$D0Om}XQK7w^**0t=`PY~>Q` zBAL6MXNGO~%S=%Ubjq0|PVDg6^6DE`&htP-2Z+eGQ^4s4zg4FhT4B?pvpu{(6UJ>3 z#%)(a(uvK)_Y?tN8pi*_frAQGs0&7J6Y4R4(?d?(A)9Etg{l<4<|x#Cu;|_PZAWeE zN|Z4SF8t$6|2234Meml`2M;8$bQk+L*z0=K^fDV@0MlDMGSnW@4K`;fbJ$T$#*P)L zZHJuzTAzXs8M4N?18WG4pITUDz^};ZTi&2T=)icdyW9mNCDH_R^XEikDMtI5JPl^! zRg4ZXizn3dB}C?^dABOdKAOKbTwMO87rSqCIZ2oyzoH1oJTI{xkkLRtAvj;?6@OfQ~BvO+06<)8_N{H)$fM1@JSF zZj4jV?+2IuE3y3+atezESKM&&rc*)8p3yY%WlDL-QF#afI5pzo;cHSw2pguG$)wk@+2lE~e`che_vH5%UJ6J+7jlldK_ncw!24kKMlvV#X zLY}14ffay{EQnV|c?0)|hLNDm=Kg&YO+g&RLQr|QUF)_6paedtT2Rw|Jg=s?!?z@4 zZU@Wtx3G5gZDjIptap69Z~+|c8`AcNFeA?zXXrh7$Qq%uKdvUqA&i+6VOc>9z}45cMs3`dT!1P9%ndap+fSB(`MU6D zVkTC=U>V@HGLJntKTy#1LnG-g8+MRPdx9CXJF`r^(1n3hp8r2gcEgdq4C93iW2yPQ zak{^0$a_H6!v7ZH`4L0yvdmAGc()?I`pS?ArW@df>>>vdDB1%Kh`v5w0NfD#Xby^T z-SG?_!_5 zVBZ!q8_FN#ymbkLo&KD`Vc=xyj)-Y8j{H+HN4WvZuQD=)&O>;L4hXY(Jy2+6TLi)k z5uh4r)m-?U0fCMfc^zi&lr~+oEFpy(22e}uh7z*X`@PUUGk@GlGCW{e-~^akn9&@~ z3wx&dsf=km6(Fx@ewqsk9q%bD4h_$vlOXC}H_>S>onjUaLmC(R#9A)Y-HZ<{&jC!O zFM+N+s{yX$po}(v;BwcWqX;CWjGml)tp6##L(*k*+@$&fd3{885S$OF?AR>n^0Ca7Cp{Jxc)?e(^>_)0d%}O25mx;syxz$k6c^lj!2lLI4z?HZaEO>=wa|8-HcU**6KrrGQo1kUiv`lC$gO<3!rLT{9c%hgTnr?G*O4aj+1uOmtR z^U_A_etH4=sK&;7o8Oa^8eQst-b%{WcE3HaUAPTioBNBpoy)Ki*Hqzfj03bGqe*lM z_hh)UZclpCgT-2v95#O?8c^wC07z}Qq!sLWiFQ-aRSTSBO^gmm$jfl2066u?SSpt% ztZt^xOa1RZ^BX_A1I@xA4>SmM&%{ji^G;bA^%H<1;f|+>Lz#+t*JJ2J&sU>lAL$eg zT;>gQ`yls!ynCp-b^xU3NA8&*#?OJczzItP{D47<&08KQ>k!aNLU0@l+$96SG|k!; zg@X0ksw*CjIv@Ws1zqIMP}!TWMaw?&5a4OMxlYf*>{RLoYnp@Wo1l5;S!#O{Kg2gV zd8%*1zS$bSI6+x!(19&H-o;Dhl7*y3IU!B<@?2u9%npDWPCRuiN_`aZ&zleORnF~e zgxf$rb%^=q7IH>=pf-G6C~Ym}Tw?7mB0-+UinLTd-Suaq`VjBZeqt}ga^T9{ubkos zu|OcLN7~xb=}dWY_ZI;!HTnFkXdBef?F+P_sY?hRMXaftzJf<{8U%pr6*jj*FYV_J z6!u?PZC2fraHM|>&OR`)I7hQdMZ-G|o(ANWSz90)c<^i=2g3S`JD?^{k?ddwS`#_D z18rduha3PuBr{>llz$xj{tEO@=VOQy;BG=$6Ez%$?`Oy`4(PfU?yFAhA7X)BTNM^j zzWI#+M@y#Ei@%^&-3Ms<2F6Bihn@IiyJtT9{k8(CZZG6&KQ>?{+m8lJ|F2jVipL(k zZQu3t_UUI|rOT1t%>x9+a_;jkSgLbu2zg)%9PQZ0k>g9SpI+UMsF>UCLuo&|^hl6}x` zuwAExSo{-PCWvV5;idZEgN0NXMvC1PI(Yh7=<$^L@cHSQ^hP}rZJEJIp$ny)do!S; z?!$=adVm6RPvyAvm6I-U4-scrBr4DoV7H-9ji`z4lB8@m{$U4|T3-4t87G&_p(FlW z9Q28WgFp&F8pa=PyC6PMP>&r15~rhWHV@$oNpsrddlwy6zqNVJ2)K>RsW)|haqzYHII3o{VlU}&*Zoiecc&kOMHmLGm{=(~r>$gO_UpG4t1 z3_WN+^G@Sa_7t8i0PS;c2%!YZQ&j?;Uwkv7(X>T?!-q@-2F8 z4*>P@Q_nQ`&aw_9j&yha?^hXo{R!Sg@OjMfkvWQ;g)4IFsb+O>!9yULUls@{`Yx)9 zPBQ0#QeiO1^p$cs-}4dCE>1GcFuXG_-ie$8>qD%*xidbRErHDs>7}fu3NM|!Q`JYf zdY6&k+VapS;kfGmBI2X3;v*;@38@3`rNEc(rOYuvkH-UlLI*tt*mbVx^p7CkFNnA! z5z4UMiW2Ra4G(=o8iKVZH*Nu5j=^vU8kmS?-Fbdjg~I{AF8v><0rxXM-Z}8B(*)4X z9Y~LoU>h8{0*=H4I2c-^Uf*}2$d^f8Yo{AM)S|?IAF&drHy5qFU-aL+S))g;9}OfA z=gIho94URsG2VPL`UYdobA3M(8n&|bKLXp20isTivtw;IRAlxEQ@c zp-?Dp#uGhyg+ifFWI_qOLZMK2I36SPIW;i%1cVTgLWsYF5Zi?i+k_BL+qM@LLi{0w z*kk*2;y4crIB z0^Mzm$vus9;5jK}EKmm+WV_Fyz#=JSIbf0PyC?EAup`s=4}cL;${64b&|ep;oX2Ot z0x4xhpc>k}d-SswLX-ra0urQ@RnSVKcf&KE^+cf6E!*z&eD6c+=TQ!=?dq7^T*xDY z9!La|rIh=ml-Gf2KnI;|ayU}Tqf*NCw$B&Pe(+18-RmUVut`8iT?}+0LWl^U4e*Wa zy8xO2Gi^gW>b`{#1%XCry&jNKUYAmy#S5M_TAz%3wr+KSz)?K#SxYIGqV?~{%$=TZ z2jDxYu?h$gLfGBN*i(c)s|M!I>9T7Ogb-0c4PZ4;R7!c&w(A&BR4==n$VlKtz-s#* z3Y?WvrrW-I+B1&^P688b;i_+h5c%=&XDy{X415e6L=*UPz&bqpS!d=>&o@sR=P5$( zRR|%1fLXv0DdqXh`)n;7Jt}8b2+nTF-HBHEc5WZ+4KSfG8+qU_5#%rAI?-P32=OpR_?S&Bg(ZX`g z(Bju2w%2>`yl0*Qq(~`O+rH046Mj#B*7pGaN-0lDDbs-2XwTo1pLJ&L^nCNAah@Xd z_B1eedu(y(@xTr#<)@i;Wp2*i2bkk(+nkA%@?|Mygp@K0?b$AtQuYM?LTg439(LbS z%45JGAw&h+rsin9mp%GfqlIkh2q6M(|KBt2+hd>gE!^q(&INkgvTy1D=Q4$Oaz;w& z5r2Tzz;0kKFidZIoOF*3&|c*>0~>9BTBF6Q7Xf>KKY_w}+v>!BgC@W)w$EjNH9#`( zBjB&MeGb0Y>Ohk1dH)O)@`h&(yb0_CwgQs@e@}eYz*Mw(h%~e}xc+x~zJNf>EZhrh zLCZ?A`&z05f_g0RsaA1 literal 0 HcmV?d00001 From fa6586173a114dbbc7f3fa6840124cd5af29facd Mon Sep 17 00:00:00 2001 From: ayasyrev Date: Mon, 31 Jan 2022 14:42:20 +0300 Subject: [PATCH 5/5] docs fix collapse flag --- docs/PolyLR.html | 12 ++++++++++-- docs/SGDR.html | 19 ++++++++++++++++++- nbs/07g_PolyLR.ipynb | 4 ++-- 3 files changed, 30 insertions(+), 5 deletions(-) diff --git a/docs/PolyLR.html b/docs/PolyLR.html index 080b15c..e1fc8a1 100644 --- a/docs/PolyLR.html +++ b/docs/PolyLR.html @@ -211,7 +211,10 @@

    Required arguments. -
    +
    + + +
    @@ -223,6 +226,7 @@

    Required arguments.
    @@ -268,7 +272,10 @@

    power

    {% raw %}
    -
    +
    + + +
    @@ -291,6 +298,7 @@

    power

    +
    diff --git a/docs/SGDR.html b/docs/SGDR.html index 44326c7..33fbaef 100644 --- a/docs/SGDR.html +++ b/docs/SGDR.html @@ -79,13 +79,30 @@
    -

    class CosineLRScheduler[source]

    CosineLRScheduler(optimizer:Optimizer, t_initial:int, lr_min:float=0.0, cycle_mul:float=1.0, cycle_decay:float=1.0, cycle_limit:int=1, warmup_t=0, warmup_lr_init=0, warmup_prefix=False, t_in_epochs=True, noise_range_t=None, noise_pct=0.67, noise_std=1.0, noise_seed=42, k_decay=1.0, initialize=True) :: Scheduler

    +

    class CosineLRScheduler[source]

    CosineLRScheduler(optimizer:Optimizer, t_initial:int, lr_min:float=0.0, cycle_mul:float=1.0, cycle_decay:float=1.0, cycle_limit:int=1, warmup_t:int=0, warmup_lr_init:float=0, warmup_prefix:bool=False, t_in_epochs:bool=True, noise_range_t:Union[int, float, List[Union[int, float]]]=None, noise_pct:float=0.67, noise_std:float=1.0, noise_seed:int=42, k_decay:float=1.0, initialize:bool=True) :: Scheduler

    Cosine decay with restarts. This is described in the paper https://arxiv.org/abs/1608.03983.

    Inspiration from https://github.com/allenai/allennlp/blob/master/allennlp/training/learning_rate_schedulers/cosine.py

    k-decay option based on k-decay: A New Method For Learning Rate Schedule - https://arxiv.org/abs/2004.05909

    +

    Args:
    + optimizer (torch.optim.Optimizer): torch optimizer to schedule
    + t_initial (int): Number of epochs it initial (first) cycle.
    + lr_min (float, optional): Minimum learning rate to use during the scheduling. Defaults to 0..
    + cycle_mul (float, optional): Multiplyer for cycle length. Defaults to 1..
    + cycle_decay (float, optional): Factor to decay lr at next cycle. Defaults to 1..
    + cycle_limit (int, optional): Number of cycles. Defaults to 1.
    + warmup_t (int, optional): Number of epochs to warmup. Defaults to 0.
    + warmup_lr_init (float, optional): Initial learning rate during warmup . Defaults to 0.
    + warmup_prefix (bool, optional): If True, after warmup annealing starts from initial LR. Defaults to False.
    + t_in_epochs (bool, optional): If set to False, returned lr are None. Defaults to True.
    + noise_range_t (Union[int, float, List[int, float]], optional): Epoch when noise starts. If list or tuple - epoch range, when noise applied. Defaults to None.
    + noise_pct (float, optional): Percentage of noise to add. Defaults to 0.67.
    + noise_std (float, optional): Noise standard deviation. Defaults to 1.0.
    + noise_seed (int, optional): Seed to use to add random noise. Defaults to 42.
    + k_decay (float, optional): Power for kdecay. Defaults to 1.0.
    + initialize (bool, optional): Add initial
    {field_name} to optimizer param group. Defaults to True.

    diff --git a/nbs/07g_PolyLR.ipynb b/nbs/07g_PolyLR.ipynb index 4f512d7..afea5c7 100644 --- a/nbs/07g_PolyLR.ipynb +++ b/nbs/07g_PolyLR.ipynb @@ -312,7 +312,7 @@ } ], "source": [ - "#collapse_input open\n", + "#collapse\n", "scheduler = PolyLRScheduler(optimizer, t_initial=50)\n", "plot_lr(scheduler)" ] @@ -362,7 +362,7 @@ } ], "source": [ - "#collapse_output open\n", + "#collapse\n", "scheduler = PolyLRScheduler(optimizer, t_initial=t_initial)\n", "plot_lr(scheduler, label='power=0.5, default')\n", "\n",