# Deep Learning Dictionary

**Additive Smoothing** : When calculating the maximum likelihood estimate $\theta_j$, you want to make sure that even unlikely possibilities could be generated in the additive model.
Discussed in Generative Deep Learning - July 2019 - David Foster - Chapter 1

**Activations** : These are tne nonlinearities that rae introduced within Dense layers.

- Sigmoid - This is used for multiclass classification (when an item can belong to more than one class ). It is used when you want the values to be between 0 and 1. This is represented as $\frac{1}{1 + e^{-x}}$

Discussed in Generative Deep Learning - July 2019 - David Foster - Chapter 2

**Autoregressive Model** : This is a unidirectional model that attempts to predict data from past input.

**Catastrophic Forgetting** : A situation where the model forgets the task on which it was originally trained.
Discussed in NAACL 2019 Transfer Learning Tutorial Slides

**Naive Bayes** : This modeling technique makes the assumption that each feature is independent of every other feature.

**Sequential Adaptation** : Intermediate fine-tuning on related datasets and tasks.
Discussed in Generative Deep Learning - July 2019 - David Foster - Chapter 1