CS 454 • Final • Introduction to Machine Learning and Artificial Neural Networks

Counting and Probability

Conditional Probability and Independence

Bayes' Rule

Discrete Random Variables

Continuous Random Variables

Expected Value and Variance

Bernoulli and Binomial Distributions

Continuous Uniform Distribution

Exponential Distribution

Normal Distribution

Laplace and Logistic Distributions

What Nonparametric Means

Density Estimation

Nonparametric Classification

Condensed Nearest Neighbor

Outlier Detection

Nonparametric Regression

Additive Models & How to chose h/k

What is a Decision Tree?

Splitting in Classification Trees

Pruning Trees

From Trees to Rules

Multivariate/Oblique Trees

The Problem

Linear Discriminant

Two Classes/Multiple Classes/Pairwise Seperation

From Discriminants to Posteriors

Gradient Descent

Logistic Discrimination: Two Classes

Logistic Discrimination: K>2 Classes

Generalizing the Linear Model

Discrimination by Regression

Learning to Rank

Perceptron

Training a Perceptron

Limitation: XOR

MLP Architecture & Representation View

Backpropagation

Regression

Discrimination

Introduction to Deep Learning & Activation Functions

Training Deep Networks

Regularization Techniques

Tuning Network Structure

Learning Time

Time-Delay Neural Networks (TDNN)

RNN / LSTM / GRU

Generative Adversarial Networks (GANs)

Why Combine Multiple Learners?

Voting & Linear Combination

Bayesian Perspective & Effect of Dependence

Fixed Combination Rules & ECOC

Bagging & AdaBoost

Mixture of Experts and Stacking

Fine-Tuning an Ensemble

Cascading

Combining Multiple Sources/Views

Weighted Least Squares (Closed-Form Solution, Matrix View & Interpretation)

Linear Discriminant with Equal Variance

MLP with Hard-Threshold Units

Should we initialize all MLP weights to zero?

One Shared Network vs. Three Separate Networks

Naive Histogram Estimator vs. Parzen Windows (Kernel)