ENGR 421 • Final • Introduction to Machine Learning

Counting and Probability

Conditional Probability and Independence

Bayes' Rule

Discrete Random Variables

Continuous Random Variables

Expected Value and Variance

Bernoulli and Binomial Distributions

Continuous Uniform Distribution

Exponential Distribution

Normal Distribution

Laplace and Logistic Distributions

Dimensionality Reduction

Principal Component Analysis (PCA)

Feature Embedding & Factor Analysis (FA)

Singular Value Decomposition and Matrix Factorization

Multidimensional Scaling

Linear Discriminant Analysis (LDA)

Canonical Correlation Analysis

Isomap, Locally Linear Embedding, Laplacian Eigenmaps

Introduction and Mixture Densities

K-Means Clustering

Expectation-Maximization (EM)

Mixture Models & Practical Use of Clusters

Spectral and Hierarchical Clustering

Why Combine Multiple Learners?

Voting & Linear Combination

Bayesian Perspective & Effect of Dependence

Fixed Combination Rules & ECOC

Bagging & AdaBoost

Mixture of Experts and Stacking

Fine-Tuning an Ensemble

Cascading

Combining Multiple Sources/Views

Pass Rates & Majors (Bayes; Law of Total Probability)

Linear Discriminant with Equal Variance

MLP with Hard-Threshold Units

Naive Histogram Estimator vs. Parzen Windows (Kernel)

Comparing Two Splits (Gini vs. Misclassification)

Why Not Regression for Classification?

Adaptive Learning Rates in Gradient Descent

When Do Direct Input Output Links Help in an MLP?

Mahalanobis vs. Euclidean: Why and When?

Regularized Least Squares

Gaussian Generative Model → Logistic Posterior

Naive Bayes Text Classification with Binary Features

Derivative of Softmax

Kernel Density Estimation

Choosing Between Two Splits: Gini vs. Misclassification