AIN 3001 • Midterm • Machine Learning

Introduction to Machine Learning

Machine Learning Notation Explained

Parameter vs Hyperparameter

Data Splitting

K-Fold Cross Validation

Train–Validation–Test Split

Generalization

Underfitting & Overfitting

Bias Variance Perspective

Evaluation

Closing Checklist

Bias–Variance & Model Complexity

Why Supervised?

Hypothesis Space & Occam's Razor

Loss Functions: Measuring Mistakes

Example: Least-Squares Linear Regression

Introduction and Mixture Densities

K-Means Clustering

K-Means Iteration Calculation

Expectation-Maximization (EM)

Mixture Models & Practical Use of Clusters

Spectral and Hierarchical Clustering

Dimensionality Reduction

Principal Component Analysis (PCA)

Covariance Matrix Calculation

Feature Embedding & Factor Analysis (FA)

Singular Value Decomposition and Matrix Factorization

Multidimensional Scaling

Linear Discriminant Analysis (LDA)

Canonical Correlation Analysis

Isomap, Locally Linear Embedding, Laplacian Eigenmaps

What & Why

Maximum Margin Classification

Maximizing the Margin

Lagrangian Formulation of the Hard-Margin SVM

From Primal to Dual: Solving the SVM Optimization

Why only a few points matter (KKT & sparsity)

From 𝛼 to parameters

Prediction uses only support vectors

Soft Margin SVM

Soft Margin Dual

Counting and Probability

Conditional Probability and Independence

Bayes' Rule

Discrete Random Variables

Continuous Random Variables

Expected Value and Variance

Bernoulli and Binomial Distributions

Continuous Uniform Distribution

Exponential Distribution

Normal Distribution

Laplace and Logistic Distributions

Why Not Regression for Classification?

From Binary to Multiclass: One-vs-All / One-vs-One with a Binary Classifier

Mahalanobis vs. Euclidean: Why and When?

Feature Mapping & Transformation

True/False on Scaling, k-NN, Intrinsic Error and Model Complexity