CS 454 • Midterm • Introduction to Machine Learning and Artificial Neural Networks

Introduction to Machine Learning

Machine Learning Notation Explained

Machine Learning Preliminaries

Why Supervised?

Hypothesis Space & Occam's Razor

Loss Functions: Measuring Mistakes

Example: Least-Squares Linear Regression

Counting and Probability

Conditional Probability and Independence

Bayes' Rule

Discrete Random Variables

Continuous Random Variables

Expected Value and Variance

Bernoulli and Binomial Distributions

Continuous Uniform Distribution

Exponential Distribution

Normal Distribution

Laplace and Logistic Distributions

Maximum Likelihood Estimation(MLE)

Bernoulli Likelihood

Multinomial Likelihood and Smoothing

Bayes' Theorem

Parametric Classification

Unequal Variances → Quadratic Boundary

Gaussian Classification Boundary

Parametric & Polynomial Regression

Modeling Multivariate Data: Estimation, Normal Distributions, and Naive Bayes

Multivariate Classification: Linear, Quadratic, and Model Selection

Discrete Features & Multivariate Regression

Dimensionality Reduction

Principal Component Analysis (PCA)

Feature Embedding & Factor Analysis (FA)

Singular Value Decomposition and Matrix Factorization

Multidimensional Scaling

Linear Discriminant Analysis (LDA)

Canonical Correlation Analysis

Isomap, Locally Linear Embedding, Laplacian Eigenmaps

Introduction and Mixture Densities

K-Means Clustering

Expectation-Maximization (EM)

Mixture Models & Practical Use of Clusters

Spectral and Hierarchical Clustering

Pass Rates & Majors (Bayes; Law of Total Probability)

Weighted Least Squares (Closed-Form Solution, Matrix View & Interpretation)

MLE for α (positive support, exponential tail)

Naive Histogram Estimator vs. Parzen Windows (Kernel)

Kernel Smoother

Naive Density Estimator (Bandwidth effect & validity)

Comparing Two Splits (Gini vs. Misclassification)

Prepruning vs. Postpruning (Which and Why?)