CMPE 468 • Final • Machine Learning for Engineers

Counting and Probability

Conditional Probability and Independence

Bayes' Rule

Discrete Random Variables

Continuous Random Variables

Expected Value and Variance

Bernoulli and Binomial Distributions

Continuous Uniform Distribution

Exponential Distribution

Normal Distribution

Laplace and Logistic Distributions

What is a Decision Tree?

Splitting in Classification Trees

Pruning Trees

From Trees to Rules

Multivariate/Oblique Trees

What & Why

Maximum Margin Classification

Maximizing the Margin

Lagrangian Formulation of the Hard-Margin SVM

From Primal to Dual: Solving the SVM Optimization

Why only a few points matter (KKT & sparsity)

From 𝛼 to parameters

Prediction uses only support vectors

Soft Margin SVM

Soft Margin Dual

Introduction and Mixture Densities

K-Means Clustering

One Iteration of k-Means

Expectation-Maximization (EM)

Mixture Models & Practical Use of Clusters

Spectral and Hierarchical Clustering

Choose the Right Clustering Tool

“Clustering as Preprocessing” Pitfall

Cross-validation, Generalization, Bias-Variance Trade-off

Evaluation/Performance Metrics

Loss Functions: Measuring Mistakes

Feature Selection vs Feature Extraction

Principal Component Analysis (PCA)

Feature Embedding & Factor Analysis (FA)

Pass Rates & Majors (Bayes; Law of Total Probability)

Linear Discriminant with Equal Variance

Comparing Two Splits (Gini vs. Misclassification)

Prepruning vs. Postpruning (Which and Why?)

Weighted Least Squares (Closed-Form Solution, Matrix View & Interpretation)

Mean Square Error for Linear Regression

Gradient Descent Update

k-NN Regression Prediction

Decision Boundary and Building a Network for Binary Classification

Derivative of Squared Error

Computing Input and Output of a Convolution Node

True/False Reasoning on Activation, Linear Networks, and Gradient Descent

Computing Total Probability

True/False on Scaling, k-NN, Intrinsic Error and Model Complexity

Regression: Test-Set MSE

Generalization & Overfitting: True/False

Baseline Error: ZeroR vs Random Guessing

Entropy: Fair Die & Bias Effect

Decision Trees: ID3 Optimality + Key Advantage

Decision Tree Split: Remaining Entropy

Discrete Attribute in Decision Trees

From Binary to Multiclass: One-vs-All / One-vs-One with a Binary Classifier

Why Not Regression for Classification?

Adaptive Learning Rates in Gradient Descent

Mahalanobis vs. Euclidean: Why and When?

Regularized Least Squares

Gaussian Generative Model → Logistic Posterior

Why Initialize Weights Near Zero?

Naive Bayes Text Classification with Binary Features

Choosing Between Two Splits: Gini vs. Misclassification