CMPE 468 • Midterm • Machine Learning for Engineers

Introduction to Machine Learning

Machine Learning Notation Explained

Counting and Probability

Conditional Probability and Independence

Bayes' Rule

Discrete Random Variables

Continuous Random Variables

Expected Value and Variance

Bernoulli and Binomial Distributions

Continuous Uniform Distribution

Exponential Distribution

Normal Distribution

Laplace and Logistic Distributions

Why Supervised?

Hypothesis Space & Occam's Razor

Example: Least-Squares Linear Regression

What Is Regression?

Linear Regression

Multiple Linear Regression

Polynomial Regression

Summary: Linear, Multiple & Polynomial Regression

Feature Transformations & Feature Engineering

Nearest Neighbor Approach

Value of k

Geometric View: Voronoi Intuition

Distance Measure

Distances for Real Vectors

Example: Computing Distance Between Two Points

Distance for Non-Numeric Data

Scaling and Normalization

Voting Mechanism

k-NN Regression

The Problem

Linear Discriminant

Two Classes/Multiple Classes/Pairwise Seperation

From Discriminants to Posteriors

Gradient Descent

Logistic Discrimination: Two Classes

Logistic Discrimination: K>2 Classes

Generalizing the Linear Model

Discrimination by Regression

Learning to Rank

Naive Bayes Approach

Curse of Dimensionality

Bayes Classifier vs. Naive Bayes

Independence & Conditional Independence

Naive Bayes Classification

How Naive Bayes Simplifies Parameter Estimation

Example with categorical variables

Naive Bayes Subtleties

What is a Decision Tree?

Splitting in Classification Trees

Pruning Trees

From Trees to Rules

Multivariate/Oblique Trees

Weighted Least Squares (Closed-Form Solution, Matrix View & Interpretation)

Comparing Two Splits (Gini vs. Misclassification)

Prepruning vs. Postpruning (Which and Why?)

Pass Rates & Majors (Bayes; Law of Total Probability)

Why Not Regression for Classification?

From Binary to Multiclass: One-vs-All / One-vs-One with a Binary Classifier

Mahalanobis vs. Euclidean: Why and When?

Discrete Attribute in Decision Trees

Regularized Least Squares

Gaussian Generative Model → Logistic Posterior

Choosing Between Two Splits: Gini vs. Misclassification

Naive Bayes Text Classification with Binary Features

Mean Square Error for Linear Regression

k-NN Regression Prediction

Decision Boundary and Building a Network for Binary Classification

Derivative of Squared Error

True/False Reasoning on Activation, Linear Networks, and Gradient Descent

Computing Total Probability

True/False on Scaling, k-NN, Intrinsic Error and Model Complexity