CS 464 • Tüm Sınavlar • Introduction to Machine Learning

Introduction to Machine Learning

Machine Learning Notation Explained

Machine Learning Preliminaries

Counting and Probability

Conditional Probability and Independence

Bayes' Rule

Discrete Random Variables

Continuous Random Variables

Expected Value and Variance

Bernoulli and Binomial Distributions

Continuous Uniform Distribution

Exponential Distribution

Normal Distribution

Laplace and Logistic Distributions

Maximum Likelihood Estimation(MLE)

Bernoulli Likelihood

Multinomial Likelihood and Smoothing

Bayes' Theorem

Parametric Classification

Unequal Variances → Quadratic Boundary

Gaussian Classification Boundary

Modeling Multivariate Data: Estimation, Normal Distributions, and Naive Bayes

Multivariate Classification: Linear, Quadratic, and Model Selection

Discrete Features & Multivariate Regression

Sample Mean and Covariance Matrix

Mahalanobis Distance

LDA vs QDA Classification

Naive Bayes Classification (Discrete Features)

Feature Selection vs Feature Extraction

Principal Component Analysis (PCA)

Feature Embedding & Factor Analysis (FA)

Evaluation/Performance Metrics

Why Supervised?

Hypothesis Space & Occam's Razor

Loss Functions: Measuring Mistakes

Linear & Polynomial Regression

Example: Least-Squares Linear Regression

The Problem in Linear Regression

Linear Discriminant

Two Classes/Multiple Classes/Pairwise Seperation

From Discriminants to Posteriors

Gradient Descent

Motivation

Probabilistic Interpretation

Binary Cross Entropy / Log-loss

Optimization with Gradient Descent

Classification with Logistic Regression

Summary & Multi-Class Logistic Regression

Pass Rates & Majors (Bayes; Law of Total Probability)

Weighted Least Squares (Closed-Form Solution, Matrix View & Interpretation)

MLE for α (positive support, exponential tail)

Naive Histogram Estimator vs. Parzen Windows (Kernel)

Naive Density Estimator (Bandwidth effect & validity)

Why Not Regression for Classification?

From Binary to Multiclass: One-vs-All / One-vs-One with a Binary Classifier

Max-shift for SoftMax

Mahalanobis vs. Euclidean: Why and When?

Regularized Least Squares

Gaussian Generative Model → Logistic Posterior

Naive Bayes Text Classification with Binary Features

Derivative of Softmax

Kernel Density Estimation

MLE for a Discrete PMF

1-NN LOOCV on Patient Dataset

Linear Regression + MSE: Gradient Descent Step Size Effects

Motivation

Probabilistic Interpretation

Parametric & Polynomial Regression

Binary Cross Entropy / Log-loss

Optimization with Gradient Descent

Classification with Logistic Regression

Summary & Multi-Class Logistic Regression

Why Not Regression for Classification?

What & Why

Maximum Margin Classification

Maximizing the Margin

Lagrangian Formulation of the Hard-Margin SVM

From Primal to Dual: Solving the SVM Optimization

Why only a few points matter (KKT & sparsity)

From 𝛼 to parameters

Prediction uses only support vectors

Soft Margin SVM

Soft Margin Dual

Margin, distance, and support vectors

Solving a tiny SVM dual problem (linear kernel)

Polynomial kernel and feature map

Perceptron

Training a Perceptron

Limitation: XOR

MLP Architecture & Representation View

Backpropagation

Regression

Discrimination

MLP with Hard-Threshold Units

Should we initialize all MLP weights to zero?

Introduction to Deep Learning & Activation Functions

Training Deep Networks

Regularization Techniques

Tuning Network Structure

Learning Time

Time-Delay Neural Networks (TDNN)

RNN / LSTM / GRU

Generative Adversarial Networks (GANs)

Convolution vs Fully Connected

Forward pass in a small neural network

Softmax and cross-entropy

Vanishing gradients (True/False with explanation)

What is a Decision Tree?

Splitting in Classification Trees

Pruning Trees

From Trees to Rules

Multivariate/Oblique Trees

Choosing Between Two Splits: Gini vs. Misclassification

Pass Rates & Majors (Bayes; Law of Total Probability)

Weighted Least Squares (Closed-Form Solution, Matrix View & Interpretation)

MLE for α (positive support, exponential tail)

Linear Discriminant with Equal Variance

One Shared Network vs. Three Separate Networks

Naive Histogram Estimator vs. Parzen Windows (Kernel)

Kernel Smoother

Naive Density Estimator (Bandwidth effect & validity)

Comparing Two Splits (Gini vs. Misclassification)

Prepruning vs. Postpruning (Which and Why?)

From Binary to Multiclass: One-vs-All / One-vs-One with a Binary Classifier

Max-shift for SoftMax

Why Initialize Weights Near Zero?

Adaptive Learning Rates in Gradient Descent

When Do Direct Input Output Links Help in an MLP?

Mahalanobis vs. Euclidean: Why and When?

Discrete Attribute in Decision Trees

Regularized Least Squares

Gaussian Generative Model → Logistic Posterior

Naive Bayes Text Classification with Binary Features

Derivative of Softmax

Kernel Density Estimation