CS 412 • Tüm Sınavlar • Machine Learning

Introduction to Machine Learning

Machine Learning Notation Explained

Machine Learning Preliminaries

Why Supervised?

Hypothesis Space & Occam's Razor

Loss Functions: Measuring Mistakes

Example: Least-Squares Linear Regression

Nearest Neighbor Approach

Value of k

Geometric View: Voronoi Intuition

Distance Measure

Distances for Real Vectors

Example: Computing Distance Between Two Points

Distance for Non-Numeric Data

Scaling and Normalization

Voting Mechanism

k-NN Regression

What is a Decision Tree?

Splitting in Classification Trees

Pruning Trees

From Trees to Rules

Multivariate/Oblique Trees

What Is Regression?

Linear Regression

Multiple Linear Regression

Polynomial Regression

Summary: Linear, Multiple & Polynomial Regression

Feature Transformations & Feature Engineering

Feature Selection vs Feature Extraction

Feature Embedding & Factor Analysis (FA)

Motivation

Probabilistic Interpretation

Binary Cross Entropy / Log-loss

Optimization with Gradient Descent

Classification with Logistic Regression

Summary & Multi-Class Logistic Regression

Perceptron

Training a Perceptron

Limitation: XOR

MLP Architecture & Representation View

Backpropagation

Regression

Discrimination

Introduction to Deep Learning & Activation Functions

Training Deep Networks

Regularization Techniques

Tuning Network Structure

Learning Time

Time-Delay Neural Networks (TDNN)

RNN / LSTM / GRU

Generative Adversarial Networks (GANs)

Counting and Probability

Conditional Probability and Independence

Bayes' Rule

Discrete Random Variables

Continuous Random Variables

Expected Value and Variance

Bernoulli and Binomial Distributions

Continuous Uniform Distribution

Exponential Distribution

Normal Distribution

Laplace and Logistic Distributions

Maximum Likelihood Estimation(MLE)

Bernoulli Likelihood

Multinomial Likelihood and Smoothing

Bayes' Theorem

Parametric Classification

Unequal Variances → Quadratic Boundary

Gaussian Classification Boundary

Parametric & Polynomial Regression

Naive Bayes Approach

Curse of Dimensionality

Bayes Classifier vs. Naive Bayes

Independence & Conditional Independence

Naive Bayes Classification

How Naive Bayes Simplifies Parameter Estimation

Model Selection Using Validation Performance and Test MSE

k-NN Decision Boundaries and the Effect of k

Pass Rates & Majors (Bayes; Law of Total Probability)

Weighted Least Squares (Closed-Form Solution, Matrix View & Interpretation)

MLE for α (positive support, exponential tail)

MLP with Hard-Threshold Units

Should we initialize all MLP weights to zero?

One Shared Network vs. Three Separate Networks

Naive Histogram Estimator vs. Parzen Windows (Kernel)

Comparing Two Splits (Gini vs. Misclassification)

Prepruning vs. Postpruning (Which and Why?)

Why Not Regression for Classification?

From Binary to Multiclass: One-vs-All / One-vs-One with a Binary Classifier

Max-shift for SoftMax

Why Initialize Weights Near Zero?

Adaptive Learning Rates in Gradient Descent

When Do Direct Input Output Links Help in an MLP?

Mahalanobis vs. Euclidean: Why and When?

Discrete Attribute in Decision Trees

Regularized Least Squares

Gaussian Generative Model → Logistic Posterior

Naive Bayes Text Classification with Binary Features

Derivative of Softmax

Choosing Between Two Splits: Gini vs. Misclassification

Mean Square Error for Linear Regression

Gradient Descent Update

k-NN Regression Prediction

Decision Boundary and Building a Network for Binary Classification

Derivative of Squared Error

Computing Input and Output of a Convolution Node

True/False Reasoning on Activation, Linear Networks, and Gradient Descent

Computing Total Probability

True/False on Scaling, k-NN, Intrinsic Error and Model Complexity

Output Size of a Conv Layer

Naive Bayes Approach

Curse of Dimensionality

Bayes Classifier vs. Naive Bayes

Independence & Conditional Independence

Naive Bayes Classification

How Naive Bayes Simplifies Parameter Estimation

Example with categorical variables

Naive Bayes Subtleties

Modeling Multivariate Data: Estimation, Normal Distributions, and Naive Bayes

Discrete Features & Multivariate Regression

Sample Mean and Covariance Matrix

Mahalanobis Distance

Naive Bayes Classification (Discrete Features)

Maximum Likelihood Estimation(MLE)

Bernoulli Likelihood

Multinomial Likelihood and Smoothing

Parametric Classification

Unequal Variances → Quadratic Boundary

Gaussian Classification Boundary

Multivariate Classification: Linear, Quadratic, and Model Selection

LDA vs QDA Classification

Text Classification with Naïve Bayes

Text Classification Tasks

Bag-of-Words (BoW)

Naive Bayes scoring with BoW

Bernoulli Naive Bayes

Multinomial Naive Bayes

Multinomial Naive Bayes - 2

Laplace smoothing

Practical Notes

Why raw term counts aren’t enough

Transforming TF

IDF

TF-IDF

Text Preprocessing

Tokenization and Token normalization

Stop words + Stemming vs Lemmatization

Why text is harder than “normal” ML inputs

Bag of Words / TF-IDF

One-Hot Encoding & Distributional Hypothesis

Word Embeddings

Word2Vec idea & CBOW vs Skip-gram

Training objective & Negative Sampling

Scoring function

What embeddings capture

Limitations of Word2Vec embeddings

Language Models

Word2Vec Training Data

Semi/Self-Supervised Learning

Pretext & Pseudo-labeling

Consistency Regularization

Representation Learning & Contrastive Learning

Semi-Supervised Learning & Entropy Minimization

Early Work in SSL

Single-Neuron Sigmoid + MSE

Decision Trees: Gini Impurity Split Comparison

Decision Trees: Entropy & Information Gain Split Comparison

MLE for a Discrete PMF

1-NN LOOCV on Patient Dataset

Linear Regression + MSE: Gradient Descent Step Size Effects

Mean Square Error for Linear Regression

Gradient Descent Update

k-NN Regression Prediction

Decision Boundary and Building a Network for Binary Classification

Derivative of Squared Error

Computing Input and Output of a Convolution Node

True/False Reasoning on Activation, Linear Networks, and Gradient Descent

Computing Total Probability

True/False on Scaling, k-NN, Intrinsic Error and Model Complexity

Output Size of a Conv Layer

Regression: Test-Set MSE

Generalization & Overfitting: True/False

Baseline Error: ZeroR vs Random Guessing

Entropy: Fair Die & Bias Effect

Decision Trees: ID3 Optimality + Key Advantage

Decision Tree Split: Remaining Entropy