IE 3554 • Midterm • Stochastic Operations Research

Random Variables and Probability Distributions

Conditional Probability

Total Probability Rule

Expected Value

What is a Markov Chain?

One-Step Transition Probabilities

Example

n-Step Transition Probabilities

Chapman-Kolmogorov Equations

Example

Unconditional State Probabilities

Example

Steady State Distribution

Example

Classes and State Properties

Periodicity and Ergodic Markov Chains

Example

Example

Example

First Passage Times

Example

Expected First Passage Time

Expected Recurrence Time

Example

What is an Absorbing MC?

Absorption Probabilities

Example

Probability of Ever Visiting a State

Example

Expected Number of State Visits

Example

Expected Time Until Absoption

Example

Probability Review

Probability Review

Probability Review

Discrete Time Markov Chains

Discrete Time Markov Chains

Discrete Time Markov Chains

Limiting Distribution and State Classifications

Limiting Distribution and State Classifications

Limiting Distribution and State Classifications

First Step Analysis

First Step Analysis

Probability Review

Probability Review

Probability Review

Discrete Time Markov Chains

Discrete Time Markov Chains

Limiting Distribution and State Classifications

Limiting Distribution and State Classifications

Limiting Distribution and State Classifications

First Step Analysis

Absorbing Chains

Probability Review

Discrete Time Markov Chains

Limiting Distribution and State Classifications

Limiting Distribution and State Classifications?

First Step Analysis

Absorbing Chains

Discrete Time Markov Chains

Discrete Time Markov Chains

Limiting Distribution and State Classifications

Limiting Distribution and State Classifications

First Step Analysis

First Step Analysis

Probability Review

Discrete Time Markov Chains

Limiting Distribution and State Classifications

Limiting Distributions and State Classifications

First Step Analysis

True/False