IE 313 • Operations Research III • Midterm

Random Variables and Probability Distributions

Conditional Probability

Total Probability Rule

Expected Value

What is a Markov Chain?

One-Step Transition Probabilities

Example

n-Step Transition Probabilities

Chapman-Kolmogorov Equations

Example

Unconditional State Probabilities

Example

Steady State Distribution

Example

Classes and State Properties

Periodicity and Ergodic Markov Chains

Example

Example

Example

First Passage Times

Example

Expected First Passage Time

Expected Recurrence Time

Example

What is an Absorbing MC?

Absorption Probabilities

Example

Probability of Ever Visiting a State

Example

Expected Number of State Visits

Example

Expected Time Until Absoption

Example

Counting Processes

Assumptions of Poisson Processes

Example 1

Memoryless Property

Example 2

Minimum of Exponential Random Variables

Example 3

Thinning

Example 4

Superposition

Example 5

Non-homogenous Poisson Processes

Example 6

Question 1 (Spring 2021)

Question 2 (Fall 2022)

Question 3 (Fall 2022)

Question 4 (Spring 2021)

Question 5 (Fall 2023)

Question 6 (Fall 2023)

Question 7 (Fall 2022)

Question 8 (Fall 2023)

Question 9 (Fall 2022)

Question 10 (Spring 2021)

Question 11 (Fall 2023)

Question 12 (Spring 23)

Question 13 (Spring 22)

Python Question 1 (Fall 2023)

Python Question 2 (Spring 2022)

Discrete Time Markov Chains 1

Discrete Time Markov Chains 2

Limiting Distribution and State Classifications 1

Limiting Distribution and State Classifications 2

Limiting Distribution and State Classifications 3

Long Term Properties of Markov Chains 1

Long Term Properties of Markov Chains 2

Poisson Processes 1

Poisson Processes 2

Poisson Processes 3

Poisson Processes 4