IE 336 • Midterm • Stochastic Modeling

Random Variables and Probability Distributions

Conditional Probability

Total Probability Rule

Expected Value

What is a Markov Chain?

One-Step Transition Probabilities

Example 1

n-Step Transition Probabilities

Chapman-Kolmogorov Equations

Example 2

Unconditional State Probabilities

Example 3

Steady State Distribution

Example 1

Classes and State Properties

Periodicity and Ergodic Markov Chains

Example 2

Example 3

Example 4

First Passage Times

Example 1

Expected First Passage Time

Expected Recurrence Time

Example 2

Problem 1

Problem 2

Problem 3

Problem 4

Problem 5

Problem 6 (Very Hard)

Problem 7 (Very Hard)

Problem 8 (Very Hard)

Problem 1

Problem 2

Problem 3

Problem 4

Problem 5

Problem 6

Problem 7

Problem 8

Problem 9