IE 301 • Midterm • Operations Research II

Random Variables and Probability Distributions

Conditional Probability

Total Probability Rule

Expected Value

What is a Markov Chain?

One-Step Transition Probabilities

Example

n-Step Transition Probabilities

Chapman-Kolmogorov Equations

Example

Unconditional State Probabilities

Example

Steady State Distribution

Example

Classes and State Properties

Periodicity and Ergodic Markov Chains

Example

Example

Example

First Passage Times

Example

Expected First Passage Time

Expected Recurrence Time

Example

Absorbing Markov Chains

Matrix Structure

Inverse of a Matrix

Expected Number of State Visits

Example

Expected Time Until Absoption

Example

Absorption Probabilities

Example

Counting Processes

Assumptions of Poisson Processes

Example 1

Memoryless Property

Example 2

Minimum of Exponential Random Variables

Example 3

Thinning

Example 4

Superposition

Example 5

Non-homogenous Poisson Processes

Example 6

Markov Chains 1

Markov Chains 2

Markov Chains 3

Markov Chains 4

Markov Chains 5

Markov Chains 6

Steady State Probabilities and Applications 1

Steady State Probabilities and Applications 2

Steady State Probabilities and Applications 3

Steady State Probabilities and Applications 4

Steady State Probabilities and Applications 5

Steady State Probabilities and Applications 6

Steady State Probabilities and Applications 7

Steady State Probabilities and Applications 8

Steady State Probabilities and Applications 9

Steady State Probabilities and Applications 10

Steady State Probabilities and Applications 11

Absorbing Markov Chains 1

Absorbing Markov Chains 2

Exponential Distribution and Poisson Process 1

Exponential Distribution and Poisson Process 2

Exponential Distribution and Poisson Process 3

Exponential Distribution and Poisson Process 5