IE 334 • Midterm • Operations Research III

Random Variables and Probability Distributions

Conditional Probability

Total Probability Rule

Expected Value

Introduction

A Decision Tree Example

Interpretation

EV or EMV

Example 1

Introduction

Calculation of EVPI

Some notes about EVPI

Example EVPI

EVII or EVSI

Example EVII

What is a Markov Chain?

One-Step Transition Probabilities

Example

n-Step Transition Probabilities

Chapman-Kolmogorov Equations

Example

Unconditional State Probabilities

Example

Steady State Distribution

Example

Classes and State Properties

Periodicity and Ergodic Markov Chains

Example

Example

Example

First Passage Times

Example

Expected First Passage Time

Expected Recurrence Time

Example

What is an Absorbing MC?

Absorption Probabilities

Example

Probability of Ever Visiting a State

Example

Expected Number of State Visits

Example

Expected Time Until Absoption

Example

Decision Making Under Uncertainty 1

Decision Making Under Uncertainty 2

Decision Making Under Uncertainty 3

Markov Chains 1

Markov Chains 2

Markov Chains 3

Markov Chains 4

Markov Chains 5

Markov Chains 6

Markov Chains 7

Markov Chains 8

Markov Chains 9

Markov Chains 10

Markov Chains 11

Markov Chains 12

Markov Chains 13

Markov Chains 14

Markov Chains 15

Markov Chains 16

Markov Chains 17

Markov Chains 18

Markov Chains 19

Markov Chains 20