IE 325 • Midterm • Stochastic Models

Counting and Probability

Conditional Probability and Independence

Bayes' Rule

Discrete Random Variables

Continuous Random Variables

Expected Value and Variance

Bernoulli and Binomial Distributions

Continuous Uniform Distribution

Exponential Distribution

Problem 1

Problem 2

Problem 3

Problem 4

Problem 5

Problem 6

Problem 7

Problem 8

Problem 9

Problem 10

Problem 11

Problem 12

Problem 13

Problem 14

Problem 15

Problem 16

Problem 17

Problem 18

What is a Markov Chain?

One-Step Transition Probabilities

Example

n-Step Transition Probabilities

Chapman-Kolmogorov Equations

Example

Unconditional State Probabilities

Example

Steady State Distribution

Example

Classes and State Properties

Periodicity and Ergodic Markov Chains

Example

Example

Example

First Passage Times

Example

Expected First Passage Time

Expected Recurrence Time

Example

What is an Absorbing MC?

Absorption Probabilities

Example

Probability of Ever Visiting a State

Example

Expected Number of State Visits

Example

Expected Time Until Absoption

Example

Counting Processes

Assumptions of Poisson Processes

Example 1

Memoryless Property

Example 2

Minimum of Exponential Random Variables

Example 3

Thinning

Example 4

Superposition

Example 5

Non-homogenous Poisson Processes

Example 6

Probability Review 1

Probability Review 2

Probability Review 3

Probability Review 4

Probability Review 5

Markov Chains 1

Markov Chains 2

Markov Chains 3

Markov Chains 4

Markov Chains 5

Markov Chains 6

Markov Chains 7 (NEW)

Markov Chains 8 (NEW)

Markov Chains 9 (NEW)

Markov Chains 10 (NEW)

Limiting Probabilities and State Classifications 1

Limiting Probabilities and State Classifications 2

Limiting Probabilities and State Classifications 3

Limiting Probabilities and State Classifications 4

Limiting Probabilities and State Classifications 5

Limiting Probabilities and State Classifications 6

Limiting Probabilities and State Classifications 7

Limiting Probabilities and State Classifications 8

Limiting Probabilities and State Classifications 9

Limiting Probabilities and State Classifications 10

First-step Analysis 1

First-step Analysis 2

First-step Analysis 3

First-step Analysis 4

First-step Analysis 5 (NEW)

First-step Analysis 6 (NEW)

First-step Analysis 7 (NEW)

Absorbing Markov Chains 1 (NEW)

Absorbing Markov Chains 2 (NEW)

Poisson Processes 1

Poisson Processes 2

Poisson Processes 3

Poisson Processes 4

Poisson Processes 5

Poisson Processes 6

Poisson Processes 7