IE 332 • Final • Mathematical Modeling and Optimization III

Continuous Time Markov Chains

Example 1

Example 2

Steady State Distribution

Birth and Death Processes

Random Rates, Finite Chain

Example 3

Random Rates, Infinite Chain

Example 4

Equal Rates, Infinite Chain, Single Server

Example 5

Equal Rates, Finite Chain, Single Server

Example 6

Equal Rates, Finite Chain, Multiple Servers

Example 7

Kendall's Notation

Terminology

M/M/1 Queue

Example 1

M/M/s Queue

Example 2

M/M/1/K Queue

Example 3

M/M/s/K Queue

Example 4

A New Approach to Optimization Problems

Inventory Problems

Example 1

Resource Allocation Problems

Example 2

What is it?

Example 1

Example 2

Example 3

Introduction

Policy Improvement Algorithm: Step 1

Policy Improvement Algorithm: Step 2

Example 1

LP Solution

LP Solution with Discount

Birth and Death 1

Birth and Death 2

Birth and Death 3

Birth and Death 4

Queueing Theory 1

Queueing Theory 2

Queueing Theory 3

Queueing Theory 4

Queueing Theory 5

Queueing Theory 6

Queueing Theory 7

Queueing Theory 8

Probabilistic Dynamic Programming 1

Probabilistic Dynamic Programming 2

Probabilistic Dynamic Programming 3

Probabilistic Dynamic Programming 4

Probabilistic Dynamic Programming 5

Probabilistic Dynamic Programming 6

Markov Decision Process 1