# E2 204, Spring 2024

## Stochastic Processes & Queueuing Theory

### Course Syllabus

Poisson process, Renewal theory, Markov chains, Reversibility, Queueing networks, Martingales, Random walk.

### Course Description

Basic mathematical modeling is at the heart of engineering. In both electrical and computer engineering, many complex systems are modeled using stochastic processes. This course will introduce students to basic stochastic processes tools that can be utilized for performance analysis and stochastic modeling of dynamic systems and networks.

### Teams/GitHub Information

#### Teams

We will use Microsoft Teams for all the course related communication.

**Please do not send any email regarding the course.**

You can signup for the course team Stochastic-Processes-2024 using the following code **jeib7m2**.

### Instructor

Parimal Parag

**Office:** EC 2.17

**Hours:** By appointment.

### Time and Location

**Classroom:** EC 1.07, ECE main building

**Hours:** Tue/Thu 10:00am-11:30am.

### Teaching Assistants

Moonmoon Mohanty

**Email:** moonmoonm@iisc.ac.in

**Hours:** By appointment.

### Lectures

- 02 Jan 2024: Lecture-01 Introduction
- 04 Jan 2024: Lecture-02 Probability Review
- 09 Jan 2024: Lecture-03 Conditional Expectation
- 11 Jan 2024: Lecture-04 Progressive Measurability
- 16 Jan 2024: Lecture-05 Stopped Event Spaces
- 18 Jan 2024: Lecture-06 Strong Markov Property
- 23 Jan 2024: Lecture-07 Renewal Processes
- 25 Jan 2024: Lecture-08 Renewal Processes: Distribution Functions
- 30 Jan 2024: Lecture-09 Renewal Processes: Limit Theorems
- 01 Feb 2024: Lecture-10 Regenerative Processes
- 06 Feb 2024: Lecture-11 Blackwell’s Theorem
- 08 Feb 2024: Lecture-12 Key Renewal Theorem
- 13 Feb 2024: Lecture-13 Key Renewal Theorem: Applications
- 15 Feb 2024: Lecture-14 Discrete Time Markov Chains
- 20 Feb 2024: Lecture-15 DTMC: Class Properties
- 22 Feb 2024: Lecture-16 DTMC: Invariant Distribution
- 27 Feb 2024: Lecture-17 Continuous Time Markov Chains
- 29 Feb 2024: Lecture-18 CTMC: Embedded Markov Chain and Jump Times
- 05 Mar 2024: Lecture-19 CTMC: Uniformization
- 07 Mar 2024: Lecture-20 CTMC: Invariant Distribution
- 12 Mar 2024: Lecture-21 Reversibility
- 14 Mar 2024: Lecture-22 Queues
- 19 Mar 2024: Lecture-23 Reversed Processes
- 21 Mar 2024: Lecture-24 Stochastic Networks
- 26 Mar 2024: Lecture-25 Martingales
- 28 Mar 2024: Lecture-26 Martingale Convergence Theorem
- 02 Apr 2024: Lecture-27 Martingale Concentration Inequalities
- 04 Apr 2024: Lecture-28 Exchangeability
- 09 Apr 2024: Lecture-29 Random Walks
- 11 Apr 2024: Lecture-30 GI/G/1 Queues

### Homework

- 04 Jan 2024: Homework-01
- 18 Jan 2024: Homework-02
- 01 Feb 2024: Homework-03
- 15 Feb 2024: Homework-04
- 29 Feb 2024: Homework-05
- 14 Mar 2024: Homework-06
- 28 Mar 2024: Homework-07
- 11 Apr 2024: Homework-08

### Tests

**Quiz Hours:** 10:00-10:30am

**Test Hours:** 10:00am-01:00pm

**Venue:** EC 1.07, ECE main building

- 13 Jan 2024: Quiz 1
- 27 Jan 2024: Quiz 2
- 10 Feb 2024: Mid Term 1
- 24 Feb 2024: Quiz 3
- 09 Mar 2024: Quiz 4
- 23 Mar 2024: Mid Term 2
- 06 Apr 2024: Quiz 5
- 20 Apr 2024: Quiz 6
- 26 Apr 2024: Final (09:00am-01:00pm)

### Grading Policy

**Quizzes:** 30

**Mid Terms:** 40

**Final:** 30

### Textbooks

Stochastic Processes, P. Parag and Vinod Sharma.

Stochastic Processes, Sheldon M. Ross, 2nd edition, 1996. You can get a copy of the textbook from the campus book store.

Introduction to Stochastic Processes, Erhan Cinlar, 2013.

Markov Chains: Gibbs Fields, Monte Carlo Simulation, and Queues, Pierre Bremaud, 1999.

Markov Chains, James R. Norris, 1998.

Reversibility and Stochastic Networks, Frank P. Kelly, 2011.

Probability: Theory and Examples, Rick Durett, 4th edition, 2010.