# E2 204, Spring 2023

## Stochastic Processes & Queueuing Theory

### Lectures

- 03 Jan 2023: Lecture-01 Introduction
- 05 Jan 2023: Lecture-02 Probability Review
- 10 Jan 2023: Lecture-03 Conditional Expectation
- 12 Jan 2023: Lecture-04 Progressive Measurability
- 17 Jan 2023: Lecture-05 Stopped Event Spaces
- 19 Jan 2023: Lecture-06 Strong Markov Property
- 24 Jan 2023: Lecture-07 Renewal Processes
- 26 Jan 2023: Lecture-08 Renewal Processes: Distribution Functions
- 31 Jan 2023: Lecture-09 Renewal Processes: Limit Theorems
- 02 Feb 2023: Lecture-10 Regenerative Processes
- 07 Feb 2023: Lecture-11 Blackwell’s Theorem
- 09 Feb 2023: Lecture-12 Key Renewal Theorem
- 14 Feb 2023: Lecture-13 Key Renewal Theorem: Applications
- 16 Feb 2023: Lecture-14 Discrete Time Markov Chains
- 21 Feb 2023: Lecture-15 DTMC: Class Properties
- 23 Feb 2023: Lecture-16 DTMC: Invariant Distribution
- 28 Feb 2023: Lecture-17 Continuous Time Markov Chains
- 02 Mar 2023: Lecture-18 CTMC: Embedded Markov Chain and Jump Times
- 07 Mar 2023: Lecture-19 CTMC: Uniformization
- 09 Mar 2023: Lecture-20 CTMC: Invariant Distribution
- 14 Mar 2023: Lecture-21 Reversibility
- 16 Mar 2023: Lecture-22 Queues
- 21 Mar 2023: Lecture-23 Reversed Processes
- 23 Mar 2023: Lecture-24 Stochastic Networks
- 28 Mar 2023: Lecture-25 Martingales
- 30 Mar 2023: Lecture-26 Martingale Convergence Theorem
- 04 Apr 2023: Lecture-27 Martingale Concentration Inequalities
- 06 Apr 2023: Lecture-28 Exchangeability
- 11 Apr 2023: Lecture-29 Random Walks
- 13 Apr 2023: Lecture-30 GI/G/1 Queues

### Homework

- 05 Jan 2023: Homework-01
- 19 Jan 2023: Homework-02
- 02 Feb 2023: Homework-03
- 16 Feb 2023: Homework-04
- 02 Mar 2023: Homework-05
- 16 Mar 2023: Homework-06
- 30 Mar 2023: Homework-07
- 13 Apr 2023: Homework-08

### Tests

**Quiz Hours:** 10:00-10:30am

**Test Hours:** 10:00am-01:00pm

**Venue:** EC 1.07

- 21 Jan 2023: Quiz 1
- 04 Feb 2023: Quiz 2
- 18 Feb 2023: Mid Term 1
- 04 Mar 2023: Quiz 3
- 18 Mar 2023: Quiz 4
- 01 Apr 2023: Mid Term 2
- 15 Apr 2023: Quiz 5
- 22 Apr 2023: Quiz 6
- 26 Apr 2023: Final (09:00am-01:00pm)

### Grading Policy

**Quizzes:** 30

**Mid Terms:** 40

**Final:** 30

### Course Syllabus

Poisson process, Renewal theory, Markov chains, Reversibility, Queueing networks, Martingales, Random walk.

### Course Description

Basic mathematical modeling is at the heart of engineering. In both electrical and computer engineering, many complex systems are modeled using stochastic processes. This course will introduce students to basic stochastic processes tools that can be utilized for performance analysis and stochastic modeling of dynamic systems and networks.

### Teams/GitHub Information

#### Teams

We will use Microsoft Teams for all the course related communication.

**Please do not send any email regarding the course.**

You can signup for the course team Stochastic-Processes-2023 using the following code **pi3h8to**.

#### GitHub

All the students in the class have read access to Stochastic-Processes public repository on GitHub.

Please follow the guidelines in the sample lecture.

The source file for the sample lecture is in the repository.

It is recommended you save it with another name in your local repository for creating a new lecture.

Here is a good book for Git and a simple tutorial.

### Instructor

Parimal Parag

**Office:** EC 2.17

**Hours:** By appointment.

### Time and Location

**Classroom:** Auditorium 2, MP 30, ECE MP Building

**Hours:** Tue/Thu 10:00am-11:30am.

### Teaching Assistants

Nitika

**Email:** nitika2021@iisc.ac.in

**Hours:** By appointment.

### Textbooks

Stochastic Processes, P. Parag and Vinod Sharma.

Stochastic Processes, Sheldon M. Ross, 2nd edition, 1996. You can get a copy of the textbook from the campus book store.

Introduction to Stochastic Processes, Erhan Cinlar, 2013.

Markov Chains: Gibbs Fields, Monte Carlo Simulation, and Queues, Pierre Bremaud, 1999.

Markov Chains, James R. Norris, 1998.

Reversibility and Stochastic Networks, Frank P. Kelly, 2011.

Probability: Theory and Examples, Rick Durett, 4th edition, 2010.