Math/Stat 632 Introduction to Stochastic Processes

Spring 2018, Lecture 2


Meetings: TuTh 11:00AM - 12:15PM, 6203 Social Sciences.
Instructor: Daniele Cappelletti.
Office: 415 Van Vleck.
Office Hours: Wednesday 4.00 - 6.00 PM, and by appointment (offered by me, in 415 Van Vleck);
Sunday 4.00-6.00 PM (with the student Thomas Hameister in B205 Van Vleck).
Wednesday 7.00-9.00 PM (with the student Thomas Hameister in B205 Van Vleck).
You can also partecipate to the office hours offered for Section 1, since the material we will cover is the same. You can find the relevant information for Section 1 here
E-mail: cappelletti@math.wisc.edu


This is the course homepage that also serves as the syllabus for the course. Here you will find our weekly schedule, and updates on scheduling matters. The Mathematics Department also has a general information page on this course.

I will use the class email list to send out corrections, announcements, etc.  Please check your wisc.edu email regularly.

Number of credits: 3. This course meets for 3 hours per week, and there is an expectation that students will work an additional 6 hours per week outside of class.

Course description

Math 632 provides an introduction to stochastic processes, which describe the time evolution of a system in a random setting. We will work with basic stochastic processes and applications with an emphasis on problem solving. Topics will include  discrete-time Markov chains, Poisson point processes, continuous-time Markov chains, and renewal processes.

A typical advanced math course follows a strict theorem-proof format. 632 is not of this type. Mathematical theory is discussed in a precise fashion but only some results can be rigorously proved in class. This is a consequence of time limitations and the desire to leave measure theory outside the scope of this course. Interested students can find the proofs in the textbook. For a thoroughly rigorous probability course students should sign up for the graduate probability sequence 733-734.

To go beyond Math 632 in probability you should consider taking Math 635 - Introduction to Brownian Motion and Stochastic Calculus.

Learning Goals

Where are stochastic processes used?

Stochastic processes are a mathematical tool to model the time evolution of systems with aleatory features. Hence, they are used ubiquitously throughout the sciences and industry.  For example, in biology many cellular phenomena are now modeled as stochastic processes rather than as deterministic solution of ODEs  (this is the subject I am doing research on). As for industry, many models of interaction with customers or machine maintenance are probabilistic in nature, as we cannot determine customer choices or the time when a machine breaks deterministically. In informatics, a solid understanding of stochastic processes is extremely useful in designing computer algorithms that for example learn to play chess, or perform speech recognition.

Prerequisites

The material is treated at a level that does not require measure theory. Consequently technical prerequisites needed for this course are not too heavy: calculus, linear algebra, and an introduction to probability (at the level of Math 431) are sufficient. However, the material is sophisticated, so a degree of intellectual maturity and a willingness to work hard are required. For this reason some 500-level work in mathematics is recommended for background, preferably in analysis (521).

Good knowledge of undergraduate probability at the level of UW-Madison Math 431 (or an equivalent course) is required. This means familiarity with basic probability models, random variables and their probability mass functions and distributions, expectations, joint distributions, independence, conditional probabilities, the law of large numbers and the central limit theorem.  If you need a thorough review of basic probability, the textbook "Introduction to Probability" (on reserve in math library) by David F. Anderson, Timo Seppäläinen, and Benedek Valko is recommended.

Textbook

I will provide you with lecture notes that contain almost everything that is said in class. These should be the primary source for your studies. The lecture notes are based on (and sometimes reference to) the following book: Durrett: Essentials of Stochastic Processes,  Springer, 2nd edition. Available on Prof. Durrett's website.  Note that you do not need to purchase a book. However you need to download the pdf file, since you will be assigned exercises from it.
Other textbooks which could be used for supplemental reading:

Canvas

We will use the Canvas website of the course to post homework assignments and solutions. The lecture notes will also be posted there.

Piazza

We will be using Piazza for class discussion.  The system is catered to getting you help fast and efficiently from classmates and myself.  Rather than emailing math questions to me, I encourage you to post your questions on Piazza.  Students of Lecture 1 and Lecture 3 of the course share the same Piazza page. If you have any problems or feedback for the developers, email team@piazza.com.

Find our class Piazza page here.

Evaluation

Course grades will be based on homework (15%),  two midterm exams (2x25%), and a comprehensive final exam (35%).  Your lowest homework grade will be dropped. No calculators, cell phones, or other gadgets will be permitted in exams, only pencils, pens and paper.
The final grades will be determined according to the following scale:

A:[100,90], AB: (90,86], B: (86,77], BC: (77,73], C: (73,64], D: (64,50), F: [50,0].

There will be no curving in the class, but the instructor reserves the right to modify the final grade lines.

How to succeed in this course

The midterm and final exams will contain problems which will be similar to the homework problems in difficulty. The best way to prepare for these is to do as many practice problems from the book as you can. This will also help you understand the theory a lot better!
If you have trouble solving the homework (or practice) problems then come see me in my office hours (or set up an appointment).

Homework

Homework assignments will be posted on the Learn@UW site of the course. Weekly homework assignments are due on Fridays at 4 PM.

Instructions for homework

Gradescope

We will use the software Gradescope, which is already being succesfully tested in other courses here at UW-Madison. The advantages for you are the following:

Here is a guide on how you have to submit your homework. You can also watch the video "For students: submitting homework" here.

Weekly schedule

Here is a tentative weekly schedule, to be adjusted as we go. The numbers refer to sections in the textbook (but we will do something differently from how it is done in the textbook - please follow the lecture notes you will be provided with).


Week Covered topics
1
1/23-1/26 		Review of basic concepts of probability. Discrete Markov chains: definitions and examples,
the transition probability matrix, multistep probabilities.
Sections 1.1-1.2
2
1/29-2/2 		Classification of states, strong Markov property, transience and recurrence, closed and irreducible sets.
Section 1.3
3
2/5-2/9		 Stationary distributions.
Section 1.4
4
2/12-2/16		 Periodicity, limit behavior, one dimensional random walk, expected return time.
Sections 1.5-1.6, 1.10
5
2/19-2/23 		Detailed balanced stationary distributions, birth and death chains, the Metropolis-Hastings algorithm, exit times and exit distributions.
Sections 1.6, 1.8, 1.9, 1.10
6
2/26-3/2 		The gambler's ruin problem, Birth and death chains with infinite state space, extinction probability in a branching process.
Sections 1.9, 1.10
7
3/5-3/9		 Properties of the exponential distribution, the Poisson process, number of arrivals in an interval, the spatial Poisson process.
First midterm exam
Sections 2.1-2.2
8
3/12-3/16		 Compound Poisson process, transformations of Poisson processes. Renewal processes.
Sections 2.3, 2.4, 3.1
9
3/19-3/23 		Renewal processes. Age and residual life.
Sections 3.1, 3.3
​​​​​-- Spring Break --
10
4/2-4/6 		Continuous time Markov chains, Markov property in continuous time, the embedded discrete time MC, infinitesimal rates.
Section 4.1
11
4/9-4/13		 Examples, the Poisson clock construction, Kolmogorov's backward and forward equations.
Section 4.1-4.2
12
4/16-4/20 		Solving the Kolmogorov equations, stationary distribution, limiting behavior.
Section 4.3
13
4/23-4/27 		Stationary distribution, detailed balance.
Second midterm exam
Section 4.3
14
4/30-5/4 		Birth and death processes, exit times, Queueing Theory.