Math/Stat 632 Introduction to Stochastic Processes - Lecture 1

Spring 2017, Lecture 1

Meetings: MWF, 12:05-12:55 PM Sewell Social Sciences 6102
Instructor: Daniele Cappelletti
Office: 415 Van Vleck.
Office Hours: Wednesday 3.30 - 5.00 pm, and by appointment.
E-mail:cappelletti@math.wisc.edu


This is the course homepage that also serves as the syllabus for the course. Here you will find our weekly schedule, and updates on scheduling matters. The Mathematics Department also has a general information page on this course.

I will use the class email list to send out corrections, announcements, etc.  Please check your wisc.edu email regularly.

Course description

Math 431 provides an introduction to stochastic processes, which describe the time evolution of a system in a aleatory setting. We will work with basic stochastic processes and applications with an emphasis on problem solving. Topics will include  discrete-time Markov chains, Poisson point processes, continuous-time Markov chains, and renewal processes.

A typical advanced math course follows a strict theorem-proof format. 632 is not of this type. Mathematical theory is discussed in a precise fashion but only some results can be rigorously proved in class. This is a consequence of time limitations and the desire to leave measure theory outside the scope of this course. Interested students can find the proofs in the textbook. For a thoroughly rigorous probability course students should sign up for the graduate probability sequence 831-832.

To go beyond Math 632 in probability you should consider taking Math 635 - Introduction to Brownian Motion and Stochastic Calculus.

Where are stochastic processes used?

Stochastic processes are a mathematical tool to model the time evolution of systems with aleatory features. Hence, they are used ubiquitously throughout the sciences and industry.  For example, in biology many cellular phenomena are now modeled as stochastic processes rather than as deterministic solution of ODEs  (this is the subject I am doing research on). As for industry, many models of interaction with customers or machine maintenance are probabilistic in nature, as we cannot determine customer choices or the time when a machine breaks deterministically. In informatics, a solid understanding of stochastic processes is extremely useful in designing computer algorithms that for example learn to play chess, or perform speech recognition.

Prerequisites

The material is treated at a level that does not require measure theory. Consequently technical prerequisites needed for this course are not too heavy: calculus, linear algebra, and an introduction to probability (at the level of Math 431) are sufficient. However, the material is sophisticated, so a degree of intellectual maturity and a willingness to work hard are required. For this reason some 500-level work in mathematics is recommended for background, preferably in analysis (521).

Good knowledge of undergraduate probability at the level of UW-Madison Math 431 (or an equivalent course) is required. This means familiarity with basic probability models, random variables and their probability mass functions and distributions, expectations, joint distributions, independence, conditional probabilities, the law of large numbers and the central limit theorem.  If you need a thorough review of basic probability, the textbook A First Course in Probability (on reserve in math library) by Sheldon Ross is recommended.

Textbook

Durrett: Essentials of Stochastic Processes,  Springer, 2nd edition. Available on Prof. Durrett's website.  Note that you do not need to purchase a book.

Other textbooks which could be used for supplemental reading:

Learn@UW

We will use the Learn@UW website of the course to post homework assignments and solutions. The lecture notes will also be posted there.

 Piazza
We will be using Piazza for class discussion.  The system is catered to getting you help fast and efficiently from classmates and myself.  Rather than emailing math questions to me, I encourage you to post your questions on Piazza.  If you have any problems or feedback for the developers, email team@piazza.com.

Find our class Piazza page here.

Evaluation

Course grades will be based on homework (15%),  two midterm exams (2x25%), and a comprehensive final exam (35%).  Your lowest homework grade will be dropped. ​​​​​​​​​​As part of your homework, you are asked to use Webwork, an online platform where you can enter the solution of some exercises (Scroll down for more information). No calculators, cell phones, or other gadgets will be permitted in exams, only pencils, pens and paper.
The final grades will be determined according to the following scale: 
A: [100,89), AB: [89,87), B: [87,76), BC: [76,74), C: [74,62), D: [62,50), F: [50,0]. 
There will be no curving in the class, but the instructor reserves the right to modify the final grade lines.

How to succeed in this course

The midterm and final exams will contain problems which will be similar to the homework problems in difficulty. The best way to prepare for these is to do as many practice problems from the book as you can. This will also help you understand the theory a lot better!
If you have trouble solving the homework (or practice) problems then come see me in my office hours (or set up an appointment).

Homework

Homework assignments will be posted on the Learn@UW site of the course. Weekly homework assignments are due Fridays at the beginning of the class.

Instructions for homework

Weekly schedule

Here is a tentative weekly schedule, to be adjusted as we go. The numbers refer to sections in the textbook.


Week Covered topics
1
1/16-9/20 		Review of basic concepts of probability. Discrete Markov chains: definitions and examples,
the transition probability matrix, multistep probabilities.
Sections 1.1-1.2
2
1/23-1/27 		Classification of states, strong Markov property, transience and recurrence, closed and irreducible sets.
Section 1.3
3
1/30-2/3		 Stationary distributions, periodicity.
Sections 1.4, 1.5
4 2/6-2/10 Limit behavior, one dimensional random walk, expected return time.
Sections 1.5-1.6, 1.10
5 2/13-2/17 Detailed balanced stationary distributions, birth and death chains, the Metropolis-Hastings algorithm, exit times and exit distributions.
Sections 1.6, 1.8, 1.9, 1.10
6 2/20-2/24 The gambler's ruin problem, Birth and death chains with infinite state space, extinction probability in a branching process.
Sections 1.9, 1.10
7 2/27-3/3 Properties of the exponential distribution, the Poisson process, number of arrivals in an interval, the spatial Poisson process.
First midterm exam
Sections 2.1-2.2
8 3/6-3/10 Compound Poisson process, transformations of Poisson processes. Renewal processes.
Sections 2.3, 2.4, 3.1
9 3/13-3/17 Renewal processes. Age and residual life.
Sections 3.1, 3.3
​​​​​-- Spring Break --
10 3/27-3/31 Continuous time Markov chains, Markov property in continuous time, the embedded discrete time MC, infinitesimal rates.
Section 4.1
11 4/3-4/7 Examples, the Poisson clock construction, Kolmogorov's backward and forward equations.
Section 4.1-4.2
12 4/10-4/14 Solving the Kolmogorov equations, stationary distribution, limiting behavior.
Section 4.3
13 4/17-4/21 Stationary distribution, detailed balance.
Second midterm exam
Section 4.3
14 4/24-4/28 Birth and death processes, exit times, queuing examples,
15 5/1-5/5 Brief introduction to Brownian motion, review.