Math/Stat 632 Introduction to Stochastic Processes - Lecture 3

Fall 2017, Lecture 3

Meetings: MWF, 2:25-3:15 PM, 594 Van Hise Hall.

Instructor: Daniele Cappelletti.

Office: 415 Van Vleck.

Office Hours: Wednesday 3.30 - 5.30 PM, and by appointment (with me, in my office);

Sunday 4.00-6.00 PM (with the student Thomas Hameister in B205 Van Vleck).

Thursday 2.00-5.30 PM (with the student Thomas Hameister in B205 Van Vleck). Note that here students from Math 431 have the priority.

E-mail: cappelletti@math.wisc.edu

This is the course homepage that also serves as the syllabus for the course. Here you will find our weekly schedule, and updates on scheduling matters. The Mathematics Department also has a general information page on this course.

I will use the class email list to send out corrections, announcements, etc. Please check your wisc.edu email regularly.

Course description

Math 632 provides an introduction to stochastic processes, which describe the time evolution of a system in a aleatory setting. We will work with basic stochastic processes and applications with an emphasis on problem solving. Topics will include discrete-time Markov chains, Poisson point processes, continuous-time Markov chains, and renewal processes.

A typical advanced math course follows a strict theorem-proof format. 632 is not of this type. Mathematical theory is discussed in a precise fashion but only some results can be rigorously proved in class. This is a consequence of time limitations and the desire to leave measure theory outside the scope of this course. Interested students can find the proofs in the textbook. For a thoroughly rigorous probability course students should sign up for the graduate probability sequence 831-832.

To go beyond Math 632 in probability you should consider taking Math 635 - Introduction to Brownian Motion and Stochastic Calculus.

Where are stochastic processes used?

Stochastic processes are a mathematical tool to model the time evolution of systems with aleatory features. Hence, they are used ubiquitously throughout the sciences and industry. For example, in biology many cellular phenomena are now modeled as stochastic processes rather than as deterministic solution of ODEs (this is the subject I am doing research on). As for industry, many models of interaction with customers or machine maintenance are probabilistic in nature, as we cannot determine customer choices or the time when a machine breaks deterministically. In informatics, a solid understanding of stochastic processes is extremely useful in designing computer algorithms that for example learn to play chess, or perform speech recognition.

Prerequisites

The material is treated at a level that does not require measure theory. Consequently technical prerequisites needed for this course are not too heavy: calculus, linear algebra, and an introduction to probability (at the level of Math 431) are sufficient. However, the material is sophisticated, so a degree of intellectual maturity and a willingness to work hard are required. For this reason some 500-level work in mathematics is recommended for background, preferably in analysis (521).

Good knowledge of undergraduate probability at the level of UW-Madison Math 431 (or an equivalent course) is required. This means familiarity with basic probability models, random variables and their probability mass functions and distributions, expectations, joint distributions, independence, conditional probabilities, the law of large numbers and the central limit theorem. If you need a thorough review of basic probability, the textbook A First Course in Probability (on reserve in math library) by Sheldon Ross is recommended.

Textbook

I will provide you with lecture notes that contain almost everything that is said in class. These should be the primary source for your studies. The lecture notes are based on (and sometimes reference to) the following book: Durrett: Essentials of Stochastic Processes, Springer, 2nd edition. Available on Prof. Durrett's website. Note that you do not need to purchase a book. However you need to download the pdf file, since you will be assigned exercises from it.
Other textbooks which could be used for supplemental reading:

Greg Lawler: Introduction to Stochastic Processes, Chapman and Hall

Sidney Resnick: Adventures in Stochastic Processes, Birkhäuser.
Sheldon Ross: Stochastic Processes, Wiley
Sheldon Ross: Introduction to Probability Models, Academic Press

Learn@UW

We will use the Learn@UW website of the course to post homework assignments and solutions. The lecture notes will also be posted there.

Piazza
We will be using Piazza for class discussion. The system is catered to getting you help fast and efficiently from classmates and myself. Rather than emailing math questions to me, I encourage you to post your questions on Piazza. Students of Lecture 1 and Lecture 3 of the course share the same Piazza page. If you have any problems or feedback for the developers, email team@piazza.com.

Find our class Piazza page here.

Evaluation

Course grades will be based on homework (15%), two midterm exams (2x25%), and a comprehensive final exam (35%). Your lowest homework grade will be dropped. As part of your homework, you are asked to use Webwork, an online platform where you can enter the solution of some exercises (Scroll down for more information).

Midterm exam 1: Wednesday, October 11th, 7:15 PM - 8:45 PM,B130 Van Vleck.
Midterm exam 2: Thursday, November 30th, 7:15 PM - 8:45 PM,B130 Van Vleck.
Final exam: Thursday, December 21st, 5:05 PM - 7:05 PM,Ingraham 19.

No calculators, cell phones, or other gadgets will be permitted in exams, only pencils, pens and paper.
The final grades will be determined according to the following scale:
A: [100,89), AB: [89,87), B: [87,76), BC: [76,74), C: [74,62), D: [62,50), F: [50,0].
There will be no curving in the class, but the instructor reserves the right to modify the final grade lines.

How to succeed in this course

The midterm and final exams will contain problems which will be similar to the homework problems in difficulty. The best way to prepare for these is to do as many practice problems from the book as you can. This will also help you understand the theory a lot better!
If you have trouble solving the homework (or practice) problems then come see me in my office hours (or set up an appointment).

Homework

Homework assignments will be posted on the Learn@UW site of the course. Weekly homework assignments are due Fridays at the beginning of the class.

Instructions for homework

Observe rules of academic integrity. Handing in plagiarized work, whether copied from a fellow student or off the web, is not acceptable. Plagiarism cases will lead to sanctions.
Homework is collected at the beginning of the class period on the due date. No late papers will be accepted. You can bring the homework earlier to the instructor's office or mailbox.
Working in groups on homework assignments is strongly encouraged; however, every student must write their own assignments.
Organize your work neatly. Use proper English. Write in complete English or mathematical sentences. Answers should be simplified as much as possible. If the answer is a simple fraction or expression, a decimal answer from a calculator is not necessary. But for some exercises you need a calculator to get the final answer.
Answers to some exercises are in the back of the book, so answers alone carry no credit. It's all in the reasoning you write down.
Put problems in the correct order and staple your pages together.
Be neat. There should not be text crossed out.
Recopy your problems. Do not hand in your rough draft or first attempt.
Papers that are messy, disorganized or unreadable cannot be graded.
Typing up your solutions (perhaps using Latex) is not necessary but it is encouraged.

Weekly schedule

Here is a tentative weekly schedule, to be adjusted as we go. The numbers refer to sections in the textbook (but we will do something differently from how it is done in the textbook - please follow the lecture notes you will be provided with).

Week	Covered topics
1 9/6-9/8	Review of basic concepts of probability. Discrete Markov chains: definitions and examples, the transition probability matrix, multistep probabilities. Sections 1.1-1.2
2 9/11-9/15	Classification of states, strong Markov property, transience and recurrence, closed and irreducible sets. Section 1.3
3 9/18-9/22	Stationary distributions. Section 1.4
4 9/25-9/29	Periodicity, limit behavior, one dimensional random walk, expected return time. Sections 1.5-1.6, 1.10
5 10/2-10/6	Detailed balanced stationary distributions, birth and death chains, the Metropolis-Hastings algorithm, exit times and exit distributions. Sections 1.6, 1.8, 1.9, 1.10
6 10/9-10/13	The gambler's ruin problem, Birth and death chains with infinite state space, extinction probability in a branching process. First midterm exam Sections 1.9, 1.10
7 10/16-10/20	Properties of the exponential distribution, the Poisson process, number of arrivals in an interval, the spatial Poisson process. Sections 2.1-2.2
8 10/23-10/27	Compound Poisson process, transformations of Poisson processes. Renewal processes. Sections 2.3, 2.4, 3.1
9 10/30-11/3	Renewal processes. Age and residual life. Sections 3.1, 3.3
10 11/6-11/10	Continuous time Markov chains, Markov property in continuous time, the embedded discrete time MC, infinitesimal rates. Section 4.1
11 11/13-11/17	Examples, the Poisson clock construction, Kolmogorov's backward and forward equations. Section 4.1-4.2
12 11/20-11/24	Solving the Kolmogorov equations, stationary distribution, limiting behavior. Section 4.3
13 11/27-12/1	Stationary distribution, detailed balance. Second midterm exam Section 4.3
14 12/4-12/8	Birth and death processes, exit times, queuing examples,
15 12/11-12/15	More on queueing theory. if time permits an introduction to Brownian motion.

Meetings:	MWF, 2:25-3:15 PM, 594 Van Hise Hall.
Instructor:	Daniele Cappelletti.
Office:	415 Van Vleck.
Office Hours:	Wednesday 3.30 - 5.30 PM, and by appointment (with me, in my office);
	Sunday 4.00-6.00 PM (with the student Thomas Hameister in B205 Van Vleck).
	Thursday 2.00-5.30 PM (with the student Thomas Hameister in B205 Van Vleck). Note that here students from Math 431 have the priority.
E-mail:	cappelletti@math.wisc.edu