Math 5061: Theory of Statistics I

Fall 2016

Fall 2016

Instructor:
Todd Kuffner (kuffner@math.wustl.edu)

Lecture: 1:00-2:30pm, Monday/Wednesday, Cupples I, Room 199

Office Hours: Monday 8:00-9:00am, Tuesday 4:00-6:00pm; Cupples I Room 18 (basement).

Course Overview: This course is intended for Ph.D. students in Statistics and Mathematics. Math 5061-5062 together form a year-long sequence in mathematical statistics leading to the Ph.D. qualifying exam in statistical theory. The first semester will cover introductory measure-theoretic probability, decision theory, notions of optimality, principles of data reduction, and finite sample estimation and inference. We will discuss foundational issues, and consider several paradigms for testing, such as the Neyman-Pearson, neo-Fisherian, and Bayesian approaches. Roughly half of the first semester is devoted to the measure-theoretic foundations of probability theory and statistics. The second semester will cover asymptotic theory, including convergence in measure, limit theorems, integral and density approximations, and higher-order asymptotics. Maximum likelihood, Bayesian, and bootstrap methods will be considered. Empirical processes, large deviations, and modern topics (e.g. Bayesian nonparametric asymptotics) will be introduced as time permits. The style of the course is theorem-proof based; applications will not be emphasized, and examples will be theoretical. Statistical software is not part of the course.

Prerequisite: It is assumed that students have taken a first course in real analysis, probability, and mathematical statistics, and are familiar with basic topology, multivariate calculus, and matrix algebra. Ph.D. students are strongly encouraged to enroll in Math 5051 concurrently (Ph.D.-level measure theory and functional analysis).

If you are undecided about whether or not to take this course, it may be helpful to look at the Ph.D. qualifying exam from the last time I taught the course (2014-2015). This time there will be more measure theory and probability theory on exams.

Textbook: There are many excellent books and online resources for the material in this course. However, no single book is suitable. Due to the cost of purchasing several books, I will not require that students use any particular books. The recommended readings for each lecture are accompanied by sections of three books listed below, but students are welcome to look at other references for the same material. I will use the same books for Math 5061 and Math 5062. The links give electronic access to two of the books for Washington University students (logged in to library account) through SpringerLink, but I also recommend purchasing these books as they are excellent references for researchers.

Homework: There will be weekly homework assignments. You are strongly encouraged to write your solutions in LaTeX. If not, then handwritten submissions must be clear and organized. Homework will be graded, but solutions will not be provided to students.

Homework grader: Qiyiwen Zhang (qiyiwenzhang@wustl.edu)

Blackboard: During the semester, homework assignments, homework and midterm exam grades and any other course-related announcements will be posted to Blackboard or sent by email using Blackboard.

Attendance: Attendance is required for all lectures. The student who misses a lecture is responsible for any assignments and/or announcements made.

Grades: 15% Homework, 20% Midterm 1, 20% Midterm 2, 45% Final

Exams: 2 in-class midterms and 1 final. The dates of the exams should not be considered fixed until the first day of class. What appears on Course Listings may be incorrect.

Homework: There will be weekly homework assignments. The lowest homework grade will be dropped. If you added the class late and missed the first homework, then that will count as your dropped homework.

Final Course Grade: The letter grades for the course will be determined according to the following numerical grades on a 0-100 scale.

Other Course Policies: Students are encouraged to look at the Faculty of Arts & Sciences policies.

Lecture: 1:00-2:30pm, Monday/Wednesday, Cupples I, Room 199

Office Hours: Monday 8:00-9:00am, Tuesday 4:00-6:00pm; Cupples I Room 18 (basement).

Course Overview: This course is intended for Ph.D. students in Statistics and Mathematics. Math 5061-5062 together form a year-long sequence in mathematical statistics leading to the Ph.D. qualifying exam in statistical theory. The first semester will cover introductory measure-theoretic probability, decision theory, notions of optimality, principles of data reduction, and finite sample estimation and inference. We will discuss foundational issues, and consider several paradigms for testing, such as the Neyman-Pearson, neo-Fisherian, and Bayesian approaches. Roughly half of the first semester is devoted to the measure-theoretic foundations of probability theory and statistics. The second semester will cover asymptotic theory, including convergence in measure, limit theorems, integral and density approximations, and higher-order asymptotics. Maximum likelihood, Bayesian, and bootstrap methods will be considered. Empirical processes, large deviations, and modern topics (e.g. Bayesian nonparametric asymptotics) will be introduced as time permits. The style of the course is theorem-proof based; applications will not be emphasized, and examples will be theoretical. Statistical software is not part of the course.

Prerequisite: It is assumed that students have taken a first course in real analysis, probability, and mathematical statistics, and are familiar with basic topology, multivariate calculus, and matrix algebra. Ph.D. students are strongly encouraged to enroll in Math 5051 concurrently (Ph.D.-level measure theory and functional analysis).

If you are undecided about whether or not to take this course, it may be helpful to look at the Ph.D. qualifying exam from the last time I taught the course (2014-2015). This time there will be more measure theory and probability theory on exams.

Textbook: There are many excellent books and online resources for the material in this course. However, no single book is suitable. Due to the cost of purchasing several books, I will not require that students use any particular books. The recommended readings for each lecture are accompanied by sections of three books listed below, but students are welcome to look at other references for the same material. I will use the same books for Math 5061 and Math 5062. The links give electronic access to two of the books for Washington University students (logged in to library account) through SpringerLink, but I also recommend purchasing these books as they are excellent references for researchers.

- K.B. Athreya and S.N. Lahiri's Measure
Theory and Probability Theory, First Edition, Springer. electronic access errata ; thanks to Soumendra Lahiri for sending this to me!

- E.L. Lehmann and G. Casella's Theory of Point Estimation, Second Edition, Springer. errata more errata

- E.L. Lehmann & J.P. Romano's Testing
Statistical Hypotheses, Third Edition, Springer. electronic access errata

Homework: There will be weekly homework assignments. You are strongly encouraged to write your solutions in LaTeX. If not, then handwritten submissions must be clear and organized. Homework will be graded, but solutions will not be provided to students.

Homework grader: Qiyiwen Zhang (qiyiwenzhang@wustl.edu)

Blackboard: During the semester, homework assignments, homework and midterm exam grades and any other course-related announcements will be posted to Blackboard or sent by email using Blackboard.

Attendance: Attendance is required for all lectures. The student who misses a lecture is responsible for any assignments and/or announcements made.

Grades: 15% Homework, 20% Midterm 1, 20% Midterm 2, 45% Final

Exams: 2 in-class midterms and 1 final. The dates of the exams should not be considered fixed until the first day of class. What appears on Course Listings may be incorrect.

Homework: There will be weekly homework assignments. The lowest homework grade will be dropped. If you added the class late and missed the first homework, then that will count as your dropped homework.

Final Course Grade: The letter grades for the course will be determined according to the following numerical grades on a 0-100 scale.

A+ |
impress me |
B+ |
[87, 90) |
C+ |
[77, 80) |
D+ |
[67, 70) |
F |
[0,60) |

A |
93+ |
B |
[83, 87) |
C |
[73, 77) |
D |
[63, 67) |
||

A- |
[90, 93) |
B- |
[80, 83) |
C- |
[70, 73) |
D- |
[60, 63) |

Other Course Policies: Students are encouraged to look at the Faculty of Arts & Sciences policies.

- Academic integrity: Students are expected to adhere to the University's policy on academic integrity.
- Auditing: There is an
option to audit, but this
still involves enrolling in the course. See the Faculty of Arts &
Sciences policy
on auditing.
Auditing students will still be expected to attend all
lectures and compete all required coursework and exams. A course grade of 75 is required for a successful audit.

- Collaboration: Students are encouraged to discuss homework with one another, but each student must submit separate solutions, and these must be the original work of the student.
- Exam conflicts: Read the
University policy.
The exam dates for this course are posted before the semester begins,
and thus you are expected to be present at all exams.

- Late homework: Only by
prior arrangement. If a valid reason for an exception is not presented at least 36
hours before a homework due date, then it will not be accepted late (a
zero will be given for that assignment).

- Missed exams: There are no make-up exams. For valid excused absences with midterm exams - such as medical, family, transportation and weather-related emergencies - the contribution of that midterm to the final course grade will be redistributed equally to the other midterm exam and final exam. Students missing both midterm exams and/or the final exam cannot earn a passing grade for the course.

08/29 |
Lecture 1 Review of set theory; algebras, sigma algebras; Borel sets Reading: Appendix of AL HW1 assigned |

08/31 |
Lecture 2 Measures; extensions Reading: AL 1.1-1.2 HW1 due HW2 assigned |

09/05 |
Labor Day; no class |

09/07 |
Lecture 3 Completeness; measurable transformations Reading: AL 1.3-1.4; AL 2.1-2.2 HW2 due Friday 09/09 |

09/12 |
Lecture 4 Induced measures; distribution functions; Lebesgue and Riemann integration Reading: AL 2.1-2.4 HW3 assigned |

09/14 |
Lecture 5 Convergence for measurable functions Reading: AL 2.5 |

09/19 |
Lecture 6 (guest lecturer) Important Inequalities (Markov, Chebychev, Cramer, Jensen, Holder, Cauchy-Schwarz, Minkowski) Reading: AL 3.1 |

09/21 |
Lecture 7 (guest lecturer) L^p spaces, Banach spaces, and Hilbert spaces Reading: AL 3.2-3.3 |

09/26 |
Lecture 8 Radon-Nikodym theorem; signed measures Reading: AL 4.1-4.2 |

09/28 |
Lecture 9 Functions of bounded variation; absolutely continuous function on R; singular distributions; product spaces; product measures Reading: AL 4.3-4.4, 5.1 |

10/03 |
Lecture 10 Fubini-Tonelli theorems; sample spaces; random variables; Kolmogorov's consistency theorem Reading: AL 5.2 (note: 5.3-5.8 would be part of an analysis course), 6.1-6.3 |

10/05 |
Lecture 11 Expectation; moment generating functions; pi-lambda Theorem; independence Reading: AL 6.1-6.3, 7.1 |

10/10 |
Lecture 12 Exchangeability; Representation Theorems; Borel-Cantelli lemmas; Kolmogorov's 0-1 Law Reading: 7.1-7.2 |

10/12 |
Midterm 1 during class Material: Lectures 1-10 |

10/17 |
Fall Break; no class |

10/24 |
Lecture 13 Conditional expectations/probability; regular conditional distributions; Bayesian statistical experiments Reading: 12.1-12.3 Recommended review material before next lecture: TPE 1.1-1.4 |

10/26 |
Lecture 14 Decision theory; data reduction via sufficiency Reading: TSH 1.1-1.2, 1.4; TPE 1.6 |

10/31 |
Lecture 15 Exponential families; optimal data reduction via minimal sufficiency and completeness Reading: TPE 1.5-1.6 |

11/02 |
Lecture 16 More data reduction; risk reduction Reading: TPE 1.6-1.7, 2.1-2.3 |

11/07 |
Lecture 17 Optimal unbiased and location equivariant estimation; risk unbiasedness Reading: TPE 2.1-2.3, 3.1-3.3 |

11/09 |
Midterm 2 during class Material: Lectures 11-17 |

11/14 |
Lecture 18 Bayes estimators and average risk optimality Reading: TPE 4.1-4.3 |

11/16 |
Lecture 19 Bayes estimators and average risk optimality Reading: TPE 4.1-4.3 |

11/21 |
Lecture 20 Minimax estimators and worst-case optimality Reading: TPE 5.1-5.2 |

11/28 |
Thanksgiving Break; no class |

11/30 |
Lecture 21 Minimax estimators and worst-case optimality Reading: TPE 5.1-5.2 |

12/05 |
Lecture 22 Minimax estimation; admissibility; simultaneous estimation Reading: TPE 5.1-5.2, 4.7 (p. 272-277), 5.5 (p. 355-360) |

12/07 |
Lecture 23 Robust estimation; high-dimensional estimation Reading: handout |

12/09 |
Last day of fall semester classes |

12/19 |
Final Exam scheduled 6:00-8:00pm in Cupples I Room 199 Material: Lectures 1-23 |