STA238: Probability, Statistics and Data Analysis II: Index

Someone hands you eighty hourly patient counts from a hospital and asks how busy the emergency room is on an average night.

You can average the eighty numbers, and that is clearly a reasonable thing to do. But it raises a question that turns out to be harder than the arithmetic: how much should anyone trust that average? Count a different eighty hours and you get a different average from the same hospital. So the number you produced depends partly on the emergency room and partly on which hours you happened to pick, and those two influences are mixed together in the single figure you are about to report.

This course is about that situation, and about what you can honestly say in it.

The reason it is hard is that questions like this have no single right answer. Told that patients arrive at an average rate of 3 an hour, you can work out how often a completely quiet hour occurs, and there is exactly one correct result. Running it the other way, from eighty observed counts back to the underlying rate, does not work like that: plenty of different rates could have produced those particular eighty numbers, some more plausibly than others, and nothing in the data picks one out with certainty.

So you never solve for the answer. You build a procedure for guessing, and then you spend most of your effort working out how much that procedure can be trusted, which is a question you can answer even when the guess itself stays uncertain.

That second half is what most of this course actually is. Not clever guesses, but ways of judging guesses.

STA238 covers four things.

1. Building the model. Deciding which family of distributions could have produced the data, and what has to be true of the data for that decision to be legitimate.
2. Building estimators and judging them. A rule for turning data into a guess is an estimator. Most of the course is criteria for telling a good one from a bad one: unbiasedness, variance, mean squared error, efficiency, and the two general recipes that manufacture estimators on demand.
3. Quantifying what you do not know. A single number hides its own uncertainty. Confidence intervals and hypothesis tests are the two ways of reporting it, and they turn out to be the same machinery read two different ways.
4. Relating one variable to another. The linear model extends everything above from a single unknown parameter to a relationship between two measured quantities.

The official description names the same content: statistical models and parameters, estimators of parameters and their statistical properties, methods of estimation, confidence intervals, hypothesis testing, the likelihood function, and the linear model, with data analysis and simulation in R used to guide the theoretical development.

One distinction to get straight early

There is a difference between a recipe and the thing you cooked, and in statistics both get called much the same names, so it is worth separating them now.

Suppose your recipe is “average the numbers.” That recipe exists before you have any data. You can ask sensible questions about it: does it tend to land on the right answer, how much do its results bounce around, is there a better recipe. Those questions are about the procedure itself, across all the data sets it might ever be applied to.

Now apply it to your actual eighty numbers and get 2.5. That is one output. It does not bounce around, because it already happened. Asking whether 2.5 tends to land on the right answer does not mean anything, because 2.5 does not tend to do anything.

The recipe is called an estimator, written with capitals, $T = h(X_1, \dots, X_n)$. It is a random variable, so it has a distribution, an expectation and a variance.

The output is called an estimate, written with lowercase, $t = h(x_1, \dots, x_n)$. It is a number.

Every property this course studies, bias, variance, efficiency, is a property of the estimator. When a question asks whether something is unbiased, it is asking about the recipe, and the answer has to be an argument about the procedure rather than a statement about the number you got.

The pages

The order builds: each page uses the previous ones, but each is written to be read on its own.

1. The Statistical Model, what a random sample is and what it assumes
2. Summarizing a Sample, the numerical and graphical summaries and what each one hides
3. Estimators and Sampling Distributions, the sampling distribution, the law of large numbers, Chebyshev, the central limit theorem
4. Bias, unbiasedness and the arguments that establish it without computing an expectation
5. Mean Squared Error, the decomposition and the bias-variance trade
6. Maximum Likelihood, the recipe, invariance, and the case where calculus fails
7. Efficiency and the Cramer-Rao Bound, comparing estimators and proving one is optimal
8. Confidence Intervals, three constructions and what the interval actually claims
9. The Bootstrap, replacing an unknown distribution with the data itself
10. Hypothesis Testing, test statistics, p-values, critical regions, and the two errors
11. The Linear Model, modelling a relationship between two variables rather than one parameter
12. Reading R Output, the exam skill of interpreting printed output
13. Worked Problems, full-length problems in exam format

Two topics from the end of the course, goodness of fit and Bayesian estimation, are not yet written.

Index

• The Statistical Model. Why you need a model at all, what random variables and expectation and variance actually mean, what a random sample assumes and how those assumptions break, and how to pick a family of distributions for a given kind of data.
• Summarizing a Sample. How to describe a set of numbers with a few numbers or a picture: mean and median, quantiles and the five-number summary, variance and MAD, histograms, kernel density estimates, the empirical CDF, and box plots, with the 1.5 IQR rule worked by hand.
• Estimators and Sampling Distributions. Estimator against estimate, the sampling distribution, Chebyshev inequality, the law of large numbers with its proof, the central limit theorem, standardization, and the distributions derived from the normal: chi-squared, t, and F.
• Bias. Unbiasedness and bias of an estimator, how to prove unbiasedness directly, and the three structural arguments that establish bias without computing an expectation: Jensen inequality, bounded support, and the method of moments variance.
• Mean Squared Error. The mean squared error of an estimator, its decomposition into variance plus squared bias, how to derive it and minimize it over a tuning constant, and the bias-variance tradeoff worked on the exponential and the discrete uniform.
• Maximum Likelihood. The likelihood function, the four-step recipe for finding a maximum likelihood estimator including the second-order check, the invariance principle, the case where calculus fails, and the method of moments as the alternative recipe.
• Efficiency and the Cramer-Rao Bound. Comparing unbiased estimators by variance, relative efficiency, and the Cramer-Rao lower bound: the theoretical floor on the variance of any unbiased estimator, worked in full for the exponential distribution.
• Confidence Intervals. Building an interval estimate: the pivot and inversion argument, intervals from Chebyshev, from the normal with known variance, and from the t distribution with unknown variance, and what a confidence interval does and does not claim.
• The Bootstrap. Approximating a sampling distribution by resampling: the bootstrap principle, the empirical and parametric procedures step by step, why the statistic is centred, and the bootstrap confidence interval for a mean.
• Hypothesis Testing. Null and alternative hypotheses, test statistics, p-values, significance level, critical regions, choosing the right test statistic, the equivalence with confidence intervals, and Type I and Type II errors.
• The Linear Model. Modelling a relationship between two variables: bivariate data, the deterministic line against the statistical one, the simple linear regression model and its assumptions, why the response variables are independent but not identically distributed, and what the regression line predicts.
• Reading R Output. Interpreting printed R output without writing code: the four function families, reading simulation results into bias and MSE, checking a model with mean against variance, and the logical-vector idioms that appear on exams.
• Worked Problems. Full-length problems in the format of the course assessments, with solutions written the way a marker rewards them: the principle named, the justification given, and the arithmetic shown.