Random samples and sample means

The logic of statistical hypothesis testing is a bit different from that of probability theory, where ideal conditions are assumed. First off, we will evaluate a large data set, or statistical population, based on samples.

It's not necessary to analyze the entire data set. All you need is a small but representative portion of the data that reflects the features of the population as a whole. The easiest way to ensure representativeness is to take a random sample. Random elements are selected from the entire data set using a random number generator. Using these elements, we can draw conclusions about the population as a whole.

Some data sets may have several parts of unequal size that differ greatly with respect to the parameter you're studying. In these cases, it's a good idea to take proportional random samples of each part and then combine them. The result is a stratified sample that's more representative than a regular random sample would be. We call it "stratified" because we divided the population into strata, or groups, that have something in common. These strata are then used to produce random samples.

With a sample, you can draw conclusions about the population — about its statistical parameters, to be precise. It's usually enough to estimate mean values and variance in order to draw a conclusion about equality or inequality with regard to populations' mean values.

What can we learn about the mean and variance of a population based on the mean and variance that we calculate for a sample (also called the sample mean and sample variance)? Almost everything, as long as our sample is sufficiently large.

Here is one way to state the central limit theorem: if there are sufficient observations in a sample, the sampling distribution of the sample mean of any statistical population is normally distributed around the mean of this population. “Any statistical population” means that the statistical population can have any distribution. The data set of mean values of the samples will still be normally distributed around the mean of the entire statistical population.

The standard deviation of the sample mean from the mean of the population is called the standard error and is found using the formula:

$$ E.S.E. = \frac{S}{\sqrt{n}} $$

E.S.E. stands for estimated standard error. It’s “estimated” because we only have a sample. We don’t know the exact error, we just estimate it based on the data we have.

S is the population's estimated standard deviation.

n is sample size. Since the square root of n is in the denominator, the standard error decreases as the sample size increases.

Formulating hypotheses

No experimentally obtained data will ever confirm any hypothesis. This is our fundamental limitation. The data can only contradict the hypothesis or, conversely, show that the results are extremely unlikely (provided that the hypothesis is true). But in both cases we have no grounds to state that the hypothesis has been proven.

If the data doesn't contradict the hypothesis, we simply don't reject it. But if, assuming the hypothesis is true, we'd be highly unlikely to get such data, we have a reason to reject the hypothesis.

Typical hypotheses pertain to the means of statistical populations and look like this:

—The mean of a population equals a certain value.

—The means of two populations are equal to each other.

A test of a statistical hypothesis always begins with a statement of the hypothesis. First, we state the null hypothesis, H₀. For example, “The mean of the population in question equals A,” where A is some number. The alternative hypothesis, H₁, is based on H₀. For this H₀, H₁ would be, “The mean of the population does not equal A.” H₀ is always stated with an equal sign.

We construct a distribution based on the assumption that H₀ is true. In this case, it would be a normal distribution around the parameter of interest, the mean. The variance, or standard deviation (remember, it's a measure of the distribution's dispersion, or scatter) is estimated based on the sample.