Hypothesis Testing I

Week 11

Author

Maghfira Ramadhani

Published

Oct 27, 2025

Plan

In today’s lecture, we will learn about:

Problem of testing hypotheses
Null and alternative hypothesis (two-sided and one-sided)
Rejection region
Significance level, p-value, and power of test

Textbook Reference: JA 16.1, SDG 9.1

Problem of testing hypotheses

In general, hypothesis testing concerns trying to decide whether a parameter $\theta$ lies in one subset of the parameter space.
Consider a statistical problem involving a parameter $\theta$ whose value is unknown but must lie in a certain parameter space $\bf\Omega$.
Suppose now that $\bf\Omega$ can be partitioned into two disjoint subsets $\bf\Omega_0$ and $\bf\Omega_1$ , and the statistician is interested in whether $\theta$ lies $\bf\Omega_0$ or $\bf\Omega_1$

Problem of testing hypotheses

Let $H_0$ denotes hypothesis that $\theta\in\bf\Omega_0$ and $H_1$ denote the hypothesis that $\theta\in\bf\Omega_1$.
Since $\bf\Omega_0$ and $\bf\Omega_0$ are disjoint, and $\bf\Omega_0\cup\bf\Omega_1$, exactly one of the hypothesis must be true.
A problem of this type, in which there are only two possible decisions, is called a problem of testing hypotheses
A procedure for deciding which hypothesis to choose is called a test procedure or simply a test

Null and altenative hypotheses

The hypothesis $H_0$ is called the null hypothesis
The hypothesis $H_1$ is called the alternative hypothesis
When performing a test, if we decide that $\theta$ lies in $\bf\Omega_1$, then we are said to reject $H_0$.
If we decide that $\theta$ lies in $\bf\Omega_0$, we do are said not to reject $H_0$.

Example 16.2 Investment Opportunity

You are interested in the possibility of buying a business that produces and sells a certain product.
By your calculations, the true average of weekly sales would need to beat least $10,000 for the investment to be worthwhile.
As part of due diligence, you obtain weekly sales figures from the business for 10 randomly chosen weeks.
For those 10 weeks, the sample mean of weekly sales is $11,200, and the sample standard deviation of weekly sales is $3,400.

Example 16.2 Investment Opportunity

Two-sided test

For a two-sided test of an unknown parameter $\theta$

The null hypothesis is \[H_0:\theta=c,\]
The alternative hypothesis is \[H_1:\theta\neq c.\]
The hypothesis test of $H_0$ determines whether there is statistical evidence to reject $H_0$.

One-sided test

For a one-sided test of an unknown parameter $\theta$

The null and alternative hypothesis are \[H_0:\theta\geq c,\quad\quad H_1:\theta< c.\]
Another version of null and alternative hypothesis are \[H_0:\theta\leq c,\quad\quad H_1:\theta> c.\]
The hypothesis test of $H_0$ determines whether there is statistical evidence to reject $H_0$.

Example 16.2 Investment Opportunity

Using our example of investment opportunity, we have n=10 sample of weekly sales with $\bar{x}=11,200$ and $s_x=3,400$.
When we discussed confidence interval we know that \[\text{t-ratio}=\frac{\bar{X}-\mu}{s_X/\sqrt{n}}\overset{}{\sim}t_{n-1}\]
Ultimately, we want to test whether $\mu\leq 10,000$ (then don’t invest) or $\mu> 10,000$ (then invest)

Example 16.2 Investment Opportunity

Using our example of investment opportunity, we have n=10 sample of weekly sales with $\bar{x}=11,200$ and $s_x=3,400$.
But for now let’s test whether $H_0=\mu=10,000$ is true.
Plugging the 10,000 value into our t-ratio, now we call it t-statistics, then \[\text{t-statistics}=\frac{\bar{X}-10,000}{s_X/\sqrt{n}}\overset{}{\sim}t_{n-1} \quad \text{ when }H_0 \text{ is true}.\]
This t-statistics tells us the number of standard deviations that our estimator $X$ is away from 10,000.
The t-statistics is positive when our estimate is above 10,000 and negative if our estimate is below 10,000.

Example 16.2 Investment Opportunity

Using our example of investment opportunity, we have n=10 sample of weekly sales with $\bar{x}=11,200$ and $s_x=3,400$.
Intuitively, our realized t-statistics (plug in the sample mean and s.d. from sample) should also distributed following $t_{n-1}$ if $H_0$ is true.
Consider the 95% probability interval of our t-statistics when $H_0$ is true:\[P\left(\left|\frac{\bar{X}-10,000}{s_X/\sqrt{n}}\right|<t_{n-1,0.025}\right)=0.95\text{ when }H_0 \text{ is true}.\]
Thus, our 95% confidence interval of our t-statistics when $H_0$ is true:\[P\left(\left|\frac{\bar{x}-10,000}{s_x/\sqrt{n}}\right|<t_{n-1,0.025}\right)=0.95\text{ when }H_0 \text{ is true}.\]

Example 16.2 Investment Opportunity

Using our example of investment opportunity, we have n=10 sample of weekly sales with $\bar{x}=11,200$ and $s_x=3,400$.
Thus, our 95% confidence interval of our t-statistics when $H_0$ is true:\[P\left(\left|\frac{\bar{x}-10,000}{s_x/\sqrt{n}}\right|<t_{n-1,0.025}\right)=0.95\text{ when }H_0 \text{ is true}.\]
Our realized t-statistics is $\frac{11,200-10,000}{3,400\sqrt{10}}=0.1117$
$t_{n-1,0.025}=2.262$ (quantile of t-distribution with 9 d.o.f. at 1-0.025)
Since our t-statistics is lower than the critical value, how do we interpret this statistical evidence?

Level or significance level

Level or significance level of a test, denoted $\alpha$, is the probability that the null hypothessi $H_0$ is rejected when $H_0$ is true.

It is also called the type I error of the test

In previous example, we arbitrarily chose $\alpha=0.05$ as our level, you can work with smaller (more conservative) or larger (more lenient) depending on your preferred precision.

Two sided t-test rejection rules

The relevant probability statement is: \[P\left(\left|\frac{\bar{X}-c}{s_X/\sqrt{n}}\right|<t_{n-1,\alpha/2}\right)=1-\alpha\text{ when }H_0:\mu=c \text{ is true}.\]

Rejection rule based on t-statistics (test at $\alpha$-level)
Reject $H_0:\mu=c$ if $\|\text{t-stat}\|=\left\|\frac{\bar{x}-c}{s_x/\sqrt{n}}\right\|\geq t_{n-1,\alpha/2}$ Do not reject $H_0:\mu=c$ if $\|\text{t-stat}\|=\left\|\frac{\bar{x}-c}{s_x/\sqrt{n}}\right\|< t_{n-1,\alpha/2}$

Two sided t-test rejection rules

The relevant probability statement is: \[P\left(\left|\frac{\bar{X}-c}{s_X/\sqrt{n}}\right|<t_{n-1,\alpha/2}\right)=1-\alpha\text{ when }H_0:\mu=c \text{ is true}.\]

Rejection rule based on confidence interval (test at $\alpha$-level)
Reject $H_0:\mu=c$ if $c$ is not within the two-sided $1-\alpha$ confidence interval for $\mu$ Do not reject $H_0:\mu=c$ if $c$ is within the two-sided $1-\alpha$ confidence interval for $\mu$

Two sided t-test rejection rules

P-value for two-sided t-test

The p-value of a test of the null hypothesis $H_0$ is the smallest level $alpha^*$ such that the test rejects $H_0$ at $\alpha$-level.

\[\text{p-value}=P(|T|>|\text{t-stat}|) \text{ when }H_0:\mu=c \text{ is true, where }T\sim t_{n-1}.\]

Rejection rule based on p-value (test at $\alpha$-level)
Reject $H_0:\mu=c$ if $\text{p-value}<\alpha$ Do not reject $H_0:\mu=c$ if $\text{p-value}>\alpha$

Power of the t-test

So far the level of $\alpha$ indicates the probability that we reject $H_0$ given that $H_0$ is true.
Power ($\beta$) is the probability of rejecting $H_0$ when $H_0$ is not true

Null Hypothesis is

True

False

Rejected

Type I error

False positive

Probability=$\alpha$

Correct decision

True positive

Probability=$1-\beta$

Not rejected

Correct decision

True negative

Probability=$1-\alpha$

Type II error

False negative

Probability=$\beta$

Power of the t-test

Up next

One-sided t-test
Asymptotic hypothesis testing
HW4 canceled