12 Hypothesis Testing
In the previous chapters, we have learned how to describe and summarize data using measures of central tendency and dispersion. These tools help us understand what a sample of data looks like. But in agricultural research or in any research, we often go a step further, where we want to use the sample to draw conclusions about the larger population. This is where the branch of statistics named as inferential statistics come in to play, and at its core lies the process of hypothesis testing.
12.1 A hypothesis
Scientific research often involves conducting experiments to study natural phenomena or to evaluate the effect of different conditions or practices. In a scientific experiment, observations are made systematically and data are collected to answer specific research questions.
To conduct an experiment in a meaningful way, the researcher must begin with a clear hypothesis. A hypothesis is a tentative and testable statement formulated on the basis of prior knowledge, observations, or scientific reasoning. It provides direction to the study by clearly stating what is to be examined and guides the collection and interpretation of experimental data.
Examples of hypotheses from agriculture:
“Application of nitrogen fertilizer increases the grain yield of rice.”
“Improved seed treatment results in higher germination percentage than untreated seeds.”
“Drip irrigation leads to better water-use efficiency compared to surface irrigation.”
“A new crop variety produces higher average yield than the existing variety.”
These hypotheses can be tested by conducting a properly designed scientific experiment.
ImportantNote
A statistical hypothesis is a specific, testable statement about a population parameter. It expresses an assumption or claim that can be tested using statistical methods based on sample data. Statistical hypotheses are generally formulated in pairs: the null hypothesis (denoted as \(H_0\)), which represents the default or no-effect assumption, and the alternative hypothesis (denoted as \(H_1\) or \(H_a\)), which represents a statement that contradicts the null hypothesis.
The Falsification Principle
The falsification principle, proposed by the philosopher Karl Popper, is a way of distinguishing science from non-science. It suggests that for a theory to be considered scientific, it must be capable of being tested and potentially proven false. For example, the hypothesis “all swans are white” can be falsified by observing a single black swan. According to Popper, science should attempt to disprove a theory, rather than continuously seek evidence that confirms it. (Popper 1959)
This idea forms the philosophical backbone of hypothesis testing in statistics.
12.2 Null and alternative hypothesis
Considering the Popperian Principle of Falsification, we need to translate the working hypothesis into a framework of two competing statements. These are termed the null hypothesis and the alternative hypothesis.
Null hypothesis
The null hypothesis (\(H_0\)) is a statement that there is no effect, no difference, or no relationship between variables. It represents the default or status quo assumption. In hypothesis testing, we assume the null hypothesis is true unless we have strong statistical evidence against it.
Why do we need a null hypothesis?
In an experiment, the results seen in the sample may differ from expectation either because a real effect exists, or simply because of random chance. The null hypothesis begins with the assumption that no real difference or effect exists in the population. Using this assumption, statistical tests examine the sample data and calculate how likely it is to obtain such results purely by chance. This helps us decide whether the observed difference is likely to be real or just a fluke.
For example, suppose a researcher tests a new fertilizer and finds that plants in the sample are 3 cm taller than usual. This difference may be due to the fertilizer or simply to natural variation among plants. By assuming that the fertilizer has no effect (the null hypothesis), a statistical test determines whether the observed difference is likely to have occurred by chance. If it is very unlikely, we reject the null hypothesis and conclude that the fertilizer probably has a real effect.
Alternative hypothesis
The alternative hypothesis (denoted by \(H_1\) or \(H_a\)) is a statement that contradicts the null hypothesis. It proposes that a real effect, difference, or relationship exists in the population and that the observed results are not due to random chance. In most studies, the alternative hypothesis represents the researcher’s expectation or claim that they aim to find evidence to support.
NoteImportant
In statistics, we say “fail to reject” the null hypothesis - not “accept” the null hypothesis. Failing to reject \(H_0\) does not prove that it is true; it only means we do not have sufficient evidence against it.
Definitions
Null hypothesis (\(H_0\)): A statement of ‘no effect’ or ‘no difference’. It is the hypothesis that the researcher attempts to disprove.
Alternative hypothesis (\(H_1\)): The opposite of the null hypothesis. It represents the effect or difference the researcher is looking for.
Example
Not so long ago, people believed that the world was flat. The research question was: Is the Earth flat?
Null hypothesis, \(H_0\): The Earth is flat.
Alternative hypothesis, \(H_1\): The Earth is round.
Several scientists, including Copernicus, set out to disprove the null hypothesis. This eventually led to the rejection of \(H_0\) and the acceptance of \(H_1\).
Example 12.1: A plant biologist wants to test whether different fertilizers affect plant height.
\(H_0\): There is no difference in plant height among different fertilizer treatments.
\(H_1\): There is a difference in plant height among different fertilizer treatments.
Example 12.2: A pharmaceutical company wants to know if a new drug is more effective than the standard treatment.
\(H_0\): The new drug is equally effective as the standard treatment.
\(H_1\): The new drug is more effective than the standard treatment.
12.2.1 Stating a hypothesis
Problem 1: A researcher applies a certain chemical twice a week after flowering on a fruit tree and expects that the average fruit weight per plant is 8 kg.
Step 1: Identify the hypothesis from the problem.
The researcher expects the average fruit weight per plant to be 8 kg. If \(\mu\) (pronounced as ‘mu’) denotes the average fruit weight, the null hypothesis is:
\(H_0\): \(\mu\) = 8
Step 2: State the alternative hypothesis.
If the average fruit weight is not 8 kg, and there is no reason to believe the chemical will increase yield, the only remaining possibility is that the weight is less than 8 kg. So:
\(H_1\): \(\mu\) < 8
But what if the researcher has no prior idea of the direction of the effect?
Problem 2: A researcher is studying the effect of a pesticide chemical on plant yield. He wants to determine whether this chemical has any effect on yield. The chemical could either reduce or boost the yield.
Step 1: Identify the hypothesis.
The researcher expects the change in average yield before and after application of the chemical to be zero. If \(\mu\) denotes this change:
\(H_0\): \(\mu\) = 0
Step 2: Since the researcher does not know whether the chemical will increase or decrease the yield, the alternative hypothesis is:
\(H_1\): \(\mu\) \(\mathbf{\neq}\) 0
12.3 Hypothesis testing
In hypothesis testing, a decision must be made between two alternatives i.e. the null hypothesis and the alternative hypothesis. To make the decision, an experiment is performed, and the null hypothesis is either rejected or not rejected based on a decision rule.
Example 12.3: You have a coin and want to check whether it is biased or unbiased. An unbiased coin has a 50:50 chance of landing heads or tails.
If the coin is unbiased, the probability of obtaining a head is 0.5, i.e., \(p\) = 0.5. The hypotheses are:
\(H_0\): \(p\) = 0.5 (coin is unbiased)
\(H_1\): \(p\) \(\mathbf{\neq}\) 0.5 (coin is biased)
You toss the coin 10 times and note the outcomes, as shown in Table 12.1.
| Toss No. | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 |
|---|---|---|---|---|---|---|---|---|---|---|
| Result | H | H | H | H | T | H | T | H | H | H |
In hypothesis testing, acceptance or rejection of the null hypothesis is based on a decision rule, and this rule is driven by a quantity called the test statistic.
A test statistic is a function of the sample observations that summarizes the sample data into a single numerical value. It is calculated using a specific formula and is used to determine whether the observed sample result is consistent with the null hypothesis. Different statistical tests use different test statistics. For example, the (z)-statistic is used in a (z)-test, the (t)-statistic in a (t)-test, the (^2)-statistic in a chi-square test, and the (F)-statistic in an (F)-test. These test statistics will be discussed in detail in the following chapters.
In this example, suppose you decide to reject \(H_0\) if 8 or more heads occur in 10 tosses, i.e., reject \(H_0\) if \(p \geq \frac{8}{10} = 0.8\).
From the experiment:
Number of heads = 8; Number of tosses = 10
Test statistic: \(p\) = 8/10 = 0.8
Based on the decision rule, you reject the null hypothesis and conclude that the coin is likely biased.
NoteNote
In this example, the decision rule was set informally for illustration. In practice, decision rules are derived from statistical theory, taking into account the sample size and the chosen level of significance. This is discussed in the sections that follow.
12.4 Errors in hypothesis testing
Since hypothesis testing is based on sample data, it is possible to arrive at an incorrect conclusion. Two types of errors can occur, as shown in Table 12.2.
Type I error: Rejecting the null hypothesis when it is actually true.
Type II error: Failing to reject the null hypothesis when it is actually false.
| In fact \(H_0\) is true | In fact \(H_0\) is false | |
|---|---|---|
| Test concludes \(H_0\) is true | Correct Decision | Type II error |
| Test concludes \(H_0\) is false | Type I error | Correct Decision |
Seriousness of Type I and Type II errors
Which error is more serious? The answer depends on the context. Consider the following examples.
Comparing two medications
\(H_0\): \(\mu_1 = \mu_2\) - The two medications are equally effective.
\(H_1\): \(\mu_1 \neq \mu_2\) - The two medications differ in effectiveness.
A Type I error here means concluding the medications differ when they actually do not - perhaps leading to an unnecessary change in treatment. A Type II error means concluding they are equally effective when one is actually better - potentially life-threatening if the inferior medication continues to be prescribed.
Cancer diagnosis
\(H_0\): The patient does not have cancer.
\(H_1\): The patient has cancer.
A Type I error means falsely diagnosing a healthy patient with cancer, leading to unnecessary treatment and distress. A Type II error means missing the diagnosis in a patient who actually has cancer - which could be fatal.
There is an inherent trade-off: reducing Type I error tends to increase Type II error, and vice versa. Think of it this way - a Type I error is like a false alarm: a fire alarm ringing when there is no fire. A Type II error is like a missed alarm: the alarm fails to ring when there actually is a fire.
Consider a person accused of a crime and facing a death sentence.
\(H_0\): He is innocent. \(\quad\) \(H_1\): He is guilty.
A Type I error → an innocent person is executed.
A Type II error → a guilty person is set free.
Most people would agree that wrongly executing an innocent person is the more severe error. This is why many textbooks treat Type I error as more serious.
NoteExam Tip
If an exam question asks which error is more serious, the safe answer is Type I error. But the honest answer is - it depends on the context.
12.5 Level of significance and power of test
Level of significance (\(\alpha\))
The significance level, denoted as \(\alpha\) (alpha), is the probability of rejecting the null hypothesis when it is true. In other words, the level of significance is the probability of committing a Type I error. (Fisher 1925). We fix a small value of \(\alpha\) before the experiment to keep Type I error fixed.
In practice, we fix the Type I error by selecting a suitable value of \(\alpha\) before the experiment, and reduce Type II error by using an adequate sample size. Commonly used values are \(\alpha\) = 0.05 (for agricultural research) or \(\alpha\) = 0.01 (medical research).
Power of test (1 - \(\beta\))
The probability of a Type II error is denoted as \(\beta\) (beta). The quantity \(1 - \beta\) is called the power of the test. Power is the probability of correctly rejecting the null hypothesis when it is false - in other words, the ability of the test to detect a real effect when it exists. (Neyman and Pearson 1933)
Both \(\alpha\) and \(\beta\) play a role in deciding the decision rule for hypothesis testing. Power is primarily determined by the sample size of the experiment: larger samples give more power.
12.6 Region of acceptance and rejection
The test statistic calculated from the sample will follow a probability distribution. For example, in the coin-tossing experiment, the value of the test statistic obtained by person A will differ from that obtained by person B, since both may get different outcomes from the same experiment. The probability distribution of the test statistic is called its sampling distribution.
You will reject the null hypothesis if the test statistic falls in a particular region of the sampling distribution. This region is called the region of rejection (also known as the critical region). The complementary region, where the test statistic leads to non-rejection of \(H_0\), is called the region of acceptance.
- Size of the region of rejection, which is also the probability of type I error = level of significance = \(\alpha\)
- Size of the region of acceptance = \(1 - \alpha\)
The value of the test statistic that separates the region of acceptance from the region of rejection is called the critical value.
12.7 One-tailed and two-tailed tests
The type of alternative hypothesis \(H_1\) determines whether a test is one-tailed or two-tailed.
One-tailed tests
Consider Problem 1 in (state?), where the alternative hypothesis is:
\(H_1\): \(\mu\) < 8
Here, we reject \(H_0\) if the test statistic falls towards the left side of the sampling distribution. This is called a left-tailed test.
If instead the alternative hypothesis were:
\(H_1\): \(\mu\) > 8
We would reject \(H_0\) if the test statistic falls towards the right side. This is called a right-tailed test.
Left-tailed test: Critical region is on the left side of the sampling distribution.
Right-tailed test: Critical region is on the right side of the sampling distribution.
Two-tailed test
Consider Problem 2 from (state?), where the alternative hypothesis is:
\(H_1\): \(\mu\) \(\mathbf{\neq}\) 0
Or consider another example:
\(H_1\): \(\mu\) \(\mathbf{\neq}\) 8
In both cases, the critical region lies on both sides of the sampling distribution. Each tail has an area of \(\alpha/2\), giving a total critical area of \(\alpha\).
12.8 Decision Rule
After calculating the test statistic from the sample, how do we decide whether to reject the null hypothesis?
The decision rule specifies whether the null hypothesis (\(H_0\)) should be rejected based on the calculated value of the test statistic. It depends on the level of significance (()) selected by the researcher and the alternative hypothesis (\(H_1\)), which determines whether the test is right-tailed, left-tailed, or two-tailed.
The chosen significance level (()) determines the critical value(s) that divide the sampling distribution into the rejection region and the non-rejection region. After computing the test statistic, it is compared with the critical value(s) to make a decision.
The decision rules are:
- Right-tailed test: Reject (\(H_0\)) if the calculated test statistic is greater than the critical value.
- Left-tailed test: Reject (\(H_0\)) if the calculated test statistic is less than the critical value.
- Two-tailed test: Reject (\(H_0\)) if the calculated test statistic is less than the lower critical value or greater than the upper critical value.
If the calculated test statistic does not fall in the rejection region, we do not reject (\(H_0\)). This does not prove that \(H_0\) is true; it simply indicates that the sample does not provide sufficient evidence to reject it at the chosen level of significance.
A well-designed statistical test has high power ((1-)), meaning it has a high probability of correctly rejecting a false null hypothesis while controlling the probability of a Type I error at the chosen significance level (()).
12.9 A worked example
Returning to the coin-tossing experiment from (hypo?). Let \(X\) be the number of heads in 10 tosses. Under the null hypothesis (\(p\) = 0.5), \(X\) follows a binomial distribution with \(n\) = 10 and \(p\) = 0.5.
The binomial probability formula is:
\[p(X = x) = \binom{n}{x} p^x q^{n-x} = \frac{n!}{(n-x)!\, x!} \, p^x q^{n-x}\]
The probability distribution of \(X\) under \(H_0\) is given in Table 12.3. This table gives the probabilities of \(X\) when the \(H_0\) is true.
| \(X\) | \(p(x)\) |
|---|---|
| 0 | 0.001 |
| 1 | 0.010 |
| 2 | 0.044 |
| 3 | 0.117 |
| 4 | 0.205 |
| 5 | 0.246 |
| 6 | 0.205 |
| 7 | 0.117 |
| 8 | 0.044 |
| 9 | 0.010 |
| 10 | 0.001 |
For a two-tailed test at \(\alpha\) = 0.05, each tail should have an area of \(\alpha/2\) = 0.025. From Table 12.3:
- \(P(X \leq 1)\) = 0.001 + 0.010 = 0.011
- \(P(X \leq 2)\) = 0.011 + 0.044 = 0.055
The left critical value is approximately \(X\) = 2, and by symmetry the right critical value is \(X\) = 8. This means: if the number of heads is less than 2 or more than 8, reject \(H_0\) at \(\alpha\) = 0.05.
In our experiment, we obtained 8 heads. Since 8 equals - but does not exceed - the critical value, we fail to reject \(H_0\) at \(\alpha\) = 0.05. We do not have sufficient evidence to conclude the coin is biased.
Commonly used test statistics are \(t\), \(F\), \(Z\), and \(\chi^2\) (chi-square). Critical values of these statistics are available in standard statistical tables and will be used in later chapters.
12.10 Confidence Interval
A confidence interval (CI) is a range of values that is likely to contain the true value of a population parameter. The confidence level of the interval is 100(1 - ())%, where () is the level of significance.
For example:
- If (= 0.05), the confidence level is 95%.
- If (= 0.01), the confidence level is 99%.
A 95% confidence interval means that if the same study were repeated many times and a confidence interval were calculated from each sample, about 95% of those intervals would contain the true population parameter. Similarly, a 99% confidence interval would contain the true parameter in about 99% of repeated studies.
Note: A common misconception is that “if an experiment is repeated 100 times, the same conclusion will be obtained exactly 95 times.” This is not the correct interpretation. The confidence level refers to the proportion of confidence intervals that would contain the true population parameter when the study is repeated many times.
In simple terms, a 95% confidence interval is a range of plausible values for the true population parameter based on the sample data.
12.11 p-value
Traditionally, hypothesis testing involved comparing the calculated test statistic with a critical value obtained from statistical tables. If the test statistic exceeded the critical value (or fell beyond it in the rejection region), the null hypothesis was rejected.
Today, statistical softwares automatically calculates the p-value along with the test statistic. As a result, researchers rarely use printed statistical tables to make decisions. Instead, they compare the p-value directly with the chosen level of significance (()).
The p-value provides a more informative and convenient approach because it indicates the smallest level of significance at which the null hypothesis would be rejected. Therefore, modern statistical analysis is primarily based on the p-value rather than on critical values.
The p-value is the probability of obtaining a test statistic as extreme as or more extreme than the one observed in the sample, assuming that the null hypothesis is true.
A small p-value indicates that the observed result is unlikely to have occurred by random chance alone if the null hypothesis is true. Therefore, it provides strong evidence against the null hypothesis. On the other hand, a large p-value indicates that the observed result is reasonably consistent with the null hypothesis, and there is not enough evidence to reject it.
The decision rule based on the p-value is simple:
- If p-value (), reject the null hypothesis (\(H_0\)).
- If p-value (> ), do not reject the null hypothesis (\(H_0\)).
Thus, the p-value helps us decide whether the sample provides sufficient evidence to reject the null hypothesis at the chosen level of significance.
12.12 Steps in hypothesis testing
The following eight steps summarize the process of hypothesis testing.
Step 1: State the null hypothesis (\(H_0\)).
Step 2: State the alternative hypothesis (\(H_1\)).
Step 3: Set the level of significance (\(\alpha\)).
Step 4: Collect data from the experiment or other scientific methods.
Step 5: Calculate the test statistic.
Step 6: Identify the critical value (or critical region) for the specified \(\alpha\).
Step 7: Compare the calculated test statistic with the critical value.
Step 8: If the calculated value falls in the critical region, reject \(H_0\) with 100(1 - \(\alpha\))% confidence. Otherwise, state that there is insufficient evidence to reject \(H_0\).
CautionHistorical Insights
The Lady, the Tea, and the Birth of Hypothesis Testing
It was an otherwise unremarkable summer afternoon at an agricultural research station in Cambridge, England, in the 1920s. A group of colleagues sat down for tea. When statistician Ronald Fisher poured a cup and offered it to his colleague Dr. Muriel Bristol, she politely declined. She preferred tea into which the milk had been added first, she explained - and she could tell the difference just by tasting.
Fisher was sceptical. “Nonsense,” he reportedly said. “Surely it makes no difference.” But Bristol was adamant. A third colleague, William Roach, suggested they settle the matter with an experiment.
Fisher prepared eight cups of tea - four with milk added first, four with tea added first - and presented them to Dr. Bristol in a random order. She was told that four cups had been prepared each way, and her task was to identify them. She correctly identified all eight.
Fisher used this experiment in his landmark 1935 book The Design of Experiments to introduce the concept of the null hypothesis and significance testing. The null hypothesis was that Dr. Bristol had no ability to tell the difference and was merely guessing. Fisher showed that the probability of correctly identifying all eight cups by chance alone was only 1 in 70, or about 1.4% - well below any reasonable threshold of significance. (Fisher 1935)
This charming episode gave the world a framework that now underlies every clinical trial, every agricultural experiment, and every scientific study that draws a conclusion from data. The next time you test a hypothesis in your research, you are following in the footsteps of a tea party in Cambridge.
TipQuotes to Inspire
“To consult the statistician after an experiment is finished is often merely to ask him to conduct a post mortem examination. He can perhaps say what the experiment died of.” - Sir Ronald A. Fisher