The Very Basics of a Statistical Test
There are multiple types of statistical tests, but they all share same basic elements, which are random variables, distribution, null/alternative hypothesis and critical value, and concepts that are generated from these elements, e.g., Type I error, Type II error, significance level,α,confidence level, p-value, effect size, β and power of the test.
Assuming there are two types (100 acre each) of soybean lands, one received fertilizer (experiment groups, G1) and the other did not (control groups: G0), all other factors are the same. We want to compare the soybean yields of these two types of lands. Assuming variance of yields of G0 and G1 are equal. Thus, we could leverage two sample t test with equal variance to test whether the mean of yields of two groups are the same.
The null hypothesis of this test is that the difference of yield y1-y0 follows the following distribution:
Mean: mu_1 — mu_0
Var: Var_1 + Var_0
Std: Var/N (the number of observations in each group)
assumption: (mu_1 — mu_0)/std follows t distribution with 2*(N-1) degree of freedom
(The details of why y1-y0 follows the following distribution will not be discussed here).
