Making Smarter Decisions With Data: A Simple Guide to Hypothesis Testing

What is Hypothesis Testing?

Hypothesis testing is a statistical method used to determine whether a sample of data provides sufficient evidence to support or reject a claim about a population.

Think of it as a courtroom:

• The Null Hypothesis (H₀) is the default assumption (“no change”, “no difference”).
• The Alternative Hypothesis (H₁) is what you want to prove.

Your data acts as the evidence.

Statistics helps you objectively decide whether that evidence is strong enough to reject H₀.

It helps answer key questions like:

• Is the new medicine effective?
• Has customer satisfaction improved?
• Is the coin fair?
• Has the population changed over time?

How Does It Work?

Every hypothesis test follows four core steps:

1. Define H₀ and H₁

Example:

H₀: The new onboarding process does not improve retention.
H₁: The new onboarding process does improve retention.

2. Choose a significance level (α)

Most commonly 0.05 (5% chance of making a wrong decision).

3. Run the appropriate test

Depending on the data, analysts use:

• Z-tests (large samples)
• t-tests (small samples)
• Chi-square tests (categorical data)
• ANOVA/F-tests (3+ group comparisons)

4. Make a decision using p-value or critical value

• If p ≤ 0.05, the result is statistically significant → Reject H₀
• If p > 0.05, there isn’t enough evidence → Fail to reject H₀

This approach ensures decisions are guided by evidence rather than bias.

Key Concepts You Must Know

1. Null & Alternate Hypothesis

• H₀: “Nothing has changed.”
• H₁: “Something has changed.”

Example from the slides:

A college claims average pass percentage = 85%.
Sample average = 90%.
We test whether this new result truly shows improvement.

2. Significance Level (α)

Normally 0.05 (5%) or 0.01 (1%).
This represents the maximum risk you’re willing to take to reject a true H₀.

3. Test Statistic (Z or t)

A formula-based number that tells you how far your sample result is from the claim.

4. Critical Value

A threshold from statistical tables.
If your test statistic crosses it → Reject H₀.

5. P-Value

The most important concept in modern statistics.

It tells you:

“How likely is my data if H₀ is true?”

Lower p-value → stronger evidence against H₀.

Decision rule:

• p ≤ α → Reject H₀
• p > α → Fail to reject H₀

Z-Test Explained (Large Samples | σ Known)

Use when:

• Sample size ≥ 30
• Population standard deviation (σ) is known
• Data is normally distributed

⭐ Example

Population mean = 168 cm
Sample mean = 169.5
n = 36
σ = 3.9

Compute Z-value using:

You compare this value with critical Z = ±1.96 (95% confidence).

If |Z| > 1.96 → Reject H₀

Student’s t-Test (Small Samples | σ Unknown)

Used when:

• n < 30
• Population standard deviation unknown
• Replace σ with sample SD (s)

Example:

IQ sample
μ = 100
= 140
s = 20
n = 30
df = n – 1 = 29
Critical t = ±2.045 (95%)

Calculated t = 10.96 → Reject H₀

This means the medication significantly affects IQ.

Type I & Type II Errors

Very important for:

• Medical testing
• Machine learning (confusion matrix)
• Quality control

Chi-Square Test (Categorical Data)

Used when comparing observed vs expected frequencies.

Weight Distribution 2010 vs 2020

Calculated χ² = 26.66
Critical χ² = 5.991
Since 26.66 > 5.991 → Reject H₀

Conclusion: Weight distribution changed significantly over 10 years

ANOVA (Analysis of Variance)

Used to compare 3 or more group means.

Key Concepts:

• Factors: Variables (e.g., medication)
• Levels: Categories (e.g., 5 mg, 10 mg, 15 mg)

 Example:

Three medication dosages (15 mg, 30 mg, 45 mg)

Computed F = 43.46
Critical F = 3.5546
Since F > critical F → Reject H₀

Meaning: At least one dosage works better than others.

 Confidence Interval & Margin of Error

CI provides a range, not just a single estimate.

Example (CAT exam):

• Mean = 520
• σ = 100
• n = 25
• Z = 1.96

CI = (480.8, 559.2)

Meaning: With 95% confidence, the population mean lies in this range.

Conclusion

Hypothesis testing is the core of statistical decision-making.

From Z-tests to ANOVA, every method helps you answer one question:

“Is my observed data significantly different from what I expected?”

Using hypothesis testing, businesses improve decisions, researchers validate claims, and analysts turn raw data into meaningful insights.

Blog Contributed By Dr Prerna Singh