1. Multiple Testing Adjustment
Importance: ★
Ease: LL
What it means
- Every time you run a statistical test, you accept the risk of being wrong.
- If you accept P = 0.05, you are accepting a 5% chance of a false positive (Type I error).
- If you run many tests, each one brings its own 5% error → the overall chance of a wrong conclusion increases.
Why adjustment is needed
Because with multiple tests, the probability of making at least one false-positive conclusion becomes large.
How adjustment works
Multiple testing adjustment tightens the P-value threshold to keep the overall false-positive rate at 5%.
Most common method
- Bonferroni adjustment
- Divide 0.05 by the number of tests.
- Example: 5 tests → new threshold = 0.05/5 = 0.01
2. One- and Two-Tailed Tests
Importance: ★
Ease: L
Two-tailed test
- Used most of the time.
- You reject the null hypothesis if:
- the new treatment is better
- the new treatment is worse
- You check both “tails” of the distribution.
OR
One-tailed test
Used rarely.
- Only used when the ONLY meaningful direction is:
- “Is the new treatment worse?” (or only “better”, but that is extremely rare in clinical work)
- One-tailed tests can make a non-significant two-tailed result suddenly look significant → be sceptical.
3. Incidence
Importance: ★★★★
Ease: LLLL
Definition
Number of new cases of a condition in a population during a defined time period, expressed as a percentage (or rate).
Example
A practice of 1000 patients → 15 new cases of Brett’s palsy this year:
- Incidence = (15/1000) × 100 = 1.5% per year
Key point
- Used for risk over time
- Falls in chronic diseases
- Increases in conditions with rapid onset/outbreaks
4. Prevalence
Importance: ★★★★
Ease: LLLL
Definition
Number of existing cases at a single point in time, as a percentage.
Example
At the study moment:
- 90 patients have Brett’s palsy in a 1000-patient practice.
Prevalence = (90/1000) × 100 = 9%
Key point
- Chronic diseases: prevalence >> incidence
- Short illnesses (e.g., colds): incidence is high, but point prevalence low
5. Power
Importance: ★★
Ease: LLL
Definition
Power is the probability that the study will detect a statistically significant difference if a real difference actually exists.
Why it matters
If a study is too small:
- It may miss real differences
- Results may be “non-significant” simply due to low sample size
Example
- If the expected difference between treatments is huge (e.g., 100% cure vs 0%), a small sample has enough power.
- If the expected difference is tiny (e.g., 1%), you need a very large sample to have enough power.
6. Bayesian Statistics
Importance: ★
Ease: L
Key idea
Bayesian statistics incorporate:
- Prior knowledge/opinion (previous studies, clinical experience)
- New data from the study
The two combine to produce a posterior distribution (updated belief after seeing the new data).
Why it is different
- Classical (frequentist) statistics: uses only the sample at hand
- Bayesian: combines prior + new sample
Example (conceptual)
A clinician believes Drug A is highly effective based on prior studies.
A new study is added.
Bayesian analysis blends:
- Prior belief (weighted numerically)
- New study data
→ produces updated probabilities.
Caution
- Different researchers may choose different priors → results differ.
- Only recently feasible due to computing power.
SUMMARY TABLE
Concept | Meaning | Key Takeaway |
Multiple Testing Adjustment | Controls false positives when doing many tests | Bonferroni most common |
One- vs Two-Tailed Tests | Two-tailed tests look for difference in either direction | Be sceptical of one-tailed tests |
Incidence | New cases over time | Used for “risk over time” |
Prevalence | Existing cases at a point in time | Chronic diseases have high prevalence |
Power | Chance a study detects true difference | Low power → false negative |
Bayesian Statistics | Combines prior knowledge + new data | Reflects real clinical thinking |