This content originally appeared on DEV Community and was authored by MakenaKinyua
Errors in Hypothesis testing.
In hypothesis testing, we conduct statistical tests in order to determine the validity of our tests at a specific level of significance.
We start by setting the null and alternate hypothesis. The alternate hypothesis is what we observe from an experiment and the null hypothesis is the opposite of the alternate hypothesis.
In the process, there is always a chance of encountering errors when it comes to rejecting or failing to reject the null hypothesis. With these errors, there is a question of how to balance the errors and what we are willing to trade off.
There are two types of errors:
Type i error – occurs when we reject null when we should not have rejected it.
Type ii error – occurs when we fail to reject the null hypothesis when we should have rejected it.
Medical Dilemma : A Cancer Scenario
A patient walks into a hospital with several signs and symptoms. The doctor suspects that the patient’s symptoms are consistent with cancer.
In this case:
H0 : The patient does not have cancer
H1 : The patient has cancer
Is it better for a type i or a type ii error to occur? Where do we trade off between the two and how do we decide which one is better?
Type i error
A type i error would be that we reject null; that the patient does not have cancer, when we should not have rejected it.
The implication:
- The patient is in turn put on cancer treatments such as chemotherapy, a draining treatment, when they are in fact healthy.
- It leads to physical, mental and financial drain on the patient.
Type ii error
A type ii error would mean that we fail to reject null; that the patient does not have cancer, when we should have rejected it.
The implication:
- The patient would be sent home thinking that they are not sick when in fact, they are sick.
- They do not receive any sort of care and might end up having a sudden decline in health which might lead to death.
Reflection
So which one is more acceptable in this case? A false positive or a false negative? How can we balance between the two?
- A type i error is reduced by setting an acceptable alpha value for the hypothesis test.
- A type ii error is reduced by increasing the statistical power.
It is important to strike a good balance between the two because both extremes can be dangerous.
I say we need to consider our priorities, our morality, the cost of each type of error and the effects in the long run. What are your thoughts?
This content originally appeared on DEV Community and was authored by MakenaKinyua