Research is meant to be a bridge from a question to an answer. But what happens when half of your participants fall off the bridge before they reach the other side? Attrition bias occurs when the loss of participants is not random. If your study is too long, too boring, or too difficult, you will lose a specific type of person, usually the busiest or the most frustrated.
Bias is a term often heard in research and statistical circles, indicating the presence of systematic error in a study. While many forms of cognitive bias can impact the validity and reliability of a study’s findings, attrition bias is one of the most common yet overlooked types.
What Is Attrition Bias
At its simplest level, attrition bias occurs when the people who leave a study (the “attriters”) are systematically different from the people who stay.
Looking for research help?
Research Prospect to the rescue then!
We have expert writers on our team who are skilled at helping students with their research across a variety of disciplines. Guaranteeing 100% satisfaction!
In a perfect world, if 10% of people leave a study, they would be a random mix. You would lose a few tall people, a few short people, a few optimists, and a few pessimists. The “flavour” of your group would not change. This is called attrition at random. It is annoying because you have less data, but it does not necessarily ruin your results.
Attrition Bias is the “Not-at-Random” version. It happens when there is a specific reason related to the study that causes certain types of people to vanish. When this happens, the final group of participants no longer represents the original group. The “survivors” are a biased sample, leading to “Survivor Bias“, a close cousin of attrition bias that paints a dangerously rosy picture of reality.
Attrition Bias Definition
A type of selection bias caused by the unequal loss of participants (dropout) from the groups being compared in a study. If the people who leave have different characteristics than those who stay, the final results will be biased and unrepresentative of the original group.
Categories Of Attrition Bias
To understand how this ruins a research paper or a multi-million dollar business strategy, we have to look at why people actually leave. It usually falls into a few “red flag” categories:
1. The “Too Hard” Factor
This is common in medical and psychological research. If a new medication has a side effect, say, extreme nausea, the people who experience that nausea will quit the trial. The people who do not feel sick will stay. When the researchers analyse the data at the end, they might conclude the drug is “perfectly tolerated” because nobody in the final group complained of nausea. They have accidentally filtered out the very evidence they needed to find.
2. The “It’s Not Working” Factor
In educational studies or self-improvement programs, people who do not see immediate progress often lose motivation and stop responding to surveys. If a study is measuring the effectiveness of a new tutoring method, and only the students who “got it” stuck around to take the final exam, the tutoring method will look like a miracle cure for low grades, even if it failed 80% of the class.
3. The Socio-Economic Filter
Research often requires time, internet access, or transportation. If a long-term study requires participants to drive to a clinic every week, people with low incomes, unreliable cars, or inflexible jobs will drop out at higher rates. The final results will then only reflect the experiences of a privileged demographic, but the study might be incorrectly applied to the general population.
Why Is Attrition Bias A Problem
You might be thinking, “Okay, so a few people left the study. Is it really that big of a deal?” In the world of evidence-based decision-making, it is a massive deal.
It is because attrition bias:
- Creates false positives
- Compromises internal validity
- Misleads consumers
It Creates “False Positives”
Attrition bias almost always makes an intervention look more effective than it actually is. It hides the failures and highlights the successes. If you are a policy maker deciding whether to fund a new social program based on a biased study, you might waste millions of dollars on a program that only helps the “easy” cases while failing the people who need it most.
It Compromises Internal Validity
Internal validity is the “truth” of a study, the confidence that Variable A actually caused Result B. When attrition kicks in, you can no longer be sure if the results happened because of your intervention or because your group’s composition changed mid-way through.
It Misleads Consumers
We see this in marketing every day. “9 out of 10 people recommend this skincare routine!” (What they don’t tell you is that 500 people tried it, 490 got a rash and stopped using it, and they only surveyed the 10 who liked it).
How To Spot Attrition Bias
Whether you are reviewing an academic paper or looking at your own business metrics, there are three primary questions you should ask to sniff out this bias:
- What was the “N” at the start vs. the end? (N = number of participants). If a study starts with 500 people and ends with 150, your “bias alarm” should be ringing loudly.
- Was there a “Baseline Comparison” for dropouts? A good researcher will compare the characteristics of the people who left with the people who stayed. If the dropouts had lower baseline test scores or higher health risks than the stayers, the results are compromised.
- Why did they leave? If the paper just says “participants were lost to follow-up” without explaining why, they might be glossing over a major flaw in the study design.
How To Fix Attrition Bias
Researchers have developed a few clever ways to deal with the “ghosts” in their data. If you’re writing about this or conducting your own research, you’ll want to be familiar with these terms:
Intent-to-Treat (ITT) Analysis
This is the gold standard, especially in clinical trials. In an ITT analysis, you include everyone who started the study in the final results, regardless of whether they finished or even took the medication.
If someone drops out, you treat their result as a “failure” or use their last known data point. This provides a “real-world” view of how effective a program is. It’s honest. It says, “In the real world, people forget their pills or get bored, and our results reflect that reality.”
Multiple Imputation
This sounds fancy, but it’s basically “educated guessing” backed by high-level statistics. If a participant drops out, researchers use the data from similar participants who stayed to predict what the missing person’s data would have looked like. It’s not perfect, but it’s often better than just pretending those people never existed.
Attrition Bias Example In The Digital Age
Think about mobile apps. A developer launches a new app and sees that “Average Session Time” is 20 minutes. They think, “Wow, people love our app!” But what if 90% of people who download the app delete it within 30 seconds because the interface is confusing? The only people left are the “power users” who spent 20 minutes figuring it out. The developer is looking at a “survivor” metric. If they don’t account for the 90% who left (the attrition), they will never fix the onboarding issue that is killing their business.
Similarly, in customer satisfaction surveys, you usually only hear from the people who are extremely happy or extremely angry. The “silent middle” who just stopped using your service and moved on to a competitor represents a form of attrition bias. If you only listen to the survey results, you’re missing the biggest part of the story.
How To Minimise Attrition In Your Own Work
If you are designing a project, a blog series, or a study, your goal is to keep the “N” as high as possible. Here are a few “human” ways to prevent attrition before it starts:
- Reduce Participant Burden: Don’t ask people to fill out a 50 question survey every day. They will quit. Make it easy.
- Offer Incentives: Sometimes a small “thank you” or a gift card at the end of a study is the difference between someone finishing or ghosting.
- Build a Relationship: In long-term research, keeping in touch with participants through newsletters or updates makes them feel like part of a team, rather than just a data point.
- Be Transparent: Tell people exactly how long it will take and what is expected. Surprise “work” is a leading cause of attrition.
Frequently Asked Questions
Attrition bias occurs when participants drop out of a longitudinal study over time, potentially skewing results. For instance, in a study examining a new drug’s effects, if those experiencing side effects disproportionately drop out, the drug may seem safer than it actually is, leading to biased conclusions.
Attrition bias arises from participants dropping out of a study over time, potentially skewing results. Exclusion bias occurs when certain individuals or data are systematically excluded from analysis, leading to non-representative results. While both can affect a study’s validity, attrition relates to dropout, and exclusion concerns initial data selection.
Yes, attrition can cause bias. When participants drop out of a study, and their dropout is related to the study’s outcome or exposure, it can skew the results, leading to inaccurate conclusions. This non-random loss of participants, if unaccounted for, can compromise the study’s validity and generalisability.
In randomised controlled trials (RCTs), attrition bias occurs when there is a differential loss of participants between comparison groups, which can distort the true effect of the intervention. If participants drop out due to side effects or lack of efficacy, and they are not analysed, results may not accurately represent the intervention’s effect.
