Selection bias occurs when researchers select participants for a study in a way that is not random, potentially leading to results that are not representative of the larger population. This bias in sampling can significantly affect the outcome of the research, as it can introduce systematic differences between the selected group and those not chosen.
Researchers need to be aware of this risk, as selection bias can compromise the validity of a study. The distortion arising from selection bias may result in an overestimation or underestimation of the true relationships between variables.
Particularly in observational studies, like case-control studies, cohort studies, and cross-sectional studies, certain groups may be overrepresented or underrepresented, which impacts the study’s conclusions.
Types and Sources of Selection Bias
The various forms of this bias, each with its unique impact on research, are essential for researchers to identify and mitigate.
Self-Selection Bias
With self-selection bias, the participants have the autonomy to decide whether to participate. This can result in a sample that holds certain characteristics that are significantly different from the parent population.
Sampling Bias
Sampling bias is systematic error due to a non-random sample of a population, causing some members of the population to be less likely to be included than others, resulting in a biased sample, defined as a statistical sample of a population (or non-human factors) in which all participants are not equally balanced or objectively represented. It is mostly classified as a subtype of selection bias, sometimes specifically termed sample selection bias, but some classify it as a separate type of bias.
One difference between sampling bias and selection bias is that sampling bias undermines a test’s external validity (the ability of its results to be generalized to the rest of the population), whereas selection bias primarily addresses internal validity for differences or similarities found in the sample at hand. faults in the process of acquiring the sample or cohort generate sampling bias, whereas faults in any subsequent procedure cause selection bias.
Examples of sampling bias include self-selection, pre-screening of trial participants, discounting trial subjects/tests that did not run to completion and migration bias by excluding subjects who have recently moved into or out of the study area, length-time bias, where slowly developing disease with better prognosis is detected, and lead time bias, where disease is diagnosed earlier participants than in comparison populations, although the average course of disease is the same.
Self-Selection Bias
Self-selection bias, where the participants have the autonomy to decide whether to participate in studies, poses an additional challenge to study validity because these participants may have intrinsically different characteristics from the study’s target population. Research has shown that volunteers tend to come from a higher social standing than from a lower socio-economic background.
Another study found that women are more likely than men to volunteer for research. Volunteer bias can be found throughout the study’s life cycle, from recruitment to follow-up.
Attrition Bias
Attrition bias is a type of selection bias produced by participant attrition, which discounts trial subjects/tests that did not run to completion. It is closely connected to the survivorship bias, which includes only subjects who “survived” a procedure, or the failure bias, which includes only subjects who “failed” a process.
Dropout, nonresponse (lower response rate), withdrawal, and protocol deviators are all included. It produces biased outcomes when exposure and/or outcome are unequal.
For example, in a dieting program study, the researcher may simply dismiss everyone who drops out, although the majority of people who drop out are individuals for whom the program was not working. Different losses of people in the intervention and comparison groups may influence the characteristics and results of these groups regardless of the researched intervention.
Recall Bias
Recall Bias (also called observer bias) is a systematic error caused by differences in the accuracy or completeness of the recollections retrieved by participants regarding events or experiences from the past. It can sometimes also be referred to as response bias, responder bias or reporting bias.
Recall bias is a sort of measurement bias that can arise in research employing interviews or questionnaires. It may result in misclassification of numerous types of exposure in this scenario. Recall bias is especially problematic in case-control studies used to examine the etiology of a disease or psychiatric condition.
In studies of risk factors for breast cancer, for example, women who have had the disease may search their memory more deeply for possible causes of their cancer than members of the unaffected control group. Those in the case group (those with breast cancer) may be able to recall a greater number of potential risk factors than those in the control group (those who have not been diagnosed with breast cancer). This selection effect may overstate the link between a putative risk factor and the disease.
Nonresponse Bias
Nonresponse bias, also known as participation bias, happens when the results of elections, research, polls, and other similar events become unrepresentative because the participants disproportionately possess particular characteristics that influence the outcome. These characteristics indicate that the sample population is systematically different from the target population, which may result in skewed estimates.
For instance, one study found that those who refused to answer a survey on AIDS tended to be “older, attend church more often, are less likely to believe in the confidentiality of surveys, and have lower sexual self disclosure.” Non-response bias can be a problem in longitudinal research due to attrition during the study.
Consequences of Selection Bias
Selection bias has several far-reaching consequences that can seriously undermine the credibility and utility of research findings. Firstly, it can lead to a distortion of relationships. When selection bias occurs, the observed relationship between variables may not represent the true association.
This form of bias primarily impacts internal validity, as it questions the cause-and-effect relationship within the study group. For instance, if a study sample is not representative due to selection bias, any conclusions drawn about causal relationships within that group may be inaccurate.
In terms of external validity, selection bias raises concerns about the generalizability of study results. If the participants are not a good representation of the larger population, then the findings cannot be confidently applied to other groups.
The presence of selection bias can also inflate the risk of drawing incorrect conclusions. In medical research, for example, the perceived effectiveness of a treatment could be misleading if the study doesn’t accurately reflect the population’s diversity.
Selection bias may be involuntary, but it requires researchers to be vigilant during the research design and implementation of studies to minimize its effects.
Minimizing Selection Bias
Selection biases cannot be overcome solely through statistical analysis of available data. Examining correlations between exogenous (background) factors and a treatment indicator can help determine the degree of selection bias.
However, in regression models, it is the link between unobserved determinants of outcome and unobserved determinants of sample selection that biases estimates, and this correlation between unobservables cannot be detected simply by the observed determinants of treatment.
Randomization
Randomization is the cornerstone of a scientifically rigorous study. It involves assigning participants or samples to various groups using a random process. This method ensures that each participant has an equal chance of being assigned to any given group, which helps to balance out known and unknown factors that could lead to selection bias. For example, in clinical trials, randomization can help ensure that treatment and control groups are comparable at baseline.
Study Design
Effective study design is essential for preventing selection bias. Researchers should clearly define inclusion and exclusion criteria and adhere to these guidelines strictly. They should also consider using blinding methods, where participants and experimenters are unaware of the group assignments.
Cross-disciplinary strategies are also useful, as they may create robust frameworks capable of reducing bias across various types of studies.
Data Collection
During data collection, selection bias occurs when the sample does not accurately represent the population. For instance, an internet survey on health behaviors might unintentionally exclude individuals without internet access, skewing results since the sample is not representative of the general population.
Cherry picking of data, which is actually a confirmation bias, rather than selection bias, occurs when certain subsets of data are picked to support a conclusion (e.g. mentioning incidents of plane crashes as evidence of airline flight safety while omitting the considerably more prevalent example of flights that complete safely).
Active efforts to minimize nonresponse and missing data, such as follow-up with participants, are essential in reducing the potential for bias. The goal is to ensure a sample that accurately reflects the entire population, thus maintaining the validity of the data collection process.
Sampling Methods
Selection bias can be prevented or minimized through careful sampling methods. Probability sampling is a technique wherein every member of the population has a known non-zero probability of being selected.
Simple random sampling, a type of probability sampling, further reduces selection bias by giving every individual in the population an equal opportunity to be included in the sample. When a random sample is not feasible, researchers may employ other methods such as stratified or cluster sampling to approximate a random distribution and control for bias.
Research Oversight
Research oversight mechanisms, such as Institutional Review Boards (IRBs) or ethics committees, play a pivotal role in counteracting both selection effect and exclusion bias. They ensure that studies adhere to ethical standards by reviewing the methodologies for potential biases and evaluating the measures in place for informed consent.
Especially in observational studies, where researchers observe subjects in their natural settings without manipulation, proactive measures must be taken to ensure that the results are not skewed due to a non-representative sample or systematic differences between groups being compared.
References:
- Ards, Sheila; Chung, Chanjin; Myers, Samuel L. (1998). The effects of sample selection bias on racial differences in child abuse reporting. Child Abuse & Neglect. 22 (2): 103–115. doi:10.1016/S0145-2134(97)00131-2
- Cortes, Corinna; Mohri, Mehryar; Riley, Michael; Rostamizadeh, Afshin (2008). Sample Selection Bias Correction Theory. Algorithmic Learning Theory. Lecture Notes in Computer Science. Vol. 5254. pp. 38–53. arXiv:0805.2775. doi: 10.1007/978-3-540-87987-9_8
- Fadem, Barbara (2009). Behavioral Science. Lippincott Williams & Wilkins. ISBN 978-0-7817-8257-9
- Jüni, P.; Egger, Matthias (2005). Empirical evidence of attrition bias in clinical trials. International Journal of Epidemiology. 34 (1): 87–88. doi:10.1093/ije/dyh406
- Kopec, JA; Esdaile, JM (September 1990). Bias in case-control studies. A review. Journal of Epidemiology and Community Health. 44 (3): 179–86. doi:10.1136/jech.44.3.179
- Slonim, R; Wang, C; Garbarino, E; Merrett, D (2013). Opting-in: Participation bias in economic experiments. Journal of Economic Behavior & Organization. 90: 43–70. doi:10.1016/j.jebo.2013.03.013
- Tripepi, G., Jager, K. J., Dekker, F. W., & Zoccali, C. (2010). Selection bias and information bias in clinical research. Nephron. Clinical practice, 115(2), c94–c99.
- Turner, Heather A. (1999) Participation bias in AIDS‐related telephone surveys: Results from the national AIDS behavioral survey (NABS) non‐response study, The Journal of Sex Research, 36:1, 52-58, DOI: 10.1080/00224499909551967
- Winship, Christopher, and Robert D. Mare. Models for sample selection bias. Annual review of sociology 18, no. 1 (1992): 327-350.