In probability judgments, ignoring the base rates can lead to errors called the base rate fallacy. This occurs when people fail to consider the base rates while assessing the likelihood of an event or attribute happening.
Instead, they rely on specific information about a case, causing them to overlook the fundamental probabilities derived from a broader population. According to the base rate fallacy, this error can result in incorrect assumptions and flawed decision-making.
To demonstrate the importance of base rates, consider this classic example:
- 1% of a population have a particular disease.
- A medical test can detect the disease with 90% accuracy.
- A randomly selected individual tests positive for the disease.
Ignoring base rate (1% prevalence) and only considering the test’s accuracy (90%) might lead someone to assume that the individual has a 90% chance of having the disease. However, this overlooks the fact that the disease is relatively rare and that the test could yield false positives in some cases. By properly taking into account the base rate and test accuracy, the actual probability of the individual having the disease is significantly lower.
A base rate is the fundamental probability of an event or attribute occurring within a population. It serves as a baseline or prior probability for making more specific statistical inferences.
In simple terms, base-rates represent the likelihood of an event happening without taking into account any extra information. For example, if 20% of people in a city are vegetarians, the base rate of being a vegetarian in that city would be 20%.
Base rates play a crucial role in the field of statistics and probability. They serve as the foundation for assessing likelihoods in various contexts, such as medical diagnoses, financial investment decisions, and social stereotypes.
The Base Rate Fallacy in Probability Judgement
The base rate fallacy, also called base rate neglect or base rate bias, and prosecutor’s fallacy or defense attorney’s fallacy when applied to the results of statistical tests (such as DNA tests) in the context of law proceedings, is a cognitive bias wherein people tend to ignore or underweight the base rate information when making probability judgments.
Some key features of the base rate fallacy:
- Occurs when people focus on specific information (usually recent or vivid) and disregard the general statistical information
- Base rates are essential for accurate probability assessment, yet are often ignored or underweight
- Contributes to an increased likelihood of false positives in certain situations
While base rate fallacy can occur in various contexts, it typically arises in situations involving medical diagnoses, legal judgments, and other decision-making processes where relevant statistical information is available.
Base Rate Fallacy Examples
Jury Decisions and Evidence Evaluation
The base rate fallacy can have significant consequences on legal decisions made by a jury. In the context of a courtroom, base-rate information refers to the prevalence of a certain characteristic in the general population. However, juries tend to disregard this information, which results in the misinterpretation of evidence.
For example, let’s assume that in a certain criminal case, a jury is presented with evidence that only a small fraction (e.g., 1%) of the population carries a rare genetic marker. The defendant is found to be part of this 1%.
The jury may fall into the base rate fallacy and assume this evidence is highly incriminating. However, they might overlook the fact that in a city of one million people, there would be 10,000 individuals with the same genetic marker. The rarity of the genetic marker alone does not suffice for inferring guilt.
False Positives in Legal Testing
Another legal implication of the base rate fallacy concerns the interpretation of test results, particularly in regards to false positives. Legal testing often relies on tests with low false positive rates, which may not necessarily convey accurate information when base rates are not considered.
For example, consider drug testing. Assume that a drug test has a false positive rate of 5% and a true positive rate of 95%. If 1,000 employees are tested and 10% of them are drug users, the results would look like this:
- True Positives: 95 individuals (95% of 100 drug users)
- False Positives: 45 individuals (5% of 900 non-drug users)
- Total Positives: 140 individuals (95 true positives + 45 false positives)
In this scenario, of the 140 individuals who tested positive, only 95 were actually drug users, meaning that the test would correctly identify drug users in only 67.9% of the cases. While the test’s false positive rate is low (5%), the proportion of false positives amongst the total positives is relatively high, and base rates have a significant impact on the interpretation of test results.
Or, suppose a crime is committed. According to forensic analysis, the assailant possesses a blood type that is prevalent among 10% of the general population. It is discovered that the suspect arrested has the identical blood type.
A prosecutor may charge the suspect solely on that basis and assert during the trial that there is a 90% chance that the defendant is culpable. Nevertheless, this deduction is merely marginally accurate if the defendant was identified as the primary suspect on the basis of substantial evidence uncovered earlier than the blood test and unrelated to it.
Otherwise, the reasoning presented is flawed, as it overlooks the high prior probability (that is, prior to the blood test) that he is a random innocent person. Assume, for instance, that 1000 people live in the town where the crime occurred.
This means that 100 people live there who have the perpetrator’s blood type, of whom only one is the true perpetrator; therefore, the true probability that the defendant is guilty – based only on the fact that his blood type matches that of the killer – is only 1%, far less than the 90% argued by the prosecutor.
The prosecutor’s fallacy involves assuming that the prior probability of a random match is equal to the probability that the defendant is innocent. When using it, a prosecutor questioning an expert witness may ask: “The odds of finding this evidence on an innocent man are so small that the jury can safely disregard the possibility that this defendant is innocent, correct?”
The claim assumes that the probability of finding evidence on an innocent man is the same as the probability of a man being innocent given that evidence was discovered on him, which is not true.
While the former is usually small (10% in the previous example) due to good forensic evidence procedures, the latter (99% in that example) does not directly relate to it and is often much higher, because it is dependent on the likely quite high prior odds of the defendant being a random innocent person.
Employment
Suppose an employer is trying to hire engineers and lawyers. They receive ten applications, with nine from engineers and only one from a lawyer.
If the employer bases their decision on individual merits without considering the base rate (9:1), they may commit a base rate fallacy by overlooking the greater likelihood of hiring an engineer based on the application pool.
Cognitive Biases and Heuristics
Heuristics are mental shortcuts that help individuals make quick decisions and judgments. They are essential in simplifying the complex information processing tasks that human beings face daily. However, relying on heuristics can sometimes lead to systematic errors known as cognitive biases.
The psychologists Daniel Kahneman and Amos Tversky have conducted extensive research on heuristics and their related cognitive biases. Some of the most well-known heuristics are the representativeness heuristic and the availability heuristic. These mental shortcuts can lead to biases such as the base rate fallacy.
Bayesian Reasoning in Statistics
Bayesian reasoning is a method in statistics that uses probability to model uncertainty in the world. The key idea behind Bayesian reasoning is to update our beliefs about events as we gather more data.
The foundation of Bayesian reasoning is Bayes’ theorem, which states that the probability of an event occurring, given specific information, is equal to the product of the probability of the event and the probability that the specific information occurs, divided by the total probability that the specific information occurs.
In mathematical terms, Bayes’ theorem is represented as:
P(A|B) = P(B|A) * P(A) / P(B)
where P(A|B) is the probability of event A occurring given event B has occurred, P(B|A) is the probability of event B occurring given event A has occurred, P(A) is the probability of event A occurring, and P(B) is the probability of event B occurring.
A crucial aspect of Bayesian probability is the concept of the prior probability, which represents our initial belief about an event before we have any data or new information.
As we collect more data, we update our beliefs by applying the posterior probability, which is the probability of the event after taking new data into account. This updating process continues as more data becomes available, allowing us to refine our predictions and make better-informed decisions.
The base rate fallacy is a classic application of Bayesian reasoning. By disregarding the base rate, people may misjudge the actual probability of an event occurring. Bayesian reasoning helps us to correctly update our beliefs and predictions by taking both the base rate and the specific information into account.
Experimental Findings
Experiments have revealed that when individuating information is available, people prefer it over general information.
In one study, students were asked to estimate the grade point averages (GPAs) of fictitious students in various experiments. Students tended to dismiss relevant statistics about GPA distribution when provided descriptive information on a specific student, even if the new descriptive information was obviously of little or no significance to school performance. This finding has been used to imply that interviews are a waste of time in the college admissions process since interviewers cannot choose successful candidates better than basic statistics.
There is much dispute in psychology about the circumstances under which people value or dislike base rate information. The heuristics-and-biases program’s researchers have emphasized empirical data demonstrating that people neglect base rates and make inferences that violate certain probabilistic reasoning rules, such as Bayes’ theorem.
This line of research led to the conclusion that human probabilistic thinking is essentially defective and error-prone. Others have stressed the relationship between cognitive processes and information formats, claiming that such findings are not generally warranted.
References:
- Bar-Hillel, Maya (1980). The base-rate fallacy in probability judgments. Acta Psychologica. 44 (3): 211–233. doi:10.1016/0001-6918(80)90046-3
- Barbey, Aron & Steven Sloman (2007). Base-rate respect: From ecological rationality to dual processes. Behavioral and Brain Sciences, 30(3):241–254
- Fenton, Norman; Neil, Martin; Berger, Daniel (June 2016). Bayes and the Law. Annual Review of Statistics and Its Application. 3 (1): 51–77. doi: 10.1146/annurev-statistics-041715-033428
- Kahneman, Daniel; Amos Tversky (1973). On the psychology of prediction. Psychological Review. 80 (4): 237–251
- Koehler JJ. (1996) The base rate fallacy reconsidered: Descriptive, normative, and methodological challenges. Behavioral and Brain Sciences. 19 (1) 1-17. doi:10.1017/S0140525X00041157
- (2001) Deterrence and the base rate fallacy: An examination of perceived certainty, Justice Quarterly, 18:1, 63-86, DOI: 10.1080/07418820100094821
- Nisbett, Richard E.; E. Borgida; R. Crandall; H. Reed (1976). Popular induction: Information is not always informative. In J. S. Carroll & J. W. Payne (ed.). Cognition and social behavior. Vol. 2. John Wiley & Sons, Incorporated. ISBN 0-470-99007-4.
- Tversky, A.; Kahneman, D. (1974). Judgment under Uncertainty: Heuristics and Biases. Science. 185 (4157): 1124–1131. doi: 10.1126/science.185.4157.1124
- Welsh, Matthew B.; Navarro, Daniel J. (2012). Seeing is believing: Priors, trust, and base rate neglect. Organizational Behavior and Human Decision Processes. 119 (1): 1–14. doi:10.1016/j.obhdp.2012.04.001