There are lies, damn lies - and statistics.
To help you prepare for the many statistical reasoning questions that you will likely encounter on test day, we provide a primer on statistical reasoning similar to what you would get in a college-level introductory statistics class.
1. The Biased Sample Fallacy
The Fallacy of the Biased Sample is committed whenever the data for a statistical inference is drawn from a sample that is not representative of the population under consideration. The data drawn and used to make a generalization is drawn from a group that does not represent the whole. Here is an argument that commits the fallacy of the biased sample:
A recent study showed that over 60% of Oregon residents watched cartoons. Based on this study, executives at Cartoon Channel spent $10 million to expand their access to Oregonians, who appear to be avid fans of cartoons.
Note that this survey doesn't say anything about the specific Oregon residents polled. Are they school children? The results would seem to indicate this. A sample must be representative of the overall population that wants to be studied in order to make a general conclusion.
Here is another example:
In a recent survey conducted by Wall Street Weekly of its readers, 80% of the respondents indicated their strong disapproval of increased capital gains taxes. This survey clearly shows that increased capital gains taxes will meet with strong opposition from the electorate.
The data for the inference in this argument is drawn from a sample that is not representative of the entire electorate. The survey was conducted of just people who invest, and not random members of the electorate. People who read about investing are more likely to have an opinion on the topic of taxes on investment different than the population at large.
2. The Insufficient Sample Fallacy (Hasty Generalization/Sweeping Generalization)
The Fallacy of the Insufficient Sample is committed whenever an inadequate sample is used to justify the conclusion drawn. In a Biased Sample, people are pulled from a non-representative group, in an Insufficient Sample, not enough people are polled to make a statistically significant result.
I have worked with three people from New York City and found them to be obnoxious, pushy and rude. It is obvious that people from New York City have a bad attitude.
Observations of three people are not sufficient to support a conclusion about 10 million. Bad luck could account for meeting three bad people. Try this one:
After living and working in New York City for 12 years, I have met thousands of people and with very rare exception, I have found them to be obnoxious, pushy and rude. It is obvious that people from New York City have a bad attitude.
This latter argument is something to take more seriously given the larger pool from which the observation is drawn.
3. Correlation does not prove causality
A correlation is a statistical linking between two items that seems to be parallel. One of the LSAT's Greatest Hits that you seem time and time again is the attempt to link up two separate items that seem to statistically correlate and then establish one of the two as the "cause".
The relation between an association and a cause is difficult.
- Heavier people tend to be taller.
- Weight is correlated with height.
- Gaining weight will make you taller.
This assumes a relationship between correlated data where if you change one element you can change the other one.
Another obvious one:
- More fire trucks tend to be at more serious fires
- We can reduce the severity of fires by reducing the fire trucks.
Here is a more challenging example:
- Young people who watch more TV violence are more likely to engage in violence.
- The recent increase in TV violence is associated with an increase in violence society-wide.
- If children would watch less TV, they would be less violent.
This one seems intuitive enough and it's the "sentimental favorite", but the reality is that (3) can't be proven from 1 and/or 2. You can't assume that just because things correlate you can change one factor and it will automatically change the other. Children who watch large amounts of TV may have inattentive parents, and this may be the underlying hidden causal factor (not watching too much TV violence in itself). This argument could use more evidence, like a study showing that violent children are more successfully rehabilitated by cutting off violent shows.
Studies have shown that men aged 18-27 who have owned a pet for at least 2 years before marrying are 35% less likely to divorce. Researchers conclude that caring for a pet prepares men for long-term, healthy relationships in marriage.
Which of the following, if true, most strengthens the conclusion that men who have owned pets are prepared for healthy marriages?
A. Studies have shown that pet ownership drastically reduces daily stress levels.
B. Many successful marriages are based on emotional investment in a common interest, such as a pet.
C. Many men who have been married for 25 years or more continue to own pets.
D. Men who have not owned pets for at least two years before marrying are more likely to divorce.
E. Men whose wives who owned a pet for at least two years are equally as unlikely to divorce.
Situation: Researchers have concluded that men who have owned a pet for at least 2 years are prepared for healthy marriages.
Reasoning: Which option most strengthens the conclusion? Researchers base their conclusion on an assumed connection between sustained care for a pet and care for a spouse. Men who care for pets before marriage, the argument runs, are also statistically more likely to sustain marriage relationships. The problem is that correlation doesn't prove causality, so that link alone is not enough.
A. While this may be true, it does not introduce additional evidence to support the conclusion.
B. This option does not address the question of why men who own pets are less likely to divorce.
C. The question concerns men who have owned pets before marrying, not after.
D. Correct. This option provides additional evidence of a causal correlation between pet ownership and the likelihood of divorce.
E. The question concerns men, not their wives.
The correct answer is D.
4. Confounding Factors (also called the "Lurking Variable")
A confounding factor is an additional factor that may be responsible for a correlation. "Con" is a latin root for "with", so confounding means literally to found with.
Example 1: The Miracle Hospital
A sports injury treatment center in New York has the lowest rate of recovery for sports injuries. A treatment center in rural Pennsylvania has the highest and quickest recovery rate. If you have just been severely injured while playing softball, should you go to Pennsylvania?
In this example it appears pretty obvious that this hospital in New York is bad for your health. So you follow the statistics and go to Pennsylvania, right? The treatment center in New York is an option of last resort for serious sports injury patients like you. The Pennsylvania hospital is so poor that no one with a serious injury ever goes there. The hospital's patients consist of those with minor injuries who recover quickly. The hidden confounding factor in this argument is that people with more severe injuries are choosing to go to New York, meaning that looking at their injuries is a biased sample.
Example 2: The Secret Conspiracy Against Men
At University of California at Berkeley, the school had a much lower acceptance rate for men than for women and administrators could not determine why since the male applicants had higher SAT scores and grades.
Are the lower admissions rates of men a result of systematic bias?
Looking at the information, it appears that someone in the admissions department doesn't like men and has been secretly rejecting their applications.
When looking more carefully at the data, men were much more likely to apply to the highly-competitive engineering program. The result was that men had lower rates of admission overall at the Berkeley. In non-engineering programs, however, the acceptance rates were identical. So gender played no direct role in admissions rates, the factor was the major chosen by the applicants.