Unrepresentative Sample

If the means of collecting the sample from the population are likely to produce a sample that is unrepresentative of the population, then a generalization upon the sample data is an inference committing the fallacy of unrepresentative sample. A kind of hasty generalization. When some of the statistical evidence is expected to be relevant to the results but is hidden or overlooked, the fallacy is called suppressed evidence.

Example:

The two men in the matching green suits that I met at the Star Trek Convention in Las Vegas had a terrible fear of cats. I remember their saying they were from Delaware. I've never met anyone else from Delaware, so I suppose everyone there has a terrible fear of cats.

Most people's background information is sufficient to tell them that people at this sort of convention are unlikely to be representative, that is, typical members of society.

Large samples can be unrepresentative, too.
Example:

We've polled over 400,000 Southern Baptists and asked them whether the best religion in the world is Southern Baptist. We have over 99% agreement, which proves our point about which religion is best.

Getting a larger sample size does not overcome sampling bias.

Small Sample

This is the fallacy of using too small a sample. If the sample is too small to provide a representative sample of the population, and if we have the background information to know that there is this problem with sample size, yet we still accept the generalization upon the sample results, then we commit the fallacy. This fallacy is the fallacy of hasty generalization, but it emphasizes statistical sampling techniques.

Example:

I've eaten in restaurants twice in my life, and both times I've gotten sick. I've learned one thing from these experiences: restaurants make me sick.

How big a sample do you need to avoid the fallacy? Relying on background knowledge about a population's lack of diversity can reduce the sample size needed for the generalization. With a completely homogeneous population, a sample of one is large enough to be representative of the population; if we've seen one electron, we've seen them all. However, eating in one restaurant is not like eating in any restaurant, so far as getting sick is concerned. We cannot place a specific number on sample size below which the fallacy is produced unless we know about homogeneity of the population and the margin of error and the confidence level.

Hasty Generalization

A hasty generalization is a fallacy of jumping to conclusions in which the conclusion is a generalization. See also Biased Statistics.

Example:

I've met two people in Nicaragua so far, and they were both nice to me. So, all people I will meet in Nicaragua will be nice to me.

Converse Accident

If we reason by paying too much attention to exceptions to the rule, and generalize on the exceptions, we commit this fallacy. This fallacy is the converse of the accident fallacy. It is a kind of Hasty Generalization.

Example:

I've heard that turtles live longer than tarantulas, but the one turtle I bought lived only two days. I bought it at Dowden's Pet Store. So, I think that turtles bought from pet stores do not live longer than tarantulas.

The original generalization is "Turtles live longer than tarantulas." There are exceptions, such as the turtle bought from the pet store. Rather than seeing this for what it is, namely an exception, the reasoner places too much trust in this exception and generalizes on it to produce the faulty generalization that turtles bought from pet stores do not live longer than tarantulas.