Training pigeons to become as dumb as humans
There’s an interesting series of studies from Edmund Fantino, Inna Kanevsky, Shawn Charlton, J Hartl, A Goodie, and D Case about heuristics in human decision-making. They find a task at which pigeons are actually better than humans, because they don’t have the same biases. Then they managed to train the pigeons to show the same biases as the humans. The studies are about base-rate neglect.
What is base rate neglect? (skip down if you already know)
To explain base-rate neglect, let’s say that in a certain sort of situation, one of two mutually exclusive events occurs, and generally event A happens with X% probability, event B with Y% probability (those probabilities are the “base rates”). Your job is to determine which event happened in some particular situation. There is also evidence left behind which helps you decide which event happened (for instance, event A may cause E to happen with a 5% chance, but event B has a 20% chance of causing E; so if you see evidence E, that pushes you in the direction of thinking that event B happened).
The logically optimal thing to do in these sorts of situations is to crunch the numbers using Bayes’ rule; there’s an equation which, when given all of the relevant probabilities and evidence, will tell you whether event A or B is more likely to have occurred.
But what humans often actually do is underestimate the effect of the base rates, and rely too much on the immediate evidence. To see why this is stupid, imagine that you’re coming home at the end of a cloudy day — you think it may have rained but you were inside at work all day and you’re not sure. When you get home you find that the window is open and your carpet is all wet next to the window. Your roommate accidentally leaves the window open about 50% of the time, so you get mad at him for leaving it open and letting the rain in. But he says, “Wait a minute man, yeah, okay, maybe I left the window open, I really don’t remember, and maybe it rained, I didn’t notice, but before you get all mad at me, consider this: we both agree there’s about a 50% chance of it raining on a day like this. So even if i HAD left the window open, there’s only a 50% chance that it would cause the carpet to get wet. But let’s say that wet, slimey aliens came through the window to look at our house during the day. If that had happened, there would be a 100% chance of causing the carpet to get wet. So, the evidence (the wet carpet) really supports the slimey aliens hypothesis better than your I-left-the-window-open hypothesis. So it probably wasn’t my fault”.
What’s wrong with your roommate’s argument? Base rate neglect. Sure, slimey aliens would be more certain to get the carpet wet than leaving the window open. But the prior probability of slimey aliens coming in on any particular day is much, much lower than the prior probability of your roomate leaving the window open by accident. The huge difference in the priors outweighs the fact that the aliens would be more certain to cause the evidence that you see.
Any adult human can see what’s wrong with the slimey aliens excuse, although they probably wouldn’t phrase it in terms of “base rate neglect”. But here’s a similar situation that humans tend to get wrong.
1% of women at age forty who participate in routine screening have breast cancer. 80% of women with breast cancer will get positive mammographies. 9.6% of women without breast cancer will also get positive mammographies. A woman in this age group had a positive mammography in a routine screening. What is the probability that she actually has breast cancer?
I won’t give away the answer here in case you want to work it out yourself. Please see this link for the correct answer, as well as a detailed explanation of how to get it. If you get it wrong, don’t feel bad; so do 85% of doctors (heh heh, I bet now you feel even worse, but for a different reason).
OK so what’s the experiment with the pigeons?
On each trial, first they showed the pigeons a picture of either a vertical or a horizontal line (they call this the “sample”). Then they gave the pigeons two choices; peck at a button with a picture of a vertical line, or peck at a button with a picture of a horizontal line. If they picked the vertical line, there was a 75% chance of getting a reward; if they picked the horizontal line, there was a 25% chance of getting a reward. The sample was a red herring; it didn’t give you any information about the right answer. The optimal strategy is always to pick the vertical line.
This is the task at which pigeons are better than humans. Pigeons do pretty close to the optimal thing. Humans choose the choice that matches the sample about 55% of the time (Goodie and Fantino (1995), but that’s not online; this article summarizes on pages 18-19).
The researchers hypothesize (on page 19 of the previous link) that this is because human have been trained over years and years to do tasks in which matching is important. In other words, for this task, humans are “overeducated”:
…from early childhood humans learn to match like colors and shapes at home, at play, and at school
(e.g., in playing with blocks and puzzles and in reading picture books with their parents).
Pigeons, on the other hand, have not experienced a rich history of matching.
They test this hypothesis by training the pigeons to do a matching task, and then they have them redo the original task (creating “overeducated” pigeons; Fantino has a funny cartoon picture in his talk but I couldn’t find it online). And indeed, they report that these pigeons do about the same thing that humans do, choosing the choice that matches the same about half the time.
So, they have succeeded in educating the pigeons until they become as stupid as us.