Are All the Marbles Black?

A very simple model of confirmation holds that a general law is confirmed by its positive instances; for example, the theory that all ravens are black is supported by each new black raven we find.

We can find some support for this intuitively plausible idea in Bayes's Theorem. Under normal circumstances, positive instances will in fact support a hypothesized generalization. (But there are exceptions, which Bayes' Theorem will also help us to understand.)

It can be tedious to do the calculations needed to apply Bayes' Theorem to particular cases on a calculator or by hand. To make things simpler, we will use a little Bayesian calculator: once we enter the necessary input values, we can just press a button to reach the posterior probability.

The calculator (a Java applet) appears below. (It probably will not work correctly unless you have Sun's Java Plugin installed; you can get that here. Just click the "get it now" button.)

Suppose someone tells you all the 500 marbles in a bag are black. You haven't seen any of them, and you are agnostic to begin with about whether this is true are not. Suppose you also know that the marbles were purchased at Joe's Marble Store, where they only sell two kinds of bags of marbles: all-black bags, and bags of multicolored marbles of which 20% are black. You draw out a marble and look at it: it's black. How should this affect your belief in the generalization that all the marbles in the bag are black?

Let's say that P(T) is .5, the same as P(~T), representing your initial agnosticism. P(E|T) is 1, since if all the marbles in the bag are black, then it must be the case that the particular marble you have just examined is black. We only need one further value, namely P(E|~T). This is trickier, and in fact unless you have some further background information, it is not clear exactly how you could assign a value to it. However, given our background knowledge about Joe's Marble Shop, we would expect that if ~T is the case, then the bag contains 20% black marbles, so the probability of observing a black marble is .2.

With this as background, try the following experiments.

Experiment 1: The initial marbles example.

Just plug in the values from the example above, and see what you get!

The calculator is just solving the following equation: P(T|E) = (1 * .5)/((1 * .5) + (.2 * .5)), and, if my own calculations are correct, it should come up with .83.

Wow! That went up by a lot. But it's reasonable that it should, because the odds of getting a black marble unless the bag was all-black were rather low.

Experiment 2: Drawing a second marble.

Consider what happens if you draw a second marble. This time your prior probability, P(T), should be the posterior probability from Experiment 1.

(Here's a slightly subtle question. Do you need to change any of the input values other than the prior probability? Remember that whereas the bag formerly held 500 marbles, it now contains only 499, since you removed one. That shouldn't make much difference in this case -- try it both ways and see! -- but it is an important point to keep in mind. We can ignore this complication if we assume that after every drawing, we replace the marble in the bag.)

Experiment 3: What if P(E|~T) were higher?

Let's change the example a little. Suppose the marble bags at Joe's Marble Shop were either all-black or 80% black. (Maybe Joe really likes black a lot.) Try the calculation for these assumptions and notice how much different the effect of a black marble on the posterior probability is! Why?

Morals

So we observe two things about confirmation by positive instances, on this model. First, under normal circumstances, positive instances will confirm a generalization, as we might expect. Second, the extent to which a positive instance confirms a generalization depends on the situation, and especially on how likely the positive instance is if the generalization is not true.