Statistical insights into politics

As most people are aware, the beta distribution has the interesting property that there are instances when more data is collected and the result of collecting this additional information is, surprisingly, that the variance of the distribution actually goes up.

This is a rare property: virtually all distributions have the consistency property in which the more you know the smaller the variance. Consider any polling venture: the larger the sample, the smaller the variance in the estimate. This aligns with most people's intuition: if you want to be more sure about something, collect more information, get a larger sample, increase the number of observations. But this may not help if you're working with the beta distribution.

According to this report, the Kerry campaign appears to have the "beta effect" on voters.

In the latest NBC/Wall Street Journal poll, "just 57 percent of the respondents say they know a lot or a fair amount about Kerry," reports NBC's Mark Murray. That's "a real drop from 68 percent in the NBC/Journal March survey." The voters actually know less about Kerry the more the campaign progresses. It's working! At this impressive rate of memory loss, most of the electorate won't even recognize Kerry's name on the Nov. 2 ballot.

Should the polls take larger samples? Perhaps that will only increase how uncertain they are about the increasing uncertainty of voters.

The beta distribution may actually be a realistic model of the political process -- the more we learn about anything the less certain we are that we understand it.