The Unz Review • An Alternative Media Selection$
A Collection of Interesting, Important, and Controversial Perspectives Largely Excluded from the American Mainstream Media
 BlogviewJames Thompson Archive
Electoral Behaviour: Lurking Factors and Visible Causes
Email This Page to Someone

 Remember My Information


Bookmark Toggle AllToCAdd to LibraryRemove from Library • B
Show CommentNext New CommentNext New ReplyRead More
ReplyAgree/Disagree/Etc. More... This Commenter This Thread Hide Thread Display All Comments
These buttons register your public Agreement, Disagreement, Thanks, LOL, or Troll with the selected comment. They are ONLY available to recent, frequent commenters who have saved their Name+Email using the 'Remember My Information' checkbox, and may also ONLY be used three times during any eight hour period.
Ignore Commenter Follow Commenter
Search Text Case Sensitive  Exact Words  Include Comments
List of Bookmarks

I have no idea what you will be thinking or doing on 12th December, but efforts are being made to determine how UK citizens will vote on that day. Why the fuss? A rational approach to elections is to read the party manifestos, judge the personal and societal impact of the proposals, calculate the probability of the promises being kept, and then vote in advance by postal ballot. That done, the good citizen has no need of reading the news, watching debates, or even waiting in a queue on election day. Certainly, the rational citizen has no need of opinion polls: they are not relevant to informed decision-making.

In a further show of rationality, the wise elector will not stay up on election night, since the event cannot be influenced by watching it. Wiser still, the elector should not check the results for a few days, to avoid wasting time with dangling chads, recounts and other frivolities.

Sadly, the current culture of instant gratification requires a daily feed of speculation about how the election is going, that is, how other people are thinking about which way they might vote on election day, and whether they are likely to vote at all. That provides hours of comment. Then, in a meta-analytic frenzy, commentators discuss how much the opinion polls themselves are causing citizens to change their intended behaviours, as they realize that they are in a minority or, conversely, in such a majority that they might be giving too much power to the most popular party. Others, feeling exposed at being revealed to be in a minority, switch to the most popular party, just as others are abandoning it.

Opinion polls took off in the US in the 1920s. Gradually pollsters learned about the effects of selection bias. If you only poll the readers of your newspaper, you leave out the majority who don’t read it, and certainly those that don’t read any newspapers. Sampling is crucial. However good your sample, you can never include those who don’t want to be sampled. Different avenues of approach lead to different types of voter. Telephones (remember those?) only catch older people at home, and that may be only 20% of home numbers dialed.

To keep calm and avoid election fever, I have been reading a book on statistics. The best statistics books are those one does not admit reading. They explain complex matters in simple terms, and one is forever grateful, without admitting it.

David Spiegelhalter. The Art of Statistics: Learning from data. Random House, 2019.

This is a good book, with helpful explanations which concentrate on key concepts, not specific formulae. Brian Everitt, one of the developers of cluster analysis, always said to me that it was a deficiency of statistics that when you asked a statistical question you got a number instead of an answer. “Yes” or “no” are generally the answers one is searching for.

Far from bringing solace, the book discussed the problem of using opinion polls to predict election results, using the UK election of 2017 as a prime example of the shortcomings of conventional techniques, which failed to spot a late surge for the Labour party, leading to a hung Parliament, and a precarious working majority derived from a cobbled together Conservative coalition.

Can statisticians do better in 2019?

Spiegelhalter’s book is worth reading because he poses interesting questions, and answers them without numbers (or at least, without too many complicated numbers in the first instance). His aim is to get you to think straight about problems, and to solve them in a systematic way, leaving the number and calculation issues till later. Think hard, plan carefully, and then you can let the (properly selected and presented) numbers do the talking. OK, numbers don’t talk, but if you have thought things through, then you can explain the findings in ordinary language.

For example, has a nice family doctor been murdering his patients? How does one estimate normal death rates in medical practices? What counts as an excess? Anything else worth measuring? Is time of the day the death occurs something worth counting?

Spiegelhalter shows in a simple figure (page 5) that Dr Harold Shipman’s victims disproportionately died in the afternoons, when he did his home visits, and administered a lethal opiate overdose to at least 215 elderly patients. As Spiegelhalter dryly observes: “The pattern does not require sophisticated statistical analysis.”

Spiegelhalter’s approach is immensely sensible. He shows that statistics require careful thinking, and only then some number crunching, followed by an honest depiction of the findings. He is a good guide to statistics, particularly for those who panic at the sight of mathematical notation. He is good at explaining (yet again) the difference between relative and absolute risk; the distorting effects of question framing (in the UK 57% supported “giving 16 and 17 year olds the right to vote” but only 37% agreed with the logically identical proposal to “reduce the voting age from 18 to 16”; the distorting effects of telephone polls which do not declare what proportion of telephone numbers dialled never answered; he explains that a causal link does not mean that every single person shows an effect (many people smoke and don’t get cancer, but smoking causes more people who smoke to get cancer than those who don’t smoke, and some non-smokers get cancer); don’t rely on a single study, instead review all studies systematically; some potential causes can be called “lurking factors” but actual causes follow the Bradford Hill criteria (page 115): effect size so large it cannot be explained by plausible confounding; appropriate temporal or spatial proximity in that cause precedes effect and effect occurs after a plausible interval, or at the same site; effect increases as exposure increases, and reduces upon reduction of the dose; there is a plausible mechanism of action; the effect fits in with what is known already; the effect replicates; and the effect is found in similar but not identical studies.

He is excellent at explaining regression to the mean (two thirds of the apparently beneficial effect of speed cameras are due to regression to the mean); at discussing the bias/variance trade-off (over-fitting predictors to correct “bias” and reflect local circumstances at the cost of lower reliability); and in accusing algorithms of prejudice when predicting whether criminals will re-offend, notions of justice are being favoured over predictive accuracy.

There is much to recommend in this book. He covers a wide range of statistical issue with clarity, particularly on probability, so Chapters 8 and 9 are worth buying the book for. I will probably refer further to this book in subsequent posts.

What does this mean for the UK election on 12 December? I will try to explain. The current YouGov snapshot shows the following voting intentions, and what they are likely to mean in terms of parliamentary seats.

UK elections are carried out in 650 constituencies, and use the “first past the post” system, in which the winner is the one with most votes, and becomes the Member of Parliament, and all the other votes contribute nothing to the election. Harsh but effective, like the race between spermatozoa.

Most constituencies vote solidly one way or the other, so can almost be counted in advance, and are ignored by campaigners who concentrate on the “marginals” where a change of winner is possible. The constituencies are somewhat biased towards urban centres, and are usually smaller, giving city voters an advantage. This is not democratic, but the parties have not agreed to a proper reform of the boundary setting process. Currently cities tend to vote Labour. After the election the political parties can do some horse-trading. In 2017 the Conservatives were short of seats and borrowed them, at significant cost, from the Unionists in Ulster, with whom they have traditional links. This time if the Labour Party do not win they are likely to seek support from the Scottish Nationalists (who don’t want to be part of the Union anyway) and the Liberal Democrats, who say they won’t work with Labour, but the prospect of power will make them pause for only a few minutes. So, it is possible for the Conservatives to win the popular vote (which is irrelevant) and gain the largest number of seats under their own banner, but still lose power if the other minority powers decide, after the election, on a coalition of convenience in which they bury their differences to thwart the winner.

Because of these complications, pollsters can only attempt to predict voting intentions, and then need to use some interesting statistics to predict what it will mean for each constituency. They use available demographic and social data at constituency level to estimate how much the national figures will vary in each constituency, and therefor what the likely winner will be. There are several ways of doing this, and different groups have come up with somewhat different results.

Into this mix we must put that large tribe, about one tenth of voters, who don’t yet know how they will vote. They will possibly get inspiration as they look at the ballot sheet. Animal charities may benefit. They could jump any way, or stay at home if it is cold and rainy. Indeed, all voters could stay at home. Furthermore, all the polling data is always a day or two late, and changes of mind happen at the moment of voting.

Plus, all the predictions come with a margin of error of 3 points. Spiegelhalter recommends doubling that to 6 points to account for samples which are too small, sampling approaches which leave out particular groups, questions which are too leading, polling which is too late to detect surges of support (in 2017 there was a late Labour surge which was missed by pollsters) and plain ordinary human cussedness. Those factors, plus idiosyncratic marginal constituency three-way races complicate the picture, even before you factor in deliberate tactical voting, make this a hard election to call. The current state of the polls may be deceiving, and already out of date as the crunch vote hoves into view. Yet, it seems likely at the moment that the Conservatives will scrape in with a working majority. With any luck, that will spare us from elections until late 2024.

Vote early, vote often.

Hide 24 CommentsLeave a Comment
Commenters to FollowEndorsed Only
Trim Comments?
  1. I can’t speak for the UK, but I have been popular for telephone polls for the last couple of years. I do not recall one that has asked multiple questions in one question. For example, on a scale of 1-5 do you support a named party’s platform of A, B, C, and D. I confuse the pollster by saying, you’ve asked me four questions. They still don’t understand that there are multiple possible answers if I do not support all, or none, of the 4 elements in the platform, so I tell them I have no answer.
    I am fully aware that there are not a lot of people like me, and that most will answer positively even if they only support one of the 4. This also skews polls.
    The British Parliamentary system was developed to keep power out of the public’s hands, and keep it in the hands of the elite. The elite is not necessarily the aristocracy, and most times is not.

    • Replies: @dc.sunsets
  2. dearieme says:

    Welcome back, Doc. I was beginning to worry.

    “the rational citizen has no need of opinion polls: they are not relevant to informed decision-making”: I beg to differ. In our seat the only likely winner is the candidate of either (i) the anti-semitic socialist party led by a sausage-dodger with a six letter surname, or (ii) The Illiberal Antidemocrats.

    I reckon I must hold my nose and vote for the less repulsive option: either the Commie-Nazi or the Euro-Quisling. Now: could opinion polls alter my assessment of the possible winner? That is the question.

  3. @Curmudgeon

    The elite is not necessarily the aristocracy, and most times is not.

    These days, it’s surely almost the polar opposite of a natural aristocracy. Nobel-Prize-Winning economist F.A. Hayek (author of The Road to Serfdom) wryly observed that, “in a democracy, the scum rises to the top.”

    From Hoppe’s brief description of his book, Democracy: The God that Failed.

    Theoretically speaking, the transition from monarchy to democracy involves no more or less than a hereditary monopoly “owner” (the prince or king) being replaced by temporary and interchangeable monopoly “caretakers” (presidents, prime ministers, and members of parliament). Both kings and presidents will produce bads, yet a king, because he “owns” the monopoly and may sell or bequeath it, will care about the repercussions of his actions on capital values.

    As the owner of the capital stock on “his” territory, the king will be comparatively future-oriented. In order to preserve or enhance the value of his property, he will exploit only moderately and calculatingly. In contrast, a temporary and interchangeable democratic caretaker does not own the country, but as long as he is in office he is permitted to use it to his advantage. He owns its current use but not its capital stock. This does not eliminate exploitation. Instead, it makes exploitation shortsighted (present-oriented) and uncalculated, i.e., carried out without regard for the value of the capital stock.

    Nor is it an advantage of democracy that free entry into every state position exists (whereas under monarchy entry is restricted by the king’s discretion). To the contrary, only competition in the production of goods is a good thing. Competition in the production of bads is not good; in fact, it is sheer evil. Kings, coming into their position by virtue of birth, might be harmless dilettantes or decent men (and if they are “madmen,” they will be quickly restrained or if need be, killed, by close relatives concerned with the possessions of the dynasty).

    In sharp contrast, the selection of government rulers by means of popular elections makes it essentially impossible for a harmless or decent person to ever rise to the top. Presidents and prime ministers come into their position as a result of their efficiency as morally uninhibited demagogues. Hence, democracy virtually assures that only dangerous men will rise to the top of government.

    • Replies: @Curmudgeon
  4. As I understand it, no serious school of political science credits voters with any agency at all. If this is remotely so, polling is just Kabuki Theater.

    The approach of Public Choice Theory sheds some light on this, discussing “rational ignorance” as a voter strategy, the ubiquity of log-rolling in legislation and such.

    Orwell’s book-within-a-book, The Theory and Practice of Oligarchical Collectivism holds more insight into today’s political realm than any poll or statistical reduction of current events.

    In the end, I believe it’s impossible to simultaneously have huge nation-states and political management of nearly 100% of every aspect of life. What emerges from this is constant warfare as innumerable factions attempt to force their peculiar vision on everyone for every question of life.

    We sail toward a storm. At least in the USA, the chasm between people’s beliefs and preferences is now so wide and deep that sharing a single polity is increasingly a powder keg. I suspect the UK is no different, just much, much smaller.

  5. Pity elections pay no attention to past proven incompetence, malice & lies. Thus, the Tories are poised for another victory…it really reinforces your faith in the wisdom of “people”

    • Replies: @dearieme
  6. dearieme says:

    the Tories are poised for another victory

    How do you know that?

    • Replies: @MJB
  7. Realist says:

    Electoral Behaviour: Lurking Factors and Visible Causes

    The strangest behaviour of the electorate is that any of them believe their vote makes a difference.

    • Replies: @dearieme
  8. dearieme says:

    Those people who cast 50 votes might reasonably hope that they might make a difference once-in-a-lifetime. Even more those who cast 100.

    • Replies: @Realist
    , @obwandiyag
  9. Realist says:

    Those people who cast 50 votes might reasonably hope that they might make a difference once-in-a-lifetime. Even more those who cast 100.

    My point is both parties are wholly owned subsidiaries of the Deep State…so it doesn’t matter how many times you vote.

    • Replies: @dearieme
  10. On this side of the pond our statistics bible is this book:

    _Every_ “scientific study” uses it!

    • Replies: @Reg Cæsar
  11. dearieme says:

    My point is both parties are wholly owned subsidiaries of the Deep State

    Could be. But if Boris actually does deliver something reasonably recognisable as Brexit there’s every chance the Deep State will be fuming.

    I suppose there is a chance that the Commie/Nazi party is owned by a Deep State – but which? Cuban? Venezuelan?

    • Replies: @Realist
  12. @dearieme

    The Great McGinty!

    • Replies: @dearieme
  13. Realist says:

    Could be. But if Boris actually does deliver something reasonably recognisable as Brexit there’s every chance the Deep State will be fuming.

    Yes, but little chance he will.

    • Replies: @dearieme
  14. dearieme says:

    Alas, you may well prove right.

  15. dearieme says:

    It used to be that the only elections that everyone assumed to be corrupt were (i) trade union elections, and (ii) local and general elections in constituencies with lots of Irish voters.

  16. MJB says:

    The British Pound is surging.

  17. @dc.sunsets

    “in a democracy, the scum rises to the top.”

    I used to call that “the septic tank theory of management – the big chunks float to the top”.

    As for the rest, I agree. As I have often pointed out, the large titled landowners in the UK, may not be nice people, but they understand that for the estate to be passed on, they have to be nice enough to gain other people’s trust. It’s long term planning at it’s best in the Anglo world now obsessed with short term gain. Going up the food chain to the monarch, historically, it was more important to gain the trust of the peasants, and to keep them reasonably satisfied to prevent uprisings.

    I do note that many insist that a republic is not a democracy. Technically true, but those who insist on that refuse to recognize that a republic is an elite supported by an army. Given the founders wanted “a well regulated militia” and not a standing army, the US today, is a republic, but not true to the vision of the founders.

    • Replies: @dc.sunsets
    , @Reg Cæsar
  18. @Curmudgeon

    Almost everyone steeps in a cesspool of illogic, one that they’ll defend violently against any effort to show just how absurd are their beliefs.

    Few concepts illustrate this better than what most people might call political science. Democracy is, in actual studies of Poli Sci, a joke. Voters know quite literally LESS than nothing about the tens of thousands of policy questions to be addressed by various levels of government. The myth of the informed voter is as powerful as are the myths taught as history of the American Revolution.

    Hans Hermann Hoppe wrote a brief synopsis of his book, Democracy, the God That Failed. The book title is a riff on an earlier book, written by communists disillusioned by the literally mountainous (as in mountains of corpses) evils of Mao’s China, Lenin’s (and then Stalin’s) USSR, Pol Pot’s Cambodia, etc., titled, The God That Failed. In it, they did what communists do best: claim that those unimaginable mass-murdering communists “just didn’t do it right, so it wasn’t really communism.”

    I heartily recommend Hoppe’s book, but the synopsis is actually adequate all by itself. It shows just how absurd are several examples of what most people, including those who are very high IQ and sporting advanced degrees, believe.

    Theory is indispensable in correctly interpreting history. History–the sequence of events unfolding in time–is “blind.” It reveals nothing about causes and effects. We may agree, for instance, that feudal Europe was poor, that monarchical Europe was wealthier, and that democratic Europe is wealthier still, or that nineteenth-century America with its low taxes and few regulations was poor, while contemporary America with its high taxes and many regulations is rich.

    Yet was Europe poor because of feudalism, and did it grow richer because of monarchy and democracy? Or did Europe grow richer in spite of monarchy and democracy? Or are these phenomena unrelated? Likewise, we might ask whether contemporary America is wealthier because of higher taxes and more regulations or in spite of them. That is, would America be even more prosperous if taxes and regulations had remained at their nineteenth-century levels?

    Historians qua historians cannot answer such questions, and no amount of statistical data manipulation can change this fact. Every sequence of empirical events is compatible with any of a number of rival, mutually incompatible interpretations. [Emphasis added]

    If there’s really an interested God of creation, our existence is proof he has a wicked sense of humor. Taking a step back, watching humanity is like watching the antics on Monkey Island at the zoo. Reality must constantly be laughing its ass off at us.

  19. @Justvisiting

    Don’t miss its natural companion volume:

    Remember when the “Peters projection” was all the rage among the politically bien-pensant? Now you hardly ever hear of it. It makes the “North” look crowded and the “South” presumptuous.

    • Replies: @dearieme
  20. @Curmudgeon

    I do note that many insist that a republic is not a democracy. Technically true, but those who insist on that refuse to recognize that a republic is an elite supported by an army.

    But not a God.




    אַ שפּראַך איז אַ דיאַלעקט מיט אַן אַרמיי און פֿלאָט,

    “A language is a dialect with an army and a navy.” — anonymous, via Max Weinreich

    It’s also said that France is a monarchy with a president, while the UK is a republic with a queen.

  21. dearieme says:
    @Reg Cæsar

    The principle behind the Peters projection was that the Mercator projection was just an Eskimo plot to exaggerate the amount of land in the far north. Or some such drivel.

    • Replies: @Reg Cæsar
  22. @dearieme

    The principle behind the Peters projection was that the Mercator projection was just an Eskimo plot to exaggerate the amount of land in the far north. Or some such drivel.

    Imperialistic Siberians and Greenlanders.

    My point is that it works better for us. The Third World has plenty of land, They don’t need ours.

    And they don’t want it. They want our money.

  23. And not only politics, there is climatology, indeed, even debates over IQ. Unless there is a (reproducible) experiment, we are in the domain of statistical modelling designed in Cambridge for testing plant growth with highly controlled environments. Apply that to people or deep geomorphic processes and the concepts start to break down.

    I feel that most observational science is a case of seek and you shall find. I speak as a former market researcher during one phase of my life. It was always possible to find some way to make the client feel good.

    Thank you for the book recommendation. I don’t supose it deal with my latest bugbear (Kalman filtering of temperature data since 1998) but less mathematical texts are always welcome on a five or 6 hour trip on a Cross Country train. Staring out of the window at Brexitland gets depressing. Places like Derby, Chesterfield, Sheffield, Wetherby, Leeds, York could be defined by what they made once. Now Yorkies come from Poland. It was the fault of oil (with some help from monetarism) not the EU but who wants to know the truth? There’s always a statistic to the contrary. Even so, Boris Johnson …? More Brandy.

  24. res says:

    OT: Dr. Thompson, I have run across what I think is an important methodological issue with some cases of how David Becker (and others, I believe) calculate average IQs for low scoring countries (here Nigeria) from papers presenting raw test score averages (here the SPM+). Could you please take a look at this comment and see whether I am mistaken or not?
    And if you do not see a problem with my analysis, maybe you could pass it along to David Becker?

    Thank you in advance!

    P.S. In short, most of the conversion formulas for IQ tests have a limited valid range which does not cover much of the population IQ distribution for low scoring countries (in this example more than 20%). Averaging the raw test scores and then using the conversion of that raw average as an estimate of average IQ is flawed if the conversion formulas are invalid and extremely nonlinear (and most of Becker’s conversion formulas are cubics!) in a range where there are a significant number of people. This is probably more of an issue for estimating SDs than means, but both are affected.

Current Commenter

Leave a Reply -

 Remember My InformationWhy?
 Email Replies to my Comment
Submitted comments have been licensed to The Unz Review and may be republished elsewhere at the sole discretion of the latter
Commenting Disabled While in Translation Mode
Subscribe to This Comment Thread via RSS Subscribe to All James Thompson Comments via RSS