The Unz Review • An Alternative Media Selection
A Collection of Interesting, Important, and Controversial Perspectives Largely Excluded from the American Mainstream Media
 TeasersiSteve Blog
Kahneman Fesses Up on "Priming"
Email This Page to Someone

 Remember My Information



=>

Bookmark Toggle AllToCAdd to LibraryRemove from Library • BShow CommentNext New CommentNext New ReplyRead More
ReplyAgree/Disagree/Etc. More... This Commenter This Thread Hide Thread Display All Comments
AgreeDisagreeThanksLOLTroll
These buttons register your public Agreement, Disagreement, Thanks, LOL, or Troll with the selected comment. They are ONLY available to recent, frequent commenters who have saved their Name+Email using the 'Remember My Information' checkbox, and may also ONLY be used three times during any eight hour period.
Ignore Commenter Follow Commenter
Search Text Case Sensitive  Exact Words  Include Comments
List of Bookmarks

I was one of the few critics to be less than utterly awestruck by psychologist Daniel Kahneman’s 2011 bestseller Thinking Fast and Slow.

Lately, the chapter in Thinking Fast and Slow on “priming” has come in for severe criticism as celebrated priming experiments, such as how if you show college students word scrambles that include a lot of words about the elderly, they will then walk slower (or maybe faster), have often failed to replicate.

The Replicability-Index blog has a lengthy takedown of the priming chapter in Kahneman’s book:

Reconstruction of a Train Wreck: How Priming Research Went off the Rails
February 2, 2017

Authors: Ulrich Schimmack, Moritz Heene, and Kamini Kesavan

Kahneman replied in the comments:

Daniel Kahneman February 14, 2017 at 8:37 pm
From Daniel Kahneman

I accept the basic conclusions of this blog. …

What the blog gets absolutely right is that I placed too much faith in underpowered studies. As pointed out in the blog, and earlier by Andrew Gelman, there is a special irony in my mistake because the first paper that Amos Tversky and I published was about the belief in the “law of small numbers,” which allows researchers to trust the results of underpowered studies with unreasonably small samples. …

My position when I wrote “Thinking, Fast and Slow” was that if a large body of evidence published in reputable journals supports an initially implausible conclusion, then scientific norms require us to believe that conclusion. …

I knew, of course, that the results of priming studies were based on small samples, that the effect sizes were perhaps implausibly large, and that no single study was conclusive on its own.What impressed me was the unanimity and coherence of the results reported by many laboratories. I concluded that priming effects are easy for skilled experimenters to induce, and that they are robust.

However, I now understand that my reasoning was flawed and that I should have known better. Unanimity of underpowered studies provides compelling evidence for the existence of a severe file-drawer problem (and/or p-hacking). The argument is inescapable: Studies that are underpowered for the detection of plausible effects must occasionally return non-significant results even when the research hypothesis is true – the absence of these results is evidence that something is amiss in the published record. …

I still believe that actions can be primed, sometimes even by stimuli of which the person is unaware. There is adequate evidence for all the building blocks: semantic priming, significant processing of stimuli that are not consciously perceived, and ideo-motor activation. I see no reason to draw a sharp line between the priming of thoughts and the priming of actions. …

Think of “priming” as a euphemism for “manipulability.” Is it plausible that college students could be manipulable?

Yes. Definitely yes.

Rock stars, for example, are, more or less, experts at manipulating college students.

Is it plausible that people are consistently, endlessly manipulable in important ways by simple minded tricks repeated over and over?

Maybe, maybe not. And certainly less as time goes by.

For ten years, for instance, Jerry Lewis could manipulate America into howling with laughter. For the next fifty years, not so much.

Further, some people are better than other people at manipulating, and who primes whom is highly variable.

My guess is that manipulation is very real, but also expensive and hard to do, especially over time. There’s always a manipulation arms race going on.

Scams get tired, so you need new scams.

For example, here’s an article from some site called The Outline about some other site called Mic:

For about five years, Mic.com was a place where readers could go to get moral clarity. In the Mic universe, heroes fought for equality against villains who tried to take it away. Every day, there was someone, like plus-size model Ashley Graham, to cheer for, and someone else, like manspreaders, to excoriate. Kim Kardashian annihilated slut shamers, George Takei clapped back at transphobes. “In a Single Tweet, One Man Beautifully Destroys the Hypocrisy of Anti-Muslim Bigotry.” “This Brave Woman’s Horrifying Photo Has Become a Viral Rallying Cry Against Sexual Harassment.” “Young Conservative Tries to Mansplain Hijab in Viral Olympic Photo, Gets It All Wrong.” “The Problematic Disney Body Image Trend We’re Not Talking About.” “The Very Problematic Reason This Woman Is Taking a Stand Against Leggings.” …

The success of personal, identity-driven essays like “5 Powerful Reasons I’m a (Male) Feminist,” “An Open Letter to the Pope From a Gay Man,” and “An Open Letter to Abercrombie and Fitch from a Formerly Homeless Kid” inspired Mic to launch an “Identities” section in October 2013 “dedicated to examining the intersections of sexuality, gender, class and race in politics and culture for the millennial generation.” These stories got traction on Facebook, so Mic replicated them, attracting more social justice readers as well as more social justice writers, who then wrote more social justice stories. “Mic realized earlier than most places that they could commodify people’s feelings about race and gender,” was the view of one early staffer who has since left.

But now readers have started to get tired of this kind of priming, so Mic is pivoting to some other kind of priming.

 
Hide 39 CommentsLeave a Comment
Commenters to Ignore...to FollowEndorsed Only
Trim Comments?
  1. Fair play to Mr Kahneman. He appears to be a scientist (or at least a follower of Keynes – “when the facts change, I change my mind – what do you do?“).

    • Agree: TomSchmidt
  2. It was always strange and disturbing to me how quickly academics liked to believe bizarre theories about how they could control people.

    Chomskys brilliant essay that blew behaviorism out of the water is worth a read for lay people imo: https://chomsky.info/1967____/

    • Replies: @Dieter Kief
    @anonymouslee

    Chomsky beat a pretty dead horse, though - the usual argument against behaviorism - current before Chomsky ranted against Skinner, went like this: Baviorism might be interesting for little mammals (mice etc.), or dogs even - unfortunately, those animals have - as of now - not shown the least bit of interest in those scientific findings.

  3. Wouldn’t it be odd if the whole SJW subculture was based on clickbait algorithms? Yass queen, that would be odd. But what of all the energy spent criticizing that culture? Essentially, it’s fulminating against digital formulas rather than people and ideas. In criticizing, do I just get on the same merry go round? Perhaps that’s a waste too? Better to be above the fray?

  4. “You can fool some of the people all of the time, and all of the people some of the time, but you can not fool all of the people all of the time.”

    • Replies: @backup
    @donut

    Once every four years is enough, though

    , @Comms
    @donut

    If you can fool enough people for a while, then monetize that and pursue mezzanine financing. Try to keep voting control.

  5. Rock stars are better at manipulating people that don’t go to college, except for subgenres that appeal heavily to “the college crowd”.

    • Agree: Peter Akuleyev
  6. The founders of Mic.com were called ‘Chris Altchek’ (a former Goldman banker) and ‘Jake Horowitz’ and they met in a New York prep school. Just for anyone who is interested

    • Replies: @TelfoedJohn
    @DFH

    Just goes to show - the alt-right and alt-left are providing the content.. but the Alt-Checks are making the money.

    , @duncsbaby
    @DFH

    We are never not interested.

  7. But now readers have started to get tired of this kind of stuff

    Hard to sell snobbery about stuff everybody is doing.

  8. @DFH
    The founders of Mic.com were called 'Chris Altchek' (a former Goldman banker) and 'Jake Horowitz' and they met in a New York prep school. Just for anyone who is interested

    Replies: @TelfoedJohn, @duncsbaby

    Just goes to show – the alt-right and alt-left are providing the content.. but the Alt-Checks are making the money.

  9. Impressive. Priming is now an embarrassment to social psychologists.

    Can people be influenced? Mad Men answered it. But it takes Don Draper rather than some frumpy professors. And if it were easy, you wouldn’t need Don.

    • Agree: International Jew
    • Replies: @International Jew
    @anon

    Yep. And along the same lines: if you want to learn people, you'll learn more reading great writers than frumpy professors. (Of course getting out there and having real interactions is the best of all.)

  10. OT. He’s at it again.

    http://www.telegraph.co.uk/news/2017/08/23/separated-chechen-couples-pressured-reunite-leader-believes/

    “Almost 1,000 separated couples in Chechnya have been pressured to reunite after strongman leader Ramzan Kadyrov declared that divorce lay at the root of the republic’s social problems.

    Chechen state television announced this week that “948 families have been reunited” by a special commissions of government, law enforcement and religious officials, which some residents have said pressure people to live together even in adverse circumstances.

    A mostly Muslim republic in Russia’s Caucasus mountains, Chechnya has grown increasingly conservative and authoritarian under Mr Kadyrov, who was recently re-appointed by Vladimir Putin and is seen to have impunity as long as he suppresses the region’s Islamic insurgency.

    The campaign began after Mr Kadyrov said in July that children who don’t live with both parents are more vulnerable to extremists.

    He claimed that most of the young men who had committed crimes in Chechnya in recent years grew up in “incomplete families” and argued that “out of 100 of those families, at the most five or six are normal”.

    “Our first task is to return women who left their husbands, reconcile them,” Mr Kadyrov said.”

    • Replies: @AM
    @YetAnotherAnon

    I'm a bit torn.

    There is the immoral use of force, so that's a huge problem.

    On the other hand, he's actually right. Statistics in the US backup everything he says about broken homes. Staying married for the sake of the children, absent abuse, works. The kids really don't care if you're only fair to middling happy or sleeping in separate beds.

    My Dad stayed in a hopeless marriage for my sister and I. It would be my mentally ill Mom that left after my sister was married. I don't want to think about that otherwise shaky childhood if they had separated.

    , @Daniel Chieh
    @YetAnotherAnon

    We need to bring American FREEDOM to these thots!

    , @AndrewR
    @YetAnotherAnon

    Discouraging divorce/separation doesn't make marriages any healthier, nor of course does making divorce easier make people any happier.

  11. @donut
    "You can fool some of the people all of the time, and all of the people some of the time, but you can not fool all of the people all of the time."

    Replies: @backup, @Comms

    Once every four years is enough, though

  12. Sounds like the sort of problem that bedevils the pharmaceutical industry’s clinical trials. You need volunteers with the targeted malady to see if your drug works but , unless you are targeting a terminal cancer, the sort of people who volunteer for drug trials tend to be hypochondriacs,

  13. My guess is that manipulation is very real, but also expensive and hard to do, especially over time. There’s always a manipulation arms race going on.

    At 22, I had no idea what my in-laws were up to, other than noticing a)I was upset at them all the time and b)they ended up winning a fight I hardly noticed we were having at the time.

    Twenty years later, I am mostly very, very, very civil and they spontaneously find other places to be.

    I’m not sure I could sit through college classes either anymore for similar reasons.

  14. @YetAnotherAnon
    OT. He's at it again.

    http://www.telegraph.co.uk/news/2017/08/23/separated-chechen-couples-pressured-reunite-leader-believes/

    "Almost 1,000 separated couples in Chechnya have been pressured to reunite after strongman leader Ramzan Kadyrov declared that divorce lay at the root of the republic's social problems.

    Chechen state television announced this week that “948 families have been reunited” by a special commissions of government, law enforcement and religious officials, which some residents have said pressure people to live together even in adverse circumstances.

    A mostly Muslim republic in Russia's Caucasus mountains, Chechnya has grown increasingly conservative and authoritarian under Mr Kadyrov, who was recently re-appointed by Vladimir Putin and is seen to have impunity as long as he suppresses the region's Islamic insurgency.

    The campaign began after Mr Kadyrov said in July that children who don't live with both parents are more vulnerable to extremists.

    He claimed that most of the young men who had committed crimes in Chechnya in recent years grew up in “incomplete families” and argued that “out of 100 of those families, at the most five or six are normal”.

    “Our first task is to return women who left their husbands, reconcile them,” Mr Kadyrov said."
     

    Replies: @AM, @Daniel Chieh, @AndrewR

    I’m a bit torn.

    There is the immoral use of force, so that’s a huge problem.

    On the other hand, he’s actually right. Statistics in the US backup everything he says about broken homes. Staying married for the sake of the children, absent abuse, works. The kids really don’t care if you’re only fair to middling happy or sleeping in separate beds.

    My Dad stayed in a hopeless marriage for my sister and I. It would be my mentally ill Mom that left after my sister was married. I don’t want to think about that otherwise shaky childhood if they had separated.

  15. S says:

    I think the correct take on priming is that it is real with respect to beliefs, through confirmation bias, but bullshit with respect physical action. White guys really do run slower, which is why we don’t make great wide receivers, and blacks really are worse at IQ dependent activities. However, the reason so many people believe that right wing extremist commit more terrorism than Muslims, or that blacks are being killed off by cops and the KKK is that they are primed to believe so by the media. So when the inevitable, but low probability cop on black shooting, or abortion clinic bombing comes along, it confirms their priors, but they are at a loss for causality when the now bi-monthly jihad event occurs. The funny thing is, confirmation bias probably played a large role in duping so many intelligent “scientists” in to believing these obviously retarded studies.

    • Replies: @Dieter Kief
    @S


    The funny thing is, confirmation bias probably played a large role in duping so many intelligent “scientists” in to believing these obviously retarded studies
     
    It's rather dumb than funny, isn't it?
  16. BTW, about alt-right white-separatist and “provocateur “Jason Kessler, this from his Wikipedia entry:

    Prior to 2016 Kessler was virtually unknown…Many on white nationalist forums have questioned his background as it seems he was involved with the Occupy Wall Street movement and previously supported President Barack Obama. People who know Kessler have confirmed that he previously voted for Democrats and was involved with the Occupy movement in Charlottesville…

    Kessler is also President of Unity and Security for America, 501c “non-profit”.

    Think about it. What better way to make money these days than to set up a 501c and have a Crony board of directors pay you $300,000? Washington is saturated with Crony funded Think Pimp Tanks who thrive on that model.

    So Kessler flips to an alt-right firebrand because he knows that’s where the hot-button money is. And he also knows that the only way he could get alt-right chumps to shovel donations to his non-profit (himself really) is by getting a lot of exposure.

    Like there are various flavors of “Fake News”, there are also various flavors of priming…

  17. Is Kahneman the scholar who put forward the fat man/bystander/necessary homicide/trolley/save people/ thesis?

    • Replies: @Stebbing Heuer
    @Studley

    The Trolley Problem.

    First appears in 1905. Modern version (the one we are familiar with) created by Philippa Foot in 1967.

    So, no, not Kahneman.

  18. @YetAnotherAnon
    OT. He's at it again.

    http://www.telegraph.co.uk/news/2017/08/23/separated-chechen-couples-pressured-reunite-leader-believes/

    "Almost 1,000 separated couples in Chechnya have been pressured to reunite after strongman leader Ramzan Kadyrov declared that divorce lay at the root of the republic's social problems.

    Chechen state television announced this week that “948 families have been reunited” by a special commissions of government, law enforcement and religious officials, which some residents have said pressure people to live together even in adverse circumstances.

    A mostly Muslim republic in Russia's Caucasus mountains, Chechnya has grown increasingly conservative and authoritarian under Mr Kadyrov, who was recently re-appointed by Vladimir Putin and is seen to have impunity as long as he suppresses the region's Islamic insurgency.

    The campaign began after Mr Kadyrov said in July that children who don't live with both parents are more vulnerable to extremists.

    He claimed that most of the young men who had committed crimes in Chechnya in recent years grew up in “incomplete families” and argued that “out of 100 of those families, at the most five or six are normal”.

    “Our first task is to return women who left their husbands, reconcile them,” Mr Kadyrov said."
     

    Replies: @AM, @Daniel Chieh, @AndrewR

    We need to bring American FREEDOM to these thots!

  19. @donut
    "You can fool some of the people all of the time, and all of the people some of the time, but you can not fool all of the people all of the time."

    Replies: @backup, @Comms

    If you can fool enough people for a while, then monetize that and pursue mezzanine financing. Try to keep voting control.

  20. @YetAnotherAnon
    OT. He's at it again.

    http://www.telegraph.co.uk/news/2017/08/23/separated-chechen-couples-pressured-reunite-leader-believes/

    "Almost 1,000 separated couples in Chechnya have been pressured to reunite after strongman leader Ramzan Kadyrov declared that divorce lay at the root of the republic's social problems.

    Chechen state television announced this week that “948 families have been reunited” by a special commissions of government, law enforcement and religious officials, which some residents have said pressure people to live together even in adverse circumstances.

    A mostly Muslim republic in Russia's Caucasus mountains, Chechnya has grown increasingly conservative and authoritarian under Mr Kadyrov, who was recently re-appointed by Vladimir Putin and is seen to have impunity as long as he suppresses the region's Islamic insurgency.

    The campaign began after Mr Kadyrov said in July that children who don't live with both parents are more vulnerable to extremists.

    He claimed that most of the young men who had committed crimes in Chechnya in recent years grew up in “incomplete families” and argued that “out of 100 of those families, at the most five or six are normal”.

    “Our first task is to return women who left their husbands, reconcile them,” Mr Kadyrov said."
     

    Replies: @AM, @Daniel Chieh, @AndrewR

    Discouraging divorce/separation doesn’t make marriages any healthier, nor of course does making divorce easier make people any happier.

  21. Kahneman’s book is huge in the quant investing world. It’s used all the time to back up behavioral finance work. The idea being that investor biases cause them to make various mistakes that can be taken advantage.

    Wonder if this admission will cause any pause in the investing world’s love Kahneman’s work. Probably not. It sells well.

  22. Rock stars, for example, are, more or less, experts at manipulating college students.

    If you’re a guitar-man who can’t make them love, make them cry, bring them down, and get them high, you should probably find another line of work.

  23. I searched for “In a Single Tweet, One Man Beautifully Destroys the Hypocrisy of Anti-Muslim Bigotry.” Though I was prepared (perhaps having been suitably primed) to read this article and change my entire world view, I was let down by the results.

  24. Unanimity of underpowered studies provides compelling evidence for the existence of a severe file-drawer problem

    This is key! Steve, it would have been nice if you’d explained this for the benefit of your readers who aren’t trained in statistics. So I’ll do it for you:

    These studies prove their points by applying a statistical test. (I won’t define statistical test formally, just think of a test, which a hypothesis — eg priming works — either passes or fails.) Now the thing about these tests is that they’re not infallible; they are liable to give a wrong result a certain % of the time.

    In the article, the “underpowered tests” they mention are tests that give wrong results rather more often than we’d like. They’re saying that even if priming is real, then a certain % of the studies that look for priming will fail to detect it. That % is a known number, based on mathematical considerations. And yet, among published reports about priming, none of them (that’s the “unanimity” mentioned) reject priming. Which implies that psychologists have relegated the “unsuccessful” results to that metaphorical “desk drawer”. I.e. they’re practicing advocacy and not science.

    Whew! I thought I could explain all that in fewer words, and even so I cut corners that would raise the hair on every pedantic statistician’s head. If you’re such a person, though, please, instead of yelling at me, try explaining this stuff as well or better than I have, in no more words.

    • Replies: @res
    @International Jew

    I'm not sure it is better, but here is how I think about it.

    First, postulate an environment where positive (rather than negative) results for trendy topics are easier to publish and also more happily received by one's colleagues and the media.

    Second, think of a research study on a nonexistent effect as the roll of a 20 sided die with a 1 indicating a positive effect (i.e. the 5% p value threshold for "statistical significance").

    Lastly, assume 40 researchers do underpowered studies on the nonexistent effect. 38 (on average) of those will get either no or a negative result--those go into a file drawer somewhere. 2 researchers obtain positive results that are statistically significant. These are published with resulting fame and approbation from their colleagues and the media.

    Because of behavior like p-value fishing the reality is even worse than this. True believers really want those positive results to further their agenda (and themselves). There is also a variant where there really is a small effect in some cases, but the published studies overstate both the size and universality of the effect. I think this is most likely the case for priming and stereotype threat.

    If we are lucky there is a final step debunking the original idea as we see in Steve's Kahneman example.

    A good way to know people are actively looking for (and presumably trying to correct) this effect is to look for a funnel plot included in a metastudy: https://en.wikipedia.org/wiki/Funnel_plot

    Replies: @Jim Don Bob, @Dieter Kief

    , @gregor
    @International Jew

    I think the file drawer effect is the charitable explanation.

  25. @anon
    Impressive. Priming is now an embarrassment to social psychologists.

    Can people be influenced? Mad Men answered it. But it takes Don Draper rather than some frumpy professors. And if it were easy, you wouldn't need Don.

    Replies: @International Jew

    Yep. And along the same lines: if you want to learn people, you’ll learn more reading great writers than frumpy professors. (Of course getting out there and having real interactions is the best of all.)

  26. @International Jew

    Unanimity of underpowered studies provides compelling evidence for the existence of a severe file-drawer problem
     
    This is key! Steve, it would have been nice if you'd explained this for the benefit of your readers who aren't trained in statistics. So I'll do it for you:

    These studies prove their points by applying a statistical test. (I won't define statistical test formally, just think of a test, which a hypothesis — eg priming works — either passes or fails.) Now the thing about these tests is that they're not infallible; they are liable to give a wrong result a certain % of the time.

    In the article, the "underpowered tests" they mention are tests that give wrong results rather more often than we'd like. They're saying that even if priming is real, then a certain % of the studies that look for priming will fail to detect it. That % is a known number, based on mathematical considerations. And yet, among published reports about priming, none of them (that's the "unanimity" mentioned) reject priming. Which implies that psychologists have relegated the "unsuccessful" results to that metaphorical "desk drawer". I.e. they're practicing advocacy and not science.

    Whew! I thought I could explain all that in fewer words, and even so I cut corners that would raise the hair on every pedantic statistician's head. If you're such a person, though, please, instead of yelling at me, try explaining this stuff as well or better than I have, in no more words.

    Replies: @res, @gregor

    I’m not sure it is better, but here is how I think about it.

    First, postulate an environment where positive (rather than negative) results for trendy topics are easier to publish and also more happily received by one’s colleagues and the media.

    Second, think of a research study on a nonexistent effect as the roll of a 20 sided die with a 1 indicating a positive effect (i.e. the 5% p value threshold for “statistical significance”).

    Lastly, assume 40 researchers do underpowered studies on the nonexistent effect. 38 (on average) of those will get either no or a negative result–those go into a file drawer somewhere. 2 researchers obtain positive results that are statistically significant. These are published with resulting fame and approbation from their colleagues and the media.

    Because of behavior like p-value fishing the reality is even worse than this. True believers really want those positive results to further their agenda (and themselves). There is also a variant where there really is a small effect in some cases, but the published studies overstate both the size and universality of the effect. I think this is most likely the case for priming and stereotype threat.

    If we are lucky there is a final step debunking the original idea as we see in Steve’s Kahneman example.

    A good way to know people are actively looking for (and presumably trying to correct) this effect is to look for a funnel plot included in a metastudy: https://en.wikipedia.org/wiki/Funnel_plot

    • Replies: @Jim Don Bob
    @res

    Many of these studies are done on a small sample size of paid college students, and the results are then extrapolated to the population as a whole. bs multiplied by 1000 is still BS.

    Good explanation, Res.

    , @Dieter Kief
    @res


    "Lastly, assume 40 researchers do underpowered studies on the nonexistent effect. 38 (on average) of those will get either no or a negative result–those go into a file drawer somewhere. 2 researchers obtain positive results that are statistically significant. These are published with resulting fame and approbation from their colleagues and the media."
     
    Would it need a genius of otherworldly powers to make a funny novel out of this stuff. Or a movie. I'm meta-scientifically day-dreaming, so to speak.

    (Not sure how rooted I still am in our earthly matters myself - but I laughed out loud about the way you managed to describe this madness, thanks a lot!)
  27. @Studley
    Is Kahneman the scholar who put forward the fat man/bystander/necessary homicide/trolley/save people/ thesis?

    Replies: @Stebbing Heuer

    The Trolley Problem.

    First appears in 1905. Modern version (the one we are familiar with) created by Philippa Foot in 1967.

    So, no, not Kahneman.

  28. @res
    @International Jew

    I'm not sure it is better, but here is how I think about it.

    First, postulate an environment where positive (rather than negative) results for trendy topics are easier to publish and also more happily received by one's colleagues and the media.

    Second, think of a research study on a nonexistent effect as the roll of a 20 sided die with a 1 indicating a positive effect (i.e. the 5% p value threshold for "statistical significance").

    Lastly, assume 40 researchers do underpowered studies on the nonexistent effect. 38 (on average) of those will get either no or a negative result--those go into a file drawer somewhere. 2 researchers obtain positive results that are statistically significant. These are published with resulting fame and approbation from their colleagues and the media.

    Because of behavior like p-value fishing the reality is even worse than this. True believers really want those positive results to further their agenda (and themselves). There is also a variant where there really is a small effect in some cases, but the published studies overstate both the size and universality of the effect. I think this is most likely the case for priming and stereotype threat.

    If we are lucky there is a final step debunking the original idea as we see in Steve's Kahneman example.

    A good way to know people are actively looking for (and presumably trying to correct) this effect is to look for a funnel plot included in a metastudy: https://en.wikipedia.org/wiki/Funnel_plot

    Replies: @Jim Don Bob, @Dieter Kief

    Many of these studies are done on a small sample size of paid college students, and the results are then extrapolated to the population as a whole. bs multiplied by 1000 is still BS.

    Good explanation, Res.

  29. Wasn’t the whole idea of priming a fraud from the start? It was made up to prove that racism is the cause of black underachievement, even when individual blacks had never actually dealt with racism. I swear I read about this somewhere.

  30. @anonymouslee
    It was always strange and disturbing to me how quickly academics liked to believe bizarre theories about how they could control people.

    Chomskys brilliant essay that blew behaviorism out of the water is worth a read for lay people imo: https://chomsky.info/1967____/

    Replies: @Dieter Kief

    Chomsky beat a pretty dead horse, though – the usual argument against behaviorism – current before Chomsky ranted against Skinner, went like this: Baviorism might be interesting for little mammals (mice etc.), or dogs even – unfortunately, those animals have – as of now – not shown the least bit of interest in those scientific findings.

  31. bored identity is not amused – priming screamers are more than ready to unleash Siri,Cortana, Alexa, and all other AI ANTIFA Whores on you:

  32. @S
    I think the correct take on priming is that it is real with respect to beliefs, through confirmation bias, but bullshit with respect physical action. White guys really do run slower, which is why we don't make great wide receivers, and blacks really are worse at IQ dependent activities. However, the reason so many people believe that right wing extremist commit more terrorism than Muslims, or that blacks are being killed off by cops and the KKK is that they are primed to believe so by the media. So when the inevitable, but low probability cop on black shooting, or abortion clinic bombing comes along, it confirms their priors, but they are at a loss for causality when the now bi-monthly jihad event occurs. The funny thing is, confirmation bias probably played a large role in duping so many intelligent "scientists" in to believing these obviously retarded studies.

    Replies: @Dieter Kief

    The funny thing is, confirmation bias probably played a large role in duping so many intelligent “scientists” in to believing these obviously retarded studies

    It’s rather dumb than funny, isn’t it?

  33. @res
    @International Jew

    I'm not sure it is better, but here is how I think about it.

    First, postulate an environment where positive (rather than negative) results for trendy topics are easier to publish and also more happily received by one's colleagues and the media.

    Second, think of a research study on a nonexistent effect as the roll of a 20 sided die with a 1 indicating a positive effect (i.e. the 5% p value threshold for "statistical significance").

    Lastly, assume 40 researchers do underpowered studies on the nonexistent effect. 38 (on average) of those will get either no or a negative result--those go into a file drawer somewhere. 2 researchers obtain positive results that are statistically significant. These are published with resulting fame and approbation from their colleagues and the media.

    Because of behavior like p-value fishing the reality is even worse than this. True believers really want those positive results to further their agenda (and themselves). There is also a variant where there really is a small effect in some cases, but the published studies overstate both the size and universality of the effect. I think this is most likely the case for priming and stereotype threat.

    If we are lucky there is a final step debunking the original idea as we see in Steve's Kahneman example.

    A good way to know people are actively looking for (and presumably trying to correct) this effect is to look for a funnel plot included in a metastudy: https://en.wikipedia.org/wiki/Funnel_plot

    Replies: @Jim Don Bob, @Dieter Kief

    “Lastly, assume 40 researchers do underpowered studies on the nonexistent effect. 38 (on average) of those will get either no or a negative result–those go into a file drawer somewhere. 2 researchers obtain positive results that are statistically significant. These are published with resulting fame and approbation from their colleagues and the media.”

    Would it need a genius of otherworldly powers to make a funny novel out of this stuff. Or a movie. I’m meta-scientifically day-dreaming, so to speak.

    (Not sure how rooted I still am in our earthly matters myself – but I laughed out loud about the way you managed to describe this madness, thanks a lot!)

  34. The great digital media culling of 2017

    In recent weeks and months, several digital publishers have executed a “pivot to video,” a now-notorious euphemism for “firing most or all of your writers to chase video advertising dollars.” Mashable, Vocativ, MTV News, Fox Sports, Vice, ATTN, and now Mic have tried this, at least to some extent. It probably sounded like a sexy and forward-thinking strategy in whatever corporate boardroom it was cooked up in. But as a long-term business proposition, it will almost certainly fail — just as dependence on Facebook traffic did.

    There are many reasons why this all-in-on-video strategy is ill-conceived — not least of which is that we have not quite reached Idiocracy-levels of vacuousness yet, and many if not most news consumers still prefer to, you know, read things. There is plenty of good video on the internet, some of it even made by digital media organizations. But not every publisher needs to be in the video business. And the fact that so many quality news sites are canning their writers, reporters, and editors to churn out glorified slideshows in pursuit of those sweet CPMs misunderstands both the audience and the industry. This is a fool’s bargain.

    Why? Because even as digital publishers relentlessly pursue quick profits — often by trying to cynically engineer referral traffic bonanzas from monopoly platforms like Google and Facebook — these same monopoly platforms are devouring almost all new online advertising dollars. And it’s not hard to see why. Facebook has more than 2 billion monthly users. Google handles billions of searches every day. Their audiences are so massive that they can serve advertisers the exact sort of eyeballs they want. A website with 1 million or 10 million or even 100 million monthly visitors can offer advertisers the audience it has. Google and Facebook can offer advertisers whatever audience they want.

    [MORE]

    Mic was hardly alone in chasing the gold rush of Facebook traffic. Viral content factories like Upworthy and ViralNova based their entire business models — and absurdly astronomical valuations — on gaming Facebook. This probably seemed like an awesome idea until Facebook tweaked its algorithm in a way that penalized clickbait. Mic, for one, attempted to recoup its losses by pivoting from churning out Facebook-optimized dreck to churning out search-optimized dreck, which worked for a bit. But all the while, Google and Facebook were themselves scooping up more and more of the online ad market. In 2015, Facebook and Google gobbled up 60 percent of the growth in online ad spending. In 2016, that figure was 99 percent. For every $100 in new digital ad spending, only $1 went to a company not named Facebook and Google. That ought to terrify every sentient being working in the digital news media.

    Online advertising has become a duopoly, badly squeezing nearly every digital media company. And that has led to today’s “pivot to video.” Video ads are the last pot of serious money left — and so that is where many digital publishers are piling in. But make no mistake: This is foolish.

  35. @International Jew

    Unanimity of underpowered studies provides compelling evidence for the existence of a severe file-drawer problem
     
    This is key! Steve, it would have been nice if you'd explained this for the benefit of your readers who aren't trained in statistics. So I'll do it for you:

    These studies prove their points by applying a statistical test. (I won't define statistical test formally, just think of a test, which a hypothesis — eg priming works — either passes or fails.) Now the thing about these tests is that they're not infallible; they are liable to give a wrong result a certain % of the time.

    In the article, the "underpowered tests" they mention are tests that give wrong results rather more often than we'd like. They're saying that even if priming is real, then a certain % of the studies that look for priming will fail to detect it. That % is a known number, based on mathematical considerations. And yet, among published reports about priming, none of them (that's the "unanimity" mentioned) reject priming. Which implies that psychologists have relegated the "unsuccessful" results to that metaphorical "desk drawer". I.e. they're practicing advocacy and not science.

    Whew! I thought I could explain all that in fewer words, and even so I cut corners that would raise the hair on every pedantic statistician's head. If you're such a person, though, please, instead of yelling at me, try explaining this stuff as well or better than I have, in no more words.

    Replies: @res, @gregor

    I think the file drawer effect is the charitable explanation.

  36. @DFH
    The founders of Mic.com were called 'Chris Altchek' (a former Goldman banker) and 'Jake Horowitz' and they met in a New York prep school. Just for anyone who is interested

    Replies: @TelfoedJohn, @duncsbaby

    We are never not interested.

  37. Haven’t professional magicians proven “priming” beyond a shadow of a doubt for many decades? With mentalism, predictive tricks, mind reading, etc, I believe it’s a solid area of study.

    Not to mention hypnotism, which – while definitely not being mind control – is surely priming to an extreme degree.

    • Replies: @Steve Sailer
    @Pansophic_Gothi

    Right, as I pointed out in 2011, many of Kahneman's experiments are reminiscent of stage magicians' tricks, so it's hardly surprising to learn that you can fool some of the people some of the time.

  38. @Pansophic_Gothi
    Haven't professional magicians proven "priming" beyond a shadow of a doubt for many decades? With mentalism, predictive tricks, mind reading, etc, I believe it's a solid area of study.

    Not to mention hypnotism, which - while definitely not being mind control - is surely priming to an extreme degree.

    Replies: @Steve Sailer

    Right, as I pointed out in 2011, many of Kahneman’s experiments are reminiscent of stage magicians’ tricks, so it’s hardly surprising to learn that you can fool some of the people some of the time.

  39. Steve:

    You owe me a hat tip or whatever the bloggers call it. And you left out the MIT chick.

Comments are closed.

Subscribe to All Steve Sailer Comments via RSS