One of the joys of attending a conference on intelligence, such as ISIR2017, is hearing people report the results of their detailed investigations into overblown claims in science reporting that one had oneself thought questionable. Will the proper researcher come to the same conclusions as you had in your own preliminary reactions?
Years ago I was asked to comment on a paper which was being touted as revealing that IQ test results were strongly affected by the vagaries of motivation. In fact, the paper actually said that the main factor in an intelligence test result was intelligence, but it did go on to discuss the contribution of motivation at more length. Press reports took the hint, and stressed the interpretation that motivation distorted IQ results.
However, science does not proceed by immediate impressions, but by studies which give any hypothesis a chance to succeed, before then testing it further by giving it a chance to fail.
Into this controversy steps the highly motivated Gilles Gignac, whose conference paper I show below. It is fun to see how a more disciplined mind takes a subject apart. First, Gignac shows that the claim that motivation can boost IQ by 10 points was skewed by the fact that only 2 of the 46 studies were carried out on adults, and that a only a few papers claimed very big results. Other conference delegates said that those were of less good quality. Gignac also found out that motivation was measured by counting the number of times children said: “I don’t know” quickly to questions. As he points out, this might be something that an unmotivated child does, but it could also be due to a child simply not knowing something, admitting it, and wanting to move on to the next question.
In Gignac’s study 1 he measured ability in university students, and measured their motivation by the Student Opinion Scale. There was a small but non-fluke r= .3 correlation which was reduced to .25 with the inclusion of the most substantial interference effect.
Case proved? Not really. He carefully explained that there are two explanations for this correlation. The first is that low motivation lowered the ability scores a bit. The second is that intelligence affects the amount of effort students put in to the ability tests, but in a positive way. Intelligent subjects could be simply more motivated when confronted by intelligence tests. Gignac concludes: “the correlation between motivation and IQ scores is not simply due to interference.”
So, intelligence is affecting both problem solving and the motivation to solve problems. It is an additional correlated force, and not a separate interfering factor.
Even though he had probably proved his case, Gignac tried to test the motivational hypothesis further, by seeing whether the strong prospect of winning $75 boosted performance on tests of ability, having established that was the sum that undergraduates found motivating. He gave no fewer than 5 IQ tests: Visual-Spatial Ability, Letter-Number Sequencing in 2 forms, Fluid intelligence, Working memory. He imagined that the processing speed tasks should be most sensitive to motivation, but although there was a bit of an effect on the numbers task, even that task fell short of significance, and all the others were flat. Motivation did not boost performance. That is even the case for the APM shown below, which is the Advanced Progressive Matrices, a test of Fluid Intelligence.
1. It is likely that there is a positive correlation between test-taking motivation and IQ scores in low-stakes settings.
2. The correlation is substantially due to intelligence impacting test-taking motivation, rather than test-taking motivation impacting IQ scores.
3. Attempts to increase IQ scores via financial incentive have failed in adults.
4. There is an absence of a causal effect between test-taking motivation and IQ scores.
Get his whole conference presentation here:
Happily, my off-the-cuff comments can now be substantiated by proper research. That is worth much more than $75.