RSS
◄►Bookmark◄❌►▲ ▼Toggle AllToC▲▼Add to LibraryRemove from Library • BShow CommentNext New CommentNext New Reply
One of the joys of attending a conference on intelligence, such as ISIR2017, is hearing people report the results of their detailed investigations into overblown claims in science reporting that one had oneself thought questionable. Will the proper researcher come to the same conclusions as you had in your own preliminary reactions?
Years ago I was asked to comment on a paper which was being touted as revealing that IQ test results were strongly affected by the vagaries of motivation. In fact, the paper actually said that the main factor in an intelligence test result was intelligence, but it did go on to discuss the contribution of motivation at more length. Press reports took the hint, and stressed the interpretation that motivation distorted IQ results.
http://www.bbc.co.uk/news/health-13156817
However, science does not proceed by immediate impressions, but by studies which give any hypothesis a chance to succeed, before then testing it further by giving it a chance to fail.
Into this controversy steps the highly motivated Gilles Gignac, whose conference paper I show below. It is fun to see how a more disciplined mind takes a subject apart. First, Gignac shows that the claim that motivation can boost IQ by 10 points was skewed by the fact that only 2 of the 46 studies were carried out on adults, and that a only a few papers claimed very big results. Other conference delegates said that those were of less good quality. Gignac also found out that motivation was measured by counting the number of times children said: “I don’t know” quickly to questions. As he points out, this might be something that an unmotivated child does, but it could also be due to a child simply not knowing something, admitting it, and wanting to move on to the next question.
In Gignac’s study 1 he measured ability in university students, and measured their motivation by the Student Opinion Scale. There was a small but non-fluke r= .3 correlation which was reduced to .25 with the inclusion of the most substantial interference effect.
Case proved? Not really. He carefully explained that there are two explanations for this correlation. The first is that low motivation lowered the ability scores a bit. The second is that intelligence affects the amount of effort students put in to the ability tests, but in a positive way. Intelligent subjects could be simply more motivated when confronted by intelligence tests. Gignac concludes: “the correlation between motivation and IQ scores is not simply due to interference.”
So, intelligence is affecting both problem solving and the motivation to solve problems. It is an additional correlated force, and not a separate interfering factor.
Even though he had probably proved his case, Gignac tried to test the motivational hypothesis further, by seeing whether the strong prospect of winning $75 boosted performance on tests of ability, having established that was the sum that undergraduates found motivating. He gave no fewer than 5 IQ tests: Visual-Spatial Ability, Letter-Number Sequencing in 2 forms, Fluid intelligence, Working memory. He imagined that the processing speed tasks should be most sensitive to motivation, but although there was a bit of an effect on the numbers task, even that task fell short of significance, and all the others were flat. Motivation did not boost performance. That is even the case for the APM shown below, which is the Advanced Progressive Matrices, a test of Fluid Intelligence.
1. It is likely that there is a positive correlation between test-taking motivation and IQ scores in low-stakes settings.
2. The correlation is substantially due to intelligence impacting test-taking motivation, rather than test-taking motivation impacting IQ scores.
3. Attempts to increase IQ scores via financial incentive have failed in adults.
4. There is an absence of a causal effect between test-taking motivation and IQ scores.
Get his whole conference presentation here:
https://drive.google.com/open?id=0B3c4TxciNeJZVlA1dlNRNWFMSDA
Happily, my off-the-cuff comments can now be substantiated by proper research. That is worth much more than $75.
If motivation made no difference, you wouldn’t need active control groups in brain training studies and there would be no difference in effect sizes between active or passive control groups. But there are.
/jthompson/the-motivational-quotient/#comment-1939514
How were the students selected? Unless they were caught at random in the university halls and forced to participate in the experiment, they already came in with some basic level of motivation.
The world’s former smartest man – I believe he’s now the world’s second smartest man – Rick Rosner talks about motivation here:
https://www.youtube.com/watch?v=yNM2a_P0t
Motivation did not boost performance.
That is not a correct conclusion. He can only conclude that the promise of $75 did not boost performance.
IQ tests are fun. It’s as simple as that. Our brains are wired to problem-solve and the tests are deliberately created to start with simple tasks. By the time it gets trickier you’re already sailing on a string of “victories” so there’s no problem with motivation. Getting discouraged after that point is a sure sign of reaching the limit of one’s IQ potential.
The proponents of motivation explanations for psychometric differences are probably the people who are most opposed to the free market, and, for instance, using IQ tests for job placement.
Does motivation matter in life? You bet! Can it explain the observed differences between categorized groups. Not likely. The motivation to eliminate these differences is intensely high, yet it has not been done, despite billions being spent.
Hello moribund…
Cognitive tests trivialize the importance and / or meaning of intelligence by restricting its concept and application to issues that are dissociated from reality, but not from the world of work. Solving these silly questions says absolutely nothing if the person is able, for example, to recognize ((((predators)) or (((parasites))) potential.
Cognitive tests are for domesticated humans [more domesticated than the baseline population].
So the tests transform something that is inescapable in the natural world [question of life or death], intelligence, into something more recreational and secondary in importance. You do not have to know about your rights, about who the real elite are and what the plans are for you [at long term], as a subordinate, you just need to be smart to learn and apply non-social cultural techniques, to use sophisticated language and to have A useless intellectual social status, as useless as the world of celebrities.
Thank you goooogol translator.
Don’t hate me for needling you, but would’t ‘affecting’ or ‘influencing’ be better word choices than ‘impacting’ in this or just about any context?
The 2015 OECD PISA survey has data on motivation from 540,000 students. On first glance in the majority of countries the result is the reverse of that reported by BBC, the average score of the three subjects PISA3 is negatively correlated to percentage of self assessed motivated students in the country samples,
PISA3 = -53.5598*Motiv +479.174; # n=57; Rsq=0.1888; p=0.0007323
The plot of data points showed that the scatter about the linear regresssion line increased with Motiv hence there were are factors in play.
There are also data on anxiety,
Anxiety = +0.323805*Motiv +0.0705132; # n=57; Rsq=0.2003; p=0.0004811
Increased motivation also increases the anxiety level which traditionally assumed that it would degrade the performance,
PISA3 = -43.5129*Anxiety +479.632; # n=57; Rsq=0.06521; p=0.05522
but from the PISA data the trend is weak and borderline statistically not significant. More on that later.
There are also data on more specific levels of motivation, i.e. wanted to be selected (into better universities ?) and wanted to be the best. All these are correlated. A multivariable linear regression procedure initially with all the variables and iteratively dropping variable that is not significant yields the final equation,
PISA3 = -1.28487*WantBestPct +558.715; # n=57; Rsq=0.2372; p=0.0001222
i.e. all the other variables contain latent characteristic best described by the
variable WantBestPct without which they were not statistically significant. The
plot with the other variables though showed clustering around the respective regression lines there are much scatter. The plot of PISA3 vs WantBestPct shows a
clearer pattern, a sloping pitchfork shape. While about 75% of the PISA3 scores
decrease with WantBestPct, about 25% have the opposite trend, PISA3 increases wi
th WantBestPct. That explains the borderline statistically not significant PISA3
~Anxiety regression.
The overall average point so happen to be at the junction of the handle and the two forks. The data points in this upper fork can be objectively isolated by using the overall group average point as the origin and partitioned them into 4 quadrants. The regression line for the first quadrant is statistically significant,
PISA3=+1.54352*WantBestPct+400.991; # n=14; Rsq=0.3779; p=0.01931
Another attribute of WantBestPct is the intensity of competition. Some students thrive with competitions while others suffer from performance anxiety. The countries in this upper fork give a hint of their common attributes, the descendents of the Vikings Denmark, Sweden and Norway, recent historical frontier countries Canada, Australia and New Zealand, and the East Asian countries Singapore, Korea, Japan, China, Taiwan, HongKong.
On the sloping pitchfork handle where PISA3 increases with lesser competition is examplify by Finland which boasted their low competition education environment. However Finland has since being eclipsed by Estonia with higher competition level as the top European PISA country.
Hmm. The changes in article formatting are mucking up the display in my old browser. Not sure if this will get through. Anyway,
The PISA data naturally split the data points in the performance~competition space into three groups, the competition driven, the stress aversed and the slogging it out. It is naturally trying to speculate the results from the genetic point of views.
It is well known that the ‘warrior gene’ marker rs4680 on the average distinguishes groups of ‘warrior’ who thrive under pressure and the ‘worrier’,
https://www.snpedia.com/index.php/Rs4680
The histogram showed that the East Asians have higher percentage of the G allele Val158Met. Though the data for the descendents of Vikings and frontier pioneers are not available, it could be assumed that they also could have high percentage of ‘warrior gene’
The geographical distribution of the ‘warrior gene’, http://popgen.uchicago.edu/ggv/?data=%221000genomes%22&chr=22&pos=19951271
Of the four European countries data points, the Fin have the highest ‘worrier’ percentage at 59% which explains their position at the top of the PISA pitchfork handle. The Italians, Spaniads and Brits are at 45.4%, 47.2% and 52.7% respectively. The corresponding data for SouthChina/Singapore and Japan are 28.2% and 28.4% respectively.
I stated at elsewhere that this ‘warrior’ gene might explain the grit attribute. Recent I heard in the SamHarris podcast that Charles Murray also stated that to be successful those with high IQ also required a high dose of grittiness.
The PISA data naturally split the data points in the performance~competition space into three groups, the competition driven, the stress aversed and the slogging it out. It is naturally trying to speculate the results from the genetic point of views.
It is well known that the 'warrior gene' marker rs4680 on the average distinguishes groups of 'warrior' who thrive under pressure and the 'worrier',
https://www.snpedia.com/index.php/Rs4680The histogram showed that the East Asians have higher percentage of the G allele Val158Met. Though the data for the descendents of Vikings and frontier pioneers are not available, it could be assumed that they also could have high percentage of 'warrior gene'
The geographical distribution of the 'warrior gene', http://popgen.uchicago.edu/ggv/?data=%221000genomes%22&chr=22&pos=19951271
Of the four European countries data points, the Fin have the highest 'worrier' percentage at 59% which explains their position at the top of the PISA pitchfork handle. The Italians, Spaniads and Brits are at 45.4%, 47.2% and 52.7% respectively. The corresponding data for SouthChina/Singapore and Japan are 28.2% and 28.4% respectively.
I stated at elsewhere that this 'warrior' gene might explain the grit attribute. Recent I heard in the SamHarris podcast that Charles Murray also stated that to be successful those with high IQ also required a high dose of grittiness.
I laugh when I see someone attributing complex traits to single SNPs.
I laughed when some thought that an example is the totality.
I laughed when I saw someone think they know what genes/DNA do in the body.
I laugh when you do this, charlaToni…
Oh the guy who can’t answer simple premises.
Maybe you can explain to this to me?
As I said before it was speculation with the obvious low hanging fruit. With Rsq=0.3779 those with understanding of statistics will implicitly understand that that was not the complete explanation.
A correction on one particular data point. In genomics it is often that the data for China, Japan and Korea are pooled together that I let it slipped that Japan is on the upper fork of the PISA data. In fact it is on the pitchfork handle with WantBestPct as reported by PISA to be even less than that for Finland. The values for WantBestPct and WantSelectPct for most countries are usually comparable, e.g. for BSJG_China they are 81.1 and 96.6, but for Japan they are 32.9 and 87.3. The Japanese students seem to be lack of competitiveness. Explanations on why the descendents of the Samurai, the Goth warriors and the Roman Legions are not on the upper fork are needed.
Specific genomic data are rare. Common data are usually used to cover the characteristics of several related countries, e.g. the CEU data are used to specify Western and Northern Europeans, the FIN data are used for Baltics countries, IBS data for Spain and Portugal, TIS for Italy and the Balkans, the average of CHB and CHS for BSJG_China, CHS for Singapore, HongKong, Macao and Taiwan, JPT for Japan and Korea, etc. Thus the analysis is approximate. As it stand there are little or no data for countries on the lower fork. See regression with proxy,
https://tamino.wordpress.com/2010/05/22/regression/
https://en.wikipedia.org/wiki/Proxy_%28statistics%29
It is interesting that the plot of PISA3 vs ComtGFrq (pop fraction with warrior gene COMT with G allele) still shows a ‘V’ shape with the lower fork missing. With the data splitted about the mean ComtGFrq=0.5677, the upper fork is statistically significant (but with Rsq=0.5526 it is implicitly known that it is not the complete explanation),
PISA3 = +883.735*ComtGFreqU -137.259; # n=20; Rsq=0.5526; p=0.0001726
However, for the ‘handle’ it is statistically not significant,
PISA3 = -251.265*ComtGFreqH +614.487; # n=36; Rsq=0.08606; p=0.08248
The single Japan data point with extremely high ComtGFrq and yet with low WantBestPct (at the extreme left side), at the other ends from the East Asians (at the extreme right side). The Italian with ComtGFrq greater than that for Britain and yet UK is on the upper fork. Some considered these countries are temporarily suffering from the so called ‘German Angst’ (Leistungsdruck?), being the losing side in WW2 with the trauma of the Allies forces fighting foot by foot closed to or through their homelands and with competitiveness significantly driven out from them. The Japanese were further traumatized as the sole nation that had experienced the effects of nuclear devices.
http://www.ozy.com/acumen/german-angst-its-in-the-genes/36547
with reference to the research which studied the effects of methylation on several genes and in particular with reaction to stress and the handling of it, and they might passed down to a few generations after the events,
http://www.biologicalpsychiatryjournal.com/article/S0006-3223%2809%2900394-1/fulltext
Whether these are valid or not are beside the point that in the PISA data they were behaving differently from their genetic peers while others are behaving as expected with reference to the tolerance with competitive stresses.
The results are directly expected from the hypothesis to be tested. The population that are competitive stress resilient might behave as expected while those that are not competitive stress resilient are not resilent in their own separate ways that the results for them might be less conclusive or even not statistically significant. Closer investigations are needed for this group.
So talking about intelligence is taboo, but talking about motivation is not. If “motivated” researchers “discover” that IQ tests are only measuring motivation and not intelligence, does this not legitimize IQ tests? We could just rename them “motivation tests” and discover (unsurprisingly) that the “motivation” they measure is strongly correlated with job performance, academic outcomes, health outcomes, crime rates, etc. It might seem perfectly right and proper to allow employers and admissions committees to measure the motivation of applicants. We might even discover that the various ways of measuring motivation all seem to measure a common trait we will name “m” and we can rank motivation tests by their m-loading. This might be the end of the empirical study of intelligence, but a revolution in the empirical study of motivation – using the very tools of the now-defunct intelligence researchers. And in one fell swoop, everybody feels better.