The Unz Review - Mobile
A Collection of Interesting, Important, and Controversial Perspectives Largely Excluded from the American Mainstream Media
Email This Page to Someone

 Remember My Information



=>
Authors Filter?
Razib Khan
Nothing found
 TeasersGene Expression Blog
/
GRE

Bookmark Toggle AllToCAdd to LibraryRemove from Library • BShow CommentNext New CommentNext New Reply
🔊 Listen RSS

Screenshot - 12012015 - 10:22:42 PM

The above is from an article in Nature, A test that fails. Two stories first. One of my good friends who went to grad school at MIT got a good ribbing from his roommates because he was the only one who didn’t get a perfect score on the math portion of the GRE. Luckily for him, he was a chemist, so they let him into the program. It is a truth universally acknowledged among the quantitatively ept that the quantitative GRE is just way too easy, and is compressed at the top scale and does not allow for differentiation of the good from the great. That is, there are a wide range of competencies which are bracketed among those who score a “perfect” 800 on the quantitative GRE. And there are many people in fields like physics who score 800; the average score on quantitative reasoning for those who intend to study physics in graduate school (not those who get accepted!) is in the 740s. Second, a friend of mine was complaining about the lack of underrepresented minorities in the biological sciences at my graduate school. To her surprise and irritation I just pointed out that all the underrepresented minorities within the range of GRE scores that our program takes would be going to Stanford or Berkeley. There weren’t enough of them that we’d be competitive. Data like the above is just not well known.

Another point is that the article above is very anti-GRE. They claim that the GRE score is not very predictive of ultimate outcome. One of my professors pointed to a study at University of California San Francisco (UCSF) where they tracked future successes (e.g., tenured position in academia x years out), and correlated them with GPA and GRE. Neither were very strong predictors. Rather, their Ph.D. research productivity was highly predictive. This isn’t that surprising, because GPA and GRE are just proxies to get at whether one can be a productive researcher, and being productive in graduate school is probably the best guide as to whether you’ll be productive later. But, one thing I want to point out is that UCSF is a very selective school. The range of GRE scores in particular is likely to be narrow, because they’re going to simply not even look at applicants with low scores. Whenever people point out that MCAT or GRE is poorly correlated with professional outcomes, remember that you’ve already compressed the distribution toward the higher end. If schools allowed a much wider range of applicants in, then these aptitude tests would be much more predictive.

Screenshot - 12012015 - 11:12:23 PM In fact, the reality is that there is variation in outcomes according to general intelligence among graduate students. As I stated above, the maximum score of the GRE, especially the quantitative reasoning section, is too low to get at that. But Camilla Benbow’s group has been tracking mathematically precocious children for decades. As the data to the left shows, the smartest-of-the-smart are more likely to become scientists, and much more likely to attain tenure. The cut-off was scoring in the top 1% of their age group on the mathematical SAT test, a 390 score. You can see how much better those very rare students who score 700 or more at age 13 are doing later in life.

Finally, obviously these tests are very robust and predictive, but they’re population statistics. There are people who do not do well on the GRE who do well in academia, and vice versa. But, the reality is that these tests are not useless, and just how “not useless” they are will become more obvious if no one made recourse to them.

Addendum: My physicist friends always enjoy a chuckle whenever I honestly state that physicists are smarter than biologists, as I am a biologist. There are rare cases, such as Ed Witten, of people entering physics from other fields, but in general it’s the physicists who are the imperialists. And that’s because they’re smart, able to decompose general problems rapidly and decisively. In contrast, biologists are somewhat narrow in their focus, and plodding in their reasoning. These are generalizations, but I think they’re roughly correct (I had a friend at a prominent non-profit who was irritated with how difficult it was to find Ph.D. biologists who were flexible thinkers in interviews). And standardized tests bear out my generalization (though honestly, it is a pleasure talking to physicists and mathematicians about out of topic fields compared to biologists partly because they’re so mentally acute; you don’t need GRE stats to get this).

But, another implication of this logic is that some minority groups are also not too bright. If you don’t think these tests are accurately reflecting real intellectual skills that groups have though you don’t have to go there. And my experience is that this is a common belief, including among physicists. But then I suppose they shouldn’t get so full of themselves about their GRE scores in relation to biologists?

 
• Category: Science • Tags: Academia, GRE, Psychometrics 
🔊 Listen RSS

The post below on teachers elicited some strange responses. Its ultimate aim was to show that teachers are not as dull as the average education major may imply to you. Instead many people were highly offended at the idea that physical education teachers may not be the sharpest tools in the shed due to their weak standardized test scores. On average. It turns out that the idea of average, and the reality of variation, is so novel that unless you elaborate in exquisite detail all the common sense qualifications, people feel the need to emphasize exceptions to the rule. For example, over at Fark:

Apparently what had happened was this: He played college football. He majored in math, minored in education. When he went to go get a job, he took it as a math teacher. When the football coach retired/quit, he took over. When funding for an advance computer class was offered, he said he could teach it after he got the certs – he easily got them within a month.

So the anecdote here is a math teacher who also coached. Obviously the primary issue happens to be physical education teachers who become math teachers! (it happened to me, and it happened to other readers apparently) In the course of double checking the previous post I found some more interesting GRE numbers. You remember the post where I analyzed and reported on GRE scores by intended graduate school concentration? It was a very popular post (for example, philosophy departments like it because it highlights that people who want to study philosophy have very strong GRE scores).

As it happens the table which I reported on is relatively coarse. ETS has a much more fine-grained set of results. Want to know how aspiring geneticists stack up against aspiring ecologists? Look no further! There are a lot of disciplines. I wanted to focus on the ones of interest to me, and I limited them to cases where the N was 100 or greater (though many of these have N’s in the thousands).

You’re going to have to click the image to make out where the different disciplines are. But wait! First I need to tell you what I did. I looked at the average verbal and mathematical score for each discipline. Then I converted them to standard deviation units away from the mean. This is useful because there’s an unfortunate compression and inflation on the mathematical scores. Disciplines which are stronger in math are going to have a greater average because the math averages are higher all around. You can see that I divided the chart into quadrants. There are no great surprises. People who want to pursue a doctorate in physical education are in the bottom left quadrant. Sorry. As in my previous post physicists, economists, and philosophers do rather well. But there were some surprises at the more detailed scale. Historians of science, and those graduate students who wish to pursue classics or classical languages are very bright. Budding historians of science have a relatively balanced intellectual profile, and the strongest writing scores of any group except for philosophers. I think I know why: many of these individuals have a science background, but later became interested in history. They are by nature relatively broad generalists. I have no idea why people drawn to traditionally classical fields are bright, but I wonder if it is because these are not “sexy” domains, to the point where you have to have a proactive interest in the intellectual enterprise.

I also wanted to compare aggregate smarts to intellectual balance. In the plot to the right on the x-axis you have the combined value of math and verbal scores in standard deviation units. A negative value indicates lower values combined, and a positive value higher. Obviously though you can have a case where two disciplines have the same average, but the individual scores differ a lot. So I wanted to compare that with the difference between the two scores. You can see then in the plot that disciplines like classics are much more verbal, while engineering is more mathematical. Physical scientists tend to be more balanced and brighter than engineers. Interestingly linguists have a different profile than other social scientists, and cognitive psych people don’t cluster with others in their broader field. Economists are rather like duller physicists. Which makes sense since many economists are washed out or bored physicists. And political science and international relations people don’t stack up very well against the economists. Perhaps this is the source of the problem whereby economists think they’re smarter than they are? Some humility might be instilled if economics was always put in the same building as physics.

In regards to my own field of interest, the biological sciences, not too many surprises. As you should expect biologists are not as smart as physicists or chemists, but there seems to be two clusters, with a quant and verbal bias. This somewhat surprised me. I didn’t expect ecology to be more verbal than genetics! And much respect to the neuroscience people, they’re definitely the smartest biologists in this data set (unless you count biophysicists!). I think that points to the fact that neuroscience is sucking up a lot of talent right now.

The main caution I would offer is that converting to standard deviation units probably means that I underweighted the mathematical fields in their aptitudes, because such a large fraction max out at a perfect 800. That means you can’t get the full range of the distribution and impose an artificial ceiling. In any case, the raw data in the table below. SDU = standard deviation units.

 

Field V-mean M-mean V-SDU M-SDU Average-SDU Difference-SDU
Anatomy 443 568 -0.16 -0.11 -0.13 -0.05
Biochemistry 486 669 0.20 0.56 0.38 -0.36
Biology 477 606 0.13 0.15 0.14 -0.02
Biophysics 523 727 0.51 0.95 0.73 -0.43
Botany 513 626 0.43 0.28 0.35 0.15
Cell & Mol Bio 497 658 0.29 0.49 0.39 -0.20
Ecology 535 638 0.61 0.36 0.49 0.26
Develop Bio 490 623 0.24 0.26 0.25 -0.02
Entomology 505 606 0.36 0.15 0.25 0.22
Genetics 496 651 0.29 0.44 0.36 -0.16
Marine Biology 499 611 0.31 0.18 0.24 0.13
Microbiology 482 615 0.17 0.21 0.19 -0.04
Neuroscience 533 665 0.60 0.54 0.57 0.06
Nutrition 432 542 -0.25 -0.28 -0.27 0.03
Pathology 468 594 0.05 0.07 0.06 -0.02
Pharmacology 429 634 -0.28 0.33 0.03 -0.61
Physiology 464 606 0.02 0.15 0.08 -0.13
Toxicology 465 610 0.03 0.17 0.10 -0.15
Zoology 505 609 0.36 0.17 0.26 0.20
Other Biology 473 626 0.09 0.28 0.19 -0.19
Chemistry, Gen 483 681 0.18 0.64 0.41 -0.47
Chemistry, Analytical 464 652 0.02 0.45 0.23 -0.43
Chemistry, Inorganic 502 690 0.34 0.70 0.52 -0.37
Chemistry, Organic 490 683 0.24 0.66 0.45 -0.42
Chemistry, Pharm 429 647 -0.28 0.42 0.07 -0.69
Chemistry, Physical 513 708 0.43 0.82 0.62 -0.39
Chemistry, Other 477 659 0.13 0.50 0.31 -0.37
Computer Programming 407 681 -0.46 0.64 0.09 -1.10
Computer Science 453 702 -0.08 0.78 0.35 -0.86
Information Science 446 621 -0.13 0.25 0.06 -0.38
Atmospheric Science 490 673 0.24 0.59 0.41 -0.35
Environ Science 493 615 0.26 0.21 0.23 0.06
Geochemistry 514 657 0.44 0.48 0.46 -0.05
Geology 495 625 0.28 0.27 0.27 0.01
Geophysics 487 676 0.21 0.61 0.41 -0.40
Paleontology 531 621 0.58 0.25 0.41 0.33
Meteology 470 663 0.07 0.52 0.30 -0.46
Epidemiology 485 610 0.19 0.17 0.18 0.02
Immunology 492 662 0.25 0.52 0.38 -0.26
Nursing 452 531 -0.08 -0.35 -0.22 0.27
Actuarial Science 460 726 -0.02 0.94 0.46 -0.96
Applied Math 487 730 0.21 0.97 0.59 -0.76
Mathematics 523 740 0.51 1.03 0.77 -0.52
Probability & Stats 486 728 0.20 0.95 0.58 -0.75
Math, Other 474 715 0.10 0.87 0.48 -0.77
Astronomy 525 706 0.53 0.81 0.67 -0.28
Astrophysics 540 727 0.66 0.95 0.80 -0.29
Atomic Physics 522 739 0.50 1.03 0.77 -0.52
Nuclear Physicsl 506 715 0.37 0.87 0.62 -0.50
Optics 495 729 0.28 0.96 0.62 -0.68
Physics 540 743 0.66 1.05 0.85 -0.40
Planetary Science 545 694 0.70 0.73 0.71 -0.03
Solid State Physics 514 743 0.44 1.05 0.74 -0.62
Physics, Other 519 723 0.48 0.92 0.70 -0.44
Chemical Engineering 490 729 0.24 0.96 0.60 -0.72
Civil Engineering 456 705 -0.05 0.80 0.38 -0.85
Computer Engineering 465 716 0.03 0.87 0.45 -0.85
Electrical Engineering 465 722 0.03 0.91 0.47 -0.89
Industrial Engineering 426 699 -0.30 0.76 0.23 -1.06
Operations Research 483 743 0.18 1.05 0.61 -0.88
Materials Science 509 728 0.39 0.95 0.67 -0.56
Mechanical Engineering 471 721 0.08 0.91 0.49 -0.83
Aerospace Engineering 498 725 0.30 0.93 0.62 -0.63
Biomedical Engineering 504 717 0.35 0.88 0.62 -0.53
Nuclear Engineering 500 720 0.32 0.90 0.61 -0.58
Petroleum Engineering 414 676 -0.40 0.61 0.10 -1.01
Anthropology 532 562 0.59 -0.15 0.22 0.73
Economics 508 707 0.39 0.81 0.60 -0.43
International Relations 531 588 0.58 0.03 0.30 0.55
Political Science 523 574 0.51 -0.07 0.22 0.58
Clinical Psychology 484 554 0.18 -0.20 -0.01 0.38
Cognitive Psychology 532 627 0.59 0.28 0.44 0.30
Community Psychology 441 493 -0.18 -0.60 -0.39 0.43
Counseling Psychology 444 500 -0.15 -0.56 -0.35 0.41
Developmental Psychology 476 563 0.12 -0.14 -0.01 0.26
Psychology 476 546 0.12 -0.25 -0.07 0.37
Quantitative Psychology 515 629 0.45 0.30 0.37 0.15
Social Psychology 518 594 0.47 0.07 0.27 0.40
Sociology 490 541 0.24 -0.28 -0.02 0.52
Criminal Justice/Criminology 418 477 -0.37 -0.71 -0.54 0.34
Art history 536 549 0.62 -0.23 0.20 0.85
Music History 536 596 0.62 0.08 0.35 0.54
Drama 514 541 0.44 -0.28 0.08 0.72
Music History 490 559 0.24 -0.17 0.03 0.40
Creative Writing 553 540 0.76 -0.29 0.24 1.06
Classical Language 619 633 1.32 0.32 0.82 0.99
Russian 584 611 1.03 0.18 0.60 0.85
American History 533 541 0.60 -0.28 0.16 0.88
European History 554 555 0.77 -0.19 0.29 0.97
History of Science 596 661 1.13 0.51 0.82 0.62
Philosophy 591 630 1.08 0.30 0.69 0.78
Classics 609 616 1.24 0.21 0.72 1.02
Comp Lit 591 588 1.08 0.03 0.56 1.06
Linguistics 566 630 0.87 0.30 0.59 0.57
Elementary Education 438 520 -0.20 -0.42 -0.31 0.22
Early Childhood Education 420 497 -0.35 -0.58 -0.46 0.22
Secondary Education 484 576 0.18 -0.05 0.07 0.24
Special Education 424 497 -0.32 -0.58 -0.45 0.26
Physical Education 389 487 -0.61 -0.64 -0.63 0.03
Finance 466 721 0.03 0.91 0.47 -0.87
Business Adminstraiton 434 570 -0.24 -0.09 -0.16 -0.14
Communication 458 517 -0.03 -0.44 -0.24 0.41
Theology 537 583 0.63 -0.01 0.31 0.64
Social Work 428 463 -0.29 -0.80 -0.54 0.52
(Republished from Discover/GNXP by permission of author or representative)
 
• Category: Science • Tags: Data Analysis, GRE, Intelligence, Social Science 
🔊 Listen RSS

It isn’t too difficult to find GRE scores by intended major online. In reviewing articles/posts for my post below on anthropology I noted the distinction made between quant & qual methods, and aversions to regressions and scatter plots (or the supposed love of biological anthropologists for these tools). That got me wondering about the average mathematical and verbal aptitudes of those who intend to pursue graduate work in anthropology. I removed some extraneous disciplines which I don’t think add anything, and naturally I created three scatter plots, quantitative score vs. verbal score, writing score vs. verbal score, and writing score vs. quantitative score.

I was more interested in the spatial relationships between disciplines. But, I was a but surprised by the low correlations between quant and verbal scores at the level of disciplines. On the individual level there’s naturally some correlation. People who score very high in one are unlikely to score very low in another. That’s why the variance in scores of a simple 10 word vocabulary test can predict 50% of the variance in general intelligence. In any case, here are the r-squareds:

quant-verbal = 0
writing-verbal = 0.81
writing-quant = 0.08

So 81% of the variance in writing scores on the scale of disciplines can be explained by verbal scores. Below are the three scatter plots:


gre1gre2gre3

Some observations:

- Social work people have more EQ than IQ (this is not a major achievement because of the scale obviously).

- Accountants never made it into the “blue bird” reading group.

- Philosophers are the smartest humanists, physicists the smartest scientists, economists the smartest social scientists.

- Yes, anthropologists can read and write far better than they can do math.

The raw data below.

Major Verbal Quant Writing
Philosophy 589 636 5.1
English 559 552 4.9
History 543 556 4.8
Art History 538 554 4.7
Religion 538 583 4.8
Physics 534 738 4.5
Anthropology 532 571 4.7
Foreign Language 529 573 4.6
Political Science 522 589 4.8
Economics 504 706 4.5
Math 502 733 4.4
Earth Science 495 637 4.4
Engineering, Materials 494 729 4.3
Biology 491 632 4.4
Art & Performance 489 571 4.3
Chemistry 487 682 4.4
Sociology 487 545 4.6
Education, Secondary 486 577 4.5
Engineering, Chemical 485 727 4.3
Architecture 477 614 4.3
Banking & Finance 476 709 4.3
Communications 470 533 4.5
Psychology 470 543 4.5
Computer Science 469 704 4.2
Engineering, Mechanical 467 723 4.2
Education, Higher 465 548 4.6
Agriculture 461 596 4.2
Engineering, Electrical 461 728 4.1
Engineering, Civil 457 702 4.2
Public Administration 452 513 4.3
Education, Elementary 443 527 4.3
Engineering, Industrial 440 710 4.1
Business Administration 439 562 4.2
Social Work 428 468 4.1
Accounting 415 595 3.9
(Republished from Discover/GNXP by permission of author or representative)
 
• Category: Science • Tags: Anthropology, Data Analysis, GRE 
No Items Found
Razib Khan
About Razib Khan

"I have degrees in biology and biochemistry, a passion for genetics, history, and philosophy, and shrimp is my favorite food. If you want to know more, see the links at http://www.razib.com"