The Unz Review: An Alternative Media Selection
A Collection of Interesting, Important, and Controversial Perspectives Largely Excluded from the American Mainstream Media
 TeasersiSteve Blog
How to Fix the SAT / ACT
🔊 Listen RSS
Email This Page to Someone

 Remember My Information



=>

Bookmark Toggle AllToCAdd to LibraryRemove from Library • BShow CommentNext New CommentNext New ReplyRead More
ReplyAgree/Disagree/Etc. More... This Commenter This Thread Hide Thread Display All Comments
AgreeDisagreeLOLTroll
These buttons register your public Agreement, Disagreement, Troll, or LOL with the selected comment. They are ONLY available to recent, frequent commenters who have saved their Name+Email using the 'Remember My Information' checkbox, and may also ONLY be used once per hour.
Ignore Commenter Follow Commenter
Search Text Case Sensitive  Exact Words  Include Comments
List of Bookmarks

America’s two college admissions tests, the SAT and the ACT, have a duopoly that has become more competitive in this century, with unfortunate results. Competition usually produces more of what the customer wants, but what the customers (the parents of the kids taking the tests) want is higher scores. And so do the customers of the customers (the colleges, which are ranked in part on their students’ test scores).

In the old days, the SAT (coasts) and the ACT (heartland) had regional semi-monopolies. Moreover, they both saw themselves, during the Cold War, as being entrusted with the nation’s future, so they delivered quite impressive tests for the technology available in the mid-20th Century.

Over time, however, both have responded to market pressures for higher scores. For example, in 1995 about 70 or 80 points were added to the previously quite hard SAT Verbal test. The 2017 renovation of the SAT appears to have boosted scores another 50 points. Similarly, something is going on with the ACT, judging from all the perfect scores this year.

Meanwhile, neither test has upgraded to adaptive testing, which varies the difficulty of questions in regard to how well the test-taker has done on previous questions. Both SAT and ACT tests are now generally taken online, but the question mix is still static.The problem with static testing, of course, is that the question selection is usually pitched at the middle of the bell curve, so it doesn’t do as good a job distinguishing between people on the right tail as it could. From the NYT in 2018:

There’s talk that the online test might one day become adaptive. What does that mean?

Adaptive tests adjust the level of questioning according to how the test taker performs on prior questions, so that low scorers are asked fewer of the hardest questions and high scorers don’t need to waste as much time on easy ones. That kind of test can provide a more detailed picture of what students have mastered. It also means test takers get different questions in a different order from one another and from previous exams that may have been leaked or stolen.

Does anyone use these adaptive tests?

The GRE and G.M.A.T. graduate exams are adaptive tests, as is the Smarter Balanced test some states use to measure Common Core skills in grades 3 to 11. Language placement exams are often adaptive, as are licensing exams for pharmacists, accountants, paramedics and other professions.

What’s the holdup on going adaptive?

Just to go online has required exhaustive studies and statistical analysis to ensure comparability of a paper exam score with one from a computer, time the test loading and scrolling speeds of various laptops, and assure that computerized testing doesn’t work to the advantage of some groups of students over others.

Beyond that, adaptive testing also demands a much larger store of potential questions.

So, here’s my suggestion:

First, implement adaptive testing.

Second, keep the scoring the same as today, but just raise the maximum. Currently, the SAT is scored from 400 to 1600 and the ACT from 12 to 36. So, just add another standard deviation of headroom: score the SAT from 400 to 1800 and the ACT from 12 to 40.

Keep the current scoring the same: somebody who gets a 1200 on the current test will still get a 1200. A college whose average is 1200 will still average 1200. The only difference will be at the high end, where much finer and more revealing gradations will be possible.

 
Hide 56 CommentsLeave a Comment
Commenters to Ignore...to FollowEndorsed Only
Trim Comments?
  1. THERE IS NO SCIENCE ON THE SAT. PUT SCIENCE BACK IN, PLEASE.

    Many Americans don’t know or understand much science, because it is not taught well in schools and they took out science section from the SAT.

    • Agree: Endgame Napoleon
    • Replies: @Galactic Overlord
    Actually, science wasn't taken out of the SAT. The SAT has never had a science section.
    , @Redneck farmer
    SAT has always been math and English. Also, define "science". For example, there are relatively few countries that beat the US on the biology portions of science questions of international tests. But you don't know about that, because that would imply the US education system isn't quite as bad as some say it is. The fact that Americans do relatively poorly on physics, when they may have not yet taken a physics course is given more weight than it should.
    , @Alec Leamas
    It's the Scholastic Aptitude Test, not the Test of Stuff I Learned in High School.
  2. How can we get people with different levels of intelligence to score the same?

  3. “So, just add another standard deviation of headroom: score the SAT from 400 to 1800 and the ACT from 12 to 40.”

    You trying to make Ed Realist rich and Tiger Moms and Cubs stroke out from overwork?

    • Replies: @International Jew
    How would that make EducationRealist rich? He teaches test preparation? If so, then from what I've see, he wouldn't be of much use to anyone above the 75th percentile.
  4. Somebody will point out that dynamic headroom and signal to noise ratio, in terms of scholastic performance for 17-year-olds, amounts to evaluating them as though they were machines, like they are radiotrons and not human beings.

    I might be sympathetic to such an argument Steve.

  5. The standards have to be standards.

    The Romans had their XII Tables originally, covering weights and measures. I remember seeing the Imperial Standard Inch, Foot and Yard displayed on the wall of the City Chambers (City Hall) of Glasgow and I daresay that within were certified standard weights.

    If the evidence exists to distribute test results predictably, honestly and fairly then no grade inflation should be seen to occur.

    Not everyone was born to produce monographs on Greek sculpture, win the US Masters, play guitar like John Fahey etc.

    Like Michael Corleone says in The Godfather “Don’t insult my intelligence, Carlo.”

  6. Any standardized test now is completely invalid because any child whose parents get a note stating the child is ADHD, or whatever psychological syndrome is in vogue, gets unlimited time to take the test – and no reference is allowed to be made to this in the scoring results, thanks to the Americans with “Disabilities” Act. This is not hard, it only takes a doctor’s note and parents can get them easily. What percentage of perfect scores had this time advantage?

    The only way to fix the SAT or any standardized test is to give everyone unlimited time. Until then the tests are invalid.

  7. “..keep the scoring the same as today, but just raise the maximum…”

    Nigel understood this concept and explains it well here:

  8. Pretty sure the whole point of the last quarter century of changes was to obscure those finer gradations at the top. They would tell a story the customers don’t want to hear.

    • Agree: Triumph104
    • Replies: @Calvin Hobbes

    Pretty sure the whole point of the last quarter century of changes was to obscure those finer gradations at the top. They would tell a story the customers don’t want to hear.
     
    Lots of unwelcome gradations at the top here:

    https://www.mathcounts.org/sites/default/files/2019%20National%20Final%20Standings_0.pdf

    and here:

    http://economics.mit.edu/files/4298

    The AMC tests discussed in the second link and also here:

    https://www.ivyzen.net/amc/

    have lots of headroom.

    , @Will

    Pretty sure the whole point of the last quarter century of changes was to obscure those finer gradations at the top. They would tell a story the customers don’t want to hear.
     
    This.
  9. Nice idea, though I see some issues.

    1. With different people in effect taking different tests, there’d be less transparency, and more room for arbitrariness, in how scores come about. One person could get an 800 by doing great on the easier “basic” questions but terrible on the “adaptive” hard questions, while someone else does less well on the easy questions but shows some brilliance on a few hard questions.

    2. The “manufacture” of SAT questions is an expensive process. Inventing the questions is just the beginning. What follows that is statistical assessment of the question’s usefulness as a discriminator (in the statistical, not SJW sense), a problem that becomes a good deal harder if we go to an adaptive test.

    Or maybe I should look on the bright side and see these issues as a stimulus to the hiring of well-trained mathematical statisticians.

    • Replies: @Lot
    GMAT has used computer adaptive testing for 15+ years.

    It works better.
    , @Jon

    One person could get an 800 by doing great on the easier “basic” questions but terrible on the “adaptive” hard questions, while someone else does less well on the easy questions but shows some brilliance on a few hard questions.
     
    With adaptive testing, would the second example ever get the chance to show their brilliance? Adaptive tests weren't around when I was going through, but that would be my worry -- a couple of careless errors and now you are on track for a mediocre score.
    , @EH
    That's not how scoring works in adaptive testing. Each question and test-taker is measured on the same scale of difficulty/ability, so that a person of a particular ability score has a 50/50 chance of getting right a question with the same difficulty score. For each test administration, the questions start with an average-difficulty question (or other best estimate of test-taker ability). If the answer is wrong, an easier question is tried, if right then a harder question is asked. When a level is reached where the odds of getting a question right are not distinguishable from 50%, that's the score.

    (In simplest form. Question banks run out of questions of a given difficulty, but you can use harder questions as long as that test-taker gets enough of them right to be able to quantify the percentage gotten right at higher difficulty levels, together with a knowledge of the the questions' statistical ability to discriminate between ability levels, which is different from the difficulty level of the question in the same way a standard deviation is different from an average.)

    Getting the abilities and difficulties on the same scale is called a "Rasch measure" and it is done by making a (often sparse, with many blanks) binary-valued (right/wrong) matrix of test-taker rows and question columns. Then there's more math - something about a log-transformation, but it's all been done routinely since the 60s and 70s.

    , @AnotherDad

    1. With different people in effect taking different tests, there’d be less transparency, and more room for arbitrariness, in how scores come about. One person could get an 800 by doing great on the easier “basic” questions but terrible on the “adaptive” hard questions, while someone else does less well on the easy questions but shows some brilliance on a few hard questions.
     
    LOL.

    The whole point of adaptive testing is that if you start nailing the questions at any level--you "level up" and get tougher questions; and if you keep missing questions at a level you go down a level and get easier questions.

    The test finds the level where you're nailing some questions and missing others--and that's your score, not how many total questions you got right.

    The beauty of it--especially for the world we live in now--is that it can quickly find the rough level of difficulty where a student both gets and misses questions and then absolutely pepper him at that level. So it's actually giving the student a real *test*. If done properly adaptive testing can discriminate between the real 750 kid and the 650 kid who is well test prepped, and maybe in the old static test could get an extra 5 or 10 questions right beyond his smarts, but now is hit with 40 or 50 questions at his level revealing his limitations.
  10. you’re thinking about it the wrong way. our modern media in fact would say that’s the white way of thinking about these tests.

    the demand was not for higher performance. the demand was for more equal performance.

    the point of compressing the range of scores was to eliminate the ability discrimination function of the test.

    they are doing the same thing to SAT that they did to the civil service exams. reduce the difficulty enough so that a lot of the test takers appear to be capable, then pick admissions at random from that pool.

    raise the ceiling of the test and the low scorers will demand bonus points, along with their lawyers and all the politicians on their side.

    this is, truly, the proverbial zero sum game.

    not equality of opportunity. equality of outcome.

    • Agree: Cortes
  11. @Lot
    “So, just add another standard deviation of headroom: score the SAT from 400 to 1800 and the ACT from 12 to 40.”

    You trying to make Ed Realist rich and Tiger Moms and Cubs stroke out from overwork?

    How would that make EducationRealist rich? He teaches test preparation? If so, then from what I’ve see, he wouldn’t be of much use to anyone above the 75th percentile.

    • Replies: @Lot
    Her test prep clients are nearly all asians.
  12. @International Jew
    How would that make EducationRealist rich? He teaches test preparation? If so, then from what I've see, he wouldn't be of much use to anyone above the 75th percentile.

    Her test prep clients are nearly all asians.

  13. The easiest solution would just be for especially bright high schoolers to start taking the GRE in addition to the SAT or ACT.

    It’s basically just a souped-up critical thinking test, like the SAT. But doing well on the GRE would distinguish you from the run-of-the-mill perfect SAT scorer. Plus, it’s already been normed and, presumably, reverse engineered to maximize female and POC scores within the universe of the potential questions that have any predictive validity.

    • Replies: @dvorak

    The easiest solution would just be for especially bright high schoolers to start taking the GRE in addition to the SAT or ACT.
     
    The GRE and GMAT are still too easy. What you are looking for is the LSAT, especially its logic games section. Hoo mama.
    , @StemPete
    Have you seen the histogram of GRE scores? It has a tight right hand tail and a long left hand tail to give fine grain cutoff points for universities to reject students. It is not designed to find genius.
  14. @International Jew
    Nice idea, though I see some issues.

    1. With different people in effect taking different tests, there'd be less transparency, and more room for arbitrariness, in how scores come about. One person could get an 800 by doing great on the easier "basic" questions but terrible on the "adaptive" hard questions, while someone else does less well on the easy questions but shows some brilliance on a few hard questions.

    2. The "manufacture" of SAT questions is an expensive process. Inventing the questions is just the beginning. What follows that is statistical assessment of the question's usefulness as a discriminator (in the statistical, not SJW sense), a problem that becomes a good deal harder if we go to an adaptive test.

    Or maybe I should look on the bright side and see these issues as a stimulus to the hiring of well-trained mathematical statisticians.

    GMAT has used computer adaptive testing for 15+ years.

    It works better.

  15. @Rational
    THERE IS NO SCIENCE ON THE SAT. PUT SCIENCE BACK IN, PLEASE.

    Many Americans don't know or understand much science, because it is not taught well in schools and they took out science section from the SAT.

    Actually, science wasn’t taken out of the SAT. The SAT has never had a science section.

    • Replies: @Jon
    Yeah, I think he is confusing SAT and ACT. As far back as I remember, the SAT just had Math and Verbal.
  16. Anon[267] • Disclaimer says:

    I could have sworn that I read an article somewhere that the College Board had spent years making a new adaptive SAT, and had trialed it in Hong Kong (apperently where you go to test tests, because the world’s more aggressive cheating gangs are there), but the test was somehow compromised, and they dropped it and gave up.

    I can’t pull this up in Google, but it may be because there is so much other stuff about Chinese SAT cheating.

  17. having said that, an adaptive test could be, the ultimate academic test. but the old SAT was already more than good enough. it was developed by geniuses who really, really knew what they were doing. it had a range beyond the wechsler. and it sorted teenagers into 100 different ability categories exceedingly well. 1 million people took it every year, and only a few were able to get everything right. that’s more than sufficient.

    an adaptive test, if developed by the same serious researchers, with zero attention paid to modern year complaints, could make something totally incredible. a test capable of finely discerning ability over almost the entire human spectrum of capability. single point of capability differentiation anywhere on the human brainpower scale you wanted to investigate. answering questions that even current tests couldn’t answer.

    the really dumb humans. zeroing in on whether some guy has a capability of 52, or 53. making the test so short and accessible that even morons could engage it, and do their best. answering questions about the capabilities of the humans in the super low range, questions which aren’t important to answer for us, but are of academic interest for purposes of brain damage, genetic disorders, developmental psychology.

    the super smart guys. going beyond the wechsler 170 equivalent of the old SAT. raising the ceiling to 180, or 190. every year about 15 or 20 people answered every question correctly on the old SAT. what if you pushed them? like star athletes in sports. who REALLY is the smartest? every year some team wins the superbowl, some guy wins wimbledon, somebody is the WBC boxing champion, some guy is the 400 meter world champion. but it’s obvious to everybody, the champions aren’t all equal, year to year. who was actually better?

    there is no such thing as a perfect score in decathlon – the scoring in unlimited. there is no ceiling. should there even be a ceiling that a human could reach on a ron hoeflin designed intelligence test? should there even be such a thing as a perfect SAT score? should it simply be, how far can you push it, like any other world record. perhaps practical concerns about time constraints would be the only limiting factor. certainly you can’t have the smartest 18 year old in america sitting there, answering questions for 12 hours, trying to exhaust the computer’s collection of ultra difficult problems, while the average guy is done in 45 minutes. the test would turn into a video game.

  18. Anon[267] • Disclaimer says:

    The way that these tests are clustering the smart kids at the top with perfect scores is just a trick to reduce “the gap,” as explained by La Griffe:

    http://www.lagriffedulion.f2s.com/adverse.htm

    Prodigy: Though we know the achievement gap is some function of student proficiency (measured here by the white pass rate), we do not yet know the function’s details. We have learned, however, that because the gap vanishes at its endpoints and is nonzero at points in between, as students improve there will be times when the pass-rate difference will increase and other times when it will decrease. And this variation in the gap, Judge, has nothing to do with real relative changes in group performance. The difference between the mean scores of blacks and whites could remain perfectly constant, while all this is taking place. That is, the perceived gap variation could simply be a mathematical artifact!

    Judge: Do you mean that there was no real gap reduction in Flyover?

    Prodigy:
    Be patient, Judge, all will come out in the wash. First we need to find the function depicted in my sketch. It should not be difficult.

  19. @Desiderius
    Pretty sure the whole point of the last quarter century of changes was to obscure those finer gradations at the top. They would tell a story the customers don’t want to hear.

    Pretty sure the whole point of the last quarter century of changes was to obscure those finer gradations at the top. They would tell a story the customers don’t want to hear.

    Lots of unwelcome gradations at the top here:

    https://www.mathcounts.org/sites/default/files/2019%20National%20Final%20Standings_0.pdf

    and here:

    http://economics.mit.edu/files/4298

    The AMC tests discussed in the second link and also here:

    https://www.ivyzen.net/amc/

    have lots of headroom.

    • Replies: @Will
    The SAT is deeply broken on math with a terribly low ceiling. (Which as noted above, the AMC competitions solve for anyone applying to Caltech / MIT.)

    Their real problem is they removed the single most g loaded verbal item, analogies, and took out the guessing penalty. This makes scores more random, fails to reward a sense of your own confidence in getting down to three answers, etc.
  20. “Currently, the SAT is scored from 400 to 1600 and the ACT from 12 to 36. So, just add another standard deviation of headroom: score the SAT from 400 to 1800 and the ACT from 12 to 40.”

    Why is this a problem that needs to be solved? The top test takers (36 on ACT, say, the top 2,000 test takers) aren’t properly distinguished to make it easy for the top 10 or so colleges to rank them.
    But the freshman class of those 10 colleges is much larger than 2,000, so that extra distinguishing isn’t really necessary-those 10 colleges have a vast majority of freshman that are already distinguished well enough (they score 35 to say 31 or so).

    Making Harvard’s job easier isn’t something that requires very much attention from the rest of us.

    joe

    • Replies: @Anon
    Honestly, things like these corporate-sponsored national math and science competitions have an ethnic bias in that certain ethnic groups know about them more than others and consider them more important than others. It's sort of like the way Cambodians pass around donut shop information, and Indians with motels.

    Maybe some math teachers push their students into these, but students have to care, and parents have to think it's worth flying to Florida with the sprog to do something that is worth zip city on a college application unless you get a really high placement, and since the SAT and AP are pretty decent math gauntlets, how much more does one of these competitions add?

    I think they are more useful for minorities, because, especially with the poster-based science ones, they show a tiny bit of public speaking skill and lack of stage fright, which is useful to Ivies and other universities for showing you're not just another generic chank.
    , @Anon

    Why is this a problem that needs to be solved?
     
    I'm not sure that it constitutes a market, but it would be nice to know who the super genius kids are. If you look at the perfect scoreers and then sort by personality and extracuriculars, you will miss the Einsteins if they are socially maladjusted and politically unwoke. Caltech might want them anyway. But if not, some small college might decide to go full brainiac and pay them to attend.
    , @Hibernian
    Have a second test for further screening 0f geniuses.
  21. No worries.

    We ‘re getting there:

  22. Both SAT and ACT tests are now generally taken online

    As someone who proctored the SAT just this last year, this isn’t a true statement. The SAT is taken solely via pencil-and-paper

  23. Degrading the top scores might be doing more good than harm in the long run if it mixes together the top 10% or 20% together in the same schools. Steve has argued that the main, if hidden, conflict today is the right end of the curve against everybody dumber than them.

    Bringing back the finer gradations of the past can only worsen that. I mean, aren’t they what got us into this mess in the first place?

  24. Anon[267] • Disclaimer says:
    @joeyjoejoe
    "Currently, the SAT is scored from 400 to 1600 and the ACT from 12 to 36. So, just add another standard deviation of headroom: score the SAT from 400 to 1800 and the ACT from 12 to 40."

    Why is this a problem that needs to be solved? The top test takers (36 on ACT, say, the top 2,000 test takers) aren't properly distinguished to make it easy for the top 10 or so colleges to rank them.
    But the freshman class of those 10 colleges is much larger than 2,000, so that extra distinguishing isn't really necessary-those 10 colleges have a vast majority of freshman that are already distinguished well enough (they score 35 to say 31 or so).

    Making Harvard's job easier isn't something that requires very much attention from the rest of us.

    joe

    Honestly, things like these corporate-sponsored national math and science competitions have an ethnic bias in that certain ethnic groups know about them more than others and consider them more important than others. It’s sort of like the way Cambodians pass around donut shop information, and Indians with motels.

    Maybe some math teachers push their students into these, but students have to care, and parents have to think it’s worth flying to Florida with the sprog to do something that is worth zip city on a college application unless you get a really high placement, and since the SAT and AP are pretty decent math gauntlets, how much more does one of these competitions add?

    I think they are more useful for minorities, because, especially with the poster-based science ones, they show a tiny bit of public speaking skill and lack of stage fright, which is useful to Ivies and other universities for showing you’re not just another generic chank.

    • Replies: @Anonymous

    Honestly, things like these corporate-sponsored national math and science competitions have an ethnic bias in that certain ethnic groups know about them more than others and consider them more important than others. It’s sort of like the way Cambodians pass around donut shop information, and Indians with motels.
     
    Cmbodian donut shops and Indian motels ("Patels" we call them) are not particularly good and best avoided in most cases if possible.

    Most supermarkets with bakery departments make better donuts than all but a select few donut shops. Almost always these shops are owned by Core Americans who have been in the business forever.

    We have chains here, Lamars and Krispy Kreme. Lamars is ok but overpriced. KK is yecch.
  25. Anon[267] • Disclaimer says:
    @joeyjoejoe
    "Currently, the SAT is scored from 400 to 1600 and the ACT from 12 to 36. So, just add another standard deviation of headroom: score the SAT from 400 to 1800 and the ACT from 12 to 40."

    Why is this a problem that needs to be solved? The top test takers (36 on ACT, say, the top 2,000 test takers) aren't properly distinguished to make it easy for the top 10 or so colleges to rank them.
    But the freshman class of those 10 colleges is much larger than 2,000, so that extra distinguishing isn't really necessary-those 10 colleges have a vast majority of freshman that are already distinguished well enough (they score 35 to say 31 or so).

    Making Harvard's job easier isn't something that requires very much attention from the rest of us.

    joe

    Why is this a problem that needs to be solved?

    I’m not sure that it constitutes a market, but it would be nice to know who the super genius kids are. If you look at the perfect scoreers and then sort by personality and extracuriculars, you will miss the Einsteins if they are socially maladjusted and politically unwoke. Caltech might want them anyway. But if not, some small college might decide to go full brainiac and pay them to attend.

  26. @Hypnotoad666
    The easiest solution would just be for especially bright high schoolers to start taking the GRE in addition to the SAT or ACT.

    It's basically just a souped-up critical thinking test, like the SAT. But doing well on the GRE would distinguish you from the run-of-the-mill perfect SAT scorer. Plus, it's already been normed and, presumably, reverse engineered to maximize female and POC scores within the universe of the potential questions that have any predictive validity.

    The easiest solution would just be for especially bright high schoolers to start taking the GRE in addition to the SAT or ACT.

    The GRE and GMAT are still too easy. What you are looking for is the LSAT, especially its logic games section. Hoo mama.

    • Replies: @Jon
    Back around the turn of the century, I spent some time teaching for Kaplan. One of the fun facts I picked up is that the GRE Math section was easier than the SAT. Outside of STEM majors, there is very little math required, so in math the average high schooler is actually better prepared than the average colleg grad. As Trump would say: Sad!
  27. Degrading the top scores might be doing more good than harm in the long run if it mixes together the top 10% or 20% together in the same schools.

    MIT, Harvard, etc. have a variety of ways to find the top of the top, of whatever they are looking for.

  28. I like Steve’s plan, but he’s confused. Adaptive Testing that provides accurate information about the top end students is exactly what everyone (and I mean everyone) is trying to avoid.

    Here’s what happened. Your daughter is in middle school. It’s 1980. You realize she’s just not getting A’s in math. To get into the best colleges, she needs A’s across the board. Whose fault is this?

    You and your wife go to the principal. You tell him in no uncertain terms that your daughter should be getting A’s. He mumbles something about standardized tests show that on average there are two boys with 130 IQs for every girl. There are two boys with 700+ SAT math scores for every girl. She’s fine.

    The missus shakes her head. Fix math so girls will do better. Require students to show their penmanship (work) for all problems. Give more points for social skills (group work in class). More points for clerical work (maintaining a neat notebook). Extra Credit!! Lots of that.

    Oh, and when brilliant boys show up who just blurt out correct answers after glancing at problems, give them C’s, that’ll fix’em (this actually happened to my son, now working on a math PhD).

    Standardized tests should reflect what parents want! They’re the ones paying the bills, mind you. After a generation of hard work, girls are as good as boys at math. Grades prove it and grades don’t lie. Girls get better grades in high school math than boys. Look it up. Yay!

    The SAT and ACT have been fixed and now Steve Sailer wants to rock the boat. Standardized tests are supposed to bury the truth, not dig it up.

    • Replies: @Redneck farmer
    To quote Warren Zevon: "Abandoned all hope and don't rock the boat/ and we'll all make a few hundred grand"
  29. Anonymous[427] • Disclaimer says:
    @Anon
    Honestly, things like these corporate-sponsored national math and science competitions have an ethnic bias in that certain ethnic groups know about them more than others and consider them more important than others. It's sort of like the way Cambodians pass around donut shop information, and Indians with motels.

    Maybe some math teachers push their students into these, but students have to care, and parents have to think it's worth flying to Florida with the sprog to do something that is worth zip city on a college application unless you get a really high placement, and since the SAT and AP are pretty decent math gauntlets, how much more does one of these competitions add?

    I think they are more useful for minorities, because, especially with the poster-based science ones, they show a tiny bit of public speaking skill and lack of stage fright, which is useful to Ivies and other universities for showing you're not just another generic chank.

    Honestly, things like these corporate-sponsored national math and science competitions have an ethnic bias in that certain ethnic groups know about them more than others and consider them more important than others. It’s sort of like the way Cambodians pass around donut shop information, and Indians with motels.

    Cmbodian donut shops and Indian motels (“Patels” we call them) are not particularly good and best avoided in most cases if possible.

    Most supermarkets with bakery departments make better donuts than all but a select few donut shops. Almost always these shops are owned by Core Americans who have been in the business forever.

    We have chains here, Lamars and Krispy Kreme. Lamars is ok but overpriced. KK is yecch.

    • Replies: @Anon
    I worked in a donut shop. Donuts are donuts. They all use the same mixes from the same makers. There are four kinds: cake, raised, buttermilk, and French. The only distinction is if you make them fresh throughout the day, which Krispy Kreme does with raised glazed. You can form the donuts by hand cutting them or with an extruder, but the taste is the same.
  30. @Rational
    THERE IS NO SCIENCE ON THE SAT. PUT SCIENCE BACK IN, PLEASE.

    Many Americans don't know or understand much science, because it is not taught well in schools and they took out science section from the SAT.

    SAT has always been math and English. Also, define “science”. For example, there are relatively few countries that beat the US on the biology portions of science questions of international tests. But you don’t know about that, because that would imply the US education system isn’t quite as bad as some say it is. The fact that Americans do relatively poorly on physics, when they may have not yet taken a physics course is given more weight than it should.

    • Replies: @Realist

    For example, there are relatively few countries that beat the US on the biology portions of science questions of international tests.
     
    So being an 'also ran' in biological science is a good thing???

    The fact that Americans do relatively poorly on physics, when they may have not yet taken a physics course is given more weight than it should.
     
    That is the point, same can be said for chemistry and math. The fact that American education sucks is the problem.
  31. @Anon7
    I like Steve’s plan, but he’s confused. Adaptive Testing that provides accurate information about the top end students is exactly what everyone (and I mean everyone) is trying to avoid.

    Here’s what happened. Your daughter is in middle school. It’s 1980. You realize she’s just not getting A’s in math. To get into the best colleges, she needs A’s across the board. Whose fault is this?

    You and your wife go to the principal. You tell him in no uncertain terms that your daughter should be getting A’s. He mumbles something about standardized tests show that on average there are two boys with 130 IQs for every girl. There are two boys with 700+ SAT math scores for every girl. She’s fine.

    The missus shakes her head. Fix math so girls will do better. Require students to show their penmanship (work) for all problems. Give more points for social skills (group work in class). More points for clerical work (maintaining a neat notebook). Extra Credit!! Lots of that.

    Oh, and when brilliant boys show up who just blurt out correct answers after glancing at problems, give them C’s, that’ll fix’em (this actually happened to my son, now working on a math PhD).

    Standardized tests should reflect what parents want! They’re the ones paying the bills, mind you. After a generation of hard work, girls are as good as boys at math. Grades prove it and grades don’t lie. Girls get better grades in high school math than boys. Look it up. Yay!

    The SAT and ACT have been fixed and now Steve Sailer wants to rock the boat. Standardized tests are supposed to bury the truth, not dig it up.

    To quote Warren Zevon: “Abandoned all hope and don’t rock the boat/ and we’ll all make a few hundred grand”

  32. @Galactic Overlord
    Actually, science wasn't taken out of the SAT. The SAT has never had a science section.

    Yeah, I think he is confusing SAT and ACT. As far back as I remember, the SAT just had Math and Verbal.

  33. Jon says:
    @dvorak

    The easiest solution would just be for especially bright high schoolers to start taking the GRE in addition to the SAT or ACT.
     
    The GRE and GMAT are still too easy. What you are looking for is the LSAT, especially its logic games section. Hoo mama.

    Back around the turn of the century, I spent some time teaching for Kaplan. One of the fun facts I picked up is that the GRE Math section was easier than the SAT. Outside of STEM majors, there is very little math required, so in math the average high schooler is actually better prepared than the average colleg grad. As Trump would say: Sad!

  34. Jon says:
    @International Jew
    Nice idea, though I see some issues.

    1. With different people in effect taking different tests, there'd be less transparency, and more room for arbitrariness, in how scores come about. One person could get an 800 by doing great on the easier "basic" questions but terrible on the "adaptive" hard questions, while someone else does less well on the easy questions but shows some brilliance on a few hard questions.

    2. The "manufacture" of SAT questions is an expensive process. Inventing the questions is just the beginning. What follows that is statistical assessment of the question's usefulness as a discriminator (in the statistical, not SJW sense), a problem that becomes a good deal harder if we go to an adaptive test.

    Or maybe I should look on the bright side and see these issues as a stimulus to the hiring of well-trained mathematical statisticians.

    One person could get an 800 by doing great on the easier “basic” questions but terrible on the “adaptive” hard questions, while someone else does less well on the easy questions but shows some brilliance on a few hard questions.

    With adaptive testing, would the second example ever get the chance to show their brilliance? Adaptive tests weren’t around when I was going through, but that would be my worry — a couple of careless errors and now you are on track for a mediocre score.

    • Replies: @International Jew
    The second person would need to do fairly well on the easy questions, sure. He'd get to the hard questions a little later than the first person. Of course I'm just speculating about the adaptive algorithm.
  35. Anon[213] • Disclaimer says:
    @Anonymous

    Honestly, things like these corporate-sponsored national math and science competitions have an ethnic bias in that certain ethnic groups know about them more than others and consider them more important than others. It’s sort of like the way Cambodians pass around donut shop information, and Indians with motels.
     
    Cmbodian donut shops and Indian motels ("Patels" we call them) are not particularly good and best avoided in most cases if possible.

    Most supermarkets with bakery departments make better donuts than all but a select few donut shops. Almost always these shops are owned by Core Americans who have been in the business forever.

    We have chains here, Lamars and Krispy Kreme. Lamars is ok but overpriced. KK is yecch.

    I worked in a donut shop. Donuts are donuts. They all use the same mixes from the same makers. There are four kinds: cake, raised, buttermilk, and French. The only distinction is if you make them fresh throughout the day, which Krispy Kreme does with raised glazed. You can form the donuts by hand cutting them or with an extruder, but the taste is the same.

    • Replies: @Anonymous
    How would you characterize the beignet?
  36. @Jon

    One person could get an 800 by doing great on the easier “basic” questions but terrible on the “adaptive” hard questions, while someone else does less well on the easy questions but shows some brilliance on a few hard questions.
     
    With adaptive testing, would the second example ever get the chance to show their brilliance? Adaptive tests weren't around when I was going through, but that would be my worry -- a couple of careless errors and now you are on track for a mediocre score.

    The second person would need to do fairly well on the easy questions, sure. He’d get to the hard questions a little later than the first person. Of course I’m just speculating about the adaptive algorithm.

  37. @Hypnotoad666
    The easiest solution would just be for especially bright high schoolers to start taking the GRE in addition to the SAT or ACT.

    It's basically just a souped-up critical thinking test, like the SAT. But doing well on the GRE would distinguish you from the run-of-the-mill perfect SAT scorer. Plus, it's already been normed and, presumably, reverse engineered to maximize female and POC scores within the universe of the potential questions that have any predictive validity.

    Have you seen the histogram of GRE scores? It has a tight right hand tail and a long left hand tail to give fine grain cutoff points for universities to reject students. It is not designed to find genius.

  38. Opening up the tests for higher achievement would be good for prospective Caltech students.

    More importantly, extremely high SAT scores might reveal some romantic opportunities for high school needs.

  39. Anonymous[427] • Disclaimer says:
    @Anon
    I worked in a donut shop. Donuts are donuts. They all use the same mixes from the same makers. There are four kinds: cake, raised, buttermilk, and French. The only distinction is if you make them fresh throughout the day, which Krispy Kreme does with raised glazed. You can form the donuts by hand cutting them or with an extruder, but the taste is the same.

    How would you characterize the beignet?

  40. Adaptive testing is good for finding genius but very very bad for comparing the performance of one to another as they are not tested with exactly the same test and the scoring is opened up for manipulation. May be somebody want a test where they can easily fiddle with the scores.

    • Replies: @AnotherDad

    Adaptive testing is good for finding genius but very very bad for comparing the performance of one to another as they are not tested with exactly the same test and the scoring is opened up for manipulation. May be somebody want a test where they can easily fiddle with the scores.
     
    No, it's much better at both.

    Already the SAT does not have the same questions test to test. The December test isn't the same as the November test, neither of which will be the same as March test, the May test, the June test ...
    So what you get from the SAT is a score from a test that--in theory--"norms" evenly with the other tests. But you may have gotten a higher or lower score because the test you took just happened to have a few questions on it that were particularly easy or difficult for you relative to other questions of that difficulty level.
  41. eah says:

    I guess it doesn’t look too good for test scores to be rising when ‘IQ rates are dropping’ — that’s not logical.

    NBC: “IQ Rates Are Dropping in Many Developed Countries and That Doesn’t Bode Well for Humanity”

    But:

    Over time, however, both have responded to market pressures for higher scores.

    Since we all know IQ is pseudoscience (except NBC it seems), we’ll fix the problem of rising test scores by tinkering with the testing, instead of addressing the problem of America filling up with low IQ third-worlders — first things first after all: we must get the much bigger problem of rising test scores under control.

  42. It’s time to move into Post-Collegiate America.

  43. So adaptive testing is a euphemism for lowering standards. Got it.

  44. How to Fix the SAT / ACT

    The way to fix SAT/ACT tests and college admissions to to make them merit only

  45. @Redneck farmer
    SAT has always been math and English. Also, define "science". For example, there are relatively few countries that beat the US on the biology portions of science questions of international tests. But you don't know about that, because that would imply the US education system isn't quite as bad as some say it is. The fact that Americans do relatively poorly on physics, when they may have not yet taken a physics course is given more weight than it should.

    For example, there are relatively few countries that beat the US on the biology portions of science questions of international tests.

    So being an ‘also ran’ in biological science is a good thing???

    The fact that Americans do relatively poorly on physics, when they may have not yet taken a physics course is given more weight than it should.

    That is the point, same can be said for chemistry and math. The fact that American education sucks is the problem.

  46. EH says:
    @International Jew
    Nice idea, though I see some issues.

    1. With different people in effect taking different tests, there'd be less transparency, and more room for arbitrariness, in how scores come about. One person could get an 800 by doing great on the easier "basic" questions but terrible on the "adaptive" hard questions, while someone else does less well on the easy questions but shows some brilliance on a few hard questions.

    2. The "manufacture" of SAT questions is an expensive process. Inventing the questions is just the beginning. What follows that is statistical assessment of the question's usefulness as a discriminator (in the statistical, not SJW sense), a problem that becomes a good deal harder if we go to an adaptive test.

    Or maybe I should look on the bright side and see these issues as a stimulus to the hiring of well-trained mathematical statisticians.

    That’s not how scoring works in adaptive testing. Each question and test-taker is measured on the same scale of difficulty/ability, so that a person of a particular ability score has a 50/50 chance of getting right a question with the same difficulty score. For each test administration, the questions start with an average-difficulty question (or other best estimate of test-taker ability). If the answer is wrong, an easier question is tried, if right then a harder question is asked. When a level is reached where the odds of getting a question right are not distinguishable from 50%, that’s the score.

    (In simplest form. Question banks run out of questions of a given difficulty, but you can use harder questions as long as that test-taker gets enough of them right to be able to quantify the percentage gotten right at higher difficulty levels, together with a knowledge of the the questions’ statistical ability to discriminate between ability levels, which is different from the difficulty level of the question in the same way a standard deviation is different from an average.)

    Getting the abilities and difficulties on the same scale is called a “Rasch measure” and it is done by making a (often sparse, with many blanks) binary-valued (right/wrong) matrix of test-taker rows and question columns. Then there’s more math – something about a log-transformation, but it’s all been done routinely since the 60s and 70s.

  47. @joeyjoejoe
    "Currently, the SAT is scored from 400 to 1600 and the ACT from 12 to 36. So, just add another standard deviation of headroom: score the SAT from 400 to 1800 and the ACT from 12 to 40."

    Why is this a problem that needs to be solved? The top test takers (36 on ACT, say, the top 2,000 test takers) aren't properly distinguished to make it easy for the top 10 or so colleges to rank them.
    But the freshman class of those 10 colleges is much larger than 2,000, so that extra distinguishing isn't really necessary-those 10 colleges have a vast majority of freshman that are already distinguished well enough (they score 35 to say 31 or so).

    Making Harvard's job easier isn't something that requires very much attention from the rest of us.

    joe

    Have a second test for further screening 0f geniuses.

  48. @Rational
    THERE IS NO SCIENCE ON THE SAT. PUT SCIENCE BACK IN, PLEASE.

    Many Americans don't know or understand much science, because it is not taught well in schools and they took out science section from the SAT.

    It’s the Scholastic Aptitude Test, not the Test of Stuff I Learned in High School.

  49. @International Jew
    Nice idea, though I see some issues.

    1. With different people in effect taking different tests, there'd be less transparency, and more room for arbitrariness, in how scores come about. One person could get an 800 by doing great on the easier "basic" questions but terrible on the "adaptive" hard questions, while someone else does less well on the easy questions but shows some brilliance on a few hard questions.

    2. The "manufacture" of SAT questions is an expensive process. Inventing the questions is just the beginning. What follows that is statistical assessment of the question's usefulness as a discriminator (in the statistical, not SJW sense), a problem that becomes a good deal harder if we go to an adaptive test.

    Or maybe I should look on the bright side and see these issues as a stimulus to the hiring of well-trained mathematical statisticians.

    1. With different people in effect taking different tests, there’d be less transparency, and more room for arbitrariness, in how scores come about. One person could get an 800 by doing great on the easier “basic” questions but terrible on the “adaptive” hard questions, while someone else does less well on the easy questions but shows some brilliance on a few hard questions.

    LOL.

    The whole point of adaptive testing is that if you start nailing the questions at any level–you “level up” and get tougher questions; and if you keep missing questions at a level you go down a level and get easier questions.

    The test finds the level where you’re nailing some questions and missing others–and that’s your score, not how many total questions you got right.

    The beauty of it–especially for the world we live in now–is that it can quickly find the rough level of difficulty where a student both gets and misses questions and then absolutely pepper him at that level. So it’s actually giving the student a real *test*. If done properly adaptive testing can discriminate between the real 750 kid and the 650 kid who is well test prepped, and maybe in the old static test could get an extra 5 or 10 questions right beyond his smarts, but now is hit with 40 or 50 questions at his level revealing his limitations.

  50. @StemPete
    Adaptive testing is good for finding genius but very very bad for comparing the performance of one to another as they are not tested with exactly the same test and the scoring is opened up for manipulation. May be somebody want a test where they can easily fiddle with the scores.

    Adaptive testing is good for finding genius but very very bad for comparing the performance of one to another as they are not tested with exactly the same test and the scoring is opened up for manipulation. May be somebody want a test where they can easily fiddle with the scores.

    No, it’s much better at both.

    Already the SAT does not have the same questions test to test. The December test isn’t the same as the November test, neither of which will be the same as March test, the May test, the June test …
    So what you get from the SAT is a score from a test that–in theory–“norms” evenly with the other tests. But you may have gotten a higher or lower score because the test you took just happened to have a few questions on it that were particularly easy or difficult for you relative to other questions of that difficulty level.

  51. Great post. No snark here, no sarcasm. Simply put, a great post.

    • Replies: @Desiderius
    Ok, that’s good faith. Keep a balance of praise/agreement/charity and criticism and your posting will improve immensely.
  52. What are the odds that adaptive testing on SATs, if implemented today – considering the actual goals of university admission boards and the role of universities as gatekeepers to the elite professions – would actually be used to discriminate between smart and genius-level intellect, as opposed to artificially boosting the representation of “under-represented” groups?

  53. @Corvinus
    Great post. No snark here, no sarcasm. Simply put, a great post.

    Ok, that’s good faith. Keep a balance of praise/agreement/charity and criticism and your posting will improve immensely.

    • Replies: @Corvinus
    'Ok, that’s good faith."

    Just because my opinion falls in line with yours and Mr. Sailer's that it is "good faith"? LOL.

    "Keep a balance of praise/agreement/charity and criticism..."

    Pot calling the kettle black.

    "and your posting will improve immensely."

    My posting is stellar. Tread lightly, friend.
  54. @Desiderius
    Ok, that’s good faith. Keep a balance of praise/agreement/charity and criticism and your posting will improve immensely.

    ‘Ok, that’s good faith.”

    Just because my opinion falls in line with yours and Mr. Sailer’s that it is “good faith”? LOL.

    “Keep a balance of praise/agreement/charity and criticism…”

    Pot calling the kettle black.

    “and your posting will improve immensely.”

    My posting is stellar. Tread lightly, friend.

  55. @Desiderius
    Pretty sure the whole point of the last quarter century of changes was to obscure those finer gradations at the top. They would tell a story the customers don’t want to hear.

    Pretty sure the whole point of the last quarter century of changes was to obscure those finer gradations at the top. They would tell a story the customers don’t want to hear.

    This.

  56. Will says:
    @Calvin Hobbes

    Pretty sure the whole point of the last quarter century of changes was to obscure those finer gradations at the top. They would tell a story the customers don’t want to hear.
     
    Lots of unwelcome gradations at the top here:

    https://www.mathcounts.org/sites/default/files/2019%20National%20Final%20Standings_0.pdf

    and here:

    http://economics.mit.edu/files/4298

    The AMC tests discussed in the second link and also here:

    https://www.ivyzen.net/amc/

    have lots of headroom.

    The SAT is deeply broken on math with a terribly low ceiling. (Which as noted above, the AMC competitions solve for anyone applying to Caltech / MIT.)

    Their real problem is they removed the single most g loaded verbal item, analogies, and took out the guessing penalty. This makes scores more random, fails to reward a sense of your own confidence in getting down to three answers, etc.

Comments are closed.

Subscribe to All Steve Sailer Comments via RSS
PastClassics
Which superpower is more threatened by its “extractive elites”?
The “war hero” candidate buried information about POWs left behind in Vietnam.
What Was John McCain's True Wartime Record in Vietnam?
Are elite university admissions based on meritocracy and diversity as claimed?