Diversity and the SAT

by guest author Jennifer Karan, Executive Director of the SAT Program at the College Board

In a move that has been much discussed among its current student body, Ithaca College recently announced that students applying for 2013 admission will have the option of withholding their SAT scores from consideration during the admissions process.  The statement by Ithaca College maintains that, as a result of dropping the SAT as a requirement, it is “anticipated that the applicant pool will become more diverse, as underrepresented students tend to take greater advantage of test optional policies.”

More than ever the population of students taking the SAT reflects the diverse makeup of America’s classrooms.  In fact, SAT takers from the class of 2012 were the most diverse SAT class ever: 45% self-identified as being minority students; 28% reported that English was not exclusively their first language; and, most tellingly, 36% reported that they would be the first generation of college goers in their family.

The SAT was created to democratize access to college for all students.  SAT scores provide a national, standardized and fair benchmark that neutralizes the risk of grade inflation – a particularly important point when more than 40 percent of SAT takers report an “A” GPA.

Furthermore, the SAT is the most rigorously researched college entrance exam and is consistently shown to be a fair and valid predictor of college success for all students, regardless of gender, race or socioeconomic status.  Each potential SAT question is reviewed by external subject matter experts, subjected to an independent and external sensitivity review process, and pretested on a diverse sample of students from around the world.  Any question that performs substantially different for any gender or ethnic group is eliminated.

Ithaca College and all schools should be recognized for constantly examining their admissions processes, making adjustments to expand opportunities to new applicants and diversifying their student bodies.  As a true believer in the mission of the College Board – helping to connect all students with college opportunity and success – I hope that colleges and universities choosing a test optional admissions policy continue to take the same thoughtful approach as they review the results.

As a former high school teacher and dean of students, I believe in giving students every opportunity to showcase their strengths.  In this case, Ithaca may well be short-changing both the university and potential applicants by eliminating a valid and reliable measure such as the SAT from the admission process.


Will New Measures Halt Cheating on the SAT and ACT?

In the wake of a recent cheating scandal on Long Island, NY, new security measures were announced for the SAT and ACT exams. The principal change is that students will now be required to upload a photo of themselves when registering for these exams. The photos will appear on the students admission tickets, and also on test site rosters that will be available to proctors. Also, the photos will be attached to any score reports that are sent to high schools and colleges.

It seems to me that these measures, while not foolproof, do add an additional layer of anti-cheating protection. Consider the recent cheating scandal on Long Island. Students paid a college student to take the exams for them. The college student, Sam Eshagoff, had only to furnish cheap false IDs, and was granted entry into the testing center. Since the name on each ID matched that of the student who would receive the scores, Eshagoff was able to get away with his scam for quite some time.

If another student were to attempt a similar scam, he would now have to upload a photo of himself, or at least one that looked similar enough to fool a proctor. Of course, that wouldn’t mean he’d be any easier to detect than Eshagoff was. However, there would now be a long-lasting record of his chicanery. If at any time, a high school guidance counselor, or a college admissions officer were to notice that the photo was not that of the student, the deception would be exposed.

I think that this measure will act as a deterrent to most people who might consider cheating in this fashion. However, some students may take the risk, hoping that no one pays close attention to the photos. I suppose it’s also remotely possible that a look-alike could stand in for a student, or perhaps even an expert in disguise might give it a try.

Masked man

My major criticism of the new measures is that they only deter one method of cheating. Students can still share information in various ways. They could easily develop methods of signaling to each other. In this technological age, it’s also hard to imagine that some students haven’t tried using hand-held devices to communicate with sources outside the testing center.

I have heard other objections to the new measures. There will be no more standby registrations. As Akil Bello of Bell Curves pointed out, poorer students who lack easy access to connected computers will be most affected.

This article points out some other possible negative consequences.

Edit: Also read this article on the Bell Curves Blog.

Finally, the question remains: what should be done to the students who are caught cheating? Currently, the only penalty is having their scores cancelled.

What do you think?

Obama’s Proposal Tying Financial Aid to Tuition Rates

In last week’s State of the Union Address, President Obama proposed withholding federal aid to colleges that continue to raise tuition.

English: President Barack Obama gives his Stat...

Some facts: 1) Tuition (and other college costs) have been rising, making college less affordable and less accessible to poor families. 2) In recent years, states have been pulling aid from colleges. 3) To some extent, the loss of state aid has been compensated by increased federal aid, although much of that has been in the form of loans.

What that means is that poorer students are seeking alternatives to college, and those who do attend will find themselves saddled with a pile of debt.

Education is our future. Obama is certainly right to try to make college more affordable. And his proposal has “teeth” – that is, it would put tremendous pressure on colleges to rein in costs.

But is it misguided and simplistic?

If colleges have been hiking tuition to compensate for losing state aid, it isn’t right to point the accusatory finger at them. I fear that withholding federal aid will only exacerbate the problem. Some colleges will not be able to afford to teach need-worthy students. Their student bodies will shrink, and that could lead to bankruptcy.

It’s also hard to blame the states; in this economy, their income has shrunk, and they have cut their budgets out of necessity. One could say that decreasing funding for higher education is shortsighted; they would reply that it’s a matter of survival.

The simple fact is that, during an economic downturn, people and institutions suffer. The best way to get education back on the right track is to right the economy. Fortunately, that seems to happening, if shakily and slowly.

It seems to me that tying federal aid directly to tuition rates is indeed simplistic. State aid needs to be factored into the formula. Perhaps private endowments could be included as well; they also decrease during a recession. Maybe the federal government could offer incentives to states for maintaining college aid.

What do you think?

Should We Succumb to Students’ Aversion to the Printed Word?

Last week, Apple launched its iBooks 2 software. Apple was not the first company to create software for the development of multimedia etextbooks, but as usual, they created the biggest buzz. Naturally, their motives aren’t entirely altruistic; Apple wants a big piece of the etextbooks pie (now, there’s a metaphor that works overtime).

English: The logo for Apple Computer, now Appl...

I have seen demonstrations of multimedia etexts, and they are very impressive. Embedded photos can be quickly expanded to fill the page of your monitor or iPad; a flick of your fingers returns them to normal size. Three-dimensional images can be rotated in a similar fashion. You can watch videos also, and of course, audio is included. Note taking and highlighting are also easy to do.

There is no doubt in my mind that etextbooks are the way of the future. They simply amount to a better way of transmitting information than the old, stodgy paper books. And who wouldn’t rather carry a single iPad than a heavy load of textbooks?

Still, something gnaws at me. To hear the majority of my students tell it, the disappearance of printed text as we know it wouldn’t be a bad thing. Who needs novels? – we have movies. Biographies? – watch a documentary. Who wants to read when you can watch a screen?

Educators have been lamenting the decrease in reading by our youngsters since long before Apple was formed. I tell my students that I understand the phenomenon. It used to be that kids didn’t have much to do after the sun went down. Once they tired of drawing or playing checkers, reading a good book was their most entertaining option. But then radio was invented, and that took up a share of their time. Television was a big one. Now there are computers, video games, smart phones, etc., and they are all easy and fun. Who has the time to read a book? Only geeky bookworms seem to do very much of it.

However, practice does makes perfect, and only students who read regularly tend to make good readers. Many of my students have told me that they don’t want to read my blog because it’s all text (hey – what about those SpongeBob pictures?). Even my own son said he’d prefer video lessons, or at least animations.

I don’t want to sound old fashioned, and I’m certainly not anti-technology. But I believe that many animated or video lessons are inefficient. They have their place, when visual imagery is paramount. But students are forced to go at the pace of the video, and few of them will bother to rewind even if they should. Learning is very passive when one watches an animation; text is sometimes more effective, and writing material down for one’s self can be best of all. Yes, students like on-screen learning because it’s interesting, but they also like it because it lets them be lazy.

Some educators say “if technology makes learning fun, go with it, because students will use it more.” That’s certainly true, but I think a compromise is called for. Printed text is not going to disappear anytime soon, and if our students don’t have a facility with it, we’re going to be in big trouble. Electronic devices are fine, but educators should make sure that they encourage students to read plenty of printed text, whether on paper or screens.

That’s not an Earth-shattering proposal, but students’ reluctance to read (and write) is disturbing. We educators need to find ways to make reading enjoyable for our students.

English: A 1st generation Apple iPad showing i...

Fancy electronics alone won’t do that. We must find ways to make content compelling to today’s students. We need to find stories that reach today’s youth personally. Also, students might prefer short stories or novellas to full-length novels – that’s fine. As for textbooks, perhaps their writing style needs some spicing up. Old-school teachers might be horrified at the thought, but again, it’s worth it if it gets the kids to read more.

Now, I’m not saying that we should “de-professionalize” every textbook. What I am saying is that we need to find alternative strategies to reach students who are reluctant to read much. To be sure, we won’t succeed with every student. But such efforts could be implemented at all grade levels from elementary to high school. If we can succeed at rescuing a few percent of the non-readers in each grade, we’ll have many more competent readers and writers and more of them will go on to college.

We cannot let our children abandon the printed word. If we do, we abandon them.

SAT Essay: In Search of a Perfect Score

I have had several students who were excellent writers, yet were disappointed at not being able to score above a 10 on their essays after taking the SAT multiple times. I have also read numerous reports on the internet about students who had similar experiences.

My advice to all of you – don’t lose sleep over it. Let’s see why:

Two graders read your essay. Each assigns a grade of 1 – 6, so you get a total of 2 – 12 (note that a third reader will be asked to grade your essay if the first two scores are more than 1 point apart).

The graders are presented with a scoring rubric. Let’s have a look at The College Board’s description of what a “5 essay” and a “6 essay” should be:

Score of 6

An essay in this category demonstrates clear and consistent mastery, although it may have a few minor errors. A typical essay:

  • Effectively and insightfully develops a point of view on the issue and demonstrates outstanding critical thinking, using clearly appropriate examples, reasons, and other evidence to support its position
  • Is well organized and clearly focused, demonstrating clear coherence and smooth progression of ideas
  • Exhibits skillful use of language, using a varied, accurate, and apt vocabulary
  • Demonstrates meaningful variety in sentence structure
  • Is free of most errors in grammar, usage, and mechanics

Score of 5

An essay in this category demonstrates reasonably consistent mastery, although it will have occasional errors or lapses in quality. A typical essay:

  • Effectively develops a point of view on the issue and demonstrates strong critical thinking, generally using appropriate examples, reasons, and other evidence to support its position
  • Is well organized and focused, demonstrating coherence and progression of ideas
  • Exhibits facility in the use of language, using appropriate vocabulary
  • Demonstrates variety in sentence structure
  • Is generally free of most errors in grammar, usage, and mechanics

You can see that all of the differences are subjective. Arguably, any well-written essay could be described as belonging to either of the categories.

Many of my students order Question-and-Answer service, and I’ve looked at their essays as they were written. And in my mind, many of the essays that received a score of 10 or 11 were superior to others that received a 12. Other SAT experts have reported similar findings.

I have read about professional authors who have received less-than-perfect scores, despite repeated attempts at the SAT. How is that possible? A score of 6 is not supposed to be reserved for a perfect essay (if one even exists). No, a 6 is supposed to be assigned to a decent first draft written at advanced high school level. That’s a far cry from what a professional author will churn out easily.

The problem is that the graders’ ideas of what constitutes a “6 essay” are subjective. I have read reports from graders who say that they only assign a 6 if an essay is “special.” Fine, but what exactly constitutes “special”?

I have read that some graders are impressed by students who cite classic works of literature. This isn’t surprising; a large percentage of the graders are English teachers. But I’ve read that other graders frown upon cited sources, preferring the “creativity” of personal experience. And since graders are looking for their own idea of “special,” there is simply no way to ensure that you’ll get that perfect grade.

The College Board seems to have recognized this fact; the scoring scales for the first 3 tests in the “Blue Book” (The Official SAT Study Guide, 2nd ed.) reveal that you can achieve a perfect 800 score on the writing section even if you score only a 9 on the essay. However, if you don’t do well on the multiple choice, you can lose 40 or 50 points because your grader doesn’t think your essay was “special.” So much for the claims of pinpoint accuracy of SAT scores.

Another issue may be that graders are reluctant to give many essays a score of 6, given that the other grader of any particular essay may assign it a 4. The College Board keeps track of how often a third reader needs to be called; the consequences for exceeding a certain threshold include mandatory retraining, and eventually, the possibility of being fired.

A few years ago, I signed up for The College Board’s online course. I was able to submit essays for scoring. Since the scores were returned immediately, it was obvious that they were using some form of software to do the grading. I cut-and-pasted the example essay from the Scorebusters course manual, and received a score of 5. Then I wrote some low-level nonsense (including a paragraph about an invented book titled Lizard Man and the Green Meanies by Harold Wheresmyshorts; the plot was as inane as the title implies), and sure enough, reached a 6. I can assure you that the first essay was superior in every way mentioned on the rubric.

Lizard Knows Best

I offer a solution to this problem: Let the graders continue to assign grades of 1 – 6, but convert all the 6’s to 5’s before calculating scaled scores (i.e render scores of 5 and 6 identical). Stop penalizing students because of graders’ subjective opinions of what is “special.”

Is the SAT a Good Test?

Polarize vb. to cause people to adopt extreme opposing positions (from thefreedictionary.com)

A web search reveals many different opinions of the SAT, most either very negative or stoutheartedly supportive.

I’ve been teaching and analyzing the SAT for 28 years, and I strive to form opinions that are as free from personal bias as I can make them. And my opinion is the boring, uncontroversial, middle-of-the-road  one – the SAT is neither an awful test nor a very well-developed one.

Here I will give my best answers to several questions about the quality of the SAT. Given the format of this blog, I will attempt to be informative, but by no means comprehensive.

SAT Subject Tests

Does the SAT do what it is designed to do?

The primary stated goal of the SAT is to predict grades in the freshman year. The second goal is to predict grades throughout undergraduate college. Various studies have been published, most of which indicate that there is a significant correlation between SAT scores and college grades, but that the SAT doesn’t provide much information beyond what high school grades do.

One of the first studies was sponsored by Educational Testing Service, who wrote the SATs. They were unhappy with the results, but the authors released their findings, which were published in several articles and at least one book. One of the researchers reported that he had expected the results to support the usefulness of the SAT, and was surprised when they did not.

More recent studies have yielded similar conclusions, although some scholars have questioned their methodology. For example, check out this article in Pencil Nerd’s blog.

It stands to reason that a grade average of 90 at one high school may not equate to a similar one at another school. On the other hand, one would hope that most admissions officers have detailed data on just what different high schools’ grades mean. Nevertheless, the SAT should serve to “level the playing field” by providing colleges with uniform scores.

In conclusion, the SAT probably does some of what it’s supposed to do, but not necessarily as well as it might.

Is the SAT a fair test?

Numerous studies have demonstrated that women, minorities, and residents of some states perform below average on some or all of the sections of the SAT. Groups such as the National Organization for Women and FairTest have claimed that the SAT is biased against these groups.

The obvious question is: are they blaming the messenger? In today’s hyper-politically-correct climate, many people are eager to point the finger at anyone nearby whenever any minority is at a disadvantage. But if men outscore women on the math SAT, does that mean the test is bad, or that men tend to outperform women on math, period? Perhaps women shy away from math because of gender biases. Perhaps something in our genetics predisposes men to like/do better at math more (shudder – did I just suggest that people might be different?). What about minorities, such as blacks or Hispanics? Don’t they tend to have lower incomes than whites, and thus live in poorer districts with poorer schools. Should we blame a test for that?

Some years ago, when the SAT had analogy questions, FairTest pointed out an analogy for which the correct answer was oarsman:regatta. If you don’t know, a regatta is a series of boat races, and wealthy people who live in coastal areas tend to take part in them. Certainly, such people (or children in their families) would have an advantage on this question.

FairTest cited the regatta question as evidence of SAT bias. But I’m very familiar with the test, and I have seen very few questions that are biased in this fashion. How many SATs, with well over 100 questions each, did FairTest have to pore through before they gleefully discovered this question (and a couple of  others of its ilk). ETS and The College Board have been dealing with accusations of bias for years; you can bet that they now take painful steps to avoid “rich kid” questions.

That last sentence is my conclusion. The SAT writers bend over backwards to make the SAT politically correct, so that, if anything, it would tend to benefit minorities (the Reading section abounds with passages about feminists and abolitionists). It’s time to point the finger somewhere else.

Are the questions well written?

Yes! An impressive amount of time is devoted to developing, reviewing, and revising the questions. Then new questions are seeded into experimental sections on the SAT, so that thousands of students “review” them. Very few flawed questions make it to the actual SAT.

By contrast, I have seen a lot of practice tests in commercial prep books, and I can instantly recognize many flaws in all of them.

English: Welcome sign at the entrance to the h...

Are the question types good ones?

Again, my review is mixed here. The general types are okay, but the test developers deliberately make the questions tricky in various ways (e.g. trap answers, tricky wording, etc.).

The fact is that a single exam of only a few hours length is not the optimal tool for measuring the college-readiness of all students. There is a huge range of abilities in the population of students who take the SAT, from those who hope to achieve a minimum score in order to be eligible for sports scholarships to those who need top scores for top schools. That means the test writers need to include questions that are aimed at all of these students – i.e. very easy to very difficult questions. The result is that there are very few questions directly aimed at any particular student.

Useful hard questions are particularly difficult to create. It is not enough that few students answer them correctly; to be meaningful it is vital that only the best students nail them. The test writers employ several strategies to help achieve that result, but they are flawed, since it is possible to analyze them and coach students accordingly.

Has the SAT improved, and what can be done to improve it?

The SAT has improved. Some flawed question types have been dumped. There are also fewer tricky questions and answers than there used to be. The addition of the Writing section was also an improvement.

The SAT could be improved by adding more questions such as those found on the ACT. The ACT is more of a test of learned knowledge; the SAT tests reasoning more. Why not test more of both?

Clearly, eliminating tricks and traps would be a positive step. Hard questions should require higher level thinking, rather than the ability to navigate a sea of tricks and traps.

The SAT tests the good ole’ “3 Rs” (reading, ‘riting, and ‘rithmetic). It couldn’t hurt to add some science and social studies to the mix.

I feel that the best way to improve the SAT would be to administer two separate tests on different days. The first test would assess the student’s general level of ability in each section. The second test would be tailored toward that ability. For example, if a student scored 4 out of 5 on the Math section of the preliminary test, she would then take a “Level 4” Math test afterwards, with only questions of difficulty 3-5.

Naturally, there would be some resistance to this proposal, since it would take up more time overall, and the first test would need to be graded before the second was administered. However, I think it makes good sense, and would even help reduce pressure on many students.

Finally, The College Board has been working hard to prevent cheating on the SAT.

A Little Logic, Please!

Time for a quick rant. Yesterday, several good folks on Twitter pointed me to an article entitled SAT scandal shows tyranny of standardized testing. For awhile, it was ranked at the top of the Education page on Google News. The author blames high schools and colleges for requiring standardized tests such as the SAT, and putting undue strain on students.

Today, I’m not even going to bother debating whether the pressure of taking the SAT is unfair, or whether our nation’s permissive ethos is making us soft. My point of this rant is that, if you want people to take you seriously, don’t begin your article with patently absurd logic.

A handful of students cheated on a test which is administered to over 2 million students each year, and that’s supposed to prove that it’s tyrannical? Sometimes kids steal their classmates’ pocket change – does that mean that lunch is tyrannical? A student recently cheated on his girlfriend – love must be monstrous.

And now back to my Gummi Worms.

Gummi Worms