Education should drop the guise of being a science. New book attacking SATs has right message, wrong tone

ONTROVERSY over Scholastic Aptitude Test (SAT) examinations and the validity of other standardized tests administered by the Educational Testing Service (ETS) has heated up again as a result of David Owen's new book, None of the Above: Behind the Myth of Scholastic Aptitude (Houghton Mifflin Company, Boston). The book is a sustained polemic against the SAT and ETS, which operates under the aegis of the venerable College Board. Owen's book could not have been better timed. In the United States, two years' worth of extensive investigation and debate about the quality of schools and colleges has just been completed. In today's frantic search for improvements in our schools and for reliable measures of quality, we are likely to place even greater reliance on the results of SAT-like, uniform multiple-choice examinations.

The stakes are high. Students with high scores on SAT exams (and their law school and medical school equivalents) are assumed to be America's most talented. High schools where a high proportion of graduates score well are thought to be the best. Colleges that boast a high level of SAT scores among entering freshmen are considered selective and therefore superior. There is no guidebook to American colleges that does not cite SAT averages as a key measure of educational quality. Now, SAT-like tests may be used to evaluate the work of teachers and determine who gets the much-talked-about ``merit pay'' increases.

The College Board and ETS have taken Owen's excessively long, often petulant new book seriously. They issued a defensive rebuttal to their member institutions which, sadly, fails to redeem ETS or the SAT. In fact, ETS president Gregory Anrig, responding to Owen, has claimed that testing of the SAT variety ``is an integral part of current school reform efforts to strengthen educational standards.'' If Anrig should be proved right, today's momentum for long-overdue school reform in our decade will not have fulfilled its early promise.

The time has come to construct a simpler, more rigorous, straightforward approach to testing, one that raises standards and jettisons the superficiality, the pseudo-scientific pretensions, and the self-serving character of what ETS now gives us. It is time to turn our eyes away from testing and back to the quality and content of the curriculum, the training of teachers in their fields, and to the capacity of students to write extensive, well-formulated answers to straightforward questions in those old-fashioned instruments, the essay test, and the mathematical problem set. In the popularity of the phrase ``back to basics'' lies a hidden truth: a call to a return to the era before multiple-choice tests and the growth of ETS. Not only are they not agents of reform, they are part of the problem with American schools. Testing and the burden of democracy

The sustained popularity of the SATs stems from a national desire to create a system that would permit merit to triumph over old school ties, wealth, who one's parents were, the advantages of private schooling, and the like. The ETS and the SAT flourished in a postwar era that sought to open the doors of our most prestigious colleges and professional schools to new populations. It is ironic that one of the most persistent correlations is the link between family income and SAT scores. The rich do better. They have better schools and live in environments more conducive to achievement in school. In fact, the system of measuring so-called aptitude on a standardized comparative basis, although consistently defended in terms of democratic ideals, was developed in the 1920s and '30s (as Owen ably describes) by men who believed in racial differences and social stereotypes, even in eugenics.

The use and popularity of SAT tests has done nothing to improve schooling for those populations that need it most. It has merely reinforced the educational advantages possessed by high-income white Americans by placing a respected ``scientific'' measure on those advantages, and redefining them as signs of ``aptitude.'' The performance of students over a 15-year period in New York State's Equal Opportunity Program and Higher Education Opportunity Program (college and university programs designed for low-income students who score poorly on such tests) indicates that the poor and minorities with low SATs can do perfectly well, tests or no tests. As Bowdoin College, which pioneered dropping SAT scores as part of the admissions process, has demonstrated, educational excellence in the classroom and high standards in admissions do not need validation or help from ETS.

Education is no science. Few fields of endeavor possess such a wide range of well-reasoned but conflicting opinions about materials, methods, and measures of quality. Few fields have been subject to as much thoughtless preaching, politically inspired ideologies, half-baked but scientific-looking theories, public criticism, and vitriol. Few fields seem to hold so much potential and yet deliver so little. Owen's book attempts to show that the SATs and the cult of authority around them are matters of subjective belief cloaked in the objective-sounding language of science.

Education had always been considered a key to the quality of our democracy. It is a route to greater social equality, a road out of poverty, an exit from prejudice and personal stagnation. We still believe that if we could only raise the level of educational achievement among our fellow citizens, there might be less crime and a healthier economy. Furthermore, if we could only find an objective way of measuring excellence, of identifying merit fairly and objectively, we might lessen the lingering sense that privilege and mediocrity persist where they ought not to. Our attachment to the SATs and similar tests, however misplaced, reflects these aspirations. In clinging to the SATs, we express the hope that the seemingly endless debate of what does and does not work in education can be ended.

So strong is our desire for some common measuring stick for merit and educational excellence that we have accepted appearance for reality. One of the poignant ironies of the American spirit is that for nearly half a century the nation that most values individuality, self-reliance, and the spirit of innovation -- those residues of America's founding, its historic tolerance for religious iconoclasm, and its 19th-century encounter with the frontier -- has clung to a seemingly unshakable faith in a timed, formulaic, and superficial uniform test of intellectual talent and promise. It seems never to have occurred to commentators to note that there is no defensible correlation between an individual's SAT scores and the capacity for usefulness, even greatness, later in life. A cluster of contradictions

David Owen's book, therefore, not only attacks the tests and ETS but also the validity of such psychometric testing as such. He describes ETS as a profitable, lush, and overfed corporation hiding behind the veil of its not-for-profit status and its claim to provide service and research in the public interest.

Owen uncovers a host of contradictions in ETS positions, ranging from how significant differences in scores are and whether one should guess on the tests (Owen says yes), to whether coaching helps (which it does). Owen attacks the jargon of the ETS, its statistical premises, and the fairness and quality of its ``holistic grading'' concept. He attempts to destroy the myth of the SAT's accuracy, its quality, and the security at its test sites. Most of all, Owen tries to get at why ETS has flourished, at why so many of us are taken in and even traumatized by SATs. (Anxiety is the greatest cause of lower-than-expected scores.) On practically every page, Owen's outrage at the influence and impact of ETS and the SATs -- on students, employers, government, and the educational establishment -- can be felt.

Unfortunately, Owen gets too caught up in his own personal vendetta. He writes like a man bent on uncovering a conspiracy. Even though he demolishes many sample questions on the exams in order to show how wrong and silly much of what is on these tests is, he too frequently relies on his own clever rebuttals, much like a pupil eager to show up the teacher. He does show that the tests are based on a system that can be cracked by coaching; that getting good scores is indeed a matter of learning to play a self-referential game, one with rules pitifully unrelated to intellectual curiosity, knowledge, and understanding.

Owen's zeal to demolish ETS and the tests does lead him astray. For example, in discussing the ``cult of mental measurement'' behind ETS, Owen cites the 1928 publication of an article by Carl Brigham, ETS's spiritual father, in a eugenics journal that in 1933 printed a Nazi sterilization statute written by Hitler. Owen's attempt to link Brigham with Nazism in this way is sheer calumny, guilt by association of the worst sort. Owen's book is marred by less egregious but similar overreachings, leaps of logic and evidence. He even fails to reveal that he is employed by the particular coaching service to which he devotes so much favorable space.

Despite these flaws, Owen is on the right track, and his essential message is sound. Yes, ETS should be cut down to size and the inadequate methodology behind its tests exposed. If SATs and their ilk cannot be abolished, the tests should at least be severely limited. Owen is, however, no educational reformer, and his final chapter, ``Testing and Society: What Can Be Done,'' is vague and unconvincing.

Prescriptions for the future -- for a system that can demonstrate greater quality, fairness, and a higher standard of excellence -- are frighteningly simple. We need essay and math problem set tests based on rigorous in-school requirements, particularly in English, history, science, and mathematics; reduced class size; teachers well trained in their disciplines; tougher curricula; and better teaching materials (including great traditional texts), both in high school and college.

But even if well-financed reforms toward these goals fail to occur, we must work to shore up the standards behind the only proven measure of a student's potential for the next stage of education: earned grades. We must work to raise the levels of expectation in our classrooms. No elaborate instruments of measurement are required when we test what our children have learned and can do -- just common sense. Our love affair with bad standardized tests has gotten us nowhere. We need better schools, not better tests.

Leon Botstein is president of Bard College.