

With a different kind of analysis, it may indeed be appropriate to conclude that many undergraduates are not benefiting as much as they should from their college experience. In short, these considerations suggest that the claim that 45 percent of America’s college undergraduates fail to improve their reasoning and writing skills during their first two years of college cannot be taken seriously. In fact, one of those technical reports flatly states, without explanation, that “student-level reliability coefficients are not computed for this study.” I even looked up several technical articles referred to in the authors’ Methodological Appendix, but was still unable to find any information on the reliability of individual students’ scores. the authors provide no information on the CLA’s reliability at the individual student level. Here’s the dilemma: The more error inherent in the test, the less likely you are to conclude that any given student’s improvement is “significant.” But how much error is present in the CLA?. they concluded that the student “improved.” By that standard, 55 percent of the students showed “significant” improvement-which led, erroneously, to the assertion that 45 percent of the students showed no improvement.

If the improvement was at least roughly twice as large as the standard error. The method used to determine whether a student’s sophomore score was “significantly” better than his or her freshman score is ill suited to the researchers’ conclusion. It is not my custom to offer technical critiques of other researchers’ work in a public forum, but the fact that this 45-percent conclusion is well on its way to becoming part of the folklore about American higher education prompts me to write. The principal finding giving rise to such opinions is the claim that 45 percent of the more than 2,000 students tested in the study failed to show significant gains in reasoning and writing skills during their freshman and sophomore years. in apocalyptic terms: “extremely devastating in what it says about American higher education today”. The implications of the study recently released with the book Academically Adrift: Limited Learning on College Campuses, by Richard Arum and Josipa Roksa, have been portrayed. Jacob Felson points me to this discussion by Alexander Astin of a recent book on college education:
