As Josipa Roksa greeted the University of Iowa crowd that gathered for her presentation, she acknowledged the gloomy weather outside and warned, “I don’t have a happy topic for you this morning.”

The topic of her presentation was the controversial book Academically Adrift: Limited Learning on Campuses, which she cowrote with Richard Arum, professor of sociology and education at New York University’  Steinhardt School of Culture, Education, and Human Development. The book argues that a substantial number of undergraduate students are not learning as much as they should be at college.  

Roksa spoke about her book and the research it details at the Lindquist Center Thursday morning.

In the book, Roksa and Arnum concluded that there was no significant difference or improvement in certain cognitive skills during a student’s first two years of college.

Roksa and her research team conducted a longitudinal study over a two-year period of 2,300 freshmen from 24 institutions who entered college at four-year institutions to track their academic progress. The researchers collected their data based on students’ scores on the Collegiate Learning Assessment, which measures critical thinking, complex reasoning, and written-communication skills, in order to measure the students’ progress over a two-year span.

While Roksa’s book has been highly praised as a wake-up call to higher education, Alexander Astin of UCLA recommends that her findings be taken with a grain of salt.

Astin, a professor emiritus of higher education, criticized the learning assessment as a limited instrument of measurement that allows for a great deal of error. He said he was also concerned that the study made strong conclusions about higher education based on only three skills that a student is expected to develop over their college careers.

“You need much broader gauges of measures to draw such sweeping conclusions of whether higher education is effective or not,” Astin said. “It’s like having a beauty contest and saying, ‘We are only going to judge the contestants’ beauty based on their left ear and ignore all of their other qualities.’ ”

UI sophomore Will Aden said that although he does not think every class he has taken since entering college has contributed to his critical thinking and writing skills, he does believe that they have improved.

“I don’t think they improved that much during my first year, but in my second year and since changing my major to journalism, my writing skills have especially improved,” he said.

Ernest Pascarella, a UI professor of educational policy and leadership studies, and two UI graduates conducted a follow-up to Arum and Roksa’s study after the legitimacy of their study was questioned.

Rather than using the measure that Roksa and her colleagues used, Pascarella and his team used a multiple-choice measure of critical thinking that was developed by the ACT.  They also used a different sample of institutions and students from those evaluated in Roksa’s research. Pascarella’s study focused on first-year students rather than students in their first two years of college.

“We came up with almost exactly the same results,” Pascarella said. “What that means is their findings are robust irrespective of what kinds of measures they used.”

While Pascarella is confident that his study proves that Roksa’s findings were not just a fluke, he still acknowledges that both studies do not entirely measure the effect of college.

“You probably learn a lot of skills in college that are useful in a job that have nothing to do with how you score on a test,” he said. “There are lots of other impacts of college; [these studies] just show one little piece of it.”

Roksa and her colleagues are working on advancing their research by writing another book that considers what recent graduates are doing two years after college and how their scores on the student assessment correlate to their post-college outcomes.