Latest News

« back to 2010 | Back to NCAA News Archive Index

NCAA grad rate a success by any measure

By Michelle Brutlag Hosick
NCAA.org

The NCAA will release the latest Graduation Success Rate figures on October 19, once again bringing attention to a rate that college and university presidents believe is more useful than the methodology used to produce the federal graduation rate.

The difference in methodology each year produces a GSR that is markedly higher in aggregate than the federal calculation of student-athlete graduation rates. Most recently, the GSR was about 15 percentage points higher than the graduation rate collected using the federal methodology.

Some critics argue that the disparity is nothing more than artificially induced graduation-rate inflation, but those who created the rate – and those who support the metric – say the difference in the two methodologies is the reason why the GSR was created.

“One of the primary things that gave rise to the GSR was concern about the way the federal graduation rate seemed to deal incorrectly with transfer students,” said Jack Evans, faculty athletics representative at North Carolina and longtime member of the NCAA Division I Committee on Academic Performance. “The federal rate treats all transfers out as graduation failures, creating a loss in the federal graduation rate, and people who transfer into an institution do not become the responsibility of that new institution. The federal rate essentially makes transfers invisible, which doesn’t reflect what’s actually going on in our institutions.”

The Division I Board of Directors’ adoption of the GSR methodology reflected a desire from college and university presidents in general who were frustrated with a system that didn’t make sense to them. Even though the federal methodology showed rates that compared favorably with rates for the general student body, presidents wanted a more accurate picture.

University of Michigan President Mary Sue Coleman told The NCAA News when the GSR was adopted in 2003 that presidents had long been disappointed with a federal methodology in which so many student-athletes are simply lost in the calculation.

“We know that not only in athletics programs but in student bodies as well that many students begin college, then they transfer, and they are counted in the federal rate as dropping out and never getting a degree. That’s not the case at all,” she said.

Todd Petr, NCAA managing director of research, said research shows that many students have dynamic collegiate careers – moving from one institution to another. More than a quarter of men’s basketball and baseball student-athletes transfer at least once in their collegiate careers. That movement is not accounted for under the federal methodology.

“The federal rate assumes a very static model of student progression through college where student movement is ignored for incoming transfers but penalized as academic failure for those who transfer out of a school,” Petr said. “Our membership has been clear that these transfer students be included in any measurement of academic performance.”

Differences are clear

The GSR includes student-athletes who transfer into Division I institutions from either two-year or four-year institutions in the new school’s cohort. Those who leave an institution but would have been eligible to compete had they stayed are removed from that school’s cohort (not counted as successful or failures) and potentially picked up by another school’s cohort if the student-athlete remains in Divisions I or II. The GSR also includes nonscholarship athletes participating in programs that do not offer scholarships.

“Generally speaking, for most squads, the GSR turns out to be higher than the federal graduation rate for the same cohorts,” Evans said. “That doesn’t mean that this is an inflated metric. I would argue that the federal graduation metric incorrectly deflates what is actually going on. The GSR in reality is the metric we really would have wanted from the beginning. It’s not athletics-friendly inflation of graduation results.”

In fact, the additional NCAA research shows the GSR may actually underestimate how many student-athletes eventually earn their bachelor’s degrees.

The NCAA’s Study of College Outcomes and Recent Experiences (SCORE) conducted in 2005-06 longitudinally followed more than 4,000 former Division I student-athletes who graduated from high school in 1994 and surveyed them about 10 years after their initial enrollment in college. The sample was weighted to represent all Division I student-athletes, and the data from that sample indicated that 88 percent of the former Division I student-athletes had graduated from a four-year college within 10 years.

“It’s interesting to note that only 62 percent of all student-athletes graduated from their initial institution of enrollment, illustrating the importance of taking transfer behavior into account,” Petr said. “The GSR, even though a better approximation of what really happens to the student-athletes themselves than the federal rate, might actually be underestimating the long-term student-level graduation performance.”

Including – and excluding – these transfer student-athletes leads to a gap between the federal rate and the GSR that, while substantial, is easily explained. The NCAA still sees some value in the federal rate, however, because it is currently the only rate in which student-athletes can be compared to the rest of the student body. Though the Association has approached the U.S. Department of Education about modifying its methodology to acknowledge the significant transfer behavior among college students, the federal government has not been receptive.

The resistance is at least in part due to the difficulty of changing the way the federal government collects and reports graduation rates. Any change would require a change in the law (not just departmental regulations). While many in higher education – including a fair number of university presidents – agree that the rate is flawed, no one with significant political clout has become a champion.

Petr also identified a methodological issue that might make a shift difficult. Because student-athletes are held to specific academic standards that must be met for them to be eligible, it is easy to determine whether a student-athlete who leaves an institution would be eligible to continue. The standards for the student body are much different, and the difference could make writing a new law that allows for direct comparisons between student-athletes and the student body challenging.

Countering misperceptions

For all of its improvements over the federal rate, the GSR is not the answer to every academic problem. Some officials have argued that teams with substandard graduation rates should be excluded from championship participation. Evans calls that a “bad idea” that would “impose a penalty on a group of people that were not responsible for creating the metric on which the penalty was based.”

By definition, graduation rates cannot be compiled until the student-athletes responsible for graduating in a set period have either done so or not. By the time a penalty would be imposed, the student-athletes in the cohort that created the poor graduation rate are no longer part of the team and, in most cases, no longer at the university. 

The NCAA imposes a championships ban for poor academic performance in another area. The loss of the opportunity to participate in championships is a saber in the Committee on Academic Performance scabbard of penalties, assigned to teams that, in real time, fail to achieve academically for a certain period. Later this month, the CAP will discuss refining its Academic Progress Rate with the possible addition of a graduation component.

Some misuse the GSR in other ways. A recent paper by Woodrow Eckard from the University of Colorado-Denver questioned whether the federal rate and the GSR grossly overstate student-athlete academic performance because NCAA regulations require student-athletes to be full-time students. Eckard claims that other students tend not to stay full time because they are subject to no rule requiring it, and he believes the student body graduation rates should be adjusted to eliminate part-time students.

“He gets to this conclusion and a subsequent large adjustment in student body graduation rates by assuming that at schools with high numbers of part-time students, full-time degree-seeking members of the student body tend to fall to part-time status in the same proportion as there are part-time students at that school,” Petr said.

In his paper, Eckard adjusted the student-body graduation figures to account for what he believed to be more part-time students, but he did not adjust the student-athlete numbers, making the student-athlete rates look poor in comparison. Evans at North Carolina said Eckard’s claims of poor student-athlete performance miss the point of the GSR calculation – which is as accurate as possible.

“It doesn’t matter all that much to us how the GSR compares to some estimated GSR for non-athletes because we don’t have a GSR for non-athletes. That’s not the point,” Evans said. “We wanted some internally consistent graduation metric that we could use for athletes that dealt with the simple fact that there are transfers in and out, and to have an accurate measure of graduation activity you need to consider both of those things.”

This is not implying that Eckard is creating an adjusted GSR.

Evans believes the GSR has been successful at doing what was intended – be a more accurate measurement of graduation. In itself, it is not designed to improve academic performance – a number of other reform efforts have that aim (the APR with its penalties and increased eligibility standards). But when working in concert, all of the elements of academic reform are intended to work together to produce higher graduation rates.

“Academic reform is a three-part system that consists of new standards, both initial eligibility and progress toward degree; new metrics, the APR and the GSR; and new consequences through APR penalties. These three taken together are the essential pieces of academic reform,” Evans said. “They are causing coaches to pay more attention to academic performance. We’ve seen the APR improve. What we don’t know yet is whether improved APR will lead to improved graduation.”