National Collegiate Athletic Association
NCAA News Archive - 2007
« back to 2007 | Back to NCAA News Archive Index
Assumptions put to test by emphasis on research
|
By Jack Copeland
The NCAA News
The NCAA News
It’s one thing to “assume” that a problem exists in Division III — whether it’s improperly awarding financial aid, using performance-enhancing drugs without fear of discovery, or assuming that wide differences of philosophy or mission exist among division members.
It’s another thing to objectively test those assumptions — something Division III increasingly is attempting to do by allocating funds for research.
Observing how research has helped Division I better understand academic issues and shape policy to place more emphasis on student-athlete success — Division III leaders are tapping the same resources for measuring such distinctively Division III concerns as whether student-athletes are treated like other students.
Research also is helping the division better understand its diversity of institutional types and missions, which Division III Vice President Dan Dutcher says underscores the importance of using research to explore the division’s issues.
“Because of that diversity, it’s become even more important to base decisions on objective information,” he said. “The margin of error has grown related to assumptions. It’s become more foolish, if not dangerous, to make assumptions.”
After relying heavily on existing analytical capabilities in the NCAA national office to establish a financial aid reporting process and design a drug education and testing pilot program that will be conducted during the next two years, the Division III Presidents Council recently provided its clearest endorsement yet of research — it joined Division II in authorizing the use of division-specific funds to support a new staff research position.
That post is held by Eric Hartung, who already has done extensive work for Division III in recent years. In addition to helping design membership surveys conducted during the recent Future of Division III initiative, Hartung teamed with other NCAA research staff members and various consultants in determining the best method for measuring institutional variances in the awarding of financial aid and determining ways to assess the effectiveness of drug education and testing at Division III institutions.
Hartung won’t be establishing policy or otherwise making decisions for Division III in his new role. “Data, or research, or empirical evidence only take you so far,” he said. “At some point, people sitting around a table or voting at the annual Convention make the decisions.”
But Hartung agrees with Dutcher that research helps reduce the margin of error. The more systematic the process, the more highly rigorous the investigations. “You lessen the arbitrariness — and that’s what we’re trying to do,” he said.
Best example
Probably the financial aid reporting requirement adopted by Division III in 2004 — and the subsequent use of what Hartung called “hard science” to create a fair but practical means of measuring institutions’ compliance with the division’s financial aid legislation — is the most noteworthy product thus far of the increased reliance on research.
“The program began with an assumption — there must be a reason why certain schools or teams are always on top,” Hartung said. “There were two ways to attack or confront the issue. One, you can release the enforcement staff based on suspicion, and you begin looking for evidence of wrongdoing; or you build a systematic approach to assess financial aid packaging at Division III institutions in a fair, justifiable, and scientifically and legally defensible manner.
“We developed the systematic approach and have built a foundation of information that allows us to understand financial aid packaging.”
Rather than attempt to compare institutions’ financial aid awarding practices with each other, Division III followed Hartung’s recommendation to establish criteria for comparing aid awarded by each institution to its student-athletes against aid awarded to that school’s general student body.
That recommendation, as well as analysis of data collected from a 2004 pilot study in which Division III institutions provided financial aid data to the national office, resulted in establishment of a 4 percent variance boundary that now is being used to determine whether an institution’s practices justify a review by the Division III Financial Aid Committee.
“What we were able to do was use a body of evidence to determine the criteria or parameters of an entire compliance-based system,” Hartung said. “We were able to measure scientifically the differences in financial aid packaging.
“We were able to model various criteria, and use the information to make an informed decision about what are the proper criteria. The criteria are based on that balance of being scientifically and legally defensible against the practical issues of implementing the program.”
Dutcher said the financial aid reporting program is the best example yet of how Division III can use research to help clarify difficult concepts.
“What emerged was a defensible and understandable approach that helped us get our arms around a fundamental issue,” he said. “That might be the best example of how, once we’re able to approach a problem from the research perspective, we are able to find a solution that works.”
Informed leadership
The capacity for addressing difficult problems through research gains importance in an environment in which people make policy or legislative decisions.
People ultimately make all of the decisions in Division III — whether it’s a policy decision endorsed by the Presidents Council or proposed legislation the membership votes on at the Convention.
“It’s important because you want to make decisions based on the best data you can,” Dutcher said. “You want to assume as little as possible and know as much as possible. Research is a valuable tool because it helps make discussions more objective.”
Division III has used research to reduce assumptions and build an information base for several years — and has come quite a ways from using Convention straw votes during the mid-1990s to take the pulse of the membership on issues such as whether to subdivide the membership.
A decade later, as continuing growth of Division III and the Association as a whole continues to pressure the membership to decide whether it can maintain the current divisional structure, research is helping leaders navigate more objectively through discussions that easily might be driven by emotion.
The Division III Working Group on Membership Issues is making extensive use of research data to guide its discussions about ways to approach possible restructuring of the Association (see story on page 16). Meanwhile, Hartung is beginning to consider questions that will be asked in a planned membership survey early next year to solicit opinions about specific models for addressing Division III’s growth and diversity.
In the past, Hartung said, the membership was asked whether it agreed or disagreed with subdivision, and the discussion stopped right there, because no one knew what a subdivision would look like or who would land where.
“We didn’t know what the principles were,” he said. “But this time, we’re building those principles first — modeling what those principles may be, and modeling what the criteria may be. We can begin with institutional characteristics.
“Then, when we survey, we can say here’s Model 1, here’s Model 2, 3, 4, 5, 7, A, B — whatever you want to call them. And there’s a much better understanding of what you’re actually trying to find out.”
Ultimately, though, people will decide how or even whether to use results from research to address issues.
“Research is vital to the decision-making process, but it’s just one piece in the process,” Hartung said. “If anyone’s looking to research findings or data to answer all of their questions, they’re mistaken. At some point, a group of people still needs to make a decision.”
Those decisions may be made by various groups — by a committee, or the Division III Management and Presidents Councils, or ultimately by the Division III membership — and research is at best a tool to help inform leadership.
“Ultimately, Division III’s policy decisions are still made one-institution, one-vote,” he said. “Research, such as a survey, is a valuable tool in that process, but it’s only that. It’s not the end-all, be-all; it doesn’t necessarily resolve an issue.
“When a survey suggests a significant difference of opinion in the membership regarding a particular proposal, all that means is that the leadership needs to take that into consideration in recommending a course of action to the membership.”
Upcoming issues
Still, research informs those decisions, and it will play an important role in dealing with several issues facing Division III in coming years.
“The drug education and testing pilot should be viewed as being all about research,” Dutcher said, “because we are going to use the pilot to determine to what extent substance abuse is a problem — and to what extent education, testing or a combination of the two effectively address the problem.”
Hartung said financial aid reporting remains an important focus of research as the financial aid committee seeks to focus on specific areas of concern, such as sport-specific aid patterns or improper uses of athletics criteria in packaging decisions.
He also sees other developing areas of research, including student-athlete well-being, and health and safety.
Dutcher suggested that research also is valuable not only in making decisions, but testing whether actions already taken are achieving the intended results. He points to results of the recent GOALS study (see article on page 18) as helping describe the impact that recent legislative actions to support integration of athletics into institutions’ broader academic and community missions may be having on student-athletes.
Hartung said research also is giving Division III a better picture of itself.
“Institutions are disclosing information at record levels — from finances to hiring to athletics — which absolutely allows us to build a storehouse of information,” he said. “Building that storehouse is extremely valuable, because it helps us understand the diversity of the division. We know institutions are different, but to understand how different, we can measure it — commonalities, too.”
Dutcher is sure research will continue to be used to examine assumptions and help Division III solve problems.
“Research has played an ever-increasing role,” he said. “That has been accepted and appreciated by our membership and governance structure. It’s also consistent with the Association’s strategic plan. (The creation of Hartung’s position) reflects the recognition of the importance of research as a tool to help us solve problems that wasn’t recognized before.”
It’s another thing to objectively test those assumptions — something Division III increasingly is attempting to do by allocating funds for research.
Observing how research has helped Division I better understand academic issues and shape policy to place more emphasis on student-athlete success — Division III leaders are tapping the same resources for measuring such distinctively Division III concerns as whether student-athletes are treated like other students.
Research also is helping the division better understand its diversity of institutional types and missions, which Division III Vice President Dan Dutcher says underscores the importance of using research to explore the division’s issues.
“Because of that diversity, it’s become even more important to base decisions on objective information,” he said. “The margin of error has grown related to assumptions. It’s become more foolish, if not dangerous, to make assumptions.”
After relying heavily on existing analytical capabilities in the NCAA national office to establish a financial aid reporting process and design a drug education and testing pilot program that will be conducted during the next two years, the Division III Presidents Council recently provided its clearest endorsement yet of research — it joined Division II in authorizing the use of division-specific funds to support a new staff research position.
That post is held by Eric Hartung, who already has done extensive work for Division III in recent years. In addition to helping design membership surveys conducted during the recent Future of Division III initiative, Hartung teamed with other NCAA research staff members and various consultants in determining the best method for measuring institutional variances in the awarding of financial aid and determining ways to assess the effectiveness of drug education and testing at Division III institutions.Hartung won’t be establishing policy or otherwise making decisions for Division III in his new role. “Data, or research, or empirical evidence only take you so far,” he said. “At some point, people sitting around a table or voting at the annual Convention make the decisions.”
But Hartung agrees with Dutcher that research helps reduce the margin of error. The more systematic the process, the more highly rigorous the investigations. “You lessen the arbitrariness — and that’s what we’re trying to do,” he said.
Best example
Probably the financial aid reporting requirement adopted by Division III in 2004 — and the subsequent use of what Hartung called “hard science” to create a fair but practical means of measuring institutions’ compliance with the division’s financial aid legislation — is the most noteworthy product thus far of the increased reliance on research.
“The program began with an assumption — there must be a reason why certain schools or teams are always on top,” Hartung said. “There were two ways to attack or confront the issue. One, you can release the enforcement staff based on suspicion, and you begin looking for evidence of wrongdoing; or you build a systematic approach to assess financial aid packaging at Division III institutions in a fair, justifiable, and scientifically and legally defensible manner.
“We developed the systematic approach and have built a foundation of information that allows us to understand financial aid packaging.”
Rather than attempt to compare institutions’ financial aid awarding practices with each other, Division III followed Hartung’s recommendation to establish criteria for comparing aid awarded by each institution to its student-athletes against aid awarded to that school’s general student body.
That recommendation, as well as analysis of data collected from a 2004 pilot study in which Division III institutions provided financial aid data to the national office, resulted in establishment of a 4 percent variance boundary that now is being used to determine whether an institution’s practices justify a review by the Division III Financial Aid Committee.
“What we were able to do was use a body of evidence to determine the criteria or parameters of an entire compliance-based system,” Hartung said. “We were able to measure scientifically the differences in financial aid packaging.
“We were able to model various criteria, and use the information to make an informed decision about what are the proper criteria. The criteria are based on that balance of being scientifically and legally defensible against the practical issues of implementing the program.”
Dutcher said the financial aid reporting program is the best example yet of how Division III can use research to help clarify difficult concepts.
“What emerged was a defensible and understandable approach that helped us get our arms around a fundamental issue,” he said. “That might be the best example of how, once we’re able to approach a problem from the research perspective, we are able to find a solution that works.”
Informed leadership
The capacity for addressing difficult problems through research gains importance in an environment in which people make policy or legislative decisions.
People ultimately make all of the decisions in Division III — whether it’s a policy decision endorsed by the Presidents Council or proposed legislation the membership votes on at the Convention.
“It’s important because you want to make decisions based on the best data you can,” Dutcher said. “You want to assume as little as possible and know as much as possible. Research is a valuable tool because it helps make discussions more objective.”
Division III has used research to reduce assumptions and build an information base for several years — and has come quite a ways from using Convention straw votes during the mid-1990s to take the pulse of the membership on issues such as whether to subdivide the membership.
A decade later, as continuing growth of Division III and the Association as a whole continues to pressure the membership to decide whether it can maintain the current divisional structure, research is helping leaders navigate more objectively through discussions that easily might be driven by emotion.
The Division III Working Group on Membership Issues is making extensive use of research data to guide its discussions about ways to approach possible restructuring of the Association (see story on page 16). Meanwhile, Hartung is beginning to consider questions that will be asked in a planned membership survey early next year to solicit opinions about specific models for addressing Division III’s growth and diversity.
In the past, Hartung said, the membership was asked whether it agreed or disagreed with subdivision, and the discussion stopped right there, because no one knew what a subdivision would look like or who would land where.
“We didn’t know what the principles were,” he said. “But this time, we’re building those principles first — modeling what those principles may be, and modeling what the criteria may be. We can begin with institutional characteristics.
“Then, when we survey, we can say here’s Model 1, here’s Model 2, 3, 4, 5, 7, A, B — whatever you want to call them. And there’s a much better understanding of what you’re actually trying to find out.”
Ultimately, though, people will decide how or even whether to use results from research to address issues.
“Research is vital to the decision-making process, but it’s just one piece in the process,” Hartung said. “If anyone’s looking to research findings or data to answer all of their questions, they’re mistaken. At some point, a group of people still needs to make a decision.”
Those decisions may be made by various groups — by a committee, or the Division III Management and Presidents Councils, or ultimately by the Division III membership — and research is at best a tool to help inform leadership.
“Ultimately, Division III’s policy decisions are still made one-institution, one-vote,” he said. “Research, such as a survey, is a valuable tool in that process, but it’s only that. It’s not the end-all, be-all; it doesn’t necessarily resolve an issue.
“When a survey suggests a significant difference of opinion in the membership regarding a particular proposal, all that means is that the leadership needs to take that into consideration in recommending a course of action to the membership.”
Upcoming issues
Still, research informs those decisions, and it will play an important role in dealing with several issues facing Division III in coming years.
“The drug education and testing pilot should be viewed as being all about research,” Dutcher said, “because we are going to use the pilot to determine to what extent substance abuse is a problem — and to what extent education, testing or a combination of the two effectively address the problem.”
Hartung said financial aid reporting remains an important focus of research as the financial aid committee seeks to focus on specific areas of concern, such as sport-specific aid patterns or improper uses of athletics criteria in packaging decisions.
He also sees other developing areas of research, including student-athlete well-being, and health and safety.
Dutcher suggested that research also is valuable not only in making decisions, but testing whether actions already taken are achieving the intended results. He points to results of the recent GOALS study (see article on page 18) as helping describe the impact that recent legislative actions to support integration of athletics into institutions’ broader academic and community missions may be having on student-athletes.
Hartung said research also is giving Division III a better picture of itself.
“Institutions are disclosing information at record levels — from finances to hiring to athletics — which absolutely allows us to build a storehouse of information,” he said. “Building that storehouse is extremely valuable, because it helps us understand the diversity of the division. We know institutions are different, but to understand how different, we can measure it — commonalities, too.”
Dutcher is sure research will continue to be used to examine assumptions and help Division III solve problems.
“Research has played an ever-increasing role,” he said. “That has been accepted and appreciated by our membership and governance structure. It’s also consistent with the Association’s strategic plan. (The creation of Hartung’s position) reflects the recognition of the importance of research as a tool to help us solve problems that wasn’t recognized before.”
© 2010 The National Collegiate Athletic Association
Terms and Conditions | Privacy Policy