National Collegiate Athletic Association |
The NCAA News - News and FeaturesThe NCAA News -- February 15, 1999 Dissecting the rating-percentage index
RPIs are excellent data source, but not panacea for championship selection
BY GARY T. BROWN Sports committees have long sought the perfect balance between objective data and subjective selection during those long meetings when they choose NCAA championships fields.
Designed for pinpoint accuracy in rating teams according to various formulas, the RPIs offer sports committees the facts they need to make subjective decisions. In fact, because the RPIs are so accurate, the line between objectivity and subjectivity can sometimes blur. After all, this year's mythical college football championship was determined by a computerized ranking system. But the RPI is no BCS. Though the general public may see RPIs as the great regulators of championship selection, sports committees are adamant that they exist purely as another source of data from which to draw guidance. "It would be just as bad for a computer to spit out a championship field as it would for a committee to make some very subjective decisions without data," said Jonathan B. LeCrone, chair of the Division I Men's Soccer Committee. LeCrone's committee began using an RPI as a selection tool for the first time last year. Before that, the committee was responsible for culling its own data and spending hours during deliberations reaching decisions that the RPI has helped to make more efficient. But LeCrone said the committee wants no part of having the RPI become the sole button to push during selection weekend. "The point is that none of this is a perfect science -- the RPI or any other data source," he said. "You're never going to be able to eliminate or make championship selection a fully objective process -- some subjectivity still is important. But that doesn't mean the RPI isn't a usable tool. It's not perfect, however, and it's not a panacea." Even the BCS found that out. But as the public is bombarded with rankings, polls and lists, the RPI sometimes is misconceived as the bottom line. "We get calls from the general public all the time about the RPI," said William R. Hancock, director of the Division I Men's Basketball Championship, administration. "But in fact the committee downplays the importance of the RPI. It's used to provide information, but not as any sort of directive." Basketball was first Jim Wright, NCAA director of statistics, said talk of a rating-percentage index first emanated from the Men's Basketball Committee in the late 1970s and was kicked around by some mathematicians at Stanford University. NCAA staff, particularly the late Jim Van Valkenberg, then-director of statistics, also had a hand in various early permutations of the RPI. Those combined efforts led to the first RPI being used to help the Men's Basketball Committee select the Division I Men's Basketball Championship field in 1981. An RPI for Division I women's basketball was quick to follow, then baseball jumped on the RPI bandwagon in 1988. Today, there are RPIs for eight NCAA Division I team sports, the latest being field hockey, which was added this fall. All committees with RPIs at their disposals use them in much the same way -- as another tool in a bevy of selection criteria and data resources. "It's one of many criteria we use," said Cindy Masner, chair of the Division I Women's Softball Committee and sen- ior woman administrator at California State University, Long Beach. "We don't weight our cri- teria. So it's not a matter of saying, 'Well, they're this in the RPI so they have to be selected.' It's been a useful tool, but it's not something where if you're one-through-64 you're going to get in." Wright, who as an NCAA staff member since 1975 has been involved with each of the indexes, said he sees the same caveats in committees relying too heavily on the RPI despite its value. "Even as a statistician who spends a lot of time with RPIs, I'd never want just a computer program to pick teams because the computer can't evaluate the rivalries, or injuries, or a transfer suddenly becoming active, or a chemistry problem that develops within a particular team," he said. "That's why you have sports committees and regional advisory committees out there. "Yes, the RPI is an important tool, and yes, it gives a sports committee a lot of information, but none of us who edit RPIs here at the national office would ever expect or want a situation where a computer is picking teams for a tournament." Anatomy of an RPI The RPI skeleton is the same for each sport. Each comprises the following three factors:
The third factor is important in comparing two teams' opponents, which may have the same average winning percentage, but a closer look at the strength of schedule may reveal that those two sets of opponents came to their records by dramatically different methods. The first and third factors each count 25 percent toward the total index, while factor No. 2 is weighted at 50 percent. The weighting used to be 20-40-40 until it was determined that such a heavy emphasis on factor No. 3 at times created what Wright calls false impressions, depending on the strength of a particular team's conference. There have been further refinements to the RPIs over the years that are unique to each sports committee. Some award bonus points for wins over a highly rated team on the road or at a neutral site. Some assess penalties for losses at home to lower-rated teams. Bonus or penalty points also are awarded for strong or weak nonconference schedules. In baseball and softball, there are penalties for playing too many non-Division I teams. The bonus/penalty system also distinguishes the NCAA RPIs from other sources that provide similar rating-percentage indexes to various media outlets. A time-consuming process To be sure, the proliferation of RPI-mania is a heavy weight on Wright's staff. He said each edition of a sport's RPI can take several days to compile, edit and compute. And because there's such a premium on accuracy, each statistician charged with compiling an RPI must make sure his or her desk is free of other duties to reduce the risk of error. "That's why we become concerned about every sports committee saying they'd want an RPI," Wright said. "It would be great if that's all we were doing, but it's not." Compiling every result for every team in a given sport entails a specially coded questionnaire that is filled out by member conferences and sent in to the national office to be edited for style and then sent to a data-entry firm in Kansas City, Missouri. The ensuing computer disk provides what Wright calls an edit-listing, which essentially is all the scores as provided by the conferences and independents. The program then finds any discrepancies (score differences, games added, games deleted, incorrect site, overtime games, etc.). "That's what takes the time," Wright said. "You'd think it would be a fairly error-free process, but literally hundreds of errors show up on these things, even in conference games." Wright's staff then must call participating schools to make sure the data match reality. "If a conference says a team is 10-8, we have to make sure the program says that team is 10-8," Wright said. "There's literally no room for error. If you get one fact wrong, it not only can mess up that team, but its opponents and their opponents. That's why we have to take so much time and be so sure that all these numbers add up correctly." Automatic qualification The RPI is used not only to rate individual teams but conferences as well. In fact, in 1993, the NCAA Executive Committee stipulated that sports committees that award fewer automatic-qualification berths than there are eligible conferences for that sport must use the RPI as the sole determinant. For sports like soccer, which allots 11 automatic qualifiers among 21 conferences (with the remaining conferences participating in play-ins), the issue can get dicey. "It's a little more problematic in terms of automatic qualification because the line is so thin," said LeCrone, who in addition to chairing the soccer committee is commissioner of the Midwestern Collegiate Conference. "It's very difficult to define 11 conferences by the numbers. Even more interesting is the very small number of points that separate conference No. 11 from conference No. 21. I would not have preferred the RPI as the sole determinant but only as part of the process." And it still is just part of the process as far as championship selections are concerned. While the temptation may exist for committees to make more of the RPI than necessary, LeCrone said it is each group's responsibility to keep the RPIs in the proper perspective. "I trust the committee system," LeCrone said. "I've never been involved with a sports committee whose members did not have the ability to put aside their own personal preferences and do what's best for the sport. At the same time, I want a committee to have all the tools at its disposal so that they can make well-informed decisions. The RPI does that." Masner agreed, saying that having more criteria to look at or more statistical guidance to study gives a committee the empirical support it needs to feel more comfortable in its selection. Wright said if nothing else, the sheer numbers provide committee members with a one-stop shopping list of information that can't hurt come selection time. "The best element of the RPI," he said, "is that it is all based on the scores of all the teams in that particular sport, which means that a sports committee using the RPI has access to every team, best to worst, and every game, beginning of the season to end. Even if you have no computer rankings at all -- if you just had the results -- that is a tremendous advantage to a sports committee."
RPIs in Division I SportsSport Year Began Men's Basketball 1981 Women's Basketball 1984 Baseball 1988 Women's Volleyball 1992 Softball 1996 Men's Soccer 1997 Women's Soccer 1997 Field Hockey 1998 |
And increasingly for team sports in Division I, those objective data include a rating-percentage index (RPI) produced by statisticians at the NCAA national office.