Behind the panel: A closer look at the UFC’s rankings system

It’s every fighter’s goal to achieve success inside the octagon, with that success typically illustrated by two triangles.

Wake up one day and find a green triangle next to your name, which means you’re moving up in the UFC world. An inverted red triangle the opposite. The direction you’re heading determines any number of things: Who will be your next opponent? Where will you be placed on the card? How will your next contract look?

It’s up to the competitors, their coaches, the referee, and the judges to sort out the business on fight night. In the days that follow, it’s the 22 media members that comprise the official UFC rankings voting panel who will decide what happens to the winners and losers.

The rankings have been the subject of much consternation since their inception in February 2013, with questions swirling around the media’s involvement in the rankings, the qualifications of the outlets responsible for voting, the level of transparency, and the UFC’s potential influence on the voting process.

That latter issue recently drew renewed interest when panelist Eric Kowal of My MMA News shared a tweet that included a sample of the briefing notes given to voters before every event. Specifically, Kowal singled out the wording in notes for the June 20 UFC on ESPN 11 co-main event matchup between featherweights Josh Emmett and Shane Burgos.

Kowal told MMA Fighting that his motivation for sharing this information was to help explain to fans why Burgos fell four spots in the rankings despite putting on a Fight of the Year contender in a losing effort against Emmett. As of now, Burgos currently sits at No. 14 in the featherweight rankings, three spots behind former UFC champion Jose Aldo, who is currently competing in the bantamweight division.

“I didn’t drop Shane Burgos four spots,” Kowal said. “But people were complaining about it so basically I just said after this last weekend, for people complaining about it I said, ‘Hey look, this is what the panel’s putting out. They’re giving information before each event and saying, ‘Hey, this is what a win or loss could or should mean.’ Not saying that they’re telling people how to vote, but if you’re on the panel—and you look at some of the websites on the panel, you’ve never heard of them. … I don’t know if they even watch the events. They could be going off the PowerPoint that was sent to them the week before.

“So for us to all be getting blamed about how the rankings are going—and rankings are supposed to be there to determine the next fight that the UFC sets up and also fighter pay—we’re getting blamed for it and I don’t think that’s right. I put it out there to say this isn’t all my doing. I didn’t vote for Burgos to drop four spots. This is what’s going on.”

Kowal is just one of 22 panelists whose votes are compiled by a third party that serves to keep the UFC separate from the voters. That third party is led by Kirik Jenness, the longtime record keeper for the Association of Boxing and Combative Sports Commissions, also known to fight fans as the man who runs www.mixedmartialarts.com, a.k.a. “The Underground.” It is actually his company Mixed Martial Arts LLC that was subcontracted by another company called Adept Mobile LLC to handle the rankings. According to Jenness, Adept Mobile first contacted him in 2017.

It was Jenness who reached out to media members in January to expand the panel from 14 members to 22. When the rankings started in 2013, it was the UFC that handled recruiting panelists both directly and through an application on its website, with FightMetric (now known as UFC Stats) handling the polling.

Asked directly about the UFC’s level of involvement today, Jenness answered that the promotion does not directly influence the rankings.

“The UFC is hands off,” Jenness said via e-mail. “Myself and my partner Chris Palmquist reach out to them on an ongoing but infrequent basis with relevant questions, like for example, ‘Is fighter X staying in this division or is it a one-off?’ I wouldn’t describe which fighters are eligible and in which class as a decision. If a fighter is contracted with the UFC, and fights in a division, he or she is eligible for ranking.

“I want to state unambiguously that the UFC has never reached out in any way, either directly or subtly, about any direction they want a ranking to go. It’s like a live rail — they don’t touch it. There is no oversight or editorial input whatsoever, ever. I say this on my mother’s life. Further, the software is mine, and sits on my server. No one in the UFC has access to it. They couldn’t tweak the figures if they wanted to. And they very, very, very much don’t want to. They want a ranking that mathematically reflects the honest opinion of the panel members, and they get it.”

Taking a closer look at the rankings panel, we find a smorgasbord of dedicated MMA websites, podcasts, international sites, and radio stations among others. Originally, anyone visiting the UFC’s official website could see not only the aggregate rankings, but that of each panelist. That individual information was removed from the page when the system received an overhaul in January.

MMA Fighting was able to contact 10 of the 22 panelists to get their take on the briefing notes as well as their overall impressions of the rankings system, their motivation for taking part in the process, and what they’d like to see improve.

---

Esther Lin, MMA Fighting

When talking to rankings panelists, one phrase keeps popping up: “Broken system.”

It isn’t a universal sentiment, but to some degree all of the panelists agree that there is no perfect system for ranking fighters and that the one currently in place has its flaws. With around 40 events in a typical year, the sheer speed of the UFC schedule makes the search for order difficult.

Shawn-Dallas Hall of MMA Fight Coverage, one of the media members that joined the panel in January (a list that also includes Kowal, Scott Lewis of Inside Fighting Radio, and Ashah Tafari of MMA NYTT), described his own system in which he focuses on a fighter’s previous five fights with an added emphasis on the most recent three.

Even then, it’s complicated.

“The landscape of this sport changes every single week so if you’re focusing on something that happened two and a half years ago, your rankings are never gonna be accurate to what’s really happening right now,” Hall said.

Most of the panelists who spoke to MMA Fighting said they favor recent activity and results over legacy and reputation, even though a fighter’s rise can come to a halt if they don’t get the chance to face off with a more famous name ranked above them. Other factors that drive decision-making include method of victory, quality of opponents, and hypothetical matchups. Still, so much is dependent on “MMA Math,” which rarely offers concrete answers.

Rob DeMello of KHON Honolulu, a panelist since 2015, pointed out that even an algorithm-based website like Fight Matrix can produce results that fans may disagree with, since its emphasis on strength of schedule can keep a fighter like Conor McGregor locked in near the top of the rankings off of a few big wins. It’s not the strength of schedule that concerns DeMello so much as how unbalanced it is.

“With mixed martial arts, there will never, ever, ever be a perfect system because people aren’t fighting the same amount of times a year,” DeMello said. “And not only that, unfortunately they’re not fighting as competitively as they should be, where 12 fights 11, 10 fights 9, 8 fights 7 and then let’s see what happens.

“They’re all independent contractors and can accept or not accept fights and therefore you’re going to have situations where the No. 10-ranked fighter isn’t taking fights against No. 9 or 8 and he’s fighting against No. 17 and 18. So it’s really hard to judge on where a guy is ranked.”

Another problem that has come up with some frequency over the past few years is highly-ranked fighters taking a break from action. McGregor has thrice announced his retirement from competition, most recently in June and yet still remains eligible to be ranked. Several of the panelists expressed confusion about McGregor’s case specifically, especially since Henry Cejudo has already been removed from the rankings following his recent retirement announcement.

Similarly, David Brown of the Cherokee Scout, a panelist since the nascent days of the rankings, told MMA Fighting that panelists should have some say as a group regarding when fighters should be are ineligible to be ranked, rather than just making the call on their own, which can lead to discrepancies in the aggregate rankings.

“I wish we had a little bit of influence on that, honestly, because it would be nice to have a higher level of consistency,” Brown said. “I’ve handled these situations before in the past and thought to myself, ‘Should I take it upon myself?’ For instance, when Nate Diaz went that long time between fights, he was still available to be ranked for a long time before he was finally pulled and it was very difficult knowing should I keep him in or not?”

Esther Lin, MMA Fighting

Even if one feels they have a foolproof system, the math behind the rankings can lead to results that leave the panelists themselves scratching their heads. Lewis, a radio host who has known Jenness for years and is best recognized in the MMA community as the cornerman and mitt holder for 71-fight veteran Joe Riggs, acknowledged that one has to accept that their opinion is a small part of a greater whole.

“I would say yes and no,” Lewis said when asked if he feels his opinion is accurately reflected in the public rankings. “Some things I don’t understand, but again, if there’s 22 people in a panel and of those 22 people, let’s say 11 of them feel a certain way, those 11 are probably going to have enough power to put somebody in a ranking that they feel strongly about.

“Personally, I watch a lot of fights. I’m very motivated to get the right ranking—I watch every fight. I’ve been like this since the day I got involved in this sport back in 1999—but even more so now because the rankings I take really seriously. I don’t understand little things, like for a while Max Holloway was still ahead of Dustin Poirier on the pound-for-pound rankings after Poirier had just beaten him. I didn’t get that.”

For Jenness, he’d like to see the system evolve to the point where the panel can grow even larger both in terms of size and international reach. He recalls that there may have been closer to 100 panelists when the rankings started, which is in contrast to the 28 figure mentioned in a rankings feature published by THE SCRAP in September. He believes the greater the number of voters, the less likely the process will be affected by “outlying votes” or “temporary absences.”

“In an ideal world, I would like to expand the ranking panel further; from a technology standpoint, it can accommodate any number of panel members,” Jenness said. “That said, as evidenced by the experience with the original ranking panel, it can be problematic. We are working on creating ranking algorithms that have the potential to provide further information to the ranking panel members in making their decision.”

---

So why do it? If a panelist runs the risk of not only raising the ire of the MMA community but possibly negatively impacting the livelihood of an athlete, what is the upside in participating when you know that the deck is stacked against you?

The general consensus among the voters is that they feel it’s their responsibility to provide an educated opinion because if they don’t, someone possibly less qualified will. If the UFC is determined to have X amount of panelists, then one could argue that dropping out of the process is not unlike abandoning one’s station.

“I’ve had multiple conversations with colleagues of mine and friends of mine that are prominent MMA writers that know that I’ve voted,” DeMello said. “Some of them tell me, ‘You really shouldn’t be voting because a lot of people want to see this system collapse.’ My thing has always been, hey, I feel you, I totally understand where you’re coming from, but it is the rankings that is in place and if I know that I’m putting in the effort and I’m putting in the hours that I do to do these rankings, then at least I can feel good about at least [1/22nd] of it is being done the right way.

“Now, should I be the person voting? Do people agree that I should be one of the people? That [question] I understand. But the fact of the matter is that I was offered the opportunity and I trust myself that I will put in the proper work and treat it with respect, so as long as this system is in place I feel like I’m not gonna give it up so that some other person can come in, because who knows who that person is?”

As far as backlash goes, the panelists understand that criticism come with the territory. MMA Soldier’s Rodney James Edgar, a former panelist who rejoined in January, recalls being publicly called out by top bantamweight contender Aljamain Sterling.

“Most of the time when people criticize, they’re actually wrong,” Edgar said. “Aljamain Sterling called me out on Ariel [Helwani]’s show, I was like, ‘What?’ He was like, ‘Who are these people? I don’t even know who these people are.’ So I confronted him and he was like, ‘Yeah, I know who you are, Rodney, I didn’t know that was your brand.’ And I just said, ‘Look, Aljo’-I love Aljo-‘if something’s wrong with them, tell me what’s wrong and what you think I should fix.’ I never got a response and that’s usually what happens.

“People complain and I will respond to them on social media. What should we change? And they never have anything of substance if they even reply.”

Esther Lin, MMA Fighting

For better or for worse, the panelists are as passionate about the sport as the fans, and while that may mean being completely unbiased is impossible, it also explains why they’ve chosen to be take part in a project that provides no compensation or satisfaction outside of the sense that one has supposedly contributed to a greater cause.

“I personally enjoy it,” Hall said. “I’m an MMA junkie. I watch everything. Every fight, everything from local cards to the LFAs to Bellators to Invicta, if it’s on I’m watching it. For me, I enjoy it. It’s a lot of work, at least the way I do it is a lot of work. But for me it’s a lot of work. It gives me an insight into the landscape of mixed martial arts that provides me with talking points anywhere I go.”

---

Now about those briefing notes.

One thing all the panelists wanted to make explicitly clear is that they’ve never felt pressure from the UFC or Jenness to vote a certain way, regardless of what is suggested in the information provided to them. Even Kowal, who has been vocal for months about his concerns with the system, said the panelists primarily communicate with the team at Adept Mobile and that he’s only received the occasional note directly from Jenness, including one incident in which he was told not to announce that a fighter was dropped from the rankings for switching weight classes because the fighter themselves had not been informed of the change yet.

Usually, the notes are strictly to provide information and context, though there is always an “upside (win)” and “downside (lose)” section included. According to the panelists, the Emmett-Burgos notes were atypically detailed as far as level of guidance that is offered.

“Now what they do though is they say, for example, I saw this comment on Burgos: ‘This may remove him from the top-15,’” Lewis said. “I took that as assuming that if he had a really bad loss, then he might be removed from the top-15. I don’t think he should have moved beyond 12. I know he’s at 14 right now. I would have kept him above Aldo, but behind Arnold Allen. But that’s me.

“At the end of the day, I don’t think him moving to 14 as opposed to 12 is that egregious, but I also know that I don’t look at those briefs and I wasn’t given those briefs with the mindset of ‘That’s how you have to pick’ because I haven’t picked that way. I haven’t followed the briefs and I haven’t gotten one call to say, ‘Hey Scott, you’re not following the briefs. Why not?’”

Edgar was also quick to describe the value of the notes, though he and several other panelists stated that because of the level of research they conduct on their own, they don’t always read them cover to cover.

“Those briefings are in general pretty outstanding,” Edgar said. “I do want to defend them in that regard. They’re usually really well-informed. But I did notice some discrepancies in the last one, I kind of glossed over the briefing on that one because I didn’t feel like I needed that information as much because I felt I already had a grasp on it.”

Photo by Chris Unger/Zuffa LLC

Some of the voters feel that they could be given more criteria when it comes to shaping their rankings, if only to see more consistency in the final results. Currently, they’re explicitly instructed to rank as they see fit, which leads to freedom of opinion but also frustration when that opinion is lost in the shuffle due to everyone following different parameters.

“The UFC Rankings can definitely use more of a criteria than this vague answer we get in the FAQ over at the ranking portal,” Tafari said via e-mail. “There is no single answer to this question, rather, the decision of where a fighter ranks is left entirely to your judgment.

“In general, there are two main strategies people use: Rank fighters based on who is ‘better.’ In this way, you should rank fighters in an order so that you would expect a fighter to be able to beat those fighters ranked below him most of the time. It is possible in this case that a fighter might still be ranked ahead of someone who just defeated him if you believe the loss was unjustified or unlikely to happen if given another shot. [The other is to] rank fighters based on who is more deserving of a title shot. In this method, more credence would be given to results, so that a fighter who wins will almost always be ranked ahead of a fighter he defeats because the victory gets him closer to the title shot.”

On the other hand, Brian Hemminger of MMAOddsBreaker counters, “I think if there were too strict of outlines, then all the rankings would be the same and it would take out some of the subjectivity, so I’m kind of okay with it.

“I just want to make sure that if you’re being subjective about it, do it for a reason. If you have something different, you need to be able to justify it.”

Photo by Chris Unger/Zuffa LLC

Of the 10 panelists spoken to for this feature, all but one said that for the most part they handle the rankings independent of their respective site. Only Jaser Davari of Sweden’s Kimura.se said that he votes by committee, compiling the rankings of nine of his staff members including himself before submitting his rankings.

Suggestions for improvement ranged from taking elements from college football’s ranking system such as RPI (Rating Percentage Index, which uses a formula to measure strength of schedule), to having a more diverse panel (currently, 17 of the 22 panelists are based in the United States), to being more aggressive with removing panelists whose rankings are consistently outliers.

Kowal would like to see more clarity in the voting process, because when you vote, you’re tweaking your own previous rankings as opposed to the latest official rankings, which can lead to discrepancies in the aggregate. Pappani went as far as to suggest that a fan element be added given how vocal the public can be about the rankings, though he acknowledged that could be problematic.

“Any system is going to have flaws,” Pappani said. “The thing that I may do is have a fan vote be part of the aggregate. That way, the fans also have a voice and then there wouldn’t be maybe as much complaining because they have a say in it as well.

“The only thing you get tricky with the fan vote is that maybe you’re just voting for your favorite fighter at No. 1 and then it can throw the whole thing off.”

MMA Fighting reached out to the UFC for comment on the rankings, but did not receive a statement as of this publication.

Despite their frustrations, the panelists all expressed a desire to continue working towards a better system, even in the face of criticism of fans and peers. There’s a certain pride to be taken in believing that order can be attained in the lawless land that is MMA, even if the results leave something to be desired.

That optimism extends to Jenness himself, who spoke highly of his working relationship with the panel and the UFC.

“It is an honor to serve this sport as I am able to, however modestly, and handling the rankings in a fair and effective manner is a central part of that,” Jenness said. “I also appreciate the criticism. The passion fans feel for the sport is like no other; it buoys and inspires me. People say stuff like that all the time, but I really, really mean it.”