Developer Insights: Ratings and Matchmaking in Hearthstone Mercenaries PvP

Greetings! I’m Tian, a Lead Data Scientist on the Hearthstone team. In this blog post, I am going to talk about some details behind ratings and matchmaking in the PvP Fighting Pit of our brand-new game mode, Mercenaries!

The Mercenaries rating and matchmaking system is the most complicated rating and matchmaking system we’ve ever used in one of our game modes! At its core, though, we had two simple ideas:

• For ratings, our main goal was to make the value best indicate player skill over time, rather than Mercenary level, Ability/Equipment level, or team composition.
• For matchmaking, our main goal was to achieve “fair” matches within a reasonable queue time, taking into account your rating, Merc levels, Ability/Equipment levels, and Party Roles.

There are several factors that impact how we update your post game ratings and how we find you an opponent during the matchmaking phase, including: your ratings, the level of your Mercenaries, the Ability and Equipment tiers of your Mercenaries, the Role breakdown in your Party, and how long you have been in queue. Let’s talk about each of those things here.

1. Ratings

Similar to Battlegrounds and Duels, there are two ratings in this system: your internal rating and your external rating.

The internal rating is a value that represents our current belief of your current skill level, and does not get reset at the start of each season. It is set after you’ve played a number of initial placement games, and then it slowly adjusts over time as you play (or if you take a long time away from the Fighting Pit). That value is invisible to you.

The external rating is the number you see, which is reset at the start of each season to add meaningful stakes and progression each season. As more games are played in a season, it can also come to closely represent our belief of your current skill level.

The leaderboard uses your external rating whereas matchmaking uses your internal rating. The discrepancy between your internal rating and your opponent’s is an important factor in predicting your win probability against the opponent before each match (other factors are discussed below). The win probability is then compared to the actual result to calculate ratings changes.

For some basics of dual-rating system, you can refer to my previous blog for Battlegrounds (https://playhearthstone.com/en-us/news/23523064 ).

2. Mercenary Levels

The levels of the individual Mercenaries on your Party play a big role in both rating updates after a match and matchmaking before the match begins. For example, the system has a preference for you to face against someone with the same level Mercenaries as you (especially when ratings and other factors are the same). If you are paired against an opponent with a higher-level Party, though, that would tend to make your win probability lower. That, in turn, means you will likely be rewarded more rating points after the match if you win, or lose fewer points if you lose.

Of course, players won’t always have perfectly equally-leveled Parties. We created a formula to evaluate when that common situation happens. In mathematical terms, the formula takes account of the maximum level, average level, variation, and skewness of the Mercenaries in your Party to generate a “proxy” for how strong that particular Party of Mercenaries is. We call that your “level score.”

After each game, your level score is considered, along with your relative internal ratings and the other things discussed in this article, to provide a “rating compensation value” that dictates how much rating you gain or lose for that game.

3. Mercenary Ability Tiers

The tiers of each Mercenary’s Abilities are also taken into account in both matchmaking and the rating system. Similar to your Party’s level score, an “ability score” is calculated to represent how strong your Mercenaries’ Abilities are. That ability score is then transformed, by a non-linear function, to help predict your win probability against your opponent and to calculate post-game rating updates.

Note that your ability score matters much less if you and your opponent’s Mercenaries are all at maximum level (the level cap at launch is 30). Under those circumstances, the other factors, including relative internal ratings, are more heavily weighed.

4. Mercenary Equipment Tiers

The algorithms for calculating scores and win probabilities regarding your Merc’s Equipment are very similar to those about its Ability tiers, mentioned above, but with different parameters. One big difference is that you are not required to have any Equipment equipped to your Mercenaries. If you don’t equip up, that Mercenary would be treated as if it were bringing a “tier 0” Equipment.

5. Mercenary Roles

As you know by now, there are three different types of Roles in Mercenaries: Fighter, Caster, and Protector. Each Role deals double damage to one and takes double damage from another.

Mercenary Role choice does not have any effect on your post-match rating updates, but it does have an impact during the matchmaking phase. The impact is much less significant than your Party’s levels and Ability/Equipment tiers, but we do consider Roles to lower the chance of “extreme” match cases that could result in unfun play experiences, such as six Fighters versus six Casters, or a six Protectors mirror match. To achieve this, we use an idea very similar to the mathematical concept of “information entropy”—where higher entropy means more variety in Mercenary Role composition. We then multiply that entropy score by a characteristic function to make sure that only the extreme match cases are affected. Our goal is to find the right balance where players can play with any Mercenaries they want, but where players are also still incentivized to consider the mode’s core Role system.

6. Queue Time and AI Opponent

Time spent in queue is an important factor for any matchmaking system. While getting a good quality match is important for fun gameplay, a very long queue time also tends to create negative play experiences. With so many variables considered in the Mercenaries matchmaking system, we wanted to avoid a situation where matchmaking took an inordinately long time to find an opponent that was just right.

In current Mercenaries, if your queue time runs longer than 1-1.5 minutes, and both your internal and external ratings are below certain thresholds (the external rating threshold is 7000), you will be given an A.I. opponent. That threshold means that more casual players will always have a quick queue, but the most committed players can still compete with each other for top leaderboard spots.

At this time, we have only one A.I. difficulty level, so we adjust the A.I. opponent’s difficulty by adjusting the A.I.’s Party’s levels to accommodate your internal rating. If you bring a Party with a really wide range of levels and you get paired against an A.I. opponent, the A.I.’s team will mirror your team’s level (instead of picking some sort of average level), and then adjusting the A.I.’s Party’s level based on your rating. Your post-match rating updates will also be affected by whether you played against A.I.

Additional notes

We are not considering which exact Mercenary, Ability, or Equipment you bring into the Fighting Pit in rating updates or matchmaking. Two different Mercenaries with the same level, Ability and Equipment tiers, and Roles are treated the same. There is no calculation for whether Tyrande or Rexxar is “inherently” stronger, or more meta. We’ve chosen to leave those kinds of edges for the community to find.

Last but not least, I want to say that there are many hyper-parameters that control this rating/matchmaking system, most of which we obtained through predictive, mass simulation of A.I. vs A.I. matchups. We will closely monitor the rating and matchmaking system after launch and update the algorithms and/or variable values as warranted by the live data. We are constantly listening to your feedback, so please share your rating and matchmaking comments with us.

We hope you enjoy building your Party, heading into the Fighting Pit, and climbing up the ladder. We’ll see you all in the Tavern!