Throughout Destiny 2's life cycle, Bungie has never been able to capture the same magic it did with the original iteration of Destiny's Trials of Osiris. Of course, it is an uphill battle for the developer, as it is not just battling sandbox differences between the original and the sequel, but also diminishing returns on the fans' part as they reminisce about weekends of Flawless run after Flawless run in the original Destiny.
This season, however, Bungie did the seemingly impossible by making Trials of Osiris rewarding and fun to play once again. During the first weekend, the people voted simply by showing up, playing more matches than any weekend before (whether in Destiny 2 or the original Destiny), and by playing more consistently each day of the weekend. By the end of the weekend, Bungie reported that players played a total of 2.8 million hours of Trials (600,000 more than the previous record). It also revealed that 750,000 players participated in Trials of Osiris, with 237,000 of them going flawless that weekend.
This successful launch could have been brought about by several things: Bungie partnered with BattlEye Anti-cheat at the beginning of the season, Bungie implemented more pathways to receiving Trials rewards and reworked Trials of Osiris, and the weekend was the first week of Trials for the season after a short hiatus. Immediately after this successful weekend, however, Bungie made it much harder to pinpoint what led to the high player count when it started experimenting as opposed to allowing the new system to play out.
The Problem With Bungie's Approach
After this first weekend, with many confounding variables that made it difficult to attribute what exactly led to its success, Bungie threw some more variables into the mix. Rather than seeing how the multitude of changes it made to Trials (including reduced round times, changes to special ammo economy and the addition of a reputation rank system) would play out after the initial burst of enthusiasm, Bungie opted to change the mode even more after receiving only a weekend's worth of data. In week two, Bungie implemented the flawless matchmaking pool, a system that placed players into a separate pool of players once they achieved flawless in a given weekend. The pool would reset each week, and players would be locked into it after going flawless. Meaning, even if players reset their cards, they would match against others who had gone flawless and reset their own cards.
After week two, Bungie tested many different changes including Trials Labs modes that featured a capture point added to the normal Trials game mode on one weekend, and a freelance solo queue option another weekend. Bungie refined the flawless pool, made it kick in on Sundays, got rid of it in favor of overall win-based matchmaking, and then reverted it. All of this has taken place since the first weekend on Sept. 9, but Trials has never gone back to the first weekend's ruleset.
This is a problem for a few reasons. First, some Destiny 2 fans look to that first weekend as the ideal Trials experience, and they often point to the numbers as support for that opinion. While that does not completely work because the first weekend of Trials during a given season or expansion almost always has an inflated player count, it is worth noting. Next, to experiment like Bungie has been doing with Trials, there is a need for a control group. One weekend's worth of data that was collected during a weekend where the player base may have been skewed in a positive direction does not make for a solid control group. Lastly, Bungie made many changes with the first weekend of the season that have been overshadowed in large part by the Labs modes and the addition of the flawless pool.
Each one of the changes made in the first Trials revamp this season deserves to be looked at individually before moving swiftly to the next problem. Bungie has signaled many times this season that Trials of Osiris is in an experimental state so that it can be in a relatively stable state by the release of The Witch Queen expansion, but the experiment it has been conducting is fundamentally flawed. Each weekend brings new variables such as map changes and players getting burned out with the game, so adding on the flawless pool only creates more confusion when trying to pinpoint what makes the mode popular and what it is getting right.
How Does Bungie Fix The Trials Experiments?
Destiny 2 needs to revert Trials of Osiris to the first weekend's set of rules and see how the game plays out on various maps before trying out new things such as the flawless matchmaking pool. Doing so would ensure that the team has a strong sample size of data to look back on when implementing new changes, using a sample that serves as an accurate control group this time around. This is not to say that the first weekend of Trials of Osiris this season was the best, but there is no way to tell which way is the best if the first ruleset is not given a fair chance to play out later in the season.
This point in the season would be the best time for Bungie to implement this change, as the player count is at a low in anticipation of next month's content drop as part of Bungie's 30th Anniversary Celebration. The data collected now would certainly not be affected by the beginning of the season surge in the player base, and it would serve as a more reliable baseline moving forward.
Regardless of what Bungie chooses to do with Trials moving forward, this 'experiment' lacks a sufficient control group to measure the success of various changes against. This is a shame, as a majority of the changes implemented this season resulted in a system that already encouraged players to keep their flawless cards to continue earning adept loot and to keep playing other teams with seven wins. It would have been nice to see if that system could have shone more brightly without the addition of the flawless pool so early into the life of these changes.
Destiny 2 is available now on PC, PS4, PS5, Stadia, Xbox One, and Xbox Series X/S.
Source: Bungie