‘Fallout’ Devs Lost Out: How Publishers Use and Abuse Game Reviews

By | 2 years ago 
Fallout: New Vegas Screenshot

Fallout: New Vegas didn’t get good enough review scores to earn its devs a bonus. Image Source: Welshpixie via Flickr.

Review scores are an easily digestible way to determine whether you want to buy a game or not. While each publication differs slightly in their scoring system, one thing is for certain: those scores are for the convenience of their audience, not for game publishers to use to further their own ends.

A certain amount of use by publishers is expected—if a game gets a good score, publishers are going to use that to promote the game. That’s understandable, but that’s not the only way publishers are using game reviews these days. The relationship between publications providing review scores, aggregators like Metacritic, and game publishers is currently under a lot of scrutiny for the potential use and misuse of game scores.

Obsidian Takes the Financial Hit for a Near Miss 

Fallout: New Vegas was a divisive game at launch, earning high praise for its inventive story and tried-and-true Fallout worldbuilding, but attracting a lot of criticism for frequently game-breaking bugs, including crashing, freezing, and saved game corruption. It currently holds an 84 for Xbox 360 on Metacritic, meaning it received generally good reviews despite glitches and other issues.

Unfortunately, that 84 wasn’t enough to earn the developers a bonus. Obsidian Entertainment, Fallout: New Vegas‘ developer team, signed a contract with publisher Bethesda, stating that if the game received an 85 on Metacritic, the developers would earn a bonus.

Admittedly, Obsidian knew the bonus wasn’t guaranteed when they signed the contract, but that doesn’t change the shady nature of the deal. Publishers know developer teams have a lot to gain from taking on a big title, and that titles like Fallout will sell no matter what—making it easy for publishers to push their own advantage.

Fallout: New Vegas Screenshot

Though Obsidian admits they bit off more than they could chew, that doesn’t make the promise of a bonus for good review scores ethical.

When Contracts Set Devs Up for a Fall

Feargus Urquhart, CEO of Obsidian Entertainment, later said that they agreed to a compressed timeline for Fallout: New Vegas, and subsequently they wound up scrambling a bit in development. That compressed timeline, which the publisher pushed for, ultimately worked in Bethesda’s favor. Fallout was released with bugs that drove down its Metacritic score (to one point shy of the target), and subsequently the publisher didn’t have to pay Obsidian a bonus.

The issue here is not that Obsidian’s product didn’t live up to publisher expectations; by most counts it did. Fallout: New Vegas sold almost eight million copies globally, earning Bethesda a solid chunk of change. Despite that, Obsidian Entertainment faced layoffs and had to cancel an upcoming game just two years after Fallout‘s release.

Misuse of Game Reviews Reflects a Larger Problem

It’s no secret that the relationship between publishers, developers, and consumers can be a frustrating one. But seeing a company use review scores—intended as a convenience for game consumers—as a dangling carrot bonus with a significant chance of failure is disheartening. It would make more sense to award bonuses for hitting higher sales goals, such as if the game sold five million or ten million copies. Basing bonuses on Metacritic review scores, which are based on varied, subjective numbers often scaled by a publication’s size and reputation, is kind of like offering a bonus based on the daily weather. Sure, you can make an informed estimation of what the weather will be like on a given day, but it doesn’t account for a stray breeze or an inaccurate thermometer or human errors in prediction.

Fallout 3 Brotherhood Soldier

Awarding bonuses for a certain Metacritic score also seems to undermine or dismiss the high-pressure environments developers work in, or the looming intensity of release deadlines. Long hours, overtime, and frequent layoffs are a common theme in the gaming industry, and contracts such as the one between Bethesda and Obsidian may only encourage the exploitation of smaller developers. No risk, no reward, but when big-name publishers reap the benefits of developers’ rush jobs and overtime, there’s an incredible power imbalance.

Publishers have been known to push for the release of games that aren’t quite ready, and more and more they seem to directly influence the actual game content as well. When you couple these facts with the outcome of contracts like Obsidian’s, it leaves a bad taste in your mouth. In spite of the fact that publishers often set the stakes, the developers are hit hardest when a game fails to meet expectations. When layoffs happen in the games industry, it’s far more common to see them at the developer level than at the publisher level. This is partly because developers typically don’t need as many people after a large release, but it’s also reflective of the way the industry treats devs as being disposable.

Gamers will always want a quick and easy way to evaluate a game before they buy, so game reviews will always be needed. But using review scores to contractually exploit developers, to the benefit of publishers, needs to stop. There may be significant growing pains before we see a more satisfying relationship develop between publishers, developers, and consumers. But hopefully, in the end, game reviews will return to being just a useful tool for gamers, and a means of valuable feedback for developers—not a bargaining chip for publishers.