James Gwertzman, the head of cloud gaming at Microsoft, is one of the bright minds responsible for the massive growth of the video games industry in recent years. His company, PlayFab, was one of the first to offer cloud technology specifically built to support online video games, and Game Rant got to speak with him to celebrate the three year anniversary of PlayFab's acquisition by Microsoft. While at Microsoft, PlayFab merged with other cloud services and exploded in size, innovating new tech that, behind the scenes, powers countless games this very moment.
Gwertzman was able to talk at length about the potential of cloud technologies in gaming, suggesting that many games will become more like platforms than services, that massive worlds like Flight Simulator will become commonplace, and that there will be a democratization of professional-level game development tools enabled by cloud computing. However, some of the most interesting examples of cutting edge tech came from his discussion of AI, and the ways that Microsoft is experimenting with machine learning.
Our first questions about the future of artificial intelligence came on the heels of Gwertzman's discussion of Microsoft Flight Simulator, and his description of the AI process that only took 72 hours to render the entire earth in 3D from satellite data. This took an immense amount of compute power, but it was spread out across many servers worldwide. We asked Gwertzman where he sees machine learning and AI going in game development, and if gamers will see more advancements like DLSS soon.
It’s going to hit at every level of development from the early stages of development to the deepest layers of gameplay. So just an example, on the content creation side we already talked about Flight Simulator, but we also see machine learning models doing some pretty crazy stuff. We have a research project in our studios where they built a texture decompression and compression algorithm, where they trained a machine learning model up on textures. They can take a big texture, shrink it down till it’s really ugly, and then use the machine learning model to decompress it again in real time. It’s not exactly the same, but it looks really realistic. It’s almost creepy how it works. There’s a lot of areas where you can use AI as a creative aid.
The ability to scale textures up in real time has interesting implications for storage size of future games, suggesting we soon may be able to download 720P textures and then play with them scaled up to 4K. However, Gwertzman's suggestion that AI can be used as a creative aid goes a little deeper than that.
I just saw on VentureBeat the other day a new company using machine learning to help come up with game concepts. Literally the machine spits out concept art and designs. It almost felt like an April Fools article, but I think it’s legit. So you can see it from the early stage of creating concepts, all the way up to creating art and assets.
We’re already doing some things at Microsoft where we have AI bots in a testing loop running through the game, or they can watch streams and look for texture errors and geometry errors. We also see AI used for personalization. We had a project recently where we built some personalization or recommendation engines for retail. It’s being used by Walmart to make consumer recommendations. We then tried to use the exact same tech in Minecraft, for example, and it turns out it’s way more effective.... So we’re going to try and roll that out and make it more available to developers. I love finding opportunities where we find something in another industry, and we can bring it over and apply it to gaming.
The sort of industry cross-collaboration that Gwertzman finds so exciting is certainly easy to imagine at a company the size of Microsoft. Gwertzman also gave an example of a technological development that might seem worrying to most people being used for mundane creative purposes in video games.
We have voice-font technology where we can record and listen to a couple dozen hours of someone talking, and then build a model that can recreate that person’s voice saying whatever we want them to say. When you think about that in the context of deep fakes it’s deeply troubling, but when you think about its use in casting voices for games, it could allow an indie developer with a small budget to create thousands of hours of dialogue. Even better, have the computer speak using AI in whatever voice you want, you can get some really cool creative experiences. I love the idea of using voice fonts to let the computer speak in a more natural way than you typically get today. There’s neat opportunities like that coming.
Gwertzman went on to discuss the growing role of the cloud in allowing games to grow to massive sizes, whether they come from AAA studios or small indie teams. He explained that all of these professional-level development tools belong in the hands of even ordinary consumers, and the power of cloud computing can put them there. We can only imagine the kind of creativity that will bring about, but we hope Gwertzman is right when he says we'll be seeing it very soon.