Game Rant recently conducted an interview with James Gwertzman, the head of cloud gaming at Microsoft. Gwertzman was a fountain of knowledge and insight, providing a wealth of information on cloud computing, cloud services, and things to come in the games industry. From the future of machine learning to the history of cloud services, Gwertzman gave us a lot to think about. Considering the success of his company, PlayFab, after its acquisition by Microsoft, he's probably right in most of his prognostications too.
Gwertzman spoke with us in celebration of the three-year anniversary of his company's acquisition by Microsoft, during which time it grew nearly 10X. PlayFab was one of the first companies to build cloud services specifically for games, providing the backbone for multiplayer components and anything else that requires internet connectivity. At Microsoft, it's only broadened in scope, and Gwertzman was happy to give examples of how. The following interview transcript has been edited for clarity and brevity.
GR: 'Cloud gaming services' is a term a lot of readers might not be totally familiar with. Can you briefly summarize what you do? What are the goals behind the kinds of projects you work on?
JG: Games have been through an evolution in the last few years, even decades, as they’ve become more internet dependent. It used to be you bought a game on a cartridge or CD, you played it, you finished it, and you were done. It was a packaged-goods experience.... What’s been happening is that the category of games that depend on servers and internet connectivity has been expanding, and now almost any game you can think of has at least some element of being a service.... and I would argue that we’re going beyond services now and games are now communities....
Even mostly single-player games are doing this. There’s updates, there’s parts of the game that are personalized, and every player now might have a different experience. There’s analytics and data being captured from the game, which you’re using as a game designer to figure out what’s working and what isn’t. You’re not shipping a game every 2-3 years as a developer, you’re constantly updating and revising the game you’ve got with seasons, content, updates.
All of these things, the updates, the personalization, multiplayer, matchmaking, user generated content in the case of Minecraft or Roblox, all of these collectively require stuff that isn’t on your computer, it’s out on a server somewhere. More and more now those servers are not hosted on proprietary datacenters that the company builds itself, they’re hosted on the cloud, hosted by Amazon or Google or Azure.... You can launch a game and suddenly have 10X the players you expected, but as long as you’re in the cloud it’s easy to scale up and scale down again. Games are especially notorious for spiky loads, which makes the cloud attractive because it can scale to meet those workloads.
Where things are getting interesting is that companies are starting to add levels of services that aren’t just raw compute and storage, but that are dedicated specifically to gaming. That’s exactly what PlayFab was. PlayFab was one of the first companies providing a set of services that lives in the cloud that you as a developer could use in your game, and you didn’t have to worry about running the server.... You can use a service like a leaderboard, or our matchmaker, or our analytics, or content updates, or any of those services we offer. You get to just use the service, and it just works without having to worry about updating or managing it.
GR: So you’re providing a lot of backend infrastructure for the online components of games, and part of the appeal of that is it’s pre-packaged and scalable, so developers can rely on that instead of having to do it themselves.
JG: Exactly, and because of that it speeds up development, and as a developer you can focus on the good bits and the fun bits instead of focusing on infrastructure that, frankly, you just want it to work and you don’t want to focus on.
GR: We reviewed Microsoft Flight Sim, and one thing that stood out about it is the number of resources that Microsoft put into it. There’s real time weather data, real time commercial flight data, satellite data, you're streaming in assets as people are playing, and that allows the entire planet to be depicted in a game. Are we going to see more games at that scale? What comes next after a project of that size?
JG: Well, I love talking about Flight Simulator. In my mind, this is the perfect example of a cloud-native game that would not be possible without the cloud. But I would parse what you said a little more narrowly: it’s not that this is a game that was only possible at the scale of a Microsoft. It was only possible at the scale of the cloud. One of the cool things is you don’t have to be a very big company to be able to use the cloud at that level.
One of the things this game did that was so remarkable was that it took satellite data from all over the earth... so that AI could take the satellite images and imply geometry, and make up geometry for the whole planet in a way that would have been impossible for humans to do. It would’ve been tens of thousands of artists building the earth in 3D, but the computer can do it. Once you’ve built the model once and it works, once you’ve trained it, you can run the model on the entire earth. And, it only took 72 hours. It took a massive amount of compute power, but that’s the advantage of the cloud. You can grab thousands of servers for 72 hours and then release them when you’re done. That’s expensive, but you don’t have to be a big company to do that. You can lease the data from somewhere and rent the server space.
GR: The ability to render an entire planet in 72 hours is incredible. Will we see it again?
JG: First of all yes, we will see more projects of that scale.... I think what’s gonna happen is you’re gonna start to see specialization of the value chain. I’ve often said, for example, that the world of Red Dead Redemption, the amount of effort that Rockstar put into that, and to only use it for one game almost feels like a waste. It’s almost like you want to build that entire world and then have multiple games in that world. I could see a company emerging that only builds worlds, and then other companies can just lease or rent out the world for other games.
Look at a game like Cyberpunk. Part of the cost of building that game was building the whole world out. What if someone else built the world, and you just borrowed it like a backlot at a movie studio and had a live updated version for your game? We’re already seeing things like that with Roblox, with mods, with Minecraft. On one level these are games but they’re also platforms for building other games.
GR: You talk a lot about games as a platform and a community. You also just mentioned mods, and games in which players make a lot of their own content. Do you think the future of games as platforms also involves people building the games themselves?
JG: Absolutely. 100%. I think one of the things we’re going to change here at Microsoft in the coming months is that we’re going to talk less about game developers and more about game creators. The reason is that we really do believe that the class of who is making games has been broadening over time. It used to be only AAA, now there’s indies, and now we’re in a new generation and we’re in for more citizen developers. The average player can also now be a developer. Maybe it’s a simple as making a texture map and selling it, maybe it’s as complicated as making a whole level and adding it on to the world, but we’re putting these creative tools in the hands of the masses.
I’m excited for that. And for us, there’s an opportunity to take all of these services I’ve described and make them available all the way down to citizen creators. One example is Roblox.... Microsoft-PlayFab is powering the analytics platform inside of Roblox. It realized that one of the basic building blocks of any live game is analytics, so you can tell what players are doing. It’s expensive and hard to do, but you've got to do it. We have an off-the shelf-solution for that, so now the average Roblox developer can get access to that data and use it to figure out what’s going on with their players. There will be more and more opportunities to make professional grade services available to players everywhere.
GR: Everyone in the gaming community has had to pay attention to the rise of Among Us, Fall Guys, and Phasmophobia during this pandemic. Very light, easy to run games that offer an open field for human interaction. That’s similar to a lot of the things that you talk about, so is there anything you think you’ve learned from this big uncontrolled experiment that is the pandemic? A lot of these games were very small and weren’t necessarily trying to be what they became.
JG: A couple of thoughts: One is... this is a supporting vote for the cloud. When these games blew up, developers were able to keep it going because of the cloud. A couple years ago a game like Among Us wouldn’t have happened. They would have stalled out. They would have had five servers, and they would’ve saturated them, taken six weeks to get new ones, and they would’ve lost the opportunity. It’s cool that now the cloud is enabling that kind of runaway growth.
I agree that games are providing social interactions and opportunities to interact with our colleagues when we can’t in real life. I go to Burning Man every year, and last year I couldn’t go because it was cancelled. They tried to host it online, but nobody went because, frankly, I don’t think they asked a game designer to help make it. If they had asked a game company to help build it, it would have worked. We could’ve had 100,00 people be able to interact and engage with each other.
There are already virtual concert companies now, and people are starting to see that games can really bring people together from the micro level of four people up to hundreds of thousands of players, even millions of players. We’re learning from that and learning how to embrace that.
But, going back to these smaller games, it’s a reminder that a big hit can come from anywhere, and it’s not just the EAs and Activisions of the world who can afford to build big games. A garage developer with two employees can build a game and as long as they did it right, if the game takes off, they can take off with it. It means the playing field is pretty level. That’s exciting because big AAA games have huge budgets, which means they have to play it safe. They’re not going to spend $200,000,000 on a game if they’re worried it’s not gonna sell. They have to have a franchise or a hook that will ensure it’s going to sell.... What these smaller games show us is that you can afford to take risks, and many will fail, but if you hit it right you can have tens of millions of people playing your game overnight.
GR: To shift a bit, how do you see this future of creation and delivery of content through the cloud integrating with platforms like VR and mobile?
JG: ...We have a number of "industry priority scenarios." It doesn’t roll of the tongue, but they’re things that we think the industry really cares about today that may be pain points that we’re trying to help with. The first one on our list of five is to accelerate game production with the cloud....
This act of content creation, once you have it in the cloud, distribution becomes more fluid. We see this with Xcloud. It started out as just racking Xboxes in datacenters and streaming it. Now we’re getting more experience with it, and you may be able to build game experiences that would not be possible without running in the cloud. Games were you can have lots of players in a single environment interacting in new ways...
Xcloud was about putting Xboxes in the cloud *laugh* but the broader term is pixel streaming, where you’re running GPUs in the cloud and streaming it down. Initially pixel streaming is going to be useful in non-gaming scenarios like architecture or retail where you want a 3D experience but you don’t have to have the hardware. That will then move into gaming and you’ll se developers leveraging experiences that go beyond what was possible before.
GR: Can you talk about where you see machine learning and AI going in game development? We’ve seen things like DLSS, but is there more to it than that?
JG: It’s going to hit at every level of development. From the early stages of development to the deepest layers of gameplay. So just an example: on the content creation side we already talked about Flight Simulator, but we also see machine learning models doing some pretty crazy stuff. We have a research project in our studios where they built a texture decompression and compression algorithm, where they trained a machine learning model up on textures. They can take a big texture, shrink it down till it’s really ugly, and then use the machine learning model to decompress it again in real time. It’s not exactly the same, but it looks really realistic. It’s almost creepy how it works. There’s a lot of areas where you can use AI as a creative aid.
I just saw on VentureBeat the other day a new company using machine learning to help come up with game concepts. Literally, the machine spits out concept art and designs. It almost felt like an April Fools article, but I think it’s legit. So you can see it from the early stage of creating concepts to creating art and assets.
We’re already doing some things at Microsoft in testing where we have AI bots in a testing loop running through the game, or they can watch streams and look for texture errors and geometry errors.... We had a project recently where we built some personalization engines, or recommendation engines, that we built for retail. It’s being used by Walmart to make consumer recommendations. We then tried to use the exact same tech in Minecraft, for example, and it turns out it’s way more effective. It does a pretty good job of making recommendations on the marketplace. So, we’re going to try and roll that out and make it more available to developers. I love opportunities where we find something in another industry, and we can bring it over and apply it to gaming.
Another example: we have voice-font technology where we can record and listen to a couple dozen hours of someone talking, and then build a model that can recreate that person’s voice saying whatever we want them to say. When you think about that in the context of deep fakes it’s deeply troubling, but when you think about its use in casting voices for games, it could allow an indie developer with a small budget to create thousands of hours of dialogue. Even better, have the computer speak using AI in whatever voice you want, you can get some really cool creative experiences. I love the idea of using voice fonts to let the computer speak in a more natural way than you typically get today. There’s neat opportunities like that coming.
GR: Something we’ve seen from Microsoft is this very open approach, with games subscriptions going across platforms. That’s very different from the 'walled garden' approach you get from Sony and Nintendo. It seems like you’ve talked a lot about that openness, the accessibility of assets and the accessibility of platforms to more developers.
JG: One of the things I love about our team is we have a service called PlayFab party. It’s our voice chat solution. And it’s supported on a bunch of platforms including Switch, Playstation, even Stadia. We had to get special clearance from our lawyers, but we want to supply services that work across all these platforms. Every developer has the same goal which is to target gamers. They want to reach the broadest audience they can. Our Xbox team will compete with Playstation to try and get more gamers to buy our device, but at the same time when it comes to helping game developers, the goal is to help them reach gamers wherever they are.
GR: So would you say that console exclusives are not the answer to moving platforms forward?
JG: The average game developer doesn’t want exclusivity. The average game developer wants to be on every device that players are on. They want a mobile version, a console version. Console makers sometimes want exclusivity because they want their console to be special or better or whatever, but no, we don’t think it’s the answer. We’ve been pretty open about trying to support cross platform experiences.
GR: We have so many more questions, but unfortunately it looks like we’re running out of time.
JG: One thing that we didn’t talk about that I would like to cover is that we’re celebrating the 3-year anniversary of Microsoft acquiring PlayFab. I would like to close with how it’s been an interesting journey. I came to Microsoft because of PlayFab, and they acquired PlayFab because Phil Spencer had this vision of building the world’s best gaming platform. That’s no longer just about silicon on the Xbox, it’s increasingly about the cloud. We’ve had a really good run and we’ve grown almost 10X since we came to Microsoft. Now my role is shifting and I get the role of representing Microsoft more broadly across our entire cloud space. The fun job I have now is selling tools and services to developers of all sizes.
[End.]