Microsoft wants your PC’s hardware to get smart—and your gaming foes to become even more devious. Earlier this month, the company revealed Windows ML, an API that taps into your computer’s CPU and GPU to bolster your software with machine learning capabilities. That wasn’t the end of it though. At GDC 2018 on Monday, Microsoft explained how Windows ML can benefit video games, and introduced new “DirectML” tools that provide GPU hardware acceleration for games that use WinML, built on the same no-hassle-for-gamers principles as the DirectX standard.
That’s a lot of buzzwords. What does it all mean? Machine learning can make games prettier, more adaptable to individual playstyles, and easier to create, Microsoft says. Let’s start with the eye candy.
“[Deep neural network] models can learn to determine the best color for each pixel to benefit small images that are upscaled, or images that are zoomed in on,” Microsoft says, pointing to this machine learning-enhanced image upsample by Nvidia as an example. (Be sure to zoom in!) “You may have had the experience when playing a game where objects look great from afar, but when you move close to a wall or hide behind a crate, things start to look a bit blocky or fuzzy – with ML we may see the end of those types of experiences.”
Nvidia, via Microsoft
ML super sampling (left) vs. bilinear upsampling (right)
Improving up-close visuals isn’t the only way machine learning can level-up games. Developers can use the new Windows ML and DirectML tools to adapt your gaming experience to match your specific tastes. Here’s what Microsoft says:
“If a DNN model can be trained on a gamer’s style, it can improve games or the gaming environment by altering everything from difficulty level to avatar appearance to suit personal preferences. DNN models can be trained to adjust difficulty or add custom content [that] can make games more fun as you play along. If your NPC companion is more work than they are worth, DNNs can help solve this issue by making them smarter and more adaptable as they understand your in-game habits in real time. If you’re someone who likes to find treasures in game but don’t care to engage in combat, DNNs could prioritize and amplify those activities while reducing the amount or difficulty of battles. When games can learn and transform along with the players, there is an opportunity to maximize fun and make games better reflect their players.”
It’s exciting, intriguing stuff—at least in theory. We’ll need to see Microsoft’s new tools used in real life to see if the reality matches the hype.
On the developer side of things, machine learning can be embraced to simplify workflow. Nvidia recently partnered with Remedy to help create facial animations in Quantum Break. Facial animation can take a lot of time and effort, but after training Nvidia’s machine learning solution on the voice and faces of actors, it was able to generate facial animations automatically that were 80-percent complete simply by listening to lines of dialogue. Remedy’s artists only needed to focus on the finishing work.
Microsoft plans to publish a deeper technical look at these machine learning tools on the DirectX blog at the same time this post goes live, and it’s partnering with Nvidia to present two sessions at GDC 2018 focused on “using artificial intelligence to enhance your game.” The company is also partnering with Unity to provide support for the engine’s ML-Agents framework. Microsoft’s PIX for Windows tool will let developers start playing with WinML and DirectML today if they sign up for the Windows Insider program and snag the latest flight. Look for official support to land sometime later this year.