Google and Amazon make PyTorch easier to move beyond Nvidia

PyTorch’s Nvidia roots

PyTorch is a popular open-source toolkit for building, training and running AI applications. Some estimates put it behind 85% of them. Meta developed it in 2016, when it was still Facebook, and handed it to the Linux Foundation in 2022.

Nvidia GPUs were already standard equipment for AI research when PyTorch was created, and many of the framework’s capabilities were built with them in mind. That history helps explain why cloud companies are now working not just to offer alternative chips, but to make PyTorch work well on them.

Two paths off Nvidia

Google’s TorchTPU lets developers move existing PyTorch applications to Google’s tensor processing units, or TPUs, and Google Cloud by changing settings rather than rewriting the application.

Amazon announced TorchNeuron on Monday. It moves PyTorch applications to Amazon’s Trainium chip and AWS without a code rewrite. Amazon is also contributing code to the main PyTorch project that separates framework capabilities from Nvidia chips, with the aim of making support for other vendors’ hardware easier by default.

Google says it has worked directly with Meta, Hugging Face, Inferact, the creator of vLLM, RadixArk, the creator of SGLang, and other partners to make TorchTPU and its code libraries work with popular AI development tools.

The company also released TPU agent suite, a toolset for moving applications to Google chips and cloud services and finding and fixing errors during the process. It updated LiteRT, its tool for running AI on phones, laptops and embedded devices, to work better with PyTorch applications.

Easier to move is not a reason to move

I think the more important test is not whether an application can run on an alternative chip, but whether developers have a reason to take it there. Google and Amazon are reducing migration work, but the announcement gives no evidence that developers will switch at scale. Nvidia’s advantage is not erased by a simpler settings change.

That leaves the companies competing to change what “default” means in AI development. If PyTorch becomes more hardware-agnostic, Nvidia may face less friction from developers who want to use alternatives. But making alternatives easier to reach is only the opening move; the tools still have to make those alternatives worth choosing.