Amazon’s TorchNeuron takes aim at Nvidia’s PyTorch advantage

The easier part is the port

PyTorch is an open-source toolkit for developing, training and running AI models. Nvidia’s CUDA platform has spent years supporting PyTorch applications on Nvidia GPUs and other company hardware, helping make Nvidia the default choice for many developers.

TorchNeuron is built on PyTorch, so developers familiar with the framework do not need to adopt a new tool. But Amazon is not alone in trying to make its chips easier to use through PyTorch: users can already move applications to AMD GPUs and Apple chips by changing settings.

The harder question is whether a simpler port is enough to shift where developers run their workloads. Amazon is also selling AWS as a broader alternative to Nvidia-linked “neoclouds” such as Coreweave, Nebius and Firmus. Its pitch is that startups can use the company’s cloud infrastructure as they move from prototype to product, then scale their computing needs.

AWS says the same Trainium chip can handle both model training and inference, the process of running a trained model to respond to a user. That could lower costs across a workflow even if AWS does not offer the lowest price per token.

Colin Brace, vice president of software at AWS Annapurna Labs, Amazon’s custom-chip division, told TechCrunch by email that AWS gives developers the infrastructure to grow a project without assembling it around a separate computing provider.

Brace pointed to Deepgram, a voice AI company that competes with ElevenLabs. He said Deepgram moved to Trainium using PyTorch and now processes 70% more audio per dollar, at production-level accuracy.

The bigger bet is on PyTorch

Amazon’s ambition extends beyond making Trainium easier to try. The company says it is working with the PyTorch community to remove code that ties the framework to specific chips, with the goal of making it less dependent on Nvidia hardware for all developers.

That is a more consequential move than a one-line configuration change. If PyTorch becomes less tied to Nvidia, competing chip and cloud providers could have an easier time attracting developers. But PyTorch’s purpose is also to hide the complexity of running AI code on different devices; loosening hardware ties could give developers more choice, or leave them with more setup work.

I think TorchNeuron’s immediate value is straightforward: it lowers the friction of testing Trainium for teams already using PyTorch. The larger promise—that AWS can make the framework itself less Nvidia-dependent—will be harder to judge. Amazon has described the goal, but not what changes developers will need to make or how much easier it will be to move between chips.

If Amazon succeeds, Nvidia’s exposure would extend beyond losing some chip workloads. The contest would also be over how much control CUDA has over the software path developers already use.