Google challenges Nvidia's AI grip with open-source TorchTPU

Google said Tuesday that TorchTPU, its own version of the PyTorch toolkit, is now available as an open-source project — the latest move to chip away at Nvidia’s hold on the AI industry, where most AI applications default to running on Nvidia GPUs. PyTorch, the open-source toolkit AI developers use to build, train, and run models, powers an estimated 85% of AI apps; Meta created it in 2016, when Nvidia chips already dominated AI research hardware, before handing it to the nonprofit Linux Foundation in 2022.

TorchTPU lets apps run on Google’s own AI chips, Tensor Processing Units, and migrate existing PyTorch code to TPUs — and by extension Google Cloud — through a simple configuration change rather than a rewrite. The move echoes a step Amazon took a day earlier, releasing its own PyTorch version, TorchNeuron, which ports AI apps to Amazon’s in-house Trainium chip and AWS without rewriting code. Amazon also said it is submitting code to the core PyTorch project to decouple various features from Nvidia hardware, making the toolkit more open to any chip by default.

Google may be going further than Amazon in some respects: it says it worked directly with Meta, Hugging Face, Inferact (the maker of vLLM), and RadixArk (the maker of SGLang) to make sure TorchTPU’s code libraries work as smoothly with those popular AI tools as the original PyTorch does. Google also launched a dedicated TPU agent suite to help developers migrate apps and troubleshoot errors, and refreshed LiteRT, the tool it uses to run AI models on phones, laptops, and other embedded devices, so that it handles PyTorch-based apps more smoothly.

None of this guarantees developers will abandon Nvidia hardware en masse now that alternative chips and clouds are easier to reach. But Amazon and Google are both working deliberately to chip away at Nvidia’s advantage.

Why it matters

Lowering the switching cost from Nvidia GPUs to custom silicon strikes at the software lock-in that has underpinned Nvidia’s AI dominance — even if hardware performance gaps mean developers won’t necessarily switch right away.