Google has open-sourced TorchTPU, a backend that lets PyTorch programs run on its Cloud TPUs as a native device. It was co-developed with the PyTorch team at Meta and announced at PyTorch Conference 2026 on October 20. Google recommends it for all new workloads and says it replaces PyTorch/XLA, the older route for running PyTorch on TPUs.

What TorchTPU changes

For years, the main obstacle to using TPUs has been software rather than silicon. Most AI developers write in PyTorch, and Nvidia’s CUDA ecosystem works with it by default. PyTorch/XLA, the earlier bridge to TPUs, recorded lazy graphs of operations and often needed developers to mark which parts of their code could be compiled. It also supported only a uniform style of parallel code known as SPMD.

TorchTPU treats the TPU like a GPU or CPU. According to MarkTechPost’s summary of the release, existing code can target a TPU through a standard torch.device("tpu"). Tensors are ordinary PyTorch tensors rather than special wrapper types, the system runs one process per device, and it preserves tensor views and in-place changes. It also handles divergent execution across devices, which PyTorch/XLA did not.

Three eager modes and a compiler

TorchTPU ships three eager modes. Strict, the default, sends each operator to the chip asynchronously with normal PyTorch behavior. Debug runs synchronously and checks tensor bounds. DeferAndFuse combines runs of operators to reduce memory traffic; the accompanying paper says it can be 50% or more faster than Strict.

For top speed, developers call torch.compile(backend="tpu"). That captures the program graph, lowers it through StableHLO to OpenXLA, and produces TPU machine code. Compiled programs are cached across processes, hosts and restarts. Custom Pallas kernels can be called like normal PyTorch functions, and distributed training uses a tpu_dist process group that supports DDP, FSDP2 and DTensor.

The numbers, and their limits

All performance figures come from Google and its partners, and we have not verified them independently. In one fine-tuning example, step time fell from 11.49 seconds with out-of-the-box eager execution to 2.78 seconds fully tuned. DeferAndFuse alone gave a 1.15x speedup, and full compilation 3.80x. That example shows the tuning work the new backend still asks of developers.

Google reports a best training result of 52% model FLOPs utilization on Qwen 3 8B using TorchTitan on Ironwood TPUs, and near-linear scaling up to 8,192 Ironwood chips. The company says it continuously tests about 5,000 Hugging Face models in both eager and compiled modes. Zenteiq, an early user, grew daily output from 1.47 billion to 5.97 billion tokens per 1,000 TPU v6e chips, though that gain came from llm-d routing on Google Kubernetes Engine, not TorchTPU alone.

What is in the release

The release spans four open repositories: TorchTPU itself, vLLM-TorchTPU, SGLang-TorchTPU and TorchTitan. TorchTPU is at version 0.2.0 on PyPI, requires Python 3.11 or newer, and is licensed under Apache 2.0. It supports TPU generations from v5e up, and serving covers FP8, MXFP4 and NVFP4 checkpoints. Hugging Face published its own post on supporting TorchTPU.

Part of a wider push

The launch follows Amazon’s TorchNeuron, which lets PyTorch apps move to Trainium with a configuration change; see our coverage of Amazon’s TorchNeuron. Together they show cloud providers attacking the same moat. To understand why that moat matters, read why Nvidia dominates AI through CUDA and how GPUs, TPUs and NPUs differ.

What to watch next

  • Independent benchmarks comparing TorchTPU with CUDA on the same models.
  • How quickly the 0.2.0 release stabilizes and whether the out-of-box eager gap narrows.
  • Whether large model builders announce a switch of training workloads to TPUs.
  • How Nvidia and AMD respond on PyTorch tooling.
  • Whether other chip makers adopt the same PrivateUse1 backend path.

Key takeaways

  • TorchTPU makes TPUs a native PyTorch device and replaces PyTorch/XLA for new work.
  • It is open source under Apache 2.0, built with Meta’s PyTorch team.
  • Performance figures are self-reported; out-of-box eager can be much slower than a tuned setup.
  • The goal is to remove the software barrier that keeps developers on Nvidia GPUs.
  • Amazon’s TorchNeuron shows rivals pursuing the same strategy.

Frequently asked questions

Do I have to rewrite my PyTorch code?

Google says existing code can target a TPU through a standard device setting, although getting top speed takes compilation and tuning.

Does TorchTPU replace PyTorch/XLA?

For new workloads, yes. Google recommends TorchTPU and says it supersedes PyTorch/XLA.

Is it free to use?

The software is open source under Apache 2.0. Running it on Google Cloud TPUs is billed as usual.