Google AI chips: TPU v5p vs Trillium for training

on-demand price than TPU v5p. But the first training run often exposes the real constraint. The model does not fit comfortably into 32 GB of HBM per chip, the parallelism plan becomes more complicated, or the available capacity sits in a region the data and compliance teams cannot use.

That is the practical distinction in the Google AI chips discussion. TPU v5p and Trillium, commercially known as TPU v6e, overlap as large-scale training infrastructure, but they optimize different parts of the operating problem. One puts unusually large memory and high on-chip data movement at the center. The other emphasizes compute density, smaller pod units and scaling across pods through the data-center network.

For IT leaders, the question is not which TPU has the larger headline number. It is where training jobs are currently losing time, where workflow friction will emerge during migration, and whether the lower chip-hour price survives contact with the full training bill.

Architectural divergence: a large 3D fabric versus modular pods

TPU v5p was announced in December 2023 as Google Cloud’s high-performance training system. Its physical Pod contains 8,960 chips connected in a 3D-torus topology. Google documents a maximum schedulable single job of 6,144 chips, or 96 cubes, with Multislice configurations extending training to 18,432 chips.

That architecture matters because distributed training is not merely a matter of adding accelerators. Every additional chip adds communication work: gradient synchronization, tensor exchanges, optimizer-state sharding, checkpoint handling and coordination between hosts. A large, tightly integrated topology can simplify the assumptions behind a very large training run, especially where the model architecture demands frequent communication.

Trillium, announced in May 2024 and generally available from December of that year, takes a more modular route. A TPU v6e Pod contains 256 chips arranged in a 2D torus. Supported slice configurations begin with one chip and run to 256 chips per pod; larger training deployments span pods using the data-center network.

This is not a cosmetic topology change. It changes how an infrastructure team plans capacity and how an ML platform team thinks about scale-out behavior. With v5p, the natural unit of thought is a large, high-bandwidth single-slice environment. With Trillium, teams can start smaller and build across pod boundaries as a job proves it needs more capacity.

The table makes one point clear: calling Trillium the universal replacement for v5p would be too neat. Trillium is a sixth-generation TPU, but it is not simply v5p with faster clocks and more memory. In several of the specifications most relevant to communication-heavy training, v5p remains the larger per-chip system.

A TPU decision is usually a model-shape decision before it becomes a procurement decision.

Compute density and memory: the trade-off hidden behind “2x faster”

Trillium supplies 918 TFLOPs of peak BF16 compute per chip, exactly double TPU v5p’s listed 459 TFLOPs. For training teams whose runs are primarily compute-bound, that is an attractive starting point. More peak arithmetic per chip can reduce the number of chip-hours needed for a given training workload—provided the model and its data pipeline keep those units busy.

Here is the catch: peak compute is not equivalent to training throughput.

A modern foundation-model training run does not consume compute in isolation. It moves activations, gradients and parameters through memory. It may carry large embedding tables. It may use long context windows that increase activation memory. It may be constrained by the optimizer state rather than the matrix multiplications that dominate a benchmark chart.

TPU v5p has 95 GiB of HBM per chip, while Trillium has 32 GB. The unit labels in Google’s specifications differ—GiB for v5p and GB for v6e—so the precise ratio should not be overstated. Operationally, though, the gap is substantial: v5p offers roughly three times the per-chip memory capacity.

That memory headroom can materially reduce implementation complexity. A team can fit a larger local shard, use a less aggressive partitioning approach, or preserve a batch and sequence configuration that has already been validated for training stability. These are not minor conveniences. Each change to a parallelism strategy can introduce weeks of profiling, compiler tuning, reproducibility testing and retraining of internal support staff.

For an organization moving a model from a v5p environment to Trillium, the relevant engineering questions are concrete:

1. What is the peak memory footprint at the intended sequence length? Measure parameters, optimizer states, gradients, activations and temporary buffers—not only the parameter count shown in a model card.

2. How much model sharding will Trillium require? Extra sharding can make a workload feasible on smaller-memory chips, but it can also increase communication overhead and complicate failure recovery.

3. Is the workload compute-bound, memory-bandwidth-bound or communication-bound? A chip with twice the BF16 peak rate is most valuable when the workload can sustain that rate.

4. What does the checkpoint workflow look like? Large training jobs are operational systems. Checkpoint cadence, storage throughput and restart behavior affect both cost and schedule certainty.

5. Which configuration can the platform team actually support? A theoretically efficient parallelism scheme has weak ROI if only one specialist understands how to debug it at 2 a.m.

The implication is not that v5p is inherently easier to operate. Trillium may be a better fit for teams building many repeatable training jobs at varying scales. But a migration should begin with memory traces and communication profiles, not a spreadsheet based on TFLOPs alone.

Data movement remains the quiet constraint

The TPU v5p specification lists 2,765 GB/s of HBM bandwidth per chip and 1,200 GB/s of bidirectional Inter-Chip Interconnect, or ICI, bandwidth. Trillium lists 1,638 GB/s of HBM bandwidth and 800 GB/s of bidirectional ICI bandwidth per chip.

Those figures are easy to overlook because compute is easier to explain in executive reviews. Yet data movement frequently determines whether an expensive cluster behaves like a training engine or an expensive waiting room.

High HBM bandwidth supports workloads where chips need to feed arithmetic units rapidly from local memory. High interconnect bandwidth matters when the training strategy repeatedly exchanges tensors among chips. Transformer layers, mixture-of-experts routing, pipeline stages and sharded optimizer designs can all shift the bottleneck toward communication, depending on model design and configuration.

In practice, v5p’s larger memory and higher listed local bandwidth figures give it a credible argument for workloads that are both memory-intensive and communication-sensitive. This is especially relevant when a team has already tuned a very large model around a particular topology and needs predictable scaling behavior more than it needs the newest accelerator label.

Trillium’s architectural answer is different rather than inadequate. It places 256 chips into a 2D-torus pod and uses data-center networking to extend training beyond that boundary. For organizations that want to grow training capacity in more incremental blocks, that can align better with capacity planning and internal chargeback models.

The change management issue is that these approaches distribute responsibility differently. A large v5p deployment may concentrate the problem around a single, high-performance training fabric. A Trillium deployment at multi-pod scale makes the network, scheduling layer and cross-pod workload design more visible parts of the performance equation.

The cheapest chip-hour becomes expensive when a job spends it waiting on memory, networking or an avoidable reconfiguration.

Scaling claims are useful evidence, not a promise

Google reported 99% scaling efficiency for a 3,072-chip Trillium deployment across 12 pods, and 94% efficiency across 24 pods, or 6,144 chips, on GPT-3 175B pre-training. Those are strong results and demonstrate that cross-pod Trillium scaling can work at substantial scale.

They are also benchmark results under a stated workload and configuration, not a guarantee that every enterprise model will see the same outcome.

This distinction matters in corporate planning because the deployment question is rarely “Can this hardware scale?” The more useful question is “Can our model, our compiler settings, our data pipeline and our operating controls scale in the same way?”

A financial-services team training on proprietary documents, for example, may impose data residency rules that constrain region selection. A healthcare organization may need stricter access controls around training data and checkpoints. A product group building an internal coding assistant may have shorter iteration cycles and place greater value on quickly available mid-size slices than on maximum theoretical cluster scale.

Those constraints create silos if infrastructure selection happens separately from security, data governance and ML engineering. The accelerator team may choose the most attractive price-performance option, only for compliance to discover that the target zone is not acceptable or for platform engineering to find that a tested deployment template is missing.

A disciplined pilot should test more than tokens per second. It should capture:

• end-to-end elapsed time, including environment creation and data staging;
• achieved accelerator utilization over the full run rather than during a favorable benchmark window;
• checkpoint duration and restart reliability;
• behavior when a job crosses a pod boundary;
• engineering time spent changing sharding, batch size or compiler settings;
• the ability to reproduce a run after an update to the software environment.

These are operational metrics, but they belong in the ROI calculation. A faster training step that requires a bespoke workflow can be rational for a flagship model program. It can be a poor fit for a central platform serving dozens of teams with different skills and release schedules.

Pricing is meaningful, but chip-hours are not total cost

Google’s published U.S. on-demand pricing lists Trillium at $2.70 per chip-hour in us-east1 and us-east5, compared with $4.20 per chip-hour for TPU v5p in those regions. On its face, Trillium costs about 36% less per listed chip-hour while offering twice the stated BF16 peak compute.

That is a meaningful economic signal. It should encourage teams running compute-bound training to test Trillium first.

It should not be presented as a complete cost comparison.

A lower chip-hour rate does not establish a lower cost to a finished model. The final bill depends on the number of chips required, achieved utilization, time to convergence, storage, CPU and host resources, networking, reservation or committed-use terms, interruptions and the human cost of tuning. Regional pricing and consumption options can also change the calculation.

For a memory-constrained workload, for instance, using Trillium may require more aggressive partitioning or a different batch strategy. If that change slows convergence, raises communication overhead or consumes engineering capacity, the nominal discount can narrow quickly. Conversely, a compute-heavy workload that runs efficiently on v6e may obtain both a lower unit price and a shorter wall-clock training cycle.

A useful internal comparison does not ask, “Which TPU is cheaper?” It asks:

\[

\text{effective training cost} = \text{infrastructure spend} + \text{engineering time} + \text{schedule risk}

\]

The formula is simple; collecting honest inputs is harder. Finance teams tend to see the first term, while ML teams carry the second and product leaders feel the third. Bringing those three groups into the same capacity review prevents a familiar outcome: a low-cost infrastructure decision that delays the model launch.

Availability can decide the architecture before performance does

The comparison between Google AI hardware options is also constrained by where capacity is available. Google’s regional documentation has listed v6e in several U.S. zones, including locations in us-central1, us-east1, us-east5 and us-south1, while v5p has been listed in a narrower set including us-central1 and us-east5. Capacity and supported zones change, so deployment teams should confirm the live position at the time they schedule a program.

This is not administrative detail. Regional availability touches data location, network design, disaster recovery, latency to source data and internal compliance commitments. It also affects whether a proof of concept can become a production-grade training workflow without rework.

For an enterprise with strong data residency requirements, the right accelerator in an unavailable or unsuitable location is not a viable option. For a global AI platform team, the ability to standardize templates around available regions may be worth more than a marginal difference in chip-level specifications.

The practical path is to separate the decision into two gates. First, establish which TPU options meet data, security and capacity constraints. Then benchmark the remaining candidates against the actual model and operating workflow. This reduces the temptation to invest heavily in performance testing for an option that cannot clear compliance or procurement.

Choose the constraint you want to remove

TPU v5p remains compelling when the limiting factor is memory capacity per chip, high local memory bandwidth, high ICI bandwidth or the operational need for a very large, tightly integrated training topology. Its 95 GiB of HBM per chip is not a legacy footnote. For some large-model configurations, it is the decisive feature.

Trillium is the more compelling first test when the workload is compute-heavy, can operate effectively within its 32 GB memory configuration, and benefits from a lower listed U.S. on-demand chip-hour rate. Its 918 TFLOPs of BF16 peak compute per chip and documented multi-pod scaling results make it a serious training platform rather than a budget tier.

The right Google Cloud TPU comparison is therefore less about “TPU v6 versus TPU v5p” and more about matching infrastructure to organizational reality. Profile the model’s memory behavior. Benchmark the intended parallelism strategy. Include compliance, region availability and platform support in the same decision process as chip pricing.

For IT leaders, the actionable takeaway is modest but valuable: approve a representative pilot, not a hardware verdict. A two-week test that measures end-to-end training economics and employee workflow friction will usually tell you more than a quarter spent debating peak specifications.