Google TPU 8tx and 8i Max Explained

Read Time:5 Minute, 38 Second

Google introduced two AI accelerator systems on November 9. TPU 8tx is designed to protect sensitive data while AI models process it. TPU 8i Max combines two TPU 8i chips in one package for work that needs more memory bandwidth, such as an AI assistant reading across a large codebase. The two systems bring Google’s eighth-generation TPU lineup to four. The announcement gives no launch date or pricing for India.

The two designs reuse existing Google silicon. That makes this a change in how Google puts the hardware together, including the processors, networking and memory around the chips. Google Cloud’s announcement lays out what each configuration can do. Reuters’ coverage puts the announcement in the context of Google’s effort to win more cloud customers for its own AI hardware.

TPU 8tx protects data while a model uses it

When a company encrypts its data at rest, it protects the copy on disk. Encryption in transit protects it as it moves across a network. When the data is being processed, it can still be exposed inside the computer unless the system protects that stage too. Google says TPU 8tx is designed to keep sensitive data and model weights isolated while they are in use.

Google built the system around TPU 8t silicon and paired it with fifth-generation AMD EPYC host processors. The 8tx also uses the fast network Google developed for the TPU 8i. Google reports up to twofold lower latency compared with TPU 8t. That comparison comes from Google and refers to the systems it named. How much a particular application gains will depend on how it uses the hardware.

Who would use confidential inference?

Inference is the work a model does when it responds to a request. Imagine a bank using a model to pull together details from several internal records for an employee. The bank has to protect those records while the model is using them as well as when they are stored. The same issue comes up when a government agency runs a model on a private collection of documents.

Google also describes 8tx as suitable for smaller training jobs, including fine-tuning a model with private data. If teams can run and adapt a model on the same system, they can avoid moving certain datasets between separate clusters. Teams would still have to review who can send data to the model, where the data and logs are stored, and how the workload fits their security requirements. Hardware protections cover one part of that review.

TPU 8i Max puts two chips in one package

Google built 8i Max for applications that keep a lot of information in play at once. When an assistant reads a repository, for instance, it may need to refer to code from several files while it generates and checks a change. The model has to pull data from memory repeatedly. If the memory cannot feed the processors fast enough, adding processing capacity alone can leave part of the system waiting.

Google says 8i Max doubles compute capacity, on-chip SRAM and high-bandwidth memory bandwidth compared with TPU 8i. Memory bandwidth is how much data can move in a given time. More bandwidth can help keep processors supplied during demanding workloads. The figures describe hardware capacity. They leave open how much faster a specific tool will respond, how much a workload will cost and how many simultaneous sessions an application can sustain.

Why the context window matters

A context window is how much information a model can consider in a request or session. The context can include instructions, code, a document or material from earlier in the conversation. A coding tool that opens one short file has a different memory workload from a tool that reads across a repository while keeping track of earlier steps. Google points to workloads with very long context windows as a reason for the 8i Max design.

The application team still decides what information to include and when to retrieve it. Even if a model can read more in a single session, engineers may want to measure the quality of its output alongside response time and cost. For background on how repeated steps fit into AI workflows, see BoringDude’s guide to loop engineering.

How the two systems fit into Google’s cloud lineup

Google introduced TPU 8t and 8i in April. The former was designed for training large models, and the latter for running models that need to reason with low delay. With 8tx and 8i Max, cloud buyers have two more configurations to evaluate. A team weighing the four can start with its main constraint, such as a need to protect data during processing or a lack of memory bandwidth for large sessions.

Google is also working to make the hardware easier to use from common software. It says its open-source TorchTPU project supports PyTorch and Hugging Face models, and that TPU backends are available for serving tools such as vLLM and SGLang. The November announcement separately identifies its TPU agent suite as a preview. The status of an onboarding tool or a cloud management feature tells buyers little about when they can reserve the newly announced systems.

What Indian teams can verify next

Google’s announcement describes the systems and the workloads they target. It leaves unanswered when customers can reserve them, which regions will get them and what a deployment will cost. A developer reading the announcement in India should distinguish between access from India and having the workload run in a cloud region located in India. Those can be different arrangements, especially where a company’s rules restrict where sensitive data can be processed.

The next concrete evidence will be product and regional availability pages, pricing and tests on applications that resemble what teams run in production. A useful comparison would keep the model and workload fixed, then measure response time, throughput and cost across configurations. For teams considering 8tx, the security review matters alongside those figures. For teams considering 8i Max, the practical question is whether the added bandwidth helps their own longest or busiest sessions.

Frequently asked questions

What is the difference between TPU 8tx and TPU 8i Max?

Google designed 8tx for confidential AI processing and 8i Max for workloads that need more memory bandwidth, including long-context AI sessions.

How many chips does TPU 8i Max use?

It combines two TPU 8i chips in a single package.

When will the new TPUs be available in India?

Google gave no India release date or Indian cloud region availability in its November 9 announcement.

Is the 8i Max twice as fast as the 8i?

Google reports double the compute capacity, on-chip SRAM and high-bandwidth memory bandwidth. It gives no universal twofold speed guarantee for applications.