← All news
Global · AI & Technology
Global·AI & Technology

Exa reports roughly 50% higher embedding throughput on AWS AI chips

Exa says optimizations to its AWS-based embedding infrastructure increased throughput by roughly 50% over its initial implementation. The work followed a decision to re-embed 30 billion priority documents for a new document model.

AI Hubs United Editorial ·

Exa reports that it increased document embedding throughput by roughly 50% over its initial implementation through optimizations on AWS AI hardware, according to its account on the AWS Machine Learning Blog. The search company describes infrastructure using Inferentia2 and Trainium2 chips to process data for its semantic index, which supports natural-language queries alongside keyword search.

The work began when Exa selected 30 billion high-priority documents to re-embed after adopting a new document model. Running the existing and replacement embedding pipelines in parallel temporarily doubled the embedding workload, prompting a review of hardware and performance. Exa used the Neuron SDK with support from the AWS Trainium team, adjusting model weight layouts to remove unnecessary transposes and enabling lower-precision matrix operations. It reports that the lower-precision outputs passed a correctness threshold of 0.999 cosine similarity against a reference.

For Exa, embedding capacity affects both processing economics and how quickly newly discovered documents become searchable. Every document accepted for the index must pass through embedding before serving, making higher throughput practically important during both ongoing ingestion and large model migrations.

AI Hubs United original short summary of the source linked above. It is not the original article; read the full story at the source.