Meta Andromeda: Powering the Next Generation of Ads Retrieval Through Model-System Co-Design

This post describes the next generation of Meta Andromeda, Meta’s personalized ads retrieval engine, and how co-designing the model, system, and GPU hardware unlocks a new efficiency frontier for ads retrieval. In production, the system scores every eligible ad with near ranking-level model performance at retrieval scale and surfaces the top candidates at over one billion ads per second, a new throughput benchmark for ads retrieval-stage models (i.e., a 100x lift compared to the 2024 Meta Andromeda system). For advertisers, this means better performance automatically from the retrieval system matching ads across a larger candidate pool, within the same latency budget. Since 2025, Meta Andromeda has delivered a 2.4% increase in conversions on Facebook. Meta Andromeda is Meta’s machine learning system for personalized ads retrieval, the first stage of the ads recommendation pipeline. Retrieval narrows a vast pool of eligible ads down to a small set of strong candidates in milliseconds; a later, more computationally expensive ranking stage then scores those survivors in fine detail. Historically, the deep modeling quality of the ranking stage has been too costly to apply at retrieval’s scale. Building on the retrieval foundation established with Andromeda’s 2024 launch on the NVIDIA GH200 Grace Hopper Superchip, this post shares how co-designing the model, system, and hardware together brings near ranking-level performance to the retrieval scale. Challenges and Opportunities in Scaling Retrieval The 2024 Andromeda deployment reimplemented retrieval with an end-to-end optimized retrieval implementation from custom AI hardware with hierarchical indexing and GPU-native operators to a purpose-built ML model, growing model capacity by four orders of magnitude. Since then, generative AI creative tools have helped advertisers rapidly create hundreds of millions of new ads, growing Meta’s ads candidate pool to an unprecedented scale. Scoring such a large ad candidate pool with a more accurate model at a much higher system throughput requires fundamental technology breakthroughs in model quality, system architecture, and hardware utilization jointly. Higher Model Performance Requires a New Architecture Ads retrieval has an outsized ROI on incremental quality: Small improvements in retrieval model performance can drive large gains in advertiser value. The rapid expansion of our ads candidate pool makes optimizing for hardware capacity ROI an increasingly critical factor. The 2024 Meta Andromeda retrieval models have established a highly effective foundation with their GPU-optimized deep neural network architecture. To keep improving retrieval performance at a sustainable compute cost at our current scale, however, we must move beyond that paradigm. This requires a fundamental architectural evolution to push model performance boundaries significantly without a proportional increase in costs. New System Bottlenecks at Billion Scale The 2024 Meta Andromeda system evaluates at tens of millions of ads per second per GPU card. Meeting the new business demand requires a throughput at multi-billion ads evaluated per second per card in the system. At billions of ads per second throughput, the memory bandwidth of passing ad tensors through memory hierarchy becomes the new bottleneck. Intermediate tensors between GPU kernels account for terabytes per second of memory traffic, and shipping ads between local CPU hosts and remote hardware accelerators evaluating models saturates the network link. This requires a new system co-design that unifies these fragmented, distributed workloads into a single GPU pipeline. Pushing Towards the Hardware Limits With retrieval running on GPU, the important optimization question is “How close are we to the hardware ceiling, and how much room remains for future model growth?” Answering this requires pushing each component of the retrieval pipeline to the limits of the current hardware generation, a challenge that motivated a joint collaboration with NVIDIA. Each of these challenges drove a corresponding advance. Unlocking Retrieval Performance Through Model-System Co-Design Next-Gen Retrieval Model Architecture — Retrieval Mixture-of-Experts The latest iteration of Meta Andromeda introduces three synergistic architectural innovations that bring large language model (LLM)-inspired mixture-of-experts (MoE) into the retrieval (see Figure 1). With this highly expressive yet inference-efficient architecture, a new retrieval performance-efficiency Pareto frontier has been established: model performance approaches ranking-level in a typical retrieval-then-ranking system, at approximately 300x fewer inference-time FLOPs than the 2024 Meta Andromeda retrieval model. On the user modeling side , we transitioned from a single user tower to an ensemble of specialized user towers as experts. This captures diverse, time-evolving, and contextualized user interests by integrating a vast array of high-fidelity signals: sequential ad engagement events, user-cohort representations, LLM-derived semantic features, and cross-surface co-engagement signals. In the candidate ads tower , we employ a joint encoding strategy that incorporates real-time serving context, ensuring that downstream interaction models receive context-aware representations. In the interaction overarch layer , the interaction model evolved from dense neural networks into an MoE attention-based gating network that dynamically routes candidates across the most relevant user experts, delivering near-ranking-level performance at a fraction of the FLOPs. A defining property of the new architecture is that its performance is co-designed with the underlying hardware execution model. Model inference throughput has reached over 10 billion ads per second on a single GH200 GPU, and this efficiency allows model capacity to grow further, through richer feature interactions, additional expert sub-networks, and deeper architectures, without a proportional increase in GPU cost. Each incremental model advancement translates into greater advertiser value. Co-Designing the Model and System Onto a Single GPU Pipeline The 2026 Meta Andromeda pushes the model-system co-design to the next level with two innovations — a single GPU kernel fusing both model and system workloads per user-ad interaction and a strategy to co-locate per user request logics with per user-ad GPU kernels in a single GPU box. To this end, a highly optimized and fully fused GPU pipeline is built to host the entire end-to-end Meta Andromeda workflow. Fused Kernel At a high level, Meta Andromeda’s inference time per user-ad workload is a compact sequence of operations across model execution and system logics — gathering embeddings, executing user-ad interaction models, and scoring. The ~300x FLOPs reduction in the retrieval MoE gating architecture creates the necessary headroom and is less resistant to fully fuse both model execution and non-model system workloads into a single GPU kernel, compared to the 2024 launch where only operations within the model execution were fused. Consequently, the fused kernel now runs at over 10 billion ads per second on a single GH200. This matters because retrieval workloads are fundamentally memory-bandwidth bound. At billions of ads per second, even a small intermediate tensor per ad multiplies into terabytes per second of high bandwidth memory (HBM) traffic. The memory workload of these intermediate tensors, not arithmetic, becomes the dominant bottleneck in multi-kernel retrieval pipelines. The fused kernel eliminates this bottleneck entirely. Rather than dispatching multiple small GPU kernels for model and system operations, it executes the entire per user-ad computation in one pass. Intermediate results stay in GPU registers and thread block shared memory and are never materialized to HBM, achieving high utilization of the GPU’s peak memory bandwidth rather than wasting it on redundant intermediate traffic.