Apple announced a new method for Maximum Inner Product Search (MIPS) in retrieval-augmented generation (RAG) systems. The method uses small neural networks to route queries faster and more accurately, trading off upfront amortization costs for better performance at inference time.
What happened
Apple's Machine Learning Research team introduced a new method for Maximum Inner Product Search (MIPS) in retrieval-augmented generation (RAG) systems. The method, detailed in the paper 'Amortizing Maximum Inner Product Search with Learned Support Functions' presented at ICML 2026, uses small neural networks (NNs) to route queries faster and more accurately. These NNs are trained offline for a fixed database of keys, learning to route queries toward their best match. The approach aims to improve retrieval efficiency by amortizing the cost of repeatedly solving MIPS for queries drawn from a known distribution over a fixed key database.
What to weigh
- The method requires upfront training of neural networks.
- The method is designed for specific query distributions.
Source: Apple
