Query Understanding With Large Language Models: Techniques and...

Query Understanding With Large Language Models: Techniques and Optimizations in OpenSearch - Hajer Bouafif & Cédric Pelvet, Amazon Web Services This session presents a range of techniques that integrate Large Language Models (LLMs) with OpenSearch to enhance query understanding in search systems. We will examine the use of LLMs for query expansion, highlighting their ability to generate semantically enriched tokens that bridge lexical gaps between user queries and document content. Additionally, we explore query rewriting strategies that leverage LLMs to infer and apply relevant metadata filters, improving retrieval precision. While LLMs offer substantial improvements in relevance, they also introduce challenges related to inference cost and query latency. To address these, we introduce semantic caching using OpenSearch, an effective approach to mitigate the performance overhead of LLMs in production-grade search pipelines. The session includes a live demonstration of these methods in the context of an e-commerce use case using OpenSearch.