Introducing OpenSearch’s Search Relevance Workbench! - Eric Pugh, OpenSource Connections & Stavros Macrakis, Amazon Web Services This talk introduces you to OpenSearch’s integrated new capability to measure and enhance search quality: Search Relevance Workbench (SRW). We’ll introduce the mental model for improving search quality espoused by OpenSource Connections. We’ll show you how SRW in OpenSearch 3 implements this approach. We’ll teach you the three phases of improving search using SRW. Understand Search Quality: * Define explicit Search Configurations. This is a new concept in OpenSearch for structuring query plans * Run evaluation metrics like NDCG. * Scale up with the “LLM as a Judge”. Improve Search Quality: * SRW makes it easy to evaluate changes to your query templates. We’ll show you how to make new search configurations in OpenSearch and quickly test them. * We’ll introduce the new Hybrid Optimizer capability in SRW that goes beyond manual experimentation by introducing automated experimentation. Deploy new Search Configurations: * We’ll demonstrate how to A/B test your search configurations using the User Behavior Insights event tracking system. At the end of this talk, you’ll leave understanding how to take use these new capabilities in OpenSearch 3 to enhance your own Search.