Deep Dive Into OpenSearch Pull-Based Ingestion at Uber

Deep Dive Into OpenSearch Pull-Based Ingestion at Uber - Varun Bharadwaj, Uber & Yupeng Fu, Uber Inc In this session, Uber, a premier member of the OpenSearch Foundation, will present an in-depth overview of the architecture of the pull-based ingestion feature contributed to OpenSearch. Unlike traditional push-based approaches, pull-based ingestion enables OpenSearch to actively fetch data from streaming sources such as Apache Kafka and Amazon Kinesis. This provides fine-grained control over data flow, allowing OpenSearch to natively handle backpressure and pacing, resulting in more stable, scalable, and resource-efficient ingestion pipelines. Join this session to explore the design and capabilities of pull-based ingestion, and discover how it powers large-scale search infrastructure within Uber.