Using OpenSearch as a Database With DataFusion
Using OpenSearch as a Database With DataFusion - Marc Handalian, Amazon Web Services & Michael Froh, Uber OpenSearch supports fast full-text, numeric range, and vector queries, but how does it compare to a relational database? In this talk, we explore integrating DataFusion with OpenSearch to add rich SQL capabilities like planning, joins, and aggregations. We’ll walk through the process of querying OpenSearch with DataFusion, starting with a naive table scan, moving on to projection push-down, and then predicate push-down to leverage OpenSearch’s powerful query capabilities. We’ll use DataFusion to join multiple OpenSearch indices with other data sources.
