Digging Into OpenSearch’s Memory: Lucene and JVM Garbage Collection Under the...

Digging Into OpenSearch’s Memory: Lucene and JVM Garbage Collection Under the Microscope - Yeonghyeon Ko, SK hynix OpenSearch indexing creates short-lived Java objects from Lucene analyzers and field types, which impact heap behavior, GC frequency, and node performance. This session explores how the JVM memory model and garbage collector handle Lucene-generated objects during indexing, focusing on log-heavy OpenSearch clusters. Let’s explore the lifecycle of key objects as they move from Eden to Survivor and the Old Generation. By reviewing GC logs and heap, you’ll learn to spot inefficiencies, like high promotion rates and excessive heap waste, that degrade performance and cause GC pauses. We’ll also cover JVM tuning strategies, including the UseStringDeduplication, G1UseAdaptiveIHOP, and G1NewSizePercent/G1MaxNewSizePercent flags, which help optimizing our cluster. In addition to JVM tuning, it’s beneficial to examine the future roadmap of OpenSearch—separation of search/indexing workloads and Generational ZGC. Whether scaling OpenSearch or optimizing in resource-constrained environments, this session helps you control memory usage and implement heap optimization best practices.