In this article, we will discuss Bloom filters. An HBase Bloom Filter is an efficient mechanism to test whether a StoreFile contains a specific row or row-col cell.

Without Bloom filters, The only way to find a row key in a StoreFile is to check the store file’s block index. The StoreFile’s block index stores the start row key of each block in the StoreFile. Bloom Filters provides an in-memory structure to reduce disk reads to only the files likely to contain that row. In short, finding a row in a StoreFile can be considered an in-memory index.

If your application modifies all or the majority of the rows of Hbase regularly, the majority of StoreFiles will have a piece of the row you are searching for. Thus, Bloom filters may only help a little. In time-series data, when a few records are updated at a time or when updated in batches, each row is written in a separate Store file.

In this case, the Bloom filter helps a lot in improving the performance of HBase reads. It is done by discarding Store files that do not contain the row being searched.

After testing the above settings on test data of about 10 GB, we implemented the same in the streaming data HBase database. We observed a performance gain in line with the above experimental results.

Emergys Blog

Recent Articles

  • Maximizing Customer Engagement with Salesforce

    Maximizing Customer Engagement with Salesforce

    Maximizing Customer Engagement with Salesforce

    Forget about closing deals – in today's business world, customer [...]

    Forget about closing deals – in today's business world, customer engagement is all about building bridges, [...]

  • Bridging the Gap Between Humans and Machines with Generative AI

    Bridging the Gap Between Humans and Machines with Generative AI

    Bridging the Gap Between Humans and Machines with Generative AI

    Nowadays, customers expect quick and thorough help whenever they reach [...]

    Nowadays, customers expect quick and thorough help whenever they reach out, whether it’s to order something, [...]

  • Exploring Generative AI for Manufacturing Demand Forecasting

    Exploring Generative AI for Manufacturing Demand Forecasting

    Recent disruptions in global supply chains have shown that [...]

    Recent disruptions in global supply chains have shown that traditional ways of predicting customer demand [...]