Simple Hbase paging scheme

Keywords: Big Data HBase

Simple Hbase paging scheme

Most of the paging schemes on the Internet are divided into paging from the server or paging from the client. The paging mode of the server mainly uses the PageFilter filter. First, it is too complex. Secondly, the compatibility of the cluster is not very good. The author gives a simple and easy intermediate scheme by using the combination of paging from the server and paging from the client.

1. Use the PageFilter filter to page from the server to filter out the maximum number of pieces required,
Note: the author thinks that most users will not turn the page too deep. Assuming pageSize=5, a total of 500 pieces of data will be retrieved from the server for 100 pages of the customer's meal, which will not cause pressure on the Hbase cluster. And output to the client will not put too much pressure on the client

Filter pageSize = new PageFilter(pageSize *pageNo);

2. Client paging skills

            List<Result> resultList = new ArrayList<>();
            // Calculate start and end pages
            Integer firstPage = (pageNo) *pageSize;
            Integer endPage = firstPage + pageSize;

            //Client pages 
            int i = 0;

            for (Result rs : scanner) {
                if (!rs.isEmpty() && i >= firstPage && i < endPage) {
                    resultList.add(rs);
                }
                if (resultList.size() == log.getPageSize()) {
                    break;
                }
                i++;
            }

In this way, you can get the desired paging effect
Full code:

    public static void mai(String[] args){
            Connection connection = HBaseClientUtil.getConnection();
            table = connection.getTable(TableName.valueOf(ProcessLogUtil.HB_TB_NAME));
            Scan scan = new Scan();
            Filter pageSizeFilter = new PageFilter(pageSize *pageNo);
            scan.setFilter(pageSizeFilter );
            ResultScanner scanner = table.getScanner(scan);
            List<Result> resultList = new ArrayList<>();
            // Calculate start and end pages
            Integer firstPage = (pageNo) *pageSize;
            Integer endPage = firstPage + pageSize;

            //Client pages 
            int i = 0;

            for (Result rs : scanner) {
                if (!rs.isEmpty() && i >= firstPage && i < endPage) {
                    resultList.add(rs);
                }
                if (resultList.size() == log.getPageSize()) {
                    break;
                }
                i++;
            }
    }
           

Posted by lifeson2112 on Sat, 14 Dec 2019 12:29:49 -0800