1. Purpose of setting up

As an online service, it needs to be able to guarantee the characteristics of fast failure, fault-tolerant retry and so on. Fast failures can ensure low latency of the system and prevent the temporary use of service resources due to waiting for a resource, which eventually leads to the unavailability of services. Failure tolerance can provide stability of service, and service failure is retry. therefore Hbase The retry mechanism provided by the client can ensure the fault tolerance of the client and the low latency of the system by configuring reasonable parameters.

2. Important parameters of HBase client

• hbase.client.pause 
  The waiting time of failed retries is longer with the number of retries. The calculation method is as follows:

  public static int RETRY_BACKOFF[] = { 1, 2, 3, 5, 10, 20, 40, 100, 100, 100, 100, 200, 200 }; 
  long normalPause = pause * HConstants.RETRY_BACKOFF[ntries];
  long jitter = (long)(normalPause * RANDOM.nextFloat() * 0.01f); 

  • 1
  • 2
  • 3
  • 1
  • 2
  • 3

  So if you retry 10 times, hbase.client.pause=50ms, the waiting time for each retry is {50, 100, 150, 250, 500, 1000, 2000, 5000, 5000, 5000}.  
  The default value of the property is 100ms, which can be set to 50ms or even smaller.

• hbase.client.retries.number 
  The default number of retries is 31. The value can be adjusted relatively small according to the requirements of its own application. For example, if the time-out time of the whole application is 3 seconds, the number of retries can be adjusted to 3 times according to the above calculation method of retry time.

• hbase.rpc.timeout 
  This parameter represents the timeout time of an RPC request. If the RPC time exceeds this value at a time, the client will actively close the socket.  
  The default value is 1 minute, which can be set according to the application's timeout time when the application is online service. If the application's total timeout is 3 seconds, the value should be 3 seconds or less.

• hbase.client.operation.timeout 
  This parameter represents the total time-out between the initiation of a data operation by the HBase client and the response. The data operation types include get, append, increment, delete, put, etc. The difference between this value and hbase.rpc.timeout is that hbase.rpc.timeout is the timeout time of an RPC call. hbase.client.operation.timeout is the total time of an operation (from the start of the call to the total time of failure after n retries).  
  For example, for a Put request, the client first encapsulates the request as a caller object, which sends RPC requests to the server. If a serious Full GC happens on the server side at this time, the RPC time-out causes SocketTimeoutException, corresponding to hbase.rpc.timeout. If the caller object sends an RPC request and then throws a network exception, the HBase client will retry. If the total operation time exceeds several times and causes the SocketTimeoutException, the corresponding is hbase.client.operation.timeout.

• hbase.client.scanner.timeout.period 
  This parameter represents the total time-out between the rpc call of a scan operation initiated by the hbase client and the response. A scan operation is an operation that initiates a region server rpc call. hbase divides the scan operation into multiple rpc operations according to the cacheing and batch settings of the scan query conditions. For example, if the number of rowkey s satisfying the scan condition is 10,000 and the cacheing of the scan query is 200, then the number of rpc calls required to execute for all the results of the query is 50. This value refers to the maximum of 50 rpc calls for a single corresponding time.

3. Examples of HBase client usage

/**
 * hbase client Singleton connection
 * @author chun
 *
 */
public class HbaseClient {
    private static final Logger LOGGER = LoggerFactory.getLogger(HbaseClient.class);
    private Configuration conf = null;
    private Connection conn = null;
    private static HbaseClient instance = null;

    private HbaseClient(){
        init();
    }
    public void init(){
        try{
            conf =  HBaseConfiguration.create();  
            conf.set("hbase.zookeeper.quorum", "127.0.0.1");  
            conf.set("zookeeper.znode.parent", "/hbase");
            conf.set("hbase.zookeeper.property.clientPort", "2181"); 
            conf.set("hbase.client.pause", "50"); 
            conf.set("hbase.client.retries.number", "3"); 
            conf.set("hbase.rpc.timeout", "2000"); 
            conf.set("hbase.client.operation.timeout", "3000"); 
            conf.set("hbase.client.scanner.timeout.period", "10000"); 

            conn = ConnectionFactory.createConnection(conf);
        }catch(Exception e){
            LOGGER.error("Initialization hbase connection failed"+e.getMessage(),e);
        }
    }

    public static HbaseClient getInstance(){
        if(instance == null){
            synchronized (HbaseClient.class) {
                if(instance == null){
                    instance = new HbaseClient();
                }
            }
        }
        return instance;
    }

    /**
     * Get the htable action class
     * @param tableName
     * @return
     * @throws IOException
     */
    public Table getHtable(String tableName) throws IOException{
        return conn.getTable(TableName.valueOf(tableName));
    }

    /**
     * 
     * @param hTableInterface
     */
    public void relaseHtable(Table table){
        if(table == null){
            return;
        }
        try {
            table.close();
        } catch (IOException e) {
            LOGGER.error(e.getMessage(),e);
        }
    }

    /**
     * Close the hbase connection
     */
    public void destory(){
        try {
            conn.close();
            instance = null;
        } catch (IOException e) {
            LOGGER.error(e.getMessage(),e);
        }
    }

}

References are as follows:
HBase Best Practice - Client Retry Mechanism 
HBase Best Practice-Client Timeout Mechanism