hbase is a member of hadoop ecology. To build hbase first, you need to install hadoop, then zookeeper, and then hbase. Now hbase can be installed directly through docker, and hadoop is not required in the container.
The installation is simple. Directly pull the image and run it.
docker run -d --name hbase -p 2181:2181 -p 16010:16010 -p 16020:16020 -p 16030:16030 harisekhon/hbase
Note that hbase uses many ports for port mapping. In some places, it directly uses the - P parameter instead of using - P for port mapping one by one.
docker run -d --name hbase -P harisekhon/hbase
If the - P parameter is passed, all listening ports in the container will be mapped to random ports. We can see the container status:
[root@docker ~]# docker ps CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES 748f76bf11f3 harisekhon/hbase "/entrypoint.sh" 31 minutes ago Up 31 minutes 0.0.0.0:32776->2181/tcp, 0.0.0.0:32775->8080/tcp, 0.0.0.0:32774->8085/tcp, 0.0.0.0:32773->9090/tcp, 0.0.0.0:32772->9095/tcp, 0.0.0.0:32771->16000/tcp, 0.0.0.0:32770->16010/tcp, 0.0.0.0:32769->16201/tcp, 0.0.0.0:32768->16301/tcp hbase
Among them, port 16010 is the exposed port of the Web UI interface provided by hbase, which is mapped to 32770. We can access hbase Web UI through the virtual machine address and this port.
Here, you can see the hbase status through the web interface. Then, you can enter the container, run the hbase shell, create tables, add data, and scan data. However, if external programs, such as java, need to operate hbase, it is very troublesome.
Therefore, it is not recommended to start hbase directly with - P (uppercase).
Recommended startup method: docker run -d --name hbase -p 2181:2181 -p 16010:16010 -p 16020:16020 -p 16030:16030 harisekhon/hbase
Not recommended startup method: docker run -d --name -P harisekhon/hbase
Integration of spring boot and hbase
1. Although it is simple to build hbase environment through docker, there is a problem that its host mapping directly uses the container ID. therefore, if our external java program needs to connect to hbase, we need to add the container ID to the host list of our machine.
192.168.61.150 docker dd13ff2dca8e
2. Introduce hbase dependency library. Here, you only need to introduce additional hbase client dependency.
<dependency> <groupId>org.apache.hbase</groupId> <artifactId>hbase-client</artifactId> <version>2.4.3</version> </dependency>
3. Configuration file application.properties
hbase.config.hbase.zookeeper.quorum=192.168.61.150 hbase.config.hbase.zookeeper.property.clientPort=2181
4. The configuration file corresponds to the Java class, HbaseProperties.java
package com.xx.hbase.config;
import java.util.Map;
import org.springframework.boot.context.properties.ConfigurationProperties;
@ConfigurationProperties(prefix = "hbase")
public class HbaseProperties {
private Map<String,String> config;
public void setConfig(Map<String, String> config) {
this.config = config;
}
public Map<String, String> getConfig() {
return config;
}
}
5,HbaseConfig.java
package com.xx.hbase.config;
import java.io.IOException;
import java.util.Map;
import org.apache.hadoop.hbase.HBaseConfiguration;
import org.apache.hadoop.hbase.client.Connection;
import org.apache.hadoop.hbase.client.ConnectionFactory;
import org.apache.hadoop.hbase.client.HBaseAdmin;
import org.springframework.boot.context.properties.EnableConfigurationProperties;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
@Configuration
@EnableConfigurationProperties(HbaseProperties.class)
public class HbaseConfig {
private final HbaseProperties props;
public HbaseConfig(HbaseProperties props) {
this.props = props;
}
@Bean
public org.apache.hadoop.conf.Configuration configuration(){
org.apache.hadoop.conf.Configuration conf = HBaseConfiguration.create();
Map<String, String> config = props.getConfig();
config.forEach(conf::set);
return conf;
}
@Bean
public Connection getConnection() throws IOException{
return ConnectionFactory.createConnection(configuration());
}
@Bean
public HBaseAdmin hBaseAdmin() throws IOException {
return (HBaseAdmin) getConnection().getAdmin();
}
}
6,HbaseService.java
package com.xx.hbase.config;
import java.io.IOException;
import java.util.Map;
import org.apache.hadoop.hbase.Cell;
import org.apache.hadoop.hbase.TableName;
import org.apache.hadoop.hbase.client.ColumnFamilyDescriptor;
import org.apache.hadoop.hbase.client.ColumnFamilyDescriptorBuilder;
import org.apache.hadoop.hbase.client.Connection;
import org.apache.hadoop.hbase.client.HBaseAdmin;
import org.apache.hadoop.hbase.client.Put;
import org.apache.hadoop.hbase.client.Result;
import org.apache.hadoop.hbase.client.ResultScanner;
import org.apache.hadoop.hbase.client.Scan;
import org.apache.hadoop.hbase.client.Table;
import org.apache.hadoop.hbase.client.TableDescriptor;
import org.apache.hadoop.hbase.client.TableDescriptorBuilder;
import org.apache.hadoop.hbase.util.Bytes;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.stereotype.Service;
@Service
public class HbaseService {
@Autowired
private HBaseAdmin admin;
@Autowired
private Connection connection;
public void createTable(String name,String colFamily) throws IOException {
TableName table = TableName.valueOf(name);
if(admin.tableExists(table)) {
System.out.println("table ["+name+"] exist.");
}
ColumnFamilyDescriptor cfd = ColumnFamilyDescriptorBuilder.newBuilder(Bytes.toBytes(colFamily))
.setMaxVersions(1).build();
TableDescriptor tableDes = TableDescriptorBuilder.newBuilder(table).setColumnFamily(cfd).build();
admin.createTable(tableDes);
}
public void putData(String name,String colFamily,String rowKey,Map<String, String> data) throws IOException {
TableName table = TableName.valueOf(name);
if(admin.tableExists(table)) {
Table t = connection.getTable(table);
Put put = new Put(Bytes.toBytes(rowKey));
for(Map.Entry<String, String> entry:data.entrySet()) {
put.addColumn(Bytes.toBytes(colFamily), Bytes.toBytes(entry.getKey()), Bytes.toBytes(entry.getValue()));
}
t.put(put);
}else {
System.out.println("table ["+name+"] does not exist.");
}
}
public void getData(String name) throws IOException{
TableName table = TableName.valueOf(name);
Table t = connection.getTable(table);
ResultScanner rs = t.getScanner(new Scan());
for(Result r:rs) {
System.out.println("row:"+new String(r.getRow()));
for(Cell cell:r.rawCells()) {
System.out.println("colFamily:"+Bytes.toString(cell.getFamilyArray(),cell.getFamilyOffset(),cell.getFamilyLength())+""
+",qualifier:"+Bytes.toString(cell.getQualifierArray(),cell.getQualifierOffset(),cell.getQualifierLength())+
",value:"+Bytes.toString(cell.getValueArray(),cell.getValueOffset(),cell.getValueLength()));
}
}
}
}
7. springboot startup class
package com.xx.hbase;
import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;
@SpringBootApplication
public class HbaseApp {
public static void main(String[] args) {
SpringApplication.run(HbaseApp.class, args);
}
}
8. hbase unit test class
package com.xx.mybatis;
import java.io.IOException;
import java.util.Map;
import org.apache.hbase.thirdparty.org.apache.commons.collections4.map.HashedMap;
import org.junit.Test;
import org.junit.runner.RunWith;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.boot.test.context.SpringBootTest;
import org.springframework.test.context.junit4.SpringJUnit4ClassRunner;
import com.xx.hbase.HbaseApp;
import com.xx.hbase.config.HbaseService;
@SpringBootTest(classes = HbaseApp.class)
@RunWith(SpringJUnit4ClassRunner.class)
public class HbaseTest {
@Autowired
private HbaseService hbaseService;
@Test
public void createTable() {
try {
hbaseService.createTable("stu", "info");
} catch (IOException e) {
e.printStackTrace();
}
}
@Test
public void putData() {
String name = "stu";
String colFamily = "info";
String rowKey = "2";
Map<String, String> data = new HashedMap<>();
data.put("name", "bbb");
data.put("email", "bb@126.com");
try {
hbaseService.putData(name, colFamily, rowKey, data);
} catch (IOException e) {
e.printStackTrace();
}
}
@Test
public void getData() {
try {
hbaseService.getData("stu");
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
}
The unit test class here does not use assertions, but only calls the three methods of creating tables, inserting data and querying data, which proves that there is no problem for java to call the hbase interface to operate hbase.
9. Operate hbase in docker container
Summary:
1. Start the container and do not use - P random port mapping, which will cause great trouble for subsequent java external programs to access hbase.
2. The hbase started by docker uses the container ID as the host. Therefore, if an external program wants to access hbase, it is necessary to add the host mapping corresponding to the container ID to the local hosts.