Hadoop 2.9.1 Install Hive 2.3.3 on Ubuntu 16.04

Preface

http://hive.apache.org/downloads.html There are instructions. hadoop3.x Version needs hive3.0.0，and hadoop2.x Need hive2.3.3. Because of mine. hadoop It's 2..9，So choose to download hive2.3.3. Hive yes hadoop Tools, so you only need to install them NameNode Up, no need to install DataNode Up.

Move Hadoop Installation

http://blog.51cto.com/ljohn/2307655

Install Hive 2.3.3

1. Download Hive

# Download address:
wget http://mirrors.shu.edu.cn/apache/hive/stable-2/apache-hive-2.3.3-bin.tar.gz

# Decompress Hive 2.3.3
tar xvf apache-hive-2.3.3-bin.tar.gz -C /usr/local/
chown -R hadoop.hadoop  /usr/local/apache-hive-2.3.3-bin/

2. Configuring Hive 2.3.3 environment variables

cat << EOF >> ~/.bashrc
HIVE_HOME=/usr/local/apache-hive-2.3.3-bin
PATH=$PATH:$JAVA_HOME/bin:$JRE_HOME/bin:$HADOOP_HOME/bin:$HIVE_HOME/bin:$HIVE_HOME/conf
export JAVA_HOME JRE_HOME CLASS_PATH PATH HADOOP_HOME HIVE_HOME
EOF
source /etc/profile

3. Configure hive-env.sh

cat < EOF > /usr/local/apache-hive-2.3.3-bin/hive-env.sh

export JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
export HADOOP_HOME=/usr/local/hadoop
export HIVE_HOME=/usr/local/apache-hive-2.3.3-bin

# Hive Configuration Directory can be controlled by:
export HIVE_CONF_DIR=$HIVE_HOME/conf

# Folder containing extra libraries required for hive compilation/execution can be controlled by:
export HIVE_AUX_JARS_PATH=$HIVE_HOME/lib/*
EOF 

4. Configure hive-site.xml

There are three Hive environment patterns, and in this case the embedded pattern is used
cp hive-default.xml.template hive-site.xml

hive-site.xml file content does not need to be modified, the main configuration

<property>
    <name>hive.metastore.warehouse.dir</name>
    <value>/user/hive/warehouse</value>
    <description>location of default database for the warehouse</description>
</property>
..
<property>
    <name>hive.exec.scratchdir</name>
    <value>/tmp/hive</value>
    <description>HDFS root scratch dir for Hive jobs which gets created with write all (733) permission. For each connecting user, an HDFS scratch dir: ${hive.exec.scratchdir}/&lt;username&gt; is created, with ${hive.scratch.dir.permission}.</description>
</property>

Note:

• hive.metastore.warehouse.dir

  This parameter specifies the Hive data storage directory, which is located by default in the / user/hive/warehouse path above HDFS.

• hive.exec.scratchdir

  This parameter specifies Hive's temporary data file directory, which defaults to the / tmp/hive path above HDFS.

5. Create HDFS directories

There are two important HDFS paths in the hive-site.xml file: user/hive/warehouse and/tmp/hive, which need to be created in HDFS and permissions modified.

hadoop fs -mkdir /user
hadoop fs -mkdir /user/hive
hadoop fs -mkdir /user/hive/warehouse
hadoop fs -mkdir /tmp/hive
hadoop fs -chmod  777 /user/hive/warehouse
hadoop fs -chmod  777 /tmp/hive

6. Initialization of database

The embedded schema uses the local database derby

./schematool -initSchema -dbType derby 
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/usr/local/apache-hive-2.3.3-bin/lib/log4j-slf4j-impl-2.6.2.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/usr/local/hadoop/share/hadoop/common/lib/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
Metastore connection URL:        jdbc:derby:;databaseName=metastore_db;create=true
Metastore Connection Driver :    org.apache.derby.jdbc.EmbeddedDriver
Metastore connection User:       APP
Starting metastore schema initialization to 2.3.0
Initialization script hive-schema-2.3.0.derby.sql
Initialization script completed
schemaTool completed

7. Start hive

Enter the $HIVE_HOME/bin directory

cd $HIVE_HOME/bin
./hive

# test
hive> show tables;
OK
Time taken: 7.101 seconds

FAQ

Q1: Start Hive error reporting

Logging initialized using configuration in jar:file:/usr/local/apache-hive-2.3.3-bin/lib/hive-common-2.3.3.jar!/hive-log4j2.properties Async: true
Exception in thread "main" java.lang.IllegalArgumentException: java.net.URISyntaxException: Relative path in absolute URI: ${system:java.io.tmpdir%7D/$%7Bsystem:user.name%7D
        at org.apache.hadoop.fs.Path.initialize(Path.java:254)
        at org.apache.hadoop.fs.Path.<init>(Path.java:212)
        at org.apache.hadoop.hive.ql.session.SessionState.createSessionDirs(SessionState.java:659)
        at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:582)
        at org.apache.hadoop.hive.ql.session.SessionState.beginStart(SessionState.java:549)
        at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:750)
        at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:686)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at org.apache.hadoop.util.RunJar.run(RunJar.java:239)
        at org.apache.hadoop.util.RunJar.main(RunJar.java:153)
Caused by: java.net.URISyntaxException: Relative path in absolute URI: ${system:java.io.tmpdir%7D/$%7Bsystem:user.name%7D
        at java.net.URI.checkPath(URI.java:1823)
        at java.net.URI.<init>(URI.java:745)
        at org.apache.hadoop.fs.Path.initialize(Path.java:251)
        ... 12 more

A1:

Add the following configuration at the front of the hive-site.xml file to restart hive

Create a cache directory: mkdir-p/usr/local/apache-hive-2.3.3-bin/tmpdir

<property>
     <name>system:java.io.tmpdir</name>
     <value>/usr/local/apache-hive-2.3.3-bin/tmpdir</value>
  </property>
  <property>
     <name>system:user.name</name>
     <value>hive</value>
</property>