Large Data Hive Installation Configuration

Keywords: Big Data hive MySQL Database xml

Installation of Hive libraries for large data components:
1. First, download it from the official website of hive.
https://hive.apache.org/downloads.html

Note that when choosing the hive version, you need to pay attention to the hadoop version. The hive must be based on hadoop before it can be started.

The following error is reported in the installation process. It will be ready after modification in the installation document.

Copyright Statement: This article is the original article of the blogger, follow CC 4.0 BY-SA Copyright Agreement, reproduced please attach the link of origin and this statement.
Links to this article: https://blog.csdn.net/walykyy/article/details/82971562
Relative path in absolute URI: ${system:java.io.tmpdir%7D/$%7Bsystem:user.name%7D

modify hive-site.xml file

<property>
  <name>hive.exec.local.scratchdir</name>
    <value>$HIVE_HOME/iotmp</value>
    <description>Local scratch space for Hive jobs</description>
</property>

<property>
   <name>hive.querylog.location</name>
    <value>$HIVE_HOME/iotmp</value>
    <description>Location of Hive run time structured log file</description>
 </property>

 <property>
   <name>hive.downloaded.resources.dir</name>
    <value>$HIVE_HOME/iotmp</value>
    <description>Temporary local directory for added resources in the remote file system.</description>
 </property>
--------
Copyright Statement: CSDN Blogger「Fight against yourself」Original articles, follow CC 4.0 BY-SA Copyright Agreement, reproduced please attach a link to the original source and this statement.
Links to the original text: https://blog.csdn.net/walykyy/article/details/82971562

Errors were reported when the execution was re-executed:

schematool -dbType mysql -initSchema

Schema initialization FAILED! Metastore state would be inconsistent !!

 

After that, the solution is as follows (I put the configuration at the bottom of the solution):

Copyright Statement: This article is the original article of the blogger. It follows the CC 4.0 BY-SA Copyright Agreement. Please attach the link of origin and this statement to reproduce it.
Links to this article: https://blog.csdn.net/qq_22650745/article/details/82853039
Schematool-dbType mysql-initSchema time error. Online tutorials are all about adding MySQL connection configuration to the file header, but hive-site. xml. template was originally derby configuration, which will be overwritten by the derby configuration below, resulting in initialization failure. The way to do this is to put the MySQL configuration at the bottom, or to delete the derby configuration
--------
Copyright Statement: This is the original article of "Ja de King Puppy" by CSDN blogger. It follows CC 4.0 BY-SA Copyright Agreement. Please attach the link of origin and this statement for reproducing.
Links to the original text: https://blog.csdn.net/qq_22650745/article/details/82853039

But when hive enters the command line, query the database in it:

The following error occurred:

hive> 
    > show databases;
FAILED: IllegalArgumentException java.net.URISyntaxException: Relative path in absolute URI: file:./$HIVE_HOME/iotmp/fef28b9a-a421-4a53-8a2d-e43cf2ca4e40/hive_2019-09-06_11-43-53_217_6769807244516030197-1
hive> 
As shown in the figure:

After changing the path to absolute path, the following is done:


modify hive-site.xml file

<property>
  <name>hive.exec.local.scratchdir</name>
    <value>/data/hive/iotmp</value>
    <description>Local scratch space for Hive jobs</description>
</property>

<property>
   <name>hive.querylog.location</name>
    <value>/data/hive/iotmp</value>
    <description>Location of Hive run time structured log file</description>
 </property>

 <property>
   <name>hive.downloaded.resources.dir</name>
    <value>/data/hive/iotmp</value>
    <description>Temporary local directory for added resources in the remote file system.</description>
 </property>

 

Re-execution: schematool-dbType mysql-initSchema

Error: Table'CTLGS'already exists (state = 42S01, code = 1050)

Then I found that some tables had been initialized in the mysql library, and the tables in the graph already existed, but could not be deleted separately because of the foreign key, so I deleted and rebuilt the library, and then reinitialized it.

Then I went to hive:

Then execute show databases to view databases:

As shown in the figure, it's normal to view the database with a default, but when I create the database, it's unusual: I feel that there is a problem with my creation command. Fix the following successful execution

show databases;
create database myhive;

Posted by Jak-S on Thu, 05 Sep 2019 23:57:36 -0700