Linux ordinary user performs sudo permission setting
Execute the command sudo useradd --help
The error is reported as follows:fuyun is not in the sudoers file. This incident will be reported.
Reason: the user does not have root permission temporarily
Solution: modify sudoers file
Switch to the root user, su -, run the visudo command, which is used ...
Posted by undecided name 01 on Thu, 31 Oct 2019 12:52:24 -0700
Pit that Hive stepped on during use
Error 1 when hive starts
Cannot execute statement:impossible to write to binary long since BINLOG_FORMAT = STATEMENT...
//Error when starting
Caused by: javax.jdo.JDOException:Couldnt obtain a new sequence(unique id):Cannot execute statement:impossible to write to binary log since BINLOG_FORMAT = STATEMENT and at least one table uses a storage ...
Posted by FortMyersDrew on Mon, 28 Oct 2019 14:17:31 -0700
Add LZO compression support for Hadoop
The compression mode with lzo enabled is very useful for small-scale clusters. The compression ratio can be reduced to about 1 / 3 of the original log size. At the same time, the speed of decompression is faster.
install
Prepare jar package
1) Download lzo's jar project firsthttps://github.com/twitter/hadoop-lzo/archive/master.zip
2) the na ...
Posted by fastfingertips on Tue, 15 Oct 2019 10:36:22 -0700
Hive Homework - Return the name of the student with the highest score for each course
Assignment: Return the name of the student who scored the highest mark for each course.
t_score_data.txt
zhangsan math:90,english:60
lisi chinese:80,math:66,english:77
wangwu chinese:66,math:55,english:80
Return the names of the students for each course and for the highest score
chinese ...
Posted by nephish on Sun, 13 Oct 2019 10:23:26 -0700
The reason why Hive failed to connect to the database (Nested Throwables)
NestedThrowables:
java.sql.SQLException: Unable to open a test connection to the given database. JDBC url = jdbc:mysql://hadoop:3306/metastore?createDatabaseIfNotExist=true, username = root. Terminating connection pool (set lazyInit to true if you expect to start your database after your app). Origin ...
Posted by mm00 on Sun, 06 Oct 2019 14:23:36 -0700
hive array, map, struct use
Links to the original text: https://my.oschina.net/zipu888/blog/549644
Structs: Data within structs can be accessed through DOT (.). For example, the type of column c in the table is ...
Posted by mortimerdude on Wed, 02 Oct 2019 17:08:18 -0700
Hive Pre-installation
premise
Log in with the hive user and create a Hive database: mysql-uhive-phive under your own database
mysql>create database hive character set utf8;
mysql>flush privileges;
mysql>show databases;
install
Start installing hive on hadoop1
tar -zxvf apache-hive-1.2.1-bin.tar.gz -C /usr ...
Posted by nincha on Mon, 30 Sep 2019 04:47:14 -0700
Hive custom UDF function
In hive, sometimes you need to customize some functions according to business requirements. Here are the steps to customize functions
1. Create a new maven project and introduce dependencies in the project's pom file
<dependency>
<groupId>org.apache.hive</groupId&g ...
Posted by ded on Mon, 30 Sep 2019 03:59:50 -0700
Using flume to monitor hive files to HDFS in real time
Flume's main function is to read the data from the server's local disk in real time and write the data to HDFS.
The Flume architecture is as follows:
The internal principle of Flume Agent is shown in the figure.
1. Configure hive log file storage location:
2. Modify configuration items:
...
Posted by bradley252 on Mon, 30 Sep 2019 03:16:06 -0700
HiveQL table and table query
HiveQL Data Operation
1. Loading data into tables
load data local inpath '/data/employees'
overwrite into table employees
partition (country='US',state='CA')
If the partition directory does not exist, this command automatically creates the part ...
Posted by scross on Wed, 18 Sep 2019 04:24:15 -0700