Linux ordinary user performs sudo permission setting

Execute the command sudo useradd --help The error is reported as follows:fuyun is not in the sudoers file. This incident will be reported. Reason: the user does not have root permission temporarily Solution: modify sudoers file Switch to the root user, su -, run the visudo command, which is used ...

Posted by undecided name 01 on Thu, 31 Oct 2019 12:52:24 -0700

Pit that Hive stepped on during use

Error 1 when hive starts Cannot execute statement:impossible to write to binary long since BINLOG_FORMAT = STATEMENT... //Error when starting Caused by: javax.jdo.JDOException:Couldnt obtain a new sequence(unique id):Cannot execute statement:impossible to write to binary log since BINLOG_FORMAT = STATEMENT and at least one table uses a storage ...

Posted by FortMyersDrew on Mon, 28 Oct 2019 14:17:31 -0700

Add LZO compression support for Hadoop

The compression mode with lzo enabled is very useful for small-scale clusters. The compression ratio can be reduced to about 1 / 3 of the original log size. At the same time, the speed of decompression is faster. install Prepare jar package 1) Download lzo's jar project firsthttps://github.com/twitter/hadoop-lzo/archive/master.zip 2) the na ...

Posted by fastfingertips on Tue, 15 Oct 2019 10:36:22 -0700

Hive Homework - Return the name of the student with the highest score for each course

Assignment: Return the name of the student who scored the highest mark for each course. t_score_data.txt zhangsan math:90,english:60 lisi chinese:80,math:66,english:77 wangwu chinese:66,math:55,english:80 Return the names of the students for each course and for the highest score chinese ...

Posted by nephish on Sun, 13 Oct 2019 10:23:26 -0700

The reason why Hive failed to connect to the database (Nested Throwables)

NestedThrowables: java.sql.SQLException: Unable to open a test connection to the given database. JDBC url = jdbc:mysql://hadoop:3306/metastore?createDatabaseIfNotExist=true, username = root. Terminating connection pool (set lazyInit to true if you expect to start your database after your app). Origin ...

Posted by mm00 on Sun, 06 Oct 2019 14:23:36 -0700

hive array, map, struct use

Links to the original text: https://my.oschina.net/zipu888/blog/549644 Structs: Data within structs can be accessed through DOT (.). For example, the type of column c in the table is ...

Posted by mortimerdude on Wed, 02 Oct 2019 17:08:18 -0700

Hive Pre-installation

premise Log in with the hive user and create a Hive database: mysql-uhive-phive under your own database mysql>create database hive character set utf8; mysql>flush privileges; mysql>show databases; install Start installing hive on hadoop1 tar -zxvf apache-hive-1.2.1-bin.tar.gz -C /usr ...

Posted by nincha on Mon, 30 Sep 2019 04:47:14 -0700

Hive custom UDF function

In hive, sometimes you need to customize some functions according to business requirements. Here are the steps to customize functions 1. Create a new maven project and introduce dependencies in the project's pom file <dependency> <groupId>org.apache.hive</groupId&g ...

Posted by ded on Mon, 30 Sep 2019 03:59:50 -0700

Using flume to monitor hive files to HDFS in real time

Flume's main function is to read the data from the server's local disk in real time and write the data to HDFS. The Flume architecture is as follows: The internal principle of Flume Agent is shown in the figure.   1. Configure hive log file storage location: 2. Modify configuration items: ...

Posted by bradley252 on Mon, 30 Sep 2019 03:16:06 -0700

HiveQL table and table query

HiveQL Data Operation 1. Loading data into tables load data local inpath '/data/employees' overwrite into table employees partition (country='US',state='CA') If the partition directory does not exist, this command automatically creates the part ...

Posted by scross on Wed, 18 Sep 2019 04:24:15 -0700