Hadoop Programming Tour I

Configure the programming environment on Hadoop and test sample run Pre-preparation: Hadoop runs successfully"Hadoop" Installation of the Sixth Bullet from Zero - Brief Bookhttp://www.jianshu.com/p/96ed8b7886d2 Install the Linux version of eclipseeclipse-jee-mars-2-linux-gtk-x86_64.tar.gzUse tar command to decompress under Linux, ...

Posted by haynest on Sun, 09 Jun 2019 18:28:54 -0700

Hive Learning: Hive Installation

Installation Prerequisites Hadoop cluster has been installed and configured (single or fully distributed) Software Download Hive: https://hive.apache.org/index.html Hive Installation Configuring environment variables Upload the downloaded Live package to the machine and extract it to the specified path Edit / etc/profile to configure Hi ...

Posted by Adeus on Sat, 08 Jun 2019 14:45:09 -0700

Android Fragment Learning and Use - Advanced Chapter

I. overview The last article has explained Fragment's life cycle, basic usage methods and the role of some api s. But if you want to use Fragment well in your project, you must be able to clearly manage its status. Here are some scenarios that you will encounter in actual development. 2. Fragment Back Stack Management Activity is managed ...

Posted by ludjer on Wed, 29 May 2019 11:27:53 -0700

hue oozie stepped on the pit again, workflow,coordinator could finally run away.

The front summarizes some pits of sqoop1, oozie and HBase under hue. When the project expires today, we must work out oozie workflow and schedule execution on time to skr skr. 1. The pits of sqoop mysql imported and exported from the front have been trampled. Later, it was found that besides cdh(5.15), sqoop1 was not automatically configured. ...

Posted by Ree on Sat, 18 May 2019 22:13:03 -0700

hadoop configuration and wordcount

Most of the reference blogs are based on Hadoop 2.x and low-level java. The configuration process seems very simple to write. Looking at other people's blogs, it feels the same steps. But there are many problems when configuring the blog: data node can't start, the page can't display properly, data node inexplicably dies, resourcemanager can't ...

Posted by dieselmachine on Sat, 18 May 2019 09:45:22 -0700

Installation of hadoop and hbase on Linux

Hadoop installation steps hadoop Download address: http://mirrors.advancedhosters.com/apache/hadoop/common/   1. Install JDK and hadoop (jdk requires version 1.7 or more) The user installing hadoop is root Create installation directory and unzip related files: mkdir /usr/local/java mkdir /usr/local/hadoop mkdir /usr/loc ...

Posted by rickead2000 on Fri, 17 May 2019 12:51:02 -0700

UDF, UDTF and UDAF in hive

New table "apache_log" in hive CREATE TABLE apachelog ( host STRING, identity STRING, user STRING, time STRING, request STRING, status STRING, size STRING, referer STRING, agent STRING) ROW FORMAT SERDE 'org.apache.hadoop.hive.serde2.RegexSerDe' WITH SERDEPROPERTIES ( "input.regex" = "([^ ] ...

Posted by DJH on Fri, 17 May 2019 11:36:59 -0700

Flink reality: Statistics website PV, UV

Flink reality: Statistics website PV, UV PV,UV PV (Page View): Page clicks UV (User View): Number of visits by independent users Assume the requirements are as follows, every minute interval, count the last 5 minutes of UV, PV.It is easy to imagine that the correct results can be obtained from the count and count distinct of the database.Tra ...

Posted by jlr2k8 on Thu, 16 May 2019 01:57:37 -0700

Spark Cluster Installation

spark version 1.6.0, Scala version 2.12, jdk version 1.8. spark has been used recently and is recorded here. One master and three workers, together with Hadoop-2.7.7 cluster, namdenode on master and two datanode s on worker 1 and worker 2.     List-1 192.168.33.30 worker1 master 192.168.33.31 worker2 192.168.33.32 worker3 Modify the h ...

Posted by gidiag on Wed, 15 May 2019 19:09:29 -0700

[Java] AQS Source Code Analysis

The full name of AQS is AbstractQueued Synchronizer, which is a synchronizer design framework provided by JDK. Many concurrent data structures, such as ReentrantLock, ReentrantReadWriteLock, Semaphore and so on, are implemented based on AQS. Next, the principle of AQS is analyzed.   I. underlying data structure The AQS under ...

Posted by giraffemedia on Wed, 15 May 2019 16:13:37 -0700