Fully distributed cluster in installation mode
1. Introduction to fully distributed mode
Fully distributed refers to using multiple machines to build a complete distributed file system in a real environment.
In the real world, hdfs Related daemons in are also distributed in different machines, such as:
-1. namenode As far as possible, the dae ...
Posted by Stryves on Mon, 06 Dec 2021 19:34:03 -0800
If we store a large amount of HBase data at one time, the processing speed is slow, and the Region resources are particularly occupied, a more efficient and convenient method is to use the "Bulk Loading" method, that is, the HFileOutputFormat class provided by HBase. It uses the principle that hbase data information is ...
Posted by rharter on Sun, 05 Dec 2021 20:42:35 -0800
How HDFS works
1. NameNode and DataNode
HDFS adopts master/slave architecture. An HDFS cluster consists of a NameNode and a certain number of datanodes. NameNode is a central server, which is responsible for managing the namespace of the file system and the access of clients to files. A DataNode in a cluster is usually one node, which is resp ...
Posted by SamLiu on Fri, 03 Dec 2021 06:17:26 -0800
If Hadoop has not been configured, you can click the link to see how to configure it
Basic technical teaching and practical development teaching of major technologies (under continuous updating ··················&middo ...
Posted by ducey on Sun, 21 Nov 2021 22:15:57 -0800
1 what is safe mode
Security mode is a special state of HDFS. In this state, the file system only accepts data read requests, but does not accept change requests such as deletion and modification.
When the namenode master node is started, HDFS first enters the safe mode. When the datanode is started, it will report the available block and oth ...
Posted by Thumper on Sat, 06 Nov 2021 11:06:32 -0700
In fact, the construction methods of fully distributed and pseudo distributed are basically the same. Only individual files need to be changed. Let's start now!
The first step is to configure the IP address, host name, and the mapping between host and IP address. The tutorial is included in my previous pseudo distributed building, which will n ...
Posted by smudge on Sun, 31 Oct 2021 12:30:57 -0700
This document is compiled with reference to the official website of Hadoop and many other materials. For neat typesetting and comfortable reading, blurred and unclear pictures and black-and-white pictures are redrawn into high-definition color pictures.
At present, Hadoop 2. X is widely used in enterprises, so this article focuses on Hado ...
Posted by b-real on Thu, 21 Oct 2021 21:42:54 -0700
Common HDFS commands
<path>... One or more paths in hdfs, if not specified, default to/user/<currentUser> <localsrc>...one or more paths to the local file system Target Path in <dst> HDFS
Command: hdfs dfs -help [cmd ...]
cmd... One or more commands to query
Command: hd ...
Posted by dr.wong on Sat, 02 Oct 2021 09:30:52 -0700