IPFS private network and Cluster construction

Keywords: ipfs

This article uses virtual machines to build IPFS private networks and clusters, and uses vagrant and virtualbox to quickly build the environment.

Private network

Download binaries

If you use the go IPFs source code to compile, you need to install the go environment. It is recommended to download the binary file:

  1. https://dist.ipfs.io

  2. https://github.com/ipfs/go-ipfs/releases

Use 1 to download other components such as go IPFs

Build virtual machine environment

install vagrant And virtualbox

# mac
brew install vagrant
brew intall virtualbox

# ubuntu
sudo apt update
sudo apt install virtualbox

curl -fsSL https://apt.releases.hashicorp.com/gpg | sudo apt-key add -
sudo apt-add-repository "deb [arch=amd64] https://apt.releases.hashicorp.com $(lsb_release -cs) main"
sudo apt-get update && sudo apt-get install vagrant

Create virtual machine

Using Ubuntu / focal64( vagrant boxes list ), create Vagrantfile as follows:

mkdir ~/vagrant-project
cd /vagrant-project
vagrant init ubuntu/focal64  # Create Vagrantfile

The initialized Vagrantfile configuration is very simple. Four virtual machine environments are used. The simple modifications are as follows:

# -*- mode: ruby -*-
# vi: set ft=ruby :

Vagrant.configure("2") do |config|

	(1..4).each do |i|
    	config.vm.define "node#{i}" do |node|
			node.vm.box = "ubuntu/focal64"
			node.vm.hostname = "node#{i}"
            node.vm.network "private_network", ip: "{i}"

			node.vm.provider "virtualbox" do |v|
				v.name = "node#{i}"
				v.memory = 2048
				v.cpus = 1

Installing IPFS and configuring

Move the downloaded binary file to the project directory ~ / vagrant project, which will be mounted to the virtual machine / vagrant by default.

Start the virtual machine:

vagrant up

The following is the configuration of node node1, and the other nodes are similar

Install IPFS binaries:

vagrant ssh node1   # Enter node1
cd /vagrant         # Enter the mount directory
tar -C /usr/local -zxvf go-ipfs_v0.10.0_linux-amd64.tar.gz # decompression
cd /usr/local/go-ipfs
sudo ./install.sh  # Copy to / usr/local/bin

ipfs node initialization:

ipfs init

Generate shared key for private network

Generate the shared key file swarm.key. Peer nodes will establish connections with each other only when their contents are the same, and the go environment needs to be installed Installation steps

  1. Download build tool
go get -u github.com/Kubuxu/go-ipfs-swarm-key-gen/ipfs-swarm-key-gen

Generate the file swarm.key and copy it to the ~ /. ipfs warehouse directory

ipfs-swarm-key-gen > ~/.ipfs/swarm.key
  1. Copy the file to the ~ /. ipfs directory on the remaining nodes
  • Copy the file to the file shared by the node
  • Or cat ~/.ipfs/swarm.key copy the contents of the file and recreate it on other nodes

Delete the default boot node for the node

 ipfs bootstrap rm all

(add default boot node: ipfs bootstrap add default)

Start multiple nodes

Note that the above operations should be performed in all nodes; Start multiple nodes. After the nodes are configured as described above, you need to connect to the local boot node. For example, set node node1 as the boot node and start other nodes (such as node2) to connect:

First, you need to start the ipfs daemon node node1

View the address of node1

ipfs id
# /ip4/

Add node1

ipfs bootstrap add /ip4/

Start other nodes

ipfs daemon

View peer nodes

ipfs swarm peers


You can use "peering": {"peers": null} in the configuration file to add a connection that needs to be protected. It will also be connected automatically at startup

Problem: the announcement address may not be specified after the node is started. You can add the published address in the announcement in the configuration file (or connect directly)

So far, the private network composed of multiple nodes has been built, and the external nodes cannot connect or access the files in the network; For managing multiple IPFS nodes and ensuring data security and reliability, you can build an IPFS Cluster

IPFS Cluster setup

IPFS cluster node and IPFS are in one-to-one correspondence. First build the above IPFS multi node network and continue to build IPFS cluster. Official documents

Briefly introduce the relationship between the two:

IPFS cluster service starts a cluster peer node, which depends on an ipfs daemon node; The cluster node will join another swarm network independent of the IPFS network

cluster peer will participate in the consensus of the cluster, follow a distributed log of fixed pin and unpin requests, and manage related pin operations for the configured IPFS daemon

cluster peer provides an API for cluster management and an IPFS Proxy API to forward requests to IPFS daemon and internal communication components

             | ipfs-cluster-ctl |
                       | HTTP(s)
ipfs-cluster-service   |                           HTTP
+----------+--------+--v--+----------------------+      +-------------+
| RPC      | Peer 1 | API | IPFS Connector/Proxy +------> IPFS daemon |
+----^-----+--------+-----+----------------------+      +-------------+
     | libp2p
+----v-----+--------+-----+----------------------+      +-------------+
| RPC      | Peer 2 | API | IPFS Connector/Proxy +------> IPFS daemon |
+----^-----+--------+-----+----------------------+      +-------------+
+----v-----+--------+-----+----------------------+      +-------------+
| RPC      | Peer 3 | API | IPFS Connector/Proxy +------> IPFS daemon |
+----------+--------+-----+----------------------+      +-------------+

By default, the cluster uses:

  • 9096/tcp, as a cluster swarm endpoint, should be open and can be called by other cluster peers.
  • 9094/tcp as HTTP API endpoint
  • 9095/tcp as proxy API endpoint


according to the above address Download related components

  • IPFs cluster service (installed per node) - used to start a cluster peer node
  • IPFs cluster CTL (one node can be installed) - used to interact with IPFs cluster service
  • IPFs cluster follower (optional. Running a follower peer without write permission is used to join the collaboration cluster)

After downloading, unzip it directly and move the corresponding binary file to / usr/local/bin

to configure

All cluster peer s need to use the same secret and consensus components. First initialize at node1 node (CRDT is used by default):

You need to start the IPFS daemon of this node first

ipfs-cluster-service init

Three files will be generated under ~ / IPFs cluster /

  • Identity: id and private key of cluster peer
  • peerstore: stores known peer node addresses
  • service.json: configuration file

You can view the secret key of the current node of service.json; Other nodes are set to be the same as node node1:

ipfs-cluster-service init --secrect <node1-secret>

The same environment variable cluster secrets can also be used to set the key

In addition, you can use the remote configuration file or the configuration file IPFS cluster service init stored in IPFS http://localhost:8080/ipns/config.mydomain.com

Start the cluster peer corresponding to node node1 first:

ipfs-cluster-service daemon

Start other nodes

After other nodes have set the same secret, directly add node node1 and start the corresponding cluster peer

View the id of node node1:

ipfs-cluster-ctl id
#  /ip4/

If the private address is not displayed, you can use it directly

Use the – bootstrap option when starting the remaining nodes:

ipfs-cluster-service daemon --bootstrap /ip4/

The IPFS Cluster of multiple nodes has been built

Other configurations

For the production environment, the configuration file provides many configurations. Please refer to the documentation for details

IPFs cluster configuration reference document

Cluster interaction

Peer node

ipfs-cluster-ctl peers list

Add file

The default Pinset plan allocation strategy of the cluster is fixed once for each node. In the corresponding configuration file, service.json:

    "replication_factor_min": -1,
    "replication_factor_max": -1,

Add a haha.gif file on node node1: [you can display the specified replication factor]

ipfs-cluster-ctl add haha.gif
# added QmRZ1eBKEvgFx38o1gGB8K2KKsno1dzzunG9cCYuhZLzwp haha.gif

Options you can add:

  • Specify the minimum and maximum replication factors -- replication min 2 -- replication max 3 to maximize the number of copies
  • Specify replication factor -- replication 2
  • Set the name associated with the pin, for example -- name website

Note: use ipfs cluster CTL   add  < File > only the files added will be managed by the cluster and follow the allocation policy; Use ipfs directly   add  < File > is only added to the local storage of this node

Fix the CID (or accessible CID) that has been added to the IPFS node to the Cluster:

ipfs-cluster-ctl pin add <CID> 

ipfs cluster CTL add is equivalent to adding a file to the ipfs network and then fixing its pin to the cluster

You can refer to the detailed adding process and operation Official documents

File status


  • IPFs cluster CTL Pin LS displays information from cluster share status or global pinset, which is fully available in each peer. It shows the Pin's allocation node and the Pin's related configuration
  • IPFS cluster CTL status requests information about the status of each pin on each cluster peer, including whether the CID is PINNED on IPFS, or is still PINNING, or there is an error for some reason (actual storage). These conditions are supported Filtering results

View the allocation strategy of CID and pin tracked by cluster: [only CID shared by cluster nodes]

ipfs-cluster-ctl pin ls # all pins
ipfs-cluster-ctl pin ls QmRZ1eBKEvgFx38o1gGB8K2KKsno1dzzunG9cCYuhZLzwp

View the actual fixed status of pins/CID: status command

ipfs-cluster-ctl status # Status of all cids tracked
ipfs-cluster-ctl status QmRZ1eBKEvgFx38o1gGB8K2KKsno1dzzunG9cCYuhZLzwp

Delete pin (ipfs will automatically unpin)

ipfs-cluster-ctl pin rm <CID>

access files

All IPFS nodes form a private network. You can directly use IPFS nodes to access files:

ipfs get -o haha.gif QmRZ1eBKEvgFx38o1gGB8K2KKsno1dzzunG9cCYuhZLzwp


restful API of ipfs cluster

Description and summary

  • The IPFS node in this article uses a private network, but it can also be a public node; IPFS Cluster nodes can form a private network; The two networks are independent. If it is a public network, the public gateway can access the content in the cluster

  • The cluster fixed pin can be regarded as two processes. One is the consensus component, which is the set of all pins maintained and tracked by the cluster node, and the other is the file content corresponding to the fixed pin of the corresponding IPFS node

Link Links

  1. restful API of ipfs cluster
  2. https://cluster.ipfs.io/documentation/reference/configuration/

Posted by turing_machine on Thu, 28 Oct 2021 02:24:15 -0700