Persistent Ephemeral Node / Persistent Node

A persistent node that can be maintained when a link or session is interrupted.

1. key API

org.apache.curator.framework.recipes.nodes.PersistentEphemeralNode

2. mechanism description

Actually, it's org.apache.curator.framework.recipes.nodes.PersistentNode.

Considering a lot of abnormal situations, it can automatically retry the recovery node when the abnormal occurs.

Compared with creating nodes directly, PersistentNode is more secure and convenient.

3. usage

3.1 create

public PersistentNode(CuratorFramework client,
                               CreateMode mode,
                               boolean useProtection,
                               String basePath,
                               byte[] data)

3.2 use

1. Before using: node.start();
2. Need to call after using: node.close();
   • close() deletes nodes

4. error handling

The PersistentNode instance handles all errors internally and recreates (restores) nodes as needed.

5. source code analysis

@Deprecated
public class PersistentEphemeralNode extends PersistentNode
{
    @Deprecated
    public enum Mode
    {
        /**
         * Same as {@link CreateMode#EPHEMERAL}
         */
        EPHEMERAL()
            {
                @Override
                protected CreateMode getCreateMode(boolean pathIsSet)
                {
                    return CreateMode.EPHEMERAL;
                }

                @Override
                protected boolean isProtected()
                {
                    return false;
                }
            },

        /**
         * Same as {@link CreateMode#EPHEMERAL_SEQUENTIAL}
         */
        EPHEMERAL_SEQUENTIAL()
            {
                @Override
                protected CreateMode getCreateMode(boolean pathIsSet)
                {
                    return pathIsSet ? CreateMode.EPHEMERAL : CreateMode.EPHEMERAL_SEQUENTIAL;
                }

                @Override
                protected boolean isProtected()
                {
                    return false;
                }
            },

        /**
         * Same as {@link CreateMode#EPHEMERAL} with protection
         */
        PROTECTED_EPHEMERAL()
            {
                @Override
                protected CreateMode getCreateMode(boolean pathIsSet)
                {
                    return CreateMode.EPHEMERAL;
                }

                @Override
                protected boolean isProtected()
                {
                    return true;
                }
            },

        /**
         * Same as {@link CreateMode#EPHEMERAL_SEQUENTIAL} with protection
         */
        PROTECTED_EPHEMERAL_SEQUENTIAL()
            {
                @Override
                protected CreateMode getCreateMode(boolean pathIsSet)
                {
                    return pathIsSet ? CreateMode.EPHEMERAL : CreateMode.EPHEMERAL_SEQUENTIAL;
                }

                @Override
                protected boolean isProtected()
                {
                    return true;
                }
            };

        protected abstract CreateMode getCreateMode(boolean pathIsSet);

        protected abstract boolean isProtected();
    }

    @SuppressWarnings("deprecation")
    public PersistentEphemeralNode(CuratorFramework client, Mode mode, String basePath, byte[] initData)
    {
        super(client, mode.getCreateMode(false), mode.isProtected(), basePath, initData);
    }

• Inheritance and org.apache.curator.framework.recipes.nodes.PersistentNode
• It is not recommended to use.
  • It is recommended to use org.apache.curator.framework.recipes.nodes.PersistentNode directly. - Defines an internal enumeration: org. apache. curator. framework. recipes. nodes. Persistent EphemeralNode. Mode
  • Its function is compatible with org.apache.zookeeper.CreateMode.
  • It can be seen that Persistent Ephemeral Node is basically abandoned.
    • Downward compatibility through Mode
    • Actual logic and function are fully used by org.apache.curator.framework.recipes.nodes.PersistentNode

So the next step is to target the source code of org.apache.curator.framework.recipes.nodes.PersistentNode.

The 5.1 definition

public class PersistentNode implements Closeable{}

• Implement java.io.Closeable interface

5.2 member variables

public class PersistentNode implements Closeable {
    private final AtomicReference<CountDownLatch> initialCreateLatch = new AtomicReference<CountDownLatch>(new CountDownLatch(1));
    private final Logger log = LoggerFactory.getLogger(getClass());
    private final CuratorFramework client;
    private final AtomicReference<String> nodePath = new AtomicReference<String>(null);
    private final String basePath;
    private final CreateMode mode;
    private final AtomicReference<byte[]> data = new AtomicReference<byte[]>();
    private final AtomicReference<State> state = new AtomicReference<State>(State.LATENT);
    private final AtomicBoolean authFailure = new AtomicBoolean(false);
    private final BackgroundCallback backgroundCallback;
    private final boolean useProtection;
    private final AtomicReference<CreateModable<ACLBackgroundPathAndBytesable<String>>> createMethod = new AtomicReference<CreateModable<ACLBackgroundPathAndBytesable<String>>>(null);
    private final CuratorWatcher watcher = new CuratorWatcher()
    {
        @Override
        public void process(WatchedEvent event) throws Exception
        {
            if ( event.getType() == EventType.NodeDeleted )
            {
                createNode();
            }
            else if ( event.getType() == EventType.NodeDataChanged )
            {
                watchNode();
            }
        }
    };
    private final BackgroundCallback checkExistsCallback = new BackgroundCallback()
    {
        @Override
        public void processResult(CuratorFramework client, CuratorEvent event) throws Exception
        {
            if ( event.getResultCode() == KeeperException.Code.NONODE.intValue() )
            {
                createNode();
            }
            else
            {
                boolean isEphemeral = event.getStat().getEphemeralOwner() != 0;
                if ( isEphemeral != mode.isEphemeral() )
                {
                    log.warn("Existing node ephemeral state doesn't match requested state. Maybe the node was created outside of PersistentNode? " + basePath);
                }
            }
        }
    };
    private final BackgroundCallback setDataCallback = new BackgroundCallback()
    {

        @Override
        public void processResult(CuratorFramework client, CuratorEvent event)
            throws Exception
        {
            //If the result is ok then initialisation is complete (if we're still initialising)
            //Don't retry on other errors as the only recoverable cases will be connection loss
            //and the node not existing, both of which are already handled by other watches.
            if ( event.getResultCode() == KeeperException.Code.OK.intValue() )
            {
                //Update is ok, mark initialisation as complete if required.
                initialisationComplete();
            }
        }
    };
    private final ConnectionStateListener connectionStateListener = new ConnectionStateListener()
    {
        @Override
        public void stateChanged(CuratorFramework client, ConnectionState newState)
        {
            if ( newState == ConnectionState.RECONNECTED )
            {
                createNode();
            }
        }
    };

    private enum State
    {
        LATENT,
        STARTED,
        CLOSED
    }
}

• initialCreateLatch
  • A java.util.concurrent.CountDownLatch for atomic references
  • The number of locking is 1.
  • Blocking processing for waiting for initialization
• Log log
• client
• nodePath
  • Node path
  • Atomic Reference
  • It contains some automatically generated serial numbers or GUID s, so it needs to be distinguished from basePath.
  • The nodePath is used as the basis for judging whether initialization is completed, so Atomic Reference is needed here.
• basePath
• mode
  • org.apache.zookeeper.CreateMode
  • How nodes are created (temporary / persistent)
• data
  • Node data
  • Atomic Reference
• state
  • state
  • Internal enumeration
    • LATENT
    • STARTED
    • CLOSED
  • Atomic Reference
• authFailure
  • Is the zk password wrong?
  • Atomic Boolean
• backgroundCallback
  • org.apache.curator.framework.api.BackgroundCallback
  • Callback
• useProtection
  • Whether to use protection mode or not
  • Using protected mode, a GUID prefix is added to the node name
    • GUID: Global Unique Identifier
    • When creating a node (especially a temporarily ordered node), if it fails to create a successful return to the customer service, the client will not know which temporarily ordered node it is when it is its own.
    • When the creation fails and retries are made, the current GUID under the parent node is checked first.
      • If so, it indicates that the last time it was created successfully, but no successful response was received.
• createMethod
  • How to create template methods (functions) for nodes
• watcher
  • Node listeners (observers)
• checkExistsCallback
  • Callback checks whether the node exists
  • Create node
• connectionStateListener
  • Link status listener
  • When the link is restored, the node is recreated

5.3 constructor

public PersistentNode(CuratorFramework client, final CreateMode mode, boolean useProtection, final String basePath, byte[] initData)
{
    this.useProtection = useProtection;
    this.client = Preconditions.checkNotNull(client, "client cannot be null");
    this.basePath = PathUtils.validatePath(basePath);
    this.mode = Preconditions.checkNotNull(mode, "mode cannot be null");
    final byte[] data = Preconditions.checkNotNull(initData, "data cannot be null");

    backgroundCallback = new BackgroundCallback()
    {
        @Override
        public void processResult(CuratorFramework client, CuratorEvent event) throws Exception
        {
            if ( state.get() == State.STARTED )
            {
                processBackgroundCallback(event);
            }
            else
            {
                processBackgroundCallbackClosedState(event);
            }
        }
    };

    this.data.set(Arrays.copyOf(data, data.length));
}

• Basic attribute assignment
  • Note that data uses a copy
• Initialized backgroundCallback callback logic
  • This callback triggers various event handling

Here we can go back to the initialization of Persistent EphemeralNode:

@SuppressWarnings("deprecation")
public PersistentEphemeralNode(CuratorFramework client, Mode mode, String basePath, byte[] initData)
{
    super(client, mode.getCreateMode(false), mode.isProtected(), basePath, initData);
}

• The main thing is the conversion of Mode.
  • If it is a general temporary node, no protection mode is used.
  • If it is a protected temporary node, use protection mode

5.4 boot

You need to call the start() method before you use it

5.4.1 start

public void start()
{
    Preconditions.checkState(state.compareAndSet(State.LATENT, State.STARTED), "Already started");

    client.getConnectionStateListenable().addListener(connectionStateListener);
    createNode();
}

1. Atomization update state to start state STARTED
   • Determine by an assertion that no duplicate startup is allowed
2. Add the link status listener connectionStateListener to the link
   • In this way, the node can be automatically recreated after the link interruption is restored.
3. Call the createNode() method to create a node

5.4.2 createNode

private void createNode()
{
    if ( !isActive() )
    {
        return;
    }

    try
    {
        String existingPath = nodePath.get();
        String createPath = (existingPath != null && !useProtection) ? existingPath : basePath;

        CreateModable<ACLBackgroundPathAndBytesable<String>> localCreateMethod = createMethod.get();
        if ( localCreateMethod == null )
        {
            CreateModable<ACLBackgroundPathAndBytesable<String>> tempCreateMethod = useProtection ? client.create().creatingParentContainersIfNeeded().withProtection() : client.create().creatingParentContainersIfNeeded();
            if ( createMethod.compareAndSet(null, tempCreateMethod) )
            {
                localCreateMethod = tempCreateMethod;
            }
        }
        localCreateMethod.withMode(getCreateMode(existingPath != null)).inBackground(backgroundCallback).forPath(createPath, data.get());
    }
    catch ( Exception e )
    {
        ThreadUtils.checkInterrupted(e);
        throw new RuntimeException("Creating node. BasePath: " + basePath, e);  // should never happen unless there's a programming error - so throw RuntimeException
    }
}

1. If the current state is not STARTED, no nodes need to be created.
2. On the first boot, use basePath to create
3. Decide which creation method (function) to use depending on whether protection mode is used or not.
4. If the node has been created, basePath does not use ordered nodes
   • If there are ordered nodes, the path cannot be maintained (the serial number will change)
5. Adding callback processing logic for creating nodes through backgroundCallback
   • The state is STARTED, so backgroundCallback calls processBackgroundCallback(event) to handle events

5.4.2 processBackgroundCallback

private void processBackgroundCallback(CuratorEvent event) throws Exception
{
    String path = null;
    boolean nodeExists = false;
    if ( event.getResultCode() == KeeperException.Code.NODEEXISTS.intValue() )
    {
        path = event.getPath();
        nodeExists = true;
    }
    else if ( event.getResultCode() == KeeperException.Code.OK.intValue() )
    {
        path = event.getName();
    }
    else if ( event.getResultCode() == KeeperException.Code.NOAUTH.intValue() )
    {
        log.warn("Client does not have authorisation to write node at path {}", event.getPath());
        authFailure.set(true);
        return;
    }
    if ( path != null )
    {
        authFailure.set(false);
        nodePath.set(path);
        watchNode();

        if ( nodeExists )
        {
            client.setData().inBackground(setDataCallback).forPath(getActualPath(), getData());
        }
        else
        {
            initialisationComplete();
        }
    }
    else
    {
        createNode();
    }
}

1. If you do not have the right to create a node
   1. Authorization failed by setting authFailure to true
   2. Return
2. If the node already exists, or if the creation is successful
   1. Set nodePath as the node path
      • Marks completion of initialization
   2. Adding checkExistsCallback listeners to nodes
      • When a node is deleted, it can be recreated through this listener
   3. If the node exists
      1. Data coverage
         • After coverage is complete, initialization is complete
   4. Otherwise, initialization is complete
3. In other cases, try createNode() again and initialize again.

5.6 setting data

public void setData(byte[] data) throws Exception Using this method, we should pay attention to:

1. start() until initialization is complete, there is a process
2. So, after start, you may encounter situations where initialization has not been completed by calling setup data directly.
   • At this point an exception will be thrown
3. So it's better to call public Boolean waitForInitial Create (long timeout, TimeUnit unit) and wait for initialization to complete before calling this method.

public void setData(byte[] data) throws Exception
{
    data = Preconditions.checkNotNull(data, "data cannot be null");
    Preconditions.checkState(nodePath.get() != null, "initial create has not been processed. Call waitForInitialCreate() to ensure.");
    this.data.set(Arrays.copyOf(data, data.length));
    if ( isActive() )
    {
        client.setData().inBackground().forPath(getActualPath(), getData());
    }
}

5.5 Get the Actual Path

In the process of creating a node, the program will also deal with the node name according to the conditions of creating a node, whether it is a collection, whether it uses protection mode and so on. So, in some cases, you need to get the real node path: public String get ActualPath ()

public String getActualPath()
{
    return nodePath.get();
}

• NoePath was analyzed in 5.5 and has value only after initialization is completed.
• So you also need to judge public Boolean wait for Initial Create (long timeout, TimeUnit unit)

6. summary

Creating zk nodes through org.apache.curator.framework.recipes.nodes.PersistentNode can avoid many anomalies:

• Link state change
  • Automatic recovery of nodes after link interruption recovery
• The creation was successful, but the client did not receive a response
• Nodes were accidentally deleted
• Unexpected modification of node data

Therefore, ZK nodes can be created easily and safely through PersistentNode.