abstract
In Pulsar, Function, Source, and Link are all running on Function Worker, and you can refer to Function for reference An article about Pulsar Functions , you can refer to the use of Source and Link
This paper introduces the election mechanism of Functions Worker.Read this article to gain a deeper understanding of Functions Worker.
Function Worker Architecture
The following is a schematic diagram of the Pulsar Functions Worker
Each component in the schema diagram, Metadata Mananger, Scheduler Manager, Runtime Manager, and Membership Manager, is running on Worker.
Worker's Election Mechanism
- The blue background is Worker
- Ellipse light green is subscription, also a Consumer
- The square green background is Broker
- Square red background is Topic
Let's look at Worker's election.
- Three Workers named Worker Service1, Worker Service2, and Worker Service3 are started in the diagram.These Workers can be on the same machine or on different machines.
2. At startup, a Consumer is started inside each Worker, which is used for elections, and each Worker's id, host name, and port number are bound to the Consumer.
3. The Consumer is started based on Failover mode (refer to [here] for Failover subscription mode for Consumer). http://pulsar.apache.org/docs... ).Connecting to the same Topic, only one Constumer with the same subscription name is active at one time.
4. The Worker using this Consumer is the Leader, which is responsible for scheduling and handling other operations.
Worker Service1 and Worker Service2 are one cluster, connected to Topic1 at the same time, with the same subscription name sub, Leader is generated in both of them.
Worker Service3 is another cluster, connected to Topic2, and the subscription name is sub3, because there is only one Worker, it is itself a Leader.
Test the election process
Let's practice the election process.
The following dependencies need to be installed on your computer before you can build this test, which was conducted on a Mac system.
Start Single Pulsar Service
Pull a mirror of apachepulsar/pulsar:2.3.1 and start it with the docker run command, the -d parameter runs the service in background mode, the -it runs the container in interactive mode, and assigns a pseudo input terminal to the container.
--name specifies that the container uses the name pulsar-standalone-function-leader.
docker run -d -it -p 6650:6650 -p 8080:8080 -v $PWD/data:/pulsar/data --name pulsar-standalone-function-leader apachepulsar/pulsar:2.3.1 bin/pulsar standalone
docker logs -f 11:17:17.363 [pulsar-web-55-4] INFO org.apache.pulsar.broker.web.PulsarWebResource - Successfully validated clusters on tenant [public] 11:17:17.369 [pulsar-web-55-4] INFO org.apache.pulsar.broker.admin.impl.NamespacesBase - [null] Created namespace public/default 11:17:17.370 [pulsar-web-55-4] INFO org.eclipse.jetty.server.RequestLog - 172.17.0.2 - - [18/May/2019:11:17:17 +0000] "PUT /admin/v2/namespaces/public/default HTTP/1.1" 204 0 "-" "Pulsar-Java-v2.3.1" 10 11:17:17.377 [pulsar-web-55-12] INFO org.apache.pulsar.broker.web.PulsarWebResource - Successfully validated clusters on tenant [public]
The log above shows that the service started successfully after starting Pulsar.
Simulate the election process
Worker's election was based on the Consumer's Failover model, so in this test, the Consumer was started directly to simulate the election.
As shown in the electoral chart above, this test will start Consumers with three Failover modes, two of which are connected to Topic1 using a subscription name of sub, which represents one Worker cluster; the other Consumer is connected to Topic2, and the subscription name of sub3 is used to represent another Worker cluster.
For this test, four windows named window1, window2, window3, and window4 need to be opened.
window4 is used for validation.
window1
Start a consumer in Failover mode, subscribe to public/default/topic1 using the subscription name sub
docker exec -it pulsar-standalone-function-leader /bin/bash ./bin/pulsar-client consume persistent://public/default/topic1 --num-messages 0 --subscription-name sub -t Failover
window2
Start a consumer in Failover mode, subscribe to public/default/topic1 using the subscription name sub
docker exec -it pulsar-standalone-function-leader /bin/bash ./bin/pulsar-client consume persistent://public/default/topic1 --num-messages 0 --subscription-name sub -t Failover
window3
Start a consumer in Failover mode, subscribe to public/default/topic2 using the subscription name sub3
docker exec -it pulsar-standalone-function-leader /bin/bash ./bin/pulsar-client consume persistent://public/default/topic2 --num-messages 0 --subscription-name sub3 -t Failover
All three windows operate using the same subscription mode Failover, while window1 and window2 subscribe to the same Topic with the same subscription name.
window4
Get topic1 statistics by commanding topics stats
./bin/pulsar-admin topics stats topic1 { "msgRateIn" : 0.0, "msgThroughputIn" : 0.0, "msgRateOut" : 0.0, "msgThroughputOut" : 0.0, "averageMsgSize" : 0.0, "storageSize" : 0, "publishers" : [ ], "subscriptions" : { "sub" : { "msgRateOut" : 0.0, "msgThroughputOut" : 0.0, "msgRateRedeliver" : 0.0, "msgBacklog" : 0, "blockedSubscriptionOnUnackedMsgs" : false, "unackedMessages" : 0, "type" : "Failover", "activeConsumerName" : "383dc", "msgRateExpired" : 0.0, "consumers" : [ { "msgRateOut" : 0.0, "msgThroughputOut" : 0.0, "msgRateRedeliver" : 0.0, "consumerName" : "383dc", "availablePermits" : 1000, "unackedMessages" : 0, "blockedConsumerOnUnackedMsgs" : false, "metadata" : { }, "address" : "/127.0.0.1:50014", "connectedSince" : "2019-05-18T11:30:34.161Z", "clientVersion" : "2.3.1" }, { "msgRateOut" : 0.0, "msgThroughputOut" : 0.0, "msgRateRedeliver" : 0.0, "consumerName" : "51911", "availablePermits" : 1000, "unackedMessages" : 0, "blockedConsumerOnUnackedMsgs" : false, "metadata" : { }, "address" : "/127.0.0.1:50018", "connectedSince" : "2019-05-18T11:30:42.742Z", "clientVersion" : "2.3.1" } ] } }, "replication" : { }, "deduplicationStatus" : "Disabled" }
A subscription appears under Topic1, two Consumers. The current active ConsumerName is 383dc, which is the subscription in window1.This indicates that the Worker currently holding the subscription is a Leader.
When you turn off your subscription for window1, take a look.In window1, use CTRL + C to turn off the Consumer and return to window4 to view the subscription.
./bin/pulsar-admin topics stats topic1 { "msgRateIn" : 0.0, "msgThroughputIn" : 0.0, "msgRateOut" : 0.0, "msgThroughputOut" : 0.0, "averageMsgSize" : 0.0, "storageSize" : 0, "publishers" : [ ], "subscriptions" : { "sub" : { "msgRateOut" : 0.0, "msgThroughputOut" : 0.0, "msgRateRedeliver" : 0.0, "msgBacklog" : 0, "blockedSubscriptionOnUnackedMsgs" : false, "unackedMessages" : 0, "type" : "Failover", "activeConsumerName" : "51911", "msgRateExpired" : 0.0, "consumers" : [ { "msgRateOut" : 0.0, "msgThroughputOut" : 0.0, "msgRateRedeliver" : 0.0, "consumerName" : "51911", "availablePermits" : 1000, "unackedMessages" : 0, "blockedConsumerOnUnackedMsgs" : false, "metadata" : { }, "connectedSince" : "2019-05-18T11:30:42.742Z", "clientVersion" : "2.3.1", "address" : "/127.0.0.1:50018" } ] } }, "replication" : { }, "deduplicationStatus" : "Disabled" }
You can see that the subscription started in window2 was successfully activated and the current activeConsumerName is 51911, at which point the Worker holding the Consumer becomes the Leader.
summary
These are the election mechanisms that Connector uses to run Worker, and you can see how cleverly it uses Consumer's Failover mode to achieve a high-availability mechanism for Worker.