Dapr publish / subscribe

Keywords: Dapr

preface

 Previous article This article will learn about the state management of Dapr, and this article will continue to learn about the subscription / publication building blocks.

1, Definition:

The concept of publish and subscribe comes from the design idea of event driven architecture (EDA), which is a main way to decouple programs (applications and services). Asynchronous calls between services can also be realized through the idea of publish and subscribe. Most distributed applications rely on this publish subscribe decoupling model.

 

Steps:

  1. The publisher sends the message to the message broker.
  2. The subscriber will bind to the subscription on the message broker.
  3. The message broker forwards a copy of the message to the interested subscription.
  4. Subscribers use messages from their subscriptions.

However, there are subtle differences between different message oriented middleware. Projects using different products need to implement different implementation classes. Although it is a wise decision, the abstraction and its basic implementation must be written and maintained. This method requires complex, repetitive, and error prone custom code.

In order to solve this problem, Dapr provides out of the box messaging abstraction and implementation, which are encapsulated in Dapr building blocks. The business system only needs to call to implement subscription and publication according to the requirements of Dapr.

2, Working principle:

The Dapr publish & subscribe building block provides a platform independent API framework to send and receive messages. Your service publishes messages to a named topic. The service subscribes to a topic to use messages.

The service calls the pub/sub API on the Dapr Sidecar. Sidecar then calls a predefined Dapr pub/sub component to encapsulate a specific message broker product. The following figure shows the Dapr publish / subscribe messaging stack.

 

3, Function:

  • Publish / subscribe API

The Dapr publish & subscribe building block provides a platform independent API framework to send and receive messages.

The service publishes the message to the specified topic, and the business service subscribes to the topic to use the message.

The service calls the pub/sub API on the Dapr sidecar. Sidecar then calls the predefined Dapr pub/sub component.

Any programming platform can use the Dapr native API to call building blocks via HTTP or gRPC. To publish a message, make the following API calls:

http://localhost:<dapr-port>/v1.0/publish/<pub-sub-name>/<topic>

There are several Dapr specific URL segments in the above call:

    • <dapr-port>   Provide the port number on which Dapr sidecar listens.

    • <pub-sub-name>   Provide the name of the selected Dapr pub/sub component.

    • <topic>   Provide the name of the topic to which the message is published.

  • message format

To enable message routing and provide additional context for each message, Dapr uses   CloudEvents 1.0 specification   As its message format. Any information sent using the Dapr application will be automatically wrapped in the Cloud Events envelope, datacontenttype   Attribute use   Content-Type   Header value.

Dapr implements the following Cloud Events fields:

    • id 
    • source
    • specversion
    • type
    • datacontenttype   (optional)

The following example shows the XML content serialized as JSON in CloudEvent v1.0:

{
    "specversion" : "1.0",
    "type" : "xml.message",
    "source" : "https://example.com/message",
    "subject" : "Test XML Message",
    "id" : "id-1234-5678-9101",
    "time" : "2020-09-23T06:23:21Z",
    "datacontenttype" : "text/xml",
    "data" : "<note><to>User1</to><from>user2</from><message>hi</message></note>"
}
  • Subscription message

Dapr applications can subscribe to published topics. Dapr allows your application to subscribe to topics in two ways:

Declarative: defined in an external file:

apiVersion: dapr.io/v1alpha1
kind: Subscription
metadata:
  name: myevent-subscription
spec:
  topic: test_topic  //theme
  route: /TestPubSub //route
  pubsubname: pubsub //name
scopes:
- frontend       //Enable subscriptions for this app

The example above shows   test_topic topic event subscription, using component   pubsub.

    • route   Tell Dapr to send all subject messages to the in the application  / TestPubSub   Endpoint.

    • scopes   Enable subscription for frontend app

Programming mode: subscription is defined in user code

  • message passing

Dapr guarantees messaging at least once semantics. This means that when an application publishes a message to a topic using the publish / subscribe API, dapr ensures that the message is delivered to each subscriber at least once

  • Consumer groups and competitive models

Multiple consumption groups and multiple application instances use one consumption group, which will be automatically processed by Dapr. When multiple instances of the same application (with the same ID) subscribe to a topic, Dapr delivers each message to only one instance of the application.

   

Similarly, if two different applications (different IDS) subscribe to the same topic, Dapr delivers each message to only one instance of each application.

  •   Topic scope:

By default, all topics that support the Dapr publish / subscribe component (for example, Kafka, Redis, RabbitMQ) can be used to configure each application of the component. To limit which application can publish or subscribe to topic, Dapr provides topic scope qualification. This enables you to let your application decide which topics to publish and which topics to subscribe to.

Scope qualification of pub/sub topic

Define publish / subscribe ranges for each pub/sub component. You may have one named   pubsub   The pub/sub component of, which has a set of range settings and another   pubsub2   There is another set of range settings.

To use this topic scope, you can set three metadata properties of a pub/sub component:

    • spec.metadata.publishingScopes
      • Semicolon separated application list & comma separated topic list allows the app to publish information to the topic list
      • If in   publishingScopes   If nothing is specified in (the default behavior), all applications can be published to all topics
      • To reject the application from publishing information to any topic, leave the topic list blank (app1=;app2=topic2)
      • For example,   app1=topic1;app2=topic2,topic3;app3=   App1 is allowed to publish information to topic1, app2 is allowed to publish information to Topic2 and Topic3, and app3 is not allowed to publish information to any topic.
    • spec.metadata.subscriptionScopes
      • Semicolon separated application list & comma separated topic list allows the app to subscribe to the topic list
      • If in   subscriptionScopes   If nothing is specified in (the default behavior), all applications can subscribe to all topics
      • For example,   app1=topic1;app2=topic2,topic3   App1 is allowed to subscribe to topic1, and app2 can subscribe to Topic2 and Topic3
    • spec.metadata.allowedTopics
      • A comma separated list of allowed topics for all applications.
      • If not set   allowedTopics   (default behavior), then all topics are valid.   subscriptionScopes   and   publishingScopes   If it exists, it still takes effect.
      • publishingScopes   or   subscriptionScopes   Can be used with   allowedTopics   To add restricted granularity
  • Message lifetime:

Dapr can set the timeout on a per message basis. Indicates that if the message is not read from the Pub/Sub component, the message will be discarded. This is to prevent the accumulation of unread messages. Messages that exceed the configured TTL in the queue can be said to be hung.   

4, . NET Core application

1. Set the Pub/Sub component:

Redis Staram is installed on this computer by default. Open% UserProfile%\.dapr\components\pubsub.yaml on Windows   Component files to verify:

apiVersion: dapr.io/v1alpha1
kind: Component
metadata:
  name: pubsub
spec:
  type: pubsub.redis
  metadata:
  - name: redisHost
    value: localhost:6379
  - name: redisPassword
    value: ""

2. Realize publish / subscribe function:

Add controller (PubSubController)

[Route("api/[controller]")]
[ApiController]
public class PubSubController : ControllerBase
{
    private readonly ILogger<PubSubController> _logger;
    private readonly DaprClient _daprClient;
    public PubSubController(ILogger<PubSubController> logger, DaprClient daprClient)
    {
        _logger = logger;
        _daprClient = daprClient;
    }

    /// <summary>
    /// Release news
    /// </summary>
    /// <returns></returns>
    [HttpGet("pub")]
    public async Task<ActionResult> PubAsync()
    {
        var data = new WeatherForecast() { Summary = "city", Date = DateTime.Now };
        await _daprClient.PublishEventAsync<WeatherForecast>("pubsub", "test_topic", data);
        return Ok();
    }

    /// <summary>
    /// Consumption news
    /// </summary>
    /// <returns></returns>
    [Topic("pubsub", "test_topic")]
    [HttpPost("sub")]
    public async Task<ActionResult> Sub()
    {
        Stream stream = Request.Body;
        byte[] buffer = new byte[Request.ContentLength.Value];
        stream.Position = 0L;
        await stream.ReadAsync(buffer, 0, buffer.Length);
        string content = Encoding.UTF8.GetString(buffer);
        _logger.LogInformation("testsub" + content);
        return Ok(content);
    }
}

Adjustment in Startup.cs:

public class Startup
{
    public Startup(IConfiguration configuration)
    {
        Configuration = configuration;
    }
    public IConfiguration Configuration { get; }
    // This method gets called by the runtime. Use this method to add services to the container.
    public void ConfigureServices(IServiceCollection services)
    {
        //injection Dapr
        services.AddControllers().AddDapr();
    }
    // This method gets called by the runtime. Use this method to configure the HTTP request pipeline.
    public void Configure(IApplicationBuilder app, IWebHostEnvironment env)
    {
        // use CoudEvent
        app.UseCloudEvents();
        if (env.IsDevelopment())
        {
            app.UseDeveloperExceptionPage();
        }
        app.Use((context, next) =>
        {
            context.Request.EnableBuffering();
            return next();
        });
        app.UseRouting();
        app.UseAuthorization();
        app.UseEndpoints(endpoints =>
        {
            endpoints.MapControllers()
            //Subscription processing
            endpoints.MapSubscribeHandler();;
        });
    }
}

3. dapr operation procedure:

dapr run --dapr-http-port 3501 --app-port 5001  --app-id frontend dotnet  .\FrontEnd.dll

4. Call and issue command:

http://127.0.0.1:3501/v1.0/invoke/frontend/method/api/pubsub/pub

5. Publish the message through Dapr cli:

dapr publish --publish-app-id frontend --pubsub pubsub --topic test_topic --data '{"date":"0001-01-01T00:00:00","temperatureC":0,"temperatureF":32,"summary":null}'

summary

The pub/sub pattern helps you separate services from distributed applications. The Dapr publish & subscribe building block simplifies implementing this behavior in your application.

With Dapr pub/sub, you can publish messages to specific sites   Theme. The building block also queries the service to determine (subscribe to) topics.

Dapr pub/sub can be used natively through HTTP or one of the language specific SDKs, such as the. NET SDK for dapr NET SDK is closely integrated with ASP.NET platform.

With Dapr, supported message broker products can be inserted into applications. The message broker can then be exchanged without changing the application's code.

Posted by DoctorWho on Sun, 07 Nov 2021 14:15:57 -0800