Open source distributed task scheduling system ScheduleMaster-v2.0 under. NET Core

Keywords: github Docker JSON less

It's almost 4 months since the first public introduction of this project in January, during which some repairs have been made and there is no major change in the whole. 2.0 is the first major version update since its release. It brings many new features and fixes some known bug s. Thank you for your comments on blogs, issues and QQ groups, as well as my predecessors.

In my opinion, this project does not use any advanced technology and architecture, and even some of the code written by myself is not satisfied and dare not show it to you. Compared with other big open source projects in the community, it is shameful. But in recent months, I have received the support and encouragement from some small partners, and also been included and recommended by some source code websites. I have the courage and confidence to continue to do it. Please insist.

The new version mainly adds HTTP task scheduling and node management functions. During the development process, part of the code has been refactored. Originally, it was intended to improve the unit test, but due to the time relationship, it can't be delayed.

For unfamiliar friends, please read the introduction below:


New version features

  • Improved mission lifecycle events

  • Task list supports node name search and display optimization

  • Support to configure HTTP tasks

  • Support node manual management

  • Support for specifying custom profiles in assembly tasks

  • Support long task cancellation

  • New system policy configuration

  • New dynamic parameter startup, more friendly to container deployment

  • Launch official documents

  • Added some use of demo

  • Fix some bug s

It's a little pity for this update to deal with HTTP task authorization authentication. The open API s I have learned have basically implemented a set of signature verification mechanism of their own, and the more perfect ones will use the OAuth2 set of things. The use of simple BASIC authentication or JWT is less, so I feel that it's not meaningful to access BASIC and JWT, but in the face of a variety of interface signatures or OAuth, I will temporarily At that time, there was no good solution, so we had to give a simple HTTP scheduling function first, and we will continue to think and explore in the future.

Now the core scheduling function of the system is almost the same. Next, we will focus on asynchronization, resource monitoring, performance monitoring and more free policy configuration. Some friends also suggest that you can support multiple database types, change to SPA applications, etc. at present, this demand is not very urgent, and you will put it later.


be all eagerness to see it

Due to the maintenance of official documents, there will be content updates in the middle and later stages of the blog, please github wiki Subject to.

1. The task initialization entry is provided:

public virtual void Initialize()
{
    ///TODO:
}

We can load our own configuration file here (support hot update):

    /// <summary>
    ///Show me how to set up my own profile and read configuration items
    /// </summary>
    public class CustomConfigFile : TaskBase
    {
        public override void Initialize()
        {
            //Specify profile
            base.SetConfigurationFile("myconfig.json");
        }

        public override void Run(TaskContext context)
        {
            context.WriteLog($"My configuration TestKey1: {Configuration["TestKey1"]}");
            context.WriteLog($"My configuration TestKey2: name->{Configuration["TestKey2:Name"]}  age->{Configuration["TestKey2:Age"]}");
        }
    }
// myconfig.json
{
  "TestKey1": "MyValue1",
  "TestKey2": {
    "Name": "hoho",
    "Age": 18
  }
}

Or do some DI binding:

    /// <summary>
    ///Show me how to use DI in business
    /// </summary>
    public class TaskUseDI : TaskBase
    {
        IServiceProvider serviceProvider;

        public override void Initialize()
        {
            var serviceCollection = new ServiceCollection();
            serviceCollection.AddTransient<ITemplateService1, TemplateService1>();
            serviceCollection.AddTransient<ITemplateService2, TemplateService2>();
            serviceProvider = serviceCollection.BuildServiceProvider();

            base.Initialize();
        }

        public override void Run(TaskContext context)
        {
            context.WriteLog($"I use DI Results obtained:{new TaskUseDITest(serviceProvider.GetService<ITemplateService1>()).GetResult()}");
        }
    }

The feature of the Initialize() method is that it will only be executed once when the task instance is created (you can also use the constructor, of course), along with the whole cycle until the task instance is destroyed (stopped).

2. Configuring HTTP tasks in the console

   [HttpPost]
   public string JsonPost(Blog model)
   {
       return "ok";
   }



3. Some new configuration items



4. Disable automatic registration mode, use master to actively create and take over worker s



5. Overwrite the configuration file with command line parameters

// Configuration files in docker image
{
  "NodeSetting": {
    "IdentityName": "worker1", 
    "Role": "worker", 
    "Protocol": "http", 
    "IP": "localhost", 
    "Port": 30001, 
    "Priority": 1
  }
}
[root@master1 ms_worker1]# docker run  -d -p 40001:80  --name="myworker1" ms_worker bash --identity=docker-worker1 --ip=192.168.174.5 --port=40001
5e446997d4a28b3c6ec0708a88d42a4d6baad1e5d5ae686d88c03e99c4e2003f

More waiting for you to explore slowly~


Future outlook

The project always adheres to the core concept of simple deployment + simple operation, and continuously updates iteratively, actively responds to the suggestions and needs of all parties, and contributes to the open source community~

Author's only open source address:

Documents (in progress):

Thank you for your support~

My blog will be synced to Tencent cloud + community, inviting you to join us: https://cloud.tencent.com/developer/support-plan?invite_code=oeevm2f6y7qt

Posted by srdva59 on Sun, 26 Apr 2020 19:06:26 -0700