0x00 introduction
DotBPE. RPC is an RPC framework based on dotnet core. Its father, DotBPE, aims to achieve an out-of-the-box micro-service framework, but it is almost meaningful, only in the stage of conception and attempt. But anyway, RPC is the foundation of micro-service, let's talk about the implementation of RPC first. The default implementation of DotBPE.RPC underlying communication is based on DotNetty This is a Netty implementation of C# developed by Microsoft Azure team. It's very cool. Of course, you can also replace it with other Socket communication components. DotBPE.RPC uses the default protocol name Amp, and Google's Proteobuf3 for encoding and decoding, but these default implementations are replaceable.
Source address: https://github.com/xuanye/dotbpe.git
<!--more-->
0x01 About Amp Protocol and Google Protobuf
Amp(A Message Protocol)
Amp(A Message Protocol), which is called a message protocol in Chinese, is a message protocol implemented by DotBPE.RPC by default. In practice, it is not necessary to know how messages are coded, decoded and transmitted, but it is helpful to understand the framework. The basic structure of the protocol is shown in the following figure:
0 1 2 3 4 5 6 7 8 9 10 11 12 13 <length>-14 +------------+----------+---------+------+-------+---------+------------+ | <ver/argc> | <length> | <seq> |<type>|<serId>| <msgid> | <data> | +------------+----------+---------+------+-------+---------+------------+
The default header length of Amp protocol is 14 bytes and does not contain extended header.
Number 0: ver/argc// is the version number, temporarily, the default is 0
Bit 1-4: length // is the total length of the package (including the length of the head)
Bit 5-8: sequence // is the message sequence number, which corresponds to the request < - > response
Number 9: type // Message type, with five present values, as follows:
> Request = 1, Response = 2, Notify = 3,NotFound = 4, ERROR = 5
Bits 10-11: serviceId//message ID ushort type
Bits 12-13: msgId//message ID ushort type
In Amp protocol, serviceId identifies a class of requests, similar to modules in applications, while msgId identifies specific methods in modules.
Then follow the actual data.
Google Protobuf
Google Protocol Buffer (hereinafter referred to as Protobuf) is a mixed language data standard within Google. At present, more than 48,162 message format definitions and more than 12,183. proto files are being used. They are used in RPC systems and persistent data storage systems.
Protocol Buffers is a portable and efficient structured data storage format, which can be used for structured data serialization, or serialization. It is very suitable for data storage or RPC data exchange format. It can be used for language-independent, platform-independent and extensible serialized structured data formats in communication protocols, data storage and other fields. API s in many languages are currently available, including C++, C#, GO, JAVA, PYTHON
My previous blog Using CSharp to write Google Protobuf plug-ins It has been introduced that if we write plug-ins, we can define proto files and generate the code we need.
In DotBPE.RPC, I use protobuf as the description file of the service, and generate the proxy classes of the server and client through a custom plug-in.
0x02 Quick Start
0. premise
Because DotBPE is based on dotnet core, you must have a dotnet core development environment locally.
Use github managed code, so you must have installed the git client
You need to generate template code through protoc, so you must have installed the google protobuf command line tool
1. Download the sample program
In order to explain our quick start program, you need a sample code that can run locally. Downloading the sample code I've written from github allows you to quickly build programs without some tedious but necessary steps.
>$ # Clone the repository to get the example code: >$ git clone https://github.com/xuanye/dotbpe-sample.git >$ cd dotbpe-sample
Use VS2017 or VSCode to open the downloaded code. The directory structure is as follows:
If you use VS2017, you can automatically help you back to the original. If you use VSCode, you need to run dotnet restore download dependency. After success, you can use dotnet build to compile and see the result: It looks perfect.
2. Running program
Run Server
>$ cd HelloDotBPE.Server >$ dotnet run
Run Client
>$ cd HelloDotBPE.Client >$ dotnet run
Congratulations! A Server/Client application has been run using DotBPE.RPC.
3. Let's take a look at the code.
3.1 Service Description File proto
First of all, the proto extension file in the DotBPE.RPC framework. All projects need this file. My article is about how to extend the proto. Blog With more detailed introduction, I won't repeat it here.
//dotbpe_option.proto file syntax = "proto3"; package dotbpe; option csharp_namespace = "DotBPE.ProtoBuf"; import "google/protobuf/descriptor.proto"; //Extended services extend google.protobuf.ServiceOptions { int32 service_id = 51001; bool disable_generic_service_client = 51003; //Prohibit client code generation bool disable_generic_service_server = 51004; //Prohibit the generation of server-side code } extend google.protobuf.MethodOptions { int32 message_id = 51002; } extend google.protobuf.FileOptions { bool disable_generic_services_client = 51003; //Prohibit client code generation bool disable_generic_services_server = 51004; //Prohibit the generation of server-side code bool generic_markdown_doc = 51005; //Document generation is not useful in this example bool generic_objectfactory = 51006; //Whether or not to generate object factories is not useful in this example }
The following service description file, greeter.proto, is the real example service description file: Simpler, define a Greeter Rpc service, and define a Hello method
//greeter.proto syntax = "proto3"; package dotbpe; option csharp_namespace = "HelloDotBPE.Common"; // Introducing extensions import public "dotbpe_option.proto"; // Define a service service Greeter { option (service_id)= 100 ;//Message ID, global must be unique // Sends a greeting rpc Hello (HelloRequest) returns (HelloResponse) { option (message_id)= 1 ;//Set the message ID, unique within the same service } } // The request message containing the user's name. message HelloRequest { string name = 1; } // The response message containing the greetings message HelloResponse { string message = 1; }
The template code is generated by protoc tool, and the code in the example is generated in HelloDotBPE. Common_g directory. Students who can run shell commands locally can go directly to the HelloDotBPE.Common_g directory. The dotbpe-samplescriptgenerate directory runs sh generate_hello.sh (cgywin is usually installed under windows). Students who cannot run can also run the command line directly under Hello DotBPE directory.
protoc -I=../protos --csharp_out=./HelloDotBPE.Common/_g/ --dotbpe_out=./HelloDotBPE.Common/_g/ ../protos/dotbpe_option.proto ../protos/greeter.proto --plugin=protoc-gen-dotbpe=../../tool/protoc_plugin/Protobuf.Gen.exe
Of course, I recommend that you install the following cgywin runtime environment to run some common commands on unix. At the same time, some scripts of the development environment can be shared when deployed to the formal environment.
3.2 Server Code
Service implementation:
// Service implementation code public class GreeterImpl : GreeterBase { public override Task<HelloResponse> HelloAsync(HelloRequest request) { // Return directly to Hello Name return Task.FromResult(new HelloResponse() { Message = "Hello " + request.Name }); } }
Server Startup Class
public class Startup : IStartup { public void Configure(IAppBuilder app, IHostingEnvironment env) { } public IServiceProvider ConfigureServices(IServiceCollection services) { services.AddDotBPE(); // Adding DotBPE.RPC Core Dependencies services.AddServiceActors<AmpMessage>(actors => { actors.Add<GreeterImpl>(); // Registration Service Implementation }); return services.BuildServiceProvider(); } }
Start the server
class Program { static void Main(string[] args) { Console.OutputEncoding = System.Text.Encoding.UTF8; //Output debug log in console DotBPE.Rpc.Environment.SetLogger(new DotBPE.Rpc.Logging.ConsoleLogger()); var host = new RpcHostBuilder() .UseServer("0.0.0.0:6201") //Binding Local Port 6201 .UseStartup<Startup>() .Build(); host.StartAsync().Wait(); Console.WriteLine("Press any key to quit!"); Console.ReadKey(); host.ShutdownAsync().Wait(); } }
3.3 Client Code
class Program { static void Main(string[] args) { Console.OutputEncoding = Encoding.UTF8; var client = AmpClient.Create("127.0.0.1:6201"); //Establishing Link Channels var greeter = new GreeterClient(client); //Client proxy class while (true) { Console.WriteLine("input your name and press enter:"); string name = Console.ReadLine(); if ("bye".Equals(name)) { break; } try { var request = new HelloRequest() { Name = name }; var result = greeter.HelloAsync(request).Result; Console.WriteLine($"---------------receive form server:{result.Message}-----------"); } catch (Exception ex) { Console.WriteLine("Errors occurred:" + ex.Message); } } Console.WriteLine($"---------------close connection-----------"); client.CloseAsync(); } }
0x03 Next Step
In the next article, I will describe in detail the main classes and invocation relationships in DotBPE.RPC, and how to use DotNetty to achieve RPC communication. In fact, I'm writing a more complex example. https://github.com/xuanye/PiggyMetrics.git， This was originally a sample program of spring cloud. I used DotBPE to transform and describe the application of DotBPE in real scenes with examples. Including service registration and discovery, inter-service invocation, disclosure of HttpApi, monitoring and inspection functions, and through practice to further improve the DotBPE. Preliminary functions have been implemented, but there is no time to write documentation. The implementation of this system will be described in detail later in this series.