Go language, Protobuf speed entry!

Protobuf fer is the abbreviation of Protocol Buffers. It is an extensible data description language for serializing structured data independent of language and platform. As the description language of interface specification, protobuffer can be used as the basic tool for designing secure cross language PRC interface.

Basic grammar

hello.proto file

syntax = "proto3";

package main;

message String {
    string value = 1;
}
  1. The first line declares the use of proto3 syntax. Otherwise, the proto2 syntax is used by default, and v3 is recommended by the mainstream. This declaration must be the first non empty, non commented line of the file.
  2. The package instruction indicates that the current package is the main package. You can also customize the corresponding package path and name for different languages.
  3. Message keyword defines a String type message body, which corresponds to a String structure in the finally generated Go language code. The field of each message body contains three attributes: type, field name and field number. In the definition of message body, it cannot be repeated except for type. Here, there is only one value member of String type in String type. When encoding this member, 1 number is used instead of name.
  4. The most basic data unit in Protobuf is message, which is similar to the structure in Go language. Members of message or other basic data types can be nested in message.

About identification number

The field in the message body defines a unique numeric value. These numbers are used to identify each field in the binary format of the message and cannot be changed once they are used. Note: the identification number within [1,15] will occupy one byte during encoding. The identification number within [162047] occupies 2 bytes. Therefore, the identification number within [1,15] should be reserved for those frequently occurring message elements.

The smallest identification number can start from 1, up to 2 ^ 29 - 1, or 536870911. The identification numbers of [19000-19999] can not be used, which are reserved in the implementation of Protobuf protocol. If you have to use these reserved identification numbers in the. proto file, an alarm will be sent at compile time. Similarly, you cannot use any identifiers previously reserved.

Add comments

Add comments to the. proto file. You can use the C/C + + style / / and / *... * / syntax formats

Reserved fields

If and fields are deleted from the previously defined message, keep their field numbers and use the keyword reserved:

syntax "proto3";

message Stock {
    reserved 3, 4;
    // ...
}

You can also use the reserved keyword as a placeholder for fields that may be added in the future. You can use the to keyword to place consecutive field numbers.

syntax "proto3";

message Info {
    reserved 2, 9 to 11, 15;
    // ...
}

Generate corresponding Go code

The core tool set of Protobuf is developed in C + +, and the official protoc ol compiler does not support Go language. To generate the corresponding Go code based on the above hello.proto file, you need to install the corresponding plug-in.

  • Install the official protoc ol tool from https://github.com/google/protobuf/releases Download.
  • Install the code generation plug-in for Go language through the go get github.com/golang/protobuf/protoc-gen-go command.

Generate the corresponding Go code with the following command:

$ protoc --go_out=. hello.proto

  • go_ The out parameter tells the protocol compiler to load the corresponding protocol Gen go tool and put the generated code into the current directory. Finally, there is a list of protobuf files to process.
  • plugins=plugin1+plugin2: specify the list of sub plug-ins to be loaded. The proto file we defined involves RPC services, and RPC code will not be generated by default, so it needs to be in go_ The plugins parameter given in out is passed to protocol Gen go to tell the compiler to support RPC (the built-in grpc plug-in is specified here).

Basic data type

The data types generated by protobuf are not completely consistent with the original types. Here are some common type mappings:

Generated hello.pb.go file

The pb.go file is the corresponding Go code generated from the proto file, which will be referenced in practical application.

// Code generated by protoc-gen-go. DO NOT EDIT.
// source: hello.proto

package main

import (
 fmt "fmt"
 proto "github.com/golang/protobuf/proto"
 math "math"
)

// Reference imports to suppress errors if they are not otherwise used.
var _ = proto.Marshal
var _ = fmt.Errorf
var _ = math.Inf

// This is a compile-time assertion to ensure that this generated file
// is compatible with the proto package it is being compiled against.
// A compilation error at this line likely means your copy of the
// proto package needs to be updated.
const _ = proto.ProtoPackageIsVersion3 // please upgrade the proto package

type String struct {
 Value                *String  `protobuf:"bytes,1,opt,name=value,proto3" json:"value,omitempty"`
 XXX_NoUnkeyedLiteral struct{} `json:"-"`
 XXX_unrecognized     []byte   `json:"-"`
 XXX_sizecache        int32    `json:"-"`
}

func (m *String) Reset()         { *m = String{} }
func (m *String) String() string { return proto.CompactTextString(m) }
func (*String) ProtoMessage()    {}
func (*String) Descriptor() ([]byte, []int) {
 return fileDescriptor_61ef911816e0a8ce, []int{0}
}

func (m *String) XXX_Unmarshal(b []byte) error {
 return xxx_messageInfo_String.Unmarshal(m, b)
}
func (m *String) XXX_Marshal(b []byte, deterministic bool) ([]byte, error) {
 return xxx_messageInfo_String.Marshal(b, m, deterministic)
}
func (m *String) XXX_Merge(src proto.Message) {
 xxx_messageInfo_String.Merge(m, src)
}
func (m *String) XXX_Size() int {
 return xxx_messageInfo_String.Size(m)
}
func (m *String) XXX_DiscardUnknown() {
 xxx_messageInfo_String.DiscardUnknown(m)
}

var xxx_messageInfo_String proto.InternalMessageInfo

func (m *String) GetValue() *String {
 if m != nil {
  return m.Value
 }
 return nil
}

func init() {
 proto.RegisterType((*String)(nil), "main.String")
}

func init() { proto.RegisterFile("hello.proto", fileDescriptor_61ef911816e0a8ce) }

var fileDescriptor_61ef911816e0a8ce = []byte{
 // 84 bytes of a gzipped FileDescriptorProto
 0x1f, 0x8b, 0x08, 0x00, 0x00, 0x00, 0x00, 0x00, 0x02, 0xff, 0xe2, 0xe2, 0xce, 0x48, 0xcd, 0xc9,
 0xc9, 0xd7, 0x2b, 0x28, 0xca, 0x2f, 0xc9, 0x17, 0x62, 0xc9, 0x4d, 0xcc, 0xcc, 0x53, 0xd2, 0xe1,
 0x62, 0x0b, 0x2e, 0x29, 0xca, 0xcc, 0x4b, 0x17, 0x52, 0xe2, 0x62, 0x2d, 0x4b, 0xcc, 0x29, 0x4d,
 0x95, 0x60, 0x54, 0x60, 0xd4, 0xe0, 0x36, 0xe2, 0xd1, 0x03, 0xc9, 0xeb, 0x41, 0x24, 0x83, 0x20,
 0x52, 0x49, 0x6c, 0x60, 0xad, 0xc6, 0x80, 0x00, 0x00, 0x00, 0xff, 0xff, 0x76, 0x0c, 0x0e, 0x54,
 0x49, 0x00, 0x00, 0x00,
}
  • String type automatically generates a group of methods, in which the ProtoMessage method indicates that it is a method that implements the proto.Message interface. In addition, Protobuf also generates a Get method for each member, which can provide a convenient way to Get values, handle some null pointer values, and Reset the parameter through the Reset method.
  • For the initialization method of. pb.go file, pay attention to the relevant statements of filedescriptor. fileDescriptor_61ef911816e0a8ce represents a compiled proto file, which is the overall description of the proto file, including the proto file name, import content, package name, option settings, all defined message bodies (messages), all defined enums (enums), all defined services (services), and all defined methods (RPC methods) And so on.
  • Each Message Type contains a Descriptor method, which refers to the description of a message body definition. This method will find the location of its own Message Field in the fileDescriptor and then return.

Protobuf and RPC combination

Re implement the HelloService service based on the String type

package main

import (
 "log"
 "net"
 "net/rpc"
 "rpc/protoc"
)

// HelloService is rpc server obj
type HelloService struct{}

//The input and output parameters of the Hello method are represented by the String type defined by Protobuf.
//Because the new input parameter is a structure type, the pointer type is used as the input parameter, and the internal code of the function is adjusted accordingly.
func (p *HelloService) Hello(request *protoc.String, reply *protoc.String) error {
 reply.Value = "hello:" + request.GetValue()
 return nil
}

func main() {
 rpc.RegisterName("HelloService", new(HelloService))

 listener, err := net.Listen("tcp", ":1234")
 if err != nil {
  log.Fatal("ListenTCP error:", err)
 }

 conn, err := listener.Accept()
 if err != nil {
  log.Fatal("Accept error", err)
 }

 rpc.ServeConn(conn)
}

The following is the code client.go for the client to request the HelloService service:

package main

import (
 "fmt"
 "log"
 "net/rpc"
 "rpc/protoc"
)

func main() {
 client, err := rpc.Dial("tcp", "localhost:1234")
 if err != nil {
  log.Fatal("dialing err:", err)
 }

 var reply = &protoc.String{}
 var param = &protoc.String{
  Value: "hello wekenw",
 }

 err = client.Call("HelloService.Hello", &param, &reply)
 if err != nil {
  log.Fatal(err)
 }
 fmt.Println(reply)
}

Open the server and the client. The execution results of the client are as follows:

$ go run client.go
value:"hello:hello wekenw"

Pictures and some relevant technical knowledge points come from network search, infringement deletion!

reference material:

https://blog.csdn.net/qq_22660775/article/details/89163881

https://docs.microsoft.com/zh-cn/dotnet/architecture/grpc-for-wcf-developers/protobuf-reserved

https://golang2.eddycjy.com/posts/ch3/01-simple-grpc-protobuf/

Go Language Advanced Programming

Posted by foid025 on Thu, 11 Nov 2021 21:57:14 -0800

Hot Keywords

©2025 Programmer Group