Introduction and Use of protobuf for t-io Series Documents (3)

Keywords: Java JSON Google Python

  1. Why Write

    So far (2017-04-23), protobuf is a message format that I prefer, for the following reasons
    1. It's small. After coding the same object, its volume is about 1/8 of that of json.
    2. There's nothing to say about the speed of coding and decoding. Who doesn't like fast things?
    3. "Cross-language": We only need to write an xxx.proto, which can be used to generate the corresponding code in java, python, c++ and other languages.
    and t-io Protobuf is also used in some of the cases provided, so I'll sort out the relevant information about protobuf here.
  2. Get to know the proto source file first

    Here's one. t-io The source files used in the im examples provided are examples at the production level. Considering that this example is quite complete, I will not introduce it one by one. The name of the source file is chat.proto .
    syntax = "proto3";
    package org.tio.examples.im.common.packets;
    
    option java_package = "org.tio.examples.im.common.packets";  //Setting the package corresponding to java
    option java_multiple_files = true;                           //It is recommended to set it to true so that each object is placed in a file, otherwise all objects are in a java file.
    
    //type
    enum DeviceType {
    	DEVICE_TYPE_UNKNOW = 0;
    	DEVICE_TYPE_PC = 1;             //PC
    	DEVICE_TYPE_ANDROID = 2;        //Android
    	DEVICE_TYPE_IOS = 3;            //IOS
    }
    
    enum Command {
    	COMMAND_UNKNOW = 0;
    	COMMAND_HANDSHAKE_REQ = 1; //Handshake request, web socket handshake request with http
    	COMMAND_HANDSHAKE_RESP = 2; //Handshake response, web socket handshake response with http
    	COMMAND_AUTH_REQ = 3; //authentication request
    	COMMAND_AUTH_RESP = 4; // Authentication response
    	COMMAND_JOIN_GROUP_REQ = 5; //Apply for membership in a group
    	COMMAND_JOIN_GROUP_RESP = 6; //Application for Entry Group Response
    	COMMAND_JOIN_GROUP_NOTIFY_RESP = 7; //Room Entry Notice
    	COMMAND_CHAT_REQ = 8; //Chat request
    	COMMAND_CHAT_RESP = 9; //Chat response
    	COMMAND_START_SHOW_REQ = 10; //Broadcast Request
    	COMMAND_START_SHOW_RESP = 11; //Start-up response
    	COMMAND_END_SHOW_REQ = 12; //Stop broadcasting request
    	COMMAND_END_SHOW_NOTIFY_RESP = 13; //Discontinuation notice
    	COMMAND_HEARTBEAT_REQ = 14; //Heart beat request
    	COMMAND_CLOSE_REQ = 15; //Close the request
    }
    
    /**
     * authentication request
     */
    message AuthReqBody {
    	int64 time = 1;//Message delivery time
    	string deviceId = 2;// Device ID (mobile device id)
    	string token = 3;
    	DeviceType deviceType = 4;//clienttype client type 1-pc 2-android 3-ios
    	string cid = 5;//Channel number
    	string appVersion = 6;//app version number
    	string deviceInfo = 7;//Mobile phone model information, such as Huawei model
    	int64 seq = 8;//serial number
     	string sign = 9;//autograph
    }
    
    /**
     * Authentication response
     */
    message AuthRespBody {
    	int64 time = 1;//Message delivery time
    }
    
    /**
     * Join Group Request
     */
    message JoinReqBody {
    	int64 time = 1;//Message delivery time
    	string group = 2;
    }
    
    
    /**
     * Results of application for membership in a group
     */
    enum JoinGroupResult {
    	JOIN_GROUP_RESULT_UNKNOW = 0;//No entry is allowed for other reasons
    	JOIN_GROUP_RESULT_OK = 1;//Entry allowed
    	JOIN_GROUP_RESULT_NOT_EXIST = 2;//Group does not exist
    	JOIN_GROUP_RESULT_GROUP_FULL = 3;//Group full
    	JOIN_GROUP_RESULT_IN_BACKLIST = 4;//On the blacklist
    	JOIN_GROUP_RESULT_TAKEOUTED = 5;//Being kicked
    }
    
    /**
     * Join group response
     */
    message JoinRespBody {
    	int64 time = 1;//Message delivery time
    	JoinGroupResult result = 2;
    	string group = 3;
    }
    
    /**
     * Chat type
     */
    enum ChatType {
    	CHAT_TYPE_UNKNOW = 0;//Unknown
    	CHAT_TYPE_PUBLIC = 1;//Public chat
    	CHAT_TYPE_PRIVATE = 2;//Private chat
    }
    /**
     * Chat request
     */
    message ChatReqBody {
    	int64 time = 1;//Message delivery time
    	ChatType type = 2; //Chat type
    	string text = 3; //Chat content
    	string group = 4; //Target group id
    	int32 toId = 5; //The target user id,
    	string toNick = 6; //Target user nick
    }
    
    /**
     * Chat response
     */
    message ChatRespBody {
    	int64 time = 1;//Message delivery time
    	ChatType type = 2; //Chat type
    	string text = 3; //Chat content
    	int32 fromId = 4; //User id for sending chat messages
    	string fromNick = 5; //nick, the user who sends chat messages
    	int32 toId = 6; //Target user id
    	string toNick = 7; //Target user nick
    	string group = 8; //Target group id
    }
    
    
    /**
     * Start Live Request
     */
    message BeginToLiveReqBody {
    	int64 time = 1;//Message delivery time
    }
    
    /**
     * Start Live Response
     */
    message BeginToLiveRespBody {
    	int64 time = 1;//Message delivery time
    	int64 liveid = 2;       //id of this live broadcast
    	string rtmppublishurl = 3;  //rtmp push-flow address
    	string rtmpliveurl = 4;  //rtmp playback address
    }
    
    /**
     * End Live Request
     */
    message EndLiveReqBody {
    	int64 time = 1;//Message delivery time
    }
    
    /**
     * End Live Response
     */
    message EndLiveRespBody {
    	int64 time = 1;//Message delivery time
    	int64 liveid = 2;       //id of this live broadcast
    }
    
    

     

  3. Use tools to turn proto source files into the code we need

    1. Download and install tools

      Google provides the tool proto C. exe, which can generate proto source files corresponding to java, c++, python and other languages. https://github.com/google/protobufo You can find it here. If you can't find it here, for your convenience, the author has uploaded it to the t-io project, please click https://git.oschina.net/tywo45/t-io Then find the docs/tools directory, which contains the protoc. exe file. After downloading, add the protoc.exe directory to your path.
    2. Executing commands to generate code

      md .\src\c
      md .\src\java
      
      protoc.exe --cpp_out=.\src\c chat.proto
      protoc.exe --java_out=.\src\java chat.proto
      t-io Off-the-shelf scripts are provided, located in https://git.oschina.net/tywo45/t-io In the "src/example/im/common/src/main/protobuf" directory, you can run the generate.bat script directly to see what the generated code looks like, so as to form a perceptual understanding. However, we generally don't care about the details of the generated code, we only focus on a few methods, see the following coding examples and decoding examples.
    3. Copy the generated code to your project directory

      No, please duplicate it yourself.
  4. Application of protobuf in t-io

    1. Introducing related jar

      To use protobuf in java, we first need to introduce the relevant jar. The pom coordinates of maven are as follows
      <dependency>
      	<groupId>com.google.protobuf</groupId>
      	<artifactId>protobuf-java</artifactId>
      	<version>3.2.0</version>
      </dependency>
    2. Coding examples

      Coding is converted from java to byte [], as shown in the following example
      ChatRespBody.Builder builder = ChatRespBody.newBuilder();
      builder.setType(chatReqBody.getType());
      builder.setText(chatReqBody.getText());
      builder.setFromId(fromId);
      builder.setFromNick(fromNick);
      builder.setToId(toId);
      builder.setToNick(toNick);
      builder.setGroup(toGroup);
      builder.setTime(SystemTimer.currentTimeMillis());
      ChatRespBody chatRespBody = builder.build();
      byte[] bodybyte = chatRespBody.toByteArray();
      The code is in: org.tio.examples.im.server.handler.ChatReqHandler.handler(ImPacket, ChannelContext)
    3. Decoding example

      Decoding is converted from byte [] to java objects, as shown in the following example
      ChatReqBody chatReqBody = ChatReqBody.parseFrom(packet.getBody());
      The code is in: org.tio.examples.im.server.handler.ChatReqHandler.handler(ImPacket, ChannelContext)
  5. Expanding knowledge

    1. protobuf and json transform each other

      Specific reference https://github.com/bivas/protobuf-java-format
      1. Introducing jar
        <dependency>
        	<groupId>com.googlecode.protobuf-java-format</groupId>
        	<artifactId>protobuf-java-format</artifactId>
        	<version>1.4</version>
        </dependency>
      2. From protobuf to json
        Message someProto = SomeProto.getDefaultInstance();
        JsonFormat jsonFormat = new JsonFormat();
        String asJson = jsonFormat.printToString(someProto);
      3. From json to protobuf
        Message.Builder builder = SomeProto.newBuilder();
        String asJson = "{\"name\":\"tan\"}";
        JsonFormat jsonFormat = new JsonFormat();
        jsonFormat.merge(asJson, builder);

Posted by nathanr on Sat, 06 Jul 2019 11:12:56 -0700