Java communicates with Python via gRPC

Keywords: Python Java grpc

Java end

gRPC transmits data through protobuf by default. Protobuf provides a flexible, efficient and automated mechanism for serializing structured data. Protobuf only needs to customize the required data format once, and then we can use the protobuf compiler to automatically generate the source code of various languages, which is convenient for us to read and write custom formatted data.

Installing plug-ins in idea

Configure maven files

It is mainly to add references and plug-ins of protobuf and grpc packages.

The references of protobuf and grpc are:

        <dependency>
            <groupId>com.google.protobuf</groupId>
            <artifactId>protobuf-java</artifactId>
            <version>3.5.1</version>
        </dependency>
        <dependency>
            <groupId>io.grpc</groupId>
            <artifactId>grpc-all</artifactId>
            <version>1.12.0</version>
        </dependency>

Generate protobuf and grpc interface plug-in configuration

            <plugin>
                <groupId>org.xolstice.maven.plugins</groupId>
                <artifactId>protobuf-maven-plugin</artifactId>
                <version>0.5.0</version>
                <configuration>
                    <protocArtifact>com.google.protobuf:protoc:3.0.0:exe:${os.detected.classifier}</protocArtifact>
                    <pluginId>grpc-java</pluginId>
                    <pluginArtifact>io.grpc:protoc-gen-grpc-java:1.0.0:exe:${os.detected.classifier}</pluginArtifact>
                </configuration>
                <executions>
                    <execution>
                        <goals>
                            <goal>compile</goal>
                            <goal>compile-custom</goal>
                        </goals>
                    </execution>
                </executions>
            </plugin>

Full profile

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>
    <parent>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-starter-parent</artifactId>
        <version>2.5.5</version>
        <relativePath/> <!-- lookup parent from repository -->
    </parent>
    <groupId>com.example</groupId>
    <artifactId>grpcdemo</artifactId>
    <version>0.0.1-SNAPSHOT</version>
    <name>grpcdemo</name>
    <description>Demo project for Spring Boot</description>
    <properties>
        <java.version>1.8</java.version>
    </properties>
    <dependencies>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-web</artifactId>
        </dependency>

        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-test</artifactId>
            <scope>test</scope>
        </dependency>

        <dependency>
            <groupId>com.google.protobuf</groupId>
            <artifactId>protobuf-java</artifactId>
            <version>3.5.1</version>
        </dependency>
        <dependency>
            <groupId>io.grpc</groupId>
            <artifactId>grpc-all</artifactId>
            <version>1.12.0</version>
        </dependency>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-autoconfigure</artifactId>
            <version>2.5.5</version>
        </dependency>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot</artifactId>
            <version>2.5.5</version>
        </dependency>

    </dependencies>

    <build>
        <extensions>
            <extension>
                <groupId>kr.motd.maven</groupId>
                <artifactId>os-maven-plugin</artifactId>
                <version>1.4.1.Final</version>
            </extension>
        </extensions>

        <plugins>
            <plugin>
                <groupId>org.xolstice.maven.plugins</groupId>
                <artifactId>protobuf-maven-plugin</artifactId>
                <version>0.5.0</version>
                <configuration>
                    <protocArtifact>com.google.protobuf:protoc:3.0.0:exe:${os.detected.classifier}</protocArtifact>
                    <pluginId>grpc-java</pluginId>
                    <pluginArtifact>io.grpc:protoc-gen-grpc-java:1.0.0:exe:${os.detected.classifier}</pluginArtifact>
                </configuration>
                <executions>
                    <execution>
                        <goals>
                            <goal>compile</goal>
                            <goal>compile-custom</goal>
                        </goals>
                    </execution>
                </executions>
            </plugin>
<!--            <plugin>-->
<!--                <groupId>org.springframework.boot</groupId>-->
<!--                <artifactId>spring-boot-maven-plugin</artifactId>-->
<!--            </plugin>-->
        </plugins>
    </build>

</project>

Define an interface service file (*. proto), which needs to be placed at the same level as main/java. You can create a folder.

  The following is the content of a sample file calc.proto

syntax = "proto3";

option java_multiple_files = false;
option java_package = "com.example.grpcdemo.grpc.calc";
option java_outer_classname = "PhmCalcProto";
// option objc_class_prefix = "Hello";

package com.example.grpcdemo.grpc.calc;

//Define service
service PhmCalcService {
  //Note: This is returns, not return
  rpc calc (PhmCalcRequest) returns (PhmCalcResponse) {
  }
}

//Define request data structure
message PhmCalcRequest {
  string calcName = 1;
  string calcCode = 2;
  string calcParams = 3;
}

//Define return data structure
message PhmCalcResponse {
  string status = 1;
  string code = 2;
  string message = 3;
  string data = 4;
}

Here, a service interface "PhmCalcService" and two transmission objects "PhmCalcRequest" and "PhmCalcResponse" are defined.

Generate the protobuf interface template file through maven's install

After clicking run, a target folder will be generated in the project (to be opened)  

  The target folder contains sample programs, interface templates, and java files.

Copy template code to project

 

To avoid class conflicts, you can delete or move the template "target" folder for backup.

Write client request test class

package com.example.grpcdemo;

import com.example.grpcdemo.grpc.calc.PhmCalcProto;
import com.example.grpcdemo.grpc.calc.PhmCalcServiceGrpc;
import io.grpc.ManagedChannel;
import io.grpc.ManagedChannelBuilder;

import java.util.concurrent.TimeUnit;

/**
 * @author Administrator
 */
public class PhmCalcTestClient {
    private final ManagedChannel channel;
    private final PhmCalcServiceGrpc.PhmCalcServiceBlockingStub blockingStub;
    private static final String GRPC_SERVER_HOST = "127.0.0.1";
    private static final int GRPC_SERVER_PORT = 50051;

    public PhmCalcTestClient(String host, int port) {
        // usePlaintext means plaintext transmission, otherwise SSL needs to be configured, and channel means communication channel
        channel = ManagedChannelBuilder.forAddress(host, port).usePlaintext(true).build();
        // Stub, which is used to call the interface method of the server.
        blockingStub = PhmCalcServiceGrpc.newBlockingStub(channel);
    }

    public void shutdown() throws InterruptedException {
        channel.shutdown().awaitTermination(5, TimeUnit.SECONDS);
    }

    public void testResult() {
        PhmCalcProto.PhmCalcRequest request = PhmCalcProto.PhmCalcRequest.newBuilder()
                .setCalcCode("code123")
                .setCalcName("name123")
                .setCalcParams("{\"p1\":\"p1Value\"}").build();

        System.out.println("request: ");
        System.out.println(request.toString());

        PhmCalcProto.PhmCalcResponse response = blockingStub.calc(request);

        System.out.println("response: ");
        System.out.println(response.toString());
    }

    public static void main(String[] args) {
        PhmCalcTestClient client = new PhmCalcTestClient(GRPC_SERVER_HOST, GRPC_SERVER_PORT);
        client.testResult();
    }
}

  Python end

Create a new Python code project. If it is not installed, install Python first.

The gRPC service template code on the python side is not generated in the IDE tool, but is completed using a tool plug-in of Python.

When installing python's grpc runtime, the command line is executed

pip install grpcio
  • Compile the proto file with protoc to generate the implementation of python language

Install the grpc tool and execute it from the command line

pip install grpcio-tools

Edit the protobuf file to generate the service template code. Copy the same protobuf file "calc.proto" defined in JAVA in the previous step to the Python project directory, locate the command line to the proto directory, and execute the command

python -m grpc_tools.protoc --python_out=. --grpc_python_out=. -I. calc.proto

After compilation, the code Calc is generated_ Pb2.py and calc_pb2_grpc.py file.

Write the server test class calcServer.py

 

from concurrent import futures
import time
import grpc
import calc_pb2
import calc_pb2_grpc

# Implement the PhmCalcServiceService defined in the proto file
class CalcServer(calc_pb2_grpc.PhmCalcServiceServicer):
    # Implement the rpc call defined in the proto file
    def calc(self, request, context):
        print("request: ")
        print(request)

        # The results are obtained through actual calculation.
        # ...
        response = calc_pb2.PhmCalcResponse(status="0", message="success", data="{\"p2\": \"p2Value\"}")

        print("response: ")
        print(response)

        return response

def serve():
    # Start rpc service
    server = grpc.server(futures.ThreadPoolExecutor(max_workers=10))
    calc_pb2_grpc.add_PhmCalcServiceServicer_to_server(CalcServer(), server)
    server.add_insecure_port('[::]:50051')
    server.start()
    try:
        while True:
            time.sleep(60*60*24) # one day in seconds
    except KeyboardInterrupt:
        server.stop(0)

if __name__ == '__main__':
    serve()

Posted by adrianpeyton on Mon, 27 Sep 2021 23:12:44 -0700