[DEVOPS] realize real-time update of application interface documents

Keywords: DevOps word swagger2

With the help of maven plugin + custom service Swagger2Word service, the real-time update and online display of RESTful API documents meeting the requirements of custom format are realized. (Note: the API interface documents here are independent of the application. Unlike Swagger, you must start the application to see the corresponding documents.)

1. Preface

Through the automatic real-time online update of documents, low-level manual errors are eliminated, the rapid discovery of introduced errors is realized, the communication friction cost of both sides is reduced, and the development efficiency is accelerated.

2. Realization

We need to use three Maven plugins and a custom service.

2.1 generation of swagger2.0 format specification file

In this step, we need to use Swagger Maven Plugin of Maven Plugin to generate JSON files conforming to Swagger 2.0 format specification based on Swagger annotations in the project source code in Maven's Compile phase. (for specific requirements of Swagger2.0 format specification, see Swagger 2.0 document format specification )

The biggest difficulty in this step is that the configuration of different versions of swagger Maven plugin is slightly different. Therefore, the author directly gives a clear version number here (in fact, the parent dependency of SpringBoot sets the default version number). See the source code below for other details.

<!--	Swagger Export annotations as Word Document start		-->
<plugin>
    <groupId>com.github.kongchen</groupId>
    <artifactId>swagger-maven-plugin</artifactId>
    <version>3.1.8</version>
    <configuration>
        <apiSources>
            <apiSource>
               <!-- Tell the plugin your project is a JAX-RS(false) or a SpringMvc(true) project-->
                <springmvc>true</springmvc>
                <locations>
                    <!-- required-->
                    <!--   Classes containing Swagger's annotation @Api,
                    or packages containing those classes can be configured here.
                     Each item must be located inside a tag-->
                    <location>org.xx.xx.xx.controller</location>
                </locations>
                <schemes>
                    <scheme>http</scheme>
                    <scheme>https</scheme>
                </schemes>
                <host></host>
                <basePath></basePath>
                <!-- <typesToSkip></typesToSkip>  -->
                <info>
                    <!--required-->
                    <title>title</title>
                    <version>v1</version>
                    <description>description</description>
                    <termsOfService>
                        http://www.github.com/kongchen/swagger-maven-plugin
                    </termsOfService>
                    <contact>
                        <email>x</email>
                        <name>x</name>
                        <url>x</url>
                    </contact>
                    <license>
                        <url>http://www.apache.org/licenses/LICENSE-2.0.html</url>
                        <name>Apache 2.0</name>
                    </license>
                </info>
                <outputFormats>json</outputFormats>
                <!--Based on the project root directory -->
                <swaggerDirectory>generated/swagger-ui</swaggerDirectory>
               	<!-- Here we use a custom template -->
                <!--  Support classpath or file absolute path here. 1) classpath e.g: "classpath:/markdown.hbs", "classpath:/templates/hello.html" 2) file e.g: "${basedir}/src/main/resources/markdown.hbs", "${basedir}/src/main/resources/template/hello.html" 
                            <templatePath>${basedir}/templates/strapdown.html.hbs</templatePath>
                            <outputPath>${basedir}/generated/document.html</outputPath> -->              
            </apiSource>
        </apiSources>
    </configuration>
    <executions>
        <execution>
            <phase>compile</phase>
            <goals>
                <goal>generate</goal>
            </goals>
        </execution>
    </executions>
</plugin>

2.2 generate Word document

With the help of our custom RESTFul service, convert the JSON file generated in the previous step into a Word / HTML file with custom content format (where the HTML format is used for online update). This step will use the exec Maven Plugin of Maven Plugin.

In this step, the author's initial idea was to implement a custom Maven Plugin, but at the beginning, he suddenly thought of why he didn't learn from the idea of microservice. Such advantages:

  1. It can realize the single service function, which makes maintenance and update convenient. In the past, we have been trying to build a unified platform to integrate the internal required functions into one application. In the end, there are often problems in service update and code synchronization. Moreover, due to different requirements, it is easy to conflict after the integration of various dependencies, which makes the startup and debugging of applications in the development environment They all tremble, which will dampen the enthusiasm of developers.
  2. Improve the sense of ownership of relevant personnel and ensure the response speed and quality of requirements. In the past, in the form of large-scale application services, the writers of relevant sub services are easy to show a laissez faire attitude. After the separation of micro services, the improvement of autonomy can significantly improve the enthusiasm of personnel. (in addition to interface specifications, we will not limit the language and third-party dependency types)
  3. Accumulate practical experience related to microservices. As a business software company with extreme lack of infrastructure and strong demand for independent deployment, rashly pursuing the fashion of microservices will only lead to the suffering of all parties. By realizing the microservicing of internal services, we can feel and accumulate relevant experience of microservices in practice and explore microservices suitable for our company's business characteristics Service practical technical route.

The gossip is far away. Let's pull it back. Here are the relevant configurations of exec Maven plugin:

<!--	Swagger Export annotations as Word file 		-->
<plugin>
    <groupId>org.codehaus.mojo</groupId>
    <artifactId>exec-maven-plugin</artifactId>
    <version>3.0.0</version>
    <executions>
        <execution>
            <id>generateSwaggerWordFile</id>
            <phase>pre-package</phase>
            <goals>
                <goal>java</goal>
            </goals>
        </execution>
    </executions>
    <configuration>
        <mainClass>
            org.xx.xx.MavenExecPluginGenerateSwaggerWordFile
        </mainClass>
        <cleanupDaemonThreads>false</cleanupDaemonThreads>
        <!-- -->
        <addResourcesToClasspath>
            true
        </addResourcesToClasspath>
        <addOutputToClasspath>true</addOutputToClasspath>

        <arguments>
            <!-- // Command line arguments for the program / / -- >
            <!-- Build directory, default to target -->
            <argument>${project.build.directory}</argument>
            <argument>${session.executionRootDirectory}</argument>
            <!-- -->
            <argument>${project.groupId}</argument>
            <argument>${project.artifactId}</argument>
            <argument>${project.version}</argument>
            <argument>${project.packaging}</argument>
            <!-- swagger Document generation related -->
            <argument>generated/swagger-ui/swagger.json</argument>
            <argument>http://xxx.xx.x.x:9527/strToWord</argument>
            <!-- This is classpath Property whose value is<classpath/> -->
            <argument>-classpath</argument>
        </arguments>
    </configuration>
</plugin>

Related Java code:

	// ====Mavenexecplugingenenerateswaggerwordfile.java core code
	public static void main(String[] args) {
		if (args.length < 5) {
			output("Missing required parameters, Do not execute MavenExecPluginGenerateMetaFile.java");
			return;
		}

		// debugInputParams(args);

		MavenProjectEntity entity = readArgs(args);

		final String swaggerJsonFileRelativePath = args[6];
		final String swagger2WordServerUrl = args[7];

		final String swaggerJsonContent = FileUtil.readString(
				FilenameUtil.concat(entity.getExecutionRootDirectory(), swaggerJsonFileRelativePath),
				CharsetUtil.CHARSET_UTF_8);

		final File wordFileGenerated = FileUtil
				.file(FilenameUtil.concat(entity.getExecutionRootDirectory(), swaggerJsonFileRelativePath + ".doc"));
				// Request remote RESTful services and convert JSON files
		HttpRequest.post(swagger2WordServerUrl).queryMap(Collections.singletonMap("jsonStr", swaggerJsonContent))//
				.connectTimeout(Duration.ofSeconds(6000))// 
				.readTimeout(Duration.ofSeconds(6000))//
				.writeTimeout(Duration.ofSeconds(6000))//
				.execute() //
				.onFailed((request, ex) -> Console.error("exception occurred:" + ex))//
				.onSuccess(s -> s.rawBody(t -> FileUtil.writeFromStream(t.byteStream(), wordFileGenerated)));

		// Generate an html file at the same time
		FileUtil.copy(wordFileGenerated, FileUtil.file(FilenameUtil.concat(entity.getExecutionRootDirectory(),
				FilenameUtil.getFullPath(swaggerJsonFileRelativePath), "xxx-yyy.html")), true);
		output("Interface word Document generation completed! Path is: " + wordFileGenerated.getAbsolutePath());

	}

	static MavenProjectEntity readArgs(String[] args) {
		MavenProjectEntity entity = new MavenProjectEntity();
		entity.setProjectBuildDirectory(args[0]);
		entity.setExecutionRootDirectory(args[1]);
		entity.setGroupId(args[2]);
		entity.setArtifactId(args[3]);
		entity.setVersion(args[4]);
		entity.setPacking(args[5]);
		return entity;
	}

2.3 uploading documents

Finally, only the generated interface document in HTML format is pushed to the specified directory of the deployed file server. In this step, we need to use the Dragon Maven Plugin of Maven Plugin.

The relevant source code is as follows:

<!--Upload to online file server-->
<!-- https://www.cnblogs.com/tqyysm/articles/9815092.html -->
<plugin>
	<groupId>org.codehaus.mojo</groupId>
	<artifactId>wagon-maven-plugin</artifactId>
	<dependencies>
		<dependency>
			<groupId>com.jcraft</groupId>
			<artifactId>jsch</artifactId>
			<version>0.1.54</version>
		</dependency>
		<dependency>
			<groupId>org.bouncycastle</groupId>
			<artifactId>bcprov-jdk16</artifactId>
			<version>1.46</version>
		</dependency>
	</dependencies>
	<configuration>
		<serverId>txYunServer</serverId>
		<!-- Files to deploy -->
		<fromFile>generated/swagger-ui/xxx-yyy.html</fromFile>
		<!-- Deployment Directory User: password@ip+Deployment address: Port root:password@ -->
		<url>scp://xxx.yy.zzz.24/usr/local/docker-nginx/html</url>					
		<commands>						
			 <!-- <command>pkill java</command>  -->  
			 <!-- Updates the document in days -->
			<command><![CDATA[mv /usr/local/docker-nginx/html/kanq-authcenter.html /usr/local/docker-nginx/html/kanq-authcenter-`date +%Y-%m-%d`.html]]></command>
		</commands>
		<!-- Displays the output of the run command  -->
		<displayCommandOutputs>true</displayCommandOutputs>
	</configuration>
</plugin>

2.4 complete operation

After the above configuration is completed, the next thing to do is to type the following commands on the command line and press enter:

cd {Project root directory}
# Generate swagger2.0 JSON file
mvn swagger:generate
# Generate word/html custom content format document
mvn exec:java
# Push html formatted documents to the static file server
mvn wagon:upload-single mvn wagon:sshexec

###### Assemble the above commands
mvn swagger:generate exec:java wagon:upload-single mvn wagon:sshexec

3. Supplement - build a static file server

In order to send Buddha to the west, the following also gives the relevant commands and configurations of the author in using docker to build nginx static file server.

##### nginx configuration files to be prepared on the host
# /Add the following files in the usr / local / docker nginx directory (see the end of this section for the contents):
nginx.conf   
conf.d/default.conf

docker run --name nginx_container -p 9527:80 -d -e TZ="Asia/Shanghai" -v /etc/localtime:/etc/localtime:ro -v /usr/local/docker-nginx/html:/usr/share/nginx/html -v /usr/local/docker-nginx/nginx.conf:/etc/nginx/nginx.conf:ro -v /usr/local/docker-nginx/conf.d/:/etc/nginx/conf.d/  -v /usr/local/docker-nginx/logs:/var/log/nginx nginx

###################################### default.conf content
server {
    listen       80;
    listen  [::]:80;
    server_name  localhost;

    #access_log  /var/log/nginx/host.access.log  main;

    location / {
		autoindex on;
		autoindex_exact_size off;    #Set the file size displayed in MB, GB, etc
		autoindex_localtime on;    #Set the time attribute of the displayed directory or file

        alias /usr/share/nginx/html/;
    }

    #error_page  404              /404.html;

    # redirect server error pages to the static page /50x.html
    #
    error_page   500 502 503 504  /50x.html;
    location = /50x.html {
        root   /usr/share/nginx/html;
    }

}

###################################### nginx.conf content
user  nginx;
worker_processes  auto;

error_log  /var/log/nginx/error.log notice;
pid        /var/run/nginx.pid;


events {
    worker_connections  1024;
}


http {
    include       /etc/nginx/mime.types;
    default_type  application/octet-stream;

    log_format  main  '$remote_addr - $remote_user [$time_local] "$request" '
                      '$status $body_bytes_sent "$http_referer" '
                      '"$http_user_agent" "$http_x_forwarded_for"';

    access_log  /var/log/nginx/access.log  main;

    sendfile        on;
    #tcp_nopush     on;

    keepalive_timeout  65;

    #gzip  on;

    include /etc/nginx/conf.d/*.conf;
}



4. Supplement - integrated into DEVOPS process

In fact, the idea has been given in Section 2.4 above. Where it is embedded depends on the automation progress and process characteristics of the reader's company.

5. Wordy words

Through automatic document generation, the implementation of regular review mechanism in the R & D process is promoted. The inspector can check the basic situation of the application - whether the annotation is complete, whether the design is reasonable / violated, etc. without starting the relevant application. (in fact, the lack of automation may lead to very troublesome system startup) , regular inspection is far better than the final one-time test and inspection after completing the first service, and make efforts at ordinary times.

6. Reference

  1. GitHub - Swagger2Word We convert the Swagger annotation to Word document based on this open source project.

Posted by Flyier on Sat, 23 Oct 2021 06:38:14 -0700