Modification of spring cloud gateway server webexchange core method and request or response content
premise
- The Spring Cloud Gateway version used at the time of writing this article is the latest version Greenwich.SR1 at that time.
When using Spring Cloud Gateway, we noticed that filters (including GatewayFilter, GlobalFilter and filter chain GatewayFilterChain) all depend on ServerWebExchange:
public interface GlobalFilter {
Mono<Void> filter(ServerWebExchange exchange, GatewayFilterChain chain);
}
public interface GatewayFilter extends ShortcutConfigurable {
Mono<Void> filter(ServerWebExchange exchange, GatewayFilterChain chain);
}
public interface GatewayFilterChain {
Mono<Void> filter(ServerWebExchange exchange);
}
The design here is similar to the Filter in the Servlet. The current Filter can decide whether to execute the logic of the next Filter, which is determined by whether gateway filterchain#Filter () is called. Server web exchange is equivalent to the context of the current Request and Response. The ServerWebExchange instance not only stores the Request and Response objects, but also provides some extension methods. If you want to modify the Request parameters or Response parameters, you must deeply understand ServerWebExchange.
Understanding ServerWebExchange
First look at the notes of ServerWebExchange:
Contract for an HTTP request-response interaction. Provides access to the HTTP request and response and also exposes additional server-side processing related properties and features such as request attributes.
The translation is about:
Server web exchange is a contract for HTTP request response interaction. Provides access to HTTP requests and responses, and exposes additional server-side processing related properties and features, such as request properties.
In fact, ServerWebExchange is named service network switch, which stores important request response properties, request instances and response instances, which is a bit like the role of Context.
ServerWebExchange interface
All methods of ServerWebExchange interface:
public interface ServerWebExchange {
// The KEY of the log prefix attribute is org.springframework.web.server.ServerWebExchange.LOG_ID
// It can be understood as attributes.set("org.springframework.web.server.ServerWebExchange.LOG_ID", "specific value of log prefix");
// The function is to splice the prefix value of the KEY when printing the log. The default value is ""
String LOG_ID_ATTRIBUTE = ServerWebExchange.class.getName() + ".LOG_ID";
String getLogPrefix();
// Get ServerHttpRequest object
ServerHttpRequest getRequest();
// Get ServerHttpResponse object
ServerHttpResponse getResponse();
// Returns the request attribute of the current exchange, and the returned result is a variable Map
Map<String, Object> getAttributes();
// Get request properties according to KEY
@Nullable
default <T> T getAttribute(String name) {
return (T) getAttributes().get(name);
}
// Get the request attribute according to the KEY and make a non empty judgment
@SuppressWarnings("unchecked")
default <T> T getRequiredAttribute(String name) {
T value = getAttribute(name);
Assert.notNull(value, () -> "Required attribute '" + name + "' is missing");
return value;
}
// Get the request attribute according to the KEY, and you need to provide a default value
@SuppressWarnings("unchecked")
default <T> T getAttributeOrDefault(String name, T defaultValue) {
return (T) getAttributes().getOrDefault(name, defaultValue);
}
// Returns the currently requested network session
Mono<WebSession> getSession();
// Returns the currently requested authenticated user, if any
<T extends Principal> Mono<T> getPrincipal();
// Return the requested form data or an empty Map. This method will return a non empty Map only when the content type is application/x-www-form-urlencoded -- this is usually used for form data submission
Mono<MultiValueMap<String, String>> getFormData();
// Return the part data or an empty Map requested by multipart. This method will return a non empty Map only when the content type is multipart / form data -- this is usually used for file upload
Mono<MultiValueMap<String, Part>> getMultipartData();
// Returns the context of Spring
@Nullable
ApplicationContext getApplicationContext();
// These methods are related to the lastModified attribute
boolean isNotModified();
boolean checkNotModified(Instant lastModified);
boolean checkNotModified(String etag);
boolean checkNotModified(@Nullable String etag, Instant lastModified);
// URL conversion
String transformUrl(String url);
// URL translation mapping
void addUrlTransformer(Function<String, String> transformer);
// Note that the method name is change. This is a method to modify the properties of ServerWebExchange. It returns a Builder instance, which is the internal class of ServerWebExchange
default Builder mutate() {
return new DefaultServerWebExchangeBuilder(this);
}
interface Builder {
// Overwrite ServerHttpRequest
Builder request(Consumer<ServerHttpRequest.Builder> requestBuilderConsumer);
Builder request(ServerHttpRequest request);
// Override ServerHttpResponse
Builder response(ServerHttpResponse response);
// Overwrite the currently requested authenticated user
Builder principal(Mono<Principal> principalMono);
// Build a new ServerWebExchange instance
ServerWebExchange build();
}
}
Note the ServerWebExchange#mutate() method. The ServerWebExchange instance can be understood as an immutable instance. If we want to modify it, we need to generate a new instance through the mutate() method, for example:
public class CustomGlobalFilter implements GlobalFilter {
@Override
public Mono<Void> filter(ServerWebExchange exchange, GatewayFilterChain chain) {
ServerHttpRequest request = exchange.getRequest();
// The ServerHttpRequest instance can be modified here
ServerHttpRequest newRequest = ...
ServerHttpResponse response = exchange.getResponse();
// You can modify the ServerHttpResponse instance here
ServerHttpResponse newResponse = ...
// Build a new ServerWebExchange instance
ServerWebExchange newExchange = exchange.mutate().request(newRequest).response(newResponse).build();
return chain.filter(newExchange);
}
}
ServerHttpRequest interface
The ServerHttpRequest instance is used to carry the attributes and request body related to the request. The bottom layer of the Spring Cloud Gateway uses Netty to process network requests. By tracing the source code, you can know from the reactorthtphandleradapter that the specific implementation of the ServerHttpRequest instance held in the ServerWebExchange instance is reactoserverhttprequest. The reason for listing the relationship between these examples is that it is easier to clarify some implicit problems, such as:
- When initializing the internal property headers in the parent class AbstractServerHttpRequest of reactoserverhttprequest, the HTTP header of the request is encapsulated as a read-only instance:
public AbstractServerHttpRequest(URI uri, @Nullable String contextPath, HttpHeaders headers) {
this.uri = uri;
this.path = RequestPath.parse(uri, contextPath);
this.headers = HttpHeaders.readOnlyHttpHeaders(headers);
}
// The readonlyhttpaheaders method in the HttpHeaders class. Readonlyhttpaheaders shields all methods that modify the request header and directly throws an unsupported operationexception
public static HttpHeaders readOnlyHttpHeaders(HttpHeaders headers) {
Assert.notNull(headers, "HttpHeaders must not be null");
if (headers instanceof ReadOnlyHttpHeaders) {
return headers;
}
else {
return new ReadOnlyHttpHeaders(headers);
}
}
Therefore, the request header HttpHeaders instance cannot be obtained and modified directly from the ServerHttpRequest instance.
The ServerHttpRequest interface is as follows:
public interface HttpMessage {
// Get the request header. The readonlyhttpaheaders instance returned in the current implementation is read-only
HttpHeaders getHeaders();
}
public interface ReactiveHttpInputMessage extends HttpMessage {
// Return the Flux encapsulation of the request body
Flux<DataBuffer> getBody();
}
public interface HttpRequest extends HttpMessage {
// Returns the HTTP request method and resolves it to an HttpMethod instance
@Nullable
default HttpMethod getMethod() {
return HttpMethod.resolve(getMethodValue());
}
// Return HTTP request method, string
String getMethodValue();
// Requested URI
URI getURI();
}
public interface ServerHttpRequest extends HttpRequest, ReactiveHttpInputMessage {
// Unique identification of the connection or for log processing
String getId();
// Get the request path and encapsulate it as a RequestPath object
RequestPath getPath();
// The returned query parameter is a read-only MultiValueMap instance
MultiValueMap<String, String> getQueryParams();
// Returns the Cookie collection, which is a read-only MultiValueMap instance
MultiValueMap<String, HttpCookie> getCookies();
// Remote server address information
@Nullable
default InetSocketAddress getRemoteAddress() {
return null;
}
// Information about SSL session implementation
@Nullable
default SslInfo getSslInfo() {
return null;
}
// Modify the requested method and return a Builder instance Builder, which is an internal class
default ServerHttpRequest.Builder mutate() {
return new DefaultServerHttpRequestBuilder(this);
}
interface Builder {
// Override request method
Builder method(HttpMethod httpMethod);
// The URI, request path or context of the overlay request are mutually restrictive. For details, please refer to the API notes
Builder uri(URI uri);
Builder path(String path);
Builder contextPath(String contextPath);
// Overwrite request header
Builder header(String key, String value);
Builder headers(Consumer<HttpHeaders> headersConsumer);
// Overwrite sslino
Builder sslInfo(SslInfo sslInfo);
// Build a new ServerHttpRequest instance
ServerHttpRequest build();
}
}
If you want to modify the ServerHttpRequest instance, you need to do this:
ServerHttpRequest request = exchange.getRequest();
ServerHttpRequest newRequest = request.mutate().headers("key","value").path("/myPath").build();
The most noteworthy thing here is: the method HttpHeaders getHeaders() provided by ServerHttpRequest or HttpMessage interface to get request headers; The returned result is a read-only instance, specifically the readonlyhttpaheaders type. I mention it again here. The Spring Cloud Gateway version used by the author when writing this blog post is Greenwich.SR1.
ServerHttpResponse interface
The ServerHttpResponse instance is used to carry the attributes and response body related to the response. The bottom layer of the Spring Cloud Gateway uses Netty to process network requests. By tracing the source code, you can know from the reactorthtphandleradapter that the specific implementation of the ServerHttpResponse instance held in the ServerWebExchange instance is reactoserverhttpresponse. The reason for listing the relationship between these examples is that it is easier to clarify some implicit problems, such as:
// Parent class of reactoserverhttpresponse
public AbstractServerHttpResponse(DataBufferFactory dataBufferFactory, HttpHeaders headers) {
Assert.notNull(dataBufferFactory, "DataBufferFactory must not be null");
Assert.notNull(headers, "HttpHeaders must not be null");
this.dataBufferFactory = dataBufferFactory;
this.headers = headers;
this.cookies = new LinkedMultiValueMap<>();
}
public ReactorServerHttpResponse(HttpServerResponse response, DataBufferFactory bufferFactory) {
super(bufferFactory, new HttpHeaders(new NettyHeadersAdapter(response.responseHeaders())));
Assert.notNull(response, "HttpServerResponse must not be null");
this.response = response;
}
It can be seen that when the reactorserverhtpresponse constructor initializes the instance, the HttpHeaders instance stores the response Header, that is, the response Header can be modified directly.
The ServerHttpResponse interface is as follows:
public interface HttpMessage {
// Get the response Header. In the current implementation, the returned HttpHeaders instance can be modified directly
HttpHeaders getHeaders();
}
public interface ReactiveHttpOutputMessage extends HttpMessage {
// Get the DataBufferFactory instance, which is used to wrap or generate the DataBuffer instance of the data buffer (create the response body)
DataBufferFactory bufferFactory();
// Register an action. This action will call back before HttpOutputMessage is submitted
void beforeCommit(Supplier<? extends Mono<Void>> action);
// Judge whether HttpOutputMessage has been submitted
boolean isCommitted();
// Write message body to HTTP protocol layer
Mono<Void> writeWith(Publisher<? extends DataBuffer> body);
// Write the message body to the HTTP protocol layer and flush the buffer
Mono<Void> writeAndFlushWith(Publisher<? extends Publisher<? extends DataBuffer>> body);
// Indicates that the message processing has ended. Generally, this method is called automatically at the end of message processing. Multiple calls will not have side effects
Mono<Void> setComplete();
}
public interface ServerHttpResponse extends ReactiveHttpOutputMessage {
// Set response status code
boolean setStatusCode(@Nullable HttpStatus status);
// Get response status code
@Nullable
HttpStatus getStatusCode();
// Get the response Cookie and encapsulate it as a MultiValueMap instance, which can be modified
MultiValueMap<String, ResponseCookie> getCookies();
// Add response Cookie
void addCookie(ResponseCookie cookie);
}
It can be seen here that except that the response body is difficult to modify, other properties are variable.
ServerWebExchangeUtils and context properties
ServerWebExchangeUtils stores many static public string KEY values (the actual value of these string keys is org.springframework.cloud.gateway.support.ServerWebExchangeUtils. + any static public KEY below). These string KEY values are generally used for the attributes of ServerWebExchange (Attribute, see ServerWebExchange#getAttributes() method above), These Attribute values have special meanings. When using the filter, if the time is appropriate, they can be directly taken out for use. Next, analyze them one by one.
- PRESERVE_HOST_HEADER_ATTRIBUTE: whether to save the Host property. The value is Boolean. The write location is PreserveHostHeaderGatewayFilterFactory. The location used is NettyRoutingFilter. The purpose is that if it is set to true, the Host property in the HTTP request Header will be written to the request Header property of the underlying reactor netty.
- CLIENT_RESPONSE_ATTR: save the response object of the underlying reactor netty. The type is reactor.netty.http.client.HttpClientResponse.
- CLIENT_RESPONSE_CONN_ATTR: saves the connection object of the underlying reactor netty. The type is reactor.netty.Connection.
- URI_TEMPLATE_VARIABLES_ATTRIBUTE: after the pathroutepredictefactory parses the Path parameters, store the placeholder KEY Path map after parsing in the ServerWebExchange property. KEY is the URI_TEMPLATE_VARIABLES_ATTRIBUTE.
- CLIENT_RESPONSE_HEADER_NAMES: a collection of names that hold the response headers of the underlying reactor netty.
- GATEWAY_ROUTE_ATTR: used to store the specific route (org.springframework.cloud.gateway.route.Route) instance matched in RoutePredicateHandlerMapping. Through this route instance, you can know which downstream service the current request will be routed to.
- GATEWAY_REQUEST_URL_ATTR: an instance of java.net.URI type, which represents the real URI that needs to be requested to the downstream service after direct request or load balancing processing.
- GATEWAY_ORIGINAL_REQUEST_URL_ATTR: an instance of java.net.URI type. When the request URI needs to be rewritten, the original request URI is saved.
- GATEWAY_HANDLER_MAPPER_ATTR: short name of the type of HandlerMapping instance currently used (usually the string "routepredictehandlermapping").
- GATEWAY_SCHEME_PREFIX_ATTR: determines that if there is a schemespecicpart attribute in the target route URI, the scheme of the URI will be saved. In this attribute, the route URI will be reconstructed. See routetorequest urlfilter.
- GATEWAY_PREDICATE_ROUTE_ATTR: used to store the ID of the specific route (org.springframework.cloud.gateway.route.Route) instance matched in RoutePredicateHandlerMapping.
- WEIGHT_ATTR: experimental function (this version is not recommended to be used in the formal version) to store attributes related to grouping weight. See WeightCalculatorWebFilter.
- ORIGINAL_RESPONSE_CONTENT_TYPE_ATTR: stores the value of ContentType in the response Header.
- HYSTRIX_EXECUTION_EXCEPTION_ATTR: Throwable instance. It stores the exception instance when Hystrix executes an exception. See HystrixGatewayFilterFactory.
- GATEWAY_ALREADY_ROUTED_ATTR: Boolean value used to judge whether routing has been performed. See NettyRoutingFilter.
- GATEWAY_ALREADY_PREFIXED_ATTR: Boolean value, used to judge whether the request path has been added with a pre part. See PrefixPathGatewayFilterFactory.
The context attribute provided by ServerWebExchangeUtils is used for the secure transmission and use of some important internal instances or identity attributes when the ServerWebExchange component of Spring Cloud Gateway processes requests and responses. There may be some risks in using them, because no one can determine whether the original attribute KEY or VALUE will change after the version upgrade, If you have assessed the risk or avoided the risk, you can use it at ease. For example, we can rely on gateway when making request and response logs (similar to Nginx's Access Log)_ ROUTE_ Attr, because we want to print the destination information of the route. A simple example:
@Slf4j
@Component
public class AccessLogFilter implements GlobalFilter {
@Override
public Mono<Void> filter(ServerWebExchange exchange, GatewayFilterChain chain) {
ServerHttpRequest request = exchange.getRequest();
String path = request.getPath().pathWithinApplication().value();
HttpMethod method = request.getMethod();
// Gets the destination URI of the route
URI targetUri = exchange.getAttribute(ServerWebExchangeUtils.GATEWAY_REQUEST_URL_ATTR);
InetSocketAddress remoteAddress = request.getRemoteAddress();
return chain.filter(exchange.mutate().build()).then(Mono.fromRunnable(() -> {
ServerHttpResponse response = exchange.getResponse();
HttpStatus statusCode = response.getStatusCode();
log.info("Request path:{},Client remote IP address:{},Request method:{},target URI:{},Response code:{}",
path, remoteAddress, method, targetUri, statusCode);
}));
}
}
Modify request body
Modifying the request body is a common requirement. For example, when we use Spring Cloud Gateway to implement the gateway, we need to implement a function: parse the JWT stored in the request header, extract the user ID in it, and then write it to the request body. Let's simplify this scenario. Suppose we store the userId plaintext in the accessToken in the request header, and the request body is a JSON structure:
{
"serialNumber": "Request serial number",
"payload" : {
// ... here is the payload, storing specific data
}
}
We need to extract the accessToken, that is, insert the userId into the request body JSON as follows:
{
"userId": "user ID",
"serialNumber": "Request serial number",
"payload" : {
// ... here is the payload, storing specific data
}
}
In order to simplify the design, the global filter GlobalFilter is used to implement it. In practice, it needs to be considered in combination with specific scenarios:
@Slf4j
@Component
public class ModifyRequestBodyGlobalFilter implements GlobalFilter {
private final DataBufferFactory dataBufferFactory = new NettyDataBufferFactory(ByteBufAllocator.DEFAULT);
@Autowired
private ObjectMapper objectMapper;
@Override
public Mono<Void> filter(ServerWebExchange exchange, GatewayFilterChain chain) {
ServerHttpRequest request = exchange.getRequest();
String accessToken = request.getHeaders().getFirst("accessToken");
if (!StringUtils.hasLength(accessToken)) {
throw new IllegalArgumentException("accessToken");
}
// Create a new ServerHttpRequest decorator to cover the methods to be decorated
ServerHttpRequestDecorator decorator = new ServerHttpRequestDecorator(request) {
@Override
public Flux<DataBuffer> getBody() {
Flux<DataBuffer> body = super.getBody();
InputStreamHolder holder = new InputStreamHolder();
body.subscribe(buffer -> holder.inputStream = buffer.asInputStream());
if (null != holder.inputStream) {
try {
// Parsing JSON nodes
JsonNode jsonNode = objectMapper.readTree(holder.inputStream);
Assert.isTrue(jsonNode instanceof ObjectNode, "JSON Format exception");
ObjectNode objectNode = (ObjectNode) jsonNode;
// Write a new attribute to the outermost layer of the JSON node
objectNode.put("userId", accessToken);
DataBuffer dataBuffer = dataBufferFactory.allocateBuffer();
String json = objectNode.toString();
log.info("Final JSON Data is:{}", json);
dataBuffer.write(json.getBytes(StandardCharsets.UTF_8));
return Flux.just(dataBuffer);
} catch (Exception e) {
throw new IllegalStateException(e);
}
} else {
return super.getBody();
}
}
};
// Regenerate a new ServerWebExchange using the modified ServerHttpRequestDecorator
return chain.filter(exchange.mutate().request(decorator).build());
}
private class InputStreamHolder {
InputStream inputStream;
}
}
Test:
// HTTP
POST /order/json HTTP/1.1
Host: localhost:9090
Content-Type: application/json
accessToken: 10086
Accept: */*
Cache-Control: no-cache
Host: localhost:9090
accept-encoding: gzip, deflate
content-length: 94
Connection: keep-alive
cache-control: no-cache
{
"serialNumber": "Request serial number",
"payload": {
"name": "doge"
}
}
// Log output
Final JSON Data is:{"serialNumber":"Request serial number","payload":{"name":"doge"},"userId":"10086"}
The most important thing is to use the ServerHttpRequest decorator ServerHttpRequest decorator, which mainly covers the method of obtaining the data buffer of the request body. As for how to deal with other logic, you need to consider it yourself. Here is just a simple demonstration. The general code logic is as follows:
ServerHttpRequest request = exchange.getRequest();
ServerHttpRequestDecorator requestDecorator = new ServerHttpRequestDecorator(request) {
@Override
public Flux<DataBuffer> getBody() {
// Get the Flux that carries the original request body
Flux<DataBuffer> body = super.getBody();
// Here, a new Flux carrying the request body is generated in a custom way
Flux<DataBuffer> newBody = ...
return newBody;
}
}
return chain.filter(exchange.mutate().request(requestDecorator).build());
Modify response body
It is also common to modify the requirements of the response body. The specific method is similar to that of modifying the request body. For example, we want to realize the following functions: the third-party service request passes through the gateway, and the original message is ciphertext. We need to decrypt the ciphertext at the gateway, and then route the decrypted plaintext to the downstream service. The downstream service successfully responds to the plaintext. We need to encrypt the plaintext into ciphertext at the gateway and then return it to the third-party service. Now simplify the whole process, use AES encryption algorithm and unify the password into the string "throwable". Suppose that the request message and response message are as follows:
// Request ciphertext
{
"serialNumber": "Request serial number",
"payload" : "Encrypted request message payload"
}
// Request plaintext (only as a hint)
{
"serialNumber": "Request serial number",
"payload" : "{\"name:\":\"doge\"}"
}
// Response ciphertext
{
"code": 200,
"message":"ok",
"payload" : "Encrypted response message payload"
}
// Response plaintext (only as a hint)
{
"code": 200,
"message":"ok",
"payload" : "{\"name:\":\"doge\",\"age\":26}"
}
In order to facilitate the implementation of encryption, decryption or encoding and decoding, it is necessary to introduce the Commons codec class library of Apache:
<dependency>
<groupId>commons-codec</groupId>
<artifactId>commons-codec</artifactId>
<version>1.12</version>
</dependency>
A global filter is defined here to deal with encryption and decryption. In fact, it is best to determine whether it is suitable for the global filter in combination with the real scene. Here is only an example:
// AES encryption and decryption tool class
public enum AesUtils {
// Single case
X;
private static final String PASSWORD = "throwable";
private static final String KEY_ALGORITHM = "AES";
private static final String SECURE_RANDOM_ALGORITHM = "SHA1PRNG";
private static final String DEFAULT_CIPHER_ALGORITHM = "AES/ECB/PKCS5Padding";
public String encrypt(String content) {
try {
Cipher cipher = Cipher.getInstance(DEFAULT_CIPHER_ALGORITHM);
cipher.init(Cipher.ENCRYPT_MODE, provideSecretKey());
return Hex.encodeHexString(cipher.doFinal(content.getBytes(StandardCharsets.UTF_8)));
} catch (Exception e) {
throw new IllegalArgumentException(e);
}
}
public byte[] decrypt(String content) {
try {
Cipher cipher = Cipher.getInstance(DEFAULT_CIPHER_ALGORITHM);
cipher.init(Cipher.DECRYPT_MODE, provideSecretKey());
return cipher.doFinal(Hex.decodeHex(content));
} catch (Exception e) {
throw new IllegalArgumentException(e);
}
}
private SecretKey provideSecretKey() {
try {
KeyGenerator keyGen = KeyGenerator.getInstance(KEY_ALGORITHM);
SecureRandom secureRandom = SecureRandom.getInstance(SECURE_RANDOM_ALGORITHM);
secureRandom.setSeed(PASSWORD.getBytes(StandardCharsets.UTF_8));
keyGen.init(128, secureRandom);
return new SecretKeySpec(keyGen.generateKey().getEncoded(), KEY_ALGORITHM);
} catch (Exception e) {
throw new IllegalArgumentException(e);
}
}
}
// EncryptionGlobalFilter
@Slf4j
@Component
public class EncryptionGlobalFilter implements GlobalFilter, Ordered {
@Autowired
private ObjectMapper objectMapper;
@Override
public int getOrder() {
return -2;
}
@Override
public Mono<Void> filter(ServerWebExchange exchange, GatewayFilterChain chain) {
ServerHttpRequest request = exchange.getRequest();
ServerHttpResponse response = exchange.getResponse();
DataBufferFactory bufferFactory = exchange.getResponse().bufferFactory();
ServerHttpRequestDecorator requestDecorator = processRequest(request, bufferFactory);
ServerHttpResponseDecorator responseDecorator = processResponse(response, bufferFactory);
return chain.filter(exchange.mutate().request(requestDecorator).response(responseDecorator).build());
}
private ServerHttpRequestDecorator processRequest(ServerHttpRequest request, DataBufferFactory bufferFactory) {
Flux<DataBuffer> body = request.getBody();
DataBufferHolder holder = new DataBufferHolder();
body.subscribe(dataBuffer -> {
int len = dataBuffer.readableByteCount();
holder.length = len;
byte[] bytes = new byte[len];
dataBuffer.read(bytes);
DataBufferUtils.release(dataBuffer);
String text = new String(bytes, StandardCharsets.UTF_8);
JsonNode jsonNode = readNode(text);
JsonNode payload = jsonNode.get("payload");
String payloadText = payload.asText();
byte[] content = AesUtils.X.decrypt(payloadText);
String requestBody = new String(content, StandardCharsets.UTF_8);
log.info("Modify request body payload,Before modification:{},After modification:{}", payloadText, requestBody);
rewritePayloadNode(requestBody, jsonNode);
DataBuffer data = bufferFactory.allocateBuffer();
data.write(jsonNode.toString().getBytes(StandardCharsets.UTF_8));
holder.dataBuffer = data;
});
HttpHeaders headers = new HttpHeaders();
headers.putAll(request.getHeaders());
headers.remove(HttpHeaders.CONTENT_LENGTH);
return new ServerHttpRequestDecorator(request) {
@Override
public HttpHeaders getHeaders() {
int contentLength = holder.length;
if (contentLength > 0) {
headers.setContentLength(contentLength);
} else {
headers.set(HttpHeaders.TRANSFER_ENCODING, "chunked");
}
return headers;
}
@Override
public Flux<DataBuffer> getBody() {
return Flux.just(holder.dataBuffer);
}
};
}
private ServerHttpResponseDecorator processResponse(ServerHttpResponse response, DataBufferFactory bufferFactory) {
return new ServerHttpResponseDecorator(response) {
@SuppressWarnings("unchecked")
@Override
public Mono<Void> writeWith(Publisher<? extends DataBuffer> body) {
if (body instanceof Flux) {
Flux<? extends DataBuffer> flux = (Flux<? extends DataBuffer>) body;
return super.writeWith(flux.map(buffer -> {
CharBuffer charBuffer = StandardCharsets.UTF_8.decode(buffer.asByteBuffer());
DataBufferUtils.release(buffer);
JsonNode jsonNode = readNode(charBuffer.toString());
JsonNode payload = jsonNode.get("payload");
String text = payload.toString();
String content = AesUtils.X.encrypt(text);
log.info("Modify response body payload,Before modification:{},After modification:{}", text, content);
setPayloadTextNode(content, jsonNode);
return bufferFactory.wrap(jsonNode.toString().getBytes(StandardCharsets.UTF_8));
}));
}
return super.writeWith(body);
}
};
}
private void rewritePayloadNode(String text, JsonNode root) {
try {
JsonNode node = objectMapper.readTree(text);
ObjectNode objectNode = (ObjectNode) root;
objectNode.set("payload", node);
} catch (Exception e) {
throw new IllegalStateException(e);
}
}
private void setPayloadTextNode(String text, JsonNode root) {
try {
ObjectNode objectNode = (ObjectNode) root;
objectNode.set("payload", new TextNode(text));
} catch (Exception e) {
throw new IllegalStateException(e);
}
}
private JsonNode readNode(String in) {
try {
return objectMapper.readTree(in);
} catch (Exception e) {
throw new IllegalStateException(e);
}
}
private class DataBufferHolder {
DataBuffer dataBuffer;
int length;
}
}
Prepare a ciphertext first:
Map<String, Object> json = new HashMap<>(8);
json.put("serialNumber", "Request serial number");
String content = "{\"name\": \"doge\"}";
json.put("payload", AesUtils.X.encrypt(content));
System.out.println(new ObjectMapper().writeValueAsString(json));
// output
{"serialNumber":"Request serial number","payload":"144e3dc734743f5709f1adf857bca473da683246fd612f86ac70edeb5f2d2729"}
Simulation request:
POST /order/json HTTP/1.1
Host: localhost:9090
accessToken: 10086
Content-Type: application/json
User-Agent: PostmanRuntime/7.13.0
Accept: */*
Cache-Control: no-cache
Postman-Token: bda07fc3-ea1a-478c-b4d7-754fe6f37200,634734d9-feed-4fc9-ba20-7618bd986e1c
Host: localhost:9090
cookie: customCookieName=customCookieValue
accept-encoding: gzip, deflate
content-length: 104
Connection: keep-alive
cache-control: no-cache
{
"serialNumber": "Request serial number",
"payload": "FE49xzR0P1cJ8a34V7ykc9poMkb9YS+GrHDt618tJyk="
}
// Response results
{
"serialNumber": "Request serial number",
"payload": "oo/K1igg2t/S8EExkBVGWOfI1gAh5pBpZ0wyjNPW6e8=" # < --- after decryption: {name":"doge","age":26}
}
Problems encountered:
- The Ordered interface must be implemented to return an order value less than - 1. This is because the order value of NettyWriteResponseFilter is - 1. We need to override the logic of returning the response body. The customized GlobalFilter must be executed prior to NettyWriteResponseFilter.
- After the gateway restarts each time, the first request cannot read the valid Body from the original ServerHttpRequest. To be exact, the phenomenon is that NettyRoutingFilter obtains an empty object when calling ServerHttpRequest#getBody(), resulting in a null pointer; Strangely, it can be called normally from the second request. The author reduced the version of Spring Cloud Gateway to Finchley.SR3 and Spring Boot to 2.0.8.RELEASE. The problem no longer occurs. It is preliminarily determined that the compatibility problem or BUG caused by the version upgrade of Spring Cloud Gateway.
The most important thing is to use the ServerHttpResponseDecorator, which mainly covers the part written to the data buffer of the response body. As for how to deal with other logic, you need to consider it yourself. Here is just a simple demonstration. The general code logic is as follows:
ServerHttpResponse response = exchange.getResponse();
ServerHttpResponseDecorator responseDecorator = new ServerHttpResponseDecorator(response) {
@Override
public Mono<Void> writeWith(Publisher<? extends DataBuffer> body) {
if (body instanceof Flux) {
Flux<? extends DataBuffer> flux = (Flux<? extends DataBuffer>) body;
return super.writeWith(flux.map(buffer -> {
// Buffer is the buffer of the original response data
// After processing, you can return the buffer of new response data
return bufferFactory.wrap(...);
}));
}
return super.writeWith(body);
}
};
return chain.filter(exchange.mutate().response(responseDecorator).build());
The message size of the requestor or responder is too large
Some enthusiastic students told the author that if the request message is too large or the response message is too large, there will be problems in the method of modifying the request and response message in the previous two sections. Here, try to reproduce the specific problems encountered. Try to lengthen the request message first:
Map<String, Object> json = new HashMap<>(8);
json.put("serialNumber", "Request serial number");
StringBuilder builder = new StringBuilder();
for (int i = 0; i < 1000; i++) {
builder.append("doge");
}
String content = String.format("{\"name\": \"%s\"}", builder.toString());
json.put("payload", AesUtils.X.encrypt(content));
System.out.println(new ObjectMapper().writeValueAsString(json));
// The requested JSON message is as follows:
{
"serialNumber": "Request serial number",
"payload": "......"
}
Copy code
There is a problem when the above request message is used to initiate the request:
The main problems are:
- After the flux < databuffer > instance loaded with the request body packet data is subscribed, the length of the read byte array is truncated. The length of the string in the provided original request message should be greater than 1000, and it must be greater than 1000 when converted into a byte array. However, in the above example, only the byte array with length of 673 is read.
- After the read byte array is truncated, when Jackson is used for deserialization, it will be prompted that the EOF ID of the string is not read, resulting in deserialization failure.
Now that you have a problem, find a way to solve it. First, the first step is to locate the reason. My intuition tells the author to open the DEBUG log for observation. If there is no clue, I may want to track the source code.
After the DEBUG log level is enabled, a request is made and some suspicious log information is found:
2019-05-19 11:16:15.660 [reactor-http-nio-2] DEBUG reactor.ipc.netty.http.server.HttpServer - [id: 0xa9b527e5, L:/0:0:0:0:0:0:0:1:9090 - R:/0:0:0:0:0:0:0:1:58012] READ COMPLETE
2019-05-19 11:16:15.660 [reactor-http-nio-2] DEBUG reactor.ipc.netty.http.server.HttpServer - [id: 0xa9b527e5, L:/0:0:0:0:0:0:0:1:9090 ! R:/0:0:0:0:0:0:0:1:58012] INACTIVE
2019-05-19 11:16:15.660 [reactor-http-nio-3] DEBUG reactor.ipc.netty.http.server.HttpServer - [id: 0x5554e091, L:/0:0:0:0:0:0:0:1:9090 - R:/0:0:0:0:0:0:0:1:58013] READ: 1024B
+-------------------------------------------------+
| 0 1 2 3 4 5 6 7 8 9 a b c d e f |
+--------+-------------------------------------------------+----------------+
|00000000| 50 4f 53 54 20 2f 6f 72 64 65 72 2f 6a 73 6f 6e |POST /order/json|
|00000010| 20 48 54 54 50 2f 31 2e 31 0d 0a 61 63 63 65 73 | HTTP/1.1..acces|
|00000020| 73 54 6f 6b 65 6e 3a 20 31 30 30 38 36 0d 0a 43 |sToken: 10086..C|
... ...
|000003f0| 49 41 72 6b 64 37 58 57 35 4c 6c 32 2f 71 61 42 |IArkd7XW5Ll2/qaB|
+--------+-------------------------------------------------+----------------+
2019-05-19 11:16:15.662 [reactor-http-nio-2] DEBUG reactor.ipc.netty.http.server.HttpServer - [id: 0xa9b527e5, L:/0:0:0:0:0:0:0:1:9090 ! R:/0:0:0:0:0:0:0:1:58012] UNREGISTERED
2019-05-19 11:16:15.665 [reactor-http-nio-3] DEBUG reactor.ipc.netty.http.server.HttpServerOperations - [id: 0x5554e091, L:/0:0:0:0:0:0:0:1:9090 - R:/0:0:0:0:0:0:0:1:58013] Increasing pending responses, now 1
2019-05-19 11:16:15.671 [reactor-http-nio-3] DEBUG reactor.ipc.netty.http.server.HttpServer - [id: 0x5554e091, L:/0:0:0:0:0:0:0:1:9090 - R:/0:0:0:0:0:0:0:1:58013] READ COMPLETE
Note the keyword READ: 1024B, which should be the length limit of the maximum datagram read by the underlying reactor netty. The printed datagram is just the size of 1024B, which should be the root cause of the truncation of the request body; This problem occurs not only in the acquisition of the request body, but also in the writing of the response body. Since this is a common problem, there must be a corresponding Issue on the project Github. Find one with long interaction gateway request size limit 1024B because netty default limit 1024,how to solve it? #581 From the answer, it is suggested that ModifyRequestBodyGatewayFilterFactory and ModifyResponseBodyGatewayFilterFactory be used to complete the corresponding functions. Here, we can try to learn from the implementation of ModifyRequestBodyGatewayFilterFactory and modify the previous code because the logic of the code is relatively long and complex. The filter for decrypting the request body is split into a new class RequestEncryptionGlobalFilter, and the filter for encrypting the response body is split into ResponseDecryptionGlobalFilter:
The code of RequestEncryptionGlobalFilter is as follows:
@Slf4j
@Component
public class RequestEncryptionGlobalFilter implements GlobalFilter, Ordered {
@Autowired
private ObjectMapper objectMapper;
private final List<HttpMessageReader<?>> messageReaders = HandlerStrategies.withDefaults().messageReaders();
@Override
public int getOrder() {
return -2;
}
@Override
public Mono<Void> filter(ServerWebExchange exchange, GatewayFilterChain chain) {
return processRequest(exchange, chain);
}
private Mono<Void> processRequest(ServerWebExchange exchange, GatewayFilterChain chain) {
ServerRequest serverRequest = new DefaultServerRequest(exchange, messageReaders);
DataBufferFactory bufferFactory = exchange.getResponse().bufferFactory();
Mono<String> rawBody = serverRequest.bodyToMono(String.class).map(s -> s);
BodyInserter<Mono<String>, ReactiveHttpOutputMessage> bodyInserter = BodyInserters.fromPublisher(rawBody, String.class);
HttpHeaders tempHeaders = new HttpHeaders();
tempHeaders.putAll(exchange.getRequest().getHeaders());
tempHeaders.remove(HttpHeaders.CONTENT_LENGTH);
CachedBodyOutputMessage outputMessage = new CachedBodyOutputMessage(exchange, tempHeaders);
return bodyInserter.insert(outputMessage, new BodyInserterContext()).then(Mono.defer(() -> {
Flux<DataBuffer> body = outputMessage.getBody();
DataBufferHolder holder = new DataBufferHolder();
body.subscribe(dataBuffer -> {
int len = dataBuffer.readableByteCount();
holder.length = len;
byte[] bytes = new byte[len];
dataBuffer.read(bytes);
DataBufferUtils.release(dataBuffer);
String text = new String(bytes, StandardCharsets.UTF_8);
JsonNode jsonNode = readNode(text);
JsonNode payload = jsonNode.get("payload");
String payloadText = payload.asText();
byte[] content = AesUtils.X.decrypt(payloadText);
String requestBody = new String(content, StandardCharsets.UTF_8);
log.info("Modify request body payload,Before modification:{},After modification:{}", payloadText, requestBody);
rewritePayloadNode(requestBody, jsonNode);
DataBuffer data = bufferFactory.allocateBuffer();
data.write(jsonNode.toString().getBytes(StandardCharsets.UTF_8));
holder.dataBuffer = data;
});
ServerHttpRequestDecorator requestDecorator = new ServerHttpRequestDecorator(exchange.getRequest()) {
@Override
public HttpHeaders getHeaders() {
long contentLength = tempHeaders.getContentLength();
HttpHeaders httpHeaders = new HttpHeaders();
httpHeaders.putAll(super.getHeaders());
if (contentLength > 0) {
httpHeaders.setContentLength(contentLength);
} else {
httpHeaders.set(HttpHeaders.TRANSFER_ENCODING, "chunked");
}
return httpHeaders;
}
@Override
public Flux<DataBuffer> getBody() {
return Flux.just(holder.dataBuffer);
}
};
return chain.filter(exchange.mutate().request(requestDecorator).build());
}));
}
private void rewritePayloadNode(String text, JsonNode root) {
try {
JsonNode node = objectMapper.readTree(text);
ObjectNode objectNode = (ObjectNode) root;
objectNode.set("payload", node);
} catch (Exception e) {
throw new IllegalStateException(e);
}
}
private void setPayloadTextNode(String text, JsonNode root) {
try {
ObjectNode objectNode = (ObjectNode) root;
objectNode.set("payload", new TextNode(text));
} catch (Exception e) {
throw new IllegalStateException(e);
}
}
private JsonNode readNode(String in) {
try {
return objectMapper.readTree(in);
} catch (Exception e) {
throw new IllegalStateException(e);
}
}
private class DataBufferHolder {
DataBuffer dataBuffer;
int length;
}
}
The code of ResponseDecryptionGlobalFilter is as follows:
@Slf4j
@Component
public class ResponseDecryptionGlobalFilter implements GlobalFilter, Ordered {
@Autowired
private ObjectMapper objectMapper;
@Override
public int getOrder() {
return NettyWriteResponseFilter.WRITE_RESPONSE_FILTER_ORDER - 1;
}
@Override
public Mono<Void> filter(ServerWebExchange exchange, GatewayFilterChain chain) {
return processResponse(exchange, chain);
}
private Mono<Void> processResponse(ServerWebExchange exchange, GatewayFilterChain chain) {
ServerHttpResponseDecorator responseDecorator = new ServerHttpResponseDecorator(exchange.getResponse()) {
@Override
public Mono<Void> writeWith(Publisher<? extends DataBuffer> body) {
String originalResponseContentType = exchange.getAttribute(ORIGINAL_RESPONSE_CONTENT_TYPE_ATTR);
HttpHeaders httpHeaders = new HttpHeaders();
httpHeaders.add(HttpHeaders.CONTENT_TYPE, originalResponseContentType);
ResponseAdapter responseAdapter = new ResponseAdapter(body, httpHeaders);
DefaultClientResponse clientResponse = new DefaultClientResponse(responseAdapter, ExchangeStrategies.withDefaults());
Mono<String> rawBody = clientResponse.bodyToMono(String.class).map(s -> s);
BodyInserter<Mono<String>, ReactiveHttpOutputMessage> bodyInserter = BodyInserters.fromPublisher(rawBody, String.class);
CachedBodyOutputMessage outputMessage = new CachedBodyOutputMessage(exchange, exchange.getResponse().getHeaders());
return bodyInserter.insert(outputMessage, new BodyInserterContext())
.then(Mono.defer(() -> {
Flux<DataBuffer> messageBody = outputMessage.getBody();
Flux<DataBuffer> flux = messageBody.map(buffer -> {
CharBuffer charBuffer = StandardCharsets.UTF_8.decode(buffer.asByteBuffer());
DataBufferUtils.release(buffer);
JsonNode jsonNode = readNode(charBuffer.toString());
JsonNode payload = jsonNode.get("payload");
String text = payload.toString();
String content = AesUtils.X.encrypt(text);
log.info("Modify response body payload,Before modification:{},After modification:{}", text, content);
setPayloadTextNode(content, jsonNode);
return getDelegate().bufferFactory().wrap(jsonNode.toString().getBytes(StandardCharsets.UTF_8));
});
HttpHeaders headers = getDelegate().getHeaders();
if (!headers.containsKey(HttpHeaders.TRANSFER_ENCODING)) {
flux = flux.doOnNext(data -> headers.setContentLength(data.readableByteCount()));
}
return getDelegate().writeWith(flux);
}));
}
};
return chain.filter(exchange.mutate().response(responseDecorator).build());
}
private void setPayloadTextNode(String text, JsonNode root) {
try {
ObjectNode objectNode = (ObjectNode) root;
objectNode.set("payload", new TextNode(text));
} catch (Exception e) {
throw new IllegalStateException(e);
}
}
private JsonNode readNode(String in) {
try {
return objectMapper.readTree(in);
} catch (Exception e) {
throw new IllegalStateException(e);
}
}
private class ResponseAdapter implements ClientHttpResponse {
private final Flux<DataBuffer> flux;
private final HttpHeaders headers;
@SuppressWarnings("unchecked")
private ResponseAdapter(Publisher<? extends DataBuffer> body, HttpHeaders headers) {
this.headers = headers;
if (body instanceof Flux) {
flux = (Flux) body;
} else {
flux = ((Mono) body).flux();
}
}
@Override
public Flux<DataBuffer> getBody() {
return flux;
}
@Override
public HttpHeaders getHeaders() {
return headers;
}
@Override
public HttpStatus getStatusCode() {
return null;
}
@Override
public int getRawStatusCode() {
return 0;
}
@Override
public MultiValueMap<String, ResponseCookie> getCookies() {
return null;
}
}
}
Simulation request:
POST /order/json HTTP/1.1
Host: localhost:9090
accessToken: 10086
Content-Type: application/json
User-Agent: PostmanRuntime/7.13.0
Accept: */*
Cache-Control: no-cache
Postman-Token: 3a830202-f3d1-450e-839f-ae8f3b88bced,b229feb1-7c8b-4d25-a039-09345f3fe8f0
Host: localhost:9090
cookie: customCookieName=customCookieValue
accept-encoding: gzip, deflate
content-length: 5416
Connection: keep-alive
cache-control: no-cache
{
"serialNumber": "Request serial number",
"payload": "... ...."
}
// response
{"serialNumber":"Request serial number","userId":null,"payload":"... ..."}
Copy code
It completely solves the problem of previous request or response message truncation. The author found that many blog posts are changing (copying) the code logic when reading DataBuffer instances. In fact, that logic is irrelevant. You can try to use BufferedReader to read based on lines and then use StringBuilder to carry it, or directly read it as a byte array as described in this article, The fundamental reason is that the data in the obtained DataBuffer instance is incomplete due to the reading size limit of the underlying reactor netty data block. The solution is to transform it with reference to the basic class library provided by Spring Cloud Gateway itself (no entry is found to adjust the configuration of reactor netty temporarily), and it is not difficult.
Summary
There is just a need to do the encryption and decryption of the gateway, including the modification of the request body and response body. By the way, I sorted out some contents related to this aspect of the Spring Cloud Gateway, stepped on the pit and filled in it. Next, try to modify the custom logic according to the available components officially provided, including Hystrix, load balancing based on Eureka and Ribbon, current limiting, etc.
Original link
- Github Page: www.throwable.club/2019/05/18/...
- Coding Page: throwable.coding.me/2019/05/18/...