Online problem handling - feign call error (legal character ((ctrl-char, code 31)): only regular white space (\ R \ n \ T))

Keywords: Java Spring Back-end

Online problem handling - feign call error

Business scenario: Service 1 invokes service 2 through Feign. In the test phase, everything is normal and there is data loss online (to avoid simple reproduction of sensitive local). Errors are reported as follows:

2021-12-04 13:47:47.774 DEBUG 29480 --- [io-10011-exec-1] .w.s.m.m.a.ServletInvocableHandlerMethod : Could not resolve parameter [0]
 in public void com.example.service1.controller.TaskControler.syncTaskFile(com.example.service1.domain.rep.SyncTaskFileRep): JSON parse
  error: Illegal character ((CTRL-CHAR, code 31)): only regular white space (\r, \n, \t) is allowed between tokens; nested exception is 
  com.fasterxml.jackson.core.JsonParseException: Illegal character ((CTRL-CHAR, code 31)): only regular white space (\r, \n, \t) is allowed
   between tokens at [Source: (PushbackInputStream); line: 1, column: 2]

Error reporting is not very simple and clear. First, open Feign log and make specific analysis:
To open Feign log:

//Set the configuration class in the annotation
@FeignClient(value = "service1",contextId = "service1-client",
        configuration = {FeignLogConfiguration.class},
        fallbackFactory = Service1ClientFallbackFactory.class)
public interface Service1Client {

    @PostMapping(value = "/service1/task/sync/task_file")
    Void syncTaskFile(@RequestBody SyncTaskFileReq rep);
}

//The log level is promoted to FULL
//feign has four log levels: none, Basic, headers and full. It is recommended to default to Basic
@Component
public class FeignLogConfiguration {
    @Bean
    Logger.Level feignLoggerLevel() {
        return Logger.Level.FULL;
    }
}

The query log is:

: [Service1Client#syncTaskFile] ---> POST http://service1/service1/task/sync/task_file HTTP/1.1
: [Service1Client#syncTaskFile] Accept-Encoding: gzip
: [Service1Client#syncTaskFile] Accept-Encoding: deflate
: [Service1Client#syncTaskFile] Content-Encoding: gzip
: [Service1Client#syncTaskFile] Content-Encoding: deflate
: [Service1Client#syncTaskFile] Content-Length: 2762
: [Service1Client#syncTaskFile] Content-Type: application/json

It can be seen that the transmission body is too large. gzip compression is enabled to transmit requests to improve efficiency. Check the nacos configuration:

feign:
  sentinel:
    enabled: true
  okhttp:
    enabled: true
  httpclient:
    enabled: false
  client:
    config:
      default:
        connectTimeout: 10000
        readTimeout: 10000
  compression:
    request:
      enabled: true
    response:
      enabled: true

okhttp is used and compression is enabled. okhttp and httpclient are unimportant here, mainly because compression is enabled and gzip compression is adopted by default. The default request body size is enabled when it exceeds 2048. It can be seen above that the request body has exceeded 2048. It should be that the test did not create too much data during the previous test, so this problem did not occur, There is a problem with larger data online.
ps. the temporary solution here is to change feign.compression.request.enabled to false, that is, do not compress the transmission.
It is expected that the compression problem of gzip leads to an error. The error is JSON because the request body transmitted in the past is accepted in JSON format. An error occurs during the conversion of class objects. You can try to change the @ RequestBody of service 2 to String type, and then the breakpoint is found, which is garbled.

    //Test feign problem
    @RequestMapping(value = "/sync/task_file", method = RequestMethod.POST)
    public void syncTaskFile(@RequestBody SyncTaskFileRep rep) {
        log.error(rep.toString());
    }
    //Replace with:
    @RequestMapping(value = "/sync/task_file", method = RequestMethod.POST)
    public void syncTaskFile(@RequestBody String rep) {
        log.error(rep);
    }

Printing the log is obvious:

2021-12-04 14:22:57.915 ERROR 13688 --- [io-10011-exec-1] c.e.service1.controller.TaskControler    :        ݕ�jA�_E����� ��"�����I����z�a��[X����Ω�����U�>)�pu�hR1)�o\=�������bT@��[4��K7*���	�f���FǡݻS��V�v���IQ�9�g�._�8{O�է�dܸ�������W�[���LaŌ|4�3V��`u�����j�s����nc6w��޴G�����[�]n�F��ԃf�r���t��v���>��A�i9hR�g`\)�d ��a�a�<���=H)��L�3�+�b:1H�����dr06� ���{*H�d�6�HβO�0���k5dR)���=Rv�Cp��9&p��>����I�O�z�����6Y$���b�AJq�/R�N���

The reason is that service 2 cannot parse the gzip encoded request body when receiving. How to solve it?
1. Someone on the Internet said that feign.compression.response.useGzipDecoder can be set to true to take effect. It's useless to try.
2. Add a filter to handle requests with gzip content encoding:

@Slf4j
public class MyGzipRequestWrapper extends HttpServletRequestWrapper {

    private HttpServletRequest request;

    public MyGzipRequestWrapper(HttpServletRequest request) {
        super(request);
        this.request = request;
    }
    
    @Override
    public ServletInputStream getInputStream() throws IOException {

        ServletInputStream inputStream = request.getInputStream();
        try {
            GZIPInputStream gzipInputStream = new GZIPInputStream(inputStream);
            ServletInputStream newStream = new ServletInputStream() {
                @Override
                public boolean isFinished() {
                    return false;
                }

                @Override
                public boolean isReady() {
                    return false;
                }

                @Override
                public void setReadListener(ReadListener readListener) {
                }

                @Override
                public int read() throws IOException {
                    return gzipInputStream.read();
                }
            };
            return newStream;
        } catch (Exception e) {
            log.error("ungzip fail, ", e);
        }
        return inputStream;
    }
}
@Slf4j
public class MyGzipFilter implements Filter {
    @Override
    public void doFilter(ServletRequest servletRequest, ServletResponse servletResponse, FilterChain filterChain) throws IOException, ServletException {
        HttpServletRequest request = (HttpServletRequest) servletRequest;
        String contentEncoding = request.getHeader("Content-Encoding");
        if(StringUtils.isNotBlank(contentEncoding) && contentEncoding.contains("gzip")){
            request = new MyGzipRequestWrapper(request);
        }
        filterChain.doFilter(request, servletResponse);
    }
}
@Configuration
public class FilterConfiguration {

    @Bean
    public FilterRegistrationBean<MyGzipFilter> gzipFilter(){
        FilterRegistrationBean<MyGzipFilter> registration = new FilterRegistrationBean<>();
        registration.setFilter(new MyGzipFilter());
        registration.addUrlPatterns("/task/*");
        registration.setName("gzipFilter");
        registration.setOrder(5);  //The smaller the value, the higher the Filter.
        return registration;
    }
}

solve! In this scenario, the online environment registration center is nacos. It is said that eureka is the registration center, which has not been verified.

Posted by jokeruk on Sun, 05 Dec 2021 18:01:22 -0800