Code address of this series: https://github.com/JoJoTec/sp...

Scenarios requiring retry

In the microservice system, online publishing will be encountered. The general publishing update strategy is to start a new one. After successful startup, close an old one until all the old ones are closed. Spring Boot has the function of graceful shutdown, which can ensure that the request is closed after processing, and will reject new requests at the same time. These rejected requests need to be retried to ensure that the user experience is not affected.

For a micro service deployed on the cloud, it is likely that all instances of the same service and request will not be abnormal at the same time, for example:

1. For instances deployed in Kubernetes cluster, multiple different micro service instances may be deployed in the same virtual machine Node during idle time. When the pressure becomes greater, migration and capacity expansion are required. At this time, because different microservices have different pressures, which Node they are in at that time may be under high pressure or low pressure. For the same microservice, the nodes where all instances are located may not be under great pressure.
2. Cloud deployment will generally be deployed across availability zones. If one availability zone is abnormal, the other availability zone can continue to provide services.
3. A business triggers a Bug, which causes the instance to be in GC all the time. However, such requests are generally uncommon and will not be sent to all instances.

At this time, we need to retry the request imperceptibly.

Retry issues to consider

1. Retrying requires retrying instances that are different from the previous instances, or even instances that are not in the same virtual machine Node. This is mainly implemented by LoadBalancer. Please refer to the previous LoadBalancer section. In the following articles, we will also improve the LoadBalancer

2. Retry needs to consider what requests can be retried and what exceptions can be retried:

   • Assuming that we have a query interface and a deduction interface without idempotency, we can intuitively feel that the query interface can be retried and the deduction interface without idempotency cannot be retried.
   • For interfaces that cannot be retried in business, we can retry special exceptions (in fact, exceptions indicating that the request has not been sent). Although there is no idempotent deduction interface, if the reason thrown is the IOException of Connect Timeout, such an exception indicates that the request has not been sent and can be retried.
3. Retry strategy: retry several times and retry interval. Similar to the Busy Spin strategy in multiprocessor programming mode, it will cause a large bus flux and reduce the performance. If you fail to retry immediately, many requests will be retried to other instances at the same time when an instance exception causes a timeout. It's best to add a certain delay.

Using resilience4j to implement FeignClient retry

FeignClient itself has retry, but the retry strategy is relatively simple. At the same time, we also want to use circuit breaker, current limiter and thread isolation. resilience4j includes these components.

Principle introduction

Resilience4J provides Retryer retrier. The official document address is: https://resilience4j.readme.i...

We can understand the principle from the configuration, but the official document configuration is not comprehensive. If you want to see all the configurations, you'd better use the source code:

RetryConfigurationProperties.java

//Retry interval: 500ms by default
@Nullable
private Duration waitDuration;

//Only one of the retry interval function and waitDuration can be set. The default is waitDuration
@Nullable
private Class<? extends IntervalBiFunction<Object>> intervalBiFunction;

//The maximum number of retries, including the call itself
@Nullable
private Integer maxAttempts;

//Judge whether to retry by the exception thrown. The default is to retry as long as there is an exception
@Nullable
private Class<? extends Predicate<Throwable>> retryExceptionPredicate;

//Judge whether to retry by the result. The default is not to retry as long as the result is obtained
@Nullable
private Class<? extends Predicate<Object>> resultPredicate;

//If the configuration throws these exceptions and subclasses, it will try again
@SuppressWarnings("unchecked")
@Nullable
private Class<? extends Throwable>[] retryExceptions;

//If the configuration throws these exceptions and subclasses, it will not retry
@SuppressWarnings("unchecked")
@Nullable
private Class<? extends Throwable>[] ignoreExceptions;

//Enable the ExponentialBackoff delay algorithm. The delay time of the first retry is waitDuration. After that, the delay time of each retry is multiplied by the exponentialbackoffmultipler until the exponentialMaxWaitDuration
@Nullable
private Boolean enableExponentialBackoff;

private Double exponentialBackoffMultiplier;

private Duration exponentialMaxWaitDuration;

//Enable the random delay algorithm. The range is waitDuration - randomizedWaitFactor*waitDuration ~ waitDuration + randomizedWaitFactor*waitDuration
@Nullable
private Boolean enableRandomizedWait;

private Double randomizedWaitFactor;

@Nullable
private Boolean failAfterMaxAttempts;

By introducing the dependency of resilience4j-spring-boot2, you can configure all resilience4j components such as Retryer through Properties configuration, for example:

application.yml

resilience4j.retry:
  configs:
    default:
      ## Maximum number of retries, including the first call
      maxRetryAttempts: 2
      ## Retry wait time
      waitDuration: 500ms
      ## Enable random waiting time. The range is waitDuration - randomizedWaitFactor*waitDuration ~ waitDuration + randomizedWaitFactor*waitDuration
      enableRandomizedWait: true
      randomizedWaitFactor: 0.5
    test-client1:
      ## Maximum number of retries, including the first call
      maxRetryAttempts: 3
      ## Retry wait time
      waitDuration: 800ms
      ## Enable random waiting time. The range is waitDuration - randomizedWaitFactor*waitDuration ~ waitDuration + randomizedWaitFactor*waitDuration
      enableRandomizedWait: true
      randomizedWaitFactor: 0.5  

In this way, we can obtain the Retryer corresponding to the configuration through the following code:

@Autowired
RetryRegistry retryRegistry;
//Read the configuration under resilience4j.Retry.configurations.test-client1 and build a Retry, which is named retry1
Retry retry1 = retryRegistry.retry("retry1", "test-client1");
//Read the configuration under resilience4j.Retry.configurations.default and build a Retry. This Retry is named retry1
//If the configuration name is not specified, the configuration under default is used
Retry retry2 = retryRegistry.retry("retry2");

Introducing the dependency of resilience4j-spring-cloud2 is equivalent to introducing the dependency of resilience4j-spring-boot2. On the basis of it, the dynamic refresh RefreshScope mechanism for spring cloud config increases compatibility.

<dependency>
    <groupId>io.github.resilience4j</groupId>
    <artifactId>resilience4j-spring-cloud2</artifactId>
</dependency>

Add retry to OpenFeign using resilience4j feign

The official provides a dependency library for bonding OpenFeign, namely resilience4j feign

<dependency>
    <groupId>io.github.resilience4j</groupId>
    <artifactId>resilience4j-feign</artifactId>
</dependency>

Next, we use this dependency to add retry to OpenFeign. First, enable OpenFeign Client and specify the default configuration:

OpenFeignAutoConfiguration

@EnableFeignClients(value = "com.github.jojotech", defaultConfiguration = DefaultOpenFeignConfiguration.class)

In this default configuration, glue resilience4j add retry by overriding the default Feign.Builder:

@Bean
public Feign.Builder resilience4jFeignBuilder(
        List<FeignDecoratorBuilderInterceptor> feignDecoratorBuilderInterceptors,
        FeignDecorators.Builder builder
) {
    feignDecoratorBuilderInterceptors.forEach(feignDecoratorBuilderInterceptor -> feignDecoratorBuilderInterceptor.intercept(builder));
    return Resilience4jFeign.builder(builder.build());
}

@Bean
public FeignDecorators.Builder defaultBuilder(
        Environment environment,
        RetryRegistry retryRegistry
) {
    String name = environment.getProperty("feign.client.name");
    Retry retry = null;
    try {
        retry = retryRegistry.retry(name, name);
    } catch (ConfigurationNotFoundException e) {
        retry = retryRegistry.retry(name);
    }

    //Override the exception judgment and retry only for feign.RetryableException. All exceptions that need to be retried are encapsulated as RetryableException in DefaultErrorDecoder and Resilience4jFeignClient
    retry = Retry.of(name, RetryConfig.from(retry.getRetryConfig()).retryOnException(throwable -> {
        return throwable instanceof feign.RetryableException;
    }).build());

    return FeignDecorators.builder().withRetry(
            retry
    );
}

WeChat search "my programming meow" attention to the official account, daily brush, easy to upgrade technology, and capture all kinds of offer: