1. Overview
1-1. Problems faced by distributed systems
In the complex distributed architecture system, the mutual call between services can not avoid failure, which will lead to service avalanche. If the call link of a service is long, the probability of call failure will be higher. For example, a calls B, B calls C, C calls D, or the call of a service needs to be completed by multiple services, and the probability of call failure will also be higher. For example, a needs to call B, C, D services, and only B, C, and D services are completed, A service can be completed. This is the so-called "fan out".
If the response time of a service call on the fan out link is too long or unavailable, the call to the current module will occupy more and more resources, and even cause system crash. When a module has a problem, if the module still accepts the request and calls other modules, it will lead to cascade failure and system avalanche effect. This is the so-called "avalanche effect" of the system.
For a high traffic application, a single back-end dependency may lead to the saturation of all resources on all servers in a very short time. Worse, the application may also lead to the increase of delay between services and the shortage of resources such as queues and threads, resulting in more failures of the whole system.
In order to avoid such a situation, it is necessary to isolate faults and delays, so as to avoid dragging down the normal operation of other services due to the problem of one service. We need a thorough scheme or a scheme of link interruption, the so-called service degradation.
1-2. What is Hystrix
Hystrix is an Open-Source Library for dealing with delay and fault tolerance of distributed systems. In distributed systems, many dependencies inevitably fail to call, such as common timeout and program exception error reporting. Hystrix can ensure that when a dependency fails, it will not lead to overall service failure and avoid cascading failures, So as to improve the elasticity of distributed system.
"Circuit breaker" itself is a kind of switching device. When a service fails, it returns an expected and treatable alternative response (FallBack) to the caller through the fault monitoring of the circuit breaker (similar to fuse blowing), rather than waiting for a long time or throwing an exception that the caller cannot handle, This ensures that the thread of the service caller will not be occupied unnecessarily for a long time, so as to avoid the spread and even avalanche of faults in the distributed system.
1-3. What can Hystrix do
Service degradation, service fusing, current limiting, isolation and near real-time monitoring.
1-4. Hystrix official website
https://github.com/Netflix/Hystrix/wiki/How-To-Use
from https://github.com/Netflix/Hystrix It can be seen from the information of Hystrix Status that hystrix is no longer being developed. Judging from the update of github, it has not been updated for a long time. Hystrix recommends using Resilience4j instead. However, few large factories in China use Resilience4j, and more large factories use Spring Cloud Alibaba Sentinel to realize fusing and current limiting, Or it can be packaged on the existing open source framework to realize its own functions.
Although Hystrix is no longer updated, the design concept of Hystrix is worth learning and paves the way for learning Spring Cloud Alibaba Sentinel later.
2. Key concepts of Hystrix
2-1. Service degradation (fallback)
When the program is abnormal, timeout, service fusing, thread pool and semaphore is full, service degradation will occur. For example, the service will return a friendly prompt. Please try again later. Do not let the client wait all the time and return a good prompt immediately.
2-2. Service break
When the number of service requests reaches a certain limit, the service can be blown, which is similar to the fuse at home. When there is a high current, the fuse will be triggered. For example, at this time, the service is directly inaccessible, and the subsequent service degradation can be used to immediately return a response to the client (service degradation - > fusing - > recovering the calling link).
2-3. Service flow limit
It is often used to kill high concurrent requests at a certain time. For example, in the second kill phase, in order not to kill the service, some requests will be limited, such as putting them into the queue and waiting temporarily.
3. Hystrix case
3-1. Construction project
First, change cloud Eureka server7001 and cloud Eureka server7002 to stand-alone version, and then the sequence will start faster. Modify the value of defaultZone to point to its own service address instead of registering with each other.
# Configure service port number
server:
port: 7001
# Configure eureka
eureka:
instance:
# hostname: localhost # Eureka server instance name (stand-alone mode)
hostname: eureka7001.com # Eureka server instance name (cluster mode)
client:
register-with-eureka: false # Indicates that you do not register yourself with the container center
fetch-registry: false # It means that it is the registry. Its responsibility is to maintain service instances, and there is no need to retrieve services
service-url:
# Set the address to interact with Eureka Server. Both query service and registration service need to rely on this address
defaultZone: http://${eureka.instance.hostname}:${server.port}/eureka / # (stand-alone mode)
# defaultZone: http://Eureka 7002. Com: 7002 / Eureka / # mutual registration and mutual watch (cluster mode)
# server:
# enable-self-preservation: false # Turn off self-protection mechanism (default true)
# eviction-interval-timer-in-ms: 2000 # The interval between cleaning up the microservice, in milliseconds
# Configure service port number
server:
port: 7002
# Configure eureka
eureka:
instance:
# hostname: localhost # Eureka server instance name (stand-alone mode)
hostname: eureka7002.com # Eureka server instance name (cluster mode)
client:
register-with-eureka: false # Indicates that you do not register yourself with the container center
fetch-registry: false # It means that it is the registry. Its responsibility is to maintain service instances, and there is no need to retrieve services
service-url:
# Set the address to interact with Eureka Server. Both query service and registration service need to rely on this address
defaultZone: http://${eureka.instance.hostname}:${server.port}/eureka / # (stand-alone mode)
# defaultZone: http://eureka7001.com:7001/eureka / # mutual registration and mutual watch (cluster mode)
Create a new cloud provider hystrix payment8001 module, modify pom.xml, and add spring cloud starter Netflix hystrix dependency here.
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<!-- Import parent module name -->
<parent>
<artifactId>cloud2020</artifactId>
<groupId>com.king.springcloud</groupId>
<version>1.0-SNAPSHOT</version>
</parent>
<modelVersion>4.0.0</modelVersion>
<!-- Sub module name -->
<artifactId>cloud-provider-hystrix-payment8001</artifactId>
<dependencies>
<!-- introduce hystrix Fuse dependency -->
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-starter-netflix-hystrix</artifactId>
</dependency>
<!-- quote eureka-client Register service client dependencies -->
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-starter-netflix-eureka-client</artifactId>
</dependency>
<!-- quote cloud-api-common Common module -->
<dependency>
<groupId>com.king.springcloud</groupId>
<artifactId>cloud-api-commons</artifactId>
<version>${project.version}</version>
</dependency>
<!-- Reference parent spring boot Dependence of -->
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-web</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-actuator</artifactId>
</dependency>
<!-- Reference parent mybatis Follow spring boot Dependence of -->
<dependency>
<groupId>org.mybatis.spring.boot</groupId>
<artifactId>mybatis-spring-boot-starter</artifactId>
</dependency>
<!-- Reference parent druid Alibaba connection pool -->
<dependency>
<groupId>com.alibaba</groupId>
<artifactId>druid-spring-boot-starter</artifactId>
</dependency>
<!-- Reference parent mysql Dependence of -->
<dependency>
<groupId>mysql</groupId>
<artifactId>mysql-connector-java</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-jdbc</artifactId>
</dependency>
<!-- Configure hot deployment -->
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-devtools</artifactId>
<scope>runtime</scope>
<optional>true</optional>
</dependency>
<!-- Reference parent lombok rely on -->
<dependency>
<groupId>org.projectlombok</groupId>
<artifactId>lombok</artifactId>
<optional>true</optional>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-test</artifactId>
<scope>test</scope>
</dependency>
</dependencies>
</project>
Add the application.yml configuration file.
# Configure service port number
server:
port: 8001
# Configure application information
spring:
application:
name: cloud-provider-hystrix-payment # Configure app name
datasource:
type: com.alibaba.druid.pool.DruidDataSource # Data source type
driver-class-name: com.mysql.cj.jdbc.Driver # Database driven
url: jdbc:mysql://localhost:3306/cloud_ DB_ 2020? Useunicode = true & charcatencoding = UTF-8 & usessl = false & servertimezone = Asia / Shanghai # database connection
username: root # Database user name
password: rootroot # Database password
# Configure eureka
eureka:
instance:
hostname: eureka7001.com # Eureka server instance name
client:
register-with-eureka: true # Means to register yourself with the container center
fetch-registry: true # It means that it is the registry. Its responsibility is to maintain service instances and retrieve services
service-url:
# Set the address to interact with Eureka Server. Both query service and registration service need to rely on this address
defaultZone: http://${eureka.instance.hostname}:${server.port}/eureka/
# mybatis configuration
mybatis:
mapper-locations: classpath:mapper/*.xml # Location of mapper file
type-aliases-package: com.king.springcloud.entities # Package (alias) where all entity classes are located
Create the main startup class.
package com.king.springcloud;
import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;
import org.springframework.cloud.netflix.eureka.EnableEurekaClient;
/**
* Main startup class of micro service provider service side payment module
* @EnableEurekaClient: Indicates that it is an Eureka service registry client
*/
@EnableEurekaClient
@SpringBootApplication
public class PaymentHystrixMain8001 {
public static void main(String[] args) {
SpringApplication.run(PaymentHystrixMain8001.class, args);
}
}
Create a business class.
package com.king.springcloud.service;
import org.springframework.stereotype.Service;
@Service
public class PaymentService {
/**
* Simulation: normal method
* @param id
* @return
*/
public String paymentInfo_OK(Integer id) {
// The name of the current thread pool
return "Thread pool:" + Thread.currentThread().getName() + "\tpaymentInfo_OK, id=" + id;
}
/**
* Simulation: business complex calling timeout exception method
* @param id
* @return
*/
public String paymentInfo_TimeOut(Integer id) {
int time = 3000;
try {
Thread.sleep(time);
} catch (InterruptedException e) {
e.printStackTrace();
}
return "Thread pool:" + Thread.currentThread().getName() + "\tpaymentInfo_TimeOut, id=" + id + "\t，Time (seconds):" + time;
}
}
Create a control class.
package com.king.springcloud.controller;
import com.king.springcloud.service.PaymentService;
import lombok.extern.slf4j.Slf4j;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.PathVariable;
import org.springframework.web.bind.annotation.RestController;
import javax.annotation.Resource;
@Slf4j
@RestController
public class PaymentController {
@Resource
private PaymentService paymentService;
@Value("${server.port}")
private String serverPort;
/**
* Simulation: normal method
* @param id
* @return
*/
@GetMapping("/payment/hystrix/ok/{id}")
public String paymentInfo_OK(@PathVariable("id") Integer id) {
String result = paymentService.paymentInfo_OK(id);
log.info("result={}", result);
return result;
}
/**
* Simulation: business complex calling timeout exception method
* @param id
* @return
*/
@GetMapping("/payment/hystrix/timeout/{id}")
public String paymentInfo_TimeOut(@PathVariable("id") Integer id) throws InterruptedException {
String result = paymentService.paymentInfo_TimeOut(id);
log.info("result={}", result);
return result;
}
}
Start the cloud Eureka server7001 module first, and then the cloud provider hystrix payment8001 module, which can be accessed through the browser http://eureka7001.com:7001/ It can be seen that the cloud provider hystrix payment8001 module has been registered:
Revisit http://localhost:8001/payment/hystrix/ok/1 and http://localhost:8001/payment/hystrix/timeout/1 Check the effect. At this time, both requests can be accessed normally, except paymentinfo_ The request of the timeout () method should be slower. Next, we will expand based on these two requests to demonstrate the functions in Hystrix.
3-2. High concurrency test
Test with Apache Jmeter, a high concurrency testing tool.
Right click the test plan → add → thread (user) → thread group, add a thread group named Spring Cloud Hystrix, the number of threads is 200, the ramp up time is 1, the number of cycles is 100, and the other parameters can be kept by default, that is, 20000 concurrent threads. Save the thread group. Right click the Spring Cloud Hystrix thread group → add → sampler → HTTP request, name it cloud provider hystrix payment8001, server name or IP is localhost, port number is 8001, HTTP request is GET request, path is http://localhost:8001/payment/hystrix/timeout/1 , click Save to try the timeout request first. Click the green arrow in the menu bar to initiate the request. At this point, you can access it through the browser http://localhost:8001/payment/hystrix/ok/1 You'll find that it's also slowed down.
Create a new cloud-consumer-feign-hystrix-order80 module and deepen the simulation scenario. Hystrix has always been used on the consumer side. Modify pom.xml.
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<!-- Import parent module name -->
<parent>
<artifactId>cloud2020</artifactId>
<groupId>com.king.springcloud</groupId>
<version>1.0-SNAPSHOT</version>
</parent>
<modelVersion>4.0.0</modelVersion>
<!-- Sub module name -->
<artifactId>cloud-consumer-feign-hystrix-order80</artifactId>
<dependencies>
<!-- introduce spring cloud openfeign rely on -->
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-starter-openfeign</artifactId>
</dependency>
<!-- introduce hystrix Fuse dependency -->
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-starter-netflix-hystrix</artifactId>
</dependency>
<!-- quote eureka-client Register service client dependencies -->
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-starter-netflix-eureka-client</artifactId>
</dependency>
<!-- quote cloud-api-common Common module -->
<dependency>
<groupId>com.king.springcloud</groupId>
<artifactId>cloud-api-commons</artifactId>
<version>${project.version}</version>
</dependency>
<!-- Reference parent spring boot Dependence of -->
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-web</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-actuator</artifactId>
</dependency>
<!-- Reference parent mybatis Follow spring boot Dependence of -->
<dependency>
<groupId>org.mybatis.spring.boot</groupId>
<artifactId>mybatis-spring-boot-starter</artifactId>
</dependency>
<!-- Reference parent druid Alibaba connection pool -->
<dependency>
<groupId>com.alibaba</groupId>
<artifactId>druid-spring-boot-starter</artifactId>
</dependency>
<!-- Reference parent mysql Dependence of -->
<dependency>
<groupId>mysql</groupId>
<artifactId>mysql-connector-java</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-jdbc</artifactId>
</dependency>
<!-- Configure hot deployment -->
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-devtools</artifactId>
<scope>runtime</scope>
<optional>true</optional>
</dependency>
<!-- Reference parent lombok rely on -->
<dependency>
<groupId>org.projectlombok</groupId>
<artifactId>lombok</artifactId>
<optional>true</optional>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-test</artifactId>
<scope>test</scope>
</dependency>
</dependencies>
</project>
Add the application.yml configuration file.
# Configure service port number
server:
port: 80
# Configure application information
spring:
application:
name: cloud-consumer-feign-hystrix-order # Configure app name
datasource:
type: com.alibaba.druid.pool.DruidDataSource # Data source type
driver-class-name: com.mysql.cj.jdbc.Driver # Database driven
url: jdbc:mysql://localhost:3306/cloud_ DB_ 2020? Useunicode = true & charcatencoding = UTF-8 & usessl = false & servertimezone = Asia / Shanghai # database connection
username: root # Database user name
password: rootroot # Database password
# Configure eureka
eureka:
instance:
hostname: eureka7001.com # Eureka server instance name
client:
register-with-eureka: true # Means to register yourself with the container center
fetch-registry: true # It means that it is the registry. Its responsibility is to maintain service instances and retrieve services
service-url:
# Set the address to interact with Eureka Server. Both query service and registration service need to rely on this address
defaultZone: http://${eureka.instance.hostname}:7001/eureka/
# mybatis configuration
mybatis:
mapper-locations: classpath:mapper/*.xml # Location of mapper file
type-aliases-package: com.king.springcloud.entities # Package (alias) where all entity classes are located
Add the main startup class.
package com.king.springcloud;
import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;
import org.springframework.cloud.netflix.eureka.EnableEurekaClient;
import org.springframework.cloud.openfeign.EnableFeignClients;
/**
* Consumer client order module main startup class
* @EnableEurekaClient: Indicates that it is an Eureka service registry client
* @EnableFeignClients: Tell the framework to scan all feign clients defined with the annotation @ FeignClient and add the feign client to the spring IOC container
*/
@EnableFeignClients
@EnableEurekaClient
@SpringBootApplication
public class OrderHystrixMain80 {
public static void main(String[] args) {
SpringApplication.run(OrderHystrixMain80.class, args);
}
}
Add a business class.
package com.king.springcloud.service;
import org.springframework.cloud.openfeign.FeignClient;
import org.springframework.stereotype.Component;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.PathVariable;
/**
* @FeignClient: If the service provider has registered in the registry, the value of name or value is: the service name of the service provider.
*/
@Component
@FeignClient(value = "CLOUD-PROVIDER-HYSTRIX-PAYMENT")
public interface PaymentHystrixService {
/**
* Call the paymentinfo of the cloud-provider-hierarchy-payment module_ OK method
* @param id
* @return
*/
@GetMapping("/payment/hystrix/ok/{id}")
public String paymentInfo_OK(@PathVariable("id") Integer id);
/**
* Call the paymentinfo of the cloud-provider-hierarchy-payment module_ Timeout method
* @param id
* @return
*/
@GetMapping("/payment/hystrix/timeout/{id}")
public String paymentInfo_TimeOut(@PathVariable("id") Integer id);
}
Add a control class.
package com.king.springcloud.controller;
import com.king.springcloud.service.PaymentHystrixService;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.PathVariable;
import org.springframework.web.bind.annotation.RestController;
import javax.annotation.Resource;
@RestController
public class OrderHystrixController {
@Resource
PaymentHystrixService paymentHystrixService;
@GetMapping("/consumer/payment/hystrix/ok/{id}")
public String paymentInfo_OK(@PathVariable("id") Integer id) {
return paymentHystrixService.paymentInfo_OK(id);
}
@GetMapping("/consumer/payment/hystrix/timeout/{id}")
public String paymentInfo_TimeOut(@PathVariable("id") Integer id) {
return paymentHystrixService.paymentInfo_TimeOut(id);
}
}
Start cloud Eureka server7001 service registry, cloud provider hystrix payment8001 service provider and cloud consumer feign hystrix order80 service consumer module successively.
Browser access http://eureka7001.com:7001/ , http://localhost:8001/payment/hystrix/ok/1 , http://localhost:8001/payment/hystrix/timeout/1 , http://localhost/consumer/payment/hystrix/ok/1 Can be accessed normally when accessing http://localhost/consumer/payment/hystrix/timeout/1 The prompt timed out because of feign. Feign's default connection timeout is 1 second, The default read resource timeout is 1 second, which is ignored for the time being. Of course, it can also be based on the previous article OpenFeign notes , modify the timeout.
At this time, start Apache JMeter to http://localhost:8001/payment/hystrix/timeout/1 Request to send high concurrency 20000 access through the browser http://localhost/consumer/payment/hystrix/ok/1 , you will find that the direct timeout occurs. At this time, the simulated situation is: a large number of requests hit the producer. At this time, another external request through Feign will timeout, This request is made when the pressure test is not run http://localhost/consumer/payment/hystrix/ok/1 The response time is still very fast.
Reason: the 8001 service interface is occupied by a large number of requests, and the processing speed can not keep up. Tomcat's working threads have been occupied, and later arriving threads can only wait, which leads to the browser accessing the requests that should be returned immediately http://localhost/consumer/payment/hystrix/ok/1 Timeout occurred.
It is because of such failures or poor conditions that we need degradation, fault tolerance, current limiting and other technologies to deal with this problem.
- Producer 8001 has timed out, and consumer 80 cannot wait all the time. There must be service degradation
- Producer 8001 is down. Consumer 80 can't wait all the time. There must be service degradation
- The producer 8001 is normal, the consumer 80 has its own failure or the required waiting time is less than the producer's processing time, and the consumer 80 handles the service degradation by itself
3-3. Service degradation
First, transform from the service producer to find the paymentinfo in the service business layer of the cloud provider hystrix payment8001 service provider_ Timeout () method, because the execution of this method is a little long, we add a standard to it. When the method execution does not meet the standard requirements, we will make the target method fail quickly, and then execute the method of service degradation, and no longer wait for the return of the target method.
package com.king.springcloud.service;
import com.netflix.hystrix.contrib.javanica.annotation.HystrixCommand;
import com.netflix.hystrix.contrib.javanica.annotation.HystrixProperty;
import org.springframework.stereotype.Service;
@Service
public class PaymentService {
/**
* Simulation: normal method
* @param id
* @return
*/
public String paymentInfo_OK(Integer id) {
// The name of the current thread pool
return "Thread pool:" + Thread.currentThread().getName() + "\tpaymentInfo_OK, id=" + id;
}
/**
* Simulation: abnormal methods that do not meet service conditions
* @param id
* @HystrixCommand: The annotation indicates that when the target method does not meet the parameters specified by commandProperties, the current method is terminated, and then the bottom method specified by fallbackMethod is executed
* An error is reported when the execution.isolation.thread.timeoutInMilliseconds thread exceeds 3 seconds
* @return
*/
@HystrixCommand(fallbackMethod = "paymentInfo_TimeOutHandler", commandProperties = {
@HystrixProperty(name = "execution.isolation.thread.timeoutInMilliseconds", value = "3000")
})
public String paymentInfo_TimeOut(Integer id) {
int time = 0;
try {
time = 5000; // Sleep for 5 seconds and let it report an error
// int a = 1 / 0;// When the program reports an error, it will also trigger the service degradation method
Thread.sleep(time);
} catch (InterruptedException e) {
e.printStackTrace();
}
return "Thread pool:" + Thread.currentThread().getName() + "\tpaymentInfo_TimeOut, id=" + id + "\t Time consuming:" + time;
}
/**
* When paymentinfo_ Paymentinfo is executed when the timeout method does not meet the service conditions_ Timeouthandler method
* @param id
* @return
*/
public String paymentInfo_TimeOutHandler(Integer id) {
return "Thread pool:" + Thread.currentThread().getName() + "\tpaymentInfo_TimeOutHandler Bottom covering method, id=" + id + "\t The service degradation method was run";
}
}
Then modify the main startup class of cloud provider hystrix payment8001 service provider.
package com.king.springcloud;
import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;
import org.springframework.cloud.client.circuitbreaker.EnableCircuitBreaker;
import org.springframework.cloud.netflix.eureka.EnableEurekaClient;
/**
* Main startup class of micro service provider service side payment module
* @EnableEurekaClient: Indicates that it is an Eureka service registry client
* @EnableCircuitBreaker: Indicates the function of opening the circuit breaker
*/
@EnableCircuitBreaker
@EnableEurekaClient
@SpringBootApplication
public class PaymentHystrixMain8001 {
public static void main(String[] args) {
SpringApplication.run(PaymentHystrixMain8001.class, args);
}
}
Start EurekaMain7001 and PaymentHystrixMain8001 modules and visit the browser http://localhost:8001/payment/hystrix/timeout/1 For the test, the method body business takes 5 seconds, and the caller can wait up to 3 seconds. When the time exceeds 3 seconds and has not returned, paymentinfo is directly executed_ Timeouthandler() method. On the browser side, we can see the specific output.
Above, we deal with the service degradation of service producers. Below, we deal with service consumers. It should be noted that service degradation can be placed on both producers and consumers, and more often on consumers.
Add the @ EnableHystrix annotation on the OrderHystrixMain80 main startup class.
package com.king.springcloud;
import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;
import org.springframework.cloud.netflix.eureka.EnableEurekaClient;
import org.springframework.cloud.netflix.hystrix.EnableHystrix;
import org.springframework.cloud.openfeign.EnableFeignClients;
/**
* Consumer client order module main startup class
* @EnableEurekaClient: Indicates that it is an Eureka service registry client
* @EnableFeignClients: Tell the framework to scan all feign clients defined with the annotation @ FeignClient and add the feign client to the spring IOC container
* @EnableCircuitBreaker: Indicates the function of opening the circuit breaker
* @EnableHystrix: It inherits the function of opening circuit breaker of @ EnableCricuitBreaker
*/
@EnableHystrix
@EnableFeignClients
@EnableEurekaClient
@SpringBootApplication
public class OrderHystrixMain80 {
public static void main(String[] args) {
SpringApplication.run(OrderHystrixMain80.class, args);
}
}
Cloud consumer feign hystrix order80 service consumer module business layer.
package com.king.springcloud.service;
import org.springframework.cloud.openfeign.FeignClient;
import org.springframework.stereotype.Component;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.PathVariable;
/**
* @FeignClient: If the service provider has registered in the registry, the value of name or value is: the service name of the service provider.
*/
@Component
@FeignClient(value = "CLOUD-PROVIDER-HYSTRIX-PAYMENT")
public interface PaymentHystrixService {
/**
* Call the paymentinfo of the cloud-provider-hierarchy-payment module_ OK method
* @param id
* @return
*/
@GetMapping("/payment/hystrix/ok/{id}")
public String paymentInfo_OK(@PathVariable("id") Integer id);
/**
* Call the paymentinfo of the cloud-provider-hierarchy-payment module_ Timeout method
* @param id
* @return
*/
@GetMapping("/payment/hystrix/timeout/{id}")
public String paymentInfo_TimeOut(@PathVariable("id") Integer id);
}
Cloud consumer feign hystrix order80 service consumer module control layer.
package com.king.springcloud.controller;
import com.king.springcloud.service.PaymentHystrixService;
import com.netflix.hystrix.contrib.javanica.annotation.HystrixCommand;
import com.netflix.hystrix.contrib.javanica.annotation.HystrixProperty;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.PathVariable;
import org.springframework.web.bind.annotation.RestController;
import javax.annotation.Resource;
@RestController
public class OrderHystrixController {
@Resource
PaymentHystrixService paymentHystrixService;
@GetMapping("/consumer/payment/hystrix/ok/{id}")
public String paymentInfo_OK(@PathVariable("id") Integer id) {
return paymentHystrixService.paymentInfo_OK(id);
}
/**
* Exception methods that do not meet service conditions
* @param id
* @HystrixCommand Annotation indicates that when the target method does not meet the parameters specified by commandProperties, the current method is terminated, and then the method specified by fallbackMethod is executed
* An error is reported when the execution.isolation.thread.timeoutInMilliseconds thread exceeds 2.5 seconds
* @return
*/
@GetMapping("/consumer/payment/hystrix/timeout/{id}")
@HystrixCommand(fallbackMethod = "paymentTimeOutFallbackMethod", commandProperties = {
@HystrixProperty(name = "execution.isolation.thread.timeoutInMilliseconds", value = "2500")
})
public String paymentInfo_TimeOut(@PathVariable("id") Integer id) {
// int a = 1 / 0; // When the program reports an error, it will also trigger the service degradation method
return paymentHystrixService.paymentInfo_TimeOut(id);
}
/**
* For the service degradation method, it should be noted that the service degradation method should be consistent with the method signature of the target method, that is, the parameters and return values should be consistent, otherwise it will prompt that the service degradation method cannot be found
* @param id
* @return
*/
public String paymentTimeOutFallbackMethod(@PathVariable("id") Integer id) {
return "Consumer 80 terminal, consumer calling producer busy, please wait and try again";
}
}
At this point, the service producer side, paymentinfo_ The service running time of timeout () method is 5S. After adding the service degradation policy, paymentinfo_ If the timeout() method cannot be executed within 3s, the service degradation method will be executed. The requirements of the service consumer are more stringent. The consumer can wait up to 2.5s. If it cannot return within 2.5s, the degradation method will be executed. Access via browser http://localhost/consumer/payment/hystrix/timeout/1 , in the browser, it is found that the service degradation method is triggered at more than 2.5 seconds. This is because we do not set the default timeout of the Ribbon (connection establishment timeout 1s, read resource timeout 1s) to 2s, which is less than 2.5 seconds set by HystrixProperty. The timeout of the Ribbon is triggered first, and the code reports an error, The degradation method is then triggered. We modify the application.yml configuration file and the Ribbon timeout.
# Configure service port number
server:
port: 80
# Configure application information
spring:
application:
name: cloud-consumer-feign-hystrix-order # Configure app name
datasource:
type: com.alibaba.druid.pool.DruidDataSource # Data source type
driver-class-name: com.mysql.cj.jdbc.Driver # Database driven
url: jdbc:mysql://localhost:3306/cloud_ DB_ 2020? Useunicode = true & charcatencoding = UTF-8 & usessl = false & servertimezone = Asia / Shanghai # database connection
username: root # Database user name
password: rootroot # Database password
# Configure eureka
eureka:
instance:
hostname: eureka7001.com # Eureka server instance name
client:
register-with-eureka: true # Means to register yourself with the container center
fetch-registry: true # It means that it is the registry. Its responsibility is to maintain service instances and retrieve services
service-url:
# Set the address to interact with Eureka Server. Both query service and registration service need to rely on this address
defaultZone: http://${eureka.instance.hostname}:7001/eureka/
# mybatis configuration
mybatis:
mapper-locations: classpath:mapper/*.xml # Location of mapper file
type-aliases-package: com.king.springcloud.entities # Package (alias) where all entity classes are located
# Set feign client timeout. OpenFeign supports Ribbon by default
ribbon:
ReadTimeout: 6000 # The timeout for processing a request after a connection is established. The default is 1 second
ConnectTimeout: 6000 # The timeout length of connection establishment, which is 1 second by default
MaxAutoRetries: 1 # The maximum number of retries for the same instance, excluding the first call, is 1 by default
MaxAutoRetriesNextServer: 0 # The maximum number of retries to retry other instances of load balancing, excluding the first call. The default is 0
OkToRetryOnAllOperations: false # Whether to retry all operations. The default is false
At this point, send again http://localhost/consumer/payment/hystrix/timeout/1 Request. Check the time through the browser's Network. It is found that the return time is more than 2.5 seconds. At this time, the 2.5 seconds set by HystrixProperty works. Moreover, the information output by the browser is the output of the degradation method in the client.
For another demonstration, change the time of the client's HystrixProperty to 3.5s, which is generally not set, because the 3.5s of the client is greater than the 3s of the server. Here is only a test and access through the browser http://localhost/consumer/payment/hystrix/timeout/1 , observe the time of the Network, 3S will return, and the content displayed by the browser, Is the output of the server degradation method.
Look at these timeout times. Take the minimum timeout time as the standard. As long as the minimum timeout time is reached, service degradation will be caused. The cause of service degradation is determined by the minimum timeout time.
When the service degradation is added to the producer, the service producer is called directly. The degradation is triggered by 3s of HystrixProperty in the producer Controller.
When the service degradation is added to the consumer, directly call the service consumer:
- If the default timeout of the Ribbon has not been modified and the consumer's call exceeds the default 2s, then min (2s of the Ribbon, the time specified by the consumer's HystrixProperty) triggers the service degradation. Whoever has a small value is the one who causes the service degradation.
- If the default timeout of the Ribbon is modified, it is assumed that the modified value is very large, excluding the service degradation caused by the Ribbon call timeout. Then, the trigger condition of service degradation is the time specified by the consumer's HystrixProperty. If the service producer also specifies the time in the HystrixProperty at this time, it is expressed in min (the time specified by the consumer's HystrixProperty and the time specified by the producer's HystrixProperty) as the degradation time, which value is small is the degradation triggered by which degradation method to take.
Looking back at the configured service degradation methods, if there are 10 methods that need service degradation, you need to configure 10 additional service degradation methods, which will lead to code expansion. Is there a way to handle service degradation with a general method and customize the rest? The combination of service degradation methods and business methods leads to confusion in the business class. Can you take out the degradation methods?
Optimize code expansion, add a global service degradation method, and add @ DefaultProperties(defaultFallback = "payment_Global_FallbackMethod") on the class, indicating that all methods with @ HystrixCommand do not write fallbackMethod property in this class. If there is no special indication on the method, this payment will be taken when the method is degraded_ Global_ fallbackMethod() method.
package com.king.springcloud.controller;
import com.king.springcloud.service.PaymentHystrixService;
import com.netflix.hystrix.contrib.javanica.annotation.DefaultProperties;
import com.netflix.hystrix.contrib.javanica.annotation.HystrixCommand;
import com.netflix.hystrix.contrib.javanica.annotation.HystrixProperty;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.PathVariable;
import org.springframework.web.bind.annotation.RestController;
import javax.annotation.Resource;
/**
* @DefaultProperties: Specify the global degradation method. If the fallbackMethod property is not specified in @ HystrixCommand, you will find the specified degradation method of @ DefaultProperties
*/
@DefaultProperties(defaultFallback = "payment_Global_FallbackMethod")
@RestController
public class OrderHystrixController {
@Resource
PaymentHystrixService paymentHystrixService;
@GetMapping("/consumer/payment/hystrix/ok/{id}")
public String paymentInfo_OK(@PathVariable("id") Integer id) {
return paymentHystrixService.paymentInfo_OK(id);
}
/**
* Exception methods that do not meet service conditions
* @param id
* @HystrixCommand Annotation indicates that when the target method does not meet the parameters specified by commandProperties, the current method is terminated, and then the method specified by fallbackMethod is executed
* An error is reported when the execution.isolation.thread.timeoutInMilliseconds thread exceeds 2.5 seconds
* @return
*/
@GetMapping("/consumer/payment/hystrix/timeout/{id}")
@HystrixCommand(fallbackMethod = "paymentTimeOutFallbackMethod", commandProperties = {
@HystrixProperty(name = "execution.isolation.thread.timeoutInMilliseconds", value = "2500")
})
public String paymentInfo_TimeOut(@PathVariable("id") Integer id) {
// int a = 1 / 0; // When the program reports an error, it will also trigger the service degradation method
return paymentHystrixService.paymentInfo_TimeOut(id);
}
/**
* For the service degradation method, it should be noted that the service degradation method should be consistent with the method signature of the target method, that is, the parameters and return values should be consistent, otherwise it will prompt that the service degradation method cannot be found
* @param id
* @return
*/
public String paymentTimeOutFallbackMethod(@PathVariable("id") Integer id) {
return "Consumer 80 terminal, consumer calling producer busy, please wait and try again";
}
/**
* The fallbackMethod property of @ HystrixCommand is not specified to test the global exception degradation method
* @param id
* @return
*/
@GetMapping("/consumer/payment/hystrix/global/timeout/{id}")
@HystrixCommand(commandProperties = {
@HystrixProperty(name = "execution.isolation.thread.timeoutInMilliseconds", value = "2500")
})
public String paymentInfo_Global_TimeOut(@PathVariable("id") Integer id) {
// int a = 1 / 0; // When the program reports an error, it will also trigger the service degradation method
return paymentHystrixService.paymentInfo_TimeOut(id);
}
/**
* Global exception degradation method
* @return
*/
public String payment_Global_FallbackMethod(){
return "Consumer end 80, Global Exception handling information, please try again later.";
}
}
Browser access http://localhost/consumer/payment/hystrix/global/timeout/1 , the service can still be degraded according to the timeout rules specified by us. When the service is degraded, the method executed is payment_Global_FallbackMethod().
The optimization code is chaotic, the service is degraded, the client calls the server, and the server is down or shut down. In order to realize decoupling, the fallback parameter in the @ FeignClient annotation can be used to realize decoupling. Create a new PaymentHystrixServiceImpl.java implementation class to implement the PaymentHystrixService interface and rewrite the methods in the interface. The two methods rewritten here are the methods used for service degradation execution. Finally, the PaymentHystrixService interface can decouple the business method from the degradation method by writing the fallback parameter in the @ FeignClient annotation.
PaymentHystrixService interface:
package com.king.springcloud.service;
import com.king.springcloud.service.impl.PaymentHystrixServiceImpl;
import org.springframework.cloud.openfeign.FeignClient;
import org.springframework.stereotype.Component;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.PathVariable;
/**
* @FeignClient: If the service provider has registered in the registry, the value of name or value is: the service name of the service provider. fallback: specify a method to realize the degradation of class services
*/
@Component
@FeignClient(value = "CLOUD-PROVIDER-HYSTRIX-PAYMENT",fallback = PaymentHystrixServiceImpl.class)
public interface PaymentHystrixService {
/**
* Call the paymentinfo of the cloud-provider-hierarchy-payment module_ OK method
* @param id
* @return
*/
@GetMapping("/payment/hystrix/ok/{id}")
public String paymentInfo_OK(@PathVariable("id") Integer id);
/**
* Call the paymentinfo of the cloud-provider-hierarchy-payment module_ Timeout method
* @param id
* @return
*/
@GetMapping("/payment/hystrix/timeout/{id}")
public String paymentInfo_TimeOut(@PathVariable("id") Integer id);
}
PaymentHystrixServiceImpl.java implementation class:
package com.king.springcloud.service.impl;
import com.king.springcloud.service.PaymentHystrixService;
import org.springframework.stereotype.Component;
/**
* Implement fallback s of all methods in the PaymentHystrixService interface
*/
@Component
public class PaymentHystrixServiceImpl implements PaymentHystrixService {
/**
* Implements the paymentinfo of the PaymentHystrixService interface_ fallBack method of ok() method
* @param id
* @return
*/
@Override
public String paymentInfo_OK(Integer id) {
return "PaymentHystrixServiceImpl fall back paymentInfo_OK Method";
}
/**
* Implements the paymentinfo of the PaymentHystrixService interface_ fallBack method of timeout() method
* @param id
* @return
*/
@Override
public String paymentInfo_TimeOut(Integer id) {
return "PaymentHystrixServiceImpl fall back paymentInfo_TimeOut Method";
}
}
Modify application.yml and open Hystrix. The default is false.
# Configure service port number
server:
port: 80
# Configure application information
spring:
application:
name: cloud-consumer-feign-hystrix-order # Configure app name
datasource:
type: com.alibaba.druid.pool.DruidDataSource # Data source type
driver-class-name: com.mysql.cj.jdbc.Driver # Database driven
url: jdbc:mysql://localhost:3306/cloud_ DB_ 2020? Useunicode = true & charcatencoding = UTF-8 & usessl = false & servertimezone = Asia / Shanghai # database connection
username: root # Database user name
password: rootroot # Database password
# Configure eureka
eureka:
instance:
hostname: eureka7001.com # Eureka server instance name
client:
register-with-eureka: true # Means to register yourself with the container center
fetch-registry: true # It means that it is the registry. Its responsibility is to maintain service instances and retrieve services
service-url:
# Set the address to interact with Eureka Server. Both query service and registration service need to rely on this address
defaultZone: http://${eureka.instance.hostname}:7001/eureka/
# mybatis configuration
mybatis:
mapper-locations: classpath:mapper/*.xml # Location of mapper file
type-aliases-package: com.king.springcloud.entities # Package (alias) where all entity classes are located
# Set feign client timeout. OpenFeign supports Ribbon by default
ribbon:
ReadTimeout: 6000 # The timeout for processing a request after a connection is established. The default is 1 second
ConnectTimeout: 6000 # The timeout length of connection establishment, which is 1 second by default
MaxAutoRetries: 1 # The maximum number of retries for the same instance, excluding the first call, is 1 by default
MaxAutoRetriesNextServer: 0 # The maximum number of retries to retry other instances of load balancing, excluding the first call. The default is 0
OkToRetryOnAllOperations: false # Whether to retry all operations. The default is false
# For service degradation, add the fallback attribute in the annotation @ FeignClient
feign:
hystrix:
enabled: true # Turn on the Hystrix fuse in Feign
First access through the browser http://localhost/consumer/payment/hystrix/ok/1 , you can get the value returned by the producer through the service producer. At this time, turn off the producer, simulate the downtime of the producer, and visit again http://localhost/consumer/payment/hystrix/ok/1 , you will see that it executes paymentinfo in the PaymentHystrixServiceImpl.java implementation class_ OK () method, now the service degradation method and the business method are separated.
Review the location of fallback:
- The @ DefaultProperties annotation on the control class OrderHystrixController defaults to fallback
- Paymentinfo of control class OrderHystrixController_ fallback set separately by @ HystrixCommand annotation on timeout() method
- The @ feignclient (value = "cloud-provider-hierarchy-payment", fallback = paymenthystrixserviceimpl. Class) annotation on the PaymentHystrixService interface of the business class implements the fallback of the class
fallback priority display: 3 > 2 > 1
3-4. Service fuse
The circuit breaker mechanism is a medium micro service link protection mechanism to deal with the avalanche effect. When a micro service with fan out filing rate makes an error or the response time is too long, it will degrade the service, and then fuse the call of the node's Micro service to quickly return the wrong response information. When it is detected that the microservice call response of the node is normal, the call link is restored.
In a distributed system, each microservice cannot guarantee its own operation and will never have problems. When microservices call each other, there may be problems with dependent services due to various reasons (network exceptions, dependent services hang up, problems relying on the service itself). Fusing mainly solves the above problems. When the service on which a service depends is unavailable, the service will not be affected, so as to prevent it from collapsing itself, resulting in the unavailability of more micro services (isolate the bad services), and provide protection and control for the calls between services.
Think of the fuse as a fuse. In the circuit system, a fuse is usually added in the middle of all the lines connected to the external power supply of the home appliance system. When the external voltage is too high and reaches the melting point of the fuse, the fuse will be blown, so as to cut off the connection between the home appliance system and the external circuit, So as to ensure that the home appliance system will not be damaged due to high voltage.
Fuses are components located in front of the thread pool. After a user requests a service, Hystrix will pass through the fuse first. At this time, if the fuse is open, it will be degraded directly and will not continue to send the request to the thread pool. The fuse is equivalent to a barrier in front of the process pool. Each fuse maintains 10 buckets by default and creates one bucket per second. Each bucket records the times of success, failure, timeout and rejection. When a new bucket is created, the oldest bucket will be discarded.
The fuse mode defines the logic of mutual conversion of fuse switches:
Three states:
- Open: open the fuse, that is, the service caller executes the local degradation policy and does not make remote calls.
- Closed: when the fuse is closed, the service caller directly initiates the remote call.
- Half open: an intermediate (half open) state that allows quantitative service requests to directly initiate remote calls.
Health of service = number of failed requests / total number of requests
The state transition of the fuse switch from off to on is determined by comparing the current service health status with the set threshold.
The switch of the fuse can ensure that the service caller can quickly return the result when calling the abnormal service, avoid a large number of synchronous waiting, and the fuse can continue to detect the request execution result after a period of time, providing the possibility of restoring the service call.
The fuse provided by hystrix has a similar function. When the number of times the service caller calls the service of the service provider reaches the set threshold within a certain period of time, and the number of errors also reaches the set error threshold, the service will be degraded, allowing the service callers to implement the locally set degradation strategy instead of initiating remote calls. However, the fuse provided by hystrix has the function of self feedback and self recovery. Hystrix will automatically switch the fuse between open, closed and half open according to the calling interface.
- Closed - > Open: normally, the fuse is closed. If the current health status is higher than the set threshold, it remains closed. If the current health status is below the set threshold, switch to open.
- Open - > half open: when the fuse corresponding to the service interface is open, all service callers execute the local degradation method when calling the service method. Hystrix provides a test strategy, that is, it sets a time window (generally set as the average fault handling time, i.e. MTTR). Within a time window from the fuse to the open state, it delegates the service degradation method to execute when calling the service interface. If the time exceeds the time window, change the fusing status from open - > half open
- Half open - > closed: when the fuse is in the half open state, quantitative service requests are allowed. If all (or a certain proportion) requests are called successfully, the fuse will be restored to closed. Otherwise, the fuse status will be open, the next request will be prohibited, and the start time of the time window will be recorded again.
Fuse and degradation
Common ground: ensure the availability of services, and then ensure the availability of the whole system.
difference:
The fusing is triggered by the unavailability of downstream services, so it is necessary to actively monitor the fusing.
Degradation is triggered by the upstream caller according to the availability of the downstream service and is passively abandoned.
Main functions:
- Prevent avalanche and provide protection mechanism for the system.
- Resource isolation: Hystrix maintains a separate thread pool for each requested downstream service. If the thread pool is full, the request to call the downstream service will immediately fail to prevent cascading failure.
- Circuit breaker: when the failure rate of calling a service reaches the threshold, Hystrix realizes the function of circuit breaker. The circuit breaker opens and suspends the request for the service for a period of time.
- Degradation: when the circuit breaker is open, directly request the fallback written by yourself to perform specific fallback degradation.
- Monitoring: Hystrix can monitor the changes of operation indicators and configurations in real time and give alarms to facilitate operation and maintenance.
- Self repair: when the circuit breaker is in the half open state, carry out service recovery (see the half open state in the circuit breaker state for details)
- Quick failure: when the Fail Fast error occurs, it directly returns the failure and does not request the real service.
In the Spring Cloud framework, the circuit breaker mechanism is implemented through hystrix. Hystrix will monitor the calls between microservices. When the failed calls reach a certain threshold, the circuit breaker mechanism will be triggered (20 failures in 5 seconds by default). The annotation of the circuit breaker mechanism is @ HystrixCommand.
Add a method in the PaymentService of cloud provider hystrix payment8001 service provider module to demonstrate the service.
PaymentService.java file:
package com.king.springcloud.service;
import com.netflix.hystrix.contrib.javanica.annotation.HystrixCommand;
import com.netflix.hystrix.contrib.javanica.annotation.HystrixProperty;
import org.springframework.stereotype.Service;
import org.springframework.web.bind.annotation.PathVariable;
import java.util.UUID;
@Service
public class PaymentService {
/*Here is the code that demonstrates service degradation*/
/**
* Simulation: normal method
* @param id
* @return
*/
public String paymentInfo_OK(Integer id) {
// The name of the current thread pool
return "Thread pool:" + Thread.currentThread().getName() + "\tpaymentInfo_OK, id=" + id;
}
/**
* Simulation: abnormal methods that do not meet service conditions
* @param id
* @HystrixCommand: The annotation indicates that when the target method does not meet the parameters specified by commandProperties, the current method is terminated, and then the bottom method specified by fallbackMethod is executed
* An error is reported when the execution.isolation.thread.timeoutInMilliseconds thread exceeds 3 seconds
* @return
*/
@HystrixCommand(fallbackMethod = "paymentInfo_TimeOutHandler", commandProperties = {
@HystrixProperty(name = "execution.isolation.thread.timeoutInMilliseconds", value = "3000")
})
public String paymentInfo_TimeOut(Integer id) {
int time = 0;
try {
time = 5000; // Sleep for 5 seconds and let it report an error
// int a = 1 / 0;// When the program reports an error, it will also trigger the service degradation method
Thread.sleep(time);
} catch (InterruptedException e) {
e.printStackTrace();
}
return "Thread pool:" + Thread.currentThread().getName() + "\tpaymentInfo_TimeOut, id=" + id + "\t Time consuming:" + time;
}
/**
* When paymentinfo_ Paymentinfo is executed when the timeout method does not meet the service conditions_ Timeouthandler method
* @param id
* @return
*/
public String paymentInfo_TimeOutHandler(Integer id) {
return "Thread pool:" + Thread.currentThread().getName() + "\tpaymentInfo_TimeOutHandler Bottom covering method, id=" + id + "\t The service degradation method was run";
}
/*Here is the code that demonstrates the service*/
/**
* Service fusing method
* The @ HystrixProperty configured here can be viewed by searching the HystrixCommandProperties abstract class
* @param id
* @return
*/
@HystrixCommand(fallbackMethod = "paymentCircuitBreakerFallback", commandProperties = {
@HystrixProperty(name = "circuitBreaker.enabled", value = "true"),// Is the circuit breaker open
// The number of requests in a rolling window. When the number of requests reaches 10, the failure rate is calculated to judge whether the open circuit condition is met. Judge whether the fuse is open or closed
@HystrixProperty(name = "circuitBreaker.requestVolumeThreshold", value = "10"),
// During the time window period, the circuit breaker will fail directly for 10s after triggering. After more than 10s, try to recover once. Fuse half open
@HystrixProperty(name = "circuitBreaker.sleepWindowInMilliseconds", value = "10000"),
// Error threshold. When the failure rate reaches 60%, open the circuit. Fuse open status
@HystrixProperty(name = "circuitBreaker.errorThresholdPercentage", value = "60")
})
public String paymentCircuitBreaker(@PathVariable("id") Integer id) {
if (id < 0) {
// The simulation service is unavailable
throw new RuntimeException("id Cannot be negative");
}
String uuid = UUID.randomUUID().toString();
return Thread.currentThread().getName() + "\t Call succeeded, Serial No UUID=" + uuid;
}
/**
* paymentCircuitBreaker Service degradation method
* @param id
* @return
*/
public String paymentCircuitBreakerFallback(@PathVariable("id") Integer id) {
return "id Cannot be negative. Please try again later, id=" + id;
}
}
PaymentController.java file:
package com.king.springcloud.controller;
import com.king.springcloud.service.PaymentService;
import lombok.extern.slf4j.Slf4j;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.PathVariable;
import org.springframework.web.bind.annotation.RestController;
import javax.annotation.Resource;
@Slf4j
@RestController
public class PaymentController {
@Resource
private PaymentService paymentService;
@Value("${server.port}")
private String serverPort;
/*Here is the code that demonstrates service degradation*/
/**
* Simulation: normal method
* @param id
* @return
*/
@GetMapping("/payment/hystrix/ok/{id}")
public String paymentInfo_OK(@PathVariable("id") Integer id) {
String result = paymentService.paymentInfo_OK(id);
log.info("result={}", result);
return result;
}
/**
* Simulation: business complex calling timeout exception method
* @param id
* @return
*/
@GetMapping("/payment/hystrix/timeout/{id}")
public String paymentInfo_TimeOut(@PathVariable("id") Integer id) {
String result = paymentService.paymentInfo_TimeOut(id);
log.info("result={}", result);
return result;
}
/*Here is the code that demonstrates the service*/
/**
* Service fusing method
* @param id
* @return
*/
@GetMapping("/payment/circuit/{id}")
public String paymentCircuitBreaker(@PathVariable("id") Integer id) {
String result = paymentService.paymentCircuitBreaker(id);
log.info("result={}", result);
return result;
}
}
Start Eureka main7001 registry module and PaymentHystrixMain8001 service provider module. Browser send http://localhost:8001/payment/circuit/1 Request, you can get the correct return value.
Trying to send a large number of http://localhost:8001/payment/circuit/-1 request, you can get the value returned by the service degradation method.
Send another http://localhost:8001/payment/circuit/1 Request. For this normal request, the method of service degradation is still adopted, indicating that the circuit breaker is open at this time, because the error rate has reached the set error threshold and has not been recovered. After sending several correct requests again, the fuse will be closed if the error rate is lower than the set error threshold, Normal requests can be processed.
In case of fusing, the fusing time will be recorded. According to the configuration here, the request will go through the method of service degradation in the following 10s. After 10s, try to call the service. If it can be adjusted, restore the calling link. If it cannot be adjusted, continue to maintain the fusing and record the time again.
There are three types of fusing: fusing open, fusing half open and fusing closed.
- Fusing on: when the fusing occurs in Hystrix, the new request is not calling the service, but directly running the service degradation method to record the fusing time (the built-in setting clock is generally the MTTR mean fault processing time). When the fusing duration reaches the set sleepWindowInMilliseconds, the fusing is half open
- Fuse half open: some requests make service calls. If the call is successful, the fuse will be closed. If the call fails, the fuse will continue to be open and the time will be recorded again
- Switch off: call the service normally
@HystrixCommand all configurations:
// @HystrixCommand(fallbackMethod = "str_fallbackMethod",groupKey = "strGroupCommand", commandKey = "strCommand",threadPoolKey = "strThreadPool",
// commandProperties = {
// Set the isolation policy. THREAD indicates THREAD pool. SEMAPHORE: signal pool isolation
// @HystrixProperty(name = "execution.isolation.strategy", value = "THREAD"),
// When the isolation strategy selects signal pool isolation, it is used to set the size of the signal pool (maximum concurrent number)
// @HystrixProperty(name = "execution.isolation.semaphore.maxConcurrentRequests", value = "10"),
// Configure the timeout for command execution
// @HystrixProperty(name = "execution.isolation.thread.timeoutinMilliseconds", value = "10"),
// Enable timeout
// @HystrixProperty(name = "execution.timeout.enabled", value = "true"),
// Is the execution interrupted when it times out
// @HystrixProperty(name = "execution.isolation.thread.interruptOnTimeout", value = "true"),
// Is the execution interrupted when it is cancelled
// @HystrixProperty(name = "execution.isolation.thread.interruptOnCancel", value = "true"),
// Maximum concurrent number of callback method executions allowed
// @HystrixProperty(name = "fallback.isolation.semaphore.maxConcurrentRequests", value = "10"),
// Whether the service degradation is enabled and whether the callback function is executed
// @HystrixProperty(name = "fallback.enabled", value = "true"),
// Is the circuit breaker enabled
// @HystrixProperty(name = "circuitBreaker.enabled", value = "true"),
// This attribute is used to set the minimum number of requests for circuit breaker fusing in the rolling time window. For example, when the default value is 20,
// If only 19 requests are received within the rolling time window (default 10 seconds), the circuit breaker will not open even if all 19 requests fail.
// @HystrixProperty(name = "circuitBreaker.requestVolumeThreshold", value = "20"),
// This attribute is used to set in the rolling time window, which means that when the number of requests exceeds circuitBreaker.requestVolumeThreshold, if the percentage of wrong requests exceeds 50, the circuit breaker will be set to "on", otherwise it will be set to "off".
// @HystrixProperty(name = "circuitBreaker.errorThresholdPercentage", value = "50"),
// This attribute is used to set the sleep time window after the circuit breaker is opened. After the sleep time window ends, the circuit breaker will be set to the "half open" state and the request command to try to fuse. If it still fails, the circuit breaker will continue to be set to the "open" state, and if it succeeds, it will be set to the "closed" state.
// @HystrixProperty(name = "circuitBreaker.sleepWindowinMilliseconds", value = "5000"),
// Forced opening of circuit breaker
// @HystrixProperty(name = "circuitBreaker.forceOpen", value = "false"),
// Forced closing of circuit breaker
// @HystrixProperty(name = "circuitBreaker.forceClosed", value = "false"),
// Rolling time window setting, which is used for the duration of information to be collected when judging the health of the circuit breaker
// @HystrixProperty(name = "metrics.rollingStats.timeinMilliseconds", value = "10000"),
// This attribute is used to set the number of "buckets" divided when rolling time window statistics indicator information. When collecting indicator information, the circuit breaker will
// The set time window length is divided into multiple "buckets" to accumulate each measurement value. Each "bucket" records the collection indicators over a period of time.
// For example, it can be divided into 10 "buckets" in 10 seconds, so timeinMilliseconds must be divisible by numBuckets. Otherwise, an exception will be thrown
// @HystrixProperty(name = "metrics.rollingStats.numBuckets", value = "10"),
// This property is used to set whether the delay in command execution is tracked and calculated using percentiles. If set to false, all summary statistics will return - 1.
// @HystrixProperty(name = "metrics.rollingPercentile.enabled", value = "false"),
// This property is used to set the duration of the scrolling window for percentile statistics, in milliseconds.
// @HystrixProperty(name = "metrics.rollingPercentile.timeInMilliseconds", value = "60000"),
// This attribute is used to set the number of buckets used in the percentile statistics scroll window.
// @HystrixProperty(name = "metrics.rollingPercentile.numBuckets", value = "60000"),
// This attribute is used to set the maximum number of executions to keep in each bucket during execution. If the number of executions exceeding the set value occurs within the rolling time window,
// Start rewriting from the original position. For example, set the value to 100 and scroll the window for 10 seconds. If 500 executions occur in a "bucket" within 10 seconds,
// Then only the statistics of the last 100 executions are retained in the "bucket". In addition, increasing the size of this value will increase the consumption of memory and the calculation time required to sort percentiles.
// @HystrixProperty(name = "metrics.rollingPercentile.bucketSize", value = "100"),
// This attribute is used to set the interval waiting time for collecting health snapshots (requested success, error percentage) that affect the status of the circuit breaker.
// @HystrixProperty(name = "metrics.healthSnapshot.intervalinMilliseconds", value = "500"),
// Enable request cache
// @HystrixProperty(name = "requestCache.enabled", value = "true"),
// Whether the execution and events of the HystrixCommand are printed into the HystrixRequestLog
// @HystrixProperty(name = "requestLog.enabled", value = "true"), },
// threadPoolProperties = {
// This parameter is used to set the number of core threads in the command execution thread pool, which is the maximum concurrency of command execution
// @HystrixProperty(name = "coreSize", value = "10"),
// This parameter is used to set the maximum queue size of the thread pool. When set to - 1, the thread pool will use the queue implemented by SynchronousQueue, otherwise it will use the queue implemented by LinkedBlockingQueue.
// @HystrixProperty(name = "maxQueueSize", value = "-1"),
// This parameter is used to set the reject threshold for the queue. With this parameter, the request can be rejected even if the queue does not reach the maximum value.
// This parameter is mainly a supplement to the LinkedBlockingQueue queue, because LinkedBlockingQueue
// The queue cannot dynamically modify its object size, but the size of the queue that rejects requests can be adjusted through this attribute.
// @HystrixProperty(name = "queueSizeRejectionThreshold", value = "5"),
// })
3-5. Service current limit
It will be described in the subsequent Sentinel of Alibaba.
4. Hystrix workflow
The whole process consists of 9 steps:
- Create a HystrixCommand or hystrixobservercommand object
- Execute Command
- Check whether there is a cache available. If so, directly return the contents of the cache. If not, go to step 4
- Check the opening and closing of the fuse. If it is opened, run the service degradation method and go to step 8. If it is not opened, go to step 5
- Check whether the thread pool, queue and semaphore are sufficient. If not, run the service degradation method and go to step 8. If sufficient, go to step 6
- Execute the construct() method of hystrixobservatlecommand or the run() method of HystrixCommand
- According to the execution results, judge whether the execution is successful and whether there is timeout, and feed back to the health judgment function in the circuit, so as to affect the opening and closing of the circuit breaker
- In case of fuse disconnection, insufficient resources, execution failure, execution timeout, etc., take the service degradation method to obtain the return value
- Everything is normal, the circuit breaker is closed, and the return value is obtained by calling the service normally
5. Hystrix Dashboard for graphical monitoring of services
In addition to isolating the calls of dependent services, hystrix also provides a quasi real-time call monitoring (Hystrix Dashboard). Hystrix will continuously record the execution information of all requests initiated through hystrix and display it to users in the form of statistical reports and graphics, including how many successful and failed requests are executed every second. Netflix monitors the above indicators through the hystrix metrics event stream project. Spring Cloud also provides the integration of the Hystrix Dashboard to convert the monitoring content into a visual interface.
Create a new cloud consumer hystrix dashboard 9001 module, modify pom.xml, and introduce spring cloud starter Netflix hystrix dashboard dependency.
pom.xml file:
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<!-- Import parent module name -->
<parent>
<artifactId>cloud2020</artifactId>
<groupId>com.king.springcloud</groupId>
<version>1.0-SNAPSHOT</version>
</parent>
<modelVersion>4.0.0</modelVersion>
<!-- Sub module name -->
<artifactId>cloud-consumer-hystrix-dashboard9001</artifactId>
<dependencies>
<!-- quote hystrix Graphical monitor dependencies -->
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-starter-netflix-hystrix-dashboard</artifactId>
</dependency>
<!-- actuator monitor -->
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-actuator</artifactId>
</dependency>
<!-- Reference parent spring boot Dependence of -->
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-web</artifactId>
</dependency>
<!-- Configure hot deployment -->
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-devtools</artifactId>
<scope>runtime</scope>
<optional>true</optional>
</dependency>
<!-- Reference parent lombok rely on -->
<dependency>
<groupId>org.projectlombok</groupId>
<artifactId>lombok</artifactId>
<optional>true</optional>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-test</artifactId>
<scope>test</scope>
</dependency>
</dependencies>
</project>
Add the application.yml configuration file to indicate the port number.
# Configure service port number server: port: 9001
Add the main startup class, add @ EnableHystrixDashboard and @ enablercircuitbreaker annotations, and add a bean to specify the monitoring path, otherwise the content cannot be monitored.
package com.king.springcloud;
import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;
import org.springframework.cloud.netflix.hystrix.dashboard.EnableHystrixDashboard;
/**
* @EnableHystrixDashboard: Turn on the hystrix graphical monitor
*/
@EnableHystrixDashboard
@SpringBootApplication
public class HystrixDashboardMain9001 {
public static void main(String[] args) {
SpringApplication.run(HystrixDashboardMain9001.class, args);
}
}
In order to realize the monitoring function, which module needs to have the dependency of spring boot starter actuator in pom.xml. Start the HystrixDashboardMain9001 module and visit the browser http://localhost:9001/hystrix You can see the interface.
In order to demonstrate the monitoring effect, that is, module 9001 monitors the operation of module 8001, modify the cloud provider hystrix payment8001 module, and check whether there is a spring boot starter actuator dependency in the pom.xml of module 8001. If it is not added, it must be.
Then modify the main startup class of the monitored PaymentHystrixMain8001 and inject a @ Bean to specify the monitoring path. Otherwise, an error of Unable to connect Command Metric Stream will be reported and the @ enablercircuitbreaker annotation function will be enabled. Note that the main startup class of PaymentHystrixMain8001 is modified.
package com.king.springcloud;
import com.netflix.hystrix.contrib.metrics.eventstream.HystrixMetricsStreamServlet;
import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;
import org.springframework.boot.web.servlet.ServletRegistrationBean;
import org.springframework.cloud.client.circuitbreaker.EnableCircuitBreaker;
import org.springframework.cloud.netflix.eureka.EnableEurekaClient;
import org.springframework.context.annotation.Bean;
/**
* Main startup class of micro service provider service side payment module
* @EnableEurekaClient: Indicates that it is an Eureka service registry client
* @EnableCircuitBreaker: Indicates the function of opening the circuit breaker
*/
@EnableCircuitBreaker
@EnableEurekaClient
@SpringBootApplication
public class PaymentHystrixMain8001 {
public static void main(String[] args) {
SpringApplication.run(PaymentHystrixMain8001.class, args);
}
/**
* This configuration is for service monitoring and has nothing to do with service fault tolerance itself. It is the result of Spring Cloud upgrade
* Because the default path of ServletRegistrationBean in SpringBoot is not "/ hystrix.stream“
* Therefore, you need to configure the upper and lower servlet s in your own project
*/
@Bean
public ServletRegistrationBean getServlet() {
HystrixMetricsStreamServlet streamServlet = new HystrixMetricsStreamServlet() ;
ServletRegistrationBean registrationBean = new ServletRegistrationBean(streamServlet);
registrationBean.setLoadOnStartup(1);
registrationBean.addUrlMappings("/hystrix.stream");
registrationBean.setName("HystrixMetricsStreamServlet");
return registrationBean;
}
}
After the modification, start Eureka7001 registration center, Provider8001 service producer, Dashboard9001 service monitoring, and http://localhost:9001/hystrix In the input box of the page, enter http://localhost:8001/hystrix.stream , that is, which service to monitor, Delay can default to 2000, and choose any name in the Title.
Click "Monitor Stream" to start monitoring and open a new window.
Multiple visits http://localhost:8001/payment/circuit/1 , you can see a broken line and a dot under the Circuit label. The broken line records the relative change of access traffic in the last 2 minutes. When the access traffic is large, the broken line rises and the dot becomes larger. At the same time, the dot will change its color according to the health status, and the health degree will change from green → yellow → orange → red Decreasing.
Multiple visits http://localhost:8001/payment/circuit/-1. It can be seen that there is another Circuit, indicating whether it is blown. If it is Closed, it means it is not blown. If it is Open, it means it is blown.
In the upper right corner of the monitoring page, there are seven colors, corresponding from left to right: number of successful requests, number of fused requests, number of wrong requests, number of timeout requests, number of rejected requests by thread pool, number of failed requests, and error rate in the last 10 seconds.
