High concurrency current limiting solution current limiting algorithm (token bucket, leaky bucket, counter), application layer current limiting (Nginx)

Current limiting algorithm

Common current limiting algorithms include token bucket and leaky bucket. The counter can also be implemented by current limiting.

Counter

   It is the simplest and easiest algorithm among current limiting algorithms. For example, if we require an interface, the number of requests in one minute cannot exceed 10. We can set a counter at the beginning. For each request, the counter + 1; If the value of the counter is greater than 10 and the time interval with the first request is within 1 minute, it indicates that there are too many requests. If the time interval between the request and the first request is greater than 1 minute and the value of the counter is still within the current limit, reset the counter

 

Sliding window count

Sliding window counting has many usage scenarios, such as current limiting to prevent system avalanche. Compared with the counting implementation, the sliding window implementation will be smoother and can automatically eliminate burrs.
The principle of sliding window is to judge whether the total access volume in the first N unit times exceeds the set threshold every time there is access, and add 1 to the number of requests on the current time slice.

 

Token Bucket

Token bucket algorithm is a bucket for storing tokens with a fixed capacity. Tokens are added to the bucket at a fixed rate. The token bucket algorithm is described as follows:

Assuming a limit of 2r/s, add tokens to the bucket at a fixed rate of 500 milliseconds;

A maximum of b tokens are stored in the bucket. When the bucket is full, the newly added tokens are discarded or rejected;

When a packet of N bytes arrives, n tokens will be deleted from the bucket, and then the packet will be sent to the network;

If there are less than n tokens in the bucket, the token will not be deleted and the packet will be throttled (either discarded or waiting in the buffer).

Using RateLimiter to implement token bucket current limiting

RateLimiter is an implementation class based on token bucket algorithm provided by guava. It can easily complete the current limiting stunt and adjust the rate of generating tokens according to the actual situation of the system.

Generally, it can be applied to rush purchase current limiting to prevent the system from crashing; Limit the access volume of an interface and service unit within a certain time. For example, some third-party services will limit the user access volume; Limit the network speed, how many bytes are allowed to upload and download per unit time, etc.

maven dependency of guava is introduced:

<parent>
		<groupId>org.springframework.boot</groupId>
		<artifactId>spring-boot-starter-parent</artifactId>
		<version>2.0.0.RELEASE</version>
	</parent>
	<dependencies>
		<dependency>
			<groupId>org.springframework.boot</groupId>
			<artifactId>spring-boot-starter-web</artifactId>
		</dependency>
		<dependency>
			<groupId>com.google.guava</groupId>
			<artifactId>guava</artifactId>
			<version>25.1-jre</version>
		</dependency>
	</dependencies>

 

/**
 * Function Description: use RateLimiter to realize token bucket algorithm
 * 
 */
@RestController
public class IndexController {
	@Autowired
	private OrderService orderService;
	// Explanation: 1.0 means that 1 token is generated every second and stored in the bucket
	RateLimiter rateLimiter = RateLimiter.create(1.0);

	// Order request
	@RequestMapping("/order")
	public String order() {
		// 1. Current limiting judgment
		// If no token is obtained within 500 milliseconds, it will wait all the time
		System.out.println("Generate token wait time:" + rateLimiter.acquire());
		boolean acquire = rateLimiter.tryAcquire(500, TimeUnit.MILLISECONDS);
		if (!acquire) {
			System.out.println("You can't grab it, because you'll be waiting. Give up first!");
			return "You can't grab it, because you'll be waiting. Give up first!";
		}

		// 2. If the current limiting requirements are not met, directly call the order interface
		boolean isOrderAdd = orderService.addOrder();
		if (isOrderAdd) {
			return "Congratulations,Successful rush purchase!";
		}
		return "Rush purchase failed!";
	}

}

Encapsulate RateLimiter

Custom annotation encapsulates RateLimiter

Custom annotation

@Target(value = ElementType.METHOD)
@Retention(RetentionPolicy.RUNTIME)
public @interface ExtRateLimiter {
	double value();
	long timeOut();
}

Write AOP

@Aspect
@Component
public class RateLimiterAop {
	// Does the storage interface already exist
	private static ConcurrentHashMap<String, RateLimiter> rateLimiterMap = new ConcurrentHashMap<String, RateLimiter>();

	@Pointcut("execution(public * com.itmayeidu.api.*.*(..))")
	public void rlAop() {
	}

	@Around("rlAop()")
	public Object doBefore(ProceedingJoinPoint proceedingJoinPoint) throws Throwable {
		MethodSignature signature = (MethodSignature) proceedingJoinPoint.getSignature();
		// Use Java reflection technology to obtain whether there is @ ExtRateLimiter annotation class on the method
		ExtRateLimiter extRateLimiter = signature.getMethod().getDeclaredAnnotation(ExtRateLimiter.class);
		if (extRateLimiter == null) {
			// Normal execution method
			Object proceed = proceedingJoinPoint.proceed();
			return proceed;
		}
		// ############Gets the parameter configuration fixed rate on the annotation###############
		// Get configured rate
		double value = extRateLimiter.value();
		// Get wait token wait time
		long timeOut = extRateLimiter.timeOut();
		RateLimiter rateLimiter = getRateLimiter(value, timeOut);
		// Judge whether the token bucket timed out to obtain the token
		boolean tryAcquire = rateLimiter.tryAcquire(timeOut, TimeUnit.MILLISECONDS);
		if (!tryAcquire) {
			serviceDowng();
			return null;
		}
		// Get the token and execute it directly
		Object proceed = proceedingJoinPoint.proceed();
		return proceed;

	}

	// Get RateLimiter object
	private RateLimiter getRateLimiter(double value, long timeOut) {
		// Get current URL
		ServletRequestAttributes attributes = (ServletRequestAttributes) RequestContextHolder.getRequestAttributes();
		HttpServletRequest request = attributes.getRequest();
		String requestURI = request.getRequestURI();
		RateLimiter rateLimiter = null;
		if (!rateLimiterMap.containsKey(requestURI)) {
			// Open token flow limit
			rateLimiter = RateLimiter.create(value); // Independent thread
			rateLimiterMap.put(requestURI, rateLimiter);
		} else {
			rateLimiter = rateLimiterMap.get(requestURI);
		}
		return rateLimiter;
	}

	// service degradation 
	private void serviceDowng() throws IOException {
		// Perform service degradation processing
		System.out.println("Execute degradation method,dear,Server busy! Please try again later!");
		ServletRequestAttributes attributes = (ServletRequestAttributes) RequestContextHolder.getRequestAttributes();
		HttpServletResponse response = attributes.getResponse();
		response.setHeader("Content-type", "text/html;charset=UTF-8");
		PrintWriter writer = response.getWriter();
		try {
			writer.println("Execute degradation method,dear,Server busy! Please try again later!");
		} catch (Exception e) {

		} finally {
			writer.close();
		}

	}

}

  call

@RequestMapping("/myOrder")
@ExtRateLimiter(value = 10.0, timeOut = 500)
public String myOrder() throws InterruptedException {
		System.out.println("myOrder");
		return "SUCCESS";
}

Leaky bucket algorithm

A leaky bucket with a fixed capacity flows out water droplets at a constant fixed rate;

If the bucket is empty, no water droplets need to flow out;

The water can flow into the leakage bucket at any rate;