Tip: This post has been added Index of.NET Core Microservice Basic Series Articles
Load Balancing and Request Caching
1.1 Load Balancing
To verify load balancing, we have configured two Consul Client nodes where ClientService is deployed within each node (192.168.80.70 and 192.168.80.71).
To better show which node API Repsonse comes from, let's change the return value:
[Route("api/[controller]")] public class ValuesController : Controller { // GET api/values [HttpGet] public IEnumerable<string> Get() { return new string[] { $"ClinetService: {DateTime.Now.ToString()} {Environment.MachineName} " + $"OS: {Environment.OSVersion.VersionString}" }; } ...... }
Make sure there are load balancing settings in Ocelot's configuration file:
{ "ReRoutes": [ ...... "LoadBalancerOptions": { "Type": "RoundRobin" }, ...... }
Next, publish and deploy to these two nodes, then launch our API gateway, where I launch it from the command line:
Then you can test load balancing, enter URL s in your browser and refresh them continuously: what you can see through the hostname is really load balancing based on polling.
Load Balance Optional Values:
- RoundRobin - Poll, Next to each other, rain and dew
- LeastConnection - Minimum number of connections, who has the least tasks, who receives the guests
- NoLoadBalance - Don't load balance, let me die alone.
1.2 Request Cache
Ocelot currently supports caching URL s for downstream services and can set a TTL in seconds to expire the cache.We can also clear a Region's cache by calling Ocelot's administrative API.
In order to use caching in routes, the following settings need to be added to ReRoute:
"FileCacheOptions": { "TtlSeconds": 10, "Region": "somename" }
This means that the cache will expire in 10 seconds.Additionally, it appears that only get mode is supported, and as long as the requested URL remains unchanged, it will be cached.
For example, let's continue with demo above, after adding the FileCacheOptions configuration, do a small test: because the 10s we set expires, all we get within 10s are caches, otherwise load balancing will be triggered to pick up data from different nodes.
2. Current Limiting and Fuses (QoS)
2.1 RateLimit
Limiting the flow of requests prevents downstream servers from crashing due to overloaded access. We can do this by adding a few simple configurations to the route.In addition, looking at the documentation, we found that this function was contributed by Captain Zhang Youyou, really 666.Also see a garden friend catcherwong , has been in practice for a long time, great.
For flow limiting, we can configure each service as follows:
"RateLimitOptions": { "ClientWhitelist": [ "admin" ], // Whitelist "EnableRateLimiting": true, // Is Current Limiting Enabled "Period": "1m", // Statistics period: 1 s, 5m, 1h, 1d "PeriodTimespan": 15, // How many seconds before the client can try again "Limit": 5 // Maximum number of requests allowed in statistical time period }
At the same time, we can do some global configuration:
"RateLimitOptions": { "DisableRateLimitHeaders": false, // Http head X-Rate-Limit and Retry-After Is Disabled "QuotaExceededMessage": "Too many requests, are you OK?", // Message returned when request overload is truncated "HttpStatusCode": 999, // Return when request overload is truncated http status "ClientIdHeader": "client_id" // Used to identify the client's request header, default is ClientId }
Each field here has a comment and is no longer interpreted.Let's test this:
Scenario 1: Access the clientservice without header, more than five times in a minute, will be truncated, returning the truncated message directly, HttpStatusCode:999
You can verify that a status code of 999 was returned by looking at the details of Repsonse:
Scenario 2: with header (client_id:admin) Access the clientservice, and have unrestricted access to the API within one minute
2.2 Fuser (QoS)
A break means stopping forwarding requests to downstream services.Requests are also unsuccessful when downstream services have failed and add to the burden of downstream servers and API gateways.This functionality is implemented using Sollly, and we only need to do some simple configuration for routing.If you are unfamiliar with Polly, you can read one of my previous articles, " A Polly+AspectCore-based Fuse and Degradation Mechanism for.NET Core Microservices>
"QoSOptions": { "ExceptionsAllowedBeforeBreaking": 2, // How many exception requests are allowed "DurationOfBreak": 5000, // Fuse time in milliseconds "TimeoutValue": 3000 // If the downstream request takes more time to process, it will be considered as if the request timed out },
*. Here for DurationOfBreak, the unit stated in the official document is seconds, but I found in the test that it should be milliseconds.I don't know if I'm using the wrong version or what.anyway, this is not the focus of the experiment.OK, here's our setting: If the service server takes more than three seconds to execute, a Timeout Exception will be thrown.If Service Server throws a second Timeout Exception, stop service access for five seconds.
Now let's transform the Service so that it manually times out so that Ocelot triggers the fuse protection mechanism.The TimeOutValue set in Ocelot is 3 seconds, so here we simply and roughly delayed it by 5 seconds (only for the first three requests).
[Route("api/[controller]")] public class ValuesController : Controller { ...... private static int _count = 0; // GET api/values [HttpGet] public IEnumerable<string> Get() { _count++; Console.WriteLine($"Get...{_count}"); if (_count <= 3) { System.Threading.Thread.Sleep(5000); } return new string[] { $"ClinetService: {DateTime.Now.ToString()} {Environment.MachineName} " + $"OS: {Environment.OSVersion.VersionString}" }; } ...... }
Here's a test: once you see the exception, you enter a service inaccessibility period of five seconds (returning 503 Service Unavaliable directly) and you can access the interface normally after five seconds (you won't enter a hard-code delay code at that time)
The log also confirms that Ocelot triggered the fuse protection:
3. Dynamic Routing
Remember a gardener commented in the previous article that he had 500 API services. If you configure them all in one place, it would be a huge project, all copies, but it would increase the chance of error and make it difficult to troubleshoot.At this point, we can sacrifice some specialties for universality, and Ocelot provides us with Dynamic Routing capabilities.This functionality was added after issue 340 (see the official document below) to reduce the ReRoutes configuration items in the configuration file by locating directly through service discovery after using service discovery.
Example: http://Api.edc.com/productservice/api/products=> Ocelot calls the Consul Service Discovery API as key to get IP and Port, then adds the following request URL section (api/products) to access the final URL:http://ip:port/api/products.
The experimental node structure shown in the following figure is still used: one API gateway node, three Consul Server nodes, and one Consul Client node.
Since there is no longer a need to configure ReRoutes, we need to make some "generic" changes, as shown in the GlobalConfiguration below:
{ "ReRoutes": [], "Aggregates": [], "GlobalConfiguration": { "RequestIdKey": null, "ServiceDiscoveryProvider": { "Host": "192.168.80.100", // Consul Service IP "Port": 8500 // Consul Service Port }, "RateLimitOptions": { "DisableRateLimitHeaders": false, // Http head X-Rate-Limit and Retry-After Is Disabled "QuotaExceededMessage": "Too many requests, are you OK?", // Message returned when request overload is truncated "HttpStatusCode": 999, // Return when request overload is truncated http status "ClientIdHeader": "client_id" // Used to identify the client's request header, default is ClientId }, "QoSOptions": { "ExceptionsAllowedBeforeBreaking": 3, "DurationOfBreak": 10000, "TimeoutValue": 5000 }, "BaseUrl": null, "LoadBalancerOptions": { "Type": "LeastConnection", "Key": null, "Expiry": 0 }, "DownstreamScheme": "http", "HttpHandlerOptions": { "AllowAutoRedirect": false, "UseCookieContainer": false, "UseTracing": false } } }
For more information, please browse: http://ocelot.readthedocs.io/en/latest/features/servicediscovery.html#dynamic-routing
Let's do a little test to see if clientservice and productservice can be accessed successfully.
(1) Access clientservice
(2) Access to productservice
You can see that service discovery-based access is normal as long as we enter the request URL correctly.Just here we need to enter the correct service name, which is registered in consul as shown in the highlighted section below:
{ "services":[ { "id": "EDC_DNC_MSAD_CLIENT_SERVICE_01", "name" : "CAS.ClientService", "tags": [ "urlprefix-/ClientService01" ], "address": "192.168.80.71", "port": 8810, "checks": [ { "name": "clientservice_check", "http": "http://192.168.80.71:8810/api/health", "interval": "10s", "timeout": "5s" } ] } ] }
4. Integrated Swagger Unified API Document Entry
Today, the only link between the front and back ends has become the API interface; API documentation has become the link between front and back end developers, and it is becoming more and more important. swagger is a framework for better writing API documentation.
4.1 Integrate Swagger for each Service
Step1.NuGet Install Swagger
NuGet>Install-Package Swashbuckle.AspNetCore
Override StartUp class
public class Startup { public Startup(IConfiguration configuration) { Configuration = configuration; } public IConfiguration Configuration { get; } // This method gets called by the runtime. Use this method to add services to the container. public IServiceProvider ConfigureServices(IServiceCollection services) { ....... services.AddMvc(); // Swagger services.AddSwaggerGen(s => { s.SwaggerDoc(Configuration["Service:DocName"], new Info { Title = Configuration["Service:Title"], Version = Configuration["Service:Version"], Description = Configuration["Service:Description"], Contact = new Contact { Name = Configuration["Service:Contact:Name"], Email = Configuration["Service:Contact:Email"] } }); var basePath = PlatformServices.Default.Application.ApplicationBasePath; var xmlPath = Path.Combine(basePath, Configuration["Service:XmlFile"]); s.IncludeXmlComments(xmlPath); }); ...... } // This method gets called by the runtime. Use this method to configure the HTTP request pipeline. public void Configure(IApplicationBuilder app, IHostingEnvironment env, IApplicationLifetime lifetime) { if (env.IsDevelopment()) { app.UseDeveloperExceptionPage(); } app.UseMvc(); // swagger app.UseSwagger(c=> { c.RouteTemplate = "doc/{documentName}/swagger.json"; }); app.UseSwaggerUI(s => { s.SwaggerEndpoint($"/doc/{Configuration["Service:DocName"]}/swagger.json", $"{Configuration["Service:Name"]} {Configuration["Service:Version"]}"); }); } }
This section of the configuration file is as follows:
{ "Service": { "Name": "CAS.NB.ClientService", "Port": "8810", "DocName": "clientservice", "Version": "v1", "Title": "CAS Client Service API", "Description": "CAS Client Service API provide some API to help you get client information from CAS", "Contact": { "Name": "CAS 2.0 Team", "Email": "EdisonZhou@manulife.com" }, "XmlFile": "Manulife.DNC.MSAD.NB.ClientService.xml" } }
Note that the output XML document file is checked and copied to the published directory (if there is no automatic copy):
4.2 Integrating Swagger for API Gateway
Step1.NuGet Installation Swagger =>Reference 4.1
Override StartUp class
public class Startup { public Startup(IConfiguration configuration) { Configuration = configuration; } public IConfiguration Configuration { get; } // This method gets called by the runtime. Use this method to add services to the container. public void ConfigureServices(IServiceCollection services) { // Ocelot services.AddOcelot(Configuration); // Swagger services.AddMvc(); services.AddSwaggerGen(options => { options.SwaggerDoc($"{Configuration["Swagger:DocName"]}", new Info { Title = Configuration["Swagger:Title"], Version = Configuration["Swagger:Version"] }); }); } // This method gets called by the runtime. Use this method to configure the HTTP request pipeline. public void Configure(IApplicationBuilder app, IHostingEnvironment env) { if (env.IsDevelopment()) { app.UseDeveloperExceptionPage(); } // get from service discovery later var apiList = new List<string>() { "clientservice", "productservice", "noticeservice" }; app.UseMvc() .UseSwagger() .UseSwaggerUI(options => { apiList.ForEach(apiItem => { options.SwaggerEndpoint($"/doc/{apiItem}/swagger.json", apiItem); }); }); // Ocelot app.UseOcelot().Wait(); } }
*. Here's an apiNameList directly hard-code, which should actually take a configuration file or call Service Discovery to get the service name (assuming your docName and serviceName are identical, otherwise your document cannot be located correctly)
Step3. ChangeConfiguration.jsonConfiguration file=> matches the name of hard-code, which makes it easy to get API documentation by directly formatting URL s upstream and downstream
{ "ReRoutes": [ // API01:CAS.ClientService // --> swagger part { "DownstreamPathTemplate": "/doc/clientservice/swagger.json", "DownstreamScheme": "http", "ServiceName": "CAS.ClientService", "LoadBalancer": "RoundRobin", "UseServiceDiscovery": true, "UpstreamPathTemplate": "/doc/clientservice/swagger.json", "UpstreamHttpMethod": [ "GET", "POST", "DELETE", "PUT" ] }, // --> service part { "UseServiceDiscovery": true, // use Consul service discovery "DownstreamPathTemplate": "/api/{url}", "DownstreamScheme": "http", "ServiceName": "CAS.ClientService", "LoadBalancerOptions": { "Type": "RoundRobin" }, "UpstreamPathTemplate": "/api/clientservice/{url}", "UpstreamHttpMethod": [ "Get", "Post" ], "RateLimitOptions": { "ClientWhitelist": [ "admin" ], // Whitelist "EnableRateLimiting": true, // Is Current Limiting Enabled "Period": "1m", // Statistics period: 1 s, 5m, 1h, 1d "PeriodTimespan": 15, // How many seconds before the client can try again "Limit": 10 // Maximum number of requests allowed in statistical time period }, "QoSOptions": { "ExceptionsAllowedBeforeBreaking": 2, // How many exception requests are allowed "DurationOfBreak": 5000, // Time of melting in seconds "TimeoutValue": 3000 // If the downstream request takes more time to process, it will be considered as if the request timed out }, "ReRoutesCaseSensitive": false // non case sensitive }, // API02:CAS.ProductService // --> swagger part { "DownstreamPathTemplate": "/doc/productservice/swagger.json", "DownstreamScheme": "http", "ServiceName": "CAS.ProductService", "LoadBalancer": "RoundRobin", "UseServiceDiscovery": true, "UpstreamPathTemplate": "/doc/productservice/swagger.json", "UpstreamHttpMethod": [ "GET", "POST", "DELETE", "PUT" ] }, // --> service part { "UseServiceDiscovery": true, // use Consul service discovery "DownstreamPathTemplate": "/api/{url}", "DownstreamScheme": "http", "ServiceName": "CAS.ProductService", "LoadBalancerOptions": { "Type": "RoundRobin" }, "FileCacheOptions": { // cache response data - ttl: 10s "TtlSeconds": 10, "Region": "" }, "UpstreamPathTemplate": "/api/productservice/{url}", "UpstreamHttpMethod": [ "Get", "Post" ], "RateLimitOptions": { "ClientWhitelist": [ "admin" ], "EnableRateLimiting": true, "Period": "1m", "PeriodTimespan": 15, "Limit": 10 }, "QoSOptions": { "ExceptionsAllowedBeforeBreaking": 2, // How many exception requests are allowed "DurationOfBreak": 5000, // Time of melting in seconds "TimeoutValue": 3000 // If the downstream request takes more time to process, it will be considered as if the request timed out }, "ReRoutesCaseSensitive": false // non case sensitive } ], "GlobalConfiguration": { //"BaseUrl": "https://api.mybusiness.com" "ServiceDiscoveryProvider": { "Host": "192.168.80.100", // Consul Service IP "Port": 8500 // Consul Service Port }, "RateLimitOptions": { "DisableRateLimitHeaders": false, // Http head X-Rate-Limit and Retry-After Is Disabled "QuotaExceededMessage": "Too many requests, are you OK?", // Message returned when request overload is truncated "HttpStatusCode": 999, // Return when request overload is truncated http status "ClientIdHeader": "client_id" // Used to identify the client's request header, default is ClientId } } }
*. Here you need to note the new swagger part configuration, specifically forSwagger.jsonMapping done.
4.3 Test
From then on, we can browse the API documentation for all services only through the API gateway. It's awesome!
V. Summary
Based on Ocelot's official documentation, this article takes a look at some of Ocelot's useful functions: load balancing (although only two basic algorithm strategies are provided), caching, throttling, QoS, and Dynamic Routing, and validates them with a few simple Demos.Finally, by inheriting Swagger as the uniform API document entry, you can view all the swagger-based API documents from one URL only.By looking at Ocelot's official documentation, you can see that Ocelot also supports many other useful features that are not described here (some may be added in subsequent sections such as authentication, authorization, Trace, and so on).In addition, some friends look for the source I want to demo, and I'll upload it to github later.The content in these articles can be built by sharing code s and configurations, so don't post it=>It's already posted. Click Download.
Sample Code
Click here => Click me to download
Reference material
jesse, .NET Core Open Source Gateway - Ocelot Chinese Documentation>
catcher wong，<Building API Gateway Using Ocelot In ASP.NET Core - QoS (Quality of Service)>
focus-lei，<.NET Core Unified Configuration of Swagger in Ocelot Gateway>
Official Ocelot Documentation: http://ocelot.readthedocs.io/en/latest/index.html
Author: Zhou Xulong
Source: http://edisonchou.cnblogs.com
Copyright of this article is owned by both the author and the blog park. Reproduction is welcome, but this statement must be retained without the author's consent, and a link to the original text is provided in an obvious place on the article page.