1, Nacos configuration center
Official documents: https://github.com/alibaba/spring-cloud-alibaba/wiki/Nacos-config
Nacos provides key/value storage for storing configuration and other metadata, and provides server-side and client-side support for externalized configuration in distributed systems. Using Spring Cloud Alibaba Nacos Config, you can centrally manage the external attribute configuration of your Spring Cloud application in the Nacos Server.
1.1 quick start
Prepare the configuration and create a new nacos-config.properties in the nacos server
Note that the configuration format is properties
Note that the dataid is extended by properties (the default file extension).
1.2 build Nacos config service
Dynamic configuration change is realized through Nacos Server and spring cloud starter Alibaba Nacos config.
1) Introduce dependency
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<parent>
<artifactId>spring-cloud-alibaba-mall</artifactId>
<groupId>org.jihu.mall</groupId>
<version>1.0-SNAPSHOT</version>
</parent>
<modelVersion>4.0.0</modelVersion>
<artifactId>nacos-config</artifactId>
<dependencies>
<dependency>
<groupId>com.alibaba.cloud</groupId>
<artifactId>spring-cloud-starter-alibaba-nacos-config</artifactId>
</dependency>
</dependencies>
</project>
2) Add bootstrap.properties (note that it is not application.yml, otherwise it will always report errors, and the connection address is localhost:8848)
spring.application.name=nacos-config # Configuration center address spring.cloud.nacos.config.server-addr=192.168.131.172:8848
Note that when you use the domain name to access Nacos, the spring.cloud.nacos.config.server-addr configuration method is the domain name: port. For example, if the domain name of Nacos is abc.com.nacos and the listening port is 80, then spring. Cloud. Nacos. Config. Server addr = ABC. Com. Nacos: 80. Note that port 80 cannot be omitted.
3) Start the service and test whether the micro service uses the configuration of the configuration center
@SpringBootApplication
public class NacosConfigApplication {
public static void main(String[] args) {
ConfigurableApplicationContext applicationContext = SpringApplication.run(NacosConfigApplication.class, args);
String userName = applicationContext.getEnvironment().getProperty("user.name");
String userAge = applicationContext.getEnvironment().getProperty("user.age");
System.out.println("user name :" + userName + "; age: " + userAge);
}
}
Configuration of file extension based on yaml with dataid
Spring cloud starter Alibaba Nacos config also perfectly supports yaml format. At this time, you only need to complete the following two steps:
1. The declared dataid file extension displayed in the bootstrap.properties configuration file of the application. As follows:
spring.cloud.nacos.config.file-extension=yaml
2. Add a configuration with dataid yaml as extension on the console of Nacos, as shown below:
After these two steps are completed, restart the test program and you can see the following output results:
Note: by default, the microservice name will be used for matching, so the default configuration file can be extended (. yaml/.properties).
1.3 Config related configuration
The Key of the Nacos data model is uniquely determined by triples. By default, the Namespace is an empty string, the public Namespace (public), and the grouping is DEFAULT_GROUP
Support dynamic update of configuration
@SpringBootApplication
public class NacosConfigApplication {
public static void main(String[] args) throws InterruptedException {
ConfigurableApplicationContext applicationContext = SpringApplication.run(NacosConfigApplication.class, args);
while (true) {
//When the dynamic configuration is refreshed, it will be updated to the environment, so the configuration is obtained from the environment every second
String userName = applicationContext.getEnvironment().getProperty("user.name");
String userAge = applicationContext.getEnvironment().getProperty("user.age");
System.err.println("user name :" + userName + "; age: " + userAge);
TimeUnit.SECONDS.sleep(1);
}
}
}
Let's modify the configuration file on nacos, and then observe the log:
You can see that the modified value is read here very quickly.
Support configuration of profile granularity
When loading the configuration, Spring cloud starter Alibaba Nacos config not only loads the basic configuration with the dataid of ${Spring. Application. Name}. ${file extension: properties}, but also loads the basic configuration with the dataid of ${Spring. Application. Name} - ${profile}. ${file extension: properties}. In daily development, if you encounter different configurations in multiple environments, you can configure them through the ${spring.profiles.active} configuration item provided by Spring.
spring.application.name=nacos-config
# Configuration center address
spring.cloud.nacos.config.server-addr=192.168.131.172:8848
# Configuration method of file extension with dataid yaml
# `${spring.application.name}.${file-extension:properties}`
# Declare dataid file extension
spring.cloud.nacos.config.file-extension=yaml
#Profile granularity configuration ` ${spring. Application. Name} - ${profile}. ${file extension: properties}`
spring.profiles.active=develop
${spring.profiles.active} must be placed in the bootstrap.properties file when specified through the configuration file.
A basic configuration with a data ID of: nacos-config-development.yaml is added to Nacos, as shown below:
The code to start the Spring Boot application test is as follows:
@SpringBootApplication
public class NacosConfigApplication {
public static void main(String[] args) throws InterruptedException {
ConfigurableApplicationContext applicationContext = SpringApplication.run(NacosConfigApplication.class, args);
while (true) {
String userName = applicationContext.getEnvironment().getProperty("user.name");
String userAge = applicationContext.getEnvironment().getProperty("user.age");
//Get the currently deployed environment
String currentEnv = applicationContext.getEnvironment().getProperty("current.env");
System.err.println("in " + currentEnv + " enviroment; " + "user name :" + userName + "; age: " + userAge);
TimeUnit.SECONDS.sleep(1);
}
}
}
After startup, you can see the output results of the console:
If you need to switch to the production environment, you only need to change the ${spring.profiles.active} parameter configuration. As follows:
spring.profiles.active=product
At the same time, Nacos needs to add the basic configuration of the corresponding dataid in the production environment. For example, Naocs in the generation environment adds a configuration with a data ID of: nacos-config-product.yaml:
Start the test program, and the output results are as follows:
In this case, we write it in the configuration file through spring.profiles.active =. In the real project implementation process, the value of this variable needs different environments. At this time, the common practice is to specify its configuration through the - dspring. Profiles. Active = < profile > parameter to achieve flexible switching between environments.
Supports custom namespace configuration
First, let's take a look at the concept of Nacos Namespace, Nacos concept
Used for tenant granularity configuration isolation. Configurations with the same Group or Data ID can exist in different namespaces. One of the common scenarios of Namespace is to distinguish and isolate the configurations of different environments, such as the isolation of resources (such as configuration and service) between development test environment and production environment.
If the ${spring.cloud.nacos.config.namespace} configuration is not explicitly specified, the Public namespace on Nacos is used by default. If you need to use a custom namespace, you can implement it through the following configuration:
spring.cloud.nacos.config.namespace=b3404bc0-d7dc-4855-b519-570ed34b62d7
The configuration must be placed in the bootstrap.properties file. In addition, the value of spring.cloud.nacos.config.namespace is the id corresponding to the namespace. The id value can be obtained from the console of Nacos. And be careful not to select other namepages when adding configurations, otherwise the correct configuration will not be read.
Note that the command space here is actually very flexible. We can divide it according to project modules or developers, not necessarily limited to the environment.
Supports the configuration of custom groups
Default is used by default when the ${spring.cloud.nacos.config.group} configuration is not explicitly specified_ GROUP . If you need to customize your own group, you can configure it as follows:
spring.cloud.nacos.config.group=DEVELOP_GROUP
The configuration must be placed in the bootstrap.properties file. When adding configuration, the value of Group must be consistent with the configuration value of spring.cloud.nacos.config.group.
Supports Data Id configuration for custom extensions
You can define a Data Id according to mybats, redis, mq and other services and configure it separately.
Data ID is one of the dimensions of organization partition configuration. Data ID is usually used to organize the configuration set of the partition system. A system or application can contain multiple configuration sets, and each configuration set can be identified by a meaningful name. Data ID usually adopts the naming rules of Java like package (such as com.taobao.tc.refund.log.level) to ensure global uniqueness. This naming convention is not mandatory.
By customizing the extended Data Id configuration, you can not only solve the problem of configuration sharing among multiple applications, but also support multiple configuration files for one application.
After Spring Cloud Alibaba Nacos Config is released from version 0.2.1, it can support the configuration of custom Data Id. For the detailed design of this part, please refer to here. A complete configuration case is as follows:
spring.application.name=opensource-service-provider spring.cloud.nacos.config.server-addr=127.0.0.1:8848 # config external configuration # 1. The Data Id is in the default group DEFAULT_GROUP, dynamic refresh of configuration is not supported spring.cloud.nacos.config.extension-configs[0].data-id=ext-config-common01.properties # 2. Data Id is not in the default group and dynamic refresh is not supported spring.cloud.nacos.config.extension-configs[1].data-id=ext-config-common02.properties spring.cloud.nacos.config.extension-configs[1].group=GLOBALE_GROUP # 3. The Data Id is neither in the default group nor supports dynamic refresh spring.cloud.nacos.config.extension-configs[2].data-id=ext-config-common03.properties spring.cloud.nacos.config.extension-configs[2].group=REFRESH_GROUP spring.cloud.nacos.config.extension-configs[2].refresh=true
You can see:
- The configuration of multiple data IDS is supported through the configuration method of spring.cloud.nacos.config.extension-configurations [n]. Data Id.
- Customize the group of Data Id through the configuration method of spring.cloud.nacos.config.extension-configurations [n]. Group. If the configuration is not clear, the default is DEFAULT_GROUP.
- The configuration mode of spring.cloud.nacos.config.extension-configurations [n]. Refresh is used to control whether the Data Id supports dynamic refresh in the application and senses the latest configuration value in case of configuration change. Default is not supported.
When multiple data IDs are configured at the same time, their priority relationship is spring.cloud.nacos.config.extension-configurations [n]. Data Id, where the greater the value of N, the higher the priority.
The value of spring.cloud.nacos.config.extension-configurations [n]. Data Id must have a file extension that supports both properties and yaml/yml. At this time, the configuration of spring.cloud.nacos.config.file-extension has no effect on the Data Id file extension of the custom extension configuration.
By customizing the extended Data Id configuration, you can not only solve the problem of configuration sharing among multiple applications, but also support multiple configuration files for one application.
To more clearly configure shared data IDS among multiple applications, you can configure them in the following ways:
# Configure shared Data Id spring.cloud.nacos.config.shared-configs[0].data-id=common.yaml # Configure the group where the Data Id is located. The default is DEFAULT_GROUP spring.cloud.nacos.config.shared-configs[0].group=GROUP_APP1 # Whether to dynamically refresh the configuration Data Id when the configuration is changed. The default is false spring.cloud.nacos.config.shared-configs[0].refresh=true
You can see:
- Support the configuration of multiple shared data IDS through spring.cloud.nacos.config.shared-configurations [n]. Data Id.
- Configure the group of custom Data Id through spring.cloud.nacos.config.shared-configurations [n]. Group. If the configuration is not clear, the default is DEFAULT_GROUP.
- Use spring.cloud.nacos.config.shared-configurations [n]. Refresh to control whether the Data Id supports dynamic refresh in the application when the configuration is changed. The default is false.
Common configuration
General configuration of different projects:
spring.cloud.nacos.config.shared-configs[0].data-id=common.yaml spring.cloud.nacos.config.shared-configs[0].group=DEFAULT_GROUP spring.cloud.nacos.config.shared-configs[0].refresh=true
Configured priority
Spring Cloud Alibaba Nacos Config currently provides three configuration capabilities to pull relevant configurations from Nacos.
- A: Support multiple shared data through spring.cloud.nacos.config.shared-configurations [n]. Data ID
Configuration of Id - B: Through spring.cloud.nacos.config.extension-configurations [n]. Data ID
Supports the configuration of multiple extended data IDs - C: Relevant Data Id configuration is automatically generated through internal relevant rules (application name, application name + Profile)
When the three methods are used together, one of their priority relationships is: a < B < C
Completely close the configuration
Completely close Spring Cloud Nacos Config by setting spring.cloud.nacos.config.enabled = false
@RefreshScope annotation
@The Value annotation can get the Value of the configuration center, but it cannot dynamically perceive the modified Value. You need to use the @ RefreshScope annotation
@RestController
@RefreshScope
public class TestController {
@Value("${common.age}")
private String age;
@GetMapping("/common")
public String hello() {
return age;
}
}
@NacosValue annotation
If spring boot integrates the nacos configuration center, you can also use the @ NacosValue annotation to dynamically read the updated configuration:
First, introduce related dependencies:
<!-- nacos-config -->
<dependency>
<groupId>com.alibaba.boot</groupId>
<artifactId>nacos-config-spring-boot-starter</artifactId>
<version>0.2.7</version>
</dependency>
Then add the configuration file to read on the service startup class:
1.4 configuration practice
We now have a user micro service. Let's configure it on nacos.
1) Add dependencies to microservices:
<dependency>
<groupId>com.alibaba.cloud</groupId>
<artifactId>spring-cloud-starter-alibaba-nacos-config</artifactId>
</dependency>
2) Add the bootstrap.yml configuration file:
spring:
application:
name: mall-user # Microservice name
cloud:
nacos:
config:
# nacos service address
server-addr: 192.168.131.172:8848
# Configure service suffix (later versions support yml)
file-extension: yml
3) Add a configuration file with dataId of mall-user.yml on nacos (copy the contents of the application.yml file in the original service)
server:
port: 8040
spring:
application:
name: mall-user #Microservice name
#Configure nacos registry address
cloud:
nacos:
discovery:
server-addr: 192.168.131.172:8848
loadbalancer:
rinnon:
enabled: false
feign:
client:
config:
mall-order: # Corresponding microservices
loggerLevel: FULL
mall-user: # Corresponding microservices
loggerLevel: FULL
Then delete the old application.yml file.
4) Start microservice
5) Pull out the nacos configuration. Because many of our modules depend on nacos service discovery, we now pull out the nacos related configuration
# Configure nacos registry address
spring:
cloud:
nacos:
discovery:
server-addr: 192.168.131.172:8848
Create a new configuration on nacos:
Then delete the nacos related configuration in mall-user.yml.
6) Merge mall-user.yml and nacos.yml
Modify the bootstrap.yml file of the mall user microservice:
spring:
application:
name: mall-user # Microservice name
cloud:
nacos:
config:
# nacos service address
server-addr: 192.168.131.172:8848
# Configure service suffix (later versions support yml)
file-extension: yml
# Introducing nacos to expand configuration
extension-configs[0]:
dataId: nacos.yml
refersh: true
7) Restart user microservice:
The configuration is successful and has been successfully registered with nacos.
Next, there is no need to configure other micro services related to Nacos. You can directly expand nacos.yml.
Of course, shared configurations [0] can also be used for these public configurations, but extension configurations [0] has higher priority.
2, Source code analysis of Nacos configuration center
2.1 architecture of configuration center
demo used by configuration center:
public class ConfigServerDemo {
public static void main(String[] args) throws NacosException, InterruptedException {
String serverAddr = "localhost";
String dataId = "nacos-config-demo.yaml";
String group = "DEFAULT_GROUP";
Properties properties = new Properties();
properties.put(PropertyKeyConst.SERVER_ADDR, serverAddr);
//Get configuration service
ConfigService configService = NacosFactory.createConfigService(properties);
//Get configuration
String content = configService.getConfig(dataId, group, 5000);
System.out.println(content);
//Register listener
configService.addListener(dataId, group, new Listener() {
@Override
public void receiveConfigInfo(String configInfo) {
System.out.println("===recieve:" + configInfo);
}
@Override
public Executor getExecutor() {
return null;
}
});
//Publish configuration
//boolean isPublishOk = configService.publishConfig(dataId, group, "content");
//System.out.println(isPublishOk);
//Send properties format
configService.publishConfig(dataId,group,"common.age=30", ConfigType.PROPERTIES.getType());
Thread.sleep(3000);
content = configService.getConfig(dataId, group, 5000);
System.out.println(content);
// boolean isRemoveOk = configService.removeConfig(dataId, group);
// System.out.println(isRemoveOk);
// Thread.sleep(3000);
// content = configService.getConfig(dataId, group, 5000);
// System.out.println(content);
// Thread.sleep(300000);
}
}
2.2 source code analysis of Nacos config client
Configuration center core interface ConfigService
2.2.1 get configuration
The main method to obtain the configuration is the getConfig method of the NacosConfigService class. Usually, this method directly obtains the configuration value from the local file. If the local file does not exist or the content is empty, then pull the configuration from the remote through the HTTP GET method and save it to the local snapshot. When obtaining remote configuration through HTTP, Nacos provides two fusing strategies: one is the timeout time, the other is the maximum number of retries, and the default is three retries.
private String getConfigInner(String tenant, String dataId, String group, long timeoutMs) throws NacosException {
group = this.null2defaultGroup(group);
ParamUtils.checkKeyParam(dataId, group);
ConfigResponse cr = new ConfigResponse();
cr.setDataId(dataId);
cr.setTenant(tenant);
cr.setGroup(group);
// Get the configuration information from the local file first
String content = LocalConfigInfoProcessor.getFailover(this.agent.getName(), dataId, group, tenant);
if (content != null) {
LOGGER.warn("[{}] [get-config] get failover ok, dataId={}, group={}, tenant={}, config={}", new Object[]{this.agent.getName(), dataId, group, tenant, ContentUtils.truncateContent(content)});
cr.setContent(content);
this.configFilterChainManager.doFilter((IConfigRequest)null, cr);
content = cr.getContent();
return content;
} else { // Remote acquisition
try {
String[] ct = this.worker.getServerConfig(dataId, group, tenant, timeoutMs);
cr.setContent(ct[0]);
this.configFilterChainManager.doFilter((IConfigRequest)null, cr);
content = cr.getContent();
return content;
} catch (NacosException var9) {
if (403 == var9.getErrCode()) {
throw var9;
} else {
LOGGER.warn("[{}] [get-config] get from server error, dataId={}, group={}, tenant={}, msg={}", new Object[]{this.agent.getName(), dataId, group, tenant, var9.toString()});
LOGGER.warn("[{}] [get-config] get snapshot ok, dataId={}, group={}, tenant={}, config={}", new Object[]{this.agent.getName(), dataId, group, tenant, ContentUtils.truncateContent(content)});
content = LocalConfigInfoProcessor.getSnapshot(this.agent.getName(), dataId, group, tenant);
cr.setContent(content);
this.configFilterChainManager.doFilter((IConfigRequest)null, cr);
content = cr.getContent();
return content;
}
}
}
}
2.2.2 registered listener
The configuration center client will register a listener for the configuration item to perform the callback function when the configuration item is changed.
NacosConfigService#getConfigAndSignListener ConfigService#addListener
Nacos can register listeners in the above ways, and their internal implementation is to call addCacheDataIfAbsent of ClientWorker class. CacheData is an instance that maintains the configuration item and all listeners registered under it. All CacheData are saved in the atomic cacheMap in the ClientWorker class. Its internal core members are:
private void registerNacosListener(final String groupKey, final String dataKey) {
String key = NacosPropertySourceRepository.getMapKey(dataKey, groupKey);
Listener listener = (Listener)this.listenerMap.computeIfAbsent(key, (lst) -> {
return new AbstractSharedListener() {
public void innerReceive(String dataId, String group, String configInfo) {
NacosContextRefresher.refreshCountIncrement();
NacosContextRefresher.this.nacosRefreshHistory.addRefreshRecord(dataId, group, configInfo);
// If a service change is detected, a spring container refresh event is issued to update the bean
NacosContextRefresher.this.applicationContext.publishEvent(new RefreshEvent(this, (Object)null, "Refresh Nacos config"));
if (NacosContextRefresher.log.isDebugEnabled()) {
NacosContextRefresher.log.debug(String.format("Refresh Nacos config group=%s,dataId=%s,configInfo=%s", group, dataId, configInfo));
}
}
};
});
try {
this.configService.addListener(dataKey, groupKey, listener);
} catch (NacosException var6) {
log.warn(String.format("register fail for nacos listener ,dataId=[%s],group=[%s]", dataKey, groupKey), var6);
}
}
2.2.3 configure long polling
ClientWorker completes the configuration of long polling through its two thread pools. One is a single thread executor. It retrieves the cacheData instances to be polled for every 3000 configuration items every 10ms, packages them into a LongPollingTask, and submits them to the executorService of the second thread pool for processing.
2.3 source code analysis of Nacos config server
2.3.1 configuring dump
When the server starts, it will rely on the init method of DumpService, load the configuration from the database and store it on the local disk, and cache some important meta information, such as MD5 value, in memory. The server will judge whether to dump full configuration data or partial incremental configuration data from the database according to the last heartbeat time saved in the heartbeat file (if the last heartbeat interval of the machine is within 6h).
Full dump, of course, first empties the disk cache, and then swipes 1000 configurations into the disk and memory according to the primary key ID. Incremental dump is to retrieve the newly added configurations (including updated and deleted) in the last six hours. First refresh the memory and files according to this batch of data, and then compare the database according to the total amount of data in the memory. If there are changes, synchronize again, which will reduce a certain number of database IO and disk IO compared with full dump.
2.3.2 configuration release
The code for publishing configuration is located in ConfigController#publishConfig. In cluster deployment, the request will only hit one machine at the beginning, which will insert the configuration into MySQL for persistence. The server does not access MySQL for each configuration query, but will rely on the dump function to cache the configuration in the local file. Therefore, after a single machine saves the configuration, it needs to notify other machines to refresh the file contents in memory and local disk. Therefore, it will publish an event called ConfigDataChangeEvent, which will notify all cluster nodes (including itself) through HTTP call to trigger the refresh of local files and memory.
2.3.3 handling long polling
The client will have a long polling task to pull the configuration changes of the server. The processing logic of the server is in the LongPollingService class. Among them, there is a Runnable task named ClientLongPolling. The server will package the received polling request into a ClientLongPolling task, which holds an AsyncContext response object, The execution is delayed by 29.5s through the timed thread pool. It is returned 500ms earlier than the 30s timeout of the client to ensure that the client will not timeout due to network delay to the greatest extent.
