The study of NoSql database -- the study of Redis database

Keywords: Jedis Redis Database JSON

Article catalog

1. Concept: redis is a high-performance NOSQL series non relational database

1.1. What is NOSQL

		NoSQL(NoSQL = Not Only SQL), which means "not only SQL", is a new database concept, generally referring to non relational database.
		With the rise of Internet Web 2.0 sites, traditional relational databases have been unable to cope with Web 2.0 sites, especially the super large-scale and high concurrency of SNS type Web 2.0 pure dynamic sites, which has exposed many problems that are difficult to overcome, while non relational databases have developed rapidly due to their own characteristics. The production of NoSQL database is to solve the challenges brought by large-scale data collection and multiple data types, especially the application problems of big data.

1.1.1. Comparison between NoSQL and relational database

advantage:
	1) Cost: nosql database is simple and easy to deploy. It is basically open-source software. It doesn't need to spend a lot of money to purchase and use like oracle. It's cheaper than relational database.
	2) Query speed: nosql database stores data in cache, relational database stores data in hard disk, and natural query speed is far slower than nosql database.
	3) Format of data storage: the storage format of nosql is key,value, document, picture, etc., so it can store various formats such as basic types, objects or collections, while the database only supports basic types.
	4) Extensibility: it is difficult to extend a relational database due to the limitation of multi table query mechanism such as join.

Disadvantages:
	1) The maintenance tools and materials are limited, because nosql is a new technology, and it can't keep pace with the technology of relational database for more than 10 years.
	2) It does not provide support for sql. If it does not support industrial standards such as sql, there will be a certain cost for users to learn and use.
	3) Transaction processing by a relational database is not provided.

1.1.2. Advantages of non relational database:

1) The performance of NOSQL is based on key value pairs, which can be imagined as the corresponding relationship between the primary key and value in the table, and does not need to be parsed by the SQL layer, so the performance is very high.
2) Scalability is also because there is no coupling between data based on key value pairs, so it is very easy to expand horizontally.

1.1.3. Advantages of relational database:

1) Complex query can be easily used in SQL statements to do a very complex data query between a table and multiple tables.
2) Transaction support enables the implementation of data access requirements with high security performance. For these two types of databases, the advantage of each other is its own weakness, and vice versa.

1.1.4. Summary

Relational database and NoSQL database are not opposite but complementary. That is to say, they usually use relational database and NoSQL database when it is suitable for NoSQL,
Let NoSQL database make up for the deficiency of relational database.
Generally, the data will be stored in the relational database, and the data of the relational database will be backed up and stored in the nosql database

1.2. Mainstream NOSQL products

• key value storage database

Related products: Tokyo Cabinet/Tyrant, Redis, Voldemort, Berkeley DB
 Typical application: content caching, which is mainly used to deal with the high access load of a large amount of data. 
Data model: a series of key value pairs
 Advantage: fast query
 Disadvantage: data stored is not structured

• column storage database

Related products: Cassandra, HBase, Riak
 Typical application: distributed file system
 Data model: store in column cluster, and store the same column of data together
 Advantages: fast search speed, strong scalability and easier distributed expansion
 Disadvantages: relatively limited functions

• document database

Related products: CouchDB, MongoDB
 Typical application: Web application (similar to key Value, Value is structured)
Data model: a series of key value pairs
 Advantage: data structure requirements are not strict
 Disadvantage: poor query performance and lack of unified query syntax

• graph database

Related databases: Neo4J, InfoGrid, Infinite Graph
 Typical application: Social Network
 Data model: graph structure
 Advantages: using graph structure related algorithm.
Disadvantage: it needs to calculate the whole figure to get the result, and it is not easy to do a distributed clustering scheme.

1.3. What is Redis

Redis is an open-source high-performance key value database developed in C language. It officially provides test data, 50 concurrent execution of 100000 requests, 110000 reads / s, 81000 writes / s, and redis provides a variety of key value data types to meet the storage requirements in different scenarios. So far, redis supports the following key value data types:
	1) string type
	2) Hash type hash
	3) List type list
	4) Set type set
	5) Ordered set type sortedset

1.3.1.redis application scenario

• caching (data query, short connection, news content, product content, etc.)
• chat room's online friends list
 • task queues. (seckill, snap up, 12306, etc.)
• app leaderboards
 • website visit statistics
 • data expiration processing (can be accurate to milliseconds
 session separation in distributed cluster architecture

2. Download and install

1. Official website: https://redis.io
 2. Chinanet: http://www.redis.net.cn/
3. Decompression can be used directly:
	 redis.windows.conf : Profile
	 redis-cli.exe : redis client
	 redis-server.exe : redis server side

3. Command operation

3.1. Data structure of redis:

	redis stores data in the format of key and value, in which key is a string and value has five different data structures
		 Data structure of value:
			1) string type
			2) Hash type hash: map format  
			3) List type list: linkedlist format. Support for repeating elements
			4) Set type set: duplicate elements are not allowed
			5) Ordered set type sortedset: duplicate elements are not allowed, and elements have order

3.2. string type

	1. Storage: set key value
			127.0.0.1:6379> set username zhangsan
			OK
	2. obtain: get key
			127.0.0.1:6379> get username
			"zhangsan"
	3. Delete: del key
			127.0.0.1:6379> del age
			(integer) 1

3.3. Hash type hash

		1. Storage: hset key field value
			127.0.0.1:6379> hset myhash username lisi
			(integer) 1
			127.0.0.1:6379> hset myhash password 123
			(integer) 1
		2. obtain: 
			* hget key field: Get the specified field Corresponding value
				127.0.0.1:6379> hget myhash username
				"lisi"
			* hgetall key: Get all field and value
				127.0.0.1:6379> hgetall myhash
				1) "username"
				2) "lisi"
				3) "password"
				4) "123"
				
		3. Delete: hdel key field
			127.0.0.1:6379> hdel myhash username
			(integer) 1

3.4. List type list: you can add an element to the head (left) or tail (right) of the list

		1. add to:
			1. lpush key value: Add elements to the left table of the list
				
			2. rpush key value: Add elements to the right side of the list
				
				127.0.0.1:6379> lpush myList a
				(integer) 1
				127.0.0.1:6379> lpush myList b
				(integer) 2
				127.0.0.1:6379> rpush myList c
				(integer) 3
		2. obtain:
			 lrange key start end : Scope acquisition
				127.0.0.1:6379> lrange myList 0 -1
				1) "b"
				2) "a"
				3) "c"
		3. Delete:
			 lpop key:  Delete the leftmost element of the list and return the element
			 rpop key:  Delete the rightmost element of the list and return the element

3.5. Set type set: duplicate elements are not allowed

		1. Storage: sadd key value
			127.0.0.1:6379> sadd myset a
			(integer) 1
			127.0.0.1:6379> sadd myset a
			(integer) 0
		2. obtain: smembers key:obtain set All elements in the collection
			127.0.0.1:6379> smembers myset
			1) "a"
		3. Delete: srem key value:delete set An element in the collection	
			127.0.0.1:6379> srem myset a
			(integer) 1

3.6. Ordered set type sortedset: duplicate elements are not allowed, and elements have order. Each element will be associated with a score of double type. redis sorts the members of the collection from small to large by scores.

		1. Storage: zadd key score value
			127.0.0.1:6379> zadd mysort 60 zhangsan
			(integer) 1
			127.0.0.1:6379> zadd mysort 50 lisi
			(integer) 1
			127.0.0.1:6379> zadd mysort 80 wangwu
			(integer) 1
		2. obtain: zrange key start end [withscores]
			127.0.0.1:6379> zrange mysort 0 -1
			1) "lisi"
			2) "zhangsan"
			3) "wangwu"

			127.0.0.1:6379> zrange mysort 0 -1 withscores
			1) "zhangsan"
			2) "60"
			3) "wangwu"
			4) "80"
			5) "lisi"
			6) "500"
		3. Delete: zrem key value
			127.0.0.1:6379> zrem mysort lisi
			(integer) 1

3.7. General order

		1. keys *: query all keys
		2. type key: get the type of value corresponding to the key
		3. del key: delete the specified key value

4. Persistence

	1. redis Is a memory database, when redis Restart the server, restart the computer, data will be lost, we can redis Data in memory is persisted to files on the hard disk.
	2. redis Persistence mechanism:
		1. RDB: By default, no configuration is required, and this mechanism is used by default
			* In a certain interval of time, detection key And then persist the data
			1. edit redis.windwos.conf file
				#   after 900 sec (15 min) if at least 1 key changed
				save 900 1
				#   after 300 sec (5 min) if at least 10 keys changed
				save 300 10
				#   after 60 sec if at least 10000 keys changed
				save 60 10000
				
			2. Restart redis Server and specify the profile name
				D:\JavaWeb2018\day23_redis\data\redis\windows-64\redis-2.8.9>redis-server.exe redis.windows.conf	
			
		2. AOF: In the way of logging, the operation of each command can be recorded. Data can be persisted after each command operation
			1. edit redis.windwos.conf file
				appendonly no(close aof) --> appendonly yes (open aof)
				
				# appendfsync always: persist every operation
				appendfsync everysec :  Persistence every second
				# appendfsync no: no persistence

5.Java client Jedis

	Jedis: One java operation redis Tools for databases.
	//Use steps:
		1. download jedis Of jar package
		2. use
			//1. Get connection
    		Jedis jedis = new Jedis("localhost",6379);
   			//2. Operation
   			jedis.set("username","zhangsan");
    		//3. Close the connection
    		jedis.close();

	
	Jedis Various operations redis Data structure in
		1) String type string
			set
			get
			
			 //1. Get connection
	        Jedis jedis = new Jedis();//If the null parameter structure is used, the default value is "localhost", port 6379
	        //2. Operation
	        //storage
	        jedis.set("username","zhangsan");
	        //obtain
	        String username = jedis.get("username");
	        System.out.println(username);
	
	        //You can use the setex() method to store the key value that can specify the expiration time
	        jedis.setex("activecode",20,"hehe");//Store the key value pair of activecode: hehe into redis, and automatically delete the key value pair after 20 seconds
	
	        //3. Close the connection
	        jedis.close();

		2) Hash type hash :  map format  
			hset
			hget
			hgetAll
			//1. Get connection
	        Jedis jedis = new Jedis();//If the null parameter structure is used, the default value is "localhost", port 6379
	        //2. Operation
	        // Storage hash
	        jedis.hset("user","name","lisi");
	        jedis.hset("user","age","23");
	        jedis.hset("user","gender","female");
	
	        // Get hash
	        String name = jedis.hget("user", "name");
	        System.out.println(name);
	
	
	        // Get data from all map s of hash
	        Map<String, String> user = jedis.hgetAll("user");
	
	        // keyset
	        Set<String> keySet = user.keySet();
	        for (String key : keySet) {
	            //Get value
	            String value = user.get(key);
	            System.out.println(key + ":" + value);
	        }
	
	        //3. Close the connection
	        jedis.close();


		3) List type list :  linkedlist Format. Support for repeating elements
			lpush / rpush
			lpop / rpop
			lrange start end : Scope acquisition
			
			 //1. Get connection
	        Jedis jedis = new Jedis();//If the null parameter structure is used, the default value is "localhost", port 6379
	        //2. operation
	        // list store
	        jedis.lpush("mylist","a","b","c");//Save from left
	        jedis.rpush("mylist","a","b","c");//Save from right
	
	        // Get list range
	        List<String> mylist = jedis.lrange("mylist", 0, -1);
	        System.out.println(mylist);
	        
	        // list pop up
	        String element1 = jedis.lpop("mylist");//c
	        System.out.println(element1);
	
	        String element2 = jedis.rpop("mylist");//c
	        System.out.println(element2);
	
	        // Get list range
	        List<String> mylist2 = jedis.lrange("mylist", 0, -1);
	        System.out.println(mylist2);
	
	        //3. Close the connection
	        jedis.close();


		4) Collection type set  :  Duplicate elements are not allowed
			sadd
			smembers:Get all elements

			//1. Get connection
	        Jedis jedis = new Jedis();//If the null parameter structure is used, the default value is "localhost", port 6379
	        //2. Operation
	
	
	        // set storage
	        jedis.sadd("myset","java","php","c++");
	
	        // set get
	        Set<String> myset = jedis.smembers("myset");
	        System.out.println(myset);
	
	        //3. Close the connection
	        jedis.close();
		5) Ordered set type sortedset: Duplicate elements are not allowed, and elements have order
			zadd
			zrange

			//1. Get connection
	        Jedis jedis = new Jedis();//If the null parameter structure is used, the default value is "localhost", port 6379
	        //2. Operation
	        // sortedset storage
	        jedis.zadd("mysortedset",3,"Arthur");
	        jedis.zadd("mysortedset",30,"offspring");
	        jedis.zadd("mysortedset",55,"Sun WuKong");
	
	        // sortedset get
	        Set<String> mysortedset = jedis.zrange("mysortedset", 0, -1);
	
	        System.out.println(mysortedset);
	
	
	        //3. Close the connection
	        jedis.close();


	
	jedis Connection pool: JedisPool
		//use:
			1. establish JedisPool Connection pool object
			2. Call method getResource()Method acquisition Jedis connect
				//0. Create a configuration object
		        JedisPoolConfig config = new JedisPoolConfig();
		        config.setMaxTotal(50);
		        config.setMaxIdle(10);
		
		        //1. Create a Jedis connection pool object
		        JedisPool jedisPool = new JedisPool(config,"localhost",6379);
		
		        //2. Get connection
		        Jedis jedis = jedisPool.getResource();
		        //3. Use
		        jedis.set("hehe","heihei");
		
		
		        //4. Close return to connection pool
		        jedis.close();
		
		//Connection pool tool class
			public class JedisPoolUtils {

			    private static JedisPool jedisPool;
			
			    static{
			        //Read profile
			        InputStream is = JedisPoolUtils.class.getClassLoader().getResourceAsStream("jedis.properties");
			        //Create Properties object
			        Properties pro = new Properties();
			        //Associated files
			        try {
			            pro.load(is);
			        } catch (IOException e) {
			            e.printStackTrace();
			        }
			        //Get data and set it to JedisPoolConfig
			        JedisPoolConfig config = new JedisPoolConfig();
			        config.setMaxTotal(Integer.parseInt(pro.getProperty("maxTotal")));
			        config.setMaxIdle(Integer.parseInt(pro.getProperty("maxIdle")));
			
			        //Initialize JedisPool
			        jedisPool = new JedisPool(config,pro.getProperty("host"),Integer.parseInt(pro.getProperty("port")));
			
			
			
			    }
			
			
			    /**
			     * Get connection method
			     */
			    public static Jedis getJedis(){
			        return jedisPool.getResource();
			    }
			}

6. Case:

Only the key code is posted: the complete one will be given in github

Case requirements:
	1. Provide index.html Page with a drop-down list of provinces
	2. When the page is loaded, send an ajax request to load all provinces


Note: use redis to cache some data that doesn't change frequently.
	Once the data of the database changes, the cache needs to be updated.
		The tables in the database need to add, delete, and modify. The redis cache data needs to be saved again
		In the add / delete / modify method corresponding to service, delete the redis data.

File of mysql database:

CREATE DATABASE day23; -- Create database
USE day23; 			   -- Use database
CREATE TABLE province(   -- Create table
	id INT PRIMARY KEY AUTO_INCREMENT,
	NAME VARCHAR(20) NOT NULL
	
);
-- insert data
INSERT INTO province VALUES(NULL,'Beijing');
INSERT INTO province VALUES(NULL,'Shanghai');
INSERT INTO province VALUES(NULL,'Guangzhou');
INSERT INTO province VALUES(NULL,'Shaanxi');

code:

<!DOCTYPE html>
<html lang="en">
<head>
    <meta charset="UTF-8">
    <title>Title</title>
    <script src="js/jquery-3.3.1.min.js"></script>
    
    <script>
        $(function(){
            //Send ajax request and load data of all provinces
            $.get("ProvinceServlet",{},function (data) {
                // [{"id":1,"name": "Beijing"}, {"id":2,"name": "Shanghai"}, {"id":3,"name": "Guangzhou"}, {"id":4,"name": "Shaanxi"}]
                //1. Get select
                var province=$("#province")
                //2. Traverse the json array
                $(data).each(function () {
                    //3. Create < option >
                    var option="<option name='"+this.id+"'>"+this.name+"</option>"
                    //4. Call append of select to append option
                    province.append(option);
                })
            });
        })
    </script>
</head>
<body>
    <select id="province">
        <option>--Please select a province--</option>
    </select>
</body>
</html>

Servlet Code:

package Per.Belief.RedisStudy.web.servlet;
import Per.Belief.RedisStudy.domain.Province;
import Per.Belief.RedisStudy.service.ProvinceService;
import Per.Belief.RedisStudy.service.impl.ProvinceServiceImpl;
import com.fasterxml.jackson.databind.ObjectMapper;

import javax.servlet.ServletException;
import javax.servlet.annotation.WebServlet;
import javax.servlet.http.HttpServlet;
import javax.servlet.http.HttpServletRequest;
import javax.servlet.http.HttpServletResponse;
import java.io.IOException;
import java.util.List;

@WebServlet("/ProvinceServlet")
public class ProvinceServlet extends HttpServlet {

    protected void doGet(HttpServletRequest req, HttpServletResponse resp) throws ServletException, IOException {
        this.doPost(req,resp);
    }

    protected void doPost(HttpServletRequest req, HttpServletResponse resp) throws ServletException, IOException {
        ProvinceService service=new ProvinceServiceImpl();
//        List<Province> list=service.findAll();
//        ObjectMapper mapper=new ObjectMapper();
//        String json=mapper.writeValueAsString(list);
        String json=service.findAllJson();


        System.out.println(json);
        resp.setContentType("application/json;charset=utf-8");
        resp.getWriter().write(json);
    }
}

ProvinceServiceIpml Code:

package Per.Belief.RedisStudy.service.impl;

import Per.Belief.RedisStudy.dao.ProvinceDao;
import Per.Belief.RedisStudy.dao.impl.ProvinceDaoImpl;
import Per.Belief.RedisStudy.domain.Province;
import Per.Belief.RedisStudy.jedis.util.JedisPoolUtils;
import Per.Belief.RedisStudy.service.ProvinceService;
import com.fasterxml.jackson.core.JsonProcessingException;
import com.fasterxml.jackson.databind.ObjectMapper;
import redis.clients.jedis.Jedis;

import java.util.List;

public class ProvinceServiceImpl implements ProvinceService {
    ProvinceDao pd=new ProvinceDaoImpl();
    @Override
    public List<Province> findAll() {
        return pd.findAll();
    }
    @Override
    public String findAllJson(){
        //1. Query data from redis first
        //1.1 get redis client connection
        Jedis jedis= JedisPoolUtils.getJedis();
        String province_json=jedis.get("province");

        //2 judge Province_ Whether the JSON data is null
        if(province_json==null||province_json.length()==0){
            //No data in redis
            System.out.println("redis No data in, query database!");

            //2.1 query from data
            List<Province> list= pd.findAll();

            ObjectMapper mapper=new ObjectMapper();
            try {
                province_json=mapper.writeValueAsString(list);

            } catch (JsonProcessingException e) {
                e.printStackTrace();
            }
            //2.3 saving json data into redis
            jedis.set("province",province_json);
            //Return connection
            jedis.close();

        }else{
            System.out.println("redis Data in, query cache!");
        }
        return province_json;
    }
}

effect:
First entry:


Second entry:

Posted by chriskiely on Sun, 07 Jun 2020 19:47:34 -0700

Hot Keywords

©2024 Programmer Group