title: Redis6 Introduction (I)

NoSQL database

NoSQL database

summary

NoSQL(NoSQL = Not Only SQL), which means "not just SQL", generally refers to non relational databases. NoSQL does not rely on business logic storage, but is stored in a simple key value mode. Therefore, it greatly increases the expansion ability of the database.

• SQL standards are not followed.
• ACID is not supported. (four characteristics of transactions: atomicity, consistency, isolation and persistence)
• Far exceeds the performance of SQL.

Applicable scenario

• Highly concurrent reading and writing of data
• Massive data reading and writing
• For high data scalability

Inapplicable scenario

• Transaction support required
• Structured query storage based on sql, dealing with complex relationships, requires ad hoc query.

(consider using NoSql if you don't need sql or if you can't use sql)

Memcache

• Early NoSql database
• The data is in memory and is generally not persistent
• It supports simple key value mode and single type
• Generally, it is a database that is secondary persistent as a cache database

Redis

• It covers almost most functions of Memcached
• The data is in memory and supports persistence. It is mainly used for backup and recovery
• In addition to supporting the simple key value mode, it also supports the storage of a variety of data structures, such as list, set, hash, zset, etc.
• Generally, it is a database that is secondary persistent as a cache database

MongoDB

• High performance, open source, schema free document database
• The data is in memory. If the memory is insufficient, save the infrequent data to the hard disk
• Although it is a key value mode, it provides rich query functions for value (especially json)
• Support binary data and large objects
• It can replace RDBMS and become an independent database according to the characteristics of data. Or with RDBMS, store specific data

Line storage database

Column storage database

Graph relational database

Main applications: social relations, public transport network, map and network topology (n*(n-1)/2)

Cassandra[kəˈsændrə]

Apache Cassandra is a free open source NoSQL database, which is designed to manage massive data sets (usually up to PB level) on a large cluster built by a large number of commercial servers. Among the many significant features, Cassandra's most outstanding advantage is to adjust the scale of write and read operations, and it does not emphasize the design idea of the main cluster, which can simplify the creation and expansion process of each cluster in a relatively intuitive way.

Hbase

HBase is the database in the Hadoop project. It is used in scenarios that require random and real-time read and write operations on a large amount of data.

The goal of HBase is to process tables with a very large amount of data. It can process more than 1 billion rows of data with ordinary computers and data tables with millions of columns of elements.

Redis installation

introduce

• Redis is an open source key value storage system.
• Similar to Memcached, it supports relatively more stored value types, including string (string), list (linked list), set (set), zset(sorted set -- ordered set) and hash (hash type).
• These data types support push/pop, add/remove, intersection, union, difference and richer operations, and these operations are atomic.
• On this basis, Redis supports sorting in different ways.
• Like memcached, data is cached in memory to ensure efficiency.
• The difference is that Redis will periodically write the updated data to the disk or write the modification operation to the additional record file.
• On this basis, master-slave synchronization is realized

Applicable scenario

Cache with relational database

• High frequency, hot access data, reduce database IO
• Distributed architecture for session sharing

A variety of data structures store persistent data

Install Redis for Ubuntu

#install
sudo apt update
sudo apt install redis-server

#start-up
redis-server

#Check whether redis is started
redis-cli
root@iZbp13941xpzjmefjge9chZ:~# redis-cli
127.0.0.1:6379> auth "lxb991209"
127.0.0.1:6379> ping
PONG

Ubuntu default directory / usr/bin

redis-benchmark:The performance test tool can be run on your own machine to see how your own machine performs
redis-check-aof: Fix the problem AOF Documents, rdb and aof Later 
redis-check-dump: Fix the problem dump.rdb file
redis-sentinel: Redis Cluster use
redis-server: Redis Server start command
redis-cli: Client, operation portal

Foreground start (not recommended)

redis-server

Background startup (recommended)

cp -r redis.conf /etc/redis.conf
redis-server /etc/redis.conf

Relevant knowledge

Default port 6379
 There are 16 databases by default. Similar to the array, the subscript starts from 0. The initial default is library 0
 Use command select   <dbid>To switch databases. as: select 8
 Unified password management, all libraries have the same password.
dbsize View the current database key Number of
flushdb Empty current library
flushall Kill all libraries

Redis is a single thread + multiple IO multiplexing technology

Multiplexing refers to using a thread to check the ready status of multiple file descriptors (sockets). For example, call the select and poll functions to pass in multiple file descriptors. If one file descriptor is ready, it will return. Otherwise, it will be blocked until timeout. After the ready state is obtained, the real operation can be performed in the same thread, or the thread execution can be started (such as using thread pool)

Serial vs multithreading + memcached vs single thread + IO multiplexing (Redis)

(

There are three differences from Memcache:

• Support multiple data types
• Support persistence
• Single thread + multiplex IO

)

Five data types

Redis key

keys *                ==View all of the current library key==    (Match: keys *1)
exists key            Judge a key Does it exist
type key              View your key What type is it
del key               Delete the specified key data
unlink key            according to value Select non blocking delete to delete only keys from keyspace     The metadata is deleted. The real deletion will be performed in subsequent asynchronous operations.
expire key 10         10 Seconds: for a given key Set expiration time
ttl key               See how many seconds are left to expire,-1 Means never expires,-2 Indicates that it has expired
 
select                Command switch database
dbsize               View the current database key Number of
flushdb              Empty current library
flushall             Kill all libraries

Redis string (String)

• As like as two peas, String is the most basic type of Redis. You can understand it as a type that is exactly the same as Memcached, and a key corresponds to a value.
• The string type is binary safe. This means that the Redis string can contain any data. For example, jpg images or serialized objects.
• String type is the most basic data type of Redis. The string value in a Redis can be 512M at most

Common commands

set <key><value>      Add key value pair
    *NX: When in the database key When it does not exist, you can key-value Add database
    *XX: When in the database key When present, you can key-value Add database, and NX Parameter mutual exclusion
    *EX: key Timeout seconds
    *PX: key Timeout milliseconds, and EX mutex

get     <key>             Query corresponding key value
append  <key><value>      Will the given<value> Append to end of original value
strlen  <key>             Gets the length of the value
setnx  <key><value>      Only in key When not present    set up key Value of
 
incr  <key>             take key The numeric value stored in the is incremented by 1,You can only operate on numeric values. If it is blank, the new increment is 1
decr  <key>             take key The numeric value stored in minus 1,You can only operate on numeric values. If it is blank, the new value added is-1
incrby / decrby  <key> <step>   take key Increase or decrease the digital value stored in. Custom step size.        
mset  <key1><value1><key2><value2>  .....    Set one or more at the same time key-value yes  
mget  <key1><key2><key3> .....               Get one or more at the same time value  
msetnx <key1><value1><key2><value2>  .....   Set one or more at the same time key-value Yes, if and only if all given key None of them exist.
Atomicity, if one fails, all fail
 
getrange  <key><Starting position><End position>             Get the range of values, similar to java Medium substring，Front package
setrange  <key><Starting position><value>               use <value>  Overwrite<key>Stored string value, from<Starting position>start(Index starts at 0). 
 
setex  <key><Expiration time><value>                  Set the expiration time in seconds while setting the key value.
getset <key><value>                           Replace the old with the new, set the new value and obtain the old value at the same time.

incr key: performs atomic plus 1 operation on the value stored in the specified key

Atomic operation refers to the operation that will not be interrupted by thread scheduling mechanism;

Once this operation starts, it runs until the end without any context switch (switching to another thread).

(1) In a single thread, any operation that can be completed in a single instruction can be regarded as an "atomic operation", because interrupts can only occur between instructions.

(2) In multithreading, operations that cannot be interrupted by other processes (threads) are called atomic operations.

The atomicity of Redis single command mainly benefits from the single thread of Redis

data structure

The data structure of String is Simple Dynamic String (abbreviated as SDS). It is a String that can be modified. Its internal structure is similar to the ArrayList of Java. It uses pre allocation of redundant space to reduce frequent memory allocation

The internal space capacity actually allocated for the current string is generally higher than the actual string length len. When the string length is less than 1M, the expansion is to double the existing space. If it exceeds 1M, only 1M more space will be expanded at a time. Note that the maximum length of the string is 512M.

Redis list

brief introduction

Single key multi value

Redis list is a simple string list, sorted by insertion order. You can add an element to the head (left) or tail (right) of the list. Its bottom layer is actually a two-way linked list, which has high operation performance at both ends. The performance of the middle node through index subscript operation will be poor.

Common commands

lpush/rpush  <key><value1><value2><value3> .... From the left/Insert one or more values to the right.
lpop/rpop  <key>            From the left/Spit out a value on the right. The value is in the key, and the light key dies.
rpoplpush  <key1><key2>     from<key1>Spit out a value on the right side of the list and insert it into<key2>To the left of the list.
lrange <key><start><stop>     Get elements by index subscript(From left to right)
lrange mylist 0 -1             0 First on the left,-1 First on the right, (0-1 (indicates get all)
lindex <key><index>          Get elements by index subscript(From left to right)
llen <key>                    Get list length
linsert <key>  before/after <value><newvalue>     stay<value>Insert after<newvalue>insert values
lrem <key><n><value>                         Delete from left n individual value(From left to right)
lset<key><index><value>                    Will list key Subscript is index Replace the value of with value

data structure

The data structure of List is = = quickList==

First, when there are few list elements, a continuous memory storage will be used. This structure is ziplost, that is, compressed list. It stores all the elements next to each other and allocates a continuous piece of memory. When there is a large amount of data, it will be changed to quicklist. Because the additional pointer space required by ordinary linked lists is too large, it will waste space. For example, only int type data is stored in this list, and two additional pointers prev and next are required in the structure.

Redis combines the linked list and zipplist to form a quicklist. That is to string multiple ziplist s using bidirectional pointers. This not only meets the fast insertion and deletion performance, but also does not appear too much spatial redundancy.

Redis set

brief introduction

Redis set's external functions are similar to those of a list. The special feature is that set can automatically eliminate duplication. Set provides an important interface to judge whether a member is in a set set, which is not provided by a list. Redis set is an unordered set of string type. Its bottom layer is actually a hash table with null value, so the complexity of adding, deleting and searching is O(1).

An algorithm, with the increase of data, the length of execution time. If it is O(1), the data increases and the time to find the data remains the same

Common commands

sadd <key><value1><value2> .....   One or more member Element is added to the collection key Existing in member Element will be ignored
smembers <key>                     Gets all the values of the collection.
sismember <key><value>             Judgment set<key>Whether it contains the<value>Value, with 1, without 0
scard<key>                         Returns the number of elements in the collection.
srem <key><value1><value2> ....    Delete an element in the collection.
spop <key>                         Spit out a value randomly from the set.
srandmember <key><n>               Randomly from the set n Values. Is not removed from the collection.
smove <source><destination>value        Moves a value in a set from one set to another
sinter <key1><key2>                        Returns the intersection element of two collections.
sunion <key1><key2>                        Returns the union element of two collections.
sdiff <key1><key2>                Returns the difference element of two sets(key1 In, excluding key2 Medium)

data structure

The set data structure is a dict dictionary, which is implemented with a hash table. The internal implementation of HashSet in Java uses HashMap, but all values point to the same object. The same is true for Redis's set structure. It also uses a hash structure internally. All values point to the same internal value.

Redis hash

brief introduction

Redis hash is a collection of key value pairs.

Redis hash is a mapping table of field and value of string type. Hash is especially suitable for storing objects.

Similar to map < string, Object > in Java

The user ID is the searched key and the stored value. The user object contains name, age, birthday and other information. If it is stored in an ordinary key/value structure

The corresponding attribute data can be operated through key (user ID) + field (attribute tag). There is no need to store data repeatedly, and there will be no problems of serialization and concurrent modification control

Common commands

hset <key><field><value>         to<key>In the collection<field>Key assignment<value>
hget <key1><field>                from<key1>aggregate<field>take out value
hmset <key1><field1><value1><field2><value2>...         Batch settings hash Value of
hexists<key1><field>                        View hash table key Given domain field Whether it exists.
hkeys <key>                                List the hash All of the collection field
hvals <key>                                List the hash All of the collection value
hincrby <key><field><increment>           Is a hash table key Domain in field Value of plus increment 1   -1
hsetnx <key><field><value>                Hash table key Domain in field The value of is set to value ，If and only if domain field non-existent .

data structure

There are two data structures corresponding to the Hash type: ziplost (compressed list) and hashtable (Hash table). When the field value length is short and the number is small, use ziplist; otherwise, use hashtable.

Redis ordered set (Zset)

brief introduction

Redis ordered set zset is very similar to ordinary set. It is a string set without duplicate elements.

The difference is that each member of the ordered set is associated with a score, which is used to sort the members of the set from the lowest score to the highest score. The members of the collection are unique, but the scores can be repeated.

Because the elements are ordered, you can also quickly get a range of elements according to score or position.

Accessing the intermediate elements of an ordered set is also very fast, so you can use an ordered set as a smart list without duplicate members.

Common commands

zadd  <key><score1><value1><score2><value2>...        One or more member Elements and their score Add value to ordered set key among.
zrange <key><start><stop>  [WITHSCORES]            Returns an ordered set key Middle, subscript in<start><stop>Element bands between WITHSCORES，You can return scores together with values to the result set.
zrangebyscore key minmax [withscores] [limit offset count]       Returns an ordered set key In, all score Value between min and max between(Including equal min or max )Members of. Ordered set members by score Value increment(from small to large)Order.
zrevrangebyscore key maxmin [withscores] [limit offset count]    Ditto, change to order from large to small.
zincrby <key><increment><value>                    For element score Plus increment
zrem  <key><value>                                 Delete the element with the specified value under the collection
zcount <key><min><max>                             Count the number of elements in the set and score interval
zrank <key><value>                                   Returns the ranking of the value in the collection, starting from 0.

data structure

SortedSet(zset) is a very special data structure provided by Redis. On the one hand, it is equivalent to the Java data structure map < string, double >, which can give each element value a weight score. On the other hand, it is similar to TreeSet. The internal elements will be sorted according to the weight score to get the ranking of each element, You can also get the list of elements through the scope of score.

The underlying zset uses two data structures

(1) hash is used to associate the element value with the weight score to ensure the uniqueness of the element value. The corresponding score value can be found through the element value.

(2) Jump table. The purpose of jump table is to sort the element value and obtain the element list according to the range of score.

Jump table

brief introduction

Orderly collection is common in life, such as ranking students according to their grades, ranking players according to their scores, etc. For the underlying implementation of ordered sets, arrays, balanced trees, linked lists, etc. can be used. Insertion and deletion of array elements; Balanced tree or red black tree has high efficiency but complex structure; The linked list query needs to traverse all, which is inefficient. Redis uses a jump table. The efficiency of jump table is comparable to that of red black tree, and the implementation is much simpler than that of red black tree.

example

Compare the ordered linked list and jump list, and query 51 from the linked list

(1) Ordered linked list

To find the element with the value of 51, you need to start from the first element and find it by searching and comparing in turn. A total of 6 comparisons are required.

(2) Jump table

Starting from layer 2, node 1 is smaller than node 51, which is compared backward.

Node 21 is smaller than node 51. Continue to compare backward, followed by NULL, so go down from node 21 to layer 1

In layer 1, node 41 is smaller than node 51. Continue backward, and node 61 is larger than node 51, so it goes down from 41

In layer 0, node 51 is the node to be found. The node is found for 4 times.

It can be seen that the efficiency of jump list is higher than that of ordered linked list