NoSQL database

NoSQL generally refers to non relational databases, which can be a good supplement to relational databases. With the rise of Internet web2.0 websites, the traditional relational database has been unable to cope with web2.0 websites, especially the super large-scale and highly concurrent SNS type web2.0 pure dynamic websites, which has exposed many insurmountable problems, while the non relational database has developed very rapidly due to its own characteristics.

​ The generation of NoSQL database is to solve the challenges brought by large-scale data sets and multiple data types, especially the problems of big data application. NoSQL database can not use sql, not relational database, to solve the problem of high concurrency. Because a single machine like mysql or oracle can withstand a maximum of several thousand concurrencies

Mainstream NoSQL

The four categories of NoSQL database are as follows:

• Key value storage database (map)

  • Related products: Tokyo Cabinet/Tyrant, Redis, Voldemort, Berkeley DB

    Typical application: content caching, which is mainly used to handle the high access load of a large amount of data.

    Data model: a series of key value pairs

    Advantage: quick query

    Disadvantages: the stored data is not structured and needs to be designed by yourself

• Column storage database

  • Related products: Cassandra, HBase, Riak

    Typical application: distributed file system under big data

    Data model: it is stored in column clusters to store the same column of data together

    Advantages: fast search speed, strong scalability and easier distributed expansion

    Disadvantages: relatively limited functions

• Document database

  • Related products: CouchDB, MongoDB

    Typical application: Web application (similar to key Value, Value is a structured bson) user1001 {"username": "admin", "age": 12}

    Data model: a series of key value pairs

    Advantage: the data structure requirements are not strict

    Disadvantages: low query performance and lack of unified query syntax

• Graph database

  • Related databases: Neo4J, InfoGrid, Infinite Graph

    Typical application: Social Network

    Data model: diagram structure

    Advantages: using graph structure related algorithms.

    Disadvantages: you need to calculate the whole graph to get the results. It is not easy to make a distributed cluster scheme.

What is redis

​ Redis is an open source high-performance key value database developed in C language. It provides a variety of key value data types to meet the storage requirements in different scenarios. So far, redis supports the following key value data types:

• String type string

• Hash type hash

• List type list

• Set type set

• Ordered collection type sortedSet.

redis application scenario

• Content cache (data query, short connection, news content, commodity content, etc.). (maximum use)

• session separation in distributed cluster architecture.

• Chat room's online friends list.

• Task queue. (second kill, rush purchase, 12306, etc.) 100 iPhones are free,

• App leaderboard.

• Website access statistics.

• Data expiration processing (accurate to milliseconds)

redis installation steps

1:wget downloads a tar package or scp online to CentOS 7
2:tar -zxvf redis-4.0.9.tar.gz
3:cd redis-4.0.9
4:make
5:make PREFIX=/usr/app/redis install
6:cp /usr/app/redis-4.0.9/redis.conf /usr/app/redis/bin/
7: cd /usr/app/redis/bin
9:./redis-server redis.conf
10: ./redis-cli -h 127.0.0.1 -p 6379
11:set test libai
12:get test
13:del test

Three connection modes of redis
1: . / redis cli - H IP address - p port number
Shut down the redis server
Send the shutdown command through the client
2: Connect through the visualization tool (Note: the configuration of redis.conf needs to be modified for remote connection)
1: Comment out bind 127.0.0.1
2: Change protected mode to no
Modify the number of databases 16 to 1
You can set the access password of redis and the requirepass password
Daemon yes to run in the background
3: Through the java client (the java client mostly uses jedis jar package)
)You can also connect to the redis database

Does redis need a single machine to open more?
Redis is a single threaded key value database, which can make rational use of redis resources (when redis clusters)

redis data types and commands

1.string

​ redis does not use the string representation of C language, but a custom data structure called SDS (simple dynamic string), that is, simple dynamic string.

//Assignment and value set key value get key value
set test 1234
get test 
Jedis jedis = new Jedis("192.168.192.3",6379);
jedis.set("test","123456");
System.out.println(jedis.get("test"));

//Incrementing the number incr key (if a number is stored) 
incr test 
jedis.incr("test");
//Increase the specified integer incrby key number
incrby test 2
jedis.incrBy("test",2);
//Reduce integer decr key   
decr test 
jedis.decr("test");
//Decrease the specified number by key number
decrby key test 2    
jedis.decrBy("test",2);

//append key value to the tail
append test "world"
jedis.set("test","hello");
jedis.append("test"," world!");
System.out.println(jedis.get("test"));

//Get string length strlen key
strlen test
System.out.println(jedis.strlen("test"));

//Set / get multiple key values at the same time Mset key value mget key value
jedis.mset("1","1","2","2","3","3");
System.out.println(jedis.mget("1","2","3"));

//Self increasing primary key (self splicing)
set its: id 1
its: 1  2
its: 2  2
String key="its:"+request.getParameter("id");
String value=jedis.get(key);

2.hash

• Map<String,HashMap<String,String>>

Problems with using String

User:1 {"id":1,"username":zhangsan,"age":20}

Update the user with id 1 and age changed to 22

User object -- JSON (string) -- redis

If you only modify age without modifying other attributes, it will cause a waste of resources

• Hash is called hash type, which provides the mapping of fields and field values. The field value can only be string type, and other types such as hash type and collection type are not supported.

User:1 username zhangsan

​ Password 123456

​ Age 23

//hset sets one value at a time and hget gets one value at a time
//hmset sets multiple values at once 		 hmget gets multiple values at once
//Assignment and value hmset key key1 value key2 value hmget key key1 key2
hmset a name zhangsan age 20
hmget a name age
Map map =  new HashMap();
map.put("name","zhangsan");
map.put("age","20");
jedis.hmset("a",map);
System.out.println(jedis.hmget("a","name","age"));
//Get all attribute hgetall key s
System.out.println(jedis.hgetAll("a"));

//Determine whether there are hexists key key1 in the field
hexists a age
System.out.println(jedis.hexists("a","age"));

//Assign hsetnx key key1 value1 when the field does not exist
hsetnx a age 22
jedis.hsetnx("a","age","22");
System.out.println(jedis.hget("a","age"));

//Add the number hincrby key key1 number
hincrby a age 2
hget a age
jedis.hincrBy("a","age",2);
System.out.println(jedis.hget("a","age"));

//Delete field Deletes one or more fields. The return value is the number of deleted fields
//hdel key key1 key2
jedis.hdel("a","name");

//Get only the field name or field value hkeys key hvals key
hkeys a 
hvals a
System.out.println(jedis.hkeys("a"));
System.out.println(jedis.hvals("a"));
//Get field quantity hlen key
hlen a
System.out.println(jedis.hlen("a"));

• Application store commodity information

3.list

• The difference between ArrayList and linkedList
  • ArrayList uses array to store data. When opening up storage space, continuous storage space is required. Therefore, querying data according to index is fast, and displacement operation needs to be designed when adding or deleting elements, so it is relatively slow.
  • LinkedList uses two-way links to store data without continuous storage space. Each element records the pointers of the front and rear elements. Therefore, when inserting and deleting data, it is only necessary to change the pointer of the front and rear elements. The speed is very fast. When querying elements through subscripts, it needs to index from the beginning, so it is relatively slow, However, it is faster to query the first or last elements.

//Add the element lpush key value1 Value2 rpush key value1 Value2 to both ends of the list
jedis.lpush("b","1","2","3");
jedis.rpush("b","1","2","3");
//Get list fragment lrange list startindex endindex
System.out.println(jedis.lrange("b",0,-1));

//Pop up lpop key and rpop key from both ends of the list
jedis.lpop("b");
jedis.rpop("b");
System.out.println(jedis.lrange("b",0,-1));

//Get the number of elements in the list len key
System.out.println(jedis.llen("b"));

//Delete the value krem key count value specified in the list
//When count > 0, LREM will be deleted from the left of the list. 
//When count < 0, LREM will delete from the back of the list. 
//When count=0, LREM deletes all elements with value.
jedis.lrem("b",2,"2");
System.out.println(jedis.lrange("b",0,-1));

//Gets / sets the element value of the specified index lindex key index lset key index value
System.out.println(jedis.lindex("b",2));
jedis.lset("b",2,"3");

//Only the specified segments in the list are retained, and the specified range is consistent with lrange. ltrim key start stop


//Insert the element linsert key before|after pivot value into the list
 The command first looks for the value from left to right in the list pivot The element, then according to the second parameter is BEFORE still AFTER To decide to value Insert before or after the element

    
//Move elements from one list to another rpoplpush list newlist
    

• Apply product review list

You can also realize the scene of rush buying and second killing: act as a queue (first in, first out) lpush rpop

You can also comment on users' products

Idea:

Create a product comment list in redis

The user publishes product comments, converts the comment information into json and stores it in the list.

The user queries the comment list on the page, and takes out json data from redis and displays it on the page.

Items: item ID hash type

Items:comments: item ID '{"Id": 1, "name": "good, good!!", "date":1430295077289}'

'{"id":2,"name": "garbage!!", "date":1430295077289}'

Define item comment list key:

Item comment with item number 1001 key: items: comment:1001

192.168.101.3:7001 > lpush items: comment: 1001 '{"Id": 1, "name": "good, good!!", "date":1430295077289}'

4.set

set in Java is an interface, which is out of order and cannot be repeated

The common operations of a collection type are adding or deleting elements to the collection, judging whether an element exists, etc. because Redis of a collection type uses a hash table with an empty value, the time complexity of all these operations is 0 (1).

Redis also provides the operations of intersection, union and difference sets between multiple sets.

//Add delete element Sadd key member

//Get the smember key of all elements in the collection

//Determine whether the element is in the collection. sismember key member

5.sorted set Zset

Based on the set type, the ordered set type associates each element in the set with a score, which enables us to not only insert, delete and judge whether the element exists in the set, but also obtain the top N elements with the highest or lowest score, obtain the elements within the specified score range and other score related operations.

In some ways, ordered sets and list types are somewhat similar.

1. Both are orderly.

2. Both can get a range of elements.

However, there are great differences between the two:

1. The list type is realized through a linked list. It is very fast to obtain data near both ends. When the number of elements increases, the speed of accessing intermediate data will slow down.

2. The ordered collection type is implemented using a hash table, and all data can be read quickly even if it is in the middle.

3. You can't simply adjust the position of an element in the list, but the ordered collection can be achieved (by changing the score)

4. Ordered collections consume more memory than list types.

//Add element adds an element and its score to the ordered set. If the element already exists, the original score will be replaced with a new score. The return value is the number of elements newly added to the collection, excluding the existing elements. zadd key score member

//Get the score of element zscore key member

//Zrange key start stop WithCores