Index and query optimization of MongoDB

Keywords: Attribute MongoDB Mobile Database

M
The indexing mechanism of ongoDB is basically similar to that of ordinary databases, mainly in the following parts:

Single field index

By default, MongoDB creates a single field index of the _id field for all collections, which is unique and cannot be deleted (_id is the primary key of the collection)

Index creation method:

db.customers.ensureIndex({name:1},{unique:false} )

Query index:

db.system.indexes.find()

Query results:

{ "v" : 1, "name" : "_id_", "key" : { "_id" : 1 }, "ns" : "test.customers" }
{ "v" : 1, "name" : "name_1", "key" : { "name" : 1 }, "ns" : "test.customers" }

Explain indexed query selectors:

> db.customers.find({name:'zhangsan'}).explain()
{
"cursor" : "BtreeCursor name_1",//Indicates that the index is used for the query
"isMultiKey" : false,//Unused multi-key composite index
"n" : 10,//Query selection matches the number of records
"nscannedObjects" : 10,//Number of document objects scanned by executing queries
"nscanned" : 10,//Total number of documents or indexes scanned
"nscannedObjectsAllPlans" : 10,//Scanning Wendan Total Among All Query Plans
"nscannedAllPlans" : 10,//Total number of documents or lines scanned in all query plans
"scanAndOrder" : false,//Whether to sort the data when retrieving the queried data from the cursor
"indexOnly" : false,//
"nYields" : 0,//Produced reading locks
"nChunkSkips" : 0,
"millis" : 0,//Query time consuming (ms)
"indexBounds" : {
"name" : [
[
"zhangsan",
"zhangsan"
]
]
},
"server" : "raspberrypi:27017"
}

Explain the meaning of the above // the following part;

Note: The above annotations will be used later, and they are often used when analyzing queries. It's better to write them down.

Composite index

Composite index mainly refers to adding indexes to multiple fields at the same time, so composite index supports queries matching multiple fields.
Create a composite index:

db.customers.ensureIndex({id:1,age:1})

Query index results:

> db.system.indexes.find()
{ "v" : 1, "name" : "_id_", "key" : { "_id" : 1 }, "ns" : "test.customers" }
{ "v" : 1, "name" : "name_1", "key" : { "name" : 1 }, "ns" : "test.customers" }
{ "v" : 1, "name" : "id_1_age_1", "key" : { "id" : 1, "age" : 1 }, "ns" : "test.customers" }

Using an index example:

> db.customers.find({id:{$lt:5},age:{$gt:12}})
{ "_id" : ObjectId("589835c41c85cb68725f789b"), "id" : 3, "name" : "zhangsan", "age" : 13 }
{ "_id" : ObjectId("589835c41c85cb68725f789c"), "id" : 4, "name" : "zhangsan", "age" : 14 }

Explain the execution example:

> db.customers.find({id:{$lt:5},age:{$gt:12}}).explain()
{
"cursor" : "BtreeCursor id_1_age_1",
"isMultiKey" : false,
"n" : 2,
"nscannedObjects" : 2,
"nscanned" : 4,
"nscannedObjectsAllPlans" : 5,
"nscannedAllPlans" : 9,
"scanAndOrder" : false,
"indexOnly" : false,
"nYields" : 0,
"nChunkSkips" : 0,
"millis" : 0,
"indexBounds" : {
"id" : [
[
-1.7976931348623157e+308,
5
]
],
"age" : [
[
12,
1.7976931348623157e+308
]
]
},
"server" : "raspberrypi:27017"
}

Multi-key index of arrays

Creating an index for a field whose value is an array type defaults to creating an index for every element in the array.

> db.customers.ensureIndex({'orders.orders_id':1})

See:

> db.system.indexes.find()
{ "v" : 1, "name" : "_id_", "key" : { "_id" : 1 }, "ns" : "test.customers" }
{ "v" : 1, "name" : "name_1", "key" : { "name" : 1 }, "ns" : "test.customers" }
{ "v" : 1, "name" : "id_1_age_1", "key" : { "id" : 1, "age" : 1 }, "ns" : "test.customers" }
{ "v" : 1, "name" : "orders.products.product_name_1", "key" : { "orders.products.product_name" : 1 }, "ns" : "test.customers" }
{ "v" : 1, "name" : "orders.orders_id_1", "key" : { "orders.orders_id" : 1 }, "ns" : "test.customers" }

Interpretation and implementation:

> db.customers.find({'orders.orders_id':{$gte:0}}).explain()
{
"cursor" : "BtreeCursor orders.orders_id_1",
"isMultiKey" : false,
"n" : 1,
"nscannedObjects" : 1,
"nscanned" : 1,
"nscannedObjectsAllPlans" : 1,
"nscannedAllPlans" : 1,
"scanAndOrder" : false,
"indexOnly" : false,
"nYields" : 0,
"nChunkSkips" : 0,
"millis" : 0,
"indexBounds" : {
"orders.orders_id" : [
[
0,
1.7976931348623157e+308
]
]
},
"server" : "raspberrypi:27017"
}

Another one:

> db.customers.find({'orders.products.product_name':'iphone'})
{ "_id" : ObjectId("58983d0fc55e261327343eab"), "id" : 11, "name" : "lisi", "orders" : [ { "orders_id" : 1, "create_time" : "2017-02-06", "products" : [ { "product_name" : "MiPad", "price" : "$100.00" }, { "product_name" : "iphone", "price" : "$399.00" } ] } ], "mobile" : "13161020110", "address" : { "city" : "beijing", "street" : "taiyanggong" } }
> db.customers.find({'orders.products.product_name':'iphone'}).explain()
{
"cursor" : "BtreeCursor orders.products.product_name_1",
"isMultiKey" : true,
"n" : 1,
"nscannedObjects" : 1,
"nscanned" : 1,
"nscannedObjectsAllPlans" : 1,
"nscannedAllPlans" : 1,
"scanAndOrder" : false,
"indexOnly" : false,
"nYields" : 0,
"nChunkSkips" : 0,
"millis" : 0,
"indexBounds" : {
"orders.products.product_name" : [
[
"iphone",
"iphone"
]
]
},
"server" : "raspberrypi:27017"
}

Index management

The creation of index has been described above. The following is a summary of the format of index creation:

db.collection.ensureIndex(keys,options)

key is a document document, which contains fields that need to be indexed and the direction of index ranking; option is optional to control the way of index creation;
All indexes are stored in the collection system.indexes.
The index is deleted by:

db.collection.dropIndex(indexName)

For example, delete the name_1 index of customers:

> db.customers.dropIndex('name_1')
{ "nIndexesWas" : 5, "ok" : 1 }
> db.system.indexes.find()
{ "v" : 1, "name" : "_id_", "key" : { "_id" : 1 }, "ns" : "test.customers" }
{ "v" : 1, "name" : "id_1_age_1", "key" : { "id" : 1, "age" : 1 }, "ns" : "test.customers" }
{ "v" : 1, "name" : "orders.products.product_name_1", "key" : { "orders.products.product_name" : 1 }, "ns" : "test.customers" }
{ "v" : 1, "name" : "orders.orders_id_1", "key" : { "orders.orders_id" : 1 }, "ns" : "test.customers" }

Slow Query Monitoring

1. MongoDB will automatically output the query statement with execution time exceeding 100 ms to the log. Among them, 100ms can be set by mongod's startup option slow Ms. The default is 100ms.
2. It can also be opened by opening the monitoring function of the database, which is turned off by default. It can be opened by following commands

db.setProfilingLevel(level,[slowms])

Level: Monitor level, 0 closed, 1 slow log only, 2 slow log all operations
The results of the monitoring are stored in the system.profile.

Examples are as follows:

> db.setProfilingLevel(2)
{ "was" : 0, "slowms" : 100, "ok" : 1 }
> db.customers.find()
{ "_id" : ObjectId("589835c41c85cb68725f7899"), "id" : 1, "name" : "zhangsan", "age" : 11 }
{ "_id" : ObjectId("589835c41c85cb68725f789a"), "id" : 2, "name" : "zhangsan", "age" : 12 }
{ "_id" : ObjectId("589835c41c85cb68725f789b"), "id" : 3, "name" : "zhangsan", "age" : 13 }
{ "_id" : ObjectId("589835c41c85cb68725f789c"), "id" : 4, "name" : "zhangsan", "age" : 14 }
{ "_id" : ObjectId("589835c41c85cb68725f789d"), "id" : 5, "name" : "zhangsan", "age" : 15 }
{ "_id" : ObjectId("589835c41c85cb68725f789e"), "id" : 6, "name" : "zhangsan", "age" : 16 }
{ "_id" : ObjectId("589835c41c85cb68725f789f"), "id" : 7, "name" : "zhangsan", "age" : 17 }
{ "_id" : ObjectId("589835c41c85cb68725f78a0"), "id" : 8, "name" : "zhangsan", "age" : 18 }
{ "_id" : ObjectId("589835c41c85cb68725f78a1"), "id" : 9, "name" : "zhangsan", "age" : 19 }
{ "_id" : ObjectId("589835c41c85cb68725f78a2"), "id" : 10, "name" : "zhangsan", "age" : 20 }
{ "_id" : ObjectId("58983d0fc55e261327343eab"), "id" : 11, "name" : "lisi", "orders" : [ { "orders_id" : 1, "create_time" : "2017-02-06", "products" : [ { "product_name" : "MiPad", "price" : "$100.00" }, { "product_name" : "iphone", "price" : "$399.00" } ] } ], "mobile" : "13161020110", "address" : { "city" : "beijing", "street" : "taiyanggong" } }
> db.system.profile.find()
{ "op" : "query", "ns" : "test.system.indexes", "query" : { "expireAfterSeconds" : { "$exists" : true } }, "ntoreturn" : 0, "ntoskip" : 0, "nscanned" : 4, "keyUpdates" : 0, "numYield" : 0, "lockStats" : { "timeLockedMicros" : { "r" : NumberLong(294), "w" : NumberLong(0) }, "timeAcquiringMicros" : { "r" : NumberLong(13), "w" : NumberLong(15) } }, "nreturned" : 0, "responseLength" : 20, "millis" : 0, "ts" : ISODate("2017-02-09T03:45:41.845Z"), "client" : "0.0.0.0", "allUsers" : [ ], "user" : "" }
{ "op" : "query", "ns" : "test.customers", "query" : { }, "ntoreturn" : 0, "ntoskip" : 0, "nscanned" : 11, "keyUpdates" : 0, "numYield" : 0, "lockStats" : { "timeLockedMicros" : { "r" : NumberLong(252), "w" : NumberLong(0) }, "timeAcquiringMicros" : { "r" : NumberLong(19), "w" : NumberLong(16) } }, "nreturned" : 11, "responseLength" : 995, "millis" : 0, "ts" : ISODate("2017-02-09T03:45:49.972Z"), "client" : "127.0.0.1", "allUsers" : [ ], "user" : "" }
>

Posted by mishasoni on Mon, 25 Mar 2019 06:24:28 -0700