HTTP request operation of ElasticSearch

Keywords: ElasticSearch search engine http

Mapping operation

The mapping in the index library is similar to the table structure in the database. To create a database table, you need to set the field name, type, length, constraint, etc; The same is true for the index library. You need to know what fields are under this type and what constraint information each field has. This is called mapping.

Create mapping

Field name: optional

Type: type. Elasticsearch supports a wide range of data types. There are several key points:

There are two types of String:

text: Separable word

keyword: Non separable words, the data will be matched as a complete field

Numerical: numerical type, divided into two categories

Basic data type: long,integer,short,byte,double,float,half_float

High precision type of floating point number: scaled_float

Date: date type

Array: array type

Object: object

Index: whether to index. The default value is true. All fields will be indexed

true: If the field is indexed, it can be used for search

false: Fields are not indexed and cannot be searched

Store: whether to store data independently. The default value is false

The original text is stored in the_ In source, by default, other extracted fields are not stored independently, but from_ Extracted from source. Of course, you can also store a field independently, as long as you set "store": true. It is easier to obtain the independently stored field than from_ Source parsing is much faster, but it also takes up more space, so it should be set according to the actual business needs.

analyzer: word splitter, ik here_ max_ Word uses ik participle

http://127.0.0.1:9200/user

Put http://127.0.0.1:9200/user/_mapping

{
	"properties": {
		"username": {
			"type": "text",
			"index": true
		},
		"sex": {
			"type": "keyword",
			"index": true
		},
		"age": {
			"type": "long",
			"index": false
		}
	}
}

view map

Get http://127.0.0.1:9200/user/_mapping

{
	"user": {
		"mappings": {
			"properties": {
				"age": {
					"type": "long",
					"index": false
				},
				"sex": {
					"type": "keyword"
				},
				"username": {
					"type": "text"
				}
			}
		}
	}
}

Index direct mapping Association

Put http://127.0.0.1:9200/user1

{
	"settings": {},
	"mappings": {
		"properties": {
			"username": {
				"type": "text",
				"index": true
			},
			"sex": {
				"type": "keyword",
				"index": true
			},
			"age": {
				"type": "long",
				"index": false
			}
		}
	}
}

{
	"acknowledged": true,
	"shards_acknowledged": true,
	"index": "user1"
}

Index operation

Create index

Send PUT request to ES server

http://127.0.0.1:9200/demo

Response:

{
	"acknowledged": true,
	"shards_acknowledged": true,
	"index": "demo"
}

View a single index

Send GET request to ES server

http://127.0.0.1:9200/demo

Response:

{
	"demo": {
		"aliases": {},
		"mappings": {},
		"settings": {
			"index": {
				"creation_date": "1632061699362",
				"number_of_shards": "1",
				"number_of_replicas": "1",
				"uuid": "PMskybxnT6WHoA2hWHFETw",
				"version": {
					"created": "7080099"
				},
				"provided_name": "demo"
			}
		}
	}
}

View all indexes

Send GET request to ES server

http://127.0.0.1:9200/_cat/indices?v

Response:

health status index uuid                   pri rep docs.count docs.deleted store.size pri.store.size
yellow open   demo  PMskybxnT6WHoA2hWHFETw   1   1          0            0       208b           208b

Header	meaning
health	Current server health status: green: cluster integrity yellow: the single point is normal and the cluster is incomplete red: single point abnormal
status	Index on / off status
index	Index name
uuid	Index unified number
pri	Number of main segments
rep	Number of copies
docs.count	Number of documents available
docs.deleted	Document deletion status (logical deletion)
store.size	Overall space occupied by main partition and sub partition
pri.store.size	Space occupied by main partition

Delete index

Send DELETE request to ES server

http://127.0.0.1:9200/demo

{
	"acknowledged": true
}

Document operation

create documents

Send POST request

http://127.0.0.1:9200/demo/_doc

Request data:

{
    "username":"Little white",
    "age":22,
    "address":"Shanghai China"
}

Response:

{
	"_index": "demo",
	"_type": "_doc",
	"_id": "wwKD_nsBW1VSDLfjekOE",
	"_version": 1,
	"result": "created",
	"_shards": {
		"total": 2,
		"successful": 1,
		"failed": 0
	},
	"_seq_no": 0,
	"_primary_term": 1
}

Because the data uniqueness ID (ID) is not specified, the ES server will randomly generate one by default.

The user-defined uniqueness ID needs to be specified when creating

http://127.0.0.1:9200/demo/_doc/1

http://127.0.0.1:9200/demo/_create/2

Response:

{
	"_index": "demo",
	"_type": "_doc",
	"_id": "1",
	"_version": 1,
	"result": "created",
	"_shards": {
		"total": 2,
		"successful": 1,
		"failed": 0
	},
	"_seq_no": 1,
	"_primary_term": 1
}

Note: to add data and specify the data primary key, you can use POST and PUT requests

View document

Send GET request:

http://127.0.0.1:9200/demo/_doc/1

Response:

{
	"_index": "demo",
	"_type": "_doc",
	"_id": "1",
	"_version": 1,
	"_seq_no": 1,
	"_primary_term": 1,
	"found": true,
	"_source": {
		"username": "Xiaobai 2",
		"age": 22,
		"address": "Shanghai, China 2"
	}
}

Modify document

POST request:

http://127.0.0.1:9200/demo/_doc/1

Request parameters:

{
    "username":"Modify document",
    "age":22,
    "address":"Shanghai, China 2"
}

Response:

{
	"_index": "demo",
	"_type": "_doc",
	"_id": "1",
	"_version": 2,
	"result": "updated",
	"_shards": {
		"total": 2,
		"successful": 1,
		"failed": 0
	},
	"_seq_no": 2,
	"_primary_term": 1
}

Modify field

When modifying data, only local information of the data is modified

POST request:

http://127.0.0.1:9200/demo/_update/1

Request data:

{ 
  "doc": {
    "username":"update field"
  } 
}

Response:

{
	"_index": "demo",
	"_type": "_doc",
	"_id": "1",
	"_version": 3,
	"result": "updated",
	"_shards": {
		"total": 2,
		"successful": 1,
		"failed": 0
	},
	"_seq_no": 3,
	"_primary_term": 2
}

remove document

Deleted documents are not immediately removed from disk, but are marked as deleted (logical deletion)

Delete according to the unique identification of the document

Delete multiple pieces of data according to conditions

Send DELETE request:

http://127.0.0.1:9200/demo/_doc/4

Response:

{
	"_index": "demo",
	"_type": "_doc",
	"_id": "4",
	"_version": 2,
	"result": "deleted",
	"_shards": {
		"total": 2,
		"successful": 1,
		"failed": 0
	},
	"_seq_no": 6,
	"_primary_term": 2
}

Send POST request:

http://127.0.0.1:9200/demo/_delete_by_query

Request data:

{
	"query": {
		"match": {
			"username": "update field"
		}
	}
}

Response:

{
	"took": 81,
	"timed_out": false,
	"total": 1,
	"deleted": 1,
	"batches": 1,
	"version_conflicts": 0,
	"noops": 0,
	"retries": {
		"bulk": 0,
		"search": 0
	},
	"throttled_millis": 0,
	"requests_per_second": -1,
	"throttled_until_millis": 0,
	"failures": []
}

Query all

Send Get request:

http://127.0.0.1:9200/demo/_search

Response:

{
	"took": 632,
	"timed_out": false,
	"_shards": {
		"total": 1,
		"successful": 1,
		"skipped": 0,
		"failed": 0
	},
	"hits": {
		"total": {
			"value": 4,
			"relation": "eq"
		},
		"max_score": 1,
		"hits": [
			{
				"_index": "demo",
				"_type": "_doc",
				"_id": "wwKD_nsBW1VSDLfjekOE",
				"_score": 1,
				"_source": {
					"username": "Little white",
					"age": 22,
					"address": "Shanghai China"
				}
			},
			{
				"_index": "demo",
				"_type": "_doc",
				"_id": "1",
				"_score": 1,
				"_source": {
					"username": "update field",
					"age": 22,
					"address": "Shanghai, China 2"
				}
			}
		]
	}
}

Advanced query

Query all

Query represents a query object, which can have different query properties

Get http://127.0.0.1:9200/demo/_search

Request parameters:

{
  "query": {
    "match_all": {}
  }
}

Field matching query

Get http://127.0.0.1:9200/demo/_search?q=age:22

Response:

{
	"took": 40,
	"timed_out": false,
	"_shards": {
		"total": 1,
		"successful": 1,
		"skipped": 0,
		"failed": 0
	},
	"hits": {
		"total": {
			"value": 3,
			"relation": "eq"
		},
		"max_score": 1,
		"hits": [
			{
				"_index": "demo",
				"_type": "_doc",
				"_id": "wwKD_nsBW1VSDLfjekOE",
				"_score": 1,
				"_source": {
					"username": "Little white",
					"age": 22,
					"address": "Shanghai China"
				}
			},
			{
				"_index": "demo",
				"_type": "_doc",
				"_id": "1",
				"_score": 1,
				"_source": {
					"username": "update field",
					"age": 22,
					"address": "Shanghai, China 2"
				}
			}
		]
	}
}

Get http://127.0.0.1:9200/demo/_search

Request parameters:

{
	"query": {
		"match": {
			"age": 22
		}
	}
}

Response:

{
	"took": 2,
	"timed_out": false,
	"_shards": {
		"total": 1,
		"successful": 1,
		"skipped": 0,
		"failed": 0
	},
	"hits": {
		"total": {
			"value": 3,
			"relation": "eq"
		},
		"max_score": 1,
		"hits": [
			{
				"_index": "demo",
				"_type": "_doc",
				"_id": "wwKD_nsBW1VSDLfjekOE",
				"_score": 1,
				"_source": {
					"username": "Little white",
					"age": 22,
					"address": "Shanghai China"
				}
			},
			{
				"_index": "demo",
				"_type": "_doc",
				"_id": "1",
				"_score": 1,
				"_source": {
					"username": "update field",
					"age": 22,
					"address": "Shanghai, China 2"
				}
			}
		]
	}
}

multi_match is similar to match, except that it can be queried in multiple fields

{
	"query": {
		"multi_match": {
			"query": "2",
			"fields": [
				"username",
				"address"
			]
		}
	}
}

{
	"took": 13,
	"timed_out": false,
	"_shards": {
		"total": 1,
		"successful": 1,
		"skipped": 0,
		"failed": 0
	},
	"hits": {
		"total": {
			"value": 2,
			"relation": "eq"
		},
		"max_score": 0.6682933,
		"hits": [
			{
				"_index": "demo",
				"_type": "_doc",
				"_id": "2",
				"_score": 0.6682933,
				"_source": {
					"username": "Xiaobai 2",
					"age": 22,
					"address": "Shanghai, China 2"
				}
			},
			{
				"_index": "demo",
				"_type": "_doc",
				"_id": "9",
				"_score": 0.6682933,
				"_source": {
					"username": "Xiaobai 2",
					"age": 25,
					"address": "Shanghai, China 2"
				}
			}
		]
	}
}

Exact matching query

term query, accurate keyword matching query, without word segmentation of query conditions

{
	"query": {
		"term": {
			"username": {
				"value": "Xiaobai 2"
			}
		}
	}
}

Multi keyword exact query

terms Query and term Like queries, multiple values can be specified for matching.

If this field contains any of the specified values, the document meets the conditions, similar to mysql of in

{
	"query": {
		"terms": {
			"username": [
				"Little white",
				"Xiaobai"
			]
		}
	}
}

Query specified fields

By default, ES will save the document in the search results_ All fields of source are returned. If you only want to get some of these fields, you can add_ Source filtering

Get http://127.0.0.1:9200/demo/_search

Request data:

{
	"query": {
		"match_all": {
			
		},
	},
		"_source":['username']
}

Response:

{
	"took": 3,
	"timed_out": false,
	"_shards": {
		"total": 1,
		"successful": 1,
		"skipped": 0,
		"failed": 0
	},
	"hits": {
		"total": {
			"value": 3,
			"relation": "eq"
		},
		"max_score": 1,
		"hits": [
			{
				"_index": "demo",
				"_type": "_doc",
				"_id": "wwKD_nsBW1VSDLfjekOE",
				"_score": 1,
				"_source": {
					"username": "Little white"
				}
			},
			{
				"_index": "demo",
				"_type": "_doc",
				"_id": "2",
				"_score": 1,
				"_source": {
					"username": "Xiaobai 2"
				}
			}
		]
	}
}

Field filtering

includes: To specify the fields you want to display

excludes: To specify the fields you do not want to display

Get http://127.0.0.1:9200/demo/_search

Request data:

{
  "_source": {
    "includes": ["age","username"]
  },  
  "query": {
    "match_all": {
			
		},
  }
}

Paging query

from: The starting index of the current page, starting from 0 by default. from = (pageNum - 1) * size

size: How many are displayed per page

Get http://127.0.0.1:9200/demo/_search

Request parameters:

{
	"query": {
		"match_all": {
			
		},
	},
		"from":0,
		"size":2
}

{
	"took": 1,
	"timed_out": false,
	"_shards": {
		"total": 1,
		"successful": 1,
		"skipped": 0,
		"failed": 0
	},
	"hits": {
		"total": {
			"value": 3,
			"relation": "eq"
		},
		"max_score": 1,
		"hits": [
			{
				"_index": "demo",
				"_type": "_doc",
				"_id": "wwKD_nsBW1VSDLfjekOE",
				"_score": 1,
				"_source": {
					"username": "Little white",
					"age": 22,
					"address": "Shanghai China"
				}
			},
			{
				"_index": "demo",
				"_type": "_doc",
				"_id": "1",
				"_score": 1,
				"_source": {
					"username": "update field",
					"age": 22,
					"address": "Shanghai, China 2"
				}
			}
		]
	}
}

Single field sorting

adopt order Specifies how to sort. desc In descending order, asc In ascending order.

Get http://127.0.0.1:9200/demo/_search

Request data:

{
	"query": {
		"match_all": {
			
		},
	},
		"from":0,
		"size":2,
		"_source":['username'],
		"sort":{
		    "age":{
		        "order":"desc"
		    }
		}
}

Response:

{
	"took": 469,
	"timed_out": false,
	"_shards": {
		"total": 1,
		"successful": 1,
		"skipped": 0,
		"failed": 0
	},
	"hits": {
		"total": {
			"value": 4,
			"relation": "eq"
		},
		"max_score": null,
		"hits": [
			{
				"_index": "demo",
				"_type": "_doc",
				"_id": "9",
				"_score": null,
				"_source": {
					"username": "Xiaobai 2"
				},
				"sort": [
					25
				]
			},
			{
				"_index": "demo",
				"_type": "_doc",
				"_id": "wwKD_nsBW1VSDLfjekOE",
				"_score": null,
				"_source": {
					"username": "Little white"
				},
				"sort": [
					22
				]
			}
		]
	}
}

Multi field sorting

Get http://127.0.0.1:9200/demo/_search

Request data:

{
	"query": {
		"match_all": {
			
		},
	},
		"from":0,
		"size":2,
		"_source":['username'],
		"sort":[
		    {
		    "age":{
		        "order":"desc"
		    }
		},
		{
      "_score":{
        "order": "desc"
      }
    }
		    ]
}

Response:

{
	"took": 4,
	"timed_out": false,
	"_shards": {
		"total": 1,
		"successful": 1,
		"skipped": 0,
		"failed": 0
	},
	"hits": {
		"total": {
			"value": 3,
			"relation": "eq"
		},
		"max_score": null,
		"hits": [
			{
				"_index": "demo",
				"_type": "_doc",
				"_id": "9",
				"_score": 1,
				"_source": {
					"username": "Xiaobai 2"
				},
				"sort": [
					25,
					1
				]
			},
			{
				"_index": "demo",
				"_type": "_doc",
				"_id": "wwKD_nsBW1VSDLfjekOE",
				"_score": 1,
				"_source": {
					"username": "Little white"
				},
				"sort": [
					22,
					1
				]
			}
		]
	}
}

Fuzzy query

Returns a document containing words similar to the search term.

Edit distance is the number of character changes required to convert one term to another. These changes can include:

Change character( box → fox)

Delete character( black → lack)

Insert character( sic → sick)

Transpose two adjacent characters( act → cat)

To find similar terms, a fuzzy query creates all possible variants or extensions of a set of search terms within a specified editing distance. Then the query returns an exact match for each extension.

Modify the editing distance through fuzziness. Generally, the default value AUTO is used to generate the editing distance according to the length of the term.

Get http://127.0.0.1:9200/demo/_search

{
  "query": {
    "fuzzy": {
      "username": {
        "value": "white",
        "fuzziness": 2
      }
    }
  }
}

Combined query

bool combines various other queries through must, must_not and should `

Get http://127.0.0.1:9200/demo/_search

Request data:

{
	"query": {
		"bool": {
			"must": [
				{
					"match": {
						"age": 22
					}
				},
				{
					"match": {
						"username": "Xiaobai"
					}
				}
			]
		}
	}
}

Response data:

{
	"took": 5,
	"timed_out": false,
	"_shards": {
		"total": 1,
		"successful": 1,
		"skipped": 0,
		"failed": 0
	},
	"hits": {
		"total": {
			"value": 2,
			"relation": "eq"
		},
		"max_score": 1.8220872,
		"hits": [
			{
				"_index": "demo",
				"_type": "_doc",
				"_id": "wwKD_nsBW1VSDLfjekOE",
				"_score": 1.8220872,
				"_source": {
					"username": "Little white",
					"age": 22,
					"address": "Shanghai China"
				}
			},
			{
				"_index": "demo",
				"_type": "_doc",
				"_id": "2",
				"_score": 1.6877716,
				"_source": {
					"username": "Xiaobai 2",
					"age": 22,
					"address": "Shanghai, China 2"
				}
			}
		]
	}
}

Or query

{
	"query": {
		"bool": {
			"should": [
				{
					"match": {
						"age": 22
					}
				},
				{
					"match": {
						"username": "123"
					}
				}
			]
		}
	}
}

Range query

The range query finds out the numbers or times that fall within the specified range, and supports the following characters

Operator	explain	Symbol
gt	greater than	>
gte	Greater than or equal to	>=
lt	less than	<
lte	Less than or equal to	<=

Get http://127.0.0.1:9200/demo/_search

Request data:

{
	"query": {
		"bool": {
			"should": [
				{
					"match": {
						"age": 22
					}
				},
				{
					"match": {
						"username": "123"
					}
				}
			],
			"filter":{
			    "range":{
			        "age":{
			            "gt":23
			        }
			    }
			}
		}
	}
}

Response:

{
	"took": 5,
	"timed_out": false,
	"_shards": {
		"total": 1,
		"successful": 1,
		"skipped": 0,
		"failed": 0
	},
	"hits": {
		"total": {
			"value": 1,
			"relation": "eq"
		},
		"max_score": 0,
		"hits": [
			{
				"_index": "demo",
				"_type": "_doc",
				"_id": "9",
				"_score": 0,
				"_source": {
					"username": "Xiaobai 2",
					"age": 25,
					"address": "Shanghai, China 2"
				}
			}
		]
	}
}

Exact match query

Get   http://127.0.0.1:9200/demo/_search

Request data:

{
   "query":{
       "match_phrase":{
           "username":"Little white"
       }
   }
}

Response:

{
	"took": 20,
	"timed_out": false,
	"_shards": {
		"total": 1,
		"successful": 1,
		"skipped": 0,
		"failed": 0
	},
	"hits": {
		"total": {
			"value": 1,
			"relation": "eq"
		},
		"max_score": 1.0316575,
		"hits": [
			{
				"_index": "demo",
				"_type": "_doc",
				"_id": "wwKD_nsBW1VSDLfjekOE",
				"_score": 1.0316575,
				"_source": {
					"username": "Little white",
					"age": 22,
					"address": "Shanghai China"
				}
			}
		]
	}
}

Highlight query

Get  http://127.0.0.1:9200/demo/_search

{
	"query": {
		"match_phrase": {
			"username": "Little white"
		}
	},
	"highlight": {
		"pre_tags": "<font color='red'>",
		"post_tags": "</font>",
		"fields": {
			"username": {}
		}
	}
}

Response:

{
	"took": 5,
	"timed_out": false,
	"_shards": {
		"total": 1,
		"successful": 1,
		"skipped": 0,
		"failed": 0
	},
	"hits": {
		"total": {
			"value": 1,
			"relation": "eq"
		},
		"max_score": 1.0316575,
		"hits": [
			{
				"_index": "demo",
				"_type": "_doc",
				"_id": "wwKD_nsBW1VSDLfjekOE",
				"_score": 1.0316575,
				"_source": {
					"username": "Little white",
					"age": 22,
					"address": "Shanghai China"
				},
				"highlight": {
					"username": [
						"<font color='red'>Small</font><font color='red'>white</font><font color='red'>white</font>"
					]
				}
			}
		]
	}
}

Aggregate query

Get http://127.0.0.1:9200/demo/_search

max for a field

{
    "aggs":{
      "max_age":{
        "max":{"field":"age"}
      }
    },
    "size":0
}

Maximum value min for a field

{
    "aggs":{
      "min_age":{
        "min":{"field":"age"}
      }
    },
    "size":0
}

Sum a field sum

{
    "aggs":{
      "sum_age":{
        "sum":{"field":"age"}
      }
    },
    "size":0
}

Average a field avg

{
    "aggs":{
      "avg_age":{
        "avg":{"field":"age"}
      }
    },
    "size":0
}

The total number is obtained after de duplication of the value of a field

{
    "aggs":{
      "distinct_age":{
        "cardinality":{"field":"age"}
      }
    },
    "size":0
}

Bucket aggregation query

Bucket aggregation and are equivalent to group by statements, terms aggregation and group statistics in sql

Get http://127.0.0.1:9200/demo/_search

Request data:

{
	"aggs": {
		"group_age": {
			"terms": {
				"field": "age"
			}
		}
	},
	"size": 0
}

Response:

{
	"took": 165,
	"timed_out": false,
	"_shards": {
		"total": 1,
		"successful": 1,
		"skipped": 0,
		"failed": 0
	},
	"hits": {
		"total": {
			"value": 3,
			"relation": "eq"
		},
		"max_score": null,
		"hits": []
	},
	"aggregations": {
		"group_age": {
			"doc_count_error_upper_bound": 0,
			"sum_other_doc_count": 0,
			"buckets": [
				{
					"key": 22,
					"doc_count": 2
				},
				{
					"key": 25,
					"doc_count": 1
				}
			]
		}
	}
}

Stats aggregation

stats aggregation returns count, max, min, avg and sum for a field at one time

Get http://127.0.0.1:9200/demo/_search

Request data:

{
	"aggs": {
		"stats_age": {
			"stats": {
				"field": "age"
			}
		}
	},
	"size": 0
}

Response:

{
	"took": 8,
	"timed_out": false,
	"_shards": {
		"total": 1,
		"successful": 1,
		"skipped": 0,
		"failed": 0
	},
	"hits": {
		"total": {
			"value": 3,
			"relation": "eq"
		},
		"max_score": null,
		"hits": []
	},
	"aggregations": {
		"stats_age": {
			"count": 3,
			"min": 22,
			"max": 25,
			"avg": 23,
			"sum": 69
		}
	}
}

Posted by Gordonator on Thu, 02 Dec 2021 19:00:58 -0800

Programmer Group

HTTP request operation of ElasticSearch

Mapping operation

Create mapping

view map

Index direct mapping Association

Index operation

Create index

View a single index

View all indexes

Delete index

Document operation

create documents

View document

Modify document

Modify field

remove document

Query all

Advanced query

Query all

Field matching query

Exact matching query

Multi keyword exact query

Query specified fields

Field filtering

Paging query

Single field sorting

Multi field sorting

Fuzzy query

Combined query

Range query

Exact match query

Highlight query

Aggregate query

Bucket aggregation query

Stats aggregation

Hot Keywords