- document data format
- Background introduction of commodity management cases in e-commerce websites
- Simple Cluster Management
- CRUD operations for commodities (document curd)
1. Document data format
Document-Oriented Search and Analysis Engine
- The data structure of application system is object-oriented and complex.
- Object data stored in the database, can only be disassembled into flat tables, each time the query is still restored to the object format, quite troublesome.
- ES is document-oriented. The data structure stored in the document is the same as object-oriented data structure. It is based on this kind of document data structure. ES can provide complex indexing, full-text retrieval, analysis and aggregation functions.
- The document of ES is expressed in json data format.
public class Employee { private String email; private String firstName; private String lastName; private EmployeeInfo info; private Date joinDate; } private class EmployeeInfo { private String bio; private Integer age; private String[] interesets; } EmployeeInfo info = new EmployeeInfo(); info.SetBio("youdi"); info.setAge(24); info.setInteresets(new String[]{"dance", "code"}); Employee employee = new Employee(); employee.setEmail("@"); employee.setInfo(info); employee Object: It contains Employee Class's own properties. Two tables need to be saved. //Using ORM. database is flat and can not reflect the object-oriented structure.
2. Background Introduction of Commodity Management Case of E-commerce Website
The following functions are provided:
- CRUD of Commodity Information
- Perform simple full-text retrieval and complex phrase retrieval
- For full-text retrieval results, highlighting can be done
- Simple aggregation analysis of data
3. Simple Cluster Management
-
Quick check of cluster health
ES provides a set of cat APIs to view a wide variety of data in ES
GET /_cat/health?pretty epoch timestamp cluster status node.total node.data shards pri relo init unassign pending_tasks max_task_wait_time active_shards_percent 1524842572 23:22:52 elasticsearch_youdi yellow 1 1 10 10 0 0 10 0 - 50.0%View status
Green: The primary share and replica share of each index are active states
yellow: The primary share of each index is active, but part of replica share is not active and is unavailable.
red: Not all primary shares of indexes are active, and some of the miniatures are missing data
Why is it yellow ing now?
primary shard replica shard has no second node
- Quickly view all indexes in the cluster?
GET /_cat/indices?v health status index uuid pri rep docs.count docs.deleted store.size pri.store.size yellow open website -NVtisruRUO1buiENwh7Vw 5 1 1 0 5.5kb 5.5kb yellow open megacorp 3YlG1FRPTDynSETmZBDIhg 5 1 3 0 17.5kb 17.5kb
- Simple index operation
PUT /test_index?pretty DELETE /test_index?pretty
CRUD of Commodities
- New products
PUT /index/type/id PUT /products/goods/1 { "name": "test", "desc": "hello world", "price": 11.0, "producer": "hello", "tags": ["youdi", "haha"] } { "_index": "products", "_type": "goods", "_id": "1", "_version": 1, "result": "created", "_shards": { "total": 2, "successful": 1, "failed": 0 }, "_seq_no": 0, "_primary_term": 1 }
- Enquiry goods
GET /products/goods/1 { "_index": "products", "_type": "goods", "_id": "1", "_version": 1, "found": true, "_source": { "name": "test", "desc": "hello world", "price": 11, "producer": "hello", "tags": [ "youdi", "haha" ] } }
- update operation
Replacement: All information must be submitted PUT /products/goods/1 { "name": "test3", "desc": "hello world", "price": 11.5, "producer": "hello", "tags": ["youdi", "haha"] } { "_index": "products", "_type": "goods", "_id": "1", "_version": 2, "result": "updated", "_shards": { "total": 2, "successful": 1, "failed": 0 }, "_seq_no": 1, "_primary_term": 1 } PUT /products/goods/1 { "name": "test3" } GET /products/goods/1 { "_index": "products", "_type": "goods", "_id": "1", "_version": 3, "found": true, "_source": { "name": "test3" } } POST products/goods/1/_update { "doc": { "name": "test6" } } { "_index": "products", "_type": "goods", "_id": "1", "_version": 5, "result": "updated", "_shards": { "total": 2, "successful": 1, "failed": 0 }, "_seq_no": 4, "_primary_term": 1 }
- remove document
DELETE /products/goods/1 { "_index": "products", "_type": "goods", "_id": "1", "_version": 6, "result": "deleted", "_shards": { "total": 2, "successful": 1, "failed": 0 }, "_seq_no": 5, "_primary_term": 1 } { "_index": "products", "_type": "goods", "_id": "1", "found": false }
The Way of Searching for Goods
- query string search
- Query DSL
- query filter
- Full-text search
1. query string search
Search for all goods
GET /products/goods/_search { "took": 104, //time consuming "timed_out": false, //Is time out? "_shards": { //_shards "total": 5, "successful": 5, "skipped": 0, "failed": 0 }, "hits": { "total": 1, // Number of query results 1 data "max_score": 1, //Matching Score of ES for Relevance Degree "hits": [ //Contains detailed search data { "_index": "products", "_type": "goods", "_id": "2", "_score": 1, "_source": { "name": "tes2", "desc": "hello world", "price": 14, "producer": "hello", "tags": [ "youdi", "haha" ] } } ] } } { "took": 8, "timed_out": false, "_shards": { "total": 5, "successful": 5, "skipped": 0, "failed": 0 }, "hits": { "total": 2, "max_score": 1, "hits": [ { "_index": "products", "_type": "goods", "_id": "2", "_score": 1, "_source": { "name": "tes2", "desc": "hello world", "price": 14, "producer": "hello", "tags": [ "youdi", "haha" ] } }, { "_index": "products", "_type": "goods", "_id": "1", "_score": 1, "_source": { "name": "test3", "desc": "hello world", "price": 11.5, "producer": "hello", "tags": [ "youdi", "haha" ] } } ] } } { "_index": "bank", "_type": "_doc", "_id": "25", "_score": 1, "_source": { "account_number": 25, "balance": 40540, "firstname": "Virginia", "lastname": "Ayala", "age": 39, "gender": "F", "address": "171 Putnam Avenue", "employer": "Filodyne", "email": "virginiaayala@filodyne.com", "city": "Nicholson", "state": "PA" } },
- Search account names with ber, sorted by age
GET /bank/_doc/_search?q=firstname:Virginia&sort=age:des { "took": 1, "timed_out": false, "_shards": { "total": 5, "successful": 5, "skipped": 0, "failed": 0 }, "hits": { "total": 1, "max_score": 4.882802, "hits": [ { "_index": "bank", "_type": "_doc", "_id": "25", "_score": 4.882802, "_source": { "account_number": 25, "balance": 40540, "firstname": "Virginia", "lastname": "Ayala", "age": 39, "gender": "F", "address": "171 Putnam Avenue", "employer": "Filodyne", "email": "virginiaayala@filodyne.com", "city": "Nicholson", "state": "PA" } } ] } }
- query DSL
DSL: Domain Specific Language Domain Specific Language Domain Specific Language Domain Specific Language Domain Specific Language
Query all account s
http request body: request body, can use json format to build query grammar, more convenient, can build a variety of complex grammar.
GET /bank/_doc/_search { "query": { "match_all": {} } } { "took": 1, "timed_out": false, "_shards": { "total": 5, "successful": 5, "skipped": 0, "failed": 0 }, "hits": { "total": 1000, "max_score": 1, "hits": [ { "_index": "bank", "_type": "_doc", "_id": "25", "_score": 1, "_source": { "account_number": 25, "balance": 40540, "firstname": "Virginia", "lastname": "Ayala", "age": 39, "gender": "F", "address": "171 Putnam Avenue", "employer": "Filodyne", "email": "virginiaayala@filodyne.com", "city": "Nicholson", "state": "PA" } }, { "_index": "bank", "_type": "_doc", "_id": "44", "_score": 1, "_source": { "account_number": 44, "balance": 34487, "firstname": "Aurelia", "lastname": "Harding", "age": 37, "gender": "M", "address": "502 Baycliff Terrace", "employer": "Orbalix", "email": "aureliaharding@orbalix.com", "city": "Yardville", "state": "DE" } }, { "_index": "bank", "_type": "_doc", "_id": "99", "_score": 1, "_source": { "account_number": 99, "balance": 47159, "firstname": "Ratliff", "lastname": "Heath", "age": 39, "gender": "F", "address": "806 Rockwell Place", "employer": "Zappix", "email": "ratliffheath@zappix.com", "city": "Shaft", "state": "ND" } }, { "_index": "bank", "_type": "_doc", "_id": "119", "_score": 1, "_source": { "account_number": 119, "balance": 49222, "firstname": "Laverne", "lastname": "Johnson", "age": 28, "gender": "F", "address": "302 Howard Place", "employer": "Senmei", "email": "lavernejohnson@senmei.com", "city": "Herlong", "state": "DC" } }, { "_index": "bank", "_type": "_doc", "_id": "126", "_score": 1, "_source": { "account_number": 126, "balance": 3607, "firstname": "Effie", "lastname": "Gates", "age": 39, "gender": "F", "address": "620 National Drive", "employer": "Digitalus", "email": "effiegates@digitalus.com", "city": "Blodgett", "state": "MD" } }, { "_index": "bank", "_type": "_doc", "_id": "145", "_score": 1, "_source": { "account_number": 145, "balance": 47406, "firstname": "Rowena", "lastname": "Wilkinson", "age": 32, "gender": "M", "address": "891 Elton Street", "employer": "Asimiline", "email": "rowenawilkinson@asimiline.com", "city": "Ripley", "state": "NH" } }, { "_index": "bank", "_type": "_doc", "_id": "183", "_score": 1, "_source": { "account_number": 183, "balance": 14223, "firstname": "Hudson", "lastname": "English", "age": 26, "gender": "F", "address": "823 Herkimer Place", "employer": "Xinware", "email": "hudsonenglish@xinware.com", "city": "Robbins", "state": "ND" } }, { "_index": "bank", "_type": "_doc", "_id": "190", "_score": 1, "_source": { "account_number": 190, "balance": 3150, "firstname": "Blake", "lastname": "Davidson", "age": 30, "gender": "F", "address": "636 Diamond Street", "employer": "Quantasis", "email": "blakedavidson@quantasis.com", "city": "Crumpler", "state": "KY" } }, { "_index": "bank", "_type": "_doc", "_id": "208", "_score": 1, "_source": { "account_number": 208, "balance": 40760, "firstname": "Garcia", "lastname": "Hess", "age": 26, "gender": "F", "address": "810 Nostrand Avenue", "employer": "Quiltigen", "email": "garciahess@quiltigen.com", "city": "Brooktrails", "state": "GA" } }, { "_index": "bank", "_type": "_doc", "_id": "222", "_score": 1, "_source": { "account_number": 222, "balance": 14764, "firstname": "Rachelle", "lastname": "Rice", "age": 36, "gender": "M", "address": "333 Narrows Avenue", "employer": "Enaut", "email": "rachellerice@enaut.com", "city": "Wright", "state": "AZ" } } ] } }