Elastic search 5.2.2 plug-in development (1)

Keywords: ElasticSearch Apache Maven log4j

Address of this article http://blog.csdn.net/makefriend7/article/details/60323717


First put the official address https://www.elastic.co/guide/en/elasticsearch/plugins/5.2/index.html


1. Introduction to Plug-ins

Plug-in, as you know by name, is to insert something on ES to enhance ES's ability.

Here's a brief introduction to the classification of plug-ins

  • API extension plugin (Action Plugin): This is the API function that extends ES. Most are used for search and mapping, such as enabling ES to support SQL statements
  • Alerting plugins: Monitor the index and automatically trigger an alarm when the threshold is exceeded (on behalf of plug-in X-PACK, which periodically calls after you have set some query condition, and if it meets the condition, does what is specified).
  • Analysis plugins: An analysis plug-in, in short, is a plug-in that formulates indexing rules, such as SmartCN, which is a Chinese word segmentation plug-in that builds indexing based on Chinese (rather than English blanks) and indexing by sentence and word.
  • Discovery plugins: Discovery plugins, in short. It's how the cluster finds its own servers. Officially, it's how to elect a major node in a cluster.
  • The ingest plugins: The main function of this plug-in is to enhance the functionality of each node. For example, Ingest Attcahment Processor Plugin allows each node to decompress files and process file formats such as PPT, XLD, PDF.
  • Management plugins: Management plugins, of course, interact with and manage ES (such as X-PACK)
  • Mapper plugins: This plug-in mainly enhances the data type of ES. For example, add an attachment type, which can store PDF or WORD data.
  • Scripting plugins: This plug-in essentially calls the user's script, so you can execute any program, for example, through this plug-in, support javascript language, python language, or any user-defined language or program.
  • Security plugins: Provide security controls, of course, such as those API s that control which users can use ES
  • Repository plugins: Provide snapshots and recovery, a simple understanding is that you put data on the server, others can access your data through shared folders, you can also restore your data through shared folders. At present, the core plug-ins of ES already support many access modes such as S3,HDFS, etc.
  • Store plugins, we know that ES actually uses Lucene for storage. We can also use Store SMB.(windows Shared File Protocol)
Second, additional tools, which are used to help other systems use ES.

1. Content Management System

  • Drupal: Drupal is an open source content management framework (CMF) written in PHP language.
  • Wp-Elastic search: ES's WordPress plug-in, WordPress, can use ES directly
  • And Elastic search, Tiki Wiki Cms Groupware, XWIKI Next Generation Wiki
2. Data import, export and verification
  • LogStash output to ES and ES input to LogStash
  • ES event filtering in Logstash
  • ES bulk codec
  • JDBC importer: Importing jdbc's source data into ES
  • Kafka Standalone consumer(Indexer): Importing kafka data into ES
  • Mongolastic: Importing ES data into MongoDB
  • Scrutineer: A Check Tool for Index and Content
  • IMAP/POP3/MAIL importer: Import IMAP POP3 data into ES (mailbox data can also enter ES)
  • FS Crawler: Index file systems (e.g. PDF, OPEN OFFICE.) Local, or through SSH
3. deployment
ES Provides Puppet Community Provides Chef

3. Framework integration

4.Hadoop integrations

4. Other Integration
pes: DSL queries supporting JS https://github.com/kodcu/pes

Second, the first PLUGIN program
Plug-ins are very simple. The code is as follows

  1. public class MyFirstPlugin extends Plugin{  
  2.     private final static Logger LOGGER = LogManager.getLogger(MyFirstPlugin.class);  
  3.     public MyFirstPlugin() {  
  4.         super();  
  5.         LOGGER.warn("This is my fisrt Plugin");  
  6.     }  
  7. }  


Add pomx.ml

  1. <?xml version="1.0" encoding="UTF-8"?>  
  2. <project xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"  
  3.          xmlns="http://maven.apache.org/POM/4.0.0"  
  4.          xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">  
  5.     <modelVersion>4.0.0</modelVersion>  
  6.   
  7.     <groupId>mygroup1</groupId>  
  8.     <artifactId>myart1</artifactId>  
  9.     <version>5.2.2-SNAPSHOT</version>  
  10.     <name>Plugin: Basic</name>  
  11.     <description>Only for test</description>  
  12.   
  13.   
  14.     <properties>  
  15.         <elasticsearch.version>5.2.2</elasticsearch.version>  
  16.         <lucene.version>6.4.1</lucene.version>  
  17.     </properties>  
  18.     <dependencies>  
  19.         <dependency>  
  20.             <groupId>org.elasticsearch</groupId>  
  21.             <artifactId>elasticsearch</artifactId>  
  22.             <version>${elasticsearch.version}</version>  
  23.             <scope>provided</scope>  
  24.         </dependency>  
  25.   
  26.         <!-- Testing -->  
  27.         <dependency>  
  28.             <groupId>org.apache.logging.log4j</groupId>  
  29.             <artifactId>log4j-api</artifactId>  
  30.             <version>2.7</version>  
  31.             <scope>provided</scope>  
  32.         </dependency>  
  33.         <dependency>  
  34.             <groupId>org.apache.logging.log4j</groupId>  
  35.             <artifactId>log4j-core</artifactId>  
  36.             <version>2.7</version>  
  37.             <scope>test</scope>  
  38.         </dependency>  
  39.         <dependency>  
  40.             <groupId>org.elasticsearch.test</groupId>  
  41.             <artifactId>framework</artifactId>  
  42.             <version>${elasticsearch.version}</version>  
  43.             <scope>test</scope>  
  44.         </dependency>  
  45.   
  46.         <dependency>  
  47.             <groupId>org.apache.lucene</groupId>  
  48.             <artifactId>lucene-test-framework</artifactId>  
  49.             <version>${lucene.version}</version>  
  50.             <scope>test</scope>  
  51.         </dependency>  
  52.     </dependencies>  
  53.   
  54.     <build>  
  55.         <resources>  
  56.             <resource>  
  57.                 <directory>src/main/resources</directory>  
  58.                 <filtering>false</filtering>  
  59.                 <excludes>  
  60.                     <exclude>*.properties</exclude>  
  61.                 </excludes>  
  62.             </resource>  
  63.         </resources>  
  64.         <plugins>  
  65.             <plugin>  
  66.                 <groupId>org.apache.maven.plugins</groupId>  
  67.                 <artifactId>maven-assembly-plugin</artifactId>  
  68.                 <version>2.6</version>  
  69.                 <configuration>  
  70.                     <appendAssemblyId>false</appendAssemblyId>  
  71.                     <outputDirectory>${project.build.directory}/releases/</outputDirectory>  
  72.                     <descriptors>  
  73.                         <descriptor>${basedir}/src/main/assemblies/plugin.xml</descriptor>  
  74.                     </descriptors>  
  75.                 </configuration>  
  76.                 <executions>  
  77.                     <execution>  
  78.                         <phase>package</phase>  
  79.                         <goals>  
  80.                             <goal>single</goal>  
  81.                         </goals>  
  82.                     </execution>  
  83.                 </executions>  
  84.             </plugin>  
  85.             <plugin>  
  86.                 <groupId>org.apache.maven.plugins</groupId>  
  87.                 <artifactId>maven-compiler-plugin</artifactId>  
  88.                 <version>3.3</version>  
  89.                 <configuration>  
  90.                     <source>1.8</source>  
  91.                     <target>1.8</target>  
  92.                 </configuration>  
  93.             </plugin>  
  94.         </plugins>  
  95.     </build>  
  96. </project>  


Run the command mvn clean install
Get myart1-5.2.2-SNAPSHOT.zip file in release directory
Running in the ES directory
bin\elasticsearch-plugin install file:///D:/greesoft/elastic/myart1-5.2.2-SNAPSHOT.zip
The plug-in is installed.

There is a small pit here. Under windows. The directories installed in java usually have spaces. Here is a simple modification to the elastic search-plugin.bat file as follows

Set JAVA = D: jdk1.8.0_102 bin java.exe


Run ES. You can see the information we typed out.


Add pom.xml

  1. <?xml version="1.0" encoding="UTF-8"?>  
  2. <project xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"  
  3.          xmlns="http://maven.apache.org/POM/4.0.0"  
  4.          xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">  
  5.     <modelVersion>4.0.0</modelVersion>  
  6.   
  7.     <groupId>mygroup1</groupId>  
  8.     <artifactId>myart1</artifactId>  
  9.     <version>5.2.2-SNAPSHOT</version>  
  10.     <name>Plugin: Basic</name>  
  11.     <description>Only for test</description>  
  12.   
  13.   
  14.     <properties>  
  15.         <elasticsearch.version>5.2.2</elasticsearch.version>  
  16.         <lucene.version>6.4.1</lucene.version>  
  17.     </properties>  
  18.     <dependencies>  
  19.         <dependency>  
  20.             <groupId>org.elasticsearch</groupId>  
  21.             <artifactId>elasticsearch</artifactId>  
  22.             <version>${elasticsearch.version}</version>  
  23.             <scope>provided</scope>  
  24.         </dependency>  
  25.   
  26.         <!-- Testing -->  
  27.         <dependency>  
  28.             <groupId>org.apache.logging.log4j</groupId>  
  29.             <artifactId>log4j-api</artifactId>  
  30.             <version>2.7</version>  
  31.             <scope>provided</scope>  
  32.         </dependency>  
  33.         <dependency>  
  34.             <groupId>org.apache.logging.log4j</groupId>  
  35.             <artifactId>log4j-core</artifactId>  
  36.             <version>2.7</version>  
  37.             <scope>test</scope>  
  38.         </dependency>  
  39.         <dependency>  
  40.             <groupId>org.elasticsearch.test</groupId>  
  41.             <artifactId>framework</artifactId>  
  42.             <version>${elasticsearch.version}</version>  
  43.             <scope>test</scope>  
  44.         </dependency>  
  45.   
  46.         <dependency>  
  47.             <groupId>org.apache.lucene</groupId>  
  48.             <artifactId>lucene-test-framework</artifactId>  
  49.             <version>${lucene.version}</version>  
  50.             <scope>test</scope>  
  51.         </dependency>  
  52.     </dependencies>  
  53.   
  54.     <build>  
  55.         <resources>  
  56.             <resource>  
  57.                 <directory>src/main/resources</directory>  
  58.                 <filtering>false</filtering>  
  59.                 <excludes>  
  60.                     <exclude>*.properties</exclude>  
  61.                 </excludes>  
  62.             </resource>  
  63.         </resources>  
  64.         <plugins>  
  65.             <plugin>  
  66.                 <groupId>org.apache.maven.plugins</groupId>  
  67.                 <artifactId>maven-assembly-plugin</artifactId>  
  68.                 <version>2.6</version>  
  69.                 <configuration>  
  70.                     <appendAssemblyId>false</appendAssemblyId>  
  71.                     <outputDirectory>${project.build.directory}/releases/</outputDirectory>  
  72.                     <descriptors>  
  73.                         <descriptor>${basedir}/src/main/assemblies/plugin.xml</descriptor>  
  74.                     </descriptors>  
  75.                 </configuration>  
  76.                 <executions>  
  77.                     <execution>  
  78.                         <phase>package</phase>  
  79.                         <goals>  
  80.                             <goal>single</goal>  
  81.                         </goals>  
  82.                     </execution>  
  83.                 </executions>  
  84.             </plugin>  
  85.             <plugin>  
  86.                 <groupId>org.apache.maven.plugins</groupId>  
  87.                 <artifactId>maven-compiler-plugin</artifactId>  
  88.                 <version>3.3</version>  
  89.                 <configuration>  
  90.                     <source>1.8</source>  
  91.                     <target>1.8</target>  
  92.                 </configuration>  
  93.             </plugin>  
  94.         </plugins>  
  95.     </build>  
  96. </project>  


Posted by sweetmaster on Sat, 11 May 2019 13:26:08 -0700

Hot Keywords

©2024 Programmer Group