Instances of conversion operations for the core DStream of Spark Streaming

Conversion operation of DStream The DStream API provides the following methods related to transformation operations: Examples of transform(func) and updateStateByKey(func) methods are given below: (1), transform(func) method transform methods and similar transformWith(func) methods allow any RDD-to-RDD function to be applied on DStream and can ...

Posted by buildakicker on Sat, 23 May 2020 12:15:37 -0700

Swagger permission control part 2: Implementation Based on Spring Security

>Following the previous article "Apache Shiro takes over the authentication and authorization of Swagger", some enthusiastic netizens said that Apache Shiro seems too simple. In response to this problem, the individual does not make any evaluation (all technologies serve the needs). Today's main share is: how to take over the authe ...

Posted by Chrisinsd on Fri, 22 May 2020 00:13:44 -0700

ELK 7.X Use kiana's built-in analysis module to display icon problem records

Built a set of elk by myself 7.x Version environment found that using filebeat to collect nginx or apapche could not customize the directory log directory. If you customize the log directory, it would not be segmented. If you use the default log location, it would be segmented The solution is hereby recorded Default load path [root@kangcw e ...

Posted by phithe on Mon, 18 May 2020 09:03:04 -0700

Deploy multiple projects in Tomcat, and configure the process of secondary domain name through Nginx reverse agent

Purchase domain name. Example: example.com Set multiple secondary domain names. As shown in the figure:   To configure the tomcat file: Modify the server.xml File: The following configuration configures three containers with three different ports. Please note three points: ① port number: Connector port; ② container name: portservice ...

Posted by kenchucky on Sun, 17 May 2020 08:11:31 -0700

The application of Stream stream, a new feature of java8

As a qualified programmer, how to make the code more concise and clear, and improve the coding speed. Today, follow me to learn the application of java 8 stream. Don't talk much nonsense, go straight to the point. Considering the following business scenario, there are four personnel information, and we need to count the names of personnel accor ...

Posted by sajidfiaz on Sat, 16 May 2020 07:12:15 -0700

New features of JDK11: new HTTP API

brief introduction Before JDK11, the HTTP function of java was very weak, only HttpURLConnection was provided for HTTP connection, and it was very complex to use. So we usually use the third-party HTTP client (Apache HttpComponents or OkHttp) to make HTTP requests. Everything has changed in JDK11. In the java.net.http package, the latest HttpCl ...

Posted by PhilipXVIII18 on Wed, 13 May 2020 15:58:37 -0700

phoenix-5.0.0 and CDH6.0.1 compatibility cause secondary index unavailable

Today, when testing phoenix's secondary index function, the following exception occurred when the index was created to write data: Caused by: org.apache.hadoop.hbase.client.RetriesExhaustedWithDetailsException: Failed 1 action: org.apache.phoenix.hbase.index.builder.IndexBuildingFailureException: Failed to build index for unexpected reason! ...

Posted by superhaggis on Sat, 09 May 2020 07:38:49 -0700

RocketMq cluster construction steps teaching package teaching package meeting

mq cluster construction I haven't written these technical articles for a while. I'll spare some time to write them today, or I'll forget them This article records the process of building rocketmq cluster, and it's also a half day's achievement record! Interested friends like to go! Well, there's not much bullshit. Let's start. This article i ...

Posted by scoppc on Thu, 07 May 2020 00:07:47 -0700

Hive quick start series (12) | introduction and use of hive data compression

                   . Catalog 1, Hadoop source compilation supports Snappy compression 1.1 resource preparation 1.2 jar package installation 1.3 compiling source code 2, Hadoop compression configuration 2 ...

Posted by pavanpuligandla on Tue, 05 May 2020 23:29:55 -0700

Hbase API create table error record for Docker container deployment cluster

Hbase API create table error record Demo method: import org.apache.hadoop.conf.Configuration; import org.apache.hadoop.hbase.HBaseConfiguration; import org.apache.hadoop.hbase.HColumnDescriptor; import org.apache.hadoop.hbase.HTableDescriptor; import org.apache.hadoop.hbase.TableName; import org.apache.hadoop.hbase.client.Admin; import ...

Posted by spartan7 on Tue, 05 May 2020 16:37:36 -0700