Hadoop: Flink on Yan service configuration and settings
Previously, I planned to directly install and configure the flink service on ambari to facilitate management, but I found that the flink integrated by ambari would have many problems, which would be inconvenient to manage (maybe I didn't find the correct method), so I planned to configure the service separately
Download two files
Flink-1.10.1 ...
Posted by sangamon on Tue, 26 Oct 2021 18:36:19 -0700
Submit the sql script file with Flink sql client
Title: submit sql script file with flynk sql client Date: twenty 21-10-22 22:11:34 Tag: [Flink, SQL client] Category: flink
We know that sql-client.sh can provide us with an SQL interactive interface, so that we can see the execution results or interactive query table results without executing an SQL.
In fact, we can also submit sql scripts t ...
Posted by Drannon on Fri, 22 Oct 2021 07:39:30 -0700
Flink source code note 02 - generating StreamGraph from DataStream
StreanGraph
It is a DAG (stored in the form of adjacency table), which stores the topology information of the whole flow and is composed of a series of StreamEdge and StreamNode.
Call StreamExecutionEnvironment.getStreamGraph() to generate a StreamGraph. StreamExecutionEnvironment.getStreamGraph() clears the transformations saved in StreamExe ...
Posted by dr_overload on Sun, 10 Oct 2021 20:32:39 -0700
Restrictions on Flink DataStream Java Lambda expressions
Original text: Restrictions on Flink DataStream Java Lambda expressions
Java 8 introduces some new language features aimed at faster and clearer coding. With the most important feature 'lambda expression', it opens the door to functional programming. Lambda expressions allow functions to be implemented and passed directly without declaring ...
Posted by system_critical on Fri, 08 Oct 2021 11:30:18 -0700
[vernacular analysis] Flink's Watermark mechanism
catalogue
0x00 summary
0x01 problem
Q1. What are the common requirements / solutions in Flink stream processing applications
Question 2. Watermark should be translated into water mark
Question 3. What is the essence of watermark
Question 4. How does watermark solve the problem
0x02 background concept
Stream processing
Out of order
0 ...
Posted by schilly on Tue, 28 Sep 2021 20:06:11 -0700
FlinkSQL custom function (UDF) dimension table conversion
preface
Relationship between Table and SQL: SQL is the inheritance encapsulation of Table (this point is in Flink concept For example, StreamTableEnvironment can be embodied by inheriting from TableEnvironment. Therefore, the use of tables in official documents can be reflected in SQL, such as user-defined functions, Table API & custom fun ...
Posted by scofansnags on Tue, 07 Sep 2021 19:38:53 -0700
Kakfa Flink hive integration principle and actual combat code
Hello, I'm brother Tu.
At present, he works as a big data algorithm engineer in a large Internet factory.
Today, a fan sent a message in the group asking if there is any information about Flink DDL, saying that he is a new learner.
In order to let this fan quickly learn how to use Flink DDL, I will explain the principle and attach the actu ...
Posted by amazinggrace1983 on Tue, 07 Sep 2021 16:02:28 -0700