Small pit record of flink performance monitoring based on Prometheus

background The flink cluster in the company has been running for a long time and has not been fully monitored. Recently, it plans to do this. Through online research, the deployment mode adopted by the company is per job mode. Finally, the job index is pushed to the push gateway of the intermediate gateway based on prometheus, Then prometheus ...

Posted by jtgraphic on Fri, 26 Nov 2021 16:43:59 -0800

Big data Flume custom type

1. Customize the Interceptor 1.1 case requirements When Flume is used to collect the local logs of the server, different types of logs need to be sent to different analysis systems according to different log types. 1.2 demand analysis: Interceptor and Multiplexing ChannelSelector cases In actual development, there may be many types of ...

Posted by nainil on Fri, 26 Nov 2021 08:12:12 -0800

scala -- format and usage of function + lazy lazy usage + difference between method and function + method calling method

1. Method 1.1 general In actual development, we need to write a large number of logic codes, which is bound to involve repeated requirements. For example, for the maximum values of 10 and 20 and the maximum values of 11 and 22, the logic code for comparison needs to be written twice. If the logic code for comparison is put into the method, it ...

Posted by tony-kidsdirect on Thu, 25 Nov 2021 16:45:14 -0800

Figure calculation: Processing hierarchical data using Spark Graphx Pregel API

Today, distributed computing engines are the backbone of many analysis, batch, and streaming applications. Spark provides many advanced functions (pivot, analysis window function, etc.) to convert data out of the box. Sometimes you need to process hierarchical data or perform hierarchical calculations. Many database vendors provide functions su ...

Posted by newyear498 on Thu, 25 Nov 2021 15:14:49 -0800

Experiment 6 is familiar with Hive's basic operations

1. Experimental Purpose (1) Understand Hive's role as a data warehouse in the Hadoop architecture. (2) Skilled in using commonly used HiveQL. 2. Experimental Platform Operating system: Ubuntu 18.04 (or Ubuntu 16.04);Hadoop version: 3.1.3;Hive version: 3.1.2;JDK version: 1.8. 3. Data Sets Dead work: Provided by Hive Programming Guide ...

Posted by abcd1234 on Thu, 25 Nov 2021 09:12:34 -0800

Big data Azkaban Work Flow practice

1 HelloWorld case 1) In the windows environment, create a new azkaban.project file. The editing content is as follows: Azkaban flow version: 2.0 Note: the function of this file is to parse the flow file using the new flow API method. 2.0 azkaban supports both properties configuration files and yml configuration files! 3.0 azkaban sup ...

Posted by aeonsky on Wed, 24 Nov 2021 13:25:18 -0800

Big data foundation

Big data concept Big Data: refers to a data set that cannot be captured, managed and processed by conventional software tools within a certain time range. It is a massive, high growth rate and diversified information asset that requires a new processing mode to have stronger decision-making power, insight and discovery power and process op ...

Posted by mykg4orce on Tue, 23 Nov 2021 18:58:52 -0800

Grinding 23 kinds of big talk design modes ----- decorator mode (decorator mode)

Hello, everyone. I'm a student in the java learning circle who doesn't want to disclose my name and lives a shoddy life. If there are mistakes in the article, I hope Haihan. You're welcome to make more corrections If you get useful dry goods knowledge from this article, please give me a favor. It is said that all the likes have got offer s ...

Posted by andr923 on Tue, 23 Nov 2021 17:39:27 -0800

Introduction to Scala -- data types

1. Output statements and semicolons 1.1 output statement Mode 1: line feed output format: println(Write the data you want to print to the console); Mode 2: output without line feed format: print(Write the data you want to print to the console); Note: multiple values can be printed at the same time regardless of println() or print() ...

Posted by theresandy on Tue, 23 Nov 2021 15:02:59 -0800

Understand the modularity of NgModule~Angular

  catalogue @Class of NgModule() decorator Common system modules of Angular 1. Example - create and import feature modules order.module.ts list.component.ts app.module.ts app.component.html 2. Example - inert loading module order.module order-routing.module Configure in the top-level routing AppRoutingModule app.component.html im ...

Posted by optimiss on Tue, 23 Nov 2021 01:33:30 -0800