Programmer Group

1. Customize the Interceptor 1.1 case requirements When Flume is used to collect the local logs of the server, different types of logs need to be sent to different analysis systems according to different log types. 1.2 demand analysis: Interceptor and Multiplexing ChannelSelector cases In actual development, there may be many types of ...

1. Method 1.1 general In actual development, we need to write a large number of logic codes, which is bound to involve repeated requirements. For example, for the maximum values of 10 and 20 and the maximum values of 11 and 22, the logic code for comparison needs to be written twice. If the logic code for comparison is put into the method, it ...

1. Arithmetic operator 1.1 introduction to operators The symbol used to splice variables or constants is called: operator, and the formula connected by operators is called: expression. We often use it in practical development For example: 10 + 3 is an expression, and the + sign is an operator Note: in Scala, operators are not only operator ...

1. Output statements and semicolons 1.1 output statement Mode 1: line feed output format: println(Write the data you want to print to the console); Mode 2: output without line feed format: print(Write the data you want to print to the console); Note: multiple values can be printed at the same time regardless of println() or print() ...

  import com.bean.Yyds1 import org.apache.spark.sql.SparkSession object TestReadCSV { def main(args: Array[String]): Unit = { val spark = SparkSession.builder() .appName("CSV Reader") .master("local") .getOrCreate() /** * Parameters can be either strings or specific types, such as boolean * delimiter Separat ...

Process introduction CI/CD is a method to frequently deliver applications to customers by introducing automation in the application development phase. The core concepts of CI/CD are continuous integration, continuous delivery and continuous deployment. In this article, I will introduce the practice of automated build and release based on GitLa ...

10,scala 10.1 introduction to scala scala is a multi paradigm programming language running on the JVM. It supports both object-oriented and function oriented programming. 10.2. scala interpreter To start the scala interpreter, you only need the following steps: Press and hold the windows key + rJust enter scala Execute: quit in the scal ...

Methods and functionsDescription of the distinction between the two Scala has methods and functions. There is little semantic difference between them. In most cases, they are regarded as equivalent. Scala method is a part of a class, and function is an object that can be assigned to a variable, that is, the function defined in the class is a me ...

Spark common RDD operators for big data development map map passes in a piece of data and returns a piece of data Map is to perform function operations on the elements in the RDD one by one and map them to another RDD, Each data item in an RDD is transformed into a new element through the function mapping in the map. Input partition and o ...

Scala express syntax (11) | data structure map mapping function The map function returns a new collection Pass the elements of the collection through the functions in () //Create list val list = List(1, 2, 3, 4, 5) //Each element * 2 println(list.map(_ * 2)) flatmap mapping: flat is flattening, flattening and flattening mapping flat ...