scala -- Set, Map, iterator, flattening, filtering, sorting, grouping, aggregation

Keywords: Scala Big Data Hadoop

4. Set set

Set (also known as: Set) represents a set without duplicate elements. Features: unique, disordered

  • The only meaning is that the elements in the Set are unique and there are no duplicate elements
  • Unordered means that the order in which the elements in the Set are added and taken out is inconsistent
Format I: Create an empty immutable set
val/var Variable name = Set[type]()
Format II: Given elements to create an immutable set
val/var Variable name = Set(Element 1, Element 2, Element 3...)

Example

//Case: demonstrate immutable sets
object ClassDemo16 {
    def main(args: Array[String]): Unit = {
        //1. Define an empty integer immutable set
        val set1 = Set[Int]()
        //2. Define an immutable set and save the following elements: 1,1,3,2,4,8
        val set2 = Set(1, 1, 3, 2, 4, 8)
        //3. Print results
        println(s"set1: ${set1}")
        println(s"set2: ${set2}")
    }
}

Common operations of immutable sets

  • Gets the size of the set
  • Traversal set (consistent with traversal array)
  • Add an element to generate a new Set (+)
  • Splice two sets to generate a new Set (+ +)
  • Splice sets and lists to generate a new Set (+ +)
//Case: demonstrate common operations of immutable sets
object ClassDemo17 {
  def main(args: Array[String]): Unit = {
    //1. Create a set containing the following elements: 1,1,2,3,4,5
    val set1 = Set(1, 1, 2, 3, 4, 5)
    //2. Get the size of the set
    println("set1 The length of the is: " + set1.size)
    //3. Traverse the set and print each element
    println("set1 The elements in the set are: ")
    for (i <- set1) println(i)
    println("-" * 15)
    //4. Delete element 1 to generate a new set
    val set2 = set1 - 1
    println("set2: " + set2)
    //5. Splice another set (6, 7, 8)
    val set3 = set1 ++ Set(6, 7, 8)
    println("set3: " + set3)
    //6. Splice a list (6, 7, 8, 9)
    val set4 = set1 ++ List(6, 7, 8, 9)
    println("set4: " + set4)
  }
}

Variable set

Variable sets refer to elements. The length of sets is variable. Its creation method is the same as that of immutable sets, except that the variable set class needs to be imported first.

Manual import: import scala.collection.mutable.Set

import scala.collection.mutable.Set

//Case: demonstrate variable sets
object ClassDemo18 {
  def main(args: Array[String]): Unit = {
    //1. Define a variable set that contains the following elements: 1, 2, 3, 4
    val set1 = Set(1, 2, 3, 4)
    //2. Add element 5 to the variable set
    set1 += 5
    //3. Add elements 6, 7 and 8 to the variable set
    //set1 ++= Set(6, 7, 8)
    set1 ++= List(6, 7, 8) //Either way
    //4. Remove element 1 from the variable set
    set1 -= 1
    //5. Remove elements 3, 5, 7 from the variable set
    //set1 --= Set(3, 5, 7)
    set1 --= List(3, 5, 7) //Either way
    //6. Print results
    println(set1)
  }
}

mapping

Mapping refers to a Map. It is a collection of key value pairs. The feature is that keys are unique, but values can be repeated. In Scala, Map is also divided into immutable Map and variable Map.

Immutable Map

Immutable Map refers to elements whose length is immutable

val/var map = Map(key->value, key->value, key->value...) // Recommended, better readability
val/var map = Map((key, value), (key, value), (key, value), (key, value)...)

Example

//Case: demonstrating immutable Map
object ClassDemo19 {
    def main(args: Array[String]): Unit = {
        //1. Define a mapping that contains the following student names and age data
        val map1 = Map("Zhang San" -> 23, "Li Si" -> 24, "Li Si" -> 40)
        val map2 = Map(("Zhang San", 23),("Li Si", 24), ("Li Si" -> 40))
        //2. Print results
        println(s"map1: ${map1}")
        println(s"map2: ${map2}")
    }
}

Variable Map

Variable Map refers to elements with variable length. The definition syntax is consistent with that of immutable Map,

Just manually import the package: import scala.collection.mutable.Map

Example

import scala.collection.mutable.Map
//Case: demonstrate variable Map
object ClassDemo20 {
    def main(args: Array[String]): Unit = {
        //1. Define a mapping that contains the following student names and age data
        val map1 = Map("Zhang San" -> 23, "Li Si" -> 24)
        val map2 = Map(("Zhang San", 23),("Li Si", 24))
        //2. Change Zhang San's age to 30
        map1("Zhang San") = 30
        //3. Print results
        println(s"map1: ${map1}")
        println(s"map2: ${map2}")
    }
}

Basic operation of Map

  • map(key): get the corresponding value according to the key. If the key does not exist, return None

  • map.keys: get all the keys

  • map.values: get all values

  • Traversing the map set: it can be implemented through ordinary for

  • getOrElse: get the corresponding value according to the key. If the key does not exist, the specified default value will be returned

  • +No.: add key value pairs and generate a new Map

be careful: If variable Map, You can use +=perhaps++= Direct to the variable Map Add key value pair element in
  • -No.: delete the corresponding key value pair element according to the key, and generate a new Map

    be careful: If variable Map, You can use -=perhaps--= Directly from this variable Map Delete key value pair elements in.
    

Example

import scala.collection.mutable.Map

//Case: demonstrate common operations of Map
object ClassDemo21 {
  def main(args: Array[String]): Unit = {
    //1. Define a mapping that contains the following student names and age data: Zhang San - > 23, Li Si - > 24
    val map1 = Map("Zhang San" -> 23, "Li Si" -> 24)
    //2. Get Zhang San's age and print it
    println(map1.get("Zhang San"))
    //3. Get the names of all students and print them
    println(map1.keys)
    //4. Obtain all student ages and print them
    println(map1.values)
    //5. Print the names and ages of all students
    for ((k, v) <- map1) println(s"key:${k}, value:${v}")
    println("-" * 15)
    //6. Obtain the age of "king five". If "king five" does not exist, return - 1 and print
    println(map1.getOrElse("Wang Wu", -1))
    println("-" * 15)
    //7. Add a new student: Wang Wu, 25, and print the results
    /*//Immutable Map
    val map2 = map1 + ("Wang Wu "-> 25)
    println(s"map1: ${map1}")
    println(s"map2: ${map2}")*/
    map1 += ("Wang Wu" -> 25)
    //8. Remove 'Li Si' from the variable mapping and print it
    map1 -= "Li Si"
    println(s"map1: ${map1}")
  }
}

iterator

Scala provides an iterator for each kind of collection to iterate access to the collection

There are two methods in the iterator:

  • hasNext method: query whether there is the next element in the container
  • Next method: returns the next element of the iterator. If not, throw NoSuchElementException

Example

//Case: demonstrating iterators
object ClassDemo22 {
  def main(args: Array[String]): Unit = {
    //1. Define a list containing the following elements: 1,2,3,4,5
    val list1 = List(1, 2, 3, 4, 5)
    //2. Use the while loop and iterator to traverse and print the list
    //2.1 get its corresponding iterator object according to the list
    val it = list1.iterator
    //2.2 judge whether there is the next element in the iterator
    while(it.hasNext){
      //2.3 if yes, get the next element and print it
      println(it.next)
    }
    //Split line
    println("-" * 15)
    //After the iteration, if the iterator is used to get the element again, an exception is thrown: NoSuchElementException
    println(it.next)
  }
}

Functional programming

Traversal (foreach)

def foreach(f:(A) => Unit): Unit
//Abbreviated form
def foreach(function)

Example

//Case: demonstrating foreach function
object ClassDemo23 {
    def main(args: Array[String]): Unit = {
        //1. Define a list containing 1, 2, 3 and 4
        val list1 = List(1, 2, 3, 4)
        //2. Traverse the above list through foreach function
        //x: Represents each element in the set. The function body represents each element in the output set
        list1.foreach((x:Int) => println(x))
    }
}
//Case: demonstrate simplified function definition
object ClassDemo24 {
    def main(args: Array[String]): Unit = {
        //1. There is a list containing elements 1, 2, 3 and 4. Please use the foreach method to traverse and print each element
        val list1 = List(1, 2, 3, 4)
        list1.foreach((x:Int) => println(x))
        println("-" * 15)
        //2. Use type inference to simplify function definition
        list1.foreach(x => println(x))
        println("-" * 15)
        //3. Use underscores to simplify function definitions
        list1.foreach(println(_))
    }
}

Map

def map[B](f: (A) ⇒ B): TraversableOnce[B]
//Short form:
def map(Function object)

Example

//Case: demonstrate map function (mapping)
object ClassDemo25 {
    def main(args: Array[String]): Unit = {
        //1. Create a list containing elements 1, 2, 3, and 4
        val list1 = List(1, 2, 3, 4)
        //2. Convert the above numbers into ` * 'of the corresponding number, that is, 1 becomes *, 2 becomes * *, and so on
        //Method 1: common writing
        val list2 = list1.map((x:Int) => "*" * x)
        println(s"list2: ${list2}")
        //Method 2: implemented by type inference
        val list3 = list1.map(x => "*" * x)
        println(s"list3: ${list3}")
        //Method 3: realized by underlining
        val list4 = list1.map("*" * _)
        println(s"list4: ${list4}")
    }
}

Flat map

Flattening mapping can be understood as map first and then flatten

def flatMap[B](f:(A) => GenTraversableOnce[B]): TraversableOnce[B]
//Short form:
def flatMap(f:(A) => To add an element A Set converted to B List of)

demand

  1. There is a list containing several text lines: "Hadoop hive spark flick flume", "kudu HBase spoon storm"
  2. Get each word in the text line and put each word in the list

Example

//Case: demonstrate flatmap
object ClassDemo26 {
  def main(args: Array[String]): Unit = {
    //1. There is a list containing several text lines: "Hadoop hive spark flick flume", "kudu HBase spoon storm"
      val list1 = List("hadoop hive spark flink flume", "kudu hbase sqoop storm")
      //2. Get each word in the text line and put each word in the list
      //Method 1: implemented by map and flatten
      val list2 = list1.map(_.split(" "))
      val list3 = list2.flatten
      println(s"list3: ${list3}")
    //Mode 2: implemented through flatMap
    val list4 = list1.flatMap(_.split(" "))
    println(s"list4: ${list4}")
  }
}

Filter

Filtering refers to filtering out (filtering out) elements that meet certain conditions

def filter(f:(A) => Boolean): TraversableOnce[A]
//Short form:
def filter(f:(A) => Screening conditions)

Example

//Case: demonstration filter
object ClassDemo27 {
    def main(args: Array[String]): Unit = {
        //1. There is a numeric list with elements of 1,2,3,4,5,6,7,8,9
        val list1 = (1 to 9).toList
        //2. Please filter out all even numbers
        val list2 = list1.filter(_ % 2 == 0)
        println(s"list2: ${list2}")
    }
}

sort

sorted is used to sort the set elements by default and arrange the list elements in ascending order. If descending order is required, the ascending order is followed by reverse

sortBy is used to sort the collection according to the specified field

sortWith is used to customize the sorting of collections

Example

//Case: demonstrating default sorting
object ClassDemo28 {
    def main(args: Array[String]): Unit = {
        //1. Define a list containing the following elements: 3, 1, 2, 9, 7
        val list1 = List(3, 1, 2, 9, 7)
        //2. Sort the list in ascending order
        val list2 = list1.sorted
        println(s"list2: ${list2}")
        //3. Arrange the list in descending order
        val list3 = list2.reverse
        println(s"list3: ${list3}")
    }
}

//Case: demonstrate sorting according to specified fields (sortBy)
object ClassDemo29 {
    def main(args: Array[String]): Unit = {
        //1. There is a list containing several text lines: "01 hadoop", "02 flume", "03 hive", "04 spark"
        val list1 = List("01 hadoop", "02 flume", "03 hive", "04 spark")
        //2. Please sort by word letters
        //val list2 = list1.sortBy(x => x.split(" ")(1))
        //Short form:
        val list2 = list1.sortBy(_.split(" ")(1))
        println(s"list2: ${list2}")
    }
}
//Case: demonstrating custom sorting (sortWith)
object ClassDemo30 {
    def main(args: Array[String]): Unit = {
        //1. There is a list containing the following elements: 2,3,1,6,4,5
        val list1 = List(2,3,1,6,4,5)
        //2. Use sortWith to sort the list in descending order
        //Val List2 = list1.sortwith ((x, y) = > x > y) / / descending order
        //Short form:
        val list2 = list1.sortWith(_ > _) //Descending order
        println(s"list2: ${list2}")
    }
}		

Group by

Grouping refers to grouping data according to specified conditions, so as to facilitate our statistical analysis of data

//Case: demonstrate grouping function (groupBy)
object ClassDemo31 {
  def main(args: Array[String]): Unit = {
    //1. There is a list containing the names and gender of students: "Andy Lau" - > "male", "Liu Yifei" - > "female", "Hu Ge" - > "male"
    val list1 = List("Lau Andy" -> "male", "Liu Yifei" -> "female", "Hu Ge" -> "male")
    //2. Please group according to gender
    //val list2 = list1.groupBy(x => x._2)
    //Abbreviated form
    val list2 = list1.groupBy(_._2)
    //println(s"list2: ${list2}")
    //3. Count the number of students of different genders
    val list3 = list2.map(x => x._1 -> x._2.size)
    println(s"list3: ${list3}")
  }
}

Aggregation operation

The so-called aggregation operation refers to merging the data in a list into one. This operation is often used in statistical analysis. There are two common aggregation operations:

  • reduce: used to aggregate and calculate collection elements

    reduce and reduceLeft The effect is consistent, indicating that it is calculated from left to right
    reduceRight Indicates right to left calculation
    
  • fold: used to calculate the folding of collection elements

fold And reduce Very similar, except for an additional parameter that specifies the initial value.

Example

object ClassDemo32 {
def main(args: Array[String]): Unit = {
        //1. Define a list containing the following elements: 1,2,3,4,5,6,7,8,9,10
        val list1 = (1 to 10).toList
        //2. Use reduce to calculate the sum of all elements
        //val list2 = list1.reduce((x, y) => x + y)
        //Short form:
        val list2 = list1.reduce(_ + _)
        val list3 = list1.reduceLeft(_ + _)
        val list4 = list1.reduceRight(_ + _)
        println(s"list2: ${list2}")
        println(s"list3: ${list3}")
        println(s"list4: ${list4}")
    }
}
//Case: demonstrate fold calculation
object ClassDemo33 {
    def main(args: Array[String]): Unit = {
        //1. Define a list containing the following elements: 1,2,3,4,5,6,7,8,9,10
        val list1 = (1 to 10).toList
        //2. Assuming the initialization value is 100, use fold to calculate the sum of all elements
        //val list2 = list1.fold(100)((x, y) => x + y)
        //Short form:
        val list2 = list1.fold(100)(_ + _)
        val list3 = list1.foldLeft(100)(_ + _)
        val list4 = list1.foldRight(100)(_ + _)
        println(s"list2: ${list2}")
        println(s"list3: ${list3}")
        println(s"list4: ${list4}")
    }
}

Posted by wee493 on Mon, 29 Nov 2021 15:14:24 -0800