Parallel programming of Scala Actor in spark notes
1.1. Course objectives1.1.1. Objective 1: familiar with Scala Actor concurrent programming1.1.2. Goal 2: prepare for Akka learning
Note: Scala Actor is scala 2.10.x and earlier.
Scala added Akka as its default Actor in version 2.11.x, and the old one has been abandoned.
1.2. What is Scala actor 1.2.1. Concept
The Actor in scala can realize the ...
Posted by bogu on Wed, 16 Oct 2019 01:13:40 -0700
Spark streaming manually saves offset to zk java implementation
Article directory
Preface
pom Dependent Version
Dome
Preface
There are some cases on the Internet about setting offset manually in kafka, but most of them use 0.8 version of kafka, which is written by scala. The version of kafka-0.10 is rarely mentioned or incomplete. Version 0.10 is compatible ...
Posted by artin on Tue, 08 Oct 2019 21:41:57 -0700
158 Scala Basic Grammar
1. Declare variables
package cn.itcast.scala
object VariableDemo {
def main(args: Array[String]) {
//Variable values defined with val are immutable, equivalent to variables modified with final in java
val i = 1
//Variables defined with var are changeable, and val is encouraged in Sc ...
Posted by pure_skill_2000 on Sun, 06 Oct 2019 09:45:47 -0700
173 Scala Implicit Conversion and Implicit Parameters
Implicit transformation and implicit parameters are two very powerful functions in Scala. With implicit transformation and implicit parameters, you can provide elegant class libraries and hide boring details from the users of class libraries.
Implicit enhancements of class methods enrich the functions o ...
Posted by Braveheartt on Sun, 06 Oct 2019 09:34:08 -0700
Spark Learning 02 - A Method to Create DStream
Spark Streaming provides two types of built-in streaming media sources.
Basic source: The source directly provided in the StreamingContext API. Example: File system and socket connection.
Advanced resources: Kafka, Flume, Kinesis and other resources can be obtained through additional utility classes.
...
Posted by leebo on Thu, 03 Oct 2019 10:21:03 -0700
Spark is exported to an existing directory
Links to the original text: https://my.oschina.net/asparagus/blog/699814
Spark saves the output data to an existing directory
Save by the saveAsTextFile method of RDD
String input = ...
Posted by webguync on Thu, 03 Oct 2019 07:17:07 -0700
scala_Akka Concurrent Programming Framework
Article Directory
Introduction to Akka Concurrent Programming Framework
Introduction to Akka
Akka characteristics
Akka communication process
Create Actor
Introduction to API
Getting Started Cases
Implementation Steps
1. Create Maven modules
2. Create and load Actor s
3. Send/receive messages
Akka ...
Posted by Stonewall on Fri, 20 Sep 2019 20:24:53 -0700
Apache Flink Zero Foundation Initial Flink Data Stream Programming
Data sources can be created by Stream Execution Environment. addSource (sourceFunction). Flink also provides some built-in data sources for easy use, such as readTextFile(path) readFile(), and of course, it can also write a custom data source (by implementing the SourceFunction method, but can not be implemented in parallel). That's ok. Or impl ...
Posted by McInfo on Tue, 10 Sep 2019 02:57:29 -0700
Implicit conversion of scala
1. Implicit Conversion
1 Basic Introduction
Implicit conversion functions are functions declared with a single parameter using the implicit keyword.This function is automatically applied to convert values from one type to another
(2) Examples
obj ...
Posted by hasjem on Fri, 16 Aug 2019 20:16:53 -0700
Scala for Large Data Learning Notes: Chapter 9 Packages and Introduction
Articles Catalogue
9.1 Pack
9.2 Scope
9.3 Package Objects
9.4 Packet Visibility
9.5 Introduction
9.6 Rename and Hide Method
9.1 Pack
Packets, like packages in Java, are designed to organize programs, and packages in Scala do not require associations between directories and packages. The package ...
Posted by hoffmeister on Sun, 04 Aug 2019 22:58:40 -0700