Parallel programming of Scala Actor in spark notes

1.1. Course objectives1.1.1. Objective 1: familiar with Scala Actor concurrent programming1.1.2. Goal 2: prepare for Akka learning Note: Scala Actor is scala 2.10.x and earlier. Scala added Akka as its default Actor in version 2.11.x, and the old one has been abandoned. 1.2. What is Scala actor 1.2.1. Concept The Actor in scala can realize the ...

Posted by bogu on Wed, 16 Oct 2019 01:13:40 -0700

Spark streaming manually saves offset to zk java implementation

Article directory Preface pom Dependent Version Dome Preface There are some cases on the Internet about setting offset manually in kafka, but most of them use 0.8 version of kafka, which is written by scala. The version of kafka-0.10 is rarely mentioned or incomplete. Version 0.10 is compatible ...

Posted by artin on Tue, 08 Oct 2019 21:41:57 -0700

158 Scala Basic Grammar

1. Declare variables package cn.itcast.scala object VariableDemo { def main(args: Array[String]) { //Variable values defined with val are immutable, equivalent to variables modified with final in java val i = 1 //Variables defined with var are changeable, and val is encouraged in Sc ...

Posted by pure_skill_2000 on Sun, 06 Oct 2019 09:45:47 -0700

173 Scala Implicit Conversion and Implicit Parameters

Implicit transformation and implicit parameters are two very powerful functions in Scala. With implicit transformation and implicit parameters, you can provide elegant class libraries and hide boring details from the users of class libraries. Implicit enhancements of class methods enrich the functions o ...

Posted by Braveheartt on Sun, 06 Oct 2019 09:34:08 -0700

Spark Learning 02 - A Method to Create DStream

Spark Streaming provides two types of built-in streaming media sources. Basic source: The source directly provided in the StreamingContext API. Example: File system and socket connection. Advanced resources: Kafka, Flume, Kinesis and other resources can be obtained through additional utility classes. ...

Posted by leebo on Thu, 03 Oct 2019 10:21:03 -0700

Spark is exported to an existing directory

Links to the original text: https://my.oschina.net/asparagus/blog/699814 Spark saves the output data to an existing directory Save by the saveAsTextFile method of RDD String input =  ...

Posted by webguync on Thu, 03 Oct 2019 07:17:07 -0700

scala_Akka Concurrent Programming Framework

Article Directory Introduction to Akka Concurrent Programming Framework Introduction to Akka Akka characteristics Akka communication process Create Actor Introduction to API Getting Started Cases Implementation Steps 1. Create Maven modules 2. Create and load Actor s 3. Send/receive messages Akka ...

Posted by Stonewall on Fri, 20 Sep 2019 20:24:53 -0700

Apache Flink Zero Foundation Initial Flink Data Stream Programming

Data sources can be created by Stream Execution Environment. addSource (sourceFunction). Flink also provides some built-in data sources for easy use, such as readTextFile(path) readFile(), and of course, it can also write a custom data source (by implementing the SourceFunction method, but can not be implemented in parallel). That's ok. Or impl ...

Posted by McInfo on Tue, 10 Sep 2019 02:57:29 -0700

Implicit conversion of scala

1. Implicit Conversion 1 Basic Introduction Implicit conversion functions are functions declared with a single parameter using the implicit keyword.This function is automatically applied to convert values from one type to another (2) Examples obj ...

Posted by hasjem on Fri, 16 Aug 2019 20:16:53 -0700

Scala for Large Data Learning Notes: Chapter 9 Packages and Introduction

Articles Catalogue 9.1 Pack 9.2 Scope 9.3 Package Objects 9.4 Packet Visibility 9.5 Introduction 9.6 Rename and Hide Method 9.1 Pack Packets, like packages in Java, are designed to organize programs, and packages in Scala do not require associations between directories and packages. The package ...

Posted by hoffmeister on Sun, 04 Aug 2019 22:58:40 -0700