Hadoop running environment building tutorial
1. Environmental preparation:
A linux machine can be a virtual machine installed by local VMware or a real linux machine.
If it is a locally installed virtual machine, the following points need to be pre configured:
Configure the static IP of the machine (to prevent IP changes during restart)
Modify host name (easy to configure)
Turn off the ...
Posted by crazylegseddie on Tue, 25 Feb 2020 19:02:32 -0800
Spring Boot bottom level Exploration Series 04 - Web Development
Article directory
Handling JSON data
1. Edit the Book class
2. Edit BookController controller
3. Convert set data
4. Replace the converter
1) Use Gson
2) Using fastjson
Jiang Shuai, Naixue education, is good at system architecture design, big data, operation and maintenance and other technica ...
Posted by glitch003 on Sat, 22 Feb 2020 03:57:14 -0800
oracle 19c virtual automatic index test
1. About testing
Recently oracle held a series of online sharing, one of the new features of 19c is automatic indexing. I tested it in docker and failed.
[oracle@8aa96a41b58b ~]$ sqlplus / as sysdba
SQL*Plus: Release 19.0.0.0.0 - Production on Thu Feb 20 09:44:52 2020
Version 19.3.0.0.0
SQL> EXEC DBMS_AUTO_INDEX.CONFIGURE('AUTO_INDEX_MODE ...
Posted by jabbaonthedais on Fri, 21 Feb 2020 04:58:37 -0800
Python Crawler Serial 5-Proxy, Cookie Resolution
1. ProxyHandler Processing (Proxy Server)
1. Using proxy IP is a common means of crawling
2. Get the address of the proxy server:
www.xicidaili.com
www.goubanjia.com
3. Proxies are used to hide real access. Proxies do not allow frequent access to a fixed site, so proxies must be many.
4. Basic usage steps:
(1) Set proxy address
(2) Create a Pox ...
Posted by landavia on Wed, 05 Feb 2020 08:49:12 -0800
Spring cloud series - get started quickly hystrix
What is hystrix?
hystrix is a fuse protection middleware used by netflix for microservice distributed system. hystrix provides elegant response results to clients after the service is not reachable or the request times out.
Why use hystrix?
In the microservice scenario, many services depend on e ...
Posted by Sturm on Tue, 04 Feb 2020 06:02:08 -0800
Algorithm learning series - hash table, bloom filter, consistent hash, island problem, and concurrent query set
Article directory
1. Understand hash function and hash table
1.1 what is Hash
1.2 why there should be Hash
1.3 for example, chestnuts:
1. To use array storage, you need to create new arrays new int[]{2,5,9,13}, and then write a loop to traverse the search:
2. If the hash function is used for calcul ...
Posted by duk on Tue, 04 Feb 2020 00:09:04 -0800
Find the number of adjacent words in large amount of data
This topic is similar to some of the search topics in Leetcode.
The problem you want to deal with is: count the number of two adjacent digits of a word. If there are w1,w2,w3,w4,w5,w6, then:
The final output is (word,neighbor,frequency).
We implement it in five ways:
MapReduce
Spark
Spark SQL method
Scala method
Spark SQL for Scala
MapReduce ...
Posted by olechka on Sun, 02 Feb 2020 08:18:59 -0800
Spark SQL/DataFrame/DataSet operation ----- read data
1, Read data source
(1) Read json and use spark.read. Note: the path is from HDFS by default. If you want to read the native file, you need to prefix it file: / /, as follows
scala> val people = spark.read.format("json").load("file:///opt/software/data/people.json")
people: org.apache.spark.sql.DataFrame = [age: bigint, name: string]
scal ...
Posted by Pie on Sun, 02 Feb 2020 08:18:33 -0800
Baidu feijiang deep learning
Catalog
1. In depth learning development process
1.1 Turing test
1.2 hierarchical processing information
1.3 in depth learning
2. Machine learning
2.1 category
2.2 classification
3. Neural network
3.1 initial understanding of neural network
3.2 example of neural network - paddle learning handwri ...
Posted by TWD on Wed, 29 Jan 2020 01:30:39 -0800
The way of Flutter state management
Pick up another article. The way of Flutter state management (4)
This article mainly introduces the shuttle ﹣ mobx
Fish Redux
Version: 0.2.7
Library address: https://github.com/alibaba/fish-redux/
Evolution process
concept
object
Explain
Subordinate Library
Action
Represents an intent that ...
Posted by LuAn on Sun, 19 Jan 2020 01:37:59 -0800