Flink Summary - set Jvm parameters

Keywords: jvm Apache snapshot Java

jvm settings

http://www.360doc.com/content/12/1023/16/9615799_243296263.shtml

Heap setup
 -Xms: initial heap size
 -Xmx: maximum heap size
 -20: Newsize = n: set younger generation size
 -20: Newratio = n: set the ratio of young generation to old generation. For example, if it is 3, it means that the ratio of the young generation to the old generation is 1:3, and the young generation accounts for 1 / 4 of the sum of the young generation and the old generation
 -20: Survivorratio = n: ratio of Eden region to two survivor regions in the younger generation. Note that there are two survivor areas. For example, 3 means Eden: Survivor=3:2, and one survivor area accounts for 1 / 5 of the whole young generation
 -20: Maxpermsize = n: set persistent generation size
 Collector settings
 -20: + useserialgc: set serial collector
 -20: + useparallelgc: set parallel collector
 -20: + useparalleledoldgc: set up parallel aging collector
 -20: + useconcmarksweepgc: set concurrent collector
 Garbage collection statistics
 -20: + printheatgc GC's heap details
 -20: + printgcdetails GC details
 -20: + printgctimestamps print GC time information
 -20: + printtenuringdistribution print age information, etc
 -20: + handlepromotionfailure old age distribution guarantee (true or false)
Parallel collector settings
 -20: Parallelgcthreads = n: sets the number of CPU s used by the parallel collector to collect. Number of parallel collection threads.
-20: Maxgcpausemillis = n: set the maximum pause time of parallel collection
 -20: Gctimeratio = n: set the percentage of garbage collection time in the running time of the program. The formula is 1/(1+n)
Concurrent collector settings
 -20: + cmsincrementalmode: set to incremental mode. It is suitable for single CPU.
-20: Parallelgcthreads = n: sets the number of CPU s used when the collection method of the younger generation of the concurrent collector is parallel collection. Number of parallel collection threads

flink setting jvm parameters

https://ci.apache.org/projects/flink/flink-docs-release-1.3/setup/config.html

./flink-1.2.0/bin/flink run -m yarn-cluster -yn 4 -yjm 2048 -ytm 8086 -c beam.count.OpsCount -yqu data-default \
-yD taskmanager.heap.mb=4096 -yD yarn.heap-cutoff-ratio=0.6 -yD taskmanager.debug.memory.startLogThread=true -yD taskmanager.debug.memory.logIntervalMs=600000 \
-yz toratest -yst -yd ./beampoc-bundled-0.0.1-SNAPSHOT.jar --parallelism=4

https://mingminxu.com/2017/01/16/run-flink-jobs-on-yarn/

Add jvm parameters of flink

-yD env.java.opts="-XX:NewRatio=2"

jmap output of the default flink 4container slot4

Heap Configuration:
   MinHeapFreeRatio         = 0
   MaxHeapFreeRatio         = 100
   MaxHeapSize              = 2359296000 (2250.0MB)
   NewSize                  = 786432000 (750.0MB)
   MaxNewSize               = 786432000 (750.0MB)
   OldSize                  = 1572864000 (1500.0MB)
   NewRatio                 = 2
   SurvivorRatio            = 8
   MetaspaceSize            = 21807104 (20.796875MB)
   CompressedClassSpaceSize = 1073741824 (1024.0MB)
   MaxMetaspaceSize         = 17592186044415 MB
   G1HeapRegionSize         = 0 (0.0MB)

Heap Usage:
PS Young Generation
Eden Space:
   capacity = 476053504 (454.0MB)
   used     = 224818576 (214.40370178222656MB)
   free     = 251234928 (239.59629821777344MB)
   47.225484974058716% used
From Space:
   capacity = 101187584 (96.5MB)
   used     = 85147536 (81.20301818847656MB)
   free     = 16040048 (15.296981811523438MB)
   84.14820537665965% used
To Space:
   capacity = 155189248 (148.0MB)
   used     = 0 (0.0MB)
   free     = 155189248 (148.0MB)
   0.0% used
PS Old Generation
   capacity = 1572864000 (1500.0MB)
   used     = 922110224 (879.3928375244141MB)
   free     = 650753776 (620.6071624755859MB)
   58.62618916829427% used

22206 interned Strings occupying 2255248 bytes.

 

0 people praise

 

Flink flow calculation

 



By Zachary Ju 1db5
Link: https://www.jianshu.com/p/6f2f53358b9c
Source: Jianshu
The copyright belongs to the author. For commercial reprint, please contact the author for authorization. For non-commercial reprint, please indicate the source.

172 original articles published, 158 praised, 2.57 million visitors+
His message board follow

Posted by rinkhals on Thu, 12 Mar 2020 04:03:51 -0700