Question scenario:
The kittle tool for data extraction on the server side may sometimes have various problems. The abnormal closure of dos window results in abnormal data extraction work. Therefore, a solution to this problem is proposed.
Reason analysis:
After Baidu, many of them are kittle memory problems, but what our leaders want is that the memory is normal, and then consider its own self-startup problem.
Solution:
Attach the memory solution technology paste: http://blog.csdn.net/huangyanlong/article/details/42453831
Considering from the aspect of application self-startup:
Solutions:
The batch script is used to monitor whether the kittle process has been started or not. If started, the script does nothing and ends automatically. If the process is not started, the kittle is started using the command line mode. This monitoring task checks it regularly by using the timed tasks that come with windows system.
First, you must clearly use the command line to start the kittle:
Kitchen/file:(full path of KJB file)/level:basic> (full path of log file)
Instruction meaning is not explained here.
From the instruction point of view, he is calling a kitchen.bat script in the local kittle file to perform the task.
So check in the kitchen.bat script:
@echo off
setlocal
SET initialDir=%cd%
pushd %~dp0
SET STARTTITLE="Kitchen"
SET SPOON_CONSOLE=1
call Spoon.bat -main org.pentaho.di.kitchen.Kitchen -initialDir "%initialDir%"\ %*
popd
As can be seen from the script content, it is essentially the spoon.bat batch handler invoked. Similarly, let's go to spoon.bat to see:
@echo off
setlocal
cd /D %~dp0
REM **************************************************
REM ** Set console window properties **
REM **************************************************
REM TITLE Spoon console
REM COLOR F0
:: **************************************************
:: ** Kettle home **
:: **************************************************
if "%KETTLE_DIR%"=="" set KETTLE_DIR=%~dp0
if %KETTLE_DIR:~-1%==\ set KETTLE_DIR=%KETTLE_DIR:~0,-1%
cd %KETTLE_DIR%
REM Special console/debug options when called from SpoonConsole.bat or SpoonDebug.bat
if "%SPOON_CONSOLE%"=="1" set PENTAHO_JAVA=java.exe
if not "%SPOON_CONSOLE%"=="1" set PENTAHO_JAVA=javaw.exe
set IS64BITJAVA=0
call "%~dp0set-pentaho-env.bat"
REM **************************************************
REM Platform Specific SWT **
REM **************************************************
REM The following line is predicated on the 64-bit Sun
REM java output from -version which
REM looks like this (at the time of this writing):
REM
REM java version "1.6.0_17"
REM Java(TM) SE Runtime Environment (build 1.6.0_17-b04)
REM Java HotSpot(TM) 64-Bit Server VM (build 14.3-b01, mixed mode)
REM
REM Below is a logic to find the directory where java can found. We will
REM temporarily change the directory to that folder where we can run java there
pushd "%_PENTAHO_JAVA_HOME%"
if exist java.exe goto USEJAVAFROMPENTAHOJAVAHOME
cd bin
if exist java.exe goto USEJAVAFROMPENTAHOJAVAHOME
popd
pushd "%_PENTAHO_JAVA_HOME%\jre\bin"
if exist java.exe goto USEJAVAFROMPATH
goto USEJAVAFROMPATH
:USEJAVAFROMPENTAHOJAVAHOME
FOR /F %%a IN ('.\java.exe -version 2^>^&1^|%windir%\system32\find /C "64-Bit"') DO (SET /a IS64BITJAVA=%%a)
GOTO CHECK32VS64BITJAVA
:USEJAVAFROMPATH
FOR /F %%a IN ('java -version 2^>^&1^|%windir%\system32\find /C "64-Bit"') DO (SET /a IS64BITJAVA=%%a)
GOTO CHECK32VS64BITJAVA
:CHECK32VS64BITJAVA
IF %IS64BITJAVA% == 1 GOTO :USE64
:USE32
REM ===========================================
REM Using 32bit Java, so include 32bit SWT Jar
REM ===========================================
set LIBSPATH=libswt\win32
GOTO :CONTINUE
:USE64
REM ===========================================
REM Using 64bit java, so include 64bit SWT Jar
REM ===========================================
set LIBSPATH=libswt\win64
set SWTJAR=..\libswt\win64
:CONTINUE
popd
REM **********************
REM Collect arguments
REM **********************
set _cmdline=
:TopArg
if %1!==! goto EndArg
set _cmdline=%_cmdline% %1
shift
goto TopArg
:EndArg
REM ******************************************************************
REM ** Set java runtime options **
REM ** Change 2048m to higher values in case you run out of memory **
REM ** or set the PENTAHO_DI_JAVA_OPTIONS environment variable **
REM ******************************************************************
if "%PENTAHO_DI_JAVA_OPTIONS%"=="" set PENTAHO_DI_JAVA_OPTIONS="-Xms512m" "-Xmx512m" "-XX:MaxPermSize=256m"
set OPT=%OPT% %PENTAHO_DI_JAVA_OPTIONS% "-Dhttps.protocols=TLSv1,TLSv1.1,TLSv1.2" "-Djava.library.path=%LIBSPATH%" "-DKETTLE_HOME=%KETTLE_HOME%" "-DKETTLE_REPOSITORY=%KETTLE_REPOSITORY%" "-DKETTLE_USER=%KETTLE_USER%" "-DKETTLE_PASSWORD=%KETTLE_PASSWORD%" "-DKETTLE_PLUGIN_PACKAGES=%KETTLE_PLUGIN_PACKAGES%" "-DKETTLE_LOG_SIZE_LIMIT=%KETTLE_LOG_SIZE_LIMIT%" "-DKETTLE_JNDI_ROOT=%KETTLE_JNDI_ROOT%"
REM ***************
REM ** Run... **
REM ***************
if %STARTTITLE%!==! SET STARTTITLE="Spoon"
REM Eventually call java instead of javaw and do not run in a separate window
if not "%SPOON_CONSOLE%"=="1" set SPOON_START_OPTION=start %STARTTITLE%
@echo on
%SPOON_START_OPTION% "%_PENTAHO_JAVA%" %OPT% -jar launcher\pentaho-application-launcher-7.0.0.0-25.jar -lib ..\%LIBSPATH% %_cmdline%
@echo off
if "%SPOON_PAUSE%"=="1" pause
As can be seen above:
1. Actually, this file is the startup entry of kittle software. By judging the parameters of the previous step, we can judge whether it is started directly or from kitchen.bat. If it is started from kitchen.bat, we can start the software by java.exe. If it is started by double-clicking directly, it is not read and transmitted to kitchen.bat. Parameters, so start using javax.exe (as is known to all, javax.exe is a program that starts the Java interface).
2. Call chain initiated by command line: command line input instruction > kitchen. bat > spoon. bat > java. exe
Analysis call chain:
1. The command line specifies a call to the kitchen.bat script, which defines a parameter: SPOON_CONSOLE and assigns a value of 1.
2. Kichen.bat calls spoon.bat. In spoon.bat, the runner java.exe is specified by reading the parameters given in the previous step.
Students who have used tomcat may know that the essence of tomcat startup is also to call java.exe. In order to prevent conflicts, we must change the object we monitor java.exe. Then, the question arises: how can we change the process name of the monitored object we need? Thus, Baidu can create its own process method: http://blog.csdn.net/fangdengfu123/article/details/70051498
From this, we can conclude that we must change the original pointing program in spoon, but we can not change the normal use of kittle s, so there are two schemes here:
1. Create a copy of spoon.bat and change the value of PENTAHO_JAVA in that copy (that is, point the running program to our own startup program);
2. Create a copy of kitchen.bat, in which spoon.bat is still called, but modify the parameter value of SPOON_CONSOLE, and then judge the parameter in spoon.bat. If it is equal to the value we set, then point the value of PENTAHO_JAVA to our program entry.
So essentially, we just need to monitor java.exe, which gives birth to our own batch scripts for the surface layer:
@echo off
:after
:check javaKittleProHuizong.exe
tasklist >list.txt
find /i "javaKittleProHuizong.exe" list.txt
if "%errorlevel%"=="1" (goto before) else (goto over)
:before
cd e:\data-integration
start kittlePreHuizong.bat
echo "use ping to delay"
set SLEEP=ping 127.0.0.1 /n
echo %time%
%SLEEP% 30 > nul
echo %time%
:over
end
The main task of the script is to check whether the specified javaKittleProHuizong.exe exists, if it exists, jump directly to the end of the program; if it does not exist, call the copy of kittlePreHuizong.bat we created.
KitlePreHuizong. bat is an instruction to start kittle s.
Then create our kittlePreHuizong.bat batch program:
@echo off
setlocal
SET initialDir=%cd%
pushd %~dp0
SET STARTTITLE="KitchenPreHuizong"
SET SPOON_CONSOLE=1
call SpoonPreHuizong.bat -main org.pentaho.di.kitchen.Kitchen -initialDir "%initialDir%"\ %*
popd
This script calls our replicated spoon.bat copy:
SpoonPreHuizong.bat
In this copy, we only modify the quotation address of the variable.
if "%SPOON_CONSOLE%"=="1" set PENTAHO_JAVA=javaKittleProHuizong.exe
Then create a timed task in windows timed task and call the monitoring script regularly.
Although the monitoring dos can be kept on for a time delay and then check the process, it is not allowed to shut down the dos which is responsible for monitoring abnormally, so the timing task is safer.