Solution to Abnormal Closure in kittle Startup: Batch Processing Solution

Keywords: Java Windows Tomcat

Question scenario:
The kittle tool for data extraction on the server side may sometimes have various problems. The abnormal closure of dos window results in abnormal data extraction work. Therefore, a solution to this problem is proposed.

Reason analysis:
After Baidu, many of them are kittle memory problems, but what our leaders want is that the memory is normal, and then consider its own self-startup problem.

Solution:
Attach the memory solution technology paste: http://blog.csdn.net/huangyanlong/article/details/42453831
Considering from the aspect of application self-startup:
Solutions:
The batch script is used to monitor whether the kittle process has been started or not. If started, the script does nothing and ends automatically. If the process is not started, the kittle is started using the command line mode. This monitoring task checks it regularly by using the timed tasks that come with windows system.
First, you must clearly use the command line to start the kittle:

Kitchen/file:(full path of KJB file)/level:basic> (full path of log file)

Instruction meaning is not explained here.
From the instruction point of view, he is calling a kitchen.bat script in the local kittle file to perform the task.
So check in the kitchen.bat script:

    @echo off
    setlocal
    SET initialDir=%cd%
    pushd %~dp0
    SET STARTTITLE="Kitchen"
    SET SPOON_CONSOLE=1
    call Spoon.bat -main org.pentaho.di.kitchen.Kitchen -initialDir "%initialDir%"\ %*
    popd

As can be seen from the script content, it is essentially the spoon.bat batch handler invoked. Similarly, let's go to spoon.bat to see:

    @echo off
    setlocal 

    cd /D %~dp0

    REM **************************************************
    REM ** Set console window properties                **
    REM **************************************************
    REM TITLE Spoon console
    REM COLOR F0

    :: **************************************************
    :: ** Kettle home                                  **
    :: **************************************************

    if "%KETTLE_DIR%"=="" set KETTLE_DIR=%~dp0
    if %KETTLE_DIR:~-1%==\ set KETTLE_DIR=%KETTLE_DIR:~0,-1%

    cd %KETTLE_DIR%

    REM Special console/debug options when called from SpoonConsole.bat or SpoonDebug.bat
    if "%SPOON_CONSOLE%"=="1" set PENTAHO_JAVA=java.exe
    if not "%SPOON_CONSOLE%"=="1" set PENTAHO_JAVA=javaw.exe
    set IS64BITJAVA=0

    call "%~dp0set-pentaho-env.bat"

    REM **************************************************
    REM   Platform Specific SWT       **
    REM **************************************************

    REM The following line is predicated on the 64-bit Sun
    REM java output from -version which
    REM looks like this (at the time of this writing):
    REM
    REM java version "1.6.0_17"
    REM Java(TM) SE Runtime Environment (build 1.6.0_17-b04)
    REM Java HotSpot(TM) 64-Bit Server VM (build 14.3-b01, mixed mode)
    REM
    REM Below is a logic to find the directory where java can found. We will
    REM temporarily change the directory to that folder where we can run java there
    pushd "%_PENTAHO_JAVA_HOME%"
    if exist java.exe goto USEJAVAFROMPENTAHOJAVAHOME
    cd bin
    if exist java.exe goto USEJAVAFROMPENTAHOJAVAHOME
    popd
    pushd "%_PENTAHO_JAVA_HOME%\jre\bin"
    if exist java.exe goto USEJAVAFROMPATH
    goto USEJAVAFROMPATH
    :USEJAVAFROMPENTAHOJAVAHOME
    FOR /F %%a IN ('.\java.exe -version 2^>^&1^|%windir%\system32\find /C "64-Bit"') DO (SET /a IS64BITJAVA=%%a)
    GOTO CHECK32VS64BITJAVA
    :USEJAVAFROMPATH
    FOR /F %%a IN ('java -version 2^>^&1^|%windir%\system32\find /C "64-Bit"') DO (SET /a IS64BITJAVA=%%a)
    GOTO CHECK32VS64BITJAVA
    :CHECK32VS64BITJAVA


    IF %IS64BITJAVA% == 1 GOTO :USE64

    :USE32
    REM ===========================================
    REM Using 32bit Java, so include 32bit SWT Jar
    REM ===========================================
    set LIBSPATH=libswt\win32
    GOTO :CONTINUE
    :USE64
    REM ===========================================
    REM Using 64bit java, so include 64bit SWT Jar
    REM ===========================================
    set LIBSPATH=libswt\win64
    set SWTJAR=..\libswt\win64
    :CONTINUE
    popd

    REM **********************
    REM   Collect arguments
    REM **********************

    set _cmdline=
    :TopArg
    if %1!==! goto EndArg
    set _cmdline=%_cmdline% %1
    shift
    goto TopArg
    :EndArg

    REM ******************************************************************
    REM ** Set java runtime options                                     **
    REM ** Change 2048m to higher values in case you run out of memory  **
    REM ** or set the PENTAHO_DI_JAVA_OPTIONS environment variable      **
    REM ******************************************************************

    if "%PENTAHO_DI_JAVA_OPTIONS%"=="" set PENTAHO_DI_JAVA_OPTIONS="-Xms512m" "-Xmx512m" "-XX:MaxPermSize=256m"

    set OPT=%OPT% %PENTAHO_DI_JAVA_OPTIONS% "-Dhttps.protocols=TLSv1,TLSv1.1,TLSv1.2" "-Djava.library.path=%LIBSPATH%" "-DKETTLE_HOME=%KETTLE_HOME%" "-DKETTLE_REPOSITORY=%KETTLE_REPOSITORY%" "-DKETTLE_USER=%KETTLE_USER%" "-DKETTLE_PASSWORD=%KETTLE_PASSWORD%" "-DKETTLE_PLUGIN_PACKAGES=%KETTLE_PLUGIN_PACKAGES%" "-DKETTLE_LOG_SIZE_LIMIT=%KETTLE_LOG_SIZE_LIMIT%" "-DKETTLE_JNDI_ROOT=%KETTLE_JNDI_ROOT%"

    REM ***************
    REM ** Run...    **
    REM ***************

    if %STARTTITLE%!==! SET STARTTITLE="Spoon"
    REM Eventually call java instead of javaw and do not run in a separate window
    if not "%SPOON_CONSOLE%"=="1" set SPOON_START_OPTION=start %STARTTITLE%

    @echo on
    %SPOON_START_OPTION% "%_PENTAHO_JAVA%" %OPT% -jar launcher\pentaho-application-launcher-7.0.0.0-25.jar -lib ..\%LIBSPATH% %_cmdline%
    @echo off
    if "%SPOON_PAUSE%"=="1" pause

As can be seen above:
1. Actually, this file is the startup entry of kittle software. By judging the parameters of the previous step, we can judge whether it is started directly or from kitchen.bat. If it is started from kitchen.bat, we can start the software by java.exe. If it is started by double-clicking directly, it is not read and transmitted to kitchen.bat. Parameters, so start using javax.exe (as is known to all, javax.exe is a program that starts the Java interface).
2. Call chain initiated by command line: command line input instruction > kitchen. bat > spoon. bat > java. exe

Analysis call chain:
1. The command line specifies a call to the kitchen.bat script, which defines a parameter: SPOON_CONSOLE and assigns a value of 1.
2. Kichen.bat calls spoon.bat. In spoon.bat, the runner java.exe is specified by reading the parameters given in the previous step.

Students who have used tomcat may know that the essence of tomcat startup is also to call java.exe. In order to prevent conflicts, we must change the object we monitor java.exe. Then, the question arises: how can we change the process name of the monitored object we need? Thus, Baidu can create its own process method: http://blog.csdn.net/fangdengfu123/article/details/70051498

From this, we can conclude that we must change the original pointing program in spoon, but we can not change the normal use of kittle s, so there are two schemes here:
1. Create a copy of spoon.bat and change the value of PENTAHO_JAVA in that copy (that is, point the running program to our own startup program);
2. Create a copy of kitchen.bat, in which spoon.bat is still called, but modify the parameter value of SPOON_CONSOLE, and then judge the parameter in spoon.bat. If it is equal to the value we set, then point the value of PENTAHO_JAVA to our program entry.

So essentially, we just need to monitor java.exe, which gives birth to our own batch scripts for the surface layer:

@echo off

:after
:check javaKittleProHuizong.exe 
tasklist >list.txt 
find /i "javaKittleProHuizong.exe" list.txt 

if "%errorlevel%"=="1" (goto before) else (goto over) 

:before
cd e:\data-integration
start kittlePreHuizong.bat

echo "use ping to delay"
set SLEEP=ping 127.0.0.1 /n
echo %time%
%SLEEP% 30 > nul
echo %time%
:over
end

The main task of the script is to check whether the specified javaKittleProHuizong.exe exists, if it exists, jump directly to the end of the program; if it does not exist, call the copy of kittlePreHuizong.bat we created.

KitlePreHuizong. bat is an instruction to start kittle s.

Then create our kittlePreHuizong.bat batch program:

@echo off
setlocal
SET initialDir=%cd%
pushd %~dp0
SET STARTTITLE="KitchenPreHuizong"
SET SPOON_CONSOLE=1
call SpoonPreHuizong.bat -main org.pentaho.di.kitchen.Kitchen -initialDir "%initialDir%"\ %*
popd

This script calls our replicated spoon.bat copy:
SpoonPreHuizong.bat
In this copy, we only modify the quotation address of the variable.
if "%SPOON_CONSOLE%"=="1" set PENTAHO_JAVA=javaKittleProHuizong.exe

Then create a timed task in windows timed task and call the monitoring script regularly.

Although the monitoring dos can be kept on for a time delay and then check the process, it is not allowed to shut down the dos which is responsible for monitoring abnormally, so the timing task is safer.

Posted by snipesh0tz on Wed, 10 Jul 2019 14:52:09 -0700