Installation and usage of irace

Since many of the functions of irace can only be implemented on GNU/Linux systems, it is recommended that you run this tuning tool on Linux, which is also described below for Linux.

Note: The irace runtime requires the R language runtime environment. The R language runtime environment dependency may conflict with older versions of gnome desktop systems, try to operate on newer versions of the Linux system, or on servers that do not have desktop systems.

1.Install the R language runtime environment

Using debian Linux as an example, install using the following command

sudo apt-get install r-base

If a pop-up window appears during installation, always choose yes. Restart the system after installation is complete. If you can enter the login interface smoothly, there is no problem.

If not, try the following command to restore the initial settings of the Linux desktop system.

dconf reset -f /

2.Install irace and add to environment variables

Enter the R language interaction environment, and the command line does the following

R

install.package("irace")

After installation, use the following command to output the irace installation path.

library("irace")
system.file(package = "irace")

Path to copy output

q()

Exit R Language Interactive Environment

Open environment variable profile with vim, nano, gedit, vscode and other tools

One of ~/.bash_profile, ~/.bashrc or ~/.profile

Append the following to the end of the file:

# Replace <IRACE_HOME> with the irace installation path
export IRACE_HOME=<IRACE_HOME>
export PATH=${IRACE_HOME}/bin/:$PATH
# Tell R where to find R_LIBS_USER
# Use the following line only if local installation was forced
export R_LIBS=${R_LIBS_USER}:${R_LIBS}

Where <irace_HOME>is modified to the installation path of the irace copied above.

Save after modifications, using the source directive to take effect immediately. (Examples with *.bashrc*)

source ~/.bashrc

Then enter irace --help on the command line to verify that the configuration is complete.

3.Project Configuration

A typical irace project configuration consists of the following components

• Executable for debugging algorithms
• target-runner, algorithm execution script
• scenario.txt, scene configuration file
• parameters.txt, parameter profile to debug
• /Instances folder, place test cases
• /arena folder, data storage folder where irace runs experiments
• (Optional) default.txt, default settings for debug parameters
• (optional) forbidden.txt, the prohibited range or condition of the parameter

3.1 Algorithmic Executable

Since irace splices command lines during parameter invocation, executables must have parametric parsing capabilities, such as

./numvc --i test.mtx --c 15 --s 234789434

3.2 target-runner Settings

Most of the target-runner content for the official template does not need to be moved, only the parts related to our project need to be modified

1. Modify algorithm executable path

# Path to the exe of your alghorithm:
EXE=/home/eric/repository/numvc_code/numvc

1. Fixed parameters on command line, parts that don't require adjustments, note that spaces are left before and after

FIXED_PARAMS=" --c 15 "

1. Core instructions for executing algorithms

# Now we can call exe by building a command line with all parameters for it
$EXE ${FIXED_PARAMS} --i $INSTANCE --s $SEED ${CONFIG_PARAMS} 1> $STDOUT 2> $STDERR

Specific parameter settings, according to their own algorithm to perform the required modifications.

1. Output result interception

COST=$(cat ${STDOUT} | grep -e '^[0-9.e]*$')

This directive intercepts the data from the last line of the executable output text, which is used to evaluate the quality of a set of parameter choices and to sort the last output parameters. It is recommended that only the key values used to determine the quality of parameter choices be output in the final output of the executable file, such as the time it takes for the program to find the best solution.

When irace evaluates a set of parameters, it considers that the smaller the target value, the better the set of parameters. If you want the larger the target value, the better, you need a result value of -1 at program output. *

1. Intermediate File Processing for Program Running

rm -f "${STDOUT}" "${STDERR}"

By default, the script deletes all intermediate files. If you want to keep intermediate file observations, change this line to:

rm -f "${STDERR}"

3.3 scenario.txt settings

Only key information is provided, and all settings are recommended for irace/templates/scenario.txt.tmpl

The irace runs until either maxExperiments or maxTime is reached, so you must configure one of them in scenario.txt.

## File that contains the description of the parameters.
## Parameter Settings File
parameterFile = "./parameters-numvc.txt"

## Directory where the programs will be run.
## irace workspace, the / arena folder mentioned above
execDir = "./numvc-arena1"

## Directory where tuning instances are located, either absolute path or
## relative to current directory.
## test case folder
trainInstancesDir = "./Instances"

## The maximum number of runs (invocations of targetRunner) that will performed. It
## determines the (maximum) budget of experiments for the tuning.
## Maximum number of experiments, need to be estimated by yourself
maxExperiments = 200

## Indicates the number of decimal places to be considered for the
## real parameters.
## When the parameter is a real number, the precision of the parameter, digits = 1, is exactly one decimal place after the decimal point
digits = 1

## Number of iterations.
## Number of experiment iterations
nbIterations = 2

3.4 parameters.txt parameter settings

Examples of parameter settings:

# name		switch		     type	values			[conditions (using R syntax)]
# nnls            "--nnls "            i  	(5, 50)			| localsearch %in% c(1, 2, 3)
# dlb             "--dlb "             c  	(0, 1) 			| localsearch %in% c(1,2,3)
optimal_size        "--o "       c       (0)
forget_scale        "--f "       r       (0.1,1.0)
threshold_scale     "--t "       r       (0.1,1.0)

The name column is used only for display in irace output.

The switch column is used for passing parameters, so the parse command line instructions set for the algorithm to be executed are followed by spaces to stitch parameters.

The type column is the parameter type, and the following are common parameter types:

• c, the classification type, which is a list with values ranging from all available values. In particular, it is used when there is only one fixed value for a parameter.
• i, Integer type, all integers in the range (lb, ub).
• r, real type, all real numbers whose values range from (lb, ub). The exact number of digits after the decimal point is determined by the digits field in scenario.txt, where digits = 1 is exactly one digit after the decimal point.

Values column is the range of values.

Conditions are value conditions that need to be written in R language syntax. This field is optional.

3.5 default.txt settings

default.txt is optional. Contents are the default settings for parameters within parameters.txt.

3.6 forbidden.txt settings

forbidden.txt is optional. Contents are prohibited ranges for parameters within parameters.txt.

4.Irace run

Once all the above are configured, you can try running irace.

The command line switches to the project directory and runs irace directly. irace looks for scenario.txt in the current directory, calls the scenario.txt configuration, and runs target-runner.

irace

This way, only one process will be consumed, and if you want to run multi-process parallel operations, you need to add the parallel parameter.

irace --parallel 4

This allows you to run four parallel threads, specifying the number of parallel programs, looking at the number of threads supported by the CPU, and then modifying them yourself.

Or add the following configuration to scenario.txt (this is recommended):

## Number is the number of threads you want to parallel
parallel = 10

View the number of CPU support threads by

# View number of CPU s
cat /proc/cpuinfo |grep "physical id"|sort|uniq|wc -l
# View the number of cores per physical CPU
cat /proc/cpuinfo |grep "cpu cores"|uniq|wc -l
# View the number of threads per CPU core
cat /proc/cpuinfo |grep "processor"|wc -l
# So the number of threads supported by the cpu*Number of cores per cpu*Number of threads per core

If you need to see the command line parameters supported by irace, you can use:

irace --help

The irace process outputs the appropriate parameters that have been calculated for each iteration, and when all experiments are completed, three optimal parameter results are output in descending order, that is, the desired results.

5. Error handling

1.Configuration issues

Error: == irace == target runner '~/tuning/target-runner' does not exist

No target-runner configured

2.Permission issues

Error: == irace == target runner '~/tuning/target-runner' is not executable

target-runner does not have executable permissions, modify permissions is sufficient

chmod 777 target-runner

Similarly, a prompt similar to the following appears

== irace == The output was:
Tue May 3 19:00:37 UTC 2016: error: ~/bin/acotsp: not found or not executable
(pwd: ~/tuning/acotsp-arena)

The build file of the program does not have permission to execute, just modify the permission.

3.Output Result String Interception Problem

Error: == irace == The output of '~/tuning/target-runner 1 25 365157769
~/tuning/Instances/1000-31.tsp --ras --localsearch 1 --alpha 0.26 --beta
6.95 --rho 0.69 --ants 56 --nnls 10 --dlb 0 --rasranks 7' is not numeric!
== irace == The output was:
Solution: 24479793

It is recommended that the output data have only one result, and there is only one criterion for the irace evaluation parameters. If there are multiple evaluation criteria such as weight and time-consuming, choose the most important one

6.Higher order operation

More advanced operations for irace, accessible Here An official guide book.