Supervisor (http://supervisord.org ) is a process management tool written in Python that can be easily used to start, restart, and close processes (not just Python processes).In addition to controlling a single process, multiple processes can be started and shut down at the same time, such as when unfortunately the server is down and all applications are killed, you can start all applications at the same time with supervisor instead of tapping commands one by one.

install

Supervisor runs on Linux, Mac OS X.As mentioned earlier, the supervisor is written in Python, so it is easy to install and can be directly pip:

sudo pip install supervisor

If it's a Ubuntu system, you can also install it using apt-get.

supervisord configuration

Supervisor is quite powerful and provides a lot of functionality, but we may only need a small part of it.Once the installation is complete, you can write a configuration file to meet your needs.For convenience, we divide the configuration into two parts: supervisord (supervisor is a C/S model program, this is the server side, corresponding to the client side: supervisorctl) and the application (that is, the program we want to manage).

First, look at the supervisord configuration file.After installing the supervisor, you can either run the echo_supervisord_conf command to output the default configuration item or redirect it to a configuration file:

echo_supervisord_conf > /etc/supervisord.conf

After removing most of the comments and "irrelevant" sections, we can first look at these configurations:

[unix_http_server]
file=/tmp/supervisor.sock   ; UNIX socket Files, supervisorctl Will use
;chmod=0700                 ; socket File mode，Default is 0700
;chown=nobody:nogroup       ; socket File owner，Format: uid:gid

;[inet_http_server]         ; HTTP Server, Provide web Management interface
;port=127.0.0.1:9001        ; Web Manage Background Running IP And ports, if open to the public network, need to be aware of security
;username=user              ; Logon administration background user name
;password=123               ; Login Administration Background Password

[supervisord]
logfile=/tmp/supervisord.log ; Log file, default is $CWD/supervisord.log
logfile_maxbytes=50MB        ; Log file size, exceeds rotate，default 50MB
logfile_backups=10           ; Log Files Keep Backup Quantity Default 10
loglevel=info                ; Log level, default info，Other: debug,warn,trace
pidfile=/tmp/supervisord.pid ; pid file
nodaemon=false               ; Whether to start in the foreground, default is false，That is to say, daemon Way to start
minfds=1024                  ; Minimum value of file descriptors that can be opened, default 1024
minprocs=200                 ; Minimum number of processes that can be opened, default 200

; the below section must remain in the config file for RPC
; (supervisorctl/web interface) to work, additional interfaces may be
; added by defining them in separate rpcinterface: sections
[rpcinterface:supervisor]
supervisor.rpcinterface_factory = supervisor.rpcinterface:make_main_rpcinterface

[supervisorctl]
serverurl=unix:///tmp/supervisor.sock; connects supervisord through UNIX socket with path identical to file in unix_http_server section
;serverurl=http://127.0.0.1:9001; connect supervisord over HTTP

; Contains additional configuration files
[include]
files = relative/directory/*.ini    ; Can be *.conf or *.ini

We save the above allocation to/etc/supervisord.conf (or any other file with permission to access it), then start supervisord (specify the path to the configuration file through the -c option, if not specified it will look for the configuration file in this order: $CWD/supervisord.conf, $CWD/etc/supervisord.conf, /etc/supervisord.conf):

supervisord -c /etc/supervisord.conf

Check if supervisord is running:

ps aux | grep supervisord

program configuration

Now that we've run supervisrod above, we can add the configuration files for the processes we want to manage.All configuration items can be written to the supervisord.conf file, but this is not recommended. Instead, different programs (groups) can be written to different configuration files by include.

For example, let's create a new directory / etc/supervisor / to store these configuration files, and modify the configuration of the include section in / etc/supervisord.conf accordingly:

[include]
files = /etc/supervisor/*.conf

Suppose you have a user-centric system written with Python and Flask frameworks, named usercenter, and gunicorn ( http://gunicorn.org/ ) Make a web server.The project code is located in /home/leon/projects/usercenter, the gunicorn configuration file is gunicorn.py, and WSGI callable is the app attribute in wsgi.py.So starting directly from the command line might be like this:

cd /home/leon/projects/usercenter
gunicorn -c gunicorn.py wsgi:app

Now write a configuration file to manage the process (Note: gunicorn's daemon option needs to be set to False when managed with supervisord):

[program:usercenter]
directory = /home/leon/projects/usercenter ; Startup directory for programs
command = gunicorn -c gunicorn.py wsgi:app  ; Start command, you can see that it is the same as the command that was started manually on the command line
autostart = true     ; stay supervisord Start automatically at startup
startsecs = 5        ; start-up 5 No abnormal exit after seconds, as if it had started normally
autorestart = true   ; Automatic restart after abnormal program exit
startretries = 3     ; Startup failure automatic retries, default is 3
user = leon          ; Start with which user
redirect_stderr = true  ; hold stderr Redirect to stdout，default false
stdout_logfile_maxbytes = 20MB  ; stdout Log file size, default 50MB
stdout_logfile_backups = 20     ; stdout Number of log file backups
; stdout Log files, note that the specified directory does not start properly when it does not exist, so you need to create the directory manually. supervisord Will automatically create a log file)
stdout_logfile = /data/logs/usercenter_stdout.log

; By environment To add the required environment variables, a common use is to modify PYTHONPATH
; environment=PYTHONPATH=$PYTHONPATH:/path/to/somewhere

A configuration file requires at least one configuration in the [program:x] section to tell supervisor that the process needs to be managed.The X in [program:x] syntax represents the program name, which is displayed on the client (supervisorctl or web interface) and is used in supervisorctl to start, restart, stop, and so on.

A configuration example:

[program:api-cache-server]
command=python api_cache_server.py -c /root/sxadp-prod-conf
numprocs=1
directory=/root/api_trigger_script/
stdout_logfile=/var/log/api-cache-server/api-cache-server.log
stdout_logfile_maxbytes=10MB
stdout_logfile_backups=3
redirect_stderr=true
autostart=true
autorestart=true
user=root
stopasgroup=true

Use supervisorctl

Supervisorctl is a command-line client tool for supervisord, and you need to specify that you use the same profile as supervisord at startup, otherwise you will find the profiles in the same order as supervisord.

supervisorctl -c /etc/supervisord.conf

The above command enters the shell interface of supervisorctl, and then different commands can be executed:

> status #View program status
> stop usercenter #Close usercenter program
> start usercenter #Start usercenter program
> Restart usercenter #Restart usercenter program
> reread # Reads configuration files with updates (additions) and does not start newly added programs
> update Restart the program whose profile was modified

These commands have corresponding outputs, which can be run directly in the bash terminal in addition to entering the shell interface of supervisorctl:

$ supervisorctl status
$ supervisorctl stop usercenter
$ supervisorctl start usercenter
$ supervisorctl restart usercenter
$ supervisorctl reread
$ supervisorctl update 

Other

In addition to supervisorctl, you can configure supervisrod to launch a web management interface that uses Basic Auth for authentication in the web background.

In addition to the control of a single process, you can configure groups for group management.

Check the log files frequently, including those of supervisord and pragram. Half of the information that the program crash or throws an exception is output to stderr. You can look at the corresponding log files to find problems.

Supervisor has a rich set of features and many other configurations for more information in the official documentation: http://supervisord.org/index.html

Reference resources:

http://liyangliang.me/posts/2015/06/using-supervisor/