I. Introduction

In engineering practice, it is common to design Cron Protect scripts to monitor task execution when a task fails or fails for other reasons, but it will not work if the task execution times out and Cron Protect does not exit. Kill will be forced at this timeDrop the task and restart it, but every time you need to log in to the server, it is very troublesome. At this time, you can start the program automatically by designing the interface + listener program, only by accessing the interface of your mobile phone.

2. Practice

After the analysis above, the function of this monitoring mainly involves monitoring interface parameters and implementing startup tasks, which requires the following three tools:

=>Listening status: Redis

The latest values passed in by the real-time monitoring interface, implemented using Redis queues

=>Interface:

Interfaces are implemented concisely through Python's Fast API

=>Start Task:

Python launches sh tasks through SubProcess

1. Listen Status + Start Task

Since the Redis listening state and the restart task are two consecutive steps, that is, to listen for the startup state and then perform the startup task operation, the two are implemented together, of course, without Redis, you can use any storage side to transfer state values.

#coding=utf-8
import redis
import subprocess
import datetime
import time

# Initialize Client
r = redis.Redis(host='your_host', port='your_port', decode_responses=True)
# Other Pipelines
# client = new MyClient(host, port)

# Queue Key
read_key = 'value_state'

def getValue():
    now = datetime.datetime.now()
    length = r.llen(read_key)
    # Keep only the latest and empty the queue
    r.ltrim(read_key,0,0)
    value = r.rpop(read_key)
    curLength = r.llen(read_key)
    print("time:%s value: %s oriLength:%s curLength:%s" % (now, value, length, curLength))
    if value == "self_defined":
        log = subprocess.Popen("nohup sh test.sh  > log 2>&1 &", stdout=subprocess.PIPE, stderr=subprocess.PIPE, shell=True)
        print("Application Started At:", now, log.communicate()[0].strip())

while(1):
    getValue()
    time.sleep(10)

The A.getValue() function takes care of getting the latest values from the Redis queue and clearing the redundant values from the current queue, so the r.ltrim operation is performed to prevent the interface from abnormally passing in too many status values and causing the task to restart multiple times

B.rpop returns the latest element, depending on the value of the element and the custom value, you can decide whether or not subprocess.Popen suspends its own task, where Popen is executed asynchronously, so it does not cause python programs to wait synchronously. If you need to listen for the status of the task being executed, you can modify the way nohup submits, so log.communi()Blocks the operation until test.sh finishes executing and determines the status of task execution

C. Use While True to implement the 7x24 service of the listening system, where sleep sets the interval for each listening

2. Interface

Pthon implements a simple interface via the FastAPI, which is convenient and efficient. The interface only needs to implement a basic Get function. It can pass parameters to Redis via the mobile phone, so that the listener can get the latest status values.

#!/usr/bin/python
# -*- coding: UTF-8 -*-

from fastapi import FastAPI
from redis import Redis

send = Redis(host='your_host', port='your_port')
app = FastAPI()


@app.get("/monitor/{value}")
async def send_info(value: str = None):
    send.lpush("value_state", value)
    return {"value": value}

Deploy to your own server via uvicorn or Netstat, reload updates the APP on behalf of updated code

# uvicorn Monitor:app --reload

Can be passed directly through the URL:

http://127.0.0.1:8000/monitor/self_difined

You can also enter an interactive interface to pass on parameters:

 http://127.0.0.1:8000/docs

 

3. Testing

Build a test script, test.sh:

#!/bin/bash

echo `date` Task Start...

Then nohup suspends the python program and deploys the interface on the server to execute the mobile control task, which is monitored every 10 s econds, and you can see the log of test.sh:

Thu Sep 16 10:21:04 CST 2021 Task Start...

10:21:04 coincides with the Application Start time of our interface log, queue empties after restart, continuous monitoring

Subsequent restart tasks no longer require login to the server, just login to the mobile phone to access:

(http://127.0.0.1:8000/monitor/self_difined You can complete the wake-up task at any time, very Ness