Serverless engineering practice | self built Apache opentask platform

Keywords: Apache serverless Open Source

Author Liu Yu (Jiang Yu)

Opentask is an open source, serverless cloud platform. It can respond to various events by executing extended code in the runtime container without users' concern for the relevant infrastructure.

Introduction to opentask

Opentask is a cloud based distributed event driven programming service. Opentask provides a programming model that registers event handlers with cloud services to handle a variety of different services. It can support thousands of triggers and calls, and can respond to events of different sizes.

Openwhite is built by many components, which make openwhite an excellent open source FaaS platform.


Apache opentask component structure

Opentask deployment

The operating system of the experimental machine is Ubuntu 18.04 Desktop. Use the initiator openwhisk provided on GitHub for installation. If Git is not installed on this machine, you need to install git first:

apt install git

Next, clone the repo to the local directory:

git clone https://github.com/apache/incubator-openwhisk.git openwhisk

After cloning is complete, the display is as shown in the figure.


Apache OpenWhisk project Clone

Enter the opentask directory and execute the script. Opentask is developed by Scala, and the Java environment needs to be installed to run. The following script implements the installation of Java environment and other required software:

cd openwhisk && cd tools/ubuntu-setup && ./all.sh

The Apache opentask installation configuration is shown in the figure.


Apache opentask installation configuration

Opentask is deployed using ansible, and the environment variable is defined in ansible / environments / group_ Under vars / all:

limits:
invocationsPerMinute: "{{ limit_invocations_per_minute | default(60) }}"
concurrentInvocations: "{{ limit_invocations_concurrent | default(30) }}"
concurrentInvocationsSystem:  "{{ limit_invocations_concurrent_system | default
    (5000) }}"
firesPerMinute: "{{ limit_fires_per_minute | default(60) }}"
sequenceMaxLength: "{{ limit_sequence_max_length | default(50) }}"

The above program defines the limitations of opentask in the system.

  • invocationsPerMinute indicates the number of actions called per minute in the same Namespace.
  • Concurrent invocations indicates the number of concurrent calls in the same Namespace.
  • Concurrent invocations system indicates the number of concurrent calls to all namespaces in the system.
  • firesPerMinute indicates the number of Trigger calls per minute in the same Namespace.
  • sequenceMaxLength indicates the maximum sequence length of an Action.

If you need to modify the above default values, you can add the modified values to the file ansible / environments / local / group_ End of vars / all. For example, if the maximum sequence length of an Action is 100, you can add sequenceMaxLength: 120 to the end of the file.

Next, configure a persistent storage database for opentask, with CouchDB and Cloudant options. Taking CouchDB as an example, configure the environment:

export OW_DB=CouchDB
export OW_DB_USERNAME=root
export OW_DB_PASSWORD=PASSWORD
export OW_DB_PROTOCOL=http
export OW_DB_HOST=172.17.0.1
export OW_DB_PORT=5984

Run the script in the openwhisk/ansible directory, as shown in the figure.

ansible-playbook -i environments/local/ setup.yml


Execute script process

Next, deploy opentask using CouchDB to ensure that there is already a local database_ local.ini. Execute the deployment command in the openwhisk / Directory:

./gradlew distDocker

If a problem occurs during deployment (as shown in the figure below), it may be caused by not installing npm. At this time, you can execute the following instructions.


Examples of possible errors during deployment

apt install npm

After a moment, you can see the Build success page, as shown in the figure.


Build success example

Next, enter the openwhisk/ansible Directory:

ansible-playbook -i environments/local/ couchdb.yml
ansible-playbook -i environments/local/ initdb.yml
ansible-playbook -i environments/local/ wipe.yml
ansible-playbook -i environments/local/ apigateway.yml
ansible-playbook -i environments/local/ openwhisk.yml
ansible-playbook -i environments/local/ postdeploy.yml

Execute the script as shown in the figure.


Picture execution script process

After successful deployment, opentask will start several Docker containers in the system. We can view it through docker ps:

docker ps --format "{{.Image}} \t {{.Names }}"

The list of containers after successful installation is shown in the figure.


List of containers after successful installation

Developer Tools

Opentask provides a unified command line interface wsk. The generated wsk is in openwhisk/bin. It has two properties that need to be configured.

  • API host the API used to deploy the hostname or IP address of openwhite.
  • The Authorization key (user name or password) is used to authorize the API to operate opentask.

Set API host, and the IP in the stand-alone configuration should be 172.17.0.1, as shown in the figure.

./bin/wsk property set --apihost '172.17.0.1'


Set API host

Set key:

./bin/wsk property set --auth `cat ansible/files/auth.guest

Permission settings are shown in the figure.


Picture setting permissions

Opentask stores CLI configuration information in ~ /. wskprops. The location of this file can also be through the environment variable WSK_CONFIG_FILE.

Verify CLI:

wsk action invoke /whisk.system/utils/echo –p message hello –result
{
    "message": "hello"
}

Experience test

Create a simple action with the following code:

# test.py
def main(args):
    num = args.get("number", "30")
    return {"fibonacci": F(int(num))}
def F(n):
    if n == 0:
        return 0
    elif n == 1:
        return 1
    else:
        return F(n - 1) + F(n - 2)

Create action:

/bin/wsk action create myfunction ./test.py  --insecure

The function creation is shown in the figure.

Create function

Trigger action:

./bin/wsk -i action invoke myfunction --result --blocking --param nember 20

The results are as shown in the figure.


Execution function

So far, we have completed the deployment and testing of the opentask project.

Posted by jboku on Mon, 01 Nov 2021 06:58:55 -0700