Writing reusable tasks

There are different ways to achieve similar goal, to define the Task. In chapter about Syntax you can learn differences between those multiple ways.

Now we will focus on Classic Python syntax which allows to define Tasks as classes, those classes can be packaged into Python packages and reused across projects and event organizations.

Importing packages

Everytime a new project is created there is no need to duplicate same solutions over and over again. Even in simplest makefiles there are ready-to-use tasks from rkd.core.standardlib imported and used.

version: org.riotkit.rkd/yaml/v2
imports:
    - my_org.my_package1

Package index

A makefile can import a class or whole package. There is no any automatic class discovery, every package exports what was intended to export.

Below is explained how does it work that Makefile can import multiple tasks from my_org.my_package1 without specifying classes one-by-one.

Example package structure

my_package1/
my_package1/__init__.py
my_package1/script.py
my_package1/composer.py

Example __init__.py inside Python package e.g. my_org.my_package1

from rkd.core.api.syntax import TaskDeclaration
from .composer import ComposerIntegrationTask                 # (1)
from .script import PhpScriptTask, imports as script_imports  # (2)


# (3)
def imports():
    return [
        TaskDeclaration(ComposerIntegrationTask()) # (5)
    ] + script_imports()  # (4)

• (1): ComposerIntegrationTask was imported from composer.py file

• (2): imports as script_imports other def imports() from script.py was loaded and used in (4)

• (3): def imports() defines which tasks will appear automatically in your build, when you import whole module, not a single class

• (5): TaskDeclaration can decide about custom task name, custom working directory, if the task is internal which means - if should be listed on :tasks

Task construction

Basic example of how the Task looks

class GetEnvTask(TaskInterface):
    """Gets environment variable value"""

    def get_name(self) -> str:
        return ':get'

    def get_group_name(self) -> str:
        return ':env'

    def configure_argparse(self, parser: ArgumentParser):
        parser.add_argument('--name', '-e', help='Environment variable name', required=True)

    def execute(self, context: ExecutionContext) -> bool:
        self.io().out(os.getenv(context.get_arg('--name'), ''))

        return True

Basic configuration methods to implement

• get_name(): Define a name e.g. :my-task

• get_group_name(): Optionally a group name e.g. :app1

• get_declared_envs(): List of allowed environment variables to be used inside of this Task

• configure_argparse(): Commandline switches configuration, uses Python’s native ArgParse

• get_configuration_attributes() : Optionally. If our Task is designed to be used as Base Task of other Task, then there we can limit which methods and class attributes can be called from configure() method

class rkd.core.api.contract. TaskInterface

abstract configure_argparse ( parser : argparse.ArgumentParser )

Allows a task to configure ArgumentParser (argparse)

def configure_argparse(self, parser: ArgumentParser):
    parser.add_argument('--php', help='PHP version ("php" docker image tag)', default='8.0-alpine')
    parser.add_argument('--image', help='Docker image name', default='php')

classmethod get_declared_envs ( ) → Dict [ str , Union [ str , rkd.core.api.contract.ArgumentEnv ] ]

Dictionary of allowed envs to override: KEY -> DEFAULT VALUE

All environment variables fetched from the ExecutionContext needs to be defined there. Declared values there are automatically documented in –help

@classmethod
def get_declared_envs(cls) -> Dict[str, Union[str, ArgumentEnv]]:
    return {
        'PHP': ArgumentEnv('PHP', '--php', '8.0-alpine'),
        'IMAGE': ArgumentEnv('IMAGE', '--image', 'php')
    }

abstract get_group_name ( ) → str

Group name where the task belongs eg. “:publishing”, can be empty.

abstract get_name ( ) → str

Task name eg. “:sh”

Basic action methods

• execute(): Contains the Task logic, there is access to environment variables, commandline switches and class attributes

• inner_execute(): If you want to create a Base Task, then implement a call to this method inside execute() , so the Task that extends your Base Task can inject code inside execute() you defined

• configure(): If our Task extends other Task, then there is a possibility to configure Base Task in this method

• compile(): Code that will execute on compilation stage. There is an access to CompilationLifecycleEvent which allows several operations such as task expansion (converting current task into a Pipeline with dynamically created Tasks)

class rkd.core.api.contract. ExtendableTaskInterface

compile ( event : CompilationLifecycleEvent ) → None

Execute code after all tasks were collected into a single context

configure ( event : ConfigurationLifecycleEvent ) → None

Executes before all tasks are executed. ORDER DOES NOT MATTER, can be executed in parallel.

abstract execute ( context : rkd.core.api.contract.ExecutionContext ) → bool

Executes a task. True/False should be returned as return

inner_execute ( ctx : rkd.core.api.contract.ExecutionContext ) → bool

Method that can be executed inside execute() - if implemented.

Use cases:

• Allow child Task to inject code between e.g. database startup and database shutdown to execute some operations on the database

Parameters

ctx –

Returns

Additional methods that can be called inside execute() and inner_execute()

• io(): Provides logging inside execute() and configure()

• rkd() and sh(): Executes commands in subshells

• py(): Executes Python code isolated in a subshell

class rkd.core.api.contract. ExtendableTaskInterface

io ( ) → rkd.core.api.inputoutput.IO

Gives access to Input/Output object

py ( code : str = '' , become : Optional [ str ] = None , capture : bool = False , script_path : Optional [ str ] = None , arguments : str = '' ) → Optional [ str ]

Executes a Python code in a separate process

NOTICE: Use instead of subprocess. Raw subprocess is less supported and output from raw subprocess

may be not catch properly into the logs

rkd ( args : list , verbose : bool = False , capture : bool = False ) → str

Spawns an RKD subprocess

NOTICE: Use instead of subprocess. Raw subprocess is less supported and output from raw subprocess

may be not catch properly into the logs

sh ( cmd : str , capture : bool = False , verbose : bool = False , strict : bool = True , env : Optional [ dict ] = None , use_subprocess : bool = False ) → Optional [ str ]

Executes a shell script in bash. Throws exception on error. To capture output set capture=True

NOTICE: Use instead of subprocess. Raw subprocess is less supported and output from raw subprocess

may be not catch properly into the logs