Beginners guide - on YAML syntax example

Where to place files

.rkd directory must always exists in your project. Inside .rkd directory you should place your makefile.yaml that will contain all of the required tasks.

Just like in UNIX/Linux, and just like in Python - there is an environment variable RKD_PATH that allows to define multiple paths to .rkd directories placed in other places - for example outside of your project. This gives a flexibility and possibility to build system-wide tools installable via Python’s PIP.

Environment variables

RKD natively reads .env (called also “dot-env files”) at startup. You can define default environment values in .env, or in other .env-some-name files that can be included in env_files section of the YAML.

Scope of environment variables

env_files and environment blocks can be defined globally, which will end in including that fact in each task, second possibility is to define those blocks per task. Having both global and per-task block merges those values together and makes per-task more important.

Example

version: org.riotkit.rkd/yaml/v1
environment:
    PYTHONPATH: "/project"
tasks:
    :hello:
        description: Prints variables
        environment:
            SOME_VAR: "HELLO"
        steps: |
            echo "SOME_VAR is ${SOME_VAR}, PYTHONPATH is ${PYTHONPATH}"

Arguments parsing

Arguments parsing is a strong side of RKD. Each task has it’s own argument parsing, it’s own generated –help command. Python’s argparse library is used, so Python programmers should feel like in home.

Example

version: org.riotkit.rkd/yaml/v1
environment:
    PYTHONPATH: "/project"
tasks:
    :hello:
        description: Prints your name
        arguments:
            "--name":
                required: true
                #option: store_true # for booleans/flags
                #default: "Unknown" # for default values
        steps: |
            echo "Hello ${ARG_NAME}"
rkd :hello --name Peter

Defining tasks in Python code

Defining tasks in Python gives wider possibilities - to access Python’s libraries, better handle errors, write less tricky code. RKD has a similar concept to hashbangs in UNIX/Linux.

There are two supported hashbangs + no hashbang:

  • #!python
  • #!bash
  • (just none there)

What can I do in such Python code? Everything! Import, print messages, execute shell commands, everything.

Example

version: org.riotkit.rkd/yaml/v1
environment:
    PYTHONPATH: "/project"
tasks:
    :hello:
        description: Prints your name
        arguments:
            "--name":
                required: true
                #option: store_true # for booleans/flags
                #default: "Unknown" # for default values
        steps: |
            #!python
            print('Hello %s' % ctx.get_arg('--name'))

Special variables

  • this - instance of current TaskInterface implementation
  • ctx - instance of ExecutionContext

Please check Tasks API for those classes reference.

YAML syntax reference

Let’s at the beginning start from analyzing an example.

version: org.riotkit.rkd/yaml/v1

# optional: Import tasks from Python packages
# This gives a possibility to publish tasks and share across projects, teams, organizations
imports:
    - rkt_utils.db.WaitForDatabaseTask

# optional environment section would append those variables to all tasks
# of course the tasks can overwrite those values in per-task syntax
environment:
    PYTHONPATH: "/project/src"

# optional env files loaded there would append loaded variables to all tasks
# of course the tasks can overwrite those values in per-task syntax
#env_files:
#    - .some-dotenv-file

tasks:
    :check-is-using-linux:
        description: Are you using Linux?
        # use sudo to become a other user, optional
        become: root
        steps:
            # steps can be defined as single step, or multiple steps
            # each step can be in a different language
            # each step can be a multiline string
            - "[[ $(uname -s) == \"Linux\" ]] && echo \"You are using Linux, cool\""
            - echo "step 2"
            - |
                #!python
                print('Step 3')

    :hello:
        description: Say hello
        arguments:
            "--name":
                help: "Your name"
                required: true
                #default: "Peter"
                #option: "store_true" # for booleans
        steps: |
            echo "Hello ${ARG_NAME}"

            if [[ $(uname -s) == "Linux" ]]; then
                echo "You are a Linux user"
            fi

imports - Imports external tasks installed via Python’ PIP. That’s the way to easily share code across projects

environment - Can define default values for environment variables. Environment section can be defined for all tasks, or per task

env_files - Includes .env files, can be used also per task

tasks - List of available tasks, each task has a name, descripton, list of steps (or a single step), arguments

Running the example:

  1. Create a .rkd directory
  2. Create .rkd/makefile.yaml file
  3. Paste/rewrite the example into the .rkd/makefile.yaml
  4. Run rkd :tasks from the directory where the .rkd directory is placed
  5. Run defined tasks rkd :hello :check-is-using-linux

Example projects using Makefile YAML syntax:

Extended usage - Makefile in Python syntax

Not only tasks can be written in Python code, but Makefile too - such makefile is called makefile.py, and placed in .rkd directory.

Example:

import os
from rkd.api.syntax import TaskAliasDeclaration as Task  # RKD API (for defining shortcuts/aliases for whole tasks lists)
from rkd.api.syntax import TaskDeclaration               # RKD API (for declaring usage of given task, importing it)
from rkd.api.contract import ExecutionContext            # RKD API (one of dependencies - context gives us access to commandline arguments and environment variables)
from rkd_python import imports as PythonImports          # group of imports (not all packages supports it, but most of them)
from rkd.standardlib.jinja import FileRendererTask       # single task
from rkd.standardlib import CallableTask                 # Basic Python callable task for a little bit advanced usage
# from .mypackage import MyTask                          # import your task from local package


def example_method(ctx: ExecutionContext, task: CallableTask) -> bool:
    os.system('xdg-open https://twitter.com/wrkclasshistory')
    return True


IMPORTS = [
    # We declare that we will use this task.
    # Declaration can take some additional arguments like args= or env=, to always append environment and/or commandline switches
    # regardless of if user used it
    TaskDeclaration(FileRendererTask()),
    # remember about the "," between tasks, it's an array/list ;)
    # TaskDeclaration(MyTask())

    TaskDeclaration(CallableTask(':read-real-history', example_method, description='Example task with simple Python code'))
]

IMPORTS += PythonImports()

TASKS = [
    # declared task-aliases. A Task Alias is a shortcut eg. ":release" that will expands to ":py:build :py:publish --username= (...)"
    # the best feature in task-aliases is that you can append and overwrite last commandline arguments, those will be added
    # at the end of the command
    Task(':release', description='Release to PyPI (snapshot when on master, release on tag)',
         to_execute=[
            ':py:build', ':py:publish', '--username=__token__', '--password=${PYPI_TOKEN}'
         ]),

    Task(':test', [':py:unittest'], description='Run unit tests'),
    Task(':docs', [':sh', '-c', ''' set -x
        cd docs
        rm -rf build
        sphinx-build -M html "source" "build"
    '''])
]
  • The Python syntax is very flexible
  • You can create your own local packages and import them here, create own advanced structure
  • Possibility to declare aliases and adjust TaskDeclarations for advanced usage (YAML syntax does not offer this)

Example projects using Makefile.py syntax:

Check ADVANCED usage page for description of all environment variables, mechanisms, good practices and more