Beginners guide - on YAML syntax example¶
Where to place files¶
.rkd
directory must always exists in your project. Inside .rkd
directory you should place your makefile.yaml that will contain
all of the required tasks.
Just like in UNIX/Linux, and just like in Python - there is an environment variable RKD_PATH
that allows to define
multiple paths to .rkd
directories placed in other places - for example outside of your project. This gives a flexibility and possibility
to build system-wide tools installable via Python’s PIP.
Environment variables¶
RKD natively reads .env (called also “dot-env files”) at startup. You can define default environment values in .env, or in other .env-some-name files
that can be included in env_files
section of the YAML.
Scope of environment variables
env_files
and environment
blocks can be defined globally, which will end in including that fact in each task, second possibility is to
define those blocks per task. Having both global and per-task block merges those values together and makes per-task more important.
Example
version: org.riotkit.rkd/yaml/v1
environment:
PYTHONPATH: "/project"
tasks:
:hello:
description: Prints variables
environment:
SOME_VAR: "HELLO"
steps: |
echo "SOME_VAR is ${SOME_VAR}, PYTHONPATH is ${PYTHONPATH}"
Arguments parsing¶
Arguments parsing is a strong side of RKD. Each task has it’s own argument parsing, it’s own generated –help command. Python’s argparse library is used, so Python programmers should feel like in home.
Example
version: org.riotkit.rkd/yaml/v1
environment:
PYTHONPATH: "/project"
tasks:
:hello:
description: Prints your name
arguments:
"--name":
required: true
#option: store_true # for booleans/flags
#default: "Unknown" # for default values
steps: |
echo "Hello ${ARG_NAME}"
rkd :hello --name Peter
Defining tasks in Python code¶
Defining tasks in Python gives wider possibilities - to access Python’s libraries, better handle errors, write less tricky code. RKD has a similar concept to hashbangs in UNIX/Linux.
There are two supported hashbangs + no hashbang:
- #!python
- #!bash
- (just none there)
What can I do in such Python code? Everything! Import, print messages, execute shell commands, everything.
Example
version: org.riotkit.rkd/yaml/v1
environment:
PYTHONPATH: "/project"
tasks:
:hello:
description: Prints your name
arguments:
"--name":
required: true
#option: store_true # for booleans/flags
#default: "Unknown" # for default values
steps: |
#!python
print('Hello %s' % ctx.get_arg('--name'))
Special variables
- this - instance of current TaskInterface implementation
- ctx - instance of ExecutionContext
Please check Tasks API for those classes reference.
YAML syntax reference¶
Let’s at the beginning start from analyzing an example.
version: org.riotkit.rkd/yaml/v1
# optional: Import tasks from Python packages
# This gives a possibility to publish tasks and share across projects, teams, organizations
imports:
- rkt_utils.db.WaitForDatabaseTask
# optional environment section would append those variables to all tasks
# of course the tasks can overwrite those values in per-task syntax
environment:
PYTHONPATH: "/project/src"
# optional env files loaded there would append loaded variables to all tasks
# of course the tasks can overwrite those values in per-task syntax
#env_files:
# - .some-dotenv-file
tasks:
:check-is-using-linux:
description: Are you using Linux?
# use sudo to become a other user, optional
become: root
steps:
# steps can be defined as single step, or multiple steps
# each step can be in a different language
# each step can be a multiline string
- "[[ $(uname -s) == \"Linux\" ]] && echo \"You are using Linux, cool\""
- echo "step 2"
- |
#!python
print('Step 3')
:hello:
description: Say hello
arguments:
"--name":
help: "Your name"
required: true
#default: "Peter"
#option: "store_true" # for booleans
steps: |
echo "Hello ${ARG_NAME}"
if [[ $(uname -s) == "Linux" ]]; then
echo "You are a Linux user"
fi
imports - Imports external tasks installed via Python’ PIP. That’s the way to easily share code across projects
environment - Can define default values for environment variables. Environment section can be defined for all tasks, or per task
env_files - Includes .env files, can be used also per task
tasks - List of available tasks, each task has a name, descripton, list of steps (or a single step), arguments
Running the example:
- Create a .rkd directory
- Create .rkd/makefile.yaml file
- Paste/rewrite the example into the .rkd/makefile.yaml
- Run
rkd :tasks
from the directory where the .rkd directory is placed - Run defined tasks
rkd :hello :check-is-using-linux
Extended usage - Makefile in Python syntax¶
Not only tasks can be written in Python code, but Makefile too - such makefile is called makefile.py
, and placed in .rkd
directory.
Example:
import os
from rkd.api.syntax import TaskAliasDeclaration as Task # RKD API (for defining shortcuts/aliases for whole tasks lists)
from rkd.api.syntax import TaskDeclaration # RKD API (for declaring usage of given task, importing it)
from rkd.api.contract import ExecutionContext # RKD API (one of dependencies - context gives us access to commandline arguments and environment variables)
from rkd_python import imports as PythonImports # group of imports (not all packages supports it, but most of them)
from rkd.standardlib.jinja import FileRendererTask # single task
from rkd.standardlib import CallableTask # Basic Python callable task for a little bit advanced usage
# from .mypackage import MyTask # import your task from local package
def example_method(ctx: ExecutionContext, task: CallableTask) -> bool:
os.system('xdg-open https://twitter.com/wrkclasshistory')
return True
IMPORTS = [
# We declare that we will use this task.
# Declaration can take some additional arguments like args= or env=, to always append environment and/or commandline switches
# regardless of if user used it
TaskDeclaration(FileRendererTask()),
# remember about the "," between tasks, it's an array/list ;)
# TaskDeclaration(MyTask())
TaskDeclaration(CallableTask(':read-real-history', example_method, description='Example task with simple Python code'))
]
IMPORTS += PythonImports()
TASKS = [
# declared task-aliases. A Task Alias is a shortcut eg. ":release" that will expands to ":py:build :py:publish --username= (...)"
# the best feature in task-aliases is that you can append and overwrite last commandline arguments, those will be added
# at the end of the command
Task(':release', description='Release to PyPI (snapshot when on master, release on tag)',
to_execute=[
':py:build', ':py:publish', '--username=__token__', '--password=${PYPI_TOKEN}'
]),
Task(':test', [':py:unittest'], description='Run unit tests'),
Task(':docs', [':sh', '-c', ''' set -x
cd docs
rm -rf build
sphinx-build -M html "source" "build"
'''])
]
- The Python syntax is very flexible
- You can create your own local packages and import them here, create own advanced structure
- Possibility to declare aliases and adjust TaskDeclarations for advanced usage (YAML syntax does not offer this)