Python scripting

Historically, the walt command was the only way to interact with a WALT platform. So, to automate experiments, users had to resort to shell scripts. Since then, a handy python scripting feature was introduced with WALT 8.0. This post proposes a short overview of this feature.

You can get more details by looking at the related documentation pages.

Python setup

If not already in a python virtual environment, we should initialize one and activate it:

~$ mkdir experiment && cd experiment
~/experiment$ python3 -m venv .venv && source .venv/bin/activate
(.venv) ~/experiment$ pip install --upgrade pip  # just in case
(.venv) ~/experiment$

For more information about virtual environments, checkout the official python documentation.

Then we can install the walt-client python package:

(.venv) ~/experiment$ pip install walt-client

This will install both the walt command line tool and the python scripting features in the virtual environment. If you did not use walt on this machine yet, you will be asked for configuration (i.e., IP or hostname of the WALT server and credentials for remote registries) when you start to use the command line tool or to execute python scripts.

An exploratory tour of the python API

The python API entrypoint can be imported by using the following statement:

from walt.client import api

Then you can easily explore the API features using an interactive python shell, because objects returned by the python API are self-descriptive¹: if you display such an object it will print its attributes, methods, and possibly any sub-objects.

For instance, in the following terminal recording, an object representing the node named rpi2b will be printed, with all its attributes (hardware model, IP address, WALT image currently booted, etc.) and methods (reboot(), wait(), etc.). Earlier in the recording, another object representing the set of all nodes available on the platform will also be shown.²

In this recording, I am actually checking how to toggle the netsetup attribute of a node. This attribute has two possible values, LAN and NAT³.

As you can see the API is very responsive and therefore well suited for this kind of exploratory session.

After this test, I can quickly draft a working script for the feature I was looking for:

#!/usr/bin/env python
import sys
from walt.client import api

if len(sys.argv) < 2:
    sys.exit(f"Usage: {sys.argv[0]} <node-name>")
node_name = sys.argv[1]

nodes = api.nodes.get_nodes()
if node_name not in nodes:
    sys.exit(f"No such node '{sys.argv[1]}'")

node = nodes[node_name]
if node.config.netsetup == 'NAT':
    print('NAT -> LAN')
    node.config.netsetup = 'LAN'
else:
    print('LAN -> NAT')
    node.config.netsetup = 'NAT'
node.reboot()

And this works as expected:

(.venv) ~/experiment$ python toggle-netsetup.py rpi2b
LAN -> NAT
Done. Reboot node(s) to see new settings in effect.
Node rpi2b: rebooted ok.
(.venv) ~/experiment$

Note about sub-items shortcuts and interactive completion

When running an interactive session, it can be handy to use autocompletion of object attributes using the <TAB> key.

However, the <object>[<sub-item>] notation breaks this autocompletion. For instance typing nodes['rpi2b'].<TAB><TAB> will fail to list the attributes of node rpi2b.

For this reason, object sets (such as variable nodes in the terminal recording) also provide access to sub-items using point-based notation. In our example, one can for example type nodes.rpi2b instead of nodes['rpi2b'], and in this case <TAB>-completion works.

If a sub-item name contains characters forbidden in an attribute (such as a dash), they will be replaced by underscores.

API features

When designing the python API, we focused on experiment automation features first.

So we first implemented the features to explore the WALT platform (i.e., inspect nodes and OS images available) and to work with nodes, such as rebooting (see previous example), changing configuration parameters (see previous example), letting them boot a given OS image, etc. For the same reason, log retrieval and synchronisation features were also part of the first features we implemented. Finally, we also implemented creation and removal of virtual nodes, making it easy to write fully self-contained experiment scripts when no specific hardware is needed.

Over time we started to implement other features in the API, to be able to automate other tasks, not only the experiment itself. For instance we implemented api.images.build() and api.images.clone(), the API counterparts of walt image build and walt image clone.

We will continue to implement more features, including porting features only available through the walt command line tool at the moment, as user needs arise. For instance, we may implement an API counterpart for walt image search in a near future to automate advanced OS deployment workflows involving several image repositories⁴.

For more information, and if exploring the API interactively is not enough, the features currently available are fully documented in dedicated documentation pages.

Last words

We hope this post has inspired you to explore the python API further.

Thanks to Jérémie Finiel for proofreading. If you have questions, we can answer you on the mailing list.