Transport plugins

The term transport in AiiDA refers to a class that the engine uses to perform operations on local or remote machines where its CalcJob are submitted. The base class Transport defines an interface for these operations, such as copying files and executing commands. A transport plugin is a class that implements this base class for a specific connection method. The aiida-core package ships with two transport plugins: the LocalTransport and SshTransport classes. The local transport can be used to connect with the localhost and makes use only of some standard python modules like os and shutil. The ssh transport, which can be used for machines that can be connected to over ssh, is simply a wrapper around the library paramiko that is installed as a required dependency of aiida-core.

Developing a plugin

The transport class is actually almost never used directly by the user. It is mostly utilized by the engine that uses the transport plugin to connect to the machine where the calculation job, that it is managing, is running. The engine has to be able to use always the same methods regardless of which kind of transport is required to connect to the computer in question.

The generic transport class contains a set of minimal methods that an implementation must support in order to be fully compatible with the other plugins. If not, a NotImplementedError will be raised, interrupting the managing of the calculation or whatever is using the transport plugin.

As for the general functioning of the plugin, the __init__() method is used only to initialize the class instance, without actually opening the transport channel. The connection must be opened only by the __enter__() method, (and closed by __exit__()). The __enter__() method lets you use the transport class using the with statement (see python docs), in a way similar to the following:

with TransportPlugin() as transport:
    transport.some_method()

To ensure this, for example, the local plugin uses a hidden boolean variable _is_open that is set when the __enter__() and __exit__() methods are called. The ssh logic is instead given by the property sftp.

The other functions that require some care are the copying functions, called using the following terminology:

1. put: from local source to remote destination

2. get: from remote source to local destination

3. copy: copying files from remote source to remote destination

Note that these functions must accept both files and folders and internally they will fallback to functions like putfile or puttree.

The last function requiring care is exec_command_wait(), which is an analogue to the subprocess python module. The function gives the freedom to execute a string as a remote command, thus it could produce nasty effects if not written with care.

Warning

Be sure to escape any strings for bash!

Download this template as a starting point to implementing a new transport plugin. It contains the interface with all the methods that need to be implemented, including docstrings that will work with Sphinx documentation.

Note

To inform AiiDA about your new transport plugin you must register an entry point in the aiida.transports entry point group. Please visit the AiiDA registry to see an example of how this can be done.

Login shells

The base transport class aiida.transports.transport.Transport defines the use_login_shell option. When set to True, all commands executed over the transport will use the -l/--login option of bash. This instructs bash to load a login shell, which, according to the bash manpage, means:

When bash is invoked as an interactive login shell, or as a non-interactive shell with the --login option, it first reads and executes commands from the file /etc/profile, if that file exists. After reading that file, it looks for ~/.bash_profile, ~/.bash_login, and ~/.profile, in that order, and reads and executes commands from the first one that exists and is readable.

By default use_login_shell is set to True as it ensures that the commands executed over the transport by AiiDA see the same shell environment as the user, if they were to login manually and execute the command. However, in certain cases, the login scripts might not be necessary for AiiDA to properly run codes on the target computer. At the same time, it is possible that the login scripts have a non-negligible run time, and so can significantly slow down all the commands AiiDA has to execute. In this case, it may be useful to set the use_login_shell to False.