Automated parser tests¶
AiiDA testing facility can check for the proper functionality of parsers automatically. To facilitate the creation of new tests, we provide a simple tool to create a new parser test from a calculation that you already run in your AiiDA database, described below.
Test folders¶
Each folder inside the path aiida/djsite/db/subtests/parser_tests
constitutes a single test.
The naming convention for folders is the following:
- it should contain only digits, letters and underscores, otherwise the
folder will be ignored when running
verdi devel tests db.parsers
; - the folder name should start with
test_
; - the name should be followed by the parser plugin name, as returned by calculation.get_parser_name(), and with dots replaced with underscores;
- it should be followed by an underscore;
- finally it should be followed by a string that explains what is tested.
For instance, a valid name is test_quantumespresso_pw_vanderwaals
.
Note that the naming scheme is only a convention, and that the parser to
use for the test is selected automatically.
Creation of a test from an existing calculation¶
In order to create the folder, you can open verdi shell
while being
in the folder aiida/djsite/db/subtests/parser_tests
, import
the following function:
from aiida.djsite.db.subtests.parsers import output_test
and then run it with the correct parameters. The documentation of the function
can be found here
.
An example call could be:
output_test(
pk=21,
testname='vanderwaals',
skip_uuids_from_inputs=[
'f579974c-6a9e-4eb4-9b41-e72486f86ac5',
'ee0df234-955e-4f99-9808-17e168e6a769']
)
where:
21
is the PK of the calculation that you want to exportvanderwaals
is the name of the test: if for instance the node withpk=21
is a Quantum ESPRESSO pw.x calculation, the script will create a folder namedtest_quantumespresso_pw_vanderwaals
- the (optional)
skip_uuids_from_inputs
is a list of UUIDs of input nodes that will not be exported.
The script will create a new folder, containing the exported content of the calculation,
its direct inputs (except those listed in the
skip_uuids_from_inputs
list), and the output retrieved
node.
The format of the exported data is the same of the export files of
AiiDA, but the folder is not zipped.
Note
The skip_uuids_from_inputs
parameter is typically
useful for input nodes containing large files that are not
needed for parsing and would just create a large test; a
typical example is given by pseudopotential input nodes for
Quantum ESPRESSO.
After having run the command, the existence of the folder will only test that the parser is able to parse the calculation without errors. Typically, however, you will also want to check some parsed values.
In this case, you need to modify the _aiida_checks.json
JSON file inside the folder. The syntax is the following:
each key represents an output node that should be generated by the parser;
each value is a dictionary with multiple keys (an empty dictionary will just check for the existence of the output node);
each key of the subdictionary is an attribute to check for. The value is a list of dictionaries, one for each test to perform on the given value; multiple tests are therefore possible. The dictionary should have at least have one key: “comparison”, a string to specifies the type of comparison. The other keys depend on the type of comparison, and typically there is at least a “value” key, the value to compare with. An example:
{ "output_parameters": { "energy": [ { "comparison": "AlmostEqual", "value": -3699.26590536037 } ], "energy_units": [ { "comparison": "Equal", "value": "eV" } ] }, "output_array": { } }
The list of valid comparisons is hardcoded inside the
aiida.djsite.db.subtests.parsers
module;
if you need new comparison types, add them directly to the module.
Running tests¶
Finally, in order to run all tests contained in the folder aiida/djsite/db/subtests/parser_tests
one can use the following verdi
command:
verdi devel tests db.parsers
If no fail message appears it means that the test was successful.