aiida.tools.importexport.dbexport package

Provides export functionalities.

aiida.tools.importexport.dbexport.export(what, outfile='export_data.aiida.tar.gz', overwrite=False, silent=False, **kwargs)

Export the entries passed in the ‘what’ list to a file tree.

Parameters

• what (list) – a list of entity instances; they can belong to different models/entities.

• outfile (str) – the filename (possibly including the absolute path) of the file on which to export.

• overwrite (bool) – if True, overwrite the output file without asking, if it exists. If False, raise an ArchiveExportError if the output file already exists.

• silent (bool) – suppress prints.

• allowed_licenses (list) – List or function. If a list, then checks whether all licenses of Data nodes are in the list. If a function, then calls function for licenses of Data nodes expecting True if license is allowed, False otherwise.

• forbidden_licenses (list) – List or function. If a list, then checks whether all licenses of Data nodes are in the list. If a function, then calls function for licenses of Data nodes expecting True if license is allowed, False otherwise.

• include_comments (bool) – In-/exclude export of comments for given node(s) in what. Default: True, include comments in export (as well as relevant users).

• include_logs (bool) – In-/exclude export of logs for given node(s) in what. Default: True, include logs in export.

• kwargs – graph traversal rules. See aiida.common.links.GraphTraversalRules what rule names are toggleable and what the defaults are.

Raises

• ArchiveExportError – if there are any internal errors when exporting.

• LicensingException – if any node is licensed under forbidden license.

aiida.tools.importexport.dbexport.export_zip(what, outfile='testzip', overwrite=False, silent=False, use_compression=True, **kwargs)

Export in a zipped folder

Parameters

• what (list) – a list of entity instances; they can belong to different models/entities.

• outfile (str) – the filename (possibly including the absolute path) of the file on which to export.

• overwrite (bool) – if True, overwrite the output file without asking, if it exists. If False, raise an ArchiveExportError if the output file already exists.

• silent (bool) – suppress prints.

• use_compression (bool) – Whether or not to compress the zip file.

• allowed_licenses (list) – List or function. If a list, then checks whether all licenses of Data nodes are in the list. If a function, then calls function for licenses of Data nodes expecting True if license is allowed, False otherwise.

• forbidden_licenses (list) – List or function. If a list, then checks whether all licenses of Data nodes are in the list. If a function, then calls function for licenses of Data nodes expecting True if license is allowed, False otherwise.

• include_comments (bool) – In-/exclude export of comments for given node(s) in what. Default: True, include comments in export (as well as relevant users).

• include_logs (bool) – In-/exclude export of logs for given node(s) in what. Default: True, include logs in export.

• kwargs – graph traversal rules. See aiida.common.links.GraphTraversalRules what rule names are toggleable and what the defaults are.

Raises

• ArchiveExportError – if there are any internal errors when exporting.

• LicensingException – if any node is licensed under forbidden license.

Submodules

Utility functions for export of AiiDA entities

aiida.tools.importexport.dbexport.utils._retrieve_linked_nodes_query(current_node, input_type, output_type, direction, link_type_value)

Helper function for retrieve_linked_nodes()

A general QueryBuilder query, retrieving linked Nodes and returning link information and the found Nodes.

Parameters

• current_node (int) – The current Node’s PK.

• input_type (Data, ProcessNode.) – Source Node class for Link

• output_type (Data, ProcessNode.) – Target Node class for Link

• direction (str) – Link direction, must be either 'forward' or 'backward'.

• link_type_value (str) – A LinkType value, e.g. LinkType.RETURN.value.

Returns

Dictionary of link information to be used for the export archive and set of found Nodes.

aiida.tools.importexport.dbexport.utils.check_licenses(node_licenses, allowed_licenses, forbidden_licenses)

Check licenses

aiida.tools.importexport.dbexport.utils.check_process_nodes_sealed(nodes)

Check ProcessNode s are sealed Only sealed ProcessNode s may be exported.

Parameters

nodes (list, int) – ProcessNode s to be checked. Should be their PK(s).

Raises

ExportValidationError – if a ProcessNode is not sealed or nodes is not a list, set, or int.

aiida.tools.importexport.dbexport.utils.fill_in_query(partial_query, originating_entity_str, current_entity_str, tag_suffixes=None, entity_separator='_')

This function recursively constructs QueryBuilder queries that are needed for the SQLA export function. To manage to construct such queries, the relationship dictionary is consulted (which shows how to reference different AiiDA entities in QueryBuilder. To find the dependencies of the relationships of the exported data, the get_all_fields_info (which described the exported schema and its dependencies) is consulted.

aiida.tools.importexport.dbexport.utils.retrieve_linked_nodes(process_nodes, data_nodes, **kwargs)

Recursively retrieve linked Nodes and the links

The rules for recursively following links/edges in the provenance graph are as follows, where the Node types in bold symbolize the Node that is currently being exported, i.e., it is this Node onto which the Link in question has been found.

LinkType_Direction	From	To	Follow (default)	Togglable
INPUT_CALC_FORWARD	Data	CalculationNode	False	True
INPUT_CALC_BACKWARD	Data	CalculationNode	True	False
CREATE_FORWARD	CalculationNode	Data	True	False
CREATE_BACKWARD	CalculationNode	Data	True	True
RETURN_FORWARD	WorkflowNode	Data	True	False
RETURN_BACKWARD	WorkflowNode	Data	False	True
INPUT_WORK_FORWARD	Data	WorkflowNode	False	True
INPUT_WORK_BACKWARD	Data	WorkflowNode	True	False
CALL_CALC_FORWARD	WorkflowNode	CalculationNode	True	False
CALL_CALC_BACKWARD	WorkflowNode	CalculationNode	False	True
CALL_WORK_FORWARD	WorkflowNode	WorkflowNode	True	False
CALL_WORK_BACKWARD	WorkflowNode	WorkflowNode	False	True

Parameters

• process_nodes – Set of ProcessNode node PKs.

• data_nodes – Set of Data node PKs.

• input_calc_forward – Follow INPUT_CALC links in the forward direction (recursively).

• create_backward – Follow CREATE links in the backward direction (recursively).

• return_backward – Follow RETURN links in the backward direction (recursively).

• input_work_forward – Follow INPUT_WORK links in the forward direction (recursively

• call_calc_backward – Follow CALL_CALC links in the backward direction (recursively).

• call_work_backward – Follow CALL_WORK links in the backward direction (recursively).

Returns

Set of retrieved Nodes, list of links information, and updated dict of LINK_FLAGS.

Raises

ExportValidationError – if wrong or too many kwargs are given.

aiida.tools.importexport.dbexport.utils.serialize_dict(datadict, remove_fields=None, rename_fields=None, track_conversion=False)

Serialize the dict using the serialize_field function to serialize each field.

Parameters

• remove_fields –

  a list of strings. If a field with key inside the remove_fields list is found, it is removed from the dict.

  This is only used at level-0, no removal is possible at deeper levels.

• rename_fields –

  a dictionary in the format {"oldname": "newname"}.

  If the “oldname” key is found, it is replaced with the “newname” string in the output dictionary.

  This is only used at level-0, no renaming is possible at deeper levels.

• track_conversion – if True, a tuple is returned, where the first element is the serialized dictionary, and the second element is a dictionary with the information on the serialized fields.

aiida.tools.importexport.dbexport.utils.serialize_field(data, track_conversion=False)

Serialize a single field.

Todo

Generalize such that it the proper function is selected also during import

Export a zip-file.

class aiida.tools.importexport.dbexport.zip.MyWritingZipFile(zip_file, fname)

Bases: object

__dict__ = mappingproxy({'__module__': 'aiida.tools.importexport.dbexport.zip', '__init__': <function MyWritingZipFile.__init__>, 'open': <function MyWritingZipFile.open>, 'write': <function MyWritingZipFile.write>, 'close': <function MyWritingZipFile.close>, '__enter__': <function MyWritingZipFile.__enter__>, '__exit__': <function MyWritingZipFile.__exit__>, '__dict__': <attribute '__dict__' of 'MyWritingZipFile' objects>, '__weakref__': <attribute '__weakref__' of 'MyWritingZipFile' objects>, '__doc__': None})

__enter__()

__exit__(type, value, traceback)

__init__(zip_file, fname)

Initialize self. See help(type(self)) for accurate signature.

__module__ = 'aiida.tools.importexport.dbexport.zip'

__weakref__

list of weak references to the object (if defined)

close()

open()

write(data)

class aiida.tools.importexport.dbexport.zip.ZipFolder(zipfolder_or_fname, mode=None, subfolder='.', use_compression=True, allowZip64=True)

Bases: object

To improve: if zipfile is closed, do something (e.g. add explicit open method, rename open to openfile, set _zipfile to None, …)

__dict__ = mappingproxy({'__module__': 'aiida.tools.importexport.dbexport.zip', '__doc__': '\n To improve: if zipfile is closed, do something\n (e.g. add explicit open method, rename open to openfile,\n set _zipfile to None, ...)\n ', '__init__': <function ZipFolder.__init__>, '__enter__': <function ZipFolder.__enter__>, '__exit__': <function ZipFolder.__exit__>, 'close': <function ZipFolder.close>, 'pwd': <property object>, 'open': <function ZipFolder.open>, '_get_internal_path': <function ZipFolder._get_internal_path>, 'get_subfolder': <function ZipFolder.get_subfolder>, 'exists': <function ZipFolder.exists>, 'insert_path': <function ZipFolder.insert_path>, '__dict__': <attribute '__dict__' of 'ZipFolder' objects>, '__weakref__': <attribute '__weakref__' of 'ZipFolder' objects>})

__enter__()

__exit__(type, value, traceback)

__init__(zipfolder_or_fname, mode=None, subfolder='.', use_compression=True, allowZip64=True)

Parameters

• zipfolder_or_fname – either another ZipFolder instance, of which you want to get a subfolder, or a filename to create.

• mode – the file mode; see the zipfile.ZipFile docs for valid strings. Note: can be specified only if zipfolder_or_fname is a string (the filename to generate)

• subfolder – the subfolder that specified the “current working directory” in the zip file. If zipfolder_or_fname is a ZipFolder, subfolder is a relative path from zipfolder_or_fname.subfolder

• use_compression – either True, to compress files in the Zip, or False if you just want to pack them together without compressing. It is ignored if zipfolder_or_fname is a ZipFolder isntance.

__module__ = 'aiida.tools.importexport.dbexport.zip'

__weakref__

list of weak references to the object (if defined)

_get_internal_path(filename)

close()

exists(path)

Check whether path already exists in the ZipFolder

get_subfolder(subfolder, create=False, reset_limit=False)

insert_path(src, dest_name=None, overwrite=True)

open(fname, mode='r')

property pwd