aiida.backends.djsite.db.subtests.migrations package¶
Submodules¶
Tests for the migrations of the attributes, extras and settings from EAV to JSONB Migration 0037_attributes_extras_settings_json
-
class
aiida.backends.djsite.db.subtests.migrations.test_migrations_0037_attributes_extras_settings_json.
DbAttributeBaseClass
[source]¶ -
Abstract base class for tables storing element-attribute-value data. Element is the dbnode; attribute is the key name. Value is the specific value to store.
This table had different SQL columns to store different types of data, and a datatype field to know the actual datatype.
Moreover, this class unpacks dictionaries and lists when possible, so that it is possible to query inside recursive lists and dicts.
-
__module__
= 'aiida.backends.djsite.db.subtests.migrations.test_migrations_0037_attributes_extras_settings_json'¶
-
_subspecifier_field_name
= 'dbnode'¶
-
classmethod
set_value_for_node
(dbnode, key, value, with_transaction=True, stop_if_existing=False)[source]¶ This is the raw-level method that accesses the DB. No checks are done to prevent the user from (re)setting a valid key. To be used only internally.
- Todo
there may be some error on concurrent write; not checked in this unlucky case!
- Parameters
dbnode – the dbnode for which the attribute should be stored; if an integer is passed, it will raise, since this functionality is not supported in the models for the migrations.
key – the key of the attribute to store; must be a level-zero attribute (i.e., no separators in the key)
value – the value of the attribute to store
with_transaction – if True (default), do this within a transaction, so that nothing gets stored if a subitem cannot be created. Otherwise, if this parameter is False, no transaction management is performed.
stop_if_existing – if True, it will stop with an UniquenessError exception if the key already exists for the given node. Otherwise, it will first delete the old value, if existent. The use with True is useful if you want to use a given attribute as a “locking” value, e.g. to avoid to perform an action twice on the same node. Note that, if you are using transactions, you may get the error only when the transaction is committed.
- Raises
ValueError – if the key contains the separator symbol used internally to unpack dictionaries and lists (defined in cls._sep).
-
-
class
aiida.backends.djsite.db.subtests.migrations.test_migrations_0037_attributes_extras_settings_json.
DbAttributeFunctionality
[source]¶ -
This class defines all the methods that are needed for the correct deserialization of given attribute dictionaries to the EAV table. It is a stripped-down Django EAV schema to the absolutely necessary methods for this deserialization.
-
__module__
= 'aiida.backends.djsite.db.subtests.migrations.test_migrations_0037_attributes_extras_settings_json'¶
-
-
class
aiida.backends.djsite.db.subtests.migrations.test_migrations_0037_attributes_extras_settings_json.
DbExtraFunctionality
[source]¶ -
This class defines all the methods that are needed for the correct deserialization of given extras dictionaries to the EAV table. It is a stripped-down Django EAV schema to the absolutely necessary methods for this deserialization.
-
__module__
= 'aiida.backends.djsite.db.subtests.migrations.test_migrations_0037_attributes_extras_settings_json'¶
-
-
class
aiida.backends.djsite.db.subtests.migrations.test_migrations_0037_attributes_extras_settings_json.
DbMultipleValueAttributeBaseClass
[source]¶ Bases:
object
Abstract base class for tables storing attribute + value data, of different data types (without any association to a Node).
-
class
Meta
[source]¶ Bases:
object
-
__dict__
= mappingproxy({'__module__': 'aiida.backends.djsite.db.subtests.migrations.test_migrations_0037_attributes_extras_settings_json', 'abstract': True, 'unique_together': (('key',),), '__dict__': <attribute '__dict__' of 'Meta' objects>, '__weakref__': <attribute '__weakref__' of 'Meta' objects>, '__doc__': None})¶
-
__module__
= 'aiida.backends.djsite.db.subtests.migrations.test_migrations_0037_attributes_extras_settings_json'¶
-
__weakref__
¶ list of weak references to the object (if defined)
-
abstract
= True¶
-
unique_together
= (('key',),)¶
-
-
__dict__
= mappingproxy({'__module__': 'aiida.backends.djsite.db.subtests.migrations.test_migrations_0037_attributes_extras_settings_json', '__doc__': '\n Abstract base class for tables storing attribute + value data, of\n different data types (without any association to a Node).\n ', '_sep': '.', 'Meta': <class 'aiida.backends.djsite.db.subtests.migrations.test_migrations_0037_attributes_extras_settings_json.DbMultipleValueAttributeBaseClass.Meta'>, '_subspecifier_field_name': None, 'subspecifier_pk': <property object>, 'validate_key': <classmethod object>, 'set_value': <classmethod object>, 'create_value': <classmethod object>, '__dict__': <attribute '__dict__' of 'DbMultipleValueAttributeBaseClass' objects>, '__weakref__': <attribute '__weakref__' of 'DbMultipleValueAttributeBaseClass' objects>})¶
-
__module__
= 'aiida.backends.djsite.db.subtests.migrations.test_migrations_0037_attributes_extras_settings_json'¶
-
__weakref__
¶ list of weak references to the object (if defined)
-
_sep
= '.'¶
-
_subspecifier_field_name
= None¶
-
classmethod
create_value
(key, value, subspecifier_value=None, other_attribs={})[source]¶ Create a new list of attributes, without storing them, associated with the current key/value pair (and to the given subspecifier, e.g. the DbNode for DbAttributes and DbExtras).
- Note
No hits are done on the DB, in particular no check is done on the existence of the given nodes.
- Parameters
key – a string with the key to create (can contain the separator cls._sep if this is a sub-attribute: indeed, this function calls itself recursively)
value – the value to store (a basic data type or a list or a dict)
subspecifier_value – must be None if this class has no subspecifier set (e.g., the DbSetting class). Must be the value of the subspecifier (e.g., the dbnode) for classes that define it (e.g. DbAttribute and DbExtra)
other_attribs – a dictionary of other parameters, to store only on the level-zero attribute (e.g. for description in DbSetting).
- Returns
always a list of class instances; it is the user responsibility to store such entries (typically with a Django bulk_create() call).
-
classmethod
set_value
(key, value, with_transaction=True, subspecifier_value=None, other_attribs={}, stop_if_existing=False)[source]¶ Set a new value in the DB, possibly associated to the given subspecifier.
- Note
This method also stored directly in the DB.
- Parameters
key – a string with the key to create (must be a level-0 attribute, that is it cannot contain the separator cls._sep).
value – the value to store (a basic data type or a list or a dict)
subspecifier_value – must be None if this class has no subspecifier set (e.g., the DbSetting class). Must be the value of the subspecifier (e.g., the dbnode) for classes that define it (e.g. DbAttribute and DbExtra)
with_transaction – True if you want this function to be managed with transactions. Set to False if you already have a manual management of transactions in the block where you are calling this function (useful for speed improvements to avoid recursive transactions)
other_attribs – a dictionary of other parameters, to store only on the level-zero attribute (e.g. for description in DbSetting).
stop_if_existing – if True, it will stop with an UniquenessError exception if the new entry would violate an uniqueness constraint in the DB (same key, or same key+node, depending on the specific subclass). Otherwise, it will first delete the old value, if existent. The use with True is useful if you want to use a given attribute as a “locking” value, e.g. to avoid to perform an action twice on the same node. Note that, if you are using transactions, you may get the error only when the transaction is committed.
-
property
subspecifier_pk
¶ Return the subspecifier PK in the database (or None, if no subspecifier should be used)
-
classmethod
validate_key
(key)[source]¶ Validate the key string to check if it is valid (e.g., if it does not contain the separator symbol.).
- Returns
None if the key is valid
- Raises
aiida.common.ValidationError – if the key is not valid
-
class
-
class
aiida.backends.djsite.db.subtests.migrations.test_migrations_0037_attributes_extras_settings_json.
TestAttributesExtrasToJSONMigrationManyNodes
(methodName='runTest')[source]¶ Bases:
aiida.backends.djsite.db.subtests.migrations.test_migrations_common.TestMigrations
This test comparing to the previous one (TestAttributesExtrasToJSONMigrationSimple), it creates several nodes with different atributes and extras and checks their correct migration one-by-one.
-
__module__
= 'aiida.backends.djsite.db.subtests.migrations.test_migrations_0037_attributes_extras_settings_json'¶
-
migrate_from
= '0036_drop_computer_transport_params'¶
-
migrate_to
= '0037_attributes_extras_settings_json'¶
-
nodes_no_to_create
= 20¶
-
nodes_to_verify
= {}¶
-
-
class
aiida.backends.djsite.db.subtests.migrations.test_migrations_0037_attributes_extras_settings_json.
TestAttributesExtrasToJSONMigrationSimple
(methodName='runTest')[source]¶ Bases:
aiida.backends.djsite.db.subtests.migrations.test_migrations_common.TestMigrations
A “simple” test for the attributes and extra migration from EAV to JSONB. It stores a sample dictionary using the EAV deserialization of AiiDA Django for the attributes and extras. Then the test checks that they are corerctly converted to JSONB.
-
__module__
= 'aiida.backends.djsite.db.subtests.migrations.test_migrations_0037_attributes_extras_settings_json'¶
-
migrate_from
= '0036_drop_computer_transport_params'¶
-
migrate_to
= '0037_attributes_extras_settings_json'¶
-
nodes_to_verify
= {}¶
-
-
class
aiida.backends.djsite.db.subtests.migrations.test_migrations_0037_attributes_extras_settings_json.
TestSettingsToJSONMigration
(methodName='runTest')[source]¶ Bases:
aiida.backends.djsite.db.subtests.migrations.test_migrations_common.TestMigrations
This test checks the correct migration of the settings. Setting records were used as an example from a typical settings table of Django EAV.
-
__module__
= 'aiida.backends.djsite.db.subtests.migrations.test_migrations_0037_attributes_extras_settings_json'¶
-
migrate_from
= '0036_drop_computer_transport_params'¶
-
migrate_to
= '0037_attributes_extras_settings_json'¶
-
setUpBeforeMigration
()[source]¶ Anything to do before running the migrations, which should be implemented in test subclasses.
-
settings_info
= {}¶
-
Tests for the migrations of the attributes, extras and settings from EAV to JSONB Migration 0037_attributes_extras_settings_json
-
class
aiida.backends.djsite.db.subtests.migrations.test_migrations_0038_data_migration_legacy_job_calculations.
TestLegacyJobCalcStateDataMigration
(methodName='runTest')[source]¶ Bases:
aiida.backends.djsite.db.subtests.migrations.test_migrations_common.TestMigrations
Test the migration that performs a data migration of legacy JobCalcState.
-
__module__
= 'aiida.backends.djsite.db.subtests.migrations.test_migrations_0038_data_migration_legacy_job_calculations'¶
-
migrate_from
= '0037_attributes_extras_settings_json'¶
-
migrate_to
= '0038_data_migration_legacy_job_calculations'¶
-
Tests for the migrations of legacy process attributes.
-
class
aiida.backends.djsite.db.subtests.migrations.test_migrations_0040_data_migration_legacy_process_attributes.
TestLegacyProcessAttributeDataMigration
(methodName='runTest')[source]¶ Bases:
aiida.backends.djsite.db.subtests.migrations.test_migrations_common.TestMigrations
Test the migration that performs a data migration of legacy JobCalcState.
-
__module__
= 'aiida.backends.djsite.db.subtests.migrations.test_migrations_0040_data_migration_legacy_process_attributes'¶
-
migrate_from
= '0039_reset_hash'¶
-
migrate_to
= '0040_data_migration_legacy_process_attributes'¶
-
Tests for the migrations of legacy process attributes.
-
class
aiida.backends.djsite.db.subtests.migrations.test_migrations_0041_seal_unsealed_processes.
TestSealUnsealedProcessesMigration
(methodName='runTest')[source]¶ Bases:
aiida.backends.djsite.db.subtests.migrations.test_migrations_common.TestMigrations
Test the migration that performs a data migration of legacy JobCalcState.
-
__module__
= 'aiida.backends.djsite.db.subtests.migrations.test_migrations_0041_seal_unsealed_processes'¶
-
migrate_from
= '0040_data_migration_legacy_process_attributes'¶
-
migrate_to
= '0041_seal_unsealed_processes'¶
-
Tests for the migrations of legacy process attributes.
-
class
aiida.backends.djsite.db.subtests.migrations.test_migrations_0043_default_link_label.
TestSealUnsealedProcessesMigration
(methodName='runTest')[source]¶ Bases:
aiida.backends.djsite.db.subtests.migrations.test_migrations_common.TestMigrations
Test the migration that performs a data migration of legacy JobCalcState.
-
__module__
= 'aiida.backends.djsite.db.subtests.migrations.test_migrations_0043_default_link_label'¶
-
migrate_from
= '0042_prepare_schema_reset'¶
-
migrate_to
= '0043_default_link_label'¶
-
The basic functionality for the migration tests
-
class
aiida.backends.djsite.db.subtests.migrations.test_migrations_common.
TestMigrations
(methodName='runTest')[source]¶ Bases:
aiida.backends.testbase.AiidaTestCase
This is the common test class that is used by all migration tests. It migrates to a given migration point, allows you to set up the database & AiiDA at that point with the necessary data and migrates then to the final migration point. In the end it forwards the database at the final migration (as it should be and found before the migration tests).
-
__module__
= 'aiida.backends.djsite.db.subtests.migrations.test_migrations_common'¶
-
property
app
¶
-
migrate_from
= None¶
-
migrate_to
= None¶
-
This file contains the majority of the migration tests that are too short to go to a separate file.
-
class
aiida.backends.djsite.db.subtests.migrations.test_migrations_many.
TestCalcAttributeKeysMigration
(methodName='runTest')[source]¶ Bases:
aiida.backends.djsite.db.subtests.migrations.test_migrations_many.TestMigrationsModelModifierV0025
This test class checks that the migration 0023_calc_job_option_attribute_keys works as expected which migrates CalcJobNode attributes for metadata options whose key changed.
-
KEY_ENVIRONMENT_VARIABLES_NEW
= 'environment_variables'¶
-
KEY_ENVIRONMENT_VARIABLES_OLD
= 'custom_environment_variables'¶
-
KEY_PARSER_NAME_NEW
= 'parser_name'¶
-
KEY_PARSER_NAME_OLD
= 'parser'¶
-
KEY_PROCESS_LABEL_NEW
= 'process_label'¶
-
KEY_PROCESS_LABEL_OLD
= '_process_label'¶
-
KEY_RESOURCES_NEW
= 'resources'¶
-
KEY_RESOURCES_OLD
= 'jobresource_params'¶
-
__module__
= 'aiida.backends.djsite.db.subtests.migrations.test_migrations_many'¶
-
migrate_from
= '0022_dbgroup_type_string_change_content'¶
-
migrate_to
= '0023_calc_job_option_attribute_keys'¶
-
-
class
aiida.backends.djsite.db.subtests.migrations.test_migrations_many.
TestDataMoveWithinNodeMigration
(methodName='runTest')[source]¶ Bases:
aiida.backends.djsite.db.subtests.migrations.test_migrations_common.TestMigrations
Check that backward migrations work also for the DbLog migration(s).
-
__module__
= 'aiida.backends.djsite.db.subtests.migrations.test_migrations_many'¶
-
migrate_from
= '0024_dblog_update'¶
-
migrate_to
= '0025_move_data_within_node_module'¶
-
-
class
aiida.backends.djsite.db.subtests.migrations.test_migrations_many.
TestDbLogMigrationBackward
(methodName='runTest')[source]¶ Bases:
aiida.backends.djsite.db.subtests.migrations.test_migrations_common.TestMigrations
Check that backward migrations work also for the DbLog migration(s).
-
__module__
= 'aiida.backends.djsite.db.subtests.migrations.test_migrations_many'¶
-
migrate_from
= '0024_dblog_update'¶
-
migrate_to
= '0023_calc_job_option_attribute_keys'¶
-
-
class
aiida.backends.djsite.db.subtests.migrations.test_migrations_many.
TestDbLogMigrationRecordCleaning
(methodName='runTest')[source]¶ Bases:
aiida.backends.djsite.db.subtests.migrations.test_migrations_common.TestMigrations
This test class checks that the migration 0024_dblog_update works as expected. That migration updates of the DbLog table and adds uuids
-
__module__
= 'aiida.backends.djsite.db.subtests.migrations.test_migrations_many'¶
-
migrate_from
= '0023_calc_job_option_attribute_keys'¶
-
migrate_to
= '0024_dblog_update'¶
-
setUpBeforeMigration
()[source]¶ Anything to do before running the migrations, which should be implemented in test subclasses.
-
test_dblog_calculation_node
()[source]¶ Verify that after the migration there is only two log records left and verify that they corresponds to the CalculationNodes.
-
-
class
aiida.backends.djsite.db.subtests.migrations.test_migrations_many.
TestDuplicateNodeUuidMigration
(methodName='runTest')[source]¶ Bases:
aiida.backends.djsite.db.subtests.migrations.test_migrations_common.TestMigrations
Test the migration that verifies that there are no duplicate UUIDs
-
__module__
= 'aiida.backends.djsite.db.subtests.migrations.test_migrations_many'¶
-
migrate_from
= '0013_django_1_8'¶
-
migrate_to
= '0014_add_node_uuid_unique_constraint'¶
-
-
class
aiida.backends.djsite.db.subtests.migrations.test_migrations_many.
TestGroupRenamingMigration
(methodName='runTest')[source]¶ Bases:
aiida.backends.djsite.db.subtests.migrations.test_migrations_common.TestMigrations
This test class checks the migration 0022_dbgroup_type_string_change_content which updates the type_string column of the groups.
-
__module__
= 'aiida.backends.djsite.db.subtests.migrations.test_migrations_many'¶
-
migrate_from
= '0021_dbgroup_name_to_label_type_to_type_string'¶
-
migrate_to
= '0022_dbgroup_type_string_change_content'¶
-
-
class
aiida.backends.djsite.db.subtests.migrations.test_migrations_many.
TestMigrationsModelModifierV0025
(methodName='runTest')[source]¶ Bases:
aiida.backends.djsite.db.subtests.migrations.test_migrations_common.TestMigrations
Sub class of TestMigrations that need to work on node attributes using the ModelModifierV0025.
-
__module__
= 'aiida.backends.djsite.db.subtests.migrations.test_migrations_many'¶
-
-
class
aiida.backends.djsite.db.subtests.migrations.test_migrations_many.
TestNoMigrations
(methodName='runTest')[source]¶ Bases:
aiida.backends.testbase.AiidaTestCase
Verify that no django migrations remain.
-
__module__
= 'aiida.backends.djsite.db.subtests.migrations.test_migrations_many'¶
-
-
class
aiida.backends.djsite.db.subtests.migrations.test_migrations_many.
TestNodePrefixRemovalMigration
(methodName='runTest')[source]¶ Bases:
aiida.backends.djsite.db.subtests.migrations.test_migrations_common.TestMigrations
This test class checks that the migration 0028_remove_node_prefix works as expected.
That is the final data migration for Nodes after aiida.orm.nodes reorganization was finalized to remove the node. prefix
-
__module__
= 'aiida.backends.djsite.db.subtests.migrations.test_migrations_many'¶
-
migrate_from
= '0027_delete_trajectory_symbols_array'¶
-
migrate_to
= '0028_remove_node_prefix'¶
-
-
class
aiida.backends.djsite.db.subtests.migrations.test_migrations_many.
TestParameterDataToDictMigration
(methodName='runTest')[source]¶ Bases:
aiida.backends.djsite.db.subtests.migrations.test_migrations_common.TestMigrations
This test class checks that the migration 0029_rename_parameter_data_to_dict works as expected.
This is a data migration for the renaming of ParameterData to Dict.
-
__module__
= 'aiida.backends.djsite.db.subtests.migrations.test_migrations_many'¶
-
migrate_from
= '0028_remove_node_prefix'¶
-
migrate_to
= '0029_rename_parameter_data_to_dict'¶
-
-
class
aiida.backends.djsite.db.subtests.migrations.test_migrations_many.
TestResetHash
(methodName='runTest')[source]¶ Bases:
aiida.backends.djsite.db.subtests.migrations.test_migrations_common.TestMigrations
This test class checks that only the hash extra is removed.
-
__module__
= 'aiida.backends.djsite.db.subtests.migrations.test_migrations_many'¶
-
migrate_from
= '0038_data_migration_legacy_job_calculations'¶
-
migrate_to
= '0039_reset_hash'¶
-
-
class
aiida.backends.djsite.db.subtests.migrations.test_migrations_many.
TestTextFieldToJSONFieldMigration
(methodName='runTest')[source]¶ Bases:
aiida.backends.djsite.db.subtests.migrations.test_migrations_common.TestMigrations
This test class checks that the migration 0033_replace_text_field_with_json_field works as expected.
That migration replaces the use of text fields to store JSON data with builtin JSONFields.
-
__module__
= 'aiida.backends.djsite.db.subtests.migrations.test_migrations_many'¶
-
migrate_from
= '0032_remove_legacy_workflows'¶
-
migrate_to
= '0033_replace_text_field_with_json_field'¶
-
-
class
aiida.backends.djsite.db.subtests.migrations.test_migrations_many.
TestTrajectoryDataMigration
(methodName='runTest')[source]¶ Bases:
aiida.backends.djsite.db.subtests.migrations.test_migrations_many.TestMigrationsModelModifierV0025
This test class checks that the migrations 0026_trajectory_symbols_to_attribute and 0027_delete_trajectory_symbols_array work as expected. These are data migrations for TrajectoryData nodes where symbol lists are moved from repository array to attributes.
-
__module__
= 'aiida.backends.djsite.db.subtests.migrations.test_migrations_many'¶
-
cells
= array([[[2., 0., 0.], [0., 2., 0.], [0., 0., 2.]], [[3., 0., 0.], [0., 3., 0.], [0., 0., 3.]]])¶
-
migrate_from
= '0025_move_data_within_node_module'¶
-
migrate_to
= '0027_delete_trajectory_symbols_array'¶
-
positions
= array([[[0. , 0. , 0. ], [0.5, 0.5, 0.5], [1.5, 1.5, 1.5]], [[0. , 0. , 0. ], [0.5, 0.5, 0.5], [1.5, 1.5, 1.5]]])¶
-
setUpBeforeMigration
()[source]¶ Anything to do before running the migrations, which should be implemented in test subclasses.
-
stepids
= array([60, 70])¶
-
times
= array([0.6, 0.7])¶
-
velocities
= array([[[ 0. , 0. , 0. ], [ 0. , 0. , 0. ], [ 0. , 0. , 0. ]], [[ 0.5, 0.5, 0.5], [ 0.5, 0.5, 0.5], [-0.5, -0.5, -0.5]]])¶
-
-
class
aiida.backends.djsite.db.subtests.migrations.test_migrations_many.
TestUuidMigration
(methodName='runTest')[source]¶ Bases:
aiida.backends.djsite.db.subtests.migrations.test_migrations_common.TestMigrations
This test class checks the migration 0018_django_1_11 which switches from the django_extensions UUID field to the native UUIDField of django 1.11. It also introduces unique constraints on all uuid columns (previously existed only on dbnode).
-
__module__
= 'aiida.backends.djsite.db.subtests.migrations.test_migrations_many'¶
-
migrate_from
= '0017_drop_dbcalcstate'¶
-
migrate_to
= '0018_django_1_11'¶
-