Writing validation rulesets for IMAS-Validator

This section explains how to write your own tests. First set up your ruledir with a ruleset folder and a rule file. For a quick reminder of the ruleset structure look at Defining validation rules.

Hint

To create a new ruleset my_ruleset with one (empty) rule file, run

$ mkdir -p tmp/my_ruledir/my_ruleset/
$ touch tmp/my_ruledir/my_ruleset/my_rulefile.py

If you have your new empty rule file, we need to define validation rules inside. These are python functions structured like:

  1. @validator decorator specifying which IDSs to target with the function.

  2. Function definition accepting IDS instances as arguments.

  3. Docstring with a short description of the tests.

  4. The tests, which can use standard python logic, IMAS-Python IDSs and predefined helper methods. assert statements describe which conditions should be adhered to.

See Rule definition for more information.

Note

There is no need to separately import the helper functions and @validator decorator.

Exercise 1

Write a simple test to determine whether all core_profiles IDSs have a comment in their ids_properties attribute. Does the DBentry for imas:hdf5?path=imas-validator-course/good pass the test?

Note

If the assert statement is clear on its own, no need to add a custom message. Better to use those if the problem is not immediately recognizable from the test and/or code.

Add core_profiles in the @validator decorator to make sure the test is run against all core_profiles IDSs.

Use assert to make sure your rule is actually being tested.

my_ruledir/my_ruleset/my_rulefile.py
"""Very informative docstring for the rule file"""

@validator('core_profiles')
def test_core_profiles_comment(cp):
    """Test whether the comments are filled in for all core_profiles IDSs"""
    assert cp.ids_properties.comment is not None

Exercise 2

Write a test for core_profiles IDSs to determine whether the time array is strictly increasing. Use the Increasing helper function. Does the DBentry for imas:hdf5?path=imas-validator-course/good pass the test?

my_ruledir/my_ruleset/my_rulefile.py
"""Very informative docstring for the rule file"""

@validator('core_profiles')
def test_core_profiles_comment(cp):
    """Test whether the core_profiles base level time arrays are strictly increasing"""
    assert Increasing(cp.time)

Exercise 3

Write a test for core_profiles IDSs to determine whether the profiles follow electroneutrality. Use the Approx helper function. Does the DBentry for imas:hdf5?path=imas-validator-course/good pass the test? What if you use a comparison operator == instead of the Approx helper function?

The positive and negative charges can be determined using

  • profiles_1d[i].ion[j].density

  • profiles_1d[i].ion[j].z_ion

  • profiles_1d[i].electrons.density

my_ruledir/my_ruleset/my_rulefile.py
"""Very informative docstring for the rule file"""

@validator("core_profiles")
def validate_electroneutrality_core_profiles(cp):
    """Test whether the core_profiles have electroneutrality"""
    for profiles_1d in cp.profiles_1d:
        if len(profiles_1d.ion) == 0 or not profiles_1d.ion[0].density.has_value:
            continue
        ni_zi = sum(ion.density * ion.z_ion for ion in profiles_1d.ion)
        assert Approx(profiles_1d.electrons.density, ni_zi)

Exercise 4

Write a test for all IDSs to determine whether any _error_lower values are positive. Use the Select helper function. Does the DBentry for imas:hdf5?path=imas-validator-course/good pass the test?

Select all IDSs in the @validator decorator using a wildcard selector '*'. The filtering in the Select helper is done using Regex logic. Select(ids, "_error_lower$", has_value=True) will get the needed nodes for this test.

my_ruledir/my_ruleset/my_rulefile.py
"""Very informative docstring for the rule file"""

@validator("*")
def validate_errors_positive(ids):
    """Validate whether all error bar values are positive"""
    for error_lower in Select(ids, "_error_lower$", has_value=True):
        assert error_lower >= 0

Exercise 5

Write a test for all IDSs to determine whether in any case where a _min and its corresponding _max value both exist, the _min is lower than the _max. Use the Parent helper function. Does the DBentry for imas:hdf5?path=imas-validator-course/good pass the test?

You can get the name of a _min attribute using attr.metadata.name and then get its _max counterpart using getattr on the parent node.

my_ruledir/my_ruleset/my_rulefile.py
"""Very informative docstring for the rule file"""

@validator("*")
def validate_min_max(ids):
    """Validate that ``*_min`` values are lower than ``*_max`` values"""
    for quantity_min in Select(ids, "_min$", has_value=True):
        quantity_name = str(quantity_min.metadata.name)[:-4]  # strip off _min
        quantity_max = getattr(Parent(quantity_min), quantity_name + "_max", None)

        # If _max exists and is filled, check that it is >= _min
        if quantity_max is not None and quantity_max.has_value:
            assert quantity_min <= quantity_max

You can write tests that are only run on specific DD versions. In that case the dd_version needs to be to the added to the @validator decorator like @validator("core_profiles", version="3.40"). You can also specify ranges. For more information, see python packaging specifiers docs

Exercise 6

In the DD version 3.41, ids.ids_properties.provenance.node(i).sources(:) was changed to ids.ids_properties.provenance.node(i).reference(j)

1) Write a test that checks whether the length of references for all nodes is larger than zero for core_profiles. The DBEntry imas:hdf5?path=imas-validator-course/good has DD version 3.40.1 while imas:hdf5?path=imas-validator-course/new has version 3.42.0. Run your test for both.

2) Specify a version in the @validator decorator so that it only covers tests after version 3.41.0. Run your test for both and check if there are no errors left.

3) Write a test that checks whether the length of sources for all nodes is larger than zero for core_profiles specifically for versions before 3.41.0. Run your test for both and check if there are no errors left.

You can specify ranges of versions like <3.0.0 and >=2.0.0,<3.0.0

my_ruledir/my_ruleset/my_rulefile.py
@validator("core_profiles", version=">=3.41.0")
def test_sources_new(cp):
    """Validate that references are filled for versions after 3.41.0"""
    for node in cp.ids_properties.provenance.node:
        assert len(node.reference) > 0

@validator("core_profiles", version="<3.41.0")
def test_sources_old(cp):
    """Validate that sources are filled for versions before 3.41.0"""
    for node in cp.ids_properties.provenance.node:
        assert len(node.sources) > 0

You can write tests that combine multiple IDSs by adding both of their names in the @validator decorator. In that case the occurrence numbers need to be explicitly added like @validator("summary:0", "core_profiles:0").

Exercise 7

Write a test that checks if the time arrays for core_profiles and waves are approximately the same. Use the Approx helper function. Keep in mind that the occurrence number of an IDS needs to be specified for multi-ids validation. You can specify the occurrence number by writing the ids name like core_profiles:0 in the @validator decorator. Does the DBentry for imas:hdf5?path=imas-validator-course/good pass the test? What happens if you do not specify the occurrence number?

my_ruledir/my_ruleset/my_rulefile.py
"""Very informative docstring for the rule file"""

@validator("core_profiles:0", "waves:0")
def validate_min_max(cp, wv):
    """Validate that time array of core_profiles and waves are approximately the same"""
    assert Approx(cp.time, wv.time)
Last update: 2025-08-29