Debugging with IMAS-Validator¶
In this section we zoom in on what you can do when a validation test fails
First, let's filter the tests so that we only run the failing test. We do this by adding flags to our command. If you only want to test specific rules you can filter on:
Rule name (-f, --filter, --filter_name)
IDS name (-f, --filter, --filter_ids)
Whether or not to use bundled rules (-b, --no-bundled)
Whether or not to use tests for all IDSs (-g, --no-generic)
Note
Rule name consists of <ruleset_name>/<file_name>:<rule_name>,
so you can use the --filter option to filter on a specific rule file.
Exercise 1¶
Call the IMAS-Validator for the db_entry with uri imas:hdf5?path=imas-validator-course/bad
Run only the failing tests.
(You can also try out all the filtering options.)
Use the -f, --filter or --filter_name flag to filter on the name of the failing test.
$ imas_validator validate 'imas:hdf5?path=imas-validator-course/bad' -f increasing_time
The summary report should only show the failing test for all IDS instances.
Sometimes you want to take a closer look at the data when a test fails. The IMAS-Validator has an optional feature to drop into a Python debugger (pdb) console when a test returns an assertion error. This gives you access in the terminal to the stack traces and local variables in the code. While the documentation shows a lot more functionality, in this case we focus only on the local variables. To activate the Python debugger, use the flag:
Python debugger (-d, --debug)
Exercise 2¶
Call the IMAS-Validator bundled tests for the db_entry with uri imas:hdf5?path=imas-validator-course/bad with the debugger argument.
What is the problem with this DBEntry?
Use the locals() function to list all local variables when in the Python debugger.
$ imas_validator validate 'imas:hdf5?path=imas-validator-course/bad' -f increasing_time -d
Time axis at toplevel is decreasing instead of increasing.