Running tests

In TARDIS, we focus primarily on unit tests. These tests check the outputs of individual functions, ensuring that each component behaves as expected.

Unit tests run quickly and are executed after every suggested change to TARDIS, allowing for immediate feedback and maintaining code quality.

All of them are based on pytest.

For policy-level expectations on what to run for different change types, see TARDIS Development Playbook.

Running the Unit Tests

This is very straightforward to run on your own machine. For very simple unit tests, you can run this with:

> pytest tardis

You can also run tests using VS Code test tooling where available. The pytest command line remains the standard.

If you do not have local regression data available yet, this unit-test run is the recommended minimum local validation before opening a PR.

Running Regression Data Tests

Note

We have migrated from tardis-refdata to tardis-regression-data.

Running the more advanced unit tests requires TARDIS Regression data that can be downloaded from the tardis-regression-data repository. The regression data contains data in different formats (npy, hdf5, csv etc.) which is used by the testing pipeline to verify the accuracy of the codebase. Git LFS is used to download and track the files in the tardis-regression-data repository.

> GIT_LFS_SKIP_SMUDGE=1 git clone https://github.com/tardis-sn/tardis-regression-data.git
> cd tardis-regression-data
> git lfs fetch && git lfs checkout

Note

GIT_LFS_SKIP_SMUDGE=1 defers downloading large files until the explicit git lfs fetch command, which provides better visibility into download progress and allows you to monitor which files are being retrieved.

The tardis-regression-data repository should be located outside of the main tardis repository, rather than being nested within it. Finally, the tests can be run using the following command

> pytest tardis --tardis-regression-data=/path/to/tardis-regression-data/

Or, to run tests for a particular file or directory

> pytest tardis/path/to/test_file_or_directory --tardis-regression-data=/path/to/tardis-regression-data/

Note

In some cases you might have to update the regression data. The steps to update the regression data are outlined in the Update the Regression Data.

For regression-data expectations, see TARDIS Development Playbook.

Note

The tests workflow runs on pull requests and on push events. To prevent leaking LFS quota, tests have been disabled on forks. If, by any chance, you need to run tests on your fork, make sure to run the tests workflow on master branch first. The LFS cache generated in the master branch should be available in all child branches. You can check if cache was generated by looking in the Setup LFS step of the workflow run. Cache can also be found under the “Management” Section under “Actions” tab.

For instructions to update regression data, please see Update the Regression Data.