holo-test - test runner for Holo plugins
holo-test <name> <directory>...
Holo plugins can be tested by executing Holo commands on fabricated file system trees, and comparing the output and resulting file system with the expected results. The holo-test program is a test runner for such test cases.
Each test case is a directory that must contain the following files:
source-tree <-- a dump of the root directory before running `holo apply` expected-tree <-- a dump of the expected root directory after running `holo apply` (see "DUMP FORMAT" below for details) expected-scan-output <-- expected output of `holo scan` expected-diff-output <-- expected output of `holo diff` expected-apply-output <-- expected output of `holo apply` expected-apply-force-output <-- expected output of `holo apply --force` (not always required, see below)
For each file like
expected-%, holo-test places the actual outputs in the file
scan-output, and so on). For files like
%-output, the outputs have any color codes stripped. The verbatim program outputs are placed at
colored-%-output contains a final line indicating the exit code of the holo process, in the format
exit status 2
source-tree is materialized into a directory
target/ directly inside the test case directory. holo-test then runs, in that order,
holo scan holo diff holo apply holo apply --force # maybe, see below
in a quasi-chroot here and seeing what output it produces and what it does to this filesystem tree. If the output of
holo apply mentions the word
holo apply --force is run, too. This covers cases where plugins refuse to overwrite modified entities, printing instead something like:
!! Target has been modified (use --force to overwrite)
Since you're probably testing a plugin that's not yet installed, you need to tell Holo to pick it up from the proper location. There's a special syntax allowed in holorc for that:
The path must be relative to the test case directory, since that's the working directory where Holo is run.
To run any number of tests, invoke holo-test with the name of the thing you're testing (typically the name of a Holo plugin), followed by the paths of the test case directories. The first argument is only used for formatting the output.
$ holo-test 'holo-files plugin' ./tests/holo-files/* >> Running test case tests/holo-files/01-first... >> Running test case tests/holo-files/02-second... >> Running test case tests/holo-files/03-third... >> All tests for holo-files plugin completed successfully.
The test will usually output just a progress display (one line per test case). If a test fails, the failing parts will be printed on the command line. To flag test failure when run in Continuous Integration setups,
holo-test will exit with non-zero status when one or more test cases fail.
expected-tree use a simple format to describe a hierarchy of directories, files and symlinks in a single file. For example:
file 0755 ./usr/lib/example.sh #!/bin/sh while read LINE; do echo "example: $LINE" done ---------------------------------------- symlink 0777 ./usr/lib/other-example.sh example.sh ---------------------------------------- directory 0700 ./var/lib/example/ ----------------------------------------
The dump consists of multiple sections, each terminated by a line containing a nonzero number of dashes. The first line of each section is a header with three space-separated fields: a file type (
file for regular files,
symlink for symbolic links, or
directory for directories), the file's access rights in octal notation, and the path to the file (relative to the root directory of the
For regular files, the header is followed by the contents of the file (possibly none). For symbolic links, the header is followed by exactly one nonempty line containing the link's target. For directories, the section contains only the header and no further lines.
Further documentation is available at the project homepage: https://holocm.org
Please report any issues and feature requests at GitHub: https://github.com/holocm/holo/issues