NAME

holo-test - test runner for Holo plugins

SYNOPSIS

holo-test <name> <directory>...

DESCRIPTION

Holo plugins can be tested by executing Holo commands on fabricated file system trees, and comparing the output and resulting file system with the expected results. The holo-test program is a test runner for such test cases.

Each test case is a directory that must contain the following files:

source-tree                 <-- a dump of the root directory before running `holo apply`
expected-tree               <-- a dump of the expected root directory after running `holo apply`
                                (see "DUMP FORMAT" below for details)
expected-scan-output        <-- expected output of `holo scan`
expected-diff-output        <-- expected output of `holo diff`
expected-apply-output       <-- expected output of `holo apply`
expected-apply-force-output <-- expected output of `holo apply --force`
                                (not always required, see below)

For each file like expected-%, holo-test places the actual outputs in the file % (i.e. tree, scan-output, and so on). For files like %-output, the outputs have any color codes stripped. The verbatim program outputs are placed at colored-%-output (e.g. colored-scan-output).

Each %-output and colored-%-output contains a final line indicating the exit code of the holo process, in the format

exit status 2

How tests are run

The file source-tree is materialized into a directory target/ directly inside the test case directory. holo-test then runs, in that order,

holo scan
holo diff
holo apply
holo apply --force # maybe, see below

in a quasi-chroot here and seeing what output it produces and what it does to this filesystem tree. If the output of holo apply mentions the word --force, then holo apply --force is run, too. This covers cases where plugins refuse to overwrite modified entities, printing instead something like:

!! Target has been modified (use --force to overwrite)

Since you're probably testing a plugin that's not yet installed, you need to tell Holo to pick it up from the proper location. There's a special syntax allowed in holorc for that:

plugin foobar=../../../build/holo-foobar

The path must be relative to the test case directory, since that's the working directory where Holo is run.

Running a test

To run any number of tests, invoke holo-test with the name of the thing you're testing (typically the name of a Holo plugin), followed by the paths of the test case directories. The first argument is only used for formatting the output.

$ holo-test 'holo-files plugin' ./tests/holo-files/*
>> Running test case tests/holo-files/01-first...
>> Running test case tests/holo-files/02-second...
>> Running test case tests/holo-files/03-third...
>> All tests for holo-files plugin completed successfully.

The test will usually output just a progress display (one line per test case). If a test fails, the failing parts will be printed on the command line. To flag test failure when run in Continuous Integration setups, holo-test will exit with non-zero status when one or more test cases fail.

DUMP FORMAT

The files source-tree, tree and expected-tree use a simple format to describe a hierarchy of directories, files and symlinks in a single file. For example:

file 0755 ./usr/lib/example.sh
#!/bin/sh
while read LINE; do
    echo "example: $LINE"
done
----------------------------------------
symlink 0777 ./usr/lib/other-example.sh
example.sh
----------------------------------------
directory 0700 ./var/lib/example/
----------------------------------------

The dump consists of multiple sections, each terminated by a line containing a nonzero number of dashes. The first line of each section is a header with three space-separated fields: a file type (file for regular files, symlink for symbolic links, or directory for directories), the file's access rights in octal notation, and the path to the file (relative to the root directory of the target/ directory).

For regular files, the header is followed by the contents of the file (possibly none). For symbolic links, the header is followed by exactly one nonempty line containing the link's target. For directories, the section contains only the header and no further lines.

SEE ALSO

holo(8), holo-plugin-interface(7)

AUTHOR

Stefan Majewsky

Further documentation is available at the project homepage: https://holocm.org

Please report any issues and feature requests at GitHub: https://github.com/holocm/holo/issues