Name | Updated at | |
---|---|---|
.clang-format | ||
Android.bp | ||
README.md | ||
cli-test.cpp |
cli-test
makes integration testing of command-line tools easier.
Readable syntax. Common cases should be concise, and pretty much anyone should be able to read tests even if they've never seen this tool before.
Minimal issues with quoting. The toybox tests -- being shell scripts -- quickly become a nightmare of quoting. Using a non ad hoc format (such as JSON) would have introduced similar but different quoting issues. A custom format, while annoying, side-steps this.
Sensible defaults. We expect your exit status to be 0 unless you say otherwise. We expect nothing on stderr unless you say otherwise. And so on.
Convention over configuration. Related to sensible defaults, we don't let you
configure things that aren't absolutely necessary. So you can't keep your test
data anywhere except in the files/
subdirectory of the directory containing
your test, for example.
Any all-whitespace line, or line starting with #
is ignored.
A test looks like this:
name: unzip -l
command: unzip -l $FILES/example.zip d1/d2/x.txt
after: [ ! -f d1/d2/x.txt ]
expected-stdout:
Archive: $FILES/example.zip
Length Date Time Name
--------- ---------- ----- ----
1024 2017-06-04 08:45 d1/d2/x.txt
--------- -------
1024 1 file
---
The name:
line names the test, and is only for human consumption.
The command:
line is the command to be run. Additional commands can be
supplied as zero or more before:
lines (run before command:
) and zero or
more after:
lines (run after command:
). These are useful for both
setup/teardown but also for testing post conditions (as in the example above).
Any command:
, before:
, or after:
line is expected to exit with status 0.
Anything else is considered a test failure.
The expected-stdout:
line is followed by zero or more tab-prefixed lines that
are otherwise the exact output expected from the command. (There's magic behind
the scenes to rewrite the test files directory to $FILES
because otherwise any
path in the output would depend on the temporary directory used to run the test.)
There is currently no expected-stderr:
line. Standard error is implicitly
expected to be empty, and any output will cause a test failure. (The support is
there, but not wired up because we haven't needed it yet.)
The fields can appear in any order, but every test must contain at least a
name:
line and a command:
line.
The output is intended to resemble gtest.
It's often useful to be able to match against stdout/stderr/a file rather
than give exact expected output. We might want to add explicit support for
this. In the meantime, it's possible to use an after:
with grep -q
if
you redirect in your command:
.
In addition to using a before:
(which will fail a test), it can be useful
to be able to specify tests that would cause us to skip a test. An example
would be "am I running as root?".
It might be useful to be able to make exit status assertions other than 0?
There's currently no way (other than the files/
directory) to share repeated
setup between tests.