Currently, todo.sh produces no indented output, so this hasn't affected tests so far. I found this while testing one of my custom actions with the test-lib.
By unsetting IFS, the read command won't strip off leading and trailing whitespace while parsing the test session definition.
This extension of test_todo_completion additionally takes the completion function to use, so that customized completions for todo.sh aliases can be tested.
The completion may unexpectedly abort somewhere.
Additionally, reformat the failure messages (don't prepend a "*", this should be reserved to mark the individual test runs) and use "compreply" instead of "output", so that it's already clear from the diff output that the completion is being tested.
There should be a trailing newline when there are completions (to avoid diff complaining about "no trailing newline", but not when there are no completions at all.
Before adding any more features to todo_completion, I feel like I need test coverage, so this is a first stab at testing the completion results, via a new test function test_todo_completion.
Some basic tests showcase the capabilities.
Note: test-lib.sh now uses arrays, therefore all tests must use /bin/bash, not /bin/sh to avoid errors when sourcing test-lib. For consistency with todo.sh, we should have used Bash everywhere, anyway. Also note that t2000-multiline.sh needs some more quoting to avoid "Bash: ambiguous redirect" errors.
Now that differences in the output (and exit code) are already printed by default, we can make the verbose mode actually "verbose" by including all todo.sh output generated during the test run. This may help in reviewing the tests and for troubleshooting.
By moving the redirection to the output file to test_run_(), all testing-related embellishments have been removed from the command under test itself, resulting in much cleaner test messages.
Additionally, also capture stderr in output. todo.sh itself currently prints everything to stdout (but the die() output probably belongs to stderr), so as of now, that has no consequences, but seems to be more consistent and future-proof.
Remove the check for the todo.sh exit code (scripted via "=== N") from the command under test into a separate assertion test_expect_code_and_output. This allows for reporting of expected vs. actual exit code (also in the default non-verbose mode), and unexpected output from the same test is now reported, too.
Differences in the expected and actual todo.sh output are critical to analyzing the error, so they should always be printed, not just when the -verbose argument is given.
This refactoring moves the test_cmp call from the command under test (as seen from test-lib's perspective) into a new assertion test_expect_output, derived from test_expect_success.
- test-lib uses 'read -r' to parse the test session input literally, without interpretation of backslashes.
- FIX: Use quoting to maintain original whitespace (tabs and multiple spaces) from the test session input (instead of condensing into a single space).
- Using Bash instead of POSIX shell for t1340-listescapes.sh, so that the interpretation of escape sequences is not dependent on the POSIX shell being used.
- Changed Makefile so that the shell selected by the shebang line is actually used when invoking tests, not the POSIX shell.
- Above changes obsolete the multiple escaping of the test session data; now, the test session can actually be pasted as-is into a test script, even when it contains backslashes. (I.e. works as expected now.)
BUG 1: didn't report non-error exit
A test that enforces a non-zero exit code via "=== N" would succeed when the command under test returned with success (0); only differing exit codes would be correctly reported.
Corrected logic so that the check for the exit code would always run, not just when the command under test failed.
BUG 2: too early evaluation of $?
In the test expression, the evaluation of $? must be deferred until the test expression itself is evaluated.
Escaping $? to prevent premature evaluation in the context of the expression definition.
While preparing a setup where env vars overload config file,
I had the bad experience that my actual todo.txt was messed up
by the testsuite so it's safer to unset everything.
Even so, some tests could have failed if some TODOTXT_ were set
in the user environment, for example:
export TODOTXT_PLAIN=1
./t1200-pri.sh
Since test-lib.sh does not depend on bash currently, calculate
the list of variables to be unset using set|sed.
Signed-off-by: Emil Sit <sit@emilsit.net>
Add a simple helper script testshell.sh that can be called
to set up an environment for generating test case transcripts
that can then be pasted into an actual test script.
Port old testsuite.txt into t9999-testsuite_example.sh
with the aim of refactoring it later into smaller tests.
Also, change test_todo_session to count from one instead of zero when
autogenerating tests.
Signed-off-by: Emil Sit <sit@emilsit.net>
Create a custom date view for todo.sh tests -- the time
is set in TODO_TEST_TIME as seconds since the Unix epoch
and a custom wrapper in the testing directory is put at the
front of the path to intercept todo.sh calls to date for
date formatting.
This will only work as long as todo.sh calls $(date) for
date formatting. It also assumes a date command that
supports the -d flag (such as the one in GNU coreutils.)
Add a test for TODOTXT_DATE_ON_ADD to exercise this functionality.
Signed-off-by: Emil Sit <sit@emilsit.net>
Add a test_todo_session function to test-lib.sh that enables
easy recording of input and output from todo.sh (including
annotation of exit status for testing error cases).
Begin to port and re-factor testsuite.txt into smaller
test cases, starting with basic add list functionality
and replace functionality.
Thanks to Philippe Teuwen <phil@teuwen.org> for feedback
and improvements.
Signed-off-by: Emil Sit <sit@emilsit.net>
Add a basic test framework, borrowed from the framework used
by git.git. A shell script library (tests/test-lib.sh) helps
generate the fixtures, and simplifies the process of writing
test scripts. Tests can be run as a suite (via 'make test')
or individually (sh tests/t0000-config.sh). Results are aggregated
upon completion. Includes a detailed README.
A basic test of config file processing is part of this commit.
Signed-off-by: Emil Sit <sit@emilsit.net>
