Compare commits

..

4 Commits

Author SHA1 Message Date
Ingo Karkat
c31716af47 test-report: Use color highlighting as in test-lib.
The test aggregate results are easy to miss when running the entire test suite via "make test", as the status of the last test case is highlighted, but the aggregate results appear in an unformatted, uncolored block of text.
Copy the say_color() function from test-lib.sh. (Sorry for the duplication, I found no simple way to import or share just this piece of functionality without adding much complexity.)
Successes, errors and broken summaries will now be highlighted in the appropriate colors (unless --no-color is given or output is not to a terminal), but only if the number is more than zero.
2012-01-26 12:15:26 +01:00
Ingo Karkat
ebe9fb868b test-lib: Show full todo.sh output in verbose mode.
Now that differences in the output (and exit code) are already printed by default, we can make the verbose mode actually "verbose" by including all todo.sh output generated during the test run. This may help in reviewing the tests and for troubleshooting.
By moving the redirection to the output file to test_run_(), all testing-related embellishments have been removed from the command under test itself, resulting in much cleaner test messages.
Additionally, also capture stderr in output. todo.sh itself currently prints everything to stdout (but the die() output probably belongs to stderr), so as of now, that has no consequences, but seems to be more consistent and future-proof.
2012-01-26 09:54:30 +01:00
Ingo Karkat
189779c6de test-lib: Separate function for exit code assertion.
Remove the check for the todo.sh exit code (scripted via "=== N") from the command under test into a separate assertion test_expect_code_and_output. This allows for reporting of expected vs. actual exit code (also in the default non-verbose mode), and unexpected output from the same test is now reported, too.
2012-01-26 09:54:25 +01:00
Ingo Karkat
516f806d58 test-lib: Always print output differences, not just in verbose mode.
Differences in the expected and actual todo.sh output are critical to analyzing the error, so they should always be printed, not just when the -verbose argument is given.

This refactoring moves the test_cmp call from the command under test (as seen from test-lib's perspective) into a new assertion test_expect_output, derived from test_expect_success.
2012-01-26 09:35:38 +01:00
4 changed files with 126 additions and 57 deletions

View File

@@ -1,4 +1,57 @@
#!/bin/sh #!/bin/bash
[ "x$TERM" != "xdumb" ] && (
export TERM &&
[ -t 1 ] &&
tput bold >/dev/null 2>&1 &&
tput setaf 1 >/dev/null 2>&1 &&
tput sgr0 >/dev/null 2>&1
) &&
color=t
case "$1" in
--no-color)
color=; shift ;;
esac
if test -n "$color"; then
say_color () {
(
export TERM
case "$1" in
error) tput bold; tput setaf 1;; # bold red
skip) tput bold; tput setaf 2;; # bold green
pass) tput setaf 2;; # green
info) tput setaf 3;; # brown
*) test -n "$quiet" && return;;
esac
shift
printf "* %s" "$*"
tput sgr0
echo
)
}
else
say_color() {
test -z "$1" && test -n "$quiet" && return
shift
echo "* $*"
}
fi
get_color()
{
# Only use the supplied color if there are actually instances of that
# type, so that a clean test run does not distract the user by the
# appearance of the error highlighting.
if [ ${1:?} -eq 0 ]
then
echo 'info'
else
echo "${2:-info}"
fi
}
fixed=0 fixed=0
success=0 success=0
@@ -27,8 +80,8 @@ do
done <"$file" done <"$file"
done done
printf "%-8s%d\n" fixed $fixed say_color 'info' "$(printf "%-8s%d\n" fixed $fixed)"
printf "%-8s%d\n" success $success say_color "$(get_color "$success" 'pass')" "$(printf "%-8s%d\n" success $success)"
printf "%-8s%d\n" failed $failed say_color "$(get_color "$failed" 'error')" "$(printf "%-8s%d\n" failed $failed)"
printf "%-8s%d\n" broken $broken say_color "$(get_color "$broken" 'error')" "$(printf "%-8s%d\n" broken $broken)"
printf "%-8s%d\n" total $total say_color 'info' "$(printf "%-8s%d\n" total $total)"

View File

@@ -61,42 +61,6 @@ TODO: 0 of 5 tasks shown
TODO: 0 of 5 tasks shown TODO: 0 of 5 tasks shown
EOF EOF
cat > todo.txt <<EOF
(B) smell the uppercase Roses +flowers @outside
(X) clean the house from A-Z
(C) notice the sunflowers
(X) listen to music
buy more records from artists A-Z
EOF
test_todo_session 'listpri filtering priority ranges' <<EOF
>>> todo.sh -p listpri a-c
1 (B) smell the uppercase Roses +flowers @outside
3 (C) notice the sunflowers
--
TODO: 2 of 5 tasks shown
>>> todo.sh -p listpri c-Z
3 (C) notice the sunflowers
2 (X) clean the house from A-Z
4 (X) listen to music
--
TODO: 3 of 5 tasks shown
>>> todo.sh -p listpri A-
2 (X) clean the house from A-Z
--
TODO: 1 of 5 tasks shown
>>> todo.sh -p listpri A-C A-Z
--
TODO: 0 of 5 tasks shown
>>> todo.sh -p listpri X A-Z
2 (X) clean the house from A-Z
--
TODO: 1 of 5 tasks shown
EOF
cat > todo.txt <<EOF cat > todo.txt <<EOF
(B) ccc xxx this line should be third. (B) ccc xxx this line should be third.
ccc xxx this line should be third. ccc xxx this line should be third.

View File

@@ -180,7 +180,7 @@ test_failure_ () {
test_failure=$(($test_failure + 1)) test_failure=$(($test_failure + 1))
say_color error "FAIL $test_count: $1" say_color error "FAIL $test_count: $1"
shift shift
echo "$@" | sed -e 's/^/ /' echo "$@"
test "$immediate" = "" || { trap - EXIT; exit 1; } test "$immediate" = "" || { trap - EXIT; exit 1; }
} }
@@ -199,8 +199,9 @@ test_debug () {
} }
test_run_ () { test_run_ () {
eval >&3 2>&4 "$1" eval > output 2>&1 "$1"
eval_ret="$?" eval_ret="$?"
cat >&3 output
return 0 return 0
} }
@@ -260,6 +261,57 @@ test_expect_success () {
echo >&3 "" echo >&3 ""
} }
test_expect_output () {
test "$#" = 2 ||
error "bug in the test script: not 2 parameters to test-expect-output"
if ! test_skip "$@"
then
say >&3 "expecting success and output: $2"
test_run_ "$2"
if [ "$?" = 0 -a "$eval_ret" = 0 ]
then
cmp_output=$(test_cmp expect output)
if [ "$?" = 0 ]
then
test_ok_ "$1"
else
test_failure_ "$@" "
$cmp_output"
fi
else
test_failure_ "$@"
fi
fi
echo >&3 ""
}
test_expect_code_and_output () {
test "$#" = 3 ||
error "bug in the test script: not 3 parameters to test-expect-code-and-output"
if ! test_skip "$@"
then
say >&3 "expecting exit code $1 and output: $3"
test_run_ "$3"
if [ "$?" = 0 -a "$eval_ret" = "$1" ]
then
cmp_output=$(test_cmp expect output)
if [ "$?" = 0 ]
then
test_ok_ "$2"
else
test_failure_ "$2" "$3" "
$cmp_output"
fi
else
cmp_output=$(test_cmp expect output)
test_failure_ "$2" "$3" "
* expected exit code $1, actual ${eval_ret}${cmp_output:+
}${cmp_output}"
fi
fi
echo >&3 ""
}
test_expect_code () { test_expect_code () {
test "$#" = 3 || test "$#" = 3 ||
error "bug in the test script: not 3 parameters to test-expect-code" error "bug in the test script: not 3 parameters to test-expect-code"
@@ -271,7 +323,8 @@ test_expect_code () {
then then
test_ok_ "$2" test_ok_ "$2"
else else
test_failure_ "$@" test_failure_ "$2" "$3" "
* expected exit code $1, actual ${eval_ret}"
fi fi
fi fi
echo >&3 "" echo >&3 ""
@@ -542,9 +595,9 @@ test_todo_session () {
"") "")
if [ ! -z "$cmd" ]; then if [ ! -z "$cmd" ]; then
if [ $status = 0 ]; then if [ $status = 0 ]; then
test_expect_success "$1 $subnum" "$cmd > output && test_cmp expect output" test_expect_output "$1 $subnum" "$cmd"
else else
test_expect_success "$1 $subnum" "$cmd > output ; test \$? = $status && test_cmp expect output" test_expect_code_and_output "$status" "$1 $subnum" "$cmd"
fi fi
subnum=$(($subnum + 1)) subnum=$(($subnum + 1))
@@ -560,9 +613,9 @@ test_todo_session () {
done done
if [ ! -z "$cmd" ]; then if [ ! -z "$cmd" ]; then
if [ $status = 0 ]; then if [ $status = 0 ]; then
test_expect_success "$1 $subnum" "$cmd > output && test_cmp expect output" test_expect_output "$1 $subnum" "$cmd"
else else
test_expect_success "$1 $subnum" "$cmd > output ; test \$? = $status && test_cmp expect output" test_expect_code_and_output "$status" "$1 $subnum" "$cmd"
fi fi
fi fi
} }

15
todo.sh
View File

@@ -59,7 +59,7 @@ shorthelp()
listall|lsa [TERM...] listall|lsa [TERM...]
listcon|lsc listcon|lsc
listfile|lf [SRC [TERM...]] listfile|lf [SRC [TERM...]]
listpri|lsp [PRIORITIES] [TERM...] listpri|lsp [PRIORITY] [TERM...]
listproj|lsprj [TERM...] listproj|lsprj [TERM...]
move|mv ITEM# DEST [SRC] move|mv ITEM# DEST [SRC]
prepend|prep ITEM# "TEXT TO PREPEND" prepend|prep ITEM# "TEXT TO PREPEND"
@@ -231,11 +231,10 @@ help()
Without any arguments, the names of all text files in the todo.txt Without any arguments, the names of all text files in the todo.txt
directory are listed. directory are listed.
listpri [PRIORITIES] [TERM...] listpri [PRIORITY] [TERM...]
lsp [PRIORITIES] [TERM...] lsp [PRIORITY] [TERM...]
Displays all tasks prioritized PRIORITIES. Displays all tasks prioritized PRIORITY.
PRIORITIES can be a single one (A) or a range (A-C). If no PRIORITY specified, lists all prioritized tasks.
If no PRIORITIES specified, lists all prioritized tasks.
If TERM specified, lists only prioritized tasks that contain TERM(s). If TERM specified, lists only prioritized tasks that contain TERM(s).
Hides all tasks that contain TERM(s) preceded by a minus sign Hides all tasks that contain TERM(s) preceded by a minus sign
(i.e. -TERM). (i.e. -TERM).
@@ -1125,8 +1124,8 @@ case $action in
"listpri" | "lsp" ) "listpri" | "lsp" )
shift ## was "listpri", new $1 is priority to list or first TERM shift ## was "listpri", new $1 is priority to list or first TERM
pri=$(printf "%s\n" "$1" | tr 'a-z' 'A-Z' | grep -e '^[A-Z]$' -e '^[A-Z]-[A-Z]$') && shift || pri="A-Z" pri=$(printf "%s\n" "$1" | tr 'a-z' 'A-Z' | grep '^[A-Z]$') && shift || pri="[A-Z]"
post_filter_command="grep '^ *[0-9]\+ ([${pri}]) '" post_filter_command="grep '^ *[0-9]\+ (${pri}) '"
_list "$TODO_FILE" "$@" _list "$TODO_FILE" "$@"
;; ;;