@rrahn commented on Mon Mar 30 2020

Description

In many systems to install an application multiple steps are involved:

configure / cmake
make 
make check
make install

First the application is configured (in our case using cmake), then the binary is built using make and then the extra target check is called to check if the built binary is correct.
In our case it should run CLI tests or a subset of them defined by the application developer.

Acceptance Criteria

• calling make check after make and make cli_tests runs the cli tests of the application
• if the binary hasn't been built before make check is called a diagnostic message is issued to inform that the binary has to be built before.
• if the cli tests haven't been built before a diagnostic message is issued to inform that the cli tests have to be built before.
• if the binary and cli tests are present the target check executes the cli tests.
• the target fails if some cli tests don't pass and outputs a corresonding diagnostic message
• if only a subset of cli tests is registered for the target only those get executed

Tasks

• add a check target which runs the cli tests
• allow to register only a subset of cli tests to the check target

Definition of Done

• Implementation and design approved
• Unit tests pass
• Test coverage = 100%
• Microbenchmarks added and/or affected microbenchmarks < 5% performance drop
• API documentation added
• Tutorial/teaching material added
• Test suite compiles in less than 30 seconds (on travis)
• Changelog entry added

@marehr commented on Thu May 07 2020

Core-Meeting:

This would only execute a subset of tests to test basic functionality. For a complete unit test; build all tests.