Advanced Precommit

Process Reaper

A common problem is the 'stuck' unit test. If bash v4.0 or higher is in use, Apache Yetus may be told to turn on the process reaper functionality. Using the --reapearmode option, this feature may be configured to either report and even kill left over processes that match provided regular expressions.

WARNING: Using --reapermode outside of Docker will report or kill ALL matching processes on the system. It is recommended to only use those options whilst in Docker mode.

The reaper will run after every 'external' command that is printed on the console. This includes almost all build tool commands and individual test commands.

Plug-ins

test-patch allows one to add to its basic feature set via plug-ins. There is a directory called plugins.d inside the directory where test-patch.sh lives. Inside this directory one may place some bash shell fragments that, if setup with proper functions, will allow for test-patch to call it as necessary. Different plug-ins have specific functions for that particular functionality. In this document, the common functions available to all/most plug-ins are covered. Test plugins are covered below. See other documentation for pertinent information for the other plug-in types.

Common Plug-in Functions

Every plug-in must have one line in order to be recognized, usually an 'add' statement. Test plug-ins, for example, have this statement:

add_test_type <pluginname>

This function call registers the pluginname so that test-patch knows that it exists. Plug-in names must be unique across all the different plug-in types. Additionally, the 'all' plug-in is reserved. The pluginname also acts as the key to the custom functions that you can define. For example:

function pluginname_filefilter

defines the filefilter for the pluginname plug-in.

Similarly, there are other functions that may be defined during the test-patch run:

HINT: It is recommended to make the pluginname relatively small, 10 characters at the most.  Otherwise, the ASCII output table may be skewed.

Plug-in Importation

Plug-ins are imported from several key directories:

If the --skip-system-plugins flag is passed, then only core.d is imported.

Test Plug-ins

Plug-ins geared towards independent tests are registered via:

add_test_type <pluginname>

Personalities

It is impossible for any general framework to be predictive about what types of special rules any given project may have, especially when it comes to ordering and Maven profiles. In order to direct test-patch to do the correct action, a project personality should be added that enacts these custom rules.

A personality consists of one or more functions.

There can be only one of each personality function defined.

Global Settings

Globals for personalities should be defined in the personality_globals function. This function is called after the other plug-ins have been imported. This allows one to configure any settings for plug-ins that have been imported safely:

function personality_globals
{
  GITHUB_REPO="apache/yetus"
  PATCH_BRANCH_DEFAULT=main
  PROJECT_NAME="yetus"
}

Custom Argument Parsing

Additionally, a personality may require some outside help from the user. The personality_parse_args
function is called almost immediately after the personality is loaded and plug-ins parse arguments.

function personality_parse_args
{
  echo "$*"
}

It is important to note that this function is called AFTER personality_globals.

Test Determination

The personality_file_tests function determines which tests to turn on based upon the file name. It is relatively simple. For example, to turn on a full suite of tests for Java files:

function personality_file_tests
{
  local filename=$1

  if [[ ${filename} =~ \.java$ ]]; then
    add_test javac
    add_test javadoc
    add_test mvninstall
    add_test spotbugs
    add_test unit
  fi

}

The add_test function is used to activate the standard tests. Additional plug-ins (such as checkstyle), will get queried on their own.

This function may also be defined as PROJECT_NAME_personality_file_tests, where PROJECT_NAME matches the value passed via --project
or autodetermined by various means. The PROJECT_NAME version takes precedence over the generic version.

Module & Profile Determination

Once the tests are determined, it is now time to pick which modules should get used. That's the job of the personality_modules function.

function personality_modules
{

    clear_personality_queue

...

    personality_enqueue_module <module> <flags>

}

It takes exactly two parameters repostatus and testtype.

The repostatus parameter tells the personality function exactly what state the source repository is in. It can only be in one of two states: branch or patch. branch means the patch has not been applied. The patch state is after the patch has been applied.

The testtype state tells the personality exactly which test is about to be executed.

In order to communicate back to test-patch, there are two functions for the personality to use.

The first is clear_personality_queue. This removes the previous test's configuration so that a new module queue may be built. Custom personality_modules will almost always want to do this as the first action.

The second is personality_enqueue_module. This function takes two parameters. The first parameter is the name of the module to add to this test's queue. The second parameter is an option list of additional flags to pass to Maven when processing it. personality_enqueue_module may be called as many times as necessary for your project.

NOTE: A module name of . signifies the root of the repository.

For example, let's say your project uses a special configuration to skip unit tests (-DskipTests). Running unit tests during a javadoc build isn't very useful and wastes a lot of time. We can write a simple personality check to disable the unit tests:

function personality_modules
{
    local repostatus=$1
    local testtype=$2

    if [[ ${testtype} == 'javadoc' ]]; then
        personality_enqueue_module . -DskipTests
        return
    fi
    ...

This function will tell test-patch that when the javadoc test is being run, do the documentation build at the base of the source repository and make sure the -DskipTests flag is passed to our build tool.

This function may also be defined as PROJECT_NAME_personality_modules, where PROJECT_NAME matches the value passed via --project
or autodetermined by various means. The PROJECT_NAME version takes precedence over the generic version.

Enabling Plug-ins

Personalities can set the base list of plug-ins to enable and disable for their project via the personality_plugins function. Just call it with the same pattern as the --plugins command line option:

personality_plugins "all,-checkstyle,-spotbugs,-asflicense"

This list is used if the user does not provide a list of plug-ins.

Important Variables

There are a handful of extremely important system variables that make life easier for personality and plug-in writers. Other variables may be provided by individual plug-ins. Check their development documentation for more information.