A typical local configuration is to have two repositories. One with the code you are working on and another, clean repository. The workflow looks similar to the following:
$ cd <workrepo>
$ git format-patch master > /tmp/patchfile
$ test-patch --basedir=/some/path/testrepo --build-tool=nobuild /tmp/patchfile
--basedir sets the location of the repository to use for testing. The /tmp/patchfile is the patch file that is generated by git format-patch or git diff or any number of other types of input.
When you run it, you should see something similar to this:
$ test-patch --basedir=/tmp/yetus.test --build-tool=nobuild /tmp/patchfile
/tmp/yetus-14497.22984 has been created
Modes:
Processing: /tmp/patchfile
Patch file /tmp/patchfile copied to /private/tmp/yetus-14497.22984
============================================================================
============================================================================
Confirming git environment
============================================================================
============================================================================
Testing patch on master.
============================================================================
============================================================================
Determining needed tests
============================================================================
============================================================================
(Depending upon input size and number of plug-ins, this may take a while)
============================================================================
============================================================================
Cleaning the source tree
============================================================================
============================================================================
cd /tmp/yetus.test
true > /private/tmp/yetus-14497.22984/branch-distclean-root.txt 2>&1
Elapsed: 0m 00s
============================================================================
============================================================================
Applying patch to master
============================================================================
============================================================================
Applying the changes:
Sun Jun 9 10:17:17 PDT 2019
cd /tmp/yetus.test
git apply --binary -v --stat --apply -p1 /private/tmp/yetus-14497.22984/input.patch
Applied patch README.md cleanly.
README.md | 1 +
1 file changed, 1 insertion(+)
Total Elapsed time: 0m 01s
+1 overall
____ _
/ ___| _ _ ___ ___ ___ ___ ___| |
\___ \| | | |/ __/ __/ _ \/ __/ __| |
___) | |_| | (_| (_| __/\__ \__ \_|
|____/ \__,_|\___\___\___||___/___(_)
| Vote | Subsystem | Runtime | Comment
============================================================================
| | | | Prechecks
| | | | master Compile Tests
| | | | Patch Compile Tests
| | | | Other Tests
| | | 0m 01s |
|| Subsystem || Report/Notes ||
============================================================================
| Optional Tests | |
| uname | Darwin hostname 18.6.0 Darwin Kernel Version 18.6.0: Thu Apr 25 23:16:27 PDT 2019; root:xnu-4903.261.4~2/RELEASE_X86_64 x86_64 |
| Build tool | nobuild |
| git revision | master / 6b94d9df |
| modules | C: . U: . |
| versions | git=2.21.0 |
| Powered by | Apache Yetus 0.11.0 https://yetus.apache.org |
============================================================================
============================================================================
Finished build.
============================================================================
============================================================================
test-patch gives extensive information about what it is doing at run time and then provides a report at the end to give a summary about what happened. In this particular case, this output just shows that the patch was applied and that no tests were actually run.
After running that command, the test repository will contain the remnants of the build for later debugging. It is inconvenient to instead use:
$ cd <workrepo>
$ git format-patch main > /tmp/patchfile
$ test-patch --basedir=/some/path/testrepo --resetrepo --build-tool=nobuild /tmp/patchfile
--resetrepo tells test patch that it can go into destructive mode. Destructive mode will wipe out any changes made to that repository, so use it with care!
In this particular case, the output will look nearly identical expect for one change:
Modes: ResetRepo
Some features are mostly invisible or auto-detected during runtime. The Modes line provides hints about those features.
An alternative to --resetrepo is the --dirty-workspace flag. It tells test-patch that the repository is not clean and it is ok to continue. This is especially useful on a repository that is in some way 'live,' such as a repository that is actively being used for development.
In general, almost all features are enabled via the plug-in framework. To see what has been enabled, use the --list-plugins option:
$ test-patch --list-plugins
You should see output similar to this:
BUILDTOOLS:
nobuild ant autoconf cmake gradle make maven
TESTTYPES:
asflicense author blanks buf cc checkmake checkstyle dupname golang golangcilint hadolint javac javadoc jshint markdownlint mvnsite pathlen perlcritic pylint revive rubocop scalac scaladoc shellcheck shelldocs test4tests unitveto xml yamllint
BUGSYSTEMS:
briefreport bugzilla github gitlab htmlout jira junit slack
TESTFORMATS:
ctest junit tap
This output shows all of the plug-ins that are available as well as what type of plug-in they are.
From this list, the specific plug-ins can be enabled:
$ test-patch --plugins="ant,maven,shellcheck,xml" <other options>
As a short-cut, every plug-in may be enabled via the special 'all' type:
$ test-patch --plugins="all" <other options>
--plugins also allows some basic "arithmetic":
$ test-patch --plugins="all,-checkstyle,-spotbugs" <other options>
This will enable all plug-ins for potential usage, except for checkstyle and spotbugs.
NOTE: Many examples in this section will use --plugins=all. Users should set the --plugins option as appropriate.
This command will execute basic patch testing using every available plug-in against a patch file stored in "filename":
$ cd <your repo>
$ test-patch --plugins=all <other options> <filename>
By default, unit tests are not run since they may take a significant amount of time.
To turn them on, we need to provide the –run-tests option:
$ cd <your repo>
$ test-patch --plugins=all --dirty-workspace --run-tests <filename>
This is the same command, but now runs the unit tests.
After the tests have run, there is a directory that contains all of the test-patch related artifacts. This is generally referred to as the patch directory. By default, test-patch tries to make something off of /tmp to contain this content. Using the --patch-dir option, one can specify exactly which directory to use. This is helpful for automated precommit testing so that continuous integration systems knows where to look to gather up the output.
For example:
$ test-patch --plugins=all --patch-dir=${WORKSPACE}/patchdir --basedir=${WORKSPACE}/source ${WORKSPACE}/patchfile
… will trigger test-patch to run in fully automated mode, using ${WORKSPACE}/patchdir as its scratch space, ${WORKSPACE}/source as the source repository, and ${WORKSPACE}/patchfile as the name of the patch to test against. This will always run the unit tests, write answers back to bug systems, remove old, stopped/exited Docker containers after 24 hours and images after 1 week, forcibly use --resetrepo, and more.
NOTE: Make sure to add the patch directory to .gitignore if the directory is inside the source tree to avoid deleting it, as test-patch does a git clean to remove untracked files from previous runs.
Out of the box, test-patch will try to figure out which build tool the project uses. But what if you want to override it? The --build-tool option allows a manual setting:
$ test-patch --plugins=all --build-tool=ant (other options)
will tell test-patch to use ant instead of maven to drive the project.
To disable the build tool entirely, use the nobuild setting:
$ test-patch --plugins=all --build-tool=nobuild (other options)
NOTE: More in-depth information may be found in the bugsystems section.
It is a fairly common practice within the Apache community to use Apache's JIRA instance to store potential patches. As a result, test-patch supports providing just a JIRA issue number via the jira plug-in. test-patch will find the last attachment, download it, then process it.
NOTE: test-patch expects the patch files to follow a particular naming convention. For complete details
on the naming convention please refer to patch-naming-conventions
For example:
$ test-patch --plugins=all HADOOP-9905 (other options)
… will process the patch file associated with this JIRA issue.
If the Apache JIRA system is not in use, then override options may be provided on the command line to point to a different JIRA instance.
$ test-patch --plugins=all --jira-issue-re='^PROJECT-[0-9]+$' --jira-base-url='https://example.com/jira' PROJECT-90
… will process the patch file attached to PROJECT-90 on the JIRA instance located on the example.com server.
test-patch has built-in support for Github via the github plug-in. test-patch supports many forms of providing pull requests to work on:
$ test-patch --plugins=all --github-repo=apache/pig GH:99
or
$ test-patch --plugins=all https://github.com/apache/pig/pulls/99
or
$ test-patch --plugins=all https://github.com/apache/pig/pulls/99.patch
… will process PR #99 on the apache/pig repo.
test-patch has support for Gitlab via the gitlab plug-in. test-patch supports many forms of providing merge requests to work on:
$ test-patch --plugins=all --gitlab-repo=_a__w_/yetus GL:1
or
$ test-patch --plugins=all https://gitlab.com/_a__w_/yetus/merge_requests/3
or
$ test-patch --plugins=all https://gitlab.com/_a__w_/yetus/merge_requests/3.patch
… will process MR #3 on the _a__w_/yetus repo.
Luckily, test-patch supports ways to provide unified diffs via URLs.
For example:
$ test-patch --plugins=all https://example.com/webserver/file.patch
process the file.patch from the example.com webserver.
Some repositories have content that is either imported from other sources (aka "vendored") or in some
other way have files that are known to break tests. By default, .yetus/excludes.txt will be read for any
file, directory, or regular expression as the list to remove from test results. The file to read
may be overwritten by using the --excludes flag:
$ test-patch --plugins=all --excludes=(filename) (other options)
NOTE: Do not confuse file globs for regular expressions. With regular expressions, `a*` will match `aaaaaaaa`
but not `ab`. To match all characters any number of times, you need `.*` as your wildcard expression.
In some cases, test plug-ins may always generate fail results in ways that cannot be avoided or are so
monumental to fix that you just want to keep track of the results without ever failing the run. test-patch
provides the --tests-filter option to do just that; print the results but don't fail the job. For example:
$ test-patch --plugins=all --tests-filter=checkstyle,javadoc (other options)
… will always force the checkstyle and javadoc tests to never vote -1.
Due to the extensible nature of the system, test-patch allows for projects to define project-specific rules which we call personalities. (How to build those rules is covered elsewhere.) There are two ways to specify which personality to use:
$ test-patch --plugins=all --personality=(filename)
This tells test-patch to use the personality in the given file.
However, test-patch can detect if it is a personality that is in its "personality" directory based upon the project name:
$ test-patch --plugins=all --project=(project)
By default, test-patch will set the user soft limit (ulimit -Su) to a relatively low 1,000 processes (and, on some operating systems with some languages such as Java, threads!). This is to prevent errant processes from eating up all system resources. If this limit is too low, it may be necessary to use the --proclimit option. For example:
$ test-patch --plugins=all --proclimit=10000
… will set it to be 10,000 processes.
NOTE: The actual implementation of this feature is dependent upon the version of Bash. For bash v4 and higher (most operating systems), the fork bomb protection is generally only used for the build and QA tools. This means Apache Yetus should continue to function. For earlier versions of bash (e.g., OS X), the limit is applied to all of test-patch. If the limit is hit, Apache Yetus will itself likely crash.
For many projects, it is useful to test Java code against multiple versions of JDKs at the same time. In combination with the java plug-in, test-patch can do this with the --multijdkdirs option:
$ test-patch --plugins=all --multijdkdirs="/j/d/k/1,/j/d/k/2"
Not all Java tests support this mode, but those that do will now run their tests with all of the given versions of Java consecutively (e.g., javac–the Java compilation test). Tests that do not support MultiJDK mode (e.g., checkstyle, mvn install) will use JAVA_HOME.
NOTE: JAVA_HOME is always appended to the list of JDKs in MultiJDK mode. If JAVA_HOME is in the list, it will be moved to the end.
test-patch also has a built-in mode (i.e., no plug-in required) to utilize Docker:
$ test-patch --docker
This command will do some preliminary setup and then re-execute itself inside a Docker container. For more information on how to provide a custom Dockerfile and other Docker-specific features, see the specific precommit Docker support page and the Apache Yetus Docker Hub Images page for more information on the convenience Docker images.
Currently, Apache Yetus is still undergoing incompatible changes from time to time. Despite that, in many cases
the upgrade process for test-patch and friends is usually just verifying what flags are being passed. To help out,
there is an option to --ignore-unknown-options so that test-patch does not error out if it is given flags it no
longer understands. It will print a list of those unknown options in the end report. In situations where that is
also undesirable, the --report-unknown-options may also be set simultaneously to remove the list from the report.
test-patch has many other features and command line options for the basic user. Many of these are self-explanatory. To see the list of options, run test-patch without any options or with --help.