Apache Yetus 0.14.0 Release Notes

These release notes cover new developer and user-facing incompatibilities, important issues, features, and major improvements.

YETUS-1057 Major Remove findbugs support

Support for FindBugs has been removed as SpotBugs has completely replaced it.

YETUS-1122 Major pylint: provide option to ignore 'bad-option-value'

pylint will no longer flag options that are unsupported by the running version of pylint by default. The --pylint-ignore-bad-option-value has been added to control this behavior.

YETUS-1121 Major Rewrite shelldocs

shelldocs errors now go to stderr. Previously they went to stdout.

YETUS-1119 Major Update dependencies

Utilities within the Docker container have been upgraded, including several to major versions. There will likely be incompatible changes as a result.

List:

bats 1.2.1 -> 1.4.1
buf 0.21.0 -> 0.56.0
bundler 2.1.4 -> 2.2.7
checkmake now set at 8915bd409
codespell pre-2.0 -> 2.1.0
golangci-lint 1.31.0 -> 1.42.1
hadolint 1.18.0 -> 2.7.0
jshint 2.12.0 -> 2.13.1
markdownlint-cli 0.23.2 -> 0.28.1
pylint 2.5.3 -> 2.11.1
rake 13.0.1 -> 13.0.3
rubocop 0.90.0 -> 1.21.0
shellcheck 0.7.1 -> 0.7.2
spotbugs 4.2.2 -> 4.4.1
yamllint 1.24.2 -> 1.26.3

Anything not mentioned is either unchanged or the latest top of tree

YETUS-1127 Major exclusions rework

Users:

Exclusion support has been modified to be slightly incompatible with previous releases:

By default, .yetus/excludes.txt was supposed to have been read. However in previous releases it was ignored unless manually specified.
--excludes has been restricted such that only files within the source repository, especially when running in --docker mode
During a PR, the working set of changed files will now be completely recalculated, not just have additional files excluded.
The asflicense plug-in will now honor the global excludes file unless --asflicense-rat-globalexcludes=false is passed.
As a reminder, the excludes file needs to be regular expressions, not file globs. Therefore .* is the 'match everything' string.
Several bugs with exclusion in various plug-ins have been simplified/cleaned up and should result in fewer corner-case issues.

Developers:

yetus_file_to_array has been modified to additionally strip out lines that contain spaces in addition to empty lines and lines that begin with a comment.
The private function find_changed_dirs and the CHANGED_DIRS variable has been removed. It has always been broken and nothing was known to use it.

YETUS-1139 Major Option to limit docker build output

A new option to send the output of docker build to a log. This option is automatically enabled on Travis CI as it kills jobs that send too much data to console (~30k).

YETUS-1141 Major Write container images to ghcr.io

Container images are now available at ghcr.io/apache/yetus and ghcr.io./apache/yetus-base. The Dockerhub source will continue to be updated for the time being, however only the x86 image will be available. Users should migrate to the Github repository as the Dockerhub images will likely be removed in the future.

YETUS-1138 Major Better support for multiple architectures

The various Dockerfiles should now be much more compatible with various chipets. In addition to x86, arm64 container images are now published on the ghcr.io registry, enabling easier access for machines like Apple's M1.

YETUS-1152 Major Upgrade buf

As of this release, test-patch only supports buf v0.34.0 and higher.

YETUS-557 Major Investigate switching to jdk.javadoc.doclet

The audience annotations doclet code now requires JDK 11+. Additionally, the default Apache Yetus container no longer has JDK 8.

YETUS-1156 Major [April 2022] Update Dependencies

More updates to versions:

ansiblelint 5.2.1 -> 5.5.0
buf 1.0.0-rc10 -> 1.3.1
bundler 2.2.27 -> 2.3.10
checkmake is now 0.2.1
detect-secrets IBM 0.13 -> Yelp 1.2.0
golangcilint 1.42.1 -> 1.45.2
hadolint 2.8.0 -> 2.10.0
jshint 2.13.1 -> 2.13.4
markdownlint-cli 0.28.1 -> 0.31.1
maven 3.8.4 -> 3.8.5
pylint 2.11.1 ->2.13.4
rake 13.0.3 -> 13.0.6
rubocop 1.21.0 -> 1.26.1
shellcheck 0.7.2 -> 0.8.0
spotbugs 4.4.1 -> 4.6.0

As usual, versions not mentioned were either unchanged or latest top of tree.

YETUS-1132 Blocker EOL JDK8 Support

JDK8 support for audience annotations has been removed. JDK11 or higher is required.

The Apache Yetus container now has JDK11 as the base JDK.

YETUS-1159 Blocker fixes for CVE-2022-24765

Users:

If precommit is running within a container, set GIT_DIR and GIT_CEILING_DIRECTORIES to provide some assistance with CVE-2022-24765. If running outside of a container, there is an assumption that the user has a properly configured environment.

Developers:

This change now adds a new yetus_is_container function. It should be noted that there is no guaranteed way to determine if a process is in a container (especially from within the container) but there are some parts of the environment that are able to be checked to provide at least a pretty good guess.

YETUS-1030 Major Support linecomments in junit

The JUnit report output has been changed to support two formats, one generated using line-level and the other being a full report with URLs. By default, line is the default with the exception of Circle CI.

Jenkins users should now be able to process the output with the WarningsNG plug-in to provide better feedback in the Jenkins UI.