There are two major options to automatically add license headers to source files:

• Mojo License Maven plugin from codehaus
• Mycila Maven License plugin from mycila

I think that the “mycila” plugin is easier to use than the codehaus plugin. However, it seems as if the codehaus plugin is the go-forward tool, despite its problems and quirks. I found it very difficult to make the mojo Maven license plugin work in a way so that I could use my own license file in a centralized configuration artifact (i.e. a dedicated jar file holding the custom license file).

The good thing about the mojo Maven license plugin is that, in addition to the source code license headers, it can also create (although unfortunately not reliably update) the project license file.

As a first step, add a “license” section on the project level to document the license choice. It is commonly recommended to use a compatible name and identifier for the license. For open source projects, most licenses already have an SPDX name identifier assigned.

Once the license is defined and chosen, configure it for the project as follows:

1. Create a centralized configuration artifact, with a “resources” directory containing a “licenses” parent folder, and within the “licenses” folder, a folder named with the license identifier (e.g. the SPDX identifier). Only use lowercase letters for this directory name, and do not use spaces. This directory contains two files: One for the license header that goes into each file (“header.txt.ftl”), the other one for the license that goes into each Maven module’s root folder (“license.txt.ftl”).
   The license header file contains the body to be used in a generic template that comes with the plugin, while the license file that will be placed into the Maven root folder actually is a template that allows substitution. The content from the header file will later on be completed with a package description line and a copyright line, while the Maven root folder license file will not go through this transformation. For some reason, the header file cannot contain blank lines, as blank lines in this file will lead to a misleading error message.
2. Create a “licenses.properties” file in the license parent folder in the resources folder created in step 1. This file must contain a mapping of the license folder name (aka the license name or SPDX identifier) to a human readable name. This makes the license available to the plugin (the name is displayed, referencing the identifier, which by convention identifies the folder that holds the license files).
3. Build and release the configuration artifact
4. Configure the license maven plugin. Reference the folder name in the resource folder of the configuration artifact from step 1 as the <licenseName>. The <licenseResolver> tag must contain the path to the folder from step 1. As an example, if the license is stored in ./src/main/resources/license/mylicense, then “mylicense” is the licenseName and “license” is the licenseResolver. Include the dependency to the configuration artifact as a dependency of the License Maven plugin, and complete the configuration of the plugin as needed (e.g. if it should fail the build in case of missing / incorrect licenses, if the plugin is allowed to modify the license section in files, which files to ignore, etc).
5. Add the following required project meta information:
   1. <inceptionYear>
      This is used in conjunction with the current year as the value for “${copyright.years}” in the Maven module folder license file, and it will automatically be inserted in the file headers before the body of the header license notice
   2. <licenses> with <license> block containing name, URL, and distribution (set to “repo” in this example)
   3. Name and URL of the organization holding the copyright
      This is used as the value for ${copyright.holder} in the Maven module folder license file, and it will automatically be inserted in the file headers before the body of the header license notice after the “${copyright.years}” value
6. For each Maven module, add a ./src/license directory, with a “licenseDescription.ftl” file. This file should contain a single-line module specific description, which will be prefixed to all headers in the respective module. The content of this file is added into the file header right before the copyright years, copyright holder, and header license body.
   This file must not contain any blank lines. Some IDEs add a newline at the end of this file when editing / saving it. The license plugin removes this newline when it updates the license headers (good!), but it is not smart enough to also do this during the subsequent checks when it tries to find out if a file header needs to be changed. Consequently, every subsequent build will fail with an error of “file headers need to be updated”, but an update of the headers will not fix the issue. The solution is to remove the newline in this template file.

This is how the overall configuration in the parent POM should look like. Note that the name in the organization contains an html entities encoded representation of the email address to make it work with the License Maven plugin. Some of the configuration values are parameterized, but the example works equally well for hardcoded values:

<project>
  ...
  <inceptionYear>2014</inceptionYear>
  ...
  <licenses>
    <license>
      <name>BSD 3-clause Revised License</name>
      <url>LICENSE.txt</url>
      <distribution>repo</distribution>
    </license>
  </licenses>
  ...
  <organization>
    <name>Your Name &lt;your@email.com&gt;</name>
    <url>http://www.your.website.com</url>
  </organization>
  ...
  <build>
    ...
    <plugins>
      ...
      <plugin>
        <groupId>org.codehaus.mojo</groupId>
        <artifactId>license-maven-plugin</artifactId>
        <version>${licenseMavenPluginVersion}</version>
        <configuration>
          <licenseName>3-bsd-mb</licenseName>
          <licenseResolver>classpath://license</licenseResolver>
          <descriptionTemplate>${basedir}/src/license/licenseDescription.ftl</descriptionTemplate>
          <canUpdateCopyright>true</canUpdateCopyright>
          <canUpdateDescription>true</canUpdateDescription>
          <addJavaLicenseAfterPackage>false</addJavaLicenseAfterPackage>
          <failOnMissingHeader>true</failOnMissingHeader>
          <failOnNotUptodateHeader>true</failOnNotUptodateHeader>
          <excludes>
            <exclude>**/licenseDescription.ftl</exclude>
            <exclude>**/package-info.java</exclude>
            <exclude>**/*.txt</exclude>
            <exclude>**/*.properties</exclude>
          </excludes>
        </configuration>
        <dependencies>
          <dependency>
            <groupId>${buildToolsGroupId}</groupId>
            <artifactId>${buildToolsArtifactId}</artifactId>
            <version>${buildToolsVersion}</version>
          </dependency>
        </dependencies>
        <executions>
          <execution>
            <phase>validate</phase>
            <goals>
              <goal>check-file-header</goal>
            </goals>
          </execution>
        </executions>
      </plugin>
      ...
    </plugins>
    ...
  </build>
</project>

The build tools project at https://github.com/mbeiter/util/tree/master/build-tools contains an example for a centralized configuration artifact holding a custom 3 clause BSD license configuration that can be used with this plugin. See https://github.com/mbeiter/util for an example on how to configure the License Maven plugin as described in this post for a multi-module build, using the license from the configuration artifact.

Usage:

• To check all available licenses (note that custom licenses do not show up here, even if they are available!):
  # mvn license:license-list
• To check if all files have a valid license header (this function can also be used to check which files would be modified when a header is added automatically):
  # mvn license:check-file-header
• To add the license header to all files:
  # mvn license:update-file-header
• To create the project license file (for some reason, an update to existing files does not work):
  # mvn license:update-project-license

It should normally only take 10-15 minutes for an artifact to propagate to Maven Central after the staging repositories have been released in Sonatype OSSRH. If the artifacts do not show up on the browse view at http://search.maven.org/#browse, then there might be something broken with the index that powers this view.

I asked Sonatype support about this, and luckily, the artifacts are still available in Maven Central, even if they do not show up in the search index on search.maven.org. The search index and the browsing view are, in fact, completely decoupled from the actual storage backend, and http://search.maven.org is regularly updated about 1 to 2 hours AFTER an artifact has been pushed to Maven Central.

If an artifact does not show up in the browse view on http://search.maven.org, browse the Maven Central repository directly at http://repo1.maven.org/maven2/. Artifacts should show up there within 10-15 minutes after doing a release. If they are not visible after 20 minutes, the deployment most likely failed.

In this series of posts, we discuss how to publish Maven bits to Maven Central using the standard Maven deploy plugin. This is an eight step process to set up and execute, and we discussed the first two steps in the first post of the series and steps three to five in the previous post.

In this post, we discuss the last three steps with particular focus on how to do the code signing with GnuPG in an automated multi-module Maven build.

Step 6: Configure GnuPG for artifact signing

Neither Maven nor Nexus can properly handle GnuPG subkeys as signing keys. Using a subkey for signing is actually a very good security best practice, because it allows protecting the main key in automated code signing setups. Unfortunately, due to the limitations imposed by Maven and Nexus, if you are used to subkeys signing, then you must change your habit to make GnuPG work with Maven.

It also seems to be impossible to specify the singing key in the Maven GnuPG plugin. Consequently, you must delete not only all of your signing subkeys from the primary (master) key, but also all the other secret (master) keys you may have on your key ring. Once you deleted all secret keys save one, and all signing subkeys of that remaining primary key, the build should produce valid signatures.

For the basic Maven configuration, add the following section to the parent POM, preferably in a dedicated profile so that the signing is only executed when needed (e.g. on a centralized build server):

<build>
  ...
  <plugins>
  ...
    <plugin>
      <groupId>org.apache.maven.plugins</groupId>
      <artifactId>maven-gpg-plugin</artifactId>
      <version>${mavenGpgPluginVersion}</version>
      <executions>
        <execution>
          <id>sign-artifacts</id>
          <phase>verify</phase>
          <goals>
            <goal>sign</goal>
          </goals>
        </execution>
      </executions>
    </plugin>
  ...
  </plugins>
  ...
</build>

As GnuPG keys are usually protected by a passphrase, it is required to somehow inject the signing key’s passphrase into the Maven build. There are three common options for this:

1. The passphrase can be stored in a settings file.
   This is really a bad idea, in particular as Maven requires using the master key, and is incapable of operating on an easily revocable subkey.
2. The passphrase can be submitted as a parameter when starting the build.
   Use the -Dgpg.passphrase=keyPassphrase parameter when starting the Maven build. This is a reasonably safe option on single users systems (e.g. on dedicated build servers), but may allow users on multi-user systems to steal the passphrase from the process list. For multi-user systems, the parameter can point to an environment variable so that the passphrase does not show up in the process list (this is really a must on multi-user systems).
3. Provide the passphrase during the build when it is needed
   The plugin will ask for the passphrase if it cannot find it in a file or as a parameter. This is the most secure option, but it may prevent build automation if not properly configured. By default, GnuPG uses its agent to retrieve the password, and the default setup is an interactive prompt. Depending on how the pinentry component of the agent is configured, it can retrieve the password interactively or non-interactively e.g. from a password vault.

Step 7: Build and deploy to staging

Build the binaries according to the project’s release process. Assuming that all the release related plugin confirmations are in the “release” profile, such a build could look similar like this:

    mvn versions:set -DnewVersion=1.0
    mvn versions:commit
    mvn clean deploy site -P release

The build instructions (and POM configuration in the underlying project) of Mike’s utility library give an example on how to configure an enterprise style Maven multi-module build and set it up to automatically deploy to Sonatype Nexus.

Step 8: Release the deployment to Maven Central

As indicated earlier, the automated deployment to OSSRH with the standard Maven Deploy plugin does not automatically release the deployed artifacts to Maven Central. While this process can be automated, I generally prefer to review the build results once more before releasing them to the public.

Sonatype describes their release process in detail. Once the deployment has been released, Sonatype will do an (automated) evaluation of the staging repository contents, and promote to Maven Central if everything is okay. Otherwise, the process will fail and the (then obsolete) staging repository can be “Dropped”.

In the first post of this series, we discussed how to publish Maven bits to Maven Central using the standard Maven deploy plugin. This is an eight step process to set up and execute, and we discussed the first two steps in the previous post of this series.

We will discuss the five of the remaining steps in this post, and I will create another post with particular focus on how to do the code signing with GnuPG in an automated multi-module Maven build.

Step 3: Request an account in Sonatype JIRA and create a ticket to set up a Nexus repository

The process of how to request an account and have a Nexus staging repository set up is described in the OSSRH guide. The staging repository is restricted to a specific group ID. If you maintain multiple projects and eventually want to publish all of them, request a top-level group ID in the ticket.

As an example, if you own the domain “domain.com” and you have two projects with the planned group IDs “com.domain.project1” and “com.domain.project2”, request the repository to being set up for “com.domain”, even if the concrete request is for the project with the group ID “com.domain.project1”.

Step 4: Add the OSSRH repository to the distribution management section in the POM

I do not publish SNAPSHOT builds in my project, but Sonatype generally supports them. They are located in a different repository though.

Configure the distribution management section to deploy the bits automatically into the staging repository on Sonatype using the standard Maven deploy plugin:

<distributionManagement>
  <!--
  uncomment this section if you want to deploy SNAPSHOT builds
  <snapshotRepository>
    <id>ossrh</id>
    <url>https://oss.sonatype.org/content/repositories/snapshots</url>
  </snapshotRepository>
  -->
  <repository>
    <id>ossrh</id>
    <url>https://oss.sonatype.org/service/local/staging/deploy/maven2/</url>
  </repository>
</distributionManagement>

See the “util” project parent POM and related source file here for an example.

Step 5: Add your credentials for OSSRH to the “settings.xml” file in your home directory

The Maven deploy plugin will need the Sonatype jira credentials to submit the bits to the staging repository. It is really not very secure to use the primary credentials directly here, but unfortunately Sonatype does not support using a tokenized form of the credentials at this point.

<settings>
  ...
  <servers>
    ...
    <server>
      <id>ossrh</id>
      <username>jira-username</username>
      <password>jira-password</password>
    </server>
    ...
  </servers>
  ...
</settings>

Be sure to always use the same ID whenever referring to the OSSRH repository: The ID used in the repository ID of the “distribution management” section of the POM must match the ID used in the settings’ server section.

Edit: continue reading…

There are several automated options to publish artifacts that have been created in a Maven build to a Maven repository. The deploy process may depend somewhat on the repository being used. For example, if a project is not hosted by one of the big players, it might be necessary to deploy through a “proxy” mechanism that eventually forwards the artifacts to Maven Central, rather than deploying there directly.

One of these proxy services is provided by Sonatype as their “OSS Repository Hosting” or “OSSRH” service. The OSSRH service provided by Sonatype is free, and allows staging of one’s artifacts and then promoting them to Maven Central.

Sonatype provides detailed instructions on how to use the service, heavily promoting their “Nexus Staging Maven Plugin” for Maven based builds in combination with the Maven Release plugin. While there is nothing wrong with using these plugins, I find it hard to integrate into enterprise builds where the build process is a little more formalized and a well-defined branching and tagging process should be obeyed.

Following the eight steps outlined below, projects can be deployed to Maven Central using the standard Maven deploy process. The client side (i.e. the Maven build, tagging, branching, etc) can still be easily automated by employing a build system such as Jenkins, giving the build engineer tight control over how the build is performed. However, once the artifacts have been deployed, it is still necessary to “promote” them from the OSSRH Staging Area to Maven Central. While this step can be fully automated when using the Nexus Staging and the Maven Release plugins as recommended by Sonatype, I do not mind this last manual step and find it useful to be able to review the staged artifacts before they are published to Maven – in particular as releases are not that frequent. On the other hand, if a project uses OSSRH to also publish SNAPSHOT artifacts, which are commonly released with a much higher frequency than regular releases, it might be prudent to either follow a hybrid approach (using the staging plugin for SNAPSHOTS, and the regular Maven process for other releases), or to adopt a project workflow that fits in the boundaries defined by the Nexus Staging Maven Plugin.

The examples below are based on a utility library that I published through this process and commonly use as a Maven dependency. This particular project is a multi-module Maven build.

Step 1: Create a GnuPG key pair, and publish the public key

Sonatype has an excellent guide on how to generate and manage keys with GnuPG. They are very clear about the requirement that the key used to sign the artifacts cannot be a subkey. This is not very desirable from a security and key management perspective, but unfortunately necessary due to limitations of Maven and Nexus, which can only verify signatures against a primary key.

Step 2: Set up the Maven build so that it meets the OSSRH requirements

Like for the key management guide with GnuPG, Sonatype has published very detailed instructions on the requirements a project must meet so that it can be published to Maven Central through Sonatype’s OSSRH.

I noticed a few things than merit additional clarification in addition to what is mentioned in Sonatype’s guide:

• The build must produce a source and a Javadoc package in addition to the artifact. This is tricky in case a project is a mere resource project, that is, a container for resource files. In this case, it will usually not contain Javadocs. However, as Sonatype requires Javadocs to be present, it is possible to satisfy this requirement by creating a “fake” Javadocs file. This can be done by creating a src/main/javadoc directory and adding a README.txt file. The fake Javadoc can then be included by adding the plugin below. See the “build-tools” project POM and related source file here for an example.

  <build>
    ...
    <plugins>
    ...
      <plugin>
        <groupId>org.apache.maven.plugins</groupId>
        <artifactId>maven-jar-plugin</artifactId>
        <executions>
          <execution>
            <id>empty-javadoc-jar</id>
            <phase>package</phase>
            <goals>
              <goal>jar</goal>
            </goals>
            <configuration>
              <classifier>javadoc</classifier>
              <classesDirectory>${basedir}/src/main/javadoc</classesDirectory>
              <includes>
                <include>**/*</include>
              </includes>
            </configuration>
          </execution>
        </executions>
      </plugin>
    ...
    </plugins>
    ...
  </build>
  

• The group ID must be owned by the project. For example, a project cannot write to the google.com group ID unless it is authorized to do so. If necessary, reserve a group ID for the project when requesting access to OSSRH.
• The parent POM of the project must contain a “developers” section, but that section does not necessarily have to contain an email address that is trivially machine readable. You may try to protect your email address against UBE, like in the example below.

      <developers>
      <developer>
        <name>Mike Beiter</name>
        <email>michael AT beiter.org</email>
        <url>https://www.michael.beiter.org</url>
      </developer>
    </developers>
  

Edit: continue reading…

I recently published a small project on GitHub, mostly to play around with the features that GitHub offers over plain git. I came across the GitHub pages features, and wanted to try creating a site from my multi-module Maven project, and have that automatically published on GitHub pages.

It turns out that building a Maven site is a very fragile process, in particular for multi-module builds. There are countless blog posts and posts on stack overflow that deal with this problem, and I am adding just one more after spending half a day figuring things out. I just could not get the internal project links to work: Either the link worked from the direction of the site created by the parent POM to the child POM, or the other way around. At some point, I got the links working in both directions for the site’s side menu, but not for the links in the content section (e.g. the Project Modules overview page).

The key to the problem lies in how Maven creates the relative URLs in the project. Obviously, the project needs a “distribution management” section for the parent module (aggregator project) of the multi-module build. It also needs a “distribution management” section in each child module for the links to work. The URL in each of these sections has to be different in each project, otherwise the links in the report are broken. The reason for this is that some of the relative links in the generated site are computed based on the URL from the “distribution management” section.

In addition to the URL in the “distribution management” section, each module must also have a top-level URL parameter, because some of the links are computed relative to the URL from this top-level parameter.

To make things extra tricky, the two URLs must be the same, because otherwise the two algorithms that compute the links produce a site where some of the links work, and others do not.

As an example, I incorrectly used one of the links as a pointer to the project’s start page, and the other one as the pointer to the project’s Maven page. They shared the same domain, which probably contributed to the mess: The index.html page that the project-report plugin produces (“project information” -> “about” page -> “project modules” overview section) uses the main project URL to derive links, which all ended up having a “../” too much in the relative URL. The maven site plugin uses the “distribution management” URL in “mvn site:stage“, which resulted in the same links being correct in the “modules” side menu.

It is crucial to set the project URL correctly on each level, and then reference a module’s project URL again in its distribution management section to overwrite anything that may be inherited from a higher level POM.

The relevant section of the parent POM is set up like so:

<project>
  ...
  <url>http://domain.invalid/root/path/to/project/site/${project.version}/</url>
  ...
  <distributionManagement>
    ...
    <site>
      <id>some-id</id>
      <url>${project.url}</url>
    </site>
    ...
  </distributionManagement>
  ...
</build>

The id in the “site” section may have to match an entry in the ~/.m2/settings.xml file, depending on how the site is being distributed (e.g. automated upload to a webserver). For the setup on GitHub that I am using in this project, this is not required (I decided to copy locally, and then commit to the gh-pages branch – much less brittle, and not a lot of extra work).

Each child POM is set up like so:

<project>
  ...
  <url>http://domain.invalid/root/path/to/project/site/${project.version}/${project.artifactId}/
</url>
  ...
  <distributionManagement>
    ...
    <site>
      <id>some-id</id>
      <url>${project.url}</url>
    </site>
    ...
  </distributionManagement>
  ...
</build>

Note that the format of the URL must be specified as in the example, and the distribution management section must be present.

The distribution management section in the child POM looks exactly like the one in the parent POM. However, if it is omitted, then the distribution management section of the parent will eventually be used. It took me a while to figure this out, but apparently the order in which Maven seems to process this is to first do variable expansion, and then inheritance second. This means that, if the distribution management section were removed in the child, the child POM would inherit the parent’s distribution management section with the expanded URL, putting the value of the parent’s project URL into the child’s POM, and not the child’s URL that contains the artifact ID. Or, in other words, the URL value in the child POM would be ignored.

The interesting thing is that, according to the Maven Site Plugin documentation, the inheritance should work correctly as long as the multi-module build follows the Maven naming conventions:

• If subprojects inherit the (distribution) site URL from a parent POM, they will automatically append their artifactId to form their effective deployment location. This goes for both the project url and the url defined in the <distributionManagement> element of the pom.
• If your multi-module tree does not follow the Maven conventions, or if module directories are named differently than module artifacts, you have to specify the url’s for each child project.

As my project follows the standard naming convention, and removing both the <url> and the <distributionMananegemt> sections from the child POM leads to erratic behavior with broken links, the “automatic appending” is apparently not working correctly.

See https://github.com/mbeiter/util for an example on how to configure Maven to create the site in a multi-module build as described in this post. You may find the parent POM and the child POM of the DB module particularly interesting.

The BUILD.md file contains instructions on how to perform a release build using this setup. This in particular includes automatically (and consistently) setting the version for all POM files in the multi-module build, building the site, staging it locally, and then publish it on GitHub pages.

Note that it is recommended to always build the project before building the site, as the site goal may or may not report missing dependencies if it does not find them. A possible symptom of this problem is that the Javadoc plugin will not find your classes if “mvn install” is not run before “mvn site“:

I do not use a plugin to automatically publish the site on GitHub, but instead copy it manually from the staging folder in the “target” folder to the appropriate folder in the “gh-pages” branch of the same repository. This requires me to check out the project twice on my machine, once on the main branch, and once on the “gh-pages” branch. I still like this approach better than automating it, because building the site for multi-module builds seems to be very brittle in Maven, and this gives me the opportunity to review that everything has been built correctly (and all the links work) before I publish the site with a git commit. Also, while there are a few plugins that automate this process, such as com.github.github:site-maven-plugin and other plugins that implement wagon for GitHub in Maven, none of them seem to be able to properly deal with multi-module sites at this time.

I also use a small project intro page on GitHub pages with some basic project information and links to relevant documentation (e.g. the Maven site, and the Javadocs). The page I used there is a GitHub standard template. It looks good enough for my needs, and offers convenient links to download the project source code as a zip file or tar ball.