Committers' Guide

Apache Isis documentation is broken out into a number of user, reference and "supporting procedures" guides.

The user guides available are:

The reference guides are:

The remaining guides are:

This guide provides guidance for Apache Isis' own committers.

If a diff file has been provided, then it can easily be applied using eGit’s wizards (Eclipse’s Git integration)…

If a patch file has been provided, then it can be applied using command line tools.

The overall process is as follows:

This screencast also shows the process.

The script uses 'jq' to parse JSON. To install:

The syntax is:

where:

The listing below shows the steps taken by the script:

The build now commences. Once done, the script continues:

The screenshot belows shows the history we end up with:

This shows the fork being merged into the temporary branch ("ISIS-1162_pr-31"), then some further tidy-up, and finally the merging of the temporary branch into mainline.

Note that there is no rebasing in this model. This is intentional: when the merged branch is pushed, github will automatically close the original pull request.

Before releasing core, ensure there is consensus on the dev mailing list that this is the right time for a release. The discussion should include confirming the version number to be used, and to confirm content.

These discussions should also confirm the version number of the module being released. This should be in line with our semantic versioning policy.

Make sure you have a JIRA ticket open against which to perform all commits. In most cases a JIRA ticket will have been created at the beginning of the previous release cycle.

We use environment variables to parameterize as many of the steps as possible. For example:

Obviously, alter $ISISDEV and $ISISREL as required, and bump $ISISRC for re-releasing following an unsuccessful releases.

Set the HEAD of your local git repo to the commit to be released. This will usually be the tip of the origin’s master branch. Then, create a release branch for the version number being released; eg:

All release preparation is done locally; if we are successful, this branch will be merged back into master.

Double check that the version number of the parent pom should reflect the branch name that you are now on (with a -SNAPSHOT suffix). his will normally have been done already during earlier development; but confirm that it has been updated. If it has not, make the change.

Double check that the version number of the core POM (core/pom.xml) should reflect the branch name that you are now on. For example, if releasing version 1.16.0, the POM should read:

Also, check that there are no snapshot dependencies:

The only mention of SNAPSHOT should be for the Isis modules about to be released.

First, we release core. Switch to the appropriate directory:

Apache Isis archetypes are reverse engineered from example applications. Once reverse engineered, the source is checked into git (replacing any earlier version of the archetype) and released.

There are currently two archetypes, simpleapp and helloworld.

The mvn release:perform commands will have put release artifacts for both core and the simpleapp archetype into a newly created staging repository on the ASF Nexus repository server.

Log onto repository.apache.org (using your ASF LDAP account):

And then check that the release has been staged (select staging repositories from left-hand side):

If nothing appears in a staging repo you should stop here and work out why.

Assuming that the repo has been populated, make a note of its repo id; this is needed for the voting thread. In the screenshot above the id is org.apache.isis-008.

After checking that the staging repository contains the artifacts that you expect you should close the staging repository. This will make it available so that people can check the release.

Press the Close button and complete the dialog:

Nexus should start the process of closing the repository.

All being well, the close should (eventually) complete successfully (keep hitting refresh):

The Nexus repository manager will also email you with confirmation of a successful close.

If Nexus has problems with the key signature, however, then the close will be aborted:

Use gpg --keyserver hkp://pgp.mit.edu --recv-keys nnnnnnnn to confirm that the key is available.

Push the release branch to origin:

and also push tags for both core and the archetype:

Once the artifacts have been uploaded, you can call a vote.

In all cases, votes last for 72 hours and require a +3 (binding) vote from members.

Whenever a release manager announces a vote on a release (as per the release process) on the dev mailing list, it is the responsibility of the project’s PMC to cast their vote on the release. Anyone else can also vote, but only members of the Apache Isis PMC’s vote are binding.

Per this ASF documentation, the legal requirements for an ASF release are:

Note that the binaries are not an ASF release, they merely exist on the Maven central repo as a convenience. That said, you might also want to verify the release by pulling the binaries from the Maven staging repository. Details of how to do this are also documented below.

To verify the source ZIP files, you will need to have imported the public keys used for signing Apache Isis releases. These can be downloaded from the root of the Apache Isis source tree.

Since the Apache Isis source is mirrored on github.com, you can just use:

Also, we will be rebuilding Isis from source. Therefore delete all Isis artifacts from your local Maven repo:

You can either verify the source artifacts manually, or use a script that automates the steps.

The Apache Creadur project exists to provide a set of tools to ensure compliance with Apache’s licensing standards.

The main release auditing tool, Apache RAT is used in the release process.

Creadur’s remaining tools - Tentacles and Whisker - are to support the verification process.

For example, Tentacles generates a report called archives.html. This lists all of the top-level binaires, their LICENSE and NOTICE files and any LICENSE and NOTICE files of any binaries they may contain.

Validation of the output at this point is all still manual. Things to check include:

In this report, each binary will have three links listed after its name '(licenses, notices, contents)'

Assuming that everything builds ok, then test the archetypes (adjust version as necessary):

If the archetypes run up ok, then it’s time to vote!

When you have made the above checks (and any other checks you think may be relevant), cast your vote by replying to the email thread on the mailing list.

If you are casting -1, please provide details of the problem(s) you have found.

Post the results to the dev@isis.a.o mailing list:

using the body (alter last line as appropriate):

Replace the -RCn tag with another without the qualifier.

You can do this using the scripts/promoterctag.sh script; for example:

This script pushes the tag under refs/tags/rel. As per Apache policy (communicated on 10th Jan 2016 to Apache PMCs), this path is 'protected' and is unmodifiable (guaranteeing the provenance that the ASF needs for releases).

Then, continue onto the next section for the steps to promote and announce the release.

From the ASF Nexus repository, select the staging repository and select 'release' from the top menu.

This moves the release artifacts into an Apache releases repository; from there they will be automatically moved to the Maven repository.

As described in the Apache documentation, each Apache TLP has a release/TLP-name directory in the distribution Subversion repository at https://dist.apache.org/repos/dist. Once a release vote passes, the release manager should svn add the artifacts (plus signature and hash files) into this location. The release is then automatically pushed to http://www.apache.org/dist/ by svnpubsub. Only the most recent release of each supported release line should be contained here, old versions should be deleted.

Each project is responsible for the structure of its directory. The directory structure of Apache Isis reflects the directory structure in our git source code repo:

If necessary, checkout this directory structure:

Next, add the new release into the appropriate directory, and delete any previous release. The upd.sh script (also downloadable from this gist) can be used to automate this:

The script downloads the artifacts from the Nexus release repository, adds the artifacts to subversion and deletes the previous version.

Double check that the files are correct; there is sometimes a small delay in the files becoming available in the release repository. It should be sufficient to check just the md5 or .asc files that these look valid (aren’t HTML 404 error pages):

Assuming all is good, commit the changes:

If the files are invalid, then revert using svn revert . --recursive and try again in a little while.

Update the Apache Isis (asciidoc) website:

Don’t forget to commit the .adoc changes and publish to the isis-site repo.

Announce the release to users mailing list.

For example, for a release of Apache Isis Core, use the following subject:

And use the following body (summarizing the main points as required):

Log onto the Apache blog and create a new post. Copy-n-paste the above mailing list announcement should suffice.

Because we release from a branch, the changes made in the branch (changes to pom.xml made by the maven-release-plugin, or any manual edits) should be merged back from the release branch back into the master branch:

Finally, update helloworld’s pom.xml and simpleapp’s root pom.xml to reference the next SNAPSHOT release (1.17.0-SNAPSHOT)

With the release complete, now is a good time to bump versions of dependencies (so that there is a full release cycle to identify any possible issues).

You will probably want to create a new JIRA ticket for these updates (or if minor then use the "catch-all" JIRA ticket raised earlier for the next release).

This is also a good time to make source code has been cleaned up and formatted according to the Apache Isis and ASF conventions. Use this Eclipse template and this import order.

Finally, push the changes up to origin:

The (non-ASF) Incode Platform should also be released, as per their release guide.

Post the results to the dev@isis.a.o mailing list.

For example, use the following subject for a vote on Apache Isis Core:

using the body (alter last line as appropriate):

Tidy up remote branches in the git repo:

Drop staging repositories:

Finally, rewind the release branch to prior to the previous release candidate, and continue from there.

Before you start, make sure you’ve defined the snapshots repo in your local ~/.m2/settings.xml file:

where xxxxxxx and yyyyyyy are your Apache LDAP username and password. For more information, see these ASF docs.

{note It is also possible to configure to use .ssh secure keys, and thereby avoid hardcoding your Apache LDAP password into your .m2/settings.xml file. A description of how to do this can be found, for example, here. }

Before deploying the snapshot, perform a quick sanity check.

First, delete all Isis artifacts from your local Maven repo:

Next, check that the framework builds ok:

Confirm that the versions of the Isis artifacts now cached in your local repository are correct (both those pulled down from Maven central repo, as well as those of the component built locally).

Deploy the framework using:

This will deploy all the modules that make up a release.

To confirm that they are present, browse to Apache’s Nexus repository manager and search for "isis".

Create a remote fork/clone of the Apache Isis repository (eg using github or bitbucket or similar), and ensure that your local fork specifies this remote.

Also, set up a CI server against your fork/clone, to build against any branches called origin/interim/*. To build it should use the command:

that is, skipping the example simpleapp and archetype; only core framework is built

The CI server should then also publish the resultant artifacts to a local Maven repository. For example, Jenkins provides post-build plugins to perform such a task.

Ensure that the framework builds ok using the same command that your CI server is set up to execute (see section above).

Deploy the framework using:

where xxx is the most recent release of Isis (to act as the base), and yyyyy is the name of the remote.

For example,

This will result in a new branch and tagged being pushed to the remote, with an appropriate version (details below).

As noted in the prereqs (above), the CI server configured should then detect the new branch (or tag, if you prefer), build the framework (skipping application and architecture, as in the prerequisites) and then publish the resultant artifacts to a private Maven repo.

The generated site is published by copying into the content/ directory of the isis-site git repo. You therefore need to check this out this repo.

The procedure assumes that two git repos (for isis itself and for isis-site) are checked out into the same parent directory, eg:

If this isn’t the case, then it is possible to override the relative directory by passing in a system property to the mvn goal; see below.

You also need to know that ASF’s publishing script work from the 'asf-site' branch, NOT from the 'master' branch. Therefore, in the isis.git repo site:

Back in the adocs/documentation directory of the main isis-git.repo, to copy the generated documents to the isis-site.git repo, run:

This deletes the entire content of contents, and replaces with the content under target/site. It also fixes up line endings, standardizing on unix-style LFs.

To copy and to also commit the generated documents to the isis-site.git repo , run:

Behind the scenes this just calls mvn clean install -Dmessage=…​.

It’s also possible to omit the message, in which case publish.sh will reuse the most recent commit message from the main isis.git repo.

Pushing the commits (in the isis-site.git directory, of course) will publishing the changes:

Double check at isis.apache.org.

If none of the guides have been changed, and if you have run the full rebuild recently, then you can skip the generation of PDFs using:

The clean goal must not be included though (else all the guides will disappear from the site content).

Download and install GnuPG (gpg), version 1.4.10 or higher.

Then, edit ~/.gnupg/gpg.conf (on Windows, the file to edit is C:\Users\xxx\AppData\Roaming\gnupg\gpg.conf) so that the default is to generate a strong key:

The Apache Software Foundation requires that keys are signed with a key (or subkey) based on RSA 4096 bits. To do this:

Specify RSA key:

Specify key length as 4096 bits:

Specify key as non-expiring:

Enter your name, email and comment:

Real name: Xxx Xxxxxxxxx Email address: xxx@apache.org Comment: CODE SIGNING KEY You selected this USER-ID: "Xxx Xxxxxxxxx (CODE SIGNING KEY) xxx@apache.org"

Change (N)ame, ©omment, (E)mail or (O)kay/(Q)uit? O

You need a Passphrase to protect your secret key. Enter passphrase:

Provide a passphrase to secure your key.

GPG will goes on to generate your key:

The public key with id nnnnnnnn should now be stored in ~/.gnupg/pubring.pgp (on Windows 7, this is in c:/Users/xxx/AppData/Roaming/gnupg/pubring.pgp).

To confirm the key has been generated, use:

The key Id is the one true way to identify the key, and is also the last 8 digits of the fingerprint. The corresponding secret key for id nnnnnnnn is stored in ~/.gnupg/secring.pgp (on Windows 7, this is in c:/Users/xxx/AppData/Roaming/gnupg/secring.pgp).

It’s also worth confirming the key has the correct preference of algorithms (reflecting the initial configuration we did earlier). For this, enter the gpg shell for your new key:

where nnnnnnnn is your key id. Now, use the 'showpref' subcommand to list details:

The Digest line should list SHA-512 first and SHA-1 last. If it doesn’t, use the "setpref" command:

Finally, remember to take a backup of your key and the keyring (ie, backup the .gnupg directory and its contents).

It’s recommended to use a subkey with an expiry date to sign releases, rather than your main, non-expiring key. If a subkey is present, then gpg will use it for signing in preference to the main key.

To create a subkey Enter the gpg shell using (the identifier of) your main key:

Type 'addkey' to create a subkey, and enter your passphrase for the main key:

Select (6) to choose an RSA key for encryption:

Specify that the key is valid for 1 year:

Quit the gpg shell; you now have a subkey.

It’s good practice to generate a number of revocation certificates so that the key can be revoked if it happens to be compromised. See the gpg page on the Apache wiki for more background on this topic.

First, generate a "no reason specified" key:

Select 0.

Provide a description:

Confirm this is ok.

The file revoke-nnnnnnnn-0.asc should be created: Then, backup this file.

Now repeat the process to create two further revocation certificates:

Specify reason as "1 = Key has been compromised"

and:

Specify reason as "3 = Key is no longer used"

Backup these files also.

It is also necessary to publish your key. There are several places where this should be done. In most cases, you’ll need the "armored" " (ie ASCII) representation of your key. This can be generated using:

where nnnnnnnn is the id of your public key.

You’ll also need the fingerprint of your key. This can be generated using:

The output from this command includes a line beginning "Key fingerprint", followed by a (space delimited) 40 character hexadecimal fingerprint. The last 8 characters should be the same as the key id (nnnnnnnn).

It is strongly advised that the contributor attend a key signing party at an Apache event, in order that other Apache committers/members can in person verify their identity against the key. The process for this is described here and here.

The Maven release plugin will automatically sign the release, however it is necessary to update the ~/.m2/settings.xml file with your GPG acronym passphrase in order that it can use your secret key. This is defined under a profile so that it is activated only when we perform a release (as defined by [org,apache:apache] parent POM.

Therefore, make the following edits:

In addition, to allow the release plugin to tag SVN changes, you must either add in your LDAP username/password or configure .ssh:

Apache Isis releases are built using Java 7, enforced using the maven toolchains plugin. Ensure that Java 7 is installed and the toolchains plugin is configured, as described in the contributors' guide.

The most important configuration you require is to set up public/private key pair. This is used by the maven-release-plugin to sign the code artifacts. See the page on key generation for more details.

In order to prepare the release, you’ll (need to) have a ~/.gnupg directory with the relevant files (gpg.conf, pubring.gpg, secring.gpg etc), and have gpg on your operating system PATH.

During the release process the maven-deploy-plugin uploads the generated artifacts to a staging repo on the Apache repository manager. This requires your Apache LDAP credentials to be specified in your ~/.m2/settings.xml file:

where xxxxxxx and yyyyyyy are your Apache LDAP username and password. For more information, see these ASF docs.

Also, set up keyphrase for gpg; this avoids being prompted during release:

These notes recommend how contributors should work with git. To understand these notes, the only real concepts that you need to grok are:

For more background reading, see:

And, of course, there is loads of good advice on stackoverflow.com

To complete the procedures documented here, you’ll need ssh access to minotaur.apache.org. For example:

When prompted, provide passphrase for private key.

Currently we don’t distinguish between committers and PMC members: every committer is automatically invited to also be a member of the PMC.

Further notes on the steps for new PMC members can be found here.