Cutting a Release

The release process consists of:

the release manager cutting the release (documented below)
Members of the Apache Isis PMC verifying and voting on the release
the release manager performing post-release tasks, for either a successful or an unsuccessful vote.

Apache Isis itself is released in one go, everything is ultimately a child of bom/pom.xml (relative to the source code root). This section details the process for formally releasing this module.

The subsequent sections describe how other committers can verify a release and how the release manager can then perform post-release activities and set up for the next development iteration.

If you’ve not performed a release before, then note that there are some configuration prerequisites that must be configured first. In particular, you’ll need signed public/private keys, and the ASF Nexus staging repo inlocal ~/.m2/settings.xml file.

These release notes using bash command line tools. They should work on Linux and MacOS; for Windows, use mSysGit.

Preparation

The release process should be performed on Java 11.

The deploy process has been tested only with gpg 2.0.x ; you may need to adjust if using a later version.

Obtain Consensus

Before releasing the framework, ensure there is consensus on the dev mailing list that this is the right time for a release. The discussion should include confirming the version number to be used, and to confirm content.

These discussions should also confirm the version number of the module being released. This should be in line with our semantic versioning policy.

Make sure you have a JIRA ticket open against which to perform all commits. In most cases a JIRA ticket will have been created at the beginning of the previous release cycle.

Pull down code to release

Set the HEAD of your local git repo to the commit to be released. This will usually be the tip of the origin’s master branch:

git checkout master
git pull --ff-only

License headers

The Apache Release Audit Tool RAT (from the Apache Creadur project) checks for missing license header files. The parent pom.xml of each releasable module specifies the RAT Maven plugin, with a number of custom exclusions.

We run the tool runs over all submodules, including non-released modules. To run the RAT tool, use:

find unapproved/missing licenses

pushd bom
mvn -Dapache-release clean
mvn -Dapache-release org.apache.rat:apache-rat-plugin:check -D rat.numUnapprovedLicenses=1000     (1)
popd

for a in `/bin/find . -name rat.txt -print`; do grep '!???' $a; done > /tmp/rat-qn.txt            (2)
for a in `/bin/find . -name rat.txt -print`; do grep '!AL' $a; done > /tmp/rat-al.txt            (2)

cat /tmp/rat-qn.txt
cat /tmp/rat-al.txt

1	The command writes out a `target\rat.txt` for each submodule. missing license notes are indicated using the key `!???`.
2	Collates all the errors.

Inspect the contents of the two /tmp/rat-*.txt files and fix any reported violations, typically by either:

adding genuinely missing license headers from Java (or other) source files, or
updating the <excludes> element for the apache-rat-plugin plugin to ignore test files, log files and any other non-source code files

Also, look to remove any stale <exclude> entries

Once you’ve fixed all issues, run the script again to confirm that all license violations have been fixed.

Missing License Check

Although Apache Isis has no dependencies on artifacts with incompatible licenses, the POMs for some of these dependencies (in the Maven central repo) do not necessarily contain the required license information. Without appropriate additional configuration, this would result in the generated DEPENDENCIES file and generated Maven site indicating dependencies as having "unknown" licenses.

Fortunately, Maven allows the missing information to be provided by configuring the maven-remote-resources-plugin. This is stored in the src/main/appended-resources/supplemental-models.xml file, relative to the root of each releasable module.

pushd bom
mvn -Dgithub clean install -T1C -DskipTests (1)
mvn -Dgithub license:download-licenses      (2)
popd
groovy scripts/checkmissinglicenses.groovy  (3)

1	builds the framework. It’s necessary to have built the framework locally at least once (ok to skip tests). The `-Dgithub` activates the "github" profile which references the exact same `<module>`s as the official "apache-release" profile.
2	Captures the missing license information. The Maven plugin creates a `license.xml` file in the `target/generated-resources` directory of each module.
3	Reports on the missing license information The script searches for these `licenses.xml` files, and compares them against the contents of the `supplemental-models.xml` file.

For example, the output could be something like:

licenses to add to supplemental-models.xml:

[org.slf4j, slf4j-api, 1.5.7]
[org.codehaus.groovy, groovy-all, 1.7.2]

licenses to remove from supplemental-models.xml (are spurious):

[org.slf4j, slf4j-api, 1.5.2]

If any missing entries are listed or are spurious, then update supplemental-models.xml and try again.

Reconcile isis-bom pom

The bom/pom.xml is a customisation of the org.apache:apache pom, with each section of customisation clearly identified.

Check to see if there has been a new version of org.apache:apache; if so, merge in the changes.

Update the project build timestamp

For reproducibility, the project.build.outputTimestamp property must be updated to a hard-coded value.

Locate this property in bom/pom.xml and update to the current date/time.

Update and preview website

The next step is to generate the website, ensuring that the config, examples, projdoc (system overview and global index) are all updated.

Be careful that sometimes the configuration property pages (that are generated from the core/config/target/classes/META-INF/spring-configuration-metadata.json file) may be malformed; in particular for default values going missing.

This seems to occur if the IDE (IntelliJ) has compiled the codebase.

The workaround is to rebuild the core/config Maven module using mvn command line first.
```
mvn clean install -pl core/config -DskipTests
```

It’s also necessary to make sure the tooling is built:

mvn -D module-tooling -D skip.essential install -DskipTests

Then, generate the website:
```
sh preview.sh
```

Check for any Asciidoc errors, and fix. Also double-check that the config property files are correctly formatted.

Releasing the Framework

Set environment variables

We use environment variables to parameterize as many of the steps as possible. For example:

export ISISJIRA=ISIS-9999                           (1)
export ISISTMP=/c/tmp                               (2)
export ISISREL={page-isisrel}                             (3)
export ISISRC=RC1                                   (4)
export ISISBRANCH=release-$ISISREL-$ISISRC
export ISISART=isis
env | grep ISIS | sort

1	set to an "umbrella" ticket for all release activities. (One should exist already, created at the beginning of the development cycle now completing).
2	adjust by platform
3	adjust as required
4	adjust as necessary if this is not the first attempt to release

The branch name is intentionally not the same as the eventual tag names (eg isis-{page-isisrel}).

Commit changes

Commit any changes from the preceding steps:

git add ..
git commit -m "$ISISJIRA: updates to pom.xml etc for release"

Create a release branch and worktree

The release is performed on a branch; if we are successful, this branch will be merged back into master.

We also recommend performing this work in a separate git worktree. The original worktree can stay with the master branch and be used for documentation fixes, etc.

create (but don’t checkout) a release branch for the version number being released; eg:
```
git branch $ISISBRANCH
```

Create a worktree for this branch:

git worktree add ../isis-release $ISISBRANCH

Switch to the new worktree and push the branch:

cd ../isis-release
git push origin $ISISBRANCH -u

Switch to the `bom` directory

The remainder of the release process is performed from the context of the bom directory:

cd bom

Bump projects to `$ISISREL`

We use mvn versions:set to manually bump the release version. (After release, there is a similar step at the end to reset back to a -SNAPSHOT version).

(Still in the bom directory), the steps are:

First we bump the framework’s pom.xml files:
```
mvn versions:set -DnewVersion=$ISISREL
```
next we build the framework so that these versioned pom.xml files are available for the next step:
```
mvn install -o -DskipTests -T1C -Dgithub
```
The -Dgithub property activates the "github" profile that references all modules to be released.

next we also update the starters (as used by applications built using the framework):

pushd ../starters
mvn versions:set -DnewVersion=$ISISREL
mvn install -DskipTests -o
popd

finally we commit the changes:

git add ..
git commit -m "$ISISJIRA: bumps version to $ISISREL"

Sanity check

Perform one last sanity check on the codebase. Delete all Isis artifacts from your local Maven repo, then build using the -o offline flag.

Still in the bom directory:

rm -rf ~/.m2/repository/org/apache/isis
mvn clean install -o -T1C -Dgithub
git clean -dfx ..

Deploy

Since the <version> has already been updated, we just use mvn deploy to upload the artifacts. We activate the (inherited) apache-release profile to bring in the gpg plugin for code signing.

Prerequisites

Make sure that the version of gpg on the $PATH is correct, because the configuration in the pom.xml assumes gpg v2.0, rather than v2.1+.

The older version of gpg picks up the secret keys from pubring.gpg and secring.gpg (and does not use an agent); the newer version uses pubring.pbx.

On Windows, be careful using git-bash as this will pick up git’s version of gpg in /usr/bin, which is gpg v2.1. Instead, consider building in a Powershell console, described below.

If on Windows, we recommend you build under Powershell rather than git-bash, which means there are some prerequisites.

install the correct version of choco:

choco install gnupg -version 2.3.7

so that

gpg --version

returns:

gpg (GnuPG) 2.0.30 (Gpg4win 2.3.4)
...
Home: C:/Users/xxx/AppData/Roaming/gnupg
...
Compression: Uncompressed, ZIP, ZLIB, BZIP2

ensure your public key is installed in pubring.gpg.

For example:

gpg --list-keys

returns:

C:/Users/dan/AppData/Roaming/gnupg/pubring.gpg
----------------------------------------------
...
pub   4096R/77AD2E23 2011-02-01
uid       [ultimate] Dan Haywood (CODE SIGNING KEY) <danhaywood@apache.org>
...

ensure your secret key is installed in secring.gpg

For example:

gpg --list-secret-keys

returns:

C:/Users/dan/AppData/Roaming/gnupg/secring.gpg
----------------------------------------------
...
sec   4096R/77AD2E23 2011-02-01
uid                  Dan Haywood (CODE SIGNING KEY) <danhaywood@apache.org>
...

in ~/m2/settings.xml, ensure that the gpg.passphrase server is set:

~/.m2/settings.xml

<settings>
  ...
  <servers>
    <server>
      <id>gpg.passphrase</id>
      <passphrase>XXXX</passphrase>
    </server>
    ...
  </servers>
</settings>

Perform the deploy

The build creates a zip of the directory, so before executing the release we remove any other files.

Still in the bom directory, we deploy (upload the artifacts) using:

mvn deploy -Dapache-release -Dgit

Tag the Release

Finally, tag the release:

git tag $ISISART-$ISISREL --force
git tag $ISISART-$ISISREL-$ISISRC --force

Check/Close Staging Repo

The mvn deploy commands will have uploaded all the release artifacts into a newly created staging repository on the ASF Nexus repository server.

Log onto repository.apache.org (using your ASF LDAP account):

And then check that the release has been staged (select staging repositories from left-hand side):

If nothing appears in a staging repo you should stop here and work out why.

Assuming that the repo has been populated, make a note of its repo id; this is needed for the voting thread. In the screenshot above the id is org.apache.isis-008.

After checking that the staging repository contains the artifacts that you expect you should close the staging repository. This will make it available so that people can check the release.

Press the Close button and complete the dialog:

Nexus should start the process of closing the repository.

All being well, the close should (eventually) complete successfully (keep hitting refresh):

The Nexus repository manager will also email you with confirmation of a successful close.

If Nexus has problems with the key signature, however, then the close will be aborted:

Use gpg --keyserver hkp://pgp.mit.edu --recv-keys nnnnnnnn to confirm that the key is available.

Unfortunately, Nexus does not seem to allow subkeys to be used for signing. See Key Generation for more details.

Reset `revision` property

At the beginning of the release process we bumped the version to the release version, ie $ISISREL. With the release now deployed we now need to reset the revision back down to the base snapshot, ie 2.0.0-SNAPSHOT.

Therefore, still in the bom directory:

mvn versions:set -DnewVersion=2.0.0-SNAPSHOT
mvn install -DskipTests -o -T1C -Dgithub

pushd ../starters
mvn versions:set -DnewVersion=2.0.0-SNAPSHOT
mvn install -DskipTests -o
popd

git add ..
git commit -m "$ISISJIRA: resetting version"

Push branch & tag

Push the release branch to origin:

git push -u origin $ISISBRANCH

and also push tag:

git push origin refs/tags/isis-$ISISREL:refs/tags/isis-$ISISREL-$ISISRC
git fetch

The remote tags aren’t visible locally but can be seen online.

Update starter apps

For each of the two starter apps, we maintain four branches:

jdo and jpa

These are intended to reference the most recently released version, the first demonstrating persistence using JDO, the second using JPA. These are the branches referenced from the home page and getting started pages of the website.
jdo-SNAPSHOT and jpa-SNAPSHOT

These reference the most current snapshot nightly build.

The general idea is that a release will fast-forward jdo to jdo-SNAPSHOT and similarly moves jpa up to jpa-SNAPSHOT, bumping to the newly released version of the framework in the process.

In order that we don’t break the starter apps while a release is being voted on, we do the changes in work branches, $ISISBRANCH-jdo and $ISISBRANCH-jpa:

HelloWorld

For helloworld, we create a release branch for both variants:

for jpa:
- Checkout the branch, bump versions, and commit:
  git checkout jpa-SNAPSHOT git pull --ff-only git checkout -b $ISISBRANCH-jpa mvn versions:update-parent -DparentVersion=$ISISREL (1) mvn versions:set -DnewVersion=$ISISREL git add . git commit -m "$ISISJIRA - updates to $ISISREL (jpa)"
  1 the top-level pom.xml references the ASF staging repository, so this will pull down the release if not already present in ~/.m2/repository.
- Test the app
  mvn clean install mvn spring-boot:run
- Make any additional changes that might be required (eg update to menubars.layout.xml) commit and retest
- Push the branch to origin:
  git push -u origin $ISISBRANCH-jpa
for jdo:
- Checkout the branch, bump versions, and commit:
  git checkout jdo-SNAPSHOT git pull --ff-only git checkout -b $ISISBRANCH-jdo mvn versions:update-parent -DparentVersion=$ISISREL mvn versions:set -DnewVersion=$ISISREL git add . git commit -m "$ISISJIRA - updates to $ISISREL (jdo)"
- Test the app
  mvn clean install mvn spring-boot:run
- Make any additional changes that might be required, and re-test.
  
  This might be easiest to just cherry-pick commits made on the jpa branch.
- Merge in the jpa branch, discarding any conflicts.
  git merge $ISISBRANCH-jpa --no-commit
  The idea here is to show equivalence between the two branches. The merge commit should end up with no changes:
  git commit -m "$ISISJIRA - merge in jpa"
- Push the branch to origin:
  git push -u origin $ISISBRANCH-jdo

SimpleApp

For simple app, the steps are almost the same:

for jpa:
- Checkout the branch, bump versions, and commit:
  git checkout jpa-SNAPSHOT git pull --ff-only git checkout -b $ISISBRANCH-jpa mvn versions:update-parent -DparentVersion=$ISISREL (1) mvn versions:set -DnewVersion=$ISISREL git add . git commit -m "$ISISJIRA - updates to $ISISREL (jpa)"
  1 the top-level pom.xml references the ASF staging repository, so this will pull down the release if not already present in ~/.m2/repository.
- Test the app
  mvn clean install mvn -pl webapp spring-boot:run
- Make any additional changes that might be required (eg update to menubars.layout.xml) commit and retest
- Push the branch to origin:
  git push -u origin $ISISBRANCH-jpa

for jdo:

Checkout the branch, bump versions, and commit:

git checkout jdo-SNAPSHOT
git pull --ff-only

git checkout -b $ISISBRANCH-jdo

mvn versions:update-parent -DparentVersion=$ISISREL
mvn versions:set -DnewVersion=$ISISREL

git add .
git commit -m "$ISISJIRA - updates to $ISISREL (jdo)"

Test the app

mvn clean install
mvn -pl webapp spring-boot:run

Make any additional changes that might be required, and re-test.

This might be easiest to just cherry-pick commits made on the jpa branch.
Merge in the jpa branch, discarding any conflicts.
```
git merge $ISISBRANCH-jpa --no-commit
```
The idea here is to show equivalence between the two branches. The merge commit should end up with no changes:
```
git commit -m "$ISISJIRA - merge in jpa"
```
Push the branch to origin:
```
git push -u origin $ISISBRANCH-jdo
```

Preview website

We also prepare a preview of the next version of the website, then made accessible from https://isis.staged.apache.org.

Prerequisites:
- clone the https://github.com/apache/isis-site repo, alongside the isis repo:
  git clone https://github.com/apache/isis-site ../isis-site
- in the isis-site repo, check out the asf-staging branch:
  cd ../isis-site git checkout asf-staging git pull --ff-only
still in the isis-site repo, delete all the files in content/ except for the schema and versions directories:
```
pushd content
for a in $(ls -1 | grep -v schema | grep -v versions)
do
    rm -rf $a
done
popd
```

Back in the isis repo’s release worktree, generate the Antora site (from the top-level directory).

cd ../isis-release

sh preview.sh -A

This procedure doesn’t yet work; Antora does not yet support worktrees.

So instead it’s currently necessary to perform the preview in the main ../isis directory, switching branches. You’ll also need to rebuild all and then regenerate config files and tooling:

git checkout $ISISBRANCH
pushd bom
mvn clean install -DskipTests -T1C -Dgithub
popd
mvn clean install -pl core/config
mvn -D module-tooling -D skip.essential install -DskipTests

Copy the generated Antora site to isis-site repo’s contents directory:
```
cp -Rf antora/target/site/* ../isis-site/content/.
```

Back in the isis-site repo, commit the changes and preview:

cd ../isis-site

git add .
git commit -m "$ISISJIRA : staging changes to website"

sh preview.sh

If happy, then push the changes:
```
git push origin asf-staging -u
```

Wait a minute or two; the site should be available at https://isis.staged.apache.org (nb: 'staged', not 'staging').

Voting

Once the artifacts have been uploaded, you can call a vote.

In all cases, votes last for 72 hours and require a +3 (binding) vote from members.

Start voting thread on dev mailing list

That is, mailto:dev@apache.isis.org

The following boilerplate is for a release of the Apache Isis Core. Adapt as required:

Use the following subject, eg:

[VOTE] Apache Isis Core release {page-isisrel} RC1

And use the following body:

I've just cut a new release of the Apache Isis Framework.

The source code zip artifact has been uploaded to a staging repository on
https://repository.apache.org, along with its corresponding .asc signature.

In the source code repo the code has been tagged as isis-{page-isisrel}-RC1;
see https://github.com/apache/isis/tags

To verify the source code itself, you can use the following commands
(in an empty directory):

----------------------------------------------------------------
VERSION={page-isisrel}
RC=RC1
NEXUSREPONUM=11xx

curl https://downloads.apache.org/isis/KEYS > /tmp/KEYS
gpg --import /tmp/KEYS

rm -rf isis-$VERSION

curl -O -L https://raw.githubusercontent.com/apache/isis/release-$VERSION-$RC/scripts/verify-isis-release.sh

chmod +x ./verify-isis-release.sh
./verify-isis-release.sh $NEXUSREPONUM $VERSION $RC
----------------------------------------------------------------

You can then test the helloworld or simpleapp starter apps, see:
https://isis.staged.apache.org/comguide/latest/verifying-releases.html.

You can also inspect the website in general, available at:
https://isis.staged.apache.org.

Please verify the release and cast your vote.
The vote will be open for a minimum of 72 hours.

[ ] +1
[ ]  0
[ ] -1

Remember to update:

the version number ({page-isisrel} or whatever)
the release candidate number (RC1 or whatever)
the NEXUSREPONUM to the repository id as provided by Nexus earlier (11xx or whatever)

Note that the email also references the procedure for other committers to verify the release.

Cutting a Release

Preparation

Obtain Consensus

Pull down code to release

License headers

Missing License Check

Reconcile isis-bom pom

Update the project build timestamp

Update and preview website

Releasing the Framework

Set environment variables

Commit changes

Create a release branch and worktree

Switch to the bom directory

Bump projects to $ISISREL

Sanity check

Deploy

Prerequisites

Perform the deploy

Tag the Release

Check/Close Staging Repo

Reset revision property

Push branch & tag

Update starter apps

HelloWorld

SimpleApp

Preview website

Voting

Start voting thread on dev mailing list

Switch to the `bom` directory

Bump projects to `$ISISREL`

Reset `revision` property