ReleaseProcedure

From Eigenpedia

Jump to: navigation, search

This page documents the procedure used to make Eigenbase source and binary releases. It covers Farrago, Fennel, and LucidDB; other subprojects (blackhawk, saffron, enki, JGraphT and resgen) have their own release procedures.

Contents

Version Numbering

Currently, we use a unified version numbering scheme across these projects, with no release codenames. In the future, individual projects may go on their own release tracks, but for now, we keep it simple.

The version numbering is x.y.z, e.g. 0.7.4.

So far, LucidDB has been driving the release process, since it has an actual user community as opposed to just a developer community. We have kept x=0 because there are still a lot of gaps to be filled before calling anything "1.0". We increment y for introduction of major new user-visible features or architectural changes, and z otherwise. Numbering has been linear so far, but in the future could be branching for maintenance of older releases.

In some cases, development prereleases may be put out before an official release. For this, the convention is to reserve a release number which will be skipped in the official release track, e.g. LucidDB 0.7.5 was a prerelease before the official 0.8.0 release, and 0.7.4 was the last official release on the 0.7 track. If the 0.7 track had continued, the next official release there would have been 0.7.6 (which we'll use as an example version number throughout this page).

Release versions should also be defined in JIRA (issues.eigenbase.org) so that issues can be assigned to releases as appropriate. Because Fennel, Farrago, and LucidDB are different projects in JIRA, it is necessary to create and manage a release definition for each one rather than just once.

Release Specific File Updates

Once the release number has been decided, a number of files need to be updated: 

Change 11941 by jvs@jvs.bagheera.eigenbase on 2008/11/10 13:25:57

        DEV:  version number and doc updates for 0.8.0 release

Affected files ...

... //open/dev/bootstrap/dist/README.bin#14 edit
... //open/dev/bootstrap/dist/README.src#13 edit
... //open/dev/farrago/README#16 edit
... //open/dev/fennel/README#20 edit
... //open/dev/fennel/configure.in#61 edit
... //open/dev/luciddb/doc/arch.html#15 edit
... //open/dev/luciddb/doc/features.html#9 edit
... //open/dev/luciddb/doc/roadmap.html#11 edit
... //open/dev/luciddb/license/README#13 edit

In some cases, this is just a matter of updating the release number (a step that really should be automated/generated, but for now it's manual via Perforce edits).

In other cases, it's necessary to add or change text in the file. For example, for LucidDB, we maintain a summary of changes in each release; this requires a human to scan through the Perforce changes since the last release and come up with a short description for each (including a JIRA reference from issues.eigenbase.org if any).

Besides bug fixes, new features and change impact notes, here are some other items to consider for release documentation:

  • have any significant new thirdparty dependencies been introduced?
    • if so, are there any unresolved licensing issues? are new GPL exceptions needed in the files named "COPYING" under fennel, farrago, or luciddb?
  • any special migration instructions?
  • any platform-specific limitations?
  • deprecation for existing features or addons?

Perforce Label Creation

After release-specific files have been updated, the next step in the release process is to create a Perforce label for the first release candidate. Before doing this, it is a good idea to make sure that:

  • all features intended for the release are good to go
  • all bugs of interest are fixed
  • any desired branch integrations to //open/dev have been performed
  • all tests of interest are passing

The label name should be eigenbase_rx.y.z, and is created with the p4 label command: 

p4 label eigenbase_r0.7.6

Initially, the label should be defined like this: 

Label:  eigenbase_r0.7.6

Update: 2008/07/16 09:47:58

Access: 2008/07/13 16:36:32

Owner:  jvs

Description:
        Eigenbase release 0.7.6 (candidate)

Options:        unlocked

View:
        //open/dev/...

This assumes that the release is being made directly off the main Eigenbase trunk. Alternatively, it may come from a release branch, e.g. //open/release/r0.7.4. Perforce is quite powerful for retroactive branching, so it is fine to start with a label on the trunk and then later create a branch for fixing problems with the release; you can move the label from trunk to branch at that time.

Once the label has been defined as above, run a command like this one to take the initial snapshot of the code: 

p4 labelsync -l eigenbase_r0.7.6

Release Build

The script //open/dev/bootstrap/dist/buildEigenbaseRelease.sh automates the entire release build process for a given platform. It expects you to have set up and tested a Perforce sandbox and build environment for the branch corresponding to the current label definition. Official builds should only be done with JDK 1.5 for now.

Run it as follows: 

buildEigenbaseRelease.sh eigenbase_r0.7.6 0 7 6

The script checks to make sure you don't have any files open for edit in your Perforce client, and fails immediately if you do. If that check passes, the next step taken by the script is to rm -rf most of the sandbox directories for the branch and then p4 sync -f to the label (to make sure the build is pristine), so make sure you don't have anything of value lying around in your Perforce sandbox directories!.

On Linux, the script requires the chrpath utility to be available, so be sure to install that.

If the script completes successfully, it will produce files as follows on Linux: 

eigenbase-0.7.6.tar.bz2
eigenbase-src-0.7.6.tar.bz2
fennel-0.7.6.tar.bz2
farrago-0.7.6.tar.bz2
luciddb-0.7.6.tar.bz2

On Windows, the files will have .zip extensions instead.

The files produced do not match the actual file release naming conventions, so it's necessary to rename them. The source packages can come from any OS build, so assuming the 32-bit Linux build is used for that, the renames are as follows for the 3 platforms currently supported:

  • Linux 32-bit
    • eigenbase-0.7.6.tar.bz2 -> eigenbase-bin-linux32-0.7.6.tar.bz2
    • eigenbase-src-0.7.6.tar.bz2 (no rename)
    • fennel-0.7.6.tar.bz2 -> fennel-src-0.7.6.tar.bz2
    • farrago-0.7.6.tar.bz2 -> farrago-src-0.7.6.tar.bz2
    • luciddb-0.7.6.tar.bz2 -> luciddb-bin-linux32-0.7.6.tar.bz2
  • Linux 64-bit (AMD)
    • eigenbase-0.7.6.tar.bz2 -> eigenbase-bin-linux64-0.7.6.tar.bz2
    • luciddb-0.7.6.tar.bz2 -> luciddb-bin-linux32-0.7.6.tar.bz2
  • Win32
    • eigenbase-0.7.6.zip -> eigenbase-bin-win32-0.7.6.zip
    • luciddb-0.7.6.zip -> luciddb-bin-win32-0.7.6.zip

(It would be nice if the script were enhanced to produce the correct filenames automatically.)

So far, OS/runtime dependencies for binary packages have been random based on whatever build machines have been available to Eigenbase. On Windows, this isn't a problem, but on Linux there is a large amount of variation in ABI's for different gcc versions. Eventually, we would like to have distro+version-specific packages (.debs, .rpms, etc); for now, users have to fall back to building from source in cases where the released binaries are incompatible with a particular distribution.

Release Candidate Validation

Once builds have been produced, they should get as much validation as possible before release. We don't currently have an automated release package testing process, so the suggested activities below are currently all manual.

  • Install the package (ideally on a clean machine not used for normal development/builds) and follow the setup and test instructions.
  • For LucidDB, test out upgrade from an installation of the previous release
  • Compare the size of the release package files against the previous release. If there is significant growth, make sure that it's expected (e.g. a new unneeded build-time dependency may have slipped into the distribution accidentally). If there is significant size reduction, figure out what has gone away and make sure that it isn't still needed.

For LucidDB, add-ons and clients should also be tested since these don't get exercised as part of the main build/test sequence due to dependencies:

If time permits, verifying fixes for issues in JIRA which have been marked as resolved for the release helps to ensure a quality release.

It's a good idea to farm out the validation work to as many volunteers as possible to get more testing before release. Here is an example from the LucidDB 0.7.5 prerelease. (Eventually nightly snapshot publication should be automated to reduce the effort and latency here.)

When issues are found which are considered urgent enough to require a fix for the release, a new release candidate must be produced. Once the fix has been checked in and integrated to the target branch, the label can be updated by repeating the original labelsync command. If the fix is mixed in with other changes on the branch, and you don't want to include those in the release, make sure to p4 sync selectively to get just the state you want before updating the label. (It's easy to make mistakes here, so for anything non-trivial, consider setting up a release branch for isolating fixes.) 

p4 labelsync -l eigenbase_r0.7.6

Then repeat the release build procedure and carry on with validating the new packages. Note that it may be tempting to stitch together "frankenbuild" packages to expedite a release, but best practice is to always execute the full automated build procedure and only use the output of that (and to use the same label definition for all platforms).

Suffixing package filenames (e.g. luciddb-bin-win32-0.7.6-rc2.zip) can be a good way to keep release candidates straight.

Label Lock

Once a golden release candidate has been finalized, lock the Perforce label to indicate that it should no longer be updated, since there will be no more release candidates. (Anything after this will be either a patch or a new release.) Use this command: 

p4 label eigenbase_r0.7.6

And change the label definition as follows: 

Label:  eigenbase_r0.7.6

Update: 2008/07/16 09:47:58

Access: 2008/07/13 16:36:32

Owner:  jvs

Description:
        Eigenbase release 0.7.6

Options:        locked

View:
        //open/dev/...

Sourceforge Release

Now it's time to create the releases in Sourceforge. This involves a large amount of pointing, clicking, and sftp, which we won't go into here since it's documented at http://alexandria.wiki.sourceforge.net/File+Release+System+-+Offering+Files+for+Download.

Conventions for Sourceforge package and release names are as follows:

  • package: release
  • eigenbase: eigenbase-0.7.6
  • fennel: fennel-0.7.6
  • farrago: farrago-0.7.6
  • luciddb: luciddb-0.7.6

When uploading files, be sure to fill in all the information such as file type and architecture (following the pattern from previous releases), otherwise the files won't show up as available for download.

Website Updates

The LucidDB and Eigenbase websites have news pages which need to be updated as part of the release. (It's not possible to do this up front when changing version numbers in files since the final release date typically isn't known at that time.) For LucidDB, update luciddb/doc/main.html; also update luciddb/doc/navigation.html with the new download links. For Eigenbase, update intra/eigenbase/htdocs/news.html.

If there are wiki pages in Eigenpedia describing new features pending, update them to indicate that they are now available in a specific release number.

Mailing List Announcements

Currently, announcements only go out for LucidDB releases, to luciddb-users. See http://n2.nabble.com/LucidDB-0.7.4-release-is-now-available-td1441449.html for an example.

Sourceforge News

Submit news articles for the release of each project, following the pattern from previous releases. Also be sure to use the file release system to alert any users who have subscribed to file release notifications.

JIRA Update

JIRA's "Manage Versions" feature includes an operation for marking a version as released. Do this for all projects on the actual release date. At this point, you may want to create future versions, and move any unresolved issues which were still assigned to the release which just completed.

TBD

  • patching, other maintenance
Retrieved from "http://pub.eigenbase.org/wiki/ReleaseProcedure"
Personal tools