Developper reference for building and packaging

Building a distribution

Building a distribution consists of the building of both the compiler & the different engines. The distribution must meet the following goals:

• contains only the code we want to distribute (ie. no source code)
• requires as less installation steps as possible for the user
• ability to keep track (version) of distributed files

The distribution scripts involve several steps:

• compute a distribution version number and inject it in all following steps
• compile the compiler: it uses ant/ivy
• compile the different engines (eg. reference engine, optimized engine)
• provide install/setup script
• package compiler & engines in archives to be distributed

All these steps are scheduled from the script called wrap.sh located, as everything related to the distribution, in the distribution directory.

Important

Please be aware that all steps needed in the release process are not automated in scripts. You still need to do manual steps (see below) in order to build a complete release. Failing to do so will lead to incoherence and most probably headaches for solving problems.

Invoking wrap.sh script

The wrap script accepts several command line parameters:

-r              When building a revision to release :)
-v              Give the version name instead of it being generated
-p              Build profile for engine (default: Release)
-s              Skip directly to engine building, no compiler
-h              This help

If the -r flag is used, then the version used throughout the build uses the pattern YYYY.MM. It has the nice property of being easy to compute, to understand and is always increasing.

The -v allows the specification of the full version string. It has priority over -r.

If none of -v and -r are used, then the pattern used is YYYY.MM.HHmmSS-DEV.

The -s can be used to skip the compiler compilation, which is responsible for most of the build time. The remaining steps are still executed.

The last parameter, -p, is used to set the cmake build profile used when building the engines. Default is Release, meaning that the code may be optimized and debugging symbols stripped. If you need to build a debugging release, use the Debug profile. Please note that this profile is only used for building the engines, it has nothing to do with the build of the code that may be generated by the bip compiler later.

What wrap.sh does

It starts by cleaning everything : its own build directory along with the separate build directories for the difference parts of the compiler. And immediately starts the building of all parts of the compilers. It does that by using ant with the following targets:

• clean: to remove previous build artefacts
• publish-local-all: build the compiler

The publish-local-all target is a frontend target that uses ivy.

After the compiler has been successfully built, the wrap script continues by copying the resulting artefacts (jar files) and the frontend script that will be used by the users (ie. bipc.sh).

At this point, the bip compiler is ready. The wrap script now compiles the different engines found in the Engines/ directory. The result of the engine compilation is directly a self-contained archive.

What wrap.sh produces

After everything has been executed, you can find the distribution in the build/ directory:

build
+── bipc-2012.04.110853-DEV.FILES
+── bipc_2012.04.110853-DEV.tar.gz
+── BIP-optimized-engine-2012.04.110853-DEV_Linux-i686.tar.gz
`── BIP-reference-engine-2012.04.110853-DEV_Linux-i686.tar.gz

The .FILES file should be kept as it contains all the filenames included in the compiler distribution archive. It is useful, as the compiler contains several external dependencies whose versions are not encoded in the distribution version.

Single archive distribution using single-archive-dist.sh

For even easier distribution, the single-archive-dist.sh script can be used. It produces a single archive with the compiler and the engines inside. Installation is only a matter of extracting and running a script that sets up the environment correctly. It relies on wrap.sh and simply rearrange the products of the latter. The script accepts only -r and -v command line parameters, which are exactly the same as for wrap.sh.

The result of running this script is a single archive called bip-full_<ARCH>.tar.gz. It contains the compiler and the engines. It also has a setup.sh script that can be used to setup the environment correctly. By default, the script configures the environment for using the reference-engine, but you can give it any engine (provided it is shipped in the archive) name:

$ . ./setup.sh optimized-engine
Environment configured for engine:  optimized-engine
$ . ./setup.sh
Environment configured for engine:  reference-engine

Do not forget the leading . in the command above.

Publishing a distribution

Publishing a distribution must always includes the following steps:

1. compute a new version name. You must never reuse a previous version name. Never! If you have published a distribution with a huge bug inside, it’s already too late, you need a new version. Commonly, a version has a major part and a minor or revision part. The major corresponds to a new release, and the minor can be changed when you want to distribute a new revision for the same release (eg. with bug fixes).
2. build the compiler using the previously computed version name
3. build the engines using the previously computed version name
4. build the document. The documentation should always match the distributed software. Never provide outdated documentation. If you can’t update the documentation (it’s a shame), outdated part must be explicitly marked.
5. tag all the different parts of the software, script, documents in the SCM (ie. subversion).
6. publish !

The scripts presented in previous paragraphs can help for steps 1,2,3. Steps 4,5 and 6 must be done carefully by hand.

Manual steps

Building the documentation

Building the documentation is simple. 2 documentations can be built and published: APIs & user/dev documentations. As of this writing, only the user/dev documentation is published though, as APIs still need some work (they are based on javadoc & doxygen).

The user/dev documentation are using Sphinx.

First, you need to configure the documentation build to include the correct version number and release name. Edit the file source/conf.py to include the matching version/name:

# The short X.Y version.
version = '2012.04'
# The full version, including alpha/beta/rc tags.
release = '2012.04 (RC3)'

Building is as easy as running:

$ make html latexpdf

The targets are self-described and produce static HTML pages and a PDF. The script sync-to-www.sh provides an easy way to build and publish the user/dev documentations. The script also creates a tree-hugger-friendly PDF with 2 pages per side and publishes the example files.

Tagging

You must tag all parts (compiler, engines, documentation, distribution scripts), even the ones that have not moved since the previous release. use the tag command:

$ svn tag version-name

Of course, replace version-name by the release version name. You must repeat this operation for all svn module that need to be tagged.

Publishing

Publishing means copying files in the web directory. The sync-to-www.sh moves the documentations in the doc/ subdirectory:

/doc
 +── examples
 +── html
 `── pdf

Compiler & engines releases must be copied by hand.

Things to keep in mind

Unless you know exactly what you are doing (and why!), you should:

• never commit u.v.b.userinterface.cli/src/main/java/u/v/b/userinterface/cli/Version.java : it is modified when building a distribution but these modifications should never make their way into the code repository.
• never commit any .project, .classpath or any other eclipse dot-files. Having local modifications is also often a sign of wrong configuration (but not always). Be careful to never commit these as this will break other developer setup.
• never commit Documents/sphinx-doc/source/conf.py if you’ve only changed the version/release information.