Introductory Guide

Emdebian seeks to provide minimised Debian packages, with the same sort of consistency that Debian itself offers, but a much smaller footprint, a complete Debian-based environment in which to build and customise them and complete distributions for various devices or types of devices.

This guide covers general topics relating to crossbuilding Debian, organised as a series of hints and tips. Other guides cover starting out, fixing problems when crossbuilding, preparing a root filesystem and possible problems of mixing Debian and Emdebian packages.

Crossbuilding Debian

The em1 version suffix.
debian-xcontrol.
source autodetection in ./configure.
Omitting make check.
Cache files
Modifying files in the debian/ directory.
Modifying files outside the debian directory.
Changes for CDBS.
X-Build-Cross-Libtool: yes.
Handling udebs.
Modifying maintainer scripts.
Easier patch handling with meld.
Filing bugs for crossbuild support in Debian packages.

Version suffix

The emdebian scripts implement one of the methods of preparing packages suitable for embedded devices, called the composite method, reliant on an emdebian version suffix: 'emN' : the 'em' prefix and a sequential digit.

In common with Debian versioning itself, the emdebian version resets to 1 each time a new package is available. Emdebian considers Debian as upstream so a new Debian version is still a new package as far as emdebian is concerned.

The suffix separates the emdebian archive files from other Debian archives and associated files being built from the same source tree on the developer system and identifies the archive as an adapted, stripped out, archive that is not necessarily compatible with any other .deb or .udeb files or programs. The method allows emdebian developers to have a version number that is independent of the full Debian version string.

Omitting `make check`.

Cross-building involves preventing certain architecture-specific elements of a normal build from running on the host when compiled for the target architecture. Other methods use emulators or native environments but these are often limited to specific architecture combinations. During ./configure, macros like AC_TRY_* therefore need to be passed cached values using an architecture-specific cache file. Values for the cache file should be obtained from the Debian native buildd logs. em_make patches debian/rules to omit make check which would also attempt to run cross-built programs on the host architecture. Creating $arch.cache files needs to be done by hand - each time the build fails during configure.

Auto detection by `./configure`.

Some packages enable components if a particular dependency is located at build time. This auto detection is useful but if the package lacks a working option to ./configure that can --disable the component, this is a bug in the package. Report the problem under the long term mass bug filing for cross build support and add a note that such systems risk causing native bugs by generating different builds on a developer system compared to a buildd. As a workaround, remove these dependencies from debian/xcontrol to ensure the relevant feature is disabled. When the bug is fixed, these dependencies will need to be added back.

Although debian/xcontrol simplifies various elements of changing the dependency chain of packages for embedded systems, the debian-xcontrol package expects the Debian maintainer to edit debian/xcontrol and use xcontrol to update debian/control prior to the native build. Future versions of emdebuild may incorporate checks on whether debian/control is in sync with debian/xcontrol.

Using `debian/xcontrol` with `xcontrol`.

emdebuild initially improvised a Build-Cross-Depends: field in debian/xcontrol.

Build-Depends: libsqlite0-dev, libglib2.0-dev, libgpewidget-dev
Build-Depends-Tools: cdbs, debhelper (>= 5), autotools-dev, automake1.9, dpkg-dev (>= 1.13.19)

instead of:

Build-Depends: cdbs, debhelper (>= 5), autotools-dev, libsqlite0-dev, libglib2.0-dev,
 dpkg-dev (>= 1.13.19), libgpewidget-dev, automake1.9

Only packages that need to be installed with apt-cross are now listed in Build-Depends and packages that only need to be installed on the build machine (including all packages that are Architecture: all) go into Build-Depends-Tools. xcontrol build can then be used to integrate the two sections for normal Debian builds.

Using $architecture.cache files.

Some packages build test programs that are compiled and executed during ./configure and these programs will inevitably fail when cross building because the test is compiled to run on the target architecture, not the build architecture. Unfortunately, the autotools are not currently able to handle this situation automatically. Specific tests can be omitted by specifying cached values from a native build on the target architecture. These values can be obtained by inspecting the Debian buildd logs for the architecture involved.

An example cache file (from libglib2.0-0 on arm) shows the format:

glib_cv_stack_grows=no
glib_cv_monotonic_clock=yes
glib_cv_uscore=no
ac_cv_func_posix_getpwuid_r=yes

Variable names can be obtained from careful inspection of the ./configure script at the point at which the cross-build fails. Alternatively, build a normal Debian version of the package using ./configure --cache-file=$file and lookup the variable names from that file. Only use the variables that are necessary to allow the cross-build to complete in the cache file so that other values can be calculated normally. Scripts to help this process would be welcome.

To use the cache file, place it in the top level source directory of the package (same directory as ./configure). If the package uses the internal tarball method (where dpkg-source unpacks a debian directory and leaves one or more tarballs containing the upstream source), create a patch for debian/patches that creates the file in the top level directory of the build-tree. The cache file should still be present in the top level source directory so that emdebuild can maintain the relevant emdebian patch file. Then edit debian/rules to add this line to the options passed to ./configure:

--cache-file=$(DEB_HOST_GNU_TYPE).cache

(ensure you specify HOST not BUILD).

Modifying other files in the debian directory.

Sometimes, other files need to be modified in the debian/ directory, besides rules, changelog, control and control.in (if present). Some of these files are modified automatically during the build (e.g. the files list, shlibs files and temporary directories). To get a list of files that have been modified to create a successful build use:

$ interdiff -z PACKAGE_VERSION.diff.gz PACKAGE_VERSIONem1.diff.gz | \
 grep "+++" | grep debian | grep -v locale | grep -v rules | \
 grep -v control | grep -v changelog

If you have had to edit a file in the debian/ directory, subsequent builds will need the same changes so a secondary emdebian patch file is needed. emsource version 0.2.2 or later will back up all files in the debian/ and debian/patches/ directories to ../$package.old/.

emdebuild will check all files in the debian/ directory compared to the versions in ../$PACKAGE.old/ and create the necessary emdebian patch files.

Modifying files outside the debian directory.

Wherever possible, Emdebian changes should be limited to files in the debian directory, in particular debian/rules. Currently, it is preferable to let the build process run as normal, even if this means generating documentation or other content that is destined to be removed before the package itself is built. If the package supports a --no-docs or --disable-docs option to ./configure then it should be used. If not, simply remove the unwanted content by adding a rule to debian/rules rather than trying to patch or modify Makefiles or similar. If you still need to modify files outside the debian/ directory, use patches in debian/patches/. For CDBS packages, add the simplepatchsys rules:

include /usr/share/cdbs/1/rules/simple-patchsys.mk

For debhelper and dpkg packages, use dpatch - implementing dpatch support in debian/rules if necessary. See : HowTo use dpatch

Patch files created for the debian/patches/ directory should follow the naming conventions for that directory and must not use the emdebian- prefix. However, it is advisable to use some form of emdebian "tag" for the file so that future upstream versions of the package do not inadvertently use the same filename. You must also ensure that all upstream patches that may affect the relevant files are applied before the emdebian patch. As many patch systems use a numerical prefix to determine the sequence of applying patches, one suggestion is to use filenames like 99_emdebian_avoid_running_foo. Remember that these patches will be applied by debian/rules, not by emdebuild. emdebuild checks for new files in debian/patches and then creates a copy of the patch file for emdebian SVN (called debian-patch-$file, ready to be copied into debian/patches by emsource.

If the package already uses patches via CDBS, nothing needs to be done.

Changes for CDBS.

Versions of dpkg-cross prior to 2.0.0 used overrides in the MAKEFLAGS environment variable that conflicted with the needs of packages like libx11 that need to be able to build certain internal tools using the native compiler. CDBS tends to rely on this override mechanism and many CDBS packages now fail to use the cross-compiler because CDBS retains an internal override that is no longer cleared by dpkg-buildpackage. To reset CDBS to not set the override use the following in debian/rules:

DEB_CONFIGURE_SCRIPT_ENV :=

When Debian bug #450483 is fixed, this change will no longer be required.

CROSS-BUILDING DEBIAN

Gradually, changes are being made to the Debian package building tools to make cross building easier in Debian. Some problems remain and emdebuild uses three main workarounds:

Packages can implement their own support for cross building their internal objects by specifying -l flags to libtool:

Specifying relinking path for internal objects in CDBS.

This example is from libqof1.

ifneq ($(DEB_BUILD_GNU_TYPE),$(DEB_HOST_GNU_TYPE))
DEB_MAKE_INVOKE+="QOF_LIBS=-L/usr/$(DEB_HOST_GNU_TYPE)/lib -lqof"
endif

Specifying relinking path for internal objects via $(MAKE).

For an example package, FOO using -lfoo during native relinking and passing ${FOO_LIBS} for libfoo_la_LIBADD in the upstream Makefile.am.

ifneq ($(DEB_BUILD_GNU_TYPE),$(DEB_HOST_GNU_TYPE))
RELINK="FOO_LIBS=-L/usr/$(DEB_HOST_GNU_TYPE)/lib -lfoo"
endif

Upstream may need to be patched.

Note that upstream needs to support a variable already - if the upstream Makefile.am hardcodes -lfoo then a patch will be needed to replace this value with a suitable variable.

These problems will also be reported under the long term mass bug filing for cross building support agreed on debian-devel in November 2007.

Handle udeb packages intended for the Debian Installer.

Emdebian does not, yet, use udeb packages because the Emdebian installation process is handled separately. However, most udeb packages can be cross-built and the availability of udeb packages will assist when the time comes to fold the emsandbox tarball into the Debian installer to make the installation process itself easier, e.g. using D-I to setup the root password, users in a pleasant GUI environment. Whether this will work or not is unknown.

Maintainer scripts.

Maintainer scripts include preinst, postinst, prerm and postrm package scripts that handle the installation, upgrade, removal and purge process for each package. Source packages can contain separate maintainer scripts for each binary package built. For information on maintainer scripts, see Debian Policy.

Many maintainer scripts will need patches for Emdebian, reasons include (but are not limited to):

perl

Maintainer scripts are often written in shell but may use perl or any interpreter if the dependencies are handled appropriately. Scripts that are written in perl will need to be reimplemented in shell, C or simply removed.
alternatives

Maintainer scripts may call update-alternatives with details of commands, man pages and info pages - these calls need to be removed until a non-perl form of update-alternatives is available.
install-info

Maintainer scripts may call install-info to put info pages into usable locations. Without info pages in the package, install-info will fail so these calls must be removed.
backwards compatibility

Maintainer scripts are often written to handle issues that only arise when upgrading from an old version to a fixed version. These sections can be removed from the Emdebian scripts, especially if any of the above problems exist in these sections or if the compatibility code makes the script particularly long.

At present, emdebuild does only basic checks on the maintainer scripts.

Easier patch handling.

meld simplifies the process of handling the emdebian patch files. An example usage may help:

$ meld ../apt.old/debian/rules debian/rules

meld allows you to migrate a line from the old file back into the patched file as well as showing you exactly which parts of long lines have changed. Note that although meld can produce a new patch from the loaded files, meld should be started from the directory above to ensure that patch has no problems applying the results:

$ pushd ../
$ meld apt.old/debian/rules apt-0.7.6/debian/rules
$ popd

Another use of meld is in preparing patches for bug reports against the Debian package.

Using meld to generate patches for Debian bugs.

The difficulty in preparing suitable patches to support cross-building in Debian is that the Emdebian patches necessarily remove package components that Debian requires by Policy. Some Emdebian changes are handled within the package, some within other tools and the patch files need to be split accordingly.

meld can assist by migrating sections of a patch between different versions of the relevant files, typically debian/rules.

$ cp $package-$version.orig.tar.gz ../branches/
$ cp $package_$version.dsc ../branches/
$ mkdir -p ../branches/$package.new/debian
$ cp $package-$version/debian/rules ../branches/$package.new/debian/rules
$ cd ../branches
$ dpkg-source -x $package_$version.dsc
$ meld $package-$version/debian/rules $package.new/debian/rules &

The $package.new directory provides a usable path so that the final patch is easier to apply.

Identify the components of the patch that are relevant to the package, rather than the tools (e.g. changes to support --build and --host or package changes to support nocheck DEB_BUILD_OPTIONS etc.) and use the arrows in the centre of the meld window to move other changes into rules.emdebian so that the temporary rules.emdebian file represents a midstage between the original debian/rules and the Emdebian patched debian/rules in ../trunk/$package-$version/.

Note that some Debian maintainers prefer to have separate patches for separate issues, even if the patches apply to the same file. Others may agree to have one patch for one file but would prefer to have two bug reports for the two issues. patch will usually work out how to offset one patch to take into account lines added or removed by another.

Once you only have the changes you need for this bug report, right click the rules.emdebian window and select "Create patch" from the context menu. Apply this patch and test the build using normal Debian build tools like debuild or dpkg-buildpackage -rfakeroot. In the majority of cases, Emdebian changes can be restricted to the files in the debian/ directory and there is no need to use dpatch or other systems to apply these patches. Just use the patch command directly. If the patch works and does not change the normal Debian build behaviour or package contents, file a wishlist bug against the package quoting the long term mass bug filing for cross building support and specify the usertag either in the body of the email: (See Additional Pseudoheaders for the Debian BTS). In reportbug, you can attach the patch direct from the command line with reportbug -A FILE if the diff is large or reportbug -i FILE to include the diff within the text of the report.

See also: Current list of cross build support bugs and the Emdebian wiki support: Emdebian QuickStart and The Emdebian Developer Guide

Back to the Emdebian Project homepage.