|
|
Introductory GuideEmdebian seeks to provide minimised Debian packages, with the same sort of consistency that Debian itself offers, but a much smaller footprint, a complete Debian-based environment in which to build and customise them and complete distributions for various devices or types of devices. This guide covers general topics relating to crossbuilding Debian using emdebian-tools, organised as a series of hints and tips. Other guides cover starting out, fixing problems when crossbuilding, preparing a root filesystem and possible problems of mixing Debian and Emdebian packages. See also
Crossbuilding Debian
Version suffixThe emdebian-tools implement one of the methods of preparing packages suitable for embedded devices, called the composite method, reliant on an emdebian version suffix: 'emN' : the 'em' prefix and a sequential digit. In common with Debian versioning itself, the emdebian version resets to 1 each time a new package is available. Emdebian considers Debian as upstream so a new Debian version is still a new package as far as emdebian is concerned. The suffix separates the emdebian archive files from other Debian archives and associated files being built from the same source tree on the developer system and identifies the archive as an adapted, stripped out, archive that is not necessarily compatible with any other .deb or .udeb files or programs. The method allows emdebian developers to have a version number that is independent of the full Debian version string. Omitting make check.Cross-building with emdebian-tools involves preventing certain architecture-specific elements of
a normal build from running on the host when compiled for the target architecture. Other methods use emulators
or native environments but these are often limited to specific architecture combinations. During
./configure, macros like Autobuilding Emdebian.emsource supports the -p and -b options to begin an automated build using the emdebian patch files from SVN. emsource -b calls emdebuild which will create a package_version_arch.build log file. To get a complete log containing the relevant output from emsource, use the -q option to emsource and redirect the output, e.g. using tee. emsource -p calls empdebuild to unpack the Emdebian build chroot, copy the package source into the chroot, copy and apply the emdebian patch files, installs the Build-Cross-Depends packages from debian/xcontrol using apt-cross -q -i and then runs emdebuild inside the chroot. See also debian-xcontrol. The build chroot is then discarded at the end of the build. If successful, the .changes file will be copied into the $WORK/pbuilder/result/ tree (organised by package name initial letter and full name in the same layout as the subversion tree used by emsource itself. Generating a complete build log with a disposable chroot.To supplement the emdebuild build log and generate a typical buildd log, use:
ANSI_COLORS_DISABLED=1 emsource -q -p $PACKAGE | tee $PATH/$FILENAME
ANSI_COLORS_DISABLED prevents the colour escape codes from being output to the logfile and the -q option to emsource prevents apt-cross from outputting the progress bar that would also include escape codes. Note that you may need to modify the sudo configuration (or run as root) to allow the environment to be modified. Alternatively, you may want a static chroot or buildd environment and then use emsource -b. Note that your buildd scripts will need to subsequently uninstall the build dependencies (native and cross) once the build process finishes. A disposable chroot is recommended when preparing packages so that the list of Build-Cross-Depends packages in debian/xcontrol is complete. However, a static chroot is faster than a disposable one as the chroot does not have to be unpacked and the apt cache remains in place. emdebuild support for debian/xcontrol.emdebuild supports the use of Build-Depends and Build-Depends-Tools when using the debian-xcontrol package. (In xcontrol, Build-Depends-Tools are the build dependencies that are needed only on the build machine - including all Architecture: all packages - and Build-Depends are the build dependencies that will need to be installed using apt-cross to make the cross architecture objects and headers available to the cross compiler.) emdebuild supports all valid dpkg constructs in debian/xcontrol and the easiest way to create a new debian/xcontrol file is to copy debian/control and then edit it. Migrate all build system tools (e.g. debhelper), patch systems (e.g. quilt), configuration tools (e.g. pkg-config) and documentation generation packages (e.g. doxygen) into Build-Depends-Tools. All packages left in Build-Depends in debian/xcontrol must be suitable for apt-cross, typically libraries and dev packages. (Unless the contents of the package are linked against the objects built by the cross compiler, leave the package out of Build-Depends in debian/xcontrol. Auto detection by ./configure.Some packages enable components if a particular dependency is located at build time. This auto detection is useful but if the package lacks a working option to ./configure that can --disable the component, this is a bug in the package. Report the problem under the long term mass bug filing for cross build support and add a note that such systems risk causing native bugs by generating different builds on a developer system compared to a buildd. As a workaround, remove these dependencies from debian/xcontrol to ensure the relevant feature is disabled. When the bug is fixed, these dependencies will need to be added back. Although debian/xcontrol simplifies various elements of changing the dependency chain of packages for embedded systems, the debian-xcontrol package expects the Debian maintainer to edit debian/xcontrol and use xcontrol to update debian/control prior to the native build. Future versions of emdebuild may incorporate checks on whether debian/control is in sync with debian/xcontrol. Using debian/xcontrol with xcontrol.emdebuild initially improvised a Build-Cross-Depends: field in debian/xcontrol and this will continue to be supported until emdebian-tools is fully integrated with xcontrol from the debian-xcontrol package. For compatibility with xcontrol, use:
Build-Depends: libsqlite0-dev, libglib2.0-dev, libgpewidget-dev
Build-Depends-Tools: cdbs, debhelper (>= 5), autotools-dev, automake1.9, dpkg-dev (>= 1.13.19)
instead of:
Build-Depends: cdbs, debhelper (>= 5), autotools-dev, libsqlite0-dev, libglib2.0-dev,
dpkg-dev (>= 1.13.19), libgpewidget-dev, automake1.9
Only packages that need to be installed with apt-cross are now listed in Build-Depends and packages that only need to be installed on the build machine (including all packages that are Architecture: all) go into Build-Depends-Tools. xcontrol build can then be used to integrate the two sections for normal Debian builds. Using $architecture.cache files.Some packages build test programs that are compiled and executed during ./configure and these programs will inevitably fail when cross building because the test is compiled to run on the target architecture, not the build architecture. Unfortunately, the autotools are not currently able to handle this situation automatically. Specific tests can be omitted by specifying cached values from a native build on the target architecture. These values can be obtained by inspecting the Debian buildd logs for the architecture involved. An example cache file (from libglib2.0-0 on arm) shows the format: glib_cv_stack_grows=no glib_cv_monotonic_clock=yes glib_cv_uscore=no ac_cv_func_posix_getpwuid_r=yesVariable names can be obtained from careful inspection of the ./configure script at the point at which the cross-build fails. Alternatively, build a normal Debian version of the package using ./configure --cache-file=$file and lookup the variable names from that file. Only use the variables that are necessary to allow the cross-build to complete in the cache file so that other values can be calculated normally. Scripts to help this process would be welcome. To use the cache file, place it in the top level source directory of the package (same directory as ./configure). If the package uses the internal tarball method (where dpkg-source unpacks a debian directory and leaves one or more tarballs containing the upstream source), create a patch for debian/patches that creates the file in the top level directory of the build-tree. The cache file should still be present in the top level source directory so that emdebuild can maintain the relevant emdebian patch file. Then edit debian/rules to add this line to the options passed to ./configure: --cache-file=$(DEB_HOST_GNU_TYPE).cache(ensure you specify HOST not BUILD). Modifying other files in the debian directory.Sometimes, other files need to be modified in the debian/ directory, besides rules, changelog, control and control.in (if present). Some of these files are modified automatically during the build (e.g. the files list, shlibs files and temporary directories). To get a list of files that have been modified to create a successful build use: $ interdiff -z PACKAGE_VERSION.diff.gz PACKAGE_VERSIONem1.diff.gz | \ grep "+++" | grep debian | grep -v locale | grep -v rules | \ grep -v control | grep -v changelog If you have had to edit a file in the debian/ directory, subsequent builds will need the same changes so a secondary emdebian patch file is needed. emsource version 0.2.2 or later will back up all files in the debian/ and debian/patches/ directories to ../$package.old/. emdebuild will check all files in the debian/ directory compared to the versions in ../$PACKAGE.old/ and create the necessary emdebian patch files. Modifying files outside the debian directory.Wherever possible, Emdebian changes should be limited to files in the debian directory, in particular debian/rules. Currently, it is preferable to let the build process run as normal, even if this means generating documentation or other content that is destined to be removed before the package itself is built. If the package supports a --no-docs or --disable-docs option to ./configure then it should be used. If not, simply remove the unwanted content by adding a rule to debian/rules rather than trying to patch or modify Makefiles or similar. If you still need to modify files outside the debian/ directory, use patches in debian/patches/. For CDBS packages, add the simplepatchsys rules: include /usr/share/cdbs/1/rules/simple-patchsys.mk For debhelper and dpkg packages, use dpatch - implementing dpatch support in debian/rules if necessary. See : HowTo use dpatch Patch files created for the debian/patches/ directory should follow the naming conventions for that directory and must not use the emdebian- prefix. However, it is advisable to use some form of emdebian "tag" for the file so that future upstream versions of the package do not inadvertently use the same filename. You must also ensure that all upstream patches that may affect the relevant files are applied before the emdebian patch. As many patch systems use a numerical prefix to determine the sequence of applying patches, one suggestion is to use filenames like 99_emdebian_avoid_running_foo. Remember that these patches will be applied by debian/rules, not by emdebuild. emdebuild checks for new files in debian/patches and then creates a copy of the patch file for emdebian SVN (called debian-patch-$file, ready to be copied into debian/patches by emsource. Listing of emdebian patch files with associated debian patch.$ ls *.patch debian-patch-99_emdebian_avoid_making_docs.patch emdebian-changelog.patch emdebian-control.patch emdebian-rules.patch emdebian-postinst.patch In this example, debian-patch-99_emdebian_avoid_making_docs.patch is copied by emdebuild into debian/patches as 99_emdebian_avoid_making_docs. emdebian-rules.patch then needs to include any extra code necessary to allow the build to apply and deapply this patch. If the package already uses patches via CDBS, nothing needs to be done. If the existing patches use dpatch, a second debian-patch will be needed to modify debian/patches/00list. If the package does not use any patch system, debian/rules will need patch support added via emdebian-rules.patch. Changes for CDBS.Versions of dpkg-cross prior to 2.0.0 used overrides in the MAKEFLAGS environment variable that conflicted with the needs of packages like libx11 that need to be able to build certain internal tools using the native compiler. CDBS tends to rely on this override mechanism and many CDBS packages now fail to use the cross-compiler because CDBS retains an internal override that is no longer cleared by dpkg-buildpackage. To reset CDBS to not set the override use the following in debian/rules:
When Debian bug #450483 is fixed, this change will no longer be required. CROSS-BUILDING DEBIANGradually, changes are being made to the Debian package building tools to make cross building easier in Debian. Some problems remain and emdebuild uses three main workarounds: gccross : from dpkg-cross. This script sits between and the cross compiler. When the cross compiler is called, all -I and -L paths must be set for the cross packages installed by dpkg-cross so that the compiler can find the right headers and the right shared objects. In most cases, a combination of pkg-config and libtool ensure that these paths are available. Packages that build internal libraries and then link other libraries or executables against those object files will experience problems when installing the object file during the Debian package building. During the final install, object files are relinked for the package directories - except when cross building, the relinking gets confused and tries to relink against libraries in /usr/lib/ instead of /usr/$arch-triplet/lib/ and the build fails. For some years, dpkg-cross has contained a utility called gccross that rewrites the paths passed to the cross compiler. It does this by pretending to be the cross compiler and then rewriting the @ARGV before passing control to the real compiler. Since emdebian-tools (>=0.7.3), gccross is optional - packages need to opt-in to gccross support using a flag in debian/xcontrol : X-Build-Cross-Libtool: yes (or Build-Cross-Libtool: yes but hopefully this will be fixed before this field needs to become an official dpkg control field). Packages can implement their own support for cross building their internal objects by specifying -l flags to libtool: Specifying relinking path for internal objects in CDBS.This example is from libqof1.
ifneq ($(DEB_BUILD_GNU_TYPE),$(DEB_HOST_GNU_TYPE))
DEB_MAKE_INVOKE+="QOF_LIBS=-L/usr/$(DEB_HOST_GNU_TYPE)/lib -lqof"
endif
Specifying relinking path for internal objects via $(MAKE).For an example package, FOO using -lfoo during native relinking and passing ${FOO_LIBS} for libfoo_la_LIBADD in the upstream Makefile.am.
ifneq ($(DEB_BUILD_GNU_TYPE),$(DEB_HOST_GNU_TYPE))
RELINK="FOO_LIBS=-L/usr/$(DEB_HOST_GNU_TYPE)/lib -lfoo"
endif
Upstream may need to be patched.Note that upstream needs to support a variable already - if the upstream Makefile.am hardcodes -lfoo then a patch will be needed to replace this value with a suitable variable. These problems will also be reported under the long term mass bug filing for cross building support agreed on debian-devel in November 2007. Existing workarounds for dpkg crossbuilding bugs.LD_LIBRARY_PATH : this is a workaround for dpkg-shlibdeps and when bug #453267 is fixed, this workaround will disappear from emdebuild. The basis of this workaround is to make /usr/${crossprefix}/lib/ available to dpkg-shlibdeps where ${crossprefix} is the architecture triplet; for ARM the triplet is arm-linux-gnu. PKG_CONFIG_LIBDIR : this is a workaround for dpkg-buildpackage -a and when bug #439979 is fixed, this workaround will disappear from emdebuild. The basis of this workaround is to make /usr/${crossprefix}/include/ and /usr/${crossprefix}/lib available to gcc where ${crossprefix} is the architecture triplet; for ARM the triplet is arm-linux-gnu. Omit udeb packages intended for the Debian Installer.Emdebian does not use udeb packages because the Emdebian installation process is handled separately and the dependencies of udeb packages can cause difficulties when cross building. emdebuild tries to prevent such packages being built but some editing of debian/rules may still be necessary to remove udeb build instructions. Typically it is necessary to remove udeb targets from the configure: and binary-arch: targets as well as removing / adapting custom rules that handle udebs within more general targets. Maintainer scripts.Maintainer scripts include preinst, postinst, prerm and postrm package scripts that handle the installation, upgrade, removal and purge process for each package. Source packages can contain separate maintainer scripts for each binary package built. For information on maintainer scripts, see Debian Policy. Many maintainer scripts will need patches for Emdebian, reasons include (but are not limited to):
At present, emdebuild does only basic checks on the maintainer scripts. Checking the final package.emdebuild will make basic checks on the package(s) including:
Further checks may be added but manual checks on the packages are still essential - including whether the package picks up the correct Depends: information from dpkg-shlibdeps and whether the maintainer scripts are usable. In a Gnome environment, you can pass the .changes file directly to deb-gview and view the contents and descriptions of all packages listed in the .changes. You can also use debc using the -a $arch option to view the contents in the shell, including details of which interpreter is called by each maintainer script. Wherever possible, check your package on real hardware (or in an emulator), to ensure that the package installs, upgrades and removes cleanly. (If anyone wants to port piuparts - or something like it - to work on a cross-built package, feel free.) Also, if you are using emsource, run 'emsource -b $package' after committing patch changes to SVN but before uploading the package itself. A successful build will output the location of the .changes file. Use the --verbose option to emdebuild and a successful build will output a suitable command line for both of these additional checks. In time, these check routines will need to be ported to lintian - with support for not running certain lintian checks that valid emdebian packages would fail - or an emdebian version of lintian. Easier patch handling.emdebian-tools recommends meld to simplify the process of handling the emdebian patch files. An example usage may help (run from the emsource directory):
$ meld ../apt.old/debian/rules debian/rules
meld allows you to migrate a line from the old file back into the patched file as well as showing you exactly which parts of long lines have changed. Note that although meld can produce a new patch from the loaded files, meld should be started from the directory above to ensure that patch has no problems applying the results:
$ pushd ../
$ meld apt.old/debian/rules apt-0.7.6/debian/rules
$ popd
Generally, this is not necessary because emdebuild will update ../emdebian-rules.patch for you at the next build. Another use of meld is in preparing patches for bug reports against the Debian package. Using meld to generate patches for Debian bugs.The difficulty in preparing suitable patches to support cross-building in Debian is that the Emdebian patches necessarily remove package components that Debian requires by Policy. Some Emdebian changes are handled within the package, some within other tools and the patch files need to be split accordingly. meld can assist by migrating sections of a patch between different versions of the relevant files, typically debian/rules.
$ cp $package-$version.orig.tar.gz ../branches/
$ cp $package_$version.dsc ../branches/
$ mkdir -p ../branches/$package.new/debian
$ cp $package-$version/debian/rules ../branches/$package.new/debian/rules
$ cd ../branches
$ dpkg-source -x $package_$version.dsc
$ meld $package-$version/debian/rules $package.new/debian/rules &
The $package.new directory provides a usable path so that the final patch is easier to apply. Identify the components of the patch that are relevant to the package, rather than the tools (e.g. changes to support --build and --host or package changes to support nocheck DEB_BUILD_OPTIONS etc.) and use the arrows in the centre of the meld window to move other changes into rules.emdebian so that the temporary rules.emdebian file represents a midstage between the original debian/rules and the Emdebian patched debian/rules in ../trunk/$package-$version/. Note that some Debian maintainers prefer to have separate patches for separate issues, even if the patches apply to the same file. Others may agree to have one patch for one file but would prefer to have two bug reports for the two issues. patch will usually work out how to offset one patch to take into account lines added or removed by another. Once you only have the changes you need for this bug report, right click the rules.emdebian window and select "Create patch" from the context menu. Apply this patch and test the build using normal Debian build tools like debuild or dpkg-buildpackage -rfakeroot. In the majority of cases, Emdebian changes can be restricted to the files in the debian/ directory and there is no need to use dpatch or other systems to apply these patches. Just use the patch command directly. If the patch works and does not change the normal Debian build behaviour or package contents, file a wishlist bug against the package quoting the long term mass bug filing for cross building support and specify the usertag either in the body of the email: (See Additional Pseudoheaders for the Debian BTS). In reportbug, you can attach the patch direct from the command line with reportbug -A FILE if the diff is large or reportbug -i FILE to include the diff within the text of the report.
Tag: patch
User: codehelp@debian.org
Usertags: crossbuilt
or once the bug number has been received:
$ bts user codehelp@debian.org , usertag NUMBER + crossbuilt
See also: Current list of cross build support bugs and the Emdebian wiki support: Emdebian QuickStart and The Emdebian Developer Guide Making Patches using embug.Preparing a patch suitable for the Debian package can also involve a little extra work, separating the Emdebian changes from the cross-building changes and ensuring that the patch has no negative effects on the Debian package. embug--prepare does NOT support --package. 'embug --prepare' must be run from the source directory of the emdebianised package and the package must have been successfully built using emdebuild. embug makes no attempt to remove or clean the temporary directory, ever. Running 'embug --prepare' more than once will result in multiple temporary directories. It is advisable to retain the temporary directory until the bug is fixed as the package maintainer may have questions about the patch and request changes that you will need to test. Preparing a bug for aptitude.
$ emsource -c aptitude
$ cd /opt/emdebian/trunk/a/aptitude/trunk/aptitude-0.4.10/
...
aptitude-0.4.10$ emdebuild
...
aptitude-0.4.10$ embug --prepare
dpkg-source: extracting aptitude in aptitude-0.4.10
dpkg-source: unpacking aptitude_0.4.10.orig.tar.gz
dpkg-source: applying ./aptitude_0.4.10-1.diff.gz
dpkg-source: extracting aptitude in aptitude-0.4.10
dpkg-source: unpacking aptitude_0.4.10.orig.tar.gz
dpkg-source: applying ./aptitude_0.4.10-1em1.diff.gz
embug: prepared source is in ../../branches/bugp21413
total 5124
-rw-r--r-- 1 neil neil 33218 2008-02-09 23:05 aptitude_0.4.10-1.diff.gz
-rw-r--r-- 1 neil neil 968 2008-02-09 23:05 aptitude_0.4.10-1.dsc
-rw-r--r-- 1 neil neil 46699 2008-02-09 23:05 aptitude_0.4.10-1em1.diff.gz
-rw-r--r-- 1 neil neil 652 2008-02-09 23:05 aptitude_0.4.10-1em1.dsc
drwxr-xr-x 8 neil neil 4096 2008-02-09 23:05 aptitude-0.4.10.debian
drwxr-xr-x 8 neil neil 4096 2008-02-09 23:05 aptitude-0.4.10.emdebian
-rw-r--r-- 1 neil neil 5129549 2008-02-09 23:05 aptitude_0.4.10.orig.tar.gz
embug: aptitude-0.4.10 is ready for patch creation.
e.g. 'meld aptitude-0.4.10.debian/debian/rules aptitude-0.4.10.emdebian/debian/rules'
aptitude-0.4.10$ cd ../../branches/bugp21413
bugp21413$ meld aptitude-0.4.10.debian/debian/rules aptitude-0.4.10.emdebian/debian/rules
It is worth checking that the .dsc is the same as in current Debian - check the md5sum for the files against the .dsc available from the Debian Package Tracking System. In meld, you need to isolate the cross-building changes from the rest of the Emdebian changes by returning removed lines into the Emdebian file and removing added lines until you are left with only the changes that are needed to allow the package to cross-build. Right click the Emdebian rules file, select "Create Patch" and save the patch file in the temporary directory. Avoid making changes to aptitude-0.4.10.debian/debian/rules within meld! Now do test builds from the aptitude-0.4.10.debian/ directory with and without the patch. (This allows you to use tools like debdiff, interdiff and lsdiff -z to check for differences before and after applying the patch.)
aptitude-0.4.10.debian$ debuild
aptitude-0.4.10.debian$ patch -p1 < ../crossbuild.diff
aptitude-0.4.10.debian$ dch -n "Cross build support"
aptitude-0.4.10.debian$ debuild
The use of dch -n ensures that you have two .changes and two .dsc files as well as two sets of packages for the comparisons as well as preventing lintian from adding extra complaints. The package MUST build as a Debian package without any changes to the contents or lintian errors or warnings. It must also cross-build (although emdebuild should be expected to complain about the contents of the cross built package which would normally be removed. As more bugs are filed and more are fixed, the task of separating changes out of the emdebian rules patch becomes easier. In most cases, only debian/rules should be patched for the mass bug filing for cross build support. Only bugs tagged with the crossbuilt usertag (user: codehelp@debian.org) are queried by embug. When filing bugs, note that reportbug will silently remove usertags, so save the contents of the reportbug report and include it in a new email using your favourite email client. The usertag is set by adding these pseudo-headers at the top of your email:
Tag: patch
User: codehelp@debian.org
Usertags: crossbuilt
If you use mutt, use reportbug -M -i ../crossbuild.diff to use mutt with the diff already included. Add the usertag headers and content then sign and send as normal. (Content should include a reference to the mass bug filing for crossbuilding support in Debian by linking to http://lists.debian.org/debian-devel/2007/11/msg00116.html to indicate that this bug is part of a process already agreed on debian-devel.) A quick way of checking for existing bugs is to use reportbug-ng which supports searching the title of all open bugs or search the bugs.debian.org page for the package using your WWW browser. Once checked, skip this part of the reportbug process using the -b option to reportbug. All bugs filed for crossbuilding support should be wishlist. One method for other email clients is to use the -p option to reportbug, redirect the output to a file and insert that file into your new message. embug only records flags on your local system. Packages should be flagged if ../emdebian-rules.patch contains patch lines to allow the package to crossbuild instead of just adding Emdebian features. Examples of crossbuild changes:
Remember not to include changes that do various Emdebian things like removing manpages, removing documentation or udeb packages etc. Patches for crossbuilding support must not add lintian warnings or other errors to the package. Build the existing package and compare the output carefully against the patched build before submitting the bug report. Commands will generally involve:
$ emsource -a arm --where avahi
Checking the apt-cross cache is up to date for arm.
avahi is located in: '/opt/emdebian/trunk/a/avahi/trunk/avahi-0.6.21/'
$ less /opt/emdebian/trunk/a/avahi/trunk/avahi-0.6.21/../emdebian-rules.patch
In due course, once other bugs are fixed, the removal of manpages, info pages, documentation etc. is expected to be supported within the build tools themselves. Crossbuilding bugs will then start to be filed for support of these updated build tools. Ensure the Emdebian build is up to date before starting work on patches. Viewing emdebian-rules.patch via emsource.A simple script that accepts a package name and then displays the Emdebian patch for debian/rules using less.
set -e
if [ ! $1 ]; then
echo "Please specify a package to check";
exit;
fi
DIR=`emsource --where $1 | grep located | cut -d"'" -f2`
less $DIR/../emdebian-rules.patch
Then, if the patch warrants a crossbuilding bug report, use embug -f -p package_name. (If a bug has already been filed, embug -f will retain the existing data.) |
Last Modified: Sun, Apr 6 02:00:19 UTC 2008
Copyright © 2000-2008
The Embedded Debian Project;
Emdebian is an offical subproject of Debian.
