TDeb translation packages
Cross building Debian
tdebs in Debian and Emdebian
Emdebian borrows a technique from OpenEmbedded to handle
translations which is already outlined within Debian as "Tdeb" - translation
debian packages.
Emdebian tdebs are not identical to the Debian tdeb plans
The Emdebian implementation of tdebs differs from the proposed
tdebs for Debian because Emdebian does not care about manpages in
general, let alone translated manpages. Once the 'nodocs' DEB_BUILD_OPTION is
supported in debhelper, this will not be an issue as the tdebs can be built for
Emdebian without any manpages. Other translated documentation would be omitted
under 'nodocs' too. Images containing translated text are relatively few.
Therefore, the majority of the Emdebian tdeb implementation can be applicable
to Debian.
Current Debian practice for translations
Debian packages collate all available translations into a single
package which can easily lead to over 250Mb of unused translation files on a
typical installation. i.e. the translations alone could take up four times the
amount of space intended for the entire embedded operating system.
Repackaging translations into individual locales per
package.
One locale, one package, one tdeb
em_installtdeb is a new script in
emdebian-tools based on debhelper, to
create translation packages (tdebs). em_installtdeb is
intended to separate out the individual translation files from the current
Debian packages into packages without any translation files and a series of
tdeb locale packages, one per translation. Generated packages use the syntax:
$package-locale-$languagecode_$version_all.deb.
Once a package uses em_installtdeb, translation
files should be removed from all packages in the normal build.
em_installtdeb runs as a second build (very small, very
fast) that simply converts and packages the *.po files into tdebs. A tdeb
source package is created (.dsc, tdeb.tar.gz and .changes) alongside the
existing build data. The source package can be used by translators to build
updated or new tdeb packages. Tdeb .changes files need to be uploaded only to
secondary locale repositories instead of the main Debian mirrors and these
repositories can have much more relaxed upload policies. Tdeb packages have no
dependencies and no packages may depend upon them.
Each package now built for Emdebian includes tdeb support so that
if the package contains gettext-compliant translations, tdeb packages are
created. Emdebian also supports a locale repository with search support.
Currently, certain compromises are made to allow tdebs to work without
changes to packages like dpkg or reprepro
- including the need to make two uploads, one for the main package (without
translations) and one for the generated tdeb .changes file including
generated tdeb source and all of the tdeb packages. For more information on
em_installtdeb, see the em_instal
ltdeb manpage.
Handling a ten fold increase in binary package numbers
Emdebian generates a single package for every translation of each
Emdebian package, leading to a 70% reduction in installation size but a tenfold
increase in the number of binary packages built from each source package. To
solve this scalability problem, Emdebian provides a new C/C++ application
(langupdate) handling the installation and update of tdeb
packages according to the list of supported locales and the list of installed
packages. Data cached by langupdate is temporary and is not intended to be
stored between runs of langupdate (i.e. it can be stored completely in
RAM).
The secondary locale repository organises the tdeb packages by
locale root, e.g. 'en_GB' is beneath 'en' and 'sr@Latn' beneath 'sr'. Each
locale root becomes a single apt source in the locale repository to support
fallback from a specific locale to a the more general locale root. In this
manner, each device only has to cope with cache data for a fraction of the
total number of tdeb packages (roughly 1 part in 30). The tdeb cache itself is
temporary so the only effect on the device package data is the small number of
locale packages that are actually installed. tdebs have no long description and
no dependencies sothe effect on the dpkg lists should be
minimal.
langupdate queries the secondary apt cache to
get a list of all translation packages (tdebs) that match the list of locales
supported on the device. This list is then compared with the list of installed
binaries. The translation packages that match the installed package are then
installed by passing options to apt-get.
langupdate uses a temporary file for the sources.list and
updates it from the list of supported locales each time langupdate is run, i.e.
it combines the effects of 'apt-get update' with 'apt-get upgrade'. tdeb
packages have no dependencies so 'dist-upgrade' is not required.
langupdate is a native Emdebian package - it may
appear in Debian at a later date when Debian supports tdebs.
Translator upload support (outline only)
em_installtdeb includes outline support for
translator uploads. This is intended to allow translators to add the locale
deb-src source to their apt sources lists and use apt-get source
$package-locale-$lang to download the tdeb source: the POT file, existing PO
files and enough metadata to be able to build a new or updated tdeb package for
a specific language using em_installtdeb $lang. The build generates a new
(binary-only) .changes file suitable for upload direct to the locale
repository. There are issues to be resolved within this support before it can
be made widely available, relating to merging the new translations back into
the Debian package, but everything related to tdebs in Emdebian should be
considered "experimental".