|
|
April 30th, 2006
The Embedded Debian team held a work session in Extremadura, Spain, between 12th and 16th April 2006, as part of an ongoing series of Sponsored Debian work sessions. A lot of very useful work was done and real progress was more on a number of fronts - scratchbox2, the Emdebian distribution, integrating cross-toolchains into Debian, the arm-eabi architecture and a host of other items.
These sessions are made possible by the Linex team who arrange Government sponsorship to cover attendees' costs and taking care of all our logistical, accomodation, food and hacking needs.
The workshop was over 5 days, with most of the first and all of the last taken up with travelling, and the chance to experience some culture one day, leaving 2 days and 4 evenings for work.
Nearly everyone flew in to Madrid Airport on Wednesday and came by Bus to the University Building at Jarandilla de la Vera. Wookey and the Spanish contingent came overland.
We arrived to find that our hack lab was already set up with power and networking, so after settling in we had an evening to get to work.
We had a relatively formal meeting this first evening and the last evening. The first to get introduced and decide what people would work on, and the second to review the event and work out where we got to. There were also various smaller group discussions in the hack-room during the event, including some quite animated debate about the more contentious issues.
Most of the time was spent on solid hacking, with some discussion and debate, and a timeout to see some of the locality.
On the last day, after some more hacking, the wrap-up meeting and the final dinner there was a certain amount of drinking and talking bollocks from some members of the team, with Yevgeny winning the 'most pissed' award to general amusement.
Here is a summary of what got done and disucussed.
It was explained to the group that as well as providing a cross-build environment scratchbox is probably the best way of solving the problem of running install-time scripts on the target filesystem, rather than the build-machine filessytem. Maintainer scripts currently assume they are running on the native host and there is no way to tell them they are actually running on a filesytem in /var/arm-linux/target/ (for example). Scratchbox's path magic solves this problem, so it is useful even if you are not using it for cross-building. This feature was of serious interest to the SLIND team.
The team worked on the next-generation scratchbox, scratchbox2. The existing scratchbox works very well but has a large maintainence overhead because all the packages needed to build things have to be compiled with special scratchbox paths and kept up to date. All this stuff is on the system twice, and the idea of scratchbox 2 is that the normal debian packages can be used. For this to work scratchbox2 needs to be able to arbitrarily munge paths in order to point at native or target architecture files.
The scratchbox team worked on a couple of possible ways of doing this and proved both in principle. The first is a redirector based on LD_PRELOAD functionality. The second uses the fuse userland filesystem. The disadvantage of the LD_PRELOAD scheme is that it doesn't work for static binaries. Fuse is kernel 2.6 only.
Both are configured using lua scripts (lua is small, quick and simple).
Given this excellent progress a pre-alpha version looks reasonable in the summer and people can see how well it works in practice. The main current task is writing the configuration rules to make the correct redicrections happen.
They are using the Nokia 770 as a development platform.
Ed and Yevgeny have produced an embedded Debian distribution based on the STAG ideas and having a /emdebian directory containing modified /debian files, but using scratchbox to do the cross-building. There are a small number of tools which need to be modified to recognise a changed rules directory (dpkg, debhelper, devscripts, etc). The mechanism is to use $DEBIAN_DIR/ throughout instead of /debian/. This is surprisingly powerful. if $DEBIAN_DIR is unset or set to /debian then everything builds just as normal. If you set it to /emdebian you get the default emdebian build. You can set it to some other title of your own invention to have your own version of a package override the emdebian or debian versions, so it is easy to have local modifications.
In discussions with the SLIND team it became clear that you could build SLIND using these tools simply by having the SLIND version of the debian dir renamed to slind and $DEBIAN_DIR set to point at it. Thus the two projects agreed to produce a common set of modified packages under the emdebian banner.
During the session they updated their tools for etch, and produced a scratchbox devkit containing the same version of tools as scratchbox 0.9.8
SLIND is an embedded Debian distribution built using dpkg-cross. It has support for uclibc as well as glibc. It has support for full 3-stage bootstrapping of cross-toolchains from debian sources. It contains mdodifications to the debian dependecies and build rules very much like those of STAGE. Both projects produce a set of packages with the same names as standard debian packages but smaller and with the cruft removed.
Updated the Slind toolchains to build gcc 4.0 & 4.1 with current debian verions, added mips & mipsel cross-toolchains and got x86->x86 cross-toolchain working.
Worked on build ordering
Agreed common approach with STAGE team, as described above.
One thing that numerous people expressed a desire for was to have a set of cross-toolchains in debian main itself so that people could simply apt-get them. Emdebian.org has been maintaining a set of cross-toolchains, but keeping in-sync with debian main has proved taxing. Whilst having all possible cross-compilers built is clearly silly, having some of the most used ones (fast machines -> common targets) in the archive makes a lot of sense.
Simon discussed this issue with an ftp-master (Ganeff) and it was agreed that we could upload compilers from x86, amd64 and powerpc to arm, powerpc, mips, and mipsel.
We also agreed to kill off toolchain-source as it is no longer necessary now that the normal tools can be cross-built. We do need a simple mechanism to build 'unsupplied combinations' though, so that people can easily do this for themselves when required.
He worked largely on qemu, and spent some hours talking to the local media (from EFE), which resulted in this article: http://www.regiondigital.com/modulos/mod_periodico/pub/mostrar_noticia.php?id=37118
Worked on updating Emdebian/Debian cross-tools. First took latest binutils patch (updated by Josh Triplett from Nikita Youschenko's original) and checked that it built and worked. Composed a mail to the binutils maintainers to get the 2.5 yr old bug on cross-building this app closed and the patches incorporated so we don't have to keep it updated. This did the trick and a new version including the patch was uploaded a few days later.
With a bit of help from virtuoso patched gcc 3.4 in testing and built full cross-toolchian. This tested OK on building a Balloon 2.4 kernel and some utils.
Spent some time getting emdebian cross-build script to work with an independent version of dpkg as well as apt-get. Decided that we were slowly re-writing wanna-build and associated tools so started work on using those instead of our own home-grown effort in the home that it would be easier in the long term. Hopefully all this will be superceded by most of the tools people want being in Debian main anyway soon.
Also looked at merging existing crosscompiler script with Virtuoso's slind stuff.
Colin particularly ended up doing quite a lot of hacking trying to get YAFFS2 working on Balloon2. Partly due to exhortations from home and also because it is a necessary prerequiste to building STAGE for Balloon, which we wanted to try, but didn't get to.
Wook created a couple of new wiki pages and updated existing ones to reflect current developments. Started one to cover the major remaining area of disgreement amongst those present - about exactly how to store our metadata, and another document on how package maintainers can make their packages more embeded- and cross-compile- friendly. This still needs a lot of work.
Uploaded a new version of qemu
Made sure we discussed and decided on how to handle the change of arm to eabi (a new abi). The short version is that the new architecture is called armel. A new version of dpkg containing a patch for this should now be uploaded.
The longer version is that we can see no viable mechanism for migrating to eabi within the existing arm architecture. Any attempt to do so would result in everything appearing in the archive twice, and/or things remaining uninstallable for months, and an upgrade being done in one fell swoop. This is not really any different to having a new arch and people switching to it. It is just like going from i386->amd64 on a 64 bit machine. So we decided a new architecture is necessary and proportionate. The actual name is surprisingly contentious and hedged with considerations of externalities. Names iwth hyphens in cause trouble as that currently separates os and cpu (and maybe libc too in the future). Eventually a consensus was reached that armel was not too long, clearly arm-related (starts with arm), not taken and can potentially form a neat pair with armeb.
The only potential problem is that the unofficial armeb is old-abi (corresponding to the existing 'arm'). However it was felt by the representative of that port present (Lennert) that it was reasonable for them to move to eabi before becoming an official port. We decided to proceed on this basis but allow a short period after announcing it in which this team could veto this descision (we would ignore people _not_ actually doing this work who whinged - you need to be doing the work to get a say of any significance).
Another consideration is that a pure eabi system will not work on arm v4 (Strongarm), due to the need for the BX instruction to be thumb interworking safe. This appeared with arm v4t. We could work around this, at the cost of some inefficiency (an extra instruction on every function call, modified toolchains and binary incompatiblity with other distributions), but it may make more sense just to leave strongarm using the existing 'arm' architecture.
You can read more about the whole ARM-eabi thing on the wiki
Guillem also worked on adding dpkg-exclude functionality to dpkg. This will allow things to be 'left out' at install time by dpkg. He also looked at adding a uclibc-arm architecture.
Finally he updated DirectFB and generated a summary of issues doc from our initial meeting which Wookey used as a basis for the wiki pages, and spent a long time talking to the journalist with Simon.
Worked on Debian-installer support for the nslu-2.
Looked at making initramfs use klibc and kinit in order to have enough system in the initramfs to run tests for cases where it is necessary.
Worked on the MTD support for d-i, which would allow it to install to systems containing only flash storage.
Hector was a day late due to being stricken with a stomach bug the week before. He didn't enjpoy the drive over much as a result.
He worked on the web pages including a new look and feel, provided virtuoso with a mips box (WRT54G) to get slind working on mips, and tested the testing toolchains (finding a bug when building 2.6 kernels which does not happen with the slind versions of the toolchains).
He was also heavily involved with the organisation beforehand.
Spent most of his time working on a free driver for NSLU2 ethernet driver, to replace the existing Intel non-free driver. Not yet working.
He tried building QTE2 for opie, but found that the opie build system is broken, and he needs to talk to upstream in order to make further progress.
Next he tested GTK on framebuffer (using DirectFB). This works, but needs support in GTK to select this option rather than the default of X. THis needs dicussion with upstream in order to decide how best to do this.
Next he turned to getting an sbuild setup using scratchbox working. Jesus Climent at Nokia has had this going - they will confer.
On the Saturday we largely had a day off as our hosts had set up a day-trip to visit some local places of interest and experience a bit of Extremaduran culture (food, religion, arhitecture, countryside). For many of the non-smokers this was the first time we had stepped out of the building :-) Amaya Rodrigo joined us for the day, as did the 4 spouses of team members who had come along to enjoy a bit of Spanish sunshine.
We visited the Monastery de Yuste, which is the Palace of King Charles 1st of Spain and the 5th of Germany, which was restored in 1950s for the 400th anniversary of his death, and is now a monastery. That was heaving as it was easter weekend so the car-park chaos and traffic was quite entertaining. We took a group photo, and Wookey climbed round a bridge.
Next we went via a village famed for it's topiary, which did indeed have a lot of topiary to a restaurant by the Garganta de Cuartos in Losar de la Vera. Here a fine long lunch was provided. It took about 2.5 hours and we were introduced to the concept of putting lemonade or water in your wine, which I think is a brilliant idea. After a few rounds of tapas some of us wanted a progress bar so we had some idea of many more courses there would be. There was the famous Spanish dish Gazpacho (cold soup, which I have to admit I didn't like much) and lots of good food including local homebrew alcohol (something like schnapps but really sweet) to finish off.
Now thouroughly stuffed we got to walk it off round another village, Valverde de la Vera, which still has a great deal of the mediaeval character of villages in the area with no cars, cobbled streets containing small central streams, and strange overhanging and random buildings, often in quite a poor state of repair. They were all set up for the '12 nights of easter', where various rituals and services go on. This day it was processions through flower-festooned arches (that evening - we just saw the arches set up). Zumbi asked the locals what it was all about but was soon telling them all about Debian and trying to get converts to _our_ church :-)
You can see some pictures of the event.
So, a great deal of stuff was done. But we are of course left with many more things to do.
A few things noted at the wrapup meeting and not adressed above are:
Cross-compiling/toolchains:We need a unified framework, particularly for an easy way to build arbitrary toolchains. Even with the common ones in main, there will be a need for people to make special-purpose toolchains or unusual combinations (mips->arm, powerpc->avr)
Toolchain source has previously filled that slot, but now we are intending to get rid of it we should work out what will replace it. Perhaps this can be dealt with at Debconf6
We still need to actually produce some prebuilt emdebian rootfs tarballs for people to download and use. We are closer to that goal, but have not yet actually attained it.
We need to detail DEBUILD_OPTIONS such as 'nodocs' and 'notest' and promote them to package maintainers. We still also have to deal with the DEBIAN_DIR/udeb dicotomy somehow. Possibly by continuing to develop both until a clear winner emerges.
Thanx to nchip (Riku Voipio) for deciding this would be a good idea and getting it off the ground, then arranging things with the Linex people, with the help of Zumbi (Hector Oron). Also thanks to Dario Rapisardi, Cesar Gomez Martin and a the rest of the Linex people for flawless organisation on-site.
The attendees were:Here is a pic of us all so you can see who is who.
Last Modified: Sat, Mar 24 01:39:18 UTC 2007
Copyright © 2006-2007
The Embedded Debian Project;
Emdebian is an offical subproject of Debian.
